It hurts when I shoot myself in the foot
A Cautionary Tale for Silly People
----------------------------------
For the past year now, I've had "ping" problems.
It pauses when it runs, and always returns warnings like so:
"Warning: time of day goes back, taking countermeasures"
I looked _everywhere_ on the net trying to find the problem.
I upgraded the kernel many times.
I upgraded glibc a few times.
I upgraded iputils a few times as well.
Nothing helped.
The clock was sync'd with an atomic clock every night.
Still, I kept getting the problem.
Then while moving log files around today, I noticed that the clock 'second' didn't move. Weird.
So I ran this:
while :
do
date "+%H:%M:%S"
done
I got interesting results:
17:05:24
17:05:24
17:05:24
17:05:33
17:05:33
17:05:25
17:05:25
17:05:33
17:05:25
17:05:25
Nice huh!
Why?
I looked inside the box and found a Pentium II 400, and a Pentium II 450.
Oddly enough they run together as a 266.
[root@hercules root]#cat /proc/cpuinfo
processor : 0vendor_id : GenuineIntel
cpu family : 6
model : 5
model name : Pentium II (Deschutes)
stepping : 2
cpu MHz : 265.915
cache size : 512 KBfdiv_bug : nohlt_bug : nof00f_bug : nocoma_bug : no
fpu : yesfpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips : 530.84
processor : 1vendor_id : GenuineIntel
cpu family : 6
model : 5
model name : Pentium II (Deschutes)
stepping : 1
cpu MHz : 265.915
cache size : 512 KBfdiv_bug : nohlt_bug : nof00f_bug : nocoma_bug : no
fpu : yesfpu_exception : yes
cpuid level : 2
wp : yes
flags : fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 mmx fxsr
bogomips : 663.55
Followup to: <[email protected]> By author: Chris <[email protected]> In newsgroup: linux.dev.kernel
Quoted 5 lines
I looked inside the box and found a Pentium II 400, and a Pentium II 450.
Oddly enough they run together as a 266.
The fact that they have different BogoMIPS figures indicate that that is not really the case. It looks more like the second processor is running at 333 MHz or something. You definitely have a bizarre box here, and you probably should be running with the "notsc" option. Heck, maybe you can reconfigure your mobo and actually run both processors at 400 MHz. You'd get quite a performance boost, too...
-hpa
Signature · 3 lines
<[email protected]> at work, <[email protected]> in private! "Unix gives you enough rope to shoot yourself in the foot." http://www.zytor.com/~hpa/puzzle.txt <[email protected]>
Quoted 6 lines · H. Peter Anvin
Followup to: <[email protected]> By author: Chris <[email protected]> In newsgroup: linux.dev.kernelQuoted 3 lines
I looked inside the box and found a Pentium II 400, and a Pentium II 450.
Quoted 8 lines
Quoted 1 line
Oddly enough they run together as a 266.
The fact that they have different BogoMIPS figures indicate that that is not really the case. It looks more like the second processor is running at 333 MHz or something. You definitely have a bizarre box here, and you probably should be running with the "notsc" option. Heck, maybe you can reconfigure your mobo and actually run both processors at 400 MHz. You'd get quite a performance boost, too...
There was a kernel patch posted about5 or so months ago which would "handle" this setup (CPUs with the same clock speed, but different multipliers). Alan Cox said it probably was a bad idea, so it wouldn't go into the kernel, but the patch may still be usable.
This is sometimes called "asymmetric multiprocessing", and the thread is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/
Quoted 16 lines · Andreas Dilger
Quoted 6 lines · H. Peter Anvin
Followup to: <[email protected]> By author: Chris <[email protected]> In newsgroup: linux.dev.kernelQuoted 3 lines
I looked inside the box and found a Pentium II 400, and a Pentium II 450.
Quoted 8 lines
Quoted 1 line
Oddly enough they run together as a 266.
The fact that they have different BogoMIPS figures indicate that that is not really the case. It looks more like the second processor is running at 333 MHz or something. You definitely have a bizarre box here, and you probably should be running with the "notsc" option. Heck, maybe you can reconfigure your mobo and actually run both processors at 400 MHz. You'd get quite a performance boost, too...
Quoted 7 lines
There was a kernel patch posted about5 or so months ago which would "handle" this setup (CPUs with the same clock speed, but different multipliers). Alan Cox said it probably was a bad idea, so it wouldn't go into the kernel, but the patch may still be usable.
This is sometimes called "asymmetric multiprocessing", and the thread is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
I thought asymmetric multiprocessing would support CPUs with different speeds. ie, 400 & 450mhz. How would you get different multipliers and same Mhz when the CPUs are on the same FSB(ignoring AMD SMP where each processor has an exclusive FSB, and this might be possible)?
IIRC, Linux supports SMP on CPUs with different stepping as long as the features are the same, and even then it'll use the features of the boot CPU. There was a patch to compare the different features on the CPUs available and use the subset available on all processors.
Mike
Quoted 6 lines · Mike Fedyk
Quoted 5 lines · Andreas Dilger
There was a kernel patch posted about 5 or so months ago which would "handle" this setup (CPUs with the same clock speed, but different multipliers). Alan Cox said it probably was a bad idea, so it wouldn't go into the kernel, but the patch may still be usable.
Quoted 7 lines
Quoted 2 lines
This is sometimes called "asymmetric multiprocessing", and the thread is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
I thought asymmetric multiprocessing would support CPUs with different speeds. ie, 400 & 450mhz. How would you get different multipliers and same Mhz when the CPUs are on the same
FSB(ignoring AMD SMP where each processor has an exclusive FSB, and this might be possible)?
That was what I was trying to say: same FSB speed * different multipliers = different CPU MHZ, like what the original poster is asking about. I don't think it is possible to configure a motherboard to have different FSB speeds for two processors.
Quoted 2 lines
There was a patch to compare the different features on the CPUs available and use the subset available on all processors.
Hmm, maybe if you had actually read my email and followed the URL I posted, you would have found the patch to which you refer ;-).
Cheers, Andreas
--
Andreas Dilger
http://www-mddsp.enel.ucalgary.ca/People/adilger/
http://sourceforge.net/projects/ext2resize/
Quoted 4 lines
That was what I was trying to say: same FSB speed * different multipliers = different CPU MHZ, like what the original poster is asking about. I don't think it is possible to configure a motherboard to have different FSB speeds for two processors.
The Intel pentium II/III FSB at least can't handle this with normal chipsets. You can run same FSB different multipliers but this not officially supported. I have code to detect the multiplier of each CPU but never got around to making the kernel detect this and turn off TSC usage so you need to boot with "notsc" in such a case
Quoted 4 lines · Alan Cox
I have code to detect the multiplier of each CPU but never got around to making the kernel detect this and turn off TSC usage so you need to boot with "notsc" in such a case
If the kernel knew multipliers couldn't it actually use the TSCs anyway? Of course it would take some work, but is there any reason why it would not be posible?
Signature · 2 lines
Kasper Dupont -- der bruger for meget tid på usenet. For sending spam use mailto:[email protected]
Quoted 3 lines
If the kernel knew multipliers couldn't it actually use the TSCs anyway? Of course it would take some work, but is there any reason why it would not be posible?
In 2.4 yes. In 2.5 it would be close to impossible due to the pre-empt code
Quoted 6 lines · Alan Cox
Quoted 4 lines
If the kernel knew multipliers couldn't it actually use the TSCs anyway? Of course it would take some work, but is there any reason why it would not be posible?
In 2.4 yes. In 2.5 it would be close to impossible due to the pre-empt code
Couldn't that be solved in one of the following ways?
1) Disable pre-emption while reading TSC and CPU nr.
2) Use affinity for processes pre-empted in kernel mode.
3) Disable pre-emption for SMP systems.
Signature · 2 lines
Kasper Dupont -- der bruger for meget tid på usenet. For sending spam use mailto:[email protected]
Quoted 5 lines
Couldn't that be solved in one of the following ways?
1) Disable pre-emption while reading TSC and CPU nr.
2) Use affinity for processes pre-empted in kernel mode.
3) Disable pre-emption for SMP systems.
You can solve it by disabling pre-emption (and given its questionable value doing so permanently might not be a bad idea). However if you simply disable pre-emption during udelay() calls then you've just screwed yourself by removing 99% of the use pre-emption had.
Given all the pain its probably better to not use the TSC
Quoted 9 lines · Alan Cox
Quoted 6 lines
Couldn't that be solved in one of the following ways?
1) Disable pre-emption while reading TSC and CPU nr.
2) Use affinity for processes pre-empted in kernel mode.
3) Disable pre-emption for SMP systems.You can solve it by disabling pre-emption (and given its questionable value doing so permanently might not be a bad idea).
Questionable value of what? TSC or preemption?
Quoted 3 lines
However if you simply
disable pre-emption duringudelay()calls then you've just screwed yourself by removing 99% of the use pre-emption had.
I wouldn't want to disable preemption during udelays. Either I would disable and enable preemption on every pass through the loop. Or I would just manually check for every pass if I should give up the CPU. This obviously requires more computation for every pass, but being a busy waiting loop I don't see a problem.
Otherwise I would lock the process to a fixed CPU for the duration of udelay.
Quoted 2 lines
Given all the pain its probably better to not use the TSC
Do we have better alternatives for high resolution time meassurements?
Signature · 2 lines
Kasper Dupont -- der bruger for meget tid på usenet. For sending spam use mailto:[email protected]
Quoted 4 lines
Quoted 2 lines
You can solve it by disabling pre-emption (and given its questionable value doing so permanently might not be a bad idea).
Questionable value of what? TSC or preemption?
Pre-emption
Quoted 3 lines
I wouldn't want to disable preemption during udelays. Either I would disable and enable preemption on every pass through the loop. Or I would just manually check
You've already made it too expensive too bother with
Quoted 6 lines · Kasper Dupont
Quoted 5 lines · Alan Cox
Quoted 4 lines
If the kernel knew multipliers couldn't it actually use the TSCs anyway? Of course it would take some work, but is there any reason why it would not be posible?
Quoted 8 lines
Quoted 1 line
In 2.4 yes. In 2.5 it would be close to impossible due to the pre-empt code
Couldn't that be solved in one of the following ways?
1) Disable pre-emption while reading TSC and CPU nr.
2) Use affinity for processes pre-empted in kernel mode.
3) Disable pre-emption for SMP systems.
Sorry to chime in on this if it isn't appreciated, but wouldn't #3 open some old wounds again? It seems that it would FMPOV.
Austin
Quoted 15 lines · Andreas Dilger
Quoted 6 lines · Mike Fedyk
Quoted 5 lines · Andreas Dilger
There was a kernel patch posted about 5 or so months ago which would "handle" this setup (CPUs with the same clock speed, but different multipliers). Alan Cox said it probably was a bad idea, so it wouldn't go into the kernel, but the patch may still be usable.
Quoted 7 lines
Quoted 2 lines
This is sometimes called "asymmetric multiprocessing", and the thread is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
I thought asymmetric multiprocessing would support CPUs with different speeds. ie, 400 & 450mhz. How would you get different multipliers and same Mhz when the CPUs are on the same
FSB(ignoring AMD SMP where each processor has an exclusive FSB, and this might be possible)?
Quoted 5 lines
That was what I was trying to say: same FSB speed * different multipliers = different CPU MHZ, like what the original poster is asking about. I don't think it is possible to configure a motherboard to have different FSB speeds for two processors.
Me neither, but it seems theoretically possible.
Quoted 4 lines
Quoted 2 lines
There was a patch to compare the different features on the CPUs available and use the subset available on all processors.
Hmm, maybe if you had actually read my email
I did. ;)
Quoted 1 line
and followed the URL I posted,
That I didn't do. :(
Quoted 1 line
you would have found the patch to which you refer ;-).
And you are correct. That is the exact patch to which I was referring.
Mike
Quoted 16 lines · Mike Fedyk
Quoted 15 lines · Andreas Dilger
Quoted 13 lines · Mike Fedyk
Quoted 7 lines · Andreas Dilger
There was a kernel patch posted about 5 or so months ago which would "handle" this setup (CPUs with the same clock speed, but different multipliers). Alan Cox said it probably was a bad idea, so it wouldn't go into the kernel, but the patch may still be usable.
This is sometimes called "asymmetric multiprocessing", and the thread is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
I thought asymmetric multiprocessing would support CPUs with different speeds. ie, 400 & 450mhz. How would you get different multipliers and same Mhz when the CPUs are on the same
FSB(ignoring AMD SMP where each processor has an exclusive FSB, and this might be possible)?
Quoted 7 lines
Quoted 5 lines
That was what I was trying to say: same FSB speed * different multipliers = different CPU MHZ, like what the original poster is asking about. I don't think it is possible to configure a motherboard to have different FSB speeds for two processors.
Me neither, but it seems theoretically possible.
It is not, they are both on the same FSB, at least in Pentium II/III case.
Signature · 2 lines
Vojtech Pavlik SuSE Labs
Quoted 23 lines · Vojtech Pavlik
Quoted 22 lines · Mike Fedyk
Quoted 20 lines · Andreas Dilger
Quoted 6 lines · Mike Fedyk
Quoted 5 lines · Andreas Dilger
There was a kernel patch posted about 5 or so months ago which would "handle" this setup (CPUs with the same clock speed, but different multipliers). Alan Cox said it probably was a bad idea, so it wouldn't go into the kernel, but the patch may still be usable.
Quoted 7 lines
Quoted 2 lines
This is sometimes called "asymmetric multiprocessing", and the thread is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
I thought asymmetric multiprocessing would support CPUs with different speeds. ie, 400 & 450mhz. How would you get different multipliers and same Mhz when the CPUs are on the same
FSB(ignoring AMD SMP where each processor has an exclusive FSB, and this might be possible)?That was what I was trying to say: same FSB speed * different multipliers = different CPU MHZ, like what the original poster is asking about. I don't think it is possible to configure a motherboard to have different FSB speeds for two processors.
Quoted 3 lines
Quoted 1 line
Me neither, but it seems theoretically possible.
It is not, they are both on the same FSB, at least in Pentium II/III case.
And if you read even what is quoted above in this message you would notice that we are not talking about Intel SMP, but AMD SMP, or more correctly, AMD SMP licensed from Alpha. What you say is true for Intel SMP though.
Quoted 24 lines · Mike Fedyk
Quoted 23 lines · Vojtech Pavlik
Quoted 22 lines · Mike Fedyk
Quoted 20 lines · Andreas Dilger
Quoted 6 lines · Mike Fedyk
Quoted 5 lines · Andreas Dilger
There was a kernel patch posted about 5 or so months ago which would "handle" this setup (CPUs with the same clock speed, but different multipliers). Alan Cox said it probably was a bad idea, so it wouldn't go into the kernel, but the patch may still be usable.
Quoted 7 lines
Quoted 2 lines
This is sometimes called "asymmetric multiprocessing", and the thread is at http://marc.theaimsgroup.com/?l=linux-kernel&m=98519070331478&w=4
I thought asymmetric multiprocessing would support CPUs with different speeds. ie, 400 & 450mhz. How would you get different multipliers and same Mhz when the CPUs are on the same
FSB(ignoring AMD SMP where each processor has an exclusive FSB, and this might be possible)?That was what I was trying to say: same FSB speed * different multipliers = different CPU MHZ, like what the original poster is asking about. I don't think it is possible to configure a motherboard to have different FSB speeds for two processors.
Quoted 7 lines
Quoted 3 lines
Quoted 1 line
Me neither, but it seems theoretically possible.
It is not, they are both on the same FSB, at least in Pentium II/III case.
And if you read even what is quoted above in this message you would notice that we are not talking about Intel SMP, but AMD SMP, or more correctly, AMD SMP licensed from Alpha. What you say is true for Intel SMP though.
"ignoring AMD SMP" doesn't seem you're talking about it ... anyway even with AMD chipsets while there are two different FSBs, they're still driven by a single northbridge with a single clock. Now with HyperTransport (like for example nVidia uses now in their nForce), I think it could work ...
Signature · 2 lines
Vojtech Pavlik SuSE Labs