Dual core Athlons and unsynced TSCs
It's been known for quite some time that the TSCs are not synced between cores on Athlon X2 machines and this screws up the kernel's timekeeping, as it still uses the TSC as the default time source on these machines.
This problem still seems to be present in the latest kernels. What is the plan to fix it? Is the fix simply to make the kernel use the ACPI PM timer by default on Athlon X2?
Lee
Quoted 9 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Lee Revell
Sent: Thursday, January 12, 2006 4:18 PM
To: linux-kernel
Subject: Dual core Athlons and unsynced TSCs
Quoted 8 lines
It's been known for quite some time that the TSCs are not synced between cores on Athlon X2 machines and this screws up the kernel's timekeeping, as it still uses the TSC as the default time source on these machines.
This problem still seems to be present in the latest kernels.
What is the plan to fix it? Is the fix simply to make the
kernel use the ACPI PM timer by default on Athlon X2?
Do we know if this also affects dual-core opterons?
The symptoms are that the clocks run at 2x the speed, correct?
Roger
Quoted 13 lines · Roger Heflin
Quoted 13 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Lee Revell
Sent: Thursday, January 12, 2006 4:18 PM
To: linux-kernel
Subject: Dual core Athlons and unsynced TSCsIt's been known for quite some time that the TSCs are not synced between cores on Athlon X2 machines and this screws up the kernel's timekeeping, as it still uses the TSC as the default time source on these machines.
Quoted 8 lines
Quoted 3 lines
This problem still seems to be present in the latest kernels.
What is the plan to fix it? Is the fix simply to make the
kernel use the ACPI PM timer by default on Athlon X2?Do we know if this also affects dual-core opterons?
The symptoms are that the clocks run at 2x the speed, correct?
No, worse. The monotonic clock can go backwards. The tscs of the CPUs are not in sync when one slows down due to idle. So if you read from two different CPUs, you may get the second read have an earlier time than the first. Breaks the rule of what a monotonic clock is.
-- Steve
Quoted 19 lines · Steven Rostedt
Quoted 18 lines · Roger Heflin
Quoted 16 lines
-----Original Message-----
From: [email protected] [mailto:[email protected]] On Behalf Of Lee Revell
Sent: Thursday, January 12, 2006 4:18 PM
To: linux-kernel
Subject: Dual core Athlons and unsynced TSCsIt's been known for quite some time that the TSCs are not synced between cores on Athlon X2 machines and this screws up the kernel's timekeeping, as it still uses the TSC as the default time source on these machines.
This problem still seems to be present in the latest kernels.
What is the plan to fix it? Is the fix simply to make the
kernel use the ACPI PM timer by default on Athlon X2?
Quoted 9 lines
Quoted 3 lines
Do we know if this also affects dual-core opterons?
The symptoms are that the clocks run at 2x the speed, correct?
No, worse. The monotonic clock can go backwards. The tscs of the CPUs are not in sync when one slows down due to idle. So if you read from two different CPUs, you may get the second read have an earlier time than the first. Breaks the rule of what a monotonic clock is.
Steve,
I don't have hardware to test this, can you confirm that the only workaround needed is to boot with "clock=pmtmr"?
Lee
Quoted 2 lines · Lee Revell
I don't have hardware to test this, can you confirm that the only workaround needed is to boot with "clock=pmtmr"?
For which kernel?
-- Steve
Quoted 6 lines · Steven Rostedt
Quoted 2 lines · Lee Revell
I don't have hardware to test this, can you confirm that the only workaround needed is to boot with "clock=pmtmr"?
For which kernel?
I believe the problem exists on all kernels, it should not matter.
Lee
Quoted 7 lines · Lee Revell
It's been known for quite some time that the TSCs are not synced between cores on Athlon X2 machines and this screws up the kernel's timekeeping, as it still uses the TSC as the default time source on these machines.
This problem still seems to be present in the latest kernels. What is the plan to fix it? Is the fix simply to make the kernel use the ACPI PM timer by default on Athlon X2?
If your BIOS has an ACPI "HPET" table, then the kernel can use the HPET timer instead. It doesn't solve the problem for apps or other kernel bits that use rdtsc directly, but there are other fixes for that floating around which will hopefully get consideration when they're mature.
Quoted 5 lines · On Fri, 2006-01-13 at 10:06 -0800, [email protected] wrote:
Quoted 4 lines · Lee Revell
It's been known for quite some time that the TSCs are not synced between cores on Athlon X2 machines and this screws up the kernel's timekeeping, as it still uses the TSC as the default time source on these machines.
Quoted 9 lines
Quoted 3 lines
This problem still seems to be present in the latest kernels. What is the plan to fix it? Is the fix simply to make the kernel use the ACPI PM timer by default on Athlon X2?
If your BIOS has an ACPI "HPET" table, then the kernel can use the HPET timer instead. It doesn't solve the problem for apps or other kernel bits that use rdtsc directly, but there are other fixes for that floating around which will hopefully get consideration when they're mature.
The apps can be converted to use clock_gettime(), but that does not help if the kernel can't keep reasonable time on these machines.
But if we just use "clock=pmtmr" by default on these machines the TSC is not a problem right?
Lee
Quoted 7 lines · Lee Revell
Quoted 4 lines
If your BIOS has an ACPI "HPET" table, then the kernel can use the HPET timer instead. It doesn't solve the problem for apps or other kernel bits that use rdtsc directly, but there are other fixes for that floating around which will hopefully get consideration when they're mature.
The apps can be converted to use
clock_gettime(), but that does not help if the kernel can't keep reasonable time on these machines.
Some apps/users need higher resolution and lower overhead that only rdtsc can offer currently.
Quoted 2 lines
But if we just use "clock=pmtmr" by default on these machines the TSC is not a problem right?
I never tried it with pmtimer, we had HPET available. Empirically TSC did not work (and we had a simple test case to show how bad it could get). HPET made that go away for users of gettimeofday().
We're exploring rdtsc-compatible solutions.
Quoted 6 lines · On Fri, 2006-01-13 at 10:16 -0800, [email protected] wrote:
Quoted 5 lines · Lee Revell
Quoted 4 lines
If your BIOS has an ACPI "HPET" table, then the kernel can use the HPET timer instead. It doesn't solve the problem for apps or other kernel bits that use rdtsc directly, but there are other fixes for that floating around which will hopefully get consideration when they're mature.
Quoted 6 lines
Quoted 2 lines
The apps can be converted to use
clock_gettime(), but that does not help if the kernel can't keep reasonable time on these machines.Some apps/users need higher resolution and lower overhead that only rdtsc can offer currently.
But obviously if the TSC gives wildly inaccurate results, it cannot be used no matter how low the overhead.
Quoted 9 lines
Quoted 2 lines
But if we just use "clock=pmtmr" by default on these machines the TSC is not a problem right?
I never tried it with pmtimer, we had HPET available. Empirically TSC did not work (and we had a simple test case to show how bad it could get). HPET made that go away for users of
gettimeofday().We're exploring rdtsc-compatible solutions.
Since timekeeping on these machines has always been completely broken, shouldn't the default time source have been changed to the PM timer or HPET as soon as the problem was known?
Lee
Quoted 9 lines · Lee Revell
Quoted 6 lines · Steven Rostedt
Quoted 2 lines · Lee Revell
I don't have hardware to test this, can you confirm that the only workaround needed is to boot with "clock=pmtmr"?
For which kernel?
I believe the problem exists on all kernels, it should not matter.
Well, I just booted up (and am currently running) 2.6.15-rt4-sr2 and I used "clock=pmtmr". I haven't got any
read_tsc: ACK! TSC went backward! Unsynced TSCs?
If that's what you want to know?
But the latency traces are still screwed up.
-- Steve
Quoted 5 lines · Lee Revell
Quoted 2 lines
Some apps/users need higher resolution and lower overhead that only rdtsc can offer currently.
But obviously if the TSC gives wildly inaccurate results, it cannot be used no matter how low the overhead.
unless we can re-sync the TSCs often enough that apps don't notice.
Quoted 10 lines
Quoted 6 lines
I never tried it with pmtimer, we had HPET available. Empirically TSC did not work (and we had a simple test case to show how bad it could get). HPET made that go away for users of
gettimeofday().We're exploring rdtsc-compatible solutions.
Since timekeeping on these machines has always been completely broken, shouldn't the default time source have been changed to the PM timer or HPET as soon as the problem was known?
If you have HPET, the kernel will prefer that. I don't know enough about the PMtimer overhead or resolution to say for sure.
Quoted 13 lines · Steven Rostedt
Quoted 9 lines · Lee Revell
Quoted 6 lines · Steven Rostedt
Quoted 2 lines · Lee Revell
I don't have hardware to test this, can you confirm that the only workaround needed is to boot with "clock=pmtmr"?
For which kernel?
I believe the problem exists on all kernels, it should not matter.
Quoted 7 lines
Well, I just booted up (and am currently running) 2.6.15-rt4-sr2 and I used "clock=pmtmr". I haven't got any
read_tsc: ACK! TSC went backward! Unsynced TSCs?If that's what you want to know?
I want to know if clock_gettime(CLOCK_MONOTONIC, *ts) is actually guaranteed to be monotonic on these machines AKA do we break POSIX compliance or not.
Lee
Quoted 9 lines · On Fri, 2006-01-13 at 10:55 -0800, [email protected] wrote:
Quoted 5 lines · Lee Revell
Quoted 2 lines
Some apps/users need higher resolution and lower overhead that only rdtsc can offer currently.
But obviously if the TSC gives wildly inaccurate results, it cannot be used no matter how low the overhead.
unless we can re-sync the TSCs often enough that apps don't notice.
You'd have to quantify that somehow, in terms of the max drift rate (ppm), and the max resolution available (< tsc frequency).
Either that, or track an offset, and use one TSC as truth, and update the correction factor for the other TSCs as often as needed, maybe?
This is kind of analogous to the "drift" NTP calculates against a free-running oscillator.
So you'd be pushing that functionality deeper into the OS-core.
Dave Mills had that "hardpps" stuff in there for a while, it might be a starting point.
Just some thoughts for now...
Sven
Quoted 5 lines
-
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/
Signature · 13 lines
*********************************** Sven-Thorsten Dietrich Real-Time Software Architect MontaVista Software, Inc. 1237 East Arques Ave. Sunnyvale, CA 94085 Phone: 408.992.4515 Fax: 408.328.9204 http://www.mvista.com Platform To Innovate ***********************************
Quoted 20 lines · Sven-Thorsten Dietrich
Quoted 9 lines · On Fri, 2006-01-13 at 10:55 -0800, [email protected] wrote:
Quoted 5 lines · Lee Revell
Quoted 2 lines
Some apps/users need higher resolution and lower overhead that only rdtsc can offer currently.
But obviously if the TSC gives wildly inaccurate results, it cannot be used no matter how low the overhead.
unless we can re-sync the TSCs often enough that apps don't notice.
You'd have to quantify that somehow, in terms of the max drift rate (ppm), and the max resolution available (< tsc frequency).
Either that, or track an offset, and use one TSC as truth, and update the correction factor for the other TSCs as often as needed, maybe?
This is kind of analogous to the "drift" NTP calculates against a free-running oscillator.
Quoted 7 lines
So you'd be pushing that functionality deeper into the OS-core.
Dave Mills had that "hardpps" stuff in there for a while, it might be a starting point.
Just some thoughts for now...
It kind of makes you wonder what in the heck AMD were thinking, whether they realized that this design decision would cause so many problems at the OS level (it's broken at least Linux and Solaris). Maybe Windows keeps time in a way that was unaffected by this?
Lee
Quoted 26 lines · Lee Revell
Quoted 25 lines · Sven-Thorsten Dietrich
Quoted 9 lines · On Fri, 2006-01-13 at 10:55 -0800, [email protected] wrote:
Quoted 5 lines · Lee Revell
Quoted 2 lines
Some apps/users need higher resolution and lower overhead that only rdtsc can offer currently.
But obviously if the TSC gives wildly inaccurate results, it cannot be used no matter how low the overhead.
unless we can re-sync the TSCs often enough that apps don't notice.
You'd have to quantify that somehow, in terms of the max drift rate (ppm), and the max resolution available (< tsc frequency).
Either that, or track an offset, and use one TSC as truth, and update the correction factor for the other TSCs as often as needed, maybe?
This is kind of analogous to the "drift" NTP calculates against a free-running oscillator.
So you'd be pushing that functionality deeper into the OS-core.
Dave Mills had that "hardpps" stuff in there for a while, it might be a starting point.
Quoted 8 lines
Quoted 2 lines
Just some thoughts for now...
It kind of makes you wonder what in the heck AMD were thinking, whether they realized that this design decision would cause so many problems at the OS level (it's broken at least Linux and Solaris). Maybe Windows keeps time in a way that was unaffected by this?
Last time I looked at Windows, we had to create an RT task to interpolate between jiffies, b/c Windows was only keeping time at the 10 / 16 (smp) ms resolution. (to get NTP to run)
That was 6 years ago.
I haven't looked back since, and may I turn to salt or stone if I ever do.
Quoted 2 lines
Lee
Signature · 13 lines
*********************************** Sven-Thorsten Dietrich Real-Time Software Architect MontaVista Software, Inc. 1237 East Arques Ave. Sunnyvale, CA 94085 Phone: 408.992.4515 Fax: 408.328.9204 http://www.mvista.com Platform To Innovate ***********************************
Le vendredi 13 janvier 2006 à 10:56 -0800, Sven-Thorsten Dietrich a écrit :
Quoted 8 lines · On Fri, 2006-01-13 at 10:55 -0800, [email protected] wrote:
Quoted 7 lines · Lee Revell
Quoted 5 lines
Quoted 2 lines
Some apps/users need higher resolution and lower overhead that only rdtsc can offer currently.
But obviously if the TSC gives wildly inaccurate results, it cannot be used no matter how low the overhead.
Quoted 8 lines
Quoted 2 lines
unless we can re-sync the TSCs often enough that apps don't notice.
You'd have to quantify that somehow, in terms of the max drift rate (ppm), and the max resolution available (< tsc frequency).
Either that, or track an offset, and use one TSC as truth, and update the correction factor for the other TSCs as often as needed, maybe?
As often as needed being each time a thread changes CPU ?
Quoted 12 lines · Sven-Thorsten Dietrich
Quoted 1 line · On Fri, 2006-01-13 at 10:55 -0800, [email protected] wrote:
unless we can re-sync the TSCs often enough that apps don't notice.
You'd have to quantify that somehow, in terms of the max drift rate (ppm), and the max resolution available (< tsc frequency).
Either that, or track an offset, and use one TSC as truth, and update the correction factor for the other TSCs as often as needed, maybe?
This is kind of analogous to the "drift" NTP calculates against a free-running oscillator.
Quoted 6 lines
So you'd be pushing that functionality deeper into the OS-core.
Dave Mills had that "hardpps" stuff in there for a while, it might be a starting point.
Just some thoughts for now...
There's some chatter here about a solution involving a lazy sync of TSCs to the HPET (or other) whenever an app calls rdtsc after a potentially unsyncing event.
For example, 'hlt' will initiate C1 which may cause clock ramping (and TSC skew). We can trap rdtsc after a hlt and re-sync the TSCs to some truly monotonic source, like HPET.
I don't have all the details, some problems remain, and the work is not quite done yet, but it looks promising.
Even if we eventually get synced TSCs, it's too little too late. basically, anything in-kernel that uses rdtsc is bound to break, and any app that uses rdtsc had better be using CPU affinity.
Tim
Quoted 6 lines · Lee Revell
Quoted 2 lines
If that's what you want to know?
I want to know if
clock_gettime(CLOCK_MONOTONIC, *ts)is actually guaranteed to be monotonic on these machines AKA do we break POSIX compliance or not.
Nope it doesn't work :-(
I ran this test:
http://www.kihontech.com/tests/rt/monotonic.c
And got this:
$ ./monotonic main program pid=8477 hello from thread 0! hello from thread 1! hello from thread 2! hello from thread 3! hello from thread 4! Failed! prev: 6279.925610302 current: 6279.874615349
$ cat /proc/cmdline root=/dev/md0 ro console=ttyS0,115200 console=tty0 nmi_watchdog=2 lapic earlyprintk=ttyS0,115200 clock=pmtmr
$ uname -r 2.6.15-rt4-sr2
(I also added Ingo, Thomas, and John in the CC)
I'll reboot to vanilla 2.6.15 and see if that is broken too (just to make sure)
-- Steve
Quoted 5 lines · Steven Rostedt
Nope it doesn't work :-(
I ran this test:
http://www.kihontech.com/tests/rt/monotonic.c
[...]
Quoted 4 lines
I'll reboot to vanilla 2.6.15 and see if that is broken too (just to make sure)
Failed on 2.6.15 too:
$ ./monotonic main program pid=6724 hello from thread 0! hello from thread 4! Failed! prev: 316.952599248 current: 316.951899248 hello from thread 3! hello from thread 2! hello from thread 1!
$ uname -r 2.6.15 $ cat /proc/cmdline root=/dev/md0 ro console=tty0 nmi_watchdog=2 lapic clock=pmtmr
:-(
-- Steve
Quoted 7 lines · Steven Rostedt
Quoted 5 lines · Steven Rostedt
Nope it doesn't work :-(
I ran this test:
http://www.kihontech.com/tests/rt/monotonic.c
Quoted 7 lines
[...]
Quoted 4 lines
I'll reboot to vanilla 2.6.15 and see if that is broken too (just to make sure)
Failed on 2.6.15 too:
That's what I suspected based on reports from several JACK users (JACK used to use rdtsc directly, it now uses clock_gettime).
Hmm, it looks like the kernel uses the TSC left and right, even if clock=pmtmr is used. There's even an rdtscll in pmtimer_mark_offset().
Wow, what a mess. Thanks, AMD...
Lee
Quoted 7 lines · Steven Rostedt
Quoted 5 lines · Steven Rostedt
Nope it doesn't work :-(
I ran this test:
http://www.kihontech.com/tests/rt/monotonic.c
Quoted 8 lines
[...]
Quoted 4 lines
I'll reboot to vanilla 2.6.15 and see if that is broken too (just to make sure)
Failed on 2.6.15 too:
Ugh. In arch/x86_64/kernel/time.c monotonic_clock() uses the TSC unconditionally.
Lee
Quoted 2 lines · Lee Revell
Ugh. In
arch/x86_64/kernel/time.cmonotonic_clock()uses the TSC unconditionally.
Can you try
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt2.patch
please ?
tglx
Quoted 10 lines · Thomas Gleixner
Quoted 2 lines · Lee Revell
Ugh. In
arch/x86_64/kernel/time.cmonotonic_clock()uses the TSC unconditionally.Can you try
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt2.patch
please ?
I can try it (well I can pass it on to someone who has the hardware), but that's a huge patch, not likely to be mergeable in the 2.6.16 timeframe. Surely there has to be a way this can be fixed for 2.6.16?
Lee
Quoted 8 lines · Lee Revell
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines · Lee Revell
Ugh. In
arch/x86_64/kernel/time.cmonotonic_clock()uses the TSC unconditionally.Can you try
Quoted 7 lines
Quoted 3 lines
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt2.patch
please ?
I can try it (well I can pass it on to someone who has the hardware), but that's a huge patch, not likely to be mergeable in the 2.6.16 timeframe. Surely there has to be a way this can be fixed for 2.6.16?
Parts of the patch are in mainline (-gitX), but those alone will not fix the problem AFAICS
tglx
Quoted 8 lines · Lee Revell
Quoted 7 lines · Thomas Gleixner
Quoted 2 lines · Lee Revell
Ugh. In
arch/x86_64/kernel/time.cmonotonic_clock()uses the TSC unconditionally.Can you try
Quoted 7 lines
Quoted 3 lines
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt2.patch
please ?
I can try it (well I can pass it on to someone who has the hardware), but that's a huge patch, not likely to be mergeable in the 2.6.16 timeframe. Surely there has to be a way this can be fixed for 2.6.16?
I'll try it.
If it works, at least it might give us a clue to what to do.
-- Steve
Quoted 28 lines · On Fri, 2006-01-13 at 11:32 -0800, [email protected] wrote:
Quoted 12 lines · Sven-Thorsten Dietrich
Quoted 1 line · On Fri, 2006-01-13 at 10:55 -0800, [email protected] wrote:
unless we can re-sync the TSCs often enough that apps don't notice.
You'd have to quantify that somehow, in terms of the max drift rate (ppm), and the max resolution available (< tsc frequency).
Either that, or track an offset, and use one TSC as truth, and update the correction factor for the other TSCs as often as needed, maybe?
This is kind of analogous to the "drift" NTP calculates against a free-running oscillator.
Quoted 6 lines
So you'd be pushing that functionality deeper into the OS-core.
Dave Mills had that "hardpps" stuff in there for a while, it might be a starting point.
Just some thoughts for now...
There's some chatter here about a solution involving a lazy sync of TSCs to the HPET (or other) whenever an app calls rdtsc after a potentially unsyncing event.
For example, 'hlt' will initiate C1 which may cause clock ramping (and TSC skew). We can trap rdtsc after a hlt and re-sync the TSCs to some truly monotonic source, like HPET.
Quoted 6 lines
I don't have all the details, some problems remain, and the work is not quite done yet, but it looks promising.
Even if we eventually get synced TSCs, it's too little too late. basically, anything in-kernel that uses rdtsc is bound to break, and any app that uses rdtsc had better be using CPU affinity.
I know this might be somewhat of an overhead, but what about resyncing on wakeup of the idle function? Isn't that where they fall apart?
-- Steve
Quoted 5 lines · Thomas Gleixner
Quoted 2 lines · Lee Revell
Ugh. In
arch/x86_64/kernel/time.cmonotonic_clock()uses the TSC unconditionally.
Quoted 9 lines
Can you try
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt2.patch
please ?
tglx
Hmm, it doesn't compile :(
-- Steve
rostedt@gandalf:~/work/kernels/linux-2.6.15-hrt2$ make
amdmake
CHK include/linux/version.h
CC arch/x86_64/kernel/asm-offsets.s
In file included from include/linux/timex.h:61,
from include/linux/sched.h:11,
from arch/x86_64/kernel/asm-offsets.c:7:
include/asm/timex.h: In function 'get_cycles_sync':
include/asm/timex.h:36: warning: implicit declaration of function 'alternative_io'
include/asm/timex.h:37: error: called object '"=a"' is not a function
include/asm/timex.h:37: error: called object '"0"' is not a function
include/asm/timex.h:37: error: syntax error before ':' token
make[1]: *** [arch/x86_64/kernel/asm-offsets.s] Error 1
make: *** [prepare0] Error 2Quoted 10 lines · Steven Rostedt
Quoted 9 lines · Thomas Gleixner
Quoted 2 lines · Lee Revell
Ugh. In
arch/x86_64/kernel/time.cmonotonic_clock()uses the TSC unconditionally.Can you try
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt2.patch
Quoted 7 lines
Quoted 5 lines
please ?
tglx
Hmm, it doesn't compile :(
Grmbl, I check this tomorrow. Getting late here
tglx
Quoted 13 lines · Thomas Gleixner
Quoted 12 lines · Steven Rostedt
Quoted 11 lines · Thomas Gleixner
Quoted 2 lines · Lee Revell
Ugh. In
arch/x86_64/kernel/time.cmonotonic_clock()uses the TSC unconditionally.Can you try
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt2.patch
please ?
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
tglx
Hmm, it doesn't compile :(
Grmbl, I check this tomorrow. Getting late here
OK, I'll probably wont work on it till Monday then (wifes rules ;)
-- Steve
Quoted 26 lines · Lee Revell
Quoted 25 lines · Sven-Thorsten Dietrich
Quoted 9 lines · On Fri, 2006-01-13 at 10:55 -0800, [email protected] wrote:
Quoted 5 lines · Lee Revell
Quoted 2 lines
Some apps/users need higher resolution and lower overhead that only rdtsc can offer currently.
But obviously if the TSC gives wildly inaccurate results, it cannot be used no matter how low the overhead.
unless we can re-sync the TSCs often enough that apps don't notice.
You'd have to quantify that somehow, in terms of the max drift rate (ppm), and the max resolution available (< tsc frequency).
Either that, or track an offset, and use one TSC as truth, and update the correction factor for the other TSCs as often as needed, maybe?
This is kind of analogous to the "drift" NTP calculates against a free-running oscillator.
So you'd be pushing that functionality deeper into the OS-core.
Dave Mills had that "hardpps" stuff in there for a while, it might be a starting point.
Quoted 7 lines
Quoted 2 lines
Just some thoughts for now...
It kind of makes you wonder what in the heck AMD were thinking, whether they realized that this design decision would cause so many problems at the OS level (it's broken at least Linux and Solaris). Maybe Windows keeps time in a way that was unaffected by this?
Lee, the last time I saw this discussion I thought it was identified that the multiple cores (or IIRC the multiple chips in a SMP motherboard) would only get out of sync when power management calls were made (hlt or switching the c-state). IIRC the workaround that was posted then was to just disable these in the kernel build.
if this is the case then I could easily see the hardware engineers thinking that if the software did a sleep thing then it was up to the software to re-sync the TSC clocks when the wakeup takes place.
David Lang
Signature · 2 lines
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare
Quoted 5 lines · David Lang
Lee, the last time I saw this discussion I thought it was identified that the multiple cores (or IIRC the multiple chips in a SMP motherboard) would only get out of sync when power management calls were made (hlt or switching the c-state). IIRC the workaround that was posted then was to just disable these in the kernel build.
not using 'hlt' when idling means that you spend 10s of Watts more power on mostly idle systems.
Quoted 9 lines · On Fri, 13 Jan 2006 [email protected] wrote:
Quoted 5 lines · David Lang
Lee, the last time I saw this discussion I thought it was identified that the multiple cores (or IIRC the multiple chips in a SMP motherboard) would only get out of sync when power management calls were made (hlt or switching the c-state). IIRC the workaround that was posted then was to just disable these in the kernel build.
not using 'hlt' when idling means that you spend 10s of Watts more power on mostly idle systems.
true, but for people who need better time accruacy then the other workaround this may be very acceptable.
David Lang
Signature · 2 lines
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare
Quoted 9 lines · David Lang
Quoted 7 lines · On Fri, 13 Jan 2006 [email protected] wrote:
Quoted 5 lines · David Lang
Lee, the last time I saw this discussion I thought it was identified that the multiple cores (or IIRC the multiple chips in a SMP motherboard) would only get out of sync when power management calls were made (hlt or switching the c-state). IIRC the workaround that was posted then was to just disable these in the kernel build.
Quoted 5 lines
Quoted 2 lines
not using 'hlt' when idling means that you spend 10s of Watts more power on mostly idle systems.
true, but for people who need better time accruacy then the other workaround this may be very acceptable.
sure, if power doesn't matter use idle=poll => problem solved.
Quoted 9 lines · David Lang
Quoted 7 lines · On Fri, 13 Jan 2006 [email protected] wrote:
Quoted 5 lines · David Lang
Lee, the last time I saw this discussion I thought it was identified that the multiple cores (or IIRC the multiple chips in a SMP motherboard) would only get out of sync when power management calls were made (hlt or switching the c-state). IIRC the workaround that was posted then was to just disable these in the kernel build.
Quoted 6 lines
Quoted 2 lines
not using 'hlt' when idling means that you spend 10s of Watts more power on mostly idle systems.
true, but for people who need better time accruacy then the other workaround this may be very acceptable.
1/4 KW / day for time synchronisation.
The power company would love that.
Quoted 2 lines
David Lang
Signature · 13 lines
*********************************** Sven-Thorsten Dietrich Real-Time Software Architect MontaVista Software, Inc. 1237 East Arques Ave. Sunnyvale, CA 94085 Phone: 408.992.4515 Fax: 408.328.9204 http://www.mvista.com Platform To Innovate ***********************************
Quoted 13 lines · Sven-Thorsten Dietrich
Quoted 12 lines · David Lang
Quoted 9 lines · On Fri, 13 Jan 2006 [email protected] wrote:
Quoted 5 lines · David Lang
Lee, the last time I saw this discussion I thought it was identified that the multiple cores (or IIRC the multiple chips in a SMP motherboard) would only get out of sync when power management calls were made (hlt or switching the c-state). IIRC the workaround that was posted then was to just disable these in the kernel build.
not using 'hlt' when idling means that you spend 10s of Watts more power on mostly idle systems.
Quoted 7 lines
Quoted 3 lines
true, but for people who need better time accruacy then the other workaround this may be very acceptable.
1/4 KW / day for time synchronisation.
The power company would love that.
more precisely 1/4 KW Hour / day
$0.01 - $0.02/day (I had to lookup the current rates)
they probably won't notice.
David Lang
Signature · 2 lines
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare
Quoted 16 lines · Steven Rostedt
Quoted 6 lines · Lee Revell
Quoted 2 lines
If that's what you want to know?
I want to know if
clock_gettime(CLOCK_MONOTONIC, *ts)is actually guaranteed to be monotonic on these machines AKA do we break POSIX compliance or not.Nope it doesn't work :-(
I ran this test:
http://www.kihontech.com/tests/rt/monotonic.cAnd got this:
Quoted 8 lines
$ ./monotonic main program pid=8477 hello from thread 0! hello from thread 1! hello from thread 2! hello from thread 3! hello from thread 4! Failed! prev: 6279.925610302 current: 6279.874615349
Not sure, but I think your test is broken.
While it serializes the checking and exchange of current and last, it doesn't serialize the actual calling of clock_gettime().
Thus you can get something like:
last=0
A: B:
1: now = clock_gettime()
2: now = clock_gettime()
3: atomic {
test(now, last) [2>0]
last = now
}
4: atomic {
test(now, last) [1>2]
FAIL!
}Attached is a similar testcase that I've been using myself that avoids this issue (although I just converted it from gettimeofday to clock_gettime, so there may still be an issue). Let me know if you have any comments on it.
thanks
-john
/* threadtest.c
* by: john stultz ([email protected])
* (C) Copyright IBM 2004, 2005, 2006
* Licensed under the GPL
*/
#include <stdio.h>
#include <sys/time.h>
#include <pthread.h>
/* serializes shared list access */ pthread_mutex_t list_lock = PTHREAD_MUTEX_INITIALIZER; /* serializes console output */ pthread_mutex_t print_lock = PTHREAD_MUTEX_INITIALIZER;
#define MAX_THREADS 128
#define LISTSIZE 128
int done = 0;
struct timespec global_list[LISTSIZE];
int listcount = 0;
void checklist(struct timespec* list, int size)
{
int i,j;
struct timespec *a,*b;
static int dbgct = 0;
if(!(dbgct++%7000)){
printf(".");
fflush(0);
}
/* scan the list */
for(i=0; i < size-1; i++){
a = &list[i];
b = &list[i+1];
/* look for any time inconsistencies */
if((b->tv_sec <= a->tv_sec)&&
(b->tv_nsec < a->tv_nsec)){
/* flag other threads */
done = 1;
/*serialize printing to avoid junky output*/
pthread_mutex_lock(&print_lock);
/* dump the list */
printf("\n");
for(j=0; j< size; j++){
if(j == i)
printf("---------------\n");
printf("%lu:%lu\n", list[j].tv_sec,list[j].tv_nsec);
if(j == i+1)
printf("---------------\n");
}
printf("TEST FAILED\n");
pthread_mutex_unlock(&print_lock);
}
}
}/* The Shared Thread shares a global list
* that each thread fills while holding the lock.
* This stresses clock syncronization across cpus. */
void* shared_thread(void* arg)
{
while(!done){
/* protect the list */
pthread_mutex_lock(&list_lock);
/* see if we're ready to check the list */
if(listcount >= LISTSIZE){
checklist(global_list, LISTSIZE);
listcount = 0;
}
clock_gettime(CLOCK_MONOTONIC, &global_list[listcount++]);
pthread_mutex_unlock(&list_lock);
}
}/* Each independent thread fills in its own
* list. This stresses clock_gettime() lock contention. */
void* independent_thread(void* arg)
{
struct timespec my_list[LISTSIZE];
int count;
while(!done){
/* fill the list */
for(count=0; count < LISTSIZE; count++)
clock_gettime(CLOCK_MONOTONIC, &my_list[count]);
checklist(my_list, LISTSIZE);
}
}int main(int argc, void** argv)
{
int thread_count = 1, i;
pthread_t pth[MAX_THREADS];
void* ret;
void* (*thread)(void*) = shared_thread;
/* pull the thread count */
if(argc > 1)
thread_count = strtol(argv[1],0,0);
if(thread_count > MAX_THREADS)
thread_count = MAX_THREADS;
/* check if we're running independent threads */
if(argc == 3 && !strncmp(argv[2],"indie", 5)){
thread = independent_thread;
printf("using independent threads\n");
}
system("date");
printf("spawning %i threads\n", thread_count,0,0);
/* spawn */
for(i=0; i < thread_count; i++)
pthread_create(&pth[i], 0, thread, 0);
/* wait */
for(i=0; i< thread_count; i++)
pthread_join(pth[i],&ret);
system("date");
/* die */
return 0;
}Quoted 29 lines · David Lang
Quoted 27 lines · Sven-Thorsten Dietrich
Quoted 14 lines · David Lang
Quoted 12 lines · On Fri, 13 Jan 2006 [email protected] wrote:
Quoted 8 lines · David Lang
Lee, the last time I saw this discussion I thought it was identified that
the multiple cores (or IIRC the multiple chips in a SMP motherboard) would
only get out of sync when power management calls were made (hlt or switching the c-state). IIRC the workaround that was posted then was to
just disable these in the kernel build.Quoted 8 lines
Quoted 3 lines
not using 'hlt' when idling means that you spend 10s of Watts more power
on mostly idle systems.true, but for people who need better time accruacy then the other workaround this may be very acceptable.
1/4 KW / day for time synchronisation.
Quoted 9 lines
Quoted 1 line
The power company would love that.
more precisely 1/4 KW Hour / day
$0.01 - $0.02/day (I had to lookup the current rates)
they probably won't notice.
Well, wait until there's AMD based dual core x86_64 laptops out there (this email being written on a single core x86_64 one). I can already see the faces of the unhappy future owners being told "use idle=poll" when on battery and anyway going deaf by fan noise.
(/me ducks and runs)
Signature · 1 line
Andreas Steinmetz SPAMmers use [email protected]
Quoted 10 lines · Andreas Steinmetz
Quoted 5 lines
more precisely 1/4 KW Hour / day
$0.01 - $0.02/day (I had to lookup the current rates)
they probably won't notice.
Well, wait until there's AMD based dual core
x86_64laptops out there (this email being written on a single corex86_64one). I can already see the faces of the unhappy future owners being told "use idle=poll" when on battery and anyway going deaf by fan noise.
Yeah, and then multiply that cost by a few tens of thousands of servers.
idle=poll is a hack at best.
Quoted 40 lines · Andreas Steinmetz
Quoted 29 lines · David Lang
Quoted 27 lines · Sven-Thorsten Dietrich
Quoted 14 lines · David Lang
Quoted 12 lines · On Fri, 13 Jan 2006 [email protected] wrote:
Quoted 8 lines · David Lang
Lee, the last time I saw this discussion I thought it was identified that
the multiple cores (or IIRC the multiple chips in a SMP motherboard) would
only get out of sync when power management calls were made (hlt or switching the c-state). IIRC the workaround that was posted then was to
just disable these in the kernel build.Quoted 8 lines
Quoted 3 lines
not using 'hlt' when idling means that you spend 10s of Watts more power
on mostly idle systems.true, but for people who need better time accruacy then the other workaround this may be very acceptable.
1/4 KW / day for time synchronisation.
Quoted 9 lines
Quoted 1 line
The power company would love that.
more precisely 1/4 KW Hour / day
$0.01 - $0.02/day (I had to lookup the current rates)
they probably won't notice.
Quoted 6 lines
Well, wait until there's AMD based dual core
x86_64laptops out there (this email being written on a single corex86_64one). I can already see the faces of the unhappy future owners being told "use idle=poll" when on battery and anyway going deaf by fan noise.(/me ducks and runs)
I'm not saying it's the right answer, but it's one of two workarounds currently available.
idle=poll causes increased power useage
timer source change (mentioned earlier in this thread) limits timer precision
neither of these are fixes, but by understanding the different costs people can choose the work around they want to use while waiting for a better fix.
David Lang
Signature · 2 lines
There are two ways of constructing a software design. One way is to make it so simple that there are obviously no deficiencies. And the other way is to make it so complicated that there are no obvious deficiencies. -- C.A.R. Hoare
Quoted 3 lines · john stultz
Quoted 1 line
Failed! prev: 6279.925610302 current: 6279.874615349
Not sure, but I think your test is broken.
Yep and it is now fixed:
http://www.kihontech.com/tests/rt/monotonic.c
Quoted 5 lines
Attached is a similar testcase that I've been using myself that avoids this issue (although I just converted it from gettimeofday to
clock_gettime, so there may still be an issue). Let me know if you have any comments on it.
Thanks, I'll add that to my list of tests too.
Oh and 2.6.15 passed as well (with clock=pmtmr)
-- Steve
Quoted 4 lines · Steven Rostedt
Thanks, I'll add that to my list of tests too.
Oh and 2.6.15 passed as well (with clock=pmtmr)
It really seems like it would fail if you gave it enough time due to the rdtsc in monotonic_clock()...
Lee
David Lang wrote:
[snip]
Quoted 11 lines
I'm not saying it's the right answer, but it's one of two workarounds currently available.
idle=poll causes increased power useage
timer source change (mentioned earlier in this thread) limits timer precision
neither of these are fixes, but by understanding the different costs people can choose the work around they want to use while waiting for a better fix.
The problem being that some of us use their laptops for audio work too. And then high battery usage, noisy fans or lack of high res timers will be really bad.
Simple example:
I do final mastering work using my laptop and Ardour/Jack/JAMin out of house in a place with a good stage audio system (Mackie mixer, 2KW Dynacord Amp/Syrincs S3) and a sufficient listening space to get a proper bass mix. I run on battery in this case to avoid any kind of audio interference (ground loops, etc). Now thinking of a dual core laptop...
Signature · 1 line
Andreas Steinmetz SPAMmers use [email protected]
Quoted 8 lines · Lee Revell
Quoted 4 lines · Steven Rostedt
Thanks, I'll add that to my list of tests too.
Oh and 2.6.15 passed as well (with clock=pmtmr)
It really seems like it would fail if you gave it enough time due to the rdtsc in
monotonic_clock()...
Currently monotonic_clock()is only used by the hangcheck-timer, and is not used by gettimeofday/clock_gettime (even w/ CLOCK_MONOTONIC).
So there may still be an issue there w/ the hangcheck-timer(for x86-64, on i386 the acpi pm timer can be used for monotonic_clock), but its doesn't affect the time related userland interfaces.
thanks
-john
Quoted 10 lines · john stultz
Quoted 8 lines · Lee Revell
Quoted 4 lines · Steven Rostedt
Thanks, I'll add that to my list of tests too.
Oh and 2.6.15 passed as well (with clock=pmtmr)
It really seems like it would fail if you gave it enough time due to the rdtsc in
monotonic_clock()...
Quoted 6 lines
Currently monotonic_clock()is only used by the hangcheck-timer, and is not used by gettimeofday/clock_gettime (even w/
CLOCK_MONOTONIC).So there may still be an issue there w/ the hangcheck-timer(for x86-64, on i386 the acpi pm timer can be used for
monotonic_clock), but its doesn't affect the time related userland interfaces.
OK so the last question is how do we make sure the kernel uses the clock=pmtmr behavior by default on those machines?
Lee
Quoted 14 lines · Lee Revell
Quoted 13 lines · john stultz
Quoted 8 lines · Lee Revell
Quoted 4 lines · Steven Rostedt
Thanks, I'll add that to my list of tests too.
Oh and 2.6.15 passed as well (with clock=pmtmr)
It really seems like it would fail if you gave it enough time due to the rdtsc in
monotonic_clock()...Currently monotonic_clock()is only used by the hangcheck-timer, and is not used by gettimeofday/clock_gettime (even w/
CLOCK_MONOTONIC).
Quoted 6 lines
Quoted 3 lines
So there may still be an issue there w/ the hangcheck-timer(for x86-64, on i386 the acpi pm timer can be used for
monotonic_clock), but its doesn't affect the time related userland interfaces.OK so the last question is how do we make sure the kernel uses the clock=pmtmr behavior by default on those machines?
This is as I understand it:
With 2.6.15 on x86-64:
If available, alternate timesources (HPET, ACPI PM) will be used if
available on AMD SMP systems. (clock= is i386 only)
With 2.6.15 on i386:
If CONFIG_X86_PM_TIMER is enabled, and available it is the preferred
clocksource over the TSC. Some distros have changed this priority causing the TSC to be preferred. In these cases clock=pmtmr is needed.
How's that?
-john
Quoted 8 lines · Andreas Steinmetz
Quoted 3 lines
neither of these are fixes, but by understanding the different costs people can choose the work around they want to use while waiting for a better fix.
The problem being that some of us use their laptops for audio work too. And then high battery usage, noisy fans or lack of high res timers will be really bad.
Quoted 7 lines
Simple example:
I do final mastering work using my laptop and Ardour/Jack/JAMin out of house in a place with a good stage audio system (Mackie mixer, 2KW Dynacord Amp/Syrincs S3) and a sufficient listening space to get a proper bass mix. I run on battery in this case to avoid any kind of audio interference (ground loops, etc). Now thinking of a dual core laptop...
Fixes are coming, just need to sort out the fidlly bits.
Quoted 6 lines · john stultz
This is as I understand it:
With 2.6.15 on x86-64:
If available, alternate timesources (HPET, ACPI PM) will be used if
available on AMD SMP systems. (clock= is i386 only)
Hmm, should I boot without the clock= to prove this?
-- Steve
Quoted 11 lines
With 2.6.15 on i386:
IfCONFIG_X86_PM_TIMERis enabled, and available it is the preferred
clocksource over the TSC. Some distros have changed this priority causing the TSC to be preferred. In these cases clock=pmtmr is needed.How's that?
-john
Quoted 10 lines · Steven Rostedt
Quoted 6 lines · john stultz
This is as I understand it:
With 2.6.15 on x86-64:
If available, alternate timesources (HPET, ACPI PM) will be used if
available on AMD SMP systems. (clock= is i386 only)Hmm, should I boot without the clock= to prove this?
Feel free. That or grep the x86-64 time.c code.
Look for:
time.c: Using PM based timekeeping.
To verify the timesource selection.
thanks
-john
Quoted 10 lines · john stultz
Quoted 9 lines · Steven Rostedt
Quoted 6 lines · john stultz
This is as I understand it:
With 2.6.15 on x86-64:
If available, alternate timesources (HPET, ACPI PM) will be used if
available on AMD SMP systems. (clock= is i386 only)
Quoted 8 lines
Quoted 1 line
Hmm, should I boot without the clock= to prove this?
Feel free. That or grep the x86-64
time.ccode.Look for:
time.c: Using PM based timekeeping.To verify the timesource selection.
I'm too busy trying to install linux on my powerbook to look at code at the moment :-)
I just rebooted and ran the test on 2.6.15 without the clock= option and it passed my test. I'll wait a while and run it again to see if I get any other drift too prove it more.
-- Steve
Quoted 10 lines · john stultz
Quoted 9 lines · Steven Rostedt
Quoted 6 lines · john stultz
This is as I understand it:
With 2.6.15 on x86-64:
If available, alternate timesources (HPET, ACPI PM) will be used if
available on AMD SMP systems. (clock= is i386 only)
Quoted 8 lines
Quoted 1 line
Hmm, should I boot without the clock= to prove this?
Feel free. That or grep the x86-64
time.ccode.Look for:
time.c: Using PM based timekeeping.To verify the timesource selection.
So much for not looking at code!
I uploaded yet another version of
http://www.kihontech.com/tests/rt/monotonic.c
that now also tests the tsc too, to see if it goes backwards. If it does, then it prints (only once) that fact and continues testing the gettime function.
So this proves that the tsc does go back and the gettime still is working.
$ ./monotonic main program pid=7137 hello from thread 0! hello from thread 4! last tsc is 4389431102233 this tsc is 4389429636684 hello from thread 1! hello from thread 2! hello from thread 3!
-- Steve
Quoted 3 lines · Lee Revell
I want to know if
clock_gettime(CLOCK_MONOTONIC, *ts)is actually guaranteed to be monotonic on these machines AKA do we break POSIX compliance or not.
Yes you do. Other arches have had the same issues.... See IA64 code.
Quoted 13 lines · David Lang
Quoted 7 lines · Andreas Steinmetz
David Lang wrote:
Well, wait until there's AMD based dual corex86_64laptops out there (this email being written on a single corex86_64one). I can already see the faces of the unhappy future owners being told "use idle=poll" when on battery and anyway going deaf by fan noise.(/me ducks and runs)
I'm not saying it's the right answer, but it's one of two workarounds currently available.
Quoted 8 lines
idle=poll causes increased power useage
timer source change (mentioned earlier in this thread) limits timer precision
neither of these are fixes, but by understanding the different costs people can choose the work around they want to use while waiting for a better fix.
A laptop user could also bind a process to a single CPU, and use the scaling min/max values to lock CPU speed to a single value. The TSC may still stop during HLT, but software must be handling that already.
Wouldn't that provide an accurate TSC?
Signature · 1 line
Zan Lynx <[email protected]>
Quoted 5 lines · Zan Lynx
A laptop user could also bind a process to a single CPU, and use the scaling min/max values to lock CPU speed to a single value. The TSC may still stop during HLT, but software must be handling that already.
Wouldn't that provide an accurate TSC?
monotonic but not linear. Also remember that the OS will use rdtsc here and there, and you can't affine the OS :)
Quoted 10 lines · On Sun, 2006-01-15 at 08:25 -0800, [email protected] wrote:
Quoted 5 lines · Zan Lynx
A laptop user could also bind a process to a single CPU, and use the scaling min/max values to lock CPU speed to a single value. The TSC may still stop during HLT, but software must be handling that already.
Wouldn't that provide an accurate TSC?
monotonic but not linear. Also remember that the OS will use rdtsc here and there, and you can't affine the OS :)
So the options are either to fix the TSC handling on these systems (by resyncing the TSCs when exiting from HLT), or eliminate the use of rdtsc by the OS?
Lee
Quoted 8 lines · Lee Revell
Quoted 7 lines · On Sun, 2006-01-15 at 08:25 -0800, [email protected] wrote:
Quoted 5 lines · Zan Lynx
A laptop user could also bind a process to a single CPU, and use the scaling min/max values to lock CPU speed to a single value. The TSC may still stop during HLT, but software must be handling that already.
Wouldn't that provide an accurate TSC?
Quoted 7 lines
Quoted 3 lines
monotonic but not linear. Also remember that the OS will use rdtsc here and there, and you can't affine the OS :)
So the options are either to fix the TSC handling on these systems (by resyncing the TSCs when exiting from HLT), or eliminate the use of rdtsc by the OS?
Or both. You can trap rdtsc users in userland, you have to manually audit kernel users.
It should be abolished or properly wrapped in kernel code, as soon as a resync path is available.
Quoted 17 lines · On Sun, 2006-01-15 at 10:21 -0800, [email protected] wrote:
Quoted 8 lines · Lee Revell
Quoted 7 lines · On Sun, 2006-01-15 at 08:25 -0800, [email protected] wrote:
Quoted 5 lines · Zan Lynx
A laptop user could also bind a process to a single CPU, and use the scaling min/max values to lock CPU speed to a single value. The TSC may still stop during HLT, but software must be handling that already.
Wouldn't that provide an accurate TSC?
Quoted 7 lines
Quoted 3 lines
monotonic but not linear. Also remember that the OS will use rdtsc here and there, and you can't affine the OS :)
So the options are either to fix the TSC handling on these systems (by resyncing the TSCs when exiting from HLT), or eliminate the use of rdtsc by the OS?
Quoted 6 lines
Or both. You can trap rdtsc users in userland, you have to manually audit kernel users.
It should be abolished or properly wrapped in kernel code, as soon as a resync path is available.
For the purposes of this discussion I was not considering direct use of rdtsc from userland for timing, I've accepted the arguments that this is a lost cause with today's hardware (although I maintain it was viable in practice on previous generations of hardware). Besides, rdtsc is useless from userspace if the kernel traps it, because the whole point of using it over gettimeofday() was that it was dirt cheap.
I'm content to make sure the kernel's internal timekeeping is solid.
Lee
Quoted 6 lines · Lee Revell
Quoted 5 lines
Or both. You can trap rdtsc users in userland, you have to manually audit kernel users.
It should be abolished or properly wrapped in kernel code, as soon as a resync path is available.
Quoted 8 lines
For the purposes of this discussion I was not considering direct use of rdtsc from userland for timing, I've accepted the arguments that this is a lost cause with today's hardware (although I maintain it was viable in practice on previous generations of hardware). Besides, rdtsc is useless from userspace if the kernel traps it, because the whole point of using it over
gettimeofday()was that it was dirt cheap.I'm content to make sure the kernel's internal timekeeping is solid.
You only need to trap rdtsc if you know that some unsyncronizing event has happened. E.g. 'hlt' or some other sleep state. If we sync every time we hlt, then we don't need to trap. But we can maybe be lazier about it.
The sync operation itself may not be cheap, and it may be useful to skip it if possible.
Quoted 4 lines · john stultz
With 2.6.15 on x86-64:
If available, alternate timesources (HPET, ACPI PM) will be used if
available on AMD SMP systems. (clock= is i386 only)
It would be good if it worked on x86-64 too - simply to unconfuse people. It's somewhere on my todo list, but patches welcome.
Quoted 4 lines
With 2.6.15 on i386:
IfCONFIG_X86_PM_TIMERis enabled, and available it is the preferred
clocksource over the TSC. Some distros have changed this priority causing the TSC to be preferred. In these cases clock=pmtmr is needed.
One problem is that it is not obvious enough to people that CONFIG_X86_PM_TIMER is really needed for correct timing on many system and they just don't enable it. That is why we got so many bogus reports.
I just changed it on x86-64 to be dependent on EMBEDDED, on by default. i386 probably should do this change too.
-Andi
Quoted 7 lines · Steven Rostedt
Quoted 5 lines
Quoted 2 lines
Hmm, it doesn't compile :(
Grmbl, I check this tomorrow. Getting late here
OK, I'll probably wont work on it till Monday then (wifes rules ;)
Can you try this one please ?
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt3.patch
tglx
Quoted 4 lines · Thomas Gleixner
Quoted 3 lines · Steven Rostedt
Quoted 3 lines
Quoted 2 lines
Hmm, it doesn't compile :(
Quoted 8 lines
Quoted 4 lines
Quoted 2 lines
Grmbl, I check this tomorrow. Getting late here
OK, I'll probably wont work on it till Monday then (wifes rules ;)
Can you try this one please ?
http://tglx.de/projects/hrtimers/2.6.15/patch-2.6.15-hrt3.patch
It compiled.
I'll boot into it now.
-- Steve