get_cycles() on i386
Folks,
Certain distributions are building all of their SMP kernels
NUMA-aware. This is great, as the kernels support boxes like the x440 with no trouble. However, this implicitly disables CONFIG_X86_TSC. While that is good for NUMA systems, and fine from a kernel timing standpoint, it also eliminates any generic access to the TSC via get_cycles(). With CONFIG_X86_TSC not defined, get_cycles() always returns 0.
Given that >95% of machines will not be x440s, this means that a
user of that kernel cannot access a high resolution timer via get_cycles(). I don't want to have to litter my code with rdtscll() when I managed to remove it!
The proposed patch is trivial. If the system has a TSC, it is
available get_cycles(). This makes no change to the other parts of the kernel protected by CONFIG_X86_TSC.
Joel
include/asm-i386/timex.h renamed from kernel-2.4.21-4.0.1.EL/linux-2.4.21/include/asm-i386/timex.h +1 -1
@@ -40,7 +40,7 @@static inline cycles_t get_cycles (void ) { -#ifndef CONFIG_X86_TSC +#ifndef CONFIG_X86_HAS_TSC return 0 ;#else unsigned long long ret;
Signature · 8 lines
"Hey mister if you're gonna walk on water, Could you drop a line my way?" Joel Becker Senior Member of Technical Staff Oracle Corporation E-mail: [email protected] Phone: (650) 506-8127
Quoted 7 lines · Joel Becker
Folks,
Certain distributions are building all of their SMP kernels
NUMA-aware. This is great, as the kernels support boxes like the x440 with no trouble. However, this implicitly disablesCONFIG_X86_TSC. While that is good for NUMA systems, and fine from a kernel timing standpoint, it also eliminates any generic access to the TSC viaget_cycles(). WithCONFIG_X86_TSCnot defined,get_cycles()always
Quoted 8 lines
returns 0.
Given that >95% of machines will not be x440s, this means that a
user of that kernel cannot access a high resolution timer viaget_cycles(). I don't want to have to litter my code withrdtscll()when I managed to remove it!
The proposed patch is trivial. If the system has a TSC, it is
availableget_cycles(). This makes no change to the other parts of the kernel protected byCONFIG_X86_TSC.
CONFIG_X86_TSC be the devil. Personally, I'd much prefer dropping the compile time option and using dynamic detection. Something like (not recently tested and i believe against 2.5.something, but you get the idea):
include/asm-i386/timex.h +3 -7
@@ -40,14 +40,10 @@static inline cycles_t get_cycles (void ) { -#ifndef CONFIG_X86_TSC -return 0 ; -#else -unsigned long long ret; - -rdtscll (ret); +unsigned long long ret =0 ; +if (cpu_has_tsc) +rdtscll (ret);return ret; -#endif }extern unsigned long cpu_khz;
thanks
-john
Quoted 5 lines · john stultz
CONFIG_X86_TSCbe the devil. Personally, I'd much prefer dropping the compile time option and using dynamic detection. Something like (not recently tested and i believe against 2.5.something, but you get the idea):
Some of the users are really timing-critical (eg scheduler).
How about just using the "alternative()" infrastructure that we already have in 2.6.x for this? See <asm-i386/system.h> for details.
We don't have an "alternative_output()" available yet, but using that it would look something like:
static inline unsigned long long get_cycle(void)
{
unsigned long long tsc;
alternative_output(
"xorl %%eax,%%eax ; xorl %%edx,%%edx",
"rdtsc",
X86_FEATURE_TSC,
"=A" (tsc));
return tsc;
}which should allow for "perfect" code (well, gcc tends to mess up 64-bit stuff, but you get the idea).
We use the "alternative_input()" thing for prefetch() handling (see <asm-i386/processor.h>).
Linus
Quoted 10 lines · Linus Torvalds
Quoted 5 lines · john stultz
CONFIG_X86_TSCbe the devil. Personally, I'd much prefer dropping the compile time option and using dynamic detection. Something like (not recently tested and i believe against 2.5.something, but you get the idea):Some of the users are really timing-critical (eg scheduler).
The scheduler uses its own sched_clock which only gives jiffies resolution if CONFIG_NUMA is defined. Unfortunate because I think its interactive behaviour isn't so good with ms resolution.
The scheduler does not need to have synchronised TSCs though, I think. It just means 2 more calls to sched_clock in a slow path (smp migration).
Quoted 15 lines · Nick Piggin
Quoted 10 lines · Linus Torvalds
Quoted 5 lines · john stultz
CONFIG_X86_TSCbe the devil. Personally, I'd much prefer dropping the compile time option and using dynamic detection. Something like (not recently tested and i believe against 2.5.something, but you get the idea):Some of the users are really timing-critical (eg scheduler).
Quoted 7 lines
The scheduler uses its own
sched_clockwhich only gives jiffies resolution ifCONFIG_NUMAis defined. Unfortunate because I think its interactive behaviour isn't so good with ms resolution.The scheduler does not need to have synchronised TSCs though, I think. It just means 2 more calls to
sched_clockin a slow path (smp migration).
Well no, its much trickier than that I think :(
Quoted 28 lines · Linus Torvalds
Quoted 5 lines · john stultz
CONFIG_X86_TSCbe the devil. Personally, I'd much prefer dropping the compile time option and using dynamic detection. Something like (not recently tested and i believe against 2.5.something, but you get the idea):Some of the users are really timing-critical (eg scheduler).
How about just using the "
alternative()" infrastructure that we already have in 2.6.x for this? See <asm-i386/system.h> for details.We don't have an "
alternative_output()" available yet, but using that it would look something like:static inline unsigned long long get_cycle(void) { unsigned long long tsc; alternative_output( "xorl %%eax,%%eax ; xorl %%edx,%%edx", "rdtsc", X86_FEATURE_TSC, "=A" (tsc)); return tsc; }
Quoted 5 lines
which should allow for "perfect" code (well, gcc tends to mess up 64-bit stuff, but you get the idea).
We use the "
alternative_input()" thing forprefetch()handling (see <asm-i386/processor.h>).
I'm not confident this is something for 2.4.
The "if (cpu_has_tsc)" fix from John sounds fine.
Any concerns?
Quoted 34 lines · john stultz
Quoted 7 lines · Joel Becker
Folks,
Certain distributions are building all of their SMP kernels
NUMA-aware. This is great, as the kernels support boxes like the x440 with no trouble. However, this implicitly disablesCONFIG_X86_TSC. While that is good for NUMA systems, and fine from a kernel timing standpoint, it also eliminates any generic access to the TSC viaget_cycles(). WithCONFIG_X86_TSCnot defined,get_cycles()alwaysQuoted 8 lines
returns 0.
Given that >95% of machines will not be x440s, this means that a
user of that kernel cannot access a high resolution timer viaget_cycles(). I don't want to have to litter my code withrdtscll()when I managed to remove it!
The proposed patch is trivial. If the system has a TSC, it is
availableget_cycles(). This makes no change to the other parts of the kernel protected byCONFIG_X86_TSC.
CONFIG_X86_TSCbe the devil. Personally, I'd much prefer dropping the compile time option and using dynamic detection. Something like (not recently tested and i believe against 2.5.something, but you get the idea):diff -Nru
a/include/asm-i386/timex.hb/include/asm-i386/timex.hinclude/asm-i386/timex.h +0
-4@@ -40,14 +40,10 @@static inline cycles_t get_cycles (void ) { -#ifndef CONFIG_X86_TSC -return 0 ; -#else -unsigned long long ret;
include/asm-i386/timex.h line 49
Quoted 10 lines
include/asm-i386/timex.h +3
-3- -rdtscll (ret); +unsigned long long ret =0 ; +if (cpu_has_tsc) +rdtscll (ret);return ret; -#endif }extern unsigned long cpu_khz;
John, Joel,
I believe this is reliable. I'll apply it.
Any concerns?