i386: vgetcpu(), take 2
In-Reply-To: <[email protected]>
Quoted 10 lines · Ingo Molnar
Quoted 3 lines · Chuck Ebbert
Use a GDT entry's limit field to store per-cpu data for fast access from userspace, and provide a vsyscall to access the current CPU number stored there.
very nice idea! I thought of doing
sys_get_cpu()too, but my idea was to use the scheduler to keep a writable [and permanently pinned, per-thread] VDSO data page uptodate with the current CPU# [and other interesting data]. Btw., do we know how fast LSL is on modern CPUs?
Now that the GDT is a full page for each CPU there's plenty of space for all kinds of per-cpu data, even if we waste 75% of it. LSL seems pretty fast; I got 13 clocks for the whole lsl/jnz/and sequence on K8 and 21 clocks on PII. Myabe you can test P4?
/* test how fast lsl/jnz/and runs.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>
#define rdtscll(t) asm volatile ("rdtsc" : "=A" (t))
#ifndef ITERS
#define ITERS 1000000
#endif
int main(int argc, char * const argv[])
{
unsigned long long tsc1, tsc2;
int count, cpu, junk;
rdtscll(tsc1);
asm (
" pushl %%ds \n"
" popl %2 \n"
"1: \n"
#ifdef DO_TEST
" lsl %2,%0 \n"
" jnz 2f \n"
" and $0xff,%0 \n"
#endif
" dec %1 \n"
" jnz 1b \n"
"2: \n"
: "=&r" (cpu), "=&r" (count), "=&r" (junk)
: "1" (ITERS), "0" (-1)
);
rdtscll(tsc2);
if (count == 0)
printf("loops: %d, avg: %llu clocks\n",
ITERS, (tsc2 - tsc1) / ITERS);
return 0;
}Quoted 10 lines
Review
Quoted 9 lines
(quoted diff) +9
-0+__vgetcpu: +.LSTART_vgetcpu: + movl $-EFAULT,%eax + movl $((27<<3)|3),%edx + lsll %edx,%eax + jnz 1f + andl $0xff,%eax +1: + ret
Quoted 2 lines
this needs unwinder annotations as well to make this a proper DSO, so that for example a breakpoint here does not confuse gdb.
I can't write those.
Quoted 1 line
also, would be nice to do something like this in 64-bit mode too.
Andi has x86_64 patches in his tree and is considering this method for ia32 support.
Signature · 2 lines
Chuck "You can't read a newspaper if you can't read." --George W. Bush
Quoted 10 lines · Chuck Ebbert
In-Reply-To: <[email protected]>
Quoted 6 lines · Ingo Molnar
Quoted 3 lines · Chuck Ebbert
Use a GDT entry's limit field to store per-cpu data for fast access from userspace, and provide a vsyscall to access the current CPU number stored there.
Quoted 8 lines
Quoted 4 lines
very nice idea! I thought of doing
sys_get_cpu()too, but my idea was to use the scheduler to keep a writable [and permanently pinned, per-thread] VDSO data page uptodate with the current CPU# [and other interesting data]. Btw., do we know how fast LSL is on modern CPUs?Now that the GDT is a full page for each CPU there's plenty of space for all kinds of per-cpu data, even if we waste 75% of it. LSL seems pretty fast; I got 13 clocks for the whole lsl/jnz/and sequence on K8
My measurements show different - i get 60+ cycles on K8 and 150+ cycles on P4. That is with a full vsyscall around it. However it is still far better than CPUID, however slower than RDTSCP on those CPUs that support it.
I changed the CPUID fallback path to use LSL on x86-64
Quoted 28 lines
and 21 clocks on PII. Myabe you can test P4?
/* test how fast lsl/jnz/and runs.
*/
#define_GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>#define rdtscll(t) asm volatile ("rdtsc" : "=A" (t))
#ifndef ITERS
#define ITERS 1000000
#endifint
main(int argc, char * const argv[]){ unsigned long long tsc1, tsc2; int count, cpu, junk; rdtscll(tsc1); asm ( " pushl %%ds \n" " popl %2 \n" "1: \n" #ifdef DO_TEST " lsl %2,%0 \n" " jnz 2f \n" " and $0xff,%0 \n"
Quoted 8 lines
#endif " dec %1 \n" " jnz 1b \n" "2: \n" : "=&r" (cpu), "=&r" (count), "=&r" (junk) : "1" (ITERS), "0" (-1) ); rdtscll(tsc2);
Measuring this way is a bad idea because you get far too much noise from the RDTSCs. Usually you need to put a a few thousands entry loop inside the RDTSCP and devide the result by the loop count
-Andi
Quoted 1 line
Quoted 6 lines · Andi Kleen
My measurements show different - i get 60+ cycles on K8 and 150+ cycles on P4. That is with a full vsyscall around it. However it is still far better than CPUID, however slower than RDTSCP on those CPUs that support it.
I changed the CPUID fallback path to use LSL on x86-64
One note of warning:
Playing "clever games" has a real tendency to suck badly eventually. I'm betting LSL is pretty damn low on any list of instructions to be optimized by the CPU core, so it would tend to always be microcoded, while other ops might get faster.
Quoted 3 lines
Measuring this way is a bad idea because you get far too much noise from the RDTSCs. Usually you need to put a a few thousands entry loop inside the RDTSCP and devide the result by the loop count
And measuring that way isn't perfect either, because it tends to show you how well an instruction works in that particular instruction mix, but not necessarily in real life.
Benchmarking single instructions is simply damn hard. It's often better to try to find a real load where that particular sequence is important enough to be measurable at all, and then try the alternatives. Not perfect either, but if you can't find such a load, maybe you shouldn't be doing it in the first place.. And if you _can_ find such a real load, at least you measured something that was actually real.
Linus
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Andi Kleen
My measurements show different - i get 60+ cycles on K8 and 150+ cycles on P4. That is with a full vsyscall around it. However it is still far better than CPUID, however slower than RDTSCP on those CPUs that support it.
Quoted 8 lines
Quoted 1 line
I changed the CPUID fallback path to use LSL on x86-64
One note of warning:
Playing "clever games" has a real tendency to suck badly eventually. I'm betting LSL is pretty damn low on any list of instructions to be optimized by the CPU core, so it would tend to always be microcoded, while other ops might get faster.
Any way we use to get the current CPU number is microcoded. Unless RDTSCP and CPUID LSL is not defined to flush any pipelines fortunately. And with the cache it is not THAT critical.
Quoted 8 lines
Quoted 3 lines
Measuring this way is a bad idea because you get far too much noise from the RDTSCs. Usually you need to put a a few thousands entry loop inside the RDTSCP and devide the result by the loop count
And measuring that way isn't perfect either, because it tends to show you how well an instruction works in that particular instruction mix, but not necessarily in real life.
Quoted 6 lines
Benchmarking single instructions is simply damn hard. It's often better to try to find a real load where that particular sequence is important enough to be measurable at all, and then try the alternatives. Not perfect either, but if you can't find such a load, maybe you shouldn't be doing it in the first place.. And if you _can_ find such a real load, at least you measured something that was actually real.
I benchmarked it in a faithful simulation of a x86-64 vsyscall
-Andi