Skip to contentSkip to conversation

i386: vgetcpu(), take 2

RFC PATCH Chuck Ebbert 3 participants 4 messages Jun 21, 2006

lore
Share & tools
lore
#0 lore raw

In-Reply-To: <[email protected]>

Quoted 10 lines · Ingo Molnar
Quoted 3 lines · Chuck Ebbert

Use a GDT entry's limit field to store per-cpu data for fast access from userspace, and provide a vsyscall to access the current CPU number stored there.

very nice idea! I thought of doing sys_get_cpu() too, but my idea was to use the scheduler to keep a writable [and permanently pinned, per-thread] VDSO data page uptodate with the current CPU# [and other interesting data]. Btw., do we know how fast LSL is on modern CPUs?

Now that the GDT is a full page for each CPU there's plenty of space for all kinds of per-cpu data, even if we waste 75% of it. LSL seems pretty fast; I got 13 clocks for the whole lsl/jnz/and sequence on K8 and 21 clocks on PII. Myabe you can test P4?

/* test how fast lsl/jnz/and runs.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>

#define rdtscll(t) asm volatile ("rdtsc" : "=A" (t))

#ifndef ITERS
#define ITERS 1000000
#endif

int main(int argc, char * const argv[])

{
	unsigned long long tsc1, tsc2;
	int count, cpu, junk;

	rdtscll(tsc1);
	asm (
		"	pushl %%ds		\n"
		"	popl %2			\n"
		"1:				\n"
#ifdef DO_TEST
		"	lsl %2,%0		\n"
		"	jnz 2f			\n"
		"	and $0xff,%0		\n"
#endif
		"	dec %1			\n"
		"	jnz 1b			\n"
		"2:				\n"
		: "=&r" (cpu), "=&r" (count), "=&r" (junk)
		: "1" (ITERS), "0" (-1)
	);
	rdtscll(tsc2);

	if (count == 0)
		printf("loops: %d, avg: %llu clocks\n",
			ITERS, (tsc2 - tsc1) / ITERS);
	return 0;
}
Quoted 10 lines

Review

Quoted 9 lines
(quoted diff) +9 -0
+__vgetcpu:
+.LSTART_vgetcpu:
+   movl $-EFAULT,%eax
+   movl $((27<<3)|3),%edx
+   lsll %edx,%eax
+   jnz 1f
+   andl $0xff,%eax
+1:
+   ret
Quoted 2 lines

this needs unwinder annotations as well to make this a proper DSO, so that for example a breakpoint here does not confuse gdb.

I can't write those.

Quoted 1 line

also, would be nice to do something like this in 64-bit mode too.

Andi has x86_64 patches in his tree and is considering this method for ia32 support.

Signature · 2 lines
Chuck
 "You can't read a newspaper if you can't read."  --George W. Bush
#1 lore raw
Quoted 10 lines · Chuck Ebbert

In-Reply-To: <[email protected]>

Quoted 6 lines · Ingo Molnar
Quoted 3 lines · Chuck Ebbert

Use a GDT entry's limit field to store per-cpu data for fast access from userspace, and provide a vsyscall to access the current CPU number stored there.

Quoted 8 lines
Quoted 4 lines

very nice idea! I thought of doing sys_get_cpu() too, but my idea was to use the scheduler to keep a writable [and permanently pinned, per-thread] VDSO data page uptodate with the current CPU# [and other interesting data]. Btw., do we know how fast LSL is on modern CPUs?

Now that the GDT is a full page for each CPU there's plenty of space for all kinds of per-cpu data, even if we waste 75% of it. LSL seems pretty fast; I got 13 clocks for the whole lsl/jnz/and sequence on K8

My measurements show different - i get 60+ cycles on K8 and 150+ cycles on P4. That is with a full vsyscall around it. However it is still far better than CPUID, however slower than RDTSCP on those CPUs that support it.

I changed the CPUID fallback path to use LSL on x86-64

Quoted 28 lines

and 21 clocks on PII. Myabe you can test P4?

/* test how fast lsl/jnz/and runs.
*/
#define _GNU_SOURCE
#include <stdio.h>
#include <stdlib.h>

#define rdtscll(t) asm volatile ("rdtsc" : "=A" (t))

#ifndef ITERS
#define ITERS 1000000
#endif

int main(int argc, char * const argv[])

{
	unsigned long long tsc1, tsc2;
	int count, cpu, junk;

	rdtscll(tsc1);
	asm (
		"	pushl %%ds		\n"
		"	popl %2			\n"
		"1:				\n"
#ifdef DO_TEST
		"	lsl %2,%0		\n"
		"	jnz 2f			\n"
		"	and $0xff,%0		\n"
Quoted 8 lines
#endif
		"	dec %1			\n"
		"	jnz 1b			\n"
		"2:				\n"
		: "=&r" (cpu), "=&r" (count), "=&r" (junk)
		: "1" (ITERS), "0" (-1)
	);
	rdtscll(tsc2);

Measuring this way is a bad idea because you get far too much noise from the RDTSCs. Usually you need to put a a few thousands entry loop inside the RDTSCP and devide the result by the loop count

-Andi

Quoted 1 line
#2 lore raw
Quoted 6 lines · Andi Kleen

My measurements show different - i get 60+ cycles on K8 and 150+ cycles on P4. That is with a full vsyscall around it. However it is still far better than CPUID, however slower than RDTSCP on those CPUs that support it.

I changed the CPUID fallback path to use LSL on x86-64

One note of warning:

Playing "clever games" has a real tendency to suck badly eventually. I'm betting LSL is pretty damn low on any list of instructions to be optimized by the CPU core, so it would tend to always be microcoded, while other ops might get faster.

Quoted 3 lines

Measuring this way is a bad idea because you get far too much noise from the RDTSCs. Usually you need to put a a few thousands entry loop inside the RDTSCP and devide the result by the loop count

And measuring that way isn't perfect either, because it tends to show you how well an instruction works in that particular instruction mix, but not necessarily in real life.

Benchmarking single instructions is simply damn hard. It's often better to try to find a real load where that particular sequence is important enough to be measurable at all, and then try the alternatives. Not perfect either, but if you can't find such a load, maybe you shouldn't be doing it in the first place.. And if you _can_ find such a real load, at least you measured something that was actually real.

Linus

#3 lore raw
Quoted 7 lines · Linus Torvalds
Quoted 5 lines · Andi Kleen

My measurements show different - i get 60+ cycles on K8 and 150+ cycles on P4. That is with a full vsyscall around it. However it is still far better than CPUID, however slower than RDTSCP on those CPUs that support it.

Quoted 8 lines
Quoted 1 line

I changed the CPUID fallback path to use LSL on x86-64

One note of warning:

Playing "clever games" has a real tendency to suck badly eventually. I'm betting LSL is pretty damn low on any list of instructions to be optimized by the CPU core, so it would tend to always be microcoded, while other ops might get faster.

Any way we use to get the current CPU number is microcoded. Unless RDTSCP and CPUID LSL is not defined to flush any pipelines fortunately. And with the cache it is not THAT critical.

Quoted 8 lines
Quoted 3 lines

Measuring this way is a bad idea because you get far too much noise from the RDTSCs. Usually you need to put a a few thousands entry loop inside the RDTSCP and devide the result by the loop count

And measuring that way isn't perfect either, because it tends to show you how well an instruction works in that particular instruction mix, but not necessarily in real life.

Quoted 6 lines

Benchmarking single instructions is simply damn hard. It's often better to try to find a real load where that particular sequence is important enough to be measurable at all, and then try the alternatives. Not perfect either, but if you can't find such a load, maybe you shouldn't be doing it in the first place.. And if you _can_ find such a real load, at least you measured something that was actually real.

I benchmarked it in a faithful simulation of a x86-64 vsyscall

-Andi