x86/vdso: RDPID support and a test

PATCH New Andy Lutomirski 2 participants 5 messages Nov 16, 2016 – Nov 17, 2016

Subsystems (2)

x86 Kselftest

lore

Share & tools

lore

Andy Lutomirski

author

2016-11-16 18:23 UTC #0 lore raw

cover x86/vdso: RDPID support and a test

RDPID is a new instruction in unspecified future Intel processors (sigh) that returns MSR_TSC_AUX. This will finally let the vDSO stop using LSL.

Patch 1 is the meat and patch 2 is the test. Megha has kindly tested them for me, since I don't own an unspecified Intel processor :)

Andy Lutomirski (2):
x86/vdso: Use RDPID in preference to LSL when available selftests/x86: Add test_vdso to test getcpu

 arch/x86/include/asm/cpufeatures.h      |   1 +
 arch/x86/include/asm/vgtod.h            |   7 +-
 tools/testing/selftests/x86/Makefile    |   2 +-
 tools/testing/selftests/x86/test_vdso.c | 123 ++++++++++++++++++++++++++++++++
 4 files changed, 131 insertions(+), 2 deletions(-)
 create mode 100644 tools/testing/selftests/x86/test_vdso.c

Signature · 1 line

2.7.4

Andy Lutomirski

author

2016-11-16 18:23 UTC #1 lore raw

PATCH 1/2 x86/vdso: Use RDPID in preference to LSL when available

RDPID is a new instruction that reads MSR_TSC_AUX quickly. This should be considerably faster than reading the GDT. Add a cpufeature for it and use it from __vdso_getcpu when available.

Tested-by Megha Dey <[email protected]>

Signed-off-by Andy Lutomirski <[email protected]>

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 arch/x86/include/asm/vgtod.h       | 7 ++++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

arch/x86/include/asm/cpufeatures.h +1 -0

index a39629206864..3068edf43abe 100644
@@ -281,6 +281,7 @@
 /* Intel-defined CPU features, CPUID level 0x00000007:0 (ecx), word 16 */
 #define X86_FEATURE_PKU		(16*32+ 3) /* Protection Keys for Userspace */
 #define X86_FEATURE_OSPKE	(16*32+ 4) /* OS Protection Keys Enable */
+#define X86_FEATURE_RDPID	(16*32+ 22) /* RDPID instruction */
 
 /* AMD-defined CPU features, CPUID level 0x80000007 (ebx), word 17 */
 #define X86_FEATURE_OVERFLOW_RECOV (17*32+0) /* MCA overflow recovery support */

arch/x86/include/asm/vgtod.h +6 -1

index e728699db774..3a01996db58f 100644
@@ -89,8 +89,13 @@ static inline unsigned int __getcpu(void)
 	 * works on all CPUs.  This is volatile so that it orders
 	 * correctly wrt barrier() and to keep gcc from cleverly
 	 * hoisting it out of the calling function.
+	 *
+	 * If RDPID is available, use it.
 	 */
-	asm volatile ("lsl %1,%0" : "=r" (p) : "r" (__PER_CPU_SEG));
+	alternative_io ("lsl %[p],%[seg]",
+			".byte 0xf3,0x0f,0xc7,0xf8", /* RDPID %eax/rax */
+			X86_FEATURE_RDPID,
+			[p] "=a" (p), [seg] "r" (__PER_CPU_SEG));
 
 	return p;
 }

Signature · 1 line

2.7.4

Andy Lutomirski

author

2016-11-16 18:23 UTC #3 lore raw

PATCH 2/2 selftests/x86: Add test_vdso to test getcpu

In reply to Andy Lutomirski

I'll eventually add tests for more vdso functions here.

Signed-off-by Andy Lutomirski <[email protected]>

---
 tools/testing/selftests/x86/Makefile    |   2 +-
 tools/testing/selftests/x86/test_vdso.c | 123 ++++++++++++++++++++++++++++++++
 2 files changed, 124 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/x86/test_vdso.c

tools/testing/selftests/x86/Makefile +1 -1

index a89f80a5b711..8c1cb423cfe6 100644
@@ -6,7 +6,7 @@ include ../lib.mk
 
 TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt ptrace_syscall test_mremap_vdso \
 			check_initial_reg_state sigreturn ldt_gdt iopl \
-			protection_keys
+			protection_keys test_vdso
 TARGETS_C_32BIT_ONLY := entry_from_vm86 syscall_arg_fault test_syscall_vdso unwind_vdso \
 			test_FCMOV test_FCOMI test_FISTTP \
 			vdso_restorer

tools/testing/selftests/x86/test_vdso.c new file +123 -0

123 lines not on this page · raw message

Signature · 1 line

2.7.4

tip-bot for Andy Lutomirski

merge bot

2016-11-17 07:50 UTC #2 lore raw

[tip:x86/asm] x86/vdso: Use RDPID in preference to LSL when available

In reply to Andy Lutomirski

x86/vdso: Use RDPID in preference to LSL when available

Commit-ID:  a582c540ac1b10f0a7d37415e04c4af42409fd08
Gitweb:     http://git.kernel.org/tip/a582c540ac1b10f0a7d37415e04c4af42409fd08
Author:     Andy Lutomirski <[email protected]>
AuthorDate: Wed, 16 Nov 2016 10:23:27 -0800
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 17 Nov 2016 08:31:14 +0100

x86/vdso: Use RDPID in preference to LSL when available

RDPID is a new instruction that reads MSR_TSC_AUX quickly. This should be considerably faster than reading the GDT. Add a cpufeature for it and use it from __vdso_getcpu() when available.

Tested-by Megha Dey <[email protected]>

Signed-off-by Andy Lutomirski <[email protected]>

Cc Borislav Petkov <[email protected]>

Cc Brian Gerst <[email protected]>

Cc Denys Vlasenko <[email protected]>

Cc H. Peter Anvin <[email protected]>

Cc Josh Poimboeuf <[email protected]>

Cc Linus Torvalds <[email protected]>

Cc Peter Zijlstra <[email protected]>

Cc Thomas Gleixner <[email protected]>

Link http://lkml.kernel.org/r/4f6c3a22012d10f1c65b9ca15800e01b42c7d39d.1479320367.git.luto@kernel.org open here

Signed-off-by Ingo Molnar <[email protected]>

---
 arch/x86/include/asm/cpufeatures.h | 1 +
 arch/x86/include/asm/vgtod.h       | 7 ++++++-
 2 files changed, 7 insertions(+), 1 deletion(-)

arch/x86/include/asm/cpufeatures.h +1 -0

index a3e8cd7..eac7572 100644
@@ -283,6 +283,7 @@
 #define X86_FEATURE_AVX512VBMI  (16*32+ 1) /* AVX512 Vector Bit Manipulation instructions*/
 #define X86_FEATURE_PKU		(16*32+ 3) /* Protection Keys for Userspace */
 #define X86_FEATURE_OSPKE	(16*32+ 4) /* OS Protection Keys Enable */
+#define X86_FEATURE_RDPID	(16*32+ 22) /* RDPID instruction */
 
 /* AMD-defined CPU features, CPUID level 0x80000007 (ebx), word 17 */
 #define X86_FEATURE_OVERFLOW_RECOV (17*32+0) /* MCA overflow recovery support */

arch/x86/include/asm/vgtod.h +6 -1

index e728699..3a01996 100644
@@ -89,8 +89,13 @@ static inline unsigned int __getcpu(void)
 	 * works on all CPUs.  This is volatile so that it orders
 	 * correctly wrt barrier() and to keep gcc from cleverly
 	 * hoisting it out of the calling function.
+	 *
+	 * If RDPID is available, use it.
 	 */
-	asm volatile ("lsl %1,%0" : "=r" (p) : "r" (__PER_CPU_SEG));
+	alternative_io ("lsl %[p],%[seg]",
+			".byte 0xf3,0x0f,0xc7,0xf8", /* RDPID %eax/rax */
+			X86_FEATURE_RDPID,
+			[p] "=a" (p), [seg] "r" (__PER_CPU_SEG));
 
 	return p;
 }

tip-bot for Andy Lutomirski

merge bot

2016-11-17 07:51 UTC #4 lore raw

[tip:x86/asm] selftests/x86: Add test_vdso to test getcpu()

In reply to Andy Lutomirski

selftests/x86: Add test_vdso to test getcpu()

Commit-ID:  3200ca806942d3d8f78d0d0c822e345dfb8789e7
Gitweb:     http://git.kernel.org/tip/3200ca806942d3d8f78d0d0c822e345dfb8789e7
Author:     Andy Lutomirski <[email protected]>
AuthorDate: Wed, 16 Nov 2016 10:23:28 -0800
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 17 Nov 2016 08:31:22 +0100

selftests/x86: Add test_vdso to test getcpu()

I'll eventually add tests for more vDSO functions here.

Signed-off-by Andy Lutomirski <[email protected]>

Cc Borislav Petkov <[email protected]>

Cc Brian Gerst <[email protected]>

Cc Denys Vlasenko <[email protected]>

Cc H. Peter Anvin <[email protected]>

Cc Josh Poimboeuf <[email protected]>

Cc Linus Torvalds <[email protected]>

Cc Megha <[email protected]>

Cc Peter Zijlstra <[email protected]>

Cc Thomas Gleixner <[email protected]>

Link http://lkml.kernel.org/r/945cd29901a62a3cc6ea7d6ee5e389ab1ec1ac0c.1479320367.git.luto@kernel.org open here

Signed-off-by Ingo Molnar <[email protected]>

---
 tools/testing/selftests/x86/Makefile    |   2 +-
 tools/testing/selftests/x86/test_vdso.c | 123 ++++++++++++++++++++++++++++++++
 2 files changed, 124 insertions(+), 1 deletion(-)

tools/testing/selftests/x86/Makefile +1 -1

index a89f80a..8c1cb42 100644
@@ -6,7 +6,7 @@ include ../lib.mk
 
 TARGETS_C_BOTHBITS := single_step_syscall sysret_ss_attrs syscall_nt ptrace_syscall test_mremap_vdso \
 			check_initial_reg_state sigreturn ldt_gdt iopl \
-			protection_keys
+			protection_keys test_vdso
 TARGETS_C_32BIT_ONLY := entry_from_vm86 syscall_arg_fault test_syscall_vdso unwind_vdso \
 			test_FCMOV test_FCOMI test_FISTTP \
 			vdso_restorer

tools/testing/selftests/x86/test_vdso.c new file +123 -0

123 lines not on this page · raw message