x86: Enable LKGS instruction
Subsystems (4)
v5 cover x86: Enable LKGS instruction
LKGS instruction is introduced with Intel FRED (flexible return and event delivery) specification https://cdrdv2.intel.com/v1/dl/getContent/678938.
LKGS is independent of FRED, so we enable it as a standalone CPU feature.
LKGS behaves like the MOV to GS instruction except that it loads the base address into the IA32_KERNEL_GS_BASE MSR instead of the GS segment’s descriptor cache, which is exactly what Linux kernel does to load user level GS base. Thus, with LKGS, there is no need to SWAPGS away from the kernel GS base.
Changes since v4:
* Clear the LKGS feature from Xen PV guests (Juergen Gross).
Changes since v3:
* We want less ASM not more, thus keep local_irq_save/restore() inside native_load_gs_index() (Thomas Gleixner).
* For paravirt enabled kernels, initialize pv_ops.cpu.load_gs_index to native_lkgs (Thomas Gleixner).
Changes since V2:
* Add "" not to show "lkgs" in /proc/cpuinfo (Chang S. Bae).
* Mark DI as input and output (+D) as in V1, since the exception handler modifies it (Brian Gerst).
Changes since V1:
* Use EX_TYPE_ZERO_REG instead of fixup code in the obsolete .fixup code section (Peter Zijlstra).
* Add a comment that states the LKGS_DI macro will be repalced with "lkgs %di" once the binutils support the LKGS instruction (Peter Zijlstra).
H. Peter Anvin (Intel) (5):
x86/cpufeature: add the cpu feature bit for LKGS x86/opcode: add the LKGS instruction to x86-opcode-map x86/gsseg: make asm_load_gs_index() take an u16 x86/gsseg: move load_gs_index() to its own new header file x86/gsseg: use the LKGS instruction if available for load_gs_index()
arch/x86/entry/entry_64.S | 2 +- arch/x86/ia32/ia32_signal.c | 1 + arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/gsseg.h | 66 ++++++++++++++++++++++++ arch/x86/include/asm/mmu_context.h | 1 + arch/x86/include/asm/special_insns.h | 21 -------- arch/x86/kernel/cpu/common.c | 1 + arch/x86/kernel/paravirt.c | 1 + arch/x86/kernel/tls.c | 1 + arch/x86/lib/x86-opcode-map.txt | 1 + arch/x86/xen/enlighten_pv.c | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + tools/arch/x86/lib/x86-opcode-map.txt | 1 + 13 files changed, 77 insertions(+), 22 deletions(-) create mode 100644 arch/x86/include/asm/gsseg.h
Signature · 1 line
2.34.1
PATCH v5 1/5 x86/cpufeature: add the cpu feature bit for LKGS
From: "H. Peter Anvin (Intel)" <[email protected]>
Add the CPU feature bit for LKGS (Load "Kernel" GS).
LKGS instruction is introduced with Intel FRED (flexible return and event delivery) specificaton
https://cdrdv2.intel.com/v1/dl/getContent/678938.
LKGS behaves like the MOV to GS instruction except that it loads the base address into the IA32_KERNEL_GS_BASE MSR instead of the GS segment’s descriptor cache, which is exactly what Linux kernel does to load a user level GS base. Thus, with LKGS, there is no need to SWAPGS away from the kernel GS base.
---
Change since V2:
* add "" not to show "lkgs" in /proc/cpuinfo (Chang S. Bae).
--- arch/x86/include/asm/cpufeatures.h | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 2 files changed, 2 insertions(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -311,6 +311,7 @@/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4)/* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ +#define X86_FEATURE_LKGS (12*32+ 18)/* "" Load "kernel" (userspace) gs */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
tools/arch/x86/include/asm/cpufeatures.h +1 -0
@@ -308,6 +308,7 @@/* Intel-defined CPU features, CPUID level 0x00000007:1 (EAX), word 12 */ #define X86_FEATURE_AVX_VNNI (12*32+ 4)/* AVX VNNI instructions */ #define X86_FEATURE_AVX512_BF16 (12*32+ 5)/* AVX512 BFLOAT16 instructions */ +#define X86_FEATURE_LKGS (12*32+ 18)/* "" Load "kernel" (userspace) gs */ /* AMD-defined CPU features, CPUID level 0x80000008 (EBX), word 13 */ #define X86_FEATURE_CLZERO (13*32+ 0)/* CLZERO instruction */
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v5 1/5] x86/cpufeature: add the cpu feature bit for LKGS
Quoted 6 lines · Xin Li
From: "H. Peter Anvin (Intel)" <[email protected]>
Add the CPU feature bit for LKGS (Load "Kernel" GS).
LKGS instruction is introduced with Intel FRED (flexible return and event delivery) specificaton
Unknown word [specificaton] in commit message.
Suggestions: ['specification',
Please use a spellchecker for your commit messages.
Quoted 1 line
This URL is most likely going to be unstable.
Write instead
"Search for the latest spec in most search engines by doing:
site:intel.com FRED (flexible return and event delivery) specification"
Google etc has indexed it already so it should always pull out the newest revision.
$ test-apply.sh /tmp/01-x86-cpufeature-add_the_cpu_feature_bit_for_lkgs-new.patch checking file arch/x86/include/asm/cpufeatures.h Hunk #1 FAILED at 311. 1 out of 1 hunk FAILED checking file tools/arch/x86/include/asm/cpufeatures.h Hunk #1 FAILED at 308. 1 out of 1 hunk FAILED Apply? (y/n)
I guess you'd need to refresh the patchset against latest tip/master.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRE: [PATCH v5 1/5] x86/cpufeature: add the cpu feature bit for LKGS
Quoted 12 lines
Quoted 4 lines
Add the CPU feature bit for LKGS (Load "Kernel" GS).
LKGS instruction is introduced with Intel FRED (flexible return and event delivery) specificaton
Unknown word [specificaton] in commit message.
Suggestions: ['specification',Please use a spellchecker for your commit messages.
Quoted 1 line
Quoted 7 lines
This URL is most likely going to be unstable.
Write instead
"Search for the latest spec in most search engines by doing:
site:intel.com FRED (flexible return and event delivery) specification"
Thanks for the suggestion.
Quoted 12 lines
Google etc has indexed it already so it should always pull out the newest revision.
$ test-apply.sh /tmp/01-x86-cpufeature-add_the_cpu_feature_bit_for_lkgs- new.patch checking file arch/x86/include/asm/cpufeatures.h Hunk #1 FAILED at 311. 1 out of 1 hunk FAILED checking file tools/arch/x86/include/asm/cpufeatures.h Hunk #1 FAILED at 308. 1 out of 1 hunk FAILED Apply? (y/n)
Quoted 1 line
I guess you'd need to refresh the patchset against latest tip/master.
Will rebase on top of latest master and send v6 of the patch set.
Xin
Quoted 7 lines
Thx.
--
Regards/Gruss,
Boris.
PATCH v5 2/5 x86/opcode: add the LKGS instruction to x86-opcode-map
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Add the instruction opcode used by LKGS.
Opcode number is per public FRED draft spec v3.0
https://cdrdv2.intel.com/v1/dl/getContent/678938.
--- arch/x86/lib/x86-opcode-map.txt | 1 + tools/arch/x86/lib/x86-opcode-map.txt | 1 + 2 files changed, 2 insertions(+)
arch/x86/lib/x86-opcode-map.txt +1 -0
@@ -1047,6 +1047,7 @@ GrpTable: Grp6 3: LTR Ew 4: VERR Ew 5: VERW Ew +6: LKGS Ew (F2) EndTable GrpTable: Grp7
tools/arch/x86/lib/x86-opcode-map.txt +1 -0
@@ -1047,6 +1047,7 @@ GrpTable: Grp6 3: LTR Ew 4: VERR Ew 5: VERW Ew +6: LKGS Ew (F2) EndTable GrpTable: Grp7
Signature · 1 line
2.34.1
PATCH v5 3/5 x86/gsseg: make asm_load_gs_index() take an u16
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Let gcc know that only the low 16 bits of load_gs_index() argument actually matter. It might allow it to create slightly better code. However, do not propagate this into the prototypes of functions that end up being paravirtualized, to avoid unnecessary changes.
--- arch/x86/entry/entry_64.S | 2 +- arch/x86/include/asm/special_insns.h | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/entry/entry_64.S +1 -1
@@ -779,7 +779,7 @@ _ASM_NOKPROBE(common_interrupt_return) /* * Reload gs selector with exception handling - * edi: new selector + * di: new selector * * Is in entry.text as it shouldn't be instrumented. */
arch/x86/include/asm/special_insns.h +1 -1
@@ -120,7 +120,7 @@ static inline void native_wbinvd(void) asmvolatile ("wbinvd" : : :"memory" ); } -extern asmlinkage void asm_load_gs_index(unsigned int selector); +extern asmlinkage void asm_load_gs_index(u16 selector);static inline void native_load_gs_index (unsigned int selector) {
Signature · 1 line
2.34.1
PATCH v5 4/5 x86/gsseg: move load_gs_index() to its own new header file
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
GS is a special segment on x86_64, move load_gs_index() to its own new header file.
--- arch/x86/ia32/ia32_signal.c | 1 + arch/x86/include/asm/gsseg.h | 41 ++++++++++++++++++++++++++++ arch/x86/include/asm/mmu_context.h | 1 + arch/x86/include/asm/special_insns.h | 21 -------------- arch/x86/kernel/paravirt.c | 1 + arch/x86/kernel/tls.c | 1 + 6 files changed, 45 insertions(+), 21 deletions(-) create mode 100644 arch/x86/include/asm/gsseg.h
arch/x86/ia32/ia32_signal.c +1 -0
@@ -34,6 +34,7 @@#include <asm/sigframe.h> #include <asm/sighandling.h> #include <asm/smap.h> +#include <asm/gsseg.h> static inline void reload_segments (struct sigcontext_32 *sc) {
arch/x86/include/asm/gsseg.h new file +41 -0
41 lines not on this page · raw message
arch/x86/include/asm/mmu_context.h +1 -0
@@ -12,6 +12,7 @@#include <asm/tlbflush.h> #include <asm/paravirt.h> #include <asm/debugreg.h> +#include <asm/gsseg.h> extern atomic64_t last_mm_ctx_id;
arch/x86/include/asm/special_insns.h +0 -21
@@ -120,17 +120,6 @@ static inline void native_wbinvd(void) asmvolatile ("wbinvd" : : :"memory" ); } -extern asmlinkage void asm_load_gs_index(u16 selector); - -static inline void native_load_gs_index (unsigned int selector) -{ -unsigned long flags; - -local_irq_save (flags); -asm_load_gs_index (selector); -local_irq_restore (flags); -} -static inline unsigned long __read_cr4 (void ) {return native_read_cr4 ();
@@ -184,16 +173,6 @@ static inline void wbinvd(void) native_wbinvd(); } - -static inline void load_gs_index (unsigned int selector) -{ -#ifdef CONFIG_X86_64 -native_load_gs_index (selector); -#else -loadsegment (gs, selector); -#endif -} -#endif /* CONFIG_PARAVIRT_XXL */ static inline void clflush (volatile void *__p)
arch/x86/kernel/paravirt.c +1 -0
@@ -32,6 +32,7 @@#include <asm/special_insns.h> #include <asm/tlb.h> #include <asm/io_bitmap.h> +#include <asm/gsseg.h> /* * nop stub, which must not clobber anything *including the stack* to
arch/x86/kernel/tls.c +1 -0
@@ -12,6 +12,7 @@#include <asm/ldt.h> #include <asm/processor.h> #include <asm/proto.h> +#include <asm/gsseg.h> #include "tls.h"
Signature · 1 line
2.34.1
PATCH v5 5/5 x86/gsseg: use the LKGS instruction if available for load_gs_index()
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
The LKGS instruction atomically loads a segment descriptor into the %gs descriptor registers, *except* that %gs.base is unchanged, and the base is instead loaded into MSR_IA32_KERNEL_GS_BASE, which is exactly what we want this function to do.
---
Changes since v4:
* Clear the LKGS feature from Xen PV guests (Juergen Gross).
Changes since v3:
* We want less ASM not more, thus keep local_irq_save/restore() inside native_load_gs_index() (Thomas Gleixner).
* For paravirt enabled kernels, initialize pv_ops.cpu.load_gs_index to native_lkgs (Thomas Gleixner).
Changes since v2:
* Mark DI as input and output (+D) as in V1, since the exception handler modifies it (Brian Gerst).
Changes since v1:
* Use EX_TYPE_ZERO_REG instead of fixup code in the obsolete .fixup code section (Peter Zijlstra).
* Add a comment that states the LKGS_DI macro will be repalced with "lkgs %di" once the binutils support the LKGS instruction (Peter Zijlstra).
--- arch/x86/include/asm/gsseg.h | 33 +++++++++++++++++++++++++++++---- arch/x86/kernel/cpu/common.c | 1 + arch/x86/xen/enlighten_pv.c | 1 + 3 files changed, 31 insertions(+), 4 deletions(-)
arch/x86/include/asm/gsseg.h +29 -4
@@ -14,17 +14,42 @@ extern asmlinkage void asm_load_gs_index(u16 selector); +/* Replace with "lkgs %di" once binutils support LKGS instruction */ +#define LKGS_DI _ASM_BYTES(0xf2,0x0f,0x00,0xf7) + +static inline void native_lkgs(unsigned int selector) +{ + u16 sel = selector; + asm_inline volatile("1: " LKGS_DI + _ASM_EXTABLE_TYPE_REG(1b, 1b, EX_TYPE_ZERO_REG, %k[sel]) + : [sel] "+D" (sel)); +} + static inline void native_load_gs_index(unsigned int selector) { -unsigned long flags; + if (cpu_feature_enabled(X86_FEATURE_LKGS)) { + native_lkgs(selector); + } else { + unsigned long flags; -local_irq_save (flags); -asm_load_gs_index (selector); -local_irq_restore (flags); + local_irq_save(flags); + asm_load_gs_index(selector); + local_irq_restore(flags); + } } #endif /* CONFIG_X86_64 */ +static inline void __init lkgs_init(void) +{ +#ifdef CONFIG_PARAVIRT_XXL +#ifdef CONFIG_X86_64 + if (cpu_feature_enabled(X86_FEATURE_LKGS)) + pv_ops.cpu.load_gs_index = native_lkgs; +#endif +#endif +} + #ifndef CONFIG_PARAVIRT_XXL static inline void load_gs_index(unsigned int selector)
arch/x86/kernel/cpu/common.c +1 -0
@@ -1939,6 +1939,7 @@ void __init identify_boot_cpu(void)setup_cr_pinning ();tsx_init (); +lkgs_init (); }void identify_secondary_cpu (struct cpuinfo_x86 *c)
arch/x86/xen/enlighten_pv.c +1 -0
@@ -276,6 +276,7 @@ static void __init xen_init_capabilities(void)setup_clear_cpu_cap (X86_FEATURE_ACC );setup_clear_cpu_cap (X86_FEATURE_X2APIC );setup_clear_cpu_cap (X86_FEATURE_SME ); +setup_clear_cpu_cap (X86_FEATURE_LKGS );/* * Xen PV would need some work to support PCID: CR3 handling as well
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v5 5/5] x86/gsseg: use the LKGS instruction if available for load_gs_index()
Quoted 7 lines · Xin Li
From: "H. Peter Anvin (Intel)" <[email protected]>
The LKGS instruction atomically loads a segment descriptor into the %gs descriptor registers, *except* that %gs.base is unchanged, and the base is instead loaded into
MSR_IA32_KERNEL_GS_BASE, which is exactly what we want this function to do.
Quoted 5 lines
Signed-off-by H. Peter Anvin (Intel) <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Brian Gerst <[email protected]>Signed-off-by Juergen Gross <[email protected]>Signed-off-by Xin Li <[email protected]>
I'm reading this SOB chain as
hpa wrote it -> then it went to Peter -> then to Brian -> then to Juergen -> and you're sending it.
I'm pretty sure that cannot be right.
Quoted 15 lines
---
Changes since v4:
* Clear the LKGS feature from Xen PV guests (Juergen Gross).Changes since v3:
* We want less ASM not more, thus keep local_irq_save/restore() insidenative_load_gs_index()(Thomas Gleixner).
* For paravirt enabled kernels, initialize pv_ops.cpu.load_gs_index tonative_lkgs(Thomas Gleixner).Changes since v2:
* Mark DI as input and output (+D) as in V1, since the exception handler modifies it (Brian Gerst).
Quoted 5 lines
Changes since v1:
* UseEX_TYPE_ZERO_REGinstead of fixup code in the obsolete .fixup code section (Peter Zijlstra).
* Add a comment that states theLKGS_DImacro will be repalced with "lkgs %di" once the binutils support the LKGS instruction (Peter Zijlstra).
I guess that explains what the SOB chain is supposed to mean - you've gotten review feedback. But that doesn't need such a SOB chain. Sounds like you need to refresh on
Documentation/process/submitting-patches.rst
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRE: [PATCH v5 5/5] x86/gsseg: use the LKGS instruction if available for load_gs_index()
Quoted 38 lines
Quoted 11 lines
The LKGS instruction atomically loads a segment descriptor into the %gs descriptor registers, *except* that %gs.base is unchanged, and the base is instead loaded into
MSR_IA32_KERNEL_GS_BASE, which is exactly what we want this function to do.Signed-off-by H. Peter Anvin (Intel) <[email protected]>Signed-off-by Peter Zijlstra (Intel) <[email protected]>Signed-off-by Brian Gerst <[email protected]>Signed-off-by Juergen Gross <[email protected]>Signed-off-by Xin Li <[email protected]>I'm reading this SOB chain as
hpa wrote it -> then it went to Peter -> then to Brian -> then to Juergen -> and you're sending it.
I'm pretty sure that cannot be right.
Quoted 11 lines
---
Changes since v4:
* Clear the LKGS feature from Xen PV guests (Juergen Gross).Changes since v3:
* We want less ASM not more, thus keep local_irq_save/restore() insidenative_load_gs_index()(Thomas Gleixner).
* For paravirt enabled kernels, initialize pv_ops.cpu.load_gs_index tonative_lkgs(Thomas Gleixner).Quoted 8 lines
Changes since v2:
* Mark DI as input and output (+D) as in V1, since the exception handler modifies it (Brian Gerst).Changes since v1:
* UseEX_TYPE_ZERO_REGinstead of fixup code in the obsolete .fixup code section (Peter Zijlstra).
* Add a comment that states theLKGS_DImacro will be repalced with "lkgs
Quoted 6 lines
%di"
Quoted 1 line
once the binutils support the LKGS instruction (Peter Zijlstra).
I guess that explains what the SOB chain is supposed to mean - you've gotten review feedback. But that doesn't need such a SOB chain. Sounds like you need to refresh on
Your guess is correct, will remove those SOBs in v6.
Quoted 9 lines
Documentation/process/submitting-patches.rstThx.
--
Regards/Gruss,
Boris.