x86: enable FRED for x86-64
Subsystems (15)
v12 cover x86: enable FRED for x86-64
This patch set enables the Intel flexible return and event delivery (FRED) architecture for x86-64.
The FRED architecture defines simple new transitions that change privilege level (ring transitions). The FRED architecture was designed with the following goals:
1) Improve overall performance and response time by replacing event delivery through the interrupt descriptor table (IDT event delivery) and event return by the IRET instruction with lower latency transitions.
2) Improve software robustness by ensuring that event delivery establishes the full supervisor context and that event return establishes the full user context.
The new transitions defined by the FRED architecture are FRED event delivery and, for returning from events, two FRED return instructions. FRED event delivery can effect a transition from ring 3 to ring 0, but it is used also to deliver events incident to ring 0. One FRED instruction (ERETU) effects a return from ring 0 to ring 3, while the other (ERETS) returns while remaining in ring 0. Collectively, FRED event delivery and the FRED return instructions are FRED transitions.
Search for the latest FRED spec in most search engines with this search pattern:
site:intel.com FRED (flexible return and event delivery) specification
As of now there is no publicly avaiable CPU supporting FRED, thus the Intel Simics® Simulator is used as software development and testing vehicles. And it can be downloaded from:
https://www.intel.com/content/www/us/en/developer/articles/tool/simics-simulator.html
To enable FRED, the Simics package 8112 QSP-CPU needs to be installed with CPU model configured as:
$cpu_comp_class = "x86-experimental-fred"
Changes since v11:
* Add a new structure fred_cs to denote the FRED flags above CS selector as what is done for SS (H. Peter Anvin).
Changes since v10:
* No need to invalidate SYSCALL and SYSENTER MSRs (Thomas Gleixner).
* Better explain the reason why no need to check current stack level (Paolo Bonzini).
* Replace "IS_ENABLED(CONFIG_IA32_EMULATION)" with the new ia32_enabled() API (Nikolay Borisov).
* FRED feature is defined in cpuid word 12, not 13 (Nikolay Borisov).
* Reword a sentence in the new FRED documentation to improve readability (Nikolay Borisov).
* A few comment fixes and improvements to event type definitions (Andrew Cooper).
Changes since v9:
* Set unused sysvec table entries to fred_handle_spurious_interrupt() in fred_complete_exception_setup() (Thomas Gleixner).
* Shove the whole thing into arch/x86/entry/entry_64_fred.S for invoking external_interrupt() and fred_exc_nmi() (Sean Christopherson).
* Correct and improve a few comments (Sean Christopherson).
* Merge the two IRQ/NMI asm entries into one as it's fine to invoke noinstr code from regular code (Thomas Gleixner).
* Setup the long mode and NMI flags in the augmented SS field of FRED stack frame in C instead of asm (Thomas Gleixner).
* Don't use jump tables, indirect jumps are expensive (Thomas Gleixner).
* Except #NMI/#DB/#MCE, FRED really can share the exception handlers with IDT (Thomas Gleixner).
* Avoid the sysvec_* idt_entry muck, do it at a central place, reuse code instead of blindly copying it, which breaks the performance optimized sysvec entries like reschedule_ipi (Thomas Gleixner).
* Add asm_ prefix to FRED asm entry points (Thomas Gleixner).
* Disable #DB to avoid endless recursion and stack overflow when a watchpoint/breakpoint is set in the code path which is executed by #DB handler (Thomas Gleixner).
* Introduce a new structure fred_ss to denote the FRED flags above SS selector, which avoids FRED_SSX_ macros and makes the code simpler and easier to read (Thomas Gleixner).
* Use type u64 to define FRED bit fields instead of type unsigned int (Thomas Gleixner).
* Avoid a type cast by defining X86_CR4_FRED as 0 on 32-bit (Thomas Gleixner).
* Add the WRMSRNS instruction support (Thomas Gleixner).
Changes since v8:
* Move the FRED initialization patch after all required changes are in place (Thomas Gleixner).
* Don't do syscall early out in fred_entry_from_user() before there are proper performance numbers and justifications (Thomas Gleixner).
* Add the control exception handler to the FRED exception handler table (Thomas Gleixner).
* Introduce a macro sysvec_install() to derive the asm handler name from a C handler, which simplifies the code and avoids an ugly typecast (Thomas Gleixner).
* Remove junk code that assumes no local APIC on x86_64 (Thomas Gleixner).
* Put IDTENTRY changes in a separate patch (Thomas Gleixner).
* Use high-order 48 bits above the lowest 16 bit SS only when FRED is enabled (Thomas Gleixner).
* Explain why writing directly to the IA32_KERNEL_GS_BASE MSR is doing the right thing (Thomas Gleixner).
* Reword some patch descriptions (Thomas Gleixner).
* Add a new macro VMX_DO_FRED_EVENT_IRQOFF for FRED instead of refactoring VMX_DO_EVENT_IRQOFF (Sean Christopherson).
* Do NOT use a trampoline, just LEA+PUSH the return RIP, PUSH the error code, and jump to the FRED kernel entry point for NMI or call external_interrupt() for IRQs (Sean Christopherson).
* Call external_interrupt() only when FRED is enabled, and convert the non-FRED handling to external_interrupt() after FRED lands (Sean Christopherson).
* Use __packed instead of __attribute__((__packed__)) (Borislav Petkov).
* Put all comments above the members, like the rest of the file does (Borislav Petkov).
* Reflect the FRED spec 5.0 change that ERETS and ERETU add 8 to %rsp before popping the return context from the stack.
* Reflect stack frame definition changes from FRED spec 3.0 to 5.0.
* Add ENDBR to the FRED_ENTER asm macro after kernel IBT is added to FRED base line in FRED spec 5.0.
* Add a document which briefly introduces FRED features.
* Remove 2 patches, "allow FRED systems to use interrupt vectors 0x10-0x1f" and "allow dynamic stack frame size", from this patch set, as they are "optimizations" only.
* Send 2 patches, "header file for event types" and "do not modify the DPL bits for a null selector", as pre-FRED patches.
Changes since v7:
* Always call external_interrupt() for VMX IRQ handling on x86_64, thus avoid re-entering the noinstr code.
* Create a FRED stack frame when FRED is compiled-in but not enabled, which uses some extra stack space but simplifies the code.
* Add a log message when FRED is enabled.
Changes since v6:
* Add a comment to explain why it is safe to write to a previous FRED stack frame. (Lai Jiangshan).
* Export fred_entrypoint_kernel(), required when kvm-intel built as a module.
* Reserve a REDZONE for CALL emulation and Align RSP to a 64-byte boundary before pushing a new FRED stack frame.
* Replace pt_regs csx flags prefix FRED_CSL_ with FRED_CSX_.
Changes since v5:
* Initialize system_interrupt_handlers with dispatch_table_spurious_interrupt() instead of NULL to get rid of a branch (Peter Zijlstra).
* Disallow #DB inside #MCE for robustness sake (Peter Zijlstra).
* Add a comment for FRED stack level settings (Lai Jiangshan).
* Move the NMI bit from an invalid stack frame, which caused ERETU to fault, to the fault handler's stack frame, thus to unblock NMI ASAP if NMI is blocked (Lai Jiangshan).
* Refactor VMX_DO_EVENT_IRQOFF to handle IRQ/NMI in IRQ/NMI induced VM exits when FRED is enabled (Sean Christopherson).
Changes since v4:
* Do NOT use the term "injection", which in the KVM context means to reinject an event into the guest (Sean Christopherson).
* Add the explanation of why to execute "int $2" to invoke the NMI handler in NMI caused VM exits (Sean Christopherson).
* Use cs/ss instead of csx/ssx when initializing the pt_regs structure for calling external_interrupt(), otherwise it breaks i386 build.
Changes since v3:
* Call external_interrupt() to handle IRQ in IRQ caused VM exits.
* Execute "int $2" to handle NMI in NMI caused VM exits.
* Rename csl/ssl of the pt_regs structure to csx/ssx (x for extended) (Andrew Cooper).
Changes since v2:
* Improve comments for changes in arch/x86/include/asm/idtentry.h.
Changes since v1:
* call irqentry_nmi_{enter,exit}() in both IDT and FRED debug fault kernel handler (Peter Zijlstra).
* Initialize a FRED exception handler to fred_bad_event() instead of NULL if no FRED handler defined for an exception vector (Peter Zijlstra).
* Push calling irqentry_{enter,exit}() and instrumentation_{begin,end}() down into individual FRED exception handlers, instead of in the dispatch framework (Peter Zijlstra).
H. Peter Anvin (Intel) (20):
x86/fred: Add Kconfig option for FRED (CONFIG_X86_FRED) x86/cpufeatures: Add the cpu feature bit for FRED x86/fred: Disable FRED support if CONFIG_X86_FRED is disabled x86/opcode: Add ERET[US] instructions to the x86 opcode map x86/objtool: Teach objtool about ERET[US]
x86/cpu: Add X86_CR4_FRED macro
x86/cpu: Add MSR numbers for FRED configuration
x86/fred: Add a new header file for FRED definitions x86/fred: Reserve space for the FRED stack frame
x86/fred: Update MSR_IA32_FRED_RSP0 during task switch x86/fred: Disallow the swapgs instruction when FRED is enabled x86/fred: No ESPFIX needed when FRED is enabled
x86/fred: Allow single-step trap and NMI when starting a new task x86/fred: Make exc_page_fault() work for FRED
x86/fred: Add a debug fault entry stub for FRED
x86/fred: Add a NMI entry stub for FRED
x86/fred: FRED entry/exit and dispatch code
x86/fred: Let ret_from_fork_asm() jmp to asm_fred_exit_user when FRED
is enabled
x86/fred: Add FRED initialization functions
x86/fred: Invoke FRED initialization code to enable FRED
Peter Zijlstra (Intel) (1):
x86/entry/calling: Allow PUSH_AND_CLEAR_REGS being used beyond actual
entry code
Xin Li (16):
x86/cpufeatures: Add the cpu feature bit for WRMSRNS x86/opcode: Add the WRMSRNS instruction to the x86 opcode map x86/msr: Add the WRMSRNS instruction support
x86/entry: Remove idtentry_sysvec from entry_{32,64}.S x86/trapnr: Add event type macros to <asm/trapnr.h> Documentation/x86/64: Add a documentation for FRED x86/fred: Disable FRED by default in its early stage x86/ptrace: Cleanup the definition of the pt_regs structure x86/ptrace: Add FRED additional information to the pt_regs structure x86/idtentry: Incorporate definitions/declarations of the FRED entries x86/fred: Add a machine check entry stub for FRED x86/traps: Add sysvec_install() to install a system interrupt handler x86/fred: Fixup fault on ERETU by jumping to fred_entrypoint_user x86/entry: Add fred_entry_from_kvm() for VMX to handle IRQ/NMI
KVM: VMX: Call fred_entry_from_kvm() for IRQ/NMI handling x86/syscall: Split IDT syscall setup code into idt_syscall_init()
.../admin-guide/kernel-parameters.txt | 3 + Documentation/arch/x86/x86_64/fred.rst | 96 ++++++ Documentation/arch/x86/x86_64/index.rst | 1 + arch/x86/Kconfig | 9 + arch/x86/entry/Makefile | 5 +- arch/x86/entry/calling.h | 15 +- arch/x86/entry/entry_32.S | 4 - arch/x86/entry/entry_64.S | 14 +- arch/x86/entry/entry_64_fred.S | 132 +++++++++ arch/x86/entry/entry_fred.c | 279 ++++++++++++++++++ arch/x86/entry/vsyscall/vsyscall_64.c | 2 +- arch/x86/include/asm/asm-prototypes.h | 1 + arch/x86/include/asm/cpufeatures.h | 2 + arch/x86/include/asm/desc.h | 2 - arch/x86/include/asm/disabled-features.h | 8 +- arch/x86/include/asm/extable_fixup_types.h | 4 +- arch/x86/include/asm/fred.h | 97 ++++++ arch/x86/include/asm/idtentry.h | 88 +++++- arch/x86/include/asm/msr-index.h | 13 +- arch/x86/include/asm/msr.h | 18 ++ arch/x86/include/asm/ptrace.h | 104 ++++++- arch/x86/include/asm/switch_to.h | 8 +- arch/x86/include/asm/thread_info.h | 12 +- arch/x86/include/asm/trapnr.h | 12 + arch/x86/include/asm/vmx.h | 17 +- arch/x86/include/uapi/asm/processor-flags.h | 7 + arch/x86/kernel/Makefile | 1 + arch/x86/kernel/cpu/acrn.c | 4 +- arch/x86/kernel/cpu/common.c | 41 ++- arch/x86/kernel/cpu/cpuid-deps.c | 2 + arch/x86/kernel/cpu/mce/core.c | 26 ++ arch/x86/kernel/cpu/mshyperv.c | 15 +- arch/x86/kernel/espfix_64.c | 8 + arch/x86/kernel/fred.c | 59 ++++ arch/x86/kernel/idt.c | 4 +- arch/x86/kernel/irqinit.c | 7 +- arch/x86/kernel/kvm.c | 2 +- arch/x86/kernel/nmi.c | 28 ++ arch/x86/kernel/process_64.c | 67 ++++- arch/x86/kernel/traps.c | 48 ++- arch/x86/kvm/vmx/vmx.c | 12 +- arch/x86/lib/x86-opcode-map.txt | 4 +- arch/x86/mm/extable.c | 79 +++++ arch/x86/mm/fault.c | 5 +- drivers/xen/events/events_base.c | 2 +- tools/arch/x86/include/asm/cpufeatures.h | 2 + .../arch/x86/include/asm/disabled-features.h | 8 +- tools/arch/x86/include/asm/msr-index.h | 13 +- tools/arch/x86/lib/x86-opcode-map.txt | 4 +- tools/objtool/arch/x86/decode.c | 19 +- 50 files changed, 1299 insertions(+), 114 deletions(-) create mode 100644 Documentation/arch/x86/x86_64/fred.rst create mode 100644 arch/x86/entry/entry_64_fred.S create mode 100644 arch/x86/entry/entry_fred.c create mode 100644 arch/x86/include/asm/fred.h create mode 100644 arch/x86/kernel/fred.c
Signature · 1 line
2.34.1
PATCH v12 1/37 x86/cpufeatures: Add the cpu feature bit for WRMSRNS
WRMSRNS is an instruction that behaves exactly like WRMSR, with the only difference being that it is not a serializing instruction by default. Under certain conditions, WRMSRNS may replace WRMSR to improve performance.
Add the CPU feature bit for WRMSRNS.
--- arch/x86/include/asm/cpufeatures.h | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 2 files changed, 2 insertions(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -322,6 +322,7 @@#define X86_FEATURE_FSRS (12*32+11)/* "" Fast short REP STOSB */ #define X86_FEATURE_FSRC (12*32+12)/* "" Fast short REP {CMPSB,SCASB} */ #define X86_FEATURE_LKGS (12*32+18)/* "" Load "kernel" (userspace) GS */ +#define X86_FEATURE_WRMSRNS (12*32+19)/* "" Non-Serializing Write to Model Specific Register instruction */ #define X86_FEATURE_AMX_FP16 (12*32+21)/* "" AMX fp16 Support */ #define X86_FEATURE_AVX_IFMA (12*32+23)/* "" Support for VPMADD52[H,L]UQ */ #define X86_FEATURE_LAM (12*32+26)/* Linear Address Masking */
tools/arch/x86/include/asm/cpufeatures.h +1 -0
@@ -318,6 +318,7 @@#define X86_FEATURE_FSRS (12*32+11)/* "" Fast short REP STOSB */ #define X86_FEATURE_FSRC (12*32+12)/* "" Fast short REP {CMPSB,SCASB} */ #define X86_FEATURE_LKGS (12*32+18)/* "" Load "kernel" (userspace) GS */ +#define X86_FEATURE_WRMSRNS (12*32+19)/* "" Non-Serializing Write to Model Specific Register instruction */ #define X86_FEATURE_AMX_FP16 (12*32+21)/* "" AMX fp16 Support */ #define X86_FEATURE_AVX_IFMA (12*32+23)/* "" Support for VPMADD52[H,L]UQ */ #define X86_FEATURE_LAM (12*32+26)/* Linear Address Masking */
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v12 01/37] x86/cpufeatures: Add the cpu feature bit for WRMSRNS
Quoted 1 line · Xin Li
Subject: Re: [PATCH v12 01/37] x86/cpufeatures: Add the cpu feature bit for WRMSRNS
^^^^
For all your text:
s/cpu/CPU/g
Quoted 5 lines
WRMSRNS is an instruction that behaves exactly like WRMSR, with the only difference being that it is not a serializing instruction by default. Under certain conditions, WRMSRNS may replace WRMSR to improve performance.
Quoted 8 lines
Add the CPU feature bit for WRMSRNS.
Tested-by Shan Kang <[email protected]>Signed-off-by Xin Li <[email protected]>--- arch/x86/include/asm/cpufeatures.h | 1 + tools/arch/x86/include/asm/cpufeatures.h | 1 + 2 files changed, 2 insertions(+)
It looks to me like you can merge the first three patches into one as all they do is add that insn support.
Then, further down in the patchset, it says:
+ if (cpu_feature_enabled(X86_FEATURE_FRED)) {
+ /* WRMSRNS is a baseline feature for FRED. */
but WRMSRNS is not mentioned in the FRED spec "Document Number:
346446-005US, Revision: 5.0" which, according to
is the latest.
Am I looking at the wrong one?
arch/x86/include/asm/cpufeatures.h line 325
Quoted 9 lines
arch/x86/include/asm/cpufeatures.h +1
-0@@ -322,6 +322,7 @@#define X86_FEATURE_FSRS (12*32+11)/* "" Fast short REP STOSB */ #define X86_FEATURE_FSRC (12*32+12)/* "" Fast short REP {CMPSB,SCASB} */ #define X86_FEATURE_LKGS (12*32+18)/* "" Load "kernel" (userspace) GS */ +#define X86_FEATURE_WRMSRNS (12*32+19)/* "" Non-Serializing Write to Model Specific Register instruction */
/* "" Non-serializing WRMSR */
is more than enough.
And now I'm wondering: when you're adding a separate CPUID bit, then the above should be
+ if (cpu_feature_enabled(X86_FEATURE_WRMSRNS)) {
+ /* WRMSRNS is a baseline feature for FRED. */
I see that you're adding a dependency:
+ { X86_FEATURE_FRED, X86_FEATURE_WRMSRNS },
which then means you don't need the X86_FEATURE_WRMSRNS definition at all and can use X86_FEATURE_FRED only.
So, what's up?
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRE: [PATCH v12 01/37] x86/cpufeatures: Add the cpu feature bit for WRMSRNS
Quoted 8 lines
Then, further down in the patchset, it says:
(quoted diff) +2
-0+ if (cpu_feature_enabled(X86_FEATURE_FRED)) { + /* WRMSRNS is a baseline feature for FRED. */but WRMSRNS is not mentioned in the FRED spec "Document Number:
346446-005US, Revision: 5.0" which, according to
Quoted 6 lines
https://www.intel.com/content/www/us/en/content-details/780121/flexible-
return-and-event-delivery-fred-specification.htmlis the latest.
Am I looking at the wrong one?
No. tglx asked for it:
https://lkml.kernel.org/kvm/87y1h81ht4.ffs@tglx/
Review
Quoted 6 lines
And now I'm wondering: when you're adding a separate CPUID bit, then the above should be
(quoted diff) +2
-0+ if (cpu_feature_enabled(X86_FEATURE_WRMSRNS)) { + /* WRMSRNS is a baseline feature for FRED. */
Because we are doing
wrmsrns(MSR_IA32_FRED_RSP0, ...)
here, and X86_FEATURE_WRMSRNS doesn't guarantee MSR_IA32_FRED_RSP0 exists.
Or I missed something?
Quoted 9 lines
I see that you're adding a dependency:
(quoted diff) +1
-0+ { X86_FEATURE_FRED, X86_FEATURE_WRMSRNS },which then means you don't need the
X86_FEATURE_WRMSRNSdefinition at all and can useX86_FEATURE_FREDonly.So, what's up?
FRED just gets the honor to introduce WRMSRNS and its first usage:
https://lkml.kernel.org/kvm/[email protected]/
Another patch set should replace WRMSR with WRMSRNS, with SERIALIZE added when needed.
Sorry for the late response, it was a long weekend in the US.
Thanks!
Xin
Borislav Petkov
maintainerRe: [PATCH v12 01/37] x86/cpufeatures: Add the cpu feature bit for WRMSRNS
Quoted 2 lines · On Tue, Nov 14, 2023 at 12:43:38AM +0000, Li, Xin3 wrote:
No. tglx asked for it:
https://lkml.kernel.org/kvm/87y1h81ht4.ffs@tglx/
Aha
"According to the CPU folks FRED systems are guaranteed to have WRMSRNS - I asked for that :). It's just not yet documented."
so I'm going to expect that to appear in the next FRED spec revision...
Quoted 5 lines
Because we are doing
wrmsrns(MSR_IA32_FRED_RSP0, ...)here, and
X86_FEATURE_WRMSRNSdoesn't guaranteeMSR_IA32_FRED_RSP0exists.Or I missed something?
Well, according to what I'm hearing and reading so far:
FRED means WRMSRNS
FRED means MSR_IA32_FRED_RSP0
and if you had to be precise, the code should do:
if (cpu_feature_enabled(X86_FEATURE_FRED)) {
if (cpu_feature_enabled(X86_FEATURE_WRMSRNS))
wrmsrns(MSR_IA32_FRED_RSP0, (unsigned long)task_stack_page(task) + THREAD_SIZE);
else
wrmsr(MSR_IA32_FRED_RSP0, (unsigned long)task_stack_page(task) + THREAD_SIZE);
}but apparently FRED implies WRMSRNS - not documented anywhere currently
- so you can save yourself one check.
But your version checks FRED if it can do WRMSRNS while there's a separate WRMSRNS flag and that made me wonder...
Quoted 2 lines
Another patch set should replace WRMSR with WRMSRNS, with SERIALIZE added when needed.
I sense someone wants to optimize MSR writes ... :-)
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRE: [PATCH v12 01/37] x86/cpufeatures: Add the cpu feature bit for WRMSRNS
Quoted 10 lines
and if you had to be precise, the code should do:
if (cpu_feature_enabled(X86_FEATURE_FRED)) { if (cpu_feature_enabled(X86_FEATURE_WRMSRNS)) wrmsrns(MSR_IA32_FRED_RSP0, (unsigned long)task_stack_page(task) + THREAD_SIZE); else wrmsr(MSR_IA32_FRED_RSP0, (unsigned long)task_stack_page(task) + THREAD_SIZE); }
This is exactly what tglx wanted to avoid.
And I love the idea "baseline", especially we have a ton of CPU features.
Quoted 5 lines
Quoted 3 lines
Another patch set should replace WRMSR with WRMSRNS, with SERIALIZE added when needed.
I sense someone wants to optimize MSR writes ... :-)
:-)
PATCH v12 2/37 x86/opcode: Add the WRMSRNS instruction to the x86 opcode map
In reply to Xin LiAdd the opcode used by WRMSRNS, which is the non-serializing version of WRMSR and may replace it to improve performance, to the x86 opcode map.
--- arch/x86/lib/x86-opcode-map.txt | 2 +- tools/arch/x86/lib/x86-opcode-map.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/lib/x86-opcode-map.txt +1 -1
@@ -1051,7 +1051,7 @@ GrpTable: Grp6 EndTable GrpTable: Grp7 -0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) +0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) | WRMSRNS (110),(11B) 1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) 2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) | ENCLU (111),(11B) 3: LIDT Ms
tools/arch/x86/lib/x86-opcode-map.txt +1 -1
@@ -1051,7 +1051,7 @@ GrpTable: Grp6 EndTable GrpTable: Grp7 -0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) +0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) | WRMSRNS (110),(11B) 1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) 2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) | ENCLU (111),(11B) 3: LIDT Ms
Signature · 1 line
2.34.1
RE: [PATCH v12 02/37] x86/opcode: Add the WRMSRNS instruction to the x86 opcode map
Quoted 6 lines
Add the opcode used by WRMSRNS, which is the non-serializing version of WRMSR and may replace it to improve performance, to the x86 opcode map.
Tested-by Shan Kang <[email protected]>Signed-off-by Xin Li <[email protected]>
Hi Masami,
Boris prefers to merge the first 3 patches into one:
https://lore.kernel.org/lkml/20231108123647.GBZUuA31zntox0W0gu@fat_crate.local/#t
So I'm thinking could you please also ack the other 2 patches (1st and 3rd of this patch set)?
Thus I can keep your acked-by 😊
Thanks!
Xin
Quoted 4 lines
--- arch/x86/lib/x86-opcode-map.txt | 2 +- tools/arch/x86/lib/x86-opcode-map.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
PATCH v12 3/37 x86/msr: Add the WRMSRNS instruction support
In reply to Xin LiAdd an always inline API __wrmsrns() to embed the WRMSRNS instruction into the code.
--- arch/x86/include/asm/msr.h | 18 ++++++++++++++++++ 1 file changed, 18 insertions(+)
arch/x86/include/asm/msr.h +18 -0
@@ -97,6 +97,19 @@ static __always_inline void __wrmsr(unsigned int msr, u32 low, u32 high) : : "c" (msr), "a"(low), "d" (high) : "memory"); } +/* +* WRMSRNS behaves exactly like WRMSR with the only difference being +* that it is not a serializing instruction by default. +*/ +static __always_inline void __wrmsrns(u32 msr, u32 low, u32 high) +{ +/* Instruction opcode for WRMSRNS; supported in binutils >= 2.40. */ + asm volatile("1: .byte 0x0f,0x01,0xc6\n" + "2:\n" + _ASM_EXTABLE_TYPE(1b, 2b, EX_TYPE_WRMSR) + : : "c" (msr), "a"(low), "d" (high)); +} +#define native_rdmsr (msr, val1, val2) \ do { \ u64 __val = __rdmsr((msr)); \
@@ -297,6 +310,11 @@ do { \#endif /* !CONFIG_PARAVIRT_XXL */ +static __always_inline void wrmsrns(u32 msr, u64 val) +{ +__wrmsrns (msr, val, val >>32 ); +} +/* * 64-bit version of wrmsr_safe(): */
Signature · 1 line
2.34.1
PATCH v12 4/37 x86/entry: Remove idtentry_sysvec from entry_{32,64}.S
In reply to Xin Liidtentry_sysvec is really just DECLARE_IDTENTRY defined in <asm/idtentry.h>, no need to define it separately.
--- arch/x86/entry/entry_32.S | 4 ---- arch/x86/entry/entry_64.S | 8 -------- arch/x86/include/asm/idtentry.h | 2 +- 3 files changed, 1 insertion(+), 13 deletions(-)
arch/x86/entry/entry_32.S +0 -4
@@ -649,10 +649,6 @@ SYM_CODE_START_LOCAL(asm_\cfunc) SYM_CODE_END(asm_\cfunc) .endm -.macro idtentry_sysvec vector cfunc - idtentry \vector asm_\cfunc \cfunc has_error_code=0 -.endm - /* * Include the defines which emit the idt entries which are shared * shared between 32 and 64 bit and emit the __irqentry_text_* markers
arch/x86/entry/entry_64.S +0 -8
@@ -432,14 +432,6 @@ SYM_CODE_END(\asmsym) idtentry \vector asm_\cfunc \cfunc has_error_code=1 .endm -/* - * System vectors which invoke their handlers directly and are not - * going through the regular common device interrupt handling code. - */ -.macro idtentry_sysvec vector cfunc - idtentry \vector asm_\cfunc \cfunc has_error_code=0 -.endm - /** * idtentry_mce_db - Macro to generate entry stubs for #MC and #DB * @vector: Vector number
arch/x86/include/asm/idtentry.h +1 -1
@@ -447,7 +447,7 @@ __visible noinstr void func(struct pt_regs *regs, \/* System vector entries */ #define DECLARE_IDTENTRY_SYSVEC (vector, func) \ - idtentry_sysvec vector func + DECLARE_IDTENTRY(vector, func)#ifdef CONFIG_X86_64 # define DECLARE_IDTENTRY_MCE (vector, func) \
Signature · 1 line
2.34.1
PATCH v12 5/37 x86/trapnr: Add event type macros to <asm/trapnr.h>
In reply to Xin LiIntel VT-x classifies events into eight different types, which is inherited by FRED for event identification. As such, event type becomes a common x86 concept, and should be defined in a common x86 header.
Add event type macros to <asm/trapnr.h>, and use it in <asm/vmx.h>.
---
Changes since v10:
* A few comment fixes and improvements (Andrew Cooper).
--- arch/x86/include/asm/trapnr.h | 12 ++++++++++++ arch/x86/include/asm/vmx.h | 17 +++++++++-------- 2 files changed, 21 insertions(+), 8 deletions(-)
arch/x86/include/asm/trapnr.h +12 -0
@@ -2,6 +2,18 @@#ifndef _ASM_X86_TRAPNR_H#define _ASM_X86_TRAPNR_H +/* +* Event type codes used by FRED, Intel VT-x and AMD SVM +*/ +#define EVENT_TYPE_EXTINT 0 // External interrupt +#define EVENT_TYPE_RESERVED 1 +#define EVENT_TYPE_NMI 2 // NMI +#define EVENT_TYPE_HWEXC 3 // Hardware originated traps, exceptions +#define EVENT_TYPE_SWINT 4 // INT n +#define EVENT_TYPE_PRIV_SWEXC 5 // INT1 +#define EVENT_TYPE_SWEXC 6 // INTO, INT3 +#define EVENT_TYPE_OTHER 7 // FRED SYSCALL/SYSENTER, VT-x MTF +/* Interrupts/Exceptions */ #define X86_TRAP_DE 0/* Divide-by-zero */
arch/x86/include/asm/vmx.h +9 -8
@@ -17,6 +17,7 @@#include <linux/types.h> #include <uapi/asm/vmx.h> +#include <asm/trapnr.h> #include <asm/vmxfeatures.h> #define VMCS_CONTROL_BIT (x) BIT(VMX_FEATURE_##x & 0x1f)
@@ -374,14 +375,14 @@ enum vmcs_field {#define VECTORING_INFO_DELIVER_CODE_MASK INTR_INFO_DELIVER_CODE_MASK#define VECTORING_INFO_VALID_MASK INTR_INFO_VALID_MASK -#define INTR_TYPE_EXT_INTR (0 << 8)/* external interrupt */ -#define INTR_TYPE_RESERVED (1 << 8)/* reserved */ -#define INTR_TYPE_NMI_INTR (2 << 8)/* NMI */ -#define INTR_TYPE_HARD_EXCEPTION (3 << 8)/* processor exception */ -#define INTR_TYPE_SOFT_INTR (4 << 8)/* software interrupt */ -#define INTR_TYPE_PRIV_SW_EXCEPTION (5 << 8)/* ICE breakpoint - undocumented */ -#define INTR_TYPE_SOFT_EXCEPTION (6 << 8)/* software exception */ -#define INTR_TYPE_OTHER_EVENT (7 << 8)/* other event */ +#define INTR_TYPE_EXT_INTR (EVENT_TYPE_EXTINT << 8)/* external interrupt */ +#define INTR_TYPE_RESERVED (EVENT_TYPE_RESERVED << 8)/* reserved */ +#define INTR_TYPE_NMI_INTR (EVENT_TYPE_NMI << 8)/* NMI */ +#define INTR_TYPE_HARD_EXCEPTION (EVENT_TYPE_HWEXC << 8)/* processor exception */ +#define INTR_TYPE_SOFT_INTR (EVENT_TYPE_SWINT << 8)/* software interrupt */ +#define INTR_TYPE_PRIV_SW_EXCEPTION (EVENT_TYPE_PRIV_SWEXC << 8)/* ICE breakpoint */ +#define INTR_TYPE_SOFT_EXCEPTION (EVENT_TYPE_SWEXC << 8)/* software exception */ +#define INTR_TYPE_OTHER_EVENT (EVENT_TYPE_OTHER << 8)/* other event */ /* GUEST_INTERRUPTIBILITY_INFO flags. */ #define GUEST_INTR_STATE_STI 0x00000001
Signature · 1 line
2.34.1
PATCH v12 6/37 Documentation/x86/64: Add a documentation for FRED
In reply to Xin LiBriefly introduce FRED, and its advantages compared to IDT.
---
Changes since v10:
* Reword a sentence to improve readability (Nikolay Borisov).
--- Documentation/arch/x86/x86_64/fred.rst | 96 +++++++++++++++++++++++++ Documentation/arch/x86/x86_64/index.rst | 1 + 2 files changed, 97 insertions(+) create mode 100644 Documentation/arch/x86/x86_64/fred.rst
Documentation/arch/x86/x86_64/fred.rst new file +96 -0
@@ -0,0 +1,96 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================= +Flexible Return and Event Delivery (FRED) +========================================= + +Overview +======== + +The FRED architecture defines simple new transitions that change +privilege level (ring transitions). The FRED architecture was +designed with the following goals: + +1) Improve overall performance and response time by replacing event + delivery through the interrupt descriptor table (IDT event + delivery) and event return by the IRET instruction with lower + latency transitions. + +2) Improve software robustness by ensuring that event delivery + establishes the full supervisor context and that event return + establishes the full user context. + +The new transitions defined by the FRED architecture are FRED event +delivery and, for returning from events, two FRED return instructions. +FRED event delivery can effect a transition from ring 3 to ring 0, but +it is used also to deliver events incident to ring 0. One FRED +instruction (ERETU) effects a return from ring 0 to ring 3, while the +other (ERETS) returns while remaining in ring 0. Collectively, FRED +event delivery and the FRED return instructions are FRED transitions. + +In addition to these transitions, the FRED architecture defines a new +instruction (LKGS) for managing the state of the GS segment register. +The LKGS instruction can be used by 64-bit operating systems that do +not use the new FRED transitions. + +Furthermore, the FRED architecture is easy to extend for future CPU +architectures. + +Software based event dispatching +================================ + +FRED operates differently from IDT in terms of event handling. Instead +of directly dispatching an event to its handler based on the event +vector, FRED requires the software to dispatch an event to its handler +based on both the event's type and vector. Therefore, an event dispatch +framework must be implemented to facilitate the event-to-handler +dispatch process. The FRED event dispatch framework takes control +once an event is delivered, and employs a two-level dispatch. + +The first level dispatching is event type based, and the second level +dispatching is event vector based. + +Full supervisor/user context +============================ + +FRED event delivery atomically save and restore full supervisor/user +context upon event delivery and return. Thus it avoids the problem of +transient states due to %cr2 and/or %dr6, and it is no longer needed +to handle all the ugly corner cases caused by half baked entry states. + +FRED allows explicit unblock of NMI with new event return instructions +ERETS/ERETU, avoiding the mess caused by IRET which unconditionally +unblocks NMI, e.g., when an exception happens during NMI handling. + +FRED always restores the full value of %rsp, thus ESPFIX is no longer +needed when FRED is enabled. + +LKGS +==== + +LKGS behaves like the MOV to GS instruction except that it loads the +base address into the IA32_KERNEL_GS_BASE MSR instead of the GS +segment’s descriptor cache. With LKGS, it ends up with avoiding +mucking with kernel GS, i.e., an operating system can always operate +with its own GS base address. + +Because FRED event delivery from ring 3 and ERETU both swap the value +of the GS base address and that of the IA32_KERNEL_GS_BASE MSR, plus +the introduction of LKGS instruction, the SWAPGS instruction is no +longer needed when FRED is enabled, thus is disallowed (#UD). + +Stack levels +============ + +4 stack levels 0~3 are introduced to replace the nonreentrant IST for +event handling, and each stack level should be configured to use a +dedicated stack. + +The current stack level could be unchanged or go higher upon FRED +event delivery. If unchanged, the CPU keeps using the current event +stack. If higher, the CPU switches to a new event stack specified by +the MSR of the new stack level, i.e., MSR_IA32_FRED_RSP[123]. + +Only execution of a FRED return instruction ERET[US], could lower the +current stack level, causing the CPU to switch back to the stack it was +on before a previous event delivery that promoted the stack level.
Documentation/arch/x86/x86_64/index.rst +1 -0
@@ -15,3 +15,4 @@ x86_64 Support cpu-hotplug-spec machinecheck fsgs + fred
Signature · 1 line
2.34.1
Re: [PATCH v12 06/37] Documentation/x86/64: Add a documentation for FRED
Quoted 92 lines · Xin Li
Documentation/arch/x86/x86_64/fred.rst new file +86
-0@@ -0,0 +1,96 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================= +Flexible Return and Event Delivery (FRED) +========================================= + +Overview +======== + +The FRED architecture defines simple new transitions that change +privilege level (ring transitions). The FRED architecture was +designed with the following goals: + +1) Improve overall performance and response time by replacing event + delivery through the interrupt descriptor table (IDT event + delivery) and event return by the IRET instruction with lower + latency transitions. + +2) Improve software robustness by ensuring that event delivery + establishes the full supervisor context and that event return + establishes the full user context. + +The new transitions defined by the FRED architecture are FRED event +delivery and, for returning from events, two FRED return instructions. +FRED event delivery can effect a transition from ring 3 to ring 0, but +it is used also to deliver events incident to ring 0. One FRED +instruction (ERETU) effects a return from ring 0 to ring 3, while the +other (ERETS) returns while remaining in ring 0. Collectively, FRED +event delivery and the FRED return instructions are FRED transitions. + +In addition to these transitions, the FRED architecture defines a new +instruction (LKGS) for managing the state of the GS segment register. +The LKGS instruction can be used by 64-bit operating systems that do +not use the new FRED transitions. + +Furthermore, the FRED architecture is easy to extend for future CPU +architectures. + +Software based event dispatching +================================ + +FRED operates differently from IDT in terms of event handling. Instead +of directly dispatching an event to its handler based on the event +vector, FRED requires the software to dispatch an event to its handler +based on both the event's type and vector. Therefore, an event dispatch +framework must be implemented to facilitate the event-to-handler +dispatch process. The FRED event dispatch framework takes control +once an event is delivered, and employs a two-level dispatch. + +The first level dispatching is event type based, and the second level +dispatching is event vector based. + +Full supervisor/user context +============================ + +FRED event delivery atomically save and restore full supervisor/user +context upon event delivery and return. Thus it avoids the problem of +transient states due to %cr2 and/or %dr6, and it is no longer needed +to handle all the ugly corner cases caused by half baked entry states. + +FRED allows explicit unblock of NMI with new event return instructions +ERETS/ERETU, avoiding the mess caused by IRET which unconditionally +unblocks NMI, e.g., when an exception happens during NMI handling. + +FRED always restores the full value of %rsp, thus ESPFIX is no longer +needed when FRED is enabled. + +LKGS +==== + +LKGS behaves like the MOV to GS instruction except that it loads the +base address into the IA32_KERNEL_GS_BASE MSR instead of the GS +segment’s descriptor cache. With LKGS, it ends up with avoiding +mucking with kernel GS, i.e., an operating system can always operate +with its own GS base address. + +Because FRED event delivery from ring 3 and ERETU both swap the value +of the GS base address and that of the IA32_KERNEL_GS_BASE MSR, plus +the introduction of LKGS instruction, the SWAPGS instruction is no +longer needed when FRED is enabled, thus is disallowed (#UD). + +Stack levels +============ + +4 stack levels 0~3 are introduced to replace the nonreentrant IST for +event handling, and each stack level should be configured to use a
Documentation/arch/x86/x86_64/fred.rst line 96
Quoted 10 lines
Documentation/arch/x86/x86_64/fred.rst new file +10
-0+dedicated stack. + +The current stack level could be unchanged or go higher upon FRED +event delivery. If unchanged, the CPU keeps using the current event +stack. If higher, the CPU switches to a new event stack specified by +the MSR of the new stack level, i.e., MSR_IA32_FRED_RSP[123]. + +Only execution of a FRED return instruction ERET[US], could lower the +current stack level, causing the CPU to switch back to the stack it was +on before a previous event delivery that promoted the stack level.
LGTM, thanks!
Signature · 1 line
An old man doll... just what I always wanted! - Clara
RE: [PATCH v12 06/37] Documentation/x86/64: Add a documentation for FRED
Quoted 15 lines
Documentation/arch/x86/x86_64/fred.rst line 5
Quoted 12 lines
Documentation/arch/x86/x86_64/fred.rst new file +5
-0@@ -0,0 +1,96 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================================= +Flexible Return and Event Delivery (FRED) +=========================================LGTM, thanks!
Quoted 1 line
Reviewed-by Bagas Sanjaya <[email protected]>
Thanks a lot for reviewing it!
Xin
PATCH v12 7/37 x86/fred: Add Kconfig option for FRED (CONFIG_X86_FRED)
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Add the configuration option CONFIG_X86_FRED to enable FRED.
--- arch/x86/Kconfig | 9 +++++++++ 1 file changed, 9 insertions(+)
arch/x86/Kconfig +9 -0
@@ -496,6 +496,15 @@ config X86_CPU_RESCTRL Say N if unsure. +config X86_FRED + bool "Flexible Return and Event Delivery" + depends on X86_64 + help + When enabled, try to use Flexible Return and Event Delivery + instead of the legacy SYSCALL/SYSENTER/IDT architecture for + ring transitions and exception/interrupt handling if the + system supports. + if X86_32 config X86_BIGSMP bool "Support for big SMP systems with more than 8 CPUs"
Signature · 1 line
2.34.1
PATCH v12 8/37 x86/cpufeatures: Add the cpu feature bit for FRED
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Any FRED CPU will always have the following features as its baseline:
1) LKGS, load attributes of the GS segment but the base address into the IA32_KERNEL_GS_BASE MSR instead of the GS segment’s descriptor cache.
2) WRMSRNS, non-serializing WRMSR for faster MSR writes.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/cpu/cpuid-deps.c | 2 ++ tools/arch/x86/include/asm/cpufeatures.h | 1 + 3 files changed, 4 insertions(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -321,6 +321,7 @@#define X86_FEATURE_FZRM (12*32+10)/* "" Fast zero-length REP MOVSB */ #define X86_FEATURE_FSRS (12*32+11)/* "" Fast short REP STOSB */ #define X86_FEATURE_FSRC (12*32+12)/* "" Fast short REP {CMPSB,SCASB} */ +#define X86_FEATURE_FRED (12*32+17)/* Flexible Return and Event Delivery */ #define X86_FEATURE_LKGS (12*32+18)/* "" Load "kernel" (userspace) GS */ #define X86_FEATURE_WRMSRNS (12*32+19)/* "" Non-Serializing Write to Model Specific Register instruction */ #define X86_FEATURE_AMX_FP16 (12*32+21)/* "" AMX fp16 Support */
arch/x86/kernel/cpu/cpuid-deps.c +2 -0
@@ -82,6 +82,8 @@ static const struct cpuid_dep cpuid_deps[] = { { X86_FEATURE_XFD, X86_FEATURE_XGETBV1 }, { X86_FEATURE_AMX_TILE, X86_FEATURE_XFD }, { X86_FEATURE_SHSTK, X86_FEATURE_XSAVES }, + { X86_FEATURE_FRED, X86_FEATURE_LKGS }, + { X86_FEATURE_FRED, X86_FEATURE_WRMSRNS }, {} };
tools/arch/x86/include/asm/cpufeatures.h +1 -0
@@ -317,6 +317,7 @@#define X86_FEATURE_FZRM (12*32+10)/* "" Fast zero-length REP MOVSB */ #define X86_FEATURE_FSRS (12*32+11)/* "" Fast short REP STOSB */ #define X86_FEATURE_FSRC (12*32+12)/* "" Fast short REP {CMPSB,SCASB} */ +#define X86_FEATURE_FRED (12*32+17)/* Flexible Return and Event Delivery */ #define X86_FEATURE_LKGS (12*32+18)/* "" Load "kernel" (userspace) GS */ #define X86_FEATURE_WRMSRNS (12*32+19)/* "" Non-Serializing Write to Model Specific Register instruction */ #define X86_FEATURE_AMX_FP16 (12*32+21)/* "" AMX fp16 Support */
Signature · 1 line
2.34.1
PATCH v12 9/37 x86/fred: Disable FRED support if CONFIG_X86_FRED is disabled
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Add CONFIG_X86_FRED to <asm/disabled-features.h> to make cpu_feature_enabled() work correctly with FRED.
---
Changes since v10:
* FRED feature is defined in cpuid word 12, not 13 (Nikolay Borisov).
--- arch/x86/include/asm/disabled-features.h | 8 +++++++- tools/arch/x86/include/asm/disabled-features.h | 8 +++++++- 2 files changed, 14 insertions(+), 2 deletions(-)
arch/x86/include/asm/disabled-features.h +7 -1
@@ -117,6 +117,12 @@#define DISABLE_IBT (1 << (X86_FEATURE_IBT & 31))#endif +#ifdef CONFIG_X86_FRED +# define DISABLE_FRED 0 +#else +# define DISABLE_FRED (1 << (X86_FEATURE_FRED & 31)) +#endif +/* * Make sure to add features to the correct mask */
@@ -133,7 +139,7 @@#define DISABLED_MASK10 0#define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING|DISABLE_USER_SHSTK) -#define DISABLED_MASK12 (DISABLE_LAM) +#define DISABLED_MASK12 (DISABLE_FRED|DISABLE_LAM)#define DISABLED_MASK13 0#define DISABLED_MASK14 0#define DISABLED_MASK15 0
tools/arch/x86/include/asm/disabled-features.h +7 -1
@@ -105,6 +105,12 @@# define DISABLE_TDX_GUEST (1 << (X86_FEATURE_TDX_GUEST & 31))#endif +#ifdef CONFIG_X86_FRED +# define DISABLE_FRED 0 +#else +# define DISABLE_FRED (1 << (X86_FEATURE_FRED & 31)) +#endif +/* * Make sure to add features to the correct mask */
@@ -121,7 +127,7 @@#define DISABLED_MASK10 0#define DISABLED_MASK11 (DISABLE_RETPOLINE|DISABLE_RETHUNK|DISABLE_UNRET| \ DISABLE_CALL_DEPTH_TRACKING) -#define DISABLED_MASK12 (DISABLE_LAM) +#define DISABLED_MASK12 (DISABLE_FRED|DISABLE_LAM)#define DISABLED_MASK13 0#define DISABLED_MASK14 0#define DISABLED_MASK15 0
Signature · 1 line
2.34.1
PATCH v12 10/37 x86/fred: Disable FRED by default in its early stage
In reply to Xin LiTo enable FRED, a new kernel command line option "fred" needs to be added.
--- Documentation/admin-guide/kernel-parameters.txt | 3 +++ arch/x86/kernel/cpu/common.c | 3 +++ 2 files changed, 6 insertions(+)
Documentation/admin-guide/kernel-parameters.txt +3 -0
@@ -1525,6 +1525,9 @@ Warning: use of this parameter will taint the kernel and may cause unknown problems. + fred [X86-64] + Enable flexible return and event delivery + ftrace=[tracer] [FTRACE] will set and start the specified tracer as early as possible in order to facilitate early
arch/x86/kernel/cpu/common.c +3 -0
@@ -1502,6 +1502,9 @@ static void __init cpu_parse_early_param(void)char *argptr = arg, *opt;int arglen, taint =0 ; +if (!cmdline_find_option_bool (boot_command_line,"fred" )) +setup_clear_cpu_cap (X86_FEATURE_FRED ); +#ifdef CONFIG_X86_32 if (cmdline_find_option_bool (boot_command_line,"no387" )) #ifdef CONFIG_MATH_EMULATION
Signature · 1 line
2.34.1
PATCH v12 11/37 x86/opcode: Add ERET[US] instructions to the x86 opcode map
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
ERETU returns from an event handler while making a transition to ring 3, and ERETS returns from an event handler while staying in ring 0.
Add instruction opcodes used by ERET[US] to the x86 opcode map; opcode numbers are per FRED spec v5.0.
--- arch/x86/lib/x86-opcode-map.txt | 2 +- tools/arch/x86/lib/x86-opcode-map.txt | 2 +- 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/lib/x86-opcode-map.txt +1 -1
@@ -1052,7 +1052,7 @@ EndTable GrpTable: Grp7 0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) | WRMSRNS (110),(11B) -1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) +1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) | ERETU (F3),(010),(11B) | ERETS (F2),(010),(11B) 2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) | ENCLU (111),(11B) 3: LIDT Ms 4: SMSW Mw/Rv
tools/arch/x86/lib/x86-opcode-map.txt +1 -1
@@ -1052,7 +1052,7 @@ EndTable GrpTable: Grp7 0: SGDT Ms | VMCALL (001),(11B) | VMLAUNCH (010),(11B) | VMRESUME (011),(11B) | VMXOFF (100),(11B) | PCONFIG (101),(11B) | ENCLV (000),(11B) | WRMSRNS (110),(11B) -1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) +1: SIDT Ms | MONITOR (000),(11B) | MWAIT (001),(11B) | CLAC (010),(11B) | STAC (011),(11B) | ENCLS (111),(11B) | ERETU (F3),(010),(11B) | ERETS (F2),(010),(11B) 2: LGDT Ms | XGETBV (000),(11B) | XSETBV (001),(11B) | VMFUNC (100),(11B) | XEND (101)(11B) | XTEST (110)(11B) | ENCLU (111),(11B) 3: LIDT Ms 4: SMSW Mw/Rv
Signature · 1 line
2.34.1
PATCH v12 12/37 x86/objtool: Teach objtool about ERET[US]
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Update the objtool decoder to know about the ERET[US] instructions (type INSN_CONTEXT_SWITCH).
--- tools/objtool/arch/x86/decode.c | 19 ++++++++++++++----- 1 file changed, 14 insertions(+), 5 deletions(-)
tools/objtool/arch/x86/decode.c +14 -5
@@ -509,11 +509,20 @@ int arch_decode_instruction(struct objtool_file *file, const struct section *secif (op2 ==0x01 ) { -if (modrm ==0xca ) - insn->type =INSN_CLAC ; -else if (modrm ==0xcb ) - insn->type =INSN_STAC ; - +switch (insn_last_prefix_id (&ins)) { +case INAT_PFX_REPE : +case INAT_PFX_REPNE : +if (modrm ==0xca ) +/* eretu/erets */ + insn->type =INSN_CONTEXT_SWITCH ; +break ; +default : +if (modrm ==0xca ) + insn->type =INSN_CLAC ; +else if (modrm ==0xcb ) + insn->type =INSN_STAC ; +break ; + } }else if (op2 >=0x80 && op2 <=0x8f ) { insn->type =INSN_JUMP_CONDITIONAL ;
Signature · 1 line
2.34.1
PATCH v12 13/37 x86/cpu: Add X86_CR4_FRED macro
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Add X86_CR4_FRED macro for the FRED bit in %cr4. This bit must not be changed after initialization, so add it to the pinned CR4 bits.
---
Changes since v9:
* Avoid a type cast by defining X86_CR4_FRED as 0 on 32-bit (Thomas Gleixner).
--- arch/x86/include/uapi/asm/processor-flags.h | 7 +++++++ arch/x86/kernel/cpu/common.c | 5 ++--- 2 files changed, 9 insertions(+), 3 deletions(-)
arch/x86/include/uapi/asm/processor-flags.h +7 -0
@@ -139,6 +139,13 @@#define X86_CR4_LAM_SUP_BIT 28/* LAM for supervisor pointers */ #define X86_CR4_LAM_SUP _BITUL(X86_CR4_LAM_SUP_BIT) +#ifdef __x86_64__ +#define X86_CR4_FRED_BIT 32/* enable FRED kernel entry */ +#define X86_CR4_FRED _BITUL(X86_CR4_FRED_BIT) +#else +#define X86_CR4_FRED (0) +#endif +/* * x86-64 Task Priority Register, CR8 */
arch/x86/kernel/cpu/common.c +2 -3
@@ -401,9 +401,8 @@ static __always_inline void setup_umip(struct cpuinfo_x86 *c) }/* These bits should not change their value after CPU init is finished. */ -static const unsigned long cr4_pinned_mask = -X86_CR4_SMEP |X86_CR4_SMAP |X86_CR4_UMIP | -X86_CR4_FSGSBASE |X86_CR4_CET ; +static const unsigned long cr4_pinned_mask =X86_CR4_SMEP |X86_CR4_SMAP |X86_CR4_UMIP | +X86_CR4_FSGSBASE |X86_CR4_CET |X86_CR4_FRED ;static DEFINE_STATIC_KEY_FALSE_RO (cr_pinning);static unsigned long cr4_pinned_bits__ro_after_init ;
Signature · 1 line
2.34.1
PATCH v12 14/37 x86/cpu: Add MSR numbers for FRED configuration
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Add MSR numbers for the FRED configuration registers per FRED spec 5.0.
--- arch/x86/include/asm/msr-index.h | 13 ++++++++++++- tools/arch/x86/include/asm/msr-index.h | 13 ++++++++++++- 2 files changed, 24 insertions(+), 2 deletions(-)
arch/x86/include/asm/msr-index.h +12 -1
@@ -36,8 +36,19 @@#define EFER_FFXSR (1<<_EFER_FFXSR)#define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) -/* Intel MSRs. Some also available on other CPUs */ +/* FRED MSRs */ +#define MSR_IA32_FRED_RSP0 0x1cc/* Level 0 stack pointer */ +#define MSR_IA32_FRED_RSP1 0x1cd/* Level 1 stack pointer */ +#define MSR_IA32_FRED_RSP2 0x1ce/* Level 2 stack pointer */ +#define MSR_IA32_FRED_RSP3 0x1cf/* Level 3 stack pointer */ +#define MSR_IA32_FRED_STKLVLS 0x1d0/* Exception stack levels */ +#define MSR_IA32_FRED_SSP0 MSR_IA32_PL0_SSP/* Level 0 shadow stack pointer */ +#define MSR_IA32_FRED_SSP1 0x1d1/* Level 1 shadow stack pointer */ +#define MSR_IA32_FRED_SSP2 0x1d2/* Level 2 shadow stack pointer */ +#define MSR_IA32_FRED_SSP3 0x1d3/* Level 3 shadow stack pointer */ +#define MSR_IA32_FRED_CONFIG 0x1d4/* Entrypoint and interrupt stack level */ +/* Intel MSRs. Some also available on other CPUs */ #define MSR_TEST_CTRL 0x00000033#define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29#define MSR_TEST_CTRL_SPLIT_LOCK_DETECT BIT(MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT)
tools/arch/x86/include/asm/msr-index.h +12 -1
@@ -36,8 +36,19 @@#define EFER_FFXSR (1<<_EFER_FFXSR)#define EFER_AUTOIBRS (1<<_EFER_AUTOIBRS) -/* Intel MSRs. Some also available on other CPUs */ +/* FRED MSRs */ +#define MSR_IA32_FRED_RSP0 0x1cc/* Level 0 stack pointer */ +#define MSR_IA32_FRED_RSP1 0x1cd/* Level 1 stack pointer */ +#define MSR_IA32_FRED_RSP2 0x1ce/* Level 2 stack pointer */ +#define MSR_IA32_FRED_RSP3 0x1cf/* Level 3 stack pointer */ +#define MSR_IA32_FRED_STKLVLS 0x1d0/* Exception stack levels */ +#define MSR_IA32_FRED_SSP0 MSR_IA32_PL0_SSP/* Level 0 shadow stack pointer */ +#define MSR_IA32_FRED_SSP1 0x1d1/* Level 1 shadow stack pointer */ +#define MSR_IA32_FRED_SSP2 0x1d2/* Level 2 shadow stack pointer */ +#define MSR_IA32_FRED_SSP3 0x1d3/* Level 3 shadow stack pointer */ +#define MSR_IA32_FRED_CONFIG 0x1d4/* Entrypoint and interrupt stack level */ +/* Intel MSRs. Some also available on other CPUs */ #define MSR_TEST_CTRL 0x00000033#define MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT 29#define MSR_TEST_CTRL_SPLIT_LOCK_DETECT BIT(MSR_TEST_CTRL_SPLIT_LOCK_DETECT_BIT)
Signature · 1 line
2.34.1
PATCH v12 15/37 x86/ptrace: Cleanup the definition of the pt_regs structure
In reply to Xin Listruct pt_regs is hard to read because the member or section related comments are not aligned with the members.
The 'cs' and 'ss' members of pt_regs are type of 'unsigned long' while in reality they are only 16-bit wide. This works so far as the remaining space is unused, but FRED will use the remaining bits for other purposes.
To prepare for FRED:
- Cleanup the formatting
- Convert 'cs' and 'ss' to u16 and embed them into an union with a u64
- Fixup the related printk() format strings
--- arch/x86/entry/vsyscall/vsyscall_64.c | 2 +- arch/x86/include/asm/ptrace.h | 44 +++++++++++++++++++-------- arch/x86/kernel/process_64.c | 2 +- 3 files changed, 33 insertions(+), 15 deletions(-)
arch/x86/entry/vsyscall/vsyscall_64.c +1 -1
@@ -76,7 +76,7 @@ static void warn_bad_vsyscall(const char *level, struct pt_regs *regs,if (!show_unhandled_signals)return ; -printk_ratelimited ("%s%s[%d] %s ip:%lx cs:%lx sp:%lx ax:%lx si:%lx di:%lx\n" , +printk_ratelimited ("%s%s[%d] %s ip:%lx cs:%x sp:%lx ax:%lx si:%lx di:%lx\n" , level, current->comm,task_pid_nr (current), message, regs->ip, regs->cs, regs->sp, regs->ax, regs->si, regs->di);
arch/x86/include/asm/ptrace.h +31 -13
66 lines not on this page · raw message
arch/x86/kernel/process_64.c +1 -1
@@ -117,7 +117,7 @@ void __show_regs(struct pt_regs *regs, enum show_regs_mode mode,printk ("%sFS: %016lx(%04x) GS:%016lx(%04x) knlGS:%016lx\n" , log_lvl, fs, fsindex, gs, gsindex, shadowgs); -printk ("%sCS: %04lx DS: %04x ES: %04x CR0: %016lx\n" , +printk ("%sCS: %04x DS: %04x ES: %04x CR0: %016lx\n" , log_lvl, regs->cs, ds, es, cr0);printk ("%sCR2: %016lx CR3: %016lx CR4: %016lx\n" , log_lvl, cr2, cr3, cr4);
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v12 15/37] x86/ptrace: Cleanup the definition of the pt_regs structure
arch/x86/include/asm/ptrace.h struct pt_regs { · line 96
Quoted 9 lines · Xin Li
arch/x86/include/asm/ptrace.h +6
-2-/* Return frame for iretq */ + +/* The IRETQ return frame starts here */ unsigned long ip; -unsigned long cs; + +union { +u64 csx;// The full 64-bit data slot containing CS +u16 cs;// CS selector
I know people want to start using those // one-line comments now but this struct already uses the /* multiline ones. Can we stick to one type pls, otherwise it'll turn into a mess.
And pls put those comments ontop, not on the side.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquettePATCH v12 16/37 x86/ptrace: Add FRED additional information to the pt_regs structure
In reply to Xin LiFRED defines additional information in the upper 48 bits of cs/ss fields. Therefore add the information definitions into the pt_regs structure.
Specially introduce a new structure fred_ss to denote the FRED flags above SS selector, which avoids FRED_SSX_ macros and makes the code simpler and easier to read.
---
Changes since v11:
* Add a new structure fred_cs to denote the FRED flags above CS selector as what is done for SS (H. Peter Anvin).
Changes since v9:
* Introduce a new structure fred_ss to denote the FRED flags above SS selector, which avoids FRED_SSX_ macros and makes the code simpler and easier to read (Thomas Gleixner).
* Use type u64 to define FRED bit fields instead of type unsigned int (Thomas Gleixner).
Changes since v8:
* Reflect stack frame definition changes from FRED spec 3.0 to 5.0.
* Use __packed instead of __attribute__((__packed__)) (Borislav Petkov).
* Put all comments above the members, like the rest of the file does (Borislav Petkov).
Changes since v3:
* Rename csl/ssl of the pt_regs structure to csx/ssx (x for extended) (Andrew Cooper).
--- arch/x86/include/asm/ptrace.h | 70 ++++++++++++++++++++++++++++++++--- 1 file changed, 65 insertions(+), 5 deletions(-)
arch/x86/include/asm/ptrace.h +65 -5
@@ -56,6 +56,50 @@ struct pt_regs {#else /* __i386__ */ +struct fred_cs { +/* CS selector */ +u64 cs :16 , +/* Stack level at event time */ + sl :2 , +/* IBT in WAIT_FOR_ENDBRANCH state */ + wfe : 1, + :45 ; +}; + +struct fred_ss { +/* SS selector */ +u64 ss :16 , +/* STI state */ + sti :1 , +/* Set if syscall, sysenter or INT n */ + swevent :1 , +/* Event is NMI type */ + nmi : 1, + :13 , +/* Event vector */ + vector : 8, + :8 , +/* Event type */ + type : 4, + :4 , +/* Event was incident to enclave execution */ + enclave :1 , +/* CPU was in long mode */ + lm :1 , +/* +* Nested exception during FRED delivery, not set +* for #DF. +*/ + nested : 1, + :1 , +/* +* The length of the instruction causing the event. +* Only set for INTO, INT1, INT3, INT n, SYSCALL +* and SYSENTER. 0 otherwise. +*/ + insnlen :4 ; +}; + struct pt_regs {/* * C ABI says these regs are callee-preserved. They aren't saved on
@@ -85,6 +129,12 @@ struct pt_regs {* - the syscall number (syscall, sysenter, int80) * - error_code stored by the CPU on traps and exceptions * - the interrupt number for device interrupts +* +* A FRED stack frame starts here: +* 1) It _always_ includes an error code; +* +* 2) The return frame for ERET[US] starts here, but +* the content of orig_ax is ignored. */ unsigned long orig_ax;
@@ -92,20 +142,30 @@ struct pt_regs {unsigned long ip;union { -u64 csx;// The full 64-bit data slot containing CS -u16 cs;// CS selector +/* CS selector */ +u16 cs; +/* The extended 64-bit data slot containing CS */ +u64 csx; +/* The FRED CS extension */ +struct fred_cs fred_cs; };unsigned long flags;unsigned long sp;union { -u64 ssx;// The full 64-bit data slot containing SS -u16 ss;// SS selector +/* SS selector */ +u16 ss; +/* The extended 64-bit data slot containing SS */ +u64 ssx; +/* The FRED SS extension */ +struct fred_ss fred_ss; };/* -* Top of stack on IDT systems. +* Top of stack on IDT systems, while FRED systems have extra fields +* defined above for storing exception related information, e.g. CR2 or +* DR6. */ };
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v12 16/37] x86/ptrace: Add FRED additional information to the pt_regs structure
Quoted 9 lines · Xin Li
FRED defines additional information in the upper 48 bits of cs/ss fields. Therefore add the information definitions into the
pt_regsstructure.Specially introduce a new structure
fred_ssto denote the FRED flags above SS selector, which avoidsFRED_SSX_macros and makes the code simpler and easier to read.Signed-off-by H. Peter Anvin (Intel) <[email protected]>
You and hpa need to go through all the patches and figure out who's the author that's going to land in git.
Because this and others have hpa's SOB first, suggesting he's the author. However, the mail doesn't start with
From: H. Peter Anvin (Intel) <[email protected]>
and then git will make *you* the author.
Quoted 3 lines
Tested-by Shan Kang <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Xin Li <[email protected]>
...
arch/x86/include/asm/ptrace.h struct pt_regs { · line 162
Quoted 9 lines
arch/x86/include/asm/ptrace.h +6
-2union { -u64 ssx;// The full 64-bit data slot containing SS -u16 ss;// SS selector +/* SS selector */ +u16 ss; +/* The extended 64-bit data slot containing SS */ +u64 ssx; +/* The FRED SS extension */ +struct fred_ss fred_ss;
Aha, sanity about the right comments has come to your mind in this next patch. :-P
Just do them right in the previous one.
arch/x86/include/asm/ptrace.h struct pt_regs { · line 168
Quoted 5 lines
arch/x86/include/asm/ptrace.h +3
-1/* -* Top of stack on IDT systems. +* Top of stack on IDT systems, while FRED systems have extra fields +* defined above for storing exception related information, e.g. CR2 or +* DR6.
Btw, I really appreciate the good commenting - thanks for that!
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteH. Peter Anvin
maintainerRe: [PATCH v12 16/37] x86/ptrace: Add FRED additional information to the pt_regs structure
Quoted 42 lines · Borislav Petkov
Quoted 9 lines · Xin Li
FRED defines additional information in the upper 48 bits of cs/ss fields. Therefore add the information definitions into the
pt_regsstructure.Specially introduce a new structure
fred_ssto denote the FRED flags above SS selector, which avoidsFRED_SSX_macros and makes the code simpler and easier to read.Signed-off-by H. Peter Anvin (Intel) <[email protected]>You and hpa need to go through all the patches and figure out who's the author that's going to land in git.
Because this and others have hpa's SOB first, suggesting he's the author. However, the mail doesn't start with
From: H. Peter Anvin (Intel) <[email protected]>
and then git will make *you* the author.
Quoted 3 lines
Tested-by Shan Kang <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Xin Li <[email protected]>...
arch/x86/include/asm/ptrace.h struct pt_regs { · line 162
Quoted 9 lines
arch/x86/include/asm/ptrace.h +6
-2union { -u64 ssx;// The full 64-bit data slot containing SS -u16 ss;// SS selector +/* SS selector */ +u16 ss; +/* The extended 64-bit data slot containing SS */ +u64 ssx; +/* The FRED SS extension */ +struct fred_ss fred_ss;Aha, sanity about the right comments has come to your mind in this next patch. :-P
Just do them right in the previous one.
Quoted 8 lines
arch/x86/include/asm/ptrace.h struct pt_regs { · line 168
Quoted 5 lines
arch/x86/include/asm/ptrace.h +3
-1/* -* Top of stack on IDT systems. +* Top of stack on IDT systems, while FRED systems have extra fields +* defined above for storing exception related information, e.g. CR2 or +* DR6. Btw, I really appreciate the good commenting - thanks for that!
For Xin, mainly:
Standard practice is:
1. For a patch with relatively small modifications, or where the changes are mainly in comments or the patch message:
Keep the authorship, but put a description of what you have changed in brackets with your username at the bottom of the description, immediately before Signed-off-by:
[ xin: changed foo, bar, baz ]
2. For a patch with major rewrites:
Take authorship on the From: line, but have an Originally-by: tag (rather than a Signed-off-by: by the original author):
3. For a patch which is fully or nearly fully your own work (a total rewrite, or based on a concept idea rather than actual code), credit the original in the patch comment:
Based on an idea by Someone Else <[email protected]> (optional link to lore.kernel.org).
PATCH v12 17/37 x86/fred: Add a new header file for FRED definitions
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Add a header file for FRED prototypes and definitions.
---
Changes since v6:
* Replace pt_regs csx flags prefix FRED_CSL_ with FRED_CSX_.
--- arch/x86/include/asm/fred.h | 68 +++++++++++++++++++++++++++++++++++++ 1 file changed, 68 insertions(+) create mode 100644 arch/x86/include/asm/fred.h
arch/x86/include/asm/fred.h new file +68 -0
@@ -0,0 +1,68 @@ +/* SPDX-License-Identifier: GPL-2.0 */ +/* +* Macros for Flexible Return and Event Delivery (FRED) +*/ + +#ifndef ASM_X86_FRED_H +#define ASM_X86_FRED_H + +#include <linux/const.h> + +#include <asm/asm.h> + +/* +* FRED event return instruction opcodes for ERET{S,U}; supported in +* binutils >= 2.41. +*/ +#define ERETS _ASM_BYTES(0xf2,0x0f,0x01,0xca) +#define ERETU _ASM_BYTES(0xf3,0x0f,0x01,0xca) + +/* +* RSP is aligned to a 64-byte boundary before used to push a new stack frame +*/ +#define FRED_STACK_FRAME_RSP_MASK _AT(unsigned long, (~0x3f)) + +/* +* Used for the return address for call emulation during code patching, +* and measured in 64-byte cache lines. +*/ +#define FRED_CONFIG_REDZONE_AMOUNT 1 +#define FRED_CONFIG_REDZONE (_AT(unsigned long, FRED_CONFIG_REDZONE_AMOUNT) << 6) +#define FRED_CONFIG_INT_STKLVL (l) (_AT(unsigned long, l) << 9) +#define FRED_CONFIG_ENTRYPOINT (p) _AT(unsigned long, (p)) + +#ifndef __ASSEMBLY__ + +#ifdef CONFIG_X86_FRED +#include <linux/kernel.h> + +#include <asm/ptrace.h> + +struct fred_info { +/* Event data: CR2, DR6, ... */ +unsigned long edata; +unsigned long resv; +}; + +/* Full format of the FRED stack frame */ +struct fred_frame { +struct pt_regs regs; +struct fred_info info; +}; + +static __always_inline struct fred_info *fred_info(struct pt_regs *regs) +{ + return &container_of(regs, struct fred_frame, regs)->info; +} + +static __always_inline unsigned long fred_event_data (struct pt_regs *regs) +{ +return fred_info (regs)->edata; +} + +#else /* CONFIG_X86_FRED */ +static __always_inline unsigned long fred_event_data (struct pt_regs *regs) {return 0 ; } +#endif /* CONFIG_X86_FRED */ +#endif /* !__ASSEMBLY__ */ + +#endif /* ASM_X86_FRED_H */
Signature · 1 line
2.34.1
PATCH v12 18/37 x86/fred: Reserve space for the FRED stack frame
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
When using FRED, reserve space at the top of the stack frame, just like i386 does.
--- arch/x86/include/asm/thread_info.h | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
arch/x86/include/asm/thread_info.h +9 -3
@@ -31,7 +31,9 @@* In vm86 mode, the hardware frame is much longer still, so add 16 * bytes to make room for the real-mode segments. * -* x86_64 has a fixed-length stack frame. +* x86-64 has a fixed-length stack frame, but it depends on whether +* or not FRED is enabled. Future versions of FRED might make this +* dynamic, but for now it is always 2 words longer. */ #ifdef CONFIG_X86_32 # ifdef CONFIG_VM86
@@ -39,8 +41,12 @@# else # define TOP_OF_KERNEL_STACK_PADDING 8# endif -#else -# define TOP_OF_KERNEL_STACK_PADDING 0 +#else /* x86-64 */ +# ifdef CONFIG_X86_FRED +# define TOP_OF_KERNEL_STACK_PADDING (2 * 8) +# else +# define TOP_OF_KERNEL_STACK_PADDING 0 +# endif #endif /*
Signature · 1 line
2.34.1
PATCH v12 19/37 x86/fred: Update MSR_IA32_FRED_RSP0 during task switch
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
MSR_IA32_FRED_RSP0 is used during ring 3 event delivery, and needs to be updated to point to the top of next task stack during task switch.
--- arch/x86/include/asm/switch_to.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)
arch/x86/include/asm/switch_to.h +6 -2
@@ -70,9 +70,13 @@ static inline void update_task_stack(struct task_struct *task)#ifdef CONFIG_X86_32 this_cpu_write (cpu_tss_rw.x86_tss.sp1, task->thread.sp0);#else -/* Xen PV enters the kernel on the thread stack. */ -if (cpu_feature_enabled (X86_FEATURE_XENPV )) +if (cpu_feature_enabled (X86_FEATURE_FRED )) { +/* WRMSRNS is a baseline feature for FRED. */ +wrmsrns (MSR_IA32_FRED_RSP0 , (unsigned long )task_stack_page (task) +THREAD_SIZE ); + }else if (cpu_feature_enabled (X86_FEATURE_XENPV )) { +/* Xen PV enters the kernel on the thread stack. */ load_sp0 (task_top_of_stack (task)); + }#endif }
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v12 19/37] x86/fred: Update MSR_IA32_FRED_RSP0 during task switch
Quoted 17 lines · Xin Li
From: "H. Peter Anvin (Intel)" <[email protected]>
MSR_IA32_FRED_RSP0is used during ring 3 event delivery, and needs to be updated to point to the top of next task stack during task switch.Signed-off-by H. Peter Anvin (Intel) <[email protected]>Tested-by Shan Kang <[email protected]>Signed-off-by Xin Li <[email protected]>--- arch/x86/include/asm/switch_to.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/include/asm/switch_to.h +0
-0
arch/x86/include/asm/switch_to.h static inline void update_task_stack(struct task_struct *task) · line 75
Quoted 8 lines
arch/x86/include/asm/switch_to.h +3
-2@@ -70,9 +70,13 @@ static inline void update_task_stack(struct task_struct *task)#ifdef CONFIG_X86_32 this_cpu_write (cpu_tss_rw.x86_tss.sp1, task->thread.sp0);#else -/* Xen PV enters the kernel on the thread stack. */ -if (cpu_feature_enabled (X86_FEATURE_XENPV )) +if (cpu_feature_enabled (X86_FEATURE_FRED )) { +/* WRMSRNS is a baseline feature for FRED. */ +wrmsrns (MSR_IA32_FRED_RSP0 , (unsigned long )task_stack_page (task) +THREAD_SIZE );
If this non-serializing write happens now and, AFAICT, the CR3 write during the task switch has already happened in switch_mm* earlier, what is the serialization point that's going to make sure that write is committed before the new task starts executing?
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteH. Peter Anvin
maintainerRe: [PATCH v12 19/37] x86/fred: Update MSR_IA32_FRED_RSP0 during task switch
Quoted 32 lines · Borislav Petkov
Quoted 17 lines · Xin Li
From: "H. Peter Anvin (Intel)" <[email protected]>
MSR_IA32_FRED_RSP0is used during ring 3 event delivery, and needs to be updated to point to the top of next task stack during task switch.Signed-off-by H. Peter Anvin (Intel) <[email protected]>Tested-by Shan Kang <[email protected]>Signed-off-by Xin Li <[email protected]>--- arch/x86/include/asm/switch_to.h | 8 ++++++-- 1 file changed, 6 insertions(+), 2 deletions(-)arch/x86/include/asm/switch_to.h +0
-0arch/x86/include/asm/switch_to.h static inline void update_task_stack(struct task_struct *task) · line 75
Quoted 8 lines
arch/x86/include/asm/switch_to.h +3
-2@@ -70,9 +70,13 @@ static inline void update_task_stack(struct task_struct *task)#ifdef CONFIG_X86_32 this_cpu_write (cpu_tss_rw.x86_tss.sp1, task->thread.sp0);#else -/* Xen PV enters the kernel on the thread stack. */ -if (cpu_feature_enabled (X86_FEATURE_XENPV )) +if (cpu_feature_enabled (X86_FEATURE_FRED )) { +/* WRMSRNS is a baseline feature for FRED. */ +wrmsrns (MSR_IA32_FRED_RSP0 , (unsigned long )task_stack_page (task) +THREAD_SIZE );If this non-serializing write happens now and, AFAICT, the CR3 write during the task switch has already happened in switch_mm* earlier, what is the serialization point that's going to make sure that write is committed before the new task starts executing?
Quoted 2 lines
Thx.
A resource cannot be consumed after the value has been written; this is the only necessary level of serialization, equivalent to, say, RAX.
A serializing instruction stops the entire pipeline until everything has retired and any stores have become globally visible.
Borislav Petkov
maintainerRe: [PATCH v12 19/37] x86/fred: Update MSR_IA32_FRED_RSP0 during task switch
Quoted 2 lines · H. Peter Anvin
A resource cannot be consumed after the value has been written; this is the only necessary level of serialization, equivalent to, say, RAX.
Lemme see if I understand this correctly using this context as an
example: after this MSR_IA32_FRED_RSP0 write, any FRED events determined to be delivered to level 0 will use this new task stack ptr?
And since the new task is not running yet and the old one isn't running either, we're fine here. So the "serialization point" I was talking about above is bollocks.
Close? :)
Quoted 2 lines
A serializing instruction stops the entire pipeline until everything has retired and any stores have become globally visible.
Right, we don't need that here.
Thx.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteH. Peter Anvin
maintainerRe: [PATCH v12 19/37] x86/fred: Update MSR_IA32_FRED_RSP0 during task switch
Quoted 12 lines · Borislav Petkov
Quoted 2 lines · H. Peter Anvin
A resource cannot be consumed after the value has been written; this is the only necessary level of serialization, equivalent to, say, RAX.
Lemme see if I understand this correctly using this context as an
example: after thisMSR_IA32_FRED_RSP0write, any FRED events determined to be delivered to level 0 will use this new task stack ptr?And since the new task is not running yet and the old one isn't running either, we're fine here. So the "serialization point" I was talking about above is bollocks.
Quoted 9 lines
Close? :)
Quoted 2 lines
A serializing instruction stops the entire pipeline until everything has retired and any stores have become globally visible.
Right, we don't need that here.
Thx.
Yep!
PATCH v12 20/37 x86/fred: Disallow the swapgs instruction when FRED is enabled
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
SWAPGS is no longer needed thus NOT allowed with FRED because FRED transitions ensure that an operating system can _always_ operate with its own GS base address:
- For events that occur in ring 3, FRED event delivery swaps the GS base address with the IA32_KERNEL_GS_BASE MSR.
- ERETU (the FRED transition that returns to ring 3) also swaps the GS base address with the IA32_KERNEL_GS_BASE MSR.
And the operating system can still setup the GS segment for a user thread without the need of loading a user thread GS with:
- Using LKGS, available with FRED, to modify other attributes of the GS segment without compromising its ability always to operate with its own GS base address.
- Accessing the GS segment base address for a user thread as before using RDMSR or WRMSR on the IA32_KERNEL_GS_BASE MSR.
Note, LKGS loads the GS base address into the IA32_KERNEL_GS_BASE MSR instead of the GS segment’s descriptor cache. As such, the operating system never changes its runtime GS base address.
---
Changes since v8:
* Explain why writing directly to the IA32_KERNEL_GS_BASE MSR is doing the right thing (Thomas Gleixner).
--- arch/x86/kernel/process_64.c | 27 +++++++++++++++++++++++++-- 1 file changed, 25 insertions(+), 2 deletions(-)
arch/x86/kernel/process_64.c +25 -2
@@ -166,7 +166,29 @@ static noinstr unsigned long __rdgsbase_inactive(void)lockdep_assert_irqs_disabled (); -if (!cpu_feature_enabled (X86_FEATURE_XENPV )) { +/* +* SWAPGS is no longer needed thus NOT allowed with FRED because +* FRED transitions ensure that an operating system can _always_ +* operate with its own GS base address: +* - For events that occur in ring 3, FRED event delivery swaps +* the GS base address with the IA32_KERNEL_GS_BASE MSR. +* - ERETU (the FRED transition that returns to ring 3) also swaps +* the GS base address with the IA32_KERNEL_GS_BASE MSR. +* +* And the operating system can still setup the GS segment for a +* user thread without the need of loading a user thread GS with: +* - Using LKGS, available with FRED, to modify other attributes +* of the GS segment without compromising its ability always to +* operate with its own GS base address. +* - Accessing the GS segment base address for a user thread as +* before using RDMSR or WRMSR on the IA32_KERNEL_GS_BASE MSR. +* +* Note, LKGS loads the GS base address into the IA32_KERNEL_GS_BASE +* MSR instead of the GS segment’s descriptor cache. As such, the +* operating system never changes its runtime GS base address. +*/ +if (!cpu_feature_enabled (X86_FEATURE_FRED ) && + !cpu_feature_enabled (X86_FEATURE_XENPV )) {native_swapgs (); gsbase =rdgsbase ();native_swapgs ();
@@ -191,7 +213,8 @@ static noinstr void __wrgsbase_inactive(unsigned long gsbase) {lockdep_assert_irqs_disabled (); -if (!cpu_feature_enabled (X86_FEATURE_XENPV )) { +if (!cpu_feature_enabled (X86_FEATURE_FRED ) && + !cpu_feature_enabled (X86_FEATURE_XENPV )) {native_swapgs ();wrgsbase (gsbase);native_swapgs ();
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v12 20/37] x86/fred: Disallow the swapgs instruction when FRED is enabled
arch/x86/kernel/process_64.c static noinstr unsigned long __rdgsbase_inactive(void) · line 187
Quoted 2 lines · Xin Li
arch/x86/kernel/process_64.c +2
-0+* Note, LKGS loads the GS base address into the IA32_KERNEL_GS_BASE +* MSR instead of the GS segment’s descriptor cache. As such, the
:verify_diff: WARNING: Unicode char [’] (0x8217 in line: + * MSR instead of the GS segment’s descriptor cache. As such, the
Just do a normal ' - char number 0x27.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquettePATCH v12 21/37 x86/fred: No ESPFIX needed when FRED is enabled
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Because FRED always restores the full value of %rsp, ESPFIX is no longer needed when it's enabled.
--- arch/x86/kernel/espfix_64.c | 8 ++++++++ 1 file changed, 8 insertions(+)
arch/x86/kernel/espfix_64.c +8 -0
@@ -106,6 +106,10 @@ void __init init_espfix_bsp(void)pgd_t *pgd;p4d_t *p4d; +/* FRED systems always restore the full value of %rsp */ +if (cpu_feature_enabled (X86_FEATURE_FRED )) +return ; +/* Install the espfix pud into the kernel page directory */ pgd = &init_top_pgt[pgd_index (ESPFIX_BASE_ADDR )]; p4d =p4d_alloc (&init_mm, pgd,ESPFIX_BASE_ADDR );
@@ -129,6 +133,10 @@ void init_espfix_ap(int cpu)void *stack_page;pteval_t ptemask; +/* FRED systems always restore the full value of %rsp */ +if (cpu_feature_enabled (X86_FEATURE_FRED )) +return ; +/* We only have to do this once... */ if (likely (per_cpu (espfix_stack, cpu)))return ;/* Already initialized */
Signature · 1 line
2.34.1
PATCH v12 22/37 x86/fred: Allow single-step trap and NMI when starting a new task
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Entering a new task is logically speaking a return from a system call (exec, fork, clone, etc.). As such, if ptrace enables single stepping a single step exception should be allowed to trigger immediately upon entering user space. This is not optional.
NMI should *never* be disabled in user space. As such, this is an optional, opportunistic way to catch errors.
Allow single-step trap and NMI when starting a new task, thus once the new task enters user space, single-step trap and NMI are both enabled immediately.
---
Changes since v8:
* Use high-order 48 bits above the lowest 16 bit SS only when FRED is enabled (Thomas Gleixner).
--- arch/x86/kernel/process_64.c | 38 ++++++++++++++++++++++++++++++------ 1 file changed, 32 insertions(+), 6 deletions(-)
arch/x86/kernel/process_64.c +32 -6
@@ -56,6 +56,7 @@#include <asm/resctrl.h> #include <asm/unistd.h> #include <asm/fsgsbase.h> +#include <asm/fred.h> #ifdef CONFIG_IA32_EMULATION /* Not included via unistd.h */ #include <asm/unistd_32_ia32.h>
@@ -528,7 +529,7 @@ void x86_gsbase_write_task(struct task_struct *task, unsigned long gsbase)static void start_thread_common (struct pt_regs *regs,unsigned long new_ip,unsigned long new_sp, -unsigned int _cs,unsigned int _ss,unsigned int _ds) +u16 _cs,u16 _ss,u16 _ds) {WARN_ON_ONCE (regs !=current_pt_regs ());
@@ -545,11 +546,36 @@ start_thread_common(struct pt_regs *regs, unsigned long new_ip,loadsegment (ds, _ds);load_gs_index (0 ); - regs->ip = new_ip; - regs->sp = new_sp; - regs->cs = _cs; - regs->ss = _ss; - regs->flags =X86_EFLAGS_IF ; + regs->ip = new_ip; + regs->sp = new_sp; + regs->csx = _cs; + regs->ssx = _ss; +/* +* Allow single-step trap and NMI when starting a new task, thus +* once the new task enters user space, single-step trap and NMI +* are both enabled immediately. +* +* Entering a new task is logically speaking a return from a +* system call (exec, fork, clone, etc.). As such, if ptrace +* enables single stepping a single step exception should be +* allowed to trigger immediately upon entering user space. +* This is not optional. +* +* NMI should *never* be disabled in user space. As such, this +* is an optional, opportunistic way to catch errors. +* +* Paranoia: High-order 48 bits above the lowest 16 bit SS are +* discarded by the legacy IRET instruction on all Intel, AMD, +* and Cyrix/Centaur/VIA CPUs, thus can be set unconditionally, +* even when FRED is not enabled. But we choose the safer side +* to use these bits only when FRED is enabled. +*/ +if (cpu_feature_enabled (X86_FEATURE_FRED )) { + regs->fred_ss.swevent = true; + regs->fred_ss.nmi = true; + } + + regs->flags =X86_EFLAGS_IF |X86_EFLAGS_FIXED ; }void
Signature · 1 line
2.34.1
PATCH v12 23/37 x86/fred: Make exc_page_fault() work for FRED
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
On a FRED system, the faulting address (CR2) is passed on the stack, to avoid the problem of transient state. Thus we get the page fault address from the stack instead of CR2.
--- arch/x86/mm/fault.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
arch/x86/mm/fault.c +4 -1
@@ -34,6 +34,7 @@#include <asm/kvm_para.h> /* kvm_handle_async_pf */ #include <asm/vdso.h> /* fixup_vdso_exception() */ #include <asm/irq_stack.h> +#include <asm/fred.h> #define CREATE_TRACE_POINTS #include <asm/trace/exceptions.h>
@@ -1516,8 +1517,10 @@ handle_page_fault(struct pt_regs *regs, unsigned long error_code,DEFINE_IDTENTRY_RAW_ERRORCODE (exc_page_fault) { -unsigned long address =read_cr2 ();irqentry_state_t state; +unsigned long address; + + address =cpu_feature_enabled (X86_FEATURE_FRED ) ?fred_event_data (regs) :read_cr2 ();prefetchw (¤t->mm->mmap_lock);
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v12 23/37] x86/fred: Make exc_page_fault() work for FRED
Quoted 4 lines · Xin Li
From: "H. Peter Anvin (Intel)" <[email protected]>
On a FRED system, the faulting address (CR2) is passed on the stack, to avoid the problem of transient state. Thus we get the page fault
^^
Please use passive voice in your commit message: no "we" or "I", etc, and describe your changes in imperative mood.
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquettePATCH v12 24/37 x86/idtentry: Incorporate definitions/declarations of the FRED entries
In reply to Xin LiFRED and IDT can share most of the definitions and declarations so that in the majority of cases the actual handler implementation is the same.
The differences are the exceptions where FRED stores exception related information on the stack and the sysvec implementations as FRED can handle irqentry/exit() in the dispatcher instead of having it in each handler.
Also add stub defines for vectors which are not used due to Kconfig decisions to spare the ifdeffery in the actual FRED dispatch code.
---
Changes since v9:
* Except NMI/#DB/#MCE, FRED really should share the exception handlers with IDT (Thomas Gleixner).
Changes since v8:
* Put IDTENTRY changes in a separate patch (Thomas Gleixner).
--- arch/x86/include/asm/idtentry.h | 71 +++++++++++++++++++++++++++++---- 1 file changed, 63 insertions(+), 8 deletions(-)
arch/x86/include/asm/idtentry.h +63 -8
@@ -13,15 +13,18 @@#include <asm/irq_stack.h> +typedef void (*idtentry_t )(struct pt_regs *regs); +/** * DECLARE_IDTENTRY - Declare functions for simple IDT entry points * No error code pushed by hardware * @vector: Vector number (ignored for C) * @func: Function name of the entry point * -* Declares three functions: +* Declares four functions: * - The ASM entry point: asm_##func * - The XEN PV trap entry point: xen_##func (maybe unused) +* - The C handler called from the FRED event dispatcher (maybe unused) * - The C handler called from the ASM entry point * * Note: This is the C variant of DECLARE_IDTENTRY(). As the name says it
@@ -31,6 +34,7 @@#define DECLARE_IDTENTRY (vector, func) \ asmlinkage void asm_##func(void); \ asmlinkage void xen_asm_##func(void); \ + void fred_##func(struct pt_regs *regs); \ __visible void func(struct pt_regs *regs)/**
@@ -137,6 +141,17 @@ static __always_inline void __##func(struct pt_regs *regs, \#define DEFINE_IDTENTRY_RAW (func) \ __visible noinstr void func(struct pt_regs *regs) +/** +* DEFINE_FREDENTRY_RAW - Emit code for raw FRED entry points +* @func: Function name of the entry point +* +* @func is called from the FRED event dispatcher with interrupts disabled. +* +* See @DEFINE_IDTENTRY_RAW for further details. +*/ +#define DEFINE_FREDENTRY_RAW (func) \ +noinstr void fred_##func(struct pt_regs *regs) +/** * DECLARE_IDTENTRY_RAW_ERRORCODE - Declare functions for raw IDT entry points * Error code pushed by hardware
@@ -233,17 +248,27 @@ static noinline void __##func(struct pt_regs *regs, u32 vector)#define DEFINE_IDTENTRY_SYSVEC (func) \ static void __##func(struct pt_regs *regs); \ \ +static __always_inline void instr_##func(struct pt_regs *regs) \ +{ \ + kvm_set_cpu_l1tf_flush_l1d(); \ + run_sysvec_on_irqstack_cond(__##func, regs); \ +} \ + \ __visible noinstr void func(struct pt_regs *regs) \ { \ irqentry_state_t state = irqentry_enter(regs); \ \ instrumentation_begin(); \ - kvm_set_cpu_l1tf_flush_l1d(); \ - run_sysvec_on_irqstack_cond(__##func, regs); \ + instr_##func (regs); \ instrumentation_end(); \ irqentry_exit(regs, state); \ } \ \ +void fred_##func(struct pt_regs *regs) \ +{ \ + instr_##func (regs); \ +} \ + \ static noinline void __##func(struct pt_regs *regs)/**
@@ -260,19 +285,29 @@ static noinline void __##func(struct pt_regs *regs)#define DEFINE_IDTENTRY_SYSVEC_SIMPLE (func) \ static __always_inline void __##func(struct pt_regs *regs); \ \ -__visible noinstr void func(struct pt_regs *regs) \ +static __always_inline void instr_##func(struct pt_regs *regs) \ { \ - irqentry_state_t state = irqentry_enter(regs); \ - \ - instrumentation_begin(); \ __irq_enter_raw(); \ kvm_set_cpu_l1tf_flush_l1d(); \ __##func (regs); \ __irq_exit_raw(); \ +} \ + \ +__visible noinstr void func(struct pt_regs *regs) \ +{ \ + irqentry_state_t state = irqentry_enter(regs); \ + \ + instrumentation_begin(); \ + instr_##func (regs); \ instrumentation_end(); \ irqentry_exit(regs, state); \ } \ \ +void fred_##func(struct pt_regs *regs) \ +{ \ + instr_##func (regs); \ +} \ + \ static __always_inline void __##func(struct pt_regs *regs)/**
@@ -410,15 +445,18 @@ __visible noinstr void func(struct pt_regs *regs, \/* C-Code mapping */ #define DECLARE_IDTENTRY_NMI DECLARE_IDTENTRY_RAW#define DEFINE_IDTENTRY_NMI DEFINE_IDTENTRY_RAW +#define DEFINE_FREDENTRY_NMI DEFINE_FREDENTRY_RAW#ifdef CONFIG_X86_64 #define DECLARE_IDTENTRY_MCE DECLARE_IDTENTRY_IST#define DEFINE_IDTENTRY_MCE DEFINE_IDTENTRY_IST#define DEFINE_IDTENTRY_MCE_USER DEFINE_IDTENTRY_NOIST +#define DEFINE_FREDENTRY_MCE DEFINE_FREDENTRY_RAW#define DECLARE_IDTENTRY_DEBUG DECLARE_IDTENTRY_IST#define DEFINE_IDTENTRY_DEBUG DEFINE_IDTENTRY_IST#define DEFINE_IDTENTRY_DEBUG_USER DEFINE_IDTENTRY_NOIST +#define DEFINE_FREDENTRY_DEBUG DEFINE_FREDENTRY_RAW#endif #else /* !__ASSEMBLY__ */
@@ -651,23 +689,36 @@ DECLARE_IDTENTRY(RESCHEDULE_VECTOR, sysvec_reschedule_ipi);DECLARE_IDTENTRY_SYSVEC (REBOOT_VECTOR , sysvec_reboot);DECLARE_IDTENTRY_SYSVEC (CALL_FUNCTION_SINGLE_VECTOR , sysvec_call_function_single);DECLARE_IDTENTRY_SYSVEC (CALL_FUNCTION_VECTOR , sysvec_call_function); +#else +# define fred_sysvec_reschedule_ipi NULL +# define fred_sysvec_reboot NULL +# define fred_sysvec_call_function_single NULL +# define fred_sysvec_call_function NULL#endif #ifdef CONFIG_X86_LOCAL_APIC # ifdef CONFIG_X86_MCE_THRESHOLD DECLARE_IDTENTRY_SYSVEC (THRESHOLD_APIC_VECTOR , sysvec_threshold); +# else +# define fred_sysvec_threshold NULL# endif # ifdef CONFIG_X86_MCE_AMD DECLARE_IDTENTRY_SYSVEC (DEFERRED_ERROR_VECTOR , sysvec_deferred_error); +# else +# define fred_sysvec_deferred_error NULL# endif # ifdef CONFIG_X86_THERMAL_VECTOR DECLARE_IDTENTRY_SYSVEC (THERMAL_APIC_VECTOR , sysvec_thermal); +# else +# define fred_sysvec_thermal NULL# endif # ifdef CONFIG_IRQ_WORK DECLARE_IDTENTRY_SYSVEC (IRQ_WORK_VECTOR , sysvec_irq_work); +# else +# define fred_sysvec_irq_work NULL# endif #endif
@@ -675,12 +726,16 @@ DECLARE_IDTENTRY_SYSVEC(IRQ_WORK_VECTOR, sysvec_irq_work);DECLARE_IDTENTRY_SYSVEC (POSTED_INTR_VECTOR , sysvec_kvm_posted_intr_ipi);DECLARE_IDTENTRY_SYSVEC (POSTED_INTR_WAKEUP_VECTOR , sysvec_kvm_posted_intr_wakeup_ipi);DECLARE_IDTENTRY_SYSVEC (POSTED_INTR_NESTED_VECTOR , sysvec_kvm_posted_intr_nested_ipi); +#else +# define fred_sysvec_kvm_posted_intr_ipi NULL +# define fred_sysvec_kvm_posted_intr_wakeup_ipi NULL +# define fred_sysvec_kvm_posted_intr_nested_ipi NULL#endif #if IS_ENABLED (CONFIG_HYPERV )DECLARE_IDTENTRY_SYSVEC (HYPERVISOR_CALLBACK_VECTOR , sysvec_hyperv_callback);DECLARE_IDTENTRY_SYSVEC (HYPERV_REENLIGHTENMENT_VECTOR , sysvec_hyperv_reenlightenment); -DECLARE_IDTENTRY_SYSVEC (HYPERV_STIMER0_VECTOR , sysvec_hyperv_stimer0); +DECLARE_IDTENTRY_SYSVEC (HYPERV_STIMER0_VECTOR , sysvec_hyperv_stimer0);#endif #if IS_ENABLED (CONFIG_ACRN_GUEST )
Signature · 1 line
2.34.1
Borislav Petkov
maintainerRe: [PATCH v12 24/37] x86/idtentry: Incorporate definitions/declarations of the FRED entries
Quoted 9 lines · Xin Li
FRED and IDT can share most of the definitions and declarations so that in the majority of cases the actual handler implementation is the same.
The differences are the exceptions where FRED stores exception related information on the stack and the sysvec implementations as FRED can handle irqentry/exit() in the dispatcher instead of having it in each handler.
Quoted 6 lines
Also add stub defines for vectors which are not used due to
Kconfigdecisions to spare the ifdeffery in the actual FRED dispatch code.Tested-by Shan Kang <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Xin Li <[email protected]>
This makes me wonder too who the author is. The commit message text sounds like tglx. :)
arch/x86/include/asm/idtentry.h static __always_inline void __##func(struct pt_regs *regs, \ · line 145
Quoted 6 lines
arch/x86/include/asm/idtentry.h +2
-0@@ -137,6 +141,17 @@ static __always_inline void __##func(struct pt_regs *regs, \#define DEFINE_IDTENTRY_RAW (func) \ __visible noinstr void func(struct pt_regs *regs) +/** +* DEFINE_FREDENTRY_RAW - Emit code for raw FRED entry points
LOL, "FREDENTRY"
...
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquetteRE: [PATCH v12 24/37] x86/idtentry: Incorporate definitions/declarations of the FRED entries
Quoted 12 lines
Quoted 12 lines
FRED and IDT can share most of the definitions and declarations so that in the majority of cases the actual handler implementation is the same.
The differences are the exceptions where FRED stores exception related information on the stack and the sysvec implementations as FRED can handle irqentry/exit() in the dispatcher instead of having it in each handler.
Also add stub defines for vectors which are not used due to
Kconfigdecisions to spare the ifdeffery in the actual FRED dispatch code.
Quoted 6 lines
Quoted 3 lines
Tested-by Shan Kang <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Xin Li <[email protected]>This makes me wonder too who the author is. The commit message text sounds like tglx. :)
You have a very good sense 😊
This is mostly from his review comments and suggestions on my original changes to IDTENTRY. So probably I should put a "Suggested-by" instead of "Signed-off-by" as HPA pointed out!
Thanks!
Xin
H. Peter Anvin
maintainerRE: [PATCH v12 24/37] x86/idtentry: Incorporate definitions/declarations of the FRED entries
Quoted 19 lines · On November 28, 2023 10:39:01 AM PST, "Li, Xin3" <[email protected]> wrote:
Quoted 12 lines
Quoted 12 lines
FRED and IDT can share most of the definitions and declarations so that in the majority of cases the actual handler implementation is the same.
The differences are the exceptions where FRED stores exception related information on the stack and the sysvec implementations as FRED can handle irqentry/exit() in the dispatcher instead of having it in each handler.
Also add stub defines for vectors which are not used due to
Kconfigdecisions to spare the ifdeffery in the actual FRED dispatch code.Quoted 6 lines
Quoted 3 lines
Tested-by Shan Kang <[email protected]>Signed-off-by Thomas Gleixner <[email protected]>Signed-off-by Xin Li <[email protected]>This makes me wonder too who the author is. The commit message text sounds like tglx. :)
Quoted 9 lines
You have a very good sense 😊
This is mostly from his review comments and suggestions on my original changes to IDTENTRY. So probably I should put a "Suggested-by" instead of "Signed-off-by" as HPA pointed out!
Thanks!
Xin
Remember that Signed-off-by: relates to the *patch flow*.
Borislav Petkov
maintainerRe: [PATCH v12 24/37] x86/idtentry: Incorporate definitions/declarations of the FRED entries
Quoted 1 line · H. Peter Anvin
Quoted 1 line
You have a very good sense 😊
I've been reading code of a couple of people for a couple of years. :-)
Quoted 1 line
Remember that Signed-off-by: relates to the *patch flow*.
Yes, you should take the time to read Documentation/process/ and especially submitting-patches.rst.
We have it all written down and I point new(-er) people at that directory at least once a week. :-\
Signature · 4 lines
Regards/Gruss,
Boris.
https://people.kernel.org/tglx/notes-about-netiquettePATCH v12 25/37 x86/fred: Add a debug fault entry stub for FRED
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
When occurred on different ring level, i.e., from user or kernel context, #DB needs to be handled on different stack: User #DB on current task stack, while kernel #DB on a dedicated stack. This is exactly how FRED event delivery invokes an exception handler: ring 3 event on level 0 stack, i.e., current task stack; ring 0 event on the #DB dedicated stack specified in the IA32_FRED_STKLVLS MSR. So unlike IDT, the FRED debug exception entry stub doesn't do stack switch.
On a FRED system, the debug trap status information (DR6) is passed on the stack, to avoid the problem of transient state. Furthermore, FRED transitions avoid a lot of ugly corner cases the handling of which can, and should be, skipped.
The FRED debug trap status information saved on the stack differs from DR6 in both stickiness and polarity; it is exactly in the format which debug_read_clear_dr6() returns for the IDT entry points.
---
Changes since v9:
* Disable #DB to avoid endless recursion and stack overflow when a watchpoint/breakpoint is set in the code path which is executed by #DB handler (Thomas Gleixner).
Changes since v1:
* call irqentry_nmi_{enter,exit}() in both IDT and FRED debug fault kernel handler (Peter Zijlstra).
--- arch/x86/kernel/traps.c | 43 ++++++++++++++++++++++++++++++++++++----- 1 file changed, 38 insertions(+), 5 deletions(-)
arch/x86/kernel/traps.c +38 -5
@@ -50,6 +50,7 @@#include <asm/ftrace.h> #include <asm/traps.h> #include <asm/desc.h> +#include <asm/fred.h> #include <asm/fpu/api.h> #include <asm/cpu.h> #include <asm/cpu_entry_area.h>
@@ -934,8 +935,7 @@ static bool notify_debug(struct pt_regs *regs, unsigned long *dr6)return false; } -static __always_inline void exc_debug_kernel(struct pt_regs *regs, -unsigned long dr6) +static noinstr void exc_debug_kernel(struct pt_regs *regs, unsigned long dr6) {/* * Disable breakpoints during exception handling; recursive exceptions
@@ -947,6 +947,11 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs,* * Entry text is excluded for HW_BP_X and cpu_entry_area, which * includes the entry stack is excluded for everything. +* +* For FRED, nested #DB should just work fine. But when a watchpoint or +* breakpoint is set in the code path which is executed by #DB handler, +* it results in an endless recursion and stack overflow. Thus we stay +* with the IDT approach, i.e., save DR7 and disable #DB. */ unsigned long dr7 =local_db_save ();irqentry_state_t irq_state =irqentry_nmi_enter (regs);
@@ -976,7 +981,8 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs,* Catch SYSENTER with TF set and clear DR_STEP. If this hit a * watchpoint at the same time then that will still be handled. */ -if ((dr6 &DR_STEP ) &&is_sysenter_singlestep (regs)) +if (!cpu_feature_enabled (X86_FEATURE_FRED ) && + (dr6 &DR_STEP ) &&is_sysenter_singlestep (regs)) dr6 &= ~DR_STEP ;/*
@@ -1008,8 +1014,7 @@ static __always_inline void exc_debug_kernel(struct pt_regs *regs,local_db_restore (dr7); } -static __always_inline void exc_debug_user(struct pt_regs *regs, -unsigned long dr6) +static noinstr void exc_debug_user(struct pt_regs *regs, unsigned long dr6) {bool icebp;
@@ -1093,6 +1098,34 @@ DEFINE_IDTENTRY_DEBUG_USER(exc_debug) {exc_debug_user (regs,debug_read_clear_dr6 ()); } + +#ifdef CONFIG_X86_FRED +/* +* When occurred on different ring level, i.e., from user or kernel +* context, #DB needs to be handled on different stack: User #DB on +* current task stack, while kernel #DB on a dedicated stack. +* +* This is exactly how FRED event delivery invokes an exception +* handler: ring 3 event on level 0 stack, i.e., current task stack; +* ring 0 event on the #DB dedicated stack specified in the +* IA32_FRED_STKLVLS MSR. So unlike IDT, the FRED debug exception +* entry stub doesn't do stack switch. +*/ +DEFINE_FREDENTRY_DEBUG (exc_debug) +{ +/* +* FRED #DB stores DR6 on the stack in the format which +* debug_read_clear_dr6() returns for the IDT entry points. +*/ +unsigned long dr6 =fred_event_data (regs); + +if (user_mode (regs)) +exc_debug_user (regs, dr6); +else +exc_debug_kernel (regs, dr6); +} +#endif /* CONFIG_X86_FRED */ +#else /* 32 bit does not have separate entry points. */ DEFINE_IDTENTRY_RAW (exc_debug)
Signature · 1 line
2.34.1
PATCH v12 26/37 x86/fred: Add a NMI entry stub for FRED
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
On a FRED system, NMIs nest both with themselves and faults, transient information is saved into the stack frame, and NMI unblocking only happens when the stack frame indicates that so should happen.
Thus, the NMI entry stub for FRED is really quite small...
--- arch/x86/kernel/nmi.c | 28 ++++++++++++++++++++++++++++ 1 file changed, 28 insertions(+)
arch/x86/kernel/nmi.c +28 -0
@@ -34,6 +34,7 @@#include <asm/cache.h> #include <asm/nospec-branch.h> #include <asm/sev.h> +#include <asm/fred.h> #define CREATE_TRACE_POINTS #include <trace/events/nmi.h>
@@ -643,6 +644,33 @@ void nmi_backtrace_stall_check(const struct cpumask *btp)#endif +#ifdef CONFIG_X86_FRED +/* +* With FRED, CR2/DR6 is pushed to #PF/#DB stack frame during FRED +* event delivery, i.e., there is no problem of transient states. +* And NMI unblocking only happens when the stack frame indicates +* that so should happen. +* +* Thus, the NMI entry stub for FRED is really straightforward and +* as simple as most exception handlers. As such, #DB is allowed +* during NMI handling. +*/ +DEFINE_FREDENTRY_NMI (exc_nmi) +{ +irqentry_state_t irq_state; + +if (IS_ENABLED (CONFIG_SMP ) &&arch_cpu_is_offline (smp_processor_id ())) +return ; + + irq_state =irqentry_nmi_enter (regs); + +inc_irq_stat (__nmi_count); +default_do_nmi (regs); + +irqentry_nmi_exit (regs, irq_state); +} +#endif +void stop_nmi (void ) { ignore_nmis++;
Signature · 1 line
2.34.1
PATCH v12 27/37 x86/fred: Add a machine check entry stub for FRED
In reply to Xin LiLike #DB, when occurred on different ring level, i.e., from user or kernel context, #MCE needs to be handled on different stack: User #MCE on current task stack, while kernel #MCE on a dedicated stack.
This is exactly how FRED event delivery invokes an exception handler: ring 3 event on level 0 stack, i.e., current task stack; ring 0 event on the #MCE dedicated stack specified in the IA32_FRED_STKLVLS MSR. So unlike IDT, the FRED machine check entry stub doesn't do stack switch.
---
Changes since v5:
* Disallow #DB inside #MCE for robustness sake (Peter Zijlstra).
--- arch/x86/kernel/cpu/mce/core.c | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+)
arch/x86/kernel/cpu/mce/core.c +26 -0
@@ -52,6 +52,7 @@#include <asm/mce.h> #include <asm/msr.h> #include <asm/reboot.h> +#include <asm/fred.h> #include "internal.h"
@@ -2144,6 +2145,31 @@ DEFINE_IDTENTRY_MCE_USER(exc_machine_check)exc_machine_check_user (regs); local_db_restore(dr7); } + +#ifdef CONFIG_X86_FRED +/* +* When occurred on different ring level, i.e., from user or kernel +* context, #MCE needs to be handled on different stack: User #MCE +* on current task stack, while kernel #MCE on a dedicated stack. +* +* This is exactly how FRED event delivery invokes an exception +* handler: ring 3 event on level 0 stack, i.e., current task stack; +* ring 0 event on the #MCE dedicated stack specified in the +* IA32_FRED_STKLVLS MSR. So unlike IDT, the FRED machine check entry +* stub doesn't do stack switch. +*/ +DEFINE_FREDENTRY_MCE (exc_machine_check) +{ +unsigned long dr7; + + dr7 =local_db_save (); +if (user_mode (regs)) +exc_machine_check_user (regs); +else +exc_machine_check_kernel (regs); +local_db_restore (dr7); +} +#endif #else /* 32bit unified entry point */ DEFINE_IDTENTRY_RAW (exc_machine_check)
Signature · 1 line
2.34.1
PATCH v12 28/37 x86/fred: FRED entry/exit and dispatch code
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
The code to actually handle kernel and event entry/exit using FRED. It is split up into two files thus:
- entry_64_fred.S contains the actual entrypoints and exit code, and saves and restores registers.
- entry_fred.c contains the two-level event dispatch code for FRED. The first-level dispatch is on the event type, and the second-level is on the event vector.
---
Changes since v10:
* Replace "IS_ENABLED(CONFIG_IA32_EMULATION)" with the new ia32_enabled() API (Nikolay Borisov).
Changes since v9:
* Don't use jump tables, indirect jumps are expensive (Thomas Gleixner).
* Except NMI/#DB/#MCE, FRED really can share the exception handlers with IDT (Thomas Gleixner).
* Avoid the sysvec_* idt_entry muck, do it at a central place, reuse code instead of blindly copying it, which breaks the performance optimized sysvec entries like reschedule_ipi (Thomas Gleixner).
* Add asm_ prefix to FRED asm entry points (Thomas Gleixner).
Changes since v8:
* Don't do syscall early out in fred_entry_from_user() before there are proper performance numbers and justifications (Thomas Gleixner).
* Add the control exception handler to the FRED exception handler table (Thomas Gleixner).
* Add ENDBR to the FRED_ENTER asm macro.
* Reflect the FRED spec 5.0 change that ERETS and ERETU add 8 to %rsp before popping the return context from the stack.
Changes since v1:
* Initialize a FRED exception handler to fred_bad_event() instead of NULL if no FRED handler defined for an exception vector (Peter Zijlstra).
* Push calling irqentry_{enter,exit}() and instrumentation_{begin,end}() down into individual FRED exception handlers, instead of in the dispatch framework (Peter Zijlstra).
--- arch/x86/entry/Makefile | 5 +- arch/x86/entry/entry_64_fred.S | 52 ++++++ arch/x86/entry/entry_fred.c | 230 ++++++++++++++++++++++++++ arch/x86/include/asm/asm-prototypes.h | 1 + arch/x86/include/asm/fred.h | 6 + 5 files changed, 293 insertions(+), 1 deletion(-) create mode 100644 arch/x86/entry/entry_64_fred.S create mode 100644 arch/x86/entry/entry_fred.c
arch/x86/entry/Makefile +4 -1
@@ -18,6 +18,9 @@ obj-y += vdso/ obj-y += vsyscall/ obj-$(CONFIG_PREEMPTION) += thunk_$(BITS).o +CFLAGS_entry_fred.o += -fno-stack-protector +CFLAGS_REMOVE_entry_fred.o += -pg $(CC_FLAGS_FTRACE) +obj-$(CONFIG_X86_FRED) += entry_64_fred.o entry_fred.o + obj-$(CONFIG_IA32_EMULATION) += entry_64_compat.o syscall_32.o obj-$(CONFIG_X86_X32_ABI) += syscall_x32.o -
arch/x86/entry/entry_64_fred.S new file +52 -0
52 lines not on this page · raw message
arch/x86/entry/entry_fred.c new file +230 -0
230 lines not on this page · raw message
arch/x86/include/asm/asm-prototypes.h +1 -0
@@ -12,6 +12,7 @@#include <asm/special_insns.h> #include <asm/preempt.h> #include <asm/asm.h> +#include <asm/fred.h> #include <asm/gsseg.h> #ifndef CONFIG_X86_CMPXCHG64
arch/x86/include/asm/fred.h +6 -0
@@ -60,6 +60,12 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs)return fred_info (regs)->edata; } +void asm_fred_entrypoint_user (void ); +void asm_fred_entrypoint_kernel (void ); + +__visible void fred_entry_from_user(struct pt_regs *regs); +__visible void fred_entry_from_kernel(struct pt_regs *regs); +#else /* CONFIG_X86_FRED */ static __always_inline unsigned long fred_event_data (struct pt_regs *regs) {return 0 ; }#endif /* CONFIG_X86_FRED */
Signature · 1 line
2.34.1
PATCH v12 29/37 x86/traps: Add sysvec_install() to install a system interrupt handler
In reply to Xin LiAdd sysvec_install() to install a system interrupt handler into the IDT or the FRED system interrupt handler table.
---
Changes since v8:
* Introduce a macro sysvec_install() to derive the asm handler name from a C handler, which simplifies the code and avoids an ugly typecast (Thomas Gleixner).
--- arch/x86/entry/entry_fred.c | 14 ++++++++++++++ arch/x86/include/asm/desc.h | 2 -- arch/x86/include/asm/idtentry.h | 15 +++++++++++++++ arch/x86/kernel/cpu/acrn.c | 4 ++-- arch/x86/kernel/cpu/mshyperv.c | 15 +++++++-------- arch/x86/kernel/idt.c | 4 ++-- arch/x86/kernel/kvm.c | 2 +- drivers/xen/events/events_base.c | 2 +- 8 files changed, 42 insertions(+), 16 deletions(-)
arch/x86/entry/entry_fred.c +14 -0
@@ -126,6 +126,20 @@ static idtentry_t sysvec_table[NR_SYSTEM_VECTORS] __ro_after_init = { SYSVEC(POSTED_INTR_NESTED_VECTOR, kvm_posted_intr_nested_ipi), }; +static bool fred_setup_done__initdata ; + +void __init fred_install_sysvec (unsigned int sysvec,idtentry_t handler) +{ +if (WARN_ON_ONCE (sysvec <FIRST_SYSTEM_VECTOR )) +return ; + +if (WARN_ON_ONCE (fred_setup_done)) +return ; + +if (!WARN_ON_ONCE (sysvec_table[sysvec -FIRST_SYSTEM_VECTOR ])) + sysvec_table[sysvec -FIRST_SYSTEM_VECTOR ] = handler; +} + static noinstr void fred_extint(struct pt_regs *regs) {unsigned int vector = regs->fred_ss.vector;
arch/x86/include/asm/desc.h +0 -2
@@ -402,8 +402,6 @@ static inline void set_desc_limit(struct desc_struct *desc, unsigned long limit) desc->limit1 = (limit >>16 ) &0xf ; } -void alloc_intr_gate (unsigned int n,const void *addr); -static inline void init_idt_data (struct idt_data *data,unsigned int n,const void *addr) {
arch/x86/include/asm/idtentry.h +15 -0
@@ -459,6 +459,21 @@ __visible noinstr void func(struct pt_regs *regs, \#define DEFINE_FREDENTRY_DEBUG DEFINE_FREDENTRY_RAW#endif +void idt_install_sysvec (unsigned int n,const void *function); + +#ifdef CONFIG_X86_FRED +void fred_install_sysvec (unsigned int vector,const idtentry_t function); +#else +static inline void fred_install_sysvec (unsigned int vector,const idtentry_t function) { } +#endif + +#define sysvec_install (vector, function) { \ + if (cpu_feature_enabled(X86_FEATURE_FRED)) \ + fred_install_sysvec(vector, function); \ + else \ + idt_install_sysvec(vector, asm_##function); \ +} +#else /* !__ASSEMBLY__ */ /*
arch/x86/kernel/cpu/acrn.c +2 -2
@@ -26,8 +26,8 @@ static u32 __init acrn_detect(void)static void __init acrn_init_platform (void ) { -/* Setup the IDT for ACRN hypervisor callback */ -alloc_intr_gate (HYPERVISOR_CALLBACK_VECTOR , asm_sysvec_acrn_hv_callback); +/* Install system interrupt handler for ACRN hypervisor callback */ +sysvec_install (HYPERVISOR_CALLBACK_VECTOR , sysvec_acrn_hv_callback); x86_platform.calibrate_tsc = acrn_get_tsc_khz; x86_platform.calibrate_cpu = acrn_get_tsc_khz;
arch/x86/kernel/cpu/mshyperv.c +7 -8
@@ -536,19 +536,18 @@ static void __init ms_hyperv_init_platform(void)*/ x86_platform.apic_post_init = hyperv_init;hyperv_setup_mmu_ops (); -/* Setup the IDT for hypervisor callback */ -alloc_intr_gate (HYPERVISOR_CALLBACK_VECTOR , asm_sysvec_hyperv_callback); -/* Setup the IDT for reenlightenment notifications */ +/* Install system interrupt handler for hypervisor callback */ +sysvec_install (HYPERVISOR_CALLBACK_VECTOR , sysvec_hyperv_callback); + +/* Install system interrupt handler for reenlightenment notifications */ if (ms_hyperv.features &HV_ACCESS_REENLIGHTENMENT ) { -alloc_intr_gate (HYPERV_REENLIGHTENMENT_VECTOR , - asm_sysvec_hyperv_reenlightenment); +sysvec_install (HYPERV_REENLIGHTENMENT_VECTOR , sysvec_hyperv_reenlightenment); } -/* Setup the IDT for stimer0 */ +/* Install system interrupt handler for stimer0 */ if (ms_hyperv.misc_features &HV_STIMER_DIRECT_MODE_AVAILABLE ) { -alloc_intr_gate (HYPERV_STIMER0_VECTOR , - asm_sysvec_hyperv_stimer0); +sysvec_install (HYPERV_STIMER0_VECTOR , sysvec_hyperv_stimer0); }# ifdef CONFIG_SMP
arch/x86/kernel/idt.c +2 -2
@@ -337,7 +337,7 @@ void idt_invalidate(void)load_idt (&idt); } -void __init alloc_intr_gate (unsigned int n,const void *addr) +void __init idt_install_sysvec (unsigned int n,const void *function) {if (WARN_ON (n <FIRST_SYSTEM_VECTOR ))return ;
@@ -346,5 +346,5 @@ void __init alloc_intr_gate(unsigned int n, const void *addr)return ;if (!WARN_ON (test_and_set_bit (n, system_vectors))) -set_intr_gate (n, addr); +set_intr_gate (n, function); }
arch/x86/kernel/kvm.c +1 -1
@@ -829,7 +829,7 @@ static void __init kvm_guest_init(void)if (kvm_para_has_feature (KVM_FEATURE_ASYNC_PF_INT ) && kvmapf) {static_branch_enable (&kvm_async_pf_enabled); -alloc_intr_gate (HYPERVISOR_CALLBACK_VECTOR , asm_sysvec_kvm_asyncpf_interrupt); +sysvec_install (HYPERVISOR_CALLBACK_VECTOR , sysvec_kvm_asyncpf_interrupt); }#ifdef CONFIG_SMP
drivers/xen/events/events_base.c +1 -1
@@ -2226,7 +2226,7 @@ static __init void xen_alloc_callback_vector(void)return ;pr_info ("Xen HVM callback vector for event delivery is enabled\n" ); - alloc_intr_gate(HYPERVISOR_CALLBACK_VECTOR, asm_sysvec_xen_hvm_callback); + sysvec_install(HYPERVISOR_CALLBACK_VECTOR, sysvec_xen_hvm_callback); }#else void xen_setup_callback_vector (void ) {}
Signature · 1 line
2.34.1
PATCH v12 30/37 x86/fred: Let ret_from_fork_asm() jmp to asm_fred_exit_user when FRED is enabled
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Let ret_from_fork_asm() jmp to asm_fred_exit_user when FRED is enabled, otherwise the existing IDT code is chosen.
--- arch/x86/entry/entry_64.S | 6 ++++++ arch/x86/entry/entry_64_fred.S | 1 + 2 files changed, 7 insertions(+)
arch/x86/entry/entry_64.S +6 -0
@@ -309,7 +309,13 @@ SYM_CODE_START(ret_from_fork_asm) * and unwind should work normally. */ UNWIND_HINT_REGS + +#ifdef CONFIG_X86_FRED + ALTERNATIVE "jmp swapgs_restore_regs_and_return_to_usermode", \ + "jmp asm_fred_exit_user", X86_FEATURE_FRED +#else jmp swapgs_restore_regs_and_return_to_usermode +#endif SYM_CODE_END(ret_from_fork_asm) .popsection
arch/x86/entry/entry_64_fred.S +1 -0
@@ -32,6 +32,7 @@ SYM_CODE_START_NOALIGN(asm_fred_entrypoint_user) FRED_ENTER call fred_entry_from_user +SYM_INNER_LABEL(asm_fred_exit_user, SYM_L_GLOBAL) FRED_EXIT ERETU SYM_CODE_END(asm_fred_entrypoint_user)
Signature · 1 line
2.34.1
PATCH v12 31/37 x86/fred: Fixup fault on ERETU by jumping to fred_entrypoint_user
In reply to Xin LiIf the stack frame contains an invalid user context (e.g. due to invalid SS, a non-canonical RIP, etc.) the ERETU instruction will trap (#SS or #GP).
From a Linux point of view, this really should be considered a user space failure, so use the standard fault fixup mechanism to intercept the fault, fix up the exception frame, and redirect execution to fred_entrypoint_user. The end result is that it appears just as if the hardware had taken the exception immediately after completing the transition to user space.
---
Changes since v8:
* Reflect the FRED spec 5.0 change that ERETS and ERETU add 8 to %rsp before popping the return context from the stack.
Changes since v6:
* Add a comment to explain why it is safe to write to the previous FRED stack frame. (Lai Jiangshan).
Changes since v5:
* Move the NMI bit from an invalid stack frame, which caused ERETU to fault, to the fault handler's stack frame, thus to unblock NMI ASAP if NMI is blocked (Lai Jiangshan).
--- arch/x86/entry/entry_64_fred.S | 5 +- arch/x86/include/asm/extable_fixup_types.h | 4 +- arch/x86/mm/extable.c | 79 ++++++++++++++++++++++ 3 files changed, 86 insertions(+), 2 deletions(-)
arch/x86/entry/entry_64_fred.S +4 -1
@@ -3,6 +3,7 @@ * The actual FRED entry points. */ +#include <asm/asm.h> #include <asm/fred.h> #include "calling.h"
@@ -34,7 +35,9 @@ SYM_CODE_START_NOALIGN(asm_fred_entrypoint_user) call fred_entry_from_user SYM_INNER_LABEL(asm_fred_exit_user, SYM_L_GLOBAL) FRED_EXIT - ERETU +1: ERETU + + _ASM_EXTABLE_TYPE(1b, asm_fred_entrypoint_user, EX_TYPE_ERETU) SYM_CODE_END(asm_fred_entrypoint_user) .fill asm_fred_entrypoint_kernel - ., 1, 0xcc
arch/x86/include/asm/extable_fixup_types.h +3 -1
@@ -64,6 +64,8 @@#define EX_TYPE_UCOPY_LEN4 (EX_TYPE_UCOPY_LEN | EX_DATA_IMM(4))#define EX_TYPE_UCOPY_LEN8 (EX_TYPE_UCOPY_LEN | EX_DATA_IMM(8)) -#define EX_TYPE_ZEROPAD 20/* longword load with zeropad on fault */ +#define EX_TYPE_ZEROPAD 20/* longword load with zeropad on fault */ + +#define EX_TYPE_ERETU 21#endif
arch/x86/mm/extable.c +79 -0
97 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 32/37 x86/entry/calling: Allow PUSH_AND_CLEAR_REGS being used beyond actual entry code
In reply to Xin LiFrom: "Peter Zijlstra (Intel)" <[email protected]>
PUSH_AND_CLEAR_REGS could be used besides actual entry code; in that case %rbp shouldn't be cleared (otherwise the frame pointer is destroyed) and UNWIND_HINT shouldn't be added.
--- arch/x86/entry/calling.h | 15 ++++++++++----- 1 file changed, 10 insertions(+), 5 deletions(-)
arch/x86/entry/calling.h +10 -5
@@ -65,7 +65,7 @@ For 32-bit we have the following conventions - kernel is built with* for assembly code: */ -.macro PUSH_REGS rdx=%rdx rcx=%rcx rax=%rax save_ret=0 +.macro PUSH_REGS rdx=%rdx rcx=%rcx rax=%rax save_ret=0 unwind_hint=1 .if \save_ret pushq %rsi /* pt_regs->si */ movq 8(%rsp), %rsi /* temporarily store the return address in %rsi */
@@ -87,14 +87,17 @@ For 32-bit we have the following conventions - kernel is built with pushq %r13 /* pt_regs->r13 */ pushq %r14 /* pt_regs->r14 */ pushq %r15/* pt_regs->r15 */ + + .if \unwind_hint UNWIND_HINT_REGS + .endif .if \save_ret pushq %rsi /* return address on top of stack */ .endif .endm -.macro CLEAR_REGS +.macro CLEAR_REGS clear_bp=1/* * Sanitize registers of values that a speculation attack might * otherwise want to exploit. The lower registers are likely clobbered
@@ -109,7 +112,9 @@ For 32-bit we have the following conventions - kernel is built with xorl %r10d, %r10d /* nospec r10 */ xorl %r11d, %r11d /* nospec r11 */ xorl %ebx, %ebx /* nospec rbx */ + .if \clear_bp xorl %ebp, %ebp /* nospec rbp */ + .endif xorl %r12d, %r12d /* nospec r12 */ xorl %r13d, %r13d /* nospec r13 */ xorl %r14d, %r14d /* nospec r14 */
@@ -117,9 +122,9 @@ For 32-bit we have the following conventions - kernel is built with .endm -.macro PUSH_AND_CLEAR_REGS rdx=%rdx rcx=%rcx rax=%rax save_ret=0 - PUSH_REGS rdx=\rdx, rcx=\rcx, rax=\rax, save_ret=\save_ret - CLEAR_REGS +.macro PUSH_AND_CLEAR_REGS rdx=%rdx rcx=%rcx rax=%rax save_ret=0 clear_bp=1 unwind_hint=1 + PUSH_REGS rdx=\rdx, rcx=\rcx, rax=\rax, save_ret=\save_ret unwind_hint=\unwind_hint + CLEAR_REGS clear_bp=\clear_bp .endm .macro POP_REGS pop_rdi=1
Signature · 1 line
2.34.1
PATCH v12 33/37 x86/entry: Add fred_entry_from_kvm() for VMX to handle IRQ/NMI
In reply to Xin LiIn IRQ/NMI induced VM exits, KVM VMX needs to execute the respective handlers, which requires the software to create a FRED stack frame, and use it to invoke the handlers. Add fred_irq_entry_from_kvm() for this job.
Export fred_entry_from_kvm() because VMX can be compiled as a module.
---
Changes since v10:
* Better explain the reason why no need to check current stack level (Paolo Bonzini).
Changes since v9:
* Shove the whole thing into arch/x86/entry/entry_64_fred.S for invoking external_interrupt() and fred_exc_nmi() (Sean Christopherson).
* Correct and improve a few comments (Sean Christopherson).
* Merge the two IRQ/NMI asm entries into one as it's fine to invoke noinstr code from regular code (Thomas Gleixner).
* Setup the long mode and NMI flags in the augmented SS field of FRED stack frame in C instead of asm (Thomas Gleixner).
* Add UNWIND_HINT_{SAVE,RESTORE} to get rid of the warning: "objtool:
asm_fred_entry_from_kvm+0x0: unreachable instruction" (Peter Zijlstra).
Changes since v8:
* Add a new macro VMX_DO_FRED_EVENT_IRQOFF for FRED instead of refactoring VMX_DO_EVENT_IRQOFF (Sean Christopherson).
* Do NOT use a trampoline, just LEA+PUSH the return RIP, PUSH the error code, and jump to the FRED kernel entry point for NMI or call external_interrupt() for IRQs (Sean Christopherson).
* Call external_interrupt() only when FRED is enabled, and convert the non-FRED handling to external_interrupt() after FRED lands (Sean Christopherson).
--- arch/x86/entry/entry_64_fred.S | 76 ++++++++++++++++++++++++++++++++++ arch/x86/entry/entry_fred.c | 14 +++++++ arch/x86/include/asm/fred.h | 18 ++++++++ 3 files changed, 108 insertions(+)
arch/x86/entry/entry_64_fred.S +76 -0
@@ -4,7 +4,9 @@ */ #include <asm/asm.h> +#include <asm/export.h> #include <asm/fred.h> +#include <asm/segment.h> #include "calling.h"
@@ -54,3 +56,77 @@ SYM_CODE_START_NOALIGN(asm_fred_entrypoint_kernel) FRED_EXIT ERETS SYM_CODE_END(asm_fred_entrypoint_kernel) + +#if IS_ENABLED(CONFIG_KVM_INTEL) +SYM_FUNC_START(asm_fred_entry_from_kvm) + push %rbp + mov %rsp, %rbp + + UNWIND_HINT_SAVE + + /* + * Both IRQ and NMI from VMX can be handled on current task stack + * because there is no need to protect from reentrancy and the call + * stack leading to this helper is effectively constant and shallow + * (relatively speaking). Do the same when FRED is active, i.e., no + * need to check current stack level for a stack switch. + * + * Emulate the FRED-defined redzone and stack alignment. + */ + sub $(FRED_CONFIG_REDZONE_AMOUNT << 6), %rsp + and $FRED_STACK_FRAME_RSP_MASK, %rsp + + /* + * Start to push a FRED stack frame, which is always 64 bytes: + * + * +--------+-----------------+ + * | Bytes | Usage | + * +--------+-----------------+ + * | 63:56 | Reserved | + * | 55:48 | Event Data | + * | 47:40 | SS + Event Info | + * | 39:32 | RSP | + * | 31:24 | RFLAGS | + * | 23:16 | CS + Aux Info | + * | 15:8 | RIP | + * | 7:0 | Error Code | + * +--------+-----------------+ + */ + push $0 /* Reserved, must be 0 */ + push $0 /* Event data, 0 for IRQ/NMI */ + push %rdi /* fred_ss handed in by the caller */ + push %rbp + pushf + mov $__KERNEL_CS, %rax + push %rax + + /* + * Unlike the IDT event delivery, FRED _always_ pushes an error code + * after pushing the return RIP, thus the CALL instruction CANNOT be + * used here to push the return RIP, otherwise there is no chance to + * push an error code before invoking the IRQ/NMI handler. + * + * Use LEA to get the return RIP and push it, then push an error code. + */ + lea 1f(%rip), %rax + push %rax /* Return RIP */ + push $0 /* Error code, 0 for IRQ/NMI */ + + PUSH_AND_CLEAR_REGS clear_bp=0 unwind_hint=0 + movq %rsp, %rdi /* %rdi -> pt_regs */ + call __fred_entry_from_kvm /* Call the C entry point */ + POP_REGS + ERETS +1: + /* + * Objtool doesn't understand what ERETS does, this hint tells it that + * yes, we'll reach here and with what stack state. A save/restore pair + * isn't strictly needed, but it's the simplest form. + */ + UNWIND_HINT_RESTORE + pop %rbp + RET + +SYM_FUNC_END(asm_fred_entry_from_kvm) +EXPORT_SYMBOL_GPL(asm_fred_entry_from_kvm); +#endif
arch/x86/entry/entry_fred.c +14 -0
@@ -242,3 +242,17 @@ __visible noinstr void fred_entry_from_kernel(struct pt_regs *regs)return fred_bad_type (regs, error_code); } } + +#if IS_ENABLED (CONFIG_KVM_INTEL ) +__visible noinstr void __fred_entry_from_kvm(struct pt_regs *regs) +{ +switch (regs->fred_ss.type) { +case EVENT_TYPE_EXTINT : +return fred_extint (regs); +case EVENT_TYPE_NMI : +return fred_exc_nmi (regs); +default : +WARN_ON_ONCE (1 ); + } +} +#endif
arch/x86/include/asm/fred.h +18 -0
@@ -9,6 +9,7 @@#include <linux/const.h> #include <asm/asm.h> +#include <asm/trapnr.h> /* * FRED event return instruction opcodes for ERET{S,U}; supported in
@@ -62,12 +63,29 @@ static __always_inline unsigned long fred_event_data(struct pt_regs *regs)void asm_fred_entrypoint_user (void );void asm_fred_entrypoint_kernel (void ); +void asm_fred_entry_from_kvm (struct fred_ss ); __visible void fred_entry_from_user(struct pt_regs *regs); __visible void fred_entry_from_kernel(struct pt_regs *regs); +__visible void __fred_entry_from_kvm(struct pt_regs *regs); + +/* Can be called from noinstr code, thus __always_inline */ +static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) +{ +struct fred_ss ss = { + .ss =__KERNEL_DS, + .type = type, + .vector = vector, + .nmi = type ==EVENT_TYPE_NMI , + .lm =1 , + }; + +asm_fred_entry_from_kvm (ss); +}#else /* CONFIG_X86_FRED */ static __always_inline unsigned long fred_event_data (struct pt_regs *regs) {return 0 ; } +static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { }#endif /* CONFIG_X86_FRED */ #endif /* !__ASSEMBLY__ */
Signature · 1 line
2.34.1
PATCH v12 34/37 KVM: VMX: Call fred_entry_from_kvm() for IRQ/NMI handling
In reply to Xin LiWhen FRED is enabled, call fred_entry_from_kvm() to handle IRQ/NMI in IRQ/NMI induced VM exits.
--- arch/x86/kvm/vmx/vmx.c | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-)
arch/x86/kvm/vmx/vmx.c +9 -3
@@ -38,6 +38,7 @@#include <asm/desc.h> #include <asm/fpu/api.h> #include <asm/fpu/xstate.h> +#include <asm/fred.h> #include <asm/idtentry.h> #include <asm/io.h> #include <asm/irq_remapping.h>
@@ -6962,14 +6963,16 @@ static void handle_external_interrupt_irqoff(struct kvm_vcpu *vcpu) {u32 intr_info =vmx_get_intr_info (vcpu);unsigned int vector = intr_info &INTR_INFO_VECTOR_MASK ; -gate_desc *desc = (gate_desc *)host_idt_base + vector;if (KVM_BUG (!is_external_intr (intr_info), vcpu->kvm,"unexpected VM-Exit interrupt info: 0x%x" , intr_info))return ;kvm_before_interrupt (vcpu,KVM_HANDLING_IRQ ); -vmx_do_interrupt_irqoff (gate_offset (desc)); +if (cpu_feature_enabled (X86_FEATURE_FRED )) +fred_entry_from_kvm (EVENT_TYPE_EXTINT , vector); +else +vmx_do_interrupt_irqoff (gate_offset ((gate_desc *)host_idt_base + vector));kvm_after_interrupt (vcpu); vcpu->arch.at_instruction_boundary = true;
@@ -7262,7 +7265,10 @@ static noinstr void vmx_vcpu_enter_exit(struct kvm_vcpu *vcpu,if ((u16 )vmx->exit_reason.basic ==EXIT_REASON_EXCEPTION_NMI &&is_nmi (vmx_get_intr_info (vcpu))) {kvm_before_interrupt (vcpu,KVM_HANDLING_NMI ); -vmx_do_nmi_irqoff (); +if (cpu_feature_enabled (X86_FEATURE_FRED )) +fred_entry_from_kvm (EVENT_TYPE_NMI ,NMI_VECTOR ); +else +vmx_do_nmi_irqoff ();kvm_after_interrupt (vcpu); }
Signature · 1 line
2.34.1
PATCH v12 35/37 x86/syscall: Split IDT syscall setup code into idt_syscall_init()
In reply to Xin LiBecause FRED uses the ring 3 FRED entrypoint for SYSCALL and SYSENTER and ERETU is the only legit instruction to return to ring 3, there is NO need to setup SYSCALL and SYSENTER MSRs for FRED, except the IA32_STAR MSR.
Split IDT syscall setup code into idt_syscall_init() to make it easy to skip syscall setup code when FRED is enabled.
--- arch/x86/kernel/cpu/common.c | 13 ++++++++++--- 1 file changed, 10 insertions(+), 3 deletions(-)
arch/x86/kernel/cpu/common.c +10 -3
@@ -2078,10 +2078,8 @@ static void wrmsrl_cstar(unsigned long val)wrmsrl (MSR_CSTAR , val); } -/* May not be marked __init: used by software suspend */ -void syscall_init (void ) +static inline void idt_syscall_init (void ) { -wrmsr (MSR_STAR ,0 , (__USER32_CS <<16 ) | __KERNEL_CS);wrmsrl (MSR_LSTAR , (unsigned long )entry_SYSCALL_64);if (ia32_enabled ()) {
@@ -2115,6 +2113,15 @@ void syscall_init(void) X86_EFLAGS_AC|X86_EFLAGS_ID); } +/* May not be marked __init: used by software suspend */ +void syscall_init (void ) +{ +/* The default user and kernel segments */ +wrmsr (MSR_STAR ,0 , (__USER32_CS <<16 ) | __KERNEL_CS); + +idt_syscall_init (); +} +#else /* CONFIG_X86_64 */ #ifdef CONFIG_STACKPROTECTOR
Signature · 1 line
2.34.1
PATCH v12 36/37 x86/fred: Add FRED initialization functions
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Add cpu_init_fred_exceptions() to:
- Set FRED entrypoints for events happening in ring 0 and 3.
- Specify the stack level for IRQs occurred ring 0.
- Specify dedicated event stacks for #DB/NMI/#MCE/#DF.
- Enable FRED and invalidtes IDT.
- Force 32-bit system calls to use "int $0x80" only.
Add fred_complete_exception_setup() to:
- Initialize system_vectors as done for IDT systems.
- Set unused sysvec_table entries to fred_handle_spurious_interrupt().
---
Changes since v9:
* Set unused sysvec table entries to fred_handle_spurious_interrupt() in fred_complete_exception_setup() (Thomas Gleixner).
Changes since v5:
* Add a comment for FRED stack level settings (Lai Jiangshan).
* Define NMI/#DB/#MCE/#DF stack levels using macros.
--- arch/x86/entry/entry_fred.c | 21 +++++++++++++ arch/x86/include/asm/fred.h | 5 ++++ arch/x86/kernel/Makefile | 1 + arch/x86/kernel/fred.c | 59 +++++++++++++++++++++++++++++++++++++ 4 files changed, 86 insertions(+) create mode 100644 arch/x86/kernel/fred.c
arch/x86/entry/entry_fred.c +21 -0
@@ -140,6 +140,27 @@ void __init fred_install_sysvec(unsigned int sysvec, idtentry_t handler) sysvec_table[sysvec -FIRST_SYSTEM_VECTOR ] = handler; } +static noinstr void fred_handle_spurious_interrupt(struct pt_regs *regs) +{ +spurious_interrupt (regs, regs->fred_ss.vector); +} + +void __init fred_complete_exception_setup (void ) +{ +unsigned int vector; + +for (vector =0 ; vector <FIRST_EXTERNAL_VECTOR ; vector++) +set_bit (vector, system_vectors); + +for (vector =0 ; vector <NR_SYSTEM_VECTORS ; vector++) { +if (sysvec_table[vector]) +set_bit (vector +FIRST_SYSTEM_VECTOR , system_vectors); +else + sysvec_table[vector] = fred_handle_spurious_interrupt; + } + fred_setup_done = true; +} + static noinstr void fred_extint(struct pt_regs *regs) {unsigned int vector = regs->fred_ss.vector;
arch/x86/include/asm/fred.h +5 -0
@@ -83,8 +83,13 @@ static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned intasm_fred_entry_from_kvm (ss); } +void cpu_init_fred_exceptions (void ); +void fred_complete_exception_setup (void ); +#else /* CONFIG_X86_FRED */ static __always_inline unsigned long fred_event_data (struct pt_regs *regs) {return 0 ; } +static inline void cpu_init_fred_exceptions (void ) { } +static inline void fred_complete_exception_setup (void ) { } static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { }#endif /* CONFIG_X86_FRED */ #endif /* !__ASSEMBLY__ */
arch/x86/kernel/Makefile +1 -0
@@ -47,6 +47,7 @@ obj-y += platform-quirks.o obj-y += process_$(BITS).o signal.o signal_$(BITS).o obj-y += traps.o idt.o irq.o irq_$(BITS).o dumpstack_$(BITS).o obj-y += time.o ioport.o dumpstack.o nmi.o +obj-$(CONFIG_X86_FRED) += fred.o obj-$(CONFIG_MODIFY_LDT_SYSCALL) += ldt.o obj-$(CONFIG_X86_KERNEL_IBT) += ibt_selftest.o obj-y += setup.o x86_init.o i8259.o irqinit.o
arch/x86/kernel/fred.c new file +59 -0
59 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 37/37 x86/fred: Invoke FRED initialization code to enable FRED
In reply to Xin LiFrom: "H. Peter Anvin (Intel)" <[email protected]>
Let cpu_init_exception_handling() call cpu_init_fred_exceptions() to initialize FRED. However if FRED is unavailable or disabled, it falls back to set up TSS IST and initialize IDT.
---
Changes since v10:
* No need to invalidate SYSCALL and SYSENTER MSRs (Thomas Gleixner).
Changes since v8:
* Move this patch after all required changes are in place (Thomas Gleixner).
--- arch/x86/kernel/cpu/common.c | 22 +++++++++++++++++----- arch/x86/kernel/irqinit.c | 7 ++++++- arch/x86/kernel/traps.c | 5 ++++- 3 files changed, 27 insertions(+), 7 deletions(-)
arch/x86/kernel/cpu/common.c +17 -5
@@ -61,6 +61,7 @@#include <asm/microcode.h> #include <asm/intel-family.h> #include <asm/cpu_device_id.h> +#include <asm/fred.h> #include <asm/uv/uv.h> #include <asm/ia32.h> #include <asm/set_memory.h>
@@ -2119,7 +2120,15 @@ void syscall_init(void)/* The default user and kernel segments */ wrmsr (MSR_STAR ,0 , (__USER32_CS <<16 ) | __KERNEL_CS); - idt_syscall_init(); +/* +* Except the IA32_STAR MSR, there is NO need to setup SYSCALL and +* SYSENTER MSRs for FRED, because FRED uses the ring 3 FRED +* entrypoint for SYSCALL and SYSENTER, and ERETU is the only legit +* instruction to return to ring 3 (both sysexit and sysret cause +* #UD when FRED is enabled). +*/ +if (!cpu_feature_enabled (X86_FEATURE_FRED )) + idt_syscall_init(); }#else /* CONFIG_X86_64 */
@@ -2235,8 +2244,9 @@ void cpu_init_exception_handling(void)/* paranoid_entry() gets the CPU number from the GDT */ setup_getcpu (cpu); -/* IST vectors need TSS to be set up. */ -tss_setup_ist (tss); +/* For IDT mode, IST vectors need to be set in TSS. */ +if (!cpu_feature_enabled (X86_FEATURE_FRED )) +tss_setup_ist (tss);tss_setup_io_bitmap (tss);set_tss_desc (cpu, &get_cpu_entry_area (cpu)->tss.x86_tss);
@@ -2245,8 +2255,10 @@ void cpu_init_exception_handling(void)/* GHCB needs to be setup to handle #VC. */ setup_ghcb (); -/* Finally load the IDT */ -load_current_idt (); +if (cpu_feature_enabled (X86_FEATURE_FRED )) +cpu_init_fred_exceptions (); +else +load_current_idt (); }/*
arch/x86/kernel/irqinit.c +6 -1
@@ -28,6 +28,7 @@#include <asm/setup.h> #include <asm/i8259.h> #include <asm/traps.h> +#include <asm/fred.h> #include <asm/prom.h> /*
@@ -96,7 +97,11 @@ void __init native_init_IRQ(void)/* Execute any quirks before the call gates are initialised: */ x86_init.irqs.pre_vector_init (); -idt_setup_apic_and_irq_gates (); +if (cpu_feature_enabled (X86_FEATURE_FRED )) +fred_complete_exception_setup (); +else +idt_setup_apic_and_irq_gates (); +lapic_assign_system_vectors ();if (!acpi_ioapic && !of_ioapic &&nr_legacy_irqs ()) {
arch/x86/kernel/traps.c +4 -1
@@ -1411,7 +1411,10 @@ void __init trap_init(void)/* Initialize TSS before setting up traps so ISTs work */ cpu_init_exception_handling (); +/* Setup traps as cpu_init() might #GP */ -idt_setup_traps (); +if (!cpu_feature_enabled (X86_FEATURE_FRED )) +idt_setup_traps (); +cpu_init (); }
Signature · 1 line
2.34.1
Quoted 3 lines · Xin Li
This patch set enables the Intel flexible return and event delivery (FRED) architecture for x86-64.
Which tree is this based on now? I tried running 'b4 diff' but it complains it can't find some context around arch/x86/kernel/cpu/common.c . I have the tip tree updated.
Quoted 1 line
Quoted 7 lines
Quoted 3 lines
This patch set enables the Intel flexible return and event delivery (FRED) architecture for x86-64.
Which tree is this based on now?
It was based on the tip master on the day I sent the v12 patch set, i.e., Monday night in the US west coast.
Quoted 3 lines
I tried running 'b4 diff' but it complains it can't find some context around
arch/x86/kernel/cpu/common.c.
What does it complain?
Quoted 1 line
I have the tip tree updated.
I just did a rebase on the latest tip master, and did find a build bug caused by:
[PATCH v12 33/37] x86/entry: Add fred_entry_from_kvm() for VMX to handle IRQ/NMI
as <asm/export.h> is deprecated by <linux/export.h> yesterday.
After I replaced it, it compiles and boots. What should I do now?
Thanks!
Xin