Skip to contentSkip to conversation

Enable FRED earlier

PATCH Applied Xin Li (Intel) 4 participants 15 messages Jul 9, 2024 – Aug 13, 2024

Subsystems (2)
lore
Share & tools
lore

Xin Li (Intel)

author maintainer
#0 lore raw

v2 cover Enable FRED earlier

Wenlong Hou from Ant group reported two problems during the FRED initialization:
https://lore.kernel.org/lkml/[email protected]/

The first problem is that spurious_interrupt() gets called on the HYPERVISOR_CALLBACK_VECTOR vector. Because kvm_guest_init(), being executed way before trap_init() in which it is decided that whether FRED will be enabled or not, calls sysvec_install() to install HYPERVISOR_CALLBACK_VECTOR's interrupt handler into FRED system vector dispatch table or IDT depending on whether FRED is enabled.

The other problem is that the #PF handler gets a wrong faulting address from the stack instead of CR2 before FRED is enabled. Because the #PF handler fetches its faulting addresss from the stack or CR2 based on whether FRED is available rather than active.

This patchset fixes the 2 problems with suggestions from tglx:

1) Parse cmdline param "fred=" in cpu_parse_early_param() to minimize the gap mentioned above, before kvm_guest_init().

2) Enable FRED right after init_mem_mapping() to switch to FRED from early IDT ASAP, avoid intermediately using the IDT #PF handler.

Link to v1:
https://lore.kernel.org/lkml/[email protected]/

Changes since v1:
* Drop the patch that changes wrmsrl() to wrmsrns().
* Use strncmp() instead of strcmp().

Xin Li (Intel) (3):
x86/fred: Parse cmdline param "fred=" in cpu_parse_early_param() x86/fred: Split FRED RSP initialization into a separate function x86/fred: Enable FRED right after init_mem_mapping()

 arch/x86/include/asm/fred.h  |  2 ++
 arch/x86/kernel/cpu/common.c |  9 ++++++---
 arch/x86/kernel/fred.c       | 28 +++++++++++++++++++---------
 arch/x86/kernel/setup.c      | 11 ++++++++++-
 arch/x86/kernel/smpboot.c    |  6 ++++++
 arch/x86/kernel/traps.c      | 30 ++++--------------------------
 6 files changed, 47 insertions(+), 39 deletions(-)

base-commit: aa9d8caba6e40b0b02a6f2b5f2bd9177cd76cacf

Signature · 1 line
2.45.2
author maintainer
#1 lore raw

PATCH v2 1/3 x86/fred: Parse cmdline param "fred=" in cpu_parse_early_param()

Depending on whether FRED will be used, sysvec_install() installs a system interrupt handler into FRED system vector dispatch table or IDT. However FRED can be disabled later in trap_init(), after sysvec_install() is called. E.g., the HYPERVISOR_CALLBACK_VECTOR handler is registered with sysvec_install() in kvm_guest_init(), which is called in setup_arch() but way before trap_init(). IOW, there is a gap between FRED is available and available but disabled. As a result, when FRED is available but disabled, its IDT handler is not installed thus spurious_interrupt() will be invoked.

Fix it by parsing cmdline param "fred=" in cpu_parse_early_param() to minimize the gap between FRED is available and available but disabled.

Fixes 3810da12710a ("x86/fred: Add a fred= cmdline param")

---

Change since v1:
* Use strncmp() instead of strcmp().

---
 arch/x86/kernel/cpu/common.c |  5 +++++
 arch/x86/kernel/traps.c      | 26 --------------------------
 2 files changed, 5 insertions(+), 26 deletions(-)
arch/x86/kernel/cpu/common.c +5 -0
index d4e539d4e158..10a5402d8297 100644
@@ -1510,6 +1510,11 @@ static void __init cpu_parse_early_param(void)
 	if (cmdline_find_option_bool(boot_command_line, "nousershstk"))
 		setup_clear_cpu_cap(X86_FEATURE_USER_SHSTK);
 
+	/* Minimize the gap between FRED is available and available but disabled. */
+	arglen = cmdline_find_option(boot_command_line, "fred", arg, sizeof(arg));
+	if (arglen != 2 || strncmp(arg, "on", 2))
+		setup_clear_cpu_cap(X86_FEATURE_FRED);
+
 	arglen = cmdline_find_option(boot_command_line, "clearcpuid", arg, sizeof(arg));
 	if (arglen <= 0)
 		return;
arch/x86/kernel/traps.c +0 -26
index 4fa0b17e5043..6afb41e6cbbb 100644
@@ -1402,34 +1402,8 @@ DEFINE_IDTENTRY_SW(iret_error)
 }
 #endif
 
-/* Do not enable FRED by default yet. */
-static bool enable_fred __ro_after_init = false;
-
-#ifdef CONFIG_X86_FRED
-static int __init fred_setup(char *str)
-{
-	if (!str)
-		return -EINVAL;
-
-	if (!cpu_feature_enabled(X86_FEATURE_FRED))
-		return 0;
-
-	if (!strcmp(str, "on"))
-		enable_fred = true;
-	else if (!strcmp(str, "off"))
-		enable_fred = false;
-	else
-		pr_warn("invalid FRED option: 'fred=%s'\n", str);
-	return 0;
-}
-early_param("fred", fred_setup);
-#endif
-
 void __init trap_init(void)
 {
-	if (cpu_feature_enabled(X86_FEATURE_FRED) && !enable_fred)
-		setup_clear_cpu_cap(X86_FEATURE_FRED);
-
 	/* Init cpu_entry_area before IST entries are set up */
 	setup_cpu_entry_areas();
 
Signature · 1 line
2.45.2
#2 lore raw

Re: [PATCH v2 1/3] x86/fred: Parse cmdline param "fred=" in cpu_parse_early_param()

Quoted 32 lines · Xin Li (Intel)

Depending on whether FRED will be used, sysvec_install() installs a system interrupt handler into FRED system vector dispatch table or IDT. However FRED can be disabled later in trap_init(), after sysvec_install() is called. E.g., the HYPERVISOR_CALLBACK_VECTOR handler is registered with sysvec_install() in kvm_guest_init(), which is called in setup_arch() but way before trap_init(). IOW, there is a gap between FRED is available and available but disabled. As a result, when FRED is available but disabled, its IDT handler is not installed thus spurious_interrupt() will be invoked.

Fix it by parsing cmdline param "fred=" in cpu_parse_early_param() to minimize the gap between FRED is available and available but disabled.

Fixes 3810da12710a ("x86/fred: Add a fred= cmdline param")

---

Change since v1:
* Use strncmp() instead of strcmp().

---
  arch/x86/kernel/cpu/common.c |  5 +++++
  arch/x86/kernel/traps.c      | 26 --------------------------
  2 files changed, 5 insertions(+), 26 deletions(-)
arch/x86/kernel/cpu/common.c +0 -0
index d4e539d4e158..10a5402d8297 100644

arch/x86/kernel/cpu/common.c static void __init cpu_parse_early_param(void) · line 1515

Quoted 6 lines
arch/x86/kernel/cpu/common.c +3 -0
@@ -1510,6 +1510,11 @@ static void __init cpu_parse_early_param(void)
  	if (cmdline_find_option_bool(boot_command_line, "nousershstk"))
  		setup_clear_cpu_cap(X86_FEATURE_USER_SHSTK);
 
+	/* Minimize the gap between FRED is available and available but disabled. */
+	arglen = cmdline_find_option(boot_command_line, "fred", arg, sizeof(arg));
+	if (arglen != 2 || strncmp(arg, "on", 2))

I'm confused why you keep perverting the calling convention of cmdline_find_option. The doc clearly states:

* Returns the position of that @option (starts counting with 1)

* or 0 on not found. @option will only be found if it is found

* as an entire word in @cmdline. For instance, if @option="car"

* then a cmdline which contains "cart" will not match.

You should only care if arglen is non 0, which if it is you check if its value equal 'on', why bother with its starting position?

Quoted 44 lines
arch/x86/kernel/cpu/common.c +2 -0
+		setup_clear_cpu_cap(X86_FEATURE_FRED);
+
  	arglen = cmdline_find_option(boot_command_line, "clearcpuid", arg, sizeof(arg));
  	if (arglen <= 0)
  		return;
arch/x86/kernel/traps.c +0 -26
index 4fa0b17e5043..6afb41e6cbbb 100644
@@ -1402,34 +1402,8 @@ DEFINE_IDTENTRY_SW(iret_error)
  }
  #endif
 
-/* Do not enable FRED by default yet. */
-static bool enable_fred __ro_after_init = false;
-
-#ifdef CONFIG_X86_FRED
-static int __init fred_setup(char *str)
-{
-	if (!str)
-		return -EINVAL;
-
-	if (!cpu_feature_enabled(X86_FEATURE_FRED))
-		return 0;
-
-	if (!strcmp(str, "on"))
-		enable_fred = true;
-	else if (!strcmp(str, "off"))
-		enable_fred = false;
-	else
-		pr_warn("invalid FRED option: 'fred=%s'\n", str);
-	return 0;
-}
-early_param("fred", fred_setup);
-#endif
-
  void __init trap_init(void)
  {
-	if (cpu_feature_enabled(X86_FEATURE_FRED) && !enable_fred)
-		setup_clear_cpu_cap(X86_FEATURE_FRED);
-
  	/* Init cpu_entry_area before IST entries are set up */
  	setup_cpu_entry_areas();
 
author maintainer
#3 lore raw

Re: [PATCH v2 1/3] x86/fred: Parse cmdline param "fred=" in cpu_parse_early_param()

Quoted 13 lines · Nikolay Borisov
Quoted 8 lines · Xin Li (Intel)
arch/x86/kernel/cpu/common.c +1 -0
@@ -1510,6 +1510,11 @@ static void __init cpu_parse_early_param(void)
       if (cmdline_find_option_bool(boot_command_line, "nousershstk"))
           setup_clear_cpu_cap(X86_FEATURE_USER_SHSTK);
+    /* Minimize the gap between FRED is available and available but 

disabled. */
+    arglen = cmdline_find_option(boot_command_line, "fred", arg, sizeof(arg));
+    if (arglen != 2 || strncmp(arg, "on", 2))

I'm confused why you keep perverting the calling convention of cmdline_find_option. The doc clearly states:

Quoted 7 lines

    * Returns the position of that @option (starts counting with 1)
    * or 0 on not found.  @option will only be found if it is found
    * as an entire word in @cmdline.  For instance, if @option="car"
    * then a cmdline which contains "cart" will not match.

You should only care if arglen is non 0, which if it is you check if its value equal 'on', why bother with its starting position?

Well, just look at how it is used in match_option() in arch/x86/kernel/cpu/bugs.c and arch/x86/kernel/cpu/intel.c.

This is a short version and it will be expanded once we have more option strings well defined (match_option() should be a common lib function then).

#4 lore raw

Re: [PATCH v2 1/3] x86/fred: Parse cmdline param "fred=" in cpu_parse_early_param()

Quoted 14 lines · Xin Li
Quoted 13 lines · Nikolay Borisov
Quoted 8 lines · Xin Li (Intel)
arch/x86/kernel/cpu/common.c +1 -0
@@ -1510,6 +1510,11 @@ static void __init cpu_parse_early_param(void)
       if (cmdline_find_option_bool(boot_command_line, "nousershstk"))
           setup_clear_cpu_cap(X86_FEATURE_USER_SHSTK);
+    /* Minimize the gap between FRED is available and available but 

disabled. */
+    arglen = cmdline_find_option(boot_command_line, "fred", arg, sizeof(arg));
+    if (arglen != 2 || strncmp(arg, "on", 2))

I'm confused why you keep perverting the calling convention of cmdline_find_option. The doc clearly states:

Quoted 7 lines
Quoted 7 lines

     * Returns the position of that @option (starts counting with 1)
     * or 0 on not found.  @option will only be found if it is found
     * as an entire word in @cmdline.  For instance, if @option="car"
     * then a cmdline which contains "cart" will not match.

You should only care if arglen is non 0, which if it is you check if its value equal 'on', why bother with its starting position?

Actually, I have quoted the wrong doc, the correct one is:

"
Returns the length of the argument (regardless of if it was truncated to fit in the buffer), or -1 on not found. "

Quoted 3 lines

Well, just look at how it is used in match_option() in arch/x86/kernel/cpu/bugs.c and arch/x86/kernel/cpu/intel.c.

Exactly, in bugs.c it's used as I've suggested:

In spectre_v2_parse_user_cmdline it checks if spectre_v2_user is present (if a negative value is returned) and if not it returns some default.

In spectre_v2_parse_cmdline it's used exactly the same way - return some default if that function returns a negative value (spectre_v2 check) or return some specific value if it found nospectre_v2.

And in sld_state_setup the code just checks for a non-negative value i.e the argument has been found.

Otoh, I see what you are trying to say if I look at the usage of this function in arch/x86/boot/compressed/acpi.c

Still I find this convention a bit counter-intuitive, but given it's not a precedent I'm fine with leaving it as is.

Quoted 5 lines

This is a short version and it will be expanded once we have more option strings well defined (match_option() should be a common lib function then).

merge bot Applied
#5 lore raw

[tip: x86/fred] x86/fred: Parse cmdline param "fred=" in cpu_parse_early_param()

In reply to Xin Li (Intel)
The following commit has been merged into the x86/fred branch of tip:

The following commit has been merged into the x86/fred branch of tip:

Commit-ID:     989b5cfaa7b6054f4e1bde914470ee091c23e6a5
Gitweb:        https://git.kernel.org/tip/989b5cfaa7b6054f4e1bde914470ee091c23e6a5
Author:        Xin Li (Intel) <[email protected]>
AuthorDate:    Tue, 09 Jul 2024 08:40:46 -07:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Tue, 13 Aug 2024 21:59:21 +02:00

x86/fred: Parse cmdline param "fred=" in cpu_parse_early_param()

Depending on whether FRED is enabled, sysvec_install() installs a system interrupt handler into either into FRED's system vector dispatch table or into the IDT.

However FRED can be disabled later in trap_init(), after sysvec_install() has been invoked already; e.g., the HYPERVISOR_CALLBACK_VECTOR handler is registered with sysvec_install() in kvm_guest_init(), which is called in setup_arch() but way before trap_init().

IOW, there is a gap between FRED is available and available but disabled. As a result, when FRED is available but disabled, early sysvec_install() invocations fail to install the IDT handler resulting in spurious interrupts.

Fix it by parsing cmdline param "fred=" in cpu_parse_early_param() to ensure that FRED is disabled before the first sysvec_install() incovations.

---
 arch/x86/kernel/cpu/common.c |  5 +++++
 arch/x86/kernel/traps.c      | 26 --------------------------
 2 files changed, 5 insertions(+), 26 deletions(-)
arch/x86/kernel/cpu/common.c +5 -0
index d4e539d..10a5402 100644
@@ -1510,6 +1510,11 @@ static void __init cpu_parse_early_param(void)
 	if (cmdline_find_option_bool(boot_command_line, "nousershstk"))
 		setup_clear_cpu_cap(X86_FEATURE_USER_SHSTK);
 
+	/* Minimize the gap between FRED is available and available but disabled. */
+	arglen = cmdline_find_option(boot_command_line, "fred", arg, sizeof(arg));
+	if (arglen != 2 || strncmp(arg, "on", 2))
+		setup_clear_cpu_cap(X86_FEATURE_FRED);
+
 	arglen = cmdline_find_option(boot_command_line, "clearcpuid", arg, sizeof(arg));
 	if (arglen <= 0)
 		return;
arch/x86/kernel/traps.c +0 -26
index 4fa0b17..6afb41e 100644
@@ -1402,34 +1402,8 @@ DEFINE_IDTENTRY_SW(iret_error)
 }
 #endif
 
-/* Do not enable FRED by default yet. */
-static bool enable_fred __ro_after_init = false;
-
-#ifdef CONFIG_X86_FRED
-static int __init fred_setup(char *str)
-{
-	if (!str)
-		return -EINVAL;
-
-	if (!cpu_feature_enabled(X86_FEATURE_FRED))
-		return 0;
-
-	if (!strcmp(str, "on"))
-		enable_fred = true;
-	else if (!strcmp(str, "off"))
-		enable_fred = false;
-	else
-		pr_warn("invalid FRED option: 'fred=%s'\n", str);
-	return 0;
-}
-early_param("fred", fred_setup);
-#endif
-
 void __init trap_init(void)
 {
-	if (cpu_feature_enabled(X86_FEATURE_FRED) && !enable_fred)
-		setup_clear_cpu_cap(X86_FEATURE_FRED);
-
 	/* Init cpu_entry_area before IST entries are set up */
 	setup_cpu_entry_areas();
 
author maintainer
#6 lore raw

PATCH v2 2/3 x86/fred: Split FRED RSP initialization into a separate function

In reply to Xin Li (Intel)

To enable FRED earlier, split FRED RSP initialization into a separate function, as they are initialized with memory from CPU entry areas, thus their initialization has to be kept after setup_cpu_entry_areas().

No functional change intended.

---
 arch/x86/include/asm/fred.h  |  2 ++
 arch/x86/kernel/cpu/common.c |  6 ++++--
 arch/x86/kernel/fred.c       | 28 +++++++++++++++++++---------
 3 files changed, 25 insertions(+), 11 deletions(-)
arch/x86/include/asm/fred.h +2 -0
index e86c7ba32435..66d7dbe2d314 100644
@@ -84,11 +84,13 @@ static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int
 }
 
 void cpu_init_fred_exceptions(void);
+void cpu_init_fred_rsps(void);
 void fred_complete_exception_setup(void);
 
 #else /* CONFIG_X86_FRED */
 static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { return 0; }
 static inline void cpu_init_fred_exceptions(void) { }
+static inline void cpu_init_fred_rsps(void) { }
 static inline void fred_complete_exception_setup(void) { }
 static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { }
 #endif /* CONFIG_X86_FRED */
arch/x86/kernel/cpu/common.c +4 -2
index 10a5402d8297..6de12b3c1b04 100644
@@ -2195,10 +2195,12 @@ void cpu_init_exception_handling(void)
 	/* GHCB needs to be setup to handle #VC. */
 	setup_ghcb();
 
-	if (cpu_feature_enabled(X86_FEATURE_FRED))
+	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
 		cpu_init_fred_exceptions();
-	else
+		cpu_init_fred_rsps();
+	} else {
 		load_current_idt();
+	}
 }
 
 /*
arch/x86/kernel/fred.c +19 -9
index 4bcd8791ad96..99a134fcd5bf 100644
@@ -32,6 +32,25 @@ void cpu_init_fred_exceptions(void)
 	       FRED_CONFIG_INT_STKLVL(0) |
 	       FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user));
 
+	wrmsrl(MSR_IA32_FRED_STKLVLS, 0);
+	wrmsrl(MSR_IA32_FRED_RSP0, 0);
+	wrmsrl(MSR_IA32_FRED_RSP1, 0);
+	wrmsrl(MSR_IA32_FRED_RSP2, 0);
+	wrmsrl(MSR_IA32_FRED_RSP3, 0);
+
+	/* Enable FRED */
+	cr4_set_bits(X86_CR4_FRED);
+	/* Any further IDT use is a bug */
+	idt_invalidate();
+
+	/* Use int $0x80 for 32-bit system calls in FRED mode */
+	setup_clear_cpu_cap(X86_FEATURE_SYSENTER32);
+	setup_clear_cpu_cap(X86_FEATURE_SYSCALL32);
+}
+
+/* Must be called after setup_cpu_entry_areas() */
+void cpu_init_fred_rsps(void)
+{
 	/*
 	 * The purpose of separate stacks for NMI, #DB and #MC *in the kernel*
 	 * (remember that user space faults are always taken on stack level 0)
@@ -47,13 +66,4 @@ void cpu_init_fred_exceptions(void)
 	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
 	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
 	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
-
-	/* Enable FRED */
-	cr4_set_bits(X86_CR4_FRED);
-	/* Any further IDT use is a bug */
-	idt_invalidate();
-
-	/* Use int $0x80 for 32-bit system calls in FRED mode */
-	setup_clear_cpu_cap(X86_FEATURE_SYSENTER32);
-	setup_clear_cpu_cap(X86_FEATURE_SYSCALL32);
 }
Signature · 1 line
2.45.2
merge bot Applied
#7 lore raw

[tip: x86/fred] x86/fred: Move FRED RSP initialization into a separate function

The following commit has been merged into the x86/fred branch of tip:

The following commit has been merged into the x86/fred branch of tip:

Commit-ID:     73270c1f2369fb37683121ebe097cd37172602b6
Gitweb:        https://git.kernel.org/tip/73270c1f2369fb37683121ebe097cd37172602b6
Author:        Xin Li (Intel) <[email protected]>
AuthorDate:    Tue, 09 Jul 2024 08:40:47 -07:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Tue, 13 Aug 2024 21:59:21 +02:00

x86/fred: Move FRED RSP initialization into a separate function

To enable FRED earlier, move the RSP initialization out of cpu_init_fred_exceptions() into cpu_init_fred_rsps().

This is required as the FRED RSP initialization depends on the availability of the CPU entry areas which are set up late in trap_init(),

No functional change intended. Marked with Fixes as it's a depedency for the real fix.

Fixes 14619d912b65 ("x86/fred: FRED entry/exit and dispatch code")
---
 arch/x86/include/asm/fred.h  |  2 ++
 arch/x86/kernel/cpu/common.c |  6 ++++--
 arch/x86/kernel/fred.c       | 28 +++++++++++++++++++---------
 3 files changed, 25 insertions(+), 11 deletions(-)
arch/x86/include/asm/fred.h +2 -0
index e86c7ba..66d7dbe 100644
@@ -84,11 +84,13 @@ static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int 
 }
 
 void cpu_init_fred_exceptions(void);
+void cpu_init_fred_rsps(void);
 void fred_complete_exception_setup(void);
 
 #else /* CONFIG_X86_FRED */
 static __always_inline unsigned long fred_event_data(struct pt_regs *regs) { return 0; }
 static inline void cpu_init_fred_exceptions(void) { }
+static inline void cpu_init_fred_rsps(void) { }
 static inline void fred_complete_exception_setup(void) { }
 static __always_inline void fred_entry_from_kvm(unsigned int type, unsigned int vector) { }
 #endif /* CONFIG_X86_FRED */
arch/x86/kernel/cpu/common.c +4 -2
index 10a5402..6de12b3 100644
@@ -2195,10 +2195,12 @@ void cpu_init_exception_handling(void)
 	/* GHCB needs to be setup to handle #VC. */
 	setup_ghcb();
 
-	if (cpu_feature_enabled(X86_FEATURE_FRED))
+	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
 		cpu_init_fred_exceptions();
-	else
+		cpu_init_fred_rsps();
+	} else {
 		load_current_idt();
+	}
 }
 
 /*
arch/x86/kernel/fred.c +19 -9
index 4bcd879..99a134f 100644
@@ -32,6 +32,25 @@ void cpu_init_fred_exceptions(void)
 	       FRED_CONFIG_INT_STKLVL(0) |
 	       FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user));
 
+	wrmsrl(MSR_IA32_FRED_STKLVLS, 0);
+	wrmsrl(MSR_IA32_FRED_RSP0, 0);
+	wrmsrl(MSR_IA32_FRED_RSP1, 0);
+	wrmsrl(MSR_IA32_FRED_RSP2, 0);
+	wrmsrl(MSR_IA32_FRED_RSP3, 0);
+
+	/* Enable FRED */
+	cr4_set_bits(X86_CR4_FRED);
+	/* Any further IDT use is a bug */
+	idt_invalidate();
+
+	/* Use int $0x80 for 32-bit system calls in FRED mode */
+	setup_clear_cpu_cap(X86_FEATURE_SYSENTER32);
+	setup_clear_cpu_cap(X86_FEATURE_SYSCALL32);
+}
+
+/* Must be called after setup_cpu_entry_areas() */
+void cpu_init_fred_rsps(void)
+{
 	/*
 	 * The purpose of separate stacks for NMI, #DB and #MC *in the kernel*
 	 * (remember that user space faults are always taken on stack level 0)
@@ -47,13 +66,4 @@ void cpu_init_fred_exceptions(void)
 	wrmsrl(MSR_IA32_FRED_RSP1, __this_cpu_ist_top_va(DB));
 	wrmsrl(MSR_IA32_FRED_RSP2, __this_cpu_ist_top_va(NMI));
 	wrmsrl(MSR_IA32_FRED_RSP3, __this_cpu_ist_top_va(DF));
-
-	/* Enable FRED */
-	cr4_set_bits(X86_CR4_FRED);
-	/* Any further IDT use is a bug */
-	idt_invalidate();
-
-	/* Use int $0x80 for 32-bit system calls in FRED mode */
-	setup_clear_cpu_cap(X86_FEATURE_SYSENTER32);
-	setup_clear_cpu_cap(X86_FEATURE_SYSCALL32);
 }
author maintainer
#8 lore raw

PATCH v2 3/3 x86/fred: Enable FRED right after init_mem_mapping()

In reply to Xin Li (Intel)

Enable FRED right after init_mem_mapping() to avoid #PF handler, exc_page_fault(), fetching its faulting address from the stack before FRED is enabled.

Fixes 14619d912b65 ("x86/fred: FRED entry/exit and dispatch code")
---
 arch/x86/kernel/cpu/common.c |  6 +-----
 arch/x86/kernel/setup.c      | 11 ++++++++++-
 arch/x86/kernel/smpboot.c    |  6 ++++++
 arch/x86/kernel/traps.c      |  4 ++++
 4 files changed, 21 insertions(+), 6 deletions(-)
arch/x86/kernel/cpu/common.c +1 -5
index 6de12b3c1b04..42d4136ed6ac 100644
@@ -2195,12 +2195,8 @@ void cpu_init_exception_handling(void)
 	/* GHCB needs to be setup to handle #VC. */
 	setup_ghcb();
 
-	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
-		cpu_init_fred_exceptions();
-		cpu_init_fred_rsps();
-	} else {
+	if (!cpu_feature_enabled(X86_FEATURE_FRED))
 		load_current_idt();
-	}
 }
 
 /*
arch/x86/kernel/setup.c +10 -1
index 728927e4ba51..36403b901eb2 100644
@@ -39,6 +39,7 @@
 #include <asm/coco.h>
 #include <asm/cpu.h>
 #include <asm/efi.h>
+#include <asm/fred.h>
 #include <asm/gart.h>
 #include <asm/hypervisor.h>
 #include <asm/io_apic.h>
@@ -1040,7 +1041,15 @@ void __init setup_arch(char **cmdline_p)
 
 	init_mem_mapping();
 
-	idt_setup_early_pf();
+	/*
+	 * init_mem_mapping() uses early IDT to setup memory mappings, thus FRED
+	 * can't be enabled earlier than that, unless FRED adds support to setup
+	 * memory mappings.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_exceptions();
+	else
+		idt_setup_early_pf();
 
 	/*
 	 * Update mmu_cr4_features (and, indirectly, trampoline_cr4_features)
arch/x86/kernel/smpboot.c +6 -0
index 0c35207320cb..0d83377f9dcd 100644
@@ -64,6 +64,7 @@
 #include <asm/acpi.h>
 #include <asm/cacheinfo.h>
 #include <asm/desc.h>
+#include <asm/fred.h>
 #include <asm/nmi.h>
 #include <asm/irq.h>
 #include <asm/realmode.h>
@@ -248,6 +249,11 @@ static void notrace start_secondary(void *unused)
 
 	cpu_init_exception_handling();
 
+	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
+		cpu_init_fred_exceptions();
+		cpu_init_fred_rsps();
+	}
+
 	/*
 	 * Load the microcode before reaching the AP alive synchronization
 	 * point below so it is not part of the full per CPU serialized
arch/x86/kernel/traps.c +4 -0
index 6afb41e6cbbb..81648bd07576 100644
@@ -1407,6 +1407,10 @@ void __init trap_init(void)
 	/* Init cpu_entry_area before IST entries are set up */
 	setup_cpu_entry_areas();
 
+	/* FRED RSPs pointing to memory from CPU entry areas */
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_rsps();
+
 	/* Init GHCB memory pages when running as an SEV-ES guest */
 	sev_es_init_vc_handling();
 
Signature · 1 line
2.45.2
#9 lore raw

Re: [PATCH v2 3/3] x86/fred: Enable FRED right after init_mem_mapping()

On Tue, Jul 09 2024 at 08:40, Xin Li wrote:

I'm really unhappy about sprinkling all these FRED conditionals all over the place:

Quoted 21 lines
(quoted diff) +13 -1
 	init_mem_mapping();
 
-	idt_setup_early_pf();
+	/*
+	 * init_mem_mapping() uses early IDT to setup memory mappings, thus FRED
+	 * can't be enabled earlier than that, unless FRED adds support to setup
+	 * memory mappings.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_exceptions();
+	else
+		idt_setup_early_pf();
@@ -248,6 +249,11 @@ static void notrace start_secondary(void *unused)
 
 	cpu_init_exception_handling();
 
+	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
+		cpu_init_fred_exceptions();
+		cpu_init_fred_rsps();
+	}

Review static void notrace start_secondary(void *unused)

Quoted 10 lines
(quoted diff) +4 -0
 
 	/* Init cpu_entry_area before IST entries are set up */
 	setup_cpu_entry_areas();
 
+	/* FRED RSPs pointing to memory from CPU entry areas */
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_rsps();
+
 	/* Init GHCB memory pages when running as an SEV-ES guest */
 	sev_es_init_vc_handling();

This really can be encapsulated and kept in places which need to know about FRED already. See below. Can you please validate?

Thanks,

tglx
---
From: "Xin Li (Intel)" <[email protected]>
Subject: x86/fred: Enable FRED right after init_mem_mapping()
Date: Tue, 09 Jul 2024 08:40:48 -0700

From: "Xin Li (Intel)" <[email protected]>

On 64-bit init_mem_mapping() relies on the minimal page fault handler provided by the early IDT mechanism. The real page fault handler is installed right afterwards into the IDT.

This is problematic on CPUs which have X86_FEATURE_FRED set because the real page fault handler retrieves the faulting address from the FRED exception stack frame and not from CR2, but that does obviously not work when FRED is not yet enabled in the CPU.

To prevent this enable FRED right after init_mem_mapping() without interrupt stacks. Those are enabled later in trap_init() after the CPU entry area is set up.

[ tglx: Encapsulate the FRED details ]

---
 arch/x86/include/asm/processor.h |    3 ++-
 arch/x86/kernel/cpu/common.c     |   15 +++++++++++++--
 arch/x86/kernel/setup.c          |    7 ++++++-
 arch/x86/kernel/smpboot.c        |    2 +-
 arch/x86/kernel/traps.c          |    2 +-
 5 files changed, 23 insertions(+), 6 deletions(-)
arch/x86/include/asm/processor.h +2 -1
@@ -582,7 +582,8 @@ extern void switch_gdt_and_percpu_base(i
 extern void load_direct_gdt(int);
 extern void load_fixmap_gdt(int);
 extern void cpu_init(void);
-extern void cpu_init_exception_handling(void);
+extern void cpu_init_exception_handling(bool boot_cpu);
+extern void cpu_init_replace_early_idt(void);
 extern void cr4_init(void);
 
 extern void set_task_blockstep(struct task_struct *task, bool on);
arch/x86/kernel/cpu/common.c +13 -2
@@ -2176,7 +2176,7 @@ static inline void tss_setup_io_bitmap(s
  * Setup everything needed to handle exceptions from the IDT, including the IST
  * exceptions which use paranoid_entry().
  */
-void cpu_init_exception_handling(void)
+void cpu_init_exception_handling(bool boot_cpu)
 {
 	struct tss_struct *tss = this_cpu_ptr(&cpu_tss_rw);
 	int cpu = raw_smp_processor_id();
@@ -2196,13 +2196,24 @@ void cpu_init_exception_handling(void)
 	setup_ghcb();
 
 	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
-		cpu_init_fred_exceptions();
+		/* The boot CPU has enabled FRED during early boot */
+		if (!boot_cpu)
+			cpu_init_fred_exceptions();
+
 		cpu_init_fred_rsps();
 	} else {
 		load_current_idt();
 	}
 }
 
+void __init cpu_init_replace_early_idt(void)
+{
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_exceptions();
+	else
+		idt_setup_early_pf();
+}
+
 /*
  * cpu_init() initializes state that is per-CPU. Some data is already
  * initialized (naturally) in the bootstrap process, such as the GDT.  We
arch/x86/kernel/setup.c +6 -1
@@ -1039,7 +1039,12 @@ void __init setup_arch(char **cmdline_p)
 
 	init_mem_mapping();
 
-	idt_setup_early_pf();
+	/*
+	 * init_mem_mapping() relies on the early IDT page fault handling.
+	 * Now either enable FRED or install the real page fault handler
+	 * for 64-bit in the IDT.
+	 */
+	cpu_init_replace_early_idt();
 
 	/*
 	 * Update mmu_cr4_features (and, indirectly, trampoline_cr4_features)
arch/x86/kernel/smpboot.c +1 -1
@@ -246,7 +246,7 @@ static void notrace start_secondary(void
 		__flush_tlb_all();
 	}
 
-	cpu_init_exception_handling();
+	cpu_init_exception_handling(false);
 
 	/*
 	 * Load the microcode before reaching the AP alive synchronization
arch/x86/kernel/traps.c +1 -1
@@ -1411,7 +1411,7 @@ void __init trap_init(void)
 	sev_es_init_vc_handling();
 
 	/* Initialize TSS before setting up traps so ISTs work */
-	cpu_init_exception_handling();
+	cpu_init_exception_handling(true);
 
 	/* Setup traps as cpu_init() might #GP */
 	if (!cpu_feature_enabled(X86_FEATURE_FRED))
author maintainer
#10 lore raw

Re: [PATCH v2 3/3] x86/fred: Enable FRED right after init_mem_mapping()

Quoted 4 lines · Thomas Gleixner

On Tue, Jul 09 2024 at 08:40, Xin Li wrote:

I'm really unhappy about sprinkling all these FRED conditionals all over the place:

Sigh, for the reason of a bad taste...

Quoted 25 lines
Quoted 25 lines
  	init_mem_mapping();
(quoted diff) +13 -1
-	idt_setup_early_pf();
+	/*
+	 * init_mem_mapping() uses early IDT to setup memory mappings, thus FRED
+	 * can't be enabled earlier than that, unless FRED adds support to setup
+	 * memory mappings.
+	 */
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_exceptions();
+	else
+		idt_setup_early_pf();
@@ -248,6 +249,11 @@ static void notrace start_secondary(void *unused)
 
  	cpu_init_exception_handling();
 
+	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
+		cpu_init_fred_exceptions();
+		cpu_init_fred_rsps();
+	}
 
  	/* Init cpu_entry_area before IST entries are set up */
  	setup_cpu_entry_areas();
Quoted 9 lines

Review static void notrace start_secondary(void *unused)

Quoted 6 lines
(quoted diff) +4 -0
+	/* FRED RSPs pointing to memory from CPU entry areas */
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_rsps();
+
  	/* Init GHCB memory pages when running as an SEV-ES guest */
  	sev_es_init_vc_handling();

This really can be encapsulated and kept in places which need to know about FRED already. See below. Can you please validate?

I reviewed your patch, and there is no reason it won't work; my tests show no problem with it.

Thanks!
Xin

Quoted 124 lines

---
From: "Xin Li (Intel)" <[email protected]>
Subject: x86/fred: Enable FRED right after init_mem_mapping()
Date: Tue, 09 Jul 2024 08:40:48 -0700

From: "Xin Li (Intel)" <[email protected]>

On 64-bit init_mem_mapping() relies on the minimal page fault handler provided by the early IDT mechanism. The real page fault handler is installed right afterwards into the IDT.

This is problematic on CPUs which have X86_FEATURE_FRED set because the real page fault handler retrieves the faulting address from the FRED exception stack frame and not from CR2, but that does obviously not work when FRED is not yet enabled in the CPU.

To prevent this enable FRED right after init_mem_mapping() without interrupt stacks. Those are enabled later in trap_init() after the CPU entry area is set up.

[ tglx: Encapsulate the FRED details ]

---
  arch/x86/include/asm/processor.h |    3 ++-
  arch/x86/kernel/cpu/common.c     |   15 +++++++++++++--
  arch/x86/kernel/setup.c          |    7 ++++++-
  arch/x86/kernel/smpboot.c        |    2 +-
  arch/x86/kernel/traps.c          |    2 +-
  5 files changed, 23 insertions(+), 6 deletions(-)
arch/x86/include/asm/processor.h +2 -1
@@ -582,7 +582,8 @@ extern void switch_gdt_and_percpu_base(i
  extern void load_direct_gdt(int);
  extern void load_fixmap_gdt(int);
  extern void cpu_init(void);
-extern void cpu_init_exception_handling(void);
+extern void cpu_init_exception_handling(bool boot_cpu);
+extern void cpu_init_replace_early_idt(void);
  extern void cr4_init(void);
 
  extern void set_task_blockstep(struct task_struct *task, bool on);
arch/x86/kernel/cpu/common.c +13 -2
@@ -2176,7 +2176,7 @@ static inline void tss_setup_io_bitmap(s
   * Setup everything needed to handle exceptions from the IDT, including the IST
   * exceptions which use paranoid_entry().
   */
-void cpu_init_exception_handling(void)
+void cpu_init_exception_handling(bool boot_cpu)
  {
  	struct tss_struct *tss = this_cpu_ptr(&cpu_tss_rw);
  	int cpu = raw_smp_processor_id();
@@ -2196,13 +2196,24 @@ void cpu_init_exception_handling(void)
  	setup_ghcb();
 
  	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
-		cpu_init_fred_exceptions();
+		/* The boot CPU has enabled FRED during early boot */
+		if (!boot_cpu)
+			cpu_init_fred_exceptions();
+
  		cpu_init_fred_rsps();
  	} else {
  		load_current_idt();
  	}
  }
 
+void __init cpu_init_replace_early_idt(void)
+{
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_exceptions();
+	else
+		idt_setup_early_pf();
+}
+
  /*
   * cpu_init() initializes state that is per-CPU. Some data is already
   * initialized (naturally) in the bootstrap process, such as the GDT.  We
arch/x86/kernel/setup.c +6 -1
@@ -1039,7 +1039,12 @@ void __init setup_arch(char **cmdline_p)
 
  	init_mem_mapping();
 
-	idt_setup_early_pf();
+	/*
+	 * init_mem_mapping() relies on the early IDT page fault handling.
+	 * Now either enable FRED or install the real page fault handler
+	 * for 64-bit in the IDT.
+	 */
+	cpu_init_replace_early_idt();
 
  	/*
  	 * Update mmu_cr4_features (and, indirectly, trampoline_cr4_features)
arch/x86/kernel/smpboot.c +1 -1
@@ -246,7 +246,7 @@ static void notrace start_secondary(void
  		__flush_tlb_all();
  	}
 
-	cpu_init_exception_handling();
+	cpu_init_exception_handling(false);
 
  	/*
  	 * Load the microcode before reaching the AP alive synchronization
arch/x86/kernel/traps.c +1 -1
@@ -1411,7 +1411,7 @@ void __init trap_init(void)
  	sev_es_init_vc_handling();
 
  	/* Initialize TSS before setting up traps so ISTs work */
-	cpu_init_exception_handling();
+	cpu_init_exception_handling(true);
 
  	/* Setup traps as cpu_init() might #GP */
  	if (!cpu_feature_enabled(X86_FEATURE_FRED))
merge bot Applied
#11 lore raw

[tip: x86/fred] x86/fred: Enable FRED right after init_mem_mapping()

In reply to Xin Li (Intel)
The following commit has been merged into the x86/fred branch of tip:

The following commit has been merged into the x86/fred branch of tip:

Commit-ID:     a97756cbec448032f84b5bbfe4e101478d1e01e0
Gitweb:        https://git.kernel.org/tip/a97756cbec448032f84b5bbfe4e101478d1e01e0
Author:        Xin Li (Intel) <[email protected]>
AuthorDate:    Tue, 09 Jul 2024 08:40:48 -07:00
Committer:     Thomas Gleixner <[email protected]>
CommitterDate: Tue, 13 Aug 2024 21:59:21 +02:00

x86/fred: Enable FRED right after init_mem_mapping()

On 64-bit init_mem_mapping() relies on the minimal page fault handler provided by the early IDT mechanism. The real page fault handler is installed right afterwards into the IDT.

This is problematic on CPUs which have X86_FEATURE_FRED set because the real page fault handler retrieves the faulting address from the FRED exception stack frame and not from CR2, but that does obviously not work when FRED is not yet enabled in the CPU.

To prevent this enable FRED right after init_mem_mapping() without interrupt stacks. Those are enabled later in trap_init() after the CPU entry area is set up.

[ tglx: Encapsulate the FRED details ]

---
 arch/x86/include/asm/processor.h |  3 ++-
 arch/x86/kernel/cpu/common.c     | 15 +++++++++++++--
 arch/x86/kernel/setup.c          |  7 ++++++-
 arch/x86/kernel/smpboot.c        |  2 +-
 arch/x86/kernel/traps.c          |  2 +-
 5 files changed, 23 insertions(+), 6 deletions(-)
arch/x86/include/asm/processor.h +2 -1
index a75a07f..399f7d1 100644
@@ -582,7 +582,8 @@ extern void switch_gdt_and_percpu_base(int);
 extern void load_direct_gdt(int);
 extern void load_fixmap_gdt(int);
 extern void cpu_init(void);
-extern void cpu_init_exception_handling(void);
+extern void cpu_init_exception_handling(bool boot_cpu);
+extern void cpu_init_replace_early_idt(void);
 extern void cr4_init(void);
 
 extern void set_task_blockstep(struct task_struct *task, bool on);
arch/x86/kernel/cpu/common.c +13 -2
index 6de12b3..a4735d9 100644
@@ -2176,7 +2176,7 @@ static inline void tss_setup_io_bitmap(struct tss_struct *tss)
  * Setup everything needed to handle exceptions from the IDT, including the IST
  * exceptions which use paranoid_entry().
  */
-void cpu_init_exception_handling(void)
+void cpu_init_exception_handling(bool boot_cpu)
 {
 	struct tss_struct *tss = this_cpu_ptr(&cpu_tss_rw);
 	int cpu = raw_smp_processor_id();
@@ -2196,13 +2196,24 @@ void cpu_init_exception_handling(void)
 	setup_ghcb();
 
 	if (cpu_feature_enabled(X86_FEATURE_FRED)) {
-		cpu_init_fred_exceptions();
+		/* The boot CPU has enabled FRED during early boot */
+		if (!boot_cpu)
+			cpu_init_fred_exceptions();
+
 		cpu_init_fred_rsps();
 	} else {
 		load_current_idt();
 	}
 }
 
+void __init cpu_init_replace_early_idt(void)
+{
+	if (cpu_feature_enabled(X86_FEATURE_FRED))
+		cpu_init_fred_exceptions();
+	else
+		idt_setup_early_pf();
+}
+
 /*
  * cpu_init() initializes state that is per-CPU. Some data is already
  * initialized (naturally) in the bootstrap process, such as the GDT.  We
arch/x86/kernel/setup.c +6 -1
index 6129dc2..f1fea50 100644
@@ -1039,7 +1039,12 @@ void __init setup_arch(char **cmdline_p)
 
 	init_mem_mapping();
 
-	idt_setup_early_pf();
+	/*
+	 * init_mem_mapping() relies on the early IDT page fault handling.
+	 * Now either enable FRED or install the real page fault handler
+	 * for 64-bit in the IDT.
+	 */
+	cpu_init_replace_early_idt();
 
 	/*
 	 * Update mmu_cr4_features (and, indirectly, trampoline_cr4_features)
arch/x86/kernel/smpboot.c +1 -1
index 0c35207..dc4fff8 100644
@@ -246,7 +246,7 @@ static void notrace start_secondary(void *unused)
 		__flush_tlb_all();
 	}
 
-	cpu_init_exception_handling();
+	cpu_init_exception_handling(false);
 
 	/*
 	 * Load the microcode before reaching the AP alive synchronization
arch/x86/kernel/traps.c +1 -1
index 6afb41e..197d588 100644
@@ -1411,7 +1411,7 @@ void __init trap_init(void)
 	sev_es_init_vc_handling();
 
 	/* Initialize TSS before setting up traps so ISTs work */
-	cpu_init_exception_handling();
+	cpu_init_exception_handling(true);
 
 	/* Setup traps as cpu_init() might #GP */
 	if (!cpu_feature_enabled(X86_FEATURE_FRED))
#12 lore raw
In reply to Xin Li (Intel)
Quoted 16 lines · Xin Li

Wenlong Hou from Ant group reported two problems during the FRED initialization:
https://lore.kernel.org/lkml/[email protected]/

The first problem is that spurious_interrupt() gets called on the HYPERVISOR_CALLBACK_VECTOR vector. Because kvm_guest_init(), being executed way before trap_init() in which it is decided that whether FRED will be enabled or not, calls sysvec_install() to install HYPERVISOR_CALLBACK_VECTOR's interrupt handler into FRED system vector dispatch table or IDT depending on whether FRED is enabled.

The other problem is that the #PF handler gets a wrong faulting address from the stack instead of CR2 before FRED is enabled. Because the #PF handler fetches its faulting addresss from the stack or CR2 based on whether FRED is available rather than active.

Quoted 8 lines

This patchset fixes the 2 problems with suggestions from tglx:

1) Parse cmdline param "fred=" in cpu_parse_early_param() to minimize the gap mentioned above, before kvm_guest_init().

2) Enable FRED right after init_mem_mapping() to switch to FRED from early IDT ASAP, avoid intermediately using the IDT #PF handler.

I just noticed that there is another leftover regarding FRED:

arch/x86/kernel/cpu/cpuid-deps.c:86: { X86_FEATURE_FRED, X86_FEATURE_WRMSRNS },

We removed the dependency on X86_FEATURE_WRMSRNS, right? So this is stale and should be removed too.

Thanks,

tglx

author maintainer
#13 lore raw
Quoted 6 lines

I just noticed that there is another leftover regarding FRED:

arch/x86/kernel/cpu/cpuid-deps.c:86: { X86_FEATURE_FRED, X86_FEATURE_WRMSRNS },

We removed the dependency on X86_FEATURE_WRMSRNS, right? So this is stale and should be removed too.

Right, I did plan to include such a change per your ask to remove the comment in arch/x86/include/asm/switch_to.h:

https://lore.kernel.org/lkml/87ttfw18jy.ffs@tglx/

So it will be in the next iteration of that patch set.

Thanks!
Xin

author maintainer
#14 lore raw
Quoted 6 lines · Xin Li
Quoted 6 lines

I just noticed that there is another leftover regarding FRED:

arch/x86/kernel/cpu/cpuid-deps.c:86: { X86_FEATURE_FRED, X86_FEATURE_WRMSRNS },

We removed the dependency on X86_FEATURE_WRMSRNS, right? So this is stale and should be removed too.

Sorry, this dependency is not removed yet, but will be as mentioned in my previous reply.

Thanks!
Xin

Quoted 6 lines

Right, I did plan to include such a change per your ask to remove the comment in arch/x86/include/asm/switch_to.h:

https://lore.kernel.org/lkml/87ttfw18jy.ffs@tglx/

So it will be in the next iteration of that patch set.