Skip to contentSkip to conversation

x86/fred: Fix the FRED RSP0 MSR out of sync with its per CPU cache

PATCH Superseded Xin Li (Intel) 4 participants 10 messages Jan 8, 2025 – Jan 9, 2025

Subsystems (2)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Xin Li (Intel)

author maintainer
#0 lore raw

PATCH 1/1 x86/fred: Fix the FRED RSP0 MSR out of sync with its per CPU cache

The FRED RSP0 MSR (pointing to the top of the kernel stack for user level event delivery) and its per CPU cache should be kept in sync to avoid redundant writes in the exit to user space path, as a result, a write to the FRED RSP0 MSR is paired with a write to its per CPU cache as fred_update_rsp0() does.

However as the FRED RSP0 MSR is set to 0 in cpu_init_fred_exceptions(), it gets out of sync with its per CPU cache during a CPU offline/online cycle, which causes #DF exceptions if no context switch happens after a CPU offline/online cycle and before exit to user space.

Fix the bug through resynchronizing the FRED RSP0 MSR with its per CPU cache value in cpu_init_fred_exceptions().

Fixes fe85ee391966 ("x86/entry: Set FRED RSP0 on return to userspace instead of context switch")
---
 arch/x86/kernel/fred.c | 13 ++++++++++++-
 1 file changed, 12 insertions(+), 1 deletion(-)
arch/x86/kernel/fred.c +12 -1
index 8d32c3f48abc..9524ace96bfa 100644
@@ -50,7 +50,18 @@ void cpu_init_fred_exceptions(void)
 	       FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user));
 
 	wrmsrl(MSR_IA32_FRED_STKLVLS, 0);
-	wrmsrl(MSR_IA32_FRED_RSP0, 0);
+
+	/*
+	 * Resynchronize the FRED RSP0 MSR with its per CPU cache value.
+	 *
+	 * Another option is to leave the FRED RSP0 MSR as-is, because the RESET
+	 * state of FRED MSRs is zero and INIT does not change the value of the
+	 * FRED MSRs in a CPU offline/online cycle.  But it doesn't seem safe to
+	 * depend on the properties of INIT as that's way too many things that
+	 * could cause bugs.
+	 */
+	wrmsrl(MSR_IA32_FRED_RSP0, __this_cpu_read(fred_rsp0));
+
 	wrmsrl(MSR_IA32_FRED_RSP1, 0);
 	wrmsrl(MSR_IA32_FRED_RSP2, 0);
 	wrmsrl(MSR_IA32_FRED_RSP3, 0);

base-commit: cf6b067860f69fe01b08ebae9138fe0f89854398

Signature · 1 line
2.47.1
maintainer Reviewed-by on v1 1/1
#1 lore raw
Quoted 46 lines · Xin Li (Intel)

The FRED RSP0 MSR (pointing to the top of the kernel stack for user level event delivery) and its per CPU cache should be kept in sync to avoid redundant writes in the exit to user space path, as a result, a write to the FRED RSP0 MSR is paired with a write to its per CPU cache as fred_update_rsp0() does.

However as the FRED RSP0 MSR is set to 0 in cpu_init_fred_exceptions(), it gets out of sync with its per CPU cache during a CPU offline/online cycle, which causes #DF exceptions if no context switch happens after a CPU offline/online cycle and before exit to user space.

Fix the bug through resynchronizing the FRED RSP0 MSR with its per CPU cache value in cpu_init_fred_exceptions().

Fixes fe85ee391966 ("x86/entry: Set FRED RSP0 on return to userspace instead of context switch")

---
arch/x86/kernel/fred.c | 13 ++++++++++++-

1 file changed, 12 insertions(+), 1 deletion(-)
arch/x86/kernel/fred.c +12 -1
index 8d32c3f48abc..9524ace96bfa 100644
@@ -50,7 +50,18 @@ void cpu_init_fred_exceptions(void)
 	       FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user));
 
 	wrmsrl(MSR_IA32_FRED_STKLVLS, 0);
-	wrmsrl(MSR_IA32_FRED_RSP0, 0);
+
+	/*
+	 * Resynchronize the FRED RSP0 MSR with its per CPU cache value.
+	 *
+	 * Another option is to leave the FRED RSP0 MSR as-is, because the RESET
+	 * state of FRED MSRs is zero and INIT does not change the value of the
+	 * FRED MSRs in a CPU offline/online cycle.  But it doesn't seem safe to
+	 * depend on the properties of INIT as that's way too many things that
+	 * could cause bugs.
+	 */
+	wrmsrl(MSR_IA32_FRED_RSP0, __this_cpu_read(fred_rsp0));
+
 	wrmsrl(MSR_IA32_FRED_RSP1, 0);
 	wrmsrl(MSR_IA32_FRED_RSP2, 0);
 	wrmsrl(MSR_IA32_FRED_RSP3, 0);
Quoted 1 line

base-commit: cf6b067860f69fe01b08ebae9138fe0f89854398

Looks straightforward enough.

#2 lore raw
In reply to Xin Li (Intel)

arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61

Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5 -0
+	 * Another option is to leave the FRED RSP0 MSR as-is, because the RESET
+	 * state of FRED MSRs is zero and INIT does not change the value of the
+	 * FRED MSRs in a CPU offline/online cycle.  But it doesn't seem safe to
+	 * depend on the properties of INIT as that's way too many things that
+	 * could cause bugs.

Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.

#3 lore raw
Quoted 9 lines · Dave Hansen

arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61

Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5 -0
+	 * Another option is to leave the FRED RSP0 MSR as-is, because the RESET
+	 * state of FRED MSRs is zero and INIT does not change the value of the
+	 * FRED MSRs in a CPU offline/online cycle.  But it doesn't seem safe to
+	 * depend on the properties of INIT as that's way too many things that
+	 * could cause bugs.

Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.

That's why the init code needs to write every control MSR, even if it's filling in 0 because it doesn't want to use the feature.

The job of the init code is to go from an unknown state into a good state, not from the INIT/#RESET state.

Or someone could finally write down an ABI for Kexec which details who's responsibility it is :)

~Andrew

#4 lore raw
Quoted 10 lines · Andrew Cooper
Quoted 9 lines · Dave Hansen

arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61

Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5 -0
+	 * Another option is to leave the FRED RSP0 MSR as-is, because the RESET
+	 * state of FRED MSRs is zero and INIT does not change the value of the
+	 * FRED MSRs in a CPU offline/online cycle.  But it doesn't seem safe to
+	 * depend on the properties of INIT as that's way too many things that
+	 * could cause bugs.

Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.

Quoted 5 lines

That's why the init code needs to write every control MSR, even if it's filling in 0 because it doesn't want to use the feature.

The job of the init code is to go from an unknown state into a good state, not from the INIT/#RESET state.

Right, so I think the quoted "Another option..." comment needs to get zapped. Because it's not really another option. Discussing alternatives is better left for changelogs anyway, not code comments.

author maintainer
#5 lore raw
Quoted 14 lines · Dave Hansen
Quoted 13 lines · Andrew Cooper
Quoted 9 lines · Dave Hansen

arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61

Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5 -0
+	 * Another option is to leave the FRED RSP0 MSR as-is, because the RESET
+	 * state of FRED MSRs is zero and INIT does not change the value of the
+	 * FRED MSRs in a CPU offline/online cycle.  But it doesn't seem safe to
+	 * depend on the properties of INIT as that's way too many things that
+	 * could cause bugs.

Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.

That's why the init code needs to write every control MSR, even if it's filling in 0 because it doesn't want to use the feature.

Quoted 7 lines
Quoted 2 lines

The job of the init code is to go from an unknown state into a good state, not from the INIT/#RESET state.

Right, so I think the quoted "Another option..." comment needs to get zapped. Because it's not really another option. Discussing alternatives is better left for changelogs anyway, not code comments.

Will do!

maintainer
#6 lore raw
In reply to Dave Hansen
Quoted 11 lines · Dave Hansen

arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61

Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5 -0
+	 * Another option is to leave the FRED RSP0 MSR as-is, because the RESET
+	 * state of FRED MSRs is zero and INIT does not change the value of the
+	 * FRED MSRs in a CPU offline/online cycle.  But it doesn't seem safe to
+	 * depend on the properties of INIT as that's way too many things that
+	 * could cause bugs.

Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.

Yeah, it is a nonstarter. I believe Xin was trying to document the problems with *not* doing this, so someone doesn't have the bright idea to "optimize" this later.

author maintainer
#7 lore raw
Quoted 13 lines · H. Peter Anvin
Quoted 11 lines · Dave Hansen

arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61

Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5 -0
+	 * Another option is to leave the FRED RSP0 MSR as-is, because the RESET
+	 * state of FRED MSRs is zero and INIT does not change the value of the
+	 * FRED MSRs in a CPU offline/online cycle.  But it doesn't seem safe to
+	 * depend on the properties of INIT as that's way too many things that
+	 * could cause bugs.

Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.

Quoted 2 lines

Yeah, it is a nonstarter. I believe Xin was trying to document the problems with *not* doing this, so someone doesn't have the bright idea to "optimize" this later.

exactly ;)

#8 lore raw
In reply to Xin Li (Intel)
Quoted 5 lines · Xin Li (Intel)

The FRED RSP0 MSR (pointing to the top of the kernel stack for user level event delivery) and its per CPU cache should be kept in sync to avoid redundant writes in the exit to user space path, as a result, a write to the FRED RSP0 MSR is paired with a write to its per CPU cache as fred_update_rsp0() does.

I _think_ you're trying to explain the general use of a per-cpu MSR cache. That's good. But I was reading this paragraph and thinking at this point that the bug had something to do with redundant writes to the MSR.

How about this?

The FRED RSP0 MSR is only used for delivering events when running userspace. This kernel leverages this property to reduce expensive MSR writes and optimize context switches. The kernel only writes the MSR when about to run userspace *and* when the MSR has actually changed since the last time userspace ran.

This optimization is implemented by maintaining a per-cpu cache of FRED RSP0 and then checking that against the value for the current task's stack before running userspace.

However cpu_init_fred_exceptions() writes the MSR without updating the per-cpu cache. This means that the kernel might return to userspace with MSR_IA32_FRED_RSP0==0 when it needed to point to the current task stack. This would induce a double fault (#DF), which is bad.

A context switch after cpu_init_fred_exceptions() can paper over the issue since it updates the cached value. That evidently happens most of the time explaining how this bug got through.

author maintainer
#9 lore raw
Quoted 30 lines · Dave Hansen
Quoted 5 lines · Xin Li (Intel)

The FRED RSP0 MSR (pointing to the top of the kernel stack for user level event delivery) and its per CPU cache should be kept in sync to avoid redundant writes in the exit to user space path, as a result, a write to the FRED RSP0 MSR is paired with a write to its per CPU cache as fred_update_rsp0() does.

I _think_ you're trying to explain the general use of a per-cpu MSR cache. That's good. But I was reading this paragraph and thinking at this point that the bug had something to do with redundant writes to the MSR.

How about this?

The FRED RSP0 MSR is only used for delivering events when running userspace. This kernel leverages this property to reduce expensive MSR writes and optimize context switches. The kernel only writes the MSR when about to run userspace *and* when the MSR has actually changed since the last time userspace ran.

This optimization is implemented by maintaining a per-cpu cache of FRED RSP0 and then checking that against the value for the current task's stack before running userspace.

However cpu_init_fred_exceptions() writes the MSR without updating the per-cpu cache. This means that the kernel might return to userspace with MSR_IA32_FRED_RSP0==0 when it needed to point to the current task stack. This would induce a double fault (#DF), which is bad.

Quoted 4 lines

A context switch after cpu_init_fred_exceptions() can paper over the issue since it updates the cached value. That evidently happens most of the time explaining how this bug got through.

This is a way better explanation, thanks a lot!