x86/fred: Fix the FRED RSP0 MSR out of sync with its per CPU cache
Subsystems (2)
Xin Li (Intel)
maintainerPATCH 1/1 x86/fred: Fix the FRED RSP0 MSR out of sync with its per CPU cache
The FRED RSP0 MSR (pointing to the top of the kernel stack for user level event delivery) and its per CPU cache should be kept in sync to avoid redundant writes in the exit to user space path, as a result, a write to the FRED RSP0 MSR is paired with a write to its per CPU cache as fred_update_rsp0() does.
However as the FRED RSP0 MSR is set to 0 in cpu_init_fred_exceptions(), it gets out of sync with its per CPU cache during a CPU offline/online cycle, which causes #DF exceptions if no context switch happens after a CPU offline/online cycle and before exit to user space.
Fix the bug through resynchronizing the FRED RSP0 MSR with its per CPU cache value in cpu_init_fred_exceptions().
--- arch/x86/kernel/fred.c | 13 ++++++++++++- 1 file changed, 12 insertions(+), 1 deletion(-)
arch/x86/kernel/fred.c +12 -1
@@ -50,7 +50,18 @@ void cpu_init_fred_exceptions(void) FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user));wrmsrl (MSR_IA32_FRED_STKLVLS ,0 ); -wrmsrl (MSR_IA32_FRED_RSP0 ,0 ); + +/* +* Resynchronize the FRED RSP0 MSR with its per CPU cache value. +* +* Another option is to leave the FRED RSP0 MSR as-is, because the RESET +* state of FRED MSRs is zero and INIT does not change the value of the +* FRED MSRs in a CPU offline/online cycle. But it doesn't seem safe to +* depend on the properties of INIT as that's way too many things that +* could cause bugs. +*/ +wrmsrl (MSR_IA32_FRED_RSP0 ,__this_cpu_read (fred_rsp0)); +wrmsrl (MSR_IA32_FRED_RSP1 ,0 );wrmsrl (MSR_IA32_FRED_RSP2 ,0 );wrmsrl (MSR_IA32_FRED_RSP3 ,0 );
base-commit: cf6b067860f69fe01b08ebae9138fe0f89854398
Signature · 1 line
2.47.1
Quoted 46 lines · Xin Li (Intel)
The FRED RSP0 MSR (pointing to the top of the kernel stack for user level event delivery) and its per CPU cache should be kept in sync to avoid redundant writes in the exit to user space path, as a result, a write to the FRED RSP0 MSR is paired with a write to its per CPU cache as
fred_update_rsp0()does.However as the FRED RSP0 MSR is set to 0 in
cpu_init_fred_exceptions(), it gets out of sync with its per CPU cache during a CPU offline/online cycle, which causes #DF exceptions if no context switch happens after a CPU offline/online cycle and before exit to user space.Fix the bug through resynchronizing the FRED RSP0 MSR with its per CPU cache value in
cpu_init_fred_exceptions().Fixes fe85ee391966 ("x86/entry: Set FRED RSP0 on return to userspace instead of context switch")Signed-off-by Xin Li (Intel) <[email protected]>---
arch/x86/kernel/fred.c| 13 ++++++++++++-1 file changed, 12 insertions(+), 1 deletion(-)arch/x86/kernel/fred.c +12
-1@@ -50,7 +50,18 @@ void cpu_init_fred_exceptions(void) FRED_CONFIG_ENTRYPOINT(asm_fred_entrypoint_user));wrmsrl (MSR_IA32_FRED_STKLVLS ,0 ); -wrmsrl (MSR_IA32_FRED_RSP0 ,0 ); + +/* +* Resynchronize the FRED RSP0 MSR with its per CPU cache value. +* +* Another option is to leave the FRED RSP0 MSR as-is, because the RESET +* state of FRED MSRs is zero and INIT does not change the value of the +* FRED MSRs in a CPU offline/online cycle. But it doesn't seem safe to +* depend on the properties of INIT as that's way too many things that +* could cause bugs. +*/ +wrmsrl (MSR_IA32_FRED_RSP0 ,__this_cpu_read (fred_rsp0)); +wrmsrl (MSR_IA32_FRED_RSP1 ,0 );wrmsrl (MSR_IA32_FRED_RSP2 ,0 );wrmsrl (MSR_IA32_FRED_RSP3 ,0 );
Quoted 1 line
base-commit:
cf6b067860f69fe01b08ebae9138fe0f89854398
Looks straightforward enough.
arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61
Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5
-0+* Another option is to leave the FRED RSP0 MSR as-is, because the RESET +* state of FRED MSRs is zero and INIT does not change the value of the +* FRED MSRs in a CPU offline/online cycle. But it doesn't seem safe to +* depend on the properties of INIT as that's way too many things that +* could cause bugs.
Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.
Quoted 9 lines · Dave Hansen
arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61
Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5
-0+* Another option is to leave the FRED RSP0 MSR as-is, because the RESET +* state of FRED MSRs is zero and INIT does not change the value of the +* FRED MSRs in a CPU offline/online cycle. But it doesn't seem safe to +* depend on the properties of INIT as that's way too many things that +* could cause bugs. Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.
That's why the init code needs to write every control MSR, even if it's filling in 0 because it doesn't want to use the feature.
The job of the init code is to go from an unknown state into a good state, not from the INIT/#RESET state.
Or someone could finally write down an ABI for Kexec which details who's responsibility it is :)
~Andrew
Quoted 10 lines · Andrew Cooper
Quoted 9 lines · Dave Hansen
arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61
Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5
-0+* Another option is to leave the FRED RSP0 MSR as-is, because the RESET +* state of FRED MSRs is zero and INIT does not change the value of the +* FRED MSRs in a CPU offline/online cycle. But it doesn't seem safe to +* depend on the properties of INIT as that's way too many things that +* could cause bugs. Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.
Quoted 5 lines
That's why the init code needs to write every control MSR, even if it's filling in 0 because it doesn't want to use the feature.
The job of the init code is to go from an unknown state into a good state, not from the INIT/#RESET state.
Right, so I think the quoted "Another option..." comment needs to get zapped. Because it's not really another option. Discussing alternatives is better left for changelogs anyway, not code comments.
Xin Li
maintainerQuoted 14 lines · Dave Hansen
Quoted 13 lines · Andrew Cooper
Quoted 9 lines · Dave Hansen
arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61
Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5
-0+* Another option is to leave the FRED RSP0 MSR as-is, because the RESET +* state of FRED MSRs is zero and INIT does not change the value of the +* FRED MSRs in a CPU offline/online cycle. But it doesn't seem safe to +* depend on the properties of INIT as that's way too many things that +* could cause bugs. Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.
That's why the init code needs to write every control MSR, even if it's filling in 0 because it doesn't want to use the feature.
Quoted 7 lines
Quoted 2 lines
The job of the init code is to go from an unknown state into a good state, not from the INIT/#RESET state.
Right, so I think the quoted "Another option..." comment needs to get zapped. Because it's not really another option. Discussing alternatives is better left for changelogs anyway, not code comments.
Will do!
H. Peter Anvin
maintainerQuoted 11 lines · Dave Hansen
arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61
Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5
-0+* Another option is to leave the FRED RSP0 MSR as-is, because the RESET +* state of FRED MSRs is zero and INIT does not change the value of the +* FRED MSRs in a CPU offline/online cycle. But it doesn't seem safe to +* depend on the properties of INIT as that's way too many things that +* could cause bugs. Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.
Yeah, it is a nonstarter. I believe Xin was trying to document the problems with *not* doing this, so someone doesn't have the bright idea to "optimize" this later.
Xin Li
maintainerQuoted 13 lines · H. Peter Anvin
Quoted 11 lines · Dave Hansen
arch/x86/kernel/fred.c void cpu_init_fred_exceptions(void) · line 61
Quoted 5 lines · Xin Li (Intel)
arch/x86/kernel/fred.c +5
-0+* Another option is to leave the FRED RSP0 MSR as-is, because the RESET +* state of FRED MSRs is zero and INIT does not change the value of the +* FRED MSRs in a CPU offline/online cycle. But it doesn't seem safe to +* depend on the properties of INIT as that's way too many things that +* could cause bugs. Wouldn't that also open us up to kexec problems? If the last kernel (maybe not even Linux) left RSP0 set, then the current kernel might eventually try to run userspace with the old kernel's RSP0 value.
Quoted 2 lines
Yeah, it is a nonstarter. I believe Xin was trying to document the problems with *not* doing this, so someone doesn't have the bright idea to "optimize" this later.
exactly ;)
Quoted 5 lines · Xin Li (Intel)
The FRED RSP0 MSR (pointing to the top of the kernel stack for user level event delivery) and its per CPU cache should be kept in sync to avoid redundant writes in the exit to user space path, as a result, a write to the FRED RSP0 MSR is paired with a write to its per CPU cache as
fred_update_rsp0()does.
I _think_ you're trying to explain the general use of a per-cpu MSR cache. That's good. But I was reading this paragraph and thinking at this point that the bug had something to do with redundant writes to the MSR.
How about this?
The FRED RSP0 MSR is only used for delivering events when running userspace. This kernel leverages this property to reduce expensive MSR writes and optimize context switches. The kernel only writes the MSR when about to run userspace *and* when the MSR has actually changed since the last time userspace ran.
This optimization is implemented by maintaining a per-cpu cache of FRED RSP0 and then checking that against the value for the current task's stack before running userspace.
However cpu_init_fred_exceptions() writes the MSR without updating the per-cpu cache. This means that the kernel might return to userspace with MSR_IA32_FRED_RSP0==0 when it needed to point to the current task stack. This would induce a double fault (#DF), which is bad.
A context switch after cpu_init_fred_exceptions() can paper over the issue since it updates the cached value. That evidently happens most of the time explaining how this bug got through.
Xin Li
maintainerQuoted 30 lines · Dave Hansen
Quoted 5 lines · Xin Li (Intel)
The FRED RSP0 MSR (pointing to the top of the kernel stack for user level event delivery) and its per CPU cache should be kept in sync to avoid redundant writes in the exit to user space path, as a result, a write to the FRED RSP0 MSR is paired with a write to its per CPU cache as
fred_update_rsp0()does.I _think_ you're trying to explain the general use of a per-cpu MSR cache. That's good. But I was reading this paragraph and thinking at this point that the bug had something to do with redundant writes to the MSR.
How about this?
The FRED RSP0 MSR is only used for delivering events when running userspace. This kernel leverages this property to reduce expensive MSR writes and optimize context switches. The kernel only writes the MSR when about to run userspace *and* when the MSR has actually changed since the last time userspace ran.
This optimization is implemented by maintaining a per-cpu cache of FRED RSP0 and then checking that against the value for the current task's stack before running userspace.
However
cpu_init_fred_exceptions()writes the MSR without updating the per-cpu cache. This means that the kernel might return to userspace with MSR_IA32_FRED_RSP0==0 when it needed to point to the current task stack. This would induce a double fault (#DF), which is bad.
Quoted 4 lines
A context switch after
cpu_init_fred_exceptions()can paper over the issue since it updates the cached value. That evidently happens most of the time explaining how this bug got through.
This is a way better explanation, thanks a lot!