Skip to contentSkip to conversation

Kernel Shadow Stacks Versus XFEATURE_MASK_GUEST_SUPERVISOR

DISCUSSION Dave Hansen 4 participants 6 messages Oct 23 – Oct 27

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

So, I'm working on kernel shadow stacks. I have some nice optimization code like fred_update_rsp0() for SSP0. It avoids WRMSR's on exits to userspace if FRED_SSP0 hasn't changed, just like FRED_RSP0.

Well, turns out that FRED_SSP0 isn't like FRED_RSP0 in one very important way: it is an alias for PL0_SSP, which is managed via XFEATURE_CET_KERNEL which is in:

	/* Supervisor features which are enabled only in guest FPUs */
	#define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL

So, I was running along just fine in my nice shadow stack kernel going in and out of the kernel just fine. But the first time I did an XRSTOR, the XFEATURE_CET_KERNEL feature got set to its init state. My SSP0 cache didn't notice and I tried to run userspace with SSP0=0. Much hilarity ensued on the next kernel entry.

The most adorable part of all this is that I have KVM compiled completely out on my config. The "only in guest FPUs" code is biting me on a kernel that can't even run guests.

I think this all started because of 8b05b3c98816 ("x86/fpu/xstate: Add CET supervisor xfeature support as a guest-only feature"), which seems like mostly my doing. So I have mostly myself to blame.

Any thoughts on what we should do?

Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a lot of ways. PL0_SSP can always be derived _and_ it isn't actually even needed at context switch time, only when running userspace.

I could just aggressively blow away my SSP0 cache any time an XRSTOR touches XFEATURE_CET_KERNEL. That would be easy. But it also seems silly that XFEATURE_CET_KERNEL is being blown away at _all_ during normal context switches. Heck, it goes further than that because:

fpregs_restore_userregs() does:

	restore_fpregs_from_fpstate(fpu->fpstate,
				    XFEATURE_MASK_FPSTATE);

Where XFEATURE_MASK_FPSTATE contains XFEATURE_MASK_SUPERVISOR_SUPPORTED.

So a "restore_userregs" function is blowing away supervisor state.

IOW, I think we're using XFEATURE_MASK_FPSTATE a bit too broadly.

I'd like to get to a place where only the KVM code even has XFEATURE_CET_KERNEL in its RFBM.

Anybody have any bright ideas?

maintainer
#1 lore raw
Quoted 17 lines · Dave Hansen

So, I'm working on kernel shadow stacks. I have some nice optimization code like fred_update_rsp0() for SSP0. It avoids WRMSR's on exits to userspace if FRED_SSP0 hasn't changed, just like FRED_RSP0.

Well, turns out that FRED_SSP0 isn't like FRED_RSP0 in one very important way: it is an alias for PL0_SSP, which is managed via XFEATURE_CET_KERNEL which is in:

	/* Supervisor features which are enabled only in guest FPUs */
	#define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL

So, I was running along just fine in my nice shadow stack kernel going in and out of the kernel just fine. But the first time I did an XRSTOR, the XFEATURE_CET_KERNEL feature got set to its init state. My SSP0 cache didn't notice and I tried to run userspace with SSP0=0. Much hilarity ensued on the next kernel entry.

Quoted 3 lines

The most adorable part of all this is that I have KVM compiled completely out on my config. The "only in guest FPUs" code is biting me on a kernel that can't even run guests.

If you want to quickly work around for your !CONFIG_KVM kernel, I would do

#ifdef CONFIG_KVM
/* Supervisor features which are enabled only in guest FPUs */ #define XFEATURE_MASK_GUEST_SUPERVISOR XFEATURE_MASK_CET_KERNEL #else
#define XFEATURE_MASK_GUEST_SUPERVISOR 0
#endif

and a small adjustment like

        BUILD_BUG_ON((XFEATURE_MASK_USER_SUPPORTED |
                      XFEATURE_MASK_SUPERVISOR_SUPPORTED) &
                     ~XFEATURES_INIT_FPSTATE_HANDLED);

That will get rid of the bit in XSS.

Quoted 5 lines

Any thoughts on what we should do?

Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a lot of ways. PL0_SSP can always be derived _and_ it isn't actually even needed at context switch time, only when running userspace.

True but, just to understand, what is the performance like if you actually use XSAVES/XRSTORS to update SSP0 on context switches instead of WRMSR+cache? Hopefully the microcode does a bunch of optimizations as well.

Quoted 11 lines

I could just aggressively blow away my SSP0 cache any time an XRSTOR touches XFEATURE_CET_KERNEL. That would be easy. But it also seems silly that XFEATURE_CET_KERNEL is being blown away at _all_ during normal context switches. Heck, it goes further than that because:

fpregs_restore_userregs() does:

	restore_fpregs_from_fpstate(fpu->fpstate,
				    XFEATURE_MASK_FPSTATE);

Where XFEATURE_MASK_FPSTATE contains XFEATURE_MASK_SUPERVISOR_SUPPORTED.

The actual mask used by restore_fpregs_from_fpstate is tighter though, it's "fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSTATE".

Can you replace fpu_kernel_cfg.max_features with some fpstate->xyz (sorry too late to think it through :)) that has CET_KERNEL for the guest FPU and not for the regular one?

Quoted 6 lines

So a "restore_userregs" function is blowing away supervisor state.

IOW, I think we're using XFEATURE_MASK_FPSTATE a bit too broadly.

I'd like to get to a place where only the KVM code even has XFEATURE_CET_KERNEL in its RFBM.

That I agree, but I think it can be handled in {save,restore}_fpregs_to_fpstate().

Paolo

#2 lore raw
Quoted 1 line · Paolo Bonzini

On 10/23/26 21:03, Dave Hansen wrote:

...

Quoted 8 lines

If you want to quickly work around for your !CONFIG_KVM kernel, I would do

#ifdef CONFIG_KVM
/* Supervisor features which are enabled only in guest FPUs */ #define XFEATURE_MASK_GUEST_SUPERVISOR  XFEATURE_MASK_CET_KERNEL #else
#define XFEATURE_MASK_GUEST_SUPERVISOR  0
#endif

Yup, that'd work for sure. Thanks!

Quoted 10 lines
Quoted 5 lines

Any thoughts on what we should do?

Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a lot of ways. PL0_SSP can always be derived _and_ it isn't actually even needed at context switch time, only when running userspace.

True but, just to understand, what is the performance like if you actually use XSAVES/XRSTORS to update SSP0 on context switches instead of WRMSR+cache?  Hopefully the microcode does a bunch of optimizations as well.

First, we can use WRMSRNS for this MSR, so it's faster than usual. Second, on kernel=>kernel context switches that don't go out to userspace, it doesn't need to be written at all. It's kinda like the user FPU state. Third, I'm actually reusing the fred_ssp0 cache, so further reduces the incremental cost. Fourth, it's not _really_ context-switched. There's never a need to read the register. It can always be conjured out of thin air. It's possible to only write the register, never read it.

I can benchmark it, but I seriously doubt that an XSAVE/XRSTOR at context switch of three fields is going to beat an WRMSRNS of one.

Quoted 10 lines
Quoted 10 lines

I could just aggressively blow away my SSP0 cache any time an XRSTOR touches XFEATURE_CET_KERNEL. That would be easy. But it also seems silly that XFEATURE_CET_KERNEL is being blown away at _all_ during normal context switches. Heck, it goes further than that because:

fpregs_restore_userregs() does:

    restore_fpregs_from_fpstate(fpu->fpstate,
                    XFEATURE_MASK_FPSTATE);

Quoted 8 lines
Quoted 1 line

Where XFEATURE_MASK_FPSTATE contains XFEATURE_MASK_SUPERVISOR_SUPPORTED.

The actual mask used by restore_fpregs_from_fpstate is tighter though, it's "fpu_kernel_cfg.max_features & XFEATURE_MASK_FPSTATE".

Can you replace fpu_kernel_cfg.max_features with some fpstate->xyz (sorry too late to think it through :)) that has CET_KERNEL for the guest FPU and not for the regular one?

Sure I can give that a shot.

There's one minor issue in that we need to init dynamic features which _aren't_ in fpstate->xfeatures when context switching. There's a little note about this in restore_fpregs_from_fpstate().

I think what you're getting at is that we need _something_ to say that KVM fpstates should muck with XFEATURE_CET_KERNEL and task fpstates should not, even though the task ones have space for XFEATURE_CET_KERNEL.

That's certainly doable.

Quoted 8 lines
Quoted 6 lines

So a "restore_userregs" function is blowing away supervisor state.

IOW, I think we're using XFEATURE_MASK_FPSTATE a bit too broadly.

I'd like to get to a place where only the KVM code even has XFEATURE_CET_KERNEL in its RFBM.

That I agree, but I think it can be handled in {save,restore} _fpregs_to_fpstate().

I think I understand what you're going after at least. I'll poke at it a bit more.

Thanks for the thoughts!

#3 lore raw
Quoted 11 lines · Dave Hansen
Quoted 10 lines · Paolo Bonzini
Quoted 5 lines · Dave Hansen

Any thoughts on what we should do?

Actually context switching XFEATURE_CET_KERNEL seems kinda silly in a lot of ways. PL0_SSP can always be derived _and_ it isn't actually even needed at context switch time, only when running userspace.

True but, just to understand, what is the performance like if you actually use XSAVES/XRSTORS to update SSP0 on context switches instead of WRMSR+cache?  Hopefully the microcode does a bunch of optimizations as well.

Quoted 5 lines

First, we can use WRMSRNS for this MSR, so it's faster than usual. Second, on kernel=>kernel context switches that don't go out to userspace, it doesn't need to be written at all. It's kinda like the user FPU state. Third, I'm actually reusing the fred_ssp0 cache, so further reduces the incremental cost.

XSTATE_CET_S is a weird thing.  It contains MSRs for PL{0..2}_SSP which are the shstk equivalents of tss->rsp{0..2}.

The Ring1/2 aspects are practically useless; CET-SS was the thing which finally caused me to put Ring1 PV kernels to rest.  All of tss->rsp{0..2} are switched together, so I suppose PL{0..2}_SSP were grouped for the same reason, not that 64bit OSes use task switching.

Windows uses %xmm in the main ABI, so already need to XSAVE/XRSTOR when switching between kernel threads.  So it's "simple" to add one extra bit to RFBM while doing this.

VMMs probably want to use XSTATE_CET_S simply to have the guest state held together.  Even then, it does feel like a waste; PL1/2_SSP are going to be 0 ~all of the time, while PL0_SSP is nonzero ~all of the time in a CET-enabled kernel, so you can't even rely on the INIT optimisation to speed things up.

Linux obviously doesn't want to do an X{SAVE,RSTOR}S between kernel threads, and isn't using Rings 1/2, so the WRMSRNS approach seems fine. And I agree that it only needs to be good at the point of returning to userspace.

Quoted 3 lines

Fourth, it's not _really_
context-switched. There's never a need to read the register. It can always be conjured out of thin air.

The requirements are different, depending on FRED or not.

Without FRED, PL0_SSP must point to a Supervisor Shadow Stack token. IRET marks the token non-busy (squashing any errors), and IDT delivery checks it can atomically set the token busy, yielding #CP if not.

With FRED, ERETU checks that SSP == SL0_SSP and yields #CP on a mismatch.  There are no checks on entry.

~Andrew

#4 lore raw
In reply to Dave Hansen
Quoted 10 lines
Quoted 3 lines

Can you replace fpu_kernel_cfg.max_features with some fpstate->xyz (sorry too late to think it through :)) that has CET_KERNEL for the guest FPU and not for the regular one?

Sure I can give that a shot.

There's one minor issue in that we need to init dynamic features which _aren't_ in fpstate->xfeatures when context switching. There's a little note about this in restore_fpregs_from_fpstate().

Quoted 5 lines

I think what you're getting at is that we need _something_ to say that KVM fpstates should muck with XFEATURE_CET_KERNEL and task fpstates should not, even though the task ones have space for XFEATURE_CET_KERNEL.

That's certainly doable.

Sounds like you are talking about something like this, which Sean suggested in [1]:

arch/x86/kernel/fpu/core.c +10 -0
index 1015af1ae562..8e7599e3b923 100644
@@ -167,6 +167,16 @@ void restore_fpregs_from_fpstate(struct fpstate *fpstate, u64 mask)
                 */
                xfd_update_state(fpstate);
 
+               /*
+                * Leave supervisor CET state as-is when loading host state
+                * (kernel or userspace).  Supervisor CET state is managed via
+                * XSTATE for KVM guests, but the host never consumes said
+                * state (doesn't support supervisor shadow stacks), i.e. it's
+                * safe to keep guest state loaded into hardware.
+                */
+               if (!fpstate->is_guest)
+                       mask &= ~XFEATURE_MASK_CET_KERNEL;
+
 

We first looked at enabling CET_KERNEL for all tasks, and that version included this optimization so that restoring a non-KVM fpstate doesn't touch CET_KERNEL.

We then switched to the guest-only approach to avoid allocating CET_KERNEL space in non-KVM fpstates. The optimization could still be applied on top of that, but it was dropped after its benefit was questioned off-list:

"""
1. It hacks in an optimization with conditionals around the edges, instead of integrating with the way the rest of the FPU works.
2. There is no proven benefit of this narrow optimization and x86 maintainers are already skeptical.
"""

This optimization can be added back if it benefits kernel shadow stacks. But os_xrstor_supervisor() and os_xrstor_safe() would need the same handling.

[1]: https://lore.kernel.org/all/[email protected]/

#5 lore raw
Quoted 11 lines · Chao Gao

We then switched to the guest-only approach to avoid allocating CET_KERNEL space in non-KVM fpstates. The optimization could still be applied on top of that, but it was dropped after its benefit was questioned off-list:

"""
1. It hacks in an optimization with conditionals around the edges, instead of integrating with the way the rest of the FPU works.
2. There is no proven benefit of this narrow optimization and x86 maintainers are already skeptical.
"""

Quoted 3 lines

This optimization can be added back if it benefits kernel shadow stacks. But os_xrstor_supervisor() and os_xrstor_safe() would need the same handling.

That makes it more functional and less of an optimization. But, no less of a hack.