Skip to contentSkip to conversation

x86/kexec: Fix kexec-reboot with CET active

PATCH In review Andrew Cooper 3 participants 6 messages Mar 7, 2022 – Mar 30, 2022

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/kexec: Fix kexec-reboot with CET active

The kexec_reloc() asm has an indirect jump to relocate onto the identity trampoline. While we clear CET in machine_crash_shutdown(), we fail to clear CET for the non-crash path. This in turn highlights that the same is true of resetting the CPUID masking/faulting.

Move both pieces of logic from machine_crash_shutdown() to machine_kexec(), the latter being common for all kexec transitions. Adjust the condition for CET being considered active to check in CR4, which is simpler and more robust.

Fixes 311434bfc9d1 ("x86/setup: Rework MSR_S_CET handling for CET-IBT")
Fixes b60ab42db2f0 ("x86/shstk: Activate Supervisor Shadow Stacks")
Fixes 5ab9564c6fa1 ("x86/cpu: Context switch cpuid masks and faulting state in context_switch()")
Reported-by David Vrabel (XXX which alias to use?)

---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
CC: David Vrabel <[email protected]>

---
 xen/arch/x86/crash.c         | 10 ----------
 xen/arch/x86/machine_kexec.c | 10 ++++++++++
 2 files changed, 10 insertions(+), 10 deletions(-)
xen/arch/x86/crash.c +0 -10
index 003222c0f1ac..99089f77a7eb 100644
@@ -187,16 +187,6 @@ void machine_crash_shutdown(void)
 
     nmi_shootdown_cpus();
 
-    /* Reset CPUID masking and faulting to the host's default. */
-    ctxt_switch_levelling(NULL);
-
-    /* Disable CET. */
-    if ( cpu_has_xen_shstk || cpu_has_xen_ibt )
-    {
-        wrmsrl(MSR_S_CET, 0);
-        write_cr4(read_cr4() & ~X86_CR4_CET);
-    }
-
     info = kexec_crash_save_info();
     info->xen_phys_start = xen_phys_start;
     info->dom0_pfn_to_mfn_frame_list_list =
xen/arch/x86/machine_kexec.c +10 -0
index 751a9efcaf6a..d83aa4e7e93b 100644
@@ -156,6 +156,16 @@ void machine_kexec(struct kexec_image *image)
      */
     local_irq_disable();
 
+    /* Reset CPUID masking and faulting to the host's default. */
+    ctxt_switch_levelling(NULL);
+
+    /* Disable CET. */
+    if ( read_cr4() & X86_CR4_CET )
+    {
+        wrmsrl(MSR_S_CET, 0);
+        write_cr4(read_cr4() & ~X86_CR4_CET);
+    }
+
     /* Now regular interrupts are disabled, we need to reduce the impact
      * of interrupts not disabled by 'cli'.
      *
Signature · 1 line
2.11.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 8 lines · Andrew Cooper

The kexec_reloc() asm has an indirect jump to relocate onto the identity trampoline. While we clear CET in machine_crash_shutdown(), we fail to clear CET for the non-crash path. This in turn highlights that the same is true of resetting the CPUID masking/faulting.

Move both pieces of logic from machine_crash_shutdown() to machine_kexec(), the latter being common for all kexec transitions. Adjust the condition for CET being considered active to check in CR4, which is simpler and more robust.

Quoted 4 lines
Fixes 311434bfc9d1 ("x86/setup: Rework MSR_S_CET handling for CET-IBT")
Fixes b60ab42db2f0 ("x86/shstk: Activate Supervisor Shadow Stacks")
Fixes 5ab9564c6fa1 ("x86/cpu: Context switch cpuid masks and faulting state in context_switch()")
Reported-by David Vrabel (XXX which alias to use?)

Amazon, please.

David

#2 lore raw
In reply to Andrew Cooper
Quoted 9 lines · Andrew Cooper
xen/arch/x86/machine_kexec.c +3 -0
@@ -156,6 +156,16 @@ void machine_kexec(struct kexec_image *image)
      */
     local_irq_disable();
 
+    /* Reset CPUID masking and faulting to the host's default. */
+    ctxt_switch_levelling(NULL);
+

xen/arch/x86/machine_kexec.c void machine_kexec(struct kexec_image *image) · line 171

Quoted 10 lines
xen/arch/x86/machine_kexec.c +7 -0
+    /* Disable CET. */
+    if ( read_cr4() & X86_CR4_CET )
+    {
+        wrmsrl(MSR_S_CET, 0);
+        write_cr4(read_cr4() & ~X86_CR4_CET);
+    }
+
     /* Now regular interrupts are disabled, we need to reduce the impact
      * of interrupts not disabled by 'cli'.
      *

Besides introducing somewhat of a disconnect between the comment in context here and the earlier local_irq_disable(), is it really necessary to do both actions with IRQs off?

Jan

#3 lore raw
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Andrew Cooper
xen/arch/x86/machine_kexec.c +0 -0
@@ -156,6 +156,16 @@ void machine_kexec(struct kexec_image *image)
      */
     local_irq_disable();
 
Quoted 16 lines

xen/arch/x86/machine_kexec.c void machine_kexec(struct kexec_image *image) · line 171

Quoted 13 lines
xen/arch/x86/machine_kexec.c +10 -0
+    /* Reset CPUID masking and faulting to the host's default. */
+    ctxt_switch_levelling(NULL);
+
+    /* Disable CET. */
+    if ( read_cr4() & X86_CR4_CET )
+    {
+        wrmsrl(MSR_S_CET, 0);
+        write_cr4(read_cr4() & ~X86_CR4_CET);
+    }
+
     /* Now regular interrupts are disabled, we need to reduce the impact
      * of interrupts not disabled by 'cli'.
      *

Besides introducing somewhat of a disconnect between the comment in context here and the earlier local_irq_disable(), is it really necessary to do both actions with IRQs off?

We are a handful of instructions away from discarding Xen's context entirely.  IRQs are not a relevant concern.

If we're nitpicking, irqs want to be off before kexecing gets set, because absolutely nothing good can come of handling interrupts later than that point.

~Andrew

#4 lore raw
Quoted 25 lines · Andrew Cooper
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Andrew Cooper
xen/arch/x86/machine_kexec.c +0 -0
@@ -156,6 +156,16 @@ void machine_kexec(struct kexec_image *image)
      */
     local_irq_disable();
 
Quoted 16 lines

xen/arch/x86/machine_kexec.c void machine_kexec(struct kexec_image *image) · line 171

Quoted 13 lines
xen/arch/x86/machine_kexec.c +10 -0
+    /* Reset CPUID masking and faulting to the host's default. */
+    ctxt_switch_levelling(NULL);
+
+    /* Disable CET. */
+    if ( read_cr4() & X86_CR4_CET )
+    {
+        wrmsrl(MSR_S_CET, 0);
+        write_cr4(read_cr4() & ~X86_CR4_CET);
+    }
+
     /* Now regular interrupts are disabled, we need to reduce the impact
      * of interrupts not disabled by 'cli'.
      *

Besides introducing somewhat of a disconnect between the comment in context here and the earlier local_irq_disable(), is it really necessary to do both actions with IRQs off?

Quoted 2 lines

We are a handful of instructions away from discarding Xen's context entirely.  IRQs are not a relevant concern.

Well, as said - the comment was what caught my eye. But as you appear to think that slight disconnect is not an issue: I don't mean my remark to be an objection. Feel free to commit with David's R-b.

Jan

#5 lore raw
In reply to Andrew Cooper
Quoted 5 lines · Andrew Cooper
xen/arch/x86/machine_kexec.c +0 -0
@@ -156,6 +156,16 @@ void machine_kexec(struct kexec_image *image)
      */
     local_irq_disable();

xen/arch/x86/machine_kexec.c void machine_kexec(struct kexec_image *image) · line 167

Quoted 10 lines
xen/arch/x86/machine_kexec.c +9 -0
 
+    /* Reset CPUID masking and faulting to the host's default. */
+    ctxt_switch_levelling(NULL);
+
+    /* Disable CET. */
+    if ( read_cr4() & X86_CR4_CET )
+    {
+        wrmsrl(MSR_S_CET, 0);
+        write_cr4(read_cr4() & ~X86_CR4_CET);
+    }

It just occurred to me: Isn't using read_cr4() here somewhat risky? If we crashed on the CR4 write in write_cr4(), we'd try to load CR4 here again with a value which may cause another fault.

Jan