Skip to contentSkip to conversation

x86: IST cleanup

PATCH In review Andrew Cooper 3 participants 12 messages Apr 20, 2020 – Apr 24, 2020

lore
Share & tools
lore
#0 lore raw

cover x86: IST cleanup

Various bits of cleanup for !CONFIG_PV builds.

Andrew Cooper (3):
x86/S3: Use percpu_traps_init() rather than opencoding SYSCALL/SYSENTER restoration x86/boot: Don't enable EFER.SCE for !CONFIG_PV builds
x86/pv: Don't use IST for NMI/#MC/#DB in !CONFIG_PV builds

 xen/arch/x86/acpi/suspend.c     | 25 ++-----------------------
 xen/arch/x86/boot/trampoline.S  |  4 ++--
 xen/arch/x86/cpu/common.c       |  8 +++++---
 xen/arch/x86/efi/efi-boot.h     |  3 ++-
 xen/arch/x86/hvm/vmx/vmx.c      |  4 ++++
 xen/arch/x86/x86_64/traps.c     | 19 ++++++-------------
 xen/include/asm-x86/processor.h | 12 +++++++++++-
 7 files changed, 32 insertions(+), 43 deletions(-)
Signature · 1 line
2.11.0
#1 lore raw

PATCH 1/3 x86/S3: Use percpu_traps_init() rather than opencoding SYSCALL/SYSENTER restoration

This make the S3 BSP path consistent with AP paths, and reduces the amount of state needing stashing specially. Also, it takes care of re-setting up Xen's LBR configuration if requested, which was missing previously.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

---
 xen/arch/x86/acpi/suspend.c | 25 ++-----------------------
 1 file changed, 2 insertions(+), 23 deletions(-)
xen/arch/x86/acpi/suspend.c +2 -23
index 32d0f71ffd..1c2f1c470e 100644
@@ -15,8 +15,6 @@
 #include <asm/xstate.h>
 #include <xen/hypercall.h>
 
-static unsigned long saved_lstar, saved_cstar;
-static unsigned long saved_sysenter_esp, saved_sysenter_eip;
 static unsigned long saved_fs_base, saved_gs_base, saved_kernel_gs_base;
 static uint64_t saved_xcr0;
 
@@ -25,14 +23,6 @@ void save_rest_processor_state(void)
     saved_fs_base = rdfsbase();
     saved_gs_base = rdgsbase();
     rdmsrl(MSR_SHADOW_GS_BASE, saved_kernel_gs_base);
-    rdmsrl(MSR_CSTAR, saved_cstar);
-    rdmsrl(MSR_LSTAR, saved_lstar);
-
-    if ( cpu_has_sep )
-    {
-        rdmsrl(MSR_IA32_SYSENTER_ESP, saved_sysenter_esp);
-        rdmsrl(MSR_IA32_SYSENTER_EIP, saved_sysenter_eip);
-    }
 
     if ( cpu_has_xsave )
         saved_xcr0 = get_xcr0();
@@ -46,24 +36,13 @@ void restore_rest_processor_state(void)
     /* Restore full CR4 (inc MCE) now that the IDT is in place. */
     write_cr4(mmu_cr4_features);
 
-    /* Recover syscall MSRs */
-    wrmsrl(MSR_LSTAR, saved_lstar);
-    wrmsrl(MSR_CSTAR, saved_cstar);
-    wrmsrl(MSR_STAR, XEN_MSR_STAR);
-    wrmsrl(MSR_SYSCALL_MASK, XEN_SYSCALL_MASK);
+    /* (re)initialise SYSCALL/SYSENTER state, amongst other things. */
+    percpu_traps_init();
 
     wrfsbase(saved_fs_base);
     wrgsbase(saved_gs_base);
     wrmsrl(MSR_SHADOW_GS_BASE, saved_kernel_gs_base);
 
-    if ( cpu_has_sep )
-    {
-        /* Recover sysenter MSRs */
-        wrmsrl(MSR_IA32_SYSENTER_ESP, saved_sysenter_esp);
-        wrmsrl(MSR_IA32_SYSENTER_EIP, saved_sysenter_eip);
-        wrmsr(MSR_IA32_SYSENTER_CS, __HYPERVISOR_CS, 0);
-    }
-
     if ( cpu_has_xsave && !set_xcr0(saved_xcr0) )
         BUG();
 
Signature · 1 line
2.11.0
Acked-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] x86/S3: Use percpu_traps_init() rather than opencoding SYSCALL/SYSENTER restoration

xen/arch/x86/acpi/suspend.c void restore_rest_processor_state(void) · line 40

Quoted 11 lines · Andrew Cooper
xen/arch/x86/acpi/suspend.c +2 -5
@@ -46,24 +36,13 @@ void restore_rest_processor_state(void)
     /* Restore full CR4 (inc MCE) now that the IDT is in place. */
     write_cr4(mmu_cr4_features);
 
-    /* Recover syscall MSRs */
-    wrmsrl(MSR_LSTAR, saved_lstar);
-    wrmsrl(MSR_CSTAR, saved_cstar);
-    wrmsrl(MSR_STAR, XEN_MSR_STAR);
-    wrmsrl(MSR_SYSCALL_MASK, XEN_SYSCALL_MASK);
+    /* (re)initialise SYSCALL/SYSENTER state, amongst other things. */
+    percpu_traps_init();

Without tweaks to subarch_percpu_traps_init() this assumes that, now and going forward, map_domain_page() will work reliably at this early point of resume. I'm not opposed, i.e.

but it would feel less fragile if the redundant re-writing of the stubs would be skipped in the BSP resume case (I didn't check whether it's also redundant for APs, but I suspect it's not, as the stub pages may get allocated anew).

Jan

#3 lore raw

Re: [PATCH 1/3] x86/S3: Use percpu_traps_init() rather than opencoding SYSCALL/SYSENTER restoration

Quoted 15 lines · Jan Beulich

Review void restore_rest_processor_state(void) · line 40

Quoted 11 lines · Andrew Cooper
(quoted diff) +2 -5
@@ -46,24 +36,13 @@ void restore_rest_processor_state(void)
     /* Restore full CR4 (inc MCE) now that the IDT is in place. */
     write_cr4(mmu_cr4_features);
 
-    /* Recover syscall MSRs */
-    wrmsrl(MSR_LSTAR, saved_lstar);
-    wrmsrl(MSR_CSTAR, saved_cstar);
-    wrmsrl(MSR_STAR, XEN_MSR_STAR);
-    wrmsrl(MSR_SYSCALL_MASK, XEN_SYSCALL_MASK);
+    /* (re)initialise SYSCALL/SYSENTER state, amongst other things. */
+    percpu_traps_init();

Without tweaks to subarch_percpu_traps_init() this assumes that, now and going forward, map_domain_page() will work reliably at this early point of resume. I'm not opposed, i.e.

Quoted 1 line

I think this reasonable to expect, and robust going forward.

We are properly in d[IDLE]v0 context when it comes to pagetables, and there is nothing interesting between this point and coming fully back online.

That said, I could easily move the call to later in the resume path for even more certainty.

xen/arch/x86/acpi/power.c +0 -0
index 3ad7dfc9a3..d5a468eddd 100644
@@ -297,6 +297,8 @@ static int enter_state(u32 state)

     ci->spec_ctrl_flags |= (default_spec_ctrl_flags & SCF_ist_wrmsr); spec_ctrl_exit_idle(ci);

+    /* (re)initialise SYSCALL/SYSENTER state, amongst other things. */ +    percpu_traps_init();
+
  done:
     spin_debug_enable();
     local_irq_restore(flags);

In fact - I prefer this, because it works towards one low priority goal of deleting {save,restore}_rest_processor_state() which I've still got a pending series for.

Would your ack still stand if I tweak the patch in this way?

Quoted 4 lines

but it would feel less fragile if the redundant re-writing of the stubs would be skipped in the BSP resume case (I didn't check whether it's also redundant for APs, but I suspect it's not, as the stub pages may get allocated anew).

I don't really agree.  Symmetry (even if it is expected to be redundant) is much more easy to reason about in terms of robustness.  S3 is not a fastpath.

~Andrew

#4 lore raw

Re: [PATCH 1/3] x86/S3: Use percpu_traps_init() rather than opencoding SYSCALL/SYSENTER restoration

Quoted 42 lines · Andrew Cooper
Quoted 15 lines · Jan Beulich

Review void restore_rest_processor_state(void) · line 40

Quoted 11 lines · Andrew Cooper
(quoted diff) +2 -5
@@ -46,24 +36,13 @@ void restore_rest_processor_state(void)
     /* Restore full CR4 (inc MCE) now that the IDT is in place. */
     write_cr4(mmu_cr4_features);
 
-    /* Recover syscall MSRs */
-    wrmsrl(MSR_LSTAR, saved_lstar);
-    wrmsrl(MSR_CSTAR, saved_cstar);
-    wrmsrl(MSR_STAR, XEN_MSR_STAR);
-    wrmsrl(MSR_SYSCALL_MASK, XEN_SYSCALL_MASK);
+    /* (re)initialise SYSCALL/SYSENTER state, amongst other things. */
+    percpu_traps_init();

Without tweaks to subarch_percpu_traps_init() this assumes that, now and going forward, map_domain_page() will work reliably at this early point of resume. I'm not opposed, i.e.

Quoted 1 line

I think this reasonable to expect, and robust going forward.

We are properly in d[IDLE]v0 context when it comes to pagetables, and there is nothing interesting between this point and coming fully back online.

That said, I could easily move the call to later in the resume path for even more certainty.

xen/arch/x86/acpi/power.c +0 -0
index 3ad7dfc9a3..d5a468eddd 100644
@@ -297,6 +297,8 @@ static int enter_state(u32 state)
      ci->spec_ctrl_flags |= (default_spec_ctrl_flags & SCF_ist_wrmsr);
      spec_ctrl_exit_idle(ci);


+    /* (re)initialise SYSCALL/SYSENTER state, amongst other things. */ +    percpu_traps_init();
+
  done:
     spin_debug_enable();
     local_irq_restore(flags);

Quoted 5 lines

In fact - I prefer this, because it works towards one low priority goal of deleting {save,restore}_rest_processor_state() which I've still got a pending series for.

Would your ack still stand if I tweak the patch in this way?

Yes.

Jan

#5 lore raw

PATCH 2/3 x86/boot: Don't enable EFER.SCE for !CONFIG_PV builds

In reply to Andrew Cooper

This will cause all SYSCALL/SYSRET instructions to suffer #UD rather than following the MSR_{L,C}STAR pointers, allowing us to drop the star_enter() panic helper, allowing us to clean up the IST stacks in a subsequent patch.

Drop the now-dead conditional SYSENTER logic in the middle of subarch_percpu_traps_init().

In addition, vmx_restore_host_msrs() need not restore any host state. (Regarding the asymmetric changes, VT-x automatically restores SYSENTER state on vmexit, and SVM restores both SYSCALL/SYSENTER state with the VMSAVE/VMLOAD instructions.)

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Kevin Tian <[email protected]>

---
 xen/arch/x86/boot/trampoline.S |  4 ++--
 xen/arch/x86/efi/efi-boot.h    |  3 ++-
 xen/arch/x86/hvm/vmx/vmx.c     |  4 ++++
 xen/arch/x86/x86_64/traps.c    | 19 ++++++-------------
 4 files changed, 14 insertions(+), 16 deletions(-)
xen/arch/x86/boot/trampoline.S +2 -2
index 18c6638924..928b4ad4ce 100644
@@ -140,9 +140,9 @@ gdt_48:
 GLOBAL(trampoline_misc_enable_off)
         .quad   0
 
-/* EFER OR-mask for boot paths.  This gets adjusted with NX when available. */
+/* EFER OR-mask for boot paths.  SCE conditional on PV support, NX added when available. */
 GLOBAL(trampoline_efer)
-        .long   EFER_LME | EFER_SCE
+        .long   EFER_LME | (EFER_SCE * IS_ENABLED(CONFIG_PV))
 
 GLOBAL(trampoline_xen_phys_start)
         .long   0
xen/arch/x86/efi/efi-boot.h +2 -1
index a13304201f..bf0fae4f89 100644
@@ -238,7 +238,8 @@ static void __init noreturn efi_arch_post_exit_boot(void)
     /* Set system registers and transfer control. */
     asm volatile("pushq $0\n\tpopfq");
     rdmsrl(MSR_EFER, efer);
-    efer |= EFER_SCE;
+    if ( IS_ENABLED(CONFIG_PV) )
+        efer |= EFER_SCE;
     if ( cpu_has_nx )
         efer |= EFER_NX;
     wrmsrl(MSR_EFER, efer);
xen/arch/x86/hvm/vmx/vmx.c +4 -0
index 869339062b..3e4b2e9a58 100644
@@ -496,6 +496,10 @@ static void vmx_vcpu_destroy(struct vcpu *v)
  */
 static void vmx_restore_host_msrs(void)
 {
+    /* No PV guests?  No need to set restore host SYSCALL infrastructure. */
+    if ( !IS_ENABLED(CONFIG_PV) )
+        return;
+
     /* Relies on the SYSCALL trampoline being at the start of the stubs. */
     wrmsrl(MSR_STAR,         XEN_MSR_STAR);
     wrmsrl(MSR_LSTAR,        this_cpu(stubs.addr));
xen/arch/x86/x86_64/traps.c +6 -13
index c3d4faea6b..93af0c5e87 100644
@@ -299,17 +299,8 @@ static unsigned int write_stub_trampoline(
 
 DEFINE_PER_CPU(struct stubs, stubs);
 
-#ifdef CONFIG_PV
 void lstar_enter(void);
 void cstar_enter(void);
-#else
-static void __cold star_enter(void)
-{
-    panic("lstar/cstar\n");
-}
-#define lstar_enter star_enter
-#define cstar_enter star_enter
-#endif /* CONFIG_PV */
 
 void subarch_percpu_traps_init(void)
 {
@@ -321,6 +312,10 @@ void subarch_percpu_traps_init(void)
     /* IST_MAX IST pages + at least 1 guard page + primary stack. */
     BUILD_BUG_ON((IST_MAX + 1) * PAGE_SIZE + PRIMARY_STACK_SIZE > STACK_SIZE);
 
+    /* No PV guests?  No need to set up SYSCALL/SYSENTER infrastructure. */
+    if ( !IS_ENABLED(CONFIG_PV) )
+        return;
+
     stub_page = map_domain_page(_mfn(this_cpu(stubs.mfn)));
 
     /*
@@ -338,10 +333,8 @@ void subarch_percpu_traps_init(void)
     {
         /* SYSENTER entry. */
         wrmsrl(MSR_IA32_SYSENTER_ESP, stack_bottom);
-        wrmsrl(MSR_IA32_SYSENTER_EIP,
-               IS_ENABLED(CONFIG_PV) ? (unsigned long)sysenter_entry : 0);
-        wrmsr(MSR_IA32_SYSENTER_CS,
-              IS_ENABLED(CONFIG_PV) ? __HYPERVISOR_CS : 0, 0);
+        wrmsrl(MSR_IA32_SYSENTER_EIP, (unsigned long)sysenter_entry);
+        wrmsr(MSR_IA32_SYSENTER_CS, __HYPERVISOR_CS, 0);
     }
 
     /* Trampoline for SYSCALL entry from compatibility mode. */
Signature · 1 line
2.11.0
Reviewed-by on v1 2/3
#6 lore raw

Re: [PATCH 2/3] x86/boot: Don't enable EFER.SCE for !CONFIG_PV builds

xen/arch/x86/efi/efi-boot.h static void __init noreturn efi_arch_post_exit_boot(void) · line 245

Quoted 12 lines · Andrew Cooper
xen/arch/x86/efi/efi-boot.h +2 -1
@@ -238,7 +238,8 @@ static void __init noreturn efi_arch_post_exit_boot(void)
     /* Set system registers and transfer control. */
     asm volatile("pushq $0\n\tpopfq");
     rdmsrl(MSR_EFER, efer);
-    efer |= EFER_SCE;
+    if ( IS_ENABLED(CONFIG_PV) )
+        efer |= EFER_SCE;
     if ( cpu_has_nx )
         efer |= EFER_NX;
     wrmsrl(MSR_EFER, efer);

Switch to simply ORing in trampoline_efer here?

With or without the adjustment

Jan

Reviewed-by on v1 2/3
#7 lore raw

RE: [PATCH 2/3] x86/boot: Don't enable EFER.SCE for !CONFIG_PV builds

In reply to Andrew Cooper
Quoted 10 lines

From: Andrew Cooper <[email protected]>
Sent: Monday, April 20, 2020 10:59 PM

This will cause all SYSCALL/SYSRET instructions to suffer #UD rather than following the MSR_{L,C}STAR pointers, allowing us to drop the star_enter() panic helper, allowing us to clean up the IST stacks in a subsequent patch.

Drop the now-dead conditional SYSENTER logic in the middle of subarch_percpu_traps_init().

Quoted 7 lines

In addition, vmx_restore_host_msrs() need not restore any host state. (Regarding the asymmetric changes, VT-x automatically restores SYSENTER state on vmexit, and SVM restores both SYSCALL/SYSENTER state with
the VMSAVE/VMLOAD instructions.)

#8 lore raw

PATCH 3/3 x86/pv: Don't use IST for NMI/#MC/#DB in !CONFIG_PV builds

In reply to Andrew Cooper

ISTs are used to force a stack switch on CPL0=>0 interrupts/exceptions. They however come with a nasty corner case in the case of reentrancy where the outer exception frame gets clobbered.

When the SYSCALL/SYSRET instructions aren't used, there is no need to use IST for anything other than #DF, which reduces the number of corner cases.

---
CC: Jan Beulich <[email protected]>
CC: Wei Liu <[email protected]>
CC: Roger Pau Monné <[email protected]>

---
 xen/arch/x86/cpu/common.c       |  8 +++++---
 xen/include/asm-x86/processor.h | 12 +++++++++++-
 2 files changed, 16 insertions(+), 4 deletions(-)
xen/arch/x86/cpu/common.c +5 -3
index 7b093cb421..d45495c701 100644
@@ -732,15 +732,17 @@ void load_system_tables(void)
 		.rsp2 = 0x8600111111111111ul,
 
 		/*
-		 * MCE, NMI and Double Fault handlers get their own stacks.
+		 * #DF always uses a separate stack. NMI/#MC/#DB only need a
+		 * separate stacks when PV guests are used.
 		 * All others poisoned.
 		 */
 		.ist = {
-			[IST_MCE - 1] = stack_top + IST_MCE * PAGE_SIZE,
 			[IST_DF  - 1] = stack_top + IST_DF  * PAGE_SIZE,
+#ifdef CONFIG_PV
 			[IST_NMI - 1] = stack_top + IST_NMI * PAGE_SIZE,
+			[IST_MCE - 1] = stack_top + IST_MCE * PAGE_SIZE,
 			[IST_DB  - 1] = stack_top + IST_DB  * PAGE_SIZE,
-
+#endif
 			[IST_MAX ... ARRAY_SIZE(tss->ist) - 1] =
 				0x8600111111111111ul,
 		},
xen/include/asm-x86/processor.h +11 -1
index ea6e5497f4..33f2052c8e 100644
@@ -441,12 +441,18 @@ struct tss_page {
 };
 DECLARE_PER_CPU(struct tss_page, tss_page);
 
+/*
+ * Interrupt Stack Tables.  Used to force a stack switch on a CPL0=>0
+ * interrupt/exception.  #DF uses IST all the time to detect stack overflows
+ * cleanly.  NMI/#MC/#DB only need IST to cover the SYSCALL gap, and therefore
+ * only necessary with PV guests.
+ */
 #define IST_NONE 0UL
 #define IST_DF   1UL
 #define IST_NMI  2UL
 #define IST_MCE  3UL
 #define IST_DB   4UL
-#define IST_MAX  4UL
+#define IST_MAX  (IS_ENABLED(CONFIG_PV) ? 4ul : 1ul)
 
 /* Set the Interrupt Stack Table used by a particular IDT entry. */
 static inline void set_ist(idt_entry_t *idt, unsigned int ist)
@@ -461,6 +467,8 @@ static inline void set_ist(idt_entry_t *idt, unsigned int ist)
 static inline void enable_each_ist(idt_entry_t *idt)
 {
     set_ist(&idt[TRAP_double_fault],  IST_DF);
+    if ( !IS_ENABLED(CONFIG_PV) )
+        return;
     set_ist(&idt[TRAP_nmi],           IST_NMI);
     set_ist(&idt[TRAP_machine_check], IST_MCE);
     set_ist(&idt[TRAP_debug],         IST_DB);
@@ -469,6 +477,8 @@ static inline void enable_each_ist(idt_entry_t *idt)
 static inline void disable_each_ist(idt_entry_t *idt)
 {
     set_ist(&idt[TRAP_double_fault],  IST_NONE);
+    if ( !IS_ENABLED(CONFIG_PV) )
+        return;
     set_ist(&idt[TRAP_nmi],           IST_NONE);
     set_ist(&idt[TRAP_machine_check], IST_NONE);
     set_ist(&idt[TRAP_debug],         IST_NONE);
Signature · 1 line
2.11.0
#9 lore raw

Re: [PATCH 3/3] x86/pv: Don't use IST for NMI/#MC/#DB in !CONFIG_PV builds

xen/include/asm-x86/processor.h struct tss_page { · line 449

Quoted 12 lines · Andrew Cooper
xen/include/asm-x86/processor.h +6 -0
@@ -441,12 +441,18 @@ struct tss_page {
 };
 DECLARE_PER_CPU(struct tss_page, tss_page);
 
+/*
+ * Interrupt Stack Tables.  Used to force a stack switch on a CPL0=>0
+ * interrupt/exception.  #DF uses IST all the time to detect stack overflows
+ * cleanly.  NMI/#MC/#DB only need IST to cover the SYSCALL gap, and therefore
+ * only necessary with PV guests.
+ */

Is it really only the SYSCALL gap that we mean to cover? In particular for #MC I'd suspect it is a good idea to switch stacks as well, to get onto a distinct memory page in case the #MC was stack related. With NMI it might as well be better to switch; I agree we don't need any switching for #DB.

I also think that the comment at the top of current.h would want updating with these adjustments (which I notice lacks the #DB part anyway).

Jan

#10 lore raw

Re: [PATCH 3/3] x86/pv: Don't use IST for NMI/#MC/#DB in !CONFIG_PV builds

Quoted 7 lines · Jan Beulich
Quoted 6 lines · Andrew Cooper
xen/include/asm-x86/processor.h +0 -0
@@ -441,12 +441,18 @@ struct tss_page {
 };
 DECLARE_PER_CPU(struct tss_page, tss_page);
 
Quoted 9 lines

xen/include/asm-x86/processor.h struct tss_page { · line 449

Quoted 6 lines
xen/include/asm-x86/processor.h +6 -0
+/*
+ * Interrupt Stack Tables.  Used to force a stack switch on a CPL0=>0
+ * interrupt/exception.  #DF uses IST all the time to detect stack overflows
+ * cleanly.  NMI/#MC/#DB only need IST to cover the SYSCALL gap, and therefore
+ * only necessary with PV guests.
+ */

Is it really only the SYSCALL gap that we mean to cover? In particular for #MC I'd suspect it is a good idea to switch stacks as well, to get onto a distinct memory page in case the #MC was stack related.

If #MC occurs on your stack, you have already lost.  The chances of only taking a single #MC are tiny because the next-line prefetcher will be doing its job (or it hits when the lines (plural) leave L3, which will be in guest context at some point in the future.)

The very best you can hope for is to cleanly print something and crash - even if you manage to run the handler, you've got no idea if the interrupted context had a spinlock held, and Xen has no support for changing to a different pcpu stack.

Quoted 1 line

With NMI it might as well be better to switch;

Why?  There is no benefit (with no SYSCALL in the picture), and a downside which causes state loss.

Quoted 6 lines

I agree we don't need any
switching for #DB.

I also think that the comment at the top of current.h would want updating with these adjustments (which I notice lacks the #DB part anyway).

Oops - I totally forgot that for the XSA-260 fix.

~Andrew

Reviewed-by
#11 lore raw

Re: [PATCH 3/3] x86/pv: Don't use IST for NMI/#MC/#DB in !CONFIG_PV builds

Quoted 18 lines · Andrew Cooper
Quoted 7 lines · Jan Beulich
Quoted 6 lines · Andrew Cooper
xen/include/asm-x86/processor.h +0 -0
@@ -441,12 +441,18 @@ struct tss_page {
 };
 DECLARE_PER_CPU(struct tss_page, tss_page);
 
Quoted 9 lines

xen/include/asm-x86/processor.h struct tss_page { · line 449

Quoted 6 lines
xen/include/asm-x86/processor.h +6 -0
+/*
+ * Interrupt Stack Tables.  Used to force a stack switch on a CPL0=>0
+ * interrupt/exception.  #DF uses IST all the time to detect stack overflows
+ * cleanly.  NMI/#MC/#DB only need IST to cover the SYSCALL gap, and therefore
+ * only necessary with PV guests.
+ */

Is it really only the SYSCALL gap that we mean to cover? In particular for #MC I'd suspect it is a good idea to switch stacks as well, to get onto a distinct memory page in case the #MC was stack related.

Quoted 4 lines

If #MC occurs on your stack, you have already lost.  The chances of only taking a single #MC are tiny because the next-line prefetcher will be doing its job (or it hits when the lines (plural) leave L3, which will be in guest context at some point in the future.)

It would seem fishy behavior to me for the hardware to continue issuing prefetches against a page that has just been noticed to cause issues when accessed. Surely hardware at least _could_ internally mark such a page #UC or some such, to prevent further prefetches to actually hit the bus.

Quoted 4 lines

The very best you can hope for is to cleanly print something and crash - even if you manage to run the handler, you've got no idea if the interrupted context had a spinlock held, and Xen has no support for changing to a different pcpu stack.

But getting something printed is magnitudes better that hanging or rebooting entirely silently.

Nevertheless, as long as you extend the description somewhat to express this reasoning in some way

(a little hesitantly though).

Jan