Skip to contentSkip to conversation

x86/S3: Restore CR4 earlier during resume

PATCH In review Andrew Cooper 3 participants 4 messages Oct 2, 2020 – Oct 5, 2020

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 x86/S3: Restore CR4 earlier during resume

c/s 4304ff420e5 "x86/S3: Drop {save,restore}_rest_processor_state() completely" moved CR4 restoration up into C, to account for the fact that MCE was explicitly handled later.

However, time_resume() ends up making an EFI Runtime Service call, and EFI explodes without OSFXSR, presumably when trying to spill %xmm registers onto the stack.

Given this codepath, and the potential for other issues of a similar kind (TLB flushing vs INVPCID, HVM logic vs VMXE, etc), restore CR4 in asm before entering C.

Ignore the previous MCE special case, because its not actually necessary. The handler is already suitably configured from before suspend.

Fixes 4304ff420e5 ("x86/S3: Drop {save,restore}_rest_processor_state() completely")

---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>

This is one definite bug fix. It doesn't appear to be the only S3 bug however.

---
 xen/arch/x86/acpi/power.c       | 3 ---
 xen/arch/x86/acpi/wakeup_prot.S | 5 +++++
 2 files changed, 5 insertions(+), 3 deletions(-)
xen/arch/x86/acpi/power.c +0 -3
index 4fb1e7a148..7f162a4df9 100644
@@ -276,9 +276,6 @@ static int enter_state(u32 state)
 
     mcheck_init(&boot_cpu_data, false);
 
-    /* Restore CR4 from cached value, now MCE is set up. */
-    write_cr4(read_cr4());
-
     printk(XENLOG_INFO "Finishing wakeup from ACPI S%d state.\n", state);
 
     if ( (state == ACPI_STATE_S3) && error )
xen/arch/x86/acpi/wakeup_prot.S +5 -0
index c6b3fcc93d..1ee5551fb5 100644
@@ -110,6 +110,11 @@ ENTRY(s3_resume)
 
         call    load_system_tables
 
+        /* Restore CR4 from the cpuinfo block. */
+        GET_STACK_END(bx)
+        mov     STACK_CPUINFO_FIELD(cr4)(%rbx), %rax
+        mov     %rax, %cr4
+
 .Lsuspend_err:
         pop     %r15
         pop     %r14
Signature · 1 line
2.11.0
Reviewed-by on v1 1/1
#1 lore raw
Quoted 8 lines · Andrew Cooper

c/s 4304ff420e5 "x86/S3: Drop {save,restore}_rest_processor_state() completely" moved CR4 restoration up into C, to account for the fact that MCE was explicitly handled later.

However, time_resume() ends up making an EFI Runtime Service call, and EFI explodes without OSFXSR, presumably when trying to spill %xmm registers onto the stack.

Quoted 6 lines

Given this codepath, and the potential for other issues of a similar kind (TLB flushing vs INVPCID, HVM logic vs VMXE, etc), restore CR4 in asm before entering C.

Ignore the previous MCE special case, because its not actually necessary. The handler is already suitably configured from before suspend.

Are you suggesting we could drop the call to mcheck_init() altogether?

Quoted 3 lines
Fixes 4304ff420e5 ("x86/S3: Drop {save,restore}_rest_processor_state() completely")

Jan

#2 lore raw
Quoted 9 lines · Jan Beulich
Quoted 8 lines · Andrew Cooper

c/s 4304ff420e5 "x86/S3: Drop {save,restore}_rest_processor_state() completely" moved CR4 restoration up into C, to account for the fact that MCE was explicitly handled later.

However, time_resume() ends up making an EFI Runtime Service call, and EFI explodes without OSFXSR, presumably when trying to spill %xmm registers onto the stack.

Quoted 7 lines
Quoted 6 lines

Given this codepath, and the potential for other issues of a similar kind (TLB flushing vs INVPCID, HVM logic vs VMXE, etc), restore CR4 in asm before entering C.

Ignore the previous MCE special case, because its not actually necessary. The handler is already suitably configured from before suspend.

Are you suggesting we could drop the call to mcheck_init() altogether?

Not completely.  It reconfigures some of the MCE bank controls, which probably won't survive S3, but the #MC handler itself is fully intact once the IDT is re-established.

It probably wants splitting in two, but I think some part of it needs to remain.

Quoted 5 lines
Quoted 4 lines
Fixes 4304ff420e5 ("x86/S3: Drop {save,restore}_rest_processor_state() completely")

Thanks,

~Andrew

Tested-by on v1 1/1
#3 lore raw
In reply to Andrew Cooper
Quoted 12 lines · Andrew Cooper

c/s 4304ff420e5 "x86/S3: Drop {save,restore}_rest_processor_state() completely" moved CR4 restoration up into C, to account for the fact that MCE was explicitly handled later.

However, time_resume() ends up making an EFI Runtime Service call, and EFI explodes without OSFXSR, presumably when trying to spill %xmm registers onto the stack.

Given this codepath, and the potential for other issues of a similar kind (TLB flushing vs INVPCID, HVM logic vs VMXE, etc), restore CR4 in asm before entering C.

Quoted 6 lines

Ignore the previous MCE special case, because its not actually necessary. The handler is already suitably configured from before suspend.

Fixes 4304ff420e5 ("x86/S3: Drop {save,restore}_rest_processor_state() completely")

This one doesn't build, wakeup_prot.S misses #include <asm/asm_defns.h> . With that fixed:

Quoted 43 lines

---
CC: Jan Beulich <[email protected]>
CC: Roger Pau Monné <[email protected]>
CC: Wei Liu <[email protected]>
CC: Marek Marczykowski-Górecki <[email protected]>

This is one definite bug fix. It doesn't appear to be the only S3 bug however.

---
 xen/arch/x86/acpi/power.c       | 3 ---
 xen/arch/x86/acpi/wakeup_prot.S | 5 +++++
 2 files changed, 5 insertions(+), 3 deletions(-)
xen/arch/x86/acpi/power.c +0 -3
index 4fb1e7a148..7f162a4df9 100644
@@ -276,9 +276,6 @@ static int enter_state(u32 state)
 
     mcheck_init(&boot_cpu_data, false);
 
-    /* Restore CR4 from cached value, now MCE is set up. */
-    write_cr4(read_cr4());
-
     printk(XENLOG_INFO "Finishing wakeup from ACPI S%d state.\n", state);
 
     if ( (state == ACPI_STATE_S3) && error )
xen/arch/x86/acpi/wakeup_prot.S +5 -0
index c6b3fcc93d..1ee5551fb5 100644
@@ -110,6 +110,11 @@ ENTRY(s3_resume)
 
         call    load_system_tables
 
+        /* Restore CR4 from the cpuinfo block. */
+        GET_STACK_END(bx)
+        mov     STACK_CPUINFO_FIELD(cr4)(%rbx), %rax
+        mov     %rax, %cr4
+
 .Lsuspend_err:
         pop     %r15
         pop     %r14
Signature · 5 lines
Best Regards,
Marek Marczykowski-Górecki
Invisible Things Lab
A: Because it messes up the order in which people normally read text.
Q: Why is top-posting such a bad thing?