Skip to contentSkip to conversation

x86/fred: Fix system hang during S4 resume with FRED enabled

PATCH Applied Xin Li (Intel) 4 participants 4 messages Apr 1, 2025

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 x86/fred: Fix system hang during S4 resume with FRED enabled

Upon a wakeup from S4, the restore kernel starts and initializes the FRED MSRs as needed from its perspective. It then loads a hibernation image, including the image kernel, and attempts to load image pages directly into their original page frames used before hibernation unless those frames are currently in use. Once all pages are moved to their original locations, it jumps to a "trampoline" page in the image kernel.

At this point, the image kernel takes control, but the FRED MSRs still contain values set by the restore kernel, which may differ from those set by the image kernel before hibernation. Therefore, the image kernel must ensure the FRED MSRs have the same values as before hibernation. Since these values depend only on the location of the kernel text and data, they can be recomputed from scratch.

---

Change in v2:
* Rewrite the change log and in-code comments based on Rafael's feedback.

---
 arch/x86/power/cpu.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/x86/power/cpu.c +14 -0
index 63230ff8cf4f..08e76a5ca155 100644
@@ -27,6 +27,7 @@
 #include <asm/mmu_context.h>
 #include <asm/cpu_device_id.h>
 #include <asm/microcode.h>
+#include <asm/fred.h>
 
 #ifdef CONFIG_X86_32
 __visible unsigned long saved_context_ebx;
@@ -231,6 +232,19 @@ static void notrace __restore_processor_state(struct saved_context *ctxt)
 	 */
 #ifdef CONFIG_X86_64
 	wrmsrl(MSR_GS_BASE, ctxt->kernelmode_gs_base);
+
+	/*
+	 * Reinitialize FRED to ensure the FRED MSRs contain the same values
+	 * as before hibernation.
+	 *
+	 * Note, the setup of FRED RSPs requires access to percpu data
+	 * structures.  Therefore, FRED reinitialization can only occur after
+	 * the percpu access pointer (i.e., MSR_GS_BASE) is restored.
+	 */
+	if (ctxt->cr4 & X86_CR4_FRED) {
+		cpu_init_fred_exceptions();
+		cpu_init_fred_rsps();
+	}
 #else
 	loadsegment(fs, __KERNEL_PERCPU);
 #endif

base-commit: 535bd326c5657fe570f41b1f76941e449d9e2062

Signature · 1 line
2.49.0
maintainer Reviewed-by on v2 1/1
#1 lore raw
Quoted 15 lines · Xin Li (Intel)

Upon a wakeup from S4, the restore kernel starts and initializes the FRED MSRs as needed from its perspective. It then loads a hibernation image, including the image kernel, and attempts to load image pages directly into their original page frames used before hibernation unless those frames are currently in use. Once all pages are moved to their original locations, it jumps to a "trampoline" page in the image kernel.

At this point, the image kernel takes control, but the FRED MSRs still contain values set by the restore kernel, which may differ from those set by the image kernel before hibernation. Therefore, the image kernel must ensure the FRED MSRs have the same values as before hibernation. Since these values depend only on the location of the kernel text and data, they can be recomputed from scratch.

Quoted 10 lines
Quoted 4 lines

---

Change in v2:
* Rewrite the change log and in-code comments based on Rafael's feedback.

Thanks!

Quoted 41 lines
---
 arch/x86/power/cpu.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/x86/power/cpu.c +14 -0
index 63230ff8cf4f..08e76a5ca155 100644
@@ -27,6 +27,7 @@
 #include <asm/mmu_context.h>
 #include <asm/cpu_device_id.h>
 #include <asm/microcode.h>
+#include <asm/fred.h>
 
 #ifdef CONFIG_X86_32
 __visible unsigned long saved_context_ebx;
@@ -231,6 +232,19 @@ static void notrace __restore_processor_state(struct saved_context *ctxt)
         */
 #ifdef CONFIG_X86_64
        wrmsrl(MSR_GS_BASE, ctxt->kernelmode_gs_base);
+
+       /*
+        * Reinitialize FRED to ensure the FRED MSRs contain the same values
+        * as before hibernation.
+        *
+        * Note, the setup of FRED RSPs requires access to percpu data
+        * structures.  Therefore, FRED reinitialization can only occur after
+        * the percpu access pointer (i.e., MSR_GS_BASE) is restored.
+        */
+       if (ctxt->cr4 & X86_CR4_FRED) {
+               cpu_init_fred_exceptions();
+               cpu_init_fred_rsps();
+       }
 #else
        loadsegment(fs, __KERNEL_PERCPU);
 #endif

base-commit: 535bd326c5657fe570f41b1f76941e449d9e2062 --
2.49.0

reviewer Reviewed-by on v2 1/1
#2 lore raw
In reply to Xin Li (Intel)
Quoted 65 lines · Xin Li (Intel)

Upon a wakeup from S4, the restore kernel starts and initializes the FRED MSRs as needed from its perspective. It then loads a hibernation image, including the image kernel, and attempts to load image pages directly into their original page frames used before hibernation unless those frames are currently in use. Once all pages are moved to their original locations, it jumps to a "trampoline" page in the image kernel.

At this point, the image kernel takes control, but the FRED MSRs still contain values set by the restore kernel, which may differ from those set by the image kernel before hibernation. Therefore, the image kernel must ensure the FRED MSRs have the same values as before hibernation. Since these values depend only on the location of the kernel text and data, they can be recomputed from scratch.

---

Change in v2:
* Rewrite the change log and in-code comments based on Rafael's feedback.
---
arch/x86/power/cpu.c | 14 ++++++++++++++

1 file changed, 14 insertions(+)
arch/x86/power/cpu.c +14 -0
index 63230ff8cf4f..08e76a5ca155 100644
@@ -27,6 +27,7 @@
 #include <asm/mmu_context.h>
 #include <asm/cpu_device_id.h>
 #include <asm/microcode.h>
+#include <asm/fred.h>
 
 #ifdef CONFIG_X86_32
 __visible unsigned long saved_context_ebx;
@@ -231,6 +232,19 @@ static void notrace __restore_processor_state(struct saved_context *ctxt)
 	 */
 #ifdef CONFIG_X86_64
 	wrmsrl(MSR_GS_BASE, ctxt->kernelmode_gs_base);
+
+	/*
+	 * Reinitialize FRED to ensure the FRED MSRs contain the same values
+	 * as before hibernation.
+	 *
+	 * Note, the setup of FRED RSPs requires access to percpu data
+	 * structures.  Therefore, FRED reinitialization can only occur after
+	 * the percpu access pointer (i.e., MSR_GS_BASE) is restored.
+	 */
+	if (ctxt->cr4 & X86_CR4_FRED) {
+		cpu_init_fred_exceptions();
+		cpu_init_fred_rsps();
+	}
 #else
 	loadsegment(fs, __KERNEL_PERCPU);
 #endif
Quoted 1 line

base-commit: 535bd326c5657fe570f41b1f76941e449d9e2062

merge bot Applied
#3 lore raw
In reply to Xin Li (Intel)
The following commit has been merged into the x86/urgent branch of tip:

The following commit has been merged into the x86/urgent branch of tip:

Commit-ID:     e5f1e8af9c9e151ecd665f6d2e36fb25fec3b110
Gitweb:        https://git.kernel.org/tip/e5f1e8af9c9e151ecd665f6d2e36fb25fec3b110
Author:        Xin Li (Intel) <[email protected]>
AuthorDate:    Tue, 01 Apr 2025 00:57:27 -07:00
Committer:     Ingo Molnar <[email protected]>
CommitterDate: Tue, 01 Apr 2025 22:29:02 +02:00

x86/fred: Fix system hang during S4 resume with FRED enabled

Upon a wakeup from S4, the restore kernel starts and initializes the FRED MSRs as needed from its perspective. It then loads a hibernation image, including the image kernel, and attempts to load image pages directly into their original page frames used before hibernation unless those frames are currently in use. Once all pages are moved to their original locations, it jumps to a "trampoline" page in the image kernel.

At this point, the image kernel takes control, but the FRED MSRs still contain values set by the restore kernel, which may differ from those set by the image kernel before hibernation. Therefore, the image kernel must ensure the FRED MSRs have the same values as before hibernation. Since these values depend only on the location of the kernel text and data, they can be recomputed from scratch.

---
 arch/x86/power/cpu.c | 14 ++++++++++++++
 1 file changed, 14 insertions(+)
arch/x86/power/cpu.c +14 -0
index 63230ff..08e76a5 100644
@@ -27,6 +27,7 @@
 #include <asm/mmu_context.h>
 #include <asm/cpu_device_id.h>
 #include <asm/microcode.h>
+#include <asm/fred.h>
 
 #ifdef CONFIG_X86_32
 __visible unsigned long saved_context_ebx;
@@ -231,6 +232,19 @@ static void notrace __restore_processor_state(struct saved_context *ctxt)
 	 */
 #ifdef CONFIG_X86_64
 	wrmsrl(MSR_GS_BASE, ctxt->kernelmode_gs_base);
+
+	/*
+	 * Reinitialize FRED to ensure the FRED MSRs contain the same values
+	 * as before hibernation.
+	 *
+	 * Note, the setup of FRED RSPs requires access to percpu data
+	 * structures.  Therefore, FRED reinitialization can only occur after
+	 * the percpu access pointer (i.e., MSR_GS_BASE) is restored.
+	 */
+	if (ctxt->cr4 & X86_CR4_FRED) {
+		cpu_init_fred_exceptions();
+		cpu_init_fred_rsps();
+	}
 #else
 	loadsegment(fs, __KERNEL_PERCPU);
 #endif