Skip to contentSkip to conversation

x86/cpu: Skip reading MSR_IA32_PLATFORM_ID in virtualized environment

PATCH In review Binbin Wu 4 participants 19 messages Apr 30 – May 14

Subsystems (3)
lore
Share & tools
lore

Binbin Wu

author
#0 lore raw

PATCH v2 1/1 x86/cpu: Skip reading MSR_IA32_PLATFORM_ID in virtualized environment

The kernel now reads MSR_IA32_PLATFORM_ID during CPU init. When running as a guest, if the underlying hypervisor does not emulate the MSR, the RDMSR from MSR_IA32_PLATFORM_ID can trigger an unchecked MSR access during early boot. This MSR is not emulated in the case for KVM TDX, where the following is observed in the TD guest:

    unchecked MSR access error: RDMSR from 0x17 at rIP: 0xffffffffba38d6fc (intel_get_platform_id+0x7c/0xb0)
    Call Trace:
     <TASK>
     ? early_init_intel+0x28/0x2c0
     ? early_cpu_init+0x9b/0x930
     ? setup_arch+0xbf/0xbb0
     ? _printk+0x6b/0x90
     ? start_kernel+0x7f/0xaa0
     ? x86_64_start_reservations+0x24/0x30
     ? x86_64_start_kernel+0xda/0xe0
     ? common_startup_64+0x13e/0x141
     </TASK>

The platform ID is used for one thing and one thing only: microcode updates. Those updates are solely the domain of the bare-metal OS. The guest kernel code should not even try to touch the MSR. Skip reading the MSR when the kernel is running in a virtualized environment. 0 is a valid platform ID, however, microcode related logic is skipped in a virtualized environment.

Since intel_get_platform_id() could be called early before cpuinfo_x86 is fully initialized in the case of CONFIG_MICROCODE_DBG, check whether the kernel is running in a virtualized environment from CPUID. Use cpuid_ecx() instead of native_cpuid_ecx() so that Xen PV guest will see the virtualized bit.

Fixes d8630b67ca1ed ("x86/cpu: Add platform ID to CPU info structure")

---
v2:
- Drop the patch on KVM side. (Sean, Dave)
- Use X86_FEATURE_HYPERVISOR for better readability. (Dave)
- Use cpuid_ecx() instead of native_cpuid_ecx() to check the hypervisor bit.
- Add RB from Rick.

---
 arch/x86/kernel/cpu/microcode/intel.c | 4 ++++
 1 file changed, 4 insertions(+)
arch/x86/kernel/cpu/microcode/intel.c +4 -0
index 37ac4afe0972..1bc0c350726c 100644
@@ -147,6 +147,10 @@ u32 intel_get_platform_id(void)
 	if (intel_cpuid_vfm() <= INTEL_PENTIUM_II_KLAMATH)
 		return 0;
 
+	/* Don't try to read microcode bits when virtualized. */
+	if (cpuid_ecx(1) & BIT(X86_FEATURE_HYPERVISOR & 0x1f))
+		return 0;
+
 	/* get processor flags from MSR 0x17 */
 	native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
 

base-commit: 9974969c14031a097d6b45bcb7a06bb4aa525c40

Signature · 1 line
2.46.0
Acked-by on v2 1/1
#1 lore raw
Quoted 32 lines · Binbin Wu

The kernel now reads MSR_IA32_PLATFORM_ID during CPU init. When running as a guest, if the underlying hypervisor does not emulate the MSR, the RDMSR from MSR_IA32_PLATFORM_ID can trigger an unchecked MSR access during early boot. This MSR is not emulated in the case for KVM TDX, where the following is observed in the TD guest:

    unchecked MSR access error: RDMSR from 0x17 at rIP: 0xffffffffba38d6fc (intel_get_platform_id+0x7c/0xb0)
    Call Trace:
     <TASK>
     ? early_init_intel+0x28/0x2c0
     ? early_cpu_init+0x9b/0x930
     ? setup_arch+0xbf/0xbb0
     ? _printk+0x6b/0x90
     ? start_kernel+0x7f/0xaa0
     ? x86_64_start_reservations+0x24/0x30
     ? x86_64_start_kernel+0xda/0xe0
     ? common_startup_64+0x13e/0x141
     </TASK>

The platform ID is used for one thing and one thing only: microcode updates. Those updates are solely the domain of the bare-metal OS. The guest kernel code should not even try to touch the MSR. Skip reading the MSR when the kernel is running in a virtualized environment. 0 is a valid platform ID, however, microcode related logic is skipped in a virtualized environment.

Since intel_get_platform_id() could be called early before cpuinfo_x86 is fully initialized in the case of CONFIG_MICROCODE_DBG, check whether the kernel is running in a virtualized environment from CPUID. Use cpuid_ecx() instead of native_cpuid_ecx() so that Xen PV guest will see the virtualized bit.

Quoted 4 lines
Fixes d8630b67ca1ed ("x86/cpu: Add platform ID to CPU info structure")
Signature · 1 line
  Kiryl Shutsemau / Kirill A. Shutemov
maintainer
#2 lore raw
In reply to Binbin Wu
Quoted 16 lines · Binbin Wu
arch/x86/kernel/cpu/microcode/intel.c +4 -0
index 37ac4afe0972..1bc0c350726c 100644
@@ -147,6 +147,10 @@ u32 intel_get_platform_id(void)
 	if (intel_cpuid_vfm() <= INTEL_PENTIUM_II_KLAMATH)
 		return 0;
 
+	/* Don't try to read microcode bits when virtualized. */
+	if (cpuid_ecx(1) & BIT(X86_FEATURE_HYPERVISOR & 0x1f))
+		return 0;
+
 	/* get processor flags from MSR 0x17 */
 	native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
 
Quoted 2 lines

base-commit: 9974969c14031a097d6b45bcb7a06bb4aa525c40

Does that work too?

arch/x86/kernel/cpu/microcode/intel.c +3 -0
index 18d2eff7a4b7..1b24de94bdd5 100644
@@ -139,6 +139,9 @@ u32 intel_get_platform_id(void)
 {
 	unsigned int val[2];
 
+	if (hypervisor_present)
+		return 0;
+
 	/*
 	 * This can be called early. Use CPUID directly instead of
 	 * relying on cpuinfo_x86 which may not be fully initialized.
Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author
#3 lore raw
Quoted 27 lines · Borislav Petkov
Quoted 16 lines · Binbin Wu
arch/x86/kernel/cpu/microcode/intel.c +4 -0
index 37ac4afe0972..1bc0c350726c 100644
@@ -147,6 +147,10 @@ u32 intel_get_platform_id(void)
 	if (intel_cpuid_vfm() <= INTEL_PENTIUM_II_KLAMATH)
 		return 0;
 
+	/* Don't try to read microcode bits when virtualized. */
+	if (cpuid_ecx(1) & BIT(X86_FEATURE_HYPERVISOR & 0x1f))
+		return 0;
+
 	/* get processor flags from MSR 0x17 */
 	native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
 
Quoted 2 lines

base-commit: 9974969c14031a097d6b45bcb7a06bb4aa525c40

Does that work too?

arch/x86/kernel/cpu/microcode/intel.c +0 -0
index 18d2eff7a4b7..1b24de94bdd5 100644

arch/x86/kernel/cpu/microcode/intel.c u32 intel_get_platform_id(void) · line 142

Quoted 4 lines
arch/x86/kernel/cpu/microcode/intel.c +1 -0
@@ -139,6 +139,9 @@ u32 intel_get_platform_id(void)
 {
 	unsigned int val[2];
 
+	if (hypervisor_present)

hypervisor_present could be uninitialized if dis_ucode_ldr is true. intel_get_platform_id() is also called during the normal cpu initialization.

Quoted 6 lines
arch/x86/kernel/cpu/microcode/intel.c +2 -0
+		return 0;
+
 	/*
 	 * This can be called early. Use CPUID directly instead of
 	 * relying on cpuinfo_x86 which may not be fully initialized.
maintainer
#4 lore raw
Quoted 2 lines · Binbin Wu

hypervisor_present could be uninitialized if dis_ucode_ldr is true. intel_get_platform_id() is also called during the normal cpu initialization.

Right, that needs more surgery. See if the below works instead pls.

We might as well do it - the question whether we run on a HV comes very often recenly - might as well make it an "official" variable.

arch/x86/include/asm/processor.h +1 -0
index 10b5355b323e..67dd932305db 100644
@@ -733,6 +733,7 @@ bool xen_set_default_idle(void);
 #endif
 
 void __noreturn stop_this_cpu(void *dummy);
+extern bool x86_hypervisor_present;
 void microcode_check(struct cpuinfo_x86 *prev_info);
 void store_cpu_caps(struct cpuinfo_x86 *info);
 
arch/x86/kernel/cpu/microcode/amd.c +2 -2
index e533881284a1..5c0afae75e9f 100644
@@ -322,7 +322,7 @@ static u32 get_patch_level(void)
 {
 	u32 rev, dummy __always_unused;
 
-	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present) {
+	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present) {
 		int cpu = smp_processor_id();
 
 		if (!microcode_rev[cpu]) {
@@ -714,7 +714,7 @@ static bool __apply_microcode_amd(struct microcode_amd *mc, u32 *cur_rev,
 			invlpg(p_addr_end);
 	}
 
-	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present)
+	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present)
 		microcode_rev[smp_processor_id()] = mc->hdr.patch_id;
 
 	/* verify patch application was successful */
arch/x86/kernel/cpu/microcode/core.c +10 -12
index 68a1a893246c..f6722c9618e0 100644
@@ -57,7 +57,7 @@ bool force_minrev = IS_ENABLED(CONFIG_MICROCODE_LATE_FORCE_MINREV);
 u32 base_rev;
 u32 microcode_rev[NR_CPUS] = {};
 
-bool hypervisor_present;
+bool __ro_after_init x86_hypervisor_present;
 
 /*
  * Synchronization.
@@ -118,14 +118,9 @@ bool __init microcode_loader_disabled(void)
 	/*
 	 * Disable when:
 	 *
-	 * 1) The CPU does not support CPUID.
-	 */
-	if (!cpuid_feature()) {
-		dis_ucode_ldr = true;
-		return dis_ucode_ldr;
-	}
-
-	/*
+	 * 1) The CPU does not support CPUID detected below in
+	 *    load_ucode_bsp().
+	 *
 	 * 2) Bit 31 in CPUID[1]:ECX is set
 	 *    The bit is reserved for hypervisor use. This is still not
 	 *    completely accurate as XEN PV guests don't see that CPUID bit
@@ -135,9 +130,7 @@ bool __init microcode_loader_disabled(void)
 	 * 3) Certain AMD patch levels are not allowed to be
 	 *    overwritten.
 	 */
-	hypervisor_present = native_cpuid_ecx(1) & BIT(31);
-
-	if ((hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) ||
+	if ((x86_hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) ||
 	    amd_check_current_patch_level())
 		dis_ucode_ldr = true;
 
@@ -179,6 +172,11 @@ void __init load_ucode_bsp(void)
 
 	early_parse_cmdline();
 
+	if (!cpuid_feature())
+		dis_ucode_ldr = true;
+	else
+		x86_hypervisor_present = native_cpuid_ecx(1) & BIT(31);
+
 	if (microcode_loader_disabled())
 		return;
 
arch/x86/kernel/cpu/microcode/intel.c +3 -0
index 37ac4afe0972..a4c0a0cf928b 100644
@@ -138,6 +138,9 @@ u32 intel_get_platform_id(void)
 {
 	unsigned int val[2];
 
+	if (x86_hypervisor_present)
+		return 0;
+
 	/*
 	 * This can be called early. Use CPUID directly instead of
 	 * relying on cpuinfo_x86 which may not be fully initialized.
arch/x86/kernel/cpu/microcode/internal.h +0 -1
index 3b93c0676b4f..a10b547eda1e 100644
@@ -48,7 +48,6 @@ extern struct early_load_data early_data;
 extern struct ucode_cpu_info ucode_cpu_info[];
 extern u32 microcode_rev[NR_CPUS];
 extern u32 base_rev;
-extern bool hypervisor_present;
 
 struct cpio_data find_microcode_in_initrd(const char *path);
 
Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author
#5 lore raw
Quoted 8 lines · Borislav Petkov
Quoted 2 lines · Binbin Wu

hypervisor_present could be uninitialized if dis_ucode_ldr is true. intel_get_platform_id() is also called during the normal cpu initialization.

Right, that needs more surgery. See if the below works instead pls.

We might as well do it - the question whether we run on a HV comes very often recenly - might as well make it an "official" variable.

Make sense.
My version treated it as a bug fix, so I tried to minimize the code change.

[...]

arch/x86/kernel/cpu/microcode/core.c void __init load_ucode_bsp(void) · line 179

Quoted 10 lines
arch/x86/kernel/cpu/microcode/core.c +5 -0
@@ -179,6 +172,11 @@ void __init load_ucode_bsp(void)
 
 	early_parse_cmdline();
 
+	if (!cpuid_feature())
+		dis_ucode_ldr = true;
+	else
+		x86_hypervisor_present = native_cpuid_ecx(1) & BIT(31);
+

There is one case not covered by setting the global x86_hypervisor_present at this point. Xen PV guest will not go this path, so Xen PV guest will not see the hypervisor bit.

But maybe we can just ignore Xen PV guest case today?

Quoted 29 lines
arch/x86/kernel/cpu/microcode/core.c +0 -0
 	if (microcode_loader_disabled())
 		return;
 
arch/x86/kernel/cpu/microcode/intel.c +3 -0
index 37ac4afe0972..a4c0a0cf928b 100644
@@ -138,6 +138,9 @@ u32 intel_get_platform_id(void)
 {
 	unsigned int val[2];
 
+	if (x86_hypervisor_present)
+		return 0;
+
 	/*
 	 * This can be called early. Use CPUID directly instead of
 	 * relying on cpuinfo_x86 which may not be fully initialized.
arch/x86/kernel/cpu/microcode/internal.h +0 -1
index 3b93c0676b4f..a10b547eda1e 100644
@@ -48,7 +48,6 @@ extern struct early_load_data early_data;
 extern struct ucode_cpu_info ucode_cpu_info[];
 extern u32 microcode_rev[NR_CPUS];
 extern u32 base_rev;
-extern bool hypervisor_present;
 
 struct cpio_data find_microcode_in_initrd(const char *path);
 
maintainer
#6 lore raw
Quoted 1 line · Binbin Wu

My version treated it as a bug fix, so I tried to minimize the code change.

I didn't like

	BIT(X86_FEATURE_HYPERVISOR & 0x1f))

in your patch. I'd use BIT(31) as it does.

But, more importantly, it added yet another place where we test whether we run on a HV. And it is high time we unified those because we run more and more on HVs nowadays.

Quoted 4 lines

There is one case not covered by setting the global x86_hypervisor_present at this point. Xen PV guest will not go this path, so Xen PV guest will not see the hypervisor bit.

But maybe we can just ignore Xen PV guest case today?

Nothing changes before or after this fix. So Xen PV can be handled completely separated from this.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author
#7 lore raw
Quoted 8 lines · Borislav Petkov
Quoted 1 line · Binbin Wu

My version treated it as a bug fix, so I tried to minimize the code change.

I didn't like

	BIT(X86_FEATURE_HYPERVISOR & 0x1f))

in your patch. I'd use BIT(31) as it does.

It was suggested by Dave in the v1 comment.
https://lore.kernel.org/kvm/[email protected]/

Quoted 4 lines

But, more importantly, it added yet another place where we test whether we run on a HV. And it is high time we unified those because we run more and more on HVs nowadays.

Agree.

Quoted 8 lines
Quoted 5 lines

There is one case not covered by setting the global x86_hypervisor_present at this point. Xen PV guest will not go this path, so Xen PV guest will not see the hypervisor bit.

But maybe we can just ignore Xen PV guest case today?

Nothing changes before or after this fix. So Xen PV can be handled completely separated from this.

intel_get_platform_id() can be called in Xen PV guest during normal boot. Using x86_hypervisor_present just doesn't cover the case.

My fix was intended to skip reading MSR_IA32_PLATFORM_ID if hypervisor is detected, I used cpuid_ecx() instead of cpuid_ecx_native() so that Xen PV guest can also read the hypervisor bit using xen_cpuid() (Xen PV guest has overwritten the pv ops) to make the coverage complete.

Quoted 3 lines

Thx.

maintainer
#8 lore raw
Quoted 4 lines · Binbin Wu

My fix was intended to skip reading MSR_IA32_PLATFORM_ID if hypervisor is detected, I used cpuid_ecx() instead of cpuid_ecx_native() so that Xen PV guest can also read the hypervisor bit using xen_cpuid() (Xen PV guest has overwritten the pv ops) to make the coverage

Which part of "can be handled completely separated from this" wasn't clear?

The current code uses the native cpuid variant. If you want to fix Xen, send a patch ontop please.

Signature · 1 line
Small device. Typos and formatting crap
author Tested-by
#9 lore raw
In reply to Borislav Petkov
Quoted 8 lines · Borislav Petkov
Quoted 2 lines · Binbin Wu

hypervisor_present could be uninitialized if dis_ucode_ldr is true. intel_get_platform_id() is also called during the normal cpu initialization.

Right, that needs more surgery. See if the below works instead pls.

We might as well do it - the question whether we run on a HV comes very often recenly - might as well make it an "official" variable.

Tested the diff by running as a TDX guest and it fixed the unchecked MSR access error.

Quoted 115 lines
arch/x86/include/asm/processor.h +1 -0
index 10b5355b323e..67dd932305db 100644
@@ -733,6 +733,7 @@ bool xen_set_default_idle(void);
 #endif
 
 void __noreturn stop_this_cpu(void *dummy);
+extern bool x86_hypervisor_present;
 void microcode_check(struct cpuinfo_x86 *prev_info);
 void store_cpu_caps(struct cpuinfo_x86 *info);
 
arch/x86/kernel/cpu/microcode/amd.c +2 -2
index e533881284a1..5c0afae75e9f 100644
@@ -322,7 +322,7 @@ static u32 get_patch_level(void)
 {
 	u32 rev, dummy __always_unused;
 
-	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present) {
+	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present) {
 		int cpu = smp_processor_id();
 
 		if (!microcode_rev[cpu]) {
@@ -714,7 +714,7 @@ static bool __apply_microcode_amd(struct microcode_amd *mc, u32 *cur_rev,
 			invlpg(p_addr_end);
 	}
 
-	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present)
+	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present)
 		microcode_rev[smp_processor_id()] = mc->hdr.patch_id;
 
 	/* verify patch application was successful */
arch/x86/kernel/cpu/microcode/core.c +10 -12
index 68a1a893246c..f6722c9618e0 100644
@@ -57,7 +57,7 @@ bool force_minrev = IS_ENABLED(CONFIG_MICROCODE_LATE_FORCE_MINREV);
 u32 base_rev;
 u32 microcode_rev[NR_CPUS] = {};
 
-bool hypervisor_present;
+bool __ro_after_init x86_hypervisor_present;
 
 /*
  * Synchronization.
@@ -118,14 +118,9 @@ bool __init microcode_loader_disabled(void)
 	/*
 	 * Disable when:
 	 *
-	 * 1) The CPU does not support CPUID.
-	 */
-	if (!cpuid_feature()) {
-		dis_ucode_ldr = true;
-		return dis_ucode_ldr;
-	}
-
-	/*
+	 * 1) The CPU does not support CPUID detected below in
+	 *    load_ucode_bsp().
+	 *
 	 * 2) Bit 31 in CPUID[1]:ECX is set
 	 *    The bit is reserved for hypervisor use. This is still not
 	 *    completely accurate as XEN PV guests don't see that CPUID bit
@@ -135,9 +130,7 @@ bool __init microcode_loader_disabled(void)
 	 * 3) Certain AMD patch levels are not allowed to be
 	 *    overwritten.
 	 */
-	hypervisor_present = native_cpuid_ecx(1) & BIT(31);
-
-	if ((hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) ||
+	if ((x86_hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) ||
 	    amd_check_current_patch_level())
 		dis_ucode_ldr = true;
 
@@ -179,6 +172,11 @@ void __init load_ucode_bsp(void)
 
 	early_parse_cmdline();
 
+	if (!cpuid_feature())
+		dis_ucode_ldr = true;
+	else
+		x86_hypervisor_present = native_cpuid_ecx(1) & BIT(31);
+
 	if (microcode_loader_disabled())
 		return;
 
arch/x86/kernel/cpu/microcode/intel.c +3 -0
index 37ac4afe0972..a4c0a0cf928b 100644
@@ -138,6 +138,9 @@ u32 intel_get_platform_id(void)
 {
 	unsigned int val[2];
 
+	if (x86_hypervisor_present)
+		return 0;
+
 	/*
 	 * This can be called early. Use CPUID directly instead of
 	 * relying on cpuinfo_x86 which may not be fully initialized.
arch/x86/kernel/cpu/microcode/internal.h +0 -1
index 3b93c0676b4f..a10b547eda1e 100644
@@ -48,7 +48,6 @@ extern struct early_load_data early_data;
 extern struct ucode_cpu_info ucode_cpu_info[];
 extern u32 microcode_rev[NR_CPUS];
 extern u32 base_rev;
-extern bool hypervisor_present;
 
 struct cpio_data find_microcode_in_initrd(const char *path);
 
maintainer
#10 lore raw
Quoted 3 lines · Binbin Wu

Tested the diff by running as a TDX guest and it fixed the unchecked MSR access error.

Thanks.

Patches coming up as a reply to this message.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
maintainer
#11 lore raw

PATCH 1/2 x86/microcode: Do not access MSR_IA32_PLATFORM_ID when running as a guest

Patch in Fixes: causes the usual:

unchecked MSR access error: RDMSR from 0x17 at ... (intel_get_platform_id)

  Call Trace:
   early_init_intel
   early_cpu_init
   setup_arch
   _printk
   start_kernel
   x86_64_start_reservations
   x86_64_start_kernel
   common_startup_64

because the kernel is booted in a guest.

In order to avoid it, this MSR access needs to be prevented when running virtualized. That is usually done by checking X86_FEATURE_HYPERVISOR but for this particular case it is too early yet.

The platform ID needs to be read as early as when microcode is loaded on the BSP:

load_ucode_bsp ... -> get_microcode_blob ... -> intel_find_matching_signature

and by that time, CPUID leafs haven't been parsed yet.

The microcode loader already has logic to check early whether the kernel is running virtualized so make that globally available to arch/x86/. The query whether running virtualized is getting more and more prominent in recent times so might as well make it an arch-global var which the rest of the code can use.

Fixes d8630b67ca1ed ("x86/cpu: Add platform ID to CPU info structure")
---
 arch/x86/include/asm/processor.h         |  1 +
 arch/x86/kernel/cpu/microcode/amd.c      |  4 ++--
 arch/x86/kernel/cpu/microcode/core.c     | 22 ++++++++++------------
 arch/x86/kernel/cpu/microcode/intel.c    |  3 +++
 arch/x86/kernel/cpu/microcode/internal.h |  1 -
 5 files changed, 16 insertions(+), 15 deletions(-)
arch/x86/include/asm/processor.h +1 -0
index 10b5355b323e..67dd932305db 100644
@@ -733,6 +733,7 @@ bool xen_set_default_idle(void);
 #endif
 
 void __noreturn stop_this_cpu(void *dummy);
+extern bool x86_hypervisor_present;
 void microcode_check(struct cpuinfo_x86 *prev_info);
 void store_cpu_caps(struct cpuinfo_x86 *info);
 
arch/x86/kernel/cpu/microcode/amd.c +2 -2
index e533881284a1..5c0afae75e9f 100644
@@ -322,7 +322,7 @@ static u32 get_patch_level(void)
 {
 	u32 rev, dummy __always_unused;
 
-	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present) {
+	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present) {
 		int cpu = smp_processor_id();
 
 		if (!microcode_rev[cpu]) {
@@ -714,7 +714,7 @@ static bool __apply_microcode_amd(struct microcode_amd *mc, u32 *cur_rev,
 			invlpg(p_addr_end);
 	}
 
-	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && hypervisor_present)
+	if (IS_ENABLED(CONFIG_MICROCODE_DBG) && x86_hypervisor_present)
 		microcode_rev[smp_processor_id()] = mc->hdr.patch_id;
 
 	/* verify patch application was successful */
arch/x86/kernel/cpu/microcode/core.c +10 -12
index 68a1a893246c..dcc688f4d35f 100644
@@ -57,7 +57,7 @@ bool force_minrev = IS_ENABLED(CONFIG_MICROCODE_LATE_FORCE_MINREV);
 u32 base_rev;
 u32 microcode_rev[NR_CPUS] = {};
 
-bool hypervisor_present;
+bool __ro_after_init x86_hypervisor_present;
 
 /*
  * Synchronization.
@@ -118,14 +118,9 @@ bool __init microcode_loader_disabled(void)
 	/*
 	 * Disable when:
 	 *
-	 * 1) The CPU does not support CPUID.
-	 */
-	if (!cpuid_feature()) {
-		dis_ucode_ldr = true;
-		return dis_ucode_ldr;
-	}
-
-	/*
+	 * 1) The CPU does not support CPUID, detected below in
+	 *    load_ucode_bsp().
+	 *
 	 * 2) Bit 31 in CPUID[1]:ECX is set
 	 *    The bit is reserved for hypervisor use. This is still not
 	 *    completely accurate as XEN PV guests don't see that CPUID bit
@@ -135,9 +130,7 @@ bool __init microcode_loader_disabled(void)
 	 * 3) Certain AMD patch levels are not allowed to be
 	 *    overwritten.
 	 */
-	hypervisor_present = native_cpuid_ecx(1) & BIT(31);
-
-	if ((hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) ||
+	if ((x86_hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) ||
 	    amd_check_current_patch_level())
 		dis_ucode_ldr = true;
 
@@ -179,6 +172,11 @@ void __init load_ucode_bsp(void)
 
 	early_parse_cmdline();
 
+	if (!cpuid_feature())
+		dis_ucode_ldr = true;
+	else
+		x86_hypervisor_present = native_cpuid_ecx(1) & BIT(31);
+
 	if (microcode_loader_disabled())
 		return;
 
arch/x86/kernel/cpu/microcode/intel.c +3 -0
index 37ac4afe0972..a4c0a0cf928b 100644
@@ -138,6 +138,9 @@ u32 intel_get_platform_id(void)
 {
 	unsigned int val[2];
 
+	if (x86_hypervisor_present)
+		return 0;
+
 	/*
 	 * This can be called early. Use CPUID directly instead of
 	 * relying on cpuinfo_x86 which may not be fully initialized.
arch/x86/kernel/cpu/microcode/internal.h +0 -1
index 3b93c0676b4f..a10b547eda1e 100644
@@ -48,7 +48,6 @@ extern struct early_load_data early_data;
 extern struct ucode_cpu_info ucode_cpu_info[];
 extern u32 microcode_rev[NR_CPUS];
 extern u32 base_rev;
-extern bool hypervisor_present;
 
 struct cpio_data find_microcode_in_initrd(const char *path);
 
Signature · 8 lines
2.51.0


-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
maintainer
#16 lore raw

PATCH 2/2 x86/cpu: Move intel_get_platform_id() to cpu/intel.c

In reply to Borislav Petkov

It is not only used in the microcode loader anymore and the platform ID is cached in the cpuindo_x86 structure so move the getter to Intel CPU-specific code.

No functional changes.

---
 arch/x86/include/asm/microcode.h      |  2 --
 arch/x86/include/asm/processor.h      |  1 +
 arch/x86/kernel/cpu/intel.c           | 35 ++++++++++++++++++++++++++
 arch/x86/kernel/cpu/microcode/intel.c | 36 ---------------------------
 4 files changed, 36 insertions(+), 38 deletions(-)
arch/x86/include/asm/microcode.h +0 -2
index 3c317d155771..8b41f26f003b 100644
@@ -61,8 +61,6 @@ static inline int intel_microcode_get_datasize(struct microcode_header_intel *hd
 	return hdr->datasize ? : DEFAULT_UCODE_DATASIZE;
 }
 
-extern u32 intel_get_platform_id(void);
-
 static inline u32 intel_get_microcode_revision(void)
 {
 	u32 rev, dummy;
arch/x86/include/asm/processor.h +1 -0
index 67dd932305db..1a68a955863d 100644
@@ -237,6 +237,7 @@ extern void early_cpu_init(void);
 extern void identify_secondary_cpu(unsigned int cpu);
 extern void print_cpu_info(struct cpuinfo_x86 *);
 void print_cpu_msr(struct cpuinfo_x86 *);
+extern u32 intel_get_platform_id(void);
 
 /*
  * Friendlier CR3 helpers.
arch/x86/kernel/cpu/intel.c +35 -0
index f28c0efb7c8f..3473b89e6907 100644
@@ -199,6 +199,41 @@ void intel_unlock_cpuid_leafs(struct cpuinfo_x86 *c)
 		c->cpuid_level = cpuid_eax(0);
 }
 
+/*
+ * Use CPUID to generate a "vfm" value. Useful before cpuinfo_x86
+ * structures are populated.
+ */
+static u32 intel_cpuid_vfm(void)
+{
+	u32 eax   = cpuid_eax(1);
+	u32 fam   = x86_family(eax);
+	u32 model = x86_model(eax);
+
+	return IFM(fam, model);
+}
+
+u32 intel_get_platform_id(void)
+{
+	unsigned int val[2];
+
+	if (x86_hypervisor_present)
+		return 0;
+
+	/*
+	 * This can be called early. Use CPUID directly instead of
+	 * relying on cpuinfo_x86 which may not be fully initialized.
+	 * The PII does not have MSR_IA32_PLATFORM_ID. Everything
+	 * before _it_ has no microcode (for Linux at least).
+	 */
+	if (intel_cpuid_vfm() <= INTEL_PENTIUM_II_KLAMATH)
+		return 0;
+
+	/* get processor flags from MSR 0x17 */
+	native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
+
+	return (val[1] >> 18) & 7;
+}
+
 static void early_init_intel(struct cpuinfo_x86 *c)
 {
 	u64 misc_enable;
arch/x86/kernel/cpu/microcode/intel.c +0 -36
index a4c0a0cf928b..f55ff81682e5 100644
@@ -120,42 +120,6 @@ static inline unsigned int exttable_size(struct extended_sigtable *et)
 	return et->count * EXT_SIGNATURE_SIZE + EXT_HEADER_SIZE;
 }
 
-
-/*
- * Use CPUID to generate a "vfm" value. Useful before cpuinfo_x86
- * structures are populated.
- */
-static u32 intel_cpuid_vfm(void)
-{
-	u32 eax   = cpuid_eax(1);
-	u32 fam   = x86_family(eax);
-	u32 model = x86_model(eax);
-
-	return IFM(fam, model);
-}
-
-u32 intel_get_platform_id(void)
-{
-	unsigned int val[2];
-
-	if (x86_hypervisor_present)
-		return 0;
-
-	/*
-	 * This can be called early. Use CPUID directly instead of
-	 * relying on cpuinfo_x86 which may not be fully initialized.
-	 * The PII does not have MSR_IA32_PLATFORM_ID. Everything
-	 * before _it_ has no microcode (for Linux at least).
-	 */
-	if (intel_cpuid_vfm() <= INTEL_PENTIUM_II_KLAMATH)
-		return 0;
-
-	/* get processor flags from MSR 0x17 */
-	native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
-
-	return (val[1] >> 18) & 7;
-}
-
 void intel_collect_cpu_info(struct cpu_signature *sig)
 {
 	sig->sig = cpuid_eax(1);
Signature · 8 lines
2.51.0


-- 
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author Reviewed-by on v1 1/2
#12 lore raw

Re: [PATCH 1/2] x86/microcode: Do not access MSR_IA32_PLATFORM_ID when running as a guest

In reply to Borislav Petkov
Quoted 32 lines · Borislav Petkov

Patch in Fixes: causes the usual:

unchecked MSR access error: RDMSR from 0x17 at ... (intel_get_platform_id)

  Call Trace:
   early_init_intel
   early_cpu_init
   setup_arch
   _printk
   start_kernel
   x86_64_start_reservations
   x86_64_start_kernel
   common_startup_64

because the kernel is booted in a guest.

In order to avoid it, this MSR access needs to be prevented when running virtualized. That is usually done by checking X86_FEATURE_HYPERVISOR but for this particular case it is too early yet.

The platform ID needs to be read as early as when microcode is loaded on the BSP:

load_ucode_bsp ... -> get_microcode_blob ... -> intel_find_matching_signature

and by that time, CPUID leafs haven't been parsed yet.

The microcode loader already has logic to check early whether the kernel is running virtualized so make that globally available to arch/x86/. The query whether running virtualized is getting more and more prominent in recent times so might as well make it an arch-global var which the rest of the code can use.

Quoted 5 lines
Fixes d8630b67ca1ed ("x86/cpu: Add platform ID to CPU info structure")

One nit below.

[...]

arch/x86/kernel/cpu/microcode/core.c bool __init microcode_loader_disabled(void) · line 121

Quoted 13 lines
arch/x86/kernel/cpu/microcode/core.c +1 -8
@@ -118,14 +118,9 @@ bool __init microcode_loader_disabled(void)
 	/*
 	 * Disable when:
 	 *
-	 * 1) The CPU does not support CPUID.
-	 */
-	if (!cpuid_feature()) {
-		dis_ucode_ldr = true;
-		return dis_ucode_ldr;
-	}
-
-	/*
+	 * 1) The CPU does not support CPUID, detected below in
                                                         ^

Nit:
"below" is a bit ambiguous, maybe "earlier"?

Quoted 54 lines
arch/x86/kernel/cpu/microcode/core.c +8 -3
+	 *    load_ucode_bsp().
+	 *
 	 * 2) Bit 31 in CPUID[1]:ECX is set
 	 *    The bit is reserved for hypervisor use. This is still not
 	 *    completely accurate as XEN PV guests don't see that CPUID bit
@@ -135,9 +130,7 @@ bool __init microcode_loader_disabled(void)
 	 * 3) Certain AMD patch levels are not allowed to be
 	 *    overwritten.
 	 */
-	hypervisor_present = native_cpuid_ecx(1) & BIT(31);
-
-	if ((hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) ||
+	if ((x86_hypervisor_present && !IS_ENABLED(CONFIG_MICROCODE_DBG)) ||
 	    amd_check_current_patch_level())
 		dis_ucode_ldr = true;
 
@@ -179,6 +172,11 @@ void __init load_ucode_bsp(void)
 
 	early_parse_cmdline();
 
+	if (!cpuid_feature())
+		dis_ucode_ldr = true;
+	else
+		x86_hypervisor_present = native_cpuid_ecx(1) & BIT(31);
+
 	if (microcode_loader_disabled())
 		return;
 
arch/x86/kernel/cpu/microcode/intel.c +3 -0
index 37ac4afe0972..a4c0a0cf928b 100644
@@ -138,6 +138,9 @@ u32 intel_get_platform_id(void)
 {
 	unsigned int val[2];
 
+	if (x86_hypervisor_present)
+		return 0;
+
 	/*
 	 * This can be called early. Use CPUID directly instead of
 	 * relying on cpuinfo_x86 which may not be fully initialized.
arch/x86/kernel/cpu/microcode/internal.h +0 -1
index 3b93c0676b4f..a10b547eda1e 100644
@@ -48,7 +48,6 @@ extern struct early_load_data early_data;
 extern struct ucode_cpu_info ucode_cpu_info[];
 extern u32 microcode_rev[NR_CPUS];
 extern u32 base_rev;
-extern bool hypervisor_present;
 
 struct cpio_data find_microcode_in_initrd(const char *path);
 
author Reviewed-by on v1 2/2
#17 lore raw

Re: [PATCH 2/2] x86/cpu: Move intel_get_platform_id() to cpu/intel.c

In reply to Borislav Petkov
Quoted 2 lines · Borislav Petkov

It is not only used in the microcode loader anymore and the platform ID is cached in the cpuindo_x86 structure so move the getter to

Typo:
cpuindo_x86 -> cpuinfo_x86

Quoted 5 lines

Intel CPU-specific code.

No functional changes.

Quoted 125 lines
---
 arch/x86/include/asm/microcode.h      |  2 --
 arch/x86/include/asm/processor.h      |  1 +
 arch/x86/kernel/cpu/intel.c           | 35 ++++++++++++++++++++++++++
 arch/x86/kernel/cpu/microcode/intel.c | 36 ---------------------------
 4 files changed, 36 insertions(+), 38 deletions(-)
arch/x86/include/asm/microcode.h +0 -2
index 3c317d155771..8b41f26f003b 100644
@@ -61,8 +61,6 @@ static inline int intel_microcode_get_datasize(struct microcode_header_intel *hd
 	return hdr->datasize ? : DEFAULT_UCODE_DATASIZE;
 }
 
-extern u32 intel_get_platform_id(void);
-
 static inline u32 intel_get_microcode_revision(void)
 {
 	u32 rev, dummy;
arch/x86/include/asm/processor.h +1 -0
index 67dd932305db..1a68a955863d 100644
@@ -237,6 +237,7 @@ extern void early_cpu_init(void);
 extern void identify_secondary_cpu(unsigned int cpu);
 extern void print_cpu_info(struct cpuinfo_x86 *);
 void print_cpu_msr(struct cpuinfo_x86 *);
+extern u32 intel_get_platform_id(void);
 
 /*
  * Friendlier CR3 helpers.
arch/x86/kernel/cpu/intel.c +35 -0
index f28c0efb7c8f..3473b89e6907 100644
@@ -199,6 +199,41 @@ void intel_unlock_cpuid_leafs(struct cpuinfo_x86 *c)
 		c->cpuid_level = cpuid_eax(0);
 }
 
+/*
+ * Use CPUID to generate a "vfm" value. Useful before cpuinfo_x86
+ * structures are populated.
+ */
+static u32 intel_cpuid_vfm(void)
+{
+	u32 eax   = cpuid_eax(1);
+	u32 fam   = x86_family(eax);
+	u32 model = x86_model(eax);
+
+	return IFM(fam, model);
+}
+
+u32 intel_get_platform_id(void)
+{
+	unsigned int val[2];
+
+	if (x86_hypervisor_present)
+		return 0;
+
+	/*
+	 * This can be called early. Use CPUID directly instead of
+	 * relying on cpuinfo_x86 which may not be fully initialized.
+	 * The PII does not have MSR_IA32_PLATFORM_ID. Everything
+	 * before _it_ has no microcode (for Linux at least).
+	 */
+	if (intel_cpuid_vfm() <= INTEL_PENTIUM_II_KLAMATH)
+		return 0;
+
+	/* get processor flags from MSR 0x17 */
+	native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
+
+	return (val[1] >> 18) & 7;
+}
+
 static void early_init_intel(struct cpuinfo_x86 *c)
 {
 	u64 misc_enable;
arch/x86/kernel/cpu/microcode/intel.c +0 -36
index a4c0a0cf928b..f55ff81682e5 100644
@@ -120,42 +120,6 @@ static inline unsigned int exttable_size(struct extended_sigtable *et)
 	return et->count * EXT_SIGNATURE_SIZE + EXT_HEADER_SIZE;
 }
 
-
-/*
- * Use CPUID to generate a "vfm" value. Useful before cpuinfo_x86
- * structures are populated.
- */
-static u32 intel_cpuid_vfm(void)
-{
-	u32 eax   = cpuid_eax(1);
-	u32 fam   = x86_family(eax);
-	u32 model = x86_model(eax);
-
-	return IFM(fam, model);
-}
-
-u32 intel_get_platform_id(void)
-{
-	unsigned int val[2];
-
-	if (x86_hypervisor_present)
-		return 0;
-
-	/*
-	 * This can be called early. Use CPUID directly instead of
-	 * relying on cpuinfo_x86 which may not be fully initialized.
-	 * The PII does not have MSR_IA32_PLATFORM_ID. Everything
-	 * before _it_ has no microcode (for Linux at least).
-	 */
-	if (intel_cpuid_vfm() <= INTEL_PENTIUM_II_KLAMATH)
-		return 0;
-
-	/* get processor flags from MSR 0x17 */
-	native_rdmsr(MSR_IA32_PLATFORM_ID, val[0], val[1]);
-
-	return (val[1] >> 18) & 7;
-}
-
 void intel_collect_cpu_info(struct cpu_signature *sig)
 {
 	sig->sig = cpuid_eax(1);
maintainer
#13 lore raw

Re: [PATCH 1/2] x86/microcode: Do not access MSR_IA32_PLATFORM_ID when running as a guest

In reply to Binbin Wu
Quoted 7 lines · Binbin Wu

arch/x86/kernel/cpu/microcode/core.c bool __init microcode_loader_disabled(void) · line 121

Quoted 2 lines
arch/x86/kernel/cpu/microcode/core.c +1 -1
-	/*
+	 * 1) The CPU does not support CPUID, detected below in
                                                         ^

Nit:
"below" is a bit ambiguous, maybe "earlier"?

arch/x86/kernel/cpu/microcode/core.c bool __init microcode_loader_disabled(void) · line 122

Quoted 1 line
arch/x86/kernel/cpu/microcode/core.c +1 -0
+	 *    load_ucode_bsp().

The sentence says "... below in load_ucode_bsp()." It can't any clearer than that.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
Reviewed-by on v1 1/2
#14 lore raw

Re: [PATCH 1/2] x86/microcode: Do not access MSR_IA32_PLATFORM_ID when running as a guest

In reply to Borislav Petkov
Quoted 32 lines · Borislav Petkov

Patch in Fixes: causes the usual:

   unchecked MSR access error: RDMSR from 0x17 at ... (intel_get_platform_id)
   Call Trace:
    early_init_intel
    early_cpu_init
    setup_arch
    _printk
    start_kernel
    x86_64_start_reservations
    x86_64_start_kernel
    common_startup_64

because the kernel is booted in a guest.

In order to avoid it, this MSR access needs to be prevented when running virtualized. That is usually done by checking X86_FEATURE_HYPERVISOR but for this particular case it is too early yet.

The platform ID needs to be read as early as when microcode is loaded on the BSP:

   load_ucode_bsp ... -> get_microcode_blob ... -> intel_find_matching_signature

and by that time, CPUID leafs haven't been parsed yet.

The microcode loader already has logic to check early whether the kernel is running virtualized so make that globally available to arch/x86/. The query whether running virtualized is getting more and more prominent in recent times so might as well make it an arch-global var which the rest of the code can use.

Quoted 5 lines
Fixes d8630b67ca1ed ("x86/cpu: Add platform ID to CPU info structure")

arch/x86/kernel/cpu/microcode/core.c void __init load_ucode_bsp(void) · line 179

Quoted 9 lines
arch/x86/kernel/cpu/microcode/core.c +5 -0
@@ -179,6 +172,11 @@ void __init load_ucode_bsp(void)
 
  	early_parse_cmdline();
 
+	if (!cpuid_feature())
+		dis_ucode_ldr = true;
+	else
+		x86_hypervisor_present = native_cpuid_ecx(1) & BIT(31);
+

IIUC, the reason why move the initialization of x86_hypervisor_present from microcode_loader_disabled to here is to make sure x86_hypervisor_present is always initialized, right?

But initializing x86_hypervisor_present by load_ucode_bsp() depends on CONFIG_MICROCODE, which depends on CPU_SUP_AMD || CPU_SUP_INTEL. So when both CPU_SUP_AMD and CPU_SUP_INTEL are not enabled, x86_hypervisor_present will always be 0. It's not a problem for now, we can solve it when x86_hypervisor_present is going to be used by CENTAUR or ZHAOXIN.

Reviewed-by on v1 2/2
#18 lore raw

Re: [PATCH 2/2] x86/cpu: Move intel_get_platform_id() to cpu/intel.c

In reply to Borislav Petkov
Quoted 7 lines · Borislav Petkov

It is not only used in the microcode loader anymore and the platform ID is cached in the cpuindo_x86 structure so move the getter to Intel CPU-specific code.

No functional changes.

with the typo pointed by Binbin fixed,

maintainer
#15 lore raw

Re: [PATCH 1/2] x86/microcode: Do not access MSR_IA32_PLATFORM_ID when running as a guest

In reply to Xiaoyao Li
Quoted 3 lines · Xiaoyao Li

But initializing x86_hypervisor_present by load_ucode_bsp() depends on CONFIG_MICROCODE, which depends on CPU_SUP_AMD || CPU_SUP_INTEL. So when both CPU_SUP_AMD and CPU_SUP_INTEL are not enabled,

... you don't need microcode loader either.

Quoted 3 lines

x86_hypervisor_present will always be 0. It's not a problem for now, we can solve it when x86_hypervisor_present is going to be used by CENTAUR or ZHAOXIN.

Yes, we'll cross that bridge when we get to it.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette