Skip to contentSkip to conversation

Prevent attempting updates known to fail

PATCH Superseded Alejandro Vallejo 2 participants 7 messages Aug 8, 2023

v7 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

v7 cover Prevent attempting updates known to fail

Under certain conditions a CPU may not be able to perform microcode updates even if hardware exists to that effect. In particular:

* If Xen runs under certain hypervisors they won't allow microcode updates, and will signal this fact by reporting a microcode revision of -1.
* If the DIS_MCU_LOAD bit is set, which is expected in some baremetal clouds where the owner may not trust the tenant, then the CPU is not capable of loading new microcode.

This series adds logic so that in both of these cases we don't needlessly attempt updates that are not going to succeed. Patch summary:

Patch 1 Ignores microcode facilities when the current microcode revision is -1

Patch 2 Moves the MSR_ARCH_CAPS read in tsx_init() to early_cpu_init() and
early_microcode_init()

Patch 3 Adds the logic to detect microcode updates being disabled on Intel.

Alejandro Vallejo (3):
x86/microcode: Ignore microcode loading interface for revision = -1
x86: Read MSR_ARCH_CAPS immediately after early_microcode_init() x86/microcode: Disable microcode update handler if DIS_MCU_UPDATE is
set

 xen/arch/x86/cpu/common.c             | 20 ++++++++++----
 xen/arch/x86/cpu/microcode/core.c     | 38 ++++++++++++++++++++++++---
 xen/arch/x86/cpu/microcode/intel.c    | 13 +++++++++
 xen/arch/x86/cpu/microcode/private.h  |  7 +++++
 xen/arch/x86/include/asm/cpufeature.h |  1 +
 xen/arch/x86/include/asm/msr-index.h  |  5 ++++
 xen/arch/x86/include/asm/setup.h      |  2 +-
 xen/arch/x86/setup.c                  |  2 +-
 xen/arch/x86/tsx.c                    | 16 +++--------
 9 files changed, 81 insertions(+), 23 deletions(-)
Signature · 1 line
2.34.1
#1 lore raw

PATCH v7 1/3 x86/microcode: Ignore microcode loading interface for revision = -1

Some hypervisors report ~0 as the microcode revision to mean "don't issue microcode updates". Ignore the microcode loading interface in that case.

---
v7:
* Removed R-by tag. It wasn't meant to be there!
* Fix wrong substring inserted in printk statement
* Returned the printk severity to WARNING

I still think both this new print and the previous one in the function should have severity INFO instead, but I'm not bothered either way and they should both have the same severity because they highlight equivalent events.

---
 xen/arch/x86/cpu/microcode/core.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
xen/arch/x86/cpu/microcode/core.c +15 -2
index bec8b55db2..205bb52201 100644
@@ -867,10 +867,23 @@ int __init early_microcode_init(unsigned long *module_map,
         return -ENODEV;
     }
 
-    microcode_grab_module(module_map, mbi);
-
     ucode_ops.collect_cpu_info();
 
+    /*
+     * Some hypervisors deliberately report a microcode revision of -1 to
+     * mean that they will not accept microcode updates. We take the hint
+     * and ignore the microcode interface in that case.
+     */
+    if ( this_cpu(cpu_sig).rev == ~0 )
+    {
+        printk(XENLOG_WARNING "Microcode loading disabled due to: %s",
+                              "rev = ~0");
+        ucode_ops.apply_microcode = NULL;
+        return -ENODEV;
+    }
+
+    microcode_grab_module(module_map, mbi);
+
     if ( ucode_mod.mod_end || ucode_blob.size )
         rc = early_microcode_update_cpu();
 
Signature · 1 line
2.34.1
Reviewed-by on v7 1/3
#2 lore raw

Re: [PATCH v7 1/3] x86/microcode: Ignore microcode loading interface for revision = -1

Quoted 9 lines · Alejandro Vallejo
xen/arch/x86/cpu/microcode/core.c +0 -2
@@ -867,10 +867,23 @@ int __init early_microcode_init(unsigned long *module_map,
         return -ENODEV;
     }
 
-    microcode_grab_module(module_map, mbi);
-
     ucode_ops.collect_cpu_info();

xen/arch/x86/cpu/microcode/core.c int __init early_microcode_init(unsigned long *module_map, · line 880

Quoted 10 lines
xen/arch/x86/cpu/microcode/core.c +9 -0
 
+    /*
+     * Some hypervisors deliberately report a microcode revision of -1 to
+     * mean that they will not accept microcode updates. We take the hint
+     * and ignore the microcode interface in that case.
+     */
+    if ( this_cpu(cpu_sig).rev == ~0 )
+    {
+        printk(XENLOG_WARNING "Microcode loading disabled due to: %s",
+                              "rev = ~0");

Sadly there now looks to be a newline missing. Easy to add while committing, I suppose, and with it added (back)

Jan

#3 lore raw

Re: [PATCH v7 1/3] x86/microcode: Ignore microcode loading interface for revision = -1

Quoted 23 lines · Jan Beulich
Quoted 9 lines · Alejandro Vallejo
xen/arch/x86/cpu/microcode/core.c +0 -2
@@ -867,10 +867,23 @@ int __init early_microcode_init(unsigned long *module_map,
         return -ENODEV;
     }
 
-    microcode_grab_module(module_map, mbi);
-
     ucode_ops.collect_cpu_info();

xen/arch/x86/cpu/microcode/core.c int __init early_microcode_init(unsigned long *module_map, · line 880

Quoted 10 lines
xen/arch/x86/cpu/microcode/core.c +9 -0
 
+    /*
+     * Some hypervisors deliberately report a microcode revision of -1 to
+     * mean that they will not accept microcode updates. We take the hint
+     * and ignore the microcode interface in that case.
+     */
+    if ( this_cpu(cpu_sig).rev == ~0 )
+    {
+        printk(XENLOG_WARNING "Microcode loading disabled due to: %s",
+                              "rev = ~0");

Sadly there now looks to be a newline missing. Easy to add while committing, I suppose, and with it added (back)

Quoted 1 line

Actually, the 2nd string also is indented too deeply; unindenting will in particular help with line length in the 3rd patch (which isn't to say that the line's getting too long there, but it also doesn't need to grow as long as it does).

Jan

#4 lore raw

Re: [PATCH v7 1/3] x86/microcode: Ignore microcode loading interface for revision = -1

In reply to Jan Beulich

Hi,

Quoted 23 lines · Jan Beulich
Quoted 9 lines · Alejandro Vallejo
xen/arch/x86/cpu/microcode/core.c +0 -2
@@ -867,10 +867,23 @@ int __init early_microcode_init(unsigned long *module_map,
         return -ENODEV;
     }
 
-    microcode_grab_module(module_map, mbi);
-
     ucode_ops.collect_cpu_info();

xen/arch/x86/cpu/microcode/core.c int __init early_microcode_init(unsigned long *module_map, · line 880

Quoted 10 lines
xen/arch/x86/cpu/microcode/core.c +9 -0
 
+    /*
+     * Some hypervisors deliberately report a microcode revision of -1 to
+     * mean that they will not accept microcode updates. We take the hint
+     * and ignore the microcode interface in that case.
+     */
+    if ( this_cpu(cpu_sig).rev == ~0 )
+    {
+        printk(XENLOG_WARNING "Microcode loading disabled due to: %s",
+                              "rev = ~0");

Sadly there now looks to be a newline missing. Easy to add while committing, I suppose, and with it added (back)

Quoted 3 lines

Jan

You mean the missing newline after "%s"? It fell under the cracks while adding the substring in v6. It should indeed be there, cheers.

Thanks,
Alejandro

#5 lore raw

PATCH v7 2/3 x86: Read MSR_ARCH_CAPS immediately after early_microcode_init()

In reply to Alejandro Vallejo

Move MSR_ARCH_CAPS read code from tsx_init() to early_cpu_init(). Because microcode updates might make them that MSR to appear/have different values we also must reload it after a microcode update in early_microcode_init().

---
v7:
* No change

---
 xen/arch/x86/cpu/common.c         | 20 +++++++++++++++-----
 xen/arch/x86/cpu/microcode/core.c |  9 +++++++++
 xen/arch/x86/include/asm/setup.h  |  2 +-
 xen/arch/x86/setup.c              |  2 +-
 xen/arch/x86/tsx.c                | 16 ++++------------
 5 files changed, 30 insertions(+), 19 deletions(-)
xen/arch/x86/cpu/common.c +15 -5
index 6fada384a1..3fd4fd0654 100644
@@ -299,7 +299,7 @@ static inline u32 phys_pkg_id(u32 cpuid_apic, int index_msb)
 
    WARNING: this function is only called on the BP.  Don't add code here
    that is supposed to run on all CPUs. */
-void __init early_cpu_init(void)
+void __init early_cpu_init(bool verbose)
 {
 	struct cpuinfo_x86 *c = &boot_cpu_data;
 	u32 eax, ebx, ecx, edx;
@@ -320,6 +320,8 @@ void __init early_cpu_init(void)
 	case X86_VENDOR_SHANGHAI: this_cpu = &shanghai_cpu_dev; break;
 	case X86_VENDOR_HYGON:    this_cpu = &hygon_cpu_dev;    break;
 	default:
+		if (!verbose)
+			break;
 		printk(XENLOG_ERR
 		       "Unrecognised or unsupported CPU vendor '%.12s'\n",
 		       c->x86_vendor_id);
@@ -336,10 +338,13 @@ void __init early_cpu_init(void)
 	c->x86_capability[FEATURESET_1d] = edx;
 	c->x86_capability[FEATURESET_1c] = ecx;
 
-	printk(XENLOG_INFO
-	       "CPU Vendor: %s, Family %u (%#x), Model %u (%#x), Stepping %u (raw %08x)\n",
-	       x86_cpuid_vendor_to_str(c->x86_vendor), c->x86, c->x86,
-	       c->x86_model, c->x86_model, c->x86_mask, eax);
+	if (verbose)
+		printk(XENLOG_INFO
+		       "CPU Vendor: %s, Family %u (%#x), "
+		       "Model %u (%#x), Stepping %u (raw %08x)\n",
+		       x86_cpuid_vendor_to_str(c->x86_vendor), c->x86,
+		       c->x86, c->x86_model, c->x86_model, c->x86_mask,
+		       eax);
 
 	if (c->cpuid_level >= 7) {
 		uint32_t max_subleaf;
@@ -348,6 +353,11 @@ void __init early_cpu_init(void)
 			    &c->x86_capability[FEATURESET_7c0],
 			    &c->x86_capability[FEATURESET_7d0]);
 
+		if (test_bit(X86_FEATURE_ARCH_CAPS, c->x86_capability))
+			rdmsr(MSR_ARCH_CAPABILITIES,
+			      c->x86_capability[FEATURESET_m10Al],
+			      c->x86_capability[FEATURESET_m10Ah]);
+
 		if (max_subleaf >= 1)
 			cpuid_count(7, 1, &eax, &ebx, &ecx,
 				    &c->x86_capability[FEATURESET_7d1]);
xen/arch/x86/cpu/microcode/core.c +9 -0
index 205bb52201..273840fbcf 100644
@@ -887,5 +887,14 @@ int __init early_microcode_init(unsigned long *module_map,
     if ( ucode_mod.mod_end || ucode_blob.size )
         rc = early_microcode_update_cpu();
 
+    /*
+     * Some CPUID leaves and MSRs are only present after microcode updates
+     * on some processors. We take the chance here to make sure what little
+     * state we have already probed is re-probed in order to ensure we do
+     * not use stale values. tsx_init() in particular needs to have up to
+     * date MSR_ARCH_CAPS.
+     */
+    early_cpu_init(false);
+
     return rc;
 }
xen/arch/x86/include/asm/setup.h +1 -1
index 51fce66607..dad8b3e605 100644
@@ -15,7 +15,7 @@ extern uint64_t boot_tsc_stamp;
 
 extern void *stack_start;
 
-void early_cpu_init(void);
+void early_cpu_init(bool verbose);
 void early_time_init(void);
 
 void set_nr_cpu_ids(unsigned int max_cpus);
xen/arch/x86/setup.c +1 -1
index 2dbe9857aa..12cb28e969 100644
@@ -1211,7 +1211,7 @@ void __init noreturn __start_xen(unsigned long mbi_p)
         panic("Bootloader provided no memory information\n");
 
     /* This must come before e820 code because it sets paddr_bits. */
-    early_cpu_init();
+    early_cpu_init(true);
 
     /* Choose shadow stack early, to set infrastructure up appropriately. */
     if ( !boot_cpu_has(X86_FEATURE_CET_SS) )
xen/arch/x86/tsx.c +4 -12
index 80c6f4cedd..50d8059f23 100644
@@ -39,9 +39,10 @@ void tsx_init(void)
     static bool __read_mostly once;
 
     /*
-     * This function is first called between microcode being loaded, and CPUID
-     * being scanned generally.  Read into boot_cpu_data.x86_capability[] for
-     * the cpu_has_* bits we care about using here.
+     * This function is first called between microcode being loaded, and
+     * CPUID being scanned generally. early_cpu_init() has already prepared
+     * the feature bits needed here. And early_microcode_init() has ensured
+     * they are not stale after the microcode update.
      */
     if ( unlikely(!once) )
     {
@@ -49,15 +50,6 @@ void tsx_init(void)
 
         once = true;
 
-        if ( boot_cpu_data.cpuid_level >= 7 )
-            boot_cpu_data.x86_capability[FEATURESET_7d0]
-                = cpuid_count_edx(7, 0);
-
-        if ( cpu_has_arch_caps )
-            rdmsr(MSR_ARCH_CAPABILITIES,
-                  boot_cpu_data.x86_capability[FEATURESET_m10Al],
-                  boot_cpu_data.x86_capability[FEATURESET_m10Ah]);
-
         has_rtm_always_abort = cpu_has_rtm_always_abort;
 
         if ( cpu_has_tsx_ctrl && cpu_has_srbds_ctrl )
Signature · 1 line
2.34.1
#6 lore raw

PATCH v7 3/3 x86/microcode: Disable microcode update handler if DIS_MCU_UPDATE is set

In reply to Alejandro Vallejo

If IA32_MSR_MCU_CONTROL exists then it's possible a CPU may be unable to perform microcode updates. This is controlled through the DIS_MCU_LOAD bit and is intended for baremetal clouds where the owner may not trust the tenant to choose the microcode version in use. If we notice that bit being set then simply disable the "apply_microcode" handler so we can't even try to perform update (as it's known to be silently dropped).

While at it, remove the Intel family check, as microcode loading is supported on every Intel64 CPU.

---
v7:
* No change

---
 xen/arch/x86/cpu/microcode/core.c     | 20 ++++++++++++++------
 xen/arch/x86/cpu/microcode/intel.c    | 13 +++++++++++++
 xen/arch/x86/cpu/microcode/private.h  |  7 +++++++
 xen/arch/x86/include/asm/cpufeature.h |  1 +
 xen/arch/x86/include/asm/msr-index.h  |  5 +++++
 5 files changed, 40 insertions(+), 6 deletions(-)
xen/arch/x86/cpu/microcode/core.c +14 -6
index 273840fbcf..ba9c9a8b1d 100644
@@ -847,17 +847,21 @@ int __init early_microcode_init(unsigned long *module_map,
 {
     const struct cpuinfo_x86 *c = &boot_cpu_data;
     int rc = 0;
+    bool can_load = false;
 
     switch ( c->x86_vendor )
     {
     case X86_VENDOR_AMD:
         if ( c->x86 >= 0x10 )
+        {
             ucode_ops = amd_ucode_ops;
+            can_load = true;
+        }
         break;
 
     case X86_VENDOR_INTEL:
-        if ( c->x86 >= 6 )
-            ucode_ops = intel_ucode_ops;
+        ucode_ops = intel_ucode_ops;
+        can_load = intel_can_load_microcode();
         break;
     }
 
@@ -871,13 +875,17 @@ int __init early_microcode_init(unsigned long *module_map,
 
     /*
      * Some hypervisors deliberately report a microcode revision of -1 to
-     * mean that they will not accept microcode updates. We take the hint
-     * and ignore the microcode interface in that case.
+     * mean that they will not accept microcode updates.
+     *
+     * It's also possible the hardware might have built-in support to disable
+     * updates and someone (e.g: a baremetal cloud provider) disabled them.
+     *
+     * Take the hint in either case and ignore the microcode interface.
      */
-    if ( this_cpu(cpu_sig).rev == ~0 )
+    if ( this_cpu(cpu_sig).rev == ~0 || !can_load )
     {
         printk(XENLOG_WARNING "Microcode loading disabled due to: %s",
-                              "rev = ~0");
+                              can_load ? "rev = ~0" : "HW toggle");
         ucode_ops.apply_microcode = NULL;
         return -ENODEV;
     }
xen/arch/x86/cpu/microcode/intel.c +13 -0
index 8d4d6574aa..060c529a6e 100644
@@ -385,6 +385,19 @@ static struct microcode_patch *cf_check cpu_request_microcode(
     return patch;
 }
 
+bool __init intel_can_load_microcode(void)
+{
+    uint64_t mcu_ctrl;
+
+    if ( !cpu_has_mcu_ctrl )
+        return true;
+
+    rdmsrl(MSR_MCU_CONTROL, mcu_ctrl);
+
+    /* If DIS_MCU_LOAD is set applying microcode updates won't work */
+    return !(mcu_ctrl & MCU_CONTROL_DIS_MCU_LOAD);
+}
+
 const struct microcode_ops __initconst_cf_clobber intel_ucode_ops = {
     .cpu_request_microcode            = cpu_request_microcode,
     .collect_cpu_info                 = collect_cpu_info,
xen/arch/x86/cpu/microcode/private.h +7 -0
index 626aeb4d08..d80787205a 100644
@@ -60,6 +60,13 @@ struct microcode_ops {
         const struct microcode_patch *new, const struct microcode_patch *old);
 };
 
+/**
+ * Checks whether we can perform microcode updates on this Intel system
+ *
+ * @return True iff the microcode update facilities are enabled
+ */
+bool intel_can_load_microcode(void);
+
 extern const struct microcode_ops amd_ucode_ops, intel_ucode_ops;
 
 #endif /* ASM_X86_MICROCODE_PRIVATE_H */
xen/arch/x86/include/asm/cpufeature.h +1 -0
index 64e1dad225..5422baed27 100644
@@ -193,6 +193,7 @@ static inline bool boot_cpu_has(unsigned int feat)
 #define cpu_has_if_pschange_mc_no boot_cpu_has(X86_FEATURE_IF_PSCHANGE_MC_NO)
 #define cpu_has_tsx_ctrl        boot_cpu_has(X86_FEATURE_TSX_CTRL)
 #define cpu_has_taa_no          boot_cpu_has(X86_FEATURE_TAA_NO)
+#define cpu_has_mcu_ctrl        boot_cpu_has(X86_FEATURE_MCU_CTRL)
 #define cpu_has_fb_clear        boot_cpu_has(X86_FEATURE_FB_CLEAR)
 #define cpu_has_rrsba           boot_cpu_has(X86_FEATURE_RRSBA)
 
xen/arch/x86/include/asm/msr-index.h +5 -0
index 4d41c171d2..cf8b2fa15c 100644
@@ -165,6 +165,11 @@
 #define  PASID_PASID_MASK                   0x000fffff
 #define  PASID_VALID                        (_AC(1, ULL) << 31)
 
+#define MSR_MCU_CONTROL                     0x00001406
+#define  MCU_CONTROL_LOCK                   (_AC(1, ULL) <<  0)
+#define  MCU_CONTROL_DIS_MCU_LOAD           (_AC(1, ULL) <<  1)
+#define  MCU_CONTROL_EN_SMM_BYPASS          (_AC(1, ULL) <<  2)
+
 #define MSR_UARCH_MISC_CTRL                 0x00001b01
 #define  UARCH_CTRL_DOITM                   (_AC(1, ULL) <<  0)
 
Signature · 1 line
2.34.1