Skip to contentSkip to conversation

x86: proposed new ARCH_CAPABILITIES MSR bit for RSB-underflow

RFC PATCH In review Dave Hansen 4 participants 5 messages Feb 16, 2018 – Feb 19, 2018

Subsystems (2)
lore
Share & tools
lore

Dave Hansen

author maintainer
#0 lore raw

RFC PATCH 1/1 x86: proposed new ARCH_CAPABILITIES MSR bit for RSB-underflow

Intel is considering adding a new bit to the IA32_ARCH_CAPABILITIES MSR to tell when RSB underflow might be happen. Feedback on this would be greatly appreciated before the specification is finalized.

---

Background:

The RSB is a microarchitectural structure that attempts to help predict the branch target of RET instructions. It is implemented as a stack that is pushed on CALL and popped on RET. Being a stack, it can become empty. On some processors, an empty condition leads to use of the other indirect branch predictors which have been targeted by Spectre variant 2 (branch target injection) exploits.

Processors based on Skylake and its close derivatives have this fallback behavior and need additional mitigation to avoid RSB-empty conditions. Right now, the only place we do this "RSB stuffing" operation is at context switch. We currently have a model/family list to decide where to deploy this.

Problem:

However, that causes a problem in virtualization environments. They routinely expose a different model/family to guests than what the bare-metal hardware has. This, among other things, makes it easy to migrate guests between different bare-metal systems with different capabilities. However, this defeats the Skylake-generation model/family detection.

Solution:

To help address this issue, Intel is proposing a new bit in the IA32_ARCH_CAPABILITIES MSR. This bit, "RSB Override" (RSBO) would indicate:

The CPU may predict the target of RET instructions with a predictor other than the RSB when the RSB is 'empty'.

Hardware implementations may choose to set this, but it can also be set by a hypervisor that traps RDMSR and simply wants to indicate to a guest that it should deploy RSB-underflow mitigations.

An OS should assume that RSB-underflow mitigations are needed both when RSBO=1 or when running on Skylake-generation processors with RSBO=0.

Cc Linus Torvalds <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Rik van Riel <[email protected]>
Cc Josh Poimboeuf <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Jiri Kosina <[email protected]>
Cc Andy Lutomirski <[email protected]>
Cc Kees Cook <[email protected]>
Cc Greg Kroah-Hartman <[email protected]>
Cc Paul Turner <[email protected]>
Cc David Woodhouse <[email protected]>
Cc Andi Kleen <[email protected]>
Cc Tim Chen <[email protected]>
Cc Arjan van de Ven <[email protected]>
Cc Dan Williams <[email protected]>
Cc Asit Mallick <[email protected]>

---

 b/arch/x86/include/asm/msr-index.h |    1 
 b/arch/x86/kernel/cpu/bugs.c       |   47 +++++++++++++++++++++++++++++++++----
 2 files changed, 43 insertions(+), 5 deletions(-)
arch/x86/kernel/cpu/bugs.c renamed from arch/x86/kernel/cpu/bugs.c~need-rsb-stuffing +42 -5
diff -puN arch/x86/kernel/cpu/bugs.c~need-rsb-stuffing arch/x86/kernel/cpu/bugs.c
@@ -218,6 +218,43 @@ static bool __init is_skylake_era(void)
 	return false;
 }
 
+/*
+ * This MSR has a bit to indicate whether the processor might fall back to
+ * the BTB.  Hypervisors might lie about the model/family, breaking the
+ * is_skylake_era() check.  They might also want the OS to deploy
+ * mitigations because it *might* get migrated to other hardware that has
+ * this behavior, even if current bare-metal hardware is not exposed.
+ */
+static bool cpu_has_rsb_override(void)
+{
+	u64 ia32_cap = 0;
+
+        if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES))
+		rdmsrl(MSR_IA32_ARCH_CAPABILITIES, ia32_cap);
+
+        /* RSBO == RSB Override */
+        if (ia32_cap & ARCH_CAP_RSBO)
+		return true;
+
+	return false;
+}
+
+/*
+ * Can a RET instruction on this CPU fall back to the BTB?
+ */
+static bool __init cpu_ret_uses_btb(void)
+{
+	/* All Skylake-era processors fall back to BTB */
+	if (is_skylake_era())
+		return true;
+
+	/* Does the ARCH_CAPABILITIES override the model/family we see? */
+	if (cpu_has_rsb_override())
+		return true;
+
+	return false;
+}
+
 static void __init spectre_v2_select_mitigation(void)
 {
 	enum spectre_v2_mitigation_cmd cmd = spectre_v2_parse_cmdline();
@@ -283,14 +320,14 @@ retpoline_auto:
 	 * from a shallow call stack to a deeper one. To prevent this fill
 	 * the entire RSB, even when using IBRS.
 	 *
-	 * Skylake era CPUs have a separate issue with *underflow* of the
-	 * RSB, when they will predict 'ret' targets from the generic BTB.
-	 * The proper mitigation for this is IBRS. If IBRS is not supported
-	 * or deactivated in favour of retpolines the RSB fill on context
+	 * Some CPUs have a separate issue with *underflow* of the RSB,
+	 * when they will predict 'ret' targets from the generic BTB.  The
+	 * proper mitigation for this is IBRS. If IBRS is not supported or
+	 * deactivated in favour of retpolines the RSB fill on context
 	 * switch is required.
 	 */
 	if ((!boot_cpu_has(X86_FEATURE_PTI) &&
-	     !boot_cpu_has(X86_FEATURE_SMEP)) || is_skylake_era()) {
+	     !boot_cpu_has(X86_FEATURE_SMEP)) || cpu_ret_uses_btb()) {
 		setup_force_cpu_cap(X86_FEATURE_RSB_CTXSW);
 		pr_info("Spectre v2 mitigation: Filling RSB on context switch\n");
 	}
arch/x86/include/asm/msr-index.h renamed from arch/x86/include/asm/msr-index.h~need-rsb-stuffing +1 -0
diff -puN arch/x86/include/asm/msr-index.h~need-rsb-stuffing arch/x86/include/asm/msr-index.h
@@ -68,6 +68,7 @@
 #define MSR_IA32_ARCH_CAPABILITIES	0x0000010a
 #define ARCH_CAP_RDCL_NO		(1 << 0)   /* Not susceptible to Meltdown */
 #define ARCH_CAP_IBRS_ALL		(1 << 1)   /* Enhanced IBRS support */
+#define ARCH_CAP_RSBO			(1 << 2)   /* Needs RSB Stuffing */
 
 #define MSR_IA32_BBL_CR_CTL		0x00000119
 #define MSR_IA32_BBL_CR_CTL3		0x0000011e

_

#1 lore raw
Quoted 4 lines · Dave Hansen

Intel is considering adding a new bit to the IA32_ARCH_CAPABILITIES MSR to tell when RSB underflow might be happen. Feedback on this would be greatly appreciated before the specification is finalized.

Yes, please. It would be lovely to not have any "this model" kind of checks.

Of course, your patch still doesn't allow for "we claim to be skylake for various other independent reasons, but the RSB issue is fixed".

So it might actually be even better with _two_ bits: "explicitly needs RSB stuffing" and "explicitly fixed and does _not_ need RSB stuffing".

And then if neither bit it set, we fall back to the implicit "we know Skylake needs it".

If both bits are set, we just go with a "CPU is batshit schitzo" message, and assume it needs RSB stuffing just because it's obviously broken.

Linus

#2 lore raw
Quoted 3 lines · Linus Torvalds

Of course, your patch still doesn't allow for "we claim to be skylake for various other independent reasons, but the RSB issue is fixed".

.. maybe nobody ever has a reason to do that, though?

Who knows, virtualization people may simply want the user to specify the model, but then make the Spectre decisions be based on actual hardware capabilities (whether those are "current" or "some minimum base").

Two bits allow that. One bit means "if you claim you're running skylake, we'll always have to stuff, whether you _really_ are or not".

Linus

#3 lore raw
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Linus Torvalds

Of course, your patch still doesn't allow for "we claim to be skylake for various other independent reasons, but the RSB issue is fixed".

.. maybe nobody ever has a reason to do that, though?

yeah I would be extremely surprised

Quoted 5 lines

Who knows, virtualization people may simply want the user to specify the model, but then make the Spectre decisions be based on actual hardware capabilities (whether those are "current" or "some minimum base").

once you fake to be skylake when you're not, you do that for a reason; normallyt that reason is that you COULD migrate to a skylake.
(and migration is not supposed to be visble to the guest OS)

and at that point you are a skylake for all intents and purposes.

(and the virtualization people also really hate it when the hardware burst the bubble of this fakeing hardware to be not what it is)

#4 lore raw
In reply to Dave Hansen

arch/x86/kernel/cpu/bugs.c static bool __init is_skylake_era(void) · line 237

Quoted 10 lines · Dave Hansen
arch/x86/kernel/cpu/bugs.c renamed from arch/x86/kernel/cpu/bugs.c~need-rsb-stuffing +10 -0
+static bool cpu_has_rsb_override(void)
+{
+       u64 ia32_cap = 0;
+
+        if (boot_cpu_has(X86_FEATURE_ARCH_CAPABILITIES))
+               rdmsrl(MSR_IA32_ARCH_CAPABILITIES, ia32_cap);
+
+        /* RSBO == RSB Override */
+        if (ia32_cap & ARCH_CAP_RSBO)
+               return true;

That would be the third such construct. As some point we should probably put the bits of MSR_IA32_ARCH_CAPABILITIES into cpufeatures a bit like the 'scattered' CPUID bits.

Amazon Web Services UK Limited. Registered in England and Wales with registration number 08650665 with its registered office at 1 Principal Place, Worship Street, London, EC2A 2FA, United Kingdom.