Skip to contentSkip to conversation

x86/bugs: RSB mitigation fixes and documentation

PATCH Superseded Josh Poimboeuf 9 participants 32 messages Apr 2, 2025 – Apr 9, 2025

Subsystems (6)
Documentation x86 KVM x86 MM +2 more subsystems

v3 of 4 Superseded Latest is v4 →

lore
Share & tools
lore

Josh Poimboeuf

author maintainer
#0 lore raw

v3 cover x86/bugs: RSB mitigation fixes and documentation

v3:
- fancy new doc
- more fixes

v2: https://lore.kernel.org/[email protected]

Josh Poimboeuf (6):
x86/bugs: Rename entry_ibpb()
x86/bugs: Use SBPB in __write_ibpb() if applicable x86/bugs: Fix RSB clearing in indirect_branch_prediction_barrier() x86/bugs: Don't fill RSB on VMEXIT with eIBRS+retpoline x86/bugs: Don't fill RSB on context switch with eIBRS x86/bugs: Add RSB mitigation document

 Documentation/admin-guide/hw-vuln/index.rst |   1 +
 Documentation/admin-guide/hw-vuln/rsb.rst   | 241 ++++++++++++++++++++
 arch/x86/entry/entry.S                      |   9 +-
 arch/x86/include/asm/nospec-branch.h        |  12 +-
 arch/x86/kernel/cpu/bugs.c                  | 109 +++------
 arch/x86/mm/tlb.c                           |   6 +-
 6 files changed, 291 insertions(+), 87 deletions(-)
 create mode 100644 Documentation/admin-guide/hw-vuln/rsb.rst
Signature · 1 line
2.48.1
author maintainer
#1 lore raw

PATCH v3 1/6 x86/bugs: Rename entry_ibpb()

There's nothing entry-specific about entry_ibpb(). In preparation for calling it from elsewhere, rename it to __write_ibpb().

---
 arch/x86/entry/entry.S               | 7 ++++---
 arch/x86/include/asm/nospec-branch.h | 6 +++---
 arch/x86/kernel/cpu/bugs.c           | 6 +++---
 3 files changed, 10 insertions(+), 9 deletions(-)
arch/x86/entry/entry.S +4 -3
index d3caa31240ed..3a53319988b9 100644
@@ -17,7 +17,8 @@
 
 .pushsection .noinstr.text, "ax"
 
-SYM_FUNC_START(entry_ibpb)
+// Clobbers AX, CX, DX
+SYM_FUNC_START(__write_ibpb)
 	ANNOTATE_NOENDBR
 	movl	$MSR_IA32_PRED_CMD, %ecx
 	movl	$PRED_CMD_IBPB, %eax
@@ -27,9 +28,9 @@ SYM_FUNC_START(entry_ibpb)
 	/* Make sure IBPB clears return stack preductions too. */
 	FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS, X86_BUG_IBPB_NO_RET
 	RET
-SYM_FUNC_END(entry_ibpb)
+SYM_FUNC_END(__write_ibpb)
 /* For KVM */
-EXPORT_SYMBOL_GPL(entry_ibpb);
+EXPORT_SYMBOL_GPL(__write_ibpb);
 
 .popsection
 
arch/x86/include/asm/nospec-branch.h +3 -3
index 8a5cc8e70439..bbac79cad04c 100644
@@ -269,7 +269,7 @@
  * typically has NO_MELTDOWN).
  *
  * While retbleed_untrain_ret() doesn't clobber anything but requires stack,
- * entry_ibpb() will clobber AX, CX, DX.
+ * __write_ibpb() will clobber AX, CX, DX.
  *
  * As such, this must be placed after every *SWITCH_TO_KERNEL_CR3 at a point
  * where we have a stack but before any RET instruction.
@@ -279,7 +279,7 @@
 	VALIDATE_UNRET_END
 	CALL_UNTRAIN_RET
 	ALTERNATIVE_2 "",						\
-		      "call entry_ibpb", \ibpb_feature,			\
+		      "call __write_ibpb", \ibpb_feature,			\
 		     __stringify(\call_depth_insns), X86_FEATURE_CALL_DEPTH
 #endif
 .endm
@@ -368,7 +368,7 @@ extern void srso_return_thunk(void);
 extern void srso_alias_return_thunk(void);
 
 extern void entry_untrain_ret(void);
-extern void entry_ibpb(void);
+extern void __write_ibpb(void);
 
 #ifdef CONFIG_X86_64
 extern void clear_bhb_loop(void);
arch/x86/kernel/cpu/bugs.c +3 -3
index 4386aa6c69e1..310cb3f7139c 100644
@@ -1142,7 +1142,7 @@ static void __init retbleed_select_mitigation(void)
 		setup_clear_cpu_cap(X86_FEATURE_RETHUNK);
 
 		/*
-		 * There is no need for RSB filling: entry_ibpb() ensures
+		 * There is no need for RSB filling: __write_ibpb() ensures
 		 * all predictions, including the RSB, are invalidated,
 		 * regardless of IBPB implementation.
 		 */
@@ -2676,7 +2676,7 @@ static void __init srso_select_mitigation(void)
 				setup_clear_cpu_cap(X86_FEATURE_RETHUNK);
 
 				/*
-				 * There is no need for RSB filling: entry_ibpb() ensures
+				 * There is no need for RSB filling: __write_ibpb() ensures
 				 * all predictions, including the RSB, are invalidated,
 				 * regardless of IBPB implementation.
 				 */
@@ -2701,7 +2701,7 @@ static void __init srso_select_mitigation(void)
 				srso_mitigation = SRSO_MITIGATION_IBPB_ON_VMEXIT;
 
 				/*
-				 * There is no need for RSB filling: entry_ibpb() ensures
+				 * There is no need for RSB filling: __write_ibpb() ensures
 				 * all predictions, including the RSB, are invalidated,
 				 * regardless of IBPB implementation.
 				 */
Signature · 1 line
2.48.1
maintainer
#2 lore raw

Re: [PATCH v3 1/6] x86/bugs: Rename entry_ibpb()

Quoted 1 line · Josh Poimboeuf

There's nothing entry-specific about entry_ibpb(). In preparation for

Not anymore - it was done on entry back then AFAIR.

Quoted 14 lines

calling it from elsewhere, rename it to __write_ibpb().

---
 arch/x86/entry/entry.S               | 7 ++++---
 arch/x86/include/asm/nospec-branch.h | 6 +++---
 arch/x86/kernel/cpu/bugs.c           | 6 +++---
 3 files changed, 10 insertions(+), 9 deletions(-)
arch/x86/entry/entry.S +0 -0
index d3caa31240ed..3a53319988b9 100644

arch/x86/entry/entry.S line 20

Quoted 5 lines
arch/x86/entry/entry.S +1 -1
@@ -17,7 +17,8 @@
 
 .pushsection .noinstr.text, "ax"
 
-SYM_FUNC_START(entry_ibpb)
+// Clobbers AX, CX, DX

Why the ugly comment style if the rest of the file is already using the multiline one...

arch/x86/entry/entry.S line 21

Quoted 1 line
arch/x86/entry/entry.S +1 -0
+SYM_FUNC_START(__write_ibpb)

... and why the __ ?

Quoted 3 lines
arch/x86/entry/entry.S +0 -0
 	ANNOTATE_NOENDBR
 	movl	$MSR_IA32_PRED_CMD, %ecx
 	movl	$PRED_CMD_IBPB, %eax
Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
author maintainer
#3 lore raw

Re: [PATCH v3 1/6] x86/bugs: Rename entry_ibpb()

Quoted 20 lines · Borislav Petkov
Quoted 1 line · Josh Poimboeuf

There's nothing entry-specific about entry_ibpb(). In preparation for

Not anymore - it was done on entry back then AFAIR.

Quoted 15 lines

calling it from elsewhere, rename it to __write_ibpb().

---
 arch/x86/entry/entry.S               | 7 ++++---
 arch/x86/include/asm/nospec-branch.h | 6 +++---
 arch/x86/kernel/cpu/bugs.c           | 6 +++---
 3 files changed, 10 insertions(+), 9 deletions(-)
arch/x86/entry/entry.S +0 -0
index d3caa31240ed..3a53319988b9 100644
@@ -17,7 +17,8 @@
 
Quoted 7 lines

arch/x86/entry/entry.S line 20

Quoted 4 lines
arch/x86/entry/entry.S +1 -1
 .pushsection .noinstr.text, "ax"
 
-SYM_FUNC_START(entry_ibpb)
+// Clobbers AX, CX, DX

Why the ugly comment style if the rest of the file is already using the multiline one...

It helps it stand out more? :-)

Quoted 3 lines

arch/x86/entry/entry.S line 21

Quoted 1 line
arch/x86/entry/entry.S +1 -0
+SYM_FUNC_START(__write_ibpb)

... and why the __ ?

I was thinking the calling interface is a bit nonstandard. But actually it's fine to call from C as those registers are already caller-saved anyway. So yeah, let's drop the '__'.

Signature · 1 line
Josh
maintainer
#4 lore raw

Re: [PATCH v3 1/6] x86/bugs: Rename entry_ibpb()

Quoted 1 line · Josh Poimboeuf

It helps it stand out more? :-)

Please don't.

Someone thought that it is a good idea to start using that // ugliness all of a sudden.

So we decided we should limit it in tip:

Documentation/process/maintainer-tip.rst

The paragrapn that starts with "Use C++ style, tail comments when documenting ..."

Quoted 3 lines

I was thinking the calling interface is a bit nonstandard. But actually it's fine to call from C as those registers are already caller-saved anyway. So yeah, let's drop the '__'.

Thx.

Signature · 4 lines
Regards/Gruss,
    Boris.

https://people.kernel.org/tglx/notes-about-netiquette
#5 lore raw

Re: [PATCH v3 1/6] x86/bugs: Rename entry_ibpb()

In reply to Josh Poimboeuf
Quoted 2 lines · Josh Poimboeuf

There's nothing entry-specific about entry_ibpb(). In preparation for calling it from elsewhere, rename it to __write_ibpb().

Minor Git-log hygienic request, could you please mention in such patches what the *new* name is?

This title is annoyingly passive-aggressive:

x86/bugs: Rename entry_ibpb()

Let's make it:

x86/bugs: Rename entry_ibpb() to write_ibpb()

... or so. Because the new name is infinitely more important going forward than the old name is ever going to be. :-)

Thanks,

Ingo

author maintainer
#6 lore raw

Re: [PATCH v3 1/6] x86/bugs: Rename entry_ibpb()

Quoted 10 lines · Ingo Molnar
Quoted 2 lines · Josh Poimboeuf

There's nothing entry-specific about entry_ibpb(). In preparation for calling it from elsewhere, rename it to __write_ibpb().

Minor Git-log hygienic request, could you please mention in such patches what the *new* name is?

This title is annoyingly passive-aggressive:

LOL

Quoted 8 lines

x86/bugs: Rename entry_ibpb()

Let's make it:

x86/bugs: Rename entry_ibpb() to write_ibpb()

... or so. Because the new name is infinitely more important going forward than the old name is ever going to be. :-)

Indeed, thanks.

Signature · 1 line
Josh
author maintainer
#7 lore raw

PATCH v3 2/6 x86/bugs: Use SBPB in __write_ibpb() if applicable

In reply to Josh Poimboeuf

__write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

---
 arch/x86/entry/entry.S     | 2 +-
 arch/x86/kernel/cpu/bugs.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/entry/entry.S +1 -1
index 3a53319988b9..a5b421ec19c0 100644
@@ -21,7 +21,7 @@
 SYM_FUNC_START(__write_ibpb)
 	ANNOTATE_NOENDBR
 	movl	$MSR_IA32_PRED_CMD, %ecx
-	movl	$PRED_CMD_IBPB, %eax
+	movl	_ASM_RIP(x86_pred_cmd), %eax
 	xorl	%edx, %edx
 	wrmsr
 
arch/x86/kernel/cpu/bugs.c +1 -1
index 310cb3f7139c..c8b8dc829046 100644
@@ -58,7 +58,7 @@ EXPORT_SYMBOL_GPL(x86_spec_ctrl_base);
 DEFINE_PER_CPU(u64, x86_spec_ctrl_current);
 EXPORT_PER_CPU_SYMBOL_GPL(x86_spec_ctrl_current);
 
-u64 x86_pred_cmd __ro_after_init = PRED_CMD_IBPB;
+u32 x86_pred_cmd __ro_after_init = PRED_CMD_IBPB;
 EXPORT_SYMBOL_GPL(x86_pred_cmd);
 
 static u64 __ro_after_init x86_arch_cap_msr;
Signature · 1 line
2.48.1
#8 lore raw

Re: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

Quoted 4 lines · Josh Poimboeuf

__write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

Maybe add something here that indicates the x86_pred_cmd variable tracks this optimization so switch to using that variable vs the hardcoded IBPB?

Thanks,
Tom

Quoted 33 lines
---
 arch/x86/entry/entry.S     | 2 +-
 arch/x86/kernel/cpu/bugs.c | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
arch/x86/entry/entry.S +1 -1
index 3a53319988b9..a5b421ec19c0 100644
@@ -21,7 +21,7 @@
 SYM_FUNC_START(__write_ibpb)
 	ANNOTATE_NOENDBR
 	movl	$MSR_IA32_PRED_CMD, %ecx
-	movl	$PRED_CMD_IBPB, %eax
+	movl	_ASM_RIP(x86_pred_cmd), %eax
 	xorl	%edx, %edx
 	wrmsr
 
arch/x86/kernel/cpu/bugs.c +1 -1
index 310cb3f7139c..c8b8dc829046 100644
@@ -58,7 +58,7 @@ EXPORT_SYMBOL_GPL(x86_spec_ctrl_base);
 DEFINE_PER_CPU(u64, x86_spec_ctrl_current);
 EXPORT_PER_CPU_SYMBOL_GPL(x86_spec_ctrl_current);
 
-u64 x86_pred_cmd __ro_after_init = PRED_CMD_IBPB;
+u32 x86_pred_cmd __ro_after_init = PRED_CMD_IBPB;
 EXPORT_SYMBOL_GPL(x86_pred_cmd);
 
 static u64 __ro_after_init x86_arch_cap_msr;
author maintainer
#9 lore raw

Re: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

Quoted 8 lines · Tom Lendacky
Quoted 4 lines · Josh Poimboeuf

__write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

Maybe add something here that indicates the x86_pred_cmd variable tracks this optimization so switch to using that variable vs the hardcoded IBPB?

Indeed, adding a second paragraph to clarify that:

x86/bugs: Use SBPB in write_ibpb() if applicable

write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

The 'x86_pred_cmd' variable already keeps track of whether IBPB or SBPB should be used. Use that instead of hardcoding IBPB.

Signature · 1 line
Josh
#10 lore raw

Re: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

In reply to Josh Poimboeuf
Quoted 5 lines · Josh Poimboeuf

__write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

When nested SVM is not supported, should KVM "promote" SRSO_USER_KERNEL_NO on the host to SRSO_NO in KVM_GET_SUPPORTED_CPUID? Or is a Linux guest clever enough to do the promotion itself if CPUID.80000001H:ECX.SVM[bit 2] is clear?

author maintainer
#11 lore raw

Re: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

Quoted 11 lines · Jim Mattson
Quoted 5 lines · Josh Poimboeuf

__write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

When nested SVM is not supported, should KVM "promote" SRSO_USER_KERNEL_NO on the host to SRSO_NO in KVM_GET_SUPPORTED_CPUID? Or is a Linux guest clever enough to do the promotion itself if CPUID.80000001H:ECX.SVM[bit 2] is clear?

I'm afraid that question is beyond my pay grade, maybe some AMD or virt folks can chime in.

Signature · 1 line
Josh
#12 lore raw

Re: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

Quoted 9 lines · Josh Poimboeuf
Quoted 7 lines · Jim Mattson
Quoted 5 lines · Josh Poimboeuf

__write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

Quoted 7 lines
Quoted 4 lines

When nested SVM is not supported, should KVM "promote" SRSO_USER_KERNEL_NO on the host to SRSO_NO in KVM_GET_SUPPORTED_CPUID? Or is a Linux guest clever enough to do the promotion itself if CPUID.80000001H:ECX.SVM[bit 2] is clear?

I'm afraid that question is beyond my pay grade, maybe some AMD or virt folks can chime in.

That question aside, I'm not sure that this series is safe with respect to nested virtualization.

If the CPU has SRSO_NO, then KVM will report SRSO_NO in KVM_GET_SUPPORTED_CPUID. However, in nested virtualization, the L1 guest and the L2 guest share a prediction domain. KVM currently ensures isolation between L1 and L2 with a call to indirect_branch_prediction_barrier() in svm_vcpu_load(). I think that particular barrier should *always* be a full IBPB--even if the host has SRSO_NO.

#13 lore raw

RE: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

[AMD Official Use Only - AMD Internal Distribution Only]

Quoted 43 lines

-----Original Message-----
From: Jim Mattson <[email protected]>
Sent: Wednesday, April 9, 2025 11:07 AM
To: Josh Poimboeuf <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Shah, Amit <[email protected]>; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Das1, Sandipan <[email protected]>; [email protected]; Moger, Babu <[email protected]>; Kaplan, David <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 3 lines · Josh Poimboeuf
Quoted 1 line · Jim Mattson

On Wed, Apr 2, 2025 at 11:20 AM Josh Poimboeuf <[email protected]>

Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 5 lines

__write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

When nested SVM is not supported, should KVM "promote" SRSO_USER_KERNEL_NO on the host to SRSO_NO in

KVM_GET_SUPPORTED_CPUID?

Quoted 5 lines
Quoted 2 lines

Or is a Linux guest clever enough to do the promotion itself if CPUID.80000001H:ECX.SVM[bit 2] is clear?

I'm afraid that question is beyond my pay grade, maybe some AMD or virt folks can chime in.

That question aside, I'm not sure that this series is safe with respect to nested virtualization.

Quoted 6 lines

If the CPU has SRSO_NO, then KVM will report SRSO_NO in KVM_GET_SUPPORTED_CPUID. However, in nested virtualization, the L1 guest and the L2 guest share a prediction domain. KVM currently ensures isolation between L1 and L2 with a call to indirect_branch_prediction_barrier() in svm_vcpu_load(). I think that particular barrier should *always* be a full IBPB--even if the host has SRSO_NO.

I don't think that's true.

If SRSO_NO=1, the indirect_branch_prediction_barrier() in svm_vcpu_load() I believe only needs to prevent indirect predictions from leaking from one VM to another, which is what SBPB provides. Keep in mind that before SRSO came out, IBPB on these parts was only flushing indirect predictions. SBPB become the 'legacy' IBPB functionality while IBPB turned into a full flush (on certain parts). If the CPU is immune to SRSO, you don't need the full flush.

I also don't think promoting SRSO_USER_KERNEL_NO to SRSO_NO should ever be done. When SRSO_NO=1, it tells the OS that it can use SBPB on context switches because the only process->process BTB concern is with indirect predictions. The OS has to use IBPB if SRSO_NO=0 (regardless of SRSO_USER_KERNEL_NO) to prevent SRSO attacks from process->process.

--David Kaplan

#14 lore raw

Re: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

Quoted 53 lines · David

[AMD Official Use Only - AMD Internal Distribution Only]

Quoted 43 lines

-----Original Message-----
From: Jim Mattson <[email protected]>
Sent: Wednesday, April 9, 2025 11:07 AM
To: Josh Poimboeuf <[email protected]>
Cc: [email protected]; [email protected]; [email protected]; [email protected]; Shah, Amit <[email protected]>; Lendacky, Thomas <[email protected]>; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; [email protected]; Das1, Sandipan <[email protected]>; [email protected]; Moger, Babu <[email protected]>; Kaplan, David <[email protected]>; [email protected]; [email protected]
Subject: Re: [PATCH v3 2/6] x86/bugs: Use SBPB in __write_ibpb() if applicable

Caution: This message originated from an External Source. Use proper caution when opening attachments, clicking links, or responding.

Quoted 3 lines · Josh Poimboeuf
Quoted 1 line · Jim Mattson

On Wed, Apr 2, 2025 at 11:20 AM Josh Poimboeuf <[email protected]>

Quoted 8 lines · wrote:
Quoted 8 lines
Quoted 5 lines

__write_ibpb() does IBPB, which (among other things) flushes branch type predictions on AMD. If the CPU has SRSO_NO, or if the SRSO mitigation has been disabled, branch type flushing isn't needed, in which case the lighter-weight SBPB can be used.

When nested SVM is not supported, should KVM "promote" SRSO_USER_KERNEL_NO on the host to SRSO_NO in

KVM_GET_SUPPORTED_CPUID?

Quoted 5 lines
Quoted 2 lines

Or is a Linux guest clever enough to do the promotion itself if CPUID.80000001H:ECX.SVM[bit 2] is clear?

I'm afraid that question is beyond my pay grade, maybe some AMD or virt folks can chime in.

That question aside, I'm not sure that this series is safe with respect to nested virtualization.

Quoted 6 lines

If the CPU has SRSO_NO, then KVM will report SRSO_NO in KVM_GET_SUPPORTED_CPUID. However, in nested virtualization, the L1 guest and the L2 guest share a prediction domain. KVM currently ensures isolation between L1 and L2 with a call to indirect_branch_prediction_barrier() in svm_vcpu_load(). I think that particular barrier should *always* be a full IBPB--even if the host has SRSO_NO.

Quoted 5 lines

I don't think that's true.

If SRSO_NO=1, the indirect_branch_prediction_barrier() in svm_vcpu_load() I believe only needs to prevent indirect predictions from leaking from one VM to another, which is what SBPB provides. Keep in mind that before SRSO came out, IBPB on these parts was only flushing indirect predictions. SBPB become the 'legacy' IBPB functionality while IBPB turned into a full flush (on certain parts). If the CPU is immune to SRSO, you don't need the full flush.

I also don't think promoting SRSO_USER_KERNEL_NO to SRSO_NO should ever be done. When SRSO_NO=1, it tells the OS that it can use SBPB on context switches because the only process->process BTB concern is with indirect predictions. The OS has to use IBPB if SRSO_NO=0 (regardless of SRSO_USER_KERNEL_NO) to prevent SRSO attacks from process->process.

Thanks, David!

Conceptually, it sounds like SRSO_NO = (SRSO_USER_KERNEL_NO + SRSO_SAME_MODE_NO). You don't need an IBPB between L1 and L2 on SRSO_NO parts, because SRSO_SAME_MODE_NO is implied. Similarly, you can't "promote" SRSO_USER_KERNEL_NO to SRSO_NO, even absent SVM, because SRSO_SAME_MODE_NO is missing.

author maintainer
#15 lore raw

PATCH v3 3/6 x86/bugs: Fix RSB clearing in indirect_branch_prediction_barrier()

In reply to Josh Poimboeuf

IBPB is expected to clear the RSB. However, if X86_BUG_IBPB_NO_RET is set, that doesn't happen. Make indirect_branch_prediction_barrier() take that into account by calling __write_ibpb() which already does the right thing.

Fixes 50e4b3b94090 ("x86/entry: Have entry_ibpb() invalidate return predictions")
---
 arch/x86/include/asm/nospec-branch.h | 6 +++---
 arch/x86/kernel/cpu/bugs.c           | 1 -
 2 files changed, 3 insertions(+), 4 deletions(-)
arch/x86/include/asm/nospec-branch.h +3 -3
index bbac79cad04c..f99b32f014ec 100644
@@ -514,11 +514,11 @@ void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature)
 		: "memory");
 }
 
-extern u64 x86_pred_cmd;
-
 static inline void indirect_branch_prediction_barrier(void)
 {
-	alternative_msr_write(MSR_IA32_PRED_CMD, x86_pred_cmd, X86_FEATURE_IBPB);
+	asm_inline volatile(ALTERNATIVE("", "call __write_ibpb", X86_FEATURE_IBPB)
+			    : ASM_CALL_CONSTRAINT
+			    :: "rax", "rcx", "rdx", "memory");
 }
 
 /* The Intel SPEC CTRL MSR base value cache */
arch/x86/kernel/cpu/bugs.c +0 -1
index c8b8dc829046..9f9637cff7a3 100644
@@ -59,7 +59,6 @@ DEFINE_PER_CPU(u64, x86_spec_ctrl_current);
 EXPORT_PER_CPU_SYMBOL_GPL(x86_spec_ctrl_current);
 
 u32 x86_pred_cmd __ro_after_init = PRED_CMD_IBPB;
-EXPORT_SYMBOL_GPL(x86_pred_cmd);
 
 static u64 __ro_after_init x86_arch_cap_msr;
 
Signature · 1 line
2.48.1
#16 lore raw

Re: [PATCH v3 3/6] x86/bugs: Fix RSB clearing in indirect_branch_prediction_barrier()

Quoted 4 lines · Josh Poimboeuf

IBPB is expected to clear the RSB. However, if X86_BUG_IBPB_NO_RET is set, that doesn't happen. Make indirect_branch_prediction_barrier() take that into account by calling __write_ibpb() which already does the right thing.

I find this changelog somewhat dubious. So zen < 4 basically have IBPB_NO_RET, your patch 2 in this series makes using SBPB for cores which have SRSO_NO or if the mitigation is disabled. So if you have a core which is zen <4 and doesn't use SBPB then what happens?

Quoted 39 lines
Fixes 50e4b3b94090 ("x86/entry: Have entry_ibpb() invalidate return predictions")
---
  arch/x86/include/asm/nospec-branch.h | 6 +++---
  arch/x86/kernel/cpu/bugs.c           | 1 -
  2 files changed, 3 insertions(+), 4 deletions(-)
arch/x86/include/asm/nospec-branch.h +3 -3
index bbac79cad04c..f99b32f014ec 100644
@@ -514,11 +514,11 @@ void alternative_msr_write(unsigned int msr, u64 val, unsigned int feature)
  		: "memory");
  }
 
-extern u64 x86_pred_cmd;
-
  static inline void indirect_branch_prediction_barrier(void)
  {
-	alternative_msr_write(MSR_IA32_PRED_CMD, x86_pred_cmd, X86_FEATURE_IBPB);
+	asm_inline volatile(ALTERNATIVE("", "call __write_ibpb", X86_FEATURE_IBPB)
+			    : ASM_CALL_CONSTRAINT
+			    :: "rax", "rcx", "rdx", "memory");
  }
 
  /* The Intel SPEC CTRL MSR base value cache */
arch/x86/kernel/cpu/bugs.c +0 -1
index c8b8dc829046..9f9637cff7a3 100644
@@ -59,7 +59,6 @@ DEFINE_PER_CPU(u64, x86_spec_ctrl_current);
  EXPORT_PER_CPU_SYMBOL_GPL(x86_spec_ctrl_current);
 
  u32 x86_pred_cmd __ro_after_init = PRED_CMD_IBPB;
-EXPORT_SYMBOL_GPL(x86_pred_cmd);
 
  static u64 __ro_after_init x86_arch_cap_msr;
 
author maintainer
#17 lore raw

Re: [PATCH v3 3/6] x86/bugs: Fix RSB clearing in indirect_branch_prediction_barrier()

Quoted 8 lines · Nikolay Borisov
Quoted 4 lines · Josh Poimboeuf

IBPB is expected to clear the RSB. However, if X86_BUG_IBPB_NO_RET is set, that doesn't happen. Make indirect_branch_prediction_barrier() take that into account by calling __write_ibpb() which already does the right thing.

Quoted 4 lines

I find this changelog somewhat dubious. So zen < 4 basically have IBPB_NO_RET, your patch 2 in this series makes using SBPB for cores which have SRSO_NO or if the mitigation is disabled. So if you have a core which is zen <4 and doesn't use SBPB then what happens?

I'm afraid I don't understand the question. In that case write_ibpb() uses IBPB and manually clears the RSB.

Signature · 1 line
Josh
#18 lore raw

Re: [PATCH v3 3/6] x86/bugs: Fix RSB clearing in indirect_branch_prediction_barrier()

Quoted 9 lines · Josh Poimboeuf
Quoted 8 lines · Nikolay Borisov
Quoted 4 lines · Josh Poimboeuf

IBPB is expected to clear the RSB. However, if X86_BUG_IBPB_NO_RET is set, that doesn't happen. Make indirect_branch_prediction_barrier() take that into account by calling __write_ibpb() which already does the right thing.

Quoted 8 lines
Quoted 4 lines

I find this changelog somewhat dubious. So zen < 4 basically have IBPB_NO_RET, your patch 2 in this series makes using SBPB for cores which have SRSO_NO or if the mitigation is disabled. So if you have a core which is zen <4 and doesn't use SBPB then what happens?

I'm afraid I don't understand the question. In that case write_ibpb() uses IBPB and manually clears the RSB.

Actually isn't this patch a noop. The old code simply wrote the value of x86_pred_cmd to the IA32-PRED_CMD register iff FEATURE_IBPB was set. So x86_pred_cmd might contain either PRED_CMD_IBPB or PRED_CMD_SBPB, meaning the correct value was written.

With your change you now call __write_ibpb() which does effectively the same thing.

author maintainer
#19 lore raw

Re: [PATCH v3 3/6] x86/bugs: Fix RSB clearing in indirect_branch_prediction_barrier()

Quoted 19 lines · Nikolay Borisov
Quoted 9 lines · Josh Poimboeuf
Quoted 8 lines · Nikolay Borisov
Quoted 4 lines · Josh Poimboeuf

IBPB is expected to clear the RSB. However, if X86_BUG_IBPB_NO_RET is set, that doesn't happen. Make indirect_branch_prediction_barrier() take that into account by calling __write_ibpb() which already does the right thing.

Quoted 8 lines
Quoted 4 lines

I find this changelog somewhat dubious. So zen < 4 basically have IBPB_NO_RET, your patch 2 in this series makes using SBPB for cores which have SRSO_NO or if the mitigation is disabled. So if you have a core which is zen <4 and doesn't use SBPB then what happens?

I'm afraid I don't understand the question. In that case write_ibpb() uses IBPB and manually clears the RSB.

Quoted 7 lines

Actually isn't this patch a noop. The old code simply wrote the value of x86_pred_cmd to the IA32-PRED_CMD register iff FEATURE_IBPB was set. So x86_pred_cmd might contain either PRED_CMD_IBPB or PRED_CMD_SBPB, meaning the correct value was written.

With your change you now call __write_ibpb() which does effectively the same thing.

Hm, are you getting SBPB and IBPB_NO_RET mixed up? They're completely separate and distinct:

- SBPB is an AMD feature which is just like IBPB, except it doesn't flush branch type predictions. It can be used when the SRSO mitigation isn't needed. That was fixed by the previous patch.

- AMD has a bug on older CPUs where IBPB doesn't flush the RSB. Such CPUs have X86_BUG_IBPB_NO_RET set. That's fixed with this patch due to the fact that write_ibpb() has this:

/* Make sure IBPB clears return stack preductions too. */ FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS, X86_BUG_IBPB_NO_RET

So you're right in that this patch doesn't change SBPB behavior. But that's not what it intends to do :-)

Signature · 1 line
Josh
Reviewed-by
#20 lore raw

Re: [PATCH v3 3/6] x86/bugs: Fix RSB clearing in indirect_branch_prediction_barrier()

Quoted 39 lines · Josh Poimboeuf
Quoted 19 lines · Nikolay Borisov
Quoted 9 lines · Josh Poimboeuf
Quoted 8 lines · Nikolay Borisov
Quoted 4 lines · Josh Poimboeuf

IBPB is expected to clear the RSB. However, if X86_BUG_IBPB_NO_RET is set, that doesn't happen. Make indirect_branch_prediction_barrier() take that into account by calling __write_ibpb() which already does the right thing.

Quoted 8 lines
Quoted 4 lines

I find this changelog somewhat dubious. So zen < 4 basically have IBPB_NO_RET, your patch 2 in this series makes using SBPB for cores which have SRSO_NO or if the mitigation is disabled. So if you have a core which is zen <4 and doesn't use SBPB then what happens?

I'm afraid I don't understand the question. In that case write_ibpb() uses IBPB and manually clears the RSB.

Quoted 7 lines

Actually isn't this patch a noop. The old code simply wrote the value of x86_pred_cmd to the IA32-PRED_CMD register iff FEATURE_IBPB was set. So x86_pred_cmd might contain either PRED_CMD_IBPB or PRED_CMD_SBPB, meaning the correct value was written.

With your change you now call __write_ibpb() which does effectively the same thing.

Hm, are you getting SBPB and IBPB_NO_RET mixed up? They're completely separate and distinct:

- SBPB is an AMD feature which is just like IBPB, except it doesn't flush branch type predictions. It can be used when the SRSO mitigation isn't needed. That was fixed by the previous patch.

- AMD has a bug on older CPUs where IBPB doesn't flush the RSB. Such CPUs have X86_BUG_IBPB_NO_RET set. That's fixed with this patch due to the fact that write_ibpb() has this:

Quoted 5 lines

/* Make sure IBPB clears return stack preductions too. */ FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS, X86_BUG_IBPB_NO_RET

So you're right in that this patch doesn't change SBPB behavior. But that's not what it intends to do :-)

Fair enough, it just wasn't obvious from the diff context that there is the FILL_RETURN_BUFFER , my bad for not looking at __write_ibpb.

author maintainer
#21 lore raw

PATCH v3 4/6 x86/bugs: Don't fill RSB on VMEXIT with eIBRS+retpoline

In reply to Josh Poimboeuf

eIBRS protects against guest->host RSB underflow/poisoning attacks. Adding retpoline to the mix doesn't change that. Retpoline has a balanced CALL/RET anyway.

So the current full RSB filling on VMEXIT with eIBRS+retpoline is overkill. Disable it or do the VMEXIT_LITE mitigation if needed.

---
 arch/x86/kernel/cpu/bugs.c | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
arch/x86/kernel/cpu/bugs.c +4 -4
index 9f9637cff7a3..354411fd4800 100644
@@ -1617,20 +1617,20 @@ static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_
 	case SPECTRE_V2_NONE:
 		return;
 
-	case SPECTRE_V2_EIBRS_LFENCE:
 	case SPECTRE_V2_EIBRS:
+	case SPECTRE_V2_EIBRS_LFENCE:
+	case SPECTRE_V2_EIBRS_RETPOLINE:
 		if (boot_cpu_has_bug(X86_BUG_EIBRS_PBRSB)) {
-			setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT_LITE);
 			pr_info("Spectre v2 / PBRSB-eIBRS: Retire a single CALL on VMEXIT\n");
+			setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT_LITE);
 		}
 		return;
 
-	case SPECTRE_V2_EIBRS_RETPOLINE:
 	case SPECTRE_V2_RETPOLINE:
 	case SPECTRE_V2_LFENCE:
 	case SPECTRE_V2_IBRS:
-		setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT);
 		pr_info("Spectre v2 / SpectreRSB : Filling RSB on VMEXIT\n");
+		setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT);
 		return;
 	}
 
Signature · 1 line
2.48.1
Reviewed-by on v3 4/6
#22 lore raw

Re: [PATCH v3 4/6] x86/bugs: Don't fill RSB on VMEXIT with eIBRS+retpoline

Quoted 11 lines · Josh Poimboeuf

eIBRS protects against guest->host RSB underflow/poisoning attacks. Adding retpoline to the mix doesn't change that. Retpoline has a balanced CALL/RET anyway.

So the current full RSB filling on VMEXIT with eIBRS+retpoline is overkill. Disable it or do the VMEXIT_LITE mitigation if needed.

author maintainer
#23 lore raw

PATCH v3 5/6 x86/bugs: Don't fill RSB on context switch with eIBRS

In reply to Josh Poimboeuf

User->user Spectre v2 attacks (including RSB) across context switches are already mitigated by IBPB in cond_mitigation(), if enabled globally or if either the prev or the next task has opted in to protection. RSB filling without IBPB serves no purpose for protecting user space, as indirect branches are still vulnerable.

User->kernel RSB attacks are mitigated by eIBRS. In which case the RSB filling on context switch isn't needed, so remove it.

---
 arch/x86/kernel/cpu/bugs.c | 24 ++++++++++++------------
 arch/x86/mm/tlb.c          |  6 +++---
 2 files changed, 15 insertions(+), 15 deletions(-)
arch/x86/kernel/cpu/bugs.c +12 -12
index 354411fd4800..680c779e9711 100644
@@ -1591,7 +1591,7 @@ static void __init spec_ctrl_disable_kernel_rrsba(void)
 	rrsba_disabled = true;
 }
 
-static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_mitigation mode)
+static void __init spectre_v2_select_rsb_mitigation(enum spectre_v2_mitigation mode)
 {
 	/*
 	 * Similar to context switches, there are two types of RSB attacks
@@ -1615,7 +1615,7 @@ static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_
 	 */
 	switch (mode) {
 	case SPECTRE_V2_NONE:
-		return;
+		break;
 
 	case SPECTRE_V2_EIBRS:
 	case SPECTRE_V2_EIBRS_LFENCE:
@@ -1624,18 +1624,21 @@ static void __init spectre_v2_determine_rsb_fill_type_at_vmexit(enum spectre_v2_
 			pr_info("Spectre v2 / PBRSB-eIBRS: Retire a single CALL on VMEXIT\n");
 			setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT_LITE);
 		}
-		return;
+		break;
 
 	case SPECTRE_V2_RETPOLINE:
 	case SPECTRE_V2_LFENCE:
 	case SPECTRE_V2_IBRS:
-		pr_info("Spectre v2 / SpectreRSB : Filling RSB on VMEXIT\n");
+		pr_info("Spectre v2 / SpectreRSB: Filling RSB on context switch and VMEXIT\n");
+		setup_force_cpu_cap(X86_FEATURE_RSB_CTXSW);
 		setup_force_cpu_cap(X86_FEATURE_RSB_VMEXIT);
-		return;
-	}
+		break;
 
-	pr_warn_once("Unknown Spectre v2 mode, disabling RSB mitigation at VM exit");
-	dump_stack();
+	default:
+		pr_warn_once("Unknown Spectre v2 mode, disabling RSB mitigation\n");
+		dump_stack();
+		break;
+	}
 }
 
 /*
@@ -1867,10 +1870,7 @@ static void __init spectre_v2_select_mitigation(void)
 	 *
 	 * FIXME: Is this pointless for retbleed-affected AMD?
 	 */
-	setup_force_cpu_cap(X86_FEATURE_RSB_CTXSW);
-	pr_info("Spectre v2 / SpectreRSB mitigation: Filling RSB on context switch\n");
-
-	spectre_v2_determine_rsb_fill_type_at_vmexit(mode);
+	spectre_v2_select_rsb_mitigation(mode);
 
 	/*
 	 * Retpoline protects the kernel, but doesn't protect firmware.  IBRS
arch/x86/mm/tlb.c +3 -3
index e459d97ef397..eb83348f9305 100644
@@ -667,9 +667,9 @@ static void cond_mitigation(struct task_struct *next)
 	prev_mm = this_cpu_read(cpu_tlbstate.last_user_mm_spec);
 
 	/*
-	 * Avoid user/user BTB poisoning by flushing the branch predictor
-	 * when switching between processes. This stops one process from
-	 * doing Spectre-v2 attacks on another.
+	 * Avoid user->user BTB/RSB poisoning by flushing them when switching
+	 * between processes. This stops one process from doing Spectre-v2
+	 * attacks on another.
 	 *
 	 * Both, the conditional and the always IBPB mode use the mm
 	 * pointer to avoid the IBPB when switching between tasks of the
Signature · 1 line
2.48.1
Reviewed-by on v3 5/6
#24 lore raw

Re: [PATCH v3 5/6] x86/bugs: Don't fill RSB on context switch with eIBRS

Quoted 6 lines · Josh Poimboeuf

User->user Spectre v2 attacks (including RSB) across context switches are already mitigated by IBPB in cond_mitigation(), if enabled globally or if either the prev or the next task has opted in to protection. RSB filling without IBPB serves no purpose for protecting user space, as indirect branches are still vulnerable.

Quoted 7 lines

User->kernel RSB attacks are mitigated by eIBRS. In which case the RSB filling on context switch isn't needed, so remove it.

author maintainer
#25 lore raw

PATCH v3 6/6 x86/bugs: Add RSB mitigation document

In reply to Josh Poimboeuf

Create a document to summarize hard-earned knowledge about RSB-related mitigations, with references, and replace the overly verbose yet incomplete comments with a reference to the document.

---
 Documentation/admin-guide/hw-vuln/index.rst |   1 +
 Documentation/admin-guide/hw-vuln/rsb.rst   | 241 ++++++++++++++++++++
 arch/x86/kernel/cpu/bugs.c                  |  64 ++----
 3 files changed, 255 insertions(+), 51 deletions(-)
 create mode 100644 Documentation/admin-guide/hw-vuln/rsb.rst
Documentation/admin-guide/hw-vuln/index.rst +1 -0
index ff0b440ef2dc..451874b8135d 100644
@@ -22,3 +22,4 @@ are configurable at compile, boot or run time.
    srso
    gather_data_sampling
    reg-file-data-sampling
+   rsb
Documentation/admin-guide/hw-vuln/rsb.rst new file +241 -0

241 lines not on this page · raw message

arch/x86/kernel/cpu/bugs.c +13 -51

82 lines not on this page · raw message

Signature · 1 line
2.48.1
#26 lore raw

Re: [PATCH v3 6/6] x86/bugs: Add RSB mitigation document

Quoted 3 lines · Josh Poimboeuf

Create a document to summarize hard-earned knowledge about RSB-related mitigations, with references, and replace the overly verbose yet incomplete comments with a reference to the document.

Just a few nits:

Documentation/admin-guide/hw-vuln/rsb.rst line 36

Quoted 11 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +10 -0
+RSB poisoning (Intel and AMD)
+=============================
+
+SpectreRSB
+~~~~~~~~~~
 
+
+RSB poisoning is a technique used by Spectre-RSB [#spectre-rsb]_ where
+an attacker poisons an RSB entry to cause a victim's return instruction
+to speculate to an attacker-controlled address.  This can happen when
+there are unbalanced CALLs/RETs after a context switch or VMEXIT.

s/Spectre-RSB
/SpectreRSB

Which is the name the title just a few lines above uses, and which appears to be broadly the in-tree consensus spelling as well.

Documentation/admin-guide/hw-vuln/rsb.rst line 110

Quoted 3 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +3 -0
+
+AMD Retbleed / SRSO / Branch Type Confusion
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Nit: the underline is one character too long. :-)

Documentation/admin-guide/hw-vuln/rsb.rst line 117

Quoted 6 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +6 -0
+On AMD, poisoned RSB entries can also be created by the AMD Retbleed
+variant [#retbleed-paper]_ and/or Speculative Return Stack Overflow
+[#amd-srso]_ (Inception [#inception-paper]_).  These attacks are made
+possible by Branch Type Confusion [#amd-btc]_.  The kernel protects
+itself by replacing every RET in the kernel with a branch to a single
+safe RET.

s/Retbleed
/RETBleed

Seems to be the consensus spelling in-tree. (There's a few more cases in this document as well.)

arch/x86/kernel/cpu/bugs.c static void __init spec_ctrl_disable_kernel_rrsba(void) · line 1606

Quoted 10 lines
arch/x86/kernel/cpu/bugs.c +7 -0
+	 * WARNING! There are many subtleties to consider when changing *any*
+	 * code related to RSB-related mitigations.  Before doing so, carefully
+	 * read the following document, and update if necessary:
 	 *
+	 *   Documentation/admin-guide/hw-vuln/rsb.rst
 	 *
+	 * In an overly simplified nutshell:
 	 *
+	 *   - User->user RSB attacks are conditionally mitigated during
+	 *     context switch by cond_mitigation -> __write_ibpb().

s/during context switch
/during context switches

arch/x86/kernel/cpu/bugs.c static void __init spec_ctrl_disable_kernel_rrsba(void) · line 1614

Quoted 8 lines
arch/x86/kernel/cpu/bugs.c +5 -0
 	 *
+	 *   - User->kernel and guest->host attacks are mitigated by eIBRS or
+	 *     RSB filling.
 	 *
+	 *     Though, depending on config, note that other alternative
+	 *     mitigations may end up getting used instead, e.g., IBPB on
+	 *     entry/vmexit, call depth tracking, or return thunks.
 	 */

s/__write_ibpb()
/write_ibpb()

as per the discussion under patch #1.

Thanks,

Ingo

author maintainer
#27 lore raw

Re: [PATCH v3 6/6] x86/bugs: Add RSB mitigation document

Quoted 8 lines · Ingo Molnar
Quoted 3 lines · Josh Poimboeuf

Create a document to summarize hard-earned knowledge about RSB-related mitigations, with references, and replace the overly verbose yet incomplete comments with a reference to the document.

Just a few nits:

Ack, thanks!

Signature · 1 line
Josh
#28 lore raw

Re: [PATCH v3 6/6] x86/bugs: Add RSB mitigation document

In reply to Josh Poimboeuf

Review

Quoted 4 lines · Josh Poimboeuf
(quoted diff) +4 -0
+    Note that some Intel CPUs are susceptible to Post-barrier Return
+    Stack Buffer Predictions (PBRSB)[#intel-pbrsb]_, where the last CALL
+    from the guest can be used to predict the first unbalanced RET.  In
+    this case the PBRSB mitigation is needed in addition to eIBRS.

I get Sphinx unreferenced footnote warning:

Documentation/admin-guide/hw-vuln/rsb.rst:221: WARNING: Footnote [#] is not referenced. [ref.footnote]

To fix that, I have to separate the footnote:

---- >8 ----

Documentation/admin-guide/hw-vuln/rsb.rst +1 -1
index 97bf75993d5d43..dd727048b00204 100644
@@ -102,7 +102,7 @@ there are unbalanced CALLs/RETs after a context switch or VMEXIT.
 	at the time of the VM exit." [#intel-eibrs-vmexit]_
 
     Note that some Intel CPUs are susceptible to Post-barrier Return
-    Stack Buffer Predictions (PBRSB)[#intel-pbrsb]_, where the last CALL
+    Stack Buffer Predictions (PBRSB) [#intel-pbrsb]_, where the last CALL
     from the guest can be used to predict the first unbalanced RET.  In
     this case the PBRSB mitigation is needed in addition to eIBRS.
 

Thanks.

Signature · 1 line
An old man doll... just what I always wanted! - Clara
author maintainer
#29 lore raw

Re: [PATCH v3 6/6] x86/bugs: Add RSB mitigation document

Quoted 18 lines · Bagas Sanjaya

Review

Quoted 4 lines · Josh Poimboeuf
(quoted diff) +4 -0
+    Note that some Intel CPUs are susceptible to Post-barrier Return
+    Stack Buffer Predictions (PBRSB)[#intel-pbrsb]_, where the last CALL
+    from the guest can be used to predict the first unbalanced RET.  In
+    this case the PBRSB mitigation is needed in addition to eIBRS.

I get Sphinx unreferenced footnote warning:

Documentation/admin-guide/hw-vuln/rsb.rst:221: WARNING: Footnote [#] is not referenced. [ref.footnote]

To fix that, I have to separate the footnote:

---- >8 ----

Documentation/admin-guide/hw-vuln/rsb.rst +0 -0
index 97bf75993d5d43..dd727048b00204 100644

Documentation/admin-guide/hw-vuln/rsb.rst there are unbalanced CALLs/RETs after a context switch or VMEXIT. · line 107

Quoted 7 lines
Documentation/admin-guide/hw-vuln/rsb.rst +1 -1
@@ -102,7 +102,7 @@ there are unbalanced CALLs/RETs after a context switch or VMEXIT.
 	at the time of the VM exit." [#intel-eibrs-vmexit]_
 
     Note that some Intel CPUs are susceptible to Post-barrier Return
-    Stack Buffer Predictions (PBRSB)[#intel-pbrsb]_, where the last CALL
+    Stack Buffer Predictions (PBRSB) [#intel-pbrsb]_, where the last CALL
     from the guest can be used to predict the first unbalanced RET.  In
     this case the PBRSB mitigation is needed in addition to eIBRS.

Fixed, thanks!

Signature · 1 line
Josh
#30 lore raw

Re: [PATCH v3 6/6] x86/bugs: Add RSB mitigation document

In reply to Josh Poimboeuf
Quoted 5 lines · Josh Poimboeuf

Create a document to summarize hard-earned knowledge about RSB-related mitigations, with references, and replace the overly verbose yet incomplete comments with a reference to the document.

Excellent.  I've been trying to do something like this for a while, and not managing to get this far.

Quoted 9 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +3 -0
new file mode 100644
index 000000000000..97bf75993d5d
@@ -0,0 +1,241 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=======================

Documentation/admin-guide/hw-vuln/rsb.rst line 13

Quoted 10 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +10 -0
+RSB-related mitigations
+=======================
+
+.. warning::
+   Please keep this document up-to-date, otherwise you will be
+   volunteered to update it and convert it to a very long comment in
+   bugs.c!
+
+Since 2018 there have been many Spectre CVEs related to the Return Stack
+Buffer (RSB).

2017.  I, and many others, spent the whole second half of 2017 tied up in all of this.

I'd drop the Spectre, or swap it for Speculation-related.  Simply "Spectre" CVEs tend to focus on the conditional and indirect predictors.

Quoted 2 lines

Information about these CVEs and how to mitigate them is
+scattered amongst a myriad of microarchitecture-specific documents.

You should note that the AMD terms RAS and RAP are the same thing, considering that you link to the documents.

Documentation/admin-guide/hw-vuln/rsb.rst line 24

Quoted 10 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +10 -0
+
+This document attempts to consolidate all the relevant information in
+once place and clarify the reasoning behind the current RSB-related
+mitigations.
+
+At a high level, there are two classes of RSB attacks: RSB poisoning
+(Intel and AMD) and RSB underflow (Intel only).  They must each be
+considered individually for each attack vector (and microarchitecture
+where applicable).
+

One question before getting further.  This seems to be focused on (mis)prediction of RETs ?

That's fine, but it wants spelling out, because it is distinct from the other class of issues when RET instructions execute in bad ways.

When lecturing, my go-to example is Spectre-v1.1 / BCBS (store which clobbers or aliases the return address), because an astounding number of things can go wrong in different ways from there.

Next, before diving into the specifics, it's incredibly relevant to have a section briefly describing how the RSB typically works.  It's key to understanding the rest of the documents, and there will definitely be people reading the document who don't know it.

The salient points are (on CPUs since ~Nehalem era.  Confirm with Intel and AMD.  You can spot it in the optimisation guides, because it's where the phrase such as "only taken branches consume prediction resource" began appearing):

* Branch prediction is **prior** to instruction decode, and guesses at the location, type, and target of all near branches.
* The RSB is a prediction structure used by branches predicted as CALLs or RETs.
* When a CALL is predicted, the predicted return-address is pushed on the RSB
* When a RET is predicted, the RSB is popped
* Later, decode will cross-check the prediction with the instruction stream.  It can issue corrections to the predictor state, and restart prediction/fetch from the point that things appeared to go wrong.  This can include editing transient state in the RSB.

For the observant reader, yes, the RSB is filled using predicted targets.  This is why the SRSO vuln is so evil.

So, with the behaviour summarised, next some properties (disclaimer:
varies by vendor)
* It is logically a stack, but has finite capacity.  Executing more RET instructions than CALLs will underflow it.
** AMD reuses the -1'th entry and doesn't move the pointer ** Intel may fall back to a prediction from a different predictor
* It is a structure shared across all security domains in Core/Thread. Guest & Host is most relevant to the doc, but SMM/ACM/SEAM/XuCode are all included.
** Some AMD CPUs dynamically re-partition the RSB(RAS) when a sibling thread goes idle
** Some Intel CPUs only have a 32-bit wide RSB, and reuse the upper bits of the location for the predicted target
** Some Intel CPUs hardwire bit 47(?) which causes the kernel to follow userspace predictions.

Quoted 8 lines

+----

Documentation/admin-guide/hw-vuln/rsb.rst new file +7 -0
+
+RSB poisoning (Intel and AMD)
+=============================
+
+SpectreRSB
+~~~~~~~~~~
+

Documentation/admin-guide/hw-vuln/rsb.rst line 42

Quoted 10 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +10 -0
+RSB poisoning is a technique used by Spectre-RSB [#spectre-rsb]_ where
+an attacker poisons an RSB entry to cause a victim's return instruction
+to speculate to an attacker-controlled address.  This can happen when
+there are unbalanced CALLs/RETs after a context switch or VMEXIT.
+
+* All attack vectors can potentially be mitigated by flushing out any
+  poisoned RSB entries using an RSB filling sequence
+  [#intel-rsb-filling]_ [#amd-rsb-filling]_ when transitioning between
+  untrusted and trusted domains.  But this has a performance impact and
+  should be avoided whenever possible.

More importantly, 32-entry RSB stuffing loops are only applicable to pre-eIBRS and pre-ERAPS hardware.

They are known unsafe to use on newer microarchitectures, inc Gracemont (128 entries) and Zen5 (64 entries).

Documentation/admin-guide/hw-vuln/rsb.rst line 52

Quoted 10 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +10 -0
+
+* On context switch, the user->user mitigation requires ensuring the
+  RSB gets filled or cleared whenever IBPB gets written [#cond-ibpb]_
+  during a context switch:
+
+  * AMD:
+	IBPB (or SBPB [#amd-sbpb]_ if used) automatically clears the RSB
+	if IBPB_RET is set in CPUID [#amd-ibpb-rsb]_.  Otherwise the RSB
+	filling sequence [#amd-rsb-filling]_ must be always be done in
+	addition to IBPB.

Honestly, I dislike this way of characterising it.   IBPB was very clearly spec'd to flush all indirect prediction structures, and some AMD CPUs have an errata where this isn't true and has to be filled in by the OS.

Documentation/admin-guide/hw-vuln/rsb.rst line 64

Quoted 12 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +12 -0
+
+  * Intel:
+	IBPB automatically clears the RSB:
+
+	"Software that executed before the IBPB command cannot control
+	the predicted targets of indirect branches executed after the
+	command on the same logical processor. The term indirect branch
+	in this context includes near return instructions, so these
+	predicted targets may come from the RSB." [#intel-ibpb-rsb]_
+
+* On context switch, user->kernel attacks are mitigated by SMEP, as user
+  space can only insert its own return addresses into the RSB:

It's more subtle than this (see the 32-bit wide prediction).

A user/supervisor split address space limits the ranges of addresses that userspace can insert into the predictor.

There is a corner case at the canonical boundary.  Userspace can insert the first non-canonincal address, and on some CPUs, this is interpreted as the first high canonical address.  Guard pages on both sides of the canonical boundary mitigate this.

In the unbalanced case for user->kernel, a bad prediction really is made (coming out of the RSB), and it's only the SMEP #PF at instruction fetch which prevents you speculatively executing in userspace.

In the 32-bit width case, the kernel predicts to {high_kern:low_user} target.

Quoted 43 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +43 -0
+
+  * AMD:
+	"Finally, branches that are predicted as 'ret' instructions get
+	their predicted targets from the Return Address Predictor (RAP).
+	AMD recommends software use a RAP stuffing sequence (mitigation
+	V2-3 in [2]) and/or Supervisor Mode Execution Protection (SMEP)
+	to ensure that the addresses in the RAP are safe for
+	speculation. Collectively, we refer to these mitigations as "RAP
+	Protection"." [#amd-smep-rsb]_
+
+  * Intel:
+	"On processors with enhanced IBRS, an RSB overwrite sequence may
+	not suffice to prevent the predicted target of a near return
+	from using an RSB entry created in a less privileged predictor
+	mode.  Software can prevent this by enabling SMEP (for
+	transitions from user mode to supervisor mode) and by having
+	IA32_SPEC_CTRL.IBRS set during VM exits." [#intel-smep-rsb]_
+
+* On VMEXIT, guest->host attacks are mitigated by eIBRS (and PBRSB
+  mitigation if needed):
+
+  * AMD:
+	"When Automatic IBRS is enabled, the internal return address
+	stack used for return address predictions is cleared on VMEXIT."
+	[#amd-eibrs-vmexit]_
+
+  * Intel:
+	"On processors with enhanced IBRS, an RSB overwrite sequence may
+	not suffice to prevent the predicted target of a near return
+	from using an RSB entry created in a less privileged predictor
+	mode.  Software can prevent this by enabling SMEP (for
+	transitions from user mode to supervisor mode) and by having
+	IA32_SPEC_CTRL.IBRS set during VM exits. Processors with
+	enhanced IBRS still support the usage model where IBRS is set
+	only in the OS/VMM for OSes that enable SMEP. To do this, such
+	processors will ensure that guest behavior cannot control the
+	RSB after a VM exit once IBRS is set, even if IBRS was not set
+	at the time of the VM exit." [#intel-eibrs-vmexit]_
+
+    Note that some Intel CPUs are susceptible to Post-barrier Return
+    Stack Buffer Predictions (PBRSB)[#intel-pbrsb]_, where the last CALL
+    from the guest can be used to predict the first unbalanced RET.  In
+    this case the PBRSB mitigation is needed in addition to eIBRS.

Documentation/admin-guide/hw-vuln/rsb.rst line 117

Quoted 10 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +10 -0
+
+AMD Retbleed / SRSO / Branch Type Confusion
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
+
+On AMD, poisoned RSB entries can also be created by the AMD Retbleed
+variant [#retbleed-paper]_ and/or Speculative Return Stack Overflow
+[#amd-srso]_ (Inception [#inception-paper]_).  These attacks are made
+possible by Branch Type Confusion [#amd-btc]_.  The kernel protects
+itself by replacing every RET in the kernel with a branch to a single
+safe RET.

BTC and SRSO are unrelated things.

"predicted branch types" is an inherent property of the predictor.  BTC is specifically the case where decode doesn't halt, and still issues the wrong uops into the pipeline to execute.

SRSO goes entirely wrong at the BP/IF stages (BP racing ahead while IF is stalled unable to fetch anything), so the damage is done by the time Decode sees the instruction stream.

You also need to cover the AMD going-idle issue where the other thread's RSB entries magically appear in my RSB, and my head/tail pointer is reset to a random position.

Documentation/admin-guide/hw-vuln/rsb.rst line 122

Quoted 5 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +5 -0
+
+----
+
+RSB underflow (Intel only)
+==========================

Well, not really.  AMD can underflow too.  It just picks a fixed entry and keeps on reusing that.  (Great for the alarming number of programming languages which consider recursion a virtue.)

Quoted 19 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +19 -0
+
+Intel Retbleed
+~~~~~~~~~~~~~~
+
+Some Intel Skylake-generation CPUs are susceptible to the Intel variant
+of Retbleed [#retbleed-paper]_ (Return Stack Buffer Underflow
+[#intel-rsbu]_).  If a RET is executed when the RSB buffer is empty due
+to mismatched CALLs/RETs or returning from a deep call stack, the branch
+predictor can fall back to using the Branch Target Buffer (BTB).  If a
+user forces a BTB collision then the RET can speculatively branch to a
+user-controlled address.
+
+* Note that RSB filling doesn't fully mitigate this issue.  If there
+  are enough unbalanced RETs, the RSB may still underflow and fall back
+  to using a poisoned BTB entry.
+
+* On context switch, user->user underflow attacks are mitigated by the
+  conditional IBPB [#cond-ibpb]_ on context switch which clears the BTB:
+

Documentation/admin-guide/hw-vuln/rsb.rst line 151

Quoted 10 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +10 -0
+  * "The indirect branch predictor barrier (IBPB) is an indirect branch
+    control mechanism that establishes a barrier, preventing software
+    that executed before the barrier from controlling the predicted
+    targets of indirect branches executed after the barrier on the same
+    logical processor." [#intel-ibpb-btb]_
+
+    .. note::
+       I wasn't able to find any offical documentation from Intel
+       explicitly stating that IBPB clears the BTB.  However, it's
+       broadly known to be true and relied upon in several mitigations.

Part of this is because when the vendors say the BTB, they're translating their internal names into what academia calls them.

"Flush the BTB" isn't a helpful statement anyway.  See AMD's IBPB vs SBPB which controls whether the branch types predictions remain intact.

Given how many rounds of Intel microcode there have been making IBPB scrub more, it clearly wasn't scrubbing everything in the first place.

Documentation/admin-guide/hw-vuln/rsb.rst

Quoted 6 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +6 -0
+
+* On context switch and VMEXIT, user->kernel and guest->host underflows
+  are mitigated by IBRS or eIBRS:
+
+  * "Enabling IBRS (including enhanced IBRS) will mitigate the "RSBU"
+    attack demonstrated by the researchers.

Yeah, except it doesn't.  Intra-mode BTI is a thing, and that will leak your secrets too.

Documentation/admin-guide/hw-vuln/rsb.rst line 162

Quoted 6 lines

As previously documented,

Documentation/admin-guide/hw-vuln/rsb.rst new file +5 -0
+    Intel recommends the use of enhanced IBRS, where supported. This
+    includes any processor that enumerates RRSBA but not RRSBA_DIS_S."
+    [#intel-rsbu]_
+
+  As an alternative to classic IBRS, call depth tracking can be used to

legacy IBRS.  Please don't invent yet another term for it :), and "classic" further implies there might be a time when anyone looks back fondly on it.

Documentation/admin-guide/hw-vuln/rsb.rst line 167

Quoted 5 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +5 -0
+  track kernel returns and fill the RSB when it gets close to being
+  empty.
+
+Restricted RSB Alternate (RRSBA)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This needs to discuss RSBA and RRSBA.  They're distinct.  To a first approximation, RRSBA == RSBA + eIBRS.  The "restricted" nature is "mode-tagged predictions".

Except there's a narrow range of CPUs around Icelake which have eIBRS and do not suffer (R)RSBA.

Quoted 15 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +15 -0
+
+Some newer Intel CPUs have Restricted RSB Alternate (RRSBA) behavior,
+which, similar to the Intel variant of Retbleed described above, also
+falls back to using the BTB on RSB underflow.  The only difference is
+that the predicted targets are restricted to the current domain.
+
+* "Restricted RSB Alternate (RRSBA) behavior allows alternate branch
+  predictors to be used by near RET instructions when the RSB is
+  empty.  When eIBRS is enabled, the predicted targets of these
+  alternate predictors are restricted to those belonging to the
+  indirect branch predictor entries of the current prediction domain.
+  [#intel-eibrs-rrsba]_
+
+When a CPU with RRSBA is vulnerable to Branch History Injection
+[#bhi-paper]_ [#intel-bhi]_, an RSB underflow could be used for an

Documentation/admin-guide/hw-vuln/rsb.rst line 192

Quoted 10 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +10 -0
+intra-mode BTI attack.  This is mitigated by clearing the BHB on
+kernel entry.
+
+However if the kernel uses retpolines instead of eIBRS, it needs to
+disable RRSBA:
+
+* "Where software is using retpoline as a mitigation for BHI or
+  intra-mode BTI, and the processor both enumerates RRSBA and
+  enumerates RRSBA_DIS controls, it should disable this behavior. "
+  [#intel-retpoline-rrsba]_

IIRC, not all CPUs which suffer RRSBA have the RRSBA_DIS_* controls.

But, I think that's enough nitpicking for now.  I really think some Intel and AMD architects ought to weigh in too.

~Andrew

author maintainer
#31 lore raw

Re: [PATCH v3 6/6] x86/bugs: Add RSB mitigation document

Quoted 6 lines · Andrew Cooper

Review

Quoted 2 lines · Josh Poimboeuf
(quoted diff) +2 -0
+Since 2018 there have been many Spectre CVEs related to the Return Stack
+Buffer (RSB).

2017.  I, and many others, spent the whole second half of 2017 tied up in all of this.

Right. Personally I got pulled in in October 2017. Even Skylake RSB underflow was already well understood, which led to the whole distro IBRS vs upstream retpoline divergence debacle.

But with respect to the above sentence, the first Spectre CVEs were disclosed in 2018.

Quoted 2 lines

I'd drop the Spectre, or swap it for Speculation-related.  Simply "Spectre" CVEs tend to focus on the conditional and indirect predictors.

As you are well aware, a RET is basically an indirect branch. SpectreRSB literally has Spectre in the name. And rightfully so IMO, it's just another way to exploit CPU branch prediction. And the same CPU data structures are involved.

So from my perspective, Spectre refers to the entire class of branch prediction exploits.

But I can change it if you think that's incorrect.

Quoted 5 lines
Quoted 2 lines

Information about these CVEs and how to mitigate them is
+scattered amongst a myriad of microarchitecture-specific documents.

You should note that the AMD terms RAS and RAP are the same thing, considering that you link to the documents.

Ack. What does RAS stand for?

Quoted 9 lines

One question before getting further.  This seems to be focused on (mis)prediction of RETs ?

That's fine, but it wants spelling out, because it is distinct from the other class of issues when RET instructions execute in bad ways.

When lecturing, my go-to example is Spectre-v1.1 / BCBS (store which clobbers or aliases the return address), because an astounding number of things can go wrong in different ways from there.

The document is strictly related to the RSB mitigations.

Quoted 25 lines

Next, before diving into the specifics, it's incredibly relevant to have a section briefly describing how the RSB typically works.  It's key to understanding the rest of the documents, and there will definitely be people reading the document who don't know it.

The salient points are (on CPUs since ~Nehalem era.  Confirm with Intel and AMD.  You can spot it in the optimisation guides, because it's where the phrase such as "only taken branches consume prediction resource" began appearing):

* Branch prediction is **prior** to instruction decode, and guesses at the location, type, and target of all near branches.
* The RSB is a prediction structure used by branches predicted as CALLs or RETs.
* When a CALL is predicted, the predicted return-address is pushed on the RSB
* When a RET is predicted, the RSB is popped
* Later, decode will cross-check the prediction with the instruction stream.  It can issue corrections to the predictor state, and restart prediction/fetch from the point that things appeared to go wrong.  This can include editing transient state in the RSB.

For the observant reader, yes, the RSB is filled using predicted targets.  This is why the SRSO vuln is so evil.

Quoted 15 lines

So, with the behaviour summarised, next some properties (disclaimer:
varies by vendor)
* It is logically a stack, but has finite capacity.  Executing more RET instructions than CALLs will underflow it.
** AMD reuses the -1'th entry and doesn't move the pointer ** Intel may fall back to a prediction from a different predictor
* It is a structure shared across all security domains in Core/Thread. Guest & Host is most relevant to the doc, but SMM/ACM/SEAM/XuCode are all included.
** Some AMD CPUs dynamically re-partition the RSB(RAS) when a sibling thread goes idle
** Some Intel CPUs only have a 32-bit wide RSB, and reuse the upper bits of the location for the predicted target
** Some Intel CPUs hardwire bit 47(?) which causes the kernel to follow userspace predictions.

That's some really good information. However, I'm not sure gritty details about how RSBs work are within the scope for this document.

My goal is for it to be as concise as possible, focused only on the current kernel mitigations: what are the RSB-related attack vectors for each uarch and how are we currently mitigating those?

It's basically a glorified comment, but too long to actually be one. So that when the next CVE comes along, one can quickly refer to this as a refresher to see what we're already doing and why.

If the reader wants to know details about the RSB itself, or the related exploits, there are plenty of references for them to dive into.

Quoted 11 lines

Documentation/admin-guide/hw-vuln/rsb.rst line 42

Quoted 5 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +5 -0
+* All attack vectors can potentially be mitigated by flushing out any
+  poisoned RSB entries using an RSB filling sequence
+  [#intel-rsb-filling]_ [#amd-rsb-filling]_ when transitioning between
+  untrusted and trusted domains.  But this has a performance impact and
+  should be avoided whenever possible.

More importantly, 32-entry RSB stuffing loops are only applicable to pre-eIBRS and pre-ERAPS hardware.

They are known unsafe to use on newer microarchitectures, inc Gracemont (128 entries) and Zen5 (64 entries).

Right now we're doing 32 entries regardless of CPU model. Do we need to increase the loop count for newer HW? Have any references handy?

Quoted 9 lines

Documentation/admin-guide/hw-vuln/rsb.rst line 52

Quoted 5 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +5 -0
+  * AMD:
+	IBPB (or SBPB [#amd-sbpb]_ if used) automatically clears the RSB
+	if IBPB_RET is set in CPUID [#amd-ibpb-rsb]_.  Otherwise the RSB
+	filling sequence [#amd-rsb-filling]_ must be always be done in
+	addition to IBPB.

Honestly, I dislike this way of characterising it.   IBPB was very clearly spec'd to flush all indirect prediction structures, and some AMD CPUs have an errata where this isn't true and has to be filled in by the OS.

I don't see how that statement characterizes anything, it just describes the relevant facts as concisely as possible.

Quoted 5 lines

Documentation/admin-guide/hw-vuln/rsb.rst line 64

Quoted 2 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +2 -0
+* On context switch, user->kernel attacks are mitigated by SMEP, as user
+  space can only insert its own return addresses into the RSB:

It's more subtle than this (see the 32-bit wide prediction).

Quoted 7 lines

A user/supervisor split address space limits the ranges of addresses that userspace can insert into the predictor.

There is a corner case at the canonical boundary.  Userspace can insert the first non-canonincal address, and on some CPUs, this is interpreted as the first high canonical address.  Guard pages on both sides of the canonical boundary mitigate this.

Ack. Have a reference handy?

Quoted 3 lines

In the unbalanced case for user->kernel, a bad prediction really is made (coming out of the RSB), and it's only the SMEP #PF at instruction fetch which prevents you speculatively executing in userspace.

Right, I should probably clarify that a bit more.

Quoted 2 lines

In the 32-bit width case, the kernel predicts to {high_kern:low_user} target.

Ok. But blocked by SMEP regardless.

Quoted 9 lines

Documentation/admin-guide/hw-vuln/rsb.rst line 117

Quoted 6 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +6 -0
+On AMD, poisoned RSB entries can also be created by the AMD Retbleed
+variant [#retbleed-paper]_ and/or Speculative Return Stack Overflow
+[#amd-srso]_ (Inception [#inception-paper]_).  These attacks are made
+possible by Branch Type Confusion [#amd-btc]_.  The kernel protects
+itself by replacing every RET in the kernel with a branch to a single
+safe RET.

BTC and SRSO are unrelated things.

Quoted 7 lines

"predicted branch types" is an inherent property of the predictor.  BTC is specifically the case where decode doesn't halt, and still issues the wrong uops into the pipeline to execute.

SRSO goes entirely wrong at the BP/IF stages (BP racing ahead while IF is stalled unable to fetch anything), so the damage is done by the time Decode sees the instruction stream.

As a simple software engineer, I was under the impression that SRSO is a special case of BTC. Apparently I was wrong :-)

Quoted 3 lines

You also need to cover the AMD going-idle issue where the other thread's RSB entries magically appear in my RSB, and my head/tail pointer is reset to a random position.

I'm not familiar with that one, do you have a reference?

Quoted 6 lines
Quoted 2 lines

+RSB underflow (Intel only)
+==========================

Well, not really.  AMD can underflow too.  It just picks a fixed entry and keeps on reusing that.  (Great for the alarming number of programming languages which consider recursion a virtue.)

I'm not sure that's relevant here since there are no mitigations needed for that?

Quoted 17 lines

Documentation/admin-guide/hw-vuln/rsb.rst line 151

Quoted 13 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +13 -0
+* On context switch, user->user underflow attacks are mitigated by the
+  conditional IBPB [#cond-ibpb]_ on context switch which clears the BTB:
+
+  * "The indirect branch predictor barrier (IBPB) is an indirect branch
+    control mechanism that establishes a barrier, preventing software
+    that executed before the barrier from controlling the predicted
+    targets of indirect branches executed after the barrier on the same
+    logical processor." [#intel-ibpb-btb]_
+
+    .. note::
+       I wasn't able to find any offical documentation from Intel
+       explicitly stating that IBPB clears the BTB.  However, it's
+       broadly known to be true and relied upon in several mitigations.

Part of this is because when the vendors say the BTB, they're translating their internal names into what academia calls them.

Quoted 5 lines

"Flush the BTB" isn't a helpful statement anyway.  See AMD's IBPB vs SBPB which controls whether the branch types predictions remain intact.

Given how many rounds of Intel microcode there have been making IBPB scrub more, it clearly wasn't scrubbing everything in the first place.

Any suggestions for how I can improve the wording here? I want to explain why IBPB protects against RSB underflow and how we know that.

I should probably also mention IBPB_RET here as well.

Quoted 8 lines

Documentation/admin-guide/hw-vuln/rsb.rst

Quoted 5 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +5 -0
+* On context switch and VMEXIT, user->kernel and guest->host underflows
+  are mitigated by IBRS or eIBRS:
+
+  * "Enabling IBRS (including enhanced IBRS) will mitigate the "RSBU"
+    attack demonstrated by the researchers.

Yeah, except it doesn't.  Intra-mode BTI is a thing, and that will leak your secrets too.

Ok, I'll add a mention of that.

Quoted 10 lines

Documentation/admin-guide/hw-vuln/rsb.rst line 162

Quoted 6 lines

As previously documented,

Documentation/admin-guide/hw-vuln/rsb.rst new file +5 -0
+    Intel recommends the use of enhanced IBRS, where supported. This
+    includes any processor that enumerates RRSBA but not RRSBA_DIS_S."
+    [#intel-rsbu]_
+
+  As an alternative to classic IBRS, call depth tracking can be used to

legacy IBRS.  Please don't invent yet another term for it :), and "classic" further implies there might be a time when anyone looks back fondly on it.

:-)

Quoted 6 lines
Quoted 2 lines

+Restricted RSB Alternate (RRSBA)
+~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

This needs to discuss RSBA and RRSBA.  They're distinct.  To a first approximation, RRSBA == RSBA + eIBRS.  The "restricted" nature is "mode-tagged predictions".

RSBA is the original Intel Retbleed, right? That was discussed extensively in the preceding section. I can mention that Intel Retbleed may also be referred to as RSBA.

Quoted 9 lines

Documentation/admin-guide/hw-vuln/rsb.rst line 192

Quoted 7 lines
Documentation/admin-guide/hw-vuln/rsb.rst new file +7 -0
+However if the kernel uses retpolines instead of eIBRS, it needs to
+disable RRSBA:
+
+* "Where software is using retpoline as a mitigation for BHI or
+  intra-mode BTI, and the processor both enumerates RRSBA and
+  enumerates RRSBA_DIS controls, it should disable this behavior. "
+  [#intel-retpoline-rrsba]_

IIRC, not all CPUs which suffer RRSBA have the RRSBA_DIS_* controls.

Hm, have a reference for that?

Signature · 1 line
Josh