Skip to contentSkip to conversation

KVM: SVM: let alternatives handle the cases when RSB filling is required

PATCH Applied Amit Shah 3 participants 3 messages Aug 7, 2024 – Sep 10, 2024

Subsystems (3)
lore
Share & tools
lore

Amit Shah

author
#0 lore raw

PATCH v4 1/1 KVM: SVM: let alternatives handle the cases when RSB filling is required

From: Amit Shah <[email protected]>

Remove superfluous RSB filling after a VMEXIT when the CPU already has flushed the RSB after a VMEXIT when AutoIBRS is enabled.

The initial implementation for adding RETPOLINES added an ALTERNATIVES implementation for filling the RSB after a VMEXIT in

commit 117cc7a908c836 ("x86/retpoline: Fill return stack buffer on vmexit")

Later, X86_FEATURE_RSB_VMEXIT was added in

commit 2b129932201673 ("x86/speculation: Add RSB VM Exit protections")

The AutoIBRS (on AMD CPUs) feature implementation added in

commit e7862eda309ecf ("x86/cpu: Support AMD Automatic IBRS")

used the already-implemented logic for EIBRS in spectre_v2_determine_rsb_fill_type_on_vmexit() -- but did not update the code at VMEXIT to act on the mode selected in that function -- resulting in VMEXITs continuing to clear the RSB when RETPOLINES are enabled, despite the presence of AutoIBRS.

---
v4: resend of v3 with subject-prefix fixed

v3:
- Add a comment mentioning SVM does not need RSB_VMEXIT_LITE unlike VMX.
v2:
- tweak commit message re: Boris's comments.

 arch/x86/kvm/svm/vmenter.S | 24 ++++++++++++++++--------
 1 file changed, 16 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/vmenter.S +16 -8
index a0c8eb37d3e1..69d9825ebdd9 100644
@@ -209,10 +209,14 @@ SYM_FUNC_START(__svm_vcpu_run)
 7:	vmload %_ASM_AX
 8:
 
-#ifdef CONFIG_MITIGATION_RETPOLINE
-	/* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */
-	FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE
-#endif
+	/*
+	 * IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET!
+	 *
+	 * Unlike VMX, AMD does not have the hardware bug that necessitates
+	 * RSB_VMEXIT_LITE
+	 */
+
+	FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_VMEXIT
 
 	/* Clobbers RAX, RCX, RDX.  */
 	RESTORE_HOST_SPEC_CTRL
@@ -348,10 +352,14 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run)
 
 2:	cli
 
-#ifdef CONFIG_MITIGATION_RETPOLINE
-	/* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */
-	FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE
-#endif
+	/*
+	 * IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET!
+	 *
+	 * Unlike VMX, AMD does not have the hardware bug that necessitates
+	 * RSB_VMEXIT_LITE
+	 */
+
+	FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_VMEXIT
 
 	/* Clobbers RAX, RCX, RDX, consumes RDI (@svm) and RSI (@spec_ctrl_intercepted). */
 	RESTORE_HOST_SPEC_CTRL
Signature · 1 line
2.45.2
#1 lore raw

Pinging to bring attention to this fix.

Quoted 97 lines · Amit Shah

From: Amit Shah <[email protected]>

Remove superfluous RSB filling after a VMEXIT when the CPU already has
flushed the RSB after a VMEXIT when AutoIBRS is enabled.

The initial implementation for adding RETPOLINES added an ALTERNATIVES
implementation for filling the RSB after a VMEXIT in

commit 117cc7a908c836 ("x86/retpoline: Fill return stack buffer on vmexit")

Later, X86_FEATURE_RSB_VMEXIT was added in

commit 2b129932201673 ("x86/speculation: Add RSB VM Exit protections")

The AutoIBRS (on AMD CPUs) feature implementation added in

commit e7862eda309ecf ("x86/cpu: Support AMD Automatic IBRS")

used the already-implemented logic for EIBRS in spectre_v2_determine_rsb_fill_type_on_vmexit() -- but did not update the
code at VMEXIT to act on the mode selected in that function -- resulting
in VMEXITs continuing to clear the RSB when RETPOLINES are enabled, despite the presence of AutoIBRS.

---
v4: resend of v3 with subject-prefix fixed

v3:
 - Add a comment mentioning SVM does not need RSB_VMEXIT_LITE unlike VMX.
v2:
 - tweak commit message re: Boris's comments.

 arch/x86/kvm/svm/vmenter.S | 24 ++++++++++++++++--------

 1 file changed, 16 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/vmenter.S +0 -0
index a0c8eb37d3e1..69d9825ebdd9 100644
@@ -209,10 +209,14 @@ SYM_FUNC_START(__svm_vcpu_run)

 7: vmload %_ASM_AX
 8:

arch/x86/kvm/svm/vmenter.S +0 -2
-#ifdef CONFIG_MITIGATION_RETPOLINE
-	/* IMPORTANT: Stuff the RSB immediately after VM-Exit,

before RET! */

arch/x86/kvm/svm/vmenter.S +0 -1
-	FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS,

X86_FEATURE_RETPOLINE

arch/x86/kvm/svm/vmenter.S +2 -1
-#endif
+	/*
+	 * IMPORTANT: Stuff the RSB immediately after VM-Exit,

before RET!

arch/x86/kvm/svm/vmenter.S +2 -0
+	 *
+	 * Unlike VMX, AMD does not have the hardware bug that

necessitates

arch/x86/kvm/svm/vmenter.S +4 -0
+	 * RSB_VMEXIT_LITE
+	 */
+
+	FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS,

X86_FEATURE_RSB_VMEXIT

  /* Clobbers RAX, RCX, RDX.  */
  RESTORE_HOST_SPEC_CTRL

arch/x86/kvm/svm/vmenter.S +0 -0
@@ -348,10 +352,14 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run)


 2: cli

arch/x86/kvm/svm/vmenter.S +0 -2
-#ifdef CONFIG_MITIGATION_RETPOLINE
-	/* IMPORTANT: Stuff the RSB immediately after VM-Exit,

before RET! */

arch/x86/kvm/svm/vmenter.S +0 -1
-	FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS,

X86_FEATURE_RETPOLINE

arch/x86/kvm/svm/vmenter.S +2 -1
-#endif
+	/*
+	 * IMPORTANT: Stuff the RSB immediately after VM-Exit,

before RET!

arch/x86/kvm/svm/vmenter.S +2 -0
+	 *
+	 * Unlike VMX, AMD does not have the hardware bug that

necessitates

arch/x86/kvm/svm/vmenter.S +4 -0
+	 * RSB_VMEXIT_LITE
+	 */
+
+	FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS,

X86_FEATURE_RSB_VMEXIT

  /* Clobbers RAX, RCX, RDX, consumes RDI (@svm) and RSI (@spec_ctrl_intercepted). */
  RESTORE_HOST_SPEC_CTRL

maintainer Applied
#2 lore raw
In reply to Amit Shah
Quoted 9 lines · Amit Shah

From: Amit Shah <[email protected]>

Remove superfluous RSB filling after a VMEXIT when the CPU already has flushed the RSB after a VMEXIT when AutoIBRS is enabled.

The initial implementation for adding RETPOLINES added an ALTERNATIVES implementation for filling the RSB after a VMEXIT in

commit 117cc7a908c836 ("x86/retpoline: Fill return stack buffer on vmexit")

Nit, no need for 14 digits, 12 is still the "official" recommendation. To make this flow better, I would also prefer to not have each commit reference be on its own line.

Quoted 3 lines

Later, X86_FEATURE_RSB_VMEXIT was added in

commit 2b129932201673 ("x86/speculation: Add RSB VM Exit protections")

Oh, the irony. That commit didn't add RSB_VMEXIT, it added RSB_VMEXIT_LITE. Commit 9756bba28470 ("x86/speculation: Fill RSB on vmexit for IBRS") added the "heavy" version. This is also a good opportunity to call out with RSB_VMEXIT actually does.

Quoted 21 lines

The AutoIBRS (on AMD CPUs) feature implementation added in

commit e7862eda309ecf ("x86/cpu: Support AMD Automatic IBRS")

used the already-implemented logic for EIBRS in spectre_v2_determine_rsb_fill_type_on_vmexit() -- but did not update the code at VMEXIT to act on the mode selected in that function -- resulting in VMEXITs continuing to clear the RSB when RETPOLINES are enabled, despite the presence of AutoIBRS.

---

arch/x86/kvm/svm/vmenter.S +0 -0
index a0c8eb37d3e1..69d9825ebdd9 100644
@@ -209,10 +209,14 @@ SYM_FUNC_START(__svm_vcpu_run)
 7:	vmload %_ASM_AX
 8:
 

arch/x86/kvm/svm/vmenter.S SYM_FUNC_START(__svm_vcpu_run) · line 217

Quoted 10 lines
arch/x86/kvm/svm/vmenter.S +6 -4
-#ifdef CONFIG_MITIGATION_RETPOLINE
-	/* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */
-	FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE
-#endif
+	/*
+	 * IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET!
+	 *
+	 * Unlike VMX, AMD does not have the hardware bug that necessitates
+	 * RSB_VMEXIT_LITE
+	 */

I would rather do nothing than carry these comments. Long term, I would still prefer to have RSB_VMEXIT_LITE, but that's not a hill worth dying on, and I am a-ok waiting to deal with that if/when I (or someone else) takes on the task of unifying the VM-Enter/VM-Exit flows.

So, with the comment changes dropped and the changelog massaged to this:

Remove superfluous RSB filling after a VMEXIT when the CPU already has flushed the RSB after a VMEXIT when AutoIBRS is enabled.

The initial implementation for adding RETPOLINES added an ALTERNATIVES implementation for filling the RSB after a VMEXIT in commit 117cc7a908c8 ("x86/retpoline: Fill return stack buffer on vmexit").

Later, X86_FEATURE_RSB_VMEXIT was added in commit 9756bba28470 ("x86/speculation: Fill RSB on vmexit for IBRS") to handle stuffing the RSB if RETPOLINE=y *or* KERNEL_IBRS=y, i.e. to also stuff the RSB if the kernel is configured to do IBRS mitigations on entry/exit.

The AutoIBRS (on AMD) feature implementation added in commit e7862eda309e ("x86/cpu: Support AMD Automatic IBRS") used the already-implemented logic for EIBRS in spectre_v2_determine_rsb_fill_type_on_vmexit() -- but did not update the code at VMEXIT to act on the mode selected in that function -- resulting in VMEXITs continuing to clear the RSB when RETPOLINES are enabled, despite the presence of AutoIBRS.

Applied to kvm-x86 svm.

[1/1] KVM: SVM: let alternatives handle the cases when RSB filling is required

      https://github.com/kvm-x86/linux/commit/4440337af4d4

--
https://github.com/kvm-x86/linux/tree/next