KVM: SVM: let alternatives handle the cases when RSB filling is required
PATCH v4 1/1 KVM: SVM: let alternatives handle the cases when RSB filling is required
From: Amit Shah <[email protected]>
Remove superfluous RSB filling after a VMEXIT when the CPU already has flushed the RSB after a VMEXIT when AutoIBRS is enabled.
The initial implementation for adding RETPOLINES added an ALTERNATIVES implementation for filling the RSB after a VMEXIT in
commit 117cc7a908c836 ("x86/retpoline: Fill return stack buffer on vmexit")
Later, X86_FEATURE_RSB_VMEXIT was added in
commit 2b129932201673 ("x86/speculation: Add RSB VM Exit protections")
The AutoIBRS (on AMD CPUs) feature implementation added in
commit e7862eda309ecf ("x86/cpu: Support AMD Automatic IBRS")
used the already-implemented logic for EIBRS in spectre_v2_determine_rsb_fill_type_on_vmexit() -- but did not update the code at VMEXIT to act on the mode selected in that function -- resulting in VMEXITs continuing to clear the RSB when RETPOLINES are enabled, despite the presence of AutoIBRS.
---
v4: resend of v3 with subject-prefix fixed
v3:
- Add a comment mentioning SVM does not need RSB_VMEXIT_LITE unlike VMX.
v2:
- tweak commit message re: Boris's comments.
arch/x86/kvm/svm/vmenter.S | 24 ++++++++++++++++-------- 1 file changed, 16 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/vmenter.S +16 -8
@@ -209,10 +209,14 @@ SYM_FUNC_START(__svm_vcpu_run) 7: vmload %_ASM_AX 8: -#ifdef CONFIG_MITIGATION_RETPOLINE - /* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */ - FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE -#endif + /* + * IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! + * + * Unlike VMX, AMD does not have the hardware bug that necessitates + * RSB_VMEXIT_LITE + */ + + FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_VMEXIT /* Clobbers RAX, RCX, RDX. */ RESTORE_HOST_SPEC_CTRL
@@ -348,10 +352,14 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run) 2: cli -#ifdef CONFIG_MITIGATION_RETPOLINE - /* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */ - FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE -#endif + /* + * IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! + * + * Unlike VMX, AMD does not have the hardware bug that necessitates + * RSB_VMEXIT_LITE + */ + + FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS, X86_FEATURE_RSB_VMEXIT /* Clobbers RAX, RCX, RDX, consumes RDI (@svm) and RSI (@spec_ctrl_intercepted). */ RESTORE_HOST_SPEC_CTRL
Signature · 1 line
2.45.2
Pinging to bring attention to this fix.
Quoted 97 lines · Amit Shah
From: Amit Shah <[email protected]>
Remove superfluous RSB filling after a VMEXIT when the CPU already has
flushed the RSB after a VMEXIT when AutoIBRS is enabled.The initial implementation for adding RETPOLINES added an ALTERNATIVES
implementation for filling the RSB after a VMEXIT incommit
117cc7a908c836("x86/retpoline: Fill return stack buffer on vmexit")Later,
X86_FEATURE_RSB_VMEXITwas added incommit
2b129932201673("x86/speculation: Add RSB VM Exit protections")The AutoIBRS (on AMD CPUs) feature implementation added in
commit
e7862eda309ecf("x86/cpu: Support AMD Automatic IBRS")used the already-implemented logic for EIBRS in
spectre_v2_determine_rsb_fill_type_on_vmexit()-- but did not update the
code at VMEXIT to act on the mode selected in that function -- resulting
in VMEXITs continuing to clear the RSB when RETPOLINES are enabled, despite the presence of AutoIBRS.Signed-off-by Amit Shah <[email protected]>---
v4: resend of v3 with subject-prefix fixedv3:
- Add a comment mentioning SVM does not needRSB_VMEXIT_LITEunlike VMX.
v2:
- tweak commit message re: Boris's comments.arch/x86/kvm/svm/vmenter.S | 24 ++++++++++++++++--------
1 file changed, 16 insertions(+), 8 deletions(-)arch/x86/kvm/svm/vmenter.S +0
-0@@ -209,10 +209,14 @@ SYM_FUNC_START(__svm_vcpu_run)7: vmload %_ASM_AX
8:arch/x86/kvm/svm/vmenter.S +0
-2-#ifdef CONFIG_MITIGATION_RETPOLINE - /* IMPORTANT: Stuff the RSB immediately after VM-Exit,before RET! */
arch/x86/kvm/svm/vmenter.S +0
-1- FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS,
X86_FEATURE_RETPOLINEarch/x86/kvm/svm/vmenter.S +2
-1-#endif + /* + * IMPORTANT: Stuff the RSB immediately after VM-Exit,before RET!
arch/x86/kvm/svm/vmenter.S +2
-0+ * + * Unlike VMX, AMD does not have the hardware bug thatnecessitates
arch/x86/kvm/svm/vmenter.S +4
-0+ * RSB_VMEXIT_LITE + */ + + FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS,
X86_FEATURE_RSB_VMEXIT
/* Clobbers RAX, RCX, RDX. */
RESTORE_HOST_SPEC_CTRLarch/x86/kvm/svm/vmenter.S +0
-0@@ -348,10 +352,14 @@ SYM_FUNC_START(__svm_sev_es_vcpu_run)
2: cliarch/x86/kvm/svm/vmenter.S +0
-2-#ifdef CONFIG_MITIGATION_RETPOLINE - /* IMPORTANT: Stuff the RSB immediately after VM-Exit,before RET! */
arch/x86/kvm/svm/vmenter.S +0
-1- FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS,
X86_FEATURE_RETPOLINEarch/x86/kvm/svm/vmenter.S +2
-1-#endif + /* + * IMPORTANT: Stuff the RSB immediately after VM-Exit,before RET!
arch/x86/kvm/svm/vmenter.S +2
-0+ * + * Unlike VMX, AMD does not have the hardware bug thatnecessitates
arch/x86/kvm/svm/vmenter.S +4
-0+ * RSB_VMEXIT_LITE + */ + + FILL_RETURN_BUFFER %rax, RSB_CLEAR_LOOPS,
X86_FEATURE_RSB_VMEXIT
/* Clobbers RAX, RCX, RDX, consumes RDI (@svm) and RSI (@spec_ctrl_intercepted). */
RESTORE_HOST_SPEC_CTRL
Quoted 9 lines · Amit Shah
From: Amit Shah <[email protected]>
Remove superfluous RSB filling after a VMEXIT when the CPU already has flushed the RSB after a VMEXIT when AutoIBRS is enabled.
The initial implementation for adding RETPOLINES added an ALTERNATIVES implementation for filling the RSB after a VMEXIT in
commit
117cc7a908c836("x86/retpoline: Fill return stack buffer on vmexit")
Nit, no need for 14 digits, 12 is still the "official" recommendation. To make this flow better, I would also prefer to not have each commit reference be on its own line.
Quoted 3 lines
Later,
X86_FEATURE_RSB_VMEXITwas added incommit
2b129932201673("x86/speculation: Add RSB VM Exit protections")
Oh, the irony. That commit didn't add RSB_VMEXIT, it added RSB_VMEXIT_LITE. Commit 9756bba28470 ("x86/speculation: Fill RSB on vmexit for IBRS") added the "heavy" version. This is also a good opportunity to call out with RSB_VMEXIT actually does.
Quoted 21 lines
The AutoIBRS (on AMD CPUs) feature implementation added in
commit
e7862eda309ecf("x86/cpu: Support AMD Automatic IBRS")used the already-implemented logic for EIBRS in
spectre_v2_determine_rsb_fill_type_on_vmexit()-- but did not update the code at VMEXIT to act on the mode selected in that function -- resulting in VMEXITs continuing to clear the RSB when RETPOLINES are enabled, despite the presence of AutoIBRS.Signed-off-by Amit Shah <[email protected]>---
arch/x86/kvm/svm/vmenter.S +0
-0@@ -209,10 +209,14 @@ SYM_FUNC_START(__svm_vcpu_run) 7: vmload %_ASM_AX 8:
arch/x86/kvm/svm/vmenter.S SYM_FUNC_START(__svm_vcpu_run) · line 217
Quoted 10 lines
arch/x86/kvm/svm/vmenter.S +6
-4-#ifdef CONFIG_MITIGATION_RETPOLINE - /* IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! */ - FILL_RETURN_BUFFER %_ASM_AX, RSB_CLEAR_LOOPS, X86_FEATURE_RETPOLINE -#endif + /* + * IMPORTANT: Stuff the RSB immediately after VM-Exit, before RET! + * + * Unlike VMX, AMD does not have the hardware bug that necessitates + * RSB_VMEXIT_LITE + */
I would rather do nothing than carry these comments. Long term, I would still prefer to have RSB_VMEXIT_LITE, but that's not a hill worth dying on, and I am a-ok waiting to deal with that if/when I (or someone else) takes on the task of unifying the VM-Enter/VM-Exit flows.
So, with the comment changes dropped and the changelog massaged to this:
Remove superfluous RSB filling after a VMEXIT when the CPU already has flushed the RSB after a VMEXIT when AutoIBRS is enabled.
The initial implementation for adding RETPOLINES added an ALTERNATIVES implementation for filling the RSB after a VMEXIT in commit 117cc7a908c8 ("x86/retpoline: Fill return stack buffer on vmexit").
Later, X86_FEATURE_RSB_VMEXIT was added in commit 9756bba28470 ("x86/speculation: Fill RSB on vmexit for IBRS") to handle stuffing the RSB if RETPOLINE=y *or* KERNEL_IBRS=y, i.e. to also stuff the RSB if the kernel is configured to do IBRS mitigations on entry/exit.
The AutoIBRS (on AMD) feature implementation added in commit e7862eda309e ("x86/cpu: Support AMD Automatic IBRS") used the already-implemented logic for EIBRS in spectre_v2_determine_rsb_fill_type_on_vmexit() -- but did not update the code at VMEXIT to act on the mode selected in that function -- resulting in VMEXITs continuing to clear the RSB when RETPOLINES are enabled, despite the presence of AutoIBRS.
Applied to kvm-x86 svm.
[1/1] KVM: SVM: let alternatives handle the cases when RSB filling is required
https://github.com/kvm-x86/linux/commit/4440337af4d4