Align SVM with APM defined behaviors
v3 cover Align SVM with APM defined behaviors
The APM lists the following behaviors
- The VMRUN, VMLOAD, VMSAVE, CLGI, VMMCALL, and INVLPGA instructions can be used when the EFER.SVME is set to 1; otherwise, these instructions generate a #UD exception.
- If VMMCALL instruction is not intercepted, the instruction raises a #UD exception.
- STGI instruction causes a #UD exception if SVM is not enabled and neither SVM Lock nor the device exclusion vector (DEV) are supported.
The patches in this series fix current SVM bugs that do not adhere to the APM listed behaviors.
v2 -> v3:
- Elaborated on 'Move STGI and CLGI intercept handling' commit message as per Sean
- Fixed bug due to interaction with svm_enable_nmi_window() and 'Move STGI and CLGI intercept handling' as pointed out by Yosry. Code changes suggested by Sean/Yosry.
- Removed open-coded nested_svm_check_permissions() in STGI interception function as per Yosry
v2: https://lore.kernel.org/all/[email protected]/
v1 -> v2:
- Split up the series into smaller more logical changes as suggested by Sean
- Added patch for injecting #UD for STGI under APM defined conditions as suggested by Sean
- Combined EFER.SVME=0 conditional with intel CPU logic in svm_recalc_instruction_intercepts
Kevin Cheng (5):
KVM: SVM: Move STGI and CLGI intercept handling
KVM: SVM: Inject #UD for STGI if EFER.SVME=0 and SVM Lock and DEV are
not available
KVM: SVM: Inject #UD for INVLPGA if EFER.SVME=0
KVM: SVM: Recalc instructions intercepts when EFER.SVME is toggled
KVM: SVM: Raise #UD if VMMCALL instruction is not intercepted
arch/x86/kvm/svm/nested.c | 9 +++-- arch/x86/kvm/svm/svm.c | 74 +++++++++++++++++++++++++++++++++------ arch/x86/kvm/svm/svm.h | 1 + 3 files changed, 71 insertions(+), 13 deletions(-)
--
2.52.0.457.g6b5491de43-goog
PATCH v3 1/5 KVM: SVM: Move STGI and CLGI intercept handling
Move the STGI/CLGI intercept handling to svm_recalc_instruction_intercepts() in preparation for making the function EFER-aware. A later patch will recalculate instruction intercepts when EFER.SVME is toggled, which is needed to inject #UD on STGI/CLGI when the guest clears EFER.SVME.
When clearing the STGI intercept with vgif enabled, request KVM_REQ_EVENT if there is a pending GIF-controlled event. This avoids breaking NMI/SMI window tracking, as enable_{nmi,smi}_window() sets INTERCEPT_STGI to detect when NMIs become unblocked. KVM_REQ_EVENT forces kvm_check_and_inject_events() to re-evaluate pending events and re-enable the intercept if needed.
Extract the pending GIF event check into a helper function svm_has_pending_gif_event() to deduplicate the logic between svm_recalc_instruction_intercepts() and svm_set_gif().
--- arch/x86/kvm/svm/svm.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-)
arch/x86/kvm/svm/svm.c +24 -8
@@ -972,6 +972,14 @@ void svm_write_tsc_multiplier(struct kvm_vcpu *vcpu)preempt_enable (); } +static bool svm_has_pending_gif_event (struct vcpu_svm *svm) +{ +return svm->vcpu.arch.smi_pending || + svm->vcpu.arch.nmi_pending || +kvm_cpu_has_injectable_intr (&svm->vcpu) || +kvm_apic_has_pending_init_or_sipi (&svm->vcpu); +} +/* Evaluate instruction intercepts that depend on guest CPUID features. */ static void svm_recalc_instruction_intercepts (struct kvm_vcpu *vcpu) {
@@ -1010,6 +1018,20 @@ static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu)svm_clr_intercept (svm,INTERCEPT_VMSAVE ); svm->vmcb->control.virt_ext |=VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK ; } + +if (vgif) { +/* +* If there is a pending interrupt controlled by GIF, set +* KVM_REQ_EVENT to re-evaluate if the intercept needs to be set +* again to track when GIF is re-enabled (e.g. for NMI +* injection). +*/ +svm_clr_intercept (svm,INTERCEPT_STGI ); +if (svm_has_pending_gif_event (svm)) +kvm_make_request (KVM_REQ_EVENT , &svm->vcpu); + +svm_clr_intercept (svm,INTERCEPT_CLGI ); + } } }
@@ -1147,11 +1169,8 @@ static void init_vmcb(struct kvm_vcpu *vcpu, bool init_event)if (vnmi) svm->vmcb->control.int_ctl |=V_NMI_ENABLE_MASK ; -if (vgif) { -svm_clr_intercept (svm,INTERCEPT_STGI ); -svm_clr_intercept (svm,INTERCEPT_CLGI ); +if (vgif) svm->vmcb->control.int_ctl |=V_GIF_ENABLE_MASK ; - }if (vcpu->kvm->arch.bus_lock_detection_enabled)svm_set_intercept (svm,INTERCEPT_BUSLOCK );
@@ -2247,10 +2266,7 @@ void svm_set_gif(struct vcpu_svm *svm, bool value)svm_clear_vintr (svm);enable_gif (svm); -if (svm->vcpu.arch.smi_pending || - svm->vcpu.arch.nmi_pending || -kvm_cpu_has_injectable_intr (&svm->vcpu) || -kvm_apic_has_pending_init_or_sipi (&svm->vcpu)) +if (svm_has_pending_gif_event (svm))kvm_make_request (KVM_REQ_EVENT , &svm->vcpu); }else {disable_gif (svm);
Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v3 2/5 KVM: SVM: Inject #UD for STGI if EFER.SVME=0 and SVM Lock and DEV are not available
In reply to Kevin ChengThe AMD APM states that STGI causes a #UD if SVM is not enabled and neither SVM Lock nor the device exclusion vector (DEV) are supported. Support for DEV is part of the SKINIT architecture. Fix the STGI exit handler by injecting #UD when these conditions are met.
--- arch/x86/kvm/svm/nested.c | 9 +++++++-- arch/x86/kvm/svm/svm.c | 12 +++++++++++- arch/x86/kvm/svm/svm.h | 1 + 3 files changed, 19 insertions(+), 3 deletions(-)
arch/x86/kvm/svm/nested.c +7 -2
@@ -1466,9 +1466,9 @@ int nested_svm_exit_handled(struct vcpu_svm *svm)return vmexit; } -int nested_svm_check_permissions (struct kvm_vcpu *vcpu) +int __nested_svm_check_permissions (struct kvm_vcpu *vcpu,bool insn_allowed) { -if (!(vcpu->arch.efer &EFER_SVME ) || !is_paging (vcpu)) { +if (!insn_allowed || !is_paging (vcpu)) {kvm_queue_exception (vcpu,UD_VECTOR );return 1 ; }
@@ -1481,6 +1481,11 @@ int nested_svm_check_permissions(struct kvm_vcpu *vcpu)return 0 ; } +int nested_svm_check_permissions (struct kvm_vcpu *vcpu) +{ +return __nested_svm_check_permissions (vcpu, vcpu->arch.efer &EFER_SVME ); +} +static bool nested_svm_is_exception_vmexit (struct kvm_vcpu *vcpu,u8 vector,u32 error_code) {
arch/x86/kvm/svm/svm.c +11 -1
@@ -2283,9 +2283,19 @@ void svm_set_gif(struct vcpu_svm *svm, bool value)static int stgi_interception (struct kvm_vcpu *vcpu) { +bool insn_allowed;int ret; -if (nested_svm_check_permissions (vcpu)) +/* +* According to the APM, STGI is allowed even with SVM disabled if SVM +* Lock or device exclusion vector (DEV) are supported. DEV is part of +* the SKINIT architecture. +*/ + insn_allowed = (vcpu->arch.efer &EFER_SVME ) || +guest_cpu_cap_has (vcpu,X86_FEATURE_SVML ) || +guest_cpu_cap_has (vcpu,X86_FEATURE_SKINIT ); + +if (__nested_svm_check_permissions (vcpu, insn_allowed))return 1 ; ret =kvm_skip_emulated_instruction (vcpu);
arch/x86/kvm/svm/svm.h +1 -0
@@ -769,6 +769,7 @@ static inline int nested_svm_simple_vmexit(struct vcpu_svm *svm, u32 exit_code) }int nested_svm_exit_handled (struct vcpu_svm *svm); +int __nested_svm_check_permissions (struct kvm_vcpu *vcpu,bool insn_allowed);int nested_svm_check_permissions (struct kvm_vcpu *vcpu);int nested_svm_check_exception (struct vcpu_svm *svm,unsigned nr,bool has_error_code,u32 error_code);
Signature · 1 line
2.52.0.457.g6b5491de43-goog
Sean Christopherson
maintainerRe: [PATCH V3 2/5] KVM: SVM: Inject #UD for STGI if EFER.SVME=0 and SVM Lock and DEV are not available
Quoted 4 lines · Kevin Cheng
The AMD APM states that STGI causes a #UD if SVM is not enabled and neither SVM Lock nor the device exclusion vector (DEV) are supported. Support for DEV is part of the SKINIT architecture. Fix the STGI exit handler by injecting #UD when these conditions are met.
This is entirely pointless. SVML and SKINIT can never bet set in guest caps. There are many things that are documented in the SDM/APM that don't have "correct" handling in KVM, because they're completely unsupported.
_If_ this is causing someone enough heartburn to want to "fix", just add a comment in nested_svm_check_permissions() stating that KVM doesn't support SVML or SKINIT.
Sean Christopherson
maintainerRe: [PATCH V3 2/5] KVM: SVM: Inject #UD for STGI if EFER.SVME=0 and SVM Lock and DEV are not available
Quoted 6 lines · Sean Christopherson
Quoted 4 lines · Kevin Cheng
The AMD APM states that STGI causes a #UD if SVM is not enabled and neither SVM Lock nor the device exclusion vector (DEV) are supported. Support for DEV is part of the SKINIT architecture. Fix the STGI exit handler by injecting #UD when these conditions are met.
Quoted 6 lines
This is entirely pointless. SVML and SKINIT can never bet set in guest caps. There are many things that are documented in the SDM/APM that don't have "correct" handling in KVM, because they're completely unsupported.
_If_ this is causing someone enough heartburn to want to "fix", just add a comment in
nested_svm_check_permissions()stating that KVM doesn't support SVML or SKINIT.
Case in point, patch 4 is flawed because it forces interception of STGI if EFER.SVME=0. I.e. by trying to handle the impossible, you're introducing new and novel ways for KVM to do things "wrong".
Re: [PATCH V3 2/5] KVM: SVM: Inject #UD for STGI if EFER.SVME=0 and SVM Lock and DEV are not available
Quoted 12 lines · Sean Christopherson
Quoted 10 lines · Sean Christopherson
Quoted 4 lines · Kevin Cheng
The AMD APM states that STGI causes a #UD if SVM is not enabled and neither SVM Lock nor the device exclusion vector (DEV) are supported. Support for DEV is part of the SKINIT architecture. Fix the STGI exit handler by injecting #UD when these conditions are met.
This is entirely pointless. SVML and SKINIT can never bet set in guest caps. There are many things that are documented in the SDM/APM that don't have "correct" handling in KVM, because they're completely unsupported.
Quoted 6 lines
Quoted 2 lines
_If_ this is causing someone enough heartburn to want to "fix", just add a comment in
nested_svm_check_permissions()stating that KVM doesn't support SVML or SKINIT.Case in point, patch 4 is flawed because it forces interception of STGI if EFER.SVME=0. I.e. by trying to handle the impossible, you're introducing new and novel ways for KVM to do things "wrong".
Just to clarify, do you mean patch 4 is flawed with patch 2? Or is the forcing of STGI interception flawed regardless? I am assuming the former here
Sean Christopherson
maintainerRe: [PATCH V3 2/5] KVM: SVM: Inject #UD for STGI if EFER.SVME=0 and SVM Lock and DEV are not available
Quoted 16 lines · Kevin Cheng
Quoted 15 lines · Sean Christopherson
Quoted 12 lines · Sean Christopherson
Quoted 4 lines · Kevin Cheng
The AMD APM states that STGI causes a #UD if SVM is not enabled and neither SVM Lock nor the device exclusion vector (DEV) are supported. Support for DEV is part of the SKINIT architecture. Fix the STGI exit handler by injecting #UD when these conditions are met.
This is entirely pointless. SVML and SKINIT can never bet set in guest caps. There are many things that are documented in the SDM/APM that don't have "correct" handling in KVM, because they're completely unsupported.
_If_ this is causing someone enough heartburn to want to "fix", just add a comment in
nested_svm_check_permissions()stating that KVM doesn't support SVML or SKINIT.
Quoted 7 lines
Quoted 3 lines
Case in point, patch 4 is flawed because it forces interception of STGI if EFER.SVME=0. I.e. by trying to handle the impossible, you're introducing new and novel ways for KVM to do things "wrong".
Just to clarify, do you mean patch 4 is flawed with patch 2? Or is the forcing of STGI interception flawed regardless? I am assuming the former here
Yes, the former. Checking only SVME here:
if (guest_cpuid_is_intel_compatible(vcpu) || !(efer & EFER_SVME)) {
svm_set_intercept(svm, INTERCEPT_CLGI);
svm_set_intercept(svm, INTERCEPT_STGI);
svm_set_intercept(svm, INTERCEPT_VMLOAD);
svm_set_intercept(svm, INTERCEPT_VMSAVE);is confusing, because KVM's logic for injecting the #UD would be:
if (!(vcpu->arch.efer & EFER_SVME) && !guest_cpu_cap_has(vcpu, X86_FEATURE_SVML) && !guest_cpu_cap_has(vcpu, X86_FEATURE_SKINIT)) <#ud>
which raises the question of why the interception code doesn't factor in SVML and SKINIT. "wrong" was in quotes, because there's no functional bug, but it's weird and confusing because KVM is blatantly contradicting itself.
PATCH v3 3/5 KVM: SVM: Inject #UD for INVLPGA if EFER.SVME=0
In reply to Kevin ChengINVLPGA should cause a #UD when EFER.SVME is not set. Add a check to properly inject #UD when EFER.SVME=0.
--- arch/x86/kvm/svm/svm.c | 3 +++ 1 file changed, 3 insertions(+)
arch/x86/kvm/svm/svm.c +3 -0
@@ -2320,6 +2320,9 @@ static int invlpga_interception(struct kvm_vcpu *vcpu)gva_t gva =kvm_rax_read (vcpu);u32 asid =kvm_rcx_read (vcpu); +if (nested_svm_check_permissions (vcpu)) +return 1 ; +/* FIXME: Handle an address size prefix. */ if (!is_long_mode (vcpu)) gva = (u32 )gva;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v3 4/5 KVM: SVM: Recalc instructions intercepts when EFER.SVME is toggled
In reply to Kevin ChengThe AMD APM states that VMRUN, VMLOAD, VMSAVE, CLGI, VMMCALL, and INVLPGA instructions should generate a #UD when EFER.SVME is cleared. Currently, when VMLOAD, VMSAVE, or CLGI are executed in L1 with EFER.SVME cleared, no #UD is generated in certain cases. This is because the intercepts for these instructions are cleared based on whether or not vls or vgif is enabled. The #UD fails to be generated when the intercepts are absent.
Fix the missing #UD generation by ensuring that all relevant instructions have intercepts set when SVME.EFER is disabled.
VMMCALL is special because KVM's ABI is that VMCALL/VMMCALL are always supported for L1 and never fault.
--- arch/x86/kvm/svm/svm.c | 11 ++++++++++- 1 file changed, 10 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +10 -1
@@ -243,6 +243,8 @@ int svm_set_efer(struct kvm_vcpu *vcpu, u64 efer)if (svm_gp_erratum_intercept && !sev_guest (vcpu->kvm))set_exception_intercept (svm,GP_VECTOR ); } + +kvm_make_request (KVM_REQ_RECALC_INTERCEPTS , vcpu); } svm->vmcb->save.efer = efer |EFER_SVME ;
@@ -984,6 +986,7 @@ static bool svm_has_pending_gif_event(struct vcpu_svm *svm)static void svm_recalc_instruction_intercepts (struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu); +u64 efer = vcpu->arch.efer;/* * Intercept INVPCID if shadow paging is enabled to sync/free shadow
@@ -1004,7 +1007,13 @@ static void svm_recalc_instruction_intercepts(struct kvm_vcpu *vcpu)svm_set_intercept (svm,INTERCEPT_RDTSCP ); } -if (guest_cpuid_is_intel_compatible (vcpu)) { +/* +* Intercept instructions that #UD if EFER.SVME=0, as SVME must be set even +* when running the guest, i.e. hardware will only ever see EFER.SVME=1. +*/ +if (guest_cpuid_is_intel_compatible (vcpu) || !(efer &EFER_SVME )) { +svm_set_intercept (svm,INTERCEPT_CLGI ); +svm_set_intercept (svm,INTERCEPT_STGI );svm_set_intercept (svm,INTERCEPT_VMLOAD );svm_set_intercept (svm,INTERCEPT_VMSAVE ); svm->vmcb->control.virt_ext &= ~VIRTUAL_VMLOAD_VMSAVE_ENABLE_MASK ;
Signature · 1 line
2.52.0.457.g6b5491de43-goog
PATCH v3 5/5 KVM: SVM: Raise #UD if VMMCALL instruction is not intercepted
In reply to Kevin ChengThe AMD APM states that if VMMCALL instruction is not intercepted, the instruction raises a #UD exception.
Create a vmmcall exit handler that generates a #UD if a VMMCALL exit from L2 is being handled by L0, which means that L1 did not intercept the VMMCALL instruction.
--- arch/x86/kvm/svm/svm.c | 16 +++++++++++++++- 1 file changed, 15 insertions(+), 1 deletion(-)
arch/x86/kvm/svm/svm.c +15 -1
@@ -3194,6 +3194,20 @@ static int bus_lock_exit(struct kvm_vcpu *vcpu)return 0 ; } +static int vmmcall_interception (struct kvm_vcpu *vcpu) +{ +/* +* VMMCALL #UDs if it's not intercepted, and KVM reaches this point if +* and only if the VMMCALL intercept is not set in vmcb12. +*/ +if (is_guest_mode (vcpu)) { +kvm_queue_exception (vcpu,UD_VECTOR ); +return 1 ; + } + +return kvm_emulate_hypercall (vcpu); +} +static int (*const svm_exit_handlers[])(struct kvm_vcpu *vcpu) = { [SVM_EXIT_READ_CR0 ] = cr_interception, [SVM_EXIT_READ_CR3 ] = cr_interception,
@@ -3244,7 +3258,7 @@ static int (*const svm_exit_handlers[])(struct kvm_vcpu *vcpu) = { [SVM_EXIT_TASK_SWITCH] = task_switch_interception, [SVM_EXIT_SHUTDOWN] = shutdown_interception, [SVM_EXIT_VMRUN] = vmrun_interception, - [SVM_EXIT_VMMCALL] = kvm_emulate_hypercall, + [SVM_EXIT_VMMCALL] = vmmcall_interception, [SVM_EXIT_VMLOAD] = vmload_interception, [SVM_EXIT_VMSAVE] = vmsave_interception, [SVM_EXIT_STGI] = stgi_interception,
Signature · 1 line
2.52.0.457.g6b5491de43-goog