KVM: SVM: Inhibit APIC virtualization for X2APIC guest
PATCH v2 1/2 KVM: SVM: Inhibit APIC virtualization for X2APIC guest
The AVIC does not support guest use of the x2APIC interface. Currently, KVM simply chooses to squash the x2APIC feature in the guest's CPUID If the AVIC is enabled. Doing so prevents KVM from running a guest with greater than 255 vCPUs, as such a guest necessitates the use of the x2APIC interface.
Instead, inhibit AVIC enablement on a per-VM basis whenever the x2APIC feature is set in the guest's CPUID.
---
Parent commit: a93236fcbe1d ("KVM: s390: rstify new ioctls in api.rst")
v1 => v2:
- Adopt Paolo's suggestion to inhibit the AVIC by default rather than requiring opt-in
- Drop now unnecessary module parameter
arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/svm.c | 15 +++++++++------ 2 files changed, 10 insertions(+), 6 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -890,6 +890,7 @@ enum kvm_irqchip_mode {#define APICV_INHIBIT_REASON_NESTED 2#define APICV_INHIBIT_REASON_IRQWIN 3#define APICV_INHIBIT_REASON_PIT_REINJ 4 +#define APICV_INHIBIT_REASON_X2APIC 5struct kvm_arch {unsigned long n_used_mmu_pages;
arch/x86/kvm/svm.c +9 -6
@@ -6027,7 +6027,13 @@ static void svm_cpuid_update(struct kvm_vcpu *vcpu)if (!kvm_vcpu_apicv_active (vcpu))return ; -guest_cpuid_clear (vcpu,X86_FEATURE_X2APIC ); +/* +* AVIC does not work with an x2APIC mode guest. If the X2APIC feature +* is exposed to the guest, disable AVIC. +*/ +if (guest_cpuid_has (vcpu,X86_FEATURE_X2APIC )) +kvm_request_apicv_update (vcpu->kvm, false, +APICV_INHIBIT_REASON_X2APIC );/* * Currently, AVIC does not work with nested virtualization.
@@ -6043,10 +6049,6 @@ static void svm_cpuid_update(struct kvm_vcpu *vcpu)static void svm_set_supported_cpuid (u32 func,struct kvm_cpuid_entry2 *entry) {switch (func) { -case 0x1 : -if (avic) - entry->ecx &= ~F (X2APIC ); -break ;case 0x80000001 :if (nested) entry->ecx |= (1 <<2 );/* Set SVM bit */
@@ -7370,7 +7372,8 @@ static bool svm_check_apicv_inhibit_reasons(ulong bit)BIT (APICV_INHIBIT_REASON_HYPERV ) |BIT (APICV_INHIBIT_REASON_NESTED ) |BIT (APICV_INHIBIT_REASON_IRQWIN ) | -BIT (APICV_INHIBIT_REASON_PIT_REINJ ); +BIT (APICV_INHIBIT_REASON_PIT_REINJ ) | +BIT (APICV_INHIBIT_REASON_X2APIC );return supported &BIT (bit); }
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
PATCH v2 2/2 KVM: SVM: Enable AVIC by default
Switch the default value of the 'avic' module parameter to 1.
--- arch/x86/kvm/svm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm.c +1 -1
@@ -358,7 +358,7 @@ static int nested = true;module_param (nested, int,S_IRUGO );/* enable / disable AVIC */ -static int avic; +static int avic =1 ;#ifdef CONFIG_X86_LOCAL_APIC module_param (avic, int,S_IRUGO );#endif
Signature · 1 line
2.25.1.481.gfbce0eb801-goog
Re: [PATCH v2 2/2] KVM: SVM: Enable AVIC by default
Quoted 23 lines · Oliver Upton
Switch the default value of the 'avic' module parameter to 1.
Suggested-by Paolo Bonzini <[email protected]>Signed-off-by Oliver Upton <[email protected]>--- arch/x86/kvm/svm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/svm.c +1
-2@@ -358,7 +358,7 @@ static int nested = true;module_param (nested, int,S_IRUGO );/* enable / disable AVIC */ -static int avic; +static int avic =1 ;#ifdef CONFIG_X86_LOCAL_APIC module_param (avic, int,S_IRUGO );#endif --
Quoted 1 line
2.25.1.481.gfbce0eb801-goog
How extensively has this been tested? Why hasn't someone from AMD suggested this change?
Re: [PATCH v2 2/2] KVM: SVM: Enable AVIC by default
Quoted 17 lines · Jim Mattson
Quoted 16 lines · Oliver Upton
Switch the default value of the 'avic' module parameter to 1.
Suggested-by Paolo Bonzini <[email protected]>Signed-off-by Oliver Upton <[email protected]>--- arch/x86/kvm/svm.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)arch/x86/kvm/svm.c +0
-0@@ -358,7 +358,7 @@ static int nested = true;module_param (nested, int,S_IRUGO );
Quoted 11 lines
Quoted 8 lines
arch/x86/kvm/svm.c +1
-2/* enable / disable AVIC */ -static int avic; +static int avic =1 ;#ifdef CONFIG_X86_LOCAL_APIC module_param (avic, int,S_IRUGO );#endif --2.25.1.481.gfbce0eb801-goog
How extensively has this been tested? Why hasn't someone from AMD suggested this change?
I personally don't suggest enable AVIC by default. There are cases of slow AVIC doorbell delivery, due to delivery path and contention under a large number of guest cores.
Quoted 1 line
Paolo Bonzini
maintainerRe: [PATCH v2 2/2] KVM: SVM: Enable AVIC by default
Quoted 6 lines · Wei Huang
Quoted 2 lines
How extensively has this been tested? Why hasn't someone from AMD suggested this change?
I personally don't suggest enable AVIC by default. There are cases of slow AVIC doorbell delivery, due to delivery path and contention under a large number of guest cores.
To clarify, this is a hardware issue, right?
Note that in practice this patch series wouldn't change much, because x2apic is enabled by default by userspace (it has better performance than memory-mapped APIC) and patch 1 in turn inhibits APICv if the guest has the X2APIC CPUID bit set.
Paolo
Re: [PATCH v2 2/2] KVM: SVM: Enable AVIC by default
Quoted 8 lines · Paolo Bonzini
Quoted 6 lines · Wei Huang
Quoted 2 lines
How extensively has this been tested? Why hasn't someone from AMD suggested this change?
I personally don't suggest enable AVIC by default. There are cases of slow AVIC doorbell delivery, due to delivery path and contention under a large number of guest cores.
Quoted 6 lines
To clarify, this is a hardware issue, right?
Note that in practice this patch series wouldn't change much, because x2apic is enabled by default by userspace (it has better performance than memory-mapped APIC) and patch 1 in turn inhibits APICv if the guest has the X2APIC CPUID bit set.
QEMU will work fine with this option ON due to x2APIC, just like what you said. If you feel other emulators using KVM will behave similarly, I can revoke my concern.
Quoted 3 lines
Paolo
Paolo Bonzini
maintainerRe: [PATCH v2 2/2] KVM: SVM: Enable AVIC by default
Quoted 5 lines · Wei Huang
Quoted 5 lines
Quoted 3 lines
I personally don't suggest enable AVIC by default. There are cases of slow AVIC doorbell delivery, due to delivery path and contention under a large number of guest cores.
To clarify, this is a hardware issue, right?
Quoted 7 lines
Quoted 4 lines
Note that in practice this patch series wouldn't change much, because x2apic is enabled by default by userspace (it has better performance than memory-mapped APIC) and patch 1 in turn inhibits APICv if the guest has the X2APIC CPUID bit set.
QEMU will work fine with this option ON due to x2APIC, just like what you said. If you feel other emulators using KVM will behave similarly, I can revoke my concern.
No, it's okay for now to leave it disabled. It would be nice if this information would be more easily available than by you lurking on the mailing list. :)
Paolo
Re: [PATCH v2 2/2] KVM: SVM: Enable AVIC by default
In reply to Wei HuangQuoted 3 lines · Wei Huang
I personally don't suggest enable AVIC by default. There are cases of slow AVIC doorbell delivery, due to delivery path and contention under a large number of guest cores.
Under what conditions would you suggest enabling AVIC?