KVM: SVM: Add support for 4k vCPUs with x2AVIC
v4 cover KVM: SVM: Add support for 4k vCPUs with x2AVIC
This is v4 of the series posted here:
http://lkml.kernel.org/r/[email protected]
This series has been significantly re-worked based on the feedback received on v3, as well as to accommodate upstream changes to SVM and AVIC code. I have not picked up Vasant's and Pankaj's review tags for that purpose. Kindly review again.
- Naveen
Naveen N Rao (AMD) (7):
KVM: SVM: Limit AVIC physical max index based on configured
max_vcpu_ids
KVM: SVM: Add a helper to look up the max physical ID for AVIC
KVM: SVM: Replace hard-coded value 0x1FF with the corresponding macro
KVM: SVM: Expand AVIC_PHYSICAL_MAX_INDEX_MASK to be a 12-bit field
KVM: SVM: Move AVIC Physical ID table allocation to vcpu_precreate() x86/cpufeatures: Add X86_FEATURE_X2AVIC_EXT
KVM: SVM: Add AVIC support for 4k vCPUs in x2AVIC mode
arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/include/asm/svm.h | 5 +- arch/x86/kvm/svm/svm.h | 1 + arch/x86/kernel/cpu/scattered.c | 1 + arch/x86/kvm/svm/avic.c | 76 ++++++++++++++++++++++++------ arch/x86/kvm/svm/svm.c | 9 ++++ 6 files changed, 78 insertions(+), 15 deletions(-)
base-commit: 91b392ada892a2e8b1c621b9493c50f6fb49880f
Signature · 1 line
2.50.1
PATCH v4 2/7 KVM: SVM: Add a helper to look up the max physical ID for AVIC
To help with a future change, add a helper to look up the maximum physical ID depending on the vCPU AVIC mode. No functional change intended.
--- arch/x86/kvm/svm/avic.c | 26 ++++++++++++++++++++------ 1 file changed, 20 insertions(+), 6 deletions(-)
arch/x86/kvm/svm/avic.c +20 -6
@@ -79,13 +79,31 @@ static bool next_vm_id_wrapped = 0;static DEFINE_SPINLOCK (svm_vm_data_hash_lock);bool x2avic_enabled; +static u32 avic_get_max_physical_id (struct kvm_vcpu *vcpu) +{ +u32 arch_max; + +if (x2avic_enabled &&apic_x2apic_mode (vcpu->arch.apic)) + arch_max =X2AVIC_MAX_PHYSICAL_ID ; +else + arch_max =AVIC_MAX_PHYSICAL_ID ; + +/* +* Despite its name, KVM_CAP_MAX_VCPU_ID represents the maximum APIC ID plus one, +* so the max possible APIC ID is one less than that. +*/ +return min (vcpu->kvm->arch.max_vcpu_ids -1 , arch_max); +} +static void avic_activate_vmcb (struct vcpu_svm *svm) {struct vmcb *vmcb = svm->vmcb01.ptr; -struct kvm *kvm = svm->vcpu.kvm; +struct kvm_vcpu *vcpu = &svm->vcpu; vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK |X2APIC_MODE_MASK ); + vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK ; + vmcb->control.avic_physical_id |=avic_get_max_physical_id (vcpu); vmcb->control.int_ctl |=AVIC_ENABLE_MASK ;
@@ -98,8 +116,7 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)*/ if (x2avic_enabled &&apic_x2apic_mode (svm->vcpu.arch.apic)) { vmcb->control.int_ctl |=X2APIC_MODE_MASK ; - vmcb->control.avic_physical_id |=min (kvm->arch.max_vcpu_ids -1 , -X2AVIC_MAX_PHYSICAL_ID ); +/* Disabling MSR intercept for x2APIC registers */ svm_set_x2apic_msr_interception (svm, false); }else {
@@ -109,9 +126,6 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)*/ kvm_make_request (KVM_REQ_TLB_FLUSH_CURRENT , &svm->vcpu); -/* For xAVIC and hybrid-xAVIC modes */ - vmcb->control.avic_physical_id |=min (kvm->arch.max_vcpu_ids -1 , -AVIC_MAX_PHYSICAL_ID );/* Enabling MSR intercept for x2APIC registers */ svm_set_x2apic_msr_interception (svm, true); }
Signature · 1 line
2.50.1
PATCH v4 3/7 KVM: SVM: Replace hard-coded value 0x1FF with the corresponding macro
In reply to Naveen N Rao (AMD)The lower 9-bit field in EXITINFO2 represents an index into the AVIC Physical/Logical APIC ID table for a AVIC_INCOMPLETE_IPI #VMEXIT. Since the index into the Logical APIC ID table is just 8 bits, this field is actually bound by the bit-width of the index into the AVIC Physical ID table which is represented by AVIC_PHYSICAL_MAX_INDEX_MASK. So, use that macro to mask EXITINFO2.Index instead of hard coding 0x1FF in avic_incomplete_ipi_interception().
--- arch/x86/kvm/svm/avic.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/kvm/svm/avic.c +1 -1
@@ -500,7 +500,7 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)u32 icrh = svm->vmcb->control.exit_info_1 >>32 ;u32 icrl = svm->vmcb->control.exit_info_1;u32 id = svm->vmcb->control.exit_info_2 >>32 ; -u32 index = svm->vmcb->control.exit_info_2 &0x1FF ; +u32 index = svm->vmcb->control.exit_info_2 &AVIC_PHYSICAL_MAX_INDEX_MASK ;struct kvm_lapic *apic = vcpu->arch.apic;trace_kvm_avic_incomplete_ipi (vcpu->vcpu_id, icrh, icrl, id, index);
Signature · 1 line
2.50.1
PATCH v4 4/7 KVM: SVM: Expand AVIC_PHYSICAL_MAX_INDEX_MASK to be a 12-bit field
In reply to Naveen N Rao (AMD)In the latest APM describing AVIC support for 4k vCPUs, VMCB AVIC_PHYSICAL_MAX_INDEX (Offset 0xF8) and EXITINFO2.Index are both updated from 9-bit wide to 12-bit wide fields unconditionally (i.e., regardless of AVIC support for 4k vCPUs). Expand AVIC_PHYSICAL_MAX_INDEX_MASK accordingly.
While AVIC_PHYSICAL_MAX_INDEX_MASK is updated to a 12-bit field, KVM will limit the max vCPU/APIC ID based on the maximum supported on a specific processor and enforce that limit during vCPU creation. I.e., we don't need to rely on the mask to ensure that the max APIC ID being programmed in the VMCB is in range. The additional bits (11:9) were previously marked reserved and were never set/read by older processors.
--- arch/x86/include/asm/svm.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/x86/include/asm/svm.h +1 -1
@@ -279,7 +279,7 @@ enum avic_ipi_failure_cause { AVIC_IPI_FAILURE_INVALID_IPI_VECTOR, }; -#define AVIC_PHYSICAL_MAX_INDEX_MASK GENMASK_ULL(8, 0) +#define AVIC_PHYSICAL_MAX_INDEX_MASK GENMASK_ULL(11, 0)/* * For AVIC, the max index allowed for physical APIC ID table is 0xfe (254), as
Signature · 1 line
2.50.1
PATCH v4 5/7 KVM: SVM: Move AVIC Physical ID table allocation to vcpu_precreate()
In reply to Naveen N Rao (AMD)With support for 4k vCPUs in x2AVIC, the size of the AVIC Physical ID table is expanded from a single 4k page to a maximum of 8 contiguous 4k pages. The actual number of pages allocated depends on the maximum possible APIC ID in the guest, which is only known by the time the first vCPU is created. In preparation for supporting a dynamic AVIC Physical ID table size, move its allocation to vcpu_precreate().
--- arch/x86/kvm/svm/svm.h | 1 + arch/x86/kvm/svm/avic.c | 18 ++++++++++++++---- arch/x86/kvm/svm/svm.c | 9 +++++++++ 3 files changed, 24 insertions(+), 4 deletions(-)
arch/x86/kvm/svm/svm.h +1 -0
@@ -803,6 +803,7 @@ extern struct kvm_x86_nested_ops svm_nested_ops;bool avic_hardware_setup (void );int avic_ga_log_notifier (u32 ga_tag); +int avic_alloc_physical_id_table (struct kvm *kvm);void avic_vm_destroy (struct kvm *kvm);int avic_vm_init (struct kvm *kvm);void avic_init_vmcb (struct vcpu_svm *svm,struct vmcb *vmcb);
arch/x86/kvm/svm/avic.c +14 -4
@@ -185,6 +185,20 @@ int avic_ga_log_notifier(u32 ga_tag)return 0 ; } +int avic_alloc_physical_id_table (struct kvm *kvm) +{ +struct kvm_svm *kvm_svm =to_kvm_svm (kvm); + +if (kvm_svm->avic_physical_id_table || !enable_apicv || !irqchip_in_kernel (kvm)) +return 0 ; + + kvm_svm->avic_physical_id_table = (void *)get_zeroed_page (GFP_KERNEL_ACCOUNT ); +if (!kvm_svm->avic_physical_id_table) +return -ENOMEM ; + +return 0 ; +} +void avic_vm_destroy (struct kvm *kvm) {unsigned long flags;
@@ -212,10 +226,6 @@ int avic_vm_init(struct kvm *kvm)if (!enable_apicv)return 0 ; - kvm_svm->avic_physical_id_table = (void *)get_zeroed_page (GFP_KERNEL_ACCOUNT ); -if (!kvm_svm->avic_physical_id_table) - goto free_avic; - kvm_svm->avic_logical_id_table = (void *)get_zeroed_page (GFP_KERNEL_ACCOUNT );if (!kvm_svm->avic_logical_id_table) goto free_avic;
arch/x86/kvm/svm/svm.c +9 -0
@@ -1271,6 +1271,14 @@ void svm_switch_vmcb(struct vcpu_svm *svm, struct kvm_vmcb_info *target_vmcb) svm->vmcb = target_vmcb->ptr; } +static int svm_vcpu_precreate (struct kvm *kvm) +{ +if (enable_apicv) +return avic_alloc_physical_id_table (kvm); + +return 0 ; +} +static int svm_vcpu_create (struct kvm_vcpu *vcpu) {struct vcpu_svm *svm;
@@ -5063,6 +5071,7 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .emergency_disable_virtualization_cpu = svm_emergency_disable_virtualization_cpu, .has_emulated_msr = svm_has_emulated_msr, + .vcpu_precreate = svm_vcpu_precreate, .vcpu_create = svm_vcpu_create, .vcpu_free = svm_vcpu_free, .vcpu_reset = svm_vcpu_reset,
Signature · 1 line
2.50.1
PATCH v4 6/7 x86/cpufeatures: Add X86_FEATURE_X2AVIC_EXT
In reply to Naveen N Rao (AMD)Add CPUID feature bit for x2AVIC extension that enables AMD SVM to support up to 4096 vCPUs in x2AVIC mode. The primary change is in the size of the AVIC Physical ID table, which can now go up to 8 contiguous 4k pages. The number of pages allocated is controlled by the maximum APIC ID for a guest, and that controls the number of pages to allocate for the AVIC Physical ID table. AVIC hardware is enhanced to look up Physical ID table entries for vCPUs > 512 for locating the target APIC backing page and the host APIC ID of the physical core on which the guest vCPU is running.
--- arch/x86/include/asm/cpufeatures.h | 1 + arch/x86/kernel/cpu/scattered.c | 1 + 2 files changed, 2 insertions(+)
arch/x86/include/asm/cpufeatures.h +1 -0
@@ -496,6 +496,7 @@#define X86_FEATURE_TSA_L1_NO (21*32+12)/* AMD CPU not vulnerable to TSA-L1 */ #define X86_FEATURE_CLEAR_CPU_BUF_VM (21*32+13)/* Clear CPU buffers using VERW before VMRUN */ #define X86_FEATURE_MSR_IMM (21*32+14)/* MSR immediate form instructions */ +#define X86_FEATURE_X2AVIC_EXT (21*32+15)/* AMD SVM x2AVIC support for 4k vCPUs */ /* * BUG word(s)
arch/x86/kernel/cpu/scattered.c +1 -0
@@ -49,6 +49,7 @@ static const struct cpuid_bit cpuid_bits[] = { { X86_FEATURE_PROC_FEEDBACK, CPUID_EDX, 11, 0x80000007, 0 }, { X86_FEATURE_AMD_FAST_CPPC, CPUID_EDX, 15, 0x80000007, 0 }, { X86_FEATURE_MBA, CPUID_EBX, 6, 0x80000008, 0 }, + { X86_FEATURE_X2AVIC_EXT, CPUID_ECX, 6, 0x8000000a, 0 }, { X86_FEATURE_COHERENCY_SFW_NO, CPUID_EBX, 31, 0x8000001f, 0 }, { X86_FEATURE_SMBA, CPUID_EBX, 2, 0x80000020, 0 }, { X86_FEATURE_BMEC, CPUID_EBX, 3, 0x80000020, 0 },
Signature · 1 line
2.50.1
PATCH v4 7/7 KVM: SVM: Add AVIC support for 4k vCPUs in x2AVIC mode
In reply to Naveen N Rao (AMD)With AVIC support for 4k vCPUs, the maximum supported physical ID in x2AVIC mode is 4095. Since this is no longer fixed, introduce a variable (x2avic_max_physical_id) to capture the maximum supported physical ID on the current platform and use that in place of the existing macro (X2AVIC_MAX_PHYSICAL_ID).
With AVIC support for 4k vCPUs, the AVIC Physical ID table is no longer a single page and can occupy up to 8 contiguous 4k pages. Since AVIC hardware accesses of the physical ID table are limited by the physical max index programmed in the VMCB, it is sufficient to allocate only as many pages as are required to have a physical table entry for the max guest APIC ID. Since the guest APIC mode is not available at this point, provision for the maximum possible x2AVIC ID. For this purpose, add a variant of avic_get_max_physical_id() that works with a NULL vCPU pointer and returns the max x2AVIC ID. Wrap this in a new helper for obtaining the allocation order.
To make it easy to identify support for 4k vCPUs in x2AVIC mode, update the message printed to the kernel log to print the maximum number of vCPUs supported. Do this on all platforms supporting x2AVIC since it is useful to know what is supported on a specific platform.
--- arch/x86/include/asm/svm.h | 3 +++ arch/x86/kvm/svm/avic.c | 43 ++++++++++++++++++++++++++++---------- 2 files changed, 35 insertions(+), 11 deletions(-)
arch/x86/include/asm/svm.h +3 -0
@@ -289,11 +289,14 @@ enum avic_ipi_failure_cause {/* * For x2AVIC, the max index allowed for physical APIC ID table is 0x1ff (511). +* With X86_FEATURE_X2AVIC_EXT, the max index is increased to 0xfff (4095). */ #define X2AVIC_MAX_PHYSICAL_ID 0x1FFUL +#define X2AVIC_4K_MAX_PHYSICAL_ID 0xFFFULstatic_assert ((AVIC_MAX_PHYSICAL_ID &AVIC_PHYSICAL_MAX_INDEX_MASK ) ==AVIC_MAX_PHYSICAL_ID );static_assert ((X2AVIC_MAX_PHYSICAL_ID &AVIC_PHYSICAL_MAX_INDEX_MASK ) ==X2AVIC_MAX_PHYSICAL_ID ); +static_assert ((X2AVIC_4K_MAX_PHYSICAL_ID &AVIC_PHYSICAL_MAX_INDEX_MASK ) ==X2AVIC_4K_MAX_PHYSICAL_ID );#define SVM_SEV_FEAT_SNP_ACTIVE BIT(0)#define SVM_SEV_FEAT_RESTRICTED_INJECTION BIT(3)
arch/x86/kvm/svm/avic.c +32 -11
101 lines not on this page · raw message
Signature · 1 line
2.50.1
PATCH v4 1/7 KVM: SVM: Limit AVIC physical max index based on configured max_vcpu_ids
In reply to Naveen N Rao (AMD)KVM allows VMMs to specify the maximum possible APIC ID for a virtual machine through KVM_CAP_MAX_VCPU_ID capability so as to limit data structures related to APIC/x2APIC. Utilize the same to set the AVIC physical max index in the VMCB, similar to VMX. This helps hardware limit the number of entries to be scanned in the physical APIC ID table speeding up IPI broadcasts for virtual machines with smaller number of vCPUs.
Unlike VMX, SVM AVIC requires a single page to be allocated for the Physical APIC ID table and the Logical APIC ID table, so retain the existing approach of allocating those during VM init.
--- arch/x86/kvm/svm/avic.c | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-)
arch/x86/kvm/svm/avic.c +5 -2
@@ -82,6 +82,7 @@ bool x2avic_enabled;static void avic_activate_vmcb (struct vcpu_svm *svm) {struct vmcb *vmcb = svm->vmcb01.ptr; +struct kvm *kvm = svm->vcpu.kvm; vmcb->control.int_ctl &= ~(AVIC_ENABLE_MASK |X2APIC_MODE_MASK ); vmcb->control.avic_physical_id &= ~AVIC_PHYSICAL_MAX_INDEX_MASK ;
@@ -97,7 +98,8 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)*/ if (x2avic_enabled &&apic_x2apic_mode (svm->vcpu.arch.apic)) { vmcb->control.int_ctl |=X2APIC_MODE_MASK ; - vmcb->control.avic_physical_id |=X2AVIC_MAX_PHYSICAL_ID ; + vmcb->control.avic_physical_id |=min (kvm->arch.max_vcpu_ids -1 , +X2AVIC_MAX_PHYSICAL_ID );/* Disabling MSR intercept for x2APIC registers */ svm_set_x2apic_msr_interception (svm, false); }else {
@@ -108,7 +110,8 @@ static void avic_activate_vmcb(struct vcpu_svm *svm)kvm_make_request (KVM_REQ_TLB_FLUSH_CURRENT , &svm->vcpu);/* For xAVIC and hybrid-xAVIC modes */ - vmcb->control.avic_physical_id |=AVIC_MAX_PHYSICAL_ID ; + vmcb->control.avic_physical_id |=min (kvm->arch.max_vcpu_ids -1 , +AVIC_MAX_PHYSICAL_ID );/* Enabling MSR intercept for x2APIC registers */ svm_set_x2apic_msr_interception (svm, true); }
Signature · 1 line
2.50.1