RFCv2: nested AVIC
Subsystems (6)
v2 cover RFCv2: nested AVIC
This patch series implement everything that is needed to use AMD's AVIC while a nested guest is running including ability of the nested guest to use it, and brings feature parity vs APICv.
Compared to v1 of the series, there are lot of fixes, and refactoring.
This version still use unconditional read-only apic id, which will be addressed in the next version.
For the last patch, which allows to avoid cleaning is_running bit in physid pages as long as it is possible, I measured what would happen in a worst case:
- A malicious guest runs with 2 vCPUs pinned, its first vCPU pounds on ICR sending IPIs to the 2nd vCPU
and 2nd vCPU scheduled out forever and not halted (something that should not happen with Qemu though)
- A normal guest with single vCPU is pinned to run on the same CPU as the 2nd vCPU of the first guest.
The normal guest continued to run, but was observed to run about 40% slower.
Therefore AVIC doorbel is strict by default but if the admin policy is to pin guests and not allow them to share a physical cpu, then strict doorbel can be set to false and that does improves the nested (and not nested as well) AVIC perf futher.
Suggestions, comments are welcome.
Best regards,
Maxim Levitsky
Maxim Levitsky (10):
KVM: x86: mmu: allow to enable write tracking externally
x86: KVMGT: use kvm_page_track_write_tracking_enable
KVM: x86: mmu: add gfn_in_memslot helper
KVM: x86: mmu: tweak fast path for emulation of access to nested NPT
pages
KVM: x86: lapic: don't allow to change APIC ID when apic acceleration
is enabled
KVM: x86: SVM: remove avic's broken code that updated APIC ID
KVM: x86: SVM: move avic state to separate struct
KVM: x86: rename .set_apic_access_page_addr to reload_apic_access_page
KVM: nSVM: implement support for nested AVIC
KVM: SVM: allow to avoid not needed updates to is_running
arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/asm/kvm_host.h | 5 +- arch/x86/include/asm/kvm_page_track.h | 1 + arch/x86/kvm/Kconfig | 3 - arch/x86/kvm/lapic.c | 28 +- arch/x86/kvm/mmu.h | 8 +- arch/x86/kvm/mmu/mmu.c | 21 +- arch/x86/kvm/mmu/page_track.c | 10 +- arch/x86/kvm/svm/avic.c | 949 ++++++++++++++++++++++++-- arch/x86/kvm/svm/nested.c | 131 +++- arch/x86/kvm/svm/svm.c | 31 +- arch/x86/kvm/svm/svm.h | 165 ++++- arch/x86/kvm/trace.h | 140 +++- arch/x86/kvm/vmx/vmx.c | 8 +- arch/x86/kvm/x86.c | 17 +- drivers/gpu/drm/i915/Kconfig | 1 - drivers/gpu/drm/i915/gvt/kvmgt.c | 5 + include/linux/kvm_host.h | 10 +- 18 files changed, 1413 insertions(+), 122 deletions(-)
Signature · 1 line
2.26.3
RFC PATCH v2 1/10 KVM: x86: mmu: allow to enable write tracking externally
This will be used to enable write tracking from nested AVIC code and can also be used to enable write tracking in GVT-g module when it actually uses it as opposed to always enabling it, when the module is compiled in the kernel.
No functional change intended.
--- arch/x86/include/asm/kvm_host.h | 2 +- arch/x86/include/asm/kvm_page_track.h | 1 + arch/x86/kvm/mmu.h | 8 +++++--- arch/x86/kvm/mmu/mmu.c | 17 ++++++++++------- arch/x86/kvm/mmu/page_track.c | 10 ++++++++-- 5 files changed, 25 insertions(+), 13 deletions(-)
arch/x86/include/asm/kvm_host.h +1 -1
@@ -1234,7 +1234,7 @@ struct kvm_arch {* is used as one input when determining whether certain memslot * related allocations are necessary. */ -bool shadow_root_allocated; +bool mmu_page_tracking_enabled;#if IS_ENABLED (CONFIG_HYPERV )hpa_t hv_root_tdp;
arch/x86/include/asm/kvm_page_track.h +1 -0
@@ -50,6 +50,7 @@ int kvm_page_track_init(struct kvm *kvm);void kvm_page_track_cleanup (struct kvm *kvm);bool kvm_page_track_write_tracking_enabled (struct kvm *kvm); +int kvm_page_track_write_tracking_enable (struct kvm *kvm);int kvm_page_track_write_tracking_alloc (struct kvm_memory_slot *slot);void kvm_page_track_free_memslot (struct kvm_memory_slot *slot);
arch/x86/kvm/mmu.h +5 -3
@@ -269,7 +269,7 @@ int kvm_arch_write_log_dirty(struct kvm_vcpu *vcpu);int kvm_mmu_post_init_vm (struct kvm *kvm);void kvm_mmu_pre_destroy_vm (struct kvm *kvm); -static inline bool kvm_shadow_root_allocated (struct kvm *kvm) +static inline bool mmu_page_tracking_enabled (struct kvm *kvm) {/* * Read shadow_root_allocated before related pointers. Hence, threads
@@ -277,9 +277,11 @@ static inline bool kvm_shadow_root_allocated(struct kvm *kvm)* see the pointers. Pairs with smp_store_release in * mmu_first_shadow_root_alloc. */ -return smp_load_acquire (&kvm->arch.shadow_root_allocated); +return smp_load_acquire (&kvm->arch.mmu_page_tracking_enabled); } +int mmu_enable_write_tracking (struct kvm *kvm); +#ifdef CONFIG_X86_64 static inline bool is_tdp_mmu_enabled (struct kvm *kvm) {return kvm->arch.tdp_mmu_enabled; }#else
@@ -288,7 +290,7 @@ static inline bool is_tdp_mmu_enabled(struct kvm *kvm) { return false; }static inline bool kvm_memslots_have_rmaps (struct kvm *kvm) { -return !is_tdp_mmu_enabled (kvm) ||kvm_shadow_root_allocated (kvm); +return !is_tdp_mmu_enabled (kvm) ||mmu_page_tracking_enabled (kvm); }static inline gfn_t gfn_to_index (gfn_t gfn,gfn_t base_gfn,int level)
arch/x86/kvm/mmu/mmu.c +10 -7
@@ -3368,7 +3368,7 @@ static int mmu_alloc_direct_roots(struct kvm_vcpu *vcpu)return r; } -static int mmu_first_shadow_root_alloc (struct kvm *kvm) +int mmu_enable_write_tracking (struct kvm *kvm) {struct kvm_memslots *slots;struct kvm_memory_slot *slot;
@@ -3378,21 +3378,20 @@ static int mmu_first_shadow_root_alloc(struct kvm *kvm)* Check if this is the first shadow root being allocated before * taking the lock. */ -if (kvm_shadow_root_allocated (kvm)) +if (mmu_page_tracking_enabled (kvm))return 0 ;mutex_lock (&kvm->slots_arch_lock);/* Recheck, under the lock, whether this is the first shadow root. */ -if (kvm_shadow_root_allocated (kvm)) +if (mmu_page_tracking_enabled (kvm)) goto out_unlock;/* * Check if anything actually needs to be allocated, e.g. all metadata * will be allocated upfront if TDP is disabled. */ -if (kvm_memslots_have_rmaps (kvm) && -kvm_page_track_write_tracking_enabled (kvm)) +if (kvm_memslots_have_rmaps (kvm) &&mmu_page_tracking_enabled (kvm)) goto out_success;for (i =0 ; i <KVM_ADDRESS_SPACE_NUM ; i++) {
@@ -3422,7 +3421,7 @@ static int mmu_first_shadow_root_alloc(struct kvm *kvm)* all the related pointers are set. */ out_success: -smp_store_release (&kvm->arch.shadow_root_allocated, true); +smp_store_release (&kvm->arch.mmu_page_tracking_enabled, true); out_unlock:mutex_unlock (&kvm->slots_arch_lock);
@@ -3459,7 +3458,7 @@ static int mmu_alloc_shadow_roots(struct kvm_vcpu *vcpu) } } - r =mmu_first_shadow_root_alloc (vcpu->kvm); + r =mmu_enable_write_tracking (vcpu->kvm);if (r)return r;
@@ -5727,6 +5726,10 @@ int kvm_mmu_init_vm(struct kvm *kvm) node->track_write = kvm_mmu_pte_write; node->track_flush_slot = kvm_mmu_invalidate_zap_pages_in_memslot;kvm_page_track_register_notifier (kvm, node); + +if (IS_ENABLED (CONFIG_KVM_EXTERNAL_WRITE_TRACKING ) || !tdp_enabled) +mmu_enable_write_tracking (kvm); +return 0 ; }
arch/x86/kvm/mmu/page_track.c +8 -2
@@ -21,10 +21,16 @@bool kvm_page_track_write_tracking_enabled (struct kvm *kvm) { -return IS_ENABLED (CONFIG_KVM_EXTERNAL_WRITE_TRACKING ) || - !tdp_enabled ||kvm_shadow_root_allocated (kvm); +return mmu_page_tracking_enabled (kvm); } +int kvm_page_track_write_tracking_enable (struct kvm *kvm) +{ +return mmu_enable_write_tracking (kvm); +} +EXPORT_SYMBOL_GPL (kvm_page_track_write_tracking_enable); + +void kvm_page_track_free_memslot (struct kvm_memory_slot *slot) {int i;
Signature · 1 line
2.26.3
RFC PATCH v2 2/10 x86: KVMGT: use kvm_page_track_write_tracking_enable
In reply to Maxim LevitskyThis allows to enable the write tracking only when KVMGT is actually used and doesn't carry any penalty otherwise.
Tested by booting a VM with a kvmgt mdev device.
--- arch/x86/kvm/Kconfig | 3 --- arch/x86/kvm/mmu/mmu.c | 2 +- drivers/gpu/drm/i915/Kconfig | 1 - drivers/gpu/drm/i915/gvt/kvmgt.c | 5 +++++ 4 files changed, 6 insertions(+), 5 deletions(-)
arch/x86/kvm/Kconfig +0 -3
@@ -126,7 +126,4 @@ config KVM_XEN If in doubt, say "N". -config KVM_EXTERNAL_WRITE_TRACKING - bool - endif # VIRTUALIZATION
arch/x86/kvm/mmu/mmu.c +1 -1
@@ -5727,7 +5727,7 @@ int kvm_mmu_init_vm(struct kvm *kvm) node->track_flush_slot = kvm_mmu_invalidate_zap_pages_in_memslot;kvm_page_track_register_notifier (kvm, node); -if (IS_ENABLED (CONFIG_KVM_EXTERNAL_WRITE_TRACKING ) || !tdp_enabled) +if (!tdp_enabled)mmu_enable_write_tracking (kvm);return 0 ;
drivers/gpu/drm/i915/Kconfig +0 -1
@@ -130,7 +130,6 @@ config DRM_I915_GVT_KVMGT depends on DRM_I915_GVT depends on KVM depends on VFIO_MDEV - select KVM_EXTERNAL_WRITE_TRACKING default n help Choose this option if you want to enable KVMGT support for
drivers/gpu/drm/i915/gvt/kvmgt.c +5 -0
@@ -1933,6 +1933,7 @@ static int kvmgt_guest_init(struct mdev_device *mdev)struct intel_vgpu *vgpu;struct kvmgt_vdev *vdev;struct kvm *kvm; +int ret; vgpu =mdev_get_drvdata (mdev);if (handle_valid (vgpu->handle))
@@ -1948,6 +1949,10 @@ static int kvmgt_guest_init(struct mdev_device *mdev)if (__kvmgt_vgpu_exist (vgpu, kvm))return -EEXIST ; + ret =kvm_page_track_write_tracking_enable (kvm); +if (ret) +return ret; + info =vzalloc (sizeof (struct kvmgt_guest_info ));if (!info)return -ENOMEM ;
Signature · 1 line
2.26.3
RFC PATCH v2 3/10 KVM: x86: mmu: add gfn_in_memslot helper
In reply to Maxim LevitskyThis is a tiny refactoring, and can be useful to check if a GPA/GFN is within a memslot a bit more cleanly.
--- include/linux/kvm_host.h | 10 +++++++++- 1 file changed, 9 insertions(+), 1 deletion(-)
include/linux/kvm_host.h +9 -1
@@ -1580,6 +1580,13 @@ int kvm_request_irq_source_id(struct kvm *kvm);void kvm_free_irq_source_id (struct kvm *kvm,int irq_source_id);bool kvm_arch_irqfd_allowed (struct kvm *kvm,struct kvm_irqfd *args); + +static inline bool gfn_in_memslot (struct kvm_memory_slot *slot,gfn_t gfn) +{ +return (gfn >= slot->base_gfn && gfn < slot->base_gfn + slot->npages); +} + +/* * Returns a pointer to the memslot if it contains gfn. * Otherwise returns NULL.
@@ -1590,12 +1597,13 @@ try_get_memslot(struct kvm_memory_slot *slot, gfn_t gfn)if (!slot)return NULL ; -if (gfn >= slot->base_gfn && gfn < slot->base_gfn + slot->npages) +if (gfn_in_memslot (slot, gfn))return slot;else return NULL ; } +/* * Returns a pointer to the memslot that contains gfn. Otherwise returns NULL. *
Signature · 1 line
2.26.3
RFC PATCH v2 4/10 KVM: x86: mmu: tweak fast path for emulation of access to nested NPT pages
In reply to Maxim Levitsky--- arch/x86/kvm/mmu/mmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/x86/kvm/mmu/mmu.c +2 -2
@@ -5315,8 +5315,8 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,*/ if (vcpu->arch.mmu->root_role.direct && (error_code &PFERR_NESTED_GUEST_PAGE ) ==PFERR_NESTED_GUEST_PAGE ) { -kvm_mmu_unprotect_page (vcpu->kvm,gpa_to_gfn (cr2_or_gpa)); -return 1 ; +if (kvm_mmu_unprotect_page (vcpu->kvm,gpa_to_gfn (cr2_or_gpa))) +return 1 ; }/*
Signature · 1 line
2.26.3
Re: [RFC PATCH v2 04/10] KVM: x86: mmu: tweak fast path for emulation of access to nested NPT pages
Quoted 9 lines · Maxim Levitsky
--- arch/x86/kvm/mmu/mmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)arch/x86/kvm/mmu/mmu.c +0
-0
arch/x86/kvm/mmu/mmu.c int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code, · line 5322
Quoted 10 lines
arch/x86/kvm/mmu/mmu.c +2
-2@@ -5315,8 +5315,8 @@ int kvm_mmu_page_fault(struct kvm_vcpu *vcpu, gpa_t cr2_or_gpa, u64 error_code,*/ if (vcpu->arch.mmu->root_role.direct && (error_code &PFERR_NESTED_GUEST_PAGE ) ==PFERR_NESTED_GUEST_PAGE ) { -kvm_mmu_unprotect_page (vcpu->kvm,gpa_to_gfn (cr2_or_gpa)); -return 1 ; +if (kvm_mmu_unprotect_page (vcpu->kvm,gpa_to_gfn (cr2_or_gpa))) +return 1 ; }/*
I forgot to add commit description here:
If non leaf mmu page is write tracked externally for some reason, which can in theory happen if it was used for nested avic physid page before, then this code will enter an endless loop of page faults because unprotecting the page will not remove write tracking, nor will the write tracker callback be called.
Fix this by only invoking the fast patch if we succeeded in zapping the mmu page.
--
In theory, KVMGT also does external write tracking so in theory this issue can happen today, but it is highly unlikely.
Best regards,
Maxim Levitsk
RFC PATCH v2 5/10 KVM: x86: lapic: don't allow to change APIC ID when apic acceleration is enabled
In reply to Maxim LevitskyNo normal guest has any reason to change physical APIC IDs, and allowing this introduces bugs into APIC acceleration code.
--- arch/x86/kvm/lapic.c | 28 +++++++++++++++++++++++----- 1 file changed, 23 insertions(+), 5 deletions(-)
arch/x86/kvm/lapic.c +23 -5
@@ -2046,10 +2046,20 @@ static int kvm_lapic_reg_write(struct kvm_lapic *apic, u32 reg, u32 val)switch (reg) {case APIC_ID :/* Local APIC ID */ -if (!apic_x2apic_mode (apic)) -kvm_apic_set_xapic_id (apic, val >>24 ); -else +if (apic_x2apic_mode (apic)) { ret =1 ; +break ; + } +/* +* Don't allow setting APIC ID with any APIC acceleration +* enabled to avoid unexpected issues +*/ +if (enable_apicv && ((val >>24 ) != apic->vcpu->vcpu_id)) { +kvm_vm_bugged (apic->vcpu->kvm); +break ; + } + +kvm_apic_set_xapic_id (apic, val >>24 );break ;case APIC_TASKPRI :
@@ -2617,8 +2627,16 @@ int kvm_get_apic_interrupt(struct kvm_vcpu *vcpu)static int kvm_apic_state_fixup (struct kvm_vcpu *vcpu,struct kvm_lapic_state *s,bool set) { -if (apic_x2apic_mode (vcpu->arch.apic)) { -u32 *id = (u32 *)(s->regs +APIC_ID ); +u32 *id = (u32 *)(s->regs +APIC_ID ); + +if (!apic_x2apic_mode (vcpu->arch.apic)) { +/* Don't allow setting APIC ID with any APIC acceleration +* enabled to avoid unexpected issues +*/ +if (enable_apicv && (*id >>24 ) != vcpu->vcpu_id) +return -EINVAL ; + }else { +u32 *ldr = (u32 *)(s->regs +APIC_LDR );u64 icr;
Signature · 1 line
2.26.3
RFC PATCH v2 6/10 KVM: x86: SVM: remove avic's broken code that updated APIC ID
In reply to Maxim LevitskyNow that KVM doesn't allow to change APIC ID in case AVIC is enabled, remove buggy AVIC code that tried to do so.
--- arch/x86/kvm/svm/avic.c | 35 ----------------------------------- 1 file changed, 35 deletions(-)
arch/x86/kvm/svm/avic.c +0 -35
@@ -442,35 +442,6 @@ static int avic_handle_ldr_update(struct kvm_vcpu *vcpu)return ret; } -static int avic_handle_apic_id_update (struct kvm_vcpu *vcpu) -{ -u64 *old, *new; -struct vcpu_svm *svm =to_svm (vcpu); -u32 id =kvm_xapic_id (vcpu->arch.apic); - -if (vcpu->vcpu_id == id) -return 0 ; - - old =avic_get_physical_id_entry (vcpu, vcpu->vcpu_id); - new =avic_get_physical_id_entry (vcpu, id); -if (!new || !old) -return 1 ; - -/* We need to move physical_id_entry to new offset */ - *new = *old; - *old =0ULL ; -to_svm (vcpu)->avic_physical_id_cache = new; - -/* -* Also update the guest physical APIC ID in the logical -* APIC ID table entry if already setup the LDR. -*/ -if (svm->ldr_reg) -avic_handle_ldr_update (vcpu); - -return 0 ; -} -static void avic_handle_dfr_update (struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);
@@ -489,10 +460,6 @@ static int avic_unaccel_trap_write(struct kvm_vcpu *vcpu)AVIC_UNACCEL_ACCESS_OFFSET_MASK ;switch (offset) { -case APIC_ID : -if (avic_handle_apic_id_update (vcpu)) -return 0 ; -break ;case APIC_LDR :if (avic_handle_ldr_update (vcpu))return 0 ;
@@ -584,8 +551,6 @@ int avic_init_vcpu(struct vcpu_svm *svm)void avic_apicv_post_state_restore (struct kvm_vcpu *vcpu) { -if (avic_handle_apic_id_update (vcpu) !=0 ) -return ;avic_handle_dfr_update (vcpu);avic_handle_ldr_update (vcpu); }
Signature · 1 line
2.26.3
RFC PATCH v2 7/10 KVM: x86: SVM: move avic state to separate struct
In reply to Maxim LevitskyThis will make the code a bit easier to read when nested AVIC support is added.
No functional change intended.
--- arch/x86/kvm/svm/avic.c | 49 +++++++++++++++++++++++------------------ arch/x86/kvm/svm/svm.h | 14 +++++++----- 2 files changed, 36 insertions(+), 27 deletions(-)
arch/x86/kvm/svm/avic.c +28 -21
@@ -69,6 +69,8 @@ int avic_ga_log_notifier(u32 ga_tag)unsigned long flags;struct kvm_svm *kvm_svm;struct kvm_vcpu *vcpu =NULL ; +struct kvm_svm_avic *avic; +u32 vm_id =AVIC_GATAG_TO_VMID (ga_tag);u32 vcpu_id =AVIC_GATAG_TO_VCPUID (ga_tag);
@@ -76,9 +78,13 @@ int avic_ga_log_notifier(u32 ga_tag)trace_kvm_avic_ga_log (vm_id, vcpu_id);spin_lock_irqsave (&svm_vm_data_hash_lock, flags); -hash_for_each_possible (svm_vm_data_hash, kvm_svm, hnode, vm_id) { -if (kvm_svm->avic_vm_id != vm_id) +hash_for_each_possible (svm_vm_data_hash, avic, hnode, vm_id) { + + +if (avic->vm_id != vm_id)continue ; + + kvm_svm = container_of(avic, struct kvm_svm, avic); vcpu =kvm_get_vcpu_by_id (&kvm_svm->kvm, vcpu_id);break ; }
@@ -98,18 +104,18 @@ int avic_ga_log_notifier(u32 ga_tag)void avic_vm_destroy (struct kvm *kvm) {unsigned long flags; -struct kvm_svm *kvm_svm =to_kvm_svm (kvm); +struct kvm_svm_avic *avic = &to_kvm_svm (kvm)->avic;if (!enable_apicv)return ; -if (kvm_svm->avic_logical_id_table_page) -__free_page (kvm_svm->avic_logical_id_table_page); -if (kvm_svm->avic_physical_id_table_page) -__free_page (kvm_svm->avic_physical_id_table_page); +if (avic->logical_id_table_page) +__free_page (avic->logical_id_table_page); +if (avic->physical_id_table_page) +__free_page (avic->physical_id_table_page);spin_lock_irqsave (&svm_vm_data_hash_lock, flags); -hash_del (&kvm_svm->hnode); +hash_del (&avic->hnode);spin_unlock_irqrestore (&svm_vm_data_hash_lock, flags); }
@@ -117,10 +123,9 @@ int avic_vm_init(struct kvm *kvm) {unsigned long flags;int err = -ENOMEM ; -struct kvm_svm *kvm_svm =to_kvm_svm (kvm); -struct kvm_svm *k2;struct page *p_page;struct page *l_page; +struct kvm_svm_avic *avic = &to_kvm_svm (kvm)->avic;u32 vm_id;if (!enable_apicv)
@@ -131,14 +136,14 @@ int avic_vm_init(struct kvm *kvm)if (!p_page) goto free_avic; - kvm_svm->avic_physical_id_table_page = p_page; + avic->physical_id_table_page = p_page;/* Allocating logical APIC ID table (4KB) */ l_page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO);if (!l_page) goto free_avic; - kvm_svm->avic_logical_id_table_page = l_page; + avic->logical_id_table_page = l_page;spin_lock_irqsave (&svm_vm_data_hash_lock, flags); again:
@@ -149,13 +154,15 @@ int avic_vm_init(struct kvm *kvm) }/* Is it still in use? Only possible if wrapped at least once */ if (next_vm_id_wrapped) { -hash_for_each_possible (svm_vm_data_hash, k2, hnode, vm_id) { -if (k2->avic_vm_id == vm_id) +struct kvm_svm_avic *avic2; + +hash_for_each_possible (svm_vm_data_hash, avic2, hnode, vm_id) { +if (avic2->vm_id == vm_id) goto again; } } - kvm_svm->avic_vm_id = vm_id; -hash_add (svm_vm_data_hash, &kvm_svm->hnode, kvm_svm->avic_vm_id); + avic->vm_id = vm_id; +hash_add (svm_vm_data_hash, &avic->hnode, avic->vm_id);spin_unlock_irqrestore (&svm_vm_data_hash_lock, flags);return 0 ;
@@ -169,8 +176,8 @@ void avic_init_vmcb(struct vcpu_svm *svm, struct vmcb *vmcb) {struct kvm_svm *kvm_svm =to_kvm_svm (svm->vcpu.kvm);phys_addr_t bpa =__sme_set (page_to_phys (svm->avic_backing_page)); -phys_addr_t lpa =__sme_set (page_to_phys (kvm_svm->avic_logical_id_table_page)); -phys_addr_t ppa =__sme_set (page_to_phys (kvm_svm->avic_physical_id_table_page)); +phys_addr_t lpa =__sme_set (page_to_phys (kvm_svm->avic.logical_id_table_page)); +phys_addr_t ppa =__sme_set (page_to_phys (kvm_svm->avic.physical_id_table_page)); vmcb->control.avic_backing_page = bpa &AVIC_HPA_MASK ; vmcb->control.avic_logical_id = lpa &AVIC_HPA_MASK ;
@@ -193,7 +200,7 @@ static u64 *avic_get_physical_id_entry(struct kvm_vcpu *vcpu,if (index >=AVIC_MAX_PHYSICAL_ID_COUNT )return NULL ; - avic_physical_id_table =page_address (kvm_svm->avic_physical_id_table_page); + avic_physical_id_table =page_address (kvm_svm->avic.physical_id_table_page);return &avic_physical_id_table[index]; }
@@ -387,7 +394,7 @@ static u32 *avic_get_logical_id_entry(struct kvm_vcpu *vcpu, u32 ldr, bool flat) index = (cluster <<2 ) + apic; } - logical_apic_id_table = (u32 *)page_address (kvm_svm->avic_logical_id_table_page); + logical_apic_id_table = (u32 *)page_address (kvm_svm->avic.logical_id_table_page);return &logical_apic_id_table[index]; }
@@ -737,7 +744,7 @@ int avic_pi_update_irte(struct kvm *kvm, unsigned int host_irq,/* Try to enable guest_mode in IRTE */ pi.base =__sme_set (page_to_phys (svm->avic_backing_page) &AVIC_HPA_MASK ); - pi.ga_tag =AVIC_GATAG (to_kvm_svm (kvm)->avic_vm_id, + pi.ga_tag =AVIC_GATAG (to_kvm_svm (kvm)->avic.vm_id, svm->vcpu.vcpu_id); pi.is_guest_mode = true; pi.vcpu_data = &vcpu_info;
arch/x86/kvm/svm/svm.h +8 -6
@@ -88,15 +88,17 @@ struct kvm_sev_info {atomic_t migration_in_progress; }; -struct kvm_svm { -struct kvm kvm; -/* Struct members for AVIC */ -u32 avic_vm_id; -struct page *avic_logical_id_table_page; -struct page *avic_physical_id_table_page; +struct kvm_svm_avic { +u32 vm_id; +struct page *logical_id_table_page; +struct page *physical_id_table_page;struct hlist_node hnode; +}; +struct kvm_svm { +struct kvm kvm; +struct kvm_svm_avic avic;struct kvm_sev_info sev_info; };
Signature · 1 line
2.26.3
RFC PATCH v2 8/10 KVM: x86: rename .set_apic_access_page_addr to reload_apic_access_page
In reply to Maxim LevitskyThis will be used on SVM to reload shadow page of the AVIC physid table
No functional change intended
--- arch/x86/include/asm/kvm-x86-ops.h | 2 +- arch/x86/include/asm/kvm_host.h | 3 +-- arch/x86/kvm/vmx/vmx.c | 8 ++++---- arch/x86/kvm/x86.c | 6 +++--- 4 files changed, 9 insertions(+), 10 deletions(-)
arch/x86/include/asm/kvm-x86-ops.h +1 -1
@@ -82,7 +82,7 @@ KVM_X86_OP_OPTIONAL(hwapic_isr_update) KVM_X86_OP_OPTIONAL_RET0(guest_apic_has_interrupt) KVM_X86_OP_OPTIONAL(load_eoi_exitmap) KVM_X86_OP_OPTIONAL(set_virtual_apic_mode) -KVM_X86_OP_OPTIONAL(set_apic_access_page_addr) +KVM_X86_OP_OPTIONAL(reload_apic_pages) KVM_X86_OP(deliver_interrupt) KVM_X86_OP_OPTIONAL(sync_pir_to_irr) KVM_X86_OP_OPTIONAL_RET0(set_tss_addr)
arch/x86/include/asm/kvm_host.h +1 -2
@@ -1415,7 +1415,7 @@ struct kvm_x86_ops {bool (*guest_apic_has_interrupt)(struct kvm_vcpu *vcpu);void (*load_eoi_exitmap)(struct kvm_vcpu *vcpu,u64 *eoi_exit_bitmap);void (*set_virtual_apic_mode)(struct kvm_vcpu *vcpu); -void (*set_apic_access_page_addr)(struct kvm_vcpu *vcpu); +void (*reload_apic_pages)(struct kvm_vcpu *vcpu);void (*deliver_interrupt)(struct kvm_lapic *apic,int delivery_mode,int trig_mode,int vector);int (*sync_pir_to_irr)(struct kvm_vcpu *vcpu);
@@ -1888,7 +1888,6 @@ int kvm_cpu_has_extint(struct kvm_vcpu *v);int kvm_arch_interrupt_allowed (struct kvm_vcpu *vcpu);int kvm_cpu_get_interrupt (struct kvm_vcpu *v);void kvm_vcpu_reset (struct kvm_vcpu *vcpu,bool init_event); -int kvm_pv_send_ipi (struct kvm *kvm,unsigned long ipi_bitmap_low,unsigned long ipi_bitmap_high,u32 min,unsigned long icr,int op_64_bit);
arch/x86/kvm/vmx/vmx.c +4 -4
@@ -6339,7 +6339,7 @@ void vmx_set_virtual_apic_mode(struct kvm_vcpu *vcpu)vmx_update_msr_bitmap_x2apic (vcpu); } -static void vmx_set_apic_access_page_addr (struct kvm_vcpu *vcpu) +static void vmx_reload_apic_access_page (struct kvm_vcpu *vcpu) {struct page *page;
@@ -7777,7 +7777,7 @@ static struct kvm_x86_ops vmx_x86_ops __initdata = { .enable_irq_window = vmx_enable_irq_window, .update_cr8_intercept = vmx_update_cr8_intercept, .set_virtual_apic_mode = vmx_set_virtual_apic_mode, - .set_apic_access_page_addr = vmx_set_apic_access_page_addr, + .reload_apic_pages = vmx_reload_apic_access_page, .refresh_apicv_exec_ctrl = vmx_refresh_apicv_exec_ctrl, .load_eoi_exitmap = vmx_load_eoi_exitmap, .apicv_post_state_restore = vmx_apicv_post_state_restore,
@@ -7940,12 +7940,12 @@ static __init int hardware_setup(void) enable_vnmi =0 ;/* -* set_apic_access_page_addr() is used to reload apic access +* kvm_vcpu_reload_apic_pages() is used to reload apic access * page upon invalidation. No need to do anything if not * using the APIC_ACCESS_ADDR VMCS field. */ if (!flexpriority_enabled) - vmx_x86_ops.set_apic_access_page_addr =NULL ; + vmx_x86_ops.reload_apic_pages =NULL ;if (!cpu_has_vmx_tpr_shadow ()) vmx_x86_ops.update_cr8_intercept =NULL ;
arch/x86/kvm/x86.c +3 -3
@@ -9949,12 +9949,12 @@ void kvm_arch_mmu_notifier_invalidate_range(struct kvm *kvm,kvm_make_all_cpus_request (kvm,KVM_REQ_APIC_PAGE_RELOAD ); } -static void kvm_vcpu_reload_apic_access_page (struct kvm_vcpu *vcpu) +static void kvm_vcpu_reload_apic_pages (struct kvm_vcpu *vcpu) {if (!lapic_in_kernel (vcpu))return ; -static_call_cond (kvm_x86_set_apic_access_page_addr)(vcpu); +static_call_cond (kvm_x86_reload_apic_pages)(vcpu); }void __kvm_request_immediate_exit (struct kvm_vcpu *vcpu)
@@ -10071,7 +10071,7 @@ static int vcpu_enter_guest(struct kvm_vcpu *vcpu)if (kvm_check_request (KVM_REQ_LOAD_EOI_EXITMAP , vcpu))vcpu_load_eoi_exitmap (vcpu);if (kvm_check_request (KVM_REQ_APIC_PAGE_RELOAD , vcpu)) -kvm_vcpu_reload_apic_access_page (vcpu); +kvm_vcpu_reload_apic_pages (vcpu);if (kvm_check_request (KVM_REQ_HV_CRASH , vcpu)) { vcpu->run->exit_reason =KVM_EXIT_SYSTEM_EVENT ; vcpu->run->system_event.type =KVM_SYSTEM_EVENT_CRASH ;
Signature · 1 line
2.26.3
RFC PATCH v2 9/10 KVM: nSVM: implement support for nested AVIC
In reply to Maxim LevitskyThis implements initial support of using the AVIC in a nested guest
--- arch/x86/kvm/svm/avic.c | 850 +++++++++++++++++++++++++++++++++++++- arch/x86/kvm/svm/nested.c | 131 +++++- arch/x86/kvm/svm/svm.c | 18 + arch/x86/kvm/svm/svm.h | 150 +++++++ arch/x86/kvm/trace.h | 140 ++++++- arch/x86/kvm/x86.c | 11 + 6 files changed, 1282 insertions(+), 18 deletions(-)
arch/x86/kvm/svm/avic.c +837 -13
@@ -51,6 +51,526 @@ static u32 next_vm_id = 0;static bool next_vm_id_wrapped =0 ;static DEFINE_SPINLOCK (svm_vm_data_hash_lock); +static u32 nested_avic_get_reg (struct kvm_vcpu *vcpu,int reg_off) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +void *nested_apic_regs = svm->nested.l2_apic_access_page.hva; + +if (WARN_ON_ONCE (!nested_apic_regs)) +return 0 ; + +return *((u32 *) (nested_apic_regs + reg_off)); +} + +static inline struct kvm_vcpu *avic_vcpu_by_l1_apicid (struct kvm *kvm, +int l1_apicid) +{ +WARN_ON (l1_apicid ==-1 ); +return kvm_get_vcpu_by_id (kvm, l1_apicid); +} + +static void avic_physid_shadow_entry_set_vcpu (struct kvm *kvm, +struct avic_physid_table *t, +int n, +int new_l1_apicid) +{ +struct avic_physid_entry_descr *e = &t->entries[n]; +u64 sentry =READ_ONCE (*e->sentry); +u64 old_sentry = sentry; +struct kvm_svm *kvm_svm =to_kvm_svm (kvm); +struct kvm_vcpu *new_vcpu =NULL ; +int l0_apicid =-1 ; +unsigned long flags; + +raw_spin_lock_irqsave (&kvm_svm->avic.table_entries_lock, flags); + +WARN_ON (!test_bit (n, t->valid_entires)); + +if (!list_empty (&e->link)) +list_del_init (&e->link); + +if (new_l1_apicid !=-1 ) + new_vcpu =avic_vcpu_by_l1_apicid (kvm, new_l1_apicid); + +if (new_vcpu) +list_add_tail (&e->link, &to_svm (new_vcpu)->nested.physid_ref_entries); + +if (new_vcpu &&to_svm (new_vcpu)->nested_avic_active) + l0_apicid =kvm_cpu_get_apicid (new_vcpu->cpu); + +physid_entry_set_apicid (&sentry, l0_apicid); + +if (sentry != old_sentry) +WRITE_ONCE (*e->sentry, sentry); + +raw_spin_unlock_irqrestore (&kvm_svm->avic.table_entries_lock, flags); +} + +static void avic_physid_shadow_entry_create (struct kvm *kvm, +struct avic_physid_table *t, +int n, +u64 gentry) +{ +struct avic_physid_entry_descr *e = &t->entries[n]; +struct page *backing_page; +u64 backing_page_gpa =physid_entry_get_backing_table (gentry); +int l1_apic_id =physid_entry_get_apicid (gentry); +hpa_t backing_page_hpa; +u64 sentry =0 ; + + +if (backing_page_gpa ==INVALID_BACKING_PAGE ) +return ; + +/* Pin the APIC backing page */ + backing_page =gfn_to_page (kvm,gpa_to_gfn (backing_page_gpa)); + +if (is_error_page (backing_page)) +/* Invalid GPA in the guest entry - point to a dummy entry */ + backing_page_hpa = t->dummy_page_hpa; +else + backing_page_hpa =page_to_phys (backing_page); + +physid_entry_set_backing_table (&sentry, backing_page_hpa); + + e->gentry = gentry; + *e->sentry = sentry; + +if (test_and_set_bit (n, t->valid_entires)) +WARN_ON (1 ); + +if (backing_page_hpa != t->dummy_page_hpa) +avic_physid_shadow_entry_set_vcpu (kvm, t, n, l1_apic_id); +} + +static void avic_physid_shadow_entry_remove (struct kvm *kvm, +struct avic_physid_table *t, +int n) +{ +struct avic_physid_entry_descr *e = &t->entries[n]; +struct kvm_svm *kvm_svm =to_kvm_svm (kvm); +hpa_t backing_page_hpa; +unsigned long flags; + +raw_spin_lock_irqsave (&kvm_svm->avic.table_entries_lock, flags); + +if (!test_and_clear_bit (n, t->valid_entires)) +WARN_ON (1 ); + +/* Release the APIC backing page */ + backing_page_hpa =physid_entry_get_backing_table (*e->sentry); + +if (backing_page_hpa != t->dummy_page_hpa) +kvm_release_pfn_dirty (backing_page_hpa >>PAGE_SHIFT ); + +if (!list_empty (&e->link)) +list_del_init (&e->link); + + e->gentry =0 ; + *e->sentry =0 ; + +raw_spin_unlock_irqrestore (&kvm_svm->avic.table_entries_lock, flags); +} + +static void avic_update_peer_physid_entries (struct kvm_vcpu *vcpu,int cpu) +{ +/* +* Update all shadow physid tables which contain entries +* which reference this vCPU with its new physical location +*/ +struct kvm_svm *kvm_svm =to_kvm_svm (vcpu->kvm); +struct vcpu_svm *vcpu_svm =to_svm (vcpu); +struct avic_physid_entry_descr *e; +int nentries =0 ; +int l0_apicid =-1 ; +unsigned long flags; +bool new_active = cpu !=-1 ; + +if (vcpu_svm->nested_avic_active == new_active) +return ; + +if (cpu !=-1 ) + l0_apicid =kvm_cpu_get_apicid (cpu); + +raw_spin_lock_irqsave (&kvm_svm->avic.table_entries_lock, flags); + +list_for_each_entry (e, &vcpu_svm->nested.physid_ref_entries, link) { +u64 sentry =READ_ONCE (*e->sentry); +u64 old_sentry = sentry; + +physid_entry_set_apicid (&sentry, l0_apicid); + +if (sentry != old_sentry) +WRITE_ONCE (*e->sentry, sentry); + + nentries++; + } + +if (nentries) +trace_kvm_avic_physid_update_vcpu (vcpu->vcpu_id, cpu, nentries); + + vcpu_svm->nested_avic_active = new_active; + +raw_spin_unlock_irqrestore (&kvm_svm->avic.table_entries_lock, flags); +} + +static bool +avic_physid_shadow_table_setup_write_tracking (struct kvm *kvm, +struct avic_physid_table *t, +bool enable) +{ +struct kvm_memory_slot *slot; + +write_lock (&kvm->mmu_lock); + slot =gfn_to_memslot (kvm, t->gfn); +if (!slot) { +write_unlock (&kvm->mmu_lock); +return false; + } + +if (enable) +kvm_slot_page_track_add_page (kvm, slot, t->gfn,KVM_PAGE_TRACK_WRITE ); +else +kvm_slot_page_track_remove_page (kvm, slot, t->gfn,KVM_PAGE_TRACK_WRITE ); +write_unlock (&kvm->mmu_lock); +return true; +} + +static void +avic_physid_shadow_table_erase (struct kvm *kvm,struct avic_physid_table *t) +{ +int i; + +if (!t->nentries) +return ; + +avic_physid_shadow_table_setup_write_tracking (kvm, t, false); + +for_each_set_bit (i, t->valid_entires,AVIC_MAX_PHYSICAL_ID_COUNT ) +avic_physid_shadow_entry_remove (kvm, t, i); + + t->nentries =0 ; + t->flood_count =0 ; +} + +static struct avic_physid_table * +avic_physid_shadow_table_alloc (struct kvm *kvm,gfn_t gfn) +{ +struct avic_physid_entry_descr *e; +struct avic_physid_table *t; +struct kvm_svm *kvm_svm =to_kvm_svm (kvm); +u64 *shadow_table_address; +int i; + +if (kvm_page_track_write_tracking_enable (kvm)) +return NULL ; + +lockdep_assert_held (&kvm_svm->avic.tables_lock); + + t =kzalloc (sizeof (*t),GFP_KERNEL_ACCOUNT ); +if (!t) +return NULL ; + + t->shadow_table =alloc_page (GFP_KERNEL_ACCOUNT |__GFP_ZERO); +if (!t->shadow_table) + goto err_free_table; + + shadow_table_address =page_address (t->shadow_table); + t->shadow_table_hpa =__sme_set (page_to_phys (t->shadow_table)); + +for (i =0 ; i <ARRAY_SIZE (t->entries); i++) { + e = &t->entries[i]; + e->sentry = &shadow_table_address[i]; + e->gentry =0 ; +INIT_LIST_HEAD (&e->link); + } + + t->gfn = gfn; + t->refcount =1 ; +list_add_tail (&t->link, &kvm_svm->avic.physid_tables); + + t->dummy_page_hpa =page_to_phys (kvm_svm->avic.invalid_physid_page); + +trace_kvm_avic_physid_table_alloc (gfn_to_gpa (gfn)); +return t; + +err_free_table: +kfree (t); +return NULL ; +} + +static void +avic_physid_shadow_table_free (struct kvm *kvm,struct avic_physid_table *t) +{ +struct kvm_svm *kvm_svm =to_kvm_svm (kvm); + +lockdep_assert_held (&kvm_svm->avic.tables_lock); + +WARN_ON (t->refcount); + +avic_physid_shadow_table_erase (kvm, t); + +trace_kvm_avic_physid_table_free (gfn_to_gpa (t->gfn)); + +hlist_del (&t->hash_link); +list_del (&t->link); +__free_page (t->shadow_table); +kfree (t); +} + +static struct avic_physid_table * +__avic_physid_shadow_table_get (struct hlist_head *head,gfn_t gfn) +{ +struct avic_physid_table *t; + +hlist_for_each_entry (t, head, hash_link) +if (t->gfn == gfn) { + t->refcount++; +return t; + } +return NULL ; +} + +struct avic_physid_table * +avic_physid_shadow_table_get (struct kvm_vcpu *vcpu,gfn_t gfn) +{ +struct kvm_svm *kvm_svm =to_kvm_svm (vcpu->kvm); +struct hlist_head *hlist; +struct avic_physid_table *t; + +mutex_lock (&kvm_svm->avic.tables_lock); + + hlist = &kvm_svm->avic.physid_gpa_hash[avic_physid_hash (gfn)]; + t =__avic_physid_shadow_table_get (hlist, gfn); +if (!t) { + t =avic_physid_shadow_table_alloc (vcpu->kvm, gfn); +if (!t) + goto out_unlock; +hlist_add_head (&t->hash_link, hlist); + } + t->flood_count =0 ; +out_unlock: +mutex_unlock (&kvm_svm->avic.tables_lock); +return t; +} + +static void +__avic_physid_shadow_table_put (struct kvm *kvm,struct avic_physid_table *t) +{ +WARN_ON (t->refcount <=0 ); +if (--t->refcount ==0 ) +avic_physid_shadow_table_free (kvm, t); +} + +void avic_physid_shadow_table_put (struct kvm *kvm,struct avic_physid_table *t) +{ +struct kvm_svm *kvm_svm =to_kvm_svm (kvm); + +mutex_lock (&kvm_svm->avic.tables_lock); +__avic_physid_shadow_table_put (kvm, t); +mutex_unlock (&kvm_svm->avic.tables_lock); +} + +static void avic_physid_shadow_table_invalidate (struct kvm *kvm, +struct avic_physid_table *t) +{ +avic_physid_shadow_table_erase (kvm, t); +kvm_make_all_cpus_request (kvm,KVM_REQ_APIC_PAGE_RELOAD ); +} + +int avic_physid_shadow_table_sync (struct kvm_vcpu *vcpu, +struct avic_physid_table *t,int nentries) +{ +struct kvm_svm *kvm_svm =to_kvm_svm (vcpu->kvm); +struct kvm_host_map map; +u64 *gentries; +int i; +int ret =0 ; + +mutex_lock (&kvm_svm->avic.tables_lock); + +if (t->nentries >= nentries) + goto out_unlock; + +trace_kvm_avic_physid_table_reload (gfn_to_gpa (t->gfn), t->nentries, nentries); + +if (t->nentries ==0 ) { +if (!avic_physid_shadow_table_setup_write_tracking (vcpu->kvm, t, true)) { + ret = -EFAULT ; + goto out_unlock; + } + } + +if (kvm_vcpu_map (vcpu, t->gfn, &map)) { + ret = -EFAULT ; + goto out_unlock; + } + + gentries = (u64 *)map.hva; + +for (i = t->nentries ; i < nentries ; i++) +avic_physid_shadow_entry_create (vcpu->kvm, t, i, gentries[i]); + +/* publish the table before setting nentries */ +wmb (); +WRITE_ONCE (t->nentries, nentries); + +kvm_vcpu_unmap (vcpu, &map, false); +out_unlock: +mutex_unlock (&kvm_svm->avic.tables_lock); +return ret; +} + +static void avic_physid_shadow_table_track_write (struct kvm_vcpu *vcpu, +gpa_t gpa, +const u8 *new, +int bytes, +struct kvm_page_track_notifier_node *node) +{ +struct kvm_svm *kvm_svm =to_kvm_svm (vcpu->kvm); +struct hlist_head *hlist; +struct avic_physid_table *t; +gfn_t gfn =gpa_to_gfn (gpa); +unsigned int page_offset =offset_in_page (gpa); +unsigned int entry_offset = page_offset &0x7 ; +int first = page_offset /sizeof (u64); +int last = (page_offset + bytes -1 ) /sizeof (u64); +u64 new_entry, old_entry; +int l1_apic_id; + +if (WARN_ON_ONCE (bytes ==0 )) +return ; + +mutex_lock (&kvm_svm->avic.tables_lock); + + hlist = &kvm_svm->avic.physid_gpa_hash[avic_physid_hash (gfn)]; + t =__avic_physid_shadow_table_get (hlist, gfn); + +if (!t) + goto out_unlock; + +trace_kvm_avic_physid_table_write (gpa, bytes); + +/* +* Update policy: +* +* Only a write to a single entry, entry that had a valid backing page +* on the last VM entry with this page, and only if the +* write touches only the is_running and/or apic_id part of this entry +* is allowed. +* +* Writes outside of known number of entries are ignored to support +* case when the guest is adding entries to end of the page +* in the process of a cpu hotplug. +* +* All other writes, which are not supposed to happen during +* use of the page, cause the page to be invalidated, +* and read as a whole, next time it is used by a vCPU for VM entry. +*/ + +if (first >= t->nentries) + goto out_table_put; + +if (first != last || !test_bit (first, t->valid_entires)) + goto invalidate; + +/* update the entry with written bytes */ + old_entry = t->entries[first].gentry; + new_entry = old_entry; +memcpy (((u8 *)&new_entry) + entry_offset, new, bytes); + +/* if backing page changed, invalidate the whole page*/ +if (physid_entry_get_backing_table (old_entry) != +physid_entry_get_backing_table (new_entry)) + goto invalidate; + +if (++t->flood_count > t->nentries *AVIC_PHYSID_FLOOD_COUNT ) + goto invalidate; + +/* Update the backing cpu */ + l1_apic_id =physid_entry_get_apicid (new_entry); +avic_physid_shadow_entry_set_vcpu (vcpu->kvm, t, first, l1_apic_id); + t->entries[first].gentry = new_entry; + goto out_table_put; +invalidate: +avic_physid_shadow_table_invalidate (vcpu->kvm, t); +out_table_put: +__avic_physid_shadow_table_put (vcpu->kvm, t); +out_unlock: +mutex_unlock (&kvm_svm->avic.tables_lock); +} + +static void avic_physid_shadow_table_flush_memslot (struct kvm *kvm, +struct kvm_memory_slot *slot, +struct kvm_page_track_notifier_node *node) +{ +struct kvm_svm *kvm_svm =to_kvm_svm (kvm); +struct avic_physid_table *t, *n; +int i; + +mutex_lock (&kvm_svm->avic.tables_lock); + +list_for_each_entry_safe (t, n, &kvm_svm->avic.physid_tables, link) { + +if (gfn_in_memslot (slot, t->gfn)) { +avic_physid_shadow_table_invalidate (kvm, t); +continue ; + } + +for_each_set_bit (i, t->valid_entires,AVIC_MAX_PHYSICAL_ID_COUNT ) { +u64 gentry = t->entries[i].gentry; +gpa_t gpa =physid_entry_get_backing_table (gentry); + +if (gfn_in_memslot (slot,gpa_to_gfn (gpa))) { +avic_physid_shadow_table_invalidate (kvm, t); +break ; + } + } + } +mutex_unlock (&kvm_svm->avic.tables_lock); +} + +bool avic_nested_has_interrupt (struct kvm_vcpu *vcpu) +{ +int off; + +if (!nested_avic_in_use (vcpu)) +return false; + +for (off =0x10 ; off <0x80 ; off +=0x10 ) +if (nested_avic_get_reg (vcpu,APIC_IRR + off)) +return true; +return false; +} + +void avic_reload_apic_pages (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *vcpu_svm =to_svm (vcpu); +struct avic_physid_table *t = vcpu_svm->nested.l2_physical_id_table; + +int nentries = vcpu_svm->nested.ctl.avic_physical_id & +AVIC_PHYSICAL_ID_TABLE_SIZE_MASK ; + +if (t &&is_guest_mode (vcpu) &&nested_avic_in_use (vcpu)) +avic_physid_shadow_table_sync (vcpu, t, nentries); +} + +void avic_free_nested (struct kvm_vcpu *vcpu) +{ +struct avic_physid_table *t; +struct vcpu_svm *svm =to_svm (vcpu); + + t = svm->nested.l2_physical_id_table; +if (t) { +avic_physid_shadow_table_put (vcpu->kvm, t); + svm->nested.l2_physical_id_table =NULL ; + } + +kvm_vcpu_unmap (vcpu, &svm->nested.l2_apic_access_page, true); +kvm_vcpu_unmap (vcpu, &svm->nested.l2_logical_id_table, true); +} +/* * This is a wrapper of struct amd_iommu_ir_data. */
@@ -105,26 +625,38 @@ void avic_vm_destroy(struct kvm *kvm) {unsigned long flags;struct kvm_svm_avic *avic = &to_kvm_svm (kvm)->avic; +unsigned long i; +struct kvm_vcpu *vcpu;if (!enable_apicv)return ; +kvm_for_each_vcpu (i, vcpu, kvm) { +vcpu_load (vcpu); +avic_free_nested (vcpu); +vcpu_put (vcpu); + } +if (avic->logical_id_table_page)__free_page (avic->logical_id_table_page);if (avic->physical_id_table_page)__free_page (avic->physical_id_table_page); +if (avic->invalid_physid_page) +__free_page (avic->invalid_physid_page);spin_lock_irqsave (&svm_vm_data_hash_lock, flags);hash_del (&avic->hnode);spin_unlock_irqrestore (&svm_vm_data_hash_lock, flags); + + +kvm_page_track_unregister_notifier (kvm, &avic->write_tracker); }int avic_vm_init (struct kvm *kvm) {unsigned long flags;int err = -ENOMEM ; -struct page *p_page; -struct page *l_page; +struct page *page;struct kvm_svm_avic *avic = &to_kvm_svm (kvm)->avic;u32 vm_id;
@@ -132,18 +664,26 @@ int avic_vm_init(struct kvm *kvm)return 0 ;/* Allocating physical APIC ID table (4KB) */ - p_page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO); -if (!p_page) + page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO); +if (!page) goto free_avic; - avic->physical_id_table_page = p_page; + avic->physical_id_table_page = page;/* Allocating logical APIC ID table (4KB) */ - l_page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO); -if (!l_page) + page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO); +if (!page) + goto free_avic; + + avic->logical_id_table_page = page; + + page =alloc_page (GFP_KERNEL_ACCOUNT | __GFP_ZERO); +if (!page) goto free_avic; - avic->logical_id_table_page = l_page; +/* Allocating dummy page for invalid nested avic physid entries */ + avic->invalid_physid_page = page; +spin_lock_irqsave (&svm_vm_data_hash_lock, flags); again:
@@ -165,6 +705,14 @@ int avic_vm_init(struct kvm *kvm)hash_add (svm_vm_data_hash, &avic->hnode, avic->vm_id);spin_unlock_irqrestore (&svm_vm_data_hash_lock, flags); +raw_spin_lock_init (&avic->table_entries_lock); +mutex_init (&avic->tables_lock); +INIT_LIST_HEAD (&avic->physid_tables); + + avic->write_tracker.track_write = avic_physid_shadow_table_track_write; + avic->write_tracker.track_flush_slot = avic_physid_shadow_table_flush_memslot; + +kvm_page_track_register_notifier (kvm, &avic->write_tracker);return 0 ; free_avic:
@@ -316,6 +864,161 @@ static void avic_kick_target_vcpus(struct kvm *kvm, struct kvm_lapic *source, } } +static void +avic_kick_target_vcpu_nested_physical (struct vcpu_svm *svm, +int target_l2_apic_id, +int *index, +bool *invalid_page) +{ +u64 gentry, sentry; +int target_l1_apicid; +struct avic_physid_table *t = svm->nested.l2_physical_id_table; + +if (WARN_ON_ONCE (!t)) +return ; + +/* +* This shouldn't normally happen as such condition +* should cause AVIC_IPI_FAILURE_INVALID_TARGET vmexit, +* however guest can change the page under us. +*/ +if (target_l2_apic_id >= t->nentries) +return ; + + gentry = t->entries[target_l2_apic_id].gentry; + sentry = *t->entries[target_l2_apic_id].sentry; + +/* Same reasoning as above */ +if (!(gentry &AVIC_PHYSICAL_ID_ENTRY_VALID_MASK )) +return ; + +/* +* This races against the guest updating is_running bit. +* Race itself happens on real hardware as well, and the guest +* should use correct means to avoid it. +* +* AVIC hardware already set IRR and should have done memory +* barrier, and then found that shadowed is_running is false. +* We are doing another is_running check, completing it, +* thus don't need additional memory barriers +*/ + + target_l1_apicid =physid_entry_get_apicid (gentry); + +if (target_l1_apicid ==-1 ) { + +/* is_running is false, need to vmexit to the guest */ +if (*index ==-1 ) { +u64 backing_page_phys =physid_entry_get_backing_table (sentry); + + *index = target_l2_apic_id; +if (backing_page_phys == t->dummy_page_hpa) + *invalid_page = true; + } + }else { +/* Wake up the target vCPU and hide the VM exit from the guest */ +struct kvm_vcpu *target =avic_vcpu_by_l1_apicid (svm->vcpu.kvm, target_l1_apicid); + +if (target && target != &svm->vcpu) +kvm_vcpu_wake_up (target); + } + +trace_kvm_avic_nested_kick_vcpu (svm->vcpu.vcpu_id, + target_l2_apic_id, + target_l1_apicid); +} + +static void +avic_kick_target_vcpus_nested_logical (struct vcpu_svm *svm,unsigned long dest, +int *index,bool *invalid_page) +{ +int logical_id; +u8 cluster =0 ; +u64 *logical_id_table = (u64 *)svm->nested.l2_logical_id_table.hva; +int physical_index =-1 ; + +if (WARN_ON_ONCE (!logical_id_table)) +return ; + +if (nested_avic_get_reg (&svm->vcpu,APIC_DFR ) ==APIC_DFR_CLUSTER ) { +if (dest >=0x40 ) +return ; + cluster = dest &0x3C ; + dest &=0x3 ; + } + +for_each_set_bit (logical_id, &dest,8 ) { +int logical_index = cluster | logical_id; +u64 log_gentry = logical_id_table[logical_index]; +int l2_apicid =logid_get_physid (log_gentry); + +/* Should not happen as in this case AVIC should VM exit +* with 'invalid target' + +* However the guest can change the entry under KVM's back, +* thus ignore this case. +*/ +if (l2_apicid ==-1 ) +continue ; + +avic_kick_target_vcpu_nested_physical (svm, l2_apicid, + &physical_index, + invalid_page); + +/* Reported index is the index of the logical entry in this case */ +if (physical_index !=-1 ) + *index = logical_index; + } +} + +static void +avic_kick_target_vcpus_nested_broadcast (struct vcpu_svm *svm, +int *index,bool *invalid_page) +{ +struct avic_physid_table *t = svm->nested.l2_physical_id_table; +int l2_apicid; + +/* +* This races against guest changing valid bit in the table and/or +* increasing nentries of the table. +* In both cases the race would happen on real hardware as well +* thus there is no need to take locks. +*/ +for_each_set_bit (l2_apicid, t->valid_entires,AVIC_MAX_PHYSICAL_ID_COUNT ) +avic_kick_target_vcpu_nested_physical (svm, l2_apicid, + index, invalid_page); +} + + +static void avic_kick_target_vcpus_nested (struct kvm_vcpu *vcpu, +struct kvm_lapic *source, +u32 icrl,u32 icrh, +int *index,bool *invalid_page) +{ +struct vcpu_svm *svm =to_svm (vcpu); +int dest =GET_APIC_DEST_FIELD (icrh); + +switch (icrl &APIC_SHORT_MASK ) { +case APIC_DEST_NOSHORT : +if (dest ==0xFF ) +avic_kick_target_vcpus_nested_broadcast (svm, + index, invalid_page); +else if (icrl &APIC_DEST_MASK ) +avic_kick_target_vcpus_nested_logical (svm, dest, + index, invalid_page); +else +avic_kick_target_vcpu_nested_physical (svm, dest, + index, invalid_page); +break ; +case APIC_DEST_ALLINC : +case APIC_DEST_ALLBUT : +avic_kick_target_vcpus_nested_broadcast (svm, index, invalid_page); +break ; +case APIC_DEST_SELF : +break ; + } +} +int avic_incomplete_ipi_interception (struct kvm_vcpu *vcpu) {struct vcpu_svm *svm =to_svm (vcpu);
@@ -323,10 +1026,20 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)u32 icrl = svm->vmcb->control.exit_info_1;u32 id = svm->vmcb->control.exit_info_2 >>32 ;u32 index = svm->vmcb->control.exit_info_2 &0xFF ; +int nindex =-1 ; +bool invalid_page = false; +struct kvm_lapic *apic = vcpu->arch.apic;trace_kvm_avic_incomplete_ipi (vcpu->vcpu_id, icrh, icrl, id, index); +if (is_guest_mode (&svm->vcpu)) { +if (WARN_ON_ONCE (!nested_avic_in_use (vcpu))) +return 1 ; +if (WARN_ON_ONCE (!svm->nested.l2_physical_id_table)) +return 1 ; + } +switch (id) {case AVIC_IPI_FAILURE_INVALID_INT_TYPE :/*
@@ -338,23 +1051,49 @@ int avic_incomplete_ipi_interception(struct kvm_vcpu *vcpu)* which case KVM needs to emulate the ICR write as well in * order to clear the BUSY flag. */ +if (is_guest_mode (&svm->vcpu)) { +nested_svm_vmexit (svm); +break ; + } +if (icrl &APIC_ICR_BUSY )kvm_apic_write_nodecode (vcpu,APIC_ICR );else kvm_apic_send_ipi (apic, icrl, icrh); +break ;case AVIC_IPI_FAILURE_TARGET_NOT_RUNNING :/* * At this point, we expect that the AVIC HW has already * set the appropriate IRR bits on the valid target * vcpus. So, we just need to kick the appropriate vcpu. +* +* If nested we might also need to reflect the VM exit to +* the guest */ -avic_kick_target_vcpus (vcpu->kvm, apic, icrl, icrh); +if (!is_guest_mode (&svm->vcpu)) { +avic_kick_target_vcpus (vcpu->kvm, apic, icrl, icrh); +break ; + } + +avic_kick_target_vcpus_nested (vcpu, apic, icrl, icrh, + &nindex, &invalid_page); +if (nindex !=-1 ) { +if (invalid_page) + id =AVIC_IPI_FAILURE_INVALID_BACKING_PAGE ; + + svm->vmcb->control.exit_info_2 = ((u64 )id <<32 ) | nindex; +nested_svm_vmexit (svm); + }break ;case AVIC_IPI_FAILURE_INVALID_TARGET : +if (is_guest_mode (&svm->vcpu)) +nested_svm_vmexit (svm); +else +WARN_ON_ONCE (1 );break ;case AVIC_IPI_FAILURE_INVALID_BACKING_PAGE : -WARN_ONCE (1 ,"Invalid backing page\n" ); +WARN_ON_ONCE (1 );break ;default :pr_err ("Unknown IPI interception\n" );
@@ -370,6 +1109,48 @@ unsigned long avic_vcpu_get_apicv_inhibit_reasons(struct kvm_vcpu *vcpu)return 0 ; } +int avic_emulate_doorbell_write (struct kvm_vcpu *vcpu,u64 data) +{ +int source_l1_apicid = vcpu->vcpu_id; +int target_l1_apicid = data &AVIC_DOORBELL_PHYSICAL_ID_MASK ; +bool target_running, target_nested; +struct kvm_vcpu *target; + +if (data & ~AVIC_DOORBELL_PHYSICAL_ID_MASK ) +return 1 ; + + target =avic_vcpu_by_l1_apicid (vcpu->kvm, target_l1_apicid); +if (!target) +/* Guest bug: targeting invalid APIC ID. */ +return 0 ; + + target_running =READ_ONCE (target->mode) ==IN_GUEST_MODE ; + target_nested =is_guest_mode (target); + +trace_kvm_avic_nested_doorbell (source_l1_apicid, target_l1_apicid, + target_nested, target_running); + +/* +* Target is not in nested mode, thus doorbell doesn't affect it +* if it became just now nested now, +* it means that it processed the doorbell on entry +*/ +if (!target_nested) +return 0 ; + +/* +* If the target vCPU is in guest mode, kick the real doorbell. +* Otherwise we need to wake it up in case it is not scheduled to run. +*/ +if (target_running) +wrmsr (MSR_AMD64_SVM_AVIC_DOORBELL , +kvm_cpu_get_apicid (READ_ONCE (target->cpu)),0 ); +else +kvm_vcpu_wake_up (target); + +return 0 ; +} +static u32 *avic_get_logical_id_entry (struct kvm_vcpu *vcpu,u32 ldr,bool flat) {struct kvm_svm *kvm_svm =to_kvm_svm (vcpu->kvm);
@@ -463,9 +1244,13 @@ static void avic_handle_dfr_update(struct kvm_vcpu *vcpu)static int avic_unaccel_trap_write (struct kvm_vcpu *vcpu) { +struct vcpu_svm *svm =to_svm (vcpu);u32 offset =to_svm (vcpu)->vmcb->control.exit_info_1 &AVIC_UNACCEL_ACCESS_OFFSET_MASK ; +if (WARN_ON_ONCE (is_guest_mode (&svm->vcpu))) +return 0 ; +switch (offset) {case APIC_LDR :if (avic_handle_ldr_update (vcpu))
@@ -523,6 +1308,8 @@ int avic_unaccelerated_access_interception(struct kvm_vcpu *vcpu)AVIC_UNACCEL_ACCESS_WRITE_MASK ;bool trap =is_avic_unaccelerated_access_trap (offset); +WARN_ON_ONCE (is_guest_mode (&svm->vcpu)); +trace_kvm_avic_unaccelerated_access (vcpu->vcpu_id, offset, trap, write, vector);if (trap) {
@@ -908,18 +1695,51 @@ static void avic_vcpu_load(struct kvm_vcpu *vcpu)int cpu =get_cpu ();WARN_ON (cpu != vcpu->cpu); -__avic_vcpu_load (vcpu, cpu); -put_cpu (); }static void avic_vcpu_put (struct kvm_vcpu *vcpu) {preempt_disable (); -__avic_vcpu_put (vcpu); +preempt_enable (); +} + + +void __nested_avic_load (struct kvm_vcpu *vcpu,int cpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +lockdep_assert_preemption_disabled (); + +if (svm->nested.initialized && svm->avic_enabled) +avic_update_peer_physid_entries (vcpu, cpu); +} + +void __nested_avic_put (struct kvm_vcpu *vcpu) +{ +struct vcpu_svm *svm =to_svm (vcpu); + +lockdep_assert_preemption_disabled (); + +if (svm->nested.initialized && svm->avic_enabled) +avic_update_peer_physid_entries (vcpu,-1 ); +} + +void nested_avic_load (struct kvm_vcpu *vcpu) +{ +int cpu =get_cpu (); + +WARN_ON (cpu != vcpu->cpu); +__nested_avic_load (vcpu, cpu); +put_cpu (); +} +void nested_avic_put (struct kvm_vcpu *vcpu) +{ +preempt_disable (); +__nested_avic_put (vcpu);preempt_enable (); }
@@ -983,3 +1803,7 @@ void avic_vcpu_unblocking(struct kvm_vcpu *vcpu)avic_vcpu_load (vcpu); } + +/* +* TODO: Deal with AVIC errata in regard to flushing TLB on vCPU change +*/
arch/x86/kvm/svm/nested.c +127 -4
216 lines not on this page · raw message
arch/x86/kvm/svm/svm.c +18 -0
@@ -1301,6 +1301,8 @@ static int svm_vcpu_create(struct kvm_vcpu *vcpu) svm->guest_state_loaded = false; +INIT_LIST_HEAD (&svm->nested.physid_ref_entries); +return 0 ; error_free_vmsa_page:
@@ -1390,8 +1392,11 @@ static void svm_vcpu_load(struct kvm_vcpu *vcpu, int cpu) sd->current_vmcb = svm->vmcb;indirect_branch_prediction_barrier (); } +if (kvm_vcpu_apicv_active (vcpu))__avic_vcpu_load (vcpu, cpu); + +__nested_avic_load (vcpu, cpu); }static void svm_vcpu_put (struct kvm_vcpu *vcpu)
@@ -1399,6 +1404,8 @@ static void svm_vcpu_put(struct kvm_vcpu *vcpu)if (kvm_vcpu_apicv_active (vcpu))__avic_vcpu_put (vcpu); +__nested_avic_put (vcpu); +svm_prepare_host_switch (vcpu); ++vcpu->stat.host_state_reload;
@@ -2764,6 +2771,8 @@ static int svm_set_msr(struct kvm_vcpu *vcpu, struct msr_data *msr)u32 ecx = msr->index;u64 data = msr->data;switch (ecx) { +case MSR_AMD64_SVM_AVIC_DOORBELL : +return avic_emulate_doorbell_write (vcpu, data);case MSR_AMD64_TSC_RATIO :if (!svm->tsc_scaling_enabled) {
@@ -4060,6 +4069,9 @@ static void svm_vcpu_after_set_cpuid(struct kvm_vcpu *vcpu)if (guest_cpuid_has (vcpu,X86_FEATURE_X2APIC ))kvm_set_apicv_inhibit (kvm,APICV_INHIBIT_REASON_X2APIC ); } + + svm->avic_enabled = enable_apicv &&guest_cpuid_has (vcpu,X86_FEATURE_AVIC ); +init_vmcb_after_set_cpuid (vcpu); }
@@ -4669,9 +4681,11 @@ static struct kvm_x86_ops svm_x86_ops __initdata = { .enable_nmi_window = svm_enable_nmi_window, .enable_irq_window = svm_enable_irq_window, .update_cr8_intercept = svm_update_cr8_intercept, + .reload_apic_pages = avic_reload_apic_pages, .refresh_apicv_exec_ctrl = avic_refresh_apicv_exec_ctrl, .check_apicv_inhibit_reasons = avic_check_apicv_inhibit_reasons, .apicv_post_state_restore = avic_apicv_post_state_restore, + .guest_apic_has_interrupt = avic_nested_has_interrupt, .get_mt_mask = svm_get_mt_mask, .get_exit_info = svm_get_exit_info,
@@ -4798,6 +4812,9 @@ static __init void svm_set_cpu_caps(void)if (vgif)kvm_cpu_cap_set (X86_FEATURE_VGIF ); +if (enable_apicv) +kvm_cpu_cap_set (X86_FEATURE_AVIC ); +/* Nested VM can receive #VMEXIT instead of triggering #GP */ kvm_cpu_cap_set (X86_FEATURE_SVME_ADDR_CHK ); }
@@ -4923,6 +4940,7 @@ static __init int svm_hardware_setup(void) svm_x86_ops.vcpu_blocking =NULL ; svm_x86_ops.vcpu_unblocking =NULL ; svm_x86_ops.vcpu_get_apicv_inhibit_reasons =NULL ; + svm_x86_ops.guest_apic_has_interrupt =NULL ; }if (vls) {
arch/x86/kvm/svm/svm.h +150 -0
204 lines not on this page · raw message
arch/x86/kvm/trace.h +139 -1
152 lines not on this page · raw message
arch/x86/kvm/x86.c +11 -0
@@ -13063,9 +13063,20 @@ EXPORT_TRACEPOINT_SYMBOL_GPL(kvm_write_tsc_offset);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_ple_window_update);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_pml_full);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_pi_irte_update); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_unaccelerated_access);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_incomplete_ipi);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_ga_log); + +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_physid_table_alloc); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_physid_table_free); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_physid_table_reload); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_physid_table_write); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_physid_update_vcpu); + +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_nested_doorbell); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_avic_nested_kick_vcpu); +EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_apicv_accept_irq);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_enter);EXPORT_TRACEPOINT_SYMBOL_GPL (kvm_vmgexit_exit);
Signature · 1 line
2.26.3
RFC PATCH v2 10/10 KVM: SVM: allow to avoid not needed updates to is_running
In reply to Maxim LevitskyAllow optionally to make KVM not update is_running unless it is functionally needed which is only when a vCPU halts, or is in the guest mode.
This means security wise that if a vCPU is scheduled out, other vCPUs could still send doorbell messages to the last physical CPU where this vCPU was last running.
If a malicious guest tries to do it can slow down the victim CPU by about 40% in my testing, so this should only be enabled if physical CPUs are not shared among guests.
The option is avic_doorbell_strict and is true by
default, setting it to false allows this relaxed
non strict mode.
--- arch/x86/kvm/svm/avic.c | 19 ++++++++++++------- arch/x86/kvm/svm/svm.c | 19 ++++++++++++++----- arch/x86/kvm/svm/svm.h | 1 + 3 files changed, 27 insertions(+), 12 deletions(-)
arch/x86/kvm/svm/avic.c +12 -7
@@ -1641,7 +1641,7 @@ avic_update_iommu_vcpu_affinity(struct kvm_vcpu *vcpu, int cpu, bool r)void __avic_vcpu_load (struct kvm_vcpu *vcpu,int cpu) { -u64 entry; +u64 old_entry, new_entry;int h_physical_id =kvm_cpu_get_apicid (cpu);struct vcpu_svm *svm =to_svm (vcpu);
@@ -1660,14 +1660,16 @@ void __avic_vcpu_load(struct kvm_vcpu *vcpu, int cpu)if (kvm_vcpu_is_blocking (vcpu))return ; - entry =READ_ONCE (*(svm->avic_physical_id_cache)); -WARN_ON (entry &AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK ); + old_entry =READ_ONCE (*(svm->avic_physical_id_cache)); + new_entry = old_entry; - entry &= ~AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK ; - entry |= (h_physical_id &AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK ); - entry |=AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK ; + new_entry &= ~AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK ; + new_entry |= (h_physical_id &AVIC_PHYSICAL_ID_ENTRY_HOST_PHYSICAL_ID_MASK ); + new_entry |=AVIC_PHYSICAL_ID_ENTRY_IS_RUNNING_MASK ; + +if (old_entry != new_entry) +WRITE_ONCE (*(svm->avic_physical_id_cache), new_entry); -WRITE_ONCE (*(svm->avic_physical_id_cache), entry);avic_update_iommu_vcpu_affinity (vcpu, h_physical_id, true); }
@@ -1777,6 +1779,9 @@ void avic_refresh_apicv_exec_ctrl(struct kvm_vcpu *vcpu)void avic_vcpu_blocking (struct kvm_vcpu *vcpu) { +if (!avic_doorbell_strict) +__nested_avic_put (vcpu); +if (!kvm_vcpu_apicv_active (vcpu))return ;
arch/x86/kvm/svm/svm.c +14 -5
@@ -190,6 +190,10 @@ module_param(avic, bool, 0444);static bool force_avic;module_param_unsafe (force_avic, bool,0444 ); +bool avic_doorbell_strict = true; +module_param (avic_doorbell_strict, bool,0444 ); + +bool __read_mostly dump_invalid_vmcb;module_param (dump_invalid_vmcb, bool,0644 );
@@ -1395,16 +1399,21 @@ static void svm_vcpu_load(struct kvm_vcpu *vcpu, int cpu)if (kvm_vcpu_apicv_active (vcpu))__avic_vcpu_load (vcpu, cpu); -__nested_avic_load (vcpu, cpu); }static void svm_vcpu_put (struct kvm_vcpu *vcpu) { -if (kvm_vcpu_apicv_active (vcpu)) -__avic_vcpu_put (vcpu); - -__nested_avic_put (vcpu); +/* +* Forbid AVIC's peers to send interrupts +* to this CPU unless we are in non strict mode, +* in which case, we will do so only when this vCPU blocks +*/ +if (avic_doorbell_strict) { +if (kvm_vcpu_apicv_active (vcpu)) +__avic_vcpu_put (vcpu); +__nested_avic_put (vcpu); + }svm_prepare_host_switch (vcpu);
arch/x86/kvm/svm/svm.h +1 -0
@@ -36,6 +36,7 @@ extern u32 msrpm_offsets[MSRPM_OFFSETS] __read_mostly;extern bool npt_enabled;extern int vgif;extern bool intercept_smi; +extern bool avic_doorbell_strict;/* * Clean bits in VMCB.
Signature · 1 line
2.26.3
Patchwork
CI[Intel-gfx] ✗ Fi.CI.BUILD: failure for RFCv2: nested AVIC
In reply to Maxim Levitsky== Series Details ==
== Series Details ==
Series: RFCv2: nested AVIC
URL : https://patchwork.freedesktop.org/series/102905/
State : failure
== Summary ==
Error: patch https://patchwork.freedesktop.org/api/1.0/series/102905/revisions/1/mbox/ not applied
Applying: KVM: x86: mmu: allow to enable write tracking externally
Applying: x86: KVMGT: use kvm_page_track_write_tracking_enable
Applying: KVM: x86: mmu: add gfn_in_memslot helper
Applying: KVM: x86: mmu: tweak fast path for emulation of access to nested NPT pages
error: sha1 information is lacking or useless (arch/x86/kvm/mmu/mmu.c).
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0004 KVM: x86: mmu: tweak fast path for emulation of access to nested NPT pages When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".