X86/KVM/Hyper-V: Add HV ept tlb range list flush support in KVM
Subsystems (11)
v2 cover X86/KVM/Hyper-V: Add HV ept tlb range list flush support in KVM
From: Lan Tianyu <[email protected]>
This patchset is to introduce hv ept tlb range list flush function support in the KVM MMU component. Flushing ept tlbs of several address range can be done via single hypercall and new list flush function is used in the kvm_mmu_commit_zap_page() and FNAME(sync_page). This patchset also adds more hv ept tlb range flush support in more KVM MMU function.
Change since v1:
1) Make flush list as a hlist instead of list in order to
keep struct kvm_mmu_page size.
2) Add last_level flag in the struct kvm_mmu_page instead
of spte pointer
3) Move tlb flush from kvm_mmu_notifier_clear_flush_young() to kvm_age_hva()
4) Use range flush in the kvm_vm_ioctl_get/clear_dirty_log()
Lan Tianyu (10):
X86/Hyper-V: Add parameter offset for
hyperv_fill_flush_guest_mapping_list()
KVM/VMX: Fill range list in kvm_fill_hv_flush_list_func() KVM/MMU: Add last_level in the struct mmu_spte_page KVM/MMU: Introduce tlb flush with range list
KVM/MMU: Flush tlb with range list in sync_page() KVM/MMU: Flush tlb directly in the kvm_mmu_slot_gfn_write_protect()
KVM: Add kvm_get_memslot() to get memslot via slot id
KVM: Use tlb range flush in the kvm_vm_ioctl_get/clear_dirty_log()
KVM: Add flush parameter for kvm_age_hva()
KVM/MMU: Use tlb range flush in the kvm_age_hva()
arch/arm/include/asm/kvm_host.h | 3 ++- arch/arm64/include/asm/kvm_host.h | 3 ++- arch/mips/include/asm/kvm_host.h | 3 ++- arch/mips/kvm/mmu.c | 11 ++++++-- arch/powerpc/include/asm/kvm_host.h | 3 ++- arch/powerpc/kvm/book3s.c | 10 ++++++-- arch/powerpc/kvm/e500_mmu_host.c | 3 ++- arch/x86/hyperv/nested.c | 4 +-- arch/x86/include/asm/kvm_host.h | 11 +++++++- arch/x86/include/asm/mshyperv.h | 2 +- arch/x86/kvm/mmu.c | 51 +++++++++++++++++++++++++++++-------- arch/x86/kvm/mmu.h | 7 +++++ arch/x86/kvm/paging_tmpl.h | 15 ++++++++--- arch/x86/kvm/vmx/vmx.c | 18 +++++++++++-- arch/x86/kvm/x86.c | 16 +++++++++--- include/linux/kvm_host.h | 1 + virt/kvm/arm/mmu.c | 13 ++++++++-- virt/kvm/kvm_main.c | 51 +++++++++++++++---------------------- 18 files changed, 160 insertions(+), 65 deletions(-)
Signature · 1 line
2.14.4
PATCH v2 1/10 X86/Hyper-V: Add parameter offset for hyperv_fill_flush_guest_mapping_list()
From: Lan Tianyu <[email protected]>
Add parameter offset to specify start position to add flush ranges in guest address list of struct hv_guest_mapping_flush_list.
---arch/x86/hyperv/nested.c | 4 ++--
arch/x86/include/asm/mshyperv.h | 2 +- arch/x86/kvm/vmx/vmx.c | 2 +- 3 files changed, 4 insertions(+), 4 deletions(-)
arch/x86/hyperv/nested.c +2 -2
@@ -58,11 +58,11 @@ EXPORT_SYMBOL_GPL(hyperv_flush_guest_mapping);int hyperv_fill_flush_guest_mapping_list (struct hv_guest_mapping_flush_list *flush, -u64 start_gfn,u64 pages) +int offset,u64 start_gfn,u64 pages) {u64 cur = start_gfn;u64 additional_pages; -int gpa_n =0 ; +int gpa_n = offset; do { /*
arch/x86/include/asm/mshyperv.h +1 -1
@@ -357,7 +357,7 @@ int hyperv_flush_guest_mapping_range(u64 as, hyperv_fill_flush_list_func fill_func, void *data);int hyperv_fill_flush_guest_mapping_list (struct hv_guest_mapping_flush_list *flush, -u64 start_gfn,u64 end_gfn); +int offset,u64 start_gfn,u64 end_gfn);#ifdef CONFIG_X86_64 void hv_apic_init (void );
arch/x86/kvm/vmx/vmx.c +1 -1
@@ -428,7 +428,7 @@ int kvm_fill_hv_flush_list_func(struct hv_guest_mapping_flush_list *flush, {struct kvm_tlb_range *range = data; -return hyperv_fill_flush_guest_mapping_list (flush, range->start_gfn, +return hyperv_fill_flush_guest_mapping_list (flush,0 , range->start_gfn, range->pages); }
Signature · 1 line
2.14.4
PATCH v2 2/10 KVM/VMX: Fill range list in kvm_fill_hv_flush_list_func()
In reply to [email protected]From: Lan Tianyu <[email protected]>
Populate ranges on the flush list into struct hv_guest_mapping_flush_list when flush list is available in the struct kvm_tlb_range.
---
Change since v1:
Make flush list as a "hlist" instead of a "list" in order to keep struct kvm_mmu_page size.
arch/x86/include/asm/kvm_host.h | 7 +++++++
arch/x86/kvm/vmx/vmx.c | 18 ++++++++++++++++-- 2 files changed, 23 insertions(+), 2 deletions(-)
arch/x86/include/asm/kvm_host.h +7 -0
@@ -317,6 +317,12 @@ struct kvm_rmap_head {struct kvm_mmu_page {struct list_head link; + +/* +* Tlb flush with range list uses struct kvm_mmu_page as list entry +* and all list operations should be under protection of mmu_lock. +*/ +struct hlist_node flush_link;struct hlist_node hash_link;bool unsync;
@@ -443,6 +449,7 @@ struct kvm_mmu {struct kvm_tlb_range {u64 start_gfn;u64 pages; +struct hlist_head *flush_list; };enum pmc_type {
arch/x86/kvm/vmx/vmx.c +16 -2
@@ -427,9 +427,23 @@ int kvm_fill_hv_flush_list_func(struct hv_guest_mapping_flush_list *flush, void *data) {struct kvm_tlb_range *range = data; +struct kvm_mmu_page *sp; -return hyperv_fill_flush_guest_mapping_list (flush,0 , range->start_gfn, - range->pages); +if (!range->flush_list) { +return hyperv_fill_flush_guest_mapping_list (flush, +0 , range->start_gfn, range->pages); + }else { +int offset =0 ; + +hlist_for_each_entry (sp, range->flush_list, flush_link) { +int pages =KVM_PAGES_PER_HPAGE (sp->role.level); + + offset =hyperv_fill_flush_guest_mapping_list (flush, + offset, sp->gfn, pages); + } + +return offset; + } } static inline int __hv_remote_flush_tlb_with_range(struct kvm *kvm,
Signature · 1 line
2.14.4
PATCH v2 3/10 KVM/MMU: Add last_level in the struct mmu_spte_page
In reply to [email protected]From: Lan Tianyu <[email protected]>
This patch is to add last_level in the struct kvm_mmu_page. When build flush tlb range list, last_level will be used to identify whehter the page should be added into list.
--- arch/x86/include/asm/kvm_host.h | 1 + arch/x86/kvm/mmu.c | 3 +++ 2 files changed, 4 insertions(+)
arch/x86/include/asm/kvm_host.h +1 -0
@@ -325,6 +325,7 @@ struct kvm_mmu_page {struct hlist_node flush_link;struct hlist_node hash_link;bool unsync; +bool last_level;/* * The following two entries are used to key the shadow page in the
arch/x86/kvm/mmu.c +3 -0
@@ -2918,6 +2918,9 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,if (level >PT_PAGE_TABLE_LEVEL ) spte |=PT_PAGE_SIZE_MASK ; + + sp->last_level =is_last_spte (spte, level); +if (tdp_enabled) spte |= kvm_x86_ops->get_mt_mask (vcpu, gfn,kvm_is_mmio_pfn (pfn));
Signature · 1 line
2.14.4
Paolo Bonzini
maintainerRe: [PATCH V2 3/10] KVM/MMU: Add last_level in the struct mmu_spte_page
arch/x86/kvm/mmu.c static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep, · line 2922
Quoted 10 lines · On 02/02/19 02:38, [email protected] wrote:
arch/x86/kvm/mmu.c +2
-0@@ -2918,6 +2918,9 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,if (level >PT_PAGE_TABLE_LEVEL ) spte |=PT_PAGE_SIZE_MASK ; + + sp->last_level =is_last_spte (spte, level);
sp->last_level is always true here.
Paolo
Quoted 3 lines
arch/x86/kvm/mmu.c +0
-0if (tdp_enabled) spte |= kvm_x86_ops->get_mt_mask (vcpu, gfn,kvm_is_mmio_pfn (pfn));
Paolo Bonzini
maintainerRe: [PATCH V2 3/10] KVM/MMU: Add last_level in the struct mmu_spte_page
In reply to [email protected]arch/x86/kvm/mmu.c static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep, · line 2922
Quoted 10 lines · On 02/02/19 02:38, [email protected] wrote:
arch/x86/kvm/mmu.c +2
-0@@ -2918,6 +2918,9 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,if (level >PT_PAGE_TABLE_LEVEL ) spte |=PT_PAGE_SIZE_MASK ; + + sp->last_level =is_last_spte (spte, level);
Wait, I wasn't thinking straight. If a struct kvm_mmu_page exists, it is never the last level. Page table entries for the last level do not have a struct kvm_mmu_page.
Therefore you don't need the flag after all. I suspect your calculations in patch 2 are off by one, and you actually need
hlist_for_each_entry(sp, range->flush_list, flush_link) {
int pages = KVM_PAGES_PER_HPAGE(sp->role.level + 1);
...
}For example, if sp->role.level is 1 then the struct kvm_mmu_page is for a page containing PTEs and covers an area of 2 MiB.
Thanks,
Paolo
Quoted 3 lines
arch/x86/kvm/mmu.c +0
-0if (tdp_enabled) spte |= kvm_x86_ops->get_mt_mask (vcpu, gfn,kvm_is_mmio_pfn (pfn));
Re: [PATCH V2 3/10] KVM/MMU: Add last_level in the struct mmu_spte_page
Quoted 20 lines · Paolo Bonzini
arch/x86/kvm/mmu.c static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep, · line 2922
Quoted 10 lines · On 02/02/19 02:38, [email protected] wrote:
arch/x86/kvm/mmu.c +2
-0@@ -2918,6 +2918,9 @@ static int set_spte(struct kvm_vcpu *vcpu, u64 *sptep,if (level >PT_PAGE_TABLE_LEVEL ) spte |=PT_PAGE_SIZE_MASK ; + + sp->last_level =is_last_spte (spte, level);Wait, I wasn't thinking straight. If a struct
kvm_mmu_pageexists, it is never the last level. Page table entries for the last level do not have a structkvm_mmu_page.Therefore you don't need the flag after all. I suspect your calculations in patch 2 are off by one, and you actually need
Quoted 7 lines
hlist_for_each_entry(sp, range->flush_list, flush_link) { int pages = KVM_PAGES_PER_HPAGE(sp->role.level + 1); ... }For example, if
sp->role.levelis 1 then the structkvm_mmu_pageis for a page containing PTEs and covers an area of 2 MiB.
Yes, you are right. Thanks to point out and will fix. The last_level flag is to avoid adding middle page node(e.g, PGD, PMD) into flush list. The address range will be duplicated if adding both leaf, node and middle node into flush list.
Quoted 9 lines
Thanks,
Paolo
Quoted 3 lines
arch/x86/kvm/mmu.c +0
-0if (tdp_enabled) spte |= kvm_x86_ops->get_mt_mask (vcpu, gfn,kvm_is_mmio_pfn (pfn));
Signature · 2 lines
Best regards Tianyu Lan
Paolo Bonzini
maintainerRe: [PATCH V2 3/10] KVM/MMU: Add last_level in the struct mmu_spte_page
Quoted 4 lines · Tianyu Lan
Yes, you are right. Thanks to point out and will fix. The
last_levelflag is to avoid adding middle pagenode(e.g, PGD, PMD)into flush list. The address range will be duplicated if adding both leaf, node and middle node into flush list.
Hmm, that's not easy to track. One kvm_mmu_page could include both leaf and non-leaf page (for example a huge page for 0 to 2 MB and a page table for 2 MB to 4 MB).
Is this really needed? First, your benchmarks so far have been done with sp->last_level always set to true. Second, you would only encounter this optimization in kvm_mmu_commit_zap_page when zapping a 1 GB region (which then would be invalidated twice, at both the PMD and PGD level) or bigger.
Paolo
Re: [PATCH V2 3/10] KVM/MMU: Add last_level in the struct mmu_spte_page
Quoted 11 lines · Paolo Bonzini
Quoted 4 lines · Tianyu Lan
Yes, you are right. Thanks to point out and will fix. The
last_levelflag is to avoid adding middle pagenode(e.g, PGD, PMD)into flush list. The address range will be duplicated if adding both leaf, node and middle node into flush list.Hmm, that's not easy to track. One
kvm_mmu_pagecould include both leaf and non-leaf page (for example a huge page for 0 to 2 MB and a page table for 2 MB to 4 MB).
Quoted 7 lines
Is this really needed? First, your benchmarks so far have been done with
sp->last_levelalways set to true. Second, you would only encounter this optimization inkvm_mmu_commit_zap_pagewhen zapping a 1 GB region (which then would be invalidated twice, at both the PMD and PGD level) or bigger.Paolo
Hi Paolo:
Sorry for later response and I tried to figure out a bug
lead by defining wrong
max flush count. I just sent out V3. I still put the last_level flag patch in the end of patchset.
Detail please see the change log. Just like you said this was an optimization and wasn't 100%
required. If you still have some concerns, you can ignore it and other patches in this patchset
should be good. Thanks.
Signature · 2 lines
Best regards Tianyu Lan
PATCH v2 4/10 KVM/MMU: Introduce tlb flush with range list
In reply to [email protected]From: Lan Tianyu <[email protected]>
This patch is to introduce tlb flush with range list interface and use struct kvm_mmu_page as list entry. Use flush list function in the kvm_mmu_commit_zap_page().
--- arch/x86/kvm/mmu.c | 25 ++++++++++++++++++++++++- 1 file changed, 24 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu.c +24 -1
@@ -289,6 +289,20 @@ static void kvm_flush_remote_tlbs_with_address(struct kvm *kvm, range.start_gfn = start_gfn; range.pages = pages; + range.flush_list =NULL ; + +kvm_flush_remote_tlbs_with_range (kvm, &range); +} + +static void kvm_flush_remote_tlbs_with_list (struct kvm *kvm, +struct hlist_head *flush_list) +{ +struct kvm_tlb_range range; + +if (hlist_empty (flush_list)) +return ; + + range.flush_list = flush_list;kvm_flush_remote_tlbs_with_range (kvm, &range); }
@@ -2708,6 +2722,7 @@ static void kvm_mmu_commit_zap_page(struct kvm *kvm, struct list_head *invalid_list) {struct kvm_mmu_page *sp, *nsp; +HLIST_HEAD (flush_list);if (list_empty (invalid_list))return ;
@@ -2721,7 +2736,15 @@ static void kvm_mmu_commit_zap_page(struct kvm *kvm,* In addition, kvm_flush_remote_tlbs waits for all vcpus to exit * guest mode and/or lockless shadow page table walks. */ -kvm_flush_remote_tlbs (kvm); +if (kvm_available_flush_tlb_with_range ()) { +list_for_each_entry (sp, invalid_list, link) +if (sp->last_level) +hlist_add_head (&sp->flush_link, &flush_list); + +kvm_flush_remote_tlbs_with_list (kvm, &flush_list); + }else { +kvm_flush_remote_tlbs (kvm); + }list_for_each_entry_safe (sp, nsp, invalid_list, link) {WARN_ON (!sp->role.invalid || sp->root_count);
Signature · 1 line
2.14.4
PATCH v2 5/10 KVM/MMU: Flush tlb with range list in sync_page()
In reply to [email protected]From: Lan Tianyu <[email protected]>
This patch is to flush tlb via flush list function. Put page into flush list when return value of set_spte() includes flag SET_SPTE_NEED_REMOTE_TLB_FLUSH. kvm_flush_remote_ tlbs_with_list() checks whether the flush list is empty or not. It also checks whether range tlb flush is available and go back to tradiion flush if not.
---
Change since v1:
Use check of list_empty in the kvm_flush_remote_tlbs_with_list() to determine flush or not instead of checking set_spte_ret.
arch/x86/kvm/paging_tmpl.h | 15 +++++++++++----
1 file changed, 11 insertions(+), 4 deletions(-)
arch/x86/kvm/paging_tmpl.h +11 -4
@@ -970,7 +970,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp)int i, nr_present =0 ;bool host_writable;gpa_t first_pte_gpa; -int set_spte_ret =0 ; +HLIST_HEAD (flush_list);/* direct kvm_mmu_page can not be unsync. */ BUG_ON (sp->role.direct);
@@ -978,6 +978,7 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) first_pte_gpa =FNAME (get_level1_sp_gpa)(sp);for (i =0 ; i <PT64_ENT_PER_PAGE ; i++) { +int set_spte_ret =0 ;unsigned pte_access;pt_element_t gpte;gpa_t pte_gpa;
@@ -1027,14 +1028,20 @@ static int FNAME(sync_page)(struct kvm_vcpu *vcpu, struct kvm_mmu_page *sp) host_writable = sp->spt[i] &SPTE_HOST_WRITEABLE ; - set_spte_ret |=set_spte (vcpu, &sp->spt[i], + set_spte_ret =set_spte (vcpu, &sp->spt[i], pte_access,PT_PAGE_TABLE_LEVEL , gfn,spte_to_pfn (sp->spt[i]), true, false, host_writable); + +if (set_spte_ret &SET_SPTE_NEED_REMOTE_TLB_FLUSH ) { +struct kvm_mmu_page *leaf_sp =page_header (sp->spt[i] + &PT64_BASE_ADDR_MASK ); +hlist_add_head (&leaf_sp->flush_link, &flush_list); + } + } -if (set_spte_ret &SET_SPTE_NEED_REMOTE_TLB_FLUSH ) -kvm_flush_remote_tlbs (vcpu->kvm); +kvm_flush_remote_tlbs_with_list (vcpu->kvm, &flush_list);return nr_present; }
Signature · 1 line
2.14.4
PATCH v2 6/10 KVM/MMU: Flush tlb directly in the kvm_mmu_slot_gfn_write_protect()
In reply to [email protected]From: Lan Tianyu <[email protected]>
This patch is to flush tlb directly in the kvm_mmu_slot_gfn_write_protect() when range flush is available.
--- arch/x86/kvm/mmu.c | 5 +++++ 1 file changed, 5 insertions(+)
arch/x86/kvm/mmu.c +5 -0
@@ -1718,6 +1718,11 @@ bool kvm_mmu_slot_gfn_write_protect(struct kvm *kvm, write_protected |=__rmap_write_protect (kvm, rmap_head, true); } +if (write_protected &&kvm_available_flush_tlb_with_range ()) { +kvm_flush_remote_tlbs_with_address (kvm, gfn,1 ); + write_protected = false; + } +return write_protected; }
Signature · 1 line
2.14.4
PATCH v2 7/10 KVM: Add kvm_get_memslot() to get memslot via slot id
In reply to [email protected]From: Lan Tianyu <[email protected]>
This patch is to add kvm_get_memslot() to get struct kvm_memory_slot via slot it and remove redundant codes. The function will also be used in the following changes.
--- include/linux/kvm_host.h | 1 + virt/kvm/kvm_main.c | 45 +++++++++++++++++++-------------------------- 2 files changed, 20 insertions(+), 26 deletions(-)
include/linux/kvm_host.h +1 -0
@@ -758,6 +758,7 @@ int kvm_get_dirty_log_protect(struct kvm *kvm, struct kvm_dirty_log *log, bool *flush);int kvm_clear_dirty_log_protect (struct kvm *kvm,struct kvm_clear_dirty_log *log,bool *flush); +struct kvm_memory_slot *kvm_get_memslot (struct kvm *kvm,u32 slot); void kvm_arch_mmu_enable_log_dirty_pt_masked(struct kvm *kvm, struct kvm_memory_slot *slot,
virt/kvm/kvm_main.c +19 -26
85 lines not on this page · raw message
Signature · 1 line
2.14.4
PATCH v2 8/10 KVM: Use tlb range flush in the kvm_vm_ioctl_get/clear_dirty_log()
In reply to [email protected]From: Lan Tianyu <[email protected]>
This patch is to use tlb range flush to flush memslot's in the kvm_vm_ioctl_get/clear_dirty_log() instead of flushing tlbs of entire ept page table when range flush is available.
--- arch/x86/kvm/mmu.c | 8 +------- arch/x86/kvm/mmu.h | 7 +++++++ arch/x86/kvm/x86.c | 16 ++++++++++++---- 3 files changed, 20 insertions(+), 11 deletions(-)
arch/x86/kvm/mmu.c +1 -7
@@ -264,12 +264,6 @@ static void mmu_spte_set(u64 *sptep, u64 spte);static union kvm_mmu_page_role kvm_mmu_calc_root_page_role (struct kvm_vcpu *vcpu); - -static inline bool kvm_available_flush_tlb_with_range (void ) -{ -return kvm_x86_ops->tlb_remote_flush_with_range; -} -static void kvm_flush_remote_tlbs_with_range (struct kvm *kvm,struct kvm_tlb_range *range) {
@@ -282,7 +276,7 @@ static void kvm_flush_remote_tlbs_with_range(struct kvm *kvm,kvm_flush_remote_tlbs (kvm); } -static void kvm_flush_remote_tlbs_with_address (struct kvm *kvm, +void kvm_flush_remote_tlbs_with_address (struct kvm *kvm,u64 start_gfn,u64 pages) {struct kvm_tlb_range range;
arch/x86/kvm/mmu.h +7 -0
@@ -63,6 +63,13 @@ void kvm_init_shadow_ept_mmu(struct kvm_vcpu *vcpu, bool execonly,bool kvm_can_do_async_pf (struct kvm_vcpu *vcpu);int kvm_handle_page_fault (struct kvm_vcpu *vcpu,u64 error_code,u64 fault_address,char *insn,int insn_len); +void kvm_flush_remote_tlbs_with_address (struct kvm *kvm, +u64 start_gfn,u64 pages); + +static inline bool kvm_available_flush_tlb_with_range (void ) +{ +return kvm_x86_ops->tlb_remote_flush_with_range; +}static inline unsigned int kvm_mmu_available_pages (struct kvm *kvm) {
arch/x86/kvm/x86.c +12 -4
@@ -4445,9 +4445,13 @@ int kvm_vm_ioctl_get_dirty_log(struct kvm *kvm, struct kvm_dirty_log *log)* kvm_mmu_slot_remove_write_access(). */ lockdep_assert_held (&kvm->slots_lock); -if (flush) -kvm_flush_remote_tlbs (kvm); +if (flush) { +struct kvm_memory_slot *memslot =kvm_get_memslot (kvm, + log->slot); +kvm_flush_remote_tlbs_with_address (kvm, memslot->base_gfn, + memslot->npages); + }mutex_unlock (&kvm->slots_lock);return r; }
@@ -4472,9 +4476,13 @@ int kvm_vm_ioctl_clear_dirty_log(struct kvm *kvm, struct kvm_clear_dirty_log *lo* kvm_mmu_slot_remove_write_access(). */ lockdep_assert_held (&kvm->slots_lock); -if (flush) -kvm_flush_remote_tlbs (kvm); +if (flush) { +struct kvm_memory_slot *memslot =kvm_get_memslot (kvm, + log->slot); +kvm_flush_remote_tlbs_with_address (kvm, memslot->base_gfn, + memslot->npages); + }mutex_unlock (&kvm->slots_lock);return r; }
Signature · 1 line
2.14.4
PATCH v2 9/10 KVM: Add flush parameter for kvm_age_hva()
In reply to [email protected]From: Lan Tianyu <[email protected]>
This patch is to add flush parameter for kvm_aga_hva() and move tlb flush from kvm_mmu_notifier_clear_flush_young() to kvm_age_hva(). kvm_age_hva() can check whether tlb flush is necessary when return value young is more than 0. Flush tlb if both conditions are met.
--- arch/arm/include/asm/kvm_host.h | 3 ++- arch/arm64/include/asm/kvm_host.h | 3 ++- arch/mips/include/asm/kvm_host.h | 3 ++- arch/mips/kvm/mmu.c | 11 +++++++++-- arch/powerpc/include/asm/kvm_host.h | 3 ++- arch/powerpc/kvm/book3s.c | 10 ++++++++-- arch/powerpc/kvm/e500_mmu_host.c | 3 ++- arch/x86/include/asm/kvm_host.h | 3 ++- arch/x86/kvm/mmu.c | 10 ++++++++-- virt/kvm/arm/mmu.c | 13 +++++++++++-- virt/kvm/kvm_main.c | 6 ++---- 11 files changed, 50 insertions(+), 18 deletions(-)
arch/arm/include/asm/kvm_host.h +2 -1
@@ -229,7 +229,8 @@ int kvm_set_spte_hva(struct kvm *kvm, unsigned long hva, pte_t pte);unsigned long kvm_arm_num_regs (struct kvm_vcpu *vcpu);int kvm_arm_copy_reg_indices (struct kvm_vcpu *vcpu,u64 __user *indices); -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end); +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush);int kvm_test_age_hva (struct kvm *kvm,unsigned long hva);struct kvm_vcpu *kvm_arm_get_running_vcpu (void );
arch/arm64/include/asm/kvm_host.h +2 -1
@@ -361,7 +361,8 @@ int __kvm_arm_vcpu_set_events(struct kvm_vcpu *vcpu,int kvm_unmap_hva_range (struct kvm *kvm,unsigned long start,unsigned long end);int kvm_set_spte_hva (struct kvm *kvm,unsigned long hva,pte_t pte); -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end); +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush);int kvm_test_age_hva (struct kvm *kvm,unsigned long hva);struct kvm_vcpu *kvm_arm_get_running_vcpu (void );
arch/mips/include/asm/kvm_host.h +2 -1
@@ -937,7 +937,8 @@ enum kvm_mips_fault_result kvm_trap_emul_gva_fault(struct kvm_vcpu *vcpu,int kvm_unmap_hva_range (struct kvm *kvm,unsigned long start,unsigned long end);int kvm_set_spte_hva (struct kvm *kvm,unsigned long hva,pte_t pte); -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end); +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush);int kvm_test_age_hva (struct kvm *kvm,unsigned long hva);/* Emulation */
arch/mips/kvm/mmu.c +9 -2
@@ -579,9 +579,16 @@ static int kvm_test_age_hva_handler(struct kvm *kvm, gfn_t gfn, gfn_t gfn_end,return pte_young (*gpa_pte); } -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end) +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush) { -return handle_hva_to_gpa (kvm, start, end, kvm_age_hva_handler,NULL ); +int young =handle_hva_to_gpa (kvm, start, end, + kvm_age_hva_handler,NULL ); + +if (flush && young >0 ) +kvm_flush_remote_tlbs (kvm); + +return young; }int kvm_test_age_hva (struct kvm *kvm,unsigned long hva)
arch/powerpc/include/asm/kvm_host.h +2 -1
@@ -70,7 +70,8 @@extern int kvm_unmap_hva_range (struct kvm *kvm,unsigned long start,unsigned long end); -extern int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end); +extern int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush);extern int kvm_test_age_hva (struct kvm *kvm,unsigned long hva);extern int kvm_set_spte_hva (struct kvm *kvm,unsigned long hva,pte_t pte);
arch/powerpc/kvm/book3s.c +8 -2
@@ -841,9 +841,15 @@ int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end)return kvm->arch.kvm_ops->unmap_hva_range (kvm, start, end); } -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end) +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush) { -return kvm->arch.kvm_ops->age_hva (kvm, start, end); +int young = kvm->arch.kvm_ops->age_hva (kvm, start, end); + +if (flush && young >0 ) +kvm_flush_remote_tlbs (kvm); + +return young; }int kvm_test_age_hva (struct kvm *kvm,unsigned long hva)
arch/powerpc/kvm/e500_mmu_host.c +2 -1
@@ -745,7 +745,8 @@ int kvm_unmap_hva_range(struct kvm *kvm, unsigned long start, unsigned long end)return 0 ; } -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end) +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush) {/* XXX could be more clever ;) */ return 0 ;
arch/x86/include/asm/kvm_host.h +2 -1
@@ -1519,7 +1519,8 @@ asmlinkage void kvm_spurious_fault(void);#define KVM_ARCH_WANT_MMU_NOTIFIER int kvm_unmap_hva_range (struct kvm *kvm,unsigned long start,unsigned long end); -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end); +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush);int kvm_test_age_hva (struct kvm *kvm,unsigned long hva);int kvm_set_spte_hva (struct kvm *kvm,unsigned long hva,pte_t pte);int kvm_cpu_has_injectable_intr (struct kvm_vcpu *v);
arch/x86/kvm/mmu.c +8 -2
@@ -1977,9 +1977,15 @@ static void rmap_recycle(struct kvm_vcpu *vcpu, u64 *spte, gfn_t gfn) KVM_PAGES_PER_HPAGE(sp->role.level)); } -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end) +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush) { -return kvm_handle_hva_range (kvm, start, end,0 , kvm_age_rmapp); +int young =kvm_handle_hva_range (kvm, start, end,0 , kvm_age_rmapp); + +if (flush && young >0 ) +kvm_flush_remote_tlbs (kvm); + +return young; }int kvm_test_age_hva (struct kvm *kvm,unsigned long hva)
virt/kvm/arm/mmu.c +11 -2
@@ -2107,12 +2107,21 @@ static int kvm_test_age_hva_handler(struct kvm *kvm, gpa_t gpa, u64 size, void *return pte_young (*pte); } -int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end) +int kvm_age_hva (struct kvm *kvm,unsigned long start,unsigned long end, +bool flush) { +int young; +if (!kvm->arch.pgd)return 0 ;trace_kvm_age_hva (start, end); -return handle_hva_to_gpa (kvm, start, end, kvm_age_hva_handler,NULL ); + + young =handle_hva_to_gpa (kvm, start, end, kvm_age_hva_handler,NULL ); + +if (flush && young >0 ) +kvm_flush_remote_tlbs (kvm); + +return young; }int kvm_test_age_hva (struct kvm *kvm,unsigned long hva)
virt/kvm/kvm_main.c +2 -4
@@ -428,9 +428,7 @@ static int kvm_mmu_notifier_clear_flush_young(struct mmu_notifier *mn, idx =srcu_read_lock (&kvm->srcu);spin_lock (&kvm->mmu_lock); - young =kvm_age_hva (kvm, start, end); -if (young) -kvm_flush_remote_tlbs (kvm); + young =kvm_age_hva (kvm, start, end, true);spin_unlock (&kvm->mmu_lock);srcu_read_unlock (&kvm->srcu, idx);
@@ -461,7 +459,7 @@ static int kvm_mmu_notifier_clear_young(struct mmu_notifier *mn,* cadence. If we find this inaccurate, we might come up with a * more sophisticated heuristic later. */ - young =kvm_age_hva (kvm, start, end); + young =kvm_age_hva (kvm, start, end, false);spin_unlock (&kvm->mmu_lock);srcu_read_unlock (&kvm->srcu, idx);
Signature · 1 line
2.14.4
Paolo Bonzini
maintainerQuoted 8 lines · On 02/02/19 02:38, [email protected] wrote:
From: Lan Tianyu <[email protected]>
This patchset is to introduce hv ept tlb range list flush function support in the KVM MMU component. Flushing ept tlbs of several address range can be done via single hypercall and new list flush function is used in the
kvm_mmu_commit_zap_page()andFNAME(sync_page). This patchset also adds more hv ept tlb range flush support in more KVM MMU function.
Quoted 7 lines
Change since v1:
1) Make flush list as a hlist instead of list in order to
keep structkvm_mmu_pagesize.
2) Addlast_levelflag in the structkvm_mmu_pageinstead
of spte pointer
3) Move tlb flush fromkvm_mmu_notifier_clear_flush_young()tokvm_age_hva()
4) Use range flush in the kvm_vm_ioctl_get/clear_dirty_log()
Looks good except for the confusion on sp->last_level (which was mostly mine---sorry about that). I think it can still make 5.1.
However, note that I've never received "KVM/MMU: Use tlb range flush in the kvm_age_hva()".
Paolo