KVM: x86/mmu: Don't put invalid SPs back on the list of active pages
PATCH 1/1 KVM: x86/mmu: Don't put invalid SPs back on the list of active pages
Delete a shadow page from the invalidation list instead of throwing it back on the list of active pages when it's a root shadow page with active users. Invalid active root pages will be explicitly freed by mmu_free_root_page() when the root_count hits zero, i.e. they don't need to be put on the active list to avoid leakage.
Use sp->role.invalid to detect that a shadow page has already been zapped, i.e. is not on a list.
WARN if an invalid page is encountered when zapping pages, as it should now be impossible.
--- arch/x86/kvm/mmu/mmu.c | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
arch/x86/kvm/mmu/mmu.c +10 -8
@@ -2757,10 +2757,13 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm,if (!sp->root_count) {/* Count self */ (*nr_zapped)++; -list_move (&sp->link, invalid_list); +if (sp->role.invalid) +list_add (&sp->link, invalid_list); +else +list_move (&sp->link, invalid_list);kvm_mod_used_mmu_pages (kvm,-1 ); }else { -list_move (&sp->link, &kvm->arch.active_mmu_pages); +list_del (&sp->link);/* * Obsolete pages cannot be used on any vCPUs, see the comment
@@ -5732,12 +5735,11 @@ static void kvm_zap_obsolete_pages(struct kvm *kvm)break ;/* -* Skip invalid pages with a non-zero root count, zapping pages -* with a non-zero root count will never succeed, i.e. the page -* will get thrown back on active_mmu_pages and we'll get stuck -* in an infinite loop. +* Invalid pages should never land back on the list of active +* pages. Skip the bogus page, otherwise we'll get stuck in an +* infinite loop if the page gets put back on the list (again). */ -if (sp->role.invalid && sp->root_count) +if (WARN_ON (sp->role.invalid))continue ;/*
@@ -6015,7 +6017,7 @@ void kvm_mmu_zap_all(struct kvm *kvm)spin_lock (&kvm->mmu_lock); restart:list_for_each_entry_safe (sp, node, &kvm->arch.active_mmu_pages, link) { -if (sp->role.invalid && sp->root_count) +if (WARN_ON (sp->role.invalid))continue ;if (__kvm_mmu_prepare_zap_page (kvm, sp, &invalid_list, &ign)) goto restart;
Signature · 1 line
2.26.0
Paolo Bonzini
maintainerarch/x86/kvm/mmu/mmu.c static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm, · line 2763
Quoted 13 lines · Sean Christopherson
arch/x86/kvm/mmu/mmu.c +4
-1@@ -2757,10 +2757,13 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm,if (!sp->root_count) {/* Count self */ (*nr_zapped)++; -list_move (&sp->link, invalid_list); +if (sp->role.invalid) +list_add (&sp->link, invalid_list); +else +list_move (&sp->link, invalid_list);
It's late here, but I think this part needs a comment anyway...
Paolo
Quoted 16 lines · Paolo Bonzini
arch/x86/kvm/mmu/mmu.c static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm, · line 2763
Quoted 13 lines · Sean Christopherson
arch/x86/kvm/mmu/mmu.c +4
-1@@ -2757,10 +2757,13 @@ static bool __kvm_mmu_prepare_zap_page(struct kvm *kvm,if (!sp->root_count) {/* Count self */ (*nr_zapped)++; -list_move (&sp->link, invalid_list); +if (sp->role.invalid) +list_add (&sp->link, invalid_list); +else +list_move (&sp->link, invalid_list);It's late here, but I think this part needs a comment anyway...
No argument here. I'll spin a v2, I just realized there is a separate optimization that can build on this patch. I was planning on sending it separately, but I misread the loop in make_mmu_pages_available().