KVM: x86/mmu: move reused pages to the top of active_mmu_pages
PATCH 1/1 KVM: x86/mmu: move reused pages to the top of active_mmu_pages
Move reused shadow pages to the head of active_mmu_pages in __kvm_mmu_get_shadow_page(). This will allow us to move towards more of a LRU approximation eviction strategy instead of just straight FIFO.
--- arch/x86/kvm/mmu/mmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/x86/kvm/mmu/mmu.c +2 -1
@@ -2395,7 +2395,8 @@ static struct kvm_mmu_page *__kvm_mmu_get_shadow_page(struct kvm *kvm,if (!sp) { created = true; sp =kvm_mmu_alloc_shadow_page (kvm, caches, gfn, sp_list, role); - } + }else if (!list_is_head (&sp->link, &kvm->arch.active_mmu_pages)) +list_move (&sp->link, &kvm->arch.active_mmu_pages);trace_kvm_mmu_get_page (sp, created);return sp;
Signature · 1 line
2.52.0
Sean Christopherson
maintainerQuoted 3 lines · Hamza Mahfooz
Move reused shadow pages to the head of
active_mmu_pagesin__kvm_mmu_get_shadow_page(). This will allow us to move towards more of a LRU approximation eviction strategy instead of just straight FIFO.
Does this actually have a (positive) impact on real-world workloads? It seems like an obvious improvment, but there's enough subtlely around active_mmu_pages that I don't want to make any changes without a strong benefit.
Specifically, kvm_zap_obsolete_pages() has a hard dependency on the list being FIFO. We _might_ be ok if we make sure to filter out obsolete pages, but only because of KVM's behavior of (a) only allowing two memslot generations at any given time and (b) zapping all shadow pages from the old/obsolete generation prior to kvm_zap_obsolete_pages() exiting.
But it most definitely makes me nervous.
arch/x86/kvm/mmu/mmu.c +10 -0
@@ -2327,6 +2327,16 @@ static struct kvm_mmu_page *kvm_mmu_find_shadow_page(struct kvm *kvm,if (collisions > kvm->stat.max_mmu_page_hash_collisions) kvm->stat.max_mmu_page_hash_collisions = collisions; + +/* +* If a shadow page was found, move it to the head of the active pages +* as a rudimentary form of LRU-reclaim (KVM reclaims shadow pages from +* tail=>head if the VM hits the limit on the number of MMU pages). +* */ +if (sp && !WARN_ON_ONCE (is_obsolete_sp (kvm, sp)) && + !list_is_head (&sp->link, &kvm->arch.active_mmu_pages)) +list_move (&sp->link, &kvm->arch.active_mmu_pages); +return sp; }
Quoted 10 lines
Signed-off-by Hamza Mahfooz <[email protected]>--- arch/x86/kvm/mmu/mmu.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)arch/x86/kvm/mmu/mmu.c +0
-0
arch/x86/kvm/mmu/mmu.c static struct kvm_mmu_page *__kvm_mmu_get_shadow_page(struct kvm *kvm, · line 2399
Quoted 6 lines
arch/x86/kvm/mmu/mmu.c +2
-1@@ -2395,7 +2395,8 @@ static struct kvm_mmu_page *__kvm_mmu_get_shadow_page(struct kvm *kvm,if (!sp) { created = true; sp =kvm_mmu_alloc_shadow_page (kvm, caches, gfn, sp_list, role); - } + }else if (!list_is_head (&sp->link, &kvm->arch.active_mmu_pages)) +list_move (&sp->link, &kvm->arch.active_mmu_pages);
As alluded to above, I think I'd prefer to put this in kvm_mmu_find_shadow_page()? Largely a moot point, but it seems like we'd want to move a page to the head of the list if we look it up for any reason.
Quoted 4 lines · Sean Christopherson
Does this actually have a (positive) impact on real-world workloads? It seems like an obvious improvment, but there's enough subtlely around
active_mmu_pagesthat I don't want to make any changes without a strong benefit.
My testing mostly focused on correctness, though I did see the fault rate go down on a long running VM that I used to host a web server (I only only gave it around a gig of RAM, so it is on the more extreme end).
Quoted 5 lines
Specifically,
kvm_zap_obsolete_pages()has a hard dependency on the list being FIFO. We _might_ be ok if we make sure to filter out obsolete pages, but only because of KVM's behavior of (a) only allowing two memslot generations at any given time and (b) zapping all shadow pages from the old/obsolete generation prior tokvm_zap_obsolete_pages()exiting.
for_each_valid_sp() already filters out obsolete pages, so we should be good to go from that perspective.
Quoted 3 lines
As alluded to above, I think I'd prefer to put this in
kvm_mmu_find_shadow_page()? Largely a moot point, but it seems like we'd want to move a page to the head of the list if we look it up for any reason.
Sounds good to me, I put it in __kvm_mmu_get_shadow_page() since it seemed clearer and it's currently the only caller of kvm_mmu_find_shadow_page().