Subject: [RFC PATCH] kvm/x86: Fix 'lpages' kvm stat for TDM MMU
Large pages not being created properly may result in increased memory access time. The 'lpages' kvm stat used to keep track of the current number of large pages in the system, but with TDP MMU enabled the stat is not showing the correct number.
This patch extends the lpages counter to cover the TDP case.
--- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++++++ 1 file changed, 6 insertions(+)
arch/x86/kvm/mmu/tdp_mmu.c +6 -0
@@ -425,6 +425,12 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn,if (old_spte == new_spte)return ; + +if (is_large_pte (old_spte)) + --kvm->stat.lpages; + +if (is_large_pte (new_spte)) + ++kvm->stat.lpages;trace_kvm_tdp_mmu_spte_changed (as_id, gfn, level, old_spte, new_spte);
Signature · 11 lines
2.17.1 Amazon Development Center Germany GmbH Krausenstr. 38 10117 Berlin Geschaeftsfuehrung: Christian Schlaeger, Jonathan Weiss Eingetragen am Amtsgericht Charlottenburg unter HRB 149173 B Sitz: Berlin Ust-ID: DE 289 237 879
Quoted 21 lines · Md Shahadat Hossain
Large pages not being created properly may result in increased memory access time. The 'lpages' kvm stat used to keep track of the current number of large pages in the system, but with TDP MMU enabled the stat is not showing the correct number.
This patch extends the lpages counter to cover the TDP case.
Signed-off-by Md Shahadat Hossain Shahin <[email protected]>Cc Bartosz Szczepanek <[email protected]>--- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++++++ 1 file changed, 6 insertions(+)arch/x86/kvm/mmu/tdp_mmu.c +0
-0@@ -425,6 +425,12 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn,if (old_spte == new_spte)
arch/x86/kvm/mmu/tdp_mmu.c static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, · line 436
Quoted 10 lines
arch/x86/kvm/mmu/tdp_mmu.c +6
-0return ; + +if (is_large_pte (old_spte)) + --kvm->stat.lpages; + +if (is_large_pte (new_spte)) + ++kvm->stat.lpages;trace_kvm_tdp_mmu_spte_changed (as_id, gfn, level, old_spte, new_spte);
Queued, thanks.
Paolo
Sean Christopherson
maintainerQuoted 18 lines · Md Shahadat Hossain
Large pages not being created properly may result in increased memory access time. The 'lpages' kvm stat used to keep track of the current number of large pages in the system, but with TDP MMU enabled the stat is not showing the correct number.
This patch extends the lpages counter to cover the TDP case.
Signed-off-by Md Shahadat Hossain Shahin <[email protected]>Cc Bartosz Szczepanek <[email protected]>--- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++++++ 1 file changed, 6 insertions(+)arch/x86/kvm/mmu/tdp_mmu.c +0
-0
arch/x86/kvm/mmu/tdp_mmu.c static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, · line 433
Quoted 9 lines
arch/x86/kvm/mmu/tdp_mmu.c +6
-0@@ -425,6 +425,12 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn,if (old_spte == new_spte)return ; + +if (is_large_pte (old_spte)) + --kvm->stat.lpages; + +if (is_large_pte (new_spte)) + ++kvm->stat.lpages;
Hrm, kvm->stat.lpages could get corrupted when __handle_changed_spte() is called under read lock, e.g. if multiple vCPUs are faulting in memory.
Paolo Bonzini
maintainerQuoted 20 lines · Sean Christopherson
Quoted 19 lines · Md Shahadat Hossain
Large pages not being created properly may result in increased memory access time. The 'lpages' kvm stat used to keep track of the current number of large pages in the system, but with TDP MMU enabled the stat is not showing the correct number.
This patch extends the lpages counter to cover the TDP case.
Signed-off-by Md Shahadat Hossain Shahin <[email protected]>Cc Bartosz Szczepanek <[email protected]>--- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++++++ 1 file changed, 6 insertions(+)arch/x86/kvm/mmu/tdp_mmu.c +0
-0@@ -425,6 +425,12 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn,
Quoted 11 lines
arch/x86/kvm/mmu/tdp_mmu.c static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, · line 433
Quoted 8 lines
arch/x86/kvm/mmu/tdp_mmu.c +6
-0if (old_spte == new_spte)return ; + +if (is_large_pte (old_spte)) + --kvm->stat.lpages; + +if (is_large_pte (new_spte)) + ++kvm->stat.lpages;Hrm,
kvm->stat.lpagescould get corrupted when__handle_changed_spte()is called under read lock, e.g. if multiple vCPUs are faulting in memory.
Ouch, indeed!
One way to fix it without needing an atomic operation is to make it a per-vcpu stat. It would be a bit weird for the binary stats because we would have to hide this one from the vCPU statistics file descriptor (and only aggregate it in the VM statistics).
Alternatively, you can do the atomic_add only if is_large_pte(old_spte) != is_large_pte(new_spte), casting &kvm->stat.lpages to an atomic64_t*.
Paolo
Quoted 36 lines · Paolo Bonzini
Quoted 20 lines · Sean Christopherson
Quoted 19 lines · Md Shahadat Hossain
Large pages not being created properly may result in increased memory access time. The 'lpages' kvm stat used to keep track of the current number of large pages in the system, but with TDP MMU enabled the stat is not showing the correct number.
This patch extends the lpages counter to cover the TDP case.
Signed-off-by Md Shahadat Hossain Shahin <[email protected]>Cc Bartosz Szczepanek <[email protected]>--- arch/x86/kvm/mmu/tdp_mmu.c | 6 ++++++ 1 file changed, 6 insertions(+)arch/x86/kvm/mmu/tdp_mmu.c +0
-0@@ -425,6 +425,12 @@ static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn,Quoted 11 lines
arch/x86/kvm/mmu/tdp_mmu.c static void __handle_changed_spte(struct kvm *kvm, int as_id, gfn_t gfn, · line 433
Quoted 8 lines
arch/x86/kvm/mmu/tdp_mmu.c +6
-0if (old_spte == new_spte)return ; + +if (is_large_pte (old_spte)) + --kvm->stat.lpages; + +if (is_large_pte (new_spte)) + ++kvm->stat.lpages;Hrm,
kvm->stat.lpagescould get corrupted when__handle_changed_spte()is called under read lock, e.g. if multiple vCPUs are faulting in memory.Ouch, indeed!
Quoted 7 lines
One way to fix it without needing an atomic operation is to make it a per-vcpu stat. It would be a bit weird for the binary stats because we would have to hide this one from the vCPU statistics file descriptor (and only aggregate it in the VM statistics).
Alternatively, you can do the
atomic_addonly ifis_large_pte(old_spte)!=is_large_pte(new_spte), casting&kvm->stat.lpagesto an atomic64_t*.
I forgot that the lpages stat existed upstream. Internally at Google we also maintain a count for each mapping level and just update it atomically. The per-vCPU stat approach would work too, but I doubt it's worth the complexity unless we have other use cases for a similarly aggregated stat.
Quoted 3 lines
Paolo