KVM: arm64: Consistently use free_removed_table() for stage-2
PATCH 1/3 KVM: arm64: Consistently use free_removed_table() for stage-2
From: Oliver Upton <[email protected]>
free_removed_table() is essential to the RCU-protected parallel walking scheme, as behind the scenes the cleanup is deferred until an RCU grace period. Nonetheless, the stage-2 unmap path calls put_page() directly, which leads to table memory being freed inline with the table walk.
This is safe for the time being, as the stage-2 unmap walker is called while holding the write lock. A future change to KVM will further relax the locking mechanics around the stage-2 page tables to allow lock-free walkers protected only by RCU. As such, switch to the RCU-safe mechanism for freeing table memory.
--- arch/arm64/kvm/hyp/pgtable.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/hyp/pgtable.c +1 -1
@@ -1042,7 +1042,7 @@ static int stage2_unmap_walker(const struct kvm_pgtable_visit_ctx *ctx, kvm_granule_size(ctx->level));if (childp) - mm_ops->put_page (childp); + mm_ops->free_removed_table (childp, ctx->level);return 0 ; }
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 2/3 KVM: arm64: Only initiate walk if page_count() > 1 in free_removed_table()
Page table walk is unnecessary in free_removed_table() being called from the stage-2 unmap path while PTEs on the table is empty. It can be fast-pathed by only initiating a walk if page_count() > 1
Original disussion can be found in:
https://lore.kernel.org/kvmarm/[email protected]/
--- arch/arm64/kvm/hyp/pgtable.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
arch/arm64/kvm/hyp/pgtable.c +2 -1
@@ -1319,5 +1319,6 @@ void kvm_pgtable_stage2_free_removed(struct kvm_pgtable_mm_ops *mm_ops, void *pg .end = kvm_granule_size(level), }; -WARN_ON (__kvm_pgtable_walk (&data, mm_ops, ptep, level +1 )); +if (mm_ops->page_count (pgtable) >1 ) +WARN_ON (__kvm_pgtable_walk (&data, mm_ops, ptep, level +1 )); }
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
PATCH 3/3 KVM: arm64: Using rcu_read_lock() for kvm_pgtable_stage2_mkyoung()
In reply to Chun-Tse ShaoAccess bit is RCU safe and can be set without taking kvm->mmu_lock(). Replacing existing kvm->mmu_lock() with rcu_read_lock() for better performance.
Original disussion can be found in:
https://lore.kernel.org/kvmarm/CAOUHufZrfnfcbrqSzmHkejR5MA2gmGKZ3LMRhbLHV+1427z=Tw@mail.gmail.com/
--- arch/arm64/kvm/mmu.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
arch/arm64/kvm/mmu.c +2 -2
@@ -1437,10 +1437,10 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)trace_kvm_access_fault (fault_ipa); -read_lock (&vcpu->kvm->mmu_lock); +rcu_read_lock (); mmu = vcpu->arch.hw_mmu; pte =kvm_pgtable_stage2_mkyoung (mmu->pgt, fault_ipa); -read_unlock (&vcpu->kvm->mmu_lock); +rcu_read_unlock ();if (kvm_pte_valid (pte))kvm_set_pfn_accessed (kvm_pte_to_pfn (pte));
Signature · 1 line
2.41.0.162.gfafddb0af9-goog
Marc Zyngier
maintainerRe: [PATCH v1 3/3] KVM: arm64: Using rcu_read_lock() for kvm_pgtable_stage2_mkyoung()
Quoted 2 lines · Chun-Tse Shao
Access bit is RCU safe and can be set without taking
kvm->mmu_lock().
Please explain why. What happens when the page tables are *not* RCU controlled, such as in the pKVM case?
Quoted 2 lines
Replacing existing
kvm->mmu_lock()withrcu_read_lock()for better performance.
Please define "better performance", quote workloads, figures, HW setup and point to a reproducer. Please add a cover letter to your patch series explaining the context this happens in.
Also, I'm getting increasingly annoyed by the lack of coordination between seamingly overlapping patch series (this, Yu's, Anish's and Vipin's), all from a single company.
Surely you can talk to each other and devise a coordinated approach?
Thanks,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.
Re: [PATCH v1 3/3] KVM: arm64: Using rcu_read_lock() for kvm_pgtable_stage2_mkyoung()
Quoted 9 lines · Marc Zyngier
Quoted 2 lines · Chun-Tse Shao
Access bit is RCU safe and can be set without taking
kvm->mmu_lock().Please explain why. What happens when the page tables are *not* RCU controlled, such as in the pKVM case?
Quoted 6 lines
Quoted 2 lines
Replacing existing
kvm->mmu_lock()withrcu_read_lock()for better performance.Please define "better performance", quote workloads, figures, HW setup and point to a reproducer. Please add a cover letter to your patch series explaining the context this happens in.
Thanks for the suggestion, we are currently working on the performance test in parallel and will update after gathering more data.
Quoted 6 lines
Also, I'm getting increasingly annoyed by the lack of coordination between seamingly overlapping patch series (this, Yu's, Anish's and Vipin's), all from a single company.
Surely you can talk to each other and devise a coordinated approach?
Sure, I will start internal meeting as needed.
Quoted 7 lines
Thanks,
M.--
Without deviation from the norm, progress is not possible.
Thanks,
CT
Re: [PATCH v1 3/3] KVM: arm64: Using rcu_read_lock() for kvm_pgtable_stage2_mkyoung()
In reply to Chun-Tse ShaoQuoted 5 lines · Chun-Tse Shao
arch/arm64/kvm/mmu.c +0
-0
arch/arm64/kvm/mmu.c static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa) · line 1443
Quoted 9 lines
arch/arm64/kvm/mmu.c +2
-2@@ -1437,10 +1437,10 @@ static void handle_access_fault(struct kvm_vcpu *vcpu, phys_addr_t fault_ipa)trace_kvm_access_fault (fault_ipa); -read_lock (&vcpu->kvm->mmu_lock); +rcu_read_lock (); mmu = vcpu->arch.hw_mmu; pte =kvm_pgtable_stage2_mkyoung (mmu->pgt, fault_ipa); -read_unlock (&vcpu->kvm->mmu_lock); +rcu_read_unlock ();
What is the point of acquiring the RCU read lock here? kvm_pgtable_walk_{begin,end}() already do the exact same for any 'shared' walk.
I agree with Marc that this warrants some very clear benchmark data showing the value of the change. As I had mentioned to Yu, I already implemented this for my own purposes, but wasn't able to see a significant improvement over acquiring the MMU lock for read.
Signature · 2 lines
Thanks, Oliver
Quoted 8 lines · Chun-Tse Shao
From: Oliver Upton <[email protected]>
free_removed_table()is essential to the RCU-protected parallel walking scheme, as behind the scenes the cleanup is deferred until an RCU grace period. Nonetheless, the stage-2 unmap path callsput_page()directly, which leads to table memory being freed inline with the table walk.
Quoted 8 lines
This is safe for the time being, as the stage-2 unmap walker is called while holding the write lock. A future change to KVM will further relax the locking mechanics around the stage-2 page tables to allow lock-free walkers protected only by RCU. As such, switch to the RCU-safe mechanism for freeing table memory.
Signed-off-by Oliver Upton <[email protected]>Signed-off-by Chun-Tse Shao <[email protected]>