KVM: riscv: Age G-stage PTEs locklessly
Get series with b4
b4 am [email protected]Versions newest first
- v1 Latest5 patches 0 repliesNew
Patches in v1 No applies recorded yet
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/5 | KVM: riscv: Age all G-stage PTEs in a GFN rangearch/ arch/ arch/ | +40 | 0 replies | |
| 2/5 | KVM: riscv: Read G-stage PTEs once when walking page tablesarch/ | +27 | 0 replies | |
| 3/5 | KVM: riscv: Write-protect G-stage PTEs atomicallyarch/ | +9 | 0 replies | |
| 4/5 | KVM: riscv: Free G-stage page tables after an RCU grace periodarch/ arch/ arch/ +1 more | +32 | 0 replies | |
| 5/5 | KVM: riscv: Age G-stage PTEs locklesslyarch/ arch/ | +14 | 0 replies |
Reviews trailers per patch
No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.
Cover letter [PATCH v1 0/5] KVM: riscv: Age G-stage PTEs locklessly
Aging G-stage PTEs currently runs with mmu_lock held for write, taken by the common MMU notifier code. When MGLRU or kswapd ages a large range, every vCPU taking a G-stage fault blocks on the lock, and the dirty-logging read-side fast path blocks behind aging as well.
This series lets RISC-V select KVM_MMU_LOCKLESS_AGING, so G-stage aging can run without mmu_lock while protecting page-table walks with RCU.
1. Age all G-stage PTEs in a GFN range
Bug fix, posted separately [1] and included unchanged so the series applies on riscv_kvm_queue. Drop it if already applied.
2. Read G-stage PTEs once when walking page tables
A lockless walker must derive presence, leaf-ness and the child address from a single snapshot of each entry.
3. Write-protect G-stage PTEs atomically
Aging will clear the Accessed bit without mmu_lock; the plain read-modify-write in GSTAGE_OP_WP could clobber that update.
4. Free G-stage page tables after an RCU grace period Table pages and the root outlive any RCU reader. Root teardown keeps pgd_levels intact so a walker holding the old root sees consistent metadata.
5. Age G-stage PTEs locklessly
Select KVM_MMU_LOCKLESS_AGING and walk under rcu_read_lock().
This builds on Jinyu Tang's rwlock and cmpxchg helpers already in riscv_kvm_queue. Wang Yechao's huge page recovery series [2] adds a second page-table free site in kvm_riscv_gstage_recover_huge(); whichever lands second will need to switch that put_page() to gstage_free_page_table(). I can rebase the series as needed.
A possible follow-up is to run clear-dirty-log and the general fault path under the read side of mmu_lock, as arm64 and x86 do. Patch 3 provides the atomic write-protection needed for the former.
Testing
-------
QEMU virt, -cpu rv64,h=true (Svadu), 4 vCPUs, TCG. The host kernel was built with KASAN, PROVE_LOCKING, PROVE_RCU, LOCK_STAT and LRU_GEN.
After each patch:
- dirty_log_perf_test -v 2 -b 256M -i 3
- access_tracking_perf_test -v 2 -b 256M -s anonymous_thp (2M THP disabled and 64K mTHP enabled)
- 20 VM create/destroy iterations
With MGLRU aging forced on the VM's cgroup every 50-100ms:
- dirty_log_perf_test -v 3 -b 256M -i 3
- 30 VM create/destroy iterations
With CONFIG_KVM=m, 5 module load/unload cycles, each with a VM created and destroyed immediately before module unload, to exercise module teardown after queuing RCU frees.
No WARN, KASAN, lockdep or RCU reports were observed. The kvm_age_hva tracepoint recorded approximately 65k events per aging pass.
Performance results from dirty_log_perf_test -v 3 -b 256M -i 3, with lock_stat enabled for mmu_lock and 86 aging passes:
no aging aging
before after before after
guest dirty time 12.5s 12.4s 18.6s 16.7s mmu_lock W wait total 0.15s 0.17s 44.4s 0.15s mmu_lock W contentions 35k 35k 1.0M 30k mmu_lock R wait total 0 0 7.5s 0
Before the series, 93% of write-lock waiters were from kvm_mmu_notifier_clear_young(), and the dirty-log read-side fast path waited for mmu_lock 352k times. After the series, aging does not acquire mmu_lock.
Absolute numbers are inflated by TCG; only the relative change is meaningful. Under TCG, the atomic write-protect change in patch 3 increased the clear-dirty-log time from approximately 16ms to 30ms, as TCG serialises AMO instructions; hardware numbers would be welcome.
[1] https://lore.kernel.org/all/[email protected]/
[2] https://lore.kernel.org/kvm-riscv/[email protected]/
SeungJu Cheon (5):
KVM: riscv: Age all G-stage PTEs in a GFN range
KVM: riscv: Read G-stage PTEs once when walking page tables
KVM: riscv: Write-protect G-stage PTEs atomically
KVM: riscv: Free G-stage page tables after an RCU grace period
KVM: riscv: Age G-stage PTEs locklessly
arch/riscv/include/asm/kvm_gstage.h | 4 +- arch/riscv/kvm/Kconfig | 1 + arch/riscv/kvm/gstage.c | 111 +++++++++++++++++++++------- arch/riscv/kvm/main.c | 3 + arch/riscv/kvm/mmu.c | 56 +++++++------- 5 files changed, 117 insertions(+), 58 deletions(-)
base-commit: 41e81f7e3ef96594fb840445343c0ee7723aa550
Signature · 1 line
2.52.0