Skip to content

RFC KVM: x86/mmu: Rework marking folios dirty/accessed

Sean Christopherson kvm lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest4 patches 15 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/4KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf SPTEarch/x86/kvm/mmu/spte.c+13 -50 replies
2/4KVM: x86/mmu: Mark folio dirty when creating SPTE, not when zapping/modifyingarch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/paging_tmpl.h arch/x86/kvm/mmu/spte.c +1 more+17 -440 replies
3/4KVM: x86/mmu: Mark page/folio accessed only when zapping leaf SPTEsDocumentation/virt/kvm/locking.rst arch/x86/kvm/mmu/mmu.c arch/x86/kvm/mmu/tdp_mmu.c+43 -440 replies
4/4KVM: x86/mmu: Don't force flush if SPTE update clears Accessed bitarch/x86/kvm/mmu/mmu.c+9 -220 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [RFC PATCH 0/4] KVM: x86/mmu: Rework marking folios dirty/accessed

Rework KVM to mark folios dirty when creating shadow/secondary PTEs (SPTEs), i.e. when creating mappings for KVM guests, instead of when zapping or modifying SPTEs, e.g. when dropping mappings.

The motivation is twofold:

1. Marking folios dirty and accessed when zapping can be extremely expensive and wasteful, e.g. if KVM shattered a 1GiB hugepage into 512*512 4KiB SPTEs for dirty logging, then KVM marks the huge folio dirty and accessed for all 512*512 SPTEs.

2. x86 diverges from literally every other architecture, which updates folios when mappings are created. AFAIK, x86 is unique in that it's the only KVM arch that prefetches PTEs, so it's not quite an apples- to-apples comparison, but I don't see any reason for the dirty logic in particular to be different.

I tagged this RFC as it is barely tested, and because I'm not 100% positive there isn't some weird edge case I'm missing, which is why I Cc'd David H. and Matthew.

Note, I'm going to be offline from ~now until April 1st. I rushed this out as it could impact David S.'s kvm_follow_pfn series[*], which is imminent. E.g. if KVM stops marking pages dirty and accessed everywhere, adding SPTE_MMU_PAGE_REFCOUNTED just to sanity check that the refcount is elevated seems like a poor tradeoff (medium complexity and annoying to maintain, for not much benefit).

Regarding David S.'s series, I wouldn't be at all opposed to going even further and having x86 follow all architectures by marking pages accessed _only_ at map time, at which point I think KVM could simply pass in FOLL_TOUCH as appropriate, and thus dedup a fair bit of arch code.

Lastly, regarding bullet #1 above, we might be able to eke out better performance by batching calls to folio_mark_{accessed,dirty}() on the backend, e.g. when zapping SPTEs that KVM knows are covered by a single hugepage. But I think in practice any benefit would be marginal, as it would be quite odd for KVM to fault-in a 1GiB hugepage at 4KiB granularity.

And _if_ we wanted to optimize that case, I suspect we'd be better off pre-mapping all SPTEs for a pfn that is mapped at a larger granularity in the primary MMU. E.g. if KVM is dirty logging a 1GiB HugeTLB page, install MMU-writable 4KiB SPTEs for the entire 1GiB region when any pfn is accessed.

P.S. Matthew ruined the "nothing but Davids!" Cc list.

[*] https://lore.kernel.org/all/[email protected]

Sean Christopherson (4):
KVM: x86/mmu: Skip the "try unsync" path iff the old SPTE was a leaf
SPTE
KVM: x86/mmu: Mark folio dirty when creating SPTE, not when
zapping/modifying
KVM: x86/mmu: Mark page/folio accessed only when zapping leaf SPTEs
KVM: x86/mmu: Don't force flush if SPTE update clears Accessed bit

 Documentation/virt/kvm/locking.rst | 76 +++++++++++++++---------------
 arch/x86/kvm/mmu/mmu.c             | 60 +++++------------------
 arch/x86/kvm/mmu/paging_tmpl.h     |  7 ++-
 arch/x86/kvm/mmu/spte.c            | 27 ++++++++---
 arch/x86/kvm/mmu/tdp_mmu.c         | 19 ++------
 5 files changed, 78 insertions(+), 111 deletions(-)

base-commit: 964d0c614c7f71917305a5afdca9178fe8231434

Signature · 1 line
2.44.0.291.gc1ea87d7ee-goog