KVM: Do not reset dirty GFNs in a memslot not enabling dirty tracking
v3 cover KVM: Do not reset dirty GFNs in a memslot not enabling dirty tracking
Hi,
This series addresses a bug where userspace can request KVM to reset dirty GFNs in memslots that do not have dirty tracking enabled.
Patch 1 provides the fix.
Patch 2 is an optimization to avoid unnecessary invoking of handlers in
kvm_handle_hva_range() when a GFN range is entirely private.
Patch 2 is not directly related to the issue in this series, but was found while implementing the selftest in patch 3. It also enhance reliability of the selftest results in patch 3 by ruling out the zap-related changes to private mappings of the test slot.
Patch 3 converts the TDX-specific selftest in v2 to test
KVM_X86_SW_PROTECTED_VM VMs.
Unlike TDX cases which would generate KVM_BUG_ON() when GFNs are incorrectly reset in memslots not enabling dirty tracking, there are not obvious errors for KVM_X86_SW_PROTECTED_VMs. So, patch 3 detects the event kvm_tdp_mmu_spte_changed instead.
Will provide TDX cases once the TDX selftest framework is finalized.
v3:
- Rebased patch 1.
- Added patch 2.
- Converted patch 3 to test KVM_X86_SW_PROTECTED_VM VMs.
- code base: kvm-x86-next-2025.08.21
v2: https://lore.kernel.org/all/[email protected]
- Added a comment in patch 1, explaining that it's possible to try to update a memslot that isn't being dirty-logged if userspace is misbehaving. Specifically, userspace can write arbitrary data into the ring. (Sean)
v1: https://lore.kernel.org/all/[email protected]
Yan Zhao (3):
KVM: Do not reset dirty GFNs in a memslot not enabling dirty tracking
KVM: Skip invoking shared memory handler for entirely private GFN
ranges
KVM: selftests: Test resetting dirty ring in gmem slots in protected
VMs
tools/testing/selftests/kvm/Makefile.kvm | 1 + .../kvm/x86/reset_dirty_ring_on_gmem_test.c | 392 ++++++++++++++++++ virt/kvm/dirty_ring.c | 8 +- virt/kvm/kvm_main.c | 11 + 4 files changed, 411 insertions(+), 1 deletion(-) create mode 100644 tools/testing/selftests/kvm/x86/reset_dirty_ring_on_gmem_test.c
Signature · 1 line
2.43.2
PATCH v3 1/3 KVM: Do not reset dirty GFNs in a memslot not enabling dirty tracking
Do not allow resetting dirty GFNs in memslots that do not enable dirty tracking.
vCPUs' dirty rings are shared between userspace and KVM. After KVM sets dirtied entries in the dirty rings, userspace is responsible for harvesting/resetting these entries and calling the ioctl KVM_RESET_DIRTY_RINGS to inform KVM to advance the reset_index in the dirty rings and invoke kvm_arch_mmu_enable_log_dirty_pt_masked() to clear the SPTEs' dirty bits or perform write protection of the GFNs.
Although KVM does not set dirty entries for GFNs in a memslot that does not enable dirty tracking, userspace can write arbitrary data into the dirty ring. This makes it possible for misbehaving userspace to specify that it has harvested a GFN from such a memslot. When this happens, KVM will be asked to clear dirty bits or perform write protection for GFNs in a memslot that does not enable dirty tracking, which is undesirable.
For TDX, this unexpected resetting of dirty GFNs could cause inconsistency between the mirror SPTE and the external SPTE in hardware (e.g., the mirror SPTE has no write bit while the external SPTE is writable). When kvm_dirty_log_manual_protect_and_init_set() is true and huge pages are enabled in TDX, this could even lead to kvm_mmu_slot_gfn_write_protect() being called and trigger KVM_BUG_ON() due to permission reduction changes in the huge mirror SPTEs.
--- virt/kvm/dirty_ring.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)
virt/kvm/dirty_ring.c +7 -1
@@ -63,7 +63,13 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask) memslot =id_to_memslot (__kvm_memslots (kvm, as_id), id); -if (!memslot || (offset +__fls (mask)) >= memslot->npages) +/* +* Userspace can write arbitrary data into the dirty ring, making it +* possible for misbehaving userspace to try to reset an out-of-memslot +* GFN or a GFN in a memslot that isn't being dirty-logged. +*/ +if (!memslot || (offset +__fls (mask)) >= memslot->npages || + !kvm_slot_dirty_track_enabled (memslot))return ;KVM_MMU_LOCK (kvm);
Signature · 1 line
2.43.2
Sean Christopherson
maintainerQuoted 17 lines · Yan Zhao
Do not allow resetting dirty GFNs in memslots that do not enable dirty tracking.
vCPUs' dirty rings are shared between userspace and KVM. After KVM sets dirtied entries in the dirty rings, userspace is responsible for harvesting/resetting these entries and calling the ioctl
KVM_RESET_DIRTY_RINGSto inform KVM to advance thereset_indexin the dirty rings and invokekvm_arch_mmu_enable_log_dirty_pt_masked()to clear the SPTEs' dirty bits or perform write protection of the GFNs.Although KVM does not set dirty entries for GFNs in a memslot that does not enable dirty tracking, userspace can write arbitrary data into the dirty ring. This makes it possible for misbehaving userspace to specify that it has harvested a GFN from such a memslot. When this happens, KVM will be asked to clear dirty bits or perform write protection for GFNs in a memslot that does not enable dirty tracking, which is undesirable.
Quoted 8 lines
For TDX, this unexpected resetting of dirty GFNs could cause inconsistency between the mirror SPTE and the external SPTE in hardware (e.g., the mirror SPTE has no write bit while the external SPTE is writable). When
kvm_dirty_log_manual_protect_and_init_set()is true and huge pages are enabled in TDX, this could even lead tokvm_mmu_slot_gfn_write_protect()being called and triggerKVM_BUG_ON()due to permission reduction changes in the huge mirror SPTEs.
Sounds like this needs a Fixes and Cc: stable?
Quoted 11 lines
Signed-off-by Yan Zhao <[email protected]>--- virt/kvm/dirty_ring.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)virt/kvm/dirty_ring.c +0
-0@@ -63,7 +63,13 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask)
virt/kvm/dirty_ring.c static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask) · line 72
Quoted 10 lines
virt/kvm/dirty_ring.c +7
-1memslot =id_to_memslot (__kvm_memslots (kvm, as_id), id); -if (!memslot || (offset +__fls (mask)) >= memslot->npages) +/* +* Userspace can write arbitrary data into the dirty ring, making it +* possible for misbehaving userspace to try to reset an out-of-memslot +* GFN or a GFN in a memslot that isn't being dirty-logged. +*/ +if (!memslot || (offset +__fls (mask)) >= memslot->npages || + !kvm_slot_dirty_track_enabled (memslot))
Maybe check for dirty tracking being enabled before checking the range? Purely because checking if _any_ gfn can be recorded seems like something that should be checked before a specific gfn can be recorded. I.e.
if (!memslot || !kvm_slot_dirty_track_enabled(memslot) || (offset + __fls(mask)) >= memslot->npages)
Quoted 6 lines
virt/kvm/dirty_ring.c +0
-0return ;KVM_MMU_LOCK (kvm);Signature · 1 line
2.43.2
Quoted 27 lines · Sean Christopherson
Quoted 17 lines · Yan Zhao
Do not allow resetting dirty GFNs in memslots that do not enable dirty tracking.
vCPUs' dirty rings are shared between userspace and KVM. After KVM sets dirtied entries in the dirty rings, userspace is responsible for harvesting/resetting these entries and calling the ioctl
KVM_RESET_DIRTY_RINGSto inform KVM to advance thereset_indexin the dirty rings and invokekvm_arch_mmu_enable_log_dirty_pt_masked()to clear the SPTEs' dirty bits or perform write protection of the GFNs.Although KVM does not set dirty entries for GFNs in a memslot that does not enable dirty tracking, userspace can write arbitrary data into the dirty ring. This makes it possible for misbehaving userspace to specify that it has harvested a GFN from such a memslot. When this happens, KVM will be asked to clear dirty bits or perform write protection for GFNs in a memslot that does not enable dirty tracking, which is undesirable.
Quoted 8 lines
For TDX, this unexpected resetting of dirty GFNs could cause inconsistency between the mirror SPTE and the external SPTE in hardware (e.g., the mirror SPTE has no write bit while the external SPTE is writable). When
kvm_dirty_log_manual_protect_and_init_set()is true and huge pages are enabled in TDX, this could even lead tokvm_mmu_slot_gfn_write_protect()being called and triggerKVM_BUG_ON()due to permission reduction changes in the huge mirror SPTEs.
Quoted 1 line
Sounds like this needs a Fixes and Cc: stable?
Ok. Will include them in the next version.
Quoted 26 lines
Quoted 11 lines
Signed-off-by Yan Zhao <[email protected]>--- virt/kvm/dirty_ring.c | 8 +++++++- 1 file changed, 7 insertions(+), 1 deletion(-)virt/kvm/dirty_ring.c +0
-0@@ -63,7 +63,13 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask)virt/kvm/dirty_ring.c static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask) · line 72
Quoted 10 lines
virt/kvm/dirty_ring.c +7
-1memslot =id_to_memslot (__kvm_memslots (kvm, as_id), id); -if (!memslot || (offset +__fls (mask)) >= memslot->npages) +/* +* Userspace can write arbitrary data into the dirty ring, making it +* possible for misbehaving userspace to try to reset an out-of-memslot +* GFN or a GFN in a memslot that isn't being dirty-logged. +*/ +if (!memslot || (offset +__fls (mask)) >= memslot->npages || + !kvm_slot_dirty_track_enabled (memslot))Maybe check for dirty tracking being enabled before checking the range? Purely because checking if _any_ gfn can be recorded seems like something that should be checked before a specific gfn can be recorded. I.e.
Quoted 2 lines
if (!memslot || !kvm_slot_dirty_track_enabled(memslot) || (offset + __fls(mask)) >= memslot->npages)
Makes sense.
Thank you!
Quoted 6 lines
Quoted 6 lines
virt/kvm/dirty_ring.c +0
-0return ;KVM_MMU_LOCK (kvm);Signature · 1 line
2.43.2
PATCH v3 2/3 KVM: Skip invoking shared memory handler for entirely private GFN ranges
In reply to Yan ZhaoWhen a GFN range is entirely private, it's unnecessary for kvm_handle_hva_range() to invoke handlers for the GFN range, because
1) the gfn_range.attr_filter for the handler is KVM_FILTER_SHARED, which is for shared mappings only;
2) KVM has already zapped all shared mappings before setting the memory attribute to private.
This can avoid unnecessary zaps on private mappings for VMs of type KVM_X86_SW_PROTECTED_VM, e.g., during auto numa balancing scans of VMAs.
--- virt/kvm/kvm_main.c | 11 +++++++++++ 1 file changed, 11 insertions(+)
virt/kvm/kvm_main.c +11 -0
@@ -620,6 +620,17 @@ static __always_inline kvm_mn_ret_t kvm_handle_hva_range(struct kvm *kvm, gfn_range.slot = slot; gfn_range.lockless = range->lockless; +#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +/* +* If GFN range are all private, no need to invoke the +* handler. +*/ +if (kvm_range_has_memory_attributes (kvm, gfn_range.start, + gfn_range.end, ~0 , +KVM_MEMORY_ATTRIBUTE_PRIVATE )) +continue ; +#endif +if (!r.found_memslot) { r.found_memslot = true;if (!range->lockless) {
Signature · 1 line
2.43.2
Sean Christopherson
maintainerRe: [PATCH v3 2/3] KVM: Skip invoking shared memory handler for entirely private GFN ranges
Quoted 9 lines · Yan Zhao
When a GFN range is entirely private, it's unnecessary for
kvm_handle_hva_range()to invoke handlers for the GFN range, because
1) the gfn_range.attr_filter for the handler isKVM_FILTER_SHARED, which is for shared mappings only;
2) KVM has already zapped all shared mappings before setting the memory attribute to private.This can avoid unnecessary zaps on private mappings for VMs of type
KVM_X86_SW_PROTECTED_VM, e.g., during auto numa balancing scans of VMAs.
This feels like the wrong place to try and optimize spurious zaps. x86 should be skipping SPTEs that don't match. For KVM_X86_SW_PROTECTED_VM, I don't think we care about spurious zpas, because that's a testing-only type that doesn't have line of sight to be being a "real" type.
For SNP, we might care? But actually zapping private SPTEs would require userspace to retain the shared mappings across a transition, _and_ be running NUMA autobalancing in the first place. If someone actually cares about optimizing this scenario, KVM x86 could track private SPTEs via a software-available bit.
We also want to move away from KVM_MEMORY_ATTRIBUTE_PRIVATE and instead track private vs. shared in the gmem instance.
So I'm inclined to skip this...
Quoted 30 lines
Signed-off-by Yan Zhao <[email protected]>--- virt/kvm/kvm_main.c | 11 +++++++++++ 1 file changed, 11 insertions(+)virt/kvm/kvm_main.c +11
-0@@ -620,6 +620,17 @@ static __always_inline kvm_mn_ret_t kvm_handle_hva_range(struct kvm *kvm, gfn_range.slot = slot; gfn_range.lockless = range->lockless; +#ifdef CONFIG_KVM_GENERIC_MEMORY_ATTRIBUTES +/* +* If GFN range are all private, no need to invoke the +* handler. +*/ +if (kvm_range_has_memory_attributes (kvm, gfn_range.start, + gfn_range.end, ~0 , +KVM_MEMORY_ATTRIBUTE_PRIVATE )) +continue ; +#endif +if (!r.found_memslot) { r.found_memslot = true;if (!range->lockless) {Signature · 1 line
2.43.2
Re: [PATCH v3 2/3] KVM: Skip invoking shared memory handler for entirely private GFN ranges
Quoted 11 lines · Sean Christopherson
Quoted 9 lines · Yan Zhao
When a GFN range is entirely private, it's unnecessary for
kvm_handle_hva_range()to invoke handlers for the GFN range, because
1) the gfn_range.attr_filter for the handler isKVM_FILTER_SHARED, which is for shared mappings only;
2) KVM has already zapped all shared mappings before setting the memory attribute to private.This can avoid unnecessary zaps on private mappings for VMs of type
KVM_X86_SW_PROTECTED_VM, e.g., during auto numa balancing scans of VMAs.
Quoted 8 lines
This feels like the wrong place to try and optimize spurious zaps. x86 should be skipping SPTEs that don't match. For
KVM_X86_SW_PROTECTED_VM, I don't think we care about spurious zpas, because that's a testing-only type that doesn't have line of sight to be being a "real" type.For SNP, we might care? But actually zapping private SPTEs would require userspace to retain the shared mappings across a transition, _and_ be running NUMA autobalancing in the first place. If someone actually cares about optimizing
Hmm, "running NUMA autobalancing" + "madvise(MADV_DONTNEED)" can still trigger the spurious zaps.
task_numa_work ==> found a VMA
change_prot_numa
change_protection
change_pud_range ==> mmu_notifier_invalidate_range_start() if !pud_none()Let me use munmap() in patch 3 to guard againt spurious zap then.
Quoted 6 lines
this scenario, KVM x86 could track private SPTEs via a software-available bit.
We also want to move away from
KVM_MEMORY_ATTRIBUTE_PRIVATEand instead track private vs. shared in the gmem instance.So I'm inclined to skip this...
Fair enough. Thank you for the detailed explanation!
PATCH v3 3/3 KVM: selftests: Test resetting dirty ring in gmem slots in protected VMs
In reply to Yan ZhaoTest resetting dirty ring in slots with the KVM_MEM_GUEST_MEMFD flag in KVM_X86_SW_PROTECTED_VM VMs.
Purposely resetting dirty ring entries incorrectly to point to a gmem slot.
Unlike in TDX VMs, where resetting the dirty ring in a gmem slot could trigger KVM_BUG_ON(), there are no obvious errors for KVM_X86_SW_PROTECTED_VM VMs. Therefore, detect SPTE changes by reading trace messages with the kvm_tdp_mmu_spte_changed event enabled. Consequently, the test is conducted only when tdp_mmu is enabled and tracing is available.
--- tools/testing/selftests/kvm/Makefile.kvm | 1 + .../kvm/x86/reset_dirty_ring_on_gmem_test.c | 392 ++++++++++++++++++ 2 files changed, 393 insertions(+) create mode 100644 tools/testing/selftests/kvm/x86/reset_dirty_ring_on_gmem_test.c
tools/testing/selftests/kvm/Makefile.kvm +1 -0
@@ -136,6 +136,7 @@ TEST_GEN_PROGS_x86 += x86/max_vcpuid_cap_test TEST_GEN_PROGS_x86 += x86/triple_fault_event_test TEST_GEN_PROGS_x86 += x86/recalc_apic_map_test TEST_GEN_PROGS_x86 += x86/aperfmperf_test +TEST_GEN_PROGS_x86 += x86/reset_dirty_ring_on_gmem_test TEST_GEN_PROGS_x86 += access_tracking_perf_test TEST_GEN_PROGS_x86 += coalesced_io_test TEST_GEN_PROGS_x86 += dirty_log_perf_test
tools/testing/selftests/kvm/x86/reset_dirty_ring_on_gmem_test.c new file +392 -0
392 lines not on this page · raw message
Signature · 1 line
2.43.2