Skip to contentSkip to conversation

KVM: Do not reset dirty GFNs in a memslot not enabling dirty tracking

PATCH Superseded Yan Zhao 2 participants 5 messages Dec 20, 2024 – Dec 23, 2024

Subsystems (3)

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Yan Zhao

author
#0 lore raw

cover KVM: Do not reset dirty GFNs in a memslot not enabling dirty tracking

Hi
This series is for a bug where userspace can request KVM to reset dirty GFNs belonging to a memslot that does not enable dirty tracking.

Patch 1 provides the fix, which can be applied to Linux 6.13-rc3. Although the fix is a generic one, its primary motivation is to prevent userspace from triggering write permission reduction or accessed bit clearing in mirror SPTEs within TDX VMs. This could otherwise cause mismatches between mirror SPTEs and the corresponding external SPTEs, and in the worst case, lead to the removal of the external SPTEs.

Patch 2 introduces a selftest for TDX VMs to demonstrate how userspace could trigger this bug. If necessary, this test can be ported to the generic KVM selftest (e.g., dirty_log_test).

Thanks
Yan

Yan Zhao (2):
KVM: Do not reset dirty GFNs in a memslot not enabling dirty tracking
KVM: selftests: TDX: Test dirty ring on a gmemfd slot

 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/x86_64/tdx_dirty_ring.c     | 231 ++++++++++++++++++
 virt/kvm/dirty_ring.c                         |   3 +-
 3 files changed, 234 insertions(+), 1 deletion(-)
 create mode 100644 tools/testing/selftests/kvm/x86_64/tdx_dirty_ring.c
Signature · 1 line
2.43.2
author
#1 lore raw

PATCH 1/2 KVM: Do not reset dirty GFNs in a memslot not enabling dirty tracking

Do not allow resetting dirty GFNs belonging to a memslot that does not enable dirty tracking.

vCPUs' dirty rings are shared between userspace and KVM. After KVM sets dirtied entries in the dirty rings, userspace is responsible for harvesting/resetting the dirtied entries and calling the ioctl KVM_RESET_DIRTY_RINGS to inform KVM to advance the reset_index in the dirty rings and invoke kvm_arch_mmu_enable_log_dirty_pt_masked() to clear the SPTEs' dirty bits or perform write protection of GFNs.

Although KVM does not set dirty entries for GFNs in a memslot that does not enable dirty tracking, it is still possible for userspace to specify that it has harvested a GFN belonging to such a memslot. When this happens, KVM will be asked to clear dirty bits or perform write protection for GFNs in a memslot that does not enable dirty tracking, which is not desired.

For TDX, this unexpected resetting of dirty GFNs could cause inconsistency between the mirror SPTE and the external SPTE in hardware (e.g., the mirror SPTE has no write bit while it is writable in the external SPTE in hardware). When kvm_dirty_log_manual_protect_and_init_set() is true and when huge pages are enabled in TDX, this could even lead to kvm_mmu_slot_gfn_write_protect() being called and the external SPTE being removed.

---
 virt/kvm/dirty_ring.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
virt/kvm/dirty_ring.c +2 -1
index d14ffc7513ee..1ce5352ea596 100644
@@ -66,7 +66,8 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask)
 
 	memslot = id_to_memslot(__kvm_memslots(kvm, as_id), id);
 
-	if (!memslot || (offset + __fls(mask)) >= memslot->npages)
+	if (!memslot || (offset + __fls(mask)) >= memslot->npages ||
+	    !kvm_slot_dirty_track_enabled(memslot))
 		return;
 
 	KVM_MMU_LOCK(kvm);
Signature · 1 line
2.43.2
#2 lore raw
Quoted 34 lines · Yan Zhao

Do not allow resetting dirty GFNs belonging to a memslot that does not enable dirty tracking.

vCPUs' dirty rings are shared between userspace and KVM. After KVM sets dirtied entries in the dirty rings, userspace is responsible for harvesting/resetting the dirtied entries and calling the ioctl KVM_RESET_DIRTY_RINGS to inform KVM to advance the reset_index in the dirty rings and invoke kvm_arch_mmu_enable_log_dirty_pt_masked() to clear the SPTEs' dirty bits or perform write protection of GFNs.

Although KVM does not set dirty entries for GFNs in a memslot that does not enable dirty tracking, it is still possible for userspace to specify that it has harvested a GFN belonging to such a memslot. When this happens, KVM will be asked to clear dirty bits or perform write protection for GFNs in a memslot that does not enable dirty tracking, which is not desired.

For TDX, this unexpected resetting of dirty GFNs could cause inconsistency between the mirror SPTE and the external SPTE in hardware (e.g., the mirror SPTE has no write bit while it is writable in the external SPTE in hardware). When kvm_dirty_log_manual_protect_and_init_set() is true and when huge pages are enabled in TDX, this could even lead to kvm_mmu_slot_gfn_write_protect() being called and the external SPTE being removed.

---
 virt/kvm/dirty_ring.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
virt/kvm/dirty_ring.c +0 -0
index d14ffc7513ee..1ce5352ea596 100644

virt/kvm/dirty_ring.c static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask) · line 70

Quoted 6 lines
virt/kvm/dirty_ring.c +2 -1
@@ -66,7 +66,8 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask)
 
 	memslot = id_to_memslot(__kvm_memslots(kvm, as_id), id);
 
-	if (!memslot || (offset + __fls(mask)) >= memslot->npages)
+	if (!memslot || (offset + __fls(mask)) >= memslot->npages ||
+	    !kvm_slot_dirty_track_enabled(memslot))

Can you add a comment explaining that it's possible to try to update a memslot that isn't being dirty-logged if userspace is misbehaving? And specifically that userspace can write arbitrary data into the ring.

author
#3 lore raw
Quoted 38 lines · Sean Christopherson
Quoted 37 lines · Yan Zhao

Do not allow resetting dirty GFNs belonging to a memslot that does not enable dirty tracking.

vCPUs' dirty rings are shared between userspace and KVM. After KVM sets dirtied entries in the dirty rings, userspace is responsible for harvesting/resetting the dirtied entries and calling the ioctl KVM_RESET_DIRTY_RINGS to inform KVM to advance the reset_index in the dirty rings and invoke kvm_arch_mmu_enable_log_dirty_pt_masked() to clear the SPTEs' dirty bits or perform write protection of GFNs.

Although KVM does not set dirty entries for GFNs in a memslot that does not enable dirty tracking, it is still possible for userspace to specify that it has harvested a GFN belonging to such a memslot. When this happens, KVM will be asked to clear dirty bits or perform write protection for GFNs in a memslot that does not enable dirty tracking, which is not desired.

For TDX, this unexpected resetting of dirty GFNs could cause inconsistency between the mirror SPTE and the external SPTE in hardware (e.g., the mirror SPTE has no write bit while it is writable in the external SPTE in hardware). When kvm_dirty_log_manual_protect_and_init_set() is true and when huge pages are enabled in TDX, this could even lead to kvm_mmu_slot_gfn_write_protect() being called and the external SPTE being removed.

---
 virt/kvm/dirty_ring.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
virt/kvm/dirty_ring.c +0 -0
index d14ffc7513ee..1ce5352ea596 100644
@@ -66,7 +66,8 @@ static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask)
 
 	memslot = id_to_memslot(__kvm_memslots(kvm, as_id), id);
 
Quoted 7 lines

virt/kvm/dirty_ring.c static void kvm_reset_dirty_gfn(struct kvm *kvm, u32 slot, u64 offset, u64 mask) · line 70

Quoted 3 lines
virt/kvm/dirty_ring.c +2 -1
-	if (!memslot || (offset + __fls(mask)) >= memslot->npages)
+	if (!memslot || (offset + __fls(mask)) >= memslot->npages ||
+	    !kvm_slot_dirty_track_enabled(memslot))

Can you add a comment explaining that it's possible to try to update a memslot that isn't being dirty-logged if userspace is misbehaving? And specifically that userspace can write arbitrary data into the ring.

Yes, will do. Thanks!

author
#4 lore raw

PATCH 2/2 KVM: selftests: TDX: Test dirty ring on a gmemfd slot

In reply to Yan Zhao

Slots with the KVM_MEM_GUEST_MEMFD flag do not support dirty bit tracking. However, it is still possible for userspace to trigger the resetting of the dirty ring, asking KVM to reset SPTEs in the mirror root by clearing the accessed bit or write bit.

Test this to ensure that TDs are not negatively impacted.

---
 tools/testing/selftests/kvm/Makefile          |   1 +
 .../selftests/kvm/x86_64/tdx_dirty_ring.c     | 231 ++++++++++++++++++
 2 files changed, 232 insertions(+)
 create mode 100644 tools/testing/selftests/kvm/x86_64/tdx_dirty_ring.c
tools/testing/selftests/kvm/Makefile +1 -0
index 46fd2194add3..46ee465a0443 100644
@@ -157,6 +157,7 @@ TEST_GEN_PROGS_x86_64 += pre_fault_memory_test
 TEST_GEN_PROGS_x86_64 += x86_64/tdx_vm_tests
 TEST_GEN_PROGS_x86_64 += x86_64/tdx_shared_mem_test
 TEST_GEN_PROGS_x86_64 += x86_64/tdx_upm_test
+TEST_GEN_PROGS_x86_64 += x86_64/tdx_dirty_ring
 
 # Compiled outputs used by test targets
 TEST_GEN_PROGS_EXTENDED_x86_64 += x86_64/nx_huge_pages_test
tools/testing/selftests/kvm/x86_64/tdx_dirty_ring.c new file +231 -0

231 lines not on this page · raw message

Signature · 1 line
2.43.2