Skip to contentSkip to conversation

KVM: x86/mmu: WARN and flush TLBs if TDP MMU replaces present SPTE with MMIO SPTE

PATCH Applied Sean Christopherson 2 participants 4 messages Oct 22 – Nov 13

Subsystems (3)
lore
Share & tools
lore

Sean Christopherson

author maintainer
#0 lore raw

PATCH 1/1 KVM: x86/mmu: WARN and flush TLBs if TDP MMU replaces present SPTE with MMIO SPTE

If KVM is buggy and manages to replace a shadow-present SPTE with an MMIO SPTE in the TDP MMU, WARN but emit a TLB flush to prevent a potential use- after-free. It should be impossible for KVM to resolve a GFN to a memslot and then later find no memslot, but due to flaws in KVM's handling of GFNs that can be generated by the guest[*], KVM can/wrap truncate GFNs when walking SPTEs and resolve two different GFNs to the same SPTE.

Note, this is NOT a fix for the GFN truncation issues, it is purely a stopgap that also happens to be valuable long-term hardening.

---
 arch/x86/kvm/mmu/tdp_mmu.c | 1 +
 1 file changed, 1 insertion(+)
arch/x86/kvm/mmu/tdp_mmu.c +1 -0
index 44dad106fad1..2735cbb21293 100644
@@ -1151,6 +1151,7 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu,
 		return RET_PF_RETRY;
 	else if (is_shadow_present_pte(iter->old_spte) &&
 		 (!is_last_spte(iter->old_spte, iter->level) ||
+		  WARN_ON_ONCE(!is_shadow_present_pte(new_spte)) ||
 		  WARN_ON_ONCE(leaf_spte_change_needs_tlb_flush(iter->old_spte, new_spte))))
 		kvm_flush_remote_tlbs_gfn(vcpu->kvm, iter->gfn, iter->level);
 

base-commit: 814607cd40df80d5727957adaa857e0feca43497

Signature · 1 line
2.56.0.218.g1ec904b173-goog
#1 lore raw
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] KVM: x86/mmu: tdp_mmu_map_handle_target_level() flushes untruncated GFN on MMIO overwrite --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] KVM: x86/mmu: tdp_mmu_map_handle_target_level() flushes untruncated GFN on MMIO overwrite --

commit 409b2fbe972da1945be9a64a0bcdab640e996156
Author: Sean Christopherson <[email protected]>

KVM: x86/mmu: WARN and flush TLBs if TDP MMU replaces present SPTE with MMIO SPTE

This commit adds a warning and remote tlb flush in tdp_mmu_map_handle_target_level() if a shadow-present spte is replaced with a non-shadow-present mmio spte due to gfn truncation or wrapping during the page table walk.

arch/x86/kvm/mmu/tdp_mmu.c static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu, · line 1156

Quoted 11 lines
arch/x86/kvm/mmu/tdp_mmu.c +1 -0
index 44dad106fad19..2735cbb212937 100644
@@ -1151,6 +1151,7 @@ static int tdp_mmu_map_handle_target_level(struct kvm_vcpu *vcpu,
 		return RET_PF_RETRY;
 	else if (is_shadow_present_pte(iter->old_spte) &&
 		 (!is_last_spte(iter->old_spte, iter->level) ||
+		  WARN_ON_ONCE(!is_shadow_present_pte(new_spte)) ||
 		  WARN_ON_ONCE(leaf_spte_change_needs_tlb_flush(iter->old_spte, new_spte))))
 		kvm_flush_remote_tlbs_gfn(vcpu->kvm, iter->gfn, iter->level);

[Severity: High]
When a shadow-present spte mapping a valid memslot gfn is overwritten with a non-shadow-present mmio spte due to gfn truncation in the tdp iterator, does flushing iter->gfn via kvm_flush_remote_tlbs_gfn() flush the wrong address when range-based tlb invalidation is used?

Suppose a guest with a 4-level root first faults in a valid memslot at a lower gfn (caching that translation in the tlb), and then triggers a fault on an aliased gfn with upper bits set above the root level where fault->slot is null.

During kvm_tdp_mmu_map(), tdp_iter_restart() and try_step_down() compute iter->gfn via gfn_round_for_level():

arch/x86/kvm/mmu/mmu_internal.h:gfn_round_for_level() {
	return gfn & -KVM_PAGES_PER_HPAGE(level);
}

which only masks low bits below iter->level and retains the untruncated upper bits in iter->gfn. Meanwhile, tdp_iter_refresh_sptep() indexes the page table by shifting and masking out the upper bits:

arch/x86/kvm/mmu/tdp_iter.c:tdp_iter_refresh_sptep() {
	iter->sptep = iter->pt_path[iter->level - 1] +
		SPTE_INDEX((iter->gfn | iter->gfn_bits) << PAGE_SHIFT, iter->level);
}

resolving iter->sptep to the existing spte for the lower gfn.

When tdp_mmu_map_handle_target_level() overwrites iter->old_spte with the mmio spte and calls kvm_flush_remote_tlbs_gfn(vcpu->kvm, iter->gfn, iter->level), it passes the untruncated faulting iter->gfn. In kvm_flush_remote_tlbs_range():

virt/kvm/kvm_main.c:kvm_flush_remote_tlbs_range() {
	if (!kvm_arch_flush_remote_tlbs_range(kvm, gfn, nr_pages))
		return;
	...
	kvm_flush_remote_tlbs(kvm);
}

if range-based flushing is supported and succeeds (for example, via hv_flush_remote_tlbs_range()), only the untruncated iter->gfn is invalidated and kvm_flush_remote_tlbs() is skipped.

Because the spte is now a non-shadow-present mmio spte, subsequent mmu notifier invalidations in tdp_mmu_zap_leafs() will skip it without flushing tlbs, leaving the cached tlb entry for the original memslot gfn pointing to a host pfn that can be freed.

Should this path call kvm_flush_remote_tlbs(vcpu->kvm) (or mask iter->gfn to the root's address space) when !is_shadow_present_pte(new_spte) triggers?

Quoted 1 line

taker

Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
author maintainer
#2 lore raw
Quoted 4 lines · On Thu, Oct 22, 2026, [email protected] wrote:

When a shadow-present spte mapping a valid memslot gfn is overwritten with a non-shadow-present mmio spte due to gfn truncation in the tdp iterator, does flushing iter->gfn via kvm_flush_remote_tlbs_gfn() flush the wrong address when range-based tlb invalidation is used?

...

Quoted 2 lines

Should this path call kvm_flush_remote_tlbs(vcpu->kvm) (or mask iter->gfn to the root's address space) when !is_shadow_present_pte(new_spte) triggers?

Eh, I don't think we should bother. The right fix is prevent the truncation in the first place. If kvm_flush_remote_tlbs_gfn() weren't limited to KVM-on-HyperV I would care, especially since this requires userspace to effectively misconfigure the VM. I.e. someone would have to go pretty far out of their way to run afoul of this exact scenario.

author maintainer Applied
#3 lore raw
In reply to Sean Christopherson
Quoted 8 lines · Sean Christopherson

If KVM is buggy and manages to replace a shadow-present SPTE with an MMIO SPTE in the TDP MMU, WARN but emit a TLB flush to prevent a potential use- after-free. It should be impossible for KVM to resolve a GFN to a memslot and then later find no memslot, but due to flaws in KVM's handling of GFNs that can be generated by the guest[*], KVM can/wrap truncate GFNs when walking SPTEs and resolve two different GFNs to the same SPTE.

[...]

Applied to kvm-x86 mmu, thanks!

[1/1] KVM: x86/mmu: WARN and flush TLBs if TDP MMU replaces present SPTE with MMIO SPTE

      https://github.com/kvm-x86/linux/commit/94b5cb50abec

--
https://github.com/kvm-x86/linux/tree/next