KVM: arm64: nv: MMU fixes for 7.2

PATCH New Oliver Upton 1 participant 6 messages Jun 2

Subsystems (3)

Stable branch arm64 KVM arm64

lore

Share & tools

lore

Oliver Upton

author maintainer

2026-06-02 23:54 UTC #0 lore raw

cover KVM: arm64: nv: MMU fixes for 7.2

I've accumulated a small pile of nested MMU fixes as part of a larger series, flushing those out as a separate thing that can land for 7.2. As is the trend these days, some of these bugs were found by sashiko review and others were found in the process of fixing the AI bug reports.

Oliver Upton (5):
KVM: arm64: Don't leak PFN when kvm_translate_vncr() races MMU
notifier
KVM: arm64: nv: Fully update VNCR fixmap state in kvm_translate_vncr()
KVM: arm64: nv: Inject SEA TTW when desc update can't write to GPA
KVM: arm64: Restart instruction upon race in __kvm_at_s12()
KVM: arm64: nv: Restart stage-1 walk if stage-2 desc update fails

 arch/arm64/kvm/at.c     | 14 ++++++++++++--
 arch/arm64/kvm/nested.c | 38 ++++++++++++++++++++++++++------------
 2 files changed, 38 insertions(+), 14 deletions(-)

base-commit: 5d6919055dec134de3c40167a490f33c74c12581

Signature · 1 line

2.47.3

Oliver Upton

author maintainer

2026-06-02 23:54 UTC #1 lore raw

PATCH 1/5 KVM: arm64: Don't leak PFN when kvm_translate_vncr() races MMU notifier

In the case that kvm_translate_vncr() races with an MMU notifier the early return does not release a reference on the faulted in PFN. Add the necessary call to kvm_release_faultin_page() for the unused PFN.

Cc [email protected]

Fixes 069a05e535496 ("KVM: arm64: nv: Handle VNCR_EL2-triggered faults")

Reported-by Sashiko (local):gemini-3.1-pro

Signed-off-by Oliver Upton <[email protected]>

---
 arch/arm64/kvm/nested.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)

arch/arm64/kvm/nested.c +3 -1

index 883b6c1008fb..4fa82e96454d 100644
@@ -1326,8 +1326,10 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu, bool *is_gmem)
 	}
 
 	scoped_guard(write_lock, &vcpu->kvm->mmu_lock) {
-		if (mmu_invalidate_retry(vcpu->kvm, mmu_seq))
+		if (mmu_invalidate_retry(vcpu->kvm, mmu_seq)) {
+			kvm_release_faultin_page(vcpu->kvm, page, true, false);
 			return -EAGAIN;
+		}
 
 		vt->gva = va;
 		vt->hpa = pfn << PAGE_SHIFT;

Signature · 1 line

2.47.3

Oliver Upton

author maintainer

2026-06-02 23:54 UTC #2 lore raw

PATCH 2/5 KVM: arm64: nv: Fully update VNCR fixmap state in kvm_translate_vncr()

In reply to Oliver Upton

kvm_translate_vncr() first invalidates the pseudo-TLB entry and corresponding fixmap in anticipation of installing a new translation.

While the fixmap invalidation does clear the mapping from host stage-1, it does not clear the L1_VNCR_MAPPED flag. Depending on the state of the VNCR TLB at vcpu_put(), this could potentially precipitate a BUG_ON() if vt->cpu is reset.

Share a helper with kvm_vcpu_put_hw_mmu(), ensuring that KVM's view of the VNCR fixmap is in sync with the state of the VNCR TLB. Give it a slightly verbose name to make it obvious that it is meant to be used local to a CPU, unlike other VNCR TLB maintenance.

Fixes 069a05e535496 ("KVM: arm64: nv: Handle VNCR_EL2-triggered faults")

Signed-off-by Oliver Upton <[email protected]>

---
 arch/arm64/kvm/nested.c | 27 +++++++++++++++++----------
 1 file changed, 17 insertions(+), 10 deletions(-)

arch/arm64/kvm/nested.c +17 ~~-10~~

index 4fa82e96454d..d0545144eaac 100644
@@ -797,18 +797,24 @@ void kvm_vcpu_load_hw_mmu(struct kvm_vcpu *vcpu)
 	}
 }
 
+static void this_cpu_reset_vncr_fixmap(struct kvm_vcpu *vcpu)
+{
+	if (!host_data_test_flag(L1_VNCR_MAPPED))
+		return;
+
+	BUG_ON(vcpu->arch.vncr_tlb->cpu != smp_processor_id());
+	BUG_ON(is_hyp_ctxt(vcpu));
+
+	clear_fixmap(vncr_fixmap(vcpu->arch.vncr_tlb->cpu));
+	vcpu->arch.vncr_tlb->cpu = -1;
+	host_data_clear_flag(L1_VNCR_MAPPED);
+	atomic_dec(&vcpu->kvm->arch.vncr_map_count);
+}
+
 void kvm_vcpu_put_hw_mmu(struct kvm_vcpu *vcpu)
 {
 	/* Unconditionally drop the VNCR mapping if we have one */
-	if (host_data_test_flag(L1_VNCR_MAPPED)) {
-		BUG_ON(vcpu->arch.vncr_tlb->cpu != smp_processor_id());
-		BUG_ON(is_hyp_ctxt(vcpu));
-
-		clear_fixmap(vncr_fixmap(vcpu->arch.vncr_tlb->cpu));
-		vcpu->arch.vncr_tlb->cpu = -1;
-		host_data_clear_flag(L1_VNCR_MAPPED);
-		atomic_dec(&vcpu->kvm->arch.vncr_map_count);
-	}
+	this_cpu_reset_vncr_fixmap(vcpu);
 
 	/*
 	 * Keep a reference on the associated stage-2 MMU if the vCPU is

@@ -1282,7 +1288,8 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu, bool *is_gmem)
 	 * We also prepare the next walk wilst we're at it.
 	 */
 	scoped_guard(write_lock, &vcpu->kvm->mmu_lock) {
-		invalidate_vncr(vt);
+		this_cpu_reset_vncr_fixmap(vcpu);
+		vt->valid = false;
 
 		vt->wi = (struct s1_walk_info) {
 			.regime	= TR_EL20,

Signature · 1 line

2.47.3

Oliver Upton

author maintainer

2026-06-02 23:54 UTC #3 lore raw

PATCH 3/5 KVM: arm64: nv: Inject SEA TTW when desc update can't write to GPA

In reply to Oliver Upton

Similar to the handling of descriptor reads, inject an SEA during TTW when the descriptor access fails for reasons other than a race, such as a read-only memslot or a bad HVA.

Fixes bff8aa213dee ("KVM: arm64: Implement HW access flag management in stage-1 SW PTW")

Fixes e4c7dfac2f1a ("KVM: arm64: nv: Implement HW access flag management in stage-2 SW PTW")

Signed-off-by Oliver Upton <[email protected]>

---
 arch/arm64/kvm/at.c     | 6 +++++-
 arch/arm64/kvm/nested.c | 7 ++++++-
 2 files changed, 11 insertions(+), 2 deletions(-)

arch/arm64/kvm/at.c +5 -1

index 9f8f0ae8e86e..119a603e636e 100644
@@ -521,8 +521,12 @@ static int walk_s1(struct kvm_vcpu *vcpu, struct s1_walk_info *wi,
 		}
 
 		ret = kvm_swap_s1_desc(vcpu, ipa, desc, new_desc, wi);
-		if (ret)
+		if (ret == -EAGAIN)
 			return ret;
+		if (ret) {
+			fail_s1_walk(wr, ESR_ELx_FSC_SEA_TTW(level), false);
+			return ret;
+		}
 
 		desc = new_desc;
 	}

arch/arm64/kvm/nested.c +6 -1

index d0545144eaac..f8d3f3a72328 100644
@@ -352,8 +352,13 @@ static int walk_nested_s2_pgd(struct kvm_vcpu *vcpu, phys_addr_t ipa,
 
 	if (new_desc != desc) {
 		ret = swap_guest_s2_desc(vcpu, paddr, desc, new_desc, wi);
-		if (ret)
+		if (ret == -EAGAIN)
 			return ret;
+		if (ret) {
+			out->esr = ESR_ELx_FSC_SEA_TTW(level);
+			out->desc = desc;
+			return 1;
+		}
 
 		desc = new_desc;
 	}

Signature · 1 line

2.47.3

Oliver Upton

author maintainer

2026-06-02 23:54 UTC #4 lore raw

PATCH 4/5 KVM: arm64: Restart instruction upon race in __kvm_at_s12()

In reply to Oliver Upton

__kvm_at_s*() are expected to return -EAGAIN if the page table walk raced with a concurrent update to a page table descriptor, which is interpreted as a signal to restart the trapping instruction.

While this mostly works, __kvm_at_s12() silently eats the return from __kvm_at_s1e01() and consumes an uninitialized PAR value. Propagate the nonzero return instead.

Fixes 92c6443222ca ("KVM: arm64: Propagate PTW errors up to AT emulation")

Signed-off-by Oliver Upton <[email protected]>

---
 arch/arm64/kvm/at.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)

arch/arm64/kvm/at.c +4 -1

index 119a603e636e..6cc5892023dd 100644
@@ -1557,7 +1557,10 @@ int __kvm_at_s12(struct kvm_vcpu *vcpu, u32 op, u64 vaddr)
 		return 0;
 	}
 
-	__kvm_at_s1e01(vcpu, op, vaddr);
+	ret = __kvm_at_s1e01(vcpu, op, vaddr);
+	if (ret)
+		return ret;
+
 	par = vcpu_read_sys_reg(vcpu, PAR_EL1);
 	if (par & SYS_PAR_EL1_F)
 		return 0;

Signature · 1 line

2.47.3

Oliver Upton

author maintainer

2026-06-02 23:54 UTC #5 lore raw

PATCH 5/5 KVM: arm64: nv: Restart stage-1 walk if stage-2 desc update fails

In reply to Oliver Upton

kvm_walk_nested_s2() returns -EAGAIN as an indication that an underlying descriptor update fails due to a race. The expectation is that the caller restart translation, yet walk_s1() actually synthesizes an abort.

Propagate the -EAGAIN return out of walk_s1(), relying on callers to restart the translation fetch.

Fixes e4c7dfac2f1a ("KVM: arm64: nv: Implement HW access flag management in stage-2 SW PTW")

Signed-off-by Oliver Upton <[email protected]>

---
 arch/arm64/kvm/at.c | 3 +++
 1 file changed, 3 insertions(+)

arch/arm64/kvm/at.c +3 -0

index 6cc5892023dd..4d4285e60fce 100644
@@ -423,6 +423,9 @@ static int walk_s1(struct kvm_vcpu *vcpu, struct s1_walk_info *wi,
 
 		if (wi->s2) {
 			ret = kvm_walk_nested_s2(vcpu, ipa, &s2_trans);
+			if (ret == -EAGAIN)
+				return ret;
+
 			if (ret) {
 				fail_s1_walk(wr,
 					     (s2_trans.esr & ~ESR_ELx_FSC_LEVEL) | level,

Signature · 1 line

2.47.3