Skip to contentSkip to conversation

KVM: arm64: nv: Hold mmu_lock when invalidating VNCR SW-TLB before translating

PATCH Applied Marc Zyngier 2 participants 3 messages May 20, 2025 – May 21, 2025

Subsystems (2)
lore
Share & tools
lore

Marc Zyngier

author maintainer
#0 lore raw

PATCH 1/1 KVM: arm64: nv: Hold mmu_lock when invalidating VNCR SW-TLB before translating

When translating a VNCR translation fault, we start by marking the current SW-managed TLB as invalid, so that we can populate it in place. This is, however, done without the mmu_lock held.

A consequence of this is that another CPU dealing with TLBI emulation can observe a translation still flagged as valid, but with invalid walk results (such as pgshift being 0). Bad things can result from this, such as a BUG() in pgshift_level_to_ttl().

Fix it by taking the mmu_lock for write to perform this local invalidation, and use invalidate_vncr() instead of open-coding the write to the 'valid' flag.

Fixes 069a05e535496 ("KVM: arm64: nv: Handle VNCR_EL2-triggered faults")
---
 arch/arm64/kvm/nested.c | 25 ++++++++++++++++++-------
 1 file changed, 18 insertions(+), 7 deletions(-)
arch/arm64/kvm/nested.c +18 -7
index 6a9fd4e0e789c..56b732003caa7 100644
@@ -1179,13 +1179,24 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu)
 
 	vt = vcpu->arch.vncr_tlb;
 
-	vt->wi = (struct s1_walk_info) {
-		.regime	= TR_EL20,
-		.as_el0	= false,
-		.pan	= false,
-	};
-	vt->wr = (struct s1_walk_result){};
-	vt->valid = false;
+	/*
+	 * If we're about to walk the EL2 S1 PTs, we must invalidate the
+	 * current TLB, as it could be sampled from another vcpu doing a
+	 * TLBI *IS. A real CPU wouldn't do that, but we only keep a single
+	 * translation, so not much of a choice.
+	 *
+	 * We also prepare the next walk wilst we're at it.
+	 */
+	scoped_guard(write_lock, &vcpu->kvm->mmu_lock) {
+		invalidate_vncr(vt);
+
+		vt->wi = (struct s1_walk_info) {
+			.regime	= TR_EL20,
+			.as_el0	= false,
+			.pan	= false,
+		};
+		vt->wr = (struct s1_walk_result){};
+	}
 
 	guard(srcu)(&vcpu->kvm->srcu);
 
Signature · 1 line
2.39.2
Reviewed-by on v1 1/1
#1 lore raw
Quoted 9 lines · Marc Zyngier

When translating a VNCR translation fault, we start by marking the current SW-managed TLB as invalid, so that we can populate it in place. This is, however, done without the mmu_lock held.

A consequence of this is that another CPU dealing with TLBI emulation can observe a translation still flagged as valid, but with invalid walk results (such as pgshift being 0). Bad things can result from this, such as a BUG() in pgshift_level_to_ttl().

Quoted 6 lines

Fix it by taking the mmu_lock for write to perform this local invalidation, and use invalidate_vncr() instead of open-coding the write to the 'valid' flag.

Fixes 069a05e535496 ("KVM: arm64: nv: Handle VNCR_EL2-triggered faults")

Thanks,
Oliver

author maintainer Applied
#2 lore raw
In reply to Marc Zyngier
Quoted 10 lines · Marc Zyngier

When translating a VNCR translation fault, we start by marking the current SW-managed TLB as invalid, so that we can populate it in place. This is, however, done without the mmu_lock held.

A consequence of this is that another CPU dealing with TLBI emulation can observe a translation still flagged as valid, but with invalid walk results (such as pgshift being 0). Bad things can result from this, such as a BUG() in pgshift_level_to_ttl().

[...]

Applied to next, thanks!

[1/1] KVM: arm64: nv: Hold mmu_lock when invalidating VNCR SW-TLB before translating

      commit: d43548f422f27219eff5ce1897336af2c4f15091

Cheers,

	M.
Signature · 1 line
Without deviation from the norm, progress is not possible.