KVM: arm64: nv: Hold mmu_lock when invalidating VNCR SW-TLB before translating
Marc Zyngier
maintainerPATCH 1/1 KVM: arm64: nv: Hold mmu_lock when invalidating VNCR SW-TLB before translating
When translating a VNCR translation fault, we start by marking the current SW-managed TLB as invalid, so that we can populate it in place. This is, however, done without the mmu_lock held.
A consequence of this is that another CPU dealing with TLBI emulation can observe a translation still flagged as valid, but with invalid walk results (such as pgshift being 0). Bad things can result from this, such as a BUG() in pgshift_level_to_ttl().
Fix it by taking the mmu_lock for write to perform this local invalidation, and use invalidate_vncr() instead of open-coding the write to the 'valid' flag.
--- arch/arm64/kvm/nested.c | 25 ++++++++++++++++++------- 1 file changed, 18 insertions(+), 7 deletions(-)
arch/arm64/kvm/nested.c +18 -7
@@ -1179,13 +1179,24 @@ static int kvm_translate_vncr(struct kvm_vcpu *vcpu) vt = vcpu->arch.vncr_tlb; - vt->wi = (struct s1_walk_info ) { - .regime =TR_EL20 , - .as_el0 = false, - .pan = false, - }; - vt->wr = (struct s1_walk_result ){}; - vt->valid = false; +/* +* If we're about to walk the EL2 S1 PTs, we must invalidate the +* current TLB, as it could be sampled from another vcpu doing a +* TLBI *IS. A real CPU wouldn't do that, but we only keep a single +* translation, so not much of a choice. +* +* We also prepare the next walk wilst we're at it. +*/ + scoped_guard(write_lock, &vcpu->kvm->mmu_lock) { +invalidate_vncr (vt); + + vt->wi = (struct s1_walk_info ) { + .regime =TR_EL20 , + .as_el0 = false, + .pan = false, + }; + vt->wr = (struct s1_walk_result ){}; + }guard (srcu)(&vcpu->kvm->srcu);
Signature · 1 line
2.39.2
Quoted 9 lines · Marc Zyngier
When translating a VNCR translation fault, we start by marking the current SW-managed TLB as invalid, so that we can populate it in place. This is, however, done without the
mmu_lockheld.A consequence of this is that another CPU dealing with TLBI emulation can observe a translation still flagged as valid, but with invalid walk results (such as pgshift being 0). Bad things can result from this, such as a
BUG()inpgshift_level_to_ttl().
Quoted 6 lines
Fix it by taking the
mmu_lockfor write to perform this local invalidation, and useinvalidate_vncr()instead of open-coding the write to the 'valid' flag.Fixes 069a05e535496 ("KVM: arm64: nv: Handle VNCR_EL2-triggered faults")Signed-off-by Marc Zyngier <[email protected]>
Thanks,
Oliver
Quoted 10 lines · Marc Zyngier
When translating a VNCR translation fault, we start by marking the current SW-managed TLB as invalid, so that we can populate it in place. This is, however, done without the
mmu_lockheld.A consequence of this is that another CPU dealing with TLBI emulation can observe a translation still flagged as valid, but with invalid walk results (such as pgshift being 0). Bad things can result from this, such as a
BUG()inpgshift_level_to_ttl().[...]
Applied to next, thanks!
[1/1] KVM: arm64: nv: Hold mmu_lock when invalidating VNCR SW-TLB before translating
commit: d43548f422f27219eff5ce1897336af2c4f15091
Cheers,
M.
Signature · 1 line
Without deviation from the norm, progress is not possible.