Skip to content

KVM: arm64: VNCR TLB invalidation fixes

Marc Zyngier kvm kvmarm linux-arm-kernel sashiko-reviews stable 2 versions – Applied

Status from this message: Applied to next, thanks!

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v2 Latest8 patches 13 repliesApplied
  2. v16 patches 13 repliesSuperseded

Patches in v2 Applied

#Subject and filesSizeRepliesApplied
1/8KVM: arm64: Remove VM-wide VNCR mapping counterarch/arm64/include/asm/kvm_host.h arch/arm64/kvm/hyp/vhe/switch.c arch/arm64/kvm/nested.c+1 -83 replies
2/8KVM: arm64: Handle negative S1 walk levels in VNCR TLB size evaluationarch/arm64/kvm/nested.c+19 -72 replies
3/8KVM: arm64: Consider SCTLR_EL2.M when mapping the L1 VNCR pagearch/arm64/include/asm/kvm_nested.h arch/arm64/kvm/at.c arch/arm64/kvm/nested.c+11 -20 replies
4/8KVM: arm64: Correctly handle end of VA space TLBI invalidationarch/arm64/kvm/nested.c+19 -244 replies
5/8KVM: arm64: Handle VNCR TLB invalidation race with vcpu_put() VNCR unmappingarch/arm64/kvm/nested.c+32 -102 replies
6/8KVM: arm64: Sign-extend VA for range-based TLBI invalidationarch/arm64/include/asm/kvm_nested.h arch/arm64/kvm/sys_regs.c+18 -00 replies
7/8KVM: arm64: Make VNCR invalidation participate in MMU invalidation retryarch/arm64/kvm/nested.c+18 -30 replies
8/8KVM: arm64: Add VNCR TLB tracking againarch/arm64/include/asm/kvm_host.h arch/arm64/kvm/hyp/vhe/switch.c arch/arm64/kvm/nested.c+41 -91 reply

Reviews trailers per patch

3 of 8 patches have a Reviewed-by or Acked-by.

PatchYuan YaoLorenzo Stoakes (ARM)
1/8R-b A-b
2/8
3/8
4/8R-b
5/8R-b
6/8
7/8
8/8

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v2 thread

Cover letter [PATCH v2 0/8] KVM: arm64: VNCR TLB invalidation fixes

Prompted by a patch [0] from Hyunwoo Kim which was addressing a pretty annoying corner case, I spent some time looking at what was wrong in our VNCR TLBI code. And there was a few things to say about it...

- The TLB tracking to get into the slow path is wrong. It tracks what is mapped, but not the established TLBs. Kind of annoying.

- Handling IPA invalidation when the TLB was established with the S1 MMU disabled is failing for a number of reasons. This is the bug that Hyunwoo Kim found, but I decided to address it at its root rather than just fixing the symptoms.

- Deciding to run with a VNCR TLB doesn't take the state of SCTLR_EL2.M into account. Yes, it is a special sort of sport to run a hypervisor with its MMU disabled...

- TLB invalidation by VA targeting the last page/block of TTBR1_EL2 is dropped on the floor, because I cannot count. This was amusing. Not.

Additionally, Sashiko kindly provided some extra horrors to look at:

- A missing sign extension for range invalidation results in the wrong VA range being considered.

- TLBI doesn't participate in the general MMU invalidation retry machinery, meaning that VNCR faults and TLBIs race in an uncontrolled way.

- TLBI and vcpu_put() can race badly, leading to a TOCTOU pattern which results in either a BUG_ON() or a call to vncr_fixmap(-1).

I have fixes for each of these issues, all stable candidates.

On top of that, I have a patch reintroducing the tracking that the first patch removes, this time in a way that is actually functional. Or at least I think it is...

I'd like to thank Hyunwoo Kim for their initial patch and for providing a reproducer that helped me finding these issues by running it at multiple levels of nesting.

It is now too late for 7.2, so let's try to make this 7.3 material (hence the early repost).

* From v1 [1]

- Added sign extension to range invalidation (Sashiko)

- Use mmu_invalidate_seq to resolve fault vs TLBI races (Sashiko)

- Speculatively bump the TLB refcount on fault to make sure the TLBI takes the slow path when we are going for a S1 walk

- Clarify the check for the TLB matching for the S1 MMU state

- Extra sanitisation added on the S2 invalidation path

- Applied RBs from Yao Yuan to the patches that didn't change, with thanks.

[0] https://lore.kernel.org/r/ameGoxbn2wzBq2kL@v4bel
[1] https://lore.kernel.org/r/[email protected]

Marc Zyngier (8):
KVM: arm64: Remove VM-wide VNCR mapping counter
KVM: arm64: Handle negative S1 walk levels in VNCR TLB size evaluation
KVM: arm64: Consider SCTLR_EL2.M when mapping the L1 VNCR page
KVM: arm64: Correctly handle end of VA space TLBI invalidation
KVM: arm64: Handle VNCR TLB invalidation race with vcpu_put() VNCR
unmapping
KVM: arm64: Sign-extend VA for range-based TLBI invalidation
KVM: arm64: Make VNCR invalidation participate in MMU invalidation
retry
KVM: arm64: Add VNCR TLB tracking again

 arch/arm64/include/asm/kvm_host.h   |   4 +-
 arch/arm64/include/asm/kvm_nested.h |  14 +++
 arch/arm64/kvm/at.c                 |   2 -
 arch/arm64/kvm/hyp/vhe/switch.c     |  10 +-
 arch/arm64/kvm/nested.c             | 173 +++++++++++++++++++---------
 arch/arm64/kvm/sys_regs.c           |  11 ++
 6 files changed, 155 insertions(+), 59 deletions(-)
Signature · 1 line
2.47.3