KVM: arm64: nv: Shadow S2 life-cycle fixes

We free the shadow S2 structures from kvm_arch_flush_shadow_all(), which is a Bad Idea(tm). Freeing the page tables is fair game (this is what this callback is for), but freeing the container that could still be referenced by another part of the system is not great.

Instead, grow separate destructors that gets called when we tear the VM down for good. From there, we can nuke both the individual MMUs as well as the global array that points to them, safe in the knowledge that the vcpus themselves have been destroyed already.

Fixes 4f128f8e1aaac ("KVM: arm64: nv: Support multiple nested Stage-2 mmu structures")

Signed-off-by Marc Zyngier <[email protected]>

Cc [email protected]

---
 arch/arm64/include/asm/kvm_nested.h |  1 +
 arch/arm64/kvm/arm.c                |  4 ++--
 arch/arm64/kvm/nested.c             | 15 ++++++++++-----
 3 files changed, 13 insertions(+), 7 deletions(-)

arch/arm64/include/asm/kvm_nested.h +1 -0

index 5b8edb2e8a87d..586026e859030 100644
@@ -67,6 +67,7 @@ static inline u64 translate_ttbr0_el2_to_ttbr0_el1(u64 ttbr0)
 extern bool forward_smc_trap(struct kvm_vcpu *vcpu);
 extern bool forward_debug_exception(struct kvm_vcpu *vcpu);
 extern int kvm_init_nested(struct kvm *kvm);
+extern void kvm_destroy_nested(struct kvm *kvm);
 extern int kvm_vcpu_init_nested(struct kvm_vcpu *vcpu);
 extern void kvm_init_nested_s2_mmu(struct kvm_s2_mmu *mmu);
 extern struct kvm_s2_mmu *lookup_s2_mmu(struct kvm_vcpu *vcpu);

arch/arm64/kvm/arm.c +2 -2

index 7607173c1a40c..2b069c6440669 100644
@@ -269,7 +269,7 @@ int kvm_arch_init_vm(struct kvm *kvm, unsigned long type)
 
 err_uninit_mmu:
 	kvm_uninit_stage2_mmu(kvm);
-	kvfree(kvm->arch.nested_mmus);
+	kvm_destroy_nested(kvm);
 err_free_cpumask:
 	free_cpumask_var(kvm->arch.supported_cpus);
 err_unshare_kvm:

@@ -327,7 +327,7 @@ void kvm_arch_destroy_vm(struct kvm *kvm)
 
 	kvm_unshare_hyp(kvm, kvm + 1);
 
-	kvfree(kvm->arch.nested_mmus);
+	kvm_destroy_nested(kvm);
 	kvm_arm_teardown_hypercalls(kvm);
 }
 

arch/arm64/kvm/nested.c +9 -0

index 05266f8b87304..00580ba6e8ba0 100644
@@ -55,6 +55,15 @@ int kvm_init_nested(struct kvm *kvm)
 	return kvm->arch.nested_mmus ? 0 : -ENOMEM;
 }
 
+void kvm_destroy_nested(struct kvm *kvm)
+{
+	for (int i = 0; i < kvm->arch.nested_mmus_size; i+= S2_MMU_PER_VCPU)
+		kvfree(kvm->arch.nested_mmus[i]);
+
+	kvm->arch.nested_mmus_size = 0;
+	kvfree(kvm->arch.nested_mmus);
+}
+
 static int init_nested_s2_mmu(struct kvm *kvm, struct kvm_s2_mmu *mmu)
 {
 	/*

arch/arm64/kvm/nested.c +0 -1

 	kvm_uninit_stage2_mmu(kvm);
 }
 
--

2.47.3

Quoted 4 lines · Marc Zyngier

We free the shadow S2 structures from kvm_arch_flush_shadow_all(), which is a Bad Idea(tm). Freeing the page tables is fair game (this is what this callback is for), but freeing the container that could still be referenced by another part of the system is not great.

Yes.

Quoted 6 lines

Instead, grow separate destructors that gets called when we tear the VM down for good. From there, we can nuke both the individual MMUs as well as the global array that points to them, safe in the knowledge that the vcpus themselves have been destroyed already.

Quoted 2 lines

Fixes 4f128f8e1aaac ("KVM: arm64: nv: Support multiple nested Stage-2 mmu structures")

Signed-off-by Marc Zyngier <[email protected]>

LGTM so:

Reviewed-by Lorenzo Stoakes (ARM) <[email protected]>

A couple thoughts/questions below.

arch/arm64/kvm/nested.c void kvm_nested_s2_flush(struct kvm *kvm) · line 1322

Quoted 6 lines

arch/arm64/kvm/nested.c +1 -1

@@ -1310,16 +1319,12 @@ void kvm_nested_s2_flush(struct kvm *kvm)
 
 void kvm_arch_flush_shadow_all(struct kvm *kvm)
 {
-	for (int i = kvm->arch.nested_mmus_size - 1; i >= 0; i--) {
+	for (int i = 0; i < kvm->arch.nested_mmus_size; i++) {

Hmm why this was in reverse before? :) I guess some product of the kvfree() bit or maybe something else?

Quoted 9 lines · Lorenzo Stoakes (ARM)

arch/arm64/kvm/nested.c void kvm_nested_s2_flush(struct kvm *kvm) · line 1322

Quoted 6 lines

arch/arm64/kvm/nested.c +1 -1

@@ -1310,16 +1319,12 @@ void kvm_nested_s2_flush(struct kvm *kvm)
 
 void kvm_arch_flush_shadow_all(struct kvm *kvm)
 {
-	for (int i = kvm->arch.nested_mmus_size - 1; i >= 0; i--) {
+	for (int i = 0; i < kvm->arch.nested_mmus_size; i++) {

Hmm why this was in reverse before? :) I guess some product of the kvfree() bit or maybe something else?

We allocate s2_mmus in S2_MMU_PER_VCPU chunks. Which means that it complicates the freeing of the these structures, as they can only be freed once all S2 PTs of that chunk have been freed. We have three options:

- scan forward, and use complicated logic to work out that you have freed the last PTs of a chunk, freeing with a negative offset from the current point in the loop. Works, but hard to reason about in fewer than 3 seconds.

- scan backward, use simpler logic to ensure you have reached the beginning of a chunk, nuke it.

- have two loops, one for the PTs, one for the MMUs. That's what we end-up with due to the different garbage collection phases.

Cheers,

	M.

--
Without deviation from the norm, progress is not possible.

arch/arm64/kvm/nested.c +6 -0

+		kvfree(kvm->arch.nested_mmus[i]);
+
+	kvm->arch.nested_mmus_size = 0;
+	kvfree(kvm->arch.nested_mmus);
+}
+

On Fri, Aug 21, 2026 at 05:18:29PM +0100, Marc Zyngier wrote:

[...]

arch/arm64/kvm/nested.c int kvm_init_nested(struct kvm *kvm) · line 60

Quoted 3 lines

arch/arm64/kvm/nested.c +3 -0

+void kvm_destroy_nested(struct kvm *kvm)
+{
+	for (int i = 0; i < kvm->arch.nested_mmus_size; i+= S2_MMU_PER_VCPU)

nit: ^ missing space

Thanks,
Wei-Lin Chang

Quoted 6 lines

arch/arm64/kvm/nested.c +6 -0

+		kvfree(kvm->arch.nested_mmus[i]);
+
+	kvm->arch.nested_mmus_size = 0;
+	kvfree(kvm->arch.nested_mmus);
+}
+

[...]

Quoted 4 lines · Wei-Lin Chang

On Fri, Aug 21, 2026 at 05:18:29PM +0100, Marc Zyngier wrote:

[...]

Quoted 5 lines

arch/arm64/kvm/nested.c int kvm_init_nested(struct kvm *kvm) · line 60

Quoted 3 lines

arch/arm64/kvm/nested.c +3 -0

+void kvm_destroy_nested(struct kvm *kvm)
+{
+	for (int i = 0; i < kvm->arch.nested_mmus_size; i+= S2_MMU_PER_VCPU)

nit: ^ missing space

Ack will fix on respin :)

This is the (hopefully) last revision for this set of fixes for the shadow S2 life cycle.

* From v2 [1]

- Fixed the nonsensical type used for allocating the nested S2 array, switching to kvmalloc_objs() in the process, guaranteeing some level of type checking

- Dropped the extra S2 freeing after Wei-Lin pointed out that we are guaranteed to run the MMU notifier release callback, ensuring that nothing is left behind

- Collected AB from Lorenzo, with thanks.

[1] https://lore.kernel.org/all/[email protected]/

Marc Zyngier (2):
KVM: arm64: nv: Fix life cycle of the nested_mmus array
KVM: arm64: nv: Delay freeing of shadow S2 structures until VM
destruction

 arch/arm64/include/asm/kvm_host.h   |   2 +-
 arch/arm64/include/asm/kvm_nested.h |   3 +-
 arch/arm64/kvm/arm.c                |   8 ++-
 arch/arm64/kvm/nested.c             | 100 ++++++++++++++--------------
 4 files changed, 58 insertions(+), 55 deletions(-)

Signature · 1 line

2.47.3