Skip to contentSkip to conversation

KVM: arm64: vgic: Fix racy LPI release and re-registration handling

PATCH Superseded Carlos López 2 participants 4 messages Jul 3

Subsystems (2)

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover KVM: arm64: vgic: Fix racy LPI release and re-registration handling

Fix a couple of potential issues that could arise from racy LPI release and re-registration for the same INTID.

The issue fixed in patch 1 can manifest itself through either a leaked LPI structure, or a prematurely deleted LPI. The issue fixed in patch 2 could materialize as a spurious -ENOMEM failure when registering an LPI.

v3:
* Use refcount_dec_and_lock_irqsave() instead of unconditionally grabbing the xarray lock in patch 1.
* Add patch 2.

v2:
* Address Sashiko's review. Fix the direct release path by decrementing the refcount under the xarray spinlock, preventing a UAF that would have been introduced in v1.

Carlos López (2):
KVM: arm64: vgic: Fix race between LPI release and re-registration
KVM: arm64: vgic: Mitigate potential LPI registration failure

 arch/arm64/kvm/vgic/vgic-its.c | 10 +++++++++-
 arch/arm64/kvm/vgic/vgic.c     | 10 ++++++----
 2 files changed, 15 insertions(+), 5 deletions(-)

base-commit: 1ee27dacbe5dc4def481794d899d67b0d4570094

Signature · 1 line
2.51.0
#1 lore raw

PATCH v3 1/2 KVM: arm64: vgic: Fix race between LPI release and re-registration

Fix a potential race between decrementing an LPI's reference count and evicting that structure from the LPI xarray.

LPI structures are maintained in the VGIC LPI xarray (dist->lpi_xa). When the reference count of an LPI structure drops to zero, vgic_release_lpi_locked() removes the structure from the xarray and frees it under the xarray lock.

However, the release of an LPI can race with a concurrent LPI re-registration with the same INTID via vgic_add_lpi() on another CPU, since the reference count drop and the xarray eviction are not performed in a single atomic step. This can happen e.g. if the guest issues a DISCARD while the LPI is still referenced from a vCPU's active-pending list (ap_list), and the same INTID is re-mapped via MAPTI.

Particularly, vgic_release_lpi_locked() is called from two distinct
paths: direct release via vgic_put_irq(), and deferred release via vgic_release_deleted_lpis(). During direct release, the issue can result in deleting a newly registered LPI from the xarray:

CPU0 (Releasing LPI) CPU1 (Adding new LPI)
==================== =====================
vgic_put_irq()

      __vgic_put_irq()
          refcount_dec_and_test()
                                          vgic_add_lpi()
                                              xa_lock_irqsave()
                                              old_irq = xa_load(.., intid)
                                              vgic_try_get_irq_ref(old_irq) == false
                        new IRQ inserted -->  __xa_store(.., intid, ..)
                                              xa_unlock_irqrestore()
  xa_lock_irqsave();
  vgic_release_lpi_locked()
      __xa_erase(.., irq->intid)   <-- BUG: new IRQ is erased
      kfree_rcu(old_irq)

During the deferred release path, the old IRQ can be leaked:

CPU0 (Releasing LPI) CPU1 (Adding new LPI)
==================== =====================
vgic_put_irq_norelease()

      __vgic_put_irq()
          refcount_dec_and_test()
      irq->pending_release = true
                                          vgic_add_lpi()
                                              xa_lock_irqsave()
                                              old_irq = xa_load(.., intid)
                                              vgic_try_get_irq_ref(oldirq) == false
                 BUG: old IRQ overwritten --> __xa_store(.., intid, ..)
                                              xa_unlock_irqrestore()

vgic_release_deleted_lpis()
xa_lock_irqsave()
xa_for_each() { .. } <-- old IRQ with pending_release = true
is gone, so it cannot be released

To fix the direct release path, move the reference count drop inside the xarray lock, making sure that vgic_add_lpi() never encounters the to-be-released LPI.

To fix the deferred release path, since the refcount drop must happen under a raw spinlock, the same solution does not work. Instead, update vgic_add_lpi(), so that if it evicts a non-NULL refcount=0 LPI from the xarray, it takes on the responsibility of releasing it. If this happens, vgic_release_deleted_lpis() will iterate the xarray normally and will simply not find the already released structure.

Reported-by Claude:claude-opus-4-6
Fixes 3a08a6ca7c37 ("KVM: arm64: vgic-v3: Use bare refcount for VGIC LPIs")
Fixes d54594accf73 ("KVM: arm64: vgic-v3: Erase LPIs from xarray outside of raw spinlocks")
---
 arch/arm64/kvm/vgic/vgic-its.c | 10 +++++++++-
 arch/arm64/kvm/vgic/vgic.c     | 10 ++++++----
 2 files changed, 15 insertions(+), 5 deletions(-)
arch/arm64/kvm/vgic/vgic-its.c +9 -1
index 67d107e9a77d..577286069368 100644
@@ -116,7 +116,15 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
 		kfree(irq);
 		irq = oldirq;
 	} else {
-		ret = xa_err(__xa_store(&dist->lpi_xa, intid, irq, 0));
+		/*
+		 * The entry is either empty or contains a dead LPI (refcount=0)
+		 * from the deferred release path, pending cleanup by
+		 * vgic_release_deleted_lpis(). Evict and free it if present.
+		 */
+		oldirq = __xa_store(&dist->lpi_xa, intid, irq, 0);
+		ret = xa_err(oldirq);
+		if (!ret && oldirq)
+			kfree_rcu(oldirq, rcu);
 	}
 
 	xa_unlock_irqrestore(&dist->lpi_xa, flags);
arch/arm64/kvm/vgic/vgic.c +6 -4
index 5a4768d8cd4f..cc09e0c45b46 100644
@@ -167,12 +167,14 @@ void vgic_put_irq(struct kvm *kvm, struct vgic_irq *irq)
 		guard(spinlock_irqsave)(&dist->lpi_xa.xa_lock);
 	}
 
-	if (!__vgic_put_irq(kvm, irq))
+	if (!irq_is_lpi(kvm, irq->intid))
 		return;
 
-	xa_lock_irqsave(&dist->lpi_xa, flags);
-	vgic_release_lpi_locked(dist, irq);
-	xa_unlock_irqrestore(&dist->lpi_xa, flags);
+	if (refcount_dec_and_lock_irqsave(&irq->refcount,
+					  &dist->lpi_xa.xa_lock, &flags)) {
+		vgic_release_lpi_locked(dist, irq);
+		xa_unlock_irqrestore(&dist->lpi_xa, flags);
+	}
 }
 
 static void vgic_release_deleted_lpis(struct kvm *kvm)
Signature · 1 line
2.51.0
#2 lore raw

PATCH v3 2/2 KVM: arm64: vgic: Mitigate potential LPI registration failure

In reply to Carlos López

Mitigate a potential failure when inserting a new LPI into the VGIC LPI xarray.

When vgic_add_lpi() is preparing to register a new LPI, it pre-allocates an xarray entry using xa_reserve_irq(), so that it can later perform the insertion under the xarray lock without allocating.

However, since xa_reserve_irq() is called before acquiring such lock, there is a potential race where xa_reserve_irq() observes a populated entry, thus not performing the allocation, and another CPU removes that entry before the xarray lock is grabbed to perform the insertion.

CPU0 (Adding new LPI) CPU1 (Releasing LPI)
===================== ===================
vgic_add_lpi()

      /* Entry populated, does not allocate */
      xa_reserve_irq(.., intid, ..)
                                            vgic_release_deleted_lpis()
                                                xa_lock_irqsave()
                                                vgic_release_lpi_locked()
                          xarray node freed --> __xa_erase(.., intid)
                                                xa_unlock_irqrestore()
      xa_lock_irqsave()
      xa_load(.., intid) == NULL
      vgic_try_get_irq_ref(NULL) == false
      __xa_store(.., intid, irq, 0) <-- xarray node was freed, gfp=0
                                        cannot allocate, returns -ENOMEM

This can happen e.g. if the guest issues a DISCARD while the LPI is still referenced from a vCPU's active-pending list (ap_list), and the same INTID is re-mapped via MAPTI.

Mitigate this by passing GFP_NOWAIT to __xa_store(), so that the allocation can happen under the lock in the rare case that this condition is hit.

Fixes 1d6f83f60f79 ("KVM: arm64: vgic: Store LPIs in an xarray")
---
 arch/arm64/kvm/vgic/vgic-its.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
arch/arm64/kvm/vgic/vgic-its.c +1 -1
index 577286069368..ace3e59fff97 100644
@@ -121,7 +121,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
 		 * from the deferred release path, pending cleanup by
 		 * vgic_release_deleted_lpis(). Evict and free it if present.
 		 */
-		oldirq = __xa_store(&dist->lpi_xa, intid, irq, 0);
+		oldirq = __xa_store(&dist->lpi_xa, intid, irq, GFP_NOWAIT);
 		ret = xa_err(oldirq);
 		if (!ret && oldirq)
 			kfree_rcu(oldirq, rcu);
Signature · 1 line
2.51.0
#3 lore raw

Re: [PATCH v3 2/2] KVM: arm64: vgic: Mitigate potential LPI registration failure

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Fallback XArray node allocation bypasses memcg accounting --

Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Fallback XArray node allocation bypasses memcg accounting
--

commit bdc0b53cd314d49c8e7e70017bf5fb300a832c2b
Author: Carlos López <[email protected]>
KVM: arm64: vgic: Mitigate potential LPI registration failure

This commit mitigates a race condition in vgic_add_lpi() where an LPI might be discarded and re-mapped concurrently. It allows __xa_store() to allocate a new node under the lock if the previously reserved entry was freed.

Quoted 4 lines
arch/arm64/kvm/vgic/vgic-its.c +0 -0
index 577286069368d..ace3e59fff971 100644

[ ... ]

arch/arm64/kvm/vgic/vgic-its.c static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid, · line 124

Quoted 6 lines
arch/arm64/kvm/vgic/vgic-its.c +1 -1
@@ -121,7 +121,7 @@ static struct vgic_irq *vgic_add_lpi(struct kvm *kvm, u32 intid,
 		 * from the deferred release path, pending cleanup by
 		 * vgic_release_deleted_lpis(). Evict and free it if present.
 		 */
-		oldirq = __xa_store(&dist->lpi_xa, intid, irq, 0);
+		oldirq = __xa_store(&dist->lpi_xa, intid, irq, GFP_NOWAIT);

[Severity: Low]
Does this __xa_store() fallback allocation bypass memcg accounting?

The initial allocation in vgic_add_lpi() via xa_reserve_irq() correctly uses GFP_KERNEL_ACCOUNT. However, the GFP_NOWAIT flag here does not include __GFP_ACCOUNT.

If a guest races a DISCARD and MAPTI command such that the LPI is released after xa_reserve_irq() but before xa_lock_irqsave(), this __xa_store() will allocate a new node. Should this use GFP_NOWAIT | __GFP_ACCOUNT to ensure this allocation is still charged to the guest's memory cgroup?

Quoted 3 lines
arch/arm64/kvm/vgic/vgic-its.c +0 -0
 		ret = xa_err(oldirq);
 		if (!ret && oldirq)
 			kfree_rcu(oldirq, rcu);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2