Skip to contentSkip to conversation

vfio/cdx: Fix interrupt trigger races and consolidate MSI state

PATCH In review Alex Williamson 2 participants 5 messages Apr 17 – Apr 21

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

v2 cover vfio/cdx: Fix interrupt trigger races and consolidate MSI state

Prasanna Kumar's original patch [1] closes a NULL-pointer dereference that can be triggered by userspace via VFIO_DEVICE_SET_IRQS with VFIO_IRQ_SET_DATA_BOOL or VFIO_IRQ_SET_DATA_NONE before any VFIO_IRQ_SET_DATA_EVENTFD. Included here unchanged as patch 1/3.

Review of that patch identified a race in the same code path, where concurrent SET_IRQS calls can potentially result in time-of-check vs time-of-use and use-after-free inconsistencies. Patch 2/3 adds a per-device cdx_irqs_lock mutex and serializes all MSI state updates through vfio_cdx_set_msi_trigger(), which is the single choke-point covering both the ioctl path and the close-device cleanup.

Patch 3/3 is a non-stable cleanup that aligns vdev->cdx_irqs to NULL in error paths, thereby allowing removal of the now redundant config_msi.

[1] https://lore.kernel.org/all/[email protected]

Alex Williamson (2):
vfio/cdx: Serialize VFIO_DEVICE_SET_IRQS with a per-device mutex vfio/cdx: Consolidate MSI configured state onto cdx_irqs

Prasanna Kumar T S M (1):
vfio/cdx: Fix NULL pointer dereference in interrupt trigger path

 drivers/vfio/cdx/intr.c    | 38 ++++++++++++++++++--------------------
 drivers/vfio/cdx/main.c    | 19 +++++++++++++++++++
 drivers/vfio/cdx/private.h |  4 +++-
 3 files changed, 40 insertions(+), 21 deletions(-)
Signature · 1 line
2.51.0
#1 lore raw

PATCH v2 1/3 vfio/cdx: Fix NULL pointer dereference in interrupt trigger path

From: Prasanna Kumar T S M <[email protected]>

Add validation to ensure MSI is configured before accessing cdx_irqs array in vfio_cdx_set_msi_trigger(). Without this check, userspace can trigger a NULL pointer dereference by calling VFIO_DEVICE_SET_IRQS with VFIO_IRQ_SET_DATA_BOOL or VFIO_IRQ_SET_DATA_NONE flags before ever setting up interrupts via VFIO_IRQ_SET_DATA_EVENTFD.

The vfio_cdx_msi_enable() function allocates the cdx_irqs array and sets config_msi to 1 only when called through the EVENTFD path. The trigger loop (for DATA_BOOL/DATA_NONE) assumed this had already been done, but there was no enforcement of this call ordering.

This matches the protection used in the PCI VFIO driver where vfio_pci_set_msi_trigger() checks irq_is() before the trigger loop.

Fixes 848e447e000c ("vfio/cdx: add interrupt support")
---
 drivers/vfio/cdx/intr.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/vfio/cdx/intr.c +4 -0
index 8f4402cec9c5..c0eed065e8ef 100644
@@ -175,6 +175,10 @@ static int vfio_cdx_set_msi_trigger(struct vfio_cdx_device *vdev,
 		return ret;
 	}
 
+	/* Ensure MSI is configured before accessing cdx_irqs */
+	if (!vdev->config_msi)
+		return -EINVAL;
+
 	for (i = start; i < start + count; i++) {
 		if (!vdev->cdx_irqs[i].trigger)
 			continue;
Signature · 1 line
2.51.0
#2 lore raw

PATCH v2 2/3 vfio/cdx: Serialize VFIO_DEVICE_SET_IRQS with a per-device mutex

In reply to Alex Williamson

vfio_cdx_set_msi_trigger() reads vdev->config_msi and operates on the vdev->cdx_irqs array based on its value, but provides no serialization against concurrent VFIO_DEVICE_SET_IRQS ioctls. Two callers can race such that one observes config_msi as set while another clears it and frees cdx_irqs via vfio_cdx_msi_disable(), resulting in a use-after-free of the cdx_irqs array.

Add a cdx_irqs_lock mutex to struct vfio_cdx_device and acquire it in vfio_cdx_set_msi_trigger(), which is the single chokepoint through which all updates to config_msi, cdx_irqs, and msi_count flow, covering both the ioctl path and the close-device cleanup path. This keeps the test of config_msi atomic with the subsequent enable, disable, or trigger operations.

Drop the pre-call !cdx_irqs test from vfio_cdx_irqs_cleanup() as part of this change: the optimization it provided is redundant with the !config_msi early-return inside vfio_cdx_msi_disable(), and leaving the test in place would be an unsynchronized read of state the new lock is meant to protect.

Fixes 848e447e000c ("vfio/cdx: add interrupt support")
Assisted-by Claude:claude-opus-4-7
---
 drivers/vfio/cdx/intr.c    |  9 ++-------
 drivers/vfio/cdx/main.c    | 19 +++++++++++++++++++
 drivers/vfio/cdx/private.h |  3 +++
 3 files changed, 24 insertions(+), 7 deletions(-)
drivers/vfio/cdx/intr.c +2 -7
index c0eed065e8ef..6dfe0ced3bdd 100644
@@ -152,6 +152,8 @@ static int vfio_cdx_set_msi_trigger(struct vfio_cdx_device *vdev,
 	if (start + count > cdx_dev->num_msi)
 		return -EINVAL;
 
+	guard(mutex)(&vdev->cdx_irqs_lock);
+
 	if (!count && (flags & VFIO_IRQ_SET_DATA_NONE)) {
 		vfio_cdx_msi_disable(vdev);
 		return 0;
@@ -210,12 +212,5 @@ int vfio_cdx_set_irqs_ioctl(struct vfio_cdx_device *vdev,
 /* Free All IRQs for the given device */
 void vfio_cdx_irqs_cleanup(struct vfio_cdx_device *vdev)
 {
-	/*
-	 * Device does not support any interrupt or the interrupts
-	 * were not configured
-	 */
-	if (!vdev->cdx_irqs)
-		return;
-
 	vfio_cdx_set_msi_trigger(vdev, 0, 0, 0, VFIO_IRQ_SET_DATA_NONE, NULL);
 }
drivers/vfio/cdx/main.c +19 -0
index 8ab97405b2bd..b31ed4be7bdc 100644
@@ -8,6 +8,23 @@
 
 #include "private.h"
 
+static int vfio_cdx_init_dev(struct vfio_device *core_vdev)
+{
+	struct vfio_cdx_device *vdev =
+		container_of(core_vdev, struct vfio_cdx_device, vdev);
+
+	mutex_init(&vdev->cdx_irqs_lock);
+	return 0;
+}
+
+static void vfio_cdx_release_dev(struct vfio_device *core_vdev)
+{
+	struct vfio_cdx_device *vdev =
+		container_of(core_vdev, struct vfio_cdx_device, vdev);
+
+	mutex_destroy(&vdev->cdx_irqs_lock);
+}
+
 static int vfio_cdx_open_device(struct vfio_device *core_vdev)
 {
 	struct vfio_cdx_device *vdev =
@@ -273,6 +290,8 @@ static int vfio_cdx_mmap(struct vfio_device *core_vdev,
 
 static const struct vfio_device_ops vfio_cdx_ops = {
 	.name		= "vfio-cdx",
+	.init		= vfio_cdx_init_dev,
+	.release	= vfio_cdx_release_dev,
 	.open_device	= vfio_cdx_open_device,
 	.close_device	= vfio_cdx_close_device,
 	.ioctl		= vfio_cdx_ioctl,
drivers/vfio/cdx/private.h +3 -0
index 172e48caa3a0..94374b5fc989 100644
@@ -6,6 +6,8 @@
 #ifndef VFIO_CDX_PRIVATE_H
 #define VFIO_CDX_PRIVATE_H
 
+#include <linux/mutex.h>
+
 #define VFIO_CDX_OFFSET_SHIFT    40
 
 static inline u64 vfio_cdx_index_to_offset(u32 index)
@@ -31,6 +33,7 @@ struct vfio_cdx_region {
 struct vfio_cdx_device {
 	struct vfio_device	vdev;
 	struct vfio_cdx_region	*regions;
+	struct mutex		cdx_irqs_lock;
 	struct vfio_cdx_irq	*cdx_irqs;
 	u32			flags;
 #define BME_SUPPORT BIT(0)
Signature · 1 line
2.51.0
#3 lore raw

PATCH v2 3/3 vfio/cdx: Consolidate MSI configured state onto cdx_irqs

In reply to Alex Williamson

struct vfio_cdx_device carries three fields that track whether MSI has been configured: vdev->cdx_irqs (the allocated vector array), vdev-> msi_count (the array length), and vdev->config_msi (a boolean flag). The three are set together when vfio_cdx_msi_enable() succeeds and cleared together by vfio_cdx_msi_disable(). However, the error paths in vfio_cdx_msi_enable() free the cdx_irqs allocation on failure without resetting the pointer, leaving it stale and skewed from the other two fields until the next enable call overwrites it.

Clear vdev->cdx_irqs to NULL alongside the kfree() in both error paths so the pointer consistently reflects the configured state. With that invariant restored and access to the MSI state serialized by cdx_irqs_lock, vdev->config_msi is fully redundant with (vdev->cdx_irqs != NULL). Drop the config_msi field and switch all readers to test cdx_irqs directly.

Assisted-by Claude:claude-opus-4-7
---
 drivers/vfio/cdx/intr.c    | 29 ++++++++++++++---------------
 drivers/vfio/cdx/private.h |  1 -
 2 files changed, 14 insertions(+), 16 deletions(-)
drivers/vfio/cdx/intr.c +14 -15
index 6dfe0ced3bdd..4439481fe633 100644
@@ -32,26 +32,27 @@ static int vfio_cdx_msi_enable(struct vfio_cdx_device *vdev, int nvec)
 		return -ENOMEM;
 
 	ret = cdx_enable_msi(cdx_dev);
-	if (ret) {
-		kfree(vdev->cdx_irqs);
-		return ret;
-	}
+	if (ret)
+		goto err_free;
 
 	/* Allocate cdx MSIs */
 	ret = msi_domain_alloc_irqs(dev, MSI_DEFAULT_DOMAIN, nvec);
-	if (ret) {
-		cdx_disable_msi(cdx_dev);
-		kfree(vdev->cdx_irqs);
-		return ret;
-	}
+	if (ret)
+		goto err_disable;
 
 	for (msi_idx = 0; msi_idx < nvec; msi_idx++)
 		vdev->cdx_irqs[msi_idx].irq_no = msi_get_virq(dev, msi_idx);
 
 	vdev->msi_count = nvec;
-	vdev->config_msi = 1;
 
 	return 0;
+
+err_disable:
+	cdx_disable_msi(cdx_dev);
+err_free:
+	kfree(vdev->cdx_irqs);
+	vdev->cdx_irqs = NULL;
+	return ret;
 }
 
 static int vfio_cdx_msi_set_vector_signal(struct vfio_cdx_device *vdev,
@@ -129,7 +130,7 @@ static void vfio_cdx_msi_disable(struct vfio_cdx_device *vdev)
 
 	vfio_cdx_msi_set_block(vdev, 0, vdev->msi_count, NULL);
 
-	if (!vdev->config_msi)
+	if (!vdev->cdx_irqs)
 		return;
 
 	msi_domain_free_irqs_all(dev, MSI_DEFAULT_DOMAIN);
@@ -138,7 +139,6 @@ static void vfio_cdx_msi_disable(struct vfio_cdx_device *vdev)
 
 	vdev->cdx_irqs = NULL;
 	vdev->msi_count = 0;
-	vdev->config_msi = 0;
 }
 
 static int vfio_cdx_set_msi_trigger(struct vfio_cdx_device *vdev,
@@ -163,7 +163,7 @@ static int vfio_cdx_set_msi_trigger(struct vfio_cdx_device *vdev,
 		s32 *fds = data;
 		int ret;
 
-		if (vdev->config_msi)
+		if (vdev->cdx_irqs)
 			return vfio_cdx_msi_set_block(vdev, start, count,
 						  fds);
 		ret = vfio_cdx_msi_enable(vdev, cdx_dev->num_msi);
@@ -177,8 +177,7 @@ static int vfio_cdx_set_msi_trigger(struct vfio_cdx_device *vdev,
 		return ret;
 	}
 
-	/* Ensure MSI is configured before accessing cdx_irqs */
-	if (!vdev->config_msi)
+	if (!vdev->cdx_irqs)
 		return -EINVAL;
 
 	for (i = start; i < start + count; i++) {
drivers/vfio/cdx/private.h +0 -1
index 94374b5fc989..4c00bf633356 100644
@@ -38,7 +38,6 @@ struct vfio_cdx_device {
 	u32			flags;
 #define BME_SUPPORT BIT(0)
 	u32			msi_count;
-	u8			config_msi;
 };
 
 #ifdef CONFIG_GENERIC_MSI_IRQ
Signature · 1 line
2.51.0
maintainer Acked-by on the v2 series
#4 lore raw
In reply to Alex Williamson
Quoted 26 lines

-----Original Message-----
From: Alex Williamson <[email protected]>
Sent: Saturday, April 18, 2026 1:58 AM
To: [email protected]
Cc: Alex Williamson <[email protected]>; [email protected]; Gupta, Nipun <[email protected]>; Agarwal, Nikhil <[email protected]>; [email protected]; [email protected]
Subject: [PATCH v2 0/3] vfio/cdx: Fix interrupt trigger races and consolidate MSI state

Prasanna Kumar's original patch [1] closes a NULL-pointer dereference that can be triggered by userspace via VFIO_DEVICE_SET_IRQS with VFIO_IRQ_SET_DATA_BOOL or VFIO_IRQ_SET_DATA_NONE before any VFIO_IRQ_SET_DATA_EVENTFD. Included here unchanged as patch 1/3.

Review of that patch identified a race in the same code path, where concurrent SET_IRQS calls can potentially result in time-of-check vs time-of-use and use-after-free inconsistencies. Patch 2/3 adds a per-device cdx_irqs_lock mutex and serializes all MSI state updates through vfio_cdx_set_msi_trigger(), which is the single choke-point covering both the ioctl path and the close-device cleanup.

Patch 3/3 is a non-stable cleanup that aligns vdev->cdx_irqs to NULL in error paths, thereby allowing removal of the now redundant config_msi.

Quoted 9 lines

[1] https://lore.kernel.org/all/[email protected]

Alex Williamson (2):
vfio/cdx: Serialize VFIO_DEVICE_SET_IRQS with a per-device mutex vfio/cdx: Consolidate MSI configured state onto cdx_irqs

Prasanna Kumar T S M (1):
vfio/cdx: Fix NULL pointer dereference in interrupt trigger path

For series: