PCI/IOV: Fix SR-IOV locking races and AB-BA deadlock
Subsystems (2)
v13 cover PCI/IOV: Fix SR-IOV locking races and AB-BA deadlock
From: Ionut Nechita <[email protected]>
Hi Bjorn,
This is v13 of the fix for the SR-IOV race between driver .remove() and concurrent hotplug events.
Changes since v12 (Apr 21):
- Patch 1/2: replace mutex_get_owner() with explicit owner tracking (struct task_struct * + depth counter). mutex_get_owner() is not exported outside the scheduler core and caused link failures in some build configurations:
ld: vmlinux.o: in function `pci_lock_rescan_remove':
drivers/pci/probe.c: undefined reference to `mutex_get_owner'
The new implementation records the owner on first acquisition and clears it on final release, with a WARN_ON() catching mismatched unlock calls. Semantics are unchanged.
- Patch 1/2: dropped Reviewed-by/Tested-by since the locking primitive changed; please re-review.
- Patch 2/2: unchanged from v12 (keeps R-b from Niklas Schnelle and R-b/T-b from Benjamin Block).
Changes since v11 (Mar 26):
- Patch 2/2 picked up R-b Niklas Schnelle and R-b/T-b Benjamin Block
- Rebased on linux-next (next-20260420)
Changes since v10 (Mar 18):
- Patch 2/2: added kill_device() before device_release_driver() to prevent a new driver from binding between unbind and removal, closing the TOCTOU race window identified by Benjamin Block
- Patch 1/2 unchanged from v10
Changes since v9 (Mar 10):
- NEW patch 2/2: fix AB-BA deadlock in remove_store() by calling device_release_driver() before pci_stop_and_remove_bus_device_locked(), as suggested by Benjamin Block (addresses Guenter Roeck's report)
- Patch 1/2 unchanged from v9
Changes since v8 (Mar 9):
- Added Reviewed-by from Niklas Schnelle (IBM) and Tested-by (s390)
- Added Fixes tags for the three related commits
- Removed rescan/remove locking from sriov_numvfs_store() since locking is now handled in sriov_add_vfs() and sriov_del_vfs()
- Rebased on linux-next (20260309)
The AB-BA deadlock:
CPU0 (remove_store) CPU1 (unbind_store)
-------------------- --------------------
pci_lock_rescan_remove()
device_lock()
driver .remove()
sriov_del_vfs()
pci_lock_rescan_remove() <-- WAITS
pci_stop_bus_device()
device_release_driver()
device_lock() <-- WAITS
Patch 2/2 fixes this by:
1. Marking the device as dead via kill_device() so no new driver can bind (prevents TOCTOU race between unbind and removal)
2. Calling device_release_driver() before pci_stop_and_remove_bus_device_locked(), so both paths take locks in the same order: device_lock first, then pci_rescan_remove_lock
Note: the concurrent unbind_store + hotplug-event case (where the hotplug handler takes pci_rescan_remove_lock before device_lock) remains a known limitation. This is a pre-existing issue that Benjamin Block is addressing separately in:
https://lore.kernel.org/linux-pci/354b9e4a54ced67f3c89df198041df19434fe4c8.1773235561.git.bblock@linux.ibm.com/
This race has been independently observed by multiple organizations:
- IBM (s390 platform-generated hot-unplug events racing with sriov_del_vfs during PF driver unload)
- NVIDIA (tested by Dragos Tatulea in earlier versions)
- Intel (xe driver hitting lockdep warnings and deadlocks when calling pci_disable_sriov from .remove)
- Wind River (original reporter and patch author)
Test environment:
- Tested on s390 by Benjamin Block and Niklas Schnelle (IBM)
- Tested on x86_64 with Intel and NVIDIA SR-IOV devices (earlier versions)
Based on linux-next (next-20260420).
Ionut Nechita (Wind River) (2):
PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect
sriov_add_vfs/sriov_del_vfs
PCI: Fix AB-BA deadlock between device_lock and pci_rescan_remove_lock
in remove_store
drivers/pci/iov.c | 9 +++++---- drivers/pci/pci-sysfs.c | 30 +++++++++++++++++++++++++++++- drivers/pci/probe.c | 18 ++++++++++++++++-- 3 files changed, 50 insertions(+), 7 deletions(-)
Signature · 1 line
2.53.0
PATCH v13 1/2 PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect sriov_add_vfs/sriov_del_vfs
From: Ionut Nechita <[email protected]>
After reverting commit 05703271c3cd ("PCI/IOV: Add PCI rescan-remove locking when enabling/disabling SR-IOV") and moving the lock to sriov_numvfs_store(), the path through driver .remove() (e.g. rmmod, or manual unbind) that calls pci_disable_sriov() directly remains unprotected against concurrent hotplug events. This affects any SR-IOV capable driver that calls pci_disable_sriov() from its .remove() callback (i40e, ice, mlx5, bnxt, etc.).
On s390, platform-generated hot-unplug events for VFs can race with sriov_del_vfs() when a PF driver is being unloaded. The platform event handler takes pci_rescan_remove_lock, but sriov_del_vfs() does not, leading to double removal and list corruption.
We cannot use a plain mutex_lock() here because sriov_del_vfs() may also be called from paths that already hold pci_rescan_remove_lock (e.g. remove_store -> pci_stop_and_remove_bus_device_locked, or sriov_numvfs_store with the lock taken by the previous patch). Using mutex_lock() in those cases would deadlock.
Make pci_lock_rescan_remove() itself reentrant by tracking the current owner task and a recursion depth counter, as suggested by Lukas Wunner and Benjamin Block, since these recursive locking scenarios exist elsewhere in the PCI subsystem:
- If the lock is already held by the current task (owner == current):
increments the depth counter and returns without re-acquiring, avoiding deadlock.
- If the lock is held by another task: blocks until the lock is released, then records the owner and sets depth to 1.
- If the lock is not held: acquires the mutex normally.
pci_unlock_rescan_remove() decrements the depth counter and releases the mutex (clearing the owner) only when the depth reaches zero. A WARN_ON catches mismatched unlock calls from tasks that do not own the lock.
This avoids relying on mutex_get_owner(), which is not exported to modules and caused link failures for builds that inline this code outside of the core kernel image.
This approach keeps the API unchanged: callers simply pair lock/unlock calls without needing to track any return value or use separate reentrant variants.
Add pci_lock_rescan_remove()/pci_unlock_rescan_remove() calls to sriov_add_vfs() and sriov_del_vfs() to protect VF addition and removal against concurrent hotplug events.
Remove the rescan/remove locking from sriov_numvfs_store() that was introduced by commit a5338e365c45 ("PCI/IOV: Fix race between SR-IOV enable/disable and hotplug"), since the locking is now handled directly in sriov_add_vfs() and sriov_del_vfs() where it is actually needed, reducing the lock scope.
--- drivers/pci/iov.c | 9 +++++---- drivers/pci/probe.c | 18 ++++++++++++++++-- 2 files changed, 21 insertions(+), 6 deletions(-)
drivers/pci/iov.c +5 -4
@@ -495,9 +495,7 @@ static ssize_t sriov_numvfs_store(struct device *dev,if (num_vfs ==0 ) {/* disable VFs */ -pci_lock_rescan_remove (); ret = pdev->driver->sriov_configure (pdev,0 ); -pci_unlock_rescan_remove (); goto exit; }
@@ -509,9 +507,7 @@ static ssize_t sriov_numvfs_store(struct device *dev, goto exit; } -pci_lock_rescan_remove (); ret = pdev->driver->sriov_configure (pdev, num_vfs); -pci_unlock_rescan_remove ();if (ret <0 ) goto exit;
@@ -633,15 +629,18 @@ static int sriov_add_vfs(struct pci_dev *dev, u16 num_vfs)if (dev->no_vf_scan)return 0 ; +pci_lock_rescan_remove ();for (i =0 ; i < num_vfs; i++) { rc =pci_iov_add_virtfn (dev, i);if (rc) goto failed; } +pci_unlock_rescan_remove ();return 0 ; failed:while (i--)pci_iov_remove_virtfn (dev, i); +pci_unlock_rescan_remove ();return rc; }
@@ -766,8 +765,10 @@ static void sriov_del_vfs(struct pci_dev *dev)struct pci_sriov *iov = dev->sriov;int i; +pci_lock_rescan_remove ();for (i =0 ; i < iov->num_VFs; i++)pci_iov_remove_virtfn (dev, i); +pci_unlock_rescan_remove (); }static void sriov_disable (struct pci_dev *dev)
drivers/pci/probe.c +16 -2
@@ -3513,16 +3513,30 @@ EXPORT_SYMBOL_GPL(pci_rescan_bus);* routines should always be executed under this mutex. */ DEFINE_MUTEX (pci_rescan_remove_lock); +static struct task_struct *pci_rescan_remove_owner; +static unsigned int pci_rescan_remove_depth;void pci_lock_rescan_remove (void ) { -mutex_lock (&pci_rescan_remove_lock); +if (pci_rescan_remove_owner == current) { + pci_rescan_remove_depth++; + }else { +mutex_lock (&pci_rescan_remove_lock); + pci_rescan_remove_owner = current; + pci_rescan_remove_depth =1 ; + } }EXPORT_SYMBOL_GPL (pci_lock_rescan_remove);void pci_unlock_rescan_remove (void ) { -mutex_unlock (&pci_rescan_remove_lock); +if (WARN_ON (pci_rescan_remove_owner != current)) +return ; + +if (--pci_rescan_remove_depth ==0 ) { + pci_rescan_remove_owner =NULL ; +mutex_unlock (&pci_rescan_remove_lock); + } }EXPORT_SYMBOL_GPL (pci_unlock_rescan_remove);
Signature · 1 line
2.53.0
Re: [PATCH v13 1/2] PCI/IOV: Make pci_lock_rescan_remove() reentrant and protect sriov_add_vfs/sriov_del_vfs
drivers/pci/probe.c EXPORT_SYMBOL_GPL(pci_rescan_bus); · line 3516
Quoted 9 lines · Ionut Nechita (Wind River)
drivers/pci/probe.c +1
-0@@ -3513,16 +3513,30 @@ EXPORT_SYMBOL_GPL(pci_rescan_bus);* routines should always be executed under this mutex. */ DEFINE_MUTEX (pci_rescan_remove_lock); +static struct task_struct *pci_rescan_remove_owner;
You could make this variable const:
static const struct task_struct *pci_rescan_remove_owner;
so it is clear that this pointer is not meant to be used to modify the task_struct in any way - just for these comparisons.
Quoted 17 lines
drivers/pci/probe.c +8
-1+static unsigned int pci_rescan_remove_depth;void pci_lock_rescan_remove (void ) { -mutex_lock (&pci_rescan_remove_lock); +if (pci_rescan_remove_owner == current) { + pci_rescan_remove_depth++; + }else { +mutex_lock (&pci_rescan_remove_lock); + pci_rescan_remove_owner = current; + pci_rescan_remove_depth =1 ; + } }EXPORT_SYMBOL_GPL (pci_lock_rescan_remove);void pci_unlock_rescan_remove (void ) {
drivers/pci/probe.c EXPORT_SYMBOL_GPL(pci_rescan_bus); · line 3541
Quoted 10 lines
drivers/pci/probe.c +7
-1-mutex_unlock (&pci_rescan_remove_lock); +if (WARN_ON (pci_rescan_remove_owner != current)) +return ; + +if (--pci_rescan_remove_depth ==0 ) { + pci_rescan_remove_owner =NULL ; +mutex_unlock (&pci_rescan_remove_lock); + } }EXPORT_SYMBOL_GPL (pci_unlock_rescan_remove);
But anyway, with, or without this change, this looks good to me. I hadn't run into any linker issues, but I only ever used our debug config. Also holds up with my tests on my s390 rig.
Signature · 4 lines
Best Regards, Benjamin Block / Linux on IBM Z Kernel Development IBM Deutschland Research & Development GmbH / https://www.ibm.com/privacy Vors. Aufs.-R.: Wolfgang Wendt / Geschäftsführung: David Faller Sitz der Ges.: Ehningen / Registergericht: AmtsG Stuttgart, HRB 243294
PATCH v13 2/2 PCI: Fix AB-BA deadlock between device_lock and pci_rescan_remove_lock in remove_store
In reply to Ionut Nechita (Wind River)From: Ionut Nechita <[email protected]>
remove_store() calls pci_stop_and_remove_bus_device_locked() which takes pci_rescan_remove_lock first, then device_lock during driver release. Meanwhile, unbind_store() takes device_lock first (via device_driver_detach), and the driver's .remove() callback may call pci_disable_sriov() -> sriov_del_vfs() -> pci_lock_rescan_remove().
This creates an AB-BA deadlock:
CPU0 (remove_store) CPU1 (unbind_store)
-------------------- --------------------
pci_lock_rescan_remove()
device_lock()
driver .remove()
sriov_del_vfs()
pci_lock_rescan_remove() <-- WAITS
pci_stop_bus_device()
device_release_driver()
device_lock() <-- WAITS
Fix this by first marking the device as dead using kill_device() to prevent any new driver from binding, then calling device_release_driver() before pci_stop_and_remove_bus_device_locked().
Marking the device dead closes the race window between unbinding and removal where a new driver could theoretically bind: once the dead flag is set, the device core will refuse any new driver probe.
After device_release_driver() returns, the driver is already unbound, so the subsequent device_release_driver() call inside pci_stop_and_remove_bus_device_locked() becomes a no-op.
--- drivers/pci/pci-sysfs.c | 30 +++++++++++++++++++++++++++++- 1 file changed, 29 insertions(+), 1 deletion(-)
drivers/pci/pci-sysfs.c +29 -1
@@ -521,8 +521,36 @@ static ssize_t remove_store(struct device *dev, struct device_attribute *attr,if (kstrtoul (buf,0 , &val) <0 )return -EINVAL ; -if (val &&device_remove_file_self (dev, attr)) +if (val &&device_remove_file_self (dev, attr)) { +/* +* Mark the device as dead so that no new driver can bind +* between the unbind and the removal below. Once the +* dead flag is set, the device core will refuse any new +* driver probe. +*/ +device_lock (dev); +kill_device (dev); +device_unlock (dev); + +/* +* Unbind the driver before removing the device to avoid +* an AB-BA deadlock between device_lock and +* pci_rescan_remove_lock. Without this, remove_store +* takes pci_rescan_remove_lock first (via +* pci_stop_and_remove_bus_device_locked) and then +* device_lock during driver release, while a concurrent +* unbind_store (or sriov_numvfs_store) takes device_lock +* first and then pci_rescan_remove_lock (via +* sriov_del_vfs), creating a circular dependency. +* +* By unbinding first, the driver's .remove() callback +* (including any SR-IOV VF cleanup) completes before +* pci_rescan_remove_lock is acquired, ensuring both +* paths take locks in the same order. +*/ +device_release_driver (dev);pci_stop_and_remove_bus_device_locked (to_pci_dev (dev)); + }return count; } static DEVICE_ATTR_IGNORE_LOCKDEP(remove, 0220, NULL,
Signature · 1 line
2.53.0
Patchwork
CI✗ LGCI.VerificationFailed: failure for PCI/IOV: Fix SR-IOV locking races and AB-BA deadlock
In reply to Ionut Nechita (Wind River)== Series Details ==
== Series Details ==
Series: PCI/IOV: Fix SR-IOV locking races and AB-BA deadlock
URL : https://patchwork.freedesktop.org/series/165218/
State : failure
== Summary ==
Address '[email protected]' is not on the allowlist, which prevents CI from being triggered for this patch. If you want Intel GFX CI to accept this address, please contact the script maintainers at [email protected]. Exception occurred during validation, bailing out!