Skip to contentSkip to conversation

dmaengine: idxd: Fix refcount and cleanup issues on module unload

PATCH Superseded Yi Sun 2 participants 5 messages Jun 7, 2025 – Jun 14, 2025

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore

Yi Sun

author
#0 lore raw

v2 cover dmaengine: idxd: Fix refcount and cleanup issues on module unload

This patch series addresses two issues related to the device reference counting and cleanup path in the idxd driver.

Recent changes introduced improper put_device() calls and duplicated cleanup logic, leading to refcount underflow and potential use-after-free during module unload.

Patch 1 removes an unnecessary call to idxd_free(), which could result in a use-after-free when paired with asynchronous put_device().

Patch 2 refactors the cleanup path to avoid redundant put_device() calls introduced in commit a409e919ca3. The existing idxd_unregister_devices() already handles proper device reference release.

Both patches have been verified on hardware platform.

Both patches have been run through checkpatch.pl. Patch 2 gets 1 error and 1 warning. But these appear to be limitations in the checkpatch script itself, not reflect issues with the patches.

---
Changes in v2:
- Reworded commit messages supplementing the call traces (Vinicius)
- Explain why the put_device are unnecessary. (Vinicius)

Yi Sun (2):
dmaengine: idxd: Remove improper idxd_free
dmaengine: idxd: Fix refcount underflow on module unload

 drivers/dma/idxd/init.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
Signature · 1 line
2.43.0
author
#1 lore raw

PATCH v2 1/2 dmaengine: idxd: Remove improper idxd_free

The call to idxd_free() introduces a duplicate put_device() leading to a reference count underflow:
refcount_t: underflow; use-after-free.

WARNING: CPU: 15 PID: 4428 at lib/refcount.c:28 refcount_warn_saturate+0xbe/0x110
...
Call Trace:
 <TASK>
  idxd_remove+0xe4/0x120 [idxd]
  pci_device_remove+0x3f/0xb0
  device_release_driver_internal+0x197/0x200
  driver_detach+0x48/0x90
  bus_remove_driver+0x74/0xf0
  pci_unregister_driver+0x2e/0xb0
  idxd_exit_module+0x34/0x7a0 [idxd]
  __do_sys_delete_module.constprop.0+0x183/0x280
  do_syscall_64+0x54/0xd70
  entry_SYSCALL_64_after_hwframe+0x76/0x7e

The idxd_unregister_devices() which is invoked at the very beginning of idxd_remove(), already takes care of the necessary put_device() through the following call path:
idxd_unregister_devices() -> device_unregister() -> put_device()

In addition, when CONFIG_DEBUG_KOBJECT_RELEASE is enabled, put_device() may trigger asynchronous cleanup via schedule_delayed_work(). If idxd_free() is called immediately after, it can result in a use-after-free.

Remove the improper idxd_free() to avoid both the refcount underflow and potential memory corruption during module unload.

Fixes d5449ff1b04d ("dmaengine: idxd: Add missing idxd cleanup to fix memory leak in remove call")
drivers/dma/idxd/init.c +0 -1
index 760b7d81fcd8..504aca0fd597 100644
@@ -1324,7 +1324,6 @@ static void idxd_remove(struct pci_dev *pdev)
 	idxd_cleanup(idxd);
 	pci_iounmap(pdev, idxd->reg_base);
 	put_device(idxd_confdev(idxd));
-	idxd_free(idxd);
 	pci_disable_device(pdev);
 }
 
Signature · 1 line
2.43.0
author
#2 lore raw

PATCH v2 2/2 dmaengine: idxd: Fix refcount underflow on module unload

In reply to Yi Sun

A recent refactor introduced a misplaced put_device() call, leading to a reference count underflow during module unload.

There is no need to add additional put_device() calls for idxd groups, engines, or workqueues. Although commit a409e919ca3 claims:"Note, this also fixes the missing put_device() for idxd groups, engines, and wqs." It appears no such omission existed. The required cleanup is already handled by the call chain:
idxd_unregister_devices() -> device_unregister() -> put_device()

Extend idxd_cleanup() to perform the necessary cleanup, and remove idxd_cleanup_internals() which was not originally part of the driver unload path and introduced unintended reference count underflow.

Fixes a409e919ca32 ("dmaengine: idxd: Refactor remove call with idxd_cleanup() helper")
drivers/dma/idxd/init.c +6 -1
index 504aca0fd597..a5eabeb6a8bd 100644
@@ -1321,7 +1321,12 @@ static void idxd_remove(struct pci_dev *pdev)
 	device_unregister(idxd_confdev(idxd));
 	idxd_shutdown(pdev);
 	idxd_device_remove_debugfs(idxd);
-	idxd_cleanup(idxd);
+	perfmon_pmu_remove(idxd);
+	idxd_cleanup_interrupts(idxd);
+	if (device_pasid_enabled(idxd))
+		idxd_disable_system_pasid(idxd);
+	if (device_user_pasid_enabled(idxd))
+		idxd_disable_sva(idxd->pdev);
 	pci_iounmap(pdev, idxd->reg_base);
 	put_device(idxd_confdev(idxd));
 	pci_disable_device(pdev);
Signature · 1 line
2.43.0
#3 lore raw

Re: [PATCH v2 2/2] dmaengine: idxd: Fix refcount underflow on module unload

Quoted 22 lines · Yi Sun

A recent refactor introduced a misplaced put_device() call, leading to a reference count underflow during module unload.

There is no need to add additional put_device() calls for idxd groups, engines, or workqueues. Although commit a409e919ca3 claims:"Note, this also fixes the missing put_device() for idxd groups, engines, and wqs." It appears no such omission existed. The required cleanup is already handled by the call chain:
idxd_unregister_devices() -> device_unregister() -> put_device()

Extend idxd_cleanup() to perform the necessary cleanup, and remove idxd_cleanup_internals() which was not originally part of the driver unload path and introduced unintended reference count underflow.

Fixes a409e919ca32 ("dmaengine: idxd: Refactor remove call with idxd_cleanup() helper")
drivers/dma/idxd/init.c +0 -0
index 504aca0fd597..a5eabeb6a8bd 100644

drivers/dma/idxd/init.c static void idxd_remove(struct pci_dev *pdev) · line 1329

Quoted 10 lines
drivers/dma/idxd/init.c +6 -1
@@ -1321,7 +1321,12 @@ static void idxd_remove(struct pci_dev *pdev)
 	device_unregister(idxd_confdev(idxd));
 	idxd_shutdown(pdev);
 	idxd_device_remove_debugfs(idxd);
-	idxd_cleanup(idxd);
+	perfmon_pmu_remove(idxd);
+	idxd_cleanup_interrupts(idxd);
+	if (device_pasid_enabled(idxd))
+		idxd_disable_system_pasid(idxd);
+	if (device_user_pasid_enabled(idxd))
+		idxd_disable_sva(idxd->pdev);

idxd_disable_sva() got removed in commit 853b01b5efd7 ("dmaengine: idxd:
Remove unnecessary IOMMU_DEV_FEAT_IOPF")

Quoted 6 lines
drivers/dma/idxd/init.c +0 -0
 	pci_iounmap(pdev, idxd->reg_base);
 	put_device(idxd_confdev(idxd));
 	pci_disable_device(pdev);
Signature · 1 line
2.43.0

Cheers,

Signature · 1 line
Vinicius
author
#4 lore raw

Re: [PATCH v2 2/2] dmaengine: idxd: Fix refcount underflow on module unload

Quoted 29 lines · Vinicius Costa Gomes
Quoted 27 lines · Yi Sun

A recent refactor introduced a misplaced put_device() call, leading to a reference count underflow during module unload.

There is no need to add additional put_device() calls for idxd groups, engines, or workqueues. Although commit a409e919ca3 claims:"Note, this also fixes the missing put_device() for idxd groups, engines, and wqs." It appears no such omission existed. The required cleanup is already handled by the call chain:
idxd_unregister_devices() -> device_unregister() -> put_device()

Extend idxd_cleanup() to perform the necessary cleanup, and remove idxd_cleanup_internals() which was not originally part of the driver unload path and introduced unintended reference count underflow.

Fixes a409e919ca32 ("dmaengine: idxd: Refactor remove call with idxd_cleanup() helper")
drivers/dma/idxd/init.c +1 -1
index 504aca0fd597..a5eabeb6a8bd 100644
@@ -1321,7 +1321,12 @@ static void idxd_remove(struct pci_dev *pdev)
 	device_unregister(idxd_confdev(idxd));
 	idxd_shutdown(pdev);
 	idxd_device_remove_debugfs(idxd);
-	idxd_cleanup(idxd);
+	perfmon_pmu_remove(idxd);
Quoted 8 lines

drivers/dma/idxd/init.c static void idxd_remove(struct pci_dev *pdev) · line 1329

Quoted 5 lines
drivers/dma/idxd/init.c +5 -0
+	idxd_cleanup_interrupts(idxd);
+	if (device_pasid_enabled(idxd))
+		idxd_disable_system_pasid(idxd);
+	if (device_user_pasid_enabled(idxd))
+		idxd_disable_sva(idxd->pdev);

idxd_disable_sva() got removed in commit 853b01b5efd7 ("dmaengine: idxd:
Remove unnecessary IOMMU_DEV_FEAT_IOPF")

I get it, thanks Vinicius for the reminder. Will post another version.

Thanks
--Sun, Yi