dmaengine: fix kref underflow and UAF in dma_chan_put()
Subsystems (2)
v5 cover dmaengine: fix kref underflow and UAF in dma_chan_put()
Fix bugs related to dma_chan_put(), found while testing with SDXI[1].
[1]: https://lore.kernel.org/dmaengine/[email protected]
---
Changes in v5:
- Use int return type and add __must_check for dma_device_get() (Frank)
- Drop the comment above synchronize_rcu() and note in the commit message
that this will delay for grace period dma_list_mutex held (Logan)
- Link to v4: https://lore.kernel.org/r/[email protected]
Changes in v4:
- Add dma_device_get() helper (Frank)
- Drop dma_chan_put() move change (Frank)
- Link to v3: https://lore.kernel.org/r/[email protected]
Changes in v3:
- Add patch 3: add synchronize_rcu() to wait for RCU readers to prevent use-after-free. (Sashiko)
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Add patch 2 fixing the dma_chan_put()/dma_release_channel() use-after-free (sashiko)
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Shivank Garg (4):
dmaengine: add dma_device_get() helper
dmaengine: Fix device kref underflow in dma_chan_put()
dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
dmaengine: wait for RCU readers before releasing dma_device
drivers/dma/dmaengine.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)
---
base-commit: a4ff2be345d0abc943da8dd8da98151843b750dc
change-id: 20260518-dmaengine-kref-fix-7b21acb09455
Best regards,
Signature · 1 line
Shivank Garg <[email protected]>
PATCH v5 1/4 dmaengine: add dma_device_get() helper
Add dma_device_get() helper to match dma_device_put() to make code symmetric. It wraps open-coded kref_get_unless_zero() and asserts that dma_list_mutex is held, matching its put counterpart.
No functional change intended.
--- drivers/dma/dmaengine.c | 9 +++++++-- 1 file changed, 7 insertions(+), 2 deletions(-)
drivers/dma/dmaengine.c +7 -2
@@ -433,6 +433,12 @@ static void dma_device_release(struct kref *ref) device->device_release (device); } +static int __must_check dma_device_get (struct dma_device *device) +{ +lockdep_assert_held (&dma_list_mutex); +return kref_get_unless_zero (&device->ref); +} +static void dma_device_put (struct dma_device *device) {lockdep_assert_held (&dma_list_mutex);
@@ -460,8 +466,7 @@ static int dma_chan_get(struct dma_chan *chan)if (!try_module_get (owner))return -ENODEV ; - ret =kref_get_unless_zero (&chan->device->ref); -if (!ret) { +if (!dma_device_get (chan->device)) { ret = -ENODEV ; goto module_put_out; }
Signature · 1 line
2.43.0
PATCH v5 2/4 dmaengine: Fix device kref underflow in dma_chan_put()
In reply to Shivank Gargdma_chan_get() takes chan->device->ref only on the slow path:
/* no kref on fast path */
if (chan->client_count) {
__module_get(owner);
chan->client_count++;
return 0;
}
if (!try_module_get(owner))
return -ENODEV;
if (!dma_device_get(chan->device)) { // calls kref_get_unless_zero()dma_chan_put() drops the ref unconditionally, so every fast-path get/put pair drops one extra device reference.
The bug fires when two conditions hold together: a non-private provider has a persistent client holding chan->client_count > 0 and another client cycles dmaengine_get()/dmaengine_put(). When the kref hits zero, the subsequent dma_find_channel() returns NULL even though the provider module is still loaded.
Fix this by dropping device->ref only on the last put, matching the single slow-path get.
--- drivers/dma/dmaengine.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/dma/dmaengine.c +3 -1
@@ -520,7 +520,9 @@ static void dma_chan_put(struct dma_chan *chan) chan->route_data =NULL ; } -dma_device_put (chan->device); +/* This channel is not in use anymore, drop the device ref */ +if (!chan->client_count) +dma_device_put (chan->device);module_put (dma_chan_to_owner (chan)); }
Signature · 1 line
2.43.0
Re: [PATCH v5 2/4] dmaengine: Fix device kref underflow in dma_chan_put()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but dropping the last device reference in dma_chan_put() triggers device deallocation, leading to a Use-After-Free access in dma_release_channel(). While a subsequent patch in the series ('commit "dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()"') fixes a UAF inside dma_chan_put(), it fails to address the subsequent UAF in dma_release_channel() itself. --
--- Patch [2]: [PATCH v5 2/4] dmaengine: Fix device kref underflow in dma_chan_put() --- commit 3c7cc7c5c7f5082cf18d425005421e76b4f2aada
Author: Shivank Garg <[email protected]>
dmaengine: Fix device kref underflow in dma_chan_put()
dma_chan_get() takes chan->device->ref only on the slow path:
/* no kref on fast path */
if (chan->client_count) {
__module_get(owner);
chan->client_count++;
return 0;
}
if (!try_module_get(owner))
return -ENODEV;
if (!dma_device_get(chan->device)) { // calls kref_get_unless_zero()[ ... ]
drivers/dma/dmaengine.c static void dma_chan_put(struct dma_chan *chan) · line 525
Quoted 12 lines
drivers/dma/dmaengine.c +3
-1@@ -520,7 +520,9 @@ static void dma_chan_put(struct dma_chan *chan) chan->route_data =NULL ; } -dma_device_put (chan->device); +/* This channel is not in use anymore, drop the device ref */ +if (!chan->client_count) +dma_device_put (chan->device);
[Severity: High]
This is a pre-existing issue, but dropping the last device reference in dma_chan_put() can trigger device deallocation, leading to a use-after-free access in dma_release_channel().
When a DMA provider device is unregistered while a client still holds a channel, the provider's dma_async_device_unregister() drops its initial reference. Later, when the client releases the channel by calling dma_release_channel(), it calls dma_chan_put() which drops the final reference to the dma_device. This causes the device and its embedded dma_chan structures to be freed.
Immediately after dma_chan_put() returns, dma_release_channel() dereferences the freed memory:
drivers/dma/dmaengine.c:dma_release_channel() {
...
dma_chan_put(chan);
if (chan->slave) {
sysfs_remove_link(&chan->dev->device.kobj, DMA_SLAVE_NAME);
...
}While a subsequent patch in this series fixes a use-after-free inside dma_chan_put(), it appears to leave this subsequent use-after-free in dma_release_channel() unaddressed.
Could the channel sysfs cleanup and debugfs frees happen before we drop the final device reference?
Quoted 2 lines
drivers/dma/dmaengine.c +0
-0module_put (dma_chan_to_owner (chan)); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
PATCH v5 3/4 dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
In reply to Shivank GargWhen dma_device_put() drops the last reference on chan->device->ref, dma_device_release() runs and may free the dma_device along with its channels.
dma_chan_put() then still reads chan->device->owner via dma_chan_to_owner() for the trailing module_put(). KASAN catches it:
slab-use-after-free in dma_chan_put+0x3e6/0x4c0 Read of size 8 by task insmod/6319 Freed by task 6319: kfree+0x225/0x470 dma_chan_put+0x395/0x4c0 dmaengine_put+0xf8/0x160
Cache the module owner in dma_chan_put() before the put so the trailing module_put() does not need chan->device.
--- drivers/dma/dmaengine.c | 5 ++++- 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/dma/dmaengine.c +4 -1
@@ -500,10 +500,13 @@ static int dma_chan_get(struct dma_chan *chan)*/ static void dma_chan_put (struct dma_chan *chan) { +struct module *owner; +/* This channel is not in use, bail out */ if (!chan->client_count)return ; + owner =dma_chan_to_owner (chan); chan->client_count--;/* This channel is not in use anymore, free it */
@@ -523,7 +526,7 @@ static void dma_chan_put(struct dma_chan *chan)/* This channel is not in use anymore, drop the device ref */ if (!chan->client_count)dma_device_put (chan->device); -module_put (dma_chan_to_owner (chan)); +module_put (owner); }enum dma_status dma_sync_wait (struct dma_chan *chan,dma_cookie_t cookie)
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v5 3/4] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Pre-existing issues:
- [High] Unpatched Use-After-Free in dma_release_channel() due to dereferencing the channel after dma_chan_put().
- [High] Leaked channel, device, and module references in dmaengine_put() for unregistered public devices.
- [High] Double list_del_rcu() on device->global_node causes list corruption and kernel panic. --
--- Patch [3]: [PATCH v5 3/4] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel() --- commit f921c2034a3380b885abc856562df23d25421487
Author: Shivank Garg <[email protected]>
dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
When dma_device_put() drops the last reference on chan->device->ref, dma_device_release() runs and may free the dma_device along with its channels. This patch caches the module owner in dma_chan_put() before the put so the trailing module_put() does not need chan->device, preventing a use-after-free caught by kasan.
Quoted 4 lines
drivers/dma/dmaengine.c +0
-0
[ ... ]
drivers/dma/dmaengine.c static void dma_chan_put(struct dma_chan *chan) · line 530
Quoted 7 lines
drivers/dma/dmaengine.c +1
-1@@ -523,7 +526,7 @@ static void dma_chan_put(struct dma_chan *chan)/* This channel is not in use anymore, drop the device ref */ if (!chan->client_count)dma_device_put (chan->device); -module_put (dma_chan_to_owner (chan)); +module_put (owner); }
[Severity: High]
This isn't a regression introduced by this patch, but since this commit addresses a use-after-free involving dma_chan_put(), does dma_release_channel() have a similar issue?
If dma_async_device_unregister() was called earlier, dropping the base device reference, calling dma_release_channel() will invoke dma_chan_put(). This decrements the client count to zero, calls dma_device_put(), and drops the final reference, freeing the device and its embedded channel structure.
However, immediately after dma_chan_put(), dma_release_channel() dereferences the freed channel:
drivers/dma/dmaengine.c:dma_release_channel() {
...
dma_chan_put(chan);
if (chan->slave) {
sysfs_remove_link(&chan->dev->device.kobj, DMA_SLAVE_NAME);
...
}Does this cause a use-after-free when accessing chan->slave, chan->dev, and chan->name?
[Severity: High]
This is a pre-existing issue, but I noticed a potential resource leak in how unregistered public devices are handled.
When dmaengine_get() allocates public channels, it increments chan->client_count and takes device and module references. If the device is subsequently unregistered, dma_async_device_unregister() sets the DMA_PRIVATE flag to hide it:
drivers/dma/dmaengine.c:dma_async_device_unregister() {
...
dma_cap_set(DMA_PRIVATE, device->cap_mask);
...
}When dmaengine_put() runs later, it explicitly skips any device with DMA_PRIVATE set:
drivers/dma/dmaengine.c:dmaengine_put() {
...
if (dma_has_cap(DMA_PRIVATE, device->cap_mask))
continue;
list_for_each_entry(chan, &device->channels, device_node)
dma_chan_put(chan);
...
}Since dma_chan_put() is never called for the channels of the unregistered device, does this permanently leak chan->client_count, the module reference, and the device kref?
[Severity: High]
This is a pre-existing issue, but is there a risk of list corruption due to a double list deletion?
In find_candidate() and dmaengine_get(), if dma_chan_get() fails with -ENODEV (such as when the backing module is being unloaded), they proactively remove the device from the list:
drivers/dma/dmaengine.c:find_candidate() {
...
if (err == -ENODEV) {
...
list_del_rcu(&device->global_node);
...
}When the driver's exit routine subsequently executes dma_async_device_unregister(), it drops the device reference and eventually invokes dma_device_release(). This unconditionally deletes the node again:
drivers/dma/dmaengine.c:dma_device_release() {
...
list_del_rcu(&device->global_node);
...
}Can this double list_del_rcu() on device->global_node corrupt the doubly-linked list?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
PATCH v5 4/4 dmaengine: wait for RCU readers before releasing dma_device
In reply to Shivank Gargdma_issue_pending_all() walks the dma_device_list with list_for_each_entry_rcu() under rcu_read_lock(). dma_device_release() unlinks the device with list_del_rcu() and then calls device->device_release() (which in many drivers, such as plx_dma.c, directly calls kfree()).
Because there is no grace period between unlinking the device and freeing it, concurrent RCU readers in dma_issue_pending_all() can access the device after it has been freed.
The lockless walk originally relied on clients holding a dmaengine reference to pin the provider module, and therefore the device, for as long as they might traverse the list. Commit 8ad342a86359 ("dmaengine:
Add reference counting to dma_device struct") decoupled the dma_device lifetime from the module reference, so the device can now be released while a reader is still walking the list.
Add synchronize_rcu() before the device is freed, so RCU readers are guaranteed to have finished. Keep it unconditional: providers that do not implement device_release() free the device themselves once dma_async_device_unregister() returns. This call will delay for a grace period with dma_list_mutex held, which is safe and only teardown path is delayed.
--- drivers/dma/dmaengine.c | 1 + 1 file changed, 1 insertion(+)
drivers/dma/dmaengine.c +1 -0
@@ -428,6 +428,7 @@ static void dma_device_release(struct kref *ref)list_del_rcu (&device->global_node);dma_channel_rebalance (); +synchronize_rcu ();if (device->device_release) device->device_release (device);
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v5 4/4] dmaengine: wait for RCU readers before releasing dma_device
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Pre-existing issues:
- [High] Double list_del_rcu on a device's global_node leading to list corruption (LIST_POISON2 splats) or kernel panic when a provider module is unloaded.
- [High] The exported functions dma_async_device_channel_register and dma_async_device_channel_unregister call dma_channel_rebalance() without holding the required dma_list_mutex.
- [High] Lockless TOCTOU race in dma_issue_pending_all between chan->client_count check and device_issue_pending execution.
--
--- Patch [4]: [PATCH v5 4/4] dmaengine: wait for RCU readers before releasing dma_device --- commit 2e3a690f3e98bd55a6714eab025b6c1af7e3d406
Author: Shivank Garg <[email protected]>
dmaengine: wait for RCU readers before releasing dma_device
This patch introduces a synchronize_rcu() call before releasing the dma_device in dma_device_release(). It ensures concurrent RCU readers in dma_issue_pending_all() have finished traversing the list before the device is freed.
Quoted 2 lines
dma_issue_pending_all()walks thedma_device_listwithlist_for_each_entry_rcu()underrcu_read_lock().
[Severity: High]
This is a pre-existing issue, but can this lead to a lockless TOCTOU race between the client_count check and the execution of device_issue_pending()?
In dma_issue_pending_all(), the code locklessly checks if chan->client_count is non-zero before calling device->device_issue_pending(chan). Concurrently, dma_release_channel() or dmaengine_put() could acquire dma_list_mutex, decrement client_count to zero, and tear down the descriptor rings.
If dma_issue_pending_all() stalls between the check and execution, could it invoke device_issue_pending() on a channel that has just been released and disabled?
drivers/dma/dmaengine.c static void dma_device_release(struct kref *ref) · line 429
Quoted 7 lines
drivers/dma/dmaengine.c +0
-0@@ -428,6 +428,7 @@ static void dma_device_release(struct kref *ref)list_del_rcu (&device->global_node);
[Severity: High]
This isn't a bug introduced by this patch, but could this unconditional removal result in a double list_del_rcu() and list corruption during module unloading?
If try_module_get() fails in dma_chan_get() during a concurrent module unload, dmaengine_get() or find_candidate() will manually execute list_del_rcu(&device->global_node).
Later, when the unloading module calls dma_async_device_unregister() and drops the final reference, dma_device_release() is invoked. Will this cause a second list_del_rcu() on the same node and a kernel panic from dereferencing LIST_POISON2?
drivers/dma/dmaengine.c static void dma_device_release(struct kref *ref) · line 430
Quoted 1 line
drivers/dma/dmaengine.c +0
-0dma_channel_rebalance ();
[Severity: High]
This is a pre-existing issue, but is it safe for exported functions to call dma_channel_rebalance() without holding dma_list_mutex?
When drivers dynamically register or unregister channels using dma_async_device_channel_register() or dma_async_device_channel_unregister(), they call dma_channel_rebalance() completely unlocked.
Since dma_channel_rebalance() mutates the global channel_table and iterates over dma_device_list without RCU protection, could concurrent structural modifications to the list lead to data races or list corruption?
Quoted 4 lines
drivers/dma/dmaengine.c +1
-0+synchronize_rcu ();if (device->device_release) device->device_release (device);
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Quoted 33 lines · Shivank Garg
Fix bugs related to
dma_chan_put(), found while testing with SDXI[1].[1]: https://lore.kernel.org/dmaengine/[email protected]
Signed-off-by Shivank Garg <[email protected]>---
Changes in v5:
- Use int return type and add__must_checkfordma_device_get()(Frank)
- Drop the comment abovesynchronize_rcu()and note in the commit message
that this will delay for grace perioddma_list_mutexheld (Logan)
- Link to v4: https://lore.kernel.org/r/[email protected]Changes in v4:
- Adddma_device_get()helper (Frank)
- Dropdma_chan_put()move change (Frank)
- Link to v3: https://lore.kernel.org/r/[email protected]Changes in v3:
- Add patch 3: addsynchronize_rcu()to wait for RCU readers to prevent use-after-free. (Sashiko)
- Link to v2: https://lore.kernel.org/r/[email protected]Changes in v2:
- Add patch 2 fixing the dma_chan_put()/dma_release_channel() use-after-free (sashiko)
- Link to v1: https://lore.kernel.org/r/[email protected]---
Shivank Garg (4):
dmaengine: adddma_device_get()helper
dmaengine: Fix device kref underflow indma_chan_put()
dmaengine: fix use-after-free indma_chan_put()anddma_release_channel()
dmaengine: wait for RCU readers before releasingdma_device
Quoted 7 lines
drivers/dma/dmaengine.c | 19 +++++++++++++++---- 1 file changed, 15 insertions(+), 4 deletions(-)---
base-commit:a4ff2be345d0abc943da8dd8da98151843b750dc
change-id: 20260518-dmaengine-kref-fix-7b21acb09455Best regards,
Hi Vinod,
Would you consider this series for next?
It applies cleanly on v7.3-rc1.
Thanks,
Shivank
Quoted 5 lines · Shivank Garg
Fix bugs related to
dma_chan_put(), found while testing with SDXI[1].
Applied, thanks!
[1/4] dmaengine: add dma_device_get() helper
commit: 424103642d009f5a7dc33300a49d5606d1bf4fb5
[2/4] dmaengine: Fix device kref underflow in dma_chan_put()
commit: 44dab659064eb5c10adb0306510eebe848ed592d
[3/4] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
commit: e873c74132f0c5f1452816cd9bb26208f0bba1e1
[4/4] dmaengine: wait for RCU readers before releasing dma_device
commit: dc750422170a563c7a81f6e49d36bb02c62ae37fBest regards,
Signature · 1 line
~Vinod