dmaengine: fix kref underflow and UAF in dma_chan_put()
Subsystems (2)
v3 cover dmaengine: fix kref underflow and UAF in dma_chan_put()
Fix bugs related to dma_chan_put(), found while testing with SDXI[1].
[1]: https://lore.kernel.org/dmaengine/[email protected]
---
Changes in v3:
- Add patch 3: add synchronize_rcu() to wait for RCU readers to prevent use-after-free. (Sashiko)
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Add patch 2 fixing the dma_chan_put()/dma_release_channel() use-after-free (sashiko)
- Link to v1: https://lore.kernel.org/r/[email protected]
---
Shivank Garg (3):
dmaengine: Fix device kref underflow in dma_chan_put()
dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
dmaengine: wait for RCU readers before releasing dma_device
drivers/dma/dmaengine.c | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
---
base-commit: 0d995da5fb97e8c312834575604d4423eb6225b7
change-id: 20260518-dmaengine-kref-fix-7b21acb09455
Best regards,
Signature · 1 line
Shivank Garg <[email protected]>
PATCH v3 1/3 dmaengine: Fix device kref underflow in dma_chan_put()
dma_chan_get() takes chan->device->ref only on the slow path:
/* no kref on fast path */
if (chan->client_count) {
__module_get(owner);
chan->client_count++;
return 0;
}
if (!try_module_get(owner))
return -ENODEV;
ret = kref_get_unless_zero(&chan->device->ref);dma_chan_put() drops the ref unconditionally, so every fast-path get/put pair drops one extra device reference.
The bug fires when two conditions hold together: a non-private provider has a persistent client holding chan->client_count > 0 and another client cycles dmaengine_get()/dmaengine_put(). When the kref hits zero, the subsequent dma_find_channel() returns NULL even though the provider module is still loaded.
Fix this by dropping device->ref only on the last put, matching the single slow-path get.
--- drivers/dma/dmaengine.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
drivers/dma/dmaengine.c +3 -1
@@ -515,7 +515,9 @@ static void dma_chan_put(struct dma_chan *chan) chan->route_data =NULL ; } -dma_device_put (chan->device); +/* This channel is not in use anymore, drop the device ref */ +if (!chan->client_count) +dma_device_put (chan->device);module_put (dma_chan_to_owner (chan)); }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v3 1/3] dmaengine: Fix device kref underflow in dma_chan_put()
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Pre-existing issues:
- [High] dma_chan_put() accesses chan->device->owner after potentially freeing chan and chan->device via dma_device_put().
- [High] dma_release_channel() accesses chan fields after calling dma_chan_put(), which may have freed the channel.
- [High] RCU read-side critical section in dma_issue_pending_all() can access a synchronously freed struct dma_device. --
--- Patch [1]: [PATCH v3 1/3] dmaengine: Fix device kref underflow in dma_chan_put() --- commit 0f010c1f7452238748341b884bda3d3bba2c74ac
Author: Shivank Garg <[email protected]>
dmaengine: Fix device kref underflow in dma_chan_put()
This commit fixes a device reference underflow bug by ensuring the device reference is only dropped on the last put of a channel, matching the behavior of the get path.
drivers/dma/dmaengine.c static void dma_chan_put(struct dma_chan *chan) · line 521
Quoted 13 lines
drivers/dma/dmaengine.c +3
-1@@ -515,7 +515,9 @@ static void dma_chan_put(struct dma_chan *chan) chan->route_data =NULL ; } -dma_device_put (chan->device); +/* This channel is not in use anymore, drop the device ref */ +if (!chan->client_count) +dma_device_put (chan->device);module_put (dma_chan_to_owner (chan));
[Severity: High]
This is a pre-existing issue, but does accessing chan after dma_device_put() risk a use-after-free?
If dma_device_put() drops the last reference to the DMA device, it can trigger synchronous release of the device memory, which might also free the embedded dma_chan.
Could dma_chan_to_owner(chan) then dereference freed memory by evaluating chan->device->owner?
drivers/dma/dmaengine.c static void dma_chan_put(struct dma_chan *chan) · line 522
Quoted 1 line
drivers/dma/dmaengine.c +0
-0}
[Severity: High]
This isn't a bug introduced by this patch, but does dma_release_channel() also access freed memory under the same conditions?
drivers/dma/dmaengine.c:dma_release_channel() {
...
dma_chan_put(chan);
if (chan->slave) {
sysfs_remove_link(&chan->dev->device.kobj, DMA_SLAVE_NAME);
...
}If dma_chan_put() frees the channel when the last device reference is dropped, is it safe to access chan->slave and chan->dev afterwards to tear down the sysfs links?
[Severity: High]
This is also a pre-existing issue, but does the RCU read side in dma_issue_pending_all() race with synchronous device release?
dma_issue_pending_all() relies on an RCU read lock to traverse the device list:
drivers/dma/dmaengine.c:dma_issue_pending_all() {
...
rcu_read_lock();
list_for_each_entry_rcu(device, &dma_device_list, global_node) {
if (dma_has_cap(DMA_PRIVATE, device->cap_mask))
...
}However, dma_device_release() unlinks the device and synchronously frees it without observing an RCU grace period:
drivers/dma/dmaengine.c:dma_device_release() {
...
list_del_rcu(&device->global_node);
dma_channel_rebalance();
if (device->device_release)
device->device_release(device);
...
}Could the RCU reader dereference a struct dma_device that was just freed by dma_device_release()?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Re: [PATCH v3 1/3] dmaengine: Fix device kref underflow in dma_chan_put()
Quoted 36 lines · On Sun, 2026-08-16 at 16:02 +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Pre-existing issues:
- [High]dma_chan_put()accesseschan->device->ownerafter potentially freeing chan andchan->deviceviadma_device_put().
- [High]dma_release_channel()accesseschanfields after callingdma_chan_put(), which may have freed the channel.
- [High] RCU read-side critical section indma_issue_pending_all()can access a synchronously freedstruct dma_device. ----- Patch [1]: [PATCH v3 1/3] dmaengine: Fix device kref underflow in
dma_chan_put()--- commit0f010c1f7452238748341b884bda3d3bba2c74ac
Author: Shivank Garg <[email protected]>dmaengine: Fix device kref underflow in
dma_chan_put()This commit fixes a device reference underflow bug by ensuring the device reference is only dropped on the last put of a channel, matching the behavior of the get path.
drivers/dma/dmaengine.c static void dma_chan_put(struct dma_chan *chan) · line 521
Quoted 13 lines
drivers/dma/dmaengine.c +3
-1@@ -515,7 +515,9 @@ static void dma_chan_put(struct dma_chan *chan) chan->route_data =NULL ; } -dma_device_put (chan->device); +/* This channel is not in use anymore, drop the device ref */ +if (!chan->client_count) +dma_device_put (chan->device);module_put (dma_chan_to_owner (chan));[Severity: High]
This is a pre-existing issue, but does accessing chan afterdma_device_put()risk a use-after-free?
Quoted 7 lines
If
dma_device_put()drops the last reference to the DMA device, it can trigger synchronous release of the device memory, which might also free the embeddeddma_chan.Could
dma_chan_to_owner(chan)then dereference freed memory by evaluatingchan->device->owner?
I'm already fixing this in patch 2 by caching chan->device->owner.
Quoted 15 lines
drivers/dma/dmaengine.c static void dma_chan_put(struct dma_chan *chan) · line 522
Quoted 1 line
drivers/dma/dmaengine.c +0
-0}[Severity: High]
This isn't a bug introduced by this patch, but doesdma_release_channel()also access freed memory under the same conditions?drivers/dma/dmaengine.c:dma_release_channel() { ... dma_chan_put(chan); if (chan->slave) { sysfs_remove_link(&chan->dev->device.kobj, DMA_SLAVE_NAME); ... }
Quoted 3 lines
If
dma_chan_put()frees the channel when the last device reference is dropped, is it safe to accesschan->slaveandchan->devafterwards to tear down the sysfs links?
I addressed this in patch 2, but Sashiko pointed the fix would cause NULL dereference in stm32_dma3_pm_suspend().
Quoted 28 lines
[Severity: High]
This is also a pre-existing issue, but does the RCU read side indma_issue_pending_all()race with synchronous device release?
dma_issue_pending_all()relies on an RCU read lock to traverse the device list:drivers/dma/dmaengine.c:dma_issue_pending_all() { ... rcu_read_lock(); list_for_each_entry_rcu(device, &dma_device_list, global_node) { if (dma_has_cap(DMA_PRIVATE, device->cap_mask)) ... }However,
dma_device_release()unlinks the device and synchronously frees it without observing an RCU grace period:drivers/dma/dmaengine.c:dma_device_release() { ... list_del_rcu(&device->global_node); dma_channel_rebalance(); if (device->device_release) device->device_release(device); ... }
Quoted 3 lines
Could the RCU reader dereference a struct
dma_devicethat was just freed bydma_device_release()?
Fixed in Patch 3 by adding synchronize_rcu().
Thanks,
Shivank
Re: [PATCH v3 1/3] dmaengine: Fix device kref underflow in dma_chan_put()
In reply to Shivank GargQuoted 23 lines · Shivank Garg
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
dma_chan_get()takeschan->device->refonly on the slow path:/* no kref on fast path */ if (chan->client_count) { __module_get(owner); chan->client_count++; return 0; } if (!try_module_get(owner)) return -ENODEV; ret = kref_get_unless_zero(&chan->device->ref);
dma_chan_put()drops the ref unconditionally, so every fast-path get/put pair drops one extra device reference.The bug fires when two conditions hold together: a non-private provider has a persistent client holding
chan->client_count> 0 and another client cycles dmaengine_get()/dmaengine_put(). When the kref hits zero, the subsequentdma_find_channel()returns NULL even though the provider module is still loaded.
Quoted 6 lines
Fix this by dropping
device->refonly on the last put, matching the single slow-path get.Fixes 8ad342a86359 ("dmaengine: Add reference counting to dma_device struct")Signed-off-by Shivank Garg <[email protected]>---
Nit: add helper dma_device_get() to match dma_device_put() to make code more symmetry.
Quoted 22 lines
drivers/dma/dmaengine.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)drivers/dma/dmaengine.c +3
-2@@ -515,7 +515,9 @@ static void dma_chan_put(struct dma_chan *chan) chan->route_data =NULL ; } -dma_device_put (chan->device); +/* This channel is not in use anymore, drop the device ref */ +if (!chan->client_count) +dma_device_put (chan->device);module_put (dma_chan_to_owner (chan)); } --2.43.0
Re: [PATCH v3 1/3] dmaengine: Fix device kref underflow in dma_chan_put()
Quoted 29 lines · Frank Li
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Quoted 26 lines · Shivank Garg
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
dma_chan_get()takeschan->device->refonly on the slow path:/* no kref on fast path */ if (chan->client_count) { __module_get(owner); chan->client_count++; return 0; } if (!try_module_get(owner)) return -ENODEV; ret = kref_get_unless_zero(&chan->device->ref);
dma_chan_put()drops the ref unconditionally, so every fast-path get/put pair drops one extra device reference.The bug fires when two conditions hold together: a non-private provider has a persistent client holding
chan->client_count> 0 and another client cycles dmaengine_get()/dmaengine_put(). When the kref hits zero, the subsequentdma_find_channel()returns NULL even though the provider module is still loaded.Fix this by dropping
device->refonly on the last put, matching the single slow-path get.
Quoted 6 lines
Quoted 3 lines
Fixes 8ad342a86359 ("dmaengine: Add reference counting to dma_device struct")Signed-off-by Shivank Garg <[email protected]>---
Nit: add helper
dma_device_get()to matchdma_device_put()to make code more symmetry.
Ack.
Quoted 4 lines
Reviewed-by Frank Li <[email protected]>
Thanks for the review.
Best regards,
Shivank
PATCH v3 2/3 dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
In reply to Shivank GargWhen dma_device_put() drops the last reference on chan->device->ref, dma_device_release() runs and may free the dma_device along with its channels.
Two paths still read that memory after the put:
- dma_chan_put() reads chan->device->owner via dma_chan_to_owner() for the trailing module_put().
- dma_release_channel() calls dma_chan_put() before reading chan->slave, chan->name, chan->dev and chan->dbg_client_name.
KASAN catches the first one:
slab-use-after-free in dma_chan_put+0x3e6/0x4c0 Read of size 8 by task insmod/6319 Freed by task 6319: kfree+0x225/0x470 dma_chan_put+0x395/0x4c0 dmaengine_put+0xf8/0x160
Cache the module owner in dma_chan_put() before the put so the trailing module_put() does not need chan->device. In dma_release_channel(), move dma_chan_put() to the end, after every chan/device read.
--- drivers/dma/dmaengine.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)
drivers/dma/dmaengine.c +5 -3
@@ -495,10 +495,13 @@ static int dma_chan_get(struct dma_chan *chan)*/ static void dma_chan_put (struct dma_chan *chan) { +struct module *owner; +/* This channel is not in use, bail out */ if (!chan->client_count)return ; + owner =dma_chan_to_owner (chan); chan->client_count--;/* This channel is not in use anymore, free it */
@@ -518,7 +521,7 @@ static void dma_chan_put(struct dma_chan *chan)/* This channel is not in use anymore, drop the device ref */ if (!chan->client_count)dma_device_put (chan->device); -module_put (dma_chan_to_owner (chan)); +module_put (owner); }enum dma_status dma_sync_wait (struct dma_chan *chan,dma_cookie_t cookie)
@@ -916,8 +919,6 @@ void dma_release_channel(struct dma_chan *chan)if (--chan->device->privatecnt ==0 )dma_cap_clear (DMA_PRIVATE , chan->device->cap_mask); -dma_chan_put (chan); -if (chan->slave) {sysfs_remove_link (&chan->dev->device.kobj,DMA_SLAVE_NAME );sysfs_remove_link (&chan->slave->kobj, chan->name);
@@ -930,6 +931,7 @@ void dma_release_channel(struct dma_chan *chan)kfree (chan->dbg_client_name); chan->dbg_client_name =NULL ;#endif +dma_chan_put (chan);mutex_unlock (&dma_list_mutex); }EXPORT_SYMBOL_GPL (dma_release_channel);
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v3 2/3] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] Clearing `chan->slave` before stopping the hardware introduces a NULL pointer dereference in…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Clearing chan->slave before stopping the hardware introduces a NULL pointer dereference in stm32_dma3_pm_suspend(). --
--- Patch [2]: [PATCH v3 2/3] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel() --- commit c0bca60dc6977cca7352bacd1095081d89c88211
Author: Shivank Garg <[email protected]>
dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
When dma_device_put() drops the last reference on chan->device->ref, dma_device_release() may free the dma_device. This commit caches the module owner and delays dma_chan_put() to prevent reading freed memory.
Quoted 11 lines
drivers/dma/dmaengine.c +0
-2@@ -916,8 +919,6 @@ void dma_release_channel(struct dma_chan *chan)if (--chan->device->privatecnt ==0 )dma_cap_clear (DMA_PRIVATE , chan->device->cap_mask); -dma_chan_put (chan); -if (chan->slave) {sysfs_remove_link (&chan->dev->device.kobj,DMA_SLAVE_NAME );sysfs_remove_link (&chan->slave->kobj, chan->name);
drivers/dma/dmaengine.c
Quoted 10 lines
drivers/dma/dmaengine.c +1
-0kfree (chan->name); chan->name =NULL ; chan->slave =NULL ; }#ifdef CONFIG_DEBUG_FS kfree (chan->dbg_client_name); chan->dbg_client_name =NULL ;#endif +dma_chan_put (chan);
[Severity: High]
Could moving dma_chan_put() here introduce a NULL pointer dereference during system suspend?
By moving dma_chan_put() after the block that clears chan->slave, the hardware channel may still be active while chan->slave is NULL. If a system suspend occurs concurrently, drivers like stm32-dma3 might check the hardware state and attempt to print the slave device name:
drivers/dma/stm32/stm32-dma3.c:stm32_dma3_pm_suspend() {
...
ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id));
if (ccr & CCR_EN) {
dev_warn(dev, "Suspend is prevented: %s still in use by %s\n",
dma_chan_name(c), dev_name(c->slave));
...
}Since dma_chan_put() (which stops the hardware) hasn't been called yet, CCR_EN would still be active, and dev_name(c->slave) would dereference the NULL pointer we just set.
Is there a way to stop the channel before clearing the slave pointer without triggering the original use-after-free?
Quoted 2 lines
mutex_unlock(&dma_list_mutex);
}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [PATCH v3 2/3] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
Quoted 56 lines · On Sun, 2026-08-16 at 16:07 +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Clearingchan->slavebefore stopping the hardware introduces a NULL pointer dereference instm32_dma3_pm_suspend(). ----- Patch [2]: [PATCH v3 2/3] dmaengine: fix use-after-free in
dma_chan_put()anddma_release_channel()--- commitc0bca60dc6977cca7352bacd1095081d89c88211
Author: Shivank Garg <[email protected]>dmaengine: fix use-after-free in
dma_chan_put()anddma_release_channel()When
dma_device_put()drops the last reference onchan->device->ref,dma_device_release()may free thedma_device. This commit caches the module owner and delaysdma_chan_put()to prevent reading freed memory.Quoted 11 lines
drivers/dma/dmaengine.c +0
-2@@ -916,8 +919,6 @@ void dma_release_channel(struct dma_chan *chan)if (--chan->device->privatecnt ==0 )dma_cap_clear (DMA_PRIVATE , chan->device->cap_mask); -dma_chan_put (chan); -if (chan->slave) {sysfs_remove_link (&chan->dev->device.kobj,DMA_SLAVE_NAME );sysfs_remove_link (&chan->slave->kobj, chan->name);drivers/dma/dmaengine.c
Quoted 10 lines
drivers/dma/dmaengine.c +1
-0kfree (chan->name); chan->name =NULL ; chan->slave =NULL ; }#ifdef CONFIG_DEBUG_FS kfree (chan->dbg_client_name); chan->dbg_client_name =NULL ;#endif +dma_chan_put (chan);[Severity: High]
Could movingdma_chan_put()here introduce a NULL pointer dereference during system suspend?By moving
dma_chan_put()after the block that clearschan->slave, the hardware channel may still be active whilechan->slaveis NULL. If a system suspend occurs concurrently, drivers like stm32-dma3 might check the hardware state and attempt to print the slave device name:drivers/dma/stm32/stm32-dma3.c:stm32_dma3_pm_suspend() { ... ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)); if (ccr & CCR_EN) { dev_warn(dev, "Suspend is prevented: %s still in use by %s\n", dma_chan_name(c), dev_name(c->slave)); ... }
Quoted 6 lines
Since
dma_chan_put()(which stops the hardware) hasn't been called yet,CCR_ENwould still be active, anddev_name(c->slave)would dereference the NULL pointer we just set.Is there a way to stop the channel before clearing the slave pointer without triggering the original use-after-free?
This is a valid concern.
So, either we can have a NULL check in stm32 code while printing device name, or drop dma_chan_put() change for now to avoid this trigger?
Thanks,
Shivank
Re: [PATCH v3 2/3] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
Quoted 63 lines · Shivank
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Quoted 60 lines · On Sun, 2026-08-16 at 16:07 +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Clearingchan->slavebefore stopping the hardware introduces a NULL pointer dereference instm32_dma3_pm_suspend(). ----- Patch [2]: [PATCH v3 2/3] dmaengine: fix use-after-free in
dma_chan_put()anddma_release_channel()--- commitc0bca60dc6977cca7352bacd1095081d89c88211
Author: Shivank Garg <[email protected]>dmaengine: fix use-after-free in
dma_chan_put()anddma_release_channel()When
dma_device_put()drops the last reference onchan->device->ref,dma_device_release()may free thedma_device. This commit caches the module owner and delaysdma_chan_put()to prevent reading freed memory.Quoted 16 lines
drivers/dma/dmaengine.c +0
-2@@ -916,8 +919,6 @@ void dma_release_channel(struct dma_chan *chan)if (--chan->device->privatecnt ==0 )dma_cap_clear (DMA_PRIVATE , chan->device->cap_mask); -dma_chan_put (chan); -if (chan->slave) {sysfs_remove_link (&chan->dev->device.kobj,DMA_SLAVE_NAME );sysfs_remove_link (&chan->slave->kobj, chan->name);kfree(chan->name); chan->name = NULL; chan->slave = NULL; }Quoted 5 lines
#ifdef CONFIG_DEBUG_FS kfree(chan->dbg_client_name); chan->dbg_client_name = NULL;#endif
+dma_chan_put(chan);[Severity: High]
Could movingdma_chan_put()here introduce a NULL pointer dereference during system suspend?By moving
dma_chan_put()after the block that clearschan->slave, the hardware channel may still be active whilechan->slaveis NULL. If a system suspend occurs concurrently, drivers like stm32-dma3 might check the hardware state and attempt to print the slave device name:drivers/dma/stm32/stm32-dma3.c:stm32_dma3_pm_suspend() { ... ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)); if (ccr & CCR_EN) { dev_warn(dev, "Suspend is prevented: %s still in use by %s\n", dma_chan_name(c), dev_name(c->slave)); ... }Since
dma_chan_put()(which stops the hardware) hasn't been called yet,CCR_ENwould still be active, anddev_name(c->slave)would dereference the NULL pointer we just set.
Quoted 6 lines
Quoted 2 lines
Is there a way to stop the channel before clearing the slave pointer without triggering the original use-after-free?
This is a valid concern.
So, either we can have a NULL check in stm32 code while printing device name, or dropdma_chan_put()change for now to avoid this trigger?
Drop dma_chan_put() firstly.
Frank
Quoted 6 lines
Thanks,
Shivank
Re: [PATCH v3 2/3] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
Quoted 69 lines · Frank Li
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Quoted 66 lines · Shivank
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Quoted 62 lines · On Sun, 2026-08-16 at 16:07 +0000, [email protected] wrote:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] Clearingchan->slavebefore stopping the hardware introduces a NULL pointer dereference instm32_dma3_pm_suspend(). ----- Patch [2]: [PATCH v3 2/3] dmaengine: fix use-after-free in
dma_chan_put()anddma_release_channel()--- commitc0bca60dc6977cca7352bacd1095081d89c88211
Author: Shivank Garg <[email protected]>dmaengine: fix use-after-free in
dma_chan_put()anddma_release_channel()When
dma_device_put()drops the last reference onchan->device->ref,dma_device_release()may free thedma_device. This commit caches the module owner and delaysdma_chan_put()to prevent reading freed memory.Quoted 16 lines
drivers/dma/dmaengine.c +0
-2@@ -916,8 +919,6 @@ void dma_release_channel(struct dma_chan *chan)if (--chan->device->privatecnt ==0 )dma_cap_clear (DMA_PRIVATE , chan->device->cap_mask); -dma_chan_put (chan); -if (chan->slave) {sysfs_remove_link (&chan->dev->device.kobj,DMA_SLAVE_NAME );sysfs_remove_link (&chan->slave->kobj, chan->name);kfree(chan->name); chan->name = NULL; chan->slave = NULL; }Quoted 5 lines
#ifdef CONFIG_DEBUG_FS kfree(chan->dbg_client_name); chan->dbg_client_name = NULL;#endif
+dma_chan_put(chan);[Severity: High]
Could movingdma_chan_put()here introduce a NULL pointer dereference during system suspend?By moving
dma_chan_put()after the block that clearschan->slave, the hardware channel may still be active whilechan->slaveis NULL. If a system suspend occurs concurrently, drivers like stm32-dma3 might check the hardware state and attempt to print the slave device name:drivers/dma/stm32/stm32-dma3.c:stm32_dma3_pm_suspend() { ... ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)); if (ccr & CCR_EN) { dev_warn(dev, "Suspend is prevented: %s still in use by %s\n", dma_chan_name(c), dev_name(c->slave)); ... }Since
dma_chan_put()(which stops the hardware) hasn't been called yet,CCR_ENwould still be active, anddev_name(c->slave)would dereference the NULL pointer we just set.Is there a way to stop the channel before clearing the slave pointer without triggering the original use-after-free?
Quoted 9 lines
Quoted 3 lines
This is a valid concern.
So, either we can have a NULL check in stm32 code while printing device name, or dropdma_chan_put()change for now to avoid this trigger?Drop
dma_chan_put()firstly.Frank
Sure
Thanks,
Shivank
Re: [PATCH v3 2/3] dmaengine: fix use-after-free in dma_chan_put() and dma_release_channel()
In reply to Shivank GargQuoted 25 lines · Shivank Garg
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
When
dma_device_put()drops the last reference onchan->device->ref,dma_device_release()runs and may free thedma_devicealong with its channels.Two paths still read that memory after the put:
-dma_chan_put()readschan->device->ownerviadma_chan_to_owner()for the trailingmodule_put().
-dma_release_channel()callsdma_chan_put()before readingchan->slave,chan->name,chan->devandchan->dbg_client_name.KASAN catches the first one:
slab-use-after-free in dma_chan_put+0x3e6/0x4c0 Read of size 8 by task insmod/6319 Freed by task 6319: kfree+0x225/0x470 dma_chan_put+0x395/0x4c0 dmaengine_put+0xf8/0x160Cache the module owner in
dma_chan_put()before the put so the trailingmodule_put()does not needchan->device. Indma_release_channel(), movedma_chan_put()to the end, after every chan/device read.
Quoted 5 lines
Fixes 8ad342a86359 ("dmaengine: Add reference counting to dma_device struct")Suggested-by Sashiko <[email protected]>Signed-off-by Shivank Garg <[email protected]>---
Quoted 51 lines
drivers/dma/dmaengine.c | 8 +++++--- 1 file changed, 5 insertions(+), 3 deletions(-)drivers/dma/dmaengine.c +5
-4@@ -495,10 +495,13 @@ static int dma_chan_get(struct dma_chan *chan)*/ static void dma_chan_put (struct dma_chan *chan) { +struct module *owner; +/* This channel is not in use, bail out */ if (!chan->client_count)return ; + owner =dma_chan_to_owner (chan); chan->client_count--;/* This channel is not in use anymore, free it */ @@ -518,7 +521,7 @@ static void dma_chan_put(struct dma_chan *chan)/* This channel is not in use anymore, drop the device ref */ if (!chan->client_count)dma_device_put (chan->device); -module_put (dma_chan_to_owner (chan)); +module_put (owner); }enum dma_status dma_sync_wait (struct dma_chan *chan,dma_cookie_t cookie)@@ -916,8 +919,6 @@ void dma_release_channel(struct dma_chan *chan)if (--chan->device->privatecnt ==0 )dma_cap_clear (DMA_PRIVATE , chan->device->cap_mask); -dma_chan_put (chan); -if (chan->slave) {sysfs_remove_link (&chan->dev->device.kobj,DMA_SLAVE_NAME );sysfs_remove_link (&chan->slave->kobj, chan->name);@@ -930,6 +931,7 @@ void dma_release_channel(struct dma_chan *chan)kfree (chan->dbg_client_name); chan->dbg_client_name =NULL ;#endif +dma_chan_put (chan);mutex_unlock (&dma_list_mutex); }EXPORT_SYMBOL_GPL (dma_release_channel); --2.43.0
PATCH v3 3/3 dmaengine: wait for RCU readers before releasing dma_device
In reply to Shivank Gargdma_issue_pending_all() walks the dma_device_list with list_for_each_entry_rcu() under rcu_read_lock(). dma_device_release() unlinks the device with list_del_rcu() and then calls device->device_release() (which in many drivers, such as plx_dma.c, directly calls kfree()).
Because there is no grace period between unlinking the device and freeing it, concurrent RCU readers in dma_issue_pending_all() can access the device after it has been freed.
The lockless walk originally relied on clients holding a dmaengine reference to pin the provider module, and therefore the device, for as long as they might traverse the list. Commit 8ad342a86359 ("dmaengine:
Add reference counting to dma_device struct") decoupled the dma_device lifetime from the module reference, so the device can now be released while a reader is still walking the list.
Add synchronize_rcu() before the device is freed, so RCU readers are guaranteed to have finished. Keep it unconditional: providers that do not implement device_release() free the device themselves once dma_async_device_unregister() returns, so they need the same grace period.
--- drivers/dma/dmaengine.c | 6 ++++++ 1 file changed, 6 insertions(+)
drivers/dma/dmaengine.c +6 -0
@@ -429,6 +429,12 @@ static void dma_device_release(struct kref *ref)list_del_rcu (&device->global_node);dma_channel_rebalance (); +/* +* Wait for RCU readers (e.g. dma_issue_pending_all()) that may still +* be traversing dma_device_list before the device is freed. +*/ +synchronize_rcu (); +if (device->device_release) device->device_release (device); }
Signature · 1 line
2.43.0
[email protected]
AI reviewRe: [PATCH v3 3/3] dmaengine: wait for RCU readers before releasing dma_device
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
Pre-existing issues:
- [High] Double list_del_rcu() causes list corruption when DMA provider module unloads concurrently with a channel request.
- [High] A Time-of-Check to Time-of-Use (TOCTOU) race condition between dma_issue_pending_all() and dma_chan_put() leads to memory corruption by operating on freed DMA channel resources. --
--- Patch [3]: [PATCH v3 3/3] dmaengine: wait for RCU readers before releasing dma_device --- commit 6738eea6e01509c5c24f2cdb086fe3fd0ffab13a
Author: Shivank Garg <[email protected]>
dmaengine: wait for RCU readers before releasing dma_device
dma_issue_pending_all() walks the dma_device_list with list_for_each_entry_rcu() under rcu_read_lock(). dma_device_release() unlinks the device with list_del_rcu() and then calls device->device_release(). This patch adds synchronize_rcu() before freeing the device so RCU readers are guaranteed to have finished.
drivers/dma/dmaengine.c static void dma_device_release(struct kref *ref) · line 429
Quoted 6 lines
drivers/dma/dmaengine.c +0
-0@@ -429,6 +429,12 @@ static void dma_device_release(struct kref *ref)list_del_rcu (&device->global_node);
[Severity: High]
This isn't a bug introduced by this patch, but can this lead to a double list_del_rcu() execution and list corruption if the module is unloaded concurrently with a channel request?
If a thread calls dmaengine_get() or find_candidate(), it iterates dma_device_list and calls dma_chan_get(). If another thread concurrently unloads the module:
1. dma_chan_get() calls try_module_get(), which fails and returns -ENODEV
2. dmaengine_get() evaluates -ENODEV and eagerly deletes the device via list_del_rcu(&device->global_node)
3. Later, the unloading thread unregisters the device, which invokes dma_device_release()
4. dma_device_release() unconditionally calls list_del_rcu(&device->global_node) again on the already removed node
Calling list_del_rcu() twice on the same node poisons the prev pointer, which can cause a panic in __list_del_entry().
drivers/dma/dmaengine.c static void dma_device_release(struct kref *ref) · line 436
Quoted 7 lines
drivers/dma/dmaengine.c +5
-0dma_channel_rebalance (); +/* +* Wait for RCU readers (e.g. dma_issue_pending_all()) that may still +* be traversing dma_device_list before the device is freed. +*/ +synchronize_rcu ();
[Severity: High]
This is a pre-existing issue, but does dma_issue_pending_all() still have a TOCTOU race condition with dma_chan_put() that could lead to memory corruption?
While this synchronize_rcu() protects the dma_device itself during driver unbind, it doesn't seem to protect against channel release.
If dma_issue_pending_all() evaluates chan->client_count locklessly under rcu_read_lock():
Thread A:dma_issue_pending_all()
if (chan->client_count)
// preempted hereThread B:dma_release_channel()
dma_chan_put()
// drops client_count to 0
chan->device->device_free_chan_resources(chan);Thread A resumes:
device->device_issue_pending(chan);
The driver writes to the hardware, forcing it to fetch from the newly freed DMA ring. Can the hardware be safely started here if the resources were just freed?
Quoted 4 lines
drivers/dma/dmaengine.c +1
-0+if (device->device_release) device->device_release (device); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Re: [PATCH v3 3/3] dmaengine: wait for RCU readers before releasing dma_device
In reply to Shivank GargQuoted 25 lines · Shivank Garg
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
dma_issue_pending_all()walks thedma_device_listwithlist_for_each_entry_rcu()underrcu_read_lock().dma_device_release()unlinks the device withlist_del_rcu()and then callsdevice->device_release()(which in many drivers, such asplx_dma.c, directly callskfree()).Because there is no grace period between unlinking the device and freeing it, concurrent RCU readers in
dma_issue_pending_all()can access the device after it has been freed.The lockless walk originally relied on clients holding a dmaengine reference to pin the provider module, and therefore the device, for as long as they might traverse the list. Commit
8ad342a86359("dmaengine:
Add reference counting todma_devicestruct") decoupled thedma_devicelifetime from the module reference, so the device can now be released while a reader is still walking the list.Add
synchronize_rcu()before the device is freed, so RCU readers are guaranteed to have finished. Keep it unconditional: providers that do not implementdevice_release()free the device themselves oncedma_async_device_unregister()returns, so they need the same grace period.
Quoted 5 lines
Fixes 2ba05622b8b1 ("dmaengine: provide a common 'issue_pending_all' implementation")Suggested-by Sashiko <[email protected]>Signed-off-by Shivank Garg <[email protected]>---
Quoted 24 lines
drivers/dma/dmaengine.c | 6 ++++++ 1 file changed, 6 insertions(+)drivers/dma/dmaengine.c +6
-1@@ -429,6 +429,12 @@ static void dma_device_release(struct kref *ref)list_del_rcu (&device->global_node);dma_channel_rebalance (); +/* +* Wait for RCU readers (e.g. dma_issue_pending_all()) that may still +* be traversing dma_device_list before the device is freed. +*/ +synchronize_rcu (); +if (device->device_release) device->device_release (device); } --2.43.0
Re: [PATCH v3 3/3] dmaengine: wait for RCU readers before releasing dma_device
Quoted 28 lines · Frank Li
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
Quoted 25 lines · Shivank Garg
[You don't often get email from [email protected]. Learn why this is important at https://aka.ms/LearnAboutSenderIdentification ]
dma_issue_pending_all()walks thedma_device_listwithlist_for_each_entry_rcu()underrcu_read_lock().dma_device_release()unlinks the device withlist_del_rcu()and then callsdevice->device_release()(which in many drivers, such asplx_dma.c, directly callskfree()).Because there is no grace period between unlinking the device and freeing it, concurrent RCU readers in
dma_issue_pending_all()can access the device after it has been freed.The lockless walk originally relied on clients holding a dmaengine reference to pin the provider module, and therefore the device, for as long as they might traverse the list. Commit
8ad342a86359("dmaengine:
Add reference counting todma_devicestruct") decoupled thedma_devicelifetime from the module reference, so the device can now be released while a reader is still walking the list.Add
synchronize_rcu()before the device is freed, so RCU readers are guaranteed to have finished. Keep it unconditional: providers that do not implementdevice_release()free the device themselves oncedma_async_device_unregister()returns, so they need the same grace period.
Quoted 9 lines
Quoted 5 lines
Fixes 2ba05622b8b1 ("dmaengine: provide a common 'issue_pending_all' implementation")Suggested-by Sashiko <[email protected]>Signed-off-by Shivank Garg <[email protected]>---
Reviewed-by Frank Li <[email protected]>Quoted 1 line
Thank you for the review.
Best regards,
Shivank