drm/amdgpu/userq: fix a leaked fence driver and an unlocked doorbell walk
Subsystems (4)
cover drm/amdgpu/userq: fix a leaked fence driver and an unlocked doorbell walk
Patch 1 cancels the fences still linked on a queue's fence driver when the queue goes away. Each of them holds a reference on the driver, so its refcount never reaches zero and the seq64 slot stays allocated for the lifetime of the device.
Patch 2 holds xa_lock_irqsave() across the doorbell xarray walk in mes_userq_detect_and_reset(). That xarray is device wide, so the walk dereferences queues owned by other drm_files, and nothing keeps them alive for its duration.
---
Junrui Luo (2):
drm/amdgpu/userq: cancel linked fences on fence driver free drm/amdgpu/userq: hold the doorbell xa lock during hang reset
drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 71 ++++++++++++++++--------- drivers/gpu/drm/amd/amdgpu/mes_userqueue.c | 13 ++++- 2 files changed, 57 insertions(+), 27 deletions(-)
---
base-commit: f5bbbfec59b4e2fb7520a91de3df8a6174325d6a
change-id: 20260816-amdgpu-fixes-18d9598a85ca
Best regards,
Signature · 1 line
Junrui Luo <[email protected]>
PATCH 1/2 drm/amdgpu/userq: cancel linked fences on fence driver free
amdgpu_userq_fence_driver_free() drops the queue's reference to fence_drv but leaves fence_drv->fences alone. Each fence still linked there holds a fence_drv reference of its own, and the list holds a reference on the fence, so the count never drops to zero and amdgpu_userq_fence_driver_destroy() never runs.
A fence stays linked when it has not signaled by the time the queue goes away. amdgpu_userq_fence_init() uses the wptr read by amdgpu_userq_fence_read_wptr() from the user mapped wptr buffer as the fence seqno without requiring it to increase, so a signal with a wptr below the previous one leaves the earlier fence unsignaled on the list while userq->last_fence points at the new, already signaled one. amdgpu_userq_destroy() waits only on last_fence and returns at once. This leaks the seq64 slot that amdgpu_seq64_free() would release, for the lifetime of the device.
Cancel the fences that are still linked before the queue drops its reference, in a helper shared with amdgpu_userq_fence_driver_destroy(). Like amdgpu_userq_fence_driver_process(), the helper drains the list under fence_list_lock and releases each fence outside it, so dropping the fence's fence_drv_array does not recurse into the lock.
---
Found by code inspection; not tested on hardware.
--- drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c | 71 ++++++++++++++++--------- 1 file changed, 46 insertions(+), 25 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_userq_fence.c +46 -25
@@ -114,6 +114,45 @@ static void amdgpu_userq_walk_and_drop_fence_drv(struct xarray *xa)xa_unlock (xa); } +static void +amdgpu_userq_fence_put_fence_drv_array (struct amdgpu_userq_fence *userq_fence) +{ +unsigned long i; + +for (i =0 ; i < userq_fence->fence_drv_array_count; i++) +amdgpu_userq_fence_driver_put (userq_fence->fence_drv_array[i]); + userq_fence->fence_drv_array_count =0 ; +} + +static void +amdgpu_userq_fence_driver_cancel (struct amdgpu_userq_fence_driver *fence_drv) +{ +struct amdgpu_userq_fence *userq_fence, *tmp; +LIST_HEAD (to_be_cancelled); +struct dma_fence *fence; +unsigned long flags; + +spin_lock_irqsave (&fence_drv->fence_list_lock, flags); +list_splice_init (&fence_drv->fences, &to_be_cancelled); +spin_unlock_irqrestore (&fence_drv->fence_list_lock, flags); + +list_for_each_entry_safe (userq_fence, tmp, &to_be_cancelled, link) { + fence = &userq_fence->base; +list_del_init (&userq_fence->link); + +if (!dma_fence_is_signaled (fence)) { +dma_fence_set_error (fence, -ECANCELED ); +dma_fence_signal (fence); + } + +/* Drop fence_drv_array outside fence_list_lock +* to avoid the recursion lock. +*/ +amdgpu_userq_fence_put_fence_drv_array (userq_fence); +dma_fence_put (fence); + } +} +void amdgpu_userq_fence_driver_free (struct amdgpu_usermode_queue *userq) {
@@ -122,19 +161,16 @@ amdgpu_userq_fence_driver_free(struct amdgpu_usermode_queue *userq)amdgpu_userq_walk_and_drop_fence_drv (&userq->fence_drv_xa);xa_destroy (&userq->fence_drv_xa);mutex_destroy (&userq->fence_drv_lock); +/* +* Cancel the fences still linked on the driver. Each of them holds a +* fence_drv reference of its own, so leaving them behind keeps the +* driver - and its seq64 slot - allocated after the queue is gone. +*/ +amdgpu_userq_fence_driver_cancel (userq->fence_drv);/* Drop the queue's ownership reference to fence_drv explicitly */ amdgpu_userq_fence_driver_put (userq->fence_drv); } -static void -amdgpu_userq_fence_put_fence_drv_array (struct amdgpu_userq_fence *userq_fence) -{ -unsigned long i; -for (i =0 ; i < userq_fence->fence_drv_array_count; i++) -amdgpu_userq_fence_driver_put (userq_fence->fence_drv_array[i]); - userq_fence->fence_drv_array_count =0 ; -} -/* * Returns: * -ENOENT when no fences were processes
@@ -186,23 +222,8 @@ void amdgpu_userq_fence_driver_destroy(struct kref *ref) struct amdgpu_userq_fence_driver, refcount);struct amdgpu_device *adev = fence_drv->adev; -struct amdgpu_userq_fence *fence, *tmp; -unsigned long flags; -struct dma_fence *f; -spin_lock_irqsave (&fence_drv->fence_list_lock, flags); -list_for_each_entry_safe (fence, tmp, &fence_drv->fences, link) { - f = &fence->base; - -if (!dma_fence_is_signaled (f)) { -dma_fence_set_error (f, -ECANCELED ); -dma_fence_signal (f); - } - -list_del (&fence->link); -dma_fence_put (f); - } -spin_unlock_irqrestore (&fence_drv->fence_list_lock, flags); +amdgpu_userq_fence_driver_cancel (fence_drv);/* Free seq64 memory */ amdgpu_seq64_free (adev, fence_drv->va);
Signature · 1 line
2.51.2
PATCH 2/2 drm/amdgpu/userq: hold the doorbell xa lock during hang reset
In reply to Junrui Luomes_userq_detect_and_reset() walks adev->userq_doorbell_xa with a bare xa_for_each() and dereferences every entry: it reads queue->queue_type and queue->doorbell_index, writes queue->state, and passes the queue to amdgpu_userq_fence_driver_force_completion(). That xarray is device wide, so most entries belong to other drm_files.
Nothing keeps those queues alive for the walk. The caller, amdgpu_userq_mgr_reset_work(), holds no lock, and amdgpu_mes_lock() is dropped before the walk begins. Meanwhile amdgpu_userq_destroy() erases the doorbell entry via amdgpu_userq_cleanup() and kfree()s the queue after dropping its own uq_mgr->userq_mutex; that per-file mutex cannot cover another file's queue. xa_for_each() releases its internal RCU read lock before returning each entry, so the pointer can already be dangling when the loop body touches it.
Fix by holding xa_lock_irqsave() across the walk, as amdgpu_userq_process_fence_irq() and amdgpu_userq_mgr_cancel_reset_work() already do.
---
Found by code inspection; not tested on hardware.
--- drivers/gpu/drm/amd/amdgpu/mes_userqueue.c | 13 +++++++++++-- 1 file changed, 11 insertions(+), 2 deletions(-)
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +11 -2
@@ -208,7 +208,7 @@ static int mes_userq_detect_and_reset(struct amdgpu_device *adev,struct mes_detect_and_reset_queue_input input;struct amdgpu_usermode_queue *queue;unsigned int hung_db_num =0 ; -unsigned long queue_id; +unsigned long queue_id, flags;u32 db_array[8 ];bool found_hung_queue = false;int r, i;
@@ -230,6 +230,13 @@ static int mes_userq_detect_and_reset(struct amdgpu_device *adev,if (r) {dev_err (adev->dev,"Failed to detect and reset queues, err (%d)\n" , r); }else if (hung_db_num) { +/* +* The doorbell xarray is device wide, so this walks queues +* owned by other drm_files too. Hold its lock: the free path +* erases the entry under the same lock strictly before it +* frees the queue, so an entry found here stays allocated. +*/ +xa_lock_irqsave (&adev->userq_doorbell_xa, flags);xa_for_each (&adev->userq_doorbell_xa, queue_id, queue) {if (queue->queue_type == queue_type) {for (i =0 ; i < hung_db_num; i++) {
@@ -238,14 +245,16 @@ static int mes_userq_detect_and_reset(struct amdgpu_device *adev, found_hung_queue = true;atomic_inc (&adev->gpu_reset_counter);amdgpu_userq_fence_driver_force_completion (queue); -drm_dev_wedged_event (adev_to_drm (adev),DRM_WEDGE_RECOVERY_NONE ,NULL ); } } } } +xa_unlock_irqrestore (&adev->userq_doorbell_xa, flags); }if (found_hung_queue) { +drm_dev_wedged_event (adev_to_drm (adev),DRM_WEDGE_RECOVERY_NONE ,NULL ); +/* Resume scheduling after hang recovery */ r =amdgpu_mes_resume (adev, input.xcc_id); }
Signature · 1 line
2.51.2
[email protected]
AI reviewRe: [PATCH 2/2] drm/amdgpu/userq: hold the doorbell xa lock during hang reset
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
Pre-existing issues:
- [High] Unprotected xa_for_each iteration allows concurrent Use-After-Free.
- [High] Lockless and unprotected access to userq->last_fence can result in a Use-After-Free.
- [Medium] Unprotected state modification of queue->state causes a data race. --
--- Patch [2]: [PATCH 2/2] drm/amdgpu/userq: hold the doorbell xa lock during hang reset --- commit c3c720e398b80f11731a92d4798c53c135bb4886
Author: Junrui Luo <[email protected]>
drm/amdgpu/userq: hold the doorbell xa lock during hang reset
This commit fixes a use-after-free vulnerability in the AMDGPU driver. It adds an xa_lock_irqsave() lock around an xa_for_each() loop in mes_userq_detect_and_reset() to prevent the queue from being freed while iterating.
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +0
-0
[ ... ]
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c static int mes_userq_detect_and_reset(struct amdgpu_device *adev, · line 240
Quoted 12 lines
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +7
-0@@ -230,6 +230,13 @@ static int mes_userq_detect_and_reset(struct amdgpu_device *adev,if (r) {dev_err (adev->dev,"Failed to detect and reset queues, err (%d)\n" , r); }else if (hung_db_num) { +/* +* The doorbell xarray is device wide, so this walks queues +* owned by other drm_files too. Hold its lock: the free path +* erases the entry under the same lock strictly before it +* frees the queue, so an entry found here stays allocated. +*/ +xa_lock_irqsave (&adev->userq_doorbell_xa, flags);xa_for_each (&adev->userq_doorbell_xa, queue_id, queue) {
[Severity: High]
This isn't a bug introduced by this patch, but since you are fixing the xa_for_each() iteration here, I noticed several other functions iterate adev->userq_doorbell_xa locklessly.
For example, in amdgpu_userq_stop_sched_for_enforce_isolation():
xa_for_each(&adev->userq_doorbell_xa, queue_id, queue) {
uqm = queue->userq_mgr;
cancel_delayed_work_sync(&uqm->resume_work);
mutex_lock(&uqm->userq_mutex);Similar loops exist in amdgpu_userq_start_sched_for_enforce_isolation() and amdgpu_userq_suspend().
The xa_for_each macro only uses rcu_read_lock() internally during the lookup.
Could an unprivileged user trigger amdgpu_userq_destroy() concurrently and free the queue, causing a use-after-free when the loop body dereferences queue->userq_mgr?
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +0
-0if (queue->queue_type == queue_type) {for (i =0 ; i < hung_db_num; i++) {drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +0
-0if (queue->doorbell_index == db_array[i]) { queue->state =AMDGPU_USERQ_STATE_HUNG ;
[Severity: Medium]
This is a pre-existing issue, but does assigning queue->state here without holding queue->userq_mgr->userq_mutex cause a data race?
Concurrently, actions like amdgpu_userq_start_sched_for_enforce_isolation() or PM resume can call amdgpu_userq_map_helper() under userq_mutex, which also evaluates and sets queue->state.
Could this lead to lost updates where the AMDGPU_USERQ_STATE_HUNG state is overwritten?
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c static int mes_userq_detect_and_reset(struct amdgpu_device *adev, · line 247
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +0
-0found_hung_queue = true;atomic_inc (&adev->gpu_reset_counter);amdgpu_userq_fence_driver_force_completion (queue);
[Severity: High]
This is a pre-existing issue, but is it safe to call amdgpu_userq_fence_driver_force_completion() here?
This function reads userq->last_fence without taking queue->userq_mgr->userq_mutex or holding rcu_read_lock():
amdgpu_userq_fence_driver_force_completion()
struct dma_fence *f = userq->last_fence;
if (f) {
struct amdgpu_userq_fence *fence = to_amdgpu_userq_fence(f);Concurrently, amdgpu_userq_fence_init() (called under userq_mutex) updates userq->last_fence and drops the old fence reference via dma_fence_put().
Since xa_lock_irqsave() maps to a sleeping lock on PREEMPT_RT kernels and does not disable interrupts, the RCU grace period could end.
Could this allow the fence memory to be freed before to_amdgpu_userq_fence(f) accesses f->ops, resulting in a use-after-free?
Quoted 4 lines
drivers/gpu/drm/amd/amdgpu/mes_userqueue.c +0
-1-drm_dev_wedged_event (adev_to_drm (adev),DRM_WEDGE_RECOVERY_NONE ,NULL ); } } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [PATCH 2/2] drm/amdgpu/userq: hold the doorbell xa lock during hang reset
In reply to Junrui LuoHi Christian, Alex,
Sorry for the ping. No need to look at the patch itself.
I would just be grateful for a quick word on whether the issue it describes is a real one. If not, I will drop it; if it is, should I send a v2?
Thanks for your time,
Junrui Luo
Christian König
maintainerRe: [PATCH 2/2] drm/amdgpu/userq: hold the doorbell xa lock during hang reset
Hi,
Quoted 6 lines · Junrui Luo
Hi Christian, Alex,
Sorry for the ping. No need to look at the patch itself.
I would just be grateful for a quick word on whether the issue it describes is a real one. If not, I will drop it; if it is, should I send a v2?
as far as I can see it is completely nonsense what you try to do here.
The fence_drv references an amdgpu_userq object holds are the ones which the queue potentially waits on.
When the queue is freed up those references must be dropped, but that shouldn't affect fence_drv->fences in any way possible.
What exactly is the leak you are seeing?
Regards,
Christian.
Quoted 3 lines
Thanks for your time,
Junrui Luo
Re: [PATCH 2/2] drm/amdgpu/userq: hold the doorbell xa lock during hang reset
Hi Christian,
Thanks, and sorry, my commit message was inaccurate. Dropping the queue's fence_drv reference is unrelated to fence_drv->fences, as you say.
amdgpu_userq_signal_ioctl() takes the wptr from the user mapped wptr BO and hands it to dma_fence_init64() as the seqno, which is documented as "a linear increasing sequence number for this context". Nothing on that path checks that it increases.
fence_drv->fences is filled with list_add_tail() and drained by amdgpu_userq_fence_driver_process() as a sorted prefix cut, so a wptr that moves backwards leaves a fence sitting at the head of the list blocking the drain. The hang detect path does not recover it either. force_completion() takes the forced rptr from userq->last_fence->seqno, which the same regression has already moved backwards. Destroying the queue then makes it permanent:amdgpu_userq_fence_driver_process() is only reachable through a live queue.
So is a userspace that moves its own wptr backwards in scope here? I can send a v2 that rejects a non-advancing wptr instead of touching the fence list.
Thanks,
Junrui Luo