drm/scheduler: Fix lockup in drm_sched_entity_kill()
Subsystems (3)
PATCH 1/1 drm/scheduler: Fix lockup in drm_sched_entity_kill()
The drm_sched_entity_kill() is invoked twice by drm_sched_entity_destroy() while userspace process is exiting or being killed. First time it's invoked when sched entity is flushed and second time when entity is released. This causes a lockup within wait_for_completion(entity_idle) due to how completion API works.
Calling wait_for_completion() more times than complete() was invoked is a error condition that causes lockup because completion internally uses counter for complete/wait calls. The complete_all() must be used instead in such cases.
This patch fixes lockup of Panfrost driver that is reproducible by killing any application in a middle of 3d drawing operation.
--- drivers/gpu/drm/scheduler/sched_entity.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)
drivers/gpu/drm/scheduler/sched_entity.c +1 -1
@@ -81,7 +81,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,init_completion (&entity->entity_idle);/* We start in an idle state. */ -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);spin_lock_init (&entity->rq_lock);spsc_queue_init (&entity->job_queue);
drivers/gpu/drm/scheduler/sched_main.c +2 -2
@@ -1026,7 +1026,7 @@ static int drm_sched_main(void *param) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) { -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);continue ; }
@@ -1037,7 +1037,7 @@ static int drm_sched_main(void *param)trace_drm_run_job (sched_job, entity); fence = sched->ops->run_job (sched_job); -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);drm_sched_fence_scheduled (s_fence);if (!IS_ERR_OR_NULL (fence)) {
Signature · 1 line
2.38.1
Am 23.11.22 um 01:13 schrieb Dmitry Osipenko:
Quoted 11 lines
The
drm_sched_entity_kill()is invoked twice bydrm_sched_entity_destroy()while userspace process is exiting or being killed. First time it's invoked when sched entity is flushed and second time when entity is released. This causes a lockup withinwait_for_completion(entity_idle)due to how completion API works.Calling
wait_for_completion()more times thancomplete()was invoked is a error condition that causes lockup because completion internally uses counter for complete/wait calls. Thecomplete_all()must be used instead in such cases.
Quoted 5 lines
This patch fixes lockup of Panfrost driver that is reproducible by killing any application in a middle of 3d drawing operation.
Fixes 2fdb8a8f07c2 ("drm/scheduler: rework entity flush, kill and fini")Signed-off-by Dmitry Osipenko <[email protected]>
Oh, good point. Reviewed-by: Christian König <[email protected]>
Quoted 40 lines
--- drivers/gpu/drm/scheduler/sched_entity.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_entity.c +1
-1@@ -81,7 +81,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,init_completion (&entity->entity_idle);/* We start in an idle state. */ -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);spin_lock_init (&entity->rq_lock);spsc_queue_init (&entity->job_queue);drivers/gpu/drm/scheduler/sched_main.c +2
-2@@ -1026,7 +1026,7 @@ static int drm_sched_main(void *param) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) { -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);continue ; }@@ -1037,7 +1037,7 @@ static int drm_sched_main(void *param)trace_drm_run_job (sched_job, entity); fence = sched->ops->run_job (sched_job); -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);drm_sched_fence_scheduled (s_fence);if (!IS_ERR_OR_NULL (fence)) {
Dmitry Osipenko
AppliedQuoted 48 lines · Dmitry Osipenko
The
drm_sched_entity_kill()is invoked twice bydrm_sched_entity_destroy()while userspace process is exiting or being killed. First time it's invoked when sched entity is flushed and second time when entity is released. This causes a lockup withinwait_for_completion(entity_idle)due to how completion API works.Calling
wait_for_completion()more times thancomplete()was invoked is a error condition that causes lockup because completion internally uses counter for complete/wait calls. Thecomplete_all()must be used instead in such cases.This patch fixes lockup of Panfrost driver that is reproducible by killing any application in a middle of 3d drawing operation.
Fixes 2fdb8a8f07c2 ("drm/scheduler: rework entity flush, kill and fini")Signed-off-by Dmitry Osipenko <[email protected]>--- drivers/gpu/drm/scheduler/sched_entity.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_entity.c +1
-1@@ -81,7 +81,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,init_completion (&entity->entity_idle);/* We start in an idle state. */ -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);spin_lock_init (&entity->rq_lock);spsc_queue_init (&entity->job_queue);drivers/gpu/drm/scheduler/sched_main.c +1
-1@@ -1026,7 +1026,7 @@ static int drm_sched_main(void *param) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) { -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);continue ; }
drivers/gpu/drm/scheduler/sched_main.c static int drm_sched_main(void *param) · line 1043
Quoted 8 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-1@@ -1037,7 +1037,7 @@ static int drm_sched_main(void *param)trace_drm_run_job (sched_job, entity); fence = sched->ops->run_job (sched_job); -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);drm_sched_fence_scheduled (s_fence);if (!IS_ERR_OR_NULL (fence)) {
Applied to drm-misc-next
Signature · 2 lines
Best regards, Dmitry
Hi Dmitry / Christian, thanks for the fix!
(And thanks Melissa for pointing that, saving me lots of time in research heh)
Is this fix planned to be released on 6.2-rc cycle? I've just tested it on Steam Deck, and it resolved a lockup observed (since v6.2-rc1) - exactly the same thing mentioned in the commit message.
FWIW:
Cheers,
Guilherme
Hi,
Quoted 11 lines · Guilherme G. Piccoli
Hi Dmitry / Christian, thanks for the fix!
(And thanks Melissa for pointing that, saving me lots of time in research heh)
Is this fix planned to be released on 6.2-rc cycle? I've just tested it on Steam Deck, and it resolved a lockup observed (since v6.2-rc1) - exactly the same thing mentioned in the commit message.
FWIW:
I'll push the patch to misc-fixes as soon as it will be rebased on 6.2-rc. Thanks!
Best regards,
Dmitry
Quoted 7 lines · Dmitry Osipenko
[...]
I'll push the patch to misc-fixes as soon as it will be rebased on 6.2-rc. Thanks!Best regards,
Dmitry
Thank you Dmitry, much appreciated!
Cheers,
Guilherme
Dmitry Osipenko
AppliedQuoted 48 lines · Dmitry Osipenko
The
drm_sched_entity_kill()is invoked twice bydrm_sched_entity_destroy()while userspace process is exiting or being killed. First time it's invoked when sched entity is flushed and second time when entity is released. This causes a lockup withinwait_for_completion(entity_idle)due to how completion API works.Calling
wait_for_completion()more times thancomplete()was invoked is a error condition that causes lockup because completion internally uses counter for complete/wait calls. Thecomplete_all()must be used instead in such cases.This patch fixes lockup of Panfrost driver that is reproducible by killing any application in a middle of 3d drawing operation.
Fixes 2fdb8a8f07c2 ("drm/scheduler: rework entity flush, kill and fini")Signed-off-by Dmitry Osipenko <[email protected]>--- drivers/gpu/drm/scheduler/sched_entity.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 4 ++-- 2 files changed, 3 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_entity.c +1
-1@@ -81,7 +81,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,init_completion (&entity->entity_idle);/* We start in an idle state. */ -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);spin_lock_init (&entity->rq_lock);spsc_queue_init (&entity->job_queue);drivers/gpu/drm/scheduler/sched_main.c +1
-1@@ -1026,7 +1026,7 @@ static int drm_sched_main(void *param) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) { -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);continue ; }
drivers/gpu/drm/scheduler/sched_main.c static int drm_sched_main(void *param) · line 1043
Quoted 8 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-1@@ -1037,7 +1037,7 @@ static int drm_sched_main(void *param)trace_drm_run_job (sched_job, entity); fence = sched->ops->run_job (sched_job); -complete (&entity->entity_idle); +complete_all (&entity->entity_idle);drm_sched_fence_scheduled (s_fence);if (!IS_ERR_OR_NULL (fence)) {
Applied to drm-misc-next-fixes
Signature · 2 lines
Best regards, Dmitry