drm/sched: Document race condition in drm_sched_fini()
Subsystems (3)
Philipp Stanner
maintainerPATCH v2 1/1 drm/sched: Document race condition in drm_sched_fini()
In drm_sched_fini() all entities are marked as stopped - without taking the appropriate lock, because that would deadlock. That means that drm_sched_fini() and drm_sched_entity_push_job() can race against each other.
This should most likely be fixed by establishing the rule that all entities associated with a scheduler must be torn down first. Then, however, the locking should be removed from drm_sched_fini() alltogether with an appropriate comment.
---
Changes in v2:
- Fix typo.
--- drivers/gpu/drm/scheduler/sched_main.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
drivers/gpu/drm/scheduler/sched_main.c +16 -0
@@ -1424,6 +1424,22 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)* Prevents reinsertion and marks job_queue as idle, * it will be removed from the rq in drm_sched_entity_fini() * eventually +* +* FIXME: +* This lacks the proper spin_lock(&s_entity->lock) and +* is, therefore, a race condition. Most notably, it +* can race with drm_sched_entity_push_job(). The lock +* cannot be taken here, however, because this would +* lead to lock inversion -> deadlock. +* +* The best solution probably is to enforce the life +* time rule of all entities having to be torn down +* before their scheduler. Then, however, locking could +* be dropped alltogether from this function. +* +* For now, this remains a potential race in all +* drivers that keep entities alive for longer than +* the scheduler. */ s_entity->stopped = true;spin_unlock (&rq->lock);
Signature · 1 line
2.49.0
Quoted 5 lines · Philipp Stanner
In
drm_sched_fini()all entities are marked as stopped - without taking the appropriate lock, because that would deadlock. That means thatdrm_sched_fini()anddrm_sched_entity_push_job()can race against each other.
Quoted 8 lines
This should most likely be fixed by establishing the rule that all entities associated with a scheduler must be torn down first. Then, however, the locking should be removed from
drm_sched_fini()alltogether with an appropriate comment.
Quoted 34 lines
---
Changes in v2:
- Fix typo.--- drivers/gpu/drm/scheduler/sched_main.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)drivers/gpu/drm/scheduler/sched_main.c +16
-0@@ -1424,6 +1424,22 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)* Prevents reinsertion and marks job_queue as idle, * it will be removed from the rq in drm_sched_entity_fini() * eventually +* +* FIXME: +* This lacks the proper spin_lock(&s_entity->lock) and +* is, therefore, a race condition. Most notably, it +* can race with drm_sched_entity_push_job(). The lock +* cannot be taken here, however, because this would +* lead to lock inversion -> deadlock. +* +* The best solution probably is to enforce the life +* time rule of all entities having to be torn down +* before their scheduler. Then, however, locking could +* be dropped alltogether from this function. +* +* For now, this remains a potential race in all +* drivers that keep entities alive for longer than +* the scheduler. */ s_entity->stopped = true;spin_unlock (&rq->lock);
Danilo Krummrich
maintainerQuoted 30 lines · Philipp Stanner
In
drm_sched_fini()all entities are marked as stopped - without taking the appropriate lock, because that would deadlock. That means thatdrm_sched_fini()anddrm_sched_entity_push_job()can race against each other.This should most likely be fixed by establishing the rule that all entities associated with a scheduler must be torn down first. Then, however, the locking should be removed from
drm_sched_fini()alltogether with an appropriate comment.Reported-by James Flowers <[email protected]>Signed-off-by Philipp Stanner <[email protected]>---
Changes in v2:
- Fix typo.--- drivers/gpu/drm/scheduler/sched_main.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)drivers/gpu/drm/scheduler/sched_main.c +2
-0@@ -1424,6 +1424,22 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)* Prevents reinsertion and marks job_queue as idle, * it will be removed from the rq in drm_sched_entity_fini() * eventually +* +* FIXME:
drivers/gpu/drm/scheduler/sched_main.c void drm_sched_fini(struct drm_gpu_scheduler *sched) · line 1438
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +10
-0+* This lacks the proper spin_lock(&s_entity->lock) and +* is, therefore, a race condition. Most notably, it +* can race with drm_sched_entity_push_job(). The lock +* cannot be taken here, however, because this would +* lead to lock inversion -> deadlock. +* +* The best solution probably is to enforce the life +* time rule of all entities having to be torn down +* before their scheduler. Then, however, locking could +* be dropped alltogether from this function.
"Enforce the rule" is correct, since factually it's there, as a dependency in the code.
Do we know which drivers violate this lifetime rule?
@Christian: What about amdgpu (for which the below was added to begin with)?
Quoted 8 lines
drivers/gpu/drm/scheduler/sched_main.c +3
-0+* For now, this remains a potential race in all +* drivers that keep entities alive for longer than +* the scheduler. */ s_entity->stopped = true;spin_unlock (&rq->lock);Signature · 1 line
2.49.0
Philipp Stanner
AppliedQuoted 6 lines · Danilo Krummrich
Quoted 5 lines · Philipp Stanner
In
drm_sched_fini()all entities are marked as stopped - without taking the appropriate lock, because that would deadlock. That means thatdrm_sched_fini()anddrm_sched_entity_push_job()can race against each other.
Quoted 8 lines
Quoted 8 lines
This should most likely be fixed by establishing the rule that all entities associated with a scheduler must be torn down first. Then, however, the locking should be removed from
drm_sched_fini()alltogether with an appropriate comment.
Applied to drm-misc-next
Quoted 31 lines
Quoted 17 lines
---
Changes in v2:
- Fix typo.
---
drivers/gpu/drm/scheduler/sched_main.c | 16 ++++++++++++++++1 file changed, 16 insertions(+)drivers/gpu/drm/scheduler/sched_main.c +2
-0@@ -1424,6 +1424,22 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched) * Prevents reinsertion and marks job_queue as idle, * it will be removed from the rq in drm_sched_entity_fini() * eventually + * + * FIXME:drivers/gpu/drm/scheduler/sched_main.c void drm_sched_fini(struct drm_gpu_scheduler *sched) · line 1438
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +10
-0+* This lacks the proper spin_lock(&s_entity->lock) and +* is, therefore, a race condition. Most notably, it +* can race with drm_sched_entity_push_job(). The lock +* cannot be taken here, however, because this would +* lead to lock inversion -> deadlock. +* +* The best solution probably is to enforce the life +* time rule of all entities having to be torn down +* before their scheduler. Then, however, locking could +* be dropped alltogether from this function. "Enforce the rule" is correct, since factually it's there, as a dependency in the code.
Quoted 1 line
Do we know which drivers violate this lifetime rule?
I've got no idea :(
Quoted 2 lines
@Christian: What about amdgpu (for which the below was added to begin with)?
+1
P.
Quoted 10 lines
Quoted 9 lines
drivers/gpu/drm/scheduler/sched_main.c +3
-0+* For now, this remains a potential race in all +* drivers that keep entities alive for longer than +* the scheduler. */ s_entity->stopped = true; spin_unlock(&rq->lock);Signature · 1 line
2.49.0