drm/panfrost: Handle resetting on timeout better
Subsystems (3)
Steven Price
maintainerPATCH v2 1/2 drm/panfrost: Handle resetting on timeout better
Panfrost uses multiple schedulers (one for each slot, so 2 in reality), and on a timeout has to stop all the schedulers to safely perform a reset. However more than one scheduler can trigger a timeout at the same time. This race condition results in jobs being freed while they are still in use.
When stopping other slots use cancel_delayed_work_sync() to ensure that any timeout started for that slot has completed. Also use mutex_trylock() to obtain reset_lock. This means that only one thread attempts the reset, the other threads will simply complete without doing anything (the first thread will wait for this in the call to cancel_delayed_work_sync()).
While we're here and since the function is already dependent on sched_job not being NULL, let's remove the unnecessary checks.
---
v2:
* Added fixes and tested-by tags
* Moved cleanup of panfrost_core_dump() comment to separate patch
drivers/gpu/drm/panfrost/panfrost_job.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
drivers/gpu/drm/panfrost/panfrost_job.c +11 -5
@@ -381,13 +381,19 @@ static void panfrost_job_timedout(struct drm_sched_job *sched_job)job_read (pfdev,JS_TAIL_LO (js)), sched_job); -mutex_lock (&pfdev->reset_lock); +if (!mutex_trylock (&pfdev->reset_lock)) +return ; -for (i =0 ; i <NUM_JOB_SLOTS ; i++) -drm_sched_stop (&pfdev->js->queue[i].sched, sched_job); +for (i =0 ; i <NUM_JOB_SLOTS ; i++) { +struct drm_gpu_scheduler *sched = &pfdev->js->queue[i].sched; + +drm_sched_stop (sched, sched_job); +if (js != i) +/* Ensure any timeouts on other slots have finished */ +cancel_delayed_work_sync (&sched->work_tdr); + } -if (sched_job) -drm_sched_increase_karma (sched_job); +drm_sched_increase_karma (sched_job);spin_lock_irqsave (&pfdev->js->job_lock, flags);for (i =0 ; i <NUM_JOB_SLOTS ; i++) {
Signature · 1 line
2.20.1
Steven Price
maintainerPATCH v2 2/2 drm/panfrost: Remove commented out call to panfrost_core_dump
panfrost_core_dump() has never existed in mainline, so remove it and add a TODO entry that core dump support is currently lacking.
---
v2:
* New patch
drivers/gpu/drm/panfrost/TODO | 2 ++ drivers/gpu/drm/panfrost/panfrost_job.c | 2 -- 2 files changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/panfrost/TODO +2 -0
@@ -10,3 +10,5 @@ - Compute job support. So called 'compute only' jobs need to be plumbed up to userspace. + +- Support core dump on job failure
drivers/gpu/drm/panfrost/panfrost_job.c +0 -2
@@ -404,8 +404,6 @@ static void panfrost_job_timedout(struct drm_sched_job *sched_job) }spin_unlock_irqrestore (&pfdev->js->job_lock, flags); -/* panfrost_core_dump(pfdev); */ -panfrost_devfreq_record_transition (pfdev, js);panfrost_device_reset (pfdev);
Signature · 1 line
2.20.1
Rob Herring
maintainerQuoted 20 lines · Steven Price
Panfrost uses multiple schedulers (one for each slot, so 2 in reality), and on a timeout has to stop all the schedulers to safely perform a reset. However more than one scheduler can trigger a timeout at the same time. This race condition results in jobs being freed while they are still in use.
When stopping other slots use
cancel_delayed_work_sync()to ensure that any timeout started for that slot has completed. Also usemutex_trylock()to obtainreset_lock. This means that only one thread attempts the reset, the other threads will simply complete without doing anything (the first thread will wait for this in the call tocancel_delayed_work_sync()).While we're here and since the function is already dependent on
sched_jobnot being NULL, let's remove the unnecessary checks.Fixes aa20236784ab ("drm/panfrost: Prevent concurrent resets")Tested-by Neil Armstrong <[email protected]>Signed-off-by Steven Price <[email protected]>
Quoted 7 lines
---
v2:
* Added fixes and tested-by tags
* Moved cleanup ofpanfrost_core_dump()comment to separate patchdrivers/gpu/drm/panfrost/panfrost_job.c | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
Both patches applied.
Rob