DRM scheduler changes for Xe
Subsystems (10)
Matthew Brost
maintainercover DRM scheduler changes for Xe
As a prerequisite to merging the new Intel Xe DRM driver [1] [2], we have been asked to merge our common DRM scheduler patches first.
This a continuation of a RFC [3] with all comments addressed, ready for a full review, and hopefully in state which can merged in the near future. More details of this series can found in the cover letter of the RFC [3].
These changes have been tested with the Xe driver.
Matt
[1] https://gitlab.freedesktop.org/drm/xe/kernel
[2] https://patchwork.freedesktop.org/series/112188/
[3] https://patchwork.freedesktop.org/series/116055/
Matthew Brost (8):
drm/sched: Convert drm scheduler to use a work queue rather than
kthread
drm/sched: Move schedule policy to scheduler / entity drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy drm/sched: Add generic scheduler message interface drm/sched: Add drm_sched_start_timeout_unlocked helper drm/sched: Start run wq before TDR in drm_sched_start drm/sched: Submit job before starting TDR
drm/sched: Add helper to set TDR timeout
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 14 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 15 +- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 5 +- drivers/gpu/drm/lima/lima_sched.c | 5 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 6 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 5 +- drivers/gpu/drm/panfrost/panfrost_job.c | 5 +- drivers/gpu/drm/scheduler/sched_entity.c | 85 +++++- drivers/gpu/drm/scheduler/sched_fence.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 294 +++++++++++++++----- drivers/gpu/drm/v3d/v3d_sched.c | 25 +- include/drm/gpu_scheduler.h | 70 ++++- 12 files changed, 396 insertions(+), 135 deletions(-)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 1/8 drm/sched: Convert drm scheduler to use a work queue rather than kthread
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a drm_gpu_scheduler and drm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.
1. In XE the submission order from multiple drm_sched_entity is not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shared drm_gpu_scheduler across multiple drm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicated drm_gpu_scheduler per drm_sched_entity solve this problem.
2. In XE submissions are done via programming a ring buffer (circular buffer), a drm_gpu_scheduler provides a limit on number of jobs, if the limit of number jobs is set to RING_SIZE / MAX_SIZE_PER_JOB we get flow control on the ring for free.
A problem with this design is currently a drm_gpu_scheduler uses a kthread for submission / job cleanup. This doesn't scale if a large number of drm_gpu_scheduler are used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.
v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
--- drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 14 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 14 +- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +- drivers/gpu/drm/lima/lima_sched.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 6 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +- drivers/gpu/drm/panfrost/panfrost_job.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 136 +++++++++++--------- drivers/gpu/drm/v3d/v3d_sched.c | 10 +- include/drm/gpu_scheduler.h | 14 +- 10 files changed, 113 insertions(+), 89 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +7 -7
@@ -1489,9 +1489,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)for (i =0 ; i <AMDGPU_MAX_RINGS ; i++) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ; -kthread_park (ring->sched.thread); +drm_sched_run_wq_stop (&ring->sched); }seq_printf (m,"run ib test:\n" );
@@ -1505,9 +1505,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)for (i =0 ; i <AMDGPU_MAX_RINGS ; i++) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ; -kthread_unpark (ring->sched.thread); +drm_sched_run_wq_start (&ring->sched); }up_write (&adev->reset_domain->sem);
@@ -1727,7 +1727,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) ring = adev->rings[val]; -if (!ring || !ring->funcs->preempt_ib || !ring->sched.thread) +if (!ring || !ring->funcs->preempt_ib || !ring->sched.ready)return -EINVAL ;/* the last preemption failed */
@@ -1745,7 +1745,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) goto pro_end;/* stop the scheduler */ -kthread_park (ring->sched.thread); +drm_sched_run_wq_stop (&ring->sched);/* preempt the IB */ r =amdgpu_ring_preempt_ib (ring);
@@ -1779,7 +1779,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) failure:/* restart the scheduler */ -kthread_unpark (ring->sched.thread); +drm_sched_run_wq_start (&ring->sched);up_read (&adev->reset_domain->sem);
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +7 -7
@@ -2364,7 +2364,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev)break ; } - r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, + r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, NULL, ring->num_hw_submission, amdgpu_job_hang_limit, timeout, adev->reset_domain->wq, ring->sched_score, ring->name,
@@ -4627,7 +4627,7 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev)for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;spin_lock (&ring->sched.job_list_lock);
@@ -4753,7 +4753,7 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;/*clear job fence from fence drv to avoid force_completion
@@ -5294,7 +5294,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = tmp_adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_stop (&ring->sched, job ? &job->base :NULL );
@@ -5369,7 +5369,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = tmp_adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_start (&ring->sched, true);
@@ -5696,7 +5696,7 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_stafor (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_stop (&ring->sched,NULL );
@@ -5824,7 +5824,7 @@ void amdgpu_pci_resume(struct pci_dev *pdev)for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_start (&ring->sched, true);
drivers/gpu/drm/etnaviv/etnaviv_sched.c +1 -1
@@ -133,7 +133,7 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu) {int ret; - ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops, + ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops,NULL , etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,msecs_to_jiffies (500 ),NULL ,NULL ,dev_name (gpu->dev), gpu->dev);
drivers/gpu/drm/lima/lima_sched.c +1 -1
@@ -488,7 +488,7 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)INIT_WORK (&pipe->recover_work, lima_sched_recover_work); -return drm_sched_init (&pipe->base, &lima_sched_ops,1 , +return drm_sched_init (&pipe->base, &lima_sched_ops,NULL ,1 , lima_job_hang_limit,msecs_to_jiffies (timeout),NULL ,NULL , name, pipe->ldev->dev);
drivers/gpu/drm/msm/adreno/adreno_device.c +4 -2
@@ -662,7 +662,8 @@ static void suspend_scheduler(struct msm_gpu *gpu)*/ for (i =0 ; i < gpu->nr_rings; i++) {struct drm_gpu_scheduler *sched = &gpu->rb[i]->sched; -kthread_park (sched->thread); + +drm_sched_run_wq_stop (sched); } }
@@ -672,7 +673,8 @@ static void resume_scheduler(struct msm_gpu *gpu)for (i =0 ; i < gpu->nr_rings; i++) {struct drm_gpu_scheduler *sched = &gpu->rb[i]->sched; -kthread_unpark (sched->thread); + +drm_sched_run_wq_start (sched); } }
drivers/gpu/drm/msm/msm_ringbuffer.c +1 -1
@@ -95,7 +95,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,/* currently managing hangcheck ourselves: */ sched_timeout =MAX_SCHEDULE_TIMEOUT ; - ret =drm_sched_init (&ring->sched, &msm_sched_ops, + ret =drm_sched_init (&ring->sched, &msm_sched_ops,NULL , num_hw_submissions,0 , sched_timeout,NULL ,NULL ,to_msm_bo (ring->bo)->name, gpu->dev->dev);if (ret) {
drivers/gpu/drm/panfrost/panfrost_job.c +1 -1
@@ -815,7 +815,7 @@ int panfrost_job_init(struct panfrost_device *pfdev) js->queue[j].fence_context =dma_fence_context_alloc (1 ); ret = drm_sched_init(&js->queue[j].sched, - &panfrost_sched_ops, + &panfrost_sched_ops, NULL, nentries, 0, msecs_to_jiffies(JOB_TIMEOUT_MS), pfdev->reset.wq,
drivers/gpu/drm/scheduler/sched_main.c +77 -59
244 lines not on this page · raw message
drivers/gpu/drm/v3d/v3d_sched.c +5 -5
@@ -388,7 +388,7 @@ v3d_sched_init(struct v3d_dev *v3d)int ret; ret =drm_sched_init (&v3d->queue[V3D_BIN ].sched, - &v3d_bin_sched_ops, + &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_bin" , v3d->drm.dev);
@@ -396,7 +396,7 @@ v3d_sched_init(struct v3d_dev *v3d)return ret; ret =drm_sched_init (&v3d->queue[V3D_RENDER ].sched, - &v3d_render_sched_ops, + &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_render" , v3d->drm.dev);
@@ -404,7 +404,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_TFU ].sched, - &v3d_tfu_sched_ops, + &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_tfu" , v3d->drm.dev);
@@ -413,7 +413,7 @@ v3d_sched_init(struct v3d_dev *v3d)if (v3d_has_csd (v3d)) { ret =drm_sched_init (&v3d->queue[V3D_CSD ].sched, - &v3d_csd_sched_ops, + &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_csd" , v3d->drm.dev);
@@ -421,7 +421,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_CACHE_CLEAN ].sched, - &v3d_cache_clean_sched_ops, + &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_cache_clean" , v3d->drm.dev);
include/drm/gpu_scheduler.h +9 -5
@@ -473,17 +473,16 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. -* @wake_up_worker: the wait queue on which the scheduler sleeps until a job -* is ready to be scheduled. * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. +* @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr +* @work_run: schedules jobs and cleans up entities * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. -* @thread: the kthread on which the scheduler which run. * @pending_list: the list of jobs which are currently in the job queue. * @job_list_lock: lock to protect the pending_list. * @hang_limit: once the hangs by a job crosses this limit then it is marked
@@ -492,6 +491,7 @@ struct drm_sched_backend_ops {* @_score: score used when the driver doesn't provide one * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. +* @pause_run_wq: pause queuing of @work_run on @run_wq * @dev: system &struct device * * One scheduler is implemented for each hardware ring.
@@ -502,13 +502,13 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; -wait_queue_head_t wake_up_worker;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count; +struct workqueue_struct *run_wq;struct workqueue_struct *timeout_wq; +struct work_struct work_run;struct delayed_work work_tdr; -struct task_struct *thread;struct list_head pending_list;spinlock_t job_list_lock;int hang_limit;
@@ -516,11 +516,13 @@ struct drm_gpu_scheduler {atomic_t _score;bool ready;bool free_guilty; +bool pause_run_wq;struct device *dev; };int drm_sched_init (struct drm_gpu_scheduler *sched,const struct drm_sched_backend_ops *ops, +struct workqueue_struct *run_wq,uint32_t hw_submission,unsigned hang_limit,long timeout,struct workqueue_struct *timeout_wq,atomic_t *score,const char *name,struct device *dev);
@@ -550,6 +552,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched); +void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);void drm_sched_start (struct drm_gpu_scheduler *sched,bool full_recovery);void drm_sched_resubmit_jobs (struct drm_gpu_scheduler *sched);
Signature · 1 line
2.34.1
Re: [PATCH 1/8] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 47 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in workerSigned-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 14 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 14 +- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +- drivers/gpu/drm/lima/lima_sched.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 6 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +- drivers/gpu/drm/panfrost/panfrost_job.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 136 +++++++++++--------- drivers/gpu/drm/v3d/v3d_sched.c | 10 +- include/drm/gpu_scheduler.h | 14 +- 10 files changed, 113 insertions(+), 89 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +0
-0
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · line 1494
Quoted 8 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +2
-2@@ -1489,9 +1489,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)for (i =0 ; i <AMDGPU_MAX_RINGS ; i++) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ; -kthread_park (ring->sched.thread); +drm_sched_run_wq_stop (&ring->sched);
It would be good to split out adding of these wrappers (including adding one for ring->sched.thread/ready) to a standalong preceding patch. That way at least some mechanical changes to various drivers would be separated from functional changes.
Also, perhaps do not have the wq in the name if it is not really needed to be verbose with the underlying implementation like that? Like would drm_sched_run/pause. Or even __drm_sched_start/stop, dunno, just an idea.
Quoted 312 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +5
-5}seq_printf (m,"run ib test:\n" );@@ -1505,9 +1505,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)for (i =0 ; i <AMDGPU_MAX_RINGS ; i++) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ; -kthread_unpark (ring->sched.thread); +drm_sched_run_wq_start (&ring->sched); }up_write (&adev->reset_domain->sem);@@ -1727,7 +1727,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) ring = adev->rings[val]; -if (!ring || !ring->funcs->preempt_ib || !ring->sched.thread) +if (!ring || !ring->funcs->preempt_ib || !ring->sched.ready)return -EINVAL ;/* the last preemption failed */ @@ -1745,7 +1745,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) goto pro_end;/* stop the scheduler */ -kthread_park (ring->sched.thread); +drm_sched_run_wq_stop (&ring->sched);/* preempt the IB */ r =amdgpu_ring_preempt_ib (ring);@@ -1779,7 +1779,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) failure:/* restart the scheduler */ -kthread_unpark (ring->sched.thread); +drm_sched_run_wq_start (&ring->sched);up_read (&adev->reset_domain->sem);drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +7
-7@@ -2364,7 +2364,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev)break ; } - r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, + r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, NULL, ring->num_hw_submission, amdgpu_job_hang_limit, timeout, adev->reset_domain->wq, ring->sched_score, ring->name,@@ -4627,7 +4627,7 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev)for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;spin_lock (&ring->sched.job_list_lock);@@ -4753,7 +4753,7 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;/*clear job fence from fence drv to avoid force_completion @@ -5294,7 +5294,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = tmp_adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_stop (&ring->sched, job ? &job->base :NULL );@@ -5369,7 +5369,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = tmp_adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_start (&ring->sched, true);@@ -5696,7 +5696,7 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_stafor (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_stop (&ring->sched,NULL );@@ -5824,7 +5824,7 @@ void amdgpu_pci_resume(struct pci_dev *pdev)for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_start (&ring->sched, true);drivers/gpu/drm/etnaviv/etnaviv_sched.c +1
-1@@ -133,7 +133,7 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu) {int ret; - ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops, + ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops,NULL , etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,msecs_to_jiffies (500 ),NULL ,NULL ,dev_name (gpu->dev), gpu->dev);drivers/gpu/drm/lima/lima_sched.c +1
-1@@ -488,7 +488,7 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)INIT_WORK (&pipe->recover_work, lima_sched_recover_work); -return drm_sched_init (&pipe->base, &lima_sched_ops,1 , +return drm_sched_init (&pipe->base, &lima_sched_ops,NULL ,1 , lima_job_hang_limit,msecs_to_jiffies (timeout),NULL ,NULL , name, pipe->ldev->dev);drivers/gpu/drm/msm/adreno/adreno_device.c +4
-2@@ -662,7 +662,8 @@ static void suspend_scheduler(struct msm_gpu *gpu)*/ for (i =0 ; i < gpu->nr_rings; i++) {struct drm_gpu_scheduler *sched = &gpu->rb[i]->sched; -kthread_park (sched->thread); + +drm_sched_run_wq_stop (sched); } }@@ -672,7 +673,8 @@ static void resume_scheduler(struct msm_gpu *gpu)for (i =0 ; i < gpu->nr_rings; i++) {struct drm_gpu_scheduler *sched = &gpu->rb[i]->sched; -kthread_unpark (sched->thread); + +drm_sched_run_wq_start (sched); } }drivers/gpu/drm/msm/msm_ringbuffer.c +1
-1@@ -95,7 +95,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,/* currently managing hangcheck ourselves: */ sched_timeout =MAX_SCHEDULE_TIMEOUT ; - ret =drm_sched_init (&ring->sched, &msm_sched_ops, + ret =drm_sched_init (&ring->sched, &msm_sched_ops,NULL , num_hw_submissions,0 , sched_timeout,NULL ,NULL ,to_msm_bo (ring->bo)->name, gpu->dev->dev);if (ret) {drivers/gpu/drm/panfrost/panfrost_job.c +1
-1@@ -815,7 +815,7 @@ int panfrost_job_init(struct panfrost_device *pfdev) js->queue[j].fence_context =dma_fence_context_alloc (1 ); ret = drm_sched_init(&js->queue[j].sched, - &panfrost_sched_ops, + &panfrost_sched_ops, NULL, nentries, 0, msecs_to_jiffies(JOB_TIMEOUT_MS), pfdev->reset.wq,drivers/gpu/drm/scheduler/sched_main.c +46
-25@@ -44,7 +44,6 @@* The jobs in a entity are always scheduled in the order that they were pushed. */ -#include <linux/kthread.h> #include <linux/wait.h> #include <linux/sched.h> #include <linux/completion.h> @@ -252,6 +251,47 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) return rb ? rb_entry(rb, struct drm_sched_entity, rb_tree_node) : NULL; } +/** +* drm_sched_run_wq_stop - stop scheduler run worker +* +* @sched: scheduler instance to stop run worker +*/ +void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched) +{ +WRITE_ONCE (sched->pause_run_wq, true); +cancel_work_sync (&sched->work_run); +} +EXPORT_SYMBOL (drm_sched_run_wq_stop); + +/** +* drm_sched_run_wq_start - start scheduler run worker +* +* @sched: scheduler instance to start run worker +*/ +void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched) +{ +WRITE_ONCE (sched->pause_run_wq, false); +queue_work (sched->run_wq, &sched->work_run); +} +EXPORT_SYMBOL (drm_sched_run_wq_start); + +/** +* drm_sched_run_wq_queue - queue scheduler run worker +* +* @sched: scheduler instance to queue run worker +*/ +static void drm_sched_run_wq_queue (struct drm_gpu_scheduler *sched) +{ +/* +* Try not to schedule work if pause_run_wq set but not the end of world +* if we do as either it will be cancelled by the above +* cancel_work_sync, or drm_sched_main turns into a NOP while +* pause_run_wq is set. +*/ +if (!READ_ONCE (sched->pause_run_wq)) +queue_work (sched->run_wq, &sched->work_run); +} +/** * drm_sched_job_done - complete a job * @s_job: pointer to the job which is done @@ -271,7 +311,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence);dma_fence_put (&s_fence->finished); -wake_up_interruptible (&sched->wake_up_worker); +drm_sched_run_wq_queue (sched); }/** @@ -434,7 +474,7 @@ void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad) {struct drm_sched_job *s_job, *tmp; -kthread_park (sched->thread); +drm_sched_run_wq_stop (sched);/* * Reinsert back the bad job here - now it's safe as @@ -547,7 +587,7 @@ void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery)spin_unlock (&sched->job_list_lock); } -kthread_unpark (sched->thread); +drm_sched_run_wq_start (sched); }EXPORT_SYMBOL (drm_sched_start);@@ -864,7 +904,7 @@ static bool drm_sched_ready(struct drm_gpu_scheduler *sched)void drm_sched_wakeup (struct drm_gpu_scheduler *sched) {if (drm_sched_ready (sched)) -wake_up_interruptible (&sched->wake_up_worker); +drm_sched_run_wq_queue (sched); }/** @@ -974,61 +1014,42 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); -/** -* drm_sched_blocked - check if the scheduler is blocked -* -* @sched: scheduler instance -* -* Returns true if blocked, otherwise false. -*/ -static bool drm_sched_blocked (struct drm_gpu_scheduler *sched) -{ -if (kthread_should_park ()) { -kthread_parkme (); -return true; - } - -return false; -} -/** * drm_sched_main - main scheduler thread * * @param: scheduler instance -* -* Returns 0. */ -static int drm_sched_main (void *param) +static void drm_sched_main (struct work_struct *w) {
drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1031
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +6
-2-struct drm_gpu_scheduler *sched = (struct drm_gpu_scheduler *)param; +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run); +struct drm_sched_entity *entity; +struct drm_sched_job *cleanup_job;int r; -sched_set_fifo_low (current); +if (READ_ONCE (sched->pause_run_wq)) +return ;
Is there a point to this check given the comment in drm_sched_run_wq_queue?
Quoted 28 lines
drivers/gpu/drm/scheduler/sched_main.c +9
-15-while (!kthread_should_stop ()) { -struct drm_sched_entity *entity =NULL ; -struct drm_sched_fence *s_fence; -struct drm_sched_job *sched_job; -struct dma_fence *fence; -struct drm_sched_job *cleanup_job =NULL ; + cleanup_job =drm_sched_get_cleanup_job (sched); + entity =drm_sched_select_entity (sched); -wait_event_interruptible (sched->wake_up_worker, - (cleanup_job =drm_sched_get_cleanup_job (sched)) || - (!drm_sched_blocked (sched) && - (entity =drm_sched_select_entity (sched))) || -kthread_should_stop ()); +if (!entity && !cleanup_job) +return ;/* No more work */ -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (cleanup_job) + sched->ops->free_job (cleanup_job); -if (!entity) -continue ; +if (entity) { +struct dma_fence *fence; +struct drm_sched_fence *s_fence;
drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1052
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +4
-2+struct drm_sched_job *sched_job; sched_job =drm_sched_entity_pop_job (entity); -if (!sched_job) {complete_all (&entity->entity_idle); -continue ; +if (!cleanup_job) +return ;/* No more work */ + goto again;
Loop is gone but now it re-arms itself which is needed to avoid starvation? Is it guaranteed to be effective by the wq contract?
Quoted 36 lines
drivers/gpu/drm/scheduler/sched_main.c +7
-2} s_fence = sched_job->s_fence;@@ -1055,14 +1076,17 @@ static int drm_sched_main(void *param) r); }else {if (IS_ERR (fence)) -dma_fence_set_error (&s_fence->finished,PTR_ERR (fence)); +dma_fence_set_error (&s_fence->finished, +PTR_ERR (fence));drm_sched_job_done (sched_job); }wake_up (&sched->job_scheduled); } -return 0 ; + +again: +drm_sched_run_wq_queue (sched); }/** @@ -1070,6 +1094,7 @@ static int drm_sched_main(void *param)* * @sched: scheduler instance * @ops: backend operations for this scheduler +* @run_wq: workqueue to use for run work. If NULL, the system_wq is used * @hw_submission: number of hw submissions that can be in flight * @hang_limit: number of times to allow a job to hang before dropping it * @timeout: timeout value in jiffies for the scheduler @@ -1083,14 +1108,16 @@ static int drm_sched_main(void *param)*/ int drm_sched_init(struct drm_gpu_scheduler *sched, const struct drm_sched_backend_ops *ops, + struct workqueue_struct *run_wq,
drivers/gpu/drm/scheduler/sched_main.c static int drm_sched_main(void *param) · line 1120
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +2
-1unsigned hw_submission, unsigned hang_limit, long timeout, struct workqueue_struct *timeout_wq, atomic_t *score, const char *name, struct device *dev) { - int i, ret; + int i; sched->ops = ops; sched->hw_submission_limit = hw_submission; sched->name = name; + sched->run_wq = run_wq ? : system_wq;
I still think it is not nice to implicitly move everyone over to the shared system wq. Maybe even more so with now one at a time execution, since effect on latency can be even greater.
Have you considered kthread_work as a backend? Maybe it would work to have callers pass in a kthread_worker they create, or provide a drm_sched helper to create one, which would then be passed to drm_sched_init.
That would enable per driver kthread_worker, or per device, or whatever granularity each driver would want/need/desire.
driver init:
struct drm_sched_worker = drm_sched_create_worker(...);
queue/whatever init:drm_sched_init(.., worker, ...);
You could create one inside drm_sched_init if not passed in, which would keep the behaviour for existing drivers more similar - they would still have a 1:1 kthread context for their exclusive use.
And I *think* self-re-arming would be less problematic latency wise since kthread_worker consumes everything queued without relinquishing control and execution context would be guaranteed not to be shared with random system stuff.
Regards,
Tvrtko
Quoted 159 lines
drivers/gpu/drm/scheduler/sched_main.c +3
-12sched->timeout = timeout; sched->timeout_wq = timeout_wq ? : system_wq; sched->hang_limit = hang_limit;@@ -1099,23 +1126,15 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,for (i =DRM_SCHED_PRIORITY_MIN ; i <DRM_SCHED_PRIORITY_COUNT ; i++)drm_sched_rq_init (sched, &sched->sched_rq[i]); -init_waitqueue_head (&sched->wake_up_worker);init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); +INIT_WORK (&sched->work_run, drm_sched_main);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); - -/* Each scheduler will run on a seperate kernel thread */ - sched->thread =kthread_run (drm_sched_main, sched, sched->name); -if (IS_ERR (sched->thread)) { - ret =PTR_ERR (sched->thread); - sched->thread =NULL ; -DRM_DEV_ERROR (sched->dev,"Failed to create scheduler for %s.\n" , name); -return ret; - } + sched->pause_run_wq = false; sched->ready = true;return 0 ;@@ -1134,8 +1153,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)struct drm_sched_entity *s_entity;int i; -if (sched->thread) -kthread_stop (sched->thread); +drm_sched_run_wq_stop (sched);for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) {struct drm_sched_rq *rq = &sched->sched_rq[i];drivers/gpu/drm/v3d/v3d_sched.c +5
-5@@ -388,7 +388,7 @@ v3d_sched_init(struct v3d_dev *v3d)int ret; ret =drm_sched_init (&v3d->queue[V3D_BIN ].sched, - &v3d_bin_sched_ops, + &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_bin" , v3d->drm.dev);@@ -396,7 +396,7 @@ v3d_sched_init(struct v3d_dev *v3d)return ret; ret =drm_sched_init (&v3d->queue[V3D_RENDER ].sched, - &v3d_render_sched_ops, + &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_render" , v3d->drm.dev);@@ -404,7 +404,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_TFU ].sched, - &v3d_tfu_sched_ops, + &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_tfu" , v3d->drm.dev);@@ -413,7 +413,7 @@ v3d_sched_init(struct v3d_dev *v3d)if (v3d_has_csd (v3d)) { ret =drm_sched_init (&v3d->queue[V3D_CSD ].sched, - &v3d_csd_sched_ops, + &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_csd" , v3d->drm.dev);@@ -421,7 +421,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_CACHE_CLEAN ].sched, - &v3d_cache_clean_sched_ops, + &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_cache_clean" , v3d->drm.dev);include/drm/gpu_scheduler.h +9
-5@@ -473,17 +473,16 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. -* @wake_up_worker: the wait queue on which the scheduler sleeps until a job -* is ready to be scheduled. * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. +* @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr +* @work_run: schedules jobs and cleans up entities * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. -* @thread: the kthread on which the scheduler which run. * @pending_list: the list of jobs which are currently in the job queue. * @job_list_lock: lock to protect the pending_list. * @hang_limit: once the hangs by a job crosses this limit then it is marked @@ -492,6 +491,7 @@ struct drm_sched_backend_ops {* @_score: score used when the driver doesn't provide one * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. +* @pause_run_wq: pause queuing of @work_run on @run_wq * @dev: system &struct device * * One scheduler is implemented for each hardware ring. @@ -502,13 +502,13 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; -wait_queue_head_t wake_up_worker;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count; +struct workqueue_struct *run_wq;struct workqueue_struct *timeout_wq; +struct work_struct work_run;struct delayed_work work_tdr; -struct task_struct *thread;struct list_head pending_list;spinlock_t job_list_lock;int hang_limit;@@ -516,11 +516,13 @@ struct drm_gpu_scheduler {atomic_t _score;bool ready;bool free_guilty; +bool pause_run_wq;struct device *dev; };int drm_sched_init (struct drm_gpu_scheduler *sched,const struct drm_sched_backend_ops *ops, +struct workqueue_struct *run_wq,uint32_t hw_submission,unsigned hang_limit,long timeout,struct workqueue_struct *timeout_wq,atomic_t *score,const char *name,struct device *dev);@@ -550,6 +552,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched); +void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);void drm_sched_start (struct drm_gpu_scheduler *sched,bool full_recovery);void drm_sched_resubmit_jobs (struct drm_gpu_scheduler *sched);
Matthew Brost
maintainerRe: [PATCH 1/8] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 53 lines · Tvrtko Ursulin
Quoted 51 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in workerSigned-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 14 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 14 +- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +- drivers/gpu/drm/lima/lima_sched.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 6 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +- drivers/gpu/drm/panfrost/panfrost_job.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 136 +++++++++++--------- drivers/gpu/drm/v3d/v3d_sched.c | 10 +- include/drm/gpu_scheduler.h | 14 +- 10 files changed, 113 insertions(+), 89 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +1
-1@@ -1489,9 +1489,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)for (i =0 ; i <AMDGPU_MAX_RINGS ; i++) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)
Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · line 1494
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +1
-1continue ; -kthread_park (ring->sched.thread); +drm_sched_run_wq_stop (&ring->sched);It would be good to split out adding of these wrappers (including adding one for ring->sched.thread/ready) to a standalong preceding patch. That way at least some mechanical changes to various drivers would be separated from functional changes.
Sure.
Quoted 4 lines
Also, perhaps do not have the wq in the name if it is not really needed to be verbose with the underlying implementation like that? Like would drm_sched_run/pause. Or even __drm_sched_start/stop, dunno, just an idea.
Sure.
Quoted 274 lines
Quoted 274 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +5
-5}seq_printf (m,"run ib test:\n" );@@ -1505,9 +1505,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)for (i =0 ; i <AMDGPU_MAX_RINGS ; i++) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ; -kthread_unpark (ring->sched.thread); +drm_sched_run_wq_start (&ring->sched); }up_write (&adev->reset_domain->sem);@@ -1727,7 +1727,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) ring = adev->rings[val]; -if (!ring || !ring->funcs->preempt_ib || !ring->sched.thread) +if (!ring || !ring->funcs->preempt_ib || !ring->sched.ready)return -EINVAL ;/* the last preemption failed */ @@ -1745,7 +1745,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) goto pro_end;/* stop the scheduler */ -kthread_park (ring->sched.thread); +drm_sched_run_wq_stop (&ring->sched);/* preempt the IB */ r =amdgpu_ring_preempt_ib (ring);@@ -1779,7 +1779,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) failure:/* restart the scheduler */ -kthread_unpark (ring->sched.thread); +drm_sched_run_wq_start (&ring->sched);up_read (&adev->reset_domain->sem);drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +7
-7@@ -2364,7 +2364,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev)break ; } - r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, + r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, NULL, ring->num_hw_submission, amdgpu_job_hang_limit, timeout, adev->reset_domain->wq, ring->sched_score, ring->name,@@ -4627,7 +4627,7 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev)for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;spin_lock (&ring->sched.job_list_lock);@@ -4753,7 +4753,7 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;/*clear job fence from fence drv to avoid force_completion @@ -5294,7 +5294,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = tmp_adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_stop (&ring->sched, job ? &job->base :NULL );@@ -5369,7 +5369,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = tmp_adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_start (&ring->sched, true);@@ -5696,7 +5696,7 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_stafor (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_stop (&ring->sched,NULL );@@ -5824,7 +5824,7 @@ void amdgpu_pci_resume(struct pci_dev *pdev)for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_start (&ring->sched, true);drivers/gpu/drm/etnaviv/etnaviv_sched.c +1
-1@@ -133,7 +133,7 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu) {int ret; - ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops, + ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops,NULL , etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,msecs_to_jiffies (500 ),NULL ,NULL ,dev_name (gpu->dev), gpu->dev);drivers/gpu/drm/lima/lima_sched.c +1
-1@@ -488,7 +488,7 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)INIT_WORK (&pipe->recover_work, lima_sched_recover_work); -return drm_sched_init (&pipe->base, &lima_sched_ops,1 , +return drm_sched_init (&pipe->base, &lima_sched_ops,NULL ,1 , lima_job_hang_limit,msecs_to_jiffies (timeout),NULL ,NULL , name, pipe->ldev->dev);drivers/gpu/drm/msm/adreno/adreno_device.c +4
-2@@ -662,7 +662,8 @@ static void suspend_scheduler(struct msm_gpu *gpu)*/ for (i =0 ; i < gpu->nr_rings; i++) {struct drm_gpu_scheduler *sched = &gpu->rb[i]->sched; -kthread_park (sched->thread); + +drm_sched_run_wq_stop (sched); } }@@ -672,7 +673,8 @@ static void resume_scheduler(struct msm_gpu *gpu)for (i =0 ; i < gpu->nr_rings; i++) {struct drm_gpu_scheduler *sched = &gpu->rb[i]->sched; -kthread_unpark (sched->thread); + +drm_sched_run_wq_start (sched); } }drivers/gpu/drm/msm/msm_ringbuffer.c +1
-1@@ -95,7 +95,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,/* currently managing hangcheck ourselves: */ sched_timeout =MAX_SCHEDULE_TIMEOUT ; - ret =drm_sched_init (&ring->sched, &msm_sched_ops, + ret =drm_sched_init (&ring->sched, &msm_sched_ops,NULL , num_hw_submissions,0 , sched_timeout,NULL ,NULL ,to_msm_bo (ring->bo)->name, gpu->dev->dev);if (ret) {drivers/gpu/drm/panfrost/panfrost_job.c +1
-1@@ -815,7 +815,7 @@ int panfrost_job_init(struct panfrost_device *pfdev) js->queue[j].fence_context =dma_fence_context_alloc (1 ); ret = drm_sched_init(&js->queue[j].sched, - &panfrost_sched_ops, + &panfrost_sched_ops, NULL, nentries, 0, msecs_to_jiffies(JOB_TIMEOUT_MS), pfdev->reset.wq,drivers/gpu/drm/scheduler/sched_main.c +48
-26@@ -44,7 +44,6 @@* The jobs in a entity are always scheduled in the order that they were pushed. */ -#include <linux/kthread.h> #include <linux/wait.h> #include <linux/sched.h> #include <linux/completion.h> @@ -252,6 +251,47 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) return rb ? rb_entry(rb, struct drm_sched_entity, rb_tree_node) : NULL; } +/** +* drm_sched_run_wq_stop - stop scheduler run worker +* +* @sched: scheduler instance to stop run worker +*/ +void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched) +{ +WRITE_ONCE (sched->pause_run_wq, true); +cancel_work_sync (&sched->work_run); +} +EXPORT_SYMBOL (drm_sched_run_wq_stop); + +/** +* drm_sched_run_wq_start - start scheduler run worker +* +* @sched: scheduler instance to start run worker +*/ +void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched) +{ +WRITE_ONCE (sched->pause_run_wq, false); +queue_work (sched->run_wq, &sched->work_run); +} +EXPORT_SYMBOL (drm_sched_run_wq_start); + +/** +* drm_sched_run_wq_queue - queue scheduler run worker +* +* @sched: scheduler instance to queue run worker +*/ +static void drm_sched_run_wq_queue (struct drm_gpu_scheduler *sched) +{ +/* +* Try not to schedule work if pause_run_wq set but not the end of world +* if we do as either it will be cancelled by the above +* cancel_work_sync, or drm_sched_main turns into a NOP while +* pause_run_wq is set. +*/ +if (!READ_ONCE (sched->pause_run_wq)) +queue_work (sched->run_wq, &sched->work_run); +} +/** * drm_sched_job_done - complete a job * @s_job: pointer to the job which is done @@ -271,7 +311,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence);dma_fence_put (&s_fence->finished); -wake_up_interruptible (&sched->wake_up_worker); +drm_sched_run_wq_queue (sched); }/** @@ -434,7 +474,7 @@ void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad) {struct drm_sched_job *s_job, *tmp; -kthread_park (sched->thread); +drm_sched_run_wq_stop (sched);/* * Reinsert back the bad job here - now it's safe as @@ -547,7 +587,7 @@ void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery)spin_unlock (&sched->job_list_lock); } -kthread_unpark (sched->thread); +drm_sched_run_wq_start (sched); }EXPORT_SYMBOL (drm_sched_start);@@ -864,7 +904,7 @@ static bool drm_sched_ready(struct drm_gpu_scheduler *sched)void drm_sched_wakeup (struct drm_gpu_scheduler *sched) {if (drm_sched_ready (sched)) -wake_up_interruptible (&sched->wake_up_worker); +drm_sched_run_wq_queue (sched); }/** @@ -974,61 +1014,42 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); -/** -* drm_sched_blocked - check if the scheduler is blocked -* -* @sched: scheduler instance -* -* Returns true if blocked, otherwise false. -*/ -static bool drm_sched_blocked (struct drm_gpu_scheduler *sched) -{ -if (kthread_should_park ()) { -kthread_parkme (); -return true; - } - -return false; -} -/** * drm_sched_main - main scheduler thread * * @param: scheduler instance -* -* Returns 0. */ -static int drm_sched_main (void *param) +static void drm_sched_main (struct work_struct *w) { -struct drm_gpu_scheduler *sched = (struct drm_gpu_scheduler *)param; +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run);
Quoted 9 lines
drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1031
Quoted 6 lines
drivers/gpu/drm/scheduler/sched_main.c +4
-1+struct drm_sched_entity *entity; +struct drm_sched_job *cleanup_job;int r; -sched_set_fifo_low (current); +if (READ_ONCE (sched->pause_run_wq)) +return ;Is there a point to this check given the comment in
drm_sched_run_wq_queue?
I think so.
Quoted 28 lines
Quoted 28 lines
drivers/gpu/drm/scheduler/sched_main.c +10
-16-while (!kthread_should_stop ()) { -struct drm_sched_entity *entity =NULL ; -struct drm_sched_fence *s_fence; -struct drm_sched_job *sched_job; -struct dma_fence *fence; -struct drm_sched_job *cleanup_job =NULL ; + cleanup_job =drm_sched_get_cleanup_job (sched); + entity =drm_sched_select_entity (sched); -wait_event_interruptible (sched->wake_up_worker, - (cleanup_job =drm_sched_get_cleanup_job (sched)) || - (!drm_sched_blocked (sched) && - (entity =drm_sched_select_entity (sched))) || -kthread_should_stop ()); +if (!entity && !cleanup_job) +return ;/* No more work */ -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (cleanup_job) + sched->ops->free_job (cleanup_job); -if (!entity) -continue ; +if (entity) { +struct dma_fence *fence; +struct drm_sched_fence *s_fence; +struct drm_sched_job *sched_job; sched_job =drm_sched_entity_pop_job (entity); -if (!sched_job) {
Quoted 9 lines
drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1052
Quoted 5 lines
drivers/gpu/drm/scheduler/sched_main.c +3
-1complete_all (&entity->entity_idle); -continue ; +if (!cleanup_job) +return ;/* No more work */ + goto again;Loop is gone but now it re-arms itself which is needed to avoid starvation? Is it guaranteed to be effective by the wq contract?
Yea.
Quoted 38 lines
Quoted 38 lines
drivers/gpu/drm/scheduler/sched_main.c +8
-3} s_fence = sched_job->s_fence;@@ -1055,14 +1076,17 @@ static int drm_sched_main(void *param) r); }else {if (IS_ERR (fence)) -dma_fence_set_error (&s_fence->finished,PTR_ERR (fence)); +dma_fence_set_error (&s_fence->finished, +PTR_ERR (fence));drm_sched_job_done (sched_job); }wake_up (&sched->job_scheduled); } -return 0 ; + +again: +drm_sched_run_wq_queue (sched); }/** @@ -1070,6 +1094,7 @@ static int drm_sched_main(void *param)* * @sched: scheduler instance * @ops: backend operations for this scheduler +* @run_wq: workqueue to use for run work. If NULL, the system_wq is used * @hw_submission: number of hw submissions that can be in flight * @hang_limit: number of times to allow a job to hang before dropping it * @timeout: timeout value in jiffies for the scheduler @@ -1083,14 +1108,16 @@ static int drm_sched_main(void *param)*/ int drm_sched_init (struct drm_gpu_scheduler *sched,const struct drm_sched_backend_ops *ops, +struct workqueue_struct *run_wq,unsigned hw_submission,unsigned hang_limit,long timeout,struct workqueue_struct *timeout_wq,atomic_t *score,const char *name,struct device *dev) { -int i, ret; +int i;
Quoted 9 lines
drivers/gpu/drm/scheduler/sched_main.c static int drm_sched_main(void *param) · line 1120
Quoted 4 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-0sched->ops = ops; sched->hw_submission_limit = hw_submission; sched->name = name; + sched->run_wq = run_wq ? : system_wq;I still think it is not nice to implicitly move everyone over to the shared system wq. Maybe even more so with now one at a time execution, since effect on latency can be even greater.
No one that has a stake in this has pushed back that I can recall. Open to feedback stakeholders (maintainers of drivers that use the drm scheduler). The i915 doesn't use the DRM scheduler last time I looked. Has that changed?
Quoted 4 lines
Have you considered
kthread_workas a backend? Maybe it would work to have callers pass in akthread_workerthey create, or provide adrm_schedhelper to create one, which would then be passed todrm_sched_init.
Quoted 9 lines
That would enable per driver
kthread_worker, or per device, or whatever granularity each driver would want/need/desire.driver init:
structdrm_sched_worker=drm_sched_create_worker(...);queue/whatever init:
drm_sched_init(.., worker, ...);
This idea doesn't seem to work for varitey of reasons. Will type it out if needed but not going to spend time on this unless someone with a stake raises this as an issue.
Quoted 4 lines
You could create one inside
drm_sched_initif not passed in, which would keep the behaviour for existing drivers more similar - they would still have a 1:1 kthread context for their exclusive use.
Part of the idea of a work queue is so a user can't directly create a kthread via an IOCTL (XE_EXEC_QUEUE_CREATE). What you suggesting exposes this issue.
Quoted 5 lines
And I *think* self-re-arming would be less problematic latency wise since
kthread_workerconsumes everything queued without relinquishing control and execution context would be guaranteed not to be shared with random system stuff.
So this is essentially so we can use a loop? Seems like a lot effort for what is pure speculation. Again if a stakeholder raises an issue we can address then.
Matt
Quoted 152 lines
Regards,
Tvrtko
Quoted 148 lines
drivers/gpu/drm/scheduler/sched_main.c +3
-12sched->timeout = timeout; sched->timeout_wq = timeout_wq ? : system_wq; sched->hang_limit = hang_limit;@@ -1099,23 +1126,15 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,for (i =DRM_SCHED_PRIORITY_MIN ; i <DRM_SCHED_PRIORITY_COUNT ; i++)drm_sched_rq_init (sched, &sched->sched_rq[i]); -init_waitqueue_head (&sched->wake_up_worker);init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); +INIT_WORK (&sched->work_run, drm_sched_main);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); - -/* Each scheduler will run on a seperate kernel thread */ - sched->thread =kthread_run (drm_sched_main, sched, sched->name); -if (IS_ERR (sched->thread)) { - ret =PTR_ERR (sched->thread); - sched->thread =NULL ; -DRM_DEV_ERROR (sched->dev,"Failed to create scheduler for %s.\n" , name); -return ret; - } + sched->pause_run_wq = false; sched->ready = true;return 0 ;@@ -1134,8 +1153,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)struct drm_sched_entity *s_entity;int i; -if (sched->thread) -kthread_stop (sched->thread); +drm_sched_run_wq_stop (sched);for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) {struct drm_sched_rq *rq = &sched->sched_rq[i];drivers/gpu/drm/v3d/v3d_sched.c +5
-5@@ -388,7 +388,7 @@ v3d_sched_init(struct v3d_dev *v3d)int ret; ret =drm_sched_init (&v3d->queue[V3D_BIN ].sched, - &v3d_bin_sched_ops, + &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_bin" , v3d->drm.dev);@@ -396,7 +396,7 @@ v3d_sched_init(struct v3d_dev *v3d)return ret; ret =drm_sched_init (&v3d->queue[V3D_RENDER ].sched, - &v3d_render_sched_ops, + &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_render" , v3d->drm.dev);@@ -404,7 +404,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_TFU ].sched, - &v3d_tfu_sched_ops, + &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_tfu" , v3d->drm.dev);@@ -413,7 +413,7 @@ v3d_sched_init(struct v3d_dev *v3d)if (v3d_has_csd (v3d)) { ret =drm_sched_init (&v3d->queue[V3D_CSD ].sched, - &v3d_csd_sched_ops, + &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_csd" , v3d->drm.dev);@@ -421,7 +421,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_CACHE_CLEAN ].sched, - &v3d_cache_clean_sched_ops, + &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_cache_clean" , v3d->drm.dev);include/drm/gpu_scheduler.h +9
-5@@ -473,17 +473,16 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. -* @wake_up_worker: the wait queue on which the scheduler sleeps until a job -* is ready to be scheduled. * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. +* @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr +* @work_run: schedules jobs and cleans up entities * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. -* @thread: the kthread on which the scheduler which run. * @pending_list: the list of jobs which are currently in the job queue. * @job_list_lock: lock to protect the pending_list. * @hang_limit: once the hangs by a job crosses this limit then it is marked @@ -492,6 +491,7 @@ struct drm_sched_backend_ops {* @_score: score used when the driver doesn't provide one * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. +* @pause_run_wq: pause queuing of @work_run on @run_wq * @dev: system &struct device * * One scheduler is implemented for each hardware ring. @@ -502,13 +502,13 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; -wait_queue_head_t wake_up_worker;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count; +struct workqueue_struct *run_wq;struct workqueue_struct *timeout_wq; +struct work_struct work_run;struct delayed_work work_tdr; -struct task_struct *thread;struct list_head pending_list;spinlock_t job_list_lock;int hang_limit;@@ -516,11 +516,13 @@ struct drm_gpu_scheduler {atomic_t _score;bool ready;bool free_guilty; +bool pause_run_wq;struct device *dev; };int drm_sched_init (struct drm_gpu_scheduler *sched,const struct drm_sched_backend_ops *ops, +struct workqueue_struct *run_wq,uint32_t hw_submission,unsigned hang_limit,long timeout,struct workqueue_struct *timeout_wq,atomic_t *score,const char *name,struct device *dev);@@ -550,6 +552,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched); +void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);void drm_sched_start (struct drm_gpu_scheduler *sched,bool full_recovery);void drm_sched_resubmit_jobs (struct drm_gpu_scheduler *sched);
Christian König
maintainerRe: [PATCH 1/8] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 03.08.23 um 16:43 schrieb Matthew Brost:
Quoted 11 lines · Tvrtko Ursulin
Quoted 10 lines · Matthew Brost
[SNIP]
drivers/gpu/drm/scheduler/sched_main.c static int drm_sched_main(void *param) · line 1120
Quoted 4 lines
sched->ops = ops; sched->hw_submission_limit = hw_submission; sched->name = name;drivers/gpu/drm/scheduler/sched_main.c +1
-0+ sched->run_wq = run_wq ? : system_wq;I still think it is not nice to implicitly move everyone over to the shared system wq. Maybe even more so with now one at a time execution, since effect on latency can be even greater.
Quoted 3 lines
No one that has a stake in this has pushed back that I can recall. Open to feedback stakeholders (maintainers of drivers that use the drm scheduler).
No objections to using the system_wq here. Drivers can still pass in their own or simply use system_highpri_wq instead.
Additional to that the system_wq isn't single threaded, it will create as much threads as needed to fully utilize all CPUs.
Quoted 13 lines
The i915 doesn't use the DRM scheduler last time I looked.
Has that changed?Quoted 10 lines
Have you considered
kthread_workas a backend? Maybe it would work to have callers pass in akthread_workerthey create, or provide adrm_schedhelper to create one, which would then be passed todrm_sched_init.That would enable per driver
kthread_worker, or per device, or whatever granularity each driver would want/need/desire.driver init:
structdrm_sched_worker=drm_sched_create_worker(...);
Quoted 6 lines
Quoted 3 lines
queue/whatever init:
drm_sched_init(.., worker, ...);This idea doesn't seem to work for varitey of reasons. Will type it out if needed but not going to spend time on this unless someone with a stake raises this as an issue.
Agree completely. kthread_work is for real time workers IIRC.
Quoted 8 lines
Quoted 5 lines
You could create one inside
drm_sched_initif not passed in, which would keep the behaviour for existing drivers more similar - they would still have a 1:1 kthread context for their exclusive use.Part of the idea of a work queue is so a user can't directly create a kthread via an IOCTL (
XE_EXEC_QUEUE_CREATE). What you suggesting exposes this issue.
Yeah, prevent that is indeed a very good idea.
Quoted 9 lines
Quoted 6 lines
And I *think* self-re-arming would be less problematic latency wise since
kthread_workerconsumes everything queued without relinquishing control and execution context would be guaranteed not to be shared with random system stuff.So this is essentially so we can use a loop? Seems like a lot effort for what is pure speculation. Again if a stakeholder raises an issue we can address then.
Instead of a loop what you usually do in the worker is to submit one item (if possible) and then re-queue yourself if there is more work to do.
This way you give others chance to run as well and/or cancel the work etc...
Christian.
Quoted 7 lines
Matt
Quoted 4 lines
Regards,
Tvrtko
Re: [PATCH 1/8] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 32 lines · Christian König
Am 03.08.23 um 16:43 schrieb Matthew Brost:
Quoted 7 lines · Tvrtko Ursulin
Quoted 6 lines · Matthew Brost
[SNIP]
Quoted 4 lines
sched->ops= ops;sched->hw_submission_limit=hw_submission;sched->name= name;
+sched->run_wq=run_wq? :system_wq;Quoted 7 lines
Quoted 4 lines
I still think it is not nice to implicitly move everyone over to the shared system wq. Maybe even more so with now one at a time execution, since effect on latency can be even greater.
No one that has a stake in this has pushed back that I can recall. Open to feedback stakeholders (maintainers of drivers that use the drm scheduler).
No objections to using the
system_wqhere. Drivers can still pass in their own or simply usesystem_highpri_wqinstead.Additional to that the
system_wqisn't single threaded, it will create as much threads as needed to fully utilize all CPUs.Quoted 12 lines
The i915 doesn't use the DRM scheduler last time I looked. Has that changed?
Quoted 10 lines
Have you considered
kthread_workas a backend? Maybe it would work to have callers pass in akthread_workerthey create, or provide adrm_schedhelper to create one, which would then be passed todrm_sched_init.That would enable per driver
kthread_worker, or per device, or whatever granularity each driver would want/need/desire.driver init:
structdrm_sched_worker=drm_sched_create_worker(...);
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
queue/whatever init:
drm_sched_init(.., worker, ...);This idea doesn't seem to work for varitey of reasons. Will type it out if needed but not going to spend time on this unless someone with a stake raises this as an issue.
Agree completely.
kthread_workis for real time workers IIRC.
AFAIK it is indicated if one needs to tweak the kthread priority, but that is not the only use case.
I am curious to know why the idea does not work for variety of reasons.
Quoted 9 lines
Quoted 7 lines
Quoted 4 lines
You could create one inside
drm_sched_initif not passed in, which would keep the behaviour for existing drivers more similar - they would still have a 1:1 kthread context for their exclusive use.Part of the idea of a work queue is so a user can't directly create a kthread via an IOCTL (
XE_EXEC_QUEUE_CREATE). What you suggesting exposes this issue.Yeah, prevent that is indeed a very good idea.
Nope, I wasn't suggesting that at all.
I was suggesting as many kthread_workers (these are threads) as the implementation wants. Xe can create one per device. Someone else can create one per hw engine, whatever.
One kthread_*work* per entity does not mean one thread per XE_EXEC_QUEUE_CREATE. Kthread_work is just a unit of work executed by the kthread_worker thread. Same in that conceptual relationship as workqueue and workitem.
Difference is it may work better for single-shot re-arming design if regression in submission latency concerns any stakeholders.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
And I *think* self-re-arming would be less problematic latency wise since
kthread_workerconsumes everything queued without relinquishing control and execution context would be guaranteed not to be shared with random system stuff.
Quoted 7 lines
Quoted 3 lines
So this is essentially so we can use a loop? Seems like a lot effort for what is pure speculation. Again if a stakeholder raises an issue we can address then.
Instead of a loop what you usually do in the worker is to submit one item (if possible) and then re-queue yourself if there is more work to do.
This way you give others chance to run as well and/or cancel the work etc...
Yeah I was pointing out loop in the worker was bad months ago (or more) so it is not about that. Here my point is whether it can be done better than silently convert everyone to system_wq.
Hence my proposal is to *keep* closer to the thread semantics for everyone and at the same time _allow_ the option of custom workqueue/whatever.
Where is the problem there?
Regards,
Tvrtko
Re: [PATCH 1/8] drm/sched: Convert drm scheduler to use a work queue rather than kthread
In reply to Matthew BrostQuoted 357 lines · Matthew Brost
Quoted 53 lines · Tvrtko Ursulin
Quoted 51 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in workerSigned-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c | 14 +- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 14 +- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +- drivers/gpu/drm/lima/lima_sched.c | 2 +- drivers/gpu/drm/msm/adreno/adreno_device.c | 6 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +- drivers/gpu/drm/panfrost/panfrost_job.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 136 +++++++++++--------- drivers/gpu/drm/v3d/v3d_sched.c | 10 +- include/drm/gpu_scheduler.h | 14 +- 10 files changed, 113 insertions(+), 89 deletions(-)drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +1
-1@@ -1489,9 +1489,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)for (i =0 ; i <AMDGPU_MAX_RINGS ; i++) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)Quoted 9 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused) · line 1494
Quoted 3 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +1
-1continue ; -kthread_park (ring->sched.thread); +drm_sched_run_wq_stop (&ring->sched);It would be good to split out adding of these wrappers (including adding one for ring->sched.thread/ready) to a standalong preceding patch. That way at least some mechanical changes to various drivers would be separated from functional changes.
Sure.
Quoted 4 lines
Also, perhaps do not have the wq in the name if it is not really needed to be verbose with the underlying implementation like that? Like would drm_sched_run/pause. Or even __drm_sched_start/stop, dunno, just an idea.
Sure.
Quoted 274 lines
Quoted 274 lines
drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c +5
-5}seq_printf (m,"run ib test:\n" );@@ -1505,9 +1505,9 @@ static int amdgpu_debugfs_test_ib_show(struct seq_file *m, void *unused)for (i =0 ; i <AMDGPU_MAX_RINGS ; i++) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ; -kthread_unpark (ring->sched.thread); +drm_sched_run_wq_start (&ring->sched); }up_write (&adev->reset_domain->sem);@@ -1727,7 +1727,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) ring = adev->rings[val]; -if (!ring || !ring->funcs->preempt_ib || !ring->sched.thread) +if (!ring || !ring->funcs->preempt_ib || !ring->sched.ready)return -EINVAL ;/* the last preemption failed */ @@ -1745,7 +1745,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) goto pro_end;/* stop the scheduler */ -kthread_park (ring->sched.thread); +drm_sched_run_wq_stop (&ring->sched);/* preempt the IB */ r =amdgpu_ring_preempt_ib (ring);@@ -1779,7 +1779,7 @@ static int amdgpu_debugfs_ib_preempt(void *data, u64 val) failure:/* restart the scheduler */ -kthread_unpark (ring->sched.thread); +drm_sched_run_wq_start (&ring->sched);up_read (&adev->reset_domain->sem);drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +7
-7@@ -2364,7 +2364,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev)break ; } - r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, + r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, NULL, ring->num_hw_submission, amdgpu_job_hang_limit, timeout, adev->reset_domain->wq, ring->sched_score, ring->name,@@ -4627,7 +4627,7 @@ bool amdgpu_device_has_job_running(struct amdgpu_device *adev)for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;spin_lock (&ring->sched.job_list_lock);@@ -4753,7 +4753,7 @@ int amdgpu_device_pre_asic_reset(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;/*clear job fence from fence drv to avoid force_completion @@ -5294,7 +5294,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = tmp_adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_stop (&ring->sched, job ? &job->base :NULL );@@ -5369,7 +5369,7 @@ int amdgpu_device_gpu_recover(struct amdgpu_device *adev,for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = tmp_adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_start (&ring->sched, true);@@ -5696,7 +5696,7 @@ pci_ers_result_t amdgpu_pci_error_detected(struct pci_dev *pdev, pci_channel_stafor (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_stop (&ring->sched,NULL );@@ -5824,7 +5824,7 @@ void amdgpu_pci_resume(struct pci_dev *pdev)for (i =0 ; i <AMDGPU_MAX_RINGS ; ++i) {struct amdgpu_ring *ring = adev->rings[i]; -if (!ring || !ring->sched.thread) +if (!ring || !ring->sched.ready)continue ;drm_sched_start (&ring->sched, true);drivers/gpu/drm/etnaviv/etnaviv_sched.c +1
-1@@ -133,7 +133,7 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu) {int ret; - ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops, + ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops,NULL , etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,msecs_to_jiffies (500 ),NULL ,NULL ,dev_name (gpu->dev), gpu->dev);drivers/gpu/drm/lima/lima_sched.c +1
-1@@ -488,7 +488,7 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)INIT_WORK (&pipe->recover_work, lima_sched_recover_work); -return drm_sched_init (&pipe->base, &lima_sched_ops,1 , +return drm_sched_init (&pipe->base, &lima_sched_ops,NULL ,1 , lima_job_hang_limit,msecs_to_jiffies (timeout),NULL ,NULL , name, pipe->ldev->dev);drivers/gpu/drm/msm/adreno/adreno_device.c +4
-2@@ -662,7 +662,8 @@ static void suspend_scheduler(struct msm_gpu *gpu)*/ for (i =0 ; i < gpu->nr_rings; i++) {struct drm_gpu_scheduler *sched = &gpu->rb[i]->sched; -kthread_park (sched->thread); + +drm_sched_run_wq_stop (sched); } }@@ -672,7 +673,8 @@ static void resume_scheduler(struct msm_gpu *gpu)for (i =0 ; i < gpu->nr_rings; i++) {struct drm_gpu_scheduler *sched = &gpu->rb[i]->sched; -kthread_unpark (sched->thread); + +drm_sched_run_wq_start (sched); } }drivers/gpu/drm/msm/msm_ringbuffer.c +1
-1@@ -95,7 +95,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,/* currently managing hangcheck ourselves: */ sched_timeout =MAX_SCHEDULE_TIMEOUT ; - ret =drm_sched_init (&ring->sched, &msm_sched_ops, + ret =drm_sched_init (&ring->sched, &msm_sched_ops,NULL , num_hw_submissions,0 , sched_timeout,NULL ,NULL ,to_msm_bo (ring->bo)->name, gpu->dev->dev);if (ret) {drivers/gpu/drm/panfrost/panfrost_job.c +1
-1@@ -815,7 +815,7 @@ int panfrost_job_init(struct panfrost_device *pfdev) js->queue[j].fence_context =dma_fence_context_alloc (1 ); ret = drm_sched_init(&js->queue[j].sched, - &panfrost_sched_ops, + &panfrost_sched_ops, NULL, nentries, 0, msecs_to_jiffies(JOB_TIMEOUT_MS), pfdev->reset.wq,drivers/gpu/drm/scheduler/sched_main.c +48
-26@@ -44,7 +44,6 @@* The jobs in a entity are always scheduled in the order that they were pushed. */ -#include <linux/kthread.h> #include <linux/wait.h> #include <linux/sched.h> #include <linux/completion.h> @@ -252,6 +251,47 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) return rb ? rb_entry(rb, struct drm_sched_entity, rb_tree_node) : NULL; } +/** +* drm_sched_run_wq_stop - stop scheduler run worker +* +* @sched: scheduler instance to stop run worker +*/ +void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched) +{ +WRITE_ONCE (sched->pause_run_wq, true); +cancel_work_sync (&sched->work_run); +} +EXPORT_SYMBOL (drm_sched_run_wq_stop); + +/** +* drm_sched_run_wq_start - start scheduler run worker +* +* @sched: scheduler instance to start run worker +*/ +void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched) +{ +WRITE_ONCE (sched->pause_run_wq, false); +queue_work (sched->run_wq, &sched->work_run); +} +EXPORT_SYMBOL (drm_sched_run_wq_start); + +/** +* drm_sched_run_wq_queue - queue scheduler run worker +* +* @sched: scheduler instance to queue run worker +*/ +static void drm_sched_run_wq_queue (struct drm_gpu_scheduler *sched) +{ +/* +* Try not to schedule work if pause_run_wq set but not the end of world +* if we do as either it will be cancelled by the above +* cancel_work_sync, or drm_sched_main turns into a NOP while +* pause_run_wq is set. +*/ +if (!READ_ONCE (sched->pause_run_wq)) +queue_work (sched->run_wq, &sched->work_run); +} +/** * drm_sched_job_done - complete a job * @s_job: pointer to the job which is done @@ -271,7 +311,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence);dma_fence_put (&s_fence->finished); -wake_up_interruptible (&sched->wake_up_worker); +drm_sched_run_wq_queue (sched); }/** @@ -434,7 +474,7 @@ void drm_sched_stop(struct drm_gpu_scheduler *sched, struct drm_sched_job *bad) {struct drm_sched_job *s_job, *tmp; -kthread_park (sched->thread); +drm_sched_run_wq_stop (sched);/* * Reinsert back the bad job here - now it's safe as @@ -547,7 +587,7 @@ void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery)spin_unlock (&sched->job_list_lock); } -kthread_unpark (sched->thread); +drm_sched_run_wq_start (sched); }EXPORT_SYMBOL (drm_sched_start);@@ -864,7 +904,7 @@ static bool drm_sched_ready(struct drm_gpu_scheduler *sched)void drm_sched_wakeup (struct drm_gpu_scheduler *sched) {if (drm_sched_ready (sched)) -wake_up_interruptible (&sched->wake_up_worker); +drm_sched_run_wq_queue (sched); }/** @@ -974,61 +1014,42 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); -/** -* drm_sched_blocked - check if the scheduler is blocked -* -* @sched: scheduler instance -* -* Returns true if blocked, otherwise false. -*/ -static bool drm_sched_blocked (struct drm_gpu_scheduler *sched) -{ -if (kthread_should_park ()) { -kthread_parkme (); -return true; - } - -return false; -} -/** * drm_sched_main - main scheduler thread * * @param: scheduler instance -* -* Returns 0. */ -static int drm_sched_main (void *param) +static void drm_sched_main (struct work_struct *w) { -struct drm_gpu_scheduler *sched = (struct drm_gpu_scheduler *)param; +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run);Quoted 9 lines
drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1031
Quoted 6 lines
drivers/gpu/drm/scheduler/sched_main.c +4
-1+struct drm_sched_entity *entity; +struct drm_sched_job *cleanup_job;int r; -sched_set_fifo_low (current); +if (READ_ONCE (sched->pause_run_wq)) +return ;Is there a point to this check given the comment in
drm_sched_run_wq_queue?
Quoted 1 line
I think so.
What is it? The helper which arms it already checks the paused flag and the comment says it is okay if it waits for it to exit.
Quoted 38 lines
Quoted 28 lines
Quoted 28 lines
drivers/gpu/drm/scheduler/sched_main.c +10
-16-while (!kthread_should_stop ()) { -struct drm_sched_entity *entity =NULL ; -struct drm_sched_fence *s_fence; -struct drm_sched_job *sched_job; -struct dma_fence *fence; -struct drm_sched_job *cleanup_job =NULL ; + cleanup_job =drm_sched_get_cleanup_job (sched); + entity =drm_sched_select_entity (sched); -wait_event_interruptible (sched->wake_up_worker, - (cleanup_job =drm_sched_get_cleanup_job (sched)) || - (!drm_sched_blocked (sched) && - (entity =drm_sched_select_entity (sched))) || -kthread_should_stop ()); +if (!entity && !cleanup_job) +return ;/* No more work */ -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (cleanup_job) + sched->ops->free_job (cleanup_job); -if (!entity) -continue ; +if (entity) { +struct dma_fence *fence; +struct drm_sched_fence *s_fence; +struct drm_sched_job *sched_job; sched_job =drm_sched_entity_pop_job (entity); -if (!sched_job) {Quoted 9 lines
drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1052
Quoted 5 lines
drivers/gpu/drm/scheduler/sched_main.c +3
-1complete_all (&entity->entity_idle); -continue ; +if (!cleanup_job) +return ;/* No more work */ + goto again;Loop is gone but now it re-arms itself which is needed to avoid starvation? Is it guaranteed to be effective by the wq contract?
Quoted 1 line
Yea.
What exactly is the requirement for it to effectively avoid starvation? That same workitems are not executed back-to-back? That wq must be ordered or what?
Quoted 48 lines
Quoted 38 lines
Quoted 38 lines
drivers/gpu/drm/scheduler/sched_main.c +8
-3} s_fence = sched_job->s_fence;@@ -1055,14 +1076,17 @@ static int drm_sched_main(void *param) r); }else {if (IS_ERR (fence)) -dma_fence_set_error (&s_fence->finished,PTR_ERR (fence)); +dma_fence_set_error (&s_fence->finished, +PTR_ERR (fence));drm_sched_job_done (sched_job); }wake_up (&sched->job_scheduled); } -return 0 ; + +again: +drm_sched_run_wq_queue (sched); }/** @@ -1070,6 +1094,7 @@ static int drm_sched_main(void *param)* * @sched: scheduler instance * @ops: backend operations for this scheduler +* @run_wq: workqueue to use for run work. If NULL, the system_wq is used * @hw_submission: number of hw submissions that can be in flight * @hang_limit: number of times to allow a job to hang before dropping it * @timeout: timeout value in jiffies for the scheduler @@ -1083,14 +1108,16 @@ static int drm_sched_main(void *param)*/ int drm_sched_init (struct drm_gpu_scheduler *sched,const struct drm_sched_backend_ops *ops, +struct workqueue_struct *run_wq,unsigned hw_submission,unsigned hang_limit,long timeout,struct workqueue_struct *timeout_wq,atomic_t *score,const char *name,struct device *dev) { -int i, ret; +int i;Quoted 9 lines
drivers/gpu/drm/scheduler/sched_main.c static int drm_sched_main(void *param) · line 1120
Quoted 4 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-0sched->ops = ops; sched->hw_submission_limit = hw_submission; sched->name = name; + sched->run_wq = run_wq ? : system_wq;I still think it is not nice to implicitly move everyone over to the shared system wq. Maybe even more so with now one at a time execution, since effect on latency can be even greater.
Quoted 4 lines
No one that has a stake in this has pushed back that I can recall. Open to feedback stakeholders (maintainers of drivers that use the drm scheduler). The i915 doesn't use the DRM scheduler last time I looked. Has that changed?
Matt, I am not pushing back but commenting and discussing. My interest is for the stack, including Xe, to work well.
Quoted 10 lines
Quoted 10 lines
Have you considered
kthread_workas a backend? Maybe it would work to have callers pass in akthread_workerthey create, or provide adrm_schedhelper to create one, which would then be passed todrm_sched_init.That would enable per driver
kthread_worker, or per device, or whatever granularity each driver would want/need/desire.driver init:
structdrm_sched_worker=drm_sched_create_worker(...);
Quoted 7 lines
Quoted 3 lines
queue/whatever init:
drm_sched_init(.., worker, ...);This idea doesn't seem to work for varitey of reasons. Will type it out if needed but not going to spend time on this unless someone with a stake raises this as an issue.
Maybe I did not explain this well enough, or maybe you did not really read it all given the "not a stakeholder" a little bit of a knee jerk reaction. I will try again.
With what I wrote you could invisibly switch the underlying implementation between kthread_worker/work to workqueue/workitem and in many aspects it behaves exactly the same as your design.
I was not implying a kthread per context/queue/entity!
There are two main differences in what I wrote:
1) Abstract out the mechanism.
2) Provide private execution context by default.
And a third difference which kthread_worker brings over the workqueue:
3) Stay closer to current kthread solution in terms of submission latencies when mutlitple jobs are signalled at once.
So it is really similar to your design. Just discussing those few details.
Regards,
Tvrtko
Quoted 173 lines
Quoted 4 lines
You could create one inside
drm_sched_initif not passed in, which would keep the behaviour for existing drivers more similar - they would still have a 1:1 kthread context for their exclusive use.Part of the idea of a work queue is so a user can't directly create a kthread via an IOCTL (
XE_EXEC_QUEUE_CREATE). What you suggesting exposes this issue.Quoted 5 lines
And I *think* self-re-arming would be less problematic latency wise since
kthread_workerconsumes everything queued without relinquishing control and execution context would be guaranteed not to be shared with random system stuff.So this is essentially so we can use a loop? Seems like a lot effort for what is pure speculation. Again if a stakeholder raises an issue we can address then.
Matt
Quoted 152 lines
Regards,
Tvrtko
Quoted 148 lines
drivers/gpu/drm/scheduler/sched_main.c +3
-12sched->timeout = timeout; sched->timeout_wq = timeout_wq ? : system_wq; sched->hang_limit = hang_limit;@@ -1099,23 +1126,15 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,for (i =DRM_SCHED_PRIORITY_MIN ; i <DRM_SCHED_PRIORITY_COUNT ; i++)drm_sched_rq_init (sched, &sched->sched_rq[i]); -init_waitqueue_head (&sched->wake_up_worker);init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); +INIT_WORK (&sched->work_run, drm_sched_main);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); - -/* Each scheduler will run on a seperate kernel thread */ - sched->thread =kthread_run (drm_sched_main, sched, sched->name); -if (IS_ERR (sched->thread)) { - ret =PTR_ERR (sched->thread); - sched->thread =NULL ; -DRM_DEV_ERROR (sched->dev,"Failed to create scheduler for %s.\n" , name); -return ret; - } + sched->pause_run_wq = false; sched->ready = true;return 0 ;@@ -1134,8 +1153,7 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)struct drm_sched_entity *s_entity;int i; -if (sched->thread) -kthread_stop (sched->thread); +drm_sched_run_wq_stop (sched);for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) {struct drm_sched_rq *rq = &sched->sched_rq[i];drivers/gpu/drm/v3d/v3d_sched.c +5
-5@@ -388,7 +388,7 @@ v3d_sched_init(struct v3d_dev *v3d)int ret; ret =drm_sched_init (&v3d->queue[V3D_BIN ].sched, - &v3d_bin_sched_ops, + &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_bin" , v3d->drm.dev);@@ -396,7 +396,7 @@ v3d_sched_init(struct v3d_dev *v3d)return ret; ret =drm_sched_init (&v3d->queue[V3D_RENDER ].sched, - &v3d_render_sched_ops, + &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_render" , v3d->drm.dev);@@ -404,7 +404,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_TFU ].sched, - &v3d_tfu_sched_ops, + &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_tfu" , v3d->drm.dev);@@ -413,7 +413,7 @@ v3d_sched_init(struct v3d_dev *v3d)if (v3d_has_csd (v3d)) { ret =drm_sched_init (&v3d->queue[V3D_CSD ].sched, - &v3d_csd_sched_ops, + &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_csd" , v3d->drm.dev);@@ -421,7 +421,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_CACHE_CLEAN ].sched, - &v3d_cache_clean_sched_ops, + &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_cache_clean" , v3d->drm.dev);include/drm/gpu_scheduler.h +9
-5@@ -473,17 +473,16 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. -* @wake_up_worker: the wait queue on which the scheduler sleeps until a job -* is ready to be scheduled. * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. +* @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr +* @work_run: schedules jobs and cleans up entities * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. -* @thread: the kthread on which the scheduler which run. * @pending_list: the list of jobs which are currently in the job queue. * @job_list_lock: lock to protect the pending_list. * @hang_limit: once the hangs by a job crosses this limit then it is marked @@ -492,6 +491,7 @@ struct drm_sched_backend_ops {* @_score: score used when the driver doesn't provide one * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. +* @pause_run_wq: pause queuing of @work_run on @run_wq * @dev: system &struct device * * One scheduler is implemented for each hardware ring. @@ -502,13 +502,13 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; -wait_queue_head_t wake_up_worker;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count; +struct workqueue_struct *run_wq;struct workqueue_struct *timeout_wq; +struct work_struct work_run;struct delayed_work work_tdr; -struct task_struct *thread;struct list_head pending_list;spinlock_t job_list_lock;int hang_limit;@@ -516,11 +516,13 @@ struct drm_gpu_scheduler {atomic_t _score;bool ready;bool free_guilty; +bool pause_run_wq;struct device *dev; };int drm_sched_init (struct drm_gpu_scheduler *sched,const struct drm_sched_backend_ops *ops, +struct workqueue_struct *run_wq,uint32_t hw_submission,unsigned hang_limit,long timeout,struct workqueue_struct *timeout_wq,atomic_t *score,const char *name,struct device *dev);@@ -550,6 +552,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched); +void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);void drm_sched_start (struct drm_gpu_scheduler *sched,bool full_recovery);void drm_sched_resubmit_jobs (struct drm_gpu_scheduler *sched);
Matthew Brost
maintainerPATCH 2/8 drm/sched: Move schedule policy to scheduler / entity
In reply to Matthew BrostRather than a global modparam for scheduling policy, move the scheduling policy to scheduler / entity so user can control each scheduler / entity policy.
v2:
- s/DRM_SCHED_POLICY_MAX/DRM_SCHED_POLICY_COUNT (Luben)
- Only include policy in scheduler (Luben)
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 + drivers/gpu/drm/etnaviv/etnaviv_sched.c | 3 ++- drivers/gpu/drm/lima/lima_sched.c | 3 ++- drivers/gpu/drm/msm/msm_ringbuffer.c | 3 ++- drivers/gpu/drm/panfrost/panfrost_job.c | 3 ++- drivers/gpu/drm/scheduler/sched_entity.c | 24 ++++++++++++++++++---- drivers/gpu/drm/scheduler/sched_main.c | 23 +++++++++++++++------ drivers/gpu/drm/v3d/v3d_sched.c | 15 +++++++++----- include/drm/gpu_scheduler.h | 20 ++++++++++++------ 9 files changed, 70 insertions(+), 25 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
@@ -2368,6 +2368,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev) ring->num_hw_submission, amdgpu_job_hang_limit, timeout, adev->reset_domain->wq, ring->sched_score, ring->name, +DRM_SCHED_POLICY_DEFAULT , adev->dev);if (r) { DRM_ERROR("Failed to create scheduler on ring %s.\n",
drivers/gpu/drm/etnaviv/etnaviv_sched.c +2 -1
@@ -136,7 +136,8 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu) ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops,NULL , etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,msecs_to_jiffies (500 ),NULL ,NULL , -dev_name (gpu->dev), gpu->dev); +dev_name (gpu->dev),DRM_SCHED_POLICY_DEFAULT , + gpu->dev);if (ret)return ret;
drivers/gpu/drm/lima/lima_sched.c +2 -1
@@ -491,7 +491,8 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)return drm_sched_init (&pipe->base, &lima_sched_ops,NULL ,1 , lima_job_hang_limit,msecs_to_jiffies (timeout),NULL , -NULL , name, pipe->ldev->dev); +NULL , name,DRM_SCHED_POLICY_DEFAULT , + pipe->ldev->dev); }void lima_sched_pipe_fini (struct lima_sched_pipe *pipe)
drivers/gpu/drm/msm/msm_ringbuffer.c +2 -1
@@ -97,7 +97,8 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id, ret =drm_sched_init (&ring->sched, &msm_sched_ops,NULL , num_hw_submissions,0 , sched_timeout, -NULL ,NULL ,to_msm_bo (ring->bo)->name, gpu->dev->dev); +NULL ,NULL ,to_msm_bo (ring->bo)->name, +DRM_SCHED_POLICY_DEFAULT , gpu->dev->dev);if (ret) { goto fail; }
drivers/gpu/drm/panfrost/panfrost_job.c +2 -1
@@ -819,7 +819,8 @@ int panfrost_job_init(struct panfrost_device *pfdev) nentries,0 ,msecs_to_jiffies (JOB_TIMEOUT_MS ), pfdev->reset.wq, - NULL, "pan_js", pfdev->dev); +NULL ,"pan_js" ,DRM_SCHED_POLICY_DEFAULT , + pfdev->dev);if (ret) {dev_err (pfdev->dev,"Failed to create scheduler: %d." , ret); goto err_sched;
drivers/gpu/drm/scheduler/sched_entity.c +20 -4
@@ -33,6 +33,20 @@#define to_drm_sched_job (sched_job) \ container_of((sched_job), struct drm_sched_job, queue_node) +static bool bad_policies (struct drm_gpu_scheduler **sched_list, +unsigned int num_sched_list) +{ +enum drm_sched_policy sched_policy = sched_list[0 ]->sched_policy; +unsigned int i; + +/* All schedule policies must match */ +for (i =1 ; i < num_sched_list; ++i) +if (sched_policy != sched_list[i]->sched_policy) +return true; + +return false; +} +/** * drm_sched_entity_init - Init a context entity used by scheduler when * submit to HW ring.
@@ -62,7 +76,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, unsigned int num_sched_list, atomic_t *guilty) { -if (!(entity && sched_list && (num_sched_list ==0 || sched_list[0 ]))) +if (!(entity && sched_list && (num_sched_list ==0 || sched_list[0 ])) || +bad_policies (sched_list, num_sched_list))return -EINVAL ;memset (entity,0 ,sizeof (struct drm_sched_entity ));
@@ -440,7 +455,7 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)* Update the entity's location in the min heap according to * the timestamp of the next job, if any. */ -if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO ) { +if (entity->rq->sched->sched_policy ==DRM_SCHED_POLICY_FIFO ) {struct drm_sched_job *next; next =to_drm_sched_job (spsc_queue_peek (&entity->job_queue));
@@ -506,7 +521,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)void drm_sched_entity_push_job (struct drm_sched_job *sched_job) {struct drm_sched_entity *entity = sched_job->entity; -bool first; +bool first, fifo = entity->rq->sched->sched_policy == +DRM_SCHED_POLICY_FIFO ;trace_drm_sched_job (sched_job, entity);atomic_inc (entity->rq->sched->score);
@@ -528,7 +544,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)drm_sched_rq_add_entity (entity->rq, entity);spin_unlock (&entity->rq_lock); -if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO ) +if (fifo)drm_sched_rq_update_fifo (entity, sched_job->submit_ts);drm_sched_wakeup (entity->rq->sched);
drivers/gpu/drm/scheduler/sched_main.c +17 -6
@@ -62,14 +62,14 @@#define to_drm_sched_job (sched_job) \ container_of((sched_job), struct drm_sched_job, queue_node) -int drm_sched_policy =DRM_SCHED_POLICY_FIFO ; +int default_drm_sched_policy =DRM_SCHED_POLICY_FIFO ;/** * DOC: sched_policy (int) * Used to override default entities scheduling policy in a run queue. */ -MODULE_PARM_DESC(sched_policy, "Specify the scheduling policy for entities on a run-queue, " __stringify(DRM_SCHED_POLICY_RR) " = Round Robin, " __stringify(DRM_SCHED_POLICY_FIFO) " = FIFO (default)."); -module_param_named (sched_policy, drm_sched_policy, int,0444 ); +MODULE_PARM_DESC(sched_policy, "Specify the default scheduling policy for entities on a run-queue, " __stringify(DRM_SCHED_POLICY_RR) " = Round Robin, " __stringify(DRM_SCHED_POLICY_FIFO) " = FIFO (default)."); +module_param_named (sched_policy, default_drm_sched_policy, int,0444 ); static __always_inline bool drm_sched_entity_compare_before(struct rb_node *a,const struct rb_node *b)
@@ -173,7 +173,7 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,if (rq->current_entity == entity) rq->current_entity =NULL ; -if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO ) +if (rq->sched->sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity);spin_unlock (&rq->lock);
@@ -925,7 +925,7 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { - entity = drm_sched_policy ==DRM_SCHED_POLICY_FIFO ? + entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ?drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) :drm_sched_rq_select_entity_rr (&sched->sched_rq[i]);if (entity)
@@ -1102,6 +1102,7 @@ static void drm_sched_main(struct work_struct *w)* used * @score: optional score atomic shared with other schedulers * @name: name used for debugging +* @sched_policy: schedule policy * @dev: target &struct device * * Return 0 on success, otherwise error code.
@@ -1111,9 +1112,15 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,struct workqueue_struct *run_wq, unsigned hw_submission, unsigned hang_limit, long timeout, struct workqueue_struct *timeout_wq, - atomic_t *score, const char *name, struct device *dev) + atomic_t *score, const char *name, + enum drm_sched_policy sched_policy, + struct device *dev) { int i; + +if (sched_policy >=DRM_SCHED_POLICY_COUNT ) +return -EINVAL ; + sched->ops = ops; sched->hw_submission_limit = hw_submission; sched->name = name;
@@ -1123,6 +1130,10 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, sched->hang_limit = hang_limit; sched->score = score ? score : &sched->_score; sched->dev = dev; +if (sched_policy ==DRM_SCHED_POLICY_DEFAULT ) + sched->sched_policy = default_drm_sched_policy; +else + sched->sched_policy = sched_policy;for (i =DRM_SCHED_PRIORITY_MIN ; i <DRM_SCHED_PRIORITY_COUNT ; i++)drm_sched_rq_init (sched, &sched->sched_rq[i]);
drivers/gpu/drm/v3d/v3d_sched.c +10 -5
@@ -391,7 +391,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_bin", v3d->drm.dev); +NULL ,"v3d_bin" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret)return ret;
@@ -399,7 +400,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_render", v3d->drm.dev); +ULL ,"v3d_render" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;
@@ -407,7 +409,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_tfu", v3d->drm.dev); +NULL ,"v3d_tfu" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;
@@ -416,7 +419,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_csd", v3d->drm.dev); +NULL ,"v3d_csd" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;
@@ -424,7 +428,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_cache_clean", v3d->drm.dev); +NULL ,"v3d_cache_clean" , + DRM_SCHED_POLICY_DEFAULT, v3d->drm.dev);if (ret) goto fail; }
include/drm/gpu_scheduler.h +14 -6
@@ -72,11 +72,15 @@ enum drm_sched_priority { DRM_SCHED_PRIORITY_UNSET = -2 }; -/* Used to chose between FIFO and RR jobs scheduling */ -extern int drm_sched_policy; - -#define DRM_SCHED_POLICY_RR 0 -#define DRM_SCHED_POLICY_FIFO 1 +/* Used to chose default scheduling policy*/ +extern int default_drm_sched_policy; + +enum drm_sched_policy { +DRM_SCHED_POLICY_DEFAULT , +DRM_SCHED_POLICY_RR , +DRM_SCHED_POLICY_FIFO , +DRM_SCHED_POLICY_COUNT , +};/** * struct drm_sched_entity - A wrapper around a job queue (typically
@@ -489,6 +493,7 @@ struct drm_sched_backend_ops {* guilty and it will no longer be considered for scheduling. * @score: score to help loadbalancer pick a idle sched * @_score: score used when the driver doesn't provide one +* @sched_policy: Schedule policy for scheduler * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. * @pause_run_wq: pause queuing of @work_run on @run_wq
@@ -514,6 +519,7 @@ struct drm_gpu_scheduler {int hang_limit;atomic_t *score;atomic_t _score; +enum drm_sched_policy sched_policy;bool ready;bool free_guilty;bool pause_run_wq;
@@ -525,7 +531,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,struct workqueue_struct *run_wq, uint32_t hw_submission, unsigned hang_limit, long timeout, struct workqueue_struct *timeout_wq, - atomic_t *score, const char *name, struct device *dev); + atomic_t *score, const char *name, + enum drm_sched_policy sched_policy, + struct device *dev);void drm_sched_fini (struct drm_gpu_scheduler *sched); int drm_sched_job_init(struct drm_sched_job *job,
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 3/8 drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy
In reply to Matthew BrostDRM_SCHED_POLICY_SINGLE_ENTITY creates a 1 to 1 relationship between scheduler and entity. No priorities or run queue used in this mode. Intended for devices with firmware schedulers.
v2:
- Drop sched / rq union (Luben)
--- drivers/gpu/drm/scheduler/sched_entity.c | 69 ++++++++++++++++++------ drivers/gpu/drm/scheduler/sched_fence.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 62 ++++++++++++++++++--- include/drm/gpu_scheduler.h | 8 +++ 4 files changed, 118 insertions(+), 23 deletions(-)
drivers/gpu/drm/scheduler/sched_entity.c +54 -15
@@ -83,6 +83,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,memset (entity,0 ,sizeof (struct drm_sched_entity ));INIT_LIST_HEAD (&entity->list); entity->rq =NULL ; + entity->single_sched =NULL ; entity->guilty = guilty; entity->num_sched_list = num_sched_list; entity->priority = priority;
@@ -90,8 +91,17 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, entity->last_scheduled =NULL ;RB_CLEAR_NODE (&entity->rb_tree_node); -if (num_sched_list) - entity->rq = &sched_list[0 ]->sched_rq[entity->priority]; +if (num_sched_list) { +if (sched_list[0 ]->sched_policy != +DRM_SCHED_POLICY_SINGLE_ENTITY ) { + entity->rq = &sched_list[0 ]->sched_rq[entity->priority]; + }else { +if (num_sched_list !=1 || sched_list[0 ]->single_entity) +return -EINVAL ; + sched_list[0 ]->single_entity = entity; + entity->single_sched = sched_list[0 ]; + } + }init_completion (&entity->entity_idle);
@@ -124,7 +134,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list) { - WARN_ON(!num_sched_list || !sched_list); + WARN_ON(!num_sched_list || !sched_list || + !!entity->single_sched); entity->sched_list = sched_list; entity->num_sched_list = num_sched_list;
@@ -194,13 +205,15 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity) {struct drm_sched_job *job;struct dma_fence *prev; +bool single_entity = !!entity->single_sched; -if (!entity->rq) +if (!entity->rq && !single_entity)return ;spin_lock (&entity->rq_lock); entity->stopped = true; -drm_sched_rq_remove_entity (entity->rq, entity); +if (!single_entity) +drm_sched_rq_remove_entity (entity->rq, entity);spin_unlock (&entity->rq_lock);/* Make sure this entity is not used by the scheduler at the moment */
@@ -222,6 +235,20 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity)dma_fence_put (prev); } +/** +* drm_sched_entity_to_scheduler - Schedule entity to GPU scheduler +* @entity: scheduler entity +* +* Returns GPU scheduler for the entity +*/ +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity) +{ +bool single_entity = !!entity->single_sched; + +return single_entity ? entity->single_sched : entity->rq->sched; +} +/** * drm_sched_entity_flush - Flush a context entity *
@@ -239,11 +266,12 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)struct drm_gpu_scheduler *sched;struct task_struct *last_user;long ret = timeout; +bool single_entity = !!entity->single_sched; -if (!entity->rq) +if (!entity->rq && !single_entity)return 0 ; - sched = entity->rq->sched; + sched =drm_sched_entity_to_scheduler (entity);/** * The client will not queue more IBs during this fini, consume existing * queued IBs or discard them on SIGKILL
@@ -336,7 +364,7 @@ static void drm_sched_entity_wakeup(struct dma_fence *f, container_of(cb, struct drm_sched_entity, cb);drm_sched_entity_clear_dep (f, cb); -drm_sched_wakeup (entity->rq->sched); +drm_sched_wakeup (drm_sched_entity_to_scheduler (entity)); }/**
@@ -350,6 +378,8 @@ static void drm_sched_entity_wakeup(struct dma_fence *f,void drm_sched_entity_set_priority (struct drm_sched_entity *entity,enum drm_sched_priority priority) { +WARN_ON (!!entity->single_sched); +spin_lock (&entity->rq_lock); entity->priority = priority;spin_unlock (&entity->rq_lock);
@@ -362,7 +392,7 @@ EXPORT_SYMBOL(drm_sched_entity_set_priority);*/ static bool drm_sched_entity_add_dependency_cb (struct drm_sched_entity *entity) { -struct drm_gpu_scheduler *sched = entity->rq->sched; +struct drm_gpu_scheduler *sched =drm_sched_entity_to_scheduler (entity);struct dma_fence *fence = entity->dependency;struct drm_sched_fence *s_fence;
@@ -455,7 +485,8 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)* Update the entity's location in the min heap according to * the timestamp of the next job, if any. */ -if (entity->rq->sched->sched_policy ==DRM_SCHED_POLICY_FIFO ) { +if (drm_sched_entity_to_scheduler (entity)->sched_policy == +DRM_SCHED_POLICY_FIFO ) {struct drm_sched_job *next; next =to_drm_sched_job (spsc_queue_peek (&entity->job_queue));
@@ -472,6 +503,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)struct drm_gpu_scheduler *sched;struct drm_sched_rq *rq; +WARN_ON (!!entity->single_sched); +/* single possible engine and already selected */ if (!entity->sched_list)return ;
@@ -521,17 +554,22 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)void drm_sched_entity_push_job (struct drm_sched_job *sched_job) {struct drm_sched_entity *entity = sched_job->entity; -bool first, fifo = entity->rq->sched->sched_policy == -DRM_SCHED_POLICY_FIFO ; +bool single_entity = !!entity->single_sched; +bool first;trace_drm_sched_job (sched_job, entity); -atomic_inc (entity->rq->sched->score); +if (!single_entity) +atomic_inc (entity->rq->sched->score);WRITE_ONCE (entity->last_user, current->group_leader); first =spsc_queue_push (&entity->job_queue, &sched_job->queue_node); sched_job->submit_ts =ktime_get ();/* first job wakes up scheduler */ if (first) { +struct drm_gpu_scheduler *sched = +drm_sched_entity_to_scheduler (entity); +bool fifo = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ; +/* Add the entity to the run queue */ spin_lock (&entity->rq_lock);if (entity->stopped) {
@@ -541,13 +579,14 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)return ; } -drm_sched_rq_add_entity (entity->rq, entity); +if (!single_entity) +drm_sched_rq_add_entity (entity->rq, entity);spin_unlock (&entity->rq_lock);if (fifo)drm_sched_rq_update_fifo (entity, sched_job->submit_ts); -drm_sched_wakeup (entity->rq->sched); +drm_sched_wakeup (sched); } }EXPORT_SYMBOL (drm_sched_entity_push_job);
drivers/gpu/drm/scheduler/sched_fence.c +1 -1
@@ -213,7 +213,7 @@ void drm_sched_fence_init(struct drm_sched_fence *fence, {unsigned seq; - fence->sched = entity->rq->sched; + fence->sched =drm_sched_entity_to_scheduler (entity); seq =atomic_inc_return (&entity->fence_seq);dma_fence_init (&fence->scheduled, &drm_sched_fence_ops_scheduled, &fence->lock, entity->fence_context, seq);
drivers/gpu/drm/scheduler/sched_main.c +55 -7
144 lines not on this page · raw message
include/drm/gpu_scheduler.h +8 -0
@@ -79,6 +79,7 @@ enum drm_sched_policy {DRM_SCHED_POLICY_DEFAULT ,DRM_SCHED_POLICY_RR ,DRM_SCHED_POLICY_FIFO , +DRM_SCHED_POLICY_SINGLE_ENTITY , DRM_SCHED_POLICY_COUNT, };
@@ -112,6 +113,9 @@ struct drm_sched_entity {*/ struct drm_sched_rq *rq; +/** @single_sched: Single scheduler */ +struct drm_gpu_scheduler *single_sched; +/** * @sched_list: *
@@ -473,6 +477,7 @@ struct drm_sched_backend_ops {* struct drm_gpu_scheduler - scheduler instance-specific data * * @ops: backend operations provided by the driver. +* @single_entity: Single entity for the scheduler * @hw_submission_limit: the max size of the hardware queue. * @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used.
@@ -503,6 +508,7 @@ struct drm_sched_backend_ops {*/ struct drm_gpu_scheduler {const struct drm_sched_backend_ops *ops; +struct drm_sched_entity *single_entity;uint32_t hw_submission_limit;long timeout;const char *name;
@@ -584,6 +590,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list, atomic_t *guilty); +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity);long drm_sched_entity_flush (struct drm_sched_entity *entity,long timeout);void drm_sched_entity_fini (struct drm_sched_entity *entity);void drm_sched_entity_destroy (struct drm_sched_entity *entity);
Signature · 1 line
2.34.1
Christian König
maintainerRe: [PATCH 3/8] drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy
Am 01.08.23 um 22:50 schrieb Matthew Brost:
Quoted 6 lines
DRM_SCHED_POLICY_SINGLE_ENTITYcreates a 1 to 1 relationship between scheduler and entity. No priorities or run queue used in this mode. Intended for devices with firmware schedulers.v2:
- Drop sched / rq union (Luben)
I think we might rather completely remove the runqueues. Essentially we only needed them for the old round robin scheduling.
For Lubens r/b priority based scheduling we can just give kernel submissions a reasonable boost instead.
Luben what do you think?
Regards,
Christian.
Quoted 415 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_entity.c | 69 ++++++++++++++++++------ drivers/gpu/drm/scheduler/sched_fence.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 62 ++++++++++++++++++--- include/drm/gpu_scheduler.h | 8 +++ 4 files changed, 118 insertions(+), 23 deletions(-)drivers/gpu/drm/scheduler/sched_entity.c +54
-15@@ -83,6 +83,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,memset (entity,0 ,sizeof (struct drm_sched_entity ));INIT_LIST_HEAD (&entity->list); entity->rq =NULL ; + entity->single_sched =NULL ; entity->guilty = guilty; entity->num_sched_list = num_sched_list; entity->priority = priority;@@ -90,8 +91,17 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, entity->last_scheduled =NULL ;RB_CLEAR_NODE (&entity->rb_tree_node); -if (num_sched_list) - entity->rq = &sched_list[0 ]->sched_rq[entity->priority]; +if (num_sched_list) { +if (sched_list[0 ]->sched_policy != +DRM_SCHED_POLICY_SINGLE_ENTITY ) { + entity->rq = &sched_list[0 ]->sched_rq[entity->priority]; + }else { +if (num_sched_list !=1 || sched_list[0 ]->single_entity) +return -EINVAL ; + sched_list[0 ]->single_entity = entity; + entity->single_sched = sched_list[0 ]; + } + }init_completion (&entity->entity_idle);@@ -124,7 +134,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list) { - WARN_ON(!num_sched_list || !sched_list); + WARN_ON(!num_sched_list || !sched_list || + !!entity->single_sched); entity->sched_list = sched_list; entity->num_sched_list = num_sched_list;@@ -194,13 +205,15 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity) {struct drm_sched_job *job;struct dma_fence *prev; +bool single_entity = !!entity->single_sched; -if (!entity->rq) +if (!entity->rq && !single_entity)return ;spin_lock (&entity->rq_lock); entity->stopped = true; -drm_sched_rq_remove_entity (entity->rq, entity); +if (!single_entity) +drm_sched_rq_remove_entity (entity->rq, entity);spin_unlock (&entity->rq_lock);/* Make sure this entity is not used by the scheduler at the moment */ @@ -222,6 +235,20 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity)dma_fence_put (prev); } +/** +* drm_sched_entity_to_scheduler - Schedule entity to GPU scheduler +* @entity: scheduler entity +* +* Returns GPU scheduler for the entity +*/ +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity) +{ +bool single_entity = !!entity->single_sched; + +return single_entity ? entity->single_sched : entity->rq->sched; +} +/** * drm_sched_entity_flush - Flush a context entity * @@ -239,11 +266,12 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)struct drm_gpu_scheduler *sched;struct task_struct *last_user;long ret = timeout; +bool single_entity = !!entity->single_sched; -if (!entity->rq) +if (!entity->rq && !single_entity)return 0 ; - sched = entity->rq->sched; + sched =drm_sched_entity_to_scheduler (entity);/** * The client will not queue more IBs during this fini, consume existing * queued IBs or discard them on SIGKILL @@ -336,7 +364,7 @@ static void drm_sched_entity_wakeup(struct dma_fence *f, container_of(cb, struct drm_sched_entity, cb);drm_sched_entity_clear_dep (f, cb); -drm_sched_wakeup (entity->rq->sched); +drm_sched_wakeup (drm_sched_entity_to_scheduler (entity)); }/** @@ -350,6 +378,8 @@ static void drm_sched_entity_wakeup(struct dma_fence *f,void drm_sched_entity_set_priority (struct drm_sched_entity *entity,enum drm_sched_priority priority) { +WARN_ON (!!entity->single_sched); +spin_lock (&entity->rq_lock); entity->priority = priority;spin_unlock (&entity->rq_lock);@@ -362,7 +392,7 @@ EXPORT_SYMBOL(drm_sched_entity_set_priority);*/ static bool drm_sched_entity_add_dependency_cb (struct drm_sched_entity *entity) { -struct drm_gpu_scheduler *sched = entity->rq->sched; +struct drm_gpu_scheduler *sched =drm_sched_entity_to_scheduler (entity);struct dma_fence *fence = entity->dependency;struct drm_sched_fence *s_fence;@@ -455,7 +485,8 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)* Update the entity's location in the min heap according to * the timestamp of the next job, if any. */ -if (entity->rq->sched->sched_policy ==DRM_SCHED_POLICY_FIFO ) { +if (drm_sched_entity_to_scheduler (entity)->sched_policy == +DRM_SCHED_POLICY_FIFO ) {struct drm_sched_job *next; next =to_drm_sched_job (spsc_queue_peek (&entity->job_queue));@@ -472,6 +503,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)struct drm_gpu_scheduler *sched;struct drm_sched_rq *rq; +WARN_ON (!!entity->single_sched); +/* single possible engine and already selected */ if (!entity->sched_list)return ;@@ -521,17 +554,22 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)void drm_sched_entity_push_job (struct drm_sched_job *sched_job) {struct drm_sched_entity *entity = sched_job->entity; -bool first, fifo = entity->rq->sched->sched_policy == -DRM_SCHED_POLICY_FIFO ; +bool single_entity = !!entity->single_sched; +bool first;trace_drm_sched_job (sched_job, entity); -atomic_inc (entity->rq->sched->score); +if (!single_entity) +atomic_inc (entity->rq->sched->score);WRITE_ONCE (entity->last_user, current->group_leader); first =spsc_queue_push (&entity->job_queue, &sched_job->queue_node); sched_job->submit_ts =ktime_get ();/* first job wakes up scheduler */ if (first) { +struct drm_gpu_scheduler *sched = +drm_sched_entity_to_scheduler (entity); +bool fifo = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ; +/* Add the entity to the run queue */ spin_lock (&entity->rq_lock);if (entity->stopped) {@@ -541,13 +579,14 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)return ; } -drm_sched_rq_add_entity (entity->rq, entity); +if (!single_entity) +drm_sched_rq_add_entity (entity->rq, entity);spin_unlock (&entity->rq_lock);if (fifo)drm_sched_rq_update_fifo (entity, sched_job->submit_ts); -drm_sched_wakeup (entity->rq->sched); +drm_sched_wakeup (sched); } }EXPORT_SYMBOL (drm_sched_entity_push_job);drivers/gpu/drm/scheduler/sched_fence.c +1
-1@@ -213,7 +213,7 @@ void drm_sched_fence_init(struct drm_sched_fence *fence, {unsigned seq; - fence->sched = entity->rq->sched; + fence->sched =drm_sched_entity_to_scheduler (entity); seq =atomic_inc_return (&entity->fence_seq);dma_fence_init (&fence->scheduled, &drm_sched_fence_ops_scheduled, &fence->lock, entity->fence_context, seq);drivers/gpu/drm/scheduler/sched_main.c +55
-7@@ -32,7 +32,8 @@* backend operations to the scheduler like submitting a job to hardware run queue, * returning the dependencies of a job etc. * -* The organisation of the scheduler is the following: +* The organisation of the scheduler is the following for scheduling policies +* DRM_SCHED_POLICY_RR and DRM_SCHED_POLICY_FIFO: * * 1. Each hw run queue has one scheduler * 2. Each scheduler has multiple run queues with different priorities @@ -42,6 +43,22 @@* the hardware. * * The jobs in a entity are always scheduled in the order that they were pushed. +* The organisation of the scheduler is the following for scheduling policy +* DRM_SCHED_POLICY_SINGLE_ENTITY: +* +* 1. One to one relationship between scheduler and entity +* 2. No priorities implemented per scheduler (single job queue) +* 3. No run queues in scheduler rather jobs are directly dequeued from entity +* 4. The entity maintains a queue of jobs that will be scheduled on the +* hardware +* +* The jobs in a entity are always scheduled in the order that they were pushed +* regardless of scheduling policy. +* +* A policy of DRM_SCHED_POLICY_RR or DRM_SCHED_POLICY_FIFO is expected to used +* when the KMD is scheduling directly on the hardware while a scheduling policy +* of DRM_SCHED_POLICY_SINGLE_ENTITY is expected to be used when there is a +* firmware scheduler. */ #include <linux/wait.h> @@ -92,6 +109,8 @@ static inline void drm_sched_rq_remove_fifo_locked(struct drm_sched_entity *entivoid drm_sched_rq_update_fifo (struct drm_sched_entity *entity,ktime_t ts) { +WARN_ON (!!entity->single_sched); +/* * Both locks need to be grabbed, one to protect from entity->rq change * for entity from within concurrent drm_sched_entity_select_rq and the @@ -122,6 +141,8 @@ void drm_sched_rq_update_fifo(struct drm_sched_entity *entity, ktime_t ts)static void drm_sched_rq_init (struct drm_gpu_scheduler *sched,struct drm_sched_rq *rq) { +WARN_ON (sched->sched_policy ==DRM_SCHED_POLICY_SINGLE_ENTITY ); +spin_lock_init (&rq->lock);INIT_LIST_HEAD (&rq->entities); rq->rb_tree_root =RB_ROOT_CACHED ;@@ -140,6 +161,8 @@ static void drm_sched_rq_init(struct drm_gpu_scheduler *sched,void drm_sched_rq_add_entity (struct drm_sched_rq *rq,struct drm_sched_entity *entity) { +WARN_ON (!!entity->single_sched); +if (!list_empty (&entity->list))return ;@@ -162,6 +185,8 @@ void drm_sched_rq_add_entity(struct drm_sched_rq *rq,void drm_sched_rq_remove_entity (struct drm_sched_rq *rq,struct drm_sched_entity *entity) { +WARN_ON (!!entity->single_sched); +if (list_empty (&entity->list))return ;@@ -667,7 +692,7 @@ int drm_sched_job_init(struct drm_sched_job *job, struct drm_sched_entity *entity, void *owner) { -if (!entity->rq) +if (!entity->rq && !entity->single_sched)return -ENOENT ; job->entity = entity;@@ -700,13 +725,16 @@ void drm_sched_job_arm(struct drm_sched_job *job) {struct drm_gpu_scheduler *sched;struct drm_sched_entity *entity = job->entity; +bool single_entity = !!entity->single_sched;BUG_ON (!entity); -drm_sched_entity_select_rq (entity); - sched = entity->rq->sched; +if (!single_entity) +drm_sched_entity_select_rq (entity); + sched =drm_sched_entity_to_scheduler (entity); job->sched = sched; - job->s_priority = entity->rq - sched->sched_rq; +if (!single_entity) + job->s_priority = entity->rq - sched->sched_rq; job->id =atomic64_inc_return (&sched->job_id_count);drm_sched_fence_init (job->s_fence, job->entity);@@ -923,6 +951,13 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)if (!drm_sched_ready (sched))return NULL ; +if (sched->single_entity) { +if (drm_sched_entity_is_ready (sched->single_entity)) +return sched->single_entity; + +return NULL ; + } +/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy == DRM_SCHED_POLICY_FIFO ?@@ -1122,6 +1157,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,return -EINVAL ; sched->ops = ops; + sched->single_entity =NULL ; sched->hw_submission_limit = hw_submission; sched->name = name; sched->run_wq = run_wq ? : system_wq;@@ -1134,7 +1170,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, sched->sched_policy = default_drm_sched_policy; else sched->sched_policy = sched_policy; -for (i =DRM_SCHED_PRIORITY_MIN ; i <DRM_SCHED_PRIORITY_COUNT ; i++) +for (i =DRM_SCHED_PRIORITY_MIN ; sched_policy != +DRM_SCHED_POLICY_SINGLE_ENTITY && i <DRM_SCHED_PRIORITY_COUNT ; + i++)drm_sched_rq_init (sched, &sched->sched_rq[i]);init_waitqueue_head (&sched->job_scheduled);@@ -1166,7 +1204,15 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)drm_sched_run_wq_stop (sched); -for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { +if (sched->single_entity) { +spin_lock (&sched->single_entity->rq_lock); + sched->single_entity->stopped = true; +spin_unlock (&sched->single_entity->rq_lock); + } + +for (i =DRM_SCHED_PRIORITY_COUNT -1 ; sched->sched_policy != +DRM_SCHED_POLICY_SINGLE_ENTITY && i >=DRM_SCHED_PRIORITY_MIN ; + i--) {struct drm_sched_rq *rq = &sched->sched_rq[i];if (!rq)@@ -1210,6 +1256,8 @@ void drm_sched_increase_karma(struct drm_sched_job *bad)struct drm_sched_entity *entity;struct drm_gpu_scheduler *sched = bad->sched; +WARN_ON (sched->sched_policy ==DRM_SCHED_POLICY_SINGLE_ENTITY ); +/* don't change @bad's karma if it's from KERNEL RQ, * because sometimes GPU hang would cause kernel jobs (like VM updating jobs) * corrupt but keep in mind that kernel jobs always considered good. include/drm/gpu_scheduler.h +8
-0@@ -79,6 +79,7 @@ enum drm_sched_policy {DRM_SCHED_POLICY_DEFAULT ,DRM_SCHED_POLICY_RR ,DRM_SCHED_POLICY_FIFO , +DRM_SCHED_POLICY_SINGLE_ENTITY , DRM_SCHED_POLICY_COUNT, };@@ -112,6 +113,9 @@ struct drm_sched_entity {*/ struct drm_sched_rq *rq; +/** @single_sched: Single scheduler */ +struct drm_gpu_scheduler *single_sched; +/** * @sched_list: * @@ -473,6 +477,7 @@ struct drm_sched_backend_ops {* struct drm_gpu_scheduler - scheduler instance-specific data * * @ops: backend operations provided by the driver. +* @single_entity: Single entity for the scheduler * @hw_submission_limit: the max size of the hardware queue. * @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. @@ -503,6 +508,7 @@ struct drm_sched_backend_ops {*/ struct drm_gpu_scheduler {const struct drm_sched_backend_ops *ops; +struct drm_sched_entity *single_entity;uint32_t hw_submission_limit;long timeout;const char *name;@@ -584,6 +590,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list, atomic_t *guilty); +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity);long drm_sched_entity_flush (struct drm_sched_entity *entity,long timeout);void drm_sched_entity_fini (struct drm_sched_entity *entity);void drm_sched_entity_destroy (struct drm_sched_entity *entity);
Matthew Brost
maintainerPATCH 4/8 drm/sched: Add generic scheduler message interface
In reply to Matthew BrostAdd generic schedule message interface which sends messages to backend from the drm_gpu_scheduler main submission thread. The idea is some of these messages modify some state in drm_sched_entity which is also modified during submission. By scheduling these messages and submission in the same thread their is not race changing states in drm_sched_entity.
This interface will be used in XE, new Intel GPU driver, to cleanup, suspend, resume, and change scheduling properties of a drm_sched_entity.
The interface is designed to be generic and extendable with only the backend understanding the messages.
--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)
drivers/gpu/drm/scheduler/sched_main.c +50 -2
@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread *
@@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))
@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;
@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }
@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);
include/drm/gpu_scheduler.h +28 -1
@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler
@@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/**
@@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished.
@@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue.
@@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;
@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);
Signature · 1 line
2.34.1
Christian König
maintainerRe: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Am 01.08.23 um 22:50 schrieb Matthew Brost:
Quoted 7 lines
Add generic schedule message interface which sends messages to backend from the
drm_gpu_schedulermain submission thread. The idea is some of these messages modify some state indrm_sched_entitywhich is also modified during submission. By scheduling these messages and submission in the same thread their is not race changing states indrm_sched_entity.
Quoted 5 lines
This interface will be used in XE, new Intel GPU driver, to cleanup, suspend, resume, and change scheduling properties of a
drm_sched_entity.The interface is designed to be generic and extendable with only the backend understanding the messages.
I'm still extremely frowned on this.
If you need this functionality then let the drivers decide which runqueue the scheduler should use.
When you then create a single threaded runqueue you can just submit work to it and serialize this with the scheduler work.
This way we wouldn't duplicate this core kernel function inside the scheduler.
Regards,
Christian.
Quoted 178 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);
Re: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Quoted 17 lines · Christian König
Am 01.08.23 um 22:50 schrieb Matthew Brost:
Quoted 7 lines
Add generic schedule message interface which sends messages to backend from the
drm_gpu_schedulermain submission thread. The idea is some of these messages modify some state indrm_sched_entitywhich is also modified during submission. By scheduling these messages and submission in the same thread their is not race changing states indrm_sched_entity.Quoted 5 lines
This interface will be used in XE, new Intel GPU driver, to cleanup, suspend, resume, and change scheduling properties of a
drm_sched_entity.The interface is designed to be generic and extendable with only the backend understanding the messages.
I'm still extremely frowned on this.
Quoted 8 lines
If you need this functionality then let the drivers decide which runqueue the scheduler should use.
When you then create a single threaded runqueue you can just submit work to it and serialize this with the scheduler work.
This way we wouldn't duplicate this core kernel function inside the scheduler.
Yeah that's essentially the design we picked for the tdr workers, where some drivers have requirements that all tdr work must be done on the same thread (because of cross-engine coordination issues). But that would require that we rework the scheduler as a pile of self-submitting work items, and I'm not sure that actually fits all that well into the core workqueue interfaces either.
Worst case I think this isn't a dead-end and can be refactored to internally use the workqueue services, with the new functions here just being dumb wrappers until everyone is converted over. So it doesn't look like an expensive mistake, if it turns out to be a mistake.
-Daniel
Quoted 182 lines
Regards,
Christian.Quoted 178 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Christian König
maintainerRe: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Am 03.08.23 um 10:58 schrieb Daniel Vetter:
Quoted 22 lines · Christian König
Quoted 21 lines
Am 01.08.23 um 22:50 schrieb Matthew Brost:
Quoted 7 lines
Add generic schedule message interface which sends messages to backend from the
drm_gpu_schedulermain submission thread. The idea is some of these messages modify some state indrm_sched_entitywhich is also modified during submission. By scheduling these messages and submission in the same thread their is not race changing states indrm_sched_entity.Quoted 5 lines
This interface will be used in XE, new Intel GPU driver, to cleanup, suspend, resume, and change scheduling properties of a
drm_sched_entity.The interface is designed to be generic and extendable with only the backend understanding the messages.
I'm still extremely frowned on this.
If you need this functionality then let the drivers decide which runqueue the scheduler should use.
When you then create a single threaded runqueue you can just submit work to it and serialize this with the scheduler work.
Quoted 8 lines
Quoted 2 lines
This way we wouldn't duplicate this core kernel function inside the scheduler.
Yeah that's essentially the design we picked for the tdr workers, where some drivers have requirements that all tdr work must be done on the same thread (because of cross-engine coordination issues). But that would require that we rework the scheduler as a pile of self-submitting work items, and I'm not sure that actually fits all that well into the core workqueue interfaces either.
There were already patches floating around which did exactly that.
Last time I checked those were actually looking pretty good.
Additional to message passing advantage the real big issue with the scheduler and 1 to 1 mapping is that we create a kernel thread for each instance, which results in tons on overhead.
Just using a work item which is submitted to a work queue completely avoids that.
Regards,
Christian.
Quoted 190 lines
Worst case I think this isn't a dead-end and can be refactored to internally use the workqueue services, with the new functions here just being dumb wrappers until everyone is converted over. So it doesn't look like an expensive mistake, if it turns out to be a mistake.
-DanielQuoted 180 lines
Regards,
Christian.Quoted 177 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);
Re: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Quoted 34 lines · Christian König
Am 03.08.23 um 10:58 schrieb Daniel Vetter:
Quoted 22 lines · Christian König
Quoted 21 lines
Am 01.08.23 um 22:50 schrieb Matthew Brost:
Quoted 7 lines
Add generic schedule message interface which sends messages to backend from the
drm_gpu_schedulermain submission thread. The idea is some of these messages modify some state indrm_sched_entitywhich is also modified during submission. By scheduling these messages and submission in the same thread their is not race changing states indrm_sched_entity.Quoted 5 lines
This interface will be used in XE, new Intel GPU driver, to cleanup, suspend, resume, and change scheduling properties of a
drm_sched_entity.The interface is designed to be generic and extendable with only the backend understanding the messages.
I'm still extremely frowned on this.
If you need this functionality then let the drivers decide which runqueue the scheduler should use.
When you then create a single threaded runqueue you can just submit work to it and serialize this with the scheduler work.
Quoted 8 lines
Quoted 2 lines
This way we wouldn't duplicate this core kernel function inside the scheduler.
Yeah that's essentially the design we picked for the tdr workers, where some drivers have requirements that all tdr work must be done on the same thread (because of cross-engine coordination issues). But that would require that we rework the scheduler as a pile of self-submitting work items, and I'm not sure that actually fits all that well into the core workqueue interfaces either.
There were already patches floating around which did exactly that.
Quoted 8 lines
Last time I checked those were actually looking pretty good.
Additional to message passing advantage the real big issue with the scheduler and 1 to 1 mapping is that we create a kernel thread for each instance, which results in tons on overhead.
Just using a work item which is submitted to a work queue completely avoids that.
Hm I should have read the entire series first, since that does the conversion still. Apologies for the confusion, and yeah we should be able to just submit other work to the same wq with the first patch? And so hand-rolling this infra here isn't needed at all?
Or what am I missing?
Quoted 194 lines
Regards,
Christian.Quoted 190 lines
Worst case I think this isn't a dead-end and can be refactored to internally use the workqueue services, with the new functions here just being dumb wrappers until everyone is converted over. So it doesn't look like an expensive mistake, if it turns out to be a mistake.
-DanielQuoted 180 lines
Regards,
Christian.Quoted 177 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);
Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Matthew Brost
maintainerRe: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Quoted 11 lines · Daniel Vetter
Quoted 10 lines · Christian König
Am 03.08.23 um 10:58 schrieb Daniel Vetter:
Quoted 9 lines · Christian König
Quoted 8 lines
Am 01.08.23 um 22:50 schrieb Matthew Brost:
Quoted 7 lines
Add generic schedule message interface which sends messages to backend from the
drm_gpu_schedulermain submission thread. The idea is some of these messages modify some state indrm_sched_entitywhich is also modified during submission. By scheduling these messages and submission in the same thread their is not race changing states indrm_sched_entity.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
This interface will be used in XE, new Intel GPU driver, to cleanup, suspend, resume, and change scheduling properties of a
drm_sched_entity.The interface is designed to be generic and extendable with only the backend understanding the messages.
Christian / Daniel - I've read both of you comments and having a hard time parsing them. I do not really understand the issue with this patch or exactly what is being suggested instead. Let's try to work through this.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I'm still extremely frowned on this.
If you need this functionality then let the drivers decide which runqueue the scheduler should use.
What do you mean by runqueue here? Do you mean 'struct workqueue_struct'? The scheduler in this context is 'struct drm_gpu_scheduler', right?
Yes, we have added this functionality iin the first patch.
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
When you then create a single threaded runqueue you can just submit work to it and serialize this with the scheduler work.
We don't want to use a single threaded workqueue_struct in Xe, we want to use a system_wq as run_job() can be executed in parallel across multiple entites (or drm_gpu_scheduler as in Xe we have 1 to 1 relationship between entity and scheduler). What we want is on per entity / scheduler granularity to be able to communicate into the backend a message synchronously (run_job / free_job not executing, scheduler execution not paused for a reset).
If I'm underatanding what you suggesting in Xe we'd create an ordered workqueue_struct per drm_gpu_scheduler and the queue messages on the ordered workqueue_struct? This seems pretty messy to me as now we have open coded a solution bypassing the scheduler, every drm_gpu_scheduler creates its own workqueue_struct, and we'd also have to open code the pausing of these messages for resets too.
IMO this is pretty clean solution that follows the pattern of cleanup jobs already in place.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 2 lines
This way we wouldn't duplicate this core kernel function inside the scheduler.
Yeah that's essentially the design we picked for the tdr workers, where some drivers have requirements that all tdr work must be done on the same thread (because of cross-engine coordination issues). But that would require that we rework the scheduler as a pile of self-submitting work items, and I'm not sure that actually fits all that well into the core workqueue interfaces either.
This is the ordering between TDRs firing between different drm_gpu_scheduler and larger external resets (GT in Xe) an ordered workqueue_struct makes sense for this. Here we are talking about ordering jobs and messages within a single drm_gpu_scheduler. Using the main execution thread to do ordering makes sense in my opinion.
Quoted 5 lines
Quoted 5 lines
There were already patches floating around which did exactly that.
Last time I checked those were actually looking pretty good.
Link to patches for reference.
Quoted 3 lines
Quoted 3 lines
Additional to message passing advantage the real big issue with the scheduler and 1 to 1 mapping is that we create a kernel thread for each instance, which results in tons on overhead.
First patch in the series switches from kthread to work queue, that is still a good idea.
Quoted 9 lines
Quoted 3 lines
Just using a work item which is submitted to a work queue completely avoids that.
Hm I should have read the entire series first, since that does the conversion still. Apologies for the confusion, and yeah we should be able to just submit other work to the same wq with the first patch? And so hand-rolling this infra here isn't needed at all?
I wouldn't call this hand rolling, rather it following patten already in place.
Matt
Quoted 201 lines
Or what am I missing?
Quoted 194 lines
Regards,
Christian.Quoted 190 lines
Worst case I think this isn't a dead-end and can be refactored to internally use the workqueue services, with the new functions here just being dumb wrappers until everyone is converted over. So it doesn't look like an expensive mistake, if it turns out to be a mistake.
-DanielQuoted 180 lines
Regards,
Christian.Quoted 177 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Christian König
maintainerRe: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Am 04.08.23 um 16:13 schrieb Matthew Brost:
Quoted 6 lines
[SNIP]
Christian / Daniel - I've read both of you comments and having a hard time parsing them. I do not really understand the issue with this patch or exactly what is being suggested instead. Let's try to work through this.
Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I'm still extremely frowned on this.
If you need this functionality then let the drivers decide which runqueue the scheduler should use.
What do you mean by runqueue here? Do you mean 'struct
workqueue_struct'? The scheduler in this context is 'structdrm_gpu_scheduler', right?
Sorry for the confusing wording, your understanding is correct.
Quoted 13 lines
Yes, we have added this functionality iin the first patch.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
When you then create a single threaded runqueue you can just submit work to it and serialize this with the scheduler work.
We don't want to use a single threaded
workqueue_structin Xe, we want to use asystem_wqasrun_job()can be executed in parallel across multiple entites (ordrm_gpu_scheduleras in Xe we have 1 to 1 relationship between entity and scheduler). What we want is on per entity / scheduler granularity to be able to communicate into the backend a message synchronously (run_job/free_jobnot executing, scheduler execution not paused for a reset).
Quoted 3 lines
If I'm underatanding what you suggesting in Xe we'd create an ordered
workqueue_structperdrm_gpu_schedulerand the queue messages on the orderedworkqueue_struct?
Yes, correct.
Quoted 7 lines
This seems pretty messy to me as now we have
open coded a solution bypassing the scheduler, everydrm_gpu_schedulercreates its ownworkqueue_struct, and we'd also have to open code the pausing of these messages for resets too.IMO this is pretty clean solution that follows the pattern of cleanup jobs already in place.
Yeah, exactly that's the point. Moving the job cleanup into the scheduler thread is seen as very very bad idea by me.
And I really don't want to exercise that again for different use cases.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
This way we wouldn't duplicate this core kernel function inside the scheduler.
Yeah that's essentially the design we picked for the tdr workers, where some drivers have requirements that all tdr work must be done on the same thread (because of cross-engine coordination issues). But that would require that we rework the scheduler as a pile of self-submitting work items, and I'm not sure that actually fits all that well into the core workqueue interfaces either.
Quoted 5 lines
This is the ordering between TDRs firing between different
drm_gpu_schedulerand larger external resets (GT in Xe) an orderedworkqueue_structmakes sense for this. Here we are talking about ordering jobs and messages within a singledrm_gpu_scheduler. Using the main execution thread to do ordering makes sense in my opinion.
I completely disagree to that.
Take a look at how this came to be. This is a very very ugly hack and we already had a hard time making lockdep understand the different fence signaling dependencies with freeing the job and I'm pretty sure that is still not 100% correct.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
There were already patches floating around which did exactly that.
Last time I checked those were actually looking pretty good.
Quoted 7 lines
Link to patches for reference.
Quoted 3 lines
Quoted 3 lines
Additional to message passing advantage the real big issue with the scheduler and 1 to 1 mapping is that we create a kernel thread for each instance, which results in tons on overhead.
First patch in the series switches from kthread to work queue, that is still a good idea.
This was the patch I was referring to. Sorry didn't remembered that this was in the same patch set.
Quoted 10 lines
Quoted 8 lines
Quoted 3 lines
Just using a work item which is submitted to a work queue completely avoids that.
Hm I should have read the entire series first, since that does the conversion still. Apologies for the confusion, and yeah we should be able to just submit other work to the same wq with the first patch? And so hand-rolling this infra here isn't needed at all?
I wouldn't call this hand rolling, rather it following patten already in place.
Basically workqueues are the in kernel infrastructure for exactly that use case and we are trying to re-create that here and that is usually a rather bad idea.
Regards,
Christian.
Quoted 200 lines
Matt
Quoted 197 lines
Or what am I missing?
Quoted 191 lines
Regards,
Christian.Quoted 188 lines
Worst case I think this isn't a dead-end and can be refactored to internally use the workqueue services, with the new functions here just being dumb wrappers until everyone is converted over. So it doesn't look like an expensive mistake, if it turns out to be a mistake.
-DanielQuoted 180 lines
Regards,
Christian.Quoted 177 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Matthew Brost
maintainerRe: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Quoted 95 lines · Christian König
Am 04.08.23 um 16:13 schrieb Matthew Brost:
Quoted 6 lines
[SNIP]
Christian / Daniel - I've read both of you comments and having a hard time parsing them. I do not really understand the issue with this patch or exactly what is being suggested instead. Let's try to work through this.Quoted 7 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
Quoted 4 lines
I'm still extremely frowned on this.
If you need this functionality then let the drivers decide which runqueue the scheduler should use.
What do you mean by runqueue here? Do you mean 'struct
workqueue_struct'? The scheduler in this context is 'structdrm_gpu_scheduler', right?Sorry for the confusing wording, your understanding is correct.
Quoted 13 lines
Yes, we have added this functionality iin the first patch.
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
Quoted 3 lines
When you then create a single threaded runqueue you can just submit work to it and serialize this with the scheduler work.
We don't want to use a single threaded
workqueue_structin Xe, we want to use asystem_wqasrun_job()can be executed in parallel across multiple entites (ordrm_gpu_scheduleras in Xe we have 1 to 1 relationship between entity and scheduler). What we want is on per entity / scheduler granularity to be able to communicate into the backend a message synchronously (run_job/free_jobnot executing, scheduler execution not paused for a reset).Quoted 3 lines
If I'm underatanding what you suggesting in Xe we'd create an ordered
workqueue_structperdrm_gpu_schedulerand the queue messages on the orderedworkqueue_struct?Yes, correct.
Quoted 7 lines
This seems pretty messy to me as now we have
open coded a solution bypassing the scheduler, everydrm_gpu_schedulercreates its ownworkqueue_struct, and we'd also have to open code the pausing of these messages for resets too.IMO this is pretty clean solution that follows the pattern of cleanup jobs already in place.
Yeah, exactly that's the point. Moving the job cleanup into the scheduler thread is seen as very very bad idea by me.
And I really don't want to exercise that again for different use cases.
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
Quoted 3 lines
This way we wouldn't duplicate this core kernel function inside the scheduler.
Yeah that's essentially the design we picked for the tdr workers, where some drivers have requirements that all tdr work must be done on the same thread (because of cross-engine coordination issues). But that would require that we rework the scheduler as a pile of self-submitting work items, and I'm not sure that actually fits all that well into the core workqueue interfaces either.
Quoted 5 lines
This is the ordering between TDRs firing between different
drm_gpu_schedulerand larger external resets (GT in Xe) an orderedworkqueue_structmakes sense for this. Here we are talking about ordering jobs and messages within a singledrm_gpu_scheduler. Using the main execution thread to do ordering makes sense in my opinion.I completely disagree to that.
Take a look at how this came to be. This is a very very ugly hack and we already had a hard time making lockdep understand the different fence signaling dependencies with freeing the job and I'm pretty sure that is still not 100% correct.
Quoted 5 lines
Quoted 5 lines
Quoted 5 lines
There were already patches floating around which did exactly that.
Last time I checked those were actually looking pretty good.
Quoted 7 lines
Link to patches for reference.
Quoted 3 lines
Quoted 3 lines
Additional to message passing advantage the real big issue with the scheduler and 1 to 1 mapping is that we create a kernel thread for each instance, which results in tons on overhead.
First patch in the series switches from kthread to work queue, that is still a good idea.
This was the patch I was referring to. Sorry didn't remembered that this was in the same patch set.
Quoted 8 lines
Quoted 8 lines
Quoted 3 lines
Just using a work item which is submitted to a work queue completely avoids that.
Hm I should have read the entire series first, since that does the conversion still. Apologies for the confusion, and yeah we should be able to just submit other work to the same wq with the first patch? And so hand-rolling this infra here isn't needed at all?
Quoted 7 lines
Quoted 2 lines
I wouldn't call this hand rolling, rather it following patten already in place.
Basically workqueues are the in kernel infrastructure for exactly that use case and we are trying to re-create that here and that is usually a rather bad idea.
Ok let me play around with what this would look like in Xe, what you are suggesting would be ordered-wq per scheduler, work item for run job, work item for clean up job, and work item for a message. That might work I suppose? Only issue I see is scaling as this exposes an ordered-wq creation directly to an IOCTL. No idea if that is actually a concern though.
Matt
Quoted 204 lines
Regards,
Christian.Quoted 200 lines
Matt
Quoted 197 lines
Or what am I missing?
Quoted 191 lines
Regards,
Christian.Quoted 188 lines
Worst case I think this isn't a dead-end and can be refactored to internally use the workqueue services, with the new functions here just being dumb wrappers until everyone is converted over. So it doesn't look like an expensive mistake, if it turns out to be a mistake.
-DanielQuoted 180 lines
Regards,
Christian.Quoted 177 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Christian König
maintainerRe: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Am 08.08.23 um 16:06 schrieb Matthew Brost:
Quoted 11 lines
[SNIP]
Quoted 4 lines
Basically workqueues are the in kernel infrastructure for exactly that use case and we are trying to re-create that here and that is usually a rather bad idea.
Ok let me play around with what this would look like in Xe, what you are suggesting would be ordered-wq per scheduler, work item for run job, work item for clean up job, and work item for a message. That might work I suppose? Only issue I see is scaling as this exposes an ordered-wq creation directly to an IOCTL. No idea if that is actually a concern though.
That's a very good question I can't answer of hand either.
But from the history of work queues I know that they were invented to reduce the overhead/costs of having many kernel threads.
So my educated guess is that you probably won't find anything better at the moment. If work queues then indeed don't match this use case then we need to figure out how to improve them or find a different solution.
Christian.
Quoted 205 lines
Matt
Quoted 202 lines
Regards,
Christian.Quoted 199 lines
Matt
Quoted 197 lines
Or what am I missing?
Quoted 191 lines
Regards,
Christian.Quoted 188 lines
Worst case I think this isn't a dead-end and can be refactored to internally use the workqueue services, with the new functions here just being dumb wrappers until everyone is converted over. So it doesn't look like an expensive mistake, if it turns out to be a mistake.
-DanielQuoted 180 lines
Regards,
Christian.Quoted 177 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Matthew Brost
maintainerRe: [PATCH 4/8] drm/sched: Add generic scheduler message interface
Quoted 13 lines · Christian König
Am 08.08.23 um 16:06 schrieb Matthew Brost:
Quoted 11 lines
[SNIP]
Quoted 4 lines
Basically workqueues are the in kernel infrastructure for exactly that use case and we are trying to re-create that here and that is usually a rather bad idea.
Ok let me play around with what this would look like in Xe, what you are suggesting would be ordered-wq per scheduler, work item for run job, work item for clean up job, and work item for a message. That might work I suppose? Only issue I see is scaling as this exposes an ordered-wq creation directly to an IOCTL. No idea if that is actually a concern though.
Quoted 9 lines
That's a very good question I can't answer of hand either.
But from the history of work queues I know that they were invented to reduce the overhead/costs of having many kernel threads.
So my educated guess is that you probably won't find anything better at the moment. If work queues then indeed don't match this use case then we need to figure out how to improve them or find a different solution.
I looked at workqueue code and think the workqueue creation is decoupled from kthread creation in most cases so I think this fits.
Hacked together a quick PoC of this on top of Xe, seems to work, and like how the code looks too. Need to clean it up a bit and run some perf tests but looks promising. Hopely it all comes together and can get another spin of this series out fairly soon.
Matt
Quoted 208 lines
Christian.
Quoted 205 lines
Matt
Quoted 202 lines
Regards,
Christian.Quoted 199 lines
Matt
Quoted 197 lines
Or what am I missing?
Quoted 191 lines
Regards,
Christian.Quoted 188 lines
Worst case I think this isn't a dead-end and can be refactored to internally use the workqueue services, with the new functions here just being dumb wrappers until everyone is converted over. So it doesn't look like an expensive mistake, if it turns out to be a mistake.
-DanielQuoted 180 lines
Regards,
Christian.Quoted 177 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 52 +++++++++++++++++++++++++- include/drm/gpu_scheduler.h | 29 +++++++++++++- 2 files changed, 78 insertions(+), 3 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +50
-2@@ -1049,6 +1049,49 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, }EXPORT_SYMBOL (drm_sched_pick_best); +/** +* drm_sched_add_msg - add scheduler message +* +* @sched: scheduler instance +* @msg: message to be added +* +* Can and will pass an jobs waiting on dependencies or in a runnable queue. +* Messages processing will stop if schedule run wq is stopped and resume when +* run wq is started. +*/ +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg) +{ +spin_lock (&sched->job_list_lock); +list_add_tail (&msg->link, &sched->msgs); +spin_unlock (&sched->job_list_lock); + +drm_sched_run_wq_queue (sched); +} +EXPORT_SYMBOL (drm_sched_add_msg); + +/** +* drm_sched_get_msg - get scheduler message +* +* @sched: scheduler instance +* +* Returns NULL or message +*/ +static struct drm_sched_msg * +drm_sched_get_msg (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_msg *msg; + +spin_lock (&sched->job_list_lock); + msg =list_first_entry_or_null (&sched->msgs, + struct drm_sched_msg, link); +if (msg) +list_del (&msg->link); +spin_unlock (&sched->job_list_lock); + +return msg; +} +/** * drm_sched_main - main scheduler thread * @@ -1060,6 +1103,7 @@ static void drm_sched_main(struct work_struct *w) container_of(w, struct drm_gpu_scheduler, work_run);struct drm_sched_entity *entity;struct drm_sched_job *cleanup_job; +struct drm_sched_msg *msg;int r;if (READ_ONCE (sched->pause_run_wq))@@ -1067,12 +1111,15 @@ static void drm_sched_main(struct work_struct *w) cleanup_job =drm_sched_get_cleanup_job (sched); entity =drm_sched_select_entity (sched); + msg =drm_sched_get_msg (sched); -if (!entity && !cleanup_job) +if (!entity && !cleanup_job && !msg)return ;/* No more work */ if (cleanup_job) sched->ops->free_job (cleanup_job); +if (msg) + sched->ops->process_msg (msg);if (entity) {struct dma_fence *fence;@@ -1082,7 +1129,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) +if (!cleanup_job && !msg)return ;/* No more work */ goto again; }@@ -1177,6 +1224,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,init_waitqueue_head (&sched->job_scheduled);INIT_LIST_HEAD (&sched->pending_list); +INIT_LIST_HEAD (&sched->msgs);spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout);include/drm/gpu_scheduler.h +28
-1@@ -394,6 +394,23 @@ enum drm_gpu_sched_stat { DRM_GPU_SCHED_STAT_ENODEV, }; +/** +* struct drm_sched_msg - an in-band (relative to GPU scheduler run queue) +* message +* +* Generic enough for backend defined messages, backend can expand if needed. +*/ +struct drm_sched_msg { +/** @link: list link into the gpu scheduler list of messages */ +struct list_head link; +/** +* @private_data: opaque pointer to message private data (backend defined) +*/ +void *private_data; +/** @opcode: opcode of message (backend defined) */ +unsigned int opcode; +}; +/** * struct drm_sched_backend_ops - Define the backend operations * called by the scheduler @@ -471,6 +488,12 @@ struct drm_sched_backend_ops {* and it's time to clean it up. */ void (*free_job)(struct drm_sched_job *sched_job); + +/** +* @process_msg: Process a message. Allowed to block, it is this +* function's responsibility to free message if dynamically allocated. +*/ +void (*process_msg)(struct drm_sched_msg *msg); };/** @@ -482,6 +505,7 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. +* @msgs: list of messages to be processed in @work_run * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. @@ -489,7 +513,7 @@ struct drm_sched_backend_ops {* @job_id_count: used to assign unique id to the each job. * @run_wq: workqueue used to queue @work_run * @timeout_wq: workqueue used to queue @work_tdr -* @work_run: schedules jobs and cleans up entities +* @work_run: schedules jobs, cleans up jobs, and processes messages * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -513,6 +537,7 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; +struct list_head msgs;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count;@@ -566,6 +591,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); +void drm_sched_add_msg (struct drm_gpu_scheduler *sched, +struct drm_sched_msg *msg);void drm_sched_run_wq_stop (struct drm_gpu_scheduler *sched);void drm_sched_run_wq_start (struct drm_gpu_scheduler *sched);void drm_sched_stop (struct drm_gpu_scheduler *sched,struct drm_sched_job *bad);Signature · 3 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch
Matthew Brost
maintainerPATCH 5/8 drm/sched: Add drm_sched_start_timeout_unlocked helper
In reply to Matthew BrostAlso add a lockdep assert to drm_sched_start_timeout.
--- drivers/gpu/drm/scheduler/sched_main.c | 23 +++++++++++++---------- 1 file changed, 13 insertions(+), 10 deletions(-)
drivers/gpu/drm/scheduler/sched_main.c +13 -10
@@ -360,11 +360,20 @@ static void drm_sched_job_done_cb(struct dma_fence *f, struct dma_fence_cb *cb)*/ static void drm_sched_start_timeout (struct drm_gpu_scheduler *sched) { +lockdep_assert_held (&sched->job_list_lock); +if (sched->timeout !=MAX_SCHEDULE_TIMEOUT && !list_empty (&sched->pending_list))queue_delayed_work (sched->timeout_wq, &sched->work_tdr, sched->timeout); } +static void drm_sched_start_timeout_unlocked (struct drm_gpu_scheduler *sched) +{ +spin_lock (&sched->job_list_lock); +drm_sched_start_timeout (sched); +spin_unlock (&sched->job_list_lock); +} +/** * drm_sched_fault - immediately start timeout handler *
@@ -476,11 +485,8 @@ static void drm_sched_job_timedout(struct work_struct *work)spin_unlock (&sched->job_list_lock); } -if (status !=DRM_GPU_SCHED_STAT_ENODEV ) { -spin_lock (&sched->job_list_lock); -drm_sched_start_timeout (sched); -spin_unlock (&sched->job_list_lock); - } +if (status !=DRM_GPU_SCHED_STAT_ENODEV ) +drm_sched_start_timeout_unlocked (sched); }/**
@@ -606,11 +612,8 @@ void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery)drm_sched_job_done (s_job); } -if (full_recovery) { -spin_lock (&sched->job_list_lock); -drm_sched_start_timeout (sched); -spin_unlock (&sched->job_list_lock); - } +if (full_recovery) +drm_sched_start_timeout_unlocked (sched);drm_sched_run_wq_start (sched); }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 6/8 drm/sched: Start run wq before TDR in drm_sched_start
In reply to Matthew BrostIf the TDR is set to a very small value it can fire before the run wq is started in the function drm_sched_start. The run wq is expected to running when the TDR fires, fix this ordering so this expectation is always met.
--- drivers/gpu/drm/scheduler/sched_main.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/scheduler/sched_main.c +2 -2
@@ -612,10 +612,10 @@ void drm_sched_start(struct drm_gpu_scheduler *sched, bool full_recovery)drm_sched_job_done (s_job); } +drm_sched_run_wq_start (sched); +if (full_recovery)drm_sched_start_timeout_unlocked (sched); - -drm_sched_run_wq_start (sched); }EXPORT_SYMBOL (drm_sched_start);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 7/8 drm/sched: Submit job before starting TDR
In reply to Matthew BrostIf the TDR is set to a value, it can fire before a job is submitted in drm_sched_main. The job should be always be submitted before the TDR fires, fix this ordering.
v2:
- Add to pending list before run_job, start TDR after (Luben, Boris)
--- drivers/gpu/drm/scheduler/sched_main.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/scheduler/sched_main.c +1 -1
@@ -445,7 +445,6 @@ static void drm_sched_job_begin(struct drm_sched_job *s_job)spin_lock (&sched->job_list_lock);list_add_tail (&s_job->list, &sched->pending_list); -drm_sched_start_timeout (sched);spin_unlock (&sched->job_list_lock); }
@@ -1146,6 +1145,7 @@ static void drm_sched_main(struct work_struct *w) fence = sched->ops->run_job (sched_job);complete_all (&entity->entity_idle);drm_sched_fence_scheduled (s_fence); +drm_sched_start_timeout_unlocked (sched);if (!IS_ERR_OR_NULL (fence)) {drm_sched_fence_set_parent (s_fence, fence);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH 8/8 drm/sched: Add helper to set TDR timeout
In reply to Matthew BrostAdd helper to set TDR timeout and restart the TDR with new timeout value. This will be used in XE, new Intel GPU driver, to trigger the TDR to cleanup drm_sched_entity that encounter errors.
--- drivers/gpu/drm/scheduler/sched_main.c | 18 ++++++++++++++++++ include/drm/gpu_scheduler.h | 1 + 2 files changed, 19 insertions(+)
drivers/gpu/drm/scheduler/sched_main.c +18 -0
@@ -374,6 +374,24 @@ static void drm_sched_start_timeout_unlocked(struct drm_gpu_scheduler *sched)spin_unlock (&sched->job_list_lock); } +/** +* drm_sched_set_timeout - set timeout for reset worker +* +* @sched: scheduler instance to set and (re)-start the worker for +* @timeout: timeout period +* +* Set and (re)-start the timeout for the given scheduler. +*/ +void drm_sched_set_timeout (struct drm_gpu_scheduler *sched,long timeout) +{ +spin_lock (&sched->job_list_lock); + sched->timeout = timeout; +cancel_delayed_work (&sched->work_tdr); +drm_sched_start_timeout (sched); +spin_unlock (&sched->job_list_lock); +} +EXPORT_SYMBOL (drm_sched_set_timeout); +/** * drm_sched_fault - immediately start timeout handler *
include/drm/gpu_scheduler.h +1 -0
@@ -589,6 +589,7 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list); +void drm_sched_set_timeout (struct drm_gpu_scheduler *sched,long timeout);void drm_sched_job_cleanup (struct drm_sched_job *job);void drm_sched_wakeup (struct drm_gpu_scheduler *sched); void drm_sched_add_msg(struct drm_gpu_scheduler *sched,
Signature · 1 line
2.34.1
Patchwork
CI[Intel-xe] ✗ CI.Patch_applied: failure for DRM scheduler changes for Xe
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: DRM scheduler changes for Xe
URL : https://patchwork.freedesktop.org/series/121744/
State : failure
== Summary ==
=== Applying kernel patches on branch 'drm-xe-next' with base: ===
Base commit: 22bed6ae7 drm/xe: remove header variable from parse_g2h_msg
=== git am output follows ===
error: patch failed: drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c:1489
error: drivers/gpu/drm/amd/amdgpu/amdgpu_debugfs.c: patch does not apply
error: patch failed: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c:2364
error: drivers/gpu/drm/amd/amdgpu/amdgpu_device.c: patch does not apply
error: patch failed: drivers/gpu/drm/etnaviv/etnaviv_sched.c:133
error: drivers/gpu/drm/etnaviv/etnaviv_sched.c: patch does not apply
error: patch failed: drivers/gpu/drm/lima/lima_sched.c:488
error: drivers/gpu/drm/lima/lima_sched.c: patch does not apply
error: patch failed: drivers/gpu/drm/msm/adreno/adreno_device.c:662
error: drivers/gpu/drm/msm/adreno/adreno_device.c: patch does not apply
error: patch failed: drivers/gpu/drm/msm/msm_ringbuffer.c:95
error: drivers/gpu/drm/msm/msm_ringbuffer.c: patch does not apply
error: patch failed: drivers/gpu/drm/panfrost/panfrost_job.c:815
error: drivers/gpu/drm/panfrost/panfrost_job.c: patch does not apply
error: patch failed: drivers/gpu/drm/scheduler/sched_main.c:44
error: drivers/gpu/drm/scheduler/sched_main.c: patch does not apply
error: patch failed: drivers/gpu/drm/v3d/v3d_sched.c:388
error: drivers/gpu/drm/v3d/v3d_sched.c: patch does not apply
error: patch failed: include/drm/gpu_scheduler.h:473
error: include/drm/gpu_scheduler.h: patch does not apply
hint: Use 'git am --show-current-patch' to see the failed patch
Applying: drm/sched: Convert drm scheduler to use a work queue rather than kthread Patch failed at 0001 drm/sched: Convert drm scheduler to use a work queue rather than kthread When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".