DRM scheduler changes for Xe
Subsystems (11)
Matthew Brost
maintainerv2 cover DRM scheduler changes for Xe
As a prerequisite to merging the new Intel Xe DRM driver [1] [2], we have been asked to merge our common DRM scheduler patches first.
This a continuation of a RFC [3] with all comments addressed, ready for a full review, and hopefully in state which can merged in the near future. More details of this series can found in the cover letter of the RFC [3].
These changes have been tested with the Xe driver.
v2:
- Break run job, free job, and process message in own work items
- This might break other drivers as run job and free job now can run in parallel, can fix up if needed
Matt
Matthew Brost (9):
drm/sched: Convert drm scheduler to use a work queue rather than
kthread
drm/sched: Move schedule policy to scheduler / entity drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy drm/sched: Split free_job into own work item drm/sched: Add generic scheduler message interface drm/sched: Add drm_sched_start_timeout_unlocked helper drm/sched: Start run wq before TDR in drm_sched_start drm/sched: Submit job before starting TDR
drm/sched: Add helper to set TDR timeout
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 3 +- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 5 +- drivers/gpu/drm/lima/lima_sched.c | 5 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 5 +- drivers/gpu/drm/nouveau/nouveau_sched.c | 5 +- drivers/gpu/drm/panfrost/panfrost_job.c | 5 +- drivers/gpu/drm/scheduler/sched_entity.c | 85 ++++- drivers/gpu/drm/scheduler/sched_fence.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 408 ++++++++++++++++----- drivers/gpu/drm/v3d/v3d_sched.c | 25 +- include/drm/gpu_scheduler.h | 75 +++- 11 files changed, 487 insertions(+), 136 deletions(-)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v2 1/9 drm/sched: Convert drm scheduler to use a work queue rather than kthread
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a drm_gpu_scheduler and drm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.
1. In XE the submission order from multiple drm_sched_entity is not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shared drm_gpu_scheduler across multiple drm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicated drm_gpu_scheduler per drm_sched_entity solve this problem.
2. In XE submissions are done via programming a ring buffer (circular buffer), a drm_gpu_scheduler provides a limit on number of jobs, if the limit of number jobs is set to RING_SIZE / MAX_SIZE_PER_JOB we get flow control on the ring for free.
A problem with this design is currently a drm_gpu_scheduler uses a kthread for submission / job cleanup. This doesn't scale if a large number of drm_gpu_scheduler are used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.
v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patch
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 2 +- drivers/gpu/drm/etnaviv/etnaviv_sched.c | 2 +- drivers/gpu/drm/lima/lima_sched.c | 2 +- drivers/gpu/drm/msm/msm_ringbuffer.c | 2 +- drivers/gpu/drm/nouveau/nouveau_sched.c | 2 +- drivers/gpu/drm/panfrost/panfrost_job.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 106 +++++++++------------ drivers/gpu/drm/v3d/v3d_sched.c | 10 +- include/drm/gpu_scheduler.h | 12 ++- 9 files changed, 65 insertions(+), 75 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -1
@@ -2331,7 +2331,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev)break ; } - r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, + r = drm_sched_init(&ring->sched, &amdgpu_sched_ops, NULL, ring->num_hw_submission, 0, timeout, adev->reset_domain->wq, ring->sched_score, ring->name,
drivers/gpu/drm/etnaviv/etnaviv_sched.c +1 -1
@@ -133,7 +133,7 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu) {int ret; - ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops, + ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops,NULL , etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,msecs_to_jiffies (500 ),NULL ,NULL ,dev_name (gpu->dev), gpu->dev);
drivers/gpu/drm/lima/lima_sched.c +1 -1
@@ -488,7 +488,7 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)INIT_WORK (&pipe->recover_work, lima_sched_recover_work); -return drm_sched_init (&pipe->base, &lima_sched_ops,1 , +return drm_sched_init (&pipe->base, &lima_sched_ops,NULL ,1 , lima_job_hang_limit,msecs_to_jiffies (timeout),NULL ,NULL , name, pipe->ldev->dev);
drivers/gpu/drm/msm/msm_ringbuffer.c +1 -1
@@ -93,7 +93,7 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id,/* currently managing hangcheck ourselves: */ sched_timeout =MAX_SCHEDULE_TIMEOUT ; - ret =drm_sched_init (&ring->sched, &msm_sched_ops, + ret =drm_sched_init (&ring->sched, &msm_sched_ops,NULL , num_hw_submissions,0 , sched_timeout,NULL ,NULL ,to_msm_bo (ring->bo)->name, gpu->dev->dev);if (ret) {
drivers/gpu/drm/nouveau/nouveau_sched.c +1 -1
@@ -407,7 +407,7 @@ int nouveau_sched_init(struct nouveau_drm *drm)if (!drm->sched_wq)return -ENOMEM ; -return drm_sched_init (sched, &nouveau_sched_ops, +return drm_sched_init (sched, &nouveau_sched_ops,NULL ,NOUVEAU_SCHED_HW_SUBMISSIONS ,0 , job_hang_limit,NULL ,NULL ,"nouveau_sched" , drm->dev->dev); }
drivers/gpu/drm/panfrost/panfrost_job.c +1 -1
@@ -831,7 +831,7 @@ int panfrost_job_init(struct panfrost_device *pfdev) js->queue[j].fence_context =dma_fence_context_alloc (1 ); ret = drm_sched_init(&js->queue[j].sched, - &panfrost_sched_ops, + &panfrost_sched_ops, NULL, nentries, 0, msecs_to_jiffies(JOB_TIMEOUT_MS), pfdev->reset.wq,
drivers/gpu/drm/scheduler/sched_main.c +47 -59
213 lines not on this page · raw message
drivers/gpu/drm/v3d/v3d_sched.c +5 -5
@@ -388,7 +388,7 @@ v3d_sched_init(struct v3d_dev *v3d)int ret; ret =drm_sched_init (&v3d->queue[V3D_BIN ].sched, - &v3d_bin_sched_ops, + &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_bin" , v3d->drm.dev);
@@ -396,7 +396,7 @@ v3d_sched_init(struct v3d_dev *v3d)return ret; ret =drm_sched_init (&v3d->queue[V3D_RENDER ].sched, - &v3d_render_sched_ops, + &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_render" , v3d->drm.dev);
@@ -404,7 +404,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_TFU ].sched, - &v3d_tfu_sched_ops, + &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_tfu" , v3d->drm.dev);
@@ -413,7 +413,7 @@ v3d_sched_init(struct v3d_dev *v3d)if (v3d_has_csd (v3d)) { ret =drm_sched_init (&v3d->queue[V3D_CSD ].sched, - &v3d_csd_sched_ops, + &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_csd" , v3d->drm.dev);
@@ -421,7 +421,7 @@ v3d_sched_init(struct v3d_dev *v3d) goto fail; ret =drm_sched_init (&v3d->queue[V3D_CACHE_CLEAN ].sched, - &v3d_cache_clean_sched_ops, + &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL ,NULL ,"v3d_cache_clean" , v3d->drm.dev);
include/drm/gpu_scheduler.h +7 -5
@@ -473,17 +473,16 @@ struct drm_sched_backend_ops {* @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. * @sched_rq: priority wise array of run queues. -* @wake_up_worker: the wait queue on which the scheduler sleeps until a job -* is ready to be scheduled. * @job_scheduled: once @drm_sched_entity_do_release is called the scheduler * waits on this wait queue until all the scheduled jobs are * finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. +* @submit_wq: workqueue used to queue @work_submit * @timeout_wq: workqueue used to queue @work_tdr +* @work_submit: schedules jobs and cleans up entities * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. -* @thread: the kthread on which the scheduler which run. * @pending_list: the list of jobs which are currently in the job queue. * @job_list_lock: lock to protect the pending_list. * @hang_limit: once the hangs by a job crosses this limit then it is marked
@@ -492,6 +491,7 @@ struct drm_sched_backend_ops {* @_score: score used when the driver doesn't provide one * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. +* @pause_submit: pause queuing of @work_submit on @submit_wq * @dev: system &struct device * * One scheduler is implemented for each hardware ring.
@@ -502,13 +502,13 @@ struct drm_gpu_scheduler {long timeout;const char *name;struct drm_sched_rq sched_rq[DRM_SCHED_PRIORITY_COUNT ]; -wait_queue_head_t wake_up_worker;wait_queue_head_t job_scheduled;atomic_t hw_rq_count;atomic64_t job_id_count; +struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; +struct work_struct work_submit;struct delayed_work work_tdr; -struct task_struct *thread;struct list_head pending_list;spinlock_t job_list_lock;int hang_limit;
@@ -516,11 +516,13 @@ struct drm_gpu_scheduler {atomic_t _score;bool ready;bool free_guilty; +bool pause_submit;struct device *dev; };int drm_sched_init (struct drm_gpu_scheduler *sched,const struct drm_sched_backend_ops *ops, +struct workqueue_struct *submit_wq,uint32_t hw_submission,unsigned hang_limit,long timeout,struct workqueue_struct *timeout_wq,atomic_t *score,const char *name,struct device *dev);
Signature · 1 line
2.34.1
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Hi Matt,
Quoted 12 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.
Quoted 4 lines
2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.
In XE, where does the limitation of MAX_SIZE_PER_JOB come from?
In Nouveau we currently do have such a limitation as well, but it is derived from the RING_SIZE, hence RING_SIZE / MAX_SIZE_PER_JOB would always be 1. However, I think most jobs won't actually utilize the whole ring.
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing hw_submission_limit.
- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 15 lines
Hi Matt,
Quoted 12 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.
Quoted 8 lines
Quoted 4 lines
2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.
Well that should probably rather be RING_SIZE / MAX_SIZE_PER_JOB = hw_submission_limit (or even hw_submission_limit - 1 when the hw can't distinct full vs empty ring buffer).
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 21 lines · Christian König
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 20 lines
Hi Matt,
Quoted 16 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.
Quoted 5 lines
Quoted 3 lines
In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).
Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau MAX_SIZE_PER_JOB isn't really limited by anything other than the RING_SIZE and hence we'd never allow more than 1 active job.
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
Quoted 29 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Matthew Brost
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 17 lines · Danilo Krummrich
Quoted 16 lines · Christian König
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 15 lines
Hi Matt,
Quoted 12 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?
In Xe the job submission is series of ring instructions done by the KMD. The instructions are cache flushes, seqno writes, jump to user BB, etc... The exact instructions for each job vary based on hw engine type, platform, etc... We dervive MAX_SIZE_PER_JOB from the largest set of instructions to submit a job and have a define in the driver for this. I believe it is currently set to 192 bytes (the actual define is MAX_JOB_SIZE_BYTES). So a 16k ring lets Xe have 85 jobs inflight at once.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't
Yes, hw_submission_limit = RING_SIZE / MAX_SIZE_PER_JOB in Xe.
Quoted 6 lines
Quoted 1 line
distinct full vs empty ring buffer).
Not sure if I get you right, let me try to clarify what I was trying to say:
I wanted to say that in NouveauMAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.
I'm confused how there isn't a limit on the size of the job in Nouveau? Based on what you have said, a job could be larger than the ring then?
Quoted 5 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
If this doesn't work for Nouveau, feel free flow control the ring differently but this works rather well (and simple) for Xe.
Matt
Quoted 38 lines
Quoted 37 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 30 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 26 lines · Matthew Brost
Quoted 17 lines · Danilo Krummrich
Quoted 16 lines · Christian König
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 15 lines
Hi Matt,
Quoted 12 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?
Quoted 8 lines
In Xe the job submission is series of ring instructions done by the KMD. The instructions are cache flushes, seqno writes, jump to user BB, etc... The exact instructions for each job vary based on hw engine type, platform, etc... We dervive
MAX_SIZE_PER_JOBfrom the largest set of instructions to submit a job and have a define in the driver for this. I believe it is currently set to 192 bytes (the actual define isMAX_JOB_SIZE_BYTES). So a 16k ring lets Xe have 85 jobs inflight at once.
Ok, that sounds different to how Nouveau works. The "largest set of instructions to submit a job" really is a given by how the hardware works rather than an arbitrary limit?
In Nouveau, userspace can submit an arbitrary amount of addresses of indirect bufferes containing the ring instructions. The ring on the kernel side takes the addresses of the indirect buffers rather than the instructions themself. Hence, technically there isn't really a limit on the amount of IBs submitted by a job except for the ring size.
Quoted 14 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can'tYes,
hw_submission_limit=RING_SIZE/MAX_SIZE_PER_JOBin Xe.Quoted 2 lines
Quoted 1 line
distinct full vs empty ring buffer).
Quoted 7 lines
Quoted 4 lines
Not sure if I get you right, let me try to clarify what I was trying to say:
I wanted to say that in NouveauMAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.I'm confused how there isn't a limit on the size of the job in Nouveau? Based on what you have said, a job could be larger than the ring then?
As explained above, theoretically it could. It's only limited by the ring size.
Quoted 51 lines
Quoted 6 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
If this doesn't work for Nouveau, feel free flow control the ring differently but this works rather well (and simple) for Xe.
Matt
Quoted 38 lines
Quoted 37 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 30 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
In reply to Danilo KrummrichAm 16.08.23 um 14:30 schrieb Danilo Krummrich:
Quoted 22 lines · Christian König
Quoted 21 lines
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 20 lines
Hi Matt,
Quoted 16 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau
MAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.
But that lets the hw run dry between submissions. That is usually a pretty horrible idea for performance.
Quoted 2 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
That doesn't sounds like a good idea to me. See we don't limit the number of submitted jobs based on the ring size, but rather we calculate the ring size based on the number of submitted jobs.
In other words the hw_submission_limit defines the ring size, not the other way around. And you usually want the hw_submission_limit as low as possible for good scheduler granularity and to avoid extra overhead.
Christian.
Quoted 31 lines
Quoted 30 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 25 lines · Christian König
Am 16.08.23 um 14:30 schrieb Danilo Krummrich:
Quoted 24 lines · Christian König
Quoted 23 lines
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 22 lines
Hi Matt,
Quoted 12 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.Quoted 4 lines
2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau
MAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.But that lets the hw run dry between submissions. That is usually a pretty horrible idea for performance.
Correct, that's the reason why I said it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the maximum size of a job.
Quoted 6 lines
Quoted 3 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
That doesn't sounds like a good idea to me. See we don't limit the number of submitted jobs based on the ring size, but rather we calculate the ring size based on the number of submitted jobs.
My point isn't really about whether we derive the ring size from the job limit or the other way around. It's more about the job size (or its maximum size) being arbitrary.
As mentioned in my reply to Matt:
"In Nouveau, userspace can submit an arbitrary amount of addresses of indirect bufferes containing the ring instructions. The ring on the kernel side takes the addresses of the indirect buffers rather than the instructions themself. Hence, technically there isn't really a limit on the amount of IBs submitted by a job except for the ring size."
So, my point is that I don't really want to limit the job size artificially just to be able to fit multiple jobs into the ring even if they're submitted at their "artificial" maximum size, but rather track how much of the ring the submitted job actually occupies.
Quoted 1 line
In other words the
hw_submission_limitdefines the ring size, not the other way around. And you usually want thehw_submission_limitas low as possible for good scheduler granularity and to avoid extra overhead.
I don't think you really mean "as low as possible", do you? Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 35 lines
Christian.
Quoted 31 lines
Quoted 30 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 16.08.23 um 18:33 schrieb Danilo Krummrich:
Quoted 47 lines · Christian König
Quoted 25 lines
Am 16.08.23 um 14:30 schrieb Danilo Krummrich:
Quoted 24 lines · Christian König
Quoted 23 lines
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 22 lines
Hi Matt,
Quoted 12 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.Quoted 4 lines
2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau
MAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.But that lets the hw run dry between submissions. That is usually a pretty horrible idea for performance.
Correct, that's the reason why I said it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the maximum size of a job.
Quoted 6 lines
Quoted 3 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
That doesn't sounds like a good idea to me. See we don't limit the number of submitted jobs based on the ring size, but rather we calculate the ring size based on the number of submitted jobs.
My point isn't really about whether we derive the ring size from the job limit or the other way around. It's more about the job size (or its maximum size) being arbitrary.
As mentioned in my reply to Matt:
Quoted 7 lines
"In Nouveau, userspace can submit an arbitrary amount of addresses of indirect bufferes containing the ring instructions. The ring on the kernel side takes the addresses of the indirect buffers rather than the instructions themself. Hence, technically there isn't really a limit on the amount of IBs submitted by a job except for the ring size."
So, my point is that I don't really want to limit the job size artificially just to be able to fit multiple jobs into the ring even if they're submitted at their "artificial" maximum size, but rather track how much of the ring the submitted job actually occupies.
Quoted 1 line
In other words the
hw_submission_limitdefines the ring size, not the other way around. And you usually want thehw_submission_limitas low as possible for good scheduler granularity and to avoid extra overhead.I don't think you really mean "as low as possible", do you?
No, I do mean as low as possible or in other words as few as possible.
Ideally the scheduler would submit only the minimum amount of work to the hardware to keep the hardware busy.
The hardware seems to work mostly the same for all vendors, but you somehow seem to think that filling the ring is somehow beneficial which is really not the case as far as I can see.
Regards,
Christian.
Quoted 42 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 35 lines
Christian.
Quoted 31 lines
Quoted 30 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 52 lines · Christian König
Am 16.08.23 um 18:33 schrieb Danilo Krummrich:
Quoted 51 lines · Christian König
Quoted 25 lines
Am 16.08.23 um 14:30 schrieb Danilo Krummrich:
Quoted 24 lines · Christian König
Quoted 23 lines
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 22 lines
Hi Matt,
Quoted 12 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if targeting the same hardware engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls apart as the TDR expects submission order == completion order. Using a dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.Quoted 4 lines
2. In XE submissions are done via programming a ring buffer (circular buffer), a
drm_gpu_schedulerprovides a limit on number of jobs, if the limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow control on the ring for free.In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Quoted 1 line
In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau
MAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.But that lets the hw run dry between submissions. That is usually a pretty horrible idea for performance.
Correct, that's the reason why I said it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the maximum size of a job.
Quoted 6 lines
Quoted 3 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
That doesn't sounds like a good idea to me. See we don't limit the number of submitted jobs based on the ring size, but rather we calculate the ring size based on the number of submitted jobs.
My point isn't really about whether we derive the ring size from the job limit or the other way around. It's more about the job size (or its maximum size) being arbitrary.
As mentioned in my reply to Matt:
"In Nouveau, userspace can submit an arbitrary amount of addresses of indirect bufferes containing the ring instructions. The ring on the kernel side takes the addresses of the indirect buffers rather than the instructions themself. Hence, technically there isn't really a limit on the amount of IBs submitted by a job except for the ring size."
So, my point is that I don't really want to limit the job size artificially just to be able to fit multiple jobs into the ring even if they're submitted at their "artificial" maximum size, but rather track how much of the ring the submitted job actually occupies.
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
In other words the
hw_submission_limitdefines the ring size, not the other way around. And you usually want thehw_submission_limitas low as possible for good scheduler granularity and to avoid extra overhead.I don't think you really mean "as low as possible", do you?
No, I do mean as low as possible or in other words as few as possible.
Ideally the scheduler would submit only the minimum amount of work to the hardware to keep the hardware busy. >
The hardware seems to work mostly the same for all vendors, but you somehow seem to think that filling the ring is somehow beneficial which is really not the case as far as I can see.
I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to hw_submission_limit == 1.
I recognize you said that one should define hw_submission_limit and adjust the other parts of the equation accordingly, the options I see are:
(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.
(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
My proposal would be to just keep the hw_submission_limit (maybe rename it to submission_unit_limit) and add a submission_units field to struct drm_sched_job. By default a jobs submission_units field would be 0 and the scheduler would behave the exact same way as it does now.
Accordingly, jobs with submission_units > 1 would contribute more than one unit to the submission_unit_limit.
What do you think about that?
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
- Danilo
Quoted 47 lines
Regards,
Christian.Quoted 42 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 35 lines
Christian.
Quoted 31 lines
Quoted 30 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 17.08.23 um 13:13 schrieb Danilo Krummrich:
Quoted 5 lines · Christian König
Quoted 2 lines
[SNIP]
The hardware seems to work mostly the same for all vendors, but you somehow seem to think that filling the ring is somehow beneficial which is really not the case as far as I can see.I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As far as I can see this is not correct for Nouveau either.
Quoted 17 lines
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Quoted 7 lines
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
My proposal would be to just keep the
hw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.What do you think about that?
I think you are approaching this from the completely wrong side.
See the UAPI needs to be stable, so you need a maximum job size otherwise it can happen that a combination of large and small submissions work while a different combination doesn't.
So what you usually do, and this is driver independent because simply a requirement of the UAPI, is that you say here that's my maximum job size as well as the number of submission which should be pushed to the hw at the same time. And then get the resulting ring size by the product of the two.
That the ring in this use case can't be fully utilized is not a draw back, this is completely intentional design which should apply to all drivers independent of the vendor.
Quoted 2 lines
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
That results in better scheduling behavior. It's mostly beneficial if you don't have a hw scheduler, but as far as I can see there is no need to pump everything to the hw as fast as possible.
Regards,
Christian.
Quoted 51 lines
- Danilo
Quoted 47 lines
Regards,
Christian.Quoted 42 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 35 lines
Christian.
Quoted 31 lines
Quoted 30 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 28 lines · Christian König
Am 17.08.23 um 13:13 schrieb Danilo Krummrich:
Quoted 5 lines · Christian König
Quoted 2 lines
[SNIP]
The hardware seems to work mostly the same for all vendors, but you somehow seem to think that filling the ring is somehow beneficial which is really not the case as far as I can see.I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As far as I can see this is not correct for Nouveau either.
Quoted 19 lines
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
Quoted 7 lines
Quoted 5 lines
My proposal would be to just keep the
hw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.What do you think about that?
I think you are approaching this from the completely wrong side.
First of all, thanks for keeping up the discussion - I appreciate it. Some more comments / questions below.
Quoted 2 lines
See the UAPI needs to be stable, so you need a maximum job size otherwise it can happen that a combination of large and small submissions work while a different combination doesn't.
How is this related to the uAPI being stable? What do you mean by 'stable' in this context?
The Nouveau uAPI allows userspace to pass EXEC jobs by supplying the ring ID (channel), in-/out-syncs and a certain amount of indirect push buffers. The amount of IBs per job is limited by the amount of IBs fitting into the ring. Just to be clear, when I say 'job size' I mean the amount of IBs per job.
Maybe I should also mention that the rings we are talking about are software rings managed by a firmware scheduler. We can have an arbitrary amount of software rings and even multiple ones per FD.
Given a constant ring size I really don't see why I should limit the maximum amount of IBs userspace can push per job just to end up with a hw_submission_limit > 1.
For example, let's just assume the ring can take 128 IBs, why would I limit userspace to submit just e.g. 16 IBs at a time, such that the hw_submission_limit becomes 8?
What is the advantage of doing that, rather than letting userspace submit *up to* 128 IBs per job and just letting the scheduler push IBs to the ring as long as there's actually space left on the ring?
Quoted 2 lines
So what you usually do, and this is driver independent because simply a requirement of the UAPI, is that you say here that's my maximum job size as well as the number of submission which should be pushed to the hw at the same time. And then get the resulting ring size by the product of the two.
Given the above, how is that a requirement of the uAPI?
Quoted 2 lines
That the ring in this use case can't be fully utilized is not a draw back, this is completely intentional design which should apply to all drivers independent of the vendor.
Why wouldn't we want to fully utilize the ring size?
- Danilo
Quoted 61 lines
Quoted 3 lines
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
That results in better scheduling behavior. It's mostly beneficial if you don't have a hw scheduler, but as far as I can see there is no need to pump everything to the hw as fast as possible.
Regards,
Christian.Quoted 51 lines
- Danilo
Quoted 47 lines
Regards,
Christian.Quoted 42 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 35 lines
Christian.
Quoted 31 lines
Quoted 30 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 17.08.23 um 14:48 schrieb Danilo Krummrich:
Quoted 10 lines · Christian König
Quoted 9 lines
Am 17.08.23 um 13:13 schrieb Danilo Krummrich:
Quoted 7 lines · Christian König
Quoted 6 lines
[SNIP]
My proposal would be to just keep thehw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.What do you think about that?
Quoted 8 lines
Quoted 1 line
I think you are approaching this from the completely wrong side.
First of all, thanks for keeping up the discussion - I appreciate it. Some more comments / questions below.
Quoted 2 lines
See the UAPI needs to be stable, so you need a maximum job size otherwise it can happen that a combination of large and small submissions work while a different combination doesn't.
How is this related to the uAPI being stable? What do you mean by 'stable' in this context?
Stable is in you don't get indifferent behavior, not stable is in the sense of backward compatibility. Sorry for the confusing wording :)
Quoted 2 lines
The Nouveau uAPI allows userspace to pass EXEC jobs by supplying the ring ID (channel), in-/out-syncs and a certain amount of indirect push buffers. The amount of IBs per job is limited by the amount of IBs fitting into the ring. Just to be clear, when I say 'job size' I mean the amount of IBs per job.
Well that more or less sounds identical to all other hardware I know of, e.g. AMD, Intel and the different ARM chips seem to all work like this. But on those drivers the job size limit is not the ring size, but rather a fixed value (at least as far as I know).
Quoted 6 lines
Maybe I should also mention that the rings we are talking about are software rings managed by a firmware scheduler. We can have an arbitrary amount of software rings and even multiple ones per FD.
Given a constant ring size I really don't see why I should limit the maximum amount of IBs userspace can push per job just to end up with a
hw_submission_limit> 1.For example, let's just assume the ring can take 128 IBs, why would I limit userspace to submit just e.g. 16 IBs at a time, such that the
hw_submission_limitbecomes 8?
Well the question is what happens when you have two submissions back to back which use more than halve of the ring buffer?
I only see two possible outcomes:
1. You return -EBUSY (or similar) error code indicating the the hw can't receive more commands.
2. Wait on previously pushed commands to be executed.
(3. Your driver crash because you accidentally overwrite stuff in the ring buffer which is still executed. I just assume that's prevented).
Resolution #1 with -EBUSY is actually something the UAPI should not do, because your UAPI then depends on the specific timing of submissions which is a really bad idea.
Resolution #2 is usually bad because it forces the hw to run dry between submission and so degrade performance.
Quoted 2 lines
What is the advantage of doing that, rather than letting userspace submit *up to* 128 IBs per job and just letting the scheduler push IBs to the ring as long as there's actually space left on the ring?
Predictable behavior I think. Basically you want organize things so that the hw is at least kept busy all the time without depending on actual timing.
Quoted 5 lines
Quoted 3 lines
So what you usually do, and this is driver independent because simply a requirement of the UAPI, is that you say here that's my maximum job size as well as the number of submission which should be pushed to the hw at the same time. And then get the resulting ring size by the product of the two.
Given the above, how is that a requirement of the uAPI?
The requirement of the UAPI is actually pretty simple: You should get consistent results, independent of the timing (at least as long as you don't do stuff in parallel).
Otherwise you can run into issues when on a certain configuration stuff suddenly runs faster or slower than expected. In other words you should not depend on that stuff finishes in a certain amount of time.
Quoted 5 lines
Quoted 3 lines
That the ring in this use case can't be fully utilized is not a draw back, this is completely intentional design which should apply to all drivers independent of the vendor.
Why wouldn't we want to fully utilize the ring size?
As far as I know everybody restricts the submission size to something fixed which is at least smaller than halve the ring size to avoid the problems mentioned above.
Regards,
Christian.
Quoted 65 lines
- Danilo
Quoted 61 lines
Quoted 3 lines
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
That results in better scheduling behavior. It's mostly beneficial if you don't have a hw scheduler, but as far as I can see there is no need to pump everything to the hw as fast as possible.
Regards,
Christian.Quoted 51 lines
- Danilo
Quoted 47 lines
Regards,
Christian.Quoted 42 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 35 lines
Christian.
Quoted 31 lines
Quoted 30 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 36 lines · Christian König
Am 17.08.23 um 14:48 schrieb Danilo Krummrich:
Quoted 10 lines · Christian König
Quoted 9 lines
Am 17.08.23 um 13:13 schrieb Danilo Krummrich:
Quoted 7 lines · Christian König
Quoted 6 lines
[SNIP]
My proposal would be to just keep thehw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.What do you think about that?
Quoted 8 lines
Quoted 1 line
I think you are approaching this from the completely wrong side.
First of all, thanks for keeping up the discussion - I appreciate it. Some more comments / questions below.
Quoted 2 lines
See the UAPI needs to be stable, so you need a maximum job size otherwise it can happen that a combination of large and small submissions work while a different combination doesn't.
How is this related to the uAPI being stable? What do you mean by 'stable' in this context?
Stable is in you don't get indifferent behavior, not stable is in the sense of backward compatibility. Sorry for the confusing wording :)
Quoted 2 lines
The Nouveau uAPI allows userspace to pass EXEC jobs by supplying the ring ID (channel), in-/out-syncs and a certain amount of indirect push buffers. The amount of IBs per job is limited by the amount of IBs fitting into the ring. Just to be clear, when I say 'job size' I mean the amount of IBs per job.
Well that more or less sounds identical to all other hardware I know of, e.g. AMD, Intel and the different ARM chips seem to all work like this. But on those drivers the job size limit is not the ring size, but rather a fixed value (at least as far as I know).
Quoted 6 lines
Maybe I should also mention that the rings we are talking about are software rings managed by a firmware scheduler. We can have an arbitrary amount of software rings and even multiple ones per FD.
Given a constant ring size I really don't see why I should limit the maximum amount of IBs userspace can push per job just to end up with a
hw_submission_limit> 1.For example, let's just assume the ring can take 128 IBs, why would I limit userspace to submit just e.g. 16 IBs at a time, such that the
hw_submission_limitbecomes 8?Well the question is what happens when you have two submissions back to back which use more than halve of the ring buffer?
Quoted 8 lines
I only see two possible outcomes:
1. You return-EBUSY(or similar) error code indicating the the hw can't receive more commands.
2. Wait on previously pushed commands to be executed.
(3. Your driver crash because you accidentally overwrite stuff in the ring buffer which is still executed. I just assume that's prevented).Resolution #1 with
-EBUSYis actually something the UAPI should not do, because your UAPI then depends on the specific timing of submissions which is a really bad idea.Resolution #2 is usually bad because it forces the hw to run dry between submission and so degrade performance.
I agree, that is a good reason for at least limiting the maximum job size to half of the ring size.
However, there could still be cases where two subsequent jobs are submitted with just a single IB, which as is would still block subsequent jobs to be pushed to the ring although there is still plenty of space. Depending on the (CPU) scheduler latency, such a case can let the HW run dry as well.
Surely, we could just continue decrease the maximum job size even further, but this would result in further overhead on user and kernel for larger IB counts. Tracking the actual job size seems to be the better solution for drivers where the job size can vary over a rather huge range.
- Danilo
Quoted 93 lines
Quoted 3 lines
What is the advantage of doing that, rather than letting userspace submit *up to* 128 IBs per job and just letting the scheduler push IBs to the ring as long as there's actually space left on the ring?
Predictable behavior I think. Basically you want organize things so that the hw is at least kept busy all the time without depending on actual timing.
Quoted 5 lines
Quoted 3 lines
So what you usually do, and this is driver independent because simply a requirement of the UAPI, is that you say here that's my maximum job size as well as the number of submission which should be pushed to the hw at the same time. And then get the resulting ring size by the product of the two.
Given the above, how is that a requirement of the uAPI?
The requirement of the UAPI is actually pretty simple: You should get consistent results, independent of the timing (at least as long as you don't do stuff in parallel).
Otherwise you can run into issues when on a certain configuration stuff suddenly runs faster or slower than expected. In other words you should not depend on that stuff finishes in a certain amount of time.
Quoted 5 lines
Quoted 3 lines
That the ring in this use case can't be fully utilized is not a draw back, this is completely intentional design which should apply to all drivers independent of the vendor.
Why wouldn't we want to fully utilize the ring size?
As far as I know everybody restricts the submission size to something fixed which is at least smaller than halve the ring size to avoid the problems mentioned above.
Regards,
Christian.Quoted 65 lines
- Danilo
Quoted 61 lines
Quoted 3 lines
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
That results in better scheduling behavior. It's mostly beneficial if you don't have a hw scheduler, but as far as I can see there is no need to pump everything to the hw as fast as possible.
Regards,
Christian.Quoted 51 lines
- Danilo
Quoted 47 lines
Regards,
Christian.Quoted 42 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 35 lines
Christian.
Quoted 31 lines
Quoted 30 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 23 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 15 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work around the scaling issue, use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop, start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 18.08.23 um 13:58 schrieb Danilo Krummrich:
Quoted 6 lines
[SNIP]
Quoted 5 lines
I only see two possible outcomes:
1. You return-EBUSY(or similar) error code indicating the the hw can't receive more commands.
2. Wait on previously pushed commands to be executed.
(3. Your driver crash because you accidentally overwrite stuff in the ring buffer which is still executed. I just assume that's prevented).
Quoted 7 lines
Quoted 3 lines
Resolution #1 with
-EBUSYis actually something the UAPI should not do, because your UAPI then depends on the specific timing of submissions which is a really bad idea.Resolution #2 is usually bad because it forces the hw to run dry between submission and so degrade performance.
I agree, that is a good reason for at least limiting the maximum job size to half of the ring size.
However, there could still be cases where two subsequent jobs are submitted with just a single IB, which as is would still block subsequent jobs to be pushed to the ring although there is still plenty of space. Depending on the (CPU) scheduler latency, such a case can let the HW run dry as well.
Yeah, that was intentionally not done as well. The crux here is that the more you push to the hw the worse the scheduling granularity becomes. It's just that neither Xe nor Nouveau relies that much on the scheduling granularity at all (because of hw queues).
But Xe doesn't seem to need that feature and I would still try to avoid it because the more you have pushed to the hw the harder it is to get going again after a reset.
Quoted 2 lines
Surely, we could just continue decrease the maximum job size even further, but this would result in further overhead on user and kernel for larger IB counts. Tracking the actual job size seems to be the better solution for drivers where the job size can vary over a rather huge range.
I strongly disagree on that. A larger ring buffer is trivial to allocate and if userspace submissions are so small that the scheduler can't keep up submitting them then your ring buffer size is your smallest problem.
In other words the submission overhead will completely kill your performance and you should probably consider stuffing more into a single submission.
Regards,
Christian.
Quoted 2 lines
- Danilo
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 15 lines · Christian König
Am 18.08.23 um 13:58 schrieb Danilo Krummrich:
Quoted 6 lines
[SNIP]
Quoted 5 lines
I only see two possible outcomes:
1. You return-EBUSY(or similar) error code indicating the the hw can't receive more commands.
2. Wait on previously pushed commands to be executed.
(3. Your driver crash because you accidentally overwrite stuff in the ring buffer which is still executed. I just assume that's prevented).Quoted 7 lines
Quoted 3 lines
Resolution #1 with
-EBUSYis actually something the UAPI should not do, because your UAPI then depends on the specific timing of submissions which is a really bad idea.Resolution #2 is usually bad because it forces the hw to run dry between submission and so degrade performance.
I agree, that is a good reason for at least limiting the maximum job size to half of the ring size.
However, there could still be cases where two subsequent jobs are submitted with just a single IB, which as is would still block subsequent jobs to be pushed to the ring although there is still plenty of space. Depending on the (CPU) scheduler latency, such a case can let the HW run dry as well.
Quoted 8 lines
Yeah, that was intentionally not done as well. The crux here is that the more you push to the hw the worse the scheduling granularity becomes. It's just that neither Xe nor Nouveau relies that much on the scheduling granularity at all (because of hw queues).
But Xe doesn't seem to need that feature and I would still try to avoid it because the more you have pushed to the hw the harder it is to get going again after a reset.
Quoted 2 lines
Surely, we could just continue decrease the maximum job size even further, but this would result in further overhead on user and kernel for larger IB counts. Tracking the actual job size seems to be the better solution for drivers where the job size can vary over a rather huge range.
I strongly disagree on that. A larger ring buffer is trivial to allocate
That sounds like a workaround to me. The problem, in the case above, isn't that the ring buffer does not have enough space, the problem is that we account for the maximum job size although the actual job size is much smaller. And enabling the scheduler to track the actual job size is trivial as well.
Quoted 3 lines
and if userspace submissions are so small that the scheduler can't keep up submitting them then your ring buffer size is your smallest problem.
In other words the submission overhead will completely kill your performance and you should probably consider stuffing more into a single submission.
I fully agree and that is also the reason why I want to keep the maximum job size as large as possible.
However, afaik with Vulkan it's the applications themselves deciding when and with how many command buffers a queue is submitted (@Faith: please correct me if I'm wrong). Hence, why not optimize for this case as well? It's not that it would make another case worse, right?
- Danilo
Quoted 7 lines
Regards,
Christian.Quoted 2 lines
- Danilo
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 21.08.23 um 20:01 schrieb Danilo Krummrich:
Quoted 18 lines · Christian König
Quoted 17 lines
Am 18.08.23 um 13:58 schrieb Danilo Krummrich:
Quoted 6 lines
[SNIP]
Quoted 5 lines
I only see two possible outcomes:
1. You return-EBUSY(or similar) error code indicating the the hw can't receive more commands.
2. Wait on previously pushed commands to be executed.
(3. Your driver crash because you accidentally overwrite stuff in the ring buffer which is still executed. I just assume that's prevented).Quoted 7 lines
Quoted 3 lines
Resolution #1 with
-EBUSYis actually something the UAPI should not do, because your UAPI then depends on the specific timing of submissions which is a really bad idea.Resolution #2 is usually bad because it forces the hw to run dry between submission and so degrade performance.
I agree, that is a good reason for at least limiting the maximum job size to half of the ring size.
However, there could still be cases where two subsequent jobs are submitted with just a single IB, which as is would still block subsequent jobs to be pushed to the ring although there is still plenty of space. Depending on the (CPU) scheduler latency, such a case can let the HW run dry as well.
Yeah, that was intentionally not done as well. The crux here is that the more you push to the hw the worse the scheduling granularity becomes. It's just that neither Xe nor Nouveau relies that much on the scheduling granularity at all (because of hw queues).
Quoted 8 lines
Quoted 6 lines
But Xe doesn't seem to need that feature and I would still try to avoid it because the more you have pushed to the hw the harder it is to get going again after a reset.
Quoted 2 lines
Surely, we could just continue decrease the maximum job size even further, but this would result in further overhead on user and kernel for larger IB counts. Tracking the actual job size seems to be the better solution for drivers where the job size can vary over a rather huge range.
I strongly disagree on that. A larger ring buffer is trivial to allocate
That sounds like a workaround to me. The problem, in the case above, isn't that the ring buffer does not have enough space, the problem is that we account for the maximum job size although the actual job size is much smaller. And enabling the scheduler to track the actual job size is trivial as well.
That's what I agree on, so far I just didn't see the reason for doing it but at least a few reason for not doing it.
Quoted 8 lines
Quoted 4 lines
and if userspace submissions are so small that the scheduler can't keep up submitting them then your ring buffer size is your smallest problem.
In other words the submission overhead will completely kill your performance and you should probably consider stuffing more into a single submission.
I fully agree and that is also the reason why I want to keep the maximum job size as large as possible.
However, afaik with Vulkan it's the applications themselves deciding when and with how many command buffers a queue is submitted (@Faith: please correct me if I'm wrong). Hence, why not optimize for this case as well? It's not that it would make another case worse, right?
As I said it does make both the scheduling granularity as well as the reset behavior worse.
In general I think we should try to push just enough work to the hardware to keep it busy and not as much as possible.
So as long as nobody from userspace comes and says we absolutely need to optimize this use case I would rather not do it.
Regards,
Christian.
Quoted 11 lines
- Danilo
Quoted 7 lines
Regards,
Christian.Quoted 2 lines
- Danilo
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 33 lines · Christian König
Am 21.08.23 um 20:01 schrieb Danilo Krummrich:
Quoted 18 lines · Christian König
Quoted 17 lines
Am 18.08.23 um 13:58 schrieb Danilo Krummrich:
Quoted 6 lines
[SNIP]
Quoted 5 lines
I only see two possible outcomes:
1. You return-EBUSY(or similar) error code indicating the the hw can't receive more commands.
2. Wait on previously pushed commands to be executed.
(3. Your driver crash because you accidentally overwrite stuff in the ring buffer which is still executed. I just assume that's prevented).Quoted 7 lines
Quoted 3 lines
Resolution #1 with
-EBUSYis actually something the UAPI should not do, because your UAPI then depends on the specific timing of submissions which is a really bad idea.Resolution #2 is usually bad because it forces the hw to run dry between submission and so degrade performance.
I agree, that is a good reason for at least limiting the maximum job size to half of the ring size.
However, there could still be cases where two subsequent jobs are submitted with just a single IB, which as is would still block subsequent jobs to be pushed to the ring although there is still plenty of space. Depending on the (CPU) scheduler latency, such a case can let the HW run dry as well.
Yeah, that was intentionally not done as well. The crux here is that the more you push to the hw the worse the scheduling granularity becomes. It's just that neither Xe nor Nouveau relies that much on the scheduling granularity at all (because of hw queues).
Quoted 8 lines
Quoted 6 lines
But Xe doesn't seem to need that feature and I would still try to avoid it because the more you have pushed to the hw the harder it is to get going again after a reset.
Quoted 2 lines
Surely, we could just continue decrease the maximum job size even further, but this would result in further overhead on user and kernel for larger IB counts. Tracking the actual job size seems to be the better solution for drivers where the job size can vary over a rather huge range.
I strongly disagree on that. A larger ring buffer is trivial to allocate
That sounds like a workaround to me. The problem, in the case above, isn't that the ring buffer does not have enough space, the problem is that we account for the maximum job size although the actual job size is much smaller. And enabling the scheduler to track the actual job size is trivial as well.
That's what I agree on, so far I just didn't see the reason for doing it but at least a few reason for not doing it.
Quoted 3 lines
Quoted 3 lines
and if userspace submissions are so small that the scheduler can't keep up submitting them then your ring buffer size is your smallest problem.
Quoted 7 lines
Quoted 5 lines
Quoted 1 line
In other words the submission overhead will completely kill your performance and you should probably consider stuffing more into a single submission.
I fully agree and that is also the reason why I want to keep the maximum job size as large as possible.
However, afaik with Vulkan it's the applications themselves deciding when and with how many command buffers a queue is submitted (@Faith: please correct me if I'm wrong). Hence, why not optimize for this case as well? It's not that it would make another case worse, right?
As I said it does make both the scheduling granularity as well as the reset behavior worse.
As you already mentioned Nouveau (and XE) don't really rely much on scheduling granularity. For Nouveau, the same is true for the reset behavior; if things go south the channel is killed anyway. Userspace would just request a new ring in this case.
Hence, I think Nouveau would profit from accounting the actual job size. And at the same time, other drivers having a benefit of always accounting for the maximum job size would still do so, by default.
Arbitrary ratios of how much the job size contributes to the ring being considered as full would also be possible.
- Danilo
Quoted 20 lines
In general I think we should try to push just enough work to the hardware to keep it busy and not as much as possible.
So as long as nobody from userspace comes and says we absolutely need to optimize this use case I would rather not do it.
Regards,
Christian.Quoted 11 lines
- Danilo
Quoted 7 lines
Regards,
Christian.Quoted 2 lines
- Danilo
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 21.08.23 um 21:07 schrieb Danilo Krummrich:
Quoted 40 lines · Christian König
Quoted 39 lines
Am 21.08.23 um 20:01 schrieb Danilo Krummrich:
Quoted 18 lines · Christian König
Quoted 17 lines
Am 18.08.23 um 13:58 schrieb Danilo Krummrich:
Quoted 6 lines
[SNIP]
Quoted 5 lines
I only see two possible outcomes:
1. You return-EBUSY(or similar) error code indicating the the hw can't receive more commands.
2. Wait on previously pushed commands to be executed.
(3. Your driver crash because you accidentally overwrite stuff in the ring buffer which is still executed. I just assume that's prevented).Quoted 7 lines
Quoted 3 lines
Resolution #1 with
-EBUSYis actually something the UAPI should not do, because your UAPI then depends on the specific timing of submissions which is a really bad idea.Resolution #2 is usually bad because it forces the hw to run dry between submission and so degrade performance.
I agree, that is a good reason for at least limiting the maximum job size to half of the ring size.
However, there could still be cases where two subsequent jobs are submitted with just a single IB, which as is would still block subsequent jobs to be pushed to the ring although there is still plenty of space. Depending on the (CPU) scheduler latency, such a case can let the HW run dry as well.
Yeah, that was intentionally not done as well. The crux here is that the more you push to the hw the worse the scheduling granularity becomes. It's just that neither Xe nor Nouveau relies that much on the scheduling granularity at all (because of hw queues).
Quoted 8 lines
Quoted 6 lines
But Xe doesn't seem to need that feature and I would still try to avoid it because the more you have pushed to the hw the harder it is to get going again after a reset.
Quoted 2 lines
Surely, we could just continue decrease the maximum job size even further, but this would result in further overhead on user and kernel for larger IB counts. Tracking the actual job size seems to be the better solution for drivers where the job size can vary over a rather huge range.
I strongly disagree on that. A larger ring buffer is trivial to allocate
That sounds like a workaround to me. The problem, in the case above, isn't that the ring buffer does not have enough space, the problem is that we account for the maximum job size although the actual job size is much smaller. And enabling the scheduler to track the actual job size is trivial as well.
That's what I agree on, so far I just didn't see the reason for doing it but at least a few reason for not doing it.
Quoted 8 lines
Quoted 4 lines
and if userspace submissions are so small that the scheduler can't keep up submitting them then your ring buffer size is your smallest problem.
In other words the submission overhead will completely kill your performance and you should probably consider stuffing more into a single submission.
I fully agree and that is also the reason why I want to keep the maximum job size as large as possible.
However, afaik with Vulkan it's the applications themselves deciding when and with how many command buffers a queue is submitted (@Faith: please correct me if I'm wrong). Hence, why not optimize for this case as well? It's not that it would make another case worse, right?
Quoted 7 lines
Quoted 1 line
As I said it does make both the scheduling granularity as well as the reset behavior worse.
As you already mentioned Nouveau (and XE) don't really rely much on scheduling granularity. For Nouveau, the same is true for the reset behavior; if things go south the channel is killed anyway. Userspace would just request a new ring in this case.
Hence, I think Nouveau would profit from accounting the actual job size. And at the same time, other drivers having a benefit of always accounting for the maximum job size would still do so, by default.
Arbitrary ratios of how much the job size contributes to the ring being considered as full would also be possible.
That would indeed be rather interesting since for a bunch of drivers the limiting part is not the ring buffer size, but rather the utilization of engines.
But no idea how to properly design that. You would have multiple values to check instead of just one.
Christian.
Quoted 24 lines
- Danilo
Quoted 20 lines
In general I think we should try to push just enough work to the hardware to keep it busy and not as much as possible.
So as long as nobody from userspace comes and says we absolutely need to optimize this use case I would rather not do it.
Regards,
Christian.Quoted 11 lines
- Danilo
Quoted 7 lines
Regards,
Christian.Quoted 2 lines
- Danilo
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
In reply to Christian KönigQuoted 73 lines · Christian König
Am 21.08.23 um 20:01 schrieb Danilo Krummrich:
Quoted 37 lines · Christian König
Quoted 36 lines
Am 18.08.23 um 13:58 schrieb Danilo Krummrich:
Quoted 16 lines
[SNIP]
Quoted 8 lines
I only see two possible outcomes:
1. You return-EBUSY(or similar) error code indicating the the hw can't receive more commands.
2. Wait on previously pushed commands to be executed. (3. Your driver crash because you accidentally overwrite stuff in the ring buffer which is still executed. I just assume that's prevented).Quoted 6 lines
Resolution #1 with
-EBUSYis actually something the UAPI should not do, because your UAPI then depends on the specific timing of submissions which is a really bad idea.Resolution #2 is usually bad because it forces the hw to run dry between submission and so degrade performance.
Quoted 8 lines
I agree, that is a good reason for at least limiting the maximum job size to half of the ring size.
However, there could still be cases where two subsequent jobs are submitted with just a single IB, which as is would still block subsequent jobs to be pushed to the ring although there is still plenty of space. Depending on the (CPU) scheduler latency, such a case can let the HW run dry as well.
Yeah, that was intentionally not done as well. The crux here is that the more you push to the hw the worse the scheduling granularity becomes. It's just that neither Xe nor Nouveau relies that much on the scheduling granularity at all (because of hw queues).
But Xe doesn't seem to need that feature and I would still try to avoid it because the more you have pushed to the hw the harder it is to get going again after a reset.
Quoted 1 line
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Surely, we could just continue decrease the maximum job size even further, but this would result in further overhead on user and kernel for larger IB counts. Tracking the actual job size seems to be the better solution for drivers where the job size can vary over a rather huge range.
I strongly disagree on that. A larger ring buffer is trivial to
allocate
Quoted 6 lines
That sounds like a workaround to me. The problem, in the case above, isn't that the ring buffer does not have enough space, the problem is that we account for the maximum job size although the actual job size is much smaller. And enabling the scheduler to track the actual job size is trivial as well.
That's what I agree on, so far I just didn't see the reason for doing it but at least a few reason for not doing it.
Quoted 9 lines
Quoted 8 lines
and if userspace submissions are so small that the scheduler can't keep up submitting them then your ring buffer size is your smallest problem.
In other words the submission overhead will completely kill your performance and you should probably consider stuffing more into a single submission.
Quoted 7 lines
I fully agree and that is also the reason why I want to keep the maximum job size as large as possible.
However, afaik with Vulkan it's the applications themselves deciding when and with how many command buffers a queue is submitted (@Faith:
please correct me if I'm wrong). Hence, why not optimize for this case as well? It's not that it would make another case worse, right?
Quoted 9 lines
As I said it does make both the scheduling granularity as well as the reset behavior worse.
In general I think we should try to push just enough work to the hardware to keep it busy and not as much as possible.
So as long as nobody from userspace comes and says we absolutely need to optimize this use case I would rather not do it.
This is a place where nouveau's needs are legitimately different from AMD or Intel, I think. NVIDIA's command streamer model is very different from AMD and Intel. On AMD and Intel, each EXEC turns into a single small packet (on the order of 16B) which kicks off a command buffer. There may be a bit of cache management or something around it but that's it. From there, it's userspace's job to make one command buffer chain to another until it's finally done and then do a "return", whatever that looks like.
NVIDIA's model is much more static. Each packet in the HW/FW ring is an address and a size and that much data is processed and then it grabs the next packet and processes. The result is that, if we use multiple buffers of commands, there's no way to chain them together. We just have to pass the whole list of buffers to the kernel. A single EXEC ioctl / job may have 500 such addr+size packets depending on how big the command buffer is. It gets worse on pre-Turing hardware where we have to split the batch for every single DrawIndirect or DispatchIndirect.
Lest you think NVIDIA is just crazy here, it's a perfectly reasonable model if you assume that userspace is feeding the firmware. When that's happening, you just have a userspace thread that sits there and feeds the ringbuffer with whatever is next and you can marshal as much data through as you want. Sure, it'd be nice to have a 2nd level batch thing that gets launched from the FW ring and has all the individual launch commands but it's not at all necessary.
What does that mean from a gpu_scheduler PoV? Basically, it means a variable packet size.
What does this mean for implementation? IDK. One option would be to teach the scheduler about actual job sizes. Another would be to virtualize it and have another layer underneath the scheduler that does the actual feeding of the ring. Another would be to decrease the job size somewhat and then have the front-end submit as many jobs as it needs to service userspace and only put the out-fences on the last job. All the options kinda suck.
~Faith
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 21.08.23 um 21:46 schrieb Faith Ekstrand:
Quoted 11 lines · Christian König
[SNIP]
So as long as nobody from userspace comes and says we absolutely need to
optimize this use case I would rather not do it.This is a place where nouveau's needs are legitimately different from AMD or Intel, I think. NVIDIA's command streamer model is very different from AMD and Intel. On AMD and Intel, each EXEC turns into a single small packet (on the order of 16B) which kicks off a command buffer. There may be a bit of cache management or something around it but that's it. From there, it's userspace's job to make one command buffer chain to another until it's finally done and then do a "return", whatever that looks like.
NVIDIA's model is much more static. Each packet in the HW/FW ring is an address and a size and that much data is processed and then it grabs the next packet and processes. The result is that, if we use multiple buffers of commands, there's no way to chain them together. We just have to pass the whole list of buffers to the kernel.
So far that is actually completely identical to what AMD has.
Quoted 1 line
A single EXEC ioctl / job may have 500 such addr+size packets depending on how big the command buffer is.
And that is what I don't understand. Why would you need 100dreds of such addr+size packets?
This is basically identical to what AMD has (well on newer hw there is an extension in the CP packets to JUMP/CALL subsequent IBs, but this isn't widely used as far as I know).
Previously the limit was something like 4 which we extended to because Bas came up with similar requirements for the AMD side from RADV.
But essentially those approaches with 100dreds of IBs doesn't sound like a good idea to me.
Quoted 7 lines
It gets worse on pre-Turing hardware where we have to split the batch for every single DrawIndirect or DispatchIndirect.
Lest you think NVIDIA is just crazy here, it's a perfectly reasonable model if you assume that userspace is feeding the firmware. When that's happening, you just have a userspace thread that sits there and feeds the ringbuffer with whatever is next and you can marshal as much data through as you want. Sure, it'd be nice to have a 2nd level batch thing that gets launched from the FW ring and has all the individual launch commands but it's not at all necessary.
What does that mean from a
gpu_schedulerPoV? Basically, it means a variable packet size.What does this mean for implementation? IDK. One option would be to teach the scheduler about actual job sizes. Another would be to virtualize it and have another layer underneath the scheduler that does the actual feeding of the ring. Another would be to decrease the job size somewhat and then have the front-end submit as many jobs as it needs to service userspace and only put the out-fences on the last job. All the options kinda suck.
Yeah, agree. The job size Danilo suggested is still the least painful.
Christian.
Quoted 2 lines
~Faith
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 25 lines · Christian König
Am 21.08.23 um 21:46 schrieb Faith Ekstrand:
Quoted 4 lines · Christian König
[SNIP]
So as long as nobody from userspace comes and says we absolutely need to optimize this use case I would rather not do it.This is a place where nouveau's needs are legitimately different from AMD or Intel, I think. NVIDIA's command streamer model is very different from AMD and Intel. On AMD and Intel, each EXEC turns into a single small packet (on the order of 16B) which kicks off a command buffer. There may be a bit of cache management or something around it but that's it. From there, it's userspace's job to make one command buffer chain to another until it's finally done and then do a "return", whatever that looks like.
NVIDIA's model is much more static. Each packet in the HW/FW ring is an address and a size and that much data is processed and then it grabs the next packet and processes. The result is that, if we use multiple buffers of commands, there's no way to chain them together. We just have to pass the whole list of buffers to the kernel.
Quoted 9 lines
So far that is actually completely identical to what AMD has.
A single EXEC ioctl / job may have 500 such addr+size packets depending on how big the command buffer is.
And that is what I don't understand. Why would you need 100dreds of such addr+size packets?
Well, we're not really in control of it. We can control our base pushbuf size and that's something we can tune but we're still limited by the client. We have to submit another pushbuf whenever:
1. We run out of space (power-of-two growth is also possible but the size
is limited to a maximum of about 4MiB due to hardware limitations.)
2. The client calls a secondary command buffer.
3. Any usage of indirect draw or dispatch on pre-Turing hardware.
At some point we need to tune our BO size a bit to avoid (1) while also avoiding piles of tiny BOs. However, (2) and (3) are out of our control.
This is basically identical to what AMD has (well on newer hw there is an
Quoted 3 lines
extension in the CP packets to JUMP/CALL subsequent IBs, but this isn't widely used as far as I know).
According to Bas, RADV chains on recent hardware.
Quoted 6 lines
Previously the limit was something like 4 which we extended to because Bas came up with similar requirements for the AMD side from RADV.
But essentially those approaches with 100dreds of IBs doesn't sound like a good idea to me.
No one's arguing that they like it. Again, the hardware isn't designed to have a kernel in the way. It's designed to be fed by userspace. But we're going to have the kernel in the middle for a while so we need to make it not suck too bad.
~Faith
It gets worse on pre-Turing hardware where we have to split the batch for
Quoted 30 lines
every single DrawIndirect or DispatchIndirect.
Lest you think NVIDIA is just crazy here, it's a perfectly reasonable model if you assume that userspace is feeding the firmware. When that's happening, you just have a userspace thread that sits there and feeds the ringbuffer with whatever is next and you can marshal as much data through as you want. Sure, it'd be nice to have a 2nd level batch thing that gets launched from the FW ring and has all the individual launch commands but it's not at all necessary.
What does that mean from a
gpu_schedulerPoV? Basically, it means a variable packet size.What does this mean for implementation? IDK. One option would be to teach the scheduler about actual job sizes. Another would be to virtualize it and have another layer underneath the scheduler that does the actual feeding of the ring. Another would be to decrease the job size somewhat and then have the front-end submit as many jobs as it needs to service userspace and only put the out-fences on the last job. All the options kinda suck.
Yeah, agree. The job size Danilo suggested is still the least painful.
Christian.
~Faith
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 50 lines · Faith Ekstrand
Quoted 25 lines · Christian König
Am 21.08.23 um 21:46 schrieb Faith Ekstrand:
Quoted 4 lines · Christian König
[SNIP]
So as long as nobody from userspace comes and says we absolutely need to optimize this use case I would rather not do it.This is a place where nouveau's needs are legitimately different from AMD or Intel, I think. NVIDIA's command streamer model is very different from AMD and Intel. On AMD and Intel, each EXEC turns into a single small packet (on the order of 16B) which kicks off a command buffer. There may be a bit of cache management or something around it but that's it. From there, it's userspace's job to make one command buffer chain to another until it's finally done and then do a "return", whatever that looks like.
NVIDIA's model is much more static. Each packet in the HW/FW ring is an address and a size and that much data is processed and then it grabs the next packet and processes. The result is that, if we use multiple buffers of commands, there's no way to chain them together. We just have to pass the whole list of buffers to the kernel.
Quoted 9 lines
So far that is actually completely identical to what AMD has.
A single EXEC ioctl / job may have 500 such addr+size packets depending on how big the command buffer is.
And that is what I don't understand. Why would you need 100dreds of such addr+size packets?
Well, we're not really in control of it. We can control our base pushbuf size and that's something we can tune but we're still limited by the client. We have to submit another pushbuf whenever:
1. We run out of space (power-of-two growth is also possible but the size
is limited to a maximum of about 4MiB due to hardware limitations.)
2. The client calls a secondary command buffer.
3. Any usage of indirect draw or dispatch on pre-Turing hardware.At some point we need to tune our BO size a bit to avoid (1) while also avoiding piles of tiny BOs. However, (2) and (3) are out of our control.
Quoted 7 lines
This is basically identical to what AMD has (well on newer hw there is an
Quoted 3 lines
extension in the CP packets to JUMP/CALL subsequent IBs, but this isn't widely used as far as I know).
According to Bas, RADV chains on recent hardware.
well:
1) on GFX6 and older we can't chain at all
2) on Compute/DMA we can't chain at all
3) with VK_COMMAND_BUFFER_USAGE_SIMULTANEOUS_USE_BIT we can't chain between cmdbuffers
4) for some secondary use cases we can't chain.
so we have to do the "submit multiple" dance in many cases.
Quoted 48 lines
Quoted 8 lines
Previously the limit was something like 4 which we extended to because Bas came up with similar requirements for the AMD side from RADV.
But essentially those approaches with 100dreds of IBs doesn't sound like a good idea to me.
No one's arguing that they like it. Again, the hardware isn't designed to have a kernel in the way. It's designed to be fed by userspace. But we're going to have the kernel in the middle for a while so we need to make it not suck too bad.
~Faith
It gets worse on pre-Turing hardware where we have to split the batch for
Quoted 31 lines
every single DrawIndirect or DispatchIndirect.
Lest you think NVIDIA is just crazy here, it's a perfectly reasonable model if you assume that userspace is feeding the firmware. When that's happening, you just have a userspace thread that sits there and feeds the ringbuffer with whatever is next and you can marshal as much data through as you want. Sure, it'd be nice to have a 2nd level batch thing that gets launched from the FW ring and has all the individual launch commands but it's not at all necessary.
What does that mean from a
gpu_schedulerPoV? Basically, it means a variable packet size.What does this mean for implementation? IDK. One option would be to teach the scheduler about actual job sizes. Another would be to virtualize it and have another layer underneath the scheduler that does the actual feeding of the ring. Another would be to decrease the job size somewhat and then have the front-end submit as many jobs as it needs to service userspace and only put the out-fences on the last job. All the options kinda suck.
Yeah, agree. The job size Danilo suggested is still the least painful.
Christian.
~Faith
Matthew Brost
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
In reply to Danilo KrummrichQuoted 143 lines · Danilo Krummrich
Quoted 96 lines · Christian König
Am 16.08.23 um 18:33 schrieb Danilo Krummrich:
Quoted 88 lines · Christian König
Quoted 44 lines
Am 16.08.23 um 14:30 schrieb Danilo Krummrich:
Quoted 43 lines · Christian König
Quoted 41 lines
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 28 lines
Hi Matt,
Quoted 17 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this
seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if
targeting the same hardware
engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using
shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls
apart as the TDR expects submission order ==
completion order. Using a
dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.Quoted 7 lines
2. In XE submissions are done via programming a
ring buffer (circular
buffer), adrm_gpu_schedulerprovides a limit on
number of jobs, if the
limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow
control on the ring for free.Quoted 7 lines
In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).Quoted 8 lines
Quoted 5 lines
Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau
MAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.But that lets the hw run dry between submissions. That is usually a pretty horrible idea for performance.
Correct, that's the reason why I said it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the maximum size of a job.
Quoted 11 lines
Quoted 5 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
That doesn't sounds like a good idea to me. See we don't limit the number of submitted jobs based on the ring size, but rather we calculate the ring size based on the number of submitted jobs.
My point isn't really about whether we derive the ring size from the job limit or the other way around. It's more about the job size (or its maximum size) being arbitrary.
As mentioned in my reply to Matt:
"In Nouveau, userspace can submit an arbitrary amount of addresses of indirect bufferes containing the ring instructions. The ring on the kernel side takes the addresses of the indirect buffers rather than the instructions themself. Hence, technically there isn't really a limit on the amount of IBs submitted by a job except for the ring size."
So, my point is that I don't really want to limit the job size artificially just to be able to fit multiple jobs into the ring even if they're submitted at their "artificial" maximum size, but rather track how much of the ring the submitted job actually occupies.
Quoted 6 lines
Quoted 4 lines
In other words the
hw_submission_limitdefines the ring size, not the other way around. And you usually want thehw_submission_limitas low as possible for good scheduler granularity and to avoid extra overhead.I don't think you really mean "as low as possible", do you?
Quoted 7 lines
No, I do mean as low as possible or in other words as few as possible.
Ideally the scheduler would submit only the minimum amount of work to the hardware to keep the hardware busy. >
The hardware seems to work mostly the same for all vendors, but you somehow seem to think that filling the ring is somehow beneficial which is really not the case as far as I can see.I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
My proposal would be to just keep the
hw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.
Quoted 5 lines
Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.What do you think about that?
This seems reasonible to me and a very minimal change to the scheduler.
Matt
Quoted 73 lines
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
- Danilo
Quoted 66 lines
Regards,
Christian.Quoted 61 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 47 lines
Christian.
Quoted 43 lines
Quoted 42 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 34 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 17 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work
around the scaling issue,
use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop,
start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Christian König
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Am 18.08.23 um 05:08 schrieb Matthew Brost:
Quoted 132 lines · Danilo Krummrich
Quoted 131 lines · Christian König
Quoted 84 lines
Am 16.08.23 um 18:33 schrieb Danilo Krummrich:
Quoted 83 lines · Christian König
Quoted 41 lines
Am 16.08.23 um 14:30 schrieb Danilo Krummrich:
Quoted 40 lines · Christian König
Quoted 39 lines
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 27 lines
Hi Matt,
Quoted 17 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this
seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if
targeting the same hardware
engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using
shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls
apart as the TDR expects submission order ==
completion order. Using a
dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.Quoted 7 lines
2. In XE submissions are done via programming a
ring buffer (circular
buffer), adrm_gpu_schedulerprovides a limit on
number of jobs, if the
limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow
control on the ring for free.Quoted 7 lines
In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).Quoted 7 lines
Quoted 5 lines
Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau
MAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.But that lets the hw run dry between submissions. That is usually a pretty horrible idea for performance.
Correct, that's the reason why I said it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the maximum size of a job.
Quoted 8 lines
Quoted 3 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
That doesn't sounds like a good idea to me. See we don't limit the number of submitted jobs based on the ring size, but rather we calculate the ring size based on the number of submitted jobs.
My point isn't really about whether we derive the ring size from the job limit or the other way around. It's more about the job size (or its maximum size) being arbitrary.
As mentioned in my reply to Matt:
"In Nouveau, userspace can submit an arbitrary amount of addresses of indirect bufferes containing the ring instructions. The ring on the kernel side takes the addresses of the indirect buffers rather than the instructions themself. Hence, technically there isn't really a limit on the amount of IBs submitted by a job except for the ring size."
So, my point is that I don't really want to limit the job size artificially just to be able to fit multiple jobs into the ring even if they're submitted at their "artificial" maximum size, but rather track how much of the ring the submitted job actually occupies.
Quoted 4 lines
In other words the
hw_submission_limitdefines the ring size, not the other way around. And you usually want thehw_submission_limitas low as possible for good scheduler granularity and to avoid extra overhead.Quoted 8 lines
Quoted 1 line
I don't think you really mean "as low as possible", do you?
No, I do mean as low as possible or in other words as few as possible.
Ideally the scheduler would submit only the minimum amount of work to the hardware to keep the hardware busy. >
The hardware seems to work mostly the same for all vendors, but you somehow seem to think that filling the ring is somehow beneficial which is really not the case as far as I can see.I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
My proposal would be to just keep the
hw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.
Quoted 6 lines
Quoted 5 lines
Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.What do you think about that?
This seems reasonible to me and a very minimal change to the scheduler.
If you have a good use case for this then the approach sounds sane to me as well.
My question is rather how exactly does Nouveau comes to have this use case? Allowing the full ring size in the UAPI sounds a bit questionable.
Christian.
Quoted 63 lines
Matt
Quoted 60 lines
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
- Danilo
Quoted 54 lines
Regards,
Christian.Quoted 51 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 38 lines
Christian.
Quoted 36 lines
Quoted 36 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 31 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 16 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work
around the scaling issue,
use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop,
start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Matthew Brost
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 136 lines · Christian König
Am 18.08.23 um 05:08 schrieb Matthew Brost:
Quoted 135 lines · Danilo Krummrich
Quoted 134 lines · Christian König
Quoted 84 lines
Am 16.08.23 um 18:33 schrieb Danilo Krummrich:
Quoted 83 lines · Christian König
Quoted 41 lines
Am 16.08.23 um 14:30 schrieb Danilo Krummrich:
Quoted 40 lines · Christian König
Quoted 39 lines
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 27 lines
Hi Matt,
Quoted 17 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this
seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if
targeting the same hardware
engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using
shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls
apart as the TDR expects submission order ==
completion order. Using a
dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.Quoted 7 lines
2. In XE submissions are done via programming a
ring buffer (circular
buffer), adrm_gpu_schedulerprovides a limit on
number of jobs, if the
limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow
control on the ring for free.Quoted 7 lines
In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).Quoted 7 lines
Quoted 5 lines
Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau
MAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.But that lets the hw run dry between submissions. That is usually a pretty horrible idea for performance.
Correct, that's the reason why I said it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the maximum size of a job.
Quoted 8 lines
Quoted 3 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
That doesn't sounds like a good idea to me. See we don't limit the number of submitted jobs based on the ring size, but rather we calculate the ring size based on the number of submitted jobs.
My point isn't really about whether we derive the ring size from the job limit or the other way around. It's more about the job size (or its maximum size) being arbitrary.
As mentioned in my reply to Matt:
"In Nouveau, userspace can submit an arbitrary amount of addresses of indirect bufferes containing the ring instructions. The ring on the kernel side takes the addresses of the indirect buffers rather than the instructions themself. Hence, technically there isn't really a limit on the amount of IBs submitted by a job except for the ring size."
So, my point is that I don't really want to limit the job size artificially just to be able to fit multiple jobs into the ring even if they're submitted at their "artificial" maximum size, but rather track how much of the ring the submitted job actually occupies.
Quoted 4 lines
In other words the
hw_submission_limitdefines the ring size, not the other way around. And you usually want thehw_submission_limitas low as possible for good scheduler granularity and to avoid extra overhead.Quoted 8 lines
Quoted 1 line
I don't think you really mean "as low as possible", do you?
No, I do mean as low as possible or in other words as few as possible.
Ideally the scheduler would submit only the minimum amount of work to the hardware to keep the hardware busy. >
The hardware seems to work mostly the same for all vendors, but you somehow seem to think that filling the ring is somehow beneficial which is really not the case as far as I can see.I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
My proposal would be to just keep the
hw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.
Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
What do you think about that?
This seems reasonible to me and a very minimal change to the scheduler.
If you have a good use case for this then the approach sounds sane to me as well.
Xe does not have a use case as the difference between the minimum size of a job and the maximum is not all that large (maybe 100-192 bytes is the range) so the accounting of a unit of 1 per job is just fine for now even though it may waste space.
In Nouveau it seems like the min / max for size of job can vary wildly so it needs finer grained units to mke effective use of the ring space. Updating the scheduler to support this is rather trivial, hence no real opposition from me. Also I do see this valid use case that other driver or even perhaps Xe may use someday.
Quoted 3 lines
My question is rather how exactly does Nouveau comes to have this use case? Allowing the full ring size in the UAPI sounds a bit questionable.
I agree allowing the user completely fill the ring is a bit questionable, surely there has to be some upper limit. But lets say it allows 1-64 IB, that still IMO could be used to justify finer grained accouting in the DRM scheduler as stated above this make the difference between the min / max job quite large.
Matt
Quoted 66 lines
Christian.
Quoted 63 lines
Matt
Quoted 60 lines
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
- Danilo
Quoted 54 lines
Regards,
Christian.Quoted 51 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 38 lines
Christian.
Quoted 36 lines
Quoted 36 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 31 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 16 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work
around the scaling issue,
use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop,
start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 150 lines · Matthew Brost
Quoted 136 lines · Christian König
Am 18.08.23 um 05:08 schrieb Matthew Brost:
Quoted 135 lines · Danilo Krummrich
Quoted 134 lines · Christian König
Quoted 84 lines
Am 16.08.23 um 18:33 schrieb Danilo Krummrich:
Quoted 83 lines · Christian König
Quoted 41 lines
Am 16.08.23 um 14:30 schrieb Danilo Krummrich:
Quoted 40 lines · Christian König
Quoted 39 lines
Am 16.08.23 um 13:30 schrieb Danilo Krummrich:
Quoted 27 lines
Hi Matt,
Quoted 17 lines · Matthew Brost
In XE, the new Intel GPU driver, a choice has made to have a 1 to 1 mapping between a
drm_gpu_scheduleranddrm_sched_entity. At first this
seems a bit odd but let us explain the reasoning below.1. In XE the submission order from multiple
drm_sched_entityis not guaranteed to be the same completion even if
targeting the same hardware
engine. This is because in XE we have a firmware scheduler, the GuC, which allowed to reorder, timeslice, and preempt submissions. If a using
shareddrm_gpu_scheduleracross multipledrm_sched_entity, the TDR falls
apart as the TDR expects submission order ==
completion order. Using a
dedicateddrm_gpu_schedulerperdrm_sched_entitysolve this problem.Quoted 7 lines
2. In XE submissions are done via programming a
ring buffer (circular
buffer), adrm_gpu_schedulerprovides a limit on
number of jobs, if the
limit of number jobs is set toRING_SIZE/MAX_SIZE_PER_JOBwe get flow
control on the ring for free.Quoted 7 lines
In XE, where does the limitation of
MAX_SIZE_PER_JOBcome from?In Nouveau we currently do have such a limitation as well, but it is derived from the
RING_SIZE, henceRING_SIZE/MAX_SIZE_PER_JOBwould always be 1. However, I think most jobs won't actually utilize the whole ring.Well that should probably rather be
RING_SIZE/MAX_SIZE_PER_JOB=hw_submission_limit(or evenhw_submission_limit- 1 when the hw can't distinct full vs empty ring buffer).Quoted 7 lines
Quoted 5 lines
Not sure if I get you right, let me try to clarify what I was trying to say: I wanted to say that in Nouveau
MAX_SIZE_PER_JOBisn't really limited by anything other than theRING_SIZEand hence we'd never allow more than 1 active job.But that lets the hw run dry between submissions. That is usually a pretty horrible idea for performance.
Correct, that's the reason why I said it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the maximum size of a job.
Quoted 8 lines
Quoted 3 lines
However, it seems to be more efficient to base ring flow control on the actual size of each incoming job rather than the worst case, namely the maximum size of a job.
That doesn't sounds like a good idea to me. See we don't limit the number of submitted jobs based on the ring size, but rather we calculate the ring size based on the number of submitted jobs.
My point isn't really about whether we derive the ring size from the job limit or the other way around. It's more about the job size (or its maximum size) being arbitrary.
As mentioned in my reply to Matt:
"In Nouveau, userspace can submit an arbitrary amount of addresses of indirect bufferes containing the ring instructions. The ring on the kernel side takes the addresses of the indirect buffers rather than the instructions themself. Hence, technically there isn't really a limit on the amount of IBs submitted by a job except for the ring size."
So, my point is that I don't really want to limit the job size artificially just to be able to fit multiple jobs into the ring even if they're submitted at their "artificial" maximum size, but rather track how much of the ring the submitted job actually occupies.
Quoted 4 lines
In other words the
hw_submission_limitdefines the ring size, not the other way around. And you usually want thehw_submission_limitas low as possible for good scheduler granularity and to avoid extra overhead.Quoted 8 lines
Quoted 1 line
I don't think you really mean "as low as possible", do you?
No, I do mean as low as possible or in other words as few as possible.
Ideally the scheduler would submit only the minimum amount of work to the hardware to keep the hardware busy. >
The hardware seems to work mostly the same for all vendors, but you somehow seem to think that filling the ring is somehow beneficial which is really not the case as far as I can see.I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
My proposal would be to just keep the
hw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.Quoted 7 lines
Quoted 3 lines
Quoted 2 lines
What do you think about that?
This seems reasonible to me and a very minimal change to the scheduler.
If you have a good use case for this then the approach sounds sane to me as well.
Xe does not have a use case as the difference between the minimum size of a job and the maximum is not all that large (maybe 100-192 bytes is the range) so the accounting of a unit of 1 per job is just fine for now even though it may waste space.
Quoted 5 lines
In Nouveau it seems like the min / max for size of job can vary wildly so it needs finer grained units to mke effective use of the ring space. Updating the scheduler to support this is rather trivial, hence no real opposition from me. Also I do see this valid use case that other driver or even perhaps Xe may use someday.
Yes, exactly that.
Quoted 10 lines
Quoted 4 lines
My question is rather how exactly does Nouveau comes to have this use case? Allowing the full ring size in the UAPI sounds a bit questionable.
I agree allowing the user completely fill the ring is a bit questionable, surely there has to be some upper limit. But lets say it allows 1-64 IB, that still IMO could be used to justify finer grained accouting in the DRM scheduler as stated above this make the difference between the min / max job quite large.
Yes, I agree that limiting the job size to at least ring size half makes sense to guarantee a contiguous flow.
Quoted 70 lines
Matt
Quoted 66 lines
Christian.
Quoted 63 lines
Matt
Quoted 60 lines
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
- Danilo
Quoted 54 lines
Regards,
Christian.Quoted 51 lines
Because one really is the minimum if you want to do work at all, but as you mentioned above a job limit of one can let the ring run dry.
In the end my proposal comes down to tracking the actual size of a job rather than just assuming a pre-defined maximum job size, and hence a dynamic job limit.
I don't think this would hurt the scheduler granularity. In fact, it should even contribute to the desire of not letting the ring run dry even better. Especially for sequences of small jobs, where the current implementation might wrongly assume the ring is already full although actually there would still be enough space left.
Quoted 38 lines
Christian.
Quoted 36 lines
Quoted 36 lines
Otherwise your scheduler might just overwrite the ring buffer by pushing things to fast.
Christian.
Quoted 31 lines
Given that, it seems like it would be better to let the scheduler keep track of empty ring "slots" instead, such that the scheduler can deceide whether a subsequent job will still fit on the ring and if not re-evaluate once a previous job finished. Of course each submitted job would be required to carry the number of slots it requires on the ring.
What to you think of implementing this as alternative flow control mechanism? Implementation wise this could be a union with the existing
hw_submission_limit.- Danilo
Quoted 16 lines
A problem with this design is currently a
drm_gpu_scheduleruses a kthread for submission / job cleanup. This doesn't scale if a large number ofdrm_gpu_schedulerare used. To work
around the scaling issue,
use a worker rather than kthread for submission / job cleanup.v2:
- (Rob Clark) Fix msm build
- Pass in run work queue
v3:
- (Boris) don't have loop in worker
v4:
- (Tvrtko) break out submit ready, stop,
start helpers into own patchSigned-off-by Matthew Brost <[email protected]>
Boris Brezillon
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
In reply to Danilo KrummrichQuoted 25 lines · On Thu, 17 Aug 2023 13:13:31 +0200 Danilo Krummrich <[email protected]> wrote:
I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
Quoted 8 lines
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
I'm not entirely sure, but I think PowerVR is pretty close to your
description: jobs size is dynamic size, and the ring buffer size is picked by the driver at queue initialization time. What we did was to set hw_submission_limit to an arbitrarily high value of 64k (we could have used something like ringbuf_size/min_job_size instead), and then have the control flow implemented with ->prepare_job() [1] (CCCB is the PowerVR ring buffer). This allows us to maximize ring buffer utilization while still allowing dynamic-size jobs.
Quoted 9 lines
My proposal would be to just keep the
hw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.
Quoted 6 lines
What do you think about that?
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 37 lines · Boris Brezillon
Quoted 25 lines · On Thu, 17 Aug 2023 13:13:31 +0200 Danilo Krummrich <[email protected]> wrote:
I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
Quoted 8 lines
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
Quoted 8 lines
I'm not entirely sure, but I think PowerVR is pretty close to your
description: jobs size is dynamic size, and the ring buffer size is picked by the driver at queue initialization time. What we did was to sethw_submission_limitto an arbitrarily high value of 64k (we could have used something like ringbuf_size/min_job_size instead), and then have the control flow implemented with ->prepare_job() [1] (CCCB is the PowerVR ring buffer). This allows us to maximize ring buffer utilization while still allowing dynamic-size jobs.
I guess this would work, but I think it would be better to bake this in, especially if more drivers do have this need. I already have an implementation [1] for doing that in the scheduler. My plan was to push that as soon as Matt sends out V3.
[1] https://gitlab.freedesktop.org/nouvelles/kernel/-/commit/269f05d6a2255384badff8b008b3c32d640d2d95
Quoted 19 lines
Quoted 10 lines
My proposal would be to just keep the
hw_submission_limit(maybe rename it tosubmission_unit_limit) and add asubmission_unitsfield to structdrm_sched_job. By default a jobssubmission_unitsfield would be 0 and the scheduler would behave the exact same way as it does now.Accordingly, jobs with
submission_units> 1 would contribute more than one unit to thesubmission_unit_limit.Quoted 6 lines
What do you think about that?
Besides all that, you said that filling up the ring just enough to not let the HW run dry rather than filling it up entirely is desirable. Why do you think so? I tend to think that in most cases it shouldn't make difference.
Boris Brezillon
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 47 lines · On Tue, 12 Sep 2023 16:33:01 +0200 Danilo Krummrich <[email protected]> wrote:
Quoted 37 lines · Boris Brezillon
Quoted 25 lines · On Thu, 17 Aug 2023 13:13:31 +0200 Danilo Krummrich <[email protected]> wrote:
I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
Quoted 8 lines
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
Quoted 8 lines
I'm not entirely sure, but I think PowerVR is pretty close to your
description: jobs size is dynamic size, and the ring buffer size is picked by the driver at queue initialization time. What we did was to sethw_submission_limitto an arbitrarily high value of 64k (we could have used something like ringbuf_size/min_job_size instead), and then have the control flow implemented with ->prepare_job() [1] (CCCB is the PowerVR ring buffer). This allows us to maximize ring buffer utilization while still allowing dynamic-size jobs.
Quoted 6 lines
I guess this would work, but I think it would be better to bake this in, especially if more drivers do have this need. I already have an implementation [1] for doing that in the scheduler. My plan was to push that as soon as Matt sends out V3.
[1] https://gitlab.freedesktop.org/nouvelles/kernel/-/commit/269f05d6a2255384badff8b008b3c32d640d2d95
PowerVR's ->can_fit_in_ringbuf() logic is a bit more involved in that native fences waits are passed to the FW, and those add to the job size. When we know our job is ready for execution (all non-native deps are signaled), we evict already signaled native-deps (or native fences) to shrink the job size further more, but that's something we need to calculate late if we want the job size to be minimal. Of course, we can always over-estimate the job size, but if we go for a full-blown drm_sched integration, I wonder if it wouldn't be preferable to have a ->get_job_size() callback returning the number of units needed by job, and have the core pick 1 when the hook is not implemented.
Boris Brezillon
maintainerRe: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 57 lines · On Tue, 12 Sep 2023 16:49:09 +0200 Boris Brezillon…
Quoted 47 lines · On Tue, 12 Sep 2023 16:33:01 +0200 Danilo Krummrich <[email protected]> wrote:
Quoted 37 lines · Boris Brezillon
Quoted 25 lines · On Thu, 17 Aug 2023 13:13:31 +0200 Danilo Krummrich <[email protected]> wrote:
I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
Quoted 8 lines
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
Quoted 8 lines
I'm not entirely sure, but I think PowerVR is pretty close to your
description: jobs size is dynamic size, and the ring buffer size is picked by the driver at queue initialization time. What we did was to sethw_submission_limitto an arbitrarily high value of 64k (we could have used something like ringbuf_size/min_job_size instead), and then have the control flow implemented with ->prepare_job() [1] (CCCB is the PowerVR ring buffer). This allows us to maximize ring buffer utilization while still allowing dynamic-size jobs.Quoted 6 lines
I guess this would work, but I think it would be better to bake this in, especially if more drivers do have this need. I already have an implementation [1] for doing that in the scheduler. My plan was to push that as soon as Matt sends out V3.
[1] https://gitlab.freedesktop.org/nouvelles/kernel/-/commit/269f05d6a2255384badff8b008b3c32d640d2d95
Quoted 10 lines
PowerVR's ->can_fit_in_ringbuf() logic is a bit more involved in that native fences waits are passed to the FW, and those add to the job size. When we know our job is ready for execution (all non-native deps are signaled), we evict already signaled native-deps (or native fences) to shrink the job size further more, but that's something we need to calculate late if we want the job size to be minimal. Of course, we can always over-estimate the job size, but if we go for a full-blown
drm_schedintegration, I wonder if it wouldn't be preferable to have a ->get_job_size() callback returning the number of units needed by job, and have the core pick 1 when the hook is not implemented.
FWIW, I think last time I asked how to do that, I've been pointed to ->prepare_job() by someone (don't remember if it was Daniel or Christian), hence the PowerVR implementation. If that's still the preferred solution, there's some opportunity to have a generic layer to automate ringbuf utilization tracking and some helpers to prepare wait_for_ringbuf dma_fences that drivers could return from ->prepare_job() (those fences would then be signaled when the driver calls drm_ringbuf_job_done() and the next job waiting for ringbuf space now fits in the ringbuf).
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
Quoted 71 lines · Boris Brezillon
Quoted 57 lines · On Tue, 12 Sep 2023 16:49:09 +0200 Boris Brezillon…
Quoted 47 lines · On Tue, 12 Sep 2023 16:33:01 +0200 Danilo Krummrich <[email protected]> wrote:
Quoted 37 lines · Boris Brezillon
Quoted 25 lines · On Thu, 17 Aug 2023 13:13:31 +0200 Danilo Krummrich <[email protected]> wrote:
I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
Quoted 8 lines
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
Quoted 8 lines
I'm not entirely sure, but I think PowerVR is pretty close to your
description: jobs size is dynamic size, and the ring buffer size is picked by the driver at queue initialization time. What we did was to sethw_submission_limitto an arbitrarily high value of 64k (we could have used something like ringbuf_size/min_job_size instead), and then have the control flow implemented with ->prepare_job() [1] (CCCB is the PowerVR ring buffer). This allows us to maximize ring buffer utilization while still allowing dynamic-size jobs.Quoted 6 lines
I guess this would work, but I think it would be better to bake this in, especially if more drivers do have this need. I already have an implementation [1] for doing that in the scheduler. My plan was to push that as soon as Matt sends out V3.
[1] https://gitlab.freedesktop.org/nouvelles/kernel/-/commit/269f05d6a2255384badff8b008b3c32d640d2d95
Quoted 10 lines
PowerVR's ->can_fit_in_ringbuf() logic is a bit more involved in that native fences waits are passed to the FW, and those add to the job size. When we know our job is ready for execution (all non-native deps are signaled), we evict already signaled native-deps (or native fences) to shrink the job size further more, but that's something we need to calculate late if we want the job size to be minimal. Of course, we can always over-estimate the job size, but if we go for a full-blown
drm_schedintegration, I wonder if it wouldn't be preferable to have a ->get_job_size() callback returning the number of units needed by job, and have the core pick 1 when the hook is not implemented.
Quoted 10 lines
FWIW, I think last time I asked how to do that, I've been pointed to ->prepare_job() by someone (don't remember if it was Daniel or Christian), hence the PowerVR implementation. If that's still the preferred solution, there's some opportunity to have a generic layer to automate ringbuf utilization tracking and some helpers to prepare
wait_for_ringbufdma_fencesthat drivers could return from ->prepare_job() (those fences would then be signaled when the driver callsdrm_ringbuf_job_done()and the next job waiting for ringbuf space now fits in the ringbuf).
Not sure I like that, it's basically a different implementation to work around limitations of an implementation that is supposed to cover this case in general.
Re: [PATCH v2 1/9] drm/sched: Convert drm scheduler to use a work queue rather than kthread
In reply to Boris BrezillonQuoted 57 lines · Boris Brezillon
Quoted 47 lines · On Tue, 12 Sep 2023 16:33:01 +0200 Danilo Krummrich <[email protected]> wrote:
Quoted 37 lines · Boris Brezillon
Quoted 25 lines · On Thu, 17 Aug 2023 13:13:31 +0200 Danilo Krummrich <[email protected]> wrote:
I think that's a misunderstanding. I'm not trying to say that it is *always* beneficial to fill up the ring as much as possible. But I think it is under certain circumstances, exactly those circumstances I described for Nouveau.
As mentioned, in Nouveau the size of a job is only really limited by the ring size, which means that one job can (but does not necessarily) fill up the whole ring. We both agree that this is inefficient, because it potentially results into the HW run dry due to
hw_submission_limit== 1.I recognize you said that one should define
hw_submission_limitand adjust the other parts of the equation accordingly, the options I see are:(1) Increase the ring size while keeping the maximum job size.
(2) Decrease the maximum job size while keeping the ring size.
(3) Let the scheduler track the actual job size rather than the maximum job size.(1) results into potentially wasted ring memory, because we're not always reaching the maximum job size, but the scheduler assumes so.
(2) results into more IOCTLs from userspace for the same amount of IBs and more jobs result into more memory allocations and more work being submitted to the workqueue (with Matt's patches).
Quoted 8 lines
(3) doesn't seem to have any of those draw backs.
What would be your take on that?
Actually, if none of the other drivers is interested into a more precise way of keeping track of the ring utilization, I'd be totally fine to do it in a driver specific way. However, unfortunately I don't see how this would be possible.
Quoted 8 lines
I'm not entirely sure, but I think PowerVR is pretty close to your
description: jobs size is dynamic size, and the ring buffer size is picked by the driver at queue initialization time. What we did was to sethw_submission_limitto an arbitrarily high value of 64k (we could have used something like ringbuf_size/min_job_size instead), and then have the control flow implemented with ->prepare_job() [1] (CCCB is the PowerVR ring buffer). This allows us to maximize ring buffer utilization while still allowing dynamic-size jobs.Quoted 6 lines
I guess this would work, but I think it would be better to bake this in, especially if more drivers do have this need. I already have an implementation [1] for doing that in the scheduler. My plan was to push that as soon as Matt sends out V3.
[1] https://gitlab.freedesktop.org/nouvelles/kernel/-/commit/269f05d6a2255384badff8b008b3c32d640d2d95
Quoted 11 lines
PowerVR's ->can_fit_in_ringbuf() logic is a bit more involved in that native fences waits are passed to the FW, and those add to the job size. When we know our job is ready for execution (all non-native deps are signaled), we evict already signaled native-deps (or native fences) to shrink the job size further more, but that's something we need to calculate late if we want the job size to be minimal. Of course, we can always over-estimate the job size, but if we go for a full-blown
drm_schedintegration, I wonder if it wouldn't be preferable to have a ->get_job_size() callback returning the number of units needed by job, and have the core pick 1 when the hook is not implemented.
Sure, why not. Sounds reasonable to me.
Matthew Brost
maintainerPATCH v2 2/9 drm/sched: Move schedule policy to scheduler / entity
In reply to Matthew BrostRather than a global modparam for scheduling policy, move the scheduling policy to scheduler / entity so user can control each scheduler / entity policy.
v2:
- s/DRM_SCHED_POLICY_MAX/DRM_SCHED_POLICY_COUNT (Luben)
- Only include policy in scheduler (Luben)
--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 + drivers/gpu/drm/etnaviv/etnaviv_sched.c | 3 ++- drivers/gpu/drm/lima/lima_sched.c | 3 ++- drivers/gpu/drm/msm/msm_ringbuffer.c | 3 ++- drivers/gpu/drm/nouveau/nouveau_sched.c | 3 ++- drivers/gpu/drm/panfrost/panfrost_job.c | 3 ++- drivers/gpu/drm/scheduler/sched_entity.c | 24 ++++++++++++++++++---- drivers/gpu/drm/scheduler/sched_main.c | 23 +++++++++++++++------ drivers/gpu/drm/v3d/v3d_sched.c | 15 +++++++++----- include/drm/gpu_scheduler.h | 20 ++++++++++++------ 10 files changed, 72 insertions(+), 26 deletions(-)
drivers/gpu/drm/amd/amdgpu/amdgpu_device.c +1 -0
@@ -2335,6 +2335,7 @@ static int amdgpu_device_init_schedulers(struct amdgpu_device *adev) ring->num_hw_submission,0 , timeout, adev->reset_domain->wq, ring->sched_score, ring->name, +DRM_SCHED_POLICY_DEFAULT , adev->dev);if (r) { DRM_ERROR("Failed to create scheduler on ring %s.\n",
drivers/gpu/drm/etnaviv/etnaviv_sched.c +2 -1
@@ -136,7 +136,8 @@ int etnaviv_sched_init(struct etnaviv_gpu *gpu) ret =drm_sched_init (&gpu->sched, &etnaviv_sched_ops,NULL , etnaviv_hw_jobs_limit, etnaviv_job_hang_limit,msecs_to_jiffies (500 ),NULL ,NULL , -dev_name (gpu->dev), gpu->dev); +dev_name (gpu->dev),DRM_SCHED_POLICY_DEFAULT , + gpu->dev);if (ret)return ret;
drivers/gpu/drm/lima/lima_sched.c +2 -1
@@ -491,7 +491,8 @@ int lima_sched_pipe_init(struct lima_sched_pipe *pipe, const char *name)return drm_sched_init (&pipe->base, &lima_sched_ops,NULL ,1 , lima_job_hang_limit,msecs_to_jiffies (timeout),NULL , -NULL , name, pipe->ldev->dev); +NULL , name,DRM_SCHED_POLICY_DEFAULT , + pipe->ldev->dev); }void lima_sched_pipe_fini (struct lima_sched_pipe *pipe)
drivers/gpu/drm/msm/msm_ringbuffer.c +2 -1
@@ -95,7 +95,8 @@ struct msm_ringbuffer *msm_ringbuffer_new(struct msm_gpu *gpu, int id, ret =drm_sched_init (&ring->sched, &msm_sched_ops,NULL , num_hw_submissions,0 , sched_timeout, -NULL ,NULL ,to_msm_bo (ring->bo)->name, gpu->dev->dev); +NULL ,NULL ,to_msm_bo (ring->bo)->name, +DRM_SCHED_POLICY_DEFAULT , gpu->dev->dev);if (ret) { goto fail; }
drivers/gpu/drm/nouveau/nouveau_sched.c +2 -1
@@ -409,7 +409,8 @@ int nouveau_sched_init(struct nouveau_drm *drm)return drm_sched_init (sched, &nouveau_sched_ops,NULL ,NOUVEAU_SCHED_HW_SUBMISSIONS ,0 , job_hang_limit, -NULL ,NULL ,"nouveau_sched" , drm->dev->dev); +NULL ,NULL ,"nouveau_sched" , +DRM_SCHED_POLICY_DEFAULT , drm->dev->dev); }void nouveau_sched_fini (struct nouveau_drm *drm)
drivers/gpu/drm/panfrost/panfrost_job.c +2 -1
@@ -835,7 +835,8 @@ int panfrost_job_init(struct panfrost_device *pfdev) nentries,0 ,msecs_to_jiffies (JOB_TIMEOUT_MS ), pfdev->reset.wq, - NULL, "pan_js", pfdev->dev); +NULL ,"pan_js" ,DRM_SCHED_POLICY_DEFAULT , + pfdev->dev);if (ret) {dev_err (pfdev->dev,"Failed to create scheduler: %d." , ret); goto err_sched;
drivers/gpu/drm/scheduler/sched_entity.c +20 -4
@@ -33,6 +33,20 @@#define to_drm_sched_job (sched_job) \ container_of((sched_job), struct drm_sched_job, queue_node) +static bool bad_policies (struct drm_gpu_scheduler **sched_list, +unsigned int num_sched_list) +{ +enum drm_sched_policy sched_policy = sched_list[0 ]->sched_policy; +unsigned int i; + +/* All schedule policies must match */ +for (i =1 ; i < num_sched_list; ++i) +if (sched_policy != sched_list[i]->sched_policy) +return true; + +return false; +} +/** * drm_sched_entity_init - Init a context entity used by scheduler when * submit to HW ring.
@@ -62,7 +76,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, unsigned int num_sched_list, atomic_t *guilty) { -if (!(entity && sched_list && (num_sched_list ==0 || sched_list[0 ]))) +if (!(entity && sched_list && (num_sched_list ==0 || sched_list[0 ])) || +bad_policies (sched_list, num_sched_list))return -EINVAL ;memset (entity,0 ,sizeof (struct drm_sched_entity ));
@@ -486,7 +501,7 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)* Update the entity's location in the min heap according to * the timestamp of the next job, if any. */ -if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO ) { +if (entity->rq->sched->sched_policy ==DRM_SCHED_POLICY_FIFO ) {struct drm_sched_job *next; next =to_drm_sched_job (spsc_queue_peek (&entity->job_queue));
@@ -558,7 +573,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)void drm_sched_entity_push_job (struct drm_sched_job *sched_job) {struct drm_sched_entity *entity = sched_job->entity; -bool first; +bool first, fifo = entity->rq->sched->sched_policy == +DRM_SCHED_POLICY_FIFO ;ktime_t submit_ts;trace_drm_sched_job (sched_job, entity);
@@ -587,7 +603,7 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)drm_sched_rq_add_entity (entity->rq, entity);spin_unlock (&entity->rq_lock); -if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO ) +if (fifo)drm_sched_rq_update_fifo (entity, submit_ts);drm_sched_wakeup_if_can_queue (entity->rq->sched);
drivers/gpu/drm/scheduler/sched_main.c +17 -6
@@ -66,14 +66,14 @@#define to_drm_sched_job (sched_job) \ container_of((sched_job), struct drm_sched_job, queue_node) -int drm_sched_policy =DRM_SCHED_POLICY_FIFO ; +int default_drm_sched_policy =DRM_SCHED_POLICY_FIFO ;/** * DOC: sched_policy (int) * Used to override default entities scheduling policy in a run queue. */ -MODULE_PARM_DESC(sched_policy, "Specify the scheduling policy for entities on a run-queue, " __stringify(DRM_SCHED_POLICY_RR) " = Round Robin, " __stringify(DRM_SCHED_POLICY_FIFO) " = FIFO (default)."); -module_param_named (sched_policy, drm_sched_policy, int,0444 ); +MODULE_PARM_DESC(sched_policy, "Specify the default scheduling policy for entities on a run-queue, " __stringify(DRM_SCHED_POLICY_RR) " = Round Robin, " __stringify(DRM_SCHED_POLICY_FIFO) " = FIFO (default)."); +module_param_named (sched_policy, default_drm_sched_policy, int,0444 ); static __always_inline bool drm_sched_entity_compare_before(struct rb_node *a,const struct rb_node *b)
@@ -177,7 +177,7 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,if (rq->current_entity == entity) rq->current_entity =NULL ; -if (drm_sched_policy ==DRM_SCHED_POLICY_FIFO ) +if (rq->sched->sched_policy ==DRM_SCHED_POLICY_FIFO )drm_sched_rq_remove_fifo_locked (entity);spin_unlock (&rq->lock);
@@ -898,7 +898,7 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { - entity = drm_sched_policy ==DRM_SCHED_POLICY_FIFO ? + entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ?drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) :drm_sched_rq_select_entity_rr (&sched->sched_rq[i]);if (entity)
@@ -1071,6 +1071,7 @@ static void drm_sched_main(struct work_struct *w)* used * @score: optional score atomic shared with other schedulers * @name: name used for debugging +* @sched_policy: schedule policy * @dev: target &struct device * * Return 0 on success, otherwise error code.
@@ -1080,9 +1081,15 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,struct workqueue_struct *submit_wq, unsigned hw_submission, unsigned hang_limit, long timeout, struct workqueue_struct *timeout_wq, - atomic_t *score, const char *name, struct device *dev) + atomic_t *score, const char *name, + enum drm_sched_policy sched_policy, + struct device *dev) { int i; + +if (sched_policy >=DRM_SCHED_POLICY_COUNT ) +return -EINVAL ; + sched->ops = ops; sched->hw_submission_limit = hw_submission; sched->name = name;
@@ -1092,6 +1099,10 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, sched->hang_limit = hang_limit; sched->score = score ? score : &sched->_score; sched->dev = dev; +if (sched_policy ==DRM_SCHED_POLICY_DEFAULT ) + sched->sched_policy = default_drm_sched_policy; +else + sched->sched_policy = sched_policy;for (i =DRM_SCHED_PRIORITY_MIN ; i <DRM_SCHED_PRIORITY_COUNT ; i++)drm_sched_rq_init (sched, &sched->sched_rq[i]);
drivers/gpu/drm/v3d/v3d_sched.c +10 -5
@@ -391,7 +391,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_bin", v3d->drm.dev); +NULL ,"v3d_bin" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret)return ret;
@@ -399,7 +400,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_render", v3d->drm.dev); +ULL ,"v3d_render" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;
@@ -407,7 +409,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_tfu", v3d->drm.dev); +NULL ,"v3d_tfu" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;
@@ -416,7 +419,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_csd", v3d->drm.dev); +NULL ,"v3d_csd" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;
@@ -424,7 +428,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_cache_clean", v3d->drm.dev); +NULL ,"v3d_cache_clean" , + DRM_SCHED_POLICY_DEFAULT, v3d->drm.dev);if (ret) goto fail; }
include/drm/gpu_scheduler.h +14 -6
@@ -72,11 +72,15 @@ enum drm_sched_priority { DRM_SCHED_PRIORITY_UNSET = -2 }; -/* Used to chose between FIFO and RR jobs scheduling */ -extern int drm_sched_policy; - -#define DRM_SCHED_POLICY_RR 0 -#define DRM_SCHED_POLICY_FIFO 1 +/* Used to chose default scheduling policy*/ +extern int default_drm_sched_policy; + +enum drm_sched_policy { +DRM_SCHED_POLICY_DEFAULT , +DRM_SCHED_POLICY_RR , +DRM_SCHED_POLICY_FIFO , +DRM_SCHED_POLICY_COUNT , +};/** * struct drm_sched_entity - A wrapper around a job queue (typically
@@ -489,6 +493,7 @@ struct drm_sched_backend_ops {* guilty and it will no longer be considered for scheduling. * @score: score to help loadbalancer pick a idle sched * @_score: score used when the driver doesn't provide one +* @sched_policy: Schedule policy for scheduler * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. * @pause_submit: pause queuing of @work_submit on @submit_wq
@@ -514,6 +519,7 @@ struct drm_gpu_scheduler {int hang_limit;atomic_t *score;atomic_t _score; +enum drm_sched_policy sched_policy;bool ready;bool free_guilty;bool pause_submit;
@@ -525,7 +531,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,struct workqueue_struct *submit_wq, uint32_t hw_submission, unsigned hang_limit, long timeout, struct workqueue_struct *timeout_wq, - atomic_t *score, const char *name, struct device *dev); + atomic_t *score, const char *name, + enum drm_sched_policy sched_policy, + struct device *dev);void drm_sched_fini (struct drm_gpu_scheduler *sched); int drm_sched_job_init(struct drm_sched_job *job,
Signature · 1 line
2.34.1
Maira Canal
maintainerRe: [PATCH v2 2/9] drm/sched: Move schedule policy to scheduler / entity
Hi Matthew,
I'm not sure in which tree you plan to apply this series, but if you plan to apply it on drm-misc-next, it would be nice to rebase on top of it. It would make it easier for driver maintainers to review it.
Apart from the small nit below it, I tested the Xe tree on v3d and things seems to be running smoothly.
Quoted 22 lines · Matthew Brost
Rather than a global modparam for scheduling policy, move the scheduling policy to scheduler / entity so user can control each scheduler / entity policy.
v2:
- s/DRM_SCHED_POLICY_MAX/DRM_SCHED_POLICY_COUNT (Luben)
- Only include policy in scheduler (Luben)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 + drivers/gpu/drm/etnaviv/etnaviv_sched.c | 3 ++- drivers/gpu/drm/lima/lima_sched.c | 3 ++- drivers/gpu/drm/msm/msm_ringbuffer.c | 3 ++- drivers/gpu/drm/nouveau/nouveau_sched.c | 3 ++- drivers/gpu/drm/panfrost/panfrost_job.c | 3 ++- drivers/gpu/drm/scheduler/sched_entity.c | 24 ++++++++++++++++++---- drivers/gpu/drm/scheduler/sched_main.c | 23 +++++++++++++++------ drivers/gpu/drm/v3d/v3d_sched.c | 15 +++++++++----- include/drm/gpu_scheduler.h | 20 ++++++++++++------ 10 files changed, 72 insertions(+), 26 deletions(-)
[...]
Quoted 16 lines
drivers/gpu/drm/v3d/v3d_sched.c +2
-1@@ -391,7 +391,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_bin", v3d->drm.dev); +NULL ,"v3d_bin" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret)return ret;
drivers/gpu/drm/v3d/v3d_sched.c v3d_sched_init(struct v3d_dev *v3d) · line 403
Quoted 5 lines
drivers/gpu/drm/v3d/v3d_sched.c +1
-1@@ -399,7 +400,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_render", v3d->drm.dev); + ULL, "v3d_render", DRM_SCHED_POLICY_DEFAULT,
Small nit: s/ULL/NULL
Best Regards,
- Maíra
Quoted 86 lines
drivers/gpu/drm/v3d/v3d_sched.c +7
-3+ v3d->drm.dev);if (ret) goto fail;@@ -407,7 +409,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_tfu", v3d->drm.dev); +NULL ,"v3d_tfu" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;@@ -416,7 +419,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_csd", v3d->drm.dev); +NULL ,"v3d_csd" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;@@ -424,7 +428,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_cache_clean", v3d->drm.dev); +NULL ,"v3d_cache_clean" , + DRM_SCHED_POLICY_DEFAULT, v3d->drm.dev);if (ret) goto fail; }include/drm/gpu_scheduler.h +14
-6@@ -72,11 +72,15 @@ enum drm_sched_priority { DRM_SCHED_PRIORITY_UNSET = -2 }; -/* Used to chose between FIFO and RR jobs scheduling */ -extern int drm_sched_policy; - -#define DRM_SCHED_POLICY_RR 0 -#define DRM_SCHED_POLICY_FIFO 1 +/* Used to chose default scheduling policy*/ +extern int default_drm_sched_policy; + +enum drm_sched_policy { +DRM_SCHED_POLICY_DEFAULT , +DRM_SCHED_POLICY_RR , +DRM_SCHED_POLICY_FIFO , +DRM_SCHED_POLICY_COUNT , +};/** * struct drm_sched_entity - A wrapper around a job queue (typically @@ -489,6 +493,7 @@ struct drm_sched_backend_ops {* guilty and it will no longer be considered for scheduling. * @score: score to help loadbalancer pick a idle sched * @_score: score used when the driver doesn't provide one +* @sched_policy: Schedule policy for scheduler * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. * @pause_submit: pause queuing of @work_submit on @submit_wq @@ -514,6 +519,7 @@ struct drm_gpu_scheduler {int hang_limit;atomic_t *score;atomic_t _score; +enum drm_sched_policy sched_policy;bool ready;bool free_guilty;bool pause_submit;@@ -525,7 +531,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,struct workqueue_struct *submit_wq, uint32_t hw_submission, unsigned hang_limit, long timeout, struct workqueue_struct *timeout_wq, - atomic_t *score, const char *name, struct device *dev); + atomic_t *score, const char *name, + enum drm_sched_policy sched_policy, + struct device *dev);void drm_sched_fini (struct drm_gpu_scheduler *sched); int drm_sched_job_init(struct drm_sched_job *job,
Matthew Brost
maintainerRe: [PATCH v2 2/9] drm/sched: Move schedule policy to scheduler / entity
Quoted 6 lines · Maira Canal
Hi Matthew,
I'm not sure in which tree you plan to apply this series, but if you plan to apply it on drm-misc-next, it would be nice to rebase on top of it. It would make it easier for driver maintainers to review it.
I rebased this on drm-tip but forgot the first patch in the series.
Let me make sure I get this correct and will send a rev3 early next week.
Quoted 42 lines
Apart from the small nit below it, I tested the Xe tree on v3d and things seems to be running smoothly.
Quoted 22 lines · Matthew Brost
Rather than a global modparam for scheduling policy, move the scheduling policy to scheduler / entity so user can control each scheduler / entity policy.
v2:
- s/DRM_SCHED_POLICY_MAX/DRM_SCHED_POLICY_COUNT (Luben)
- Only include policy in scheduler (Luben)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/amd/amdgpu/amdgpu_device.c | 1 + drivers/gpu/drm/etnaviv/etnaviv_sched.c | 3 ++- drivers/gpu/drm/lima/lima_sched.c | 3 ++- drivers/gpu/drm/msm/msm_ringbuffer.c | 3 ++- drivers/gpu/drm/nouveau/nouveau_sched.c | 3 ++- drivers/gpu/drm/panfrost/panfrost_job.c | 3 ++- drivers/gpu/drm/scheduler/sched_entity.c | 24 ++++++++++++++++++---- drivers/gpu/drm/scheduler/sched_main.c | 23 +++++++++++++++------ drivers/gpu/drm/v3d/v3d_sched.c | 15 +++++++++----- include/drm/gpu_scheduler.h | 20 ++++++++++++------ 10 files changed, 72 insertions(+), 26 deletions(-)[...]
Quoted 13 lines
drivers/gpu/drm/v3d/v3d_sched.c +2
-1@@ -391,7 +391,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_bin_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_bin", v3d->drm.dev); +NULL ,"v3d_bin" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret)return ret;
Quoted 9 lines
drivers/gpu/drm/v3d/v3d_sched.c v3d_sched_init(struct v3d_dev *v3d) · line 403
Quoted 6 lines
drivers/gpu/drm/v3d/v3d_sched.c +1
-1@@ -399,7 +400,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_render_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_render", v3d->drm.dev); + ULL, "v3d_render", DRM_SCHED_POLICY_DEFAULT,Small nit: s/ULL/NULL
Yep, will fix.
Matt
Quoted 83 lines
Best Regards,
- MaíraQuoted 80 lines
drivers/gpu/drm/v3d/v3d_sched.c +7
-3+ v3d->drm.dev);if (ret) goto fail;@@ -407,7 +409,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_tfu_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_tfu", v3d->drm.dev); +NULL ,"v3d_tfu" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;@@ -416,7 +419,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_csd_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_csd", v3d->drm.dev); +NULL ,"v3d_csd" ,DRM_SCHED_POLICY_DEFAULT , + v3d->drm.dev);if (ret) goto fail;@@ -424,7 +428,8 @@ v3d_sched_init(struct v3d_dev *v3d) &v3d_cache_clean_sched_ops,NULL , hw_jobs_limit, job_hang_limit,msecs_to_jiffies (hang_limit_ms),NULL , - NULL, "v3d_cache_clean", v3d->drm.dev); +NULL ,"v3d_cache_clean" , + DRM_SCHED_POLICY_DEFAULT, v3d->drm.dev);if (ret) goto fail; }include/drm/gpu_scheduler.h +14
-6@@ -72,11 +72,15 @@ enum drm_sched_priority { DRM_SCHED_PRIORITY_UNSET = -2 }; -/* Used to chose between FIFO and RR jobs scheduling */ -extern int drm_sched_policy; - -#define DRM_SCHED_POLICY_RR 0 -#define DRM_SCHED_POLICY_FIFO 1 +/* Used to chose default scheduling policy*/ +extern int default_drm_sched_policy; + +enum drm_sched_policy { +DRM_SCHED_POLICY_DEFAULT , +DRM_SCHED_POLICY_RR , +DRM_SCHED_POLICY_FIFO , +DRM_SCHED_POLICY_COUNT , +};/** * struct drm_sched_entity - A wrapper around a job queue (typically @@ -489,6 +493,7 @@ struct drm_sched_backend_ops {* guilty and it will no longer be considered for scheduling. * @score: score to help loadbalancer pick a idle sched * @_score: score used when the driver doesn't provide one +* @sched_policy: Schedule policy for scheduler * @ready: marks if the underlying HW is ready to work * @free_guilty: A hit to time out handler to free the guilty job. * @pause_submit: pause queuing of @work_submit on @submit_wq @@ -514,6 +519,7 @@ struct drm_gpu_scheduler {int hang_limit;atomic_t *score;atomic_t _score; +enum drm_sched_policy sched_policy;bool ready;bool free_guilty;bool pause_submit;@@ -525,7 +531,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,struct workqueue_struct *submit_wq, uint32_t hw_submission, unsigned hang_limit, long timeout, struct workqueue_struct *timeout_wq, - atomic_t *score, const char *name, struct device *dev); + atomic_t *score, const char *name, + enum drm_sched_policy sched_policy, + struct device *dev);void drm_sched_fini (struct drm_gpu_scheduler *sched); int drm_sched_job_init(struct drm_sched_job *job,
Matthew Brost
maintainerPATCH v2 3/9 drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy
In reply to Matthew BrostDRM_SCHED_POLICY_SINGLE_ENTITY creates a 1 to 1 relationship between scheduler and entity. No priorities or run queue used in this mode. Intended for devices with firmware schedulers.
v2:
- Drop sched / rq union (Luben)
--- drivers/gpu/drm/scheduler/sched_entity.c | 69 ++++++++++++++++++------ drivers/gpu/drm/scheduler/sched_fence.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 63 +++++++++++++++++++--- include/drm/gpu_scheduler.h | 8 +++ 4 files changed, 119 insertions(+), 23 deletions(-)
drivers/gpu/drm/scheduler/sched_entity.c +54 -15
@@ -83,6 +83,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,memset (entity,0 ,sizeof (struct drm_sched_entity ));INIT_LIST_HEAD (&entity->list); entity->rq =NULL ; + entity->single_sched =NULL ; entity->guilty = guilty; entity->num_sched_list = num_sched_list; entity->priority = priority;
@@ -90,8 +91,17 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,RCU_INIT_POINTER (entity->last_scheduled,NULL );RB_CLEAR_NODE (&entity->rb_tree_node); -if (num_sched_list) - entity->rq = &sched_list[0 ]->sched_rq[entity->priority]; +if (num_sched_list) { +if (sched_list[0 ]->sched_policy != +DRM_SCHED_POLICY_SINGLE_ENTITY ) { + entity->rq = &sched_list[0 ]->sched_rq[entity->priority]; + }else { +if (num_sched_list !=1 || sched_list[0 ]->single_entity) +return -EINVAL ; + sched_list[0 ]->single_entity = entity; + entity->single_sched = sched_list[0 ]; + } + }init_completion (&entity->entity_idle);
@@ -124,7 +134,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list) { - WARN_ON(!num_sched_list || !sched_list); + WARN_ON(!num_sched_list || !sched_list || + !!entity->single_sched); entity->sched_list = sched_list; entity->num_sched_list = num_sched_list;
@@ -231,13 +242,15 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity) {struct drm_sched_job *job;struct dma_fence *prev; +bool single_entity = !!entity->single_sched; -if (!entity->rq) +if (!entity->rq && !single_entity)return ;spin_lock (&entity->rq_lock); entity->stopped = true; -drm_sched_rq_remove_entity (entity->rq, entity); +if (!single_entity) +drm_sched_rq_remove_entity (entity->rq, entity);spin_unlock (&entity->rq_lock);/* Make sure this entity is not used by the scheduler at the moment */
@@ -259,6 +272,20 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity)dma_fence_put (prev); } +/** +* drm_sched_entity_to_scheduler - Schedule entity to GPU scheduler +* @entity: scheduler entity +* +* Returns GPU scheduler for the entity +*/ +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity) +{ +bool single_entity = !!entity->single_sched; + +return single_entity ? entity->single_sched : entity->rq->sched; +} +/** * drm_sched_entity_flush - Flush a context entity *
@@ -276,11 +303,12 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)struct drm_gpu_scheduler *sched;struct task_struct *last_user;long ret = timeout; +bool single_entity = !!entity->single_sched; -if (!entity->rq) +if (!entity->rq && !single_entity)return 0 ; - sched = entity->rq->sched; + sched =drm_sched_entity_to_scheduler (entity);/** * The client will not queue more IBs during this fini, consume existing * queued IBs or discard them on SIGKILL
@@ -373,7 +401,7 @@ static void drm_sched_entity_wakeup(struct dma_fence *f, container_of(cb, struct drm_sched_entity, cb);drm_sched_entity_clear_dep (f, cb); -drm_sched_wakeup_if_can_queue (entity->rq->sched); +drm_sched_wakeup_if_can_queue (drm_sched_entity_to_scheduler (entity)); }/**
@@ -387,6 +415,8 @@ static void drm_sched_entity_wakeup(struct dma_fence *f,void drm_sched_entity_set_priority (struct drm_sched_entity *entity,enum drm_sched_priority priority) { +WARN_ON (!!entity->single_sched); +spin_lock (&entity->rq_lock); entity->priority = priority;spin_unlock (&entity->rq_lock);
@@ -399,7 +429,7 @@ EXPORT_SYMBOL(drm_sched_entity_set_priority);*/ static bool drm_sched_entity_add_dependency_cb (struct drm_sched_entity *entity) { -struct drm_gpu_scheduler *sched = entity->rq->sched; +struct drm_gpu_scheduler *sched =drm_sched_entity_to_scheduler (entity);struct dma_fence *fence = entity->dependency;struct drm_sched_fence *s_fence;
@@ -501,7 +531,8 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)* Update the entity's location in the min heap according to * the timestamp of the next job, if any. */ -if (entity->rq->sched->sched_policy ==DRM_SCHED_POLICY_FIFO ) { +if (drm_sched_entity_to_scheduler (entity)->sched_policy == +DRM_SCHED_POLICY_FIFO ) {struct drm_sched_job *next; next =to_drm_sched_job (spsc_queue_peek (&entity->job_queue));
@@ -524,6 +555,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)struct drm_gpu_scheduler *sched;struct drm_sched_rq *rq; +WARN_ON (!!entity->single_sched); +/* single possible engine and already selected */ if (!entity->sched_list)return ;
@@ -573,12 +606,13 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)void drm_sched_entity_push_job (struct drm_sched_job *sched_job) {struct drm_sched_entity *entity = sched_job->entity; -bool first, fifo = entity->rq->sched->sched_policy == -DRM_SCHED_POLICY_FIFO ; +bool single_entity = !!entity->single_sched; +bool first;ktime_t submit_ts;trace_drm_sched_job (sched_job, entity); -atomic_inc (entity->rq->sched->score); +if (!single_entity) +atomic_inc (entity->rq->sched->score);WRITE_ONCE (entity->last_user, current->group_leader);/*
@@ -591,6 +625,10 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)/* first job wakes up scheduler */ if (first) { +struct drm_gpu_scheduler *sched = +drm_sched_entity_to_scheduler (entity); +bool fifo = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ; +/* Add the entity to the run queue */ spin_lock (&entity->rq_lock);if (entity->stopped) {
@@ -600,13 +638,14 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)return ; } -drm_sched_rq_add_entity (entity->rq, entity); +if (!single_entity) +drm_sched_rq_add_entity (entity->rq, entity);spin_unlock (&entity->rq_lock);if (fifo)drm_sched_rq_update_fifo (entity, submit_ts); -drm_sched_wakeup_if_can_queue (entity->rq->sched); +drm_sched_wakeup_if_can_queue (sched); } }EXPORT_SYMBOL (drm_sched_entity_push_job);
drivers/gpu/drm/scheduler/sched_fence.c +1 -1
@@ -225,7 +225,7 @@ void drm_sched_fence_init(struct drm_sched_fence *fence, {unsigned seq; - fence->sched = entity->rq->sched; + fence->sched =drm_sched_entity_to_scheduler (entity); seq =atomic_inc_return (&entity->fence_seq);dma_fence_init (&fence->scheduled, &drm_sched_fence_ops_scheduled, &fence->lock, entity->fence_context, seq);
drivers/gpu/drm/scheduler/sched_main.c +56 -7
145 lines not on this page · raw message
include/drm/gpu_scheduler.h +8 -0
@@ -79,6 +79,7 @@ enum drm_sched_policy {DRM_SCHED_POLICY_DEFAULT ,DRM_SCHED_POLICY_RR ,DRM_SCHED_POLICY_FIFO , +DRM_SCHED_POLICY_SINGLE_ENTITY , DRM_SCHED_POLICY_COUNT, };
@@ -112,6 +113,9 @@ struct drm_sched_entity {*/ struct drm_sched_rq *rq; +/** @single_sched: Single scheduler */ +struct drm_gpu_scheduler *single_sched; +/** * @sched_list: *
@@ -473,6 +477,7 @@ struct drm_sched_backend_ops {* struct drm_gpu_scheduler - scheduler instance-specific data * * @ops: backend operations provided by the driver. +* @single_entity: Single entity for the scheduler * @hw_submission_limit: the max size of the hardware queue. * @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used.
@@ -503,6 +508,7 @@ struct drm_sched_backend_ops {*/ struct drm_gpu_scheduler {const struct drm_sched_backend_ops *ops; +struct drm_sched_entity *single_entity;uint32_t hw_submission_limit;long timeout;const char *name;
@@ -585,6 +591,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list, atomic_t *guilty); +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity);long drm_sched_entity_flush (struct drm_sched_entity *entity,long timeout);void drm_sched_entity_fini (struct drm_sched_entity *entity);void drm_sched_entity_destroy (struct drm_sched_entity *entity);
Signature · 1 line
2.34.1
Re: [PATCH v2 3/9] drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy
Quoted 62 lines · Matthew Brost
DRM_SCHED_POLICY_SINGLE_ENTITYcreates a 1 to 1 relationship between scheduler and entity. No priorities or run queue used in this mode. Intended for devices with firmware schedulers.v2:
- Drop sched / rq union (Luben)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_entity.c | 69 ++++++++++++++++++------ drivers/gpu/drm/scheduler/sched_fence.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 63 +++++++++++++++++++--- include/drm/gpu_scheduler.h | 8 +++ 4 files changed, 119 insertions(+), 23 deletions(-)drivers/gpu/drm/scheduler/sched_entity.c +15
-3@@ -83,6 +83,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,memset (entity,0 ,sizeof (struct drm_sched_entity ));INIT_LIST_HEAD (&entity->list); entity->rq =NULL ; + entity->single_sched =NULL ; entity->guilty = guilty; entity->num_sched_list = num_sched_list; entity->priority = priority;@@ -90,8 +91,17 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,RCU_INIT_POINTER (entity->last_scheduled,NULL );RB_CLEAR_NODE (&entity->rb_tree_node); -if (num_sched_list) - entity->rq = &sched_list[0 ]->sched_rq[entity->priority]; +if (num_sched_list) { +if (sched_list[0 ]->sched_policy != +DRM_SCHED_POLICY_SINGLE_ENTITY ) { + entity->rq = &sched_list[0 ]->sched_rq[entity->priority]; + }else { +if (num_sched_list !=1 || sched_list[0 ]->single_entity) +return -EINVAL ; + sched_list[0 ]->single_entity = entity; + entity->single_sched = sched_list[0 ]; + } + }init_completion (&entity->entity_idle);@@ -124,7 +134,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list) { - WARN_ON(!num_sched_list || !sched_list); + WARN_ON(!num_sched_list || !sched_list || + !!entity->single_sched); entity->sched_list = sched_list; entity->num_sched_list = num_sched_list;@@ -231,13 +242,15 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity) {struct drm_sched_job *job;struct dma_fence *prev; +bool single_entity = !!entity->single_sched;
drivers/gpu/drm/scheduler/sched_entity.c static void drm_sched_entity_kill(struct drm_sched_entity *entity) · line 253
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_entity.c +3
-2-if (!entity->rq) +if (!entity->rq && !single_entity)return ;spin_lock (&entity->rq_lock); entity->stopped = true; -drm_sched_rq_remove_entity (entity->rq, entity); +if (!single_entity) +drm_sched_rq_remove_entity (entity->rq, entity);
Looks like nothing prevents drm_sched_run_job_work() to fetch more jobs from the entity, hence if this is called for an entity still having queued up jobs and a still running scheduler, drm_sched_entity_kill() and drm_sched_run_job_work() would race for jobs, right?
Not sure if this is by intention because we don't expect the driver to drm_sched_entity_fini() as long as there are still queued up jobs. At least this is inconsistant to what drm_sched_entity_kill() does without DRM_SCHED_POLICY_SINGLE_ENTITY and should either be fixed or documented if we agree nothing else makes sense.
I think it also touches my question on how to tear down the scheduler once a ring is removed or deinitialized.
I know XE is going its own way in this respect, but I also feel like we're leaving drivers potentially being interested in DRM_SCHED_POLICY_SINGLE_ENTITY a bit alone on that. I think we should probably give drivers a bit more guidance on how to do that.
Currently, I see two approaches.
(1) Do what XE does, which means letting the scheduler run dry, which includes both the entity's job queue and the schedulers pending_list. While jobs from the entity's queue aren't pushing any more work to the ring on tear down, but just "flow through" to get freed up eventually. (Hopefully I got that right.)
(2) Kill the entity to cleanup jobs from the entity's queue, stop the scheduler and either just wait for pending jobs or signal them right away and finally free them up.
Actually there'd also be (3), which could be a mix of both, discard the entity's queued jobs, but let the pending_list run dry.
I'm not saying we should provide a whole bunch of infrastructure for drivers, e.g. for (1) as you've mentioned already as well, there is probably not much to generalize anyway. However, I think we should document the options drivers have to tear things down and do enough to enable drivers using any option (as long as we agree it is reasonable).
For Nouveau specifically, I'd probably like to go with (3).
- Danilo
Quoted 352 lines
drivers/gpu/drm/scheduler/sched_entity.c +36
-10spin_unlock (&entity->rq_lock);/* Make sure this entity is not used by the scheduler at the moment */ @@ -259,6 +272,20 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity)dma_fence_put (prev); } +/** +* drm_sched_entity_to_scheduler - Schedule entity to GPU scheduler +* @entity: scheduler entity +* +* Returns GPU scheduler for the entity +*/ +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity) +{ +bool single_entity = !!entity->single_sched; + +return single_entity ? entity->single_sched : entity->rq->sched; +} +/** * drm_sched_entity_flush - Flush a context entity * @@ -276,11 +303,12 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout)struct drm_gpu_scheduler *sched;struct task_struct *last_user;long ret = timeout; +bool single_entity = !!entity->single_sched; -if (!entity->rq) +if (!entity->rq && !single_entity)return 0 ; - sched = entity->rq->sched; + sched =drm_sched_entity_to_scheduler (entity);/** * The client will not queue more IBs during this fini, consume existing * queued IBs or discard them on SIGKILL @@ -373,7 +401,7 @@ static void drm_sched_entity_wakeup(struct dma_fence *f, container_of(cb, struct drm_sched_entity, cb);drm_sched_entity_clear_dep (f, cb); -drm_sched_wakeup_if_can_queue (entity->rq->sched); +drm_sched_wakeup_if_can_queue (drm_sched_entity_to_scheduler (entity)); }/** @@ -387,6 +415,8 @@ static void drm_sched_entity_wakeup(struct dma_fence *f,void drm_sched_entity_set_priority (struct drm_sched_entity *entity,enum drm_sched_priority priority) { +WARN_ON (!!entity->single_sched); +spin_lock (&entity->rq_lock); entity->priority = priority;spin_unlock (&entity->rq_lock);@@ -399,7 +429,7 @@ EXPORT_SYMBOL(drm_sched_entity_set_priority);*/ static bool drm_sched_entity_add_dependency_cb (struct drm_sched_entity *entity) { -struct drm_gpu_scheduler *sched = entity->rq->sched; +struct drm_gpu_scheduler *sched =drm_sched_entity_to_scheduler (entity);struct dma_fence *fence = entity->dependency;struct drm_sched_fence *s_fence;@@ -501,7 +531,8 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)* Update the entity's location in the min heap according to * the timestamp of the next job, if any. */ -if (entity->rq->sched->sched_policy ==DRM_SCHED_POLICY_FIFO ) { +if (drm_sched_entity_to_scheduler (entity)->sched_policy == +DRM_SCHED_POLICY_FIFO ) {struct drm_sched_job *next; next =to_drm_sched_job (spsc_queue_peek (&entity->job_queue));@@ -524,6 +555,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)struct drm_gpu_scheduler *sched;struct drm_sched_rq *rq; +WARN_ON (!!entity->single_sched); +/* single possible engine and already selected */ if (!entity->sched_list)return ;@@ -573,12 +606,13 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity)void drm_sched_entity_push_job (struct drm_sched_job *sched_job) {struct drm_sched_entity *entity = sched_job->entity; -bool first, fifo = entity->rq->sched->sched_policy == -DRM_SCHED_POLICY_FIFO ; +bool single_entity = !!entity->single_sched; +bool first;ktime_t submit_ts;trace_drm_sched_job (sched_job, entity); -atomic_inc (entity->rq->sched->score); +if (!single_entity) +atomic_inc (entity->rq->sched->score);WRITE_ONCE (entity->last_user, current->group_leader);/* @@ -591,6 +625,10 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)/* first job wakes up scheduler */ if (first) { +struct drm_gpu_scheduler *sched = +drm_sched_entity_to_scheduler (entity); +bool fifo = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ; +/* Add the entity to the run queue */ spin_lock (&entity->rq_lock);if (entity->stopped) {@@ -600,13 +638,14 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job)return ; } -drm_sched_rq_add_entity (entity->rq, entity); +if (!single_entity) +drm_sched_rq_add_entity (entity->rq, entity);spin_unlock (&entity->rq_lock);if (fifo)drm_sched_rq_update_fifo (entity, submit_ts); -drm_sched_wakeup_if_can_queue (entity->rq->sched); +drm_sched_wakeup_if_can_queue (sched); } }EXPORT_SYMBOL (drm_sched_entity_push_job);drivers/gpu/drm/scheduler/sched_fence.c +1
-1@@ -225,7 +225,7 @@ void drm_sched_fence_init(struct drm_sched_fence *fence, {unsigned seq; - fence->sched = entity->rq->sched; + fence->sched =drm_sched_entity_to_scheduler (entity); seq =atomic_inc_return (&entity->fence_seq);dma_fence_init (&fence->scheduled, &drm_sched_fence_ops_scheduled, &fence->lock, entity->fence_context, seq);drivers/gpu/drm/scheduler/sched_main.c +56
-7@@ -32,7 +32,8 @@* backend operations to the scheduler like submitting a job to hardware run queue, * returning the dependencies of a job etc. * -* The organisation of the scheduler is the following: +* The organisation of the scheduler is the following for scheduling policies +* DRM_SCHED_POLICY_RR and DRM_SCHED_POLICY_FIFO: * * 1. Each hw run queue has one scheduler * 2. Each scheduler has multiple run queues with different priorities @@ -43,6 +44,23 @@* * The jobs in a entity are always scheduled in the order that they were pushed. * +* The organisation of the scheduler is the following for scheduling policy +* DRM_SCHED_POLICY_SINGLE_ENTITY: +* +* 1. One to one relationship between scheduler and entity +* 2. No priorities implemented per scheduler (single job queue) +* 3. No run queues in scheduler rather jobs are directly dequeued from entity +* 4. The entity maintains a queue of jobs that will be scheduled on the +* hardware +* +* The jobs in a entity are always scheduled in the order that they were pushed +* regardless of scheduling policy. +* +* A policy of DRM_SCHED_POLICY_RR or DRM_SCHED_POLICY_FIFO is expected to used +* when the KMD is scheduling directly on the hardware while a scheduling policy +* of DRM_SCHED_POLICY_SINGLE_ENTITY is expected to be used when there is a +* firmware scheduler. +* * Note that once a job was taken from the entities queue and pushed to the * hardware, i.e. the pending queue, the entity must not be referenced anymore * through the jobs entity pointer. @@ -96,6 +114,8 @@ static inline void drm_sched_rq_remove_fifo_locked(struct drm_sched_entity *entivoid drm_sched_rq_update_fifo (struct drm_sched_entity *entity,ktime_t ts) { +WARN_ON (!!entity->single_sched); +/* * Both locks need to be grabbed, one to protect from entity->rq change * for entity from within concurrent drm_sched_entity_select_rq and the @@ -126,6 +146,8 @@ void drm_sched_rq_update_fifo(struct drm_sched_entity *entity, ktime_t ts)static void drm_sched_rq_init (struct drm_gpu_scheduler *sched,struct drm_sched_rq *rq) { +WARN_ON (sched->sched_policy ==DRM_SCHED_POLICY_SINGLE_ENTITY ); +spin_lock_init (&rq->lock);INIT_LIST_HEAD (&rq->entities); rq->rb_tree_root =RB_ROOT_CACHED ;@@ -144,6 +166,8 @@ static void drm_sched_rq_init(struct drm_gpu_scheduler *sched,void drm_sched_rq_add_entity (struct drm_sched_rq *rq,struct drm_sched_entity *entity) { +WARN_ON (!!entity->single_sched); +if (!list_empty (&entity->list))return ;@@ -166,6 +190,8 @@ void drm_sched_rq_add_entity(struct drm_sched_rq *rq,void drm_sched_rq_remove_entity (struct drm_sched_rq *rq,struct drm_sched_entity *entity) { +WARN_ON (!!entity->single_sched); +if (list_empty (&entity->list))return ;@@ -641,7 +667,7 @@ int drm_sched_job_init(struct drm_sched_job *job, struct drm_sched_entity *entity, void *owner) { -if (!entity->rq) +if (!entity->rq && !entity->single_sched)return -ENOENT ; job->entity = entity;@@ -674,13 +700,16 @@ void drm_sched_job_arm(struct drm_sched_job *job) {struct drm_gpu_scheduler *sched;struct drm_sched_entity *entity = job->entity; +bool single_entity = !!entity->single_sched;BUG_ON (!entity); -drm_sched_entity_select_rq (entity); - sched = entity->rq->sched; +if (!single_entity) +drm_sched_entity_select_rq (entity); + sched =drm_sched_entity_to_scheduler (entity); job->sched = sched; - job->s_priority = entity->rq - sched->sched_rq; +if (!single_entity) + job->s_priority = entity->rq - sched->sched_rq; job->id =atomic64_inc_return (&sched->job_id_count);drm_sched_fence_init (job->s_fence, job->entity);@@ -896,6 +925,13 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)if (!drm_sched_can_queue (sched))return NULL ; +if (sched->single_entity) { +if (drm_sched_entity_is_ready (sched->single_entity)) +return sched->single_entity; + +return NULL ; + } +/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy == DRM_SCHED_POLICY_FIFO ?@@ -1091,6 +1127,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,return -EINVAL ; sched->ops = ops; + sched->single_entity =NULL ; sched->hw_submission_limit = hw_submission; sched->name = name; sched->submit_wq = submit_wq ? : system_wq;@@ -1103,7 +1140,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, sched->sched_policy = default_drm_sched_policy; else sched->sched_policy = sched_policy; -for (i =DRM_SCHED_PRIORITY_MIN ; i <DRM_SCHED_PRIORITY_COUNT ; i++) +for (i =DRM_SCHED_PRIORITY_MIN ; sched_policy != +DRM_SCHED_POLICY_SINGLE_ENTITY && i <DRM_SCHED_PRIORITY_COUNT ; + i++)drm_sched_rq_init (sched, &sched->sched_rq[i]);init_waitqueue_head (&sched->job_scheduled);@@ -1135,7 +1174,15 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched)drm_sched_submit_stop (sched); -for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { +if (sched->single_entity) { +spin_lock (&sched->single_entity->rq_lock); + sched->single_entity->stopped = true; +spin_unlock (&sched->single_entity->rq_lock); + } + +for (i =DRM_SCHED_PRIORITY_COUNT -1 ; sched->sched_policy != +DRM_SCHED_POLICY_SINGLE_ENTITY && i >=DRM_SCHED_PRIORITY_MIN ; + i--) {struct drm_sched_rq *rq = &sched->sched_rq[i];spin_lock (&rq->lock);@@ -1176,6 +1223,8 @@ void drm_sched_increase_karma(struct drm_sched_job *bad)struct drm_sched_entity *entity;struct drm_gpu_scheduler *sched = bad->sched; +WARN_ON (sched->sched_policy ==DRM_SCHED_POLICY_SINGLE_ENTITY ); +/* don't change @bad's karma if it's from KERNEL RQ, * because sometimes GPU hang would cause kernel jobs (like VM updating jobs) * corrupt but keep in mind that kernel jobs always considered good. include/drm/gpu_scheduler.h +8
-0@@ -79,6 +79,7 @@ enum drm_sched_policy {DRM_SCHED_POLICY_DEFAULT ,DRM_SCHED_POLICY_RR ,DRM_SCHED_POLICY_FIFO , +DRM_SCHED_POLICY_SINGLE_ENTITY , DRM_SCHED_POLICY_COUNT, };@@ -112,6 +113,9 @@ struct drm_sched_entity {*/ struct drm_sched_rq *rq; +/** @single_sched: Single scheduler */ +struct drm_gpu_scheduler *single_sched; +/** * @sched_list: * @@ -473,6 +477,7 @@ struct drm_sched_backend_ops {* struct drm_gpu_scheduler - scheduler instance-specific data * * @ops: backend operations provided by the driver. +* @single_entity: Single entity for the scheduler * @hw_submission_limit: the max size of the hardware queue. * @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used. @@ -503,6 +508,7 @@ struct drm_sched_backend_ops {*/ struct drm_gpu_scheduler {const struct drm_sched_backend_ops *ops; +struct drm_sched_entity *single_entity;uint32_t hw_submission_limit;long timeout;const char *name;@@ -585,6 +591,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity,struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list, atomic_t *guilty); +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity);long drm_sched_entity_flush (struct drm_sched_entity *entity,long timeout);void drm_sched_entity_fini (struct drm_sched_entity *entity);void drm_sched_entity_destroy (struct drm_sched_entity *entity);
Re: [PATCH v2 3/9] drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy
Quoted 63 lines · Danilo Krummrich
Quoted 62 lines · Matthew Brost
DRM_SCHED_POLICY_SINGLE_ENTITYcreates a 1 to 1 relationship between scheduler and entity. No priorities or run queue used in this mode. Intended for devices with firmware schedulers.v2:
- Drop sched / rq union (Luben)Signed-off-by Matthew Brost <[email protected]>---
drivers/gpu/drm/scheduler/sched_entity.c| 69 ++++++++++++++++++------ drivers/gpu/drm/scheduler/sched_fence.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 63 +++++++++++++++++++--- include/drm/gpu_scheduler.h | 8 +++4 files changed, 119 insertions(+), 23 deletions(-)drivers/gpu/drm/scheduler/sched_entity.c +16
-4@@ -83,6 +83,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, memset(entity, 0, sizeof(struct drm_sched_entity)); INIT_LIST_HEAD(&entity->list); entity->rq = NULL; + entity->single_sched = NULL; entity->guilty = guilty; entity->num_sched_list = num_sched_list; entity->priority = priority;@@ -90,8 +91,17 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, RCU_INIT_POINTER(entity->last_scheduled, NULL); RB_CLEAR_NODE(&entity->rb_tree_node); - if(num_sched_list) - entity->rq = &sched_list[0]->sched_rq[entity->priority]; + if (num_sched_list) { + if (sched_list[0]->sched_policy != + DRM_SCHED_POLICY_SINGLE_ENTITY) { + entity->rq = &sched_list[0]->sched_rq[entity->priority]; + } else { + if (num_sched_list != 1 || sched_list[0]->single_entity) + return -EINVAL; + sched_list[0]->single_entity = entity; + entity->single_sched = sched_list[0]; + } + } init_completion(&entity->entity_idle);@@ -124,7 +134,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity, struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list) { - WARN_ON(!num_sched_list || !sched_list); + WARN_ON(!num_sched_list || !sched_list || + !!entity->single_sched); entity->sched_list = sched_list; entity->num_sched_list = num_sched_list;@@ -231,13 +242,15 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity) { struct drm_sched_job *job; struct dma_fence *prev; + bool single_entity = !!entity->single_sched; - if (!entity->rq) + if (!entity->rq && !single_entity) return; spin_lock(&entity->rq_lock);
Quoted 8 lines
drivers/gpu/drm/scheduler/sched_entity.c static void drm_sched_entity_kill(struct drm_sched_entity *entity) · line 251
Quoted 4 lines
drivers/gpu/drm/scheduler/sched_entity.c +2
-1entity->stopped = true; - drm_sched_rq_remove_entity(entity->rq, entity); + if (!single_entity) + drm_sched_rq_remove_entity(entity->rq, entity);Looks like nothing prevents
drm_sched_run_job_work()to fetch more jobs from the entity, hence if this is called for an entity still having queued up jobs and a still running scheduler,drm_sched_entity_kill()anddrm_sched_run_job_work()would race for jobs, right?
I worked around this with:
drivers/gpu/drm/scheduler/sched_main.c +2 -1
@@ -1025,7 +1025,8 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched, bool dequeue)return NULL ;if (sched->single_entity) { -if (drm_sched_entity_is_ready (sched->single_entity)) +if (drm_sched_entity_is_ready (sched->single_entity) && + !READ_ONCE (sched->single_entity->stopped))return sched->single_entity;return NULL ;
Quoted 355 lines
Not sure if this is by intention because we don't expect the driver to
drm_sched_entity_fini()as long as there are still queued up jobs. At least this is inconsistant to whatdrm_sched_entity_kill()does withoutDRM_SCHED_POLICY_SINGLE_ENTITYand should either be fixed or documented if we agree nothing else makes sense.I think it also touches my question on how to tear down the scheduler once a ring is removed or deinitialized.
I know XE is going its own way in this respect, but I also feel like we're leaving drivers potentially being interested in
DRM_SCHED_POLICY_SINGLE_ENTITYa bit alone on that. I think we should probably give drivers a bit more guidance on how to do that.Currently, I see two approaches.
(1) Do what XE does, which means letting the scheduler run dry, which includes both the entity's job queue and the schedulers
pending_list. While jobs from the entity's queue aren't pushing any more work to the ring on tear down, but just "flow through" to get freed up eventually. (Hopefully I got that right.)(2) Kill the entity to cleanup jobs from the entity's queue, stop the scheduler and either just wait for pending jobs or signal them right away and finally free them up.
Actually there'd also be (3), which could be a mix of both, discard the entity's queued jobs, but let the
pending_listrun dry.I'm not saying we should provide a whole bunch of infrastructure for drivers, e.g. for (1) as you've mentioned already as well, there is probably not much to generalize anyway. However, I think we should document the options drivers have to tear things down and do enough to enable drivers using any option (as long as we agree it is reasonable).
For Nouveau specifically, I'd probably like to go with (3).
- Danilo
Quoted 320 lines
drivers/gpu/drm/scheduler/sched_entity.c +36
-10spin_unlock(&entity->rq_lock); /* Make sure this entity is not used by the scheduler at the moment */@@ -259,6 +272,20 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity) dma_fence_put(prev); } +/** +* drm_sched_entity_to_scheduler - Schedule entity to GPU scheduler +* @entity: scheduler entity +* +* Returns GPU scheduler for the entity +*/ +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity) +{ + bool single_entity = !!entity->single_sched; + + return single_entity ? entity->single_sched : entity->rq->sched; +} + /*** drm_sched_entity_flush - Flush a context entity * @@ -276,11 +303,12 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout) struct drm_gpu_scheduler *sched; struct task_struct *last_user; long ret = timeout; + bool single_entity = !!entity->single_sched; - if (!entity->rq) + if (!entity->rq && !single_entity) return 0; - sched = entity->rq->sched; + sched = drm_sched_entity_to_scheduler(entity); /*** The client will not queue more IBs during this fini, consume existing * queued IBs or discard them on SIGKILL @@ -373,7 +401,7 @@ static void drm_sched_entity_wakeup(struct dma_fence *f, container_of(cb, struct drm_sched_entity, cb); drm_sched_entity_clear_dep(f, cb); - drm_sched_wakeup_if_can_queue(entity->rq->sched); + drm_sched_wakeup_if_can_queue(drm_sched_entity_to_scheduler(entity)); } /**@@ -387,6 +415,8 @@ static void drm_sched_entity_wakeup(struct dma_fence *f, void drm_sched_entity_set_priority(struct drm_sched_entity *entity, enum drm_sched_priority priority) { + WARN_ON(!!entity->single_sched); + spin_lock(&entity->rq_lock); entity->priority = priority; spin_unlock(&entity->rq_lock);@@ -399,7 +429,7 @@ EXPORT_SYMBOL(drm_sched_entity_set_priority);*/ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity) { - struct drm_gpu_scheduler *sched = entity->rq->sched; + struct drm_gpu_scheduler *sched = drm_sched_entity_to_scheduler(entity); struct dma_fence *fence = entity->dependency; struct drm_sched_fence *s_fence;@@ -501,7 +531,8 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)* Update the entity's location in the min heap according to * the timestamp of the next job, if any. */ - if (entity->rq->sched->sched_policy == DRM_SCHED_POLICY_FIFO) { + if (drm_sched_entity_to_scheduler(entity)->sched_policy == + DRM_SCHED_POLICY_FIFO) { struct drm_sched_job *next; next = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));@@ -524,6 +555,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity) struct drm_gpu_scheduler *sched; struct drm_sched_rq *rq; + WARN_ON(!!entity->single_sched); + /* single possible engine and already selected */ if (!entity->sched_list) return;@@ -573,12 +606,13 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity) void drm_sched_entity_push_job(struct drm_sched_job *sched_job) { struct drm_sched_entity *entity = sched_job->entity; - bool first, fifo = entity->rq->sched->sched_policy == - DRM_SCHED_POLICY_FIFO; + bool single_entity = !!entity->single_sched; + bool first; ktime_t submit_ts; trace_drm_sched_job(sched_job, entity); - atomic_inc(entity->rq->sched->score); + if (!single_entity) + atomic_inc(entity->rq->sched->score); WRITE_ONCE(entity->last_user, current->group_leader); /*@@ -591,6 +625,10 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job) /* first job wakes up scheduler */ if (first) { + struct drm_gpu_scheduler *sched = + drm_sched_entity_to_scheduler(entity); + bool fifo = sched->sched_policy == DRM_SCHED_POLICY_FIFO; + /* Add the entity to the run queue */ spin_lock(&entity->rq_lock); if (entity->stopped) {@@ -600,13 +638,14 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job) return; } - drm_sched_rq_add_entity(entity->rq, entity); + if (!single_entity) + drm_sched_rq_add_entity(entity->rq, entity); spin_unlock(&entity->rq_lock); if (fifo) drm_sched_rq_update_fifo(entity, submit_ts); - drm_sched_wakeup_if_can_queue(entity->rq->sched); + drm_sched_wakeup_if_can_queue(sched); } } EXPORT_SYMBOL(drm_sched_entity_push_job);drivers/gpu/drm/scheduler/sched_fence.c +1
-1@@ -225,7 +225,7 @@ void drm_sched_fence_init(struct drm_sched_fence *fence, { unsigned seq; - fence->sched = entity->rq->sched; + fence->sched = drm_sched_entity_to_scheduler(entity); seq = atomic_inc_return(&entity->fence_seq); dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled, &fence->lock, entity->fence_context, seq);drivers/gpu/drm/scheduler/sched_main.c +41
-5@@ -32,7 +32,8 @@ * backend operations to the scheduler like submitting a job to hardware run queue, * returning the dependencies of a job etc. * - * The organisation of the scheduler is the following: + * The organisation of the scheduler is the following for scheduling policies + * DRM_SCHED_POLICY_RR and DRM_SCHED_POLICY_FIFO: * * 1. Each hw run queue has one scheduler * 2. Each scheduler has multiple run queues with different priorities@@ -43,6 +44,23 @@ * * The jobs in a entity are always scheduled in the order that they were pushed. * + * The organisation of the scheduler is the following for scheduling policy + * DRM_SCHED_POLICY_SINGLE_ENTITY: + * + * 1. One to one relationship between scheduler and entity + * 2. No priorities implemented per scheduler (single job queue) + * 3. No run queues in scheduler rather jobs are directly dequeued from entity + * 4. The entity maintains a queue of jobs that will be scheduled on the + * hardware + * + * The jobs in a entity are always scheduled in the order that they were pushed + * regardless of scheduling policy. + * + * A policy of DRM_SCHED_POLICY_RR or DRM_SCHED_POLICY_FIFO is expected to used + * when the KMD is scheduling directly on the hardware while a scheduling policy + * of DRM_SCHED_POLICY_SINGLE_ENTITY is expected to be used when there is a + * firmware scheduler. + * * Note that once a job was taken from the entities queue and pushed to the * hardware, i.e. the pending queue, the entity must not be referenced anymore * through the jobs entity pointer.@@ -96,6 +114,8 @@ static inline void drm_sched_rq_remove_fifo_locked(struct drm_sched_entity *enti void drm_sched_rq_update_fifo(struct drm_sched_entity *entity, ktime_t ts) { + WARN_ON(!!entity->single_sched); + /** Both locks need to be grabbed, one to protect from entity->rq change * for entity from within concurrent drm_sched_entity_select_rq and the @@ -126,6 +146,8 @@ void drm_sched_rq_update_fifo(struct drm_sched_entity *entity, ktime_t ts) static void drm_sched_rq_init(struct drm_gpu_scheduler *sched, struct drm_sched_rq *rq) { + WARN_ON(sched->sched_policy == DRM_SCHED_POLICY_SINGLE_ENTITY); + spin_lock_init(&rq->lock); INIT_LIST_HEAD(&rq->entities); rq->rb_tree_root = RB_ROOT_CACHED;@@ -144,6 +166,8 @@ static void drm_sched_rq_init(struct drm_gpu_scheduler *sched, void drm_sched_rq_add_entity(struct drm_sched_rq *rq, struct drm_sched_entity *entity) { + WARN_ON(!!entity->single_sched); + if (!list_empty(&entity->list)) return;@@ -166,6 +190,8 @@ void drm_sched_rq_add_entity(struct drm_sched_rq *rq, void drm_sched_rq_remove_entity(struct drm_sched_rq *rq, struct drm_sched_entity *entity) { + WARN_ON(!!entity->single_sched); + if (list_empty(&entity->list)) return;@@ -641,7 +667,7 @@ int drm_sched_job_init(struct drm_sched_job *job, struct drm_sched_entity *entity, void *owner) { - if (!entity->rq) + if (!entity->rq && !entity->single_sched) return -ENOENT; job->entity = entity;@@ -674,13 +700,16 @@ void drm_sched_job_arm(struct drm_sched_job *job) { struct drm_gpu_scheduler *sched; struct drm_sched_entity *entity = job->entity; + bool single_entity = !!entity->single_sched; BUG_ON(!entity); - drm_sched_entity_select_rq(entity); - sched = entity->rq->sched; + if (!single_entity) + drm_sched_entity_select_rq(entity); + sched = drm_sched_entity_to_scheduler(entity); job->sched = sched; - job->s_priority = entity->rq - sched->sched_rq; + if (!single_entity) + job->s_priority = entity->rq - sched->sched_rq; job->id = atomic64_inc_return(&sched->job_id_count); drm_sched_fence_init(job->s_fence, job->entity);@@ -896,6 +925,13 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched) if (!drm_sched_can_queue(sched)) return NULL; + if (sched->single_entity) { + if (drm_sched_entity_is_ready(sched->single_entity)) + return sched->single_entity; + + return NULL; + } + /* Kernel run queue has higher priority than normal run queue*/ for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) {entity =
sched->sched_policy==DRM_SCHED_POLICY_FIFO?drivers/gpu/drm/scheduler/sched_main.c +1
-0@@ -1091,6 +1127,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, return -EINVAL; sched->ops = ops; + sched->single_entity = NULL; sched->hw_submission_limit = hw_submission; sched->name = name; sched->submit_wq = submit_wq ? : system_wq;@@ -1103,7 +1140,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, sched->sched_policy = default_drm_sched_policy;else
sched->sched_policy=sched_policy;
- for (i =DRM_SCHED_PRIORITY_MIN; i <DRM_SCHED_PRIORITY_COUNT; i++) + for (i =DRM_SCHED_PRIORITY_MIN;sched_policy!=
+DRM_SCHED_POLICY_SINGLE_ENTITY&& i <DRM_SCHED_PRIORITY_COUNT; + i++)
drm_sched_rq_init(sched, &sched->sched_rq[i]);init_waitqueue_head(&sched->job_scheduled);drivers/gpu/drm/scheduler/sched_main.c +11
-1@@ -1135,7 +1174,15 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched) drm_sched_submit_stop(sched); - for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + if (sched->single_entity) { + spin_lock(&sched->single_entity->rq_lock); + sched->single_entity->stopped = true; + spin_unlock(&sched->single_entity->rq_lock); + } + + for (i = DRM_SCHED_PRIORITY_COUNT - 1; sched->sched_policy != + DRM_SCHED_POLICY_SINGLE_ENTITY && i >= DRM_SCHED_PRIORITY_MIN; + i--) { struct drm_sched_rq *rq = &sched->sched_rq[i]; spin_lock(&rq->lock);@@ -1176,6 +1223,8 @@ void drm_sched_increase_karma(struct drm_sched_job *bad) struct drm_sched_entity *entity; struct drm_gpu_scheduler *sched = bad->sched; + WARN_ON(sched->sched_policy == DRM_SCHED_POLICY_SINGLE_ENTITY); + /* don't change @bad's karma if it's from KERNEL RQ,* because sometimes GPU hang would cause kernel jobs (like VM updating jobs) * corrupt but keep in mind that kernel jobs always considered good. include/drm/gpu_scheduler.h +8
-0@@ -79,6 +79,7 @@ enum drm_sched_policy { DRM_SCHED_POLICY_DEFAULT, DRM_SCHED_POLICY_RR, DRM_SCHED_POLICY_FIFO, + DRM_SCHED_POLICY_SINGLE_ENTITY, DRM_SCHED_POLICY_COUNT, };@@ -112,6 +113,9 @@ struct drm_sched_entity {*/ struct drm_sched_rq *rq; + /** @single_sched: Single scheduler */ + struct drm_gpu_scheduler *single_sched; + /*** @sched_list: * @@ -473,6 +477,7 @@ struct drm_sched_backend_ops { * struct drm_gpu_scheduler - scheduler instance-specific data * * @ops: backend operations provided by the driver. + * @single_entity: Single entity for the scheduler * @hw_submission_limit: the max size of the hardware queue. * @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used.@@ -503,6 +508,7 @@ struct drm_sched_backend_ops {*/ struct drm_gpu_scheduler { const struct drm_sched_backend_ops *ops; + struct drm_sched_entity *single_entity; uint32_t hw_submission_limit; long timeout; const char *name;@@ -585,6 +591,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list, atomic_t *guilty); +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity); long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout); void drm_sched_entity_fini(struct drm_sched_entity *entity); void drm_sched_entity_destroy(struct drm_sched_entity *entity);
Matthew Brost
maintainerRe: [PATCH v2 3/9] drm/sched: Add DRM_SCHED_POLICY_SINGLE_ENTITY scheduling policy
Quoted 80 lines · Danilo Krummrich
Quoted 63 lines · Danilo Krummrich
Quoted 62 lines · Matthew Brost
DRM_SCHED_POLICY_SINGLE_ENTITYcreates a 1 to 1 relationship between scheduler and entity. No priorities or run queue used in this mode. Intended for devices with firmware schedulers.v2:
- Drop sched / rq union (Luben)Signed-off-by Matthew Brost <[email protected]>---
drivers/gpu/drm/scheduler/sched_entity.c| 69 ++++++++++++++++++------ drivers/gpu/drm/scheduler/sched_fence.c | 2 +- drivers/gpu/drm/scheduler/sched_main.c | 63 +++++++++++++++++++--- include/drm/gpu_scheduler.h | 8 +++4 files changed, 119 insertions(+), 23 deletions(-)drivers/gpu/drm/scheduler/sched_entity.c +16
-4@@ -83,6 +83,7 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, memset(entity, 0, sizeof(struct drm_sched_entity)); INIT_LIST_HEAD(&entity->list); entity->rq = NULL; + entity->single_sched = NULL; entity->guilty = guilty; entity->num_sched_list = num_sched_list; entity->priority = priority;@@ -90,8 +91,17 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, RCU_INIT_POINTER(entity->last_scheduled, NULL); RB_CLEAR_NODE(&entity->rb_tree_node); - if(num_sched_list) - entity->rq = &sched_list[0]->sched_rq[entity->priority]; + if (num_sched_list) { + if (sched_list[0]->sched_policy != + DRM_SCHED_POLICY_SINGLE_ENTITY) { + entity->rq = &sched_list[0]->sched_rq[entity->priority]; + } else { + if (num_sched_list != 1 || sched_list[0]->single_entity) + return -EINVAL; + sched_list[0]->single_entity = entity; + entity->single_sched = sched_list[0]; + } + } init_completion(&entity->entity_idle);@@ -124,7 +134,8 @@ void drm_sched_entity_modify_sched(struct drm_sched_entity *entity, struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list) { - WARN_ON(!num_sched_list || !sched_list); + WARN_ON(!num_sched_list || !sched_list || + !!entity->single_sched); entity->sched_list = sched_list; entity->num_sched_list = num_sched_list;@@ -231,13 +242,15 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity) { struct drm_sched_job *job; struct dma_fence *prev; + bool single_entity = !!entity->single_sched; - if (!entity->rq) + if (!entity->rq && !single_entity) return; spin_lock(&entity->rq_lock);Quoted 8 lines
drivers/gpu/drm/scheduler/sched_entity.c static void drm_sched_entity_kill(struct drm_sched_entity *entity) · line 251
Quoted 4 lines
drivers/gpu/drm/scheduler/sched_entity.c +2
-1entity->stopped = true; - drm_sched_rq_remove_entity(entity->rq, entity); + if (!single_entity) + drm_sched_rq_remove_entity(entity->rq, entity);Looks like nothing prevents
drm_sched_run_job_work()to fetch more jobs from the entity, hence if this is called for an entity still having queued up jobs and a still running scheduler,drm_sched_entity_kill()anddrm_sched_run_job_work()would race for jobs, right?I worked around this with:
drivers/gpu/drm/scheduler/sched_main.c +0
-0
drivers/gpu/drm/scheduler/sched_main.c drm_sched_select_entity(struct drm_gpu_scheduler *sched, bool dequeue) · line 1028
Quoted 5 lines
drivers/gpu/drm/scheduler/sched_main.c +2
-1@@ -1025,7 +1025,8 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched, bool dequeue)return NULL ;if (sched->single_entity) { -if (drm_sched_entity_is_ready (sched->single_entity)) +if (drm_sched_entity_is_ready (sched->single_entity) && + !READ_ONCE (sched->single_entity->stopped))
This looks like the proper fix. Will include in next rev.
Quoted 26 lines
drivers/gpu/drm/scheduler/sched_main.c +0
-0return sched->single_entity;return NULL ;Quoted 23 lines
Not sure if this is by intention because we don't expect the driver to
drm_sched_entity_fini()as long as there are still queued up jobs. At least this is inconsistant to whatdrm_sched_entity_kill()does withoutDRM_SCHED_POLICY_SINGLE_ENTITYand should either be fixed or documented if we agree nothing else makes sense.I think it also touches my question on how to tear down the scheduler once a ring is removed or deinitialized.
I know XE is going its own way in this respect, but I also feel like we're leaving drivers potentially being interested in
DRM_SCHED_POLICY_SINGLE_ENTITYa bit alone on that. I think we should probably give drivers a bit more guidance on how to do that.Currently, I see two approaches.
(1) Do what XE does, which means letting the scheduler run dry, which includes both the entity's job queue and the schedulers
pending_list. While jobs from the entity's queue aren't pushing any more work to the ring on tear down, but just "flow through" to get freed up eventually. (Hopefully I got that right.)(2) Kill the entity to cleanup jobs from the entity's queue, stop the scheduler and either just wait for pending jobs or signal them right away and finally free them up.
Quoted 8 lines
Quoted 8 lines
Actually there'd also be (3), which could be a mix of both, discard the entity's queued jobs, but let the
pending_listrun dry.I'm not saying we should provide a whole bunch of infrastructure for drivers, e.g. for (1) as you've mentioned already as well, there is probably not much to generalize anyway. However, I think we should document the options drivers have to tear things down and do enough to enable drivers using any option (as long as we agree it is reasonable).
Agree we should document this. Agree both ways seem reasonable. Let me include a patch that documents this in my next rev.
Matt
Quoted 325 lines
Quoted 324 lines
For Nouveau specifically, I'd probably like to go with (3).
- Danilo
Quoted 320 lines
drivers/gpu/drm/scheduler/sched_entity.c +36
-10spin_unlock(&entity->rq_lock); /* Make sure this entity is not used by the scheduler at the moment */@@ -259,6 +272,20 @@ static void drm_sched_entity_kill(struct drm_sched_entity *entity) dma_fence_put(prev); } +/** +* drm_sched_entity_to_scheduler - Schedule entity to GPU scheduler +* @entity: scheduler entity +* +* Returns GPU scheduler for the entity +*/ +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity) +{ + bool single_entity = !!entity->single_sched; + + return single_entity ? entity->single_sched : entity->rq->sched; +} + /*** drm_sched_entity_flush - Flush a context entity * @@ -276,11 +303,12 @@ long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout) struct drm_gpu_scheduler *sched; struct task_struct *last_user; long ret = timeout; + bool single_entity = !!entity->single_sched; - if (!entity->rq) + if (!entity->rq && !single_entity) return 0; - sched = entity->rq->sched; + sched = drm_sched_entity_to_scheduler(entity); /*** The client will not queue more IBs during this fini, consume existing * queued IBs or discard them on SIGKILL @@ -373,7 +401,7 @@ static void drm_sched_entity_wakeup(struct dma_fence *f, container_of(cb, struct drm_sched_entity, cb); drm_sched_entity_clear_dep(f, cb); - drm_sched_wakeup_if_can_queue(entity->rq->sched); + drm_sched_wakeup_if_can_queue(drm_sched_entity_to_scheduler(entity)); } /**@@ -387,6 +415,8 @@ static void drm_sched_entity_wakeup(struct dma_fence *f, void drm_sched_entity_set_priority(struct drm_sched_entity *entity, enum drm_sched_priority priority) { + WARN_ON(!!entity->single_sched); + spin_lock(&entity->rq_lock); entity->priority = priority; spin_unlock(&entity->rq_lock);@@ -399,7 +429,7 @@ EXPORT_SYMBOL(drm_sched_entity_set_priority);*/ static bool drm_sched_entity_add_dependency_cb(struct drm_sched_entity *entity) { - struct drm_gpu_scheduler *sched = entity->rq->sched; + struct drm_gpu_scheduler *sched = drm_sched_entity_to_scheduler(entity); struct dma_fence *fence = entity->dependency; struct drm_sched_fence *s_fence;@@ -501,7 +531,8 @@ struct drm_sched_job *drm_sched_entity_pop_job(struct drm_sched_entity *entity)* Update the entity's location in the min heap according to * the timestamp of the next job, if any. */ - if (entity->rq->sched->sched_policy == DRM_SCHED_POLICY_FIFO) { + if (drm_sched_entity_to_scheduler(entity)->sched_policy == + DRM_SCHED_POLICY_FIFO) { struct drm_sched_job *next; next = to_drm_sched_job(spsc_queue_peek(&entity->job_queue));@@ -524,6 +555,8 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity) struct drm_gpu_scheduler *sched; struct drm_sched_rq *rq; + WARN_ON(!!entity->single_sched); + /* single possible engine and already selected */ if (!entity->sched_list) return;@@ -573,12 +606,13 @@ void drm_sched_entity_select_rq(struct drm_sched_entity *entity) void drm_sched_entity_push_job(struct drm_sched_job *sched_job) { struct drm_sched_entity *entity = sched_job->entity; - bool first, fifo = entity->rq->sched->sched_policy == - DRM_SCHED_POLICY_FIFO; + bool single_entity = !!entity->single_sched; + bool first; ktime_t submit_ts; trace_drm_sched_job(sched_job, entity); - atomic_inc(entity->rq->sched->score); + if (!single_entity) + atomic_inc(entity->rq->sched->score); WRITE_ONCE(entity->last_user, current->group_leader); /*@@ -591,6 +625,10 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job) /* first job wakes up scheduler */ if (first) { + struct drm_gpu_scheduler *sched = + drm_sched_entity_to_scheduler(entity); + bool fifo = sched->sched_policy == DRM_SCHED_POLICY_FIFO; + /* Add the entity to the run queue */ spin_lock(&entity->rq_lock); if (entity->stopped) {@@ -600,13 +638,14 @@ void drm_sched_entity_push_job(struct drm_sched_job *sched_job) return; } - drm_sched_rq_add_entity(entity->rq, entity); + if (!single_entity) + drm_sched_rq_add_entity(entity->rq, entity); spin_unlock(&entity->rq_lock); if (fifo) drm_sched_rq_update_fifo(entity, submit_ts); - drm_sched_wakeup_if_can_queue(entity->rq->sched); + drm_sched_wakeup_if_can_queue(sched); } } EXPORT_SYMBOL(drm_sched_entity_push_job);drivers/gpu/drm/scheduler/sched_fence.c +1
-1@@ -225,7 +225,7 @@ void drm_sched_fence_init(struct drm_sched_fence *fence, { unsigned seq; - fence->sched = entity->rq->sched; + fence->sched = drm_sched_entity_to_scheduler(entity); seq = atomic_inc_return(&entity->fence_seq); dma_fence_init(&fence->scheduled, &drm_sched_fence_ops_scheduled, &fence->lock, entity->fence_context, seq);drivers/gpu/drm/scheduler/sched_main.c +41
-5@@ -32,7 +32,8 @@ * backend operations to the scheduler like submitting a job to hardware run queue, * returning the dependencies of a job etc. * - * The organisation of the scheduler is the following: + * The organisation of the scheduler is the following for scheduling policies + * DRM_SCHED_POLICY_RR and DRM_SCHED_POLICY_FIFO: * * 1. Each hw run queue has one scheduler * 2. Each scheduler has multiple run queues with different priorities@@ -43,6 +44,23 @@ * * The jobs in a entity are always scheduled in the order that they were pushed. * + * The organisation of the scheduler is the following for scheduling policy + * DRM_SCHED_POLICY_SINGLE_ENTITY: + * + * 1. One to one relationship between scheduler and entity + * 2. No priorities implemented per scheduler (single job queue) + * 3. No run queues in scheduler rather jobs are directly dequeued from entity + * 4. The entity maintains a queue of jobs that will be scheduled on the + * hardware + * + * The jobs in a entity are always scheduled in the order that they were pushed + * regardless of scheduling policy. + * + * A policy of DRM_SCHED_POLICY_RR or DRM_SCHED_POLICY_FIFO is expected to used + * when the KMD is scheduling directly on the hardware while a scheduling policy + * of DRM_SCHED_POLICY_SINGLE_ENTITY is expected to be used when there is a + * firmware scheduler. + * * Note that once a job was taken from the entities queue and pushed to the * hardware, i.e. the pending queue, the entity must not be referenced anymore * through the jobs entity pointer.@@ -96,6 +114,8 @@ static inline void drm_sched_rq_remove_fifo_locked(struct drm_sched_entity *enti void drm_sched_rq_update_fifo(struct drm_sched_entity *entity, ktime_t ts) { + WARN_ON(!!entity->single_sched); + /** Both locks need to be grabbed, one to protect from entity->rq change * for entity from within concurrent drm_sched_entity_select_rq and the @@ -126,6 +146,8 @@ void drm_sched_rq_update_fifo(struct drm_sched_entity *entity, ktime_t ts) static void drm_sched_rq_init(struct drm_gpu_scheduler *sched, struct drm_sched_rq *rq) { + WARN_ON(sched->sched_policy == DRM_SCHED_POLICY_SINGLE_ENTITY); + spin_lock_init(&rq->lock); INIT_LIST_HEAD(&rq->entities); rq->rb_tree_root = RB_ROOT_CACHED;@@ -144,6 +166,8 @@ static void drm_sched_rq_init(struct drm_gpu_scheduler *sched, void drm_sched_rq_add_entity(struct drm_sched_rq *rq, struct drm_sched_entity *entity) { + WARN_ON(!!entity->single_sched); + if (!list_empty(&entity->list)) return;@@ -166,6 +190,8 @@ void drm_sched_rq_add_entity(struct drm_sched_rq *rq, void drm_sched_rq_remove_entity(struct drm_sched_rq *rq, struct drm_sched_entity *entity) { + WARN_ON(!!entity->single_sched); + if (list_empty(&entity->list)) return;@@ -641,7 +667,7 @@ int drm_sched_job_init(struct drm_sched_job *job, struct drm_sched_entity *entity, void *owner) { - if (!entity->rq) + if (!entity->rq && !entity->single_sched) return -ENOENT; job->entity = entity;@@ -674,13 +700,16 @@ void drm_sched_job_arm(struct drm_sched_job *job) { struct drm_gpu_scheduler *sched; struct drm_sched_entity *entity = job->entity; + bool single_entity = !!entity->single_sched; BUG_ON(!entity); - drm_sched_entity_select_rq(entity); - sched = entity->rq->sched; + if (!single_entity) + drm_sched_entity_select_rq(entity); + sched = drm_sched_entity_to_scheduler(entity); job->sched = sched; - job->s_priority = entity->rq - sched->sched_rq; + if (!single_entity) + job->s_priority = entity->rq - sched->sched_rq; job->id = atomic64_inc_return(&sched->job_id_count); drm_sched_fence_init(job->s_fence, job->entity);@@ -896,6 +925,13 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched) if (!drm_sched_can_queue(sched)) return NULL; + if (sched->single_entity) { + if (drm_sched_entity_is_ready(sched->single_entity)) + return sched->single_entity; + + return NULL; + } + /* Kernel run queue has higher priority than normal run queue*/ for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) {entity =
sched->sched_policy==DRM_SCHED_POLICY_FIFO?drivers/gpu/drm/scheduler/sched_main.c +1
-0@@ -1091,6 +1127,7 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, return -EINVAL; sched->ops = ops; + sched->single_entity = NULL; sched->hw_submission_limit = hw_submission; sched->name = name; sched->submit_wq = submit_wq ? : system_wq;@@ -1103,7 +1140,9 @@ int drm_sched_init(struct drm_gpu_scheduler *sched, sched->sched_policy = default_drm_sched_policy;else
sched->sched_policy=sched_policy;
- for (i =DRM_SCHED_PRIORITY_MIN; i <DRM_SCHED_PRIORITY_COUNT; i++) + for (i =DRM_SCHED_PRIORITY_MIN;sched_policy!=
+DRM_SCHED_POLICY_SINGLE_ENTITY&& i <DRM_SCHED_PRIORITY_COUNT; + i++)
drm_sched_rq_init(sched, &sched->sched_rq[i]);init_waitqueue_head(&sched->job_scheduled);drivers/gpu/drm/scheduler/sched_main.c +11
-1@@ -1135,7 +1174,15 @@ void drm_sched_fini(struct drm_gpu_scheduler *sched) drm_sched_submit_stop(sched); - for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) { + if (sched->single_entity) { + spin_lock(&sched->single_entity->rq_lock); + sched->single_entity->stopped = true; + spin_unlock(&sched->single_entity->rq_lock); + } + + for (i = DRM_SCHED_PRIORITY_COUNT - 1; sched->sched_policy != + DRM_SCHED_POLICY_SINGLE_ENTITY && i >= DRM_SCHED_PRIORITY_MIN; + i--) { struct drm_sched_rq *rq = &sched->sched_rq[i]; spin_lock(&rq->lock);@@ -1176,6 +1223,8 @@ void drm_sched_increase_karma(struct drm_sched_job *bad) struct drm_sched_entity *entity; struct drm_gpu_scheduler *sched = bad->sched; + WARN_ON(sched->sched_policy == DRM_SCHED_POLICY_SINGLE_ENTITY); + /* don't change @bad's karma if it's from KERNEL RQ,* because sometimes GPU hang would cause kernel jobs (like VM updating jobs) * corrupt but keep in mind that kernel jobs always considered good. include/drm/gpu_scheduler.h +8
-0@@ -79,6 +79,7 @@ enum drm_sched_policy { DRM_SCHED_POLICY_DEFAULT, DRM_SCHED_POLICY_RR, DRM_SCHED_POLICY_FIFO, + DRM_SCHED_POLICY_SINGLE_ENTITY, DRM_SCHED_POLICY_COUNT, };@@ -112,6 +113,9 @@ struct drm_sched_entity {*/ struct drm_sched_rq *rq; + /** @single_sched: Single scheduler */ + struct drm_gpu_scheduler *single_sched; + /*** @sched_list: * @@ -473,6 +477,7 @@ struct drm_sched_backend_ops { * struct drm_gpu_scheduler - scheduler instance-specific data * * @ops: backend operations provided by the driver. + * @single_entity: Single entity for the scheduler * @hw_submission_limit: the max size of the hardware queue. * @timeout: the time after which a job is removed from the scheduler. * @name: name of the ring for which this scheduler is being used.@@ -503,6 +508,7 @@ struct drm_sched_backend_ops {*/ struct drm_gpu_scheduler { const struct drm_sched_backend_ops *ops; + struct drm_sched_entity *single_entity; uint32_t hw_submission_limit; long timeout; const char *name;@@ -585,6 +591,8 @@ int drm_sched_entity_init(struct drm_sched_entity *entity, struct drm_gpu_scheduler **sched_list, unsigned int num_sched_list, atomic_t *guilty); +struct drm_gpu_scheduler * +drm_sched_entity_to_scheduler (struct drm_sched_entity *entity); long drm_sched_entity_flush(struct drm_sched_entity *entity, long timeout); void drm_sched_entity_fini(struct drm_sched_entity *entity); void drm_sched_entity_destroy(struct drm_sched_entity *entity);
Matthew Brost
maintainerPATCH v2 4/9 drm/sched: Split free_job into own work item
In reply to Matthew BrostRather than call free_job and run_job in same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.
--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)
drivers/gpu/drm/scheduler/sched_main.c +101 -36
@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;
@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }
@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }
@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;
@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }
@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/**
@@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/**
@@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;
@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }
@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;
@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;
@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/**
@@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;
@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);
@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);
include/drm/gpu_scheduler.h +5 -3
@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue.
@@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Signature · 1 line
2.34.1
Christian König
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Am 11.08.23 um 04:31 schrieb Matthew Brost:
Quoted 3 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.
I would rather say we should get completely rid of the free_job callback.
Essentially the job is just the container which carries the information which are necessary before you push it to the hw. The real representation of the submission is actually the scheduler fence.
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.
Quoted 315 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Matthew Brost
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 7 lines · Christian König
Am 11.08.23 um 04:31 schrieb Matthew Brost:
Quoted 3 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.I would rather say we should get completely rid of the
free_jobcallback.
Would we still have work item? e.g. Would we still want to call drm_sched_get_cleanup_job which removes the job from the pending list and adjusts the TDR? Trying to figure out out what this looks like. We probably can't do all of this from an IRQ context.
Quoted 4 lines
Essentially the job is just the container which carries the information which are necessary before you push it to the hw. The real representation of the submission is actually the scheduler fence.
Most of the free_jobs call plus drm_sched_job_cleanup + a put on job. In Xe this cannot be called from an IRQ context either.
I'm just confused what exactly you are suggesting here.
Matt
Quoted 304 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 297 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Christian König
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Am 17.08.23 um 19:54 schrieb Matthew Brost:
Quoted 12 lines · Christian König
Quoted 6 lines
Am 11.08.23 um 04:31 schrieb Matthew Brost:
Quoted 3 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.I would rather say we should get completely rid of the
free_jobcallback.Would we still have work item? e.g. Would we still want to call
drm_sched_get_cleanup_jobwhich removes the job from the pending list and adjusts the TDR? Trying to figure out out what this looks like. We probably can't do all of this from an IRQ context.
Quoted 8 lines
Quoted 4 lines
Essentially the job is just the container which carries the information which are necessary before you push it to the hw. The real representation of the submission is actually the scheduler fence.
Most of the
free_jobscall plusdrm_sched_job_cleanup+ a put on job. In Xe this cannot be called from an IRQ context either.I'm just confused what exactly you are suggesting here.
To summarize on one sentence: Instead of the job we keep the scheduler and hardware fences around after pushing the job to the hw.
The free_job callback would then be replaced by dropping the reference on the scheduler and hw fence.
Would that work for you?
Christian.
Quoted 305 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Matthew Brost
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 22 lines · Christian König
Am 17.08.23 um 19:54 schrieb Matthew Brost:
Quoted 12 lines · Christian König
Quoted 6 lines
Am 11.08.23 um 04:31 schrieb Matthew Brost:
Quoted 3 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.I would rather say we should get completely rid of the
free_jobcallback.Would we still have work item? e.g. Would we still want to call
drm_sched_get_cleanup_jobwhich removes the job from the pending list and adjusts the TDR? Trying to figure out out what this looks like. We probably can't do all of this from an IRQ context.Quoted 8 lines
Quoted 4 lines
Essentially the job is just the container which carries the information which are necessary before you push it to the hw. The real representation of the submission is actually the scheduler fence.
Most of the
free_jobscall plusdrm_sched_job_cleanup+ a put on job. In Xe this cannot be called from an IRQ context either.I'm just confused what exactly you are suggesting here.
Quoted 8 lines
To summarize on one sentence: Instead of the job we keep the scheduler and hardware fences around after pushing the job to the hw.
The
free_jobcallback would then be replaced by dropping the reference on the scheduler and hw fence.Would that work for you?
I don't think so for a few reasons.
The job and hw fence are different structures (also different allocs too) for a reason. The job referenced until it is complete (hw fence is signaled) and the free_job is called. This reference is needed for the TDR to work properly and also some reset flows too. Also in Xe some of things done in free_job cannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having free_job serialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.
FWIW what you suggesting sounds like how the i915 did things (i915_request and hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.
Matt
Quoted 308 lines
Christian.
Quoted 305 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Christian König
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Am 18.08.23 um 15:13 schrieb Matthew Brost:
Quoted 28 lines · Christian König
Quoted 27 lines
Am 17.08.23 um 19:54 schrieb Matthew Brost:
Quoted 12 lines · Christian König
Quoted 6 lines
Am 11.08.23 um 04:31 schrieb Matthew Brost:
Quoted 3 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.I would rather say we should get completely rid of the
free_jobcallback.Would we still have work item? e.g. Would we still want to call
drm_sched_get_cleanup_jobwhich removes the job from the pending list and adjusts the TDR? Trying to figure out out what this looks like. We probably can't do all of this from an IRQ context.Quoted 8 lines
Quoted 4 lines
Essentially the job is just the container which carries the information which are necessary before you push it to the hw. The real representation of the submission is actually the scheduler fence.
Most of the
free_jobscall plusdrm_sched_job_cleanup+ a put on job. In Xe this cannot be called from an IRQ context either.I'm just confused what exactly you are suggesting here.
To summarize on one sentence: Instead of the job we keep the scheduler and hardware fences around after pushing the job to the hw.
The
free_jobcallback would then be replaced by dropping the reference on the scheduler and hw fence.
Quoted 8 lines
Quoted 2 lines
Would that work for you?
I don't think so for a few reasons.
The job and hw fence are different structures (also different allocs too) for a reason. The job referenced until it is complete (hw fence is signaled) and the
free_jobis called. This reference is needed for the TDR to work properly and also some reset flows too.
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary structures (which include struct thread_info and struct file for this driver connection) alive until all submissions are completed.
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in drm_sched_entity_flush() as well.
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.
Well putting things for cleanup into a workitem doesn't sounds like something hard.
Question is what do you really need for TDR which is not inside the hardware fence?
Regards,
Christian.
Quoted 325 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Matthew Brost
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 38 lines · Christian König
Am 18.08.23 um 15:13 schrieb Matthew Brost:
Quoted 28 lines · Christian König
Quoted 27 lines
Am 17.08.23 um 19:54 schrieb Matthew Brost:
Quoted 12 lines · Christian König
Quoted 6 lines
Am 11.08.23 um 04:31 schrieb Matthew Brost:
Quoted 3 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.I would rather say we should get completely rid of the
free_jobcallback.Would we still have work item? e.g. Would we still want to call
drm_sched_get_cleanup_jobwhich removes the job from the pending list and adjusts the TDR? Trying to figure out out what this looks like. We probably can't do all of this from an IRQ context.Quoted 8 lines
Quoted 4 lines
Essentially the job is just the container which carries the information which are necessary before you push it to the hw. The real representation of the submission is actually the scheduler fence.
Most of the
free_jobscall plusdrm_sched_job_cleanup+ a put on job. In Xe this cannot be called from an IRQ context either.I'm just confused what exactly you are suggesting here.
To summarize on one sentence: Instead of the job we keep the scheduler and hardware fences around after pushing the job to the hw.
The
free_jobcallback would then be replaced by dropping the reference on the scheduler and hw fence.Quoted 8 lines
Quoted 2 lines
Would that work for you?
I don't think so for a few reasons.
The job and hw fence are different structures (also different allocs too) for a reason. The job referenced until it is complete (hw fence is signaled) and the
free_jobis called. This reference is needed for the TDR to work properly and also some reset flows too.
Quoted 9 lines
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
Agree the TDR shouldn't be accessing anything application specific rather just internal job state required to tear the job down on the hardware.
Quoted 2 lines
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary
I have not experenced the TDR going boom in Xe.
Quoted 3 lines
structures (which include struct
thread_infoand struct file for this driver connection) alive until all submissions are completed.
In Xe we keep everything alive until all submissions are completed. By everything I mean the drm job, entity, scheduler, and VM via a reference counting scheme. All of these structures are just kernel state which can safely be accessed even if the application has been killed.
If we need to teardown on demand we just set the TDR to a minimum value and it kicks the jobs off the hardware, gracefully cleans everything up and drops all references. This is a benefit of the 1 to 1 relationship, not sure if this works with how AMDGPU uses the scheduler.
Quoted 4 lines
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in
drm_sched_entity_flush()as well.
Not an issue for Xe, we never call drm_sched_entity_flush as our referencing counting scheme is all jobs are finished before we attempt to tear down entity / scheduler.
Quoted 6 lines
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Not even close, resets where a nightmare in the i915 (I spend years trying to get this right and probably still completely work) and in Xe basically got it right on the attempt.
Quoted 7 lines
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.Well putting things for cleanup into a workitem doesn't sounds like something hard.
That is exactly what we doing in the scheduler with the free_job workitem.
Quoted 3 lines
Question is what do you really need for TDR which is not inside the hardware fence?
A reference to the entity to be able to kick the job off the hardware. A reference to the entity, job, and VM for error capture.
We also need a reference to the job for recovery after a GPU reset so run_job can be called again for innocent jobs.
All of this leads to believe we need to stick with the design.
Matt
Quoted 329 lines
Regards,
Christian.Quoted 325 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Christian König
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Am 23.08.23 um 05:27 schrieb Matthew Brost:
Quoted 18 lines
[SNIP]
Quoted 9 lines
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
Agree the TDR shouldn't be accessing anything application specific rather just internal job state required to tear the job down on the hardware.
Quoted 2 lines
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary
I have not experenced the TDR going boom in Xe.
Quoted 7 lines
Quoted 3 lines
structures (which include struct
thread_infoand struct file for this driver connection) alive until all submissions are completed.In Xe we keep everything alive until all submissions are completed. By everything I mean the drm job, entity, scheduler, and VM via a reference counting scheme. All of these structures are just kernel state which can safely be accessed even if the application has been killed.
Yeah, but that might just not be such a good idea from memory management point of view.
When you (for example) kill a process all resource from that progress should at least be queued to be freed more or less immediately.
What Linux is doing for other I/O operations is to keep the relevant pages alive until the I/O operation is completed, but for GPUs that usually means keeping most of the memory of the process alive and that in turn is really not something you can do.
You can of course do this if your driver has a reliable way of killing your submissions and freeing resources in a reasonable amount of time. This should then be done in the flush callback.
Quoted 5 lines
If we need to teardown on demand we just set the TDR to a minimum value and it kicks the jobs off the hardware, gracefully cleans everything up and drops all references. This is a benefit of the 1 to 1 relationship, not sure if this works with how AMDGPU uses the scheduler.
Quoted 7 lines
Quoted 4 lines
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in
drm_sched_entity_flush()as well.Not an issue for Xe, we never call
drm_sched_entity_flushas our referencing counting scheme is all jobs are finished before we attempt to tear down entity / scheduler.
I don't think you can do that upstream. Calling drm_sched_entity_flush() is a must have from your flush callback for the file descriptor.
Unless you have some other method for killing your submissions this would give a path for a deny of service attack vector when the Xe driver is in use.
Quoted 10 lines
Quoted 6 lines
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Not even close, resets where a nightmare in the i915 (I spend years trying to get this right and probably still completely work) and in Xe basically got it right on the attempt.
Quoted 8 lines
Quoted 6 lines
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.Well putting things for cleanup into a workitem doesn't sounds like something hard.
That is exactly what we doing in the scheduler with the
free_jobworkitem.
Yeah, but I think that we do it in the scheduler and not the driver is problematic.
For the scheduler it shouldn't care about the job any more as soon as the driver takes over.
Quoted 9 lines
Quoted 4 lines
Question is what do you really need for TDR which is not inside the hardware fence?
A reference to the entity to be able to kick the job off the hardware. A reference to the entity, job, and VM for error capture.
We also need a reference to the job for recovery after a GPU reset so
run_jobcan be called again for innocent jobs.
Well exactly that's what I'm massively pushing back. Letting the scheduler call run_job() for the same job again is *NOT* something you can actually do.
This pretty clearly violates some of the dma_fence constrains and has cause massively headaches for me already.
What you can do is to do this inside your driver, e.g. take the jobs and push them again to the hw ring or just tell the hw to start executing again from a previous position.
BTW that re-submitting of jobs seems to be a no-go from userspace perspective as well. Take a look at the Vulkan spec for that, at least Marek pretty much pointed out that we should absolutely not do this inside the kernel.
The generally right approach seems to be to cleanly signal to userspace that something bad happened and that userspace then needs to submit things again even for innocent jobs.
Regards,
Christian.
Quoted 332 lines
All of this leads to believe we need to stick with the design.
Matt
Quoted 327 lines
Regards,
Christian.Quoted 324 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Matthew Brost
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 26 lines · Christian König
Am 23.08.23 um 05:27 schrieb Matthew Brost:
Quoted 17 lines
[SNIP]
Quoted 9 lines
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
Agree the TDR shouldn't be accessing anything application specific rather just internal job state required to tear the job down on the hardware.
Quoted 2 lines
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary
I have not experenced the TDR going boom in Xe.
Quoted 7 lines
Quoted 3 lines
structures (which include struct
thread_infoand struct file for this driver connection) alive until all submissions are completed.In Xe we keep everything alive until all submissions are completed. By everything I mean the drm job, entity, scheduler, and VM via a reference counting scheme. All of these structures are just kernel state which can safely be accessed even if the application has been killed.
Quoted 6 lines
Yeah, but that might just not be such a good idea from memory management point of view.
When you (for example) kill a process all resource from that progress should at least be queued to be freed more or less immediately.
We do this, the TDR kicks jobs off the hardware as fast as the hw interface allows and signals all pending hw fences immediately after. Free jobs then is immediately called and the reference count goes to zero. I think max time for all of this to occur is a handful of ms.
Quoted 9 lines
What Linux is doing for other I/O operations is to keep the relevant pages alive until the I/O operation is completed, but for GPUs that usually means keeping most of the memory of the process alive and that in turn is really not something you can do.
You can of course do this if your driver has a reliable way of killing your submissions and freeing resources in a reasonable amount of time. This should then be done in the flush callback.
'flush callback' - Do you mean drm_sched_entity_flush? I looked at that and think that function doesn't even work for what I tell. It flushes the spsc queue but what about jobs on the hardware, how do those get killed?
As stated we do via the TDR which is rather clean design and fits with our reference couting scheme.
Quoted 9 lines
Quoted 9 lines
If we need to teardown on demand we just set the TDR to a minimum value and it kicks the jobs off the hardware, gracefully cleans everything up and drops all references. This is a benefit of the 1 to 1 relationship, not sure if this works with how AMDGPU uses the scheduler.
Quoted 4 lines
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in
drm_sched_entity_flush()as well.
Quoted 7 lines
Quoted 3 lines
Not an issue for Xe, we never call
drm_sched_entity_flushas our referencing counting scheme is all jobs are finished before we attempt to tear down entity / scheduler.I don't think you can do that upstream. Calling
drm_sched_entity_flush()is a must have from your flush callback for the file descriptor.
Again 'flush callback'? What are you refering too.
And why does drm_sched_entity_flush need to be called, doesn't seem to do anything useful.
Quoted 4 lines
Unless you have some other method for killing your submissions this would give a path for a deny of service attack vector when the Xe driver is in use.
Yes, once th TDR fires is disallows all new submissions at the exec IOCTL plus flushes any pending submissions as fast as possible.
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Not even close, resets where a nightmare in the i915 (I spend years trying to get this right and probably still completely work) and in Xe basically got it right on the attempt.
Quoted 3 lines
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.
Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
Well putting things for cleanup into a workitem doesn't sounds like something hard.
That is exactly what we doing in the scheduler with the
free_jobworkitem.Yeah, but I think that we do it in the scheduler and not the driver is problematic.
Disagree, a common clean callback from a non-irq contexts IMO is a good design rather than each driver possibly having its own worker for cleanup.
Quoted 3 lines
For the scheduler it shouldn't care about the job any more as soon as the driver takes over.
This a massive rewrite for all users of the DRM scheduler, I'm saying for Xe what you are suggesting makes little to no sense.
I'd like other users of the DRM scheduler to chime in on what you purposing. The scope of this change affects 8ish drivers that would require buy in each of the stakeholders. I certainly can't change of these drivers as I don't feel comfortable in all of those code bases nor do I have hardware to test all of these drivers.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Question is what do you really need for TDR which is not inside the hardware fence?
A reference to the entity to be able to kick the job off the hardware. A reference to the entity, job, and VM for error capture.
Quoted 7 lines
Quoted 2 lines
We also need a reference to the job for recovery after a GPU reset so
run_jobcan be called again for innocent jobs.Well exactly that's what I'm massively pushing back. Letting the scheduler call
run_job()for the same job again is *NOT* something you can actually do.
But lots of drivers do this already and the DRM scheduler documents this.
Quoted 3 lines
This pretty clearly violates some of the
dma_fenceconstrains and has cause massively headaches for me already.
Seems to work fine in Xe.
Quoted 4 lines
What you can do is to do this inside your driver, e.g. take the jobs and push them again to the hw ring or just tell the hw to start executing again from a previous position.
Again this now is massive rewrite of many drivers.
Quoted 5 lines
BTW that re-submitting of jobs seems to be a no-go from userspace perspective as well. Take a look at the Vulkan spec for that, at least Marek pretty much pointed out that we should absolutely not do this inside the kernel.
Yes if the job causes the hang, we ban the queue. Typcially only per entity (queue) resets are done in Xe but occasionally device level resets are done (issues with hardware) and innocent jobs / entities call run_job again.
Quoted 4 lines
The generally right approach seems to be to cleanly signal to userspace that something bad happened and that userspace then needs to submit things again even for innocent jobs.
I disagree that innocent jobs should be banned. What you are suggesting is if a device reset needs to be done we kill / ban every user space queue. Thats seems like overkill. Not seeing where that is stated in this doc [1], it seems to imply that only jobs that are stuck results in bans.
Matt
[1] https://patchwork.freedesktop.org/patch/553465/?series=119883&rev=3
Quoted 336 lines
Regards,
Christian.Quoted 332 lines
All of this leads to believe we need to stick with the design.
Matt
Quoted 327 lines
Regards,
Christian.Quoted 324 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Re: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 165 lines · Matthew Brost
Quoted 26 lines · Christian König
Am 23.08.23 um 05:27 schrieb Matthew Brost:
Quoted 17 lines
[SNIP]
Quoted 9 lines
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
Agree the TDR shouldn't be accessing anything application specific rather just internal job state required to tear the job down on the hardware.
Quoted 2 lines
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary
I have not experenced the TDR going boom in Xe.
Quoted 7 lines
Quoted 3 lines
structures (which include struct
thread_infoand struct file for this driver connection) alive until all submissions are completed.In Xe we keep everything alive until all submissions are completed. By everything I mean the drm job, entity, scheduler, and VM via a reference counting scheme. All of these structures are just kernel state which can safely be accessed even if the application has been killed.
Quoted 6 lines
Yeah, but that might just not be such a good idea from memory management point of view.
When you (for example) kill a process all resource from that progress should at least be queued to be freed more or less immediately.
We do this, the TDR kicks jobs off the hardware as fast as the hw interface allows and signals all pending hw fences immediately after. Free jobs then is immediately called and the reference count goes to zero. I think max time for all of this to occur is a handful of ms.
Quoted 9 lines
What Linux is doing for other I/O operations is to keep the relevant pages alive until the I/O operation is completed, but for GPUs that usually means keeping most of the memory of the process alive and that in turn is really not something you can do.
You can of course do this if your driver has a reliable way of killing your submissions and freeing resources in a reasonable amount of time. This should then be done in the flush callback.
'flush callback' - Do you mean
drm_sched_entity_flush? I looked at that and think that function doesn't even work for what I tell. It flushes the spsc queue but what about jobs on the hardware, how do those get killed?As stated we do via the TDR which is rather clean design and fits with our reference couting scheme.
Quoted 9 lines
Quoted 9 lines
If we need to teardown on demand we just set the TDR to a minimum value and it kicks the jobs off the hardware, gracefully cleans everything up and drops all references. This is a benefit of the 1 to 1 relationship, not sure if this works with how AMDGPU uses the scheduler.
Quoted 4 lines
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in
drm_sched_entity_flush()as well.Quoted 7 lines
Quoted 3 lines
Not an issue for Xe, we never call
drm_sched_entity_flushas our referencing counting scheme is all jobs are finished before we attempt to tear down entity / scheduler.I don't think you can do that upstream. Calling
drm_sched_entity_flush()is a must have from your flush callback for the file descriptor.Again 'flush callback'? What are you refering too.
And why does
drm_sched_entity_flushneed to be called, doesn't seem to do anything useful.Quoted 4 lines
Unless you have some other method for killing your submissions this would give a path for a deny of service attack vector when the Xe driver is in use.
Yes, once th TDR fires is disallows all new submissions at the exec IOCTL plus flushes any pending submissions as fast as possible.
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Not even close, resets where a nightmare in the i915 (I spend years trying to get this right and probably still completely work) and in Xe basically got it right on the attempt.
Quoted 3 lines
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.Quoted 9 lines
Quoted 5 lines
Quoted 3 lines
Well putting things for cleanup into a workitem doesn't sounds like something hard.
That is exactly what we doing in the scheduler with the
free_jobworkitem.Yeah, but I think that we do it in the scheduler and not the driver is problematic.
Disagree, a common clean callback from a non-irq contexts IMO is a good design rather than each driver possibly having its own worker for cleanup.
Quoted 3 lines
For the scheduler it shouldn't care about the job any more as soon as the driver takes over.
This a massive rewrite for all users of the DRM scheduler, I'm saying for Xe what you are suggesting makes little to no sense.
I'd like other users of the DRM scheduler to chime in on what you purposing. The scope of this change affects 8ish drivers that would require buy in each of the stakeholders. I certainly can't change of these drivers as I don't feel comfortable in all of those code bases nor do I have hardware to test all of these drivers.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Question is what do you really need for TDR which is not inside the hardware fence?
A reference to the entity to be able to kick the job off the hardware. A reference to the entity, job, and VM for error capture.
Quoted 7 lines
Quoted 2 lines
We also need a reference to the job for recovery after a GPU reset so
run_jobcan be called again for innocent jobs.Well exactly that's what I'm massively pushing back. Letting the scheduler call
run_job()for the same job again is *NOT* something you can actually do.But lots of drivers do this already and the DRM scheduler documents this.
Quoted 3 lines
This pretty clearly violates some of the
dma_fenceconstrains and has cause massively headaches for me already.Seems to work fine in Xe.
Quoted 4 lines
What you can do is to do this inside your driver, e.g. take the jobs and push them again to the hw ring or just tell the hw to start executing again from a previous position.
Again this now is massive rewrite of many drivers.
Quoted 5 lines
BTW that re-submitting of jobs seems to be a no-go from userspace perspective as well. Take a look at the Vulkan spec for that, at least Marek pretty much pointed out that we should absolutely not do this inside the kernel.
Quoted 4 lines
Yes if the job causes the hang, we ban the queue. Typcially only per entity (queue) resets are done in Xe but occasionally device level resets are done (issues with hardware) and innocent jobs / entities call
run_jobagain.
If the engine is reset and the job was already executing, how can you determine that it's in a good state to resubmit? What if some internal fence or semaphore in memory used by the logic in the command buffer has been signaled already and then you resubmit the job and it now starts executing with different input state?
Alex
Quoted 351 lines
Quoted 5 lines
The generally right approach seems to be to cleanly signal to userspace that something bad happened and that userspace then needs to submit things again even for innocent jobs.
I disagree that innocent jobs should be banned. What you are suggesting is if a device reset needs to be done we kill / ban every user space queue. Thats seems like overkill. Not seeing where that is stated in this doc [1], it seems to imply that only jobs that are stuck results in bans.
Matt
[1] https://patchwork.freedesktop.org/patch/553465/?series=119883&rev=3
Quoted 336 lines
Regards,
Christian.Quoted 332 lines
All of this leads to believe we need to stick with the design.
Matt
Quoted 327 lines
Regards,
Christian.Quoted 324 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Re: [Intel-xe] [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 102 lines · Alex Deucher
Quoted 101 lines · Matthew Brost
Quoted 26 lines · Christian König
Am 23.08.23 um 05:27 schrieb Matthew Brost:
Quoted 17 lines
[SNIP]
Quoted 9 lines
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
Agree the TDR shouldn't be accessing anything application specific rather just internal job state required to tear the job down on the hardware.
Quoted 2 lines
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary
I have not experenced the TDR going boom in Xe.
Quoted 7 lines
Quoted 3 lines
structures (which include struct
thread_infoand struct file for this driver connection) alive until all submissions are completed.In Xe we keep everything alive until all submissions are completed. By everything I mean the drm job, entity, scheduler, and VM via a reference counting scheme. All of these structures are just kernel state which can safely be accessed even if the application has been killed.
Quoted 6 lines
Yeah, but that might just not be such a good idea from memory management point of view.
When you (for example) kill a process all resource from that progress should at least be queued to be freed more or less immediately.
We do this, the TDR kicks jobs off the hardware as fast as the hw interface allows and signals all pending hw fences immediately after. Free jobs then is immediately called and the reference count goes to zero. I think max time for all of this to occur is a handful of ms.
Quoted 9 lines
What Linux is doing for other I/O operations is to keep the relevant pages alive until the I/O operation is completed, but for GPUs that usually means keeping most of the memory of the process alive and that in turn is really not something you can do.
You can of course do this if your driver has a reliable way of killing your submissions and freeing resources in a reasonable amount of time. This should then be done in the flush callback.
'flush callback' - Do you mean
drm_sched_entity_flush? I looked at that and think that function doesn't even work for what I tell. It flushes the spsc queue but what about jobs on the hardware, how do those get killed?As stated we do via the TDR which is rather clean design and fits with our reference couting scheme.
Quoted 9 lines
Quoted 9 lines
If we need to teardown on demand we just set the TDR to a minimum value and it kicks the jobs off the hardware, gracefully cleans everything up and drops all references. This is a benefit of the 1 to 1 relationship, not sure if this works with how AMDGPU uses the scheduler.
Quoted 4 lines
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in
drm_sched_entity_flush()as well.Quoted 7 lines
Quoted 3 lines
Not an issue for Xe, we never call
drm_sched_entity_flushas our referencing counting scheme is all jobs are finished before we attempt to tear down entity / scheduler.I don't think you can do that upstream. Calling
drm_sched_entity_flush()is a must have from your flush callback for the file descriptor.Again 'flush callback'? What are you refering too.
And why does
drm_sched_entity_flushneed to be called, doesn't seem to do anything useful.Quoted 4 lines
Unless you have some other method for killing your submissions this would give a path for a deny of service attack vector when the Xe driver is in use.
Yes, once th TDR fires is disallows all new submissions at the exec IOCTL plus flushes any pending submissions as fast as possible.
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Not even close, resets where a nightmare in the i915 (I spend years trying to get this right and probably still completely work) and in Xe basically got it right on the attempt.
Quoted 3 lines
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Well putting things for cleanup into a workitem doesn't sounds like something hard.
That is exactly what we doing in the scheduler with the
free_jobworkitem.Yeah, but I think that we do it in the scheduler and not the driver is problematic.
Christian, I do see your point on simply get rid of free job callbacks here then use fence with own-driver workqueue and house cleaning. But I wonder if starting with this patch as a clear separation of that is not a step forward and that could be cleaned up on a follow up!?
Matt, why exactly do we need the separation in this patch? Commit message tells what it is doing and that it is aligned with design, but is not clear on why exactly we need this right now. Specially if in the end what we want is exactly keeping the submit_wq to ensure the serialization of the operations you mentioned. I mean, could we simply drop this patch and then work on a follow-up later and investigate the Christian suggestion when we are in-tree?
Quoted 61 lines
Quoted 56 lines
Quoted 1 line
Disagree, a common clean callback from a non-irq contexts IMO is a good design rather than each driver possibly having its own worker for cleanup.
Quoted 3 lines
For the scheduler it shouldn't care about the job any more as soon as the driver takes over.
This a massive rewrite for all users of the DRM scheduler, I'm saying for Xe what you are suggesting makes little to no sense.
I'd like other users of the DRM scheduler to chime in on what you purposing. The scope of this change affects 8ish drivers that would require buy in each of the stakeholders. I certainly can't change of these drivers as I don't feel comfortable in all of those code bases nor do I have hardware to test all of these drivers.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Question is what do you really need for TDR which is not inside the hardware fence?
A reference to the entity to be able to kick the job off the hardware. A reference to the entity, job, and VM for error capture.
Quoted 7 lines
Quoted 2 lines
We also need a reference to the job for recovery after a GPU reset so
run_jobcan be called again for innocent jobs.Well exactly that's what I'm massively pushing back. Letting the scheduler call
run_job()for the same job again is *NOT* something you can actually do.But lots of drivers do this already and the DRM scheduler documents this.
Quoted 3 lines
This pretty clearly violates some of the
dma_fenceconstrains and has cause massively headaches for me already.Seems to work fine in Xe.
Quoted 4 lines
What you can do is to do this inside your driver, e.g. take the jobs and push them again to the hw ring or just tell the hw to start executing again from a previous position.
Again this now is massive rewrite of many drivers.
Quoted 5 lines
BTW that re-submitting of jobs seems to be a no-go from userspace perspective as well. Take a look at the Vulkan spec for that, at least Marek pretty much pointed out that we should absolutely not do this inside the kernel.
Quoted 4 lines
Yes if the job causes the hang, we ban the queue. Typcially only per entity (queue) resets are done in Xe but occasionally device level resets are done (issues with hardware) and innocent jobs / entities call
run_jobagain.
Quoted 5 lines
If the engine is reset and the job was already executing, how can you determine that it's in a good state to resubmit? What if some internal fence or semaphore in memory used by the logic in the command buffer has been signaled already and then you resubmit the job and it now starts executing with different input state?
I believe we could set some more rules in the new robustness documentation:
https://lore.kernel.org/all/[email protected]/
For this robustness implementation i915 pin point the exact context that was in execution when the gpu hang and only blame that, although the ressubmission is up to the user space. While on Xe we are blaming every single context that was in the queue. So I'm actually confused on what are the innocent jobs and who are calling for resubmission, if all of them got banned and blamed.
Quoted 354 lines
Alex
Quoted 351 lines
Quoted 5 lines
The generally right approach seems to be to cleanly signal to userspace that something bad happened and that userspace then needs to submit things again even for innocent jobs.
I disagree that innocent jobs should be banned. What you are suggesting is if a device reset needs to be done we kill / ban every user space queue. Thats seems like overkill. Not seeing where that is stated in this doc [1], it seems to imply that only jobs that are stuck results in bans.
Matt
[1] https://patchwork.freedesktop.org/patch/553465/?series=119883&rev=3
Quoted 336 lines
Regards,
Christian.Quoted 332 lines
All of this leads to believe we need to stick with the design.
Matt
Quoted 327 lines
Regards,
Christian.Quoted 324 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Matthew Brost
maintainerRe: [Intel-xe] [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 117 lines · Rodrigo Vivi
Quoted 102 lines · Alex Deucher
Quoted 101 lines · Matthew Brost
Quoted 26 lines · Christian König
Am 23.08.23 um 05:27 schrieb Matthew Brost:
Quoted 17 lines
[SNIP]
Quoted 9 lines
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
Agree the TDR shouldn't be accessing anything application specific rather just internal job state required to tear the job down on the hardware.
Quoted 2 lines
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary
I have not experenced the TDR going boom in Xe.
Quoted 7 lines
Quoted 3 lines
structures (which include struct
thread_infoand struct file for this driver connection) alive until all submissions are completed.In Xe we keep everything alive until all submissions are completed. By everything I mean the drm job, entity, scheduler, and VM via a reference counting scheme. All of these structures are just kernel state which can safely be accessed even if the application has been killed.
Quoted 6 lines
Yeah, but that might just not be such a good idea from memory management point of view.
When you (for example) kill a process all resource from that progress should at least be queued to be freed more or less immediately.
We do this, the TDR kicks jobs off the hardware as fast as the hw interface allows and signals all pending hw fences immediately after. Free jobs then is immediately called and the reference count goes to zero. I think max time for all of this to occur is a handful of ms.
Quoted 9 lines
What Linux is doing for other I/O operations is to keep the relevant pages alive until the I/O operation is completed, but for GPUs that usually means keeping most of the memory of the process alive and that in turn is really not something you can do.
You can of course do this if your driver has a reliable way of killing your submissions and freeing resources in a reasonable amount of time. This should then be done in the flush callback.
'flush callback' - Do you mean
drm_sched_entity_flush? I looked at that and think that function doesn't even work for what I tell. It flushes the spsc queue but what about jobs on the hardware, how do those get killed?As stated we do via the TDR which is rather clean design and fits with our reference couting scheme.
Quoted 9 lines
Quoted 9 lines
If we need to teardown on demand we just set the TDR to a minimum value and it kicks the jobs off the hardware, gracefully cleans everything up and drops all references. This is a benefit of the 1 to 1 relationship, not sure if this works with how AMDGPU uses the scheduler.
Quoted 4 lines
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in
drm_sched_entity_flush()as well.Quoted 7 lines
Quoted 3 lines
Not an issue for Xe, we never call
drm_sched_entity_flushas our referencing counting scheme is all jobs are finished before we attempt to tear down entity / scheduler.I don't think you can do that upstream. Calling
drm_sched_entity_flush()is a must have from your flush callback for the file descriptor.Again 'flush callback'? What are you refering too.
And why does
drm_sched_entity_flushneed to be called, doesn't seem to do anything useful.Quoted 4 lines
Unless you have some other method for killing your submissions this would give a path for a deny of service attack vector when the Xe driver is in use.
Yes, once th TDR fires is disallows all new submissions at the exec IOCTL plus flushes any pending submissions as fast as possible.
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Not even close, resets where a nightmare in the i915 (I spend years trying to get this right and probably still completely work) and in Xe basically got it right on the attempt.
Quoted 3 lines
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
Well putting things for cleanup into a workitem doesn't sounds like something hard.
That is exactly what we doing in the scheduler with the
free_jobworkitem.Yeah, but I think that we do it in the scheduler and not the driver is problematic.
Christian, I do see your point on simply get rid of free job callbacks here then use fence with own-driver workqueue and house cleaning. But I wonder if starting with this patch as a clear separation of that is not a step forward and that could be cleaned up on a follow up!?
Quoted 7 lines
Matt, why exactly do we need the separation in this patch? Commit message tells what it is doing and that it is aligned with design, but is not clear on why exactly we need this right now. Specially if in the end what we want is exactly keeping the
submit_wqto ensure the serialization of the operations you mentioned. I mean, could we simply drop this patch and then work on a follow-up later and investigate the Christian suggestion when we are in-tree?
I believe Christian suggested this change in a previous rev (free_job, proccess_msg) in there own workitem [1].
Dropping free_job / calling run_job again is really a completely different topic than this patch.
[1] https://patchwork.freedesktop.org/patch/550722/?series=121745&rev=1
Quoted 56 lines
Quoted 56 lines
Quoted 56 lines
Quoted 1 line
Disagree, a common clean callback from a non-irq contexts IMO is a good design rather than each driver possibly having its own worker for cleanup.
Quoted 3 lines
For the scheduler it shouldn't care about the job any more as soon as the driver takes over.
This a massive rewrite for all users of the DRM scheduler, I'm saying for Xe what you are suggesting makes little to no sense.
I'd like other users of the DRM scheduler to chime in on what you purposing. The scope of this change affects 8ish drivers that would require buy in each of the stakeholders. I certainly can't change of these drivers as I don't feel comfortable in all of those code bases nor do I have hardware to test all of these drivers.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Question is what do you really need for TDR which is not inside the hardware fence?
A reference to the entity to be able to kick the job off the hardware. A reference to the entity, job, and VM for error capture.
Quoted 7 lines
Quoted 2 lines
We also need a reference to the job for recovery after a GPU reset so
run_jobcan be called again for innocent jobs.Well exactly that's what I'm massively pushing back. Letting the scheduler call
run_job()for the same job again is *NOT* something you can actually do.But lots of drivers do this already and the DRM scheduler documents this.
Quoted 3 lines
This pretty clearly violates some of the
dma_fenceconstrains and has cause massively headaches for me already.Seems to work fine in Xe.
Quoted 4 lines
What you can do is to do this inside your driver, e.g. take the jobs and push them again to the hw ring or just tell the hw to start executing again from a previous position.
Again this now is massive rewrite of many drivers.
Quoted 5 lines
BTW that re-submitting of jobs seems to be a no-go from userspace perspective as well. Take a look at the Vulkan spec for that, at least Marek pretty much pointed out that we should absolutely not do this inside the kernel.
Quoted 7 lines
Quoted 7 lines
Quoted 4 lines
Yes if the job causes the hang, we ban the queue. Typcially only per entity (queue) resets are done in Xe but occasionally device level resets are done (issues with hardware) and innocent jobs / entities call
run_jobagain.If the engine is reset and the job was already executing, how can you determine that it's in a good state to resubmit? What if some
If a job has started but not completed we ban the queue during device reset. If a queue have jobs submitted but not started we resubmit all jobs on the queue during device reset.
The started / completed state can be determined by looking at a seqno in memory.
Quoted 7 lines
Quoted 3 lines
internal fence or semaphore in memory used by the logic in the command buffer has been signaled already and then you resubmit the job and it now starts executing with different input state?
I believe we could set some more rules in the new robustness documentation:
https://lore.kernel.org/all/[email protected]/
Quoted 6 lines
For this robustness implementation i915 pin point the exact context that was in execution when the gpu hang and only blame that, although the ressubmission is up to the user space. While on Xe we are blaming every single context that was in the queue. So I'm actually confused on what are the innocent jobs and who are calling for resubmission, if all of them got banned and blamed.
See above, innocent job == submited job but not started (i.e. a job stuck in the FW queue not yet been put on the hardware). Because we have a FW scheduler we could have 1000s of innocent jobs that don't need to get banned. This is very different from drivers without FW schedulers as typically when run_job is called the job hits the hardware immediately.
Matt
Quoted 355 lines
Quoted 355 lines
Alex
Quoted 351 lines
Quoted 5 lines
The generally right approach seems to be to cleanly signal to userspace that something bad happened and that userspace then needs to submit things again even for innocent jobs.
I disagree that innocent jobs should be banned. What you are suggesting is if a device reset needs to be done we kill / ban every user space queue. Thats seems like overkill. Not seeing where that is stated in this doc [1], it seems to imply that only jobs that are stuck results in bans.
Matt
[1] https://patchwork.freedesktop.org/patch/553465/?series=119883&rev=3
Quoted 336 lines
Regards,
Christian.Quoted 332 lines
All of this leads to believe we need to stick with the design.
Matt
Quoted 327 lines
Regards,
Christian.Quoted 324 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Christian König
maintainerRe: [Intel-xe] [PATCH v2 4/9] drm/sched: Split free_job into own work item
Am 24.08.23 um 01:12 schrieb Matthew Brost:
Quoted 116 lines · Rodrigo Vivi
Quoted 108 lines · Alex Deucher
Quoted 95 lines · Matthew Brost
Quoted 94 lines · Christian König
Quoted 25 lines
Am 23.08.23 um 05:27 schrieb Matthew Brost:
Quoted 17 lines
[SNIP]
Quoted 9 lines
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
Agree the TDR shouldn't be accessing anything application specific rather just internal job state required to tear the job down on the hardware.
Quoted 2 lines
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary
I have not experenced the TDR going boom in Xe.
Quoted 7 lines
Quoted 3 lines
structures (which include struct
thread_infoand struct file for this driver connection) alive until all submissions are completed.In Xe we keep everything alive until all submissions are completed. By everything I mean the drm job, entity, scheduler, and VM via a reference counting scheme. All of these structures are just kernel state which can safely be accessed even if the application has been killed.
Quoted 6 lines
Yeah, but that might just not be such a good idea from memory management point of view.
When you (for example) kill a process all resource from that progress should at least be queued to be freed more or less immediately.
We do this, the TDR kicks jobs off the hardware as fast as the hw interface allows and signals all pending hw fences immediately after. Free jobs then is immediately called and the reference count goes to zero. I think max time for all of this to occur is a handful of ms.
Quoted 9 lines
What Linux is doing for other I/O operations is to keep the relevant pages alive until the I/O operation is completed, but for GPUs that usually means keeping most of the memory of the process alive and that in turn is really not something you can do.
You can of course do this if your driver has a reliable way of killing your submissions and freeing resources in a reasonable amount of time. This should then be done in the flush callback.
'flush callback' - Do you mean
drm_sched_entity_flush? I looked at that and think that function doesn't even work for what I tell. It flushes the spsc queue but what about jobs on the hardware, how do those get killed?As stated we do via the TDR which is rather clean design and fits with our reference couting scheme.
Quoted 9 lines
Quoted 9 lines
If we need to teardown on demand we just set the TDR to a minimum value and it kicks the jobs off the hardware, gracefully cleans everything up and drops all references. This is a benefit of the 1 to 1 relationship, not sure if this works with how AMDGPU uses the scheduler.
Quoted 4 lines
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in
drm_sched_entity_flush()as well.Quoted 6 lines
Quoted 3 lines
Not an issue for Xe, we never call
drm_sched_entity_flushas our referencing counting scheme is all jobs are finished before we attempt to tear down entity / scheduler.I don't think you can do that upstream. Calling
drm_sched_entity_flush()is a must have from your flush callback for the file descriptor.Again 'flush callback'? What are you refering too.
And why does
drm_sched_entity_flushneed to be called, doesn't seem to do anything useful.Quoted 4 lines
Unless you have some other method for killing your submissions this would give a path for a deny of service attack vector when the Xe driver is in use.
Yes, once th TDR fires is disallows all new submissions at the exec IOCTL plus flushes any pending submissions as fast as possible.
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Not even close, resets where a nightmare in the i915 (I spend years trying to get this right and probably still completely work) and in Xe basically got it right on the attempt.
Quoted 3 lines
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Well putting things for cleanup into a workitem doesn't sounds like something hard.
That is exactly what we doing in the scheduler with the
free_jobworkitem.Yeah, but I think that we do it in the scheduler and not the driver is problematic.
Christian, I do see your point on simply get rid of free job callbacks here then use fence with own-driver workqueue and house cleaning. But I wonder if starting with this patch as a clear separation of that is not a step forward and that could be cleaned up on a follow up!?
Quoted 7 lines
Matt, why exactly do we need the separation in this patch? Commit message tells what it is doing and that it is aligned with design, but is not clear on why exactly we need this right now. Specially if in the end what we want is exactly keeping the
submit_wqto ensure the serialization of the operations you mentioned. I mean, could we simply drop this patch and then work on a follow-up later and investigate the Christian suggestion when we are in-tree?
Quoted 5 lines
I believe Christian suggested this change in a previous rev (
free_job,proccess_msg) in there own workitem [1].Dropping
free_job/ callingrun_jobagain is really a completely different topic than this patch.
Yeah, agree. I just wanted to bring this up before we put even more effort in the free_job based approach.
Rodrigos point is a really good one, no matter if the driver or the scheduler frees the job. Doing that in a separate work item sounds like the right thing to do.
Regards,
Christian.
Quoted 432 lines
[1] https://patchwork.freedesktop.org/patch/550722/?series=121745&rev=1
Quoted 47 lines
Quoted 47 lines
Quoted 47 lines
Disagree, a common clean callback from a non-irq contexts IMO is a good design rather than each driver possibly having its own worker for cleanup.
Quoted 3 lines
For the scheduler it shouldn't care about the job any more as soon as the driver takes over.
This a massive rewrite for all users of the DRM scheduler, I'm saying for Xe what you are suggesting makes little to no sense.
I'd like other users of the DRM scheduler to chime in on what you purposing. The scope of this change affects 8ish drivers that would require buy in each of the stakeholders. I certainly can't change of these drivers as I don't feel comfortable in all of those code bases nor do I have hardware to test all of these drivers.
Quoted 12 lines
Quoted 8 lines
Quoted 3 lines
Question is what do you really need for TDR which is not inside the hardware fence?
A reference to the entity to be able to kick the job off the hardware. A reference to the entity, job, and VM for error capture.
We also need a reference to the job for recovery after a GPU reset so
run_jobcan be called again for innocent jobs.Well exactly that's what I'm massively pushing back. Letting the scheduler call
run_job()for the same job again is *NOT* something you can actually do.But lots of drivers do this already and the DRM scheduler documents this.
Quoted 3 lines
This pretty clearly violates some of the
dma_fenceconstrains and has cause massively headaches for me already.Seems to work fine in Xe.
Quoted 4 lines
What you can do is to do this inside your driver, e.g. take the jobs and push them again to the hw ring or just tell the hw to start executing again from a previous position.
Again this now is massive rewrite of many drivers.
Quoted 5 lines
BTW that re-submitting of jobs seems to be a no-go from userspace perspective as well. Take a look at the Vulkan spec for that, at least Marek pretty much pointed out that we should absolutely not do this inside the kernel.
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Yes if the job causes the hang, we ban the queue. Typcially only per entity (queue) resets are done in Xe but occasionally device level resets are done (issues with hardware) and innocent jobs / entities call
run_jobagain.If the engine is reset and the job was already executing, how can you determine that it's in a good state to resubmit? What if some
If a job has started but not completed we ban the queue during device reset. If a queue have jobs submitted but not started we resubmit all jobs on the queue during device reset.
The started / completed state can be determined by looking at a seqno in memory.
Quoted 6 lines
Quoted 3 lines
internal fence or semaphore in memory used by the logic in the command buffer has been signaled already and then you resubmit the job and it now starts executing with different input state?
I believe we could set some more rules in the new robustness documentation:
https://lore.kernel.org/all/[email protected]/Quoted 6 lines
For this robustness implementation i915 pin point the exact context that was in execution when the gpu hang and only blame that, although the ressubmission is up to the user space. While on Xe we are blaming every single context that was in the queue. So I'm actually confused on what are the innocent jobs and who are calling for resubmission, if all of them got banned and blamed.
See above, innocent job == submited job but not started (i.e. a job stuck in the FW queue not yet been put on the hardware). Because we have a FW scheduler we could have 1000s of innocent jobs that don't need to get banned. This is very different from drivers without FW schedulers as typically when
run_jobis called the job hits the hardware immediately.Matt
Quoted 349 lines
Quoted 349 lines
Alex
Quoted 347 lines
Quoted 4 lines
The generally right approach seems to be to cleanly signal to userspace that something bad happened and that userspace then needs to submit things again even for innocent jobs.
I disagree that innocent jobs should be banned. What you are suggesting is if a device reset needs to be done we kill / ban every user space queue. Thats seems like overkill. Not seeing where that is stated in this doc [1], it seems to imply that only jobs that are stuck results in bans.
Matt
[1] https://patchwork.freedesktop.org/patch/553465/?series=119883&rev=3
Quoted 334 lines
Regards,
Christian.Quoted 331 lines
All of this leads to believe we need to stick with the design.
Matt
Quoted 327 lines
Regards,
Christian.Quoted 324 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +101
-36@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) {struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched); - entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;
Matthew Brost
maintainerRe: [Intel-xe] [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 123 lines · Christian König
Am 24.08.23 um 01:12 schrieb Matthew Brost:
Quoted 116 lines · Rodrigo Vivi
Quoted 108 lines · Alex Deucher
Quoted 95 lines · Matthew Brost
Quoted 94 lines · Christian König
Quoted 25 lines
Am 23.08.23 um 05:27 schrieb Matthew Brost:
Quoted 17 lines
[SNIP]
Quoted 9 lines
That is exactly what I want to avoid, tying the TDR to the job is what some AMD engineers pushed for because it looked like a simple solution and made the whole thing similar to what Windows does.
This turned the previous relatively clean scheduler and TDR design into a complete nightmare. The job contains quite a bunch of things which are not necessarily available after the application which submitted the job is torn down.
Agree the TDR shouldn't be accessing anything application specific rather just internal job state required to tear the job down on the hardware.
Quoted 2 lines
So what happens is that you either have stale pointers in the TDR which can go boom extremely easily or we somehow find a way to keep the necessary
I have not experenced the TDR going boom in Xe.
Quoted 7 lines
Quoted 3 lines
structures (which include struct
thread_infoand struct file for this driver connection) alive until all submissions are completed.In Xe we keep everything alive until all submissions are completed. By everything I mean the drm job, entity, scheduler, and VM via a reference counting scheme. All of these structures are just kernel state which can safely be accessed even if the application has been killed.
Quoted 6 lines
Yeah, but that might just not be such a good idea from memory management point of view.
When you (for example) kill a process all resource from that progress should at least be queued to be freed more or less immediately.
We do this, the TDR kicks jobs off the hardware as fast as the hw interface allows and signals all pending hw fences immediately after. Free jobs then is immediately called and the reference count goes to zero. I think max time for all of this to occur is a handful of ms.
Quoted 9 lines
What Linux is doing for other I/O operations is to keep the relevant pages alive until the I/O operation is completed, but for GPUs that usually means keeping most of the memory of the process alive and that in turn is really not something you can do.
You can of course do this if your driver has a reliable way of killing your submissions and freeing resources in a reasonable amount of time. This should then be done in the flush callback.
'flush callback' - Do you mean
drm_sched_entity_flush? I looked at that and think that function doesn't even work for what I tell. It flushes the spsc queue but what about jobs on the hardware, how do those get killed?As stated we do via the TDR which is rather clean design and fits with our reference couting scheme.
Quoted 9 lines
Quoted 9 lines
If we need to teardown on demand we just set the TDR to a minimum value and it kicks the jobs off the hardware, gracefully cleans everything up and drops all references. This is a benefit of the 1 to 1 relationship, not sure if this works with how AMDGPU uses the scheduler.
Quoted 4 lines
Delaying application tear down is also not an option because then you run into massive trouble with the OOM killer (or more generally OOM handling). See what we do in
drm_sched_entity_flush()as well.Quoted 6 lines
Quoted 3 lines
Not an issue for Xe, we never call
drm_sched_entity_flushas our referencing counting scheme is all jobs are finished before we attempt to tear down entity / scheduler.I don't think you can do that upstream. Calling
drm_sched_entity_flush()is a must have from your flush callback for the file descriptor.Again 'flush callback'? What are you refering too.
And why does
drm_sched_entity_flushneed to be called, doesn't seem to do anything useful.Quoted 4 lines
Unless you have some other method for killing your submissions this would give a path for a deny of service attack vector when the Xe driver is in use.
Yes, once th TDR fires is disallows all new submissions at the exec IOCTL plus flushes any pending submissions as fast as possible.
Quoted 13 lines
Quoted 13 lines
Quoted 6 lines
Since adding the TDR support we completely exercised this through in the last two or three years or so. And to sum it up I would really like to get away from this mess again.
Compared to that what i915 does is actually rather clean I think.
Not even close, resets where a nightmare in the i915 (I spend years trying to get this right and probably still completely work) and in Xe basically got it right on the attempt.
Quoted 3 lines
Quoted 3 lines
Also in Xe some of
things done infree_jobcannot be from an IRQ context, hence calling this from the scheduler worker is rather helpful.Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
Well putting things for cleanup into a workitem doesn't sounds like something hard.
That is exactly what we doing in the scheduler with the
free_jobworkitem.Yeah, but I think that we do it in the scheduler and not the driver is problematic.
Christian, I do see your point on simply get rid of free job callbacks here then use fence with own-driver workqueue and house cleaning. But I wonder if starting with this patch as a clear separation of that is not a step forward and that could be cleaned up on a follow up!?
Quoted 7 lines
Matt, why exactly do we need the separation in this patch? Commit message tells what it is doing and that it is aligned with design, but is not clear on why exactly we need this right now. Specially if in the end what we want is exactly keeping the
submit_wqto ensure the serialization of the operations you mentioned. I mean, could we simply drop this patch and then work on a follow-up later and investigate the Christian suggestion when we are in-tree?Quoted 5 lines
I believe Christian suggested this change in a previous rev (
free_job,proccess_msg) in there own workitem [1].Dropping
free_job/ callingrun_jobagain is really a completely different topic than this patch.
Quoted 7 lines
Yeah, agree. I just wanted to bring this up before we put even more effort in the
free_jobbased approach.Rodrigos point is a really good one, no matter if the driver or the scheduler frees the job. Doing that in a separate work item sounds like the right thing to do.
Ok, so this patch for now is ok but as a follow up we should explore dropping free_job / scheduler refs to jobs with a wider audience as this change affects all drivers.
Matt
Quoted 436 lines
Regards,
Christian.Quoted 432 lines
[1] https://patchwork.freedesktop.org/patch/550722/?series=121745&rev=1
Quoted 47 lines
Quoted 47 lines
Quoted 47 lines
Disagree, a common clean callback from a non-irq contexts IMO is a good design rather than each driver possibly having its own worker for cleanup.
Quoted 3 lines
For the scheduler it shouldn't care about the job any more as soon as the driver takes over.
This a massive rewrite for all users of the DRM scheduler, I'm saying for Xe what you are suggesting makes little to no sense.
I'd like other users of the DRM scheduler to chime in on what you purposing. The scope of this change affects 8ish drivers that would require buy in each of the stakeholders. I certainly can't change of these drivers as I don't feel comfortable in all of those code bases nor do I have hardware to test all of these drivers.
Quoted 12 lines
Quoted 8 lines
Quoted 3 lines
Question is what do you really need for TDR which is not inside the hardware fence?
A reference to the entity to be able to kick the job off the hardware. A reference to the entity, job, and VM for error capture.
We also need a reference to the job for recovery after a GPU reset so
run_jobcan be called again for innocent jobs.Well exactly that's what I'm massively pushing back. Letting the scheduler call
run_job()for the same job again is *NOT* something you can actually do.But lots of drivers do this already and the DRM scheduler documents this.
Quoted 3 lines
This pretty clearly violates some of the
dma_fenceconstrains and has cause massively headaches for me already.Seems to work fine in Xe.
Quoted 4 lines
What you can do is to do this inside your driver, e.g. take the jobs and push them again to the hw ring or just tell the hw to start executing again from a previous position.
Again this now is massive rewrite of many drivers.
Quoted 5 lines
BTW that re-submitting of jobs seems to be a no-go from userspace perspective as well. Take a look at the Vulkan spec for that, at least Marek pretty much pointed out that we should absolutely not do this inside the kernel.
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Yes if the job causes the hang, we ban the queue. Typcially only per entity (queue) resets are done in Xe but occasionally device level resets are done (issues with hardware) and innocent jobs / entities call
run_jobagain.If the engine is reset and the job was already executing, how can you determine that it's in a good state to resubmit? What if some
If a job has started but not completed we ban the queue during device reset. If a queue have jobs submitted but not started we resubmit all jobs on the queue during device reset.
The started / completed state can be determined by looking at a seqno in memory.
Quoted 6 lines
Quoted 3 lines
internal fence or semaphore in memory used by the logic in the command buffer has been signaled already and then you resubmit the job and it now starts executing with different input state?
I believe we could set some more rules in the new robustness documentation:
https://lore.kernel.org/all/[email protected]/Quoted 6 lines
For this robustness implementation i915 pin point the exact context that was in execution when the gpu hang and only blame that, although the ressubmission is up to the user space. While on Xe we are blaming every single context that was in the queue. So I'm actually confused on what are the innocent jobs and who are calling for resubmission, if all of them got banned and blamed.
See above, innocent job == submited job but not started (i.e. a job stuck in the FW queue not yet been put on the hardware). Because we have a FW scheduler we could have 1000s of innocent jobs that don't need to get banned. This is very different from drivers without FW schedulers as typically when
run_jobis called the job hits the hardware immediately.Matt
Quoted 349 lines
Quoted 349 lines
Alex
Quoted 347 lines
Quoted 4 lines
The generally right approach seems to be to cleanly signal to userspace that something bad happened and that userspace then needs to submit things again even for innocent jobs.
I disagree that innocent jobs should be banned. What you are suggesting is if a device reset needs to be done we kill / ban every user space queue. Thats seems like overkill. Not seeing where that is stated in this doc [1], it seems to imply that only jobs that are stuck results in bans.
Matt
[1] https://patchwork.freedesktop.org/patch/553465/?series=119883&rev=3
Quoted 334 lines
Regards,
Christian.Quoted 331 lines
All of this leads to believe we need to stick with the design.
Matt
Quoted 327 lines
Regards,
Christian.Quoted 324 lines
The HW fence can live for longer as it can be installed in dma-resv slots, syncobjs, etc... If the job and hw fence are combined now we holding on the memory for the longer and perhaps at the mercy of the user. We also run the risk of the final put being done from an IRQ context which again wont work in Xe as it is currently coded. Lastly 2 jobs from the same scheduler could do the final put in parallel, so rather than having
free_jobserialized by the worker now multiple jobs are freeing themselves at the same time. This might not be an issue but adds another level of raceyness that needs to be accounted for. None of this sounds desirable to me.FWIW what you suggesting sounds like how the i915 did things (
i915_requestand hw fence in 1 memory alloc) and that turned out to be a huge mess. As rule of thumb I generally do the opposite of whatever the i915 did.Matt
Quoted 306 lines
Christian.
Quoted 304 lines
Matt
Quoted 302 lines
All the lifetime issues we had came from ignoring this fact and I think we should push for fixing this design up again.
Regards,
Christian.Quoted 296 lines
Signed-off-by: Matthew Brost <[email protected]>
---
drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++-------
include/drm/gpu_scheduler.h | 8 +-
2 files changed, 106 insertions(+), 39 deletions(-)diff --git a/drivers/gpu/drm/scheduler/sched_main.c b/drivers/gpu/drm/scheduler/sched_main.c
index cede47afc800..b67469eac179 100644
--- a/drivers/gpu/drm/scheduler/sched_main.c
+++ b/drivers/gpu/drm/scheduler/sched_main.c
@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,
* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run
*
* @rq: scheduler run queue to check.
+ * @dequeue: dequeue selected entity
*
* Try to find a ready entity, returns NULL if none found.
*/
static struct drm_sched_entity *
-drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)
+drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq, bool dequeue)
{
struct drm_sched_entity *entity;
@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)
if (entity) {
list_for_each_entry_continue(entity, &rq->entities, list) {
if (drm_sched_entity_is_ready(entity)) {
- rq->current_entity = entity;
- reinit_completion(&entity->entity_idle);
+ if (dequeue) {
+ rq->current_entity = entity;
+ reinit_completion(&entity->entity_idle);
+ }
spin_unlock(&rq->lock);
return entity;
}
@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)
list_for_each_entry(entity, &rq->entities, list) {
if (drm_sched_entity_is_ready(entity)) {
- rq->current_entity = entity;
- reinit_completion(&entity->entity_idle);
+ if (dequeue) {
+ rq->current_entity = entity;
+ reinit_completion(&entity->entity_idle);
+ }
spin_unlock(&rq->lock);
return entity;
}
@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)
* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run
*
* @rq: scheduler run queue to check.
+ * @dequeue: dequeue selected entity
*
* Find oldest waiting ready entity, returns NULL if none found.
*/
static struct drm_sched_entity *
-drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq)
+drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq, bool dequeue)
{
struct rb_node *rb;
@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq)
entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);
if (drm_sched_entity_is_ready(entity)) {
- rq->current_entity = entity;
- reinit_completion(&entity->entity_idle);
+ if (dequeue) {
+ rq->current_entity = entity;
+ reinit_completion(&entity->entity_idle);
+ }
break;
}
}
@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq)
}
/**
- * drm_sched_submit_queue - scheduler queue submission
+ * drm_sched_run_job_queue - queue job submission
* @sched: scheduler instance
*/
-static void drm_sched_submit_queue(struct drm_gpu_scheduler *sched)
+static void drm_sched_run_job_queue(struct drm_gpu_scheduler *sched)
{
if (!READ_ONCE(sched->pause_submit))
- queue_work(sched->submit_wq, &sched->work_submit);
+ queue_work(sched->submit_wq, &sched->work_run_job);
+}
+
+static struct drm_sched_entity *
+drm_sched_select_entity(struct drm_gpu_scheduler *sched, bool dequeue);
+
+/**
+ * drm_sched_run_job_queue_if_ready - queue job submission if ready
+ * @sched: scheduler instance
+ */
+static void drm_sched_run_job_queue_if_ready(struct drm_gpu_scheduler *sched)
+{
+ if (drm_sched_select_entity(sched, false))
+ drm_sched_run_job_queue(sched);
+}
+
+/**
+ * drm_sched_free_job_queue - queue free job
+ *
+ * @sched: scheduler instance to queue free job
+ */
+static void drm_sched_free_job_queue(struct drm_gpu_scheduler *sched)
+{
+ if (!READ_ONCE(sched->pause_submit))
+ queue_work(sched->submit_wq, &sched->work_free_job);
+}
+
+/**
+ * drm_sched_free_job_queue_if_ready - queue free job if ready
+ *
+ * @sched: scheduler instance to queue free job
+ */
+static void drm_sched_free_job_queue_if_ready(struct drm_gpu_scheduler *sched)
+{
+ struct drm_sched_job *job;
+
+ spin_lock(&sched->job_list_lock);
+ job = list_first_entry_or_null(&sched->pending_list,
+ struct drm_sched_job, list);
+ if (job && dma_fence_is_signaled(&job->s_fence->finished))
+ drm_sched_free_job_queue(sched);
+ spin_unlock(&sched->job_list_lock);
}
/**
@@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)
dma_fence_get(&s_fence->finished);
drm_sched_fence_finished(s_fence, result);
dma_fence_put(&s_fence->finished);
- drm_sched_submit_queue(sched);
+ drm_sched_free_job_queue(sched);
}
/**
@@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)
void drm_sched_wakeup_if_can_queue(struct drm_gpu_scheduler *sched)
{
if (drm_sched_can_queue(sched))
- drm_sched_submit_queue(sched);
+ drm_sched_run_job_queue(sched);
}
/**
* drm_sched_select_entity - Select next entity to process
*
* @sched: scheduler instance
+ * @dequeue: dequeue selected entity
*
* Returns the entity to process or NULL if none are found.
*/
static struct drm_sched_entity *
-drm_sched_select_entity(struct drm_gpu_scheduler *sched)
+drm_sched_select_entity(struct drm_gpu_scheduler *sched, bool dequeue)
{
struct drm_sched_entity *entity;
int i;
@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)
/* Kernel run queue has higher priority than normal run queue*/
for (i = DRM_SCHED_PRIORITY_COUNT - 1; i >= DRM_SCHED_PRIORITY_MIN; i--) {
entity = sched->sched_policy == DRM_SCHED_POLICY_FIFO ?
- drm_sched_rq_select_entity_fifo(&sched->sched_rq[i]) :
- drm_sched_rq_select_entity_rr(&sched->sched_rq[i]);
+ drm_sched_rq_select_entity_fifo(&sched->sched_rq[i],
+ dequeue) :
+ drm_sched_rq_select_entity_rr(&sched->sched_rq[i],
+ dequeue);
if (entity)
break;
}
@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,
EXPORT_SYMBOL(drm_sched_pick_best);
/**
- * drm_sched_main - main scheduler thread
+ * drm_sched_free_job_work - worker to call free_job
*
- * @param: scheduler instance
+ * @w: free job work
*/
-static void drm_sched_main(struct work_struct *w)
+static void drm_sched_free_job_work(struct work_struct *w)
{
struct drm_gpu_scheduler *sched =
- container_of(w, struct drm_gpu_scheduler, work_submit);
- struct drm_sched_entity *entity;
+ container_of(w, struct drm_gpu_scheduler, work_free_job);
struct drm_sched_job *cleanup_job;
- int r;
if (READ_ONCE(sched->pause_submit))
return;
cleanup_job = drm_sched_get_cleanup_job(sched);
- entity = drm_sched_select_entity(sched);
+ if (cleanup_job) {
+ sched->ops->free_job(cleanup_job);
+
+ drm_sched_free_job_queue_if_ready(sched);
+ drm_sched_run_job_queue_if_ready(sched);
+ }
+}
- if (!entity && !cleanup_job)
- return; /* No more work */
+/**
+ * drm_sched_run_job_work - worker to call run_job
+ *
+ * @w: run job work
+ */
+static void drm_sched_run_job_work(struct work_struct *w)
+{
+ struct drm_gpu_scheduler *sched =
+ container_of(w, struct drm_gpu_scheduler, work_run_job);
+ struct drm_sched_entity *entity;
+ int r;
- if (cleanup_job)
- sched->ops->free_job(cleanup_job);
+ if (READ_ONCE(sched->pause_submit))
+ return;
+ entity = drm_sched_select_entity(sched, true);
if (entity) {
struct dma_fence *fence;
struct drm_sched_fence *s_fence;
@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w)
sched_job = drm_sched_entity_pop_job(entity);
if (!sched_job) {
complete_all(&entity->entity_idle);
- if (!cleanup_job)
- return; /* No more work */
- goto again;
+ return; /* No more work */
}
s_fence = sched_job->s_fence;
@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w)
}
wake_up(&sched->job_scheduled);
+ drm_sched_run_job_queue_if_ready(sched);
}
-
-again:
- drm_sched_submit_queue(sched);
}
/**
@@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,
spin_lock_init(&sched->job_list_lock);
atomic_set(&sched->hw_rq_count, 0);
INIT_DELAYED_WORK(&sched->work_tdr, drm_sched_job_timedout);
- INIT_WORK(&sched->work_submit, drm_sched_main);
+ INIT_WORK(&sched->work_run_job, drm_sched_run_job_work);
+ INIT_WORK(&sched->work_free_job, drm_sched_free_job_work);
atomic_set(&sched->_score, 0);
atomic64_set(&sched->job_id_count, 0);
sched->pause_submit = false;
@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);
void drm_sched_submit_stop(struct drm_gpu_scheduler *sched)
{
WRITE_ONCE(sched->pause_submit, true);
- cancel_work_sync(&sched->work_submit);
+ cancel_work_sync(&sched->work_run_job);
+ cancel_work_sync(&sched->work_free_job);
}
EXPORT_SYMBOL(drm_sched_submit_stop);
@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);
void drm_sched_submit_start(struct drm_gpu_scheduler *sched)
{
WRITE_ONCE(sched->pause_submit, false);
- queue_work(sched->submit_wq, &sched->work_submit);
+ queue_work(sched->submit_wq, &sched->work_run_job);
+ queue_work(sched->submit_wq, &sched->work_free_job);
}
EXPORT_SYMBOL(drm_sched_submit_start);
diff --git a/include/drm/gpu_scheduler.h b/include/drm/gpu_scheduler.h
index 04eec2d7635f..fbc083a92757 100644
--- a/include/drm/gpu_scheduler.h
+++ b/include/drm/gpu_scheduler.h
@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {
* finished.
* @hw_rq_count: the number of jobs currently in the hardware queue.
* @job_id_count: used to assign unique id to the each job.
- * @submit_wq: workqueue used to queue @work_submit
+ * @submit_wq: workqueue used to queue @work_run_job and @work_free_job
* @timeout_wq: workqueue used to queue @work_tdr
- * @work_submit: schedules jobs and cleans up entities
+ * @work_run_job: schedules jobs
+ * @work_free_job: cleans up jobs
* @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the
* timeout interval is over.
* @pending_list: the list of jobs which are currently in the job queue.
@@ -518,7 +519,8 @@ struct drm_gpu_scheduler {
atomic64_t job_id_count;
struct workqueue_struct *submit_wq;
struct workqueue_struct *timeout_wq;
- struct work_struct work_submit;
+ struct work_struct work_run_job;
+ struct work_struct work_free_job;
struct delayed_work work_tdr;
struct list_head pending_list;
spinlock_t job_list_lock;
Re: [PATCH v2 4/9] drm/sched: Split free_job into own work item
In reply to Matthew BrostQuoted 197 lines · Matthew Brost
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +71
-19@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) { struct drm_gpu_scheduler *sched =
drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1092
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-3- container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched);
I tried this patch with Nouveau and found a race condition:
In drm_sched_run_job_work() the job is added to the pending_list via drm_sched_job_begin(), then the run_job() callback is called and the scheduled fence is signaled.
However, in parallel drm_sched_get_cleanup_job() might be called from drm_sched_free_job_work(), which picks the first job from the pending_list and for the next job on the pending_list sets the scheduled fence' timestamp field.
The job can be on the pending_list, but the scheduled fence might not yet be signaled. The call to actually signal the fence will subsequently fault because it will try to dereference the timestamp.
I'm not sure what's the best way to fix this, maybe it's enough to re-order signalling the scheduled fence and adding the job to the pending_list. Not sure if this has other implications though.
- Danilo
Quoted 114 lines
drivers/gpu/drm/scheduler/sched_main.c +29
-14- entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 219 lines · Danilo Krummrich
Quoted 197 lines · Matthew Brost
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +71
-19@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) { struct drm_gpu_scheduler *sched =drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1092
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-3- container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched);I tried this patch with Nouveau and found a race condition:
In
drm_sched_run_job_work()the job is added to thepending_listviadrm_sched_job_begin(), then therun_job()callback is called and the scheduled fence is signaled.However, in parallel
drm_sched_get_cleanup_job()might be called fromdrm_sched_free_job_work(), which picks the first job from thepending_listand for the next job on thepending_listsets the scheduled fence' timestamp field.
Quoted 8 lines
The job can be on the
pending_list, but the scheduled fence might not yet be signaled. The call to actually signal the fence will subsequently fault because it will try to dereference the timestamp.I'm not sure what's the best way to fix this, maybe it's enough to re-order signalling the scheduled fence and adding the job to the
pending_list. Not sure if this has other implications though.
We really want the job on the pending list before calling run_job.
I'm thinking we just delete the updating of the timestamp, not sure why this is useful.
Or we could do something like this where we try to update the timestamp, if we can't update the timestamp run_job worker will do it anyways.
drivers/gpu/drm/scheduler/sched_main.c +4 -2
@@ -1074,8 +1074,10 @@ drm_sched_get_cleanup_job(struct drm_gpu_scheduler *sched) typeof(*next), list);if (next) { - next->s_fence->scheduled.timestamp = - job->s_fence->finished.timestamp; +if (test_bit (DMA_FENCE_FLAG_TIMESTAMP_BIT , + &next->s_fence->scheduled.flags)) + next->s_fence->scheduled.timestamp = + job->s_fence->finished.timestamp;/* start TO timer for next job */ drm_sched_start_timeout (sched); }
I guess I'm leaning towards the latter option.
Matt
Quoted 117 lines
- Danilo
Quoted 114 lines
drivers/gpu/drm/scheduler/sched_main.c +29
-14- entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;Signature · 1 line
2.34.1
Christian König
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Am 25.08.23 um 04:58 schrieb Matthew Brost:
Quoted 211 lines · Danilo Krummrich
Quoted 210 lines · Matthew Brost
Quoted 197 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +71
-19@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w) { struct drm_gpu_scheduler *sched =drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1092
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-3- container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched);I tried this patch with Nouveau and found a race condition:
Quoted 7 lines
Quoted 7 lines
In
drm_sched_run_job_work()the job is added to thepending_listviadrm_sched_job_begin(), then therun_job()callback is called and the scheduled fence is signaled.However, in parallel
drm_sched_get_cleanup_job()might be called fromdrm_sched_free_job_work(), which picks the first job from thepending_listand for the next job on thepending_listsets the scheduled fence' timestamp field.
Well why can this happen in parallel? Either the work items are scheduled to a single threaded work queue or you have protected the pending list with some locks.
Just moving the free_job into a separate work item without such precautions won't work because of quite a bunch of other reasons as well.
Quoted 5 lines
Quoted 5 lines
The job can be on the
pending_list, but the scheduled fence might not yet be signaled. The call to actually signal the fence will subsequently fault because it will try to dereference the timestamp.
Quoted 8 lines
Quoted 4 lines
I'm not sure what's the best way to fix this, maybe it's enough to re-order signalling the scheduled fence and adding the job to the
pending_list. Not sure if this has other implications though.We really want the job on the pending list before calling
run_job.I'm thinking we just delete the updating of the timestamp, not sure why this is useful.
This is used for calculating how long each job has spend on the hw, so big NAK to deleting this.
Regards,
Christian.
Quoted 142 lines
Or we could do something like this where we try to update the timestamp, if we can't update the timestamp
run_jobworker will do it anyways.drivers/gpu/drm/scheduler/sched_main.c +4
-2@@ -1074,8 +1074,10 @@ drm_sched_get_cleanup_job(struct drm_gpu_scheduler *sched) typeof(*next), list);if (next) { - next->s_fence->scheduled.timestamp = - job->s_fence->finished.timestamp; +if (test_bit (DMA_FENCE_FLAG_TIMESTAMP_BIT , + &next->s_fence->scheduled.flags)) + next->s_fence->scheduled.timestamp = + job->s_fence->finished.timestamp;/* start TO timer for next job */ drm_sched_start_timeout (sched); }I guess I'm leaning towards the latter option.
Matt
Quoted 116 lines
- Danilo
Quoted 114 lines
drivers/gpu/drm/scheduler/sched_main.c +29
-14- entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;Signature · 1 line
2.34.1
Matthew Brost
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 205 lines · Christian König
Am 25.08.23 um 04:58 schrieb Matthew Brost:
Quoted 204 lines · Danilo Krummrich
Quoted 203 lines · Matthew Brost
Quoted 186 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +71
-19@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w)drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1089
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-3{struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched);I tried this patch with Nouveau and found a race condition:
In
drm_sched_run_job_work()the job is added to thepending_listviadrm_sched_job_begin(), then therun_job()callback is called and the scheduled fence is signaled.
Quoted 8 lines
Quoted 3 lines
Quoted 3 lines
However, in parallel
drm_sched_get_cleanup_job()might be called fromdrm_sched_free_job_work(), which picks the first job from thepending_listand for the next job on thepending_listsets the scheduled fence' timestamp field.Well why can this happen in parallel? Either the work items are scheduled to a single threaded work queue or you have protected the pending list with some locks.
Xe uses a single-threaded work queue, Nouveau does not (desired behavior).
The list of pending jobs is protected by a lock (safe), the race is:
add job to pending listrun_job
signal scheduled fence
dequeue from pending listfree_job
update timestamp
Once a job is on the pending list its timestamp can be accessed which can blow up if scheduled fence isn't signaled or more specifically unless DMA_FENCE_FLAG_TIMESTAMP_BIT is set. Logically it makes sense for the job to be in the pending list before run_job and signal the scheduled fence after run_job so I think we need to live with this race.
Quoted 3 lines
Just moving the
free_jobinto a separate work item without such precautions won't work because of quite a bunch of other reasons as well.
Yes, free_job might not be safe to run in parallel with run_job depending on the driver vfuncs. Mention this in the cover letter.
Certainly this should be safe in the scheduler code though and I think it will be after fixing this.
Matt
Quoted 9 lines
Quoted 9 lines
Quoted 9 lines
The job can be on the
pending_list, but the scheduled fence might not yet be signaled. The call to actually signal the fence will subsequently fault because it will try to dereference the timestamp.I'm not sure what's the best way to fix this, maybe it's enough to re-order signalling the scheduled fence and adding the job to the
pending_list. Not sure if this has other implications though.
Quoted 8 lines
Quoted 4 lines
We really want the job on the pending list before calling
run_job.I'm thinking we just delete the updating of the timestamp, not sure why this is useful.
This is used for calculating how long each job has spend on the hw, so big NAK to deleting this.
Ah, I see that AMDGPU uses this. Previously just checked the scheduler code.
The below patch should work just fine then.
Matt
Quoted 139 lines
Regards,
Christian.Quoted 135 lines
Or we could do something like this where we try to update the timestamp, if we can't update the timestamp
run_jobworker will do it anyways.drivers/gpu/drm/scheduler/sched_main.c +4
-2@@ -1074,8 +1074,10 @@ drm_sched_get_cleanup_job(struct drm_gpu_scheduler *sched) typeof(*next), list);if (next) { - next->s_fence->scheduled.timestamp = - job->s_fence->finished.timestamp; +if (test_bit (DMA_FENCE_FLAG_TIMESTAMP_BIT , + &next->s_fence->scheduled.flags)) + next->s_fence->scheduled.timestamp = + job->s_fence->finished.timestamp;/* start TO timer for next job */ drm_sched_start_timeout (sched); }I guess I'm leaning towards the latter option.
Matt
Quoted 109 lines
- Danilo
Quoted 107 lines
drivers/gpu/drm/scheduler/sched_main.c +29
-14- entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;Signature · 1 line
2.34.1
Christian König
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Am 25.08.23 um 15:36 schrieb Matthew Brost:
Quoted 222 lines · Christian König
Quoted 205 lines
Am 25.08.23 um 04:58 schrieb Matthew Brost:
Quoted 204 lines · Danilo Krummrich
Quoted 203 lines · Matthew Brost
Quoted 186 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +71
-19@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w)drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1089
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-3{struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched);I tried this patch with Nouveau and found a race condition:
In
drm_sched_run_job_work()the job is added to thepending_listviadrm_sched_job_begin(), then therun_job()callback is called and the scheduled fence is signaled.Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
However, in parallel
drm_sched_get_cleanup_job()might be called fromdrm_sched_free_job_work(), which picks the first job from thepending_listand for the next job on thepending_listsets the scheduled fence' timestamp field.Well why can this happen in parallel? Either the work items are scheduled to a single threaded work queue or you have protected the pending list with some locks.
Xe uses a single-threaded work queue, Nouveau does not (desired behavior).
The list of pending jobs is protected by a lock (safe), the race is:
add job to pending list
run_job
signal scheduled fence
Quoted 7 lines
dequeue from pending list
free_job
update timestampOnce a job is on the pending list its timestamp can be accessed which can blow up if scheduled fence isn't signaled or more specifically unless
DMA_FENCE_FLAG_TIMESTAMP_BITis set.
Ah, that problem again. No that is actually quite harmless.
You just need to double check if the DMA_FENCE_FLAG_TIMESTAMP_BIT is already set and if it's not set don't do anything.
Regards,
Christian.
Quoted 174 lines
Logically it makes sense for the
job to be in the pending list beforerun_joband signal the scheduled fence afterrun_jobso I think we need to live with this race.Quoted 3 lines
Just moving the
free_jobinto a separate work item without such precautions won't work because of quite a bunch of other reasons as well.Yes,
free_jobmight not be safe to run in parallel withrun_jobdepending on the driver vfuncs. Mention this in the cover letter.Certainly this should be safe in the scheduler code though and I think it will be after fixing this.
Matt
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
The job can be on the
pending_list, but the scheduled fence might not yet be signaled. The call to actually signal the fence will subsequently fault because it will try to dereference the timestamp.I'm not sure what's the best way to fix this, maybe it's enough to re-order signalling the scheduled fence and adding the job to the
pending_list. Not sure if this has other implications though.Quoted 7 lines
Quoted 4 lines
We really want the job on the pending list before calling
run_job.I'm thinking we just delete the updating of the timestamp, not sure why this is useful.
This is used for calculating how long each job has spend on the hw, so big NAK to deleting this.
Ah, I see that AMDGPU uses this. Previously just checked the scheduler code.
The below patch should work just fine then.
Matt
Quoted 137 lines
Regards,
Christian.Quoted 134 lines
Or we could do something like this where we try to update the timestamp, if we can't update the timestamp
run_jobworker will do it anyways.drivers/gpu/drm/scheduler/sched_main.c +4
-2@@ -1074,8 +1074,10 @@ drm_sched_get_cleanup_job(struct drm_gpu_scheduler *sched) typeof(*next), list);if (next) { - next->s_fence->scheduled.timestamp = - job->s_fence->finished.timestamp; +if (test_bit (DMA_FENCE_FLAG_TIMESTAMP_BIT , + &next->s_fence->scheduled.flags)) + next->s_fence->scheduled.timestamp = + job->s_fence->finished.timestamp;/* start TO timer for next job */ drm_sched_start_timeout (sched); }I guess I'm leaning towards the latter option.
Matt
Quoted 109 lines
- Danilo
Quoted 107 lines
drivers/gpu/drm/scheduler/sched_main.c +29
-14- entity =drm_sched_select_entity (sched); +if (cleanup_job) { + sched->ops->free_job (cleanup_job); + +drm_sched_free_job_queue_if_ready (sched); +drm_sched_run_job_queue_if_ready (sched); + } +} -if (!entity && !cleanup_job) -return ;/* No more work */ +/** +* drm_sched_run_job_work - worker to call run_job +* +* @w: run job work +*/ +static void drm_sched_run_job_work (struct work_struct *w) +{ +struct drm_gpu_scheduler *sched = + container_of(w, struct drm_gpu_scheduler, work_run_job); +struct drm_sched_entity *entity; +int r; -if (cleanup_job) - sched->ops->free_job (cleanup_job); +if (READ_ONCE (sched->pause_submit)) +return ; + entity =drm_sched_select_entity (sched, true);if (entity) {struct dma_fence *fence;struct drm_sched_fence *s_fence;@@ -1056,9 +1122,7 @@ static void drm_sched_main(struct work_struct *w) sched_job =drm_sched_entity_pop_job (entity);if (!sched_job) {complete_all (&entity->entity_idle); -if (!cleanup_job) -return ;/* No more work */ - goto again; +return ;/* No more work */ } s_fence = sched_job->s_fence;@@ -1088,10 +1152,8 @@ static void drm_sched_main(struct work_struct *w) }wake_up (&sched->job_scheduled); +drm_sched_run_job_queue_if_ready (sched); } - -again: -drm_sched_submit_queue (sched); }/** @@ -1150,7 +1212,8 @@ int drm_sched_init(struct drm_gpu_scheduler *sched,spin_lock_init (&sched->job_list_lock);atomic_set (&sched->hw_rq_count,0 );INIT_DELAYED_WORK (&sched->work_tdr, drm_sched_job_timedout); -INIT_WORK (&sched->work_submit, drm_sched_main); +INIT_WORK (&sched->work_run_job, drm_sched_run_job_work); +INIT_WORK (&sched->work_free_job, drm_sched_free_job_work);atomic_set (&sched->_score,0 );atomic64_set (&sched->job_id_count,0 ); sched->pause_submit = false;@@ -1275,7 +1338,8 @@ EXPORT_SYMBOL(drm_sched_submit_ready);void drm_sched_submit_stop (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, true); -cancel_work_sync (&sched->work_submit); +cancel_work_sync (&sched->work_run_job); +cancel_work_sync (&sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_stop);@@ -1287,6 +1351,7 @@ EXPORT_SYMBOL(drm_sched_submit_stop);void drm_sched_submit_start (struct drm_gpu_scheduler *sched) {WRITE_ONCE (sched->pause_submit, false); -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +queue_work (sched->submit_wq, &sched->work_free_job); }EXPORT_SYMBOL (drm_sched_submit_start);include/drm/gpu_scheduler.h +5
-3@@ -487,9 +487,10 @@ struct drm_sched_backend_ops {* finished. * @hw_rq_count: the number of jobs currently in the hardware queue. * @job_id_count: used to assign unique id to the each job. -* @submit_wq: workqueue used to queue @work_submit +* @submit_wq: workqueue used to queue @work_run_job and @work_free_job * @timeout_wq: workqueue used to queue @work_tdr -* @work_submit: schedules jobs and cleans up entities +* @work_run_job: schedules jobs +* @work_free_job: cleans up jobs * @work_tdr: schedules a delayed call to @drm_sched_job_timedout after the * timeout interval is over. * @pending_list: the list of jobs which are currently in the job queue. @@ -518,7 +519,8 @@ struct drm_gpu_scheduler {atomic64_t job_id_count;struct workqueue_struct *submit_wq;struct workqueue_struct *timeout_wq; -struct work_struct work_submit; +struct work_struct work_run_job; +struct work_struct work_free_job;struct delayed_work work_tdr;struct list_head pending_list;spinlock_t job_list_lock;Signature · 1 line
2.34.1
Boris Brezillon
maintainerRe: [PATCH v2 4/9] drm/sched: Split free_job into own work item
Quoted 216 lines · On Fri, 25 Aug 2023 15:45:49 +0200 Christian König <[email protected]>…
Am 25.08.23 um 15:36 schrieb Matthew Brost:
Quoted 215 lines · Christian König
Quoted 205 lines
Am 25.08.23 um 04:58 schrieb Matthew Brost:
Quoted 204 lines · Danilo Krummrich
Quoted 203 lines · Matthew Brost
Quoted 186 lines
Rather than call
free_jobandrun_jobin same work item have a dedicated work item for each. This aligns with the design and intended use of work queues.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/scheduler/sched_main.c | 137 ++++++++++++++++++------- include/drm/gpu_scheduler.h | 8 +- 2 files changed, 106 insertions(+), 39 deletions(-)drivers/gpu/drm/scheduler/sched_main.c +71
-19@@ -213,11 +213,12 @@ void drm_sched_rq_remove_entity(struct drm_sched_rq *rq,* drm_sched_rq_select_entity_rr - Select an entity which could provide a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Try to find a ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_rr (struct drm_sched_rq *rq,bool dequeue) {struct drm_sched_entity *entity;@@ -227,8 +228,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)if (entity) {list_for_each_entry_continue (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -238,8 +241,10 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)list_for_each_entry (entity, &rq->entities, list) {if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }spin_unlock (&rq->lock);return entity; }@@ -257,11 +262,12 @@ drm_sched_rq_select_entity_rr(struct drm_sched_rq *rq)* drm_sched_rq_select_entity_fifo - Select an entity which provides a job to run * * @rq: scheduler run queue to check. +* @dequeue: dequeue selected entity * * Find oldest waiting ready entity, returns NULL if none found. */ static struct drm_sched_entity * -drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq) +drm_sched_rq_select_entity_fifo (struct drm_sched_rq *rq,bool dequeue) {struct rb_node *rb;@@ -271,8 +277,10 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) entity = rb_entry(rb, struct drm_sched_entity, rb_tree_node);if (drm_sched_entity_is_ready (entity)) { - rq->current_entity = entity; -reinit_completion (&entity->entity_idle); +if (dequeue) { + rq->current_entity = entity; +reinit_completion (&entity->entity_idle); + }break ; } }@@ -282,13 +290,54 @@ drm_sched_rq_select_entity_fifo(struct drm_sched_rq *rq) }/** -* drm_sched_submit_queue - scheduler queue submission +* drm_sched_run_job_queue - queue job submission * @sched: scheduler instance */ -static void drm_sched_submit_queue (struct drm_gpu_scheduler *sched) +static void drm_sched_run_job_queue (struct drm_gpu_scheduler *sched) {if (!READ_ONCE (sched->pause_submit)) -queue_work (sched->submit_wq, &sched->work_submit); +queue_work (sched->submit_wq, &sched->work_run_job); +} + +static struct drm_sched_entity * +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue); + +/** +* drm_sched_run_job_queue_if_ready - queue job submission if ready +* @sched: scheduler instance +*/ +static void drm_sched_run_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +if (drm_sched_select_entity (sched, false)) +drm_sched_run_job_queue (sched); +} + +/** +* drm_sched_free_job_queue - queue free job +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue (struct drm_gpu_scheduler *sched) +{ +if (!READ_ONCE (sched->pause_submit)) +queue_work (sched->submit_wq, &sched->work_free_job); +} + +/** +* drm_sched_free_job_queue_if_ready - queue free job if ready +* +* @sched: scheduler instance to queue free job +*/ +static void drm_sched_free_job_queue_if_ready (struct drm_gpu_scheduler *sched) +{ +struct drm_sched_job *job; + +spin_lock (&sched->job_list_lock); + job =list_first_entry_or_null (&sched->pending_list, + struct drm_sched_job, list); +if (job &&dma_fence_is_signaled (&job->s_fence->finished)) +drm_sched_free_job_queue (sched); +spin_unlock (&sched->job_list_lock); }/** @@ -310,7 +359,7 @@ static void drm_sched_job_done(struct drm_sched_job *s_job, int result)dma_fence_get (&s_fence->finished);drm_sched_fence_finished (s_fence, result);dma_fence_put (&s_fence->finished); -drm_sched_submit_queue (sched); +drm_sched_free_job_queue (sched); }/** @@ -906,18 +955,19 @@ static bool drm_sched_can_queue(struct drm_gpu_scheduler *sched)void drm_sched_wakeup_if_can_queue (struct drm_gpu_scheduler *sched) {if (drm_sched_can_queue (sched)) -drm_sched_submit_queue (sched); +drm_sched_run_job_queue (sched); }/** * drm_sched_select_entity - Select next entity to process * * @sched: scheduler instance +* @dequeue: dequeue selected entity * * Returns the entity to process or NULL if none are found. */ static struct drm_sched_entity * -drm_sched_select_entity (struct drm_gpu_scheduler *sched) +drm_sched_select_entity (struct drm_gpu_scheduler *sched,bool dequeue) {struct drm_sched_entity *entity;int i;@@ -935,8 +985,10 @@ drm_sched_select_entity(struct drm_gpu_scheduler *sched)/* Kernel run queue has higher priority than normal run queue*/ for (i =DRM_SCHED_PRIORITY_COUNT -1 ; i >=DRM_SCHED_PRIORITY_MIN ; i--) { entity = sched->sched_policy ==DRM_SCHED_POLICY_FIFO ? -drm_sched_rq_select_entity_fifo (&sched->sched_rq[i]) : -drm_sched_rq_select_entity_rr (&sched->sched_rq[i]); +drm_sched_rq_select_entity_fifo (&sched->sched_rq[i], + dequeue) : +drm_sched_rq_select_entity_rr (&sched->sched_rq[i], + dequeue);if (entity)break ; }@@ -1024,30 +1076,44 @@ drm_sched_pick_best(struct drm_gpu_scheduler **sched_list,EXPORT_SYMBOL (drm_sched_pick_best);/** -* drm_sched_main - main scheduler thread +* drm_sched_free_job_work - worker to call free_job * -* @param: scheduler instance +* @w: free job work */ -static void drm_sched_main (struct work_struct *w) +static void drm_sched_free_job_work (struct work_struct *w)drivers/gpu/drm/scheduler/sched_main.c drm_sched_pick_best(struct drm_gpu_scheduler **sched_list, · line 1089
Quoted 10 lines
drivers/gpu/drm/scheduler/sched_main.c +1
-3{struct drm_gpu_scheduler *sched = - container_of(w, struct drm_gpu_scheduler, work_submit); -struct drm_sched_entity *entity; + container_of(w, struct drm_gpu_scheduler, work_free_job);struct drm_sched_job *cleanup_job; -int r;if (READ_ONCE (sched->pause_submit))return ; cleanup_job =drm_sched_get_cleanup_job (sched);I tried this patch with Nouveau and found a race condition:
In
drm_sched_run_job_work()the job is added to thepending_listviadrm_sched_job_begin(), then therun_job()callback is called and the scheduled fence is signaled.Quoted 7 lines
Quoted 3 lines
Quoted 3 lines
However, in parallel
drm_sched_get_cleanup_job()might be called fromdrm_sched_free_job_work(), which picks the first job from thepending_listand for the next job on thepending_listsets the scheduled fence' timestamp field.Well why can this happen in parallel? Either the work items are scheduled to a single threaded work queue or you have protected the pending list with some locks.
Xe uses a single-threaded work queue, Nouveau does not (desired behavior).
I'm a bit worried that leaving this single vs multi-threaded wq decision to drivers is going to cause unnecessary pain, because what was previously a granted in term of run/cleanup execution order (thanks to the kthread+static-drm_sched_main-workflow approach) is now subject to the wq ordering guarantees, which depend on the wq type picked by the driver.
Quoted 7 lines
Quoted 7 lines
The list of pending jobs is protected by a lock (safe), the race is:
add job to pending list
run_job
signal scheduled fence
Quoted 7 lines
Quoted 7 lines
dequeue from pending list
free_job
update timestampOnce a job is on the pending list its timestamp can be accessed which can blow up if scheduled fence isn't signaled or more specifically unless
DMA_FENCE_FLAG_TIMESTAMP_BITis set.
Ah, so that's the reason for the TIMESTAMP test added in v3. Sorry for the noise in my v3 review, but I still think it'd be beneficial to have that change moved to its own commit.