VF migration redesign
Subsystems (2)
Matthew Brost
maintainerv5 cover VF migration redesign
Rather than modifying buffers in place using GGTT addresses during VF migration, this approach relies on the submission backend's stop/start mechanism to issue fixups. The patch titled "Document GuC Submission Backend" provides a detailed explanation of the design.
Testing was performed using an out-of-tree PF/VFIO driver with manual triggering of VF migration while IGT test cases are running.
IGT test cases:
- A new series [1] that exercises active contexts, job resubmission, and compressd memory.
- A new test [2] that actively creates / destroys queue on each submission
- xe_exec_threads basic sections, which test context registration loss, schedule enable loss, and job resubmission.
- xe_exec_threads balancer sections, which follow the same flows as the basic sections but include a work queue (GGTT address shift).
- xe_exec_threads compute mode user pointer invalidation sections, which exercise the same flow as the basic sections, plus replaying suspend/resume flows.
All code paths in "Replay GuC submission state on pause/unpause" that replay state have been manually verified via debug messages "Add debug prints for GuC replaying state during VF recovery".
v2:
- Fix lockdep splat
- Fix checkpatch
- Fix PTL issue with LRC W/A buffer
- Fix race creating / destroying queues across migration exposed by [2]
- Include a version of Satya's patches in [3] which enable CCS save / restore across VF migration /w GGTT shift
v3:
- Address feedback
- Fix preempt fence mode deadlock /w work queues + VF recovery (Testing)
- Add NULL checks to scratch LRC allocation
v4:
- Fix CI failure
- Remove config lock
v5:
- Fix CI failures related to lockdep
- Address various comments
Matt
Matthew Brost (28):
drm/xe: Add NULL checks to scratch LRC allocation
drm/xe: Save off position in ring in which a job was programmed drm/xe/guc: Track pending-enable source in submission state drm/xe: Track LR jobs in DRM scheduler pending list drm/xe: Don't change LRC ring head on job resubmission drm/xe: Make LRC W/A scratch buffer usage consistent drm/xe/vf: Add xe_gt_recovery_pending helper
drm/xe/vf: Make VF recovery run on per-GT worker drm/xe/vf: Abort H2G sends during VF post-migration recovery drm/xe/vf: Remove memory allocations from VF post migration recovery drm/xe/vf: Close multi-GT GGTT shift race
drm/xe/vf: Teardown VF post migration worker on driver unload drm/xe/vf: Don't allow GT reset to be queued during VF post migration
recovery
drm/xe/vf: Wakeup in GuC backend on VF post migration recovery drm/xe/vf: Avoid indefinite blocking in preempt rebind worker for VFs
supporting migration
drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register drm/xe/vf: Flush and stop CTs in VF post migration recovery drm/xe/vf: Reset TLB invalidations during VF post migration recovery drm/xe/vf: Kickstart after resfix in VF post migration recovery drm/xe/vf: Start CTs before resfix VF post migration recovery drm/xe/vf: Abort VF post migration recovery on failure drm/xe/vf: Replay GuC submission state on pause / unpause drm/xe: Move queue init before LRC creation
drm/xe/vf: Add debug prints for GuC replaying state during VF recovery drm/xe/vf: Workaround for race condition in GuC firmware during VF
pause
drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VF drm/xe/vf: Ensure media GT VF recovery runs after primary GT on PTL drm/xe/vf: Rebase CCS save/restore BB GGTT addresses
Satyanarayana K V P (2):
drm/xe: Use PPGTT addresses for TLB invalidation to avoid GGTT fixups drm/xe/guc: Increase wait timeout to 2sec after BUSY reply from GuC
drivers/gpu/drm/xe/xe_device_types.h | 5 + drivers/gpu/drm/xe/xe_exec.c | 12 +- drivers/gpu/drm/xe/xe_exec_queue.c | 64 +-- drivers/gpu/drm/xe/xe_exec_queue.h | 2 - drivers/gpu/drm/xe/xe_exec_queue_types.h | 3 + drivers/gpu/drm/xe/xe_execlist.c | 2 +- drivers/gpu/drm/xe/xe_gpu_scheduler.c | 14 + drivers/gpu/drm/xe/xe_gpu_scheduler.h | 2 + drivers/gpu/drm/xe/xe_gt.c | 28 +- drivers/gpu/drm/xe/xe_gt.h | 15 +- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 458 +++++++++++++---- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 13 +- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 33 +- drivers/gpu/drm/xe/xe_guc.c | 4 +- drivers/gpu/drm/xe/xe_guc_ct.c | 121 +++-- drivers/gpu/drm/xe/xe_guc_ct.h | 11 + drivers/gpu/drm/xe/xe_guc_exec_queue_types.h | 15 + drivers/gpu/drm/xe/xe_guc_submit.c | 486 +++++++++++++++---- drivers/gpu/drm/xe/xe_guc_submit.h | 5 +- drivers/gpu/drm/xe/xe_lrc.c | 15 +- drivers/gpu/drm/xe/xe_lrc.h | 10 + drivers/gpu/drm/xe/xe_memirq.c | 48 +- drivers/gpu/drm/xe/xe_memirq.h | 2 + drivers/gpu/drm/xe/xe_migrate.c | 28 +- drivers/gpu/drm/xe/xe_pci.c | 6 +- drivers/gpu/drm/xe/xe_pci_types.h | 1 + drivers/gpu/drm/xe/xe_preempt_fence.c | 11 + drivers/gpu/drm/xe/xe_ring_ops.c | 23 +- drivers/gpu/drm/xe/xe_sched_job_types.h | 9 + drivers/gpu/drm/xe/xe_sriov_vf.c | 240 --------- drivers/gpu/drm/xe/xe_sriov_vf.h | 1 - drivers/gpu/drm/xe/xe_sriov_vf_ccs.c | 28 ++ drivers/gpu/drm/xe/xe_sriov_vf_ccs.h | 1 + drivers/gpu/drm/xe/xe_sriov_vf_types.h | 4 - drivers/gpu/drm/xe/xe_tile.c | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf.c | 30 +- drivers/gpu/drm/xe/xe_tile_sriov_vf.h | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h | 23 + drivers/gpu/drm/xe/xe_vm.c | 26 +- drivers/gpu/drm/xe/xe_vram.c | 6 +- 40 files changed, 1250 insertions(+), 559 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 1/30 drm/xe: Add NULL checks to scratch LRC allocation
kmalloc can fail, the returned value must have a NULL check. This should be immediately after kmalloc for clarity.
v5:
- Assert state->buffer in setup_bo if buffer is iomem (Tomasz)
--- drivers/gpu/drm/xe/xe_lrc.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_lrc.c +9 -4
@@ -1214,8 +1214,7 @@ static int setup_bo(struct bo_setup_state *state)ssize_t remain;if (state->lrc->bo->vmap.is_iomem) { -if (!state->buffer) -return -ENOMEM ; +xe_gt_assert (state->hwe->gt, state->buffer); state->ptr = state->buffer; }else { state->ptr = state->lrc->bo->vmap.vaddr + state->offset;
@@ -1303,8 +1302,11 @@ static int setup_wa_bb(struct xe_lrc *lrc, struct xe_hw_engine *hwe)u32 *buf =NULL ;int ret; -if (lrc->bo->vmap.is_iomem) +if (lrc->bo->vmap.is_iomem) { buf =kmalloc (LRC_WA_BB_SIZE ,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + } ret =xe_lrc_setup_wa_bb_with_scratch (lrc, hwe, buf);
@@ -1347,8 +1349,11 @@ setup_indirect_ctx(struct xe_lrc *lrc, struct xe_hw_engine *hwe)if (xe_gt_WARN_ON (lrc->gt, !state.funcs))return 0 ; -if (lrc->bo->vmap.is_iomem) +if (lrc->bo->vmap.is_iomem) { state.buffer =kmalloc (state.max_size,GFP_KERNEL ); +if (!state.buffer) +return -ENOMEM ; + } ret =setup_bo (&state);if (ret) {
Signature · 1 line
2.34.1
Re: [PATCH v5 01/30] drm/xe: Add NULL checks to scratch LRC allocation
Quoted 5 lines · Matthew Brost
kmalloc can fail, the returned value must have a NULL check. This should be immediately after kmalloc for clarity.
v5:
- Assertstate->bufferinsetup_boif buffer is iomem (Tomasz)
-Tomasz
Quoted 45 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_lrc.c | 13 +++++++++---- 1 file changed, 9 insertions(+), 4 deletions(-)drivers/gpu/drm/xe/xe_lrc.c +9
-4@@ -1214,8 +1214,7 @@ static int setup_bo(struct bo_setup_state *state)ssize_t remain;if (state->lrc->bo->vmap.is_iomem) { -if (!state->buffer) -return -ENOMEM ; +xe_gt_assert (state->hwe->gt, state->buffer); state->ptr = state->buffer; }else { state->ptr = state->lrc->bo->vmap.vaddr + state->offset;@@ -1303,8 +1302,11 @@ static int setup_wa_bb(struct xe_lrc *lrc, struct xe_hw_engine *hwe)u32 *buf =NULL ;int ret; -if (lrc->bo->vmap.is_iomem) +if (lrc->bo->vmap.is_iomem) { buf =kmalloc (LRC_WA_BB_SIZE ,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + } ret =xe_lrc_setup_wa_bb_with_scratch (lrc, hwe, buf);@@ -1347,8 +1349,11 @@ setup_indirect_ctx(struct xe_lrc *lrc, struct xe_hw_engine *hwe)if (xe_gt_WARN_ON (lrc->gt, !state.funcs))return 0 ; -if (lrc->bo->vmap.is_iomem) +if (lrc->bo->vmap.is_iomem) { state.buffer =kmalloc (state.max_size,GFP_KERNEL ); +if (!state.buffer) +return -ENOMEM ; + } ret =setup_bo (&state);if (ret) {
Matthew Brost
maintainerPATCH v5 2/30 drm/xe: Save off position in ring in which a job was programmed
In reply to Matthew BrostVF post-migration recovery needs to modify the ring with updated GGTT addresses for pending jobs. Save off position in ring in which a job was programmed to facilitate.
v4:
- s/VF resume/VF post-migration recovery (Tomasz)
--- drivers/gpu/drm/xe/xe_ring_ops.c | 23 +++++++++++++++++++---- drivers/gpu/drm/xe/xe_sched_job_types.h | 5 +++++ 2 files changed, 24 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_ring_ops.c +19 -4
@@ -245,12 +245,14 @@ static int emit_copy_timestamp(struct xe_lrc *lrc, u32 *dw, int i)/* for engines that don't require any special HW handling (no EUs, no aux inval, etc) */ static void __emit_job_gen12_simple (struct xe_sched_job *job,struct xe_lrc *lrc, -u64 batch_addr,u32 seqno) +u64 batch_addr,u32 *head,u32 seqno) {u32 dw[MAX_JOB_SIZE_DW ], i =0 ;u32 ppgtt_flag =get_ppgtt_flag (job);struct xe_gt *gt = job->q->gt; + *head = lrc->ring.tail; + i =emit_copy_timestamp (lrc, dw, i);if (job->ring_ops_flush_tlb) {
@@ -296,7 +298,7 @@ static bool has_aux_ccs(struct xe_device *xe) }static void __emit_job_gen12_video (struct xe_sched_job *job,struct xe_lrc *lrc, -u64 batch_addr,u32 seqno) +u64 batch_addr,u32 *head,u32 seqno) {u32 dw[MAX_JOB_SIZE_DW ], i =0 ;u32 ppgtt_flag =get_ppgtt_flag (job);
@@ -304,6 +306,8 @@ static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc,struct xe_device *xe =gt_to_xe (gt);bool decode = job->q->class ==XE_ENGINE_CLASS_VIDEO_DECODE ; + *head = lrc->ring.tail; + i =emit_copy_timestamp (lrc, dw, i); dw[i++] =preparser_disable (true);
@@ -346,7 +350,8 @@ static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc,static void __emit_job_gen12_render_compute (struct xe_sched_job *job,struct xe_lrc *lrc, -u64 batch_addr,u32 seqno) +u64 batch_addr,u32 *head, +u32 seqno) {u32 dw[MAX_JOB_SIZE_DW ], i =0 ;u32 ppgtt_flag =get_ppgtt_flag (job);
@@ -355,6 +360,8 @@ static void __emit_job_gen12_render_compute(struct xe_sched_job *job,bool lacks_render = !(gt->info.engine_mask &XE_HW_ENGINE_RCS_MASK );u32 mask_flags =0 ; + *head = lrc->ring.tail; + i =emit_copy_timestamp (lrc, dw, i); dw[i++] =preparser_disable (true);
@@ -396,11 +403,14 @@ static void __emit_job_gen12_render_compute(struct xe_sched_job *job, }static void emit_migration_job_gen12 (struct xe_sched_job *job, -struct xe_lrc *lrc,u32 seqno) +struct xe_lrc *lrc,u32 *head, +u32 seqno) {u32 saddr =xe_lrc_start_seqno_ggtt_addr (lrc);u32 dw[MAX_JOB_SIZE_DW ], i =0 ; + *head = lrc->ring.tail; + i =emit_copy_timestamp (lrc, dw, i); i =emit_store_imm_ggtt (saddr, seqno, dw, i);
@@ -434,6 +444,7 @@ static void emit_job_gen12_gsc(struct xe_sched_job *job)__emit_job_gen12_simple (job, job->q->lrc[0 ], job->ptrs[0 ].batch_addr, + &job->ptrs[0 ].head,xe_sched_job_lrc_seqno (job)); }
@@ -443,6 +454,7 @@ static void emit_job_gen12_copy(struct xe_sched_job *job)if (xe_sched_job_is_migration (job->q)) {emit_migration_job_gen12 (job, job->q->lrc[0 ], + &job->ptrs[0 ].head,xe_sched_job_lrc_seqno (job));return ; }
@@ -450,6 +462,7 @@ static void emit_job_gen12_copy(struct xe_sched_job *job)for (i =0 ; i < job->q->width; ++i)__emit_job_gen12_simple (job, job->q->lrc[i], job->ptrs[i].batch_addr, + &job->ptrs[i].head,xe_sched_job_lrc_seqno (job)); }
@@ -461,6 +474,7 @@ static void emit_job_gen12_video(struct xe_sched_job *job)for (i =0 ; i < job->q->width; ++i)__emit_job_gen12_video (job, job->q->lrc[i], job->ptrs[i].batch_addr, + &job->ptrs[i].head,xe_sched_job_lrc_seqno (job)); }
@@ -471,6 +485,7 @@ static void emit_job_gen12_render_compute(struct xe_sched_job *job)for (i =0 ; i < job->q->width; ++i)__emit_job_gen12_render_compute (job, job->q->lrc[i], job->ptrs[i].batch_addr, + &job->ptrs[i].head,xe_sched_job_lrc_seqno (job)); }
drivers/gpu/drm/xe/xe_sched_job_types.h +5 -0
@@ -24,6 +24,11 @@ struct xe_job_ptrs {struct dma_fence_chain *chain_fence;/** @batch_addr: Batch buffer address. */ u64 batch_addr; +/** +* @head: The tail pointer of the LRC (so head pointer of job) when the +* job was submitted +*/ +u32 head; };/**
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 3/30 drm/xe/guc: Track pending-enable source in submission state
In reply to Matthew BrostAdd explicit tracking in the GuC submission state to record the source of a pending enable (TDR vs. queue resume path vs. submission). Disambiguating the origin lets the GuC submission state machine apply the correct recovery/replay behavior.
This helps VF restore: when the device comes back, the state machine knows whether the pending enable stems from timeout recovery, from a queue resume sequence, or submission and can gate sequencing and fixups accordingly.
v4:
- Clarify commit message (Tomasz)
--- drivers/gpu/drm/xe/xe_guc_submit.c | 36 ++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
drivers/gpu/drm/xe/xe_guc_submit.c +36 -0
@@ -69,6 +69,8 @@ exec_queue_to_guc(struct xe_exec_queue *q)#define EXEC_QUEUE_STATE_BANNED (1 << 9)#define EXEC_QUEUE_STATE_CHECK_TIMEOUT (1 << 10)#define EXEC_QUEUE_STATE_EXTRA_REF (1 << 11) +#define EXEC_QUEUE_STATE_PENDING_RESUME (1 << 12) +#define EXEC_QUEUE_STATE_PENDING_TDR_EXIT (1 << 13)static bool exec_queue_registered (struct xe_exec_queue *q) {
@@ -220,6 +222,36 @@ static void set_exec_queue_extra_ref(struct xe_exec_queue *q)atomic_or (EXEC_QUEUE_STATE_EXTRA_REF , &q->guc->state); } +static bool __maybe_unused exec_queue_pending_resume (struct xe_exec_queue *q) +{ +return atomic_read (&q->guc->state) &EXEC_QUEUE_STATE_PENDING_RESUME ; +} + +static void set_exec_queue_pending_resume (struct xe_exec_queue *q) +{ +atomic_or (EXEC_QUEUE_STATE_PENDING_RESUME , &q->guc->state); +} + +static void clear_exec_queue_pending_resume (struct xe_exec_queue *q) +{ +atomic_and (~EXEC_QUEUE_STATE_PENDING_RESUME , &q->guc->state); +} + +static bool __maybe_unused exec_queue_pending_tdr_exit (struct xe_exec_queue *q) +{ +return atomic_read (&q->guc->state) &EXEC_QUEUE_STATE_PENDING_TDR_EXIT ; +} + +static void set_exec_queue_pending_tdr_exit (struct xe_exec_queue *q) +{ +atomic_or (EXEC_QUEUE_STATE_PENDING_TDR_EXIT , &q->guc->state); +} + +static void clear_exec_queue_pending_tdr_exit (struct xe_exec_queue *q) +{ +atomic_and (~EXEC_QUEUE_STATE_PENDING_TDR_EXIT , &q->guc->state); +} +static bool exec_queue_killed_or_banned_or_wedged (struct xe_exec_queue *q) { return (atomic_read(&q->guc->state) &
@@ -1334,6 +1366,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)return DRM_GPU_SCHED_STAT_RESET ; sched_enable: +set_exec_queue_pending_tdr_exit (q);enable_scheduling (q); rearm:/*
@@ -1493,6 +1526,7 @@ static void __guc_exec_queue_process_msg_resume(struct xe_sched_msg *msg)clear_exec_queue_suspended (q);if (!exec_queue_enabled (q)) { q->guc->resume_time =RESUME_PENDING ; +set_exec_queue_pending_resume (q);enable_scheduling (q); } }else {
@@ -2065,6 +2099,8 @@ static void handle_sched_done(struct xe_guc *guc, struct xe_exec_queue *q,xe_gt_assert (guc_to_gt (guc),exec_queue_pending_enable (q)); q->guc->resume_time =ktime_get (); +clear_exec_queue_pending_resume (q); +clear_exec_queue_pending_tdr_exit (q);clear_exec_queue_pending_enable (q);smp_wmb ();wake_up_all (&guc->ct.wq);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 4/30 drm/xe: Track LR jobs in DRM scheduler pending list
In reply to Matthew BrostVF migration requires jobs to remain pending so they can be replayed after the VF comes back. Previously, LR job fences were intentionally signaled immediately after submission to avoid the risk of exporting them, as these fences do not naturally signal in a timely manner and could break dma-fence contracts. A side effect of this approach was that LR jobs were never added to the DRM scheduler’s pending list, preventing them from being tracked for later resubmission.
We now avoid signaling LR job fences and ensure they are never exported; Xe already guards against exporting these internal fences. With that guarantee in place, we can safely track LR jobs in the scheduler’s pending list so they are eligible for resubmission during VF post-migration recovery (and similar recovery paths).
An added benefit is that LR queues now gain the DRM scheduler’s built-in flow control over ring usage rather than rejecting new jobs in the exec IOCTL if the ring is full.
v2:
- Ensure DRM scheduler TDR doesn't run for LR jobs
- Stack variable for killed_or_banned_or_wedged
v4:
- Clarify commit message (Tomasz)
--- drivers/gpu/drm/xe/xe_exec.c | 12 ++------- drivers/gpu/drm/xe/xe_exec_queue.c | 19 ------------- drivers/gpu/drm/xe/xe_exec_queue.h | 2 -- drivers/gpu/drm/xe/xe_guc_submit.c | 43 ++++++++++++++++++++---------- 4 files changed, 31 insertions(+), 45 deletions(-)
drivers/gpu/drm/xe/xe_exec.c +2 -10
@@ -124,7 +124,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)struct xe_validation_ctx ctx;struct xe_sched_job *job;struct xe_vm *vm; -bool write_locked, skip_retry = false; +bool write_locked;int err =0 ;struct xe_hw_engine_group *group;enum xe_hw_engine_group_execution_mode mode, previous_mode;
@@ -266,12 +266,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_exec; } -if (xe_exec_queue_is_lr (q) &&xe_exec_queue_ring_full (q)) { - err = -EWOULDBLOCK ;/* Aliased to -EAGAIN */ - skip_retry = true; - goto err_exec; - } -if (xe_exec_queue_uses_pxp (q)) { err =xe_vm_validate_protected (q->vm);if (err)
@@ -328,8 +322,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)xe_sched_job_init_user_fence (job, &syncs[i]); } -if (xe_exec_queue_is_lr (q)) - q->ring_ops->emit_job (job);if (!xe_vm_in_lr_mode (vm))xe_exec_queue_last_fence_set (q, vm, &job->drm.s_fence->finished);xe_sched_job_push (job);
@@ -355,7 +347,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)xe_validation_ctx_fini (&ctx); err_unlock_list:up_read (&vm->lock); -if (err == -EAGAIN && !skip_retry) +if (err == -EAGAIN ) goto retry; err_hw_exec_mode:if (mode ==EXEC_MODE_DMA_FENCE )
drivers/gpu/drm/xe/xe_exec_queue.c +0 -19
@@ -847,25 +847,6 @@ bool xe_exec_queue_is_lr(struct xe_exec_queue *q) !(q->flags &EXEC_QUEUE_FLAG_VM ); } -static s32 xe_exec_queue_num_job_inflight (struct xe_exec_queue *q) -{ -return q->lrc[0 ]->fence_ctx.next_seqno -xe_lrc_seqno (q->lrc[0 ]) -1 ; -} - -/** -* xe_exec_queue_ring_full() - Whether an exec_queue's ring is full -* @q: The exec_queue -* -* Return: True if the exec_queue's ring is full, false otherwise. -*/ -bool xe_exec_queue_ring_full (struct xe_exec_queue *q) -{ -struct xe_lrc *lrc = q->lrc[0 ]; -s32 max_job = lrc->ring.size /MAX_JOB_SIZE_BYTES ; - -return xe_exec_queue_num_job_inflight (q) >= max_job; -} -/** * xe_exec_queue_is_idle() - Whether an exec_queue is idle. * @q: The exec_queue
drivers/gpu/drm/xe/xe_exec_queue.h +0 -2
@@ -64,8 +64,6 @@ static inline bool xe_exec_queue_uses_pxp(struct xe_exec_queue *q)bool xe_exec_queue_is_lr (struct xe_exec_queue *q); -bool xe_exec_queue_ring_full (struct xe_exec_queue *q); -bool xe_exec_queue_is_idle (struct xe_exec_queue *q);void xe_exec_queue_kill (struct xe_exec_queue *q);
drivers/gpu/drm/xe/xe_guc_submit.c +29 -14
92 lines not on this page · raw message
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 5/30 drm/xe: Don't change LRC ring head on job resubmission
In reply to Matthew BrostNow that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.
v5:
- Add comment in guc_exec_queue_start explaning why the function works (Auld)
--- drivers/gpu/drm/xe/xe_guc_submit.c | 18 ++++++++++++++++-- 1 file changed, 16 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_submit.c +16 -2
@@ -2008,11 +2008,25 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)struct xe_gpu_scheduler *sched = &q->guc->sched;if (!exec_queue_killed_or_banned_or_wedged (q)) { +struct xe_sched_job *job =xe_sched_first_pending_job (sched);int i;trace_xe_exec_queue_resubmit (q); -for (i =0 ; i < q->width; ++i) -xe_lrc_set_ring_head (q->lrc[i], q->lrc[i]->ring.tail); +if (job) { +for (i =0 ; i < q->width; ++i) { +/* +* The GuC context is unregistered at this point +* time, adjusting software ring tail ensures +* jobs are rewritten in original placement, +* adjusting LRC tail ensures the newly loaded +* GuC / contexts only view the LRC tail +* increasing as jobs are written out. +*/ + q->lrc[i]->ring.tail = job->ptrs[i].head; +xe_lrc_set_ring_tail (q->lrc[i], +xe_lrc_ring_head (q->lrc[i])); + } + }xe_sched_resubmit_jobs (sched); }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 6/30 drm/xe: Make LRC W/A scratch buffer usage consistent
In reply to Matthew BrostThe LRC W/A currently checks for LRC being iomem in some places, while in others it checks if the scratch buffer is non-NULL. This inconsistency causes issues with the VF post-migration recovery code, which blindly passes in a scratch buffer.
This patch standardizes the check by consistently verifying whether the LRC is iomem to determine if the scratch buffer should be used.
--- drivers/gpu/drm/xe/xe_lrc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_lrc.c +1 -1
@@ -1247,7 +1247,7 @@ static int setup_bo(struct bo_setup_state *state)static void finish_bo (struct bo_setup_state *state) { -if (!state->buffer) +if (!state->lrc->bo->vmap.is_iomem)return ; xe_map_memcpy_to(gt_to_xe(state->lrc->gt), &state->lrc->bo->vmap,
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 7/30 drm/xe/vf: Add xe_gt_recovery_pending helper
In reply to Matthew BrostAdd xe_gt_recovery_pending helper.
This helper serves as the singular point to determine whether a GT recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.
v3:
- Add GT layer xe_gt_recovery_inprogress (Michal)
- Don't blow up in memirq not enabled (CI)
- Add __memirq_received with clear argument (Michal)
- xe_memirq_sw_int_0_irq_pending rename (Michal)
- Use offset in xe_memirq_sw_int_0_irq_pending (Michal)
v4:
- Refactor xe_gt_recovery_inprogress logic around memirq (Michal)
v5:
- s/inprogress/pending (Michal)
--- drivers/gpu/drm/xe/xe_gt.h | 13 ++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 27 +++++++++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++ drivers/gpu/drm/xe/xe_memirq.c | 48 +++++++++++++++++++++-- drivers/gpu/drm/xe/xe_memirq.h | 2 + 6 files changed, 98 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_gt.h +13 -0
@@ -12,6 +12,7 @@#include "xe_device.h" #include "xe_device_types.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine.h" #define for_each_hw_engine (hwe__, gt__, id__) \
@@ -124,4 +125,16 @@ static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) hwe->instance == gt->usm.reserved_bcs_instance; } +/** +* xe_gt_recovery_pending() - GT recovery pending +* @gt: the &xe_gt +* +* Return: True if GT recovery in pending, False otherwise +*/ +static inline bool xe_gt_recovery_pending (struct xe_gt *gt) +{ +return IS_SRIOV_VF (gt_to_xe (gt)) && +xe_gt_sriov_vf_recovery_pending (gt); +} +#endif
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +27 -0
@@ -26,6 +26,7 @@#include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_lrc.h" +#include "xe_memirq.h" #include "xe_mmio.h" #include "xe_sriov.h" #include "xe_sriov_vf.h"
@@ -776,6 +777,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)struct xe_device *xe =gt_to_xe (gt);xe_gt_assert (gt,IS_SRIOV_VF (xe)); +xe_gt_assert (gt,xe_gt_sriov_vf_recovery_pending (gt));set_bit (gt->info.id, &xe->sriov.vf.migration.gt_flags);/*
@@ -1118,3 +1120,28 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)drm_printf (p,"\thandshake:\t%u.%u\n" , pf_version->major, pf_version->minor); } + +/** +* xe_gt_sriov_vf_recovery_pending() - VF post migration recovery pending +* @gt: the &xe_gt +* +* This function's return value must be immediately visable upon vCPU unhalt and +* be persisent until RESFIX_DONE is issued. This guarnetee is only coded for +* platforms which support memirq, if non-memirq platforms support VF migration +* this function will need to be updated. +* +* Return: True if VF post migration recovery in pending, False otherwise +*/ +bool xe_gt_sriov_vf_recovery_pending (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +/* early detection until recovery starts */ +if (xe_device_uses_memirq (gt_to_xe (gt)) && +xe_memirq_guc_sw_int_0_irq_pending (memirq, >->uc.guc)) +return true; + +return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); +}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +bool xe_gt_sriov_vf_recovery_pending (struct xe_gt *gt); +u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10 -0
@@ -46,6 +46,14 @@ struct xe_gt_sriov_vf_runtime { } *regs; }; +/** +* xe_gt_sriov_vf_migration - VF migration data. +*/ +struct xe_gt_sriov_vf_migration { +/** @recovery_inprogress: VF post migration recovery in progress */ +bool recovery_inprogress; +}; +/** * struct xe_gt_sriov_vf - GT level VF virtualization data. */
@@ -58,6 +66,8 @@ struct xe_gt_sriov_vf {struct xe_gt_sriov_vf_selfconfig self_config;/** @runtime: runtime data retrieved from the PF. */ struct xe_gt_sriov_vf_runtime runtime; +/** @migration: migration data for the VF. */ + struct xe_gt_sriov_vf_migration migration; };#endif
drivers/gpu/drm/xe/xe_memirq.c +44 -4
78 lines not on this page · raw message
drivers/gpu/drm/xe/xe_memirq.h +2 -0
@@ -25,4 +25,6 @@ void xe_memirq_handler(struct xe_memirq *memirq);int xe_memirq_init_guc (struct xe_memirq *memirq,struct xe_guc *guc); +bool xe_memirq_guc_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc); +#endif
Signature · 1 line
2.34.1
Re: [PATCH v5 07/30] drm/xe/vf: Add xe_gt_recovery_pending helper
Quoted 7 lines · Matthew Brost
Add
xe_gt_recovery_pendinghelper.This helper serves as the singular point to determine whether a GT recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.
Quoted 10 lines
v3:
- Add GT layerxe_gt_recovery_inprogress(Michal)
- Don't blow up in memirq not enabled (CI)
- Add__memirq_receivedwith clear argument (Michal)
-xe_memirq_sw_int_0_irq_pendingrename (Michal)
- Use offset inxe_memirq_sw_int_0_irq_pending(Michal)
v4:
- Refactorxe_gt_recovery_inprogresslogic around memirq (Michal)
v5:
- s/inprogress/pending (Michal)
-Tomasz
Quoted 227 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt.h | 13 ++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 27 +++++++++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++ drivers/gpu/drm/xe/xe_memirq.c | 48 +++++++++++++++++++++-- drivers/gpu/drm/xe/xe_memirq.h | 2 + 6 files changed, 98 insertions(+), 4 deletions(-)drivers/gpu/drm/xe/xe_gt.h +13
-0@@ -12,6 +12,7 @@#include "xe_device.h" #include "xe_device_types.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine.h" #define for_each_hw_engine (hwe__, gt__, id__) \@@ -124,4 +125,16 @@ static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) hwe->instance == gt->usm.reserved_bcs_instance; } +/** +* xe_gt_recovery_pending() - GT recovery pending +* @gt: the &xe_gt +* +* Return: True if GT recovery in pending, False otherwise +*/ +static inline bool xe_gt_recovery_pending (struct xe_gt *gt) +{ +return IS_SRIOV_VF (gt_to_xe (gt)) && +xe_gt_sriov_vf_recovery_pending (gt); +} +#endif drivers/gpu/drm/xe/xe_gt_sriov_vf.c +27
-0@@ -26,6 +26,7 @@#include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_lrc.h" +#include "xe_memirq.h" #include "xe_mmio.h" #include "xe_sriov.h" #include "xe_sriov_vf.h" @@ -776,6 +777,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)struct xe_device *xe =gt_to_xe (gt);xe_gt_assert (gt,IS_SRIOV_VF (xe)); +xe_gt_assert (gt,xe_gt_sriov_vf_recovery_pending (gt));set_bit (gt->info.id, &xe->sriov.vf.migration.gt_flags);/* @@ -1118,3 +1120,28 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)drm_printf (p,"\thandshake:\t%u.%u\n" , pf_version->major, pf_version->minor); } + +/** +* xe_gt_sriov_vf_recovery_pending() - VF post migration recovery pending +* @gt: the &xe_gt +* +* This function's return value must be immediately visable upon vCPU unhalt and +* be persisent until RESFIX_DONE is issued. This guarnetee is only coded for +* platforms which support memirq, if non-memirq platforms support VF migration +* this function will need to be updated. +* +* Return: True if VF post migration recovery in pending, False otherwise +*/ +bool xe_gt_sriov_vf_recovery_pending (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +/* early detection until recovery starts */ +if (xe_device_uses_memirq (gt_to_xe (gt)) && +xe_memirq_guc_sw_int_0_irq_pending (memirq, >->uc.guc)) +return true; + +return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); +}drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2
-0@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +bool xe_gt_sriov_vf_recovery_pending (struct xe_gt *gt); +u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt);drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10
-0@@ -46,6 +46,14 @@ struct xe_gt_sriov_vf_runtime { } *regs; }; +/** +* xe_gt_sriov_vf_migration - VF migration data. +*/ +struct xe_gt_sriov_vf_migration { +/** @recovery_inprogress: VF post migration recovery in progress */ +bool recovery_inprogress; +}; +/** * struct xe_gt_sriov_vf - GT level VF virtualization data. */ @@ -58,6 +66,8 @@ struct xe_gt_sriov_vf {struct xe_gt_sriov_vf_selfconfig self_config;/** @runtime: runtime data retrieved from the PF. */ struct xe_gt_sriov_vf_runtime runtime; +/** @migration: migration data for the VF. */ + struct xe_gt_sriov_vf_migration migration; };#endif drivers/gpu/drm/xe/xe_memirq.c +44
-4@@ -398,8 +398,9 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)memirq_set_enable (memirq, true); } -static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, -u16 offset,const char *name) +static bool __memirq_received (struct xe_memirq *memirq, +struct iosys_map *vector,u16 offset, +const char *name,bool clear) {u8 value;@@ -409,12 +410,26 @@ static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,memirq_err_ratelimited (memirq,"Unexpected memirq value %#x from %s at %u\n" , value, name, offset); -iosys_map_wr (vector, offset, u8,0x00 ); +if (clear) +iosys_map_wr (vector, offset, u8,0x00 ); }return value; } +static bool memirq_received_noclear (struct xe_memirq *memirq, +struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, false); +} + +static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, true); +} +static void memirq_dispatch_engine (struct xe_memirq *memirq,struct iosys_map *status,struct xe_hw_engine *hwe) {@@ -434,8 +449,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *statif (memirq_received (memirq, status,ilog2 (GUC_INTR_GUC2HOST ), name))xe_guc_irq_handler (guc,GUC_INTR_GUC2HOST ); -if (memirq_received (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), name)) +/* +* We must wait to perform the clear operation until after +* xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race +* conditions where xe_gt_sriov_vf_recovery_pending() returns false. +*/ +if (memirq_received_noclear (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), + name)) {xe_guc_irq_handler (guc,GUC_INTR_SW_INT_0 ); +iosys_map_wr (status,ilog2 (GUC_INTR_SW_INT_0 ), u8,0x00 ); + } }/** @@ -460,6 +483,23 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) } } +/** +* xe_memirq_guc__sw_int_0_irq_pending() - SW_INT_0 IRQ is pending +* @memirq: the &xe_memirq +* @guc: the &xe_guc to check for IRQ +* +* Return: True if SW_INT_0 IRQ is pending on @guc, False otherwise +*/ +bool xe_memirq_guc_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc) +{ +struct xe_gt *gt =guc_to_gt (guc); +u32 offset =xe_gt_is_media_type (gt) ?ilog2 (INTR_MGUC ) :ilog2 (INTR_GUC ); +struct iosys_map map =IOSYS_MAP_INIT_OFFSET (&memirq->status, offset *SZ_16 ); + +return memirq_received_noclear (memirq, &map,ilog2 (GUC_INTR_SW_INT_0 ), +guc_name (guc)); +} +/** * xe_memirq_handler - The `Memory Based Interrupts`_ Handler. * @memirq: the &xe_memirq drivers/gpu/drm/xe/xe_memirq.h +2
-0@@ -25,4 +25,6 @@ void xe_memirq_handler(struct xe_memirq *memirq);int xe_memirq_init_guc (struct xe_memirq *memirq,struct xe_guc *guc); +bool xe_memirq_guc_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc); +#endif
Matthew Brost
maintainerPATCH v5 8/30 drm/xe/vf: Make VF recovery run on per-GT worker
In reply to Matthew BrostVF recovery is a per-GT operation, so it makes sense to isolate it to a per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.
v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper
v3:
- Drop xe_gt_sriov_ prefix for private functions (Michal)
- Drop message in xe_gt_sriov_vf_migration_init_early (Michal)
- Logic rework in vf_post_migration_notify_resfix_done (Michal)
- Rework init sequence layering (Michal)
--- drivers/gpu/drm/xe/xe_gt.c | 6 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 178 +++++++++++++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 3 +- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 7 + drivers/gpu/drm/xe/xe_sriov_vf.c | 240 ---------------------- drivers/gpu/drm/xe/xe_sriov_vf.h | 1 - drivers/gpu/drm/xe/xe_sriov_vf_types.h | 4 - 7 files changed, 181 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +6 -0
@@ -388,6 +388,12 @@ int xe_gt_init_early(struct xe_gt *gt)return err; } +if (IS_SRIOV_VF (gt_to_xe (gt))) { + err =xe_gt_sriov_vf_init_early (gt); +if (err) +return err; + } +xe_reg_sr_init (>->reg_sr,"GT" ,gt_to_xe (gt)); err =xe_wa_gt_init (gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +167 -11
225 lines not on this page · raw message
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,int xe_gt_sriov_vf_query_config (struct xe_gt *gt);int xe_gt_sriov_vf_connect (struct xe_gt *gt);int xe_gt_sriov_vf_query_runtime (struct xe_gt *gt); -void xe_gt_sriov_vf_default_lrcs_hwsp_rebase (struct xe_gt *gt); -int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +int xe_gt_sriov_vf_init_early (struct xe_gt *gt);bool xe_gt_sriov_vf_recovery_pending (struct xe_gt *gt);u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
@@ -7,6 +7,7 @@#define _XE_GT_SRIOV_VF_TYPES_H_#include <linux/types.h> +#include <linux/workqueue.h> #include "xe_uc_fw_types.h" /**
@@ -50,6 +51,12 @@ struct xe_gt_sriov_vf_runtime {* xe_gt_sriov_vf_migration - VF migration data. */ struct xe_gt_sriov_vf_migration { +/** @migration: VF migration recovery worker */ +struct work_struct worker; +/** @lock: Protects recovery_queued */ +spinlock_t lock; +/** @recovery_queued: VF post migration recovery in queued */ +bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */ bool recovery_inprogress; };
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -240
270 lines not on this page · raw message
drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
@@ -13,7 +13,6 @@ struct xe_device;void xe_sriov_vf_init_early (struct xe_device *xe);int xe_sriov_vf_init_late (struct xe_device *xe); -void xe_sriov_vf_start_migration_recovery (struct xe_device *xe);bool xe_sriov_vf_migration_supported (struct xe_device *xe);void xe_sriov_vf_debugfs_register (struct xe_device *xe,struct dentry *root);
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
@@ -33,10 +33,6 @@ struct xe_device_vf {/** @migration: VF Migration state data */ struct { -/** @migration.worker: VF migration recovery worker */ -struct work_struct worker; -/** @migration.gt_flags: Per-GT request flags for VF migration recovery */ -unsigned long gt_flags;/** * @migration.enabled: flag indicating if migration support * was enabled or not due to missing prerequisites
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 9/30 drm/xe/vf: Abort H2G sends during VF post-migration recovery
In reply to Matthew BrostWhile VF post-migration recovery is in progress, abort H2G sends with -ECANCEL. These messages are treated as lost, and TLB invalidation errors are suppressed. During this phase, the H2G channel is down, and VF recovery requires the CT lock to proceed.
v3:
- Use xe_gt_recovery_inprogress (Michal)
--- drivers/gpu/drm/xe/xe_guc_ct.c | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +2 -2
@@ -851,7 +851,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len, u32 g2h_len, u32 num_g2h, struct g2h_fence *g2h_fence) { - struct xe_gt *gt __maybe_unused = ct_to_gt(ct); + struct xe_gt *gt = ct_to_gt(ct);u16 seqno;int ret;
@@ -872,7 +872,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, goto out; } -if (ct->state ==XE_GUC_CT_STATE_STOPPED ) { +if (ct->state ==XE_GUC_CT_STATE_STOPPED ||xe_gt_recovery_pending (gt)) { ret = -ECANCELED ; goto out; }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 10/30 drm/xe/vf: Remove memory allocations from VF post migration recovery
In reply to Matthew BrostVF post migration recovery is the path of dma-fence signaling / reclaim, avoid memory allocations in this path.
v3:
- s/lrc_wa_bb/scratch (Tomasz)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 23 +++++++++++++---------- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 2 ++ 2 files changed, 15 insertions(+), 10 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +13 -10
@@ -1160,17 +1160,13 @@ static size_t post_migration_scratch_size(struct xe_device *xe)static int vf_post_migration_fixups (struct xe_gt *gt) { +void *buf = gt->sriov.vf.migration.scratch;s64 shift; -void *buf;int err; - buf =kmalloc (post_migration_scratch_size (gt_to_xe (gt)),GFP_ATOMIC ); -if (!buf) -return -ENOMEM ; - err =xe_gt_sriov_vf_query_config (gt);if (err) - goto out; +return err; shift =xe_gt_sriov_vf_ggtt_shift (gt);if (shift) {
@@ -1178,12 +1174,10 @@ static int vf_post_migration_fixups(struct xe_gt *gt)xe_gt_sriov_vf_default_lrcs_hwsp_rebase (gt); err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf);if (err) - goto out; +return err; } -out: -kfree (buf); -return err; +return 0 ; }static void vf_post_migration_kickstart (struct xe_gt *gt)
@@ -1268,9 +1262,18 @@ static void migration_worker_func(struct work_struct *w)*/ int xe_gt_sriov_vf_init_early (struct xe_gt *gt) { +void *buf; +if (!xe_sriov_vf_migration_supported (gt_to_xe (gt)))return 0 ; + buf =drmm_kmalloc (>_to_xe (gt)->drm, +post_migration_scratch_size (gt_to_xe (gt)), +GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + gt->sriov.vf.migration.scratch = buf;spin_lock_init (>->sriov.vf.migration.lock);INIT_WORK (>->sriov.vf.migration.worker, migration_worker_func);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +2 -0
@@ -55,6 +55,8 @@ struct xe_gt_sriov_vf_migration {struct work_struct worker;/** @lock: Protects recovery_queued */ spinlock_t lock; +/** @scratch: Scratch memory for VF recovery */ +void *scratch;/** @recovery_queued: VF post migration recovery in queued */ bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 11/30 drm/xe/vf: Close multi-GT GGTT shift race
In reply to Matthew BrostAs multi-GT VF post-migration recovery can run in parallel on different workqueues, but both GTs point to the same GGTT, only one GT needs to shift the GGTT. However, both GTs need to know when this step has completed. To coordinate this, perform the GGTT shift under the GGTT lock. With shift being done under the lock, storing the shift value becomes unnecessary.
v3:
- Update commmit message (Tomasz)
v4:
- Move GGTT values to tile state (Michal)
- Use GGTT lock (Michal)
v5:
- Only take GGTT lock during recovery (CI)
- Drop goto in vf_get_submission_cfg (Michal)
- Add kernel doc around recovery in xe_gt_sriov_vf_query_config (Michal)
--- drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 153 +++++++------------- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 5 +- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 7 +- drivers/gpu/drm/xe/xe_guc.c | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf.c | 30 +++- drivers/gpu/drm/xe/xe_tile_sriov_vf.h | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h | 23 +++ drivers/gpu/drm/xe/xe_vram.c | 6 +- 9 files changed, 112 insertions(+), 119 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h
drivers/gpu/drm/xe/xe_device_types.h +3 -0
@@ -27,6 +27,7 @@#include "xe_sriov_vf_ccs_types.h" #include "xe_step_types.h" #include "xe_survivability_mode_types.h" +#include "xe_tile_sriov_vf_types.h" #include "xe_validation.h" #if IS_ENABLED (CONFIG_DRM_XE_DEBUG )
@@ -185,6 +186,8 @@ struct xe_tile {struct {/** @sriov.vf.ggtt_balloon: GGTT regions excluded from use. */ struct xe_ggtt_node *ggtt_balloon[2 ]; +/** @sriov.vf.self_config: VF configuration data */ +struct xe_tile_sriov_vf_selfconfig self_config; } vf; } sriov;
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +55 -98
238 lines not on this page · raw message
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -4
@@ -18,7 +18,7 @@ int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);void xe_gt_sriov_vf_guc_versions (struct xe_gt *gt,struct xe_uc_fw_version *wanted,struct xe_uc_fw_version *found); -int xe_gt_sriov_vf_query_config (struct xe_gt *gt); +int xe_gt_sriov_vf_query_config (struct xe_gt *gt,bool recovery);int xe_gt_sriov_vf_connect (struct xe_gt *gt);int xe_gt_sriov_vf_query_runtime (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt);
@@ -29,9 +29,6 @@ bool xe_gt_sriov_vf_recovery_pending(struct xe_gt *gt);u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt); -u64 xe_gt_sriov_vf_ggtt (struct xe_gt *gt); -u64 xe_gt_sriov_vf_ggtt_base (struct xe_gt *gt); -s64 xe_gt_sriov_vf_ggtt_shift (struct xe_gt *gt);u32 xe_gt_sriov_vf_read32 (struct xe_gt *gt,struct xe_reg reg);void xe_gt_sriov_vf_write32 (struct xe_gt *gt,struct xe_reg reg,u32 val);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +1 -6
@@ -6,6 +6,7 @@#ifndef _XE_GT_SRIOV_VF_TYPES_H_#define _XE_GT_SRIOV_VF_TYPES_H_ +#include <linux/rwsem.h> #include <linux/types.h> #include <linux/workqueue.h> #include "xe_uc_fw_types.h"
@@ -14,12 +15,6 @@* struct xe_gt_sriov_vf_selfconfig - VF configuration data. */ struct xe_gt_sriov_vf_selfconfig { -/** @ggtt_base: assigned base offset of the GGTT region. */ -u64 ggtt_base; -/** @ggtt_size: assigned size of the GGTT region. */ -u64 ggtt_size; -/** @ggtt_shift: difference in ggtt_base on last migration */ -s64 ggtt_shift;/** @lmem_size: assigned size of the LMEM. */ u64 lmem_size;/** @num_ctxs: assigned number of GuC submission context IDs. */
drivers/gpu/drm/xe/xe_guc.c +1 -1
@@ -713,7 +713,7 @@ static int vf_guc_init_noalloc(struct xe_guc *guc)if (err)return err; - err =xe_gt_sriov_vf_query_config (gt); + err =xe_gt_sriov_vf_query_config (gt, false);if (err)return err;
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +24 -6
@@ -9,7 +9,6 @@#include "xe_assert.h" #include "xe_ggtt.h" -#include "xe_gt_sriov_vf.h" #include "xe_sriov.h" #include "xe_sriov_printk.h" #include "xe_tile_sriov_vf.h"
@@ -40,10 +39,10 @@ static int vf_init_ggtt_balloons(struct xe_tile *tile)* * Return: 0 on success or a negative error code on failure. */ -int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile) +static int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile) { -u64 ggtt_base =xe_gt_sriov_vf_ggtt_base (tile->primary_gt); -u64 ggtt_size =xe_gt_sriov_vf_ggtt (tile->primary_gt); +u64 ggtt_base = tile->sriov.vf.self_config.ggtt_base; +u64 ggtt_size = tile->sriov.vf.self_config.ggtt_size;struct xe_device *xe =tile_to_xe (tile);u64 wopcm =xe_wopcm_size (xe);u64 start, end;
@@ -244,11 +243,30 @@ void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift) {struct xe_ggtt *ggtt = tile->mem.ggtt; -mutex_lock (&ggtt->lock); +lockdep_assert_held (&ggtt->lock);xe_tile_sriov_vf_deballoon_ggtt_locked (tile);xe_ggtt_shift_nodes_locked (ggtt, shift);xe_tile_sriov_vf_balloon_ggtt_locked (tile); +} -mutex_unlock (&ggtt->lock); +/** +* xe_tile_sriov_vf_lmem - VF LMEM configuration. +* @tile: the &xe_tile +* +* This function is for VF use only. +* +* Return: size of the LMEM assigned to VF. +*/ +u64 xe_tile_sriov_vf_lmem (struct xe_tile *tile) +{ +struct xe_tile_sriov_vf_selfconfig *config = &tile->sriov.vf.self_config; +u64 val; + +xe_tile_assert (tile,IS_SRIOV_VF (tile_to_xe (tile))); + +xe_tile_assert (tile, config->lmem_size); + val = config->lmem_size; + +return val; }
drivers/gpu/drm/xe/xe_tile_sriov_vf.h +1 -1
@@ -11,8 +11,8 @@struct xe_tile ;int xe_tile_sriov_vf_prepare_ggtt (struct xe_tile *tile); -int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile);void xe_tile_sriov_vf_deballoon_ggtt_locked (struct xe_tile *tile);void xe_tile_sriov_vf_fixup_ggtt_nodes (struct xe_tile *tile,s64 shift); +u64 xe_tile_sriov_vf_lmem (struct xe_tile *tile);#endif
drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2025 Intel Corporation +*/ + +#ifndef _XE_TILE_SRIOV_VF_TYPES_H_ +#define _XE_TILE_SRIOV_VF_TYPES_H_ + +#include <linux/mutex.h> + +/** +* struct xe_tile_sriov_vf_selfconfig - VF configuration data. +*/ +struct xe_tile_sriov_vf_selfconfig { +/** @ggtt_base: assigned base offset of the GGTT region. */ +u64 ggtt_base; +/** @ggtt_size: assigned size of the GGTT region. */ +u64 ggtt_size; +/** @lmem_size: assigned size of the LMEM. */ +u64 lmem_size; +}; + +#endif
drivers/gpu/drm/xe/xe_vram.c +3 -3
@@ -16,10 +16,10 @@#include "xe_device.h" #include "xe_force_wake.h" #include "xe_gt_mcr.h" -#include "xe_gt_sriov_vf.h" #include "xe_mmio.h" #include "xe_module.h" #include "xe_sriov.h" +#include "xe_tile_sriov_vf.h" #include "xe_ttm_vram_mgr.h" #include "xe_vram.h" #include "xe_vram_types.h"
@@ -237,9 +237,9 @@ static int tile_vram_size(struct xe_tile *tile, u64 *vram_size, offset =0 ;for_each_tile (t, xe, id)for_each_if (t->id < tile->id) - offset +=xe_gt_sriov_vf_lmem (t->primary_gt); + offset +=xe_tile_sriov_vf_lmem (t); - *tile_size =xe_gt_sriov_vf_lmem (gt); + *tile_size =xe_tile_sriov_vf_lmem (tile); *vram_size = *tile_size; *tile_offset = offset;
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 12/30 drm/xe/vf: Teardown VF post migration worker on driver unload
In reply to Matthew BrostBe cautious and ensure the VF post-migration worker is not running during driver unload.
v3:
- More teardown later in driver init, use devm (Tomasz)
--- drivers/gpu/drm/xe/xe_gt.c | 6 ++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 34 ++++++++++++++++++++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 1 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 4 ++- 4 files changed, 43 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +6 -0
@@ -653,6 +653,12 @@ int xe_gt_init(struct xe_gt *gt)if (err)return err; +if (IS_SRIOV_VF (gt_to_xe (gt))) { + err =xe_gt_sriov_vf_init (gt); +if (err) +return err; + } +return 0 ; }
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +33 -1
@@ -739,7 +739,8 @@ static void vf_start_migration_recovery(struct xe_gt *gt)spin_lock (>->sriov.vf.migration.lock); -if (!gt->sriov.vf.migration.recovery_queued) { +if (!gt->sriov.vf.migration.recovery_queued || + !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true);
@@ -1211,6 +1212,17 @@ static void migration_worker_func(struct work_struct *w)vf_post_migration_recovery (gt); } +static void vf_migration_fini (void *arg) +{ +struct xe_gt *gt = arg; + +spin_lock_irq (>->sriov.vf.migration.lock); + gt->sriov.vf.migration.recovery_teardown = true; +spin_unlock_irq (>->sriov.vf.migration.lock); + +cancel_work_sync (>->sriov.vf.migration.worker); +} +/** * xe_gt_sriov_vf_init_early() - GT VF init early * @gt: the &xe_gt
@@ -1237,6 +1249,26 @@ int xe_gt_sriov_vf_init_early(struct xe_gt *gt)return 0 ; } +/** +* xe_gt_sriov_vf_init() - GT VF init +* @gt: the &xe_gt +* +* Return 0 on success, errno on failure +*/ +int xe_gt_sriov_vf_init (struct xe_gt *gt) +{ +if (!xe_sriov_vf_migration_supported (gt_to_xe (gt))) +return 0 ; + +/* +* We want to tear down the VF post-migration early during driver +* unload; therefore, we add this finalization action later during +* driver load. +*/ +return devm_add_action_or_reset (gt_to_xe (gt)->drm.dev, + vf_migration_fini, gt); +} +/** * xe_gt_sriov_vf_recovery_pending() - VF post migration recovery pending * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -0
@@ -24,6 +24,7 @@ int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt);int xe_gt_sriov_vf_init_early (struct xe_gt *gt); +int xe_gt_sriov_vf_init (struct xe_gt *gt);bool xe_gt_sriov_vf_recovery_pending (struct xe_gt *gt);u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -1
@@ -48,10 +48,12 @@ struct xe_gt_sriov_vf_runtime {struct xe_gt_sriov_vf_migration {/** @migration: VF migration recovery worker */ struct work_struct worker; -/** @lock: Protects recovery_queued */ +/** @lock: Protects recovery_queued, teardown */ spinlock_t lock;/** @scratch: Scratch memory for VF recovery */ void *scratch; +/** @recovery_teardown: VF post migration recovery is being torn down */ +bool recovery_teardown;/** @recovery_queued: VF post migration recovery in queued */ bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 13/30 drm/xe/vf: Don't allow GT reset to be queued during VF post migration recovery
In reply to Matthew BrostWith well-behaved software, a GT reset should never occur, nor should it happen during VF post-migration recovery. If it does, trigger a warning but suppress the GT reset, as VF post-migration recovery is expected to bring the VF back to a working state.
v3:
- Better commit message (Tomasz)
v5:
- Use xe_gt_WARN_ON (Michal)
--- drivers/gpu/drm/xe/xe_gt.c | 9 ------- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 ----- drivers/gpu/drm/xe/xe_guc_submit.c | 42 ++++------------------------- drivers/gpu/drm/xe/xe_guc_submit.h | 3 --- 4 files changed, 5 insertions(+), 56 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +0 -9
@@ -805,11 +805,6 @@ static int do_gt_restart(struct xe_gt *gt)return 0 ; } -static int gt_wait_reset_unblock (struct xe_gt *gt) -{ -return xe_guc_wait_reset_unblock (>->uc.guc); -} -static int gt_reset (struct xe_gt *gt) {unsigned int fw_ref;
@@ -824,10 +819,6 @@ static int gt_reset(struct xe_gt *gt)xe_gt_info (gt,"reset started\n" ); - err =gt_wait_reset_unblock (gt); -if (!err) -xe_gt_warn (gt,"reset block failed to get lifted" ); -xe_pm_runtime_get (gt_to_xe (gt));if (xe_fault_inject_gt_reset ()) {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -7
@@ -1103,17 +1103,11 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)static void vf_post_migration_shutdown (struct xe_gt *gt) { -int ret =0 ; -spin_lock_irq (>->sriov.vf.migration.lock); gt->sriov.vf.migration.recovery_queued = false;spin_unlock_irq (>->sriov.vf.migration.lock);xe_guc_submit_pause (>->uc.guc); - ret |=xe_guc_submit_reset_block (>->uc.guc); - -if (ret) -xe_gt_sriov_info (gt,"migration recovery encountered ongoing reset\n" ); }static size_t post_migration_scratch_size (struct xe_device *xe)
@@ -1147,7 +1141,6 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)*/ xe_irq_resume (gt_to_xe (gt)); -xe_guc_submit_reset_unblock (>->uc.guc);xe_guc_submit_unpause (>->uc.guc); }
drivers/gpu/drm/xe/xe_guc_submit.c +5 -37
58 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_submit.h +0 -3
@@ -22,9 +22,6 @@ void xe_guc_submit_stop(struct xe_guc *guc);int xe_guc_submit_start (struct xe_guc *guc);void xe_guc_submit_pause (struct xe_guc *guc);void xe_guc_submit_unpause (struct xe_guc *guc); -int xe_guc_submit_reset_block (struct xe_guc *guc); -void xe_guc_submit_reset_unblock (struct xe_guc *guc); -int xe_guc_wait_reset_unblock (struct xe_guc *guc);void xe_guc_submit_wedge (struct xe_guc *guc);int xe_guc_read_stopped (struct xe_guc *guc);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 14/30 drm/xe/vf: Wakeup in GuC backend on VF post migration recovery
In reply to Matthew BrostIf VF post-migration recovery is in progress, the recovery flow will rebuild all GuC submission state. In this case, exit all waiters to ensure that submission queue scheduling can also be paused. Avoid taking any adverse actions after aborting the wait.
As part of waking up the GuC backend, suspend_wait can now return -EAGAIN indicating the waiter should be retried. If the caller is running on work item, that work item need to be requeued to avoid a deadlock for the work item blocking the VF migration recovery work item.
v3:
- Don't block in preempt fence work queue as this can interfere with VF post-migration work queue scheduling leading to deadlock (Testing)
- Use xe_gt_recovery_inprogress (Michal)
v5:
- Use static function for vf_recovery (Michal)
- Add helper to wake CT waiters (Michal)
- Move some code to following patch (Michal)
- Adjust commit message to explain suspend_wait returning -EAGAIN (Michal)
- Add kernel doc to suspend_wait around returning -EAGAIN
--- drivers/gpu/drm/xe/xe_exec_queue_types.h | 3 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 4 ++ drivers/gpu/drm/xe/xe_guc_ct.h | 9 +++ drivers/gpu/drm/xe/xe_guc_submit.c | 82 ++++++++++++++++++------ drivers/gpu/drm/xe/xe_preempt_fence.c | 11 ++++ 5 files changed, 88 insertions(+), 21 deletions(-)
drivers/gpu/drm/xe/xe_exec_queue_types.h +3 -0
@@ -207,6 +207,9 @@ struct xe_exec_queue_ops {* call after suspend. In dma-fencing path thus must return within a * reasonable amount of time. -ETIME return shall indicate an error * waiting for suspend resulting in associated VM getting killed. +* -EAGAIN return indicates the wait should be tried again, if the wait +* is within a work item, the work item should be requeued as deadlock +* avoidance mechanism. */ int (*suspend_wait)(struct xe_exec_queue *q);/**
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
@@ -23,6 +23,7 @@#include "xe_gt_sriov_vf.h" #include "xe_gt_sriov_vf_types.h" #include "xe_guc.h" +#include "xe_guc_ct.h" #include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_guc_submit.h"
@@ -743,6 +744,9 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); +smp_wmb ();/* Ensure above write visable before wake */ + +xe_guc_ct_wake_waiters (>->uc.guc.ct); started =queue_work (gt->ordered_wq, >->sriov.vf.migration.worker); xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
drivers/gpu/drm/xe/xe_guc_ct.h +9 -0
@@ -72,4 +72,13 @@ xe_guc_ct_send_block_no_fail(struct xe_guc_ct *ct, const u32 *action, u32 len)long xe_guc_ct_queue_proc_time_jiffies (struct xe_guc_ct *ct); +/** +* xe_guc_ct_wake_waiters() - GuC CT wake up waiters +* @guc: GuC CT object +*/ +static inline void xe_guc_ct_wake_waiters (struct xe_guc_ct *ct) +{ +wake_up_all (&ct->wq); +} +#endif
drivers/gpu/drm/xe/xe_guc_submit.c +61 -21
189 lines not on this page · raw message
drivers/gpu/drm/xe/xe_preempt_fence.c +11 -0
@@ -8,6 +8,8 @@#include <linux/slab.h> #include "xe_exec_queue.h" +#include "xe_gt_printk.h" +#include "xe_guc_exec_queue_types.h" #include "xe_vm.h" static void preempt_fence_work_func (struct work_struct *w)
@@ -22,6 +24,15 @@ static void preempt_fence_work_func(struct work_struct *w) }else if (!q->ops->reset_status (q)) {int err = q->ops->suspend_wait (q); +if (err == -EAGAIN ) { +xe_gt_dbg (q->gt,"PREEMPT FENCE RETRY guc_id=%d" , + q->guc->id); +queue_work (q->vm->xe->preempt_fence_wq, + &pfence->preempt_work); +dma_fence_end_signalling (cookie); +return ; + } +if (err)dma_fence_set_error (&pfence->base, err); }else {
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 15/30 drm/xe/vf: Avoid indefinite blocking in preempt rebind worker for VFs supporting migration
In reply to Matthew BrostBlocking in work queues on a hardware action that may never occur — especially when it depends on a software fixup also scheduled on the a work queue — is a recipe for deadlock. This situation arises with the preempt rebind worker and VF post-migration recovery. To prevent potential deadlocks, avoid indefinite blocking in the preempt rebind worker for VFs that support migration.
v4:
- Use dma_fence_wait_timeout (CI)
--- drivers/gpu/drm/xe/xe_vm.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_vm.c +25 -1
@@ -35,6 +35,7 @@#include "xe_pt.h" #include "xe_pxp.h" #include "xe_res_cursor.h" +#include "xe_sriov_vf.h" #include "xe_svm.h" #include "xe_sync.h" #include "xe_tile.h"
@@ -111,12 +112,22 @@ static int alloc_preempt_fences(struct xe_vm *vm, struct list_head *list,static int wait_for_existing_preempt_fences (struct xe_vm *vm) {struct xe_exec_queue *q; +bool vf_migration =IS_SRIOV_VF (vm->xe) && +xe_sriov_vf_migration_supported (vm->xe); +signed long wait_time = vf_migration ?HZ /5 :MAX_SCHEDULE_TIMEOUT ;xe_vm_assert_held (vm);list_for_each_entry (q, &vm->preempt.exec_queues, lr.link) {if (q->lr.pfence) { -long timeout =dma_fence_wait (q->lr.pfence, false); +long timeout; + + timeout =dma_fence_wait_timeout (q->lr.pfence, false, + wait_time); +if (!timeout) { +xe_assert (vm->xe, vf_migration); +return -EAGAIN ; + }/* Only -ETIME on fence indicates VM needs to be killed */ if (timeout <0 || q->lr.pfence->error == -ETIME )
@@ -541,6 +552,19 @@ static void preempt_rebind_work_func(struct work_struct *w) out_unlock_outer:if (err == -EAGAIN ) {trace_xe_vm_rebind_worker_retry (vm); + +/* +* We can't block in workers on a VF which supports migration +* given this can block the VF post-migration workers from +* getting scheduled. +*/ +if (IS_SRIOV_VF (vm->xe) && +xe_sriov_vf_migration_supported (vm->xe)) { +up_write (&vm->lock); +xe_vm_queue_rebind_worker (vm); +return ; + } + goto retry; }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 16/30 drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register
In reply to Matthew BrostThe only case where the GuC submission backend cannot reason 100% correctly is when a GuC context is registered during VF post-migration recovery. In this scenario, it's possible that the GuC context register H2G is processed, but the immediately following schedule-enable H2G gets lost.
A double register is harmless when using GUC_HXG_TYPE_EVENT, as GuC simply drops the duplicate H2G. To keep things simple, use GUC_HXG_TYPE_EVENT for all context registrations on VFs.
v5:
- Check for xe_sriov_vf_migration_supported (Tomasz)
--- drivers/gpu/drm/xe/xe_guc_ct.c | 33 +++++++++++++++++++++++++-------- 1 file changed, 25 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +25 -8
@@ -32,6 +32,7 @@#include "xe_guc_tlb_inval.h" #include "xe_map.h" #include "xe_pm.h" +#include "xe_sriov_vf.h" #include "xe_trace_guc.h" static void receive_g2h (struct xe_guc_ct *ct);
@@ -736,6 +737,26 @@ static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence)return seqno; } +#define MAKE_ACTION (type, __action) \ +({ \ + FIELD_PREP(GUC_HXG_MSG_0_TYPE, type) | \ + FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION | \ + GUC_HXG_EVENT_MSG_0_DATA0, __action); \ +}) + +static bool vf_action_can_safely_fail (struct xe_device *xe,u32 action) +{ +/* +* If we are VF resuming, we can't exactly track if a context +* registration has been completed in the GuC state machine, it is +* harmless to resend as it will just fail silently if +* GUC_HXG_TYPE_EVENT is used. +*/ +return IS_SRIOV_VF (xe) &&xe_sriov_vf_migration_supported (xe) && + (action ==XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC || + action ==XE_GUC_ACTION_REGISTER_CONTEXT ); +} +#define H2G_CT_HEADERS (GUC_CTB_HDR_LEN + 1)/* one DW CTB header and one DW HxG header */ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
@@ -807,18 +828,14 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,FIELD_PREP (GUC_CTB_MSG_0_NUM_DWORDS , len) |FIELD_PREP (GUC_CTB_MSG_0_FENCE , ct_fence_value);if (want_response) { - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_REQUEST , action[0 ]); + }else if (vf_action_can_safely_fail (xe, action[0 ])) { + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_EVENT , action[0 ]); }else {fast_req_track (ct, ct_fence_value,FIELD_GET (GUC_HXG_EVENT_MSG_0_ACTION , action[0 ])); - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_FAST_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_FAST_REQUEST , action[0 ]); }/* H2G header in cmd[1] replaces action[0] so: */
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 17/30 drm/xe/vf: Flush and stop CTs in VF post migration recovery
In reply to Matthew BrostFlushing CTs (i.e., progressing all pending G2H messages) gives VF post-migration recovery an accurate view of which H2G messages the GuC has processed, enabling the GuC submission state machine to correctly rebuild all state.
Also, stop all CT traffic, as the CT is not live during VF post-migration recovery.
v3:
- xe_guc_ct_flush_and_stop rename (Michal)
- Drop extra GuC CT WQ wake up (Michal)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 1 + drivers/gpu/drm/xe/xe_guc_ct.c | 10 ++++++++++ drivers/gpu/drm/xe/xe_guc_ct.h | 1 + 3 files changed, 12 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -1111,6 +1111,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt) gt->sriov.vf.migration.recovery_queued = false;spin_unlock_irq (>->sriov.vf.migration.lock); +xe_guc_ct_flush_and_stop (>->uc.guc.ct);xe_guc_submit_pause (>->uc.guc); }
drivers/gpu/drm/xe/xe_guc_ct.c +10 -0
@@ -574,6 +574,16 @@ void xe_guc_ct_disable(struct xe_guc_ct *ct)stop_g2h_handler (ct); } +/** +* xe_guc_ct_flush_and_stop - Flush and stop all processing of G2H / H2G +* @ct: the &xe_guc_ct +*/ +void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct) +{ +receive_g2h (ct); +xe_guc_ct_stop (ct); +} +/** * xe_guc_ct_stop - Set GuC to stopped state * @ct: the &xe_guc_ct
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
@@ -17,6 +17,7 @@ int xe_guc_ct_init_post_hwconfig(struct xe_guc_ct *ct);int xe_guc_ct_enable (struct xe_guc_ct *ct);void xe_guc_ct_disable (struct xe_guc_ct *ct);void xe_guc_ct_stop (struct xe_guc_ct *ct); +void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct);void xe_guc_ct_fast_path (struct xe_guc_ct *ct);struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture (struct xe_guc_ct *ct);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 18/30 drm/xe/vf: Reset TLB invalidations during VF post migration recovery
In reply to Matthew BrostTLB invalidations requests can be lost during VF post-migration recovery. Since the VF has migrated, these invalidations are no longer needed.
Reset the TLB invalidation frontend, which will signal all pending fences.
v3:
- Move TLB invalidation reset after pausing submission (Tomasz)
- Adjust commit message (Michal)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
@@ -35,6 +35,7 @@#include "xe_sriov.h" #include "xe_sriov_vf.h" #include "xe_tile_sriov_vf.h" +#include "xe_tlb_inval.h" #include "xe_uc_fw.h" #include "xe_wopcm.h"
@@ -1113,6 +1114,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)xe_guc_ct_flush_and_stop (>->uc.guc.ct);xe_guc_submit_pause (>->uc.guc); +xe_tlb_inval_reset (>->tlb_inval); }static size_t post_migration_scratch_size (struct xe_device *xe)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 19/30 drm/xe/vf: Kickstart after resfix in VF post migration recovery
In reply to Matthew BrostGuC needs to be live for the GuC submission state machine to resubmit anything lost during VF post-migration recovery. Therefore, move the kickstart step after resfix to ensure proper resubmission.
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +9 -8
@@ -1141,13 +1141,6 @@ static int vf_post_migration_fixups(struct xe_gt *gt)static void vf_post_migration_kickstart (struct xe_gt *gt) { -/* -* Make sure interrupts on the new HW are properly set. The GuC IRQ -* must be working at this point, since the recovery did started, -* but the rest was not enabled using the procedure from spec. -*/ -xe_irq_resume (gt_to_xe (gt)); -xe_guc_submit_unpause (>->uc.guc); }
@@ -1167,6 +1160,13 @@ static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)if (skip_resfix)return -EAGAIN ; +/* +* Make sure interrupts on the new HW are properly set. The GuC IRQ +* must be working at this point, since the recovery did started, +* but the rest was not enabled using the procedure from spec. +*/ +xe_irq_resume (gt_to_xe (gt)); +return vf_notify_resfix_done (gt); }
@@ -1190,11 +1190,12 @@ static void vf_post_migration_recovery(struct xe_gt *gt)if (err) goto fail; -vf_post_migration_kickstart (gt); err =vf_post_migration_notify_resfix_done (gt);if (err && err != -EAGAIN ) goto fail; +vf_post_migration_kickstart (gt); +xe_pm_runtime_put (xe);xe_gt_sriov_notice (gt,"migration recovery ended\n" );return ;
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 20/30 drm/xe/vf: Start CTs before resfix VF post migration recovery
In reply to Matthew BrostBefore RESFIX_DONE, all CTs stuck in the H2G queue need to be squashed, as they may contain actions which contain invalid GGTT references or are unnecessary after HW change.
Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.
v3:
- Don't mess with head / tail values (Michal)
v4:
- Don't mess with broke (Michal)
- Add CTB_H2G_BUFFER_OFFSET (Michal)
v5:
- Adjust commit message (Tomasz)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 +++ drivers/gpu/drm/xe/xe_guc_ct.c | 70 +++++++++++++++++++++-------- drivers/gpu/drm/xe/xe_guc_ct.h | 1 + 3 files changed, 60 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7 -0
@@ -1139,6 +1139,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)return 0 ; } +static void vf_post_migration_rearm (struct xe_gt *gt) +{ +xe_guc_ct_restart (>->uc.guc.ct); +} +static void vf_post_migration_kickstart (struct xe_gt *gt) {xe_guc_submit_unpause (>->uc.guc);
@@ -1190,6 +1195,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)if (err) goto fail; +vf_post_migration_rearm (gt); + err =vf_post_migration_notify_resfix_done (gt);if (err && err != -EAGAIN ) goto fail;
drivers/gpu/drm/xe/xe_guc_ct.c +52 -18
127 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);int xe_guc_ct_init (struct xe_guc_ct *ct);int xe_guc_ct_init_post_hwconfig (struct xe_guc_ct *ct);int xe_guc_ct_enable (struct xe_guc_ct *ct); +int xe_guc_ct_restart (struct xe_guc_ct *ct);void xe_guc_ct_disable (struct xe_guc_ct *ct);void xe_guc_ct_stop (struct xe_guc_ct *ct);void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 21/30 drm/xe/vf: Abort VF post migration recovery on failure
In reply to Matthew BrostIf VF post-migration recovery fails, the device is wedged. However, submission queues still need to be enabled for proper cleanup. In such cases, call into the GuC submission backend to restart all queues that were previously paused.
v3:
- s/Avort/Abort (Tomasz)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 10 ++++++++++ drivers/gpu/drm/xe/xe_guc_submit.c | 20 ++++++++++++++++++++ drivers/gpu/drm/xe/xe_guc_submit.h | 1 + 3 files changed, 31 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
@@ -1149,6 +1149,15 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)xe_guc_submit_unpause (>->uc.guc); } +static void vf_post_migration_abort (struct xe_gt *gt) +{ +spin_lock_irq (>->sriov.vf.migration.lock); +WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, false); +spin_unlock_irq (>->sriov.vf.migration.lock); + +xe_guc_submit_pause_abort (>->uc.guc); +} +static int vf_post_migration_notify_resfix_done (struct xe_gt *gt) {bool skip_resfix = false;
@@ -1207,6 +1216,7 @@ static void vf_post_migration_recovery(struct xe_gt *gt)xe_gt_sriov_notice (gt,"migration recovery ended\n" );return ; fail: +vf_post_migration_abort (gt);xe_pm_runtime_put (xe);xe_gt_sriov_err (gt,"migration recovery failed (%pe)\n" ,ERR_PTR (err));xe_device_declare_wedged (xe);
drivers/gpu/drm/xe/xe_guc_submit.c +20 -0
@@ -2087,6 +2087,26 @@ void xe_guc_submit_unpause(struct xe_guc *guc)wake_up_all (&guc->ct.wq); } +/** +* xe_guc_submit_abort - Abort all paused submission task on given GuC. +* @guc: the &xe_guc struct instance whose scheduler is to be aborted +*/ +void xe_guc_submit_pause_abort (struct xe_guc *guc) +{ +struct xe_exec_queue *q; +unsigned long index; + +mutex_lock (&guc->submission_state.lock); +xa_for_each (&guc->submission_state.exec_queue_lookup, index, q) { +struct xe_gpu_scheduler *sched = &q->guc->sched; + +xe_sched_submission_start (sched); +if (exec_queue_killed_or_banned_or_wedged (q)) +xe_guc_exec_queue_trigger_cleanup (q); + } +mutex_unlock (&guc->submission_state.lock); +} +static struct xe_exec_queue *g2h_exec_queue_lookup (struct xe_guc *guc,u32 guc_id) {
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);int xe_guc_submit_start (struct xe_guc *guc);void xe_guc_submit_pause (struct xe_guc *guc);void xe_guc_submit_unpause (struct xe_guc *guc); +void xe_guc_submit_pause_abort (struct xe_guc *guc);void xe_guc_submit_wedge (struct xe_guc *guc);int xe_guc_read_stopped (struct xe_guc *guc);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 22/30 drm/xe/vf: Replay GuC submission state on pause / unpause
In reply to Matthew BrostFixup GuC submission pause / unpause functions to properly replay any possible state lost during VF post migration recovery.
v3:
- Add helpers for revert / replay (Tomasz)
- Add comment around WQ NOPs (Tomasz)
--- drivers/gpu/drm/xe/xe_gpu_scheduler.c | 14 ++ drivers/gpu/drm/xe/xe_gpu_scheduler.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 1 + drivers/gpu/drm/xe/xe_guc_exec_queue_types.h | 15 ++ drivers/gpu/drm/xe/xe_guc_submit.c | 242 +++++++++++++++++-- drivers/gpu/drm/xe/xe_guc_submit.h | 1 + drivers/gpu/drm/xe/xe_sched_job_types.h | 4 + 7 files changed, 264 insertions(+), 15 deletions(-)
drivers/gpu/drm/xe/xe_gpu_scheduler.c +14 -0
@@ -135,3 +135,17 @@ void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched,list_add_tail (&msg->link, &sched->msgs);xe_sched_process_msg_queue (sched); } + +/** +* xe_sched_add_msg_head() - Xe GPU scheduler add message to head of list +* @sched: Xe GPU scheduler +* @msg: Message to add +*/ +void xe_sched_add_msg_head (struct xe_gpu_scheduler *sched, +struct xe_sched_msg *msg) +{ +lockdep_assert_held (&sched->base.job_list_lock); + +list_add (&msg->link, &sched->msgs); +xe_sched_process_msg_queue (sched); +}
drivers/gpu/drm/xe/xe_gpu_scheduler.h +2 -0
@@ -29,6 +29,8 @@ void xe_sched_add_msg(struct xe_gpu_scheduler *sched, struct xe_sched_msg *msg);void xe_sched_add_msg_locked (struct xe_gpu_scheduler *sched,struct xe_sched_msg *msg); +void xe_sched_add_msg_head (struct xe_gpu_scheduler *sched, +struct xe_sched_msg *msg);static inline void xe_sched_msg_lock (struct xe_gpu_scheduler *sched) {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -1142,6 +1142,7 @@ static int vf_post_migration_fixups(struct xe_gt *gt)static void vf_post_migration_rearm (struct xe_gt *gt) {xe_guc_ct_restart (>->uc.guc.ct); +xe_guc_submit_unpause_prepare (>->uc.guc); }static void vf_post_migration_kickstart (struct xe_gt *gt)
drivers/gpu/drm/xe/xe_guc_exec_queue_types.h +15 -0
@@ -51,6 +51,21 @@ struct xe_guc_exec_queue {wait_queue_head_t suspend_wait;/** @suspend_pending: a suspend of the exec_queue is pending */ bool suspend_pending; +/** +* @needs_cleanup: Needs a cleanup message during VF post migration +* recovery. +*/ +bool needs_cleanup; +/** +* @needs_suspend: Needs a suspend message during VF post migration +* recovery. +*/ +bool needs_suspend; +/** +* @needs_resume: Needs a resume message during VF post migration +* recovery. +*/ + bool needs_resume; };#endif
drivers/gpu/drm/xe/xe_guc_submit.c +227 -15
325 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);int xe_guc_submit_start (struct xe_guc *guc);void xe_guc_submit_pause (struct xe_guc *guc);void xe_guc_submit_unpause (struct xe_guc *guc); +void xe_guc_submit_unpause_prepare (struct xe_guc *guc);void xe_guc_submit_pause_abort (struct xe_guc *guc);void xe_guc_submit_wedge (struct xe_guc *guc);
drivers/gpu/drm/xe/xe_sched_job_types.h +4 -0
@@ -63,6 +63,10 @@ struct xe_sched_job {bool ring_ops_flush_tlb;/** @ggtt: mapped in ggtt. */ bool ggtt; +/** @skip_emit: skip emitting the job */ +bool skip_emit; +/** @last_replay: last job being replayed */ +bool last_replay;/** @ptrs: per instance pointers. */ struct xe_job_ptrs ptrs[]; };
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 23/30 drm/xe: Move queue init before LRC creation
In reply to Matthew BrostA queue must be in the submission backend's tracking state before the LRC is created to avoid a race condition where the LRC's GGTT addresses are not properly fixed up during VF post-migration recovery.
Move the queue initialization—which adds the queue to the submission backend's tracking state—before LRC creation.
v2:
- Wait on VF GGTT fixes before creating LRC (testing)
v5:
- Adjust comment in code (Tomasz)
- Reduce race window
--- drivers/gpu/drm/xe/xe_exec_queue.c | 45 ++++++++++++++++++----- drivers/gpu/drm/xe/xe_execlist.c | 2 +- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 39 +++++++++++++++++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 5 +++ drivers/gpu/drm/xe/xe_guc_submit.c | 2 +- drivers/gpu/drm/xe/xe_lrc.h | 10 +++++ 7 files changed, 92 insertions(+), 13 deletions(-)
drivers/gpu/drm/xe/xe_exec_queue.c +35 -10
@@ -15,6 +15,7 @@#include "xe_dep_scheduler.h" #include "xe_device.h" #include "xe_gt.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine_class_sysfs.h" #include "xe_hw_engine_group.h" #include "xe_hw_fence.h"
@@ -202,17 +203,34 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q) flags |=XE_LRC_CREATE_RUNALONE ; } + err = q->ops->init (q); +if (err) +return err; + +/* +* This must occur after q->ops->init to avoid race conditions during VF +* post-migration recovery, as the fixups for the LRC GGTT addresses +* depend on the queue being present in the backend tracking structure. +* +* In addition to above, we must wait on inflight GGTT changes to avoid +* writing out stale values here. Such wait provides a solid solution +* (without a race) only if the function can detect migration instantly +* from the moment vCPU resumes execution. +*/ for (i =0 ; i < q->width; ++i) { - q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K , q->msix_vec, flags); -if (IS_ERR (q->lrc[i])) { - err =PTR_ERR (q->lrc[i]); +struct xe_lrc *lrc; + +xe_gt_sriov_vf_wait_valid_ggtt (q->gt); + lrc =xe_lrc_create (q->hwe, q->vm,xe_lrc_ring_size (), + q->msix_vec, flags); +if (IS_ERR (lrc)) { + err =PTR_ERR (lrc); goto err_lrc; } - } - err = q->ops->init (q); -if (err) - goto err_lrc; +/* Pairs with READ_ONCE to xe_exec_queue_contexts_hwsp_rebase */ +WRITE_ONCE (q->lrc[i], lrc); + }return 0 ;
@@ -1118,9 +1136,16 @@ int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch)int err =0 ;for (i =0 ; i < q->width; ++i) { -xe_lrc_update_memirq_regs_with_address (q->lrc[i], q->hwe, scratch); -xe_lrc_update_hwctx_regs_with_address (q->lrc[i]); - err =xe_lrc_setup_wa_bb_with_scratch (q->lrc[i], q->hwe, scratch); +struct xe_lrc *lrc; + +/* Pairs with WRITE_ONCE in __xe_exec_queue_init */ + lrc =READ_ONCE (q->lrc[i]); +if (!lrc) +continue ; + +xe_lrc_update_memirq_regs_with_address (lrc, q->hwe, scratch); +xe_lrc_update_hwctx_regs_with_address (lrc); + err =xe_lrc_setup_wa_bb_with_scratch (lrc, q->hwe, scratch);if (err)break ; }
drivers/gpu/drm/xe/xe_execlist.c +1 -1
@@ -339,7 +339,7 @@ static int execlist_exec_queue_init(struct xe_exec_queue *q)const struct drm_sched_init_args args = { .ops = &drm_sched_ops, .num_rqs =1 , - .credit_limit = q->lrc[0 ]->ring.size /MAX_JOB_SIZE_BYTES , + .credit_limit =xe_lrc_ring_size () /MAX_JOB_SIZE_BYTES , .hang_limit =XE_SCHED_HANG_LIMIT , .timeout =XE_SCHED_JOB_TIMEOUT , .name = q->hwe->name,
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +38 -1
@@ -487,6 +487,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery) shift, config->ggtt_base);xe_tile_sriov_vf_fixup_ggtt_nodes (gt_to_tile (gt), shift); } + +WRITE_ONCE (gt->sriov.vf.migration.ggtt_need_fixes, false); +smp_wmb ();/* Ensure above write visible before wake */ +wake_up_all (>->sriov.vf.migration.wq); + out:if (recovery)mutex_unlock (&ggtt->lock);
@@ -745,7 +750,8 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); -smp_wmb ();/* Ensure above write visable before wake */ +WRITE_ONCE (gt->sriov.vf.migration.ggtt_need_fixes, true); +smp_wmb ();/* Ensure above writes visable before wake */ xe_guc_ct_wake_waiters (>->uc.guc.ct);
@@ -1264,6 +1270,7 @@ int xe_gt_sriov_vf_init_early(struct xe_gt *gt) gt->sriov.vf.migration.scratch = buf;spin_lock_init (>->sriov.vf.migration.lock);INIT_WORK (>->sriov.vf.migration.worker, migration_worker_func); +init_waitqueue_head (>->sriov.vf.migration.wq);return 0 ; }
@@ -1312,3 +1319,33 @@ bool xe_gt_sriov_vf_recovery_pending(struct xe_gt *gt)return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); } + +static bool vf_valid_ggtt (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +if (xe_memirq_guc_sw_int_0_irq_pending (memirq, >->uc.guc) || +READ_ONCE (gt->sriov.vf.migration.ggtt_need_fixes)) +return false; + +return true; +} + +/** +* xe_gt_sriov_vf_wait_valid_ggtt() - VF wait for valid GGTT addresses +* @gt: the &xe_gt +*/ +void xe_gt_sriov_vf_wait_valid_ggtt (struct xe_gt *gt) +{ +int ret; + +if (!IS_SRIOV_VF (gt_to_xe (gt))) +return ; + + ret =wait_event_interruptible_timeout (gt->sriov.vf.migration.wq, +vf_valid_ggtt (gt), +HZ *5 ); +XE_WARN_ON (!ret); +}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
@@ -38,4 +38,6 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p);void xe_gt_sriov_vf_print_runtime (struct xe_gt *gt,struct drm_printer *p);void xe_gt_sriov_vf_print_version (struct xe_gt *gt,struct drm_printer *p); +void xe_gt_sriov_vf_wait_valid_ggtt (struct xe_gt *gt); +#endif
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +5 -0
@@ -8,6 +8,7 @@#include <linux/rwsem.h> #include <linux/types.h> +#include <linux/wait.h> #include <linux/workqueue.h> #include "xe_uc_fw_types.h"
@@ -50,6 +51,8 @@ struct xe_gt_sriov_vf_migration {struct work_struct worker;/** @lock: Protects recovery_queued, teardown */ spinlock_t lock; +/** @wq: wait queue for migration fixes */ +wait_queue_head_t wq;/** @scratch: Scratch memory for VF recovery */ void *scratch;/** @recovery_teardown: VF post migration recovery is being torn down */
@@ -58,6 +61,8 @@ struct xe_gt_sriov_vf_migration {bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */ bool recovery_inprogress; +/** @ggtt_need_fixes: VF GGTT needs fixes */ +bool ggtt_need_fixes; };/**
drivers/gpu/drm/xe/xe_guc_submit.c +1 -1
@@ -1663,7 +1663,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) timeout = (q->vm &&xe_vm_in_lr_mode (q->vm)) ?MAX_SCHEDULE_TIMEOUT :msecs_to_jiffies (q->sched_props.job_timeout_ms); err =xe_sched_init (&ge->sched, &drm_sched_ops, &xe_sched_ops, -NULL , q->lrc[0 ]->ring.size /MAX_JOB_SIZE_BYTES ,64 , +NULL ,xe_lrc_ring_size () /MAX_JOB_SIZE_BYTES ,64 , timeout,guc_to_gt (guc)->ordered_wq,NULL , q->name,gt_to_xe (q->gt)->drm.dev);if (err)
drivers/gpu/drm/xe/xe_lrc.h +10 -0
@@ -74,6 +74,16 @@ static inline void xe_lrc_put(struct xe_lrc *lrc)kref_put (&lrc->refcount, xe_lrc_destroy); } +/** +* xe_lrc_ring_size() - Xe LRC ring size +* +* Return: Size of LRC size +*/ +static inline size_t xe_lrc_ring_size (void ) +{ +return SZ_16K ; +} +size_t xe_gt_lrc_size (struct xe_gt *gt,enum xe_engine_class class);u32 xe_lrc_pphwsp_offset (struct xe_lrc *lrc);u32 xe_lrc_regs_offset (struct xe_lrc *lrc);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 24/30 drm/xe/vf: Add debug prints for GuC replaying state during VF recovery
In reply to Matthew BrostHelpful to manually verify the GuC state machine can correctly replay the state during a VF post-migration recovery. All replay paths have been manually verified as triggered and working during testing.
--- drivers/gpu/drm/xe/xe_guc_submit.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_guc_submit.c +20 -3
@@ -2026,21 +2026,27 @@ void xe_guc_submit_stop(struct xe_guc *guc) } -static void guc_exec_queue_revert_pending_state_change (struct xe_exec_queue *q) +static void guc_exec_queue_revert_pending_state_change (struct xe_guc *guc, +struct xe_exec_queue *q) {bool pending_enable, pending_disable, pending_resume; pending_enable =exec_queue_pending_enable (q); pending_resume =exec_queue_pending_resume (q); -if (pending_enable && pending_resume) +if (pending_enable && pending_resume) { q->guc->needs_resume = true; +xe_gt_dbg (guc_to_gt (guc),"Replay RESUME - guc_id=%d" , + q->guc->id); + }if (pending_enable && !pending_resume && !exec_queue_pending_tdr_exit (q)) {clear_exec_queue_registered (q);if (xe_exec_queue_is_lr (q))xe_exec_queue_put (q); +xe_gt_dbg (guc_to_gt (guc),"Replay REGISTER - guc_id=%d" , + q->guc->id); }if (pending_enable) {
@@ -2048,6 +2054,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)clear_exec_queue_pending_resume (q);clear_exec_queue_pending_tdr_exit (q);clear_exec_queue_pending_enable (q); +xe_gt_dbg (guc_to_gt (guc),"Replay ENABLE - guc_id=%d" , + q->guc->id); }if (exec_queue_destroyed (q) &&exec_queue_registered (q)) {
@@ -2057,6 +2065,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)else q->guc->needs_cleanup = true;clear_exec_queue_extra_ref (q); +xe_gt_dbg (guc_to_gt (guc),"Replay CLEANUP - guc_id=%d" , + q->guc->id); } pending_disable =exec_queue_pending_disable (q);
@@ -2064,6 +2074,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)if (pending_disable &&exec_queue_suspended (q)) {clear_exec_queue_suspended (q); q->guc->needs_suspend = true; +xe_gt_dbg (guc_to_gt (guc),"Replay SUSPEND - guc_id=%d" , + q->guc->id); }if (pending_disable) {
@@ -2071,6 +2083,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)set_exec_queue_enabled (q);clear_exec_queue_pending_disable (q);clear_exec_queue_check_timeout (q); +xe_gt_dbg (guc_to_gt (guc),"Replay DISABLE - guc_id=%d" , + q->guc->id); } q->guc->resume_time =0 ;
@@ -2096,7 +2110,7 @@ static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)else cancel_delayed_work_sync (&sched->base.work_tdr); -guc_exec_queue_revert_pending_state_change (q); +guc_exec_queue_revert_pending_state_change (guc, q);if (xe_exec_queue_is_parallel (q)) {struct xe_device *xe =guc_to_xe (guc);
@@ -2206,6 +2220,9 @@ static void guc_exec_queue_unpause_prepare(struct xe_guc *guc,list_for_each_entry (s_job, &sched->base.pending_list, list) { job =to_xe_sched_job (s_job); +xe_gt_dbg (guc_to_gt (guc),"Replay JOB - guc_id=%d, seqno=%d" , + q->guc->id,xe_sched_job_seqno (job)); + q->ring_ops->emit_job (job); job->skip_emit = true; }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 25/30 drm/xe/vf: Workaround for race condition in GuC firmware during VF pause
In reply to Matthew BrostA race condition exists where a paused VF's H2G request can be processed and subsequently rejected. This rejection results in a FAST_REQ failure being delivered to the KMD, which then terminates the CT via a dead worker and triggers a GT reset—an undesirable outcome.
This workaround mitigates the issue by checking if a VF post-migration recovery is in progress and aborting these adverse actions accordingly. The GuC firmware will address this bug in an upcoming release. Once that version is available and VF migration depends on it, this workaround can be safely removed.
--- drivers/gpu/drm/xe/xe_guc_ct.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/gpu/drm/xe/xe_guc_ct.c +4 -0
@@ -1395,6 +1395,10 @@ static int parse_g2h_response(struct xe_guc_ct *ct, u32 *msg, u32 len)fast_req_report (ct, fence); +/* FIXME: W/A race in the GuC, will get in firmware soon */ +if (xe_gt_recovery_pending (gt)) +return 0 ; +CT_DEAD (ct,NULL ,PARSE_G2H_RESPONSE );return -EPROTO ;
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 26/30 drm/xe: Use PPGTT addresses for TLB invalidation to avoid GGTT fixups
In reply to Matthew BrostFrom: Satyanarayana K V P <[email protected]>
The migrate VM builds the CCS metadata save/restore batch buffer (BB) in advance and retains it so the GuC can submit it directly when saving a VM’s state.
When a VM migrates between VFs, the GGTT base can change. Any GGTT-based addresses embedded in the BB would then have to be parsed and patched.
Use PPGTT addresses in the BB (including for TLB invalidation) so the BB remains GGTT-agnostic and requires no address fixups during migration.
--- drivers/gpu/drm/xe/xe_migrate.c | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_migrate.c +20 -8
@@ -980,15 +980,27 @@ struct xe_lrc *xe_migrate_lrc(struct xe_migrate *migrate)return migrate->q->lrc[0 ]; } -static int emit_flush_invalidate (struct xe_exec_queue *q,u32 *dw,int i, -u32 flags) +static u64 migrate_vm_ppgtt_addr_tlb_inval (void ) { -struct xe_lrc *lrc =xe_exec_queue_lrc (q); +/* +* The migrate VM is self-referential so it can modify its own PTEs (see +* pte_update_size() or emit_pte() functions). We reserve NUM_KERNEL_PDE +* entries for kernel operations (copies, clears, CCS migrate), and +* suballocate the rest to user operations (binds/unbinds). With +* NUM_KERNEL_PDE = 15, NUM_KERNEL_PDE - 1 is already used for PTE updates, +* so assign NUM_KERNEL_PDE - 2 for TLB invalidation. +*/ +return (NUM_KERNEL_PDE -2 ) *XE_PAGE_SIZE ; +} + +static int emit_flush_invalidate (u32 *dw,int i,u32 flags) +{ +u64 addr =migrate_vm_ppgtt_addr_tlb_inval (); + dw[i++] =MI_FLUSH_DW |MI_INVALIDATE_TLB |MI_FLUSH_DW_OP_STOREDW |MI_FLUSH_IMM_DW | flags; - dw[i++] =lower_32_bits (xe_lrc_start_seqno_ggtt_addr (lrc)) | -MI_FLUSH_DW_USE_GTT ; - dw[i++] =upper_32_bits (xe_lrc_start_seqno_ggtt_addr (lrc)); + dw[i++] =lower_32_bits (addr); + dw[i++] =upper_32_bits (addr); dw[i++] =MI_NOOP ; dw[i++] =MI_NOOP ;
@@ -1101,11 +1113,11 @@ int xe_migrate_ccs_rw_copy(struct xe_tile *tile, struct xe_exec_queue *q,emit_pte (m, bb, ccs_pt, false, false, &ccs_it, ccs_size, src); - bb->len =emit_flush_invalidate (q, bb->cs, bb->len, flush_flags); + bb->len =emit_flush_invalidate (bb->cs, bb->len, flush_flags); flush_flags =xe_migrate_ccs_copy (m, bb, src_L0_ofs, src_is_pltt, src_L0_ofs, dst_is_pltt, src_L0, ccs_ofs, true); - bb->len =emit_flush_invalidate (q, bb->cs, bb->len, flush_flags); + bb->len =emit_flush_invalidate (bb->cs, bb->len, flush_flags); size -= src_L0; }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 27/30 drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VF
In reply to Matthew BrostVF CCS restore is a primary GT operation on which the media GT depends. Therefore, it doesn't make much sense to run these operations in parallel. To address this, point the media GT's ordered work queue to the primary GT's ordered work queue on platforms that require (PTL VFs) CCS restore as part of VF post-migration recovery.
--- drivers/gpu/drm/xe/xe_device_types.h | 2 ++ drivers/gpu/drm/xe/xe_gt.c | 7 +++++-- drivers/gpu/drm/xe/xe_gt.h | 2 +- drivers/gpu/drm/xe/xe_pci.c | 6 +++++- drivers/gpu/drm/xe/xe_pci_types.h | 1 + drivers/gpu/drm/xe/xe_tile.c | 2 +- 6 files changed, 15 insertions(+), 5 deletions(-)
drivers/gpu/drm/xe/xe_device_types.h +2 -0
@@ -326,6 +326,8 @@ struct xe_device { u8 skip_mtcfg:1;/** @info.skip_pcode: skip access to PCODE uC */ u8 skip_pcode:1; +/** @info.needs_shared_vf_gt_wq: needs shared GT WQ on VF */ + u8 needs_shared_vf_gt_wq:1; } info;/** @wa_active: keep track of active workarounds */
drivers/gpu/drm/xe/xe_gt.c +5 -2
@@ -65,7 +65,7 @@#include "xe_wa.h" #include "xe_wopcm.h" -struct xe_gt *xe_gt_alloc (struct xe_tile *tile) +struct xe_gt *xe_gt_alloc (struct xe_tile *tile,bool use_primary_wq) {struct drm_device *drm = &tile_to_xe (tile)->drm;struct xe_gt *gt;
@@ -75,7 +75,10 @@ struct xe_gt *xe_gt_alloc(struct xe_tile *tile)return ERR_PTR (-ENOMEM ); gt->tile = tile; - gt->ordered_wq =drmm_alloc_ordered_workqueue (drm,"gt-ordered-wq" ,WQ_MEM_RECLAIM ); +if (use_primary_wq) + gt->ordered_wq = tile->primary_gt->ordered_wq; +else + gt->ordered_wq =drmm_alloc_ordered_workqueue (drm,"gt-ordered-wq" ,WQ_MEM_RECLAIM );if (IS_ERR (gt->ordered_wq))return ERR_CAST (gt->ordered_wq);
drivers/gpu/drm/xe/xe_gt.h +1 -1
@@ -28,7 +28,7 @@ static inline bool xe_fault_inject_gt_reset(void)return IS_ENABLED (CONFIG_DEBUG_FS ) &&should_fail (>_reset_failure,1 ); } -struct xe_gt *xe_gt_alloc (struct xe_tile *tile); +struct xe_gt *xe_gt_alloc (struct xe_tile *tile,bool use_primary_wq);int xe_gt_init_early (struct xe_gt *gt);int xe_gt_init (struct xe_gt *gt);void xe_gt_mmio_init (struct xe_gt *gt);
drivers/gpu/drm/xe/xe_pci.c +5 -1
@@ -347,6 +347,7 @@ static const struct xe_device_desc ptl_desc = { .has_sriov = true, .max_gt_per_tile = 2, .needs_scratch = true, + .needs_shared_vf_gt_wq = true, };#undef PLATFORM
@@ -598,6 +599,7 @@ static int xe_info_init_early(struct xe_device *xe, xe->info.skip_mtcfg = desc->skip_mtcfg; xe->info.skip_pcode = desc->skip_pcode; xe->info.needs_scratch = desc->needs_scratch; + xe->info.needs_shared_vf_gt_wq = desc->needs_shared_vf_gt_wq; xe->info.probe_display =IS_ENABLED (CONFIG_DRM_XE_DISPLAY ) && xe_modparam.probe_display &&
@@ -766,7 +768,9 @@ static int xe_info_init(struct xe_device *xe,* Allocate and setup media GT for platforms with standalone * media. */ - tile->media_gt =xe_gt_alloc (tile); + tile->media_gt =xe_gt_alloc (tile, + xe->info.needs_shared_vf_gt_wq && +IS_SRIOV_VF (xe));if (IS_ERR (tile->media_gt))return PTR_ERR (tile->media_gt);
drivers/gpu/drm/xe/xe_pci_types.h +1 -0
@@ -48,6 +48,7 @@ struct xe_device_desc { u8 skip_guc_pc:1; u8 skip_mtcfg:1; u8 skip_pcode:1; + u8 needs_shared_vf_gt_wq:1; };struct xe_graphics_desc {
drivers/gpu/drm/xe/xe_tile.c +1 -1
@@ -149,7 +149,7 @@ int xe_tile_init_early(struct xe_tile *tile, struct xe_device *xe, u8 id)if (err)return err; - tile->primary_gt =xe_gt_alloc (tile); + tile->primary_gt =xe_gt_alloc (tile, false);if (IS_ERR (tile->primary_gt))return PTR_ERR (tile->primary_gt);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 28/30 drm/xe/vf: Ensure media GT VF recovery runs after primary GT on PTL
In reply to Matthew BrostIt is possible that the media GT's VF post-migration recovery work item gets scheduled before the primary GT's work item. Since the media GT depends on the primary GT's work item to complete CCS restore, if the media GT's work item is scheduled first, detect this condition and re-queue the media GT's work item for a later time.
v5:
- Adjust debug message (Tomasz)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +27 -2
@@ -1112,8 +1112,22 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) pf_version->major, pf_version->minor); } -static void vf_post_migration_shutdown (struct xe_gt *gt) +static bool vf_post_migration_shutdown (struct xe_gt *gt) { +struct xe_device *xe =gt_to_xe (gt); + +/* +* On platforms where CCS must be restored by the primary GT, the media +* GT's VF post-migration recovery must run afterward. Detect this case +* and re-queue the media GT's restore work item if necessary. +*/ +if (xe->info.needs_shared_vf_gt_wq &&xe_gt_is_media_type (gt)) { +struct xe_gt *primary_gt =gt_to_tile (gt)->primary_gt; + +if (xe_gt_sriov_vf_recovery_pending (primary_gt)) +return true; + } +spin_lock_irq (>->sriov.vf.migration.lock); gt->sriov.vf.migration.recovery_queued = false;spin_unlock_irq (>->sriov.vf.migration.lock);
@@ -1121,6 +1135,8 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)xe_guc_ct_flush_and_stop (>->uc.guc.ct);xe_guc_submit_pause (>->uc.guc);xe_tlb_inval_reset (>->tlb_inval); + +return false; }static size_t post_migration_scratch_size (struct xe_device *xe)
@@ -1195,11 +1211,14 @@ static void vf_post_migration_recovery(struct xe_gt *gt) {struct xe_device *xe =gt_to_xe (gt);int err; +bool retry;xe_gt_sriov_dbg (gt,"migration recovery in progress\n" );xe_pm_runtime_get (xe); -vf_post_migration_shutdown (gt); + retry =vf_post_migration_shutdown (gt); +if (retry) + goto queue;if (!xe_sriov_vf_migration_supported (xe)) {xe_gt_sriov_err (gt,"migration is not supported\n" );
@@ -1227,6 +1246,12 @@ static void vf_post_migration_recovery(struct xe_gt *gt)xe_pm_runtime_put (xe);xe_gt_sriov_err (gt,"migration recovery failed (%pe)\n" ,ERR_PTR (err));xe_device_declare_wedged (xe); +return ; + +queue: +xe_gt_sriov_info (gt,"Re-queuing migration recovery\n" ); +queue_work (gt->ordered_wq, >->sriov.vf.migration.worker); +xe_pm_runtime_put (xe); }static void migration_worker_func (struct work_struct *w)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 29/30 drm/xe/vf: Rebase CCS save/restore BB GGTT addresses
In reply to Matthew BrostRebase the CCS save/restore BB's GGTT addresses during VF post-migration recovery by setting the software ring tail to zero, the LRC ring head to zero, and rewriting the jump-to-BB instructions.
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 4 ++++ drivers/gpu/drm/xe/xe_sriov_vf_ccs.c | 28 ++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_sriov_vf_ccs.h | 1 + 3 files changed, 33 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
@@ -34,6 +34,7 @@#include "xe_pm.h" #include "xe_sriov.h" #include "xe_sriov_vf.h" +#include "xe_sriov_vf_ccs.h" #include "xe_tile_sriov_vf.h" #include "xe_tlb_inval.h" #include "xe_uc_fw.h"
@@ -1153,6 +1154,9 @@ static int vf_post_migration_fixups(struct xe_gt *gt)if (err)return err; +if (xe_gt_is_main_type (gt)) +xe_sriov_vf_ccs_rebase (gt_to_xe (gt)); +xe_gt_sriov_vf_default_lrcs_hwsp_rebase (gt); err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf);if (err)
drivers/gpu/drm/xe/xe_sriov_vf_ccs.c +28 -0
@@ -175,6 +175,15 @@ static void ccs_rw_update_ring(struct xe_sriov_vf_ccs_ctx *ctx)struct xe_lrc *lrc =xe_exec_queue_lrc (ctx->mig_q);u32 dw[10 ], i =0 ; +/* +* XXX: Save/restore fixes — for some reason, the GuC only accepts the +* save/restore context if the LRC head pointer is zero. This is evident +* from repeated VF migrations failing when the LRC head pointer is +* non-zero. +*/ + lrc->ring.tail =0 ; +xe_lrc_set_ring_head (lrc,0 ); + dw[i++] =MI_ARB_ON_OFF |MI_ARB_ENABLE ; dw[i++] =MI_BATCH_BUFFER_START |XE_INSTR_NUM_DW (3 ); dw[i++] =lower_32_bits (addr);
@@ -186,6 +195,25 @@ static void ccs_rw_update_ring(struct xe_sriov_vf_ccs_ctx *ctx)xe_lrc_set_ring_tail (lrc, lrc->ring.tail); } +/** +* xe_sriov_vf_ccs_rebase - Rebase GGTT addresses for CCS save / restore +* @xe: the &xe_device. +*/ +void xe_sriov_vf_ccs_rebase (struct xe_device *xe) +{ +enum xe_sriov_vf_ccs_rw_ctxs ctx_id; + +if (!IS_VF_CCS_READY (xe)) +return ; + +for_each_ccs_rw_ctx (ctx_id) { +struct xe_sriov_vf_ccs_ctx *ctx = + &xe->sriov.vf.ccs.contexts[ctx_id]; + +ccs_rw_update_ring (ctx); + } +} +static int register_save_restore_context (struct xe_sriov_vf_ccs_ctx *ctx) {int ctx_type;
drivers/gpu/drm/xe/xe_sriov_vf_ccs.h +1 -0
@@ -18,6 +18,7 @@ int xe_sriov_vf_ccs_init(struct xe_device *xe);int xe_sriov_vf_ccs_attach_bo (struct xe_bo *bo);int xe_sriov_vf_ccs_detach_bo (struct xe_bo *bo);int xe_sriov_vf_ccs_register_context (struct xe_device *xe); +void xe_sriov_vf_ccs_rebase (struct xe_device *xe);void xe_sriov_vf_ccs_print (struct xe_device *xe,struct drm_printer *p);static inline bool xe_sriov_vf_ccs_ready (struct xe_device *xe)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v5 30/30 drm/xe/guc: Increase wait timeout to 2sec after BUSY reply from GuC
In reply to Matthew BrostFrom: Satyanarayana K V P <[email protected]>
Some VF2GUC actions may take longer to process. Increase default timeout after received BUSY indication to 2sec to cover all worst case scenarios.
--- drivers/gpu/drm/xe/xe_guc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_guc.c +1 -1
@@ -1439,7 +1439,7 @@ int xe_guc_mmio_send_recv(struct xe_guc *guc, const u32 *request,BUILD_BUG_ON ((GUC_HXG_TYPE_RESPONSE_SUCCESS ^GUC_HXG_TYPE_RESPONSE_FAILURE ) !=1 ); ret =xe_mmio_wait32 (mmio, reply_reg, resp_mask, resp_mask, -1000000 , &header, false); +2000000 , &header, false);if (unlikely (FIELD_GET (GUC_HXG_MSG_0_ORIGIN , header) !=GUC_HXG_ORIGIN_GUC ))
Signature · 1 line
2.34.1