VF migration redesign
Subsystems (4)
Matthew Brost
maintainerv4 cover VF migration redesign
Rather than modifying buffers in place using GGTT addresses during VF migration, this approach relies on the submission backend's stop/start mechanism to issue fixups. The patch titled "Document GuC Submission Backend" provides a detailed explanation of the design.
Testing was performed using an out-of-tree PF/VFIO driver with manual triggering of VF migration while IGT test cases are running.
IGT test cases:
- A new series [1] that exercises active contexts, job resubmission, and compressd memory.
- A new test [2] that actively creates / destroys queue on each submission
- xe_exec_threads basic sections, which test context registration loss, schedule enable loss, and job resubmission.
- xe_exec_threads balancer sections, which follow the same flows as the basic sections but include a work queue (GGTT address shift).
- xe_exec_threads compute mode user pointer invalidation sections, which exercise the same flow as the basic sections, plus replaying suspend/resume flows.
All code paths in "Replay GuC submission state on pause/unpause" that replay state have been manually verified via debug messages "Add debug prints for GuC replaying state during VF recovery".
v2:
- Fix lockdep splat
- Fix checkpatch
- Fix PTL issue with LRC W/A buffer
- Fix race creating / destroying queues across migration exposed by [2]
- Include a version of Satya's patches in [3] which enable CCS save / restore across VF migration /w GGTT shift
v3:
- Address feedback
- Fix preempt fence mode deadlock /w work queues + VF recovery (Testing)
- Add NULL checks to scratch LRC allocation
v4:
- Fix CI failure
- Remove config lock
Matt
[1] https://patchwork.freedesktop.org/series/154616/
[2] https://patchwork.freedesktop.org/series/154931/
[3] https://patchwork.freedesktop.org/series/154682/
Matthew Brost (32):
drm/xe: Add NULL checks to scratch LRC allocation
Revert "drm/xe/vf: Rebase exec queue parallel commands during
migration recovery"
Revert "drm/xe/vf: Post migration, repopulate ring area for pending
request"
Revert "drm/xe/vf: Fixup CTB send buffer messages after migration" drm/xe: Save off position in ring in which a job was programmed drm/xe/guc: Track pending-enable source in submission state drm/xe: Track LR jobs in DRM scheduler pending list drm/xe: Don't change LRC ring head on job resubmission drm/xe: Make LRC W/A scratch buffer usage consistent drm/xe/guc: Document GuC submission backend
drm/xe/vf: Add xe_gt_recovery_inprogress helper
drm/xe/vf: Make VF recovery run on per-GT worker drm/xe/vf: Abort H2G sends during VF post-migration recovery drm/xe/vf: Remove memory allocations from VF post migration recovery drm/xe/vf: Close multi-GT GGTT shift race
drm/xe/vf: Teardown VF post migration worker on driver unload drm/xe/vf: Don't allow GT reset to be queued during VF post migration
recovery
drm/xe/vf: Wakeup in GuC backend on VF post migration recovery drm/xe/vf: Avoid indefinite blocking in preempt rebind worker for VFs
supporting migration
drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register drm/xe/vf: Flush and stop CTs in VF post migration recovery drm/xe/vf: Reset TLB invalidations during VF post migration recovery drm/xe/vf: Kickstart after resfix in VF post migration recovery drm/xe/vf: Start CTs before resfix VF post migration recovery drm/xe/vf: Abort VF post migration recovery on failure drm/xe/vf: Replay GuC submission state on pause / unpause drm/xe: Move queue init before LRC creation
drm/xe/vf: Add debug prints for GuC replaying state during VF recovery drm/xe/vf: Workaround for race condition in GuC firmware during VF
pause
drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VF drm/xe/vf: Ensure media GT VF recovery runs after primary GT on PTL drm/xe/vf: Rebase CCS save/restore BB GGTT addresses
Satyanarayana K V P (2):
drm/xe: Use PPGTT addresses for TLB invalidation to avoid GGTT fixups drm/xe/guc: Increase wait timeout to 2sec after BUSY reply from GuC
Documentation/gpu/xe/index.rst | 1 + drivers/gpu/drm/xe/abi/guc_actions_abi.h | 8 - drivers/gpu/drm/xe/xe_device_types.h | 5 + drivers/gpu/drm/xe/xe_exec.c | 12 +- drivers/gpu/drm/xe/xe_exec_queue.c | 86 +- drivers/gpu/drm/xe/xe_exec_queue.h | 5 +- drivers/gpu/drm/xe/xe_execlist.c | 2 +- drivers/gpu/drm/xe/xe_gpu_scheduler.c | 14 + drivers/gpu/drm/xe/xe_gpu_scheduler.h | 2 + drivers/gpu/drm/xe/xe_gt.c | 37 +- drivers/gpu/drm/xe/xe_gt.h | 15 +- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 465 ++++++++--- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 13 +- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 33 +- drivers/gpu/drm/xe/xe_guc.c | 4 +- drivers/gpu/drm/xe/xe_guc_ct.c | 304 +++---- drivers/gpu/drm/xe/xe_guc_ct.h | 4 +- drivers/gpu/drm/xe/xe_guc_exec_queue_types.h | 15 + drivers/gpu/drm/xe/xe_guc_submit.c | 834 +++++++++++++++---- drivers/gpu/drm/xe/xe_guc_submit.h | 7 +- drivers/gpu/drm/xe/xe_lrc.c | 12 +- drivers/gpu/drm/xe/xe_lrc.h | 10 + drivers/gpu/drm/xe/xe_map.h | 18 - drivers/gpu/drm/xe/xe_memirq.c | 48 +- drivers/gpu/drm/xe/xe_memirq.h | 2 + drivers/gpu/drm/xe/xe_migrate.c | 28 +- drivers/gpu/drm/xe/xe_pci.c | 6 +- drivers/gpu/drm/xe/xe_pci_types.h | 1 + drivers/gpu/drm/xe/xe_preempt_fence.c | 11 + drivers/gpu/drm/xe/xe_ring_ops.c | 23 +- drivers/gpu/drm/xe/xe_sched_job_types.h | 9 + drivers/gpu/drm/xe/xe_sriov_vf.c | 243 ------ drivers/gpu/drm/xe/xe_sriov_vf.h | 1 - drivers/gpu/drm/xe/xe_sriov_vf_ccs.c | 28 + drivers/gpu/drm/xe/xe_sriov_vf_ccs.h | 1 + drivers/gpu/drm/xe/xe_sriov_vf_types.h | 4 - drivers/gpu/drm/xe/xe_tile.c | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf.c | 30 +- drivers/gpu/drm/xe/xe_tile_sriov_vf.h | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h | 23 + drivers/gpu/drm/xe/xe_vm.c | 26 +- drivers/gpu/drm/xe/xe_vram.c | 6 +- 42 files changed, 1513 insertions(+), 887 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 1/34 drm/xe: Add NULL checks to scratch LRC allocation
kmalloc can fail, the returned value must have a NULL check. This should be immediately after kmalloc for clarity.
--- drivers/gpu/drm/xe/xe_lrc.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_lrc.c +8 -2
@@ -1303,8 +1303,11 @@ static int setup_wa_bb(struct xe_lrc *lrc, struct xe_hw_engine *hwe)u32 *buf =NULL ;int ret; -if (lrc->bo->vmap.is_iomem) +if (lrc->bo->vmap.is_iomem) { buf =kmalloc (LRC_WA_BB_SIZE ,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + } ret =xe_lrc_setup_wa_bb_with_scratch (lrc, hwe, buf);
@@ -1347,8 +1350,11 @@ setup_indirect_ctx(struct xe_lrc *lrc, struct xe_hw_engine *hwe)if (xe_gt_WARN_ON (lrc->gt, !state.funcs))return 0 ; -if (lrc->bo->vmap.is_iomem) +if (lrc->bo->vmap.is_iomem) { state.buffer =kmalloc (state.max_size,GFP_KERNEL ); +if (!state.buffer) +return -ENOMEM ; + } ret =setup_bo (&state);if (ret) {
Signature · 1 line
2.34.1
Re: [PATCH v4 01/34] drm/xe: Add NULL checks to scratch LRC allocation
Quoted 27 lines · Matthew Brost
kmalloc can fail, the returned value must have a NULL check. This should be immediately after kmalloc for clarity.
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_lrc.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_lrc.c +4
-1@@ -1303,8 +1303,11 @@ static int setup_wa_bb(struct xe_lrc *lrc, struct xe_hw_engine *hwe)u32 *buf =NULL ;int ret; -if (lrc->bo->vmap.is_iomem) +if (lrc->bo->vmap.is_iomem) { buf =kmalloc (LRC_WA_BB_SIZE ,GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + } ret =xe_lrc_setup_wa_bb_with_scratch (lrc, hwe, buf);@@ -1347,8 +1350,11 @@ setup_indirect_ctx(struct xe_lrc *lrc, struct xe_hw_engine *hwe)if (xe_gt_WARN_ON (lrc->gt, !state.funcs))
drivers/gpu/drm/xe/xe_lrc.c setup_indirect_ctx(struct xe_lrc *lrc, struct xe_hw_engine *hwe) · line 1359
Quoted 10 lines
drivers/gpu/drm/xe/xe_lrc.c +4
-1return 0 ; -if (lrc->bo->vmap.is_iomem) +if (lrc->bo->vmap.is_iomem) { state.buffer =kmalloc (state.max_size,GFP_KERNEL ); +if (!state.buffer) +return -ENOMEM ; + } ret =setup_bo (&state);
Now it makes no sense to return -ENOMEM inside setup_bo(). We should assert that the pointer is not NULL there.
-Tomasz
Quoted 1 line
drivers/gpu/drm/xe/xe_lrc.c +0
-0if (ret) {
Matthew Brost
maintainerPATCH v4 2/34 Revert "drm/xe/vf: Rebase exec queue parallel commands during migration recovery"
In reply to Matthew BrostThis reverts commit ba180a362128cb71d16c3f0ce6645448011d2607.
Due to change in the VF migration recovery design this code is not needed any more.
v3:
- Add commit message (Michal / Lucas)
--- drivers/gpu/drm/xe/abi/guc_actions_abi.h | 8 ---- drivers/gpu/drm/xe/xe_guc_submit.c | 54 ------------------------ 2 files changed, 62 deletions(-)
drivers/gpu/drm/xe/abi/guc_actions_abi.h +0 -8
@@ -196,14 +196,6 @@ enum xe_guc_register_context_multi_lrc_param_offsets { XE_GUC_REGISTER_CONTEXT_MULTI_LRC_MSG_MIN_LEN = 11, }; -enum xe_guc_context_wq_item_offsets { -XE_GUC_CONTEXT_WQ_HEADER_DATA_0_TYPE_LEN =0 , -XE_GUC_CONTEXT_WQ_EL_INFO_DATA_1_CTX_DESC_LOW , -XE_GUC_CONTEXT_WQ_EL_INFO_DATA_2_GUCCTX_RINGTAIL_FREEZEPOCS , -XE_GUC_CONTEXT_WQ_EL_INFO_DATA_3_WI_FENCE_ID , -XE_GUC_CONTEXT_WQ_EL_CHILD_LIST_DATA_4_RINGTAIL , -}; -enum xe_guc_report_status {XE_GUC_REPORT_STATUS_UNKNOWN =0x0 ,XE_GUC_REPORT_STATUS_ACKED =0x1 ,
drivers/gpu/drm/xe/xe_guc_submit.c +0 -54
78 lines not on this page · raw message
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 3/34 Revert "drm/xe/vf: Post migration, repopulate ring area for pending request"
In reply to Matthew BrostThis reverts commit a0dda25d24e636df5c30a9370464b7cebc709faf.
Due to change in the VF migration recovery design this code is not needed any more.
v3:
- Add commit message (Michal / Lucas)
--- drivers/gpu/drm/xe/xe_exec_queue.c | 24 ------------------------ drivers/gpu/drm/xe/xe_exec_queue.h | 3 +-- drivers/gpu/drm/xe/xe_guc_submit.c | 24 ------------------------ drivers/gpu/drm/xe/xe_guc_submit.h | 2 -- drivers/gpu/drm/xe/xe_sriov_vf.c | 1 - 5 files changed, 1 insertion(+), 53 deletions(-)
drivers/gpu/drm/xe/xe_exec_queue.c +0 -24
@@ -1123,27 +1123,3 @@ int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch)return err; } - -/** -* xe_exec_queue_jobs_ring_restore - Re-emit ring commands of requests pending on given queue. -* @q: the &xe_exec_queue struct instance -*/ -void xe_exec_queue_jobs_ring_restore (struct xe_exec_queue *q) -{ -struct xe_gpu_scheduler *sched = &q->guc->sched; -struct xe_sched_job *job; - -/* -* This routine is used within VF migration recovery. This means -* using the lock here introduces a restriction: we cannot wait -* for any GFX HW response while the lock is taken. -*/ -spin_lock (&sched->base.job_list_lock); -list_for_each_entry (job, &sched->base.pending_list, drm.list) { -if (xe_sched_job_is_error (job)) -continue ; - - q->ring_ops->emit_job (job); - } -spin_unlock (&sched->base.job_list_lock); -}
drivers/gpu/drm/xe/xe_exec_queue.h +1 -2
@@ -92,7 +92,6 @@ void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q);int xe_exec_queue_contexts_hwsp_rebase (struct xe_exec_queue *q,void *scratch); -void xe_exec_queue_jobs_ring_restore (struct xe_exec_queue *q); -struct xe_lrc *xe_exec_queue_lrc (struct xe_exec_queue *q); +#endif
drivers/gpu/drm/xe/xe_guc_submit.c +0 -24
@@ -845,30 +845,6 @@ guc_exec_queue_run_job(struct drm_sched_job *drm_job)return fence; } -/** -* xe_guc_jobs_ring_rebase - Re-emit ring commands of requests pending -* on all queues under a guc. -* @guc: the &xe_guc struct instance -*/ -void xe_guc_jobs_ring_rebase (struct xe_guc *guc) -{ -struct xe_exec_queue *q; -unsigned long index; - -/* -* This routine is used within VF migration recovery. This means -* using the lock here introduces a restriction: we cannot wait -* for any GFX HW response while the lock is taken. -*/ -mutex_lock (&guc->submission_state.lock); -xa_for_each (&guc->submission_state.exec_queue_lookup, index, q) { -if (exec_queue_killed_or_banned_or_wedged (q)) -continue ; -xe_exec_queue_jobs_ring_restore (q); - } -mutex_unlock (&guc->submission_state.lock); -} -static void guc_exec_queue_free_job (struct drm_sched_job *drm_job) {struct xe_sched_job *job =to_xe_sched_job (drm_job);
drivers/gpu/drm/xe/xe_guc_submit.h +0 -2
@@ -36,8 +36,6 @@ int xe_guc_exec_queue_memory_cat_error_handler(struct xe_guc *guc, u32 *msg,int xe_guc_exec_queue_reset_failure_handler (struct xe_guc *guc,u32 *msg,u32 len);int xe_guc_error_capture_handler (struct xe_guc *guc,u32 *msg,u32 len); -void xe_guc_jobs_ring_rebase (struct xe_guc *guc); -struct xe_guc_submit_exec_queue_snapshot *xe_guc_exec_queue_snapshot_capture (struct xe_exec_queue *q);void
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -1
@@ -334,7 +334,6 @@ static int gt_vf_post_migration_fixups(struct xe_gt *gt) err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf);if (err) goto out; -xe_guc_jobs_ring_rebase (>->uc.guc);xe_guc_ct_fixup_messages_with_ggtt (>->uc.guc.ct, shift); }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 4/34 Revert "drm/xe/vf: Fixup CTB send buffer messages after migration"
In reply to Matthew BrostThis reverts commit cef88d1265cac7d415606af73ba58926fd3cd8b7.
Due to change in the VF migration recovery design this code is not needed any more.
v3:
- Add commit message (Michal / Lucas)
--- drivers/gpu/drm/xe/xe_guc_ct.c | 183 ------------------------------- drivers/gpu/drm/xe/xe_guc_ct.h | 2 - drivers/gpu/drm/xe/xe_map.h | 18 --- drivers/gpu/drm/xe/xe_sriov_vf.c | 2 - 4 files changed, 205 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +0 -183
@@ -25,7 +25,6 @@#include "xe_gt_printk.h" #include "xe_gt_sriov_pf_control.h" #include "xe_gt_sriov_pf_monitor.h" -#include "xe_gt_sriov_printk.h" #include "xe_guc.h" #include "xe_guc_log.h" #include "xe_guc_relay.h"
@@ -93,8 +92,6 @@ struct g2h_fence {bool done; }; -#define make_u64 (hi, lo) ((u64)((u64)(u32)(hi) << 32 | (u32)(lo))) -static void g2h_fence_init (struct g2h_fence *g2h_fence,u32 *response_buffer) {memset (g2h_fence,0 ,sizeof (*g2h_fence));
@@ -1793,186 +1790,6 @@ static void g2h_worker_func(struct work_struct *w)receive_g2h (ct); } -static void xe_fixup_u64_in_cmds (struct xe_device *xe,struct iosys_map *cmds, -u32 size,u32 idx,s64 shift) -{ -u32 hi, lo; -u64 offset; - - lo =xe_map_rd_ring_u32 (xe, cmds, idx, size); - hi =xe_map_rd_ring_u32 (xe, cmds, idx +1 , size); - offset =make_u64 (hi, lo); - offset += shift; - lo =lower_32_bits (offset); - hi =upper_32_bits (offset); -xe_map_wr_ring_u32 (xe, cmds, idx, size, lo); -xe_map_wr_ring_u32 (xe, cmds, idx +1 , size, hi); -} - -/* -* Shift any GGTT addresses within a single message left within CTB from -* before post-migration recovery. -* @ct: pointer to CT struct of the target GuC -* @cmds: iomap buffer containing CT messages -* @head: start of the target message within the buffer -* @len: length of the target message -* @size: size of the commands buffer -* @shift: the address shift to be added to each GGTT reference -* Return: true if the message was fixed or needed no fixups, false on failure -*/ -static bool ct_fixup_ggtt_in_message (struct xe_guc_ct *ct, -struct iosys_map *cmds,u32 head, -u32 len,u32 size,s64 shift) -{ -struct xe_gt *gt =ct_to_gt (ct); -struct xe_device *xe =ct_to_xe (ct); -u32 msg[GUC_HXG_MSG_MIN_LEN ]; -u32 action, i, n; - -xe_gt_assert (gt, len >=GUC_HXG_MSG_MIN_LEN ); - - msg[0 ] =xe_map_rd_ring_u32 (xe, cmds, head, size); - action =FIELD_GET (GUC_HXG_REQUEST_MSG_0_ACTION , msg[0 ]); - -xe_gt_sriov_dbg_verbose (gt,"fixing H2G %#x\n" , action); - -switch (action) { -case XE_GUC_ACTION_REGISTER_CONTEXT : -if (len !=XE_GUC_REGISTER_CONTEXT_MSG_LEN ) - goto err_len; -xe_fixup_u64_in_cmds (xe, cmds, size, head + -XE_GUC_REGISTER_CONTEXT_DATA_5_WQ_DESC_ADDR_LOWER , - shift); -xe_fixup_u64_in_cmds (xe, cmds, size, head + -XE_GUC_REGISTER_CONTEXT_DATA_7_WQ_BUF_BASE_LOWER , - shift); -xe_fixup_u64_in_cmds (xe, cmds, size, head + -XE_GUC_REGISTER_CONTEXT_DATA_10_HW_LRC_ADDR , shift); -break ; -case XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC : -if (len <XE_GUC_REGISTER_CONTEXT_MULTI_LRC_MSG_MIN_LEN ) - goto err_len; - n =xe_map_rd_ring_u32 (xe, cmds, head + -XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_10_NUM_CTXS , size); -if (len !=XE_GUC_REGISTER_CONTEXT_MULTI_LRC_MSG_MIN_LEN +2 * n) - goto err_len; -xe_fixup_u64_in_cmds (xe, cmds, size, head + -XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_5_WQ_DESC_ADDR_LOWER , - shift); -xe_fixup_u64_in_cmds (xe, cmds, size, head + -XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_7_WQ_BUF_BASE_LOWER , - shift); -for (i =0 ; i < n; i++) -xe_fixup_u64_in_cmds (xe, cmds, size, head + -XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_11_HW_LRC_ADDR - +2 * i, shift); -break ; -default : -break ; - } -return true; - -err_len: -xe_gt_err (gt,"Skipped G2G %#x message fixups, unexpected length (%u)\n" , action, len); -return false; -} - -/* -* Apply fixups to the next outgoing CT message within given CTB -* @ct: the &xe_guc_ct struct instance representing the target GuC -* @h2g: the &guc_ctb struct instance of the target buffer -* @shift: shift to be added to all GGTT addresses within the CTB -* @mhead: pointer to an integer storing message start position; the -* position is changed to next message before this function return -* @avail: size of the area available for parsing, that is length -* of all remaining messages stored within the CTB -* Return: size of the area available for parsing after one message -* has been parsed, that is length remaining from the updated mhead -*/ -static int ct_fixup_ggtt_in_buffer (struct xe_guc_ct *ct,struct guc_ctb *h2g, -s64 shift,u32 *mhead,s32 avail) -{ -struct xe_gt *gt =ct_to_gt (ct); -struct xe_device *xe =ct_to_xe (ct); -u32 msg[GUC_HXG_MSG_MIN_LEN ]; -u32 size = h2g->info.size; -u32 head = *mhead; -u32 len; - -xe_gt_assert (gt, avail >= (s32 )GUC_CTB_MSG_MIN_LEN ); - -/* Read header */ - msg[0 ] =xe_map_rd_ring_u32 (xe, &h2g->cmds, head, size); - len =FIELD_GET (GUC_CTB_MSG_0_NUM_DWORDS , msg[0 ]) +GUC_CTB_MSG_MIN_LEN ; - -if (unlikely (len > (u32 )avail)) { -xe_gt_err (gt,"H2G channel broken on read, avail=%d, len=%d, fixups skipped\n" , - avail, len); -return 0 ; - } - - head = (head +GUC_CTB_MSG_MIN_LEN ) % size; -if (!ct_fixup_ggtt_in_message (ct, &h2g->cmds, head,msg_len_to_hxg_len (len), size, shift)) -return 0 ; - *mhead = (head +msg_len_to_hxg_len (len)) % size; - -return avail - len; -} - -/** -* xe_guc_ct_fixup_messages_with_ggtt - Fixup any pending H2G CTB messages -* @ct: pointer to CT struct of the target GuC -* @ggtt_shift: shift to be added to all GGTT addresses within the CTB -* -* Messages in GuC to Host CTB are owned by GuC and any fixups in them -* are made by GuC. But content of the Host to GuC CTB is owned by the -* KMD, so fixups to GGTT references in any pending messages need to be -* applied here. -* This function updates GGTT offsets in payloads of pending H2G CTB -* messages (messages which were not consumed by GuC before the VF got -* paused). -*/ -void xe_guc_ct_fixup_messages_with_ggtt (struct xe_guc_ct *ct,s64 ggtt_shift) -{ -struct guc_ctb *h2g = &ct->ctbs.h2g; -struct xe_guc *guc =ct_to_guc (ct); -struct xe_gt *gt =guc_to_gt (guc); -u32 head, tail, size; -s32 avail; - -if (unlikely (h2g->info.broken)) -return ; - - h2g->info.head =desc_read (ct_to_xe (ct), h2g, head); - head = h2g->info.head; - tail =READ_ONCE (h2g->info.tail); - size = h2g->info.size; - -if (unlikely (head > size)) - goto corrupted; - -if (unlikely (tail >= size)) - goto corrupted; - - avail = tail - head; - -/* beware of buffer wrap case */ -if (unlikely (avail <0 )) - avail += size; -xe_gt_dbg (gt,"available %d (%u:%u:%u)\n" , avail, head, tail, size); -xe_gt_assert (gt, avail >=0 ); - -while (avail >0 ) - avail =ct_fixup_ggtt_in_buffer (ct, h2g, ggtt_shift, &head, avail); - -return ; - -corrupted: -xe_gt_err (gt,"Corrupted H2G descriptor head=%u tail=%u size=%u, fixups not applied\n" , - head, tail, size); - h2g->info.broken = true; -} -static struct xe_guc_ct_snapshot *guc_ct_snapshot_alloc (struct xe_guc_ct *ct,bool atomic,bool want_ctb) {
drivers/gpu/drm/xe/xe_guc_ct.h +0 -2
@@ -24,8 +24,6 @@ void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot, struct drm_prvoid xe_guc_ct_snapshot_free (struct xe_guc_ct_snapshot *snapshot);void xe_guc_ct_print (struct xe_guc_ct *ct,struct drm_printer *p,bool want_ctb); -void xe_guc_ct_fixup_messages_with_ggtt (struct xe_guc_ct *ct,s64 ggtt_shift); -static inline bool xe_guc_ct_initialized (struct xe_guc_ct *ct) {return ct->state !=XE_GUC_CT_STATE_NOT_INITIALIZED ;
drivers/gpu/drm/xe/xe_map.h +0 -18
@@ -78,24 +78,6 @@ static inline void xe_map_write32(struct xe_device *xe, struct iosys_map *map,iosys_map_wr (map__, offset__, type__, val__); \ }) -#define xe_map_rd_array (xe__, map__, index__, type__) \ - xe_map_rd(xe__, map__, (index__) * sizeof(type__), type__) - -#define xe_map_wr_array (xe__, map__, index__, type__, val__) \ - xe_map_wr(xe__, map__, (index__) * sizeof(type__), type__, val__) - -#define xe_map_rd_array_u32 (xe__, map__, index__) \ - xe_map_rd_array(xe__, map__, index__, u32) - -#define xe_map_wr_array_u32 (xe__, map__, index__, val__) \ - xe_map_wr_array(xe__, map__, index__, u32, val__) - -#define xe_map_rd_ring_u32 (xe__, map__, index__, size__) \ - xe_map_rd_array_u32(xe__, map__, (index__) % (size__)) - -#define xe_map_wr_ring_u32 (xe__, map__, index__, size__, val__) \ - xe_map_wr_array_u32(xe__, map__, (index__) % (size__), val__) -#define xe_map_rd_field (xe__, map__, struct_offset__, struct_type__, field__) ({ \ struct xe_device *__xe = xe__; \ xe_device_assert_mem_access(__xe); \
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -2
@@ -12,7 +12,6 @@#include "xe_gt_sriov_printk.h" #include "xe_gt_sriov_vf.h" #include "xe_guc.h" -#include "xe_guc_ct.h" #include "xe_guc_submit.h" #include "xe_irq.h" #include "xe_lrc.h"
@@ -334,7 +333,6 @@ static int gt_vf_post_migration_fixups(struct xe_gt *gt) err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf);if (err) goto out; -xe_guc_ct_fixup_messages_with_ggtt (>->uc.guc.ct, shift); } out:
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 5/34 drm/xe: Save off position in ring in which a job was programmed
In reply to Matthew BrostVF post-migration recovery needs to modify the ring with updated GGTT addresses for pending jobs. Save off position in ring in which a job was programmed to facilitate.
v4:
- s/VF resume/VF post-migration recovery (Tomasz)
--- drivers/gpu/drm/xe/xe_ring_ops.c | 23 +++++++++++++++++++---- drivers/gpu/drm/xe/xe_sched_job_types.h | 5 +++++ 2 files changed, 24 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_ring_ops.c +19 -4
@@ -245,12 +245,14 @@ static int emit_copy_timestamp(struct xe_lrc *lrc, u32 *dw, int i)/* for engines that don't require any special HW handling (no EUs, no aux inval, etc) */ static void __emit_job_gen12_simple (struct xe_sched_job *job,struct xe_lrc *lrc, -u64 batch_addr,u32 seqno) +u64 batch_addr,u32 *head,u32 seqno) {u32 dw[MAX_JOB_SIZE_DW ], i =0 ;u32 ppgtt_flag =get_ppgtt_flag (job);struct xe_gt *gt = job->q->gt; + *head = lrc->ring.tail; + i =emit_copy_timestamp (lrc, dw, i);if (job->ring_ops_flush_tlb) {
@@ -296,7 +298,7 @@ static bool has_aux_ccs(struct xe_device *xe) }static void __emit_job_gen12_video (struct xe_sched_job *job,struct xe_lrc *lrc, -u64 batch_addr,u32 seqno) +u64 batch_addr,u32 *head,u32 seqno) {u32 dw[MAX_JOB_SIZE_DW ], i =0 ;u32 ppgtt_flag =get_ppgtt_flag (job);
@@ -304,6 +306,8 @@ static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc,struct xe_device *xe =gt_to_xe (gt);bool decode = job->q->class ==XE_ENGINE_CLASS_VIDEO_DECODE ; + *head = lrc->ring.tail; + i =emit_copy_timestamp (lrc, dw, i); dw[i++] =preparser_disable (true);
@@ -346,7 +350,8 @@ static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc,static void __emit_job_gen12_render_compute (struct xe_sched_job *job,struct xe_lrc *lrc, -u64 batch_addr,u32 seqno) +u64 batch_addr,u32 *head, +u32 seqno) {u32 dw[MAX_JOB_SIZE_DW ], i =0 ;u32 ppgtt_flag =get_ppgtt_flag (job);
@@ -355,6 +360,8 @@ static void __emit_job_gen12_render_compute(struct xe_sched_job *job,bool lacks_render = !(gt->info.engine_mask &XE_HW_ENGINE_RCS_MASK );u32 mask_flags =0 ; + *head = lrc->ring.tail; + i =emit_copy_timestamp (lrc, dw, i); dw[i++] =preparser_disable (true);
@@ -396,11 +403,14 @@ static void __emit_job_gen12_render_compute(struct xe_sched_job *job, }static void emit_migration_job_gen12 (struct xe_sched_job *job, -struct xe_lrc *lrc,u32 seqno) +struct xe_lrc *lrc,u32 *head, +u32 seqno) {u32 saddr =xe_lrc_start_seqno_ggtt_addr (lrc);u32 dw[MAX_JOB_SIZE_DW ], i =0 ; + *head = lrc->ring.tail; + i =emit_copy_timestamp (lrc, dw, i); i =emit_store_imm_ggtt (saddr, seqno, dw, i);
@@ -434,6 +444,7 @@ static void emit_job_gen12_gsc(struct xe_sched_job *job)__emit_job_gen12_simple (job, job->q->lrc[0 ], job->ptrs[0 ].batch_addr, + &job->ptrs[0 ].head,xe_sched_job_lrc_seqno (job)); }
@@ -443,6 +454,7 @@ static void emit_job_gen12_copy(struct xe_sched_job *job)if (xe_sched_job_is_migration (job->q)) {emit_migration_job_gen12 (job, job->q->lrc[0 ], + &job->ptrs[0 ].head,xe_sched_job_lrc_seqno (job));return ; }
@@ -450,6 +462,7 @@ static void emit_job_gen12_copy(struct xe_sched_job *job)for (i =0 ; i < job->q->width; ++i)__emit_job_gen12_simple (job, job->q->lrc[i], job->ptrs[i].batch_addr, + &job->ptrs[i].head,xe_sched_job_lrc_seqno (job)); }
@@ -461,6 +474,7 @@ static void emit_job_gen12_video(struct xe_sched_job *job)for (i =0 ; i < job->q->width; ++i)__emit_job_gen12_video (job, job->q->lrc[i], job->ptrs[i].batch_addr, + &job->ptrs[i].head,xe_sched_job_lrc_seqno (job)); }
@@ -471,6 +485,7 @@ static void emit_job_gen12_render_compute(struct xe_sched_job *job)for (i =0 ; i < job->q->width; ++i)__emit_job_gen12_render_compute (job, job->q->lrc[i], job->ptrs[i].batch_addr, + &job->ptrs[i].head,xe_sched_job_lrc_seqno (job)); }
drivers/gpu/drm/xe/xe_sched_job_types.h +5 -0
@@ -24,6 +24,11 @@ struct xe_job_ptrs {struct dma_fence_chain *chain_fence;/** @batch_addr: Batch buffer address. */ u64 batch_addr; +/** +* @head: The tail pointer of the LRC (so head pointer of job) when the +* job was submitted +*/ +u32 head; };/**
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 6/34 drm/xe/guc: Track pending-enable source in submission state
In reply to Matthew BrostAdd explicit tracking in the GuC submission state to record the source of a pending enable (TDR vs. queue resume path vs. submission). Disambiguating the origin lets the GuC submission state machine apply the correct recovery/replay behavior.
This helps VF restore: when the device comes back, the state machine knows whether the pending enable stems from timeout recovery, from a queue resume sequence, or submission and can gate sequencing and fixups accordingly.
v4:
- Clarify commit message (Tomasz)
--- drivers/gpu/drm/xe/xe_guc_submit.c | 36 ++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
drivers/gpu/drm/xe/xe_guc_submit.c +36 -0
@@ -69,6 +69,8 @@ exec_queue_to_guc(struct xe_exec_queue *q)#define EXEC_QUEUE_STATE_BANNED (1 << 9)#define EXEC_QUEUE_STATE_CHECK_TIMEOUT (1 << 10)#define EXEC_QUEUE_STATE_EXTRA_REF (1 << 11) +#define EXEC_QUEUE_STATE_PENDING_RESUME (1 << 12) +#define EXEC_QUEUE_STATE_PENDING_TDR_EXIT (1 << 13)static bool exec_queue_registered (struct xe_exec_queue *q) {
@@ -220,6 +222,36 @@ static void set_exec_queue_extra_ref(struct xe_exec_queue *q)atomic_or (EXEC_QUEUE_STATE_EXTRA_REF , &q->guc->state); } +static bool __maybe_unused exec_queue_pending_resume (struct xe_exec_queue *q) +{ +return atomic_read (&q->guc->state) &EXEC_QUEUE_STATE_PENDING_RESUME ; +} + +static void set_exec_queue_pending_resume (struct xe_exec_queue *q) +{ +atomic_or (EXEC_QUEUE_STATE_PENDING_RESUME , &q->guc->state); +} + +static void clear_exec_queue_pending_resume (struct xe_exec_queue *q) +{ +atomic_and (~EXEC_QUEUE_STATE_PENDING_RESUME , &q->guc->state); +} + +static bool __maybe_unused exec_queue_pending_tdr_exit (struct xe_exec_queue *q) +{ +return atomic_read (&q->guc->state) &EXEC_QUEUE_STATE_PENDING_TDR_EXIT ; +} + +static void set_exec_queue_pending_tdr_exit (struct xe_exec_queue *q) +{ +atomic_or (EXEC_QUEUE_STATE_PENDING_TDR_EXIT , &q->guc->state); +} + +static void clear_exec_queue_pending_tdr_exit (struct xe_exec_queue *q) +{ +atomic_and (~EXEC_QUEUE_STATE_PENDING_TDR_EXIT , &q->guc->state); +} +static bool exec_queue_killed_or_banned_or_wedged (struct xe_exec_queue *q) { return (atomic_read(&q->guc->state) &
@@ -1334,6 +1366,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)return DRM_GPU_SCHED_STAT_RESET ; sched_enable: +set_exec_queue_pending_tdr_exit (q);enable_scheduling (q); rearm:/*
@@ -1493,6 +1526,7 @@ static void __guc_exec_queue_process_msg_resume(struct xe_sched_msg *msg)clear_exec_queue_suspended (q);if (!exec_queue_enabled (q)) { q->guc->resume_time =RESUME_PENDING ; +set_exec_queue_pending_resume (q);enable_scheduling (q); } }else {
@@ -2065,6 +2099,8 @@ static void handle_sched_done(struct xe_guc *guc, struct xe_exec_queue *q,xe_gt_assert (guc_to_gt (guc),exec_queue_pending_enable (q)); q->guc->resume_time =ktime_get (); +clear_exec_queue_pending_resume (q); +clear_exec_queue_pending_tdr_exit (q);clear_exec_queue_pending_enable (q);smp_wmb ();wake_up_all (&guc->ct.wq);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 7/34 drm/xe: Track LR jobs in DRM scheduler pending list
In reply to Matthew BrostVF migration requires jobs to remain pending so they can be replayed after the VF comes back. Previously, LR job fences were intentionally signaled immediately after submission to avoid the risk of exporting them, as these fences do not naturally signal in a timely manner and could break dma-fence contracts. A side effect of this approach was that LR jobs were never added to the DRM scheduler’s pending list, preventing them from being tracked for later resubmission.
We now avoid signaling LR job fences and ensure they are never exported; Xe already guards against exporting these internal fences. With that guarantee in place, we can safely track LR jobs in the scheduler’s pending list so they are eligible for resubmission during VF post-migration recovery (and similar recovery paths).
An added benefit is that LR queues now gain the DRM scheduler’s built-in flow control over ring usage rather than rejecting new jobs in the exec IOCTL if the ring is full.
v2:
- Ensure DRM scheduler TDR doesn't run for LR jobs
- Stack variable for killed_or_banned_or_wedged
v4:
- Clarify commit message (Tomasz)
--- drivers/gpu/drm/xe/xe_exec.c | 12 ++------- drivers/gpu/drm/xe/xe_exec_queue.c | 19 ------------- drivers/gpu/drm/xe/xe_exec_queue.h | 2 -- drivers/gpu/drm/xe/xe_guc_submit.c | 43 ++++++++++++++++++++---------- 4 files changed, 31 insertions(+), 45 deletions(-)
drivers/gpu/drm/xe/xe_exec.c +2 -10
@@ -124,7 +124,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)struct xe_validation_ctx ctx;struct xe_sched_job *job;struct xe_vm *vm; -bool write_locked, skip_retry = false; +bool write_locked;int err =0 ;struct xe_hw_engine_group *group;enum xe_hw_engine_group_execution_mode mode, previous_mode;
@@ -266,12 +266,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_exec; } -if (xe_exec_queue_is_lr (q) &&xe_exec_queue_ring_full (q)) { - err = -EWOULDBLOCK ;/* Aliased to -EAGAIN */ - skip_retry = true; - goto err_exec; - } -if (xe_exec_queue_uses_pxp (q)) { err =xe_vm_validate_protected (q->vm);if (err)
@@ -328,8 +322,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)xe_sched_job_init_user_fence (job, &syncs[i]); } -if (xe_exec_queue_is_lr (q)) - q->ring_ops->emit_job (job);if (!xe_vm_in_lr_mode (vm))xe_exec_queue_last_fence_set (q, vm, &job->drm.s_fence->finished);xe_sched_job_push (job);
@@ -355,7 +347,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)xe_validation_ctx_fini (&ctx); err_unlock_list:up_read (&vm->lock); -if (err == -EAGAIN && !skip_retry) +if (err == -EAGAIN ) goto retry; err_hw_exec_mode:if (mode ==EXEC_MODE_DMA_FENCE )
drivers/gpu/drm/xe/xe_exec_queue.c +0 -19
@@ -824,25 +824,6 @@ bool xe_exec_queue_is_lr(struct xe_exec_queue *q) !(q->flags &EXEC_QUEUE_FLAG_VM ); } -static s32 xe_exec_queue_num_job_inflight (struct xe_exec_queue *q) -{ -return q->lrc[0 ]->fence_ctx.next_seqno -xe_lrc_seqno (q->lrc[0 ]) -1 ; -} - -/** -* xe_exec_queue_ring_full() - Whether an exec_queue's ring is full -* @q: The exec_queue -* -* Return: True if the exec_queue's ring is full, false otherwise. -*/ -bool xe_exec_queue_ring_full (struct xe_exec_queue *q) -{ -struct xe_lrc *lrc = q->lrc[0 ]; -s32 max_job = lrc->ring.size /MAX_JOB_SIZE_BYTES ; - -return xe_exec_queue_num_job_inflight (q) >= max_job; -} -/** * xe_exec_queue_is_idle() - Whether an exec_queue is idle. * @q: The exec_queue
drivers/gpu/drm/xe/xe_exec_queue.h +0 -2
@@ -64,8 +64,6 @@ static inline bool xe_exec_queue_uses_pxp(struct xe_exec_queue *q)bool xe_exec_queue_is_lr (struct xe_exec_queue *q); -bool xe_exec_queue_ring_full (struct xe_exec_queue *q); -bool xe_exec_queue_is_idle (struct xe_exec_queue *q);void xe_exec_queue_kill (struct xe_exec_queue *q);
drivers/gpu/drm/xe/xe_guc_submit.c +29 -14
92 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v4 07/34] drm/xe: Track LR jobs in DRM scheduler pending list
Quoted 214 lines · Matthew Brost
VF migration requires jobs to remain pending so they can be replayed after the VF comes back. Previously, LR job fences were intentionally signaled immediately after submission to avoid the risk of exporting them, as these fences do not naturally signal in a timely manner and could break dma-fence contracts. A side effect of this approach was that LR jobs were never added to the DRM scheduler’s pending list, preventing them from being tracked for later resubmission.
We now avoid signaling LR job fences and ensure they are never exported; Xe already guards against exporting these internal fences. With that guarantee in place, we can safely track LR jobs in the scheduler’s pending list so they are eligible for resubmission during VF post-migration recovery (and similar recovery paths).
An added benefit is that LR queues now gain the DRM scheduler’s built-in flow control over ring usage rather than rejecting new jobs in the exec IOCTL if the ring is full.
v2:
- Ensure DRM scheduler TDR doesn't run for LR jobs
- Stack variable forkilled_or_banned_or_wedged
v4:
- Clarify commit message (Tomasz)Signed-off-by Matthew Brost <[email protected]>Reviewed-by Tomasz Lis <[email protected]>--- drivers/gpu/drm/xe/xe_exec.c | 12 ++------- drivers/gpu/drm/xe/xe_exec_queue.c | 19 ------------- drivers/gpu/drm/xe/xe_exec_queue.h | 2 -- drivers/gpu/drm/xe/xe_guc_submit.c | 43 ++++++++++++++++++++---------- 4 files changed, 31 insertions(+), 45 deletions(-)drivers/gpu/drm/xe/xe_exec.c +2
-10@@ -124,7 +124,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)struct xe_validation_ctx ctx;struct xe_sched_job *job;struct xe_vm *vm; -bool write_locked, skip_retry = false; +bool write_locked;int err =0 ;struct xe_hw_engine_group *group;enum xe_hw_engine_group_execution_mode mode, previous_mode;@@ -266,12 +266,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_exec; } -if (xe_exec_queue_is_lr (q) &&xe_exec_queue_ring_full (q)) { - err = -EWOULDBLOCK ;/* Aliased to -EAGAIN */ - skip_retry = true; - goto err_exec; - } -if (xe_exec_queue_uses_pxp (q)) { err =xe_vm_validate_protected (q->vm);if (err)@@ -328,8 +322,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)xe_sched_job_init_user_fence (job, &syncs[i]); } -if (xe_exec_queue_is_lr (q)) - q->ring_ops->emit_job (job);if (!xe_vm_in_lr_mode (vm))xe_exec_queue_last_fence_set (q, vm, &job->drm.s_fence->finished);xe_sched_job_push (job);@@ -355,7 +347,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)xe_validation_ctx_fini (&ctx); err_unlock_list:up_read (&vm->lock); -if (err == -EAGAIN && !skip_retry) +if (err == -EAGAIN ) goto retry; err_hw_exec_mode:if (mode ==EXEC_MODE_DMA_FENCE )drivers/gpu/drm/xe/xe_exec_queue.c +0
-19@@ -824,25 +824,6 @@ bool xe_exec_queue_is_lr(struct xe_exec_queue *q) !(q->flags &EXEC_QUEUE_FLAG_VM ); } -static s32 xe_exec_queue_num_job_inflight (struct xe_exec_queue *q) -{ -return q->lrc[0 ]->fence_ctx.next_seqno -xe_lrc_seqno (q->lrc[0 ]) -1 ; -} - -/** -* xe_exec_queue_ring_full() - Whether an exec_queue's ring is full -* @q: The exec_queue -* -* Return: True if the exec_queue's ring is full, false otherwise. -*/ -bool xe_exec_queue_ring_full (struct xe_exec_queue *q) -{ -struct xe_lrc *lrc = q->lrc[0 ]; -s32 max_job = lrc->ring.size /MAX_JOB_SIZE_BYTES ; - -return xe_exec_queue_num_job_inflight (q) >= max_job; -} -/** * xe_exec_queue_is_idle() - Whether an exec_queue is idle. * @q: The exec_queue drivers/gpu/drm/xe/xe_exec_queue.h +0
-2@@ -64,8 +64,6 @@ static inline bool xe_exec_queue_uses_pxp(struct xe_exec_queue *q)bool xe_exec_queue_is_lr (struct xe_exec_queue *q); -bool xe_exec_queue_ring_full (struct xe_exec_queue *q); -bool xe_exec_queue_is_idle (struct xe_exec_queue *q);void xe_exec_queue_kill (struct xe_exec_queue *q);drivers/gpu/drm/xe/xe_guc_submit.c +27
-14@@ -851,30 +851,31 @@ guc_exec_queue_run_job(struct drm_sched_job *drm_job)struct xe_sched_job *job =to_xe_sched_job (drm_job);struct xe_exec_queue *q = job->q;struct xe_guc *guc =exec_queue_to_guc (q); -struct dma_fence *fence =NULL ; -bool lr =xe_exec_queue_is_lr (q); +bool lr =xe_exec_queue_is_lr (q), killed_or_banned_or_wedged = +exec_queue_killed_or_banned_or_wedged (q);xe_gt_assert (guc_to_gt (guc), !(exec_queue_destroyed (q) ||exec_queue_pending_disable (q)) ||exec_queue_banned (q) ||exec_queue_suspended (q));trace_xe_sched_job_run (job); -if (!exec_queue_killed_or_banned_or_wedged (q) && !xe_sched_job_is_error (job)) { +if (!killed_or_banned_or_wedged && !xe_sched_job_is_error (job)) {if (!exec_queue_registered (q))register_exec_queue (q,GUC_CONTEXT_NORMAL ); -if (!lr)/* LR jobs are emitted in the exec IOCTL */ - q->ring_ops->emit_job (job); + q->ring_ops->emit_job (job);submit_exec_queue (q); } -if (lr) { -xe_sched_job_set_error (job, -EOPNOTSUPP ); -dma_fence_put (job->fence);/* Drop ref from xe_sched_job_arm */ - }else { - fence = job->fence; - } +/* +* We don't care about job-fence ordering in LR VMs because these fences +* are never exported; they are used solely to keep jobs on the pending +* list. Once a queue enters an error state, there's no need to track +* them. +*/ +if (killed_or_banned_or_wedged && lr) +xe_sched_job_set_error (job, -ECANCELED ); -return fence; +return job->fence; }static void guc_exec_queue_free_job (struct drm_sched_job *drm_job)@@ -916,7 +917,8 @@ static void disable_scheduling_deregister(struct xe_guc *guc,xe_gt_warn (q->gt,"Pending enable/disable failed to respond\n" );xe_sched_submission_start (sched);xe_gt_reset_async (q->gt); -xe_sched_tdr_queue_imm (sched); +if (!xe_exec_queue_is_lr (q)) +xe_sched_tdr_queue_imm (sched);return ; }@@ -1008,6 +1010,7 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)struct xe_exec_queue *q = ge->q;struct xe_guc *guc =exec_queue_to_guc (q);struct xe_gpu_scheduler *sched = &ge->sched; +struct xe_sched_job *job;bool wedged = false;xe_gt_assert (guc_to_gt (guc),xe_exec_queue_is_lr (q));@@ -1058,7 +1061,16 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)if (!exec_queue_killed (q) && !xe_lrc_ring_is_idle (q->lrc[0 ]))xe_devcoredump (q,NULL ,"LR job cleanup, guc_id=%d" , q->guc->id); +xe_hw_fence_irq_stop (q->fence_irq); +xe_sched_submission_start (sched); + +spin_lock (&sched->base.job_list_lock); +list_for_each_entry (job, &sched->base.pending_list, drm.list) +xe_sched_job_set_error (job, -ECANCELED ); +spin_unlock (&sched->base.job_list_lock); + +xe_hw_fence_irq_start (q->fence_irq); }#define ADJUST_FIVE_PERCENT (__t) mul_u64_u32_div(__t, 105, 100)@@ -1129,7 +1141,8 @@ static void enable_scheduling(struct xe_exec_queue *q)xe_gt_warn (guc_to_gt (guc),"Schedule enable failed to respond" );set_exec_queue_banned (q);xe_gt_reset_async (q->gt); -xe_sched_tdr_queue_imm (&q->guc->sched); +if (!xe_exec_queue_is_lr (q)) +xe_sched_tdr_queue_imm (&q->guc->sched); } }
drivers/gpu/drm/xe/xe_guc_submit.c guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · line 1203
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c +1
-0@@ -1187,6 +1200,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)int i =0 ;bool wedged = false, skip_timeout_check; +xe_gt_assert (guc_to_gt (guc), !xe_exec_queue_is_lr (q));
Just some questions around guc_exec_queue_stop/start(). In queue_stop there is:
struct xe_sched_job *job = xe_sched_first_pending_job(sched); bool ban = false;
if (job) {
if ((xe_sched_job_started(job) &&
!xe_sched_job_completed(job)) ||
xe_sched_invalidate_job(job, 2)) {
trace_xe_sched_job_ban(job);
ban = true;
}
} else if (xe_exec_queue_is_lr(q) &&
!xe_lrc_ring_is_idle(q->lrc[0])) {
ban = true;
}Do we still need this else if branch, since the job path is now being taken for lr?
Also I guess first_pending_job() strikes again? If it's pending/complete but we still have something else in-progress in the pending_list they get away clean? Not sure what happens if you skip the ban and get as far as resubmit?
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c +1
-0+/* * TDR has fired before free job worker. Common if exec queue * immediately closed after last fence signaled. Add back to pending
Matthew Brost
maintainerRe: [PATCH v4 07/34] drm/xe: Track LR jobs in DRM scheduler pending list
Quoted 220 lines · Matthew Auld
Quoted 197 lines · Matthew Brost
VF migration requires jobs to remain pending so they can be replayed after the VF comes back. Previously, LR job fences were intentionally signaled immediately after submission to avoid the risk of exporting them, as these fences do not naturally signal in a timely manner and could break dma-fence contracts. A side effect of this approach was that LR jobs were never added to the DRM scheduler’s pending list, preventing them from being tracked for later resubmission.
We now avoid signaling LR job fences and ensure they are never exported; Xe already guards against exporting these internal fences. With that guarantee in place, we can safely track LR jobs in the scheduler’s pending list so they are eligible for resubmission during VF post-migration recovery (and similar recovery paths).
An added benefit is that LR queues now gain the DRM scheduler’s built-in flow control over ring usage rather than rejecting new jobs in the exec IOCTL if the ring is full.
v2:
- Ensure DRM scheduler TDR doesn't run for LR jobs
- Stack variable forkilled_or_banned_or_wedged
v4:
- Clarify commit message (Tomasz)Signed-off-by Matthew Brost <[email protected]>Reviewed-by Tomasz Lis <[email protected]>--- drivers/gpu/drm/xe/xe_exec.c | 12 ++------- drivers/gpu/drm/xe/xe_exec_queue.c | 19 ------------- drivers/gpu/drm/xe/xe_exec_queue.h | 2 -- drivers/gpu/drm/xe/xe_guc_submit.c | 43 ++++++++++++++++++++---------- 4 files changed, 31 insertions(+), 45 deletions(-)drivers/gpu/drm/xe/xe_exec.c +2
-10@@ -124,7 +124,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)struct xe_validation_ctx ctx;struct xe_sched_job *job;struct xe_vm *vm; -bool write_locked, skip_retry = false; +bool write_locked;int err =0 ;struct xe_hw_engine_group *group;enum xe_hw_engine_group_execution_mode mode, previous_mode;@@ -266,12 +266,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file) goto err_exec; } -if (xe_exec_queue_is_lr (q) &&xe_exec_queue_ring_full (q)) { - err = -EWOULDBLOCK ;/* Aliased to -EAGAIN */ - skip_retry = true; - goto err_exec; - } -if (xe_exec_queue_uses_pxp (q)) { err =xe_vm_validate_protected (q->vm);if (err)@@ -328,8 +322,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)xe_sched_job_init_user_fence (job, &syncs[i]); } -if (xe_exec_queue_is_lr (q)) - q->ring_ops->emit_job (job);if (!xe_vm_in_lr_mode (vm))xe_exec_queue_last_fence_set (q, vm, &job->drm.s_fence->finished);xe_sched_job_push (job);@@ -355,7 +347,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)xe_validation_ctx_fini (&ctx); err_unlock_list:up_read (&vm->lock); -if (err == -EAGAIN && !skip_retry) +if (err == -EAGAIN ) goto retry; err_hw_exec_mode:if (mode ==EXEC_MODE_DMA_FENCE )drivers/gpu/drm/xe/xe_exec_queue.c +0
-19@@ -824,25 +824,6 @@ bool xe_exec_queue_is_lr(struct xe_exec_queue *q) !(q->flags &EXEC_QUEUE_FLAG_VM ); } -static s32 xe_exec_queue_num_job_inflight (struct xe_exec_queue *q) -{ -return q->lrc[0 ]->fence_ctx.next_seqno -xe_lrc_seqno (q->lrc[0 ]) -1 ; -} - -/** -* xe_exec_queue_ring_full() - Whether an exec_queue's ring is full -* @q: The exec_queue -* -* Return: True if the exec_queue's ring is full, false otherwise. -*/ -bool xe_exec_queue_ring_full (struct xe_exec_queue *q) -{ -struct xe_lrc *lrc = q->lrc[0 ]; -s32 max_job = lrc->ring.size /MAX_JOB_SIZE_BYTES ; - -return xe_exec_queue_num_job_inflight (q) >= max_job; -} -/** * xe_exec_queue_is_idle() - Whether an exec_queue is idle. * @q: The exec_queue drivers/gpu/drm/xe/xe_exec_queue.h +0
-2@@ -64,8 +64,6 @@ static inline bool xe_exec_queue_uses_pxp(struct xe_exec_queue *q)bool xe_exec_queue_is_lr (struct xe_exec_queue *q); -bool xe_exec_queue_ring_full (struct xe_exec_queue *q); -bool xe_exec_queue_is_idle (struct xe_exec_queue *q);void xe_exec_queue_kill (struct xe_exec_queue *q);drivers/gpu/drm/xe/xe_guc_submit.c +27
-14@@ -851,30 +851,31 @@ guc_exec_queue_run_job(struct drm_sched_job *drm_job)struct xe_sched_job *job =to_xe_sched_job (drm_job);struct xe_exec_queue *q = job->q;struct xe_guc *guc =exec_queue_to_guc (q); -struct dma_fence *fence =NULL ; -bool lr =xe_exec_queue_is_lr (q); +bool lr =xe_exec_queue_is_lr (q), killed_or_banned_or_wedged = +exec_queue_killed_or_banned_or_wedged (q);xe_gt_assert (guc_to_gt (guc), !(exec_queue_destroyed (q) ||exec_queue_pending_disable (q)) ||exec_queue_banned (q) ||exec_queue_suspended (q));trace_xe_sched_job_run (job); -if (!exec_queue_killed_or_banned_or_wedged (q) && !xe_sched_job_is_error (job)) { +if (!killed_or_banned_or_wedged && !xe_sched_job_is_error (job)) {if (!exec_queue_registered (q))register_exec_queue (q,GUC_CONTEXT_NORMAL ); -if (!lr)/* LR jobs are emitted in the exec IOCTL */ - q->ring_ops->emit_job (job); + q->ring_ops->emit_job (job);submit_exec_queue (q); } -if (lr) { -xe_sched_job_set_error (job, -EOPNOTSUPP ); -dma_fence_put (job->fence);/* Drop ref from xe_sched_job_arm */ - }else { - fence = job->fence; - } +/* +* We don't care about job-fence ordering in LR VMs because these fences +* are never exported; they are used solely to keep jobs on the pending +* list. Once a queue enters an error state, there's no need to track +* them. +*/ +if (killed_or_banned_or_wedged && lr) +xe_sched_job_set_error (job, -ECANCELED ); -return fence; +return job->fence; }static void guc_exec_queue_free_job (struct drm_sched_job *drm_job)@@ -916,7 +917,8 @@ static void disable_scheduling_deregister(struct xe_guc *guc,xe_gt_warn (q->gt,"Pending enable/disable failed to respond\n" );xe_sched_submission_start (sched);xe_gt_reset_async (q->gt); -xe_sched_tdr_queue_imm (sched); +if (!xe_exec_queue_is_lr (q)) +xe_sched_tdr_queue_imm (sched);return ; }@@ -1008,6 +1010,7 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)struct xe_exec_queue *q = ge->q;struct xe_guc *guc =exec_queue_to_guc (q);struct xe_gpu_scheduler *sched = &ge->sched; +struct xe_sched_job *job;bool wedged = false;xe_gt_assert (guc_to_gt (guc),xe_exec_queue_is_lr (q));@@ -1058,7 +1061,16 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)if (!exec_queue_killed (q) && !xe_lrc_ring_is_idle (q->lrc[0 ]))xe_devcoredump (q,NULL ,"LR job cleanup, guc_id=%d" , q->guc->id); +xe_hw_fence_irq_stop (q->fence_irq); +xe_sched_submission_start (sched); + +spin_lock (&sched->base.job_list_lock); +list_for_each_entry (job, &sched->base.pending_list, drm.list) +xe_sched_job_set_error (job, -ECANCELED ); +spin_unlock (&sched->base.job_list_lock); + +xe_hw_fence_irq_start (q->fence_irq); }#define ADJUST_FIVE_PERCENT (__t) mul_u64_u32_div(__t, 105, 100)@@ -1129,7 +1141,8 @@ static void enable_scheduling(struct xe_exec_queue *q)xe_gt_warn (guc_to_gt (guc),"Schedule enable failed to respond" );set_exec_queue_banned (q);xe_gt_reset_async (q->gt); -xe_sched_tdr_queue_imm (&q->guc->sched); +if (!xe_exec_queue_is_lr (q)) +xe_sched_tdr_queue_imm (&q->guc->sched); } }drivers/gpu/drm/xe/xe_guc_submit.c guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) · line 1202
Quoted 3 lines
drivers/gpu/drm/xe/xe_guc_submit.c +1
-0@@ -1187,6 +1200,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)int i =0 ;bool wedged = false, skip_timeout_check; +xe_gt_assert (guc_to_gt (guc), !xe_exec_queue_is_lr (q));Just some questions around guc_exec_queue_stop/start(). In
queue_stopthere is:struct
xe_sched_job*job =xe_sched_first_pending_job(sched); bool ban = false;if (job) { if ((xe_sched_job_started(job) && !xe_sched_job_completed(job)) || xe_sched_invalidate_job(job, 2)) { trace_xe_sched_job_ban(job); ban = true; } } else if (xe_exec_queue_is_lr(q) && !xe_lrc_ring_is_idle(q->lrc[0])) { ban = true; }
Quoted 7 lines
Do we still need this else if branch, since the job path is now being taken for lr?
Also I guess
first_pending_job()strikes again? If it's pending/complete but we still have something else in-progress in thepending_listthey get away clean? Not sure what happens if you skip the ban and get as far as resubmit?
I think there needs to be follow on to do a bit of cleanup related to LR jobs (e.g., drop some special casing here, drop some ref counting tricks, and drop the lr_tdr). I have patches locally which do this but I don't really want to expand the scope of this series as it already quite large. I plan on posting those patches once this series merges.
Matt
Quoted 5 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c +1
-0+/* * TDR has fired before free job worker. Common if exec queue * immediately closed after last fence signaled. Add back to pending
Matthew Brost
maintainerPATCH v4 8/34 drm/xe: Don't change LRC ring head on job resubmission
In reply to Matthew BrostNow that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.
v6:
- Also set LRC tail to head so queue is idle coming out of reset
--- drivers/gpu/drm/xe/xe_guc_submit.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_submit.c +8 -2
@@ -2008,11 +2008,17 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)struct xe_gpu_scheduler *sched = &q->guc->sched;if (!exec_queue_killed_or_banned_or_wedged (q)) { +struct xe_sched_job *job =xe_sched_first_pending_job (sched);int i;trace_xe_exec_queue_resubmit (q); -for (i =0 ; i < q->width; ++i) -xe_lrc_set_ring_head (q->lrc[i], q->lrc[i]->ring.tail); +if (job) { +for (i =0 ; i < q->width; ++i) { + q->lrc[i]->ring.tail = job->ptrs[i].head; +xe_lrc_set_ring_tail (q->lrc[i], +xe_lrc_ring_head (q->lrc[i])); + } + }xe_sched_resubmit_jobs (sched); }
Signature · 1 line
2.34.1
Re: [PATCH v4 08/34] drm/xe: Don't change LRC ring head on job resubmission
Quoted 24 lines · Matthew Brost
Now that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.
v6:
- Also set LRC tail to head so queue is idle coming out of resetSigned-off-by Matthew Brost <[email protected]>Reviewed-by Tomasz Lis <[email protected]>--- drivers/gpu/drm/xe/xe_guc_submit.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_guc_submit.c +1
-0@@ -2008,11 +2008,17 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)struct xe_gpu_scheduler *sched = &q->guc->sched;if (!exec_queue_killed_or_banned_or_wedged (q)) { +struct xe_sched_job *job =xe_sched_first_pending_job (sched);
drivers/gpu/drm/xe/xe_guc_submit.c static void guc_exec_queue_start(struct xe_exec_queue *q) · line 2019
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +5
-2int i;trace_xe_exec_queue_resubmit (q); -for (i =0 ; i < q->width; ++i) -xe_lrc_set_ring_head (q->lrc[i], q->lrc[i]->ring.tail); +if (job) { +for (i =0 ; i < q->width; ++i) { + q->lrc[i]->ring.tail = job->ptrs[i].head; +xe_lrc_set_ring_tail (q->lrc[i], +xe_lrc_ring_head (q->lrc[i]));
IIRC the sched pending_list stuff can also give back pending jobs that have completed on the hw, but are still kept pending until the final free_job()?
Suppose we have a pending_list like:
[pending/complete, pending/complete, actual pending kernel job that never completed/ran]
IIUC the sw ring.tail will actually go backwards to the first pending free/complete job head in the pending_list, with the hw tail being reset to the current hw head here. But on the next submit the sw ring.tail is where the commands are emitted to, and on the next update of the hw tail it will be synced to the sw ring.tail? But if that happens won't we get hw tail < hw head (since we used the head of an already complete job for the sw tail), which will make the hw think there is a massive ring wrap, so it will execute garbage until it wraps back around to tail?
Quoted 5 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-0+ } + }xe_sched_resubmit_jobs (sched); }
Matthew Brost
maintainerRe: [PATCH v4 08/34] drm/xe: Don't change LRC ring head on job resubmission
Quoted 42 lines · Matthew Auld
Quoted 22 lines · Matthew Brost
Now that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.
v6:
- Also set LRC tail to head so queue is idle coming out of resetSigned-off-by Matthew Brost <[email protected]>Reviewed-by Tomasz Lis <[email protected]>--- drivers/gpu/drm/xe/xe_guc_submit.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_guc_submit.c +0
-0@@ -2008,11 +2008,17 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)struct xe_gpu_scheduler *sched = &q->guc->sched;if (!exec_queue_killed_or_banned_or_wedged (q)) {drivers/gpu/drm/xe/xe_guc_submit.c static void guc_exec_queue_start(struct xe_exec_queue *q) · line 2017
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +6
-2+struct xe_sched_job *job =xe_sched_first_pending_job (sched);int i;trace_xe_exec_queue_resubmit (q); -for (i =0 ; i < q->width; ++i) -xe_lrc_set_ring_head (q->lrc[i], q->lrc[i]->ring.tail); +if (job) { +for (i =0 ; i < q->width; ++i) { + q->lrc[i]->ring.tail = job->ptrs[i].head; +xe_lrc_set_ring_tail (q->lrc[i], +xe_lrc_ring_head (q->lrc[i]));IIRC the sched
pending_liststuff can also give back pending jobs that have completed on the hw, but are still kept pending until the finalfree_job()?Suppose we have a
pending_listlike:[pending/complete, pending/complete, actual pending kernel job that never completed/ran]
Quoted 10 lines
IIUC the sw ring.tail will actually go backwards to the first pending free/complete job head in the
pending_list, with the hw tail being reset to the current hw head here. But on the next submit the sw ring.tail is where the commands are emitted to, and on the next update
of the hw tail it will be synced to the sw ring.tail? But if that happens won't we get hw tail < hw head (since we used the head of an already complete job for the sw tail), which will make the hw think there is a massive ring wrap, so it will execute garbage until it wraps back around to tail?
Let me tweak this flow to use the skip_emit / last_replay flow introduced later in series to avoid this issue.
Matt
Quoted 5 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-0+ } + }xe_sched_resubmit_jobs (sched); }
Matthew Brost
maintainerRe: [PATCH v4 08/34] drm/xe: Don't change LRC ring head on job resubmission
Quoted 54 lines · Matthew Brost
Quoted 42 lines · Matthew Auld
Quoted 22 lines · Matthew Brost
Now that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.
v6:
- Also set LRC tail to head so queue is idle coming out of resetSigned-off-by Matthew Brost <[email protected]>Reviewed-by Tomasz Lis <[email protected]>--- drivers/gpu/drm/xe/xe_guc_submit.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_guc_submit.c +0
-0@@ -2008,11 +2008,17 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)struct xe_gpu_scheduler *sched = &q->guc->sched;if (!exec_queue_killed_or_banned_or_wedged (q)) {drivers/gpu/drm/xe/xe_guc_submit.c static void guc_exec_queue_start(struct xe_exec_queue *q) · line 2017
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +6
-2+struct xe_sched_job *job =xe_sched_first_pending_job (sched);int i;trace_xe_exec_queue_resubmit (q); -for (i =0 ; i < q->width; ++i) -xe_lrc_set_ring_head (q->lrc[i], q->lrc[i]->ring.tail); +if (job) { +for (i =0 ; i < q->width; ++i) { + q->lrc[i]->ring.tail = job->ptrs[i].head; +xe_lrc_set_ring_tail (q->lrc[i], +xe_lrc_ring_head (q->lrc[i]));IIRC the sched
pending_liststuff can also give back pending jobs that have completed on the hw, but are still kept pending until the finalfree_job()?Suppose we have a
pending_listlike:[pending/complete, pending/complete, actual pending kernel job that never completed/ran]
Quoted 10 lines
IIUC the sw ring.tail will actually go backwards to the first pending free/complete job head in the
pending_list, with the hw tail being reset to the current hw head here. But on the next submit the sw ring.tail is where the commands are emitted to, and on the next update
of the hw tail it will be synced to the sw ring.tail? But if that happens won't we get hw tail < hw head (since we used the head of an already complete job for the sw tail), which will make the hw think there is a massive ring wrap, so it will execute garbage until it wraps back around to tail?
Quoted 3 lines
Let me tweak this flow to use the
skip_emit/last_replayflow introduced later in series to avoid this issue.
Actually this flow works just fine. The GuC state is completely lost during this flow - the context is not even registered. By the time contect is registered - the LRC head will be at the original postition of the first pending job and LRC tail will be at the end of the first job.
Matt
Quoted 7 lines
Matt
Quoted 5 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-0+ } + }xe_sched_resubmit_jobs (sched); }
Re: [PATCH v4 08/34] drm/xe: Don't change LRC ring head on job resubmission
Quoted 59 lines · Matthew Brost
Quoted 54 lines · Matthew Brost
Quoted 42 lines · Matthew Auld
Quoted 22 lines · Matthew Brost
Now that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.
v6:
- Also set LRC tail to head so queue is idle coming out of resetSigned-off-by Matthew Brost <[email protected]>Reviewed-by Tomasz Lis <[email protected]>--- drivers/gpu/drm/xe/xe_guc_submit.c | 10 ++++++++-- 1 file changed, 8 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_guc_submit.c +0
-0@@ -2008,11 +2008,17 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)struct xe_gpu_scheduler *sched = &q->guc->sched;if (!exec_queue_killed_or_banned_or_wedged (q)) {drivers/gpu/drm/xe/xe_guc_submit.c static void guc_exec_queue_start(struct xe_exec_queue *q) · line 2017
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +6
-2+struct xe_sched_job *job =xe_sched_first_pending_job (sched);int i;trace_xe_exec_queue_resubmit (q); -for (i =0 ; i < q->width; ++i) -xe_lrc_set_ring_head (q->lrc[i], q->lrc[i]->ring.tail); +if (job) { +for (i =0 ; i < q->width; ++i) { + q->lrc[i]->ring.tail = job->ptrs[i].head; +xe_lrc_set_ring_tail (q->lrc[i], +xe_lrc_ring_head (q->lrc[i]));IIRC the sched
pending_liststuff can also give back pending jobs that have completed on the hw, but are still kept pending until the finalfree_job()?Suppose we have a
pending_listlike:[pending/complete, pending/complete, actual pending kernel job that never completed/ran]
Quoted 10 lines
IIUC the sw ring.tail will actually go backwards to the first pending free/complete job head in the
pending_list, with the hw tail being reset to the current hw head here. But on the next submit the sw ring.tail is where the commands are emitted to, and on the next update
of the hw tail it will be synced to the sw ring.tail? But if that happens won't we get hw tail < hw head (since we used the head of an already complete job for the sw tail), which will make the hw think there is a massive ring wrap, so it will execute garbage until it wraps back around to tail?Quoted 3 lines
Let me tweak this flow to use the
skip_emit/last_replayflow introduced later in series to avoid this issue.
Quoted 5 lines
Actually this flow works just fine. The GuC state is completely lost during this flow - the context is not even registered. By the time contect is registered - the LRC head will be at the original postition of the first pending job and LRC tail will be at the end of the first job.
Can you share some more info here on the flow? I'm seeing LRC hw head being at the correct position, which must have moved past anything already completed by the hw, right? But here the sw lrc tail is being potentially moved backwards to the head of something already complete (the job we pick is pending but only because the free_job() has not run yet, so the job has already signalled/ran). Below when we do something like xe_sched_resubmit_jobs() the hw tail is then updated to the sw tail, but now we end up with hw tail < hw head?
Also the flow I'm thinking about here is forced suspend/resume.
Quoted 10 lines
Matt
Quoted 7 lines
Matt
Quoted 5 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-0+ } + }xe_sched_resubmit_jobs (sched); }
Matthew Brost
maintainerPATCH v4 9/34 drm/xe: Make LRC W/A scratch buffer usage consistent
In reply to Matthew BrostThe LRC W/A currently checks for LRC being iomem in some places, while in others it checks if the scratch buffer is non-NULL. This inconsistency causes issues with the VF post-migration recovery code, which blindly passes in a scratch buffer.
This patch standardizes the check by consistently verifying whether the LRC is iomem to determine if the scratch buffer should be used.
--- drivers/gpu/drm/xe/xe_lrc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_lrc.c +1 -1
@@ -1248,7 +1248,7 @@ static int setup_bo(struct bo_setup_state *state)static void finish_bo (struct bo_setup_state *state) { -if (!state->buffer) +if (!state->lrc->bo->vmap.is_iomem)return ; xe_map_memcpy_to(gt_to_xe(state->lrc->gt), &state->lrc->bo->vmap,
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 10/34 drm/xe/guc: Document GuC submission backend
In reply to Matthew BrostAdd kernel-doc to xe_guc_submit.c describing the submission path, the per-queue single-threaded model with pause/resume, the driver shadow state machine and lost-H2G replay, job timeout handling, recovery flows (GT reset, PM resume, VF resume), and reclaim constraints.
v2:
- Mirror tweaks for clarity
- Add new doc to Xe rst files
v3:
- Clarify global vs per-queue stop / start
- Clarify VF resume flow
- Add section for 'Waiters during VF resume'
- Add section for 'Page-faulting queues during VF migration'
- Add section for 'GuC-ID assignment'
- Add section for 'Reference counting and final queue destruction'
v4:
- s/VF resume/VF post migration recovery (Tomasz)
--- Documentation/gpu/xe/index.rst | 1 + drivers/gpu/drm/xe/xe_guc_submit.c | 282 +++++++++++++++++++++++++++++ 2 files changed, 283 insertions(+)
Documentation/gpu/xe/index.rst +1 -0
@@ -28,3 +28,4 @@ DG2, etc is provided to prototype the driver. xe_device xe-drm-usage-stats.rst xe_configfs + xe_guc_submit
drivers/gpu/drm/xe/xe_guc_submit.c +282 -0
288 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH v4 10/34] drm/xe/guc: Document GuC submission backend
Repeating on request as looks like it got lost in v3:
We've agreed on separating this patch and merging with a different series.
-Tomasz
Quoted 326 lines · Matthew Brost
Add kernel-doc to
xe_guc_submit.cdescribing the submission path, the per-queue single-threaded model with pause/resume, the driver shadow state machine and lost-H2G replay, job timeout handling, recovery flows (GT reset, PM resume, VF resume), and reclaim constraints.v2:
- Mirror tweaks for clarity
- Add new doc to Xe rst files
v3:
- Clarify global vs per-queue stop / start
- Clarify VF resume flow
- Add section for 'Waiters during VF resume'
- Add section for 'Page-faulting queues during VF migration'
- Add section for 'GuC-ID assignment'
- Add section for 'Reference counting and final queue destruction'
v4:
- s/VF resume/VF post migration recovery (Tomasz)Signed-off-by Matthew Brost <[email protected]>--- Documentation/gpu/xe/index.rst | 1 + drivers/gpu/drm/xe/xe_guc_submit.c | 282 +++++++++++++++++++++++++++++ 2 files changed, 283 insertions(+)Documentation/gpu/xe/index.rst +1
-0@@ -28,3 +28,4 @@ DG2, etc is provided to prototype the driver. xe_device xe-drm-usage-stats.rst xe_configfs + xe_guc_submitdrivers/gpu/drm/xe/xe_guc_submit.c +282
-0@@ -46,6 +46,288 @@#include "xe_trace.h" #include "xe_vm.h" +/* +* DOC: Overview +* +* The GuC submission backend is responsible for submitting GPU jobs to the GuC +* firmware, assigning per-queue GuC IDs, tracking submission state via a +* driver-side state machine, handling GuC-to-host (G2H) messages, tracking +* outstanding jobs, managing job timeouts and queue teardown, and providing +* recovery when GuC state is lost. It is built on top of the DRM scheduler +* (drm_sched). +* +* GuC ID assignment: +* ------------------ +* Each queue is assigned a unique GuC ID at queue init. The ID is used in all +* H2G/G2H to identify the queue and remains reserved until final destruction, +* when the GuC is known to hold no references to it. +* +* The backend maintains a reverse map GuC-ID -> queue to resolve targets for +* G2H handlers and to iterate all queues when required (e.g., recovery). This +* map is protected by submission_state.lock, a global (per-GT) lock. Lockless +* lookups are acceptable in paths where the queue’s lifetime is otherwise +* pinned and it cannot disappear underneath the operation (e.g., G2H handlers). +* +* Basic submission flow +* --------------------- +* Submission is driven by the DRM scheduler vfunc ->run_job(). The flow is: +* +* 1) Emit the job's ring instructions. +* 2) Advance the LRC ring tail: +* - width == 1: simple memory write, +* - width > 1: append a GuC workqueue (WQ) item. +* 3) If the queue is unregistered, issue a register H2G for the context. +* 4) Trigger execution via a scheduler enable or context submit command. +* 5) Return the job's hardware fence to the DRM scheduler. +* +* Registration, scheduler enable, and submit commands are issued as host-to-GuC +* (H2G) messages over the Command Transport (CT) layer, like all GuC +* interactions. +* +* Completion path +* --------------- +* When the job's hardware fence signals, the DRM scheduler vfunc ->free_job() +* is called; it drops the job's reference, typically freeing it. +* +* Control-plane messages: +* ----------------------- +* GuC submission scheduler messages form the control plane for queue cleanup, +* toggling runnability, and modifying queue properties (e.g., scheduler +* priority, timeslice, preemption timeout). Messages are initiated via queue +* vfuncs that append a control message to the queue. They are processed on the +* same single-threaded DRM scheduler workqueue that runs ->run_job() and +* ->free_job(). +* +* Lockless model: +* --------------- +* ->run_job(), ->free_job(), and the message handlers execute as work items on +* a single-threaded DRM scheduler workqueue. Per queue, this provides built-in +* mutual exclusion: only one of these items can run at a time. As a result, +* these paths are lockless with respect to per-queue state tracking. (Global +* or cross-queue data structures still use their own synchronization.) +* +* Stopping / starting: +* -------------------- +* The submission backend supports two scopes of quiesce control: +* +* - Per-queue stop/start: +* The single-threaded DRM scheduler workqueue for a specific queue can be +* stopped and started dynamically. Stopping synchronously quiesces that +* queue's worker (lets any in-flight item finish and prevents new items from +* starting), yielding a stable snapshot while an external operation (e.g., +* job timeout handling) inspects/updates state and performs any required +* fixups. While stopped, no submission, message, or ->free_job() work runs +* for that queue. When the operation completes, the queue is started; any +* pending items are then processed in order on the same worker. Other queues +* continue to run unaffected. +* +* - Global (per-GT) stop/start: +* Implemented on top of the per-queue stop/start primitive: the driver +* stops (or starts) each queue on the GT to obtain a device-wide stable +* snapshot. This is used by coordinated recovery flows (GT reset, PM resume, +* VF post migration recovery). Queues created while the global stop is in +* effect (i.e., future queues) initialize in the stopped state and remain +* stopped until the global start. After recovery fixups are complete, a +* global start iterates queues to start all eligible ones and resumes normal +* submission. +* +* State machine: +* -------------- +* The submission state machine is the driver's shadow of the GuC-visible queue +* state (e.g., registered, runnable, scheduler properties). It tracks the +* transitions we intend to make (issued as H2G commands), marking them pending +* until acknowledged via G2H or otherwise observed as applied. It also records +* the origin of each transition (->run_job(), timeout handler, explicit control +* message, etc.). +* +* Because H2G commands and/or GuC submission state can be lost across GT reset, +* PM resume, or VF post migration recovery, this bookkeeping lets recovery +* decide which operations to replay, which to elide, and which need fixups, +* restoring a consistent queue state without additional per-queue locks. +* +* Job timeouts: +* ------------- +* To prevent jobs from running indefinitely and violating dma-fence signaling +* rules, the DRM scheduler tracks how long each job has been running. If a +* threshold is exceeded, it calls ->timeout_job(). +* +* ->timeout_job() stops the queue, samples the LRC context timestamps to +* confirm the job actually started and has exceeded the allowed runtime, and +* then, if confirmed, signals all pending jobs' fences and initiates queue +* teardown. Finally, the queue is started. +* +* Job timeout handling runs on a per-GT, single-threaded recovery workqueue +* that is shared with other recovery paths (e.g., GT reset handling, VF +* resume). This guarantees only one recovery action executes at a time. +* +* Queue teardown: +* --------------- +* Teardown can be triggered by: (1) userspace closing the queue; (2) a G2H +* queue-reset notification; (3) a G2H memory_cat_error for the queue; or (4) +* in-flight jobs detected on the queue during GT reset. +* +* In all cases teardown is driven via the timeout path by setting the queue's +* DRM scheduler timeout to zero, forcing an immediate ->timeout_job() pass. +* +* Reference counting and final queue destruction: +* ----------------------------------------------- +* Jobs reference-count the queue; queues hold a reference to the VM. When a +* queue's reference count reaches zero (e.g., all jobs are freed and the +* userspace handle is closed), the queue is not destroyed immediately because +* the GuC may still reference its state. +* +* Instead, a control-plane cleanup message is appended to remove GuC-side +* references (e.g., disable runnability, deregister). Once the final G2H +* confirming that GuC no longer references the queue is eligible for +* destruction. +* +* To avoid freeing the queue from within its own DRM scheduler workqueue (which +* would risk use-after-free), the actual destruction is deferred to a separate +* work item queued on a dedicated destruction workqueue. +* +* GT resets: +* ---------- +* GT resets are triggered by catastrophic errors (e.g., CT channel failure). +* The GuC is reset and all GuC-side submission state is lost. Recovery proceeds +* as follows: +* +* 1) Quiesce: +* - Stop all queues (global submission stop). Per-queue workers finish any +* in-flight item and then stop; newly created queues during the window +* initialize in the stopped state. +* - Abort any waits on CT/G2H to avoid deadlock. +* +* 2) Sanitize driver shadow state: +* - For each queue, clear GuC-derived bits in the submission state machine +* (e.g., registered/enabled) and mark in-flight H2G transitions as lost. +* - Convert/flush any side effects of lost H2G. +* +* 3) Decide teardown vs. replay: +* - If a queue's LRC seqno indicates that a job started but did not +* complete, initiate teardown for that queue via the timeout path. +* - If no job started, keep the queue for replay. +* +* 4) Resume: +* - Start remaining queues; resubmit pending jobs. +* - Queues marked for teardown remain stopped/destroyed. +* +* The entire sequence runs on the per-GT single-threaded recovery worker, +* ensuring only one recovery action executes at a time; a runtime PM reference +* is held for the duration. +* +* PM resume: +* ---------- +* PM resume assumes all GuC state is lost (the device may have been powered +* down). It reuses the GT reset recovery path, but executes in the context of +* the caller that wakes the device (runtime PM or system resume). +* +* Suspend entry: +* - Control-plane message work is quiesced; state toggles that require an +* active device are not enqueued while suspended. +* - Per-queue scheduler workers are stopped before the device is allowed to +* suspend. +* - Barring driver bugs, no queues should have in-flight jobs at +* suspend/resume.. +* +* On resume, run the GT reset recovery flow and then start eligible queues. +* +* Runtime PM and state-change ordering: +* ------------------------------------- +* Runtime/system PM transitions must not race with per-queue submission and +* state updates. +* +* Execution contexts and RPM sources: +* - Scheduler callbacks (->run_job(), ->free_job(), ->timeout_job()): +* executed with an active RPM ref held by the in-flight job. +* - Control-plane message work: +* enqueued from IOCTL paths that already hold an RPM ref; the message path +* itself does not get/put RPM. State toggles are only issued while active. +* During suspend entry, message work is quiesced and no new toggles are +* enqueued until after resume. +* - G2H handlers: +* dispatched with an RPM ref guaranteed by the CT layer. +* - Recovery phases (GT reset/VF post migration recovery): +* explicitly get/put an RPM ref for their duration on the per-GT recovery +* worker. +* +* Consequence: +* - All submission/state mutations run with an RPM reference. The PM core +* cannot enter suspend while these updates are in progress, and resume is +* complete before updates execute. This prevents PM state changes from +* racing with queue state changes. +* +* VF post migration recovery: +* --------------------------- +* VF post migration recovery resembles a GT reset, but GuC submission state is +* expected to persist across migration; in-flight H2G commands may be lost, and +* GGTT base/offsets may change. Recovery proceeds as follows: +* +* 1) Quiesce: +* - Stop all queues and abort waits (as with GT reset) to obtain a stable +* snapshot. +* - Queues created while VF post migration recovery is in-flight initialize +* in the stopped state. +* +* 2) Treat H2G as lost and prepare in-place resubmission (GuC/CT down): +* - Treat in-flight H2G (enable/disable, etc.) as dropped; update shadow +* bits to a safe baseline and tag the ops as "needs replay". +* - Quarantine device-visible submission state: set the GuC-visible LRC ring +* tail equal to the head (and, for WQ-based submission, set the WQ +* descriptor head == tail) so that when the GuC comes up it will not process +* any entries that were built with stale GGTT addresses. +* - Reset the software ring tail to the original value captured at the +* submission of the oldest pending job, so the write pointer sits exactly +* where that job was originally emitted. +* +* 3) Replay and resubmit once GuC/CT is live: +* - VF post migration recovery invokes ->run_job() for pending jobs; +* ->emit_job() overwrites ring instructions in place, fixes GGTT fields, +* then advances the LRC tail (and WQ descriptor for width > 1). Required +* submission H2G(s) are reissued and fresh WQ entries are written. +* - Queue lost control-plane operations (scheduling-state toggles, cleanup) +* in order via the message path. +* - Start the queues to process the queued control-plane operations and run +* the resubmitted jobs. +* +* The goal is to preserve both job and queue state; no teardown is performed +* in this flow. The sequence runs on the per-GT single-threaded recovery +* worker with a held runtime PM reference. +* +* Waiters during VF post migration recovery +* ----------------------------------------- +* The submission backend frequently uses wait_event_timeout() to wait on +* GuC-driven conditions. Across VF migration/recovery two issues arise: +* 1) The timeout does not account for migration downtime and may expire +* prematurely, triggering undesired actions (e.g., GT reset, prematurely +* signaling a fence). +* 2) Some waits target GuC work that cannot complete until VF recovery +* finishes; these typically sit on the queue-stopping path. +* +* To handle this, all waiters must atomically test the "GuC down / VF-recovery +* in progress" condition (e.g., VF_RESFIX_BLOCKED) both before sleeping and +* after wakeup. The flag is coherent with VF migration: vCPUs observe it +* immediately on unhalt, and it is cleared only after the GuC/CT is live again. +* If set, the waiter must either (a) abort the wait without side effects, or +* (b) re-arm the wait with a fresh timeout once the GuC/CT is live. Timeouts +* that occur while GuC/CT are down are non-fatal—the VF-recovery path will +* rebuild state—and must not trigger recovery or teardown. +* +* Relation to reclaim: +* -------------------- +* Jobs signal dma-fences, and the MM may wait on those fences during reclaim. +* As a consequence, the entire GuC submission backend (DRM scheduler callbacks, +* message handling, and all recovery paths) lies on the reclaim path and must +* be reclaim-safe. +* +* Practical implications: +* - No memory allocations in these paths (avoid any allocation that could +* recurse into reclaim or sleep). +* - The global submission-state lock may be taken from reclaim-tainted contexts +* (timeout/recovery). Any path that acquires it (including queue init/destroy) +* must not allocate or take locks that can recurse into reclaim while holding +* it; keep the critical section to state/xarray updates. +*/ +static struct xe_guc *exec_queue_to_guc (struct xe_exec_queue *q) {
Matthew Brost
maintainerPATCH v4 11/34 drm/xe/vf: Add xe_gt_recovery_inprogress helper
In reply to Matthew BrostAdd xe_gt_recovery_inprogress helper.
This helper serves as the singular point to determine whether a GT recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.
v3:
- Add GT layer xe_gt_recovery_inprogress (Michal)
- Don't blow up in memirq not enabled (CI)
- Add __memirq_received with clear argument (Michal)
- xe_memirq_sw_int_0_irq_pending rename (Michal)
- Use offset in xe_memirq_sw_int_0_irq_pending (Michal)
v4:
- Refactor xe_gt_recovery_inprogress logic around memirq (Michal)
--- drivers/gpu/drm/xe/xe_gt.h | 13 ++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 22 +++++++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++ drivers/gpu/drm/xe/xe_memirq.c | 48 +++++++++++++++++++++-- drivers/gpu/drm/xe/xe_memirq.h | 2 + 6 files changed, 93 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_gt.h +13 -0
@@ -12,6 +12,7 @@#include "xe_device.h" #include "xe_device_types.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine.h" #define for_each_hw_engine (hwe__, gt__, id__) \
@@ -124,4 +125,16 @@ static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) hwe->instance == gt->usm.reserved_bcs_instance; } +/** +* xe_gt_recovery_inprogress() - GT recovery in progress +* @gt: the &xe_gt +* +* Return: True if GT recovery in progress, False otherwise +*/ +static inline bool xe_gt_recovery_inprogress (struct xe_gt *gt) +{ +return IS_SRIOV_VF (gt_to_xe (gt)) && +xe_gt_sriov_vf_recovery_inprogress (gt); +} +#endif
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +22 -0
@@ -26,6 +26,7 @@#include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_lrc.h" +#include "xe_memirq.h" #include "xe_mmio.h" #include "xe_sriov.h" #include "xe_sriov_vf.h"
@@ -776,6 +777,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)struct xe_device *xe =gt_to_xe (gt);xe_gt_assert (gt,IS_SRIOV_VF (xe)); +xe_gt_assert (gt,xe_gt_sriov_vf_recovery_inprogress (gt));set_bit (gt->info.id, &xe->sriov.vf.migration.gt_flags);/*
@@ -1118,3 +1120,23 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)drm_printf (p,"\thandshake:\t%u.%u\n" , pf_version->major, pf_version->minor); } + +/** +* xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress +* @gt: the &xe_gt +* +* Return: True if VF post migration recovery in progress, False otherwise +*/ +bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +/* early detection until recovery starts */ +if (xe_device_uses_memirq (gt_to_xe (gt)) && +xe_memirq_sw_int_0_irq_pending (memirq, >->uc.guc)) +return true; + +return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); +}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt); +u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10 -0
@@ -46,6 +46,14 @@ struct xe_gt_sriov_vf_runtime { } *regs; }; +/** +* xe_gt_sriov_vf_migration - VF migration data. +*/ +struct xe_gt_sriov_vf_migration { +/** @recovery_inprogress: VF post migration recovery in progress */ +bool recovery_inprogress; +}; +/** * struct xe_gt_sriov_vf - GT level VF virtualization data. */
@@ -58,6 +66,8 @@ struct xe_gt_sriov_vf {struct xe_gt_sriov_vf_selfconfig self_config;/** @runtime: runtime data retrieved from the PF. */ struct xe_gt_sriov_vf_runtime runtime; +/** @migration: migration data for the VF. */ + struct xe_gt_sriov_vf_migration migration; };#endif
drivers/gpu/drm/xe/xe_memirq.c +44 -4
78 lines not on this page · raw message
drivers/gpu/drm/xe/xe_memirq.h +2 -0
@@ -25,4 +25,6 @@ void xe_memirq_handler(struct xe_memirq *memirq);int xe_memirq_init_guc (struct xe_memirq *memirq,struct xe_guc *guc); +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc); +#endif
Signature · 1 line
2.34.1
Re: [PATCH v4 11/34] drm/xe/vf: Add xe_gt_recovery_inprogress helper
Quoted 39 lines · Matthew Brost
Add
xe_gt_recovery_inprogresshelper.This helper serves as the singular point to determine whether a GT recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.
v3:
- Add GT layerxe_gt_recovery_inprogress(Michal)
- Don't blow up in memirq not enabled (CI)
- Add__memirq_receivedwith clear argument (Michal)
-xe_memirq_sw_int_0_irq_pendingrename (Michal)
- Use offset inxe_memirq_sw_int_0_irq_pending(Michal)
v4:
- Refactorxe_gt_recovery_inprogresslogic around memirq (Michal)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt.h | 13 ++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 22 +++++++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++ drivers/gpu/drm/xe/xe_memirq.c | 48 +++++++++++++++++++++-- drivers/gpu/drm/xe/xe_memirq.h | 2 + 6 files changed, 93 insertions(+), 4 deletions(-)drivers/gpu/drm/xe/xe_gt.h +1
-0@@ -12,6 +12,7 @@#include "xe_device.h" #include "xe_device_types.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine.h" #define for_each_hw_engine (hwe__, gt__, id__) \
drivers/gpu/drm/xe/xe_gt.h static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) · line 132
Quoted 8 lines
drivers/gpu/drm/xe/xe_gt.h +5
-0@@ -124,4 +125,16 @@ static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) hwe->instance == gt->usm.reserved_bcs_instance; } +/** +* xe_gt_recovery_inprogress() - GT recovery in progress +* @gt: the &xe_gt +* +* Return: True if GT recovery in progress, False otherwise
``` True if migration recovery is pending or in progress ```
Both the messages we log and spec clearly establish that migration recovery is the queued work, not the whole period after vCPU starts.
Quoted 21 lines
drivers/gpu/drm/xe/xe_gt.h +7
-0+*/ +static inline bool xe_gt_recovery_inprogress (struct xe_gt *gt) +{ +return IS_SRIOV_VF (gt_to_xe (gt)) && +xe_gt_sriov_vf_recovery_inprogress (gt); +} +#endif drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0@@ -26,6 +26,7 @@#include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_lrc.h" +#include "xe_memirq.h" #include "xe_mmio.h" #include "xe_sriov.h" #include "xe_sriov_vf.h"
drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt) · line 780
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0@@ -776,6 +777,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)struct xe_device *xe =gt_to_xe (gt);xe_gt_assert (gt,IS_SRIOV_VF (xe)); +xe_gt_assert (gt,xe_gt_sriov_vf_recovery_inprogress (gt));
If the handler was called, then this will always pass. It does not fully verify the solution, as it doesn't mean the MEMIRQ was already set on the first vCPU cycle, which the patches rely on.
It only verifies that:
* the IRQ type is MEMIRQ
* the IRQ handler does not clear bytes which indicate source of the request before calling the handler
So this assert is ok to stay, but requirements on the IRQ raising moment are not established by it and need to be explicitly stated. Currently this is only mentioned in commit message, and not as requirement but rather.. a fact? Hard to tell even, as the sentence lacks pointing a subject.
I think we need this plainly stated, and not only in commit message but also in the sources or kerneldoc.
I don't think it would be a problem to expand the solution to IRQs with source in MMIO, we could expand the function to check MMIO bits if any platform requires that.
I also now think the requirement on IRQ bytes being set from first vCPU instruction is reasonable, meaning all VMM systems will meet it because it is required to finish VM memory restore of all types before GuC restore, and GuC needs to drive the HW to trigger the IRQ during its state load, it cannot defer it. (btw the MEMIRQ buffer seem to be in sys mem, not in vram as I previously thought)
So not sure if Michal will agree, but my perspective is: this is ok, and just needs comments to emphasize the "IRQ source data visible from the first vCPU cycle" requirement.
-Tomasz
Quoted 165 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +20
-0set_bit (gt->info.id, &xe->sriov.vf.migration.gt_flags);/* @@ -1118,3 +1120,23 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)drm_printf (p,"\thandshake:\t%u.%u\n" , pf_version->major, pf_version->minor); } + +/** +* xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress +* @gt: the &xe_gt +* +* Return: True if VF post migration recovery in progress, False otherwise +*/ +bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +/* early detection until recovery starts */ +if (xe_device_uses_memirq (gt_to_xe (gt)) && +xe_memirq_sw_int_0_irq_pending (memirq, >->uc.guc)) +return true; + +return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); +}drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2
-0@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt); +u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt);drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10
-0@@ -46,6 +46,14 @@ struct xe_gt_sriov_vf_runtime { } *regs; }; +/** +* xe_gt_sriov_vf_migration - VF migration data. +*/ +struct xe_gt_sriov_vf_migration { +/** @recovery_inprogress: VF post migration recovery in progress */ +bool recovery_inprogress; +}; +/** * struct xe_gt_sriov_vf - GT level VF virtualization data. */ @@ -58,6 +66,8 @@ struct xe_gt_sriov_vf {struct xe_gt_sriov_vf_selfconfig self_config;/** @runtime: runtime data retrieved from the PF. */ struct xe_gt_sriov_vf_runtime runtime; +/** @migration: migration data for the VF. */ + struct xe_gt_sriov_vf_migration migration; };#endif drivers/gpu/drm/xe/xe_memirq.c +44
-4@@ -398,8 +398,9 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)memirq_set_enable (memirq, true); } -static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, -u16 offset,const char *name) +static bool __memirq_received (struct xe_memirq *memirq, +struct iosys_map *vector,u16 offset, +const char *name,bool clear) {u8 value;@@ -409,12 +410,26 @@ static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,memirq_err_ratelimited (memirq,"Unexpected memirq value %#x from %s at %u\n" , value, name, offset); -iosys_map_wr (vector, offset, u8,0x00 ); +if (clear) +iosys_map_wr (vector, offset, u8,0x00 ); }return value; } +static bool memirq_received_noclear (struct xe_memirq *memirq, +struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, false); +} + +static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, true); +} +static void memirq_dispatch_engine (struct xe_memirq *memirq,struct iosys_map *status,struct xe_hw_engine *hwe) {@@ -434,8 +449,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *statif (memirq_received (memirq, status,ilog2 (GUC_INTR_GUC2HOST ), name))xe_guc_irq_handler (guc,GUC_INTR_GUC2HOST ); -if (memirq_received (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), name)) +/* +* We must wait to perform the clear operation until after +* xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race +* conditions where xe_gt_sriov_vf_recovery_inprogress() returns false. +*/ +if (memirq_received_noclear (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), + name)) {xe_guc_irq_handler (guc,GUC_INTR_SW_INT_0 ); +iosys_map_wr (status,ilog2 (GUC_INTR_SW_INT_0 ), u8,0x00 ); + } }/** @@ -460,6 +483,23 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) } } +/** +* xe_memirq_sw_int_0_irq_pending() - SW_INT_0 IRQ is pending +* @memirq: the &xe_memirq +* @guc: the &xe_guc to check for IRQ +* +* Return: True if SW_INT_0 IRQ is pending on @guc, False otherwise +*/ +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc) +{ +struct xe_gt *gt =guc_to_gt (guc); +u32 offset =xe_gt_is_media_type (gt) ?ilog2 (INTR_MGUC ) :ilog2 (INTR_GUC ); +struct iosys_map map =IOSYS_MAP_INIT_OFFSET (&memirq->status, offset *SZ_16 ); + +return memirq_received_noclear (memirq, &map,ilog2 (GUC_INTR_SW_INT_0 ), +guc_name (guc)); +} +/** * xe_memirq_handler - The `Memory Based Interrupts`_ Handler. * @memirq: the &xe_memirq drivers/gpu/drm/xe/xe_memirq.h +2
-0@@ -25,4 +25,6 @@ void xe_memirq_handler(struct xe_memirq *memirq);int xe_memirq_init_guc (struct xe_memirq *memirq,struct xe_guc *guc); +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc); +#endif
Matthew Brost
maintainerRe: [PATCH v4 11/34] drm/xe/vf: Add xe_gt_recovery_inprogress helper
Quoted 107 lines · Tomasz
Quoted 37 lines · Matthew Brost
Add
xe_gt_recovery_inprogresshelper.This helper serves as the singular point to determine whether a GT recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.
v3:
- Add GT layerxe_gt_recovery_inprogress(Michal)
- Don't blow up in memirq not enabled (CI)
- Add__memirq_receivedwith clear argument (Michal)
-xe_memirq_sw_int_0_irq_pendingrename (Michal)
- Use offset inxe_memirq_sw_int_0_irq_pending(Michal)
v4:
- Refactorxe_gt_recovery_inprogresslogic around memirq (Michal)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt.h | 13 ++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 22 +++++++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++ drivers/gpu/drm/xe/xe_memirq.c | 48 +++++++++++++++++++++-- drivers/gpu/drm/xe/xe_memirq.h | 2 + 6 files changed, 93 insertions(+), 4 deletions(-)drivers/gpu/drm/xe/xe_gt.h +1
-0@@ -12,6 +12,7 @@#include "xe_device.h" #include "xe_device_types.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine.h" #define for_each_hw_engine (hwe__, gt__, id__) \drivers/gpu/drm/xe/xe_gt.h static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) · line 131
Quoted 7 lines
drivers/gpu/drm/xe/xe_gt.h +5
-0@@ -124,4 +125,16 @@ static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) hwe->instance == gt->usm.reserved_bcs_instance; } +/** +* xe_gt_recovery_inprogress() - GT recovery in progress +* @gt: the &xe_gt +* +* Return: True if GT recovery in progress, False otherwise ``` True if migration recovery is pending or in progress ```Both the messages we log and spec clearly establish that migration recovery is the queued work, not the whole period after vCPU starts.
Quoted 21 lines
drivers/gpu/drm/xe/xe_gt.h +7
-0+*/ +static inline bool xe_gt_recovery_inprogress (struct xe_gt *gt) +{ +return IS_SRIOV_VF (gt_to_xe (gt)) && +xe_gt_sriov_vf_recovery_inprogress (gt); +} +#endif drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0@@ -26,6 +26,7 @@#include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_lrc.h" +#include "xe_memirq.h" #include "xe_mmio.h" #include "xe_sriov.h" #include "xe_sriov_vf.h" drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt) · line 779
Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0@@ -776,6 +777,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)struct xe_device *xe =gt_to_xe (gt);xe_gt_assert (gt,IS_SRIOV_VF (xe)); +xe_gt_assert (gt,xe_gt_sriov_vf_recovery_inprogress (gt));If the handler was called, then this will always pass. It does not fully verify the solution, as it doesn't mean the MEMIRQ was already set on the first vCPU cycle, which the patches rely on.
It only verifies that:
* the IRQ type is MEMIRQ
* the IRQ handler does not clear bytes which indicate source of the request before calling the handler
So this assert is ok to stay, but requirements on the IRQ raising moment are not established by it and need to be explicitly stated. Currently this is only mentioned in commit message, and not as requirement but rather.. a fact? Hard to tell even, as the sentence lacks pointing a subject.
I think we need this plainly stated, and not only in commit message but also in the sources or kerneldoc.
I don't think it would be a problem to expand the solution to IRQs with source in MMIO, we could expand the function to check MMIO bits if any platform requires that.
I also now think the requirement on IRQ bytes being set from first vCPU instruction is reasonable, meaning all VMM systems will meet it because it is required to finish VM memory restore of all types before GuC restore, and GuC needs to drive the HW to trigger the IRQ during its state load, it cannot defer it. (btw the MEMIRQ buffer seem to be in sys mem, not in vram as I previously thought)
Quoted 3 lines
So not sure if Michal will agree, but my perspective is: this is ok, and just needs comments to emphasize the "IRQ source data visible from the first vCPU cycle" requirement.
Will do.
Matt
Quoted 155 lines
-Tomasz
Quoted 152 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +20
-0set_bit (gt->info.id, &xe->sriov.vf.migration.gt_flags);/* @@ -1118,3 +1120,23 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)drm_printf (p,"\thandshake:\t%u.%u\n" , pf_version->major, pf_version->minor); } + +/** +* xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress +* @gt: the &xe_gt +* +* Return: True if VF post migration recovery in progress, False otherwise +*/ +bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +/* early detection until recovery starts */ +if (xe_device_uses_memirq (gt_to_xe (gt)) && +xe_memirq_sw_int_0_irq_pending (memirq, >->uc.guc)) +return true; + +return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); +}drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2
-0@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt); +u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt);drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10
-0@@ -46,6 +46,14 @@ struct xe_gt_sriov_vf_runtime { } *regs; }; +/** +* xe_gt_sriov_vf_migration - VF migration data. +*/ +struct xe_gt_sriov_vf_migration { +/** @recovery_inprogress: VF post migration recovery in progress */ +bool recovery_inprogress; +}; +/** * struct xe_gt_sriov_vf - GT level VF virtualization data. */ @@ -58,6 +66,8 @@ struct xe_gt_sriov_vf {struct xe_gt_sriov_vf_selfconfig self_config;/** @runtime: runtime data retrieved from the PF. */ struct xe_gt_sriov_vf_runtime runtime; +/** @migration: migration data for the VF. */ + struct xe_gt_sriov_vf_migration migration; };#endif drivers/gpu/drm/xe/xe_memirq.c +44
-4@@ -398,8 +398,9 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)memirq_set_enable (memirq, true); } -static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, -u16 offset,const char *name) +static bool __memirq_received (struct xe_memirq *memirq, +struct iosys_map *vector,u16 offset, +const char *name,bool clear) {u8 value;@@ -409,12 +410,26 @@ static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,memirq_err_ratelimited (memirq,"Unexpected memirq value %#x from %s at %u\n" , value, name, offset); -iosys_map_wr (vector, offset, u8,0x00 ); +if (clear) +iosys_map_wr (vector, offset, u8,0x00 ); }return value; } +static bool memirq_received_noclear (struct xe_memirq *memirq, +struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, false); +} + +static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, true); +} +static void memirq_dispatch_engine (struct xe_memirq *memirq,struct iosys_map *status,struct xe_hw_engine *hwe) {@@ -434,8 +449,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *statif (memirq_received (memirq, status,ilog2 (GUC_INTR_GUC2HOST ), name))xe_guc_irq_handler (guc,GUC_INTR_GUC2HOST ); -if (memirq_received (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), name)) +/* +* We must wait to perform the clear operation until after +* xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race +* conditions where xe_gt_sriov_vf_recovery_inprogress() returns false. +*/ +if (memirq_received_noclear (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), + name)) {xe_guc_irq_handler (guc,GUC_INTR_SW_INT_0 ); +iosys_map_wr (status,ilog2 (GUC_INTR_SW_INT_0 ), u8,0x00 ); + } }/** @@ -460,6 +483,23 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) } } +/** +* xe_memirq_sw_int_0_irq_pending() - SW_INT_0 IRQ is pending +* @memirq: the &xe_memirq +* @guc: the &xe_guc to check for IRQ +* +* Return: True if SW_INT_0 IRQ is pending on @guc, False otherwise +*/ +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc) +{ +struct xe_gt *gt =guc_to_gt (guc); +u32 offset =xe_gt_is_media_type (gt) ?ilog2 (INTR_MGUC ) :ilog2 (INTR_GUC ); +struct iosys_map map =IOSYS_MAP_INIT_OFFSET (&memirq->status, offset *SZ_16 ); + +return memirq_received_noclear (memirq, &map,ilog2 (GUC_INTR_SW_INT_0 ), +guc_name (guc)); +} +/** * xe_memirq_handler - The `Memory Based Interrupts`_ Handler. * @memirq: the &xe_memirq drivers/gpu/drm/xe/xe_memirq.h +2
-0@@ -25,4 +25,6 @@ void xe_memirq_handler(struct xe_memirq *memirq);int xe_memirq_init_guc (struct xe_memirq *memirq,struct xe_guc *guc); +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc); +#endif
Re: [PATCH v4 11/34] drm/xe/vf: Add xe_gt_recovery_inprogress helper
In reply to Matthew BrostQuoted 76 lines · Matthew Brost
Add
xe_gt_recovery_inprogresshelper.This helper serves as the singular point to determine whether a GT recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.
v3:
- Add GT layerxe_gt_recovery_inprogress(Michal)
- Don't blow up in memirq not enabled (CI)
- Add__memirq_receivedwith clear argument (Michal)
-xe_memirq_sw_int_0_irq_pendingrename (Michal)
- Use offset inxe_memirq_sw_int_0_irq_pending(Michal)
v4:
- Refactorxe_gt_recovery_inprogresslogic around memirq (Michal)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt.h | 13 ++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 22 +++++++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++ drivers/gpu/drm/xe/xe_memirq.c | 48 +++++++++++++++++++++-- drivers/gpu/drm/xe/xe_memirq.h | 2 + 6 files changed, 93 insertions(+), 4 deletions(-)drivers/gpu/drm/xe/xe_gt.h +13
-0@@ -12,6 +12,7 @@#include "xe_device.h" #include "xe_device_types.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine.h" #define for_each_hw_engine (hwe__, gt__, id__) \@@ -124,4 +125,16 @@ static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) hwe->instance == gt->usm.reserved_bcs_instance; } +/** +* xe_gt_recovery_inprogress() - GT recovery in progress +* @gt: the &xe_gt +* +* Return: True if GT recovery in progress, False otherwise +*/ +static inline bool xe_gt_recovery_inprogress (struct xe_gt *gt) +{ +return IS_SRIOV_VF (gt_to_xe (gt)) && +xe_gt_sriov_vf_recovery_inprogress (gt); +} +#endif drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-0@@ -26,6 +26,7 @@#include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_lrc.h" +#include "xe_memirq.h" #include "xe_mmio.h" #include "xe_sriov.h" #include "xe_sriov_vf.h" @@ -776,6 +777,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)struct xe_device *xe =gt_to_xe (gt);xe_gt_assert (gt,IS_SRIOV_VF (xe)); +xe_gt_assert (gt,xe_gt_sriov_vf_recovery_inprogress (gt));set_bit (gt->info.id, &xe->sriov.vf.migration.gt_flags);/*
drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1129
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7
-0@@ -1118,3 +1120,23 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)drm_printf (p,"\thandshake:\t%u.%u\n" , pf_version->major, pf_version->minor); } + +/** +* xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress +* @gt: the &xe_gt +* +* Return: True if VF post migration recovery in progress, False otherwise +*/
hmm, I'm little concern about the mismatch between function name and it's logic
drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1139
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10
-0+bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +/* early detection until recovery starts */ +if (xe_device_uses_memirq (gt_to_xe (gt)) && +xe_memirq_sw_int_0_irq_pending (memirq, >->uc.guc)) +return true;
as here it may return true even if the recovery didn't start yet
drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1141
Quoted 2 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-0+ +return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress);
and this is actually the flag that indicates that driver actually is recovering
maybe (depends on the actual need) we should either:
* rename this helper to:
xe_gt_sriov_vf_pending_recovery
* or provide two helpers:
xe_gt_sriov_vf_needs_recovery
xe_gt_sriov_vf_recovery_inprogress
Quoted 105 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0+}drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2
-0@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt); +u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt);drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10
-0@@ -46,6 +46,14 @@ struct xe_gt_sriov_vf_runtime { } *regs; }; +/** +* xe_gt_sriov_vf_migration - VF migration data. +*/ +struct xe_gt_sriov_vf_migration { +/** @recovery_inprogress: VF post migration recovery in progress */ +bool recovery_inprogress; +}; +/** * struct xe_gt_sriov_vf - GT level VF virtualization data. */ @@ -58,6 +66,8 @@ struct xe_gt_sriov_vf {struct xe_gt_sriov_vf_selfconfig self_config;/** @runtime: runtime data retrieved from the PF. */ struct xe_gt_sriov_vf_runtime runtime; +/** @migration: migration data for the VF. */ + struct xe_gt_sriov_vf_migration migration; };#endif drivers/gpu/drm/xe/xe_memirq.c +27
-4@@ -398,8 +398,9 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)memirq_set_enable (memirq, true); } -static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, -u16 offset,const char *name) +static bool __memirq_received (struct xe_memirq *memirq, +struct iosys_map *vector,u16 offset, +const char *name,bool clear) {u8 value;@@ -409,12 +410,26 @@ static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,memirq_err_ratelimited (memirq,"Unexpected memirq value %#x from %s at %u\n" , value, name, offset); -iosys_map_wr (vector, offset, u8,0x00 ); +if (clear) +iosys_map_wr (vector, offset, u8,0x00 ); }return value; } +static bool memirq_received_noclear (struct xe_memirq *memirq, +struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, false); +} + +static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, true); +} +static void memirq_dispatch_engine (struct xe_memirq *memirq,struct iosys_map *status,struct xe_hw_engine *hwe) {@@ -434,8 +449,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *statif (memirq_received (memirq, status,ilog2 (GUC_INTR_GUC2HOST ), name))xe_guc_irq_handler (guc,GUC_INTR_GUC2HOST ); -if (memirq_received (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), name)) +/* +* We must wait to perform the clear operation until after +* xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race +* conditions where xe_gt_sriov_vf_recovery_inprogress() returns false. +*/ +if (memirq_received_noclear (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), + name)) {xe_guc_irq_handler (guc,GUC_INTR_SW_INT_0 ); +iosys_map_wr (status,ilog2 (GUC_INTR_SW_INT_0 ), u8,0x00 ); + } }/**
drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 487
Quoted 5 lines
drivers/gpu/drm/xe/xe_memirq.c +2
-0@@ -460,6 +483,23 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) } } +/** +* xe_memirq_sw_int_0_irq_pending() - SW_INT_0 IRQ is pending
maybe we should add "guc" to the function name?
xe_memirq_guc_sw_int_0_irq_pending()
Quoted 29 lines
drivers/gpu/drm/xe/xe_memirq.c +15
-0+* @memirq: the &xe_memirq +* @guc: the &xe_guc to check for IRQ +* +* Return: True if SW_INT_0 IRQ is pending on @guc, False otherwise +*/ +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc) +{ +struct xe_gt *gt =guc_to_gt (guc); +u32 offset =xe_gt_is_media_type (gt) ?ilog2 (INTR_MGUC ) :ilog2 (INTR_GUC ); +struct iosys_map map =IOSYS_MAP_INIT_OFFSET (&memirq->status, offset *SZ_16 ); + +return memirq_received_noclear (memirq, &map,ilog2 (GUC_INTR_SW_INT_0 ), +guc_name (guc)); +} +/** * xe_memirq_handler - The `Memory Based Interrupts`_ Handler. * @memirq: the &xe_memirq drivers/gpu/drm/xe/xe_memirq.h +2
-0@@ -25,4 +25,6 @@ void xe_memirq_handler(struct xe_memirq *memirq);int xe_memirq_init_guc (struct xe_memirq *memirq,struct xe_guc *guc); +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc); +#endif
Matthew Brost
maintainerRe: [PATCH v4 11/34] drm/xe/vf: Add xe_gt_recovery_inprogress helper
Quoted 110 lines · Michal Wajdeczko
Quoted 76 lines · Matthew Brost
Add
xe_gt_recovery_inprogresshelper.This helper serves as the singular point to determine whether a GT recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.
v3:
- Add GT layerxe_gt_recovery_inprogress(Michal)
- Don't blow up in memirq not enabled (CI)
- Add__memirq_receivedwith clear argument (Michal)
-xe_memirq_sw_int_0_irq_pendingrename (Michal)
- Use offset inxe_memirq_sw_int_0_irq_pending(Michal)
v4:
- Refactorxe_gt_recovery_inprogresslogic around memirq (Michal)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt.h | 13 ++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 22 +++++++++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++ drivers/gpu/drm/xe/xe_memirq.c | 48 +++++++++++++++++++++-- drivers/gpu/drm/xe/xe_memirq.h | 2 + 6 files changed, 93 insertions(+), 4 deletions(-)drivers/gpu/drm/xe/xe_gt.h +13
-0@@ -12,6 +12,7 @@#include "xe_device.h" #include "xe_device_types.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine.h" #define for_each_hw_engine (hwe__, gt__, id__) \@@ -124,4 +125,16 @@ static inline bool xe_gt_is_usm_hwe(struct xe_gt *gt, struct xe_hw_engine *hwe) hwe->instance == gt->usm.reserved_bcs_instance; } +/** +* xe_gt_recovery_inprogress() - GT recovery in progress +* @gt: the &xe_gt +* +* Return: True if GT recovery in progress, False otherwise +*/ +static inline bool xe_gt_recovery_inprogress (struct xe_gt *gt) +{ +return IS_SRIOV_VF (gt_to_xe (gt)) && +xe_gt_sriov_vf_recovery_inprogress (gt); +} +#endif drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-0@@ -26,6 +26,7 @@#include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_lrc.h" +#include "xe_memirq.h" #include "xe_mmio.h" #include "xe_sriov.h" #include "xe_sriov_vf.h" @@ -776,6 +777,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)struct xe_device *xe =gt_to_xe (gt);xe_gt_assert (gt,IS_SRIOV_VF (xe)); +xe_gt_assert (gt,xe_gt_sriov_vf_recovery_inprogress (gt));set_bit (gt->info.id, &xe->sriov.vf.migration.gt_flags);/* drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1129
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7
-0@@ -1118,3 +1120,23 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)drm_printf (p,"\thandshake:\t%u.%u\n" , pf_version->major, pf_version->minor); } + +/** +* xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress +* @gt: the &xe_gt +* +* Return: True if VF post migration recovery in progress, False otherwise +*/ hmm, I'm little concern about the mismatch between function name and it's logic
drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1139
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10
-0+bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +/* early detection until recovery starts */ +if (xe_device_uses_memirq (gt_to_xe (gt)) && +xe_memirq_sw_int_0_irq_pending (memirq, >->uc.guc)) +return true;as here it may return true even if the recovery didn't start yet
drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1141
Quoted 2 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-0+ +return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress);and this is actually the flag that indicates that driver actually is recovering
Quoted 5 lines
maybe (depends on the actual need) we should either:
* rename this helper to:
xe_gt_sriov_vf_pending_recovery
Let's keep it to a single function. Will rename.
Quoted 114 lines
* or provide two helpers:
xe_gt_sriov_vf_needs_recovery
xe_gt_sriov_vf_recovery_inprogressQuoted 108 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0+}drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2
-0@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt); +u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt);drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10
-0@@ -46,6 +46,14 @@ struct xe_gt_sriov_vf_runtime { } *regs; }; +/** +* xe_gt_sriov_vf_migration - VF migration data. +*/ +struct xe_gt_sriov_vf_migration { +/** @recovery_inprogress: VF post migration recovery in progress */ +bool recovery_inprogress; +}; +/** * struct xe_gt_sriov_vf - GT level VF virtualization data. */ @@ -58,6 +66,8 @@ struct xe_gt_sriov_vf {struct xe_gt_sriov_vf_selfconfig self_config;/** @runtime: runtime data retrieved from the PF. */ struct xe_gt_sriov_vf_runtime runtime; +/** @migration: migration data for the VF. */ + struct xe_gt_sriov_vf_migration migration; };#endif drivers/gpu/drm/xe/xe_memirq.c +27
-4@@ -398,8 +398,9 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)memirq_set_enable (memirq, true); } -static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, -u16 offset,const char *name) +static bool __memirq_received (struct xe_memirq *memirq, +struct iosys_map *vector,u16 offset, +const char *name,bool clear) {u8 value;@@ -409,12 +410,26 @@ static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,memirq_err_ratelimited (memirq,"Unexpected memirq value %#x from %s at %u\n" , value, name, offset); -iosys_map_wr (vector, offset, u8,0x00 ); +if (clear) +iosys_map_wr (vector, offset, u8,0x00 ); }return value; } +static bool memirq_received_noclear (struct xe_memirq *memirq, +struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, false); +} + +static bool memirq_received (struct xe_memirq *memirq,struct iosys_map *vector, +u16 offset,const char *name) +{ +return __memirq_received (memirq, vector, offset, name, true); +} +static void memirq_dispatch_engine (struct xe_memirq *memirq,struct iosys_map *status,struct xe_hw_engine *hwe) {@@ -434,8 +449,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *statif (memirq_received (memirq, status,ilog2 (GUC_INTR_GUC2HOST ), name))xe_guc_irq_handler (guc,GUC_INTR_GUC2HOST ); -if (memirq_received (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), name)) +/* +* We must wait to perform the clear operation until after +* xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race +* conditions where xe_gt_sriov_vf_recovery_inprogress() returns false. +*/ +if (memirq_received_noclear (memirq, status,ilog2 (GUC_INTR_SW_INT_0 ), + name)) {xe_guc_irq_handler (guc,GUC_INTR_SW_INT_0 ); +iosys_map_wr (status,ilog2 (GUC_INTR_SW_INT_0 ), u8,0x00 ); + } }/** @@ -460,6 +483,23 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) } }
Quoted 7 lines
drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 487
Quoted 2 lines
drivers/gpu/drm/xe/xe_memirq.c +2
-0+/** +* xe_memirq_sw_int_0_irq_pending() - SW_INT_0 IRQ is pending maybe we should add "guc" to the function name?
xe_memirq_guc_sw_int_0_irq_pending()
Sure.
Matt
Quoted 30 lines
Quoted 29 lines
drivers/gpu/drm/xe/xe_memirq.c +15
-0+* @memirq: the &xe_memirq +* @guc: the &xe_guc to check for IRQ +* +* Return: True if SW_INT_0 IRQ is pending on @guc, False otherwise +*/ +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc) +{ +struct xe_gt *gt =guc_to_gt (guc); +u32 offset =xe_gt_is_media_type (gt) ?ilog2 (INTR_MGUC ) :ilog2 (INTR_GUC ); +struct iosys_map map =IOSYS_MAP_INIT_OFFSET (&memirq->status, offset *SZ_16 ); + +return memirq_received_noclear (memirq, &map,ilog2 (GUC_INTR_SW_INT_0 ), +guc_name (guc)); +} +/** * xe_memirq_handler - The `Memory Based Interrupts`_ Handler. * @memirq: the &xe_memirq drivers/gpu/drm/xe/xe_memirq.h +2
-0@@ -25,4 +25,6 @@ void xe_memirq_handler(struct xe_memirq *memirq);int xe_memirq_init_guc (struct xe_memirq *memirq,struct xe_guc *guc); +bool xe_memirq_sw_int_0_irq_pending (struct xe_memirq *memirq,struct xe_guc *guc); +#endif
Matthew Brost
maintainerPATCH v4 12/34 drm/xe/vf: Make VF recovery run on per-GT worker
In reply to Matthew BrostVF recovery is a per-GT operation, so it makes sense to isolate it to a per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.
v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper
v3:
- Drop xe_gt_sriov_ prefix for private functions (Michal)
- Drop message in xe_gt_sriov_vf_migration_init_early (Michal)
- Logic rework in vf_post_migration_notify_resfix_done (Michal)
- Rework init sequence layering (Michal)
--- drivers/gpu/drm/xe/xe_gt.c | 6 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 179 +++++++++++++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 3 +- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 7 + drivers/gpu/drm/xe/xe_sriov_vf.c | 240 ---------------------- drivers/gpu/drm/xe/xe_sriov_vf.h | 1 - drivers/gpu/drm/xe/xe_sriov_vf_types.h | 4 - 7 files changed, 182 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +6 -0
@@ -398,6 +398,12 @@ int xe_gt_init_early(struct xe_gt *gt)return err; } +if (IS_SRIOV_VF (gt_to_xe (gt))) { + err =xe_gt_sriov_vf_init_early (gt); +if (err) +return err; + } +xe_reg_sr_init (>->reg_sr,"GT" ,gt_to_xe (gt)); err =xe_wa_gt_init (gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +168 -11
226 lines not on this page · raw message
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,int xe_gt_sriov_vf_query_config (struct xe_gt *gt);int xe_gt_sriov_vf_connect (struct xe_gt *gt);int xe_gt_sriov_vf_query_runtime (struct xe_gt *gt); -void xe_gt_sriov_vf_default_lrcs_hwsp_rebase (struct xe_gt *gt); -int xe_gt_sriov_vf_notify_resfix_done (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt); +int xe_gt_sriov_vf_init_early (struct xe_gt *gt);bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt);u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
@@ -7,6 +7,7 @@#define _XE_GT_SRIOV_VF_TYPES_H_#include <linux/types.h> +#include <linux/workqueue.h> #include "xe_uc_fw_types.h" /**
@@ -50,6 +51,12 @@ struct xe_gt_sriov_vf_runtime {* xe_gt_sriov_vf_migration - VF migration data. */ struct xe_gt_sriov_vf_migration { +/** @migration: VF migration recovery worker */ +struct work_struct worker; +/** @lock: Protects recovery_queued */ +spinlock_t lock; +/** @recovery_queued: VF post migration recovery in queued */ +bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */ bool recovery_inprogress; };
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -240
270 lines not on this page · raw message
drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
@@ -13,7 +13,6 @@ struct xe_device;void xe_sriov_vf_init_early (struct xe_device *xe);int xe_sriov_vf_init_late (struct xe_device *xe); -void xe_sriov_vf_start_migration_recovery (struct xe_device *xe);bool xe_sriov_vf_migration_supported (struct xe_device *xe);void xe_sriov_vf_debugfs_register (struct xe_device *xe,struct dentry *root);
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
@@ -33,10 +33,6 @@ struct xe_device_vf {/** @migration: VF Migration state data */ struct { -/** @migration.worker: VF migration recovery worker */ -struct work_struct worker; -/** @migration.gt_flags: Per-GT request flags for VF migration recovery */ -unsigned long gt_flags;/** * @migration.enabled: flag indicating if migration support * was enabled or not due to missing prerequisites
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 13/34 drm/xe/vf: Abort H2G sends during VF post-migration recovery
In reply to Matthew BrostWhile VF post-migration recovery is in progress, abort H2G sends with -ECANCEL. These messages are treated as lost, and TLB invalidation errors are suppressed. During this phase, the H2G channel is down, and VF recovery requires the CT lock to proceed.
v3:
- Use xe_gt_recovery_inprogress (Michal)
--- drivers/gpu/drm/xe/xe_guc_ct.c | 5 +++-- 1 file changed, 3 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +3 -2
@@ -851,7 +851,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, u32 len, u32 g2h_len, u32 num_g2h, struct g2h_fence *g2h_fence) { - struct xe_gt *gt __maybe_unused = ct_to_gt(ct); + struct xe_gt *gt = ct_to_gt(ct);u16 seqno;int ret;
@@ -872,7 +872,8 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, goto out; } -if (ct->state ==XE_GUC_CT_STATE_STOPPED ) { +if (ct->state ==XE_GUC_CT_STATE_STOPPED || +xe_gt_recovery_inprogress (gt)) { ret = -ECANCELED ; goto out; }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 14/34 drm/xe/vf: Remove memory allocations from VF post migration recovery
In reply to Matthew BrostVF post migration recovery is the path of dma-fence signaling / reclaim, avoid memory allocations in this path.
v3:
- s/lrc_wa_bb/scratch (Tomasz)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 23 +++++++++++++---------- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 2 ++ 2 files changed, 15 insertions(+), 10 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +13 -10
@@ -1160,17 +1160,13 @@ static size_t post_migration_scratch_size(struct xe_device *xe)static int vf_post_migration_fixups (struct xe_gt *gt) { +void *buf = gt->sriov.vf.migration.scratch;s64 shift; -void *buf;int err; - buf =kmalloc (post_migration_scratch_size (gt_to_xe (gt)),GFP_ATOMIC ); -if (!buf) -return -ENOMEM ; - err =xe_gt_sriov_vf_query_config (gt);if (err) - goto out; +return err; shift =xe_gt_sriov_vf_ggtt_shift (gt);if (shift) {
@@ -1178,12 +1174,10 @@ static int vf_post_migration_fixups(struct xe_gt *gt)xe_gt_sriov_vf_default_lrcs_hwsp_rebase (gt); err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf);if (err) - goto out; +return err; } -out: -kfree (buf); -return err; +return 0 ; }static void vf_post_migration_kickstart (struct xe_gt *gt)
@@ -1268,9 +1262,18 @@ static void migration_worker_func(struct work_struct *w)*/ int xe_gt_sriov_vf_init_early (struct xe_gt *gt) { +void *buf; +if (!xe_sriov_vf_migration_supported (gt_to_xe (gt)))return 0 ; + buf =drmm_kmalloc (>_to_xe (gt)->drm, +post_migration_scratch_size (gt_to_xe (gt)), +GFP_KERNEL ); +if (!buf) +return -ENOMEM ; + + gt->sriov.vf.migration.scratch = buf;init_rwsem (>->sriov.vf.self_config.lock);spin_lock_init (>->sriov.vf.migration.lock);INIT_WORK (>->sriov.vf.migration.worker, migration_worker_func);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +2 -0
@@ -55,6 +55,8 @@ struct xe_gt_sriov_vf_migration {struct work_struct worker;/** @lock: Protects recovery_queued */ spinlock_t lock; +/** @scratch: Scratch memory for VF recovery */ +void *scratch;/** @recovery_queued: VF post migration recovery in queued */ bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 15/34 drm/xe/vf: Close multi-GT GGTT shift race
In reply to Matthew BrostAs multi-GT VF post-migration recovery can run in parallel on different workqueues, but both GTs point to the same GGTT, only one GT needs to shift the GGTT. However, both GTs need to know when this step has completed. To coordinate this, perform the GGTT shift under the GGTT lock. With shift being done under the lock, storing the shift value becomes unnecessary.
v3:
- Update commmit message (Tomasz)
v4:
- Move GGTT values to tile state (Michal)
- Use GGTT lock (Michal)
--- drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 166 +++++++------------- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 5 +- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 7 +- drivers/gpu/drm/xe/xe_guc.c | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf.c | 30 +++- drivers/gpu/drm/xe/xe_tile_sriov_vf.h | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h | 23 +++ drivers/gpu/drm/xe/xe_vram.c | 6 +- 9 files changed, 117 insertions(+), 127 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h
drivers/gpu/drm/xe/xe_device_types.h +3 -0
@@ -27,6 +27,7 @@#include "xe_sriov_vf_ccs_types.h" #include "xe_step_types.h" #include "xe_survivability_mode_types.h" +#include "xe_tile_sriov_vf_types.h" #include "xe_validation.h" #if IS_ENABLED (CONFIG_DRM_XE_DEBUG )
@@ -185,6 +186,8 @@ struct xe_tile {struct {/** @sriov.vf.ggtt_balloon: GGTT regions excluded from use. */ struct xe_ggtt_node *ggtt_balloon[2 ]; +/** @sriov.vf.self_config: VF configuration data */ +struct xe_tile_sriov_vf_selfconfig self_config; } vf; } sriov;
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +60 -106
298 lines not on this page · raw message
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -4
@@ -18,7 +18,7 @@ int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);void xe_gt_sriov_vf_guc_versions (struct xe_gt *gt,struct xe_uc_fw_version *wanted,struct xe_uc_fw_version *found); -int xe_gt_sriov_vf_query_config (struct xe_gt *gt); +int xe_gt_sriov_vf_query_config (struct xe_gt *gt,bool recovery);int xe_gt_sriov_vf_connect (struct xe_gt *gt);int xe_gt_sriov_vf_query_runtime (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt);
@@ -29,9 +29,6 @@ bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt); -u64 xe_gt_sriov_vf_ggtt (struct xe_gt *gt); -u64 xe_gt_sriov_vf_ggtt_base (struct xe_gt *gt); -s64 xe_gt_sriov_vf_ggtt_shift (struct xe_gt *gt);u32 xe_gt_sriov_vf_read32 (struct xe_gt *gt,struct xe_reg reg);void xe_gt_sriov_vf_write32 (struct xe_gt *gt,struct xe_reg reg,u32 val);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +1 -6
@@ -6,6 +6,7 @@#ifndef _XE_GT_SRIOV_VF_TYPES_H_#define _XE_GT_SRIOV_VF_TYPES_H_ +#include <linux/rwsem.h> #include <linux/types.h> #include <linux/workqueue.h> #include "xe_uc_fw_types.h"
@@ -14,12 +15,6 @@* struct xe_gt_sriov_vf_selfconfig - VF configuration data. */ struct xe_gt_sriov_vf_selfconfig { -/** @ggtt_base: assigned base offset of the GGTT region. */ -u64 ggtt_base; -/** @ggtt_size: assigned size of the GGTT region. */ -u64 ggtt_size; -/** @ggtt_shift: difference in ggtt_base on last migration */ -s64 ggtt_shift;/** @lmem_size: assigned size of the LMEM. */ u64 lmem_size;/** @num_ctxs: assigned number of GuC submission context IDs. */
drivers/gpu/drm/xe/xe_guc.c +1 -1
@@ -713,7 +713,7 @@ static int vf_guc_init_noalloc(struct xe_guc *guc)if (err)return err; - err =xe_gt_sriov_vf_query_config (gt); + err =xe_gt_sriov_vf_query_config (gt, false);if (err)return err;
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +24 -6
@@ -9,7 +9,6 @@#include "xe_assert.h" #include "xe_ggtt.h" -#include "xe_gt_sriov_vf.h" #include "xe_sriov.h" #include "xe_sriov_printk.h" #include "xe_tile_sriov_vf.h"
@@ -40,10 +39,10 @@ static int vf_init_ggtt_balloons(struct xe_tile *tile)* * Return: 0 on success or a negative error code on failure. */ -int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile) +static int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile) { -u64 ggtt_base =xe_gt_sriov_vf_ggtt_base (tile->primary_gt); -u64 ggtt_size =xe_gt_sriov_vf_ggtt (tile->primary_gt); +u64 ggtt_base = tile->sriov.vf.self_config.ggtt_base; +u64 ggtt_size = tile->sriov.vf.self_config.ggtt_size;struct xe_device *xe =tile_to_xe (tile);u64 wopcm =xe_wopcm_size (xe);u64 start, end;
@@ -244,11 +243,30 @@ void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift) {struct xe_ggtt *ggtt = tile->mem.ggtt; -mutex_lock (&ggtt->lock); +lockdep_assert_held (&ggtt->lock);xe_tile_sriov_vf_deballoon_ggtt_locked (tile);xe_ggtt_shift_nodes_locked (ggtt, shift);xe_tile_sriov_vf_balloon_ggtt_locked (tile); +} -mutex_unlock (&ggtt->lock); +/** +* xe_tile_sriov_vf_lmem - VF LMEM configuration. +* @tile: the &xe_tile +* +* This function is for VF use only. +* +* Return: size of the LMEM assigned to VF. +*/ +u64 xe_tile_sriov_vf_lmem (struct xe_tile *tile) +{ +struct xe_tile_sriov_vf_selfconfig *config = &tile->sriov.vf.self_config; +u64 val; + +xe_tile_assert (tile,IS_SRIOV_VF (tile_to_xe (tile))); + +xe_tile_assert (tile, config->lmem_size); + val = config->lmem_size; + +return val; }
drivers/gpu/drm/xe/xe_tile_sriov_vf.h +1 -1
@@ -11,8 +11,8 @@struct xe_tile ;int xe_tile_sriov_vf_prepare_ggtt (struct xe_tile *tile); -int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile);void xe_tile_sriov_vf_deballoon_ggtt_locked (struct xe_tile *tile);void xe_tile_sriov_vf_fixup_ggtt_nodes (struct xe_tile *tile,s64 shift); +u64 xe_tile_sriov_vf_lmem (struct xe_tile *tile);#endif
drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h new file +23 -0
@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2025 Intel Corporation +*/ + +#ifndef _XE_TILE_SRIOV_VF_TYPES_H_ +#define _XE_TILE_SRIOV_VF_TYPES_H_ + +#include <linux/mutex.h> + +/** +* struct xe_tile_sriov_vf_selfconfig - VF configuration data. +*/ +struct xe_tile_sriov_vf_selfconfig { +/** @ggtt_base: assigned base offset of the GGTT region. */ +u64 ggtt_base; +/** @ggtt_size: assigned size of the GGTT region. */ +u64 ggtt_size; +/** @lmem_size: assigned size of the LMEM. */ +u64 lmem_size; +}; + +#endif
drivers/gpu/drm/xe/xe_vram.c +3 -3
@@ -16,10 +16,10 @@#include "xe_device.h" #include "xe_force_wake.h" #include "xe_gt_mcr.h" -#include "xe_gt_sriov_vf.h" #include "xe_mmio.h" #include "xe_module.h" #include "xe_sriov.h" +#include "xe_tile_sriov_vf.h" #include "xe_ttm_vram_mgr.h" #include "xe_vram.h" #include "xe_vram_types.h"
@@ -237,9 +237,9 @@ static int tile_vram_size(struct xe_tile *tile, u64 *vram_size, offset =0 ;for_each_tile (t, xe, id)for_each_if (t->id < tile->id) - offset +=xe_gt_sriov_vf_lmem (t->primary_gt); + offset +=xe_tile_sriov_vf_lmem (t); - *tile_size =xe_gt_sriov_vf_lmem (gt); + *tile_size =xe_tile_sriov_vf_lmem (tile); *vram_size = *tile_size; *tile_offset = offset;
Signature · 1 line
2.34.1
Re: [PATCH v4 15/34] drm/xe/vf: Close multi-GT GGTT shift race
Quoted 11 lines · Matthew Brost
As multi-GT VF post-migration recovery can run in parallel on different workqueues, but both GTs point to the same GGTT, only one GT needs to shift the GGTT. However, both GTs need to know when this step has completed. To coordinate this, perform the GGTT shift under the GGTT lock. With shift being done under the lock, storing the shift value becomes unnecessary.
v3:
- Update commmit message (Tomasz)
v4:
- Move GGTT values to tile state (Michal)
I'm wondering if this would be huge task to make this move as a separate step/patch?
Quoted 50 lines
- Use GGTT lock (Michal)
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 166 +++++++------------- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 5 +- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 7 +- drivers/gpu/drm/xe/xe_guc.c | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf.c | 30 +++- drivers/gpu/drm/xe/xe_tile_sriov_vf.h | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h | 23 +++ drivers/gpu/drm/xe/xe_vram.c | 6 +- 9 files changed, 117 insertions(+), 127 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_tile_sriov_vf_types.hdrivers/gpu/drm/xe/xe_device_types.h +3
-0@@ -27,6 +27,7 @@#include "xe_sriov_vf_ccs_types.h" #include "xe_step_types.h" #include "xe_survivability_mode_types.h" +#include "xe_tile_sriov_vf_types.h" #include "xe_validation.h" #if IS_ENABLED (CONFIG_DRM_XE_DEBUG )@@ -185,6 +186,8 @@ struct xe_tile {struct {/** @sriov.vf.ggtt_balloon: GGTT regions excluded from use. */ struct xe_ggtt_node *ggtt_balloon[2 ]; +/** @sriov.vf.self_config: VF configuration data */ +struct xe_tile_sriov_vf_selfconfig self_config; } vf; } sriov;drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-2@@ -436,42 +436,57 @@ u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt)return value; } -static int vf_get_ggtt_info (struct xe_gt *gt) +static int vf_get_ggtt_info (struct xe_gt *gt,bool recovery) { -struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; + struct xe_tile_sriov_vf_selfconfig *config =
drivers/gpu/drm/xe/xe_gt_sriov_vf.c u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt) · line 451
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4
-0+ >_to_tile (gt)->sriov.vf.self_config; +struct xe_ggtt *ggtt =gt_to_tile (gt)->mem.ggtt;struct xe_guc *guc = >->uc.guc;u64 start, size; +s64 shift;int err;xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); +mutex_lock (&ggtt->lock);
with
guard(mutex)(&ggtt->lock);
you will avoid all below error prone goto's
Quoted 34 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +12
-5+ err =guc_action_query_single_klv64 (guc,GUC_KLV_VF_CFG_GGTT_START_KEY , &start);if (unlikely (err)) -return err; + goto out; err =guc_action_query_single_klv64 (guc,GUC_KLV_VF_CFG_GGTT_SIZE_KEY , &size);if (unlikely (err)) -return err; + goto out;if (config->ggtt_size && config->ggtt_size != size) {xe_gt_sriov_err (gt,"Unexpected GGTT reassignment: %lluK != %lluK\n" , size /SZ_1K , config->ggtt_size /SZ_1K ); -return -EREMCHG ; + err = -EREMCHG ; + goto out; }xe_gt_sriov_dbg_verbose (gt,"GGTT %#llx-%#llx = %lluK\n" , start, start + size -1 , size /SZ_1K ); - config->ggtt_shift = start - (s64 )config->ggtt_base; + shift = start - (s64 )config->ggtt_base; config->ggtt_base = start; config->ggtt_size = size; + err = config->ggtt_size ?0 : -ENODATA ; -return config->ggtt_size ?0 : -ENODATA ; +if (!err && shift && recovery) { +xe_gt_sriov_info (gt,"Shifting GGTT base by %lld to 0x%016llx\\n" , + shift, config->ggtt_base); +xe_tile_sriov_vf_fixup_ggtt_nodes (gt_to_tile (gt), shift); + }
drivers/gpu/drm/xe/xe_gt_sriov_vf.c u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt) · line 489
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +5
-1+out: +mutex_unlock (&ggtt->lock); +return err; }static int vf_get_lmem_info (struct xe_gt *gt) { -struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; +struct xe_tile_sriov_vf_selfconfig *config = + >_to_tile (gt)->sriov.vf.self_config;
I would try to avoid accessing directly tile-level data from here, but we can fix that in follow up
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_lmem_info(struct xe_gt *gt) · line 499
Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-1struct xe_guc *guc = >->uc.guc;char size_str[10 ];u64 size;@@ -481,20 +496,23 @@ static int vf_get_lmem_info(struct xe_gt *gt) err =guc_action_query_single_klv64 (guc,GUC_KLV_VF_CFG_LMEM_SIZE_KEY , &size);if (unlikely (err)) -return err; + goto out;
these goto's seems to be unnecessary now
Quoted 22 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +5
-2if (config->lmem_size && config->lmem_size != size) {xe_gt_sriov_err (gt,"Unexpected LMEM reassignment: %lluM != %lluM\n" , size /SZ_1M , config->lmem_size /SZ_1M ); -return -EREMCHG ; + err = -EREMCHG ; + goto out; }string_get_size (size,1 ,STRING_UNITS_2 , size_str,sizeof (size_str));xe_gt_sriov_dbg_verbose (gt,"LMEM %lluM %s\n" , size /SZ_1M , size_str); config->lmem_size = size; + err = config->lmem_size ?0 : -ENODATA ; -return config->lmem_size ?0 : -ENODATA ; +out: +return err; }static int vf_get_submission_cfg (struct xe_gt *gt)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_submission_cfg(struct xe_gt *gt) · line 533
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-2@@ -508,21 +526,23 @@ static int vf_get_submission_cfg(struct xe_gt *gt) err =guc_action_query_single_klv32 (guc,GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY , &num_ctxs);if (unlikely (err)) -return err; + goto out; err =guc_action_query_single_klv32 (guc,GUC_KLV_VF_CFG_NUM_DOORBELLS_KEY , &num_dbs);if (unlikely (err)) -return err; + goto out;
same here
Quoted 30 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +8
-3if (config->num_ctxs && config->num_ctxs != num_ctxs) {xe_gt_sriov_err (gt,"Unexpected CTXs reassignment: %u != %u\n" , num_ctxs, config->num_ctxs); -return -EREMCHG ; + err = -EREMCHG ; + goto out; }if (config->num_dbs && config->num_dbs != num_dbs) {xe_gt_sriov_err (gt,"Unexpected DBs reassignment: %u != %u\n" , num_dbs, config->num_dbs); -return -EREMCHG ; + err = -EREMCHG ; + goto out; }xe_gt_sriov_dbg_verbose (gt,"CTXs %u DBs %u\n" , num_ctxs, num_dbs);@@ -530,7 +550,10 @@ static int vf_get_submission_cfg(struct xe_gt *gt) config->num_ctxs = num_ctxs; config->num_dbs = num_dbs; -return config->num_ctxs ?0 : -ENODATA ; + err = config->num_ctxs ?0 : -ENODATA ; + +out: +return err; }static void vf_cache_gmdid (struct xe_gt *gt)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_cache_gmdid(struct xe_gt *gt) · line 570
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0@@ -544,17 +567,18 @@ static void vf_cache_gmdid(struct xe_gt *gt)/** * xe_gt_sriov_vf_query_config - Query SR-IOV config data over MMIO. * @gt: the &xe_gt +* @recovery: VF post migration recovery path
small note about what's different in this path would not hurt
Quoted 206 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +20
-90* * This function is for VF use only. * * Return: 0 on success or a negative error code on failure. */ -int xe_gt_sriov_vf_query_config (struct xe_gt *gt) +int xe_gt_sriov_vf_query_config (struct xe_gt *gt,bool recovery) {struct xe_device *xe =gt_to_xe (gt);int err; - err =vf_get_ggtt_info (gt); + err =vf_get_ggtt_info (gt, recovery);if (unlikely (err))return err;@@ -584,80 +608,16 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt)*/ u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt) { -xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); -xe_gt_assert (gt, gt->sriov.vf.guc_version.major); -xe_gt_assert (gt, gt->sriov.vf.self_config.num_ctxs); - -return gt->sriov.vf.self_config.num_ctxs; -} - -/** -* xe_gt_sriov_vf_lmem - VF LMEM configuration. -* @gt: the &xe_gt -* -* This function is for VF use only. -* -* Return: size of the LMEM assigned to VF. -*/ -u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt) -{ -xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); -xe_gt_assert (gt, gt->sriov.vf.guc_version.major); -xe_gt_assert (gt, gt->sriov.vf.self_config.lmem_size); - -return gt->sriov.vf.self_config.lmem_size; -} - -/** -* xe_gt_sriov_vf_ggtt - VF GGTT configuration. -* @gt: the &xe_gt -* -* This function is for VF use only. -* -* Return: size of the GGTT assigned to VF. -*/ -u64 xe_gt_sriov_vf_ggtt (struct xe_gt *gt) -{ -xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); -xe_gt_assert (gt, gt->sriov.vf.guc_version.major); -xe_gt_assert (gt, gt->sriov.vf.self_config.ggtt_size); - -return gt->sriov.vf.self_config.ggtt_size; -} +struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; +u16 val; -/** -* xe_gt_sriov_vf_ggtt_base - VF GGTT base offset. -* @gt: the &xe_gt -* -* This function is for VF use only. -* -* Return: base offset of the GGTT assigned to VF. -*/ -u64 xe_gt_sriov_vf_ggtt_base (struct xe_gt *gt) -{xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt)));xe_gt_assert (gt, gt->sriov.vf.guc_version.major); -xe_gt_assert (gt, gt->sriov.vf.self_config.ggtt_size); -return gt->sriov.vf.self_config.ggtt_base; -} +xe_gt_assert (gt, config->num_ctxs); + val = config->num_ctxs; -/** -* xe_gt_sriov_vf_ggtt_shift - Return shift in GGTT range due to VF migration -* @gt: the &xe_gt struct instance -* -* This function is for VF use only. -* -* Return: The shift value; could be negative -*/ -s64 xe_gt_sriov_vf_ggtt_shift (struct xe_gt *gt) -{ -struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; - -xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); -xe_gt_assert (gt,xe_gt_is_main_type (gt)); - -return config->ggtt_shift; +return val; }static int relay_action_handshake (struct xe_gt *gt,u32 *major,u32 *minor)@@ -1057,6 +1017,8 @@ void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val)*/ void xe_gt_sriov_vf_print_config (struct xe_gt *gt,struct drm_printer *p) { +struct xe_tile_sriov_vf_selfconfig *tconfig = + >_to_tile (gt)->sriov.vf.self_config;struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config;struct xe_device *xe =gt_to_xe (gt);char buf[10 ];@@ -1064,17 +1026,15 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt)));drm_printf (p,"GGTT range:\t%#llx-%#llx\n" , - config->ggtt_base, - config->ggtt_base + config->ggtt_size -1 ); - -string_get_size (config->ggtt_size,1 ,STRING_UNITS_2 , buf,sizeof (buf)); -drm_printf (p,"GGTT size:\t%llu (%s)\n" , config->ggtt_size, buf); + tconfig->ggtt_base, + tconfig->ggtt_base + tconfig->ggtt_size -1 ); -drm_printf (p,"GGTT shift on last restore:\t%lld\n" , config->ggtt_shift); +string_get_size (tconfig->ggtt_size,1 ,STRING_UNITS_2 , buf,sizeof (buf)); +drm_printf (p,"GGTT size:\t%llu (%s)\n" , tconfig->ggtt_size, buf);if (IS_DGFX (xe) &&xe_gt_is_main_type (gt)) { -string_get_size (config->lmem_size,1 ,STRING_UNITS_2 , buf,sizeof (buf)); -drm_printf (p,"LMEM size:\t%llu (%s)\n" , config->lmem_size, buf); +string_get_size (tconfig->lmem_size,1 ,STRING_UNITS_2 , buf,sizeof (buf)); +drm_printf (p,"LMEM size:\t%llu (%s)\n" , tconfig->lmem_size, buf); }drm_printf (p,"GuC contexts:\t%u\n" , config->num_ctxs);@@ -1161,21 +1121,16 @@ static size_t post_migration_scratch_size(struct xe_device *xe)static int vf_post_migration_fixups (struct xe_gt *gt) {void *buf = gt->sriov.vf.migration.scratch; -s64 shift;int err; - err =xe_gt_sriov_vf_query_config (gt); + err =xe_gt_sriov_vf_query_config (gt, true);if (err)return err; - shift =xe_gt_sriov_vf_ggtt_shift (gt); -if (shift) { -xe_tile_sriov_vf_fixup_ggtt_nodes (gt_to_tile (gt), shift); -xe_gt_sriov_vf_default_lrcs_hwsp_rebase (gt); - err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf); -if (err) -return err; - } +xe_gt_sriov_vf_default_lrcs_hwsp_rebase (gt); + err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf); +if (err) +return err;return 0 ; }@@ -1274,7 +1229,6 @@ int xe_gt_sriov_vf_init_early(struct xe_gt *gt)return -ENOMEM ; gt->sriov.vf.migration.scratch = buf; -init_rwsem (>->sriov.vf.self_config.lock);spin_lock_init (>->sriov.vf.migration.lock);INIT_WORK (>->sriov.vf.migration.worker, migration_worker_func);drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1
-4@@ -18,7 +18,7 @@ int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);void xe_gt_sriov_vf_guc_versions (struct xe_gt *gt,struct xe_uc_fw_version *wanted,struct xe_uc_fw_version *found); -int xe_gt_sriov_vf_query_config (struct xe_gt *gt); +int xe_gt_sriov_vf_query_config (struct xe_gt *gt,bool recovery);int xe_gt_sriov_vf_connect (struct xe_gt *gt);int xe_gt_sriov_vf_query_runtime (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt);@@ -29,9 +29,6 @@ bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt); -u64 xe_gt_sriov_vf_ggtt (struct xe_gt *gt); -u64 xe_gt_sriov_vf_ggtt_base (struct xe_gt *gt); -s64 xe_gt_sriov_vf_ggtt_shift (struct xe_gt *gt);u32 xe_gt_sriov_vf_read32 (struct xe_gt *gt,struct xe_reg reg);void xe_gt_sriov_vf_write32 (struct xe_gt *gt,struct xe_reg reg,u32 val);drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +1
-0@@ -6,6 +6,7 @@#ifndef _XE_GT_SRIOV_VF_TYPES_H_#define _XE_GT_SRIOV_VF_TYPES_H_ +#include <linux/rwsem.h> #include <linux/types.h> #include <linux/workqueue.h> #include "xe_uc_fw_types.h"
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h line 20
Quoted 7 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +0
-4@@ -14,12 +15,6 @@* struct xe_gt_sriov_vf_selfconfig - VF configuration data. */ struct xe_gt_sriov_vf_selfconfig { -/** @ggtt_base: assigned base offset of the GGTT region. */ -u64 ggtt_base; -/** @ggtt_size: assigned size of the GGTT region. */ -u64 ggtt_size;
maybe we can keep GGTT base/size at the GT level, as this is still a valid data that we get from the GuC, and the base/size at the Tile-level would be just what we actually use/setup?
in theory both (or rather all 3 is some cases) should be always the same, but you never know ...
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h line 19
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +0
-2-/** @ggtt_shift: difference in ggtt_base on last migration */ -s64 ggtt_shift;/** @lmem_size: assigned size of the LMEM. */ u64 lmem_size;
and this one should then either be removed or keep it like proposed above
Quoted 27 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +0
-0/** @num_ctxs: assigned number of GuC submission context IDs. */ drivers/gpu/drm/xe/xe_guc.c +1
-1@@ -713,7 +713,7 @@ static int vf_guc_init_noalloc(struct xe_guc *guc)if (err)return err; - err =xe_gt_sriov_vf_query_config (gt); + err =xe_gt_sriov_vf_query_config (gt, false);if (err)return err;drivers/gpu/drm/xe/xe_tile_sriov_vf.c +0
-1@@ -9,7 +9,6 @@#include "xe_assert.h" #include "xe_ggtt.h" -#include "xe_gt_sriov_vf.h" #include "xe_sriov.h" #include "xe_sriov_printk.h" #include "xe_tile_sriov_vf.h"
drivers/gpu/drm/xe/xe_tile_sriov_vf.c static int vf_init_ggtt_balloons(struct xe_tile *tile) · line 42
Quoted 5 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +1
-1@@ -40,10 +39,10 @@ static int vf_init_ggtt_balloons(struct xe_tile *tile)* * Return: 0 on success or a negative error code on failure. */ -int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile) +static int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile)
nit: if it's static then no need for "xe_tile_sriov" prefix
vf_balloon_ggtt_locked()
Quoted 26 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +9
-4{ -u64 ggtt_base =xe_gt_sriov_vf_ggtt_base (tile->primary_gt); -u64 ggtt_size =xe_gt_sriov_vf_ggtt (tile->primary_gt); +u64 ggtt_base = tile->sriov.vf.self_config.ggtt_base; +u64 ggtt_size = tile->sriov.vf.self_config.ggtt_size;struct xe_device *xe =tile_to_xe (tile);u64 wopcm =xe_wopcm_size (xe);u64 start, end;@@ -244,11 +243,30 @@ void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift) {struct xe_ggtt *ggtt = tile->mem.ggtt; -mutex_lock (&ggtt->lock); +lockdep_assert_held (&ggtt->lock);xe_tile_sriov_vf_deballoon_ggtt_locked (tile);xe_ggtt_shift_nodes_locked (ggtt, shift);xe_tile_sriov_vf_balloon_ggtt_locked (tile); +} -mutex_unlock (&ggtt->lock); +/** +* xe_tile_sriov_vf_lmem - VF LMEM configuration. +* @tile: the &xe_tile +* +* This function is for VF use only.
drivers/gpu/drm/xe/xe_tile_sriov_vf.c void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift) · line 267
Quoted 10 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +10
-0+* +* Return: size of the LMEM assigned to VF. +*/ +u64 xe_tile_sriov_vf_lmem (struct xe_tile *tile) +{ +struct xe_tile_sriov_vf_selfconfig *config = &tile->sriov.vf.self_config; +u64 val; + +xe_tile_assert (tile,IS_SRIOV_VF (tile_to_xe (tile))); +
drop this sep line
drivers/gpu/drm/xe/xe_tile_sriov_vf.c void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift) · line 268
Quoted 1 line
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +1
-0+xe_tile_assert (tile, config->lmem_size);
or rather move it here
Quoted 24 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +3
-0+ val = config->lmem_size; + +return val; }drivers/gpu/drm/xe/xe_tile_sriov_vf.h +1
-1@@ -11,8 +11,8 @@struct xe_tile ;int xe_tile_sriov_vf_prepare_ggtt (struct xe_tile *tile); -int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile);void xe_tile_sriov_vf_deballoon_ggtt_locked (struct xe_tile *tile);void xe_tile_sriov_vf_fixup_ggtt_nodes (struct xe_tile *tile,s64 shift); +u64 xe_tile_sriov_vf_lmem (struct xe_tile *tile);#endif drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h new file +0
-0
drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h line 9
Quoted 9 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h new file +9
-0@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2025 Intel Corporation +*/ + +#ifndef _XE_TILE_SRIOV_VF_TYPES_H_ +#define _XE_TILE_SRIOV_VF_TYPES_H_ + +#include <linux/mutex.h>
not needed
Quoted 42 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h new file +14
-0+ +/** +* struct xe_tile_sriov_vf_selfconfig - VF configuration data. +*/ +struct xe_tile_sriov_vf_selfconfig { +/** @ggtt_base: assigned base offset of the GGTT region. */ +u64 ggtt_base; +/** @ggtt_size: assigned size of the GGTT region. */ +u64 ggtt_size; +/** @lmem_size: assigned size of the LMEM. */ +u64 lmem_size; +}; + +#endif drivers/gpu/drm/xe/xe_vram.c +3
-3@@ -16,10 +16,10 @@#include "xe_device.h" #include "xe_force_wake.h" #include "xe_gt_mcr.h" -#include "xe_gt_sriov_vf.h" #include "xe_mmio.h" #include "xe_module.h" #include "xe_sriov.h" +#include "xe_tile_sriov_vf.h" #include "xe_ttm_vram_mgr.h" #include "xe_vram.h" #include "xe_vram_types.h" @@ -237,9 +237,9 @@ static int tile_vram_size(struct xe_tile *tile, u64 *vram_size, offset =0 ;for_each_tile (t, xe, id)for_each_if (t->id < tile->id) - offset +=xe_gt_sriov_vf_lmem (t->primary_gt); + offset +=xe_tile_sriov_vf_lmem (t); - *tile_size =xe_gt_sriov_vf_lmem (gt); + *tile_size =xe_tile_sriov_vf_lmem (tile); *vram_size = *tile_size; *tile_offset = offset;
Matthew Brost
maintainerRe: [PATCH v4 15/34] drm/xe/vf: Close multi-GT GGTT shift race
Quoted 10 lines · Michal Wajdeczko
Quoted 7 lines · Matthew Brost
As multi-GT VF post-migration recovery can run in parallel on different workqueues, but both GTs point to the same GGTT, only one GT needs to shift the GGTT. However, both GTs need to know when this step has completed. To coordinate this, perform the GGTT shift under the GGTT lock. With shift being done under the lock, storing the shift value becomes unnecessary.
Quoted 7 lines
Quoted 4 lines
v3:
- Update commmit message (Tomasz)
v4:
- Move GGTT values to tile state (Michal)I'm wondering if this would be huge task to make this move as a separate step/patch?
Let me look at that but quick reaction is there changes are fairly tighly coupled.
Quoted 57 lines
Quoted 57 lines
- Use GGTT lock (Michal)
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_device_types.h | 3 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 166 +++++++------------- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 5 +- drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 7 +- drivers/gpu/drm/xe/xe_guc.c | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf.c | 30 +++- drivers/gpu/drm/xe/xe_tile_sriov_vf.h | 2 +- drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h | 23 +++ drivers/gpu/drm/xe/xe_vram.c | 6 +- 9 files changed, 117 insertions(+), 127 deletions(-) create mode 100644 drivers/gpu/drm/xe/xe_tile_sriov_vf_types.hdrivers/gpu/drm/xe/xe_device_types.h +3
-0@@ -27,6 +27,7 @@#include "xe_sriov_vf_ccs_types.h" #include "xe_step_types.h" #include "xe_survivability_mode_types.h" +#include "xe_tile_sriov_vf_types.h" #include "xe_validation.h" #if IS_ENABLED (CONFIG_DRM_XE_DEBUG )@@ -185,6 +186,8 @@ struct xe_tile {struct {/** @sriov.vf.ggtt_balloon: GGTT regions excluded from use. */ struct xe_ggtt_node *ggtt_balloon[2 ]; +/** @sriov.vf.self_config: VF configuration data */ +struct xe_tile_sriov_vf_selfconfig self_config; } vf; } sriov;drivers/gpu/drm/xe/xe_gt_sriov_vf.c +5
-2@@ -436,42 +436,57 @@ u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt)return value; } -static int vf_get_ggtt_info (struct xe_gt *gt) +static int vf_get_ggtt_info (struct xe_gt *gt,bool recovery) { -struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; +struct xe_tile_sriov_vf_selfconfig *config = + >_to_tile (gt)->sriov.vf.self_config; +struct xe_ggtt *ggtt =gt_to_tile (gt)->mem.ggtt;struct xe_guc *guc = >->uc.guc;u64 start, size; +s64 shift;int err;
Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt) · line 451
Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); +mutex_lock (&ggtt->lock);with
guard(mutex)(&ggtt->lock);you will avoid all below error prone goto's
Yes, agree a guard would be better here. Will change.
Quoted 88 lines
Quoted 34 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +12
-5+ err =guc_action_query_single_klv64 (guc,GUC_KLV_VF_CFG_GGTT_START_KEY , &start);if (unlikely (err)) -return err; + goto out; err =guc_action_query_single_klv64 (guc,GUC_KLV_VF_CFG_GGTT_SIZE_KEY , &size);if (unlikely (err)) -return err; + goto out;if (config->ggtt_size && config->ggtt_size != size) {xe_gt_sriov_err (gt,"Unexpected GGTT reassignment: %lluK != %lluK\n" , size /SZ_1K , config->ggtt_size /SZ_1K ); -return -EREMCHG ; + err = -EREMCHG ; + goto out; }xe_gt_sriov_dbg_verbose (gt,"GGTT %#llx-%#llx = %lluK\n" , start, start + size -1 , size /SZ_1K ); - config->ggtt_shift = start - (s64 )config->ggtt_base; + shift = start - (s64 )config->ggtt_base; config->ggtt_base = start; config->ggtt_size = size; + err = config->ggtt_size ?0 : -ENODATA ; -return config->ggtt_size ?0 : -ENODATA ; +if (!err && shift && recovery) { +xe_gt_sriov_info (gt,"Shifting GGTT base by %lld to 0x%016llx\\n" , + shift, config->ggtt_base); +xe_tile_sriov_vf_fixup_ggtt_nodes (gt_to_tile (gt), shift); + }drivers/gpu/drm/xe/xe_gt_sriov_vf.c u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt) · line 489
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +5
-1+out: +mutex_unlock (&ggtt->lock); +return err; }static int vf_get_lmem_info (struct xe_gt *gt) { -struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; +struct xe_tile_sriov_vf_selfconfig *config = + >_to_tile (gt)->sriov.vf.self_config;I would try to avoid accessing directly tile-level data from here, but we can fix that in follow up
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_lmem_info(struct xe_gt *gt) · line 499
Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-1struct xe_guc *guc = >->uc.guc;char size_str[10 ];u64 size;@@ -481,20 +496,23 @@ static int vf_get_lmem_info(struct xe_gt *gt) err =guc_action_query_single_klv64 (guc,GUC_KLV_VF_CFG_LMEM_SIZE_KEY , &size);if (unlikely (err)) -return err; + goto out;these goto's seems to be unnecessary now
Quoted 28 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6
-3if (config->lmem_size && config->lmem_size != size) {xe_gt_sriov_err (gt,"Unexpected LMEM reassignment: %lluM != %lluM\n" , size /SZ_1M , config->lmem_size /SZ_1M ); -return -EREMCHG ; + err = -EREMCHG ; + goto out; }string_get_size (size,1 ,STRING_UNITS_2 , size_str,sizeof (size_str));xe_gt_sriov_dbg_verbose (gt,"LMEM %lluM %s\n" , size /SZ_1M , size_str); config->lmem_size = size; + err = config->lmem_size ?0 : -ENODATA ; -return config->lmem_size ?0 : -ENODATA ; +out: +return err; }static int vf_get_submission_cfg (struct xe_gt *gt)@@ -508,21 +526,23 @@ static int vf_get_submission_cfg(struct xe_gt *gt) err =guc_action_query_single_klv32 (guc,GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY , &num_ctxs);if (unlikely (err)) -return err; + goto out;
Quoted 7 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_submission_cfg(struct xe_gt *gt) · line 533
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-1err =guc_action_query_single_klv32 (guc,GUC_KLV_VF_CFG_NUM_DOORBELLS_KEY , &num_dbs);if (unlikely (err)) -return err; + goto out;same here
Will remove the gotos in this functions.
Quoted 28 lines
Quoted 28 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +8
-3if (config->num_ctxs && config->num_ctxs != num_ctxs) {xe_gt_sriov_err (gt,"Unexpected CTXs reassignment: %u != %u\n" , num_ctxs, config->num_ctxs); -return -EREMCHG ; + err = -EREMCHG ; + goto out; }if (config->num_dbs && config->num_dbs != num_dbs) {xe_gt_sriov_err (gt,"Unexpected DBs reassignment: %u != %u\n" , num_dbs, config->num_dbs); -return -EREMCHG ; + err = -EREMCHG ; + goto out; }xe_gt_sriov_dbg_verbose (gt,"CTXs %u DBs %u\n" , num_ctxs, num_dbs);@@ -530,7 +550,10 @@ static int vf_get_submission_cfg(struct xe_gt *gt) config->num_ctxs = num_ctxs; config->num_dbs = num_dbs; -return config->num_ctxs ?0 : -ENODATA ; + err = config->num_ctxs ?0 : -ENODATA ; + +out: +return err; }
Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_cache_gmdid(struct xe_gt *gt) · line 570
Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0static void vf_cache_gmdid (struct xe_gt *gt)@@ -544,17 +567,18 @@ static void vf_cache_gmdid(struct xe_gt *gt)/** * xe_gt_sriov_vf_query_config - Query SR-IOV config data over MMIO. * @gt: the &xe_gt +* @recovery: VF post migration recovery path small note about what's different in this path would not hurt
Will add.
Matt
Quoted 386 lines
Quoted 206 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +20
-90* * This function is for VF use only. * * Return: 0 on success or a negative error code on failure. */ -int xe_gt_sriov_vf_query_config (struct xe_gt *gt) +int xe_gt_sriov_vf_query_config (struct xe_gt *gt,bool recovery) {struct xe_device *xe =gt_to_xe (gt);int err; - err =vf_get_ggtt_info (gt); + err =vf_get_ggtt_info (gt, recovery);if (unlikely (err))return err;@@ -584,80 +608,16 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt)*/ u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt) { -xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); -xe_gt_assert (gt, gt->sriov.vf.guc_version.major); -xe_gt_assert (gt, gt->sriov.vf.self_config.num_ctxs); - -return gt->sriov.vf.self_config.num_ctxs; -} - -/** -* xe_gt_sriov_vf_lmem - VF LMEM configuration. -* @gt: the &xe_gt -* -* This function is for VF use only. -* -* Return: size of the LMEM assigned to VF. -*/ -u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt) -{ -xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); -xe_gt_assert (gt, gt->sriov.vf.guc_version.major); -xe_gt_assert (gt, gt->sriov.vf.self_config.lmem_size); - -return gt->sriov.vf.self_config.lmem_size; -} - -/** -* xe_gt_sriov_vf_ggtt - VF GGTT configuration. -* @gt: the &xe_gt -* -* This function is for VF use only. -* -* Return: size of the GGTT assigned to VF. -*/ -u64 xe_gt_sriov_vf_ggtt (struct xe_gt *gt) -{ -xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); -xe_gt_assert (gt, gt->sriov.vf.guc_version.major); -xe_gt_assert (gt, gt->sriov.vf.self_config.ggtt_size); - -return gt->sriov.vf.self_config.ggtt_size; -} +struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; +u16 val; -/** -* xe_gt_sriov_vf_ggtt_base - VF GGTT base offset. -* @gt: the &xe_gt -* -* This function is for VF use only. -* -* Return: base offset of the GGTT assigned to VF. -*/ -u64 xe_gt_sriov_vf_ggtt_base (struct xe_gt *gt) -{xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt)));xe_gt_assert (gt, gt->sriov.vf.guc_version.major); -xe_gt_assert (gt, gt->sriov.vf.self_config.ggtt_size); -return gt->sriov.vf.self_config.ggtt_base; -} +xe_gt_assert (gt, config->num_ctxs); + val = config->num_ctxs; -/** -* xe_gt_sriov_vf_ggtt_shift - Return shift in GGTT range due to VF migration -* @gt: the &xe_gt struct instance -* -* This function is for VF use only. -* -* Return: The shift value; could be negative -*/ -s64 xe_gt_sriov_vf_ggtt_shift (struct xe_gt *gt) -{ -struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config; - -xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); -xe_gt_assert (gt,xe_gt_is_main_type (gt)); - -return config->ggtt_shift; +return val; }static int relay_action_handshake (struct xe_gt *gt,u32 *major,u32 *minor)@@ -1057,6 +1017,8 @@ void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val)*/ void xe_gt_sriov_vf_print_config (struct xe_gt *gt,struct drm_printer *p) { +struct xe_tile_sriov_vf_selfconfig *tconfig = + >_to_tile (gt)->sriov.vf.self_config;struct xe_gt_sriov_vf_selfconfig *config = >->sriov.vf.self_config;struct xe_device *xe =gt_to_xe (gt);char buf[10 ];@@ -1064,17 +1026,15 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt)));drm_printf (p,"GGTT range:\t%#llx-%#llx\n" , - config->ggtt_base, - config->ggtt_base + config->ggtt_size -1 ); - -string_get_size (config->ggtt_size,1 ,STRING_UNITS_2 , buf,sizeof (buf)); -drm_printf (p,"GGTT size:\t%llu (%s)\n" , config->ggtt_size, buf); + tconfig->ggtt_base, + tconfig->ggtt_base + tconfig->ggtt_size -1 ); -drm_printf (p,"GGTT shift on last restore:\t%lld\n" , config->ggtt_shift); +string_get_size (tconfig->ggtt_size,1 ,STRING_UNITS_2 , buf,sizeof (buf)); +drm_printf (p,"GGTT size:\t%llu (%s)\n" , tconfig->ggtt_size, buf);if (IS_DGFX (xe) &&xe_gt_is_main_type (gt)) { -string_get_size (config->lmem_size,1 ,STRING_UNITS_2 , buf,sizeof (buf)); -drm_printf (p,"LMEM size:\t%llu (%s)\n" , config->lmem_size, buf); +string_get_size (tconfig->lmem_size,1 ,STRING_UNITS_2 , buf,sizeof (buf)); +drm_printf (p,"LMEM size:\t%llu (%s)\n" , tconfig->lmem_size, buf); }drm_printf (p,"GuC contexts:\t%u\n" , config->num_ctxs);@@ -1161,21 +1121,16 @@ static size_t post_migration_scratch_size(struct xe_device *xe)static int vf_post_migration_fixups (struct xe_gt *gt) {void *buf = gt->sriov.vf.migration.scratch; -s64 shift;int err; - err =xe_gt_sriov_vf_query_config (gt); + err =xe_gt_sriov_vf_query_config (gt, true);if (err)return err; - shift =xe_gt_sriov_vf_ggtt_shift (gt); -if (shift) { -xe_tile_sriov_vf_fixup_ggtt_nodes (gt_to_tile (gt), shift); -xe_gt_sriov_vf_default_lrcs_hwsp_rebase (gt); - err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf); -if (err) -return err; - } +xe_gt_sriov_vf_default_lrcs_hwsp_rebase (gt); + err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf); +if (err) +return err;return 0 ; }@@ -1274,7 +1229,6 @@ int xe_gt_sriov_vf_init_early(struct xe_gt *gt)return -ENOMEM ; gt->sriov.vf.migration.scratch = buf; -init_rwsem (>->sriov.vf.self_config.lock);spin_lock_init (>->sriov.vf.migration.lock);INIT_WORK (>->sriov.vf.migration.worker, migration_worker_func);drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1
-4@@ -18,7 +18,7 @@ int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);void xe_gt_sriov_vf_guc_versions (struct xe_gt *gt,struct xe_uc_fw_version *wanted,struct xe_uc_fw_version *found); -int xe_gt_sriov_vf_query_config (struct xe_gt *gt); +int xe_gt_sriov_vf_query_config (struct xe_gt *gt,bool recovery);int xe_gt_sriov_vf_connect (struct xe_gt *gt);int xe_gt_sriov_vf_query_runtime (struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt);@@ -29,9 +29,6 @@ bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);u16 xe_gt_sriov_vf_guc_ids (struct xe_gt *gt);u64 xe_gt_sriov_vf_lmem (struct xe_gt *gt); -u64 xe_gt_sriov_vf_ggtt (struct xe_gt *gt); -u64 xe_gt_sriov_vf_ggtt_base (struct xe_gt *gt); -s64 xe_gt_sriov_vf_ggtt_shift (struct xe_gt *gt);u32 xe_gt_sriov_vf_read32 (struct xe_gt *gt,struct xe_reg reg);void xe_gt_sriov_vf_write32 (struct xe_gt *gt,struct xe_reg reg,u32 val);drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +1
-0@@ -6,6 +6,7 @@#ifndef _XE_GT_SRIOV_VF_TYPES_H_#define _XE_GT_SRIOV_VF_TYPES_H_ +#include <linux/rwsem.h> #include <linux/types.h> #include <linux/workqueue.h> #include "xe_uc_fw_types.h" drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h line 20
Quoted 7 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +0
-4@@ -14,12 +15,6 @@* struct xe_gt_sriov_vf_selfconfig - VF configuration data. */ struct xe_gt_sriov_vf_selfconfig { -/** @ggtt_base: assigned base offset of the GGTT region. */ -u64 ggtt_base; -/** @ggtt_size: assigned size of the GGTT region. */ -u64 ggtt_size;maybe we can keep GGTT base/size at the GT level, as this is still a valid data that we get from the GuC, and the base/size at the Tile-level would be just what we actually use/setup?
in theory both (or rather all 3 is some cases) should be always the same, but you never know ...
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h line 19
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +0
-2-/** @ggtt_shift: difference in ggtt_base on last migration */ -s64 ggtt_shift;/** @lmem_size: assigned size of the LMEM. */ u64 lmem_size;and this one should then either be removed or keep it like proposed above
Quoted 27 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +0
-0/** @num_ctxs: assigned number of GuC submission context IDs. */ drivers/gpu/drm/xe/xe_guc.c +1
-1@@ -713,7 +713,7 @@ static int vf_guc_init_noalloc(struct xe_guc *guc)if (err)return err; - err =xe_gt_sriov_vf_query_config (gt); + err =xe_gt_sriov_vf_query_config (gt, false);if (err)return err;drivers/gpu/drm/xe/xe_tile_sriov_vf.c +0
-1@@ -9,7 +9,6 @@#include "xe_assert.h" #include "xe_ggtt.h" -#include "xe_gt_sriov_vf.h" #include "xe_sriov.h" #include "xe_sriov_printk.h" #include "xe_tile_sriov_vf.h" drivers/gpu/drm/xe/xe_tile_sriov_vf.c static int vf_init_ggtt_balloons(struct xe_tile *tile) · line 42
Quoted 5 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +1
-1@@ -40,10 +39,10 @@ static int vf_init_ggtt_balloons(struct xe_tile *tile)* * Return: 0 on success or a negative error code on failure. */ -int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile) +static int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile)nit: if it's static then no need for "
xe_tile_sriov" prefix
vf_balloon_ggtt_locked()Quoted 26 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +9
-4{ -u64 ggtt_base =xe_gt_sriov_vf_ggtt_base (tile->primary_gt); -u64 ggtt_size =xe_gt_sriov_vf_ggtt (tile->primary_gt); +u64 ggtt_base = tile->sriov.vf.self_config.ggtt_base; +u64 ggtt_size = tile->sriov.vf.self_config.ggtt_size;struct xe_device *xe =tile_to_xe (tile);u64 wopcm =xe_wopcm_size (xe);u64 start, end;@@ -244,11 +243,30 @@ void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift) {struct xe_ggtt *ggtt = tile->mem.ggtt; -mutex_lock (&ggtt->lock); +lockdep_assert_held (&ggtt->lock);xe_tile_sriov_vf_deballoon_ggtt_locked (tile);xe_ggtt_shift_nodes_locked (ggtt, shift);xe_tile_sriov_vf_balloon_ggtt_locked (tile); +} -mutex_unlock (&ggtt->lock); +/** +* xe_tile_sriov_vf_lmem - VF LMEM configuration. +* @tile: the &xe_tile +* +* This function is for VF use only. drivers/gpu/drm/xe/xe_tile_sriov_vf.c void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift) · line 267
Quoted 10 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +10
-0+* +* Return: size of the LMEM assigned to VF. +*/ +u64 xe_tile_sriov_vf_lmem (struct xe_tile *tile) +{ +struct xe_tile_sriov_vf_selfconfig *config = &tile->sriov.vf.self_config; +u64 val; + +xe_tile_assert (tile,IS_SRIOV_VF (tile_to_xe (tile))); +drop this sep line
drivers/gpu/drm/xe/xe_tile_sriov_vf.c void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift) · line 268
Quoted 1 line
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +1
-0+xe_tile_assert (tile, config->lmem_size);or rather move it here
Quoted 24 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +3
-0+ val = config->lmem_size; + +return val; }drivers/gpu/drm/xe/xe_tile_sriov_vf.h +1
-1@@ -11,8 +11,8 @@struct xe_tile ;int xe_tile_sriov_vf_prepare_ggtt (struct xe_tile *tile); -int xe_tile_sriov_vf_balloon_ggtt_locked (struct xe_tile *tile);void xe_tile_sriov_vf_deballoon_ggtt_locked (struct xe_tile *tile);void xe_tile_sriov_vf_fixup_ggtt_nodes (struct xe_tile *tile,s64 shift); +u64 xe_tile_sriov_vf_lmem (struct xe_tile *tile);#endif drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h new file +0
-0drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h line 9
Quoted 9 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h new file +9
-0@@ -0,0 +1,23 @@ +/* SPDX-License-Identifier: MIT */ +/* +* Copyright © 2025 Intel Corporation +*/ + +#ifndef _XE_TILE_SRIOV_VF_TYPES_H_ +#define _XE_TILE_SRIOV_VF_TYPES_H_ + +#include <linux/mutex.h> not needed
Quoted 42 lines
drivers/gpu/drm/xe/xe_tile_sriov_vf_types.h new file +14
-0+ +/** +* struct xe_tile_sriov_vf_selfconfig - VF configuration data. +*/ +struct xe_tile_sriov_vf_selfconfig { +/** @ggtt_base: assigned base offset of the GGTT region. */ +u64 ggtt_base; +/** @ggtt_size: assigned size of the GGTT region. */ +u64 ggtt_size; +/** @lmem_size: assigned size of the LMEM. */ +u64 lmem_size; +}; + +#endif drivers/gpu/drm/xe/xe_vram.c +3
-3@@ -16,10 +16,10 @@#include "xe_device.h" #include "xe_force_wake.h" #include "xe_gt_mcr.h" -#include "xe_gt_sriov_vf.h" #include "xe_mmio.h" #include "xe_module.h" #include "xe_sriov.h" +#include "xe_tile_sriov_vf.h" #include "xe_ttm_vram_mgr.h" #include "xe_vram.h" #include "xe_vram_types.h" @@ -237,9 +237,9 @@ static int tile_vram_size(struct xe_tile *tile, u64 *vram_size, offset =0 ;for_each_tile (t, xe, id)for_each_if (t->id < tile->id) - offset +=xe_gt_sriov_vf_lmem (t->primary_gt); + offset +=xe_tile_sriov_vf_lmem (t); - *tile_size =xe_gt_sriov_vf_lmem (gt); + *tile_size =xe_tile_sriov_vf_lmem (tile); *vram_size = *tile_size; *tile_offset = offset;
Matthew Brost
maintainerPATCH v4 16/34 drm/xe/vf: Teardown VF post migration worker on driver unload
In reply to Matthew BrostBe cautious and ensure the VF post-migration worker is not running during driver unload.
v3:
- More teardown later in driver init, use devm (Tomasz)
--- drivers/gpu/drm/xe/xe_gt.c | 6 ++++ drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 34 ++++++++++++++++++++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 1 + drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 4 ++- 4 files changed, 43 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +6 -0
@@ -663,6 +663,12 @@ int xe_gt_init(struct xe_gt *gt)if (err)return err; +if (IS_SRIOV_VF (gt_to_xe (gt))) { + err =xe_gt_sriov_vf_init (gt); +if (err) +return err; + } +return 0 ; }
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +33 -1
@@ -737,7 +737,8 @@ static void vf_start_migration_recovery(struct xe_gt *gt)spin_lock (>->sriov.vf.migration.lock); -if (!gt->sriov.vf.migration.recovery_queued) { +if (!gt->sriov.vf.migration.recovery_queued || + !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true);
@@ -1209,6 +1210,17 @@ static void migration_worker_func(struct work_struct *w)vf_post_migration_recovery (gt); } +static void vf_migration_fini (void *arg) +{ +struct xe_gt *gt = arg; + +spin_lock_irq (>->sriov.vf.migration.lock); + gt->sriov.vf.migration.recovery_teardown = true; +spin_unlock_irq (>->sriov.vf.migration.lock); + +cancel_work_sync (>->sriov.vf.migration.worker); +} +/** * xe_gt_sriov_vf_init_early() - GT VF init early * @gt: the &xe_gt
@@ -1235,6 +1247,26 @@ int xe_gt_sriov_vf_init_early(struct xe_gt *gt)return 0 ; } +/** +* xe_gt_sriov_vf_init() - GT VF init +* @gt: the &xe_gt +* +* Return 0 on success, errno on failure +*/ +int xe_gt_sriov_vf_init (struct xe_gt *gt) +{ +if (!xe_sriov_vf_migration_supported (gt_to_xe (gt))) +return 0 ; + +/* +* We want to tear down the VF post-migration early during driver +* unload; therefore, we add this finalization action later during +* driver load. +*/ +return devm_add_action_or_reset (gt_to_xe (gt)->drm.dev, + vf_migration_fini, gt); +} +/** * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -0
@@ -24,6 +24,7 @@ int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);void xe_gt_sriov_vf_migrated_event_handler (struct xe_gt *gt);int xe_gt_sriov_vf_init_early (struct xe_gt *gt); +int xe_gt_sriov_vf_init (struct xe_gt *gt);bool xe_gt_sriov_vf_recovery_inprogress (struct xe_gt *gt);u32 xe_gt_sriov_vf_gmdid (struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -1
@@ -48,10 +48,12 @@ struct xe_gt_sriov_vf_runtime {struct xe_gt_sriov_vf_migration {/** @migration: VF migration recovery worker */ struct work_struct worker; -/** @lock: Protects recovery_queued */ +/** @lock: Protects recovery_queued, teardown */ spinlock_t lock;/** @scratch: Scratch memory for VF recovery */ void *scratch; +/** @recovery_teardown: VF post migration recovery is being torn down */ +bool recovery_teardown;/** @recovery_queued: VF post migration recovery in queued */ bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 17/34 drm/xe/vf: Don't allow GT reset to be queued during VF post migration recovery
In reply to Matthew BrostWith well-behaved software, a GT reset should never occur, nor should it happen during VF post-migration recovery. If it does, trigger a warning but suppress the GT reset, as VF post-migration recovery is expected to bring the VF back to a working state.
v3:
- Better commit message (Tomasz)
--- drivers/gpu/drm/xe/xe_gt.c | 9 ------- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 ----- drivers/gpu/drm/xe/xe_guc_submit.c | 41 +++-------------------------- drivers/gpu/drm/xe/xe_guc_submit.h | 3 --- 4 files changed, 4 insertions(+), 56 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +0 -9
@@ -815,11 +815,6 @@ static int do_gt_restart(struct xe_gt *gt)return 0 ; } -static int gt_wait_reset_unblock (struct xe_gt *gt) -{ -return xe_guc_wait_reset_unblock (>->uc.guc); -} -static int gt_reset (struct xe_gt *gt) {unsigned int fw_ref;
@@ -834,10 +829,6 @@ static int gt_reset(struct xe_gt *gt)xe_gt_info (gt,"reset started\n" ); - err =gt_wait_reset_unblock (gt); -if (!err) -xe_gt_warn (gt,"reset block failed to get lifted" ); -xe_pm_runtime_get (gt_to_xe (gt));if (xe_fault_inject_gt_reset ()) {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -7
@@ -1101,17 +1101,11 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)static void vf_post_migration_shutdown (struct xe_gt *gt) { -int ret =0 ; -spin_lock_irq (>->sriov.vf.migration.lock); gt->sriov.vf.migration.recovery_queued = false;spin_unlock_irq (>->sriov.vf.migration.lock);xe_guc_submit_pause (>->uc.guc); - ret |=xe_guc_submit_reset_block (>->uc.guc); - -if (ret) -xe_gt_sriov_info (gt,"migration recovery encountered ongoing reset\n" ); }static size_t post_migration_scratch_size (struct xe_device *xe)
@@ -1145,7 +1139,6 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)*/ xe_irq_resume (gt_to_xe (gt)); -xe_guc_submit_reset_unblock (>->uc.guc);xe_guc_submit_unpause (>->uc.guc); }
drivers/gpu/drm/xe/xe_guc_submit.c +4 -37
57 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_submit.h +0 -3
@@ -22,9 +22,6 @@ void xe_guc_submit_stop(struct xe_guc *guc);int xe_guc_submit_start (struct xe_guc *guc);void xe_guc_submit_pause (struct xe_guc *guc);void xe_guc_submit_unpause (struct xe_guc *guc); -int xe_guc_submit_reset_block (struct xe_guc *guc); -void xe_guc_submit_reset_unblock (struct xe_guc *guc); -int xe_guc_wait_reset_unblock (struct xe_guc *guc);void xe_guc_submit_wedge (struct xe_guc *guc);int xe_guc_read_stopped (struct xe_guc *guc);
Signature · 1 line
2.34.1
Re: [PATCH v4 17/34] drm/xe/vf: Don't allow GT reset to be queued during VF post migration recovery
Quoted 7 lines · Matthew Brost
With well-behaved software, a GT reset should never occur, nor should it happen during VF post-migration recovery. If it does, trigger a warning but suppress the GT reset, as VF post-migration recovery is expected to bring the VF back to a working state.
v3:
- Better commit message (Tomasz)
-Tomasz
Quoted 143 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt.c | 9 ------- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 ----- drivers/gpu/drm/xe/xe_guc_submit.c | 41 +++-------------------------- drivers/gpu/drm/xe/xe_guc_submit.h | 3 --- 4 files changed, 4 insertions(+), 56 deletions(-)drivers/gpu/drm/xe/xe_gt.c +0
-9@@ -815,11 +815,6 @@ static int do_gt_restart(struct xe_gt *gt)return 0 ; } -static int gt_wait_reset_unblock (struct xe_gt *gt) -{ -return xe_guc_wait_reset_unblock (>->uc.guc); -} -static int gt_reset (struct xe_gt *gt) {unsigned int fw_ref;@@ -834,10 +829,6 @@ static int gt_reset(struct xe_gt *gt)xe_gt_info (gt,"reset started\n" ); - err =gt_wait_reset_unblock (gt); -if (!err) -xe_gt_warn (gt,"reset block failed to get lifted" ); -xe_pm_runtime_get (gt_to_xe (gt));if (xe_fault_inject_gt_reset ()) {drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0
-7@@ -1101,17 +1101,11 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)static void vf_post_migration_shutdown (struct xe_gt *gt) { -int ret =0 ; -spin_lock_irq (>->sriov.vf.migration.lock); gt->sriov.vf.migration.recovery_queued = false;spin_unlock_irq (>->sriov.vf.migration.lock);xe_guc_submit_pause (>->uc.guc); - ret |=xe_guc_submit_reset_block (>->uc.guc); - -if (ret) -xe_gt_sriov_info (gt,"migration recovery encountered ongoing reset\n" ); }static size_t post_migration_scratch_size (struct xe_device *xe)@@ -1145,7 +1139,6 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)*/ xe_irq_resume (gt_to_xe (gt)); -xe_guc_submit_reset_unblock (>->uc.guc);xe_guc_submit_unpause (>->uc.guc); }drivers/gpu/drm/xe/xe_guc_submit.c +4
-37@@ -27,6 +27,7 @@#include "xe_gt.h" #include "xe_gt_clock.h" #include "xe_gt_printk.h" +#include "xe_gt_sriov_vf.h" #include "xe_guc.h" #include "xe_guc_capture.h" #include "xe_guc_ct.h" @@ -2182,47 +2183,13 @@ static void guc_exec_queue_stop(struct xe_guc *guc, struct xe_exec_queue *q) } } -/** -* xe_guc_submit_reset_block - Disallow reset calls on given GuC. -* @guc: the &xe_guc struct instance -*/ -int xe_guc_submit_reset_block (struct xe_guc *guc) -{ -return atomic_fetch_or (1 , &guc->submission_state.reset_blocked); -} - -/** -* xe_guc_submit_reset_unblock - Allow back reset calls on given GuC. -* @guc: the &xe_guc struct instance -*/ -void xe_guc_submit_reset_unblock (struct xe_guc *guc) -{ -atomic_set_release (&guc->submission_state.reset_blocked,0 ); -wake_up_all (&guc->ct.wq); -} - -static int guc_submit_reset_is_blocked (struct xe_guc *guc) -{ -return atomic_read_acquire (&guc->submission_state.reset_blocked); -} - -/* Maximum time of blocking reset */ -#define RESET_BLOCK_PERIOD_MAX (HZ * 5) - -/** -* xe_guc_wait_reset_unblock - Wait until reset blocking flag is lifted, or timeout. -* @guc: the &xe_guc struct instance -*/ -int xe_guc_wait_reset_unblock (struct xe_guc *guc) -{ -return wait_event_timeout (guc->ct.wq, - !guc_submit_reset_is_blocked (guc),RESET_BLOCK_PERIOD_MAX ); -} -int xe_guc_submit_reset_prepare (struct xe_guc *guc) {int ret; +if (WARN_ON_ONCE (xe_gt_sriov_vf_recovery_inprogress (guc_to_gt (guc)))) +return 0 ; +if (!guc->submission_state.initialized)return 0 ;drivers/gpu/drm/xe/xe_guc_submit.h +0
-3@@ -22,9 +22,6 @@ void xe_guc_submit_stop(struct xe_guc *guc);int xe_guc_submit_start (struct xe_guc *guc);void xe_guc_submit_pause (struct xe_guc *guc);void xe_guc_submit_unpause (struct xe_guc *guc); -int xe_guc_submit_reset_block (struct xe_guc *guc); -void xe_guc_submit_reset_unblock (struct xe_guc *guc); -int xe_guc_wait_reset_unblock (struct xe_guc *guc);void xe_guc_submit_wedge (struct xe_guc *guc);int xe_guc_read_stopped (struct xe_guc *guc);
Matthew Brost
maintainerPATCH v4 18/34 drm/xe/vf: Wakeup in GuC backend on VF post migration recovery
In reply to Matthew BrostIf VF post-migration recovery is in progress, the recovery flow will rebuild all GuC submission state. In this case, exit all waiters to ensure that submission queue scheduling can also be paused. Avoid taking any adverse actions after aborting the wait.
v3:
- Don't block in preempt fence work queue as this can interfere with VF post-migration work queue scheduling leading to deadlock (Testing)
- Use xe_gt_recovery_inprogress (Michal)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 3 + drivers/gpu/drm/xe/xe_guc_submit.c | 79 ++++++++++++++++++++------- drivers/gpu/drm/xe/xe_preempt_fence.c | 11 ++++ 3 files changed, 73 insertions(+), 20 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
@@ -741,6 +741,9 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); +smp_wmb ();/* Ensure above write visable before wake */ + +wake_up_all (>->uc.guc.ct.wq); started =queue_work (gt->ordered_wq, >->sriov.vf.migration.worker); xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
drivers/gpu/drm/xe/xe_guc_submit.c +59 -20
186 lines not on this page · raw message
drivers/gpu/drm/xe/xe_preempt_fence.c +11 -0
@@ -8,6 +8,8 @@#include <linux/slab.h> #include "xe_exec_queue.h" +#include "xe_gt_printk.h" +#include "xe_guc_exec_queue_types.h" #include "xe_vm.h" static void preempt_fence_work_func (struct work_struct *w)
@@ -22,6 +24,15 @@ static void preempt_fence_work_func(struct work_struct *w) }else if (!q->ops->reset_status (q)) {int err = q->ops->suspend_wait (q); +if (err == -EAGAIN ) { +xe_gt_dbg (q->gt,"PREEMPT FENCE RETRY guc_id=%d" , + q->guc->id); +queue_work (q->vm->xe->preempt_fence_wq, + &pfence->preempt_work); +dma_fence_end_signalling (cookie); +return ; + } +if (err)dma_fence_set_error (&pfence->base, err); }else {
Signature · 1 line
2.34.1
Re: [PATCH v4 18/34] drm/xe/vf: Wakeup in GuC backend on VF post migration recovery
Quoted 22 lines · Matthew Brost
If VF post-migration recovery is in progress, the recovery flow will rebuild all GuC submission state. In this case, exit all waiters to ensure that submission queue scheduling can also be paused. Avoid taking any adverse actions after aborting the wait.
v3:
- Don't block in preempt fence work queue as this can interfere with VF post-migration work queue scheduling leading to deadlock (Testing)
- Usexe_gt_recovery_inprogress(Michal)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 3 + drivers/gpu/drm/xe/xe_guc_submit.c | 79 ++++++++++++++++++++------- drivers/gpu/drm/xe/xe_preempt_fence.c | 11 ++++ 3 files changed, 73 insertions(+), 20 deletions(-)drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0
-0
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_start_migration_recovery(struct xe_gt *gt) · line 744
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0@@ -741,6 +741,9 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); +smp_wmb ();/* Ensure above write visable before wake */
shouldn't this be in separate fixup patch ?
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_start_migration_recovery(struct xe_gt *gt) · line 746
Quoted 2 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-0+ +wake_up_all (>->uc.guc.ct.wq);
shouldn't this be wrapped into some CT level helper ?
Quoted 16 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0
-0started =queue_work (gt->ordered_wq, >->sriov.vf.migration.worker); xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?drivers/gpu/drm/xe/xe_guc_submit.c +0
-1@@ -27,7 +27,6 @@#include "xe_gt.h" #include "xe_gt_clock.h" #include "xe_gt_printk.h" -#include "xe_gt_sriov_vf.h" #include "xe_guc.h" #include "xe_guc_capture.h" #include "xe_guc_ct.h"
drivers/gpu/drm/xe/xe_guc_submit.c static u32 wq_space_until_wrap(struct xe_exec_queue *q) · line 987
Quoted 5 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-0@@ -984,6 +983,9 @@ static u32 wq_space_until_wrap(struct xe_exec_queue *q)return (WQ_SIZE - q->guc->wqi_tail); } +#define vf_recovery (guc) \ + xe_gt_recovery_inprogress(guc_to_gt(guc))
use static inline helper instead
Quoted 108 lines
drivers/gpu/drm/xe/xe_guc_submit.c +28
-9+static int wq_wait_for_space (struct xe_exec_queue *q,u32 wqi_size) {struct xe_guc *guc =exec_queue_to_guc (q);@@ -993,7 +995,7 @@ static int wq_wait_for_space(struct xe_exec_queue *q, u32 wqi_size)#define AVAILABLE_SPACE \ CIRC_SPACE(q->guc->wqi_tail, q->guc->wqi_head, WQ_SIZE) -if (wqi_size >AVAILABLE_SPACE ) { +if (wqi_size >AVAILABLE_SPACE && !vf_recovery (guc)) { try_again: q->guc->wqi_head =parallel_read (xe, map, wq_desc.head);if (wqi_size >AVAILABLE_SPACE ) {@@ -1192,9 +1194,10 @@ static void disable_scheduling_deregister(struct xe_guc *guc, ret =wait_event_timeout (guc->ct.wq, (!exec_queue_pending_enable (q) && !exec_queue_pending_disable (q)) || -xe_guc_read_stopped (guc), +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); -if (!ret) { +if (!ret && !vf_recovery (guc)) {struct xe_gpu_scheduler *sched = &q->guc->sched;xe_gt_warn (q->gt,"Pending enable/disable failed to respond\n" );@@ -1297,6 +1300,10 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)bool wedged = false;xe_gt_assert (guc_to_gt (guc),xe_exec_queue_is_lr (q)); + +if (vf_recovery (guc)) +return ; +trace_xe_exec_queue_lr_cleanup (q);if (!exec_queue_killed (q))@@ -1329,7 +1336,11 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)*/ ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_disable (q) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) +return ; +if (!ret) {xe_gt_warn (q->gt,"Schedule disable failed to respond, guc_id=%d\n" , q->guc->id);@@ -1419,8 +1430,9 @@ static void enable_scheduling(struct xe_exec_queue *q) ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_enable (q) || -xe_guc_read_stopped (guc),HZ *5 ); -if (!ret ||xe_guc_read_stopped (guc)) { +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if ((!ret && !vf_recovery (guc)) ||xe_guc_read_stopped (guc)) {xe_gt_warn (guc_to_gt (guc),"Schedule enable failed to respond" );set_exec_queue_banned (q);xe_gt_reset_async (q->gt);@@ -1491,7 +1503,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)* list so job can be freed and kick scheduler ensuring free job is not * lost. */ -if (test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &job->fence->flags)) +if (test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &job->fence->flags) || +vf_recovery (guc))return DRM_GPU_SCHED_STAT_NO_HANG ;/* Kill the run_job entry point */ @@ -1543,7 +1556,10 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) ret =wait_event_timeout (guc->ct.wq, (!exec_queue_pending_enable (q) && !exec_queue_pending_disable (q)) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) + goto handle_vf_resume;if (!ret ||xe_guc_read_stopped (guc)) goto trigger_reset;@@ -1568,7 +1584,10 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)smp_rmb (); ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_disable (q) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) + goto handle_vf_resume;if (!ret ||xe_guc_read_stopped (guc)) { trigger_reset:if (!ret)@@ -1673,6 +1692,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)* some thought, do this in a follow up. */ xe_sched_submission_start (sched); +handle_vf_resume:return DRM_GPU_SCHED_STAT_NO_HANG ; }@@ -1769,11 +1789,17 @@ static void __guc_exec_queue_process_msg_set_sched_props(struct xe_sched_msg *msstatic void __suspend_fence_signal (struct xe_exec_queue *q) {
drivers/gpu/drm/xe/xe_guc_submit.c static void __guc_exec_queue_process_msg_set_sched_props(struct xe_sched_msg *ms · line 1800
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +5
-1+struct xe_guc *guc =exec_queue_to_guc (q); +struct xe_device *xe =guc_to_xe (guc); +if (!q->guc->suspend_pending)return ;WRITE_ONCE (q->guc->suspend_pending, false); -wake_up (&q->guc->suspend_wait); +if (IS_SRIOV_VF (xe)) +wake_up_all (&guc->ct.wq);
so now for the all VFs (without recovery) there is now no need to wakeup suspend_wait?
Quoted 63 lines
drivers/gpu/drm/xe/xe_guc_submit.c +23
-8+else +wake_up (&q->guc->suspend_wait); }static void suspend_fence_signal (struct xe_exec_queue *q)@@ -1794,8 +1820,9 @@ static void __guc_exec_queue_process_msg_suspend(struct xe_sched_msg *msg)if (guc_exec_queue_allowed_to_change_state (q) && !exec_queue_suspended (q) &&exec_queue_enabled (q)) { -wait_event (guc->ct.wq, (q->guc->resume_time !=RESUME_PENDING || -xe_guc_read_stopped (guc)) && !exec_queue_pending_disable (q)); +wait_event (guc->ct.wq,vf_recovery (guc) || + ((q->guc->resume_time !=RESUME_PENDING || +xe_guc_read_stopped (guc)) && !exec_queue_pending_disable (q)));if (!xe_guc_read_stopped (guc)) { s64 since_resume_ms =@@ -1922,7 +1949,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) q->entity = &ge->entity; -if (xe_guc_read_stopped (guc)) +if (xe_guc_read_stopped (guc) ||vf_recovery (guc))xe_sched_stop (sched);mutex_unlock (&guc->submission_state.lock);@@ -2068,6 +2095,7 @@ static int guc_exec_queue_suspend(struct xe_exec_queue *q)static int guc_exec_queue_suspend_wait (struct xe_exec_queue *q) {struct xe_guc *guc =exec_queue_to_guc (q); +struct xe_device *xe =guc_to_xe (guc);int ret;/* @@ -2075,11 +2103,22 @@ static int guc_exec_queue_suspend_wait(struct xe_exec_queue *q)* suspend_pending upon kill but to be paranoid but races in which * suspend_pending is set after kill also check kill here. */ - ret =wait_event_interruptible_timeout (q->guc->suspend_wait, - !READ_ONCE (q->guc->suspend_pending) || -exec_queue_killed (q) || -xe_guc_read_stopped (guc), -HZ *5 ); +if (IS_SRIOV_VF (xe)) + ret =wait_event_interruptible_timeout (guc->ct.wq, + !READ_ONCE (q->guc->suspend_pending) || +exec_queue_killed (q) || +xe_guc_read_stopped (guc) || +vf_recovery (guc), +HZ *5 ); +else + ret =wait_event_interruptible_timeout (q->guc->suspend_wait, + !READ_ONCE (q->guc->suspend_pending) || +exec_queue_killed (q) || +xe_guc_read_stopped (guc), +HZ *5 ); + +if (vf_recovery (guc) && !xe_device_wedged ((guc_to_xe (guc)))) +return -EAGAIN ;if (!ret) { xe_gt_warn(guc_to_gt(guc),
drivers/gpu/drm/xe/xe_guc_submit.c int xe_guc_submit_reset_prepare(struct xe_guc *guc) · line 2229
Quoted 5 lines
drivers/gpu/drm/xe/xe_guc_submit.c +1
-1@@ -2187,7 +2226,7 @@ int xe_guc_submit_reset_prepare(struct xe_guc *guc) {int ret; -if (WARN_ON_ONCE (xe_gt_sriov_vf_recovery_inprogress (guc_to_gt (guc)))) +if (WARN_ON_ONCE (vf_recovery (guc)))
xe_gt_WARN_ONCE is available for use
Quoted 17 lines
drivers/gpu/drm/xe/xe_guc_submit.c +0
-0return 0 ;if (!guc->submission_state.initialized)drivers/gpu/drm/xe/xe_preempt_fence.c +2
-0@@ -8,6 +8,8 @@#include <linux/slab.h> #include "xe_exec_queue.h" +#include "xe_gt_printk.h" +#include "xe_guc_exec_queue_types.h" #include "xe_vm.h" static void preempt_fence_work_func (struct work_struct *w)
drivers/gpu/drm/xe/xe_preempt_fence.c static void preempt_fence_work_func(struct work_struct *w) · line 33
Quoted 10 lines
drivers/gpu/drm/xe/xe_preempt_fence.c +7
-0@@ -22,6 +24,15 @@ static void preempt_fence_work_func(struct work_struct *w) }else if (!q->ops->reset_status (q)) {int err = q->ops->suspend_wait (q); +if (err == -EAGAIN ) { +xe_gt_dbg (q->gt,"PREEMPT FENCE RETRY guc_id=%d" , + q->guc->id); +queue_work (q->vm->xe->preempt_fence_wq, + &pfence->preempt_work); +dma_fence_end_signalling (cookie); +return ;
I'm not sure this chunk matches "wakeup GuC backend" commit description
Quoted 5 lines
drivers/gpu/drm/xe/xe_preempt_fence.c +2
-0+ } +if (err)dma_fence_set_error (&pfence->base, err); }else {
Matthew Brost
maintainerRe: [PATCH v4 18/34] drm/xe/vf: Wakeup in GuC backend on VF post migration recovery
Quoted 20 lines · Michal Wajdeczko
Quoted 17 lines · Matthew Brost
If VF post-migration recovery is in progress, the recovery flow will rebuild all GuC submission state. In this case, exit all waiters to ensure that submission queue scheduling can also be paused. Avoid taking any adverse actions after aborting the wait.
v3:
- Don't block in preempt fence work queue as this can interfere with VF post-migration work queue scheduling leading to deadlock (Testing)
- Usexe_gt_recovery_inprogress(Michal)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 3 + drivers/gpu/drm/xe/xe_guc_submit.c | 79 ++++++++++++++++++++------- drivers/gpu/drm/xe/xe_preempt_fence.c | 11 ++++ 3 files changed, 73 insertions(+), 20 deletions(-)
Quoted 12 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_start_migration_recovery(struct xe_gt *gt) · line 744
Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0@@ -741,6 +741,9 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); +smp_wmb ();/* Ensure above write visable before wake */ shouldn't this be in separate fixup patch ?
No, this is waking up GuC backend.
Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_start_migration_recovery(struct xe_gt *gt) · line 746
Quoted 2 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-0+ +wake_up_all (>->uc.guc.ct.wq);shouldn't this be wrapped into some CT level helper ?
Sure.
Quoted 19 lines
Quoted 19 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0
-0started =queue_work (gt->ordered_wq, >->sriov.vf.migration.worker); xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?drivers/gpu/drm/xe/xe_guc_submit.c +0
-1@@ -27,7 +27,6 @@#include "xe_gt.h" #include "xe_gt_clock.h" #include "xe_gt_printk.h" -#include "xe_gt_sriov_vf.h" #include "xe_guc.h" #include "xe_guc_capture.h" #include "xe_guc_ct.h" @@ -984,6 +983,9 @@ static u32 wq_space_until_wrap(struct xe_exec_queue *q)return (WQ_SIZE - q->guc->wqi_tail); }
Quoted 5 lines
drivers/gpu/drm/xe/xe_guc_submit.c static u32 wq_space_until_wrap(struct xe_exec_queue *q) · line 987
Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-0+#define vf_recovery (guc) \ + xe_gt_recovery_inprogress(guc_to_gt(guc))use static inline helper instead
Done.
Quoted 114 lines
Quoted 114 lines
drivers/gpu/drm/xe/xe_guc_submit.c +31
-9+static int wq_wait_for_space (struct xe_exec_queue *q,u32 wqi_size) {struct xe_guc *guc =exec_queue_to_guc (q);@@ -993,7 +995,7 @@ static int wq_wait_for_space(struct xe_exec_queue *q, u32 wqi_size)#define AVAILABLE_SPACE \ CIRC_SPACE(q->guc->wqi_tail, q->guc->wqi_head, WQ_SIZE) -if (wqi_size >AVAILABLE_SPACE ) { +if (wqi_size >AVAILABLE_SPACE && !vf_recovery (guc)) { try_again: q->guc->wqi_head =parallel_read (xe, map, wq_desc.head);if (wqi_size >AVAILABLE_SPACE ) {@@ -1192,9 +1194,10 @@ static void disable_scheduling_deregister(struct xe_guc *guc, ret =wait_event_timeout (guc->ct.wq, (!exec_queue_pending_enable (q) && !exec_queue_pending_disable (q)) || -xe_guc_read_stopped (guc), +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); -if (!ret) { +if (!ret && !vf_recovery (guc)) {struct xe_gpu_scheduler *sched = &q->guc->sched;xe_gt_warn (q->gt,"Pending enable/disable failed to respond\n" );@@ -1297,6 +1300,10 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)bool wedged = false;xe_gt_assert (guc_to_gt (guc),xe_exec_queue_is_lr (q)); + +if (vf_recovery (guc)) +return ; +trace_xe_exec_queue_lr_cleanup (q);if (!exec_queue_killed (q))@@ -1329,7 +1336,11 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)*/ ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_disable (q) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) +return ; +if (!ret) {xe_gt_warn (q->gt,"Schedule disable failed to respond, guc_id=%d\n" , q->guc->id);@@ -1419,8 +1430,9 @@ static void enable_scheduling(struct xe_exec_queue *q) ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_enable (q) || -xe_guc_read_stopped (guc),HZ *5 ); -if (!ret ||xe_guc_read_stopped (guc)) { +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if ((!ret && !vf_recovery (guc)) ||xe_guc_read_stopped (guc)) {xe_gt_warn (guc_to_gt (guc),"Schedule enable failed to respond" );set_exec_queue_banned (q);xe_gt_reset_async (q->gt);@@ -1491,7 +1503,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)* list so job can be freed and kick scheduler ensuring free job is not * lost. */ -if (test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &job->fence->flags)) +if (test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &job->fence->flags) || +vf_recovery (guc))return DRM_GPU_SCHED_STAT_NO_HANG ;/* Kill the run_job entry point */ @@ -1543,7 +1556,10 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) ret =wait_event_timeout (guc->ct.wq, (!exec_queue_pending_enable (q) && !exec_queue_pending_disable (q)) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) + goto handle_vf_resume;if (!ret ||xe_guc_read_stopped (guc)) goto trigger_reset;@@ -1568,7 +1584,10 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)smp_rmb (); ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_disable (q) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) + goto handle_vf_resume;if (!ret ||xe_guc_read_stopped (guc)) { trigger_reset:if (!ret)@@ -1673,6 +1692,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)* some thought, do this in a follow up. */ xe_sched_submission_start (sched); +handle_vf_resume:return DRM_GPU_SCHED_STAT_NO_HANG ; }@@ -1769,11 +1789,17 @@ static void __guc_exec_queue_process_msg_set_sched_props(struct xe_sched_msg *msstatic void __suspend_fence_signal (struct xe_exec_queue *q) { +struct xe_guc *guc =exec_queue_to_guc (q); +struct xe_device *xe =guc_to_xe (guc); +if (!q->guc->suspend_pending)return ;
Quoted 7 lines
drivers/gpu/drm/xe/xe_guc_submit.c static void __guc_exec_queue_process_msg_set_sched_props(struct xe_sched_msg *ms · line 1800
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-1WRITE_ONCE (q->guc->suspend_pending, false); -wake_up (&q->guc->suspend_wait); +if (IS_SRIOV_VF (xe)) +wake_up_all (&guc->ct.wq);so now for the all VFs (without recovery) there is now no need to wakeup
suspend_wait?
Yes. VF use &guc->ct.wq (a global wait queue) vs. q->guc->suspend_wait) (dedicated wait queue). This allows a single wake up call in vf_start_migration_recovery.
Quoted 66 lines
Quoted 66 lines
drivers/gpu/drm/xe/xe_guc_submit.c +23
-8+else +wake_up (&q->guc->suspend_wait); }static void suspend_fence_signal (struct xe_exec_queue *q)@@ -1794,8 +1820,9 @@ static void __guc_exec_queue_process_msg_suspend(struct xe_sched_msg *msg)if (guc_exec_queue_allowed_to_change_state (q) && !exec_queue_suspended (q) &&exec_queue_enabled (q)) { -wait_event (guc->ct.wq, (q->guc->resume_time !=RESUME_PENDING || -xe_guc_read_stopped (guc)) && !exec_queue_pending_disable (q)); +wait_event (guc->ct.wq,vf_recovery (guc) || + ((q->guc->resume_time !=RESUME_PENDING || +xe_guc_read_stopped (guc)) && !exec_queue_pending_disable (q)));if (!xe_guc_read_stopped (guc)) { s64 since_resume_ms =@@ -1922,7 +1949,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) q->entity = &ge->entity; -if (xe_guc_read_stopped (guc)) +if (xe_guc_read_stopped (guc) ||vf_recovery (guc))xe_sched_stop (sched);mutex_unlock (&guc->submission_state.lock);@@ -2068,6 +2095,7 @@ static int guc_exec_queue_suspend(struct xe_exec_queue *q)static int guc_exec_queue_suspend_wait (struct xe_exec_queue *q) {struct xe_guc *guc =exec_queue_to_guc (q); +struct xe_device *xe =guc_to_xe (guc);int ret;/* @@ -2075,11 +2103,22 @@ static int guc_exec_queue_suspend_wait(struct xe_exec_queue *q)* suspend_pending upon kill but to be paranoid but races in which * suspend_pending is set after kill also check kill here. */ - ret =wait_event_interruptible_timeout (q->guc->suspend_wait, - !READ_ONCE (q->guc->suspend_pending) || -exec_queue_killed (q) || -xe_guc_read_stopped (guc), -HZ *5 ); +if (IS_SRIOV_VF (xe)) + ret =wait_event_interruptible_timeout (guc->ct.wq, + !READ_ONCE (q->guc->suspend_pending) || +exec_queue_killed (q) || +xe_guc_read_stopped (guc) || +vf_recovery (guc), +HZ *5 ); +else + ret =wait_event_interruptible_timeout (q->guc->suspend_wait, + !READ_ONCE (q->guc->suspend_pending) || +exec_queue_killed (q) || +xe_guc_read_stopped (guc), +HZ *5 ); + +if (vf_recovery (guc) && !xe_device_wedged ((guc_to_xe (guc)))) +return -EAGAIN ;if (!ret) { xe_gt_warn(guc_to_gt(guc),@@ -2187,7 +2226,7 @@ int xe_guc_submit_reset_prepare(struct xe_guc *guc) {int ret;
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c int xe_guc_submit_reset_prepare(struct xe_guc *guc) · line 2229
Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_submit.c +1
-1-if (WARN_ON_ONCE (xe_gt_sriov_vf_recovery_inprogress (guc_to_gt (guc)))) +if (WARN_ON_ONCE (vf_recovery (guc)))
xe_gt_WARN_ONCEis available for use
Done.
Quoted 20 lines
Quoted 20 lines
drivers/gpu/drm/xe/xe_guc_submit.c +0
-0return 0 ;if (!guc->submission_state.initialized)drivers/gpu/drm/xe/xe_preempt_fence.c +2
-0@@ -8,6 +8,8 @@#include <linux/slab.h> #include "xe_exec_queue.h" +#include "xe_gt_printk.h" +#include "xe_guc_exec_queue_types.h" #include "xe_vm.h" static void preempt_fence_work_func (struct work_struct *w)@@ -22,6 +24,15 @@ static void preempt_fence_work_func(struct work_struct *w) }else if (!q->ops->reset_status (q)) {int err = q->ops->suspend_wait (q);
Quoted 10 lines
drivers/gpu/drm/xe/xe_preempt_fence.c static void preempt_fence_work_func(struct work_struct *w) · line 33
Quoted 7 lines
drivers/gpu/drm/xe/xe_preempt_fence.c +7
-0+if (err == -EAGAIN ) { +xe_gt_dbg (q->gt,"PREEMPT FENCE RETRY guc_id=%d" , + q->guc->id); +queue_work (q->vm->xe->preempt_fence_wq, + &pfence->preempt_work); +dma_fence_end_signalling (cookie); +return ;I'm not sure this chunk matches "wakeup GuC backend" commit description
This should be in the following patch. Will move the code.
Matt
Quoted 6 lines
Quoted 5 lines
drivers/gpu/drm/xe/xe_preempt_fence.c +2
-0+ } +if (err)dma_fence_set_error (&pfence->base, err); }else {
Matthew Brost
maintainerRe: [PATCH v4 18/34] drm/xe/vf: Wakeup in GuC backend on VF post migration recovery
Quoted 301 lines · Matthew Brost
Quoted 20 lines · Michal Wajdeczko
Quoted 17 lines · Matthew Brost
If VF post-migration recovery is in progress, the recovery flow will rebuild all GuC submission state. In this case, exit all waiters to ensure that submission queue scheduling can also be paused. Avoid taking any adverse actions after aborting the wait.
v3:
- Don't block in preempt fence work queue as this can interfere with VF post-migration work queue scheduling leading to deadlock (Testing)
- Usexe_gt_recovery_inprogress(Michal)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 3 + drivers/gpu/drm/xe/xe_guc_submit.c | 79 ++++++++++++++++++++------- drivers/gpu/drm/xe/xe_preempt_fence.c | 11 ++++ 3 files changed, 73 insertions(+), 20 deletions(-)Quoted 12 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_start_migration_recovery(struct xe_gt *gt) · line 744
Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1
-0@@ -741,6 +741,9 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); +smp_wmb ();/* Ensure above write visable before wake */ shouldn't this be in separate fixup patch ?
No, this is waking up GuC backend.
Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_start_migration_recovery(struct xe_gt *gt) · line 746
Quoted 2 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-0+ +wake_up_all (>->uc.guc.ct.wq);shouldn't this be wrapped into some CT level helper ?
Sure.
Quoted 19 lines
Quoted 19 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0
-0started =queue_work (gt->ordered_wq, >->sriov.vf.migration.worker); xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?drivers/gpu/drm/xe/xe_guc_submit.c +0
-1@@ -27,7 +27,6 @@#include "xe_gt.h" #include "xe_gt_clock.h" #include "xe_gt_printk.h" -#include "xe_gt_sriov_vf.h" #include "xe_guc.h" #include "xe_guc_capture.h" #include "xe_guc_ct.h" @@ -984,6 +983,9 @@ static u32 wq_space_until_wrap(struct xe_exec_queue *q)return (WQ_SIZE - q->guc->wqi_tail); }Quoted 5 lines
drivers/gpu/drm/xe/xe_guc_submit.c static u32 wq_space_until_wrap(struct xe_exec_queue *q) · line 987
Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-0+#define vf_recovery (guc) \ + xe_gt_recovery_inprogress(guc_to_gt(guc))use static inline helper instead
Done.
Quoted 114 lines
Quoted 114 lines
drivers/gpu/drm/xe/xe_guc_submit.c +31
-9+static int wq_wait_for_space (struct xe_exec_queue *q,u32 wqi_size) {struct xe_guc *guc =exec_queue_to_guc (q);@@ -993,7 +995,7 @@ static int wq_wait_for_space(struct xe_exec_queue *q, u32 wqi_size)#define AVAILABLE_SPACE \ CIRC_SPACE(q->guc->wqi_tail, q->guc->wqi_head, WQ_SIZE) -if (wqi_size >AVAILABLE_SPACE ) { +if (wqi_size >AVAILABLE_SPACE && !vf_recovery (guc)) { try_again: q->guc->wqi_head =parallel_read (xe, map, wq_desc.head);if (wqi_size >AVAILABLE_SPACE ) {@@ -1192,9 +1194,10 @@ static void disable_scheduling_deregister(struct xe_guc *guc, ret =wait_event_timeout (guc->ct.wq, (!exec_queue_pending_enable (q) && !exec_queue_pending_disable (q)) || -xe_guc_read_stopped (guc), +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); -if (!ret) { +if (!ret && !vf_recovery (guc)) {struct xe_gpu_scheduler *sched = &q->guc->sched;xe_gt_warn (q->gt,"Pending enable/disable failed to respond\n" );@@ -1297,6 +1300,10 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)bool wedged = false;xe_gt_assert (guc_to_gt (guc),xe_exec_queue_is_lr (q)); + +if (vf_recovery (guc)) +return ; +trace_xe_exec_queue_lr_cleanup (q);if (!exec_queue_killed (q))@@ -1329,7 +1336,11 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)*/ ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_disable (q) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) +return ; +if (!ret) {xe_gt_warn (q->gt,"Schedule disable failed to respond, guc_id=%d\n" , q->guc->id);@@ -1419,8 +1430,9 @@ static void enable_scheduling(struct xe_exec_queue *q) ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_enable (q) || -xe_guc_read_stopped (guc),HZ *5 ); -if (!ret ||xe_guc_read_stopped (guc)) { +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if ((!ret && !vf_recovery (guc)) ||xe_guc_read_stopped (guc)) {xe_gt_warn (guc_to_gt (guc),"Schedule enable failed to respond" );set_exec_queue_banned (q);xe_gt_reset_async (q->gt);@@ -1491,7 +1503,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)* list so job can be freed and kick scheduler ensuring free job is not * lost. */ -if (test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &job->fence->flags)) +if (test_bit (DMA_FENCE_FLAG_SIGNALED_BIT , &job->fence->flags) || +vf_recovery (guc))return DRM_GPU_SCHED_STAT_NO_HANG ;/* Kill the run_job entry point */ @@ -1543,7 +1556,10 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) ret =wait_event_timeout (guc->ct.wq, (!exec_queue_pending_enable (q) && !exec_queue_pending_disable (q)) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) + goto handle_vf_resume;if (!ret ||xe_guc_read_stopped (guc)) goto trigger_reset;@@ -1568,7 +1584,10 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)smp_rmb (); ret =wait_event_timeout (guc->ct.wq, !exec_queue_pending_disable (q) || -xe_guc_read_stopped (guc),HZ *5 ); +xe_guc_read_stopped (guc) || +vf_recovery (guc),HZ *5 ); +if (vf_recovery (guc)) + goto handle_vf_resume;if (!ret ||xe_guc_read_stopped (guc)) { trigger_reset:if (!ret)@@ -1673,6 +1692,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)* some thought, do this in a follow up. */ xe_sched_submission_start (sched); +handle_vf_resume:return DRM_GPU_SCHED_STAT_NO_HANG ; }@@ -1769,11 +1789,17 @@ static void __guc_exec_queue_process_msg_set_sched_props(struct xe_sched_msg *msstatic void __suspend_fence_signal (struct xe_exec_queue *q) { +struct xe_guc *guc =exec_queue_to_guc (q); +struct xe_device *xe =guc_to_xe (guc); +if (!q->guc->suspend_pending)return ;Quoted 7 lines
drivers/gpu/drm/xe/xe_guc_submit.c static void __guc_exec_queue_process_msg_set_sched_props(struct xe_sched_msg *ms · line 1800
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2
-1WRITE_ONCE (q->guc->suspend_pending, false); -wake_up (&q->guc->suspend_wait); +if (IS_SRIOV_VF (xe)) +wake_up_all (&guc->ct.wq);so now for the all VFs (without recovery) there is now no need to wakeup
suspend_wait?Yes. VF use
&guc->ct.wq(a global wait queue) vs.q->guc->suspend_wait) (dedicated wait queue). This allows a single wake up call invf_start_migration_recovery.Quoted 66 lines
Quoted 66 lines
drivers/gpu/drm/xe/xe_guc_submit.c +23
-8+else +wake_up (&q->guc->suspend_wait); }static void suspend_fence_signal (struct xe_exec_queue *q)@@ -1794,8 +1820,9 @@ static void __guc_exec_queue_process_msg_suspend(struct xe_sched_msg *msg)if (guc_exec_queue_allowed_to_change_state (q) && !exec_queue_suspended (q) &&exec_queue_enabled (q)) { -wait_event (guc->ct.wq, (q->guc->resume_time !=RESUME_PENDING || -xe_guc_read_stopped (guc)) && !exec_queue_pending_disable (q)); +wait_event (guc->ct.wq,vf_recovery (guc) || + ((q->guc->resume_time !=RESUME_PENDING || +xe_guc_read_stopped (guc)) && !exec_queue_pending_disable (q)));if (!xe_guc_read_stopped (guc)) { s64 since_resume_ms =@@ -1922,7 +1949,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) q->entity = &ge->entity; -if (xe_guc_read_stopped (guc)) +if (xe_guc_read_stopped (guc) ||vf_recovery (guc))xe_sched_stop (sched);mutex_unlock (&guc->submission_state.lock);@@ -2068,6 +2095,7 @@ static int guc_exec_queue_suspend(struct xe_exec_queue *q)static int guc_exec_queue_suspend_wait (struct xe_exec_queue *q) {struct xe_guc *guc =exec_queue_to_guc (q); +struct xe_device *xe =guc_to_xe (guc);int ret;/* @@ -2075,11 +2103,22 @@ static int guc_exec_queue_suspend_wait(struct xe_exec_queue *q)* suspend_pending upon kill but to be paranoid but races in which * suspend_pending is set after kill also check kill here. */ - ret =wait_event_interruptible_timeout (q->guc->suspend_wait, - !READ_ONCE (q->guc->suspend_pending) || -exec_queue_killed (q) || -xe_guc_read_stopped (guc), -HZ *5 ); +if (IS_SRIOV_VF (xe)) + ret =wait_event_interruptible_timeout (guc->ct.wq, + !READ_ONCE (q->guc->suspend_pending) || +exec_queue_killed (q) || +xe_guc_read_stopped (guc) || +vf_recovery (guc), +HZ *5 ); +else + ret =wait_event_interruptible_timeout (q->guc->suspend_wait, + !READ_ONCE (q->guc->suspend_pending) || +exec_queue_killed (q) || +xe_guc_read_stopped (guc), +HZ *5 ); + +if (vf_recovery (guc) && !xe_device_wedged ((guc_to_xe (guc)))) +return -EAGAIN ;if (!ret) { xe_gt_warn(guc_to_gt(guc),@@ -2187,7 +2226,7 @@ int xe_guc_submit_reset_prepare(struct xe_guc *guc) {int ret;Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_submit.c int xe_guc_submit_reset_prepare(struct xe_guc *guc) · line 2229
Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_submit.c +1
-1-if (WARN_ON_ONCE (xe_gt_sriov_vf_recovery_inprogress (guc_to_gt (guc)))) +if (WARN_ON_ONCE (vf_recovery (guc)))
xe_gt_WARN_ONCEis available for useDone.
Quoted 20 lines
Quoted 20 lines
drivers/gpu/drm/xe/xe_guc_submit.c +0
-0return 0 ;if (!guc->submission_state.initialized)drivers/gpu/drm/xe/xe_preempt_fence.c +2
-0@@ -8,6 +8,8 @@#include <linux/slab.h> #include "xe_exec_queue.h" +#include "xe_gt_printk.h" +#include "xe_guc_exec_queue_types.h" #include "xe_vm.h" static void preempt_fence_work_func (struct work_struct *w)@@ -22,6 +24,15 @@ static void preempt_fence_work_func(struct work_struct *w) }else if (!q->ops->reset_status (q)) {int err = q->ops->suspend_wait (q);Quoted 10 lines
drivers/gpu/drm/xe/xe_preempt_fence.c static void preempt_fence_work_func(struct work_struct *w) · line 33
Quoted 7 lines
drivers/gpu/drm/xe/xe_preempt_fence.c +7
-0+if (err == -EAGAIN ) { +xe_gt_dbg (q->gt,"PREEMPT FENCE RETRY guc_id=%d" , + q->guc->id); +queue_work (q->vm->xe->preempt_fence_wq, + &pfence->preempt_work); +dma_fence_end_signalling (cookie); +return ;I'm not sure this chunk matches "wakeup GuC backend" commit description
Quoted 2 lines
This should be in the following patch. Will move the code.
Actually this is related to this patch. As part of the waking GuC backend, suspend_wait can return -EAGAIN which needs to requeue preempt fence worker to avoid deadlock. Let me see if I can adjust the commit message.
Matt
Quoted 8 lines
Matt
Quoted 6 lines
Quoted 5 lines
drivers/gpu/drm/xe/xe_preempt_fence.c +2
-0+ } +if (err)dma_fence_set_error (&pfence->base, err); }else {
Matthew Brost
maintainerPATCH v4 19/34 drm/xe/vf: Avoid indefinite blocking in preempt rebind worker for VFs supporting migration
In reply to Matthew BrostBlocking in work queues on a hardware action that may never occur — especially when it depends on a software fixup also scheduled on the a work queue — is a recipe for deadlock. This situation arises with the preempt rebind worker and VF post-migration recovery. To prevent potential deadlocks, avoid indefinite blocking in the preempt rebind worker for VFs that support migration.
v4:
- Use dma_fence_wait_timeout (CI)
--- drivers/gpu/drm/xe/xe_vm.c | 26 +++++++++++++++++++++++++- 1 file changed, 25 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_vm.c +25 -1
@@ -35,6 +35,7 @@#include "xe_pt.h" #include "xe_pxp.h" #include "xe_res_cursor.h" +#include "xe_sriov_vf.h" #include "xe_svm.h" #include "xe_sync.h" #include "xe_tile.h"
@@ -111,12 +112,22 @@ static int alloc_preempt_fences(struct xe_vm *vm, struct list_head *list,static int wait_for_existing_preempt_fences (struct xe_vm *vm) {struct xe_exec_queue *q; +bool vf_migration =IS_SRIOV_VF (vm->xe) && +xe_sriov_vf_migration_supported (vm->xe); +signed long wait_time = vf_migration ?HZ /5 :MAX_SCHEDULE_TIMEOUT ;xe_vm_assert_held (vm);list_for_each_entry (q, &vm->preempt.exec_queues, lr.link) {if (q->lr.pfence) { -long timeout =dma_fence_wait (q->lr.pfence, false); +long timeout; + + timeout =dma_fence_wait_timeout (q->lr.pfence, false, + wait_time); +if (!timeout) { +xe_assert (vm->xe, vf_migration); +return -EAGAIN ; + }/* Only -ETIME on fence indicates VM needs to be killed */ if (timeout <0 || q->lr.pfence->error == -ETIME )
@@ -541,6 +552,19 @@ static void preempt_rebind_work_func(struct work_struct *w) out_unlock_outer:if (err == -EAGAIN ) {trace_xe_vm_rebind_worker_retry (vm); + +/* +* We can't block in workers on a VF which supports migration +* given this can block the VF post-migration workers from +* getting scheduled. +*/ +if (IS_SRIOV_VF (vm->xe) && +xe_sriov_vf_migration_supported (vm->xe)) { +up_write (&vm->lock); +xe_vm_queue_rebind_worker (vm); +return ; + } + goto retry; }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 20/34 drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register
In reply to Matthew BrostThe only case where the GuC submission backend cannot reason 100% correctly is when a GuC context is registered during VF post-migration recovery. In this scenario, it's possible that the GuC context register H2G is processed, but the immediately following schedule-enable H2G gets lost.
A double register is harmless when using GUC_HXG_TYPE_EVENT, as GuC simply drops the duplicate H2G. To keep things simple, use GUC_HXG_TYPE_EVENT for all context registrations on VFs.
--- drivers/gpu/drm/xe/xe_guc_ct.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +24 -8
@@ -736,6 +736,26 @@ static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence)return seqno; } +#define MAKE_ACTION (type, __action) \ +({ \ + FIELD_PREP(GUC_HXG_MSG_0_TYPE, type) | \ + FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION | \ + GUC_HXG_EVENT_MSG_0_DATA0, __action); \ +}) + +static bool vf_action_can_safely_fail (struct xe_device *xe,u32 action) +{ +/* +* If we are VF resuming, we can't exactly track if a context +* registration has been completed in the GuC state machine, it is +* harmless to resend as it will just fail silently if +* GUC_HXG_TYPE_EVENT is used. +*/ +return IS_SRIOV_VF (xe) && + (action ==XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC || + action ==XE_GUC_ACTION_REGISTER_CONTEXT ); +} +#define H2G_CT_HEADERS (GUC_CTB_HDR_LEN + 1)/* one DW CTB header and one DW HxG header */ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
@@ -807,18 +827,14 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,FIELD_PREP (GUC_CTB_MSG_0_NUM_DWORDS , len) |FIELD_PREP (GUC_CTB_MSG_0_FENCE , ct_fence_value);if (want_response) { - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_REQUEST , action[0 ]); + }else if (vf_action_can_safely_fail (xe, action[0 ])) { + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_EVENT , action[0 ]); }else {fast_req_track (ct, ct_fence_value,FIELD_GET (GUC_HXG_EVENT_MSG_0_ACTION , action[0 ])); - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_FAST_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_FAST_REQUEST , action[0 ]); }/* H2G header in cmd[1] replaces action[0] so: */
Signature · 1 line
2.34.1
Re: [PATCH v4 20/34] drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register
Quoted 5 lines · Matthew Brost
The only case where the GuC submission backend cannot reason 100% correctly is when a GuC context is registered during VF post-migration recovery. In this scenario, it's possible that the GuC context register H2G is processed, but the immediately following schedule-enable H2G gets lost.
Shouldn't the solution then be to treat the schedule-enable H2G as separate state, and be able to revert to it?
Quoted 16 lines
A double register is harmless when using
GUC_HXG_TYPE_EVENT, as GuC simply drops the duplicate H2G. To keep things simple, useGUC_HXG_TYPE_EVENTfor all context registrations on VFs.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_guc_ct.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-)drivers/gpu/drm/xe/xe_guc_ct.c +0
-0@@ -736,6 +736,26 @@ static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence)return seqno;
drivers/gpu/drm/xe/xe_guc_ct.c static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence) · line 746
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +8
-0} +#define MAKE_ACTION (type, __action) \ +({ \ + FIELD_PREP(GUC_HXG_MSG_0_TYPE, type) | \ + FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION | \ + GUC_HXG_EVENT_MSG_0_DATA0, __action); \ +}) + +static bool vf_action_can_safely_fail (struct xe_device *xe,u32 action)
Something is very wrong with that name. Context registration can't "safely fail".
drivers/gpu/drm/xe/xe_guc_ct.c static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence) · line 756
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +10
-0+{ +/* +* If we are VF resuming, we can't exactly track if a context +* registration has been completed in the GuC state machine, it is +* harmless to resend as it will just fail silently if +* GUC_HXG_TYPE_EVENT is used. +*/ +return IS_SRIOV_VF (xe) && + (action ==XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC || + action ==XE_GUC_ACTION_REGISTER_CONTEXT );
Shouldn't we at the very least limit that to vf migration enabled?
Looks to me like we're completely ripping out context registration errors propagation. From the kalloc() fixup patches, I was under impression that error support and propagation is important for Xe. And here, we're willingly damaging it?
Can we really ignore them? If we really must, shouldn't we mimic GuC checks here on the KMD side, to at least decrease probability of problems?
This will cause possible error messages in GuC log, which do not indicate real errors. We're willingly sending a message which may result in an error.
Having error messages in GuC log which are considered normal flow on KMD side would have to be properly documented, to avoid people debugging non-existing issues. Best in several places.
I don't know about this, we're creating a bad situation by choice.
-Tomasz
Quoted 28 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6
-8+} +#define H2G_CT_HEADERS (GUC_CTB_HDR_LEN + 1)/* one DW CTB header and one DW HxG header */ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,@@ -807,18 +827,14 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,FIELD_PREP (GUC_CTB_MSG_0_NUM_DWORDS , len) |FIELD_PREP (GUC_CTB_MSG_0_FENCE , ct_fence_value);if (want_response) { - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_REQUEST , action[0 ]); + }else if (vf_action_can_safely_fail (xe, action[0 ])) { + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_EVENT , action[0 ]); }else {fast_req_track (ct, ct_fence_value,FIELD_GET (GUC_HXG_EVENT_MSG_0_ACTION , action[0 ])); - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_FAST_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_FAST_REQUEST , action[0 ]); }/* H2G header in cmd[1] replaces action[0] so: */
Matthew Brost
maintainerRe: [PATCH v4 20/34] drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register
Quoted 28 lines · Tomasz
Quoted 5 lines · Matthew Brost
The only case where the GuC submission backend cannot reason 100% correctly is when a GuC context is registered during VF post-migration recovery. In this scenario, it's possible that the GuC context register H2G is processed, but the immediately following schedule-enable H2G gets lost.
Shouldn't the solution then be to treat the schedule-enable H2G as separate state, and be able to revert to it?
Quoted 19 lines
A double register is harmless when using
GUC_HXG_TYPE_EVENT, as GuC simply drops the duplicate H2G. To keep things simple, useGUC_HXG_TYPE_EVENTfor all context registrations on VFs.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_guc_ct.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-)drivers/gpu/drm/xe/xe_guc_ct.c +2
-0@@ -736,6 +736,26 @@ static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence)return seqno; } +#define MAKE_ACTION (type, __action) \ +({ \
Quoted 8 lines
drivers/gpu/drm/xe/xe_guc_ct.c static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence) · line 746
Quoted 6 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6
-0+FIELD_PREP (GUC_HXG_MSG_0_TYPE , type) | \ +FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | \ +GUC_HXG_EVENT_MSG_0_DATA0 , __action); \ +}) + +static bool vf_action_can_safely_fail (struct xe_device *xe,u32 action)Something is very wrong with that name. Context registration can't "safely fail".
What? This name pefectly describes what this doing - these VF actions can safely fail in the GuC.
Quoted 4 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_ct.c +4
-0+{ +/* +* If we are VF resuming, we can't exactly track if a context +* registration has been completed in the GuC state machine, it is
Quoted 9 lines
drivers/gpu/drm/xe/xe_guc_ct.c static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence) · line 756
Quoted 6 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6
-0+* harmless to resend as it will just fail silently if +* GUC_HXG_TYPE_EVENT is used. +*/ +return IS_SRIOV_VF (xe) && + (action ==XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC || + action ==XE_GUC_ACTION_REGISTER_CONTEXT );Shouldn't we at the very least limit that to vf migration enabled?
Sure.
Quoted 5 lines
Looks to me like we're completely ripping out context registration errors propagation. From the
kalloc()fixup patches, I was under impression that error support and propagation is important for Xe. And here, we're willingly damaging it?
This is fundamentally different than omitting a bug check which crashes the kernel.
Quoted 6 lines
Can we really ignore them? If we really must, shouldn't we mimic GuC checks here on the KMD side, to at least decrease probability of problems?
This will cause possible error messages in GuC log, which do not indicate real errors. We're willingly sending a message which may result in an error.
Quoted 7 lines
Having error messages in GuC log which are considered normal flow on KMD side would have to be properly documented, to avoid people debugging non-existing issues. Best in several places.
I don't know about this, we're creating a bad situation by choice.
I've already explained this. Not ideal but I don't see any better options. If we remove this a fast req can randomly fail with the right race condition trashing the VF as that results in a GT reset.
This error check is protocol level check in Xe. If we had protocol problem with H2G it would show up all platforms in our CI. We are not losing any coverage as this check is enabled everywhere but a VF.
Matt
Quoted 27 lines
-Tomasz
Quoted 25 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6
-8+} +#define H2G_CT_HEADERS (GUC_CTB_HDR_LEN + 1)/* one DW CTB header and one DW HxG header */ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,@@ -807,18 +827,14 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,FIELD_PREP (GUC_CTB_MSG_0_NUM_DWORDS , len) |FIELD_PREP (GUC_CTB_MSG_0_FENCE , ct_fence_value);if (want_response) { - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_REQUEST , action[0 ]); + }else if (vf_action_can_safely_fail (xe, action[0 ])) { + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_EVENT , action[0 ]); }else {fast_req_track (ct, ct_fence_value,FIELD_GET (GUC_HXG_EVENT_MSG_0_ACTION , action[0 ])); - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_FAST_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_FAST_REQUEST , action[0 ]); }/* H2G header in cmd[1] replaces action[0] so: */
Re: [PATCH v4 20/34] drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register
In reply to Matthew BrostQuoted 24 lines · Matthew Brost
The only case where the GuC submission backend cannot reason 100% correctly is when a GuC context is registered during VF post-migration recovery. In this scenario, it's possible that the GuC context register H2G is processed, but the immediately following schedule-enable H2G gets lost.
A double register is harmless when using
GUC_HXG_TYPE_EVENT, as GuC simply drops the duplicate H2G. To keep things simple, useGUC_HXG_TYPE_EVENTfor all context registrations on VFs.Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_guc_ct.c | 32 ++++++++++++++++++++++++-------- 1 file changed, 24 insertions(+), 8 deletions(-)drivers/gpu/drm/xe/xe_guc_ct.c +1
-0@@ -736,6 +736,26 @@ static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence)return seqno; } +#define MAKE_ACTION (type, __action) \
drivers/gpu/drm/xe/xe_guc_ct.c static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence) · line 749
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +10
-0+({ \ +FIELD_PREP (GUC_HXG_MSG_0_TYPE , type) | \ +FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | \ +GUC_HXG_EVENT_MSG_0_DATA0 , __action); \ +}) + +static bool vf_action_can_safely_fail (struct xe_device *xe,u32 action) +{ +/* +* If we are VF resuming, we can't exactly track if a context
s/resuming/recovering
drivers/gpu/drm/xe/xe_guc_ct.c static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence) · line 750
Quoted 1 line
drivers/gpu/drm/xe/xe_guc_ct.c +1
-0+* registration has been completed in the GuC state machine, it is
well, we can (by looking at H2G if that was processed) but to "simplify" we don't
drivers/gpu/drm/xe/xe_guc_ct.c static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence) · line 754
Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_ct.c +4
-0+* harmless to resend as it will just fail silently if +* GUC_HXG_TYPE_EVENT is used. +*/ + return IS_SRIOV_VF(xe) &&
maybe also:
xe_gt_recovery_inprogress(gt) &&
to limit our trick to recovery only ?
Quoted 30 lines
drivers/gpu/drm/xe/xe_guc_ct.c +8
-8+ (action ==XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC || + action ==XE_GUC_ACTION_REGISTER_CONTEXT ); +} +#define H2G_CT_HEADERS (GUC_CTB_HDR_LEN + 1)/* one DW CTB header and one DW HxG header */ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,@@ -807,18 +827,14 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,FIELD_PREP (GUC_CTB_MSG_0_NUM_DWORDS , len) |FIELD_PREP (GUC_CTB_MSG_0_FENCE , ct_fence_value);if (want_response) { - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_REQUEST , action[0 ]); + }else if (vf_action_can_safely_fail (xe, action[0 ])) { + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_EVENT , action[0 ]); }else {fast_req_track (ct, ct_fence_value,FIELD_GET (GUC_HXG_EVENT_MSG_0_ACTION , action[0 ])); - cmd[1 ] = -FIELD_PREP (GUC_HXG_MSG_0_TYPE ,GUC_HXG_TYPE_FAST_REQUEST ) | -FIELD_PREP (GUC_HXG_EVENT_MSG_0_ACTION | -GUC_HXG_EVENT_MSG_0_DATA0 , action[0 ]); + cmd[1 ] =MAKE_ACTION (GUC_HXG_TYPE_FAST_REQUEST , action[0 ]); }/* H2G header in cmd[1] replaces action[0] so: */
Matthew Brost
maintainerPATCH v4 21/34 drm/xe/vf: Flush and stop CTs in VF post migration recovery
In reply to Matthew BrostFlushing CTs (i.e., progressing all pending G2H messages) gives VF post-migration recovery an accurate view of which H2G messages the GuC has processed, enabling the GuC submission state machine to correctly rebuild all state.
Also, stop all CT traffic, as the CT is not live during VF post-migration recovery.
v3:
- xe_guc_ct_flush_and_stop rename (Michal)
- Drop extra GuC CT WQ wake up (Michal)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 2 ++ drivers/gpu/drm/xe/xe_guc_ct.c | 10 ++++++++++ drivers/gpu/drm/xe/xe_guc_ct.h | 1 + 3 files changed, 13 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
@@ -23,6 +23,7 @@#include "xe_gt_sriov_vf.h" #include "xe_gt_sriov_vf_types.h" #include "xe_guc.h" +#include "xe_guc_ct.h" #include "xe_guc_hxg_helpers.h" #include "xe_guc_relay.h" #include "xe_guc_submit.h"
@@ -1108,6 +1109,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt) gt->sriov.vf.migration.recovery_queued = false;spin_unlock_irq (>->sriov.vf.migration.lock); +xe_guc_ct_flush_and_stop (>->uc.guc.ct);xe_guc_submit_pause (>->uc.guc); }
drivers/gpu/drm/xe/xe_guc_ct.c +10 -0
@@ -573,6 +573,16 @@ void xe_guc_ct_disable(struct xe_guc_ct *ct)stop_g2h_handler (ct); } +/** +* xe_guc_ct_flush_and_stop - Flush and stop all processing of G2H / H2G +* @ct: the &xe_guc_ct +*/ +void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct) +{ +receive_g2h (ct); +xe_guc_ct_stop (ct); +} +/** * xe_guc_ct_stop - Set GuC to stopped state * @ct: the &xe_guc_ct
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
@@ -17,6 +17,7 @@ int xe_guc_ct_init_post_hwconfig(struct xe_guc_ct *ct);int xe_guc_ct_enable (struct xe_guc_ct *ct);void xe_guc_ct_disable (struct xe_guc_ct *ct);void xe_guc_ct_stop (struct xe_guc_ct *ct); +void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct);void xe_guc_ct_fast_path (struct xe_guc_ct *ct);struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture (struct xe_guc_ct *ct);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 22/34 drm/xe/vf: Reset TLB invalidations during VF post migration recovery
In reply to Matthew BrostTLB invalidations requests can be lost during VF post-migration recovery. Since the VF has migrated, these invalidations are no longer needed.
Reset the TLB invalidation frontend, which will signal all pending fences.
v3:
- Move TLB invalidation reset after pausing submission (Tomasz)
- Adjust commit message (Michal)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 2 ++ 1 file changed, 2 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
@@ -35,6 +35,7 @@#include "xe_sriov.h" #include "xe_sriov_vf.h" #include "xe_tile_sriov_vf.h" +#include "xe_tlb_inval.h" #include "xe_uc_fw.h" #include "xe_wopcm.h"
@@ -1111,6 +1112,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)xe_guc_ct_flush_and_stop (>->uc.guc.ct);xe_guc_submit_pause (>->uc.guc); +xe_tlb_inval_reset (>->tlb_inval); }static size_t post_migration_scratch_size (struct xe_device *xe)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 23/34 drm/xe/vf: Kickstart after resfix in VF post migration recovery
In reply to Matthew BrostGuC needs to be live for the GuC submission state machine to resubmit anything lost during VF post-migration recovery. Therefore, move the kickstart step after resfix to ensure proper resubmission.
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +9 -8
@@ -1139,13 +1139,6 @@ static int vf_post_migration_fixups(struct xe_gt *gt)static void vf_post_migration_kickstart (struct xe_gt *gt) { -/* -* Make sure interrupts on the new HW are properly set. The GuC IRQ -* must be working at this point, since the recovery did started, -* but the rest was not enabled using the procedure from spec. -*/ -xe_irq_resume (gt_to_xe (gt)); -xe_guc_submit_unpause (>->uc.guc); }
@@ -1165,6 +1158,13 @@ static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)if (skip_resfix)return -EAGAIN ; +/* +* Make sure interrupts on the new HW are properly set. The GuC IRQ +* must be working at this point, since the recovery did started, +* but the rest was not enabled using the procedure from spec. +*/ +xe_irq_resume (gt_to_xe (gt)); +return vf_notify_resfix_done (gt); }
@@ -1188,11 +1188,12 @@ static void vf_post_migration_recovery(struct xe_gt *gt)if (err) goto fail; -vf_post_migration_kickstart (gt); err =vf_post_migration_notify_resfix_done (gt);if (err && err != -EAGAIN ) goto fail; +vf_post_migration_kickstart (gt); +xe_pm_runtime_put (xe);xe_gt_sriov_notice (gt,"migration recovery ended\n" );return ;
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 24/34 drm/xe/vf: Start CTs before resfix VF post migration recovery
In reply to Matthew BrostBefore resfix, all CTs stuck in the H2G queue need to be squashed, as they may contain stale or invalid data.
Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.
v3:
- Don't mess with head / tail values (Michal)
v4:
- Don't mess with broke (Michal)
- Add CTB_H2G_BUFFER_OFFSET (Michal)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 +++ drivers/gpu/drm/xe/xe_guc_ct.c | 70 +++++++++++++++++++++-------- drivers/gpu/drm/xe/xe_guc_ct.h | 1 + 3 files changed, 60 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7 -0
@@ -1137,6 +1137,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)return 0 ; } +static void vf_post_migration_rearm (struct xe_gt *gt) +{ +xe_guc_ct_restart (>->uc.guc.ct); +} +static void vf_post_migration_kickstart (struct xe_gt *gt) {xe_guc_submit_unpause (>->uc.guc);
@@ -1188,6 +1193,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)if (err) goto fail; +vf_post_migration_rearm (gt); + err =vf_post_migration_notify_resfix_done (gt);if (err && err != -EAGAIN ) goto fail;
drivers/gpu/drm/xe/xe_guc_ct.c +52 -18
127 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);int xe_guc_ct_init (struct xe_guc_ct *ct);int xe_guc_ct_init_post_hwconfig (struct xe_guc_ct *ct);int xe_guc_ct_enable (struct xe_guc_ct *ct); +int xe_guc_ct_restart (struct xe_guc_ct *ct);void xe_guc_ct_disable (struct xe_guc_ct *ct);void xe_guc_ct_stop (struct xe_guc_ct *ct);void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct);
Signature · 1 line
2.34.1
Re: [PATCH v4 24/34] drm/xe/vf: Start CTs before resfix VF post migration recovery
Quoted 1 line · Matthew Brost
Before
resfix, all CTs stuck in the H2G queue need to be squashed, as
Can we expand the shortcut here to "resource fixups"?
It's ok in patch name, as long as it's expanded here. Otherwise, it suggests we're talking about GuC states, as that it their name (RESFIX_BLOCKED and RESFIX_PAUSED).
Quoted 1 line
they may contain stale or invalid data.
maybe more detail:
``` they may contain actions which contain invalid GGTT references or are unnecessary after HW change. ```
-Tomasz
Quoted 194 lines
Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.
v3:
- Don't mess with head / tail values (Michal)
v4:
- Don't mess with broke (Michal)
- AddCTB_H2G_BUFFER_OFFSET(Michal)Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 +++ drivers/gpu/drm/xe/xe_guc_ct.c | 70 +++++++++++++++++++++-------- drivers/gpu/drm/xe/xe_guc_ct.h | 1 + 3 files changed, 60 insertions(+), 18 deletions(-)drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7
-0@@ -1137,6 +1137,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)return 0 ; } +static void vf_post_migration_rearm (struct xe_gt *gt) +{ +xe_guc_ct_restart (>->uc.guc.ct); +} +static void vf_post_migration_kickstart (struct xe_gt *gt) {xe_guc_submit_unpause (>->uc.guc);@@ -1188,6 +1193,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)if (err) goto fail; +vf_post_migration_rearm (gt); + err =vf_post_migration_notify_resfix_done (gt);if (err && err != -EAGAIN ) goto fail;drivers/gpu/drm/xe/xe_guc_ct.c +52
-18@@ -166,6 +166,7 @@ ct_to_xe(struct xe_guc_ct *ct)*/ #define CTB_DESC_SIZE ALIGN(sizeof(struct guc_ct_buffer_desc), SZ_2K) +#define CTB_H2G_BUFFER_OFFSET (CTB_DESC_SIZE * 2)#define CTB_H2G_BUFFER_SIZE (SZ_4K)#define CTB_G2H_BUFFER_SIZE (SZ_128K)#define G2H_ROOM_BUFFER_SIZE (CTB_G2H_BUFFER_SIZE / 2)@@ -189,7 +190,7 @@ long xe_guc_ct_queue_proc_time_jiffies(struct xe_guc_ct *ct)static size_t guc_ct_size (void ) { -return 2 *CTB_DESC_SIZE +CTB_H2G_BUFFER_SIZE + +return CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE +CTB_G2H_BUFFER_SIZE ; }@@ -330,7 +331,7 @@ static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g, h2g->desc = *map;xe_map_memset (xe, &h2g->desc,0 ,0 ,sizeof (struct guc_ct_buffer_desc )); - h2g->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE *2 ); + h2g->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_H2G_BUFFER_OFFSET ); } static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h,@@ -348,7 +349,7 @@ static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h, g2h->desc =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE );xe_map_memset (xe, &g2h->desc,0 ,0 ,sizeof (struct guc_ct_buffer_desc )); - g2h->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE *2 + + g2h->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE ); }@@ -359,7 +360,7 @@ static int guc_ct_ctb_h2g_register(struct xe_guc_ct *ct)int err; desc_addr =xe_bo_ggtt_addr (ct->bo); - ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE *2 ; + ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_H2G_BUFFER_OFFSET ; size = ct->ctbs.h2g.info.size *sizeof (u32); err = xe_guc_self_cfg64(guc,@@ -386,7 +387,7 @@ static int guc_ct_ctb_g2h_register(struct xe_guc_ct *ct)int err; desc_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE ; - ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE *2 + + ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE ; size = ct->ctbs.g2h.info.size *sizeof (u32);@@ -500,7 +501,7 @@ static void ct_exit_safe_mode(struct xe_guc_ct *ct)xe_gt_dbg (ct_to_gt (ct),"GuC CT safe-mode disabled\n" ); } -int xe_guc_ct_enable (struct xe_guc_ct *ct) +static int __xe_guc_ct_start (struct xe_guc_ct *ct,bool needs_register) {struct xe_device *xe =ct_to_xe (ct);struct xe_gt *gt =ct_to_gt (ct);@@ -508,21 +509,28 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)xe_gt_assert (gt, !xe_guc_ct_enabled (ct)); -xe_map_memset (xe, &ct->bo->vmap,0 ,0 ,xe_bo_size (ct->bo)); -guc_ct_ctb_h2g_init (xe, &ct->ctbs.h2g, &ct->bo->vmap); -guc_ct_ctb_g2h_init (xe, &ct->ctbs.g2h, &ct->bo->vmap); +if (needs_register) { +xe_map_memset (xe, &ct->bo->vmap,0 ,0 ,xe_bo_size (ct->bo)); +guc_ct_ctb_h2g_init (xe, &ct->ctbs.h2g, &ct->bo->vmap); +guc_ct_ctb_g2h_init (xe, &ct->ctbs.g2h, &ct->bo->vmap); - err =guc_ct_ctb_h2g_register (ct); -if (err) - goto err_out; + err =guc_ct_ctb_h2g_register (ct); +if (err) + goto err_out; - err =guc_ct_ctb_g2h_register (ct); -if (err) - goto err_out; + err =guc_ct_ctb_g2h_register (ct); +if (err) + goto err_out; - err =guc_ct_control_toggle (ct, true); -if (err) - goto err_out; + err =guc_ct_control_toggle (ct, true); +if (err) + goto err_out; + }else { + ct->ctbs.h2g.info.broken = false; + ct->ctbs.g2h.info.broken = false; +xe_map_memset (xe, &ct->bo->vmap,CTB_H2G_BUFFER_OFFSET ,0 , +CTB_H2G_BUFFER_SIZE ); + }guc_ct_change_state (ct,XE_GUC_CT_STATE_ENABLED );@@ -554,6 +562,32 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)return err; } +/** +* xe_guc_ct_restart() - Restart GuC CT +* @ct: the &xe_guc_ct +* +* Restart GuC CT to an empty state without issuing a CT register MMIO command. +* +* Return: 0 on success, or a negative errno on failure. +*/ +int xe_guc_ct_restart (struct xe_guc_ct *ct) +{ +return __xe_guc_ct_start (ct, false); +} + +/** +* xe_guc_ct_enable() - Enable GuC CT +* @ct: the &xe_guc_ct +* +* Enable GuC CT to an empty state and issue a CT register MMIO command. +* +* Return: 0 on success, or a negative errno on failure. +*/ +int xe_guc_ct_enable (struct xe_guc_ct *ct) +{ +return __xe_guc_ct_start (ct, true); +} +static void stop_g2h_handler (struct xe_guc_ct *ct) {cancel_work_sync (&ct->g2h_worker);drivers/gpu/drm/xe/xe_guc_ct.h +1
-0@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);int xe_guc_ct_init (struct xe_guc_ct *ct);int xe_guc_ct_init_post_hwconfig (struct xe_guc_ct *ct);int xe_guc_ct_enable (struct xe_guc_ct *ct); +int xe_guc_ct_restart (struct xe_guc_ct *ct);void xe_guc_ct_disable (struct xe_guc_ct *ct);void xe_guc_ct_stop (struct xe_guc_ct *ct);void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct);
Re: [PATCH v4 24/34] drm/xe/vf: Start CTs before resfix VF post migration recovery
In reply to Matthew BrostQuoted 11 lines · Matthew Brost
Before
resfix, all CTs stuck in the H2G queue need to be squashed, as they may contain stale or invalid data.Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.
v3:
- Don't mess with head / tail values (Michal)
v4:
- Don't mess with broke (Michal)
- AddCTB_H2G_BUFFER_OFFSET(Michal)
I guess those small fixes shall be done separately
Quoted 126 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 +++ drivers/gpu/drm/xe/xe_guc_ct.c | 70 +++++++++++++++++++++-------- drivers/gpu/drm/xe/xe_guc_ct.h | 1 + 3 files changed, 60 insertions(+), 18 deletions(-)drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7
-0@@ -1137,6 +1137,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)return 0 ; } +static void vf_post_migration_rearm (struct xe_gt *gt) +{ +xe_guc_ct_restart (>->uc.guc.ct); +} +static void vf_post_migration_kickstart (struct xe_gt *gt) {xe_guc_submit_unpause (>->uc.guc);@@ -1188,6 +1193,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)if (err) goto fail; +vf_post_migration_rearm (gt); + err =vf_post_migration_notify_resfix_done (gt);if (err && err != -EAGAIN ) goto fail;drivers/gpu/drm/xe/xe_guc_ct.c +17
-16@@ -166,6 +166,7 @@ ct_to_xe(struct xe_guc_ct *ct)*/ #define CTB_DESC_SIZE ALIGN(sizeof(struct guc_ct_buffer_desc), SZ_2K) +#define CTB_H2G_BUFFER_OFFSET (CTB_DESC_SIZE * 2)#define CTB_H2G_BUFFER_SIZE (SZ_4K)#define CTB_G2H_BUFFER_SIZE (SZ_128K)#define G2H_ROOM_BUFFER_SIZE (CTB_G2H_BUFFER_SIZE / 2)@@ -189,7 +190,7 @@ long xe_guc_ct_queue_proc_time_jiffies(struct xe_guc_ct *ct)static size_t guc_ct_size (void ) { -return 2 *CTB_DESC_SIZE +CTB_H2G_BUFFER_SIZE + +return CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE +CTB_G2H_BUFFER_SIZE ; }@@ -330,7 +331,7 @@ static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g, h2g->desc = *map;xe_map_memset (xe, &h2g->desc,0 ,0 ,sizeof (struct guc_ct_buffer_desc )); - h2g->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE *2 ); + h2g->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_H2G_BUFFER_OFFSET ); } static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h,@@ -348,7 +349,7 @@ static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h, g2h->desc =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE );xe_map_memset (xe, &g2h->desc,0 ,0 ,sizeof (struct guc_ct_buffer_desc )); - g2h->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE *2 + + g2h->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE ); }@@ -359,7 +360,7 @@ static int guc_ct_ctb_h2g_register(struct xe_guc_ct *ct)int err; desc_addr =xe_bo_ggtt_addr (ct->bo); - ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE *2 ; + ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_H2G_BUFFER_OFFSET ; size = ct->ctbs.h2g.info.size *sizeof (u32); err = xe_guc_self_cfg64(guc,@@ -386,7 +387,7 @@ static int guc_ct_ctb_g2h_register(struct xe_guc_ct *ct)int err; desc_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE ; - ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE *2 + + ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE ; size = ct->ctbs.g2h.info.size *sizeof (u32);@@ -500,7 +501,7 @@ static void ct_exit_safe_mode(struct xe_guc_ct *ct)xe_gt_dbg (ct_to_gt (ct),"GuC CT safe-mode disabled\n" ); } -int xe_guc_ct_enable (struct xe_guc_ct *ct) +static int __xe_guc_ct_start (struct xe_guc_ct *ct,bool needs_register) {struct xe_device *xe =ct_to_xe (ct);struct xe_gt *gt =ct_to_gt (ct);@@ -508,21 +509,28 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)xe_gt_assert (gt, !xe_guc_ct_enabled (ct)); -xe_map_memset (xe, &ct->bo->vmap,0 ,0 ,xe_bo_size (ct->bo)); -guc_ct_ctb_h2g_init (xe, &ct->ctbs.h2g, &ct->bo->vmap); -guc_ct_ctb_g2h_init (xe, &ct->ctbs.g2h, &ct->bo->vmap); +if (needs_register) { +xe_map_memset (xe, &ct->bo->vmap,0 ,0 ,xe_bo_size (ct->bo)); +guc_ct_ctb_h2g_init (xe, &ct->ctbs.h2g, &ct->bo->vmap); +guc_ct_ctb_g2h_init (xe, &ct->ctbs.g2h, &ct->bo->vmap); - err =guc_ct_ctb_h2g_register (ct); -if (err) - goto err_out; + err =guc_ct_ctb_h2g_register (ct); +if (err) + goto err_out; - err =guc_ct_ctb_g2h_register (ct); -if (err) - goto err_out; + err =guc_ct_ctb_g2h_register (ct); +if (err) + goto err_out; - err =guc_ct_control_toggle (ct, true);
drivers/gpu/drm/xe/xe_guc_ct.c int xe_guc_ct_enable(struct xe_guc_ct *ct) · line 532
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +8
-2-if (err) - goto err_out; + err =guc_ct_control_toggle (ct, true); +if (err) + goto err_out; + }else { + ct->ctbs.h2g.info.broken = false; + ct->ctbs.g2h.info.broken = false; +xe_map_memset (xe, &ct->bo->vmap,CTB_H2G_BUFFER_OFFSET ,0 , +CTB_H2G_BUFFER_SIZE );
nit: we may want to add some debug dump to see what H2G actually are about to be lost by this memset
this would also allow us to verify test scenarios which may assume something was not processed by the source GuC before VF pause
but we can do that as follow up
Quoted 42 lines
drivers/gpu/drm/xe/xe_guc_ct.c +27
-0+ }guc_ct_change_state (ct,XE_GUC_CT_STATE_ENABLED );@@ -554,6 +562,32 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)return err; } +/** +* xe_guc_ct_restart() - Restart GuC CT +* @ct: the &xe_guc_ct +* +* Restart GuC CT to an empty state without issuing a CT register MMIO command. +* +* Return: 0 on success, or a negative errno on failure. +*/ +int xe_guc_ct_restart (struct xe_guc_ct *ct) +{ +return __xe_guc_ct_start (ct, false); +} + +/** +* xe_guc_ct_enable() - Enable GuC CT +* @ct: the &xe_guc_ct +* +* Enable GuC CT to an empty state and issue a CT register MMIO command. +* +* Return: 0 on success, or a negative errno on failure. +*/ +int xe_guc_ct_enable (struct xe_guc_ct *ct) +{ +return __xe_guc_ct_start (ct, true); +} +static void stop_g2h_handler (struct xe_guc_ct *ct) {cancel_work_sync (&ct->g2h_worker);drivers/gpu/drm/xe/xe_guc_ct.h +0
-0
drivers/gpu/drm/xe/xe_guc_ct.h int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct); · line 21
Quoted 7 lines
drivers/gpu/drm/xe/xe_guc_ct.h +1
-0@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);int xe_guc_ct_init (struct xe_guc_ct *ct);int xe_guc_ct_init_post_hwconfig (struct xe_guc_ct *ct);int xe_guc_ct_enable (struct xe_guc_ct *ct); +int xe_guc_ct_restart (struct xe_guc_ct *ct);void xe_guc_ct_disable (struct xe_guc_ct *ct);void xe_guc_ct_stop (struct xe_guc_ct *ct);void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct);
otherwise, lgtm
Matthew Brost
maintainerRe: [PATCH v4 24/34] drm/xe/vf: Start CTs before resfix VF post migration recovery
Quoted 9 lines · Michal Wajdeczko
Quoted 6 lines · Matthew Brost
Before
resfix, all CTs stuck in the H2G queue need to be squashed, as they may contain stale or invalid data.Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.
Quoted 8 lines
Quoted 5 lines
v3:
- Don't mess with head / tail values (Michal)
v4:
- Don't mess with broke (Michal)
- AddCTB_H2G_BUFFER_OFFSET(Michal)I guess those small fixes shall be done separately
Are you suggesting I break this is different patch? Seems overkill and not particularly how I want to spend my time. This was basically unrelated nit of a suggestion to add CTB_H2G_BUFFER_OFFSET which I absord, now further nit to break into different patch. This is a great way to get to me just abandon this series.
Quoted 139 lines
Quoted 126 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 +++ drivers/gpu/drm/xe/xe_guc_ct.c | 70 +++++++++++++++++++++-------- drivers/gpu/drm/xe/xe_guc_ct.h | 1 + 3 files changed, 60 insertions(+), 18 deletions(-)drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7
-0@@ -1137,6 +1137,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)return 0 ; } +static void vf_post_migration_rearm (struct xe_gt *gt) +{ +xe_guc_ct_restart (>->uc.guc.ct); +} +static void vf_post_migration_kickstart (struct xe_gt *gt) {xe_guc_submit_unpause (>->uc.guc);@@ -1188,6 +1193,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)if (err) goto fail; +vf_post_migration_rearm (gt); + err =vf_post_migration_notify_resfix_done (gt);if (err && err != -EAGAIN ) goto fail;drivers/gpu/drm/xe/xe_guc_ct.c +17
-16@@ -166,6 +166,7 @@ ct_to_xe(struct xe_guc_ct *ct)*/ #define CTB_DESC_SIZE ALIGN(sizeof(struct guc_ct_buffer_desc), SZ_2K) +#define CTB_H2G_BUFFER_OFFSET (CTB_DESC_SIZE * 2)#define CTB_H2G_BUFFER_SIZE (SZ_4K)#define CTB_G2H_BUFFER_SIZE (SZ_128K)#define G2H_ROOM_BUFFER_SIZE (CTB_G2H_BUFFER_SIZE / 2)@@ -189,7 +190,7 @@ long xe_guc_ct_queue_proc_time_jiffies(struct xe_guc_ct *ct)static size_t guc_ct_size (void ) { -return 2 *CTB_DESC_SIZE +CTB_H2G_BUFFER_SIZE + +return CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE +CTB_G2H_BUFFER_SIZE ; }@@ -330,7 +331,7 @@ static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g, h2g->desc = *map;xe_map_memset (xe, &h2g->desc,0 ,0 ,sizeof (struct guc_ct_buffer_desc )); - h2g->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE *2 ); + h2g->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_H2G_BUFFER_OFFSET ); } static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h,@@ -348,7 +349,7 @@ static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h, g2h->desc =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE );xe_map_memset (xe, &g2h->desc,0 ,0 ,sizeof (struct guc_ct_buffer_desc )); - g2h->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE *2 + + g2h->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE ); }@@ -359,7 +360,7 @@ static int guc_ct_ctb_h2g_register(struct xe_guc_ct *ct)int err; desc_addr =xe_bo_ggtt_addr (ct->bo); - ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE *2 ; + ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_H2G_BUFFER_OFFSET ; size = ct->ctbs.h2g.info.size *sizeof (u32); err = xe_guc_self_cfg64(guc,@@ -386,7 +387,7 @@ static int guc_ct_ctb_g2h_register(struct xe_guc_ct *ct)int err; desc_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE ; - ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE *2 + + ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE ; size = ct->ctbs.g2h.info.size *sizeof (u32);@@ -500,7 +501,7 @@ static void ct_exit_safe_mode(struct xe_guc_ct *ct)xe_gt_dbg (ct_to_gt (ct),"GuC CT safe-mode disabled\n" ); } -int xe_guc_ct_enable (struct xe_guc_ct *ct) +static int __xe_guc_ct_start (struct xe_guc_ct *ct,bool needs_register) {struct xe_device *xe =ct_to_xe (ct);struct xe_gt *gt =ct_to_gt (ct);@@ -508,21 +509,28 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)xe_gt_assert (gt, !xe_guc_ct_enabled (ct)); -xe_map_memset (xe, &ct->bo->vmap,0 ,0 ,xe_bo_size (ct->bo)); -guc_ct_ctb_h2g_init (xe, &ct->ctbs.h2g, &ct->bo->vmap); -guc_ct_ctb_g2h_init (xe, &ct->ctbs.g2h, &ct->bo->vmap); +if (needs_register) { +xe_map_memset (xe, &ct->bo->vmap,0 ,0 ,xe_bo_size (ct->bo)); +guc_ct_ctb_h2g_init (xe, &ct->ctbs.h2g, &ct->bo->vmap); +guc_ct_ctb_g2h_init (xe, &ct->ctbs.g2h, &ct->bo->vmap); - err =guc_ct_ctb_h2g_register (ct); -if (err) - goto err_out; + err =guc_ct_ctb_h2g_register (ct); +if (err) + goto err_out; - err =guc_ct_ctb_g2h_register (ct); -if (err) - goto err_out; + err =guc_ct_ctb_g2h_register (ct); +if (err) + goto err_out; - err =guc_ct_control_toggle (ct, true);drivers/gpu/drm/xe/xe_guc_ct.c int xe_guc_ct_enable(struct xe_guc_ct *ct) · line 532
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +8
-2-if (err) - goto err_out; + err =guc_ct_control_toggle (ct, true); +if (err) + goto err_out; + }else { + ct->ctbs.h2g.info.broken = false; + ct->ctbs.g2h.info.broken = false; +xe_map_memset (xe, &ct->bo->vmap,CTB_H2G_BUFFER_OFFSET ,0 , +CTB_H2G_BUFFER_SIZE );nit: we may want to add some debug dump to see what H2G actually are about to be lost by this memset
Quoted 2 lines
this would also allow us to verify test scenarios which may assume something was not processed by the source GuC before VF pause
The debug messages in [1] provide all information needed to reason which code paths are being tested on VF recovery.
Matt
[1] https://patchwork.freedesktop.org/patch/677965/?series=154627&rev=4
Quoted 54 lines
but we can do that as follow up
Quoted 42 lines
drivers/gpu/drm/xe/xe_guc_ct.c +27
-0+ }guc_ct_change_state (ct,XE_GUC_CT_STATE_ENABLED );@@ -554,6 +562,32 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)return err; } +/** +* xe_guc_ct_restart() - Restart GuC CT +* @ct: the &xe_guc_ct +* +* Restart GuC CT to an empty state without issuing a CT register MMIO command. +* +* Return: 0 on success, or a negative errno on failure. +*/ +int xe_guc_ct_restart (struct xe_guc_ct *ct) +{ +return __xe_guc_ct_start (ct, false); +} + +/** +* xe_guc_ct_enable() - Enable GuC CT +* @ct: the &xe_guc_ct +* +* Enable GuC CT to an empty state and issue a CT register MMIO command. +* +* Return: 0 on success, or a negative errno on failure. +*/ +int xe_guc_ct_enable (struct xe_guc_ct *ct) +{ +return __xe_guc_ct_start (ct, true); +} +static void stop_g2h_handler (struct xe_guc_ct *ct) {cancel_work_sync (&ct->g2h_worker);drivers/gpu/drm/xe/xe_guc_ct.h +0
-0drivers/gpu/drm/xe/xe_guc_ct.h int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct); · line 21
Quoted 7 lines
drivers/gpu/drm/xe/xe_guc_ct.h +1
-0@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);int xe_guc_ct_init (struct xe_guc_ct *ct);int xe_guc_ct_init_post_hwconfig (struct xe_guc_ct *ct);int xe_guc_ct_enable (struct xe_guc_ct *ct); +int xe_guc_ct_restart (struct xe_guc_ct *ct);void xe_guc_ct_disable (struct xe_guc_ct *ct);void xe_guc_ct_stop (struct xe_guc_ct *ct);void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct);otherwise, lgtm
Re: [PATCH v4 24/34] drm/xe/vf: Start CTs before resfix VF post migration recovery
Quoted 16 lines · Matthew Brost
Quoted 15 lines · Michal Wajdeczko
Quoted 11 lines · Matthew Brost
Before
resfix, all CTs stuck in the H2G queue need to be squashed, as they may contain stale or invalid data.Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.
v3:
- Don't mess with head / tail values (Michal)
v4:
- Don't mess with broke (Michal)
- AddCTB_H2G_BUFFER_OFFSET(Michal)
Quoted 8 lines
Quoted 2 lines
I guess those small fixes shall be done separately
Are you suggesting I break this is different patch? Seems overkill and not particularly how I want to spend my time. This was basically unrelated nit of a suggestion to add
CTB_H2G_BUFFER_OFFSETwhich I absord, now further nit to break into different patch. This is a great way to get to me just abandon this series.
nit or not nit, still the same rules apply [2], and suggestion to use separate macro is a 'separate logical change'
so don't kill the messenger
[2] https://docs.kernel.org/process/submitting-patches.html#separate-your-changes
Quoted 140 lines
Quoted 140 lines
Quoted 127 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 7 +++ drivers/gpu/drm/xe/xe_guc_ct.c | 70 +++++++++++++++++++++-------- drivers/gpu/drm/xe/xe_guc_ct.h | 1 + 3 files changed, 60 insertions(+), 18 deletions(-)drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7
-0@@ -1137,6 +1137,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)return 0 ; } +static void vf_post_migration_rearm (struct xe_gt *gt) +{ +xe_guc_ct_restart (>->uc.guc.ct); +} +static void vf_post_migration_kickstart (struct xe_gt *gt) {xe_guc_submit_unpause (>->uc.guc);@@ -1188,6 +1193,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)if (err) goto fail; +vf_post_migration_rearm (gt); + err =vf_post_migration_notify_resfix_done (gt);if (err && err != -EAGAIN ) goto fail;drivers/gpu/drm/xe/xe_guc_ct.c +17
-16@@ -166,6 +166,7 @@ ct_to_xe(struct xe_guc_ct *ct)*/ #define CTB_DESC_SIZE ALIGN(sizeof(struct guc_ct_buffer_desc), SZ_2K) +#define CTB_H2G_BUFFER_OFFSET (CTB_DESC_SIZE * 2)#define CTB_H2G_BUFFER_SIZE (SZ_4K)#define CTB_G2H_BUFFER_SIZE (SZ_128K)#define G2H_ROOM_BUFFER_SIZE (CTB_G2H_BUFFER_SIZE / 2)@@ -189,7 +190,7 @@ long xe_guc_ct_queue_proc_time_jiffies(struct xe_guc_ct *ct)static size_t guc_ct_size (void ) { -return 2 *CTB_DESC_SIZE +CTB_H2G_BUFFER_SIZE + +return CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE +CTB_G2H_BUFFER_SIZE ; }@@ -330,7 +331,7 @@ static void guc_ct_ctb_h2g_init(struct xe_device *xe, struct guc_ctb *h2g, h2g->desc = *map;xe_map_memset (xe, &h2g->desc,0 ,0 ,sizeof (struct guc_ct_buffer_desc )); - h2g->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE *2 ); + h2g->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_H2G_BUFFER_OFFSET ); } static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h,@@ -348,7 +349,7 @@ static void guc_ct_ctb_g2h_init(struct xe_device *xe, struct guc_ctb *g2h, g2h->desc =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE );xe_map_memset (xe, &g2h->desc,0 ,0 ,sizeof (struct guc_ct_buffer_desc )); - g2h->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_DESC_SIZE *2 + + g2h->cmds =IOSYS_MAP_INIT_OFFSET (map,CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE ); }@@ -359,7 +360,7 @@ static int guc_ct_ctb_h2g_register(struct xe_guc_ct *ct)int err; desc_addr =xe_bo_ggtt_addr (ct->bo); - ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE *2 ; + ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_H2G_BUFFER_OFFSET ; size = ct->ctbs.h2g.info.size *sizeof (u32); err = xe_guc_self_cfg64(guc,@@ -386,7 +387,7 @@ static int guc_ct_ctb_g2h_register(struct xe_guc_ct *ct)int err; desc_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE ; - ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_DESC_SIZE *2 + + ctb_addr =xe_bo_ggtt_addr (ct->bo) +CTB_H2G_BUFFER_OFFSET +CTB_H2G_BUFFER_SIZE ; size = ct->ctbs.g2h.info.size *sizeof (u32);@@ -500,7 +501,7 @@ static void ct_exit_safe_mode(struct xe_guc_ct *ct)xe_gt_dbg (ct_to_gt (ct),"GuC CT safe-mode disabled\n" ); } -int xe_guc_ct_enable (struct xe_guc_ct *ct) +static int __xe_guc_ct_start (struct xe_guc_ct *ct,bool needs_register) {struct xe_device *xe =ct_to_xe (ct);struct xe_gt *gt =ct_to_gt (ct);@@ -508,21 +509,28 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)xe_gt_assert (gt, !xe_guc_ct_enabled (ct)); -xe_map_memset (xe, &ct->bo->vmap,0 ,0 ,xe_bo_size (ct->bo)); -guc_ct_ctb_h2g_init (xe, &ct->ctbs.h2g, &ct->bo->vmap); -guc_ct_ctb_g2h_init (xe, &ct->ctbs.g2h, &ct->bo->vmap); +if (needs_register) { +xe_map_memset (xe, &ct->bo->vmap,0 ,0 ,xe_bo_size (ct->bo)); +guc_ct_ctb_h2g_init (xe, &ct->ctbs.h2g, &ct->bo->vmap); +guc_ct_ctb_g2h_init (xe, &ct->ctbs.g2h, &ct->bo->vmap); - err =guc_ct_ctb_h2g_register (ct); -if (err) - goto err_out; + err =guc_ct_ctb_h2g_register (ct); +if (err) + goto err_out; - err =guc_ct_ctb_g2h_register (ct); -if (err) - goto err_out; + err =guc_ct_ctb_g2h_register (ct); +if (err) + goto err_out; - err =guc_ct_control_toggle (ct, true);drivers/gpu/drm/xe/xe_guc_ct.c int xe_guc_ct_enable(struct xe_guc_ct *ct) · line 532
Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +8
-2-if (err) - goto err_out; + err =guc_ct_control_toggle (ct, true); +if (err) + goto err_out; + }else { + ct->ctbs.h2g.info.broken = false; + ct->ctbs.g2h.info.broken = false; +xe_map_memset (xe, &ct->bo->vmap,CTB_H2G_BUFFER_OFFSET ,0 , +CTB_H2G_BUFFER_SIZE );nit: we may want to add some debug dump to see what H2G actually are about to be lost by this memset
Quoted 5 lines
Quoted 2 lines
this would also allow us to verify test scenarios which may assume something was not processed by the source GuC before VF pause
The debug messages in [1] provide all information needed to reason which code paths are being tested on VF recovery.
IMO those logs do not provide the same info (here logs are about what was lost, your logs are about what's replayed)
but if you feel it's sufficient, then I'm fine
Quoted 59 lines
Matt
[1] https://patchwork.freedesktop.org/patch/677965/?series=154627&rev=4
Quoted 54 lines
but we can do that as follow up
Quoted 42 lines
drivers/gpu/drm/xe/xe_guc_ct.c +27
-0+ }guc_ct_change_state (ct,XE_GUC_CT_STATE_ENABLED );@@ -554,6 +562,32 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)return err; } +/** +* xe_guc_ct_restart() - Restart GuC CT +* @ct: the &xe_guc_ct +* +* Restart GuC CT to an empty state without issuing a CT register MMIO command. +* +* Return: 0 on success, or a negative errno on failure. +*/ +int xe_guc_ct_restart (struct xe_guc_ct *ct) +{ +return __xe_guc_ct_start (ct, false); +} + +/** +* xe_guc_ct_enable() - Enable GuC CT +* @ct: the &xe_guc_ct +* +* Enable GuC CT to an empty state and issue a CT register MMIO command. +* +* Return: 0 on success, or a negative errno on failure. +*/ +int xe_guc_ct_enable (struct xe_guc_ct *ct) +{ +return __xe_guc_ct_start (ct, true); +} +static void stop_g2h_handler (struct xe_guc_ct *ct) {cancel_work_sync (&ct->g2h_worker);drivers/gpu/drm/xe/xe_guc_ct.h +0
-0drivers/gpu/drm/xe/xe_guc_ct.h int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct); · line 21
Quoted 7 lines
drivers/gpu/drm/xe/xe_guc_ct.h +1
-0@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);int xe_guc_ct_init (struct xe_guc_ct *ct);int xe_guc_ct_init_post_hwconfig (struct xe_guc_ct *ct);int xe_guc_ct_enable (struct xe_guc_ct *ct); +int xe_guc_ct_restart (struct xe_guc_ct *ct);void xe_guc_ct_disable (struct xe_guc_ct *ct);void xe_guc_ct_stop (struct xe_guc_ct *ct);void xe_guc_ct_flush_and_stop (struct xe_guc_ct *ct);otherwise, lgtm
Matthew Brost
maintainerPATCH v4 25/34 drm/xe/vf: Abort VF post migration recovery on failure
In reply to Matthew BrostIf VF post-migration recovery fails, the device is wedged. However, submission queues still need to be enabled for proper cleanup. In such cases, call into the GuC submission backend to restart all queues that were previously paused.
v3:
- s/Avort/Abort (Tomasz)
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 10 ++++++++++ drivers/gpu/drm/xe/xe_guc_submit.c | 20 ++++++++++++++++++++ drivers/gpu/drm/xe/xe_guc_submit.h | 1 + 3 files changed, 31 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
@@ -1147,6 +1147,15 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)xe_guc_submit_unpause (>->uc.guc); } +static void vf_post_migration_abort (struct xe_gt *gt) +{ +spin_lock_irq (>->sriov.vf.migration.lock); +WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, false); +spin_unlock_irq (>->sriov.vf.migration.lock); + +xe_guc_submit_pause_abort (>->uc.guc); +} +static int vf_post_migration_notify_resfix_done (struct xe_gt *gt) {bool skip_resfix = false;
@@ -1205,6 +1214,7 @@ static void vf_post_migration_recovery(struct xe_gt *gt)xe_gt_sriov_notice (gt,"migration recovery ended\n" );return ; fail: +vf_post_migration_abort (gt);xe_pm_runtime_put (xe);xe_gt_sriov_err (gt,"migration recovery failed (%pe)\n" ,ERR_PTR (err));xe_device_declare_wedged (xe);
drivers/gpu/drm/xe/xe_guc_submit.c +20 -0
@@ -2359,6 +2359,26 @@ void xe_guc_submit_unpause(struct xe_guc *guc)wake_up_all (&guc->ct.wq); } +/** +* xe_guc_submit_abort - Abort all paused submission task on given GuC. +* @guc: the &xe_guc struct instance whose scheduler is to be aborted +*/ +void xe_guc_submit_pause_abort (struct xe_guc *guc) +{ +struct xe_exec_queue *q; +unsigned long index; + +mutex_lock (&guc->submission_state.lock); +xa_for_each (&guc->submission_state.exec_queue_lookup, index, q) { +struct xe_gpu_scheduler *sched = &q->guc->sched; + +xe_sched_submission_start (sched); +if (exec_queue_killed_or_banned_or_wedged (q)) +xe_guc_exec_queue_trigger_cleanup (q); + } +mutex_unlock (&guc->submission_state.lock); +} +static struct xe_exec_queue *g2h_exec_queue_lookup (struct xe_guc *guc,u32 guc_id) {
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);int xe_guc_submit_start (struct xe_guc *guc);void xe_guc_submit_pause (struct xe_guc *guc);void xe_guc_submit_unpause (struct xe_guc *guc); +void xe_guc_submit_pause_abort (struct xe_guc *guc);void xe_guc_submit_wedge (struct xe_guc *guc);int xe_guc_read_stopped (struct xe_guc *guc);
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 26/34 drm/xe/vf: Replay GuC submission state on pause / unpause
In reply to Matthew BrostFixup GuC submission pause / unpause functions to properly replay any possible state lost during VF post migration recovery.
v3:
- Add helpers for revert / replay (Tomasz)
- Add comment around WQ NOPs (Tomasz)
--- drivers/gpu/drm/xe/xe_gpu_scheduler.c | 14 ++ drivers/gpu/drm/xe/xe_gpu_scheduler.h | 2 + drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 1 + drivers/gpu/drm/xe/xe_guc_exec_queue_types.h | 15 ++ drivers/gpu/drm/xe/xe_guc_submit.c | 242 +++++++++++++++++-- drivers/gpu/drm/xe/xe_guc_submit.h | 1 + drivers/gpu/drm/xe/xe_sched_job_types.h | 4 + 7 files changed, 264 insertions(+), 15 deletions(-)
drivers/gpu/drm/xe/xe_gpu_scheduler.c +14 -0
@@ -135,3 +135,17 @@ void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched,list_add_tail (&msg->link, &sched->msgs);xe_sched_process_msg_queue (sched); } + +/** +* xe_sched_add_msg_head() - Xe GPU scheduler add message to head of list +* @sched: Xe GPU scheduler +* @msg: Message to add +*/ +void xe_sched_add_msg_head (struct xe_gpu_scheduler *sched, +struct xe_sched_msg *msg) +{ +lockdep_assert_held (&sched->base.job_list_lock); + +list_add (&msg->link, &sched->msgs); +xe_sched_process_msg_queue (sched); +}
drivers/gpu/drm/xe/xe_gpu_scheduler.h +2 -0
@@ -29,6 +29,8 @@ void xe_sched_add_msg(struct xe_gpu_scheduler *sched, struct xe_sched_msg *msg);void xe_sched_add_msg_locked (struct xe_gpu_scheduler *sched,struct xe_sched_msg *msg); +void xe_sched_add_msg_head (struct xe_gpu_scheduler *sched, +struct xe_sched_msg *msg);static inline void xe_sched_msg_lock (struct xe_gpu_scheduler *sched) {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -1140,6 +1140,7 @@ static int vf_post_migration_fixups(struct xe_gt *gt)static void vf_post_migration_rearm (struct xe_gt *gt) {xe_guc_ct_restart (>->uc.guc.ct); +xe_guc_submit_unpause_prepare (>->uc.guc); }static void vf_post_migration_kickstart (struct xe_gt *gt)
drivers/gpu/drm/xe/xe_guc_exec_queue_types.h +15 -0
@@ -51,6 +51,21 @@ struct xe_guc_exec_queue {wait_queue_head_t suspend_wait;/** @suspend_pending: a suspend of the exec_queue is pending */ bool suspend_pending; +/** +* @needs_cleanup: Needs a cleanup message during VF post migration +* recovery. +*/ +bool needs_cleanup; +/** +* @needs_suspend: Needs a suspend message during VF post migration +* recovery. +*/ +bool needs_suspend; +/** +* @needs_resume: Needs a resume message during VF post migration +* recovery. +*/ + bool needs_resume; };#endif
drivers/gpu/drm/xe/xe_guc_submit.c +227 -15
325 lines not on this page · raw message
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);int xe_guc_submit_start (struct xe_guc *guc);void xe_guc_submit_pause (struct xe_guc *guc);void xe_guc_submit_unpause (struct xe_guc *guc); +void xe_guc_submit_unpause_prepare (struct xe_guc *guc);void xe_guc_submit_pause_abort (struct xe_guc *guc);void xe_guc_submit_wedge (struct xe_guc *guc);
drivers/gpu/drm/xe/xe_sched_job_types.h +4 -0
@@ -63,6 +63,10 @@ struct xe_sched_job {bool ring_ops_flush_tlb;/** @ggtt: mapped in ggtt. */ bool ggtt; +/** @skip_emit: skip emitting the job */ +bool skip_emit; +/** @last_replay: last job being replayed */ +bool last_replay;/** @ptrs: per instance pointers. */ struct xe_job_ptrs ptrs[]; };
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 27/34 drm/xe: Move queue init before LRC creation
In reply to Matthew BrostA queue must be in the submission backend's tracking state before the LRC is created to avoid a race condition where the LRC's GGTT addresses are not properly fixed up during VF post-migration recovery.
Move the queue initialization—which adds the queue to the submission backend's tracking state—before LRC creation.
v2:
- Wait on VF GGTT fixes before creating LRC (testing)
--- drivers/gpu/drm/xe/xe_exec_queue.c | 43 +++++++++++++++++------ drivers/gpu/drm/xe/xe_execlist.c | 2 +- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 39 +++++++++++++++++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 ++ drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 5 +++ drivers/gpu/drm/xe/xe_guc_submit.c | 2 +- drivers/gpu/drm/xe/xe_lrc.h | 10 ++++++ 7 files changed, 90 insertions(+), 13 deletions(-)
drivers/gpu/drm/xe/xe_exec_queue.c +33 -10
@@ -15,6 +15,7 @@#include "xe_dep_scheduler.h" #include "xe_device.h" #include "xe_gt.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine_class_sysfs.h" #include "xe_hw_engine_group.h" #include "xe_hw_fence.h"
@@ -179,17 +180,32 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q) flags |=XE_LRC_CREATE_RUNALONE ; } + err = q->ops->init (q); +if (err) +return err; + +/* +* This must occur after q->ops->init to avoid race conditions during VF +* post-migration recovery, as the fixups for the LRC GGTT addresses +* depend on the queue being present in the backend tracking structure. +* +* In addition to above, we must wait on inflight GGTT changes to +* avoid writing out stale values here. +*/ +xe_gt_sriov_vf_wait_valid_ggtt (q->gt);for (i =0 ; i < q->width; ++i) { - q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K , q->msix_vec, flags); -if (IS_ERR (q->lrc[i])) { - err =PTR_ERR (q->lrc[i]); +struct xe_lrc *lrc; + + lrc =xe_lrc_create (q->hwe, q->vm,xe_lrc_ring_size (), + q->msix_vec, flags); +if (IS_ERR (lrc)) { + err =PTR_ERR (lrc); goto err_lrc; } - } - err = q->ops->init (q); -if (err) - goto err_lrc; +/* Pairs with READ_ONCE to xe_exec_queue_contexts_hwsp_rebase */ +WRITE_ONCE (q->lrc[i], lrc); + }return 0 ;
@@ -1095,9 +1111,16 @@ int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch)int err =0 ;for (i =0 ; i < q->width; ++i) { -xe_lrc_update_memirq_regs_with_address (q->lrc[i], q->hwe, scratch); -xe_lrc_update_hwctx_regs_with_address (q->lrc[i]); - err =xe_lrc_setup_wa_bb_with_scratch (q->lrc[i], q->hwe, scratch); +struct xe_lrc *lrc; + +/* Pairs with WRITE_ONCE in __xe_exec_queue_init */ + lrc =READ_ONCE (q->lrc[i]); +if (!lrc) +continue ; + +xe_lrc_update_memirq_regs_with_address (lrc, q->hwe, scratch); +xe_lrc_update_hwctx_regs_with_address (lrc); + err =xe_lrc_setup_wa_bb_with_scratch (lrc, q->hwe, scratch);if (err)break ; }
drivers/gpu/drm/xe/xe_execlist.c +1 -1
@@ -339,7 +339,7 @@ static int execlist_exec_queue_init(struct xe_exec_queue *q)const struct drm_sched_init_args args = { .ops = &drm_sched_ops, .num_rqs =1 , - .credit_limit = q->lrc[0 ]->ring.size /MAX_JOB_SIZE_BYTES , + .credit_limit =xe_lrc_ring_size () /MAX_JOB_SIZE_BYTES , .hang_limit =XE_SCHED_HANG_LIMIT , .timeout =XE_SCHED_JOB_TIMEOUT , .name = q->hwe->name,
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +38 -1
@@ -480,6 +480,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery) shift, config->ggtt_base);xe_tile_sriov_vf_fixup_ggtt_nodes (gt_to_tile (gt), shift); } + +WRITE_ONCE (gt->sriov.vf.migration.ggtt_need_fixes, false); +smp_wmb ();/* Ensure above write visible before wake */ +wake_up_all (>->sriov.vf.migration.wq); + out:mutex_unlock (&ggtt->lock);return err;
@@ -743,7 +748,8 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); -smp_wmb ();/* Ensure above write visable before wake */ +WRITE_ONCE (gt->sriov.vf.migration.ggtt_need_fixes, true); +smp_wmb ();/* Ensure above writes visable before wake */ wake_up_all (>->uc.guc.ct.wq);
@@ -1262,6 +1268,7 @@ int xe_gt_sriov_vf_init_early(struct xe_gt *gt) gt->sriov.vf.migration.scratch = buf;spin_lock_init (>->sriov.vf.migration.lock);INIT_WORK (>->sriov.vf.migration.worker, migration_worker_func); +init_waitqueue_head (>->sriov.vf.migration.wq);return 0 ; }
@@ -1305,3 +1312,33 @@ bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); } + +static bool vf_valid_ggtt (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +if (xe_memirq_sw_int_0_irq_pending (memirq, >->uc.guc) || +READ_ONCE (gt->sriov.vf.migration.ggtt_need_fixes)) +return false; + +return true; +} + +/** +* xe_gt_sriov_vf_wait_valid_ggtt() - VF wait for valid GGTT addresses +* @gt: the &xe_gt +*/ +void xe_gt_sriov_vf_wait_valid_ggtt (struct xe_gt *gt) +{ +int ret; + +if (!IS_SRIOV_VF (gt_to_xe (gt))) +return ; + + ret =wait_event_interruptible_timeout (gt->sriov.vf.migration.wq, +vf_valid_ggtt (gt), +HZ *5 ); +XE_WARN_ON (!ret); +}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
@@ -38,4 +38,6 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p);void xe_gt_sriov_vf_print_runtime (struct xe_gt *gt,struct drm_printer *p);void xe_gt_sriov_vf_print_version (struct xe_gt *gt,struct drm_printer *p); +void xe_gt_sriov_vf_wait_valid_ggtt (struct xe_gt *gt); +#endif
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +5 -0
@@ -8,6 +8,7 @@#include <linux/rwsem.h> #include <linux/types.h> +#include <linux/wait.h> #include <linux/workqueue.h> #include "xe_uc_fw_types.h"
@@ -50,6 +51,8 @@ struct xe_gt_sriov_vf_migration {struct work_struct worker;/** @lock: Protects recovery_queued, teardown */ spinlock_t lock; +/** @wq: wait queue for migration fixes */ +wait_queue_head_t wq;/** @scratch: Scratch memory for VF recovery */ void *scratch;/** @recovery_teardown: VF post migration recovery is being torn down */
@@ -58,6 +61,8 @@ struct xe_gt_sriov_vf_migration {bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */ bool recovery_inprogress; +/** @ggtt_need_fixes: VF GGTT needs fixes */ +bool ggtt_need_fixes; };/**
drivers/gpu/drm/xe/xe_guc_submit.c +1 -1
@@ -1943,7 +1943,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) timeout = (q->vm &&xe_vm_in_lr_mode (q->vm)) ?MAX_SCHEDULE_TIMEOUT :msecs_to_jiffies (q->sched_props.job_timeout_ms); err =xe_sched_init (&ge->sched, &drm_sched_ops, &xe_sched_ops, -NULL , q->lrc[0 ]->ring.size /MAX_JOB_SIZE_BYTES ,64 , +NULL ,xe_lrc_ring_size () /MAX_JOB_SIZE_BYTES ,64 , timeout,guc_to_gt (guc)->ordered_wq,NULL , q->name,gt_to_xe (q->gt)->drm.dev);if (err)
drivers/gpu/drm/xe/xe_lrc.h +10 -0
@@ -74,6 +74,16 @@ static inline void xe_lrc_put(struct xe_lrc *lrc)kref_put (&lrc->refcount, xe_lrc_destroy); } +/** +* xe_lrc_ring_size() - Xe LRC ring size +* +* Return: Size of LRC size +*/ +static inline size_t xe_lrc_ring_size (void ) +{ +return SZ_16K ; +} +size_t xe_gt_lrc_size (struct xe_gt *gt,enum xe_engine_class class);u32 xe_lrc_pphwsp_offset (struct xe_lrc *lrc);u32 xe_lrc_regs_offset (struct xe_lrc *lrc);
Signature · 1 line
2.34.1
Re: [PATCH v4 27/34] drm/xe: Move queue init before LRC creation
Quoted 38 lines · Matthew Brost
A queue must be in the submission backend's tracking state before the LRC is created to avoid a race condition where the LRC's GGTT addresses are not properly fixed up during VF post-migration recovery.
Move the queue initialization—which adds the queue to the submission backend's tracking state—before LRC creation.
v2:
- Wait on VF GGTT fixes before creating LRC (testing)Signed-off-by Matthew Brost<[email protected]>--- drivers/gpu/drm/xe/xe_exec_queue.c | 43 +++++++++++++++++------ drivers/gpu/drm/xe/xe_execlist.c | 2 +- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 39 +++++++++++++++++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 ++ drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 5 +++ drivers/gpu/drm/xe/xe_guc_submit.c | 2 +- drivers/gpu/drm/xe/xe_lrc.h | 10 ++++++ 7 files changed, 90 insertions(+), 13 deletions(-)drivers/gpu/drm/xe/xe_exec_queue.c +2
-0@@ -15,6 +15,7 @@#include "xe_dep_scheduler.h" #include "xe_device.h" #include "xe_gt.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine_class_sysfs.h" #include "xe_hw_engine_group.h" #include "xe_hw_fence.h" @@ -179,17 +180,32 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q) flags |=XE_LRC_CREATE_RUNALONE ; } + err = q->ops->init (q);
drivers/gpu/drm/xe/xe_exec_queue.c static int __xe_exec_queue_init(struct xe_exec_queue *q) · line 193
Quoted 10 lines
drivers/gpu/drm/xe/xe_exec_queue.c +10
-0+if (err) +return err; + +/* +* This must occur after q->ops->init to avoid race conditions during VF +* post-migration recovery, as the fixups for the LRC GGTT addresses +* depend on the queue being present in the backend tracking structure. +* +* In addition to above, we must wait on inflight GGTT changes to +* avoid writing out stale values here.
This paragraph needs expansion. Maybe:
``` In addition to above, we must wait on inflight GGTT changes to avoid writing out stale values here. Such wait provides a solid solution (without a race) only if the function can detect migration instantly from the moment vCPU resumes execution. ```
drivers/gpu/drm/xe/xe_exec_queue.c static int __xe_exec_queue_init(struct xe_exec_queue *q) · line 200
Quoted 10 lines
drivers/gpu/drm/xe/xe_exec_queue.c +6
-3+*/ +xe_gt_sriov_vf_wait_valid_ggtt (q->gt);for (i =0 ; i < q->width; ++i) { - q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K , q->msix_vec, flags); -if (IS_ERR (q->lrc[i])) { - err =PTR_ERR (q->lrc[i]); +struct xe_lrc *lrc; + + lrc =xe_lrc_create (q->hwe, q->vm,xe_lrc_ring_size (), + q->msix_vec, flags);
Previous discussion still valid:
---
Quoted 8 lines
Quoted 7 lines
If migration happened at this place, it is still possible to create a context with wrong GGTT references in the one LRC which was already filled but not integrated into the queue yet.
I don't think we can avoid races without a lock.
-Tomasz
Quoted 7 lines
There might be a small race here, let me think about this. I will say this change
xe_exec_threads--r threads-many-queues though. Locking is definitely not the way solve this though - reclaim rules are in play here which make locking difficult and convoluted cross layer locks will always get nacked by myself and others.Matt
Ok, if you can find a lockless solution again, that would be beneficial.
-Tomasz
---
Quoted 191 lines
drivers/gpu/drm/xe/xe_exec_queue.c +15
-7+if (IS_ERR (lrc)) { + err =PTR_ERR (lrc); goto err_lrc; } - } - err = q->ops->init (q); -if (err) - goto err_lrc; +/* Pairs with READ_ONCE to xe_exec_queue_contexts_hwsp_rebase */ +WRITE_ONCE (q->lrc[i], lrc); + }return 0 ;@@ -1095,9 +1111,16 @@ int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch)int err =0 ;for (i =0 ; i < q->width; ++i) { -xe_lrc_update_memirq_regs_with_address (q->lrc[i], q->hwe, scratch); -xe_lrc_update_hwctx_regs_with_address (q->lrc[i]); - err =xe_lrc_setup_wa_bb_with_scratch (q->lrc[i], q->hwe, scratch); +struct xe_lrc *lrc; + +/* Pairs with WRITE_ONCE in __xe_exec_queue_init */ + lrc =READ_ONCE (q->lrc[i]); +if (!lrc) +continue ; + +xe_lrc_update_memirq_regs_with_address (lrc, q->hwe, scratch); +xe_lrc_update_hwctx_regs_with_address (lrc); + err =xe_lrc_setup_wa_bb_with_scratch (lrc, q->hwe, scratch);if (err)break ; }drivers/gpu/drm/xe/xe_execlist.c +1
-1@@ -339,7 +339,7 @@ static int execlist_exec_queue_init(struct xe_exec_queue *q)const struct drm_sched_init_args args = { .ops = &drm_sched_ops, .num_rqs =1 , - .credit_limit = q->lrc[0 ]->ring.size /MAX_JOB_SIZE_BYTES , + .credit_limit =xe_lrc_ring_size () /MAX_JOB_SIZE_BYTES , .hang_limit =XE_SCHED_HANG_LIMIT , .timeout =XE_SCHED_JOB_TIMEOUT , .name = q->hwe->name,drivers/gpu/drm/xe/xe_gt_sriov_vf.c +38
-1@@ -480,6 +480,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery) shift, config->ggtt_base);xe_tile_sriov_vf_fixup_ggtt_nodes (gt_to_tile (gt), shift); } + +WRITE_ONCE (gt->sriov.vf.migration.ggtt_need_fixes, false); +smp_wmb ();/* Ensure above write visible before wake */ +wake_up_all (>->sriov.vf.migration.wq); + out:mutex_unlock (&ggtt->lock);return err;@@ -743,7 +748,8 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); -smp_wmb ();/* Ensure above write visable before wake */ +WRITE_ONCE (gt->sriov.vf.migration.ggtt_need_fixes, true); +smp_wmb ();/* Ensure above writes visable before wake */ wake_up_all (>->uc.guc.ct.wq);@@ -1262,6 +1268,7 @@ int xe_gt_sriov_vf_init_early(struct xe_gt *gt) gt->sriov.vf.migration.scratch = buf;spin_lock_init (>->sriov.vf.migration.lock);INIT_WORK (>->sriov.vf.migration.worker, migration_worker_func); +init_waitqueue_head (>->sriov.vf.migration.wq);return 0 ; }@@ -1305,3 +1312,33 @@ bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); } + +static bool vf_valid_ggtt (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +if (xe_memirq_sw_int_0_irq_pending (memirq, >->uc.guc) || +READ_ONCE (gt->sriov.vf.migration.ggtt_need_fixes)) +return false; + +return true; +} + +/** +* xe_gt_sriov_vf_wait_valid_ggtt() - VF wait for valid GGTT addresses +* @gt: the &xe_gt +*/ +void xe_gt_sriov_vf_wait_valid_ggtt (struct xe_gt *gt) +{ +int ret; + +if (!IS_SRIOV_VF (gt_to_xe (gt))) +return ; + + ret =wait_event_interruptible_timeout (gt->sriov.vf.migration.wq, +vf_valid_ggtt (gt), +HZ *5 ); +XE_WARN_ON (!ret); +}drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2
-0@@ -38,4 +38,6 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p);void xe_gt_sriov_vf_print_runtime (struct xe_gt *gt,struct drm_printer *p);void xe_gt_sriov_vf_print_version (struct xe_gt *gt,struct drm_printer *p); +void xe_gt_sriov_vf_wait_valid_ggtt (struct xe_gt *gt); +#endif drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +5
-0@@ -8,6 +8,7 @@#include <linux/rwsem.h> #include <linux/types.h> +#include <linux/wait.h> #include <linux/workqueue.h> #include "xe_uc_fw_types.h" @@ -50,6 +51,8 @@ struct xe_gt_sriov_vf_migration {struct work_struct worker;/** @lock: Protects recovery_queued, teardown */ spinlock_t lock; +/** @wq: wait queue for migration fixes */ +wait_queue_head_t wq;/** @scratch: Scratch memory for VF recovery */ void *scratch;/** @recovery_teardown: VF post migration recovery is being torn down */ @@ -58,6 +61,8 @@ struct xe_gt_sriov_vf_migration {bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */ bool recovery_inprogress; +/** @ggtt_need_fixes: VF GGTT needs fixes */ +bool ggtt_need_fixes; };/** drivers/gpu/drm/xe/xe_guc_submit.c +1
-1@@ -1943,7 +1943,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) timeout = (q->vm &&xe_vm_in_lr_mode (q->vm)) ?MAX_SCHEDULE_TIMEOUT :msecs_to_jiffies (q->sched_props.job_timeout_ms); err =xe_sched_init (&ge->sched, &drm_sched_ops, &xe_sched_ops, -NULL , q->lrc[0 ]->ring.size /MAX_JOB_SIZE_BYTES ,64 , +NULL ,xe_lrc_ring_size () /MAX_JOB_SIZE_BYTES ,64 , timeout,guc_to_gt (guc)->ordered_wq,NULL , q->name,gt_to_xe (q->gt)->drm.dev);if (err)drivers/gpu/drm/xe/xe_lrc.h +10
-0@@ -74,6 +74,16 @@ static inline void xe_lrc_put(struct xe_lrc *lrc)kref_put (&lrc->refcount, xe_lrc_destroy); } +/** +* xe_lrc_ring_size() - Xe LRC ring size +* +* Return: Size of LRC size +*/ +static inline size_t xe_lrc_ring_size (void ) +{ +return SZ_16K ; +} +size_t xe_gt_lrc_size (struct xe_gt *gt,enum xe_engine_class class);u32 xe_lrc_pphwsp_offset (struct xe_lrc *lrc);u32 xe_lrc_regs_offset (struct xe_lrc *lrc);
Matthew Brost
maintainerRe: [PATCH v4 27/34] drm/xe: Move queue init before LRC creation
Quoted 90 lines · Tomasz
Quoted 37 lines · Matthew Brost
A queue must be in the submission backend's tracking state before the LRC is created to avoid a race condition where the LRC's GGTT addresses are not properly fixed up during VF post-migration recovery.
Move the queue initialization—which adds the queue to the submission backend's tracking state—before LRC creation.
v2:
- Wait on VF GGTT fixes before creating LRC (testing)Signed-off-by Matthew Brost<[email protected]>--- drivers/gpu/drm/xe/xe_exec_queue.c | 43 +++++++++++++++++------ drivers/gpu/drm/xe/xe_execlist.c | 2 +- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 39 +++++++++++++++++++- drivers/gpu/drm/xe/xe_gt_sriov_vf.h | 2 ++ drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 5 +++ drivers/gpu/drm/xe/xe_guc_submit.c | 2 +- drivers/gpu/drm/xe/xe_lrc.h | 10 ++++++ 7 files changed, 90 insertions(+), 13 deletions(-)drivers/gpu/drm/xe/xe_exec_queue.c +2
-0@@ -15,6 +15,7 @@#include "xe_dep_scheduler.h" #include "xe_device.h" #include "xe_gt.h" +#include "xe_gt_sriov_vf.h" #include "xe_hw_engine_class_sysfs.h" #include "xe_hw_engine_group.h" #include "xe_hw_fence.h" @@ -179,17 +180,32 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q) flags |=XE_LRC_CREATE_RUNALONE ; } + err = q->ops->init (q);drivers/gpu/drm/xe/xe_exec_queue.c static int __xe_exec_queue_init(struct xe_exec_queue *q) · line 192
Quoted 10 lines
drivers/gpu/drm/xe/xe_exec_queue.c +10
-0+if (err) +return err; + +/* +* This must occur after q->ops->init to avoid race conditions during VF +* post-migration recovery, as the fixups for the LRC GGTT addresses +* depend on the queue being present in the backend tracking structure. +* +* In addition to above, we must wait on inflight GGTT changes to +* avoid writing out stale values here. This paragraph needs expansion. Maybe:
``` In addition to above, we must wait on inflight GGTT changes to avoid writing out stale values here. Such wait provides a solid solution (without a race) only if the function can detect migration instantly from the moment vCPU resumes execution. ```drivers/gpu/drm/xe/xe_exec_queue.c static int __xe_exec_queue_init(struct xe_exec_queue *q) · line 200
Quoted 10 lines
drivers/gpu/drm/xe/xe_exec_queue.c +6
-3+*/ +xe_gt_sriov_vf_wait_valid_ggtt (q->gt);for (i =0 ; i < q->width; ++i) { - q->lrc[i] =xe_lrc_create (q->hwe, q->vm,SZ_16K , q->msix_vec, flags); -if (IS_ERR (q->lrc[i])) { - err =PTR_ERR (q->lrc[i]); +struct xe_lrc *lrc; + + lrc =xe_lrc_create (q->hwe, q->vm,xe_lrc_ring_size (), + q->msix_vec, flags);Previous discussion still valid:
---
Quoted 14 lines
Quoted 7 lines
If migration happened at this place, it is still possible to create a context with wrong GGTT references in the one LRC which was already filled but not integrated into the queue yet.
I don't think we can avoid races without a lock.
-Tomasz
There might be a small race here, let me think about this. I will say this change
xe_exec_threads--r threads-many-queues though. Locking is definitely not the way solve this though - reclaim rules are in play here which make locking difficult and convoluted cross layer locks will always get nacked by myself and others.
Quoted 4 lines
Quoted 1 line
Matt
Ok, if you can find a lockless solution again, that would be beneficial.
I thought about this part, very small race.
- A VF is creating a queue and passes xe_gt_sriov_vf_wait_valid_ggtt()
- vCPUs halt
- vCPUS unhalt
- The original VF thread programs in bad GGTT addresses before fixup, iterrupted before WRITE_ONCE which makes LRC available for fixup
- VF post migration thread fixes GGTTs
- The original VF thread completes WRITE_ONCE for LRC missing the fixup window.
A large lock doesn't work here as xe_lrc_create allocates memory and the VF post migration thread is the path of reclaim, that could invert and deadlock.
I think the solution is:
- We always blindly fixup all GGTT addresses on every LRC in a VF upon GuC context registration.
IMO this can be done in follow up but before feature enablement. The odds of hitting this race is basically impossible given the window is insanely small. I've run xe_exec_threads --r threads-many-queues 100s of times without it failing on the current code base so IMO this is code is stable enough for initial merge.
I suggest we open a Jira so this follow up doesn't get lost.
Matt
Quoted 181 lines
-Tomasz
---Quoted 178 lines
drivers/gpu/drm/xe/xe_exec_queue.c +15
-7+if (IS_ERR (lrc)) { + err =PTR_ERR (lrc); goto err_lrc; } - } - err = q->ops->init (q); -if (err) - goto err_lrc; +/* Pairs with READ_ONCE to xe_exec_queue_contexts_hwsp_rebase */ +WRITE_ONCE (q->lrc[i], lrc); + }return 0 ;@@ -1095,9 +1111,16 @@ int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch)int err =0 ;for (i =0 ; i < q->width; ++i) { -xe_lrc_update_memirq_regs_with_address (q->lrc[i], q->hwe, scratch); -xe_lrc_update_hwctx_regs_with_address (q->lrc[i]); - err =xe_lrc_setup_wa_bb_with_scratch (q->lrc[i], q->hwe, scratch); +struct xe_lrc *lrc; + +/* Pairs with WRITE_ONCE in __xe_exec_queue_init */ + lrc =READ_ONCE (q->lrc[i]); +if (!lrc) +continue ; + +xe_lrc_update_memirq_regs_with_address (lrc, q->hwe, scratch); +xe_lrc_update_hwctx_regs_with_address (lrc); + err =xe_lrc_setup_wa_bb_with_scratch (lrc, q->hwe, scratch);if (err)break ; }drivers/gpu/drm/xe/xe_execlist.c +1
-1@@ -339,7 +339,7 @@ static int execlist_exec_queue_init(struct xe_exec_queue *q)const struct drm_sched_init_args args = { .ops = &drm_sched_ops, .num_rqs =1 , - .credit_limit = q->lrc[0 ]->ring.size /MAX_JOB_SIZE_BYTES , + .credit_limit =xe_lrc_ring_size () /MAX_JOB_SIZE_BYTES , .hang_limit =XE_SCHED_HANG_LIMIT , .timeout =XE_SCHED_JOB_TIMEOUT , .name = q->hwe->name,drivers/gpu/drm/xe/xe_gt_sriov_vf.c +38
-1@@ -480,6 +480,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery) shift, config->ggtt_base);xe_tile_sriov_vf_fixup_ggtt_nodes (gt_to_tile (gt), shift); } + +WRITE_ONCE (gt->sriov.vf.migration.ggtt_need_fixes, false); +smp_wmb ();/* Ensure above write visible before wake */ +wake_up_all (>->sriov.vf.migration.wq); + out:mutex_unlock (&ggtt->lock);return err;@@ -743,7 +748,8 @@ static void vf_start_migration_recovery(struct xe_gt *gt) !gt->sriov.vf.migration.recovery_teardown) { gt->sriov.vf.migration.recovery_queued = true;WRITE_ONCE (gt->sriov.vf.migration.recovery_inprogress, true); -smp_wmb ();/* Ensure above write visable before wake */ +WRITE_ONCE (gt->sriov.vf.migration.ggtt_need_fixes, true); +smp_wmb ();/* Ensure above writes visable before wake */ wake_up_all (>->uc.guc.ct.wq);@@ -1262,6 +1268,7 @@ int xe_gt_sriov_vf_init_early(struct xe_gt *gt) gt->sriov.vf.migration.scratch = buf;spin_lock_init (>->sriov.vf.migration.lock);INIT_WORK (>->sriov.vf.migration.worker, migration_worker_func); +init_waitqueue_head (>->sriov.vf.migration.wq);return 0 ; }@@ -1305,3 +1312,33 @@ bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)return READ_ONCE (gt->sriov.vf.migration.recovery_inprogress); } + +static bool vf_valid_ggtt (struct xe_gt *gt) +{ +struct xe_memirq *memirq = >_to_tile (gt)->memirq; + +xe_gt_assert (gt,IS_SRIOV_VF (gt_to_xe (gt))); + +if (xe_memirq_sw_int_0_irq_pending (memirq, >->uc.guc) || +READ_ONCE (gt->sriov.vf.migration.ggtt_need_fixes)) +return false; + +return true; +} + +/** +* xe_gt_sriov_vf_wait_valid_ggtt() - VF wait for valid GGTT addresses +* @gt: the &xe_gt +*/ +void xe_gt_sriov_vf_wait_valid_ggtt (struct xe_gt *gt) +{ +int ret; + +if (!IS_SRIOV_VF (gt_to_xe (gt))) +return ; + + ret =wait_event_interruptible_timeout (gt->sriov.vf.migration.wq, +vf_valid_ggtt (gt), +HZ *5 ); +XE_WARN_ON (!ret); +}drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2
-0@@ -38,4 +38,6 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p);void xe_gt_sriov_vf_print_runtime (struct xe_gt *gt,struct drm_printer *p);void xe_gt_sriov_vf_print_version (struct xe_gt *gt,struct drm_printer *p); +void xe_gt_sriov_vf_wait_valid_ggtt (struct xe_gt *gt); +#endif drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +5
-0@@ -8,6 +8,7 @@#include <linux/rwsem.h> #include <linux/types.h> +#include <linux/wait.h> #include <linux/workqueue.h> #include "xe_uc_fw_types.h" @@ -50,6 +51,8 @@ struct xe_gt_sriov_vf_migration {struct work_struct worker;/** @lock: Protects recovery_queued, teardown */ spinlock_t lock; +/** @wq: wait queue for migration fixes */ +wait_queue_head_t wq;/** @scratch: Scratch memory for VF recovery */ void *scratch;/** @recovery_teardown: VF post migration recovery is being torn down */ @@ -58,6 +61,8 @@ struct xe_gt_sriov_vf_migration {bool recovery_queued;/** @recovery_inprogress: VF post migration recovery in progress */ bool recovery_inprogress; +/** @ggtt_need_fixes: VF GGTT needs fixes */ +bool ggtt_need_fixes; };/** drivers/gpu/drm/xe/xe_guc_submit.c +1
-1@@ -1943,7 +1943,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q) timeout = (q->vm &&xe_vm_in_lr_mode (q->vm)) ?MAX_SCHEDULE_TIMEOUT :msecs_to_jiffies (q->sched_props.job_timeout_ms); err =xe_sched_init (&ge->sched, &drm_sched_ops, &xe_sched_ops, -NULL , q->lrc[0 ]->ring.size /MAX_JOB_SIZE_BYTES ,64 , +NULL ,xe_lrc_ring_size () /MAX_JOB_SIZE_BYTES ,64 , timeout,guc_to_gt (guc)->ordered_wq,NULL , q->name,gt_to_xe (q->gt)->drm.dev);if (err)drivers/gpu/drm/xe/xe_lrc.h +10
-0@@ -74,6 +74,16 @@ static inline void xe_lrc_put(struct xe_lrc *lrc)kref_put (&lrc->refcount, xe_lrc_destroy); } +/** +* xe_lrc_ring_size() - Xe LRC ring size +* +* Return: Size of LRC size +*/ +static inline size_t xe_lrc_ring_size (void ) +{ +return SZ_16K ; +} +size_t xe_gt_lrc_size (struct xe_gt *gt,enum xe_engine_class class);u32 xe_lrc_pphwsp_offset (struct xe_lrc *lrc);u32 xe_lrc_regs_offset (struct xe_lrc *lrc);
Matthew Brost
maintainerPATCH v4 28/34 drm/xe/vf: Add debug prints for GuC replaying state during VF recovery
In reply to Matthew BrostHelpful to manually verify the GuC state machine can correctly replay the state during a VF post-migration recovery. All replay paths have been manually verified as triggered and working during testing.
--- drivers/gpu/drm/xe/xe_guc_submit.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_guc_submit.c +20 -3
@@ -2306,21 +2306,27 @@ void xe_guc_submit_stop(struct xe_guc *guc) } -static void guc_exec_queue_revert_pending_state_change (struct xe_exec_queue *q) +static void guc_exec_queue_revert_pending_state_change (struct xe_guc *guc, +struct xe_exec_queue *q) {bool pending_enable, pending_disable, pending_resume; pending_enable =exec_queue_pending_enable (q); pending_resume =exec_queue_pending_resume (q); -if (pending_enable && pending_resume) +if (pending_enable && pending_resume) { q->guc->needs_resume = true; +xe_gt_dbg (guc_to_gt (guc),"Replay RESUME - guc_id=%d" , + q->guc->id); + }if (pending_enable && !pending_resume && !exec_queue_pending_tdr_exit (q)) {clear_exec_queue_registered (q);if (xe_exec_queue_is_lr (q))xe_exec_queue_put (q); +xe_gt_dbg (guc_to_gt (guc),"Replay REGISTER - guc_id=%d" , + q->guc->id); }if (pending_enable) {
@@ -2328,6 +2334,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)clear_exec_queue_pending_resume (q);clear_exec_queue_pending_tdr_exit (q);clear_exec_queue_pending_enable (q); +xe_gt_dbg (guc_to_gt (guc),"Replay ENABLE - guc_id=%d" , + q->guc->id); }if (exec_queue_destroyed (q) &&exec_queue_registered (q)) {
@@ -2337,6 +2345,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)else q->guc->needs_cleanup = true;clear_exec_queue_extra_ref (q); +xe_gt_dbg (guc_to_gt (guc),"Replay CLEANUP - guc_id=%d" , + q->guc->id); } pending_disable =exec_queue_pending_disable (q);
@@ -2344,6 +2354,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)if (pending_disable &&exec_queue_suspended (q)) {clear_exec_queue_suspended (q); q->guc->needs_suspend = true; +xe_gt_dbg (guc_to_gt (guc),"Replay SUSPEND - guc_id=%d" , + q->guc->id); }if (pending_disable) {
@@ -2351,6 +2363,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)set_exec_queue_enabled (q);clear_exec_queue_pending_disable (q);clear_exec_queue_check_timeout (q); +xe_gt_dbg (guc_to_gt (guc),"Replay DISABLE - guc_id=%d" , + q->guc->id); } q->guc->resume_time =0 ;
@@ -2376,7 +2390,7 @@ static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)else cancel_delayed_work_sync (&sched->base.work_tdr); -guc_exec_queue_revert_pending_state_change (q); +guc_exec_queue_revert_pending_state_change (guc, q);if (xe_exec_queue_is_parallel (q)) {struct xe_device *xe =guc_to_xe (guc);
@@ -2478,6 +2492,9 @@ static void guc_exec_queue_unpause_prepare(struct xe_guc *guc,list_for_each_entry (s_job, &sched->base.pending_list, list) { job =to_xe_sched_job (s_job); +xe_gt_dbg (guc_to_gt (guc),"Replay JOB - guc_id=%d, seqno=%d" , + q->guc->id,xe_sched_job_seqno (job)); + q->ring_ops->emit_job (job); job->skip_emit = true; }
Signature · 1 line
2.34.1
Re: [PATCH v4 28/34] drm/xe/vf: Add debug prints for GuC replaying state during VF recovery
Quoted 3 lines · Matthew Brost
Helpful to manually verify the GuC state machine can correctly replay the state during a VF post-migration recovery. All replay paths have been manually verified as triggered and working during testing.
Repeating on request as looks like it got lost in v3:
-Tomasz
Quoted 94 lines
Signed-off-by Matthew Brost<[email protected]>--- drivers/gpu/drm/xe/xe_guc_submit.c | 23 ++++++++++++++++++++--- 1 file changed, 20 insertions(+), 3 deletions(-)drivers/gpu/drm/xe/xe_guc_submit.c +20
-3@@ -2306,21 +2306,27 @@ void xe_guc_submit_stop(struct xe_guc *guc) } -static void guc_exec_queue_revert_pending_state_change (struct xe_exec_queue *q) +static void guc_exec_queue_revert_pending_state_change (struct xe_guc *guc, +struct xe_exec_queue *q) {bool pending_enable, pending_disable, pending_resume; pending_enable =exec_queue_pending_enable (q); pending_resume =exec_queue_pending_resume (q); -if (pending_enable && pending_resume) +if (pending_enable && pending_resume) { q->guc->needs_resume = true; +xe_gt_dbg (guc_to_gt (guc),"Replay RESUME - guc_id=%d" , + q->guc->id); + }if (pending_enable && !pending_resume && !exec_queue_pending_tdr_exit (q)) {clear_exec_queue_registered (q);if (xe_exec_queue_is_lr (q))xe_exec_queue_put (q); +xe_gt_dbg (guc_to_gt (guc),"Replay REGISTER - guc_id=%d" , + q->guc->id); }if (pending_enable) {@@ -2328,6 +2334,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)clear_exec_queue_pending_resume (q);clear_exec_queue_pending_tdr_exit (q);clear_exec_queue_pending_enable (q); +xe_gt_dbg (guc_to_gt (guc),"Replay ENABLE - guc_id=%d" , + q->guc->id); }if (exec_queue_destroyed (q) &&exec_queue_registered (q)) {@@ -2337,6 +2345,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)else q->guc->needs_cleanup = true;clear_exec_queue_extra_ref (q); +xe_gt_dbg (guc_to_gt (guc),"Replay CLEANUP - guc_id=%d" , + q->guc->id); } pending_disable =exec_queue_pending_disable (q);@@ -2344,6 +2354,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)if (pending_disable &&exec_queue_suspended (q)) {clear_exec_queue_suspended (q); q->guc->needs_suspend = true; +xe_gt_dbg (guc_to_gt (guc),"Replay SUSPEND - guc_id=%d" , + q->guc->id); }if (pending_disable) {@@ -2351,6 +2363,8 @@ static void guc_exec_queue_revert_pending_state_change(struct xe_exec_queue *q)set_exec_queue_enabled (q);clear_exec_queue_pending_disable (q);clear_exec_queue_check_timeout (q); +xe_gt_dbg (guc_to_gt (guc),"Replay DISABLE - guc_id=%d" , + q->guc->id); } q->guc->resume_time =0 ;@@ -2376,7 +2390,7 @@ static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)else cancel_delayed_work_sync (&sched->base.work_tdr); -guc_exec_queue_revert_pending_state_change (q); +guc_exec_queue_revert_pending_state_change (guc, q);if (xe_exec_queue_is_parallel (q)) {struct xe_device *xe =guc_to_xe (guc);@@ -2478,6 +2492,9 @@ static void guc_exec_queue_unpause_prepare(struct xe_guc *guc,list_for_each_entry (s_job, &sched->base.pending_list, list) { job =to_xe_sched_job (s_job); +xe_gt_dbg (guc_to_gt (guc),"Replay JOB - guc_id=%d, seqno=%d" , + q->guc->id,xe_sched_job_seqno (job)); + q->ring_ops->emit_job (job); job->skip_emit = true; }
Matthew Brost
maintainerPATCH v4 29/34 drm/xe/vf: Workaround for race condition in GuC firmware during VF pause
In reply to Matthew BrostA race condition exists where a paused VF's H2G request can be processed and subsequently rejected. This rejection results in a FAST_REQ failure being delivered to the KMD, which then terminates the CT via a dead worker and triggers a GT reset—an undesirable outcome.
This workaround mitigates the issue by checking if a VF post-migration recovery is in progress and aborting these adverse actions accordingly. The GuC firmware will address this bug in an upcoming release. Once that version is available and VF migration depends on it, this workaround can be safely removed.
--- drivers/gpu/drm/xe/xe_guc_ct.c | 4 ++++ 1 file changed, 4 insertions(+)
drivers/gpu/drm/xe/xe_guc_ct.c +4 -0
@@ -1395,6 +1395,10 @@ static int parse_g2h_response(struct xe_guc_ct *ct, u32 *msg, u32 len)fast_req_report (ct, fence); +/* FIXME: W/A race in the GuC, will get in firmware soon */ +if (xe_gt_recovery_inprogress (gt)) +return 0 ; +CT_DEAD (ct,NULL ,PARSE_G2H_RESPONSE );return -EPROTO ;
Signature · 1 line
2.34.1
Re: [PATCH v4 29/34] drm/xe/vf: Workaround for race condition in GuC firmware during VF pause
Quoted 10 lines · Matthew Brost
A race condition exists where a paused VF's H2G request can be processed and subsequently rejected. This rejection results in a
FAST_REQfailure being delivered to the KMD, which then terminates the CT via a dead worker and triggers a GT reset—an undesirable outcome.This workaround mitigates the issue by checking if a VF post-migration recovery is in progress and aborting these adverse actions accordingly. The GuC firmware will address this bug in an upcoming release. Once that version is available and VF migration depends on it, this workaround can be safely removed.
Quoted 20 lines
Signed-off-by Matthew Brost <[email protected]>--- drivers/gpu/drm/xe/xe_guc_ct.c | 4 ++++ 1 file changed, 4 insertions(+)drivers/gpu/drm/xe/xe_guc_ct.c +4
-0@@ -1395,6 +1395,10 @@ static int parse_g2h_response(struct xe_guc_ct *ct, u32 *msg, u32 len)fast_req_report (ct, fence); +/* FIXME: W/A race in the GuC, will get in firmware soon */ +if (xe_gt_recovery_inprogress (gt)) +return 0 ; +CT_DEAD (ct,NULL ,PARSE_G2H_RESPONSE );return -EPROTO ;
Matthew Brost
maintainerPATCH v4 30/34 drm/xe: Use PPGTT addresses for TLB invalidation to avoid GGTT fixups
In reply to Matthew BrostFrom: Satyanarayana K V P <[email protected]>
The migrate VM builds the CCS metadata save/restore batch buffer (BB) in advance and retains it so the GuC can submit it directly when saving a VM’s state.
When a VM migrates between VFs, the GGTT base can change. Any GGTT-based addresses embedded in the BB would then have to be parsed and patched.
Use PPGTT addresses in the BB (including for TLB invalidation) so the BB remains GGTT-agnostic and requires no address fixups during migration.
--- drivers/gpu/drm/xe/xe_migrate.c | 28 ++++++++++++++++++++-------- 1 file changed, 20 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_migrate.c +20 -8
@@ -980,15 +980,27 @@ struct xe_lrc *xe_migrate_lrc(struct xe_migrate *migrate)return migrate->q->lrc[0 ]; } -static int emit_flush_invalidate (struct xe_exec_queue *q,u32 *dw,int i, -u32 flags) +static u64 migrate_vm_ppgtt_addr_tlb_inval (void ) { -struct xe_lrc *lrc =xe_exec_queue_lrc (q); +/* +* The migrate VM is self-referential so it can modify its own PTEs (see +* pte_update_size() or emit_pte() functions). We reserve NUM_KERNEL_PDE +* entries for kernel operations (copies, clears, CCS migrate), and +* suballocate the rest to user operations (binds/unbinds). With +* NUM_KERNEL_PDE = 15, NUM_KERNEL_PDE - 1 is already used for PTE updates, +* so assign NUM_KERNEL_PDE - 2 for TLB invalidation. +*/ +return (NUM_KERNEL_PDE -2 ) *XE_PAGE_SIZE ; +} + +static int emit_flush_invalidate (u32 *dw,int i,u32 flags) +{ +u64 addr =migrate_vm_ppgtt_addr_tlb_inval (); + dw[i++] =MI_FLUSH_DW |MI_INVALIDATE_TLB |MI_FLUSH_DW_OP_STOREDW |MI_FLUSH_IMM_DW | flags; - dw[i++] =lower_32_bits (xe_lrc_start_seqno_ggtt_addr (lrc)) | -MI_FLUSH_DW_USE_GTT ; - dw[i++] =upper_32_bits (xe_lrc_start_seqno_ggtt_addr (lrc)); + dw[i++] =lower_32_bits (addr); + dw[i++] =upper_32_bits (addr); dw[i++] =MI_NOOP ; dw[i++] =MI_NOOP ;
@@ -1101,11 +1113,11 @@ int xe_migrate_ccs_rw_copy(struct xe_tile *tile, struct xe_exec_queue *q,emit_pte (m, bb, ccs_pt, false, false, &ccs_it, ccs_size, src); - bb->len =emit_flush_invalidate (q, bb->cs, bb->len, flush_flags); + bb->len =emit_flush_invalidate (bb->cs, bb->len, flush_flags); flush_flags =xe_migrate_ccs_copy (m, bb, src_L0_ofs, src_is_pltt, src_L0_ofs, dst_is_pltt, src_L0, ccs_ofs, true); - bb->len =emit_flush_invalidate (q, bb->cs, bb->len, flush_flags); + bb->len =emit_flush_invalidate (bb->cs, bb->len, flush_flags); size -= src_L0; }
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 31/34 drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VF
In reply to Matthew BrostVF CCS restore is a primary GT operation on which the media GT depends. Therefore, it doesn't make much sense to run these operations in parallel. To address this, point the media GT's ordered work queue to the primary GT's ordered work queue on platforms that require (PTL VFs) CCS restore as part of VF post-migration recovery.
--- drivers/gpu/drm/xe/xe_device_types.h | 2 ++ drivers/gpu/drm/xe/xe_gt.c | 16 ++++++++++------ drivers/gpu/drm/xe/xe_gt.h | 2 +- drivers/gpu/drm/xe/xe_pci.c | 6 +++++- drivers/gpu/drm/xe/xe_pci_types.h | 1 + drivers/gpu/drm/xe/xe_tile.c | 2 +- 6 files changed, 20 insertions(+), 9 deletions(-)
drivers/gpu/drm/xe/xe_device_types.h +2 -0
@@ -321,6 +321,8 @@ struct xe_device { u8 skip_mtcfg:1;/** @info.skip_pcode: skip access to PCODE uC */ u8 skip_pcode:1; +/** @info.needs_shared_vf_gt_wq: needs shared GT WQ on VF */ + u8 needs_shared_vf_gt_wq:1; } info;/** @wa_active: keep track of active workarounds */
drivers/gpu/drm/xe/xe_gt.c +10 -6
@@ -72,7 +72,7 @@ static void gt_fini(struct drm_device *drm, void *arg)destroy_workqueue (gt->ordered_wq); } -struct xe_gt *xe_gt_alloc (struct xe_tile *tile) +struct xe_gt *xe_gt_alloc (struct xe_tile *tile,bool use_primary_wq) {struct xe_gt *gt;int err;
@@ -82,12 +82,16 @@ struct xe_gt *xe_gt_alloc(struct xe_tile *tile)return ERR_PTR (-ENOMEM ); gt->tile = tile; - gt->ordered_wq =alloc_ordered_workqueue ("gt-ordered-wq" , -WQ_MEM_RECLAIM ); +if (use_primary_wq) { + gt->ordered_wq = tile->primary_gt->ordered_wq; + }else { + gt->ordered_wq =alloc_ordered_workqueue ("gt-ordered-wq" , +WQ_MEM_RECLAIM ); - err =drmm_add_action_or_reset (>_to_xe (gt)->drm, gt_fini, gt); -if (err) -return ERR_PTR (err); + err =drmm_add_action_or_reset (>_to_xe (gt)->drm, gt_fini, gt); +if (err) +return ERR_PTR (err); + }return gt; }
drivers/gpu/drm/xe/xe_gt.h +1 -1
@@ -28,7 +28,7 @@ static inline bool xe_fault_inject_gt_reset(void)return IS_ENABLED (CONFIG_DEBUG_FS ) &&should_fail (>_reset_failure,1 ); } -struct xe_gt *xe_gt_alloc (struct xe_tile *tile); +struct xe_gt *xe_gt_alloc (struct xe_tile *tile,bool use_primary_wq);int xe_gt_init_early (struct xe_gt *gt);int xe_gt_init (struct xe_gt *gt);void xe_gt_mmio_init (struct xe_gt *gt);
drivers/gpu/drm/xe/xe_pci.c +5 -1
@@ -347,6 +347,7 @@ static const struct xe_device_desc ptl_desc = { .has_sriov = true, .max_gt_per_tile = 2, .needs_scratch = true, + .needs_shared_vf_gt_wq = true, };#undef PLATFORM
@@ -598,6 +599,7 @@ static int xe_info_init_early(struct xe_device *xe, xe->info.skip_mtcfg = desc->skip_mtcfg; xe->info.skip_pcode = desc->skip_pcode; xe->info.needs_scratch = desc->needs_scratch; + xe->info.needs_shared_vf_gt_wq = desc->needs_shared_vf_gt_wq; xe->info.probe_display =IS_ENABLED (CONFIG_DRM_XE_DISPLAY ) && xe_modparam.probe_display &&
@@ -766,7 +768,9 @@ static int xe_info_init(struct xe_device *xe,* Allocate and setup media GT for platforms with standalone * media. */ - tile->media_gt =xe_gt_alloc (tile); + tile->media_gt =xe_gt_alloc (tile, + xe->info.needs_shared_vf_gt_wq && +IS_SRIOV_VF (xe));if (IS_ERR (tile->media_gt))return PTR_ERR (tile->media_gt);
drivers/gpu/drm/xe/xe_pci_types.h +1 -0
@@ -48,6 +48,7 @@ struct xe_device_desc { u8 skip_guc_pc:1; u8 skip_mtcfg:1; u8 skip_pcode:1; + u8 needs_shared_vf_gt_wq:1; };struct xe_graphics_desc {
drivers/gpu/drm/xe/xe_tile.c +1 -1
@@ -149,7 +149,7 @@ int xe_tile_init_early(struct xe_tile *tile, struct xe_device *xe, u8 id)if (err)return err; - tile->primary_gt =xe_gt_alloc (tile); + tile->primary_gt =xe_gt_alloc (tile, false);if (IS_ERR (tile->primary_gt))return PTR_ERR (tile->primary_gt);
Signature · 1 line
2.34.1
Re: [PATCH v4 31/34] drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VF
Quoted 43 lines · Matthew Brost
VF CCS restore is a primary GT operation on which the media GT depends. Therefore, it doesn't make much sense to run these operations in parallel. To address this, point the media GT's ordered work queue to the primary GT's ordered work queue on platforms that require (PTL VFs) CCS restore as part of VF post-migration recovery.
Signed-off-by Matthew Brost<[email protected]>--- drivers/gpu/drm/xe/xe_device_types.h | 2 ++ drivers/gpu/drm/xe/xe_gt.c | 16 ++++++++++------ drivers/gpu/drm/xe/xe_gt.h | 2 +- drivers/gpu/drm/xe/xe_pci.c | 6 +++++- drivers/gpu/drm/xe/xe_pci_types.h | 1 + drivers/gpu/drm/xe/xe_tile.c | 2 +- 6 files changed, 20 insertions(+), 9 deletions(-)drivers/gpu/drm/xe/xe_device_types.h +2
-0@@ -321,6 +321,8 @@ struct xe_device { u8 skip_mtcfg:1;/** @info.skip_pcode: skip access to PCODE uC */ u8 skip_pcode:1; +/** @info.needs_shared_vf_gt_wq: needs shared GT WQ on VF */ + u8 needs_shared_vf_gt_wq:1; } info;/** @wa_active: keep track of active workarounds */ drivers/gpu/drm/xe/xe_gt.c +1
-1@@ -72,7 +72,7 @@ static void gt_fini(struct drm_device *drm, void *arg)destroy_workqueue (gt->ordered_wq); } -struct xe_gt *xe_gt_alloc (struct xe_tile *tile) +struct xe_gt *xe_gt_alloc (struct xe_tile *tile,bool use_primary_wq) {struct xe_gt *gt;int err;
drivers/gpu/drm/xe/xe_gt.c struct xe_gt *xe_gt_alloc(struct xe_tile *tile) · line 89
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt.c +5
-2@@ -82,12 +82,16 @@ struct xe_gt *xe_gt_alloc(struct xe_tile *tile)return ERR_PTR (-ENOMEM ); gt->tile = tile; - gt->ordered_wq =alloc_ordered_workqueue ("gt-ordered-wq" , -WQ_MEM_RECLAIM ); +if (use_primary_wq) { + gt->ordered_wq = tile->primary_gt->ordered_wq; + }else { + gt->ordered_wq =alloc_ordered_workqueue ("gt-ordered-wq" , +WQ_MEM_RECLAIM );
if (!gt->ordered_wq)
return ERR_PTR(-ENOMEM);
or maybe if use_primary_gt, the error returned should be different?
drivers/gpu/drm/xe/xe_gt.c struct xe_gt *xe_gt_alloc(struct xe_tile *tile) · line 91
Quoted 5 lines
drivers/gpu/drm/xe/xe_gt.c +1
-3- err =drmm_add_action_or_reset (>_to_xe (gt)->drm, gt_fini, gt); -if (err) -return ERR_PTR (err); + err =drmm_add_action_or_reset (>_to_xe (gt)->drm, gt_fini, gt);
Looks like we've transformed gt_fini into gt_wq_fini, and it's not getting back to what it was.
Then, the rename as above would make sense.
drivers/gpu/drm/xe/xe_gt.c struct xe_gt *xe_gt_alloc(struct xe_tile *tile) · line 93
Quoted 2 lines
drivers/gpu/drm/xe/xe_gt.c +2
-0+if (err) +return ERR_PTR (err);
I don't see why an issue with registering cleanup handler justifies failing the probe process, but that's out of scope of this review.
-Tomasz
Quoted 73 lines
drivers/gpu/drm/xe/xe_gt.c +1
-0+ }return gt; }drivers/gpu/drm/xe/xe_gt.h +1
-1@@ -28,7 +28,7 @@ static inline bool xe_fault_inject_gt_reset(void)return IS_ENABLED (CONFIG_DEBUG_FS ) &&should_fail (>_reset_failure,1 ); } -struct xe_gt *xe_gt_alloc (struct xe_tile *tile); +struct xe_gt *xe_gt_alloc (struct xe_tile *tile,bool use_primary_wq);int xe_gt_init_early (struct xe_gt *gt);int xe_gt_init (struct xe_gt *gt);void xe_gt_mmio_init (struct xe_gt *gt);drivers/gpu/drm/xe/xe_pci.c +5
-1@@ -347,6 +347,7 @@ static const struct xe_device_desc ptl_desc = { .has_sriov = true, .max_gt_per_tile = 2, .needs_scratch = true, + .needs_shared_vf_gt_wq = true, };#undef PLATFORM@@ -598,6 +599,7 @@ static int xe_info_init_early(struct xe_device *xe, xe->info.skip_mtcfg = desc->skip_mtcfg; xe->info.skip_pcode = desc->skip_pcode; xe->info.needs_scratch = desc->needs_scratch; + xe->info.needs_shared_vf_gt_wq = desc->needs_shared_vf_gt_wq; xe->info.probe_display =IS_ENABLED (CONFIG_DRM_XE_DISPLAY ) && xe_modparam.probe_display &&@@ -766,7 +768,9 @@ static int xe_info_init(struct xe_device *xe,* Allocate and setup media GT for platforms with standalone * media. */ - tile->media_gt =xe_gt_alloc (tile); + tile->media_gt =xe_gt_alloc (tile, + xe->info.needs_shared_vf_gt_wq && +IS_SRIOV_VF (xe));if (IS_ERR (tile->media_gt))return PTR_ERR (tile->media_gt);drivers/gpu/drm/xe/xe_pci_types.h +1
-0@@ -48,6 +48,7 @@ struct xe_device_desc { u8 skip_guc_pc:1; u8 skip_mtcfg:1; u8 skip_pcode:1; + u8 needs_shared_vf_gt_wq:1; };struct xe_graphics_desc {drivers/gpu/drm/xe/xe_tile.c +1
-1@@ -149,7 +149,7 @@ int xe_tile_init_early(struct xe_tile *tile, struct xe_device *xe, u8 id)if (err)return err; - tile->primary_gt =xe_gt_alloc (tile); + tile->primary_gt =xe_gt_alloc (tile, false);if (IS_ERR (tile->primary_gt))return PTR_ERR (tile->primary_gt);
Matthew Brost
maintainerRe: [PATCH v4 31/34] drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VF
Quoted 55 lines · Tomasz
Quoted 41 lines · Matthew Brost
VF CCS restore is a primary GT operation on which the media GT depends. Therefore, it doesn't make much sense to run these operations in parallel. To address this, point the media GT's ordered work queue to the primary GT's ordered work queue on platforms that require (PTL VFs) CCS restore as part of VF post-migration recovery.
Signed-off-by Matthew Brost<[email protected]>--- drivers/gpu/drm/xe/xe_device_types.h | 2 ++ drivers/gpu/drm/xe/xe_gt.c | 16 ++++++++++------ drivers/gpu/drm/xe/xe_gt.h | 2 +- drivers/gpu/drm/xe/xe_pci.c | 6 +++++- drivers/gpu/drm/xe/xe_pci_types.h | 1 + drivers/gpu/drm/xe/xe_tile.c | 2 +- 6 files changed, 20 insertions(+), 9 deletions(-)drivers/gpu/drm/xe/xe_device_types.h +2
-0@@ -321,6 +321,8 @@ struct xe_device { u8 skip_mtcfg:1;/** @info.skip_pcode: skip access to PCODE uC */ u8 skip_pcode:1; +/** @info.needs_shared_vf_gt_wq: needs shared GT WQ on VF */ + u8 needs_shared_vf_gt_wq:1; } info;/** @wa_active: keep track of active workarounds */ drivers/gpu/drm/xe/xe_gt.c +1
-1@@ -72,7 +72,7 @@ static void gt_fini(struct drm_device *drm, void *arg)destroy_workqueue (gt->ordered_wq); } -struct xe_gt *xe_gt_alloc (struct xe_tile *tile) +struct xe_gt *xe_gt_alloc (struct xe_tile *tile,bool use_primary_wq) {struct xe_gt *gt;int err;drivers/gpu/drm/xe/xe_gt.c struct xe_gt *xe_gt_alloc(struct xe_tile *tile) · line 88
Quoted 9 lines
drivers/gpu/drm/xe/xe_gt.c +5
-2@@ -82,12 +82,16 @@ struct xe_gt *xe_gt_alloc(struct xe_tile *tile)return ERR_PTR (-ENOMEM ); gt->tile = tile; - gt->ordered_wq =alloc_ordered_workqueue ("gt-ordered-wq" , -WQ_MEM_RECLAIM ); +if (use_primary_wq) { + gt->ordered_wq = tile->primary_gt->ordered_wq; + }else { + gt->ordered_wq =alloc_ordered_workqueue ("gt-ordered-wq" , +WQ_MEM_RECLAIM );if (!gt->ordered_wq)
Quoted 3 lines
return ERR_PTR(-ENOMEM);or maybe if
use_primary_gt, the error returned should be different?
What?
1. You can't fail in this case.
2. The error code isn't parsed at the caller.
So again huh?
Quoted 10 lines
drivers/gpu/drm/xe/xe_gt.c struct xe_gt *xe_gt_alloc(struct xe_tile *tile) · line 91
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt.c +1
-3- err =drmm_add_action_or_reset (>_to_xe (gt)->drm, gt_fini, gt); -if (err) -return ERR_PTR (err); + err =drmm_add_action_or_reset (>_to_xe (gt)->drm, gt_fini, gt);Looks like we've transformed
gt_finiintogt_wq_fini, and it's not getting back to what it was.Then, the rename as above would make sense.
This code is gone upstream. We switched over to using the drmm helper to allocate the wq.
Quoted 6 lines
drivers/gpu/drm/xe/xe_gt.c struct xe_gt *xe_gt_alloc(struct xe_tile *tile) · line 93
Quoted 2 lines
drivers/gpu/drm/xe/xe_gt.c +2
-0+if (err) +return ERR_PTR (err);I don't see why an issue with registering cleanup handler justifies failing the probe process, but that's out of scope of this review.
The entire driver does this and it is needed. The driver can't unload without leaking resources if cleanup handler fails to register. What the 'or_reset' part does is immediately run the handler upon failure ensuring no leaks. This is Linux 101 stuff - i.e., you can never leak resources in Linux.
Matt
Quoted 67 lines
-Tomasz
Quoted 65 lines
drivers/gpu/drm/xe/xe_gt.c +1
-0+ }return gt; }drivers/gpu/drm/xe/xe_gt.h +1
-1@@ -28,7 +28,7 @@ static inline bool xe_fault_inject_gt_reset(void)return IS_ENABLED (CONFIG_DEBUG_FS ) &&should_fail (>_reset_failure,1 ); } -struct xe_gt *xe_gt_alloc (struct xe_tile *tile); +struct xe_gt *xe_gt_alloc (struct xe_tile *tile,bool use_primary_wq);int xe_gt_init_early (struct xe_gt *gt);int xe_gt_init (struct xe_gt *gt);void xe_gt_mmio_init (struct xe_gt *gt);drivers/gpu/drm/xe/xe_pci.c +5
-1@@ -347,6 +347,7 @@ static const struct xe_device_desc ptl_desc = { .has_sriov = true, .max_gt_per_tile = 2, .needs_scratch = true, + .needs_shared_vf_gt_wq = true, };#undef PLATFORM@@ -598,6 +599,7 @@ static int xe_info_init_early(struct xe_device *xe, xe->info.skip_mtcfg = desc->skip_mtcfg; xe->info.skip_pcode = desc->skip_pcode; xe->info.needs_scratch = desc->needs_scratch; + xe->info.needs_shared_vf_gt_wq = desc->needs_shared_vf_gt_wq; xe->info.probe_display =IS_ENABLED (CONFIG_DRM_XE_DISPLAY ) && xe_modparam.probe_display &&@@ -766,7 +768,9 @@ static int xe_info_init(struct xe_device *xe,* Allocate and setup media GT for platforms with standalone * media. */ - tile->media_gt =xe_gt_alloc (tile); + tile->media_gt =xe_gt_alloc (tile, + xe->info.needs_shared_vf_gt_wq && +IS_SRIOV_VF (xe));if (IS_ERR (tile->media_gt))return PTR_ERR (tile->media_gt);drivers/gpu/drm/xe/xe_pci_types.h +1
-0@@ -48,6 +48,7 @@ struct xe_device_desc { u8 skip_guc_pc:1; u8 skip_mtcfg:1; u8 skip_pcode:1; + u8 needs_shared_vf_gt_wq:1; };struct xe_graphics_desc {drivers/gpu/drm/xe/xe_tile.c +1
-1@@ -149,7 +149,7 @@ int xe_tile_init_early(struct xe_tile *tile, struct xe_device *xe, u8 id)if (err)return err; - tile->primary_gt =xe_gt_alloc (tile); + tile->primary_gt =xe_gt_alloc (tile, false);if (IS_ERR (tile->primary_gt))return PTR_ERR (tile->primary_gt);
Matthew Brost
maintainerPATCH v4 32/34 drm/xe/vf: Ensure media GT VF recovery runs after primary GT on PTL
In reply to Matthew BrostIt is possible that the media GT's VF post-migration recovery work item gets scheduled before the primary GT's work item. Since the media GT depends on the primary GT's work item to complete CCS restore, if the media GT's work item is scheduled first, detect this condition and re-queue the media GT's work item for a later time.
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +27 -2
@@ -1110,8 +1110,22 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) pf_version->major, pf_version->minor); } -static void vf_post_migration_shutdown (struct xe_gt *gt) +static bool vf_post_migration_shutdown (struct xe_gt *gt) { +struct xe_device *xe =gt_to_xe (gt); + +/* +* On platforms where CCS must be restored by the primary GT, the media +* GT's VF post-migration recovery must run afterward. Detect this case +* and re-queue the media GT's restore work item if necessary. +*/ +if (xe->info.needs_shared_vf_gt_wq &&xe_gt_is_media_type (gt)) { +struct xe_gt *primary_gt =gt_to_tile (gt)->primary_gt; + +if (xe_gt_sriov_vf_recovery_inprogress (primary_gt)) +return true; + } +spin_lock_irq (>->sriov.vf.migration.lock); gt->sriov.vf.migration.recovery_queued = false;spin_unlock_irq (>->sriov.vf.migration.lock);
@@ -1119,6 +1133,8 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)xe_guc_ct_flush_and_stop (>->uc.guc.ct);xe_guc_submit_pause (>->uc.guc);xe_tlb_inval_reset (>->tlb_inval); + +return false; }static size_t post_migration_scratch_size (struct xe_device *xe)
@@ -1193,11 +1209,14 @@ static void vf_post_migration_recovery(struct xe_gt *gt) {struct xe_device *xe =gt_to_xe (gt);int err; +bool retry;xe_gt_sriov_dbg (gt,"migration recovery in progress\n" );xe_pm_runtime_get (xe); -vf_post_migration_shutdown (gt); + retry =vf_post_migration_shutdown (gt); +if (retry) + goto queue;if (!xe_sriov_vf_migration_supported (xe)) {xe_gt_sriov_err (gt,"migration is not supported\n" );
@@ -1225,6 +1244,12 @@ static void vf_post_migration_recovery(struct xe_gt *gt)xe_pm_runtime_put (xe);xe_gt_sriov_err (gt,"migration recovery failed (%pe)\n" ,ERR_PTR (err));xe_device_declare_wedged (xe); +return ; + +queue: +xe_gt_sriov_info (gt,"Re-queuing GT recovery\n" ); +queue_work (gt->ordered_wq, >->sriov.vf.migration.worker); +xe_pm_runtime_put (xe); }static void migration_worker_func (struct work_struct *w)
Signature · 1 line
2.34.1
Re: [PATCH v4 32/34] drm/xe/vf: Ensure media GT VF recovery runs after primary GT on PTL
Quoted 5 lines · Matthew Brost
It is possible that the media GT's VF post-migration recovery work item gets scheduled before the primary GT's work item. Since the media GT depends on the primary GT's work item to complete CCS restore, if the media GT's work item is scheduled first, detect this condition and re-queue the media GT's work item for a later time.
One print related comment below; other than that:
Quoted 60 lines
Signed-off-by Matthew Brost<[email protected]>--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 29 +++++++++++++++++++++++++++-- 1 file changed, 27 insertions(+), 2 deletions(-)drivers/gpu/drm/xe/xe_gt_sriov_vf.c +21
-2@@ -1110,8 +1110,22 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) pf_version->major, pf_version->minor); } -static void vf_post_migration_shutdown (struct xe_gt *gt) +static bool vf_post_migration_shutdown (struct xe_gt *gt) { +struct xe_device *xe =gt_to_xe (gt); + +/* +* On platforms where CCS must be restored by the primary GT, the media +* GT's VF post-migration recovery must run afterward. Detect this case +* and re-queue the media GT's restore work item if necessary. +*/ +if (xe->info.needs_shared_vf_gt_wq &&xe_gt_is_media_type (gt)) { +struct xe_gt *primary_gt =gt_to_tile (gt)->primary_gt; + +if (xe_gt_sriov_vf_recovery_inprogress (primary_gt)) +return true; + } +spin_lock_irq (>->sriov.vf.migration.lock); gt->sriov.vf.migration.recovery_queued = false;spin_unlock_irq (>->sriov.vf.migration.lock);@@ -1119,6 +1133,8 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)xe_guc_ct_flush_and_stop (>->uc.guc.ct);xe_guc_submit_pause (>->uc.guc);xe_tlb_inval_reset (>->tlb_inval); + +return false; }static size_t post_migration_scratch_size (struct xe_device *xe)@@ -1193,11 +1209,14 @@ static void vf_post_migration_recovery(struct xe_gt *gt) {struct xe_device *xe =gt_to_xe (gt);int err; +bool retry;xe_gt_sriov_dbg (gt,"migration recovery in progress\n" );xe_pm_runtime_get (xe); -vf_post_migration_shutdown (gt); + retry =vf_post_migration_shutdown (gt); +if (retry) + goto queue;if (!xe_sriov_vf_migration_supported (xe)) {xe_gt_sriov_err (gt,"migration is not supported\n" );
drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_post_migration_recovery(struct xe_gt *gt) · line 1250
Quoted 7 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4
-0@@ -1225,6 +1244,12 @@ static void vf_post_migration_recovery(struct xe_gt *gt)xe_pm_runtime_put (xe);xe_gt_sriov_err (gt,"migration recovery failed (%pe)\n" ,ERR_PTR (err));xe_device_declare_wedged (xe); +return ; + +queue: +xe_gt_sriov_info (gt,"Re-queuing GT recovery\n" );
In every other message we call it "migrationrecovery", let's keep that consistent; GT will be already mentioned due to selected function.
-Tomasz
Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2
-0+queue_work (gt->ordered_wq, >->sriov.vf.migration.worker); +xe_pm_runtime_put (xe); }static void migration_worker_func (struct work_struct *w)
Matthew Brost
maintainerPATCH v4 33/34 drm/xe/vf: Rebase CCS save/restore BB GGTT addresses
In reply to Matthew BrostRebase the CCS save/restore BB's GGTT addresses during VF post-migration recovery by setting the software ring tail to zero, the LRC ring head to zero, and rewriting the jump-to-BB instructions.
--- drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 4 ++++ drivers/gpu/drm/xe/xe_sriov_vf_ccs.c | 28 ++++++++++++++++++++++++++++ drivers/gpu/drm/xe/xe_sriov_vf_ccs.h | 1 + 3 files changed, 33 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
@@ -34,6 +34,7 @@#include "xe_pm.h" #include "xe_sriov.h" #include "xe_sriov_vf.h" +#include "xe_sriov_vf_ccs.h" #include "xe_tile_sriov_vf.h" #include "xe_tlb_inval.h" #include "xe_uc_fw.h"
@@ -1151,6 +1152,9 @@ static int vf_post_migration_fixups(struct xe_gt *gt)if (err)return err; +if (xe_gt_is_main_type (gt)) +xe_sriov_vf_ccs_rebase (gt_to_xe (gt)); +xe_gt_sriov_vf_default_lrcs_hwsp_rebase (gt); err =xe_guc_contexts_hwsp_rebase (>->uc.guc, buf);if (err)
drivers/gpu/drm/xe/xe_sriov_vf_ccs.c +28 -0
@@ -175,6 +175,15 @@ static void ccs_rw_update_ring(struct xe_sriov_vf_ccs_ctx *ctx)struct xe_lrc *lrc =xe_exec_queue_lrc (ctx->mig_q);u32 dw[10 ], i =0 ; +/* +* XXX: Save/restore fixes — for some reason, the GuC only accepts the +* save/restore context if the LRC head pointer is zero. This is evident +* from repeated VF migrations failing when the LRC head pointer is +* non-zero. +*/ + lrc->ring.tail =0 ; +xe_lrc_set_ring_head (lrc,0 ); + dw[i++] =MI_ARB_ON_OFF |MI_ARB_ENABLE ; dw[i++] =MI_BATCH_BUFFER_START |XE_INSTR_NUM_DW (3 ); dw[i++] =lower_32_bits (addr);
@@ -186,6 +195,25 @@ static void ccs_rw_update_ring(struct xe_sriov_vf_ccs_ctx *ctx)xe_lrc_set_ring_tail (lrc, lrc->ring.tail); } +/** +* xe_sriov_vf_ccs_rebase - Rebase GGTT addresses for CCS save / restore +* @xe: the &xe_device. +*/ +void xe_sriov_vf_ccs_rebase (struct xe_device *xe) +{ +enum xe_sriov_vf_ccs_rw_ctxs ctx_id; + +if (!IS_VF_CCS_READY (xe)) +return ; + +for_each_ccs_rw_ctx (ctx_id) { +struct xe_sriov_vf_ccs_ctx *ctx = + &xe->sriov.vf.ccs.contexts[ctx_id]; + +ccs_rw_update_ring (ctx); + } +} +static int register_save_restore_context (struct xe_sriov_vf_ccs_ctx *ctx) {int ctx_type;
drivers/gpu/drm/xe/xe_sriov_vf_ccs.h +1 -0
@@ -18,6 +18,7 @@ int xe_sriov_vf_ccs_init(struct xe_device *xe);int xe_sriov_vf_ccs_attach_bo (struct xe_bo *bo);int xe_sriov_vf_ccs_detach_bo (struct xe_bo *bo);int xe_sriov_vf_ccs_register_context (struct xe_device *xe); +void xe_sriov_vf_ccs_rebase (struct xe_device *xe);void xe_sriov_vf_ccs_print (struct xe_device *xe,struct drm_printer *p);static inline bool xe_sriov_vf_ccs_ready (struct xe_device *xe)
Signature · 1 line
2.34.1
Matthew Brost
maintainerPATCH v4 34/34 drm/xe/guc: Increase wait timeout to 2sec after BUSY reply from GuC
In reply to Matthew BrostFrom: Satyanarayana K V P <[email protected]>
Some VF2GUC actions may take longer to process. Increase default timeout after received BUSY indication to 2sec to cover all worst case scenarios.
--- drivers/gpu/drm/xe/xe_guc.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_guc.c +1 -1
@@ -1439,7 +1439,7 @@ int xe_guc_mmio_send_recv(struct xe_guc *guc, const u32 *request,BUILD_BUG_ON ((GUC_HXG_TYPE_RESPONSE_SUCCESS ^GUC_HXG_TYPE_RESPONSE_FAILURE ) !=1 ); ret =xe_mmio_wait32 (mmio, reply_reg, resp_mask, resp_mask, -1000000 , &header, false); +2000000 , &header, false);if (unlikely (FIELD_GET (GUC_HXG_MSG_0_ORIGIN , header) !=GUC_HXG_ORIGIN_GUC ))
Signature · 1 line
2.34.1
✗ CI.checkpatch: warning for VF migration redesign (rev4)
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: VF migration redesign (rev4)
URL : https://patchwork.freedesktop.org/series/154627/
State : warning
== Summary ==
+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master fbd08a78c3a3bb17964db2a326514c69c1dca660
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 41e3a0eb75524db919db455fb5047d1225b057f4
Author: Satyanarayana K V P <[email protected]> Date: Wed Oct 1 22:54:02 2025 -0700
drm/xe/guc: Increase wait timeout to 2sec after BUSY reply from GuC
Some VF2GUC actions may take longer to process. Increase default timeout after received BUSY indication to 2sec to cover all worst case scenarios.
+ /mt/dim checkpatch 5f87abb254011980c8332008cfff72d2cfab4952 drm-intel 623235e2e133 drm/xe: Add NULL checks to scratch LRC allocationda97a3b426bb Revert "drm/xe/vf: Rebase exec queue parallel commands during migration recovery" e3df2f24b222 Revert "drm/xe/vf: Post migration, repopulate ring area for pending request" 31fdc27a74ab Revert "drm/xe/vf: Fixup CTB send buffer messages after migration" ae05362e7b90 drm/xe: Save off position in ring in which a job was programmed 0c0a507c01aa drm/xe/guc: Track pending-enable source in submission state 9ddd11bb25b3 drm/xe: Track LR jobs in DRM scheduler pending list74f4a45c4c9e drm/xe: Don't change LRC ring head on job resubmission6bc8efa9c015 drm/xe: Make LRC W/A scratch buffer usage consistent8ae3c6507425 drm/xe/guc: Document GuC submission backendd7d770d2bf86 drm/xe/vf: Add xe_gt_recovery_inprogress helper0bb18f340077 drm/xe/vf: Make VF recovery run on per-GT worker36d1c68ca5c0 drm/xe/vf: Abort H2G sends during VF post-migration recovery d623e75acf95 drm/xe/vf: Remove memory allocations from VF post migration recovery 05563fc12579 drm/xe/vf: Close multi-GT GGTT shift race
-:489: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating? #489:
new file mode 100644
total: 0 errors, 1 warnings, 0 checks, 463 lines checked7e3c033eb30d drm/xe/vf: Teardown VF post migration worker on driver unload 8b6e3e0786f3 drm/xe/vf: Don't allow GT reset to be queued during VF post migration recovery 8f6a1861848f drm/xe/vf: Wakeup in GuC backend on VF post migration recovery 013380659419 drm/xe/vf: Avoid indefinite blocking in preempt rebind worker for VFs supporting migration e3d6206f1390 drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context registerfcd024d7b31c drm/xe/vf: Flush and stop CTs in VF post migration recovery3aac689dd88f drm/xe/vf: Reset TLB invalidations during VF post migration recovery af20893b143d drm/xe/vf: Kickstart after resfix in VF post migration recovery d7a8959f55db drm/xe/vf: Start CTs before resfix VF post migration recovery 86afb801ba4b drm/xe/vf: Abort VF post migration recovery on failure64e5ab26c042 drm/xe/vf: Replay GuC submission state on pause / unpauseefaeb06521d3 drm/xe: Move queue init before LRC creationd8f3c39e9bdf drm/xe/vf: Add debug prints for GuC replaying state during VF recovery 2f7188363c7a drm/xe/vf: Workaround for race condition in GuC firmware during VF pause db70bdda3e74 drm/xe: Use PPGTT addresses for TLB invalidation to avoid GGTT fixups 7efc9efae500 drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VF b31130252ecc drm/xe/vf: Ensure media GT VF recovery runs after primary GT on PTL 636718bc6c41 drm/xe/vf: Rebase CCS save/restore BB GGTT addresses41e3a0eb7552 drm/xe/guc: Increase wait timeout to 2sec after BUSY reply from GuC
Patchwork
CI✓ CI.KUnit: success for VF migration redesign (rev4)
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: VF migration redesign (rev4)
URL : https://patchwork.freedesktop.org/series/154627/
State : success
== Summary ==
+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig [06:45:56] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig [06:46:00] Building KUnit Kernel ... Populating config with: $ make ARCH=um O=.kunit olddefconfig Building with: $ make all compile_commands.json scripts_gdb ARCH=um O=.kunit --jobs=48 [06:46:30] Starting KUnit Kernel (1/1)... [06:46:30] ============================================================ Running tests with: $ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt [06:46:30] ================== guc_buf (11 subtests) =================== [06:46:30] [PASSED] test_smallest [06:46:30] [PASSED] test_largest [06:46:30] [PASSED] test_granular [06:46:30] [PASSED] test_unique [06:46:30] [PASSED] test_overlap [06:46:30] [PASSED] test_reusable [06:46:30] [PASSED] test_too_big [06:46:30] [PASSED] test_flush [06:46:30] [PASSED] test_lookup [06:46:30] [PASSED] test_data [06:46:30] [PASSED] test_class [06:46:30] ===================== [PASSED] guc_buf ===================== [06:46:30] =================== guc_dbm (7 subtests) =================== [06:46:30] [PASSED] test_empty [06:46:30] [PASSED] test_default [06:46:30] ======================== test_size ======================== [06:46:30] [PASSED] 4 [06:46:30] [PASSED] 8 [06:46:30] [PASSED] 32 [06:46:30] [PASSED] 256 [06:46:30] ==================== [PASSED] test_size ==================== [06:46:30] ======================= test_reuse ======================== [06:46:30] [PASSED] 4 [06:46:30] [PASSED] 8 [06:46:30] [PASSED] 32 [06:46:30] [PASSED] 256 [06:46:30] =================== [PASSED] test_reuse ==================== [06:46:30] =================== test_range_overlap ==================== [06:46:30] [PASSED] 4 [06:46:30] [PASSED] 8 [06:46:30] [PASSED] 32 [06:46:30] [PASSED] 256 [06:46:30] =============== [PASSED] test_range_overlap ================ [06:46:30] =================== test_range_compact ==================== [06:46:30] [PASSED] 4 [06:46:30] [PASSED] 8 [06:46:30] [PASSED] 32 [06:46:30] [PASSED] 256 [06:46:30] =============== [PASSED] test_range_compact ================ [06:46:30] ==================== test_range_spare ===================== [06:46:30] [PASSED] 4 [06:46:30] [PASSED] 8 [06:46:30] [PASSED] 32 [06:46:30] [PASSED] 256 [06:46:30] ================ [PASSED] test_range_spare ================= [06:46:30] ===================== [PASSED] guc_dbm ===================== [06:46:30] =================== guc_idm (6 subtests) =================== [06:46:30] [PASSED] bad_init [06:46:30] [PASSED] no_init [06:46:30] [PASSED] init_fini [06:46:30] [PASSED] check_used [06:46:30] [PASSED] check_quota [06:46:30] [PASSED] check_all [06:46:30] ===================== [PASSED] guc_idm ===================== [06:46:30] ================== no_relay (3 subtests) =================== [06:46:30] [PASSED] xe_drops_guc2pf_if_not_ready [06:46:30] [PASSED] xe_drops_guc2vf_if_not_ready [06:46:30] [PASSED] xe_rejects_send_if_not_ready [06:46:30] ==================== [PASSED] no_relay ===================== [06:46:30] ================== pf_relay (14 subtests) ================== [06:46:30] [PASSED] pf_rejects_guc2pf_too_short [06:46:30] [PASSED] pf_rejects_guc2pf_too_long [06:46:30] [PASSED] pf_rejects_guc2pf_no_payload [06:46:30] [PASSED] pf_fails_no_payload [06:46:30] [PASSED] pf_fails_bad_origin [06:46:30] [PASSED] pf_fails_bad_type [06:46:30] [PASSED] pf_txn_reports_error [06:46:30] [PASSED] pf_txn_sends_pf2guc [06:46:30] [PASSED] pf_sends_pf2guc [06:46:30] [SKIPPED] pf_loopback_nop [06:46:30] [SKIPPED] pf_loopback_echo [06:46:30] [SKIPPED] pf_loopback_fail [06:46:30] [SKIPPED] pf_loopback_busy [06:46:30] [SKIPPED] pf_loopback_retry [06:46:30] ==================== [PASSED] pf_relay ===================== [06:46:30] ================== vf_relay (3 subtests) =================== [06:46:30] [PASSED] vf_rejects_guc2vf_too_short [06:46:30] [PASSED] vf_rejects_guc2vf_too_long [06:46:30] [PASSED] vf_rejects_guc2vf_no_payload [06:46:30] ==================== [PASSED] vf_relay ===================== [06:46:30] ===================== lmtt (1 subtest) ===================== [06:46:30] ======================== test_ops ========================= [06:46:30] [PASSED] 2-level [06:46:30] [PASSED] multi-level [06:46:30] ==================== [PASSED] test_ops ===================== [06:46:30] ====================== [PASSED] lmtt ======================= [06:46:30] ================= pf_service (11 subtests) ================= [06:46:30] [PASSED] pf_negotiate_any [06:46:30] [PASSED] pf_negotiate_base_match [06:46:30] [PASSED] pf_negotiate_base_newer [06:46:30] [PASSED] pf_negotiate_base_next [06:46:30] [SKIPPED] pf_negotiate_base_older [06:46:30] [PASSED] pf_negotiate_base_prev [06:46:30] [PASSED] pf_negotiate_latest_match [06:46:30] [PASSED] pf_negotiate_latest_newer [06:46:30] [PASSED] pf_negotiate_latest_next [06:46:30] [SKIPPED] pf_negotiate_latest_older [06:46:30] [SKIPPED] pf_negotiate_latest_prev [06:46:30] =================== [PASSED] pf_service ==================== [06:46:30] ================= xe_guc_g2g (2 subtests) ================== [06:46:30] ============== xe_live_guc_g2g_kunit_default ============== [06:46:30] ========= [SKIPPED] xe_live_guc_g2g_kunit_default ========== [06:46:30] ============== xe_live_guc_g2g_kunit_allmem =============== [06:46:30] ========== [SKIPPED] xe_live_guc_g2g_kunit_allmem ========== [06:46:30] =================== [SKIPPED] xe_guc_g2g =================== [06:46:30] =================== xe_mocs (2 subtests) =================== [06:46:30] ================ xe_live_mocs_kernel_kunit ================ [06:46:30] =========== [SKIPPED] xe_live_mocs_kernel_kunit ============ [06:46:30] ================ xe_live_mocs_reset_kunit ================= [06:46:30] ============ [SKIPPED] xe_live_mocs_reset_kunit ============ [06:46:30] ==================== [SKIPPED] xe_mocs ===================== [06:46:30] ================= xe_migrate (2 subtests) ================== [06:46:30] ================= xe_migrate_sanity_kunit ================= [06:46:30] ============ [SKIPPED] xe_migrate_sanity_kunit ============= [06:46:30] ================== xe_validate_ccs_kunit ================== [06:46:30] ============= [SKIPPED] xe_validate_ccs_kunit ============== [06:46:30] =================== [SKIPPED] xe_migrate =================== [06:46:30] ================== xe_dma_buf (1 subtest) ================== [06:46:30] ==================== xe_dma_buf_kunit ===================== [06:46:30] ================ [SKIPPED] xe_dma_buf_kunit ================ [06:46:30] =================== [SKIPPED] xe_dma_buf =================== [06:46:30] ================= xe_bo_shrink (1 subtest) ================= [06:46:30] =================== xe_bo_shrink_kunit ==================== [06:46:30] =============== [SKIPPED] xe_bo_shrink_kunit =============== [06:46:30] ================== [SKIPPED] xe_bo_shrink ================== [06:46:30] ==================== xe_bo (2 subtests) ==================== [06:46:30] ================== xe_ccs_migrate_kunit =================== [06:46:30] ============== [SKIPPED] xe_ccs_migrate_kunit ============== [06:46:30] ==================== xe_bo_evict_kunit ==================== [06:46:30] =============== [SKIPPED] xe_bo_evict_kunit ================ [06:46:30] ===================== [SKIPPED] xe_bo ====================== [06:46:30] ==================== args (11 subtests) ==================== [06:46:30] [PASSED] count_args_test [06:46:30] [PASSED] call_args_example [06:46:30] [PASSED] call_args_test [06:46:30] [PASSED] drop_first_arg_example [06:46:30] [PASSED] drop_first_arg_test [06:46:30] [PASSED] first_arg_example [06:46:30] [PASSED] first_arg_test [06:46:30] [PASSED] last_arg_example [06:46:30] [PASSED] last_arg_test [06:46:30] [PASSED] pick_arg_example [06:46:30] [PASSED] sep_comma_example [06:46:30] ====================== [PASSED] args ======================= [06:46:30] =================== xe_pci (3 subtests) ==================== [06:46:30] ==================== check_graphics_ip ==================== [06:46:30] [PASSED] 12.00 Xe_LP [06:46:30] [PASSED] 12.10 Xe_LP+ [06:46:30] [PASSED] 12.55 Xe_HPG [06:46:30] [PASSED] 12.60 Xe_HPC [06:46:30] [PASSED] 12.70 Xe_LPG [06:46:30] [PASSED] 12.71 Xe_LPG [06:46:30] [PASSED] 12.74 Xe_LPG+ [06:46:30] [PASSED] 20.01 Xe2_HPG [06:46:30] [PASSED] 20.02 Xe2_HPG [06:46:30] [PASSED] 20.04 Xe2_LPG [06:46:30] [PASSED] 30.00 Xe3_LPG [06:46:30] [PASSED] 30.01 Xe3_LPG [06:46:30] [PASSED] 30.03 Xe3_LPG [06:46:30] ================ [PASSED] check_graphics_ip ================ [06:46:30] ===================== check_media_ip ====================== [06:46:30] [PASSED] 12.00 Xe_M [06:46:30] [PASSED] 12.55 Xe_HPM [06:46:30] [PASSED] 13.00 Xe_LPM+ [06:46:30] [PASSED] 13.01 Xe2_HPM [06:46:30] [PASSED] 20.00 Xe2_LPM [06:46:30] [PASSED] 30.00 Xe3_LPM [06:46:30] [PASSED] 30.02 Xe3_LPM [06:46:30] ================= [PASSED] check_media_ip ================== [06:46:30] ================= check_platform_gt_count ================= [06:46:30] [PASSED] 0x9A60 (TIGERLAKE) [06:46:30] [PASSED] 0x9A68 (TIGERLAKE) [06:46:30] [PASSED] 0x9A70 (TIGERLAKE) [06:46:30] [PASSED] 0x9A40 (TIGERLAKE) [06:46:30] [PASSED] 0x9A49 (TIGERLAKE) [06:46:30] [PASSED] 0x9A59 (TIGERLAKE) [06:46:30] [PASSED] 0x9A78 (TIGERLAKE) [06:46:30] [PASSED] 0x9AC0 (TIGERLAKE) [06:46:30] [PASSED] 0x9AC9 (TIGERLAKE) [06:46:30] [PASSED] 0x9AD9 (TIGERLAKE) [06:46:30] [PASSED] 0x9AF8 (TIGERLAKE) [06:46:30] [PASSED] 0x4C80 (ROCKETLAKE) [06:46:30] [PASSED] 0x4C8A (ROCKETLAKE) [06:46:30] [PASSED] 0x4C8B (ROCKETLAKE) [06:46:30] [PASSED] 0x4C8C (ROCKETLAKE) [06:46:30] [PASSED] 0x4C90 (ROCKETLAKE) [06:46:30] [PASSED] 0x4C9A (ROCKETLAKE) [06:46:30] [PASSED] 0x4680 (ALDERLAKE_S) [06:46:30] [PASSED] 0x4682 (ALDERLAKE_S) [06:46:30] [PASSED] 0x4688 (ALDERLAKE_S) [06:46:30] [PASSED] 0x468A (ALDERLAKE_S) [06:46:30] [PASSED] 0x468B (ALDERLAKE_S) [06:46:30] [PASSED] 0x4690 (ALDERLAKE_S) [06:46:30] [PASSED] 0x4692 (ALDERLAKE_S) [06:46:30] [PASSED] 0x4693 (ALDERLAKE_S) [06:46:30] [PASSED] 0x46A0 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46A1 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46A2 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46A3 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46A6 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46A8 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46AA (ALDERLAKE_P) [06:46:30] [PASSED] 0x462A (ALDERLAKE_P) [06:46:30] [PASSED] 0x4626 (ALDERLAKE_P) [06:46:30] [PASSED] 0x4628 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46B0 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46B1 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46B2 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46B3 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46C0 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46C1 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46C2 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46C3 (ALDERLAKE_P) [06:46:30] [PASSED] 0x46D0 (ALDERLAKE_N) [06:46:30] [PASSED] 0x46D1 (ALDERLAKE_N) [06:46:30] [PASSED] 0x46D2 (ALDERLAKE_N) [06:46:30] [PASSED] 0x46D3 (ALDERLAKE_N) [06:46:30] [PASSED] 0x46D4 (ALDERLAKE_N) [06:46:30] [PASSED] 0xA721 (ALDERLAKE_P) [06:46:30] [PASSED] 0xA7A1 (ALDERLAKE_P) [06:46:30] [PASSED] 0xA7A9 (ALDERLAKE_P) [06:46:30] [PASSED] 0xA7AC (ALDERLAKE_P) [06:46:30] [PASSED] 0xA7AD (ALDERLAKE_P) [06:46:30] [PASSED] 0xA720 (ALDERLAKE_P) [06:46:30] [PASSED] 0xA7A0 (ALDERLAKE_P) [06:46:30] [PASSED] 0xA7A8 (ALDERLAKE_P) [06:46:30] [PASSED] 0xA7AA (ALDERLAKE_P) [06:46:30] [PASSED] 0xA7AB (ALDERLAKE_P) [06:46:30] [PASSED] 0xA780 (ALDERLAKE_S) [06:46:30] [PASSED] 0xA781 (ALDERLAKE_S) [06:46:30] [PASSED] 0xA782 (ALDERLAKE_S) [06:46:30] [PASSED] 0xA783 (ALDERLAKE_S) [06:46:30] [PASSED] 0xA788 (ALDERLAKE_S) [06:46:30] [PASSED] 0xA789 (ALDERLAKE_S) [06:46:30] [PASSED] 0xA78A (ALDERLAKE_S) [06:46:30] [PASSED] 0xA78B (ALDERLAKE_S) [06:46:30] [PASSED] 0x4905 (DG1) [06:46:30] [PASSED] 0x4906 (DG1) [06:46:30] [PASSED] 0x4907 (DG1) [06:46:30] [PASSED] 0x4908 (DG1) [06:46:30] [PASSED] 0x4909 (DG1) [06:46:30] [PASSED] 0x56C0 (DG2) [06:46:30] [PASSED] 0x56C2 (DG2) [06:46:30] [PASSED] 0x56C1 (DG2) [06:46:30] [PASSED] 0x7D51 (METEORLAKE) [06:46:30] [PASSED] 0x7DD1 (METEORLAKE) [06:46:30] [PASSED] 0x7D41 (METEORLAKE) [06:46:30] [PASSED] 0x7D67 (METEORLAKE) [06:46:30] [PASSED] 0xB640 (METEORLAKE) [06:46:30] [PASSED] 0x56A0 (DG2) [06:46:30] [PASSED] 0x56A1 (DG2) [06:46:30] [PASSED] 0x56A2 (DG2) [06:46:30] [PASSED] 0x56BE (DG2) [06:46:30] [PASSED] 0x56BF (DG2) [06:46:30] [PASSED] 0x5690 (DG2) [06:46:30] [PASSED] 0x5691 (DG2) [06:46:30] [PASSED] 0x5692 (DG2) [06:46:30] [PASSED] 0x56A5 (DG2) [06:46:30] [PASSED] 0x56A6 (DG2) [06:46:30] [PASSED] 0x56B0 (DG2) [06:46:30] [PASSED] 0x56B1 (DG2) [06:46:30] [PASSED] 0x56BA (DG2) [06:46:30] [PASSED] 0x56BB (DG2) [06:46:30] [PASSED] 0x56BC (DG2) [06:46:30] [PASSED] 0x56BD (DG2) [06:46:30] [PASSED] 0x5693 (DG2) [06:46:30] [PASSED] 0x5694 (DG2) [06:46:30] [PASSED] 0x5695 (DG2) [06:46:30] [PASSED] 0x56A3 (DG2) [06:46:30] [PASSED] 0x56A4 (DG2) [06:46:30] [PASSED] 0x56B2 (DG2) [06:46:30] [PASSED] 0x56B3 (DG2) [06:46:30] [PASSED] 0x5696 (DG2) [06:46:30] [PASSED] 0x5697 (DG2) [06:46:30] [PASSED] 0xB69 (PVC) [06:46:30] [PASSED] 0xB6E (PVC) [06:46:30] [PASSED] 0xBD4 (PVC) [06:46:30] [PASSED] 0xBD5 (PVC) [06:46:30] [PASSED] 0xBD6 (PVC) [06:46:30] [PASSED] 0xBD7 (PVC) [06:46:30] [PASSED] 0xBD8 (PVC) [06:46:30] [PASSED] 0xBD9 (PVC) [06:46:30] [PASSED] 0xBDA (PVC) [06:46:30] [PASSED] 0xBDB (PVC) [06:46:30] [PASSED] 0xBE0 (PVC) [06:46:30] [PASSED] 0xBE1 (PVC) [06:46:30] [PASSED] 0xBE5 (PVC) [06:46:30] [PASSED] 0x7D40 (METEORLAKE) [06:46:30] [PASSED] 0x7D45 (METEORLAKE) [06:46:30] [PASSED] 0x7D55 (METEORLAKE) [06:46:30] [PASSED] 0x7D60 (METEORLAKE) [06:46:30] [PASSED] 0x7DD5 (METEORLAKE) [06:46:30] [PASSED] 0x6420 (LUNARLAKE) [06:46:30] [PASSED] 0x64A0 (LUNARLAKE) [06:46:30] [PASSED] 0x64B0 (LUNARLAKE) [06:46:30] [PASSED] 0xE202 (BATTLEMAGE) [06:46:30] [PASSED] 0xE209 (BATTLEMAGE) [06:46:30] [PASSED] 0xE20B (BATTLEMAGE) [06:46:30] [PASSED] 0xE20C (BATTLEMAGE) [06:46:30] [PASSED] 0xE20D (BATTLEMAGE) [06:46:30] [PASSED] 0xE210 (BATTLEMAGE) [06:46:30] [PASSED] 0xE211 (BATTLEMAGE) [06:46:30] [PASSED] 0xE212 (BATTLEMAGE) [06:46:30] [PASSED] 0xE216 (BATTLEMAGE) [06:46:30] [PASSED] 0xE220 (BATTLEMAGE) [06:46:30] [PASSED] 0xE221 (BATTLEMAGE) [06:46:30] [PASSED] 0xE222 (BATTLEMAGE) [06:46:30] [PASSED] 0xE223 (BATTLEMAGE) [06:46:30] [PASSED] 0xB080 (PANTHERLAKE) [06:46:30] [PASSED] 0xB081 (PANTHERLAKE) [06:46:30] [PASSED] 0xB082 (PANTHERLAKE) [06:46:30] [PASSED] 0xB083 (PANTHERLAKE) [06:46:30] [PASSED] 0xB084 (PANTHERLAKE) [06:46:30] [PASSED] 0xB085 (PANTHERLAKE) [06:46:30] [PASSED] 0xB086 (PANTHERLAKE) [06:46:30] [PASSED] 0xB087 (PANTHERLAKE) [06:46:30] [PASSED] 0xB08F (PANTHERLAKE) [06:46:30] [PASSED] 0xB090 (PANTHERLAKE) [06:46:30] [PASSED] 0xB0A0 (PANTHERLAKE) [06:46:30] [PASSED] 0xB0B0 (PANTHERLAKE) [06:46:30] [PASSED] 0xFD80 (PANTHERLAKE) [06:46:30] [PASSED] 0xFD81 (PANTHERLAKE) [06:46:30] ============= [PASSED] check_platform_gt_count ============= [06:46:30] ===================== [PASSED] xe_pci ====================== [06:46:30] =================== xe_rtp (2 subtests) ==================== [06:46:30] =============== xe_rtp_process_to_sr_tests ================ [06:46:30] [PASSED] coalesce-same-reg [06:46:30] [PASSED] no-match-no-add [06:46:30] [PASSED] match-or [06:46:30] [PASSED] match-or-xfail [06:46:30] [PASSED] no-match-no-add-multiple-rules [06:46:30] [PASSED] two-regs-two-entries [06:46:30] [PASSED] clr-one-set-other [06:46:30] [PASSED] set-field [06:46:30] [PASSED] conflict-duplicate [06:46:30] [PASSED] conflict-not-disjoint [06:46:30] [PASSED] conflict-reg-type [06:46:30] =========== [PASSED] xe_rtp_process_to_sr_tests ============ [06:46:30] ================== xe_rtp_process_tests =================== [06:46:30] [PASSED] active1 [06:46:30] [PASSED] active2 [06:46:30] [PASSED] active-inactive [06:46:30] [PASSED] inactive-active [06:46:30] [PASSED] inactive-1st_or_active-inactive [06:46:30] [PASSED] inactive-2nd_or_active-inactive [06:46:30] [PASSED] inactive-last_or_active-inactive [06:46:30] [PASSED] inactive-no_or_active-inactive [06:46:30] ============== [PASSED] xe_rtp_process_tests =============== [06:46:30] ===================== [PASSED] xe_rtp ====================== [06:46:30] ==================== xe_wa (1 subtest) ===================== [06:46:30] ======================== xe_wa_gt ========================= [06:46:30] [PASSED] TIGERLAKE B0 [06:46:30] [PASSED] DG1 A0 [06:46:30] [PASSED] DG1 B0 [06:46:30] [PASSED] ALDERLAKE_S A0 [06:46:30] [PASSED] ALDERLAKE_S B0 stty: 'standard input': Inappropriate ioctl for device [06:46:30] [PASSED] ALDERLAKE_S C0 [06:46:30] [PASSED] ALDERLAKE_S D0 [06:46:30] [PASSED] ALDERLAKE_P A0 [06:46:30] [PASSED] ALDERLAKE_P B0 [06:46:30] [PASSED] ALDERLAKE_P C0 [06:46:30] [PASSED] ALDERLAKE_S RPLS D0 [06:46:30] [PASSED] ALDERLAKE_P RPLU E0 [06:46:30] [PASSED] DG2 G10 C0 [06:46:30] [PASSED] DG2 G11 B1 [06:46:30] [PASSED] DG2 G12 A1 [06:46:30] [PASSED] METEORLAKE 12.70(Xe_LPG) A0 13.00(Xe_LPM+) A0 [06:46:30] [PASSED] METEORLAKE 12.71(Xe_LPG) A0 13.00(Xe_LPM+) A0 [06:46:30] [PASSED] METEORLAKE 12.74(Xe_LPG+) A0 13.00(Xe_LPM+) A0 [06:46:30] [PASSED] LUNARLAKE 20.04(Xe2_LPG) A0 20.00(Xe2_LPM) A0 [06:46:30] [PASSED] LUNARLAKE 20.04(Xe2_LPG) B0 20.00(Xe2_LPM) A0 [06:46:30] [PASSED] BATTLEMAGE 20.01(Xe2_HPG) A0 13.01(Xe2_HPM) A1 [06:46:30] [PASSED] PANTHERLAKE 30.00(Xe3_LPG) A0 30.00(Xe3_LPM) A0 [06:46:30] ==================== [PASSED] xe_wa_gt ===================== [06:46:30] ====================== [PASSED] xe_wa ====================== [06:46:30] ============================================================ [06:46:30] Testing complete. Ran 306 tests: passed: 288, skipped: 18 [06:46:30] Elapsed time: 33.894s total, 4.261s configuring, 29.266s building, 0.322s running
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/tests/.kunitconfig [06:46:30] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig [06:46:32] Building KUnit Kernel ... Populating config with: $ make ARCH=um O=.kunit olddefconfig Building with: $ make all compile_commands.json scripts_gdb ARCH=um O=.kunit --jobs=48 [06:46:55] Starting KUnit Kernel (1/1)... [06:46:55] ============================================================ Running tests with: $ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt [06:46:56] ============ drm_test_pick_cmdline (2 subtests) ============ [06:46:56] [PASSED] drm_test_pick_cmdline_res_1920_1080_60 [06:46:56] =============== drm_test_pick_cmdline_named =============== [06:46:56] [PASSED] NTSC [06:46:56] [PASSED] NTSC-J [06:46:56] [PASSED] PAL [06:46:56] [PASSED] PAL-M [06:46:56] =========== [PASSED] drm_test_pick_cmdline_named =========== [06:46:56] ============== [PASSED] drm_test_pick_cmdline ============== [06:46:56] == drm_test_atomic_get_connector_for_encoder (1 subtest) === [06:46:56] [PASSED] drm_test_drm_atomic_get_connector_for_encoder [06:46:56] ==== [PASSED] drm_test_atomic_get_connector_for_encoder ==== [06:46:56] =========== drm_validate_clone_mode (2 subtests) =========== [06:46:56] ============== drm_test_check_in_clone_mode =============== [06:46:56] [PASSED] in_clone_mode [06:46:56] [PASSED] not_in_clone_mode [06:46:56] ========== [PASSED] drm_test_check_in_clone_mode =========== [06:46:56] =============== drm_test_check_valid_clones =============== [06:46:56] [PASSED] not_in_clone_mode [06:46:56] [PASSED] valid_clone [06:46:56] [PASSED] invalid_clone [06:46:56] =========== [PASSED] drm_test_check_valid_clones =========== [06:46:56] ============= [PASSED] drm_validate_clone_mode ============= [06:46:56] ============= drm_validate_modeset (1 subtest) ============= [06:46:56] [PASSED] drm_test_check_connector_changed_modeset [06:46:56] ============== [PASSED] drm_validate_modeset =============== [06:46:56] ====== drm_test_bridge_get_current_state (2 subtests) ====== [06:46:56] [PASSED] drm_test_drm_bridge_get_current_state_atomic [06:46:56] [PASSED] drm_test_drm_bridge_get_current_state_legacy [06:46:56] ======== [PASSED] drm_test_bridge_get_current_state ======== [06:46:56] ====== drm_test_bridge_helper_reset_crtc (3 subtests) ====== [06:46:56] [PASSED] drm_test_drm_bridge_helper_reset_crtc_atomic [06:46:56] [PASSED] drm_test_drm_bridge_helper_reset_crtc_atomic_disabled [06:46:56] [PASSED] drm_test_drm_bridge_helper_reset_crtc_legacy [06:46:56] ======== [PASSED] drm_test_bridge_helper_reset_crtc ======== [06:46:56] ============== drm_bridge_alloc (2 subtests) =============== [06:46:56] [PASSED] drm_test_drm_bridge_alloc_basic [06:46:56] [PASSED] drm_test_drm_bridge_alloc_get_put [06:46:56] ================ [PASSED] drm_bridge_alloc ================= [06:46:56] ================== drm_buddy (7 subtests) ================== [06:46:56] [PASSED] drm_test_buddy_alloc_limit [06:46:56] [PASSED] drm_test_buddy_alloc_optimistic [06:46:56] [PASSED] drm_test_buddy_alloc_pessimistic [06:46:56] [PASSED] drm_test_buddy_alloc_pathological [06:46:56] [PASSED] drm_test_buddy_alloc_contiguous [06:46:56] [PASSED] drm_test_buddy_alloc_clear [06:46:56] [PASSED] drm_test_buddy_alloc_range_bias [06:46:56] ==================== [PASSED] drm_buddy ==================== [06:46:56] ============= drm_cmdline_parser (40 subtests) ============= [06:46:56] [PASSED] drm_test_cmdline_force_d_only [06:46:56] [PASSED] drm_test_cmdline_force_D_only_dvi [06:46:56] [PASSED] drm_test_cmdline_force_D_only_hdmi [06:46:56] [PASSED] drm_test_cmdline_force_D_only_not_digital [06:46:56] [PASSED] drm_test_cmdline_force_e_only [06:46:56] [PASSED] drm_test_cmdline_res [06:46:56] [PASSED] drm_test_cmdline_res_vesa [06:46:56] [PASSED] drm_test_cmdline_res_vesa_rblank [06:46:56] [PASSED] drm_test_cmdline_res_rblank [06:46:56] [PASSED] drm_test_cmdline_res_bpp [06:46:56] [PASSED] drm_test_cmdline_res_refresh [06:46:56] [PASSED] drm_test_cmdline_res_bpp_refresh [06:46:56] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced [06:46:56] [PASSED] drm_test_cmdline_res_bpp_refresh_margins [06:46:56] [PASSED] drm_test_cmdline_res_bpp_refresh_force_off [06:46:56] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on [06:46:56] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_analog [06:46:56] [PASSED] drm_test_cmdline_res_bpp_refresh_force_on_digital [06:46:56] [PASSED] drm_test_cmdline_res_bpp_refresh_interlaced_margins_force_on [06:46:56] [PASSED] drm_test_cmdline_res_margins_force_on [06:46:56] [PASSED] drm_test_cmdline_res_vesa_margins [06:46:56] [PASSED] drm_test_cmdline_name [06:46:56] [PASSED] drm_test_cmdline_name_bpp [06:46:56] [PASSED] drm_test_cmdline_name_option [06:46:56] [PASSED] drm_test_cmdline_name_bpp_option [06:46:56] [PASSED] drm_test_cmdline_rotate_0 [06:46:56] [PASSED] drm_test_cmdline_rotate_90 [06:46:56] [PASSED] drm_test_cmdline_rotate_180 [06:46:56] [PASSED] drm_test_cmdline_rotate_270 [06:46:56] [PASSED] drm_test_cmdline_hmirror [06:46:56] [PASSED] drm_test_cmdline_vmirror [06:46:56] [PASSED] drm_test_cmdline_margin_options [06:46:56] [PASSED] drm_test_cmdline_multiple_options [06:46:56] [PASSED] drm_test_cmdline_bpp_extra_and_option [06:46:56] [PASSED] drm_test_cmdline_extra_and_option [06:46:56] [PASSED] drm_test_cmdline_freestanding_options [06:46:56] [PASSED] drm_test_cmdline_freestanding_force_e_and_options [06:46:56] [PASSED] drm_test_cmdline_panel_orientation [06:46:56] ================ drm_test_cmdline_invalid ================= [06:46:56] [PASSED] margin_only [06:46:56] [PASSED] interlace_only [06:46:56] [PASSED] res_missing_x [06:46:56] [PASSED] res_missing_y [06:46:56] [PASSED] res_bad_y [06:46:56] [PASSED] res_missing_y_bpp [06:46:56] [PASSED] res_bad_bpp [06:46:56] [PASSED] res_bad_refresh [06:46:56] [PASSED] res_bpp_refresh_force_on_off [06:46:56] [PASSED] res_invalid_mode [06:46:56] [PASSED] res_bpp_wrong_place_mode [06:46:56] [PASSED] name_bpp_refresh [06:46:56] [PASSED] name_refresh [06:46:56] [PASSED] name_refresh_wrong_mode [06:46:56] [PASSED] name_refresh_invalid_mode [06:46:56] [PASSED] rotate_multiple [06:46:56] [PASSED] rotate_invalid_val [06:46:56] [PASSED] rotate_truncated [06:46:56] [PASSED] invalid_option [06:46:56] [PASSED] invalid_tv_option [06:46:56] [PASSED] truncated_tv_option [06:46:56] ============ [PASSED] drm_test_cmdline_invalid ============= [06:46:56] =============== drm_test_cmdline_tv_options =============== [06:46:56] [PASSED] NTSC [06:46:56] [PASSED] NTSC_443 [06:46:56] [PASSED] NTSC_J [06:46:56] [PASSED] PAL [06:46:56] [PASSED] PAL_M [06:46:56] [PASSED] PAL_N [06:46:56] [PASSED] SECAM [06:46:56] [PASSED] MONO_525 [06:46:56] [PASSED] MONO_625 [06:46:56] =========== [PASSED] drm_test_cmdline_tv_options =========== [06:46:56] =============== [PASSED] drm_cmdline_parser ================ [06:46:56] ========== drmm_connector_hdmi_init (20 subtests) ========== [06:46:56] [PASSED] drm_test_connector_hdmi_init_valid [06:46:56] [PASSED] drm_test_connector_hdmi_init_bpc_8 [06:46:56] [PASSED] drm_test_connector_hdmi_init_bpc_10 [06:46:56] [PASSED] drm_test_connector_hdmi_init_bpc_12 [06:46:56] [PASSED] drm_test_connector_hdmi_init_bpc_invalid [06:46:56] [PASSED] drm_test_connector_hdmi_init_bpc_null [06:46:56] [PASSED] drm_test_connector_hdmi_init_formats_empty [06:46:56] [PASSED] drm_test_connector_hdmi_init_formats_no_rgb [06:46:56] === drm_test_connector_hdmi_init_formats_yuv420_allowed === [06:46:56] [PASSED] supported_formats=0x9 yuv420_allowed=1 [06:46:56] [PASSED] supported_formats=0x9 yuv420_allowed=0 [06:46:56] [PASSED] supported_formats=0x3 yuv420_allowed=1 [06:46:56] [PASSED] supported_formats=0x3 yuv420_allowed=0 [06:46:56] === [PASSED] drm_test_connector_hdmi_init_formats_yuv420_allowed === [06:46:56] [PASSED] drm_test_connector_hdmi_init_null_ddc [06:46:56] [PASSED] drm_test_connector_hdmi_init_null_product [06:46:56] [PASSED] drm_test_connector_hdmi_init_null_vendor [06:46:56] [PASSED] drm_test_connector_hdmi_init_product_length_exact [06:46:56] [PASSED] drm_test_connector_hdmi_init_product_length_too_long [06:46:56] [PASSED] drm_test_connector_hdmi_init_product_valid [06:46:56] [PASSED] drm_test_connector_hdmi_init_vendor_length_exact [06:46:56] [PASSED] drm_test_connector_hdmi_init_vendor_length_too_long [06:46:56] [PASSED] drm_test_connector_hdmi_init_vendor_valid [06:46:56] ========= drm_test_connector_hdmi_init_type_valid ========= [06:46:56] [PASSED] HDMI-A [06:46:56] [PASSED] HDMI-B [06:46:56] ===== [PASSED] drm_test_connector_hdmi_init_type_valid ===== [06:46:56] ======== drm_test_connector_hdmi_init_type_invalid ======== [06:46:56] [PASSED] Unknown [06:46:56] [PASSED] VGA [06:46:56] [PASSED] DVI-I [06:46:56] [PASSED] DVI-D [06:46:56] [PASSED] DVI-A [06:46:56] [PASSED] Composite [06:46:56] [PASSED] SVIDEO [06:46:56] [PASSED] LVDS [06:46:56] [PASSED] Component [06:46:56] [PASSED] DIN [06:46:56] [PASSED] DP [06:46:56] [PASSED] TV [06:46:56] [PASSED] eDP [06:46:56] [PASSED] Virtual [06:46:56] [PASSED] DSI [06:46:56] [PASSED] DPI [06:46:56] [PASSED] Writeback [06:46:56] [PASSED] SPI [06:46:56] [PASSED] USB [06:46:56] ==== [PASSED] drm_test_connector_hdmi_init_type_invalid ==== [06:46:56] ============ [PASSED] drmm_connector_hdmi_init ============= [06:46:56] ============= drmm_connector_init (3 subtests) ============= [06:46:56] [PASSED] drm_test_drmm_connector_init [06:46:56] [PASSED] drm_test_drmm_connector_init_null_ddc [06:46:56] ========= drm_test_drmm_connector_init_type_valid ========= [06:46:56] [PASSED] Unknown [06:46:56] [PASSED] VGA [06:46:56] [PASSED] DVI-I [06:46:56] [PASSED] DVI-D [06:46:56] [PASSED] DVI-A [06:46:56] [PASSED] Composite [06:46:56] [PASSED] SVIDEO [06:46:56] [PASSED] LVDS [06:46:56] [PASSED] Component [06:46:56] [PASSED] DIN [06:46:56] [PASSED] DP [06:46:56] [PASSED] HDMI-A [06:46:56] [PASSED] HDMI-B [06:46:56] [PASSED] TV [06:46:56] [PASSED] eDP [06:46:56] [PASSED] Virtual [06:46:56] [PASSED] DSI [06:46:56] [PASSED] DPI [06:46:56] [PASSED] Writeback [06:46:56] [PASSED] SPI [06:46:56] [PASSED] USB [06:46:56] ===== [PASSED] drm_test_drmm_connector_init_type_valid ===== [06:46:56] =============== [PASSED] drmm_connector_init =============== [06:46:56] ========= drm_connector_dynamic_init (6 subtests) ========== [06:46:56] [PASSED] drm_test_drm_connector_dynamic_init [06:46:56] [PASSED] drm_test_drm_connector_dynamic_init_null_ddc [06:46:56] [PASSED] drm_test_drm_connector_dynamic_init_not_added [06:46:56] [PASSED] drm_test_drm_connector_dynamic_init_properties [06:46:56] ===== drm_test_drm_connector_dynamic_init_type_valid ====== [06:46:56] [PASSED] Unknown [06:46:56] [PASSED] VGA [06:46:56] [PASSED] DVI-I [06:46:56] [PASSED] DVI-D [06:46:56] [PASSED] DVI-A [06:46:56] [PASSED] Composite [06:46:56] [PASSED] SVIDEO [06:46:56] [PASSED] LVDS [06:46:56] [PASSED] Component [06:46:56] [PASSED] DIN [06:46:56] [PASSED] DP [06:46:56] [PASSED] HDMI-A [06:46:56] [PASSED] HDMI-B [06:46:56] [PASSED] TV [06:46:56] [PASSED] eDP [06:46:56] [PASSED] Virtual [06:46:56] [PASSED] DSI [06:46:56] [PASSED] DPI [06:46:56] [PASSED] Writeback [06:46:56] [PASSED] SPI [06:46:56] [PASSED] USB [06:46:56] = [PASSED] drm_test_drm_connector_dynamic_init_type_valid == [06:46:56] ======== drm_test_drm_connector_dynamic_init_name ========= [06:46:56] [PASSED] Unknown [06:46:56] [PASSED] VGA [06:46:56] [PASSED] DVI-I [06:46:56] [PASSED] DVI-D [06:46:56] [PASSED] DVI-A [06:46:56] [PASSED] Composite [06:46:56] [PASSED] SVIDEO [06:46:56] [PASSED] LVDS [06:46:56] [PASSED] Component [06:46:56] [PASSED] DIN [06:46:56] [PASSED] DP [06:46:56] [PASSED] HDMI-A [06:46:56] [PASSED] HDMI-B [06:46:56] [PASSED] TV [06:46:56] [PASSED] eDP [06:46:56] [PASSED] Virtual [06:46:56] [PASSED] DSI [06:46:56] [PASSED] DPI [06:46:56] [PASSED] Writeback [06:46:56] [PASSED] SPI [06:46:56] [PASSED] USB [06:46:56] ==== [PASSED] drm_test_drm_connector_dynamic_init_name ===== [06:46:56] =========== [PASSED] drm_connector_dynamic_init ============ [06:46:56] ==== drm_connector_dynamic_register_early (4 subtests) ===== [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_early_on_list [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_early_defer [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_early_no_init [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_early_no_mode_object [06:46:56] ====== [PASSED] drm_connector_dynamic_register_early ======= [06:46:56] ======= drm_connector_dynamic_register (7 subtests) ======== [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_on_list [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_no_defer [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_no_init [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_mode_object [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_sysfs [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_sysfs_name [06:46:56] [PASSED] drm_test_drm_connector_dynamic_register_debugfs [06:46:56] ========= [PASSED] drm_connector_dynamic_register ========== [06:46:56] = drm_connector_attach_broadcast_rgb_property (2 subtests) = [06:46:56] [PASSED] drm_test_drm_connector_attach_broadcast_rgb_property [06:46:56] [PASSED] drm_test_drm_connector_attach_broadcast_rgb_property_hdmi_connector [06:46:56] === [PASSED] drm_connector_attach_broadcast_rgb_property === [06:46:56] ========== drm_get_tv_mode_from_name (2 subtests) ========== [06:46:56] ========== drm_test_get_tv_mode_from_name_valid =========== [06:46:56] [PASSED] NTSC [06:46:56] [PASSED] NTSC-443 [06:46:56] [PASSED] NTSC-J [06:46:56] [PASSED] PAL [06:46:56] [PASSED] PAL-M [06:46:56] [PASSED] PAL-N [06:46:56] [PASSED] SECAM [06:46:56] [PASSED] Mono [06:46:56] ====== [PASSED] drm_test_get_tv_mode_from_name_valid ======= [06:46:56] [PASSED] drm_test_get_tv_mode_from_name_truncated [06:46:56] ============ [PASSED] drm_get_tv_mode_from_name ============ [06:46:56] = drm_test_connector_hdmi_compute_mode_clock (12 subtests) = [06:46:56] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb [06:46:56] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_10bpc [06:46:56] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_10bpc_vic_1 [06:46:56] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_12bpc [06:46:56] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_12bpc_vic_1 [06:46:56] [PASSED] drm_test_drm_hdmi_compute_mode_clock_rgb_double [06:46:56] = drm_test_connector_hdmi_compute_mode_clock_yuv420_valid = [06:46:56] [PASSED] VIC 96 [06:46:56] [PASSED] VIC 97 [06:46:56] [PASSED] VIC 101 [06:46:56] [PASSED] VIC 102 [06:46:56] [PASSED] VIC 106 [06:46:56] [PASSED] VIC 107 [06:46:56] === [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_valid === [06:46:56] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_10_bpc [06:46:56] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv420_12_bpc [06:46:56] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_8_bpc [06:46:56] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_10_bpc [06:46:56] [PASSED] drm_test_connector_hdmi_compute_mode_clock_yuv422_12_bpc [06:46:56] === [PASSED] drm_test_connector_hdmi_compute_mode_clock ==== [06:46:56] == drm_hdmi_connector_get_broadcast_rgb_name (2 subtests) == [06:46:56] === drm_test_drm_hdmi_connector_get_broadcast_rgb_name ==== [06:46:56] [PASSED] Automatic [06:46:56] [PASSED] Full [06:46:56] [PASSED] Limited 16:235 [06:46:56] === [PASSED] drm_test_drm_hdmi_connector_get_broadcast_rgb_name === [06:46:56] [PASSED] drm_test_drm_hdmi_connector_get_broadcast_rgb_name_invalid [06:46:56] ==== [PASSED] drm_hdmi_connector_get_broadcast_rgb_name ==== [06:46:56] == drm_hdmi_connector_get_output_format_name (2 subtests) == [06:46:56] === drm_test_drm_hdmi_connector_get_output_format_name ==== [06:46:56] [PASSED] RGB [06:46:56] [PASSED] YUV 4:2:0 [06:46:56] [PASSED] YUV 4:2:2 [06:46:56] [PASSED] YUV 4:4:4 [06:46:56] === [PASSED] drm_test_drm_hdmi_connector_get_output_format_name === [06:46:56] [PASSED] drm_test_drm_hdmi_connector_get_output_format_name_invalid [06:46:56] ==== [PASSED] drm_hdmi_connector_get_output_format_name ==== [06:46:56] ============= drm_damage_helper (21 subtests) ============== [06:46:56] [PASSED] drm_test_damage_iter_no_damage [06:46:56] [PASSED] drm_test_damage_iter_no_damage_fractional_src [06:46:56] [PASSED] drm_test_damage_iter_no_damage_src_moved [06:46:56] [PASSED] drm_test_damage_iter_no_damage_fractional_src_moved [06:46:56] [PASSED] drm_test_damage_iter_no_damage_not_visible [06:46:56] [PASSED] drm_test_damage_iter_no_damage_no_crtc [06:46:56] [PASSED] drm_test_damage_iter_no_damage_no_fb [06:46:56] [PASSED] drm_test_damage_iter_simple_damage [06:46:56] [PASSED] drm_test_damage_iter_single_damage [06:46:56] [PASSED] drm_test_damage_iter_single_damage_intersect_src [06:46:56] [PASSED] drm_test_damage_iter_single_damage_outside_src [06:46:56] [PASSED] drm_test_damage_iter_single_damage_fractional_src [06:46:56] [PASSED] drm_test_damage_iter_single_damage_intersect_fractional_src [06:46:56] [PASSED] drm_test_damage_iter_single_damage_outside_fractional_src [06:46:56] [PASSED] drm_test_damage_iter_single_damage_src_moved [06:46:56] [PASSED] drm_test_damage_iter_single_damage_fractional_src_moved [06:46:56] [PASSED] drm_test_damage_iter_damage [06:46:56] [PASSED] drm_test_damage_iter_damage_one_intersect [06:46:56] [PASSED] drm_test_damage_iter_damage_one_outside [06:46:56] [PASSED] drm_test_damage_iter_damage_src_moved [06:46:56] [PASSED] drm_test_damage_iter_damage_not_visible [06:46:56] ================ [PASSED] drm_damage_helper ================ [06:46:56] ============== drm_dp_mst_helper (3 subtests) ============== [06:46:56] ============== drm_test_dp_mst_calc_pbn_mode ============== [06:46:56] [PASSED] Clock 154000 BPP 30 DSC disabled [06:46:56] [PASSED] Clock 234000 BPP 30 DSC disabled [06:46:56] [PASSED] Clock 297000 BPP 24 DSC disabled [06:46:56] [PASSED] Clock 332880 BPP 24 DSC enabled [06:46:56] [PASSED] Clock 324540 BPP 24 DSC enabled [06:46:56] ========== [PASSED] drm_test_dp_mst_calc_pbn_mode ========== [06:46:56] ============== drm_test_dp_mst_calc_pbn_div =============== [06:46:56] [PASSED] Link rate 2000000 lane count 4 [06:46:56] [PASSED] Link rate 2000000 lane count 2 [06:46:56] [PASSED] Link rate 2000000 lane count 1 [06:46:56] [PASSED] Link rate 1350000 lane count 4 [06:46:56] [PASSED] Link rate 1350000 lane count 2 [06:46:56] [PASSED] Link rate 1350000 lane count 1 [06:46:56] [PASSED] Link rate 1000000 lane count 4 [06:46:56] [PASSED] Link rate 1000000 lane count 2 [06:46:56] [PASSED] Link rate 1000000 lane count 1 [06:46:56] [PASSED] Link rate 810000 lane count 4 [06:46:56] [PASSED] Link rate 810000 lane count 2 [06:46:56] [PASSED] Link rate 810000 lane count 1 [06:46:56] [PASSED] Link rate 540000 lane count 4 [06:46:56] [PASSED] Link rate 540000 lane count 2 [06:46:56] [PASSED] Link rate 540000 lane count 1 [06:46:56] [PASSED] Link rate 270000 lane count 4 [06:46:56] [PASSED] Link rate 270000 lane count 2 [06:46:56] [PASSED] Link rate 270000 lane count 1 [06:46:56] [PASSED] Link rate 162000 lane count 4 [06:46:56] [PASSED] Link rate 162000 lane count 2 [06:46:56] [PASSED] Link rate 162000 lane count 1 [06:46:56] ========== [PASSED] drm_test_dp_mst_calc_pbn_div =========== [06:46:56] ========= drm_test_dp_mst_sideband_msg_req_decode ========= [06:46:56] [PASSED] DP_ENUM_PATH_RESOURCES with port number [06:46:56] [PASSED] DP_POWER_UP_PHY with port number [06:46:56] [PASSED] DP_POWER_DOWN_PHY with port number [06:46:56] [PASSED] DP_ALLOCATE_PAYLOAD with SDP stream sinks [06:46:56] [PASSED] DP_ALLOCATE_PAYLOAD with port number [06:46:56] [PASSED] DP_ALLOCATE_PAYLOAD with VCPI [06:46:56] [PASSED] DP_ALLOCATE_PAYLOAD with PBN [06:46:56] [PASSED] DP_QUERY_PAYLOAD with port number [06:46:56] [PASSED] DP_QUERY_PAYLOAD with VCPI [06:46:56] [PASSED] DP_REMOTE_DPCD_READ with port number [06:46:56] [PASSED] DP_REMOTE_DPCD_READ with DPCD address [06:46:56] [PASSED] DP_REMOTE_DPCD_READ with max number of bytes [06:46:56] [PASSED] DP_REMOTE_DPCD_WRITE with port number [06:46:56] [PASSED] DP_REMOTE_DPCD_WRITE with DPCD address [06:46:56] [PASSED] DP_REMOTE_DPCD_WRITE with data array [06:46:56] [PASSED] DP_REMOTE_I2C_READ with port number [06:46:56] [PASSED] DP_REMOTE_I2C_READ with I2C device ID [06:46:56] [PASSED] DP_REMOTE_I2C_READ with transactions array [06:46:56] [PASSED] DP_REMOTE_I2C_WRITE with port number [06:46:56] [PASSED] DP_REMOTE_I2C_WRITE with I2C device ID [06:46:56] [PASSED] DP_REMOTE_I2C_WRITE with data array [06:46:56] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream ID [06:46:56] [PASSED] DP_QUERY_STREAM_ENC_STATUS with client ID [06:46:56] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream event [06:46:56] [PASSED] DP_QUERY_STREAM_ENC_STATUS with valid stream event [06:46:56] [PASSED] DP_QUERY_STREAM_ENC_STATUS with stream behavior [06:46:56] [PASSED] DP_QUERY_STREAM_ENC_STATUS with a valid stream behavior [06:46:56] ===== [PASSED] drm_test_dp_mst_sideband_msg_req_decode ===== [06:46:56] ================ [PASSED] drm_dp_mst_helper ================ [06:46:56] ================== drm_exec (7 subtests) =================== [06:46:56] [PASSED] sanitycheck [06:46:56] [PASSED] test_lock [06:46:56] [PASSED] test_lock_unlock [06:46:56] [PASSED] test_duplicates [06:46:56] [PASSED] test_prepare [06:46:56] [PASSED] test_prepare_array [06:46:56] [PASSED] test_multiple_loops [06:46:56] ==================== [PASSED] drm_exec ===================== [06:46:56] =========== drm_format_helper_test (17 subtests) =========== [06:46:56] ============== drm_test_fb_xrgb8888_to_gray8 ============== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ========== [PASSED] drm_test_fb_xrgb8888_to_gray8 ========== [06:46:56] ============= drm_test_fb_xrgb8888_to_rgb332 ============== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb332 ========== [06:46:56] ============= drm_test_fb_xrgb8888_to_rgb565 ============== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb565 ========== [06:46:56] ============ drm_test_fb_xrgb8888_to_xrgb1555 ============= [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ======== [PASSED] drm_test_fb_xrgb8888_to_xrgb1555 ========= [06:46:56] ============ drm_test_fb_xrgb8888_to_argb1555 ============= [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ======== [PASSED] drm_test_fb_xrgb8888_to_argb1555 ========= [06:46:56] ============ drm_test_fb_xrgb8888_to_rgba5551 ============= [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ======== [PASSED] drm_test_fb_xrgb8888_to_rgba5551 ========= [06:46:56] ============= drm_test_fb_xrgb8888_to_rgb888 ============== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ========= [PASSED] drm_test_fb_xrgb8888_to_rgb888 ========== [06:46:56] ============= drm_test_fb_xrgb8888_to_bgr888 ============== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ========= [PASSED] drm_test_fb_xrgb8888_to_bgr888 ========== [06:46:56] ============ drm_test_fb_xrgb8888_to_argb8888 ============= [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ======== [PASSED] drm_test_fb_xrgb8888_to_argb8888 ========= [06:46:56] =========== drm_test_fb_xrgb8888_to_xrgb2101010 =========== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ======= [PASSED] drm_test_fb_xrgb8888_to_xrgb2101010 ======= [06:46:56] =========== drm_test_fb_xrgb8888_to_argb2101010 =========== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ======= [PASSED] drm_test_fb_xrgb8888_to_argb2101010 ======= [06:46:56] ============== drm_test_fb_xrgb8888_to_mono =============== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ========== [PASSED] drm_test_fb_xrgb8888_to_mono =========== [06:46:56] ==================== drm_test_fb_swab ===================== [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ================ [PASSED] drm_test_fb_swab ================= [06:46:56] ============ drm_test_fb_xrgb8888_to_xbgr8888 ============= [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ======== [PASSED] drm_test_fb_xrgb8888_to_xbgr8888 ========= [06:46:56] ============ drm_test_fb_xrgb8888_to_abgr8888 ============= [06:46:56] [PASSED] single_pixel_source_buffer [06:46:56] [PASSED] single_pixel_clip_rectangle [06:46:56] [PASSED] well_known_colors [06:46:56] [PASSED] destination_pitch [06:46:56] ======== [PASSED] drm_test_fb_xrgb8888_to_abgr8888 ========= [06:46:56] ================= drm_test_fb_clip_offset ================= [06:46:56] [PASSED] pass through [06:46:56] [PASSED] horizontal offset [06:46:56] [PASSED] vertical offset [06:46:56] [PASSED] horizontal and vertical offset [06:46:56] [PASSED] horizontal offset (custom pitch) [06:46:56] [PASSED] vertical offset (custom pitch) [06:46:56] [PASSED] horizontal and vertical offset (custom pitch) [06:46:56] ============= [PASSED] drm_test_fb_clip_offset ============= [06:46:56] =================== drm_test_fb_memcpy ==================== [06:46:56] [PASSED] single_pixel_source_buffer: XR24 little-endian (0x34325258) [06:46:56] [PASSED] single_pixel_source_buffer: XRA8 little-endian (0x38415258) [06:46:56] [PASSED] single_pixel_source_buffer: YU24 little-endian (0x34325559) [06:46:56] [PASSED] single_pixel_clip_rectangle: XB24 little-endian (0x34324258) [06:46:56] [PASSED] single_pixel_clip_rectangle: XRA8 little-endian (0x38415258) [06:46:56] [PASSED] single_pixel_clip_rectangle: YU24 little-endian (0x34325559) [06:46:56] [PASSED] well_known_colors: XB24 little-endian (0x34324258) [06:46:56] [PASSED] well_known_colors: XRA8 little-endian (0x38415258) [06:46:56] [PASSED] well_known_colors: YU24 little-endian (0x34325559) [06:46:56] [PASSED] destination_pitch: XB24 little-endian (0x34324258) [06:46:56] [PASSED] destination_pitch: XRA8 little-endian (0x38415258) [06:46:56] [PASSED] destination_pitch: YU24 little-endian (0x34325559) [06:46:56] =============== [PASSED] drm_test_fb_memcpy ================ [06:46:56] ============= [PASSED] drm_format_helper_test ============== [06:46:56] ================= drm_format (18 subtests) ================= [06:46:56] [PASSED] drm_test_format_block_width_invalid [06:46:56] [PASSED] drm_test_format_block_width_one_plane [06:46:56] [PASSED] drm_test_format_block_width_two_plane [06:46:56] [PASSED] drm_test_format_block_width_three_plane [06:46:56] [PASSED] drm_test_format_block_width_tiled [06:46:56] [PASSED] drm_test_format_block_height_invalid [06:46:56] [PASSED] drm_test_format_block_height_one_plane [06:46:56] [PASSED] drm_test_format_block_height_two_plane [06:46:56] [PASSED] drm_test_format_block_height_three_plane [06:46:56] [PASSED] drm_test_format_block_height_tiled [06:46:56] [PASSED] drm_test_format_min_pitch_invalid [06:46:56] [PASSED] drm_test_format_min_pitch_one_plane_8bpp [06:46:56] [PASSED] drm_test_format_min_pitch_one_plane_16bpp [06:46:56] [PASSED] drm_test_format_min_pitch_one_plane_24bpp [06:46:56] [PASSED] drm_test_format_min_pitch_one_plane_32bpp [06:46:56] [PASSED] drm_test_format_min_pitch_two_plane [06:46:56] [PASSED] drm_test_format_min_pitch_three_plane_8bpp [06:46:56] [PASSED] drm_test_format_min_pitch_tiled [06:46:56] =================== [PASSED] drm_format ==================== [06:46:56] ============== drm_framebuffer (10 subtests) =============== [06:46:56] ========== drm_test_framebuffer_check_src_coords ========== [06:46:56] [PASSED] Success: source fits into fb [06:46:56] [PASSED] Fail: overflowing fb with x-axis coordinate [06:46:56] [PASSED] Fail: overflowing fb with y-axis coordinate [06:46:56] [PASSED] Fail: overflowing fb with source width [06:46:56] [PASSED] Fail: overflowing fb with source height [06:46:56] ====== [PASSED] drm_test_framebuffer_check_src_coords ====== [06:46:56] [PASSED] drm_test_framebuffer_cleanup [06:46:56] =============== drm_test_framebuffer_create =============== [06:46:56] [PASSED] ABGR8888 normal sizes [06:46:56] [PASSED] ABGR8888 max sizes [06:46:56] [PASSED] ABGR8888 pitch greater than min required [06:46:56] [PASSED] ABGR8888 pitch less than min required [06:46:56] [PASSED] ABGR8888 Invalid width [06:46:56] [PASSED] ABGR8888 Invalid buffer handle [06:46:56] [PASSED] No pixel format [06:46:56] [PASSED] ABGR8888 Width 0 [06:46:56] [PASSED] ABGR8888 Height 0 [06:46:56] [PASSED] ABGR8888 Out of bound height * pitch combination [06:46:56] [PASSED] ABGR8888 Large buffer offset [06:46:56] [PASSED] ABGR8888 Buffer offset for inexistent plane [06:46:56] [PASSED] ABGR8888 Invalid flag [06:46:56] [PASSED] ABGR8888 Set DRM_MODE_FB_MODIFIERS without modifiers [06:46:56] [PASSED] ABGR8888 Valid buffer modifier [06:46:56] [PASSED] ABGR8888 Invalid buffer modifier(DRM_FORMAT_MOD_SAMSUNG_64_32_TILE) [06:46:56] [PASSED] ABGR8888 Extra pitches without DRM_MODE_FB_MODIFIERS [06:46:56] [PASSED] ABGR8888 Extra pitches with DRM_MODE_FB_MODIFIERS [06:46:56] [PASSED] NV12 Normal sizes [06:46:56] [PASSED] NV12 Max sizes [06:46:56] [PASSED] NV12 Invalid pitch [06:46:56] [PASSED] NV12 Invalid modifier/missing DRM_MODE_FB_MODIFIERS flag [06:46:56] [PASSED] NV12 different modifier per-plane [06:46:56] [PASSED] NV12 with DRM_FORMAT_MOD_SAMSUNG_64_32_TILE [06:46:56] [PASSED] NV12 Valid modifiers without DRM_MODE_FB_MODIFIERS [06:46:56] [PASSED] NV12 Modifier for inexistent plane [06:46:56] [PASSED] NV12 Handle for inexistent plane [06:46:56] [PASSED] NV12 Handle for inexistent plane without DRM_MODE_FB_MODIFIERS [06:46:56] [PASSED] YVU420 DRM_MODE_FB_MODIFIERS set without modifier [06:46:56] [PASSED] YVU420 Normal sizes [06:46:56] [PASSED] YVU420 Max sizes [06:46:56] [PASSED] YVU420 Invalid pitch [06:46:56] [PASSED] YVU420 Different pitches [06:46:56] [PASSED] YVU420 Different buffer offsets/pitches [06:46:56] [PASSED] YVU420 Modifier set just for plane 0, without DRM_MODE_FB_MODIFIERS [06:46:56] [PASSED] YVU420 Modifier set just for planes 0, 1, without DRM_MODE_FB_MODIFIERS [06:46:56] [PASSED] YVU420 Modifier set just for plane 0, 1, with DRM_MODE_FB_MODIFIERS [06:46:56] [PASSED] YVU420 Valid modifier [06:46:56] [PASSED] YVU420 Different modifiers per plane [06:46:56] [PASSED] YVU420 Modifier for inexistent plane [06:46:56] [PASSED] YUV420_10BIT Invalid modifier(DRM_FORMAT_MOD_LINEAR) [06:46:56] [PASSED] X0L2 Normal sizes [06:46:56] [PASSED] X0L2 Max sizes [06:46:56] [PASSED] X0L2 Invalid pitch [06:46:56] [PASSED] X0L2 Pitch greater than minimum required [06:46:56] [PASSED] X0L2 Handle for inexistent plane [06:46:56] [PASSED] X0L2 Offset for inexistent plane, without DRM_MODE_FB_MODIFIERS set [06:46:56] [PASSED] X0L2 Modifier without DRM_MODE_FB_MODIFIERS set [06:46:56] [PASSED] X0L2 Valid modifier [06:46:56] [PASSED] X0L2 Modifier for inexistent plane [06:46:56] =========== [PASSED] drm_test_framebuffer_create =========== [06:46:56] [PASSED] drm_test_framebuffer_free [06:46:56] [PASSED] drm_test_framebuffer_init [06:46:56] [PASSED] drm_test_framebuffer_init_bad_format [06:46:56] [PASSED] drm_test_framebuffer_init_dev_mismatch [06:46:56] [PASSED] drm_test_framebuffer_lookup [06:46:56] [PASSED] drm_test_framebuffer_lookup_inexistent [06:46:56] [PASSED] drm_test_framebuffer_modifiers_not_supported [06:46:56] ================= [PASSED] drm_framebuffer ================= [06:46:56] ================ drm_gem_shmem (8 subtests) ================ [06:46:56] [PASSED] drm_gem_shmem_test_obj_create [06:46:56] [PASSED] drm_gem_shmem_test_obj_create_private [06:46:56] [PASSED] drm_gem_shmem_test_pin_pages [06:46:56] [PASSED] drm_gem_shmem_test_vmap [06:46:56] [PASSED] drm_gem_shmem_test_get_pages_sgt [06:46:56] [PASSED] drm_gem_shmem_test_get_sg_table [06:46:56] [PASSED] drm_gem_shmem_test_madvise [06:46:56] [PASSED] drm_gem_shmem_test_purge [06:46:56] ================== [PASSED] drm_gem_shmem ================== [06:46:56] === drm_atomic_helper_connector_hdmi_check (27 subtests) === [06:46:56] [PASSED] drm_test_check_broadcast_rgb_auto_cea_mode [06:46:56] [PASSED] drm_test_check_broadcast_rgb_auto_cea_mode_vic_1 [06:46:56] [PASSED] drm_test_check_broadcast_rgb_full_cea_mode [06:46:56] [PASSED] drm_test_check_broadcast_rgb_full_cea_mode_vic_1 [06:46:56] [PASSED] drm_test_check_broadcast_rgb_limited_cea_mode [06:46:56] [PASSED] drm_test_check_broadcast_rgb_limited_cea_mode_vic_1 [06:46:56] ====== drm_test_check_broadcast_rgb_cea_mode_yuv420 ======= [06:46:56] [PASSED] Automatic [06:46:56] [PASSED] Full [06:46:56] [PASSED] Limited 16:235 [06:46:56] == [PASSED] drm_test_check_broadcast_rgb_cea_mode_yuv420 === [06:46:56] [PASSED] drm_test_check_broadcast_rgb_crtc_mode_changed [06:46:56] [PASSED] drm_test_check_broadcast_rgb_crtc_mode_not_changed [06:46:56] [PASSED] drm_test_check_disable_connector [06:46:56] [PASSED] drm_test_check_hdmi_funcs_reject_rate [06:46:56] [PASSED] drm_test_check_max_tmds_rate_bpc_fallback_rgb [06:46:56] [PASSED] drm_test_check_max_tmds_rate_bpc_fallback_yuv420 [06:46:56] [PASSED] drm_test_check_max_tmds_rate_bpc_fallback_ignore_yuv422 [06:46:56] [PASSED] drm_test_check_max_tmds_rate_bpc_fallback_ignore_yuv420 [06:46:56] [PASSED] drm_test_check_driver_unsupported_fallback_yuv420 [06:46:56] [PASSED] drm_test_check_output_bpc_crtc_mode_changed [06:46:56] [PASSED] drm_test_check_output_bpc_crtc_mode_not_changed [06:46:56] [PASSED] drm_test_check_output_bpc_dvi [06:46:56] [PASSED] drm_test_check_output_bpc_format_vic_1 [06:46:56] [PASSED] drm_test_check_output_bpc_format_display_8bpc_only [06:46:56] [PASSED] drm_test_check_output_bpc_format_display_rgb_only [06:46:56] [PASSED] drm_test_check_output_bpc_format_driver_8bpc_only [06:46:56] [PASSED] drm_test_check_output_bpc_format_driver_rgb_only [06:46:56] [PASSED] drm_test_check_tmds_char_rate_rgb_8bpc [06:46:56] [PASSED] drm_test_check_tmds_char_rate_rgb_10bpc [06:46:56] [PASSED] drm_test_check_tmds_char_rate_rgb_12bpc [06:46:56] ===== [PASSED] drm_atomic_helper_connector_hdmi_check ====== [06:46:56] === drm_atomic_helper_connector_hdmi_reset (6 subtests) ==== [06:46:56] [PASSED] drm_test_check_broadcast_rgb_value [06:46:56] [PASSED] drm_test_check_bpc_8_value [06:46:56] [PASSED] drm_test_check_bpc_10_value [06:46:56] [PASSED] drm_test_check_bpc_12_value [06:46:56] [PASSED] drm_test_check_format_value [06:46:56] [PASSED] drm_test_check_tmds_char_value [06:46:56] ===== [PASSED] drm_atomic_helper_connector_hdmi_reset ====== [06:46:56] = drm_atomic_helper_connector_hdmi_mode_valid (4 subtests) = [06:46:56] [PASSED] drm_test_check_mode_valid [06:46:56] [PASSED] drm_test_check_mode_valid_reject [06:46:56] [PASSED] drm_test_check_mode_valid_reject_rate [06:46:56] [PASSED] drm_test_check_mode_valid_reject_max_clock [06:46:56] === [PASSED] drm_atomic_helper_connector_hdmi_mode_valid === [06:46:56] ================= drm_managed (2 subtests) ================= [06:46:56] [PASSED] drm_test_managed_release_action [06:46:56] [PASSED] drm_test_managed_run_action [06:46:56] =================== [PASSED] drm_managed =================== [06:46:56] =================== drm_mm (6 subtests) ==================== [06:46:56] [PASSED] drm_test_mm_init [06:46:56] [PASSED] drm_test_mm_debug [06:46:56] [PASSED] drm_test_mm_align32 [06:46:56] [PASSED] drm_test_mm_align64 [06:46:56] [PASSED] drm_test_mm_lowest [06:46:56] [PASSED] drm_test_mm_highest [06:46:56] ===================== [PASSED] drm_mm ====================== [06:46:56] ============= drm_modes_analog_tv (5 subtests) ============= [06:46:56] [PASSED] drm_test_modes_analog_tv_mono_576i [06:46:56] [PASSED] drm_test_modes_analog_tv_ntsc_480i [06:46:56] [PASSED] drm_test_modes_analog_tv_ntsc_480i_inlined [06:46:56] [PASSED] drm_test_modes_analog_tv_pal_576i [06:46:56] [PASSED] drm_test_modes_analog_tv_pal_576i_inlined [06:46:56] =============== [PASSED] drm_modes_analog_tv =============== [06:46:56] ============== drm_plane_helper (2 subtests) =============== [06:46:56] =============== drm_test_check_plane_state ================ [06:46:56] [PASSED] clipping_simple [06:46:56] [PASSED] clipping_rotate_reflect [06:46:56] [PASSED] positioning_simple [06:46:56] [PASSED] upscaling [06:46:56] [PASSED] downscaling [06:46:56] [PASSED] rounding1 [06:46:56] [PASSED] rounding2 [06:46:56] [PASSED] rounding3 [06:46:56] [PASSED] rounding4 [06:46:56] =========== [PASSED] drm_test_check_plane_state ============ [06:46:56] =========== drm_test_check_invalid_plane_state ============ [06:46:56] [PASSED] positioning_invalid [06:46:56] [PASSED] upscaling_invalid [06:46:56] [PASSED] downscaling_invalid [06:46:56] ======= [PASSED] drm_test_check_invalid_plane_state ======== [06:46:56] ================ [PASSED] drm_plane_helper ================= [06:46:56] ====== drm_connector_helper_tv_get_modes (1 subtest) ======= [06:46:56] ====== drm_test_connector_helper_tv_get_modes_check ======= [06:46:56] [PASSED] None [06:46:56] [PASSED] PAL [06:46:56] [PASSED] NTSC [06:46:56] [PASSED] Both, NTSC Default [06:46:56] [PASSED] Both, PAL Default [06:46:56] [PASSED] Both, NTSC Default, with PAL on command-line [06:46:56] [PASSED] Both, PAL Default, with NTSC on command-line [06:46:56] == [PASSED] drm_test_connector_helper_tv_get_modes_check === [06:46:56] ======== [PASSED] drm_connector_helper_tv_get_modes ======== [06:46:56] ================== drm_rect (9 subtests) =================== [06:46:56] [PASSED] drm_test_rect_clip_scaled_div_by_zero [06:46:56] [PASSED] drm_test_rect_clip_scaled_not_clipped [06:46:56] [PASSED] drm_test_rect_clip_scaled_clipped [06:46:56] [PASSED] drm_test_rect_clip_scaled_signed_vs_unsigned [06:46:56] ================= drm_test_rect_intersect ================= [06:46:56] [PASSED] top-left x bottom-right: 2x2+1+1 x 2x2+0+0 [06:46:56] [PASSED] top-right x bottom-left: 2x2+0+0 x 2x2+1-1 [06:46:56] [PASSED] bottom-left x top-right: 2x2+1-1 x 2x2+0+0 [06:46:56] [PASSED] bottom-right x top-left: 2x2+0+0 x 2x2+1+1 [06:46:56] [PASSED] right x left: 2x1+0+0 x 3x1+1+0 [06:46:56] [PASSED] left x right: 3x1+1+0 x 2x1+0+0 [06:46:56] [PASSED] up x bottom: 1x2+0+0 x 1x3+0-1 [06:46:56] [PASSED] bottom x up: 1x3+0-1 x 1x2+0+0 [06:46:56] [PASSED] touching corner: 1x1+0+0 x 2x2+1+1 [06:46:56] [PASSED] touching side: 1x1+0+0 x 1x1+1+0 [06:46:56] [PASSED] equal rects: 2x2+0+0 x 2x2+0+0 [06:46:56] [PASSED] inside another: 2x2+0+0 x 1x1+1+1 [06:46:56] [PASSED] far away: 1x1+0+0 x 1x1+3+6 [06:46:56] [PASSED] points intersecting: 0x0+5+10 x 0x0+5+10 [06:46:56] [PASSED] points not intersecting: 0x0+0+0 x 0x0+5+10 [06:46:56] ============= [PASSED] drm_test_rect_intersect ============= [06:46:56] ================ drm_test_rect_calc_hscale ================ [06:46:56] [PASSED] normal use [06:46:56] [PASSED] out of max range [06:46:56] [PASSED] out of min range [06:46:56] [PASSED] zero dst [06:46:56] [PASSED] negative src [06:46:56] [PASSED] negative dst [06:46:56] ============ [PASSED] drm_test_rect_calc_hscale ============ [06:46:56] ================ drm_test_rect_calc_vscale ================ [06:46:56] [PASSED] normal use [06:46:56] [PASSED] out of max range [06:46:56] [PASSED] out of min range [06:46:56] [PASSED] zero dst [06:46:56] [PASSED] negative src stty: 'standard input': Inappropriate ioctl for device [06:46:56] [PASSED] negative dst [06:46:56] ============ [PASSED] drm_test_rect_calc_vscale ============ [06:46:56] ================== drm_test_rect_rotate =================== [06:46:56] [PASSED] reflect-x [06:46:56] [PASSED] reflect-y [06:46:56] [PASSED] rotate-0 [06:46:56] [PASSED] rotate-90 [06:46:56] [PASSED] rotate-180 [06:46:56] [PASSED] rotate-270 [06:46:56] ============== [PASSED] drm_test_rect_rotate =============== [06:46:56] ================ drm_test_rect_rotate_inv ================= [06:46:56] [PASSED] reflect-x [06:46:56] [PASSED] reflect-y [06:46:56] [PASSED] rotate-0 [06:46:56] [PASSED] rotate-90 [06:46:56] [PASSED] rotate-180 [06:46:56] [PASSED] rotate-270 [06:46:56] ============ [PASSED] drm_test_rect_rotate_inv ============= [06:46:56] ==================== [PASSED] drm_rect ===================== [06:46:56] ============ drm_sysfb_modeset_test (1 subtest) ============ [06:46:56] ============ drm_test_sysfb_build_fourcc_list ============= [06:46:56] [PASSED] no native formats [06:46:56] [PASSED] XRGB8888 as native format [06:46:56] [PASSED] remove duplicates [06:46:56] [PASSED] convert alpha formats [06:46:56] [PASSED] random formats [06:46:56] ======== [PASSED] drm_test_sysfb_build_fourcc_list ========= [06:46:56] ============= [PASSED] drm_sysfb_modeset_test ============== [06:46:56] ============================================================ [06:46:56] Testing complete. Ran 621 tests: passed: 621 [06:46:56] Elapsed time: 25.654s total, 1.807s configuring, 23.630s building, 0.179s running
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/ttm/tests/.kunitconfig [06:46:56] Configuring KUnit Kernel ...
Regenerating .config ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig [06:46:58] Building KUnit Kernel ... Populating config with: $ make ARCH=um O=.kunit olddefconfig Building with: $ make all compile_commands.json scripts_gdb ARCH=um O=.kunit --jobs=48 [06:47:07] Starting KUnit Kernel (1/1)... [06:47:07] ============================================================ Running tests with: $ .kunit/linux kunit.enable=1 mem=1G console=tty kunit_shutdown=halt [06:47:07] ================= ttm_device (5 subtests) ================== [06:47:07] [PASSED] ttm_device_init_basic [06:47:07] [PASSED] ttm_device_init_multiple [06:47:07] [PASSED] ttm_device_fini_basic [06:47:07] [PASSED] ttm_device_init_no_vma_man [06:47:07] ================== ttm_device_init_pools ================== [06:47:07] [PASSED] No DMA allocations, no DMA32 required [06:47:07] [PASSED] DMA allocations, DMA32 required [06:47:07] [PASSED] No DMA allocations, DMA32 required [06:47:07] [PASSED] DMA allocations, no DMA32 required [06:47:07] ============== [PASSED] ttm_device_init_pools ============== [06:47:07] =================== [PASSED] ttm_device ==================== [06:47:07] ================== ttm_pool (8 subtests) =================== [06:47:07] ================== ttm_pool_alloc_basic =================== [06:47:07] [PASSED] One page [06:47:07] [PASSED] More than one page [06:47:07] [PASSED] Above the allocation limit [06:47:07] [PASSED] One page, with coherent DMA mappings enabled [06:47:07] [PASSED] Above the allocation limit, with coherent DMA mappings enabled [06:47:07] ============== [PASSED] ttm_pool_alloc_basic =============== [06:47:07] ============== ttm_pool_alloc_basic_dma_addr ============== [06:47:07] [PASSED] One page [06:47:07] [PASSED] More than one page [06:47:07] [PASSED] Above the allocation limit [06:47:07] [PASSED] One page, with coherent DMA mappings enabled [06:47:07] [PASSED] Above the allocation limit, with coherent DMA mappings enabled [06:47:07] ========== [PASSED] ttm_pool_alloc_basic_dma_addr ========== [06:47:07] [PASSED] ttm_pool_alloc_order_caching_match [06:47:07] [PASSED] ttm_pool_alloc_caching_mismatch [06:47:07] [PASSED] ttm_pool_alloc_order_mismatch [06:47:07] [PASSED] ttm_pool_free_dma_alloc [06:47:07] [PASSED] ttm_pool_free_no_dma_alloc [06:47:07] [PASSED] ttm_pool_fini_basic [06:47:07] ==================== [PASSED] ttm_pool ===================== [06:47:07] ================ ttm_resource (8 subtests) ================= [06:47:07] ================= ttm_resource_init_basic ================= [06:47:07] [PASSED] Init resource in TTM_PL_SYSTEM [06:47:07] [PASSED] Init resource in TTM_PL_VRAM [06:47:07] [PASSED] Init resource in a private placement [06:47:07] [PASSED] Init resource in TTM_PL_SYSTEM, set placement flags [06:47:07] ============= [PASSED] ttm_resource_init_basic ============= [06:47:07] [PASSED] ttm_resource_init_pinned [06:47:07] [PASSED] ttm_resource_fini_basic [06:47:07] [PASSED] ttm_resource_manager_init_basic [06:47:07] [PASSED] ttm_resource_manager_usage_basic [06:47:07] [PASSED] ttm_resource_manager_set_used_basic [06:47:07] [PASSED] ttm_sys_man_alloc_basic [06:47:07] [PASSED] ttm_sys_man_free_basic [06:47:07] ================== [PASSED] ttm_resource =================== [06:47:07] =================== ttm_tt (15 subtests) =================== [06:47:07] ==================== ttm_tt_init_basic ==================== [06:47:07] [PASSED] Page-aligned size [06:47:07] [PASSED] Extra pages requested [06:47:07] ================ [PASSED] ttm_tt_init_basic ================ [06:47:07] [PASSED] ttm_tt_init_misaligned [06:47:07] [PASSED] ttm_tt_fini_basic [06:47:07] [PASSED] ttm_tt_fini_sg [06:47:07] [PASSED] ttm_tt_fini_shmem [06:47:07] [PASSED] ttm_tt_create_basic [06:47:07] [PASSED] ttm_tt_create_invalid_bo_type [06:47:07] [PASSED] ttm_tt_create_ttm_exists [06:47:07] [PASSED] ttm_tt_create_failed [06:47:07] [PASSED] ttm_tt_destroy_basic [06:47:07] [PASSED] ttm_tt_populate_null_ttm [06:47:07] [PASSED] ttm_tt_populate_populated_ttm [06:47:07] [PASSED] ttm_tt_unpopulate_basic [06:47:07] [PASSED] ttm_tt_unpopulate_empty_ttm [06:47:07] [PASSED] ttm_tt_swapin_basic [06:47:07] ===================== [PASSED] ttm_tt ====================== [06:47:07] =================== ttm_bo (14 subtests) =================== [06:47:07] =========== ttm_bo_reserve_optimistic_no_ticket =========== [06:47:07] [PASSED] Cannot be interrupted and sleeps [06:47:07] [PASSED] Cannot be interrupted, locks straight away [06:47:07] [PASSED] Can be interrupted, sleeps [06:47:07] ======= [PASSED] ttm_bo_reserve_optimistic_no_ticket ======= [06:47:07] [PASSED] ttm_bo_reserve_locked_no_sleep [06:47:07] [PASSED] ttm_bo_reserve_no_wait_ticket [06:47:07] [PASSED] ttm_bo_reserve_double_resv [06:47:07] [PASSED] ttm_bo_reserve_interrupted [06:47:07] [PASSED] ttm_bo_reserve_deadlock [06:47:07] [PASSED] ttm_bo_unreserve_basic [06:47:07] [PASSED] ttm_bo_unreserve_pinned [06:47:07] [PASSED] ttm_bo_unreserve_bulk [06:47:07] [PASSED] ttm_bo_fini_basic [06:47:07] [PASSED] ttm_bo_fini_shared_resv [06:47:07] [PASSED] ttm_bo_pin_basic [06:47:07] [PASSED] ttm_bo_pin_unpin_resource [06:47:07] [PASSED] ttm_bo_multiple_pin_one_unpin [06:47:07] ===================== [PASSED] ttm_bo ====================== [06:47:07] ============== ttm_bo_validate (21 subtests) =============== [06:47:07] ============== ttm_bo_init_reserved_sys_man =============== [06:47:07] [PASSED] Buffer object for userspace [06:47:07] [PASSED] Kernel buffer object [06:47:07] [PASSED] Shared buffer object [06:47:07] ========== [PASSED] ttm_bo_init_reserved_sys_man =========== [06:47:07] ============== ttm_bo_init_reserved_mock_man ============== [06:47:07] [PASSED] Buffer object for userspace [06:47:07] [PASSED] Kernel buffer object [06:47:07] [PASSED] Shared buffer object [06:47:07] ========== [PASSED] ttm_bo_init_reserved_mock_man ========== [06:47:07] [PASSED] ttm_bo_init_reserved_resv [06:47:07] ================== ttm_bo_validate_basic ================== [06:47:07] [PASSED] Buffer object for userspace [06:47:07] [PASSED] Kernel buffer object [06:47:07] [PASSED] Shared buffer object [06:47:07] ============== [PASSED] ttm_bo_validate_basic ============== [06:47:07] [PASSED] ttm_bo_validate_invalid_placement [06:47:07] ============= ttm_bo_validate_same_placement ============== [06:47:07] [PASSED] System manager [06:47:07] [PASSED] VRAM manager [06:47:07] ========= [PASSED] ttm_bo_validate_same_placement ========== [06:47:07] [PASSED] ttm_bo_validate_failed_alloc [06:47:07] [PASSED] ttm_bo_validate_pinned [06:47:07] [PASSED] ttm_bo_validate_busy_placement [06:47:07] ================ ttm_bo_validate_multihop ================= [06:47:07] [PASSED] Buffer object for userspace [06:47:07] [PASSED] Kernel buffer object [06:47:07] [PASSED] Shared buffer object [06:47:07] ============ [PASSED] ttm_bo_validate_multihop ============= [06:47:07] ========== ttm_bo_validate_no_placement_signaled ========== [06:47:07] [PASSED] Buffer object in system domain, no page vector [06:47:07] [PASSED] Buffer object in system domain with an existing page vector [06:47:07] ====== [PASSED] ttm_bo_validate_no_placement_signaled ====== [06:47:07] ======== ttm_bo_validate_no_placement_not_signaled ======== [06:47:07] [PASSED] Buffer object for userspace [06:47:07] [PASSED] Kernel buffer object [06:47:07] [PASSED] Shared buffer object [06:47:07] ==== [PASSED] ttm_bo_validate_no_placement_not_signaled ==== [06:47:07] [PASSED] ttm_bo_validate_move_fence_signaled [06:47:07] ========= ttm_bo_validate_move_fence_not_signaled ========= [06:47:07] [PASSED] Waits for GPU [06:47:07] [PASSED] Tries to lock straight away [06:47:07] ===== [PASSED] ttm_bo_validate_move_fence_not_signaled ===== [06:47:07] [PASSED] ttm_bo_validate_happy_evict [06:47:07] [PASSED] ttm_bo_validate_all_pinned_evict [06:47:07] [PASSED] ttm_bo_validate_allowed_only_evict [06:47:07] [PASSED] ttm_bo_validate_deleted_evict [06:47:07] [PASSED] ttm_bo_validate_busy_domain_evict [06:47:07] [PASSED] ttm_bo_validate_evict_gutting [06:47:07] [PASSED] ttm_bo_validate_recrusive_evict stty: 'standard input': Inappropriate ioctl for device [06:47:07] ================= [PASSED] ttm_bo_validate ================= [06:47:07] ============================================================ [06:47:07] Testing complete. Ran 101 tests: passed: 101 [06:47:07] Elapsed time: 11.170s total, 1.778s configuring, 9.125s building, 0.229s running
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel
Patchwork
CI✗ Xe.CI.BAT: failure for VF migration redesign (rev4)
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: VF migration redesign (rev4)
URL : https://patchwork.freedesktop.org/series/154627/
State : failure
== Summary ==
CI Bug Log - changes from xe-3853-5f87abb254011980c8332008cfff72d2cfab4952_BAT -> xe-pw-154627v4_BAT
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with xe-pw-154627v4_BAT absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes
introduced in xe-pw-154627v4_BAT, please notify your bug team ([email protected]) to allow them to document this new failure mode, which will reduce false positives in CI.
Participating hosts (11 -> 11)
------------------------------
No changes in participating hosts
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in xe-pw-154627v4_BAT:
### IGT changes ###
#### Possible regressions ####
* igt@sriov_basic@enable-vfs-autoprobe-on:
- bat-bmg-2: [PASS][1] -> [ABORT][2] +1 other test abort
[1]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/bat-bmg-2/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/bat-bmg-2/igt@[email protected]
- bat-adlp-7: [PASS][3] -> [ABORT][4] +1 other test abort
[3]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/bat-adlp-7/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/bat-adlp-7/igt@[email protected]
* igt@sriov_basic@enable-vfs-autoprobe-on@numvfs-1:
- bat-bmg-1: [PASS][5] -> [ABORT][6] +1 other test abort
[5]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/bat-bmg-1/igt@sriov_basic@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/bat-bmg-1/igt@sriov_basic@[email protected]
- bat-atsm-2: [PASS][7] -> [ABORT][8] +1 other test abort
[7]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/bat-atsm-2/igt@sriov_basic@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/bat-atsm-2/igt@sriov_basic@[email protected]
* igt@xe_module_load@load:
- bat-adlp-vm: [PASS][9] -> [ABORT][10]
[9]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/bat-adlp-vm/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/bat-adlp-vm/igt@[email protected]
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.
* igt@sriov_basic@enable-vfs-autoprobe-on:
- {bat-ptl-1}: [PASS][11] -> [ABORT][12] +1 other test abort
[11]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/bat-ptl-1/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/bat-ptl-1/igt@[email protected]
* igt@sriov_basic@enable-vfs-autoprobe-on@numvfs-1:
- {bat-ptl-2}: [PASS][13] -> [ABORT][14] +1 other test abort
[13]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/bat-ptl-2/igt@sriov_basic@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/bat-ptl-2/igt@sriov_basic@[email protected]
* igt@xe_module_load@load:
- {bat-ptl-vm}: [PASS][15] -> [ABORT][16]
[15]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/bat-ptl-vm/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/bat-ptl-vm/igt@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
Build changes
-------------
* Linux: xe-3853-5f87abb254011980c8332008cfff72d2cfab4952 -> xe-pw-154627v4
IGT_8568: 8568
xe-3853-5f87abb254011980c8332008cfff72d2cfab4952: 5f87abb254011980c8332008cfff72d2cfab4952
xe-pw-154627v4: 154627v4
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/index.html
Patchwork
CI✗ Xe.CI.Full: failure for VF migration redesign (rev4)
In reply to Matthew Brost== Series Details ==
== Series Details ==
Series: VF migration redesign (rev4)
URL : https://patchwork.freedesktop.org/series/154627/
State : failure
== Summary ==
CI Bug Log - changes from xe-3853-5f87abb254011980c8332008cfff72d2cfab4952_FULL -> xe-pw-154627v4_FULL
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with xe-pw-154627v4_FULL absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes
introduced in xe-pw-154627v4_FULL, please notify your bug team ([email protected]) to allow them to document this new failure mode, which will reduce false positives in CI.
Participating hosts (4 -> 4)
------------------------------
No changes in participating hosts
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in xe-pw-154627v4_FULL:
### IGT changes ###
#### Possible regressions ####
* igt@xe_exec_compute_mode@non-blocking:
- shard-bmg: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-4/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-4/igt@[email protected]
- shard-dg2-set2: [PASS][3] -> [INCOMPLETE][4]
[3]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-434/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-433/igt@[email protected]
- shard-lnl: [PASS][5] -> [FAIL][6]
[5]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-lnl-2/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-lnl-2/igt@[email protected]
- shard-adlp: [PASS][7] -> [FAIL][8]
[7]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-9/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-8/igt@[email protected]
* igt@xe_pmu@engine-activity-most-load-idle:
- shard-adlp: [PASS][9] -> [ABORT][10] +20 other tests abort
[9]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-4/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-8/igt@[email protected]
* igt@xe_sriov_auto_provisioning@selfconfig-reprovision-reduce-numvfs:
- shard-bmg: [PASS][11] -> [ABORT][12] +24 other tests abort
[11]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-2/igt@xe_sriov_auto_provisioning@selfconfig-reprovision-reduce-numvfs.html
[12]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-2/igt@xe_sriov_auto_provisioning@selfconfig-reprovision-reduce-numvfs.html
#### Warnings ####
* igt@xe_pmu@all-fn-engine-activity-load:
- shard-adlp: [TIMEOUT][13] ([Intel XE#5213]) -> [ABORT][14]
[13]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-3/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-3/igt@[email protected]
* igt@xe_sriov_scheduling@equal-throughput:
- shard-adlp: [DMESG-FAIL][15] ([Intel XE#5213]) -> [ABORT][16] +1 other test abort
[15]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-3/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-6/igt@[email protected]
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.
* {igt@xe_pmu@engine-activity-accuracy-90@engine-drm_xe_engine_class_video_enhance0}:
- shard-adlp: [PASS][17] -> [ABORT][18] +8 other tests abort
[17]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-2/igt@xe_pmu@engine-activity-accuracy-90@engine-drm_xe_engine_class_video_enhance0.html
[18]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-9/igt@xe_pmu@engine-activity-accuracy-90@engine-drm_xe_engine_class_video_enhance0.html
* {igt@xe_pmu@engine-activity-gt-reset-idle}:
- shard-adlp: NOTRUN -> [ABORT][19] +1 other test abort
[19]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-8/igt@[email protected]
* {igt@xe_pmu@engine-activity-render-node-idle}:
- shard-bmg: [PASS][20] -> [ABORT][21] +10 other tests abort
[20]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-4/igt@[email protected]
[21]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-4/igt@[email protected]
Known issues
------------
Here are the changes found in xe-pw-154627v4_FULL that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@kms_big_fb@x-tiled-64bpp-rotate-270:
- shard-dg2-set2: NOTRUN -> [SKIP][22] ([Intel XE#316])
[22]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@kms_big_fb@y-tiled-addfb:
- shard-dg2-set2: NOTRUN -> [SKIP][23] ([Intel XE#619])
[23]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@kms_big_fb@yf-tiled-16bpp-rotate-180:
- shard-dg2-set2: NOTRUN -> [SKIP][24] ([Intel XE#1124]) +2 other tests skip
[24]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@kms_ccs@bad-pixel-format-4-tiled-mtl-rc-ccs-cc@pipe-a-hdmi-a-6:
- shard-dg2-set2: NOTRUN -> [SKIP][25] ([Intel XE#787]) +181 other tests skip
[25]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-435/igt@kms_ccs@[email protected]
* igt@kms_ccs@crc-primary-basic-4-tiled-lnl-ccs@pipe-b-dp-2:
- shard-bmg: NOTRUN -> [SKIP][26] ([Intel XE#2652] / [Intel XE#787]) +3 other tests skip
[26]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@kms_ccs@[email protected]
* igt@kms_ccs@crc-sprite-planes-basic-4-tiled-mtl-rc-ccs@pipe-d-dp-2:
- shard-dg2-set2: NOTRUN -> [SKIP][27] ([Intel XE#455] / [Intel XE#787]) +30 other tests skip
[27]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@kms_ccs@[email protected]
* igt@kms_ccs@random-ccs-data-4-tiled-dg2-rc-ccs-cc:
- shard-dg2-set2: [PASS][28] -> [INCOMPLETE][29] ([Intel XE#1727] / [Intel XE#3113] / [Intel XE#4212] / [Intel XE#4345] / [Intel XE#4522])
[28]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-435/igt@[email protected]
[29]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@kms_ccs@random-ccs-data-4-tiled-dg2-rc-ccs-cc@pipe-b-dp-4:
- shard-dg2-set2: [PASS][30] -> [INCOMPLETE][31] ([Intel XE#1727] / [Intel XE#3113] / [Intel XE#4212] / [Intel XE#4522])
[30]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-435/igt@kms_ccs@[email protected]
[31]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@kms_ccs@[email protected]
* igt@kms_ccs@random-ccs-data-4-tiled-dg2-rc-ccs@pipe-a-hdmi-a-6:
- shard-dg2-set2: [PASS][32] -> [INCOMPLETE][33] ([Intel XE#1727] / [Intel XE#3113] / [Intel XE#4345])
[32]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-433/igt@kms_ccs@[email protected]
[33]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-464/igt@kms_ccs@[email protected]
* igt@kms_chamelium_hpd@dp-hpd-fast:
- shard-dg2-set2: NOTRUN -> [SKIP][34] ([Intel XE#373]) +1 other test skip
[34]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@kms_content_protection@atomic-dpms@pipe-a-dp-4:
- shard-dg2-set2: NOTRUN -> [FAIL][35] ([Intel XE#1178]) +1 other test fail
[35]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-433/igt@kms_content_protection@[email protected]
* igt@kms_content_protection@dp-mst-lic-type-1:
- shard-dg2-set2: NOTRUN -> [SKIP][36] ([Intel XE#307])
[36]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@kms_cursor_crc@cursor-sliding-512x170:
- shard-dg2-set2: NOTRUN -> [SKIP][37] ([Intel XE#308]) +1 other test skip
[37]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@kms_cursor_legacy@cursorb-vs-flipb-atomic-transitions:
- shard-bmg: [PASS][38] -> [SKIP][39] ([Intel XE#2291])
[38]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-4/igt@[email protected]
[39]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-6/igt@[email protected]
* igt@kms_cursor_legacy@short-busy-flip-before-cursor-toggle:
- shard-dg2-set2: NOTRUN -> [SKIP][40] ([Intel XE#323])
[40]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@kms_display_modes@extended-mode-basic:
- shard-bmg: [PASS][41] -> [SKIP][42] ([Intel XE#4302])
[41]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-4/igt@[email protected]
[42]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-6/igt@[email protected]
* igt@kms_feature_discovery@dp-mst:
- shard-dg2-set2: NOTRUN -> [SKIP][43] ([Intel XE#1137])
[43]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@kms_flip@2x-flip-vs-absolute-wf_vblank:
- shard-dg2-set2: [PASS][44] -> [FAIL][45] ([Intel XE#5408]) +1 other test fail
[44]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-466/igt@kms_flip@2x-flip-vs-absolute-wf_vblank.html
[45]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-466/igt@kms_flip@2x-flip-vs-absolute-wf_vblank.html
* igt@kms_flip@2x-flip-vs-rmfb@ad-hdmi-a6-dp4:
- shard-dg2-set2: [PASS][46] -> [INCOMPLETE][47] ([Intel XE#2049]) +1 other test incomplete
[46]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-464/igt@kms_flip@[email protected]
[47]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-434/igt@kms_flip@[email protected]
* igt@kms_flip@2x-nonexisting-fb:
- shard-bmg: [PASS][48] -> [SKIP][49] ([Intel XE#2316]) +4 other tests skip
[48]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-4/igt@[email protected]
[49]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-6/igt@[email protected]
* igt@kms_flip@plain-flip-interruptible@b-hdmi-a1:
- shard-adlp: [PASS][50] -> [DMESG-WARN][51] ([Intel XE#4543]) +3 other tests dmesg-warn
[50]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-3/igt@kms_flip@[email protected]
[51]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-3/igt@kms_flip@[email protected]
* igt@kms_flip_scaled_crc@flip-32bpp-yftile-to-64bpp-yftile-upscaling:
- shard-dg2-set2: NOTRUN -> [SKIP][52] ([Intel XE#455]) +4 other tests skip
[52]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@kms_flip_scaled_crc@flip-32bpp-yftile-to-64bpp-yftile-upscaling.html
* igt@kms_frontbuffer_tracking@drrs-suspend:
- shard-dg2-set2: NOTRUN -> [SKIP][53] ([Intel XE#651]) +7 other tests skip
[53]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@kms_frontbuffer_tracking@fbcdrrs-2p-primscrn-cur-indfb-draw-blt:
- shard-adlp: NOTRUN -> [SKIP][54] ([Intel XE#656])
[54]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-8/igt@kms_frontbuffer_tracking@fbcdrrs-2p-primscrn-cur-indfb-draw-blt.html
* igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-shrfb-msflip-blt:
- shard-dg2-set2: NOTRUN -> [SKIP][55] ([Intel XE#653]) +10 other tests skip
[55]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-shrfb-msflip-blt.html
* igt@kms_pipe_crc_basic@suspend-read-crc@pipe-a-hdmi-a-6:
- shard-dg2-set2: [PASS][56] -> [INCOMPLETE][57] ([Intel XE#2597]) +1 other test incomplete
[56]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-463/igt@kms_pipe_crc_basic@[email protected]
[57]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-436/igt@kms_pipe_crc_basic@[email protected]
* igt@kms_pipe_stress@stress-xrgb8888-ytiled:
- shard-dg2-set2: NOTRUN -> [SKIP][58] ([Intel XE#4359])
[58]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@kms_plane_cursor@primary@pipe-a-hdmi-a-2-size-256:
- shard-dg2-set2: NOTRUN -> [FAIL][59] ([Intel XE#616]) +3 other tests fail
[59]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@kms_plane_cursor@[email protected]
* igt@kms_psr2_sf@pr-primary-plane-update-sf-dmg-area:
- shard-dg2-set2: NOTRUN -> [SKIP][60] ([Intel XE#1406] / [Intel XE#1489])
[60]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@kms_psr@fbc-psr2-cursor-plane-onoff:
- shard-dg2-set2: NOTRUN -> [SKIP][61] ([Intel XE#1406] / [Intel XE#2850] / [Intel XE#929]) +2 other tests skip
[61]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@xe_eudebug_online@single-step-one:
- shard-dg2-set2: NOTRUN -> [SKIP][62] ([Intel XE#4837]) +1 other test skip
[62]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@xe_exec_basic@multigpu-many-execqueues-many-vm-bindexecqueue-userptr-rebind:
- shard-dg2-set2: [PASS][63] -> [SKIP][64] ([Intel XE#1392]) +6 other tests skip
[63]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-466/igt@xe_exec_basic@multigpu-many-execqueues-many-vm-bindexecqueue-userptr-rebind.html
[64]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@xe_exec_basic@multigpu-many-execqueues-many-vm-bindexecqueue-userptr-rebind.html
* igt@xe_exec_fault_mode@twice-userptr-invalidate-race:
- shard-dg2-set2: NOTRUN -> [SKIP][65] ([Intel XE#288]) +4 other tests skip
[65]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@xe_exec_system_allocator@many-large-execqueues-mmap-free-huge:
- shard-adlp: NOTRUN -> [SKIP][66] ([Intel XE#4915]) +4 other tests skip
[66]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-8/igt@xe_exec_system_allocator@many-large-execqueues-mmap-free-huge.html
* igt@xe_exec_system_allocator@process-many-execqueues-free-race:
- shard-dg2-set2: NOTRUN -> [SKIP][67] ([Intel XE#4915]) +58 other tests skip
[67]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@xe_peer2peer@read@read-gpua-vram01-gpub-system-p2p:
- shard-dg2-set2: NOTRUN -> [FAIL][68] ([Intel XE#1173])
[68]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-466/igt@xe_peer2peer@[email protected]
* igt@xe_query@multigpu-query-gt-list:
- shard-dg2-set2: NOTRUN -> [SKIP][69] ([Intel XE#944]) +1 other test skip
[69]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@xe_sriov_scheduling@equal-throughput:
- shard-dg2-set2: NOTRUN -> [SKIP][70] ([Intel XE#4351])
[70]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
#### Possible fixes ####
* igt@kms_addfb_basic@small-bo:
- shard-adlp: [DMESG-WARN][71] ([Intel XE#2953] / [Intel XE#4173]) -> [PASS][72]
[71]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-2/igt@[email protected]
[72]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-9/igt@[email protected]
* igt@kms_bw@connected-linear-tiling-2-displays-2560x1440p:
- shard-bmg: [SKIP][73] ([Intel XE#2314] / [Intel XE#2894]) -> [PASS][74]
[73]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@[email protected]
[74]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-4/igt@[email protected]
* igt@kms_cursor_legacy@cursorb-vs-flipb-atomic:
- shard-bmg: [SKIP][75] ([Intel XE#2291]) -> [PASS][76]
[75]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@[email protected]
[76]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-4/igt@[email protected]
* igt@kms_feature_discovery@display-2x:
- shard-bmg: [SKIP][77] ([Intel XE#2373]) -> [PASS][78]
[77]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@[email protected]
[78]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@[email protected]
* igt@kms_flip@2x-plain-flip-fb-recreate:
- shard-bmg: [SKIP][79] ([Intel XE#2316]) -> [PASS][80] +5 other tests pass
[79]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@[email protected]
[80]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@[email protected]
* igt@kms_flip@flip-vs-expired-vblank@a-edp1:
- shard-lnl: [FAIL][81] ([Intel XE#301]) -> [PASS][82] +1 other test pass
[81]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-lnl-4/igt@kms_flip@[email protected]
[82]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-lnl-3/igt@kms_flip@[email protected]
* igt@kms_flip@plain-flip-interruptible@d-hdmi-a1:
- shard-adlp: [DMESG-WARN][83] ([Intel XE#4543]) -> [PASS][84] +1 other test pass
[83]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-3/igt@kms_flip@[email protected]
[84]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-3/igt@kms_flip@[email protected]
* igt@kms_setmode@invalid-clone-single-crtc:
- shard-bmg: [SKIP][85] ([Intel XE#1435]) -> [PASS][86]
[85]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@[email protected]
[86]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@[email protected]
* igt@kms_vrr@cmrr@pipe-a-edp-1:
- shard-lnl: [FAIL][87] ([Intel XE#4459]) -> [PASS][88] +1 other test pass
[87]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-lnl-7/igt@kms_vrr@[email protected]
[88]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-lnl-5/igt@kms_vrr@[email protected]
* igt@xe_evict_ccs@evict-overcommit-parallel-nofree-samefd:
- shard-dg2-set2: [INCOMPLETE][89] -> [PASS][90]
[89]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-435/igt@[email protected]
[90]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-463/igt@[email protected]
* igt@xe_exec_basic@multigpu-no-exec-userptr-invalidate-race:
- shard-dg2-set2: [SKIP][91] ([Intel XE#1392]) -> [PASS][92] +6 other tests pass
[91]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-432/igt@[email protected]
[92]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-435/igt@[email protected]
* {igt@xe_exec_system_allocator@many-large-execqueues-malloc-prefetch}:
- shard-lnl: [CRASH][93] ([Intel XE#6192]) -> [PASS][94] +7 other tests pass
[93]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-lnl-1/igt@xe_exec_system_allocator@many-large-execqueues-malloc-prefetch.html
[94]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-lnl-7/igt@xe_exec_system_allocator@many-large-execqueues-malloc-prefetch.html
* {igt@xe_exec_system_allocator@once-mmap-prefetch-shared}:
- shard-bmg: [CRASH][95] ([Intel XE#6192]) -> [PASS][96] +2 other tests pass
[95]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-7/igt@[email protected]
[96]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-6/igt@[email protected]
* igt@xe_fault_injection@probe-fail-guc-xe_guc_mmio_send_recv:
- shard-dg2-set2: [DMESG-WARN][97] ([Intel XE#5893]) -> [PASS][98]
[97]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-434/igt@xe_fault_injection@probe-fail-guc-xe_guc_mmio_send_recv.html
[98]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@xe_fault_injection@probe-fail-guc-xe_guc_mmio_send_recv.html
* igt@xe_pmu@gt-frequency:
- shard-dg2-set2: [FAIL][99] ([Intel XE#4819]) -> [PASS][100] +1 other test pass
[99]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-434/igt@[email protected]
[100]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-435/igt@[email protected]
#### Warnings ####
* igt@kms_ccs@random-ccs-data-4-tiled-dg2-rc-ccs:
- shard-dg2-set2: [INCOMPLETE][101] ([Intel XE#1727] / [Intel XE#2705] / [Intel XE#3113] / [Intel XE#4212] / [Intel XE#4345] / [Intel XE#4522]) -> [INCOMPLETE][102] ([Intel XE#1727] / [Intel XE#3113] / [Intel XE#4345])
[101]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-433/igt@[email protected]
[102]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-464/igt@[email protected]
* igt@kms_frontbuffer_tracking@drrs-2p-primscrn-spr-indfb-draw-blt:
- shard-bmg: [SKIP][103] ([Intel XE#2311]) -> [SKIP][104] ([Intel XE#2312]) +5 other tests skip
[103]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-7/igt@kms_frontbuffer_tracking@drrs-2p-primscrn-spr-indfb-draw-blt.html
[104]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-6/igt@kms_frontbuffer_tracking@drrs-2p-primscrn-spr-indfb-draw-blt.html
* igt@kms_frontbuffer_tracking@drrs-2p-scndscrn-indfb-msflip-blt:
- shard-bmg: [SKIP][105] ([Intel XE#2312]) -> [SKIP][106] ([Intel XE#2311]) +3 other tests skip
[105]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@[email protected]
[106]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@[email protected]
* igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-render:
- shard-bmg: [SKIP][107] ([Intel XE#5390]) -> [SKIP][108] ([Intel XE#2312]) +2 other tests skip
[107]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-4/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-render.html
[108]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-6/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-render.html
* igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-spr-indfb-draw-render:
- shard-bmg: [SKIP][109] ([Intel XE#2312]) -> [SKIP][110] ([Intel XE#5390]) +2 other tests skip
[109]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-spr-indfb-draw-render.html
[110]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-spr-indfb-draw-render.html
* igt@kms_frontbuffer_tracking@fbcpsr-2p-pri-indfb-multidraw:
- shard-bmg: [SKIP][111] ([Intel XE#2313]) -> [SKIP][112] ([Intel XE#2312]) +5 other tests skip
[111]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-4/igt@[email protected]
[112]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-6/igt@[email protected]
* igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-shrfb-draw-blt:
- shard-bmg: [SKIP][113] ([Intel XE#2312]) -> [SKIP][114] ([Intel XE#2313]) +4 other tests skip
[113]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-shrfb-draw-blt.html
[114]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@kms_frontbuffer_tracking@fbcpsr-2p-primscrn-pri-shrfb-draw-blt.html
* igt@kms_plane_multiple@2x-tiling-yf:
- shard-bmg: [SKIP][115] ([Intel XE#4596]) -> [SKIP][116] ([Intel XE#5021])
[115]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-6/igt@[email protected]
[116]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@[email protected]
* igt@xe_exec_basic@multigpu-many-execqueues-many-vm-userptr:
- shard-dg2-set2: [INCOMPLETE][117] ([Intel XE#4842]) -> [SKIP][118] ([Intel XE#1392])
[117]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-434/igt@[email protected]
[118]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-432/igt@[email protected]
* igt@xe_fault_injection@probe-fail-guc-xe_guc_ct_send_recv:
- shard-adlp: [ABORT][119] ([Intel XE#4917] / [Intel XE#5530]) -> [ABORT][120] ([Intel XE#5530])
[119]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-adlp-9/igt@xe_fault_injection@probe-fail-guc-xe_guc_ct_send_recv.html
[120]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-adlp-8/igt@xe_fault_injection@probe-fail-guc-xe_guc_ct_send_recv.html
- shard-bmg: [ABORT][121] ([Intel XE#5466] / [Intel XE#5530]) -> [ABORT][122] ([Intel XE#4917] / [Intel XE#5466] / [Intel XE#5530])
[121]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-bmg-2/igt@xe_fault_injection@probe-fail-guc-xe_guc_ct_send_recv.html
[122]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-bmg-3/igt@xe_fault_injection@probe-fail-guc-xe_guc_ct_send_recv.html
* igt@xe_peer2peer@read:
- shard-dg2-set2: [SKIP][123] ([Intel XE#1061]) -> [FAIL][124] ([Intel XE#1173])
[123]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-3853-5f87abb254011980c8332008cfff72d2cfab4952/shard-dg2-432/igt@[email protected]
[124]: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/shard-dg2-466/igt@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[Intel XE#1061]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1061
[Intel XE#1124]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1124
[Intel XE#1137]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1137
[Intel XE#1173]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1173
[Intel XE#1178]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1178
[Intel XE#1392]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1392
[Intel XE#1406]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1406
[Intel XE#1435]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1435
[Intel XE#1489]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1489
[Intel XE#1727]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1727
[Intel XE#2049]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2049
[Intel XE#2291]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2291
[Intel XE#2311]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2311
[Intel XE#2312]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2312
[Intel XE#2313]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2313
[Intel XE#2314]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2314
[Intel XE#2316]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2316
[Intel XE#2373]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2373
[Intel XE#2597]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2597
[Intel XE#2652]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2652
[Intel XE#2705]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2705
[Intel XE#2850]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2850
[Intel XE#288]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/288
[Intel XE#2894]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2894
[Intel XE#2953]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2953
[Intel XE#301]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/301
[Intel XE#307]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/307
[Intel XE#308]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/308
[Intel XE#3113]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/3113
[Intel XE#316]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/316
[Intel XE#323]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/323
[Intel XE#373]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/373
[Intel XE#4173]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4173
[Intel XE#4212]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4212
[Intel XE#4302]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4302
[Intel XE#4345]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4345
[Intel XE#4351]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4351
[Intel XE#4359]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4359
[Intel XE#4459]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4459
[Intel XE#4522]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4522
[Intel XE#4543]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4543
[Intel XE#455]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/455
[Intel XE#4596]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4596
[Intel XE#4819]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4819
[Intel XE#4837]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4837
[Intel XE#4842]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4842
[Intel XE#4915]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4915
[Intel XE#4917]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/4917
[Intel XE#5021]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/5021
[Intel XE#5213]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/5213
[Intel XE#5390]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/5390
[Intel XE#5408]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/5408
[Intel XE#5466]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/5466
[Intel XE#5530]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/5530
[Intel XE#5786]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/5786
[Intel XE#5893]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/5893
[Intel XE#616]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/616
[Intel XE#619]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/619
[Intel XE#6192]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/6192
[Intel XE#651]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/651
[Intel XE#653]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/653
[Intel XE#656]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/656
[Intel XE#787]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/787
[Intel XE#929]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/929
[Intel XE#944]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/944
Build changes
-------------
* Linux: xe-3853-5f87abb254011980c8332008cfff72d2cfab4952 -> xe-pw-154627v4
IGT_8568: 8568
xe-3853-5f87abb254011980c8332008cfff72d2cfab4952: 5f87abb254011980c8332008cfff72d2cfab4952
xe-pw-154627v4: 154627v4
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/xe-pw-154627v4/index.html