Skip to contentSkip to conversation

VF migration redesign

PATCH Superseded Matthew Brost 6 participants 90 messages Sep 24, 2025 – Sep 30, 2025 Page 1 of 2

Subsystems (4)

v2 of 10 Superseded Latest is v10 →

lore
Share & tools
lore

Matthew Brost

author maintainer
#0 lore raw

v2 cover VF migration redesign

Rather than modifying buffers in place using GGTT addresses during VF migration, this approach relies on the submission backend's stop/start mechanism to issue fixups. The patch titled "Document GuC Submission Backend" provides a detailed explanation of the design.

Testing was performed using an out-of-tree PF/VFIO driver with manual triggering of VF migration while IGT test cases are running.

IGT test cases:

- A new series [1] that exercises active contexts, job resubmission, and compressd memory.

- A new test [2] that actively creates / destroys queue on each submission

- xe_exec_threads basic sections, which test context registration loss, schedule enable loss, and job resubmission.

- xe_exec_threads balancer sections, which follow the same flows as the basic sections but include a work queue (GGTT address shift).

- xe_exec_threads compute mode user pointer invalidation sections, which exercise the same flow as the basic sections, plus replaying suspend/resume flows.

All code paths in "Replay GuC submission state on pause/unpause" that replay state have been manually verified via debug messages "Add debug prints for GuC replaying state during VF recovery".

v2:
- Fix lockdep splat
- Fix checkpatch
- Fix PTL issue with LRC W/A buffer
- Fix race creating / destroying queues across migration exposed by [2]
- Include a version of Satya's patches in [3] which enable CCS save / restore across VF migration /w GGTT shift

Matt

[1] https://patchwork.freedesktop.org/series/154616/
[2] https://patchwork.freedesktop.org/series/154931/
[3] https://patchwork.freedesktop.org/series/154682/

Matthew Brost (31):
Revert "drm/xe/vf: Rebase exec queue parallel commands during
migration recovery"
Revert "drm/xe/vf: Post migration, repopulate ring area for pending
request"
Revert "drm/xe/vf: Fixup CTB send buffer messages after migration" drm/xe: Save off position in ring in which a job was programmed drm/xe/guc: Track pending-enable source in submission state drm/xe: Track LR jobs in DRM scheduler pending list drm/xe: Don't change LRC ring head on job resubmission drm/xe: Make LRC W/A scratch buffer usage consistent drm/xe/guc: Document GuC submission backend
drm/xe/vf: Add xe_gt_sriov_vf_recovery_inprogress helper drm/xe/vf: Make VF recovery run on per-GT worker drm/xe/vf: Abort H2G sends during VF post-migration recovery drm/xe/vf: Remove memory allocations from VF post migration recovery drm/xe/vf: Close multi-GT GGTT shift race
drm/xe/vf: Teardown VF post migration worker on driver unload drm/xe/vf: Don't allow GT reset to be queued during VF post migration
recovery
drm/xe/vf: Wakeup in GuC backend on VF post migration recovery drm/xe/vf: Extra debug on GGTT shift
drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register drm/xe/vf: Stop and flush CTs in VF post migration recovery drm/xe/vf: Reset TLB invalidations during VF post migration recovery drm/xe/vf: Kickstart after resfix in VF post migration recovery drm/xe/vf: Start CTs before resfix VF post migration recovery drm/xe/vf: Abort VF post migration recovery on failure drm/xe/vf: Replay GuC submission state on pause / unpause drm/xe: Move queue init before LRC creation
drm/xe/vf: Add debug prints for GuC replaying state during VF recovery drm/xe/vf: Workaround for race condition in GuC firmware during VF
pause
drm/xe/vf: Use primary GT ordered work queue on media GT on PTL VF drm/xe/vf: Ensure media GT VF recovery runs after primary GT on PTL drm/xe/vf: Rebase CCS save/restore BB GGTT addresses

Satyanarayana K V P (2):
drm/xe: Use PPGTT addresses for TLB invalidation to avoid GGTT fixups drm/xe/guc: Increase wait timeout to 2sec after BUSY reply from GuC

Tomasz Lis (1):
drm/xe/vf: Lock querying GGTT config during driver init

 Documentation/gpu/xe/index.rst               |   1 +
 drivers/gpu/drm/xe/abi/guc_actions_abi.h     |   8 -
 drivers/gpu/drm/xe/xe_device_types.h         |   2 +
 drivers/gpu/drm/xe/xe_exec.c                 |  12 +-
 drivers/gpu/drm/xe/xe_exec_queue.c           |  86 +-
 drivers/gpu/drm/xe/xe_exec_queue.h           |   5 +-
 drivers/gpu/drm/xe/xe_execlist.c             |   2 +-
 drivers/gpu/drm/xe/xe_gpu_scheduler.c        |  14 +
 drivers/gpu/drm/xe/xe_gpu_scheduler.h        |   2 +
 drivers/gpu/drm/xe/xe_gt.c                   |  25 +-
 drivers/gpu/drm/xe/xe_gt.h                   |   2 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c          | 417 ++++++++--
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h          |  10 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h    |  36 +-
 drivers/gpu/drm/xe/xe_guc.c                  |   4 +-
 drivers/gpu/drm/xe/xe_guc_ct.c               | 284 ++-----
 drivers/gpu/drm/xe/xe_guc_ct.h               |   4 +-
 drivers/gpu/drm/xe/xe_guc_exec_queue_types.h |  15 +
 drivers/gpu/drm/xe/xe_guc_submit.c           | 790 +++++++++++++++----
 drivers/gpu/drm/xe/xe_guc_submit.h           |   7 +-
 drivers/gpu/drm/xe/xe_lrc.c                  |   2 +-
 drivers/gpu/drm/xe/xe_lrc.h                  |  10 +
 drivers/gpu/drm/xe/xe_map.h                  |  18 -
 drivers/gpu/drm/xe/xe_memirq.c               |  48 +-
 drivers/gpu/drm/xe/xe_memirq.h               |   3 +
 drivers/gpu/drm/xe/xe_migrate.c              |  28 +-
 drivers/gpu/drm/xe/xe_pci.c                  |   6 +-
 drivers/gpu/drm/xe/xe_pci_types.h            |   1 +
 drivers/gpu/drm/xe/xe_ring_ops.c             |  23 +-
 drivers/gpu/drm/xe/xe_sched_job_types.h      |   9 +
 drivers/gpu/drm/xe/xe_sriov.c                |   8 +-
 drivers/gpu/drm/xe/xe_sriov_vf.c             | 243 +-----
 drivers/gpu/drm/xe/xe_sriov_vf.h             |   3 +-
 drivers/gpu/drm/xe/xe_sriov_vf_ccs.c         |  24 +
 drivers/gpu/drm/xe/xe_sriov_vf_ccs.h         |   1 +
 drivers/gpu/drm/xe/xe_sriov_vf_types.h       |   4 -
 drivers/gpu/drm/xe/xe_tile.c                 |   2 +-
 drivers/gpu/drm/xe/xe_tile_sriov_vf.c        |   6 +-
 drivers/gpu/drm/xe/xe_tile_sriov_vf.h        |   1 -
 39 files changed, 1369 insertions(+), 797 deletions(-)
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v2 1/34 drm/xe/vf: Lock querying GGTT config during driver init

From: Tomasz Lis <[email protected]>

Protect access to GGTT config as this is non-static information.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 96 ++++++++++++++++++-----
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  3 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          |  6 ++
 3 files changed, 84 insertions(+), 21 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +75 -21
index 0461d5513487..016c867e5e2b 100644
@@ -440,18 +440,21 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_SIZE_KEY, &size);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	if (config->ggtt_size && config->ggtt_size != size) {
 		xe_gt_sriov_err(gt, "Unexpected GGTT reassignment: %lluK != %lluK\n",
 				size / SZ_1K, config->ggtt_size / SZ_1K);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
@@ -460,8 +463,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 	config->ggtt_shift = start - (s64)config->ggtt_base;
 	config->ggtt_base = start;
 	config->ggtt_size = size;
+	err = config->ggtt_size ? 0 : -ENODATA;
 
-	return config->ggtt_size ? 0 : -ENODATA;
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static int vf_get_lmem_info(struct xe_gt *gt)
@@ -474,22 +480,28 @@ static int vf_get_lmem_info(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	if (config->lmem_size && config->lmem_size != size) {
 		xe_gt_sriov_err(gt, "Unexpected LMEM reassignment: %lluM != %lluM\n",
 				size / SZ_1M, config->lmem_size / SZ_1M);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	string_get_size(size, 1, STRING_UNITS_2, size_str, sizeof(size_str));
 	xe_gt_sriov_dbg_verbose(gt, "LMEM %lluM %s\n", size / SZ_1M, size_str);
 
 	config->lmem_size = size;
+	err = config->lmem_size ? 0 : -ENODATA;
 
-	return config->lmem_size ? 0 : -ENODATA;
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static int vf_get_submission_cfg(struct xe_gt *gt)
@@ -501,23 +513,27 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
 	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_DOORBELLS_KEY, &num_dbs);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	if (config->num_ctxs && config->num_ctxs != num_ctxs) {
 		xe_gt_sriov_err(gt, "Unexpected CTXs reassignment: %u != %u\n",
 				num_ctxs, config->num_ctxs);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 	if (config->num_dbs && config->num_dbs != num_dbs) {
 		xe_gt_sriov_err(gt, "Unexpected DBs reassignment: %u != %u\n",
 				num_dbs, config->num_dbs);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	xe_gt_sriov_dbg_verbose(gt, "CTXs %u DBs %u\n", num_ctxs, num_dbs);
@@ -525,7 +541,11 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 	config->num_ctxs = num_ctxs;
 	config->num_dbs = num_dbs;
 
-	return config->num_ctxs ? 0 : -ENODATA;
+	err = config->num_ctxs ? 0 : -ENODATA;
+
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static void vf_cache_gmdid(struct xe_gt *gt)
@@ -579,11 +599,18 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
  */
 u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u16 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.num_ctxs);
 
-	return gt->sriov.vf.self_config.num_ctxs;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->num_ctxs);
+	val = config->num_ctxs;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -596,11 +623,18 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.lmem_size);
 
-	return gt->sriov.vf.self_config.lmem_size;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->lmem_size);
+	val = config->lmem_size;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -613,11 +647,17 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.ggtt_size);
 
-	return gt->sriov.vf.self_config.ggtt_size;
+	down_read(&config->lock);
+	val = config->ggtt_size;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -630,11 +670,18 @@ u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.ggtt_size);
 
-	return gt->sriov.vf.self_config.ggtt_base;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->ggtt_size);
+	val = config->ggtt_base;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -648,11 +695,16 @@ u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
 s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)
 {
 	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	s64 val;
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, xe_gt_is_main_type(gt));
 
-	return config->ggtt_shift;
+	down_read(&config->lock);
+	val = config->ggtt_shift;
+	up_read(&config->lock);
+
+	return val;
 }
 
 static int relay_action_handshake(struct xe_gt *gt, u32 *major, u32 *minor)
@@ -1044,6 +1096,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_read(&config->lock);
 	drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
 		   config->ggtt_base,
 		   config->ggtt_base + config->ggtt_size - 1);
@@ -1060,6 +1113,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
 	drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
 	drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
+	up_read(&config->lock);
 }
 
 /**
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -0
index 298dedf4b009..d95857bd789b 100644
@@ -6,6 +6,7 @@
 #ifndef _XE_GT_SRIOV_VF_TYPES_H_
 #define _XE_GT_SRIOV_VF_TYPES_H_
 
+#include <linux/rwsem.h>
 #include <linux/types.h>
 #include "xe_uc_fw_types.h"
 
@@ -25,6 +26,8 @@ struct xe_gt_sriov_vf_selfconfig {
 	u16 num_ctxs;
 	/** @num_dbs: assigned number of GuC doorbells IDs. */
 	u16 num_dbs;
+	/** @lock: lock for protecting access to all selfconfig fields. */
+	struct rw_semaphore lock;
 };
 
 /**
drivers/gpu/drm/xe/xe_sriov_vf.c +6 -0
index cdd9f8e78b2a..d6e2ed9b9bbc 100644
@@ -197,6 +197,12 @@ static void vf_migration_init_early(struct xe_device *xe)
  */
 void xe_sriov_vf_init_early(struct xe_device *xe)
 {
+	struct xe_gt *gt;
+	unsigned int id;
+
+	for_each_gt(gt, xe, id)
+		init_rwsem(&gt->sriov.vf.self_config.lock);
+
 	vf_migration_init_early(xe);
 }
 
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v2 01/34] drm/xe/vf: Lock querying GGTT config during driver init

Quoted 17 lines · Matthew Brost

From: Tomasz Lis <[email protected]>

Protect access to GGTT config as this is non-static information.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 96 ++++++++++++++++++-----
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  3 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          |  6 ++
 3 files changed, 84 insertions(+), 21 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index 0461d5513487..016c867e5e2b 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt) · line 443

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -440,18 +440,21 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);

do we need to protect also VF2GUC communication ..

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt) · line 452

Quoted 11 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -2
+
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_SIZE_KEY, &size);
 	if (unlikely(err))
-		return err;
+		goto out;
 

.. or just RW access to the data itself from here?

Quoted 100 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +28 -10
 	if (config->ggtt_size && config->ggtt_size != size) {
 		xe_gt_sriov_err(gt, "Unexpected GGTT reassignment: %lluK != %lluK\n",
 				size / SZ_1K, config->ggtt_size / SZ_1K);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
@@ -460,8 +463,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 	config->ggtt_shift = start - (s64)config->ggtt_base;
 	config->ggtt_base = start;
 	config->ggtt_size = size;
+	err = config->ggtt_size ? 0 : -ENODATA;
 
-	return config->ggtt_size ? 0 : -ENODATA;
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static int vf_get_lmem_info(struct xe_gt *gt)
@@ -474,22 +480,28 @@ static int vf_get_lmem_info(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	if (config->lmem_size && config->lmem_size != size) {
 		xe_gt_sriov_err(gt, "Unexpected LMEM reassignment: %lluM != %lluM\n",
 				size / SZ_1M, config->lmem_size / SZ_1M);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	string_get_size(size, 1, STRING_UNITS_2, size_str, sizeof(size_str));
 	xe_gt_sriov_dbg_verbose(gt, "LMEM %lluM %s\n", size / SZ_1M, size_str);
 
 	config->lmem_size = size;
+	err = config->lmem_size ? 0 : -ENODATA;
 
-	return config->lmem_size ? 0 : -ENODATA;
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static int vf_get_submission_cfg(struct xe_gt *gt)
@@ -501,23 +513,27 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
 	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_DOORBELLS_KEY, &num_dbs);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	if (config->num_ctxs && config->num_ctxs != num_ctxs) {
 		xe_gt_sriov_err(gt, "Unexpected CTXs reassignment: %u != %u\n",
 				num_ctxs, config->num_ctxs);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 	if (config->num_dbs && config->num_dbs != num_dbs) {
 		xe_gt_sriov_err(gt, "Unexpected DBs reassignment: %u != %u\n",
 				num_dbs, config->num_dbs);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	xe_gt_sriov_dbg_verbose(gt, "CTXs %u DBs %u\n", num_ctxs, num_dbs);
@@ -525,7 +541,11 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 	config->num_ctxs = num_ctxs;
 	config->num_dbs = num_dbs;
 
-	return config->num_ctxs ? 0 : -ENODATA;
+	err = config->num_ctxs ? 0 : -ENODATA;
+
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static void vf_cache_gmdid(struct xe_gt *gt)

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 584

Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -1
@@ -579,11 +599,18 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
  */
 u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u16 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.num_ctxs);

the purpose of this assert was to make sure no part of the VF driver is trying to get num_ctxs before we finish querying it from the GuC ..

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 609

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
-	return gt->sriov.vf.self_config.num_ctxs;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->num_ctxs);

.. instead of moving this assert in the middle of the lock ..

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 612

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+	val = config->num_ctxs;
+	up_read(&config->lock);
+

.. for code clarity you can add assert here as:

	xe_gt_assert(gt, val);
Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+	return val;
 }
 
 /**

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt) · line 601

Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -1
@@ -596,11 +623,18 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.lmem_size);

ditto

Quoted 112 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +31 -6
-	return gt->sriov.vf.self_config.lmem_size;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->lmem_size);
+	val = config->lmem_size;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -613,11 +647,17 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.ggtt_size);
 
-	return gt->sriov.vf.self_config.ggtt_size;
+	down_read(&config->lock);
+	val = config->ggtt_size;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -630,11 +670,18 @@ u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.ggtt_size);
 
-	return gt->sriov.vf.self_config.ggtt_base;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->ggtt_size);
+	val = config->ggtt_base;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -648,11 +695,16 @@ u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
 s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)
 {
 	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	s64 val;
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, xe_gt_is_main_type(gt));
 
-	return config->ggtt_shift;
+	down_read(&config->lock);
+	val = config->ggtt_shift;
+	up_read(&config->lock);
+
+	return val;
 }
 
 static int relay_action_handshake(struct xe_gt *gt, u32 *major, u32 *minor)
@@ -1044,6 +1096,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_read(&config->lock);
 	drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
 		   config->ggtt_base,
 		   config->ggtt_base + config->ggtt_size - 1);
@@ -1060,6 +1113,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
 	drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
 	drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
+	up_read(&config->lock);
 }
 
 /**
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -0
index 298dedf4b009..d95857bd789b 100644
@@ -6,6 +6,7 @@
 #ifndef _XE_GT_SRIOV_VF_TYPES_H_
 #define _XE_GT_SRIOV_VF_TYPES_H_
 
+#include <linux/rwsem.h>
 #include <linux/types.h>
 #include "xe_uc_fw_types.h"
 
@@ -25,6 +26,8 @@ struct xe_gt_sriov_vf_selfconfig {
 	u16 num_ctxs;
 	/** @num_dbs: assigned number of GuC doorbells IDs. */
 	u16 num_dbs;
+	/** @lock: lock for protecting access to all selfconfig fields. */
+	struct rw_semaphore lock;
 };
 
 /**
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -0
index cdd9f8e78b2a..d6e2ed9b9bbc 100644

drivers/gpu/drm/xe/xe_sriov_vf.c static void vf_migration_init_early(struct xe_device *xe) · line 204

Quoted 8 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +5 -0
@@ -197,6 +197,12 @@ static void vf_migration_init_early(struct xe_device *xe)
  */
 void xe_sriov_vf_init_early(struct xe_device *xe)
 {
+	struct xe_gt *gt;
+	unsigned int id;
+
+	for_each_gt(gt, xe, id)
+		init_rwsem(&gt->sriov.vf.self_config.lock);

this is a wrong place - VF's per-GT data shall be initialized in the per-GT code somewhere in xe_gt_sriov_vf.c

Quoted 4 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
+
 	vf_migration_init_early(xe);
 }
 
author maintainer
#3 lore raw

Re: [PATCH v2 01/34] drm/xe/vf: Lock querying GGTT config during driver init

Quoted 2 lines · Michal Wajdeczko

I'll let Tomasz respond to most of the commennts as it is his patch but replying to last comment.

Quoted 298 lines · Matthew Brost
Quoted 17 lines

From: Tomasz Lis <[email protected]>

Protect access to GGTT config as this is non-static information.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 96 ++++++++++++++++++-----
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  3 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          |  6 ++
 3 files changed, 84 insertions(+), 21 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index 0461d5513487..016c867e5e2b 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt) · line 443

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -440,18 +440,21 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);

do we need to protect also VF2GUC communication ..

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt) · line 452

Quoted 11 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -2
+
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_SIZE_KEY, &size);
 	if (unlikely(err))
-		return err;
+		goto out;
 

.. or just RW access to the data itself from here?

Quoted 100 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +28 -10
 	if (config->ggtt_size && config->ggtt_size != size) {
 		xe_gt_sriov_err(gt, "Unexpected GGTT reassignment: %lluK != %lluK\n",
 				size / SZ_1K, config->ggtt_size / SZ_1K);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
@@ -460,8 +463,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 	config->ggtt_shift = start - (s64)config->ggtt_base;
 	config->ggtt_base = start;
 	config->ggtt_size = size;
+	err = config->ggtt_size ? 0 : -ENODATA;
 
-	return config->ggtt_size ? 0 : -ENODATA;
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static int vf_get_lmem_info(struct xe_gt *gt)
@@ -474,22 +480,28 @@ static int vf_get_lmem_info(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	if (config->lmem_size && config->lmem_size != size) {
 		xe_gt_sriov_err(gt, "Unexpected LMEM reassignment: %lluM != %lluM\n",
 				size / SZ_1M, config->lmem_size / SZ_1M);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	string_get_size(size, 1, STRING_UNITS_2, size_str, sizeof(size_str));
 	xe_gt_sriov_dbg_verbose(gt, "LMEM %lluM %s\n", size / SZ_1M, size_str);
 
 	config->lmem_size = size;
+	err = config->lmem_size ? 0 : -ENODATA;
 
-	return config->lmem_size ? 0 : -ENODATA;
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static int vf_get_submission_cfg(struct xe_gt *gt)
@@ -501,23 +513,27 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
 	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_DOORBELLS_KEY, &num_dbs);
 	if (unlikely(err))
-		return err;
+		goto out;
 
 	if (config->num_ctxs && config->num_ctxs != num_ctxs) {
 		xe_gt_sriov_err(gt, "Unexpected CTXs reassignment: %u != %u\n",
 				num_ctxs, config->num_ctxs);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 	if (config->num_dbs && config->num_dbs != num_dbs) {
 		xe_gt_sriov_err(gt, "Unexpected DBs reassignment: %u != %u\n",
 				num_dbs, config->num_dbs);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
 	}
 
 	xe_gt_sriov_dbg_verbose(gt, "CTXs %u DBs %u\n", num_ctxs, num_dbs);
@@ -525,7 +541,11 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 	config->num_ctxs = num_ctxs;
 	config->num_dbs = num_dbs;
 
-	return config->num_ctxs ? 0 : -ENODATA;
+	err = config->num_ctxs ? 0 : -ENODATA;
+
+out:
+	up_write(&config->lock);
+	return err;
 }
 
 static void vf_cache_gmdid(struct xe_gt *gt)

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 584

Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -1
@@ -579,11 +599,18 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
  */
 u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u16 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.num_ctxs);

the purpose of this assert was to make sure no part of the VF driver is trying to get num_ctxs before we finish querying it from the GuC ..

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 609

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
-	return gt->sriov.vf.self_config.num_ctxs;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->num_ctxs);

.. instead of moving this assert in the middle of the lock ..

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 612

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+	val = config->num_ctxs;
+	up_read(&config->lock);
+

.. for code clarity you can add assert here as:

	xe_gt_assert(gt, val);
Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+	return val;
 }
 
 /**

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt) · line 601

Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -1
@@ -596,11 +623,18 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.lmem_size);

ditto

Quoted 115 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +31 -6
-	return gt->sriov.vf.self_config.lmem_size;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->lmem_size);
+	val = config->lmem_size;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -613,11 +647,17 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.ggtt_size);
 
-	return gt->sriov.vf.self_config.ggtt_size;
+	down_read(&config->lock);
+	val = config->ggtt_size;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -630,11 +670,18 @@ u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
  */
 u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
 {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.ggtt_size);
 
-	return gt->sriov.vf.self_config.ggtt_base;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->ggtt_size);
+	val = config->ggtt_base;
+	up_read(&config->lock);
+
+	return val;
 }
 
 /**
@@ -648,11 +695,16 @@ u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
 s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)
 {
 	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	s64 val;
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, xe_gt_is_main_type(gt));
 
-	return config->ggtt_shift;
+	down_read(&config->lock);
+	val = config->ggtt_shift;
+	up_read(&config->lock);
+
+	return val;
 }
 
 static int relay_action_handshake(struct xe_gt *gt, u32 *major, u32 *minor)
@@ -1044,6 +1096,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_read(&config->lock);
 	drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
 		   config->ggtt_base,
 		   config->ggtt_base + config->ggtt_size - 1);
@@ -1060,6 +1113,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
 	drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
 	drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
+	up_read(&config->lock);
 }
 
 /**
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -0
index 298dedf4b009..d95857bd789b 100644
@@ -6,6 +6,7 @@
 #ifndef _XE_GT_SRIOV_VF_TYPES_H_
 #define _XE_GT_SRIOV_VF_TYPES_H_
 
+#include <linux/rwsem.h>
 #include <linux/types.h>
 #include "xe_uc_fw_types.h"
 
@@ -25,6 +26,8 @@ struct xe_gt_sriov_vf_selfconfig {
 	u16 num_ctxs;
 	/** @num_dbs: assigned number of GuC doorbells IDs. */
 	u16 num_dbs;
+	/** @lock: lock for protecting access to all selfconfig fields. */
+	struct rw_semaphore lock;
 };
 
 /**
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -0
index cdd9f8e78b2a..d6e2ed9b9bbc 100644
@@ -197,6 +197,12 @@ static void vf_migration_init_early(struct xe_device *xe)
  */
 void xe_sriov_vf_init_early(struct xe_device *xe)
 {
Quoted 9 lines

drivers/gpu/drm/xe/xe_sriov_vf.c static void vf_migration_init_early(struct xe_device *xe) · line 204

Quoted 5 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +5 -0
+	struct xe_gt *gt;
+	unsigned int id;
+
+	for_each_gt(gt, xe, id)
+		init_rwsem(&gt->sriov.vf.self_config.lock);

this is a wrong place - VF's per-GT data shall be initialized in the per-GT code somewhere in xe_gt_sriov_vf.c

Yes, it is moved to the proper place in [1].

Matt

[1] https://patchwork.freedesktop.org/patch/676383/?series=154627&rev=2

Quoted 5 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
+
 	vf_migration_init_early(xe);
 }
 
#4 lore raw

Re: [PATCH v2 01/34] drm/xe/vf: Lock querying GGTT config during driver init

Quoted 4 lines · Matthew Brost
Quoted 1 line · Michal Wajdeczko

I'll let Tomasz respond to most of the commennts as it is his patch but replying to last comment.

ok, will answer.

Quoted 30 lines
Quoted 30 lines · Matthew Brost
Quoted 17 lines

From: Tomasz Lis<[email protected]>

Protect access to GGTT config as this is non-static information.

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 96 ++++++++++++++++++-----
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  3 +
  drivers/gpu/drm/xe/xe_sriov_vf.c          |  6 ++
  3 files changed, 84 insertions(+), 21 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index 0461d5513487..016c867e5e2b 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt) · line 443

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -440,18 +440,21 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);

do we need to protect also VF2GUC communication ..

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
+
  	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
  	if (unlikely(err))
-		return err;
+		goto out;
 
Quoted 6 lines
Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt) · line 452

Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -1
  	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_SIZE_KEY, &size);
  	if (unlikely(err))
-		return err;
+		goto out;
 

.. or just RW access to the data itself from here?

The protection must start before we receive GGTT start. It would make little sense to protect just manipulation of data which we've already received.

The post-migration recovery must have a clear state: either we got the GGTT already, or not. It cannot have a state "it's not written everywhere yet, but GuC may have sent it already, who knows".

The GGTT address must be protected from before the moment GuC reads it in order to write to VF G2H CTB.

Quoted 99 lines
Quoted 99 lines
Quoted 99 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +28 -10
  	if (config->ggtt_size && config->ggtt_size != size) {
  		xe_gt_sriov_err(gt, "Unexpected GGTT reassignment: %lluK != %lluK\n",
  				size / SZ_1K, config->ggtt_size / SZ_1K);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
  	}
 
  	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
@@ -460,8 +463,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
  	config->ggtt_shift = start - (s64)config->ggtt_base;
  	config->ggtt_base = start;
  	config->ggtt_size = size;
+	err = config->ggtt_size ? 0 : -ENODATA;
 
-	return config->ggtt_size ? 0 : -ENODATA;
+out:
+	up_write(&config->lock);
+	return err;
  }
 
  static int vf_get_lmem_info(struct xe_gt *gt)
@@ -474,22 +480,28 @@ static int vf_get_lmem_info(struct xe_gt *gt)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
  	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
  	if (unlikely(err))
-		return err;
+		goto out;
 
  	if (config->lmem_size && config->lmem_size != size) {
  		xe_gt_sriov_err(gt, "Unexpected LMEM reassignment: %lluM != %lluM\n",
  				size / SZ_1M, config->lmem_size / SZ_1M);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
  	}
 
  	string_get_size(size, 1, STRING_UNITS_2, size_str, sizeof(size_str));
  	xe_gt_sriov_dbg_verbose(gt, "LMEM %lluM %s\n", size / SZ_1M, size_str);
 
  	config->lmem_size = size;
+	err = config->lmem_size ? 0 : -ENODATA;
 
-	return config->lmem_size ? 0 : -ENODATA;
+out:
+	up_write(&config->lock);
+	return err;
  }
 
  static int vf_get_submission_cfg(struct xe_gt *gt)
@@ -501,23 +513,27 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_write(&config->lock);
+
  	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
  	if (unlikely(err))
-		return err;
+		goto out;
 
  	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_DOORBELLS_KEY, &num_dbs);
  	if (unlikely(err))
-		return err;
+		goto out;
 
  	if (config->num_ctxs && config->num_ctxs != num_ctxs) {
  		xe_gt_sriov_err(gt, "Unexpected CTXs reassignment: %u != %u\n",
  				num_ctxs, config->num_ctxs);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
  	}
  	if (config->num_dbs && config->num_dbs != num_dbs) {
  		xe_gt_sriov_err(gt, "Unexpected DBs reassignment: %u != %u\n",
  				num_dbs, config->num_dbs);
-		return -EREMCHG;
+		err = -EREMCHG;
+		goto out;
  	}
 
  	xe_gt_sriov_dbg_verbose(gt, "CTXs %u DBs %u\n", num_ctxs, num_dbs);
@@ -525,7 +541,11 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
  	config->num_ctxs = num_ctxs;
  	config->num_dbs = num_dbs;
 
-	return config->num_ctxs ? 0 : -ENODATA;
+	err = config->num_ctxs ? 0 : -ENODATA;
+
+out:
+	up_write(&config->lock);
+	return err;
  }
 
Quoted 13 lines
Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
  static void vf_cache_gmdid(struct xe_gt *gt)
@@ -579,11 +599,18 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
   */
  u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
  {
Quoted 8 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 584

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -1
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u16 val;
+
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.num_ctxs);

the purpose of this assert was to make sure no part of the VF driver is trying to get num_ctxs before we finish querying it from the GuC ..

We now know this will never happen _before_we_finish_querying_, as there's a lock. Read side will wait for the write side to finish.

So unless you're worried about a situation before we _start_ querying, it's ok to even remove it completely.

Quoted 7 lines
Quoted 7 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 609

Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
-	return gt->sriov.vf.self_config.num_ctxs;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->num_ctxs);

.. instead of moving this assert in the middle of the lock ..

Quoted 6 lines
Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_config(struct xe_gt *gt) · line 612

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+	val = config->num_ctxs;
+	up_read(&config->lock);
+

.. for code clarity you can add assert here as:

	xe_gt_assert(gt, val);

sure, can do that.

Quoted 15 lines
Quoted 15 lines
Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+	return val;
  }
 
  /**

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt) · line 601

Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -1
@@ -596,11 +623,18 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
   */
  u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
  {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.lmem_size);

ditto

ack, though whether this stays at all, depends on further discussion in v3 of this series.

-Tomasz

Quoted 134 lines
Quoted 107 lines
Quoted 107 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +31 -6
-	return gt->sriov.vf.self_config.lmem_size;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->lmem_size);
+	val = config->lmem_size;
+	up_read(&config->lock);
+
+	return val;
  }
 
  /**
@@ -613,11 +647,17 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
   */
  u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
  {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.ggtt_size);
 
-	return gt->sriov.vf.self_config.ggtt_size;
+	down_read(&config->lock);
+	val = config->ggtt_size;
+	up_read(&config->lock);
+
+	return val;
  }
 
  /**
@@ -630,11 +670,18 @@ u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
   */
  u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
  {
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	u64 val;
+
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	xe_gt_assert(gt, gt->sriov.vf.self_config.ggtt_size);
 
-	return gt->sriov.vf.self_config.ggtt_base;
+	down_read(&config->lock);
+	xe_gt_assert(gt, config->ggtt_size);
+	val = config->ggtt_base;
+	up_read(&config->lock);
+
+	return val;
  }
 
  /**
@@ -648,11 +695,16 @@ u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
  s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)
  {
  	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	s64 val;
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, xe_gt_is_main_type(gt));
 
-	return config->ggtt_shift;
+	down_read(&config->lock);
+	val = config->ggtt_shift;
+	up_read(&config->lock);
+
+	return val;
  }
 
  static int relay_action_handshake(struct xe_gt *gt, u32 *major, u32 *minor)
@@ -1044,6 +1096,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
+	down_read(&config->lock);
  	drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
  		   config->ggtt_base,
  		   config->ggtt_base + config->ggtt_size - 1);
@@ -1060,6 +1113,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
  	drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
  	drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
+	up_read(&config->lock);
  }
 
  /**
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -0
index 298dedf4b009..d95857bd789b 100644
@@ -6,6 +6,7 @@
  #ifndef _XE_GT_SRIOV_VF_TYPES_H_
  #define _XE_GT_SRIOV_VF_TYPES_H_
 
+#include <linux/rwsem.h>
  #include <linux/types.h>
  #include "xe_uc_fw_types.h"
 
@@ -25,6 +26,8 @@ struct xe_gt_sriov_vf_selfconfig {
  	u16 num_ctxs;
  	/** @num_dbs: assigned number of GuC doorbells IDs. */
  	u16 num_dbs;
+	/** @lock: lock for protecting access to all selfconfig fields. */
+	struct rw_semaphore lock;
  };
 
Quoted 17 lines
Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +0 -0
  /**
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -0
index cdd9f8e78b2a..d6e2ed9b9bbc 100644

drivers/gpu/drm/xe/xe_sriov_vf.c static void vf_migration_init_early(struct xe_device *xe) · line 204

Quoted 8 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +5 -0
@@ -197,6 +197,12 @@ static void vf_migration_init_early(struct xe_device *xe)
   */
  void xe_sriov_vf_init_early(struct xe_device *xe)
  {
+	struct xe_gt *gt;
+	unsigned int id;
+
+	for_each_gt(gt, xe, id)
+		init_rwsem(&gt->sriov.vf.self_config.lock);

this is a wrong place - VF's per-GT data shall be initialized in the per-GT code somewhere in xe_gt_sriov_vf.c

Yes, it is moved to the proper place in [1].

Matt

[1]https://patchwork.freedesktop.org/patch/676383/?series=154627&rev=2

Quoted 4 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
+
  	vf_migration_init_early(xe);
  }
 
author maintainer
#5 lore raw

PATCH v2 2/34 Revert "drm/xe/vf: Rebase exec queue parallel commands during migration recovery"

In reply to Matthew Brost

This reverts commit ba180a362128cb71d16c3f0ce6645448011d2607.

---
 drivers/gpu/drm/xe/abi/guc_actions_abi.h |  8 ----
 drivers/gpu/drm/xe/xe_guc_submit.c       | 54 ------------------------
 2 files changed, 62 deletions(-)
drivers/gpu/drm/xe/abi/guc_actions_abi.h +0 -8
index 31090c69dfbe..47756e4674a1 100644
@@ -196,14 +196,6 @@ enum xe_guc_register_context_multi_lrc_param_offsets {
 	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_MSG_MIN_LEN = 11,
 };
 
-enum xe_guc_context_wq_item_offsets {
-	XE_GUC_CONTEXT_WQ_HEADER_DATA_0_TYPE_LEN = 0,
-	XE_GUC_CONTEXT_WQ_EL_INFO_DATA_1_CTX_DESC_LOW,
-	XE_GUC_CONTEXT_WQ_EL_INFO_DATA_2_GUCCTX_RINGTAIL_FREEZEPOCS,
-	XE_GUC_CONTEXT_WQ_EL_INFO_DATA_3_WI_FENCE_ID,
-	XE_GUC_CONTEXT_WQ_EL_CHILD_LIST_DATA_4_RINGTAIL,
-};
-
 enum xe_guc_report_status {
 	XE_GUC_REPORT_STATUS_UNKNOWN = 0x0,
 	XE_GUC_REPORT_STATUS_ACKED = 0x1,
drivers/gpu/drm/xe/xe_guc_submit.c +0 -54

78 lines not on this page · raw message

Signature · 1 line
2.34.1
#6 lore raw

Re: [PATCH v2 02/34] Revert "drm/xe/vf: Rebase exec queue parallel commands during migration recovery"

Quoted 1 line · Matthew Brost

This reverts commit ba180a362128cb71d16c3f0ce6645448011d2607.

we still need the "why" part here, at least something like:

"Due to change in the VF migration recovery design this code is not needed any more."

Quoted 6 lines
---
 drivers/gpu/drm/xe/abi/guc_actions_abi.h |  8 ----
 drivers/gpu/drm/xe/xe_guc_submit.c       | 54 ------------------------
 2 files changed, 62 deletions(-)
author maintainer
#7 lore raw

Re: [PATCH v2 02/34] Revert "drm/xe/vf: Rebase exec queue parallel commands during migration recovery"

Quoted 10 lines · Michal Wajdeczko
Quoted 1 line · Matthew Brost

This reverts commit ba180a362128cb71d16c3f0ce6645448011d2607.

we still need the "why" part here, at least something like:

"Due to change in the VF migration recovery design this code is not needed any more."

Lucas also mentioned this me, I'll add explaination in commit messages for the reverts.

Matt

Quoted 6 lines
Quoted 6 lines
---
 drivers/gpu/drm/xe/abi/guc_actions_abi.h |  8 ----
 drivers/gpu/drm/xe/xe_guc_submit.c       | 54 ------------------------
 2 files changed, 62 deletions(-)
author maintainer
#8 lore raw

PATCH v2 3/34 Revert "drm/xe/vf: Post migration, repopulate ring area for pending request"

In reply to Matthew Brost

This reverts commit a0dda25d24e636df5c30a9370464b7cebc709faf.

---
 drivers/gpu/drm/xe/xe_exec_queue.c | 24 ------------------------
 drivers/gpu/drm/xe/xe_exec_queue.h |  3 +--
 drivers/gpu/drm/xe/xe_guc_submit.c | 24 ------------------------
 drivers/gpu/drm/xe/xe_guc_submit.h |  2 --
 drivers/gpu/drm/xe/xe_sriov_vf.c   |  1 -
 5 files changed, 1 insertion(+), 53 deletions(-)
drivers/gpu/drm/xe/xe_exec_queue.c +0 -24
index 37b2b93b73d6..6bfaca424ca3 100644
@@ -1123,27 +1123,3 @@ int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch)
 
 	return err;
 }
-
-/**
- * xe_exec_queue_jobs_ring_restore - Re-emit ring commands of requests pending on given queue.
- * @q: the &xe_exec_queue struct instance
- */
-void xe_exec_queue_jobs_ring_restore(struct xe_exec_queue *q)
-{
-	struct xe_gpu_scheduler *sched = &q->guc->sched;
-	struct xe_sched_job *job;
-
-	/*
-	 * This routine is used within VF migration recovery. This means
-	 * using the lock here introduces a restriction: we cannot wait
-	 * for any GFX HW response while the lock is taken.
-	 */
-	spin_lock(&sched->base.job_list_lock);
-	list_for_each_entry(job, &sched->base.pending_list, drm.list) {
-		if (xe_sched_job_is_error(job))
-			continue;
-
-		q->ring_ops->emit_job(job);
-	}
-	spin_unlock(&sched->base.job_list_lock);
-}
drivers/gpu/drm/xe/xe_exec_queue.h +1 -2
index 15ec852e7f7e..8821ceb838d0 100644
@@ -92,7 +92,6 @@ void xe_exec_queue_update_run_ticks(struct xe_exec_queue *q);
 
 int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch);
 
-void xe_exec_queue_jobs_ring_restore(struct xe_exec_queue *q);
-
 struct xe_lrc *xe_exec_queue_lrc(struct xe_exec_queue *q);
+
 #endif
drivers/gpu/drm/xe/xe_guc_submit.c +0 -24
index 3ac0950f55be..16f78376f196 100644
@@ -845,30 +845,6 @@ guc_exec_queue_run_job(struct drm_sched_job *drm_job)
 	return fence;
 }
 
-/**
- * xe_guc_jobs_ring_rebase - Re-emit ring commands of requests pending
- * on all queues under a guc.
- * @guc: the &xe_guc struct instance
- */
-void xe_guc_jobs_ring_rebase(struct xe_guc *guc)
-{
-	struct xe_exec_queue *q;
-	unsigned long index;
-
-	/*
-	 * This routine is used within VF migration recovery. This means
-	 * using the lock here introduces a restriction: we cannot wait
-	 * for any GFX HW response while the lock is taken.
-	 */
-	mutex_lock(&guc->submission_state.lock);
-	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) {
-		if (exec_queue_killed_or_banned_or_wedged(q))
-			continue;
-		xe_exec_queue_jobs_ring_restore(q);
-	}
-	mutex_unlock(&guc->submission_state.lock);
-}
-
 static void guc_exec_queue_free_job(struct drm_sched_job *drm_job)
 {
 	struct xe_sched_job *job = to_xe_sched_job(drm_job);
drivers/gpu/drm/xe/xe_guc_submit.h +0 -2
index 78c3f07e31a0..5b4a0a6fd818 100644
@@ -36,8 +36,6 @@ int xe_guc_exec_queue_memory_cat_error_handler(struct xe_guc *guc, u32 *msg,
 int xe_guc_exec_queue_reset_failure_handler(struct xe_guc *guc, u32 *msg, u32 len);
 int xe_guc_error_capture_handler(struct xe_guc *guc, u32 *msg, u32 len);
 
-void xe_guc_jobs_ring_rebase(struct xe_guc *guc);
-
 struct xe_guc_submit_exec_queue_snapshot *
 xe_guc_exec_queue_snapshot_capture(struct xe_exec_queue *q);
 void
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -1
index d6e2ed9b9bbc..0581b881b628 100644
@@ -340,7 +340,6 @@ static int gt_vf_post_migration_fixups(struct xe_gt *gt)
 		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
 		if (err)
 			goto out;
-		xe_guc_jobs_ring_rebase(&gt->uc.guc);
 		xe_guc_ct_fixup_messages_with_ggtt(&gt->uc.guc.ct, shift);
 	}
 
Signature · 1 line
2.34.1
author maintainer
#9 lore raw

PATCH v2 4/34 Revert "drm/xe/vf: Fixup CTB send buffer messages after migration"

In reply to Matthew Brost

This reverts commit cef88d1265cac7d415606af73ba58926fd3cd8b7.

---
 drivers/gpu/drm/xe/xe_guc_ct.c   | 183 -------------------------------
 drivers/gpu/drm/xe/xe_guc_ct.h   |   2 -
 drivers/gpu/drm/xe/xe_map.h      |  18 ---
 drivers/gpu/drm/xe/xe_sriov_vf.c |   2 -
 4 files changed, 205 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +0 -183
index 18f6327bf552..47079ab9922c 100644
@@ -25,7 +25,6 @@
 #include "xe_gt_printk.h"
 #include "xe_gt_sriov_pf_control.h"
 #include "xe_gt_sriov_pf_monitor.h"
-#include "xe_gt_sriov_printk.h"
 #include "xe_guc.h"
 #include "xe_guc_log.h"
 #include "xe_guc_relay.h"
@@ -93,8 +92,6 @@ struct g2h_fence {
 	bool done;
 };
 
-#define make_u64(hi, lo) ((u64)((u64)(u32)(hi) << 32 | (u32)(lo)))
-
 static void g2h_fence_init(struct g2h_fence *g2h_fence, u32 *response_buffer)
 {
 	memset(g2h_fence, 0, sizeof(*g2h_fence));
@@ -1793,186 +1790,6 @@ static void g2h_worker_func(struct work_struct *w)
 	receive_g2h(ct);
 }
 
-static void xe_fixup_u64_in_cmds(struct xe_device *xe, struct iosys_map *cmds,
-				 u32 size, u32 idx, s64 shift)
-{
-	u32 hi, lo;
-	u64 offset;
-
-	lo = xe_map_rd_ring_u32(xe, cmds, idx, size);
-	hi = xe_map_rd_ring_u32(xe, cmds, idx + 1, size);
-	offset = make_u64(hi, lo);
-	offset += shift;
-	lo = lower_32_bits(offset);
-	hi = upper_32_bits(offset);
-	xe_map_wr_ring_u32(xe, cmds, idx, size, lo);
-	xe_map_wr_ring_u32(xe, cmds, idx + 1, size, hi);
-}
-
-/*
- * Shift any GGTT addresses within a single message left within CTB from
- * before post-migration recovery.
- * @ct: pointer to CT struct of the target GuC
- * @cmds: iomap buffer containing CT messages
- * @head: start of the target message within the buffer
- * @len: length of the target message
- * @size: size of the commands buffer
- * @shift: the address shift to be added to each GGTT reference
- * Return: true if the message was fixed or needed no fixups, false on failure
- */
-static bool ct_fixup_ggtt_in_message(struct xe_guc_ct *ct,
-				     struct iosys_map *cmds, u32 head,
-				     u32 len, u32 size, s64 shift)
-{
-	struct xe_gt *gt = ct_to_gt(ct);
-	struct xe_device *xe = ct_to_xe(ct);
-	u32 msg[GUC_HXG_MSG_MIN_LEN];
-	u32 action, i, n;
-
-	xe_gt_assert(gt, len >= GUC_HXG_MSG_MIN_LEN);
-
-	msg[0] = xe_map_rd_ring_u32(xe, cmds, head, size);
-	action = FIELD_GET(GUC_HXG_REQUEST_MSG_0_ACTION, msg[0]);
-
-	xe_gt_sriov_dbg_verbose(gt, "fixing H2G %#x\n", action);
-
-	switch (action) {
-	case XE_GUC_ACTION_REGISTER_CONTEXT:
-		if (len != XE_GUC_REGISTER_CONTEXT_MSG_LEN)
-			goto err_len;
-		xe_fixup_u64_in_cmds(xe, cmds, size, head +
-				     XE_GUC_REGISTER_CONTEXT_DATA_5_WQ_DESC_ADDR_LOWER,
-				     shift);
-		xe_fixup_u64_in_cmds(xe, cmds, size, head +
-				     XE_GUC_REGISTER_CONTEXT_DATA_7_WQ_BUF_BASE_LOWER,
-				     shift);
-		xe_fixup_u64_in_cmds(xe, cmds, size, head +
-				     XE_GUC_REGISTER_CONTEXT_DATA_10_HW_LRC_ADDR, shift);
-		break;
-	case XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC:
-		if (len < XE_GUC_REGISTER_CONTEXT_MULTI_LRC_MSG_MIN_LEN)
-			goto err_len;
-		n = xe_map_rd_ring_u32(xe, cmds, head +
-				       XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_10_NUM_CTXS, size);
-		if (len != XE_GUC_REGISTER_CONTEXT_MULTI_LRC_MSG_MIN_LEN + 2 * n)
-			goto err_len;
-		xe_fixup_u64_in_cmds(xe, cmds, size, head +
-				     XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_5_WQ_DESC_ADDR_LOWER,
-				     shift);
-		xe_fixup_u64_in_cmds(xe, cmds, size, head +
-				     XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_7_WQ_BUF_BASE_LOWER,
-				     shift);
-		for (i = 0; i < n; i++)
-			xe_fixup_u64_in_cmds(xe, cmds, size, head +
-					     XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_11_HW_LRC_ADDR
-					     + 2 * i, shift);
-		break;
-	default:
-		break;
-	}
-	return true;
-
-err_len:
-	xe_gt_err(gt, "Skipped G2G %#x message fixups, unexpected length (%u)\n", action, len);
-	return false;
-}
-
-/*
- * Apply fixups to the next outgoing CT message within given CTB
- * @ct: the &xe_guc_ct struct instance representing the target GuC
- * @h2g: the &guc_ctb struct instance of the target buffer
- * @shift: shift to be added to all GGTT addresses within the CTB
- * @mhead: pointer to an integer storing message start position; the
- *   position is changed to next message before this function return
- * @avail: size of the area available for parsing, that is length
- *   of all remaining messages stored within the CTB
- * Return: size of the area available for parsing after one message
- *   has been parsed, that is length remaining from the updated mhead
- */
-static int ct_fixup_ggtt_in_buffer(struct xe_guc_ct *ct, struct guc_ctb *h2g,
-				   s64 shift, u32 *mhead, s32 avail)
-{
-	struct xe_gt *gt = ct_to_gt(ct);
-	struct xe_device *xe = ct_to_xe(ct);
-	u32 msg[GUC_HXG_MSG_MIN_LEN];
-	u32 size = h2g->info.size;
-	u32 head = *mhead;
-	u32 len;
-
-	xe_gt_assert(gt, avail >= (s32)GUC_CTB_MSG_MIN_LEN);
-
-	/* Read header */
-	msg[0] = xe_map_rd_ring_u32(xe, &h2g->cmds, head, size);
-	len = FIELD_GET(GUC_CTB_MSG_0_NUM_DWORDS, msg[0]) + GUC_CTB_MSG_MIN_LEN;
-
-	if (unlikely(len > (u32)avail)) {
-		xe_gt_err(gt, "H2G channel broken on read, avail=%d, len=%d, fixups skipped\n",
-			  avail, len);
-		return 0;
-	}
-
-	head = (head + GUC_CTB_MSG_MIN_LEN) % size;
-	if (!ct_fixup_ggtt_in_message(ct, &h2g->cmds, head, msg_len_to_hxg_len(len), size, shift))
-		return 0;
-	*mhead = (head + msg_len_to_hxg_len(len)) % size;
-
-	return avail - len;
-}
-
-/**
- * xe_guc_ct_fixup_messages_with_ggtt - Fixup any pending H2G CTB messages
- * @ct: pointer to CT struct of the target GuC
- * @ggtt_shift: shift to be added to all GGTT addresses within the CTB
- *
- * Messages in GuC to Host CTB are owned by GuC and any fixups in them
- * are made by GuC. But content of the Host to GuC CTB is owned by the
- * KMD, so fixups to GGTT references in any pending messages need to be
- * applied here.
- * This function updates GGTT offsets in payloads of pending H2G CTB
- * messages (messages which were not consumed by GuC before the VF got
- * paused).
- */
-void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift)
-{
-	struct guc_ctb *h2g = &ct->ctbs.h2g;
-	struct xe_guc *guc = ct_to_guc(ct);
-	struct xe_gt *gt = guc_to_gt(guc);
-	u32 head, tail, size;
-	s32 avail;
-
-	if (unlikely(h2g->info.broken))
-		return;
-
-	h2g->info.head = desc_read(ct_to_xe(ct), h2g, head);
-	head = h2g->info.head;
-	tail = READ_ONCE(h2g->info.tail);
-	size = h2g->info.size;
-
-	if (unlikely(head > size))
-		goto corrupted;
-
-	if (unlikely(tail >= size))
-		goto corrupted;
-
-	avail = tail - head;
-
-	/* beware of buffer wrap case */
-	if (unlikely(avail < 0))
-		avail += size;
-	xe_gt_dbg(gt, "available %d (%u:%u:%u)\n", avail, head, tail, size);
-	xe_gt_assert(gt, avail >= 0);
-
-	while (avail > 0)
-		avail = ct_fixup_ggtt_in_buffer(ct, h2g, ggtt_shift, &head, avail);
-
-	return;
-
-corrupted:
-	xe_gt_err(gt, "Corrupted H2G descriptor head=%u tail=%u size=%u, fixups not applied\n",
-		  head, tail, size);
-	h2g->info.broken = true;
-}
-
 static struct xe_guc_ct_snapshot *guc_ct_snapshot_alloc(struct xe_guc_ct *ct, bool atomic,
 							bool want_ctb)
 {
drivers/gpu/drm/xe/xe_guc_ct.h +0 -2
index cf41210ab30a..d6c81325a76c 100644
@@ -24,8 +24,6 @@ void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot, struct drm_pr
 void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot);
 void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p, bool want_ctb);
 
-void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift);
-
 static inline bool xe_guc_ct_initialized(struct xe_guc_ct *ct)
 {
 	return ct->state != XE_GUC_CT_STATE_NOT_INITIALIZED;
drivers/gpu/drm/xe/xe_map.h +0 -18
index 8d67f6ba2d95..f62e0c8b67ab 100644
@@ -78,24 +78,6 @@ static inline void xe_map_write32(struct xe_device *xe, struct iosys_map *map,
 	iosys_map_wr(map__, offset__, type__, val__);			\
 })
 
-#define xe_map_rd_array(xe__, map__, index__, type__) \
-	xe_map_rd(xe__, map__, (index__) * sizeof(type__), type__)
-
-#define xe_map_wr_array(xe__, map__, index__, type__, val__) \
-	xe_map_wr(xe__, map__, (index__) * sizeof(type__), type__, val__)
-
-#define xe_map_rd_array_u32(xe__, map__, index__) \
-	xe_map_rd_array(xe__, map__, index__, u32)
-
-#define xe_map_wr_array_u32(xe__, map__, index__, val__) \
-	xe_map_wr_array(xe__, map__, index__, u32, val__)
-
-#define xe_map_rd_ring_u32(xe__, map__, index__, size__) \
-	xe_map_rd_array_u32(xe__, map__, (index__) % (size__))
-
-#define xe_map_wr_ring_u32(xe__, map__, index__, size__, val__) \
-	xe_map_wr_array_u32(xe__, map__, (index__) % (size__), val__)
-
 #define xe_map_rd_field(xe__, map__, struct_offset__, struct_type__, field__) ({	\
 	struct xe_device *__xe = xe__;					\
 	xe_device_assert_mem_access(__xe);				\
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -2
index 0581b881b628..da064a1e7419 100644
@@ -12,7 +12,6 @@
 #include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
-#include "xe_guc_ct.h"
 #include "xe_guc_submit.h"
 #include "xe_irq.h"
 #include "xe_lrc.h"
@@ -340,7 +339,6 @@ static int gt_vf_post_migration_fixups(struct xe_gt *gt)
 		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
 		if (err)
 			goto out;
-		xe_guc_ct_fixup_messages_with_ggtt(&gt->uc.guc.ct, shift);
 	}
 
 out:
Signature · 1 line
2.34.1
author maintainer
#10 lore raw

PATCH v2 5/34 drm/xe: Save off position in ring in which a job was programmed

In reply to Matthew Brost

VF post-migration recovery needs to modify the ring with updated GGTT addresses for pending jobs. Save off position in ring in which a job was programmed to facilitate.

v4:
- s/VF resume/VF post-migration recovery (Tomasz)

---
 drivers/gpu/drm/xe/xe_ring_ops.c        | 23 +++++++++++++++++++----
 drivers/gpu/drm/xe/xe_sched_job_types.h |  5 +++++
 2 files changed, 24 insertions(+), 4 deletions(-)
drivers/gpu/drm/xe/xe_ring_ops.c +19 -4
index d71837773d6c..ac0c6dcffe15 100644
@@ -245,12 +245,14 @@ static int emit_copy_timestamp(struct xe_lrc *lrc, u32 *dw, int i)
 
 /* for engines that don't require any special HW handling (no EUs, no aux inval, etc) */
 static void __emit_job_gen12_simple(struct xe_sched_job *job, struct xe_lrc *lrc,
-				    u64 batch_addr, u32 seqno)
+				    u64 batch_addr, u32 *head, u32 seqno)
 {
 	u32 dw[MAX_JOB_SIZE_DW], i = 0;
 	u32 ppgtt_flag = get_ppgtt_flag(job);
 	struct xe_gt *gt = job->q->gt;
 
+	*head = lrc->ring.tail;
+
 	i = emit_copy_timestamp(lrc, dw, i);
 
 	if (job->ring_ops_flush_tlb) {
@@ -296,7 +298,7 @@ static bool has_aux_ccs(struct xe_device *xe)
 }
 
 static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc,
-				   u64 batch_addr, u32 seqno)
+				   u64 batch_addr, u32 *head, u32 seqno)
 {
 	u32 dw[MAX_JOB_SIZE_DW], i = 0;
 	u32 ppgtt_flag = get_ppgtt_flag(job);
@@ -304,6 +306,8 @@ static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc,
 	struct xe_device *xe = gt_to_xe(gt);
 	bool decode = job->q->class == XE_ENGINE_CLASS_VIDEO_DECODE;
 
+	*head = lrc->ring.tail;
+
 	i = emit_copy_timestamp(lrc, dw, i);
 
 	dw[i++] = preparser_disable(true);
@@ -346,7 +350,8 @@ static void __emit_job_gen12_video(struct xe_sched_job *job, struct xe_lrc *lrc,
 
 static void __emit_job_gen12_render_compute(struct xe_sched_job *job,
 					    struct xe_lrc *lrc,
-					    u64 batch_addr, u32 seqno)
+					    u64 batch_addr, u32 *head,
+					    u32 seqno)
 {
 	u32 dw[MAX_JOB_SIZE_DW], i = 0;
 	u32 ppgtt_flag = get_ppgtt_flag(job);
@@ -355,6 +360,8 @@ static void __emit_job_gen12_render_compute(struct xe_sched_job *job,
 	bool lacks_render = !(gt->info.engine_mask & XE_HW_ENGINE_RCS_MASK);
 	u32 mask_flags = 0;
 
+	*head = lrc->ring.tail;
+
 	i = emit_copy_timestamp(lrc, dw, i);
 
 	dw[i++] = preparser_disable(true);
@@ -396,11 +403,14 @@ static void __emit_job_gen12_render_compute(struct xe_sched_job *job,
 }
 
 static void emit_migration_job_gen12(struct xe_sched_job *job,
-				     struct xe_lrc *lrc, u32 seqno)
+				     struct xe_lrc *lrc, u32 *head,
+				     u32 seqno)
 {
 	u32 saddr = xe_lrc_start_seqno_ggtt_addr(lrc);
 	u32 dw[MAX_JOB_SIZE_DW], i = 0;
 
+	*head = lrc->ring.tail;
+
 	i = emit_copy_timestamp(lrc, dw, i);
 
 	i = emit_store_imm_ggtt(saddr, seqno, dw, i);
@@ -434,6 +444,7 @@ static void emit_job_gen12_gsc(struct xe_sched_job *job)
 
 	__emit_job_gen12_simple(job, job->q->lrc[0],
 				job->ptrs[0].batch_addr,
+				&job->ptrs[0].head,
 				xe_sched_job_lrc_seqno(job));
 }
 
@@ -443,6 +454,7 @@ static void emit_job_gen12_copy(struct xe_sched_job *job)
 
 	if (xe_sched_job_is_migration(job->q)) {
 		emit_migration_job_gen12(job, job->q->lrc[0],
+					 &job->ptrs[0].head,
 					 xe_sched_job_lrc_seqno(job));
 		return;
 	}
@@ -450,6 +462,7 @@ static void emit_job_gen12_copy(struct xe_sched_job *job)
 	for (i = 0; i < job->q->width; ++i)
 		__emit_job_gen12_simple(job, job->q->lrc[i],
 					job->ptrs[i].batch_addr,
+					&job->ptrs[i].head,
 					xe_sched_job_lrc_seqno(job));
 }
 
@@ -461,6 +474,7 @@ static void emit_job_gen12_video(struct xe_sched_job *job)
 	for (i = 0; i < job->q->width; ++i)
 		__emit_job_gen12_video(job, job->q->lrc[i],
 				       job->ptrs[i].batch_addr,
+				       &job->ptrs[i].head,
 				       xe_sched_job_lrc_seqno(job));
 }
 
@@ -471,6 +485,7 @@ static void emit_job_gen12_render_compute(struct xe_sched_job *job)
 	for (i = 0; i < job->q->width; ++i)
 		__emit_job_gen12_render_compute(job, job->q->lrc[i],
 						job->ptrs[i].batch_addr,
+						&job->ptrs[i].head,
 						xe_sched_job_lrc_seqno(job));
 }
 
drivers/gpu/drm/xe/xe_sched_job_types.h +5 -0
index dbf260dded8d..7ce58765a34a 100644
@@ -24,6 +24,11 @@ struct xe_job_ptrs {
 	struct dma_fence_chain *chain_fence;
 	/** @batch_addr: Batch buffer address. */
 	u64 batch_addr;
+	/**
+	 * @head: The tail pointer of the LRC (so head pointer of job) when the
+	 * job was submitted
+	 */
+	u32 head;
 };
 
 /**
Signature · 1 line
2.34.1
author maintainer
#11 lore raw

PATCH v2 6/34 drm/xe/guc: Track pending-enable source in submission state

In reply to Matthew Brost

Add explicit tracking in the GuC submission state to record the source of a pending enable (TDR vs. queue resume path vs. submission). Disambiguating the origin lets the GuC submission state machine apply the correct recovery/replay behavior.

This helps VF restore: when the device comes back, the state machine knows whether the pending enable stems from timeout recovery, from a queue resume sequence, or submission and can gate sequencing and fixups accordingly.

v4:
- Clarify commit message (Tomasz)

---
 drivers/gpu/drm/xe/xe_guc_submit.c | 36 ++++++++++++++++++++++++++++++
 1 file changed, 36 insertions(+)
drivers/gpu/drm/xe/xe_guc_submit.c +36 -0
index 16f78376f196..13746f32b231 100644
@@ -69,6 +69,8 @@ exec_queue_to_guc(struct xe_exec_queue *q)
 #define EXEC_QUEUE_STATE_BANNED			(1 << 9)
 #define EXEC_QUEUE_STATE_CHECK_TIMEOUT		(1 << 10)
 #define EXEC_QUEUE_STATE_EXTRA_REF		(1 << 11)
+#define EXEC_QUEUE_STATE_PENDING_RESUME		(1 << 12)
+#define EXEC_QUEUE_STATE_PENDING_TDR_EXIT	(1 << 13)
 
 static bool exec_queue_registered(struct xe_exec_queue *q)
 {
@@ -220,6 +222,36 @@ static void set_exec_queue_extra_ref(struct xe_exec_queue *q)
 	atomic_or(EXEC_QUEUE_STATE_EXTRA_REF, &q->guc->state);
 }
 
+static bool __maybe_unused exec_queue_pending_resume(struct xe_exec_queue *q)
+{
+	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_PENDING_RESUME;
+}
+
+static void set_exec_queue_pending_resume(struct xe_exec_queue *q)
+{
+	atomic_or(EXEC_QUEUE_STATE_PENDING_RESUME, &q->guc->state);
+}
+
+static void clear_exec_queue_pending_resume(struct xe_exec_queue *q)
+{
+	atomic_and(~EXEC_QUEUE_STATE_PENDING_RESUME, &q->guc->state);
+}
+
+static bool __maybe_unused exec_queue_pending_tdr_exit(struct xe_exec_queue *q)
+{
+	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_PENDING_TDR_EXIT;
+}
+
+static void set_exec_queue_pending_tdr_exit(struct xe_exec_queue *q)
+{
+	atomic_or(EXEC_QUEUE_STATE_PENDING_TDR_EXIT, &q->guc->state);
+}
+
+static void clear_exec_queue_pending_tdr_exit(struct xe_exec_queue *q)
+{
+	atomic_and(~EXEC_QUEUE_STATE_PENDING_TDR_EXIT, &q->guc->state);
+}
+
 static bool exec_queue_killed_or_banned_or_wedged(struct xe_exec_queue *q)
 {
 	return (atomic_read(&q->guc->state) &
@@ -1334,6 +1366,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)
 	return DRM_GPU_SCHED_STAT_RESET;
 
 sched_enable:
+	set_exec_queue_pending_tdr_exit(q);
 	enable_scheduling(q);
 rearm:
 	/*
@@ -1493,6 +1526,7 @@ static void __guc_exec_queue_process_msg_resume(struct xe_sched_msg *msg)
 		clear_exec_queue_suspended(q);
 		if (!exec_queue_enabled(q)) {
 			q->guc->resume_time = RESUME_PENDING;
+			set_exec_queue_pending_resume(q);
 			enable_scheduling(q);
 		}
 	} else {
@@ -2065,6 +2099,8 @@ static void handle_sched_done(struct xe_guc *guc, struct xe_exec_queue *q,
 		xe_gt_assert(guc_to_gt(guc), exec_queue_pending_enable(q));
 
 		q->guc->resume_time = ktime_get();
+		clear_exec_queue_pending_resume(q);
+		clear_exec_queue_pending_tdr_exit(q);
 		clear_exec_queue_pending_enable(q);
 		smp_wmb();
 		wake_up_all(&guc->ct.wq);
Signature · 1 line
2.34.1
author maintainer
#12 lore raw

PATCH v2 7/34 drm/xe: Track LR jobs in DRM scheduler pending list

In reply to Matthew Brost

VF migration requires jobs to remain pending so they can be replayed after the VF comes back. Previously, LR job fences were intentionally signaled immediately after submission to avoid the risk of exporting them, as these fences do not naturally signal in a timely manner and could break dma-fence contracts. A side effect of this approach was that LR jobs were never added to the DRM scheduler’s pending list, preventing them from being tracked for later resubmission.

We now avoid signaling LR job fences and ensure they are never exported; Xe already guards against exporting these internal fences. With that guarantee in place, we can safely track LR jobs in the scheduler’s pending list so they are eligible for resubmission during VF post-migration recovery (and similar recovery paths).

An added benefit is that LR queues now gain the DRM scheduler’s built-in flow control over ring usage rather than rejecting new jobs in the exec IOCTL if the ring is full.

v2:
- Ensure DRM scheduler TDR doesn't run for LR jobs
- Stack variable for killed_or_banned_or_wedged
v4:
- Clarify commit message (Tomasz)

---
 drivers/gpu/drm/xe/xe_exec.c       | 12 ++-------
 drivers/gpu/drm/xe/xe_exec_queue.c | 19 -------------
 drivers/gpu/drm/xe/xe_exec_queue.h |  2 --
 drivers/gpu/drm/xe/xe_guc_submit.c | 43 ++++++++++++++++++++----------
 4 files changed, 31 insertions(+), 45 deletions(-)
drivers/gpu/drm/xe/xe_exec.c +2 -10
index 83897950f0da..0dc27476832b 100644
@@ -124,7 +124,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 	struct xe_validation_ctx ctx;
 	struct xe_sched_job *job;
 	struct xe_vm *vm;
-	bool write_locked, skip_retry = false;
+	bool write_locked;
 	int err = 0;
 	struct xe_hw_engine_group *group;
 	enum xe_hw_engine_group_execution_mode mode, previous_mode;
@@ -266,12 +266,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		goto err_exec;
 	}
 
-	if (xe_exec_queue_is_lr(q) && xe_exec_queue_ring_full(q)) {
-		err = -EWOULDBLOCK;	/* Aliased to -EAGAIN */
-		skip_retry = true;
-		goto err_exec;
-	}
-
 	if (xe_exec_queue_uses_pxp(q)) {
 		err = xe_vm_validate_protected(q->vm);
 		if (err)
@@ -328,8 +322,6 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		xe_sched_job_init_user_fence(job, &syncs[i]);
 	}
 
-	if (xe_exec_queue_is_lr(q))
-		q->ring_ops->emit_job(job);
 	if (!xe_vm_in_lr_mode(vm))
 		xe_exec_queue_last_fence_set(q, vm, &job->drm.s_fence->finished);
 	xe_sched_job_push(job);
@@ -355,7 +347,7 @@ int xe_exec_ioctl(struct drm_device *dev, void *data, struct drm_file *file)
 		xe_validation_ctx_fini(&ctx);
 err_unlock_list:
 	up_read(&vm->lock);
-	if (err == -EAGAIN && !skip_retry)
+	if (err == -EAGAIN)
 		goto retry;
 err_hw_exec_mode:
 	if (mode == EXEC_MODE_DMA_FENCE)
drivers/gpu/drm/xe/xe_exec_queue.c +0 -19
index 6bfaca424ca3..81f707d2c388 100644
@@ -824,25 +824,6 @@ bool xe_exec_queue_is_lr(struct xe_exec_queue *q)
 		!(q->flags & EXEC_QUEUE_FLAG_VM);
 }
 
-static s32 xe_exec_queue_num_job_inflight(struct xe_exec_queue *q)
-{
-	return q->lrc[0]->fence_ctx.next_seqno - xe_lrc_seqno(q->lrc[0]) - 1;
-}
-
-/**
- * xe_exec_queue_ring_full() - Whether an exec_queue's ring is full
- * @q: The exec_queue
- *
- * Return: True if the exec_queue's ring is full, false otherwise.
- */
-bool xe_exec_queue_ring_full(struct xe_exec_queue *q)
-{
-	struct xe_lrc *lrc = q->lrc[0];
-	s32 max_job = lrc->ring.size / MAX_JOB_SIZE_BYTES;
-
-	return xe_exec_queue_num_job_inflight(q) >= max_job;
-}
-
 /**
  * xe_exec_queue_is_idle() - Whether an exec_queue is idle.
  * @q: The exec_queue
drivers/gpu/drm/xe/xe_exec_queue.h +0 -2
index 8821ceb838d0..a4dfbe858bda 100644
@@ -64,8 +64,6 @@ static inline bool xe_exec_queue_uses_pxp(struct xe_exec_queue *q)
 
 bool xe_exec_queue_is_lr(struct xe_exec_queue *q);
 
-bool xe_exec_queue_ring_full(struct xe_exec_queue *q);
-
 bool xe_exec_queue_is_idle(struct xe_exec_queue *q);
 
 void xe_exec_queue_kill(struct xe_exec_queue *q);
drivers/gpu/drm/xe/xe_guc_submit.c +29 -14

92 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#13 lore raw

PATCH v2 8/34 drm/xe: Don't change LRC ring head on job resubmission

In reply to Matthew Brost

Now that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.

v6:
- Also set LRC tail to head so queue is idle coming out of reset

---
 drivers/gpu/drm/xe/xe_guc_submit.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_submit.c +8 -2
index 3a534d93505f..70306f902ba5 100644
@@ -2008,11 +2008,17 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)
 	struct xe_gpu_scheduler *sched = &q->guc->sched;
 
 	if (!exec_queue_killed_or_banned_or_wedged(q)) {
+		struct xe_sched_job *job = xe_sched_first_pending_job(sched);
 		int i;
 
 		trace_xe_exec_queue_resubmit(q);
-		for (i = 0; i < q->width; ++i)
-			xe_lrc_set_ring_head(q->lrc[i], q->lrc[i]->ring.tail);
+		if (job) {
+			for (i = 0; i < q->width; ++i) {
+				q->lrc[i]->ring.tail = job->ptrs[i].head;
+				xe_lrc_set_ring_tail(q->lrc[i],
+						     xe_lrc_ring_head(q->lrc[i]));
+			}
+		}
 		xe_sched_resubmit_jobs(sched);
 	}
 
Signature · 1 line
2.34.1
#14 lore raw

Re: [PATCH v2 08/34] drm/xe: Don't change LRC ring head on job resubmission

Quoted 8 lines · Matthew Brost

Now that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.

v6:
- Also set LRC tail to head so queue is idle coming out of reset

This is complex.. if the job got preempted, then not touching the head makes

sense. But what if the queue was removed by reset? Are the problematic

commands NOP'ed so that we really can leave the head?

Setting tails looks ok.

-Tomasz

Quoted 29 lines
---
  drivers/gpu/drm/xe/xe_guc_submit.c | 10 ++++++++--
  1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_submit.c +8 -2
index 3a534d93505f..70306f902ba5 100644
@@ -2008,11 +2008,17 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)
  	struct xe_gpu_scheduler *sched = &q->guc->sched;
 
  	if (!exec_queue_killed_or_banned_or_wedged(q)) {
+		struct xe_sched_job *job = xe_sched_first_pending_job(sched);
  		int i;
 
  		trace_xe_exec_queue_resubmit(q);
-		for (i = 0; i < q->width; ++i)
-			xe_lrc_set_ring_head(q->lrc[i], q->lrc[i]->ring.tail);
+		if (job) {
+			for (i = 0; i < q->width; ++i) {
+				q->lrc[i]->ring.tail = job->ptrs[i].head;
+				xe_lrc_set_ring_tail(q->lrc[i],
+						     xe_lrc_ring_head(q->lrc[i]));
+			}
+		}
  		xe_sched_resubmit_jobs(sched);
  	}
 
author maintainer
#15 lore raw

Re: [PATCH v2 08/34] drm/xe: Don't change LRC ring head on job resubmission

Quoted 11 lines · Tomasz
Quoted 8 lines · Matthew Brost

Now that we save the job's head during submission, it's no longer necessary to adjust the LRC ring head during resubmission. Instead, a software-based adjustment of the tail will overwrite the old jobs in place. For some odd reason, adjusting the LRC ring head didn't work on parallel queues, which was causing issues in our CI.

v6:
- Also set LRC tail to head so queue is idle coming out of reset

Quoted 8 lines

This is complex.. if the job got preempted, then not touching the head makes

sense. But what if the queue was removed by reset? Are the problematic

commands NOP'ed so that we really can leave the head?

Setting tails looks ok.

So this case is a bit different than VF restore. If a job has started but not completed across GT reset, we just ban the queue unless it is KMD migration queue so preempted jobs are a non issue. KMD jobs could potentially have been preempted but copy jobs are simple copies which would be fine to run twice.

Also coming out of GT reset the queues are not registered with GuC as this point so I believe it is fine to move KMD own LRC tail back the original position of the job which we resubmiting.

TL;DR after some thought I believe this patch is correct.

Matt

Quoted 28 lines

-Tomasz

Quoted 26 lines
---
  drivers/gpu/drm/xe/xe_guc_submit.c | 10 ++++++++--
  1 file changed, 8 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_submit.c +8 -2
index 3a534d93505f..70306f902ba5 100644
@@ -2008,11 +2008,17 @@ static void guc_exec_queue_start(struct xe_exec_queue *q)
  	struct xe_gpu_scheduler *sched = &q->guc->sched;
  	if (!exec_queue_killed_or_banned_or_wedged(q)) {
+		struct xe_sched_job *job = xe_sched_first_pending_job(sched);
  		int i;
  		trace_xe_exec_queue_resubmit(q);
-		for (i = 0; i < q->width; ++i)
-			xe_lrc_set_ring_head(q->lrc[i], q->lrc[i]->ring.tail);
+		if (job) {
+			for (i = 0; i < q->width; ++i) {
+				q->lrc[i]->ring.tail = job->ptrs[i].head;
+				xe_lrc_set_ring_tail(q->lrc[i],
+						     xe_lrc_ring_head(q->lrc[i]));
+			}
+		}
  		xe_sched_resubmit_jobs(sched);
  	}
author maintainer
#16 lore raw

PATCH v2 9/34 drm/xe: Make LRC W/A scratch buffer usage consistent

In reply to Matthew Brost

The LRC W/A currently checks for LRC being iomem in some places, while in others it checks if the scratch buffer is non-NULL. This inconsistency causes issues with the VF post-migration recovery code, which blindly passes in a scratch buffer.

This patch standardizes the check by consistently verifying whether the LRC is iomem to determine if the scratch buffer should be used.

---
 drivers/gpu/drm/xe/xe_lrc.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_lrc.c +1 -1
index 47e9df775072..aed0a3155d1f 100644
@@ -1248,7 +1248,7 @@ static int setup_bo(struct bo_setup_state *state)
 
 static void finish_bo(struct bo_setup_state *state)
 {
-	if (!state->buffer)
+	if (!state->lrc->bo->vmap.is_iomem)
 		return;
 
 	xe_map_memcpy_to(gt_to_xe(state->lrc->gt), &state->lrc->bo->vmap,
Signature · 1 line
2.34.1
Reviewed-by on v2 9/34
#17 lore raw

Re: [PATCH v2 09/34] drm/xe: Make LRC W/A scratch buffer usage consistent

Quoted 7 lines · Matthew Brost

The LRC W/A currently checks for LRC being iomem in some places, while in others it checks if the scratch buffer is non-NULL. This inconsistency causes issues with the VF post-migration recovery code, which blindly passes in a scratch buffer.

This patch standardizes the check by consistently verifying whether the LRC is iomem to determine if the scratch buffer should be used.

No need for explicit assert as the kernel will abort by itself if state->buffer in NULL, so this is ok.

-Tomasz

Quoted 18 lines
---
  drivers/gpu/drm/xe/xe_lrc.c | 2 +-
  1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_lrc.c +1 -1
index 47e9df775072..aed0a3155d1f 100644
@@ -1248,7 +1248,7 @@ static int setup_bo(struct bo_setup_state *state)
 
  static void finish_bo(struct bo_setup_state *state)
  {
-	if (!state->buffer)
+	if (!state->lrc->bo->vmap.is_iomem)
  		return;
 
  	xe_map_memcpy_to(gt_to_xe(state->lrc->gt), &state->lrc->bo->vmap,
Reviewed-by on v2 9/34
#18 lore raw

Re: [PATCH v2 09/34] drm/xe: Make LRC W/A scratch buffer usage consistent

In reply to Matthew Brost
Quoted 18 lines · Matthew Brost

The LRC W/A currently checks for LRC being iomem in some places, while in others it checks if the scratch buffer is non-NULL. This inconsistency causes issues with the VF post-migration recovery code, which blindly passes in a scratch buffer.

This patch standardizes the check by consistently verifying whether the LRC is iomem to determine if the scratch buffer should be used.

---
drivers/gpu/drm/xe/xe_lrc.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_lrc.c +0 -0
index 47e9df775072..aed0a3155d1f 100644

drivers/gpu/drm/xe/xe_lrc.c static int setup_bo(struct bo_setup_state *state) · line 1251

Quoted 5 lines
drivers/gpu/drm/xe/xe_lrc.c +1 -1
@@ -1248,7 +1248,7 @@ static int setup_bo(struct bo_setup_state *state)
 
 static void finish_bo(struct bo_setup_state *state)
 {
-	if (!state->buffer)
+	if (!state->lrc->bo->vmap.is_iomem)

Please add the fixes as the first patches and find the commits they are fixing so they can propagate to drm-xe-fixes. With that, for a palliative fix:

Long-term thoughts below:

The fact that we exposed xe_lrc_setup_wa_bb_with_scratch() to users outside xe_lrc seems completely wrong to me... it explains why indirect_ctx was completely forgotten.

And the ENOMEM handling is in a completely different place than the allocation which makes this a code smell.

I think this change is a palliative that we could accept, but going forward I think it needs to be better handled: we shouldn't be recalculating it on every lrc init, it should rather be handled like the default_lrc and just copied in place. Then on migration what really needs to be fixed up is the ggtt address, i.e. only these calls:

	xe_lrc_write_ctx_reg(lrc,
			     CTX_CS_INDIRECT_CTX,
			     (xe_bo_ggtt_addr(lrc->bo) + state.offset) |
			     /* Size in CLs. */
			     (state.written * sizeof(u32) / 64));

	and

	xe_lrc_write_ctx_reg(lrc, CTX_BB_PER_CTX_PTR,
                    xe_bo_ggtt_addr(lrc->bo) + state.offset + 1);

without re-allocating and recalculating it over and over. I will take a look at doing this

Lucas De Marchi

Quoted 6 lines
drivers/gpu/drm/xe/xe_lrc.c +0 -0
 		return;
 
 	xe_map_memcpy_to(gt_to_xe(state->lrc->gt), &state->lrc->bo->vmap,
Signature · 1 line
2.34.1
author maintainer
#19 lore raw

Re: [PATCH v2 09/34] drm/xe: Make LRC W/A scratch buffer usage consistent

Quoted 20 lines · Lucas De Marchi
Quoted 19 lines · Matthew Brost

The LRC W/A currently checks for LRC being iomem in some places, while in others it checks if the scratch buffer is non-NULL. This inconsistency causes issues with the VF post-migration recovery code, which blindly passes in a scratch buffer.

This patch standardizes the check by consistently verifying whether the LRC is iomem to determine if the scratch buffer should be used.

---
drivers/gpu/drm/xe/xe_lrc.c | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_lrc.c +0 -0
index 47e9df775072..aed0a3155d1f 100644
@@ -1248,7 +1248,7 @@ static int setup_bo(struct bo_setup_state *state)
 
Quoted 7 lines

drivers/gpu/drm/xe/xe_lrc.c static int setup_bo(struct bo_setup_state *state) · line 1251

Quoted 4 lines
drivers/gpu/drm/xe/xe_lrc.c +1 -1
 static void finish_bo(struct bo_setup_state *state)
 {
-	if (!state->buffer)
+	if (!state->lrc->bo->vmap.is_iomem)

Please add the fixes as the first patches and find the commits they are fixing so they can propagate to drm-xe-fixes. With that, for a

VF migration doesn't work at all upstream, so I don't believe this warrents a fixes tag.

Quoted 10 lines

palliative fix:

Long-term thoughts below:

The fact that we exposed xe_lrc_setup_wa_bb_with_scratch() to users outside xe_lrc seems completely wrong to me... it explains why indirect_ctx was completely forgotten.

Let me add in indirect_ctx fixup in my next rev for completeness.

Quoted 2 lines

And the ENOMEM handling is in a completely different place than the allocation which makes this a code smell.

I have a fix for that, we will needs fixes tag for this one.

Quoted 19 lines

I think this change is a palliative that we could accept, but going forward I think it needs to be better handled: we shouldn't be recalculating it on every lrc init, it should rather be handled like the default_lrc and just copied in place. Then on migration what really needs to be fixed up is the ggtt address, i.e. only these calls:

	xe_lrc_write_ctx_reg(lrc,
			     CTX_CS_INDIRECT_CTX,
			     (xe_bo_ggtt_addr(lrc->bo) + state.offset) |
			     /* Size in CLs. */
			     (state.written * sizeof(u32) / 64));

	and

	xe_lrc_write_ctx_reg(lrc, CTX_BB_PER_CTX_PTR,
                    xe_bo_ggtt_addr(lrc->bo) + state.offset + 1);
Quoted 3 lines

without re-allocating and recalculating it over and over. I will take a look at doing this

That is not a bad idea but also not huge deal with the way the code is written as this isn't really a critical path anywhere it is called and unlike default LRC which we need to save off for the GuC ADS / W/A issued during driver load it is perfectly fine to regenerate on each LRC init.

Matt

Quoted 8 lines

Lucas De Marchi

Quoted 6 lines
drivers/gpu/drm/xe/xe_lrc.c +0 -0
 		return;
 
 	xe_map_memcpy_to(gt_to_xe(state->lrc->gt), &state->lrc->bo->vmap,
Signature · 1 line
2.34.1
author maintainer
#20 lore raw

PATCH v2 10/34 drm/xe/guc: Document GuC submission backend

In reply to Matthew Brost

Add kernel-doc to xe_guc_submit.c describing the submission path, the per-queue single-threaded model with pause/resume, the driver shadow state machine and lost-H2G replay, job timeout handling, recovery flows (GT reset, PM resume, VF resume), and reclaim constraints.

v2:
- Mirror tweaks for clarity
- Add new doc to Xe rst files
v3:
- Clarify global vs per-queue stop / start
- Clarify VF resume flow
- Add section for 'Waiters during VF resume'
- Add section for 'Page-faulting queues during VF migration'
- Add section for 'GuC-ID assignment'
- Add section for 'Reference counting and final queue destruction'
v4:
- s/VF resume/VF post migration recovery (Tomasz)

---
 Documentation/gpu/xe/index.rst     |   1 +
 drivers/gpu/drm/xe/xe_guc_submit.c | 282 +++++++++++++++++++++++++++++
 2 files changed, 283 insertions(+)
Documentation/gpu/xe/index.rst +1 -0
index 88b22fad880e..692c544b164c 100644
@@ -28,3 +28,4 @@ DG2, etc is provided to prototype the driver.
    xe_device
    xe-drm-usage-stats.rst
    xe_configfs
+   xe_guc_submit
drivers/gpu/drm/xe/xe_guc_submit.c +282 -0

288 lines not on this page · raw message

Signature · 1 line
2.34.1
#21 lore raw

Re: [PATCH v2 10/34] drm/xe/guc: Document GuC submission backend

Quoted 4 lines · Matthew Brost

Add kernel-doc to xe_guc_submit.c describing the submission path, the per-queue single-threaded model with pause/resume, the driver shadow state machine and lost-H2G replay, job timeout handling, recovery flows (GT reset, PM resume, VF resume), and reclaim constraints.

can we split this patch into "general doc of submission path" that could be reviewed and merged separately from the "VF recovery" doc that will stay in this series ?

Quoted 18 lines

v2:
- Mirror tweaks for clarity
- Add new doc to Xe rst files
v3:
- Clarify global vs per-queue stop / start
- Clarify VF resume flow
- Add section for 'Waiters during VF resume'
- Add section for 'Page-faulting queues during VF migration'
- Add section for 'GuC-ID assignment'
- Add section for 'Reference counting and final queue destruction'
v4:
- s/VF resume/VF post migration recovery (Tomasz)

---
 Documentation/gpu/xe/index.rst     |   1 +
 drivers/gpu/drm/xe/xe_guc_submit.c | 282 +++++++++++++++++++++++++++++
author maintainer
#22 lore raw

Re: [PATCH v2 10/34] drm/xe/guc: Document GuC submission backend

Quoted 12 lines · Michal Wajdeczko
Quoted 4 lines · Matthew Brost

Add kernel-doc to xe_guc_submit.c describing the submission path, the per-queue single-threaded model with pause/resume, the driver shadow state machine and lost-H2G replay, job timeout handling, recovery flows (GT reset, PM resume, VF resume), and reclaim constraints.

can we split this patch into "general doc of submission path" that could be reviewed and merged separately from the "VF recovery" doc that will stay in this series ?

I don't really see the why here? Ofc merging this patch as part of this series is a blocker but included it as largely explains everything this series does + overall design of GuC submission which was entirely in my head prior to this patch.

Matt

Quoted 18 lines
Quoted 18 lines

v2:
- Mirror tweaks for clarity
- Add new doc to Xe rst files
v3:
- Clarify global vs per-queue stop / start
- Clarify VF resume flow
- Add section for 'Waiters during VF resume'
- Add section for 'Page-faulting queues during VF migration'
- Add section for 'GuC-ID assignment'
- Add section for 'Reference counting and final queue destruction'
v4:
- s/VF resume/VF post migration recovery (Tomasz)

---
 Documentation/gpu/xe/index.rst     |   1 +
 drivers/gpu/drm/xe/xe_guc_submit.c | 282 +++++++++++++++++++++++++++++
author maintainer
#23 lore raw

PATCH v2 11/34 drm/xe/vf: Add xe_gt_sriov_vf_recovery_inprogress helper

In reply to Matthew Brost

Add xe_gt_sriov_vf_recovery_inprogress helper.

This helper serves as the singular point to determine whether a VF post-migration recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 17 ++++++++
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++
 drivers/gpu/drm/xe/xe_memirq.c            | 48 ++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_memirq.h            |  3 ++
 5 files changed, 79 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +17 -0
index 016c867e5e2b..c9d0e32e7a15 100644
@@ -26,6 +26,7 @@
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
 #include "xe_lrc.h"
+#include "xe_memirq.h"
 #include "xe_mmio.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
@@ -828,6 +829,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	struct xe_device *xe = gt_to_xe(gt);
 
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
+	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
 	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
 	/*
@@ -1172,3 +1174,18 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 	drm_printf(p, "\thandshake:\t%u.%u\n",
 		   pf_version->major, pf_version->minor);
 }
+
+/**
+ * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
+ * @gt: the &xe_gt
+ *
+ * Return: True if VF post migration recovery in progress, False otherwise
+ */
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)
+{
+	struct xe_memirq *memirq = &gt_to_tile(gt)->memirq;
+
+	return IS_SRIOV_VF(gt_to_xe(gt)) &&
+		(xe_memirq_vf_recovery_irq_pending(memirq, &gt->uc.guc) ||
+		 READ_ONCE(gt->sriov.vf.migration.recovery_inprogress));
+}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index 0af1dc769fe0..bb5f8eace19b 100644
@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
+
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
 u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10 -0
index d95857bd789b..7b10b8e1e10e 100644
@@ -49,6 +49,14 @@ struct xe_gt_sriov_vf_runtime {
 	} *regs;
 };
 
+/**
+ * xe_gt_sriov_vf_migration - VF migration data.
+ */
+struct xe_gt_sriov_vf_migration {
+	/** @recovery_inprogress: VF post migration recovery in progress */
+	bool recovery_inprogress;
+};
+
 /**
  * struct xe_gt_sriov_vf - GT level VF virtualization data.
  */
@@ -61,6 +69,8 @@ struct xe_gt_sriov_vf {
 	struct xe_gt_sriov_vf_selfconfig self_config;
 	/** @runtime: runtime data retrieved from the PF. */
 	struct xe_gt_sriov_vf_runtime runtime;
+	/** @migration: migration data for the VF. */
+	struct xe_gt_sriov_vf_migration migration;
 };
 
 #endif
drivers/gpu/drm/xe/xe_memirq.c +47 -1

67 lines not on this page · raw message

drivers/gpu/drm/xe/xe_memirq.h +3 -0
index 06130650e9d6..476b8cba179d 100644
@@ -25,4 +25,7 @@ void xe_memirq_handler(struct xe_memirq *memirq);
 
 int xe_memirq_init_guc(struct xe_memirq *memirq, struct xe_guc *guc);
 
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc);
+
 #endif
Signature · 1 line
2.34.1
#24 lore raw

Re: [PATCH v2 11/34] drm/xe/vf: Add xe_gt_sriov_vf_recovery_inprogress helper

Quoted 3 lines · Matthew Brost

Add xe_gt_sriov_vf_recovery_inprogress helper.

This helper serves as the singular point to determine whether a VF

hmm, this "singular" looks like a GT-level only, not global

Quoted 39 lines

post-migration recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 17 ++++++++
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++
 drivers/gpu/drm/xe/xe_memirq.c            | 48 ++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_memirq.h            |  3 ++
 5 files changed, 79 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
index 016c867e5e2b..c9d0e32e7a15 100644
@@ -26,6 +26,7 @@
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
 #include "xe_lrc.h"
+#include "xe_memirq.h"
 #include "xe_mmio.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
@@ -828,6 +829,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	struct xe_device *xe = gt_to_xe(gt);
 
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
+	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
 	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
 	/*
@@ -1172,3 +1174,18 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 	drm_printf(p, "\thandshake:\t%u.%u\n",
 		   pf_version->major, pf_version->minor);
 }
+
+/**

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1188

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+ * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
+ * @gt: the &xe_gt
+ *
+ * Return: True if VF post migration recovery in progress, False otherwise
+ */
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)
+{
+	struct xe_memirq *memirq = &gt_to_tile(gt)->memirq;
+
+	return IS_SRIOV_VF(gt_to_xe(gt)) &&

this is xe_gt_sriov_vf function, so it is expected to be called only by the VF code, thus we should rather use xe_gt_assert here and the caller is responsible for the IS_SRIOV_VF check

Quoted 49 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+		(xe_memirq_vf_recovery_irq_pending(memirq, &gt->uc.guc) ||
+		 READ_ONCE(gt->sriov.vf.migration.recovery_inprogress));
+}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index 0af1dc769fe0..bb5f8eace19b 100644
@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
+
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
 u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10 -0
index d95857bd789b..7b10b8e1e10e 100644
@@ -49,6 +49,14 @@ struct xe_gt_sriov_vf_runtime {
 	} *regs;
 };
 
+/**
+ * xe_gt_sriov_vf_migration - VF migration data.
+ */
+struct xe_gt_sriov_vf_migration {
+	/** @recovery_inprogress: VF post migration recovery in progress */
+	bool recovery_inprogress;
+};
+
 /**
  * struct xe_gt_sriov_vf - GT level VF virtualization data.
  */
@@ -61,6 +69,8 @@ struct xe_gt_sriov_vf {
 	struct xe_gt_sriov_vf_selfconfig self_config;
 	/** @runtime: runtime data retrieved from the PF. */
 	struct xe_gt_sriov_vf_runtime runtime;
+	/** @migration: migration data for the VF. */
+	struct xe_gt_sriov_vf_migration migration;
 };
 
 #endif
drivers/gpu/drm/xe/xe_memirq.c +0 -0
index 49c45ec3e83c..94d5d6859aab 100644

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_postinstall(struct xe_memirq *memirq) · line 403

Quoted 6 lines
drivers/gpu/drm/xe/xe_memirq.c +3 -0
@@ -398,6 +398,23 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)
 		memirq_set_enable(memirq, true);
 }
 
+static bool memirq_received_noclear(struct xe_memirq *memirq,
+				    struct iosys_map *vector,
+				    u16 offset, const char *name)

maybe instead of duplicating code of memirq_received() in 90% just add there the "bool clear" flag?

Quoted 18 lines
drivers/gpu/drm/xe/xe_memirq.c +14 -0
+{
+	u8 value;
+
+	value = iosys_map_rd(vector, offset, u8);
+	if (value) {
+		if (value != 0xff)
+			memirq_err_ratelimited(memirq,
+					       "Unexpected memirq value %#x from %s at %u\n",
+					       value, name, offset);
+	}
+
+	return value;
+}
+
 static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,
 			    u16 offset, const char *name)
 {

drivers/gpu/drm/xe/xe_memirq.c static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat · line 457

Quoted 8 lines
drivers/gpu/drm/xe/xe_memirq.c +4 -1
@@ -434,8 +451,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat
 	if (memirq_received(memirq, status, ilog2(GUC_INTR_GUC2HOST), name))
 		xe_guc_irq_handler(guc, GUC_INTR_GUC2HOST);
 
-	if (memirq_received(memirq, status, ilog2(GUC_INTR_SW_INT_0), name))
+	/*
+	 * We must wait to perform the clear operation until after
+	 * xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race
+	 * conditions where xe_gt_sriov_vf_recovery_inprogress() returns false.

but the VF recovery "inprogress" shall be already set in the top level

xe_sriov_vf_start_migration_recovery()

even before the GT-level recovery starts, where is this race ?

drivers/gpu/drm/xe/xe_memirq.c static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat · line 461

Quoted 4 lines
drivers/gpu/drm/xe/xe_memirq.c +3 -0
+	 */
+	if (memirq_received_noclear(memirq, status, ilog2(GUC_INTR_SW_INT_0),
+				    name)) {
 		xe_guc_irq_handler(guc, GUC_INTR_SW_INT_0);

what if new MEMIRQ will arrive just here

is it ok that we will clear it immediately?

the whole memirq flow is that we clear irq byte first and then process it, so if anything comes right after we finish processing will be noticed on next iteration

I assume any races due to double migration shall be handled on the VF2GUC communication level while sending RESFIX_START/DONE, not here

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 489

Quoted 11 lines
drivers/gpu/drm/xe/xe_memirq.c +4 -0
+		iosys_map_wr(status, ilog2(GUC_INTR_SW_INT_0), u8, 0x00);
+	}
 }
 
 /**
@@ -460,6 +485,27 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe)
 	}
 }
 
+/**
+ * xe_memirq_vf_recovery_irq_pending() - VF recovery IRQ is pending

this function isn't really using anything VF specific except that on the VF the SW_INT_0 means "migrated"

maybe we can drop the _vf from function name?

xe_memirq_pending_guc(memirq, guc, bit)

Quoted 5 lines
drivers/gpu/drm/xe/xe_memirq.c +5 -0
+ * @memirq: the &xe_memirq
+ * @guc: the &xe_guc to check for IRQ
+ *
+ * Return: True if VF recovery IRQ is pending on @guc, False otherwise
+ */

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 504

Quoted 10 lines
drivers/gpu/drm/xe/xe_memirq.c +10 -0
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc)
+{
+	struct xe_gt *gt = guc_to_gt(guc);
+	struct iosys_map map;
+
+	if (xe_gt_is_media_type(gt))
+		map = IOSYS_MAP_INIT_OFFSET(&memirq->status, ilog2(INTR_MGUC) * SZ_16);
+	else
+		map = IOSYS_MAP_INIT_OFFSET(&memirq->status, ilog2(INTR_GUC) * SZ_16);

nit: maybe just calc offset conditionally?

	u32 offset = is_media ? ilog2(INTR_MGUC) : ilog2(INTR_GUC);

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 506

Quoted 2 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+
+	return iosys_map_rd(&map, ilog2(GUC_INTR_SW_INT_0), u8);

as we have helpers we should use them

return memirq_received_noclear(...)

Quoted 17 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+}
+
 /**
  * xe_memirq_handler - The `Memory Based Interrupts`_ Handler.
  * @memirq: the &xe_memirq
drivers/gpu/drm/xe/xe_memirq.h +3 -0
index 06130650e9d6..476b8cba179d 100644
@@ -25,4 +25,7 @@ void xe_memirq_handler(struct xe_memirq *memirq);
 
 int xe_memirq_init_guc(struct xe_memirq *memirq, struct xe_guc *guc);
 
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc);
+
 #endif
author maintainer
#25 lore raw

Re: [PATCH v2 11/34] drm/xe/vf: Add xe_gt_sriov_vf_recovery_inprogress helper

Quoted 9 lines · Michal Wajdeczko
Quoted 3 lines · Matthew Brost

Add xe_gt_sriov_vf_recovery_inprogress helper.

This helper serves as the singular point to determine whether a VF

hmm, this "singular" looks like a GT-level only, not global

Yes, it is GT scoped. I will adjust the commit message.

Quoted 45 lines
Quoted 45 lines

post-migration recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 17 ++++++++
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++
 drivers/gpu/drm/xe/xe_memirq.c            | 48 ++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_memirq.h            |  3 ++
 5 files changed, 79 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
index 016c867e5e2b..c9d0e32e7a15 100644
@@ -26,6 +26,7 @@
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
 #include "xe_lrc.h"
+#include "xe_memirq.h"
 #include "xe_mmio.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
@@ -828,6 +829,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	struct xe_device *xe = gt_to_xe(gt);
 
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
+	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
 	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
 	/*
@@ -1172,3 +1174,18 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 	drm_printf(p, "\thandshake:\t%u.%u\n",
 		   pf_version->major, pf_version->minor);
 }
+
+/**
+ * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
+ * @gt: the &xe_gt
+ *
+ * Return: True if VF post migration recovery in progress, False otherwise
+ */
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1188

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+{
+	struct xe_memirq *memirq = &gt_to_tile(gt)->memirq;
+
+	return IS_SRIOV_VF(gt_to_xe(gt)) &&

this is xe_gt_sriov_vf function, so it is expected to be called only by the VF code, thus we should rather use xe_gt_assert here and the caller is responsible for the IS_SRIOV_VF check

That is not how I have coded this. I blindly call this in various places and I don't think it could at call site to determine if it is a VF as we have if (VF) statements all over the driver. If perf is the concern, we could move the IS_SRIOV_VF(gt_to_xe(gt)) part of function to static inline and reset of the function in an exported function.

Quoted 52 lines
Quoted 52 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+		(xe_memirq_vf_recovery_irq_pending(memirq, &gt->uc.guc) ||
+		 READ_ONCE(gt->sriov.vf.migration.recovery_inprogress));
+}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index 0af1dc769fe0..bb5f8eace19b 100644
@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
+
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
 u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10 -0
index d95857bd789b..7b10b8e1e10e 100644
@@ -49,6 +49,14 @@ struct xe_gt_sriov_vf_runtime {
 	} *regs;
 };
 
+/**
+ * xe_gt_sriov_vf_migration - VF migration data.
+ */
+struct xe_gt_sriov_vf_migration {
+	/** @recovery_inprogress: VF post migration recovery in progress */
+	bool recovery_inprogress;
+};
+
 /**
  * struct xe_gt_sriov_vf - GT level VF virtualization data.
  */
@@ -61,6 +69,8 @@ struct xe_gt_sriov_vf {
 	struct xe_gt_sriov_vf_selfconfig self_config;
 	/** @runtime: runtime data retrieved from the PF. */
 	struct xe_gt_sriov_vf_runtime runtime;
+	/** @migration: migration data for the VF. */
+	struct xe_gt_sriov_vf_migration migration;
 };
 
 #endif
drivers/gpu/drm/xe/xe_memirq.c +0 -0
index 49c45ec3e83c..94d5d6859aab 100644
@@ -398,6 +398,23 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)
 		memirq_set_enable(memirq, true);
 }
 
Quoted 7 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_postinstall(struct xe_memirq *memirq) · line 403

Quoted 3 lines
drivers/gpu/drm/xe/xe_memirq.c +3 -0
+static bool memirq_received_noclear(struct xe_memirq *memirq,
+				    struct iosys_map *vector,
+				    u16 offset, const char *name)

maybe instead of duplicating code of memirq_received() in 90% just add there the "bool clear" flag?

Sure.

Quoted 29 lines
Quoted 18 lines
drivers/gpu/drm/xe/xe_memirq.c +14 -0
+{
+	u8 value;
+
+	value = iosys_map_rd(vector, offset, u8);
+	if (value) {
+		if (value != 0xff)
+			memirq_err_ratelimited(memirq,
+					       "Unexpected memirq value %#x from %s at %u\n",
+					       value, name, offset);
+	}
+
+	return value;
+}
+
 static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,
 			    u16 offset, const char *name)
 {

drivers/gpu/drm/xe/xe_memirq.c static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat · line 457

Quoted 8 lines
drivers/gpu/drm/xe/xe_memirq.c +4 -1
@@ -434,8 +451,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat
 	if (memirq_received(memirq, status, ilog2(GUC_INTR_GUC2HOST), name))
 		xe_guc_irq_handler(guc, GUC_INTR_GUC2HOST);
 
-	if (memirq_received(memirq, status, ilog2(GUC_INTR_SW_INT_0), name))
+	/*
+	 * We must wait to perform the clear operation until after
+	 * xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race
+	 * conditions where xe_gt_sriov_vf_recovery_inprogress() returns false.

but the VF recovery "inprogress" shall be already set in the top level

Quoted 4 lines

xe_sriov_vf_start_migration_recovery()

even before the GT-level recovery starts, where is this race ?

If we clear the interrupt it here, this is before the IRQ handler is called which flips the software bit for "inprogress". There would be window where xe_gt_sriov_vf_recovery_inprogress could return false which is problematic.

Quoted 7 lines

drivers/gpu/drm/xe/xe_memirq.c static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat · line 461

Quoted 4 lines
drivers/gpu/drm/xe/xe_memirq.c +3 -0
+	 */
+	if (memirq_received_noclear(memirq, status, ilog2(GUC_INTR_SW_INT_0),
+				    name)) {
 		xe_guc_irq_handler(guc, GUC_INTR_SW_INT_0);

what if new MEMIRQ will arrive just here

Quoted 6 lines

is it ok that we will clear it immediately?

the whole memirq flow is that we clear irq byte first and then process it, so if anything comes right after we finish processing will be noticed on next iteration

I don't see why that matters in this case. In either case multiple IRQs could happen and single IRQ handler runs.

Quoted 2 lines

I assume any races due to double migration shall be handled on the VF2GUC communication level while sending RESFIX_START/DONE, not here

I don't think it should be possible to get multiple RESFIX_START IRQs before DONE is complete. This existing code upstream seems to handle these cases, so my series attempts to handle this too, but it seems like something that shouldn't be possible.

Quoted 9 lines
Quoted 9 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+		iosys_map_wr(status, ilog2(GUC_INTR_SW_INT_0), u8, 0x00);
+	}
 }
 
 /**
@@ -460,6 +485,27 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe)
 	}
 }
 
Quoted 8 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 489

Quoted 2 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+/**
+ * xe_memirq_vf_recovery_irq_pending() - VF recovery IRQ is pending

this function isn't really using anything VF specific except that on the VF the SW_INT_0 means "migrated"

maybe we can drop the _vf from function name?

Sure, so 'xe_memirq_recovery_irq_pending'?

Or 'xe_memirq_sw_int0_irq_pending'?

Quoted 19 lines

xe_memirq_pending_guc(memirq, guc, bit)

Quoted 5 lines
drivers/gpu/drm/xe/xe_memirq.c +5 -0
+ * @memirq: the &xe_memirq
+ * @guc: the &xe_guc to check for IRQ
+ *
+ * Return: True if VF recovery IRQ is pending on @guc, False otherwise
+ */

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 504

Quoted 10 lines
drivers/gpu/drm/xe/xe_memirq.c +10 -0
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc)
+{
+	struct xe_gt *gt = guc_to_gt(guc);
+	struct iosys_map map;
+
+	if (xe_gt_is_media_type(gt))
+		map = IOSYS_MAP_INIT_OFFSET(&memirq->status, ilog2(INTR_MGUC) * SZ_16);
+	else
+		map = IOSYS_MAP_INIT_OFFSET(&memirq->status, ilog2(INTR_GUC) * SZ_16);

nit: maybe just calc offset conditionally?

Quoted 2 lines
	u32 offset = is_media ? ilog2(INTR_MGUC) : ilog2(INTR_GUC);

Sure.

Quoted 6 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 506

Quoted 2 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+
+	return iosys_map_rd(&map, ilog2(GUC_INTR_SW_INT_0), u8);

as we have helpers we should use them

return memirq_received_noclear(...)

Sure.

Matt

Quoted 19 lines
Quoted 18 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+}
+
 /**
  * xe_memirq_handler - The `Memory Based Interrupts`_ Handler.
  * @memirq: the &xe_memirq
drivers/gpu/drm/xe/xe_memirq.h +3 -0
index 06130650e9d6..476b8cba179d 100644
@@ -25,4 +25,7 @@ void xe_memirq_handler(struct xe_memirq *memirq);
 
 int xe_memirq_init_guc(struct xe_memirq *memirq, struct xe_guc *guc);
 
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc);
+
 #endif
#26 lore raw

Re: [PATCH v2 11/34] drm/xe/vf: Add xe_gt_sriov_vf_recovery_inprogress helper

Quoted 68 lines · Matthew Brost
Quoted 9 lines · Michal Wajdeczko
Quoted 3 lines · Matthew Brost

Add xe_gt_sriov_vf_recovery_inprogress helper.

This helper serves as the singular point to determine whether a VF

hmm, this "singular" looks like a GT-level only, not global

Yes, it is GT scoped. I will adjust the commit message.

Quoted 45 lines
Quoted 45 lines

post-migration recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 17 ++++++++
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++
 drivers/gpu/drm/xe/xe_memirq.c            | 48 ++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_memirq.h            |  3 ++
 5 files changed, 79 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
index 016c867e5e2b..c9d0e32e7a15 100644
@@ -26,6 +26,7 @@
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
 #include "xe_lrc.h"
+#include "xe_memirq.h"
 #include "xe_mmio.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
@@ -828,6 +829,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	struct xe_device *xe = gt_to_xe(gt);
 
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
+	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
 	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
 	/*
@@ -1172,3 +1174,18 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 	drm_printf(p, "\thandshake:\t%u.%u\n",
 		   pf_version->major, pf_version->minor);
 }
+
+/**
+ * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
+ * @gt: the &xe_gt
+ *
+ * Return: True if VF post migration recovery in progress, False otherwise
+ */
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1188

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+{
+	struct xe_memirq *memirq = &gt_to_tile(gt)->memirq;
+
+	return IS_SRIOV_VF(gt_to_xe(gt)) &&

this is xe_gt_sriov_vf function, so it is expected to be called only by the VF code, thus we should rather use xe_gt_assert here and the caller is responsible for the IS_SRIOV_VF check

Quoted 5 lines

That is not how I have coded this. I blindly call this in various places and I don't think it could at call site to determine if it is a VF as we have if (VF) statements all over the driver. If perf is the concern, we could move the IS_SRIOV_VF(gt_to_xe(gt)) part of function to static inline and reset of the function in an exported function.

inline will work, preferable not in xe_gt_sriov_vf.h ;)

static inline xe_gt_recovery_inprogress(gt)

{
	return IS_SRIOV_VF(xe) && xe_gt_sriov_vf_recovery_inprogress(gt);
}
Quoted 94 lines
Quoted 53 lines
Quoted 53 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+		(xe_memirq_vf_recovery_irq_pending(memirq, &gt->uc.guc) ||
+		 READ_ONCE(gt->sriov.vf.migration.recovery_inprogress));
+}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index 0af1dc769fe0..bb5f8eace19b 100644
@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
+
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
 u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10 -0
index d95857bd789b..7b10b8e1e10e 100644
@@ -49,6 +49,14 @@ struct xe_gt_sriov_vf_runtime {
 	} *regs;
 };
 
+/**
+ * xe_gt_sriov_vf_migration - VF migration data.
+ */
+struct xe_gt_sriov_vf_migration {
+	/** @recovery_inprogress: VF post migration recovery in progress */
+	bool recovery_inprogress;
+};
+
 /**
  * struct xe_gt_sriov_vf - GT level VF virtualization data.
  */
@@ -61,6 +69,8 @@ struct xe_gt_sriov_vf {
 	struct xe_gt_sriov_vf_selfconfig self_config;
 	/** @runtime: runtime data retrieved from the PF. */
 	struct xe_gt_sriov_vf_runtime runtime;
+	/** @migration: migration data for the VF. */
+	struct xe_gt_sriov_vf_migration migration;
 };
 
 #endif
drivers/gpu/drm/xe/xe_memirq.c +0 -0
index 49c45ec3e83c..94d5d6859aab 100644
@@ -398,6 +398,23 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)
 		memirq_set_enable(memirq, true);
 }
 
Quoted 7 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_postinstall(struct xe_memirq *memirq) · line 403

Quoted 3 lines
drivers/gpu/drm/xe/xe_memirq.c +3 -0
+static bool memirq_received_noclear(struct xe_memirq *memirq,
+				    struct iosys_map *vector,
+				    u16 offset, const char *name)

maybe instead of duplicating code of memirq_received() in 90% just add there the "bool clear" flag?

Sure.

Quoted 31 lines
Quoted 18 lines
drivers/gpu/drm/xe/xe_memirq.c +14 -0
+{
+	u8 value;
+
+	value = iosys_map_rd(vector, offset, u8);
+	if (value) {
+		if (value != 0xff)
+			memirq_err_ratelimited(memirq,
+					       "Unexpected memirq value %#x from %s at %u\n",
+					       value, name, offset);
+	}
+
+	return value;
+}
+
 static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,
 			    u16 offset, const char *name)
 {

drivers/gpu/drm/xe/xe_memirq.c static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat · line 457

Quoted 8 lines
drivers/gpu/drm/xe/xe_memirq.c +4 -1
@@ -434,8 +451,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat
 	if (memirq_received(memirq, status, ilog2(GUC_INTR_GUC2HOST), name))
 		xe_guc_irq_handler(guc, GUC_INTR_GUC2HOST);
 
-	if (memirq_received(memirq, status, ilog2(GUC_INTR_SW_INT_0), name))
+	/*
+	 * We must wait to perform the clear operation until after
+	 * xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race
+	 * conditions where xe_gt_sriov_vf_recovery_inprogress() returns false.

but the VF recovery "inprogress" shall be already set in the top level

xe_sriov_vf_start_migration_recovery()

Quoted 7 lines
Quoted 2 lines

even before the GT-level recovery starts, where is this race ?

If we clear the interrupt it here, this is before the IRQ handler is called which flips the software bit for "inprogress". There would be window where xe_gt_sriov_vf_recovery_inprogress could return false which is problematic.

but who calls that recovery_inprogress() and why should it be problematic?

note there might be still other threads that will finish some actions (including sending CTB) that will just be stuck there until we start the recovery, so even if recovery_inprogress() returns false for the small window that shouldn't change the picture

Quoted 15 lines
Quoted 8 lines

drivers/gpu/drm/xe/xe_memirq.c static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat · line 461

Quoted 5 lines
drivers/gpu/drm/xe/xe_memirq.c +3 -0
+	 */
+	if (memirq_received_noclear(memirq, status, ilog2(GUC_INTR_SW_INT_0),
+				    name)) {
 		xe_guc_irq_handler(guc, GUC_INTR_SW_INT_0);

what if new MEMIRQ will arrive just here

Quoted 6 lines

is it ok that we will clear it immediately?

the whole memirq flow is that we clear irq byte first and then process it, so if anything comes right after we finish processing will be noticed on next iteration

Quoted 7 lines

I don't see why that matters in this cabuse. In either case multiple IRQs could happen and single IRQ handler runs.

Quoted 2 lines

I assume any races due to double migration shall be handled on the VF2GUC communication level while sending RESFIX_START/DONE, not here

I don't think it should be possible to get multiple RESFIX_START IRQs

I assume the SW_INT_0 is set on every migration, even in case we didn't have a chance to read it and clear it, or we didn't start (by sending RESFIX_START) nor finish (RESFIX_DONE)

Quoted 19 lines

before DONE is complete. This existing code upstream seems to handle these cases, so my series attempts to handle this too, but it seems like something that shouldn't be possible.

Quoted 15 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 489

Quoted 11 lines
drivers/gpu/drm/xe/xe_memirq.c +4 -0
+		iosys_map_wr(status, ilog2(GUC_INTR_SW_INT_0), u8, 0x00);
+	}
 }
 
 /**
@@ -460,6 +485,27 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe)
 	}
 }
 
+/**
+ * xe_memirq_vf_recovery_irq_pending() - VF recovery IRQ is pending

this function isn't really using anything VF specific except that on the VF the SW_INT_0 means "migrated"

Quoted 10 lines
Quoted 2 lines

maybe we can drop the _vf from function name?

Sure, so 'xe_memirq_recovery_irq_pending'?

Or 'xe_memirq_sw_int0_irq_pending'?

Quoted 1 line

xe_memirq_pending_guc(memirq, guc, bit)

I would make it more generic and provide bit param

note that xe_guc_irq_handler() also takes a bit

Quoted 53 lines
Quoted 18 lines
Quoted 5 lines
drivers/gpu/drm/xe/xe_memirq.c +5 -0
+ * @memirq: the &xe_memirq
+ * @guc: the &xe_guc to check for IRQ
+ *
+ * Return: True if VF recovery IRQ is pending on @guc, False otherwise
+ */

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 504

Quoted 10 lines
drivers/gpu/drm/xe/xe_memirq.c +10 -0
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc)
+{
+	struct xe_gt *gt = guc_to_gt(guc);
+	struct iosys_map map;
+
+	if (xe_gt_is_media_type(gt))
+		map = IOSYS_MAP_INIT_OFFSET(&memirq->status, ilog2(INTR_MGUC) * SZ_16);
+	else
+		map = IOSYS_MAP_INIT_OFFSET(&memirq->status, ilog2(INTR_GUC) * SZ_16);

nit: maybe just calc offset conditionally?

Quoted 2 lines
	u32 offset = is_media ? ilog2(INTR_MGUC) : ilog2(INTR_GUC);

Sure.

Quoted 6 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 506

Quoted 2 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+
+	return iosys_map_rd(&map, ilog2(GUC_INTR_SW_INT_0), u8);

as we have helpers we should use them

return memirq_received_noclear(...)

Sure.

Matt

Quoted 19 lines
Quoted 18 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+}
+
 /**
  * xe_memirq_handler - The `Memory Based Interrupts`_ Handler.
  * @memirq: the &xe_memirq
drivers/gpu/drm/xe/xe_memirq.h +3 -0
index 06130650e9d6..476b8cba179d 100644
@@ -25,4 +25,7 @@ void xe_memirq_handler(struct xe_memirq *memirq);
 
 int xe_memirq_init_guc(struct xe_memirq *memirq, struct xe_guc *guc);
 
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc);
+
 #endif
author maintainer
#27 lore raw

Re: [PATCH v2 11/34] drm/xe/vf: Add xe_gt_sriov_vf_recovery_inprogress helper

Quoted 77 lines · Michal Wajdeczko
Quoted 68 lines · Matthew Brost
Quoted 9 lines · Michal Wajdeczko
Quoted 3 lines · Matthew Brost

Add xe_gt_sriov_vf_recovery_inprogress helper.

This helper serves as the singular point to determine whether a VF

hmm, this "singular" looks like a GT-level only, not global

Yes, it is GT scoped. I will adjust the commit message.

Quoted 45 lines
Quoted 45 lines

post-migration recovery is currently in progress. Expected callers include the GuC CT layer and the GuC submission layer. Atomically visable as soon as vCPU are unhalted until VF recovery completes.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 17 ++++++++
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 10 +++++
 drivers/gpu/drm/xe/xe_memirq.c            | 48 ++++++++++++++++++++++-
 drivers/gpu/drm/xe/xe_memirq.h            |  3 ++
 5 files changed, 79 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
index 016c867e5e2b..c9d0e32e7a15 100644
@@ -26,6 +26,7 @@
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
 #include "xe_lrc.h"
+#include "xe_memirq.h"
 #include "xe_mmio.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
@@ -828,6 +829,7 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	struct xe_device *xe = gt_to_xe(gt);
 
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
+	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
 	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
 	/*
@@ -1172,3 +1174,18 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 	drm_printf(p, "\thandshake:\t%u.%u\n",
 		   pf_version->major, pf_version->minor);
 }
+
+/**
+ * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
+ * @gt: the &xe_gt
+ *
+ * Return: True if VF post migration recovery in progress, False otherwise
+ */
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1188

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+{
+	struct xe_memirq *memirq = &gt_to_tile(gt)->memirq;
+
+	return IS_SRIOV_VF(gt_to_xe(gt)) &&

this is xe_gt_sriov_vf function, so it is expected to be called only by the VF code, thus we should rather use xe_gt_assert here and the caller is responsible for the IS_SRIOV_VF check

Quoted 5 lines

That is not how I have coded this. I blindly call this in various places and I don't think it could at call site to determine if it is a VF as we have if (VF) statements all over the driver. If perf is the concern, we could move the IS_SRIOV_VF(gt_to_xe(gt)) part of function to static inline and reset of the function in an exported function.

Quoted 7 lines

inline will work, preferable not in xe_gt_sriov_vf.h ;)

static inline xe_gt_recovery_inprogress(gt)

{
	return IS_SRIOV_VF(xe) && xe_gt_sriov_vf_recovery_inprogress(gt);
}

+1

Quoted 102 lines
Quoted 94 lines
Quoted 53 lines
Quoted 53 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+		(xe_memirq_vf_recovery_irq_pending(memirq, &gt->uc.guc) ||
+		 READ_ONCE(gt->sriov.vf.migration.recovery_inprogress));
+}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index 0af1dc769fe0..bb5f8eace19b 100644
@@ -25,6 +25,8 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
+
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
 u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +10 -0
index d95857bd789b..7b10b8e1e10e 100644
@@ -49,6 +49,14 @@ struct xe_gt_sriov_vf_runtime {
 	} *regs;
 };
 
+/**
+ * xe_gt_sriov_vf_migration - VF migration data.
+ */
+struct xe_gt_sriov_vf_migration {
+	/** @recovery_inprogress: VF post migration recovery in progress */
+	bool recovery_inprogress;
+};
+
 /**
  * struct xe_gt_sriov_vf - GT level VF virtualization data.
  */
@@ -61,6 +69,8 @@ struct xe_gt_sriov_vf {
 	struct xe_gt_sriov_vf_selfconfig self_config;
 	/** @runtime: runtime data retrieved from the PF. */
 	struct xe_gt_sriov_vf_runtime runtime;
+	/** @migration: migration data for the VF. */
+	struct xe_gt_sriov_vf_migration migration;
 };
 
 #endif
drivers/gpu/drm/xe/xe_memirq.c +0 -0
index 49c45ec3e83c..94d5d6859aab 100644
@@ -398,6 +398,23 @@ void xe_memirq_postinstall(struct xe_memirq *memirq)
 		memirq_set_enable(memirq, true);
 }
 
Quoted 7 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_postinstall(struct xe_memirq *memirq) · line 403

Quoted 3 lines
drivers/gpu/drm/xe/xe_memirq.c +3 -0
+static bool memirq_received_noclear(struct xe_memirq *memirq,
+				    struct iosys_map *vector,
+				    u16 offset, const char *name)

maybe instead of duplicating code of memirq_received() in 90% just add there the "bool clear" flag?

Sure.

Quoted 31 lines
Quoted 18 lines
drivers/gpu/drm/xe/xe_memirq.c +14 -0
+{
+	u8 value;
+
+	value = iosys_map_rd(vector, offset, u8);
+	if (value) {
+		if (value != 0xff)
+			memirq_err_ratelimited(memirq,
+					       "Unexpected memirq value %#x from %s at %u\n",
+					       value, name, offset);
+	}
+
+	return value;
+}
+
 static bool memirq_received(struct xe_memirq *memirq, struct iosys_map *vector,
 			    u16 offset, const char *name)
 {

drivers/gpu/drm/xe/xe_memirq.c static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat · line 457

Quoted 8 lines
drivers/gpu/drm/xe/xe_memirq.c +4 -1
@@ -434,8 +451,16 @@ static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat
 	if (memirq_received(memirq, status, ilog2(GUC_INTR_GUC2HOST), name))
 		xe_guc_irq_handler(guc, GUC_INTR_GUC2HOST);
 
-	if (memirq_received(memirq, status, ilog2(GUC_INTR_SW_INT_0), name))
+	/*
+	 * We must wait to perform the clear operation until after
+	 * xe_gt_sriov_vf_start_migration_recovery() runs, to avoid race
+	 * conditions where xe_gt_sriov_vf_recovery_inprogress() returns false.

but the VF recovery "inprogress" shall be already set in the top level

xe_sriov_vf_start_migration_recovery()

Quoted 7 lines
Quoted 2 lines

even before the GT-level recovery starts, where is this race ?

If we clear the interrupt it here, this is before the IRQ handler is called which flips the software bit for "inprogress". There would be window where xe_gt_sriov_vf_recovery_inprogress could return false which is problematic.

Quoted 7 lines

but who calls that recovery_inprogress() and why should it be problematic?

note there might be still other threads that will finish some actions (including sending CTB) that will just be stuck there until we start the recovery, so even if recovery_inprogress() returns false for the small window that shouldn't change the picture

No. Read 'Waiters during VF post migration recovery' in [1]. That should explain the reason why this is required. Let me know if what is documented there is unclear.

[1] https://patchwork.freedesktop.org/patch/676374/?series=154627&rev=2

Quoted 18 lines
Quoted 18 lines
Quoted 8 lines

drivers/gpu/drm/xe/xe_memirq.c static void memirq_dispatch_guc(struct xe_memirq *memirq, struct iosys_map *stat · line 461

Quoted 5 lines
drivers/gpu/drm/xe/xe_memirq.c +3 -0
+	 */
+	if (memirq_received_noclear(memirq, status, ilog2(GUC_INTR_SW_INT_0),
+				    name)) {
 		xe_guc_irq_handler(guc, GUC_INTR_SW_INT_0);

what if new MEMIRQ will arrive just here

Quoted 6 lines

is it ok that we will clear it immediately?

the whole memirq flow is that we clear irq byte first and then process it, so if anything comes right after we finish processing will be noticed on next iteration

I don't see why that matters in this cabuse. In either case multiple IRQs could happen and single IRQ handler runs.

Quoted 9 lines
Quoted 4 lines
Quoted 2 lines

I assume any races due to double migration shall be handled on the VF2GUC communication level while sending RESFIX_START/DONE, not here

I don't think it should be possible to get multiple RESFIX_START IRQs

I assume the SW_INT_0 is set on every migration, even in case we didn't have a chance to read it and clear it, or we didn't start (by sending RESFIX_START) nor finish (RESFIX_DONE)

But can migration be triggered and then another one before the initial migration completes? I just don't see how that is possible or how it wouldn't break the world (i.e., The VF of GuC explodes somewhere).

Quoted 28 lines
Quoted 28 lines

before DONE is complete. This existing code upstream seems to handle these cases, so my series attempts to handle this too, but it seems like something that shouldn't be possible.

Quoted 9 lines
Quoted 9 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+		iosys_map_wr(status, ilog2(GUC_INTR_SW_INT_0), u8, 0x00);
+	}
 }
 
 /**
@@ -460,6 +485,27 @@ void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe)
 	}
 }
 
Quoted 8 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 489

Quoted 2 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+/**
+ * xe_memirq_vf_recovery_irq_pending() - VF recovery IRQ is pending

this function isn't really using anything VF specific except that on the VF the SW_INT_0 means "migrated"

maybe we can drop the _vf from function name?

Sure, so 'xe_memirq_recovery_irq_pending'?

Or 'xe_memirq_sw_int0_irq_pending'?

Quoted 6 lines
Quoted 1 line
Quoted 1 line

xe_memirq_pending_guc(memirq, guc, bit)

I would make it more generic and provide bit param

note that xe_guc_irq_handler() also takes a bit

Sure.

Matt

Quoted 55 lines
Quoted 54 lines
Quoted 19 lines
Quoted 6 lines
drivers/gpu/drm/xe/xe_memirq.c +5 -0
+ * @memirq: the &xe_memirq
+ * @guc: the &xe_guc to check for IRQ
+ *
+ * Return: True if VF recovery IRQ is pending on @guc, False otherwise
+ */

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 504

Quoted 10 lines
drivers/gpu/drm/xe/xe_memirq.c +10 -0
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc)
+{
+	struct xe_gt *gt = guc_to_gt(guc);
+	struct iosys_map map;
+
+	if (xe_gt_is_media_type(gt))
+		map = IOSYS_MAP_INIT_OFFSET(&memirq->status, ilog2(INTR_MGUC) * SZ_16);
+	else
+		map = IOSYS_MAP_INIT_OFFSET(&memirq->status, ilog2(INTR_GUC) * SZ_16);

nit: maybe just calc offset conditionally?

Quoted 2 lines
	u32 offset = is_media ? ilog2(INTR_MGUC) : ilog2(INTR_GUC);

Sure.

Quoted 6 lines

drivers/gpu/drm/xe/xe_memirq.c void xe_memirq_hwe_handler(struct xe_memirq *memirq, struct xe_hw_engine *hwe) · line 506

Quoted 2 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+
+	return iosys_map_rd(&map, ilog2(GUC_INTR_SW_INT_0), u8);

as we have helpers we should use them

return memirq_received_noclear(...)

Sure.

Matt

Quoted 19 lines
Quoted 18 lines
drivers/gpu/drm/xe/xe_memirq.c +2 -0
+}
+
 /**
  * xe_memirq_handler - The `Memory Based Interrupts`_ Handler.
  * @memirq: the &xe_memirq
drivers/gpu/drm/xe/xe_memirq.h +3 -0
index 06130650e9d6..476b8cba179d 100644
@@ -25,4 +25,7 @@ void xe_memirq_handler(struct xe_memirq *memirq);
 
 int xe_memirq_init_guc(struct xe_memirq *memirq, struct xe_guc *guc);
 
+bool xe_memirq_vf_recovery_irq_pending(struct xe_memirq *memirq,
+				       struct xe_guc *guc);
+
 #endif
author maintainer
#28 lore raw

PATCH v2 12/34 drm/xe/vf: Make VF recovery run on per-GT worker

In reply to Matthew Brost

VF recovery is a per-GT operation, so it makes sense to isolate it to a per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.

v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 170 ++++++++++++++-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   7 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          | 242 +---------------------
 drivers/gpu/drm/xe/xe_sriov_vf.h          |   1 -
 drivers/gpu/drm/xe/xe_sriov_vf_types.h    |   4 -
 6 files changed, 169 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +160 -10
index c9d0e32e7a15..cfb71b749e52 100644
@@ -25,11 +25,15 @@
 #include "xe_guc.h"
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
+#include "xe_guc_submit.h"
+#include "xe_irq.h"
 #include "xe_lrc.h"
 #include "xe_memirq.h"
 #include "xe_mmio.h"
+#include "xe_pm.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
+#include "xe_tile_sriov_vf.h"
 #include "xe_uc_fw.h"
 #include "xe_wopcm.h"
 
@@ -314,7 +318,7 @@ static int guc_action_vf_notify_resfix_done(struct xe_guc *guc)
  * Returns: 0 if the operation completed successfully, or a negative error
  * code otherwise.
  */
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
+static int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
 {
 	struct xe_guc *guc = &gt->uc.guc;
 	int err;
@@ -808,7 +812,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt)
  * xe_gt_sriov_vf_default_lrcs_hwsp_rebase - Update GGTT references in HWSP of default LRCs.
  * @gt: the &xe_gt struct instance
  */
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
+static void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 {
 	struct xe_hw_engine *hwe;
 	enum xe_hw_engine_id id;
@@ -817,6 +821,26 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 		xe_default_lrc_update_memirq_regs_with_address(hwe);
 }
 
+static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)
+{
+	bool started;
+
+	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
+
+	spin_lock(&gt->sriov.vf.migration.lock);
+
+	if (!gt->sriov.vf.migration.recovery_queued) {
+		gt->sriov.vf.migration.recovery_queued = true;
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+
+		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
+		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
+				 "scheduled" : "already in progress");
+	}
+
+	spin_unlock(&gt->sriov.vf.migration.lock);
+}
+
 /**
  * xe_gt_sriov_vf_migrated_event_handler - Start a VF migration recovery,
  *   or just mark that a GuC is ready for it.
@@ -831,15 +855,8 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
 	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
-	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
-	/*
-	 * We need to be certain that if all flags were set, at least one
-	 * thread will notice that and schedule the recovery.
-	 */
-	smp_mb__after_atomic();
-
 	xe_gt_sriov_info(gt, "ready for recovery after migration\n");
-	xe_sriov_vf_start_migration_recovery(xe);
+	xe_gt_sriov_vf_start_migration_recovery(gt);
 }
 
 static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
@@ -1175,6 +1192,139 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 		   pf_version->major, pf_version->minor);
 }
 
+static void vf_post_migration_shutdown(struct xe_gt *gt)
+{
+	int ret = 0;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_queued = false;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause(&gt->uc.guc);
+	ret |= xe_guc_submit_reset_block(&gt->uc.guc);
+
+	if (ret)
+		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");
+}
+
+static size_t post_migration_scratch_size(struct xe_device *xe)
+{
+	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
+}
+
+static int vf_post_migration_fixups(struct xe_gt *gt)
+{
+	s64 shift;
+	void *buf;
+	int err;
+
+	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
+	if (!buf)
+		return -ENOMEM;
+
+	err = xe_gt_sriov_vf_query_config(gt);
+	if (err)
+		goto out;
+
+	shift = xe_gt_sriov_vf_ggtt_shift(gt);
+	if (shift) {
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+		if (err)
+			goto out;
+	}
+
+out:
+	kfree(buf);
+	return err;
+}
+
+static void vf_post_migration_kickstart(struct xe_gt *gt)
+{
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	xe_irq_resume(gt_to_xe(gt));
+
+	xe_guc_submit_reset_unblock(&gt->uc.guc);
+	xe_guc_submit_unpause(&gt->uc.guc);
+}
+
+static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
+{
+	bool skip_resfix = false;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	if (gt->sriov.vf.migration.recovery_queued) {
+		skip_resfix = true;
+		xe_gt_sriov_dbg(gt, "another recovery imminent, skipped some notifications\n");
+	} else {
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	}
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);
+}
+
+static void vf_post_migration_recovery(struct xe_gt *gt)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	int err;
+
+	xe_gt_sriov_dbg(gt, "migration recovery in progress\n");
+
+	xe_pm_runtime_get(xe);
+	vf_post_migration_shutdown(gt);
+
+	if (!xe_sriov_vf_migration_supported(xe)) {
+		xe_gt_sriov_err(gt, "migration is not supported\n");
+		err = -ENOTRECOVERABLE;
+		goto fail;
+	}
+
+	err = vf_post_migration_fixups(gt);
+	if (err)
+		goto fail;
+
+	vf_post_migration_kickstart(gt);
+	err = vf_post_migration_notify_resfix_done(gt);
+	if (err && err != -EAGAIN)
+		goto fail;
+
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_notice(gt, "migration recovery ended\n");
+	return;
+fail:
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
+	xe_device_declare_wedged(xe);
+}
+
+static void migration_worker_func(struct work_struct *w)
+{
+	struct xe_gt *gt = container_of(w, struct xe_gt,
+					sriov.vf.migration.worker);
+
+	vf_post_migration_recovery(gt);
+}
+
+/**
+ * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
+ * @gt: the &xe_gt
+ */
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+{
+	init_rwsem(&gt->sriov.vf.self_config.lock);
+	spin_lock_init(&gt->sriov.vf.migration.lock);
+	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
+
+	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
+		xe_gt_sriov_info(gt, "migration not supported by this module version\n");
+}
+
 /**
  * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
  * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index bb5f8eace19b..2ac6775b52f0 100644
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
 int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
 int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
 bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
 
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
index 7b10b8e1e10e..53680a2f188a 100644
@@ -8,6 +8,7 @@
 
 #include <linux/rwsem.h>
 #include <linux/types.h>
+#include <linux/workqueue.h>
 #include "xe_uc_fw_types.h"
 
 /**
@@ -53,6 +54,12 @@ struct xe_gt_sriov_vf_runtime {
  * xe_gt_sriov_vf_migration - VF migration data.
  */
 struct xe_gt_sriov_vf_migration {
+	/** @migration: VF migration recovery worker */
+	struct work_struct worker;
+	/** @lock: Protects recovery_queued */
+	spinlock_t lock;
+	/** @recovery_queued: VF post migration recovery in queued */
+	bool recovery_queued;
 	/** @recovery_inprogress: VF post migration recovery in progress */
 	bool recovery_inprogress;
 };
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -241

276 lines not on this page · raw message

drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
index 9e752105ec2a..4df95266b261 100644
@@ -13,7 +13,6 @@ struct xe_device;
 
 void xe_sriov_vf_init_early(struct xe_device *xe);
 int xe_sriov_vf_init_late(struct xe_device *xe);
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe);
 bool xe_sriov_vf_migration_supported(struct xe_device *xe);
 void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
 
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
index 426cc5841958..6a0fd0f5463e 100644
@@ -33,10 +33,6 @@ struct xe_device_vf {
 
 	/** @migration: VF Migration state data */
 	struct {
-		/** @migration.worker: VF migration recovery worker */
-		struct work_struct worker;
-		/** @migration.gt_flags: Per-GT request flags for VF migration recovery */
-		unsigned long gt_flags;
 		/**
 		 * @migration.enabled: flag indicating if migration support
 		 * was enabled or not due to missing prerequisites
Signature · 1 line
2.34.1
#29 lore raw

Re: [PATCH v2 12/34] drm/xe/vf: Make VF recovery run on per-GT worker

Quoted 1 line · Matthew Brost

VF recovery is a per-GT operation, so it makes sense to isolate it to a

that was also my suggestion to make it per-GT, good to see it happen now

Quoted 3 lines

per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.

but while the recovery is per-GT, we should still protect against that one GT starts the recovery sooner then other GTs notice about it

Quoted 54 lines

v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 170 ++++++++++++++-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   7 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          | 242 +---------------------
 drivers/gpu/drm/xe/xe_sriov_vf.h          |   1 -
 drivers/gpu/drm/xe/xe_sriov_vf_types.h    |   4 -
 6 files changed, 169 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -2
index c9d0e32e7a15..cfb71b749e52 100644
@@ -25,11 +25,15 @@
 #include "xe_guc.h"
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
+#include "xe_guc_submit.h"
+#include "xe_irq.h"
 #include "xe_lrc.h"
 #include "xe_memirq.h"
 #include "xe_mmio.h"
+#include "xe_pm.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
+#include "xe_tile_sriov_vf.h"
 #include "xe_uc_fw.h"
 #include "xe_wopcm.h"
 
@@ -314,7 +318,7 @@ static int guc_action_vf_notify_resfix_done(struct xe_guc *guc)
  * Returns: 0 if the operation completed successfully, or a negative error
  * code otherwise.
  */
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
+static int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
 {
 	struct xe_guc *guc = &gt->uc.guc;
 	int err;
@@ -808,7 +812,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt)
  * xe_gt_sriov_vf_default_lrcs_hwsp_rebase - Update GGTT references in HWSP of default LRCs.
  * @gt: the &xe_gt struct instance
  */
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
+static void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 {
 	struct xe_hw_engine *hwe;
 	enum xe_hw_engine_id id;

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 824

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -817,6 +821,26 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 		xe_default_lrc_update_memirq_regs_with_address(hwe);
 }
 
+static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)

nit: if this is static then could be just:

vf_start_migration_recovery(gt)

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+{
+	bool started;
+
+	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 838

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+
+	spin_lock(&gt->sriov.vf.migration.lock);
+
+	if (!gt->sriov.vf.migration.recovery_queued) {
+		gt->sriov.vf.migration.recovery_queued = true;
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+
+		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
+		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
+				 "scheduled" : "already in progress");

with this .recovery_queued flag, can we hit "already in progress" case?

Quoted 29 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -8
+	}
+
+	spin_unlock(&gt->sriov.vf.migration.lock);
+}
+
 /**
  * xe_gt_sriov_vf_migrated_event_handler - Start a VF migration recovery,
  *   or just mark that a GuC is ready for it.
@@ -831,15 +855,8 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
 	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
-	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
-	/*
-	 * We need to be certain that if all flags were set, at least one
-	 * thread will notice that and schedule the recovery.
-	 */
-	smp_mb__after_atomic();
-
 	xe_gt_sriov_info(gt, "ready for recovery after migration\n");
-	xe_sriov_vf_start_migration_recovery(xe);
+	xe_gt_sriov_vf_start_migration_recovery(gt);
 }
 
 static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
@@ -1175,6 +1192,139 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 		   pf_version->major, pf_version->minor);
 }
 

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1204

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+static void vf_post_migration_shutdown(struct xe_gt *gt)
+{
+	int ret = 0;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_queued = false;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause(&gt->uc.guc);
+	ret |= xe_guc_submit_reset_block(&gt->uc.guc);

this |= seem unneeded

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1207

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+
+	if (ret)
+		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");

is this the only possible reason? maybe worth to add %pe ?

Quoted 52 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +52 -0
+}
+
+static size_t post_migration_scratch_size(struct xe_device *xe)
+{
+	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
+}
+
+static int vf_post_migration_fixups(struct xe_gt *gt)
+{
+	s64 shift;
+	void *buf;
+	int err;
+
+	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
+	if (!buf)
+		return -ENOMEM;
+
+	err = xe_gt_sriov_vf_query_config(gt);
+	if (err)
+		goto out;
+
+	shift = xe_gt_sriov_vf_ggtt_shift(gt);
+	if (shift) {
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+		if (err)
+			goto out;
+	}
+
+out:
+	kfree(buf);
+	return err;
+}
+
+static void vf_post_migration_kickstart(struct xe_gt *gt)
+{
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	xe_irq_resume(gt_to_xe(gt));
+
+	xe_guc_submit_reset_unblock(&gt->uc.guc);
+	xe_guc_submit_unpause(&gt->uc.guc);
+}
+
+static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
+{
+	bool skip_resfix = false;
+

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1269

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	if (gt->sriov.vf.migration.recovery_queued) {
+		skip_resfix = true;
+		xe_gt_sriov_dbg(gt, "another recovery imminent, skipped some notifications\n");
+	} else {
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	}
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);

nit: this looks cleaner:

	if (skip_resfix)
		return -EAGAIN;

	return xe_gt_sriov_vf_notify_resfix_done(gt);
Quoted 46 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +46 -0
+}
+
+static void vf_post_migration_recovery(struct xe_gt *gt)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	int err;
+
+	xe_gt_sriov_dbg(gt, "migration recovery in progress\n");
+
+	xe_pm_runtime_get(xe);
+	vf_post_migration_shutdown(gt);
+
+	if (!xe_sriov_vf_migration_supported(xe)) {
+		xe_gt_sriov_err(gt, "migration is not supported\n");
+		err = -ENOTRECOVERABLE;
+		goto fail;
+	}
+
+	err = vf_post_migration_fixups(gt);
+	if (err)
+		goto fail;
+
+	vf_post_migration_kickstart(gt);
+	err = vf_post_migration_notify_resfix_done(gt);
+	if (err && err != -EAGAIN)
+		goto fail;
+
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_notice(gt, "migration recovery ended\n");
+	return;
+fail:
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
+	xe_device_declare_wedged(xe);
+}
+
+static void migration_worker_func(struct work_struct *w)
+{
+	struct xe_gt *gt = container_of(w, struct xe_gt,
+					sriov.vf.migration.worker);
+
+	vf_post_migration_recovery(gt);
+}
+
+/**
+ * xe_gt_sriov_vf_migration_init_early() - VF post migration init early

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1325

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+ * @gt: the &xe_gt
+ */
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+{
+	init_rwsem(&gt->sriov.vf.self_config.lock);
+	spin_lock_init(&gt->sriov.vf.migration.lock);
+	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
+
+	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
+		xe_gt_sriov_info(gt, "migration not supported by this module version\n");

we likely don't want to repeat that message on every GT

Quoted 91 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
+}
+
 /**
  * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
  * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index bb5f8eace19b..2ac6775b52f0 100644
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
 int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
 int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
 bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
 
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
index 7b10b8e1e10e..53680a2f188a 100644
@@ -8,6 +8,7 @@
 
 #include <linux/rwsem.h>
 #include <linux/types.h>
+#include <linux/workqueue.h>
 #include "xe_uc_fw_types.h"
 
 /**
@@ -53,6 +54,12 @@ struct xe_gt_sriov_vf_runtime {
  * xe_gt_sriov_vf_migration - VF migration data.
  */
 struct xe_gt_sriov_vf_migration {
+	/** @migration: VF migration recovery worker */
+	struct work_struct worker;
+	/** @lock: Protects recovery_queued */
+	spinlock_t lock;
+	/** @recovery_queued: VF post migration recovery in queued */
+	bool recovery_queued;
 	/** @recovery_inprogress: VF post migration recovery in progress */
 	bool recovery_inprogress;
 };
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -13
index da064a1e7419..7d91553c4acc 100644
@@ -6,21 +6,12 @@
 #include <drm/drm_debugfs.h>
 #include <drm/drm_managed.h>
 
-#include "xe_assert.h"
-#include "xe_device.h"
 #include "xe_gt.h"
-#include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
-#include "xe_guc_submit.h"
-#include "xe_irq.h"
-#include "xe_lrc.h"
-#include "xe_pm.h"
-#include "xe_sriov.h"
 #include "xe_sriov_printk.h"
 #include "xe_sriov_vf.h"
 #include "xe_sriov_vf_ccs.h"
-#include "xe_tile_sriov_vf.h"
 
 /**
  * DOC: VF restore procedure in PF KMD and VF KMD
@@ -158,8 +149,6 @@ static void vf_disable_migration(struct xe_device *xe, const char *fmt, ...)
 	xe->sriov.vf.migration.enabled = false;
 }
 
-static void migration_worker_func(struct work_struct *w);
-
 static void vf_migration_init_early(struct xe_device *xe)
 {
 	/*
@@ -184,8 +173,6 @@ static void vf_migration_init_early(struct xe_device *xe)
 						    guc_version.major, guc_version.minor);
 	}
 
-	INIT_WORK(&xe->sriov.vf.migration.worker, migration_worker_func);
-
 	xe->sriov.vf.migration.enabled = true;
 	xe_sriov_dbg(xe, "migration support enabled\n");
 }

drivers/gpu/drm/xe/xe_sriov_vf.c void xe_sriov_vf_init_early(struct xe_device *xe) · line 190

Quoted 5 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -1
@@ -200,238 +187,11 @@ void xe_sriov_vf_init_early(struct xe_device *xe)
 	unsigned int id;
 
 	for_each_gt(gt, xe, id)
-		init_rwsem(&gt->sriov.vf.self_config.lock);
+		xe_gt_sriov_vf_migration_init_early(gt);

still, this should be called from gt_init_early kind of functions

Quoted 261 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -227
 	vf_migration_init_early(xe);
 }
 
-/**
- * vf_post_migration_shutdown - Stop the driver activities after VF migration.
- * @xe: the &xe_device struct instance
- *
- * After this VM is migrated and assigned to a new VF, it is running on a new
- * hardware, and therefore many hardware-dependent states and related structures
- * require fixups. Without fixups, the hardware cannot do any work, and therefore
- * all GPU pipelines are stalled.
- * Stop some of kernel activities to make the fixup process faster.
- */
-static void vf_post_migration_shutdown(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int ret = 0;
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_pause(&gt->uc.guc);
-		ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-	}
-
-	if (ret)
-		drm_info(&xe->drm, "migration recovery encountered ongoing reset\n");
-}
-
-/**
- * vf_post_migration_kickstart - Re-start the driver activities under new hardware.
- * @xe: the &xe_device struct instance
- *
- * After we have finished with all post-migration fixups, restart the driver
- * activities to continue feeding the GPU with workloads.
- */
-static void vf_post_migration_kickstart(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	/*
-	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
-	 * must be working at this point, since the recovery did started,
-	 * but the rest was not enabled using the procedure from spec.
-	 */
-	xe_irq_resume(xe);
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_reset_unblock(&gt->uc.guc);
-		xe_guc_submit_unpause(&gt->uc.guc);
-	}
-}
-
-static bool gt_vf_post_migration_needed(struct xe_gt *gt)
-{
-	return test_bit(gt->info.id, &gt_to_xe(gt)->sriov.vf.migration.gt_flags);
-}
-
-/*
- * Notify GuCs marked in flags about resource fixups apply finished.
- * @xe: the &xe_device struct instance
- * @gt_flags: flags marking to which GTs the notification shall be sent
- */
-static int vf_post_migration_notify_resfix_done(struct xe_device *xe, unsigned long gt_flags)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int err = 0;
-
-	for_each_gt(gt, xe, id) {
-		if (!test_bit(id, &gt_flags))
-			continue;
-		/* skip asking GuC for RESFIX exit if new recovery request arrived */
-		if (gt_vf_post_migration_needed(gt))
-			continue;
-		err = xe_gt_sriov_vf_notify_resfix_done(gt);
-		if (err)
-			break;
-		clear_bit(id, &gt_flags);
-	}
-
-	if (gt_flags && !err)
-		drm_dbg(&xe->drm, "another recovery imminent, skipped some notifications\n");
-	return err;
-}
-
-static int vf_get_next_migrated_gt_id(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_and_clear_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return id;
-	}
-	return -1;
-}
-
-static size_t post_migration_scratch_size(struct xe_device *xe)
-{
-	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
-}
-
-/**
- * Perform post-migration fixups on a single GT.
- *
- * After migration, GuC needs to be re-queried for VF configuration to check
- * if it matches previous provisioning. Most of VF provisioning shall be the
- * same, except GGTT range, since GGTT is not virtualized per-VF. If GGTT
- * range has changed, we have to perform fixups - shift all GGTT references
- * used anywhere within the driver. After the fixups in this function succeed,
- * it is allowed to ask the GuC bound to this GT to continue normal operation.
- *
- * Returns: 0 if the operation completed successfully, or a negative error
- * code otherwise.
- */
-static int gt_vf_post_migration_fixups(struct xe_gt *gt)
-{
-	s64 shift;
-	void *buf;
-	int err;
-
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_KERNEL);
-	if (!buf)
-		return -ENOMEM;
-
-	err = xe_gt_sriov_vf_query_config(gt);
-	if (err)
-		goto out;
-
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			goto out;
-	}
-
-out:
-	kfree(buf);
-	return err;
-}
-
-static void vf_post_migration_recovery(struct xe_device *xe)
-{
-	unsigned long fixed_gts = 0;
-	int id, err;
-
-	drm_dbg(&xe->drm, "migration recovery in progress\n");
-	xe_pm_runtime_get(xe);
-	vf_post_migration_shutdown(xe);
-
-	if (!xe_sriov_vf_migration_supported(xe)) {
-		xe_sriov_err(xe, "migration is not supported\n");
-		err = -ENOTRECOVERABLE;
-		goto fail;
-	}
-
-	while (id = vf_get_next_migrated_gt_id(xe), id >= 0) {
-		struct xe_gt *gt = xe_device_get_gt(xe, id);
-
-		err = gt_vf_post_migration_fixups(gt);
-		if (err)
-			goto fail;
-
-		set_bit(id, &fixed_gts);
-	}
-
-	vf_post_migration_kickstart(xe);
-	err = vf_post_migration_notify_resfix_done(xe, fixed_gts);
-	if (err)
-		goto fail;
-
-	xe_pm_runtime_put(xe);
-	drm_notice(&xe->drm, "migration recovery ended\n");
-	return;
-fail:
-	xe_pm_runtime_put(xe);
-	drm_err(&xe->drm, "migration recovery failed (%pe)\n", ERR_PTR(err));
-	xe_device_declare_wedged(xe);
-}
-
-static void migration_worker_func(struct work_struct *w)
-{
-	struct xe_device *xe = container_of(w, struct xe_device,
-					    sriov.vf.migration.worker);
-
-	vf_post_migration_recovery(xe);
-}
-
-/*
- * Check if post-restore recovery is coming on any of GTs.
- * @xe: the &xe_device struct instance
- *
- * Return: True if migration recovery worker will soon be running. Any worker currently
- * executing does not affect the result.
- */
-static bool vf_ready_to_recovery_on_any_gts(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return true;
-	}
-	return false;
-}
-
-/**
- * xe_sriov_vf_start_migration_recovery - Start VF migration recovery.
- * @xe: the &xe_device to start recovery on
- *
- * This function shall be called only by VF.
- */
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe)
-{
-	bool started;
-
-	xe_assert(xe, IS_SRIOV_VF(xe));
-
-	if (!vf_ready_to_recovery_on_any_gts(xe))
-		return;
-
-	started = queue_work(xe->sriov.wq, &xe->sriov.vf.migration.worker);
-	drm_info(&xe->drm, "VF migration recovery %s\n", started ?
-		 "scheduled" : "already in progress");
-}
-
 /**
  * xe_sriov_vf_init_late() - SR-IOV VF late initialization functions.
  * @xe: the &xe_device to initialize
drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
index 9e752105ec2a..4df95266b261 100644
@@ -13,7 +13,6 @@ struct xe_device;
 
 void xe_sriov_vf_init_early(struct xe_device *xe);
 int xe_sriov_vf_init_late(struct xe_device *xe);
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe);
 bool xe_sriov_vf_migration_supported(struct xe_device *xe);
 void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
 
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
index 426cc5841958..6a0fd0f5463e 100644
@@ -33,10 +33,6 @@ struct xe_device_vf {
 
 	/** @migration: VF Migration state data */
 	struct {
-		/** @migration.worker: VF migration recovery worker */
-		struct work_struct worker;
-		/** @migration.gt_flags: Per-GT request flags for VF migration recovery */
-		unsigned long gt_flags;
 		/**
 		 * @migration.enabled: flag indicating if migration support
 		 * was enabled or not due to missing prerequisites
author maintainer
#30 lore raw

Re: [PATCH v2 12/34] drm/xe/vf: Make VF recovery run on per-GT worker

Quoted 7 lines · Michal Wajdeczko
Quoted 1 line · Matthew Brost

VF recovery is a per-GT operation, so it makes sense to isolate it to a

that was also my suggestion to make it per-GT, good to see it happen now

+1

Quoted 7 lines
Quoted 3 lines

per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.

but while the recovery is per-GT, we should still protect against that one GT starts the recovery sooner then other GTs notice about it

Yes. There is shared state in 2 places:

- The GGTT shifting, this handled by [1] in my series.
- CCS restore on iGPU (PTL) handled by [2] [3] in my series.

[1] https://patchwork.freedesktop.org/patch/676394/?series=154627&rev=2
[2] https://patchwork.freedesktop.org/patch/676393/?series=154627&rev=2
[3] https://patchwork.freedesktop.org/patch/676397/?series=154627&rev=2

Quoted 57 lines
Quoted 57 lines

v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 170 ++++++++++++++-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   7 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          | 242 +---------------------
 drivers/gpu/drm/xe/xe_sriov_vf.h          |   1 -
 drivers/gpu/drm/xe/xe_sriov_vf_types.h    |   4 -
 6 files changed, 169 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -2
index c9d0e32e7a15..cfb71b749e52 100644
@@ -25,11 +25,15 @@
 #include "xe_guc.h"
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
+#include "xe_guc_submit.h"
+#include "xe_irq.h"
 #include "xe_lrc.h"
 #include "xe_memirq.h"
 #include "xe_mmio.h"
+#include "xe_pm.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
+#include "xe_tile_sriov_vf.h"
 #include "xe_uc_fw.h"
 #include "xe_wopcm.h"
 
@@ -314,7 +318,7 @@ static int guc_action_vf_notify_resfix_done(struct xe_guc *guc)
  * Returns: 0 if the operation completed successfully, or a negative error
  * code otherwise.
  */
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
+static int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
 {
 	struct xe_guc *guc = &gt->uc.guc;
 	int err;
@@ -808,7 +812,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt)
  * xe_gt_sriov_vf_default_lrcs_hwsp_rebase - Update GGTT references in HWSP of default LRCs.
  * @gt: the &xe_gt struct instance
  */
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
+static void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 {
 	struct xe_hw_engine *hwe;
 	enum xe_hw_engine_id id;
@@ -817,6 +821,26 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 		xe_default_lrc_update_memirq_regs_with_address(hwe);
 }
 
Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 824

Quoted 1 line
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)

nit: if this is static then could be just:

vf_start_migration_recovery(gt)

Sure.

Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +8 -0
+{
+	bool started;
+
+	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
+
+	spin_lock(&gt->sriov.vf.migration.lock);
+
+	if (!gt->sriov.vf.migration.recovery_queued) {
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 838

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -0
+		gt->sriov.vf.migration.recovery_queued = true;
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+
+		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
+		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
+				 "scheduled" : "already in progress");

with this .recovery_queued flag, can we hit "already in progress" case?

This was existing code so kept it but I'm really unclear how we can get multiple resfix IRQs without the prior resfix flow being completed. Maybe Tomasz can clear this one up? Ideally I'd like to remove multiple resfix IRQ handled code if this is not possible.

Quoted 29 lines
Quoted 29 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -8
+	}
+
+	spin_unlock(&gt->sriov.vf.migration.lock);
+}
+
 /**
  * xe_gt_sriov_vf_migrated_event_handler - Start a VF migration recovery,
  *   or just mark that a GuC is ready for it.
@@ -831,15 +855,8 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
 	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
-	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
-	/*
-	 * We need to be certain that if all flags were set, at least one
-	 * thread will notice that and schedule the recovery.
-	 */
-	smp_mb__after_atomic();
-
 	xe_gt_sriov_info(gt, "ready for recovery after migration\n");
-	xe_sriov_vf_start_migration_recovery(xe);
+	xe_gt_sriov_vf_start_migration_recovery(gt);
 }
 
 static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
@@ -1175,6 +1192,139 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 		   pf_version->major, pf_version->minor);
 }
 
Quoted 13 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1204

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+static void vf_post_migration_shutdown(struct xe_gt *gt)
+{
+	int ret = 0;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_queued = false;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause(&gt->uc.guc);
+	ret |= xe_guc_submit_reset_block(&gt->uc.guc);

this |= seem unneeded

This is existing code copy / pasted and is removed in [4].

[4] https://patchwork.freedesktop.org/patch/676382/?series=154627&rev=2

Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1207

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+
+	if (ret)
+		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");

is this the only possible reason? maybe worth to add %pe ?

Again this existing code and will be removed in [4].

So with above statements, I'd prefer just to leave as is.

Quoted 65 lines
Quoted 52 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +52 -0
+}
+
+static size_t post_migration_scratch_size(struct xe_device *xe)
+{
+	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
+}
+
+static int vf_post_migration_fixups(struct xe_gt *gt)
+{
+	s64 shift;
+	void *buf;
+	int err;
+
+	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
+	if (!buf)
+		return -ENOMEM;
+
+	err = xe_gt_sriov_vf_query_config(gt);
+	if (err)
+		goto out;
+
+	shift = xe_gt_sriov_vf_ggtt_shift(gt);
+	if (shift) {
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+		if (err)
+			goto out;
+	}
+
+out:
+	kfree(buf);
+	return err;
+}
+
+static void vf_post_migration_kickstart(struct xe_gt *gt)
+{
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	xe_irq_resume(gt_to_xe(gt));
+
+	xe_guc_submit_reset_unblock(&gt->uc.guc);
+	xe_guc_submit_unpause(&gt->uc.guc);
+}
+
+static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
+{
+	bool skip_resfix = false;
+

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1269

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	if (gt->sriov.vf.migration.recovery_queued) {
+		skip_resfix = true;
+		xe_gt_sriov_dbg(gt, "another recovery imminent, skipped some notifications\n");
+	} else {
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	}
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);

nit: this looks cleaner:

Quoted 5 lines
	if (skip_resfix)
		return -EAGAIN;

	return xe_gt_sriov_vf_notify_resfix_done(gt);

Sure.

Quoted 50 lines
Quoted 50 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +50 -0
+}
+
+static void vf_post_migration_recovery(struct xe_gt *gt)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	int err;
+
+	xe_gt_sriov_dbg(gt, "migration recovery in progress\n");
+
+	xe_pm_runtime_get(xe);
+	vf_post_migration_shutdown(gt);
+
+	if (!xe_sriov_vf_migration_supported(xe)) {
+		xe_gt_sriov_err(gt, "migration is not supported\n");
+		err = -ENOTRECOVERABLE;
+		goto fail;
+	}
+
+	err = vf_post_migration_fixups(gt);
+	if (err)
+		goto fail;
+
+	vf_post_migration_kickstart(gt);
+	err = vf_post_migration_notify_resfix_done(gt);
+	if (err && err != -EAGAIN)
+		goto fail;
+
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_notice(gt, "migration recovery ended\n");
+	return;
+fail:
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
+	xe_device_declare_wedged(xe);
+}
+
+static void migration_worker_func(struct work_struct *w)
+{
+	struct xe_gt *gt = container_of(w, struct xe_gt,
+					sriov.vf.migration.worker);
+
+	vf_post_migration_recovery(gt);
+}
+
+/**
+ * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
+ * @gt: the &xe_gt
+ */
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+{
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1325

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -0
+	init_rwsem(&gt->sriov.vf.self_config.lock);
+	spin_lock_init(&gt->sriov.vf.migration.lock);
+	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
+
+	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
+		xe_gt_sriov_info(gt, "migration not supported by this module version\n");

we likely don't want to repeat that message on every GT

So move this to xe_sriov_vf_init_early?

Quoted 93 lines
Quoted 93 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
+}
+
 /**
  * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
  * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index bb5f8eace19b..2ac6775b52f0 100644
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
 int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
 int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
 bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
 
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
index 7b10b8e1e10e..53680a2f188a 100644
@@ -8,6 +8,7 @@
 
 #include <linux/rwsem.h>
 #include <linux/types.h>
+#include <linux/workqueue.h>
 #include "xe_uc_fw_types.h"
 
 /**
@@ -53,6 +54,12 @@ struct xe_gt_sriov_vf_runtime {
  * xe_gt_sriov_vf_migration - VF migration data.
  */
 struct xe_gt_sriov_vf_migration {
+	/** @migration: VF migration recovery worker */
+	struct work_struct worker;
+	/** @lock: Protects recovery_queued */
+	spinlock_t lock;
+	/** @recovery_queued: VF post migration recovery in queued */
+	bool recovery_queued;
 	/** @recovery_inprogress: VF post migration recovery in progress */
 	bool recovery_inprogress;
 };
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -13
index da064a1e7419..7d91553c4acc 100644
@@ -6,21 +6,12 @@
 #include <drm/drm_debugfs.h>
 #include <drm/drm_managed.h>
 
-#include "xe_assert.h"
-#include "xe_device.h"
 #include "xe_gt.h"
-#include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
-#include "xe_guc_submit.h"
-#include "xe_irq.h"
-#include "xe_lrc.h"
-#include "xe_pm.h"
-#include "xe_sriov.h"
 #include "xe_sriov_printk.h"
 #include "xe_sriov_vf.h"
 #include "xe_sriov_vf_ccs.h"
-#include "xe_tile_sriov_vf.h"
 
 /**
  * DOC: VF restore procedure in PF KMD and VF KMD
@@ -158,8 +149,6 @@ static void vf_disable_migration(struct xe_device *xe, const char *fmt, ...)
 	xe->sriov.vf.migration.enabled = false;
 }
 
-static void migration_worker_func(struct work_struct *w);
-
 static void vf_migration_init_early(struct xe_device *xe)
 {
 	/*
@@ -184,8 +173,6 @@ static void vf_migration_init_early(struct xe_device *xe)
 						    guc_version.major, guc_version.minor);
 	}
 
-	INIT_WORK(&xe->sriov.vf.migration.worker, migration_worker_func);
-
 	xe->sriov.vf.migration.enabled = true;
 	xe_sriov_dbg(xe, "migration support enabled\n");
 }
@@ -200,238 +187,11 @@ void xe_sriov_vf_init_early(struct xe_device *xe)
 	unsigned int id;
 
Quoted 6 lines

drivers/gpu/drm/xe/xe_sriov_vf.c void xe_sriov_vf_init_early(struct xe_device *xe) · line 190

Quoted 3 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -1
 	for_each_gt(gt, xe, id)
-		init_rwsem(&gt->sriov.vf.self_config.lock);
+		xe_gt_sriov_vf_migration_init_early(gt);

still, this should be called from gt_init_early kind of functions

Kinda a nit that I'm not convinced is worth while to have xe_sriov_vf_init_early and then xe_gt_sriov_vf_migration_init_early called in gt_init_early...

Matt

Quoted 262 lines
Quoted 261 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -227
 	vf_migration_init_early(xe);
 }
 
-/**
- * vf_post_migration_shutdown - Stop the driver activities after VF migration.
- * @xe: the &xe_device struct instance
- *
- * After this VM is migrated and assigned to a new VF, it is running on a new
- * hardware, and therefore many hardware-dependent states and related structures
- * require fixups. Without fixups, the hardware cannot do any work, and therefore
- * all GPU pipelines are stalled.
- * Stop some of kernel activities to make the fixup process faster.
- */
-static void vf_post_migration_shutdown(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int ret = 0;
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_pause(&gt->uc.guc);
-		ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-	}
-
-	if (ret)
-		drm_info(&xe->drm, "migration recovery encountered ongoing reset\n");
-}
-
-/**
- * vf_post_migration_kickstart - Re-start the driver activities under new hardware.
- * @xe: the &xe_device struct instance
- *
- * After we have finished with all post-migration fixups, restart the driver
- * activities to continue feeding the GPU with workloads.
- */
-static void vf_post_migration_kickstart(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	/*
-	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
-	 * must be working at this point, since the recovery did started,
-	 * but the rest was not enabled using the procedure from spec.
-	 */
-	xe_irq_resume(xe);
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_reset_unblock(&gt->uc.guc);
-		xe_guc_submit_unpause(&gt->uc.guc);
-	}
-}
-
-static bool gt_vf_post_migration_needed(struct xe_gt *gt)
-{
-	return test_bit(gt->info.id, &gt_to_xe(gt)->sriov.vf.migration.gt_flags);
-}
-
-/*
- * Notify GuCs marked in flags about resource fixups apply finished.
- * @xe: the &xe_device struct instance
- * @gt_flags: flags marking to which GTs the notification shall be sent
- */
-static int vf_post_migration_notify_resfix_done(struct xe_device *xe, unsigned long gt_flags)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int err = 0;
-
-	for_each_gt(gt, xe, id) {
-		if (!test_bit(id, &gt_flags))
-			continue;
-		/* skip asking GuC for RESFIX exit if new recovery request arrived */
-		if (gt_vf_post_migration_needed(gt))
-			continue;
-		err = xe_gt_sriov_vf_notify_resfix_done(gt);
-		if (err)
-			break;
-		clear_bit(id, &gt_flags);
-	}
-
-	if (gt_flags && !err)
-		drm_dbg(&xe->drm, "another recovery imminent, skipped some notifications\n");
-	return err;
-}
-
-static int vf_get_next_migrated_gt_id(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_and_clear_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return id;
-	}
-	return -1;
-}
-
-static size_t post_migration_scratch_size(struct xe_device *xe)
-{
-	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
-}
-
-/**
- * Perform post-migration fixups on a single GT.
- *
- * After migration, GuC needs to be re-queried for VF configuration to check
- * if it matches previous provisioning. Most of VF provisioning shall be the
- * same, except GGTT range, since GGTT is not virtualized per-VF. If GGTT
- * range has changed, we have to perform fixups - shift all GGTT references
- * used anywhere within the driver. After the fixups in this function succeed,
- * it is allowed to ask the GuC bound to this GT to continue normal operation.
- *
- * Returns: 0 if the operation completed successfully, or a negative error
- * code otherwise.
- */
-static int gt_vf_post_migration_fixups(struct xe_gt *gt)
-{
-	s64 shift;
-	void *buf;
-	int err;
-
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_KERNEL);
-	if (!buf)
-		return -ENOMEM;
-
-	err = xe_gt_sriov_vf_query_config(gt);
-	if (err)
-		goto out;
-
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			goto out;
-	}
-
-out:
-	kfree(buf);
-	return err;
-}
-
-static void vf_post_migration_recovery(struct xe_device *xe)
-{
-	unsigned long fixed_gts = 0;
-	int id, err;
-
-	drm_dbg(&xe->drm, "migration recovery in progress\n");
-	xe_pm_runtime_get(xe);
-	vf_post_migration_shutdown(xe);
-
-	if (!xe_sriov_vf_migration_supported(xe)) {
-		xe_sriov_err(xe, "migration is not supported\n");
-		err = -ENOTRECOVERABLE;
-		goto fail;
-	}
-
-	while (id = vf_get_next_migrated_gt_id(xe), id >= 0) {
-		struct xe_gt *gt = xe_device_get_gt(xe, id);
-
-		err = gt_vf_post_migration_fixups(gt);
-		if (err)
-			goto fail;
-
-		set_bit(id, &fixed_gts);
-	}
-
-	vf_post_migration_kickstart(xe);
-	err = vf_post_migration_notify_resfix_done(xe, fixed_gts);
-	if (err)
-		goto fail;
-
-	xe_pm_runtime_put(xe);
-	drm_notice(&xe->drm, "migration recovery ended\n");
-	return;
-fail:
-	xe_pm_runtime_put(xe);
-	drm_err(&xe->drm, "migration recovery failed (%pe)\n", ERR_PTR(err));
-	xe_device_declare_wedged(xe);
-}
-
-static void migration_worker_func(struct work_struct *w)
-{
-	struct xe_device *xe = container_of(w, struct xe_device,
-					    sriov.vf.migration.worker);
-
-	vf_post_migration_recovery(xe);
-}
-
-/*
- * Check if post-restore recovery is coming on any of GTs.
- * @xe: the &xe_device struct instance
- *
- * Return: True if migration recovery worker will soon be running. Any worker currently
- * executing does not affect the result.
- */
-static bool vf_ready_to_recovery_on_any_gts(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return true;
-	}
-	return false;
-}
-
-/**
- * xe_sriov_vf_start_migration_recovery - Start VF migration recovery.
- * @xe: the &xe_device to start recovery on
- *
- * This function shall be called only by VF.
- */
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe)
-{
-	bool started;
-
-	xe_assert(xe, IS_SRIOV_VF(xe));
-
-	if (!vf_ready_to_recovery_on_any_gts(xe))
-		return;
-
-	started = queue_work(xe->sriov.wq, &xe->sriov.vf.migration.worker);
-	drm_info(&xe->drm, "VF migration recovery %s\n", started ?
-		 "scheduled" : "already in progress");
-}
-
 /**
  * xe_sriov_vf_init_late() - SR-IOV VF late initialization functions.
  * @xe: the &xe_device to initialize
drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
index 9e752105ec2a..4df95266b261 100644
@@ -13,7 +13,6 @@ struct xe_device;
 
 void xe_sriov_vf_init_early(struct xe_device *xe);
 int xe_sriov_vf_init_late(struct xe_device *xe);
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe);
 bool xe_sriov_vf_migration_supported(struct xe_device *xe);
 void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
 
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
index 426cc5841958..6a0fd0f5463e 100644
@@ -33,10 +33,6 @@ struct xe_device_vf {
 
 	/** @migration: VF Migration state data */
 	struct {
-		/** @migration.worker: VF migration recovery worker */
-		struct work_struct worker;
-		/** @migration.gt_flags: Per-GT request flags for VF migration recovery */
-		unsigned long gt_flags;
 		/**
 		 * @migration.enabled: flag indicating if migration support
 		 * was enabled or not due to missing prerequisites
#31 lore raw

Re: [PATCH v2 12/34] drm/xe/vf: Make VF recovery run on per-GT worker

Quoted 112 lines · Matthew Brost
Quoted 7 lines · Michal Wajdeczko
Quoted 1 line · Matthew Brost

VF recovery is a per-GT operation, so it makes sense to isolate it to a

that was also my suggestion to make it per-GT, good to see it happen now

+1

Quoted 7 lines
Quoted 3 lines

per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.

but while the recovery is per-GT, we should still protect against that one GT starts the recovery sooner then other GTs notice about it

Yes. There is shared state in 2 places:

- The GGTT shifting, this handled by [1] in my series.
- CCS restore on iGPU (PTL) handled by [2] [3] in my series.

[1] https://patchwork.freedesktop.org/patch/676394/?series=154627&rev=2
[2] https://patchwork.freedesktop.org/patch/676393/?series=154627&rev=2
[3] https://patchwork.freedesktop.org/patch/676397/?series=154627&rev=2

Quoted 57 lines
Quoted 57 lines

v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 170 ++++++++++++++-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   7 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          | 242 +---------------------
 drivers/gpu/drm/xe/xe_sriov_vf.h          |   1 -
 drivers/gpu/drm/xe/xe_sriov_vf_types.h    |   4 -
 6 files changed, 169 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -2
index c9d0e32e7a15..cfb71b749e52 100644
@@ -25,11 +25,15 @@
 #include "xe_guc.h"
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
+#include "xe_guc_submit.h"
+#include "xe_irq.h"
 #include "xe_lrc.h"
 #include "xe_memirq.h"
 #include "xe_mmio.h"
+#include "xe_pm.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
+#include "xe_tile_sriov_vf.h"
 #include "xe_uc_fw.h"
 #include "xe_wopcm.h"
 
@@ -314,7 +318,7 @@ static int guc_action_vf_notify_resfix_done(struct xe_guc *guc)
  * Returns: 0 if the operation completed successfully, or a negative error
  * code otherwise.
  */
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
+static int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
 {
 	struct xe_guc *guc = &gt->uc.guc;
 	int err;
@@ -808,7 +812,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt)
  * xe_gt_sriov_vf_default_lrcs_hwsp_rebase - Update GGTT references in HWSP of default LRCs.
  * @gt: the &xe_gt struct instance
  */
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
+static void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 {
 	struct xe_hw_engine *hwe;
 	enum xe_hw_engine_id id;
@@ -817,6 +821,26 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 		xe_default_lrc_update_memirq_regs_with_address(hwe);
 }
 
Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 824

Quoted 1 line
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)

nit: if this is static then could be just:

vf_start_migration_recovery(gt)

Sure.

Quoted 8 lines
Quoted 8 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +8 -0
+{
+	bool started;
+
+	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
+
+	spin_lock(&gt->sriov.vf.migration.lock);
+
+	if (!gt->sriov.vf.migration.recovery_queued) {
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 838

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -0
+		gt->sriov.vf.migration.recovery_queued = true;
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+
+		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
+		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
+				 "scheduled" : "already in progress");

with this .recovery_queued flag, can we hit "already in progress" case?

Quoted 1 line

This was existing code so kept it but I'm really unclear how we can get

I can't find "recovery_queued" flag in upstream code

Quoted 195 lines

multiple resfix IRQs without the prior resfix flow being completed. Maybe Tomasz can clear this one up? Ideally I'd like to remove multiple resfix IRQ handled code if this is not possible.

Quoted 29 lines
Quoted 29 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -8
+	}
+
+	spin_unlock(&gt->sriov.vf.migration.lock);
+}
+
 /**
  * xe_gt_sriov_vf_migrated_event_handler - Start a VF migration recovery,
  *   or just mark that a GuC is ready for it.
@@ -831,15 +855,8 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
 	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
-	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
-	/*
-	 * We need to be certain that if all flags were set, at least one
-	 * thread will notice that and schedule the recovery.
-	 */
-	smp_mb__after_atomic();
-
 	xe_gt_sriov_info(gt, "ready for recovery after migration\n");
-	xe_sriov_vf_start_migration_recovery(xe);
+	xe_gt_sriov_vf_start_migration_recovery(gt);
 }
 
 static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
@@ -1175,6 +1192,139 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 		   pf_version->major, pf_version->minor);
 }
 
Quoted 13 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1204

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+static void vf_post_migration_shutdown(struct xe_gt *gt)
+{
+	int ret = 0;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_queued = false;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause(&gt->uc.guc);
+	ret |= xe_guc_submit_reset_block(&gt->uc.guc);

this |= seem unneeded

This is existing code copy / pasted and is removed in [4].

[4] https://patchwork.freedesktop.org/patch/676382/?series=154627&rev=2

Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1207

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+
+	if (ret)
+		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");

is this the only possible reason? maybe worth to add %pe ?

Again this existing code and will be removed in [4].

So with above statements, I'd prefer just to leave as is.

Quoted 65 lines
Quoted 52 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +52 -0
+}
+
+static size_t post_migration_scratch_size(struct xe_device *xe)
+{
+	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
+}
+
+static int vf_post_migration_fixups(struct xe_gt *gt)
+{
+	s64 shift;
+	void *buf;
+	int err;
+
+	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
+	if (!buf)
+		return -ENOMEM;
+
+	err = xe_gt_sriov_vf_query_config(gt);
+	if (err)
+		goto out;
+
+	shift = xe_gt_sriov_vf_ggtt_shift(gt);
+	if (shift) {
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+		if (err)
+			goto out;
+	}
+
+out:
+	kfree(buf);
+	return err;
+}
+
+static void vf_post_migration_kickstart(struct xe_gt *gt)
+{
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	xe_irq_resume(gt_to_xe(gt));
+
+	xe_guc_submit_reset_unblock(&gt->uc.guc);
+	xe_guc_submit_unpause(&gt->uc.guc);
+}
+
+static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
+{
+	bool skip_resfix = false;
+

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1269

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	if (gt->sriov.vf.migration.recovery_queued) {
+		skip_resfix = true;
+		xe_gt_sriov_dbg(gt, "another recovery imminent, skipped some notifications\n");
+	} else {
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	}
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);

nit: this looks cleaner:

Quoted 5 lines
	if (skip_resfix)
		return -EAGAIN;

	return xe_gt_sriov_vf_notify_resfix_done(gt);

Sure.

Quoted 50 lines
Quoted 50 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +50 -0
+}
+
+static void vf_post_migration_recovery(struct xe_gt *gt)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	int err;
+
+	xe_gt_sriov_dbg(gt, "migration recovery in progress\n");
+
+	xe_pm_runtime_get(xe);
+	vf_post_migration_shutdown(gt);
+
+	if (!xe_sriov_vf_migration_supported(xe)) {
+		xe_gt_sriov_err(gt, "migration is not supported\n");
+		err = -ENOTRECOVERABLE;
+		goto fail;
+	}
+
+	err = vf_post_migration_fixups(gt);
+	if (err)
+		goto fail;
+
+	vf_post_migration_kickstart(gt);
+	err = vf_post_migration_notify_resfix_done(gt);
+	if (err && err != -EAGAIN)
+		goto fail;
+
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_notice(gt, "migration recovery ended\n");
+	return;
+fail:
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
+	xe_device_declare_wedged(xe);
+}
+
+static void migration_worker_func(struct work_struct *w)
+{
+	struct xe_gt *gt = container_of(w, struct xe_gt,
+					sriov.vf.migration.worker);
+
+	vf_post_migration_recovery(gt);
+}
+
+/**
+ * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
+ * @gt: the &xe_gt
+ */
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+{
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1325

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -0
+	init_rwsem(&gt->sriov.vf.self_config.lock);
+	spin_lock_init(&gt->sriov.vf.migration.lock);
+	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
+
+	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
+		xe_gt_sriov_info(gt, "migration not supported by this module version\n");

we likely don't want to repeat that message on every GT

Quoted 1 line

So move this to xe_sriov_vf_init_early?

hmm, maybe it is even already there

Quoted 101 lines
Quoted 94 lines
Quoted 94 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
+}
+
 /**
  * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
  * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index bb5f8eace19b..2ac6775b52f0 100644
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
 int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
 int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
 bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
 
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
index 7b10b8e1e10e..53680a2f188a 100644
@@ -8,6 +8,7 @@
 
 #include <linux/rwsem.h>
 #include <linux/types.h>
+#include <linux/workqueue.h>
 #include "xe_uc_fw_types.h"
 
 /**
@@ -53,6 +54,12 @@ struct xe_gt_sriov_vf_runtime {
  * xe_gt_sriov_vf_migration - VF migration data.
  */
 struct xe_gt_sriov_vf_migration {
+	/** @migration: VF migration recovery worker */
+	struct work_struct worker;
+	/** @lock: Protects recovery_queued */
+	spinlock_t lock;
+	/** @recovery_queued: VF post migration recovery in queued */
+	bool recovery_queued;
 	/** @recovery_inprogress: VF post migration recovery in progress */
 	bool recovery_inprogress;
 };
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -13
index da064a1e7419..7d91553c4acc 100644
@@ -6,21 +6,12 @@
 #include <drm/drm_debugfs.h>
 #include <drm/drm_managed.h>
 
-#include "xe_assert.h"
-#include "xe_device.h"
 #include "xe_gt.h"
-#include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
-#include "xe_guc_submit.h"
-#include "xe_irq.h"
-#include "xe_lrc.h"
-#include "xe_pm.h"
-#include "xe_sriov.h"
 #include "xe_sriov_printk.h"
 #include "xe_sriov_vf.h"
 #include "xe_sriov_vf_ccs.h"
-#include "xe_tile_sriov_vf.h"
 
 /**
  * DOC: VF restore procedure in PF KMD and VF KMD
@@ -158,8 +149,6 @@ static void vf_disable_migration(struct xe_device *xe, const char *fmt, ...)
 	xe->sriov.vf.migration.enabled = false;
 }
 
-static void migration_worker_func(struct work_struct *w);
-
 static void vf_migration_init_early(struct xe_device *xe)
 {
 	/*
@@ -184,8 +173,6 @@ static void vf_migration_init_early(struct xe_device *xe)
 						    guc_version.major, guc_version.minor);
 	}
 
-	INIT_WORK(&xe->sriov.vf.migration.worker, migration_worker_func);
-
 	xe->sriov.vf.migration.enabled = true;
 	xe_sriov_dbg(xe, "migration support enabled\n");
 }
@@ -200,238 +187,11 @@ void xe_sriov_vf_init_early(struct xe_device *xe)
 	unsigned int id;
 
Quoted 6 lines

drivers/gpu/drm/xe/xe_sriov_vf.c void xe_sriov_vf_init_early(struct xe_device *xe) · line 190

Quoted 3 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -1
 	for_each_gt(gt, xe, id)
-		init_rwsem(&gt->sriov.vf.self_config.lock);
+		xe_gt_sriov_vf_migration_init_early(gt);

still, this should be called from gt_init_early kind of functions

Quoted 3 lines

Kinda a nit that I'm not convinced is worth while to have xe_sriov_vf_init_early and then xe_gt_sriov_vf_migration_init_early called in gt_init_early...

we have global xe level initialization, something like:

xe_device_init_early
xe_sriov_vf_init_early
xe_device_init
xe_sriov_vf_init

then per-gt initialization, that follows the native flow, something like

xe_gt_init_early
xe_gt_sriov_vf_init_early
xe_gt_sriov_vf_migration_init_early
xe_gt_init
xe_gt_sriov_vf_init
xe_gt_sriov_vf_migration_init

IMO we shouldn't jump in sriov code from xe level to gt level init on our own

Quoted 265 lines

Matt

Quoted 262 lines
Quoted 261 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -227
 	vf_migration_init_early(xe);
 }
 
-/**
- * vf_post_migration_shutdown - Stop the driver activities after VF migration.
- * @xe: the &xe_device struct instance
- *
- * After this VM is migrated and assigned to a new VF, it is running on a new
- * hardware, and therefore many hardware-dependent states and related structures
- * require fixups. Without fixups, the hardware cannot do any work, and therefore
- * all GPU pipelines are stalled.
- * Stop some of kernel activities to make the fixup process faster.
- */
-static void vf_post_migration_shutdown(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int ret = 0;
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_pause(&gt->uc.guc);
-		ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-	}
-
-	if (ret)
-		drm_info(&xe->drm, "migration recovery encountered ongoing reset\n");
-}
-
-/**
- * vf_post_migration_kickstart - Re-start the driver activities under new hardware.
- * @xe: the &xe_device struct instance
- *
- * After we have finished with all post-migration fixups, restart the driver
- * activities to continue feeding the GPU with workloads.
- */
-static void vf_post_migration_kickstart(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	/*
-	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
-	 * must be working at this point, since the recovery did started,
-	 * but the rest was not enabled using the procedure from spec.
-	 */
-	xe_irq_resume(xe);
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_reset_unblock(&gt->uc.guc);
-		xe_guc_submit_unpause(&gt->uc.guc);
-	}
-}
-
-static bool gt_vf_post_migration_needed(struct xe_gt *gt)
-{
-	return test_bit(gt->info.id, &gt_to_xe(gt)->sriov.vf.migration.gt_flags);
-}
-
-/*
- * Notify GuCs marked in flags about resource fixups apply finished.
- * @xe: the &xe_device struct instance
- * @gt_flags: flags marking to which GTs the notification shall be sent
- */
-static int vf_post_migration_notify_resfix_done(struct xe_device *xe, unsigned long gt_flags)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int err = 0;
-
-	for_each_gt(gt, xe, id) {
-		if (!test_bit(id, &gt_flags))
-			continue;
-		/* skip asking GuC for RESFIX exit if new recovery request arrived */
-		if (gt_vf_post_migration_needed(gt))
-			continue;
-		err = xe_gt_sriov_vf_notify_resfix_done(gt);
-		if (err)
-			break;
-		clear_bit(id, &gt_flags);
-	}
-
-	if (gt_flags && !err)
-		drm_dbg(&xe->drm, "another recovery imminent, skipped some notifications\n");
-	return err;
-}
-
-static int vf_get_next_migrated_gt_id(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_and_clear_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return id;
-	}
-	return -1;
-}
-
-static size_t post_migration_scratch_size(struct xe_device *xe)
-{
-	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
-}
-
-/**
- * Perform post-migration fixups on a single GT.
- *
- * After migration, GuC needs to be re-queried for VF configuration to check
- * if it matches previous provisioning. Most of VF provisioning shall be the
- * same, except GGTT range, since GGTT is not virtualized per-VF. If GGTT
- * range has changed, we have to perform fixups - shift all GGTT references
- * used anywhere within the driver. After the fixups in this function succeed,
- * it is allowed to ask the GuC bound to this GT to continue normal operation.
- *
- * Returns: 0 if the operation completed successfully, or a negative error
- * code otherwise.
- */
-static int gt_vf_post_migration_fixups(struct xe_gt *gt)
-{
-	s64 shift;
-	void *buf;
-	int err;
-
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_KERNEL);
-	if (!buf)
-		return -ENOMEM;
-
-	err = xe_gt_sriov_vf_query_config(gt);
-	if (err)
-		goto out;
-
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			goto out;
-	}
-
-out:
-	kfree(buf);
-	return err;
-}
-
-static void vf_post_migration_recovery(struct xe_device *xe)
-{
-	unsigned long fixed_gts = 0;
-	int id, err;
-
-	drm_dbg(&xe->drm, "migration recovery in progress\n");
-	xe_pm_runtime_get(xe);
-	vf_post_migration_shutdown(xe);
-
-	if (!xe_sriov_vf_migration_supported(xe)) {
-		xe_sriov_err(xe, "migration is not supported\n");
-		err = -ENOTRECOVERABLE;
-		goto fail;
-	}
-
-	while (id = vf_get_next_migrated_gt_id(xe), id >= 0) {
-		struct xe_gt *gt = xe_device_get_gt(xe, id);
-
-		err = gt_vf_post_migration_fixups(gt);
-		if (err)
-			goto fail;
-
-		set_bit(id, &fixed_gts);
-	}
-
-	vf_post_migration_kickstart(xe);
-	err = vf_post_migration_notify_resfix_done(xe, fixed_gts);
-	if (err)
-		goto fail;
-
-	xe_pm_runtime_put(xe);
-	drm_notice(&xe->drm, "migration recovery ended\n");
-	return;
-fail:
-	xe_pm_runtime_put(xe);
-	drm_err(&xe->drm, "migration recovery failed (%pe)\n", ERR_PTR(err));
-	xe_device_declare_wedged(xe);
-}
-
-static void migration_worker_func(struct work_struct *w)
-{
-	struct xe_device *xe = container_of(w, struct xe_device,
-					    sriov.vf.migration.worker);
-
-	vf_post_migration_recovery(xe);
-}
-
-/*
- * Check if post-restore recovery is coming on any of GTs.
- * @xe: the &xe_device struct instance
- *
- * Return: True if migration recovery worker will soon be running. Any worker currently
- * executing does not affect the result.
- */
-static bool vf_ready_to_recovery_on_any_gts(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return true;
-	}
-	return false;
-}
-
-/**
- * xe_sriov_vf_start_migration_recovery - Start VF migration recovery.
- * @xe: the &xe_device to start recovery on
- *
- * This function shall be called only by VF.
- */
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe)
-{
-	bool started;
-
-	xe_assert(xe, IS_SRIOV_VF(xe));
-
-	if (!vf_ready_to_recovery_on_any_gts(xe))
-		return;
-
-	started = queue_work(xe->sriov.wq, &xe->sriov.vf.migration.worker);
-	drm_info(&xe->drm, "VF migration recovery %s\n", started ?
-		 "scheduled" : "already in progress");
-}
-
 /**
  * xe_sriov_vf_init_late() - SR-IOV VF late initialization functions.
  * @xe: the &xe_device to initialize
drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
index 9e752105ec2a..4df95266b261 100644
@@ -13,7 +13,6 @@ struct xe_device;
 
 void xe_sriov_vf_init_early(struct xe_device *xe);
 int xe_sriov_vf_init_late(struct xe_device *xe);
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe);
 bool xe_sriov_vf_migration_supported(struct xe_device *xe);
 void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
 
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
index 426cc5841958..6a0fd0f5463e 100644
@@ -33,10 +33,6 @@ struct xe_device_vf {
 
 	/** @migration: VF Migration state data */
 	struct {
-		/** @migration.worker: VF migration recovery worker */
-		struct work_struct worker;
-		/** @migration.gt_flags: Per-GT request flags for VF migration recovery */
-		unsigned long gt_flags;
 		/**
 		 * @migration.enabled: flag indicating if migration support
 		 * was enabled or not due to missing prerequisites
author maintainer
#32 lore raw

Re: [PATCH v2 12/34] drm/xe/vf: Make VF recovery run on per-GT worker

Quoted 115 lines · Michal Wajdeczko
Quoted 112 lines · Matthew Brost
Quoted 7 lines · Michal Wajdeczko
Quoted 1 line · Matthew Brost

VF recovery is a per-GT operation, so it makes sense to isolate it to a

that was also my suggestion to make it per-GT, good to see it happen now

+1

Quoted 7 lines
Quoted 3 lines

per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.

but while the recovery is per-GT, we should still protect against that one GT starts the recovery sooner then other GTs notice about it

Yes. There is shared state in 2 places:

- The GGTT shifting, this handled by [1] in my series.
- CCS restore on iGPU (PTL) handled by [2] [3] in my series.

[1] https://patchwork.freedesktop.org/patch/676394/?series=154627&rev=2
[2] https://patchwork.freedesktop.org/patch/676393/?series=154627&rev=2
[3] https://patchwork.freedesktop.org/patch/676397/?series=154627&rev=2

Quoted 57 lines
Quoted 57 lines

v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 170 ++++++++++++++-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   7 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          | 242 +---------------------
 drivers/gpu/drm/xe/xe_sriov_vf.h          |   1 -
 drivers/gpu/drm/xe/xe_sriov_vf_types.h    |   4 -
 6 files changed, 169 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -2
index c9d0e32e7a15..cfb71b749e52 100644
@@ -25,11 +25,15 @@
 #include "xe_guc.h"
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
+#include "xe_guc_submit.h"
+#include "xe_irq.h"
 #include "xe_lrc.h"
 #include "xe_memirq.h"
 #include "xe_mmio.h"
+#include "xe_pm.h"
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
+#include "xe_tile_sriov_vf.h"
 #include "xe_uc_fw.h"
 #include "xe_wopcm.h"
 
@@ -314,7 +318,7 @@ static int guc_action_vf_notify_resfix_done(struct xe_guc *guc)
  * Returns: 0 if the operation completed successfully, or a negative error
  * code otherwise.
  */
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
+static int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
 {
 	struct xe_guc *guc = &gt->uc.guc;
 	int err;
@@ -808,7 +812,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt)
  * xe_gt_sriov_vf_default_lrcs_hwsp_rebase - Update GGTT references in HWSP of default LRCs.
  * @gt: the &xe_gt struct instance
  */
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
+static void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 {
 	struct xe_hw_engine *hwe;
 	enum xe_hw_engine_id id;
@@ -817,6 +821,26 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
 		xe_default_lrc_update_memirq_regs_with_address(hwe);
 }
 
Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 824

Quoted 1 line
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)

nit: if this is static then could be just:

vf_start_migration_recovery(gt)

Sure.

Quoted 17 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+{
+	bool started;
+
+	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 838

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+
+	spin_lock(&gt->sriov.vf.migration.lock);
+
+	if (!gt->sriov.vf.migration.recovery_queued) {
+		gt->sriov.vf.migration.recovery_queued = true;
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+
+		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
+		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
+				 "scheduled" : "already in progress");

with this .recovery_queued flag, can we hit "already in progress" case?

Quoted 4 lines
Quoted 1 line

This was existing code so kept it but I'm really unclear how we can get

I can't find "recovery_queued" flag in upstream code

This is akin to comment in upstream code. Again, I'm not really convienced this is something we need to handle as I suspect this is not possible.

/* skip asking GuC for RESFIX exit if new recovery request arrived */

Quoted 195 lines
Quoted 195 lines

multiple resfix IRQs without the prior resfix flow being completed. Maybe Tomasz can clear this one up? Ideally I'd like to remove multiple resfix IRQ handled code if this is not possible.

Quoted 29 lines
Quoted 29 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -8
+	}
+
+	spin_unlock(&gt->sriov.vf.migration.lock);
+}
+
 /**
  * xe_gt_sriov_vf_migrated_event_handler - Start a VF migration recovery,
  *   or just mark that a GuC is ready for it.
@@ -831,15 +855,8 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
 	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
-	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
-	/*
-	 * We need to be certain that if all flags were set, at least one
-	 * thread will notice that and schedule the recovery.
-	 */
-	smp_mb__after_atomic();
-
 	xe_gt_sriov_info(gt, "ready for recovery after migration\n");
-	xe_sriov_vf_start_migration_recovery(xe);
+	xe_gt_sriov_vf_start_migration_recovery(gt);
 }
 
 static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
@@ -1175,6 +1192,139 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 		   pf_version->major, pf_version->minor);
 }
 
Quoted 13 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1204

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+static void vf_post_migration_shutdown(struct xe_gt *gt)
+{
+	int ret = 0;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_queued = false;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause(&gt->uc.guc);
+	ret |= xe_guc_submit_reset_block(&gt->uc.guc);

this |= seem unneeded

This is existing code copy / pasted and is removed in [4].

[4] https://patchwork.freedesktop.org/patch/676382/?series=154627&rev=2

Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1207

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+
+	if (ret)
+		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");

is this the only possible reason? maybe worth to add %pe ?

Again this existing code and will be removed in [4].

So with above statements, I'd prefer just to leave as is.

Quoted 65 lines
Quoted 52 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +52 -0
+}
+
+static size_t post_migration_scratch_size(struct xe_device *xe)
+{
+	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
+}
+
+static int vf_post_migration_fixups(struct xe_gt *gt)
+{
+	s64 shift;
+	void *buf;
+	int err;
+
+	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
+	if (!buf)
+		return -ENOMEM;
+
+	err = xe_gt_sriov_vf_query_config(gt);
+	if (err)
+		goto out;
+
+	shift = xe_gt_sriov_vf_ggtt_shift(gt);
+	if (shift) {
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+		if (err)
+			goto out;
+	}
+
+out:
+	kfree(buf);
+	return err;
+}
+
+static void vf_post_migration_kickstart(struct xe_gt *gt)
+{
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	xe_irq_resume(gt_to_xe(gt));
+
+	xe_guc_submit_reset_unblock(&gt->uc.guc);
+	xe_guc_submit_unpause(&gt->uc.guc);
+}
+
+static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
+{
+	bool skip_resfix = false;
+

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1269

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	if (gt->sriov.vf.migration.recovery_queued) {
+		skip_resfix = true;
+		xe_gt_sriov_dbg(gt, "another recovery imminent, skipped some notifications\n");
+	} else {
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	}
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);

nit: this looks cleaner:

Quoted 5 lines
	if (skip_resfix)
		return -EAGAIN;

	return xe_gt_sriov_vf_notify_resfix_done(gt);

Sure.

Quoted 59 lines
Quoted 46 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +46 -0
+}
+
+static void vf_post_migration_recovery(struct xe_gt *gt)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	int err;
+
+	xe_gt_sriov_dbg(gt, "migration recovery in progress\n");
+
+	xe_pm_runtime_get(xe);
+	vf_post_migration_shutdown(gt);
+
+	if (!xe_sriov_vf_migration_supported(xe)) {
+		xe_gt_sriov_err(gt, "migration is not supported\n");
+		err = -ENOTRECOVERABLE;
+		goto fail;
+	}
+
+	err = vf_post_migration_fixups(gt);
+	if (err)
+		goto fail;
+
+	vf_post_migration_kickstart(gt);
+	err = vf_post_migration_notify_resfix_done(gt);
+	if (err && err != -EAGAIN)
+		goto fail;
+
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_notice(gt, "migration recovery ended\n");
+	return;
+fail:
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
+	xe_device_declare_wedged(xe);
+}
+
+static void migration_worker_func(struct work_struct *w)
+{
+	struct xe_gt *gt = container_of(w, struct xe_gt,
+					sriov.vf.migration.worker);
+
+	vf_post_migration_recovery(gt);
+}
+
+/**
+ * xe_gt_sriov_vf_migration_init_early() - VF post migration init early

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1325

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+ * @gt: the &xe_gt
+ */
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+{
+	init_rwsem(&gt->sriov.vf.self_config.lock);
+	spin_lock_init(&gt->sriov.vf.migration.lock);
+	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
+
+	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
+		xe_gt_sriov_info(gt, "migration not supported by this module version\n");

we likely don't want to repeat that message on every GT

Quoted 4 lines
Quoted 1 line

So move this to xe_sriov_vf_init_early?

hmm, maybe it is even already there

Let me check on that. Either way will remove this from the GT layer.

Quoted 114 lines
Quoted 101 lines
Quoted 94 lines
Quoted 94 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
+}
+
 /**
  * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
  * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index bb5f8eace19b..2ac6775b52f0 100644
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
 int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
 int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
 bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
 
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
index 7b10b8e1e10e..53680a2f188a 100644
@@ -8,6 +8,7 @@
 
 #include <linux/rwsem.h>
 #include <linux/types.h>
+#include <linux/workqueue.h>
 #include "xe_uc_fw_types.h"
 
 /**
@@ -53,6 +54,12 @@ struct xe_gt_sriov_vf_runtime {
  * xe_gt_sriov_vf_migration - VF migration data.
  */
 struct xe_gt_sriov_vf_migration {
+	/** @migration: VF migration recovery worker */
+	struct work_struct worker;
+	/** @lock: Protects recovery_queued */
+	spinlock_t lock;
+	/** @recovery_queued: VF post migration recovery in queued */
+	bool recovery_queued;
 	/** @recovery_inprogress: VF post migration recovery in progress */
 	bool recovery_inprogress;
 };
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -13
index da064a1e7419..7d91553c4acc 100644
@@ -6,21 +6,12 @@
 #include <drm/drm_debugfs.h>
 #include <drm/drm_managed.h>
 
-#include "xe_assert.h"
-#include "xe_device.h"
 #include "xe_gt.h"
-#include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
-#include "xe_guc_submit.h"
-#include "xe_irq.h"
-#include "xe_lrc.h"
-#include "xe_pm.h"
-#include "xe_sriov.h"
 #include "xe_sriov_printk.h"
 #include "xe_sriov_vf.h"
 #include "xe_sriov_vf_ccs.h"
-#include "xe_tile_sriov_vf.h"
 
 /**
  * DOC: VF restore procedure in PF KMD and VF KMD
@@ -158,8 +149,6 @@ static void vf_disable_migration(struct xe_device *xe, const char *fmt, ...)
 	xe->sriov.vf.migration.enabled = false;
 }
 
-static void migration_worker_func(struct work_struct *w);
-
 static void vf_migration_init_early(struct xe_device *xe)
 {
 	/*
@@ -184,8 +173,6 @@ static void vf_migration_init_early(struct xe_device *xe)
 						    guc_version.major, guc_version.minor);
 	}
 
-	INIT_WORK(&xe->sriov.vf.migration.worker, migration_worker_func);
-
 	xe->sriov.vf.migration.enabled = true;
 	xe_sriov_dbg(xe, "migration support enabled\n");
 }
@@ -200,238 +187,11 @@ void xe_sriov_vf_init_early(struct xe_device *xe)
 	unsigned int id;
 
Quoted 6 lines

drivers/gpu/drm/xe/xe_sriov_vf.c void xe_sriov_vf_init_early(struct xe_device *xe) · line 190

Quoted 3 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -1
 	for_each_gt(gt, xe, id)
-		init_rwsem(&gt->sriov.vf.self_config.lock);
+		xe_gt_sriov_vf_migration_init_early(gt);

still, this should be called from gt_init_early kind of functions

Quoted 3 lines

Kinda a nit that I'm not convinced is worth while to have xe_sriov_vf_init_early and then xe_gt_sriov_vf_migration_init_early called in gt_init_early...

we have global xe level initialization, something like:

xe_device_init_early
xe_sriov_vf_init_early
xe_device_init
xe_sriov_vf_init

then per-gt initialization, that follows the native flow, something like

Quoted 9 lines

xe_gt_init_early
xe_gt_sriov_vf_init_early
xe_gt_sriov_vf_migration_init_early
xe_gt_init
xe_gt_sriov_vf_init
xe_gt_sriov_vf_migration_init

IMO we shouldn't jump in sriov code from xe level to gt level init on our own

Ok, let me refactor this. Easy enough to do.

Matt

Quoted 266 lines
Quoted 265 lines

Matt

Quoted 262 lines
Quoted 261 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -227
 	vf_migration_init_early(xe);
 }
 
-/**
- * vf_post_migration_shutdown - Stop the driver activities after VF migration.
- * @xe: the &xe_device struct instance
- *
- * After this VM is migrated and assigned to a new VF, it is running on a new
- * hardware, and therefore many hardware-dependent states and related structures
- * require fixups. Without fixups, the hardware cannot do any work, and therefore
- * all GPU pipelines are stalled.
- * Stop some of kernel activities to make the fixup process faster.
- */
-static void vf_post_migration_shutdown(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int ret = 0;
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_pause(&gt->uc.guc);
-		ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-	}
-
-	if (ret)
-		drm_info(&xe->drm, "migration recovery encountered ongoing reset\n");
-}
-
-/**
- * vf_post_migration_kickstart - Re-start the driver activities under new hardware.
- * @xe: the &xe_device struct instance
- *
- * After we have finished with all post-migration fixups, restart the driver
- * activities to continue feeding the GPU with workloads.
- */
-static void vf_post_migration_kickstart(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	/*
-	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
-	 * must be working at this point, since the recovery did started,
-	 * but the rest was not enabled using the procedure from spec.
-	 */
-	xe_irq_resume(xe);
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_reset_unblock(&gt->uc.guc);
-		xe_guc_submit_unpause(&gt->uc.guc);
-	}
-}
-
-static bool gt_vf_post_migration_needed(struct xe_gt *gt)
-{
-	return test_bit(gt->info.id, &gt_to_xe(gt)->sriov.vf.migration.gt_flags);
-}
-
-/*
- * Notify GuCs marked in flags about resource fixups apply finished.
- * @xe: the &xe_device struct instance
- * @gt_flags: flags marking to which GTs the notification shall be sent
- */
-static int vf_post_migration_notify_resfix_done(struct xe_device *xe, unsigned long gt_flags)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int err = 0;
-
-	for_each_gt(gt, xe, id) {
-		if (!test_bit(id, &gt_flags))
-			continue;
-		/* skip asking GuC for RESFIX exit if new recovery request arrived */
-		if (gt_vf_post_migration_needed(gt))
-			continue;
-		err = xe_gt_sriov_vf_notify_resfix_done(gt);
-		if (err)
-			break;
-		clear_bit(id, &gt_flags);
-	}
-
-	if (gt_flags && !err)
-		drm_dbg(&xe->drm, "another recovery imminent, skipped some notifications\n");
-	return err;
-}
-
-static int vf_get_next_migrated_gt_id(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_and_clear_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return id;
-	}
-	return -1;
-}
-
-static size_t post_migration_scratch_size(struct xe_device *xe)
-{
-	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
-}
-
-/**
- * Perform post-migration fixups on a single GT.
- *
- * After migration, GuC needs to be re-queried for VF configuration to check
- * if it matches previous provisioning. Most of VF provisioning shall be the
- * same, except GGTT range, since GGTT is not virtualized per-VF. If GGTT
- * range has changed, we have to perform fixups - shift all GGTT references
- * used anywhere within the driver. After the fixups in this function succeed,
- * it is allowed to ask the GuC bound to this GT to continue normal operation.
- *
- * Returns: 0 if the operation completed successfully, or a negative error
- * code otherwise.
- */
-static int gt_vf_post_migration_fixups(struct xe_gt *gt)
-{
-	s64 shift;
-	void *buf;
-	int err;
-
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_KERNEL);
-	if (!buf)
-		return -ENOMEM;
-
-	err = xe_gt_sriov_vf_query_config(gt);
-	if (err)
-		goto out;
-
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			goto out;
-	}
-
-out:
-	kfree(buf);
-	return err;
-}
-
-static void vf_post_migration_recovery(struct xe_device *xe)
-{
-	unsigned long fixed_gts = 0;
-	int id, err;
-
-	drm_dbg(&xe->drm, "migration recovery in progress\n");
-	xe_pm_runtime_get(xe);
-	vf_post_migration_shutdown(xe);
-
-	if (!xe_sriov_vf_migration_supported(xe)) {
-		xe_sriov_err(xe, "migration is not supported\n");
-		err = -ENOTRECOVERABLE;
-		goto fail;
-	}
-
-	while (id = vf_get_next_migrated_gt_id(xe), id >= 0) {
-		struct xe_gt *gt = xe_device_get_gt(xe, id);
-
-		err = gt_vf_post_migration_fixups(gt);
-		if (err)
-			goto fail;
-
-		set_bit(id, &fixed_gts);
-	}
-
-	vf_post_migration_kickstart(xe);
-	err = vf_post_migration_notify_resfix_done(xe, fixed_gts);
-	if (err)
-		goto fail;
-
-	xe_pm_runtime_put(xe);
-	drm_notice(&xe->drm, "migration recovery ended\n");
-	return;
-fail:
-	xe_pm_runtime_put(xe);
-	drm_err(&xe->drm, "migration recovery failed (%pe)\n", ERR_PTR(err));
-	xe_device_declare_wedged(xe);
-}
-
-static void migration_worker_func(struct work_struct *w)
-{
-	struct xe_device *xe = container_of(w, struct xe_device,
-					    sriov.vf.migration.worker);
-
-	vf_post_migration_recovery(xe);
-}
-
-/*
- * Check if post-restore recovery is coming on any of GTs.
- * @xe: the &xe_device struct instance
- *
- * Return: True if migration recovery worker will soon be running. Any worker currently
- * executing does not affect the result.
- */
-static bool vf_ready_to_recovery_on_any_gts(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return true;
-	}
-	return false;
-}
-
-/**
- * xe_sriov_vf_start_migration_recovery - Start VF migration recovery.
- * @xe: the &xe_device to start recovery on
- *
- * This function shall be called only by VF.
- */
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe)
-{
-	bool started;
-
-	xe_assert(xe, IS_SRIOV_VF(xe));
-
-	if (!vf_ready_to_recovery_on_any_gts(xe))
-		return;
-
-	started = queue_work(xe->sriov.wq, &xe->sriov.vf.migration.worker);
-	drm_info(&xe->drm, "VF migration recovery %s\n", started ?
-		 "scheduled" : "already in progress");
-}
-
 /**
  * xe_sriov_vf_init_late() - SR-IOV VF late initialization functions.
  * @xe: the &xe_device to initialize
drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
index 9e752105ec2a..4df95266b261 100644
@@ -13,7 +13,6 @@ struct xe_device;
 
 void xe_sriov_vf_init_early(struct xe_device *xe);
 int xe_sriov_vf_init_late(struct xe_device *xe);
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe);
 bool xe_sriov_vf_migration_supported(struct xe_device *xe);
 void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
 
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
index 426cc5841958..6a0fd0f5463e 100644
@@ -33,10 +33,6 @@ struct xe_device_vf {
 
 	/** @migration: VF Migration state data */
 	struct {
-		/** @migration.worker: VF migration recovery worker */
-		struct work_struct worker;
-		/** @migration.gt_flags: Per-GT request flags for VF migration recovery */
-		unsigned long gt_flags;
 		/**
 		 * @migration.enabled: flag indicating if migration support
 		 * was enabled or not due to missing prerequisites
#33 lore raw

Re: [PATCH v2 12/34] drm/xe/vf: Make VF recovery run on per-GT worker

Quoted 105 lines · Matthew Brost
Quoted 104 lines · Michal Wajdeczko
Quoted 102 lines · Matthew Brost
Quoted 5 lines · Michal Wajdeczko
Quoted 1 line · Matthew Brost

VF recovery is a per-GT operation, so it makes sense to isolate it to a

that was also my suggestion to make it per-GT, good to see it happen now

+1

Quoted 6 lines
Quoted 3 lines

per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.

but while the recovery is per-GT, we should still protect against that one GT starts the recovery sooner then other GTs notice about it

Yes. There is shared state in 2 places:

- The GGTT shifting, this handled by [1] in my series.
- CCS restore on iGPU (PTL) handled by [2] [3] in my series.

[1] https://patchwork.freedesktop.org/patch/676394/?series=154627&rev=2
[2] https://patchwork.freedesktop.org/patch/676393/?series=154627&rev=2
[3] https://patchwork.freedesktop.org/patch/676397/?series=154627&rev=2

Quoted 56 lines
Quoted 56 lines

v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 170 ++++++++++++++-
  drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +-
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   7 +
  drivers/gpu/drm/xe/xe_sriov_vf.c          | 242 +---------------------
  drivers/gpu/drm/xe/xe_sriov_vf.h          |   1 -
  drivers/gpu/drm/xe/xe_sriov_vf_types.h    |   4 -
  6 files changed, 169 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -2
index c9d0e32e7a15..cfb71b749e52 100644
@@ -25,11 +25,15 @@
  #include "xe_guc.h"
  #include "xe_guc_hxg_helpers.h"
  #include "xe_guc_relay.h"
+#include "xe_guc_submit.h"
+#include "xe_irq.h"
  #include "xe_lrc.h"
  #include "xe_memirq.h"
  #include "xe_mmio.h"
+#include "xe_pm.h"
  #include "xe_sriov.h"
  #include "xe_sriov_vf.h"
+#include "xe_tile_sriov_vf.h"
  #include "xe_uc_fw.h"
  #include "xe_wopcm.h"
 
@@ -314,7 +318,7 @@ static int guc_action_vf_notify_resfix_done(struct xe_guc *guc)
   * Returns: 0 if the operation completed successfully, or a negative error
   * code otherwise.
   */
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
+static int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
  {
  	struct xe_guc *guc = &gt->uc.guc;
  	int err;
@@ -808,7 +812,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt)
   * xe_gt_sriov_vf_default_lrcs_hwsp_rebase - Update GGTT references in HWSP of default LRCs.
   * @gt: the &xe_gt struct instance
   */
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
+static void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
  {
  	struct xe_hw_engine *hwe;
  	enum xe_hw_engine_id id;
@@ -817,6 +821,26 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
  		xe_default_lrc_update_memirq_regs_with_address(hwe);
  }
 
Quoted 5 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 824

Quoted 1 line
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)

nit: if this is static then could be just:

vf_start_migration_recovery(gt)

Sure.

Quoted 16 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+{
+	bool started;
+
+	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 838

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+
+	spin_lock(&gt->sriov.vf.migration.lock);
+
+	if (!gt->sriov.vf.migration.recovery_queued) {
+		gt->sriov.vf.migration.recovery_queued = true;
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+
+		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
+		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
+				 "scheduled" : "already in progress");

with this .recovery_queued flag, can we hit "already in progress" case?

Quoted 8 lines
Quoted 3 lines
Quoted 1 line

This was existing code so kept it but I'm really unclear how we can get

I can't find "recovery_queued" flag in upstream code

This is akin to comment in upstream code. Again, I'm not really convienced this is something we need to handle as I suspect this is not possible.

/* skip asking GuC for RESFIX exit if new recovery request arrived */

Second migration could happen while recovery_queued is set, I don't see why not. Migration could happen at any point.

Interrupts tend to be treated with priority compared to workers, so if migration will happen in that short period, this can and will be hit.

For the code near "/* skip asking GuC for RESFIX exit if new recovery request arrived */" - while its days are numbered, let's leave this method of double migration support for this series. We will later switch to an improved double migration support, but that depends on GuC version and should be developed and tested separately.

-Tomasz

Quoted 579 lines
Quoted 188 lines
Quoted 188 lines

multiple resfix IRQs without the prior resfix flow being completed. Maybe Tomasz can clear this one up? Ideally I'd like to remove multiple resfix IRQ handled code if this is not possible.

Quoted 29 lines
Quoted 29 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -8
+	}
+
+	spin_unlock(&gt->sriov.vf.migration.lock);
+}
+
  /**
   * xe_gt_sriov_vf_migrated_event_handler - Start a VF migration recovery,
   *   or just mark that a GuC is ready for it.
@@ -831,15 +855,8 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(xe));
  	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
 
-	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
-	/*
-	 * We need to be certain that if all flags were set, at least one
-	 * thread will notice that and schedule the recovery.
-	 */
-	smp_mb__after_atomic();
-
  	xe_gt_sriov_info(gt, "ready for recovery after migration\n");
-	xe_sriov_vf_start_migration_recovery(xe);
+	xe_gt_sriov_vf_start_migration_recovery(gt);
  }
 
  static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
@@ -1175,6 +1192,139 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
  		   pf_version->major, pf_version->minor);
  }
 
Quoted 12 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1204

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+static void vf_post_migration_shutdown(struct xe_gt *gt)
+{
+	int ret = 0;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_queued = false;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause(&gt->uc.guc);
+	ret |= xe_guc_submit_reset_block(&gt->uc.guc);

this |= seem unneeded

This is existing code copy / pasted and is removed in [4].

[4] https://patchwork.freedesktop.org/patch/676382/?series=154627&rev=2

Quoted 5 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1207

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+
+	if (ret)
+		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");

is this the only possible reason? maybe worth to add %pe ?

Again this existing code and will be removed in [4].

So with above statements, I'd prefer just to leave as is.

Quoted 64 lines
Quoted 52 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +52 -0
+}
+
+static size_t post_migration_scratch_size(struct xe_device *xe)
+{
+	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
+}
+
+static int vf_post_migration_fixups(struct xe_gt *gt)
+{
+	s64 shift;
+	void *buf;
+	int err;
+
+	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
+	if (!buf)
+		return -ENOMEM;
+
+	err = xe_gt_sriov_vf_query_config(gt);
+	if (err)
+		goto out;
+
+	shift = xe_gt_sriov_vf_ggtt_shift(gt);
+	if (shift) {
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+		if (err)
+			goto out;
+	}
+
+out:
+	kfree(buf);
+	return err;
+}
+
+static void vf_post_migration_kickstart(struct xe_gt *gt)
+{
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	xe_irq_resume(gt_to_xe(gt));
+
+	xe_guc_submit_reset_unblock(&gt->uc.guc);
+	xe_guc_submit_unpause(&gt->uc.guc);
+}
+
+static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
+{
+	bool skip_resfix = false;
+

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1269

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	if (gt->sriov.vf.migration.recovery_queued) {
+		skip_resfix = true;
+		xe_gt_sriov_dbg(gt, "another recovery imminent, skipped some notifications\n");
+	} else {
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	}
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);

nit: this looks cleaner:

Quoted 5 lines
	if (skip_resfix)
		return -EAGAIN;

	return xe_gt_sriov_vf_notify_resfix_done(gt);

Sure.

Quoted 58 lines
Quoted 46 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +46 -0
+}
+
+static void vf_post_migration_recovery(struct xe_gt *gt)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	int err;
+
+	xe_gt_sriov_dbg(gt, "migration recovery in progress\n");
+
+	xe_pm_runtime_get(xe);
+	vf_post_migration_shutdown(gt);
+
+	if (!xe_sriov_vf_migration_supported(xe)) {
+		xe_gt_sriov_err(gt, "migration is not supported\n");
+		err = -ENOTRECOVERABLE;
+		goto fail;
+	}
+
+	err = vf_post_migration_fixups(gt);
+	if (err)
+		goto fail;
+
+	vf_post_migration_kickstart(gt);
+	err = vf_post_migration_notify_resfix_done(gt);
+	if (err && err != -EAGAIN)
+		goto fail;
+
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_notice(gt, "migration recovery ended\n");
+	return;
+fail:
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
+	xe_device_declare_wedged(xe);
+}
+
+static void migration_worker_func(struct work_struct *w)
+{
+	struct xe_gt *gt = container_of(w, struct xe_gt,
+					sriov.vf.migration.worker);
+
+	vf_post_migration_recovery(gt);
+}
+
+/**
+ * xe_gt_sriov_vf_migration_init_early() - VF post migration init early

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1325

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+ * @gt: the &xe_gt
+ */
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+{
+	init_rwsem(&gt->sriov.vf.self_config.lock);
+	spin_lock_init(&gt->sriov.vf.migration.lock);
+	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
+
+	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
+		xe_gt_sriov_info(gt, "migration not supported by this module version\n");

we likely don't want to repeat that message on every GT

Quoted 3 lines
Quoted 1 line

So move this to xe_sriov_vf_init_early?

hmm, maybe it is even already there

Let me check on that. Either way will remove this from the GT layer.

Quoted 110 lines
Quoted 93 lines
Quoted 93 lines
Quoted 93 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
+}
+
  /**
   * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
   * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index bb5f8eace19b..2ac6775b52f0 100644
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
  int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
  int xe_gt_sriov_vf_connect(struct xe_gt *gt);
  int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
  void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
  bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
 
  u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
index 7b10b8e1e10e..53680a2f188a 100644
@@ -8,6 +8,7 @@
 
  #include <linux/rwsem.h>
  #include <linux/types.h>
+#include <linux/workqueue.h>
  #include "xe_uc_fw_types.h"
 
  /**
@@ -53,6 +54,12 @@ struct xe_gt_sriov_vf_runtime {
   * xe_gt_sriov_vf_migration - VF migration data.
   */
  struct xe_gt_sriov_vf_migration {
+	/** @migration: VF migration recovery worker */
+	struct work_struct worker;
+	/** @lock: Protects recovery_queued */
+	spinlock_t lock;
+	/** @recovery_queued: VF post migration recovery in queued */
+	bool recovery_queued;
  	/** @recovery_inprogress: VF post migration recovery in progress */
  	bool recovery_inprogress;
  };
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -13
index da064a1e7419..7d91553c4acc 100644
@@ -6,21 +6,12 @@
  #include <drm/drm_debugfs.h>
  #include <drm/drm_managed.h>
 
-#include "xe_assert.h"
-#include "xe_device.h"
  #include "xe_gt.h"
-#include "xe_gt_sriov_printk.h"
  #include "xe_gt_sriov_vf.h"
  #include "xe_guc.h"
-#include "xe_guc_submit.h"
-#include "xe_irq.h"
-#include "xe_lrc.h"
-#include "xe_pm.h"
-#include "xe_sriov.h"
  #include "xe_sriov_printk.h"
  #include "xe_sriov_vf.h"
  #include "xe_sriov_vf_ccs.h"
-#include "xe_tile_sriov_vf.h"
 
  /**
   * DOC: VF restore procedure in PF KMD and VF KMD
@@ -158,8 +149,6 @@ static void vf_disable_migration(struct xe_device *xe, const char *fmt, ...)
  	xe->sriov.vf.migration.enabled = false;
  }
 
-static void migration_worker_func(struct work_struct *w);
-
  static void vf_migration_init_early(struct xe_device *xe)
  {
  	/*
@@ -184,8 +173,6 @@ static void vf_migration_init_early(struct xe_device *xe)
  						    guc_version.major, guc_version.minor);
  	}
 
-	INIT_WORK(&xe->sriov.vf.migration.worker, migration_worker_func);
-
  	xe->sriov.vf.migration.enabled = true;
  	xe_sriov_dbg(xe, "migration support enabled\n");
  }
@@ -200,238 +187,11 @@ void xe_sriov_vf_init_early(struct xe_device *xe)
  	unsigned int id;
 
Quoted 8 lines
Quoted 5 lines

drivers/gpu/drm/xe/xe_sriov_vf.c void xe_sriov_vf_init_early(struct xe_device *xe) · line 190

Quoted 3 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -1
  	for_each_gt(gt, xe, id)
-		init_rwsem(&gt->sriov.vf.self_config.lock);
+		xe_gt_sriov_vf_migration_init_early(gt);

still, this should be called from gt_init_early kind of functions

Kinda a nit that I'm not convinced is worth while to have xe_sriov_vf_init_early and then xe_gt_sriov_vf_migration_init_early called in gt_init_early...

we have global xe level initialization, something like:

xe_device_init_early
xe_sriov_vf_init_early
xe_device_init
xe_sriov_vf_init

then per-gt initialization, that follows the native flow, something like

Quoted 9 lines

xe_gt_init_early
xe_gt_sriov_vf_init_early
xe_gt_sriov_vf_migration_init_early
xe_gt_init
xe_gt_sriov_vf_init
xe_gt_sriov_vf_migration_init

IMO we shouldn't jump in sriov code from xe level to gt level init on our own

Ok, let me refactor this. Easy enough to do.

Matt

Quoted 263 lines
Quoted 263 lines

Matt

Quoted 261 lines
Quoted 261 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -227
  	vf_migration_init_early(xe);
  }
 
-/**
- * vf_post_migration_shutdown - Stop the driver activities after VF migration.
- * @xe: the &xe_device struct instance
- *
- * After this VM is migrated and assigned to a new VF, it is running on a new
- * hardware, and therefore many hardware-dependent states and related structures
- * require fixups. Without fixups, the hardware cannot do any work, and therefore
- * all GPU pipelines are stalled.
- * Stop some of kernel activities to make the fixup process faster.
- */
-static void vf_post_migration_shutdown(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int ret = 0;
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_pause(&gt->uc.guc);
-		ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-	}
-
-	if (ret)
-		drm_info(&xe->drm, "migration recovery encountered ongoing reset\n");
-}
-
-/**
- * vf_post_migration_kickstart - Re-start the driver activities under new hardware.
- * @xe: the &xe_device struct instance
- *
- * After we have finished with all post-migration fixups, restart the driver
- * activities to continue feeding the GPU with workloads.
- */
-static void vf_post_migration_kickstart(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	/*
-	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
-	 * must be working at this point, since the recovery did started,
-	 * but the rest was not enabled using the procedure from spec.
-	 */
-	xe_irq_resume(xe);
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_reset_unblock(&gt->uc.guc);
-		xe_guc_submit_unpause(&gt->uc.guc);
-	}
-}
-
-static bool gt_vf_post_migration_needed(struct xe_gt *gt)
-{
-	return test_bit(gt->info.id, &gt_to_xe(gt)->sriov.vf.migration.gt_flags);
-}
-
-/*
- * Notify GuCs marked in flags about resource fixups apply finished.
- * @xe: the &xe_device struct instance
- * @gt_flags: flags marking to which GTs the notification shall be sent
- */
-static int vf_post_migration_notify_resfix_done(struct xe_device *xe, unsigned long gt_flags)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int err = 0;
-
-	for_each_gt(gt, xe, id) {
-		if (!test_bit(id, &gt_flags))
-			continue;
-		/* skip asking GuC for RESFIX exit if new recovery request arrived */
-		if (gt_vf_post_migration_needed(gt))
-			continue;
-		err = xe_gt_sriov_vf_notify_resfix_done(gt);
-		if (err)
-			break;
-		clear_bit(id, &gt_flags);
-	}
-
-	if (gt_flags && !err)
-		drm_dbg(&xe->drm, "another recovery imminent, skipped some notifications\n");
-	return err;
-}
-
-static int vf_get_next_migrated_gt_id(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_and_clear_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return id;
-	}
-	return -1;
-}
-
-static size_t post_migration_scratch_size(struct xe_device *xe)
-{
-	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
-}
-
-/**
- * Perform post-migration fixups on a single GT.
- *
- * After migration, GuC needs to be re-queried for VF configuration to check
- * if it matches previous provisioning. Most of VF provisioning shall be the
- * same, except GGTT range, since GGTT is not virtualized per-VF. If GGTT
- * range has changed, we have to perform fixups - shift all GGTT references
- * used anywhere within the driver. After the fixups in this function succeed,
- * it is allowed to ask the GuC bound to this GT to continue normal operation.
- *
- * Returns: 0 if the operation completed successfully, or a negative error
- * code otherwise.
- */
-static int gt_vf_post_migration_fixups(struct xe_gt *gt)
-{
-	s64 shift;
-	void *buf;
-	int err;
-
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_KERNEL);
-	if (!buf)
-		return -ENOMEM;
-
-	err = xe_gt_sriov_vf_query_config(gt);
-	if (err)
-		goto out;
-
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			goto out;
-	}
-
-out:
-	kfree(buf);
-	return err;
-}
-
-static void vf_post_migration_recovery(struct xe_device *xe)
-{
-	unsigned long fixed_gts = 0;
-	int id, err;
-
-	drm_dbg(&xe->drm, "migration recovery in progress\n");
-	xe_pm_runtime_get(xe);
-	vf_post_migration_shutdown(xe);
-
-	if (!xe_sriov_vf_migration_supported(xe)) {
-		xe_sriov_err(xe, "migration is not supported\n");
-		err = -ENOTRECOVERABLE;
-		goto fail;
-	}
-
-	while (id = vf_get_next_migrated_gt_id(xe), id >= 0) {
-		struct xe_gt *gt = xe_device_get_gt(xe, id);
-
-		err = gt_vf_post_migration_fixups(gt);
-		if (err)
-			goto fail;
-
-		set_bit(id, &fixed_gts);
-	}
-
-	vf_post_migration_kickstart(xe);
-	err = vf_post_migration_notify_resfix_done(xe, fixed_gts);
-	if (err)
-		goto fail;
-
-	xe_pm_runtime_put(xe);
-	drm_notice(&xe->drm, "migration recovery ended\n");
-	return;
-fail:
-	xe_pm_runtime_put(xe);
-	drm_err(&xe->drm, "migration recovery failed (%pe)\n", ERR_PTR(err));
-	xe_device_declare_wedged(xe);
-}
-
-static void migration_worker_func(struct work_struct *w)
-{
-	struct xe_device *xe = container_of(w, struct xe_device,
-					    sriov.vf.migration.worker);
-
-	vf_post_migration_recovery(xe);
-}
-
-/*
- * Check if post-restore recovery is coming on any of GTs.
- * @xe: the &xe_device struct instance
- *
- * Return: True if migration recovery worker will soon be running. Any worker currently
- * executing does not affect the result.
- */
-static bool vf_ready_to_recovery_on_any_gts(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return true;
-	}
-	return false;
-}
-
-/**
- * xe_sriov_vf_start_migration_recovery - Start VF migration recovery.
- * @xe: the &xe_device to start recovery on
- *
- * This function shall be called only by VF.
- */
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe)
-{
-	bool started;
-
-	xe_assert(xe, IS_SRIOV_VF(xe));
-
-	if (!vf_ready_to_recovery_on_any_gts(xe))
-		return;
-
-	started = queue_work(xe->sriov.wq, &xe->sriov.vf.migration.worker);
-	drm_info(&xe->drm, "VF migration recovery %s\n", started ?
-		 "scheduled" : "already in progress");
-}
-
  /**
   * xe_sriov_vf_init_late() - SR-IOV VF late initialization functions.
   * @xe: the &xe_device to initialize
drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
index 9e752105ec2a..4df95266b261 100644
@@ -13,7 +13,6 @@ struct xe_device;
 
  void xe_sriov_vf_init_early(struct xe_device *xe);
  int xe_sriov_vf_init_late(struct xe_device *xe);
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe);
  bool xe_sriov_vf_migration_supported(struct xe_device *xe);
  void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
 
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
index 426cc5841958..6a0fd0f5463e 100644
@@ -33,10 +33,6 @@ struct xe_device_vf {
 
  	/** @migration: VF Migration state data */
  	struct {
-		/** @migration.worker: VF migration recovery worker */
-		struct work_struct worker;
-		/** @migration.gt_flags: Per-GT request flags for VF migration recovery */
-		unsigned long gt_flags;
  		/**
  		 * @migration.enabled: flag indicating if migration support
  		 * was enabled or not due to missing prerequisites
author maintainer
#34 lore raw

Re: [PATCH v2 12/34] drm/xe/vf: Make VF recovery run on per-GT worker

Quoted 112 lines · Tomasz
Quoted 110 lines · Matthew Brost
Quoted 102 lines · Michal Wajdeczko
Quoted 100 lines · Matthew Brost
Quoted 5 lines · Michal Wajdeczko
Quoted 1 line · Matthew Brost

VF recovery is a per-GT operation, so it makes sense to isolate it to a

that was also my suggestion to make it per-GT, good to see it happen now

+1

Quoted 6 lines
Quoted 3 lines

per-GT queue. Scheduling this operation on the same worker as the GT reset and TDR not only aligns with this design but also helps avoid race conditions, as those operations can also modify the queue state.

but while the recovery is per-GT, we should still protect against that one GT starts the recovery sooner then other GTs notice about it

Yes. There is shared state in 2 places:

- The GGTT shifting, this handled by [1] in my series.
- CCS restore on iGPU (PTL) handled by [2] [3] in my series.

[1] https://patchwork.freedesktop.org/patch/676394/?series=154627&rev=2
[2] https://patchwork.freedesktop.org/patch/676393/?series=154627&rev=2
[3] https://patchwork.freedesktop.org/patch/676397/?series=154627&rev=2

Quoted 57 lines
Quoted 52 lines

v2:
- Fix lockdep splat (Adam)
- Use xe_sriov_vf_migration_supported helper

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 170 ++++++++++++++-
  drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +-
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   7 +
  drivers/gpu/drm/xe/xe_sriov_vf.c          | 242 +---------------------
  drivers/gpu/drm/xe/xe_sriov_vf.h          |   1 -
  drivers/gpu/drm/xe/xe_sriov_vf_types.h    |   4 -
  6 files changed, 169 insertions(+), 258 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -2
index c9d0e32e7a15..cfb71b749e52 100644
@@ -25,11 +25,15 @@
  #include "xe_guc.h"
  #include "xe_guc_hxg_helpers.h"
  #include "xe_guc_relay.h"
+#include "xe_guc_submit.h"
+#include "xe_irq.h"
  #include "xe_lrc.h"
  #include "xe_memirq.h"
  #include "xe_mmio.h"
+#include "xe_pm.h"
  #include "xe_sriov.h"
  #include "xe_sriov_vf.h"
+#include "xe_tile_sriov_vf.h"
  #include "xe_uc_fw.h"
  #include "xe_wopcm.h"
@@ -314,7 +318,7 @@ static int guc_action_vf_notify_resfix_done(struct xe_guc *guc)
   * Returns: 0 if the operation completed successfully, or a negative error
   * code otherwise.
   */
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
+static int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt)
  {
  	struct xe_guc *guc = &gt->uc.guc;
  	int err;
@@ -808,7 +812,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt)
   * xe_gt_sriov_vf_default_lrcs_hwsp_rebase - Update GGTT references in HWSP of default LRCs.
   * @gt: the &xe_gt struct instance
   */
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
+static void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
  {
  	struct xe_hw_engine *hwe;
  	enum xe_hw_engine_id id;

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 823

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -817,6 +821,26 @@ void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt)
  		xe_default_lrc_update_memirq_regs_with_address(hwe);
  }
+static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)

nit: if this is static then could be just:

Quoted 2 lines

vf_start_migration_recovery(gt)

Sure.

Quoted 16 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+{
+	bool started;
+
+	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt) · line 838

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+
+	spin_lock(&gt->sriov.vf.migration.lock);
+
+	if (!gt->sriov.vf.migration.recovery_queued) {
+		gt->sriov.vf.migration.recovery_queued = true;
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+
+		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
+		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
+				 "scheduled" : "already in progress");

with this .recovery_queued flag, can we hit "already in progress" case?

Quoted 3 lines
Quoted 1 line

This was existing code so kept it but I'm really unclear how we can get

I can't find "recovery_queued" flag in upstream code

This is akin to comment in upstream code. Again, I'm not really convienced this is something we need to handle as I suspect this is not possible.

Quoted 8 lines
Quoted 1 line

/* skip asking GuC for RESFIX exit if new recovery request arrived */

Second migration could happen while recovery_queued is set, I don't see why not. Migration could happen at any point.

Interrupts tend to be treated with priority compared to workers, so if migration will happen in that short period, this can and will be hit.

I was made aware how this can occur late yesterday.

Quoted 3 lines

For the code near "/* skip asking GuC for RESFIX exit if new recovery request arrived */" - while its days are numbered, let's leave this method of double migration support for this series. We will later switch to an

Ok, 'migration.recovery_queued' implements the same functionality as in the upstream code.

Quoted 3 lines

improved double migration support, but that depends on GuC version and should be developed and tested separately.

Yes, I am aware of this upcoming change too. I agree this is better solution as 'migration.recovery_queued' or upstream version which does the same thing tries hard to not race but it is basically impossible to seal that race without ridiculous locking which I chose not to implement in this series. I suggest we leave 'migration.recovery_queued' as in this series and remove once proper GuC support lands.

Matt

Quoted 564 lines

-Tomasz

Quoted 562 lines
Quoted 185 lines
Quoted 185 lines

multiple resfix IRQs without the prior resfix flow being completed. Maybe Tomasz can clear this one up? Ideally I'd like to remove multiple resfix IRQ handled code if this is not possible.

Quoted 29 lines
Quoted 29 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +9 -8
+	}
+
+	spin_unlock(&gt->sriov.vf.migration.lock);
+}
+
  /**
   * xe_gt_sriov_vf_migrated_event_handler - Start a VF migration recovery,
   *   or just mark that a GuC is ready for it.
@@ -831,15 +855,8 @@ void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(xe));
  	xe_gt_assert(gt, xe_gt_sriov_vf_recovery_inprogress(gt));
-	set_bit(gt->info.id, &xe->sriov.vf.migration.gt_flags);
-	/*
-	 * We need to be certain that if all flags were set, at least one
-	 * thread will notice that and schedule the recovery.
-	 */
-	smp_mb__after_atomic();
-
  	xe_gt_sriov_info(gt, "ready for recovery after migration\n");
-	xe_sriov_vf_start_migration_recovery(xe);
+	xe_gt_sriov_vf_start_migration_recovery(gt);
  }
  static bool vf_is_negotiated(struct xe_gt *gt, u16 major, u16 minor)
@@ -1175,6 +1192,139 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
  		   pf_version->major, pf_version->minor);
  }
+static void vf_post_migration_shutdown(struct xe_gt *gt)
+{
+	int ret = 0;
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1204

Quoted 7 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7 -0
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_queued = false;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause(&gt->uc.guc);
+	ret |= xe_guc_submit_reset_block(&gt->uc.guc);

this |= seem unneeded

This is existing code copy / pasted and is removed in [4].

[4] https://patchwork.freedesktop.org/patch/676382/?series=154627&rev=2

Quoted 5 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1207

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+
+	if (ret)
+		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");

is this the only possible reason? maybe worth to add %pe ?

Again this existing code and will be removed in [4].

So with above statements, I'd prefer just to leave as is.

Quoted 64 lines
Quoted 52 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +52 -0
+}
+
+static size_t post_migration_scratch_size(struct xe_device *xe)
+{
+	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
+}
+
+static int vf_post_migration_fixups(struct xe_gt *gt)
+{
+	s64 shift;
+	void *buf;
+	int err;
+
+	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
+	if (!buf)
+		return -ENOMEM;
+
+	err = xe_gt_sriov_vf_query_config(gt);
+	if (err)
+		goto out;
+
+	shift = xe_gt_sriov_vf_ggtt_shift(gt);
+	if (shift) {
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+		if (err)
+			goto out;
+	}
+
+out:
+	kfree(buf);
+	return err;
+}
+
+static void vf_post_migration_kickstart(struct xe_gt *gt)
+{
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	xe_irq_resume(gt_to_xe(gt));
+
+	xe_guc_submit_reset_unblock(&gt->uc.guc);
+	xe_guc_submit_unpause(&gt->uc.guc);
+}
+
+static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
+{
+	bool skip_resfix = false;
+

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1269

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	if (gt->sriov.vf.migration.recovery_queued) {
+		skip_resfix = true;
+		xe_gt_sriov_dbg(gt, "another recovery imminent, skipped some notifications\n");
+	} else {
+		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	}
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);

nit: this looks cleaner:

Quoted 5 lines
	if (skip_resfix)
		return -EAGAIN;

	return xe_gt_sriov_vf_notify_resfix_done(gt);

Sure.

Quoted 58 lines
Quoted 46 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +46 -0
+}
+
+static void vf_post_migration_recovery(struct xe_gt *gt)
+{
+	struct xe_device *xe = gt_to_xe(gt);
+	int err;
+
+	xe_gt_sriov_dbg(gt, "migration recovery in progress\n");
+
+	xe_pm_runtime_get(xe);
+	vf_post_migration_shutdown(gt);
+
+	if (!xe_sriov_vf_migration_supported(xe)) {
+		xe_gt_sriov_err(gt, "migration is not supported\n");
+		err = -ENOTRECOVERABLE;
+		goto fail;
+	}
+
+	err = vf_post_migration_fixups(gt);
+	if (err)
+		goto fail;
+
+	vf_post_migration_kickstart(gt);
+	err = vf_post_migration_notify_resfix_done(gt);
+	if (err && err != -EAGAIN)
+		goto fail;
+
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_notice(gt, "migration recovery ended\n");
+	return;
+fail:
+	xe_pm_runtime_put(xe);
+	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
+	xe_device_declare_wedged(xe);
+}
+
+static void migration_worker_func(struct work_struct *w)
+{
+	struct xe_gt *gt = container_of(w, struct xe_gt,
+					sriov.vf.migration.worker);
+
+	vf_post_migration_recovery(gt);
+}
+
+/**
+ * xe_gt_sriov_vf_migration_init_early() - VF post migration init early

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p) · line 1325

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+ * @gt: the &xe_gt
+ */
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+{
+	init_rwsem(&gt->sriov.vf.self_config.lock);
+	spin_lock_init(&gt->sriov.vf.migration.lock);
+	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
+
+	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
+		xe_gt_sriov_info(gt, "migration not supported by this module version\n");

we likely don't want to repeat that message on every GT

Quoted 3 lines
Quoted 1 line

So move this to xe_sriov_vf_init_early?

hmm, maybe it is even already there

Let me check on that. Either way will remove this from the GT layer.

Quoted 101 lines
Quoted 89 lines
Quoted 80 lines
Quoted 80 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
+}
+
  /**
   * xe_gt_sriov_vf_recovery_inprogress() - VF post migration recovery in progress
   * @gt: the &xe_gt
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index bb5f8eace19b..2ac6775b52f0 100644
@@ -21,10 +21,9 @@ void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
  int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
  int xe_gt_sriov_vf_connect(struct xe_gt *gt);
  int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
-void xe_gt_sriov_vf_default_lrcs_hwsp_rebase(struct xe_gt *gt);
-int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
  void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
+void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
  bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
  u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -0
index 7b10b8e1e10e..53680a2f188a 100644
@@ -8,6 +8,7 @@
  #include <linux/rwsem.h>
  #include <linux/types.h>
+#include <linux/workqueue.h>
  #include "xe_uc_fw_types.h"
  /**
@@ -53,6 +54,12 @@ struct xe_gt_sriov_vf_runtime {
   * xe_gt_sriov_vf_migration - VF migration data.
   */
  struct xe_gt_sriov_vf_migration {
+	/** @migration: VF migration recovery worker */
+	struct work_struct worker;
+	/** @lock: Protects recovery_queued */
+	spinlock_t lock;
+	/** @recovery_queued: VF post migration recovery in queued */
+	bool recovery_queued;
  	/** @recovery_inprogress: VF post migration recovery in progress */
  	bool recovery_inprogress;
  };
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -13
index da064a1e7419..7d91553c4acc 100644
@@ -6,21 +6,12 @@
  #include <drm/drm_debugfs.h>
  #include <drm/drm_managed.h>
-#include "xe_assert.h"
-#include "xe_device.h"
  #include "xe_gt.h"
-#include "xe_gt_sriov_printk.h"
  #include "xe_gt_sriov_vf.h"
  #include "xe_guc.h"
-#include "xe_guc_submit.h"
-#include "xe_irq.h"
-#include "xe_lrc.h"
-#include "xe_pm.h"
-#include "xe_sriov.h"
  #include "xe_sriov_printk.h"
  #include "xe_sriov_vf.h"
  #include "xe_sriov_vf_ccs.h"
-#include "xe_tile_sriov_vf.h"
  /**
   * DOC: VF restore procedure in PF KMD and VF KMD
@@ -158,8 +149,6 @@ static void vf_disable_migration(struct xe_device *xe, const char *fmt, ...)
  	xe->sriov.vf.migration.enabled = false;
  }
-static void migration_worker_func(struct work_struct *w);
-
  static void vf_migration_init_early(struct xe_device *xe)
  {
  	/*
@@ -184,8 +173,6 @@ static void vf_migration_init_early(struct xe_device *xe)
  						    guc_version.major, guc_version.minor);
  	}
-	INIT_WORK(&xe->sriov.vf.migration.worker, migration_worker_func);
-
  	xe->sriov.vf.migration.enabled = true;
Quoted 9 lines

drivers/gpu/drm/xe/xe_sriov_vf.c void xe_sriov_vf_init_early(struct xe_device *xe) · line 189

Quoted 7 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -1
  	xe_sriov_dbg(xe, "migration support enabled\n");
  }
@@ -200,238 +187,11 @@ void xe_sriov_vf_init_early(struct xe_device *xe)
  	unsigned int id;
  	for_each_gt(gt, xe, id)
-		init_rwsem(&gt->sriov.vf.self_config.lock);
+		xe_gt_sriov_vf_migration_init_early(gt);

still, this should be called from gt_init_early kind of functions

Quoted 3 lines

Kinda a nit that I'm not convinced is worth while to have xe_sriov_vf_init_early and then xe_gt_sriov_vf_migration_init_early called in gt_init_early...

we have global xe level initialization, something like:

xe_device_init_early
xe_sriov_vf_init_early
xe_device_init
xe_sriov_vf_init

then per-gt initialization, that follows the native flow, something like

Quoted 9 lines

xe_gt_init_early
xe_gt_sriov_vf_init_early
xe_gt_sriov_vf_migration_init_early
xe_gt_init
xe_gt_sriov_vf_init
xe_gt_sriov_vf_migration_init

IMO we shouldn't jump in sriov code from xe level to gt level init on our own

Ok, let me refactor this. Easy enough to do.

Matt

Quoted 258 lines
Quoted 258 lines

Matt

Quoted 256 lines
Quoted 256 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +0 -227
  	vf_migration_init_early(xe);
  }
-/**
- * vf_post_migration_shutdown - Stop the driver activities after VF migration.
- * @xe: the &xe_device struct instance
- *
- * After this VM is migrated and assigned to a new VF, it is running on a new
- * hardware, and therefore many hardware-dependent states and related structures
- * require fixups. Without fixups, the hardware cannot do any work, and therefore
- * all GPU pipelines are stalled.
- * Stop some of kernel activities to make the fixup process faster.
- */
-static void vf_post_migration_shutdown(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int ret = 0;
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_pause(&gt->uc.guc);
-		ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-	}
-
-	if (ret)
-		drm_info(&xe->drm, "migration recovery encountered ongoing reset\n");
-}
-
-/**
- * vf_post_migration_kickstart - Re-start the driver activities under new hardware.
- * @xe: the &xe_device struct instance
- *
- * After we have finished with all post-migration fixups, restart the driver
- * activities to continue feeding the GPU with workloads.
- */
-static void vf_post_migration_kickstart(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	/*
-	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
-	 * must be working at this point, since the recovery did started,
-	 * but the rest was not enabled using the procedure from spec.
-	 */
-	xe_irq_resume(xe);
-
-	for_each_gt(gt, xe, id) {
-		xe_guc_submit_reset_unblock(&gt->uc.guc);
-		xe_guc_submit_unpause(&gt->uc.guc);
-	}
-}
-
-static bool gt_vf_post_migration_needed(struct xe_gt *gt)
-{
-	return test_bit(gt->info.id, &gt_to_xe(gt)->sriov.vf.migration.gt_flags);
-}
-
-/*
- * Notify GuCs marked in flags about resource fixups apply finished.
- * @xe: the &xe_device struct instance
- * @gt_flags: flags marking to which GTs the notification shall be sent
- */
-static int vf_post_migration_notify_resfix_done(struct xe_device *xe, unsigned long gt_flags)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-	int err = 0;
-
-	for_each_gt(gt, xe, id) {
-		if (!test_bit(id, &gt_flags))
-			continue;
-		/* skip asking GuC for RESFIX exit if new recovery request arrived */
-		if (gt_vf_post_migration_needed(gt))
-			continue;
-		err = xe_gt_sriov_vf_notify_resfix_done(gt);
-		if (err)
-			break;
-		clear_bit(id, &gt_flags);
-	}
-
-	if (gt_flags && !err)
-		drm_dbg(&xe->drm, "another recovery imminent, skipped some notifications\n");
-	return err;
-}
-
-static int vf_get_next_migrated_gt_id(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_and_clear_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return id;
-	}
-	return -1;
-}
-
-static size_t post_migration_scratch_size(struct xe_device *xe)
-{
-	return max(xe_lrc_reg_size(xe), LRC_WA_BB_SIZE);
-}
-
-/**
- * Perform post-migration fixups on a single GT.
- *
- * After migration, GuC needs to be re-queried for VF configuration to check
- * if it matches previous provisioning. Most of VF provisioning shall be the
- * same, except GGTT range, since GGTT is not virtualized per-VF. If GGTT
- * range has changed, we have to perform fixups - shift all GGTT references
- * used anywhere within the driver. After the fixups in this function succeed,
- * it is allowed to ask the GuC bound to this GT to continue normal operation.
- *
- * Returns: 0 if the operation completed successfully, or a negative error
- * code otherwise.
- */
-static int gt_vf_post_migration_fixups(struct xe_gt *gt)
-{
-	s64 shift;
-	void *buf;
-	int err;
-
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_KERNEL);
-	if (!buf)
-		return -ENOMEM;
-
-	err = xe_gt_sriov_vf_query_config(gt);
-	if (err)
-		goto out;
-
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			goto out;
-	}
-
-out:
-	kfree(buf);
-	return err;
-}
-
-static void vf_post_migration_recovery(struct xe_device *xe)
-{
-	unsigned long fixed_gts = 0;
-	int id, err;
-
-	drm_dbg(&xe->drm, "migration recovery in progress\n");
-	xe_pm_runtime_get(xe);
-	vf_post_migration_shutdown(xe);
-
-	if (!xe_sriov_vf_migration_supported(xe)) {
-		xe_sriov_err(xe, "migration is not supported\n");
-		err = -ENOTRECOVERABLE;
-		goto fail;
-	}
-
-	while (id = vf_get_next_migrated_gt_id(xe), id >= 0) {
-		struct xe_gt *gt = xe_device_get_gt(xe, id);
-
-		err = gt_vf_post_migration_fixups(gt);
-		if (err)
-			goto fail;
-
-		set_bit(id, &fixed_gts);
-	}
-
-	vf_post_migration_kickstart(xe);
-	err = vf_post_migration_notify_resfix_done(xe, fixed_gts);
-	if (err)
-		goto fail;
-
-	xe_pm_runtime_put(xe);
-	drm_notice(&xe->drm, "migration recovery ended\n");
-	return;
-fail:
-	xe_pm_runtime_put(xe);
-	drm_err(&xe->drm, "migration recovery failed (%pe)\n", ERR_PTR(err));
-	xe_device_declare_wedged(xe);
-}
-
-static void migration_worker_func(struct work_struct *w)
-{
-	struct xe_device *xe = container_of(w, struct xe_device,
-					    sriov.vf.migration.worker);
-
-	vf_post_migration_recovery(xe);
-}
-
-/*
- * Check if post-restore recovery is coming on any of GTs.
- * @xe: the &xe_device struct instance
- *
- * Return: True if migration recovery worker will soon be running. Any worker currently
- * executing does not affect the result.
- */
-static bool vf_ready_to_recovery_on_any_gts(struct xe_device *xe)
-{
-	struct xe_gt *gt;
-	unsigned int id;
-
-	for_each_gt(gt, xe, id) {
-		if (test_bit(id, &xe->sriov.vf.migration.gt_flags))
-			return true;
-	}
-	return false;
-}
-
-/**
- * xe_sriov_vf_start_migration_recovery - Start VF migration recovery.
- * @xe: the &xe_device to start recovery on
- *
- * This function shall be called only by VF.
- */
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe)
-{
-	bool started;
-
-	xe_assert(xe, IS_SRIOV_VF(xe));
-
-	if (!vf_ready_to_recovery_on_any_gts(xe))
-		return;
-
-	started = queue_work(xe->sriov.wq, &xe->sriov.vf.migration.worker);
-	drm_info(&xe->drm, "VF migration recovery %s\n", started ?
-		 "scheduled" : "already in progress");
-}
-
  /**
   * xe_sriov_vf_init_late() - SR-IOV VF late initialization functions.
   * @xe: the &xe_device to initialize
drivers/gpu/drm/xe/xe_sriov_vf.h +0 -1
index 9e752105ec2a..4df95266b261 100644
@@ -13,7 +13,6 @@ struct xe_device;
  void xe_sriov_vf_init_early(struct xe_device *xe);
  int xe_sriov_vf_init_late(struct xe_device *xe);
-void xe_sriov_vf_start_migration_recovery(struct xe_device *xe);
  bool xe_sriov_vf_migration_supported(struct xe_device *xe);
  void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
drivers/gpu/drm/xe/xe_sriov_vf_types.h +0 -4
index 426cc5841958..6a0fd0f5463e 100644
@@ -33,10 +33,6 @@ struct xe_device_vf {
  	/** @migration: VF Migration state data */
  	struct {
-		/** @migration.worker: VF migration recovery worker */
-		struct work_struct worker;
-		/** @migration.gt_flags: Per-GT request flags for VF migration recovery */
-		unsigned long gt_flags;
  		/**
  		 * @migration.enabled: flag indicating if migration support
  		 * was enabled or not due to missing prerequisites
author maintainer
#35 lore raw

PATCH v2 13/34 drm/xe/vf: Abort H2G sends during VF post-migration recovery

In reply to Matthew Brost

While VF post-migration recovery is in progress, abort H2G sends with -ECANCEL. These messages are treated as lost, and TLB invalidation errors are suppressed. During this phase, the H2G channel is down, and VF recovery requires the CT lock to proceed.

---
 drivers/gpu/drm/xe/xe_guc_ct.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +4 -2
index 47079ab9922c..661ab1bf4502 100644
@@ -25,6 +25,7 @@
 #include "xe_gt_printk.h"
 #include "xe_gt_sriov_pf_control.h"
 #include "xe_gt_sriov_pf_monitor.h"
+#include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
 #include "xe_guc_log.h"
 #include "xe_guc_relay.h"
@@ -851,7 +852,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 				u32 len, u32 g2h_len, u32 num_g2h,
 				struct g2h_fence *g2h_fence)
 {
-	struct xe_gt *gt __maybe_unused = ct_to_gt(ct);
+	struct xe_gt *gt = ct_to_gt(ct);
 	u16 seqno;
 	int ret;
 
@@ -872,7 +873,8 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 		goto out;
 	}
 
-	if (ct->state == XE_GUC_CT_STATE_STOPPED) {
+	if (ct->state == XE_GUC_CT_STATE_STOPPED ||
+	    xe_gt_sriov_vf_recovery_inprogress(gt)) {
 		ret = -ECANCELED;
 		goto out;
 	}
Signature · 1 line
2.34.1
#36 lore raw

Re: [PATCH v2 13/34] drm/xe/vf: Abort H2G sends during VF post-migration recovery

Quoted 1 line · Matthew Brost

While VF post-migration recovery is in progress, abort H2G sends with

should be abort or give the caller a second chance ?

btw, do we care about extra errors being logged in such case ?

Quoted 31 lines

-ECANCEL. These messages are treated as lost, and TLB invalidation errors are suppressed. During this phase, the H2G channel is down, and VF recovery requires the CT lock to proceed.

---
 drivers/gpu/drm/xe/xe_guc_ct.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +2 -1
index 47079ab9922c..661ab1bf4502 100644
@@ -25,6 +25,7 @@
 #include "xe_gt_printk.h"
 #include "xe_gt_sriov_pf_control.h"
 #include "xe_gt_sriov_pf_monitor.h"
+#include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
 #include "xe_guc_log.h"
 #include "xe_guc_relay.h"
@@ -851,7 +852,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 				u32 len, u32 g2h_len, u32 num_g2h,
 				struct g2h_fence *g2h_fence)
 {
-	struct xe_gt *gt __maybe_unused = ct_to_gt(ct);
+	struct xe_gt *gt = ct_to_gt(ct);
 	u16 seqno;
 	int ret;
 

drivers/gpu/drm/xe/xe_guc_ct.c static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, · line 877

Quoted 6 lines
drivers/gpu/drm/xe/xe_guc_ct.c +2 -1
@@ -872,7 +873,8 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 		goto out;
 	}
 
-	if (ct->state == XE_GUC_CT_STATE_STOPPED) {
+	if (ct->state == XE_GUC_CT_STATE_STOPPED ||
+	    xe_gt_sriov_vf_recovery_inprogress(gt)) {

this still looks like a hack

maybe we should either explicitly STOP CTB once we start the recovery, or if that's too much, maybe introduce HALT CTB state?

btw, what should happen to already received G2H? shouldn't we process them out when starting recovery?

and maybe we should return different error code to clearly distinguish between our explicit STOP state and implicit recovery?

Quoted 3 lines
drivers/gpu/drm/xe/xe_guc_ct.c +0 -0
 		ret = -ECANCELED;
 		goto out;
 	}
author maintainer
#37 lore raw

Re: [PATCH v2 13/34] drm/xe/vf: Abort H2G sends during VF post-migration recovery

Quoted 9 lines · Michal Wajdeczko
Quoted 1 line · Matthew Brost

While VF post-migration recovery is in progress, abort H2G sends with

should be abort or give the caller a second chance ?

btw, do we care about extra errors being logged in such case ?

All submission H2G are designed to be lost and state machine recovers. TLB invalidation H2G are also designed to be lost.

If we have a case where H2G need to be relible, xe_guc_ct_send_recv_no_fail should be used. I code that function early in Xe, is unused so it probably doesn't work, but conceptually we should be able to make H2G relible across GT reset or VF migration.

IMO we audit all H2G is a follow up and if there is need fixup xe_guc_ct_send_recv_no_fail and use in cases where a H2G needs to be relible.

Quoted 34 lines
Quoted 34 lines

-ECANCEL. These messages are treated as lost, and TLB invalidation errors are suppressed. During this phase, the H2G channel is down, and VF recovery requires the CT lock to proceed.

---
 drivers/gpu/drm/xe/xe_guc_ct.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +2 -1
index 47079ab9922c..661ab1bf4502 100644
@@ -25,6 +25,7 @@
 #include "xe_gt_printk.h"
 #include "xe_gt_sriov_pf_control.h"
 #include "xe_gt_sriov_pf_monitor.h"
+#include "xe_gt_sriov_vf.h"
 #include "xe_guc.h"
 #include "xe_guc_log.h"
 #include "xe_guc_relay.h"
@@ -851,7 +852,7 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 				u32 len, u32 g2h_len, u32 num_g2h,
 				struct g2h_fence *g2h_fence)
 {
-	struct xe_gt *gt __maybe_unused = ct_to_gt(ct);
+	struct xe_gt *gt = ct_to_gt(ct);
 	u16 seqno;
 	int ret;
 
@@ -872,7 +873,8 @@ static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action,
 		goto out;
 	}
 
Quoted 6 lines

drivers/gpu/drm/xe/xe_guc_ct.c static int __guc_ct_send_locked(struct xe_guc_ct *ct, const u32 *action, · line 877

Quoted 3 lines
drivers/gpu/drm/xe/xe_guc_ct.c +2 -1
-	if (ct->state == XE_GUC_CT_STATE_STOPPED) {
+	if (ct->state == XE_GUC_CT_STATE_STOPPED ||
+	    xe_gt_sriov_vf_recovery_inprogress(gt)) {

this still looks like a hack

It is not and is required. We need to pop out of __guc_ct_send_locked during VF migration without taking the CT lock (calling STOP CTB).

Consider the case where H2G is full (no credits) and VF migration happens. If we try to call STOP CTB this requires CT mutex which is being held by __guc_ct_send_locked, __guc_ct_send_locked eventually times out and tries to trigger a GT reset which is not desired. IMO the flow of locklessly, immediately being able to detect a VF migration happend in the CT code is correct.

Quoted 3 lines

maybe we should either explicitly STOP CTB once we start the recovery, or if that's too much, maybe introduce HALT CTB state?

We do STOP the CT as soon as we can in VF migration recovery but this cover the case mentioned above.

Quoted 3 lines

btw, what should happen to already received G2H? shouldn't we process them out when starting recovery?

Yes. xe_guc_ct_flush process all G2H before stopping the CT.

Quoted 3 lines

and maybe we should return different error code to clearly distinguish between our explicit STOP state and implicit recovery?

-ECANCELED is the correct return code which the upper layer (e.g., TLB invalidation) understand that is fine the H2G was dropped.

Matt

Quoted 4 lines
Quoted 3 lines
drivers/gpu/drm/xe/xe_guc_ct.c +0 -0
 		ret = -ECANCELED;
 		goto out;
 	}
author maintainer
#38 lore raw

PATCH v2 14/34 drm/xe/vf: Remove memory allocations from VF post migration recovery

In reply to Matthew Brost

VF post migration recovery is the path of dma-fence signaling / reclaim, avoid memory allocations in this path.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 29 ++++++++++++++---------
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  2 ++
 drivers/gpu/drm/xe/xe_sriov.c             |  8 +++++--
 drivers/gpu/drm/xe/xe_sriov_vf.c          | 14 ++++++++---
 drivers/gpu/drm/xe/xe_sriov_vf.h          |  2 +-
 6 files changed, 39 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +18 -11
index cfb71b749e52..8304c26c076e 100644
@@ -1214,17 +1214,13 @@ static size_t post_migration_scratch_size(struct xe_device *xe)
 
 static int vf_post_migration_fixups(struct xe_gt *gt)
 {
+	void *buf = gt->sriov.vf.migration.lrc_wa_bb;
 	s64 shift;
-	void *buf;
 	int err;
 
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
-	if (!buf)
-		return -ENOMEM;
-
 	err = xe_gt_sriov_vf_query_config(gt);
 	if (err)
-		goto out;
+		return err;
 
 	shift = xe_gt_sriov_vf_ggtt_shift(gt);
 	if (shift) {
@@ -1232,12 +1228,10 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
 		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
 		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
 		if (err)
-			goto out;
+			return err;
 	}
 
-out:
-	kfree(buf);
-	return err;
+	return 0;
 }
 
 static void vf_post_migration_kickstart(struct xe_gt *gt)
@@ -1314,15 +1308,28 @@ static void migration_worker_func(struct work_struct *w)
 /**
  * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
  * @gt: the &xe_gt
+ *
+ * Return 0 on success, errno on failure
  */
-void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
 {
+	void *buf;
+
+	buf = drmm_kmalloc(&gt_to_xe(gt)->drm,
+			   post_migration_scratch_size(gt_to_xe(gt)),
+			   GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	gt->sriov.vf.migration.lrc_wa_bb = buf;
 	init_rwsem(&gt->sriov.vf.self_config.lock);
 	spin_lock_init(&gt->sriov.vf.migration.lock);
 	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
 
 	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
 		xe_gt_sriov_info(gt, "migration not supported by this module version\n");
+
+	return 0;
 }
 
 /**
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -1
index 2ac6775b52f0..195dbebe941e 100644
@@ -23,7 +23,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
-void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
+int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
 bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
 
 u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +2 -0
index 53680a2f188a..496b657119de 100644
@@ -58,6 +58,8 @@ struct xe_gt_sriov_vf_migration {
 	struct work_struct worker;
 	/** @lock: Protects recovery_queued */
 	spinlock_t lock;
+	/** @lrc_wa_bb: Scratch memory for LRC WA BB in recovery */
+	void *lrc_wa_bb;
 	/** @recovery_queued: VF post migration recovery in queued */
 	bool recovery_queued;
 	/** @recovery_inprogress: VF post migration recovery in progress */
drivers/gpu/drm/xe/xe_sriov.c +6 -2
index 7d2d6de2aabf..358a35c80d5a 100644
@@ -116,8 +116,12 @@ int xe_sriov_init(struct xe_device *xe)
 			return err;
 	}
 
-	if (IS_SRIOV_VF(xe))
-		xe_sriov_vf_init_early(xe);
+	if (IS_SRIOV_VF(xe)) {
+		int err = xe_sriov_vf_init_early(xe);
+
+		if (err)
+			return err;
+	}
 
 	xe_assert(xe, !xe->sriov.wq);
 	xe->sriov.wq = alloc_workqueue("xe-sriov-wq", 0, 0);
drivers/gpu/drm/xe/xe_sriov_vf.c +11 -3
index 7d91553c4acc..e622a7c562c4 100644
@@ -180,16 +180,24 @@ static void vf_migration_init_early(struct xe_device *xe)
 /**
  * xe_sriov_vf_init_early - Initialize SR-IOV VF specific data.
  * @xe: the &xe_device to initialize
+ *
+ * Return: 0 on success or a negative error code on failure.
  */
-void xe_sriov_vf_init_early(struct xe_device *xe)
+int xe_sriov_vf_init_early(struct xe_device *xe)
 {
 	struct xe_gt *gt;
 	unsigned int id;
+	int err;
 
-	for_each_gt(gt, xe, id)
-		xe_gt_sriov_vf_migration_init_early(gt);
+	for_each_gt(gt, xe, id) {
+		err = xe_gt_sriov_vf_migration_init_early(gt);
+		if (err)
+			return err;
+	}
 
 	vf_migration_init_early(xe);
+
+	return 0;
 }
 
 /**
drivers/gpu/drm/xe/xe_sriov_vf.h +1 -1
index 4df95266b261..13969c6910ce 100644
@@ -11,7 +11,7 @@
 struct dentry;
 struct xe_device;
 
-void xe_sriov_vf_init_early(struct xe_device *xe);
+int xe_sriov_vf_init_early(struct xe_device *xe);
 int xe_sriov_vf_init_late(struct xe_device *xe);
 bool xe_sriov_vf_migration_supported(struct xe_device *xe);
 void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
Signature · 1 line
2.34.1
#39 lore raw

Re: [PATCH v2 14/34] drm/xe/vf: Remove memory allocations from VF post migration recovery

Quoted 18 lines · Matthew Brost

VF post migration recovery is the path of dma-fence signaling / reclaim, avoid memory allocations in this path.

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 29 ++++++++++++++---------
  drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 +-
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  2 ++
  drivers/gpu/drm/xe/xe_sriov.c             |  8 +++++--
  drivers/gpu/drm/xe/xe_sriov_vf.c          | 14 ++++++++---
  drivers/gpu/drm/xe/xe_sriov_vf.h          |  2 +-
  6 files changed, 39 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index cfb71b749e52..8304c26c076e 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static size_t post_migration_scratch_size(struct xe_device *xe) · line 1217

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -1214,17 +1214,13 @@ static size_t post_migration_scratch_size(struct xe_device *xe)
 
  static int vf_post_migration_fixups(struct xe_gt *gt)
  {
+	void *buf = gt->sriov.vf.migration.lrc_wa_bb;

can we just call it 'scratch', or 'recovery_scratch'? The current use is limited to lrc, but besides temp buffer for WAs it also stores MEMIRQ regs.

This name makes it look like something more important than it is, and does not describe all uses.

Besides that name, LGTM.

-Tomasz

Quoted 150 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +17 -11
  	s64 shift;
-	void *buf;
  	int err;
 
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
-	if (!buf)
-		return -ENOMEM;
-
  	err = xe_gt_sriov_vf_query_config(gt);
  	if (err)
-		goto out;
+		return err;
 
  	shift = xe_gt_sriov_vf_ggtt_shift(gt);
  	if (shift) {
@@ -1232,12 +1228,10 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
  		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
  		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
  		if (err)
-			goto out;
+			return err;
  	}
 
-out:
-	kfree(buf);
-	return err;
+	return 0;
  }
 
  static void vf_post_migration_kickstart(struct xe_gt *gt)
@@ -1314,15 +1308,28 @@ static void migration_worker_func(struct work_struct *w)
  /**
   * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
   * @gt: the &xe_gt
+ *
+ * Return 0 on success, errno on failure
   */
-void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
  {
+	void *buf;
+
+	buf = drmm_kmalloc(&gt_to_xe(gt)->drm,
+			   post_migration_scratch_size(gt_to_xe(gt)),
+			   GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	gt->sriov.vf.migration.lrc_wa_bb = buf;
  	init_rwsem(&gt->sriov.vf.self_config.lock);
  	spin_lock_init(&gt->sriov.vf.migration.lock);
  	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
 
  	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
  		xe_gt_sriov_info(gt, "migration not supported by this module version\n");
+
+	return 0;
  }
 
  /**
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -1
index 2ac6775b52f0..195dbebe941e 100644
@@ -23,7 +23,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt);
  int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
  void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
-void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
+int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
  bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
 
  u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +2 -0
index 53680a2f188a..496b657119de 100644
@@ -58,6 +58,8 @@ struct xe_gt_sriov_vf_migration {
  	struct work_struct worker;
  	/** @lock: Protects recovery_queued */
  	spinlock_t lock;
+	/** @lrc_wa_bb: Scratch memory for LRC WA BB in recovery */
+	void *lrc_wa_bb;
  	/** @recovery_queued: VF post migration recovery in queued */
  	bool recovery_queued;
  	/** @recovery_inprogress: VF post migration recovery in progress */
drivers/gpu/drm/xe/xe_sriov.c +6 -2
index 7d2d6de2aabf..358a35c80d5a 100644
@@ -116,8 +116,12 @@ int xe_sriov_init(struct xe_device *xe)
  			return err;
  	}
 
-	if (IS_SRIOV_VF(xe))
-		xe_sriov_vf_init_early(xe);
+	if (IS_SRIOV_VF(xe)) {
+		int err = xe_sriov_vf_init_early(xe);
+
+		if (err)
+			return err;
+	}
 
  	xe_assert(xe, !xe->sriov.wq);
  	xe->sriov.wq = alloc_workqueue("xe-sriov-wq", 0, 0);
drivers/gpu/drm/xe/xe_sriov_vf.c +11 -3
index 7d91553c4acc..e622a7c562c4 100644
@@ -180,16 +180,24 @@ static void vf_migration_init_early(struct xe_device *xe)
  /**
   * xe_sriov_vf_init_early - Initialize SR-IOV VF specific data.
   * @xe: the &xe_device to initialize
+ *
+ * Return: 0 on success or a negative error code on failure.
   */
-void xe_sriov_vf_init_early(struct xe_device *xe)
+int xe_sriov_vf_init_early(struct xe_device *xe)
  {
  	struct xe_gt *gt;
  	unsigned int id;
+	int err;
 
-	for_each_gt(gt, xe, id)
-		xe_gt_sriov_vf_migration_init_early(gt);
+	for_each_gt(gt, xe, id) {
+		err = xe_gt_sriov_vf_migration_init_early(gt);
+		if (err)
+			return err;
+	}
 
  	vf_migration_init_early(xe);
+
+	return 0;
  }
 
  /**
drivers/gpu/drm/xe/xe_sriov_vf.h +1 -1
index 4df95266b261..13969c6910ce 100644
@@ -11,7 +11,7 @@
  struct dentry;
  struct xe_device;
 
-void xe_sriov_vf_init_early(struct xe_device *xe);
+int xe_sriov_vf_init_early(struct xe_device *xe);
  int xe_sriov_vf_init_late(struct xe_device *xe);
  bool xe_sriov_vf_migration_supported(struct xe_device *xe);
  void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
author maintainer
#40 lore raw

Re: [PATCH v2 14/34] drm/xe/vf: Remove memory allocations from VF post migration recovery

Quoted 27 lines · Tomasz
Quoted 18 lines · Matthew Brost

VF post migration recovery is the path of dma-fence signaling / reclaim, avoid memory allocations in this path.

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 29 ++++++++++++++---------
  drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 +-
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  2 ++
  drivers/gpu/drm/xe/xe_sriov.c             |  8 +++++--
  drivers/gpu/drm/xe/xe_sriov_vf.c          | 14 ++++++++---
  drivers/gpu/drm/xe/xe_sriov_vf.h          |  2 +-
  6 files changed, 39 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index cfb71b749e52..8304c26c076e 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static size_t post_migration_scratch_size(struct xe_device *xe) · line 1216

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -1214,17 +1214,13 @@ static size_t post_migration_scratch_size(struct xe_device *xe)
  static int vf_post_migration_fixups(struct xe_gt *gt)
  {
+	void *buf = gt->sriov.vf.migration.lrc_wa_bb;

can we just call it 'scratch', or 'recovery_scratch'? The current use is limited to lrc, but besides temp buffer for WAs it also stores MEMIRQ regs.

Quoted 4 lines

This name makes it look like something more important than it is, and does not describe all uses.

Besides that name, LGTM.

Yes, that is a better name. Will change.

Matt

Quoted 139 lines

-Tomasz

Quoted 136 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +17 -11
  	s64 shift;
-	void *buf;
  	int err;
-	buf = kmalloc(post_migration_scratch_size(gt_to_xe(gt)), GFP_ATOMIC);
-	if (!buf)
-		return -ENOMEM;
-
  	err = xe_gt_sriov_vf_query_config(gt);
  	if (err)
-		goto out;
+		return err;
  	shift = xe_gt_sriov_vf_ggtt_shift(gt);
  	if (shift) {
@@ -1232,12 +1228,10 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
  		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
  		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
  		if (err)
-			goto out;
+			return err;
  	}
-out:
-	kfree(buf);
-	return err;
+	return 0;
  }
  static void vf_post_migration_kickstart(struct xe_gt *gt)
@@ -1314,15 +1308,28 @@ static void migration_worker_func(struct work_struct *w)
  /**
   * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
   * @gt: the &xe_gt
+ *
+ * Return 0 on success, errno on failure
   */
-void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
+int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
  {
+	void *buf;
+
+	buf = drmm_kmalloc(&gt_to_xe(gt)->drm,
+			   post_migration_scratch_size(gt_to_xe(gt)),
+			   GFP_KERNEL);
+	if (!buf)
+		return -ENOMEM;
+
+	gt->sriov.vf.migration.lrc_wa_bb = buf;
  	init_rwsem(&gt->sriov.vf.self_config.lock);
  	spin_lock_init(&gt->sriov.vf.migration.lock);
  	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
  	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
  		xe_gt_sriov_info(gt, "migration not supported by this module version\n");
+
+	return 0;
  }
  /**
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -1
index 2ac6775b52f0..195dbebe941e 100644
@@ -23,7 +23,7 @@ int xe_gt_sriov_vf_connect(struct xe_gt *gt);
  int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
  void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
-void xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
+int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt);
  bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt);
  u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +2 -0
index 53680a2f188a..496b657119de 100644
@@ -58,6 +58,8 @@ struct xe_gt_sriov_vf_migration {
  	struct work_struct worker;
  	/** @lock: Protects recovery_queued */
  	spinlock_t lock;
+	/** @lrc_wa_bb: Scratch memory for LRC WA BB in recovery */
+	void *lrc_wa_bb;
  	/** @recovery_queued: VF post migration recovery in queued */
  	bool recovery_queued;
  	/** @recovery_inprogress: VF post migration recovery in progress */
drivers/gpu/drm/xe/xe_sriov.c +6 -2
index 7d2d6de2aabf..358a35c80d5a 100644
@@ -116,8 +116,12 @@ int xe_sriov_init(struct xe_device *xe)
  			return err;
  	}
-	if (IS_SRIOV_VF(xe))
-		xe_sriov_vf_init_early(xe);
+	if (IS_SRIOV_VF(xe)) {
+		int err = xe_sriov_vf_init_early(xe);
+
+		if (err)
+			return err;
+	}
  	xe_assert(xe, !xe->sriov.wq);
  	xe->sriov.wq = alloc_workqueue("xe-sriov-wq", 0, 0);
drivers/gpu/drm/xe/xe_sriov_vf.c +11 -3
index 7d91553c4acc..e622a7c562c4 100644
@@ -180,16 +180,24 @@ static void vf_migration_init_early(struct xe_device *xe)
  /**
   * xe_sriov_vf_init_early - Initialize SR-IOV VF specific data.
   * @xe: the &xe_device to initialize
+ *
+ * Return: 0 on success or a negative error code on failure.
   */
-void xe_sriov_vf_init_early(struct xe_device *xe)
+int xe_sriov_vf_init_early(struct xe_device *xe)
  {
  	struct xe_gt *gt;
  	unsigned int id;
+	int err;
-	for_each_gt(gt, xe, id)
-		xe_gt_sriov_vf_migration_init_early(gt);
+	for_each_gt(gt, xe, id) {
+		err = xe_gt_sriov_vf_migration_init_early(gt);
+		if (err)
+			return err;
+	}
  	vf_migration_init_early(xe);
+
+	return 0;
  }
  /**
drivers/gpu/drm/xe/xe_sriov_vf.h +1 -1
index 4df95266b261..13969c6910ce 100644
@@ -11,7 +11,7 @@
  struct dentry;
  struct xe_device;
-void xe_sriov_vf_init_early(struct xe_device *xe);
+int xe_sriov_vf_init_early(struct xe_device *xe);
  int xe_sriov_vf_init_late(struct xe_device *xe);
  bool xe_sriov_vf_migration_supported(struct xe_device *xe);
  void xe_sriov_vf_debugfs_register(struct xe_device *xe, struct dentry *root);
author maintainer
#41 lore raw

PATCH v2 15/34 drm/xe/vf: Close multi-GT GGTT shift race

In reply to Matthew Brost

As multi-GT VF post-migration recovery can run in parallel on different workqueues, but both GTs point to the same GGTT, only one GT needs to shift the GGTT. However, both GTs need to know when this step has completed. To coordinate this, share the VF config lock among all GTs that share a GGTT, and perform the GGTT shift under this lock.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 95 +++++++++--------------
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  3 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 11 ++-
 drivers/gpu/drm/xe/xe_guc.c               |  2 +-
 drivers/gpu/drm/xe/xe_tile_sriov_vf.c     |  6 +-
 drivers/gpu/drm/xe/xe_tile_sriov_vf.h     |  1 -
 6 files changed, 51 insertions(+), 67 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +37 -58
index 8304c26c076e..807fdced0228 100644
@@ -436,16 +436,19 @@ u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt)
 	return value;
 }
 
-static int vf_get_ggtt_info(struct xe_gt *gt)
+static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery)
 {
 	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	struct xe_gt_sriov_vf_selfconfig *primary_config =
+		&gt_to_tile(gt)->primary_gt->sriov.vf.self_config;
 	struct xe_guc *guc = &gt->uc.guc;
 	u64 start, size;
+	s64 shift;
 	int err;
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
-	down_write(&config->lock);
+	down_write(config->lock);
 
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
 	if (unlikely(err))
@@ -465,13 +468,17 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
 				start, start + size - 1, size / SZ_1K);
 
-	config->ggtt_shift = start - (s64)config->ggtt_base;
+	shift = start - (s64)primary_config->ggtt_base;
 	config->ggtt_base = start;
 	config->ggtt_size = size;
+	if (recovery)
+		primary_config->ggtt_base = start;
 	err = config->ggtt_size ? 0 : -ENODATA;
 
+	if (!err && shift && recovery)
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
 out:
-	up_write(&config->lock);
+	up_write(config->lock);
 	return err;
 }
 
@@ -485,7 +492,7 @@ static int vf_get_lmem_info(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
-	down_write(&config->lock);
+	down_write(config->lock);
 
 	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
 	if (unlikely(err))
@@ -505,7 +512,7 @@ static int vf_get_lmem_info(struct xe_gt *gt)
 	err = config->lmem_size ? 0 : -ENODATA;
 
 out:
-	up_write(&config->lock);
+	up_write(config->lock);
 	return err;
 }
 
@@ -518,7 +525,7 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
-	down_write(&config->lock);
+	down_write(config->lock);
 
 	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
 	if (unlikely(err))
@@ -549,7 +556,7 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 	err = config->num_ctxs ? 0 : -ENODATA;
 
 out:
-	up_write(&config->lock);
+	up_write(config->lock);
 	return err;
 }
 
@@ -564,17 +571,18 @@ static void vf_cache_gmdid(struct xe_gt *gt)
 /**
  * xe_gt_sriov_vf_query_config - Query SR-IOV config data over MMIO.
  * @gt: the &xe_gt
+ * @recovery: VF post migration recovery path
  *
  * This function is for VF use only.
  *
  * Return: 0 on success or a negative error code on failure.
  */
-int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
+int xe_gt_sriov_vf_query_config(struct xe_gt *gt, bool recovery)
 {
 	struct xe_device *xe = gt_to_xe(gt);
 	int err;
 
-	err = vf_get_ggtt_info(gt);
+	err = vf_get_ggtt_info(gt, recovery);
 	if (unlikely(err))
 		return err;
 
@@ -610,10 +618,10 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
 
-	down_read(&config->lock);
+	down_read(config->lock);
 	xe_gt_assert(gt, config->num_ctxs);
 	val = config->num_ctxs;
-	up_read(&config->lock);
+	up_read(config->lock);
 
 	return val;
 }
@@ -634,10 +642,10 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
 
-	down_read(&config->lock);
+	down_read(config->lock);
 	xe_gt_assert(gt, config->lmem_size);
 	val = config->lmem_size;
-	up_read(&config->lock);
+	up_read(config->lock);
 
 	return val;
 }
@@ -656,11 +664,9 @@ u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
 	u64 val;
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
+	lockdep_assert_held(config->lock);
 
-	down_read(&config->lock);
 	val = config->ggtt_size;
-	up_read(&config->lock);
 
 	return val;
 }
@@ -680,34 +686,10 @@ u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-
-	down_read(&config->lock);
 	xe_gt_assert(gt, config->ggtt_size);
-	val = config->ggtt_base;
-	up_read(&config->lock);
-
-	return val;
-}
+	lockdep_assert_held(config->lock);
 
-/**
- * xe_gt_sriov_vf_ggtt_shift - Return shift in GGTT range due to VF migration
- * @gt: the &xe_gt struct instance
- *
- * This function is for VF use only.
- *
- * Return: The shift value; could be negative
- */
-s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)
-{
-	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
-	s64 val;
-
-	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	xe_gt_assert(gt, xe_gt_is_main_type(gt));
-
-	down_read(&config->lock);
-	val = config->ggtt_shift;
-	up_read(&config->lock);
+	val = config->ggtt_base;
 
 	return val;
 }
@@ -1115,7 +1097,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
-	down_read(&config->lock);
+	down_read(config->lock);
 	drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
 		   config->ggtt_base,
 		   config->ggtt_base + config->ggtt_size - 1);
@@ -1123,8 +1105,6 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 	string_get_size(config->ggtt_size, 1, STRING_UNITS_2, buf, sizeof(buf));
 	drm_printf(p, "GGTT size:\t%llu (%s)\n", config->ggtt_size, buf);
 
-	drm_printf(p, "GGTT shift on last restore:\t%lld\n", config->ggtt_shift);
-
 	if (IS_DGFX(xe) && xe_gt_is_main_type(gt)) {
 		string_get_size(config->lmem_size, 1, STRING_UNITS_2, buf, sizeof(buf));
 		drm_printf(p, "LMEM size:\t%llu (%s)\n", config->lmem_size, buf);
@@ -1132,7 +1112,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
 	drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
 	drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
-	up_read(&config->lock);
+	up_read(config->lock);
 }
 
 /**
@@ -1215,21 +1195,16 @@ static size_t post_migration_scratch_size(struct xe_device *xe)
 static int vf_post_migration_fixups(struct xe_gt *gt)
 {
 	void *buf = gt->sriov.vf.migration.lrc_wa_bb;
-	s64 shift;
 	int err;
 
-	err = xe_gt_sriov_vf_query_config(gt);
+	err = xe_gt_sriov_vf_query_config(gt, true);
 	if (err)
 		return err;
 
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			return err;
-	}
+	xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+	err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+	if (err)
+		return err;
 
 	return 0;
 }
@@ -1313,6 +1288,7 @@ static void migration_worker_func(struct work_struct *w)
  */
 int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
 {
+	struct xe_tile *tile = gt_to_tile(gt);
 	void *buf;
 
 	buf = drmm_kmalloc(&gt_to_xe(gt)->drm,
@@ -1322,7 +1298,10 @@ int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
 		return -ENOMEM;
 
 	gt->sriov.vf.migration.lrc_wa_bb = buf;
-	init_rwsem(&gt->sriov.vf.self_config.lock);
+	if (xe_gt_is_main_type(gt))
+		init_rwsem(&gt->sriov.vf.self_config.__lock);
+	gt->sriov.vf.self_config.lock =
+		&tile->primary_gt->sriov.vf.self_config.__lock;
 	spin_lock_init(&gt->sriov.vf.migration.lock);
 	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
 
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index 195dbebe941e..535237003915 100644
@@ -18,7 +18,7 @@ int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);
 void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
 				 struct xe_uc_fw_version *wanted,
 				 struct xe_uc_fw_version *found);
-int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
+int xe_gt_sriov_vf_query_config(struct xe_gt *gt, bool recovery);
 int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
@@ -31,7 +31,6 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
 u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
 u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt);
 u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt);
-s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt);
 
 u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg);
 void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -4
index 496b657119de..61484c7c9a36 100644
@@ -19,16 +19,19 @@ struct xe_gt_sriov_vf_selfconfig {
 	u64 ggtt_base;
 	/** @ggtt_size: assigned size of the GGTT region. */
 	u64 ggtt_size;
-	/** @ggtt_shift: difference in ggtt_base on last migration */
-	s64 ggtt_shift;
 	/** @lmem_size: assigned size of the LMEM. */
 	u64 lmem_size;
 	/** @num_ctxs: assigned number of GuC submission context IDs. */
 	u16 num_ctxs;
 	/** @num_dbs: assigned number of GuC doorbells IDs. */
 	u16 num_dbs;
-	/** @lock: lock for protecting access to all selfconfig fields. */
-	struct rw_semaphore lock;
+	/** @__lock: lock for protecting access to all selfconfig fields. */
+	struct rw_semaphore __lock;
+	/**
+	 * @lock: pointer to lock for protecting access to all selfconfig
+	 * fields, all GTs point to primary GT.
+	 */
+	struct rw_semaphore *lock;
 };
 
 /**
drivers/gpu/drm/xe/xe_guc.c +1 -1
index 00789844ea4d..ac60da51da2c 100644
@@ -712,7 +712,7 @@ static int vf_guc_init_noalloc(struct xe_guc *guc)
 	if (err)
 		return err;
 
-	err = xe_gt_sriov_vf_query_config(gt);
+	err = xe_gt_sriov_vf_query_config(gt, false);
 	if (err)
 		return err;
 
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +5 -1
index f221dbed16f0..dc6221fc0520 100644
@@ -40,7 +40,7 @@ static int vf_init_ggtt_balloons(struct xe_tile *tile)
  *
  * Return: 0 on success or a negative error code on failure.
  */
-int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
+static int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
 {
 	u64 ggtt_base = xe_gt_sriov_vf_ggtt_base(tile->primary_gt);
 	u64 ggtt_size = xe_gt_sriov_vf_ggtt(tile->primary_gt);
@@ -100,12 +100,16 @@ int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
 
 static int vf_balloon_ggtt(struct xe_tile *tile)
 {
+	struct xe_gt_sriov_vf_selfconfig *config =
+		&tile->primary_gt->sriov.vf.self_config;
 	struct xe_ggtt *ggtt = tile->mem.ggtt;
 	int err;
 
+	down_read(config->lock);
 	mutex_lock(&ggtt->lock);
 	err = xe_tile_sriov_vf_balloon_ggtt_locked(tile);
 	mutex_unlock(&ggtt->lock);
+	up_read(config->lock);
 
 	return err;
 }
drivers/gpu/drm/xe/xe_tile_sriov_vf.h +0 -1
index 93eb043171e8..4ee68d1fb28e 100644
@@ -11,7 +11,6 @@
 struct xe_tile;
 
 int xe_tile_sriov_vf_prepare_ggtt(struct xe_tile *tile);
-int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile);
 void xe_tile_sriov_vf_deballoon_ggtt_locked(struct xe_tile *tile);
 void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift);
 
Signature · 1 line
2.34.1
#42 lore raw

Re: [PATCH v2 15/34] drm/xe/vf: Close multi-GT GGTT shift race

Quoted 5 lines · Matthew Brost

As multi-GT VF post-migration recovery can run in parallel on different workqueues, but both GTs point to the same GGTT, only one GT needs to shift the GGTT. However, both GTs need to know when this step has completed. To coordinate this, share the VF config lock among all GTs that share a GGTT, and perform the GGTT shift under this lock.

The description does not mention removal of ggtt_shift variable; this removal is not related to the locking change, so should be separately mentioned.

Quoted 199 lines
---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 95 +++++++++--------------
  drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  3 +-
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 11 ++-
  drivers/gpu/drm/xe/xe_guc.c               |  2 +-
  drivers/gpu/drm/xe/xe_tile_sriov_vf.c     |  6 +-
  drivers/gpu/drm/xe/xe_tile_sriov_vf.h     |  1 -
  6 files changed, 51 insertions(+), 67 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +26 -44
index 8304c26c076e..807fdced0228 100644
@@ -436,16 +436,19 @@ u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt)
  	return value;
  }
 
-static int vf_get_ggtt_info(struct xe_gt *gt)
+static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery)
  {
  	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	struct xe_gt_sriov_vf_selfconfig *primary_config =
+		&gt_to_tile(gt)->primary_gt->sriov.vf.self_config;
  	struct xe_guc *guc = &gt->uc.guc;
  	u64 start, size;
+	s64 shift;
  	int err;
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
-	down_write(&config->lock);
+	down_write(config->lock);
 
  	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
  	if (unlikely(err))
@@ -465,13 +468,17 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
  	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
  				start, start + size - 1, size / SZ_1K);
 
-	config->ggtt_shift = start - (s64)config->ggtt_base;
+	shift = start - (s64)primary_config->ggtt_base;
  	config->ggtt_base = start;
  	config->ggtt_size = size;
+	if (recovery)
+		primary_config->ggtt_base = start;
  	err = config->ggtt_size ? 0 : -ENODATA;
 
+	if (!err && shift && recovery)
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
  out:
-	up_write(&config->lock);
+	up_write(config->lock);
  	return err;
  }
 
@@ -485,7 +492,7 @@ static int vf_get_lmem_info(struct xe_gt *gt)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
-	down_write(&config->lock);
+	down_write(config->lock);
 
  	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
  	if (unlikely(err))
@@ -505,7 +512,7 @@ static int vf_get_lmem_info(struct xe_gt *gt)
  	err = config->lmem_size ? 0 : -ENODATA;
 
  out:
-	up_write(&config->lock);
+	up_write(config->lock);
  	return err;
  }
 
@@ -518,7 +525,7 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
-	down_write(&config->lock);
+	down_write(config->lock);
 
  	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
  	if (unlikely(err))
@@ -549,7 +556,7 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
  	err = config->num_ctxs ? 0 : -ENODATA;
 
  out:
-	up_write(&config->lock);
+	up_write(config->lock);
  	return err;
  }
 
@@ -564,17 +571,18 @@ static void vf_cache_gmdid(struct xe_gt *gt)
  /**
   * xe_gt_sriov_vf_query_config - Query SR-IOV config data over MMIO.
   * @gt: the &xe_gt
+ * @recovery: VF post migration recovery path
   *
   * This function is for VF use only.
   *
   * Return: 0 on success or a negative error code on failure.
   */
-int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
+int xe_gt_sriov_vf_query_config(struct xe_gt *gt, bool recovery)
  {
  	struct xe_device *xe = gt_to_xe(gt);
  	int err;
 
-	err = vf_get_ggtt_info(gt);
+	err = vf_get_ggtt_info(gt, recovery);
  	if (unlikely(err))
  		return err;
 
@@ -610,10 +618,10 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
 
-	down_read(&config->lock);
+	down_read(config->lock);
  	xe_gt_assert(gt, config->num_ctxs);
  	val = config->num_ctxs;
-	up_read(&config->lock);
+	up_read(config->lock);
 
  	return val;
  }
@@ -634,10 +642,10 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
 
-	down_read(&config->lock);
+	down_read(config->lock);
  	xe_gt_assert(gt, config->lmem_size);
  	val = config->lmem_size;
-	up_read(&config->lock);
+	up_read(config->lock);
 
  	return val;
  }
@@ -656,11 +664,9 @@ u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
  	u64 val;
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
+	lockdep_assert_held(config->lock);
 
-	down_read(&config->lock);
  	val = config->ggtt_size;
-	up_read(&config->lock);
 
  	return val;
  }
@@ -680,34 +686,10 @@ u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-
-	down_read(&config->lock);
  	xe_gt_assert(gt, config->ggtt_size);
-	val = config->ggtt_base;
-	up_read(&config->lock);
-
-	return val;
-}
+	lockdep_assert_held(config->lock);
 
-/**
- * xe_gt_sriov_vf_ggtt_shift - Return shift in GGTT range due to VF migration
- * @gt: the &xe_gt struct instance
- *
- * This function is for VF use only.
- *
- * Return: The shift value; could be negative
- */
-s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)
-{
-	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
-	s64 val;
-
-	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	xe_gt_assert(gt, xe_gt_is_main_type(gt));
-
-	down_read(&config->lock);
-	val = config->ggtt_shift;
-	up_read(&config->lock);
+	val = config->ggtt_base;
 
  	return val;
  }
@@ -1115,7 +1097,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
 
-	down_read(&config->lock);
+	down_read(config->lock);
  	drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
  		   config->ggtt_base,
  		   config->ggtt_base + config->ggtt_size - 1);

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p) · line 1127

Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -2
@@ -1123,8 +1105,6 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
  	string_get_size(config->ggtt_size, 1, STRING_UNITS_2, buf, sizeof(buf));
  	drm_printf(p, "GGTT size:\t%llu (%s)\n", config->ggtt_size, buf);
 
-	drm_printf(p, "GGTT shift on last restore:\t%lld\n", config->ggtt_shift);
-

Right.. so by that we're losing a useful debug value.

I'm not very attached to it as I did not originated the idea to have it there.

IMO the GGTT config is printed to dmesg and that's enough, shift can be computed from that.

For the main change - the lock and the multi-GT support - looks good, no issues.

-Tomasz

Quoted 163 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +11 -12
  	if (IS_DGFX(xe) && xe_gt_is_main_type(gt)) {
  		string_get_size(config->lmem_size, 1, STRING_UNITS_2, buf, sizeof(buf));
  		drm_printf(p, "LMEM size:\t%llu (%s)\n", config->lmem_size, buf);
@@ -1132,7 +1112,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
 
  	drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
  	drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
-	up_read(&config->lock);
+	up_read(config->lock);
  }
 
  /**
@@ -1215,21 +1195,16 @@ static size_t post_migration_scratch_size(struct xe_device *xe)
  static int vf_post_migration_fixups(struct xe_gt *gt)
  {
  	void *buf = gt->sriov.vf.migration.lrc_wa_bb;
-	s64 shift;
  	int err;
 
-	err = xe_gt_sriov_vf_query_config(gt);
+	err = xe_gt_sriov_vf_query_config(gt, true);
  	if (err)
  		return err;
 
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			return err;
-	}
+	xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+	err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+	if (err)
+		return err;
 
  	return 0;
  }
@@ -1313,6 +1288,7 @@ static void migration_worker_func(struct work_struct *w)
   */
  int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
  {
+	struct xe_tile *tile = gt_to_tile(gt);
  	void *buf;
 
  	buf = drmm_kmalloc(&gt_to_xe(gt)->drm,
@@ -1322,7 +1298,10 @@ int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
  		return -ENOMEM;
 
  	gt->sriov.vf.migration.lrc_wa_bb = buf;
-	init_rwsem(&gt->sriov.vf.self_config.lock);
+	if (xe_gt_is_main_type(gt))
+		init_rwsem(&gt->sriov.vf.self_config.__lock);
+	gt->sriov.vf.self_config.lock =
+		&tile->primary_gt->sriov.vf.self_config.__lock;
  	spin_lock_init(&gt->sriov.vf.migration.lock);
  	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
 
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index 195dbebe941e..535237003915 100644
@@ -18,7 +18,7 @@ int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);
  void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
  				 struct xe_uc_fw_version *wanted,
  				 struct xe_uc_fw_version *found);
-int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
+int xe_gt_sriov_vf_query_config(struct xe_gt *gt, bool recovery);
  int xe_gt_sriov_vf_connect(struct xe_gt *gt);
  int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
  void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
@@ -31,7 +31,6 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
  u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
  u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt);
  u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt);
-s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt);
 
  u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg);
  void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -4
index 496b657119de..61484c7c9a36 100644
@@ -19,16 +19,19 @@ struct xe_gt_sriov_vf_selfconfig {
  	u64 ggtt_base;
  	/** @ggtt_size: assigned size of the GGTT region. */
  	u64 ggtt_size;
-	/** @ggtt_shift: difference in ggtt_base on last migration */
-	s64 ggtt_shift;
  	/** @lmem_size: assigned size of the LMEM. */
  	u64 lmem_size;
  	/** @num_ctxs: assigned number of GuC submission context IDs. */
  	u16 num_ctxs;
  	/** @num_dbs: assigned number of GuC doorbells IDs. */
  	u16 num_dbs;
-	/** @lock: lock for protecting access to all selfconfig fields. */
-	struct rw_semaphore lock;
+	/** @__lock: lock for protecting access to all selfconfig fields. */
+	struct rw_semaphore __lock;
+	/**
+	 * @lock: pointer to lock for protecting access to all selfconfig
+	 * fields, all GTs point to primary GT.
+	 */
+	struct rw_semaphore *lock;
  };
 
  /**
drivers/gpu/drm/xe/xe_guc.c +1 -1
index 00789844ea4d..ac60da51da2c 100644
@@ -712,7 +712,7 @@ static int vf_guc_init_noalloc(struct xe_guc *guc)
  	if (err)
  		return err;
 
-	err = xe_gt_sriov_vf_query_config(gt);
+	err = xe_gt_sriov_vf_query_config(gt, false);
  	if (err)
  		return err;
 
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +5 -1
index f221dbed16f0..dc6221fc0520 100644
@@ -40,7 +40,7 @@ static int vf_init_ggtt_balloons(struct xe_tile *tile)
   *
   * Return: 0 on success or a negative error code on failure.
   */
-int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
+static int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
  {
  	u64 ggtt_base = xe_gt_sriov_vf_ggtt_base(tile->primary_gt);
  	u64 ggtt_size = xe_gt_sriov_vf_ggtt(tile->primary_gt);
@@ -100,12 +100,16 @@ int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
 
  static int vf_balloon_ggtt(struct xe_tile *tile)
  {
+	struct xe_gt_sriov_vf_selfconfig *config =
+		&tile->primary_gt->sriov.vf.self_config;
  	struct xe_ggtt *ggtt = tile->mem.ggtt;
  	int err;
 
+	down_read(config->lock);
  	mutex_lock(&ggtt->lock);
  	err = xe_tile_sriov_vf_balloon_ggtt_locked(tile);
  	mutex_unlock(&ggtt->lock);
+	up_read(config->lock);
 
  	return err;
  }
drivers/gpu/drm/xe/xe_tile_sriov_vf.h +0 -1
index 93eb043171e8..4ee68d1fb28e 100644
@@ -11,7 +11,6 @@
  struct xe_tile;
 
  int xe_tile_sriov_vf_prepare_ggtt(struct xe_tile *tile);
-int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile);
  void xe_tile_sriov_vf_deballoon_ggtt_locked(struct xe_tile *tile);
  void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift);
 
author maintainer
#43 lore raw

Re: [PATCH v2 15/34] drm/xe/vf: Close multi-GT GGTT shift race

Quoted 8 lines · Tomasz
Quoted 5 lines · Matthew Brost

As multi-GT VF post-migration recovery can run in parallel on different workqueues, but both GTs point to the same GGTT, only one GT needs to shift the GGTT. However, both GTs need to know when this step has completed. To coordinate this, share the VF config lock among all GTs that share a GGTT, and perform the GGTT shift under this lock.

Quoted 4 lines

The description does not mention removal of ggtt_shift variable; this removal is not related to the locking change, so should be separately mentioned.

The shift doesn't really need to be stored when shift is done under config lock, rather just calculated there. I mention something like this in the commit message.

Quoted 175 lines
Quoted 168 lines
---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 95 +++++++++--------------
  drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  3 +-
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h | 11 ++-
  drivers/gpu/drm/xe/xe_guc.c               |  2 +-
  drivers/gpu/drm/xe/xe_tile_sriov_vf.c     |  6 +-
  drivers/gpu/drm/xe/xe_tile_sriov_vf.h     |  1 -
  6 files changed, 51 insertions(+), 67 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +26 -44
index 8304c26c076e..807fdced0228 100644
@@ -436,16 +436,19 @@ u32 xe_gt_sriov_vf_gmdid(struct xe_gt *gt)
  	return value;
  }
-static int vf_get_ggtt_info(struct xe_gt *gt)
+static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery)
  {
  	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+	struct xe_gt_sriov_vf_selfconfig *primary_config =
+		&gt_to_tile(gt)->primary_gt->sriov.vf.self_config;
  	struct xe_guc *guc = &gt->uc.guc;
  	u64 start, size;
+	s64 shift;
  	int err;
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	down_write(&config->lock);
+	down_write(config->lock);
  	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_GGTT_START_KEY, &start);
  	if (unlikely(err))
@@ -465,13 +468,17 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
  	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
  				start, start + size - 1, size / SZ_1K);
-	config->ggtt_shift = start - (s64)config->ggtt_base;
+	shift = start - (s64)primary_config->ggtt_base;
  	config->ggtt_base = start;
  	config->ggtt_size = size;
+	if (recovery)
+		primary_config->ggtt_base = start;
  	err = config->ggtt_size ? 0 : -ENODATA;
+	if (!err && shift && recovery)
+		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
  out:
-	up_write(&config->lock);
+	up_write(config->lock);
  	return err;
  }
@@ -485,7 +492,7 @@ static int vf_get_lmem_info(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	down_write(&config->lock);
+	down_write(config->lock);
  	err = guc_action_query_single_klv64(guc, GUC_KLV_VF_CFG_LMEM_SIZE_KEY, &size);
  	if (unlikely(err))
@@ -505,7 +512,7 @@ static int vf_get_lmem_info(struct xe_gt *gt)
  	err = config->lmem_size ? 0 : -ENODATA;
  out:
-	up_write(&config->lock);
+	up_write(config->lock);
  	return err;
  }
@@ -518,7 +525,7 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	down_write(&config->lock);
+	down_write(config->lock);
  	err = guc_action_query_single_klv32(guc, GUC_KLV_VF_CFG_NUM_CONTEXTS_KEY, &num_ctxs);
  	if (unlikely(err))
@@ -549,7 +556,7 @@ static int vf_get_submission_cfg(struct xe_gt *gt)
  	err = config->num_ctxs ? 0 : -ENODATA;
  out:
-	up_write(&config->lock);
+	up_write(config->lock);
  	return err;
  }
@@ -564,17 +571,18 @@ static void vf_cache_gmdid(struct xe_gt *gt)
  /**
   * xe_gt_sriov_vf_query_config - Query SR-IOV config data over MMIO.
   * @gt: the &xe_gt
+ * @recovery: VF post migration recovery path
   *
   * This function is for VF use only.
   *
   * Return: 0 on success or a negative error code on failure.
   */
-int xe_gt_sriov_vf_query_config(struct xe_gt *gt)
+int xe_gt_sriov_vf_query_config(struct xe_gt *gt, bool recovery)
  {
  	struct xe_device *xe = gt_to_xe(gt);
  	int err;
-	err = vf_get_ggtt_info(gt);
+	err = vf_get_ggtt_info(gt, recovery);
  	if (unlikely(err))
  		return err;
@@ -610,10 +618,10 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	down_read(&config->lock);
+	down_read(config->lock);
  	xe_gt_assert(gt, config->num_ctxs);
  	val = config->num_ctxs;
-	up_read(&config->lock);
+	up_read(config->lock);
  	return val;
  }
@@ -634,10 +642,10 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-	down_read(&config->lock);
+	down_read(config->lock);
  	xe_gt_assert(gt, config->lmem_size);
  	val = config->lmem_size;
-	up_read(&config->lock);
+	up_read(config->lock);
  	return val;
  }
@@ -656,11 +664,9 @@ u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt)
  	u64 val;
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
+	lockdep_assert_held(config->lock);
-	down_read(&config->lock);
  	val = config->ggtt_size;
-	up_read(&config->lock);
  	return val;
  }
@@ -680,34 +686,10 @@ u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt)
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
  	xe_gt_assert(gt, gt->sriov.vf.guc_version.major);
-
-	down_read(&config->lock);
  	xe_gt_assert(gt, config->ggtt_size);
-	val = config->ggtt_base;
-	up_read(&config->lock);
-
-	return val;
-}
+	lockdep_assert_held(config->lock);
-/**
- * xe_gt_sriov_vf_ggtt_shift - Return shift in GGTT range due to VF migration
- * @gt: the &xe_gt struct instance
- *
- * This function is for VF use only.
- *
- * Return: The shift value; could be negative
- */
-s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)
-{
-	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
-	s64 val;
-
-	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	xe_gt_assert(gt, xe_gt_is_main_type(gt));
-
-	down_read(&config->lock);
-	val = config->ggtt_shift;
-	up_read(&config->lock);
+	val = config->ggtt_base;
  	return val;
  }
@@ -1115,7 +1097,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
  	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
-	down_read(&config->lock);
+	down_read(config->lock);
  	drm_printf(p, "GGTT range:\t%#llx-%#llx\n",
  		   config->ggtt_base,
  		   config->ggtt_base + config->ggtt_size - 1);

drivers/gpu/drm/xe/xe_gt_sriov_vf.c void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p) · line 1126

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -2
@@ -1123,8 +1105,6 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
  	string_get_size(config->ggtt_size, 1, STRING_UNITS_2, buf, sizeof(buf));
  	drm_printf(p, "GGTT size:\t%llu (%s)\n", config->ggtt_size, buf);
-	drm_printf(p, "GGTT shift on last restore:\t%lld\n", config->ggtt_shift);
-

Right.. so by that we're losing a useful debug value.

Quoted 6 lines

I'm not very attached to it as I did not originated the idea to have it there.

IMO the GGTT config is printed to dmesg and that's enough, shift can be computed from that.

I added this message [1] so when shift the occurs we can see it happening and also see which GT won the shift race. I've manually verified on BMG either GT can safely win this race.

Matt

[1] https://patchwork.freedesktop.org/patch/676386/?series=154627&rev=2

Quoted 152 lines

For the main change - the lock and the multi-GT support - looks good, no issues.

-Tomasz

Quoted 146 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +11 -12
  	if (IS_DGFX(xe) && xe_gt_is_main_type(gt)) {
  		string_get_size(config->lmem_size, 1, STRING_UNITS_2, buf, sizeof(buf));
  		drm_printf(p, "LMEM size:\t%llu (%s)\n", config->lmem_size, buf);
@@ -1132,7 +1112,7 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p)
  	drm_printf(p, "GuC contexts:\t%u\n", config->num_ctxs);
  	drm_printf(p, "GuC doorbells:\t%u\n", config->num_dbs);
-	up_read(&config->lock);
+	up_read(config->lock);
  }
  /**
@@ -1215,21 +1195,16 @@ static size_t post_migration_scratch_size(struct xe_device *xe)
  static int vf_post_migration_fixups(struct xe_gt *gt)
  {
  	void *buf = gt->sriov.vf.migration.lrc_wa_bb;
-	s64 shift;
  	int err;
-	err = xe_gt_sriov_vf_query_config(gt);
+	err = xe_gt_sriov_vf_query_config(gt, true);
  	if (err)
  		return err;
-	shift = xe_gt_sriov_vf_ggtt_shift(gt);
-	if (shift) {
-		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
-		xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
-		err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
-		if (err)
-			return err;
-	}
+	xe_gt_sriov_vf_default_lrcs_hwsp_rebase(gt);
+	err = xe_guc_contexts_hwsp_rebase(&gt->uc.guc, buf);
+	if (err)
+		return err;
  	return 0;
  }
@@ -1313,6 +1288,7 @@ static void migration_worker_func(struct work_struct *w)
   */
  int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
  {
+	struct xe_tile *tile = gt_to_tile(gt);
  	void *buf;
  	buf = drmm_kmalloc(&gt_to_xe(gt)->drm,
@@ -1322,7 +1298,10 @@ int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
  		return -ENOMEM;
  	gt->sriov.vf.migration.lrc_wa_bb = buf;
-	init_rwsem(&gt->sriov.vf.self_config.lock);
+	if (xe_gt_is_main_type(gt))
+		init_rwsem(&gt->sriov.vf.self_config.__lock);
+	gt->sriov.vf.self_config.lock =
+		&tile->primary_gt->sriov.vf.self_config.__lock;
  	spin_lock_init(&gt->sriov.vf.migration.lock);
  	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -2
index 195dbebe941e..535237003915 100644
@@ -18,7 +18,7 @@ int xe_gt_sriov_vf_bootstrap(struct xe_gt *gt);
  void xe_gt_sriov_vf_guc_versions(struct xe_gt *gt,
  				 struct xe_uc_fw_version *wanted,
  				 struct xe_uc_fw_version *found);
-int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
+int xe_gt_sriov_vf_query_config(struct xe_gt *gt, bool recovery);
  int xe_gt_sriov_vf_connect(struct xe_gt *gt);
  int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
  void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
@@ -31,7 +31,6 @@ u16 xe_gt_sriov_vf_guc_ids(struct xe_gt *gt);
  u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt);
  u64 xe_gt_sriov_vf_ggtt(struct xe_gt *gt);
  u64 xe_gt_sriov_vf_ggtt_base(struct xe_gt *gt);
-s64 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt);
  u32 xe_gt_sriov_vf_read32(struct xe_gt *gt, struct xe_reg reg);
  void xe_gt_sriov_vf_write32(struct xe_gt *gt, struct xe_reg reg, u32 val);
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +7 -4
index 496b657119de..61484c7c9a36 100644
@@ -19,16 +19,19 @@ struct xe_gt_sriov_vf_selfconfig {
  	u64 ggtt_base;
  	/** @ggtt_size: assigned size of the GGTT region. */
  	u64 ggtt_size;
-	/** @ggtt_shift: difference in ggtt_base on last migration */
-	s64 ggtt_shift;
  	/** @lmem_size: assigned size of the LMEM. */
  	u64 lmem_size;
  	/** @num_ctxs: assigned number of GuC submission context IDs. */
  	u16 num_ctxs;
  	/** @num_dbs: assigned number of GuC doorbells IDs. */
  	u16 num_dbs;
-	/** @lock: lock for protecting access to all selfconfig fields. */
-	struct rw_semaphore lock;
+	/** @__lock: lock for protecting access to all selfconfig fields. */
+	struct rw_semaphore __lock;
+	/**
+	 * @lock: pointer to lock for protecting access to all selfconfig
+	 * fields, all GTs point to primary GT.
+	 */
+	struct rw_semaphore *lock;
  };
  /**
drivers/gpu/drm/xe/xe_guc.c +1 -1
index 00789844ea4d..ac60da51da2c 100644
@@ -712,7 +712,7 @@ static int vf_guc_init_noalloc(struct xe_guc *guc)
  	if (err)
  		return err;
-	err = xe_gt_sriov_vf_query_config(gt);
+	err = xe_gt_sriov_vf_query_config(gt, false);
  	if (err)
  		return err;
drivers/gpu/drm/xe/xe_tile_sriov_vf.c +5 -1
index f221dbed16f0..dc6221fc0520 100644
@@ -40,7 +40,7 @@ static int vf_init_ggtt_balloons(struct xe_tile *tile)
   *
   * Return: 0 on success or a negative error code on failure.
   */
-int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
+static int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
  {
  	u64 ggtt_base = xe_gt_sriov_vf_ggtt_base(tile->primary_gt);
  	u64 ggtt_size = xe_gt_sriov_vf_ggtt(tile->primary_gt);
@@ -100,12 +100,16 @@ int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile)
  static int vf_balloon_ggtt(struct xe_tile *tile)
  {
+	struct xe_gt_sriov_vf_selfconfig *config =
+		&tile->primary_gt->sriov.vf.self_config;
  	struct xe_ggtt *ggtt = tile->mem.ggtt;
  	int err;
+	down_read(config->lock);
  	mutex_lock(&ggtt->lock);
  	err = xe_tile_sriov_vf_balloon_ggtt_locked(tile);
  	mutex_unlock(&ggtt->lock);
+	up_read(config->lock);
  	return err;
  }
drivers/gpu/drm/xe/xe_tile_sriov_vf.h +0 -1
index 93eb043171e8..4ee68d1fb28e 100644
@@ -11,7 +11,6 @@
  struct xe_tile;
  int xe_tile_sriov_vf_prepare_ggtt(struct xe_tile *tile);
-int xe_tile_sriov_vf_balloon_ggtt_locked(struct xe_tile *tile);
  void xe_tile_sriov_vf_deballoon_ggtt_locked(struct xe_tile *tile);
  void xe_tile_sriov_vf_fixup_ggtt_nodes(struct xe_tile *tile, s64 shift);
author maintainer
#44 lore raw

PATCH v2 16/34 drm/xe/vf: Teardown VF post migration worker on driver unload

In reply to Matthew Brost

Be cautious and ensure the VF post-migration worker is not running during driver unload.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 17 +++++++++++++++--
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  4 +++-
 2 files changed, 18 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +15 -2
index 807fdced0228..4eaffad6ebcf 100644
@@ -811,7 +811,8 @@ static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)
 
 	spin_lock(&gt->sriov.vf.migration.lock);
 
-	if (!gt->sriov.vf.migration.recovery_queued) {
+	if (!gt->sriov.vf.migration.recovery_queued ||
+	    !gt->sriov.vf.migration.recovery_teardown) {
 		gt->sriov.vf.migration.recovery_queued = true;
 		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
 
@@ -1280,6 +1281,17 @@ static void migration_worker_func(struct work_struct *w)
 	vf_post_migration_recovery(gt);
 }
 
+static void vf_migration_fini(struct drm_device *drm, void *arg)
+{
+	struct xe_gt *gt = arg;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_teardown = true;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	cancel_work_sync(&gt->sriov.vf.migration.worker);
+}
+
 /**
  * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
  * @gt: the &xe_gt
@@ -1308,7 +1320,8 @@ int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
 	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
 		xe_gt_sriov_info(gt, "migration not supported by this module version\n");
 
-	return 0;
+	return drmm_add_action_or_reset(&gt_to_xe(gt)->drm,
+					vf_migration_fini, gt);
 }
 
 /**
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -1
index 61484c7c9a36..beb9978336bb 100644
@@ -59,10 +59,12 @@ struct xe_gt_sriov_vf_runtime {
 struct xe_gt_sriov_vf_migration {
 	/** @migration: VF migration recovery worker */
 	struct work_struct worker;
-	/** @lock: Protects recovery_queued */
+	/** @lock: Protects recovery_queued, teardown */
 	spinlock_t lock;
 	/** @lrc_wa_bb: Scratch memory for LRC WA BB in recovery */
 	void *lrc_wa_bb;
+	/** @recovery_teardown: VF post migration recovery is being torn down */
+	bool recovery_teardown;
 	/** @recovery_queued: VF post migration recovery in queued */
 	bool recovery_queued;
 	/** @recovery_inprogress: VF post migration recovery in progress */
Signature · 1 line
2.34.1
#45 lore raw

Re: [PATCH v2 16/34] drm/xe/vf: Teardown VF post migration worker on driver unload

Quoted 14 lines · Matthew Brost

Be cautious and ensure the VF post-migration worker is not running during driver unload.

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 17 +++++++++++++++--
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  4 +++-
  2 files changed, 18 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index 807fdced0228..4eaffad6ebcf 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt) · line 815

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
@@ -811,7 +811,8 @@ static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)
 
  	spin_lock(&gt->sriov.vf.migration.lock);
 
-	if (!gt->sriov.vf.migration.recovery_queued) {
+	if (!gt->sriov.vf.migration.recovery_queued ||
+	    !gt->sriov.vf.migration.recovery_teardown) {

We're registering vf_migration_fini very early in the init. That means it will be called very late. With that in mind, is it even possible to hit?

During xe_gt_sriov_vf_migration_init_early, interrupts are not enabled yet. Doesn't that mean they are disabled already when vf_migration_fini is called?

Both ggtt_fini_early and guc_submit_fini should be finished by then; so if the recovery was running, it already crashed or errored out at this point.

So, maybe register the fini later? We need to init before IRQs are enabled, but for fini - it should be before exec_queue_lookup is torn down.

-Tomasz

Quoted 49 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +13 -1
  		gt->sriov.vf.migration.recovery_queued = true;
  		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
 
@@ -1280,6 +1281,17 @@ static void migration_worker_func(struct work_struct *w)
  	vf_post_migration_recovery(gt);
  }
 
+static void vf_migration_fini(struct drm_device *drm, void *arg)
+{
+	struct xe_gt *gt = arg;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_teardown = true;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	cancel_work_sync(&gt->sriov.vf.migration.worker);
+}
+
  /**
   * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
   * @gt: the &xe_gt
@@ -1308,7 +1320,8 @@ int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
  	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
  		xe_gt_sriov_info(gt, "migration not supported by this module version\n");
 
-	return 0;
+	return drmm_add_action_or_reset(&gt_to_xe(gt)->drm,
+					vf_migration_fini, gt);
  }
 
  /**
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -1
index 61484c7c9a36..beb9978336bb 100644
@@ -59,10 +59,12 @@ struct xe_gt_sriov_vf_runtime {
  struct xe_gt_sriov_vf_migration {
  	/** @migration: VF migration recovery worker */
  	struct work_struct worker;
-	/** @lock: Protects recovery_queued */
+	/** @lock: Protects recovery_queued, teardown */
  	spinlock_t lock;
  	/** @lrc_wa_bb: Scratch memory for LRC WA BB in recovery */
  	void *lrc_wa_bb;
+	/** @recovery_teardown: VF post migration recovery is being torn down */
+	bool recovery_teardown;
  	/** @recovery_queued: VF post migration recovery in queued */
  	bool recovery_queued;
  	/** @recovery_inprogress: VF post migration recovery in progress */
author maintainer
#46 lore raw

Re: [PATCH v2 16/34] drm/xe/vf: Teardown VF post migration worker on driver unload

Quoted 28 lines · Tomasz
Quoted 14 lines · Matthew Brost

Be cautious and ensure the VF post-migration worker is not running during driver unload.

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 17 +++++++++++++++--
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  4 +++-
  2 files changed, 18 insertions(+), 3 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index 807fdced0228..4eaffad6ebcf 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt) · line 813

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
@@ -811,7 +811,8 @@ static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)
  	spin_lock(&gt->sriov.vf.migration.lock);
-	if (!gt->sriov.vf.migration.recovery_queued) {
+	if (!gt->sriov.vf.migration.recovery_queued ||
+	    !gt->sriov.vf.migration.recovery_teardown) {

We're registering vf_migration_fini very early in the init. That means it will be called very late. With that in mind, is it even possible to hit?

During xe_gt_sriov_vf_migration_init_early, interrupts are not enabled yet. Doesn't that mean they are disabled already when vf_migration_fini is called?

Quoted 6 lines

Both ggtt_fini_early and guc_submit_fini should be finished by then; so if the recovery was running, it already crashed or errored out at this point.

So, maybe register the fini later? We need to init before IRQs are enabled, but for fini - it should be before exec_queue_lookup is torn down.

Yea, most this a bit later is probably a better idea. Let me look at that.

Matt

Quoted 48 lines

-Tomasz

Quoted 45 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +13 -1
  		gt->sriov.vf.migration.recovery_queued = true;
  		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
@@ -1280,6 +1281,17 @@ static void migration_worker_func(struct work_struct *w)
  	vf_post_migration_recovery(gt);
  }
+static void vf_migration_fini(struct drm_device *drm, void *arg)
+{
+	struct xe_gt *gt = arg;
+
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	gt->sriov.vf.migration.recovery_teardown = true;
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	cancel_work_sync(&gt->sriov.vf.migration.worker);
+}
+
  /**
   * xe_gt_sriov_vf_migration_init_early() - VF post migration init early
   * @gt: the &xe_gt
@@ -1308,7 +1320,8 @@ int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
  	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
  		xe_gt_sriov_info(gt, "migration not supported by this module version\n");
-	return 0;
+	return drmm_add_action_or_reset(&gt_to_xe(gt)->drm,
+					vf_migration_fini, gt);
  }
  /**
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +3 -1
index 61484c7c9a36..beb9978336bb 100644
@@ -59,10 +59,12 @@ struct xe_gt_sriov_vf_runtime {
  struct xe_gt_sriov_vf_migration {
  	/** @migration: VF migration recovery worker */
  	struct work_struct worker;
-	/** @lock: Protects recovery_queued */
+	/** @lock: Protects recovery_queued, teardown */
  	spinlock_t lock;
  	/** @lrc_wa_bb: Scratch memory for LRC WA BB in recovery */
  	void *lrc_wa_bb;
+	/** @recovery_teardown: VF post migration recovery is being torn down */
+	bool recovery_teardown;
  	/** @recovery_queued: VF post migration recovery in queued */
  	bool recovery_queued;
  	/** @recovery_inprogress: VF post migration recovery in progress */
author maintainer
#47 lore raw

PATCH v2 17/34 drm/xe/vf: Don't allow GT reset to be queued during VF post migration recovery

In reply to Matthew Brost

With well behaved software, this should never happen.

---
 drivers/gpu/drm/xe/xe_gt.c          |  9 -------
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  7 -----
 drivers/gpu/drm/xe/xe_guc_submit.c  | 41 +++--------------------------
 drivers/gpu/drm/xe/xe_guc_submit.h  |  3 ---
 4 files changed, 4 insertions(+), 56 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +0 -9
index 3e0ad7e5b5df..1a4796cd885b 100644
@@ -803,11 +803,6 @@ static int do_gt_restart(struct xe_gt *gt)
 	return 0;
 }
 
-static int gt_wait_reset_unblock(struct xe_gt *gt)
-{
-	return xe_guc_wait_reset_unblock(&gt->uc.guc);
-}
-
 static int gt_reset(struct xe_gt *gt)
 {
 	unsigned int fw_ref;
@@ -822,10 +817,6 @@ static int gt_reset(struct xe_gt *gt)
 
 	xe_gt_info(gt, "reset started\n");
 
-	err = gt_wait_reset_unblock(gt);
-	if (!err)
-		xe_gt_warn(gt, "reset block failed to get lifted");
-
 	xe_pm_runtime_get(gt_to_xe(gt));
 
 	if (xe_fault_inject_gt_reset()) {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -7
index 4eaffad6ebcf..b8e02bba7360 100644
@@ -1175,17 +1175,11 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 
 static void vf_post_migration_shutdown(struct xe_gt *gt)
 {
-	int ret = 0;
-
 	spin_lock_irq(&gt->sriov.vf.migration.lock);
 	gt->sriov.vf.migration.recovery_queued = false;
 	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
 	xe_guc_submit_pause(&gt->uc.guc);
-	ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-
-	if (ret)
-		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");
 }
 
 static size_t post_migration_scratch_size(struct xe_device *xe)
@@ -1219,7 +1213,6 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)
 	 */
 	xe_irq_resume(gt_to_xe(gt));
 
-	xe_guc_submit_reset_unblock(&gt->uc.guc);
 	xe_guc_submit_unpause(&gt->uc.guc);
 }
 
drivers/gpu/drm/xe/xe_guc_submit.c +4 -37

57 lines not on this page · raw message

drivers/gpu/drm/xe/xe_guc_submit.h +0 -3
index 5b4a0a6fd818..f535fe3895e5 100644
@@ -22,9 +22,6 @@ void xe_guc_submit_stop(struct xe_guc *guc);
 int xe_guc_submit_start(struct xe_guc *guc);
 void xe_guc_submit_pause(struct xe_guc *guc);
 void xe_guc_submit_unpause(struct xe_guc *guc);
-int xe_guc_submit_reset_block(struct xe_guc *guc);
-void xe_guc_submit_reset_unblock(struct xe_guc *guc);
-int xe_guc_wait_reset_unblock(struct xe_guc *guc);
 void xe_guc_submit_wedge(struct xe_guc *guc);
 
 int xe_guc_read_stopped(struct xe_guc *guc);
Signature · 1 line
2.34.1
#48 lore raw

Re: [PATCH v2 17/34] drm/xe/vf: Don't allow GT reset to be queued during VF post migration recovery

Quoted 1 line · Matthew Brost

With well behaved software, this should never happen.

Not a very reassuring description.

But the patch doesn't really lead to ignoring that situation as that sentence suggests; there is a bailout from reset if VF recovery is in progress.

This will work fine (for migration, it's actually better as we don't need any reset of the new HW), and adds way less code. I'd just improve the description.

-Tomasz

Quoted 142 lines
---
  drivers/gpu/drm/xe/xe_gt.c          |  9 -------
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  7 -----
  drivers/gpu/drm/xe/xe_guc_submit.c  | 41 +++--------------------------
  drivers/gpu/drm/xe/xe_guc_submit.h  |  3 ---
  4 files changed, 4 insertions(+), 56 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +0 -9
index 3e0ad7e5b5df..1a4796cd885b 100644
@@ -803,11 +803,6 @@ static int do_gt_restart(struct xe_gt *gt)
  	return 0;
  }
 
-static int gt_wait_reset_unblock(struct xe_gt *gt)
-{
-	return xe_guc_wait_reset_unblock(&gt->uc.guc);
-}
-
  static int gt_reset(struct xe_gt *gt)
  {
  	unsigned int fw_ref;
@@ -822,10 +817,6 @@ static int gt_reset(struct xe_gt *gt)
 
  	xe_gt_info(gt, "reset started\n");
 
-	err = gt_wait_reset_unblock(gt);
-	if (!err)
-		xe_gt_warn(gt, "reset block failed to get lifted");
-
  	xe_pm_runtime_get(gt_to_xe(gt));
 
  	if (xe_fault_inject_gt_reset()) {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -7
index 4eaffad6ebcf..b8e02bba7360 100644
@@ -1175,17 +1175,11 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
 
  static void vf_post_migration_shutdown(struct xe_gt *gt)
  {
-	int ret = 0;
-
  	spin_lock_irq(&gt->sriov.vf.migration.lock);
  	gt->sriov.vf.migration.recovery_queued = false;
  	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
  	xe_guc_submit_pause(&gt->uc.guc);
-	ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-
-	if (ret)
-		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");
  }
 
  static size_t post_migration_scratch_size(struct xe_device *xe)
@@ -1219,7 +1213,6 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)
  	 */
  	xe_irq_resume(gt_to_xe(gt));
 
-	xe_guc_submit_reset_unblock(&gt->uc.guc);
  	xe_guc_submit_unpause(&gt->uc.guc);
  }
 
drivers/gpu/drm/xe/xe_guc_submit.c +4 -37
index cd5e506527fe..b82976f031e5 100644
@@ -27,6 +27,7 @@
  #include "xe_gt.h"
  #include "xe_gt_clock.h"
  #include "xe_gt_printk.h"
+#include "xe_gt_sriov_vf.h"
  #include "xe_guc.h"
  #include "xe_guc_capture.h"
  #include "xe_guc_ct.h"
@@ -2182,47 +2183,13 @@ static void guc_exec_queue_stop(struct xe_guc *guc, struct xe_exec_queue *q)
  	}
  }
 
-/**
- * xe_guc_submit_reset_block - Disallow reset calls on given GuC.
- * @guc: the &xe_guc struct instance
- */
-int xe_guc_submit_reset_block(struct xe_guc *guc)
-{
-	return atomic_fetch_or(1, &guc->submission_state.reset_blocked);
-}
-
-/**
- * xe_guc_submit_reset_unblock - Allow back reset calls on given GuC.
- * @guc: the &xe_guc struct instance
- */
-void xe_guc_submit_reset_unblock(struct xe_guc *guc)
-{
-	atomic_set_release(&guc->submission_state.reset_blocked, 0);
-	wake_up_all(&guc->ct.wq);
-}
-
-static int guc_submit_reset_is_blocked(struct xe_guc *guc)
-{
-	return atomic_read_acquire(&guc->submission_state.reset_blocked);
-}
-
-/* Maximum time of blocking reset */
-#define RESET_BLOCK_PERIOD_MAX (HZ * 5)
-
-/**
- * xe_guc_wait_reset_unblock - Wait until reset blocking flag is lifted, or timeout.
- * @guc: the &xe_guc struct instance
- */
-int xe_guc_wait_reset_unblock(struct xe_guc *guc)
-{
-	return wait_event_timeout(guc->ct.wq,
-				  !guc_submit_reset_is_blocked(guc), RESET_BLOCK_PERIOD_MAX);
-}
-
  int xe_guc_submit_reset_prepare(struct xe_guc *guc)
  {
  	int ret;
 
+	if (WARN_ON_ONCE(xe_gt_sriov_vf_recovery_inprogress(guc_to_gt(guc))))
+		return 0;
+
  	if (!guc->submission_state.initialized)
  		return 0;
 
drivers/gpu/drm/xe/xe_guc_submit.h +0 -3
index 5b4a0a6fd818..f535fe3895e5 100644
@@ -22,9 +22,6 @@ void xe_guc_submit_stop(struct xe_guc *guc);
  int xe_guc_submit_start(struct xe_guc *guc);
  void xe_guc_submit_pause(struct xe_guc *guc);
  void xe_guc_submit_unpause(struct xe_guc *guc);
-int xe_guc_submit_reset_block(struct xe_guc *guc);
-void xe_guc_submit_reset_unblock(struct xe_guc *guc);
-int xe_guc_wait_reset_unblock(struct xe_guc *guc);
  void xe_guc_submit_wedge(struct xe_guc *guc);
 
  int xe_guc_read_stopped(struct xe_guc *guc);
author maintainer
#49 lore raw

Re: [PATCH v2 17/34] drm/xe/vf: Don't allow GT reset to be queued during VF post migration recovery

Quoted 6 lines · Tomasz
Quoted 1 line · Matthew Brost

With well behaved software, this should never happen.

Not a very reassuring description.

Quoted 8 lines

But the patch doesn't really lead to ignoring that situation as that sentence suggests; there is a bailout from reset if VF recovery is in progress.

This will work fine (for migration, it's actually better as we don't need any reset of the new HW), and adds way less code. I'd just improve the description.

Let me make the description a bit more verbose.

Matt

Quoted 131 lines

-Tomasz

Quoted 129 lines
---
  drivers/gpu/drm/xe/xe_gt.c          |  9 -------
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  7 -----
  drivers/gpu/drm/xe/xe_guc_submit.c  | 41 +++--------------------------
  drivers/gpu/drm/xe/xe_guc_submit.h  |  3 ---
  4 files changed, 4 insertions(+), 56 deletions(-)
drivers/gpu/drm/xe/xe_gt.c +0 -9
index 3e0ad7e5b5df..1a4796cd885b 100644
@@ -803,11 +803,6 @@ static int do_gt_restart(struct xe_gt *gt)
  	return 0;
  }
-static int gt_wait_reset_unblock(struct xe_gt *gt)
-{
-	return xe_guc_wait_reset_unblock(&gt->uc.guc);
-}
-
  static int gt_reset(struct xe_gt *gt)
  {
  	unsigned int fw_ref;
@@ -822,10 +817,6 @@ static int gt_reset(struct xe_gt *gt)
  	xe_gt_info(gt, "reset started\n");
-	err = gt_wait_reset_unblock(gt);
-	if (!err)
-		xe_gt_warn(gt, "reset block failed to get lifted");
-
  	xe_pm_runtime_get(gt_to_xe(gt));
  	if (xe_fault_inject_gt_reset()) {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -7
index 4eaffad6ebcf..b8e02bba7360 100644
@@ -1175,17 +1175,11 @@ void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p)
  static void vf_post_migration_shutdown(struct xe_gt *gt)
  {
-	int ret = 0;
-
  	spin_lock_irq(&gt->sriov.vf.migration.lock);
  	gt->sriov.vf.migration.recovery_queued = false;
  	spin_unlock_irq(&gt->sriov.vf.migration.lock);
  	xe_guc_submit_pause(&gt->uc.guc);
-	ret |= xe_guc_submit_reset_block(&gt->uc.guc);
-
-	if (ret)
-		xe_gt_sriov_info(gt, "migration recovery encountered ongoing reset\n");
  }
  static size_t post_migration_scratch_size(struct xe_device *xe)
@@ -1219,7 +1213,6 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)
  	 */
  	xe_irq_resume(gt_to_xe(gt));
-	xe_guc_submit_reset_unblock(&gt->uc.guc);
  	xe_guc_submit_unpause(&gt->uc.guc);
  }
drivers/gpu/drm/xe/xe_guc_submit.c +4 -37
index cd5e506527fe..b82976f031e5 100644
@@ -27,6 +27,7 @@
  #include "xe_gt.h"
  #include "xe_gt_clock.h"
  #include "xe_gt_printk.h"
+#include "xe_gt_sriov_vf.h"
  #include "xe_guc.h"
  #include "xe_guc_capture.h"
  #include "xe_guc_ct.h"
@@ -2182,47 +2183,13 @@ static void guc_exec_queue_stop(struct xe_guc *guc, struct xe_exec_queue *q)
  	}
  }
-/**
- * xe_guc_submit_reset_block - Disallow reset calls on given GuC.
- * @guc: the &xe_guc struct instance
- */
-int xe_guc_submit_reset_block(struct xe_guc *guc)
-{
-	return atomic_fetch_or(1, &guc->submission_state.reset_blocked);
-}
-
-/**
- * xe_guc_submit_reset_unblock - Allow back reset calls on given GuC.
- * @guc: the &xe_guc struct instance
- */
-void xe_guc_submit_reset_unblock(struct xe_guc *guc)
-{
-	atomic_set_release(&guc->submission_state.reset_blocked, 0);
-	wake_up_all(&guc->ct.wq);
-}
-
-static int guc_submit_reset_is_blocked(struct xe_guc *guc)
-{
-	return atomic_read_acquire(&guc->submission_state.reset_blocked);
-}
-
-/* Maximum time of blocking reset */
-#define RESET_BLOCK_PERIOD_MAX (HZ * 5)
-
-/**
- * xe_guc_wait_reset_unblock - Wait until reset blocking flag is lifted, or timeout.
- * @guc: the &xe_guc struct instance
- */
-int xe_guc_wait_reset_unblock(struct xe_guc *guc)
-{
-	return wait_event_timeout(guc->ct.wq,
-				  !guc_submit_reset_is_blocked(guc), RESET_BLOCK_PERIOD_MAX);
-}
-
  int xe_guc_submit_reset_prepare(struct xe_guc *guc)
  {
  	int ret;
+	if (WARN_ON_ONCE(xe_gt_sriov_vf_recovery_inprogress(guc_to_gt(guc))))
+		return 0;
+
  	if (!guc->submission_state.initialized)
  		return 0;
drivers/gpu/drm/xe/xe_guc_submit.h +0 -3
index 5b4a0a6fd818..f535fe3895e5 100644
@@ -22,9 +22,6 @@ void xe_guc_submit_stop(struct xe_guc *guc);
  int xe_guc_submit_start(struct xe_guc *guc);
  void xe_guc_submit_pause(struct xe_guc *guc);
  void xe_guc_submit_unpause(struct xe_guc *guc);
-int xe_guc_submit_reset_block(struct xe_guc *guc);
-void xe_guc_submit_reset_unblock(struct xe_guc *guc);
-int xe_guc_wait_reset_unblock(struct xe_guc *guc);
  void xe_guc_submit_wedge(struct xe_guc *guc);
  int xe_guc_read_stopped(struct xe_guc *guc);
author maintainer
#50 lore raw

PATCH v2 18/34 drm/xe/vf: Wakeup in GuC backend on VF post migration recovery

In reply to Matthew Brost

If VF post-migration recovery is in progress, the recovery flow will rebuild all GuC submission state. In this case, exit all waiters to ensure that submission queue scheduling can also be paused. Avoid taking any adverse actions after aborting the wait.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  3 ++
 drivers/gpu/drm/xe/xe_guc_submit.c  | 51 +++++++++++++++++++++--------
 2 files changed, 41 insertions(+), 13 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
index b8e02bba7360..a96e0dd65bc1 100644
@@ -815,6 +815,9 @@ static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)
 	    !gt->sriov.vf.migration.recovery_teardown) {
 		gt->sriov.vf.migration.recovery_queued = true;
 		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+		smp_wmb();	/* Ensure above write visable before wake */
+
+		wake_up_all(&gt->uc.guc.ct.wq);
 
 		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
 		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
drivers/gpu/drm/xe/xe_guc_submit.c +38 -13

142 lines not on this page · raw message

Signature · 1 line
2.34.1
author maintainer
#51 lore raw

Re: [PATCH v2 18/34] drm/xe/vf: Wakeup in GuC backend on VF post migration recovery

Quoted 163 lines · Matthew Brost

If VF post-migration recovery is in progress, the recovery flow will rebuild all GuC submission state. In this case, exit all waiters to ensure that submission queue scheduling can also be paused. Avoid taking any adverse actions after aborting the wait.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  3 ++
 drivers/gpu/drm/xe/xe_guc_submit.c  | 51 +++++++++++++++++++++--------
 2 files changed, 41 insertions(+), 13 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
index b8e02bba7360..a96e0dd65bc1 100644
@@ -815,6 +815,9 @@ static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)
 	    !gt->sriov.vf.migration.recovery_teardown) {
 		gt->sriov.vf.migration.recovery_queued = true;
 		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
+		smp_wmb();	/* Ensure above write visable before wake */
+
+		wake_up_all(&gt->uc.guc.ct.wq);
 
 		started = queue_work(gt->ordered_wq, &gt->sriov.vf.migration.worker);
 		xe_gt_sriov_info(gt, "VF migration recovery %s\n", started ?
drivers/gpu/drm/xe/xe_guc_submit.c +34 -12
index b82976f031e5..52b86cab4ec5 100644
@@ -984,6 +984,9 @@ static u32 wq_space_until_wrap(struct xe_exec_queue *q)
 	return (WQ_SIZE - q->guc->wqi_tail);
 }
 
+#define vf_recovery(guc)	\
+	xe_gt_sriov_vf_recovery_inprogress(guc_to_gt(guc))
+
 static int wq_wait_for_space(struct xe_exec_queue *q, u32 wqi_size)
 {
 	struct xe_guc *guc = exec_queue_to_guc(q);
@@ -993,7 +996,7 @@ static int wq_wait_for_space(struct xe_exec_queue *q, u32 wqi_size)
 
 #define AVAILABLE_SPACE \
 	CIRC_SPACE(q->guc->wqi_tail, q->guc->wqi_head, WQ_SIZE)
-	if (wqi_size > AVAILABLE_SPACE) {
+	if (wqi_size > AVAILABLE_SPACE && !vf_recovery(guc)) {
 try_again:
 		q->guc->wqi_head = parallel_read(xe, map, wq_desc.head);
 		if (wqi_size > AVAILABLE_SPACE) {
@@ -1192,9 +1195,10 @@ static void disable_scheduling_deregister(struct xe_guc *guc,
 	ret = wait_event_timeout(guc->ct.wq,
 				 (!exec_queue_pending_enable(q) &&
 				  !exec_queue_pending_disable(q)) ||
-					 xe_guc_read_stopped(guc),
+					 xe_guc_read_stopped(guc) ||
+					 vf_recovery(guc),
 				 HZ * 5);
-	if (!ret) {
+	if (!ret && !vf_recovery(guc)) {
 		struct xe_gpu_scheduler *sched = &q->guc->sched;
 
 		xe_gt_warn(q->gt, "Pending enable/disable failed to respond\n");
@@ -1297,6 +1301,10 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)
 	bool wedged = false;
 
 	xe_gt_assert(guc_to_gt(guc), xe_exec_queue_is_lr(q));
+
+	if (vf_recovery(guc))
+		return;
+
 	trace_xe_exec_queue_lr_cleanup(q);
 
 	if (!exec_queue_killed(q))
@@ -1329,7 +1337,11 @@ static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w)
 		 */
 		ret = wait_event_timeout(guc->ct.wq,
 					 !exec_queue_pending_disable(q) ||
-					 xe_guc_read_stopped(guc), HZ * 5);
+					 xe_guc_read_stopped(guc) ||
+					 vf_recovery(guc), HZ * 5);
+		if (vf_recovery(guc))
+			return;
+
 		if (!ret) {
 			xe_gt_warn(q->gt, "Schedule disable failed to respond, guc_id=%d\n",
 				   q->guc->id);
@@ -1419,8 +1431,9 @@ static void enable_scheduling(struct xe_exec_queue *q)
 
 	ret = wait_event_timeout(guc->ct.wq,
 				 !exec_queue_pending_enable(q) ||
-				 xe_guc_read_stopped(guc), HZ * 5);
-	if (!ret || xe_guc_read_stopped(guc)) {
+				 xe_guc_read_stopped(guc) ||
+				 vf_recovery(guc), HZ * 5);
+	if ((!ret && !vf_recovery(guc)) || xe_guc_read_stopped(guc)) {
 		xe_gt_warn(guc_to_gt(guc), "Schedule enable failed to respond");
 		set_exec_queue_banned(q);
 		xe_gt_reset_async(q->gt);
@@ -1491,7 +1504,8 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)
 	 * list so job can be freed and kick scheduler ensuring free job is not
 	 * lost.
 	 */
-	if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &job->fence->flags))
+	if (test_bit(DMA_FENCE_FLAG_SIGNALED_BIT, &job->fence->flags) ||
+	    vf_recovery(guc))
 		return DRM_GPU_SCHED_STAT_NO_HANG;
 
 	/* Kill the run_job entry point */
@@ -1543,7 +1557,10 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)
 			ret = wait_event_timeout(guc->ct.wq,
 						 (!exec_queue_pending_enable(q) &&
 						  !exec_queue_pending_disable(q)) ||
-						 xe_guc_read_stopped(guc), HZ * 5);
+						 xe_guc_read_stopped(guc) ||
+						 vf_recovery(guc), HZ * 5);
+			if (vf_recovery(guc))
+				goto handle_vf_resume;
 			if (!ret || xe_guc_read_stopped(guc))
 				goto trigger_reset;
 
@@ -1568,7 +1585,10 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)
 		smp_rmb();
 		ret = wait_event_timeout(guc->ct.wq,
 					 !exec_queue_pending_disable(q) ||
-					 xe_guc_read_stopped(guc), HZ * 5);
+					 xe_guc_read_stopped(guc) ||
+					 vf_recovery(guc), HZ * 5);
+		if (vf_recovery(guc))
+			goto handle_vf_resume;
 		if (!ret || xe_guc_read_stopped(guc)) {
 trigger_reset:
 			if (!ret)
@@ -1673,6 +1693,7 @@ guc_exec_queue_timedout_job(struct drm_sched_job *drm_job)
 	 * some thought, do this in a follow up.
 	 */
 	xe_sched_submission_start(sched);
+handle_vf_resume:
 	return DRM_GPU_SCHED_STAT_NO_HANG;
 }
 
@@ -1794,8 +1815,9 @@ static void __guc_exec_queue_process_msg_suspend(struct xe_sched_msg *msg)
 
 	if (guc_exec_queue_allowed_to_change_state(q) && !exec_queue_suspended(q) &&
 	    exec_queue_enabled(q)) {
-		wait_event(guc->ct.wq, (q->guc->resume_time != RESUME_PENDING ||
-			   xe_guc_read_stopped(guc)) && !exec_queue_pending_disable(q));
+		wait_event(guc->ct.wq, vf_recovery(guc) ||
+			   ((q->guc->resume_time != RESUME_PENDING ||
+			   xe_guc_read_stopped(guc)) && !exec_queue_pending_disable(q)));
 
 		if (!xe_guc_read_stopped(guc)) {
 			s64 since_resume_ms =
@@ -1922,7 +1944,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q)
 
 	q->entity = &ge->entity;
 
-	if (xe_guc_read_stopped(guc))
+	if (xe_guc_read_stopped(guc) || vf_recovery(guc))
 		xe_sched_stop(sched);
 
 	mutex_unlock(&guc->submission_state.lock);
@@ -2075,12 +2097,15 @@ static int guc_exec_queue_suspend_wait(struct xe_exec_queue *q)
 	 * suspend_pending upon kill but to be paranoid but races in which
 	 * suspend_pending is set after kill also check kill here.

drivers/gpu/drm/xe/xe_guc_submit.c static int guc_exec_queue_suspend_wait(struct xe_exec_queue *q) · line 2108

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +3 -0
 	 */
+retry:
 	ret = wait_event_interruptible_timeout(q->guc->suspend_wait,
 					       !READ_ONCE(q->guc->suspend_pending) ||
 					       exec_queue_killed(q) ||
 					       xe_guc_read_stopped(guc),
 					       HZ * 5);
 
+	if (vf_recovery(guc) && !xe_device_wedged((guc_to_xe(guc))))
+		goto retry;

This retry loop can lead to a deadlock on VF post-migration recovery. It appears this work queue / item not finishing during VF post-migration recovery can interfere with VF post-migration recovery worker getting scheduled. I have solution to return -EAGAIN if vf_recovery is true which the caller catches and reschedules the preempt fence work item. This appears to break the potential deadlock esnuring the VF post-migration recovery work item gets scheduled.

Matt

Quoted 15 lines
drivers/gpu/drm/xe/xe_guc_submit.c +1 -1
 	if (!ret) {
 		xe_gt_warn(guc_to_gt(guc),
 			   "Suspend fence, guc_id=%d, failed to respond",
@@ -2187,7 +2212,7 @@ int xe_guc_submit_reset_prepare(struct xe_guc *guc)
 {
 	int ret;
 
-	if (WARN_ON_ONCE(xe_gt_sriov_vf_recovery_inprogress(guc_to_gt(guc))))
+	if (WARN_ON_ONCE(vf_recovery(guc)))
 		return 0;
 
 	if (!guc->submission_state.initialized)
Signature · 1 line
2.34.1
author maintainer
#52 lore raw

PATCH v2 19/34 drm/xe/vf: Extra debug on GGTT shift

In reply to Matthew Brost

A little extra debug on GGTT shift printing the GGT shift value is helpful for VF post-migration recovery.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -1
index a96e0dd65bc1..e79ab4a2a273 100644
@@ -475,8 +475,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery)
 		primary_config->ggtt_base = start;
 	err = config->ggtt_size ? 0 : -ENODATA;
 
-	if (!err && shift && recovery)
+	if (!err && shift && recovery) {
+		xe_gt_sriov_info(gt, "Shifting GGTT %lld -> 0x%016llx\n",
+				 shift, config->ggtt_base);
 		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+	}
 out:
 	up_write(config->lock);
 	return err;
Signature · 1 line
2.34.1
Reviewed-by on v2 19/34
#53 lore raw

Re: [PATCH v2 19/34] drm/xe/vf: Extra debug on GGTT shift

Quoted 2 lines · Matthew Brost

A little extra debug on GGTT shift printing the GGT shift value is helpful for VF post-migration recovery.

Right, good. It is possible to compute the shift from "GGTT" message printed earlier and the same during first probe, but this has higher log level and also allows to confirm that node fixups were entered.

-Tomasz

Quoted 22 lines
---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 5 ++++-
  1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -1
index a96e0dd65bc1..e79ab4a2a273 100644
@@ -475,8 +475,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery)
  		primary_config->ggtt_base = start;
  	err = config->ggtt_size ? 0 : -ENODATA;
 
-	if (!err && shift && recovery)
+	if (!err && shift && recovery) {
+		xe_gt_sriov_info(gt, "Shifting GGTT %lld -> 0x%016llx\n",
+				 shift, config->ggtt_base);
  		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
+	}
  out:
  	up_write(config->lock);
  	return err;
#54 lore raw

Re: [PATCH v2 19/34] drm/xe/vf: Extra debug on GGTT shift

In reply to Matthew Brost
Quoted 1 line · Matthew Brost

A little extra debug on GGTT shift printing the GGT shift value is

s/GGT/GGTT

and maybe consider some rewording as I'm not sure it's right

Quoted 12 lines

helpful for VF post-migration recovery.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index a96e0dd65bc1..e79ab4a2a273 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery) · line 480

Quoted 7 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -1
@@ -475,8 +475,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery)
 		primary_config->ggtt_base = start;
 	err = config->ggtt_size ? 0 : -ENODATA;
 
-	if (!err && shift && recovery)
+	if (!err && shift && recovery) {
+		xe_gt_sriov_info(gt, "Shifting GGTT %lld -> 0x%016llx\n",
+				 shift, config->ggtt_base);

since this is "info" level message, maybe we can make it as:

		"Shifting GGTT base by %lld to 0x%016llx\n"

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery) · line 481

Quoted 1 line
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
 		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);

and print this from within this tile based function, where we could use tile-based printk function (instead of currently used GT-based)

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+	}
 out:
 	up_write(config->lock);
 	return err;
author maintainer
#55 lore raw

Re: [PATCH v2 19/34] drm/xe/vf: Extra debug on GGTT shift

Quoted 7 lines · Michal Wajdeczko
Quoted 1 line · Matthew Brost

A little extra debug on GGTT shift printing the GGT shift value is

s/GGT/GGTT

Yes.

Quoted 2 lines

and maybe consider some rewording as I'm not sure it's right

Let me reword this.

Quoted 15 lines
Quoted 15 lines

helpful for VF post-migration recovery.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index a96e0dd65bc1..e79ab4a2a273 100644
@@ -475,8 +475,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery)
 		primary_config->ggtt_base = start;
 	err = config->ggtt_size ? 0 : -ENODATA;
 
Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery) · line 480

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -1
-	if (!err && shift && recovery)
+	if (!err && shift && recovery) {
+		xe_gt_sriov_info(gt, "Shifting GGTT %lld -> 0x%016llx\n",
+				 shift, config->ggtt_base);

since this is "info" level message, maybe we can make it as:

		"Shifting GGTT base by %lld to 0x%016llx\n"

Sure.

Quoted 5 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery) · line 481

Quoted 1 line
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
 		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);

and print this from within this tile based function, where we could use tile-based printk function (instead of currently used GT-based)

Not sure about this one. It is helpful to see which GT is performing the shift which manually can be used to verify either GT doing the shift.

Matt

Quoted 5 lines
Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+	}
 out:
 	up_write(config->lock);
 	return err;
author maintainer
#56 lore raw

PATCH v2 20/34 drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register

In reply to Matthew Brost

The only case where the GuC submission backend cannot reason 100% correctly is when a GuC context is registered during VF post-migration recovery. In this scenario, it's possible that the GuC context register H2G is processed, but the immediately following schedule-enable H2G gets lost.

A double register is harmless when using GUC_HXG_TYPE_EVENT, as GuC simply drops the duplicate H2G. To keep things simple, use GUC_HXG_TYPE_EVENT for all context registrations on VFs.

---
 drivers/gpu/drm/xe/xe_guc_ct.c | 32 ++++++++++++++++++++++++--------
 1 file changed, 24 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +24 -8
index 661ab1bf4502..aa9fb08f7856 100644
@@ -737,6 +737,26 @@ static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence)
 	return seqno;
 }
 
+#define MAKE_ACTION(type, __action)				\
+({								\
+	FIELD_PREP(GUC_HXG_MSG_0_TYPE, type) |			\
+	FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |			\
+		   GUC_HXG_EVENT_MSG_0_DATA0, __action);	\
+})
+
+static bool vf_action_can_safely_fail(struct xe_device *xe, u32 action)
+{
+	/*
+	 * If we are VF resuming, we can't exactly track if a context
+	 * registration has been completed in the GuC state machine, it is
+	 * harmless to resend as it will just fail silently if
+	 * GUC_HXG_TYPE_EVENT is used.
+	 */
+	return IS_SRIOV_VF(xe) &&
+		(action == XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC ||
+		 action == XE_GUC_ACTION_REGISTER_CONTEXT);
+}
+
 #define H2G_CT_HEADERS (GUC_CTB_HDR_LEN + 1) /* one DW CTB header and one DW HxG header */
 
 static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
@@ -808,18 +828,14 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
 		FIELD_PREP(GUC_CTB_MSG_0_NUM_DWORDS, len) |
 		FIELD_PREP(GUC_CTB_MSG_0_FENCE, ct_fence_value);
 	if (want_response) {
-		cmd[1] =
-			FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) |
-			FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |
-				   GUC_HXG_EVENT_MSG_0_DATA0, action[0]);
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_REQUEST, action[0]);
+	} else if (vf_action_can_safely_fail(xe, action[0])) {
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_EVENT, action[0]);
 	} else {
 		fast_req_track(ct, ct_fence_value,
 			       FIELD_GET(GUC_HXG_EVENT_MSG_0_ACTION, action[0]));
 
-		cmd[1] =
-			FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_FAST_REQUEST) |
-			FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |
-				   GUC_HXG_EVENT_MSG_0_DATA0, action[0]);
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_FAST_REQUEST, action[0]);
 	}
 
 	/* H2G header in cmd[1] replaces action[0] so: */
Signature · 1 line
2.34.1
#57 lore raw

Re: [PATCH v2 20/34] drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register

Quoted 5 lines · Matthew Brost

The only case where the GuC submission backend cannot reason 100% correctly is when a GuC context is registered during VF post-migration recovery. In this scenario, it's possible that the GuC context register H2G is processed, but the immediately following schedule-enable H2G gets lost.

what do you mean by "lost"?

if it was not processed by the SRC GuC, then it is still in CTB and shall be processed by DST GuC, unless we strip it from the CTB by ourselves, but in this case we should notice that, no?

Quoted 4 lines

A double register is harmless when using GUC_HXG_TYPE_EVENT, as GuC simply drops the duplicate H2G. To keep things simple, use GUC_HXG_TYPE_EVENT for all context registrations on VFs.

with this we will miss any early errors from the GuC on non-migration cases while unlikely, IMO we shouldn't just hide potential errors

Quoted 60 lines
---
 drivers/gpu/drm/xe/xe_guc_ct.c | 32 ++++++++++++++++++++++++--------
 1 file changed, 24 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +24 -8
index 661ab1bf4502..aa9fb08f7856 100644
@@ -737,6 +737,26 @@ static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence)
 	return seqno;
 }
 
+#define MAKE_ACTION(type, __action)				\
+({								\
+	FIELD_PREP(GUC_HXG_MSG_0_TYPE, type) |			\
+	FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |			\
+		   GUC_HXG_EVENT_MSG_0_DATA0, __action);	\
+})
+
+static bool vf_action_can_safely_fail(struct xe_device *xe, u32 action)
+{
+	/*
+	 * If we are VF resuming, we can't exactly track if a context
+	 * registration has been completed in the GuC state machine, it is
+	 * harmless to resend as it will just fail silently if
+	 * GUC_HXG_TYPE_EVENT is used.
+	 */
+	return IS_SRIOV_VF(xe) &&
+		(action == XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC ||
+		 action == XE_GUC_ACTION_REGISTER_CONTEXT);
+}
+
 #define H2G_CT_HEADERS (GUC_CTB_HDR_LEN + 1) /* one DW CTB header and one DW HxG header */
 
 static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
@@ -808,18 +828,14 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
 		FIELD_PREP(GUC_CTB_MSG_0_NUM_DWORDS, len) |
 		FIELD_PREP(GUC_CTB_MSG_0_FENCE, ct_fence_value);
 	if (want_response) {
-		cmd[1] =
-			FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) |
-			FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |
-				   GUC_HXG_EVENT_MSG_0_DATA0, action[0]);
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_REQUEST, action[0]);
+	} else if (vf_action_can_safely_fail(xe, action[0])) {
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_EVENT, action[0]);
 	} else {
 		fast_req_track(ct, ct_fence_value,
 			       FIELD_GET(GUC_HXG_EVENT_MSG_0_ACTION, action[0]));
 
-		cmd[1] =
-			FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_FAST_REQUEST) |
-			FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |
-				   GUC_HXG_EVENT_MSG_0_DATA0, action[0]);
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_FAST_REQUEST, action[0]);
 	}
 
 	/* H2G header in cmd[1] replaces action[0] so: */
author maintainer
#58 lore raw

Re: [PATCH v2 20/34] drm/xe/vf: Use GUC_HXG_TYPE_EVENT for GuC context register

Quoted 9 lines · Michal Wajdeczko
Quoted 5 lines · Matthew Brost

The only case where the GuC submission backend cannot reason 100% correctly is when a GuC context is registered during VF post-migration recovery. In this scenario, it's possible that the GuC context register H2G is processed, but the immediately following schedule-enable H2G gets lost.

Quoted 6 lines

what do you mean by "lost"?

if it was not processed by the SRC GuC, then it is still in CTB and shall be processed by DST GuC, unless we strip it from the CTB by ourselves, but in this case we should notice that, no?

We do strip out the CTBs and GuC submission state machine will reply register H2G.

Quoted 8 lines
Quoted 4 lines

A double register is harmless when using GUC_HXG_TYPE_EVENT, as GuC simply drops the duplicate H2G. To keep things simple, use GUC_HXG_TYPE_EVENT for all context registrations on VFs.

with this we will miss any early errors from the GuC on non-migration cases while unlikely, IMO we shouldn't just hide potential errors

I agree this isn't ideal but this check is present for non-VFs. The registration code is generic so if we had a protocol problem our CI would quickly catch this on non-VFs. A fast request failure is fatal to the KMD as we take down th CT and issue GT reset when this happens.

IMO we live with this limitation.

Matt

Quoted 61 lines
Quoted 60 lines
---
 drivers/gpu/drm/xe/xe_guc_ct.c | 32 ++++++++++++++++++++++++--------
 1 file changed, 24 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_guc_ct.c +24 -8
index 661ab1bf4502..aa9fb08f7856 100644
@@ -737,6 +737,26 @@ static u16 next_ct_seqno(struct xe_guc_ct *ct, bool is_g2h_fence)
 	return seqno;
 }
 
+#define MAKE_ACTION(type, __action)				\
+({								\
+	FIELD_PREP(GUC_HXG_MSG_0_TYPE, type) |			\
+	FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |			\
+		   GUC_HXG_EVENT_MSG_0_DATA0, __action);	\
+})
+
+static bool vf_action_can_safely_fail(struct xe_device *xe, u32 action)
+{
+	/*
+	 * If we are VF resuming, we can't exactly track if a context
+	 * registration has been completed in the GuC state machine, it is
+	 * harmless to resend as it will just fail silently if
+	 * GUC_HXG_TYPE_EVENT is used.
+	 */
+	return IS_SRIOV_VF(xe) &&
+		(action == XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC ||
+		 action == XE_GUC_ACTION_REGISTER_CONTEXT);
+}
+
 #define H2G_CT_HEADERS (GUC_CTB_HDR_LEN + 1) /* one DW CTB header and one DW HxG header */
 
 static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
@@ -808,18 +828,14 @@ static int h2g_write(struct xe_guc_ct *ct, const u32 *action, u32 len,
 		FIELD_PREP(GUC_CTB_MSG_0_NUM_DWORDS, len) |
 		FIELD_PREP(GUC_CTB_MSG_0_FENCE, ct_fence_value);
 	if (want_response) {
-		cmd[1] =
-			FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_REQUEST) |
-			FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |
-				   GUC_HXG_EVENT_MSG_0_DATA0, action[0]);
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_REQUEST, action[0]);
+	} else if (vf_action_can_safely_fail(xe, action[0])) {
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_EVENT, action[0]);
 	} else {
 		fast_req_track(ct, ct_fence_value,
 			       FIELD_GET(GUC_HXG_EVENT_MSG_0_ACTION, action[0]));
 
-		cmd[1] =
-			FIELD_PREP(GUC_HXG_MSG_0_TYPE, GUC_HXG_TYPE_FAST_REQUEST) |
-			FIELD_PREP(GUC_HXG_EVENT_MSG_0_ACTION |
-				   GUC_HXG_EVENT_MSG_0_DATA0, action[0]);
+		cmd[1] = MAKE_ACTION(GUC_HXG_TYPE_FAST_REQUEST, action[0]);
 	}
 
 	/* H2G header in cmd[1] replaces action[0] so: */
author maintainer
#59 lore raw

PATCH v2 21/34 drm/xe/vf: Stop and flush CTs in VF post migration recovery

In reply to Matthew Brost

Flushing CTs (i.e., progressing all pending G2H messages) gives VF post-migration recovery an accurate view of which H2G messages the GuC has processed, enabling the GuC submission state machine to correctly rebuild all state.

Also, stop all CT traffic, as the CT is not live during VF post-migration recovery.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  2 ++
 drivers/gpu/drm/xe/xe_guc_ct.c      | 11 +++++++++++
 drivers/gpu/drm/xe/xe_guc_ct.h      |  1 +
 3 files changed, 14 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
index e79ab4a2a273..071c2c5e0d0a 100644
@@ -23,6 +23,7 @@
 #include "xe_gt_sriov_vf.h"
 #include "xe_gt_sriov_vf_types.h"
 #include "xe_guc.h"
+#include "xe_guc_ct.h"
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
 #include "xe_guc_submit.h"
@@ -1185,6 +1186,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)
 	gt->sriov.vf.migration.recovery_queued = false;
 	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
+	xe_guc_ct_flush(&gt->uc.guc.ct);
 	xe_guc_submit_pause(&gt->uc.guc);
 }
 
drivers/gpu/drm/xe/xe_guc_ct.c +11 -0
index aa9fb08f7856..d4a132b81b86 100644
@@ -574,6 +574,17 @@ void xe_guc_ct_disable(struct xe_guc_ct *ct)
 	stop_g2h_handler(ct);
 }
 
+/**
+ * xe_guc_ct_flush - Flush all processing of G2H / H2G
+ * @ct: the &xe_guc_ct
+ */
+void xe_guc_ct_flush(struct xe_guc_ct *ct)
+{
+	receive_g2h(ct);
+	wake_up_all(&ct->g2h_fence_wq);
+	xe_guc_ct_stop(ct);
+}
+
 /**
  * xe_guc_ct_stop - Set GuC to stopped state
  * @ct: the &xe_guc_ct
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
index d6c81325a76c..1e32a59817cc 100644
@@ -17,6 +17,7 @@ int xe_guc_ct_init_post_hwconfig(struct xe_guc_ct *ct);
 int xe_guc_ct_enable(struct xe_guc_ct *ct);
 void xe_guc_ct_disable(struct xe_guc_ct *ct);
 void xe_guc_ct_stop(struct xe_guc_ct *ct);
+void xe_guc_ct_flush(struct xe_guc_ct *ct);
 void xe_guc_ct_fast_path(struct xe_guc_ct *ct);
 
 struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct);
Signature · 1 line
2.34.1
#60 lore raw

Re: [PATCH v2 21/34] drm/xe/vf: Stop and flush CTs in VF post migration recovery

Quoted 7 lines · Matthew Brost

Flushing CTs (i.e., progressing all pending G2H messages) gives VF post-migration recovery an accurate view of which H2G messages the GuC has processed, enabling the GuC submission state machine to correctly rebuild all state.

Also, stop all CT traffic, as the CT is not live during VF post-migration recovery.

this is what I was looking for, some comments below, but LGTM

Quoted 33 lines
---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  2 ++
 drivers/gpu/drm/xe/xe_guc_ct.c      | 11 +++++++++++
 drivers/gpu/drm/xe/xe_guc_ct.h      |  1 +
 3 files changed, 14 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
index e79ab4a2a273..071c2c5e0d0a 100644
@@ -23,6 +23,7 @@
 #include "xe_gt_sriov_vf.h"
 #include "xe_gt_sriov_vf_types.h"
 #include "xe_guc.h"
+#include "xe_guc_ct.h"
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
 #include "xe_guc_submit.h"
@@ -1185,6 +1186,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)
 	gt->sriov.vf.migration.recovery_queued = false;
 	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
+	xe_guc_ct_flush(&gt->uc.guc.ct);
 	xe_guc_submit_pause(&gt->uc.guc);
 }
 
drivers/gpu/drm/xe/xe_guc_ct.c +0 -0
index aa9fb08f7856..d4a132b81b86 100644

drivers/gpu/drm/xe/xe_guc_ct.c void xe_guc_ct_disable(struct xe_guc_ct *ct) · line 578

Quoted 5 lines
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
@@ -574,6 +574,17 @@ void xe_guc_ct_disable(struct xe_guc_ct *ct)
 	stop_g2h_handler(ct);
 }
 
+/**
+ * xe_guc_ct_flush - Flush all processing of G2H / H2G

Flush and stop ?

drivers/gpu/drm/xe/xe_guc_ct.c void xe_guc_ct_disable(struct xe_guc_ct *ct) · line 582

Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_ct.c +4 -0
+ * @ct: the &xe_guc_ct
+ */
+void xe_guc_ct_flush(struct xe_guc_ct *ct)
+{

do we care about ct being not initialized yet?

drivers/gpu/drm/xe/xe_guc_ct.c void xe_guc_ct_disable(struct xe_guc_ct *ct) · line 584

Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
+	receive_g2h(ct);
+	wake_up_all(&ct->g2h_fence_wq);

this is part of the guc_ct_change_state() called by below stop isn't that sufficient ?

Quoted 18 lines
drivers/gpu/drm/xe/xe_guc_ct.c +3 -0
+	xe_guc_ct_stop(ct);
+}
+
 /**
  * xe_guc_ct_stop - Set GuC to stopped state
  * @ct: the &xe_guc_ct
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
index d6c81325a76c..1e32a59817cc 100644
@@ -17,6 +17,7 @@ int xe_guc_ct_init_post_hwconfig(struct xe_guc_ct *ct);
 int xe_guc_ct_enable(struct xe_guc_ct *ct);
 void xe_guc_ct_disable(struct xe_guc_ct *ct);
 void xe_guc_ct_stop(struct xe_guc_ct *ct);
+void xe_guc_ct_flush(struct xe_guc_ct *ct);
 void xe_guc_ct_fast_path(struct xe_guc_ct *ct);
 
 struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct);
author maintainer
#61 lore raw

Re: [PATCH v2 21/34] drm/xe/vf: Stop and flush CTs in VF post migration recovery

Quoted 48 lines · Michal Wajdeczko
Quoted 7 lines · Matthew Brost

Flushing CTs (i.e., progressing all pending G2H messages) gives VF post-migration recovery an accurate view of which H2G messages the GuC has processed, enabling the GuC submission state machine to correctly rebuild all state.

Also, stop all CT traffic, as the CT is not live during VF post-migration recovery.

this is what I was looking for, some comments below, but LGTM

Quoted 36 lines
---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  2 ++
 drivers/gpu/drm/xe/xe_guc_ct.c      | 11 +++++++++++
 drivers/gpu/drm/xe/xe_guc_ct.h      |  1 +
 3 files changed, 14 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
index e79ab4a2a273..071c2c5e0d0a 100644
@@ -23,6 +23,7 @@
 #include "xe_gt_sriov_vf.h"
 #include "xe_gt_sriov_vf_types.h"
 #include "xe_guc.h"
+#include "xe_guc_ct.h"
 #include "xe_guc_hxg_helpers.h"
 #include "xe_guc_relay.h"
 #include "xe_guc_submit.h"
@@ -1185,6 +1186,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)
 	gt->sriov.vf.migration.recovery_queued = false;
 	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
+	xe_guc_ct_flush(&gt->uc.guc.ct);
 	xe_guc_submit_pause(&gt->uc.guc);
 }
 
drivers/gpu/drm/xe/xe_guc_ct.c +0 -0
index aa9fb08f7856..d4a132b81b86 100644
@@ -574,6 +574,17 @@ void xe_guc_ct_disable(struct xe_guc_ct *ct)
 	stop_g2h_handler(ct);
 }
 
Quoted 5 lines

drivers/gpu/drm/xe/xe_guc_ct.c void xe_guc_ct_disable(struct xe_guc_ct *ct) · line 578

Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
+/**
+ * xe_guc_ct_flush - Flush all processing of G2H / H2G

Flush and stop ?

Sure.

Quoted 7 lines

drivers/gpu/drm/xe/xe_guc_ct.c void xe_guc_ct_disable(struct xe_guc_ct *ct) · line 582

Quoted 4 lines
drivers/gpu/drm/xe/xe_guc_ct.c +4 -0
+ * @ct: the &xe_guc_ct
+ */
+void xe_guc_ct_flush(struct xe_guc_ct *ct)
+{

do we care about ct being not initialized yet?

I don't see how a VF can be moved if the driver isn't loaded. Maybe can an assert here to ensure this assumption is correct?

Quoted 6 lines

drivers/gpu/drm/xe/xe_guc_ct.c void xe_guc_ct_disable(struct xe_guc_ct *ct) · line 584

Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
+	receive_g2h(ct);
+	wake_up_all(&ct->g2h_fence_wq);

this is part of the guc_ct_change_state() called by below stop isn't that sufficient ?

Agree not needed, let me drop this.

Matt

Quoted 19 lines
Quoted 18 lines
drivers/gpu/drm/xe/xe_guc_ct.c +3 -0
+	xe_guc_ct_stop(ct);
+}
+
 /**
  * xe_guc_ct_stop - Set GuC to stopped state
  * @ct: the &xe_guc_ct
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
index d6c81325a76c..1e32a59817cc 100644
@@ -17,6 +17,7 @@ int xe_guc_ct_init_post_hwconfig(struct xe_guc_ct *ct);
 int xe_guc_ct_enable(struct xe_guc_ct *ct);
 void xe_guc_ct_disable(struct xe_guc_ct *ct);
 void xe_guc_ct_stop(struct xe_guc_ct *ct);
+void xe_guc_ct_flush(struct xe_guc_ct *ct);
 void xe_guc_ct_fast_path(struct xe_guc_ct *ct);
 
 struct xe_guc_ct_snapshot *xe_guc_ct_snapshot_capture(struct xe_guc_ct *ct);
author maintainer
#62 lore raw

PATCH v2 22/34 drm/xe/vf: Reset TLB invalidations during VF post migration recovery

In reply to Matthew Brost

TLB invalidations can be lost during VF post-migration recovery. Since the VF has migrated, these invalidations are no longer needed.

Reset the TLB invalidation frontend, which will signal all pending fences.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 2 ++
 1 file changed, 2 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
index 071c2c5e0d0a..01399a871f12 100644
@@ -35,6 +35,7 @@
 #include "xe_sriov.h"
 #include "xe_sriov_vf.h"
 #include "xe_tile_sriov_vf.h"
+#include "xe_tlb_inval.h"
 #include "xe_uc_fw.h"
 #include "xe_wopcm.h"
 
@@ -1187,6 +1188,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)
 	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
 	xe_guc_ct_flush(&gt->uc.guc.ct);
+	xe_tlb_inval_reset(&gt->tlb_inval);
 	xe_guc_submit_pause(&gt->uc.guc);
 }
 
Signature · 1 line
2.34.1
#63 lore raw

Re: [PATCH v2 22/34] drm/xe/vf: Reset TLB invalidations during VF post migration recovery

Quoted 5 lines · Matthew Brost

TLB invalidations can be lost during VF post-migration recovery. Since the VF has migrated, these invalidations are no longer needed.

Reset the TLB invalidation frontend, which will signal all pending fences.

Can something other than timeout wait on these fences? Because if they trigger

additional processes, then it would make sense to wait with this.

If this can trigger submissions, then doing this after submit_pause would make sense.

If can trigger even more, then moving this to kickstart (to after recovery) may be a good idea.

-Tomasz

Quoted 25 lines
---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 2 ++
  1 file changed, 2 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
index 071c2c5e0d0a..01399a871f12 100644
@@ -35,6 +35,7 @@
  #include "xe_sriov.h"
  #include "xe_sriov_vf.h"
  #include "xe_tile_sriov_vf.h"
+#include "xe_tlb_inval.h"
  #include "xe_uc_fw.h"
  #include "xe_wopcm.h"
 
@@ -1187,6 +1188,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)
  	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
  	xe_guc_ct_flush(&gt->uc.guc.ct);
+	xe_tlb_inval_reset(&gt->tlb_inval);
  	xe_guc_submit_pause(&gt->uc.guc);
  }
 
author maintainer
#64 lore raw

Re: [PATCH v2 22/34] drm/xe/vf: Reset TLB invalidations during VF post migration recovery

Quoted 10 lines · Tomasz
Quoted 5 lines · Matthew Brost

TLB invalidations can be lost during VF post-migration recovery. Since the VF has migrated, these invalidations are no longer needed.

Reset the TLB invalidation frontend, which will signal all pending fences.

Can something other than timeout wait on these fences? Because if they trigger

The hardware via a G2H - xe_guc_ct_flush_and_stop find any remaining G2H, the timeout mechanism, the CT channel being stopped or via xe_tlb_inval_reset are th ways tlb invalidation fences signal.

Quoted 6 lines

additional processes, then it would make sense to wait with this.

If this can trigger submissions, then doing this after submit_pause would make sense.

Yes, fence signaling can trigger submissions. It is harmless where it is but logically perhaps after xe_guc_submit_pause makes a bit more sense. Will move this.

Quoted 27 lines

If can trigger even more, then moving this to kickstart (to after recovery) may be a good idea.

-Tomasz

Quoted 22 lines
---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 2 ++
  1 file changed, 2 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -0
index 071c2c5e0d0a..01399a871f12 100644
@@ -35,6 +35,7 @@
  #include "xe_sriov.h"
  #include "xe_sriov_vf.h"
  #include "xe_tile_sriov_vf.h"
+#include "xe_tlb_inval.h"
  #include "xe_uc_fw.h"
  #include "xe_wopcm.h"
@@ -1187,6 +1188,7 @@ static void vf_post_migration_shutdown(struct xe_gt *gt)
  	spin_unlock_irq(&gt->sriov.vf.migration.lock);
  	xe_guc_ct_flush(&gt->uc.guc.ct);
+	xe_tlb_inval_reset(&gt->tlb_inval);
  	xe_guc_submit_pause(&gt->uc.guc);
  }
author maintainer
#65 lore raw

PATCH v2 23/34 drm/xe/vf: Kickstart after resfix in VF post migration recovery

In reply to Matthew Brost

GuC needs to be live for the GuC submission state machine to resubmit anything lost during VF post-migration recovery. Therefore, move the kickstart step after resfix to ensure proper resubmission.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 18 ++++++++++--------
 1 file changed, 10 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -8
index 01399a871f12..2a077a98e92e 100644
@@ -1216,13 +1216,6 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
 
 static void vf_post_migration_kickstart(struct xe_gt *gt)
 {
-	/*
-	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
-	 * must be working at this point, since the recovery did started,
-	 * but the rest was not enabled using the procedure from spec.
-	 */
-	xe_irq_resume(gt_to_xe(gt));
-
 	xe_guc_submit_unpause(&gt->uc.guc);
 }
 
@@ -1239,6 +1232,14 @@ static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
 	}
 	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	if (!skip_resfix)
+		xe_irq_resume(gt_to_xe(gt));
+
 	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);
 }
 
@@ -1262,11 +1263,12 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
 	if (err)
 		goto fail;
 
-	vf_post_migration_kickstart(gt);
 	err = vf_post_migration_notify_resfix_done(gt);
 	if (err && err != -EAGAIN)
 		goto fail;
 
+	vf_post_migration_kickstart(gt);
+
 	xe_pm_runtime_put(xe);
 	xe_gt_sriov_notice(gt, "migration recovery ended\n");
 	return;
Signature · 1 line
2.34.1
Reviewed-by on v2 23/34
#66 lore raw

Re: [PATCH v2 23/34] drm/xe/vf: Kickstart after resfix in VF post migration recovery

Quoted 3 lines · Matthew Brost

GuC needs to be live for the GuC submission state machine to resubmit anything lost during VF post-migration recovery. Therefore, move the kickstart step after resfix to ensure proper resubmission.

All good.

-Tomasz

Quoted 52 lines
---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 18 ++++++++++--------
  1 file changed, 10 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -8
index 01399a871f12..2a077a98e92e 100644
@@ -1216,13 +1216,6 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
 
  static void vf_post_migration_kickstart(struct xe_gt *gt)
  {
-	/*
-	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
-	 * must be working at this point, since the recovery did started,
-	 * but the rest was not enabled using the procedure from spec.
-	 */
-	xe_irq_resume(gt_to_xe(gt));
-
  	xe_guc_submit_unpause(&gt->uc.guc);
  }
 
@@ -1239,6 +1232,14 @@ static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
  	}
  	spin_unlock_irq(&gt->sriov.vf.migration.lock);
 
+	/*
+	 * Make sure interrupts on the new HW are properly set. The GuC IRQ
+	 * must be working at this point, since the recovery did started,
+	 * but the rest was not enabled using the procedure from spec.
+	 */
+	if (!skip_resfix)
+		xe_irq_resume(gt_to_xe(gt));
+
  	return skip_resfix ? -EAGAIN : xe_gt_sriov_vf_notify_resfix_done(gt);
  }
 
@@ -1262,11 +1263,12 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
  	if (err)
  		goto fail;
 
-	vf_post_migration_kickstart(gt);
  	err = vf_post_migration_notify_resfix_done(gt);
  	if (err && err != -EAGAIN)
  		goto fail;
 
+	vf_post_migration_kickstart(gt);
+
  	xe_pm_runtime_put(xe);
  	xe_gt_sriov_notice(gt, "migration recovery ended\n");
  	return;
author maintainer
#67 lore raw

PATCH v2 24/34 drm/xe/vf: Start CTs before resfix VF post migration recovery

In reply to Matthew Brost

Before resfix, all CTs stuck in the H2G queue need to be squashed, as they may contain stale or invalid data.

Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  7 +++++
 drivers/gpu/drm/xe/xe_guc_ct.c      | 48 +++++++++++++++++++++++------
 drivers/gpu/drm/xe/xe_guc_ct.h      |  1 +
 3 files changed, 46 insertions(+), 10 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7 -0
index 2a077a98e92e..34e57f44fbf4 100644
@@ -1214,6 +1214,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
 	return 0;
 }
 
+static void vf_post_migration_rearm(struct xe_gt *gt)
+{
+	xe_guc_ct_restart(&gt->uc.guc.ct);
+}
+
 static void vf_post_migration_kickstart(struct xe_gt *gt)
 {
 	xe_guc_submit_unpause(&gt->uc.guc);
@@ -1263,6 +1268,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
 	if (err)
 		goto fail;
 
+	vf_post_migration_rearm(gt);
+
 	err = vf_post_migration_notify_resfix_done(gt);
 	if (err && err != -EAGAIN)
 		goto fail;
drivers/gpu/drm/xe/xe_guc_ct.c +38 -10

68 lines not on this page · raw message

drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
index 1e32a59817cc..506616ec1db0 100644
@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);
 int xe_guc_ct_init(struct xe_guc_ct *ct);
 int xe_guc_ct_init_post_hwconfig(struct xe_guc_ct *ct);
 int xe_guc_ct_enable(struct xe_guc_ct *ct);
+int xe_guc_ct_restart(struct xe_guc_ct *ct);
 void xe_guc_ct_disable(struct xe_guc_ct *ct);
 void xe_guc_ct_stop(struct xe_guc_ct *ct);
 void xe_guc_ct_flush(struct xe_guc_ct *ct);
Signature · 1 line
2.34.1
#68 lore raw

Re: [PATCH v2 24/34] drm/xe/vf: Start CTs before resfix VF post migration recovery

Quoted 52 lines · Matthew Brost

Before resfix, all CTs stuck in the H2G queue need to be squashed, as they may contain stale or invalid data.

Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  7 +++++
 drivers/gpu/drm/xe/xe_guc_ct.c      | 48 +++++++++++++++++++++++------
 drivers/gpu/drm/xe/xe_guc_ct.h      |  1 +
 3 files changed, 46 insertions(+), 10 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7 -0
index 2a077a98e92e..34e57f44fbf4 100644
@@ -1214,6 +1214,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
 	return 0;
 }
 
+static void vf_post_migration_rearm(struct xe_gt *gt)
+{
+	xe_guc_ct_restart(&gt->uc.guc.ct);
+}
+
 static void vf_post_migration_kickstart(struct xe_gt *gt)
 {
 	xe_guc_submit_unpause(&gt->uc.guc);
@@ -1263,6 +1268,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
 	if (err)
 		goto fail;
 
+	vf_post_migration_rearm(gt);
+
 	err = vf_post_migration_notify_resfix_done(gt);
 	if (err && err != -EAGAIN)
 		goto fail;
drivers/gpu/drm/xe/xe_guc_ct.c +1 -1
index d4a132b81b86..fa55e9c9de3a 100644
@@ -501,7 +501,7 @@ static void ct_exit_safe_mode(struct xe_guc_ct *ct)
 		xe_gt_dbg(ct_to_gt(ct), "GuC CT safe-mode disabled\n");
 }
 
-int xe_guc_ct_enable(struct xe_guc_ct *ct)
+static int __xe_guc_ct_start(struct xe_guc_ct *ct, bool needs_register)
 {
 	struct xe_device *xe = ct_to_xe(ct);
 	struct xe_gt *gt = ct_to_gt(ct);

drivers/gpu/drm/xe/xe_guc_ct.c int xe_guc_ct_enable(struct xe_guc_ct *ct) · line 514

Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_ct.c +0 -0
@@ -513,17 +513,19 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)
 	guc_ct_ctb_h2g_init(xe, &ct->ctbs.h2g, &ct->bo->vmap);
 	guc_ct_ctb_g2h_init(xe, &ct->ctbs.g2h, &ct->bo->vmap);

I'm not sure that this is aligned with the spec that after CTB being registered (here needs_register==false) we will modify the shared CTB descriptor, where GuC owns its head/tail data.

maybe on the restart() case we should just erase what was left unprocessed by the GuC from our tail to GuC's head in H2G CTB?

as extra bonus we can then look what will be exactly "lost"

Quoted 71 lines
drivers/gpu/drm/xe/xe_guc_ct.c +37 -9
-	err = guc_ct_ctb_h2g_register(ct);
-	if (err)
-		goto err_out;
+	if (needs_register) {
+		err = guc_ct_ctb_h2g_register(ct);
+		if (err)
+			goto err_out;
 
-	err = guc_ct_ctb_g2h_register(ct);
-	if (err)
-		goto err_out;
+		err = guc_ct_ctb_g2h_register(ct);
+		if (err)
+			goto err_out;
 
-	err = guc_ct_control_toggle(ct, true);
-	if (err)
-		goto err_out;
+		err = guc_ct_control_toggle(ct, true);
+		if (err)
+			goto err_out;
+	}
 
 	guc_ct_change_state(ct, XE_GUC_CT_STATE_ENABLED);
 
@@ -555,6 +557,32 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)
 	return err;
 }
 
+/**
+ * xe_guc_ct_restart() - Restart GuC CT
+ * @ct: the &xe_guc_ct
+ *
+ * Restart GuC CT to an empty state without issuing a CT register MMIO command.
+ *
+ * Return: 0 on success, or a negative errno on failure.
+ */
+int xe_guc_ct_restart(struct xe_guc_ct *ct)
+{
+	return __xe_guc_ct_start(ct, false);
+}
+
+/**
+ * xe_guc_ct_enable() - Enable GuC CT
+ * @ct: the &xe_guc_ct
+ *
+ * Enable GuC CT to an empty state and issue a CT register MMIO command.
+ *
+ * Return: 0 on success, or a negative errno on failure.
+ */
+int xe_guc_ct_enable(struct xe_guc_ct *ct)
+{
+	return __xe_guc_ct_start(ct, true);
+}
+
 static void stop_g2h_handler(struct xe_guc_ct *ct)
 {
 	cancel_work_sync(&ct->g2h_worker);
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
index 1e32a59817cc..506616ec1db0 100644
@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);
 int xe_guc_ct_init(struct xe_guc_ct *ct);
 int xe_guc_ct_init_post_hwconfig(struct xe_guc_ct *ct);
 int xe_guc_ct_enable(struct xe_guc_ct *ct);
+int xe_guc_ct_restart(struct xe_guc_ct *ct);
 void xe_guc_ct_disable(struct xe_guc_ct *ct);
 void xe_guc_ct_stop(struct xe_guc_ct *ct);
 void xe_guc_ct_flush(struct xe_guc_ct *ct);
author maintainer
#69 lore raw

Re: [PATCH v2 24/34] drm/xe/vf: Start CTs before resfix VF post migration recovery

Quoted 49 lines · Michal Wajdeczko
Quoted 46 lines · Matthew Brost

Before resfix, all CTs stuck in the H2G queue need to be squashed, as they may contain stale or invalid data.

Starting the CTs clears all H2Gs in the queue. Any lost H2Gs are resubmitted by the GuC submission state machine.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c |  7 +++++
 drivers/gpu/drm/xe/xe_guc_ct.c      | 48 +++++++++++++++++++++++------
 drivers/gpu/drm/xe/xe_guc_ct.h      |  1 +
 3 files changed, 46 insertions(+), 10 deletions(-)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7 -0
index 2a077a98e92e..34e57f44fbf4 100644
@@ -1214,6 +1214,11 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
 	return 0;
 }
 
+static void vf_post_migration_rearm(struct xe_gt *gt)
+{
+	xe_guc_ct_restart(&gt->uc.guc.ct);
+}
+
 static void vf_post_migration_kickstart(struct xe_gt *gt)
 {
 	xe_guc_submit_unpause(&gt->uc.guc);
@@ -1263,6 +1268,8 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
 	if (err)
 		goto fail;
 
+	vf_post_migration_rearm(gt);
+
 	err = vf_post_migration_notify_resfix_done(gt);
 	if (err && err != -EAGAIN)
 		goto fail;
drivers/gpu/drm/xe/xe_guc_ct.c +0 -0
index d4a132b81b86..fa55e9c9de3a 100644
@@ -501,7 +501,7 @@ static void ct_exit_safe_mode(struct xe_guc_ct *ct)
 		xe_gt_dbg(ct_to_gt(ct), "GuC CT safe-mode disabled\n");
 }
 
Quoted 10 lines

drivers/gpu/drm/xe/xe_guc_ct.c int xe_guc_ct_enable(struct xe_guc_ct *ct) · line 514

Quoted 8 lines
drivers/gpu/drm/xe/xe_guc_ct.c +1 -1
-int xe_guc_ct_enable(struct xe_guc_ct *ct)
+static int __xe_guc_ct_start(struct xe_guc_ct *ct, bool needs_register)
 {
 	struct xe_device *xe = ct_to_xe(ct);
 	struct xe_gt *gt = ct_to_gt(ct);
@@ -513,17 +513,19 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)
 	guc_ct_ctb_h2g_init(xe, &ct->ctbs.h2g, &ct->bo->vmap);
 	guc_ct_ctb_g2h_init(xe, &ct->ctbs.g2h, &ct->bo->vmap);

I'm not sure that this is aligned with the spec that after CTB

Not to rant to much about specs, but those are frequently wrong.

Quoted 5 lines

being registered (here needs_register==false) we will modify the shared CTB descriptor, where GuC owns its head/tail data.

maybe on the restart() case we should just erase what was left unprocessed by the GuC from our tail to GuC's head in H2G CTB?

This definietly works but it in perhaps is safer to just squash everything into NOPs rather than mess with the head / tail. The GuC didn't like messing with LRC head / tail so maybe it follows we shouldn't mess with the CT head / tail too. Let me change this as it pretty straight forward to do this.

Quoted 2 lines

as extra bonus we can then look what will be exactly "lost"

I did consider this but that is some extra complexity which really is not required as GuC submission state machine already tracks everything. Also once the CT is stopped or we detect a resfix we just throw H2G away without programming them into the buffer so this really doesn't buy us anything.

Matt

Quoted 73 lines
Quoted 72 lines
drivers/gpu/drm/xe/xe_guc_ct.c +37 -9
-	err = guc_ct_ctb_h2g_register(ct);
-	if (err)
-		goto err_out;
+	if (needs_register) {
+		err = guc_ct_ctb_h2g_register(ct);
+		if (err)
+			goto err_out;
 
-	err = guc_ct_ctb_g2h_register(ct);
-	if (err)
-		goto err_out;
+		err = guc_ct_ctb_g2h_register(ct);
+		if (err)
+			goto err_out;
 
-	err = guc_ct_control_toggle(ct, true);
-	if (err)
-		goto err_out;
+		err = guc_ct_control_toggle(ct, true);
+		if (err)
+			goto err_out;
+	}
 
 	guc_ct_change_state(ct, XE_GUC_CT_STATE_ENABLED);
 
@@ -555,6 +557,32 @@ int xe_guc_ct_enable(struct xe_guc_ct *ct)
 	return err;
 }
 
+/**
+ * xe_guc_ct_restart() - Restart GuC CT
+ * @ct: the &xe_guc_ct
+ *
+ * Restart GuC CT to an empty state without issuing a CT register MMIO command.
+ *
+ * Return: 0 on success, or a negative errno on failure.
+ */
+int xe_guc_ct_restart(struct xe_guc_ct *ct)
+{
+	return __xe_guc_ct_start(ct, false);
+}
+
+/**
+ * xe_guc_ct_enable() - Enable GuC CT
+ * @ct: the &xe_guc_ct
+ *
+ * Enable GuC CT to an empty state and issue a CT register MMIO command.
+ *
+ * Return: 0 on success, or a negative errno on failure.
+ */
+int xe_guc_ct_enable(struct xe_guc_ct *ct)
+{
+	return __xe_guc_ct_start(ct, true);
+}
+
 static void stop_g2h_handler(struct xe_guc_ct *ct)
 {
 	cancel_work_sync(&ct->g2h_worker);
drivers/gpu/drm/xe/xe_guc_ct.h +1 -0
index 1e32a59817cc..506616ec1db0 100644
@@ -15,6 +15,7 @@ int xe_guc_ct_init_noalloc(struct xe_guc_ct *ct);
 int xe_guc_ct_init(struct xe_guc_ct *ct);
 int xe_guc_ct_init_post_hwconfig(struct xe_guc_ct *ct);
 int xe_guc_ct_enable(struct xe_guc_ct *ct);
+int xe_guc_ct_restart(struct xe_guc_ct *ct);
 void xe_guc_ct_disable(struct xe_guc_ct *ct);
 void xe_guc_ct_stop(struct xe_guc_ct *ct);
 void xe_guc_ct_flush(struct xe_guc_ct *ct);
author maintainer
#70 lore raw

PATCH v2 25/34 drm/xe/vf: Abort VF post migration recovery on failure

In reply to Matthew Brost

If VF post-migration recovery fails, the device is wedged. However, submission queues still need to be enabled for proper cleanup. In such cases, call into the GuC submission backend to restart all queues that were previously paused.

---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 10 ++++++++++
 drivers/gpu/drm/xe/xe_guc_submit.c  | 20 ++++++++++++++++++++
 drivers/gpu/drm/xe/xe_guc_submit.h  |  1 +
 3 files changed, 31 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
index 34e57f44fbf4..a987560de2c7 100644
@@ -1224,6 +1224,15 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)
 	xe_guc_submit_unpause(&gt->uc.guc);
 }
 
+static void vf_post_migration_abort(struct xe_gt *gt)
+{
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause_abort(&gt->uc.guc);
+}
+
 static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
 {
 	bool skip_resfix = false;
@@ -1280,6 +1289,7 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
 	xe_gt_sriov_notice(gt, "migration recovery ended\n");
 	return;
 fail:
+	vf_post_migration_abort(gt);
 	xe_pm_runtime_put(xe);
 	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
 	xe_device_declare_wedged(xe);
drivers/gpu/drm/xe/xe_guc_submit.c +20 -0
index 52b86cab4ec5..8bee65dd9ca6 100644
@@ -2345,6 +2345,26 @@ void xe_guc_submit_unpause(struct xe_guc *guc)
 	wake_up_all(&guc->ct.wq);
 }
 
+/**
+ * xe_guc_submit_abort - Avort all paused submission task on given GuC.
+ * @guc: the &xe_guc struct instance whose scheduler is to be aborted
+ */
+void xe_guc_submit_pause_abort(struct xe_guc *guc)
+{
+	struct xe_exec_queue *q;
+	unsigned long index;
+
+	mutex_lock(&guc->submission_state.lock);
+	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) {
+		struct xe_gpu_scheduler *sched = &q->guc->sched;
+
+		xe_sched_submission_start(sched);
+		if (exec_queue_killed_or_banned_or_wedged(q))
+			xe_guc_exec_queue_trigger_cleanup(q);
+	}
+	mutex_unlock(&guc->submission_state.lock);
+}
+
 static struct xe_exec_queue *
 g2h_exec_queue_lookup(struct xe_guc *guc, u32 guc_id)
 {
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
index f535fe3895e5..fe82c317048e 100644
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);
 int xe_guc_submit_start(struct xe_guc *guc);
 void xe_guc_submit_pause(struct xe_guc *guc);
 void xe_guc_submit_unpause(struct xe_guc *guc);
+void xe_guc_submit_pause_abort(struct xe_guc *guc);
 void xe_guc_submit_wedge(struct xe_guc *guc);
 
 int xe_guc_read_stopped(struct xe_guc *guc);
Signature · 1 line
2.34.1
#71 lore raw

Re: [PATCH v2 25/34] drm/xe/vf: Abort VF post migration recovery on failure

Quoted 33 lines · Matthew Brost

If VF post-migration recovery fails, the device is wedged. However, submission queues still need to be enabled for proper cleanup. In such cases, call into the GuC submission backend to restart all queues that were previously paused.

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 10 ++++++++++
  drivers/gpu/drm/xe/xe_guc_submit.c  | 20 ++++++++++++++++++++
  drivers/gpu/drm/xe/xe_guc_submit.h  |  1 +
  3 files changed, 31 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +9 -0
index 34e57f44fbf4..a987560de2c7 100644
@@ -1224,6 +1224,15 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)
  	xe_guc_submit_unpause(&gt->uc.guc);
  }
 
+static void vf_post_migration_abort(struct xe_gt *gt)
+{
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause_abort(&gt->uc.guc);
+}
+
  static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
  {
  	bool skip_resfix = false;

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_post_migration_recovery(struct xe_gt *gt) · line 1295

Quoted 7 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -1280,6 +1289,7 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
  	xe_gt_sriov_notice(gt, "migration recovery ended\n");
  	return;
  fail:
+	vf_post_migration_abort(gt);
  	xe_pm_runtime_put(xe);
  	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
  	xe_device_declare_wedged(xe);

We could alter the order here to make sure all queues have cleanup triggered. Now only the ones which were previously killed/banned/wedged have that.

Not sure if it would bring any benefits though.

drivers/gpu/drm/xe/xe_guc_submit.c void xe_guc_submit_unpause(struct xe_guc *guc) · line 2349

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2 -0
index 52b86cab4ec5..8bee65dd9ca6 100644
@@ -2345,6 +2345,26 @@ void xe_guc_submit_unpause(struct xe_guc *guc)
  	wake_up_all(&guc->ct.wq);
  }
 
+/**
+ * xe_guc_submit_abort - Avort all paused submission task on given GuC.

typo - Avort

-Tomasz

Quoted 33 lines
drivers/gpu/drm/xe/xe_guc_submit.c +18 -0
+ * @guc: the &xe_guc struct instance whose scheduler is to be aborted
+ */
+void xe_guc_submit_pause_abort(struct xe_guc *guc)
+{
+	struct xe_exec_queue *q;
+	unsigned long index;
+
+	mutex_lock(&guc->submission_state.lock);
+	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) {
+		struct xe_gpu_scheduler *sched = &q->guc->sched;
+
+		xe_sched_submission_start(sched);
+		if (exec_queue_killed_or_banned_or_wedged(q))
+			xe_guc_exec_queue_trigger_cleanup(q);
+	}
+	mutex_unlock(&guc->submission_state.lock);
+}
+
  static struct xe_exec_queue *
  g2h_exec_queue_lookup(struct xe_guc *guc, u32 guc_id)
  {
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
index f535fe3895e5..fe82c317048e 100644
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);
  int xe_guc_submit_start(struct xe_guc *guc);
  void xe_guc_submit_pause(struct xe_guc *guc);
  void xe_guc_submit_unpause(struct xe_guc *guc);
+void xe_guc_submit_pause_abort(struct xe_guc *guc);
  void xe_guc_submit_wedge(struct xe_guc *guc);
 
  int xe_guc_read_stopped(struct xe_guc *guc);
author maintainer
#72 lore raw

Re: [PATCH v2 25/34] drm/xe/vf: Abort VF post migration recovery on failure

Quoted 46 lines · Tomasz
Quoted 32 lines · Matthew Brost

If VF post-migration recovery fails, the device is wedged. However, submission queues still need to be enabled for proper cleanup. In such cases, call into the GuC submission backend to restart all queues that were previously paused.

---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 10 ++++++++++
  drivers/gpu/drm/xe/xe_guc_submit.c  | 20 ++++++++++++++++++++
  drivers/gpu/drm/xe/xe_guc_submit.h  |  1 +
  3 files changed, 31 insertions(+)
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +9 -0
index 34e57f44fbf4..a987560de2c7 100644
@@ -1224,6 +1224,15 @@ static void vf_post_migration_kickstart(struct xe_gt *gt)
  	xe_guc_submit_unpause(&gt->uc.guc);
  }
+static void vf_post_migration_abort(struct xe_gt *gt)
+{
+	spin_lock_irq(&gt->sriov.vf.migration.lock);
+	WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, false);
+	spin_unlock_irq(&gt->sriov.vf.migration.lock);
+
+	xe_guc_submit_pause_abort(&gt->uc.guc);
+}
+
  static int vf_post_migration_notify_resfix_done(struct xe_gt *gt)
  {
  	bool skip_resfix = false;

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static void vf_post_migration_recovery(struct xe_gt *gt) · line 1295

Quoted 7 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -1280,6 +1289,7 @@ static void vf_post_migration_recovery(struct xe_gt *gt)
  	xe_gt_sriov_notice(gt, "migration recovery ended\n");
  	return;
  fail:
+	vf_post_migration_abort(gt);
  	xe_pm_runtime_put(xe);
  	xe_gt_sriov_err(gt, "migration recovery failed (%pe)\n", ERR_PTR(err));
  	xe_device_declare_wedged(xe);

We could alter the order here to make sure all queues have cleanup triggered. Now only the ones which were previously killed/banned/wedged have that.

Quoted 2 lines

Not sure if it would bring any benefits though.

In general doing things like this (doing things behind users back) has proven to be a bad idea. Also in wedge mode == 2, we intentionally stop all queues from being torn down until driver load - this enables SV to come in and run tools to debug hardware issue or we can examine general hardware state without it changing.

Quoted 12 lines

drivers/gpu/drm/xe/xe_guc_submit.c void xe_guc_submit_unpause(struct xe_guc *guc) · line 2348

Quoted 9 lines
drivers/gpu/drm/xe/xe_guc_submit.c +2 -0
index 52b86cab4ec5..8bee65dd9ca6 100644
@@ -2345,6 +2345,26 @@ void xe_guc_submit_unpause(struct xe_guc *guc)
  	wake_up_all(&guc->ct.wq);
  }
+/**
+ * xe_guc_submit_abort - Avort all paused submission task on given GuC.

typo - Avort

Yea, will fix.

Matt

Quoted 34 lines

-Tomasz

Quoted 32 lines
drivers/gpu/drm/xe/xe_guc_submit.c +18 -0
+ * @guc: the &xe_guc struct instance whose scheduler is to be aborted
+ */
+void xe_guc_submit_pause_abort(struct xe_guc *guc)
+{
+	struct xe_exec_queue *q;
+	unsigned long index;
+
+	mutex_lock(&guc->submission_state.lock);
+	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q) {
+		struct xe_gpu_scheduler *sched = &q->guc->sched;
+
+		xe_sched_submission_start(sched);
+		if (exec_queue_killed_or_banned_or_wedged(q))
+			xe_guc_exec_queue_trigger_cleanup(q);
+	}
+	mutex_unlock(&guc->submission_state.lock);
+}
+
  static struct xe_exec_queue *
  g2h_exec_queue_lookup(struct xe_guc *guc, u32 guc_id)
  {
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
index f535fe3895e5..fe82c317048e 100644
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);
  int xe_guc_submit_start(struct xe_guc *guc);
  void xe_guc_submit_pause(struct xe_guc *guc);
  void xe_guc_submit_unpause(struct xe_guc *guc);
+void xe_guc_submit_pause_abort(struct xe_guc *guc);
  void xe_guc_submit_wedge(struct xe_guc *guc);
  int xe_guc_read_stopped(struct xe_guc *guc);
author maintainer
#73 lore raw

PATCH v2 26/34 drm/xe/vf: Replay GuC submission state on pause / unpause

In reply to Matthew Brost

Fixup GuC submission pause / unpause functions to properly replay any possible state lost during VF post migration recovery.

---
 drivers/gpu/drm/xe/xe_gpu_scheduler.c        |  14 ++
 drivers/gpu/drm/xe/xe_gpu_scheduler.h        |   2 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c          |   1 +
 drivers/gpu/drm/xe/xe_guc_exec_queue_types.h |  15 ++
 drivers/gpu/drm/xe/xe_guc_submit.c           | 225 +++++++++++++++++--
 drivers/gpu/drm/xe/xe_guc_submit.h           |   1 +
 drivers/gpu/drm/xe/xe_sched_job_types.h      |   4 +
 7 files changed, 247 insertions(+), 15 deletions(-)
drivers/gpu/drm/xe/xe_gpu_scheduler.c +14 -0
index 455ccaf17314..af300adc7e1a 100644
@@ -135,3 +135,17 @@ void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched,
 	list_add_tail(&msg->link, &sched->msgs);
 	xe_sched_process_msg_queue(sched);
 }
+
+/**
+ * xe_sched_add_msg_head() - Xe GPU scheduler add message to head of list
+ * @sched: Xe GPU scheduler
+ * @msg: Message to add
+ */
+void xe_sched_add_msg_head(struct xe_gpu_scheduler *sched,
+			   struct xe_sched_msg *msg)
+{
+	lockdep_assert_held(&sched->base.job_list_lock);
+
+	list_add(&msg->link, &sched->msgs);
+	xe_sched_process_msg_queue(sched);
+}
drivers/gpu/drm/xe/xe_gpu_scheduler.h +2 -0
index e548b2aed95a..010003a6103a 100644
@@ -29,6 +29,8 @@ void xe_sched_add_msg(struct xe_gpu_scheduler *sched,
 		      struct xe_sched_msg *msg);
 void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched,
 			     struct xe_sched_msg *msg);
+void xe_sched_add_msg_head(struct xe_gpu_scheduler *sched,
+			   struct xe_sched_msg *msg);
 
 static inline void xe_sched_msg_lock(struct xe_gpu_scheduler *sched)
 {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
index a987560de2c7..91e7dbe80ab2 100644
@@ -1217,6 +1217,7 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
 static void vf_post_migration_rearm(struct xe_gt *gt)
 {
 	xe_guc_ct_restart(&gt->uc.guc.ct);
+	xe_guc_submit_unpause_prepare(&gt->uc.guc);
 }
 
 static void vf_post_migration_kickstart(struct xe_gt *gt)
drivers/gpu/drm/xe/xe_guc_exec_queue_types.h +15 -0
index c30c0e3ccbbb..a3b034e4b205 100644
@@ -51,6 +51,21 @@ struct xe_guc_exec_queue {
 	wait_queue_head_t suspend_wait;
 	/** @suspend_pending: a suspend of the exec_queue is pending */
 	bool suspend_pending;
+	/**
+	 * @needs_cleanup: Needs a cleanup message during VF post migration
+	 * recovery.
+	 */
+	bool needs_cleanup;
+	/**
+	 * @needs_suspend: Needs a suspend message during VF post migration
+	 * recovery.
+	 */
+	bool needs_suspend;
+	/**
+	 * @needs_resume: Needs a resume message during VF post migration
+	 * recovery.
+	 */
+	bool needs_resume;
 };
 
 #endif
drivers/gpu/drm/xe/xe_guc_submit.c +210 -15

308 lines not on this page · raw message

drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
index fe82c317048e..b49a2748ec46 100644
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);
 int xe_guc_submit_start(struct xe_guc *guc);
 void xe_guc_submit_pause(struct xe_guc *guc);
 void xe_guc_submit_unpause(struct xe_guc *guc);
+void xe_guc_submit_unpause_prepare(struct xe_guc *guc);
 void xe_guc_submit_pause_abort(struct xe_guc *guc);
 void xe_guc_submit_wedge(struct xe_guc *guc);
 
drivers/gpu/drm/xe/xe_sched_job_types.h +4 -0
index 7ce58765a34a..13e7a12b03ad 100644
@@ -63,6 +63,10 @@ struct xe_sched_job {
 	bool ring_ops_flush_tlb;
 	/** @ggtt: mapped in ggtt. */
 	bool ggtt;
+	/** @skip_emit: skip emitting the job */
+	bool skip_emit;
+	/** @last_replay: last job being replayed */
+	bool last_replay;
 	/** @ptrs: per instance pointers. */
 	struct xe_job_ptrs ptrs[];
 };
Signature · 1 line
2.34.1
#74 lore raw

Re: [PATCH v2 26/34] drm/xe/vf: Replay GuC submission state on pause / unpause

Quoted 214 lines · Matthew Brost

Fixup GuC submission pause / unpause functions to properly replay any possible state lost during VF post migration recovery.

---
  drivers/gpu/drm/xe/xe_gpu_scheduler.c        |  14 ++
  drivers/gpu/drm/xe/xe_gpu_scheduler.h        |   2 +
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c          |   1 +
  drivers/gpu/drm/xe/xe_guc_exec_queue_types.h |  15 ++
  drivers/gpu/drm/xe/xe_guc_submit.c           | 225 +++++++++++++++++--
  drivers/gpu/drm/xe/xe_guc_submit.h           |   1 +
  drivers/gpu/drm/xe/xe_sched_job_types.h      |   4 +
  7 files changed, 247 insertions(+), 15 deletions(-)
drivers/gpu/drm/xe/xe_gpu_scheduler.c +14 -0
index 455ccaf17314..af300adc7e1a 100644
@@ -135,3 +135,17 @@ void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched,
  	list_add_tail(&msg->link, &sched->msgs);
  	xe_sched_process_msg_queue(sched);
  }
+
+/**
+ * xe_sched_add_msg_head() - Xe GPU scheduler add message to head of list
+ * @sched: Xe GPU scheduler
+ * @msg: Message to add
+ */
+void xe_sched_add_msg_head(struct xe_gpu_scheduler *sched,
+			   struct xe_sched_msg *msg)
+{
+	lockdep_assert_held(&sched->base.job_list_lock);
+
+	list_add(&msg->link, &sched->msgs);
+	xe_sched_process_msg_queue(sched);
+}
drivers/gpu/drm/xe/xe_gpu_scheduler.h +2 -0
index e548b2aed95a..010003a6103a 100644
@@ -29,6 +29,8 @@ void xe_sched_add_msg(struct xe_gpu_scheduler *sched,
  		      struct xe_sched_msg *msg);
  void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched,
  			     struct xe_sched_msg *msg);
+void xe_sched_add_msg_head(struct xe_gpu_scheduler *sched,
+			   struct xe_sched_msg *msg);
 
  static inline void xe_sched_msg_lock(struct xe_gpu_scheduler *sched)
  {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
index a987560de2c7..91e7dbe80ab2 100644
@@ -1217,6 +1217,7 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
  static void vf_post_migration_rearm(struct xe_gt *gt)
  {
  	xe_guc_ct_restart(&gt->uc.guc.ct);
+	xe_guc_submit_unpause_prepare(&gt->uc.guc);
  }
 
  static void vf_post_migration_kickstart(struct xe_gt *gt)
drivers/gpu/drm/xe/xe_guc_exec_queue_types.h +15 -0
index c30c0e3ccbbb..a3b034e4b205 100644
@@ -51,6 +51,21 @@ struct xe_guc_exec_queue {
  	wait_queue_head_t suspend_wait;
  	/** @suspend_pending: a suspend of the exec_queue is pending */
  	bool suspend_pending;
+	/**
+	 * @needs_cleanup: Needs a cleanup message during VF post migration
+	 * recovery.
+	 */
+	bool needs_cleanup;
+	/**
+	 * @needs_suspend: Needs a suspend message during VF post migration
+	 * recovery.
+	 */
+	bool needs_suspend;
+	/**
+	 * @needs_resume: Needs a resume message during VF post migration
+	 * recovery.
+	 */
+	bool needs_resume;
  };
 
  #endif
drivers/gpu/drm/xe/xe_guc_submit.c +48 -10
index 8bee65dd9ca6..b112a4a91a5b 100644
@@ -425,6 +425,11 @@ static void set_exec_queue_destroyed(struct xe_exec_queue *q)
  	atomic_or(EXEC_QUEUE_STATE_DESTROYED, &q->guc->state);
  }
 
+static void clear_exec_queue_destroyed(struct xe_exec_queue *q)
+{
+	atomic_and(~EXEC_QUEUE_STATE_DESTROYED, &q->guc->state);
+}
+
  static bool exec_queue_banned(struct xe_exec_queue *q)
  {
  	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_BANNED;
@@ -505,7 +510,12 @@ static void set_exec_queue_extra_ref(struct xe_exec_queue *q)
  	atomic_or(EXEC_QUEUE_STATE_EXTRA_REF, &q->guc->state);
  }
 
-static bool __maybe_unused exec_queue_pending_resume(struct xe_exec_queue *q)
+static void clear_exec_queue_extra_ref(struct xe_exec_queue *q)
+{
+	atomic_and(~EXEC_QUEUE_STATE_EXTRA_REF, &q->guc->state);
+}
+
+static bool exec_queue_pending_resume(struct xe_exec_queue *q)
  {
  	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_PENDING_RESUME;
  }
@@ -520,7 +530,7 @@ static void clear_exec_queue_pending_resume(struct xe_exec_queue *q)
  	atomic_and(~EXEC_QUEUE_STATE_PENDING_RESUME, &q->guc->state);
  }
 
-static bool __maybe_unused exec_queue_pending_tdr_exit(struct xe_exec_queue *q)
+static bool exec_queue_pending_tdr_exit(struct xe_exec_queue *q)
  {
  	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_PENDING_TDR_EXIT;
  }
@@ -1080,7 +1090,7 @@ static void wq_item_append(struct xe_exec_queue *q)
  }
 
  #define RESUME_PENDING	~0x0ull
-static void submit_exec_queue(struct xe_exec_queue *q)
+static void submit_exec_queue(struct xe_exec_queue *q, struct xe_sched_job *job)
  {
  	struct xe_guc *guc = exec_queue_to_guc(q);
  	struct xe_lrc *lrc = q->lrc[0];
@@ -1092,10 +1102,13 @@ static void submit_exec_queue(struct xe_exec_queue *q)
 
  	xe_gt_assert(guc_to_gt(guc), exec_queue_registered(q));
 
-	if (xe_exec_queue_is_parallel(q))
-		wq_item_append(q);
-	else
-		xe_lrc_set_ring_tail(lrc, lrc->ring.tail);
+	if (!job->skip_emit || job->last_replay) {
+		if (xe_exec_queue_is_parallel(q))
+			wq_item_append(q);
+		else
+			xe_lrc_set_ring_tail(lrc, lrc->ring.tail);
+		job->last_replay = false;
+	}
 
  	if (exec_queue_suspended(q) && !xe_exec_queue_is_parallel(q))
  		return;
@@ -1148,8 +1161,10 @@ guc_exec_queue_run_job(struct drm_sched_job *drm_job)
  	if (!killed_or_banned_or_wedged && !xe_sched_job_is_error(job)) {
  		if (!exec_queue_registered(q))
  			register_exec_queue(q, GUC_CONTEXT_NORMAL);
-		q->ring_ops->emit_job(job);
-		submit_exec_queue(q);
+		if (!job->skip_emit)
+			q->ring_ops->emit_job(job);
+		submit_exec_queue(q, job);
+		job->skip_emit = false;
  	}
 
  	/*
@@ -1860,6 +1875,7 @@ static void __guc_exec_queue_process_msg_resume(struct xe_sched_msg *msg)
  #define RESUME		4
  #define OPCODE_MASK	0xf
  #define MSG_LOCKED	BIT(8)
+#define MSG_HEAD	BIT(9)
 
  static void guc_exec_queue_process_msg(struct xe_sched_msg *msg)
  {
@@ -1984,12 +2000,24 @@ static void guc_exec_queue_add_msg(struct xe_exec_queue *q, struct xe_sched_msg
  	msg->private_data = q;
 
  	trace_xe_sched_msg_add(msg);
-	if (opcode & MSG_LOCKED)
+	if (opcode & MSG_HEAD)
+		xe_sched_add_msg_head(&q->guc->sched, msg);
+	else if (opcode & MSG_LOCKED)
  		xe_sched_add_msg_locked(&q->guc->sched, msg);
  	else
  		xe_sched_add_msg(&q->guc->sched, msg);
  }
 
+static void guc_exec_queue_try_add_msg_head(struct xe_exec_queue *q,
+					    struct xe_sched_msg *msg,
+					    u32 opcode)
+{
+	if (!list_empty(&msg->link))
+		return;
+
+	guc_exec_queue_add_msg(q, msg, opcode | MSG_LOCKED | MSG_HEAD);
+}
+
  static bool guc_exec_queue_try_add_msg(struct xe_exec_queue *q,
  				       struct xe_sched_msg *msg,
  				       u32 opcode)
@@ -2264,6 +2292,93 @@ void xe_guc_submit_stop(struct xe_guc *guc)
 
  }
 
+/*
+ * This function is quite complex but only real way to ensure no state is lost
+ * during VF resume flows. The function scans the queue state, make adjustments
+ * as needed, and queues jobs / messages which replayed upon unpause.
+ */
+static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)
+{
+	struct xe_gpu_scheduler *sched = &q->guc->sched;
+	struct xe_sched_job *job;
+	bool pending_enable, pending_disable, pending_resume;

drivers/gpu/drm/xe/xe_guc_submit.c void xe_guc_submit_stop(struct xe_guc *guc) · line 2314

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +10 -0
+	int i;
+
+	lockdep_assert_held(&guc->submission_state.lock);
+
+	/* Stop scheduling + flush any DRM scheduler operations */
+	xe_sched_submission_stop(sched);
+	if (xe_exec_queue_is_lr(q))
+		cancel_work_sync(&q->guc->lr_tdr);
+	else
+		cancel_delayed_work_sync(&sched->base.work_tdr);

We're doing the same cancelling in __guc_exec_queue_destroy_async(), maybe close it into a function?

Quoted 33 lines
drivers/gpu/drm/xe/xe_guc_submit.c +33 -0
+
+	pending_enable = exec_queue_pending_enable(q);
+	pending_resume = exec_queue_pending_resume(q);
+
+	if (pending_enable && pending_resume)
+		q->guc->needs_resume = true;
+
+	if (pending_enable && !pending_resume &&
+	    !exec_queue_pending_tdr_exit(q)) {
+		clear_exec_queue_registered(q);
+		if (xe_exec_queue_is_lr(q))
+			xe_exec_queue_put(q);
+	}
+
+	if (pending_enable) {
+		clear_exec_queue_enabled(q);
+		clear_exec_queue_pending_resume(q);
+		clear_exec_queue_pending_tdr_exit(q);
+		clear_exec_queue_pending_enable(q);
+	}
+
+	if (exec_queue_destroyed(q) && exec_queue_registered(q)) {
+		clear_exec_queue_destroyed(q);
+		if (exec_queue_extra_ref(q))
+			xe_exec_queue_put(q);
+		else
+			q->guc->needs_cleanup = true;
+		clear_exec_queue_extra_ref(q);
+	}
+
+	pending_disable = exec_queue_pending_disable(q);
+
+	if (pending_disable && exec_queue_suspended(q)) {

drivers/gpu/drm/xe/xe_guc_submit.c void xe_guc_submit_stop(struct xe_guc *guc) · line 2357

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +10 -0
+		clear_exec_queue_suspended(q);
+		q->guc->needs_suspend = true;
+	}
+
+	if (pending_disable) {
+		if (!pending_enable)
+			set_exec_queue_enabled(q);
+		clear_exec_queue_pending_disable(q);
+		clear_exec_queue_check_timeout(q);
+	}

maybe we can close the above into a separate function as well?

ie. guc_exec_queue_undo_unfinished_state_change()?

guc_exec_queue_revert_pending_state_change()?

That would make this function easier to read, but also describe what we're doing.

Then, a counterfunction could be ripped out of guc_exec_queue_unpause().

drivers/gpu/drm/xe/xe_guc_submit.c void xe_guc_submit_stop(struct xe_guc *guc) · line 2368

Quoted 11 lines
drivers/gpu/drm/xe/xe_guc_submit.c +11 -0
+
+	q->guc->resume_time = 0;
+
+	if (xe_exec_queue_is_parallel(q)) {
+		struct xe_device *xe = guc_to_xe(guc);
+		struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
+
+		for (i = 0; i < WQ_SIZE / sizeof(u32); ++i)
+			parallel_write(xe, map, wq[i],
+				       FIELD_PREP(WQ_TYPE_MASK, WQ_TYPE_NOOP) |
+				       FIELD_PREP(WQ_LEN_MASK, 0));

ok so for parallel wq we're NOP'ing everything and adding the items back at new positions? Maybe a comment here would help in understanding that.

-Tomasz

Quoted 160 lines
drivers/gpu/drm/xe/xe_guc_submit.c +98 -5
+	}
+
+	job = xe_sched_first_pending_job(sched);
+	if (job) {
+		/*
+		 * Adjust software tail so jobs submitted overwrite previous
+		 * position in ring buffer with new GGTT addresses.
+		 */
+		for (i = 0; i < q->width; ++i)
+			q->lrc[i]->ring.tail = job->ptrs[i].head;
+	}
+}
+
  /**
   * xe_guc_submit_pause - Stop further runs of submission tasks on given GuC.
   * @guc: the &xe_guc struct instance whose scheduler is to be disabled
@@ -2273,8 +2388,12 @@ void xe_guc_submit_pause(struct xe_guc *guc)
  	struct xe_exec_queue *q;
  	unsigned long index;
 
+	xe_gt_assert(guc_to_gt(guc), vf_recovery(guc));
+
+	mutex_lock(&guc->submission_state.lock);
  	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
-		xe_sched_submission_stop_async(&q->guc->sched);
+		guc_exec_queue_pause(guc, q);
+	mutex_unlock(&guc->submission_state.lock);
  }
 
  static void guc_exec_queue_start(struct xe_exec_queue *q)
@@ -2323,11 +2442,87 @@ int xe_guc_submit_start(struct xe_guc *guc)
  	return 0;
  }
 
-static void guc_exec_queue_unpause(struct xe_exec_queue *q)
+static void guc_exec_queue_unpause_prepare(struct xe_guc *guc,
+					   struct xe_exec_queue *q)
  {
  	struct xe_gpu_scheduler *sched = &q->guc->sched;
+	struct drm_sched_job *s_job;
+	struct xe_sched_job *job = NULL;
+
+	list_for_each_entry(s_job, &sched->base.pending_list, list) {
+		job = to_xe_sched_job(s_job);
+
+		q->ring_ops->emit_job(job);
+		job->skip_emit = true;
+	}
+
+	if (job)
+		job->last_replay = true;
+}
+
+/**
+ * xe_guc_submit_unpause_prepare - Prepare unpause submission tasks on given GuC.
+ * @guc: the &xe_guc struct instance whose scheduler is to be prepared for unpause
+ */
+void xe_guc_submit_unpause_prepare(struct xe_guc *guc)
+{
+	struct xe_exec_queue *q;
+	unsigned long index;
+
+	xe_gt_assert(guc_to_gt(guc), vf_recovery(guc));
+
+	mutex_lock(&guc->submission_state.lock);
+	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
+		guc_exec_queue_unpause_prepare(guc, q);
+	mutex_unlock(&guc->submission_state.lock);
+}
+
+static void guc_exec_queue_unpause(struct xe_guc *guc, struct xe_exec_queue *q)
+{
+	struct xe_gpu_scheduler *sched = &q->guc->sched;
+	struct xe_sched_msg *msg;
+	bool needs_tdr = exec_queue_killed_or_banned_or_wedged(q);
+
+	lockdep_assert_held(&guc->submission_state.lock);
+
+	xe_sched_resubmit_jobs(sched);
+
+	if (q->guc->needs_cleanup) {
+		msg = q->guc->static_msgs + STATIC_MSG_CLEANUP;
+
+		guc_exec_queue_add_msg(q, msg, CLEANUP);
+		q->guc->needs_cleanup = false;
+	}
+
+	if (q->guc->needs_suspend) {
+		msg = q->guc->static_msgs + STATIC_MSG_SUSPEND;
+
+		xe_sched_msg_lock(sched);
+		guc_exec_queue_try_add_msg_head(q, msg, SUSPEND);
+		xe_sched_msg_unlock(sched);
+
+		q->guc->needs_suspend = false;
+	}
+
+	/*
+	 * The resume must be in the message queue before the suspend as it is
+	 * not possible for a resume to be issued if a suspend pending is, but
+	 * the inverse is possible.
+	 */
+	if (q->guc->needs_resume) {
+		msg = q->guc->static_msgs + STATIC_MSG_RESUME;
+
+		xe_sched_msg_lock(sched);
+		guc_exec_queue_try_add_msg_head(q, msg, RESUME);
+		xe_sched_msg_unlock(sched);
+
+		q->guc->needs_resume = false;
+	}
 
  	xe_sched_submission_start(sched);
+	if (needs_tdr)
+		xe_guc_exec_queue_trigger_cleanup(q);
+	xe_sched_submission_resume_tdr(sched);
  }
 
  /**
@@ -2339,10 +2534,10 @@ void xe_guc_submit_unpause(struct xe_guc *guc)
  	struct xe_exec_queue *q;
  	unsigned long index;
 
+	mutex_lock(&guc->submission_state.lock);
  	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
-		guc_exec_queue_unpause(q);
-
-	wake_up_all(&guc->ct.wq);
+		guc_exec_queue_unpause(guc, q);
+	mutex_unlock(&guc->submission_state.lock);
  }
 
  /**
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
index fe82c317048e..b49a2748ec46 100644
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);
  int xe_guc_submit_start(struct xe_guc *guc);
  void xe_guc_submit_pause(struct xe_guc *guc);
  void xe_guc_submit_unpause(struct xe_guc *guc);
+void xe_guc_submit_unpause_prepare(struct xe_guc *guc);
  void xe_guc_submit_pause_abort(struct xe_guc *guc);
  void xe_guc_submit_wedge(struct xe_guc *guc);
 
drivers/gpu/drm/xe/xe_sched_job_types.h +4 -0
index 7ce58765a34a..13e7a12b03ad 100644
@@ -63,6 +63,10 @@ struct xe_sched_job {
  	bool ring_ops_flush_tlb;
  	/** @ggtt: mapped in ggtt. */
  	bool ggtt;
+	/** @skip_emit: skip emitting the job */
+	bool skip_emit;
+	/** @last_replay: last job being replayed */
+	bool last_replay;
  	/** @ptrs: per instance pointers. */
  	struct xe_job_ptrs ptrs[];
  };
author maintainer
#75 lore raw

Re: [PATCH v2 26/34] drm/xe/vf: Replay GuC submission state on pause / unpause

Quoted 205 lines · Tomasz
Quoted 203 lines · Matthew Brost

Fixup GuC submission pause / unpause functions to properly replay any possible state lost during VF post migration recovery.

---
  drivers/gpu/drm/xe/xe_gpu_scheduler.c        |  14 ++
  drivers/gpu/drm/xe/xe_gpu_scheduler.h        |   2 +
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c          |   1 +
  drivers/gpu/drm/xe/xe_guc_exec_queue_types.h |  15 ++
  drivers/gpu/drm/xe/xe_guc_submit.c           | 225 +++++++++++++++++--
  drivers/gpu/drm/xe/xe_guc_submit.h           |   1 +
  drivers/gpu/drm/xe/xe_sched_job_types.h      |   4 +
  7 files changed, 247 insertions(+), 15 deletions(-)
drivers/gpu/drm/xe/xe_gpu_scheduler.c +14 -0
index 455ccaf17314..af300adc7e1a 100644
@@ -135,3 +135,17 @@ void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched,
  	list_add_tail(&msg->link, &sched->msgs);
  	xe_sched_process_msg_queue(sched);
  }
+
+/**
+ * xe_sched_add_msg_head() - Xe GPU scheduler add message to head of list
+ * @sched: Xe GPU scheduler
+ * @msg: Message to add
+ */
+void xe_sched_add_msg_head(struct xe_gpu_scheduler *sched,
+			   struct xe_sched_msg *msg)
+{
+	lockdep_assert_held(&sched->base.job_list_lock);
+
+	list_add(&msg->link, &sched->msgs);
+	xe_sched_process_msg_queue(sched);
+}
drivers/gpu/drm/xe/xe_gpu_scheduler.h +2 -0
index e548b2aed95a..010003a6103a 100644
@@ -29,6 +29,8 @@ void xe_sched_add_msg(struct xe_gpu_scheduler *sched,
  		      struct xe_sched_msg *msg);
  void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched,
  			     struct xe_sched_msg *msg);
+void xe_sched_add_msg_head(struct xe_gpu_scheduler *sched,
+			   struct xe_sched_msg *msg);
  static inline void xe_sched_msg_lock(struct xe_gpu_scheduler *sched)
  {
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
index a987560de2c7..91e7dbe80ab2 100644
@@ -1217,6 +1217,7 @@ static int vf_post_migration_fixups(struct xe_gt *gt)
  static void vf_post_migration_rearm(struct xe_gt *gt)
  {
  	xe_guc_ct_restart(&gt->uc.guc.ct);
+	xe_guc_submit_unpause_prepare(&gt->uc.guc);
  }
  static void vf_post_migration_kickstart(struct xe_gt *gt)
drivers/gpu/drm/xe/xe_guc_exec_queue_types.h +15 -0
index c30c0e3ccbbb..a3b034e4b205 100644
@@ -51,6 +51,21 @@ struct xe_guc_exec_queue {
  	wait_queue_head_t suspend_wait;
  	/** @suspend_pending: a suspend of the exec_queue is pending */
  	bool suspend_pending;
+	/**
+	 * @needs_cleanup: Needs a cleanup message during VF post migration
+	 * recovery.
+	 */
+	bool needs_cleanup;
+	/**
+	 * @needs_suspend: Needs a suspend message during VF post migration
+	 * recovery.
+	 */
+	bool needs_suspend;
+	/**
+	 * @needs_resume: Needs a resume message during VF post migration
+	 * recovery.
+	 */
+	bool needs_resume;
  };
  #endif
drivers/gpu/drm/xe/xe_guc_submit.c +53 -10
index 8bee65dd9ca6..b112a4a91a5b 100644
@@ -425,6 +425,11 @@ static void set_exec_queue_destroyed(struct xe_exec_queue *q)
  	atomic_or(EXEC_QUEUE_STATE_DESTROYED, &q->guc->state);
  }
+static void clear_exec_queue_destroyed(struct xe_exec_queue *q)
+{
+	atomic_and(~EXEC_QUEUE_STATE_DESTROYED, &q->guc->state);
+}
+
  static bool exec_queue_banned(struct xe_exec_queue *q)
  {
  	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_BANNED;
@@ -505,7 +510,12 @@ static void set_exec_queue_extra_ref(struct xe_exec_queue *q)
  	atomic_or(EXEC_QUEUE_STATE_EXTRA_REF, &q->guc->state);
  }
-static bool __maybe_unused exec_queue_pending_resume(struct xe_exec_queue *q)
+static void clear_exec_queue_extra_ref(struct xe_exec_queue *q)
+{
+	atomic_and(~EXEC_QUEUE_STATE_EXTRA_REF, &q->guc->state);
+}
+
+static bool exec_queue_pending_resume(struct xe_exec_queue *q)
  {
  	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_PENDING_RESUME;
  }
@@ -520,7 +530,7 @@ static void clear_exec_queue_pending_resume(struct xe_exec_queue *q)
  	atomic_and(~EXEC_QUEUE_STATE_PENDING_RESUME, &q->guc->state);
  }
-static bool __maybe_unused exec_queue_pending_tdr_exit(struct xe_exec_queue *q)
+static bool exec_queue_pending_tdr_exit(struct xe_exec_queue *q)
  {
  	return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_PENDING_TDR_EXIT;
  }
@@ -1080,7 +1090,7 @@ static void wq_item_append(struct xe_exec_queue *q)
  }
  #define RESUME_PENDING	~0x0ull
-static void submit_exec_queue(struct xe_exec_queue *q)
+static void submit_exec_queue(struct xe_exec_queue *q, struct xe_sched_job *job)
  {
  	struct xe_guc *guc = exec_queue_to_guc(q);
  	struct xe_lrc *lrc = q->lrc[0];
@@ -1092,10 +1102,13 @@ static void submit_exec_queue(struct xe_exec_queue *q)
  	xe_gt_assert(guc_to_gt(guc), exec_queue_registered(q));
-	if (xe_exec_queue_is_parallel(q))
-		wq_item_append(q);
-	else
-		xe_lrc_set_ring_tail(lrc, lrc->ring.tail);
+	if (!job->skip_emit || job->last_replay) {
+		if (xe_exec_queue_is_parallel(q))
+			wq_item_append(q);
+		else
+			xe_lrc_set_ring_tail(lrc, lrc->ring.tail);
+		job->last_replay = false;
+	}
  	if (exec_queue_suspended(q) && !xe_exec_queue_is_parallel(q))
  		return;
@@ -1148,8 +1161,10 @@ guc_exec_queue_run_job(struct drm_sched_job *drm_job)
  	if (!killed_or_banned_or_wedged && !xe_sched_job_is_error(job)) {
  		if (!exec_queue_registered(q))
  			register_exec_queue(q, GUC_CONTEXT_NORMAL);
-		q->ring_ops->emit_job(job);
-		submit_exec_queue(q);
+		if (!job->skip_emit)
+			q->ring_ops->emit_job(job);
+		submit_exec_queue(q, job);
+		job->skip_emit = false;
  	}
  	/*
@@ -1860,6 +1875,7 @@ static void __guc_exec_queue_process_msg_resume(struct xe_sched_msg *msg)
  #define RESUME		4
  #define OPCODE_MASK	0xf
  #define MSG_LOCKED	BIT(8)
+#define MSG_HEAD	BIT(9)
  static void guc_exec_queue_process_msg(struct xe_sched_msg *msg)
  {
@@ -1984,12 +2000,24 @@ static void guc_exec_queue_add_msg(struct xe_exec_queue *q, struct xe_sched_msg
  	msg->private_data = q;
  	trace_xe_sched_msg_add(msg);
-	if (opcode & MSG_LOCKED)
+	if (opcode & MSG_HEAD)
+		xe_sched_add_msg_head(&q->guc->sched, msg);
+	else if (opcode & MSG_LOCKED)
  		xe_sched_add_msg_locked(&q->guc->sched, msg);
  	else
  		xe_sched_add_msg(&q->guc->sched, msg);
  }
+static void guc_exec_queue_try_add_msg_head(struct xe_exec_queue *q,
+					    struct xe_sched_msg *msg,
+					    u32 opcode)
+{
+	if (!list_empty(&msg->link))
+		return;
+
+	guc_exec_queue_add_msg(q, msg, opcode | MSG_LOCKED | MSG_HEAD);
+}
+
  static bool guc_exec_queue_try_add_msg(struct xe_exec_queue *q,
  				       struct xe_sched_msg *msg,
  				       u32 opcode)
@@ -2264,6 +2292,93 @@ void xe_guc_submit_stop(struct xe_guc *guc)
  }
+/*
+ * This function is quite complex but only real way to ensure no state is lost
+ * during VF resume flows. The function scans the queue state, make adjustments
+ * as needed, and queues jobs / messages which replayed upon unpause.
+ */
+static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)
+{
+	struct xe_gpu_scheduler *sched = &q->guc->sched;
+	struct xe_sched_job *job;
+	bool pending_enable, pending_disable, pending_resume;
+	int i;
+
+	lockdep_assert_held(&guc->submission_state.lock);
+
+	/* Stop scheduling + flush any DRM scheduler operations */
Quoted 9 lines

drivers/gpu/drm/xe/xe_guc_submit.c void xe_guc_submit_stop(struct xe_guc *guc) · line 2314

Quoted 5 lines
drivers/gpu/drm/xe/xe_guc_submit.c +5 -0
+	xe_sched_submission_stop(sched);
+	if (xe_exec_queue_is_lr(q))
+		cancel_work_sync(&q->guc->lr_tdr);
+	else
+		cancel_delayed_work_sync(&sched->base.work_tdr);

We're doing the same cancelling in __guc_exec_queue_destroy_async(), maybe close it into a function?

I could but in follow up I'm going to drop &q->guc->lr_tdr and just use DRM scheduler TDR so it will just be cancel_delayed_work_sync. I have the code for this lying around and there a couple of upcoming features this will get simplified if &q->guc->lr_tdr is droppped.

Quoted 46 lines
Quoted 33 lines
drivers/gpu/drm/xe/xe_guc_submit.c +33 -0
+
+	pending_enable = exec_queue_pending_enable(q);
+	pending_resume = exec_queue_pending_resume(q);
+
+	if (pending_enable && pending_resume)
+		q->guc->needs_resume = true;
+
+	if (pending_enable && !pending_resume &&
+	    !exec_queue_pending_tdr_exit(q)) {
+		clear_exec_queue_registered(q);
+		if (xe_exec_queue_is_lr(q))
+			xe_exec_queue_put(q);
+	}
+
+	if (pending_enable) {
+		clear_exec_queue_enabled(q);
+		clear_exec_queue_pending_resume(q);
+		clear_exec_queue_pending_tdr_exit(q);
+		clear_exec_queue_pending_enable(q);
+	}
+
+	if (exec_queue_destroyed(q) && exec_queue_registered(q)) {
+		clear_exec_queue_destroyed(q);
+		if (exec_queue_extra_ref(q))
+			xe_exec_queue_put(q);
+		else
+			q->guc->needs_cleanup = true;
+		clear_exec_queue_extra_ref(q);
+	}
+
+	pending_disable = exec_queue_pending_disable(q);
+
+	if (pending_disable && exec_queue_suspended(q)) {

drivers/gpu/drm/xe/xe_guc_submit.c void xe_guc_submit_stop(struct xe_guc *guc) · line 2357

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +10 -0
+		clear_exec_queue_suspended(q);
+		q->guc->needs_suspend = true;
+	}
+
+	if (pending_disable) {
+		if (!pending_enable)
+			set_exec_queue_enabled(q);
+		clear_exec_queue_pending_disable(q);
+		clear_exec_queue_check_timeout(q);
+	}

maybe we can close the above into a separate function as well?

Quoted 2 lines

ie. guc_exec_queue_undo_unfinished_state_change()?

Do you mean all the parsing of exec queue state into a single function? That seems reasonable. I don't really want to abstract each individual if statement here as that in IMO that is too much abstraction and hard to figure out what is exactly going on.

Quoted 7 lines

guc_exec_queue_revert_pending_state_change()?

That would make this function easier to read, but also describe what we're doing.

Then, a counterfunction could be ripped out of guc_exec_queue_unpause().

Quoted 15 lines

drivers/gpu/drm/xe/xe_guc_submit.c void xe_guc_submit_stop(struct xe_guc *guc) · line 2368

Quoted 11 lines
drivers/gpu/drm/xe/xe_guc_submit.c +11 -0
+
+	q->guc->resume_time = 0;
+
+	if (xe_exec_queue_is_parallel(q)) {
+		struct xe_device *xe = guc_to_xe(guc);
+		struct iosys_map map = xe_lrc_parallel_map(q->lrc[0]);
+
+		for (i = 0; i < WQ_SIZE / sizeof(u32); ++i)
+			parallel_write(xe, map, wq[i],
+				       FIELD_PREP(WQ_TYPE_MASK, WQ_TYPE_NOOP) |
+				       FIELD_PREP(WQ_LEN_MASK, 0));

ok so for parallel wq we're NOP'ing everything and adding the items back at new positions? Maybe a comment here would help in understanding that.

Yes, the GuC didn't like messing with the head / tail. I can add a comment.

Matt

Quoted 154 lines

-Tomasz

Quoted 152 lines
drivers/gpu/drm/xe/xe_guc_submit.c +98 -5
+	}
+
+	job = xe_sched_first_pending_job(sched);
+	if (job) {
+		/*
+		 * Adjust software tail so jobs submitted overwrite previous
+		 * position in ring buffer with new GGTT addresses.
+		 */
+		for (i = 0; i < q->width; ++i)
+			q->lrc[i]->ring.tail = job->ptrs[i].head;
+	}
+}
+
  /**
   * xe_guc_submit_pause - Stop further runs of submission tasks on given GuC.
   * @guc: the &xe_guc struct instance whose scheduler is to be disabled
@@ -2273,8 +2388,12 @@ void xe_guc_submit_pause(struct xe_guc *guc)
  	struct xe_exec_queue *q;
  	unsigned long index;
+	xe_gt_assert(guc_to_gt(guc), vf_recovery(guc));
+
+	mutex_lock(&guc->submission_state.lock);
  	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
-		xe_sched_submission_stop_async(&q->guc->sched);
+		guc_exec_queue_pause(guc, q);
+	mutex_unlock(&guc->submission_state.lock);
  }
  static void guc_exec_queue_start(struct xe_exec_queue *q)
@@ -2323,11 +2442,87 @@ int xe_guc_submit_start(struct xe_guc *guc)
  	return 0;
  }
-static void guc_exec_queue_unpause(struct xe_exec_queue *q)
+static void guc_exec_queue_unpause_prepare(struct xe_guc *guc,
+					   struct xe_exec_queue *q)
  {
  	struct xe_gpu_scheduler *sched = &q->guc->sched;
+	struct drm_sched_job *s_job;
+	struct xe_sched_job *job = NULL;
+
+	list_for_each_entry(s_job, &sched->base.pending_list, list) {
+		job = to_xe_sched_job(s_job);
+
+		q->ring_ops->emit_job(job);
+		job->skip_emit = true;
+	}
+
+	if (job)
+		job->last_replay = true;
+}
+
+/**
+ * xe_guc_submit_unpause_prepare - Prepare unpause submission tasks on given GuC.
+ * @guc: the &xe_guc struct instance whose scheduler is to be prepared for unpause
+ */
+void xe_guc_submit_unpause_prepare(struct xe_guc *guc)
+{
+	struct xe_exec_queue *q;
+	unsigned long index;
+
+	xe_gt_assert(guc_to_gt(guc), vf_recovery(guc));
+
+	mutex_lock(&guc->submission_state.lock);
+	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
+		guc_exec_queue_unpause_prepare(guc, q);
+	mutex_unlock(&guc->submission_state.lock);
+}
+
+static void guc_exec_queue_unpause(struct xe_guc *guc, struct xe_exec_queue *q)
+{
+	struct xe_gpu_scheduler *sched = &q->guc->sched;
+	struct xe_sched_msg *msg;
+	bool needs_tdr = exec_queue_killed_or_banned_or_wedged(q);
+
+	lockdep_assert_held(&guc->submission_state.lock);
+
+	xe_sched_resubmit_jobs(sched);
+
+	if (q->guc->needs_cleanup) {
+		msg = q->guc->static_msgs + STATIC_MSG_CLEANUP;
+
+		guc_exec_queue_add_msg(q, msg, CLEANUP);
+		q->guc->needs_cleanup = false;
+	}
+
+	if (q->guc->needs_suspend) {
+		msg = q->guc->static_msgs + STATIC_MSG_SUSPEND;
+
+		xe_sched_msg_lock(sched);
+		guc_exec_queue_try_add_msg_head(q, msg, SUSPEND);
+		xe_sched_msg_unlock(sched);
+
+		q->guc->needs_suspend = false;
+	}
+
+	/*
+	 * The resume must be in the message queue before the suspend as it is
+	 * not possible for a resume to be issued if a suspend pending is, but
+	 * the inverse is possible.
+	 */
+	if (q->guc->needs_resume) {
+		msg = q->guc->static_msgs + STATIC_MSG_RESUME;
+
+		xe_sched_msg_lock(sched);
+		guc_exec_queue_try_add_msg_head(q, msg, RESUME);
+		xe_sched_msg_unlock(sched);
+
+		q->guc->needs_resume = false;
+	}
  	xe_sched_submission_start(sched);
+	if (needs_tdr)
+		xe_guc_exec_queue_trigger_cleanup(q);
+	xe_sched_submission_resume_tdr(sched);
  }
  /**
@@ -2339,10 +2534,10 @@ void xe_guc_submit_unpause(struct xe_guc *guc)
  	struct xe_exec_queue *q;
  	unsigned long index;
+	mutex_lock(&guc->submission_state.lock);
  	xa_for_each(&guc->submission_state.exec_queue_lookup, index, q)
-		guc_exec_queue_unpause(q);
-
-	wake_up_all(&guc->ct.wq);
+		guc_exec_queue_unpause(guc, q);
+	mutex_unlock(&guc->submission_state.lock);
  }
  /**
drivers/gpu/drm/xe/xe_guc_submit.h +1 -0
index fe82c317048e..b49a2748ec46 100644
@@ -22,6 +22,7 @@ void xe_guc_submit_stop(struct xe_guc *guc);
  int xe_guc_submit_start(struct xe_guc *guc);
  void xe_guc_submit_pause(struct xe_guc *guc);
  void xe_guc_submit_unpause(struct xe_guc *guc);
+void xe_guc_submit_unpause_prepare(struct xe_guc *guc);
  void xe_guc_submit_pause_abort(struct xe_guc *guc);
  void xe_guc_submit_wedge(struct xe_guc *guc);
drivers/gpu/drm/xe/xe_sched_job_types.h +4 -0
index 7ce58765a34a..13e7a12b03ad 100644
@@ -63,6 +63,10 @@ struct xe_sched_job {
  	bool ring_ops_flush_tlb;
  	/** @ggtt: mapped in ggtt. */
  	bool ggtt;
+	/** @skip_emit: skip emitting the job */
+	bool skip_emit;
+	/** @last_replay: last job being replayed */
+	bool last_replay;
  	/** @ptrs: per instance pointers. */
  	struct xe_job_ptrs ptrs[];
  };
author maintainer
#76 lore raw

PATCH v2 27/34 drm/xe: Move queue init before LRC creation

In reply to Matthew Brost

A queue must be in the submission backend's tracking state before the LRC is created to avoid a race condition where the LRC's GGTT addresses are not properly fixed up during VF post-migration recovery.

Move the queue initialization—which adds the queue to the submission backend's tracking state—before LRC creation.

v2:
- Wait on VF GGTT fixes before creating LRC (testing)

---
 drivers/gpu/drm/xe/xe_exec_queue.c        | 43 +++++++++++++++++------
 drivers/gpu/drm/xe/xe_execlist.c          |  2 +-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 39 +++++++++++++++++++-
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  2 ++
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  5 +++
 drivers/gpu/drm/xe/xe_guc_submit.c        |  2 +-
 drivers/gpu/drm/xe/xe_lrc.h               | 10 ++++++
 7 files changed, 90 insertions(+), 13 deletions(-)
drivers/gpu/drm/xe/xe_exec_queue.c +33 -10
index 81f707d2c388..3db8e64d9d13 100644
@@ -15,6 +15,7 @@
 #include "xe_dep_scheduler.h"
 #include "xe_device.h"
 #include "xe_gt.h"
+#include "xe_gt_sriov_vf.h"
 #include "xe_hw_engine_class_sysfs.h"
 #include "xe_hw_engine_group.h"
 #include "xe_hw_fence.h"
@@ -179,17 +180,32 @@ static int __xe_exec_queue_init(struct xe_exec_queue *q)
 			flags |= XE_LRC_CREATE_RUNALONE;
 	}
 
+	err = q->ops->init(q);
+	if (err)
+		return err;
+
+	/*
+	 * This must occur after q->ops->init to avoid race conditions during VF
+	 * post-migration recovery, as the fixups for the LRC GGTT addresses
+	 * depend on the queue being present in the backend tracking structure.
+	 *
+	 * In addition to above, we must wait on inflight GGTT changes to
+	 * avoid writing out stale values here.
+	 */
+	xe_gt_sriov_vf_wait_valid_ggtt(q->gt);
 	for (i = 0; i < q->width; ++i) {
-		q->lrc[i] = xe_lrc_create(q->hwe, q->vm, SZ_16K, q->msix_vec, flags);
-		if (IS_ERR(q->lrc[i])) {
-			err = PTR_ERR(q->lrc[i]);
+		struct xe_lrc *lrc;
+
+		lrc = xe_lrc_create(q->hwe, q->vm, xe_lrc_ring_size(),
+				    q->msix_vec, flags);
+		if (IS_ERR(lrc)) {
+			err = PTR_ERR(lrc);
 			goto err_lrc;
 		}
-	}
 
-	err = q->ops->init(q);
-	if (err)
-		goto err_lrc;
+		/* Pairs with READ_ONCE to xe_exec_queue_contexts_hwsp_rebase */
+		WRITE_ONCE(q->lrc[i], lrc);
+	}
 
 	return 0;
 
@@ -1095,9 +1111,16 @@ int xe_exec_queue_contexts_hwsp_rebase(struct xe_exec_queue *q, void *scratch)
 	int err = 0;
 
 	for (i = 0; i < q->width; ++i) {
-		xe_lrc_update_memirq_regs_with_address(q->lrc[i], q->hwe, scratch);
-		xe_lrc_update_hwctx_regs_with_address(q->lrc[i]);
-		err = xe_lrc_setup_wa_bb_with_scratch(q->lrc[i], q->hwe, scratch);
+		struct xe_lrc *lrc;
+
+		/* Pairs with WRITE_ONCE in __xe_exec_queue_init  */
+		lrc = READ_ONCE(q->lrc[i]);
+		if (!lrc)
+			continue;
+
+		xe_lrc_update_memirq_regs_with_address(lrc, q->hwe, scratch);
+		xe_lrc_update_hwctx_regs_with_address(lrc);
+		err = xe_lrc_setup_wa_bb_with_scratch(lrc, q->hwe, scratch);
 		if (err)
 			break;
 	}
drivers/gpu/drm/xe/xe_execlist.c +1 -1
index f83d421ac9d3..769d05517f93 100644
@@ -339,7 +339,7 @@ static int execlist_exec_queue_init(struct xe_exec_queue *q)
 	const struct drm_sched_init_args args = {
 		.ops = &drm_sched_ops,
 		.num_rqs = 1,
-		.credit_limit = q->lrc[0]->ring.size / MAX_JOB_SIZE_BYTES,
+		.credit_limit = xe_lrc_ring_size() / MAX_JOB_SIZE_BYTES,
 		.hang_limit = XE_SCHED_HANG_LIMIT,
 		.timeout = XE_SCHED_JOB_TIMEOUT,
 		.name = q->hwe->name,
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +38 -1
index 91e7dbe80ab2..5e70f3f93d13 100644
@@ -482,6 +482,11 @@ static int vf_get_ggtt_info(struct xe_gt *gt, bool recovery)
 				 shift, config->ggtt_base);
 		xe_tile_sriov_vf_fixup_ggtt_nodes(gt_to_tile(gt), shift);
 	}
+
+	WRITE_ONCE(gt->sriov.vf.migration.ggtt_need_fixes, false);
+	smp_wmb();	/* Ensure above write visible before wake */
+	wake_up_all(&gt->sriov.vf.migration.wq);
+
 out:
 	up_write(config->lock);
 	return err;
@@ -820,7 +825,8 @@ static void xe_gt_sriov_vf_start_migration_recovery(struct xe_gt *gt)
 	    !gt->sriov.vf.migration.recovery_teardown) {
 		gt->sriov.vf.migration.recovery_queued = true;
 		WRITE_ONCE(gt->sriov.vf.migration.recovery_inprogress, true);
-		smp_wmb();	/* Ensure above write visable before wake */
+		WRITE_ONCE(gt->sriov.vf.migration.ggtt_need_fixes, true);
+		smp_wmb();	/* Ensure above writes visable before wake */
 
 		wake_up_all(&gt->uc.guc.ct.wq);
 
@@ -1339,6 +1345,7 @@ int xe_gt_sriov_vf_migration_init_early(struct xe_gt *gt)
 		&tile->primary_gt->sriov.vf.self_config.__lock;
 	spin_lock_init(&gt->sriov.vf.migration.lock);
 	INIT_WORK(&gt->sriov.vf.migration.worker, migration_worker_func);
+	init_waitqueue_head(&gt->sriov.vf.migration.wq);
 
 	if (!xe_sriov_vf_migration_supported(gt_to_xe(gt)))
 		xe_gt_sriov_info(gt, "migration not supported by this module version\n");
@@ -1361,3 +1368,33 @@ bool xe_gt_sriov_vf_recovery_inprogress(struct xe_gt *gt)
 		(xe_memirq_vf_recovery_irq_pending(memirq, &gt->uc.guc) ||
 		 READ_ONCE(gt->sriov.vf.migration.recovery_inprogress));
 }
+
+static bool vf_valid_ggtt(struct xe_gt *gt)
+{
+	struct xe_memirq *memirq = &gt_to_tile(gt)->memirq;
+
+	xe_gt_assert(gt, IS_SRIOV_VF(gt_to_xe(gt)));
+
+	if (xe_memirq_vf_recovery_irq_pending(memirq, &gt->uc.guc) ||
+	    READ_ONCE(gt->sriov.vf.migration.ggtt_need_fixes))
+		return false;
+
+	return true;
+}
+
+/**
+ * xe_gt_sriov_vf_wait_valid_ggtt() - VF wait for valid GGTT addresses
+ * @gt: the &xe_gt
+ */
+void xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt)
+{
+	int ret;
+
+	if (!IS_SRIOV_VF(gt_to_xe(gt)))
+		return;
+
+	ret = wait_event_interruptible_timeout(gt->sriov.vf.migration.wq,
+					       vf_valid_ggtt(gt),
+					       HZ * 5);
+	XE_WARN_ON(!ret);
+}
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index 535237003915..d5b53ba1d4ca 100644
@@ -39,4 +39,6 @@ void xe_gt_sriov_vf_print_config(struct xe_gt *gt, struct drm_printer *p);
 void xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p);
 void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p);
 
+void xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt);
+
 #endif
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +5 -0
index beb9978336bb..79bb9d5d62bd 100644
@@ -8,6 +8,7 @@
 
 #include <linux/rwsem.h>
 #include <linux/types.h>
+#include <linux/wait.h>
 #include <linux/workqueue.h>
 #include "xe_uc_fw_types.h"
 
@@ -61,6 +62,8 @@ struct xe_gt_sriov_vf_migration {
 	struct work_struct worker;
 	/** @lock: Protects recovery_queued, teardown */
 	spinlock_t lock;
+	/** @wq: wait queue for migration fixes */
+	wait_queue_head_t wq;
 	/** @lrc_wa_bb: Scratch memory for LRC WA BB in recovery */
 	void *lrc_wa_bb;
 	/** @recovery_teardown: VF post migration recovery is being torn down */
@@ -69,6 +72,8 @@ struct xe_gt_sriov_vf_migration {
 	bool recovery_queued;
 	/** @recovery_inprogress: VF post migration recovery in progress */
 	bool recovery_inprogress;
+	/** @ggtt_need_fixes: VF GGTT needs fixes */
+	bool ggtt_need_fixes;
 };
 
 /**
drivers/gpu/drm/xe/xe_guc_submit.c +1 -1
index b112a4a91a5b..f01f2d6f97ed 100644
@@ -1938,7 +1938,7 @@ static int guc_exec_queue_init(struct xe_exec_queue *q)
 	timeout = (q->vm && xe_vm_in_lr_mode(q->vm)) ? MAX_SCHEDULE_TIMEOUT :
 		  msecs_to_jiffies(q->sched_props.job_timeout_ms);
 	err = xe_sched_init(&ge->sched, &drm_sched_ops, &xe_sched_ops,
-			    NULL, q->lrc[0]->ring.size / MAX_JOB_SIZE_BYTES, 64,
+			    NULL, xe_lrc_ring_size() / MAX_JOB_SIZE_BYTES, 64,
 			    timeout, guc_to_gt(guc)->ordered_wq, NULL,
 			    q->name, gt_to_xe(q->gt)->drm.dev);
 	if (err)
drivers/gpu/drm/xe/xe_lrc.h +10 -0
index 188565465779..5fb6c74bdab5 100644
@@ -74,6 +74,16 @@ static inline void xe_lrc_put(struct xe_lrc *lrc)
 	kref_put(&lrc->refcount, xe_lrc_destroy);
 }
 
+/**
+ * xe_lrc_ring_size() - Xe LRC ring size
+ *
+ * Return: Size of LRC size
+ */
+static inline size_t xe_lrc_ring_size(void)
+{
+	return SZ_16K;
+}
+
 size_t xe_gt_lrc_size(struct xe_gt *gt, enum xe_engine_class class);
 u32 xe_lrc_pphwsp_offset(struct xe_lrc *lrc);
 u32 xe_lrc_regs_offset(struct xe_lrc *lrc);
Signature · 1 line
2.34.1
author maintainer
#77 lore raw

PATCH v2 28/34 drm/xe/vf: Add debug prints for GuC replaying state during VF recovery

In reply to Matthew Brost

Helpful to manually verify the GuC state machine can correctly replay the state during a VF post-migration recovery. All replay paths have been manually verified as triggered and working during testing.

---
 drivers/gpu/drm/xe/xe_guc_submit.c | 18 +++++++++++++++++-
 1 file changed, 17 insertions(+), 1 deletion(-)
drivers/gpu/drm/xe/xe_guc_submit.c +17 -1
index f01f2d6f97ed..496cd1f2d83f 100644
@@ -2316,14 +2316,19 @@ static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)
 	pending_enable = exec_queue_pending_enable(q);
 	pending_resume = exec_queue_pending_resume(q);
 
-	if (pending_enable && pending_resume)
+	if (pending_enable && pending_resume) {
 		q->guc->needs_resume = true;
+		xe_gt_dbg(guc_to_gt(guc), "Replay RESUME - guc_id=%d",
+			  q->guc->id);
+	}
 
 	if (pending_enable && !pending_resume &&
 	    !exec_queue_pending_tdr_exit(q)) {
 		clear_exec_queue_registered(q);
 		if (xe_exec_queue_is_lr(q))
 			xe_exec_queue_put(q);
+		xe_gt_dbg(guc_to_gt(guc), "Replay REGISTER - guc_id=%d",
+			  q->guc->id);
 	}
 
 	if (pending_enable) {
@@ -2331,6 +2336,8 @@ static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)
 		clear_exec_queue_pending_resume(q);
 		clear_exec_queue_pending_tdr_exit(q);
 		clear_exec_queue_pending_enable(q);
+		xe_gt_dbg(guc_to_gt(guc), "Replay ENABLE - guc_id=%d",
+			  q->guc->id);
 	}
 
 	if (exec_queue_destroyed(q) && exec_queue_registered(q)) {
@@ -2340,6 +2347,8 @@ static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)
 		else
 			q->guc->needs_cleanup = true;
 		clear_exec_queue_extra_ref(q);
+		xe_gt_dbg(guc_to_gt(guc), "Replay CLEANUP - guc_id=%d",
+			  q->guc->id);
 	}
 
 	pending_disable = exec_queue_pending_disable(q);
@@ -2347,6 +2356,8 @@ static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)
 	if (pending_disable && exec_queue_suspended(q)) {
 		clear_exec_queue_suspended(q);
 		q->guc->needs_suspend = true;
+		xe_gt_dbg(guc_to_gt(guc), "Replay SUSPEND - guc_id=%d",
+			  q->guc->id);
 	}
 
 	if (pending_disable) {
@@ -2354,6 +2365,8 @@ static void guc_exec_queue_pause(struct xe_guc *guc, struct xe_exec_queue *q)
 			set_exec_queue_enabled(q);
 		clear_exec_queue_pending_disable(q);
 		clear_exec_queue_check_timeout(q);
+		xe_gt_dbg(guc_to_gt(guc), "Replay DISABLE - guc_id=%d",
+			  q->guc->id);
 	}
 
 	q->guc->resume_time = 0;
@@ -2452,6 +2465,9 @@ static void guc_exec_queue_unpause_prepare(struct xe_guc *guc,
 	list_for_each_entry(s_job, &sched->base.pending_list, list) {
 		job = to_xe_sched_job(s_job);
 
+		xe_gt_dbg(guc_to_gt(guc), "Replay JOB - guc_id=%d, seqno=%d",
+			  q->guc->id, xe_sched_job_seqno(job));
+
 		q->ring_ops->emit_job(job);
 		job->skip_emit = true;
 	}
Signature · 1 line
2.34.1
author maintainer
#78 lore raw

PATCH v2 29/34 drm/xe/vf: Workaround for race condition in GuC firmware during VF pause

In reply to Matthew Brost

A race condition exists where a paused VF's H2G request can be processed and subsequently rejected. This rejection results in a FAST_REQ failure being delivered to the KMD, which then terminates the CT via a dead worker and triggers a GT reset—an undesirable outcome.

This workaround mitigates the issue by checking if a VF post-migration recovery is in progress and aborting these adverse actions accordingly. The GuC firmware will address this bug in an upcoming release. Once that version is available and VF migration depends on it, this workaround can be safely removed.

---
 drivers/gpu/drm/xe/xe_guc_ct.c | 4 ++++
 1 file changed, 4 insertions(+)
drivers/gpu/drm/xe/xe_guc_ct.c +4 -0
index fa55e9c9de3a..794028c9d4b3 100644
@@ -1391,6 +1391,10 @@ static int parse_g2h_response(struct xe_guc_ct *ct, u32 *msg, u32 len)
 
 		fast_req_report(ct, fence);
 
+		/* FIXME: W/A race in the GuC, will get in firmware soon */
+		if (xe_gt_sriov_vf_recovery_inprogress(gt))
+			return 0;
+
 		CT_DEAD(ct, NULL, PARSE_G2H_RESPONSE);
 
 		return -EPROTO;
Signature · 1 line
2.34.1
author maintainer
#79 lore raw

PATCH v2 30/34 drm/xe: Use PPGTT addresses for TLB invalidation to avoid GGTT fixups

In reply to Matthew Brost

From: Satyanarayana K V P <[email protected]>

The migrate VM builds the CCS metadata save/restore batch buffer (BB) in advance and retains it so the GuC can submit it directly when saving a VM’s state.

When a VM migrates between VFs, the GGTT base can change. Any GGTT-based addresses embedded in the BB would then have to be parsed and patched.

Use PPGTT addresses in the BB (including for TLB invalidation) so the BB remains GGTT-agnostic and requires no address fixups during migration.

---
 drivers/gpu/drm/xe/xe_migrate.c | 28 ++++++++++++++++++++--------
 1 file changed, 20 insertions(+), 8 deletions(-)
drivers/gpu/drm/xe/xe_migrate.c +20 -8
index 1d667fa36cf3..ad03afb5145f 100644
@@ -980,15 +980,27 @@ struct xe_lrc *xe_migrate_lrc(struct xe_migrate *migrate)
 	return migrate->q->lrc[0];
 }
 
-static int emit_flush_invalidate(struct xe_exec_queue *q, u32 *dw, int i,
-				 u32 flags)
+static u64 migrate_vm_ppgtt_addr_tlb_inval(void)
 {
-	struct xe_lrc *lrc = xe_exec_queue_lrc(q);
+	/*
+	 * The migrate VM is self-referential so it can modify its own PTEs (see
+	 * pte_update_size() or emit_pte() functions). We reserve NUM_KERNEL_PDE
+	 * entries for kernel operations (copies, clears, CCS migrate), and
+	 * suballocate the rest to user operations (binds/unbinds). With
+	 * NUM_KERNEL_PDE = 15, NUM_KERNEL_PDE - 1 is already used for PTE updates,
+	 * so assign NUM_KERNEL_PDE - 2 for TLB invalidation.
+	 */
+	return (NUM_KERNEL_PDE - 2) * XE_PAGE_SIZE;
+}
+
+static int emit_flush_invalidate(u32 *dw, int i, u32 flags)
+{
+	u64 addr = migrate_vm_ppgtt_addr_tlb_inval();
+
 	dw[i++] = MI_FLUSH_DW | MI_INVALIDATE_TLB | MI_FLUSH_DW_OP_STOREDW |
 		  MI_FLUSH_IMM_DW | flags;
-	dw[i++] = lower_32_bits(xe_lrc_start_seqno_ggtt_addr(lrc)) |
-		  MI_FLUSH_DW_USE_GTT;
-	dw[i++] = upper_32_bits(xe_lrc_start_seqno_ggtt_addr(lrc));
+	dw[i++] = lower_32_bits(addr);
+	dw[i++] = upper_32_bits(addr);
 	dw[i++] = MI_NOOP;
 	dw[i++] = MI_NOOP;
 
@@ -1101,11 +1113,11 @@ int xe_migrate_ccs_rw_copy(struct xe_tile *tile, struct xe_exec_queue *q,
 
 		emit_pte(m, bb, ccs_pt, false, false, &ccs_it, ccs_size, src);
 
-		bb->len = emit_flush_invalidate(q, bb->cs, bb->len, flush_flags);
+		bb->len = emit_flush_invalidate(bb->cs, bb->len, flush_flags);
 		flush_flags = xe_migrate_ccs_copy(m, bb, src_L0_ofs, src_is_pltt,
 						  src_L0_ofs, dst_is_pltt,
 						  src_L0, ccs_ofs, true);
-		bb->len = emit_flush_invalidate(q, bb->cs, bb->len, flush_flags);
+		bb->len = emit_flush_invalidate(bb->cs, bb->len, flush_flags);
 
 		size -= src_L0;
 	}
Signature · 1 line
2.34.1