Skip to contentSkip to conversation

drm/xe/vf: Post-migration recovery of GGTT nodes and CTB

PATCH Superseded Tomasz Lis 2 participants 14 messages Mar 31, 2025 – Apr 3, 2025

Subsystems (2)

v6 of 14 Superseded Latest is v14 →

lore
Share & tools
lore

Tomasz Lis

author
#0 lore raw

v6 cover drm/xe/vf: Post-migration recovery of GGTT nodes and CTB

To support VF Migration, it is necessary to do fixups to any non-virtualized resources. These fixups need to be applied within VM, on the KMD working with VF.

This series adds two fixup functions to the recovery worker:
* for fixing drm_mm nodes which represent GGTT allocations
* for fixing content of outgoing CTB buffer

v2: Fixed missing include, made checkpatch happy
v3: Moved some functs to xe_ggtt.c; moved shift computation to just
after querying; improved documentation; switched some warns to asserts; skipping fixups when GGTT shift eq 0; iterating through tiles rather than gts; fixes in log messages
v4: Updated kerneldocs, removed unused funct, properly allocate
balloning nodes if non existent, removed inline keywords, enums for offsets in CTB messages, less error messages, if return unused then made functs void
v5: Removed drm_mm change, but added VF init modifications. These then
allowed to re-use ballooning functions during ggtt node fixing.
v6: Minor update - fixed some misplaced funct declarations

Tomasz Lis (4):
drm/xe/vf: Divide ballooning into allocation and insertion drm/xe/sriov: Shifting GGTT area post migration drm/xe/guc: Introduce enum with offsets for multi-LRC register H2G drm/xe/vf: Fixup CTB send buffer messages after migration

 drivers/gpu/drm/xe/abi/guc_actions_abi.h  |   7 ++
 drivers/gpu/drm/xe/xe_ggtt.c              |  40 ++++--
 drivers/gpu/drm/xe/xe_ggtt.h              |   1 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 147 +++++++++++++++++++---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   2 +
 drivers/gpu/drm/xe/xe_guc_ct.c            | 142 +++++++++++++++++++++
 drivers/gpu/drm/xe/xe_guc_ct.h            |   2 +
 drivers/gpu/drm/xe/xe_guc_submit.c        |   4 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          |  42 +++++++
 10 files changed, 368 insertions(+), 22 deletions(-)
Signature · 1 line
2.25.1
author
#1 lore raw

PATCH v6 1/4 drm/xe/vf: Divide ballooning into allocation and insertion

The balloon nodes used to fill areas of GGTT inaccessible for a specific VF, were allocaten and inserted into GGTT within one function. This disallowed re-using the insertion part during VF migration recovery.

This patch separates allocation (init/fini functs) from the insertion of balloons (balloon/deballoon functs). Locks are also moved to ensure calls from post-migration recovery worker will not cause a deadlock.

v2: Moved declarations to proper header

---
 drivers/gpu/drm/xe/xe_ggtt.c        |  9 +---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 64 ++++++++++++++++++++++-------
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h |  2 +
 3 files changed, 52 insertions(+), 23 deletions(-)
drivers/gpu/drm/xe/xe_ggtt.c +1 -8
index 5fcb2b4c2c13..2d7456e37ef4 100644
@@ -452,9 +452,7 @@ int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node, u64 start, u64 end)
 	node->base.start = start;
 	node->base.size = end - start;
 
-	mutex_lock(&ggtt->lock);
 	err = drm_mm_reserve_node(&ggtt->mm, &node->base);
-	mutex_unlock(&ggtt->lock);
 
 	if (xe_gt_WARN(ggtt->tile->primary_gt, err,
 		       "Failed to balloon GGTT %#llx-%#llx (%pe)\n",
@@ -477,16 +475,11 @@ void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node)
 		return;
 
 	if (!drm_mm_node_allocated(&node->base))
-		goto free_node;
+		return;
 
 	xe_ggtt_dump_node(node->ggtt, &node->base, "remove-balloon");
 
-	mutex_lock(&node->ggtt->lock);
 	drm_mm_remove_node(&node->base);
-	mutex_unlock(&node->ggtt->lock);
-
-free_node:
-	xe_ggtt_node_fini(node);
 }
 
 /**
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +49 -15

124 lines not on this page · raw message

drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index ba6c5d74e326..c87b0e9c7ebc 100644
@@ -18,6 +18,8 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
 int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
 int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt);
+int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt);
+void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
Signature · 1 line
2.25.1
#2 lore raw

Re: [PATCH v6 1/4] drm/xe/vf: Divide ballooning into allocation and insertion

Hi Tomasz,

Since we use 'ballooning' concept for GGTT, please include 'GGTT' in the title to make it more specific

Quoted 2 lines · Tomasz Lis

The balloon nodes used to fill areas of GGTT inaccessible for a specific VF, were allocaten and inserted into GGTT within

typo

Quoted 2 lines

one function. This disallowed re-using the insertion part during VF migration recovery.

The wording "This disallowed re-using" sounds like there is a bug in current implementation, while all we want is to refactor the code, to make if more reusable, so IMO we can just drop this sentence, and just add in the one below that the split is for the "reuse".

Quoted 4 lines

This patch separates allocation (init/fini functs) from the insertion of balloons (balloon/deballoon functs). Locks are also moved to ensure calls from post-migration recovery worker will not cause a deadlock.

hmm, this new locking is not looking as robust as before ... see below

Quoted 15 lines

v2: Moved declarations to proper header

---
 drivers/gpu/drm/xe/xe_ggtt.c        |  9 +---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 64 ++++++++++++++++++++++-------
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h |  2 +
 3 files changed, 52 insertions(+), 23 deletions(-)
drivers/gpu/drm/xe/xe_ggtt.c +0 -0
index 5fcb2b4c2c13..2d7456e37ef4 100644

drivers/gpu/drm/xe/xe_ggtt.c int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node, u64 start, u64 end) · line 455

Quoted 4 lines
drivers/gpu/drm/xe/xe_ggtt.c +0 -1
@@ -452,9 +452,7 @@ int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node, u64 start, u64 end)
 	node->base.start = start;
 	node->base.size = end - start;
 
-	mutex_lock(&ggtt->lock);

Functions that don't take the lock on it's own and expect lock to be already taken we usually name with "_locked" suffix and use explicit

lockdep_assert_held

to make sure it is enforced

also consider whether there should be just two variants of the function defined in xe_ggtt:

	xe_ggtt_node_insert_balloon_locked()
	{
		lockdep_assert_held
		...
	}

	xe_ggtt_node_insert_balloon()
	{
		mutex_lock
		xe_ggtt_node_insert_balloon_locked()
		mutex_unlock
	}
Quoted 7 lines
drivers/gpu/drm/xe/xe_ggtt.c +0 -1
 	err = drm_mm_reserve_node(&ggtt->mm, &node->base);
-	mutex_unlock(&ggtt->lock);
 
 	if (xe_gt_WARN(ggtt->tile->primary_gt, err,
 		       "Failed to balloon GGTT %#llx-%#llx (%pe)\n",
@@ -477,16 +475,11 @@ void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node)
 		return;

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 486

Quoted 10 lines
drivers/gpu/drm/xe/xe_ggtt.c +1 -3
 
 	if (!drm_mm_node_allocated(&node->base))
-		goto free_node;
+		return;
 
 	xe_ggtt_dump_node(node->ggtt, &node->base, "remove-balloon");
 
-	mutex_lock(&node->ggtt->lock);
 	drm_mm_remove_node(&node->base);
-	mutex_unlock(&node->ggtt->lock);

same here

	xe_ggtt_node_remove_balloon_locked()
	{
		lockdep_assert_held
		...
	}

	xe_ggtt_node_remove_balloon()
	{
		mutex_lock
		xe_ggtt_node_remove_balloon_locked()
		mutex_unlock
	}
Quoted 11 lines
drivers/gpu/drm/xe/xe_ggtt.c +0 -3
-
-free_node:
-	xe_ggtt_node_fini(node);
 }
 
 /**
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index a439261bf4d7..9edbe34f45f4 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 565

Quoted 8 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -2
@@ -560,26 +560,33 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
 	return gt->sriov.vf.self_config.lmem_size;
 }
 
-static struct xe_ggtt_node *
-vf_balloon_ggtt_node(struct xe_ggtt *ggtt, u64 start, u64 end)
+static int
+vf_balloon_ggtt_node(struct xe_ggtt *ggtt, struct xe_ggtt_node *node,
+		     u64 start, u64 end)

as this function is now just a simple wrapper around xe_ggtt_node_insert_balloon then maybe it's not needed and caller may use xe_ggtt_node_insert_balloon directly?

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 569

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -2
 {
-	struct xe_ggtt_node *node;
 	int err;
 
-	node = xe_ggtt_node_init(ggtt);
 	if (IS_ERR(node))

this runtime check shouldn't be needed any more as any node_init() failure shall trigger probe abort

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -2
-		return node;
+		return PTR_ERR(node);
 
 	err = xe_ggtt_node_insert_balloon(node, start, end);
 	if (err) {
-		xe_ggtt_node_fini(node);

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 580

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -3
-		return ERR_PTR(err);
+		return err;
 	}
 
-	return node;
+	return 0;
 }
 
-static int vf_balloon_ggtt(struct xe_gt *gt)
+static void xe_gt_sriov_vf_balloon_init(struct xe_gt *gt)

please don't use fully qualified names for static functions here it could be as simple as:

vf_init_ggtt_balloons()

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 586

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -0
+{
+	struct xe_tile *tile = gt_to_tile(gt);
+	struct xe_ggtt *ggtt = tile->mem.ggtt;
+
+	tile->sriov.vf.ggtt_balloon[0] = xe_ggtt_node_init(ggtt);
+	tile->sriov.vf.ggtt_balloon[1] = xe_ggtt_node_init(ggtt);

failure in any of xe_ggtt_node_init() is fatal from the VF POV and shall trigger immediate probe abort

btw, maybe it's good time to add new function variant:

xe_ggtt_node_init_managed()

to make sure we don't forget to call xe_ggtt_node_fini() ?

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 589

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+}
+
+int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt)

all new public functions shall have proper kernel-doc

Quoted 30 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -2
 {
 	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
 	struct xe_tile *tile = gt_to_tile(gt);
@@ -589,6 +596,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt)
 
 	xe_gt_assert(gt, IS_SRIOV_VF(xe));
 	xe_gt_assert(gt, !xe_gt_is_media_type(gt));
+	lockdep_assert_held(&tile->mem.ggtt->lock);
 
 	if (!config->ggtt_size)
 		return -ENODATA;
@@ -611,7 +619,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt)
 	start = xe_wopcm_size(xe);
 	end = config->ggtt_base;
 	if (end != start) {
-		tile->sriov.vf.ggtt_balloon[0] = vf_balloon_ggtt_node(ggtt, start, end);
+		vf_balloon_ggtt_node(ggtt, tile->sriov.vf.ggtt_balloon[0], start, end);
 		if (IS_ERR(tile->sriov.vf.ggtt_balloon[0]))
 			return PTR_ERR(tile->sriov.vf.ggtt_balloon[0]);
 	}
@@ -619,7 +627,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt)
 	start = config->ggtt_base + config->ggtt_size;
 	end = GUC_GGTT_TOP;
 	if (end != start) {
-		tile->sriov.vf.ggtt_balloon[1] = vf_balloon_ggtt_node(ggtt, start, end);
+		vf_balloon_ggtt_node(ggtt, tile->sriov.vf.ggtt_balloon[1], start, end);
 		if (IS_ERR(tile->sriov.vf.ggtt_balloon[1])) {
 			xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[0]);
 			return PTR_ERR(tile->sriov.vf.ggtt_balloon[1]);

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_balloon_ggtt(struct xe_gt *gt) · line 640

Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -1
@@ -629,15 +637,34 @@ static int vf_balloon_ggtt(struct xe_gt *gt)
 	return 0;
 }
 
-static void deballoon_ggtt(struct drm_device *drm, void *arg)
+void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt)

if this is a public function now, then kernel-doc is required

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_balloon_ggtt(struct xe_gt *gt) · line 645

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
 {
-	struct xe_tile *tile = arg;
+	struct xe_tile *tile = gt_to_tile(gt);
 
 	xe_tile_assert(tile, IS_SRIOV_VF(tile_to_xe(tile)));
+	lockdep_assert_held(&tile->mem.ggtt->lock);

why here?
it should be already part of the xe_ggtt_node_remove_balloon()

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_balloon_ggtt(struct xe_gt *gt) · line 650

Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
 	xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[1]);
 	xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[0]);
 }
 
+static void xe_gt_sriov_vf_balloon_fini(struct xe_gt *gt)

don't use fully qualified names for static functions look around how everything else is named!

Quoted 29 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +18 -0
+{
+	struct xe_tile *tile = gt_to_tile(gt);
+
+	xe_ggtt_node_fini(tile->sriov.vf.ggtt_balloon[1]);
+	xe_ggtt_node_fini(tile->sriov.vf.ggtt_balloon[0]);
+}
+
+static void deballoon_ggtt(struct drm_device *drm, void *arg)
+{
+	struct xe_tile *tile = arg;
+
+	mutex_lock(&tile->mem.ggtt->lock);
+	xe_gt_sriov_vf_deballoon_ggtt(tile->primary_gt);
+	mutex_unlock(&tile->mem.ggtt->lock);
+	xe_gt_sriov_vf_balloon_fini(tile->primary_gt);
+}
+
 /**
  * xe_gt_sriov_vf_prepare_ggtt - Prepare a VF's GGTT configuration.
  * @gt: the &xe_gt
@@ -650,14 +677,21 @@ int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt)
 {
 	struct xe_tile *tile = gt_to_tile(gt);
 	struct xe_device *xe = tile_to_xe(tile);
+	struct xe_ggtt *ggtt = tile->mem.ggtt;
 	int err;
 
 	if (xe_gt_is_media_type(gt))
 		return 0;

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt) · line 692

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7 -2
 
-	err = vf_balloon_ggtt(gt);
-	if (err)
+	xe_gt_sriov_vf_balloon_init(gt);
+
+	mutex_lock(&ggtt->lock);
+	err = xe_gt_sriov_vf_balloon_ggtt(gt);
+	mutex_unlock(&ggtt->lock);
+	if (err) {
+		xe_gt_sriov_vf_balloon_fini(gt);

since you have split ballooning into two parts, both parts deserve it's own cleanup "actions" to avoid any mistakes in manual cleanup sequence like this (or we can rely on the new xe_ggtt_node_init_managed if added)

Quoted 18 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
 		return err;
+	}
 
 	return drmm_add_action_or_reset(&xe->drm, deballoon_ggtt, tile);
 }
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index ba6c5d74e326..c87b0e9c7ebc 100644
@@ -18,6 +18,8 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
 int xe_gt_sriov_vf_connect(struct xe_gt *gt);
 int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
 int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt);
+int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt);
+void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
#3 lore raw

Re: [PATCH v6 1/4] drm/xe/vf: Divide ballooning into allocation and insertion

Quoted 4 lines · Michal Wajdeczko

Hi Tomasz,

Since we use 'ballooning' concept for GGTT, please include 'GGTT' in the title to make it more specific

ack

Quoted 4 lines · Tomasz Lis
Quoted 2 lines

The balloon nodes used to fill areas of GGTT inaccessible for a specific VF, were allocaten and inserted into GGTT within

typo

ack

Quoted 6 lines
Quoted 2 lines

one function. This disallowed re-using the insertion part during VF migration recovery.

The wording "This disallowed re-using" sounds like there is a bug in current implementation, while all we want is to refactor the code, to make if more reusable, so IMO we can just drop this sentence, and just add in the one below that the split is for the "reuse".

I like the division between "general awareness" paragraph and "what this patch does"

paragraph. But I will rephrase.

Quoted 33 lines
Quoted 3 lines

This patch separates allocation (init/fini functs) from the insertion of balloons (balloon/deballoon functs). Locks are also moved to ensure calls from post-migration recovery worker will not cause a deadlock.

hmm, this new locking is not looking as robust as before ... see below

Quoted 14 lines

v2: Moved declarations to proper header

---
  drivers/gpu/drm/xe/xe_ggtt.c        |  9 +---
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c | 64 ++++++++++++++++++++++-------
  drivers/gpu/drm/xe/xe_gt_sriov_vf.h |  2 +
  3 files changed, 52 insertions(+), 23 deletions(-)
drivers/gpu/drm/xe/xe_ggtt.c +0 -0
index 5fcb2b4c2c13..2d7456e37ef4 100644

drivers/gpu/drm/xe/xe_ggtt.c int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node, u64 start, u64 end) · line 455

Quoted 4 lines
drivers/gpu/drm/xe/xe_ggtt.c +0 -1
@@ -452,9 +452,7 @@ int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node, u64 start, u64 end)
  	node->base.start = start;
  	node->base.size = end - start;
 
-	mutex_lock(&ggtt->lock);

Functions that don't take the lock on it's own and expect lock to be already taken we usually name with "_locked" suffix and use explicit

lockdep_assert_held

to make sure it is enforced

also consider whether there should be just two variants of the function defined in xe_ggtt:

Quoted 12 lines
	xe_ggtt_node_insert_balloon_locked()
	{
		lockdep_assert_held
		...
	}

	xe_ggtt_node_insert_balloon()
	{
		mutex_lock
		xe_ggtt_node_insert_balloon_locked()
		mutex_unlock
	}

We will never use a "non-_locked" version at this level.

I think you was going for "_locked" version of xe_gt_sriov_vf_balloon_ggtt()?

I don't think it makes sense from a code flow standpoint, but it is an established convention.. will do.

Quoted 19 lines
Quoted 7 lines
drivers/gpu/drm/xe/xe_ggtt.c +0 -1
  	err = drm_mm_reserve_node(&ggtt->mm, &node->base);
-	mutex_unlock(&ggtt->lock);
 
  	if (xe_gt_WARN(ggtt->tile->primary_gt, err,
  		       "Failed to balloon GGTT %#llx-%#llx (%pe)\n",
@@ -477,16 +475,11 @@ void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node)
  		return;

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 486

Quoted 10 lines
drivers/gpu/drm/xe/xe_ggtt.c +1 -3
 
  	if (!drm_mm_node_allocated(&node->base))
-		goto free_node;
+		return;
 
  	xe_ggtt_dump_node(node->ggtt, &node->base, "remove-balloon");
 
-	mutex_lock(&node->ggtt->lock);
  	drm_mm_remove_node(&node->base);
-	mutex_unlock(&node->ggtt->lock);

same here

Quoted 12 lines
	xe_ggtt_node_remove_balloon_locked()
	{
		lockdep_assert_held
		...
	}

	xe_ggtt_node_remove_balloon()
	{
		mutex_lock
		xe_ggtt_node_remove_balloon_locked()
		mutex_unlock
	}

as before, wrong level.

Quoted 14 lines
Quoted 14 lines
drivers/gpu/drm/xe/xe_ggtt.c +0 -3
-
-free_node:
-	xe_ggtt_node_fini(node);
  }
 
  /**
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index a439261bf4d7..9edbe34f45f4 100644
@@ -560,26 +560,33 @@ u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt)
  	return gt->sriov.vf.self_config.lmem_size;
  }
 
Quoted 8 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 565

Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -2
-static struct xe_ggtt_node *
-vf_balloon_ggtt_node(struct xe_ggtt *ggtt, u64 start, u64 end)
+static int
+vf_balloon_ggtt_node(struct xe_ggtt *ggtt, struct xe_ggtt_node *node,
+		     u64 start, u64 end)

as this function is now just a simple wrapper around xe_ggtt_node_insert_balloon then maybe it's not needed and caller may use xe_ggtt_node_insert_balloon directly?

will do.

Quoted 23 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 569

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -2
  {
-	struct xe_ggtt_node *node;
  	int err;
 
-	node = xe_ggtt_node_init(ggtt);
  	if (IS_ERR(node))

this runtime check shouldn't be needed any more as any node_init() failure shall trigger probe abort

Quoted 14 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -4
-		return node;
+		return PTR_ERR(node);
 
  	err = xe_ggtt_node_insert_balloon(node, start, end);
  	if (err) {
-		xe_ggtt_node_fini(node);
-		return ERR_PTR(err);
+		return err;
  	}
 
-	return node;
+	return 0;
  }
 
Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 580

Quoted 2 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -1
-static int vf_balloon_ggtt(struct xe_gt *gt)
+static void xe_gt_sriov_vf_balloon_init(struct xe_gt *gt)

please don't use fully qualified names for static functions here it could be as simple as:

vf_init_ggtt_balloons()

ok

Quoted 8 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 586

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -0
+{
+	struct xe_tile *tile = gt_to_tile(gt);
+	struct xe_ggtt *ggtt = tile->mem.ggtt;
+
+	tile->sriov.vf.ggtt_balloon[0] = xe_ggtt_node_init(ggtt);
+	tile->sriov.vf.ggtt_balloon[1] = xe_ggtt_node_init(ggtt);

failure in any of xe_ggtt_node_init() is fatal from the VF POV and shall trigger immediate probe abort

will do that.

Quoted 5 lines

btw, maybe it's good time to add new function variant:

xe_ggtt_node_init_managed()

to make sure we don't forget to call xe_ggtt_node_fini() ?

I don't see the point. We do sometimes create "managed" versions (but that word is never added as a suffix),

but it's not really an established standard. Also it doesn't fit here. This brings no improvement.

Quoted 4 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c u64 xe_gt_sriov_vf_lmem(struct xe_gt *gt) · line 589

Quoted 3 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -0
+}
+
+int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt)

all new public functions shall have proper kernel-doc

true.

Quoted 33 lines
Quoted 33 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +3 -2
  {
  	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
  	struct xe_tile *tile = gt_to_tile(gt);
@@ -589,6 +596,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt)
 
  	xe_gt_assert(gt, IS_SRIOV_VF(xe));
  	xe_gt_assert(gt, !xe_gt_is_media_type(gt));
+	lockdep_assert_held(&tile->mem.ggtt->lock);
 
  	if (!config->ggtt_size)
  		return -ENODATA;
@@ -611,7 +619,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt)
  	start = xe_wopcm_size(xe);
  	end = config->ggtt_base;
  	if (end != start) {
-		tile->sriov.vf.ggtt_balloon[0] = vf_balloon_ggtt_node(ggtt, start, end);
+		vf_balloon_ggtt_node(ggtt, tile->sriov.vf.ggtt_balloon[0], start, end);
  		if (IS_ERR(tile->sriov.vf.ggtt_balloon[0]))
  			return PTR_ERR(tile->sriov.vf.ggtt_balloon[0]);
  	}
@@ -619,7 +627,7 @@ static int vf_balloon_ggtt(struct xe_gt *gt)
  	start = config->ggtt_base + config->ggtt_size;
  	end = GUC_GGTT_TOP;
  	if (end != start) {
-		tile->sriov.vf.ggtt_balloon[1] = vf_balloon_ggtt_node(ggtt, start, end);
+		vf_balloon_ggtt_node(ggtt, tile->sriov.vf.ggtt_balloon[1], start, end);
  		if (IS_ERR(tile->sriov.vf.ggtt_balloon[1])) {
  			xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[0]);
  			return PTR_ERR(tile->sriov.vf.ggtt_balloon[1]);
@@ -629,15 +637,34 @@ static int vf_balloon_ggtt(struct xe_gt *gt)
  	return 0;
  }
 
Quoted 3 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_balloon_ggtt(struct xe_gt *gt) · line 640

Quoted 2 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -1
-static void deballoon_ggtt(struct drm_device *drm, void *arg)
+void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt)

if this is a public function now, then kernel-doc is required

yes.

Quoted 8 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_balloon_ggtt(struct xe_gt *gt) · line 645

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +2 -1
  {
-	struct xe_tile *tile = arg;
+	struct xe_tile *tile = gt_to_tile(gt);
 
  	xe_tile_assert(tile, IS_SRIOV_VF(tile_to_xe(tile)));
+	lockdep_assert_held(&tile->mem.ggtt->lock);

why here?
it should be already part of the xe_ggtt_node_remove_balloon()

This is the function which will get the suffix "_locked", so the check here is really mandatory.

Will add to sub-function as well.

Quoted 6 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_balloon_ggtt(struct xe_gt *gt) · line 650

Quoted 5 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
  	xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[1]);
  	xe_ggtt_node_remove_balloon(tile->sriov.vf.ggtt_balloon[0]);
  }
 
+static void xe_gt_sriov_vf_balloon_fini(struct xe_gt *gt)

don't use fully qualified names for static functions

ok

Quoted 32 lines

look around how everything else is named!

Quoted 30 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +18 -0
+{
+	struct xe_tile *tile = gt_to_tile(gt);
+
+	xe_ggtt_node_fini(tile->sriov.vf.ggtt_balloon[1]);
+	xe_ggtt_node_fini(tile->sriov.vf.ggtt_balloon[0]);
+}
+
+static void deballoon_ggtt(struct drm_device *drm, void *arg)
+{
+	struct xe_tile *tile = arg;
+
+	mutex_lock(&tile->mem.ggtt->lock);
+	xe_gt_sriov_vf_deballoon_ggtt(tile->primary_gt);
+	mutex_unlock(&tile->mem.ggtt->lock);
+	xe_gt_sriov_vf_balloon_fini(tile->primary_gt);
+}
+
  /**
   * xe_gt_sriov_vf_prepare_ggtt - Prepare a VF's GGTT configuration.
   * @gt: the &xe_gt
@@ -650,14 +677,21 @@ int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt)
  {
  	struct xe_tile *tile = gt_to_tile(gt);
  	struct xe_device *xe = tile_to_xe(tile);
+	struct xe_ggtt *ggtt = tile->mem.ggtt;
  	int err;
 
  	if (xe_gt_is_media_type(gt))
  		return 0;
 
Quoted 12 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt) · line 692

Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +7 -2
-	err = vf_balloon_ggtt(gt);
-	if (err)
+	xe_gt_sriov_vf_balloon_init(gt);
+
+	mutex_lock(&ggtt->lock);
+	err = xe_gt_sriov_vf_balloon_ggtt(gt);
+	mutex_unlock(&ggtt->lock);
+	if (err) {
+		xe_gt_sriov_vf_balloon_fini(gt);

since you have split ballooning into two parts, both parts deserve it's own cleanup "actions" to avoid any mistakes in manual cleanup sequence like this (or we can rely on the new xe_ggtt_node_init_managed if added)

They do have that. balloon_ggtt has deballoon_ggtt, and balloon_init has balloon_fini.

This will get clearer with modifications from previous comments applied.

-Tomasz

Quoted 18 lines
Quoted 18 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
  		return err;
+	}
 
  	return drmm_add_action_or_reset(&xe->drm, deballoon_ggtt, tile);
  }
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +2 -0
index ba6c5d74e326..c87b0e9c7ebc 100644
@@ -18,6 +18,8 @@ int xe_gt_sriov_vf_query_config(struct xe_gt *gt);
  int xe_gt_sriov_vf_connect(struct xe_gt *gt);
  int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
  int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt);
+int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt);
+void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt);
  int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
  void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
author
#4 lore raw

PATCH v6 2/4 drm/xe/sriov: Shifting GGTT area post migration

In reply to Tomasz Lis

We have only one GGTT for all IOV functions, with each VF having assigned a range of addresses for its use. After migration, a VF can receive a different range of addresses than it had initially.

This implements shifting GGTT addresses within drm_mm nodes, so that VMAs stay valid after migration. This will make the driver use new addresses when accessing GGTT from the moment the shifting ends.

By taking the ggtt->lock for the period of VMA fixups, this change also adds constraint on that mutex. Any locks used during the recovery cannot ever wait for hardware response - because after migration, the hardware will not do anything until fixups are finished.

v2: Moved some functs to xe_ggtt.c; moved shift computation to just
after querying; improved documentation; switched some warns to asserts; skipping fixups when GGTT shift eq 0; iterating through tiles (Michal)
v3: Updated kerneldocs, removed unused funct, properly allocate
balloning nodes if non existent
v4: Re-used ballooning functions from VF init, used bool in place of
standard error codes
v5: Renamed one function

---
 drivers/gpu/drm/xe/xe_ggtt.c              | 33 +++++++++
 drivers/gpu/drm/xe/xe_ggtt.h              |  1 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 83 +++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  1 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  2 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          | 17 +++++
 6 files changed, 137 insertions(+)
drivers/gpu/drm/xe/xe_ggtt.c +33 -0
index 2d7456e37ef4..b13c4a12393e 100644
@@ -482,6 +482,39 @@ void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node)
 	drm_mm_remove_node(&node->base);
 }
 
+/**
+ * xe_ggtt_shift_mm_nodes - Shift GGTT nodes to adjust for a change in usable address range.
+ * @ggtt: the &xe_ggtt struct instance
+ * @shift: change to the location of area provisioned for current VF
+ */
+void xe_ggtt_shift_mm_nodes(struct xe_ggtt *ggtt, s64 shift)
+{
+	struct drm_mm_node *node, *tmpn;
+	LIST_HEAD(temp_list_head);
+	int err;
+
+	lockdep_assert_held(&ggtt->lock);
+
+	/*
+	 * Move nodes, from range previously assigned to this VF, to a temp list.
+	 */
+	drm_mm_for_each_node_safe(node, tmpn, &ggtt->mm) {
+		drm_mm_remove_node(node);
+		list_add(&node->node_list, &temp_list_head);
+	}
+
+	/*
+	 * Now the GGTT VM contains no nodes. We can re-add all VF nodes with
+	 * shifted offsets.
+	 */
+	list_for_each_entry_safe(node, tmpn, &temp_list_head, node_list) {
+		list_del(&node->node_list);
+		node->start += shift;
+		err = drm_mm_reserve_node(&ggtt->mm, node);
+		xe_tile_assert(ggtt->tile, !err);
+	}
+}
+
 /**
  * xe_ggtt_node_insert_locked - Locked version to insert a &xe_ggtt_node into the GGTT
  * @node: the &xe_ggtt_node to be inserted
drivers/gpu/drm/xe/xe_ggtt.h +1 -0
index 27e7d67de004..a07194cd3724 100644
@@ -18,6 +18,7 @@ void xe_ggtt_node_fini(struct xe_ggtt_node *node);
 int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node,
 				u64 start, u64 size);
 void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node);
+void xe_ggtt_shift_mm_nodes(struct xe_ggtt *ggtt, s64 shift);
 
 int xe_ggtt_node_insert(struct xe_ggtt_node *node, u32 size, u32 align);
 int xe_ggtt_node_insert_locked(struct xe_ggtt_node *node,
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +83 -0

95 lines not on this page · raw message

drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -0
index c87b0e9c7ebc..13c04e313aa6 100644
@@ -20,6 +20,7 @@ int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
 int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt);
 int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt);
 void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt);
+bool xe_gt_sriov_vf_fixup_ggtt_nodes(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +2 -0
index a57f13b5afcd..5ccbdf8d08b6 100644
@@ -40,6 +40,8 @@ struct xe_gt_sriov_vf_selfconfig {
 	u64 ggtt_base;
 	/** @ggtt_size: assigned size of the GGTT region. */
 	u64 ggtt_size;
+	/** @ggtt_shift: difference in ggtt_base on last migration */
+	s64 ggtt_shift;
 	/** @lmem_size: assigned size of the LMEM. */
 	u64 lmem_size;
 	/** @num_ctxs: assigned number of GuC submission context IDs. */
drivers/gpu/drm/xe/xe_sriov_vf.c +17 -0
index c1275e64aa9c..2eb6b8d8a217 100644
@@ -7,6 +7,7 @@
 
 #include "xe_assert.h"
 #include "xe_device.h"
+#include "xe_gt.h"
 #include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
 #include "xe_pm.h"
@@ -170,6 +171,20 @@ static bool vf_post_migration_imminent(struct xe_device *xe)
 	work_pending(&xe->sriov.vf.migration.worker);
 }
 
+static bool vf_post_migration_fixup_ggtt_nodes(struct xe_device *xe)
+{
+	struct xe_tile *tile;
+	unsigned int id;
+	bool need_fixups = false;
+
+	for_each_tile(tile, xe, id) {
+		struct xe_gt *gt = tile->primary_gt;
+
+		need_fixups |= xe_gt_sriov_vf_fixup_ggtt_nodes(gt);
+	}
+	return need_fixups;
+}
+
 /*
  * Notify all GuCs about resource fixups apply finished.
  */
@@ -191,6 +206,7 @@ static void vf_post_migration_notify_resfix_done(struct xe_device *xe)
 
 static void vf_post_migration_recovery(struct xe_device *xe)
 {
+	bool need_fixups;
 	int err;
 
 	drm_dbg(&xe->drm, "migration recovery in progress\n");
@@ -201,6 +217,7 @@ static void vf_post_migration_recovery(struct xe_device *xe)
 	if (unlikely(err))
 		goto fail;
 
+	need_fixups = vf_post_migration_fixup_ggtt_nodes(xe);
 	/* FIXME: add the recovery steps */
 	vf_post_migration_notify_resfix_done(xe);
 	xe_pm_runtime_put(xe);
Signature · 1 line
2.25.1
#5 lore raw

Re: [PATCH v6 2/4] drm/xe/sriov: Shifting GGTT area post migration

please use "drm/xe/vf:" in the subject as this patch is still more VF oriented, then general SRIOV

Quoted 37 lines · Tomasz Lis

We have only one GGTT for all IOV functions, with each VF having assigned a range of addresses for its use. After migration, a VF can receive a different range of addresses than it had initially.

This implements shifting GGTT addresses within drm_mm nodes, so that VMAs stay valid after migration. This will make the driver use new addresses when accessing GGTT from the moment the shifting ends.

By taking the ggtt->lock for the period of VMA fixups, this change also adds constraint on that mutex. Any locks used during the recovery cannot ever wait for hardware response - because after migration, the hardware will not do anything until fixups are finished.

v2: Moved some functs to xe_ggtt.c; moved shift computation to just
after querying; improved documentation; switched some warns to asserts; skipping fixups when GGTT shift eq 0; iterating through tiles (Michal)
v3: Updated kerneldocs, removed unused funct, properly allocate
balloning nodes if non existent
v4: Re-used ballooning functions from VF init, used bool in place of
standard error codes
v5: Renamed one function

---
 drivers/gpu/drm/xe/xe_ggtt.c              | 33 +++++++++
 drivers/gpu/drm/xe/xe_ggtt.h              |  1 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 83 +++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  1 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  2 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          | 17 +++++
 6 files changed, 137 insertions(+)
drivers/gpu/drm/xe/xe_ggtt.c +0 -0
index 2d7456e37ef4..b13c4a12393e 100644

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 486

Quoted 5 lines
drivers/gpu/drm/xe/xe_ggtt.c +2 -0
@@ -482,6 +482,39 @@ void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node)
 	drm_mm_remove_node(&node->base);
 }
 
+/**
+ * xe_ggtt_shift_mm_nodes - Shift GGTT nodes to adjust for a change in usable address range.

drop "mm" from the function name as it is implementation detail

xe_ggtt_shift_nodes()

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 488

Quoted 2 lines
drivers/gpu/drm/xe/xe_ggtt.c +2 -0
+ * @ggtt: the &xe_ggtt struct instance
+ * @shift: change to the location of area provisioned for current VF

this function is quite generic and while it is used by the VF only, the parameter doesn't have to be described as such

you may add here, in the 'longer description' part of the kernel-doc, after explaining how it works, that function is mostly used by the VF, and why we believe it will succeed

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 499

Quoted 11 lines
drivers/gpu/drm/xe/xe_ggtt.c +11 -0
+ */
+void xe_ggtt_shift_mm_nodes(struct xe_ggtt *ggtt, s64 shift)
+{
+	struct drm_mm_node *node, *tmpn;
+	LIST_HEAD(temp_list_head);
+	int err;
+
+	lockdep_assert_held(&ggtt->lock);
+
+	/*
+	 * Move nodes, from range previously assigned to this VF, to a temp list.

nit: no need for multi-line comment style

also, maybe it could be moved to the 'longer description' part of the function kernel-doc, after sanitize it from the 'VF' specific wording

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 509

Quoted 10 lines
drivers/gpu/drm/xe/xe_ggtt.c +10 -0
+	 */
+	drm_mm_for_each_node_safe(node, tmpn, &ggtt->mm) {
+		drm_mm_remove_node(node);
+		list_add(&node->node_list, &temp_list_head);
+	}
+
+	/*
+	 * Now the GGTT VM contains no nodes. We can re-add all VF nodes with
+	 * shifted offsets.
+	 */

also consider to move this comment to function kernel-doc

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 514

Quoted 5 lines
drivers/gpu/drm/xe/xe_ggtt.c +5 -0
+	list_for_each_entry_safe(node, tmpn, &temp_list_head, node_list) {
+		list_del(&node->node_list);
+		node->start += shift;
+		err = drm_mm_reserve_node(&ggtt->mm, node);
+		xe_tile_assert(ggtt->tile, !err);

while we believe it should be always possible to 'shift' all nodes, as we just released our balloons, I'm not sure that this assert here alone is sufficient

maybe before starting any movements, check that 'shift' is valid, and add asserts for each node that shifted location is within GGTT space?

Quoted 23 lines
drivers/gpu/drm/xe/xe_ggtt.c +3 -0
+	}
+}
+
 /**
  * xe_ggtt_node_insert_locked - Locked version to insert a &xe_ggtt_node into the GGTT
  * @node: the &xe_ggtt_node to be inserted
drivers/gpu/drm/xe/xe_ggtt.h +1 -0
index 27e7d67de004..a07194cd3724 100644
@@ -18,6 +18,7 @@ void xe_ggtt_node_fini(struct xe_ggtt_node *node);
 int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node,
 				u64 start, u64 size);
 void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node);
+void xe_ggtt_shift_mm_nodes(struct xe_ggtt *ggtt, s64 shift);
 
 int xe_ggtt_node_insert(struct xe_ggtt_node *node, u32 size, u32 align);
 int xe_ggtt_node_insert_locked(struct xe_ggtt_node *node,
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index 9edbe34f45f4..e9e7ddeb4254 100644

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt) · line 418

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
@@ -415,6 +415,7 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
 	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
 				start, start + size - 1, size / SZ_1K);
 
+	config->ggtt_shift = start - (s64)config->ggtt_base;

on the first probe, shouldn't we store it as '0' ?

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 974

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
 	config->ggtt_base = start;
 	config->ggtt_size = size;
 
@@ -972,6 +973,88 @@ int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt)
 	return err;
 }

hmm, are you sure that place between

xe_gt_sriov_vf_query_runtime
and
vf_runtime_reg_cmp

is the best place for this function?

maybe at least place it closer to

xe_gt_sriov_vf_migrated_event_handler

that is at least partially related,
but not sure if also not miss-located

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 983

Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +8 -0
+/**
+ * xe_gt_sriov_vf_fixup_ggtt_nodes - Shift GGTT allocations to match assigned range.
+ * @gt: the &xe_gt struct instance
+ * Return: True if fixups are necessary
+ *
+ * Since Global GTT is not virtualized, each VF has an assigned range
+ * within the global space. This range might have changed during migration,
+ * which requires all memory addresses pointing to GGTT to be shifted.

This 'longer description' shall be _after_ 'Return' description

see https://docs.kernel.org/doc-guide/kernel-doc.html#function-documentation

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 987

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+ */
+bool xe_gt_sriov_vf_fixup_ggtt_nodes(struct xe_gt *gt)
+{
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;

you should introduce xe_gt_sriov_vf_ggtt_shift() earlier than patch 4/4 and use it here (or better let the caller use it and pass ggtt_shift as parameter)

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 991

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+	struct xe_tile *tile = gt_to_tile(gt);
+	struct xe_ggtt *ggtt = tile->mem.ggtt;
+	s64 ggtt_shift;
+

assert that gt is not a media one

Quoted 17 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +17 -0
+	mutex_lock(&ggtt->lock);
+	ggtt_shift = config->ggtt_shift;
+	/*
+	 * Move nodes, including balloons, from range previously assigned to this VF,
+	 * into newly provisioned area.
+	 *
+	 * The balloon nodes are there to eliminate unavailable ranges from use: one
+	 * reserves the GGTT area below the range for current VF, and another one
+	 * reserves area above.
+	 *
+	 * Below is a GGTT layout of example VF, with a certain address range assigned to
+	 * said VF, and inaccessible areas above and below:
+	 *
+	 *  0                                                                        4GiB
+	 *  |<--------------------------- Total GGTT size ----------------------------->|
+	 *      WOPCM                                                         GUC_TOP
+	 *      |<-------------- Area mappable by xe_ggtt instance ---------------->|

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 1018

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	 *
+	 *  +---+---------------------------------+----------+----------------------+---+
+	 *  |\\\|/////////////////////////////////|  VF mem  |//////////////////////|\\\|
+	 *  +---+---------------------------------+----------+----------------------+---+
+	 *
+	 * Hardware enforced access rules before migration:
+	 *
+	 *  |<------- inaccessible for VF ------->|<VF owned>|<-- inaccessible for VF ->|
+	 *
+	 * drm_mm nodes used for tracking allocations:

use of drm_mm is implementation detail of the xe_ggtt and it is not relevant here, just say 'GGTT nodes'

Quoted 20 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +20 -0
+	 *
+	 *     |<----------- balloon ------------>|<- nodes->|<----- balloon ------>|
+	 *
+	 * After the migration, GGTT area assigned to the VF might have shifted, either
+	 * to lower or to higher address. But we expect the total size and extra areas to
+	 * be identical, as migration can only happen between matching platforms.
+	 * Below is an example of GGTT layout of the VF after migration. Content of the
+	 * GGTT for VF has been moved to a new area, and we receive its address from GuC:
+	 *
+	 *  +---+----------------------+----------+---------------------------------+---+
+	 *  |\\\|//////////////////////|  VF mem  |/////////////////////////////////|\\\|
+	 *  +---+----------------------+----------+---------------------------------+---+
+	 *
+	 * Hardware enforced access rules after migration:
+	 *
+	 *  |<- inaccessible for VF -->|<VF owned>|<------- inaccessible for VF ------->|
+	 *
+	 * So the VF has a new slice of GGTT assigned, and during migration process, the
+	 * memory content was copied to that new area. But the drm_mm nodes within xe kmd
+	 * are still tracking allocations using the old addresses. The nodes within VF

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 1048

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	 * owned area have to be shifted, and balloon nodes need to be resized to
+	 * properly mask out areas not owned by the VF.
+	 *
+	 * Fixed drm_mm nodes used for tracking allocations:
+	 *
+	 *     |<------ balloon ------>|<- nodes->|<----------- balloon ----------->|
+	 *
+	 * Due to use of GPU profiles, we do not expect the old and new GGTT ares to
+	 * overlap; but our node shifting will fix addresses properly regardless.
+	 */

this inline comment is now much bigger than actual implementation

why not promote it to the full DOC: as then it could be included in the master SRIOV documentation

Quoted 50 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +9 -0
+	if (ggtt_shift) {
+		xe_gt_sriov_vf_deballoon_ggtt(gt);
+		xe_ggtt_shift_mm_nodes(ggtt, ggtt_shift);
+		xe_gt_sriov_vf_balloon_ggtt(gt);
+	}
+	mutex_unlock(&ggtt->lock);
+	return ggtt_shift != 0;
+}
+
 static int vf_runtime_reg_cmp(const void *a, const void *b)
 {
 	const struct vf_runtime_reg *ra = a;
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -0
index c87b0e9c7ebc..13c04e313aa6 100644
@@ -20,6 +20,7 @@ int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
 int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt);
 int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt);
 void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt);
+bool xe_gt_sriov_vf_fixup_ggtt_nodes(struct xe_gt *gt);
 int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
 void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +2 -0
index a57f13b5afcd..5ccbdf8d08b6 100644
@@ -40,6 +40,8 @@ struct xe_gt_sriov_vf_selfconfig {
 	u64 ggtt_base;
 	/** @ggtt_size: assigned size of the GGTT region. */
 	u64 ggtt_size;
+	/** @ggtt_shift: difference in ggtt_base on last migration */
+	s64 ggtt_shift;
 	/** @lmem_size: assigned size of the LMEM. */
 	u64 lmem_size;
 	/** @num_ctxs: assigned number of GuC submission context IDs. */
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
index c1275e64aa9c..2eb6b8d8a217 100644
@@ -7,6 +7,7 @@
 
 #include "xe_assert.h"
 #include "xe_device.h"
+#include "xe_gt.h"
 #include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
 #include "xe_pm.h"

drivers/gpu/drm/xe/xe_sriov_vf.c static bool vf_post_migration_imminent(struct xe_device *xe) · line 178

Quoted 8 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +5 -0
@@ -170,6 +171,20 @@ static bool vf_post_migration_imminent(struct xe_device *xe)
 	work_pending(&xe->sriov.vf.migration.worker);
 }
 
+static bool vf_post_migration_fixup_ggtt_nodes(struct xe_device *xe)
+{
+	struct xe_tile *tile;
+	unsigned int id;
+	bool need_fixups = false;

try to define vars in reverse-xmas-tree order

	bool need_fixups = false;
	struct xe_tile *tile;
	unsigned int id;

drivers/gpu/drm/xe/xe_sriov_vf.c static bool vf_post_migration_imminent(struct xe_device *xe) · line 181

Quoted 3 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +3 -0
+
+	for_each_tile(tile, xe, id) {
+		struct xe_gt *gt = tile->primary_gt;
		shift = xe_gt_sriov_vf_ggtt_shift(gt);
		if (shift) {
			need_fixups = true;
			xe_gt_sriov_vf_fixup_ggtt_nodes(gt, shift);
		}
Quoted 25 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +8 -0
+
+		need_fixups |= xe_gt_sriov_vf_fixup_ggtt_nodes(gt);
+	}
+	return need_fixups;
+}
+
 /*
  * Notify all GuCs about resource fixups apply finished.
  */
@@ -191,6 +206,7 @@ static void vf_post_migration_notify_resfix_done(struct xe_device *xe)
 
 static void vf_post_migration_recovery(struct xe_device *xe)
 {
+	bool need_fixups;
 	int err;
 
 	drm_dbg(&xe->drm, "migration recovery in progress\n");
@@ -201,6 +217,7 @@ static void vf_post_migration_recovery(struct xe_device *xe)
 	if (unlikely(err))
 		goto fail;
 
+	need_fixups = vf_post_migration_fixup_ggtt_nodes(xe);
 	/* FIXME: add the recovery steps */
 	vf_post_migration_notify_resfix_done(xe);
 	xe_pm_runtime_put(xe);
#6 lore raw

Re: [PATCH v6 2/4] drm/xe/sriov: Shifting GGTT area post migration

Quoted 2 lines · Michal Wajdeczko

please use "drm/xe/vf:" in the subject as this patch is still more VF oriented, then general SRIOV

ok.

Quoted 41 lines · Tomasz Lis
Quoted 40 lines

We have only one GGTT for all IOV functions, with each VF having assigned a range of addresses for its use. After migration, a VF can receive a different range of addresses than it had initially.

This implements shifting GGTT addresses within drm_mm nodes, so that VMAs stay valid after migration. This will make the driver use new addresses when accessing GGTT from the moment the shifting ends.

By taking the ggtt->lock for the period of VMA fixups, this change also adds constraint on that mutex. Any locks used during the recovery cannot ever wait for hardware response - because after migration, the hardware will not do anything until fixups are finished.

v2: Moved some functs to xe_ggtt.c; moved shift computation to just
after querying; improved documentation; switched some warns to asserts; skipping fixups when GGTT shift eq 0; iterating through tiles (Michal)
v3: Updated kerneldocs, removed unused funct, properly allocate
balloning nodes if non existent
v4: Re-used ballooning functions from VF init, used bool in place of
standard error codes
v5: Renamed one function

---
  drivers/gpu/drm/xe/xe_ggtt.c              | 33 +++++++++
  drivers/gpu/drm/xe/xe_ggtt.h              |  1 +
  drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 83 +++++++++++++++++++++++
  drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |  1 +
  drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |  2 +
  drivers/gpu/drm/xe/xe_sriov_vf.c          | 17 +++++
  6 files changed, 137 insertions(+)
drivers/gpu/drm/xe/xe_ggtt.c +0 -0
index 2d7456e37ef4..b13c4a12393e 100644
@@ -482,6 +482,39 @@ void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node)
  	drm_mm_remove_node(&node->base);
  }
 
Quoted 5 lines

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 486

Quoted 2 lines
drivers/gpu/drm/xe/xe_ggtt.c +2 -0
+/**
+ * xe_ggtt_shift_mm_nodes - Shift GGTT nodes to adjust for a change in usable address range.

drop "mm" from the function name as it is implementation detail

xe_ggtt_shift_nodes()

The fact that nodes are used for Memory Management is an implementation detail?

I will remove mm from the name, because no other function uses that shortcut. But let's stick

to real arguments, ie. that no other function dealing with nodes has the _mm_ in name.

Quoted 4 lines

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 488

Quoted 2 lines
drivers/gpu/drm/xe/xe_ggtt.c +2 -0
+ * @ggtt: the &xe_ggtt struct instance
+ * @shift: change to the location of area provisioned for current VF

this function is quite generic and while it is used by the VF only, the parameter doesn't have to be described as such

I just changed the subject line because (from comment above) the patch is VF oriented.

Quoted 3 lines

you may add here, in the 'longer description' part of the kernel-doc, after explaining how it works, that function is mostly used by the VF, and why we believe it will succeed

Not "mostly" but "only". There is no other scenario where we could anticipate this function to be used.

While universalization is generally a good direction, pretending that this code will ever be used for anything else than VF is not a direction with any chance for future benefits.

For the comment about errors - will add.

Quoted 16 lines

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 499

Quoted 11 lines
drivers/gpu/drm/xe/xe_ggtt.c +11 -0
+ */
+void xe_ggtt_shift_mm_nodes(struct xe_ggtt *ggtt, s64 shift)
+{
+	struct drm_mm_node *node, *tmpn;
+	LIST_HEAD(temp_list_head);
+	int err;
+
+	lockdep_assert_held(&ggtt->lock);
+
+	/*
+	 * Move nodes, from range previously assigned to this VF, to a temp list.

nit: no need for multi-line comment style

also, maybe it could be moved to the 'longer description' part of the function kernel-doc, after sanitize it from the 'VF' specific wording

Quoted 11 lines

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 509

Quoted 10 lines
drivers/gpu/drm/xe/xe_ggtt.c +10 -0
+	 */
+	drm_mm_for_each_node_safe(node, tmpn, &ggtt->mm) {
+		drm_mm_remove_node(node);
+		list_add(&node->node_list, &temp_list_head);
+	}
+
+	/*
+	 * Now the GGTT VM contains no nodes. We can re-add all VF nodes with
+	 * shifted offsets.
+	 */

also consider to move this comment to function kernel-doc

Not sure why we want a detail on implementation in the function description.

Also not sure why we want detailed description block in this function, when we already have that in the caller.

But I don't mind, will move.

Quoted 11 lines

drivers/gpu/drm/xe/xe_ggtt.c void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node) · line 514

Quoted 5 lines
drivers/gpu/drm/xe/xe_ggtt.c +5 -0
+	list_for_each_entry_safe(node, tmpn, &temp_list_head, node_list) {
+		list_del(&node->node_list);
+		node->start += shift;
+		err = drm_mm_reserve_node(&ggtt->mm, node);
+		xe_tile_assert(ggtt->tile, !err);

while we believe it should be always possible to 'shift' all nodes, as we just released our balloons, I'm not sure that this assert here alone is sufficient

maybe before starting any movements, check that 'shift' is valid, and add asserts for each node that shifted location is within GGTT space?

Before starting? How exactly do we check the shift before we have any nodes? Without nodes, we have nothing to compare to.

I assume that what you're going for, is just to add more asserts on each node. Will do.

Quoted 26 lines
Quoted 26 lines
drivers/gpu/drm/xe/xe_ggtt.c +3 -0
+	}
+}
+
  /**
   * xe_ggtt_node_insert_locked - Locked version to insert a &xe_ggtt_node into the GGTT
   * @node: the &xe_ggtt_node to be inserted
drivers/gpu/drm/xe/xe_ggtt.h +1 -0
index 27e7d67de004..a07194cd3724 100644
@@ -18,6 +18,7 @@ void xe_ggtt_node_fini(struct xe_ggtt_node *node);
  int xe_ggtt_node_insert_balloon(struct xe_ggtt_node *node,
  				u64 start, u64 size);
  void xe_ggtt_node_remove_balloon(struct xe_ggtt_node *node);
+void xe_ggtt_shift_mm_nodes(struct xe_ggtt *ggtt, s64 shift);
 
  int xe_ggtt_node_insert(struct xe_ggtt_node *node, u32 size, u32 align);
  int xe_ggtt_node_insert_locked(struct xe_ggtt_node *node,
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
index 9edbe34f45f4..e9e7ddeb4254 100644
@@ -415,6 +415,7 @@ static int vf_get_ggtt_info(struct xe_gt *gt)
  	xe_gt_sriov_dbg_verbose(gt, "GGTT %#llx-%#llx = %lluK\n",
  				start, start + size - 1, size / SZ_1K);
 
Quoted 2 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c static int vf_get_ggtt_info(struct xe_gt *gt) · line 418

Quoted 1 line
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +1 -0
+	config->ggtt_shift = start - (s64)config->ggtt_base;

on the first probe, shouldn't we store it as '0' ?

ggtt_shift is a part ofxe_gt, allocated in xe_gt_alloc() using kzalloc variant. It is always zero on probe, regardless if first or not. But even regardless of that, the value of this parameter is valid only when within the post-migration recovery worker. It is not accessed outside, and noone cares about its value outside.

Quoted 12 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 974

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +0 -0
  	config->ggtt_base = start;
  	config->ggtt_size = size;
 
@@ -972,6 +973,88 @@ int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt)
  	return err;
  }

hmm, are you sure that place between

xe_gt_sriov_vf_query_runtime
and
vf_runtime_reg_cmp

Quoted 8 lines

is the best place for this function?

maybe at least place it closer to

xe_gt_sriov_vf_migrated_event_handler

that is at least partially related,
but not sure if also not miss-located

no problem, will place it there.

Quoted 9 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 983

Quoted 9 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +8 -0
+/**
+ * xe_gt_sriov_vf_fixup_ggtt_nodes - Shift GGTT allocations to match assigned range.
+ * @gt: the &xe_gt struct instance
+ * Return: True if fixups are necessary
+ *
+ * Since Global GTT is not virtualized, each VF has an assigned range
+ * within the global space. This range might have changed during migration,
+ * which requires all memory addresses pointing to GGTT to be shifted.
Quoted 3 lines

This 'longer description' shall be _after_ 'Return' description

seehttps://docs.kernel.org/doc-guide/kernel-doc.html#function-documentation

I see from the link than you meant Return: at the end. But I agreed to your comments below so the return vanishes completely.

Quoted 7 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 987

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+ */
+bool xe_gt_sriov_vf_fixup_ggtt_nodes(struct xe_gt *gt)
+{
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;

you should introduce xe_gt_sriov_vf_ggtt_shift() earlier than patch 4/4 and use it here (or better let the caller use it and pass ggtt_shift as parameter)

will move that function earlier.

Quoted 5 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 991

Quoted 4 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +4 -0
+	struct xe_tile *tile = gt_to_tile(gt);
+	struct xe_ggtt *ggtt = tile->mem.ggtt;
+	s64 ggtt_shift;
+

assert that gt is not a media one

ok

Quoted 21 lines
Quoted 21 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +21 -0
+	mutex_lock(&ggtt->lock);
+	ggtt_shift = config->ggtt_shift;
+	/*
+	 * Move nodes, including balloons, from range previously assigned to this VF,
+	 * into newly provisioned area.
+	 *
+	 * The balloon nodes are there to eliminate unavailable ranges from use: one
+	 * reserves the GGTT area below the range for current VF, and another one
+	 * reserves area above.
+	 *
+	 * Below is a GGTT layout of example VF, with a certain address range assigned to
+	 * said VF, and inaccessible areas above and below:
+	 *
+	 *  0                                                                        4GiB
+	 *  |<--------------------------- Total GGTT size ----------------------------->|
+	 *      WOPCM                                                         GUC_TOP
+	 *      |<-------------- Area mappable by xe_ggtt instance ---------------->|
+	 *
+	 *  +---+---------------------------------+----------+----------------------+---+
+	 *  |\\\|/////////////////////////////////|  VF mem  |//////////////////////|\\\|
+	 *  +---+---------------------------------+----------+----------------------+---+
Quoted 8 lines

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 1018

Quoted 6 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +6 -0
+	 *
+	 * Hardware enforced access rules before migration:
+	 *
+	 *  |<------- inaccessible for VF ------->|<VF owned>|<-- inaccessible for VF ->|
+	 *
+	 * drm_mm nodes used for tracking allocations:

use of drm_mm is implementation detail of the xe_ggtt and it is not relevant here, just say 'GGTT nodes'

We are trying to hide implementation details. Not because these details can vary/change - this will never happen.

We're just calling some things implementation details and hiding them because it is doable.

We've talked about this before, so will disagree but commit.

Quoted 32 lines
Quoted 20 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +20 -0
+	 *
+	 *     |<----------- balloon ------------>|<- nodes->|<----- balloon ------>|
+	 *
+	 * After the migration, GGTT area assigned to the VF might have shifted, either
+	 * to lower or to higher address. But we expect the total size and extra areas to
+	 * be identical, as migration can only happen between matching platforms.
+	 * Below is an example of GGTT layout of the VF after migration. Content of the
+	 * GGTT for VF has been moved to a new area, and we receive its address from GuC:
+	 *
+	 *  +---+----------------------+----------+---------------------------------+---+
+	 *  |\\\|//////////////////////|  VF mem  |/////////////////////////////////|\\\|
+	 *  +---+----------------------+----------+---------------------------------+---+
+	 *
+	 * Hardware enforced access rules after migration:
+	 *
+	 *  |<- inaccessible for VF -->|<VF owned>|<------- inaccessible for VF ------->|
+	 *
+	 * So the VF has a new slice of GGTT assigned, and during migration process, the
+	 * memory content was copied to that new area. But the drm_mm nodes within xe kmd
+	 * are still tracking allocations using the old addresses. The nodes within VF

drivers/gpu/drm/xe/xe_gt_sriov_vf.c int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt) · line 1048

Quoted 10 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +10 -0
+	 * owned area have to be shifted, and balloon nodes need to be resized to
+	 * properly mask out areas not owned by the VF.
+	 *
+	 * Fixed drm_mm nodes used for tracking allocations:
+	 *
+	 *     |<------ balloon ------>|<- nodes->|<----------- balloon ----------->|
+	 *
+	 * Due to use of GPU profiles, we do not expect the old and new GGTT ares to
+	 * overlap; but our node shifting will fix addresses properly regardless.
+	 */

this inline comment is now much bigger than actual implementation

Quoted 2 lines

why not promote it to the full DOC: as then it could be included in the master SRIOV documentation

will do.

..but for this case only. I will oppose to any tries of having the architecture docs rewritten as code comments.

Quoted 60 lines
Quoted 50 lines
drivers/gpu/drm/xe/xe_gt_sriov_vf.c +9 -0
+	if (ggtt_shift) {
+		xe_gt_sriov_vf_deballoon_ggtt(gt);
+		xe_ggtt_shift_mm_nodes(ggtt, ggtt_shift);
+		xe_gt_sriov_vf_balloon_ggtt(gt);
+	}
+	mutex_unlock(&ggtt->lock);
+	return ggtt_shift != 0;
+}
+
  static int vf_runtime_reg_cmp(const void *a, const void *b)
  {
  	const struct vf_runtime_reg *ra = a;
drivers/gpu/drm/xe/xe_gt_sriov_vf.h +1 -0
index c87b0e9c7ebc..13c04e313aa6 100644
@@ -20,6 +20,7 @@ int xe_gt_sriov_vf_query_runtime(struct xe_gt *gt);
  int xe_gt_sriov_vf_prepare_ggtt(struct xe_gt *gt);
  int xe_gt_sriov_vf_balloon_ggtt(struct xe_gt *gt);
  void xe_gt_sriov_vf_deballoon_ggtt(struct xe_gt *gt);
+bool xe_gt_sriov_vf_fixup_ggtt_nodes(struct xe_gt *gt);
  int xe_gt_sriov_vf_notify_resfix_done(struct xe_gt *gt);
  void xe_gt_sriov_vf_migrated_event_handler(struct xe_gt *gt);
 
drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h +2 -0
index a57f13b5afcd..5ccbdf8d08b6 100644
@@ -40,6 +40,8 @@ struct xe_gt_sriov_vf_selfconfig {
  	u64 ggtt_base;
  	/** @ggtt_size: assigned size of the GGTT region. */
  	u64 ggtt_size;
+	/** @ggtt_shift: difference in ggtt_base on last migration */
+	s64 ggtt_shift;
  	/** @lmem_size: assigned size of the LMEM. */
  	u64 lmem_size;
  	/** @num_ctxs: assigned number of GuC submission context IDs. */
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
index c1275e64aa9c..2eb6b8d8a217 100644
@@ -7,6 +7,7 @@
 
  #include "xe_assert.h"
  #include "xe_device.h"
+#include "xe_gt.h"
  #include "xe_gt_sriov_printk.h"
  #include "xe_gt_sriov_vf.h"
  #include "xe_pm.h"

drivers/gpu/drm/xe/xe_sriov_vf.c static bool vf_post_migration_imminent(struct xe_device *xe) · line 178

Quoted 8 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +5 -0
@@ -170,6 +171,20 @@ static bool vf_post_migration_imminent(struct xe_device *xe)
  	work_pending(&xe->sriov.vf.migration.worker);
  }
 
+static bool vf_post_migration_fixup_ggtt_nodes(struct xe_device *xe)
+{
+	struct xe_tile *tile;
+	unsigned int id;
+	bool need_fixups = false;

try to define vars in reverse-xmas-tree order

Quoted 3 lines
	bool need_fixups = false;
	struct xe_tile *tile;
	unsigned int id;

ok

Quoted 8 lines

drivers/gpu/drm/xe/xe_sriov_vf.c static bool vf_post_migration_imminent(struct xe_device *xe) · line 181

Quoted 3 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +3 -0
+
+	for_each_tile(tile, xe, id) {
+		struct xe_gt *gt = tile->primary_gt;
		shift = xe_gt_sriov_vf_ggtt_shift(gt);
		if (shift) {
			need_fixups = true;
			xe_gt_sriov_vf_fixup_ggtt_nodes(gt, shift);
		}

So in this specific place we want to make the function look longer and reveal a detail.

Makes no sense to me, but since you don't seem convinced by previous review round - will do this.

-Tomasz

Quoted 25 lines
Quoted 25 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +8 -0
+
+		need_fixups |= xe_gt_sriov_vf_fixup_ggtt_nodes(gt);
+	}
+	return need_fixups;
+}
+
  /*
   * Notify all GuCs about resource fixups apply finished.
   */
@@ -191,6 +206,7 @@ static void vf_post_migration_notify_resfix_done(struct xe_device *xe)
 
  static void vf_post_migration_recovery(struct xe_device *xe)
  {
+	bool need_fixups;
  	int err;
 
  	drm_dbg(&xe->drm, "migration recovery in progress\n");
@@ -201,6 +217,7 @@ static void vf_post_migration_recovery(struct xe_device *xe)
  	if (unlikely(err))
  		goto fail;
 
+	need_fixups = vf_post_migration_fixup_ggtt_nodes(xe);
  	/* FIXME: add the recovery steps */
  	vf_post_migration_notify_resfix_done(xe);
  	xe_pm_runtime_put(xe);
author
#7 lore raw

PATCH v6 3/4 drm/xe/guc: Introduce enum with offsets for multi-LRC register H2G

In reply to Tomasz Lis

Some GuC messages are constructed with incrementing dword counter rather than referencing specific DWORDs, as described in GuC interface specification.

This change introduces the definitions of DWORD numbers for parameters which will need to be referenced in a CTB parser to be added in a following patch. To ensure correctness of these DWORDs, verification in form of asserts was added to the message construction code.

---
 drivers/gpu/drm/xe/abi/guc_actions_abi.h | 7 +++++++
 drivers/gpu/drm/xe/xe_guc_submit.c       | 4 ++++
 2 files changed, 11 insertions(+)
drivers/gpu/drm/xe/abi/guc_actions_abi.h +7 -0
index 448afb86e05c..86bd4b092c7c 100644
@@ -161,6 +161,13 @@ enum xe_guc_preempt_options {
 	XE_GUC_PREEMPT_OPTION_DROP_SUBMIT_Q = 0x8,
 };
 
+enum xe_guc_register_context_multi_lrc_param_offsets {
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC = 5,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE = 7,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN = 10,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA = 11
+};
+
 enum xe_guc_report_status {
 	XE_GUC_REPORT_STATUS_UNKNOWN = 0x0,
 	XE_GUC_REPORT_STATUS_ACKED = 0x1,
drivers/gpu/drm/xe/xe_guc_submit.c +4 -0
index 31bc2022bfc2..3c8574e2c1f3 100644
@@ -469,12 +469,16 @@ static void __register_mlrc_exec_queue(struct xe_guc *guc,
 	action[len++] = info->context_idx;
 	action[len++] = info->engine_class;
 	action[len++] = info->engine_submit_mask;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC);
 	action[len++] = info->wq_desc_lo;
 	action[len++] = info->wq_desc_hi;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE);
 	action[len++] = info->wq_base_lo;
 	action[len++] = info->wq_base_hi;
 	action[len++] = info->wq_size;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN);
 	action[len++] = q->width;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA);
 	action[len++] = info->hwlrca_lo;
 	action[len++] = info->hwlrca_hi;
 
Signature · 1 line
2.25.1
#8 lore raw

Re: [PATCH v6 3/4] drm/xe/guc: Introduce enum with offsets for multi-LRC register H2G

Quoted 20 lines · Tomasz Lis

Some GuC messages are constructed with incrementing dword counter rather than referencing specific DWORDs, as described in GuC interface specification.

This change introduces the definitions of DWORD numbers for parameters which will need to be referenced in a CTB parser to be added in a following patch. To ensure correctness of these DWORDs, verification in form of asserts was added to the message construction code.

---
 drivers/gpu/drm/xe/abi/guc_actions_abi.h | 7 +++++++
 drivers/gpu/drm/xe/xe_guc_submit.c       | 4 ++++
 2 files changed, 11 insertions(+)
drivers/gpu/drm/xe/abi/guc_actions_abi.h +0 -0
index 448afb86e05c..86bd4b092c7c 100644

drivers/gpu/drm/xe/abi/guc_actions_abi.h enum xe_guc_preempt_options { · line 168

Quoted 8 lines
drivers/gpu/drm/xe/abi/guc_actions_abi.h +5 -0
@@ -161,6 +161,13 @@ enum xe_guc_preempt_options {
 	XE_GUC_PREEMPT_OPTION_DROP_SUBMIT_Q = 0x8,
 };
 
+enum xe_guc_register_context_multi_lrc_param_offsets {
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC = 5,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE = 7,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN = 10,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA = 11

while likely we only need those 4 indices, IMO it would cleaner to define all of them in ABI to reflect the whole message layout:

enum xe_guc_register_context_multi_lrc_message {
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_0_MBZ = 0,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_1_FLAGS,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_2_PARENT_CONTEXT,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_3_ENGINE_CLASS,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_4_ENGINE_SUBMIT_MASK,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_5_WQ_DESC_ADDR_LOWER,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_6_WQ_DESC_ADDR_UPPER,
	...
Quoted 16 lines
drivers/gpu/drm/xe/abi/guc_actions_abi.h +2 -0
+};
+
 enum xe_guc_report_status {
 	XE_GUC_REPORT_STATUS_UNKNOWN = 0x0,
 	XE_GUC_REPORT_STATUS_ACKED = 0x1,
drivers/gpu/drm/xe/xe_guc_submit.c +1 -0
index 31bc2022bfc2..3c8574e2c1f3 100644
@@ -469,12 +469,16 @@ static void __register_mlrc_exec_queue(struct xe_guc *guc,
 	action[len++] = info->context_idx;
 	action[len++] = info->engine_class;
 	action[len++] = info->engine_submit_mask;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC);
 	action[len++] = info->wq_desc_lo;
 	action[len++] = info->wq_desc_hi;

drivers/gpu/drm/xe/xe_guc_submit.c static void __register_mlrc_exec_queue(struct xe_guc *guc, · line 484

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +3 -0
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE);
 	action[len++] = info->wq_base_lo;
 	action[len++] = info->wq_base_hi;
 	action[len++] = info->wq_size;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN);
 	action[len++] = q->width;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA);
 	action[len++] = info->hwlrca_lo;
 	action[len++] = info->hwlrca_hi;
 

hmm, it looks little paranoid, and pollute otherwise clean code, so maybe either convert completely to index based message encoding:

	action[.._FLAGS] = info->flags;
	action[.._WQ_DESC_LO] = info->wq_desc_lo;
	action[.._WQ_DESC_HI] = info->wq_desc_hi;
	...
	for (i = 0; i < q->width; ++i)
		action[.._HWLRCA_LO + 1 + i] = ..;
		action[.._HWLRCA_HI + 1 + i] = ..;

or assert those important entries at the end:

	xe_gt_assert(gt, action[..WQ_DESC_LO] == info->wq_desc_lo);
	xe_gt_assert(gt, action[..HWLRCA_LO] == info->hwlrca_lo);

@Matt, any other ideas or you don't care?

#9 lore raw

Re: [PATCH v6 3/4] drm/xe/guc: Introduce enum with offsets for multi-LRC register H2G

Quoted 32 lines · Michal Wajdeczko
Quoted 20 lines · Tomasz Lis

Some GuC messages are constructed with incrementing dword counter rather than referencing specific DWORDs, as described in GuC interface specification.

This change introduces the definitions of DWORD numbers for parameters which will need to be referenced in a CTB parser to be added in a following patch. To ensure correctness of these DWORDs, verification in form of asserts was added to the message construction code.

---
  drivers/gpu/drm/xe/abi/guc_actions_abi.h | 7 +++++++
  drivers/gpu/drm/xe/xe_guc_submit.c       | 4 ++++
  2 files changed, 11 insertions(+)
drivers/gpu/drm/xe/abi/guc_actions_abi.h +0 -0
index 448afb86e05c..86bd4b092c7c 100644

drivers/gpu/drm/xe/abi/guc_actions_abi.h enum xe_guc_preempt_options { · line 168

Quoted 8 lines
drivers/gpu/drm/xe/abi/guc_actions_abi.h +5 -0
@@ -161,6 +161,13 @@ enum xe_guc_preempt_options {
  	XE_GUC_PREEMPT_OPTION_DROP_SUBMIT_Q = 0x8,
  };
 
+enum xe_guc_register_context_multi_lrc_param_offsets {
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC = 5,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE = 7,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN = 10,
+	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA = 11

while likely we only need those 4 indices, IMO it would cleaner to define all of them in ABI to reflect the whole message layout:

Quoted 9 lines
enum xe_guc_register_context_multi_lrc_message {
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_0_MBZ = 0,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_1_FLAGS,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_2_PARENT_CONTEXT,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_3_ENGINE_CLASS,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_4_ENGINE_SUBMIT_MASK,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_5_WQ_DESC_ADDR_LOWER,
	XE_GUC_REGISTER_CONTEXT_MULTI_LRC_DATA_6_WQ_DESC_ADDR_UPPER,
	...

Will agree to all the propositions here as this whole patch isn't my idea.

-Tomasz

Quoted 45 lines
Quoted 16 lines
drivers/gpu/drm/xe/abi/guc_actions_abi.h +2 -0
+};
+
  enum xe_guc_report_status {
  	XE_GUC_REPORT_STATUS_UNKNOWN = 0x0,
  	XE_GUC_REPORT_STATUS_ACKED = 0x1,
drivers/gpu/drm/xe/xe_guc_submit.c +1 -0
index 31bc2022bfc2..3c8574e2c1f3 100644
@@ -469,12 +469,16 @@ static void __register_mlrc_exec_queue(struct xe_guc *guc,
  	action[len++] = info->context_idx;
  	action[len++] = info->engine_class;
  	action[len++] = info->engine_submit_mask;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC);
  	action[len++] = info->wq_desc_lo;
  	action[len++] = info->wq_desc_hi;

drivers/gpu/drm/xe/xe_guc_submit.c static void __register_mlrc_exec_queue(struct xe_guc *guc, · line 484

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_submit.c +3 -0
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE);
  	action[len++] = info->wq_base_lo;
  	action[len++] = info->wq_base_hi;
  	action[len++] = info->wq_size;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN);
  	action[len++] = q->width;
+	xe_gt_assert(guc_to_gt(guc), len == XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA);
  	action[len++] = info->hwlrca_lo;
  	action[len++] = info->hwlrca_hi;
 

hmm, it looks little paranoid, and pollute otherwise clean code, so maybe either convert completely to index based message encoding:

	action[.._FLAGS] = info->flags;
	action[.._WQ_DESC_LO] = info->wq_desc_lo;
	action[.._WQ_DESC_HI] = info->wq_desc_hi;
	...
	for (i = 0; i < q->width; ++i)
		action[.._HWLRCA_LO + 1 + i] = ..;
		action[.._HWLRCA_HI + 1 + i] = ..;

or assert those important entries at the end:

	xe_gt_assert(gt, action[..WQ_DESC_LO] == info->wq_desc_lo);
	xe_gt_assert(gt, action[..HWLRCA_LO] == info->hwlrca_lo);

@Matt, any other ideas or you don't care?

author
#10 lore raw

PATCH v6 4/4 drm/xe/vf: Fixup CTB send buffer messages after migration

In reply to Tomasz Lis

During post-migration recovery of a VF, it is necessary to update GGTT references included in messages which are going to be sent to GuC. GuC will start consuming messages after VF KMD will inform it about fixups being done; before that, the VF KMD is expected to update any H2G messages which are already in send buffer but were not consumed by GuC.

Only a small subset of messages allowed for VFs have GGTT references in them. This patch adds the functionality to parse the CTB send ring buffer and shift addresses contained within.

While fixing the CTB content, ct->lock is not taken. This means the only barrier taken remains GGTT address lock - which is ok, because only requests with GGTT addresses matter, but it also means tail changes can happen during the CTB fixups execution (which may be ignored as any new messages will not have anything to fix).

The GGTT address locking will be introduced in a future series.

v2: removed storing shift as that's now done in VMA nodes patch;
macros to inlines; warns to asserts; log messages fixes (Michal)
v3: removed inline keywords, enums for offsets in CTB messages,
less error messages, if return unused then made functs void (Michal)
v4: update the cached head before starting fixups
v5: removed/updated comments, wrapped lines, converted assert into
error, enums for offsets to separate patch, reused xe_map_rd

---
 drivers/gpu/drm/xe/xe_guc_ct.c   | 142 +++++++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_guc_ct.h   |   2 +
 drivers/gpu/drm/xe/xe_sriov_vf.c |  25 ++++++
 3 files changed, 169 insertions(+)
drivers/gpu/drm/xe/xe_guc_ct.c +142 -0
index 686fe664c20d..6c80106e5e8b 100644
@@ -84,6 +84,8 @@ struct g2h_fence {
 	bool done;
 };
 
+#define make_u64(hi, lo) ((u64)((u64)(u32)(hi) << 32 | (u32)(lo)))
+
 static void g2h_fence_init(struct g2h_fence *g2h_fence, u32 *response_buffer)
 {
 	g2h_fence->response_buffer = response_buffer;
@@ -1622,6 +1624,146 @@ static void g2h_worker_func(struct work_struct *w)
 	receive_g2h(ct);
 }
 
+static u32 xe_map_rd_u32(struct xe_device *xe, struct iosys_map *cmds,
+			     u32 head, u32 pos)
+{
+	return xe_map_rd(xe, cmds, (head + pos) * sizeof(u32), u32);
+}
+
+static void xe_map_fixup_u64(struct xe_device *xe, struct iosys_map *cmds,
+			       u32 head, u32 pos, s64 shift)
+{
+	u32 msg[2];
+	u64 offset;
+
+	xe_map_memcpy_from(xe, msg, cmds, (head + pos) * sizeof(u32),
+			   2 * sizeof(u32));
+	offset = make_u64(msg[1], msg[0]);
+	offset += shift;
+	msg[0] = lower_32_bits(offset);
+	msg[1] = upper_32_bits(offset);
+	xe_map_memcpy_to(xe, cmds, (head + pos) * sizeof(u32), msg, 2 * sizeof(u32));
+}
+
+/*
+ * Shift any GGTT addresses within a single message left within CTB from
+ * before post-migration recovery.
+ * @ct: pointer to CT struct of the target GuC
+ * @cmds: iomap buffer containing CT messages
+ * @head: start of the target message within the buffer
+ * @len: length of the target message
+ * @size: size of the commands buffer
+ * @shift: the address shift to be added to each GGTT reference
+ */
+static void ct_update_addresses_in_message(struct xe_guc_ct *ct,
+					   struct iosys_map *cmds, u32 head,
+					   u32 len, u32 size, s64 shift)
+{
+	struct xe_device *xe = ct_to_xe(ct);
+	u32 action, i, n;
+	u32 msg[1];
+
+	xe_map_memcpy_from(xe, msg, cmds, head * sizeof(u32),
+			   1 * sizeof(u32));
+	action = FIELD_GET(GUC_HXG_REQUEST_MSG_0_ACTION, msg[0]);
+	switch (action) {
+	case XE_GUC_ACTION_REGISTER_CONTEXT:
+	case XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC:
+		xe_map_fixup_u64(xe, cmds, head,
+				 XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC, shift);
+		xe_map_fixup_u64(xe, cmds, head,
+				 XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE, shift);
+		if (action == XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC) {
+			n = xe_map_rd_u32(xe, cmds, head,
+				       XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN);
+			for (i = 0; i < n; i++)
+				xe_map_fixup_u64(xe, cmds, head,
+					    XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA
+					    + 2 * i, shift);
+		} else {
+			xe_map_fixup_u64(xe, cmds, head, 10, shift);
+		}
+		break;
+	default:
+		break;
+	}
+}
+
+static int ct_update_addresses_in_buffer(struct xe_guc_ct *ct,
+					 struct guc_ctb *h2g,
+					 s64 shift, u32 *mhead, s32 avail)
+{
+	struct xe_device *xe = ct_to_xe(ct);
+	u32 head = *mhead;
+	u32 size = h2g->info.size;
+	u32 msg[1];
+	u32 len;
+
+	/* Read header */
+	xe_map_memcpy_from(xe, msg, &h2g->cmds, sizeof(u32) * head,
+			   sizeof(u32));
+	len = FIELD_GET(GUC_CTB_MSG_0_NUM_DWORDS, msg[0]) + GUC_CTB_MSG_MIN_LEN;
+
+	if (unlikely(len > (u32)avail)) {
+		xe_gt_err(ct_to_gt(ct), "H2G channel broken on read, avail=%d, len=%d, fixups skipped\n",
+			  avail, len);
+		return 0;
+	}
+
+	head = (head + 1) % size;
+	ct_update_addresses_in_message(ct, &h2g->cmds, head, len - 1, size, shift);
+	*mhead = (head + len - 1) % size;
+
+	return avail - len;
+}
+
+/**
+ * xe_guc_ct_fixup_messages_with_ggtt - Fixup any pending H2G CTB messages by updating
+ * GGTT offsets in their payloads.
+ * @ct: pointer to CT struct of the target GuC
+ * @ggtt_shift: shift to be added to all GGTT addresses within the CTB
+ */
+void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift)
+{
+	struct xe_guc *guc = ct_to_guc(ct);
+	struct xe_gt *gt = guc_to_gt(guc);
+	struct guc_ctb *h2g = &ct->ctbs.h2g;
+	u32 head, tail, size;
+	s32 avail;
+
+	if (unlikely(h2g->info.broken))
+		return;
+
+	h2g->info.head = desc_read(ct_to_xe(ct), h2g, head);
+	head = h2g->info.head;
+	tail = READ_ONCE(h2g->info.tail);
+	size = h2g->info.size;
+
+	if (unlikely(head > size))
+		goto corrupted;
+
+	if (unlikely(tail >= size))
+		goto corrupted;
+
+	avail = tail - head;
+
+	/* beware of buffer wrap case */
+	if (unlikely(avail < 0))
+		avail += size;
+	xe_gt_dbg(gt, "available %d (%u:%u:%u)\n", avail, head, tail, size);
+	xe_gt_assert(gt, avail >= 0);
+
+	while (avail > 0)
+		avail = ct_update_addresses_in_buffer(ct, h2g, ggtt_shift, &head, avail);
+
+	return;
+
+corrupted:
+	xe_gt_err(gt, "Corrupted H2G descriptor head=%u tail=%u size=%u, fixups not applied\n",
+		 head, tail, size);
+	h2g->info.broken = true;
+}
+
 static struct xe_guc_ct_snapshot *guc_ct_snapshot_alloc(struct xe_guc_ct *ct, bool atomic,
 							bool want_ctb)
 {
drivers/gpu/drm/xe/xe_guc_ct.h +2 -0
index 82c4ae458dda..5649bda82823 100644
@@ -22,6 +22,8 @@ void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot, struct drm_pr
 void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot);
 void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p, bool want_ctb);
 
+void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift);
+
 static inline bool xe_guc_ct_enabled(struct xe_guc_ct *ct)
 {
 	return ct->state == XE_GUC_CT_STATE_ENABLED;
drivers/gpu/drm/xe/xe_sriov_vf.c +25 -0
index 2eb6b8d8a217..9a1b578fdd03 100644
@@ -10,6 +10,7 @@
 #include "xe_gt.h"
 #include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
+#include "xe_guc_ct.h"
 #include "xe_pm.h"
 #include "xe_sriov.h"
 #include "xe_sriov_printk.h"
@@ -158,6 +159,27 @@ static int vf_post_migration_requery_guc(struct xe_device *xe)
 	return ret;
 }
 
+static s32 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)
+{
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+
+	return config->ggtt_shift;
+}
+
+static void vf_post_migration_fixup_ctb(struct xe_device *xe)
+{
+	struct xe_gt *gt;
+	unsigned int id;
+
+	xe_assert(xe, IS_SRIOV_VF(xe));
+
+	for_each_gt(gt, xe, id) {
+		s32 shift = xe_gt_sriov_vf_ggtt_shift(gt);
+
+		xe_guc_ct_fixup_messages_with_ggtt(&gt->uc.guc.ct, shift);
+	}
+}
+
 /*
  * vf_post_migration_imminent - Check if post-restore recovery is coming.
  * @xe: the &xe_device struct instance
@@ -219,6 +241,9 @@ static void vf_post_migration_recovery(struct xe_device *xe)
 
 	need_fixups = vf_post_migration_fixup_ggtt_nodes(xe);
 	/* FIXME: add the recovery steps */
+	if (need_fixups)
+		vf_post_migration_fixup_ctb(xe);
+
 	vf_post_migration_notify_resfix_done(xe);
 	xe_pm_runtime_put(xe);
 	drm_notice(&xe->drm, "migration recovery ended\n");
Signature · 1 line
2.25.1
#11 lore raw

Re: [PATCH v6 4/4] drm/xe/vf: Fixup CTB send buffer messages after migration

Quoted 11 lines · Tomasz Lis

During post-migration recovery of a VF, it is necessary to update GGTT references included in messages which are going to be sent to GuC. GuC will start consuming messages after VF KMD will inform it about fixups being done; before that, the VF KMD is expected to update any H2G messages which are already in send buffer but were not consumed by GuC.

Only a small subset of messages allowed for VFs have GGTT references in them. This patch adds the functionality to parse the CTB send ring buffer and shift addresses contained within.

Quoted 7 lines

While fixing the CTB content, ct->lock is not taken. This means the only barrier taken remains GGTT address lock - which is ok, because only requests with GGTT addresses matter, but it also means tail changes can happen during the CTB fixups execution (which may be ignored as any new messages will not have anything to fix).

The GGTT address locking will be introduced in a future series.

can you add some FIXME to the code to at least mark places with this missing locking schema?

Quoted 30 lines

v2: removed storing shift as that's now done in VMA nodes patch;
macros to inlines; warns to asserts; log messages fixes (Michal)
v3: removed inline keywords, enums for offsets in CTB messages,
less error messages, if return unused then made functs void (Michal)
v4: update the cached head before starting fixups
v5: removed/updated comments, wrapped lines, converted assert into
error, enums for offsets to separate patch, reused xe_map_rd

---
 drivers/gpu/drm/xe/xe_guc_ct.c   | 142 +++++++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_guc_ct.h   |   2 +
 drivers/gpu/drm/xe/xe_sriov_vf.c |  25 ++++++
 3 files changed, 169 insertions(+)
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
index 686fe664c20d..6c80106e5e8b 100644
@@ -84,6 +84,8 @@ struct g2h_fence {
 	bool done;
 };
 
+#define make_u64(hi, lo) ((u64)((u64)(u32)(hi) << 32 | (u32)(lo)))
+
 static void g2h_fence_init(struct g2h_fence *g2h_fence, u32 *response_buffer)
 {
 	g2h_fence->response_buffer = response_buffer;

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1631

Quoted 8 lines
drivers/gpu/drm/xe/xe_guc_ct.c +5 -0
@@ -1622,6 +1624,146 @@ static void g2h_worker_func(struct work_struct *w)
 	receive_g2h(ct);
 }
 
+static u32 xe_map_rd_u32(struct xe_device *xe, struct iosys_map *cmds,
+			     u32 head, u32 pos)
+{
+	return xe_map_rd(xe, cmds, (head + pos) * sizeof(u32), u32);
+}

this helper as-is is little confusing as it mimics the xe_map_rd() but changes the meaning of the offset to index, so maybe we need:

#define xe_map_rd_array(xe, map, index, type) \
	xe_map_rd((xe), (map), (index) * sizeof(type), (type))

and

#define xe_map_rd_array_u32(xe, map, index) \
	xe_map_rd_array((xe), (map), (index), u32)

both defined in xe_map.h and then use:

	foo = xe_map_rd_array_u32(xe, cmds, head + pos);

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1634

Quoted 3 lines
drivers/gpu/drm/xe/xe_guc_ct.c +3 -0
+
+static void xe_map_fixup_u64(struct xe_device *xe, struct iosys_map *cmds,
+			       u32 head, u32 pos, s64 shift)

hmm, this can't be "xe_map" generic function, see below

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1640

Quoted 6 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6 -0
+{
+	u32 msg[2];
+	u64 offset;
+
+	xe_map_memcpy_from(xe, msg, cmds, (head + pos) * sizeof(u32),
+			   2 * sizeof(u32));

in general *) CTB messages could be wrapped at the buffer edge so you shouldn't assume that both dwords will be always together, and in other places you seem to take care of this case, so maybe do this as:

	lo = xe_map_rd_array_u32(xe, cmds, (head + pos) % size);
	hi = xe_map_rd_array_u32(xe, cmds, (head + pos + 1) % size);
	...
	offset = make_u64(hi, lo) + shift;
	lo = lower_32_bits(offset);
	hi = upper_32_bits(offset);
	...
	xe_map_wr_array_u32(xe, cmds, (head + pos) % size, lo);
	xe_map_wr_array_u32(xe, cmds, (head + pos + 1) % size, hi);

*) currently xe is filling CTB with undocumented NOP messages, but that could/should be changed any time, since it is sub-optimal in case of sending larger VFPF CTB messages

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1652

Quoted 12 lines
drivers/gpu/drm/xe/xe_guc_ct.c +12 -0
+	offset = make_u64(msg[1], msg[0]);
+	offset += shift;
+	msg[0] = lower_32_bits(offset);
+	msg[1] = upper_32_bits(offset);
+	xe_map_memcpy_to(xe, cmds, (head + pos) * sizeof(u32), msg, 2 * sizeof(u32));
+}
+
+/*
+ * Shift any GGTT addresses within a single message left within CTB from
+ * before post-migration recovery.
+ * @ct: pointer to CT struct of the target GuC
+ * @cmds: iomap buffer containing CT messages

we don't need to pass this, as it is always has to be ct->h2g.cmds, no?

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1655

Quoted 3 lines
drivers/gpu/drm/xe/xe_guc_ct.c +3 -0
+ * @head: start of the target message within the buffer
+ * @len: length of the target message
+ * @size: size of the commands buffer

same here

Quoted 11 lines
drivers/gpu/drm/xe/xe_guc_ct.c +11 -0
+ * @shift: the address shift to be added to each GGTT reference
+ */
+static void ct_update_addresses_in_message(struct xe_guc_ct *ct,
+					   struct iosys_map *cmds, u32 head,
+					   u32 len, u32 size, s64 shift)
+{
+	struct xe_device *xe = ct_to_xe(ct);
+	u32 action, i, n;
+	u32 msg[1];
+
+	xe_map_memcpy_from(xe, msg, cmds, head * sizeof(u32),

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1676

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +10 -0
+			   1 * sizeof(u32));
+	action = FIELD_GET(GUC_HXG_REQUEST_MSG_0_ACTION, msg[0]);
+	switch (action) {
+	case XE_GUC_ACTION_REGISTER_CONTEXT:
+	case XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC:
+		xe_map_fixup_u64(xe, cmds, head,
+				 XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC, shift);
+		xe_map_fixup_u64(xe, cmds, head,
+				 XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE, shift);
+		if (action == XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC) {

hmm, that looks ugly, maybe do this part in MRLC case and then fall into common case like:

	case XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC:
		n = ct_read_h2g(ct, head + N_CHILDREN);
		for (i = 0; i < n; i++)
			ct_fixup_u64(xe, head + HWLRCA + 2 * i, shift);
		fallthrough;
	case XE_GUC_ACTION_REGISTER_CONTEXT:
		ct_fixup_u64(xe, head + WQ_DESC, shift);
		ct_fixup_u64(xe, head + WQ_BASE, shift);
		break;

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1678

Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
+			n = xe_map_rd_u32(xe, cmds, head,
+				       XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN);

beware of the wrap!

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1684

Quoted 6 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6 -0
+			for (i = 0; i < n; i++)
+				xe_map_fixup_u64(xe, cmds, head,
+					    XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA
+					    + 2 * i, shift);
+		} else {
+			xe_map_fixup_u64(xe, cmds, head, 10, shift);

magic "10"

Quoted 28 lines
drivers/gpu/drm/xe/xe_guc_ct.c +28 -0
+		}
+		break;
+	default:
+		break;
+	}
+}
+
+static int ct_update_addresses_in_buffer(struct xe_guc_ct *ct,
+					 struct guc_ctb *h2g,
+					 s64 shift, u32 *mhead, s32 avail)
+{
+	struct xe_device *xe = ct_to_xe(ct);
+	u32 head = *mhead;
+	u32 size = h2g->info.size;
+	u32 msg[1];
+	u32 len;
+
+	/* Read header */
+	xe_map_memcpy_from(xe, msg, &h2g->cmds, sizeof(u32) * head,
+			   sizeof(u32));
+	len = FIELD_GET(GUC_CTB_MSG_0_NUM_DWORDS, msg[0]) + GUC_CTB_MSG_MIN_LEN;
+
+	if (unlikely(len > (u32)avail)) {
+		xe_gt_err(ct_to_gt(ct), "H2G channel broken on read, avail=%d, len=%d, fixups skipped\n",
+			  avail, len);
+		return 0;
+	}
+

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1722

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +10 -0
+	head = (head + 1) % size;
+	ct_update_addresses_in_message(ct, &h2g->cmds, head, len - 1, size, shift);
+	*mhead = (head + len - 1) % size;
+
+	return avail - len;
+}
+
+/**
+ * xe_guc_ct_fixup_messages_with_ggtt - Fixup any pending H2G CTB messages by updating
+ * GGTT offsets in their payloads.

nit: "by ..." part is more appropriate for the 'longer description' section of the function kernel-doc

Quoted 73 lines
drivers/gpu/drm/xe/xe_guc_ct.c +44 -0
+ * @ct: pointer to CT struct of the target GuC
+ * @ggtt_shift: shift to be added to all GGTT addresses within the CTB
+ */
+void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift)
+{
+	struct xe_guc *guc = ct_to_guc(ct);
+	struct xe_gt *gt = guc_to_gt(guc);
+	struct guc_ctb *h2g = &ct->ctbs.h2g;
+	u32 head, tail, size;
+	s32 avail;
+
+	if (unlikely(h2g->info.broken))
+		return;
+
+	h2g->info.head = desc_read(ct_to_xe(ct), h2g, head);
+	head = h2g->info.head;
+	tail = READ_ONCE(h2g->info.tail);
+	size = h2g->info.size;
+
+	if (unlikely(head > size))
+		goto corrupted;
+
+	if (unlikely(tail >= size))
+		goto corrupted;
+
+	avail = tail - head;
+
+	/* beware of buffer wrap case */
+	if (unlikely(avail < 0))
+		avail += size;
+	xe_gt_dbg(gt, "available %d (%u:%u:%u)\n", avail, head, tail, size);
+	xe_gt_assert(gt, avail >= 0);
+
+	while (avail > 0)
+		avail = ct_update_addresses_in_buffer(ct, h2g, ggtt_shift, &head, avail);
+
+	return;
+
+corrupted:
+	xe_gt_err(gt, "Corrupted H2G descriptor head=%u tail=%u size=%u, fixups not applied\n",
+		 head, tail, size);
+	h2g->info.broken = true;
+}
+
 static struct xe_guc_ct_snapshot *guc_ct_snapshot_alloc(struct xe_guc_ct *ct, bool atomic,
 							bool want_ctb)
 {
drivers/gpu/drm/xe/xe_guc_ct.h +2 -0
index 82c4ae458dda..5649bda82823 100644
@@ -22,6 +22,8 @@ void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot, struct drm_pr
 void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot);
 void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p, bool want_ctb);
 
+void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift);
+
 static inline bool xe_guc_ct_enabled(struct xe_guc_ct *ct)
 {
 	return ct->state == XE_GUC_CT_STATE_ENABLED;
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
index 2eb6b8d8a217..9a1b578fdd03 100644
@@ -10,6 +10,7 @@
 #include "xe_gt.h"
 #include "xe_gt_sriov_printk.h"
 #include "xe_gt_sriov_vf.h"
+#include "xe_guc_ct.h"
 #include "xe_pm.h"
 #include "xe_sriov.h"
 #include "xe_sriov_printk.h"

drivers/gpu/drm/xe/xe_sriov_vf.c static int vf_post_migration_requery_guc(struct xe_device *xe) · line 162

Quoted 4 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
@@ -158,6 +159,27 @@ static int vf_post_migration_requery_guc(struct xe_device *xe)
 	return ret;
 }
 
+static s32 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)

this function shall be defined in xe_gt_sriov_vf.h|c

Quoted 33 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +23 -0
+{
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+
+	return config->ggtt_shift;
+}
+
+static void vf_post_migration_fixup_ctb(struct xe_device *xe)
+{
+	struct xe_gt *gt;
+	unsigned int id;
+
+	xe_assert(xe, IS_SRIOV_VF(xe));
+
+	for_each_gt(gt, xe, id) {
+		s32 shift = xe_gt_sriov_vf_ggtt_shift(gt);
+
+		xe_guc_ct_fixup_messages_with_ggtt(&gt->uc.guc.ct, shift);
+	}
+}
+
 /*
  * vf_post_migration_imminent - Check if post-restore recovery is coming.
  * @xe: the &xe_device struct instance
@@ -219,6 +241,9 @@ static void vf_post_migration_recovery(struct xe_device *xe)
 
 	need_fixups = vf_post_migration_fixup_ggtt_nodes(xe);
 	/* FIXME: add the recovery steps */
+	if (need_fixups)
+		vf_post_migration_fixup_ctb(xe);
+
 	vf_post_migration_notify_resfix_done(xe);
 	xe_pm_runtime_put(xe);
 	drm_notice(&xe->drm, "migration recovery ended\n");
#12 lore raw

Re: [PATCH v6 4/4] drm/xe/vf: Fixup CTB send buffer messages after migration

Quoted 18 lines · Michal Wajdeczko
Quoted 17 lines · Tomasz Lis

During post-migration recovery of a VF, it is necessary to update GGTT references included in messages which are going to be sent to GuC. GuC will start consuming messages after VF KMD will inform it about fixups being done; before that, the VF KMD is expected to update any H2G messages which are already in send buffer but were not consumed by GuC.

Only a small subset of messages allowed for VFs have GGTT references in them. This patch adds the functionality to parse the CTB send ring buffer and shift addresses contained within.

While fixing the CTB content, ct->lock is not taken. This means the only barrier taken remains GGTT address lock - which is ok, because only requests with GGTT addresses matter, but it also means tail changes can happen during the CTB fixups execution (which may be ignored as any new messages will not have anything to fix).

Quoted 3 lines
Quoted 1 line

The GGTT address locking will be introduced in a future series.

can you add some FIXME to the code to at least mark places with this missing locking schema?

This patch doesn't have much to do with the locking.

Are you suggesting a wider change of introducing FIXMEs to the code for each part of every new functionality accepted by architecture?

Treating ggtt locking differently than anything else seem questionable.

Also, what do you mean by "at least" here?

Quoted 43 lines
Quoted 29 lines

v2: removed storing shift as that's now done in VMA nodes patch;
macros to inlines; warns to asserts; log messages fixes (Michal)
v3: removed inline keywords, enums for offsets in CTB messages,
less error messages, if return unused then made functs void (Michal)
v4: update the cached head before starting fixups
v5: removed/updated comments, wrapped lines, converted assert into
error, enums for offsets to separate patch, reused xe_map_rd

---
  drivers/gpu/drm/xe/xe_guc_ct.c   | 142 +++++++++++++++++++++++++++++++
  drivers/gpu/drm/xe/xe_guc_ct.h   |   2 +
  drivers/gpu/drm/xe/xe_sriov_vf.c |  25 ++++++
  3 files changed, 169 insertions(+)
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
index 686fe664c20d..6c80106e5e8b 100644
@@ -84,6 +84,8 @@ struct g2h_fence {
  	bool done;
  };
 
+#define make_u64(hi, lo) ((u64)((u64)(u32)(hi) << 32 | (u32)(lo)))
+
  static void g2h_fence_init(struct g2h_fence *g2h_fence, u32 *response_buffer)
  {
  	g2h_fence->response_buffer = response_buffer;

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1631

Quoted 8 lines
drivers/gpu/drm/xe/xe_guc_ct.c +5 -0
@@ -1622,6 +1624,146 @@ static void g2h_worker_func(struct work_struct *w)
  	receive_g2h(ct);
  }
 
+static u32 xe_map_rd_u32(struct xe_device *xe, struct iosys_map *cmds,
+			     u32 head, u32 pos)
+{
+	return xe_map_rd(xe, cmds, (head + pos) * sizeof(u32), u32);
+}

this helper as-is is little confusing as it mimics the xe_map_rd() but changes the meaning of the offset to index, so maybe we need:

#define xe_map_rd_array(xe, map, index, type) \
	xe_map_rd((xe), (map), (index) * sizeof(type), (type))
Quoted 8 lines

and

#define xe_map_rd_array_u32(xe, map, index) \
	xe_map_rd_array((xe), (map), (index), u32)

both defined in xe_map.h and then use:

	foo = xe_map_rd_array_u32(xe, cmds, head + pos);

will do.

Quoted 26 lines

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1634

Quoted 3 lines
drivers/gpu/drm/xe/xe_guc_ct.c +3 -0
+
+static void xe_map_fixup_u64(struct xe_device *xe, struct iosys_map *cmds,
+			       u32 head, u32 pos, s64 shift)

hmm, this can't be "xe_map" generic function, see below

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1640

Quoted 6 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6 -0
+{
+	u32 msg[2];
+	u64 offset;
+
+	xe_map_memcpy_from(xe, msg, cmds, (head + pos) * sizeof(u32),
+			   2 * sizeof(u32));

in general *) CTB messages could be wrapped at the buffer edge so you shouldn't assume that both dwords will be always together, and in other places you seem to take care of this case, so maybe do this as:

	lo = xe_map_rd_array_u32(xe, cmds, (head + pos) % size);
	hi = xe_map_rd_array_u32(xe, cmds, (head + pos + 1) % size);
	...
	offset = make_u64(hi, lo) + shift;
	lo = lower_32_bits(offset);
	hi = upper_32_bits(offset);
	...
	xe_map_wr_array_u32(xe, cmds, (head + pos) % size, lo);
	xe_map_wr_array_u32(xe, cmds, (head + pos + 1) % size, hi);
Quoted 3 lines

*) currently xe is filling CTB with undocumented NOP messages, but that could/should be changed any time, since it is sub-optimal in case of sending larger VFPF CTB messages

Right, I messed it up when moving from normal memory access to those xe_map wrappers.

Will change.

Quoted 5 lines
Quoted 5 lines
drivers/gpu/drm/xe/xe_guc_ct.c +5 -0
+	offset = make_u64(msg[1], msg[0]);
+	offset += shift;
+	msg[0] = lower_32_bits(offset);
+	msg[1] = upper_32_bits(offset);
+	xe_map_memcpy_to(xe, cmds, (head + pos) * sizeof(u32), msg, 2 * sizeof(u32));
Quoted 8 lines

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1652

Quoted 7 lines
drivers/gpu/drm/xe/xe_guc_ct.c +7 -0
+}
+
+/*
+ * Shift any GGTT addresses within a single message left within CTB from
+ * before post-migration recovery.
+ * @ct: pointer to CT struct of the target GuC
+ * @cmds: iomap buffer containing CT messages

we don't need to pass this, as it is always has to be ct->h2g.cmds, no?

That's true, we don't. Same as with ggtt_shift which you insisted on passing to xe_gt_sriov_vf_fixup_ggtt_nodes.

But for the similar reason, we are passing it, now in both these cases.

Quoted 4 lines

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1655

Quoted 3 lines
drivers/gpu/drm/xe/xe_guc_ct.c +3 -0
+ * @head: start of the target message within the buffer
+ * @len: length of the target message
+ * @size: size of the commands buffer

same here

as before, this is single message level and this parameter is required for logical consistency in layering of functions.

Quoted 24 lines
Quoted 11 lines
drivers/gpu/drm/xe/xe_guc_ct.c +11 -0
+ * @shift: the address shift to be added to each GGTT reference
+ */
+static void ct_update_addresses_in_message(struct xe_guc_ct *ct,
+					   struct iosys_map *cmds, u32 head,
+					   u32 len, u32 size, s64 shift)
+{
+	struct xe_device *xe = ct_to_xe(ct);
+	u32 action, i, n;
+	u32 msg[1];
+
+	xe_map_memcpy_from(xe, msg, cmds, head * sizeof(u32),

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1676

Quoted 10 lines
drivers/gpu/drm/xe/xe_guc_ct.c +10 -0
+			   1 * sizeof(u32));
+	action = FIELD_GET(GUC_HXG_REQUEST_MSG_0_ACTION, msg[0]);
+	switch (action) {
+	case XE_GUC_ACTION_REGISTER_CONTEXT:
+	case XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC:
+		xe_map_fixup_u64(xe, cmds, head,
+				 XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_DESC, shift);
+		xe_map_fixup_u64(xe, cmds, head,
+				 XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_WQ_BASE, shift);
+		if (action == XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC) {

hmm, that looks ugly, maybe do this part in MRLC case and then fall into common case like:

Quoted 10 lines
	case XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC:
		n = ct_read_h2g(ct, head + N_CHILDREN);
		for (i = 0; i < n; i++)
			ct_fixup_u64(xe, head + HWLRCA + 2 * i, shift);
		fallthrough;
	case XE_GUC_ACTION_REGISTER_CONTEXT:
		ct_fixup_u64(xe, head + WQ_DESC, shift);
		ct_fixup_u64(xe, head + WQ_BASE, shift);
		break;

and then if (action == XE_GUC_ACTION_REGISTER_CONTEXT_MULTI_LRC) would be still required because it has else, everything would be just more convoluted.

I'd prefer not.

Quoted 3 lines

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1678

Quoted 2 lines
drivers/gpu/drm/xe/xe_guc_ct.c +2 -0
+			n = xe_map_rd_u32(xe, cmds, head,
+				       XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_N_CHILDREN);

beware of the wrap!

I don't understand.

The wrap is needed because of these cartoonishly long enum members which I never voted for.

Quoted 7 lines

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1684

Quoted 6 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6 -0
+			for (i = 0; i < n; i++)
+				xe_map_fixup_u64(xe, cmds, head,
+					    XE_GUC_REGISTER_CONTEXT_MULTI_LRC_OFFS_HWLRCA
+					    + 2 * i, shift);
+		} else {
+			xe_map_fixup_u64(xe, cmds, head, 10, shift);

magic "10"

I assume you're asking for creation of another set of those XE_GUC_REGISTER_CONTEXT_ defines.

I disagree these are making the code clearer. But will comply.

Quoted 32 lines
Quoted 32 lines
drivers/gpu/drm/xe/xe_guc_ct.c +32 -0
+		}
+		break;
+	default:
+		break;
+	}
+}
+
+static int ct_update_addresses_in_buffer(struct xe_guc_ct *ct,
+					 struct guc_ctb *h2g,
+					 s64 shift, u32 *mhead, s32 avail)
+{
+	struct xe_device *xe = ct_to_xe(ct);
+	u32 head = *mhead;
+	u32 size = h2g->info.size;
+	u32 msg[1];
+	u32 len;
+
+	/* Read header */
+	xe_map_memcpy_from(xe, msg, &h2g->cmds, sizeof(u32) * head,
+			   sizeof(u32));
+	len = FIELD_GET(GUC_CTB_MSG_0_NUM_DWORDS, msg[0]) + GUC_CTB_MSG_MIN_LEN;
+
+	if (unlikely(len > (u32)avail)) {
+		xe_gt_err(ct_to_gt(ct), "H2G channel broken on read, avail=%d, len=%d, fixups skipped\n",
+			  avail, len);
+		return 0;
+	}
+
+	head = (head + 1) % size;
+	ct_update_addresses_in_message(ct, &h2g->cmds, head, len - 1, size, shift);
+	*mhead = (head + len - 1) % size;
+
Quoted 8 lines

drivers/gpu/drm/xe/xe_guc_ct.c static void g2h_worker_func(struct work_struct *w) · line 1722

Quoted 6 lines
drivers/gpu/drm/xe/xe_guc_ct.c +6 -0
+	return avail - len;
+}
+
+/**
+ * xe_guc_ct_fixup_messages_with_ggtt - Fixup any pending H2G CTB messages by updating
+ * GGTT offsets in their payloads.

nit: "by ..." part is more appropriate for the 'longer description' section of the function kernel-doc

right, will update.

Quoted 76 lines
Quoted 76 lines
drivers/gpu/drm/xe/xe_guc_ct.c +44 -0
+ * @ct: pointer to CT struct of the target GuC
+ * @ggtt_shift: shift to be added to all GGTT addresses within the CTB
+ */
+void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift)
+{
+	struct xe_guc *guc = ct_to_guc(ct);
+	struct xe_gt *gt = guc_to_gt(guc);
+	struct guc_ctb *h2g = &ct->ctbs.h2g;
+	u32 head, tail, size;
+	s32 avail;
+
+	if (unlikely(h2g->info.broken))
+		return;
+
+	h2g->info.head = desc_read(ct_to_xe(ct), h2g, head);
+	head = h2g->info.head;
+	tail = READ_ONCE(h2g->info.tail);
+	size = h2g->info.size;
+
+	if (unlikely(head > size))
+		goto corrupted;
+
+	if (unlikely(tail >= size))
+		goto corrupted;
+
+	avail = tail - head;
+
+	/* beware of buffer wrap case */
+	if (unlikely(avail < 0))
+		avail += size;
+	xe_gt_dbg(gt, "available %d (%u:%u:%u)\n", avail, head, tail, size);
+	xe_gt_assert(gt, avail >= 0);
+
+	while (avail > 0)
+		avail = ct_update_addresses_in_buffer(ct, h2g, ggtt_shift, &head, avail);
+
+	return;
+
+corrupted:
+	xe_gt_err(gt, "Corrupted H2G descriptor head=%u tail=%u size=%u, fixups not applied\n",
+		 head, tail, size);
+	h2g->info.broken = true;
+}
+
  static struct xe_guc_ct_snapshot *guc_ct_snapshot_alloc(struct xe_guc_ct *ct, bool atomic,
  							bool want_ctb)
  {
drivers/gpu/drm/xe/xe_guc_ct.h +2 -0
index 82c4ae458dda..5649bda82823 100644
@@ -22,6 +22,8 @@ void xe_guc_ct_snapshot_print(struct xe_guc_ct_snapshot *snapshot, struct drm_pr
  void xe_guc_ct_snapshot_free(struct xe_guc_ct_snapshot *snapshot);
  void xe_guc_ct_print(struct xe_guc_ct *ct, struct drm_printer *p, bool want_ctb);
 
+void xe_guc_ct_fixup_messages_with_ggtt(struct xe_guc_ct *ct, s64 ggtt_shift);
+
  static inline bool xe_guc_ct_enabled(struct xe_guc_ct *ct)
  {
  	return ct->state == XE_GUC_CT_STATE_ENABLED;
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
index 2eb6b8d8a217..9a1b578fdd03 100644
@@ -10,6 +10,7 @@
  #include "xe_gt.h"
  #include "xe_gt_sriov_printk.h"
  #include "xe_gt_sriov_vf.h"
+#include "xe_guc_ct.h"
  #include "xe_pm.h"
  #include "xe_sriov.h"
  #include "xe_sriov_printk.h"
@@ -158,6 +159,27 @@ static int vf_post_migration_requery_guc(struct xe_device *xe)
  	return ret;
  }
 
Quoted 2 lines

drivers/gpu/drm/xe/xe_sriov_vf.c static int vf_post_migration_requery_guc(struct xe_device *xe) · line 162

Quoted 1 line
drivers/gpu/drm/xe/xe_sriov_vf.c +1 -0
+static s32 xe_gt_sriov_vf_ggtt_shift(struct xe_gt *gt)

this function shall be defined in xe_gt_sriov_vf.h|c

ack

-Tomasz

Quoted 33 lines
Quoted 33 lines
drivers/gpu/drm/xe/xe_sriov_vf.c +23 -0
+{
+	struct xe_gt_sriov_vf_selfconfig *config = &gt->sriov.vf.self_config;
+
+	return config->ggtt_shift;
+}
+
+static void vf_post_migration_fixup_ctb(struct xe_device *xe)
+{
+	struct xe_gt *gt;
+	unsigned int id;
+
+	xe_assert(xe, IS_SRIOV_VF(xe));
+
+	for_each_gt(gt, xe, id) {
+		s32 shift = xe_gt_sriov_vf_ggtt_shift(gt);
+
+		xe_guc_ct_fixup_messages_with_ggtt(&gt->uc.guc.ct, shift);
+	}
+}
+
  /*
   * vf_post_migration_imminent - Check if post-restore recovery is coming.
   * @xe: the &xe_device struct instance
@@ -219,6 +241,9 @@ static void vf_post_migration_recovery(struct xe_device *xe)
 
  	need_fixups = vf_post_migration_fixup_ggtt_nodes(xe);
  	/* FIXME: add the recovery steps */
+	if (need_fixups)
+		vf_post_migration_fixup_ctb(xe);
+
  	vf_post_migration_notify_resfix_done(xe);
  	xe_pm_runtime_put(xe);
  	drm_notice(&xe->drm, "migration recovery ended\n");
#13 lore raw
In reply to Tomasz Lis

Hi Tomasz,

This series should be primary sent to the intel-xe as series is xe specific and it's the only way to get CI results for the xe driver.

Once you dropped the drm_mm patch, the dri-devel is not required.

Quoted 39 lines · Tomasz Lis

To support VF Migration, it is necessary to do fixups to any non-virtualized resources. These fixups need to be applied within VM, on the KMD working with VF.

This series adds two fixup functions to the recovery worker:
* for fixing drm_mm nodes which represent GGTT allocations
* for fixing content of outgoing CTB buffer

v2: Fixed missing include, made checkpatch happy
v3: Moved some functs to xe_ggtt.c; moved shift computation to just
after querying; improved documentation; switched some warns to asserts; skipping fixups when GGTT shift eq 0; iterating through tiles rather than gts; fixes in log messages
v4: Updated kerneldocs, removed unused funct, properly allocate
balloning nodes if non existent, removed inline keywords, enums for offsets in CTB messages, less error messages, if return unused then made functs void
v5: Removed drm_mm change, but added VF init modifications. These then
allowed to re-use ballooning functions during ggtt node fixing.
v6: Minor update - fixed some misplaced funct declarations

Tomasz Lis (4):
drm/xe/vf: Divide ballooning into allocation and insertion drm/xe/sriov: Shifting GGTT area post migration drm/xe/guc: Introduce enum with offsets for multi-LRC register H2G drm/xe/vf: Fixup CTB send buffer messages after migration

 drivers/gpu/drm/xe/abi/guc_actions_abi.h  |   7 ++
 drivers/gpu/drm/xe/xe_ggtt.c              |  40 ++++--
 drivers/gpu/drm/xe/xe_ggtt.h              |   1 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf.c       | 147 +++++++++++++++++++---
 drivers/gpu/drm/xe/xe_gt_sriov_vf.h       |   3 +
 drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h |   2 +
 drivers/gpu/drm/xe/xe_guc_ct.c            | 142 +++++++++++++++++++++
 drivers/gpu/drm/xe/xe_guc_ct.h            |   2 +
 drivers/gpu/drm/xe/xe_guc_submit.c        |   4 +
 drivers/gpu/drm/xe/xe_sriov_vf.c          |  42 +++++++
 10 files changed, 368 insertions(+), 22 deletions(-)