Skip to contentSkip to conversation

drm/xe: Per client usage

PATCH Superseded Lucas De Marchi 4 participants 15 messages May 7, 2024 – May 8, 2024

Subsystems (4)

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore
#0 lore raw

v3 cover drm/xe: Per client usage

Add per-client usage statistics to xe. This ports xe to use the common method in drm to export the usage to userspace per client (where 1 client == 1 drm fd open).

However instead of using the current format measured in nsec, this creates a new one. The intention here is not to mix the GPU clock domain with the CPU clock. It allows to cover a few more use cases without extra complications.

I tested this on DG2 and also checked gputop with i915 to make sure not regressed. Last patch also contains the documentation for the new key and sample output as requested in v1. Reproducing it partially here:

- drm-total-cycles-<keystr>: <uint>

Engine identifier string must be the same as the one specified in the drm-cycles-<keystr> tag and shall contain the total number cycles for the given engine.

This is a timestamp in GPU unspecified unit that matches the update rate of drm-cycles-<keystr>. For drivers that implement this interface, the engine utilization can be calculated entirely on the GPU clock domain, without considering the CPU sleep time between 2 samples.

The pre-existent drm-cycles-<keystr> is used as is, which allows gputop to work with xe.

This last patch still has some open discussion from v2, so we may need to hold it a little more.

Test-with: [email protected]

v2:
- Create a new drm-total-cycles instead of re-using drm-engine with a different unit
- Add documentation for the new interface and clarify usage of xe_lrc_update_timestamp()

v3:
- Fix bugs in "drm/xe: Add helper to accumulate exec queue runtime" - see commit message
- Reorder commits so the ones that are useful in other patch series come first

Lucas De Marchi (4):
drm/xe: Promote xe_hw_engine_class_to_str() drm/xe: Add XE_ENGINE_CLASS_OTHER to str conversion drm/xe: Add helper to capture engine timestamp drm/xe/client: Print runtime to fdinfo

Umesh Nerlige Ramappa (2):
drm/xe/lrc: Add helper to capture context timestamp drm/xe: Add helper to accumulate exec queue runtime

 Documentation/gpu/drm-usage-stats.rst         |  16 ++-
 Documentation/gpu/xe/index.rst                |   1 +
 Documentation/gpu/xe/xe-drm-usage-stats.rst   |  10 ++
 drivers/gpu/drm/xe/regs/xe_lrc_layout.h       |   1 +
 drivers/gpu/drm/xe/xe_device_types.h          |   9 ++
 drivers/gpu/drm/xe/xe_drm_client.c            | 136 +++++++++++++++++-
 drivers/gpu/drm/xe/xe_exec_queue.c            |  35 +++++
 drivers/gpu/drm/xe/xe_exec_queue.h            |   1 +
 drivers/gpu/drm/xe/xe_execlist.c              |   1 +
 drivers/gpu/drm/xe/xe_guc_submit.c            |   2 +
 drivers/gpu/drm/xe/xe_hw_engine.c             |  27 ++++
 drivers/gpu/drm/xe/xe_hw_engine.h             |   3 +
 drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c |  18 ---
 drivers/gpu/drm/xe/xe_lrc.c                   |  11 ++
 drivers/gpu/drm/xe/xe_lrc.h                   |  14 ++
 drivers/gpu/drm/xe/xe_lrc_types.h             |   3 +
 16 files changed, 267 insertions(+), 21 deletions(-)
 create mode 100644 Documentation/gpu/xe/xe-drm-usage-stats.rst
Signature · 1 line
2.43.0
#1 lore raw

PATCH v3 1/6 drm/xe: Promote xe_hw_engine_class_to_str()

Move it out of the sysfs compilation unit so it can be re-used in other places.

---
 drivers/gpu/drm/xe/xe_hw_engine.c             | 18 ++++++++++++++++++
 drivers/gpu/drm/xe/xe_hw_engine.h             |  2 ++
 drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c | 18 ------------------
 3 files changed, 20 insertions(+), 18 deletions(-)
drivers/gpu/drm/xe/xe_hw_engine.c +18 -0
index ec69803152a2..85712650be22 100644
@@ -1088,3 +1088,21 @@ bool xe_hw_engine_is_reserved(struct xe_hw_engine *hwe)
 	return xe->info.has_usm && hwe->class == XE_ENGINE_CLASS_COPY &&
 		hwe->instance == gt->usm.reserved_bcs_instance;
 }
+
+const char *xe_hw_engine_class_to_str(enum xe_engine_class class)
+{
+	switch (class) {
+	case XE_ENGINE_CLASS_RENDER:
+		return "rcs";
+	case XE_ENGINE_CLASS_VIDEO_DECODE:
+		return "vcs";
+	case XE_ENGINE_CLASS_VIDEO_ENHANCE:
+		return "vecs";
+	case XE_ENGINE_CLASS_COPY:
+		return "bcs";
+	case XE_ENGINE_CLASS_COMPUTE:
+		return "ccs";
+	default:
+		return NULL;
+	}
+}
drivers/gpu/drm/xe/xe_hw_engine.h +2 -0
index 71968ee2f600..843de159e47c 100644
@@ -67,4 +67,6 @@ static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe)
 	return hwe->name;
 }
 
+const char *xe_hw_engine_class_to_str(enum xe_engine_class class);
+
 #endif
drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c +0 -18
index 844ec68cbbb8..efce6c7dd2a2 100644
@@ -618,24 +618,6 @@ static void hw_engine_class_sysfs_fini(struct drm_device *drm, void *arg)
 	kobject_put(kobj);
 }
 
-static const char *xe_hw_engine_class_to_str(enum xe_engine_class class)
-{
-	switch (class) {
-	case XE_ENGINE_CLASS_RENDER:
-		return "rcs";
-	case XE_ENGINE_CLASS_VIDEO_DECODE:
-		return "vcs";
-	case XE_ENGINE_CLASS_VIDEO_ENHANCE:
-		return "vecs";
-	case XE_ENGINE_CLASS_COPY:
-		return "bcs";
-	case XE_ENGINE_CLASS_COMPUTE:
-		return "ccs";
-	default:
-		return NULL;
-	}
-}
-
 /**
  * xe_hw_engine_class_sysfs_init - Init HW engine classes on GT.
  * @gt: Xe GT.
Signature · 1 line
2.43.0
#2 lore raw

PATCH v3 2/6 drm/xe: Add XE_ENGINE_CLASS_OTHER to str conversion

In reply to Lucas De Marchi

XE_ENGINE_CLASS_OTHER was missing from the str conversion. Add it and remove the default handling so it's protected by -Wswitch. Currently the only user is xe_hw_engine_class_sysfs_init(), which already skips XE_ENGINE_CLASS_OTHER, so there's no change in behavior.

---
 drivers/gpu/drm/xe/xe_hw_engine.c | 8 ++++++--
 1 file changed, 6 insertions(+), 2 deletions(-)
drivers/gpu/drm/xe/xe_hw_engine.c +6 -2
index 85712650be22..de30b74bf253 100644
@@ -1100,9 +1100,13 @@ const char *xe_hw_engine_class_to_str(enum xe_engine_class class)
 		return "vecs";
 	case XE_ENGINE_CLASS_COPY:
 		return "bcs";
+	case XE_ENGINE_CLASS_OTHER:
+		return "other";
 	case XE_ENGINE_CLASS_COMPUTE:
 		return "ccs";
-	default:
-		return NULL;
+	case XE_ENGINE_CLASS_MAX:
+		break;
 	}
+
+	return NULL;
 }
Signature · 1 line
2.43.0
#3 lore raw

PATCH v3 3/6 drm/xe/lrc: Add helper to capture context timestamp

In reply to Lucas De Marchi

From: Umesh Nerlige Ramappa <[email protected]>

Add a helper to capture CTX_TIMESTAMP from the context image so it can be used to calculate the runtime.

v2: Add kernel-doc to clarify expectation from caller

---
 drivers/gpu/drm/xe/regs/xe_lrc_layout.h |  1 +
 drivers/gpu/drm/xe/xe_lrc.c             | 11 +++++++++++
 drivers/gpu/drm/xe/xe_lrc.h             | 14 ++++++++++++++
 drivers/gpu/drm/xe/xe_lrc_types.h       |  3 +++
 4 files changed, 29 insertions(+)
drivers/gpu/drm/xe/regs/xe_lrc_layout.h +1 -0
index 1825d8f79db6..8780e6c6b649 100644
@@ -11,6 +11,7 @@
 #define CTX_RING_TAIL			(0x06 + 1)
 #define CTX_RING_START			(0x08 + 1)
 #define CTX_RING_CTL			(0x0a + 1)
+#define CTX_TIMESTAMP			(0x22 + 1)
 #define CTX_PDP0_UDW			(0x30 + 1)
 #define CTX_PDP0_LDW			(0x32 + 1)
 
drivers/gpu/drm/xe/xe_lrc.c +11 -0
index 2066d34ddf0b..8c540f2f43e3 100644
@@ -751,6 +751,7 @@ int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe,
 	lrc->tile = gt_to_tile(hwe->gt);
 	lrc->ring.size = ring_size;
 	lrc->ring.tail = 0;
+	lrc->ctx_timestamp = 0;
 
 	xe_hw_fence_ctx_init(&lrc->fence_ctx, hwe->gt,
 			     hwe->fence_irq, hwe->name);
@@ -786,6 +787,7 @@ int xe_lrc_init(struct xe_lrc *lrc, struct xe_hw_engine *hwe,
 			xe_drm_client_add_bo(vm->xef->client, lrc->bo);
 	}
 
+	xe_lrc_write_ctx_reg(lrc, CTX_TIMESTAMP, 0);
 	xe_lrc_write_ctx_reg(lrc, CTX_RING_START, __xe_lrc_ring_ggtt_addr(lrc));
 	xe_lrc_write_ctx_reg(lrc, CTX_RING_HEAD, 0);
 	xe_lrc_write_ctx_reg(lrc, CTX_RING_TAIL, lrc->ring.tail);
@@ -1444,3 +1446,12 @@ void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot)
 		xe_bo_put(snapshot->lrc_bo);
 	kfree(snapshot);
 }
+
+u32 xe_lrc_update_timestamp(struct xe_lrc *lrc, u32 *old_ts)
+{
+	*old_ts = lrc->ctx_timestamp;
+
+	lrc->ctx_timestamp = xe_lrc_read_ctx_reg(lrc, CTX_TIMESTAMP);
+
+	return lrc->ctx_timestamp;
+}
drivers/gpu/drm/xe/xe_lrc.h +14 -0
index d32fa31faa2c..dcbc6edd80da 100644
@@ -60,4 +60,18 @@ void xe_lrc_snapshot_capture_delayed(struct xe_lrc_snapshot *snapshot);
 void xe_lrc_snapshot_print(struct xe_lrc_snapshot *snapshot, struct drm_printer *p);
 void xe_lrc_snapshot_free(struct xe_lrc_snapshot *snapshot);
 
+/**
+ * xe_lrc_update_timestamp - readout LRC timestamp and update cached value
+ * @lrc: logical ring context for this exec queue
+ * @old_ts: pointer where to save the previous timestamp
+ *
+ * Read the current timestamp for this LRC and update the cached value. The
+ * previous cached value is also returned in @old_ts so the caller can calculate
+ * the delta between 2 updates. Note that this is not intended to be called from
+ * any place, but just by the paths updating the drm client utilization.
+ *
+ * Returns the current LRC timestamp
+ */
+u32 xe_lrc_update_timestamp(struct xe_lrc *lrc, u32 *old_ts);
+
 #endif
drivers/gpu/drm/xe/xe_lrc_types.h +3 -0
index b716df0dfb4e..5765d771b901 100644
@@ -41,6 +41,9 @@ struct xe_lrc {
 
 	/** @fence_ctx: context for hw fence */
 	struct xe_hw_fence_ctx fence_ctx;
+
+	/** @ctx_timestamp: readout value of CTX_TIMESTAMP on last update */
+	u32 ctx_timestamp;
 };
 
 struct xe_lrc_snapshot;
Signature · 1 line
2.43.0
#4 lore raw

PATCH v3 4/6 drm/xe: Add helper to capture engine timestamp

In reply to Lucas De Marchi

Just like CTX_TIMESTAMP is used to calculate runtime, add a helper to get the timestamp for the engine so it can be used to calculate the "engine time" with the same unit as the runtime is recorded.

---
 drivers/gpu/drm/xe/xe_hw_engine.c | 5 +++++
 drivers/gpu/drm/xe/xe_hw_engine.h | 1 +
 2 files changed, 6 insertions(+)
drivers/gpu/drm/xe/xe_hw_engine.c +5 -0
index de30b74bf253..734f43a35b48 100644
@@ -1110,3 +1110,8 @@ const char *xe_hw_engine_class_to_str(enum xe_engine_class class)
 
 	return NULL;
 }
+
+u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe)
+{
+	return xe_mmio_read64_2x32(hwe->gt, RING_TIMESTAMP(hwe->mmio_base));
+}
drivers/gpu/drm/xe/xe_hw_engine.h +1 -0
index 843de159e47c..7f2d27c0ba1a 100644
@@ -68,5 +68,6 @@ static inline bool xe_hw_engine_is_valid(struct xe_hw_engine *hwe)
 }
 
 const char *xe_hw_engine_class_to_str(enum xe_engine_class class);
+u64 xe_hw_engine_read_timestamp(struct xe_hw_engine *hwe);
 
 #endif
Signature · 1 line
2.43.0
#5 lore raw

PATCH v3 5/6 drm/xe: Add helper to accumulate exec queue runtime

In reply to Lucas De Marchi

From: Umesh Nerlige Ramappa <[email protected]>

Add a helper to accumulate per-client runtime of all its exec queues. This is called every time a sched job is finished.

v2:
- Use guc_exec_queue_free_job() and execlist_job_free() to accumulate runtime when job is finished since xe_sched_job_completed() is not a notification that job finished.
- Stop trying to update runtime from xe_exec_queue_fini() - that is redundant and may happen after xef is closed, leading to a use-after-free
- Do not special case the first timestamp read: the default LRC sets CTX_TIMESTAMP to zero, so even the first sample should be a valid one.
- Handle the parallel submission case by multiplying the runtime by width.

---
 drivers/gpu/drm/xe/xe_device_types.h |  9 +++++++
 drivers/gpu/drm/xe/xe_exec_queue.c   | 35 ++++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_exec_queue.h   |  1 +
 drivers/gpu/drm/xe/xe_execlist.c     |  1 +
 drivers/gpu/drm/xe/xe_guc_submit.c   |  2 ++
 5 files changed, 48 insertions(+)
drivers/gpu/drm/xe/xe_device_types.h +9 -0
index 906b98fb973b..de078bdf0ab9 100644
@@ -560,6 +560,15 @@ struct xe_file {
 		struct mutex lock;
 	} exec_queue;
 
+	/**
+	 * @runtime: hw engine class runtime in ticks for this drm client
+	 *
+	 * Only stats from xe_exec_queue->lrc[0] are accumulated. For multi-lrc
+	 * case, since all jobs run in parallel on the engines, only the stats
+	 * from lrc[0] are sufficient.
+	 */
+	u64 runtime[XE_ENGINE_CLASS_MAX];
+
 	/** @client: drm client */
 	struct xe_drm_client *client;
 };
drivers/gpu/drm/xe/xe_exec_queue.c +35 -0
index 395de93579fa..86eb22e22c95 100644
@@ -769,6 +769,41 @@ bool xe_exec_queue_is_idle(struct xe_exec_queue *q)
 		q->lrc[0].fence_ctx.next_seqno - 1;
 }
 
+/**
+ * xe_exec_queue_update_runtime() - Update runtime for this exec queue from hw
+ * @q: The exec queue
+ *
+ * Update the timestamp saved by HW for this exec queue and save runtime
+ * calculated by using the delta from last update. On multi-lrc case, only the
+ * first is considered.
+ */
+void xe_exec_queue_update_runtime(struct xe_exec_queue *q)
+{
+	struct xe_file *xef;
+	struct xe_lrc *lrc;
+	u32 old_ts, new_ts;
+
+	/*
+	 * Jobs that are run during driver load may use an exec_queue, but are
+	 * not associated with a user xe file, so avoid accumulating busyness
+	 * for kernel specific work.
+	 */
+	if (!q->vm || !q->vm->xef)
+		return;
+
+	xef = q->vm->xef;
+
+	/*
+	 * Only sample the first LRC. For parallel submission, all of them are
+	 * scheduled together and we compensate that below by multiplying by
+	 * width
+	 */
+	lrc = &q->lrc[0];
+
+	new_ts = xe_lrc_update_timestamp(lrc, &old_ts);
+	xef->runtime[q->class] += (new_ts - old_ts) * q->width;
+}
+
 void xe_exec_queue_kill(struct xe_exec_queue *q)
 {
 	struct xe_exec_queue *eq = q, *next;
drivers/gpu/drm/xe/xe_exec_queue.h +1 -0
index 48f6da53a292..e0f07d28ee1a 100644
@@ -75,5 +75,6 @@ struct dma_fence *xe_exec_queue_last_fence_get(struct xe_exec_queue *e,
 					       struct xe_vm *vm);
 void xe_exec_queue_last_fence_set(struct xe_exec_queue *e, struct xe_vm *vm,
 				  struct dma_fence *fence);
+void xe_exec_queue_update_runtime(struct xe_exec_queue *q);
 
 #endif
drivers/gpu/drm/xe/xe_execlist.c +1 -0
index dece2785933c..a316431025c7 100644
@@ -307,6 +307,7 @@ static void execlist_job_free(struct drm_sched_job *drm_job)
 {
 	struct xe_sched_job *job = to_xe_sched_job(drm_job);
 
+	xe_exec_queue_update_runtime(job->q);
 	xe_sched_job_put(job);
 }
 
drivers/gpu/drm/xe/xe_guc_submit.c +2 -0
index d274a139010b..e0ebfe83dfe8 100644
@@ -749,6 +749,8 @@ static void guc_exec_queue_free_job(struct drm_sched_job *drm_job)
 {
 	struct xe_sched_job *job = to_xe_sched_job(drm_job);
 
+	xe_exec_queue_update_runtime(job->q);
+
 	trace_xe_sched_job_free(job);
 	xe_sched_job_put(job);
 }
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH v3 5/6] drm/xe: Add helper to accumulate exec queue runtime

Quoted 33 lines · Lucas De Marchi

From: Umesh Nerlige Ramappa <[email protected]>

Add a helper to accumulate per-client runtime of all its exec queues. This is called every time a sched job is finished.

v2:
- Use guc_exec_queue_free_job() and execlist_job_free() to accumulate runtime when job is finished since xe_sched_job_completed() is not a notification that job finished.
- Stop trying to update runtime from xe_exec_queue_fini() - that is redundant and may happen after xef is closed, leading to a use-after-free
- Do not special case the first timestamp read: the default LRC sets CTX_TIMESTAMP to zero, so even the first sample should be a valid one.
- Handle the parallel submission case by multiplying the runtime by width.

---

drivers/gpu/drm/xe/xe_device_types.h |  9 +++++++
drivers/gpu/drm/xe/xe_exec_queue.c   | 35 ++++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_exec_queue.h   |  1 +
drivers/gpu/drm/xe/xe_execlist.c     |  1 +
drivers/gpu/drm/xe/xe_guc_submit.c   |  2 ++
5 files changed, 48 insertions(+)
drivers/gpu/drm/xe/xe_device_types.h +0 -0
index 906b98fb973b..de078bdf0ab9 100644

drivers/gpu/drm/xe/xe_device_types.h struct xe_file { · line 568

Quoted 9 lines
drivers/gpu/drm/xe/xe_device_types.h +6 -0
@@ -560,6 +560,15 @@ struct xe_file {
 		struct mutex lock;
 	} exec_queue;
 
+	/**
+	 * @runtime: hw engine class runtime in ticks for this drm client
+	 *
+	 * Only stats from xe_exec_queue->lrc[0] are accumulated. For multi-lrc
+	 * case, since all jobs run in parallel on the engines, only the stats
+	 * from lrc[0] are sufficient.

Maybe we can drop the above comment altogether after the multi-lrc update.

Umesh

#7 lore raw

Re: [PATCH v3 5/6] drm/xe: Add helper to accumulate exec queue runtime

Quoted 37 lines · Umesh Nerlige Ramappa
Quoted 36 lines · Lucas De Marchi

From: Umesh Nerlige Ramappa <[email protected]>

Add a helper to accumulate per-client runtime of all its exec queues. This is called every time a sched job is finished.

v2:
- Use guc_exec_queue_free_job() and execlist_job_free() to accumulate runtime when job is finished since xe_sched_job_completed() is not a notification that job finished.
- Stop trying to update runtime from xe_exec_queue_fini() - that is redundant and may happen after xef is closed, leading to a use-after-free
- Do not special case the first timestamp read: the default LRC sets CTX_TIMESTAMP to zero, so even the first sample should be a valid one.
- Handle the parallel submission case by multiplying the runtime by width.

---

drivers/gpu/drm/xe/xe_device_types.h |  9 +++++++
drivers/gpu/drm/xe/xe_exec_queue.c   | 35 ++++++++++++++++++++++++++++
drivers/gpu/drm/xe/xe_exec_queue.h   |  1 +
drivers/gpu/drm/xe/xe_execlist.c     |  1 +
drivers/gpu/drm/xe/xe_guc_submit.c   |  2 ++
5 files changed, 48 insertions(+)
drivers/gpu/drm/xe/xe_device_types.h +0 -0
index 906b98fb973b..de078bdf0ab9 100644
@@ -560,6 +560,15 @@ struct xe_file {
 		struct mutex lock;
 	} exec_queue;
 
Quoted 8 lines

drivers/gpu/drm/xe/xe_device_types.h struct xe_file { · line 568

Quoted 6 lines
drivers/gpu/drm/xe/xe_device_types.h +6 -0
+	/**
+	 * @runtime: hw engine class runtime in ticks for this drm client
+	 *
+	 * Only stats from xe_exec_queue->lrc[0] are accumulated. For multi-lrc
+	 * case, since all jobs run in parallel on the engines, only the stats
+	 * from lrc[0] are sufficient.

Maybe we can drop the above comment altogether after the multi-lrc update.

right... I removed it from the middle of the function and this was a leftover. I will remove it on next version.

thanks
Lucas De Marchi

Quoted 2 lines

Umesh

#8 lore raw

Re: [PATCH v3 5/6] drm/xe: Add helper to accumulate exec queue runtime

In reply to Lucas De Marchi
Quoted 79 lines · Lucas De Marchi

From: Umesh Nerlige Ramappa <[email protected]>

Add a helper to accumulate per-client runtime of all its exec queues. This is called every time a sched job is finished.

v2:
- Use guc_exec_queue_free_job() and execlist_job_free() to accumulate runtime when job is finished since xe_sched_job_completed() is not a notification that job finished.
- Stop trying to update runtime from xe_exec_queue_fini() - that is redundant and may happen after xef is closed, leading to a use-after-free
- Do not special case the first timestamp read: the default LRC sets CTX_TIMESTAMP to zero, so even the first sample should be a valid one.
- Handle the parallel submission case by multiplying the runtime by width.

---
  drivers/gpu/drm/xe/xe_device_types.h |  9 +++++++
  drivers/gpu/drm/xe/xe_exec_queue.c   | 35 ++++++++++++++++++++++++++++
  drivers/gpu/drm/xe/xe_exec_queue.h   |  1 +
  drivers/gpu/drm/xe/xe_execlist.c     |  1 +
  drivers/gpu/drm/xe/xe_guc_submit.c   |  2 ++
  5 files changed, 48 insertions(+)
drivers/gpu/drm/xe/xe_device_types.h +9 -0
index 906b98fb973b..de078bdf0ab9 100644
@@ -560,6 +560,15 @@ struct xe_file {
  		struct mutex lock;
  	} exec_queue;
 
+	/**
+	 * @runtime: hw engine class runtime in ticks for this drm client
+	 *
+	 * Only stats from xe_exec_queue->lrc[0] are accumulated. For multi-lrc
+	 * case, since all jobs run in parallel on the engines, only the stats
+	 * from lrc[0] are sufficient.
+	 */
+	u64 runtime[XE_ENGINE_CLASS_MAX];
+
  	/** @client: drm client */
  	struct xe_drm_client *client;
  };
drivers/gpu/drm/xe/xe_exec_queue.c +23 -0
index 395de93579fa..86eb22e22c95 100644
@@ -769,6 +769,41 @@ bool xe_exec_queue_is_idle(struct xe_exec_queue *q)
  		q->lrc[0].fence_ctx.next_seqno - 1;
  }
 
+/**
+ * xe_exec_queue_update_runtime() - Update runtime for this exec queue from hw
+ * @q: The exec queue
+ *
+ * Update the timestamp saved by HW for this exec queue and save runtime
+ * calculated by using the delta from last update. On multi-lrc case, only the
+ * first is considered.
+ */
+void xe_exec_queue_update_runtime(struct xe_exec_queue *q)
+{
+	struct xe_file *xef;
+	struct xe_lrc *lrc;
+	u32 old_ts, new_ts;
+
+	/*
+	 * Jobs that are run during driver load may use an exec_queue, but are
+	 * not associated with a user xe file, so avoid accumulating busyness
+	 * for kernel specific work.
+	 */
+	if (!q->vm || !q->vm->xef)
+		return;
+
+	xef = q->vm->xef;

drivers/gpu/drm/xe/xe_exec_queue.c bool xe_exec_queue_is_idle(struct xe_exec_queue *q) · line 804

Quoted 10 lines
drivers/gpu/drm/xe/xe_exec_queue.c +10 -0
+
+	/*
+	 * Only sample the first LRC. For parallel submission, all of them are
+	 * scheduled together and we compensate that below by multiplying by
+	 * width
+	 */
+	lrc = &q->lrc[0];
+
+	new_ts = xe_lrc_update_timestamp(lrc, &old_ts);
+	xef->runtime[q->class] += (new_ts - old_ts) * q->width;

I think in theory this could be introducing a systematic error depending on how firmware handles things and tick resolution. Or even regardless of the firmware, if the timestamps are saved on context exit by the GPU hw itself and parallel contexts do not exit 100% aligned. Undershoot would be I think fine, but systematic overshoot under constant 100% parallel load from mutlitple client could constantly show >100% class utilisation. Probably not a concern in practice but worthy a comment?

Regards,

Tvrtko

Quoted 41 lines
drivers/gpu/drm/xe/xe_exec_queue.c +2 -0
+}
+
  void xe_exec_queue_kill(struct xe_exec_queue *q)
  {
  	struct xe_exec_queue *eq = q, *next;
drivers/gpu/drm/xe/xe_exec_queue.h +1 -0
index 48f6da53a292..e0f07d28ee1a 100644
@@ -75,5 +75,6 @@ struct dma_fence *xe_exec_queue_last_fence_get(struct xe_exec_queue *e,
  					       struct xe_vm *vm);
  void xe_exec_queue_last_fence_set(struct xe_exec_queue *e, struct xe_vm *vm,
  				  struct dma_fence *fence);
+void xe_exec_queue_update_runtime(struct xe_exec_queue *q);
 
  #endif
drivers/gpu/drm/xe/xe_execlist.c +1 -0
index dece2785933c..a316431025c7 100644
@@ -307,6 +307,7 @@ static void execlist_job_free(struct drm_sched_job *drm_job)
  {
  	struct xe_sched_job *job = to_xe_sched_job(drm_job);
 
+	xe_exec_queue_update_runtime(job->q);
  	xe_sched_job_put(job);
  }
 
drivers/gpu/drm/xe/xe_guc_submit.c +2 -0
index d274a139010b..e0ebfe83dfe8 100644
@@ -749,6 +749,8 @@ static void guc_exec_queue_free_job(struct drm_sched_job *drm_job)
  {
  	struct xe_sched_job *job = to_xe_sched_job(drm_job);
 
+	xe_exec_queue_update_runtime(job->q);
+
  	trace_xe_sched_job_free(job);
  	xe_sched_job_put(job);
  }
#9 lore raw

Re: [PATCH v3 5/6] drm/xe: Add helper to accumulate exec queue runtime

Quoted 83 lines · Tvrtko Ursulin
Quoted 81 lines · Lucas De Marchi

From: Umesh Nerlige Ramappa <[email protected]>

Add a helper to accumulate per-client runtime of all its exec queues. This is called every time a sched job is finished.

v2:
- Use guc_exec_queue_free_job() and execlist_job_free() to accumulate runtime when job is finished since xe_sched_job_completed() is not a notification that job finished.
- Stop trying to update runtime from xe_exec_queue_fini() - that is redundant and may happen after xef is closed, leading to a use-after-free
- Do not special case the first timestamp read: the default LRC sets CTX_TIMESTAMP to zero, so even the first sample should be a valid one.
- Handle the parallel submission case by multiplying the runtime by width.

---
 drivers/gpu/drm/xe/xe_device_types.h |  9 +++++++
 drivers/gpu/drm/xe/xe_exec_queue.c   | 35 ++++++++++++++++++++++++++++
 drivers/gpu/drm/xe/xe_exec_queue.h   |  1 +
 drivers/gpu/drm/xe/xe_execlist.c     |  1 +
 drivers/gpu/drm/xe/xe_guc_submit.c   |  2 ++
 5 files changed, 48 insertions(+)
drivers/gpu/drm/xe/xe_device_types.h +9 -0
index 906b98fb973b..de078bdf0ab9 100644
@@ -560,6 +560,15 @@ struct xe_file {
 		struct mutex lock;
 	} exec_queue;
+	/**
+	 * @runtime: hw engine class runtime in ticks for this drm client
+	 *
+	 * Only stats from xe_exec_queue->lrc[0] are accumulated. For multi-lrc
+	 * case, since all jobs run in parallel on the engines, only the stats
+	 * from lrc[0] are sufficient.
+	 */
+	u64 runtime[XE_ENGINE_CLASS_MAX];
+
 	/** @client: drm client */
 	struct xe_drm_client *client;
 };
drivers/gpu/drm/xe/xe_exec_queue.c +27 -0
index 395de93579fa..86eb22e22c95 100644
@@ -769,6 +769,41 @@ bool xe_exec_queue_is_idle(struct xe_exec_queue *q)
 		q->lrc[0].fence_ctx.next_seqno - 1;
 }
+/**
+ * xe_exec_queue_update_runtime() - Update runtime for this exec queue from hw
+ * @q: The exec queue
+ *
+ * Update the timestamp saved by HW for this exec queue and save runtime
+ * calculated by using the delta from last update. On multi-lrc case, only the
+ * first is considered.
+ */
+void xe_exec_queue_update_runtime(struct xe_exec_queue *q)
+{
+	struct xe_file *xef;
+	struct xe_lrc *lrc;
+	u32 old_ts, new_ts;
+
+	/*
+	 * Jobs that are run during driver load may use an exec_queue, but are
+	 * not associated with a user xe file, so avoid accumulating busyness
+	 * for kernel specific work.
+	 */
+	if (!q->vm || !q->vm->xef)
+		return;
+
+	xef = q->vm->xef;
+
+	/*
+	 * Only sample the first LRC. For parallel submission, all of them are
+	 * scheduled together and we compensate that below by multiplying by
Quoted 8 lines

drivers/gpu/drm/xe/xe_exec_queue.c bool xe_exec_queue_is_idle(struct xe_exec_queue *q) · line 804

Quoted 6 lines
drivers/gpu/drm/xe/xe_exec_queue.c +6 -0
+	 * width
+	 */
+	lrc = &q->lrc[0];
+
+	new_ts = xe_lrc_update_timestamp(lrc, &old_ts);
+	xef->runtime[q->class] += (new_ts - old_ts) * q->width;

I think in theory this could be introducing a systematic error depending on how firmware handles things and tick resolution. Or even regardless of the firmware, if the timestamps are saved on context exit by the GPU hw itself and parallel contexts do not exit 100% aligned. Undershoot would be I think fine, but systematic overshoot under constant 100% parallel load from mutlitple client could constantly show >100% class utilisation. Probably not a concern in practice but worthy a comment?

Ok... just extend the comment above? I could have looped through the LRCs to read the timestamp, but I didn't like reading them in slightly different times, which could give slight different results if they are in a running state. This way here it doesn't have this problem at the expense of "we have to trust the hw/firmware". We can switch to the loop eventually if our level of trust decreases after getting more data/test :)

thanks
Lucas De Marchi

Quoted 42 lines

Regards,

Tvrtko

Quoted 37 lines
drivers/gpu/drm/xe/xe_exec_queue.c +2 -0
+}
+
 void xe_exec_queue_kill(struct xe_exec_queue *q)
 {
 	struct xe_exec_queue *eq = q, *next;
drivers/gpu/drm/xe/xe_exec_queue.h +1 -0
index 48f6da53a292..e0f07d28ee1a 100644
@@ -75,5 +75,6 @@ struct dma_fence *xe_exec_queue_last_fence_get(struct xe_exec_queue *e,
 					       struct xe_vm *vm);
 void xe_exec_queue_last_fence_set(struct xe_exec_queue *e, struct xe_vm *vm,
 				  struct dma_fence *fence);
+void xe_exec_queue_update_runtime(struct xe_exec_queue *q);
 #endif
drivers/gpu/drm/xe/xe_execlist.c +1 -0
index dece2785933c..a316431025c7 100644
@@ -307,6 +307,7 @@ static void execlist_job_free(struct drm_sched_job *drm_job)
 {
 	struct xe_sched_job *job = to_xe_sched_job(drm_job);
+	xe_exec_queue_update_runtime(job->q);
 	xe_sched_job_put(job);
 }
drivers/gpu/drm/xe/xe_guc_submit.c +2 -0
index d274a139010b..e0ebfe83dfe8 100644
@@ -749,6 +749,8 @@ static void guc_exec_queue_free_job(struct drm_sched_job *drm_job)
 {
 	struct xe_sched_job *job = to_xe_sched_job(drm_job);
+	xe_exec_queue_update_runtime(job->q);
+
 	trace_xe_sched_job_free(job);
 	xe_sched_job_put(job);
 }
#10 lore raw

PATCH v3 6/6 drm/xe/client: Print runtime to fdinfo

In reply to Lucas De Marchi

Print the accumulated runtime for client when printing fdinfo. Each time a query is done it first does 2 things:

1) loop through all the exec queues for the current client and accumulate the runtime, per engine class. CTX_TIMESTAMP is used for that, being read from the context image.

2) Read a "GPU timestamp" that can be used for considering "how much GPU time has passed" and that has the same unit/refclock as the one recording the runtime. RING_TIMESTAMP is used for that via MMIO.

Since for all current platforms RING_TIMESTAMP follows the same refclock, just read it once, using any first engine.

This is exported to userspace as 2 numbers in fdinfo:

drm-cycles-<class>: <RUNTIME>
drm-total-cycles-<class>: <TIMESTAMP>

Userspace is expected to collect at least 2 samples, which allows to know the client engine busyness as per:

		    RUNTIME1 - RUNTIME0
	busyness = ---------------------
			  T1 - T0

Another thing to point out is that it's expected that userspace will read any 2 samples every few seconds. Given the update frequency of the counters involved and that CTX_TIMESTAMP is 32-bits, the counter for each exec_queue can wrap around (assuming 100% utilization) after ~200s. The wraparound is not perceived by userspace since it's just accumulated for all the exec_queues in a 64-bit counter), but the measurement will not be accurate if the samples are too far apart.

This could be mitigated by adding a workqueue to accumulate the counters every so often, but it's additional complexity for something that is done already by userspace every few seconds in tools like gputop (from igt), htop, nvtop, etc with none of them really defaulting to 1 sample per minute or more.

---
 Documentation/gpu/drm-usage-stats.rst       |  16 ++-
 Documentation/gpu/xe/index.rst              |   1 +
 Documentation/gpu/xe/xe-drm-usage-stats.rst |  10 ++
 drivers/gpu/drm/xe/xe_drm_client.c          | 136 +++++++++++++++++++-
 4 files changed, 160 insertions(+), 3 deletions(-)
 create mode 100644 Documentation/gpu/xe/xe-drm-usage-stats.rst
Documentation/gpu/drm-usage-stats.rst +14 -2
index 6dc299343b48..421766289b78 100644
@@ -112,6 +112,17 @@ larger value within a reasonable period. Upon observing a value lower than what
 was previously read, userspace is expected to stay with that larger previous
 value until a monotonic update is seen.
 
+- drm-total-cycles-<keystr>: <uint>
+
+Engine identifier string must be the same as the one specified in the
+drm-cycles-<keystr> tag and shall contain the total number cycles for the given
+engine.
+
+This is a timestamp in GPU unspecified unit that matches the update rate
+of drm-cycles-<keystr>. For drivers that implement this interface, the engine
+utilization can be calculated entirely on the GPU clock domain, without
+considering the CPU sleep time between 2 samples.
+
 - drm-maxfreq-<keystr>: <uint> [Hz|MHz|KHz]
 
 Engine identifier string must be the same as the one specified in the
@@ -168,5 +179,6 @@ be documented above and where possible, aligned with other drivers.
 Driver specific implementations
 -------------------------------
 
-:ref:`i915-usage-stats`
-:ref:`panfrost-usage-stats`
+* :ref:`i915-usage-stats`
+* :ref:`panfrost-usage-stats`
+* :ref:`xe-usage-stats`
Documentation/gpu/xe/index.rst +1 -0
index c224ecaee81e..3f07aa3b5432 100644
@@ -23,3 +23,4 @@ DG2, etc is provided to prototype the driver.
    xe_firmware
    xe_tile
    xe_debugging
+   xe-drm-usage-stats.rst
Documentation/gpu/xe/xe-drm-usage-stats.rst new file +10 -0
new file mode 100644
index 000000000000..482d503ae68a
@@ -0,0 +1,10 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+.. _xe-usage-stats:
+
+========================================
+Xe DRM client usage stats implementation
+========================================
+
+.. kernel-doc:: drivers/gpu/drm/xe/xe_drm_client.c
+   :doc: DRM Client usage stats
drivers/gpu/drm/xe/xe_drm_client.c +135 -1

161 lines not on this page · raw message

Signature · 1 line
2.43.0
#11 lore raw

Re: [PATCH v3 6/6] drm/xe/client: Print runtime to fdinfo

Quoted 212 lines · Lucas De Marchi

Print the accumulated runtime for client when printing fdinfo. Each time a query is done it first does 2 things:

1) loop through all the exec queues for the current client and accumulate the runtime, per engine class. CTX_TIMESTAMP is used for that, being read from the context image.

2) Read a "GPU timestamp" that can be used for considering "how much GPU time has passed" and that has the same unit/refclock as the one recording the runtime. RING_TIMESTAMP is used for that via MMIO.

Since for all current platforms RING_TIMESTAMP follows the same refclock, just read it once, using any first engine.

This is exported to userspace as 2 numbers in fdinfo:

drm-cycles-<class>: <RUNTIME>
drm-total-cycles-<class>: <TIMESTAMP>

Userspace is expected to collect at least 2 samples, which allows to know the client engine busyness as per:

		    RUNTIME1 - RUNTIME0
	busyness = ---------------------
			  T1 - T0

Another thing to point out is that it's expected that userspace will read any 2 samples every few seconds. Given the update frequency of the counters involved and that CTX_TIMESTAMP is 32-bits, the counter for each exec_queue can wrap around (assuming 100% utilization) after ~200s. The wraparound is not perceived by userspace since it's just accumulated for all the exec_queues in a 64-bit counter), but the measurement will not be accurate if the samples are too far apart.

This could be mitigated by adding a workqueue to accumulate the counters every so often, but it's additional complexity for something that is done already by userspace every few seconds in tools like gputop (from igt), htop, nvtop, etc with none of them really defaulting to 1 sample per minute or more.

---

Documentation/gpu/drm-usage-stats.rst       |  16 ++-
Documentation/gpu/xe/index.rst              |   1 +
Documentation/gpu/xe/xe-drm-usage-stats.rst |  10 ++
drivers/gpu/drm/xe/xe_drm_client.c          | 136 +++++++++++++++++++-
4 files changed, 160 insertions(+), 3 deletions(-)

create mode 100644 Documentation/gpu/xe/xe-drm-usage-stats.rst

Documentation/gpu/drm-usage-stats.rst +0 -0
index 6dc299343b48..421766289b78 100644
@@ -112,6 +112,17 @@ larger value within a reasonable period. Upon observing a value lower than what

was previously read, userspace is expected to stay with that larger previous value until a monotonic update is seen.

+- drm-total-cycles-<keystr>: <uint>

Documentation/gpu/drm-usage-stats.rst +10 -1
+
+Engine identifier string must be the same as the one specified in the
+drm-cycles-<keystr> tag and shall contain the total number cycles for the given
+engine.
+
+This is a timestamp in GPU unspecified unit that matches the update rate
+of drm-cycles-<keystr>. For drivers that implement this interface, the engine
+utilization can be calculated entirely on the GPU clock domain, without
+considering the CPU sleep time between 2 samples.
+
- drm-maxfreq-<keystr>: <uint> [Hz|MHz|KHz]

Engine identifier string must be the same as the one specified in the

Documentation/gpu/drm-usage-stats.rst +0 -0
@@ -168,5 +179,6 @@ be documented above and where possible, aligned with other drivers.

Driver specific implementations
-------------------------------

-:ref:i915-usage-stats
-:ref:panfrost-usage-stats

Documentation/gpu/drm-usage-stats.rst +3 -0
+* :ref:`i915-usage-stats`
+* :ref:`panfrost-usage-stats`
+* :ref:`xe-usage-stats`
Documentation/gpu/xe/index.rst +1 -0
index c224ecaee81e..3f07aa3b5432 100644
@@ -23,3 +23,4 @@ DG2, etc is provided to prototype the driver.
   xe_firmware
   xe_tile
   xe_debugging
+   xe-drm-usage-stats.rst
Documentation/gpu/xe/xe-drm-usage-stats.rst new file +10 -0
new file mode 100644
index 000000000000..482d503ae68a
@@ -0,0 +1,10 @@
+.. SPDX-License-Identifier: GPL-2.0+
+
+.. _xe-usage-stats:
+
+========================================
+Xe DRM client usage stats implementation
+========================================
+
+.. kernel-doc:: drivers/gpu/drm/xe/xe_drm_client.c
+   :doc: DRM Client usage stats
drivers/gpu/drm/xe/xe_drm_client.c +82 -1
index 08f0b7c95901..27494839d586 100644
@@ -2,6 +2,7 @@
 /*
 * Copyright © 2023 Intel Corporation
 */
+#include "xe_drm_client.h"
 
 #include <drm/drm_print.h>
 #include <drm/xe_drm.h>
@@ -12,9 +13,65 @@
 #include "xe_bo.h"
 #include "xe_bo_types.h"
 #include "xe_device_types.h"
-#include "xe_drm_client.h"
+#include "xe_exec_queue.h"
+#include "xe_gt.h"
+#include "xe_hw_engine.h"
+#include "xe_pm.h"
 #include "xe_trace.h"
 
+/**
+ * DOC: DRM Client usage stats
+ *
+ * The drm/xe driver implements the DRM client usage stats specification as
+ * documented in :ref:`drm-client-usage-stats`.
+ *
+ * Example of the output showing the implemented key value pairs and entirety of
+ * the currently possible format options:
+ *
+ * ::
+ *
+ * 	pos:    0
+ * 	flags:  0100002
+ * 	mnt_id: 26
+ * 	ino:    685
+ * 	drm-driver:     xe
+ * 	drm-client-id:  3
+ * 	drm-pdev:       0000:03:00.0
+ * 	drm-total-system:       0
+ * 	drm-shared-system:      0
+ * 	drm-active-system:      0
+ * 	drm-resident-system:    0
+ * 	drm-purgeable-system:   0
+ * 	drm-total-gtt:  192 KiB
+ * 	drm-shared-gtt: 0
+ * 	drm-active-gtt: 0
+ * 	drm-resident-gtt:       192 KiB
+ * 	drm-total-vram0:        23992 KiB
+ * 	drm-shared-vram0:       16 MiB
+ * 	drm-active-vram0:       0
+ * 	drm-resident-vram0:     23992 KiB
+ * 	drm-total-stolen:       0
+ * 	drm-shared-stolen:      0
+ * 	drm-active-stolen:      0
+ * 	drm-resident-stolen:    0
+ * 	drm-cycles-rcs: 28257900
+ * 	drm-total-cycles-rcs:   7655183225
+ * 	drm-cycles-bcs: 0
+ * 	drm-total-cycles-bcs:   7655183225
+ * 	drm-cycles-vcs: 0
+ * 	drm-total-cycles-vcs:   7655183225
+ * 	drm-engine-capacity-vcs:        2
+ * 	drm-cycles-vecs:        0
+ * 	drm-total-cycles-vecs:  7655183225
+ * 	drm-engine-capacity-vecs:       2
+ * 	drm-cycles-ccs: 0
+ * 	drm-total-cycles-ccs:   7655183225
+ * 	drm-engine-capacity-ccs:        4
+ *
+ * Possible `drm-cycles-` key names are: `rcs`, `ccs`, `bcs`, `vcs`, `vecs` and
+ * "other".
+ */
+
 /**
 * xe_drm_client_alloc() - Allocate drm client
 * @void: No arg
@@ -179,6 +236,82 @@ static void show_meminfo(struct drm_printer *p, struct drm_file *file)
 	}
 }
 
+static const u64 class_to_mask[] = {
+	[XE_ENGINE_CLASS_RENDER] = XE_HW_ENGINE_RCS_MASK,
+	[XE_ENGINE_CLASS_VIDEO_DECODE] = XE_HW_ENGINE_VCS_MASK,
+	[XE_ENGINE_CLASS_VIDEO_ENHANCE] = XE_HW_ENGINE_VECS_MASK,
+	[XE_ENGINE_CLASS_COPY] = XE_HW_ENGINE_BCS_MASK,
+	[XE_ENGINE_CLASS_OTHER] = XE_HW_ENGINE_GSCCS_MASK,
+	[XE_ENGINE_CLASS_COMPUTE] = XE_HW_ENGINE_CCS_MASK,
+};
+
+static void show_runtime(struct drm_printer *p, struct drm_file *file)
+{
+	struct xe_file *xef = file->driver_priv;
+	struct xe_device *xe = xef->xe;
+	struct xe_gt *gt;
+	struct xe_hw_engine *hwe;
+	struct xe_exec_queue *q;
+	unsigned long i, id_hwe, id_gt, capacity[XE_ENGINE_CLASS_MAX] = { };
+	u64 gpu_timestamp, engine_mask = 0;
+	bool gpu_stamp = false;
+
+	xe_pm_runtime_get(xe);
+
+	/* Accumulate all the exec queues from this client */
+	mutex_lock(&xef->exec_queue.lock);

drivers/gpu/drm/xe/xe_drm_client.c static void show_meminfo(struct drm_printer *p, struct drm_file *file) · line 272

Quoted 10 lines
drivers/gpu/drm/xe/xe_drm_client.c +10 -0
+	xa_for_each(&xef->exec_queue.xa, i, q)
+		xe_exec_queue_update_runtime(q);
+	mutex_unlock(&xef->exec_queue.lock);
+
+	/* Calculate capacity of each engine class */
+	BUILD_BUG_ON(ARRAY_SIZE(class_to_mask) != XE_ENGINE_CLASS_MAX);
+	for_each_gt(gt, xe, id_gt)
+		engine_mask |= gt->info.engine_mask;
+	for (i = 0; i < XE_ENGINE_CLASS_MAX; i++)
+		capacity[i] = hweight64(engine_mask & class_to_mask[i]);

Will this work if both tiles have identical set of engines? Are we even supporting such configs here?

If yes, I was thinking of something like this:

for_each_gt(gt, xe, id_gt)

	for (i = 0; i < XE_ENGINE_CLASS_MAX; i++)
		capacity[i] += hweight64(gt->info.engine_mask & class_to_mask[i]);
Quoted 27 lines
drivers/gpu/drm/xe/xe_drm_client.c +27 -0
+
+	/*
+	 * Iterate over all engines, printing the accumulated
+	 * runtime for this client, per engine class
+	 */
+	for_each_gt(gt, xe, id_gt) {
+		xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
+		for_each_hw_engine(hwe, gt, id_hwe) {
+			const char *class_name;
+
+			if (!capacity[hwe->class])
+				continue;
+
+			/*
+			 * Use any (first) engine to have a timestamp to be used every
+			 * time
+			 */
+			if (!gpu_stamp) {
+				gpu_timestamp = xe_hw_engine_read_timestamp(hwe);
+				gpu_stamp = true;
+			}
+
+			class_name = xe_hw_engine_class_to_str(hwe->class);
+
+			drm_printf(p, "drm-cycles-%s:\t%llu\n",
+				   class_name, xef->runtime[hwe->class]);
+			drm_printf(p, "drm-total-cycles-%s:\t%llu\n",

drivers/gpu/drm/xe/xe_drm_client.c static void show_meminfo(struct drm_printer *p, struct drm_file *file) · line 309

Quoted 10 lines
drivers/gpu/drm/xe/xe_drm_client.c +10 -0
+				   class_name, gpu_timestamp);
+
+			if (capacity[hwe->class] > 1)
+				drm_printf(p, "drm-engine-capacity-%s:\t%lu\n",
+					   class_name, capacity[hwe->class]);
+
+			/* engine class already handled, skip next iterations */
+			capacity[hwe->class] = 0;
+		}
+		xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);

Same question here. Do we support same class of engines that are present on both tiles?

Regards,
Umesh

Quoted 18 lines
drivers/gpu/drm/xe/xe_drm_client.c +6 -0
+	}
+
+	xe_pm_runtime_get(xe);
+}
+
 /**
 * xe_drm_client_fdinfo() - Callback for fdinfo interface
 * @p: The drm_printer ptr
@@ -192,5 +325,6 @@ static void show_meminfo(struct drm_printer *p, struct drm_file *file)
 void xe_drm_client_fdinfo(struct drm_printer *p, struct drm_file *file)
 {
 	show_meminfo(p, file);
+	show_runtime(p, file);
 }
 #endif
Signature · 1 line
2.43.0
#12 lore raw

✓ CI.Patch_applied: success for drm/xe: Per client usage (rev3)

In reply to Lucas De Marchi
== Series Details ==

== Series Details ==

Series: drm/xe: Per client usage (rev3)
URL : https://patchwork.freedesktop.org/series/132477/
State : success

== Summary ==

=== Applying kernel patches on branch 'drm-tip' with base: ===
Base commit: 703bc96550be drm-tip: 2024y-05m-07d-21h-25m-13s UTC integration manifest === git am output follows ===
Applying: drm/xe: Promote xe_hw_engine_class_to_str()
Applying: drm/xe: Add XE_ENGINE_CLASS_OTHER to str conversion
Applying: drm/xe/lrc: Add helper to capture context timestamp
Applying: drm/xe: Add helper to capture engine timestamp
Applying: drm/xe: Add helper to accumulate exec queue runtime
Applying: drm/xe/client: Print runtime to fdinfo

#13 lore raw

✗ CI.checkpatch: warning for drm/xe: Per client usage (rev3)

In reply to Lucas De Marchi
== Series Details ==

== Series Details ==

Series: drm/xe: Per client usage (rev3)
URL : https://patchwork.freedesktop.org/series/132477/
State : warning

== Summary ==

+ KERNEL=/kernel
+ git clone https://gitlab.freedesktop.org/drm/maintainer-tools mt Cloning into 'mt'...
warning: redirecting to https://gitlab.freedesktop.org/drm/maintainer-tools.git/
+ git -C mt rev-list -n1 origin/master 0daf0be5bb95eb0a0e42275e00a0e42d8d8fd543
+ cd /kernel
+ git config --global --add safe.directory /kernel
+ git log -n1
commit 62d75efcaf3b2ababfe9f605d2cb8532de4a3b3b

Author: Lucas De Marchi <[email protected]>
Date:   Tue May 7 15:45:10 2024 -0700

drm/xe/client: Print runtime to fdinfo

Print the accumulated runtime for client when printing fdinfo. Each time a query is done it first does 2 things:

1) loop through all the exec queues for the current client and accumulate the runtime, per engine class. CTX_TIMESTAMP is used for that, being read from the context image.

2) Read a "GPU timestamp" that can be used for considering "how much GPU time has passed" and that has the same unit/refclock as the one recording the runtime. RING_TIMESTAMP is used for that via MMIO.

Since for all current platforms RING_TIMESTAMP follows the same refclock, just read it once, using any first engine.

This is exported to userspace as 2 numbers in fdinfo:

drm-cycles-<class>: <RUNTIME>
drm-total-cycles-<class>: <TIMESTAMP>

Userspace is expected to collect at least 2 samples, which allows to know the client engine busyness as per:

RUNTIME1 - RUNTIME0
busyness = ---------------------
T1 - T0

Another thing to point out is that it's expected that userspace will read any 2 samples every few seconds. Given the update frequency of the counters involved and that CTX_TIMESTAMP is 32-bits, the counter for each exec_queue can wrap around (assuming 100% utilization) after ~200s. The wraparound is not perceived by userspace since it's just accumulated for all the exec_queues in a 64-bit counter), but the measurement will not be accurate if the samples are too far apart.

This could be mitigated by adding a workqueue to accumulate the counters every so often, but it's additional complexity for something that is done already by userspace every few seconds in tools like gputop (from igt), htop, nvtop, etc with none of them really defaulting to 1 sample per minute or more.

+ /mt/dim checkpatch 703bc96550be8cd9f4995bfa5126ca8a2e98c58c drm-intel 0e581f24ae59 drm/xe: Promote xe_hw_engine_class_to_str()
f90b2d51606a drm/xe: Add XE_ENGINE_CLASS_OTHER to str conversion
11294dbb0160 drm/xe/lrc: Add helper to capture context timestamp
187c54795475 drm/xe: Add helper to capture engine timestamp
3d8ecf1d30ce drm/xe: Add helper to accumulate exec queue runtime
62d75efcaf3b drm/xe/client: Print runtime to fdinfo
Traceback (most recent call last):
File "scripts/spdxcheck.py", line 6, in <module>
from ply import lex, yacc
ModuleNotFoundError: No module named 'ply'
-:89: WARNING:FILE_PATH_CHANGES: added, moved or deleted file(s), does MAINTAINERS need updating? #89:
new file mode 100644

-:138: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #138: FILE: drivers/gpu/drm/xe/xe_drm_client.c:33:
+ * ^Ipos: 0$

-:139: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #139: FILE: drivers/gpu/drm/xe/xe_drm_client.c:34:
+ * ^Iflags: 0100002$

-:140: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #140: FILE: drivers/gpu/drm/xe/xe_drm_client.c:35:
+ * ^Imnt_id: 26$

-:141: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #141: FILE: drivers/gpu/drm/xe/xe_drm_client.c:36:
+ * ^Iino: 685$

-:142: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #142: FILE: drivers/gpu/drm/xe/xe_drm_client.c:37:
+ * ^Idrm-driver: xe$

-:143: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #143: FILE: drivers/gpu/drm/xe/xe_drm_client.c:38:
+ * ^Idrm-client-id: 3$

-:144: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #144: FILE: drivers/gpu/drm/xe/xe_drm_client.c:39:
+ * ^Idrm-pdev: 0000:03:00.0$

-:145: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #145: FILE: drivers/gpu/drm/xe/xe_drm_client.c:40:
+ * ^Idrm-total-system: 0$

-:146: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #146: FILE: drivers/gpu/drm/xe/xe_drm_client.c:41:
+ * ^Idrm-shared-system: 0$

-:147: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #147: FILE: drivers/gpu/drm/xe/xe_drm_client.c:42:
+ * ^Idrm-active-system: 0$

-:148: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #148: FILE: drivers/gpu/drm/xe/xe_drm_client.c:43:
+ * ^Idrm-resident-system: 0$

-:149: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #149: FILE: drivers/gpu/drm/xe/xe_drm_client.c:44:
+ * ^Idrm-purgeable-system: 0$

-:150: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #150: FILE: drivers/gpu/drm/xe/xe_drm_client.c:45:
+ * ^Idrm-total-gtt: 192 KiB$

-:151: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #151: FILE: drivers/gpu/drm/xe/xe_drm_client.c:46:
+ * ^Idrm-shared-gtt: 0$

-:152: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #152: FILE: drivers/gpu/drm/xe/xe_drm_client.c:47:
+ * ^Idrm-active-gtt: 0$

-:153: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #153: FILE: drivers/gpu/drm/xe/xe_drm_client.c:48:
+ * ^Idrm-resident-gtt: 192 KiB$

-:154: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #154: FILE: drivers/gpu/drm/xe/xe_drm_client.c:49:
+ * ^Idrm-total-vram0: 23992 KiB$

-:155: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #155: FILE: drivers/gpu/drm/xe/xe_drm_client.c:50:
+ * ^Idrm-shared-vram0: 16 MiB$

-:156: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #156: FILE: drivers/gpu/drm/xe/xe_drm_client.c:51:
+ * ^Idrm-active-vram0: 0$

-:157: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #157: FILE: drivers/gpu/drm/xe/xe_drm_client.c:52:
+ * ^Idrm-resident-vram0: 23992 KiB$

-:158: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #158: FILE: drivers/gpu/drm/xe/xe_drm_client.c:53:
+ * ^Idrm-total-stolen: 0$

-:159: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #159: FILE: drivers/gpu/drm/xe/xe_drm_client.c:54:
+ * ^Idrm-shared-stolen: 0$

-:160: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #160: FILE: drivers/gpu/drm/xe/xe_drm_client.c:55:
+ * ^Idrm-active-stolen: 0$

-:161: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #161: FILE: drivers/gpu/drm/xe/xe_drm_client.c:56:
+ * ^Idrm-resident-stolen: 0$

-:162: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #162: FILE: drivers/gpu/drm/xe/xe_drm_client.c:57:
+ * ^Idrm-cycles-rcs: 28257900$

-:163: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #163: FILE: drivers/gpu/drm/xe/xe_drm_client.c:58:
+ * ^Idrm-total-cycles-rcs: 7655183225$

-:164: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #164: FILE: drivers/gpu/drm/xe/xe_drm_client.c:59:
+ * ^Idrm-cycles-bcs: 0$

-:165: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #165: FILE: drivers/gpu/drm/xe/xe_drm_client.c:60:
+ * ^Idrm-total-cycles-bcs: 7655183225$

-:166: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #166: FILE: drivers/gpu/drm/xe/xe_drm_client.c:61:
+ * ^Idrm-cycles-vcs: 0$

-:167: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #167: FILE: drivers/gpu/drm/xe/xe_drm_client.c:62:
+ * ^Idrm-total-cycles-vcs: 7655183225$

-:168: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #168: FILE: drivers/gpu/drm/xe/xe_drm_client.c:63:
+ * ^Idrm-engine-capacity-vcs: 2$

-:169: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #169: FILE: drivers/gpu/drm/xe/xe_drm_client.c:64:
+ * ^Idrm-cycles-vecs: 0$

-:170: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #170: FILE: drivers/gpu/drm/xe/xe_drm_client.c:65:
+ * ^Idrm-total-cycles-vecs: 7655183225$

-:171: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #171: FILE: drivers/gpu/drm/xe/xe_drm_client.c:66:
+ * ^Idrm-engine-capacity-vecs: 2$

-:172: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #172: FILE: drivers/gpu/drm/xe/xe_drm_client.c:67:
+ * ^Idrm-cycles-ccs: 0$

-:173: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #173: FILE: drivers/gpu/drm/xe/xe_drm_client.c:68:
+ * ^Idrm-total-cycles-ccs: 7655183225$

-:174: WARNING:SPACE_BEFORE_TAB: please, no space before tabs #174: FILE: drivers/gpu/drm/xe/xe_drm_client.c:69:
+ * ^Idrm-engine-capacity-ccs: 4$

total: 0 errors, 38 warnings, 0 checks, 200 lines checked

#14 lore raw

✗ CI.KUnit: failure for drm/xe: Per client usage (rev3)

In reply to Lucas De Marchi
== Series Details ==

== Series Details ==

Series: drm/xe: Per client usage (rev3)
URL : https://patchwork.freedesktop.org/series/132477/
State : failure

== Summary ==

+ trap cleanup EXIT
+ /kernel/tools/testing/kunit/kunit.py run --kunitconfig /kernel/drivers/gpu/drm/xe/.kunitconfig ERROR:root:../arch/x86/um/user-offsets.c:17:6: warning: no previous prototype for ‘foo’ [-Wmissing-prototypes] 17 | void foo(void)

      |      ^~~

In file included from ../arch/um/kernel/asm-offsets.c:1:
../arch/x86/um/shared/sysdep/kernel-offsets.h:9:6: warning: no previous prototype for ‘foo’ [-Wmissing-prototypes]
9 | void foo(void)
| ^~~
../arch/x86/um/bugs_64.c:9:6: warning: no previous prototype for ‘arch_check_bugs’ [-Wmissing-prototypes]
9 | void arch_check_bugs(void)
| ^~~~~~~~~~~~~~~
../arch/x86/um/bugs_64.c:13:6: warning: no previous prototype for ‘arch_examine_signal’ [-Wmissing-prototypes]
13 | void arch_examine_signal(int sig, struct uml_pt_regs *regs)

      |      ^~~~~~~~~~~~~~~~~~~
../arch/x86/um/fault.c:18:5: warning: no previous prototype for ‘arch_fixup’ [-Wmissing-prototypes]
   18 | int arch_fixup(unsigned long address, struct uml_pt_regs *regs)
      |     ^~~~~~~~~~

../arch/x86/um/os-Linux/mcontext.c:7:6: warning: no previous prototype for ‘get_regs_from_mc’ [-Wmissing-prototypes]
7 | void get_regs_from_mc(struct uml_pt_regs *regs, mcontext_t *mc)
| ^~~~~~~~~~~~~~~~
../arch/x86/um/os-Linux/registers.c:146:15: warning: no previous prototype for ‘get_thread_reg’ [-Wmissing-prototypes]
146 | unsigned long get_thread_reg(int reg, jmp_buf *buf)

      |               ^~~~~~~~~~~~~~
../arch/x86/um/vdso/um_vdso.c:16:5: warning: no previous prototype for ‘__vdso_clock_gettime’ [-Wmissing-prototypes]
   16 | int __vdso_clock_gettime(clockid_t clock, struct __kernel_old_timespec *ts)
      |     ^~~~~~~~~~~~~~~~~~~~
../arch/x86/um/vdso/um_vdso.c:30:5: warning: no previous prototype for ‘__vdso_gettimeofday’ [-Wmissing-prototypes]
   30 | int __vdso_gettimeofday(struct __kernel_old_timeval *tv, struct timezone *tz)
      |     ^~~~~~~~~~~~~~~~~~~
../arch/x86/um/vdso/um_vdso.c:44:21: warning: no previous prototype for ‘__vdso_time’ [-Wmissing-prototypes]
   44 | __kernel_old_time_t __vdso_time(__kernel_old_time_t *t)
      |                     ^~~~~~~~~~~
../arch/x86/um/vdso/um_vdso.c:57:1: warning: no previous prototype for ‘__vdso_getcpu’ [-Wmissing-prototypes]
   57 | __vdso_getcpu(unsigned *cpu, unsigned *node, struct getcpu_cache *unused)
      | ^~~~~~~~~~~~~
../arch/um/os-Linux/skas/process.c:107:6: warning: no previous prototype for ‘wait_stub_done’ [-Wmissing-prototypes]
  107 | void wait_stub_done(int pid)
      |      ^~~~~~~~~~~~~~
../arch/um/os-Linux/skas/process.c:683:6: warning: no previous prototype for ‘__switch_mm’ [-Wmissing-prototypes]
  683 | void __switch_mm(struct mm_id *mm_idp)
      |      ^~~~~~~~~~~
../arch/um/os-Linux/main.c:187:7: warning: no previous prototype for ‘__wrap_malloc’ [-Wmissing-prototypes]
  187 | void *__wrap_malloc(int size)
      |       ^~~~~~~~~~~~~
../arch/um/os-Linux/main.c:208:7: warning: no previous prototype for ‘__wrap_calloc’ [-Wmissing-prototypes]
  208 | void *__wrap_calloc(int n, int size)
      |       ^~~~~~~~~~~~~
../arch/um/os-Linux/main.c:222:6: warning: no previous prototype for ‘__wrap_free’ [-Wmissing-prototypes]
  222 | void __wrap_free(void *ptr)
      |      ^~~~~~~~~~~
../arch/x86/um/ptrace_64.c:111:5: warning: no previous prototype for ‘poke_user’ [-Wmissing-prototypes]
  111 | int poke_user(struct task_struct *child, long addr, long data)
      |     ^~~~~~~~~
../arch/x86/um/ptrace_64.c:171:5: warning: no previous prototype for ‘peek_user’ [-Wmissing-prototypes]
  171 | int peek_user(struct task_struct *child, long addr, long data)
      |     ^~~~~~~~~
../arch/um/os-Linux/mem.c:28:6: warning: no previous prototype for ‘kasan_map_memory’ [-Wmissing-prototypes]
   28 | void kasan_map_memory(void *start, size_t len)
      |      ^~~~~~~~~~~~~~~~
../arch/um/os-Linux/mem.c:212:13: warning: no previous prototype for ‘check_tmpexec’ [-Wmissing-prototypes]
  212 | void __init check_tmpexec(void)
      |             ^~~~~~~~~~~~~
../arch/um/kernel/skas/process.c:36:12: warning: no previous prototype for ‘start_uml’ [-Wmissing-prototypes]
   36 | int __init start_uml(void)
      |            ^~~~~~~~~
../arch/um/kernel/skas/mmu.c:17:5: warning: no previous prototype for ‘init_new_context’ [-Wmissing-prototypes]
   17 | int init_new_context(struct task_struct *task, struct mm_struct *mm)
      |     ^~~~~~~~~~~~~~~~
../arch/um/kernel/skas/mmu.c:60:6: warning: no previous prototype for ‘destroy_context’ [-Wmissing-prototypes]
   60 | void destroy_context(struct mm_struct *mm)
      |      ^~~~~~~~~~~~~~~
../arch/um/os-Linux/signal.c:75:6: warning: no previous prototype for ‘sig_handler’ [-Wmissing-prototypes]
   75 | void sig_handler(int sig, struct siginfo *si, mcontext_t *mc)
      |      ^~~~~~~~~~~
../arch/um/os-Linux/signal.c:111:6: warning: no previous prototype for ‘timer_alarm_handler’ [-Wmissing-prototypes]
  111 | void timer_alarm_handler(int sig, struct siginfo *unused_si, mcontext_t *mc)
      |      ^~~~~~~~~~~~~~~~~~~
../arch/x86/um/signal.c:560:6: warning: no previous prototype for ‘sys_rt_sigreturn’ [-Wmissing-prototypes]
  560 | long sys_rt_sigreturn(void)
      |      ^~~~~~~~~~~~~~~~
../arch/um/os-Linux/start_up.c:301:12: warning: no previous prototype for ‘parse_iomem’ [-Wmissing-prototypes]
  301 | int __init parse_iomem(char *str, int *add)
      |            ^~~~~~~~~~~
../arch/um/kernel/mem.c:202:8: warning: no previous prototype for ‘pgd_alloc’ [-Wmissing-prototypes]
  202 | pgd_t *pgd_alloc(struct mm_struct *mm)
      |        ^~~~~~~~~
../arch/um/kernel/mem.c:215:7: warning: no previous prototype for ‘uml_kmalloc’ [-Wmissing-prototypes]
  215 | void *uml_kmalloc(int size, int flags)
      |       ^~~~~~~~~~~
../arch/um/kernel/reboot.c:45:6: warning: no previous prototype for ‘machine_restart’ [-Wmissing-prototypes]
   45 | void machine_restart(char * __unused)
      |      ^~~~~~~~~~~~~~~
../arch/um/kernel/reboot.c:51:6: warning: no previous prototype for ‘machine_power_off’ [-Wmissing-prototypes]
   51 | void machine_power_off(void)
      |      ^~~~~~~~~~~~~~~~~
../arch/um/kernel/reboot.c:57:6: warning: no previous prototype for ‘machine_halt’ [-Wmissing-prototypes]
   57 | void machine_halt(void)
      |      ^~~~~~~~~~~~
../arch/um/kernel/process.c:51:5: warning: no previous prototype for ‘pid_to_processor_id’ [-Wmissing-prototypes]
   51 | int pid_to_processor_id(int pid)
      |     ^~~~~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:87:7: warning: no previous prototype for ‘__switch_to’ [-Wmissing-prototypes]
   87 | void *__switch_to(struct task_struct *from, struct task_struct *to)
      |       ^~~~~~~~~~~
../arch/um/kernel/process.c:140:6: warning: no previous prototype for ‘fork_handler’ [-Wmissing-prototypes]
  140 | void fork_handler(void)
      |      ^~~~~~~~~~~~
../arch/um/kernel/process.c:217:6: warning: no previous prototype for ‘arch_cpu_idle’ [-Wmissing-prototypes]
  217 | void arch_cpu_idle(void)
      |      ^~~~~~~~~~~~~
../arch/um/kernel/process.c:253:5: warning: no previous prototype for ‘copy_to_user_proc’ [-Wmissing-prototypes]
  253 | int copy_to_user_proc(void __user *to, void *from, int size)
      |     ^~~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:263:5: warning: no previous prototype for ‘clear_user_proc’ [-Wmissing-prototypes]
  263 | int clear_user_proc(void __user *buf, int size)
      |     ^~~~~~~~~~~~~~~
../arch/um/kernel/process.c:271:6: warning: no previous prototype for ‘set_using_sysemu’ [-Wmissing-prototypes]
  271 | void set_using_sysemu(int value)
      |      ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:278:5: warning: no previous prototype for ‘get_using_sysemu’ [-Wmissing-prototypes]
  278 | int get_using_sysemu(void)
      |     ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:316:12: warning: no previous prototype for ‘make_proc_sysemu’ [-Wmissing-prototypes]
  316 | int __init make_proc_sysemu(void)
      |            ^~~~~~~~~~~~~~~~
../arch/um/kernel/process.c:348:15: warning: no previous prototype for ‘arch_align_stack’ [-Wmissing-prototypes]
  348 | unsigned long arch_align_stack(unsigned long sp)
      |               ^~~~~~~~~~~~~~~~
../arch/um/kernel/tlb.c:579:6: warning: no previous prototype for ‘flush_tlb_mm_range’ [-Wmissing-prototypes]
  579 | void flush_tlb_mm_range(struct mm_struct *mm, unsigned long start,
      |      ^~~~~~~~~~~~~~~~~~
../arch/um/kernel/tlb.c:594:6: warning: no previous prototype for ‘force_flush_all’ [-Wmissing-prototypes]
  594 | void force_flush_all(void)
      |      ^~~~~~~~~~~~~~~
../arch/x86/um/syscalls_64.c:48:6: warning: no previous prototype for ‘arch_switch_to’ [-Wmissing-prototypes]
   48 | void arch_switch_to(struct task_struct *to)
      |      ^~~~~~~~~~~~~~
../arch/um/kernel/um_arch.c:408:19: warning: no previous prototype for ‘read_initrd’ [-Wmissing-prototypes]
  408 | int __init __weak read_initrd(void)
      |                   ^~~~~~~~~~~
../arch/um/kernel/um_arch.c:461:7: warning: no previous prototype for ‘text_poke’ [-Wmissing-prototypes]
  461 | void *text_poke(void *addr, const void *opcode, size_t len)
      |       ^~~~~~~~~
../arch/um/kernel/um_arch.c:473:6: warning: no previous prototype for ‘text_poke_sync’ [-Wmissing-prototypes]
  473 | void text_poke_sync(void)
      |      ^~~~~~~~~~~~~~
../arch/um/kernel/kmsg_dump.c:60:12: warning: no previous prototype for ‘kmsg_dumper_stdout_init’ [-Wmissing-prototypes]
   60 | int __init kmsg_dumper_stdout_init(void)
      |            ^~~~~~~~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_drm_client.c: In function ‘show_runtime’:
../drivers/gpu/drm/xe/xe_drm_client.c:279:3: error: implicit declaration of function ‘xe_force_wake_get’ [-Werror=implicit-function-declaration]
  279 |   xe_force_wake_get(gt_to_fw(gt), XE_FW_GT);
      |   ^~~~~~~~~~~~~~~~~
../drivers/gpu/drm/xe/xe_drm_client.c:309:3: error: implicit declaration of function ‘xe_force_wake_put’ [-Werror=implicit-function-declaration]
  309 |   xe_force_wake_put(gt_to_fw(gt), XE_FW_GT);
      |   ^~~~~~~~~~~~~~~~~

cc1: some warnings being treated as errors
make[7]: *** [../scripts/Makefile.build:244: drivers/gpu/drm/xe/xe_drm_client.o] Error 1 make[7]: *** Waiting for unfinished jobs....
make[6]: *** [../scripts/Makefile.build:485: drivers/gpu/drm/xe] Error 2 make[6]: *** Waiting for unfinished jobs....
make[5]: *** [../scripts/Makefile.build:485: drivers/gpu/drm] Error 2
make[4]: *** [../scripts/Makefile.build:485: drivers/gpu] Error 2
make[4]: *** Waiting for unfinished jobs....
make[3]: *** [../scripts/Makefile.build:485: drivers] Error 2
make[3]: *** Waiting for unfinished jobs....
../lib/iomap.c:156:5: warning: no previous prototype for ‘ioread64_lo_hi’ [-Wmissing-prototypes]
156 | u64 ioread64_lo_hi(const void __iomem *addr)

      |     ^~~~~~~~~~~~~~
../lib/iomap.c:163:5: warning: no previous prototype for ‘ioread64_hi_lo’ [-Wmissing-prototypes]
  163 | u64 ioread64_hi_lo(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~
../lib/iomap.c:170:5: warning: no previous prototype for ‘ioread64be_lo_hi’ [-Wmissing-prototypes]
  170 | u64 ioread64be_lo_hi(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~~~
../lib/iomap.c:178:5: warning: no previous prototype for ‘ioread64be_hi_lo’ [-Wmissing-prototypes]
  178 | u64 ioread64be_hi_lo(const void __iomem *addr)
      |     ^~~~~~~~~~~~~~~~
../lib/iomap.c:264:6: warning: no previous prototype for ‘iowrite64_lo_hi’ [-Wmissing-prototypes]
  264 | void iowrite64_lo_hi(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~
../lib/iomap.c:272:6: warning: no previous prototype for ‘iowrite64_hi_lo’ [-Wmissing-prototypes]
  272 | void iowrite64_hi_lo(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~
../lib/iomap.c:280:6: warning: no previous prototype for ‘iowrite64be_lo_hi’ [-Wmissing-prototypes]
  280 | void iowrite64be_lo_hi(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~~~
../lib/iomap.c:288:6: warning: no previous prototype for ‘iowrite64be_hi_lo’ [-Wmissing-prototypes]
  288 | void iowrite64be_hi_lo(u64 val, void __iomem *addr)
      |      ^~~~~~~~~~~~~~~~~

make[2]: *** [/kernel/Makefile:1919: .] Error 2 make[1]: *** [/kernel/Makefile:240: __sub-make] Error 2
make: *** [Makefile:240: __sub-make] Error 2

[01:34:25] Configuring KUnit Kernel ...
Generating .config ...
Populating config with:

$ make ARCH=um O=.kunit olddefconfig
[01:34:29] Building KUnit Kernel ...
Populating config with:
$ make ARCH=um O=.kunit olddefconfig
Building with:
$ make ARCH=um O=.kunit --jobs=48
+ cleanup
++ stat -c %u:%g /kernel
+ chown -R 1003:1003 /kernel