Skip to contentSkip to conversation

Test coverage for GPU debug support

PATCH Superseded Christoph Manszewski 3 participants 51 messages Aug 29, 2024 – Sep 4, 2024 Page 1 of 2

Subsystems (2)

v5 of 7 Superseded Latest is v7 →

lore
Share & tools
lore
#0 lore raw

v5 cover Test coverage for GPU debug support

Hi,

In this series the eudebug kernel and validation team would like to add test coverage for GPU debug support recently proposed as an RFC.
(https://patchwork.freedesktop.org/series/136572/)

This series adds 'xe_eudebug' and 'xe_eudebug_online' tests together with a library that encapsulates common paths in current and future EU debugger scenarios. It also extends the 'xe_exec_sip' test and 'gpgpu_shader' library.

The aim of the 'xe_eudebug' test is to validate the eudebug resource tracking and event delivery mechanism. The 'xe_eudebug_online' test is dedicated for 'online' scenarios which means scenarios that exercise hardware exception handling and thread state manipulation.

The xe_eudebug library provides an abstraction over debugger and debuggee processes, asynchronous event reader, and event log buffers for post-mortem analysis.

Latest kernel code can be found here:
https://gitlab.freedesktop.org/miku/kernel/-/commits/eudebug-dev

Thank you in advance for any comments and insight.

v2:
- make sure to include all patches and verify that each individual patch compiles (Zbigniew)

v3:
- fix multiple typos (Dominik Karol),
- squash subtest and eudebug lib patches (Zbigniew),
- include uapi sync/fix (Kamil)

v4:
- move all eudebug uapi changes to xe_drm_eudebug.h (Zbigniew),
- move some 'xe_exec_sip' tests to 'xe_exec_sip_eudebug' test (Zbigniew),
- control eudebug lib and test build with meson flag and disable it by default (Zbigniew),
- fix multiple checkpatch issues (Kamil),
- apply review comments from Dominik,

v5:
- add comment to 'xe_drm_eudebug' (Zbigniew),
- misc fixes and cleanups in gpgpu_shader.[ch] (Zbigniew),
- assert on offset in intel_bb_ptr_get (Zbigniew),
- use enum for shader and sip type in 'xe_exec_sip[_eudebug]' (Zbigniew),
- fix string concatenation issue for meson older than 0.49,
- more (hopefully all relevant) checkpatch issues addressed,

Andrzej Hajda (5):
lib/gpgpu_shader: Add write_on_exception template lib/gpgpu_shader: Add set/clear exception register (cr0.1) helpers lib/intel_batchbuffer: Add helper to get pointer at specified offset lib/gpgpu_shader: Allow enabling illegal opcode exceptions in shader tests/xe_exec_sip: Introduce invalid instruction tests

Christoph Manszewski (7):
drm-uapi/xe: Sync with oa uapi fix
lib/xe_ioctl: Add wrapper with vm_bind_op extension parameter lib/gpgpu_shader: Extend shader building library tests/xe_exec_sip: Add sanity-after-timeout test scripts/igt_doc: Add '--exclude-files' parameter tests/xe_exec_sip_eudebug: Add SIP tests for eudebug tests/xe_live_ktest: Add xe_eudebug live test

Dominik Grzegorzek (4):
drm-uapi/xe: Sync with eudebug uapi
lib/xe_eudebug: Introduce eu debug testing framework tests/xe_eudebug: Test eudebug resource tracking and manipulation tests/xe_eudebug_online: Debug client which runs workloads on EU

Gwan-gyeong Mun (1):
lib/intel_batchbuffer: Add support for long-running mode execution

 docs/testplan/meson.build         |   22 +-
 include/drm-uapi/xe_drm.h         |   16 +-
 include/drm-uapi/xe_drm_eudebug.h |  341 ++++
 lib/gpgpu_shader.c                |  480 ++++-
 lib/gpgpu_shader.h                |   29 +-
 lib/iga64_generated_codes.c       |  547 +++++-
 lib/intel_batchbuffer.c           |  149 +-
 lib/intel_batchbuffer.h           |   24 +
 lib/meson.build                   |    5 +
 lib/xe/xe_eudebug.c               | 2198 +++++++++++++++++++++++
 lib/xe/xe_eudebug.h               |  212 +++
 lib/xe/xe_ioctl.c                 |   20 +-
 lib/xe/xe_ioctl.h                 |    5 +
 meson.build                       |    2 +
 meson_options.txt                 |    5 +
 scripts/igt_doc.py                |    3 +
 scripts/test_list.py              |   47 +-
 tests/intel/xe_eudebug.c          | 2715 +++++++++++++++++++++++++++++
 tests/intel/xe_eudebug_online.c   | 2254 ++++++++++++++++++++++++
 tests/intel/xe_exec_sip.c         |  147 +-
 tests/intel/xe_exec_sip_eudebug.c |  351 ++++
 tests/intel/xe_live_ktest.c       |    6 +
 tests/meson.build                 |    8 +
 23 files changed, 9534 insertions(+), 52 deletions(-)
 create mode 100644 include/drm-uapi/xe_drm_eudebug.h
 create mode 100644 lib/xe/xe_eudebug.c
 create mode 100644 lib/xe/xe_eudebug.h
 create mode 100644 tests/intel/xe_eudebug.c
 create mode 100644 tests/intel/xe_eudebug_online.c
 create mode 100644 tests/intel/xe_exec_sip_eudebug.c
Signature · 1 line
2.34.1
#1 lore raw

PATCH v5 1/17 drm-uapi/xe: Sync with oa uapi fix

Align with kernel commit f2881dfdaaa9 ("drm/xe/oa/uapi: Make bit masks unsigned"). Use built header instead of raw uapi header.

---
 include/drm-uapi/xe_drm.h | 16 ++++++++--------
 1 file changed, 8 insertions(+), 8 deletions(-)
include/drm-uapi/xe_drm.h +8 -8
index 29425d7fd..f0a450db9 100644
@@ -3,8 +3,8 @@
  * Copyright © 2023 Intel Corporation
  */
 
-#ifndef _UAPI_XE_DRM_H_
-#define _UAPI_XE_DRM_H_
+#ifndef _XE_DRM_H_
+#define _XE_DRM_H_
 
 #include "drm.h"
 
@@ -134,7 +134,7 @@ extern "C" {
  * redefine the interface more easily than an ever growing struct of
  * increasing complexity, and for large parts of that interface to be
  * entirely optional. The downside is more pointer chasing; chasing across
- * the __user boundary with pointers encapsulated inside u64.
+ * the boundary with pointers encapsulated inside u64.
  *
  * Example chaining:
  *
@@ -1598,10 +1598,10 @@ enum drm_xe_oa_property_id {
 	 * b. Counter select c. Counter size and d. BC report. Also refer to the
 	 * oa_formats array in drivers/gpu/drm/xe/xe_oa.c.
 	 */
-#define DRM_XE_OA_FORMAT_MASK_FMT_TYPE		(0xff << 0)
-#define DRM_XE_OA_FORMAT_MASK_COUNTER_SEL	(0xff << 8)
-#define DRM_XE_OA_FORMAT_MASK_COUNTER_SIZE	(0xff << 16)
-#define DRM_XE_OA_FORMAT_MASK_BC_REPORT		(0xff << 24)
+#define DRM_XE_OA_FORMAT_MASK_FMT_TYPE		(0xffu << 0)
+#define DRM_XE_OA_FORMAT_MASK_COUNTER_SEL	(0xffu << 8)
+#define DRM_XE_OA_FORMAT_MASK_COUNTER_SIZE	(0xffu << 16)
+#define DRM_XE_OA_FORMAT_MASK_BC_REPORT		(0xffu << 24)
 
 	/**
 	 * @DRM_XE_OA_PROPERTY_OA_PERIOD_EXPONENT: Requests periodic OA unit
@@ -1698,4 +1698,4 @@ struct drm_xe_oa_stream_info {
 }
 #endif
 
-#endif /* _UAPI_XE_DRM_H_ */
+#endif /* _XE_DRM_H_ */
Signature · 1 line
2.34.1
#2 lore raw

PATCH v5 2/17 lib/xe_ioctl: Add wrapper with vm_bind_op extension parameter

In reply to Christoph Manszewski

Currently there is no way to set drm_xe_vm_bind_op extension field. Add vm_bind wrapper that allows pass this field as a parameter.

---
 lib/xe/xe_ioctl.c | 20 ++++++++++++++++----
 lib/xe/xe_ioctl.h |  5 +++++
 2 files changed, 21 insertions(+), 4 deletions(-)
lib/xe/xe_ioctl.c +16 -4
index ae43ffd15..6d8388918 100644
@@ -96,15 +96,17 @@ void xe_vm_bind_array(int fd, uint32_t vm, uint32_t exec_queue,
 	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_VM_BIND, &bind), 0);
 }
 
-int  __xe_vm_bind(int fd, uint32_t vm, uint32_t exec_queue, uint32_t bo,
-		  uint64_t offset, uint64_t addr, uint64_t size, uint32_t op,
-		  uint32_t flags, struct drm_xe_sync *sync, uint32_t num_syncs,
-		  uint32_t prefetch_region, uint8_t pat_index, uint64_t ext)
+int  ___xe_vm_bind(int fd, uint32_t vm, uint32_t exec_queue, uint32_t bo,
+		   uint64_t offset, uint64_t addr, uint64_t size, uint32_t op,
+		   uint32_t flags, struct drm_xe_sync *sync, uint32_t num_syncs,
+		   uint32_t prefetch_region, uint8_t pat_index, uint64_t ext,
+		   uint64_t op_ext)
 {
 	struct drm_xe_vm_bind bind = {
 		.extensions = ext,
 		.vm_id = vm,
 		.num_binds = 1,
+		.bind.extensions = op_ext,
 		.bind.obj = bo,
 		.bind.obj_offset = offset,
 		.bind.range = size,
@@ -125,6 +127,16 @@ int  __xe_vm_bind(int fd, uint32_t vm, uint32_t exec_queue, uint32_t bo,
 	return 0;
 }
 
+int  __xe_vm_bind(int fd, uint32_t vm, uint32_t exec_queue, uint32_t bo,
+		  uint64_t offset, uint64_t addr, uint64_t size, uint32_t op,
+		  uint32_t flags, struct drm_xe_sync *sync, uint32_t num_syncs,
+		  uint32_t prefetch_region, uint8_t pat_index, uint64_t ext)
+{
+	return ___xe_vm_bind(fd, vm, exec_queue, bo, offset, addr, size, op,
+			     flags, sync, num_syncs, prefetch_region,
+			     pat_index, ext, 0);
+}
+
 void  __xe_vm_bind_assert(int fd, uint32_t vm, uint32_t exec_queue, uint32_t bo,
 			  uint64_t offset, uint64_t addr, uint64_t size,
 			  uint32_t op, uint32_t flags, struct drm_xe_sync *sync,
lib/xe/xe_ioctl.h +5 -0
index b27c0053f..18cc2b72b 100644
@@ -20,6 +20,11 @@
 uint32_t xe_cs_prefetch_size(int fd);
 uint64_t xe_bb_size(int fd, uint64_t reqsize);
 uint32_t xe_vm_create(int fd, uint32_t flags, uint64_t ext);
+int  ___xe_vm_bind(int fd, uint32_t vm, uint32_t exec_queue, uint32_t bo,
+		   uint64_t offset, uint64_t addr, uint64_t size, uint32_t op,
+		   uint32_t flags, struct drm_xe_sync *sync, uint32_t num_syncs,
+		   uint32_t prefetch_region, uint8_t pat_index, uint64_t ext,
+		   uint64_t op_ext);
 int  __xe_vm_bind(int fd, uint32_t vm, uint32_t exec_queue, uint32_t bo,
 		  uint64_t offset, uint64_t addr, uint64_t size, uint32_t op,
 		  uint32_t flags, struct drm_xe_sync *sync, uint32_t num_syncs,
Signature · 1 line
2.34.1
#3 lore raw

PATCH v5 3/17 lib/gpgpu_shader: Extend shader building library

In reply to Christoph Manszewski

Add shader building functions and iga64 code used by eudebug subtests.

---
 lib/gpgpu_shader.c          | 392 ++++++++++++++++++++++++++++++++++-
 lib/gpgpu_shader.h          |  24 ++-
 lib/iga64_generated_codes.c | 401 +++++++++++++++++++++++++++++++++++-
 3 files changed, 813 insertions(+), 4 deletions(-)
lib/gpgpu_shader.c +391 -1
index 80bad342a..c723577e6 100644
@@ -7,10 +7,16 @@
 
 #include <i915_drm.h>
 
+#include "igt_map.h"
 #include "ioctl_wrappers.h"
 #include "gpgpu_shader.h"
 #include "gpu_cmds.h"
 
+struct label_entry {
+	uint32_t id;
+	uint32_t offset;
+};
+
 #define IGA64_ARG0 0xc0ded000
 #define IGA64_ARG_MASK 0xffffff00
 
@@ -32,7 +38,7 @@ static void gpgpu_shader_extend(struct gpgpu_shader *shdr)
 	igt_assert(shdr->code);
 }
 
-void
+uint32_t
 __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
 		  int argc, uint32_t *argv)
 {
@@ -60,6 +66,8 @@ __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
 	}
 
 	shdr->size += tpls->size;
+
+	return tpls->size;
 }
 
 static uint32_t fill_sip(struct intel_bb *ibb,
@@ -235,10 +243,16 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
 	shdr->gen_ver = 100 * info->graphics_ver + info->graphics_rel;
 	shdr->max_size = 16 * 4;
 	shdr->code = malloc(4 * shdr->max_size);
+	shdr->labels = igt_map_create(igt_map_hash_32, igt_map_equal_32);
 	igt_assert(shdr->code);
 	return shdr;
 }
 
+static void free_func(struct igt_map_entry *entry)
+{
+       free(entry->data);
+}
+
 /**
  * gpgpu_shader_destroy:
  * @shdr: pointer to shader struct created with 'gpgpu_shader_create'
@@ -247,10 +261,76 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
  */
 void gpgpu_shader_destroy(struct gpgpu_shader *shdr)
 {
+	igt_map_destroy(shdr->labels, free_func);
 	free(shdr->code);
 	free(shdr);
 }
 
+/**
+ * gpgpu_shader_dump:
+ * @shdr: shader to be printed
+ *
+ * Print shader instructions from @shdr in hex.
+ */
+void gpgpu_shader_dump(struct gpgpu_shader *shdr)
+{
+	for (int i = 0; i < shdr->size / 4; i++)
+		igt_info("0x%08x 0x%08x 0x%08x 0x%08x\n",
+			 shdr->instr[i][0], shdr->instr[i][1],
+			 shdr->instr[i][2], shdr->instr[i][3]);
+}
+
+/**
+ * gpgpu_shader__breakpoint_on:
+ * @shdr: shader to create breakpoint in
+ * @cmd_no: index of the instruction to break on
+ *
+ * Insert a breakpoint on the @cmd_no'th instruction within @shdr.
+ */
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t cmd_no)
+{
+	igt_assert(cmd_no < shdr->size / 4);
+	shdr->instr[cmd_no][0] |= 1<<30;
+}
+
+/**
+ * gpgpu_shader__breakpoint:
+ * @shdr: shader to create breakpoint in
+ *
+ * Insert a breakpoint on the last instruction in @shdr.
+ */
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr)
+{
+	gpgpu_shader__breakpoint_on(shdr, shdr->size / 4 - 1);
+}
+
+/**
+ * gpgpu_shader__wait:
+ * @shdr: shader to be modified
+ *
+ * Append wait instruction to @shader. This instruction raises attention
+ * and stops execution.
+ */
+void gpgpu_shader__wait(struct gpgpu_shader *shdr)
+{
+	emit_iga64_code(shdr, sync_host, "	\n\
+(W)	sync.host	        null		\n\
+	");
+}
+
+/**
+ * gpgpu_shader__nop:
+ * @shdr: shader to be modified
+ *
+ * Append a no-op instruction to @shdr.
+ */
+void gpgpu_shader__nop(struct gpgpu_shader *shdr)
+{
+	emit_iga64_code(shdr, nop, "	\n\
+(W)	nop				\n\
+	");
+}
+
 /**
  * gpgpu_shader__eot:
  * @shdr: shader to be modified
@@ -269,6 +349,246 @@ void gpgpu_shader__eot(struct gpgpu_shader *shdr)
 	");
 }
 
+/**
+ * gpgpu_shader__label:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Create a label for the last instruction within @shdr.
+ */
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id)
+{
+	struct label_entry *l = malloc(sizeof(*l));
+
+	l->id = label_id;
+	l->offset = shdr->size;
+	igt_map_insert(shdr->labels, &l->id, l);
+}
+
+#define OPCODE(x) (x & 0x7f)
+#define OPCODE_JUMP_INDEXED 0x20
+static void __patch_indexed_jump(struct gpgpu_shader *shdr, int label_id,
+				 uint32_t jump_iga64_size)
+{
+	struct label_entry *l;
+	uint32_t *start, *end, *label;
+	int32_t relative;
+
+	l = igt_map_search(shdr->labels, &label_id);
+	igt_assert(l);
+
+	igt_assert(jump_iga64_size % 4 == 0);
+
+	label = shdr->code + l->offset;
+	end = shdr->code + shdr->size;
+	start = end - jump_iga64_size;
+
+	for (; start < end; start += 4)
+		if (OPCODE(*start) == OPCODE_JUMP_INDEXED) {
+			relative = (label - start) * 4;
+			*(start + 3) = relative;
+			break;
+		}
+}
+
+/**
+ * gpgpu_shader__jump:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label @label_id.
+ */
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id)
+{
+	size_t shader_size;
+
+	shader_size = emit_iga64_code(shdr, jump, "	\n\
+L0:							\n\
+(W)	jmpi        L0					\n\
+	");
+
+	__patch_indexed_jump(shdr, label_id, shader_size);
+}
+
+/**
+ * gpgpu_shader__jump_neq:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label @label_id
+ * when @value is not equal to dword stored at @y_offset within the surface.
+ */
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+			    uint32_t y_offset, uint32_t value)
+{
+	uint32_t size;
+
+	size = emit_iga64_code(shdr, jump_dw_neq, "					\n\
+L0:											\n\
+(W)		mov (16|M0)              r30.0<1>:ud    0x0:ud				\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+	// Y offset of the block in rows := thread group id Y				\n\
+(W)		mov (1|M0)               r30.1<1>:ud    ARG(0):ud			\n\
+	// block width [0,63] representing 1 to 64 bytes, we want dword			\n\
+(W)		mov (1|M0)               r30.2<1>:ud    0x3:ud				\n\
+	// FFTID := FFTID from R0 header						\n\
+(W)		mov (1|M0)               r30.4<1>:ud    r0.5<0;1,0>:ud  		\n\
+(W)		send.dc1 (16|M0)         r31     r30      null    0x0	0x2190000	\n\
+#else // Typed 2D Block Store								\n\
+	// Store X and Y block start (160:191 and 192:223)				\n\
+(W)            mov (2|M0)               r30.6<1>:ud    ARG(0):ud			\n\
+	// Store X and Y block size (224:231 and 232:239)				\n\
+(W)            mov (1|M0)               r30.7<1>:ud    0x3:ud				\n\
+(W)            send.tgm (16|M0)         r31     r30    null:0    0x0    0x62100003	\n\
+#endif											\n\
+	// clear the flag register							\n\
+(W)		mov (1|M0)               f0.0<1>:ud    0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud   ARG(1):ud	\n\
+(W&f0.0)	jmpi                     L0						\n\
+	", y_offset, value);
+
+	__patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__loop_begin:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Begin a counting loop in @shdr. All subsequent instructions will constitute
+ * the loop body up until 'gpgpu_shader__loop_end' gets called. The first
+ * instruction of the loop will be at label @label_id. The r40 register will be
+ * overwritten as it is used as the loop counter.
+ */
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id)
+{
+	emit_iga64_code(shdr, clear_r40, "		\n\
+L0:							\n\
+(W)	mov (1|M0)               r40:ud    0x0:ud	\n\
+	");
+
+	gpgpu_shader__label(shdr, label_id);
+}
+
+/**
+ * gpgpu_shader__loop_end:
+ * @shdr: shader to be modified
+ * @label_id: label id passed to 'gpgpu_shader__loop_begin'
+ * @iter: iteration count
+ *
+ * End loop body in @shdr.
+ */
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, uint32_t iter)
+{
+	uint32_t size;
+
+	size = emit_iga64_code(shdr, inc_r40_jump_neq, "				\n\
+L0:											\n\
+(W)		add (1|M0)              r40:ud          r40.0<0;1,0>:ud 0x1:ud		\n\
+(W)		mov (1|M0)              f0.0<1>:ud      0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r40.0<0;1,0>:ud   ARG(0):ud	\n\
+(W&f0.0)	jmpi                     L0						\n\
+	", iter);
+
+	__patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__common_target_write:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: oword to be written
+ *
+ * Write the oword stored in @value to the target buffer at @y_offset.
+ */
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+				       uint32_t y_offset, const uint32_t value[4])
+{
+	emit_iga64_code(shdr, common_target_write, "				\n\
+(W)	mov (16|M0)		r30.0<1>:ud	0x0:ud				\n\
+(W)	mov (16|M0)		r31.0<1>:ud	0x0:ud				\n\
+(W)	mov (1|M0)		r31.0<1>:ud	ARG(1):ud			\n\
+(W)	mov (1|M0)		r31.1<1>:ud	ARG(2):ud			\n\
+(W)	mov (1|M0)		r31.2<1>:ud	ARG(3):ud			\n\
+(W)	mov (1|M0)		r31.3<1>:ud	ARG(4):ud			\n\
+#if GEN_VER < 2000 // Media Block Write						\n\
+	// Y offset of the block in rows					\n\
+(W)	mov (1|M0)		r30.1<1>:ud	ARG(0):ud			\n\
+	// block width [0,63] representing 1 to 64 bytes			\n\
+(W)	mov (1|M0)		r30.2<1>:ud	0xf:ud				\n\
+	// FFTID := FFTID from R0 header					\n\
+(W)	mov (1|M0)		r30.4<1>:ud	r0.5<0;1,0>:ud			\n\
+	// written value							\n\
+(W)	send.dc1 (16|M0)	null	r30	src1_null  0x0	0x40A8000	\n\
+#else	// Typed 2D Block Store							\n\
+	// Store X and Y block start (160:191 and 192:223)			\n\
+(W)	mov (2|M0)              r30.6<1>:ud     ARG(0):ud			\n\
+	// Store X and Y block size (224:231 and 232:239)			\n\
+(W)	mov (1|M0)              r30.7<1>:ud     0xf:ud				\n\
+(W)	send.tgm (16|M0)        null    r30     null:0  0x0     0x64000007	\n\
+#endif										\n\
+	", y_offset, value[0], value[1], value[2], value[3]);
+}
+
+/**
+ * gpgpu_shader__common_target_write_u32:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: dword to be written
+ *
+ * Fill oword at @y_offset with dword stored in @value.
+ */
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+					   uint32_t y_offset, uint32_t value)
+{
+	const uint32_t owblock[4] = {
+		value, value, value, value
+	};
+	gpgpu_shader__common_target_write(shdr, y_offset, owblock);
+}
+
+/**
+ * gpgpu_shader__write_aip:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within the surface in rows
+ *
+ * Write address instruction pointer to row tg_id_y + @y_offset.
+ */
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset)
+{
+	emit_iga64_code(shdr, media_block_write_aip, "				\n\
+	// Payload								\n\
+(W)	mov (1|M0)               r5.0<1>:ud    cr0.2:ud				\n\
+#if GEN_VER < 2000 // Media Block Write						\n\
+	// X offset of the block in bytes := (thread group id X << ARG(0))	\n\
+(W)	shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud    0x2:ud		\n\
+	// Y offset of the block in rows := thread group id Y			\n\
+(W)	mov (1|M0)               r4.1<1>:ud    r0.6<0;1,0>:ud			\n\
+(W)	add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud    ARG(0):ud	\n\
+	// block width [0,63] representing 1 to 64 bytes			\n\
+(W)	mov (1|M0)               r4.2<1>:ud    0x3:ud				\n\
+	// FFTID := FFTID from R0 header					\n\
+(W)	mov (1|M0)               r4.4<1>:ud    r0.5<0;1,0>:ud			\n\
+(W)	send.dc1 (16|M0)         null     r4   src1_null 0       0x40A8000	\n\
+#else // Typed 2D Block Store							\n\
+	// Load r2.0-3 with tg id X << ARG(0)					\n\
+(W)	shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud    0x2:ud		\n\
+	// Load r2.4-7 with tg id Y + ARG(1):ud					\n\
+(W)	mov (1|M0)               r2.1<1>:ud    r0.6<0;1,0>:ud			\n\
+(W)	add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud    ARG(0):ud	\n\
+	// payload setup							\n\
+(W)	mov (16|M0)              r4.0<1>:ud    0x0:ud				\n\
+	// Store X and Y block start (160:191 and 192:223)			\n\
+(W)	mov (2|M0)               r4.5<1>:ud    r2.0<2;2,1>:ud			\n\
+	// Store X and Y block max_size (224:231 and 232:239)			\n\
+(W)	mov (1|M0)               r4.7<1>:ud    0x3:ud				\n\
+(W)	send.tgm (16|M0)         null     r4   null:0    0    0x64000007	\n\
+#endif										\n\
+	", y_offset);
+}
+
 /**
  * gpgpu_shader__write_dword:
  * @shdr: shader to be modified
@@ -313,3 +633,73 @@ void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 #endif										\n\
 	", 2, y_offset, 3, value, value, value, value);
 }
+
+/**
+ * gpgpu_shader__end_system_routine:
+ * @shdr: shader to be modified
+ * @breakpoint_suppress: breakpoint suppress flag
+ *
+ * Return from system routine. To prevent infinite jumping to the system
+ * routine on a breakpoint, @breakpoint_suppress flag has to be set.
+ */
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+				      bool breakpoint_suppress)
+{
+	/*
+	 * set breakpoint suppress bit to avoid an endless loop
+	 * when sip was invoked by a breakpoint
+	 */
+	if (breakpoint_suppress)
+		emit_iga64_code(shdr, breakpoint_suppress, "			\n\
+(W)	or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x8000:ud	\n\
+		");
+
+	emit_iga64_code(shdr, end_system_routine, "				\n\
+(W)	and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   ARG(0):ud	\n\
+	// return to an application						\n\
+(W)	and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x7FFFFFFD:ud	\n\
+	", 0x7fffff | (1 << 26)); /* clear all exceptions, except read only bit */
+}
+
+/**
+ * gpgpu_shader__end_system_routine_step_if_eq:
+ * @shdr: shader to be modified
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value for single stepping execution
+ *
+ * Return from system routine. Don't clear breakpoint exception when @value
+ * is equal to value stored at @y_offset. This triggers the system routine
+ * after the subsequent instruction, resulting in single stepping execution.
+ */
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+						 uint32_t y_offset,
+						 uint32_t value)
+{
+	emit_iga64_code(shdr, end_system_routine_step_if_eq, "				\n\
+(W)		or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x8000:ud	\n\
+(W)		and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   ARG(0):ud	\n\
+(W)		mov (16|M0)              r30.0<1>:ud    0x0:ud				\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+		// Y offset of the block in rows := thread group id Y			\n\
+(W)		mov (1|M0)               r30.1<1>:ud    ARG(1):ud			\n\
+		// block width [0,63] representing 1 to 64 bytes, we want dword		\n\
+(W)		mov (1|M0)               r30.2<1>:ud    0x3:ud				\n\
+		// FFTID := FFTID from R0 header					\n\
+(W)		mov (1|M0)               r30.4<1>:ud    r0.5<0;1,0>:ud			\n\
+(W)		send.dc1 (16|M0)         r31     r30      null    0x0	0x2190000	\n\
+#else	// Typed 2D Block Store								\n\
+		// Store X and Y block start (160:191 and 192:223)			\n\
+(W)		mov (2|M0)               r30.6<1>:ud    ARG(1):ud			\n\
+		// Store X and Y block size (224:231 and 232:239)			\n\
+(W)		mov (1|M0)               r30.7<1>:ud    0x3:ud				\n\
+(W)		send.tgm (16|M0)         r31     r30    null:0    0x0    0x62100003	\n\
+#endif											\n\
+		// clear the flag register						\n\
+(W)		mov (1|M0)               f0.0<1>:ud    0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud   ARG(2):ud	\n\
+(W&f0.0)	and (1|M0)              cr0.1<1>:ud     cr0.1<0;1,0>:ud   ARG(3):ud	\n\
+		// return to an application						\n\
+(W)		and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x7FFFFFFD:ud	\n\
+	", 0x807fffff, /* leave breakpoint exception */
+	y_offset, value, 0x7fffff /* clear all exceptions */ );
+}
lib/gpgpu_shader.h +22 -2
index 255f93b4d..e4ca0be4c 100644
@@ -21,6 +21,7 @@ struct gpgpu_shader {
 		uint32_t *code;
 		uint32_t (*instr)[4];
 	};
+	struct igt_map *labels;
 };
 
 struct iga64_template {
@@ -31,7 +32,7 @@ struct iga64_template {
 
 #pragma GCC diagnostic ignored "-Wnested-externs"
 
-void
+uint32_t
 __emit_iga64_code(struct gpgpu_shader *shdr, const struct iga64_template *tpls,
 		  int argc, uint32_t *argv);
 
@@ -56,8 +57,27 @@ void gpgpu_shader_exec(struct intel_bb *ibb,
 		       struct gpgpu_shader *sip,
 		       uint64_t ring, bool explicit_engine);
 
+void gpgpu_shader__wait(struct gpgpu_shader *shdr);
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t cmd_no);
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr);
+void gpgpu_shader__nop(struct gpgpu_shader *shdr);
 void gpgpu_shader__eot(struct gpgpu_shader *shdr);
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+				       uint32_t y_offset, const uint32_t value[4]);
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+				     uint32_t y_offset, uint32_t value);
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+				      bool breakpoint_suppress);
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+						 uint32_t dw_offset,
+						 uint32_t value);
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset);
 void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 			       uint32_t y_offset);
-
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+			    uint32_t dw_offset, uint32_t value);
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, uint32_t iter);
 #endif /* GPGPU_SHADER_H */
lib/iga64_generated_codes.c +400 -1

422 lines not on this page · raw message

Signature · 1 line
2.34.1
#4 lore raw

Re: [PATCH i-g-t v5 03/17] lib/gpgpu_shader: Extend shader building library

Quoted 92 lines · Christoph Manszewski

Add shader building functions and iga64 code used by eudebug subtests.

---
 lib/gpgpu_shader.c          | 392 ++++++++++++++++++++++++++++++++++-
 lib/gpgpu_shader.h          |  24 ++-
 lib/iga64_generated_codes.c | 401 +++++++++++++++++++++++++++++++++++-
 3 files changed, 813 insertions(+), 4 deletions(-)
lib/gpgpu_shader.c +30 -1
index 80bad342a..c723577e6 100644
@@ -7,10 +7,16 @@
 
 #include <i915_drm.h>
 
+#include "igt_map.h"
 #include "ioctl_wrappers.h"
 #include "gpgpu_shader.h"
 #include "gpu_cmds.h"
 
+struct label_entry {
+	uint32_t id;
+	uint32_t offset;
+};
+
 #define IGA64_ARG0 0xc0ded000
 #define IGA64_ARG_MASK 0xffffff00
 
@@ -32,7 +38,7 @@ static void gpgpu_shader_extend(struct gpgpu_shader *shdr)
 	igt_assert(shdr->code);
 }
 
-void
+uint32_t
 __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
 		  int argc, uint32_t *argv)
 {
@@ -60,6 +66,8 @@ __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
 	}
 
 	shdr->size += tpls->size;
+
+	return tpls->size;
 }
 
 static uint32_t fill_sip(struct intel_bb *ibb,
@@ -235,10 +243,16 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
 	shdr->gen_ver = 100 * info->graphics_ver + info->graphics_rel;
 	shdr->max_size = 16 * 4;
 	shdr->code = malloc(4 * shdr->max_size);
+	shdr->labels = igt_map_create(igt_map_hash_32, igt_map_equal_32);
 	igt_assert(shdr->code);
 	return shdr;
 }
 
+static void free_func(struct igt_map_entry *entry)
+{
+       free(entry->data);
+}
+
 /**
  * gpgpu_shader_destroy:
  * @shdr: pointer to shader struct created with 'gpgpu_shader_create'
@@ -247,10 +261,76 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
  */
 void gpgpu_shader_destroy(struct gpgpu_shader *shdr)
 {
+	igt_map_destroy(shdr->labels, free_func);
 	free(shdr->code);
 	free(shdr);
 }
 
+/**
+ * gpgpu_shader_dump:
+ * @shdr: shader to be printed
+ *
+ * Print shader instructions from @shdr in hex.
+ */
+void gpgpu_shader_dump(struct gpgpu_shader *shdr)
+{
+	for (int i = 0; i < shdr->size / 4; i++)
+		igt_info("0x%08x 0x%08x 0x%08x 0x%08x\n",
+			 shdr->instr[i][0], shdr->instr[i][1],
+			 shdr->instr[i][2], shdr->instr[i][3]);
+}
+

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 292

Quoted 10 lines
lib/gpgpu_shader.c +10 -0
+/**
+ * gpgpu_shader__breakpoint_on:
+ * @shdr: shader to create breakpoint in
+ * @cmd_no: index of the instruction to break on
+ *
+ * Insert a breakpoint on the @cmd_no'th instruction within @shdr.
+ */
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t cmd_no)
+{
+	igt_assert(cmd_no < shdr->size / 4);

If I'm not wrong shdr->size contains number of dwords, not size in bytes. It is imo a little bit confusing and I would suggest to rename it to avoid confusion. Same with max_size.

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 304

Quoted 12 lines
lib/gpgpu_shader.c +12 -0
+	shdr->instr[cmd_no][0] |= 1<<30;
+}
+
+/**
+ * gpgpu_shader__breakpoint:
+ * @shdr: shader to create breakpoint in
+ *
+ * Insert a breakpoint on the last instruction in @shdr.
+ */
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr)
+{
+	gpgpu_shader__breakpoint_on(shdr, shdr->size / 4 - 1);

Likely shdr->size - 1 only.

Below code was commented in previous review.

--
Zbigniew

Quoted 836 lines
lib/gpgpu_shader.c +339 -0
+}
+
+/**
+ * gpgpu_shader__wait:
+ * @shdr: shader to be modified
+ *
+ * Append wait instruction to @shader. This instruction raises attention
+ * and stops execution.
+ */
+void gpgpu_shader__wait(struct gpgpu_shader *shdr)
+{
+	emit_iga64_code(shdr, sync_host, "	\n\
+(W)	sync.host	        null		\n\
+	");
+}
+
+/**
+ * gpgpu_shader__nop:
+ * @shdr: shader to be modified
+ *
+ * Append a no-op instruction to @shdr.
+ */
+void gpgpu_shader__nop(struct gpgpu_shader *shdr)
+{
+	emit_iga64_code(shdr, nop, "	\n\
+(W)	nop				\n\
+	");
+}
+
 /**
  * gpgpu_shader__eot:
  * @shdr: shader to be modified
@@ -269,6 +349,246 @@ void gpgpu_shader__eot(struct gpgpu_shader *shdr)
 	");
 }
 
+/**
+ * gpgpu_shader__label:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Create a label for the last instruction within @shdr.
+ */
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id)
+{
+	struct label_entry *l = malloc(sizeof(*l));
+
+	l->id = label_id;
+	l->offset = shdr->size;
+	igt_map_insert(shdr->labels, &l->id, l);
+}
+
+#define OPCODE(x) (x & 0x7f)
+#define OPCODE_JUMP_INDEXED 0x20
+static void __patch_indexed_jump(struct gpgpu_shader *shdr, int label_id,
+				 uint32_t jump_iga64_size)
+{
+	struct label_entry *l;
+	uint32_t *start, *end, *label;
+	int32_t relative;
+
+	l = igt_map_search(shdr->labels, &label_id);
+	igt_assert(l);
+
+	igt_assert(jump_iga64_size % 4 == 0);
+
+	label = shdr->code + l->offset;
+	end = shdr->code + shdr->size;
+	start = end - jump_iga64_size;
+
+	for (; start < end; start += 4)
+		if (OPCODE(*start) == OPCODE_JUMP_INDEXED) {
+			relative = (label - start) * 4;
+			*(start + 3) = relative;
+			break;
+		}
+}
+
+/**
+ * gpgpu_shader__jump:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label @label_id.
+ */
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id)
+{
+	size_t shader_size;
+
+	shader_size = emit_iga64_code(shdr, jump, "	\n\
+L0:							\n\
+(W)	jmpi        L0					\n\
+	");
+
+	__patch_indexed_jump(shdr, label_id, shader_size);
+}
+
+/**
+ * gpgpu_shader__jump_neq:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label @label_id
+ * when @value is not equal to dword stored at @y_offset within the surface.
+ */
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+			    uint32_t y_offset, uint32_t value)
+{
+	uint32_t size;
+
+	size = emit_iga64_code(shdr, jump_dw_neq, "					\n\
+L0:											\n\
+(W)		mov (16|M0)              r30.0<1>:ud    0x0:ud				\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+	// Y offset of the block in rows := thread group id Y				\n\
+(W)		mov (1|M0)               r30.1<1>:ud    ARG(0):ud			\n\
+	// block width [0,63] representing 1 to 64 bytes, we want dword			\n\
+(W)		mov (1|M0)               r30.2<1>:ud    0x3:ud				\n\
+	// FFTID := FFTID from R0 header						\n\
+(W)		mov (1|M0)               r30.4<1>:ud    r0.5<0;1,0>:ud  		\n\
+(W)		send.dc1 (16|M0)         r31     r30      null    0x0	0x2190000	\n\
+#else // Typed 2D Block Store								\n\
+	// Store X and Y block start (160:191 and 192:223)				\n\
+(W)            mov (2|M0)               r30.6<1>:ud    ARG(0):ud			\n\
+	// Store X and Y block size (224:231 and 232:239)				\n\
+(W)            mov (1|M0)               r30.7<1>:ud    0x3:ud				\n\
+(W)            send.tgm (16|M0)         r31     r30    null:0    0x0    0x62100003	\n\
+#endif											\n\
+	// clear the flag register							\n\
+(W)		mov (1|M0)               f0.0<1>:ud    0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud   ARG(1):ud	\n\
+(W&f0.0)	jmpi                     L0						\n\
+	", y_offset, value);
+
+	__patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__loop_begin:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Begin a counting loop in @shdr. All subsequent instructions will constitute
+ * the loop body up until 'gpgpu_shader__loop_end' gets called. The first
+ * instruction of the loop will be at label @label_id. The r40 register will be
+ * overwritten as it is used as the loop counter.
+ */
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id)
+{
+	emit_iga64_code(shdr, clear_r40, "		\n\
+L0:							\n\
+(W)	mov (1|M0)               r40:ud    0x0:ud	\n\
+	");
+
+	gpgpu_shader__label(shdr, label_id);
+}
+
+/**
+ * gpgpu_shader__loop_end:
+ * @shdr: shader to be modified
+ * @label_id: label id passed to 'gpgpu_shader__loop_begin'
+ * @iter: iteration count
+ *
+ * End loop body in @shdr.
+ */
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, uint32_t iter)
+{
+	uint32_t size;
+
+	size = emit_iga64_code(shdr, inc_r40_jump_neq, "				\n\
+L0:											\n\
+(W)		add (1|M0)              r40:ud          r40.0<0;1,0>:ud 0x1:ud		\n\
+(W)		mov (1|M0)              f0.0<1>:ud      0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r40.0<0;1,0>:ud   ARG(0):ud	\n\
+(W&f0.0)	jmpi                     L0						\n\
+	", iter);
+
+	__patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__common_target_write:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: oword to be written
+ *
+ * Write the oword stored in @value to the target buffer at @y_offset.
+ */
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+				       uint32_t y_offset, const uint32_t value[4])
+{
+	emit_iga64_code(shdr, common_target_write, "				\n\
+(W)	mov (16|M0)		r30.0<1>:ud	0x0:ud				\n\
+(W)	mov (16|M0)		r31.0<1>:ud	0x0:ud				\n\
+(W)	mov (1|M0)		r31.0<1>:ud	ARG(1):ud			\n\
+(W)	mov (1|M0)		r31.1<1>:ud	ARG(2):ud			\n\
+(W)	mov (1|M0)		r31.2<1>:ud	ARG(3):ud			\n\
+(W)	mov (1|M0)		r31.3<1>:ud	ARG(4):ud			\n\
+#if GEN_VER < 2000 // Media Block Write						\n\
+	// Y offset of the block in rows					\n\
+(W)	mov (1|M0)		r30.1<1>:ud	ARG(0):ud			\n\
+	// block width [0,63] representing 1 to 64 bytes			\n\
+(W)	mov (1|M0)		r30.2<1>:ud	0xf:ud				\n\
+	// FFTID := FFTID from R0 header					\n\
+(W)	mov (1|M0)		r30.4<1>:ud	r0.5<0;1,0>:ud			\n\
+	// written value							\n\
+(W)	send.dc1 (16|M0)	null	r30	src1_null  0x0	0x40A8000	\n\
+#else	// Typed 2D Block Store							\n\
+	// Store X and Y block start (160:191 and 192:223)			\n\
+(W)	mov (2|M0)              r30.6<1>:ud     ARG(0):ud			\n\
+	// Store X and Y block size (224:231 and 232:239)			\n\
+(W)	mov (1|M0)              r30.7<1>:ud     0xf:ud				\n\
+(W)	send.tgm (16|M0)        null    r30     null:0  0x0     0x64000007	\n\
+#endif										\n\
+	", y_offset, value[0], value[1], value[2], value[3]);
+}
+
+/**
+ * gpgpu_shader__common_target_write_u32:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: dword to be written
+ *
+ * Fill oword at @y_offset with dword stored in @value.
+ */
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+					   uint32_t y_offset, uint32_t value)
+{
+	const uint32_t owblock[4] = {
+		value, value, value, value
+	};
+	gpgpu_shader__common_target_write(shdr, y_offset, owblock);
+}
+
+/**
+ * gpgpu_shader__write_aip:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within the surface in rows
+ *
+ * Write address instruction pointer to row tg_id_y + @y_offset.
+ */
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset)
+{
+	emit_iga64_code(shdr, media_block_write_aip, "				\n\
+	// Payload								\n\
+(W)	mov (1|M0)               r5.0<1>:ud    cr0.2:ud				\n\
+#if GEN_VER < 2000 // Media Block Write						\n\
+	// X offset of the block in bytes := (thread group id X << ARG(0))	\n\
+(W)	shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud    0x2:ud		\n\
+	// Y offset of the block in rows := thread group id Y			\n\
+(W)	mov (1|M0)               r4.1<1>:ud    r0.6<0;1,0>:ud			\n\
+(W)	add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud    ARG(0):ud	\n\
+	// block width [0,63] representing 1 to 64 bytes			\n\
+(W)	mov (1|M0)               r4.2<1>:ud    0x3:ud				\n\
+	// FFTID := FFTID from R0 header					\n\
+(W)	mov (1|M0)               r4.4<1>:ud    r0.5<0;1,0>:ud			\n\
+(W)	send.dc1 (16|M0)         null     r4   src1_null 0       0x40A8000	\n\
+#else // Typed 2D Block Store							\n\
+	// Load r2.0-3 with tg id X << ARG(0)					\n\
+(W)	shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud    0x2:ud		\n\
+	// Load r2.4-7 with tg id Y + ARG(1):ud					\n\
+(W)	mov (1|M0)               r2.1<1>:ud    r0.6<0;1,0>:ud			\n\
+(W)	add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud    ARG(0):ud	\n\
+	// payload setup							\n\
+(W)	mov (16|M0)              r4.0<1>:ud    0x0:ud				\n\
+	// Store X and Y block start (160:191 and 192:223)			\n\
+(W)	mov (2|M0)               r4.5<1>:ud    r2.0<2;2,1>:ud			\n\
+	// Store X and Y block max_size (224:231 and 232:239)			\n\
+(W)	mov (1|M0)               r4.7<1>:ud    0x3:ud				\n\
+(W)	send.tgm (16|M0)         null     r4   null:0    0    0x64000007	\n\
+#endif										\n\
+	", y_offset);
+}
+
 /**
  * gpgpu_shader__write_dword:
  * @shdr: shader to be modified
@@ -313,3 +633,73 @@ void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 #endif										\n\
 	", 2, y_offset, 3, value, value, value, value);
 }
+
+/**
+ * gpgpu_shader__end_system_routine:
+ * @shdr: shader to be modified
+ * @breakpoint_suppress: breakpoint suppress flag
+ *
+ * Return from system routine. To prevent infinite jumping to the system
+ * routine on a breakpoint, @breakpoint_suppress flag has to be set.
+ */
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+				      bool breakpoint_suppress)
+{
+	/*
+	 * set breakpoint suppress bit to avoid an endless loop
+	 * when sip was invoked by a breakpoint
+	 */
+	if (breakpoint_suppress)
+		emit_iga64_code(shdr, breakpoint_suppress, "			\n\
+(W)	or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x8000:ud	\n\
+		");
+
+	emit_iga64_code(shdr, end_system_routine, "				\n\
+(W)	and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   ARG(0):ud	\n\
+	// return to an application						\n\
+(W)	and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x7FFFFFFD:ud	\n\
+	", 0x7fffff | (1 << 26)); /* clear all exceptions, except read only bit */
+}
+
+/**
+ * gpgpu_shader__end_system_routine_step_if_eq:
+ * @shdr: shader to be modified
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value for single stepping execution
+ *
+ * Return from system routine. Don't clear breakpoint exception when @value
+ * is equal to value stored at @y_offset. This triggers the system routine
+ * after the subsequent instruction, resulting in single stepping execution.
+ */
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+						 uint32_t y_offset,
+						 uint32_t value)
+{
+	emit_iga64_code(shdr, end_system_routine_step_if_eq, "				\n\
+(W)		or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x8000:ud	\n\
+(W)		and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   ARG(0):ud	\n\
+(W)		mov (16|M0)              r30.0<1>:ud    0x0:ud				\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+		// Y offset of the block in rows := thread group id Y			\n\
+(W)		mov (1|M0)               r30.1<1>:ud    ARG(1):ud			\n\
+		// block width [0,63] representing 1 to 64 bytes, we want dword		\n\
+(W)		mov (1|M0)               r30.2<1>:ud    0x3:ud				\n\
+		// FFTID := FFTID from R0 header					\n\
+(W)		mov (1|M0)               r30.4<1>:ud    r0.5<0;1,0>:ud			\n\
+(W)		send.dc1 (16|M0)         r31     r30      null    0x0	0x2190000	\n\
+#else	// Typed 2D Block Store								\n\
+		// Store X and Y block start (160:191 and 192:223)			\n\
+(W)		mov (2|M0)               r30.6<1>:ud    ARG(1):ud			\n\
+		// Store X and Y block size (224:231 and 232:239)			\n\
+(W)		mov (1|M0)               r30.7<1>:ud    0x3:ud				\n\
+(W)		send.tgm (16|M0)         r31     r30    null:0    0x0    0x62100003	\n\
+#endif											\n\
+		// clear the flag register						\n\
+(W)		mov (1|M0)               f0.0<1>:ud    0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud   ARG(2):ud	\n\
+(W&f0.0)	and (1|M0)              cr0.1<1>:ud     cr0.1<0;1,0>:ud   ARG(3):ud	\n\
+		// return to an application						\n\
+(W)		and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x7FFFFFFD:ud	\n\
+	", 0x807fffff, /* leave breakpoint exception */
+	y_offset, value, 0x7fffff /* clear all exceptions */ );
+}
lib/gpgpu_shader.h +22 -2
index 255f93b4d..e4ca0be4c 100644
@@ -21,6 +21,7 @@ struct gpgpu_shader {
 		uint32_t *code;
 		uint32_t (*instr)[4];
 	};
+	struct igt_map *labels;
 };
 
 struct iga64_template {
@@ -31,7 +32,7 @@ struct iga64_template {
 
 #pragma GCC diagnostic ignored "-Wnested-externs"
 
-void
+uint32_t
 __emit_iga64_code(struct gpgpu_shader *shdr, const struct iga64_template *tpls,
 		  int argc, uint32_t *argv);
 
@@ -56,8 +57,27 @@ void gpgpu_shader_exec(struct intel_bb *ibb,
 		       struct gpgpu_shader *sip,
 		       uint64_t ring, bool explicit_engine);
 
+void gpgpu_shader__wait(struct gpgpu_shader *shdr);
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t cmd_no);
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr);
+void gpgpu_shader__nop(struct gpgpu_shader *shdr);
 void gpgpu_shader__eot(struct gpgpu_shader *shdr);
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+				       uint32_t y_offset, const uint32_t value[4]);
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+				     uint32_t y_offset, uint32_t value);
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+				      bool breakpoint_suppress);
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+						 uint32_t dw_offset,
+						 uint32_t value);
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset);
 void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 			       uint32_t y_offset);
-
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+			    uint32_t dw_offset, uint32_t value);
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, uint32_t iter);
 #endif /* GPGPU_SHADER_H */
lib/iga64_generated_codes.c +400 -1
index ea8d0f097..dd849eebc 100644
@@ -3,7 +3,7 @@
 
 #include "gpgpu_shader.h"
 
-#define MD5_SUM_IGA64_ASMS 9977ade854d57c5af5c5ca9e93c0f37e
+#define MD5_SUM_IGA64_ASMS 33b7cd843e3b009c123a85a6c520d7d0
 
 struct iga64_template const iga64_code_gpgpu_fill[] = {
 	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
@@ -79,6 +79,119 @@ struct iga64_template const iga64_code_gpgpu_fill[] = {
 	}}
 };
 
+struct iga64_template const iga64_code_end_system_routine_step_if_eq[] = {
+	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+		0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 52, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 48, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 52, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 48, .code = (const uint32_t []) {
+		0x80000166, 0x80018220, 0x02008000, 0x00008000,
+		0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040161, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000165, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_end_system_routine[] = {
+	{ .gen_ver = 2000, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 12, .code = (const uint32_t []) {
+		0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_breakpoint_suppress[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000166, 0x80018220, 0x02008000, 0x00008000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
 struct iga64_template const iga64_code_media_block_write[] = {
 	{ .gen_ver = 2000, .size = 56, .code = (const uint32_t []) {
 		0x80100061, 0x04054220, 0x00000000, 0x00000000,
@@ -164,6 +277,270 @@ struct iga64_template const iga64_code_media_block_write[] = {
 	}}
 };
 
+struct iga64_template const iga64_code_media_block_write_aip[] = {
+	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008020, 0x00000000,
+		0x80000969, 0x02058220, 0x02000014, 0x00000002,
+		0x80000061, 0x02150220, 0x00000064, 0x00000000,
+		0x80001940, 0x02158220, 0x02000214, 0xc0ded000,
+		0x80100061, 0x04054220, 0x00000000, 0x00000000,
+		0x80041a61, 0x04550220, 0x00220205, 0x00000000,
+		0x80000061, 0x04754220, 0x00000000, 0x00000003,
+		0x80132031, 0x00000000, 0xd00e0494, 0x04000000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008040, 0x00000000,
+		0x80000969, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 40, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008020, 0x00000000,
+		0x80000969, 0x04058220, 0x02000014, 0x00000002,
+		0x80000061, 0x04150220, 0x00000064, 0x00000000,
+		0x80001940, 0x04158220, 0x02000414, 0xc0ded000,
+		0x80000061, 0x04254220, 0x00000000, 0x00000003,
+		0x80000061, 0x04450220, 0x00000054, 0x00000000,
+		0x80132031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008040, 0x00000000,
+		0x80000969, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 40, .code = (const uint32_t []) {
+		0x80000161, 0x05050220, 0x00008040, 0x00000000,
+		0x80000169, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80000140, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_common_target_write[] = {
+	{ .gen_ver = 2000, .size = 48, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e754220, 0x00000000, 0x0000000f,
+		0x80132031, 0x00000000, 0xd00e1e94, 0x04000000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 56, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 52, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e254220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 56, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 52, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_inc_r40_jump_neq[] = {
+	{ .gen_ver = 2000, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80000270, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_clear_r40[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000061, 0x28054220, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000061, 0x28054220, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_jump_dw_neq[] = {
+	{ .gen_ver = 2000, .size = 32, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+		0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffa0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 40, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 36, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x84000020, 0x00004000, 0x00000000, 0xffffff90,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 40, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 36, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff90,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_jump[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000020, 0x00004000, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000020, 0x00004000, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
 struct iga64_template const iga64_code_eot[] = {
 	{ .gen_ver = 2000, .size = 8, .code = (const uint32_t []) {
 		0x800c0061, 0x70050220, 0x00460005, 0x00000000,
@@ -188,3 +565,25 @@ struct iga64_template const iga64_code_eot[] = {
 		0x80049031, 0x00000004, 0x7020700c, 0x10000000,
 	}}
 };
+
+struct iga64_template const iga64_code_nop[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x00000060, 0x00000000, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x00000060, 0x00000000, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_sync_host[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
Signature · 1 line
2.34.1
#5 lore raw

Re: [PATCH i-g-t v5 03/17] lib/gpgpu_shader: Extend shader building library

Hi Zbigniew,

Quoted 99 lines · Zbigniew Kempczyński
Quoted 98 lines · Christoph Manszewski

Add shader building functions and iga64 code used by eudebug subtests.

---
  lib/gpgpu_shader.c          | 392 ++++++++++++++++++++++++++++++++++-
  lib/gpgpu_shader.h          |  24 ++-
  lib/iga64_generated_codes.c | 401 +++++++++++++++++++++++++++++++++++-
  3 files changed, 813 insertions(+), 4 deletions(-)
lib/gpgpu_shader.c +36 -1
index 80bad342a..c723577e6 100644
@@ -7,10 +7,16 @@
 
  #include <i915_drm.h>
 
+#include "igt_map.h"
  #include "ioctl_wrappers.h"
  #include "gpgpu_shader.h"
  #include "gpu_cmds.h"
 
+struct label_entry {
+	uint32_t id;
+	uint32_t offset;
+};
+
  #define IGA64_ARG0 0xc0ded000
  #define IGA64_ARG_MASK 0xffffff00
 
@@ -32,7 +38,7 @@ static void gpgpu_shader_extend(struct gpgpu_shader *shdr)
  	igt_assert(shdr->code);
  }
 
-void
+uint32_t
  __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
  		  int argc, uint32_t *argv)
  {
@@ -60,6 +66,8 @@ __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
  	}
 
  	shdr->size += tpls->size;
+
+	return tpls->size;
  }
 
  static uint32_t fill_sip(struct intel_bb *ibb,
@@ -235,10 +243,16 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
  	shdr->gen_ver = 100 * info->graphics_ver + info->graphics_rel;
  	shdr->max_size = 16 * 4;
  	shdr->code = malloc(4 * shdr->max_size);
+	shdr->labels = igt_map_create(igt_map_hash_32, igt_map_equal_32);
  	igt_assert(shdr->code);
  	return shdr;
  }
 
+static void free_func(struct igt_map_entry *entry)
+{
+       free(entry->data);
+}
+
  /**
   * gpgpu_shader_destroy:
   * @shdr: pointer to shader struct created with 'gpgpu_shader_create'
@@ -247,10 +261,76 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
   */
  void gpgpu_shader_destroy(struct gpgpu_shader *shdr)
  {
+	igt_map_destroy(shdr->labels, free_func);
  	free(shdr->code);
  	free(shdr);
  }
 
+/**
+ * gpgpu_shader_dump:
+ * @shdr: shader to be printed
+ *
+ * Print shader instructions from @shdr in hex.
+ */
+void gpgpu_shader_dump(struct gpgpu_shader *shdr)
+{
+	for (int i = 0; i < shdr->size / 4; i++)
+		igt_info("0x%08x 0x%08x 0x%08x 0x%08x\n",
+			 shdr->instr[i][0], shdr->instr[i][1],
+			 shdr->instr[i][2], shdr->instr[i][3]);
+}
+
+/**
+ * gpgpu_shader__breakpoint_on:
+ * @shdr: shader to create breakpoint in
+ * @cmd_no: index of the instruction to break on
+ *
+ * Insert a breakpoint on the @cmd_no'th instruction within @shdr.
Quoted 8 lines

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 292

Quoted 4 lines
lib/gpgpu_shader.c +4 -0
+ */
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t cmd_no)
+{
+	igt_assert(cmd_no < shdr->size / 4);

If I'm not wrong shdr->size contains number of dwords, not size in bytes. It is imo a little bit confusing and I would suggest to rename it to avoid confusion. Same with max_size.

It does, and I agree that changing it to something like 'dw_size' would get rid of the confusion, since by default I would also assume that a naked 'size' means bytes. However instructions are 4 dwords, so the division by 4 is relevant here.

Quoted 15 lines

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 304

Quoted 13 lines
lib/gpgpu_shader.c +12 -0
+	shdr->instr[cmd_no][0] |= 1<<30;
+}
+
+/**
+ * gpgpu_shader__breakpoint:
+ * @shdr: shader to create breakpoint in
+ *
+ * Insert a breakpoint on the last instruction in @shdr.
+ */
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr)
+{
+	gpgpu_shader__breakpoint_on(shdr, shdr->size / 4 - 1);

Likely shdr->size - 1 only.

Like above - instructions are 4 dwords.

Thanks,
Christoph

Quoted 842 lines

Below code was commented in previous review.

--
Zbigniew

Quoted 836 lines
lib/gpgpu_shader.c +339 -0
+}
+
+/**
+ * gpgpu_shader__wait:
+ * @shdr: shader to be modified
+ *
+ * Append wait instruction to @shader. This instruction raises attention
+ * and stops execution.
+ */
+void gpgpu_shader__wait(struct gpgpu_shader *shdr)
+{
+	emit_iga64_code(shdr, sync_host, "	\n\
+(W)	sync.host	        null		\n\
+	");
+}
+
+/**
+ * gpgpu_shader__nop:
+ * @shdr: shader to be modified
+ *
+ * Append a no-op instruction to @shdr.
+ */
+void gpgpu_shader__nop(struct gpgpu_shader *shdr)
+{
+	emit_iga64_code(shdr, nop, "	\n\
+(W)	nop				\n\
+	");
+}
+
  /**
   * gpgpu_shader__eot:
   * @shdr: shader to be modified
@@ -269,6 +349,246 @@ void gpgpu_shader__eot(struct gpgpu_shader *shdr)
  	");
  }
 
+/**
+ * gpgpu_shader__label:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Create a label for the last instruction within @shdr.
+ */
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id)
+{
+	struct label_entry *l = malloc(sizeof(*l));
+
+	l->id = label_id;
+	l->offset = shdr->size;
+	igt_map_insert(shdr->labels, &l->id, l);
+}
+
+#define OPCODE(x) (x & 0x7f)
+#define OPCODE_JUMP_INDEXED 0x20
+static void __patch_indexed_jump(struct gpgpu_shader *shdr, int label_id,
+				 uint32_t jump_iga64_size)
+{
+	struct label_entry *l;
+	uint32_t *start, *end, *label;
+	int32_t relative;
+
+	l = igt_map_search(shdr->labels, &label_id);
+	igt_assert(l);
+
+	igt_assert(jump_iga64_size % 4 == 0);
+
+	label = shdr->code + l->offset;
+	end = shdr->code + shdr->size;
+	start = end - jump_iga64_size;
+
+	for (; start < end; start += 4)
+		if (OPCODE(*start) == OPCODE_JUMP_INDEXED) {
+			relative = (label - start) * 4;
+			*(start + 3) = relative;
+			break;
+		}
+}
+
+/**
+ * gpgpu_shader__jump:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label @label_id.
+ */
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id)
+{
+	size_t shader_size;
+
+	shader_size = emit_iga64_code(shdr, jump, "	\n\
+L0:							\n\
+(W)	jmpi        L0					\n\
+	");
+
+	__patch_indexed_jump(shdr, label_id, shader_size);
+}
+
+/**
+ * gpgpu_shader__jump_neq:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label @label_id
+ * when @value is not equal to dword stored at @y_offset within the surface.
+ */
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+			    uint32_t y_offset, uint32_t value)
+{
+	uint32_t size;
+
+	size = emit_iga64_code(shdr, jump_dw_neq, "					\n\
+L0:											\n\
+(W)		mov (16|M0)              r30.0<1>:ud    0x0:ud				\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+	// Y offset of the block in rows := thread group id Y				\n\
+(W)		mov (1|M0)               r30.1<1>:ud    ARG(0):ud			\n\
+	// block width [0,63] representing 1 to 64 bytes, we want dword			\n\
+(W)		mov (1|M0)               r30.2<1>:ud    0x3:ud				\n\
+	// FFTID := FFTID from R0 header						\n\
+(W)		mov (1|M0)               r30.4<1>:ud    r0.5<0;1,0>:ud  		\n\
+(W)		send.dc1 (16|M0)         r31     r30      null    0x0	0x2190000	\n\
+#else // Typed 2D Block Store								\n\
+	// Store X and Y block start (160:191 and 192:223)				\n\
+(W)            mov (2|M0)               r30.6<1>:ud    ARG(0):ud			\n\
+	// Store X and Y block size (224:231 and 232:239)				\n\
+(W)            mov (1|M0)               r30.7<1>:ud    0x3:ud				\n\
+(W)            send.tgm (16|M0)         r31     r30    null:0    0x0    0x62100003	\n\
+#endif											\n\
+	// clear the flag register							\n\
+(W)		mov (1|M0)               f0.0<1>:ud    0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud   ARG(1):ud	\n\
+(W&f0.0)	jmpi                     L0						\n\
+	", y_offset, value);
+
+	__patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__loop_begin:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Begin a counting loop in @shdr. All subsequent instructions will constitute
+ * the loop body up until 'gpgpu_shader__loop_end' gets called. The first
+ * instruction of the loop will be at label @label_id. The r40 register will be
+ * overwritten as it is used as the loop counter.
+ */
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id)
+{
+	emit_iga64_code(shdr, clear_r40, "		\n\
+L0:							\n\
+(W)	mov (1|M0)               r40:ud    0x0:ud	\n\
+	");
+
+	gpgpu_shader__label(shdr, label_id);
+}
+
+/**
+ * gpgpu_shader__loop_end:
+ * @shdr: shader to be modified
+ * @label_id: label id passed to 'gpgpu_shader__loop_begin'
+ * @iter: iteration count
+ *
+ * End loop body in @shdr.
+ */
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, uint32_t iter)
+{
+	uint32_t size;
+
+	size = emit_iga64_code(shdr, inc_r40_jump_neq, "				\n\
+L0:											\n\
+(W)		add (1|M0)              r40:ud          r40.0<0;1,0>:ud 0x1:ud		\n\
+(W)		mov (1|M0)              f0.0<1>:ud      0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r40.0<0;1,0>:ud   ARG(0):ud	\n\
+(W&f0.0)	jmpi                     L0						\n\
+	", iter);
+
+	__patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__common_target_write:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: oword to be written
+ *
+ * Write the oword stored in @value to the target buffer at @y_offset.
+ */
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+				       uint32_t y_offset, const uint32_t value[4])
+{
+	emit_iga64_code(shdr, common_target_write, "				\n\
+(W)	mov (16|M0)		r30.0<1>:ud	0x0:ud				\n\
+(W)	mov (16|M0)		r31.0<1>:ud	0x0:ud				\n\
+(W)	mov (1|M0)		r31.0<1>:ud	ARG(1):ud			\n\
+(W)	mov (1|M0)		r31.1<1>:ud	ARG(2):ud			\n\
+(W)	mov (1|M0)		r31.2<1>:ud	ARG(3):ud			\n\
+(W)	mov (1|M0)		r31.3<1>:ud	ARG(4):ud			\n\
+#if GEN_VER < 2000 // Media Block Write						\n\
+	// Y offset of the block in rows					\n\
+(W)	mov (1|M0)		r30.1<1>:ud	ARG(0):ud			\n\
+	// block width [0,63] representing 1 to 64 bytes			\n\
+(W)	mov (1|M0)		r30.2<1>:ud	0xf:ud				\n\
+	// FFTID := FFTID from R0 header					\n\
+(W)	mov (1|M0)		r30.4<1>:ud	r0.5<0;1,0>:ud			\n\
+	// written value							\n\
+(W)	send.dc1 (16|M0)	null	r30	src1_null  0x0	0x40A8000	\n\
+#else	// Typed 2D Block Store							\n\
+	// Store X and Y block start (160:191 and 192:223)			\n\
+(W)	mov (2|M0)              r30.6<1>:ud     ARG(0):ud			\n\
+	// Store X and Y block size (224:231 and 232:239)			\n\
+(W)	mov (1|M0)              r30.7<1>:ud     0xf:ud				\n\
+(W)	send.tgm (16|M0)        null    r30     null:0  0x0     0x64000007	\n\
+#endif										\n\
+	", y_offset, value[0], value[1], value[2], value[3]);
+}
+
+/**
+ * gpgpu_shader__common_target_write_u32:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: dword to be written
+ *
+ * Fill oword at @y_offset with dword stored in @value.
+ */
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+					   uint32_t y_offset, uint32_t value)
+{
+	const uint32_t owblock[4] = {
+		value, value, value, value
+	};
+	gpgpu_shader__common_target_write(shdr, y_offset, owblock);
+}
+
+/**
+ * gpgpu_shader__write_aip:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within the surface in rows
+ *
+ * Write address instruction pointer to row tg_id_y + @y_offset.
+ */
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset)
+{
+	emit_iga64_code(shdr, media_block_write_aip, "				\n\
+	// Payload								\n\
+(W)	mov (1|M0)               r5.0<1>:ud    cr0.2:ud				\n\
+#if GEN_VER < 2000 // Media Block Write						\n\
+	// X offset of the block in bytes := (thread group id X << ARG(0))	\n\
+(W)	shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud    0x2:ud		\n\
+	// Y offset of the block in rows := thread group id Y			\n\
+(W)	mov (1|M0)               r4.1<1>:ud    r0.6<0;1,0>:ud			\n\
+(W)	add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud    ARG(0):ud	\n\
+	// block width [0,63] representing 1 to 64 bytes			\n\
+(W)	mov (1|M0)               r4.2<1>:ud    0x3:ud				\n\
+	// FFTID := FFTID from R0 header					\n\
+(W)	mov (1|M0)               r4.4<1>:ud    r0.5<0;1,0>:ud			\n\
+(W)	send.dc1 (16|M0)         null     r4   src1_null 0       0x40A8000	\n\
+#else // Typed 2D Block Store							\n\
+	// Load r2.0-3 with tg id X << ARG(0)					\n\
+(W)	shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud    0x2:ud		\n\
+	// Load r2.4-7 with tg id Y + ARG(1):ud					\n\
+(W)	mov (1|M0)               r2.1<1>:ud    r0.6<0;1,0>:ud			\n\
+(W)	add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud    ARG(0):ud	\n\
+	// payload setup							\n\
+(W)	mov (16|M0)              r4.0<1>:ud    0x0:ud				\n\
+	// Store X and Y block start (160:191 and 192:223)			\n\
+(W)	mov (2|M0)               r4.5<1>:ud    r2.0<2;2,1>:ud			\n\
+	// Store X and Y block max_size (224:231 and 232:239)			\n\
+(W)	mov (1|M0)               r4.7<1>:ud    0x3:ud				\n\
+(W)	send.tgm (16|M0)         null     r4   null:0    0    0x64000007	\n\
+#endif										\n\
+	", y_offset);
+}
+
  /**
   * gpgpu_shader__write_dword:
   * @shdr: shader to be modified
@@ -313,3 +633,73 @@ void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
  #endif										\n\
  	", 2, y_offset, 3, value, value, value, value);
  }
+
+/**
+ * gpgpu_shader__end_system_routine:
+ * @shdr: shader to be modified
+ * @breakpoint_suppress: breakpoint suppress flag
+ *
+ * Return from system routine. To prevent infinite jumping to the system
+ * routine on a breakpoint, @breakpoint_suppress flag has to be set.
+ */
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+				      bool breakpoint_suppress)
+{
+	/*
+	 * set breakpoint suppress bit to avoid an endless loop
+	 * when sip was invoked by a breakpoint
+	 */
+	if (breakpoint_suppress)
+		emit_iga64_code(shdr, breakpoint_suppress, "			\n\
+(W)	or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x8000:ud	\n\
+		");
+
+	emit_iga64_code(shdr, end_system_routine, "				\n\
+(W)	and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   ARG(0):ud	\n\
+	// return to an application						\n\
+(W)	and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x7FFFFFFD:ud	\n\
+	", 0x7fffff | (1 << 26)); /* clear all exceptions, except read only bit */
+}
+
+/**
+ * gpgpu_shader__end_system_routine_step_if_eq:
+ * @shdr: shader to be modified
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value for single stepping execution
+ *
+ * Return from system routine. Don't clear breakpoint exception when @value
+ * is equal to value stored at @y_offset. This triggers the system routine
+ * after the subsequent instruction, resulting in single stepping execution.
+ */
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+						 uint32_t y_offset,
+						 uint32_t value)
+{
+	emit_iga64_code(shdr, end_system_routine_step_if_eq, "				\n\
+(W)		or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x8000:ud	\n\
+(W)		and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   ARG(0):ud	\n\
+(W)		mov (16|M0)              r30.0<1>:ud    0x0:ud				\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+		// Y offset of the block in rows := thread group id Y			\n\
+(W)		mov (1|M0)               r30.1<1>:ud    ARG(1):ud			\n\
+		// block width [0,63] representing 1 to 64 bytes, we want dword		\n\
+(W)		mov (1|M0)               r30.2<1>:ud    0x3:ud				\n\
+		// FFTID := FFTID from R0 header					\n\
+(W)		mov (1|M0)               r30.4<1>:ud    r0.5<0;1,0>:ud			\n\
+(W)		send.dc1 (16|M0)         r31     r30      null    0x0	0x2190000	\n\
+#else	// Typed 2D Block Store								\n\
+		// Store X and Y block start (160:191 and 192:223)			\n\
+(W)		mov (2|M0)               r30.6<1>:ud    ARG(1):ud			\n\
+		// Store X and Y block size (224:231 and 232:239)			\n\
+(W)		mov (1|M0)               r30.7<1>:ud    0x3:ud				\n\
+(W)		send.tgm (16|M0)         r31     r30    null:0    0x0    0x62100003	\n\
+#endif											\n\
+		// clear the flag register						\n\
+(W)		mov (1|M0)               f0.0<1>:ud    0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud   ARG(2):ud	\n\
+(W&f0.0)	and (1|M0)              cr0.1<1>:ud     cr0.1<0;1,0>:ud   ARG(3):ud	\n\
+		// return to an application						\n\
+(W)		and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x7FFFFFFD:ud	\n\
+	", 0x807fffff, /* leave breakpoint exception */
+	y_offset, value, 0x7fffff /* clear all exceptions */ );
+}
lib/gpgpu_shader.h +22 -2
index 255f93b4d..e4ca0be4c 100644
@@ -21,6 +21,7 @@ struct gpgpu_shader {
  		uint32_t *code;
  		uint32_t (*instr)[4];
  	};
+	struct igt_map *labels;
  };
 
  struct iga64_template {
@@ -31,7 +32,7 @@ struct iga64_template {
 
  #pragma GCC diagnostic ignored "-Wnested-externs"
 
-void
+uint32_t
  __emit_iga64_code(struct gpgpu_shader *shdr, const struct iga64_template *tpls,
  		  int argc, uint32_t *argv);
 
@@ -56,8 +57,27 @@ void gpgpu_shader_exec(struct intel_bb *ibb,
  		       struct gpgpu_shader *sip,
  		       uint64_t ring, bool explicit_engine);
 
+void gpgpu_shader__wait(struct gpgpu_shader *shdr);
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t cmd_no);
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr);
+void gpgpu_shader__nop(struct gpgpu_shader *shdr);
  void gpgpu_shader__eot(struct gpgpu_shader *shdr);
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+				       uint32_t y_offset, const uint32_t value[4]);
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+				     uint32_t y_offset, uint32_t value);
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+				      bool breakpoint_suppress);
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+						 uint32_t dw_offset,
+						 uint32_t value);
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset);
  void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
  			       uint32_t y_offset);
-
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+			    uint32_t dw_offset, uint32_t value);
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, uint32_t iter);
  #endif /* GPGPU_SHADER_H */
lib/iga64_generated_codes.c +400 -1
index ea8d0f097..dd849eebc 100644
@@ -3,7 +3,7 @@
 
  #include "gpgpu_shader.h"
 
-#define MD5_SUM_IGA64_ASMS 9977ade854d57c5af5c5ca9e93c0f37e
+#define MD5_SUM_IGA64_ASMS 33b7cd843e3b009c123a85a6c520d7d0
 
  struct iga64_template const iga64_code_gpgpu_fill[] = {
  	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
@@ -79,6 +79,119 @@ struct iga64_template const iga64_code_gpgpu_fill[] = {
  	}}
  };
 
+struct iga64_template const iga64_code_end_system_routine_step_if_eq[] = {
+	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+		0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 52, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 48, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 52, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 48, .code = (const uint32_t []) {
+		0x80000166, 0x80018220, 0x02008000, 0x00008000,
+		0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040161, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000165, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_end_system_routine[] = {
+	{ .gen_ver = 2000, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 12, .code = (const uint32_t []) {
+		0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_breakpoint_suppress[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000166, 0x80018220, 0x02008000, 0x00008000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
  struct iga64_template const iga64_code_media_block_write[] = {
  	{ .gen_ver = 2000, .size = 56, .code = (const uint32_t []) {
  		0x80100061, 0x04054220, 0x00000000, 0x00000000,
@@ -164,6 +277,270 @@ struct iga64_template const iga64_code_media_block_write[] = {
  	}}
  };
 
+struct iga64_template const iga64_code_media_block_write_aip[] = {
+	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008020, 0x00000000,
+		0x80000969, 0x02058220, 0x02000014, 0x00000002,
+		0x80000061, 0x02150220, 0x00000064, 0x00000000,
+		0x80001940, 0x02158220, 0x02000214, 0xc0ded000,
+		0x80100061, 0x04054220, 0x00000000, 0x00000000,
+		0x80041a61, 0x04550220, 0x00220205, 0x00000000,
+		0x80000061, 0x04754220, 0x00000000, 0x00000003,
+		0x80132031, 0x00000000, 0xd00e0494, 0x04000000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008040, 0x00000000,
+		0x80000969, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 40, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008020, 0x00000000,
+		0x80000969, 0x04058220, 0x02000014, 0x00000002,
+		0x80000061, 0x04150220, 0x00000064, 0x00000000,
+		0x80001940, 0x04158220, 0x02000414, 0xc0ded000,
+		0x80000061, 0x04254220, 0x00000000, 0x00000003,
+		0x80000061, 0x04450220, 0x00000054, 0x00000000,
+		0x80132031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008040, 0x00000000,
+		0x80000969, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 40, .code = (const uint32_t []) {
+		0x80000161, 0x05050220, 0x00008040, 0x00000000,
+		0x80000169, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80000140, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_common_target_write[] = {
+	{ .gen_ver = 2000, .size = 48, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e754220, 0x00000000, 0x0000000f,
+		0x80132031, 0x00000000, 0xd00e1e94, 0x04000000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 56, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 52, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e254220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 56, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 52, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_inc_r40_jump_neq[] = {
+	{ .gen_ver = 2000, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80000270, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_clear_r40[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000061, 0x28054220, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000061, 0x28054220, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_jump_dw_neq[] = {
+	{ .gen_ver = 2000, .size = 32, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+		0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffa0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 40, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 36, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x84000020, 0x00004000, 0x00000000, 0xffffff90,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 40, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 36, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff90,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_jump[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000020, 0x00004000, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000020, 0x00004000, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
  struct iga64_template const iga64_code_eot[] = {
  	{ .gen_ver = 2000, .size = 8, .code = (const uint32_t []) {
  		0x800c0061, 0x70050220, 0x00460005, 0x00000000,
@@ -188,3 +565,25 @@ struct iga64_template const iga64_code_eot[] = {
  		0x80049031, 0x00000004, 0x7020700c, 0x10000000,
  	}}
  };
+
+struct iga64_template const iga64_code_nop[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x00000060, 0x00000000, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x00000060, 0x00000000, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_sync_host[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
Signature · 1 line
2.34.1
#6 lore raw

Re: [PATCH i-g-t v5 03/17] lib/gpgpu_shader: Extend shader building library

Hi Zbigniew,

Quoted 105 lines · Christoph

Hi Zbigniew,

Quoted 93 lines · Zbigniew Kempczyński
Quoted 92 lines · Christoph Manszewski

Add shader building functions and iga64 code used by eudebug subtests.

---
  lib/gpgpu_shader.c          | 392 ++++++++++++++++++++++++++++++++++- lib/gpgpu_shader.h          |  24 ++- lib/iga64_generated_codes.c | 401 +++++++++++++++++++++++++++++++++++-

  3 files changed, 813 insertions(+), 4 deletions(-)
lib/gpgpu_shader.c +37 -1
index 80bad342a..c723577e6 100644
@@ -7,10 +7,16 @@
   #include <i915_drm.h>
+#include "igt_map.h"
   #include "ioctl_wrappers.h"
   #include "gpgpu_shader.h"
   #include "gpu_cmds.h"
+struct label_entry {
+    uint32_t id;
+    uint32_t offset;
+};
+
   #define IGA64_ARG0 0xc0ded000
   #define IGA64_ARG_MASK 0xffffff00
@@ -32,7 +38,7 @@ static void gpgpu_shader_extend(struct gpgpu_shader 
 *shdr)
       igt_assert(shdr->code);
   }
-void
+uint32_t
   __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
             int argc, uint32_t *argv)
   {
@@ -60,6 +66,8 @@ __emit_iga64_code(struct gpgpu_shader *shdr, struct 
 iga64_template const *tpls,
       }
       shdr->size += tpls->size;
+
+    return tpls->size;
   }
   static uint32_t fill_sip(struct intel_bb *ibb,
@@ -235,10 +243,16 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
       shdr->gen_ver = 100 * info->graphics_ver + info->graphics_rel;
       shdr->max_size = 16 * 4;
       shdr->code = malloc(4 * shdr->max_size);
+    shdr->labels = igt_map_create(igt_map_hash_32, igt_map_equal_32);
       igt_assert(shdr->code);
       return shdr;
   }
+static void free_func(struct igt_map_entry *entry)
+{
+       free(entry->data);
+}
+
   /**
    * gpgpu_shader_destroy:
    * @shdr: pointer to shader struct created with 'gpgpu_shader_create'
@@ -247,10 +261,76 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
    */
   void gpgpu_shader_destroy(struct gpgpu_shader *shdr)
   {
+    igt_map_destroy(shdr->labels, free_func);
       free(shdr->code);
       free(shdr);
   }
+/**
+ * gpgpu_shader_dump:
+ * @shdr: shader to be printed
+ *
+ * Print shader instructions from @shdr in hex.
+ */
+void gpgpu_shader_dump(struct gpgpu_shader *shdr)
+{
+    for (int i = 0; i < shdr->size / 4; i++)
+        igt_info("0x%08x 0x%08x 0x%08x 0x%08x\n",
+             shdr->instr[i][0], shdr->instr[i][1],
+             shdr->instr[i][2], shdr->instr[i][3]);
+}
+
+/**
+ * gpgpu_shader__breakpoint_on:
+ * @shdr: shader to create breakpoint in
+ * @cmd_no: index of the instruction to break on
+ *
+ * Insert a breakpoint on the @cmd_no'th instruction within @shdr.
+ */
Quoted 8 lines

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 292

Quoted 4 lines
lib/gpgpu_shader.c +3 -0
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t 
 cmd_no)
+{
+    igt_assert(cmd_no < shdr->size / 4);

If I'm not wrong shdr->size contains number of dwords, not size in bytes. It is imo a little bit confusing and I would suggest to rename it to avoid confusion. Same with max_size.

Quoted 1 line

It does, and I agree that changing it to something like 'dw_size' would get rid of the confusion, since by default I would also assume that a naked 'size' means bytes. However instructions are 4 dwords, so the division by 4 is relevant here.

Oh I realized that this series doesn't actually introduce this field. In that case I think we can change it in a separate patch as it is unrealated to this series.

Thanks,
Christoph

Quoted 935 lines
Quoted 17 lines
Quoted 5 lines
lib/gpgpu_shader.c +2 -0
+    shdr->instr[cmd_no][0] |= 1<<30;
+}

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 304

Quoted 10 lines
lib/gpgpu_shader.c +10 -0
+
+/**
+ * gpgpu_shader__breakpoint:
+ * @shdr: shader to create breakpoint in
+ *
+ * Insert a breakpoint on the last instruction in @shdr.
+ */
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr)
+{
+    gpgpu_shader__breakpoint_on(shdr, shdr->size / 4 - 1);

Likely shdr->size - 1 only.

Like above - instructions are 4 dwords.

Thanks,
Christoph

Quoted 911 lines

Below code was commented in previous review.

Signature · 1 line
Zbigniew
Quoted 905 lines
lib/gpgpu_shader.c +7 -0
+}
+
+/**
+ * gpgpu_shader__wait:
+ * @shdr: shader to be modified
+ *
+ * Append wait instruction to @shader. This instruction raises 

attention
+ * and stops execution.
+ */

lib/gpgpu_shader.c +20 -0
+void gpgpu_shader__wait(struct gpgpu_shader *shdr)
+{
+    emit_iga64_code(shdr, sync_host, "    \n\
+(W)    sync.host            null        \n\
+    ");
+}
+
+/**
+ * gpgpu_shader__nop:
+ * @shdr: shader to be modified
+ *
+ * Append a no-op instruction to @shdr.
+ */
+void gpgpu_shader__nop(struct gpgpu_shader *shdr)
+{
+    emit_iga64_code(shdr, nop, "    \n\
+(W)    nop                \n\
+    ");
+}
+

  /**
   * gpgpu_shader__eot:
   * @shdr: shader to be modified

lib/gpgpu_shader.c +48 -0
@@ -269,6 +349,246 @@ void gpgpu_shader__eot(struct gpgpu_shader *shdr)
       ");
   }
+/**
+ * gpgpu_shader__label:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Create a label for the last instruction within @shdr.
+ */
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id)
+{
+    struct label_entry *l = malloc(sizeof(*l));
+
+    l->id = label_id;
+    l->offset = shdr->size;
+    igt_map_insert(shdr->labels, &l->id, l);
+}
+
+#define OPCODE(x) (x & 0x7f)
+#define OPCODE_JUMP_INDEXED 0x20
+static void __patch_indexed_jump(struct gpgpu_shader *shdr, int 
 label_id,
+                 uint32_t jump_iga64_size)
+{
+    struct label_entry *l;
+    uint32_t *start, *end, *label;
+    int32_t relative;
+
+    l = igt_map_search(shdr->labels, &label_id);
+    igt_assert(l);
+
+    igt_assert(jump_iga64_size % 4 == 0);
+
+    label = shdr->code + l->offset;
+    end = shdr->code + shdr->size;
+    start = end - jump_iga64_size;
+
+    for (; start < end; start += 4)
+        if (OPCODE(*start) == OPCODE_JUMP_INDEXED) {
+            relative = (label - start) * 4;
+            *(start + 3) = relative;
+            break;
+        }
+}
+
+/**
+ * gpgpu_shader__jump:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label 

@label_id.
+ */

lib/gpgpu_shader.c +20 -0
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id)
+{
+    size_t shader_size;
+
+    shader_size = emit_iga64_code(shdr, jump, "    \n\
+L0:                            \n\
+(W)    jmpi        L0                    \n\
+    ");
+
+    __patch_indexed_jump(shdr, label_id, shader_size);
+}
+
+/**
+ * gpgpu_shader__jump_neq:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label 

@label_id
+ * when @value is not equal to dword stored at @y_offset within the surface.
+ */

lib/gpgpu_shader.c +8 -0
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+                uint32_t y_offset, uint32_t value)
+{
+    uint32_t size;
+
+    size = emit_iga64_code(shdr, jump_dw_neq, "                    \n\
+L0:                                            \n\
+(W)        mov (16|M0)              r30.0<1>:ud    

0x0:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                            \n\
+    // Y offset of the block in rows := thread group id
Y                \n\
+(W)        mov (1|M0)               r30.1<1>:ud
ARG(0):ud            \n\
+    // block width [0,63] representing 1 to 64 bytes, we want dword            \n\
+(W)        mov (1|M0)               r30.2<1>:ud
0x3:ud                \n\
+    // FFTID := FFTID from R0 header                        \n\
+(W)        mov (1|M0)               r30.4<1>:ud r0.5<0;1,0>:ud          \n\
+(W)        send.dc1 (16|M0)         r31     r30      null    0x0 0x2190000    \n\
+#else // Typed 2D Block Store                                \n\
+    // Store X and Y block start (160:191 and
192:223)                \n\
+(W)            mov (2|M0)               r30.6<1>:ud ARG(0):ud            \n\
+    // Store X and Y block size (224:231 and 232:239)
\n\
+(W)            mov (1|M0)               r30.7<1>:ud
0x3:ud                \n\
+(W)            send.tgm (16|M0)         r31     r30    null:0 0x0    0x62100003    \n\

lib/gpgpu_shader.c +3 -0
+#endif                                            \n\
+    // clear the flag register                            \n\
+(W)        mov (1|M0)               f0.0<1>:ud    

0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud ARG(1):ud    \n\

lib/gpgpu_shader.c +12 -0
+(W&f0.0)    jmpi                     L0                        \n\
+    ", y_offset, value);
+
+    __patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__loop_begin:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Begin a counting loop in @shdr. All subsequent instructions will 

constitute
+ * the loop body up until 'gpgpu_shader__loop_end' gets called. The first
+ * instruction of the loop will be at label @label_id. The r40 register will be
+ * overwritten as it is used as the loop counter.
+ */

lib/gpgpu_shader.c +19 -0
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id)
+{
+    emit_iga64_code(shdr, clear_r40, "        \n\
+L0:                            \n\
+(W)    mov (1|M0)               r40:ud    0x0:ud    \n\
+    ");
+
+    gpgpu_shader__label(shdr, label_id);
+}
+
+/**
+ * gpgpu_shader__loop_end:
+ * @shdr: shader to be modified
+ * @label_id: label id passed to 'gpgpu_shader__loop_begin'
+ * @iter: iteration count
+ *
+ * End loop body in @shdr.
+ */
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, 

uint32_t iter)

lib/gpgpu_shader.c +6 -0
+{
+    uint32_t size;
+
+    size = emit_iga64_code(shdr, inc_r40_jump_neq, "                \n\
+L0:                                            \n\
+(W)        add (1|M0)              r40:ud          r40.0<0;1,0>:ud 

0x1:ud        \n\
+(W)        mov (1|M0)              f0.0<1>:ud
0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud     r40.0<0;1,0>:ud ARG(0):ud    \n\

lib/gpgpu_shader.c +33 -0
+(W&f0.0)    jmpi                     L0                        \n\
+    ", iter);
+
+    __patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__common_target_write:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: oword to be written
+ *
+ * Write the oword stored in @value to the target buffer at @y_offset.
+ */
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, const uint32_t value[4])
+{
+    emit_iga64_code(shdr, common_target_write, "                \n\
+(W)    mov (16|M0)        r30.0<1>:ud    0x0:ud                \n\
+(W)    mov (16|M0)        r31.0<1>:ud    0x0:ud                \n\
+(W)    mov (1|M0)        r31.0<1>:ud    ARG(1):ud            \n\
+(W)    mov (1|M0)        r31.1<1>:ud    ARG(2):ud            \n\
+(W)    mov (1|M0)        r31.2<1>:ud    ARG(3):ud            \n\
+(W)    mov (1|M0)        r31.3<1>:ud    ARG(4):ud            \n\
+#if GEN_VER < 2000 // Media Block Write                        \n\
+    // Y offset of the block in rows                    \n\
+(W)    mov (1|M0)        r30.1<1>:ud    ARG(0):ud            \n\
+    // block width [0,63] representing 1 to 64 bytes            \n\
+(W)    mov (1|M0)        r30.2<1>:ud    0xf:ud                \n\
+    // FFTID := FFTID from R0 header                    \n\
+(W)    mov (1|M0)        r30.4<1>:ud    r0.5<0;1,0>:ud            \n\
+    // written value                            \n\
+(W)    send.dc1 (16|M0)    null    r30    src1_null  0x0    

0x40A8000    \n\

lib/gpgpu_shader.c +5 -0
+#else    // Typed 2D Block Store                            \n\
+    // Store X and Y block start (160:191 and 192:223)            \n\
+(W)    mov (2|M0)              r30.6<1>:ud     ARG(0):ud            \n\
+    // Store X and Y block size (224:231 and 232:239)            \n\
+(W)    mov (1|M0)              r30.7<1>:ud     0xf:ud                

\n\
+(W)    send.tgm (16|M0)        null    r30     null:0  0x0 0x64000007    \n\

lib/gpgpu_shader.c +29 -0
+#endif                                        \n\
+    ", y_offset, value[0], value[1], value[2], value[3]);
+}
+
+/**
+ * gpgpu_shader__common_target_write_u32:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: dword to be written
+ *
+ * Fill oword at @y_offset with dword stored in @value.
+ */
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, uint32_t value)
+{
+    const uint32_t owblock[4] = {
+        value, value, value, value
+    };
+    gpgpu_shader__common_target_write(shdr, y_offset, owblock);
+}
+
+/**
+ * gpgpu_shader__write_aip:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within the surface in rows
+ *
+ * Write address instruction pointer to row tg_id_y + @y_offset.
+ */
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t 

y_offset)

lib/gpgpu_shader.c +4 -0
+{
+    emit_iga64_code(shdr, media_block_write_aip, "                \n\
+    // Payload                                \n\
+(W)    mov (1|M0)               r5.0<1>:ud    

cr0.2:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                        \n\
+    // X offset of the block in bytes := (thread group id X << ARG(0))    \n\
+(W)    shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud 0x2:ud        \n\
+    // Y offset of the block in rows := thread group id Y \n\
+(W)    mov (1|M0)               r4.1<1>:ud r0.6<0;1,0>:ud            \n\
+(W)    add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud ARG(0):ud    \n\
+    // block width [0,63] representing 1 to 64 bytes            \n\

lib/gpgpu_shader.c +3 -0
+(W)    mov (1|M0)               r4.2<1>:ud    0x3:ud                \n\
+    // FFTID := FFTID from R0 header                    \n\
+(W)    mov (1|M0)               r4.4<1>:ud    

r0.5<0;1,0>:ud            \n\
+(W)    send.dc1 (16|M0)         null     r4   src1_null 0 0x40A8000    \n\

lib/gpgpu_shader.c +3 -0
+#else // Typed 2D Block Store                            \n\
+    // Load r2.0-3 with tg id X << ARG(0)                    \n\
+(W)    shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud    

0x2:ud        \n\
+    // Load r2.4-7 with tg id Y + ARG(1):ud                    \n\
+(W)    mov (1|M0)               r2.1<1>:ud r0.6<0;1,0>:ud            \n\
+(W)    add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud ARG(0):ud    \n\
+    // payload setup                            \n\

lib/gpgpu_shader.c +3 -0
+(W)    mov (16|M0)              r4.0<1>:ud    0x0:ud                \n\
+    // Store X and Y block start (160:191 and 192:223)            \n\
+(W)    mov (2|M0)               r4.5<1>:ud    

r2.0<2;2,1>:ud            \n\
+    // Store X and Y block max_size (224:231 and 232:239) \n\

lib/gpgpu_shader.c +2 -0
+(W)    mov (1|M0)               r4.7<1>:ud    0x3:ud                \n\
+(W)    send.tgm (16|M0)         null     r4   null:0    0    

0x64000007    \n\

lib/gpgpu_shader.c +4 -0
+#endif                                        \n\
+    ", y_offset);
+}
+

  /**
   * gpgpu_shader__write_dword:
   * @shdr: shader to be modified

lib/gpgpu_shader.c +0 -0
@@ -313,3 +633,73 @@ void gpgpu_shader__write_dword(struct 
 gpgpu_shader *shdr, uint32_t value,
   #endif                                        \n\
       ", 2, y_offset, 3, value, value, value, value);

  }

lib/gpgpu_shader.c +7 -0
+
+/**
+ * gpgpu_shader__end_system_routine:
+ * @shdr: shader to be modified
+ * @breakpoint_suppress: breakpoint suppress flag
+ *
+ * Return from system routine. To prevent infinite jumping to the 

system
+ * routine on a breakpoint, @breakpoint_suppress flag has to be set.
+ */

lib/gpgpu_shader.c +10 -0
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+                      bool breakpoint_suppress)
+{
+    /*
+     * set breakpoint suppress bit to avoid an endless loop
+     * when sip was invoked by a breakpoint
+     */
+    if (breakpoint_suppress)
+        emit_iga64_code(shdr, breakpoint_suppress, "            \n\
+(W)    or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   

0x8000:ud    \n\
+        ");

lib/gpgpu_shader.c +3 -0
+
+    emit_iga64_code(shdr, end_system_routine, "                \n\
+(W)    and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   

ARG(0):ud    \n\
+    // return to an application                        \n\
+(W)    and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud 0x7FFFFFFD:ud    \n\
+    ", 0x7fffff | (1 << 26)); /* clear all exceptions, except read only bit */

lib/gpgpu_shader.c +9 -0
+}
+
+/**
+ * gpgpu_shader__end_system_routine_step_if_eq:
+ * @shdr: shader to be modified
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value for single stepping execution
+ *
+ * Return from system routine. Don't clear breakpoint exception when 

@value
+ * is equal to value stored at @y_offset. This triggers the system routine
+ * after the subsequent instruction, resulting in single stepping execution.
+ */
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+                         uint32_t y_offset,
+                         uint32_t value)
+{
+    emit_iga64_code(shdr, end_system_routine_step_if_eq,
"                \n\
+(W)        or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud 0x8000:ud    \n\
+(W)        and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud ARG(0):ud    \n\
+(W)        mov (16|M0)              r30.0<1>:ud
0x0:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                            \n\
+        // Y offset of the block in rows := thread group id Y            \n\
+(W)        mov (1|M0)               r30.1<1>:ud
ARG(1):ud            \n\
+        // block width [0,63] representing 1 to 64 bytes, we want dword        \n\
+(W)        mov (1|M0)               r30.2<1>:ud
0x3:ud                \n\
+        // FFTID := FFTID from R0 header                    \n\
+(W)        mov (1|M0)               r30.4<1>:ud r0.5<0;1,0>:ud            \n\
+(W)        send.dc1 (16|M0)         r31     r30      null    0x0 0x2190000    \n\
+#else    // Typed 2D Block Store                                \n\
+        // Store X and Y block start (160:191 and
192:223)            \n\
+(W)        mov (2|M0)               r30.6<1>:ud
ARG(1):ud            \n\
+        // Store X and Y block size (224:231 and 232:239) \n\
+(W)        mov (1|M0)               r30.7<1>:ud
0x3:ud                \n\
+(W)        send.tgm (16|M0)         r31     r30    null:0    0x0 0x62100003    \n\

lib/gpgpu_shader.c +3 -0
+#endif                                            \n\
+        // clear the flag register                        \n\
+(W)        mov (1|M0)               f0.0<1>:ud    

0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud ARG(2):ud    \n\
+(W&f0.0)    and (1|M0)              cr0.1<1>:ud cr0.1<0;1,0>:ud   ARG(3):ud    \n\
+        // return to an application                        \n\
+(W)        and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud 0x7FFFFFFD:ud    \n\
+    ", 0x807fffff, /* leave breakpoint exception */
+    y_offset, value, 0x7fffff /* clear all exceptions */ );

lib/gpgpu_shader.c +1 -0
+}
lib/gpgpu_shader.h +22 -2
index 255f93b4d..e4ca0be4c 100644
@@ -21,6 +21,7 @@ struct gpgpu_shader {
           uint32_t *code;
           uint32_t (*instr)[4];
       };
+    struct igt_map *labels;
   };
   struct iga64_template {
@@ -31,7 +32,7 @@ struct iga64_template {
   #pragma GCC diagnostic ignored "-Wnested-externs"
-void
+uint32_t
   __emit_iga64_code(struct gpgpu_shader *shdr, const struct iga64_template *tpls,
             int argc, uint32_t *argv);
@@ -56,8 +57,27 @@ void gpgpu_shader_exec(struct intel_bb *ibb,
                  struct gpgpu_shader *sip,
                  uint64_t ring, bool explicit_engine);
+void gpgpu_shader__wait(struct gpgpu_shader *shdr);
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t 
 cmd_no);
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr);
+void gpgpu_shader__nop(struct gpgpu_shader *shdr);
   void gpgpu_shader__eot(struct gpgpu_shader *shdr);
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, const uint32_t value[4]);
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+                     uint32_t y_offset, uint32_t value);
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+                      bool breakpoint_suppress);
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader 
 *shdr,
+                         uint32_t dw_offset,
+                         uint32_t value);
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t 
 y_offset);
   void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
                      uint32_t y_offset);
-
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+                uint32_t dw_offset, uint32_t value);
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, 

uint32_t iter);
  #endif /* GPGPU_SHADER_H */

lib/iga64_generated_codes.c +114 -1
index ea8d0f097..dd849eebc 100644
@@ -3,7 +3,7 @@
   #include "gpgpu_shader.h"
-#define MD5_SUM_IGA64_ASMS 9977ade854d57c5af5c5ca9e93c0f37e
+#define MD5_SUM_IGA64_ASMS 33b7cd843e3b009c123a85a6c520d7d0
   struct iga64_template const iga64_code_gpgpu_fill[] = {
       { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
@@ -79,6 +79,119 @@ struct iga64_template const 
 iga64_code_gpgpu_fill[] = {
       }}
   };
+struct iga64_template const 
 iga64_code_end_system_routine_step_if_eq[] = {
+    { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+        0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 52, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 48, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 52, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 48, .code = (const uint32_t []) {
+        0x80000166, 0x80018220, 0x02008000, 0x00008000,
+        0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040161, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000165, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_end_system_routine[] = {
+    { .gen_ver = 2000, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 12, .code = (const uint32_t []) {
+        0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_breakpoint_suppress[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000166, 0x80018220, 0x02008000, 0x00008000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
   struct iga64_template const iga64_code_media_block_write[] = {
       { .gen_ver = 2000, .size = 56, .code = (const uint32_t []) {

          0x80100061, 0x04054220, 0x00000000, 0x00000000,

lib/iga64_generated_codes.c +286 -0
@@ -164,6 +277,270 @@ struct iga64_template const 
 iga64_code_media_block_write[] = {
       }}
   };
+struct iga64_template const iga64_code_media_block_write_aip[] = {
+    { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008020, 0x00000000,
+        0x80000969, 0x02058220, 0x02000014, 0x00000002,
+        0x80000061, 0x02150220, 0x00000064, 0x00000000,
+        0x80001940, 0x02158220, 0x02000214, 0xc0ded000,
+        0x80100061, 0x04054220, 0x00000000, 0x00000000,
+        0x80041a61, 0x04550220, 0x00220205, 0x00000000,
+        0x80000061, 0x04754220, 0x00000000, 0x00000003,
+        0x80132031, 0x00000000, 0xd00e0494, 0x04000000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008040, 0x00000000,
+        0x80000969, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 40, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008020, 0x00000000,
+        0x80000969, 0x04058220, 0x02000014, 0x00000002,
+        0x80000061, 0x04150220, 0x00000064, 0x00000000,
+        0x80001940, 0x04158220, 0x02000414, 0xc0ded000,
+        0x80000061, 0x04254220, 0x00000000, 0x00000003,
+        0x80000061, 0x04450220, 0x00000054, 0x00000000,
+        0x80132031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008040, 0x00000000,
+        0x80000969, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 40, .code = (const uint32_t []) {
+        0x80000161, 0x05050220, 0x00008040, 0x00000000,
+        0x80000169, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80000140, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_common_target_write[] = {
+    { .gen_ver = 2000, .size = 48, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e754220, 0x00000000, 0x0000000f,
+        0x80132031, 0x00000000, 0xd00e1e94, 0x04000000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 56, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 52, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e254220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 56, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 52, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_inc_r40_jump_neq[] = {
+    { .gen_ver = 2000, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80000270, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_clear_r40[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000061, 0x28054220, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000061, 0x28054220, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_jump_dw_neq[] = {
+    { .gen_ver = 2000, .size = 32, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+        0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffa0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 40, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 36, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x84000020, 0x00004000, 0x00000000, 0xffffff90,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 40, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 36, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff90,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_jump[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000020, 0x00004000, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000020, 0x00004000, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
   struct iga64_template const iga64_code_eot[] = {
       { .gen_ver = 2000, .size = 8, .code = (const uint32_t []) {
           0x800c0061, 0x70050220, 0x00460005, 0x00000000,
@@ -188,3 +565,25 @@ struct iga64_template const iga64_code_eot[] = {
           0x80049031, 0x00000004, 0x7020700c, 0x10000000,
       }}
   };
+
+struct iga64_template const iga64_code_nop[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x00000060, 0x00000000, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x00000060, 0x00000000, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_sync_host[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
Signature · 1 line
2.34.1
#7 lore raw

Re: [PATCH i-g-t v5 03/17] lib/gpgpu_shader: Extend shader building library

Quoted 110 lines · Christoph

Hi Zbigniew,

Quoted 107 lines · Christoph

Hi Zbigniew,

Quoted 103 lines · Zbigniew Kempczyński
Quoted 88 lines · Christoph Manszewski

Add shader building functions and iga64 code used by eudebug subtests.

Signed-off-by Dominik Karol Piątkowski

<[email protected]>
Cc: Dominik Grzegorzek <[email protected]>
---
  lib/gpgpu_shader.c          | 392 ++++++++++++++++++++++++++++++++++- lib/gpgpu_shader.h          |  24 ++- lib/iga64_generated_codes.c | 401 +++++++++++++++++++++++++++++++++++-

  3 files changed, 813 insertions(+), 4 deletions(-)
lib/gpgpu_shader.c +7 -1
index 80bad342a..c723577e6 100644
@@ -7,10 +7,16 @@
   #include <i915_drm.h>
+#include "igt_map.h"
   #include "ioctl_wrappers.h"
   #include "gpgpu_shader.h"
   #include "gpu_cmds.h"
+struct label_entry {
+    uint32_t id;
+    uint32_t offset;
+};
+
   #define IGA64_ARG0 0xc0ded000
   #define IGA64_ARG_MASK 0xffffff00
@@ -32,7 +38,7 @@ static void gpgpu_shader_extend(struct
 gpgpu_shader *shdr)
       igt_assert(shdr->code);
   }
-void
+uint32_t

  __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
            int argc, uint32_t *argv)
  {

lib/gpgpu_shader.c +24 -0
@@ -60,6 +66,8 @@ __emit_iga64_code(struct gpgpu_shader *shdr,
 struct iga64_template const *tpls,
       }
       shdr->size += tpls->size;
+
+    return tpls->size;
   }
   static uint32_t fill_sip(struct intel_bb *ibb,
@@ -235,10 +243,16 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
       shdr->gen_ver = 100 * info->graphics_ver + info->graphics_rel;
       shdr->max_size = 16 * 4;
       shdr->code = malloc(4 * shdr->max_size);
+    shdr->labels = igt_map_create(igt_map_hash_32, igt_map_equal_32);
       igt_assert(shdr->code);
       return shdr;
   }
+static void free_func(struct igt_map_entry *entry)
+{
+       free(entry->data);
+}
+
   /**
    * gpgpu_shader_destroy:
    * @shdr: pointer to shader struct created with 'gpgpu_shader_create'
@@ -247,10 +261,76 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
    */
   void gpgpu_shader_destroy(struct gpgpu_shader *shdr)
   {
+    igt_map_destroy(shdr->labels, free_func);
       free(shdr->code);
       free(shdr);
   }
+/**
+ * gpgpu_shader_dump:
+ * @shdr: shader to be printed
+ *
+ * Print shader instructions from @shdr in hex.
+ */
+void gpgpu_shader_dump(struct gpgpu_shader *shdr)
+{
+    for (int i = 0; i < shdr->size / 4; i++)
+        igt_info("0x%08x 0x%08x 0x%08x 0x%08x\n",
+             shdr->instr[i][0], shdr->instr[i][1],
+             shdr->instr[i][2], shdr->instr[i][3]);
+}
+
+/**

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 292

Quoted 10 lines
lib/gpgpu_shader.c +9 -0
+ * gpgpu_shader__breakpoint_on:
+ * @shdr: shader to create breakpoint in
+ * @cmd_no: index of the instruction to break on
+ *
+ * Insert a breakpoint on the @cmd_no'th instruction within @shdr.
+ */
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr,
 uint32_t cmd_no)
+{
+    igt_assert(cmd_no < shdr->size / 4);

If I'm not wrong shdr->size contains number of dwords, not size in bytes. It is imo a little bit confusing and I would suggest to rename it to avoid confusion. Same with max_size.

Quoted 8 lines
Quoted 4 lines

It does, and I agree that changing it to something like 'dw_size' would get rid of the confusion, since by default I would also assume that a naked 'size' means bytes. However instructions are 4 dwords, so the division by 4 is relevant here.

Oh I realized that this series doesn't actually introduce this field. In that case I think we can change it in a separate patch as it is unrealated to this series.

I vote for renaming those fields as during review I always check how this sizes are used, as a dwords or real size.

@Andrzej - what's your opinion?

--
Zbigniew

Quoted 940 lines

Thanks,
Christoph

Quoted 937 lines
Quoted 17 lines
Quoted 5 lines
lib/gpgpu_shader.c +2 -0
+    shdr->instr[cmd_no][0] |= 1<<30;
+}

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 304

Quoted 10 lines
lib/gpgpu_shader.c +10 -0
+
+/**
+ * gpgpu_shader__breakpoint:
+ * @shdr: shader to create breakpoint in
+ *
+ * Insert a breakpoint on the last instruction in @shdr.
+ */
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr)
+{
+    gpgpu_shader__breakpoint_on(shdr, shdr->size / 4 - 1);

Likely shdr->size - 1 only.

Like above - instructions are 4 dwords.

Thanks,
Christoph

Quoted 913 lines

Below code was commented in previous review.

Signature · 1 line
Zbigniew
Quoted 907 lines
lib/gpgpu_shader.c +7 -0
+}
+
+/**
+ * gpgpu_shader__wait:
+ * @shdr: shader to be modified
+ *
+ * Append wait instruction to @shader. This instruction raises

attention
+ * and stops execution.
+ */

lib/gpgpu_shader.c +20 -0
+void gpgpu_shader__wait(struct gpgpu_shader *shdr)
+{
+    emit_iga64_code(shdr, sync_host, "    \n\
+(W)    sync.host            null        \n\
+    ");
+}
+
+/**
+ * gpgpu_shader__nop:
+ * @shdr: shader to be modified
+ *
+ * Append a no-op instruction to @shdr.
+ */
+void gpgpu_shader__nop(struct gpgpu_shader *shdr)
+{
+    emit_iga64_code(shdr, nop, "    \n\
+(W)    nop                \n\
+    ");
+}
+

  /**
   * gpgpu_shader__eot:
   * @shdr: shader to be modified

lib/gpgpu_shader.c +48 -0
@@ -269,6 +349,246 @@ void gpgpu_shader__eot(struct gpgpu_shader *shdr)
       ");
   }
+/**
+ * gpgpu_shader__label:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Create a label for the last instruction within @shdr.
+ */
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id)
+{
+    struct label_entry *l = malloc(sizeof(*l));
+
+    l->id = label_id;
+    l->offset = shdr->size;
+    igt_map_insert(shdr->labels, &l->id, l);
+}
+
+#define OPCODE(x) (x & 0x7f)
+#define OPCODE_JUMP_INDEXED 0x20
+static void __patch_indexed_jump(struct gpgpu_shader *shdr, int
 label_id,
+                 uint32_t jump_iga64_size)
+{
+    struct label_entry *l;
+    uint32_t *start, *end, *label;
+    int32_t relative;
+
+    l = igt_map_search(shdr->labels, &label_id);
+    igt_assert(l);
+
+    igt_assert(jump_iga64_size % 4 == 0);
+
+    label = shdr->code + l->offset;
+    end = shdr->code + shdr->size;
+    start = end - jump_iga64_size;
+
+    for (; start < end; start += 4)
+        if (OPCODE(*start) == OPCODE_JUMP_INDEXED) {
+            relative = (label - start) * 4;
+            *(start + 3) = relative;
+            break;
+        }
+}
+
+/**
+ * gpgpu_shader__jump:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ *
+ * Append jump instruction to @shdr. Jump to instruction with

label @label_id.
+ */

lib/gpgpu_shader.c +20 -0
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id)
+{
+    size_t shader_size;
+
+    shader_size = emit_iga64_code(shdr, jump, "    \n\
+L0:                            \n\
+(W)    jmpi        L0                    \n\
+    ");
+
+    __patch_indexed_jump(shdr, label_id, shader_size);
+}
+
+/**
+ * gpgpu_shader__jump_neq:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value
+ *
+ * Append jump instruction to @shdr. Jump to instruction with

label @label_id
+ * when @value is not equal to dword stored at @y_offset within the surface.
+ */

lib/gpgpu_shader.c +8 -0
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+                uint32_t y_offset, uint32_t value)
+{
+    uint32_t size;
+
+    size = emit_iga64_code(shdr, jump_dw_neq, "                    \n\
+L0:                                            \n\
+(W)        mov (16|M0)              r30.0<1>:ud

0x0:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                            \n\
+    // Y offset of the block in rows := thread group id
Y                \n\
+(W)        mov (1|M0)               r30.1<1>:ud
ARG(0):ud            \n\
+    // block width [0,63] representing 1 to 64 bytes, we want dword            \n\
+(W)        mov (1|M0)               r30.2<1>:ud
0x3:ud                \n\
+    // FFTID := FFTID from R0 header                        \n\
+(W)        mov (1|M0)               r30.4<1>:ud r0.5<0;1,0>:ud          \n\
+(W)        send.dc1 (16|M0)         r31     r30      null
0x0 0x2190000    \n\
+#else // Typed 2D Block Store                                \n\
+    // Store X and Y block start (160:191 and
192:223)                \n\
+(W)            mov (2|M0)               r30.6<1>:ud ARG(0):ud            \n\
+    // Store X and Y block size (224:231 and 232:239)
\n\
+(W)            mov (1|M0)               r30.7<1>:ud
0x3:ud                \n\
+(W)            send.tgm (16|M0)         r31     r30    null:0 0x0    0x62100003    \n\

lib/gpgpu_shader.c +3 -0
+#endif                                            \n\
+    // clear the flag register                            \n\
+(W)        mov (1|M0)               f0.0<1>:ud

0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud
r31.0<0;1,0>:ud ARG(1):ud    \n\

lib/gpgpu_shader.c +12 -0
+(W&f0.0)    jmpi                     L0                        \n\
+    ", y_offset, value);
+
+    __patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__loop_begin:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Begin a counting loop in @shdr. All subsequent instructions

will constitute
+ * the loop body up until 'gpgpu_shader__loop_end' gets called. The first
+ * instruction of the loop will be at label @label_id. The r40 register will be
+ * overwritten as it is used as the loop counter.
+ */

lib/gpgpu_shader.c +19 -0
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id)
+{
+    emit_iga64_code(shdr, clear_r40, "        \n\
+L0:                            \n\
+(W)    mov (1|M0)               r40:ud    0x0:ud    \n\
+    ");
+
+    gpgpu_shader__label(shdr, label_id);
+}
+
+/**
+ * gpgpu_shader__loop_end:
+ * @shdr: shader to be modified
+ * @label_id: label id passed to 'gpgpu_shader__loop_begin'
+ * @iter: iteration count
+ *
+ * End loop body in @shdr.
+ */
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int

label_id, uint32_t iter)

lib/gpgpu_shader.c +6 -0
+{
+    uint32_t size;
+
+    size = emit_iga64_code(shdr, inc_r40_jump_neq, "                \n\
+L0:                                            \n\
+(W)        add (1|M0)              r40:ud         

r40.0<0;1,0>:ud 0x1:ud        \n\
+(W)        mov (1|M0)              f0.0<1>:ud
0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud
r40.0<0;1,0>:ud ARG(0):ud    \n\

lib/gpgpu_shader.c +33 -0
+(W&f0.0)    jmpi                     L0                        \n\
+    ", iter);
+
+    __patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__common_target_write:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: oword to be written
+ *
+ * Write the oword stored in @value to the target buffer at @y_offset.
+ */
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, const uint32_t value[4])
+{
+    emit_iga64_code(shdr, common_target_write, "                \n\
+(W)    mov (16|M0)        r30.0<1>:ud    0x0:ud                \n\
+(W)    mov (16|M0)        r31.0<1>:ud    0x0:ud                \n\
+(W)    mov (1|M0)        r31.0<1>:ud    ARG(1):ud            \n\
+(W)    mov (1|M0)        r31.1<1>:ud    ARG(2):ud            \n\
+(W)    mov (1|M0)        r31.2<1>:ud    ARG(3):ud            \n\
+(W)    mov (1|M0)        r31.3<1>:ud    ARG(4):ud            \n\
+#if GEN_VER < 2000 // Media Block Write                        \n\
+    // Y offset of the block in rows                    \n\
+(W)    mov (1|M0)        r30.1<1>:ud    ARG(0):ud            \n\
+    // block width [0,63] representing 1 to 64 bytes            \n\
+(W)    mov (1|M0)        r30.2<1>:ud    0xf:ud                \n\
+    // FFTID := FFTID from R0 header                    \n\
+(W)    mov (1|M0)        r30.4<1>:ud    r0.5<0;1,0>:ud            \n\
+    // written value                            \n\
+(W)    send.dc1 (16|M0)    null    r30    src1_null  0x0

0x40A8000    \n\

lib/gpgpu_shader.c +5 -0
+#else    // Typed 2D Block Store                            \n\
+    // Store X and Y block start (160:191 and 192:223)            \n\
+(W)    mov (2|M0)              r30.6<1>:ud     ARG(0):ud            \n\
+    // Store X and Y block size (224:231 and 232:239)            \n\
+(W)    mov (1|M0)              r30.7<1>:ud     0xf:ud

\n\
+(W)    send.tgm (16|M0)        null    r30     null:0  0x0 0x64000007    \n\

lib/gpgpu_shader.c +29 -0
+#endif                                        \n\
+    ", y_offset, value[0], value[1], value[2], value[3]);
+}
+
+/**
+ * gpgpu_shader__common_target_write_u32:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: dword to be written
+ *
+ * Fill oword at @y_offset with dword stored in @value.
+ */
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, uint32_t value)
+{
+    const uint32_t owblock[4] = {
+        value, value, value, value
+    };
+    gpgpu_shader__common_target_write(shdr, y_offset, owblock);
+}
+
+/**
+ * gpgpu_shader__write_aip:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within the surface in rows
+ *
+ * Write address instruction pointer to row tg_id_y + @y_offset.
+ */
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr,

uint32_t y_offset)

lib/gpgpu_shader.c +4 -0
+{
+    emit_iga64_code(shdr, media_block_write_aip, "                \n\
+    // Payload                                \n\
+(W)    mov (1|M0)               r5.0<1>:ud

cr0.2:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                        \n\
+    // X offset of the block in bytes := (thread group id X << ARG(0))    \n\
+(W)    shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud 0x2:ud        \n\
+    // Y offset of the block in rows := thread group id Y \n\
+(W)    mov (1|M0)               r4.1<1>:ud r0.6<0;1,0>:ud            \n\
+(W)    add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud ARG(0):ud    \n\
+    // block width [0,63] representing 1 to 64 bytes            \n\

lib/gpgpu_shader.c +3 -0
+(W)    mov (1|M0)               r4.2<1>:ud    0x3:ud                \n\
+    // FFTID := FFTID from R0 header                    \n\
+(W)    mov (1|M0)               r4.4<1>:ud

r0.5<0;1,0>:ud            \n\
+(W)    send.dc1 (16|M0)         null     r4   src1_null 0 0x40A8000    \n\

lib/gpgpu_shader.c +3 -0
+#else // Typed 2D Block Store                            \n\
+    // Load r2.0-3 with tg id X << ARG(0)                    \n\
+(W)    shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud

0x2:ud        \n\
+    // Load r2.4-7 with tg id Y + ARG(1):ud                    \n\
+(W)    mov (1|M0)               r2.1<1>:ud r0.6<0;1,0>:ud            \n\
+(W)    add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud ARG(0):ud    \n\
+    // payload setup                            \n\

lib/gpgpu_shader.c +3 -0
+(W)    mov (16|M0)              r4.0<1>:ud    0x0:ud                \n\
+    // Store X and Y block start (160:191 and 192:223)            \n\
+(W)    mov (2|M0)               r4.5<1>:ud

r2.0<2;2,1>:ud            \n\
+    // Store X and Y block max_size (224:231 and 232:239) \n\

lib/gpgpu_shader.c +2 -0
+(W)    mov (1|M0)               r4.7<1>:ud    0x3:ud                \n\
+(W)    send.tgm (16|M0)         null     r4   null:0    0

0x64000007    \n\

lib/gpgpu_shader.c +4 -0
+#endif                                        \n\
+    ", y_offset);
+}
+

  /**
   * gpgpu_shader__write_dword:
   * @shdr: shader to be modified

lib/gpgpu_shader.c +0 -0
@@ -313,3 +633,73 @@ void gpgpu_shader__write_dword(struct
 gpgpu_shader *shdr, uint32_t value,
   #endif                                        \n\
       ", 2, y_offset, 3, value, value, value, value);

  }

lib/gpgpu_shader.c +7 -0
+
+/**
+ * gpgpu_shader__end_system_routine:
+ * @shdr: shader to be modified
+ * @breakpoint_suppress: breakpoint suppress flag
+ *
+ * Return from system routine. To prevent infinite jumping to

the system
+ * routine on a breakpoint, @breakpoint_suppress flag has to be set.
+ */

lib/gpgpu_shader.c +10 -0
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+                      bool breakpoint_suppress)
+{
+    /*
+     * set breakpoint suppress bit to avoid an endless loop
+     * when sip was invoked by a breakpoint
+     */
+    if (breakpoint_suppress)
+        emit_iga64_code(shdr, breakpoint_suppress, "            \n\
+(W)    or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud

0x8000:ud    \n\
+        ");

lib/gpgpu_shader.c +3 -0
+
+    emit_iga64_code(shdr, end_system_routine, "                \n\
+(W)    and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud

ARG(0):ud    \n\
+    // return to an application                        \n\
+(W)    and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud 0x7FFFFFFD:ud    \n\
+    ", 0x7fffff | (1 << 26)); /* clear all exceptions, except read only bit */

lib/gpgpu_shader.c +9 -0
+}
+
+/**
+ * gpgpu_shader__end_system_routine_step_if_eq:
+ * @shdr: shader to be modified
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value for single stepping execution
+ *
+ * Return from system routine. Don't clear breakpoint exception

when @value
+ * is equal to value stored at @y_offset. This triggers the system routine
+ * after the subsequent instruction, resulting in single stepping execution.
+ */
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+                         uint32_t y_offset,
+                         uint32_t value)
+{
+    emit_iga64_code(shdr, end_system_routine_step_if_eq,
"                \n\
+(W)        or  (1|M0)               cr0.0<1>:ud
cr0.0<0;1,0>:ud 0x8000:ud    \n\
+(W)        and (1|M0)               cr0.1<1>:ud
cr0.1<0;1,0>:ud ARG(0):ud    \n\
+(W)        mov (16|M0)              r30.0<1>:ud
0x0:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                            \n\
+        // Y offset of the block in rows := thread group id Y            \n\
+(W)        mov (1|M0)               r30.1<1>:ud
ARG(1):ud            \n\
+        // block width [0,63] representing 1 to 64 bytes, we want dword        \n\
+(W)        mov (1|M0)               r30.2<1>:ud
0x3:ud                \n\
+        // FFTID := FFTID from R0 header                    \n\
+(W)        mov (1|M0)               r30.4<1>:ud r0.5<0;1,0>:ud            \n\
+(W)        send.dc1 (16|M0)         r31     r30      null
0x0 0x2190000    \n\
+#else    // Typed 2D Block Store                                \n\
+        // Store X and Y block start (160:191 and
192:223)            \n\
+(W)        mov (2|M0)               r30.6<1>:ud
ARG(1):ud            \n\
+        // Store X and Y block size (224:231 and 232:239) \n\
+(W)        mov (1|M0)               r30.7<1>:ud
0x3:ud                \n\
+(W)        send.tgm (16|M0)         r31     r30    null:0
0x0 0x62100003    \n\

lib/gpgpu_shader.c +3 -0
+#endif                                            \n\
+        // clear the flag register                        \n\
+(W)        mov (1|M0)               f0.0<1>:ud

0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud
r31.0<0;1,0>:ud ARG(2):ud    \n\
+(W&f0.0)    and (1|M0)              cr0.1<1>:ud cr0.1<0;1,0>:ud   ARG(3):ud    \n\
+        // return to an application                        \n\
+(W)        and (1|M0)               cr0.0<1>:ud
cr0.0<0;1,0>:ud 0x7FFFFFFD:ud    \n\
+    ", 0x807fffff, /* leave breakpoint exception */ +    y_offset, value, 0x7fffff /* clear all exceptions */ );

lib/gpgpu_shader.c +1 -0
+}
lib/gpgpu_shader.h +2 -1
index 255f93b4d..e4ca0be4c 100644
@@ -21,6 +21,7 @@ struct gpgpu_shader {
           uint32_t *code;
           uint32_t (*instr)[4];
       };
+    struct igt_map *labels;
   };
   struct iga64_template {
@@ -31,7 +32,7 @@ struct iga64_template {
   #pragma GCC diagnostic ignored "-Wnested-externs"
-void
+uint32_t

  __emit_iga64_code(struct gpgpu_shader *shdr, const struct iga64_template *tpls,
            int argc, uint32_t *argv);

lib/gpgpu_shader.h +14 -0
@@ -56,8 +57,27 @@ void gpgpu_shader_exec(struct intel_bb *ibb,
                  struct gpgpu_shader *sip,
                  uint64_t ring, bool explicit_engine);
+void gpgpu_shader__wait(struct gpgpu_shader *shdr);
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr,
 uint32_t cmd_no);
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr);
+void gpgpu_shader__nop(struct gpgpu_shader *shdr);
   void gpgpu_shader__eot(struct gpgpu_shader *shdr);
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, const uint32_t value[4]);
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+                     uint32_t y_offset, uint32_t value);
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+                      bool breakpoint_suppress);
+void gpgpu_shader__end_system_routine_step_if_eq(struct
 gpgpu_shader *shdr,
+                         uint32_t dw_offset,
+                         uint32_t value);
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr,
 uint32_t y_offset);
   void gpgpu_shader__write_dword(struct gpgpu_shader *shdr,
 uint32_t value,

                     uint32_t y_offset);

lib/gpgpu_shader.h +6 -1
-
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+                uint32_t dw_offset, uint32_t value);
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int

label_id, uint32_t iter);
  #endif /* GPGPU_SHADER_H */

lib/iga64_generated_codes.c +114 -1
index ea8d0f097..dd849eebc 100644
@@ -3,7 +3,7 @@
   #include "gpgpu_shader.h"
-#define MD5_SUM_IGA64_ASMS 9977ade854d57c5af5c5ca9e93c0f37e
+#define MD5_SUM_IGA64_ASMS 33b7cd843e3b009c123a85a6c520d7d0
   struct iga64_template const iga64_code_gpgpu_fill[] = {
       { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
@@ -79,6 +79,119 @@ struct iga64_template const
 iga64_code_gpgpu_fill[] = {
       }}
   };
+struct iga64_template const
 iga64_code_end_system_routine_step_if_eq[] = {
+    { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+        0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 52, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 48, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 52, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 48, .code = (const uint32_t []) {
+        0x80000166, 0x80018220, 0x02008000, 0x00008000,
+        0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040161, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000165, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_end_system_routine[] = {
+    { .gen_ver = 2000, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 12, .code = (const uint32_t []) {
+        0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_breakpoint_suppress[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000166, 0x80018220, 0x02008000, 0x00008000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
   struct iga64_template const iga64_code_media_block_write[] = {
       { .gen_ver = 2000, .size = 56, .code = (const uint32_t []) {

          0x80100061, 0x04054220, 0x00000000, 0x00000000,

lib/iga64_generated_codes.c +286 -0
@@ -164,6 +277,270 @@ struct iga64_template const
 iga64_code_media_block_write[] = {
       }}
   };
+struct iga64_template const iga64_code_media_block_write_aip[] = {
+    { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008020, 0x00000000,
+        0x80000969, 0x02058220, 0x02000014, 0x00000002,
+        0x80000061, 0x02150220, 0x00000064, 0x00000000,
+        0x80001940, 0x02158220, 0x02000214, 0xc0ded000,
+        0x80100061, 0x04054220, 0x00000000, 0x00000000,
+        0x80041a61, 0x04550220, 0x00220205, 0x00000000,
+        0x80000061, 0x04754220, 0x00000000, 0x00000003,
+        0x80132031, 0x00000000, 0xd00e0494, 0x04000000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008040, 0x00000000,
+        0x80000969, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 40, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008020, 0x00000000,
+        0x80000969, 0x04058220, 0x02000014, 0x00000002,
+        0x80000061, 0x04150220, 0x00000064, 0x00000000,
+        0x80001940, 0x04158220, 0x02000414, 0xc0ded000,
+        0x80000061, 0x04254220, 0x00000000, 0x00000003,
+        0x80000061, 0x04450220, 0x00000054, 0x00000000,
+        0x80132031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008040, 0x00000000,
+        0x80000969, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 40, .code = (const uint32_t []) {
+        0x80000161, 0x05050220, 0x00008040, 0x00000000,
+        0x80000169, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80000140, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_common_target_write[] = {
+    { .gen_ver = 2000, .size = 48, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e754220, 0x00000000, 0x0000000f,
+        0x80132031, 0x00000000, 0xd00e1e94, 0x04000000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 56, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 52, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e254220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 56, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 52, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_inc_r40_jump_neq[] = {
+    { .gen_ver = 2000, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80000270, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_clear_r40[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000061, 0x28054220, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000061, 0x28054220, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_jump_dw_neq[] = {
+    { .gen_ver = 2000, .size = 32, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+        0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffa0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 40, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 36, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x84000020, 0x00004000, 0x00000000, 0xffffff90,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 40, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 36, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff90,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_jump[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000020, 0x00004000, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000020, 0x00004000, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
   struct iga64_template const iga64_code_eot[] = {
       { .gen_ver = 2000, .size = 8, .code = (const uint32_t []) {
           0x800c0061, 0x70050220, 0x00460005, 0x00000000,
@@ -188,3 +565,25 @@ struct iga64_template const iga64_code_eot[] = {
           0x80049031, 0x00000004, 0x7020700c, 0x10000000,
       }}
   };
+
+struct iga64_template const iga64_code_nop[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x00000060, 0x00000000, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x00000060, 0x00000000, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_sync_host[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
Signature · 1 line
2.34.1
#8 lore raw

Re: [PATCH i-g-t v5 03/17] lib/gpgpu_shader: Extend shader building library

Hi Zbigniew,

Quoted 116 lines · Zbigniew Kempczyński
Quoted 115 lines · Christoph

Hi Zbigniew,

Quoted 111 lines · Christoph

Hi Zbigniew,

Quoted 95 lines · Zbigniew Kempczyński
Quoted 94 lines · Christoph Manszewski

Add shader building functions and iga64 code used by eudebug subtests.

Signed-off-by Dominik Karol Piątkowski

<[email protected]>
Cc: Dominik Grzegorzek <[email protected]>

---
   lib/gpgpu_shader.c          | 392 ++++++++++++++++++++++++++++++++++-
   lib/gpgpu_shader.h          |  24 ++-
   lib/iga64_generated_codes.c | 401 +++++++++++++++++++++++++++++++++++-
   3 files changed, 813 insertions(+), 4 deletions(-)
lib/gpgpu_shader.c +7 -1
index 80bad342a..c723577e6 100644
@@ -7,10 +7,16 @@
   #include <i915_drm.h>
+#include "igt_map.h"
   #include "ioctl_wrappers.h"
   #include "gpgpu_shader.h"
   #include "gpu_cmds.h"
+struct label_entry {
+    uint32_t id;
+    uint32_t offset;
+};
+
   #define IGA64_ARG0 0xc0ded000
   #define IGA64_ARG_MASK 0xffffff00
@@ -32,7 +38,7 @@ static void gpgpu_shader_extend(struct
 gpgpu_shader *shdr)
       igt_assert(shdr->code);
   }
-void
+uint32_t
   __emit_iga64_code(struct gpgpu_shader *shdr, struct
 iga64_template const *tpls,
             int argc, uint32_t *argv)

  {

lib/gpgpu_shader.c +30 -0
@@ -60,6 +66,8 @@ __emit_iga64_code(struct gpgpu_shader *shdr,
 struct iga64_template const *tpls,
       }
       shdr->size += tpls->size;
+
+    return tpls->size;
   }
   static uint32_t fill_sip(struct intel_bb *ibb,
@@ -235,10 +243,16 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
       shdr->gen_ver = 100 * info->graphics_ver + info->graphics_rel;
       shdr->max_size = 16 * 4;
       shdr->code = malloc(4 * shdr->max_size);
+    shdr->labels = igt_map_create(igt_map_hash_32, igt_map_equal_32);
       igt_assert(shdr->code);
       return shdr;
   }
+static void free_func(struct igt_map_entry *entry)
+{
+       free(entry->data);
+}
+
   /**
    * gpgpu_shader_destroy:
    * @shdr: pointer to shader struct created with 'gpgpu_shader_create'
@@ -247,10 +261,76 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
    */
   void gpgpu_shader_destroy(struct gpgpu_shader *shdr)
   {
+    igt_map_destroy(shdr->labels, free_func);
       free(shdr->code);
       free(shdr);
   }
+/**
+ * gpgpu_shader_dump:
+ * @shdr: shader to be printed
+ *
+ * Print shader instructions from @shdr in hex.
+ */
+void gpgpu_shader_dump(struct gpgpu_shader *shdr)
+{
+    for (int i = 0; i < shdr->size / 4; i++)
+        igt_info("0x%08x 0x%08x 0x%08x 0x%08x\n",
+             shdr->instr[i][0], shdr->instr[i][1],
+             shdr->instr[i][2], shdr->instr[i][3]);
+}
+
+/**
+ * gpgpu_shader__breakpoint_on:
+ * @shdr: shader to create breakpoint in
+ * @cmd_no: index of the instruction to break on
+ *
+ * Insert a breakpoint on the @cmd_no'th instruction within @shdr.
+ */
Quoted 8 lines

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 292

Quoted 4 lines
lib/gpgpu_shader.c +3 -0
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr,
 uint32_t cmd_no)
+{
+    igt_assert(cmd_no < shdr->size / 4);

If I'm not wrong shdr->size contains number of dwords, not size in bytes. It is imo a little bit confusing and I would suggest to rename it to avoid confusion. Same with max_size.

It does, and I agree that changing it to something like 'dw_size' would get rid of the confusion, since by default I would also assume that a naked 'size' means bytes. However instructions are 4 dwords, so the division by 4 is relevant here.

Quoted 6 lines
Quoted 3 lines

Oh I realized that this series doesn't actually introduce this field. In that case I think we can change it in a separate patch as it is unrealated to this series.

I vote for renaming those fields as during review I always check how this sizes are used, as a dwords or real size.

I don't quite follow. This field already exists. It is used and treated as dwords. This series just uses an existing field and interprets it in a way it is supposed to. It doesn't change anything about how it is used - thus this is simply not related to this series. This seires contains enough changes as it is and I don't see fit for misc cleanups in it. I am fine with cleaning things like these up outside this series.

Thanks,
Christoph

Quoted 946 lines

@Andrzej - what's your opinion?

--
Zbigniew

Quoted 940 lines

Thanks,
Christoph

Quoted 937 lines
Quoted 17 lines
Quoted 5 lines
lib/gpgpu_shader.c +2 -0
+    shdr->instr[cmd_no][0] |= 1<<30;
+}

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 304

Quoted 10 lines
lib/gpgpu_shader.c +10 -0
+
+/**
+ * gpgpu_shader__breakpoint:
+ * @shdr: shader to create breakpoint in
+ *
+ * Insert a breakpoint on the last instruction in @shdr.
+ */
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr)
+{
+    gpgpu_shader__breakpoint_on(shdr, shdr->size / 4 - 1);

Likely shdr->size - 1 only.

Like above - instructions are 4 dwords.

Thanks,
Christoph

Quoted 913 lines

Below code was commented in previous review.

Signature · 1 line
Zbigniew
Quoted 907 lines
lib/gpgpu_shader.c +7 -0
+}
+
+/**
+ * gpgpu_shader__wait:
+ * @shdr: shader to be modified
+ *
+ * Append wait instruction to @shader. This instruction raises

attention
+ * and stops execution.
+ */

lib/gpgpu_shader.c +68 -0
+void gpgpu_shader__wait(struct gpgpu_shader *shdr)
+{
+    emit_iga64_code(shdr, sync_host, "    \n\
+(W)    sync.host            null        \n\
+    ");
+}
+
+/**
+ * gpgpu_shader__nop:
+ * @shdr: shader to be modified
+ *
+ * Append a no-op instruction to @shdr.
+ */
+void gpgpu_shader__nop(struct gpgpu_shader *shdr)
+{
+    emit_iga64_code(shdr, nop, "    \n\
+(W)    nop                \n\
+    ");
+}
+
   /**
    * gpgpu_shader__eot:
    * @shdr: shader to be modified
@@ -269,6 +349,246 @@ void gpgpu_shader__eot(struct gpgpu_shader *shdr)
       ");
   }
+/**
+ * gpgpu_shader__label:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Create a label for the last instruction within @shdr.
+ */
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id)
+{
+    struct label_entry *l = malloc(sizeof(*l));
+
+    l->id = label_id;
+    l->offset = shdr->size;
+    igt_map_insert(shdr->labels, &l->id, l);
+}
+
+#define OPCODE(x) (x & 0x7f)
+#define OPCODE_JUMP_INDEXED 0x20
+static void __patch_indexed_jump(struct gpgpu_shader *shdr, int
 label_id,
+                 uint32_t jump_iga64_size)
+{
+    struct label_entry *l;
+    uint32_t *start, *end, *label;
+    int32_t relative;
+
+    l = igt_map_search(shdr->labels, &label_id);
+    igt_assert(l);
+
+    igt_assert(jump_iga64_size % 4 == 0);
+
+    label = shdr->code + l->offset;
+    end = shdr->code + shdr->size;
+    start = end - jump_iga64_size;
+
+    for (; start < end; start += 4)
+        if (OPCODE(*start) == OPCODE_JUMP_INDEXED) {
+            relative = (label - start) * 4;
+            *(start + 3) = relative;
+            break;
+        }
+}
+
+/**
+ * gpgpu_shader__jump:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ *
+ * Append jump instruction to @shdr. Jump to instruction with

label @label_id.
+ */

lib/gpgpu_shader.c +20 -0
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id)
+{
+    size_t shader_size;
+
+    shader_size = emit_iga64_code(shdr, jump, "    \n\
+L0:                            \n\
+(W)    jmpi        L0                    \n\
+    ");
+
+    __patch_indexed_jump(shdr, label_id, shader_size);
+}
+
+/**
+ * gpgpu_shader__jump_neq:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value
+ *
+ * Append jump instruction to @shdr. Jump to instruction with

label @label_id
+ * when @value is not equal to dword stored at @y_offset within the surface.
+ */

lib/gpgpu_shader.c +8 -0
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+                uint32_t y_offset, uint32_t value)
+{
+    uint32_t size;
+
+    size = emit_iga64_code(shdr, jump_dw_neq, "                    \n\
+L0:                                            \n\
+(W)        mov (16|M0)              r30.0<1>:ud

0x0:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                            \n\
+    // Y offset of the block in rows := thread group id
Y                \n\
+(W)        mov (1|M0)               r30.1<1>:ud
ARG(0):ud            \n\
+    // block width [0,63] representing 1 to 64 bytes, we want dword            \n\
+(W)        mov (1|M0)               r30.2<1>:ud
0x3:ud                \n\
+    // FFTID := FFTID from R0 header                        \n\
+(W)        mov (1|M0)               r30.4<1>:ud r0.5<0;1,0>:ud          \n\
+(W)        send.dc1 (16|M0)         r31     r30      null
0x0 0x2190000    \n\
+#else // Typed 2D Block Store                                \n\
+    // Store X and Y block start (160:191 and
192:223)                \n\
+(W)            mov (2|M0)               r30.6<1>:ud ARG(0):ud            \n\
+    // Store X and Y block size (224:231 and 232:239)
\n\
+(W)            mov (1|M0)               r30.7<1>:ud
0x3:ud                \n\
+(W)            send.tgm (16|M0)         r31     r30    null:0 0x0    0x62100003    \n\

lib/gpgpu_shader.c +3 -0
+#endif                                            \n\
+    // clear the flag register                            \n\
+(W)        mov (1|M0)               f0.0<1>:ud

0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud
r31.0<0;1,0>:ud ARG(1):ud    \n\

lib/gpgpu_shader.c +12 -0
+(W&f0.0)    jmpi                     L0                        \n\
+    ", y_offset, value);
+
+    __patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__loop_begin:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Begin a counting loop in @shdr. All subsequent instructions

will constitute
+ * the loop body up until 'gpgpu_shader__loop_end' gets called. The first
+ * instruction of the loop will be at label @label_id. The r40 register will be
+ * overwritten as it is used as the loop counter.
+ */

lib/gpgpu_shader.c +19 -0
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id)
+{
+    emit_iga64_code(shdr, clear_r40, "        \n\
+L0:                            \n\
+(W)    mov (1|M0)               r40:ud    0x0:ud    \n\
+    ");
+
+    gpgpu_shader__label(shdr, label_id);
+}
+
+/**
+ * gpgpu_shader__loop_end:
+ * @shdr: shader to be modified
+ * @label_id: label id passed to 'gpgpu_shader__loop_begin'
+ * @iter: iteration count
+ *
+ * End loop body in @shdr.
+ */
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int

label_id, uint32_t iter)

lib/gpgpu_shader.c +6 -0
+{
+    uint32_t size;
+
+    size = emit_iga64_code(shdr, inc_r40_jump_neq, "                \n\
+L0:                                            \n\
+(W)        add (1|M0)              r40:ud

r40.0<0;1,0>:ud 0x1:ud        \n\
+(W)        mov (1|M0)              f0.0<1>:ud
0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud
r40.0<0;1,0>:ud ARG(0):ud    \n\

lib/gpgpu_shader.c +33 -0
+(W&f0.0)    jmpi                     L0                        \n\
+    ", iter);
+
+    __patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__common_target_write:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: oword to be written
+ *
+ * Write the oword stored in @value to the target buffer at @y_offset.
+ */
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, const uint32_t value[4])
+{
+    emit_iga64_code(shdr, common_target_write, "                \n\
+(W)    mov (16|M0)        r30.0<1>:ud    0x0:ud                \n\
+(W)    mov (16|M0)        r31.0<1>:ud    0x0:ud                \n\
+(W)    mov (1|M0)        r31.0<1>:ud    ARG(1):ud            \n\
+(W)    mov (1|M0)        r31.1<1>:ud    ARG(2):ud            \n\
+(W)    mov (1|M0)        r31.2<1>:ud    ARG(3):ud            \n\
+(W)    mov (1|M0)        r31.3<1>:ud    ARG(4):ud            \n\
+#if GEN_VER < 2000 // Media Block Write                        \n\
+    // Y offset of the block in rows                    \n\
+(W)    mov (1|M0)        r30.1<1>:ud    ARG(0):ud            \n\
+    // block width [0,63] representing 1 to 64 bytes            \n\
+(W)    mov (1|M0)        r30.2<1>:ud    0xf:ud                \n\
+    // FFTID := FFTID from R0 header                    \n\
+(W)    mov (1|M0)        r30.4<1>:ud    r0.5<0;1,0>:ud            \n\
+    // written value                            \n\
+(W)    send.dc1 (16|M0)    null    r30    src1_null  0x0

0x40A8000    \n\

lib/gpgpu_shader.c +5 -0
+#else    // Typed 2D Block Store                            \n\
+    // Store X and Y block start (160:191 and 192:223)            \n\
+(W)    mov (2|M0)              r30.6<1>:ud     ARG(0):ud            \n\
+    // Store X and Y block size (224:231 and 232:239)            \n\
+(W)    mov (1|M0)              r30.7<1>:ud     0xf:ud

\n\
+(W)    send.tgm (16|M0)        null    r30     null:0  0x0 0x64000007    \n\

lib/gpgpu_shader.c +29 -0
+#endif                                        \n\
+    ", y_offset, value[0], value[1], value[2], value[3]);
+}
+
+/**
+ * gpgpu_shader__common_target_write_u32:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: dword to be written
+ *
+ * Fill oword at @y_offset with dword stored in @value.
+ */
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, uint32_t value)
+{
+    const uint32_t owblock[4] = {
+        value, value, value, value
+    };
+    gpgpu_shader__common_target_write(shdr, y_offset, owblock);
+}
+
+/**
+ * gpgpu_shader__write_aip:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within the surface in rows
+ *
+ * Write address instruction pointer to row tg_id_y + @y_offset.
+ */
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr,

uint32_t y_offset)

lib/gpgpu_shader.c +4 -0
+{
+    emit_iga64_code(shdr, media_block_write_aip, "                \n\
+    // Payload                                \n\
+(W)    mov (1|M0)               r5.0<1>:ud

cr0.2:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                        \n\
+    // X offset of the block in bytes := (thread group id X << ARG(0))    \n\
+(W)    shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud 0x2:ud        \n\
+    // Y offset of the block in rows := thread group id Y \n\
+(W)    mov (1|M0)               r4.1<1>:ud r0.6<0;1,0>:ud            \n\
+(W)    add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud ARG(0):ud    \n\
+    // block width [0,63] representing 1 to 64 bytes            \n\

lib/gpgpu_shader.c +3 -0
+(W)    mov (1|M0)               r4.2<1>:ud    0x3:ud                \n\
+    // FFTID := FFTID from R0 header                    \n\
+(W)    mov (1|M0)               r4.4<1>:ud

r0.5<0;1,0>:ud            \n\
+(W)    send.dc1 (16|M0)         null     r4   src1_null 0 0x40A8000    \n\

lib/gpgpu_shader.c +3 -0
+#else // Typed 2D Block Store                            \n\
+    // Load r2.0-3 with tg id X << ARG(0)                    \n\
+(W)    shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud

0x2:ud        \n\
+    // Load r2.4-7 with tg id Y + ARG(1):ud                    \n\
+(W)    mov (1|M0)               r2.1<1>:ud r0.6<0;1,0>:ud            \n\
+(W)    add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud ARG(0):ud    \n\
+    // payload setup                            \n\

lib/gpgpu_shader.c +3 -0
+(W)    mov (16|M0)              r4.0<1>:ud    0x0:ud                \n\
+    // Store X and Y block start (160:191 and 192:223)            \n\
+(W)    mov (2|M0)               r4.5<1>:ud

r2.0<2;2,1>:ud            \n\
+    // Store X and Y block max_size (224:231 and 232:239) \n\

lib/gpgpu_shader.c +2 -0
+(W)    mov (1|M0)               r4.7<1>:ud    0x3:ud                \n\
+(W)    send.tgm (16|M0)         null     r4   null:0    0

0x64000007    \n\

lib/gpgpu_shader.c +11 -0
+#endif                                        \n\
+    ", y_offset);
+}
+
   /**
    * gpgpu_shader__write_dword:
    * @shdr: shader to be modified
@@ -313,3 +633,73 @@ void gpgpu_shader__write_dword(struct
 gpgpu_shader *shdr, uint32_t value,
   #endif                                        \n\
       ", 2, y_offset, 3, value, value, value, value);
   }
+
+/**
+ * gpgpu_shader__end_system_routine:
+ * @shdr: shader to be modified
+ * @breakpoint_suppress: breakpoint suppress flag
+ *
+ * Return from system routine. To prevent infinite jumping to

the system
+ * routine on a breakpoint, @breakpoint_suppress flag has to be set.
+ */

lib/gpgpu_shader.c +10 -0
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+                      bool breakpoint_suppress)
+{
+    /*
+     * set breakpoint suppress bit to avoid an endless loop
+     * when sip was invoked by a breakpoint
+     */
+    if (breakpoint_suppress)
+        emit_iga64_code(shdr, breakpoint_suppress, "            \n\
+(W)    or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud

0x8000:ud    \n\
+        ");

lib/gpgpu_shader.c +3 -0
+
+    emit_iga64_code(shdr, end_system_routine, "                \n\
+(W)    and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud

ARG(0):ud    \n\
+    // return to an application                        \n\
+(W)    and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud 0x7FFFFFFD:ud    \n\
+    ", 0x7fffff | (1 << 26)); /* clear all exceptions, except read only bit */

lib/gpgpu_shader.c +9 -0
+}
+
+/**
+ * gpgpu_shader__end_system_routine_step_if_eq:
+ * @shdr: shader to be modified
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value for single stepping execution
+ *
+ * Return from system routine. Don't clear breakpoint exception

when @value
+ * is equal to value stored at @y_offset. This triggers the system routine
+ * after the subsequent instruction, resulting in single stepping execution.
+ */
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+                         uint32_t y_offset,
+                         uint32_t value)
+{
+    emit_iga64_code(shdr, end_system_routine_step_if_eq,
"                \n\
+(W)        or  (1|M0)               cr0.0<1>:ud
cr0.0<0;1,0>:ud 0x8000:ud    \n\
+(W)        and (1|M0)               cr0.1<1>:ud
cr0.1<0;1,0>:ud ARG(0):ud    \n\
+(W)        mov (16|M0)              r30.0<1>:ud
0x0:ud                \n\
+#if GEN_VER < 2000 // Media Block Write                            \n\
+        // Y offset of the block in rows := thread group id Y            \n\
+(W)        mov (1|M0)               r30.1<1>:ud
ARG(1):ud            \n\
+        // block width [0,63] representing 1 to 64 bytes, we want dword        \n\
+(W)        mov (1|M0)               r30.2<1>:ud
0x3:ud                \n\
+        // FFTID := FFTID from R0 header                    \n\
+(W)        mov (1|M0)               r30.4<1>:ud r0.5<0;1,0>:ud            \n\
+(W)        send.dc1 (16|M0)         r31     r30      null
0x0 0x2190000    \n\
+#else    // Typed 2D Block Store                                \n\
+        // Store X and Y block start (160:191 and
192:223)            \n\
+(W)        mov (2|M0)               r30.6<1>:ud
ARG(1):ud            \n\
+        // Store X and Y block size (224:231 and 232:239) \n\
+(W)        mov (1|M0)               r30.7<1>:ud
0x3:ud                \n\
+(W)        send.tgm (16|M0)         r31     r30    null:0
0x0 0x62100003    \n\

lib/gpgpu_shader.c +3 -0
+#endif                                            \n\
+        // clear the flag register                        \n\
+(W)        mov (1|M0)               f0.0<1>:ud

0x0:ud                \n\
+(W)        cmp (1|M0)    (ne)f0.0   null<1>:ud
r31.0<0;1,0>:ud ARG(2):ud    \n\
+(W&f0.0)    and (1|M0)              cr0.1<1>:ud cr0.1<0;1,0>:ud   ARG(3):ud    \n\
+        // return to an application                        \n\
+(W)        and (1|M0)               cr0.0<1>:ud
cr0.0<0;1,0>:ud 0x7FFFFFFD:ud    \n\
+    ", 0x807fffff, /* leave breakpoint exception */ +    y_offset, value, 0x7fffff /* clear all exceptions */ );

lib/gpgpu_shader.c +1 -0
+}
lib/gpgpu_shader.h +22 -2
index 255f93b4d..e4ca0be4c 100644
@@ -21,6 +21,7 @@ struct gpgpu_shader {
           uint32_t *code;
           uint32_t (*instr)[4];
       };
+    struct igt_map *labels;
   };
   struct iga64_template {
@@ -31,7 +32,7 @@ struct iga64_template {
   #pragma GCC diagnostic ignored "-Wnested-externs"
-void
+uint32_t
   __emit_iga64_code(struct gpgpu_shader *shdr, const struct
 iga64_template *tpls,
             int argc, uint32_t *argv);
@@ -56,8 +57,27 @@ void gpgpu_shader_exec(struct intel_bb *ibb,
                  struct gpgpu_shader *sip,
                  uint64_t ring, bool explicit_engine);
+void gpgpu_shader__wait(struct gpgpu_shader *shdr);
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr,
 uint32_t cmd_no);
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr);
+void gpgpu_shader__nop(struct gpgpu_shader *shdr);
   void gpgpu_shader__eot(struct gpgpu_shader *shdr);
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+                       uint32_t y_offset, const uint32_t value[4]);
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+                     uint32_t y_offset, uint32_t value);
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+                      bool breakpoint_suppress);
+void gpgpu_shader__end_system_routine_step_if_eq(struct
 gpgpu_shader *shdr,
+                         uint32_t dw_offset,
+                         uint32_t value);
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr,
 uint32_t y_offset);
   void gpgpu_shader__write_dword(struct gpgpu_shader *shdr,
 uint32_t value,
                      uint32_t y_offset);
-
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+                uint32_t dw_offset, uint32_t value);
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int

label_id, uint32_t iter);
  #endif /* GPGPU_SHADER_H */

lib/iga64_generated_codes.c +114 -1
index ea8d0f097..dd849eebc 100644
@@ -3,7 +3,7 @@
   #include "gpgpu_shader.h"
-#define MD5_SUM_IGA64_ASMS 9977ade854d57c5af5c5ca9e93c0f37e
+#define MD5_SUM_IGA64_ASMS 33b7cd843e3b009c123a85a6c520d7d0
   struct iga64_template const iga64_code_gpgpu_fill[] = {
       { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
@@ -79,6 +79,119 @@ struct iga64_template const
 iga64_code_gpgpu_fill[] = {
       }}
   };
+struct iga64_template const
 iga64_code_end_system_routine_step_if_eq[] = {
+    { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+        0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 52, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 48, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 52, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 48, .code = (const uint32_t []) {
+        0x80000166, 0x80018220, 0x02008000, 0x00008000,
+        0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80040161, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+        0x81000165, 0x80218220, 0x02008020, 0xc0ded003,
+        0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_end_system_routine[] = {
+    { .gen_ver = 2000, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 12, .code = (const uint32_t []) {
+        0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 12, .code = (const uint32_t []) {
+   ��    0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 12, .code = (const uint32_t []) {
+        0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+        0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_breakpoint_suppress[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000966, 0x80018220, 0x02008000, 0x00008000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000166, 0x80018220, 0x02008000, 0x00008000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
   struct iga64_template const iga64_code_media_block_write[] = {
       { .gen_ver = 2000, .size = 56, .code = (const uint32_t []) {

          0x80100061, 0x04054220, 0x00000000, 0x00000000,

lib/iga64_generated_codes.c +286 -0
@@ -164,6 +277,270 @@ struct iga64_template const
 iga64_code_media_block_write[] = {
       }}
   };
+struct iga64_template const iga64_code_media_block_write_aip[] = {
+    { .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008020, 0x00000000,
+        0x80000969, 0x02058220, 0x02000014, 0x00000002,
+        0x80000061, 0x02150220, 0x00000064, 0x00000000,
+        0x80001940, 0x02158220, 0x02000214, 0xc0ded000,
+        0x80100061, 0x04054220, 0x00000000, 0x00000000,
+        0x80041a61, 0x04550220, 0x00220205, 0x00000000,
+        0x80000061, 0x04754220, 0x00000000, 0x00000003,
+        0x80132031, 0x00000000, 0xd00e0494, 0x04000000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008040, 0x00000000,
+        0x80000969, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 40, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008020, 0x00000000,
+        0x80000969, 0x04058220, 0x02000014, 0x00000002,
+        0x80000061, 0x04150220, 0x00000064, 0x00000000,
+        0x80001940, 0x04158220, 0x02000414, 0xc0ded000,
+        0x80000061, 0x04254220, 0x00000000, 0x00000003,
+        0x80000061, 0x04450220, 0x00000054, 0x00000000,
+        0x80132031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 44, .code = (const uint32_t []) {
+        0x80000961, 0x05050220, 0x00008040, 0x00000000,
+        0x80000969, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 40, .code = (const uint32_t []) {
+        0x80000161, 0x05050220, 0x00008040, 0x00000000,
+        0x80000169, 0x04058220, 0x02000024, 0x00000002,
+        0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+        0x80000140, 0x04258220, 0x02000424, 0xc0ded000,
+        0x80000061, 0x04454220, 0x00000000, 0x00000003,
+        0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x00000000, 0xc0000414, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_common_target_write[] = {
+    { .gen_ver = 2000, .size = 48, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e754220, 0x00000000, 0x0000000f,
+        0x80132031, 0x00000000, 0xd00e1e94, 0x04000000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 56, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 52, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e254220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 56, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 52, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+        0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+        0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+        0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x00000000, 0xc0001e14, 0x02a00000,
+        0x80000001, 0x00010000, 0x20000000, 0x00000000,
+        0x80000001, 0x00010000, 0x30000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_inc_r40_jump_neq[] = {
+    { .gen_ver = 2000, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 20, .code = (const uint32_t []) {
+        0x80000040, 0x28058220, 0x02002804, 0x00000001,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80000270, 0x00018220, 0x22002804, 0xc0ded000,
+        0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_clear_r40[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000061, 0x28054220, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000061, 0x28054220, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_jump_dw_neq[] = {
+    { .gen_ver = 2000, .size = 32, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+        0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x84000020, 0x00004000, 0x00000000, 0xffffffa0,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1270, .size = 40, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1260, .size = 36, .code = (const uint32_t []) {
+        0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+        0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x84000020, 0x00004000, 0x00000000, 0xffffff90,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 1250, .size = 40, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80001901, 0x00010000, 0x00000000, 0x00000000,
+        0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 36, .code = (const uint32_t []) {
+        0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+        0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+        0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+        0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+        0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+        0x80000061, 0x30014220, 0x00000000, 0x00000000,
+        0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+        0x81000020, 0x00004000, 0x00000000, 0xffffff90,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_jump[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000020, 0x00004000, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000020, 0x00004000, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
   struct iga64_template const iga64_code_eot[] = {
       { .gen_ver = 2000, .size = 8, .code = (const uint32_t []) {
           0x800c0061, 0x70050220, 0x00460005, 0x00000000,
@@ -188,3 +565,25 @@ struct iga64_template const iga64_code_eot[] = {
           0x80049031, 0x00000004, 0x7020700c, 0x10000000,
       }}
   };
+
+struct iga64_template const iga64_code_nop[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x00000060, 0x00000000, 0x00000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x00000060, 0x00000000, 0x00000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
+
+struct iga64_template const iga64_code_sync_host[] = {
+    { .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+        0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+        0x80000901, 0x00010000, 0x00000000, 0x00000000,
+    }},
+    { .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+        0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+        0x80000101, 0x00010000, 0x00000000, 0x00000000,
+    }}
+};
Signature · 1 line
2.34.1
#9 lore raw

Re: [PATCH i-g-t v5 03/17] lib/gpgpu_shader: Extend shader building library

In reply to Manszewski, Christoph
Quoted 102 lines · Christoph

Hi Zbigniew,

Quoted 90 lines · Zbigniew Kempczyński
Quoted 89 lines · Christoph Manszewski

Add shader building functions and iga64 code used by eudebug subtests.

---
  lib/gpgpu_shader.c          | 392 ++++++++++++++++++++++++++++++++++-
  lib/gpgpu_shader.h          |  24 ++-
  lib/iga64_generated_codes.c | 401 +++++++++++++++++++++++++++++++++++-
  3 files changed, 813 insertions(+), 4 deletions(-)
lib/gpgpu_shader.c +36 -1
index 80bad342a..c723577e6 100644
@@ -7,10 +7,16 @@
  #include <i915_drm.h>
+#include "igt_map.h"
  #include "ioctl_wrappers.h"
  #include "gpgpu_shader.h"
  #include "gpu_cmds.h"
+struct label_entry {
+	uint32_t id;
+	uint32_t offset;
+};
+
  #define IGA64_ARG0 0xc0ded000
  #define IGA64_ARG_MASK 0xffffff00
@@ -32,7 +38,7 @@ static void gpgpu_shader_extend(struct gpgpu_shader *shdr)
  	igt_assert(shdr->code);
  }
-void
+uint32_t
  __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
  		  int argc, uint32_t *argv)
  {
@@ -60,6 +66,8 @@ __emit_iga64_code(struct gpgpu_shader *shdr, struct iga64_template const *tpls,
  	}
  	shdr->size += tpls->size;
+
+	return tpls->size;
  }
  static uint32_t fill_sip(struct intel_bb *ibb,
@@ -235,10 +243,16 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
  	shdr->gen_ver = 100 * info->graphics_ver + info->graphics_rel;
  	shdr->max_size = 16 * 4;
  	shdr->code = malloc(4 * shdr->max_size);
+	shdr->labels = igt_map_create(igt_map_hash_32, igt_map_equal_32);
  	igt_assert(shdr->code);
  	return shdr;
  }
+static void free_func(struct igt_map_entry *entry)
+{
+       free(entry->data);
+}
+
  /**
   * gpgpu_shader_destroy:
   * @shdr: pointer to shader struct created with 'gpgpu_shader_create'
@@ -247,10 +261,76 @@ struct gpgpu_shader *gpgpu_shader_create(int fd)
   */
  void gpgpu_shader_destroy(struct gpgpu_shader *shdr)
  {
+	igt_map_destroy(shdr->labels, free_func);
  	free(shdr->code);
  	free(shdr);
  }
+/**
+ * gpgpu_shader_dump:
+ * @shdr: shader to be printed
+ *
+ * Print shader instructions from @shdr in hex.
+ */
+void gpgpu_shader_dump(struct gpgpu_shader *shdr)
+{
+	for (int i = 0; i < shdr->size / 4; i++)
+		igt_info("0x%08x 0x%08x 0x%08x 0x%08x\n",
+			 shdr->instr[i][0], shdr->instr[i][1],
+			 shdr->instr[i][2], shdr->instr[i][3]);
+}
+
+/**
+ * gpgpu_shader__breakpoint_on:
+ * @shdr: shader to create breakpoint in
+ * @cmd_no: index of the instruction to break on
+ *
+ * Insert a breakpoint on the @cmd_no'th instruction within @shdr.
Quoted 8 lines

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 292

Quoted 4 lines
lib/gpgpu_shader.c +4 -0
+ */
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t cmd_no)
+{
+	igt_assert(cmd_no < shdr->size / 4);

If I'm not wrong shdr->size contains number of dwords, not size in bytes. It is imo a little bit confusing and I would suggest to rename it to avoid confusion. Same with max_size.

Quoted 4 lines

It does, and I agree that changing it to something like 'dw_size' would get rid of the confusion, since by default I would also assume that a naked 'size' means bytes. However instructions are 4 dwords, so the division by 4 is relevant here.

Eh, right. Commands are 4 dwords, so division is necessary. Forget about my previous comment.

--
Zbigniew

Quoted 855 lines
Quoted 17 lines
Quoted 5 lines
lib/gpgpu_shader.c +2 -0
+	shdr->instr[cmd_no][0] |= 1<<30;
+}

lib/gpgpu_shader.c struct gpgpu_shader *gpgpu_shader_create(int fd) · line 304

Quoted 10 lines
lib/gpgpu_shader.c +10 -0
+
+/**
+ * gpgpu_shader__breakpoint:
+ * @shdr: shader to create breakpoint in
+ *
+ * Insert a breakpoint on the last instruction in @shdr.
+ */
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr)
+{
+	gpgpu_shader__breakpoint_on(shdr, shdr->size / 4 - 1);

Likely shdr->size - 1 only.

Like above - instructions are 4 dwords.

Thanks,
Christoph

Quoted 831 lines

Below code was commented in previous review.

--
Zbigniew

Quoted 825 lines
lib/gpgpu_shader.c +339 -0
+}
+
+/**
+ * gpgpu_shader__wait:
+ * @shdr: shader to be modified
+ *
+ * Append wait instruction to @shader. This instruction raises attention
+ * and stops execution.
+ */
+void gpgpu_shader__wait(struct gpgpu_shader *shdr)
+{
+	emit_iga64_code(shdr, sync_host, "	\n\
+(W)	sync.host	        null		\n\
+	");
+}
+
+/**
+ * gpgpu_shader__nop:
+ * @shdr: shader to be modified
+ *
+ * Append a no-op instruction to @shdr.
+ */
+void gpgpu_shader__nop(struct gpgpu_shader *shdr)
+{
+	emit_iga64_code(shdr, nop, "	\n\
+(W)	nop				\n\
+	");
+}
+
  /**
   * gpgpu_shader__eot:
   * @shdr: shader to be modified
@@ -269,6 +349,246 @@ void gpgpu_shader__eot(struct gpgpu_shader *shdr)
  	");
  }
+/**
+ * gpgpu_shader__label:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Create a label for the last instruction within @shdr.
+ */
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id)
+{
+	struct label_entry *l = malloc(sizeof(*l));
+
+	l->id = label_id;
+	l->offset = shdr->size;
+	igt_map_insert(shdr->labels, &l->id, l);
+}
+
+#define OPCODE(x) (x & 0x7f)
+#define OPCODE_JUMP_INDEXED 0x20
+static void __patch_indexed_jump(struct gpgpu_shader *shdr, int label_id,
+				 uint32_t jump_iga64_size)
+{
+	struct label_entry *l;
+	uint32_t *start, *end, *label;
+	int32_t relative;
+
+	l = igt_map_search(shdr->labels, &label_id);
+	igt_assert(l);
+
+	igt_assert(jump_iga64_size % 4 == 0);
+
+	label = shdr->code + l->offset;
+	end = shdr->code + shdr->size;
+	start = end - jump_iga64_size;
+
+	for (; start < end; start += 4)
+		if (OPCODE(*start) == OPCODE_JUMP_INDEXED) {
+			relative = (label - start) * 4;
+			*(start + 3) = relative;
+			break;
+		}
+}
+
+/**
+ * gpgpu_shader__jump:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label @label_id.
+ */
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id)
+{
+	size_t shader_size;
+
+	shader_size = emit_iga64_code(shdr, jump, "	\n\
+L0:							\n\
+(W)	jmpi        L0					\n\
+	");
+
+	__patch_indexed_jump(shdr, label_id, shader_size);
+}
+
+/**
+ * gpgpu_shader__jump_neq:
+ * @shdr: shader to be modified
+ * @label_id: label to jump to
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value
+ *
+ * Append jump instruction to @shdr. Jump to instruction with label @label_id
+ * when @value is not equal to dword stored at @y_offset within the surface.
+ */
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+			    uint32_t y_offset, uint32_t value)
+{
+	uint32_t size;
+
+	size = emit_iga64_code(shdr, jump_dw_neq, "					\n\
+L0:											\n\
+(W)		mov (16|M0)              r30.0<1>:ud    0x0:ud				\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+	// Y offset of the block in rows := thread group id Y				\n\
+(W)		mov (1|M0)               r30.1<1>:ud    ARG(0):ud			\n\
+	// block width [0,63] representing 1 to 64 bytes, we want dword			\n\
+(W)		mov (1|M0)               r30.2<1>:ud    0x3:ud				\n\
+	// FFTID := FFTID from R0 header						\n\
+(W)		mov (1|M0)               r30.4<1>:ud    r0.5<0;1,0>:ud  		\n\
+(W)		send.dc1 (16|M0)         r31     r30      null    0x0	0x2190000	\n\
+#else // Typed 2D Block Store								\n\
+	// Store X and Y block start (160:191 and 192:223)				\n\
+(W)            mov (2|M0)               r30.6<1>:ud    ARG(0):ud			\n\
+	// Store X and Y block size (224:231 and 232:239)				\n\
+(W)            mov (1|M0)               r30.7<1>:ud    0x3:ud				\n\
+(W)            send.tgm (16|M0)         r31     r30    null:0    0x0    0x62100003	\n\
+#endif											\n\
+	// clear the flag register							\n\
+(W)		mov (1|M0)               f0.0<1>:ud    0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud   ARG(1):ud	\n\
+(W&f0.0)	jmpi                     L0						\n\
+	", y_offset, value);
+
+	__patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__loop_begin:
+ * @shdr: shader to be modified
+ * @label_id: id of the label to be created
+ *
+ * Begin a counting loop in @shdr. All subsequent instructions will constitute
+ * the loop body up until 'gpgpu_shader__loop_end' gets called. The first
+ * instruction of the loop will be at label @label_id. The r40 register will be
+ * overwritten as it is used as the loop counter.
+ */
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id)
+{
+	emit_iga64_code(shdr, clear_r40, "		\n\
+L0:							\n\
+(W)	mov (1|M0)               r40:ud    0x0:ud	\n\
+	");
+
+	gpgpu_shader__label(shdr, label_id);
+}
+
+/**
+ * gpgpu_shader__loop_end:
+ * @shdr: shader to be modified
+ * @label_id: label id passed to 'gpgpu_shader__loop_begin'
+ * @iter: iteration count
+ *
+ * End loop body in @shdr.
+ */
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, uint32_t iter)
+{
+	uint32_t size;
+
+	size = emit_iga64_code(shdr, inc_r40_jump_neq, "				\n\
+L0:											\n\
+(W)		add (1|M0)              r40:ud          r40.0<0;1,0>:ud 0x1:ud		\n\
+(W)		mov (1|M0)              f0.0<1>:ud      0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r40.0<0;1,0>:ud   ARG(0):ud	\n\
+(W&f0.0)	jmpi                     L0						\n\
+	", iter);
+
+	__patch_indexed_jump(shdr, label_id, size);
+}
+
+/**
+ * gpgpu_shader__common_target_write:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: oword to be written
+ *
+ * Write the oword stored in @value to the target buffer at @y_offset.
+ */
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+				       uint32_t y_offset, const uint32_t value[4])
+{
+	emit_iga64_code(shdr, common_target_write, "				\n\
+(W)	mov (16|M0)		r30.0<1>:ud	0x0:ud				\n\
+(W)	mov (16|M0)		r31.0<1>:ud	0x0:ud				\n\
+(W)	mov (1|M0)		r31.0<1>:ud	ARG(1):ud			\n\
+(W)	mov (1|M0)		r31.1<1>:ud	ARG(2):ud			\n\
+(W)	mov (1|M0)		r31.2<1>:ud	ARG(3):ud			\n\
+(W)	mov (1|M0)		r31.3<1>:ud	ARG(4):ud			\n\
+#if GEN_VER < 2000 // Media Block Write						\n\
+	// Y offset of the block in rows					\n\
+(W)	mov (1|M0)		r30.1<1>:ud	ARG(0):ud			\n\
+	// block width [0,63] representing 1 to 64 bytes			\n\
+(W)	mov (1|M0)		r30.2<1>:ud	0xf:ud				\n\
+	// FFTID := FFTID from R0 header					\n\
+(W)	mov (1|M0)		r30.4<1>:ud	r0.5<0;1,0>:ud			\n\
+	// written value							\n\
+(W)	send.dc1 (16|M0)	null	r30	src1_null  0x0	0x40A8000	\n\
+#else	// Typed 2D Block Store							\n\
+	// Store X and Y block start (160:191 and 192:223)			\n\
+(W)	mov (2|M0)              r30.6<1>:ud     ARG(0):ud			\n\
+	// Store X and Y block size (224:231 and 232:239)			\n\
+(W)	mov (1|M0)              r30.7<1>:ud     0xf:ud				\n\
+(W)	send.tgm (16|M0)        null    r30     null:0  0x0     0x64000007	\n\
+#endif										\n\
+	", y_offset, value[0], value[1], value[2], value[3]);
+}
+
+/**
+ * gpgpu_shader__common_target_write_u32:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within target buffer in rows
+ * @value: dword to be written
+ *
+ * Fill oword at @y_offset with dword stored in @value.
+ */
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+					   uint32_t y_offset, uint32_t value)
+{
+	const uint32_t owblock[4] = {
+		value, value, value, value
+	};
+	gpgpu_shader__common_target_write(shdr, y_offset, owblock);
+}
+
+/**
+ * gpgpu_shader__write_aip:
+ * @shdr: shader to be modified
+ * @y_offset: write target offset within the surface in rows
+ *
+ * Write address instruction pointer to row tg_id_y + @y_offset.
+ */
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset)
+{
+	emit_iga64_code(shdr, media_block_write_aip, "				\n\
+	// Payload								\n\
+(W)	mov (1|M0)               r5.0<1>:ud    cr0.2:ud				\n\
+#if GEN_VER < 2000 // Media Block Write						\n\
+	// X offset of the block in bytes := (thread group id X << ARG(0))	\n\
+(W)	shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud    0x2:ud		\n\
+	// Y offset of the block in rows := thread group id Y			\n\
+(W)	mov (1|M0)               r4.1<1>:ud    r0.6<0;1,0>:ud			\n\
+(W)	add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud    ARG(0):ud	\n\
+	// block width [0,63] representing 1 to 64 bytes			\n\
+(W)	mov (1|M0)               r4.2<1>:ud    0x3:ud				\n\
+	// FFTID := FFTID from R0 header					\n\
+(W)	mov (1|M0)               r4.4<1>:ud    r0.5<0;1,0>:ud			\n\
+(W)	send.dc1 (16|M0)         null     r4   src1_null 0       0x40A8000	\n\
+#else // Typed 2D Block Store							\n\
+	// Load r2.0-3 with tg id X << ARG(0)					\n\
+(W)	shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud    0x2:ud		\n\
+	// Load r2.4-7 with tg id Y + ARG(1):ud					\n\
+(W)	mov (1|M0)               r2.1<1>:ud    r0.6<0;1,0>:ud			\n\
+(W)	add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud    ARG(0):ud	\n\
+	// payload setup							\n\
+(W)	mov (16|M0)              r4.0<1>:ud    0x0:ud				\n\
+	// Store X and Y block start (160:191 and 192:223)			\n\
+(W)	mov (2|M0)               r4.5<1>:ud    r2.0<2;2,1>:ud			\n\
+	// Store X and Y block max_size (224:231 and 232:239)			\n\
+(W)	mov (1|M0)               r4.7<1>:ud    0x3:ud				\n\
+(W)	send.tgm (16|M0)         null     r4   null:0    0    0x64000007	\n\
+#endif										\n\
+	", y_offset);
+}
+
  /**
   * gpgpu_shader__write_dword:
   * @shdr: shader to be modified
@@ -313,3 +633,73 @@ void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
  #endif										\n\
  	", 2, y_offset, 3, value, value, value, value);
  }
+
+/**
+ * gpgpu_shader__end_system_routine:
+ * @shdr: shader to be modified
+ * @breakpoint_suppress: breakpoint suppress flag
+ *
+ * Return from system routine. To prevent infinite jumping to the system
+ * routine on a breakpoint, @breakpoint_suppress flag has to be set.
+ */
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+				      bool breakpoint_suppress)
+{
+	/*
+	 * set breakpoint suppress bit to avoid an endless loop
+	 * when sip was invoked by a breakpoint
+	 */
+	if (breakpoint_suppress)
+		emit_iga64_code(shdr, breakpoint_suppress, "			\n\
+(W)	or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x8000:ud	\n\
+		");
+
+	emit_iga64_code(shdr, end_system_routine, "				\n\
+(W)	and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   ARG(0):ud	\n\
+	// return to an application						\n\
+(W)	and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x7FFFFFFD:ud	\n\
+	", 0x7fffff | (1 << 26)); /* clear all exceptions, except read only bit */
+}
+
+/**
+ * gpgpu_shader__end_system_routine_step_if_eq:
+ * @shdr: shader to be modified
+ * @y_offset: offset within target buffer in rows
+ * @value: expected value for single stepping execution
+ *
+ * Return from system routine. Don't clear breakpoint exception when @value
+ * is equal to value stored at @y_offset. This triggers the system routine
+ * after the subsequent instruction, resulting in single stepping execution.
+ */
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+						 uint32_t y_offset,
+						 uint32_t value)
+{
+	emit_iga64_code(shdr, end_system_routine_step_if_eq, "				\n\
+(W)		or  (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x8000:ud	\n\
+(W)		and (1|M0)               cr0.1<1>:ud   cr0.1<0;1,0>:ud   ARG(0):ud	\n\
+(W)		mov (16|M0)              r30.0<1>:ud    0x0:ud				\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+		// Y offset of the block in rows := thread group id Y			\n\
+(W)		mov (1|M0)               r30.1<1>:ud    ARG(1):ud			\n\
+		// block width [0,63] representing 1 to 64 bytes, we want dword		\n\
+(W)		mov (1|M0)               r30.2<1>:ud    0x3:ud				\n\
+		// FFTID := FFTID from R0 header					\n\
+(W)		mov (1|M0)               r30.4<1>:ud    r0.5<0;1,0>:ud			\n\
+(W)		send.dc1 (16|M0)         r31     r30      null    0x0	0x2190000	\n\
+#else	// Typed 2D Block Store								\n\
+		// Store X and Y block start (160:191 and 192:223)			\n\
+(W)		mov (2|M0)               r30.6<1>:ud    ARG(1):ud			\n\
+		// Store X and Y block size (224:231 and 232:239)			\n\
+(W)		mov (1|M0)               r30.7<1>:ud    0x3:ud				\n\
+(W)		send.tgm (16|M0)         r31     r30    null:0    0x0    0x62100003	\n\
+#endif											\n\
+		// clear the flag register						\n\
+(W)		mov (1|M0)               f0.0<1>:ud    0x0:ud				\n\
+(W)		cmp (1|M0)    (ne)f0.0   null<1>:ud     r31.0<0;1,0>:ud   ARG(2):ud	\n\
+(W&f0.0)	and (1|M0)              cr0.1<1>:ud     cr0.1<0;1,0>:ud   ARG(3):ud	\n\
+		// return to an application						\n\
+(W)		and (1|M0)               cr0.0<1>:ud   cr0.0<0;1,0>:ud   0x7FFFFFFD:ud	\n\
+	", 0x807fffff, /* leave breakpoint exception */
+	y_offset, value, 0x7fffff /* clear all exceptions */ );
+}
lib/gpgpu_shader.h +22 -2
index 255f93b4d..e4ca0be4c 100644
@@ -21,6 +21,7 @@ struct gpgpu_shader {
  		uint32_t *code;
  		uint32_t (*instr)[4];
  	};
+	struct igt_map *labels;
  };
  struct iga64_template {
@@ -31,7 +32,7 @@ struct iga64_template {
  #pragma GCC diagnostic ignored "-Wnested-externs"
-void
+uint32_t
  __emit_iga64_code(struct gpgpu_shader *shdr, const struct iga64_template *tpls,
  		  int argc, uint32_t *argv);
@@ -56,8 +57,27 @@ void gpgpu_shader_exec(struct intel_bb *ibb,
  		       struct gpgpu_shader *sip,
  		       uint64_t ring, bool explicit_engine);
+void gpgpu_shader__wait(struct gpgpu_shader *shdr);
+void gpgpu_shader__breakpoint_on(struct gpgpu_shader *shdr, uint32_t cmd_no);
+void gpgpu_shader__breakpoint(struct gpgpu_shader *shdr);
+void gpgpu_shader__nop(struct gpgpu_shader *shdr);
  void gpgpu_shader__eot(struct gpgpu_shader *shdr);
+void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
+				       uint32_t y_offset, const uint32_t value[4]);
+void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
+				     uint32_t y_offset, uint32_t value);
+void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
+				      bool breakpoint_suppress);
+void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
+						 uint32_t dw_offset,
+						 uint32_t value);
+void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset);
  void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
  			       uint32_t y_offset);
-
+void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
+			    uint32_t dw_offset, uint32_t value);
+void gpgpu_shader__loop_begin(struct gpgpu_shader *shdr, int label_id);
+void gpgpu_shader__loop_end(struct gpgpu_shader *shdr, int label_id, uint32_t iter);
  #endif /* GPGPU_SHADER_H */
lib/iga64_generated_codes.c +400 -1
index ea8d0f097..dd849eebc 100644
@@ -3,7 +3,7 @@
  #include "gpgpu_shader.h"
-#define MD5_SUM_IGA64_ASMS 9977ade854d57c5af5c5ca9e93c0f37e
+#define MD5_SUM_IGA64_ASMS 33b7cd843e3b009c123a85a6c520d7d0
  struct iga64_template const iga64_code_gpgpu_fill[] = {
  	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
@@ -79,6 +79,119 @@ struct iga64_template const iga64_code_gpgpu_fill[] = {
  	}}
  };
+struct iga64_template const iga64_code_end_system_routine_step_if_eq[] = {
+	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+		0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 52, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 48, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80100961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x84000965, 0x80118220, 0x02008010, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 52, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040961, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000965, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 48, .code = (const uint32_t []) {
+		0x80000166, 0x80018220, 0x02008000, 0x00008000,
+		0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80040161, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded002,
+		0x81000165, 0x80218220, 0x02008020, 0xc0ded003,
+		0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_end_system_routine[] = {
+	{ .gen_ver = 2000, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80118220, 0x02008010, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 12, .code = (const uint32_t []) {
+		0x80000965, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000965, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 12, .code = (const uint32_t []) {
+		0x80000165, 0x80218220, 0x02008020, 0xc0ded000,
+		0x80000165, 0x80018220, 0x02008000, 0x7ffffffd,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_breakpoint_suppress[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000966, 0x80018220, 0x02008000, 0x00008000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000166, 0x80018220, 0x02008000, 0x00008000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
  struct iga64_template const iga64_code_media_block_write[] = {
  	{ .gen_ver = 2000, .size = 56, .code = (const uint32_t []) {
  		0x80100061, 0x04054220, 0x00000000, 0x00000000,
@@ -164,6 +277,270 @@ struct iga64_template const iga64_code_media_block_write[] = {
  	}}
  };
+struct iga64_template const iga64_code_media_block_write_aip[] = {
+	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008020, 0x00000000,
+		0x80000969, 0x02058220, 0x02000014, 0x00000002,
+		0x80000061, 0x02150220, 0x00000064, 0x00000000,
+		0x80001940, 0x02158220, 0x02000214, 0xc0ded000,
+		0x80100061, 0x04054220, 0x00000000, 0x00000000,
+		0x80041a61, 0x04550220, 0x00220205, 0x00000000,
+		0x80000061, 0x04754220, 0x00000000, 0x00000003,
+		0x80132031, 0x00000000, 0xd00e0494, 0x04000000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008040, 0x00000000,
+		0x80000969, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 40, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008020, 0x00000000,
+		0x80000969, 0x04058220, 0x02000014, 0x00000002,
+		0x80000061, 0x04150220, 0x00000064, 0x00000000,
+		0x80001940, 0x04158220, 0x02000414, 0xc0ded000,
+		0x80000061, 0x04254220, 0x00000000, 0x00000003,
+		0x80000061, 0x04450220, 0x00000054, 0x00000000,
+		0x80132031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 44, .code = (const uint32_t []) {
+		0x80000961, 0x05050220, 0x00008040, 0x00000000,
+		0x80000969, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 40, .code = (const uint32_t []) {
+		0x80000161, 0x05050220, 0x00008040, 0x00000000,
+		0x80000169, 0x04058220, 0x02000024, 0x00000002,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80000140, 0x04258220, 0x02000424, 0xc0ded000,
+		0x80000061, 0x04454220, 0x00000000, 0x00000003,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_common_target_write[] = {
+	{ .gen_ver = 2000, .size = 48, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e754220, 0x00000000, 0x0000000f,
+		0x80132031, 0x00000000, 0xd00e1e94, 0x04000000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 56, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 52, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80100061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f154220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f354220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e254220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 56, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 52, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1f054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1f054220, 0x00000000, 0xc0ded001,
+		0x80000061, 0x1f254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x1f454220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x1f654220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x0000000f,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x00000000, 0xc0001e14, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_inc_r40_jump_neq[] = {
+	{ .gen_ver = 2000, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 20, .code = (const uint32_t []) {
+		0x80000040, 0x28058220, 0x02002804, 0x00000001,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80000270, 0x00018220, 0x22002804, 0xc0ded000,
+		0x81000020, 0x00004000, 0x00000000, 0xffffffd0,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_clear_r40[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000061, 0x28054220, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000061, 0x28054220, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_jump_dw_neq[] = {
+	{ .gen_ver = 2000, .size = 32, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80040061, 0x1e654220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e754220, 0x00000000, 0x00000003,
+		0x80132031, 0x1f0c0000, 0xd0061e8c, 0x04000000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x84000020, 0x00004000, 0x00000000, 0xffffffa0,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 40, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 36, .code = (const uint32_t []) {
+		0x80100061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e154220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e254220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e450220, 0x00000054, 0x00000000,
+		0x80132031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80008070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x84000020, 0x00004000, 0x00000000, 0xffffff90,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 40, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80001901, 0x00010000, 0x00000000, 0x00000000,
+		0x80044031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff80,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 36, .code = (const uint32_t []) {
+		0x80040061, 0x1e054220, 0x00000000, 0x00000000,
+		0x80000061, 0x1e254220, 0x00000000, 0xc0ded000,
+		0x80000061, 0x1e454220, 0x00000000, 0x00000003,
+		0x80000061, 0x1e850220, 0x000000a4, 0x00000000,
+		0x80049031, 0x1f0c0000, 0xc0001e0c, 0x02400000,
+		0x80000061, 0x30014220, 0x00000000, 0x00000000,
+		0x80002070, 0x00018220, 0x22001f04, 0xc0ded001,
+		0x81000020, 0x00004000, 0x00000000, 0xffffff90,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_jump[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000020, 0x00004000, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000020, 0x00004000, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
  struct iga64_template const iga64_code_eot[] = {
  	{ .gen_ver = 2000, .size = 8, .code = (const uint32_t []) {
  		0x800c0061, 0x70050220, 0x00460005, 0x00000000,
@@ -188,3 +565,25 @@ struct iga64_template const iga64_code_eot[] = {
  		0x80049031, 0x00000004, 0x7020700c, 0x10000000,
  	}}
  };
+
+struct iga64_template const iga64_code_nop[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x00000060, 0x00000000, 0x00000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x00000060, 0x00000000, 0x00000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
+struct iga64_template const iga64_code_sync_host[] = {
+	{ .gen_ver = 1250, .size = 8, .code = (const uint32_t []) {
+		0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 8, .code = (const uint32_t []) {
+		0x80000001, 0x00010000, 0xf0000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
Signature · 1 line
2.34.1
#10 lore raw

PATCH v5 4/17 lib/gpgpu_shader: Add write_on_exception template

In reply to Christoph Manszewski

From: Andrzej Hajda <[email protected]>

Writing specific value to memory location on unexpected value in exception register allows to report errors from inside shader or siplet.

---
 lib/gpgpu_shader.c          |  56 ++++++++++++++++++++
 lib/gpgpu_shader.h          |   2 +
 lib/iga64_generated_codes.c | 102 +++++++++++++++++++++++++++++++++++-
 3 files changed, 159 insertions(+), 1 deletion(-)
lib/gpgpu_shader.c +56 -0
index c723577e6..4097857f2 100644
@@ -634,6 +634,62 @@ void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 	", 2, y_offset, 3, value, value, value, value);
 }
 
+/**
+ * gpgpu_shader__write_on_exception:
+ * @shdr: shader to be modified
+ * @value: dword to be written
+ * @y_offset: write target offset within the surface in rows
+ * @mask: mask to be applied on exception register
+ * @expected: expected value of exception register with @mask applied
+ *
+ * Check if bits specified by @mask in exception register(cr0.1) are equal
+ * to provided ones: cr0.1 & @mask == @expected,
+ * if yes fill dword in (row, column/dword) == (tg_id_y + @y_offset, tg_id_x).
+ */
+void gpgpu_shader__write_on_exception(struct gpgpu_shader *shdr, uint32_t value,
+				      uint32_t y_offset, uint32_t mask, uint32_t expected)
+{
+	emit_iga64_code(shdr, write_on_exception, "					\n\
+	// Clear message header								\n\
+(W)	mov (16|M0)              r4.0<1>:ud    0x0:ud					\n\
+	// Payload									\n\
+(W)	mov (1|M0)               r5.0<1>:ud    ARG(3):ud				\n\
+(W)	mov (1|M0)               r5.1<1>:ud    ARG(4):ud				\n\
+(W)	mov (1|M0)               r5.2<1>:ud    ARG(5):ud				\n\
+(W)	mov (1|M0)               r5.3<1>:ud    ARG(6):ud				\n\
+#if GEN_VER < 2000 // prepare Media Block Write						\n\
+	// X offset of the block in bytes := (thread group id X << ARG(0))		\n\
+(W)	shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud    ARG(0):ud		\n\
+	// Y offset of the block in rows := thread group id Y				\n\
+(W)	mov (1|M0)               r4.1<1>:ud    r0.6<0;1,0>:ud				\n\
+(W)	add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud   ARG(1):ud		\n\
+	// block width [0,63] representing 1 to 64 bytes				\n\
+(W)	mov (1|M0)               r4.2<1>:ud    ARG(2):ud				\n\
+	// FFTID := FFTID from R0 header						\n\
+(W)	mov (1|M0)               r4.4<1>:ud    r0.5<0;1,0>:ud				\n\
+#else // prepare Typed 2D Block Store							\n\
+	// Load r2.0-3 with tg id X << ARG(0)						\n\
+(W)	shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud    ARG(0):ud		\n\
+	// Load r2.4-7 with tg id Y + ARG(1):ud						\n\
+(W)	mov (1|M0)               r2.1<1>:ud    r0.6<0;1,0>:ud				\n\
+(W)	add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud    ARG(1):ud		\n\
+	// Store X and Y block start (160:191 and 192:223)				\n\
+(W)	mov (2|M0)               r4.5<1>:ud    r2.0<2;2,1>:ud				\n\
+	// Store X and Y block max_size (224:231 and 232:239)				\n\
+(W)	mov (1|M0)               r4.7<1>:ud    ARG(2):ud				\n\
+#endif											\n\
+	// Check if masked exception is equal to provided value and write conditionally \n\
+(W)      and (1|M0)              r3.0<1>:ud     cr0.1<0;1,0>:ud ARG(7):ud		\n\
+(W)      mov (1|M0)              f0.0<1>:ud     0x0:ud					\n\
+(W)      cmp (1|M0)     (eq)f0.0 null:ud        r3.0<0;1,0>:ud  ARG(8):ud		\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+(W&f0.0) send.dc1 (16|M0)        null     r4   src1_null 0    0x40A8000			\n\
+#else // Typed 2D Block Store								\n\
+(W&f0.0) send.tgm (16|M0)        null     r4   null:0    0    0x64000007		\n\
+#endif											\n\
+	", 2, y_offset, 3, value, value, value, value, mask, expected);
+}
+
 /**
  * gpgpu_shader__end_system_routine:
  * @shdr: shader to be modified
lib/gpgpu_shader.h +2 -0
index e4ca0be4c..76ff4989e 100644
@@ -74,6 +74,8 @@ void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
 void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset);
 void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 			       uint32_t y_offset);
+void gpgpu_shader__write_on_exception(struct gpgpu_shader *shdr, uint32_t dw,
+			       uint32_t y_offset, uint32_t mask, uint32_t value);
 void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
 void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
 void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
lib/iga64_generated_codes.c +101 -1

114 lines not on this page · raw message

Signature · 1 line
2.34.1
#11 lore raw

Re: [PATCH i-g-t v5 04/17] lib/gpgpu_shader: Add write_on_exception template

Quoted 33 lines · Christoph Manszewski

From: Andrzej Hajda <[email protected]>

Writing specific value to memory location on unexpected value in exception register allows to report errors from inside shader or siplet.

---
 lib/gpgpu_shader.c          |  56 ++++++++++++++++++++
 lib/gpgpu_shader.h          |   2 +
 lib/iga64_generated_codes.c | 102 +++++++++++++++++++++++++++++++++++-
 3 files changed, 159 insertions(+), 1 deletion(-)
lib/gpgpu_shader.c +13 -0
index c723577e6..4097857f2 100644
@@ -634,6 +634,62 @@ void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 	", 2, y_offset, 3, value, value, value, value);
 }
 
+/**
+ * gpgpu_shader__write_on_exception:
+ * @shdr: shader to be modified
+ * @value: dword to be written
+ * @y_offset: write target offset within the surface in rows
+ * @mask: mask to be applied on exception register
+ * @expected: expected value of exception register with @mask applied
+ *
+ * Check if bits specified by @mask in exception register(cr0.1) are equal
+ * to provided ones: cr0.1 & @mask == @expected,
+ * if yes fill dword in (row, column/dword) == (tg_id_y + @y_offset, tg_id_x).
+ */
+void gpgpu_shader__write_on_exception(struct gpgpu_shader *shdr, uint32_t value,

lib/gpgpu_shader.c void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value, · line 659

Quoted 10 lines
lib/gpgpu_shader.c +10 -0
+				      uint32_t y_offset, uint32_t mask, uint32_t expected)
+{
+	emit_iga64_code(shdr, write_on_exception, "					\n\
+	// Clear message header								\n\
+(W)	mov (16|M0)              r4.0<1>:ud    0x0:ud					\n\
+	// Payload									\n\
+(W)	mov (1|M0)               r5.0<1>:ud    ARG(3):ud				\n\
+(W)	mov (1|M0)               r5.1<1>:ud    ARG(4):ud				\n\
+(W)	mov (1|M0)               r5.2<1>:ud    ARG(5):ud				\n\
+(W)	mov (1|M0)               r5.3<1>:ud    ARG(6):ud				\n\

ARG(4)-ARG(6) are unused and confusing here. Please get rid of them.

Quoted 21 lines
lib/gpgpu_shader.c +21 -0
+#if GEN_VER < 2000 // prepare Media Block Write						\n\
+	// X offset of the block in bytes := (thread group id X << ARG(0))		\n\
+(W)	shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud    ARG(0):ud		\n\
+	// Y offset of the block in rows := thread group id Y				\n\
+(W)	mov (1|M0)               r4.1<1>:ud    r0.6<0;1,0>:ud				\n\
+(W)	add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud   ARG(1):ud		\n\
+	// block width [0,63] representing 1 to 64 bytes				\n\
+(W)	mov (1|M0)               r4.2<1>:ud    ARG(2):ud				\n\
+	// FFTID := FFTID from R0 header						\n\
+(W)	mov (1|M0)               r4.4<1>:ud    r0.5<0;1,0>:ud				\n\
+#else // prepare Typed 2D Block Store							\n\
+	// Load r2.0-3 with tg id X << ARG(0)						\n\
+(W)	shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud    ARG(0):ud		\n\
+	// Load r2.4-7 with tg id Y + ARG(1):ud						\n\
+(W)	mov (1|M0)               r2.1<1>:ud    r0.6<0;1,0>:ud				\n\
+(W)	add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud    ARG(1):ud		\n\
+	// Store X and Y block start (160:191 and 192:223)				\n\
+(W)	mov (2|M0)               r4.5<1>:ud    r2.0<2;2,1>:ud				\n\
+	// Store X and Y block max_size (224:231 and 232:239)				\n\
+(W)	mov (1|M0)               r4.7<1>:ud    ARG(2):ud				\n\
+#endif											\n\

lib/gpgpu_shader.c void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value, · line 690

Quoted 10 lines
lib/gpgpu_shader.c +10 -0
+	// Check if masked exception is equal to provided value and write conditionally \n\
+(W)      and (1|M0)              r3.0<1>:ud     cr0.1<0;1,0>:ud ARG(7):ud		\n\
+(W)      mov (1|M0)              f0.0<1>:ud     0x0:ud					\n\
+(W)      cmp (1|M0)     (eq)f0.0 null:ud        r3.0<0;1,0>:ud  ARG(8):ud		\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+(W&f0.0) send.dc1 (16|M0)        null     r4   src1_null 0    0x40A8000			\n\
+#else // Typed 2D Block Store								\n\
+(W&f0.0) send.tgm (16|M0)        null     r4   null:0    0    0x64000007		\n\
+#endif											\n\
+	", 2, y_offset, 3, value, value, value, value, mask, expected);

I thought first and third argument might be directly entered in the code but after replace there's harder to find where they are used. I mean I looked for ARG(0) to find the place where 2 lands. So I suggest just to remove 'value' duplicates for ARG(4)-ARG(6) only.

--
Zbigniew

Quoted 141 lines
lib/gpgpu_shader.c +2 -0
+}
+
 /**
  * gpgpu_shader__end_system_routine:
  * @shdr: shader to be modified
lib/gpgpu_shader.h +2 -0
index e4ca0be4c..76ff4989e 100644
@@ -74,6 +74,8 @@ void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
 void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset);
 void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 			       uint32_t y_offset);
+void gpgpu_shader__write_on_exception(struct gpgpu_shader *shdr, uint32_t dw,
+			       uint32_t y_offset, uint32_t mask, uint32_t value);
 void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
 void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
 void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
lib/iga64_generated_codes.c +101 -1
index dd849eebc..c084ddcd7 100644
@@ -3,7 +3,7 @@
 
 #include "gpgpu_shader.h"
 
-#define MD5_SUM_IGA64_ASMS 33b7cd843e3b009c123a85a6c520d7d0
+#define MD5_SUM_IGA64_ASMS 85cd452f0cdcf3fe54bb0b2a7424e12a
 
 struct iga64_template const iga64_code_gpgpu_fill[] = {
 	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
@@ -192,6 +192,106 @@ struct iga64_template const iga64_code_breakpoint_suppress[] = {
 	}}
 };
 
+struct iga64_template const iga64_code_write_on_exception[] = {
+	{ .gen_ver = 2000, .size = 68, .code = (const uint32_t []) {
+		0x80100061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05154220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05354220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x02058220, 0x02000014, 0xc0ded000,
+		0x80000061, 0x02150220, 0x00000064, 0x00000000,
+		0x80001940, 0x02158220, 0x02000214, 0xc0ded001,
+		0x80041961, 0x04550220, 0x00220205, 0x00000000,
+		0x80000061, 0x04754220, 0x00000000, 0xc0ded002,
+		0x80000965, 0x03058220, 0x02008010, 0xc0ded007,
+		0x80000961, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x12000304, 0xc0ded008,
+		0x84134031, 0x00000000, 0xd00e0494, 0x04000000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 72, .code = (const uint32_t []) {
+		0x80040061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05454220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05654220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x04058220, 0x02000024, 0xc0ded000,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded001,
+		0x80000061, 0x04454220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80000965, 0x03058220, 0x02008020, 0xc0ded007,
+		0x80000961, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x12000304, 0xc0ded008,
+		0x80001a01, 0x00010000, 0x00000000, 0x00000000,
+		0x81044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 68, .code = (const uint32_t []) {
+		0x80100061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05154220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05354220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x04058220, 0x02000014, 0xc0ded000,
+		0x80000061, 0x04150220, 0x00000064, 0x00000000,
+		0x80001940, 0x04158220, 0x02000414, 0xc0ded001,
+		0x80000061, 0x04254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x04450220, 0x00000054, 0x00000000,
+		0x80000965, 0x03058220, 0x02008010, 0xc0ded007,
+		0x80000961, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x12000304, 0xc0ded008,
+		0x84134031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 72, .code = (const uint32_t []) {
+		0x80040061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05454220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05654220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x04058220, 0x02000024, 0xc0ded000,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded001,
+		0x80000061, 0x04454220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80000965, 0x03058220, 0x02008020, 0xc0ded007,
+		0x80000961, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x12000304, 0xc0ded008,
+		0x80001a01, 0x00010000, 0x00000000, 0x00000000,
+		0x81044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 68, .code = (const uint32_t []) {
+		0x80040061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05454220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05654220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x04058220, 0x02000024, 0xc0ded000,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80000140, 0x04258220, 0x02000424, 0xc0ded001,
+		0x80000061, 0x04454220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80000165, 0x03058220, 0x02008020, 0xc0ded007,
+		0x80000161, 0x30014220, 0x00000000, 0x00000000,
+		0x80000270, 0x00018220, 0x12000304, 0xc0ded008,
+		0x8104a031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
 struct iga64_template const iga64_code_media_block_write[] = {
 	{ .gen_ver = 2000, .size = 56, .code = (const uint32_t []) {
 		0x80100061, 0x04054220, 0x00000000, 0x00000000,
Signature · 1 line
2.34.1
#12 lore raw

Re: [PATCH i-g-t v5 04/17] lib/gpgpu_shader: Add write_on_exception template

Hi Zbigniew,

Quoted 38 lines · Zbigniew Kempczyński
Quoted 37 lines · Christoph Manszewski

From: Andrzej Hajda <[email protected]>

Writing specific value to memory location on unexpected value in exception register allows to report errors from inside shader or siplet.

---
  lib/gpgpu_shader.c          |  56 ++++++++++++++++++++
  lib/gpgpu_shader.h          |   2 +
  lib/iga64_generated_codes.c | 102 +++++++++++++++++++++++++++++++++++-
  3 files changed, 159 insertions(+), 1 deletion(-)
lib/gpgpu_shader.c +17 -0
index c723577e6..4097857f2 100644
@@ -634,6 +634,62 @@ void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
  	", 2, y_offset, 3, value, value, value, value);
  }
 
+/**
+ * gpgpu_shader__write_on_exception:
+ * @shdr: shader to be modified
+ * @value: dword to be written
+ * @y_offset: write target offset within the surface in rows
+ * @mask: mask to be applied on exception register
+ * @expected: expected value of exception register with @mask applied
+ *
+ * Check if bits specified by @mask in exception register(cr0.1) are equal
+ * to provided ones: cr0.1 & @mask == @expected,
+ * if yes fill dword in (row, column/dword) == (tg_id_y + @y_offset, tg_id_x).
+ */
+void gpgpu_shader__write_on_exception(struct gpgpu_shader *shdr, uint32_t value,
+				      uint32_t y_offset, uint32_t mask, uint32_t expected)
+{
+	emit_iga64_code(shdr, write_on_exception, "					\n\
+	// Clear message header								\n\
Quoted 8 lines

lib/gpgpu_shader.c void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value, · line 659

Quoted 6 lines
lib/gpgpu_shader.c +6 -0
+(W)	mov (16|M0)              r4.0<1>:ud    0x0:ud					\n\
+	// Payload									\n\
+(W)	mov (1|M0)               r5.0<1>:ud    ARG(3):ud				\n\
+(W)	mov (1|M0)               r5.1<1>:ud    ARG(4):ud				\n\
+(W)	mov (1|M0)               r5.2<1>:ud    ARG(5):ud				\n\
+(W)	mov (1|M0)               r5.3<1>:ud    ARG(6):ud				\n\

ARG(4)-ARG(6) are unused and confusing here. Please get rid of them.

Yup, already did it locally for the next revision based on your feedback from the previous patch as it is a similar case (this time for code actually added by this series =)). Thanks!

Quoted 29 lines
Quoted 29 lines
lib/gpgpu_shader.c +28 -0
+#if GEN_VER < 2000 // prepare Media Block Write						\n\
+	// X offset of the block in bytes := (thread group id X << ARG(0))		\n\
+(W)	shl (1|M0)               r4.0<1>:ud    r0.1<0;1,0>:ud    ARG(0):ud		\n\
+	// Y offset of the block in rows := thread group id Y				\n\
+(W)	mov (1|M0)               r4.1<1>:ud    r0.6<0;1,0>:ud				\n\
+(W)	add (1|M0)               r4.1<1>:ud    r4.1<0;1,0>:ud   ARG(1):ud		\n\
+	// block width [0,63] representing 1 to 64 bytes				\n\
+(W)	mov (1|M0)               r4.2<1>:ud    ARG(2):ud				\n\
+	// FFTID := FFTID from R0 header						\n\
+(W)	mov (1|M0)               r4.4<1>:ud    r0.5<0;1,0>:ud				\n\
+#else // prepare Typed 2D Block Store							\n\
+	// Load r2.0-3 with tg id X << ARG(0)						\n\
+(W)	shl (1|M0)               r2.0<1>:ud    r0.1<0;1,0>:ud    ARG(0):ud		\n\
+	// Load r2.4-7 with tg id Y + ARG(1):ud						\n\
+(W)	mov (1|M0)               r2.1<1>:ud    r0.6<0;1,0>:ud				\n\
+(W)	add (1|M0)               r2.1<1>:ud    r2.1<0;1,0>:ud    ARG(1):ud		\n\
+	// Store X and Y block start (160:191 and 192:223)				\n\
+(W)	mov (2|M0)               r4.5<1>:ud    r2.0<2;2,1>:ud				\n\
+	// Store X and Y block max_size (224:231 and 232:239)				\n\
+(W)	mov (1|M0)               r4.7<1>:ud    ARG(2):ud				\n\
+#endif											\n\
+	// Check if masked exception is equal to provided value and write conditionally \n\
+(W)      and (1|M0)              r3.0<1>:ud     cr0.1<0;1,0>:ud ARG(7):ud		\n\
+(W)      mov (1|M0)              f0.0<1>:ud     0x0:ud					\n\
+(W)      cmp (1|M0)     (eq)f0.0 null:ud        r3.0<0;1,0>:ud  ARG(8):ud		\n\
+#if GEN_VER < 2000 // Media Block Write							\n\
+(W&f0.0) send.dc1 (16|M0)        null     r4   src1_null 0    0x40A8000			\n\
+#else // Typed 2D Block Store								\n\
Quoted 8 lines

lib/gpgpu_shader.c void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value, · line 690

Quoted 3 lines
lib/gpgpu_shader.c +3 -0
+(W&f0.0) send.tgm (16|M0)        null     r4   null:0    0    0x64000007		\n\
+#endif											\n\
+	", 2, y_offset, 3, value, value, value, value, mask, expected);

I thought first and third argument might be directly entered in the code but after replace there's harder to find where they are used. I mean I looked for ARG(0) to find the place where 2 lands. So I suggest just to remove 'value' duplicates for ARG(4)-ARG(6) only.

Arghh. Ok then, I will bring them back before sending the next revision.

Thanks,
Christoph

Quoted 145 lines

--
Zbigniew

Quoted 141 lines
lib/gpgpu_shader.c +2 -0
+}
+
  /**
   * gpgpu_shader__end_system_routine:
   * @shdr: shader to be modified
lib/gpgpu_shader.h +2 -0
index e4ca0be4c..76ff4989e 100644
@@ -74,6 +74,8 @@ void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
  void gpgpu_shader__write_aip(struct gpgpu_shader *shdr, uint32_t y_offset);
  void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
  			       uint32_t y_offset);
+void gpgpu_shader__write_on_exception(struct gpgpu_shader *shdr, uint32_t dw,
+			       uint32_t y_offset, uint32_t mask, uint32_t value);
  void gpgpu_shader__label(struct gpgpu_shader *shdr, int label_id);
  void gpgpu_shader__jump(struct gpgpu_shader *shdr, int label_id);
  void gpgpu_shader__jump_neq(struct gpgpu_shader *shdr, int label_id,
lib/iga64_generated_codes.c +101 -1
index dd849eebc..c084ddcd7 100644
@@ -3,7 +3,7 @@
 
  #include "gpgpu_shader.h"
 
-#define MD5_SUM_IGA64_ASMS 33b7cd843e3b009c123a85a6c520d7d0
+#define MD5_SUM_IGA64_ASMS 85cd452f0cdcf3fe54bb0b2a7424e12a
 
  struct iga64_template const iga64_code_gpgpu_fill[] = {
  	{ .gen_ver = 2000, .size = 44, .code = (const uint32_t []) {
@@ -192,6 +192,106 @@ struct iga64_template const iga64_code_breakpoint_suppress[] = {
  	}}
  };
 
+struct iga64_template const iga64_code_write_on_exception[] = {
+	{ .gen_ver = 2000, .size = 68, .code = (const uint32_t []) {
+		0x80100061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05154220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05354220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x02058220, 0x02000014, 0xc0ded000,
+		0x80000061, 0x02150220, 0x00000064, 0x00000000,
+		0x80001940, 0x02158220, 0x02000214, 0xc0ded001,
+		0x80041961, 0x04550220, 0x00220205, 0x00000000,
+		0x80000061, 0x04754220, 0x00000000, 0xc0ded002,
+		0x80000965, 0x03058220, 0x02008010, 0xc0ded007,
+		0x80000961, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x12000304, 0xc0ded008,
+		0x84134031, 0x00000000, 0xd00e0494, 0x04000000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1270, .size = 72, .code = (const uint32_t []) {
+		0x80040061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05454220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05654220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x04058220, 0x02000024, 0xc0ded000,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded001,
+		0x80000061, 0x04454220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80000965, 0x03058220, 0x02008020, 0xc0ded007,
+		0x80000961, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x12000304, 0xc0ded008,
+		0x80001a01, 0x00010000, 0x00000000, 0x00000000,
+		0x81044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1260, .size = 68, .code = (const uint32_t []) {
+		0x80100061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05154220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05354220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x04058220, 0x02000014, 0xc0ded000,
+		0x80000061, 0x04150220, 0x00000064, 0x00000000,
+		0x80001940, 0x04158220, 0x02000414, 0xc0ded001,
+		0x80000061, 0x04254220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x04450220, 0x00000054, 0x00000000,
+		0x80000965, 0x03058220, 0x02008010, 0xc0ded007,
+		0x80000961, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x12000304, 0xc0ded008,
+		0x84134031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 1250, .size = 72, .code = (const uint32_t []) {
+		0x80040061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05454220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05654220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x04058220, 0x02000024, 0xc0ded000,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80001940, 0x04258220, 0x02000424, 0xc0ded001,
+		0x80000061, 0x04454220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80000965, 0x03058220, 0x02008020, 0xc0ded007,
+		0x80000961, 0x30014220, 0x00000000, 0x00000000,
+		0x80001a70, 0x00018220, 0x12000304, 0xc0ded008,
+		0x80001a01, 0x00010000, 0x00000000, 0x00000000,
+		0x81044031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000901, 0x00010000, 0x00000000, 0x00000000,
+	}},
+	{ .gen_ver = 0, .size = 68, .code = (const uint32_t []) {
+		0x80040061, 0x04054220, 0x00000000, 0x00000000,
+		0x80000061, 0x05054220, 0x00000000, 0xc0ded003,
+		0x80000061, 0x05254220, 0x00000000, 0xc0ded004,
+		0x80000061, 0x05454220, 0x00000000, 0xc0ded005,
+		0x80000061, 0x05654220, 0x00000000, 0xc0ded006,
+		0x80000069, 0x04058220, 0x02000024, 0xc0ded000,
+		0x80000061, 0x04250220, 0x000000c4, 0x00000000,
+		0x80000140, 0x04258220, 0x02000424, 0xc0ded001,
+		0x80000061, 0x04454220, 0x00000000, 0xc0ded002,
+		0x80000061, 0x04850220, 0x000000a4, 0x00000000,
+		0x80000165, 0x03058220, 0x02008020, 0xc0ded007,
+		0x80000161, 0x30014220, 0x00000000, 0x00000000,
+		0x80000270, 0x00018220, 0x12000304, 0xc0ded008,
+		0x8104a031, 0x00000000, 0xc0000414, 0x02a00000,
+		0x80000001, 0x00010000, 0x20000000, 0x00000000,
+		0x80000001, 0x00010000, 0x30000000, 0x00000000,
+		0x80000101, 0x00010000, 0x00000000, 0x00000000,
+	}}
+};
+
  struct iga64_template const iga64_code_media_block_write[] = {
  	{ .gen_ver = 2000, .size = 56, .code = (const uint32_t []) {
  		0x80100061, 0x04054220, 0x00000000, 0x00000000,
Signature · 1 line
2.34.1
#13 lore raw

PATCH v5 5/17 lib/gpgpu_shader: Add set/clear exception register (cr0.1) helpers

In reply to Christoph Manszewski

From: Andrzej Hajda <[email protected]>

To allow enabling and handling exceptions from shader and siplet proper helpers should be provided.

---
 lib/gpgpu_shader.c          | 28 ++++++++++++++++++++++
 lib/gpgpu_shader.h          |  2 ++
 lib/iga64_generated_codes.c | 48 ++++++++++++++++++++++++++++++++++++-
 3 files changed, 77 insertions(+), 1 deletion(-)
lib/gpgpu_shader.c +28 -0
index 4097857f2..c1bb70507 100644
@@ -634,6 +634,34 @@ void gpgpu_shader__write_dword(struct gpgpu_shader *shdr, uint32_t value,
 	", 2, y_offset, 3, value, value, value, value);
 }
 
+/**
+ * gpgpu_shader__clear_exception:
+ * @shdr: shader to be modified
+ * @value: exception bits to be cleared
+ *
+ * Clear provided bits in exception register: cr0.1 &= ~value.
+ */
+void gpgpu_shader__clear_exception(struct gpgpu_shader *shdr, uint32_t value)
+{
+	emit_iga64_code(shdr, clear_exception, "		\n\
+(W)	and (1|M0) cr0.1<1>:ud cr0.1<0;1,0>:ud ARG(0):ud	\n\
+	", ~value);
+}
+
+/**
+ * gpgpu_shader__set_exception:
+ * @shdr: shader to be modified
+ * @value: exception bits to be set
+ *
+ * Set provided bits in exception register: cr0.1 |= value.
+ */
+void gpgpu_shader__set_exception(struct gpgpu_shader *shdr, uint32_t value)
+{
+	emit_iga64_code(shdr, set_exception, "		\n\
+(W)	or (1|M0) cr0.1<1>:ud cr0.1<0;1,0>:ud ARG(0):ud	\n\
+	", value);
+}
+
 /**
  * gpgpu_shader__write_on_exception:
  * @shdr: shader to be modified
lib/gpgpu_shader.h +2 -0
index 76ff4989e..0bbeae66f 100644
@@ -66,6 +66,8 @@ void gpgpu_shader__common_target_write(struct gpgpu_shader *shdr,
 				       uint32_t y_offset, const uint32_t value[4]);
 void gpgpu_shader__common_target_write_u32(struct gpgpu_shader *shdr,
 				     uint32_t y_offset, uint32_t value);
+void gpgpu_shader__clear_exception(struct gpgpu_shader *shdr, uint32_t value);
+void gpgpu_shader__set_exception(struct gpgpu_shader *shdr, uint32_t value);
 void gpgpu_shader__end_system_routine(struct gpgpu_shader *shdr,
 				      bool breakpoint_suppress);
 void gpgpu_shader__end_system_routine_step_if_eq(struct gpgpu_shader *shdr,
lib/iga64_generated_codes.c +47 -1

60 lines not on this page · raw message

Signature · 1 line
2.34.1
#14 lore raw

PATCH v5 6/17 lib/intel_batchbuffer: Add helper to get pointer at specified offset

In reply to Christoph Manszewski

From: Andrzej Hajda <[email protected]>

The helper will be used to access data placed in batchbuffer.

---
 lib/intel_batchbuffer.h | 7 +++++++
 1 file changed, 7 insertions(+)
lib/intel_batchbuffer.h +7 -0
index cb32206e5..b1670d23e 100644
@@ -353,6 +353,13 @@ static inline uint32_t intel_bb_offset(struct intel_bb *ibb)
 	return (uint32_t) ((uint8_t *) ibb->ptr - (uint8_t *) ibb->batch);
 }
 
+static inline void *intel_bb_ptr_get(struct intel_bb *ibb, uint32_t offset)
+{
+	igt_assert(offset <= ibb->size);
+
+	return ((uint8_t *) ibb->batch + offset);
+}
+
 static inline void intel_bb_ptr_set(struct intel_bb *ibb, uint32_t offset)
 {
 	ibb->ptr = (void *) ((uint8_t *) ibb->batch + offset);
Signature · 1 line
2.34.1
#15 lore raw

Re: [PATCH i-g-t v5 06/17] lib/intel_batchbuffer: Add helper to get pointer at specified offset

Quoted 17 lines · Christoph Manszewski

From: Andrzej Hajda <[email protected]>

The helper will be used to access data placed in batchbuffer.

---
 lib/intel_batchbuffer.h | 7 +++++++
 1 file changed, 7 insertions(+)
lib/intel_batchbuffer.h +0 -0
index cb32206e5..b1670d23e 100644

lib/intel_batchbuffer.h static inline uint32_t intel_bb_offset(struct intel_bb *ibb) · line 358

Quoted 6 lines
lib/intel_batchbuffer.h +3 -0
@@ -353,6 +353,13 @@ static inline uint32_t intel_bb_offset(struct intel_bb *ibb)
 	return (uint32_t) ((uint8_t *) ibb->ptr - (uint8_t *) ibb->batch);
 }
 
+static inline void *intel_bb_ptr_get(struct intel_bb *ibb, uint32_t offset)
+{
+	igt_assert(offset <= ibb->size);

I think it should be < only, == means we're outside the buffer.

--
Zbigniew

Quoted 10 lines
lib/intel_batchbuffer.h +4 -0
+
+	return ((uint8_t *) ibb->batch + offset);
+}
+
 static inline void intel_bb_ptr_set(struct intel_bb *ibb, uint32_t offset)
 {
 	ibb->ptr = (void *) ((uint8_t *) ibb->batch + offset);
Signature · 1 line
2.34.1
#16 lore raw

Re: [PATCH i-g-t v5 06/17] lib/intel_batchbuffer: Add helper to get pointer at specified offset

Hi Zbigniew,

Quoted 21 lines · Zbigniew Kempczyński
Quoted 20 lines · Christoph Manszewski

From: Andrzej Hajda <[email protected]>

The helper will be used to access data placed in batchbuffer.

---
  lib/intel_batchbuffer.h | 7 +++++++
  1 file changed, 7 insertions(+)
lib/intel_batchbuffer.h +0 -0
index cb32206e5..b1670d23e 100644
@@ -353,6 +353,13 @@ static inline uint32_t intel_bb_offset(struct intel_bb *ibb)
  	return (uint32_t) ((uint8_t *) ibb->ptr - (uint8_t *) ibb->batch);
  }
 
Quoted 5 lines

lib/intel_batchbuffer.h static inline uint32_t intel_bb_offset(struct intel_bb *ibb) · line 358

Quoted 3 lines
lib/intel_batchbuffer.h +3 -0
+static inline void *intel_bb_ptr_get(struct intel_bb *ibb, uint32_t offset)
+{
+	igt_assert(offset <= ibb->size);

I think it should be < only, == means we're outside the buffer.

Now that you say it - it makes sense. To be honest I just took a similar approach to intel_bb_ptr_set. But I will change it to '<' for intel_bb_ptr_get then.

Thanks,
Christoph

Quoted 14 lines

--
Zbigniew

Quoted 10 lines
lib/intel_batchbuffer.h +4 -0
+
+	return ((uint8_t *) ibb->batch + offset);
+}
+
  static inline void intel_bb_ptr_set(struct intel_bb *ibb, uint32_t offset)
  {
  	ibb->ptr = (void *) ((uint8_t *) ibb->batch + offset);
Signature · 1 line
2.34.1
#17 lore raw

Re: [PATCH i-g-t v5 06/17] lib/intel_batchbuffer: Add helper to get pointer at specified offset

Quoted 29 lines · Christoph

Hi Zbigniew,

Quoted 13 lines · Zbigniew Kempczyński
Quoted 12 lines · Christoph Manszewski

From: Andrzej Hajda <[email protected]>

The helper will be used to access data placed in batchbuffer.

---
  lib/intel_batchbuffer.h | 7 +++++++
  1 file changed, 7 insertions(+)
Quoted 12 lines

lib/intel_batchbuffer.h static inline uint32_t intel_bb_offset(struct intel_bb *ibb) · line 357

Quoted 10 lines
lib/intel_batchbuffer.h +3 -0
index cb32206e5..b1670d23e 100644
@@ -353,6 +353,13 @@ static inline uint32_t intel_bb_offset(struct intel_bb *ibb)
  	return (uint32_t) ((uint8_t *) ibb->ptr - (uint8_t *) ibb->batch);
  }
+static inline void *intel_bb_ptr_get(struct intel_bb *ibb, uint32_t offset)
+{
+	igt_assert(offset <= ibb->size);

I think it should be < only, == means we're outside the buffer.

Quoted 3 lines

Now that you say it - it makes sense. To be honest I just took a similar approach to intel_bb_ptr_set. But I will change it to '<' for intel_bb_ptr_get then.

It seems that intel_bb_ptr_set() is also wrong, it should be < too.

Regarding 'get' - nothing will protect us from using returned pointer out of bounds leading to memory corruption. But at least we may try to catch some cases early.

Feel free to fix 'set' as well.

--
Zbigniew

Quoted 17 lines

Thanks,
Christoph

Quoted 14 lines

--
Zbigniew

Quoted 10 lines
lib/intel_batchbuffer.h +4 -0
+
+	return ((uint8_t *) ibb->batch + offset);
+}
+
  static inline void intel_bb_ptr_set(struct intel_bb *ibb, uint32_t offset)
  {
  	ibb->ptr = (void *) ((uint8_t *) ibb->batch + offset);
Signature · 1 line
2.34.1
#18 lore raw

Re: [PATCH i-g-t v5 06/17] lib/intel_batchbuffer: Add helper to get pointer at specified offset

In reply to Manszewski, Christoph
Quoted 29 lines · Christoph

Hi Zbigniew,

Quoted 13 lines · Zbigniew Kempczyński
Quoted 12 lines · Christoph Manszewski

From: Andrzej Hajda <[email protected]>

The helper will be used to access data placed in batchbuffer.

---
  lib/intel_batchbuffer.h | 7 +++++++
  1 file changed, 7 insertions(+)
Quoted 12 lines

lib/intel_batchbuffer.h static inline uint32_t intel_bb_offset(struct intel_bb *ibb) · line 357

Quoted 10 lines
lib/intel_batchbuffer.h +3 -0
index cb32206e5..b1670d23e 100644
@@ -353,6 +353,13 @@ static inline uint32_t intel_bb_offset(struct intel_bb *ibb)
  	return (uint32_t) ((uint8_t *) ibb->ptr - (uint8_t *) ibb->batch);
  }
+static inline void *intel_bb_ptr_get(struct intel_bb *ibb, uint32_t offset)
+{
+	igt_assert(offset <= ibb->size);

I think it should be < only, == means we're outside the buffer.

Quoted 7 lines

Now that you say it - it makes sense. To be honest I just took a similar approach to intel_bb_ptr_set. But I will change it to '<' for intel_bb_ptr_get then.

Thanks,
Christoph

Quoted 1 line

I forgot. Christoph, please add your s-o-b to all patches as you're responsible for transporting and altering them to igt ml.

--
Zbigniew

Quoted 13 lines
Quoted 13 lines

--
Zbigniew

Quoted 10 lines
lib/intel_batchbuffer.h +4 -0
+
+	return ((uint8_t *) ibb->batch + offset);
+}
+
  static inline void intel_bb_ptr_set(struct intel_bb *ibb, uint32_t offset)
  {
  	ibb->ptr = (void *) ((uint8_t *) ibb->batch + offset);
Signature · 1 line
2.34.1
#19 lore raw

PATCH v5 7/17 lib/gpgpu_shader: Allow enabling illegal opcode exceptions in shader

In reply to Christoph Manszewski

From: Andrzej Hajda <[email protected]>

Illegal opcode exceptions can be enabled in interface descriptor data passed to COMPUTE_WALKER instruction.

---
 lib/gpgpu_shader.c | 4 ++++
 lib/gpgpu_shader.h | 1 +
 2 files changed, 5 insertions(+)
lib/gpgpu_shader.c +4 -0
index c1bb70507..fb0bd6e4c 100644
@@ -103,6 +103,7 @@ __xelp_gpgpu_execfunc(struct intel_bb *ibb,
 		      struct gpgpu_shader *sip,
 		      uint64_t ring, bool explicit_engine)
 {
+	struct gen8_interface_descriptor_data *idd;
 	uint32_t interface_descriptor, sip_offset;
 	uint64_t engine;
 
@@ -113,6 +114,8 @@ __xelp_gpgpu_execfunc(struct intel_bb *ibb,
 	interface_descriptor = gen8_fill_interface_descriptor(ibb, target,
 							      shdr->instr,
 							      4 * shdr->size);
+	idd = intel_bb_ptr_get(ibb, interface_descriptor);
+	idd->desc2.illegal_opcode_exception_enable = shdr->illegal_opcode_exception_enable;
 
 	if (sip && sip->size)
 		sip_offset = fill_sip(ibb, sip->instr, 4 * sip->size);
@@ -163,6 +166,7 @@ __xehp_gpgpu_execfunc(struct intel_bb *ibb,
 
 	xehp_fill_interface_descriptor(ibb, target, shdr->instr,
 				       4 * shdr->size, &idd);
+	idd.desc2.illegal_opcode_exception_enable = shdr->illegal_opcode_exception_enable;
 
 	if (sip && sip->size)
 		sip_offset = fill_sip(ibb, sip->instr, 4 * sip->size);
lib/gpgpu_shader.h +1 -0
index 0bbeae66f..26a117a0b 100644
@@ -22,6 +22,7 @@ struct gpgpu_shader {
 		uint32_t (*instr)[4];
 	};
 	struct igt_map *labels;
+	bool illegal_opcode_exception_enable;
 };
 
 struct iga64_template {
Signature · 1 line
2.34.1
#20 lore raw

PATCH v5 8/17 tests/xe_exec_sip: Add sanity-after-timeout test

In reply to Christoph Manszewski

Add a subtest that checks if we are able to submit workloads after gpu was reset due to hung job.

---
 tests/intel/xe_exec_sip.c | 41 ++++++++++++++++++++++++++++++++-------
 1 file changed, 34 insertions(+), 7 deletions(-)
tests/intel/xe_exec_sip.c +34 -7
index ea1770cd6..3b46f5618 100644
@@ -31,6 +31,11 @@
 
 #define SHADER_CANARY 0x01010101
 
+enum shader_type {
+	SHADER_HANG,
+	SHADER_WRITE,
+};
+
 static struct intel_buf *
 create_fill_buf(int fd, int width, int height, uint8_t color)
 {
@@ -50,21 +55,32 @@ create_fill_buf(int fd, int width, int height, uint8_t color)
 	return buf;
 }
 
-static struct gpgpu_shader *get_shader(int fd)
+static struct gpgpu_shader *get_shader(int fd, const int shadertype)
 {
 	static struct gpgpu_shader *shader;
 
 	shader = gpgpu_shader_create(fd);
 	gpgpu_shader__write_dword(shader, SHADER_CANARY, 0);
+
+	switch (shadertype) {
+	case SHADER_HANG:
+		gpgpu_shader__label(shader, 0);
+		gpgpu_shader__nop(shader);
+		gpgpu_shader__jump(shader, 0);
+		break;
+	case SHADER_WRITE:
+		break;
+	}
+
 	gpgpu_shader__eot(shader);
 	return shader;
 }
 
-static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, unsigned int threads,
-			     unsigned int width, unsigned int height)
+static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, const int shadertype,
+			     unsigned int threads, unsigned int width, unsigned int height)
 {
 	struct intel_buf *buf = create_fill_buf(fd, width, height, COLOR_C4);
-	struct gpgpu_shader *shader = get_shader(fd);
+	struct gpgpu_shader *shader = get_shader(fd, shadertype);
 
 	gpgpu_shader_exec(ibb, buf, 1, threads, shader, NULL, 0, 0);
 	gpgpu_shader_destroy(shader);
@@ -125,8 +141,10 @@ xe_sysfs_get_job_timeout_ms(int fd, struct drm_xe_engine_class_instance *eci)
  * SUBTEST: sanity
  * Description: check basic shader with write operation
  *
+ * SUBTEST: sanity-after-timeout
+ * Description: check basic shader execution after job timeout
  */
-static void test_sip(struct drm_xe_engine_class_instance *eci, uint32_t flags)
+static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint32_t flags)
 {
 	unsigned int threads = 512;
 	unsigned int height = max_t(threads, HEIGHT, threads * 2);
@@ -153,7 +171,7 @@ static void test_sip(struct drm_xe_engine_class_instance *eci, uint32_t flags)
 	ibb = intel_bb_create_with_context(fd, exec_queue_id, vm_id, NULL, 4096);
 
 	igt_nsec_elapsed(&ts);
-	handle = gpgpu_shader(fd, ibb, threads, width, height);
+	handle = gpgpu_shader(fd, ibb, shader, threads, width, height);
 
 	intel_bb_sync(ibb);
 	igt_assert_lt_u64(igt_nsec_elapsed(&ts), timeout);
@@ -186,7 +204,16 @@ igt_main
 		fd = drm_open_driver(DRIVER_XE);
 
 	test_render_and_compute("sanity", fd, eci)
-		test_sip(eci, 0);
+		test_sip(SHADER_WRITE, eci, 0);
+
+	test_render_and_compute("sanity-after-timeout", fd, eci) {
+		test_sip(SHADER_HANG, eci, 0);
+
+		xe_for_each_engine(fd, eci)
+			if (eci->engine_class == DRM_XE_ENGINE_CLASS_RENDER ||
+			    eci->engine_class == DRM_XE_ENGINE_CLASS_COMPUTE)
+				test_sip(SHADER_WRITE, eci, 0);
+	}
 
 	igt_fixture
 		drm_close_driver(fd);
Signature · 1 line
2.34.1
Reviewed-by on v5 8/17
#21 lore raw

Re: [PATCH i-g-t v5 08/17] tests/xe_exec_sip: Add sanity-after-timeout test

Quoted 103 lines · Christoph Manszewski

Add a subtest that checks if we are able to submit workloads after gpu was reset due to hung job.

---
 tests/intel/xe_exec_sip.c | 41 ++++++++++++++++++++++++++++++++-------
 1 file changed, 34 insertions(+), 7 deletions(-)
tests/intel/xe_exec_sip.c +34 -7
index ea1770cd6..3b46f5618 100644
@@ -31,6 +31,11 @@
 
 #define SHADER_CANARY 0x01010101
 
+enum shader_type {
+	SHADER_HANG,
+	SHADER_WRITE,
+};
+
 static struct intel_buf *
 create_fill_buf(int fd, int width, int height, uint8_t color)
 {
@@ -50,21 +55,32 @@ create_fill_buf(int fd, int width, int height, uint8_t color)
 	return buf;
 }
 
-static struct gpgpu_shader *get_shader(int fd)
+static struct gpgpu_shader *get_shader(int fd, const int shadertype)
 {
 	static struct gpgpu_shader *shader;
 
 	shader = gpgpu_shader_create(fd);
 	gpgpu_shader__write_dword(shader, SHADER_CANARY, 0);
+
+	switch (shadertype) {
+	case SHADER_HANG:
+		gpgpu_shader__label(shader, 0);
+		gpgpu_shader__nop(shader);
+		gpgpu_shader__jump(shader, 0);
+		break;
+	case SHADER_WRITE:
+		break;
+	}
+
 	gpgpu_shader__eot(shader);
 	return shader;
 }
 
-static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, unsigned int threads,
-			     unsigned int width, unsigned int height)
+static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, const int shadertype,
+			     unsigned int threads, unsigned int width, unsigned int height)
 {
 	struct intel_buf *buf = create_fill_buf(fd, width, height, COLOR_C4);
-	struct gpgpu_shader *shader = get_shader(fd);
+	struct gpgpu_shader *shader = get_shader(fd, shadertype);
 
 	gpgpu_shader_exec(ibb, buf, 1, threads, shader, NULL, 0, 0);
 	gpgpu_shader_destroy(shader);
@@ -125,8 +141,10 @@ xe_sysfs_get_job_timeout_ms(int fd, struct drm_xe_engine_class_instance *eci)
  * SUBTEST: sanity
  * Description: check basic shader with write operation
  *
+ * SUBTEST: sanity-after-timeout
+ * Description: check basic shader execution after job timeout
  */
-static void test_sip(struct drm_xe_engine_class_instance *eci, uint32_t flags)
+static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint32_t flags)
 {
 	unsigned int threads = 512;
 	unsigned int height = max_t(threads, HEIGHT, threads * 2);
@@ -153,7 +171,7 @@ static void test_sip(struct drm_xe_engine_class_instance *eci, uint32_t flags)
 	ibb = intel_bb_create_with_context(fd, exec_queue_id, vm_id, NULL, 4096);
 
 	igt_nsec_elapsed(&ts);
-	handle = gpgpu_shader(fd, ibb, threads, width, height);
+	handle = gpgpu_shader(fd, ibb, shader, threads, width, height);
 
 	intel_bb_sync(ibb);
 	igt_assert_lt_u64(igt_nsec_elapsed(&ts), timeout);
@@ -186,7 +204,16 @@ igt_main
 		fd = drm_open_driver(DRIVER_XE);
 
 	test_render_and_compute("sanity", fd, eci)
-		test_sip(eci, 0);
+		test_sip(SHADER_WRITE, eci, 0);
+
+	test_render_and_compute("sanity-after-timeout", fd, eci) {
+		test_sip(SHADER_HANG, eci, 0);
+
+		xe_for_each_engine(fd, eci)
+			if (eci->engine_class == DRM_XE_ENGINE_CLASS_RENDER ||
+			    eci->engine_class == DRM_XE_ENGINE_CLASS_COMPUTE)
+				test_sip(SHADER_WRITE, eci, 0);
+	}
 
 	igt_fixture
 		drm_close_driver(fd);
Signature · 1 line
2.34.1

LGTM,

--
Zbigniew

#22 lore raw

PATCH v5 9/17 tests/xe_exec_sip: Introduce invalid instruction tests

In reply to Christoph Manszewski

From: Andrzej Hajda <[email protected]>

Xe2 and earlier gens are able to handle very limited set of invalid instructions - only illegal and undefined opcodes, other errors in instruction can cause undefined behavior.
Illegal/undefined opcode results in:
- setting illegal opcode status bit - cr0.1[28],
- calling SIP if illegal opcode bit is enabled - cr0.1[12]. cr0.1[12] can be enabled directly from the thread or by thread dispatcher from Interface Descriptor Data provided to COMPUTE_WALKER instruction.

Implemented cases:
- check if SIP is not called when exception is not enabled,
- check if SIP is called when exception is enabled from EU thread,
- check if SIP is called when exception is enabled from COMPUTE_WALKER

Cc Mika Kuoppala <[email protected]>
Cc Dominik Grzegorzek <[email protected]>
---
 tests/intel/xe_exec_sip.c | 118 ++++++++++++++++++++++++++++++++++----
 1 file changed, 108 insertions(+), 10 deletions(-)
tests/intel/xe_exec_sip.c +108 -10
index 3b46f5618..5384979a6 100644
@@ -30,12 +30,25 @@
 #define COLOR_C4 0xc4
 
 #define SHADER_CANARY 0x01010101
+#define SIP_CANARY 0x02020202
 
 enum shader_type {
 	SHADER_HANG,
+	SHADER_INV_INSTR_DISABLED,
+	SHADER_INV_INSTR_THREAD_ENABLED,
+	SHADER_INV_INSTR_WALKER_ENABLED,
 	SHADER_WRITE,
 };
 
+enum sip_type {
+	SIP_INV_INSTR,
+	SIP_NULL,
+};
+
+/* Control Register cr0.1 bits for exception handling */
+#define ILLEGAL_OPCODE_ENABLE BIT(12)
+#define ILLEGAL_OPCODE_STATUS BIT(28)
+
 static struct intel_buf *
 create_fill_buf(int fd, int width, int height, uint8_t color)
 {
@@ -58,8 +71,12 @@ create_fill_buf(int fd, int width, int height, uint8_t color)
 static struct gpgpu_shader *get_shader(int fd, const int shadertype)
 {
 	static struct gpgpu_shader *shader;
+	uint32_t bad;
 
 	shader = gpgpu_shader_create(fd);
+	if (shadertype == SHADER_INV_INSTR_WALKER_ENABLED)
+		shader->illegal_opcode_exception_enable = true;
+
 	gpgpu_shader__write_dword(shader, SHADER_CANARY, 0);
 
 	switch (shadertype) {
@@ -70,19 +87,59 @@ static struct gpgpu_shader *get_shader(int fd, const int shadertype)
 		break;
 	case SHADER_WRITE:
 		break;
+	case SHADER_INV_INSTR_THREAD_ENABLED:
+		gpgpu_shader__set_exception(shader, ILLEGAL_OPCODE_ENABLE);
+		__attribute__ ((fallthrough));
+	case SHADER_INV_INSTR_DISABLED:
+	case SHADER_INV_INSTR_WALKER_ENABLED:
+		bad = (shadertype == SHADER_INV_INSTR_DISABLED) ? ILLEGAL_OPCODE_ENABLE : 0;
+		gpgpu_shader__write_on_exception(shader, 1, 0, ILLEGAL_OPCODE_ENABLE, bad);
+		gpgpu_shader__nop(shader);
+		gpgpu_shader__nop(shader);
+		/* modify second nop, set only opcode bits[6:0] */
+		shader->instr[shader->size / 4 - 1][0] = 0x7f;
+		/* SIP should clear exception bit */
+		bad = ILLEGAL_OPCODE_STATUS;
+		gpgpu_shader__write_on_exception(shader, 2, 0, ILLEGAL_OPCODE_STATUS, bad);
+		break;
 	}
 
 	gpgpu_shader__eot(shader);
 	return shader;
 }
 
-static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, const int shadertype,
+static struct gpgpu_shader *get_sip(int fd, const int siptype, unsigned int y_offset)
+{
+	static struct gpgpu_shader *sip;
+
+	if (siptype == SIP_NULL)
+		return NULL;
+
+	sip = gpgpu_shader_create(fd);
+	gpgpu_shader__write_dword(sip, SIP_CANARY, y_offset);
+
+	switch (siptype) {
+	case SIP_INV_INSTR:
+		gpgpu_shader__write_on_exception(sip, 1, y_offset, ILLEGAL_OPCODE_STATUS, 0);
+		break;
+	}
+
+	gpgpu_shader__end_system_routine(sip, false);
+	return sip;
+}
+
+static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, const int shadertype, const int siptype,
 			     unsigned int threads, unsigned int width, unsigned int height)
 {
 	struct intel_buf *buf = create_fill_buf(fd, width, height, COLOR_C4);
+	struct gpgpu_shader *sip = get_sip(fd, siptype, height / 2);
 	struct gpgpu_shader *shader = get_shader(fd, shadertype);
 
-	gpgpu_shader_exec(ibb, buf, 1, threads, shader, NULL, 0, 0);
+	gpgpu_shader_exec(ibb, buf, 1, threads, shader, sip, 0, 0);
+
+	if (sip)
+		gpgpu_shader_destroy(sip);
+
 	gpgpu_shader_destroy(shader);
 	return buf->handle;
 }
@@ -98,10 +155,10 @@ static void check_fill_buf(uint8_t *ptr, const int width, const int x,
 }
 
 static void check_buf(int fd, uint32_t handle, int width, int height,
-		      uint8_t poison_c)
+		      int shadertype, int siptype, uint8_t poison_c)
 {
 	unsigned int sz = ALIGN(width * height, 4096);
-	int thread_count = 0;
+	int thread_count = 0, sip_count = 0;
 	uint32_t *ptr;
 	int i, j;
 
@@ -119,7 +176,27 @@ static void check_buf(int fd, uint32_t handle, int width, int height,
 		i = 0;
 	}
 
+	for (i = 0, j = height / 2; j < height; ++j) {
+		if (ptr[j * width / 4] == SIP_CANARY) {
+			++sip_count;
+			i = 4;
+		}
+
+		for (; i < width; i++)
+			check_fill_buf((uint8_t *)ptr, width, i, j, poison_c);
+
+		i = 0;
+	}
+
 	igt_assert(thread_count);
+	if (shadertype == SHADER_INV_INSTR_DISABLED)
+		igt_assert(!sip_count);
+	else if (siptype == SIP_INV_INSTR && shadertype != SHADER_INV_INSTR_DISABLED)
+		igt_assert_f(thread_count == sip_count,
+			     "Thread and SIP count mismatch, %d != %d\n",
+			     thread_count, sip_count);
+	else
+		igt_assert(sip_count == 0);
 
 	munmap(ptr, sz);
 }
@@ -143,8 +220,20 @@ xe_sysfs_get_job_timeout_ms(int fd, struct drm_xe_engine_class_instance *eci)
  *
  * SUBTEST: sanity-after-timeout
  * Description: check basic shader execution after job timeout
+ *
+ * SUBTEST: invalidinstr-disabled
+ * Description: Verify that we don't enter SIP after running into an invalid
+ *		instruction when exception is not enabled.
+ *
+ * SUBTEST: invalidinstr-thread-enabled
+ * Description: Verify that we enter SIP after running into an invalid instruction
+ *              when exception is enabled from thread.
+ *
+ * SUBTEST: invalidinstr-walker-enabled
+ * Description: Verify that we enter SIP after running into an invalid instruction
+ *              when exception is enabled from COMPUTE_WALKER.
  */
-static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint32_t flags)
+static void test_sip(int shader, int sip, struct drm_xe_engine_class_instance *eci, uint32_t flags)
 {
 	unsigned int threads = 512;
 	unsigned int height = max_t(threads, HEIGHT, threads * 2);
@@ -171,12 +260,12 @@ static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint3
 	ibb = intel_bb_create_with_context(fd, exec_queue_id, vm_id, NULL, 4096);
 
 	igt_nsec_elapsed(&ts);
-	handle = gpgpu_shader(fd, ibb, shader, threads, width, height);
+	handle = gpgpu_shader(fd, ibb, shader, sip, threads, width, height);
 
 	intel_bb_sync(ibb);
 	igt_assert_lt_u64(igt_nsec_elapsed(&ts), timeout);
 
-	check_buf(fd, handle, width, height, COLOR_C4);
+	check_buf(fd, handle, width, height, shader, sip, COLOR_C4);
 
 	gem_close(fd, handle);
 	intel_bb_destroy(ibb);
@@ -204,17 +293,26 @@ igt_main
 		fd = drm_open_driver(DRIVER_XE);
 
 	test_render_and_compute("sanity", fd, eci)
-		test_sip(SHADER_WRITE, eci, 0);
+		test_sip(SHADER_WRITE, SIP_NULL, eci, 0);
 
 	test_render_and_compute("sanity-after-timeout", fd, eci) {
-		test_sip(SHADER_HANG, eci, 0);
+		test_sip(SHADER_HANG, SIP_NULL, eci, 0);
 
 		xe_for_each_engine(fd, eci)
 			if (eci->engine_class == DRM_XE_ENGINE_CLASS_RENDER ||
 			    eci->engine_class == DRM_XE_ENGINE_CLASS_COMPUTE)
-				test_sip(SHADER_WRITE, eci, 0);
+				test_sip(SHADER_WRITE, SIP_NULL, eci, 0);
 	}
 
+	test_render_and_compute("invalidinstr-disabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_DISABLED, SIP_INV_INSTR, eci, 0);
+
+	test_render_and_compute("invalidinstr-thread-enabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_THREAD_ENABLED, SIP_INV_INSTR, eci, 0);
+
+	test_render_and_compute("invalidinstr-walker-enabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_WALKER_ENABLED, SIP_INV_INSTR, eci, 0);
+
 	igt_fixture
 		drm_close_driver(fd);
 }
Signature · 1 line
2.34.1
#23 lore raw

Re: [PATCH i-g-t v5 09/17] tests/xe_exec_sip: Introduce invalid instruction tests

Quoted 67 lines · Christoph Manszewski

From: Andrzej Hajda <[email protected]>

Xe2 and earlier gens are able to handle very limited set of invalid instructions - only illegal and undefined opcodes, other errors in instruction can cause undefined behavior.
Illegal/undefined opcode results in:
- setting illegal opcode status bit - cr0.1[28],
- calling SIP if illegal opcode bit is enabled - cr0.1[12]. cr0.1[12] can be enabled directly from the thread or by thread dispatcher from Interface Descriptor Data provided to COMPUTE_WALKER instruction.

Implemented cases:
- check if SIP is not called when exception is not enabled,
- check if SIP is called when exception is enabled from EU thread,
- check if SIP is called when exception is enabled from COMPUTE_WALKER

Cc Mika Kuoppala <[email protected]>
Cc Dominik Grzegorzek <[email protected]>
---
 tests/intel/xe_exec_sip.c | 118 ++++++++++++++++++++++++++++++++++----
 1 file changed, 108 insertions(+), 10 deletions(-)
tests/intel/xe_exec_sip.c +17 -0
index 3b46f5618..5384979a6 100644
@@ -30,12 +30,25 @@
 #define COLOR_C4 0xc4
 
 #define SHADER_CANARY 0x01010101
+#define SIP_CANARY 0x02020202
 
 enum shader_type {
 	SHADER_HANG,
+	SHADER_INV_INSTR_DISABLED,
+	SHADER_INV_INSTR_THREAD_ENABLED,
+	SHADER_INV_INSTR_WALKER_ENABLED,
 	SHADER_WRITE,
 };
 
+enum sip_type {
+	SIP_INV_INSTR,
+	SIP_NULL,
+};
+
+/* Control Register cr0.1 bits for exception handling */
+#define ILLEGAL_OPCODE_ENABLE BIT(12)
+#define ILLEGAL_OPCODE_STATUS BIT(28)
+
 static struct intel_buf *
 create_fill_buf(int fd, int width, int height, uint8_t color)
 {
@@ -58,8 +71,12 @@ create_fill_buf(int fd, int width, int height, uint8_t color)
 static struct gpgpu_shader *get_shader(int fd, const int shadertype)
 {
 	static struct gpgpu_shader *shader;
+	uint32_t bad;
 
 	shader = gpgpu_shader_create(fd);
+	if (shadertype == SHADER_INV_INSTR_WALKER_ENABLED)
+		shader->illegal_opcode_exception_enable = true;
+
 	gpgpu_shader__write_dword(shader, SHADER_CANARY, 0);
 
 	switch (shadertype) {

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 92

Quoted 6 lines
tests/intel/xe_exec_sip.c +3 -0
@@ -70,19 +87,59 @@ static struct gpgpu_shader *get_shader(int fd, const int shadertype)
 		break;
 	case SHADER_WRITE:
 		break;
+	case SHADER_INV_INSTR_THREAD_ENABLED:
+		gpgpu_shader__set_exception(shader, ILLEGAL_OPCODE_ENABLE);
+		__attribute__ ((fallthrough));

Just be curious - on which gcc you're observing this warning without fallthrough attribute? clang explicitely requires -Wimplicit-fallthrough to enable this warning.

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 100

Quoted 8 lines
tests/intel/xe_exec_sip.c +8 -0
+	case SHADER_INV_INSTR_DISABLED:
+	case SHADER_INV_INSTR_WALKER_ENABLED:
+		bad = (shadertype == SHADER_INV_INSTR_DISABLED) ? ILLEGAL_OPCODE_ENABLE : 0;
+		gpgpu_shader__write_on_exception(shader, 1, 0, ILLEGAL_OPCODE_ENABLE, bad);
+		gpgpu_shader__nop(shader);
+		gpgpu_shader__nop(shader);
+		/* modify second nop, set only opcode bits[6:0] */
+		shader->instr[shader->size / 4 - 1][0] = 0x7f;

This pattern 'shader->size / 4 - 1' deserves own macro or inline like last_instr(shader) or similar.

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 107

Quoted 7 lines
tests/intel/xe_exec_sip.c +4 -0
+		/* SIP should clear exception bit */
+		bad = ILLEGAL_OPCODE_STATUS;
+		gpgpu_shader__write_on_exception(shader, 2, 0, ILLEGAL_OPCODE_STATUS, bad);
+		break;
 	}
 
 	gpgpu_shader__eot(shader);

I would add one blank line before return on all functions in which return is glued with the code.

Quoted 11 lines
tests/intel/xe_exec_sip.c +7 -1
 	return shader;
 }
 
-static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, const int shadertype,
+static struct gpgpu_shader *get_sip(int fd, const int siptype, unsigned int y_offset)
+{
+	static struct gpgpu_shader *sip;
+
+	if (siptype == SIP_NULL)
+		return NULL;
+

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 127

Quoted 10 lines
tests/intel/xe_exec_sip.c +10 -0
+	sip = gpgpu_shader_create(fd);
+	gpgpu_shader__write_dword(sip, SIP_CANARY, y_offset);
+
+	switch (siptype) {
+	case SIP_INV_INSTR:
+		gpgpu_shader__write_on_exception(sip, 1, y_offset, ILLEGAL_OPCODE_STATUS, 0);
+		break;
+	}
+
+	gpgpu_shader__end_system_routine(sip, false);

Ditto.

Quoted 7 lines
tests/intel/xe_exec_sip.c +4 -0
+	return sip;
+}
+
+static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, const int shadertype, const int siptype,
 			     unsigned int threads, unsigned int width, unsigned int height)
 {
 	struct intel_buf *buf = create_fill_buf(fd, width, height, COLOR_C4);

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 143

Quoted 10 lines
tests/intel/xe_exec_sip.c +6 -1
+	struct gpgpu_shader *sip = get_sip(fd, siptype, height / 2);
 	struct gpgpu_shader *shader = get_shader(fd, shadertype);
 
-	gpgpu_shader_exec(ibb, buf, 1, threads, shader, NULL, 0, 0);
+	gpgpu_shader_exec(ibb, buf, 1, threads, shader, sip, 0, 0);
+
+	if (sip)
+		gpgpu_shader_destroy(sip);
+
 	gpgpu_shader_destroy(shader);

Ditto.

tests/intel/xe_exec_sip.c static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint3

Quoted 66 lines
tests/intel/xe_exec_sip.c +35 -3
 	return buf->handle;
 }
@@ -98,10 +155,10 @@ static void check_fill_buf(uint8_t *ptr, const int width, const int x,
 }
 
 static void check_buf(int fd, uint32_t handle, int width, int height,
-		      uint8_t poison_c)
+		      int shadertype, int siptype, uint8_t poison_c)
 {
 	unsigned int sz = ALIGN(width * height, 4096);
-	int thread_count = 0;
+	int thread_count = 0, sip_count = 0;
 	uint32_t *ptr;
 	int i, j;
 
@@ -119,7 +176,27 @@ static void check_buf(int fd, uint32_t handle, int width, int height,
 		i = 0;
 	}
 
+	for (i = 0, j = height / 2; j < height; ++j) {
+		if (ptr[j * width / 4] == SIP_CANARY) {
+			++sip_count;
+			i = 4;
+		}
+
+		for (; i < width; i++)
+			check_fill_buf((uint8_t *)ptr, width, i, j, poison_c);
+
+		i = 0;
+	}
+
 	igt_assert(thread_count);
+	if (shadertype == SHADER_INV_INSTR_DISABLED)
+		igt_assert(!sip_count);
+	else if (siptype == SIP_INV_INSTR && shadertype != SHADER_INV_INSTR_DISABLED)
+		igt_assert_f(thread_count == sip_count,
+			     "Thread and SIP count mismatch, %d != %d\n",
+			     thread_count, sip_count);
+	else
+		igt_assert(sip_count == 0);
 
 	munmap(ptr, sz);
 }
@@ -143,8 +220,20 @@ xe_sysfs_get_job_timeout_ms(int fd, struct drm_xe_engine_class_instance *eci)
  *
  * SUBTEST: sanity-after-timeout
  * Description: check basic shader execution after job timeout
+ *
+ * SUBTEST: invalidinstr-disabled
+ * Description: Verify that we don't enter SIP after running into an invalid
+ *		instruction when exception is not enabled.
+ *
+ * SUBTEST: invalidinstr-thread-enabled
+ * Description: Verify that we enter SIP after running into an invalid instruction
+ *              when exception is enabled from thread.
+ *
+ * SUBTEST: invalidinstr-walker-enabled
+ * Description: Verify that we enter SIP after running into an invalid instruction
+ *              when exception is enabled from COMPUTE_WALKER.
  */
-static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint32_t flags)
+static void test_sip(int shader, int sip, struct drm_xe_engine_class_instance *eci, uint32_t flags)
 {
 	unsigned int threads = 512;
 	unsigned int height = max_t(threads, HEIGHT, threads * 2);
@@ -171,12 +260,12 @@ static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint3

Likely I missed this in previous patch review, shader should be enum, not int.

--
Zbigniew

Quoted 47 lines
tests/intel/xe_exec_sip.c +14 -5
 	ibb = intel_bb_create_with_context(fd, exec_queue_id, vm_id, NULL, 4096);
 
 	igt_nsec_elapsed(&ts);
-	handle = gpgpu_shader(fd, ibb, shader, threads, width, height);
+	handle = gpgpu_shader(fd, ibb, shader, sip, threads, width, height);
 
 	intel_bb_sync(ibb);
 	igt_assert_lt_u64(igt_nsec_elapsed(&ts), timeout);
 
-	check_buf(fd, handle, width, height, COLOR_C4);
+	check_buf(fd, handle, width, height, shader, sip, COLOR_C4);
 
 	gem_close(fd, handle);
 	intel_bb_destroy(ibb);
@@ -204,17 +293,26 @@ igt_main
 		fd = drm_open_driver(DRIVER_XE);
 
 	test_render_and_compute("sanity", fd, eci)
-		test_sip(SHADER_WRITE, eci, 0);
+		test_sip(SHADER_WRITE, SIP_NULL, eci, 0);
 
 	test_render_and_compute("sanity-after-timeout", fd, eci) {
-		test_sip(SHADER_HANG, eci, 0);
+		test_sip(SHADER_HANG, SIP_NULL, eci, 0);
 
 		xe_for_each_engine(fd, eci)
 			if (eci->engine_class == DRM_XE_ENGINE_CLASS_RENDER ||
 			    eci->engine_class == DRM_XE_ENGINE_CLASS_COMPUTE)
-				test_sip(SHADER_WRITE, eci, 0);
+				test_sip(SHADER_WRITE, SIP_NULL, eci, 0);
 	}
 
+	test_render_and_compute("invalidinstr-disabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_DISABLED, SIP_INV_INSTR, eci, 0);
+
+	test_render_and_compute("invalidinstr-thread-enabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_THREAD_ENABLED, SIP_INV_INSTR, eci, 0);
+
+	test_render_and_compute("invalidinstr-walker-enabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_WALKER_ENABLED, SIP_INV_INSTR, eci, 0);
+
 	igt_fixture
 		drm_close_driver(fd);
 }
Signature · 1 line
2.34.1
#24 lore raw

Re: [PATCH i-g-t v5 09/17] tests/xe_exec_sip: Introduce invalid instruction tests

Hi Zbigniew,

Quoted 66 lines · Zbigniew Kempczyński
Quoted 65 lines · Christoph Manszewski

From: Andrzej Hajda <[email protected]>

Xe2 and earlier gens are able to handle very limited set of invalid instructions - only illegal and undefined opcodes, other errors in instruction can cause undefined behavior.
Illegal/undefined opcode results in:
- setting illegal opcode status bit - cr0.1[28],
- calling SIP if illegal opcode bit is enabled - cr0.1[12]. cr0.1[12] can be enabled directly from the thread or by thread dispatcher from Interface Descriptor Data provided to COMPUTE_WALKER instruction.

Implemented cases:
- check if SIP is not called when exception is not enabled,
- check if SIP is called when exception is enabled from EU thread,
- check if SIP is called when exception is enabled from COMPUTE_WALKER

Cc Mika Kuoppala <[email protected]>
Cc Dominik Grzegorzek <[email protected]>
---
  tests/intel/xe_exec_sip.c | 118 ++++++++++++++++++++++++++++++++++----
  1 file changed, 108 insertions(+), 10 deletions(-)
tests/intel/xe_exec_sip.c +17 -0
index 3b46f5618..5384979a6 100644
@@ -30,12 +30,25 @@
  #define COLOR_C4 0xc4
 
  #define SHADER_CANARY 0x01010101
+#define SIP_CANARY 0x02020202
 
  enum shader_type {
  	SHADER_HANG,
+	SHADER_INV_INSTR_DISABLED,
+	SHADER_INV_INSTR_THREAD_ENABLED,
+	SHADER_INV_INSTR_WALKER_ENABLED,
  	SHADER_WRITE,
  };
 
+enum sip_type {
+	SIP_INV_INSTR,
+	SIP_NULL,
+};
+
+/* Control Register cr0.1 bits for exception handling */
+#define ILLEGAL_OPCODE_ENABLE BIT(12)
+#define ILLEGAL_OPCODE_STATUS BIT(28)
+
  static struct intel_buf *
  create_fill_buf(int fd, int width, int height, uint8_t color)
  {
@@ -58,8 +71,12 @@ create_fill_buf(int fd, int width, int height, uint8_t color)
  static struct gpgpu_shader *get_shader(int fd, const int shadertype)
  {
  	static struct gpgpu_shader *shader;
+	uint32_t bad;
 
  	shader = gpgpu_shader_create(fd);
+	if (shadertype == SHADER_INV_INSTR_WALKER_ENABLED)
+		shader->illegal_opcode_exception_enable = true;
+
  	gpgpu_shader__write_dword(shader, SHADER_CANARY, 0);
 
Quoted 12 lines

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 92

Quoted 8 lines
tests/intel/xe_exec_sip.c +3 -0
  	switch (shadertype) {
@@ -70,19 +87,59 @@ static struct gpgpu_shader *get_shader(int fd, const int shadertype)
  		break;
  	case SHADER_WRITE:
  		break;
+	case SHADER_INV_INSTR_THREAD_ENABLED:
+		gpgpu_shader__set_exception(shader, ILLEGAL_OPCODE_ENABLE);
+		__attribute__ ((fallthrough));

Just be curious - on which gcc you're observing this warning without fallthrough attribute? clang explicitely requires -Wimplicit-fallthrough to enable this warning.

IIRC it was a checkpatch warning/check

Quoted 12 lines

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 100

Quoted 9 lines
tests/intel/xe_exec_sip.c +8 -0
+	case SHADER_INV_INSTR_DISABLED:
+	case SHADER_INV_INSTR_WALKER_ENABLED:
+		bad = (shadertype == SHADER_INV_INSTR_DISABLED) ? ILLEGAL_OPCODE_ENABLE : 0;
+		gpgpu_shader__write_on_exception(shader, 1, 0, ILLEGAL_OPCODE_ENABLE, bad);
+		gpgpu_shader__nop(shader);
+		gpgpu_shader__nop(shader);
+		/* modify second nop, set only opcode bits[6:0] */
+		shader->instr[shader->size / 4 - 1][0] = 0x7f;

This pattern 'shader->size / 4 - 1' deserves own macro or inline like last_instr(shader) or similar.

Ok

Quoted 11 lines

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 107

Quoted 8 lines
tests/intel/xe_exec_sip.c +4 -0
+		/* SIP should clear exception bit */
+		bad = ILLEGAL_OPCODE_STATUS;
+		gpgpu_shader__write_on_exception(shader, 2, 0, ILLEGAL_OPCODE_STATUS, bad);
+		break;
  	}
 
  	gpgpu_shader__eot(shader);

I would add one blank line before return on all functions in which return is glued with the code.

This is not code introduced in this patch nor even in this series.

Quoted 16 lines
Quoted 16 lines
tests/intel/xe_exec_sip.c +11 -1
  	return shader;
  }
 
-static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, const int shadertype,
+static struct gpgpu_shader *get_sip(int fd, const int siptype, unsigned int y_offset)
+{
+	static struct gpgpu_shader *sip;
+
+	if (siptype == SIP_NULL)
+		return NULL;
+
+	sip = gpgpu_shader_create(fd);
+	gpgpu_shader__write_dword(sip, SIP_CANARY, y_offset);
+
+	switch (siptype) {
Quoted 8 lines

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 127

Quoted 6 lines
tests/intel/xe_exec_sip.c +6 -0
+	case SIP_INV_INSTR:
+		gpgpu_shader__write_on_exception(sip, 1, y_offset, ILLEGAL_OPCODE_STATUS, 0);
+		break;
+	}
+
+	gpgpu_shader__end_system_routine(sip, false);

Ditto.

Sure.

Quoted 81 lines
Quoted 8 lines
tests/intel/xe_exec_sip.c +4 -0
+	return sip;
+}
+
+static uint32_t gpgpu_shader(int fd, struct intel_bb *ibb, const int shadertype, const int siptype,
  			     unsigned int threads, unsigned int width, unsigned int height)
  {
  	struct intel_buf *buf = create_fill_buf(fd, width, height, COLOR_C4);

tests/intel/xe_exec_sip.c static struct gpgpu_shader *get_shader(int fd, const int shadertype) · line 143

Quoted 10 lines
tests/intel/xe_exec_sip.c +6 -1
+	struct gpgpu_shader *sip = get_sip(fd, siptype, height / 2);
  	struct gpgpu_shader *shader = get_shader(fd, shadertype);
 
-	gpgpu_shader_exec(ibb, buf, 1, threads, shader, NULL, 0, 0);
+	gpgpu_shader_exec(ibb, buf, 1, threads, shader, sip, 0, 0);
+
+	if (sip)
+		gpgpu_shader_destroy(sip);
+
  	gpgpu_shader_destroy(shader);

Ditto. >

Quoted 61 lines
tests/intel/xe_exec_sip.c +34 -3
  	return buf->handle;
  }
@@ -98,10 +155,10 @@ static void check_fill_buf(uint8_t *ptr, const int width, const int x,
  }
 
  static void check_buf(int fd, uint32_t handle, int width, int height,
-		      uint8_t poison_c)
+		      int shadertype, int siptype, uint8_t poison_c)
  {
  	unsigned int sz = ALIGN(width * height, 4096);
-	int thread_count = 0;
+	int thread_count = 0, sip_count = 0;
  	uint32_t *ptr;
  	int i, j;
 
@@ -119,7 +176,27 @@ static void check_buf(int fd, uint32_t handle, int width, int height,
  		i = 0;
  	}
 
+	for (i = 0, j = height / 2; j < height; ++j) {
+		if (ptr[j * width / 4] == SIP_CANARY) {
+			++sip_count;
+			i = 4;
+		}
+
+		for (; i < width; i++)
+			check_fill_buf((uint8_t *)ptr, width, i, j, poison_c);
+
+		i = 0;
+	}
+
  	igt_assert(thread_count);
+	if (shadertype == SHADER_INV_INSTR_DISABLED)
+		igt_assert(!sip_count);
+	else if (siptype == SIP_INV_INSTR && shadertype != SHADER_INV_INSTR_DISABLED)
+		igt_assert_f(thread_count == sip_count,
+			     "Thread and SIP count mismatch, %d != %d\n",
+			     thread_count, sip_count);
+	else
+		igt_assert(sip_count == 0);
 
  	munmap(ptr, sz);
  }
@@ -143,8 +220,20 @@ xe_sysfs_get_job_timeout_ms(int fd, struct drm_xe_engine_class_instance *eci)
   *
   * SUBTEST: sanity-after-timeout
   * Description: check basic shader execution after job timeout
+ *
+ * SUBTEST: invalidinstr-disabled
+ * Description: Verify that we don't enter SIP after running into an invalid
+ *		instruction when exception is not enabled.
+ *
+ * SUBTEST: invalidinstr-thread-enabled
+ * Description: Verify that we enter SIP after running into an invalid instruction
+ *              when exception is enabled from thread.
+ *
+ * SUBTEST: invalidinstr-walker-enabled
+ * Description: Verify that we enter SIP after running into an invalid instruction
+ *              when exception is enabled from COMPUTE_WALKER.
   */
-static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint32_t flags)
Quoted 8 lines

tests/intel/xe_exec_sip.c static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint3

Quoted 5 lines
tests/intel/xe_exec_sip.c +1 -0
+static void test_sip(int shader, int sip, struct drm_xe_engine_class_instance *eci, uint32_t flags)
  {
  	unsigned int threads = 512;
  	unsigned int height = max_t(threads, HEIGHT, threads * 2);
@@ -171,12 +260,12 @@ static void test_sip(int shader, struct drm_xe_engine_class_instance *eci, uint3

Likely I missed this in previous patch review, shader should be enum, not int.

True, nice catch - thanks.

Christoph

Quoted 51 lines

--
Zbigniew

Quoted 47 lines
tests/intel/xe_exec_sip.c +14 -5
  	ibb = intel_bb_create_with_context(fd, exec_queue_id, vm_id, NULL, 4096);
 
  	igt_nsec_elapsed(&ts);
-	handle = gpgpu_shader(fd, ibb, shader, threads, width, height);
+	handle = gpgpu_shader(fd, ibb, shader, sip, threads, width, height);
 
  	intel_bb_sync(ibb);
  	igt_assert_lt_u64(igt_nsec_elapsed(&ts), timeout);
 
-	check_buf(fd, handle, width, height, COLOR_C4);
+	check_buf(fd, handle, width, height, shader, sip, COLOR_C4);
 
  	gem_close(fd, handle);
  	intel_bb_destroy(ibb);
@@ -204,17 +293,26 @@ igt_main
  		fd = drm_open_driver(DRIVER_XE);
 
  	test_render_and_compute("sanity", fd, eci)
-		test_sip(SHADER_WRITE, eci, 0);
+		test_sip(SHADER_WRITE, SIP_NULL, eci, 0);
 
  	test_render_and_compute("sanity-after-timeout", fd, eci) {
-		test_sip(SHADER_HANG, eci, 0);
+		test_sip(SHADER_HANG, SIP_NULL, eci, 0);
 
  		xe_for_each_engine(fd, eci)
  			if (eci->engine_class == DRM_XE_ENGINE_CLASS_RENDER ||
  			    eci->engine_class == DRM_XE_ENGINE_CLASS_COMPUTE)
-				test_sip(SHADER_WRITE, eci, 0);
+				test_sip(SHADER_WRITE, SIP_NULL, eci, 0);
  	}
 
+	test_render_and_compute("invalidinstr-disabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_DISABLED, SIP_INV_INSTR, eci, 0);
+
+	test_render_and_compute("invalidinstr-thread-enabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_THREAD_ENABLED, SIP_INV_INSTR, eci, 0);
+
+	test_render_and_compute("invalidinstr-walker-enabled", fd, eci)
+		test_sip(SHADER_INV_INSTR_WALKER_ENABLED, SIP_INV_INSTR, eci, 0);
+
  	igt_fixture
  		drm_close_driver(fd);
  }
Signature · 1 line
2.34.1
#25 lore raw

PATCH v5 10/17 drm-uapi/xe: Sync with eudebug uapi

In reply to Christoph Manszewski

From: Dominik Grzegorzek <[email protected]>

Align with kernel commit 09411c6ecbef ("drm/xe/eudebug: Add debug metadata support for xe_eudebug") from:

https://gitlab.freedesktop.org/miku/kernel.git

which introduces most recent changes to the eudebug uapi.

---
 include/drm-uapi/xe_drm_eudebug.h | 341 ++++++++++++++++++++++++++++++
 1 file changed, 341 insertions(+)
 create mode 100644 include/drm-uapi/xe_drm_eudebug.h
include/drm-uapi/xe_drm_eudebug.h new file +341 -0
new file mode 100644
index 000000000..13c069e00
@@ -0,0 +1,341 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#ifndef _XE_DRM_EUDEBUG_H_
+#define _XE_DRM_EUDEBUG_H_
+
+#if defined(__cplusplus)
+extern "C" {
+#endif
+
+/*
+ * The Xe EU debugger extends the uapi by both extending the api for the Xe device as well as
+ * adding an api to use with a separate debugger handle. Currently the KMD part adds the former to
+ * 'xe_drm.h' and the latter to 'xe_drm_eudebug.h'. Since the KMD part is not yet merged upstream,
+ * let's put all eudebug specific uapi here to keep the 'xe_drm.h' file synced with the upstream
+ * kernel.
+ */
+
+/* XXX: BEGIN section moved from xe_drm.h as temporary solution */
+#define DRM_XE_EUDEBUG_CONNECT		0x0c
+#define DRM_XE_DEBUG_METADATA_CREATE	0x0d
+#define DRM_XE_DEBUG_METADATA_DESTROY	0x0e
+
+/* ... */
+
+#define DRM_IOCTL_XE_EUDEBUG_CONNECT		DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_EUDEBUG_CONNECT, struct drm_xe_eudebug_connect)
+#define DRM_IOCTL_XE_DEBUG_METADATA_CREATE	 DRM_IOWR(DRM_COMMAND_BASE + DRM_XE_DEBUG_METADATA_CREATE, struct drm_xe_debug_metadata_create)
+#define DRM_IOCTL_XE_DEBUG_METADATA_DESTROY	 DRM_IOW(DRM_COMMAND_BASE + DRM_XE_DEBUG_METADATA_DESTROY, struct drm_xe_debug_metadata_destroy)
+
+/* ... */
+
+struct drm_xe_vm_bind_op_ext_attach_debug {
+	/** @base: base user extension */
+	struct drm_xe_user_extension base;
+
+	/** @id: Debug object id from create metadata */
+	__u64 metadata_id;
+
+	/** @flags: Flags */
+	__u64 flags;
+
+	/** @cookie: Cookie */
+	__u64 cookie;
+
+	/** @reserved: Reserved */
+	__u64 reserved;
+};
+
+/* ... */
+
+#define XE_VM_BIND_OP_EXTENSIONS_ATTACH_DEBUG 0
+
+/* ... */
+
+#define   DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG		2
+#define     DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE		(1 << 0)
+
+/* ... */
+
+/*
+ * Debugger ABI (ioctl and events) Version History:
+ * 0 - No debugger available
+ * 1 - Initial version
+ */
+#define DRM_XE_EUDEBUG_VERSION 1
+
+struct drm_xe_eudebug_connect {
+	/** @extensions: Pointer to the first extension struct, if any */
+	__u64 extensions;
+
+	__u64 pid; /* input: Target process ID */
+	__u32 flags; /* MBZ */
+
+	__u32 version; /* output: current ABI (ioctl / events) version */
+};
+
+/*
+ * struct drm_xe_debug_metadata_create - Create debug metadata
+ *
+ * Add a region of user memory to be marked as debug metadata.
+ * When the debugger attaches, the metadata regions will be delivered
+ * for debugger. Debugger can then map these regions to help decode
+ * the program state.
+ *
+ * Returns handle to created metadata entry.
+ */
+struct drm_xe_debug_metadata_create {
+	/** @extensions: Pointer to the first extension struct, if any */
+	__u64 extensions;
+
+#define DRM_XE_DEBUG_METADATA_ELF_BINARY     0
+#define DRM_XE_DEBUG_METADATA_PROGRAM_MODULE 1
+#define WORK_IN_PROGRESS_DRM_XE_DEBUG_METADATA_MODULE_AREA 2
+#define WORK_IN_PROGRESS_DRM_XE_DEBUG_METADATA_SBA_AREA 3
+#define WORK_IN_PROGRESS_DRM_XE_DEBUG_METADATA_SIP_AREA 4
+#define WORK_IN_PROGRESS_DRM_XE_DEBUG_METADATA_NUM (1 + \
+	  WORK_IN_PROGRESS_DRM_XE_DEBUG_METADATA_SIP_AREA)
+
+	/** @type: Type of metadata */
+	__u64 type;
+
+	/** @user_addr: pointer to start of the metadata */
+	__u64 user_addr;
+
+	/** @len: length, in bytes of the medata */
+	__u64 len;
+
+	/** @metadata_id: created metadata handle (out) */
+	__u32 metadata_id;
+};
+
+/**
+ * struct drm_xe_debug_metadata_destroy - Destroy debug metadata
+ *
+ * Destroy debug metadata.
+ */
+struct drm_xe_debug_metadata_destroy {
+	/** @extensions: Pointer to the first extension struct, if any */
+	__u64 extensions;
+
+	/** @metadata_id: metadata handle to destroy */
+	__u32 metadata_id;
+};
+
+/* XXX: END section moved from xe_drm.h as temporary solution */
+
+/**
+ * Do a eudebug event read for a debugger connection.
+ *
+ * This ioctl is available in debug version 1.
+ */
+#define DRM_XE_EUDEBUG_IOCTL_READ_EVENT		_IO('j', 0x0)
+#define DRM_XE_EUDEBUG_IOCTL_EU_CONTROL		_IOWR('j', 0x2, struct drm_xe_eudebug_eu_control)
+#define DRM_XE_EUDEBUG_IOCTL_ACK_EVENT		_IOW('j', 0x4, struct drm_xe_eudebug_ack_event)
+#define DRM_XE_EUDEBUG_IOCTL_VM_OPEN		_IOW('j', 0x1, struct drm_xe_eudebug_vm_open)
+#define DRM_XE_EUDEBUG_IOCTL_READ_METADATA	_IOWR('j', 0x3, struct drm_xe_eudebug_read_metadata)
+
+/* XXX: Document events to match their internal counterparts when moved to xe_drm.h */
+struct drm_xe_eudebug_event {
+	__u32 len;
+
+	__u16 type;
+#define DRM_XE_EUDEBUG_EVENT_NONE		0
+#define DRM_XE_EUDEBUG_EVENT_READ		1
+#define DRM_XE_EUDEBUG_EVENT_OPEN		2
+#define DRM_XE_EUDEBUG_EVENT_VM			3
+#define DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE		4
+#define DRM_XE_EUDEBUG_EVENT_EU_ATTENTION	5
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND		6
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP		7
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE	8
+#define DRM_XE_EUDEBUG_EVENT_METADATA		9
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA 10
+
+	__u16 flags;
+#define DRM_XE_EUDEBUG_EVENT_CREATE		(1 << 0)
+#define DRM_XE_EUDEBUG_EVENT_DESTROY		(1 << 1)
+#define DRM_XE_EUDEBUG_EVENT_STATE_CHANGE	(1 << 2)
+#define DRM_XE_EUDEBUG_EVENT_NEED_ACK		(1 << 3)
+
+	__u64 seqno;
+	__u64 reserved;
+};
+
+struct drm_xe_eudebug_event_client {
+	struct drm_xe_eudebug_event base;
+
+	__u64 client_handle; /* This is unique per debug connection */
+};
+
+struct drm_xe_eudebug_event_vm {
+	struct drm_xe_eudebug_event base;
+
+	__u64 client_handle;
+	__u64 vm_handle;
+};
+
+struct drm_xe_eudebug_event_exec_queue {
+	struct drm_xe_eudebug_event base;
+
+	__u64 client_handle;
+	__u64 vm_handle;
+	__u64 exec_queue_handle;
+	__u32 engine_class;
+	__u32 width;
+	__u64 lrc_handle[];
+};
+
+struct drm_xe_eudebug_event_eu_attention {
+	struct drm_xe_eudebug_event base;
+
+	__u64 client_handle;
+	__u64 exec_queue_handle;
+	__u64 lrc_handle;
+	__u32 flags;
+	__u32 bitmask_size;
+	__u8 bitmask[];
+};
+
+struct drm_xe_eudebug_eu_control {
+	__u64 client_handle;
+
+#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_INTERRUPT_ALL	0
+#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_STOPPED		1
+#define DRM_XE_EUDEBUG_EU_CONTROL_CMD_RESUME		2
+	__u32 cmd;
+	__u32 flags;
+
+	__u64 seqno;
+
+	__u64 exec_queue_handle;
+	__u64 lrc_handle;
+	__u32 reserved;
+	__u32 bitmask_size;
+	__u64 bitmask_ptr;
+};
+
+/*
+ *  When client (debuggee) does vm_bind_ioctl() following event
+ *  sequence will be created (for the debugger):
+ *
+ *  ┌───────────────────────┐
+ *  │  EVENT_VM_BIND        ├───────┬─┬─┐
+ *  └───────────────────────┘       │ │ │
+ *      ┌───────────────────────┐   │ │ │
+ *      │ EVENT_VM_BIND_OP #1   ├───┘ │ │
+ *      └───────────────────────┘     │ │
+ *                 ...                │ │
+ *      ┌───────────────────────┐     │ │
+ *      │ EVENT_VM_BIND_OP #n   ├─────┘ │
+ *      └───────────────────────┘       │
+ *                                      │
+ *      ┌───────────────────────┐       │
+ *      │ EVENT_UFENCE          ├───────┘
+ *      └───────────────────────┘
+ *
+ * All the events below VM_BIND will reference the VM_BIND
+ * they associate with, by field .vm_bind_ref_seqno.
+ * event_ufence will only be included if the client did
+ * attach sync of type UFENCE into its vm_bind_ioctl().
+ *
+ * When EVENT_UFENCE is sent by the driver, all the OPs of
+ * the original VM_BIND are completed and the [addr,range]
+ * contained in them are present and modifiable through the
+ * vm accessors. Accessing [addr, range] before related ufence
+ * event will lead to undefined results as the actual bind
+ * operations are async and the backing storage might not
+ * be there on a moment of receiving the event.
+ *
+ * Client's UFENCE sync will be held by the driver: client's
+ * drm_xe_wait_ufence will not complete and the value of the ufence
+ * won't appear until ufence is acked by the debugger process calling
+ * DRM_XE_EUDEBUG_IOCTL_ACK_EVENT with the event_ufence.base.seqno.
+ * This will signal the fence, .value will update and the wait will
+ * complete allowing the client to continue.
+ *
+ */
+
+struct drm_xe_eudebug_event_vm_bind {
+	struct drm_xe_eudebug_event base;
+
+	__u64 client_handle;
+	__u64 vm_handle;
+
+	__u32 flags;
+#define DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE (1 << 0)
+
+	__u32 num_binds;
+};
+
+struct drm_xe_eudebug_event_vm_bind_op {
+	struct drm_xe_eudebug_event base;
+	__u64 vm_bind_ref_seqno; /* *_event_vm_bind.base.seqno */
+	__u64 num_extensions;
+
+	__u64 addr; /* XXX: Zero for unmap all? */
+	__u64 range; /* XXX: Zero for unmap all? */
+};
+
+struct drm_xe_eudebug_event_vm_bind_ufence {
+	struct drm_xe_eudebug_event base;
+	__u64 vm_bind_ref_seqno; /* *_event_vm_bind.base.seqno */
+};
+
+struct drm_xe_eudebug_ack_event {
+	__u32 type;
+	__u32 flags; /* MBZ */
+	__u64 seqno;
+};
+
+struct drm_xe_eudebug_vm_open {
+	/** @extensions: Pointer to the first extension struct, if any */
+	__u64 extensions;
+
+	/** @client_handle: id of client */
+	__u64 client_handle;
+
+	/** @vm_handle: id of vm */
+	__u64 vm_handle;
+
+	/** @flags: flags */
+	__u64 flags;
+
+	/** @timeout_ns: Timeout value in nanoseconds operations (fsync) */
+	__u64 timeout_ns;
+};
+
+struct drm_xe_eudebug_read_metadata {
+	__u64 client_handle;
+	__u64 metadata_handle;
+	__u32 flags;
+	__u32 reserved;
+	__u64 ptr;
+	__u64 size;
+};
+
+struct drm_xe_eudebug_event_metadata {
+	struct drm_xe_eudebug_event base;
+
+	__u64 client_handle;
+	__u64 metadata_handle;
+	/* XXX: Refer to xe_drm.h for fields */
+	__u64 type;
+	__u64 len;
+};
+
+struct drm_xe_eudebug_event_vm_bind_op_metadata {
+	struct drm_xe_eudebug_event base;
+	__u64 vm_bind_op_ref_seqno; /* *_event_vm_bind_op.base.seqno */
+
+	__u64 metadata_handle;
+	__u64 metadata_cookie;
+};
+
+#if defined(__cplusplus)
+}
+#endif
+
+#endif
Signature · 1 line
2.34.1
#26 lore raw

PATCH v5 11/17 lib/xe_eudebug: Introduce eu debug testing framework

In reply to Christoph Manszewski

From: Dominik Grzegorzek <[email protected]>

Introduce library which simplifies testing of eu debug capability. The library provides event log helpers together with asynchronous abstraction for client proccess and the debugger itself.

xe_eudebug_client creates its own proccess with user's work function, and gives machanisms to synchronize beginning of execution and event logging.

xe_eudebug_debugger allows to attach to the given proccess, provides asynchronous thread for event reading and introduces triggers - a callback mechanism triggered every time subscribed event was read.

To build the eudebug testing framework 'xe_eudebug' meson build option has to be enabled, as it is disabled by default.

---
 lib/meson.build     |    5 +
 lib/xe/xe_eudebug.c | 2198 +++++++++++++++++++++++++++++++++++++++++++
 lib/xe/xe_eudebug.h |  212 +++++
 meson.build         |    2 +
 meson_options.txt   |    5 +
 5 files changed, 2422 insertions(+)
 create mode 100644 lib/xe/xe_eudebug.c
 create mode 100644 lib/xe/xe_eudebug.h
lib/meson.build +5 -0
index 4af2bc743..96dec9678 100644
@@ -195,6 +195,11 @@ if chamelium.found()
 	lib_sources += 'monitor_edids/monitor_edids_helper.c'
 endif
 
+if build_xe_eudebug
+	build_info += 'Xe EU debugger test framework enabled.'
+	lib_sources += 'xe/xe_eudebug.c'
+endif
+
 if libprocps.found()
 	lib_deps += libprocps
 else
lib/xe/xe_eudebug.c new file +2198 -0

2,198 lines not on this page · raw message

lib/xe/xe_eudebug.h new file +212 -0

212 lines not on this page · raw message

meson.build +2 -0
index 25bf1e442..207b5819b 100644
@@ -90,9 +90,11 @@ build_chamelium = get_option('chamelium')
 build_docs = get_option('docs')
 build_tests = not get_option('tests').disabled()
 build_xe = not get_option('xe_driver').disabled()
+build_xe_eudebug = get_option('xe_eudebug').enabled()
 with_libdrm = get_option('libdrm_drivers')
 
 build_info = ['Build type: ' + get_option('buildtype')]
+build_info += 'Build Xe EU debugger test framework: @0@'.format(build_xe_eudebug)
 
 inc = include_directories('include', 'include/drm-uapi', 'include/linux-uapi', 'lib', 'lib/stubs/syscalls', '.')
 
meson_options.txt +5 -0
index 6a9493ea6..11922523b 100644
@@ -42,6 +42,11 @@ option('xe_driver',
        value : 'enabled',
        description : 'Build tests for Xe driver')
 
+option('xe_eudebug',
+       type : 'feature',
+       value : 'disabled',
+       description : 'Build library for Xe EU debugger')
+
 option('libdrm_drivers',
        type : 'array',
        value : ['auto'],
Signature · 1 line
2.34.1
#27 lore raw

Re: [PATCH i-g-t v5 11/17] lib/xe_eudebug: Introduce eu debug testing framework

Quoted 10 lines · Christoph Manszewski

From: Dominik Grzegorzek <[email protected]>

Introduce library which simplifies testing of eu debug capability. The library provides event log helpers together with asynchronous abstraction for client proccess and the debugger itself.

xe_eudebug_client creates its own proccess with user's work function, and gives machanisms to synchronize beginning of execution and event logging.

Quoted 6 lines

xe_eudebug_debugger allows to attach to the given proccess, provides asynchronous thread for event reading and introduces triggers - a callback mechanism triggered every time subscribed event was read.

To build the eudebug testing framework 'xe_eudebug' meson build option has to be enabled, as it is disabled by default.

My review might not catch all the issues, there's +2K line of codes so I might overlook something. But I think you should at least address things I've spotted.

Quoted 134 lines
---
 lib/meson.build     |    5 +
 lib/xe/xe_eudebug.c | 2198 +++++++++++++++++++++++++++++++++++++++++++
 lib/xe/xe_eudebug.h |  212 +++++
 meson.build         |    2 +
 meson_options.txt   |    5 +
 5 files changed, 2422 insertions(+)
 create mode 100644 lib/xe/xe_eudebug.c
 create mode 100644 lib/xe/xe_eudebug.h
lib/meson.build +5 -0
index 4af2bc743..96dec9678 100644
@@ -195,6 +195,11 @@ if chamelium.found()
 	lib_sources += 'monitor_edids/monitor_edids_helper.c'
 endif
 
+if build_xe_eudebug
+	build_info += 'Xe EU debugger test framework enabled.'
+	lib_sources += 'xe/xe_eudebug.c'
+endif
+
 if libprocps.found()
 	lib_deps += libprocps
 else
lib/xe/xe_eudebug.c new file +95 -0
new file mode 100644
index 000000000..2a804222a
@@ -0,0 +1,2198 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include <fcntl.h>
+#include <poll.h>
+#include <signal.h>
+#include <sys/select.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include "igt.h"
+#include "igt_sysfs.h"
+#include "intel_pat.h"
+#include "xe_eudebug.h"
+#include "xe_ioctl.h"
+
+struct event_trigger {
+	xe_eudebug_trigger_fn fn;
+	int type;
+	struct igt_list_head link;
+};
+
+struct seqno_list_entry {
+	struct igt_list_head link;
+	uint64_t seqno;
+};
+
+struct match_dto {
+	struct drm_xe_eudebug_event *target;
+	struct igt_list_head *seqno_list;
+	uint64_t client_handle;
+	uint32_t filter;
+
+	/* store latest 'EVENT_VM_BIND' seqno */
+	uint64_t *bind_seqno;
+	/* latest vm_bind_op seqno matching bind_seqno */
+	uint64_t *bind_op_seqno;
+};
+
+#define CLIENT_PID  1
+#define CLIENT_RUN  2
+#define CLIENT_FINI 3
+#define CLIENT_STOP 4
+#define CLIENT_STAGE 5
+#define DEBUGGER_STAGE 6
+
+#define DEBUGGER_WORKER_INACTIVE  0
+#define DEBUGGER_WORKER_ACTIVE  1
+#define DEBUGGER_WORKER_QUITTING 2
+
+static const char *type_to_str(unsigned int type)
+{
+	switch (type) {
+	case DRM_XE_EUDEBUG_EVENT_NONE:
+		return "none";
+	case DRM_XE_EUDEBUG_EVENT_READ:
+		return "read";
+	case DRM_XE_EUDEBUG_EVENT_OPEN:
+		return "client";
+	case DRM_XE_EUDEBUG_EVENT_VM:
+		return "vm";
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE:
+		return "exec_queue";
+	case DRM_XE_EUDEBUG_EVENT_EU_ATTENTION:
+		return "attention";
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND:
+		return "vm_bind";
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP:
+		return "vm_bind_op";
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE:
+		return "vm_bind_ufence";
+	case DRM_XE_EUDEBUG_EVENT_METADATA:
+		return "metadata";
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA:
+		return "vm_bind_op_metadata";
+	}
+
+	return "UNKNOWN";
+}
+
+static const char *event_type_to_str(struct drm_xe_eudebug_event *e, char *buf)
+{
+	sprintf(buf, "%s(%d)", type_to_str(e->type), e->type);
+
+	return buf;
+}
+
+static const char *flags_to_str(unsigned int flags)
+{
+	if (flags & DRM_XE_EUDEBUG_EVENT_CREATE) {
+		if (flags & DRM_XE_EUDEBUG_EVENT_NEED_ACK)
+			return "create|ack";

lib/xe/xe_eudebug.c line 105

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+		else
+			return "create";
+	}
+	if (flags & DRM_XE_EUDEBUG_EVENT_DESTROY)
+		return "destroy";
+
+	if (flags & DRM_XE_EUDEBUG_EVENT_STATE_CHANGE)
+		return "state-change";
+
+	igt_assert(!(flags & DRM_XE_EUDEBUG_EVENT_NEED_ACK));

Does it mean ACK is only valid for CREATE event?

lib/xe/xe_eudebug.c line 110

Quoted 5 lines
lib/xe/xe_eudebug.c new file +5 -0
+
+	return "flags unknown";
+}
+
+static const char *event_members_to_str(struct drm_xe_eudebug_event *e, char *b)

For being consistent with above just minor nitpick, s/b/buf/

Quoted 233 lines
lib/xe/xe_eudebug.c new file +233 -0
+{
+	switch (e->type) {
+	case DRM_XE_EUDEBUG_EVENT_OPEN: {
+		struct drm_xe_eudebug_event_client *ec = (struct drm_xe_eudebug_event_client *)e;
+
+		sprintf(b, "handle=%llu", ec->client_handle);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM: {
+		struct drm_xe_eudebug_event_vm *evm = (struct drm_xe_eudebug_event_vm *)e;
+
+		sprintf(b, "client_handle=%llu, handle=%llu",
+			evm->client_handle, evm->vm_handle);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE: {
+		struct drm_xe_eudebug_event_exec_queue *ee = (void *)e;
+
+		sprintf(b, "client_handle=%llu, vm_handle=%llu, "
+			   "exec_queue_handle=%llu, engine_class=%d, exec_queue_width=%d",
+			ee->client_handle, ee->vm_handle,
+			ee->exec_queue_handle, ee->engine_class, ee->width);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_EU_ATTENTION: {
+		struct drm_xe_eudebug_event_eu_attention *ea = (void *)e;
+
+		sprintf(b, "client_handle=%llu, exec_queue_handle=%llu, "
+			   "lrc_handle=%llu, bitmask_size=%d",
+			ea->client_handle, ea->exec_queue_handle,
+			ea->lrc_handle, ea->bitmask_size);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND: {
+		struct drm_xe_eudebug_event_vm_bind *evmb = (void *)e;
+
+		sprintf(b, "client_handle=%llu, vm_handle=%llu, flags=0x%x, num_binds=%u",
+			evmb->client_handle, evmb->vm_handle, evmb->flags, evmb->num_binds);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP: {
+		struct drm_xe_eudebug_event_vm_bind_op *op = (void *)e;
+
+		sprintf(b, "vm_bind_ref_seqno=%lld, addr=%016llx, range=%llu num_extensions=%llu",
+			op->vm_bind_ref_seqno, op->addr, op->range, op->num_extensions);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE: {
+		struct drm_xe_eudebug_event_vm_bind_ufence *f = (void *)e;
+
+		sprintf(b, "vm_bind_ref_seqno=%lld", f->vm_bind_ref_seqno);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_METADATA: {
+		struct drm_xe_eudebug_event_metadata *em = (void *)e;
+
+		sprintf(b, "client_handle=%llu, metadata_handle=%llu, type=%llu, len=%llu",
+			em->client_handle, em->metadata_handle, em->type, em->len);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA: {
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *op = (void *)e;
+
+		sprintf(b, "vm_bind_op_ref_seqno=%lld, metadata_handle=%llu, metadata_cookie=%llu",
+			op->vm_bind_op_ref_seqno, op->metadata_handle, op->metadata_cookie);
+		break;
+	}
+	default:
+		strcpy(b, "<...>");
+	}
+
+	return b;
+}
+
+/**
+ * xe_eudebug_event_to_str:
+ * @e: pointer to event
+ * @buf: target to write string representation of @e
+ * @len: size of target buffer @buf
+ *
+ * Creates string representation for given event.
+ *
+ * Returns: the written input buffer pointed by @buf.
+ */
+const char *xe_eudebug_event_to_str(struct drm_xe_eudebug_event *e, char *buf, size_t len)
+{
+	char a[256];
+	char b[256];
+
+	snprintf(buf, len, "(%llu) %15s:%s: %s",
+		 e->seqno,
+		 event_type_to_str(e, a),
+		 flags_to_str(e->flags),
+		 event_members_to_str(e, b));
+
+	return buf;
+}
+
+static void catch_child_failure(void)
+{
+	pid_t pid;
+	int status;
+
+	pid = waitpid(-1, &status, WNOHANG);
+
+	if (pid == 0 || pid == -1)
+		return;
+
+	if (!WIFEXITED(status))
+		return;
+
+	igt_assert_f(WEXITSTATUS(status) == 0, "Client failed!\n");
+}
+
+static int safe_pipe_read(int pipe[2], void *buf, int nbytes, int timeout_ms)
+{
+	int ret;
+	int t = 0;
+	struct pollfd fd = {
+		.fd = pipe[0],
+		.events = POLLIN,
+		.revents = 0
+	};
+
+	/* When child fails we may get stuck forever. Check whether
+	 * the child process ended with an error.
+	 */
+	do {
+		const int interval_ms = 1000;
+
+		ret = poll(&fd, 1, interval_ms);
+
+		if (!ret) {
+			catch_child_failure();
+			t += interval_ms;
+		}
+	} while (!ret && t < timeout_ms);
+
+	if (ret > 0)
+		return read(pipe[0], buf, nbytes);
+
+	return 0;
+}
+
+static uint64_t pipe_read(int pipe[2], int timeout_ms)
+{
+	uint64_t in;
+	uint64_t ret;
+
+	ret = safe_pipe_read(pipe, &in, sizeof(in), timeout_ms);
+	igt_assert(ret == sizeof(in));
+
+	return in;
+}
+
+static void pipe_signal(int pipe[2], uint64_t token)
+{
+	igt_assert(write(pipe[1], &token, sizeof(token)) == sizeof(token));
+}
+
+static void pipe_close(int pipe[2])
+{
+	if (pipe[0] != -1)
+		close(pipe[0]);
+
+	if (pipe[1] != -1)
+		close(pipe[1]);
+}
+
+static uint64_t __wait_token(int p[2], const uint64_t token, int timeout_ms)
+{
+	uint64_t in;
+
+	in = pipe_read(p, timeout_ms);
+
+	igt_assert_eq(in, token);
+
+	return pipe_read(p, timeout_ms);
+}
+
+static uint64_t client_wait_token(struct xe_eudebug_client *c, const uint64_t token)
+{
+	return __wait_token(c->p_in, token, c->timeout_ms);
+}
+
+static uint64_t wait_from_client(struct xe_eudebug_client *c, const uint64_t token)
+{
+	return __wait_token(c->p_out, token, c->timeout_ms);
+}
+
+static void token_signal(int p[2], const uint64_t token, const uint64_t value)
+{
+	pipe_signal(p, token);
+	pipe_signal(p, value);
+}
+
+static void client_signal(struct xe_eudebug_client *c,
+			  const uint64_t token,
+			  const uint64_t value)
+{
+	token_signal(c->p_out, token, value);
+}
+
+static int __xe_eudebug_connect(int fd, pid_t pid, uint32_t flags, uint64_t events)
+{
+	struct drm_xe_eudebug_connect param = {
+		.pid = pid,
+		.flags = flags,
+	};
+	int debugfd;
+
+	debugfd = igt_ioctl(fd, DRM_IOCTL_XE_EUDEBUG_CONNECT, &param);
+
+	if (debugfd < 0)
+		return -errno;
+
+	return debugfd;
+}
+
+static void event_log_write_to_fd(struct xe_eudebug_event_log *l, int fd)
+{
+	igt_assert_eq(write(fd, &l->head, sizeof(l->head)),
+		      sizeof(l->head));
+
+	igt_assert_eq(write(fd, l->log, l->head), l->head);
+}
+
+static void read_all(int fd, void *buf, size_t nbytes)
+{
+	ssize_t remaining_size = nbytes;
+	ssize_t current_size = 0;
+	ssize_t read_size = 0;
+

lib/xe/xe_eudebug.c line 353

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+	do {
+		read_size = read(fd, buf + current_size, remaining_size);
+		igt_assert_f(read_size >= 0, "read failed: %s\n", strerror(errno));
+
+		current_size += read_size;
+		remaining_size -= read_size;
+	} while (remaining_size > 0 && read_size > 0);
+
+	igt_assert_eq(current_size, nbytes);
+}

I see fd wasn't set to nonblock. Isn't

igt_assert_eq(read(fd, buf, nbytes), nbytes) enough?

lib/xe/xe_eudebug.c line 363

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+
+static void event_log_read_from_fd(struct xe_eudebug_event_log *l, int fd)
+{
+	read_all(fd, &l->head, sizeof(l->head));
+	igt_assert_lt(l->head, l->max_size);
+
+	read_all(fd, l->log, l->head);
+}
+
+typedef int (*cmp_fn_t)(struct drm_xe_eudebug_event *, void *);

I wondered why void * here, but I see you have different compare functions below.

lib/xe/xe_eudebug.c line 373

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+
+static struct drm_xe_eudebug_event *
+event_cmp(struct xe_eudebug_event_log *l,
+	  struct drm_xe_eudebug_event *current,
+	  cmp_fn_t match,
+	  void *data)
+{
+	struct drm_xe_eudebug_event *e = current;
+
+	xe_eudebug_for_each_event(e, l) {

I've quickly took a look to macro definition and it is confusing 'for_each_event'. It seems it depends on e which really goes over all events if current is NULL. Maybe it would be worth to introduce xe_eudebug_for_each_event_startint_at macro to avoid depending on e value before the start?

Quoted 73 lines
lib/xe/xe_eudebug.c new file +73 -0
+		if (match(e, data))
+			return e;
+	}
+
+	return NULL;
+}
+
+static int match_type_and_flags(struct drm_xe_eudebug_event *a, void *data)
+{
+	struct drm_xe_eudebug_event *b = data;
+
+	if (a->type == b->type &&
+	    a->flags == b->flags)
+		return 1;
+
+	return 0;
+}
+
+static int match_fields(struct drm_xe_eudebug_event *a, void *data)
+{
+	struct drm_xe_eudebug_event *b = data;
+	int ret = 0;
+
+	ret = match_type_and_flags(a, data);
+	if (!ret)
+		return ret;
+
+	ret = 0;
+
+	switch (a->type) {
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE: {
+		struct drm_xe_eudebug_event_exec_queue *ae = (void *)a;
+		struct drm_xe_eudebug_event_exec_queue *be = (void *)b;
+
+		if (ae->engine_class == be->engine_class && ae->width == be->width)
+			ret = 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND: {
+		struct drm_xe_eudebug_event_vm_bind *ea = (void *)a;
+		struct drm_xe_eudebug_event_vm_bind *eb = (void *)b;
+
+		if (ea->num_binds == eb->num_binds)
+			ret = 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP: {
+		struct drm_xe_eudebug_event_vm_bind_op *ea = (void *)a;
+		struct drm_xe_eudebug_event_vm_bind_op *eb = (void *)b;
+
+		if (ea->addr == eb->addr && ea->range == eb->range &&
+		    ea->num_extensions == eb->num_extensions)
+			ret = 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA: {
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *ea = (void *)a;
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *eb = (void *)b;
+
+		if (ea->metadata_handle == eb->metadata_handle &&
+		    ea->metadata_cookie == eb->metadata_cookie)
+			ret = 1;
+		break;
+	}
+
+	default:
+		ret = 1;
+		break;
+	}
+
+	return ret;
+}
+

lib/xe/xe_eudebug.c line 456

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+static int match_client_handle(struct drm_xe_eudebug_event *e, void *data)
+{
+	struct match_dto *md = (void *)data;
+	uint64_t *bind_seqno = md->bind_seqno;
+	uint64_t *bind_op_seqno = md->bind_op_seqno;
+	uint64_t h = md->client_handle;
+
+	if (XE_EUDEBUG_EVENT_IS_FILTERED(e->type, md->filter))
+		return 0;
+

Instead of long line casting I would temporary add

#define xe_event_cast_init(__t, __ev, __e) \
	struct drm_xe_eudebug_event_##__t __ev = (struct drm_xe_eudebug_event_##__t)__e

lib/xe/xe_eudebug.c line 459

Quoted 3 lines
lib/xe/xe_eudebug.c new file +3 -0
+	switch (e->type) {
+	case DRM_XE_EUDEBUG_EVENT_OPEN: {
+		struct drm_xe_eudebug_event_client *client = (struct drm_xe_eudebug_event_client *)e;

and use

xe_event_cast_init(client *, client, e);

lib/xe/xe_eudebug.c line 466

Quoted 7 lines
lib/xe/xe_eudebug.c new file +7 -0
+
+		if (client->client_handle == h)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM: {
+		struct drm_xe_eudebug_event_vm *vm = (struct drm_xe_eudebug_event_vm *)e;

xe_event_cast_init(vm *, vm, e);

and so on. Finally undefine, but I see cast pattern is widely used here.

Quoted 215 lines
lib/xe/xe_eudebug.c new file +215 -0
+
+		if (vm->client_handle == h)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE: {
+		struct drm_xe_eudebug_event_exec_queue *ee = (struct drm_xe_eudebug_event_exec_queue *)e;
+
+		if (ee->client_handle == h)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND: {
+		struct drm_xe_eudebug_event_vm_bind *evmb = (struct drm_xe_eudebug_event_vm_bind *)e;
+
+		if (evmb->client_handle == h) {
+			*bind_seqno = evmb->base.seqno;
+			return 1;
+		}
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP: {
+		struct drm_xe_eudebug_event_vm_bind_op *eo = (struct drm_xe_eudebug_event_vm_bind_op *)e;
+
+		if (eo->vm_bind_ref_seqno == *bind_seqno) {
+			*bind_op_seqno = eo->base.seqno;
+			return 1;
+		}
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE: {
+		struct drm_xe_eudebug_event_vm_bind_ufence *ef = (struct drm_xe_eudebug_event_vm_bind_ufence *)e;
+
+		if (ef->vm_bind_ref_seqno == *bind_seqno)
+			return 1;
+
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_METADATA: {
+		struct drm_xe_eudebug_event_metadata *em = (struct drm_xe_eudebug_event_metadata *)e;
+
+		if (em->client_handle == h)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA: {
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *eo = (struct drm_xe_eudebug_event_vm_bind_op_metadata *)e;
+
+		if (eo->vm_bind_op_ref_seqno == *bind_op_seqno)
+			return 1;
+		break;
+	}
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+static int match_opposite_resource(struct drm_xe_eudebug_event *e, void *data)
+{
+	struct drm_xe_eudebug_event *d = (void *)data;
+	int ret;
+
+	d->flags ^= DRM_XE_EUDEBUG_EVENT_CREATE | DRM_XE_EUDEBUG_EVENT_DESTROY;
+	d->flags &= ~(DRM_XE_EUDEBUG_EVENT_NEED_ACK);
+	ret = match_type_and_flags(e, data);
+	d->flags ^= DRM_XE_EUDEBUG_EVENT_CREATE | DRM_XE_EUDEBUG_EVENT_DESTROY;
+
+	if (!ret)
+		return 0;
+
+	switch (e->type) {
+	case DRM_XE_EUDEBUG_EVENT_OPEN: {
+		struct drm_xe_eudebug_event_client *client = (struct drm_xe_eudebug_event_client *)e;
+		struct drm_xe_eudebug_event_client *filter = (struct drm_xe_eudebug_event_client *)data;
+
+		if (client->client_handle == filter->client_handle)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM: {
+		struct drm_xe_eudebug_event_vm *vm = (struct drm_xe_eudebug_event_vm *)e;
+		struct drm_xe_eudebug_event_vm *filter = (struct drm_xe_eudebug_event_vm *)data;
+
+		if (vm->vm_handle == filter->vm_handle)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE: {
+		struct drm_xe_eudebug_event_exec_queue *ee = (void *)e;
+		struct drm_xe_eudebug_event_exec_queue *filter = (struct drm_xe_eudebug_event_exec_queue *)data;
+
+		if (ee->exec_queue_handle == filter->exec_queue_handle)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND: {
+		struct drm_xe_eudebug_event_vm_bind *evmb = (void *)e;
+		struct drm_xe_eudebug_event_vm_bind *filter = (struct drm_xe_eudebug_event_vm_bind *)data;
+
+		if (evmb->vm_handle == filter->vm_handle &&
+		    evmb->num_binds == filter->num_binds)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP: {
+		struct drm_xe_eudebug_event_vm_bind_op *avmb = (void *)e;
+		struct drm_xe_eudebug_event_vm_bind_op *filter = (struct drm_xe_eudebug_event_vm_bind_op *)data;
+
+		if (avmb->addr == filter->addr &&
+		    avmb->range == filter->range)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_METADATA: {
+		struct drm_xe_eudebug_event_metadata *em = (void *)e;
+		struct drm_xe_eudebug_event_metadata *filter = (struct drm_xe_eudebug_event_metadata *)data;
+
+		if (em->metadata_handle == filter->metadata_handle)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA: {
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *avmb = (void *)e;
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *filter = (struct drm_xe_eudebug_event_vm_bind_op_metadata *)data;
+
+		if (avmb->metadata_handle == filter->metadata_handle &&
+		    avmb->metadata_cookie == filter->metadata_cookie)
+			return 1;
+		break;
+	}
+
+	default:
+		break;
+	}
+	return 0;
+}
+
+static int match_full(struct drm_xe_eudebug_event *e, void *data)
+{
+	struct seqno_list_entry *sl;
+
+	struct match_dto *md = (void *)data;
+	int ret = 0;
+
+	ret = match_client_handle(e, md);
+	if (!ret)
+		return 0;
+
+	ret = match_fields(e, md->target);
+	if (!ret)
+		return 0;
+
+	igt_list_for_each_entry(sl, md->seqno_list, link) {
+		if (sl->seqno == e->seqno)
+			return 0;
+	}
+
+	return 1;
+}
+
+static struct drm_xe_eudebug_event *
+event_type_match(struct xe_eudebug_event_log *l,
+		 struct drm_xe_eudebug_event *target,
+		 struct drm_xe_eudebug_event *current)
+{
+	return event_cmp(l, current, match_type_and_flags, target);
+}
+
+static struct drm_xe_eudebug_event *
+client_match(struct xe_eudebug_event_log *l,
+	     uint64_t client_handle,
+	     struct drm_xe_eudebug_event *current,
+	     uint32_t filter,
+	     uint64_t *bind_seqno,
+	     uint64_t *bind_op_seqno)
+{
+	struct match_dto md = {
+		.client_handle = client_handle,
+		.filter = filter,
+		.bind_seqno = bind_seqno,
+		.bind_op_seqno = bind_op_seqno,
+	};
+
+	return event_cmp(l, current, match_client_handle, &md);
+}
+
+static struct drm_xe_eudebug_event *
+opposite_event_match(struct xe_eudebug_event_log *l,
+		     struct drm_xe_eudebug_event *target,
+		     struct drm_xe_eudebug_event *current)
+{
+	return event_cmp(l, current, match_opposite_resource, target);
+}
+
+static struct drm_xe_eudebug_event *
+event_match(struct xe_eudebug_event_log *l,
+	    struct drm_xe_eudebug_event *target,
+	    uint64_t client_handle,
+	    struct igt_list_head *seqno_list,
+	    uint64_t *bind_seqno,
+	    uint64_t *bind_op_seqno)
+{
+	struct match_dto md = {
+		.target = target,
+		.client_handle = client_handle,
+		.seqno_list = seqno_list,
+		.bind_seqno = bind_seqno,
+		.bind_op_seqno = bind_op_seqno,
+	};
+
+	return event_cmp(l, NULL, match_full, &md);
+}
+

lib/xe/xe_eudebug.c line 691

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+static void compare_client(struct xe_eudebug_event_log *c, struct drm_xe_eudebug_event *_ce,
+			   struct xe_eudebug_event_log *d, struct drm_xe_eudebug_event *_de,
+			   uint32_t filter)
+{
+	struct drm_xe_eudebug_event_client *ce = (void *)_ce;
+	struct drm_xe_eudebug_event_client *de = (void *)_de;
+	uint64_t cbs = 0, dbs = 0, cbso = 0, dbso = 0;
+
+	struct igt_list_head matched_seqno_list;
+	struct drm_xe_eudebug_event *hc, *hd;

Variable names are not self-descriptive. Are you able to guess what c/d/ce/de/hc/hd means?

Quoted 61 lines
lib/xe/xe_eudebug.c new file +61 -0
+	struct seqno_list_entry *entry, *tmp;
+
+	igt_assert(ce);
+	igt_assert(de);
+
+	igt_debug("client: %llu -> %llu\n", ce->client_handle, de->client_handle);
+
+	hc = NULL;
+	hd = NULL;
+	IGT_INIT_LIST_HEAD(&matched_seqno_list);
+
+	do {
+		hc = client_match(c, ce->client_handle, hc, filter, &cbs, &cbso);
+		if (!hc)
+			break;
+
+		hd = event_match(d, hc, de->client_handle, &matched_seqno_list, &dbs, &dbso);
+
+		igt_assert_f(hd, "%s (%llu): no matching event type %u found for client %llu\n",
+			     c->name,
+			     hc->seqno,
+			     hc->type,
+			     ce->client_handle);
+
+		igt_debug("comparing %s %llu vs %s %llu\n",
+			  c->name, hc->seqno, d->name, hd->seqno);
+
+		/*
+		 * Store the seqno of the event that was matched above,
+		 * inside 'matched_seqno_list', to avoid it getting matched
+		 * by subsequent 'event_match' calls.
+		 */
+		entry = malloc(sizeof(*entry));
+		entry->seqno = hd->seqno;
+		igt_list_add(&entry->link, &matched_seqno_list);
+	} while (hc);
+
+	igt_list_for_each_entry_safe(entry, tmp, &matched_seqno_list, link)
+		free(entry);
+}
+
+/**
+ * xe_eudebug_event_log_find_seqno:
+ * @l: event log pointer
+ * @seqno: seqno of event to be found
+ *
+ * Finds the event with given seqno in the event log.
+ *
+ * Returns: pointer to the event with given seqno within @l or NULL seqno is
+ * not present.
+ */
+struct drm_xe_eudebug_event *
+xe_eudebug_event_log_find_seqno(struct xe_eudebug_event_log *l, uint64_t seqno)
+{
+	struct drm_xe_eudebug_event *e = NULL, *found = NULL;
+
+	igt_assert_neq(seqno, 0);
+	/*
+	 * Try to catch if seqno is corrupted and prevent too long tests,
+	 * as our post processing of events is not optimized.
+	 */

lib/xe/xe_eudebug.c line 762

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+	igt_assert_lt(seqno, 10 * 1000 * 1000);
+
+	xe_eudebug_for_each_event(e, l) {
+		if (e->seqno == seqno) {
+			if (found) {
+				igt_warn("Found multiple events with the same seqno %lu\n", seqno);
+				xe_eudebug_event_log_print(l, false);
+				igt_assert(!found);
+			}
+			found = e;

Do I understand correctly that you're trying to catch kernel bug here in the library? Shouldn't this be addressed in the kernel or some eudebug test itself, but not in the library?

Quoted 105 lines
lib/xe/xe_eudebug.c new file +105 -0
+		}
+	}
+
+	return found;
+}
+
+static void event_log_sort(struct xe_eudebug_event_log *l)
+{
+	struct xe_eudebug_event_log *tmp;
+	struct drm_xe_eudebug_event *e = NULL;
+	uint64_t first_seqno = 0;
+	uint64_t last_seqno = 0;
+	uint64_t events = 0, added = 0;
+	uint64_t i;
+
+	xe_eudebug_for_each_event(e, l) {
+		if (e->seqno > last_seqno)
+			last_seqno = e->seqno;
+
+		if (e->seqno < first_seqno)
+			first_seqno = e->seqno;
+
+		events++;
+	}
+
+	tmp = xe_eudebug_event_log_create("tmp", l->max_size);
+
+	for (i = 1; i <= last_seqno; i++) {
+		e = xe_eudebug_event_log_find_seqno(l, i);
+		if (e) {
+			xe_eudebug_event_log_write(tmp, e);
+			added++;
+		}
+	}
+
+	igt_assert_eq(events, added);
+	igt_assert_eq(tmp->head, l->head);
+
+	memcpy(l->log, tmp->log, tmp->head);
+
+	xe_eudebug_event_log_destroy(tmp);
+}
+
+/**
+ * xe_eudebug_connect:
+ * @fd: Xe file descriptor
+ * @pid: client PID
+ * @flags: connection flags
+ *
+ * Opens the xe eu debugger connection to the process described by @pid
+ *
+ * Returns: 0 if the debugger was successfully attached, -errno otherwise.
+ */
+int xe_eudebug_connect(int fd, pid_t pid, uint32_t flags)
+{
+	int ret;
+	uint64_t events = 0; /* events filtering not supported yet! */
+
+	ret = __xe_eudebug_connect(fd, pid, flags, events);
+
+	return ret;
+}
+
+/**
+ * xe_eudebug_event_log_create:
+ * @name: event log identifier
+ * @max_size: maximum size of created log
+ *
+ * Function creates an Eu Debugger event log with size equal to @max_size.
+ *
+ * Returns: pointer to just created log
+ */
+#define MAX_EVENT_LOG_SIZE (32 * 1024 * 1024)
+struct xe_eudebug_event_log *xe_eudebug_event_log_create(const char *name, unsigned int max_size)
+{
+	struct xe_eudebug_event_log *l;
+
+	l = calloc(1, sizeof(*l));
+	igt_assert(l);
+	l->log = calloc(1, max_size);
+	igt_assert(l->log);
+	l->max_size = max_size;
+	strncpy(l->name, name, sizeof(l->name) - 1);
+	pthread_mutex_init(&l->lock, NULL);
+
+	return l;
+}
+
+/**
+ * xe_eudebug_event_log_destroy:
+ * @l: event log pointer
+ *
+ * Frees given event log @l.
+ */
+void xe_eudebug_event_log_destroy(struct xe_eudebug_event_log *l)
+{
+	pthread_mutex_destroy(&l->lock);
+	free(l->log);
+	free(l);
+}
+
+/**
+ * xe_eudebug_event_log_write:
+ * @l: event log pointer
+ * @e: event to be written to event log

lib/xe/xe_eudebug.c line 877

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ *
+ * Writes event @e to the event log, thread-safe.
+ */
+void xe_eudebug_event_log_write(struct xe_eudebug_event_log *l, struct drm_xe_eudebug_event *e)
+{
+	igt_assert(e->seqno);
+	/*
+	 * Try to catch if seqno is corrupted and prevent too long tests,
+	 * as our post processing of events is not optimized.
+	 */

What means seqno is corrupted? Kernel issues deserve for own test, not library part.

Quoted 38 lines
lib/xe/xe_eudebug.c new file +38 -0
+	igt_assert_lt(e->seqno, 10 * 1000 * 1000);
+
+	pthread_mutex_lock(&l->lock);
+	igt_assert_lt(l->head + e->len, l->max_size);
+	memcpy(l->log + l->head, e, e->len);
+	l->head += e->len;
+	pthread_mutex_unlock(&l->lock);
+}
+
+/**
+ * xe_eudebug_event_log_print:
+ * @l: event log pointer
+ * @debug: when true function uses igt_debug instead of igt_info.
+ *
+ * Prints given event log.
+ */
+void
+xe_eudebug_event_log_print(struct xe_eudebug_event_log *l, bool debug)
+{
+	struct drm_xe_eudebug_event *e = NULL;
+	int level = debug ? IGT_LOG_DEBUG : IGT_LOG_INFO;
+	char str[XE_EUDEBUG_EVENT_STRING_MAX_LEN];
+
+	igt_log(IGT_LOG_DOMAIN, level,
+		"event log '%s' (%u bytes):\n", l->name, l->head);
+
+	xe_eudebug_for_each_event(e, l) {
+		xe_eudebug_event_to_str(e, str, XE_EUDEBUG_EVENT_STRING_MAX_LEN);
+		igt_log(IGT_LOG_DOMAIN, level, "%s\n", str);
+	}
+}
+
+/**
+ * xe_eudebug_event_log_compare:
+ * @a: event log pointer
+ * @b: event log pointer
+ * @filter: mask that represents events to be skipped during comparison, useful
+ * for events like 'VM_BIND' since they can be asymmetric. Note that

lib/xe/xe_eudebug.c line 925

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ * 'DRM_XE_EUDEBUG_EVENT_OPEN' will always be matched.
+ *
+ * Compares and asserts event logs @a, @b if the event
+ * sequence matches.
+ */
+void xe_eudebug_event_log_compare(struct xe_eudebug_event_log *a, struct xe_eudebug_event_log *b,
+				  uint32_t filter)
+{
+	struct drm_xe_eudebug_event *ae = NULL;
+	struct drm_xe_eudebug_event *be = NULL;

I'm not happy with variable naming here as well.

Quoted 15 lines
lib/xe/xe_eudebug.c new file +15 -0
+
+	xe_eudebug_for_each_event(ae, a) {
+		if (ae->type == DRM_XE_EUDEBUG_EVENT_OPEN &&
+		    ae->flags & DRM_XE_EUDEBUG_EVENT_CREATE) {
+			be = event_type_match(b, ae, be);
+
+			compare_client(a, ae, b, be, filter);
+			compare_client(b, be, a, ae, filter);
+		}
+	}
+}
+
+/**
+ * xe_eudebug_event_log_match_opposite:
+ * @l: event log pointer

lib/xe/xe_eudebug.c line 950

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ * @filter: mask that represents events to be skipped during comparison, useful
+ * for events like 'VM_BIND' since they can be asymmetric
+ *
+ * Matches and asserts content of all opposite events (create vs destroy).
+ */
+void
+xe_eudebug_event_log_match_opposite(struct xe_eudebug_event_log *l, uint32_t filter)
+{
+	struct drm_xe_eudebug_event *ce = NULL;
+	struct drm_xe_eudebug_event *de = NULL;

Ditto.

Quoted 87 lines
lib/xe/xe_eudebug.c new file +87 -0
+
+	xe_eudebug_for_each_event(ce, l) {
+		if (ce->flags & DRM_XE_EUDEBUG_EVENT_CREATE) {
+			uint8_t offset = sizeof(struct drm_xe_eudebug_event);
+			int opposite_matching;
+
+			if (XE_EUDEBUG_EVENT_IS_FILTERED(ce->type, filter))
+				continue;
+
+			/* No opposite matching for binds */
+			if ((ce->type >= DRM_XE_EUDEBUG_EVENT_VM_BIND &&
+			     ce->type <= DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE) ||
+			    ce->type == DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA)
+				continue;
+
+			de = opposite_event_match(l, ce, ce);
+
+			igt_assert_f(de, "no opposite event of type %u found\n", ce->type);
+
+			igt_assert_eq(ce->len, de->len);
+			opposite_matching = memcmp((uint8_t *)de + offset,
+						   (uint8_t *)ce + offset,
+						   de->len - offset) == 0;
+
+			igt_assert_f(opposite_matching,
+				     "%s: create|destroy event not maching (%llu) vs (%llu)\n",
+				     l->name, de->seqno, ce->seqno);
+		}
+	}
+}
+
+static void debugger_run_triggers(struct xe_eudebug_debugger *d,
+				  struct drm_xe_eudebug_event *e)
+{
+	struct event_trigger *t;
+
+	igt_list_for_each_entry(t, &d->triggers, link) {
+		if (e->type == t->type)
+			t->fn(d, e);
+	}
+}
+
+#define MAX_EVENT_SIZE (32 * 1024)
+static int
+xe_eudebug_read_event(int fd, struct drm_xe_eudebug_event *event)
+{
+	int ret;
+
+	event->type = DRM_XE_EUDEBUG_EVENT_READ;
+	event->flags = 0;
+	event->len = MAX_EVENT_SIZE;
+
+	ret = igt_ioctl(fd, DRM_XE_EUDEBUG_IOCTL_READ_EVENT, event);
+	if (ret < 0)
+		return -errno;
+
+	return ret;
+}
+
+static void *debugger_worker_loop(void *data)
+{
+	uint8_t buf[MAX_EVENT_SIZE];
+	struct drm_xe_eudebug_event *e = (void *)buf;
+	struct xe_eudebug_debugger *d = data;
+	struct pollfd p = {
+		.events = POLLIN,
+		.revents = 0,
+	};
+	int timeout_ms = 100, ret;
+
+	igt_assert(d->master_fd >= 0);
+
+	do {
+		p.fd = d->fd;
+		ret = poll(&p, 1, timeout_ms);
+
+		if (ret == -1) {
+			igt_info("poll failed with errno %d\n", errno);
+			break;
+		}
+
+		if (ret == 1 && (p.revents & POLLIN)) {
+			int err = xe_eudebug_read_event(d->fd, e);
+
+			if (!err) {
+				++d->event_count;
+

lib/xe/xe_eudebug.c line 1047

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+				xe_eudebug_event_log_write(d->log, e);
+				debugger_run_triggers(d, e);
+			} else {
+				igt_info("xe_eudebug_read_event returned %d\n", ret);
+			}
+		}
+	} while ((ret && READ_ONCE(d->worker_state) == DEBUGGER_WORKER_QUITTING) ||
+		 READ_ONCE(d->worker_state) == DEBUGGER_WORKER_ACTIVE);
+
+	d->worker_state = DEBUGGER_WORKER_INACTIVE;

Please add blank line here.

Quoted 30 lines
lib/xe/xe_eudebug.c new file +30 -0
+	return NULL;
+}
+
+/**
+ * xe_eudebug_debugger_available:
+ * @fd: Xe file descriptor
+ *
+ * Returns: true it debugger connection is available, false otherwise.
+ */
+bool xe_eudebug_debugger_available(int fd)
+{
+	struct drm_xe_eudebug_connect param = { .pid = getpid() };
+	int debugfd;
+
+	debugfd = igt_ioctl(fd, DRM_IOCTL_XE_EUDEBUG_CONNECT, &param);
+	if (debugfd >= 0)
+		close(debugfd);
+
+	return debugfd >= 0;
+}
+
+/**
+ * xe_eudebug_debugger_create:
+ * @master_fd: xe client used to open the debugger connection
+ * @flags: flags stored in a debugger structure, can be used at will
+ * of the caller, i.e. to be used inside triggers.
+ * @data: test's private data, allocated with MAP_SHARED | MAP_ANONYMOUS,
+ * can be shared between client and debugger. Can be NULL.
+ *
+ * Returns: newly created xe_eudebug_debugger structure with its

lib/xe/xe_eudebug.c line 1087

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ * event log initialized. Note that to open the connection
+ * you need call @xe_eudebug_debugger_attach.
+ */
+struct xe_eudebug_debugger *
+xe_eudebug_debugger_create(int master_fd, uint64_t flags, void *data)
+{
+	struct xe_eudebug_debugger *d;
+
+	d = calloc(1, sizeof(*d));
+	d->flags = flags;

I guess assert should be before dereference.

Quoted 22 lines
lib/xe/xe_eudebug.c new file +22 -0
+	igt_assert(d);
+	IGT_INIT_LIST_HEAD(&d->triggers);
+	d->log = xe_eudebug_event_log_create("debugger", MAX_EVENT_LOG_SIZE);
+	d->fd = -1;
+	d->master_fd = master_fd;
+	d->ptr = data;
+
+	return d;
+}
+
+static void debugger_destroy_triggers(struct xe_eudebug_debugger *d)
+{
+	struct event_trigger *t, *tmp;
+
+	igt_list_for_each_entry_safe(t, tmp, &d->triggers, link)
+		free(t);
+}
+
+/**
+ * xe_eudebug_debugger_destroy:
+ * @d: pointer to the debugger
+ *

lib/xe/xe_eudebug.c line 1119

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ * Frees xe_eudebug_debugger structure pointed by @d. If the debugger
+ * connection was still opened it terminates it.
+ */
+void xe_eudebug_debugger_destroy(struct xe_eudebug_debugger *d)
+{
+	if (d->worker_state)
+		xe_eudebug_debugger_stop_worker(d, 1);
+
+	if (d->target_pid)
+		xe_eudebug_debugger_dettach(d);

Typo, s/dettach/detach/

Quoted 29 lines
lib/xe/xe_eudebug.c new file +29 -0
+
+	xe_eudebug_event_log_destroy(d->log);
+	debugger_destroy_triggers(d);
+	free(d);
+}
+
+/**
+ * xe_eudebug_debugger_attach:
+ * @d: pointer to the debugger
+ * @c: pointer to the client
+ *
+ * Opens the xe eu debugger connection to the process described by @c (c->pid)
+ *
+ * Returns: 0 if the debugger was successfully attached, -errno otherwise.
+ */
+int xe_eudebug_debugger_attach(struct xe_eudebug_debugger *d,
+			       struct xe_eudebug_client *c)
+{
+	int ret;
+
+	igt_assert_eq(d->fd, -1);
+	igt_assert_neq(c->pid, 0);
+	ret = xe_eudebug_connect(d->master_fd, c->pid, 0);
+
+	if (ret < 0)
+		return ret;
+
+	d->fd = ret;
+	d->target_pid = c->pid;

lib/xe/xe_eudebug.c line 1158

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+	d->p_client[0] = c->p_in[0];
+	d->p_client[1] = c->p_in[1];
+
+	igt_debug("debugger connected to %lu\n", d->target_pid);
+
+	return 0;
+}
+
+/**
+ * xe_eudebug_debugger_dettach:

Ditto.

lib/xe/xe_eudebug.c line 1164

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+ * @d: pointer to the debugger
+ *
+ * Closes previously opened xe eu debugger connection. Asserts if
+ * the debugger has active session.
+ */
+void xe_eudebug_debugger_dettach(struct xe_eudebug_debugger *d)

Ditto.

Quoted 13 lines
lib/xe/xe_eudebug.c new file +13 -0
+{
+	igt_assert(d->target_pid);
+	close(d->fd);
+	d->target_pid = 0;
+	d->fd = -1;
+}
+
+/**
+ * xe_eudebug_debugger_add_trigger:
+ * @d: pointer to the debugger
+ * @type: the type of the event which activates the trigger
+ * @fn: function to be called when event of @type was read by the debugger.
+ *

lib/xe/xe_eudebug.c line 1187

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ * Adds function @fn to the list of triggers activated when event of @type
+ * has been read by worker.
+ * Note: Triggers are activated by the worker.
+ */
+void xe_eudebug_debugger_add_trigger(struct xe_eudebug_debugger *d,
+				     int type, xe_eudebug_trigger_fn fn)
+{
+	struct event_trigger *t;
+
+	t = calloc(1, sizeof(*t));

Assert if allocation failed.

Quoted 12 lines
lib/xe/xe_eudebug.c new file +12 -0
+	IGT_INIT_LIST_HEAD(&t->link);
+	t->type = type;
+	t->fn = fn;
+
+	igt_list_add_tail(&t->link, &d->triggers);
+	igt_debug("added trigger %p\n", t);
+}
+
+/**
+ * xe_eudebug_debugger_start_worker:
+ * @d: pointer to the debugger
+ *

lib/xe/xe_eudebug.c line 1209

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ * Starts the debugger worker. Worker is resposible for reading all
+ * incoming events from the debugger, put then into debugger log and
+ * execute appropriate event triggers. Note that using the debuggers
+ * event log while worker is running is not safe.
+ */
+void xe_eudebug_debugger_start_worker(struct xe_eudebug_debugger *d)
+{
+	int ret;
+
+	d->worker_state = true;

I'm not aware how rest of the code looks like, but setting worker_state to true before worker thread started looks racy.

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+	ret = pthread_create(&d->worker_thread, NULL, &debugger_worker_loop, d);
+
+	igt_assert_f(ret == 0, "Debugger worker thread creation failed!");
+}
+
+/**
+ * xe_eudebug_debugger_stop_worker:
+ * @d: pointer to the debugger
+ *
+ * Stops the debugger worker. Event log is sorted by seqno after closure.

lib/xe/xe_eudebug.c line 1229

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ */
+void xe_eudebug_debugger_stop_worker(struct xe_eudebug_debugger *d,
+				     int timeout_s)
+{
+	struct timespec t = {};
+	int ret;
+
+	igt_assert(d->worker_state);
+
+	d->worker_state = DEBUGGER_WORKER_QUITTING; /* First time be polite. */

Shoudln't this be WRITE_ONCE()? I see READ_ONCE() is used for reading this variable, but write imo may be delayed. Check other usages of this variable.

Quoted 64 lines
lib/xe/xe_eudebug.c new file +64 -0
+	igt_assert_eq(clock_gettime(CLOCK_REALTIME, &t), 0);
+	t.tv_sec += timeout_s;
+
+	ret = pthread_timedjoin_np(d->worker_thread, NULL, &t);
+
+	if (ret == ETIMEDOUT) {
+		d->worker_state = DEBUGGER_WORKER_INACTIVE;
+		ret = pthread_join(d->worker_thread, NULL);
+	}
+
+	igt_assert_f(ret == 0 || ret != ESRCH,
+		     "pthread join failed with error %d!\n", ret);
+
+	event_log_sort(d->log);
+}
+
+/**
+ * xe_eudebug_debugger_signal_stage:
+ * @d: pointer to the debugger
+ * @stage: stage to signal
+ *
+ * Signals to client, waiting in xe_eudebug_client_wait_stage(),
+ * releasing it to proceed.
+ */
+void xe_eudebug_debugger_signal_stage(struct xe_eudebug_debugger *d, uint64_t stage)
+{
+	token_signal(d->p_client, CLIENT_STAGE, stage);
+}
+
+/**
+ * xe_eudebug_debugger_wait_stage:
+ * @s: pointer to xe_eudebug_debugger structure
+ * @stage: stage to wait on
+ *
+ * Pauses debugger until the client has signalled the corresponding stage with
+ * xe_eudebug_client_signal_stage. This is only for situations where the actual
+ * event flow is not enough to coordinate between client/debugger and extra sync
+ * mechanism is needed.
+ */
+void xe_eudebug_debugger_wait_stage(struct xe_eudebug_session *s, uint64_t stage)
+{
+	u64 stage_in;
+
+	igt_debug("debugger xe client fd: %d pausing for stage %lu\n", s->d->master_fd, stage);
+
+	stage_in = wait_from_client(s->c, DEBUGGER_STAGE);
+	igt_debug("debugger xe client fd: %d stage %lu, expected %lu, stage\n", s->d->master_fd,
+		  stage_in, stage);
+
+	igt_assert_eq(stage_in, stage);
+}
+
+/**
+ * xe_eudebug_client_create:
+ * @master_fd: xe client used to open the debugger connection
+ * @work: function that opens xe device and executes arbitrary workload
+ * @flags: flags stored in a client structure, can be used at will
+ * of the caller, i.e. to provide the @work function an additional switch.
+ * @data: test's private data, allocated with MAP_SHARED | MAP_ANONYMOUS,
+ * can be shared between client and debugger. Accesible via client->ptr.
+ * Can be NULL.
+ *
+ * Forks and creates the debugger process. @work won't be called until
+ * xe_eudebug_client_start is called.

lib/xe/xe_eudebug.c line 1303

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ *
+ * Returns: newly created xe_eudebug_debugger structure with its
+ * event log initialized.
+ */
+struct xe_eudebug_client *xe_eudebug_client_create(int master_fd, xe_eudebug_client_work_fn work,
+						   uint64_t flags, void *data)
+{
+	struct xe_eudebug_client *c;
+
+	c = calloc(1, sizeof(*c));

Assert here.

Quoted 237 lines
lib/xe/xe_eudebug.c new file +237 -0
+	c->flags = flags;
+	igt_assert(c);
+	igt_assert(!pipe(c->p_in));
+	igt_assert(!pipe(c->p_out));
+	c->seqno = 1;
+	c->log = xe_eudebug_event_log_create("client", MAX_EVENT_LOG_SIZE);
+	c->done = 0;
+	c->ptr = data;
+	c->master_fd = master_fd;
+	c->timeout_ms = XE_EUDEBUG_DEFAULT_TIMEOUT_SEC * MSEC_PER_SEC;
+
+	igt_fork(child, 1) {
+		int mypid;
+
+		igt_assert_eq(c->pid, 0);
+
+		close(c->p_out[0]);
+		c->p_out[0] = -1;
+		close(c->p_in[1]);
+		c->p_in[1] = -1;
+
+		mypid = getpid();
+		client_signal(c, CLIENT_PID, mypid);
+
+		c->pid = client_wait_token(c, CLIENT_RUN);
+		igt_assert_eq(c->pid, mypid);
+		if (work)
+			work(c);
+
+		client_signal(c, CLIENT_FINI, c->seqno);
+
+		event_log_write_to_fd(c->log, c->p_out[1]);
+
+		c->pid = client_wait_token(c, CLIENT_STOP);
+		igt_assert_eq(c->pid, mypid);
+	}
+
+	close(c->p_out[1]);
+	c->p_out[1] = -1;
+	close(c->p_in[0]);
+	c->p_in[0] = -1;
+
+	c->pid = wait_from_client(c, CLIENT_PID);
+
+	igt_info("client running with pid %d\n", c->pid);
+
+	return c;
+}
+
+/**
+ * xe_eudebug_client_stop:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Waits for the end of client's work and exits the proccess.
+ */
+void xe_eudebug_client_stop(struct xe_eudebug_client *c)
+{
+	if (c->pid) {
+		int waitstatus;
+
+		xe_eudebug_client_wait_done(c);
+
+		token_signal(c->p_in, CLIENT_STOP, c->pid);
+		igt_assert_eq(waitpid(c->pid, &waitstatus, 0),
+			      c->pid);
+		c->pid = 0;
+	}
+}
+
+/**
+ * xe_eudebug_client_destroy:
+ * @c: pointer to xe_eudebug_client structure to be freed
+ *
+ * Frees the @c client structure. Note that it calls xe_eudebug_client_stop if
+ * client proccess has not terminated yet.
+ */
+void xe_eudebug_client_destroy(struct xe_eudebug_client *c)
+{
+	xe_eudebug_client_stop(c);
+	pipe_close(c->p_in);
+	pipe_close(c->p_out);
+	xe_eudebug_event_log_destroy(c->log);
+	free(c);
+}
+
+/**
+ * xe_eudebug_client_get_seqno:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Increments and returns current seqno value of the given client @c
+ *
+ * Returns: incremented seqno
+ */
+uint64_t xe_eudebug_client_get_seqno(struct xe_eudebug_client *c)
+{
+	return c->seqno++;
+}
+
+/**
+ * xe_eudebug_client_start:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Starts execution of client's work function within the client's proccess.
+ */
+void xe_eudebug_client_start(struct xe_eudebug_client *c)
+{
+	token_signal(c->p_in, CLIENT_RUN, c->pid);
+}
+
+/**
+ * xe_eudebug_client_wait_done:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Waits for the client work end updates the event log.
+ * Doesn't terminate the client's proccess yet.
+ */
+void xe_eudebug_client_wait_done(struct xe_eudebug_client *c)
+{
+	if (!c->done) {
+		c->done = 1;
+		c->seqno = wait_from_client(c, CLIENT_FINI);
+		event_log_read_from_fd(c->log, c->p_out[0]);
+	}
+}
+
+/**
+ * xe_eudebug_client_signal_stage:
+ * @c: pointer to the client
+ * @stage: stage to signal
+ *
+ * Signals to debugger, waiting in xe_eudebug_debugger_wait_stage(),
+ * releasing it to proceed.
+ */
+void xe_eudebug_client_signal_stage(struct xe_eudebug_client *c, uint64_t stage)
+{
+	token_signal(c->p_out, DEBUGGER_STAGE, stage);
+}
+
+/**
+ * xe_eudebug_client_wait_stage:
+ * @c: pointer to xe_eudebug_client structure
+ * @stage: stage to wait on
+ *
+ * Pauses client until the debugger has signalled the corresponding stage with
+ * xe_eudebug_debugger_signal_stage. This is only for situations where the
+ * actual event flow is not enough to coordinate between client/debugger and extra
+ * sync mechanism is needed.
+ *
+ */
+void xe_eudebug_client_wait_stage(struct xe_eudebug_client *c, uint64_t stage)
+{
+	u64 stage_in;
+
+	if (c->done) {
+		igt_warn("client: %d already done before %lu\n", c->pid, stage);
+		return;
+	}
+
+	igt_debug("client: %d pausing for stage %lu\n", c->pid, stage);
+
+	stage_in = client_wait_token(c, CLIENT_STAGE);
+	igt_debug("client: %d stage %lu, expected %lu, stage\n", c->pid, stage_in, stage);
+
+	igt_assert_eq(stage_in, stage);
+}
+
+/**
+ * xe_eudebug_session_create:
+ * @fd: XE file descriptor
+ * @work: function passed to the xe_eudebug_client_create
+ * @flags: flags passed to client and debugger
+ * @test_private: test's  data, allocated with MAP_SHARED | MAP_ANONYMOUS,
+ * passed to client and debugger. Can be NULL.
+ *
+ * Creates session together with client and debugger structures.
+ */
+struct xe_eudebug_session *xe_eudebug_session_create(int fd,
+						     xe_eudebug_client_work_fn work,
+						     unsigned int flags,
+						     void *test_private)
+{
+	struct xe_eudebug_session *s;
+
+	s = calloc(1, sizeof(*s));
+	igt_assert(s);
+
+	s->c = xe_eudebug_client_create(fd, work, flags, test_private);
+	s->d = xe_eudebug_debugger_create(fd, flags, test_private);
+	s->flags = flags;
+
+	return s;
+}
+
+/**
+ * xe_eudebug_session_run:
+ * @s: pointer to xe_eudebug_session structure
+ *
+ * Attaches debugger to client's proccess, starts debugger's
+ * async event reader, starts client and once client finish
+ * it stops debugger worker.
+ */
+void xe_eudebug_session_run(struct xe_eudebug_session *s)
+{
+	struct xe_eudebug_debugger *debugger = s->d;
+	struct xe_eudebug_client *client = s->c;
+
+	igt_assert_eq(xe_eudebug_debugger_attach(debugger, client), 0);
+
+	xe_eudebug_debugger_start_worker(debugger);
+
+	xe_eudebug_client_start(client);
+	xe_eudebug_client_wait_done(client);
+
+	xe_eudebug_debugger_stop_worker(debugger, 1);
+
+	xe_eudebug_event_log_print(debugger->log, true);
+	xe_eudebug_event_log_print(client->log, true);
+}
+
+/**
+ * xe_eudebug_session_check:
+ * @s: pointer to xe_eudebug_session structure
+ * @match_opposite: indicates whether check should match all
+ * create and destroy events.
+ * @filter: mask that represents events to be skipped during comparison, useful
+ * for events like 'VM_BIND' since they can be asymmetric
+ *
+ * Validate debugger's log against the log created by the client.
+ */
+void xe_eudebug_session_check(struct xe_eudebug_session *s, bool match_opposite, uint32_t filter)
+{
+	xe_eudebug_event_log_compare(s->c->log, s->d->log, filter);
+
+	if (match_opposite)
+		xe_eudebug_event_log_match_opposite(s->d->log, filter);
+}
+

lib/xe/xe_eudebug.c line 1550

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+/**
+ * xe_eudebug_session_destroy:
+ * @s: pointer to xe_eudebug_session structure
+ *
+ * Destroy session together with its debugger and client.
+ */
+void xe_eudebug_session_destroy(struct xe_eudebug_session *s)
+{
+	xe_eudebug_debugger_destroy(s->d);
+	xe_eudebug_client_destroy(s->c);

I think s->debugger and s->client would be more readable.

Quoted 71 lines
lib/xe/xe_eudebug.c new file +71 -0
+
+	free(s);
+}
+
+#define to_base(x) ((struct drm_xe_eudebug_event *)&(x))
+
+static void base_event(struct xe_eudebug_client *c,
+		       struct drm_xe_eudebug_event *e,
+		       uint32_t type,
+		       uint32_t flags,
+		       uint64_t size)
+{
+	e->type = type;
+	e->flags = flags;
+	e->seqno = xe_eudebug_client_get_seqno(c);
+	e->len = size;
+}
+
+static void client_event(struct xe_eudebug_client *c, uint32_t flags, int client_fd)
+{
+	struct drm_xe_eudebug_event_client ec;
+
+	base_event(c, to_base(ec), DRM_XE_EUDEBUG_EVENT_OPEN, flags, sizeof(ec));
+
+	ec.client_handle = client_fd;
+
+	xe_eudebug_event_log_write(c->log, (void *)&ec);
+}
+
+static void vm_event(struct xe_eudebug_client *c, uint32_t flags, int client_fd, uint32_t vm_id)
+{
+	struct drm_xe_eudebug_event_vm evm;
+
+	base_event(c, to_base(evm), DRM_XE_EUDEBUG_EVENT_VM, flags, sizeof(evm));
+
+	evm.client_handle = client_fd;
+	evm.vm_handle = vm_id;
+
+	xe_eudebug_event_log_write(c->log, (void *)&evm);
+}
+
+static void exec_queue_event(struct xe_eudebug_client *c, uint32_t flags,
+			     int client_fd, uint32_t vm_id,
+			     uint32_t exec_queue_handle, uint16_t class,
+			     uint16_t width)
+{
+	struct drm_xe_eudebug_event_exec_queue ee;
+
+	base_event(c, to_base(ee), DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE,
+		   flags, sizeof(ee));
+
+	ee.client_handle = client_fd;
+	ee.vm_handle = vm_id;
+	ee.exec_queue_handle = exec_queue_handle;
+	ee.engine_class = class;
+	ee.width = width;
+
+	xe_eudebug_event_log_write(c->log, (void *)&ee);
+}
+
+static void metadata_event(struct xe_eudebug_client *c, uint32_t flags,
+			   int client_fd, uint32_t id, uint64_t type, uint64_t len)
+{
+	struct drm_xe_eudebug_event_metadata em;
+
+	base_event(c, to_base(em), DRM_XE_EUDEBUG_EVENT_METADATA,
+		   flags, sizeof(em));
+
+	em.client_handle = client_fd;
+	em.metadata_handle = id;
+	em.type = type;

lib/xe/xe_eudebug.c line 1631

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+	em.len = len;
+
+	xe_eudebug_event_log_write(c->log, (void *)&em);
+}
+
+static int enable_getset(int fd, bool *old, bool *new)
+{
+	static const char * const fname = "enable_eudebug";
+	int ret = 0;
+

Unnecessary blank line.

lib/xe/xe_eudebug.c line 1638

Quoted 7 lines
lib/xe/xe_eudebug.c new file +7 -0
+	int sysfs, device_fd;
+	bool val_before;
+	struct stat st;
+
+	igt_assert(new || old);
+
+	igt_assert_eq(fstat(fd, &st), 0);

I would group assert within single block separated by blank line.

Quoted 19 lines
lib/xe/xe_eudebug.c new file +19 -0
+	sysfs = igt_sysfs_open(fd);
+	if (sysfs < 0)
+		return -1;
+
+	device_fd = openat(sysfs, "device", O_DIRECTORY | O_RDONLY);
+	close(sysfs);
+	if (device_fd < 0)
+		return -1;
+
+	if (!__igt_sysfs_get_boolean(device_fd, fname, &val_before)) {
+		ret = -1;
+		goto out;
+	}
+
+	igt_debug("enable_eudebug before: %d\n", val_before);
+
+	if (old)
+		*old = val_before;
+

lib/xe/xe_eudebug.c line 1667

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+	ret = 0;
+	if (new) {
+		if (__igt_sysfs_set_boolean(device_fd, fname, *new))
+			igt_assert_eq(igt_sysfs_get_boolean(device_fd, fname), *new);
+		else
+			ret = -1;
+	}
+
+out:
+	close(device_fd);

Add blank line.

Quoted 32 lines
lib/xe/xe_eudebug.c new file +32 -0
+	return ret;
+}
+
+/**
+ * xe_eudebug_enable
+ * @fd: xe client
+ * @enable: state toggle - true to enable, false to disable
+ *
+ * Enables/disables eudebug capability by writing to
+ * '/sys/class/drm/card<N>/device/enable_eudebug' sysfs entry.
+ *
+ * Returns: previous toggle value, i.e. true when eudebugging was enabled,
+ * false when eudebugging was disabled.
+ */
+bool xe_eudebug_enable(int fd, bool enable)
+{
+	bool old = false;
+	int ret = enable_getset(fd, &old, &enable);
+
+	if (ret) {
+		igt_skip_on(enable);
+		old = false;
+	}
+
+	return old;
+}
+
+/* Eu debugger wrappers around resource creating xe ioctls. */
+
+/**
+ * xe_eudebug_client_open_driver:
+ * @c: pointer to xe_eudebug_client structure

lib/xe/xe_eudebug.c line 1709

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ *
+ * Calls drm_open_client(DRIVER_XE) and logs the corresponding
+ * event in client's event log.
+ *
+ * Returns: valid DRM file descriptor
+ */
+int xe_eudebug_client_open_driver(struct xe_eudebug_client *c)
+{
+	int fd;
+

From now on for exported functions you should validate variables, like c must be not null here. And this spreads for all global functions introduced below.

Quoted 67 lines
lib/xe/xe_eudebug.c new file +67 -0
+	fd = drm_reopen_driver(c->master_fd);
+	client_event(c, DRM_XE_EUDEBUG_EVENT_CREATE, fd);
+
+	return fd;
+}
+
+/**
+ * xe_eudebug_client_close_driver:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ *
+ * Calls close driver and logs the corresponding event in
+ * client's event log.
+ */
+void xe_eudebug_client_close_driver(struct xe_eudebug_client *c, int fd)
+{
+	client_event(c, DRM_XE_EUDEBUG_EVENT_DESTROY, fd);
+	close(fd);
+}
+
+/**
+ * xe_eudebug_client_vm_create:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @flags: vm bind flags
+ * @ext: pointer to the first user extension
+ *
+ * Calls xe_vm_create() and logs corresponding events
+ * (including vm set metadata events) in client's event log.
+ *
+ * Returns: valid vm handle
+ */
+uint32_t xe_eudebug_client_vm_create(struct xe_eudebug_client *c, int fd,
+				     uint32_t flags, uint64_t ext)
+{
+	uint32_t vm;
+
+	vm = xe_vm_create(fd, flags, ext);
+	vm_event(c, DRM_XE_EUDEBUG_EVENT_CREATE, fd, vm);
+
+	return vm;
+}
+
+/**
+ * xe_eudebug_client_vm_destroy:
+ * @c: pointer to xe_eudebug_client structure
+ * fd: xe client
+ * vm: vm handle
+ *
+ * Calls xe_vm_destroy() and logs the corresponding event in
+ * client's event log.
+ */
+void xe_eudebug_client_vm_destroy(struct xe_eudebug_client *c, int fd, uint32_t vm)
+{
+	xe_vm_destroy(fd, vm);
+	vm_event(c, DRM_XE_EUDEBUG_EVENT_DESTROY, fd, vm);
+}
+
+/**
+ * xe_eudebug_client_exec_queue_create:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @create: exec_queue create drm struct
+ *
+ * Calls xe exec queue create ioctl and logs the corresponding event in
+ * client's event log.
+ *

lib/xe/xe_eudebug.c line 1786

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+ * Returns: valid exec queue handle
+ */
+uint32_t xe_eudebug_client_exec_queue_create(struct xe_eudebug_client *c, int fd,
+					     struct drm_xe_exec_queue_create *create)
+{
+	uint16_t class = ((struct drm_xe_engine_class_instance *)(create->instances))[0].engine_class;
+	struct drm_xe_ext_set_property *ext;
+	bool send = false;
+
+	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_EXEC_QUEUE_CREATE, create), 0);

I would add blank line here.

Quoted 61 lines
lib/xe/xe_eudebug.c new file +61 -0
+	for (ext = from_user_pointer(create->extensions); ext;
+	     ext = from_user_pointer(ext->base.next_extension))
+		if (ext->base.name == DRM_XE_EXEC_QUEUE_EXTENSION_SET_PROPERTY &&
+		    ext->property == DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG &&
+		    ext->value == DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE)
+			send = true;
+
+	if (send)
+		exec_queue_event(c, DRM_XE_EUDEBUG_EVENT_CREATE, fd, create->vm_id,
+				 create->exec_queue_id, class, create->width);
+
+	return create->exec_queue_id;
+}
+
+/**
+ * xe_eudebug_client_exec_queue_destroy:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @create: exec_queue create drm struct which was used for creation
+ *
+ * Calls xe exec_queue destroy ioctl and logs the corresponding event in
+ * client's event log.
+ */
+void xe_eudebug_client_exec_queue_destroy(struct xe_eudebug_client *c, int fd,
+					  struct drm_xe_exec_queue_create *create)
+{
+	struct drm_xe_exec_queue_destroy destroy = { .exec_queue_id = create->exec_queue_id, };
+	uint16_t class = ((struct drm_xe_engine_class_instance *)(create->instances))[0].engine_class;
+	struct drm_xe_ext_set_property *ext;
+	bool send = false;
+
+	for (ext = from_user_pointer(create->extensions); ext;
+	     ext = from_user_pointer(ext->base.next_extension))
+		if (ext->base.name == DRM_XE_EXEC_QUEUE_EXTENSION_SET_PROPERTY &&
+		    ext->property == DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG &&
+		    ext->value == DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE)
+			send = true;
+
+	if (send)
+		exec_queue_event(c, DRM_XE_EUDEBUG_EVENT_DESTROY, fd, create->vm_id,
+				 create->exec_queue_id, class, create->width);
+
+	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_EXEC_QUEUE_DESTROY, &destroy), 0);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_event:
+ * @c: pointer to xe_eudebug_client structure
+ * @event_flags: base event flags
+ * @fd: xe client
+ * @vm: vm handle
+ * @bind_flags: bind flags of vm_bind_event
+ * @num_binds: number of bind (operations) for event
+ * @ref_seqno: base vm bind reference seqno
+ * Logs vm bind event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_event(struct xe_eudebug_client *c,
+				     uint32_t event_flags, int fd,
+				     uint32_t vm, uint32_t bind_flags,
+				     uint32_t num_binds, u64 *ref_seqno)
+{

lib/xe/xe_eudebug.c line 1857

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+	struct drm_xe_eudebug_event_vm_bind evmb;
+
+	base_event(c, to_base(evmb), DRM_XE_EUDEBUG_EVENT_VM_BIND,
+		   event_flags, sizeof(evmb));
+	evmb.client_handle = fd;
+	evmb.vm_handle = vm;
+	evmb.flags = bind_flags;
+	evmb.num_binds = num_binds;
+
+	*ref_seqno = evmb.base.seqno;

ref_seqno requires to be valid pointer here.

--
Zbigniew

Quoted 594 lines
lib/xe/xe_eudebug.c new file +341 -0
+
+	xe_eudebug_event_log_write(c->log, (void *)&evmb);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_op_event:
+ * @c: pointer to xe_eudebug_client structure
+ * @event_flags: base event flags
+ * @bind_ref_seqno: base vm bind reference seqno
+ * @op_ref_seqno: output, the vm_bind_op event seqno
+ * @addr: ppgtt address
+ * @size: size of the binding
+ * @num_extensions: number of vm bind op extensions
+ *
+ * Logs vm bind op event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_op_event(struct xe_eudebug_client *c, uint32_t event_flags,
+					uint64_t bind_ref_seqno, uint64_t *op_ref_seqno,
+					uint64_t addr, uint64_t range,
+					uint64_t num_extensions)
+{
+	struct drm_xe_eudebug_event_vm_bind_op op;
+
+	base_event(c, to_base(op), DRM_XE_EUDEBUG_EVENT_VM_BIND_OP,
+		   event_flags, sizeof(op));
+	op.vm_bind_ref_seqno = bind_ref_seqno;
+	op.addr = addr;
+	op.range = range;
+	op.num_extensions = num_extensions;
+
+	*op_ref_seqno = op.base.seqno;
+
+	xe_eudebug_event_log_write(c->log, (void *)&op);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_op_metadata_event:
+ * @c: pointer to xe_eudebug_client structure
+ * @event_flags: base event flags
+ * @op_ref_seqno: base vm bind op reference seqno
+ * @metadata_handle: metadata handle
+ * @metadata_cookie: metadata cookie
+ *
+ * Logs vm bind op metadata event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_op_metadata_event(struct xe_eudebug_client *c,
+						 uint32_t event_flags, uint64_t op_ref_seqno,
+						 uint64_t metadata_handle, uint64_t metadata_cookie)
+{
+	struct drm_xe_eudebug_event_vm_bind_op_metadata op;
+
+	base_event(c, to_base(op), DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA,
+		   event_flags, sizeof(op));
+	op.vm_bind_op_ref_seqno = op_ref_seqno;
+	op.metadata_handle = metadata_handle;
+	op.metadata_cookie = metadata_cookie;
+
+	xe_eudebug_event_log_write(c->log, (void *)&op);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_ufence_event:
+ * @c: pointer to xe_eudebug_client structure
+ * @event_flags: base event flags
+ * @ref_seqno: base vm bind event seqno
+ *
+ * Logs vm bind ufence event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_ufence_event(struct xe_eudebug_client *c, uint32_t event_flags,
+					    uint64_t ref_seqno)
+{
+	struct drm_xe_eudebug_event_vm_bind_ufence f;
+
+	base_event(c, to_base(f), DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE,
+		   event_flags, sizeof(f));
+	f.vm_bind_ref_seqno = ref_seqno;
+
+	xe_eudebug_event_log_write(c->log, (void *)&f);
+}
+
+static bool has_user_fence(const struct drm_xe_sync *sync, uint32_t num_syncs)
+{
+	while (num_syncs--)
+		if (sync[num_syncs].type == DRM_XE_SYNC_TYPE_USER_FENCE)
+			return true;
+
+	return false;
+}
+
+#define for_each_metadata(__m, __ext)					\
+	for ((__m) = from_user_pointer(__ext);				\
+	     (__m);							\
+	     (__m) = from_user_pointer((__m)->base.next_extension))	\
+		if ((__m)->base.name == XE_VM_BIND_OP_EXTENSIONS_ATTACH_DEBUG)
+
+static int  __xe_eudebug_client_vm_bind(struct xe_eudebug_client *c,
+					int fd, uint32_t vm, uint32_t exec_queue,
+					uint32_t bo, uint64_t offset,
+					uint64_t addr, uint64_t size,
+					uint32_t op, uint32_t flags,
+					struct drm_xe_sync *sync,
+					uint32_t num_syncs,
+					uint32_t prefetch_region,
+					uint8_t pat_index, uint64_t op_ext)
+{
+	struct drm_xe_vm_bind_op_ext_attach_debug *metadata;
+	const bool ufence = has_user_fence(sync, num_syncs);
+	const uint32_t bind_flags = ufence ?
+		DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE : 0;
+	uint64_t seqno = 0, op_seqno = 0, num_metadata = 0;
+	uint32_t bind_base_flags = 0;
+	int ret;
+
+	for_each_metadata(metadata, op_ext)
+		num_metadata++;
+
+	switch (op) {
+	case DRM_XE_VM_BIND_OP_MAP:
+		bind_base_flags = DRM_XE_EUDEBUG_EVENT_CREATE;
+		break;
+	case DRM_XE_VM_BIND_OP_UNMAP:
+		bind_base_flags = DRM_XE_EUDEBUG_EVENT_DESTROY;
+		igt_assert_eq(num_metadata, 0);
+		igt_assert_eq(ufence, false);
+		break;
+	default:
+		/* XXX unmap all? */
+		igt_assert(op);
+		break;
+	}
+
+	ret = ___xe_vm_bind(fd, vm, exec_queue, bo, offset, addr, size,
+			    op, flags, sync, num_syncs, prefetch_region,
+			    pat_index, 0, op_ext);
+
+	if (ret)
+		return ret;
+
+	if (!bind_base_flags)
+		return -EINVAL;
+
+	xe_eudebug_client_vm_bind_event(c, DRM_XE_EUDEBUG_EVENT_STATE_CHANGE,
+					fd, vm, bind_flags, 1, &seqno);
+	xe_eudebug_client_vm_bind_op_event(c, bind_base_flags,
+					   seqno, &op_seqno, addr, size,
+					   num_metadata);
+
+	for_each_metadata(metadata, op_ext)
+		xe_eudebug_client_vm_bind_op_metadata_event(c,
+							    DRM_XE_EUDEBUG_EVENT_CREATE,
+							    op_seqno,
+							    metadata->metadata_id,
+							    metadata->cookie);
+	if (ufence)
+		xe_eudebug_client_vm_bind_ufence_event(c, DRM_XE_EUDEBUG_EVENT_CREATE |
+						       DRM_XE_EUDEBUG_EVENT_NEED_ACK,
+						       seqno);
+	return ret;
+}
+
+static void _xe_eudebug_client_vm_bind(struct xe_eudebug_client *c, int fd,
+				       uint32_t vm, uint32_t bo,
+				       uint64_t offset, uint64_t addr, uint64_t size,
+				       uint32_t op,
+				       uint32_t flags,
+				       struct drm_xe_sync *sync,
+				       uint32_t num_syncs,
+				       uint64_t op_ext)
+{
+	const uint32_t exec_queue_id = 0;
+	const uint32_t prefetch_region = 0;
+
+	igt_assert_eq(__xe_eudebug_client_vm_bind(c, fd, vm, exec_queue_id, bo, offset,
+						  addr, size, op, flags,
+						  sync, num_syncs, prefetch_region,
+						  DEFAULT_PAT_INDEX, op_ext),
+		      0);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_flags
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @vm: vm handle
+ * @bo: buffer object handle
+ * @offset: offset within buffer object
+ * @addr: ppgtt address
+ * @size: size of the binding
+ * @flags: vm_bind flags
+ * @sync: sync objects
+ * @num_syncs: number of sync objects
+ * @op_ext: BIND_OP extensions
+ *
+ * Calls xe vm_bind ioctl and logs the corresponding event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_flags(struct xe_eudebug_client *c, int fd, uint32_t vm,
+				     uint32_t bo, uint64_t offset,
+				     uint64_t addr, uint64_t size, uint32_t flags,
+				     struct drm_xe_sync *sync, uint32_t num_syncs,
+				     uint64_t op_ext)
+{
+	_xe_eudebug_client_vm_bind(c, fd, vm, bo, offset, addr, size,
+				   DRM_XE_VM_BIND_OP_MAP, flags,
+				   sync, num_syncs, op_ext);
+}
+
+/**
+ * xe_eudebug_client_vm_bind
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @vm: vm handle
+ * @bo: buffer object handle
+ * @offset: offset within buffer object
+ * @addr: ppgtt address
+ * @size: size of the binding
+ *
+ * Calls xe vm_bind ioctl and logs the corresponding event in client's event log.
+ */
+void xe_eudebug_client_vm_bind(struct xe_eudebug_client *c, int fd, uint32_t vm,
+			       uint32_t bo, uint64_t offset,
+			       uint64_t addr, uint64_t size)
+{
+	const uint32_t flags = 0;
+	struct drm_xe_sync *sync = NULL;
+	const uint32_t num_syncs = 0;
+	const uint64_t op_ext = 0;
+
+	xe_eudebug_client_vm_bind_flags(c, fd, vm, bo, offset, addr, size,
+					flags,
+					sync, num_syncs, op_ext);
+}
+
+/**
+ * xe_eudebug_client_vm_unbind_flags
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @vm: vm handle
+ * @offset: offset
+ * @addr: ppgtt address
+ * @size: size of the binding
+ * @flags: vm_bind flags
+ * @sync: sync objects
+ * @num_syncs: number of sync objects
+ *
+ * Calls xe vm_unbind ioctl and logs the corresponding event in client's event log.
+ */
+void xe_eudebug_client_vm_unbind_flags(struct xe_eudebug_client *c, int fd,
+				       uint32_t vm, uint64_t offset,
+				       uint64_t addr, uint64_t size, uint32_t flags,
+				       struct drm_xe_sync *sync, uint32_t num_syncs)
+{
+	_xe_eudebug_client_vm_bind(c, fd, vm, 0, offset, addr, size,
+				   DRM_XE_VM_BIND_OP_UNMAP, flags,
+				   sync, num_syncs, 0);
+}
+
+/**
+ * xe_eudebug_client_vm_unbind
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @vm: vm handle
+ * @offset: offset
+ * @addr: ppgtt address
+ * @size: size of the binding
+ *
+ * Calls xe vm_unbind ioctl and logs the corresponding event in client's event log.
+ */
+void xe_eudebug_client_vm_unbind(struct xe_eudebug_client *c, int fd, uint32_t vm,
+				 uint64_t offset, uint64_t addr, uint64_t size)
+{
+	const uint32_t flags = 0;
+	struct drm_xe_sync *sync = NULL;
+	const uint32_t num_syncs = 0;
+
+	xe_eudebug_client_vm_unbind_flags(c, fd, vm, offset, addr, size,
+					  flags, sync, num_syncs);
+}
+
+/**
+ * xe_eudebug_client_metadata_create:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @type: debug metadata type
+ * @len: size of @data
+ * @data: debug metadata paylad
+ *
+ * Calls xe metadata create ioctl and logs the corresponding event in
+ * client's event log.
+ *
+ * Return: valid debug metadata id.
+ */
+uint32_t xe_eudebug_client_metadata_create(struct xe_eudebug_client *c, int fd,
+					   int type, size_t len, void *data)
+{
+	struct drm_xe_debug_metadata_create create = {
+		.type = type,
+		.user_addr = to_user_pointer(data),
+		.len = len
+	};
+
+	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_DEBUG_METADATA_CREATE, &create), 0);
+
+	metadata_event(c, DRM_XE_EUDEBUG_EVENT_CREATE, fd, create.metadata_id, type, len);
+
+	return create.metadata_id;
+}
+
+/**
+ * xe_eudebug_client_metadata_destroy:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @id: xe debug metadata handle
+ * @type: debug metadata type
+ * @len: size of debug metadata payload
+ *
+ * Calls xe metadata destroy ioctl and logs the corresponding event in
+ * client's event log.
+ */
+void xe_eudebug_client_metadata_destroy(struct xe_eudebug_client *c, int fd,
+					uint32_t id, int type, size_t len)
+{
+	struct drm_xe_debug_metadata_destroy destroy = { .metadata_id = id };
+
+	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_DEBUG_METADATA_DESTROY, &destroy), 0);
+
+	metadata_event(c, DRM_XE_EUDEBUG_EVENT_DESTROY, fd, id, type, len);
+}
+
+void xe_eudebug_ack_ufence(int debugfd,
+			   const struct drm_xe_eudebug_event_vm_bind_ufence *f)
+{
+	struct drm_xe_eudebug_ack_event ack = { 0, };
+	char event_str[XE_EUDEBUG_EVENT_STRING_MAX_LEN];
+
+	ack.type = f->base.type;
+	ack.seqno = f->base.seqno;
+
+	xe_eudebug_event_to_str((void *)f, event_str, XE_EUDEBUG_EVENT_STRING_MAX_LEN);
+	igt_debug("delivering ack for event: %s\n", event_str);
+	igt_assert_eq(igt_ioctl(debugfd, DRM_XE_EUDEBUG_IOCTL_ACK_EVENT, &ack), 0);
+}
lib/xe/xe_eudebug.h new file +212 -0
new file mode 100644
index 000000000..d6c37d8f2
@@ -0,0 +1,212 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdint.h>
+#include <xe_drm.h>
+#include <xe_drm_eudebug.h>
+
+#include "igt_list.h"
+
+struct xe_eudebug_event_log {
+	uint8_t *log;
+	unsigned int head;
+	unsigned int max_size;
+	char name[80];
+	pthread_mutex_t lock;
+};
+
+struct xe_eudebug_debugger {
+	int fd;
+	uint64_t flags;
+
+	/* Used to smuggle private data */
+	void *ptr;
+
+	struct xe_eudebug_event_log *log;
+
+	uint64_t event_count;
+
+	uint64_t target_pid;
+
+	struct igt_list_head triggers;
+
+	int master_fd;
+
+	pthread_t worker_thread;
+	int worker_state;
+
+	int p_client[2];
+};
+
+struct xe_eudebug_client {
+	int pid;
+	uint64_t seqno;
+	uint64_t flags;
+
+	/* Used to smuggle private data */
+	void *ptr;
+
+	struct xe_eudebug_event_log *log;
+
+	int done;
+	int p_in[2];
+	int p_out[2];
+
+	/* Used to pickup right device (the one used in debugger) */
+	int master_fd;
+
+	int timeout_ms;
+};
+
+struct xe_eudebug_session {
+	uint64_t flags;
+	struct xe_eudebug_client *c;
+	struct xe_eudebug_debugger *d;
+};
+
+typedef void (*xe_eudebug_client_work_fn)(struct xe_eudebug_client *);
+typedef void (*xe_eudebug_trigger_fn)(struct xe_eudebug_debugger *,
+				      struct drm_xe_eudebug_event *);
+
+#define xe_eudebug_for_each_engine(fd__, hwe__) \
+	xe_for_each_engine(fd__, hwe__) \
+		if (hwe__->engine_class == DRM_XE_ENGINE_CLASS_RENDER || \
+		    hwe__->engine_class == DRM_XE_ENGINE_CLASS_COMPUTE)
+
+#define xe_eudebug_for_each_event(_e, _log) \
+	for ((_e) = (_e) ? (void *)(uint8_t *)(_e) + (_e)->len : \
+		    (void *)(_log)->log; \
+	    (uint8_t *)(_e) < (_log)->log + (_log)->head; \
+	    (_e) = (void *)(uint8_t *)(_e) + (_e)->len)
+
+#define xe_eudebug_assert(d, c)						\
+	do {								\
+		if (!(c)) {						\
+			xe_eudebug_event_log_print((d)->log, true);	\
+			igt_assert(c);					\
+		}							\
+	} while (0)
+
+#define xe_eudebug_assert_f(d, c, f...)					\
+	do {								\
+		if (!(c)) {						\
+			xe_eudebug_event_log_print((d)->log, true);	\
+			igt_assert_f(c, f);				\
+		}							\
+	} while (0)
+
+#define XE_EUDEBUG_EVENT_STRING_MAX_LEN		4096
+
+/*
+ * Default abort timeout to use across xe_eudebug lib and tests if no specific
+ * timeout value is required.
+ */
+#define XE_EUDEBUG_DEFAULT_TIMEOUT_SEC		25ULL
+
+#define XE_EUDEBUG_FILTER_EVENT_NONE		BIT(DRM_XE_EUDEBUG_EVENT_NONE)
+#define XE_EUDEBUG_FILTER_EVENT_READ		BIT(DRM_XE_EUDEBUG_EVENT_READ)
+#define XE_EUDEBUG_FILTER_EVENT_OPEN		BIT(DRM_XE_EUDEBUG_EVENT_OPEN)
+#define XE_EUDEBUG_FILTER_EVENT_VM		BIT(DRM_XE_EUDEBUG_EVENT_VM)
+#define XE_EUDEBUG_FILTER_EVENT_EXEC_QUEUE	BIT(DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE)
+#define XE_EUDEBUG_FILTER_EVENT_EU_ATTENTION	BIT(DRM_XE_EUDEBUG_EVENT_EU_ATTENTION)
+#define XE_EUDEBUG_FILTER_EVENT_VM_BIND		BIT(DRM_XE_EUDEBUG_EVENT_VM_BIND)
+#define XE_EUDEBUG_FILTER_EVENT_VM_BIND_OP	BIT(DRM_XE_EUDEBUG_EVENT_VM_BIND_OP)
+#define XE_EUDEBUG_FILTER_EVENT_VM_BIND_UFENCE  BIT(DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE)
+#define XE_EUDEBUG_FILTER_ALL			GENMASK(DRM_XE_EUDEBUG_EVENT_MAX_EVENT, 0)
+#define XE_EUDEBUG_EVENT_IS_FILTERED(_e, _f)	((1UL << (_e)) & (_f))
+
+int xe_eudebug_connect(int fd, pid_t pid, uint32_t flags);
+const char *xe_eudebug_event_to_str(struct drm_xe_eudebug_event *e, char *buf, size_t len);
+struct drm_xe_eudebug_event *
+xe_eudebug_event_log_find_seqno(struct xe_eudebug_event_log *l, uint64_t seqno);
+struct xe_eudebug_event_log *
+xe_eudebug_event_log_create(const char *name, unsigned int max_size);
+void xe_eudebug_event_log_destroy(struct xe_eudebug_event_log *l);
+void xe_eudebug_event_log_print(struct xe_eudebug_event_log *l, bool debug);
+void xe_eudebug_event_log_compare(struct xe_eudebug_event_log *c, struct xe_eudebug_event_log *d,
+				  uint32_t filter);
+void xe_eudebug_event_log_write(struct xe_eudebug_event_log *l, struct drm_xe_eudebug_event *e);
+void xe_eudebug_event_log_match_opposite(struct xe_eudebug_event_log *l, uint32_t filter);
+
+bool xe_eudebug_debugger_available(int fd);
+struct xe_eudebug_debugger *
+xe_eudebug_debugger_create(int xe, uint64_t flags, void *data);
+void xe_eudebug_debugger_destroy(struct xe_eudebug_debugger *d);
+int xe_eudebug_debugger_attach(struct xe_eudebug_debugger *d, struct xe_eudebug_client *c);
+void xe_eudebug_debugger_start_worker(struct xe_eudebug_debugger *d);
+void xe_eudebug_debugger_stop_worker(struct xe_eudebug_debugger *d, int timeout_s);
+void xe_eudebug_debugger_dettach(struct xe_eudebug_debugger *d);
+void xe_eudebug_debugger_set_data(struct xe_eudebug_debugger *c, void *ptr);
+void xe_eudebug_debugger_add_trigger(struct xe_eudebug_debugger *d, int type,
+				     xe_eudebug_trigger_fn fn);
+void xe_eudebug_debugger_signal_stage(struct xe_eudebug_debugger *d, uint64_t stage);
+void xe_eudebug_debugger_wait_stage(struct xe_eudebug_session *s, uint64_t stage);
+
+struct xe_eudebug_client *
+xe_eudebug_client_create(int xe, xe_eudebug_client_work_fn work, uint64_t flags, void *data);
+void xe_eudebug_client_destroy(struct xe_eudebug_client *c);
+void xe_eudebug_client_start(struct xe_eudebug_client *c);
+void xe_eudebug_client_stop(struct xe_eudebug_client *c);
+void xe_eudebug_client_wait_done(struct xe_eudebug_client *c);
+void xe_eudebug_client_signal_stage(struct xe_eudebug_client *c, uint64_t stage);
+void xe_eudebug_client_wait_stage(struct xe_eudebug_client *c, uint64_t stage);
+
+uint64_t xe_eudebug_client_get_seqno(struct xe_eudebug_client *c);
+void xe_eudebug_client_set_data(struct xe_eudebug_client *c, void *ptr);
+
+bool xe_eudebug_enable(int fd, bool enable);
+
+int xe_eudebug_client_open_driver(struct xe_eudebug_client *c);
+void xe_eudebug_client_close_driver(struct xe_eudebug_client *c, int fd);
+uint32_t xe_eudebug_client_vm_create(struct xe_eudebug_client *c, int fd,
+				     uint32_t flags, uint64_t ext);
+void xe_eudebug_client_vm_destroy(struct xe_eudebug_client *c, int fd, uint32_t vm);
+uint32_t xe_eudebug_client_exec_queue_create(struct xe_eudebug_client *c, int fd,
+					     struct drm_xe_exec_queue_create *create);
+void xe_eudebug_client_exec_queue_destroy(struct xe_eudebug_client *c, int fd,
+					  struct drm_xe_exec_queue_create *create);
+void xe_eudebug_client_vm_bind_event(struct xe_eudebug_client *c, uint32_t event_flags, int fd,
+				     uint32_t vm, uint32_t bind_flags,
+				     uint32_t num_ops, uint64_t *ref_seqno);
+void xe_eudebug_client_vm_bind_op_event(struct xe_eudebug_client *c, uint32_t event_flags,
+					uint64_t ref_seqno, uint64_t *op_ref_seqno,
+					uint64_t addr, uint64_t range,
+					uint64_t num_extensions);
+void xe_eudebug_client_vm_bind_op_metadata_event(struct xe_eudebug_client *c,
+						 uint32_t event_flags, uint64_t op_ref_seqno,
+						 uint64_t metadata_handle, uint64_t metadata_cookie);
+void xe_eudebug_client_vm_bind_ufence_event(struct xe_eudebug_client *c, uint32_t event_flags,
+					    uint64_t ref_seqno);
+void xe_eudebug_ack_ufence(int debugfd,
+			   const struct drm_xe_eudebug_event_vm_bind_ufence *f);
+
+void xe_eudebug_client_vm_bind_flags(struct xe_eudebug_client *c, int fd, uint32_t vm,
+				     uint32_t bo, uint64_t offset,
+				     uint64_t addr, uint64_t size, uint32_t flags,
+				     struct drm_xe_sync *sync, uint32_t num_syncs,
+				     uint64_t op_ext);
+void xe_eudebug_client_vm_bind(struct xe_eudebug_client *c, int fd, uint32_t vm,
+			       uint32_t bo, uint64_t offset,
+			       uint64_t addr, uint64_t size);
+void xe_eudebug_client_vm_unbind_flags(struct xe_eudebug_client *c, int fd,
+				       uint32_t vm, uint64_t offset,
+				       uint64_t addr, uint64_t size, uint32_t flags,
+				       struct drm_xe_sync *sync, uint32_t num_syncs);
+void xe_eudebug_client_vm_unbind(struct xe_eudebug_client *c, int fd, uint32_t vm,
+				 uint64_t offset, uint64_t addr, uint64_t size);
+
+uint32_t xe_eudebug_client_metadata_create(struct xe_eudebug_client *c, int fd,
+					   int type, size_t len, void *data);
+void xe_eudebug_client_metadata_destroy(struct xe_eudebug_client *c, int fd,
+					uint32_t id, int type, size_t len);
+
+struct xe_eudebug_session *xe_eudebug_session_create(int fd,
+						     xe_eudebug_client_work_fn work,
+						     unsigned int flags,
+						     void *test_private);
+void xe_eudebug_session_destroy(struct xe_eudebug_session *s);
+void xe_eudebug_session_run(struct xe_eudebug_session *s);
+void xe_eudebug_session_check(struct xe_eudebug_session *s, bool match_opposite, uint32_t filter);
meson.build +2 -0
index 25bf1e442..207b5819b 100644
@@ -90,9 +90,11 @@ build_chamelium = get_option('chamelium')
 build_docs = get_option('docs')
 build_tests = not get_option('tests').disabled()
 build_xe = not get_option('xe_driver').disabled()
+build_xe_eudebug = get_option('xe_eudebug').enabled()
 with_libdrm = get_option('libdrm_drivers')
 
 build_info = ['Build type: ' + get_option('buildtype')]
+build_info += 'Build Xe EU debugger test framework: @0@'.format(build_xe_eudebug)
 
 inc = include_directories('include', 'include/drm-uapi', 'include/linux-uapi', 'lib', 'lib/stubs/syscalls', '.')
 
meson_options.txt +5 -0
index 6a9493ea6..11922523b 100644
@@ -42,6 +42,11 @@ option('xe_driver',
        value : 'enabled',
        description : 'Build tests for Xe driver')
 
+option('xe_eudebug',
+       type : 'feature',
+       value : 'disabled',
+       description : 'Build library for Xe EU debugger')
+
 option('libdrm_drivers',
        type : 'array',
        value : ['auto'],
Signature · 1 line
2.34.1
#28 lore raw

Re: [PATCH i-g-t v5 11/17] lib/xe_eudebug: Introduce eu debug testing framework

Hi Zbigniew,

Quoted 160 lines · Zbigniew Kempczyński
Quoted 10 lines · Christoph Manszewski

From: Dominik Grzegorzek <[email protected]>

Introduce library which simplifies testing of eu debug capability. The library provides event log helpers together with asynchronous abstraction for client proccess and the debugger itself.

xe_eudebug_client creates its own proccess with user's work function, and gives machanisms to synchronize beginning of execution and event logging.

Quoted 6 lines

xe_eudebug_debugger allows to attach to the given proccess, provides asynchronous thread for event reading and introduces triggers - a callback mechanism triggered every time subscribed event was read.

To build the eudebug testing framework 'xe_eudebug' meson build option has to be enabled, as it is disabled by default.

My review might not catch all the issues, there's +2K line of codes so I might overlook something. But I think you should at least address things I've spotted.

Quoted 138 lines
---
  lib/meson.build     |    5 +
  lib/xe/xe_eudebug.c | 2198 +++++++++++++++++++++++++++++++++++++++++++
  lib/xe/xe_eudebug.h |  212 +++++
  meson.build         |    2 +
  meson_options.txt   |    5 +
  5 files changed, 2422 insertions(+)

create mode 100644 lib/xe/xe_eudebug.c
create mode 100644 lib/xe/xe_eudebug.h

lib/meson.build +5 -0
index 4af2bc743..96dec9678 100644
@@ -195,6 +195,11 @@ if chamelium.found()
  	lib_sources += 'monitor_edids/monitor_edids_helper.c'
  endif
 
+if build_xe_eudebug
+	build_info += 'Xe EU debugger test framework enabled.'
+	lib_sources += 'xe/xe_eudebug.c'
+endif
+
  if libprocps.found()
  	lib_deps += libprocps
  else
lib/xe/xe_eudebug.c new file +99 -0
new file mode 100644
index 000000000..2a804222a
@@ -0,0 +1,2198 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+
+#include <fcntl.h>
+#include <poll.h>
+#include <signal.h>
+#include <sys/select.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+
+#include "igt.h"
+#include "igt_sysfs.h"
+#include "intel_pat.h"
+#include "xe_eudebug.h"
+#include "xe_ioctl.h"
+
+struct event_trigger {
+	xe_eudebug_trigger_fn fn;
+	int type;
+	struct igt_list_head link;
+};
+
+struct seqno_list_entry {
+	struct igt_list_head link;
+	uint64_t seqno;
+};
+
+struct match_dto {
+	struct drm_xe_eudebug_event *target;
+	struct igt_list_head *seqno_list;
+	uint64_t client_handle;
+	uint32_t filter;
+
+	/* store latest 'EVENT_VM_BIND' seqno */
+	uint64_t *bind_seqno;
+	/* latest vm_bind_op seqno matching bind_seqno */
+	uint64_t *bind_op_seqno;
+};
+
+#define CLIENT_PID  1
+#define CLIENT_RUN  2
+#define CLIENT_FINI 3
+#define CLIENT_STOP 4
+#define CLIENT_STAGE 5
+#define DEBUGGER_STAGE 6
+
+#define DEBUGGER_WORKER_INACTIVE  0
+#define DEBUGGER_WORKER_ACTIVE  1
+#define DEBUGGER_WORKER_QUITTING 2
+
+static const char *type_to_str(unsigned int type)
+{
+	switch (type) {
+	case DRM_XE_EUDEBUG_EVENT_NONE:
+		return "none";
+	case DRM_XE_EUDEBUG_EVENT_READ:
+		return "read";
+	case DRM_XE_EUDEBUG_EVENT_OPEN:
+		return "client";
+	case DRM_XE_EUDEBUG_EVENT_VM:
+		return "vm";
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE:
+		return "exec_queue";
+	case DRM_XE_EUDEBUG_EVENT_EU_ATTENTION:
+		return "attention";
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND:
+		return "vm_bind";
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP:
+		return "vm_bind_op";
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE:
+		return "vm_bind_ufence";
+	case DRM_XE_EUDEBUG_EVENT_METADATA:
+		return "metadata";
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA:
+		return "vm_bind_op_metadata";
+	}
+
+	return "UNKNOWN";
+}
+
+static const char *event_type_to_str(struct drm_xe_eudebug_event *e, char *buf)
+{
+	sprintf(buf, "%s(%d)", type_to_str(e->type), e->type);
+
+	return buf;
+}
+
+static const char *flags_to_str(unsigned int flags)
+{
+	if (flags & DRM_XE_EUDEBUG_EVENT_CREATE) {
+		if (flags & DRM_XE_EUDEBUG_EVENT_NEED_ACK)
+			return "create|ack";
+		else
+			return "create";
+	}
+	if (flags & DRM_XE_EUDEBUG_EVENT_DESTROY)
Quoted 8 lines

lib/xe/xe_eudebug.c line 105

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+		return "destroy";
+
+	if (flags & DRM_XE_EUDEBUG_EVENT_STATE_CHANGE)
+		return "state-change";
+
+	igt_assert(!(flags & DRM_XE_EUDEBUG_EVENT_NEED_ACK));

Does it mean ACK is only valid for CREATE event?

I almost sure that currently 'ACK' is only valid for ufence events which indeed can only carry the 'create' flag.

Quoted 8 lines

lib/xe/xe_eudebug.c line 110

Quoted 6 lines
lib/xe/xe_eudebug.c new file +5 -0
+
+	return "flags unknown";
+}
+
+static const char *event_members_to_str(struct drm_xe_eudebug_event *e, char *b)

For being consistent with above just minor nitpick, s/b/buf/

Sure.

Quoted 247 lines
Quoted 234 lines
lib/xe/xe_eudebug.c new file +233 -0
+{
+	switch (e->type) {
+	case DRM_XE_EUDEBUG_EVENT_OPEN: {
+		struct drm_xe_eudebug_event_client *ec = (struct drm_xe_eudebug_event_client *)e;
+
+		sprintf(b, "handle=%llu", ec->client_handle);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM: {
+		struct drm_xe_eudebug_event_vm *evm = (struct drm_xe_eudebug_event_vm *)e;
+
+		sprintf(b, "client_handle=%llu, handle=%llu",
+			evm->client_handle, evm->vm_handle);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE: {
+		struct drm_xe_eudebug_event_exec_queue *ee = (void *)e;
+
+		sprintf(b, "client_handle=%llu, vm_handle=%llu, "
+			   "exec_queue_handle=%llu, engine_class=%d, exec_queue_width=%d",
+			ee->client_handle, ee->vm_handle,
+			ee->exec_queue_handle, ee->engine_class, ee->width);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_EU_ATTENTION: {
+		struct drm_xe_eudebug_event_eu_attention *ea = (void *)e;
+
+		sprintf(b, "client_handle=%llu, exec_queue_handle=%llu, "
+			   "lrc_handle=%llu, bitmask_size=%d",
+			ea->client_handle, ea->exec_queue_handle,
+			ea->lrc_handle, ea->bitmask_size);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND: {
+		struct drm_xe_eudebug_event_vm_bind *evmb = (void *)e;
+
+		sprintf(b, "client_handle=%llu, vm_handle=%llu, flags=0x%x, num_binds=%u",
+			evmb->client_handle, evmb->vm_handle, evmb->flags, evmb->num_binds);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP: {
+		struct drm_xe_eudebug_event_vm_bind_op *op = (void *)e;
+
+		sprintf(b, "vm_bind_ref_seqno=%lld, addr=%016llx, range=%llu num_extensions=%llu",
+			op->vm_bind_ref_seqno, op->addr, op->range, op->num_extensions);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE: {
+		struct drm_xe_eudebug_event_vm_bind_ufence *f = (void *)e;
+
+		sprintf(b, "vm_bind_ref_seqno=%lld", f->vm_bind_ref_seqno);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_METADATA: {
+		struct drm_xe_eudebug_event_metadata *em = (void *)e;
+
+		sprintf(b, "client_handle=%llu, metadata_handle=%llu, type=%llu, len=%llu",
+			em->client_handle, em->metadata_handle, em->type, em->len);
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA: {
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *op = (void *)e;
+
+		sprintf(b, "vm_bind_op_ref_seqno=%lld, metadata_handle=%llu, metadata_cookie=%llu",
+			op->vm_bind_op_ref_seqno, op->metadata_handle, op->metadata_cookie);
+		break;
+	}
+	default:
+		strcpy(b, "<...>");
+	}
+
+	return b;
+}
+
+/**
+ * xe_eudebug_event_to_str:
+ * @e: pointer to event
+ * @buf: target to write string representation of @e
+ * @len: size of target buffer @buf
+ *
+ * Creates string representation for given event.
+ *
+ * Returns: the written input buffer pointed by @buf.
+ */
+const char *xe_eudebug_event_to_str(struct drm_xe_eudebug_event *e, char *buf, size_t len)
+{
+	char a[256];
+	char b[256];
+
+	snprintf(buf, len, "(%llu) %15s:%s: %s",
+		 e->seqno,
+		 event_type_to_str(e, a),
+		 flags_to_str(e->flags),
+		 event_members_to_str(e, b));
+
+	return buf;
+}
+
+static void catch_child_failure(void)
+{
+	pid_t pid;
+	int status;
+
+	pid = waitpid(-1, &status, WNOHANG);
+
+	if (pid == 0 || pid == -1)
+		return;
+
+	if (!WIFEXITED(status))
+		return;
+
+	igt_assert_f(WEXITSTATUS(status) == 0, "Client failed!\n");
+}
+
+static int safe_pipe_read(int pipe[2], void *buf, int nbytes, int timeout_ms)
+{
+	int ret;
+	int t = 0;
+	struct pollfd fd = {
+		.fd = pipe[0],
+		.events = POLLIN,
+		.revents = 0
+	};
+
+	/* When child fails we may get stuck forever. Check whether
+	 * the child process ended with an error.
+	 */
+	do {
+		const int interval_ms = 1000;
+
+		ret = poll(&fd, 1, interval_ms);
+
+		if (!ret) {
+			catch_child_failure();
+			t += interval_ms;
+		}
+	} while (!ret && t < timeout_ms);
+
+	if (ret > 0)
+		return read(pipe[0], buf, nbytes);
+
+	return 0;
+}
+
+static uint64_t pipe_read(int pipe[2], int timeout_ms)
+{
+	uint64_t in;
+	uint64_t ret;
+
+	ret = safe_pipe_read(pipe, &in, sizeof(in), timeout_ms);
+	igt_assert(ret == sizeof(in));
+
+	return in;
+}
+
+static void pipe_signal(int pipe[2], uint64_t token)
+{
+	igt_assert(write(pipe[1], &token, sizeof(token)) == sizeof(token));
+}
+
+static void pipe_close(int pipe[2])
+{
+	if (pipe[0] != -1)
+		close(pipe[0]);
+
+	if (pipe[1] != -1)
+		close(pipe[1]);
+}
+
+static uint64_t __wait_token(int p[2], const uint64_t token, int timeout_ms)
+{
+	uint64_t in;
+
+	in = pipe_read(p, timeout_ms);
+
+	igt_assert_eq(in, token);
+
+	return pipe_read(p, timeout_ms);
+}
+
+static uint64_t client_wait_token(struct xe_eudebug_client *c, const uint64_t token)
+{
+	return __wait_token(c->p_in, token, c->timeout_ms);
+}
+
+static uint64_t wait_from_client(struct xe_eudebug_client *c, const uint64_t token)
+{
+	return __wait_token(c->p_out, token, c->timeout_ms);
+}
+
+static void token_signal(int p[2], const uint64_t token, const uint64_t value)
+{
+	pipe_signal(p, token);
+	pipe_signal(p, value);
+}
+
+static void client_signal(struct xe_eudebug_client *c,
+			  const uint64_t token,
+			  const uint64_t value)
+{
+	token_signal(c->p_out, token, value);
+}
+
+static int __xe_eudebug_connect(int fd, pid_t pid, uint32_t flags, uint64_t events)
+{
+	struct drm_xe_eudebug_connect param = {
+		.pid = pid,
+		.flags = flags,
+	};
+	int debugfd;
+
+	debugfd = igt_ioctl(fd, DRM_IOCTL_XE_EUDEBUG_CONNECT, &param);
+
+	if (debugfd < 0)
+		return -errno;
+
+	return debugfd;
+}
+
+static void event_log_write_to_fd(struct xe_eudebug_event_log *l, int fd)
+{
+	igt_assert_eq(write(fd, &l->head, sizeof(l->head)),
+		      sizeof(l->head));
+
+	igt_assert_eq(write(fd, l->log, l->head), l->head);
+}
+
+static void read_all(int fd, void *buf, size_t nbytes)
+{
+	ssize_t remaining_size = nbytes;
+	ssize_t current_size = 0;
+	ssize_t read_size = 0;
+

lib/xe/xe_eudebug.c line 353

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+	do {
+		read_size = read(fd, buf + current_size, remaining_size);
+		igt_assert_f(read_size >= 0, "read failed: %s\n", strerror(errno));
+
+		current_size += read_size;
+		remaining_size -= read_size;
+	} while (remaining_size > 0 && read_size > 0);
+
+	igt_assert_eq(current_size, nbytes);
+}

I see fd wasn't set to nonblock. Isn't

Quoted 1 line

igt_assert_eq(read(fd, buf, nbytes), nbytes) enough?

I saw read fail to read all bytes in one go here and only then added this. It seems to related to the fact that we are reading from a pipe here. The read manpage says:

[...]
On success, the number of bytes read is returned (zero indicates end of file), and the file position is advanced by this number. It is not an error if this number is smaller than the number of bytes requested; this may happen for example because fewer bytes are actually available right now (maybe because we were close to end-of-file, or because we are reading from a pipe [...]

Quoted 16 lines

lib/xe/xe_eudebug.c line 363

Quoted 12 lines
lib/xe/xe_eudebug.c new file +10 -0
+
+static void event_log_read_from_fd(struct xe_eudebug_event_log *l, int fd)
+{
+	read_all(fd, &l->head, sizeof(l->head));
+	igt_assert_lt(l->head, l->max_size);
+
+	read_all(fd, l->log, l->head);
+}
+
+typedef int (*cmp_fn_t)(struct drm_xe_eudebug_event *, void *);

I wondered why void * here, but I see you have different compare functions below.

Quoted 16 lines

lib/xe/xe_eudebug.c line 373

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+
+static struct drm_xe_eudebug_event *
+event_cmp(struct xe_eudebug_event_log *l,
+	  struct drm_xe_eudebug_event *current,
+	  cmp_fn_t match,
+	  void *data)
+{
+	struct drm_xe_eudebug_event *e = current;
+
+	xe_eudebug_for_each_event(e, l) {

I've quickly took a look to macro definition and it is confusing 'for_each_event'. It seems it depends on e which really goes over all events if current is NULL. Maybe it would be worth to introduce xe_eudebug_for_each_event_startint_at macro to avoid depending on e value before the start?

Unfortunately I don't follow, can you ellaborate? How is this macro supposed to work and what would be the benefit of it?

Quoted 87 lines
Quoted 74 lines
lib/xe/xe_eudebug.c new file +73 -0
+		if (match(e, data))
+			return e;
+	}
+
+	return NULL;
+}
+
+static int match_type_and_flags(struct drm_xe_eudebug_event *a, void *data)
+{
+	struct drm_xe_eudebug_event *b = data;
+
+	if (a->type == b->type &&
+	    a->flags == b->flags)
+		return 1;
+
+	return 0;
+}
+
+static int match_fields(struct drm_xe_eudebug_event *a, void *data)
+{
+	struct drm_xe_eudebug_event *b = data;
+	int ret = 0;
+
+	ret = match_type_and_flags(a, data);
+	if (!ret)
+		return ret;
+
+	ret = 0;
+
+	switch (a->type) {
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE: {
+		struct drm_xe_eudebug_event_exec_queue *ae = (void *)a;
+		struct drm_xe_eudebug_event_exec_queue *be = (void *)b;
+
+		if (ae->engine_class == be->engine_class && ae->width == be->width)
+			ret = 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND: {
+		struct drm_xe_eudebug_event_vm_bind *ea = (void *)a;
+		struct drm_xe_eudebug_event_vm_bind *eb = (void *)b;
+
+		if (ea->num_binds == eb->num_binds)
+			ret = 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP: {
+		struct drm_xe_eudebug_event_vm_bind_op *ea = (void *)a;
+		struct drm_xe_eudebug_event_vm_bind_op *eb = (void *)b;
+
+		if (ea->addr == eb->addr && ea->range == eb->range &&
+		    ea->num_extensions == eb->num_extensions)
+			ret = 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA: {
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *ea = (void *)a;
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *eb = (void *)b;
+
+		if (ea->metadata_handle == eb->metadata_handle &&
+		    ea->metadata_cookie == eb->metadata_cookie)
+			ret = 1;
+		break;
+	}
+
+	default:
+		ret = 1;
+		break;
+	}
+
+	return ret;
+}
+

lib/xe/xe_eudebug.c line 456

Quoted 10 lines
lib/xe/xe_eudebug.c new file +10 -0
+static int match_client_handle(struct drm_xe_eudebug_event *e, void *data)
+{
+	struct match_dto *md = (void *)data;
+	uint64_t *bind_seqno = md->bind_seqno;
+	uint64_t *bind_op_seqno = md->bind_op_seqno;
+	uint64_t h = md->client_handle;
+
+	if (XE_EUDEBUG_EVENT_IS_FILTERED(e->type, md->filter))
+		return 0;
+

Instead of long line casting I would temporary add

Quoted 2 lines
#define xe_event_cast_init(__t, __ev, __e) \
	struct drm_xe_eudebug_event_##__t __ev = (struct drm_xe_eudebug_event_##__t)__e

Agreed that some of these lines exceed 100 characters. However I am not a big fan of macros like these to be honnest. A naked cast is clear, a macro like this makes the code less obvious and it doesn't gel well with code completion.

Would a line break work for cases where we are over 100 characters?

Quoted 241 lines

lib/xe/xe_eudebug.c line 459

Quoted 4 lines
lib/xe/xe_eudebug.c new file +3 -0
+	switch (e->type) {
+	case DRM_XE_EUDEBUG_EVENT_OPEN: {
+		struct drm_xe_eudebug_event_client *client = (struct drm_xe_eudebug_event_client *)e;

and use

xe_event_cast_init(client *, client, e);

lib/xe/xe_eudebug.c line 466

Quoted 7 lines
lib/xe/xe_eudebug.c new file +7 -0
+
+		if (client->client_handle == h)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM: {
+		struct drm_xe_eudebug_event_vm *vm = (struct drm_xe_eudebug_event_vm *)e;

xe_event_cast_init(vm *, vm, e);

and so on. Finally undefine, but I see cast pattern is widely used here.

Quoted 220 lines
lib/xe/xe_eudebug.c new file +220 -0
+
+		if (vm->client_handle == h)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE: {
+		struct drm_xe_eudebug_event_exec_queue *ee = (struct drm_xe_eudebug_event_exec_queue *)e;
+
+		if (ee->client_handle == h)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND: {
+		struct drm_xe_eudebug_event_vm_bind *evmb = (struct drm_xe_eudebug_event_vm_bind *)e;
+
+		if (evmb->client_handle == h) {
+			*bind_seqno = evmb->base.seqno;
+			return 1;
+		}
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP: {
+		struct drm_xe_eudebug_event_vm_bind_op *eo = (struct drm_xe_eudebug_event_vm_bind_op *)e;
+
+		if (eo->vm_bind_ref_seqno == *bind_seqno) {
+			*bind_op_seqno = eo->base.seqno;
+			return 1;
+		}
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE: {
+		struct drm_xe_eudebug_event_vm_bind_ufence *ef = (struct drm_xe_eudebug_event_vm_bind_ufence *)e;
+
+		if (ef->vm_bind_ref_seqno == *bind_seqno)
+			return 1;
+
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_METADATA: {
+		struct drm_xe_eudebug_event_metadata *em = (struct drm_xe_eudebug_event_metadata *)e;
+
+		if (em->client_handle == h)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA: {
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *eo = (struct drm_xe_eudebug_event_vm_bind_op_metadata *)e;
+
+		if (eo->vm_bind_op_ref_seqno == *bind_op_seqno)
+			return 1;
+		break;
+	}
+	default:
+		break;
+	}
+
+	return 0;
+}
+
+static int match_opposite_resource(struct drm_xe_eudebug_event *e, void *data)
+{
+	struct drm_xe_eudebug_event *d = (void *)data;
+	int ret;
+
+	d->flags ^= DRM_XE_EUDEBUG_EVENT_CREATE | DRM_XE_EUDEBUG_EVENT_DESTROY;
+	d->flags &= ~(DRM_XE_EUDEBUG_EVENT_NEED_ACK);
+	ret = match_type_and_flags(e, data);
+	d->flags ^= DRM_XE_EUDEBUG_EVENT_CREATE | DRM_XE_EUDEBUG_EVENT_DESTROY;
+
+	if (!ret)
+		return 0;
+
+	switch (e->type) {
+	case DRM_XE_EUDEBUG_EVENT_OPEN: {
+		struct drm_xe_eudebug_event_client *client = (struct drm_xe_eudebug_event_client *)e;
+		struct drm_xe_eudebug_event_client *filter = (struct drm_xe_eudebug_event_client *)data;
+
+		if (client->client_handle == filter->client_handle)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM: {
+		struct drm_xe_eudebug_event_vm *vm = (struct drm_xe_eudebug_event_vm *)e;
+		struct drm_xe_eudebug_event_vm *filter = (struct drm_xe_eudebug_event_vm *)data;
+
+		if (vm->vm_handle == filter->vm_handle)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE: {
+		struct drm_xe_eudebug_event_exec_queue *ee = (void *)e;
+		struct drm_xe_eudebug_event_exec_queue *filter = (struct drm_xe_eudebug_event_exec_queue *)data;
+
+		if (ee->exec_queue_handle == filter->exec_queue_handle)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND: {
+		struct drm_xe_eudebug_event_vm_bind *evmb = (void *)e;
+		struct drm_xe_eudebug_event_vm_bind *filter = (struct drm_xe_eudebug_event_vm_bind *)data;
+
+		if (evmb->vm_handle == filter->vm_handle &&
+		    evmb->num_binds == filter->num_binds)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP: {
+		struct drm_xe_eudebug_event_vm_bind_op *avmb = (void *)e;
+		struct drm_xe_eudebug_event_vm_bind_op *filter = (struct drm_xe_eudebug_event_vm_bind_op *)data;
+
+		if (avmb->addr == filter->addr &&
+		    avmb->range == filter->range)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_METADATA: {
+		struct drm_xe_eudebug_event_metadata *em = (void *)e;
+		struct drm_xe_eudebug_event_metadata *filter = (struct drm_xe_eudebug_event_metadata *)data;
+
+		if (em->metadata_handle == filter->metadata_handle)
+			return 1;
+		break;
+	}
+	case DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA: {
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *avmb = (void *)e;
+		struct drm_xe_eudebug_event_vm_bind_op_metadata *filter = (struct drm_xe_eudebug_event_vm_bind_op_metadata *)data;
+
+		if (avmb->metadata_handle == filter->metadata_handle &&
+		    avmb->metadata_cookie == filter->metadata_cookie)
+			return 1;
+		break;
+	}
+
+	default:
+		break;
+	}
+	return 0;
+}
+
+static int match_full(struct drm_xe_eudebug_event *e, void *data)
+{
+	struct seqno_list_entry *sl;
+
+	struct match_dto *md = (void *)data;
+	int ret = 0;
+
+	ret = match_client_handle(e, md);
+	if (!ret)
+		return 0;
+
+	ret = match_fields(e, md->target);
+	if (!ret)
+		return 0;
+
+	igt_list_for_each_entry(sl, md->seqno_list, link) {
+		if (sl->seqno == e->seqno)
+			return 0;
+	}
+
+	return 1;
+}
+
+static struct drm_xe_eudebug_event *
+event_type_match(struct xe_eudebug_event_log *l,
+		 struct drm_xe_eudebug_event *target,
+		 struct drm_xe_eudebug_event *current)
+{
+	return event_cmp(l, current, match_type_and_flags, target);
+}
+
+static struct drm_xe_eudebug_event *
+client_match(struct xe_eudebug_event_log *l,
+	     uint64_t client_handle,
+	     struct drm_xe_eudebug_event *current,
+	     uint32_t filter,
+	     uint64_t *bind_seqno,
+	     uint64_t *bind_op_seqno)
+{
+	struct match_dto md = {
+		.client_handle = client_handle,
+		.filter = filter,
+		.bind_seqno = bind_seqno,
+		.bind_op_seqno = bind_op_seqno,
+	};
+
+	return event_cmp(l, current, match_client_handle, &md);
+}
+
+static struct drm_xe_eudebug_event *
+opposite_event_match(struct xe_eudebug_event_log *l,
+		     struct drm_xe_eudebug_event *target,
+		     struct drm_xe_eudebug_event *current)
+{
+	return event_cmp(l, current, match_opposite_resource, target);
+}
+
+static struct drm_xe_eudebug_event *
+event_match(struct xe_eudebug_event_log *l,
+	    struct drm_xe_eudebug_event *target,
+	    uint64_t client_handle,
+	    struct igt_list_head *seqno_list,
+	    uint64_t *bind_seqno,
+	    uint64_t *bind_op_seqno)
+{
+	struct match_dto md = {
+		.target = target,
+		.client_handle = client_handle,
+		.seqno_list = seqno_list,
+		.bind_seqno = bind_seqno,
+		.bind_op_seqno = bind_op_seqno,
+	};
+
+	return event_cmp(l, NULL, match_full, &md);
+}
+
+static void compare_client(struct xe_eudebug_event_log *c, struct drm_xe_eudebug_event *_ce,
+			   struct xe_eudebug_event_log *d, struct drm_xe_eudebug_event *_de,
+			   uint32_t filter)
+{
+	struct drm_xe_eudebug_event_client *ce = (void *)_ce;
Quoted 8 lines

lib/xe/xe_eudebug.c line 691

Quoted 5 lines
lib/xe/xe_eudebug.c new file +5 -0
+	struct drm_xe_eudebug_event_client *de = (void *)_de;
+	uint64_t cbs = 0, dbs = 0, cbso = 0, dbso = 0;
+
+	struct igt_list_head matched_seqno_list;
+	struct drm_xe_eudebug_event *hc, *hd;

Variable names are not self-descriptive. Are you able to guess what c/d/ce/de/hc/hd means?

I understand. The 'c' and 'd' maybe could mean 'client' and 'debugger' but since the order of the parameters isn't enforced in anyway it may as well be 'a' and 'b' or 'x' and 'y'. In the end these are just temporary variables to represent some 'a' and 'b' value to compare.

would:
log_a/log_b/ev_a/ev_b/ev_ptr_a/ev_ptr_b
work better? I am open for suggestions.

Quoted 68 lines
Quoted 68 lines
lib/xe/xe_eudebug.c new file +67 -0
+	struct seqno_list_entry *entry, *tmp;
+
+	igt_assert(ce);
+	igt_assert(de);
+
+	igt_debug("client: %llu -> %llu\n", ce->client_handle, de->client_handle);
+
+	hc = NULL;
+	hd = NULL;
+	IGT_INIT_LIST_HEAD(&matched_seqno_list);
+
+	do {
+		hc = client_match(c, ce->client_handle, hc, filter, &cbs, &cbso);
+		if (!hc)
+			break;
+
+		hd = event_match(d, hc, de->client_handle, &matched_seqno_list, &dbs, &dbso);
+
+		igt_assert_f(hd, "%s (%llu): no matching event type %u found for client %llu\n",
+			     c->name,
+			     hc->seqno,
+			     hc->type,
+			     ce->client_handle);
+
+		igt_debug("comparing %s %llu vs %s %llu\n",
+			  c->name, hc->seqno, d->name, hd->seqno);
+
+		/*
+		 * Store the seqno of the event that was matched above,
+		 * inside 'matched_seqno_list', to avoid it getting matched
+		 * by subsequent 'event_match' calls.
+		 */
+		entry = malloc(sizeof(*entry));
+		entry->seqno = hd->seqno;
+		igt_list_add(&entry->link, &matched_seqno_list);
+	} while (hc);
+
+	igt_list_for_each_entry_safe(entry, tmp, &matched_seqno_list, link)
+		free(entry);
+}
+
+/**
+ * xe_eudebug_event_log_find_seqno:
+ * @l: event log pointer
+ * @seqno: seqno of event to be found
+ *
+ * Finds the event with given seqno in the event log.
+ *
+ * Returns: pointer to the event with given seqno within @l or NULL seqno is
+ * not present.
+ */
+struct drm_xe_eudebug_event *
+xe_eudebug_event_log_find_seqno(struct xe_eudebug_event_log *l, uint64_t seqno)
+{
+	struct drm_xe_eudebug_event *e = NULL, *found = NULL;
+
+	igt_assert_neq(seqno, 0);
+	/*
+	 * Try to catch if seqno is corrupted and prevent too long tests,
+	 * as our post processing of events is not optimized.
+	 */
+	igt_assert_lt(seqno, 10 * 1000 * 1000);
+
+	xe_eudebug_for_each_event(e, l) {
+		if (e->seqno == seqno) {
+			if (found) {
+				igt_warn("Found multiple events with the same seqno %lu\n", seqno);
Quoted 8 lines

lib/xe/xe_eudebug.c line 762

Quoted 4 lines
lib/xe/xe_eudebug.c new file +4 -0
+				xe_eudebug_event_log_print(l, false);
+				igt_assert(!found);
+			}
+			found = e;

Do I understand correctly that you're trying to catch kernel bug here in the library? Shouldn't this be addressed in the kernel or some eudebug test itself, but not in the library?

From what I recall this stems from a error condition that was difficult to catch and involved some race condition. Adding this check here did ensure maximum coverage for this issue.

Furthermore this function could also be used to search the expected event log, not only the one from the event worker loop. So it serves as some kind of sanity check not only for kernel errors.

Removing this check comes with the disadvantage of reducing coverage and missing a potential chance to warn early that something is broken.

I am not able come up with any advantage of removing it from here so I would ask you to provide some if you are conviced I should remove it.

Quoted 111 lines
Quoted 111 lines
lib/xe/xe_eudebug.c new file +110 -0
+		}
+	}
+
+	return found;
+}
+
+static void event_log_sort(struct xe_eudebug_event_log *l)
+{
+	struct xe_eudebug_event_log *tmp;
+	struct drm_xe_eudebug_event *e = NULL;
+	uint64_t first_seqno = 0;
+	uint64_t last_seqno = 0;
+	uint64_t events = 0, added = 0;
+	uint64_t i;
+
+	xe_eudebug_for_each_event(e, l) {
+		if (e->seqno > last_seqno)
+			last_seqno = e->seqno;
+
+		if (e->seqno < first_seqno)
+			first_seqno = e->seqno;
+
+		events++;
+	}
+
+	tmp = xe_eudebug_event_log_create("tmp", l->max_size);
+
+	for (i = 1; i <= last_seqno; i++) {
+		e = xe_eudebug_event_log_find_seqno(l, i);
+		if (e) {
+			xe_eudebug_event_log_write(tmp, e);
+			added++;
+		}
+	}
+
+	igt_assert_eq(events, added);
+	igt_assert_eq(tmp->head, l->head);
+
+	memcpy(l->log, tmp->log, tmp->head);
+
+	xe_eudebug_event_log_destroy(tmp);
+}
+
+/**
+ * xe_eudebug_connect:
+ * @fd: Xe file descriptor
+ * @pid: client PID
+ * @flags: connection flags
+ *
+ * Opens the xe eu debugger connection to the process described by @pid
+ *
+ * Returns: 0 if the debugger was successfully attached, -errno otherwise.
+ */
+int xe_eudebug_connect(int fd, pid_t pid, uint32_t flags)
+{
+	int ret;
+	uint64_t events = 0; /* events filtering not supported yet! */
+
+	ret = __xe_eudebug_connect(fd, pid, flags, events);
+
+	return ret;
+}
+
+/**
+ * xe_eudebug_event_log_create:
+ * @name: event log identifier
+ * @max_size: maximum size of created log
+ *
+ * Function creates an Eu Debugger event log with size equal to @max_size.
+ *
+ * Returns: pointer to just created log
+ */
+#define MAX_EVENT_LOG_SIZE (32 * 1024 * 1024)
+struct xe_eudebug_event_log *xe_eudebug_event_log_create(const char *name, unsigned int max_size)
+{
+	struct xe_eudebug_event_log *l;
+
+	l = calloc(1, sizeof(*l));
+	igt_assert(l);
+	l->log = calloc(1, max_size);
+	igt_assert(l->log);
+	l->max_size = max_size;
+	strncpy(l->name, name, sizeof(l->name) - 1);
+	pthread_mutex_init(&l->lock, NULL);
+
+	return l;
+}
+
+/**
+ * xe_eudebug_event_log_destroy:
+ * @l: event log pointer
+ *
+ * Frees given event log @l.
+ */
+void xe_eudebug_event_log_destroy(struct xe_eudebug_event_log *l)
+{
+	pthread_mutex_destroy(&l->lock);
+	free(l->log);
+	free(l);
+}
+
+/**
+ * xe_eudebug_event_log_write:
+ * @l: event log pointer
+ * @e: event to be written to event log
+ *
+ * Writes event @e to the event log, thread-safe.
+ */
+void xe_eudebug_event_log_write(struct xe_eudebug_event_log *l, struct drm_xe_eudebug_event *e)
+{
Quoted 8 lines

lib/xe/xe_eudebug.c line 877

Quoted 5 lines
lib/xe/xe_eudebug.c new file +5 -0
+	igt_assert(e->seqno);
+	/*
+	 * Try to catch if seqno is corrupted and prevent too long tests,
+	 * as our post processing of events is not optimized.
+	 */

What means seqno is corrupted? Kernel issues deserve for own test, not library part.

So like above - the provided log is not neccessarily from the event worker loop. It can also be the test/client build log.

Quoted 43 lines
Quoted 43 lines
lib/xe/xe_eudebug.c new file +42 -0
+	igt_assert_lt(e->seqno, 10 * 1000 * 1000);
+
+	pthread_mutex_lock(&l->lock);
+	igt_assert_lt(l->head + e->len, l->max_size);
+	memcpy(l->log + l->head, e, e->len);
+	l->head += e->len;
+	pthread_mutex_unlock(&l->lock);
+}
+
+/**
+ * xe_eudebug_event_log_print:
+ * @l: event log pointer
+ * @debug: when true function uses igt_debug instead of igt_info.
+ *
+ * Prints given event log.
+ */
+void
+xe_eudebug_event_log_print(struct xe_eudebug_event_log *l, bool debug)
+{
+	struct drm_xe_eudebug_event *e = NULL;
+	int level = debug ? IGT_LOG_DEBUG : IGT_LOG_INFO;
+	char str[XE_EUDEBUG_EVENT_STRING_MAX_LEN];
+
+	igt_log(IGT_LOG_DOMAIN, level,
+		"event log '%s' (%u bytes):\n", l->name, l->head);
+
+	xe_eudebug_for_each_event(e, l) {
+		xe_eudebug_event_to_str(e, str, XE_EUDEBUG_EVENT_STRING_MAX_LEN);
+		igt_log(IGT_LOG_DOMAIN, level, "%s\n", str);
+	}
+}
+
+/**
+ * xe_eudebug_event_log_compare:
+ * @a: event log pointer
+ * @b: event log pointer
+ * @filter: mask that represents events to be skipped during comparison, useful
+ * for events like 'VM_BIND' since they can be asymmetric. Note that
+ * 'DRM_XE_EUDEBUG_EVENT_OPEN' will always be matched.
+ *
+ * Compares and asserts event logs @a, @b if the event
+ * sequence matches.
Quoted 8 lines

lib/xe/xe_eudebug.c line 925

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+ */
+void xe_eudebug_event_log_compare(struct xe_eudebug_event_log *a, struct xe_eudebug_event_log *b,
+				  uint32_t filter)
+{
+	struct drm_xe_eudebug_event *ae = NULL;
+	struct drm_xe_eudebug_event *be = NULL;

I'm not happy with variable naming here as well.

Ok, would:

log_a/log_b/ev_a/ev_b

work better?

Quoted 20 lines
Quoted 20 lines
lib/xe/xe_eudebug.c new file +19 -0
+
+	xe_eudebug_for_each_event(ae, a) {
+		if (ae->type == DRM_XE_EUDEBUG_EVENT_OPEN &&
+		    ae->flags & DRM_XE_EUDEBUG_EVENT_CREATE) {
+			be = event_type_match(b, ae, be);
+
+			compare_client(a, ae, b, be, filter);
+			compare_client(b, be, a, ae, filter);
+		}
+	}
+}
+
+/**
+ * xe_eudebug_event_log_match_opposite:
+ * @l: event log pointer
+ * @filter: mask that represents events to be skipped during comparison, useful
+ * for events like 'VM_BIND' since they can be asymmetric
+ *
+ * Matches and asserts content of all opposite events (create vs destroy).
Quoted 8 lines

lib/xe/xe_eudebug.c line 950

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+ */
+void
+xe_eudebug_event_log_match_opposite(struct xe_eudebug_event_log *l, uint32_t filter)
+{
+	struct drm_xe_eudebug_event *ce = NULL;
+	struct drm_xe_eudebug_event *de = NULL;

Ditto

See above.

Quoted 92 lines
Quoted 92 lines
lib/xe/xe_eudebug.c new file +91 -0
+
+	xe_eudebug_for_each_event(ce, l) {
+		if (ce->flags & DRM_XE_EUDEBUG_EVENT_CREATE) {
+			uint8_t offset = sizeof(struct drm_xe_eudebug_event);
+			int opposite_matching;
+
+			if (XE_EUDEBUG_EVENT_IS_FILTERED(ce->type, filter))
+				continue;
+
+			/* No opposite matching for binds */
+			if ((ce->type >= DRM_XE_EUDEBUG_EVENT_VM_BIND &&
+			     ce->type <= DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE) ||
+			    ce->type == DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA)
+				continue;
+
+			de = opposite_event_match(l, ce, ce);
+
+			igt_assert_f(de, "no opposite event of type %u found\n", ce->type);
+
+			igt_assert_eq(ce->len, de->len);
+			opposite_matching = memcmp((uint8_t *)de + offset,
+						   (uint8_t *)ce + offset,
+						   de->len - offset) == 0;
+
+			igt_assert_f(opposite_matching,
+				     "%s: create|destroy event not maching (%llu) vs (%llu)\n",
+				     l->name, de->seqno, ce->seqno);
+		}
+	}
+}
+
+static void debugger_run_triggers(struct xe_eudebug_debugger *d,
+				  struct drm_xe_eudebug_event *e)
+{
+	struct event_trigger *t;
+
+	igt_list_for_each_entry(t, &d->triggers, link) {
+		if (e->type == t->type)
+			t->fn(d, e);
+	}
+}
+
+#define MAX_EVENT_SIZE (32 * 1024)
+static int
+xe_eudebug_read_event(int fd, struct drm_xe_eudebug_event *event)
+{
+	int ret;
+
+	event->type = DRM_XE_EUDEBUG_EVENT_READ;
+	event->flags = 0;
+	event->len = MAX_EVENT_SIZE;
+
+	ret = igt_ioctl(fd, DRM_XE_EUDEBUG_IOCTL_READ_EVENT, event);
+	if (ret < 0)
+		return -errno;
+
+	return ret;
+}
+
+static void *debugger_worker_loop(void *data)
+{
+	uint8_t buf[MAX_EVENT_SIZE];
+	struct drm_xe_eudebug_event *e = (void *)buf;
+	struct xe_eudebug_debugger *d = data;
+	struct pollfd p = {
+		.events = POLLIN,
+		.revents = 0,
+	};
+	int timeout_ms = 100, ret;
+
+	igt_assert(d->master_fd >= 0);
+
+	do {
+		p.fd = d->fd;
+		ret = poll(&p, 1, timeout_ms);
+
+		if (ret == -1) {
+			igt_info("poll failed with errno %d\n", errno);
+			break;
+		}
+
+		if (ret == 1 && (p.revents & POLLIN)) {
+			int err = xe_eudebug_read_event(d->fd, e);
+
+			if (!err) {
+				++d->event_count;
+
+				xe_eudebug_event_log_write(d->log, e);
+				debugger_run_triggers(d, e);
+			} else {
+				igt_info("xe_eudebug_read_event returned %d\n", ret);
Quoted 8 lines

lib/xe/xe_eudebug.c line 1047

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+			}
+		}
+	} while ((ret && READ_ONCE(d->worker_state) == DEBUGGER_WORKER_QUITTING) ||
+		 READ_ONCE(d->worker_state) == DEBUGGER_WORKER_ACTIVE);
+
+	d->worker_state = DEBUGGER_WORKER_INACTIVE;

Please add blank line here.

Sure, will do.

Quoted 35 lines
Quoted 35 lines
lib/xe/xe_eudebug.c new file +34 -0
+	return NULL;
+}
+
+/**
+ * xe_eudebug_debugger_available:
+ * @fd: Xe file descriptor
+ *
+ * Returns: true it debugger connection is available, false otherwise.
+ */
+bool xe_eudebug_debugger_available(int fd)
+{
+	struct drm_xe_eudebug_connect param = { .pid = getpid() };
+	int debugfd;
+
+	debugfd = igt_ioctl(fd, DRM_IOCTL_XE_EUDEBUG_CONNECT, &param);
+	if (debugfd >= 0)
+		close(debugfd);
+
+	return debugfd >= 0;
+}
+
+/**
+ * xe_eudebug_debugger_create:
+ * @master_fd: xe client used to open the debugger connection
+ * @flags: flags stored in a debugger structure, can be used at will
+ * of the caller, i.e. to be used inside triggers.
+ * @data: test's private data, allocated with MAP_SHARED | MAP_ANONYMOUS,
+ * can be shared between client and debugger. Can be NULL.
+ *
+ * Returns: newly created xe_eudebug_debugger structure with its
+ * event log initialized. Note that to open the connection
+ * you need call @xe_eudebug_debugger_attach.
+ */
+struct xe_eudebug_debugger *
Quoted 8 lines

lib/xe/xe_eudebug.c line 1087

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+xe_eudebug_debugger_create(int master_fd, uint64_t flags, void *data)
+{
+	struct xe_eudebug_debugger *d;
+
+	d = calloc(1, sizeof(*d));
+	d->flags = flags;

I guess assert should be before dereference.

Indeed, thanks.

Quoted 27 lines
Quoted 27 lines
lib/xe/xe_eudebug.c new file +26 -0
+	igt_assert(d);
+	IGT_INIT_LIST_HEAD(&d->triggers);
+	d->log = xe_eudebug_event_log_create("debugger", MAX_EVENT_LOG_SIZE);
+	d->fd = -1;
+	d->master_fd = master_fd;
+	d->ptr = data;
+
+	return d;
+}
+
+static void debugger_destroy_triggers(struct xe_eudebug_debugger *d)
+{
+	struct event_trigger *t, *tmp;
+
+	igt_list_for_each_entry_safe(t, tmp, &d->triggers, link)
+		free(t);
+}
+
+/**
+ * xe_eudebug_debugger_destroy:
+ * @d: pointer to the debugger
+ *
+ * Frees xe_eudebug_debugger structure pointed by @d. If the debugger
+ * connection was still opened it terminates it.
+ */
+void xe_eudebug_debugger_destroy(struct xe_eudebug_debugger *d)
Quoted 8 lines

lib/xe/xe_eudebug.c line 1119

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+{
+	if (d->worker_state)
+		xe_eudebug_debugger_stop_worker(d, 1);
+
+	if (d->target_pid)
+		xe_eudebug_debugger_dettach(d);

Typo, s/dettach/detach/

Will fix.

Quoted 34 lines
Quoted 34 lines
lib/xe/xe_eudebug.c new file +33 -0
+
+	xe_eudebug_event_log_destroy(d->log);
+	debugger_destroy_triggers(d);
+	free(d);
+}
+
+/**
+ * xe_eudebug_debugger_attach:
+ * @d: pointer to the debugger
+ * @c: pointer to the client
+ *
+ * Opens the xe eu debugger connection to the process described by @c (c->pid)
+ *
+ * Returns: 0 if the debugger was successfully attached, -errno otherwise.
+ */
+int xe_eudebug_debugger_attach(struct xe_eudebug_debugger *d,
+			       struct xe_eudebug_client *c)
+{
+	int ret;
+
+	igt_assert_eq(d->fd, -1);
+	igt_assert_neq(c->pid, 0);
+	ret = xe_eudebug_connect(d->master_fd, c->pid, 0);
+
+	if (ret < 0)
+		return ret;
+
+	d->fd = ret;
+	d->target_pid = c->pid;
+	d->p_client[0] = c->p_in[0];
+	d->p_client[1] = c->p_in[1];
+
+	igt_debug("debugger connected to %lu\n", d->target_pid);
Quoted 8 lines

lib/xe/xe_eudebug.c line 1158

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+
+	return 0;
+}
+
+/**
+ * xe_eudebug_debugger_dettach:

Ditto.

Yup.

Quoted 26 lines

lib/xe/xe_eudebug.c line 1164

Quoted 7 lines
lib/xe/xe_eudebug.c new file +6 -0
+ * @d: pointer to the debugger
+ *
+ * Closes previously opened xe eu debugger connection. Asserts if
+ * the debugger has active session.
+ */
+void xe_eudebug_debugger_dettach(struct xe_eudebug_debugger *d)

Ditto >

Quoted 17 lines
lib/xe/xe_eudebug.c new file +17 -0
+{
+	igt_assert(d->target_pid);
+	close(d->fd);
+	d->target_pid = 0;
+	d->fd = -1;
+}
+
+/**
+ * xe_eudebug_debugger_add_trigger:
+ * @d: pointer to the debugger
+ * @type: the type of the event which activates the trigger
+ * @fn: function to be called when event of @type was read by the debugger.
+ *
+ * Adds function @fn to the list of triggers activated when event of @type
+ * has been read by worker.
+ * Note: Triggers are activated by the worker.
+ */
Quoted 8 lines

lib/xe/xe_eudebug.c line 1187

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+void xe_eudebug_debugger_add_trigger(struct xe_eudebug_debugger *d,
+				     int type, xe_eudebug_trigger_fn fn)
+{
+	struct event_trigger *t;
+
+	t = calloc(1, sizeof(*t));

Assert if allocation failed.

Sure!

Quoted 18 lines
Quoted 18 lines
lib/xe/xe_eudebug.c new file +17 -0
+	IGT_INIT_LIST_HEAD(&t->link);
+	t->type = type;
+	t->fn = fn;
+
+	igt_list_add_tail(&t->link, &d->triggers);
+	igt_debug("added trigger %p\n", t);
+}
+
+/**
+ * xe_eudebug_debugger_start_worker:
+ * @d: pointer to the debugger
+ *
+ * Starts the debugger worker. Worker is resposible for reading all
+ * incoming events from the debugger, put then into debugger log and
+ * execute appropriate event triggers. Note that using the debuggers
+ * event log while worker is running is not safe.
+ */
Quoted 8 lines

lib/xe/xe_eudebug.c line 1209

Quoted 5 lines
lib/xe/xe_eudebug.c new file +5 -0
+void xe_eudebug_debugger_start_worker(struct xe_eudebug_debugger *d)
+{
+	int ret;
+
+	d->worker_state = true;

I'm not aware how rest of the code looks like, but setting worker_state to true before worker thread started looks racy.

It's not my code but when thinking about it I don't quite see why that would be racy.

Basically we have two threads - one that starts/stops the worker, the other is the worker loop.

Setting the worker_state after starting the worker thread won't work since the worker loop may exit.

Setting the worker_state inside the worker thread could lead to a scenario where the debugger_destroy function doesn't call the stop_worker function as the worker_state is not yet set.

I can't seem to find a race for the case where we initialize it before unless we call the start/stop functions from different threads which is a not forseen usecase.

I think however that we could set the state to an explicit state instead of simply setting 'true'.

Quoted 18 lines
Quoted 18 lines
lib/xe/xe_eudebug.c new file +16 -0
+	ret = pthread_create(&d->worker_thread, NULL, &debugger_worker_loop, d);
+
+	igt_assert_f(ret == 0, "Debugger worker thread creation failed!");
+}
+
+/**
+ * xe_eudebug_debugger_stop_worker:
+ * @d: pointer to the debugger
+ *
+ * Stops the debugger worker. Event log is sorted by seqno after closure.
+ */
+void xe_eudebug_debugger_stop_worker(struct xe_eudebug_debugger *d,
+				     int timeout_s)
+{
+	struct timespec t = {};
+	int ret;
Quoted 8 lines

lib/xe/xe_eudebug.c line 1229

Quoted 4 lines
lib/xe/xe_eudebug.c new file +4 -0
+
+	igt_assert(d->worker_state);
+
+	d->worker_state = DEBUGGER_WORKER_QUITTING; /* First time be polite. */

Shoudln't this be WRITE_ONCE()? I see READ_ONCE() is used for reading this variable, but write imo may be delayed. Check other usages of this variable.

So from our offline discussion I conclude that this is not required since the fucntion is exported.

Quoted 69 lines
Quoted 69 lines
lib/xe/xe_eudebug.c new file +68 -0
+	igt_assert_eq(clock_gettime(CLOCK_REALTIME, &t), 0);
+	t.tv_sec += timeout_s;
+
+	ret = pthread_timedjoin_np(d->worker_thread, NULL, &t);
+
+	if (ret == ETIMEDOUT) {
+		d->worker_state = DEBUGGER_WORKER_INACTIVE;
+		ret = pthread_join(d->worker_thread, NULL);
+	}
+
+	igt_assert_f(ret == 0 || ret != ESRCH,
+		     "pthread join failed with error %d!\n", ret);
+
+	event_log_sort(d->log);
+}
+
+/**
+ * xe_eudebug_debugger_signal_stage:
+ * @d: pointer to the debugger
+ * @stage: stage to signal
+ *
+ * Signals to client, waiting in xe_eudebug_client_wait_stage(),
+ * releasing it to proceed.
+ */
+void xe_eudebug_debugger_signal_stage(struct xe_eudebug_debugger *d, uint64_t stage)
+{
+	token_signal(d->p_client, CLIENT_STAGE, stage);
+}
+
+/**
+ * xe_eudebug_debugger_wait_stage:
+ * @s: pointer to xe_eudebug_debugger structure
+ * @stage: stage to wait on
+ *
+ * Pauses debugger until the client has signalled the corresponding stage with
+ * xe_eudebug_client_signal_stage. This is only for situations where the actual
+ * event flow is not enough to coordinate between client/debugger and extra sync
+ * mechanism is needed.
+ */
+void xe_eudebug_debugger_wait_stage(struct xe_eudebug_session *s, uint64_t stage)
+{
+	u64 stage_in;
+
+	igt_debug("debugger xe client fd: %d pausing for stage %lu\n", s->d->master_fd, stage);
+
+	stage_in = wait_from_client(s->c, DEBUGGER_STAGE);
+	igt_debug("debugger xe client fd: %d stage %lu, expected %lu, stage\n", s->d->master_fd,
+		  stage_in, stage);
+
+	igt_assert_eq(stage_in, stage);
+}
+
+/**
+ * xe_eudebug_client_create:
+ * @master_fd: xe client used to open the debugger connection
+ * @work: function that opens xe device and executes arbitrary workload
+ * @flags: flags stored in a client structure, can be used at will
+ * of the caller, i.e. to provide the @work function an additional switch.
+ * @data: test's private data, allocated with MAP_SHARED | MAP_ANONYMOUS,
+ * can be shared between client and debugger. Accesible via client->ptr.
+ * Can be NULL.
+ *
+ * Forks and creates the debugger process. @work won't be called until
+ * xe_eudebug_client_start is called.
+ *
+ * Returns: newly created xe_eudebug_debugger structure with its
+ * event log initialized.
+ */
Quoted 8 lines

lib/xe/xe_eudebug.c line 1303

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+struct xe_eudebug_client *xe_eudebug_client_create(int master_fd, xe_eudebug_client_work_fn work,
+						   uint64_t flags, void *data)
+{
+	struct xe_eudebug_client *c;
+
+	c = calloc(1, sizeof(*c));

Assert here.

Sure.

Quoted 242 lines
Quoted 242 lines
lib/xe/xe_eudebug.c new file +241 -0
+	c->flags = flags;
+	igt_assert(c);
+	igt_assert(!pipe(c->p_in));
+	igt_assert(!pipe(c->p_out));
+	c->seqno = 1;
+	c->log = xe_eudebug_event_log_create("client", MAX_EVENT_LOG_SIZE);
+	c->done = 0;
+	c->ptr = data;
+	c->master_fd = master_fd;
+	c->timeout_ms = XE_EUDEBUG_DEFAULT_TIMEOUT_SEC * MSEC_PER_SEC;
+
+	igt_fork(child, 1) {
+		int mypid;
+
+		igt_assert_eq(c->pid, 0);
+
+		close(c->p_out[0]);
+		c->p_out[0] = -1;
+		close(c->p_in[1]);
+		c->p_in[1] = -1;
+
+		mypid = getpid();
+		client_signal(c, CLIENT_PID, mypid);
+
+		c->pid = client_wait_token(c, CLIENT_RUN);
+		igt_assert_eq(c->pid, mypid);
+		if (work)
+			work(c);
+
+		client_signal(c, CLIENT_FINI, c->seqno);
+
+		event_log_write_to_fd(c->log, c->p_out[1]);
+
+		c->pid = client_wait_token(c, CLIENT_STOP);
+		igt_assert_eq(c->pid, mypid);
+	}
+
+	close(c->p_out[1]);
+	c->p_out[1] = -1;
+	close(c->p_in[0]);
+	c->p_in[0] = -1;
+
+	c->pid = wait_from_client(c, CLIENT_PID);
+
+	igt_info("client running with pid %d\n", c->pid);
+
+	return c;
+}
+
+/**
+ * xe_eudebug_client_stop:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Waits for the end of client's work and exits the proccess.
+ */
+void xe_eudebug_client_stop(struct xe_eudebug_client *c)
+{
+	if (c->pid) {
+		int waitstatus;
+
+		xe_eudebug_client_wait_done(c);
+
+		token_signal(c->p_in, CLIENT_STOP, c->pid);
+		igt_assert_eq(waitpid(c->pid, &waitstatus, 0),
+			      c->pid);
+		c->pid = 0;
+	}
+}
+
+/**
+ * xe_eudebug_client_destroy:
+ * @c: pointer to xe_eudebug_client structure to be freed
+ *
+ * Frees the @c client structure. Note that it calls xe_eudebug_client_stop if
+ * client proccess has not terminated yet.
+ */
+void xe_eudebug_client_destroy(struct xe_eudebug_client *c)
+{
+	xe_eudebug_client_stop(c);
+	pipe_close(c->p_in);
+	pipe_close(c->p_out);
+	xe_eudebug_event_log_destroy(c->log);
+	free(c);
+}
+
+/**
+ * xe_eudebug_client_get_seqno:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Increments and returns current seqno value of the given client @c
+ *
+ * Returns: incremented seqno
+ */
+uint64_t xe_eudebug_client_get_seqno(struct xe_eudebug_client *c)
+{
+	return c->seqno++;
+}
+
+/**
+ * xe_eudebug_client_start:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Starts execution of client's work function within the client's proccess.
+ */
+void xe_eudebug_client_start(struct xe_eudebug_client *c)
+{
+	token_signal(c->p_in, CLIENT_RUN, c->pid);
+}
+
+/**
+ * xe_eudebug_client_wait_done:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Waits for the client work end updates the event log.
+ * Doesn't terminate the client's proccess yet.
+ */
+void xe_eudebug_client_wait_done(struct xe_eudebug_client *c)
+{
+	if (!c->done) {
+		c->done = 1;
+		c->seqno = wait_from_client(c, CLIENT_FINI);
+		event_log_read_from_fd(c->log, c->p_out[0]);
+	}
+}
+
+/**
+ * xe_eudebug_client_signal_stage:
+ * @c: pointer to the client
+ * @stage: stage to signal
+ *
+ * Signals to debugger, waiting in xe_eudebug_debugger_wait_stage(),
+ * releasing it to proceed.
+ */
+void xe_eudebug_client_signal_stage(struct xe_eudebug_client *c, uint64_t stage)
+{
+	token_signal(c->p_out, DEBUGGER_STAGE, stage);
+}
+
+/**
+ * xe_eudebug_client_wait_stage:
+ * @c: pointer to xe_eudebug_client structure
+ * @stage: stage to wait on
+ *
+ * Pauses client until the debugger has signalled the corresponding stage with
+ * xe_eudebug_debugger_signal_stage. This is only for situations where the
+ * actual event flow is not enough to coordinate between client/debugger and extra
+ * sync mechanism is needed.
+ *
+ */
+void xe_eudebug_client_wait_stage(struct xe_eudebug_client *c, uint64_t stage)
+{
+	u64 stage_in;
+
+	if (c->done) {
+		igt_warn("client: %d already done before %lu\n", c->pid, stage);
+		return;
+	}
+
+	igt_debug("client: %d pausing for stage %lu\n", c->pid, stage);
+
+	stage_in = client_wait_token(c, CLIENT_STAGE);
+	igt_debug("client: %d stage %lu, expected %lu, stage\n", c->pid, stage_in, stage);
+
+	igt_assert_eq(stage_in, stage);
+}
+
+/**
+ * xe_eudebug_session_create:
+ * @fd: XE file descriptor
+ * @work: function passed to the xe_eudebug_client_create
+ * @flags: flags passed to client and debugger
+ * @test_private: test's  data, allocated with MAP_SHARED | MAP_ANONYMOUS,
+ * passed to client and debugger. Can be NULL.
+ *
+ * Creates session together with client and debugger structures.
+ */
+struct xe_eudebug_session *xe_eudebug_session_create(int fd,
+						     xe_eudebug_client_work_fn work,
+						     unsigned int flags,
+						     void *test_private)
+{
+	struct xe_eudebug_session *s;
+
+	s = calloc(1, sizeof(*s));
+	igt_assert(s);
+
+	s->c = xe_eudebug_client_create(fd, work, flags, test_private);
+	s->d = xe_eudebug_debugger_create(fd, flags, test_private);
+	s->flags = flags;
+
+	return s;
+}
+
+/**
+ * xe_eudebug_session_run:
+ * @s: pointer to xe_eudebug_session structure
+ *
+ * Attaches debugger to client's proccess, starts debugger's
+ * async event reader, starts client and once client finish
+ * it stops debugger worker.
+ */
+void xe_eudebug_session_run(struct xe_eudebug_session *s)
+{
+	struct xe_eudebug_debugger *debugger = s->d;
+	struct xe_eudebug_client *client = s->c;
+
+	igt_assert_eq(xe_eudebug_debugger_attach(debugger, client), 0);
+
+	xe_eudebug_debugger_start_worker(debugger);
+
+	xe_eudebug_client_start(client);
+	xe_eudebug_client_wait_done(client);
+
+	xe_eudebug_debugger_stop_worker(debugger, 1);
+
+	xe_eudebug_event_log_print(debugger->log, true);
+	xe_eudebug_event_log_print(client->log, true);
+}
+
+/**
+ * xe_eudebug_session_check:
+ * @s: pointer to xe_eudebug_session structure
+ * @match_opposite: indicates whether check should match all
+ * create and destroy events.
+ * @filter: mask that represents events to be skipped during comparison, useful
+ * for events like 'VM_BIND' since they can be asymmetric
+ *
+ * Validate debugger's log against the log created by the client.
+ */
+void xe_eudebug_session_check(struct xe_eudebug_session *s, bool match_opposite, uint32_t filter)
+{
+	xe_eudebug_event_log_compare(s->c->log, s->d->log, filter);
+
+	if (match_opposite)
+		xe_eudebug_event_log_match_opposite(s->d->log, filter);
+}
+
+/**
+ * xe_eudebug_session_destroy:
+ * @s: pointer to xe_eudebug_session structure
+ *
Quoted 8 lines

lib/xe/xe_eudebug.c line 1550

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+ * Destroy session together with its debugger and client.
+ */
+void xe_eudebug_session_destroy(struct xe_eudebug_session *s)
+{
+	xe_eudebug_debugger_destroy(s->d);
+	xe_eudebug_client_destroy(s->c);

I think s->debugger and s->client would be more readable.

Ok, I will rename it.

Quoted 75 lines
Quoted 75 lines
lib/xe/xe_eudebug.c new file +75 -0
+
+	free(s);
+}
+
+#define to_base(x) ((struct drm_xe_eudebug_event *)&(x))
+
+static void base_event(struct xe_eudebug_client *c,
+		       struct drm_xe_eudebug_event *e,
+		       uint32_t type,
+		       uint32_t flags,
+		       uint64_t size)
+{
+	e->type = type;
+	e->flags = flags;
+	e->seqno = xe_eudebug_client_get_seqno(c);
+	e->len = size;
+}
+
+static void client_event(struct xe_eudebug_client *c, uint32_t flags, int client_fd)
+{
+	struct drm_xe_eudebug_event_client ec;
+
+	base_event(c, to_base(ec), DRM_XE_EUDEBUG_EVENT_OPEN, flags, sizeof(ec));
+
+	ec.client_handle = client_fd;
+
+	xe_eudebug_event_log_write(c->log, (void *)&ec);
+}
+
+static void vm_event(struct xe_eudebug_client *c, uint32_t flags, int client_fd, uint32_t vm_id)
+{
+	struct drm_xe_eudebug_event_vm evm;
+
+	base_event(c, to_base(evm), DRM_XE_EUDEBUG_EVENT_VM, flags, sizeof(evm));
+
+	evm.client_handle = client_fd;
+	evm.vm_handle = vm_id;
+
+	xe_eudebug_event_log_write(c->log, (void *)&evm);
+}
+
+static void exec_queue_event(struct xe_eudebug_client *c, uint32_t flags,
+			     int client_fd, uint32_t vm_id,
+			     uint32_t exec_queue_handle, uint16_t class,
+			     uint16_t width)
+{
+	struct drm_xe_eudebug_event_exec_queue ee;
+
+	base_event(c, to_base(ee), DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE,
+		   flags, sizeof(ee));
+
+	ee.client_handle = client_fd;
+	ee.vm_handle = vm_id;
+	ee.exec_queue_handle = exec_queue_handle;
+	ee.engine_class = class;
+	ee.width = width;
+
+	xe_eudebug_event_log_write(c->log, (void *)&ee);
+}
+
+static void metadata_event(struct xe_eudebug_client *c, uint32_t flags,
+			   int client_fd, uint32_t id, uint64_t type, uint64_t len)
+{
+	struct drm_xe_eudebug_event_metadata em;
+
+	base_event(c, to_base(em), DRM_XE_EUDEBUG_EVENT_METADATA,
+		   flags, sizeof(em));
+
+	em.client_handle = client_fd;
+	em.metadata_handle = id;
+	em.type = type;
+	em.len = len;
+
+	xe_eudebug_event_log_write(c->log, (void *)&em);
+}
Quoted 8 lines

lib/xe/xe_eudebug.c line 1631

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+
+static int enable_getset(int fd, bool *old, bool *new)
+{
+	static const char * const fname = "enable_eudebug";
+	int ret = 0;
+

Unnecessary blank line.

Sure - I will remove it. It's a weird habbit of mine to split initialized and uninitialized variables.

Quoted 10 lines

lib/xe/xe_eudebug.c line 1638

Quoted 8 lines
lib/xe/xe_eudebug.c new file +7 -0
+	int sysfs, device_fd;
+	bool val_before;
+	struct stat st;
+
+	igt_assert(new || old);
+
+	igt_assert_eq(fstat(fd, &st), 0);

I would group assert within single block separated by blank line.

Ok.

Quoted 24 lines
Quoted 24 lines
lib/xe/xe_eudebug.c new file +23 -0
+	sysfs = igt_sysfs_open(fd);
+	if (sysfs < 0)
+		return -1;
+
+	device_fd = openat(sysfs, "device", O_DIRECTORY | O_RDONLY);
+	close(sysfs);
+	if (device_fd < 0)
+		return -1;
+
+	if (!__igt_sysfs_get_boolean(device_fd, fname, &val_before)) {
+		ret = -1;
+		goto out;
+	}
+
+	igt_debug("enable_eudebug before: %d\n", val_before);
+
+	if (old)
+		*old = val_before;
+
+	ret = 0;
+	if (new) {
+		if (__igt_sysfs_set_boolean(device_fd, fname, *new))
+			igt_assert_eq(igt_sysfs_get_boolean(device_fd, fname), *new);
Quoted 8 lines

lib/xe/xe_eudebug.c line 1667

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+		else
+			ret = -1;
+	}
+
+out:
+	close(device_fd);

Add blank line.

Ok.

Quoted 39 lines
Quoted 39 lines
lib/xe/xe_eudebug.c new file +38 -0
+	return ret;
+}
+
+/**
+ * xe_eudebug_enable
+ * @fd: xe client
+ * @enable: state toggle - true to enable, false to disable
+ *
+ * Enables/disables eudebug capability by writing to
+ * '/sys/class/drm/card<N>/device/enable_eudebug' sysfs entry.
+ *
+ * Returns: previous toggle value, i.e. true when eudebugging was enabled,
+ * false when eudebugging was disabled.
+ */
+bool xe_eudebug_enable(int fd, bool enable)
+{
+	bool old = false;
+	int ret = enable_getset(fd, &old, &enable);
+
+	if (ret) {
+		igt_skip_on(enable);
+		old = false;
+	}
+
+	return old;
+}
+
+/* Eu debugger wrappers around resource creating xe ioctls. */
+
+/**
+ * xe_eudebug_client_open_driver:
+ * @c: pointer to xe_eudebug_client structure
+ *
+ * Calls drm_open_client(DRIVER_XE) and logs the corresponding
+ * event in client's event log.
+ *
+ * Returns: valid DRM file descriptor
+ */
Quoted 8 lines

lib/xe/xe_eudebug.c line 1709

Quoted 4 lines
lib/xe/xe_eudebug.c new file +4 -0
+int xe_eudebug_client_open_driver(struct xe_eudebug_client *c)
+{
+	int fd;
+

From now on for exported functions you should validate variables,
like c must be not null here. And this spreads for all global functions introduced below.

Ok.

Quoted 72 lines
Quoted 72 lines
lib/xe/xe_eudebug.c new file +71 -0
+	fd = drm_reopen_driver(c->master_fd);
+	client_event(c, DRM_XE_EUDEBUG_EVENT_CREATE, fd);
+
+	return fd;
+}
+
+/**
+ * xe_eudebug_client_close_driver:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ *
+ * Calls close driver and logs the corresponding event in
+ * client's event log.
+ */
+void xe_eudebug_client_close_driver(struct xe_eudebug_client *c, int fd)
+{
+	client_event(c, DRM_XE_EUDEBUG_EVENT_DESTROY, fd);
+	close(fd);
+}
+
+/**
+ * xe_eudebug_client_vm_create:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @flags: vm bind flags
+ * @ext: pointer to the first user extension
+ *
+ * Calls xe_vm_create() and logs corresponding events
+ * (including vm set metadata events) in client's event log.
+ *
+ * Returns: valid vm handle
+ */
+uint32_t xe_eudebug_client_vm_create(struct xe_eudebug_client *c, int fd,
+				     uint32_t flags, uint64_t ext)
+{
+	uint32_t vm;
+
+	vm = xe_vm_create(fd, flags, ext);
+	vm_event(c, DRM_XE_EUDEBUG_EVENT_CREATE, fd, vm);
+
+	return vm;
+}
+
+/**
+ * xe_eudebug_client_vm_destroy:
+ * @c: pointer to xe_eudebug_client structure
+ * fd: xe client
+ * vm: vm handle
+ *
+ * Calls xe_vm_destroy() and logs the corresponding event in
+ * client's event log.
+ */
+void xe_eudebug_client_vm_destroy(struct xe_eudebug_client *c, int fd, uint32_t vm)
+{
+	xe_vm_destroy(fd, vm);
+	vm_event(c, DRM_XE_EUDEBUG_EVENT_DESTROY, fd, vm);
+}
+
+/**
+ * xe_eudebug_client_exec_queue_create:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @create: exec_queue create drm struct
+ *
+ * Calls xe exec queue create ioctl and logs the corresponding event in
+ * client's event log.
+ *
+ * Returns: valid exec queue handle
+ */
+uint32_t xe_eudebug_client_exec_queue_create(struct xe_eudebug_client *c, int fd,
+					     struct drm_xe_exec_queue_create *create)
Quoted 8 lines

lib/xe/xe_eudebug.c line 1786

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+{
+	uint16_t class = ((struct drm_xe_engine_class_instance *)(create->instances))[0].engine_class;
+	struct drm_xe_ext_set_property *ext;
+	bool send = false;
+
+	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_EXEC_QUEUE_CREATE, create), 0);

I would add blank line here.

Ok.

Quoted 66 lines
Quoted 66 lines
lib/xe/xe_eudebug.c new file +65 -0
+	for (ext = from_user_pointer(create->extensions); ext;
+	     ext = from_user_pointer(ext->base.next_extension))
+		if (ext->base.name == DRM_XE_EXEC_QUEUE_EXTENSION_SET_PROPERTY &&
+		    ext->property == DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG &&
+		    ext->value == DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE)
+			send = true;
+
+	if (send)
+		exec_queue_event(c, DRM_XE_EUDEBUG_EVENT_CREATE, fd, create->vm_id,
+				 create->exec_queue_id, class, create->width);
+
+	return create->exec_queue_id;
+}
+
+/**
+ * xe_eudebug_client_exec_queue_destroy:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @create: exec_queue create drm struct which was used for creation
+ *
+ * Calls xe exec_queue destroy ioctl and logs the corresponding event in
+ * client's event log.
+ */
+void xe_eudebug_client_exec_queue_destroy(struct xe_eudebug_client *c, int fd,
+					  struct drm_xe_exec_queue_create *create)
+{
+	struct drm_xe_exec_queue_destroy destroy = { .exec_queue_id = create->exec_queue_id, };
+	uint16_t class = ((struct drm_xe_engine_class_instance *)(create->instances))[0].engine_class;
+	struct drm_xe_ext_set_property *ext;
+	bool send = false;
+
+	for (ext = from_user_pointer(create->extensions); ext;
+	     ext = from_user_pointer(ext->base.next_extension))
+		if (ext->base.name == DRM_XE_EXEC_QUEUE_EXTENSION_SET_PROPERTY &&
+		    ext->property == DRM_XE_EXEC_QUEUE_SET_PROPERTY_EUDEBUG &&
+		    ext->value == DRM_XE_EXEC_QUEUE_EUDEBUG_FLAG_ENABLE)
+			send = true;
+
+	if (send)
+		exec_queue_event(c, DRM_XE_EUDEBUG_EVENT_DESTROY, fd, create->vm_id,
+				 create->exec_queue_id, class, create->width);
+
+	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_EXEC_QUEUE_DESTROY, &destroy), 0);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_event:
+ * @c: pointer to xe_eudebug_client structure
+ * @event_flags: base event flags
+ * @fd: xe client
+ * @vm: vm handle
+ * @bind_flags: bind flags of vm_bind_event
+ * @num_binds: number of bind (operations) for event
+ * @ref_seqno: base vm bind reference seqno
+ * Logs vm bind event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_event(struct xe_eudebug_client *c,
+				     uint32_t event_flags, int fd,
+				     uint32_t vm, uint32_t bind_flags,
+				     uint32_t num_binds, u64 *ref_seqno)
+{
+	struct drm_xe_eudebug_event_vm_bind evmb;
+
+	base_event(c, to_base(evmb), DRM_XE_EUDEBUG_EVENT_VM_BIND,
+		   event_flags, sizeof(evmb));
Quoted 8 lines

lib/xe/xe_eudebug.c line 1857

Quoted 6 lines
lib/xe/xe_eudebug.c new file +6 -0
+	evmb.client_handle = fd;
+	evmb.vm_handle = vm;
+	evmb.flags = bind_flags;
+	evmb.num_binds = num_binds;
+
+	*ref_seqno = evmb.base.seqno;

ref_seqno requires to be valid pointer here.

Sure, will ensure that it is.

Thanks,
Christoph

Quoted 598 lines

--
Zbigniew

Quoted 594 lines
lib/xe/xe_eudebug.c new file +341 -0
+
+	xe_eudebug_event_log_write(c->log, (void *)&evmb);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_op_event:
+ * @c: pointer to xe_eudebug_client structure
+ * @event_flags: base event flags
+ * @bind_ref_seqno: base vm bind reference seqno
+ * @op_ref_seqno: output, the vm_bind_op event seqno
+ * @addr: ppgtt address
+ * @size: size of the binding
+ * @num_extensions: number of vm bind op extensions
+ *
+ * Logs vm bind op event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_op_event(struct xe_eudebug_client *c, uint32_t event_flags,
+					uint64_t bind_ref_seqno, uint64_t *op_ref_seqno,
+					uint64_t addr, uint64_t range,
+					uint64_t num_extensions)
+{
+	struct drm_xe_eudebug_event_vm_bind_op op;
+
+	base_event(c, to_base(op), DRM_XE_EUDEBUG_EVENT_VM_BIND_OP,
+		   event_flags, sizeof(op));
+	op.vm_bind_ref_seqno = bind_ref_seqno;
+	op.addr = addr;
+	op.range = range;
+	op.num_extensions = num_extensions;
+
+	*op_ref_seqno = op.base.seqno;
+
+	xe_eudebug_event_log_write(c->log, (void *)&op);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_op_metadata_event:
+ * @c: pointer to xe_eudebug_client structure
+ * @event_flags: base event flags
+ * @op_ref_seqno: base vm bind op reference seqno
+ * @metadata_handle: metadata handle
+ * @metadata_cookie: metadata cookie
+ *
+ * Logs vm bind op metadata event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_op_metadata_event(struct xe_eudebug_client *c,
+						 uint32_t event_flags, uint64_t op_ref_seqno,
+						 uint64_t metadata_handle, uint64_t metadata_cookie)
+{
+	struct drm_xe_eudebug_event_vm_bind_op_metadata op;
+
+	base_event(c, to_base(op), DRM_XE_EUDEBUG_EVENT_VM_BIND_OP_METADATA,
+		   event_flags, sizeof(op));
+	op.vm_bind_op_ref_seqno = op_ref_seqno;
+	op.metadata_handle = metadata_handle;
+	op.metadata_cookie = metadata_cookie;
+
+	xe_eudebug_event_log_write(c->log, (void *)&op);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_ufence_event:
+ * @c: pointer to xe_eudebug_client structure
+ * @event_flags: base event flags
+ * @ref_seqno: base vm bind event seqno
+ *
+ * Logs vm bind ufence event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_ufence_event(struct xe_eudebug_client *c, uint32_t event_flags,
+					    uint64_t ref_seqno)
+{
+	struct drm_xe_eudebug_event_vm_bind_ufence f;
+
+	base_event(c, to_base(f), DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE,
+		   event_flags, sizeof(f));
+	f.vm_bind_ref_seqno = ref_seqno;
+
+	xe_eudebug_event_log_write(c->log, (void *)&f);
+}
+
+static bool has_user_fence(const struct drm_xe_sync *sync, uint32_t num_syncs)
+{
+	while (num_syncs--)
+		if (sync[num_syncs].type == DRM_XE_SYNC_TYPE_USER_FENCE)
+			return true;
+
+	return false;
+}
+
+#define for_each_metadata(__m, __ext)					\
+	for ((__m) = from_user_pointer(__ext);				\
+	     (__m);							\
+	     (__m) = from_user_pointer((__m)->base.next_extension))	\
+		if ((__m)->base.name == XE_VM_BIND_OP_EXTENSIONS_ATTACH_DEBUG)
+
+static int  __xe_eudebug_client_vm_bind(struct xe_eudebug_client *c,
+					int fd, uint32_t vm, uint32_t exec_queue,
+					uint32_t bo, uint64_t offset,
+					uint64_t addr, uint64_t size,
+					uint32_t op, uint32_t flags,
+					struct drm_xe_sync *sync,
+					uint32_t num_syncs,
+					uint32_t prefetch_region,
+					uint8_t pat_index, uint64_t op_ext)
+{
+	struct drm_xe_vm_bind_op_ext_attach_debug *metadata;
+	const bool ufence = has_user_fence(sync, num_syncs);
+	const uint32_t bind_flags = ufence ?
+		DRM_XE_EUDEBUG_EVENT_VM_BIND_FLAG_UFENCE : 0;
+	uint64_t seqno = 0, op_seqno = 0, num_metadata = 0;
+	uint32_t bind_base_flags = 0;
+	int ret;
+
+	for_each_metadata(metadata, op_ext)
+		num_metadata++;
+
+	switch (op) {
+	case DRM_XE_VM_BIND_OP_MAP:
+		bind_base_flags = DRM_XE_EUDEBUG_EVENT_CREATE;
+		break;
+	case DRM_XE_VM_BIND_OP_UNMAP:
+		bind_base_flags = DRM_XE_EUDEBUG_EVENT_DESTROY;
+		igt_assert_eq(num_metadata, 0);
+		igt_assert_eq(ufence, false);
+		break;
+	default:
+		/* XXX unmap all? */
+		igt_assert(op);
+		break;
+	}
+
+	ret = ___xe_vm_bind(fd, vm, exec_queue, bo, offset, addr, size,
+			    op, flags, sync, num_syncs, prefetch_region,
+			    pat_index, 0, op_ext);
+
+	if (ret)
+		return ret;
+
+	if (!bind_base_flags)
+		return -EINVAL;
+
+	xe_eudebug_client_vm_bind_event(c, DRM_XE_EUDEBUG_EVENT_STATE_CHANGE,
+					fd, vm, bind_flags, 1, &seqno);
+	xe_eudebug_client_vm_bind_op_event(c, bind_base_flags,
+					   seqno, &op_seqno, addr, size,
+					   num_metadata);
+
+	for_each_metadata(metadata, op_ext)
+		xe_eudebug_client_vm_bind_op_metadata_event(c,
+							    DRM_XE_EUDEBUG_EVENT_CREATE,
+							    op_seqno,
+							    metadata->metadata_id,
+							    metadata->cookie);
+	if (ufence)
+		xe_eudebug_client_vm_bind_ufence_event(c, DRM_XE_EUDEBUG_EVENT_CREATE |
+						       DRM_XE_EUDEBUG_EVENT_NEED_ACK,
+						       seqno);
+	return ret;
+}
+
+static void _xe_eudebug_client_vm_bind(struct xe_eudebug_client *c, int fd,
+				       uint32_t vm, uint32_t bo,
+				       uint64_t offset, uint64_t addr, uint64_t size,
+				       uint32_t op,
+				       uint32_t flags,
+				       struct drm_xe_sync *sync,
+				       uint32_t num_syncs,
+				       uint64_t op_ext)
+{
+	const uint32_t exec_queue_id = 0;
+	const uint32_t prefetch_region = 0;
+
+	igt_assert_eq(__xe_eudebug_client_vm_bind(c, fd, vm, exec_queue_id, bo, offset,
+						  addr, size, op, flags,
+						  sync, num_syncs, prefetch_region,
+						  DEFAULT_PAT_INDEX, op_ext),
+		      0);
+}
+
+/**
+ * xe_eudebug_client_vm_bind_flags
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @vm: vm handle
+ * @bo: buffer object handle
+ * @offset: offset within buffer object
+ * @addr: ppgtt address
+ * @size: size of the binding
+ * @flags: vm_bind flags
+ * @sync: sync objects
+ * @num_syncs: number of sync objects
+ * @op_ext: BIND_OP extensions
+ *
+ * Calls xe vm_bind ioctl and logs the corresponding event in client's event log.
+ */
+void xe_eudebug_client_vm_bind_flags(struct xe_eudebug_client *c, int fd, uint32_t vm,
+				     uint32_t bo, uint64_t offset,
+				     uint64_t addr, uint64_t size, uint32_t flags,
+				     struct drm_xe_sync *sync, uint32_t num_syncs,
+				     uint64_t op_ext)
+{
+	_xe_eudebug_client_vm_bind(c, fd, vm, bo, offset, addr, size,
+				   DRM_XE_VM_BIND_OP_MAP, flags,
+				   sync, num_syncs, op_ext);
+}
+
+/**
+ * xe_eudebug_client_vm_bind
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @vm: vm handle
+ * @bo: buffer object handle
+ * @offset: offset within buffer object
+ * @addr: ppgtt address
+ * @size: size of the binding
+ *
+ * Calls xe vm_bind ioctl and logs the corresponding event in client's event log.
+ */
+void xe_eudebug_client_vm_bind(struct xe_eudebug_client *c, int fd, uint32_t vm,
+			       uint32_t bo, uint64_t offset,
+			       uint64_t addr, uint64_t size)
+{
+	const uint32_t flags = 0;
+	struct drm_xe_sync *sync = NULL;
+	const uint32_t num_syncs = 0;
+	const uint64_t op_ext = 0;
+
+	xe_eudebug_client_vm_bind_flags(c, fd, vm, bo, offset, addr, size,
+					flags,
+					sync, num_syncs, op_ext);
+}
+
+/**
+ * xe_eudebug_client_vm_unbind_flags
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @vm: vm handle
+ * @offset: offset
+ * @addr: ppgtt address
+ * @size: size of the binding
+ * @flags: vm_bind flags
+ * @sync: sync objects
+ * @num_syncs: number of sync objects
+ *
+ * Calls xe vm_unbind ioctl and logs the corresponding event in client's event log.
+ */
+void xe_eudebug_client_vm_unbind_flags(struct xe_eudebug_client *c, int fd,
+				       uint32_t vm, uint64_t offset,
+				       uint64_t addr, uint64_t size, uint32_t flags,
+				       struct drm_xe_sync *sync, uint32_t num_syncs)
+{
+	_xe_eudebug_client_vm_bind(c, fd, vm, 0, offset, addr, size,
+				   DRM_XE_VM_BIND_OP_UNMAP, flags,
+				   sync, num_syncs, 0);
+}
+
+/**
+ * xe_eudebug_client_vm_unbind
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @vm: vm handle
+ * @offset: offset
+ * @addr: ppgtt address
+ * @size: size of the binding
+ *
+ * Calls xe vm_unbind ioctl and logs the corresponding event in client's event log.
+ */
+void xe_eudebug_client_vm_unbind(struct xe_eudebug_client *c, int fd, uint32_t vm,
+				 uint64_t offset, uint64_t addr, uint64_t size)
+{
+	const uint32_t flags = 0;
+	struct drm_xe_sync *sync = NULL;
+	const uint32_t num_syncs = 0;
+
+	xe_eudebug_client_vm_unbind_flags(c, fd, vm, offset, addr, size,
+					  flags, sync, num_syncs);
+}
+
+/**
+ * xe_eudebug_client_metadata_create:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @type: debug metadata type
+ * @len: size of @data
+ * @data: debug metadata paylad
+ *
+ * Calls xe metadata create ioctl and logs the corresponding event in
+ * client's event log.
+ *
+ * Return: valid debug metadata id.
+ */
+uint32_t xe_eudebug_client_metadata_create(struct xe_eudebug_client *c, int fd,
+					   int type, size_t len, void *data)
+{
+	struct drm_xe_debug_metadata_create create = {
+		.type = type,
+		.user_addr = to_user_pointer(data),
+		.len = len
+	};
+
+	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_DEBUG_METADATA_CREATE, &create), 0);
+
+	metadata_event(c, DRM_XE_EUDEBUG_EVENT_CREATE, fd, create.metadata_id, type, len);
+
+	return create.metadata_id;
+}
+
+/**
+ * xe_eudebug_client_metadata_destroy:
+ * @c: pointer to xe_eudebug_client structure
+ * @fd: xe client
+ * @id: xe debug metadata handle
+ * @type: debug metadata type
+ * @len: size of debug metadata payload
+ *
+ * Calls xe metadata destroy ioctl and logs the corresponding event in
+ * client's event log.
+ */
+void xe_eudebug_client_metadata_destroy(struct xe_eudebug_client *c, int fd,
+					uint32_t id, int type, size_t len)
+{
+	struct drm_xe_debug_metadata_destroy destroy = { .metadata_id = id };
+
+	igt_assert_eq(igt_ioctl(fd, DRM_IOCTL_XE_DEBUG_METADATA_DESTROY, &destroy), 0);
+
+	metadata_event(c, DRM_XE_EUDEBUG_EVENT_DESTROY, fd, id, type, len);
+}
+
+void xe_eudebug_ack_ufence(int debugfd,
+			   const struct drm_xe_eudebug_event_vm_bind_ufence *f)
+{
+	struct drm_xe_eudebug_ack_event ack = { 0, };
+	char event_str[XE_EUDEBUG_EVENT_STRING_MAX_LEN];
+
+	ack.type = f->base.type;
+	ack.seqno = f->base.seqno;
+
+	xe_eudebug_event_to_str((void *)f, event_str, XE_EUDEBUG_EVENT_STRING_MAX_LEN);
+	igt_debug("delivering ack for event: %s\n", event_str);
+	igt_assert_eq(igt_ioctl(debugfd, DRM_XE_EUDEBUG_IOCTL_ACK_EVENT, &ack), 0);
+}
lib/xe/xe_eudebug.h new file +212 -0
new file mode 100644
index 000000000..d6c37d8f2
@@ -0,0 +1,212 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2023 Intel Corporation
+ */
+#include <fcntl.h>
+#include <pthread.h>
+#include <stdint.h>
+#include <xe_drm.h>
+#include <xe_drm_eudebug.h>
+
+#include "igt_list.h"
+
+struct xe_eudebug_event_log {
+	uint8_t *log;
+	unsigned int head;
+	unsigned int max_size;
+	char name[80];
+	pthread_mutex_t lock;
+};
+
+struct xe_eudebug_debugger {
+	int fd;
+	uint64_t flags;
+
+	/* Used to smuggle private data */
+	void *ptr;
+
+	struct xe_eudebug_event_log *log;
+
+	uint64_t event_count;
+
+	uint64_t target_pid;
+
+	struct igt_list_head triggers;
+
+	int master_fd;
+
+	pthread_t worker_thread;
+	int worker_state;
+
+	int p_client[2];
+};
+
+struct xe_eudebug_client {
+	int pid;
+	uint64_t seqno;
+	uint64_t flags;
+
+	/* Used to smuggle private data */
+	void *ptr;
+
+	struct xe_eudebug_event_log *log;
+
+	int done;
+	int p_in[2];
+	int p_out[2];
+
+	/* Used to pickup right device (the one used in debugger) */
+	int master_fd;
+
+	int timeout_ms;
+};
+
+struct xe_eudebug_session {
+	uint64_t flags;
+	struct xe_eudebug_client *c;
+	struct xe_eudebug_debugger *d;
+};
+
+typedef void (*xe_eudebug_client_work_fn)(struct xe_eudebug_client *);
+typedef void (*xe_eudebug_trigger_fn)(struct xe_eudebug_debugger *,
+				      struct drm_xe_eudebug_event *);
+
+#define xe_eudebug_for_each_engine(fd__, hwe__) \
+	xe_for_each_engine(fd__, hwe__) \
+		if (hwe__->engine_class == DRM_XE_ENGINE_CLASS_RENDER || \
+		    hwe__->engine_class == DRM_XE_ENGINE_CLASS_COMPUTE)
+
+#define xe_eudebug_for_each_event(_e, _log) \
+	for ((_e) = (_e) ? (void *)(uint8_t *)(_e) + (_e)->len : \
+		    (void *)(_log)->log; \
+	    (uint8_t *)(_e) < (_log)->log + (_log)->head; \
+	    (_e) = (void *)(uint8_t *)(_e) + (_e)->len)
+
+#define xe_eudebug_assert(d, c)						\
+	do {								\
+		if (!(c)) {						\
+			xe_eudebug_event_log_print((d)->log, true);	\
+			igt_assert(c);					\
+		}							\
+	} while (0)
+
+#define xe_eudebug_assert_f(d, c, f...)					\
+	do {								\
+		if (!(c)) {						\
+			xe_eudebug_event_log_print((d)->log, true);	\
+			igt_assert_f(c, f);				\
+		}							\
+	} while (0)
+
+#define XE_EUDEBUG_EVENT_STRING_MAX_LEN		4096
+
+/*
+ * Default abort timeout to use across xe_eudebug lib and tests if no specific
+ * timeout value is required.
+ */
+#define XE_EUDEBUG_DEFAULT_TIMEOUT_SEC		25ULL
+
+#define XE_EUDEBUG_FILTER_EVENT_NONE		BIT(DRM_XE_EUDEBUG_EVENT_NONE)
+#define XE_EUDEBUG_FILTER_EVENT_READ		BIT(DRM_XE_EUDEBUG_EVENT_READ)
+#define XE_EUDEBUG_FILTER_EVENT_OPEN		BIT(DRM_XE_EUDEBUG_EVENT_OPEN)
+#define XE_EUDEBUG_FILTER_EVENT_VM		BIT(DRM_XE_EUDEBUG_EVENT_VM)
+#define XE_EUDEBUG_FILTER_EVENT_EXEC_QUEUE	BIT(DRM_XE_EUDEBUG_EVENT_EXEC_QUEUE)
+#define XE_EUDEBUG_FILTER_EVENT_EU_ATTENTION	BIT(DRM_XE_EUDEBUG_EVENT_EU_ATTENTION)
+#define XE_EUDEBUG_FILTER_EVENT_VM_BIND		BIT(DRM_XE_EUDEBUG_EVENT_VM_BIND)
+#define XE_EUDEBUG_FILTER_EVENT_VM_BIND_OP	BIT(DRM_XE_EUDEBUG_EVENT_VM_BIND_OP)
+#define XE_EUDEBUG_FILTER_EVENT_VM_BIND_UFENCE  BIT(DRM_XE_EUDEBUG_EVENT_VM_BIND_UFENCE)
+#define XE_EUDEBUG_FILTER_ALL			GENMASK(DRM_XE_EUDEBUG_EVENT_MAX_EVENT, 0)
+#define XE_EUDEBUG_EVENT_IS_FILTERED(_e, _f)	((1UL << (_e)) & (_f))
+
+int xe_eudebug_connect(int fd, pid_t pid, uint32_t flags);
+const char *xe_eudebug_event_to_str(struct drm_xe_eudebug_event *e, char *buf, size_t len);
+struct drm_xe_eudebug_event *
+xe_eudebug_event_log_find_seqno(struct xe_eudebug_event_log *l, uint64_t seqno);
+struct xe_eudebug_event_log *
+xe_eudebug_event_log_create(const char *name, unsigned int max_size);
+void xe_eudebug_event_log_destroy(struct xe_eudebug_event_log *l);
+void xe_eudebug_event_log_print(struct xe_eudebug_event_log *l, bool debug);
+void xe_eudebug_event_log_compare(struct xe_eudebug_event_log *c, struct xe_eudebug_event_log *d,
+				  uint32_t filter);
+void xe_eudebug_event_log_write(struct xe_eudebug_event_log *l, struct drm_xe_eudebug_event *e);
+void xe_eudebug_event_log_match_opposite(struct xe_eudebug_event_log *l, uint32_t filter);
+
+bool xe_eudebug_debugger_available(int fd);
+struct xe_eudebug_debugger *
+xe_eudebug_debugger_create(int xe, uint64_t flags, void *data);
+void xe_eudebug_debugger_destroy(struct xe_eudebug_debugger *d);
+int xe_eudebug_debugger_attach(struct xe_eudebug_debugger *d, struct xe_eudebug_client *c);
+void xe_eudebug_debugger_start_worker(struct xe_eudebug_debugger *d);
+void xe_eudebug_debugger_stop_worker(struct xe_eudebug_debugger *d, int timeout_s);
+void xe_eudebug_debugger_dettach(struct xe_eudebug_debugger *d);
+void xe_eudebug_debugger_set_data(struct xe_eudebug_debugger *c, void *ptr);
+void xe_eudebug_debugger_add_trigger(struct xe_eudebug_debugger *d, int type,
+				     xe_eudebug_trigger_fn fn);
+void xe_eudebug_debugger_signal_stage(struct xe_eudebug_debugger *d, uint64_t stage);
+void xe_eudebug_debugger_wait_stage(struct xe_eudebug_session *s, uint64_t stage);
+
+struct xe_eudebug_client *
+xe_eudebug_client_create(int xe, xe_eudebug_client_work_fn work, uint64_t flags, void *data);
+void xe_eudebug_client_destroy(struct xe_eudebug_client *c);
+void xe_eudebug_client_start(struct xe_eudebug_client *c);
+void xe_eudebug_client_stop(struct xe_eudebug_client *c);
+void xe_eudebug_client_wait_done(struct xe_eudebug_client *c);
+void xe_eudebug_client_signal_stage(struct xe_eudebug_client *c, uint64_t stage);
+void xe_eudebug_client_wait_stage(struct xe_eudebug_client *c, uint64_t stage);
+
+uint64_t xe_eudebug_client_get_seqno(struct xe_eudebug_client *c);
+void xe_eudebug_client_set_data(struct xe_eudebug_client *c, void *ptr);
+
+bool xe_eudebug_enable(int fd, bool enable);
+
+int xe_eudebug_client_open_driver(struct xe_eudebug_client *c);
+void xe_eudebug_client_close_driver(struct xe_eudebug_client *c, int fd);
+uint32_t xe_eudebug_client_vm_create(struct xe_eudebug_client *c, int fd,
+				     uint32_t flags, uint64_t ext);
+void xe_eudebug_client_vm_destroy(struct xe_eudebug_client *c, int fd, uint32_t vm);
+uint32_t xe_eudebug_client_exec_queue_create(struct xe_eudebug_client *c, int fd,
+					     struct drm_xe_exec_queue_create *create);
+void xe_eudebug_client_exec_queue_destroy(struct xe_eudebug_client *c, int fd,
+					  struct drm_xe_exec_queue_create *create);
+void xe_eudebug_client_vm_bind_event(struct xe_eudebug_client *c, uint32_t event_flags, int fd,
+				     uint32_t vm, uint32_t bind_flags,
+				     uint32_t num_ops, uint64_t *ref_seqno);
+void xe_eudebug_client_vm_bind_op_event(struct xe_eudebug_client *c, uint32_t event_flags,
+					uint64_t ref_seqno, uint64_t *op_ref_seqno,
+					uint64_t addr, uint64_t range,
+					uint64_t num_extensions);
+void xe_eudebug_client_vm_bind_op_metadata_event(struct xe_eudebug_client *c,
+						 uint32_t event_flags, uint64_t op_ref_seqno,
+						 uint64_t metadata_handle, uint64_t metadata_cookie);
+void xe_eudebug_client_vm_bind_ufence_event(struct xe_eudebug_client *c, uint32_t event_flags,
+					    uint64_t ref_seqno);
+void xe_eudebug_ack_ufence(int debugfd,
+			   const struct drm_xe_eudebug_event_vm_bind_ufence *f);
+
+void xe_eudebug_client_vm_bind_flags(struct xe_eudebug_client *c, int fd, uint32_t vm,
+				     uint32_t bo, uint64_t offset,
+				     uint64_t addr, uint64_t size, uint32_t flags,
+				     struct drm_xe_sync *sync, uint32_t num_syncs,
+				     uint64_t op_ext);
+void xe_eudebug_client_vm_bind(struct xe_eudebug_client *c, int fd, uint32_t vm,
+			       uint32_t bo, uint64_t offset,
+			       uint64_t addr, uint64_t size);
+void xe_eudebug_client_vm_unbind_flags(struct xe_eudebug_client *c, int fd,
+				       uint32_t vm, uint64_t offset,
+				       uint64_t addr, uint64_t size, uint32_t flags,
+				       struct drm_xe_sync *sync, uint32_t num_syncs);
+void xe_eudebug_client_vm_unbind(struct xe_eudebug_client *c, int fd, uint32_t vm,
+				 uint64_t offset, uint64_t addr, uint64_t size);
+
+uint32_t xe_eudebug_client_metadata_create(struct xe_eudebug_client *c, int fd,
+					   int type, size_t len, void *data);
+void xe_eudebug_client_metadata_destroy(struct xe_eudebug_client *c, int fd,
+					uint32_t id, int type, size_t len);
+
+struct xe_eudebug_session *xe_eudebug_session_create(int fd,
+						     xe_eudebug_client_work_fn work,
+						     unsigned int flags,
+						     void *test_private);
+void xe_eudebug_session_destroy(struct xe_eudebug_session *s);
+void xe_eudebug_session_run(struct xe_eudebug_session *s);
+void xe_eudebug_session_check(struct xe_eudebug_session *s, bool match_opposite, uint32_t filter);
meson.build +2 -0
index 25bf1e442..207b5819b 100644
@@ -90,9 +90,11 @@ build_chamelium = get_option('chamelium')
  build_docs = get_option('docs')
  build_tests = not get_option('tests').disabled()
  build_xe = not get_option('xe_driver').disabled()
+build_xe_eudebug = get_option('xe_eudebug').enabled()
  with_libdrm = get_option('libdrm_drivers')
 
  build_info = ['Build type: ' + get_option('buildtype')]
+build_info += 'Build Xe EU debugger test framework: @0@'.format(build_xe_eudebug)
 
  inc = include_directories('include', 'include/drm-uapi', 'include/linux-uapi', 'lib', 'lib/stubs/syscalls', '.')
 
meson_options.txt +5 -0
index 6a9493ea6..11922523b 100644
@@ -42,6 +42,11 @@ option('xe_driver',
         value : 'enabled',
         description : 'Build tests for Xe driver')
 
+option('xe_eudebug',
+       type : 'feature',
+       value : 'disabled',
+       description : 'Build library for Xe EU debugger')
+
  option('libdrm_drivers',
         type : 'array',
         value : ['auto'],
Signature · 1 line
2.34.1