Skip to contentSkip to conversation

Add per-client engine utilization tests

PATCH In review Umesh Nerlige Ramappa 5 participants 36 messages Jul 3, 2024 – Aug 15, 2024

lore
Share & tools
lore
#0 lore raw

v2 cover Add per-client engine utilization tests

Port per client engine utilization tests from i915 to xe.

v2:
- Add a test to destroy exec queue before capturing sample
- Include review comments from Lucas/Riana

Opens from v1 review
---
1) Documenting mapping for TEST_BUSY/TEST_TRAILING_IDLE etc to tests
- Not seeing a clean way to do this since some tests iterate over all engine and some over virtual engines and then some tests omit hwe/width/count params. Would leave it as is if acceptable.

2) Busyness > 100%
- sampling RING_TIMESTAMP before utilization values is likely leading to this error. Ideally RING TIMESTAMP should be sampled before 1st sample and after 2nd sample, but that's not feasible. It can only be one way for each sample - either before or after.

Umesh Nerlige Ramappa (10):
tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test tests/intel/xe_drm_fdinfo: Add basic-engine-utilization test tests/intel/xe_drm_fdinfo: Add helper to read utilization for all
classes
tests/intel/xe_drm_fdinfo: Add helpers for spinning batches tests/intel/xe_drm_fdinfo: Add single engine tests tests/intel/xe_drm_fdinfo: Add tests to verify all class utilization tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines tests/intel/xe_drm_fdinfo: Add tests for virtual engines tests/intel/xe_drm_fdinfo: Add tests for parallel engines tests/intel/xe_drm_fdinfo: Ensure queue destroy records load correctly

 tests/intel/xe_drm_fdinfo.c | 587 +++++++++++++++++++++++++++++++++++-
 1 file changed, 577 insertions(+), 10 deletions(-)
Signature · 1 line
2.38.1
#1 lore raw

PATCH v2 1/10 tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

Existing basic test is checking memory info in fdinfo. Rename accordingly.

---
 tests/intel/xe_drm_fdinfo.c | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
tests/intel/xe_drm_fdinfo.c +6 -6
index 2ceafba24..f0aa23e2f 100644
@@ -22,8 +22,8 @@
  * Feature: SMI, core
  * Test category: SysMan
  *
- * SUBTEST: basic
- * Description: Check if basic fdinfo content is present
+ * SUBTEST: basic-memory
+ * Description: Check if basic fdinfo content is present for memory
  *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
 	xe_vm_destroy(xe, vm);
 }
 
-static void basic(int xe)
+static void basic_memory(int xe)
 {
 	struct drm_xe_mem_region *memregion;
 	uint64_t memreg = all_memory_regions(xe), region;
@@ -299,9 +299,9 @@ igt_main
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
 	}
 
-	igt_describe("Check if basic fdinfo content is present");
-	igt_subtest("basic")
-		basic(xe);
+	igt_describe("Check if basic fdinfo content is present for memory");
+	igt_subtest("basic-memory")
+		basic_memory(xe);
 
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
Signature · 1 line
2.38.1
Reviewed-by on v2 1/10
#2 lore raw

Re: [PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

tests/intel/xe_drm_fdinfo.c

Quoted 12 lines · Umesh Nerlige Ramappa

Existing basic test is checking memory info in fdinfo. Rename accordingly.

---
tests/intel/xe_drm_fdinfo.c | 12 ++++++------

1 file changed, 6 insertions(+), 6 deletions(-)
tests/intel/xe_drm_fdinfo.c +0 -0
index 2ceafba24..f0aa23e2f 100644

I was also expecting a change to the testlist... but it seems we don't really have this there.

https://intel-gfx-ci.01.org/tree/intel-xe/index.html?testfilter=xe_drm_fdinfo
returns an empty result so it seems we are not even running these tests?

Cc'ing some people. For the changes here:

thanks
Lucas De Marchi

Quoted 36 lines
tests/intel/xe_drm_fdinfo.c +6 -6
@@ -22,8 +22,8 @@
 * Feature: SMI, core
 * Test category: SysMan
 *
- * SUBTEST: basic
- * Description: Check if basic fdinfo content is present
+ * SUBTEST: basic-memory
+ * Description: Check if basic fdinfo content is present for memory
 *
 * SUBTEST: drm-total-resident
 * Description: Create and compare total and resident memory consumption by client
@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
 	xe_vm_destroy(xe, vm);
 }
 
-static void basic(int xe)
+static void basic_memory(int xe)
 {
 	struct drm_xe_mem_region *memregion;
 	uint64_t memreg = all_memory_regions(xe), region;
@@ -299,9 +299,9 @@ igt_main
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
 	}
 
-	igt_describe("Check if basic fdinfo content is present");
-	igt_subtest("basic")
-		basic(xe);
+	igt_describe("Check if basic fdinfo content is present for memory");
+	igt_subtest("basic-memory")
+		basic_memory(xe);
 
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
Signature · 1 line
2.38.1
#3 lore raw

Re: [PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

Quoted 17 lines · Lucas De Marchi

tests/intel/xe_drm_fdinfo.c

Quoted 12 lines · Umesh Nerlige Ramappa

Existing basic test is checking memory info in fdinfo. Rename accordingly.

---
tests/intel/xe_drm_fdinfo.c | 12 ++++++------

1 file changed, 6 insertions(+), 6 deletions(-)
tests/intel/xe_drm_fdinfo.c +0 -0
index 2ceafba24..f0aa23e2f 100644

I was also expecting a change to the testlist... but it seems we don't really have this there.

Quoted 2 lines

https://intel-gfx-ci.01.org/tree/intel-xe/index.html?testfilter=xe_drm_fdinfo
returns an empty result so it seems we are not even running these tests?

Strange, for rev2 I actually saw these new tests run in the shards list and was expecting the same for newer revs.

https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11334/shards-all.html?testfilter=xe_drm_fdinfo

Umesh

Quoted 44 lines

Cc'ing some people. For the changes here:

thanks
Lucas De Marchi

Quoted 36 lines
tests/intel/xe_drm_fdinfo.c +6 -6
@@ -22,8 +22,8 @@
 * Feature: SMI, core
 * Test category: SysMan
 *
- * SUBTEST: basic
- * Description: Check if basic fdinfo content is present
+ * SUBTEST: basic-memory
+ * Description: Check if basic fdinfo content is present for memory
 *
 * SUBTEST: drm-total-resident
 * Description: Create and compare total and resident memory consumption by client
@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
 	xe_vm_destroy(xe, vm);
 }
 
-static void basic(int xe)
+static void basic_memory(int xe)
 {
 	struct drm_xe_mem_region *memregion;
 	uint64_t memreg = all_memory_regions(xe), region;
@@ -299,9 +299,9 @@ igt_main
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
 	}
 
-	igt_describe("Check if basic fdinfo content is present");
-	igt_subtest("basic")
-		basic(xe);
+	igt_describe("Check if basic fdinfo content is present for memory");
+	igt_subtest("basic-memory")
+		basic_memory(xe);
 
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
Signature · 1 line
2.38.1
#4 lore raw

Re: [PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

Quoted 18 lines · Umesh Nerlige Ramappa
Quoted 17 lines · Lucas De Marchi

tests/intel/xe_drm_fdinfo.c

Quoted 12 lines · Umesh Nerlige Ramappa

Existing basic test is checking memory info in fdinfo. Rename accordingly.

---
tests/intel/xe_drm_fdinfo.c | 12 ++++++------

1 file changed, 6 insertions(+), 6 deletions(-)
tests/intel/xe_drm_fdinfo.c +0 -0
index 2ceafba24..f0aa23e2f 100644

I was also expecting a change to the testlist... but it seems we don't really have this there.

Quoted 6 lines
Quoted 2 lines

https://intel-gfx-ci.01.org/tree/intel-xe/index.html?testfilter=xe_drm_fdinfo
returns an empty result so it seems we are not even running these tests?

Strange, for rev2 I actually saw these new tests run in the shards list and was expecting the same for newer revs.

https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11334/shards-all.html?testfilter=xe_drm_fdinfo

yeah... if I use the shards-all.html I can see it.. and that one doesn't need a testlist since it's actually "all the tests".

Lucas De Marchi

Quoted 47 lines

Umesh

Quoted 44 lines

Cc'ing some people. For the changes here:

thanks
Lucas De Marchi

Quoted 36 lines
tests/intel/xe_drm_fdinfo.c +6 -6
@@ -22,8 +22,8 @@
 * Feature: SMI, core
 * Test category: SysMan
 *
- * SUBTEST: basic
- * Description: Check if basic fdinfo content is present
+ * SUBTEST: basic-memory
+ * Description: Check if basic fdinfo content is present for memory
 *
 * SUBTEST: drm-total-resident
 * Description: Create and compare total and resident memory consumption by client
@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
 	xe_vm_destroy(xe, vm);
 }
 
-static void basic(int xe)
+static void basic_memory(int xe)
 {
 	struct drm_xe_mem_region *memregion;
 	uint64_t memreg = all_memory_regions(xe), region;
@@ -299,9 +299,9 @@ igt_main
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
 	}
 
-	igt_describe("Check if basic fdinfo content is present");
-	igt_subtest("basic")
-		basic(xe);
+	igt_describe("Check if basic fdinfo content is present for memory");
+	igt_subtest("basic-memory")
+		basic_memory(xe);
 
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
Signature · 1 line
2.38.1
#5 lore raw

Re: [PATCH i-g-t v2 01/10] tests/intel/xe_drm_fdinfo: Rename basic to basic-memory test

Hi Lucas,

Quoted 22 lines · Lucas De Marchi
Quoted 21 lines · Umesh Nerlige Ramappa
Quoted 19 lines · Lucas De Marchi

tests/intel/xe_drm_fdinfo.c

Quoted 12 lines · Umesh Nerlige Ramappa

Existing basic test is checking memory info in fdinfo. Rename accordingly.

---
tests/intel/xe_drm_fdinfo.c | 12 ++++++------

1 file changed, 6 insertions(+), 6 deletions(-)
tests/intel/xe_drm_fdinfo.c +0 -0
index 2ceafba24..f0aa23e2f 100644

I was also expecting a change to the testlist... but it seems we don't really have this there.

https://intel-gfx-ci.01.org/tree/intel-xe/index.html?testfilter=xe_drm_fdinfo
returns an empty result so it seems we are not even running these tests?

Quoted 7 lines
Quoted 4 lines

Strange, for rev2 I actually saw these new tests run in the shards list and was expecting the same for newer revs.

https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11334/shards-all.html?testfilter=xe_drm_fdinfo

yeah... if I use the shards-all.html I can see it.. and that one doesn't need a testlist since it's actually "all the tests".

We need change only for hard-coded xe-fast-feedback or if it was in blocklist. Btw is this correct:

xe.blocklist.txt:igt@drm_fdinfo

imho this should also be running in our tests? Or am I missing something?

Regards,
Kamil

Quoted 50 lines

Lucas De Marchi

Quoted 47 lines

Umesh

Quoted 44 lines

Cc'ing some people. For the changes here:

thanks
Lucas De Marchi

Quoted 36 lines
tests/intel/xe_drm_fdinfo.c +6 -6
@@ -22,8 +22,8 @@
 * Feature: SMI, core
 * Test category: SysMan
 *
- * SUBTEST: basic
- * Description: Check if basic fdinfo content is present
+ * SUBTEST: basic-memory
+ * Description: Check if basic fdinfo content is present for memory
 *
 * SUBTEST: drm-total-resident
 * Description: Create and compare total and resident memory consumption by client
@@ -259,7 +259,7 @@ static void test_total_resident(int xe)
 	xe_vm_destroy(xe, vm);
 }
 
-static void basic(int xe)
+static void basic_memory(int xe)
 {
 	struct drm_xe_mem_region *memregion;
 	uint64_t memreg = all_memory_regions(xe), region;
@@ -299,9 +299,9 @@ igt_main
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
 	}
 
-	igt_describe("Check if basic fdinfo content is present");
-	igt_subtest("basic")
-		basic(xe);
+	igt_describe("Check if basic fdinfo content is present for memory");
+	igt_subtest("basic-memory")
+		basic_memory(xe);
 
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
Signature · 1 line
2.38.1
#6 lore raw

PATCH v2 2/10 tests/intel/xe_drm_fdinfo: Add basic-engine-utilization test

In reply to Umesh Nerlige Ramappa

Add the basic test for engine utilization.

---
 tests/intel/xe_drm_fdinfo.c | 32 +++++++++++++++++++++++++++++---
 1 file changed, 29 insertions(+), 3 deletions(-)
tests/intel/xe_drm_fdinfo.c +29 -3
index f0aa23e2f..b64bef339 100644
@@ -14,17 +14,20 @@
 #include "xe/xe_spin.h"
 /**
  * TEST: xe drm fdinfo
- * Description: Read and verify drm client memory consumption using fdinfo
+ * Description: Read and verify drm client memory consumption and engine utilization using fdinfo
  * Category: Core
  * Mega feature: General Core features
  * Sub-category: driver
- * Functionality: Per client memory statistics
+ * Functionality: Per client memory and engine utilization statistics
  * Feature: SMI, core
  * Test category: SysMan
  *
  * SUBTEST: basic-memory
  * Description: Check if basic fdinfo content is present for memory
  *
+ * SUBTEST: basic-engine-utilization
+ * Description: Check if basic fdinfo content is present for engine utilization
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -35,10 +38,17 @@
  * Description: Create and compare active memory consumption by client
  */
 
-IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption using fdinfo");
+IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine utilization using fdinfo");
 
 #define BO_SIZE (65536)
 
+static const char *engine_map[] = {
+	"rcs",
+	"bcs",
+	"vcs",
+	"vecs",
+	"ccs",
+};
 /* Subtests */
 static void test_active(int fd, struct drm_xe_engine *engine)
 {
@@ -287,6 +297,18 @@ static void basic_memory(int xe)
 	}
 }
 
+static void basic_engine_utilization(int xe)
+{
+	struct drm_client_fdinfo info = { };
+	unsigned int ret;
+
+	ret = igt_parse_drm_fdinfo(xe, &info, engine_map,
+				   ARRAY_SIZE(engine_map), NULL, 0);
+	igt_assert_f(ret != 0, "failed with err:%d\n", errno);
+	igt_assert(!strcmp(info.driver, "xe"));
+	igt_require(info.num_engines);
+}
+
 igt_main
 {
 	int xe;
@@ -303,6 +325,10 @@ igt_main
 	igt_subtest("basic-memory")
 		basic_memory(xe);
 
+	igt_describe("Check if basic fdinfo content is present for engine utilization");
+	igt_subtest("basic-engine-utilization")
+		basic_engine_utilization(xe);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#8 lore raw

Re: [PATCH i-g-t v2 02/10] tests/intel/xe_drm_fdinfo: Add basic-engine-utilization test

Quoted 7 lines · Lucas De Marchi
Quoted 3 lines · Umesh Nerlige Ramappa

Add the basic test for engine utilization.

applied these first 2 patches while I think (after vacations) about the rest. Also sent some minor fixes while studying this code:
https://patchwork.freedesktop.org/series/136713/

Lucas De Marchi

Quoted 2 lines

Lucas De Marchi

#9 lore raw

PATCH v2 3/10 tests/intel/xe_drm_fdinfo: Add helper to read utilization for all classes

In reply to Umesh Nerlige Ramappa

Add a helper that captures utilization for all classes.

---
 tests/intel/xe_drm_fdinfo.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
tests/intel/xe_drm_fdinfo.c +20 -0
index b64bef339..c697f3e53 100644
@@ -42,6 +42,11 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+struct pceu_cycles {
+	uint64_t cycles;
+	uint64_t total_cycles;
+};
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -49,6 +54,21 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };
+static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
+{
+	struct drm_client_fdinfo info = { };
+	int class;
+
+	igt_assert(pceu);
+	igt_assert(igt_parse_drm_fdinfo(xe, &info, engine_map,
+					ARRAY_SIZE(engine_map), NULL, 0));
+
+	xe_for_each_engine_class(class) {
+		pceu[class].cycles = info.cycles[class];
+		pceu[class].total_cycles = info.total_cycles[class];
+	}
+}
+
 /* Subtests */
 static void test_active(int fd, struct drm_xe_engine *engine)
 {
Signature · 1 line
2.38.1
#10 lore raw

Re: [PATCH i-g-t v2 03/10] tests/intel/xe_drm_fdinfo: Add helper to read utilization for all classes

Quoted 24 lines · Umesh Nerlige Ramappa

Add a helper that captures utilization for all classes.

---
tests/intel/xe_drm_fdinfo.c | 20 ++++++++++++++++++++

1 file changed, 20 insertions(+)
tests/intel/xe_drm_fdinfo.c +5 -0
index b64bef339..c697f3e53 100644
@@ -42,6 +42,11 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+struct pceu_cycles {
+	uint64_t cycles;
+	uint64_t total_cycles;
+};
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 56

Quoted 3 lines
tests/intel/xe_drm_fdinfo.c +0 -0
@@ -49,6 +54,21 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };

nit: missing newline here, otherwise:

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 57

Quoted 1 line
tests/intel/xe_drm_fdinfo.c +1 -0
+static void read_engine_cycles(int xe, struct pceu_cycles *pceu)

I think that would be better to give the compiler a hint about the size of the array:

static void read_engine_cycles(int xe,
struct pceu_cycles pceu[static DRM_XE_ENGINE_CLASS_COMPUTE + 1])

(it would not be a bad idea to have a local __IGT_DRM_XE_ENGINE_CLASS_LAST too, but outside of the scope here)

Otherwise,

// Reviewed-by: Lucas De Marchi <[email protected]>

Lucas De Marchi

Quoted 20 lines
tests/intel/xe_drm_fdinfo.c +14 -0
+{
+	struct drm_client_fdinfo info = { };
+	int class;
+
+	igt_assert(pceu);
+	igt_assert(igt_parse_drm_fdinfo(xe, &info, engine_map,
+					ARRAY_SIZE(engine_map), NULL, 0));
+
+	xe_for_each_engine_class(class) {
+		pceu[class].cycles = info.cycles[class];
+		pceu[class].total_cycles = info.total_cycles[class];
+	}
+}
+
 /* Subtests */
 static void test_active(int fd, struct drm_xe_engine *engine)
 {
Signature · 1 line
2.38.1
#11 lore raw

Re: [PATCH i-g-t v2 03/10] tests/intel/xe_drm_fdinfo: Add helper to read utilization for all classes

Quoted 16 lines · Lucas De Marchi
Quoted 15 lines · Umesh Nerlige Ramappa

Add a helper that captures utilization for all classes.

---
tests/intel/xe_drm_fdinfo.c | 20 ++++++++++++++++++++

1 file changed, 20 insertions(+)
tests/intel/xe_drm_fdinfo.c +0 -0
index b64bef339..c697f3e53 100644
@@ -42,6 +42,11 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
Quoted 14 lines

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 56

Quoted 12 lines
tests/intel/xe_drm_fdinfo.c +5 -0
+struct pceu_cycles {
+	uint64_t cycles;
+	uint64_t total_cycles;
+};
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -49,6 +54,21 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };

nit: missing newline here, otherwise:

will add

Quoted 8 lines

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 57

Quoted 2 lines
tests/intel/xe_drm_fdinfo.c +1 -0
+static void read_engine_cycles(int xe, struct pceu_cycles *pceu)

I think that would be better to give the compiler a hint about the size of the array:

static void read_engine_cycles(int xe,
struct pceu_cycles pceu[static DRM_XE_ENGINE_CLASS_COMPUTE + 1])

Never knew static could be placed there. Will add.

Quoted 7 lines

(it would not be a bad idea to have a local __IGT_DRM_XE_ENGINE_CLASS_LAST too, but outside of the scope here)

Otherwise,

// Reviewed-by: Lucas De Marchi <[email protected]>

Thanks,
Umesh

Quoted 23 lines

Lucas De Marchi

Quoted 20 lines
tests/intel/xe_drm_fdinfo.c +14 -0
+{
+	struct drm_client_fdinfo info = { };
+	int class;
+
+	igt_assert(pceu);
+	igt_assert(igt_parse_drm_fdinfo(xe, &info, engine_map,
+					ARRAY_SIZE(engine_map), NULL, 0));
+
+	xe_for_each_engine_class(class) {
+		pceu[class].cycles = info.cycles[class];
+		pceu[class].total_cycles = info.total_cycles[class];
+	}
+}
+
 /* Subtests */
 static void test_active(int fd, struct drm_xe_engine *engine)
 {
Signature · 1 line
2.38.1
#12 lore raw

Re: [PATCH i-g-t v2 03/10] tests/intel/xe_drm_fdinfo: Add helper to read utilization for all classes

Quoted 37 lines · Umesh Nerlige Ramappa
Quoted 16 lines · Lucas De Marchi
Quoted 15 lines · Umesh Nerlige Ramappa

Add a helper that captures utilization for all classes.

---
tests/intel/xe_drm_fdinfo.c | 20 ++++++++++++++++++++

1 file changed, 20 insertions(+)
tests/intel/xe_drm_fdinfo.c +0 -0
index b64bef339..c697f3e53 100644
@@ -42,6 +42,11 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
Quoted 14 lines

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 56

Quoted 12 lines
tests/intel/xe_drm_fdinfo.c +5 -0
+struct pceu_cycles {
+	uint64_t cycles;
+	uint64_t total_cycles;
+};
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -49,6 +54,21 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };

nit: missing newline here, otherwise:

will add

Quoted 3 lines
Quoted 2 lines
tests/intel/xe_drm_fdinfo.c +1 -0
+static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
Quoted 15 lines
Quoted 5 lines

I think that would be better to give the compiler a hint about the size of the array:

static void read_engine_cycles(int xe,
struct pceu_cycles pceu[static DRM_XE_ENGINE_CLASS_COMPUTE + 1])

Never knew static could be placed there. Will add.

Quoted 7 lines

(it would not be a bad idea to have a local __IGT_DRM_XE_ENGINE_CLASS_LAST too, but outside of the scope here)

Otherwise,

// Reviewed-by: Lucas De Marchi <[email protected]>

I was checking if we could apply this and rework on top. But both this and the next commit have the same issue of adding a static helper without a real caller (so we get warnings). I will keep looking if I can split this differently to get things going.

Lucas De Marchi

Quoted 26 lines

Thanks,
Umesh

Quoted 23 lines

Lucas De Marchi

Quoted 20 lines
tests/intel/xe_drm_fdinfo.c +14 -0
+{
+	struct drm_client_fdinfo info = { };
+	int class;
+
+	igt_assert(pceu);
+	igt_assert(igt_parse_drm_fdinfo(xe, &info, engine_map,
+					ARRAY_SIZE(engine_map), NULL, 0));
+
+	xe_for_each_engine_class(class) {
+		pceu[class].cycles = info.cycles[class];
+		pceu[class].total_cycles = info.total_cycles[class];
+	}
+}
+
 /* Subtests */
 static void test_active(int fd, struct drm_xe_engine *engine)
 {
Signature · 1 line
2.38.1
#13 lore raw

PATCH v2 4/10 tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

In reply to Umesh Nerlige Ramappa

Add helpers for submitting batches and waiting for them to start.

v2: Remove xe prefixes from the structures and helpers (Lucas)

---
 tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++
 1 file changed, 135 insertions(+)
tests/intel/xe_drm_fdinfo.c +135 -0
index c697f3e53..037f25e53 100644
@@ -54,6 +54,17 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };
+
+static const uint64_t batch_addr[] = {
+	0x170000,
+	0x180000,
+	0x190000,
+	0x1a0000,
+	0x1b0000,
+	0x1c0000,
+	0x1d0000,
+	0x1e0000,
+};
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
 	igt_require(info.num_engines);
 }
 
+#define MAX_PARALLEL 8
+struct spin_ctx {
+	uint32_t vm;
+	uint64_t addr[MAX_PARALLEL];
+	struct drm_xe_sync sync[2];
+	struct drm_xe_exec exec;
+	uint32_t exec_queue;
+	size_t bo_size;
+	uint32_t bo;
+	struct xe_spin *spin;
+	struct xe_spin_opts spin_opts;
+	bool ended;
+	uint16_t class;
+	uint16_t width;
+	uint16_t num_placements;
+};
+
+static struct spin_ctx *
+spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
+	      uint16_t width, uint16_t num_placements)
+{
+	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
+
+	igt_assert(width && num_placements &&
+		   (width == 1 || num_placements == 1));
+
+	igt_assert(width <= MAX_PARALLEL);
+
+	ctx->class = hwe->engine_class;
+	ctx->width = width;
+	ctx->num_placements = num_placements;
+	ctx->vm = vm;
+	for (int i = 0; i < ctx->width; i++)
+		ctx->addr[i] = batch_addr[hwe->engine_class];
+
+	ctx->exec.num_batch_buffer = width;
+	ctx->exec.num_syncs = 2;
+	ctx->exec.syncs = to_user_pointer(ctx->sync);
+
+	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[0].handle = syncobj_create(fd, 0);
+
+	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[1].handle = syncobj_create(fd, 0);
+
+	ctx->bo_size = sizeof(struct xe_spin);
+	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
+	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
+			       vram_if_possible(fd, hwe->gt_id),
+			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
+	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
+
+	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
+					     hwe, 0, &ctx->exec_queue), 0);
+
+	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
+			 ctx->sync, 1);
+
+	return ctx;
+}
+
+static void
+spin_sync_start(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	ctx->spin_opts.addr = ctx->addr[0];
+	ctx->spin_opts.preempt = true;
+	xe_spin_init(ctx->spin, &ctx->spin_opts);
+
+	/* re-use sync[0] for exec */
+	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
+
+	ctx->exec.exec_queue_id = ctx->exec_queue;
+	if (ctx->width > 1)
+		ctx->exec.address = to_user_pointer(ctx->addr);
+	else
+		ctx->exec.address = ctx->addr[0];
+	xe_exec(fd, &ctx->exec);
+
+	xe_spin_wait_started(ctx->spin);
+	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
+
+	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
+}
+
+static void
+spin_sync_end(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx || ctx->ended)
+		return;
+
+	xe_spin_end(ctx->spin);
+
+	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
+	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->ended = true;
+	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
+}
+
+static void
+spin_ctx_destroy(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	syncobj_destroy(fd, ctx->sync[0].handle);
+	syncobj_destroy(fd, ctx->sync[1].handle);
+	xe_exec_queue_destroy(fd, ctx->exec_queue);
+
+	munmap(ctx->spin, ctx->bo_size);
+	gem_close(fd, ctx->bo);
+
+	free(ctx);
+}
+
 igt_main
 {
 	int xe;
Signature · 1 line
2.38.1
#14 lore raw

Re: [PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

Quoted 18 lines · Umesh Nerlige Ramappa

Add helpers for submitting batches and waiting for them to start.

v2: Remove xe prefixes from the structures and helpers (Lucas)

---
tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++

1 file changed, 135 insertions(+)
tests/intel/xe_drm_fdinfo.c +1 -0
index c697f3e53..037f25e53 100644
@@ -54,6 +54,17 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };
+

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 67

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+static const uint64_t batch_addr[] = {
+	0x170000,
+	0x180000,
+	0x190000,
+	0x1a0000,
+	0x1b0000,
+	0x1c0000,
+	0x1d0000,
+	0x1e0000,
+};

where are these numbers coming from? I'm looking at tests/intel/xe_spin_batch.c and the lib/xe/xe_spin.c implementation and I don't understand why we need to add so many helpers and handcrafted things in this particular test.

+Matt Brost, +Zbigniew Kempczyński to help help here

Lucas De Marchi

Quoted 137 lines
tests/intel/xe_drm_fdinfo.c +124 -0
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
 	igt_require(info.num_engines);
 }
 
+#define MAX_PARALLEL 8
+struct spin_ctx {
+	uint32_t vm;
+	uint64_t addr[MAX_PARALLEL];
+	struct drm_xe_sync sync[2];
+	struct drm_xe_exec exec;
+	uint32_t exec_queue;
+	size_t bo_size;
+	uint32_t bo;
+	struct xe_spin *spin;
+	struct xe_spin_opts spin_opts;
+	bool ended;
+	uint16_t class;
+	uint16_t width;
+	uint16_t num_placements;
+};
+
+static struct spin_ctx *
+spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
+	      uint16_t width, uint16_t num_placements)
+{
+	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
+
+	igt_assert(width && num_placements &&
+		   (width == 1 || num_placements == 1));
+
+	igt_assert(width <= MAX_PARALLEL);
+
+	ctx->class = hwe->engine_class;
+	ctx->width = width;
+	ctx->num_placements = num_placements;
+	ctx->vm = vm;
+	for (int i = 0; i < ctx->width; i++)
+		ctx->addr[i] = batch_addr[hwe->engine_class];
+
+	ctx->exec.num_batch_buffer = width;
+	ctx->exec.num_syncs = 2;
+	ctx->exec.syncs = to_user_pointer(ctx->sync);
+
+	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[0].handle = syncobj_create(fd, 0);
+
+	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[1].handle = syncobj_create(fd, 0);
+
+	ctx->bo_size = sizeof(struct xe_spin);
+	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
+	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
+			       vram_if_possible(fd, hwe->gt_id),
+			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
+	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
+
+	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
+					     hwe, 0, &ctx->exec_queue), 0);
+
+	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
+			 ctx->sync, 1);
+
+	return ctx;
+}
+
+static void
+spin_sync_start(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	ctx->spin_opts.addr = ctx->addr[0];
+	ctx->spin_opts.preempt = true;
+	xe_spin_init(ctx->spin, &ctx->spin_opts);
+
+	/* re-use sync[0] for exec */
+	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
+
+	ctx->exec.exec_queue_id = ctx->exec_queue;
+	if (ctx->width > 1)
+		ctx->exec.address = to_user_pointer(ctx->addr);
+	else
+		ctx->exec.address = ctx->addr[0];
+	xe_exec(fd, &ctx->exec);
+
+	xe_spin_wait_started(ctx->spin);
+	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
+
+	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
+}
+
+static void
+spin_sync_end(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx || ctx->ended)
+		return;
+
+	xe_spin_end(ctx->spin);
+
+	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
+	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->ended = true;
+	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
+}
+
+static void
+spin_ctx_destroy(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	syncobj_destroy(fd, ctx->sync[0].handle);
+	syncobj_destroy(fd, ctx->sync[1].handle);
+	xe_exec_queue_destroy(fd, ctx->exec_queue);
+
+	munmap(ctx->spin, ctx->bo_size);
+	gem_close(fd, ctx->bo);
+
+	free(ctx);
+}
+

igt_main

{
	int xe;
Signature · 1 line
2.38.1
#15 lore raw

Re: [PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

Quoted 23 lines · Lucas De Marchi
Quoted 22 lines · Umesh Nerlige Ramappa

Add helpers for submitting batches and waiting for them to start.

v2: Remove xe prefixes from the structures and helpers (Lucas)

---
tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++

1 file changed, 135 insertions(+)
tests/intel/xe_drm_fdinfo.c +5 -0
index c697f3e53..037f25e53 100644
@@ -54,6 +54,17 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };
+
+static const uint64_t batch_addr[] = {
+	0x170000,
+	0x180000,
+	0x190000,
Quoted 8 lines

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 67

Quoted 6 lines
tests/intel/xe_drm_fdinfo.c +6 -0
+	0x1a0000,
+	0x1b0000,
+	0x1c0000,
+	0x1d0000,
+	0x1e0000,
+};

where are these numbers coming from?

The hardcoded number above are arbitrary, although I don't know if that's an issue. I did see similar numbers in use with other xe tests.

Quoted 4 lines

I'm looking at
tests/intel/xe_spin_batch.c and the lib/xe/xe_spin.c implementation and I don't understand why we need to add so many helpers and handcrafted things in this particular test.

These helpers are just a break down of something like xe_exec_balancer.c -> test_exec(). The helpers are still using functions from xe_spin.c.

The breakdown helps control some execution elements for batches like (1) using a single address space when running batches on all classes, (2) configuring width and num_placements for parallel/virtual cases and (3) provide some flexibility to test utilization when the batch has started running.

Regards,
Umesh

Quoted 143 lines

+Matt Brost, +Zbigniew Kempczyński to help help here

Lucas De Marchi

Quoted 137 lines
tests/intel/xe_drm_fdinfo.c +124 -0
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
 	igt_require(info.num_engines);
 }
 
+#define MAX_PARALLEL 8
+struct spin_ctx {
+	uint32_t vm;
+	uint64_t addr[MAX_PARALLEL];
+	struct drm_xe_sync sync[2];
+	struct drm_xe_exec exec;
+	uint32_t exec_queue;
+	size_t bo_size;
+	uint32_t bo;
+	struct xe_spin *spin;
+	struct xe_spin_opts spin_opts;
+	bool ended;
+	uint16_t class;
+	uint16_t width;
+	uint16_t num_placements;
+};
+
+static struct spin_ctx *
+spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
+	      uint16_t width, uint16_t num_placements)
+{
+	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
+
+	igt_assert(width && num_placements &&
+		   (width == 1 || num_placements == 1));
+
+	igt_assert(width <= MAX_PARALLEL);
+
+	ctx->class = hwe->engine_class;
+	ctx->width = width;
+	ctx->num_placements = num_placements;
+	ctx->vm = vm;
+	for (int i = 0; i < ctx->width; i++)
+		ctx->addr[i] = batch_addr[hwe->engine_class];
+
+	ctx->exec.num_batch_buffer = width;
+	ctx->exec.num_syncs = 2;
+	ctx->exec.syncs = to_user_pointer(ctx->sync);
+
+	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[0].handle = syncobj_create(fd, 0);
+
+	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[1].handle = syncobj_create(fd, 0);
+
+	ctx->bo_size = sizeof(struct xe_spin);
+	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
+	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
+			       vram_if_possible(fd, hwe->gt_id),
+			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
+	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
+
+	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
+					     hwe, 0, &ctx->exec_queue), 0);
+
+	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
+			 ctx->sync, 1);
+
+	return ctx;
+}
+
+static void
+spin_sync_start(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	ctx->spin_opts.addr = ctx->addr[0];
+	ctx->spin_opts.preempt = true;
+	xe_spin_init(ctx->spin, &ctx->spin_opts);
+
+	/* re-use sync[0] for exec */
+	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
+
+	ctx->exec.exec_queue_id = ctx->exec_queue;
+	if (ctx->width > 1)
+		ctx->exec.address = to_user_pointer(ctx->addr);
+	else
+		ctx->exec.address = ctx->addr[0];
+	xe_exec(fd, &ctx->exec);
+
+	xe_spin_wait_started(ctx->spin);
+	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
+
+	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
+}
+
+static void
+spin_sync_end(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx || ctx->ended)
+		return;
+
+	xe_spin_end(ctx->spin);
+
+	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
+	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->ended = true;
+	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
+}
+
+static void
+spin_ctx_destroy(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	syncobj_destroy(fd, ctx->sync[0].handle);
+	syncobj_destroy(fd, ctx->sync[1].handle);
+	xe_exec_queue_destroy(fd, ctx->exec_queue);
+
+	munmap(ctx->spin, ctx->bo_size);
+	gem_close(fd, ctx->bo);
+
+	free(ctx);
+}
+

igt_main

{
	int xe;
Signature · 1 line
2.38.1
#16 lore raw

Re: [PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

Quoted 33 lines · Umesh Nerlige Ramappa
Quoted 23 lines · Lucas De Marchi
Quoted 22 lines · Umesh Nerlige Ramappa

Add helpers for submitting batches and waiting for them to start.

v2: Remove xe prefixes from the structures and helpers (Lucas)

---
tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++

1 file changed, 135 insertions(+)
tests/intel/xe_drm_fdinfo.c +5 -0
index c697f3e53..037f25e53 100644
@@ -54,6 +54,17 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };
+
+static const uint64_t batch_addr[] = {
+	0x170000,
+	0x180000,
+	0x190000,
Quoted 8 lines

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 67

Quoted 6 lines
tests/intel/xe_drm_fdinfo.c +6 -0
+	0x1a0000,
+	0x1b0000,
+	0x1c0000,
+	0x1d0000,
+	0x1e0000,
+};

where are these numbers coming from?

Quoted 3 lines

The hardcoded number above are arbitrary, although I don't know if that's an issue. I did see similar numbers in use with other xe tests.

See below, I think you can just use a single address when setting up the cork (see below).

Quoted 5 lines
Quoted 4 lines

I'm looking at
tests/intel/xe_spin_batch.c and the lib/xe/xe_spin.c implementation and I don't understand why we need to add so many helpers and handcrafted things in this particular test.

Quoted 9 lines

These helpers are just a break down of something like xe_exec_balancer.c -> test_exec(). The helpers are still using functions from xe_spin.c.

The breakdown helps control some execution elements for batches like (1) using a single address space when running batches on all classes, (2) configuring width and num_placements for parallel/virtual cases and (3) provide some flexibility to test utilization when the batch has started running.

I see what you are trying to do, create spinners for virtual and parallel exec queues.

I think the proper place to put this would be in xe_spin.c adding 1 or 2 new functions that look like xe_cork_init() but for virtual and parallel exec queues. Then just use existing cork functions for everything else.

I think that will work.

I don't think any IGTs actually use the cork stuff yet (several probably should) but I did post an IGT recently [1] that made use of this.

Matt

[1] https://patchwork.freedesktop.org/patch/600254/?series=135143&rev=1

Quoted 146 lines

Regards,
Umesh

Quoted 143 lines

+Matt Brost, +Zbigniew Kempczyński to help help here

Lucas De Marchi

Quoted 137 lines
tests/intel/xe_drm_fdinfo.c +124 -0
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
 	igt_require(info.num_engines);
 }
 
+#define MAX_PARALLEL 8
+struct spin_ctx {
+	uint32_t vm;
+	uint64_t addr[MAX_PARALLEL];
+	struct drm_xe_sync sync[2];
+	struct drm_xe_exec exec;
+	uint32_t exec_queue;
+	size_t bo_size;
+	uint32_t bo;
+	struct xe_spin *spin;
+	struct xe_spin_opts spin_opts;
+	bool ended;
+	uint16_t class;
+	uint16_t width;
+	uint16_t num_placements;
+};
+
+static struct spin_ctx *
+spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
+	      uint16_t width, uint16_t num_placements)
+{
+	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
+
+	igt_assert(width && num_placements &&
+		   (width == 1 || num_placements == 1));
+
+	igt_assert(width <= MAX_PARALLEL);
+
+	ctx->class = hwe->engine_class;
+	ctx->width = width;
+	ctx->num_placements = num_placements;
+	ctx->vm = vm;
+	for (int i = 0; i < ctx->width; i++)
+		ctx->addr[i] = batch_addr[hwe->engine_class];
+
+	ctx->exec.num_batch_buffer = width;
+	ctx->exec.num_syncs = 2;
+	ctx->exec.syncs = to_user_pointer(ctx->sync);
+
+	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[0].handle = syncobj_create(fd, 0);
+
+	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[1].handle = syncobj_create(fd, 0);
+
+	ctx->bo_size = sizeof(struct xe_spin);
+	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
+	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
+			       vram_if_possible(fd, hwe->gt_id),
+			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
+	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
+
+	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
+					     hwe, 0, &ctx->exec_queue), 0);
+
+	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
+			 ctx->sync, 1);
+
+	return ctx;
+}
+
+static void
+spin_sync_start(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	ctx->spin_opts.addr = ctx->addr[0];
+	ctx->spin_opts.preempt = true;
+	xe_spin_init(ctx->spin, &ctx->spin_opts);
+
+	/* re-use sync[0] for exec */
+	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
+
+	ctx->exec.exec_queue_id = ctx->exec_queue;
+	if (ctx->width > 1)
+		ctx->exec.address = to_user_pointer(ctx->addr);
+	else
+		ctx->exec.address = ctx->addr[0];
+	xe_exec(fd, &ctx->exec);
+
+	xe_spin_wait_started(ctx->spin);
+	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
+
+	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
+}
+
+static void
+spin_sync_end(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx || ctx->ended)
+		return;
+
+	xe_spin_end(ctx->spin);
+
+	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
+	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->ended = true;
+	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
+}
+
+static void
+spin_ctx_destroy(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	syncobj_destroy(fd, ctx->sync[0].handle);
+	syncobj_destroy(fd, ctx->sync[1].handle);
+	xe_exec_queue_destroy(fd, ctx->exec_queue);
+
+	munmap(ctx->spin, ctx->bo_size);
+	gem_close(fd, ctx->bo);
+
+	free(ctx);
+}
+

igt_main

{
	int xe;
Signature · 1 line
2.38.1
#17 lore raw

Re: [PATCH i-g-t v2 04/10] tests/intel/xe_drm_fdinfo: Add helpers for spinning batches

Quoted 34 lines · Matthew Brost
Quoted 33 lines · Umesh Nerlige Ramappa
Quoted 31 lines · Lucas De Marchi
Quoted 18 lines · Umesh Nerlige Ramappa

Add helpers for submitting batches and waiting for them to start.

v2: Remove xe prefixes from the structures and helpers (Lucas)

---
tests/intel/xe_drm_fdinfo.c | 135 ++++++++++++++++++++++++++++++++++++

1 file changed, 135 insertions(+)
tests/intel/xe_drm_fdinfo.c +1 -0
index c697f3e53..037f25e53 100644
@@ -54,6 +54,17 @@ static const char *engine_map[] = {
 	"vecs",
 	"ccs",
 };
+

tests/intel/xe_drm_fdinfo.c static const char *engine_map[] = { · line 67

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+static const uint64_t batch_addr[] = {
+	0x170000,
+	0x180000,
+	0x190000,
+	0x1a0000,
+	0x1b0000,
+	0x1c0000,
+	0x1d0000,
+	0x1e0000,
+};

where are these numbers coming from?

Quoted 6 lines
Quoted 3 lines

The hardcoded number above are arbitrary, although I don't know if that's an issue. I did see similar numbers in use with other xe tests.

See below, I think you can just use a single address when setting up the cork (see below).

because each cork uses a different vm rather than the shared vm used here, right?

Quoted 19 lines
Quoted 6 lines
Quoted 5 lines

I'm looking at
tests/intel/xe_spin_batch.c and the lib/xe/xe_spin.c implementation and I don't understand why we need to add so many helpers and handcrafted things in this particular test.

Quoted 9 lines

These helpers are just a break down of something like xe_exec_balancer.c -> test_exec(). The helpers are still using functions from xe_spin.c.

The breakdown helps control some execution elements for batches like (1) using a single address space when running batches on all classes, (2) configuring width and num_placements for parallel/virtual cases and (3) provide some flexibility to test utilization when the batch has started running.

I see what you are trying to do, create spinners for virtual and parallel exec queues.

Quoted 6 lines

I think the proper place to put this would be in xe_spin.c adding 1 or 2 new functions that look like xe_cork_init() but for virtual and parallel exec queues. Then just use existing cork functions for everything else.

I think that will work.

It seems like one problem with cork is that it exec on init. Maybe adding a xe_cork_start_sync() so we can separate the setup part from the start part would be ok?

We could do _init() + _wait_started(), but with the current state of the test we are trying to minimize the jitter to calculate the time in which it ran. Although I was thinking... if we stop checking the cpu time and only rely on the gpu time we collect from fdinfo the igt asserts should be more reliable. And another thing we could use is the time saved by the gpu in the spin itself.

I sent a few patches to better abstract the parallel/virtual setup, similarly to what you suggested in another thread:
https://patchwork.freedesktop.org/series/137317/

I think that should mostly take care about the parallel/virtual tests added here. For the single() ones, I squashed locally the patches 3-5 (because otherwise they create warnings) and I'm thinking on merging them mostly as is. Then I add the missing cork stuff and migrate the test to use that instead of the local functions. Thoughts?

Long term I think we could do:

	s/xe_spin/xe_spin_bath/
	s/xe_cork/xe_spinner/

Thoughts?

Thanks
Lucas De Marchi

Quoted 154 lines

I don't think any IGTs actually use the cork stuff yet (several probably should) but I did post an IGT recently [1] that made use of this.

Matt

[1] https://patchwork.freedesktop.org/patch/600254/?series=135143&rev=1

Quoted 146 lines

Regards,
Umesh

Quoted 143 lines

+Matt Brost, +Zbigniew Kempczyński to help help here

Lucas De Marchi

Quoted 137 lines
tests/intel/xe_drm_fdinfo.c +124 -0
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -329,6 +340,130 @@ static void basic_engine_utilization(int xe)
 	igt_require(info.num_engines);
 }
 
+#define MAX_PARALLEL 8
+struct spin_ctx {
+	uint32_t vm;
+	uint64_t addr[MAX_PARALLEL];
+	struct drm_xe_sync sync[2];
+	struct drm_xe_exec exec;
+	uint32_t exec_queue;
+	size_t bo_size;
+	uint32_t bo;
+	struct xe_spin *spin;
+	struct xe_spin_opts spin_opts;
+	bool ended;
+	uint16_t class;
+	uint16_t width;
+	uint16_t num_placements;
+};
+
+static struct spin_ctx *
+spin_ctx_init(int fd, struct drm_xe_engine_class_instance *hwe, uint32_t vm,
+	      uint16_t width, uint16_t num_placements)
+{
+	struct spin_ctx *ctx = calloc(1, sizeof(*ctx));
+
+	igt_assert(width && num_placements &&
+		   (width == 1 || num_placements == 1));
+
+	igt_assert(width <= MAX_PARALLEL);
+
+	ctx->class = hwe->engine_class;
+	ctx->width = width;
+	ctx->num_placements = num_placements;
+	ctx->vm = vm;
+	for (int i = 0; i < ctx->width; i++)
+		ctx->addr[i] = batch_addr[hwe->engine_class];
+
+	ctx->exec.num_batch_buffer = width;
+	ctx->exec.num_syncs = 2;
+	ctx->exec.syncs = to_user_pointer(ctx->sync);
+
+	ctx->sync[0].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[0].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[0].handle = syncobj_create(fd, 0);
+
+	ctx->sync[1].type = DRM_XE_SYNC_TYPE_SYNCOBJ;
+	ctx->sync[1].flags = DRM_XE_SYNC_FLAG_SIGNAL;
+	ctx->sync[1].handle = syncobj_create(fd, 0);
+
+	ctx->bo_size = sizeof(struct xe_spin);
+	ctx->bo_size = xe_bb_size(fd, ctx->bo_size);
+	ctx->bo = xe_bo_create(fd, ctx->vm, ctx->bo_size,
+			       vram_if_possible(fd, hwe->gt_id),
+			       DRM_XE_GEM_CREATE_FLAG_NEEDS_VISIBLE_VRAM);
+	ctx->spin = xe_bo_map(fd, ctx->bo, ctx->bo_size);
+
+	igt_assert_eq(__xe_exec_queue_create(fd, ctx->vm, width, num_placements,
+					     hwe, 0, &ctx->exec_queue), 0);
+
+	xe_vm_bind_async(fd, ctx->vm, 0, ctx->bo, 0, ctx->addr[0], ctx->bo_size,
+			 ctx->sync, 1);
+
+	return ctx;
+}
+
+static void
+spin_sync_start(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	ctx->spin_opts.addr = ctx->addr[0];
+	ctx->spin_opts.preempt = true;
+	xe_spin_init(ctx->spin, &ctx->spin_opts);
+
+	/* re-use sync[0] for exec */
+	ctx->sync[0].flags &= ~DRM_XE_SYNC_FLAG_SIGNAL;
+
+	ctx->exec.exec_queue_id = ctx->exec_queue;
+	if (ctx->width > 1)
+		ctx->exec.address = to_user_pointer(ctx->addr);
+	else
+		ctx->exec.address = ctx->addr[0];
+	xe_exec(fd, &ctx->exec);
+
+	xe_spin_wait_started(ctx->spin);
+	igt_assert(!syncobj_wait(fd, &ctx->sync[1].handle, 1, 1, 0, NULL));
+
+	igt_debug("%s: spinner started\n", engine_map[ctx->class]);
+}
+
+static void
+spin_sync_end(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx || ctx->ended)
+		return;
+
+	xe_spin_end(ctx->spin);
+
+	igt_assert(syncobj_wait(fd, &ctx->sync[1].handle, 1, INT64_MAX, 0, NULL));
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->sync[0].flags |= DRM_XE_SYNC_FLAG_SIGNAL;
+	xe_vm_unbind_async(fd, ctx->vm, 0, 0, ctx->addr[0], ctx->bo_size, ctx->sync, 1);
+	igt_assert(syncobj_wait(fd, &ctx->sync[0].handle, 1, INT64_MAX, 0, NULL));
+
+	ctx->ended = true;
+	igt_debug("%s: spinner ended\n", engine_map[ctx->class]);
+}
+
+static void
+spin_ctx_destroy(int fd, struct spin_ctx *ctx)
+{
+	if (!ctx)
+		return;
+
+	syncobj_destroy(fd, ctx->sync[0].handle);
+	syncobj_destroy(fd, ctx->sync[1].handle);
+	xe_exec_queue_destroy(fd, ctx->exec_queue);
+
+	munmap(ctx->spin, ctx->bo_size);
+	gem_close(fd, ctx->bo);
+
+	free(ctx);
+}
+

igt_main

{
	int xe;

--
2.38.1

#18 lore raw

PATCH v2 5/10 tests/intel/xe_drm_fdinfo: Add single engine tests

In reply to Umesh Nerlige Ramappa

Add simple tests that submit work to one engine and measure utilization per class.

v2:
- Drop measured_usleep since return value is not used
- s/parallel engines/parallel submission/ in comment
- Use NSEC_PER_SEC for batch_duration_ns
- Percent should not be > 100
- Check utilization for both clients for isolation case

---
 tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++
 1 file changed, 97 insertions(+)
tests/intel/xe_drm_fdinfo.c +97 -0
index 037f25e53..410c885e7 100644
@@ -28,6 +28,15 @@
  * SUBTEST: basic-engine-utilization
  * Description: Check if basic fdinfo content is present for engine utilization
  *
+ * SUBTEST: drm-idle
+ * Description: Check that engines show no load when idle
+ *
+ * SUBTEST: drm-busy-idle
+ * Description: Check that engines show load when idle after busy
+ *
+ * SUBTEST: drm-busy-idle-isolation
+ * Description: Check that engine load does not spill over to other drm clients
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+/* flag masks */
+#define TEST_BUSY		(1 << 0)
+#define TEST_TRAILING_IDLE	(1 << 1)
+#define TEST_ISOLATION		(1 << 2)
+
 struct pceu_cycles {
 	uint64_t cycles;
 	uint64_t total_cycles;
 };
 
+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
 	free(ctx);
 }
 
+static void
+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
+	      int class, int width, unsigned int flags)
+{
+	double percent;
+
+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
+
+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
+
+	/* for parallel submission scale the busyness with width */
+	percent = percent / width;
+
+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
+
+	if (flags & TEST_BUSY)
+		igt_assert(percent >= 95 && percent <= 100);
+	else
+		igt_assert(!percent);
+}
+
+static void
+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+       unsigned int flags)
+{
+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+	int new_fd;
+
+	if (flags & TEST_ISOLATION)
+		new_fd = drm_reopen_driver(fd);
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
+		spin_sync_start(fd, ctx);
+	}
+
+	read_engine_cycles(fd, pceu1[0]);
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu1[1]);
+
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE)
+		spin_sync_end(fd, ctx);
+
+	read_engine_cycles(fd, pceu2[0]);
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu2[1]);
+
+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);
+
+	if (flags & TEST_ISOLATION) {
+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);
+		close(new_fd);
+	}
+
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);
+	xe_vm_destroy(fd, vm);
+}
+
 igt_main
 {
+	struct drm_xe_engine_class_instance *hwe;
 	int xe;
 
 	igt_fixture {
@@ -484,6 +569,18 @@ igt_main
 	igt_subtest("basic-engine-utilization")
 		basic_engine_utilization(xe);
 
+	igt_subtest("drm-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, 0);
+
+	igt_subtest("drm-busy-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-busy-idle-isolation")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#19 lore raw

Re: [PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

Quoted 8 lines · Umesh Nerlige Ramappa

Add simple tests that submit work to one engine and measure utilization per class.

v2:
- Drop measured_usleep since return value is not used
- s/parallel engines/parallel submission/ in comment
- Use NSEC_PER_SEC for batch_duration_ns
- Percent should not be > 100

but as we chatted later in the review, it's actually possible to have it greater than 100, so I think we need to leave some room

Quoted 61 lines

- Check utilization for both clients for isolation case

---
tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++

1 file changed, 97 insertions(+)
tests/intel/xe_drm_fdinfo.c +27 -0
index 037f25e53..410c885e7 100644
@@ -28,6 +28,15 @@
 * SUBTEST: basic-engine-utilization
 * Description: Check if basic fdinfo content is present for engine utilization
 *
+ * SUBTEST: drm-idle
+ * Description: Check that engines show no load when idle
+ *
+ * SUBTEST: drm-busy-idle
+ * Description: Check that engines show load when idle after busy
+ *
+ * SUBTEST: drm-busy-idle-isolation
+ * Description: Check that engine load does not spill over to other drm clients
+ *
 * SUBTEST: drm-total-resident
 * Description: Create and compare total and resident memory consumption by client
 *
@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+/* flag masks */
+#define TEST_BUSY		(1 << 0)
+#define TEST_TRAILING_IDLE	(1 << 1)
+#define TEST_ISOLATION		(1 << 2)
+
 struct pceu_cycles {
 	uint64_t cycles;
 	uint64_t total_cycles;
 };
 
+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
 	free(ctx);
 }
 
+static void
+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
+	      int class, int width, unsigned int flags)
+{
+	double percent;
+
+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
+

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 503

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
+
+	/* for parallel submission scale the busyness with width */
+	percent = percent / width;
+
+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
+
+	if (flags & TEST_BUSY)
+		igt_assert(percent >= 95 && percent <= 100);

<= 105 like you had before seems good, otherwise we may have too much noise in CI. What are the typical numbers you're seeing?

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 513

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	else
+		igt_assert(!percent);
+}
+
+static void
+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+       unsigned int flags)
+{
+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];

nit: either name it pceu_start/pceu_end:

pceu_start[0] is the pceu for client0 on start
pceu_start[1] is the pceu for client1 on start
pceu_end[0] is the pceu for client0 on end
pceu_end[1] is the pceu for client1 on end

or pceu_client0/pceu_client1 if grouping by client

pceu_client0[0] is the pceu for client0 on start
pceu_client0[1] is the pceu for client0 on end
pceu_client1[1] is the pceu for client1 on start
pceu_client1[2] is the pceu for client1 on end

... this makes it less prone to typos.

But see below as I think 2 samples are not sufficient.

Quoted 8 lines
tests/intel/xe_drm_fdinfo.c +8 -0
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+	int new_fd;
+
+	if (flags & TEST_ISOLATION)
+		new_fd = drm_reopen_driver(fd);
+
+	vm = xe_vm_create(fd, 0, 0);

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 531

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	if (flags & TEST_BUSY) {
+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
+		spin_sync_start(fd, ctx);
+	}
+
+	read_engine_cycles(fd, pceu1[0]);
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu1[1]);
+
+	usleep(batch_duration_ns / 1000);

NSEC_PER_USEC

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 535

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +4 -0
+	if (flags & TEST_TRAILING_IDLE)
+		spin_sync_end(fd, ctx);
+
+	read_engine_cycles(fd, pceu2[0]);

... and here we have another source of noise.... when doing TEST_TRAILING_IDLE, should we read the first sample after the spin ended, sleep, and read the second sample?

but then how are we testing all at the same time here? I can't see any igt_assert() for the trailing idle test and it will actually impact the busy test since it will wait the spin to end.... Am I missing anything here? I think we need to add a third sample when testing trailing idle and ensure:

considering the indexing as sample[client][idx]

	sample[0][1] - sample[0][0] == 100% if BUSY else 0%
	sample[0][2] - sample[0][1] == 0%  -> idle on trailing test

and for isolation:

	sample[1][1] - sample[0][0] == 0%
	sample[1][2] - sample[1][1] == 0%

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 547

Quoted 12 lines
tests/intel/xe_drm_fdinfo.c +12 -0
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu2[1]);
+
+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);
+
+	if (flags & TEST_ISOLATION) {
+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);
+		close(new_fd);
+	}
+
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);

if !BUSY you are not even calling spin_ctx_init()

Lucas De Marchi

Quoted 31 lines
tests/intel/xe_drm_fdinfo.c +3 -0
+	xe_vm_destroy(fd, vm);
+}
+

igt_main
{

tests/intel/xe_drm_fdinfo.c +1 -0
+	struct drm_xe_engine_class_instance *hwe;
	int xe;

	igt_fixture {
tests/intel/xe_drm_fdinfo.c +12 -0
@@ -484,6 +569,18 @@ igt_main
 	igt_subtest("basic-engine-utilization")
 		basic_engine_utilization(xe);
 
+	igt_subtest("drm-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, 0);
+
+	igt_subtest("drm-busy-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-busy-idle-isolation")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#20 lore raw

Re: [PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

Quoted 79 lines · Lucas De Marchi
Quoted 8 lines · Umesh Nerlige Ramappa

Add simple tests that submit work to one engine and measure utilization per class.

v2:
- Drop measured_usleep since return value is not used
- s/parallel engines/parallel submission/ in comment
- Use NSEC_PER_SEC for batch_duration_ns
- Percent should not be > 100

but as we chatted later in the review, it's actually possible to have it greater than 100, so I think we need to leave some room

Quoted 66 lines

- Check utilization for both clients for isolation case

---
tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++

1 file changed, 97 insertions(+)
tests/intel/xe_drm_fdinfo.c +32 -0
index 037f25e53..410c885e7 100644
@@ -28,6 +28,15 @@
 * SUBTEST: basic-engine-utilization
 * Description: Check if basic fdinfo content is present for engine utilization
 *
+ * SUBTEST: drm-idle
+ * Description: Check that engines show no load when idle
+ *
+ * SUBTEST: drm-busy-idle
+ * Description: Check that engines show load when idle after busy
+ *
+ * SUBTEST: drm-busy-idle-isolation
+ * Description: Check that engine load does not spill over to other drm clients
+ *
 * SUBTEST: drm-total-resident
 * Description: Create and compare total and resident memory consumption by client
 *
@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+/* flag masks */
+#define TEST_BUSY		(1 << 0)
+#define TEST_TRAILING_IDLE	(1 << 1)
+#define TEST_ISOLATION		(1 << 2)
+
 struct pceu_cycles {
 	uint64_t cycles;
 	uint64_t total_cycles;
 };
 
+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
 	free(ctx);
 }
 
+static void
+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
+	      int class, int width, unsigned int flags)
+{
+	double percent;
+
+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
+
+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
+
+	/* for parallel submission scale the busyness with width */
+	percent = percent / width;
Quoted 8 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 503

Quoted 5 lines
tests/intel/xe_drm_fdinfo.c +5 -0
+
+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
+
+	if (flags & TEST_BUSY)
+		igt_assert(percent >= 95 && percent <= 100);

<= 105 like you had before seems good, otherwise we may have too much noise in CI. What are the typical numbers you're seeing?

I see 101 whenever it fails. Also I see it only when I run batches on all classes (all_busy_all/most_busy_all etc.). The difference in numerator and denominator deltas is close to 5ms though. Not convinced that we should see 100 still because we are ending the spinners (and waiting for fences and all) before taking sample 2. Still debugging a little more on this one.

Quoted 51 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 513

Quoted 11 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	else
+		igt_assert(!percent);
+}
+
+static void
+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+       unsigned int flags)
+{
+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];

nit: either name it pceu_start/pceu_end:

pceu_start[0] is the pceu for client0 on start
pceu_start[1] is the pceu for client1 on start
pceu_end[0] is the pceu for client0 on end
pceu_end[1] is the pceu for client1 on end

or pceu_client0/pceu_client1 if grouping by client

pceu_client0[0] is the pceu for client0 on start
pceu_client0[1] is the pceu for client0 on end
pceu_client1[1] is the pceu for client1 on start
pceu_client1[2] is the pceu for client1 on end

... this makes it less prone to typos.

But see below as I think 2 samples are not sufficient.

Quoted 8 lines
tests/intel/xe_drm_fdinfo.c +8 -0
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+	int new_fd;
+
+	if (flags & TEST_ISOLATION)
+		new_fd = drm_reopen_driver(fd);
+
+	vm = xe_vm_create(fd, 0, 0);

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 531

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	if (flags & TEST_BUSY) {
+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
+		spin_sync_start(fd, ctx);
+	}
+
+	read_engine_cycles(fd, pceu1[0]);
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu1[1]);
+
+	usleep(batch_duration_ns / 1000);

NSEC_PER_USEC

Quoted 8 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 535

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +4 -0
+	if (flags & TEST_TRAILING_IDLE)
+		spin_sync_end(fd, ctx);
+
+	read_engine_cycles(fd, pceu2[0]);

... and here we have another source of noise.... when doing TEST_TRAILING_IDLE, should we read the first sample after the spin ended, sleep, and read the second sample?

That would always make utilization = 0% (numerator delta is 0). We should take sample 1 when the spinner has started and sample 2 when it has ended (for PCEU).

TEST_TRAILING_IDLE is only valid when TEST_BUSY is also specified. Maybe I should enforce that in the test. This is the flow for common flag configs:

1) flag = TEST_BUSY | TEST_TRAILING_IDLE
- wait till spinner starts
- take sample 1
- usleep
- end the batch
- take sample 2.
- check_results (expected 100% util)

2) flag = TEST_BUSY
- wait till spinner starts
- take sample 1
- usleep
- take sample 2.
- end the batch
- check_results (expected 100% util)

3) flag = 0 means truly IDLE since there is no spinner.

Quoted 5 lines

but then how are we testing all at the same time here? I can't see any igt_assert() for the trailing idle test and it will actually impact the busy test since it will wait the spin to end.... Am I missing anything here?

Case (2) above does not exist for PCEU tests because CTX_TIMESTAMP is only updates when the context switches out, which means we must end the spinner before taking the second sample (or preempt it, but ending is simpler). This series only has tests for case (1) and (3) implemented.

Case (2) is applicable for a completely different feature which is still wip - per engine utilization via kernel perf framework.

Quoted 10 lines

I think we need to add a third sample when testing trailing idle and ensure:

considering the indexing as sample[client][idx]

	sample[0][1] - sample[0][0] == 100% if BUSY else 0%
	sample[0][2] - sample[0][1] == 0%  -> idle on trailing test

and for isolation:

Quoted 7 lines
	sample[1][1] - sample[0][0] == 0%
	sample[1][2] - sample[1][1] == 0%

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 539

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +4 -0
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu2[1]);
+
+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);

This check_results will check for TEST_BUSY case.

Quoted 3 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 542

Quoted 3 lines
tests/intel/xe_drm_fdinfo.c +3 -0
+
+	if (flags & TEST_ISOLATION) {
+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);

This check_results will check for idle case (flags = 0).

Quoted 7 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 547

Quoted 5 lines
tests/intel/xe_drm_fdinfo.c +5 -0
+		close(new_fd);
+	}
+
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);

if !BUSY you are not even calling spin_ctx_init()

:) yeah, sorry about the readability here. The spin_sync_end and spin_ctx_destroy bail out if the ctx is NULL which is the case for !BUSY. I can add and explicit check to not call these instead.

Thanks,
Umesh

Quoted 34 lines

Lucas De Marchi

Quoted 31 lines
tests/intel/xe_drm_fdinfo.c +3 -0
+	xe_vm_destroy(fd, vm);
+}
+

igt_main
{

tests/intel/xe_drm_fdinfo.c +1 -0
+	struct drm_xe_engine_class_instance *hwe;
	int xe;

	igt_fixture {
tests/intel/xe_drm_fdinfo.c +12 -0
@@ -484,6 +569,18 @@ igt_main
 	igt_subtest("basic-engine-utilization")
 		basic_engine_utilization(xe);
 
+	igt_subtest("drm-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, 0);
+
+	igt_subtest("drm-busy-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-busy-idle-isolation")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#21 lore raw

Re: [PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

Quoted 155 lines · Umesh Nerlige Ramappa
Quoted 79 lines · Lucas De Marchi
Quoted 8 lines · Umesh Nerlige Ramappa

Add simple tests that submit work to one engine and measure utilization per class.

v2:
- Drop measured_usleep since return value is not used
- s/parallel engines/parallel submission/ in comment
- Use NSEC_PER_SEC for batch_duration_ns
- Percent should not be > 100

but as we chatted later in the review, it's actually possible to have it greater than 100, so I think we need to leave some room

Quoted 66 lines

- Check utilization for both clients for isolation case

---
tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++

1 file changed, 97 insertions(+)
tests/intel/xe_drm_fdinfo.c +32 -0
index 037f25e53..410c885e7 100644
@@ -28,6 +28,15 @@
 * SUBTEST: basic-engine-utilization
 * Description: Check if basic fdinfo content is present for engine utilization
 *
+ * SUBTEST: drm-idle
+ * Description: Check that engines show no load when idle
+ *
+ * SUBTEST: drm-busy-idle
+ * Description: Check that engines show load when idle after busy
+ *
+ * SUBTEST: drm-busy-idle-isolation
+ * Description: Check that engine load does not spill over to other drm clients
+ *
 * SUBTEST: drm-total-resident
 * Description: Create and compare total and resident memory consumption by client
 *
@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+/* flag masks */
+#define TEST_BUSY		(1 << 0)
+#define TEST_TRAILING_IDLE	(1 << 1)
+#define TEST_ISOLATION		(1 << 2)
+
 struct pceu_cycles {
 	uint64_t cycles;
 	uint64_t total_cycles;
 };
 
+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
 	free(ctx);
 }
 
+static void
+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
+	      int class, int width, unsigned int flags)
+{
+	double percent;
+
+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
+
+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
+
+	/* for parallel submission scale the busyness with width */
+	percent = percent / width;
Quoted 8 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 503

Quoted 5 lines
tests/intel/xe_drm_fdinfo.c +5 -0
+
+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
+
+	if (flags & TEST_BUSY)
+		igt_assert(percent >= 95 && percent <= 100);

<= 105 like you had before seems good, otherwise we may have too much noise in CI. What are the typical numbers you're seeing?

I see 101 whenever it fails. Also I see it only when I run batches on all classes (all_busy_all/most_busy_all etc.). The difference in numerator and denominator deltas is close to 5ms though. Not convinced that we should see 100 still because we are ending the spinners (and waiting for fences and all) before taking sample 2. Still debugging a little more on this one.

Quoted 51 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 513

Quoted 11 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	else
+		igt_assert(!percent);
+}
+
+static void
+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+       unsigned int flags)
+{
+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];

nit: either name it pceu_start/pceu_end:

pceu_start[0] is the pceu for client0 on start
pceu_start[1] is the pceu for client1 on start
pceu_end[0] is the pceu for client0 on end
pceu_end[1] is the pceu for client1 on end

or pceu_client0/pceu_client1 if grouping by client

pceu_client0[0] is the pceu for client0 on start
pceu_client0[1] is the pceu for client0 on end
pceu_client1[1] is the pceu for client1 on start
pceu_client1[2] is the pceu for client1 on end

... this makes it less prone to typos.

But see below as I think 2 samples are not sufficient.

Quoted 8 lines
tests/intel/xe_drm_fdinfo.c +8 -0
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+	int new_fd;
+
+	if (flags & TEST_ISOLATION)
+		new_fd = drm_reopen_driver(fd);
+
+	vm = xe_vm_create(fd, 0, 0);

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 531

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	if (flags & TEST_BUSY) {
+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
+		spin_sync_start(fd, ctx);
+	}
+
+	read_engine_cycles(fd, pceu1[0]);
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu1[1]);
+
+	usleep(batch_duration_ns / 1000);

NSEC_PER_USEC

Quoted 8 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 535

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +4 -0
+	if (flags & TEST_TRAILING_IDLE)
+		spin_sync_end(fd, ctx);
+
+	read_engine_cycles(fd, pceu2[0]);

... and here we have another source of noise.... when doing TEST_TRAILING_IDLE, should we read the first sample after the spin ended, sleep, and read the second sample?

That would always make utilization = 0% (numerator delta is 0). We should take sample 1 when the spinner has started and sample 2 when it has ended (for PCEU).

TEST_TRAILING_IDLE is only valid when TEST_BUSY is also specified. Maybe I should enforce that in the test. This is the flow for common flag configs:

Quoted 7 lines

1) flag = TEST_BUSY | TEST_TRAILING_IDLE
- wait till spinner starts
- take sample 1
- usleep
- end the batch
- take sample 2.
- check_results (expected 100% util)

I think the way the test is written, the expectation is wrong. By absurd:

1) flag = TEST_BUSY | TEST_TRAILING_IDLE
- wait till spinner starts
- take sample 1
- usleep
- end the batch

		<--- long preempt here emulated with
		     usleep(500000)

- take sample 2.
- check_results

So we'd still expect the same ballpark number of delta ticks, but utilization overall would go down as now you are ending the batch and sleeping a little more before taking another sample. In the example above I'd expect utilization to be 50%

Quoted 12 lines

2) flag = TEST_BUSY
- wait till spinner starts
- take sample 1
- usleep
- take sample 2.
- end the batch
- check_results (expected 100% util)

3) flag = 0 means truly IDLE since there is no spinner.

Quoted 1 line
Quoted 6 lines
Quoted 4 lines

but then how are we testing all at the same time here? I can't see any igt_assert() for the trailing idle test and it will actually impact the busy test since it will wait the spin to end.... Am I missing anything here?

Case (2) above does not exist for PCEU tests because CTX_TIMESTAMP is only updates when the context switches out, which means we must end the spinner before taking the second sample (or preempt it, but ending is simpler). This series only has tests for case (1) and (3) implemented.

true, we are only reading the value from the context image, but does the hardware update it only on context switch? What I had understood was that the value was updated while the context was running. Otherwise I think it's utterly broken:

Considering
s* are the samples
c* are the context timestamps
g* are the gpu timestamps

- not running
| running

g0 g1 g2 g3 g4
c0 c1 c2 c3 c4

ctx0 ----||||||||||||||||||||||||||||||||||||-------
            ^                     ^              ^
            s0                    s1             s3                    

For simplicity, assuming g0 == c0 == 0

in s0: g1 == 3

       c1 == 0

in s1: g2 == 25

       c2 == 0

-> utilization returns 0%

in s3: g4 == 40

       c4 == c3 == 35

-> utilization returns 35 / (40 - 25) == 233%

how can we get this right if the value is not updated for running contexts?

Lucas De Marchi

Quoted 75 lines

Case (2) is applicable for a completely different feature which is still wip - per engine utilization via kernel perf framework.

Quoted 10 lines

I think we need to add a third sample when testing trailing idle and ensure:

considering the indexing as sample[client][idx]

	sample[0][1] - sample[0][0] == 100% if BUSY else 0%
	sample[0][2] - sample[0][1] == 0%  -> idle on trailing test

and for isolation:

Quoted 7 lines
	sample[1][1] - sample[0][0] == 0%
	sample[1][2] - sample[1][1] == 0%

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 539

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +4 -0
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu2[1]);
+
+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);

This check_results will check for TEST_BUSY case.

Quoted 3 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 542

Quoted 3 lines
tests/intel/xe_drm_fdinfo.c +3 -0
+
+	if (flags & TEST_ISOLATION) {
+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);

This check_results will check for idle case (flags = 0).

Quoted 7 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 547

Quoted 5 lines
tests/intel/xe_drm_fdinfo.c +5 -0
+		close(new_fd);
+	}
+
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);

if !BUSY you are not even calling spin_ctx_init()

:) yeah, sorry about the readability here. The spin_sync_end and spin_ctx_destroy bail out if the ctx is NULL which is the case for !BUSY. I can add and explicit check to not call these instead.

Thanks,
Umesh

Quoted 34 lines

Lucas De Marchi

Quoted 31 lines
tests/intel/xe_drm_fdinfo.c +3 -0
+	xe_vm_destroy(fd, vm);
+}
+

igt_main
{

tests/intel/xe_drm_fdinfo.c +1 -0
+	struct drm_xe_engine_class_instance *hwe;
	int xe;

	igt_fixture {
tests/intel/xe_drm_fdinfo.c +12 -0
@@ -484,6 +569,18 @@ igt_main
 	igt_subtest("basic-engine-utilization")
 		basic_engine_utilization(xe);
 
+	igt_subtest("drm-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, 0);
+
+	igt_subtest("drm-busy-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-busy-idle-isolation")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#22 lore raw

Re: [PATCH i-g-t v2 05/10] tests/intel/xe_drm_fdinfo: Add single engine tests

Quoted 174 lines · Lucas De Marchi
Quoted 155 lines · Umesh Nerlige Ramappa
Quoted 79 lines · Lucas De Marchi
Quoted 8 lines · Umesh Nerlige Ramappa

Add simple tests that submit work to one engine and measure utilization per class.

v2:
- Drop measured_usleep since return value is not used
- s/parallel engines/parallel submission/ in comment
- Use NSEC_PER_SEC for batch_duration_ns
- Percent should not be > 100

but as we chatted later in the review, it's actually possible to have it greater than 100, so I think we need to leave some room

Quoted 66 lines

- Check utilization for both clients for isolation case

---
tests/intel/xe_drm_fdinfo.c | 97 +++++++++++++++++++++++++++++++++++++

1 file changed, 97 insertions(+)
tests/intel/xe_drm_fdinfo.c +32 -0
index 037f25e53..410c885e7 100644
@@ -28,6 +28,15 @@
 * SUBTEST: basic-engine-utilization
 * Description: Check if basic fdinfo content is present for engine utilization
 *
+ * SUBTEST: drm-idle
+ * Description: Check that engines show no load when idle
+ *
+ * SUBTEST: drm-busy-idle
+ * Description: Check that engines show load when idle after busy
+ *
+ * SUBTEST: drm-busy-idle-isolation
+ * Description: Check that engine load does not spill over to other drm clients
+ *
 * SUBTEST: drm-total-resident
 * Description: Create and compare total and resident memory consumption by client
 *
@@ -42,11 +51,18 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 
 #define BO_SIZE (65536)
 
+/* flag masks */
+#define TEST_BUSY		(1 << 0)
+#define TEST_TRAILING_IDLE	(1 << 1)
+#define TEST_ISOLATION		(1 << 2)
+
 struct pceu_cycles {
 	uint64_t cycles;
 	uint64_t total_cycles;
 };
 
+const unsigned long batch_duration_ns = (1 * NSEC_PER_SEC) / 2;
+
 static const char *engine_map[] = {
 	"rcs",
 	"bcs",
@@ -464,8 +480,77 @@ spin_ctx_destroy(int fd, struct spin_ctx *ctx)
 	free(ctx);
 }
 
+static void
+check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
+	      int class, int width, unsigned int flags)
+{
+	double percent;
+
+	igt_debug("%s: sample 1: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s1[class].cycles, s1[class].total_cycles);
+	igt_debug("%s: sample 2: cycles %lu, total_cycles %lu\n",
+		  engine_map[class], s2[class].cycles, s2[class].total_cycles);
+
+	percent = ((s2[class].cycles - s1[class].cycles) * 100) /
+		  ((s2[class].total_cycles + 1) - s1[class].total_cycles);
+
+	/* for parallel submission scale the busyness with width */
+	percent = percent / width;
Quoted 8 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 503

Quoted 5 lines
tests/intel/xe_drm_fdinfo.c +5 -0
+
+	igt_debug("%s: percent: %f\n", engine_map[class], percent);
+
+	if (flags & TEST_BUSY)
+		igt_assert(percent >= 95 && percent <= 100);

<= 105 like you had before seems good, otherwise we may have too much noise in CI. What are the typical numbers you're seeing?

I see 101 whenever it fails. Also I see it only when I run batches on all classes (all_busy_all/most_busy_all etc.). The difference in numerator and denominator deltas is close to 5ms though. Not convinced that we should see 100 still because we are ending the spinners (and waiting for fences and all) before taking sample 2. Still debugging a little more on this one.

Quoted 51 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 513

Quoted 11 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	else
+		igt_assert(!percent);
+}
+
+static void
+single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+       unsigned int flags)
+{
+	struct pceu_cycles pceu1[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[2][DRM_XE_ENGINE_CLASS_COMPUTE + 1];

nit: either name it pceu_start/pceu_end:

pceu_start[0] is the pceu for client0 on start
pceu_start[1] is the pceu for client1 on start
pceu_end[0] is the pceu for client0 on end
pceu_end[1] is the pceu for client1 on end

or pceu_client0/pceu_client1 if grouping by client

pceu_client0[0] is the pceu for client0 on start
pceu_client0[1] is the pceu for client0 on end
pceu_client1[1] is the pceu for client1 on start
pceu_client1[2] is the pceu for client1 on end

... this makes it less prone to typos.

But see below as I think 2 samples are not sufficient.

Quoted 8 lines
tests/intel/xe_drm_fdinfo.c +8 -0
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+	int new_fd;
+
+	if (flags & TEST_ISOLATION)
+		new_fd = drm_reopen_driver(fd);
+
+	vm = xe_vm_create(fd, 0, 0);

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 531

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +10 -0
+	if (flags & TEST_BUSY) {
+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
+		spin_sync_start(fd, ctx);
+	}
+
+	read_engine_cycles(fd, pceu1[0]);
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu1[1]);
+
+	usleep(batch_duration_ns / 1000);

NSEC_PER_USEC

Quoted 8 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 535

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +4 -0
+	if (flags & TEST_TRAILING_IDLE)
+		spin_sync_end(fd, ctx);
+
+	read_engine_cycles(fd, pceu2[0]);

... and here we have another source of noise.... when doing TEST_TRAILING_IDLE, should we read the first sample after the spin ended, sleep, and read the second sample?

That would always make utilization = 0% (numerator delta is 0). We should take sample 1 when the spinner has started and sample 2 when it has ended (for PCEU).

TEST_TRAILING_IDLE is only valid when TEST_BUSY is also specified. Maybe I should enforce that in the test. This is the flow for common flag configs:

Quoted 7 lines

1) flag = TEST_BUSY | TEST_TRAILING_IDLE
- wait till spinner starts
- take sample 1
- usleep
- end the batch
- take sample 2.
- check_results (expected 100% util)

I think the way the test is written, the expectation is wrong. By absurd:

1) flag = TEST_BUSY | TEST_TRAILING_IDLE
- wait till spinner starts
- take sample 1
- usleep
- end the batch

		<--- long preempt here emulated with
		     usleep(500000)
Quoted 7 lines

- take sample 2.
- check_results

So we'd still expect the same ballpark number of delta ticks, but utilization overall would go down as now you are ending the batch and sleeping a little more before taking another sample. In the example above I'd expect utilization to be 50%

Indeed. if preempted after ending batch, then the numbers will be off, but that would be for this test. In other words, the test will not be able to verify this accurately. In reality, user will see the correct utilization - 50%.

Quoted 18 lines
Quoted 18 lines

2) flag = TEST_BUSY
- wait till spinner starts
- take sample 1
- usleep
- take sample 2.
- end the batch
- check_results (expected 100% util)

3) flag = 0 means truly IDLE since there is no spinner.

Quoted 5 lines

but then how are we testing all at the same time here? I can't see any igt_assert() for the trailing idle test and it will actually impact the busy test since it will wait the spin to end.... Am I missing anything here?

Quoted 6 lines
Quoted 1 line

Case (2) above does not exist for PCEU tests because CTX_TIMESTAMP is only updates when the context switches out, which means we must end the spinner before taking the second sample (or preempt it, but ending is simpler). This series only has tests for case (1) and (3) implemented.

true, we are only reading the value from the context image, but does the hardware update it only on context switch? What I had understood was that the value was updated while the context was running. Otherwise I think it's utterly broken:

CTX_TIMESTAMP in the context image is only updated on a context switch.

Quoted 24 lines

Considering
s* are the samples
c* are the context timestamps
g* are the gpu timestamps

- not running
| running

g0 g1 g2 g3 g4
c0 c1 c2 c3 c4

ctx0 ----||||||||||||||||||||||||||||||||||||-------
           ^                     ^              ^
           s0                    s1             s3

For simplicity, assuming g0 == c0 == 0

in s0: g1 == 3

      c1 == 0

in s1: g2 == 25

      c2 == 0
Quoted 7 lines

-> utilization returns 0%

in s3: g4 == 40

      c4 == c3 == 35

-> utilization returns 35 / (40 - 25) == 233%

well... I think that's the issue with > 100% utilization that I see with tests!! If sample 1 is delayed, then calculations are off.

Quoted 3 lines

how can we get this right if the value is not updated for running contexts?

I don't think we can. This is an inherent limitation of the HW counter that we are exposing. It is updated only when context switches happen and can be reliable only for frequently switching contexts. Tools would have to use some sort of averaging algorithm over a period of time to make sense of the data.

I think this sort of sampling affects any counters that we export that are updated infrequently. You will run into internediate values that are larger than 100%. Results may need to be averaged out.

Although, for this specific case, we should try to support getting c2 as well since that would be required for long-running contexts anyways. GuC had some level of support for this but with some limitations. I would have to dig that up and consult GuC if we can use/extend that.

I would probably not merge the IGTs, but would appreciate it if they can still be reviewed. I would still use the same tests once the above support is added.

Thanks,
Umesh

Quoted 78 lines

Lucas De Marchi

Quoted 75 lines

Case (2) is applicable for a completely different feature which is still wip - per engine utilization via kernel perf framework.

Quoted 10 lines

I think we need to add a third sample when testing trailing idle and ensure:

considering the indexing as sample[client][idx]

	sample[0][1] - sample[0][0] == 100% if BUSY else 0%
	sample[0][2] - sample[0][1] == 0%  -> idle on trailing test

and for isolation:

Quoted 7 lines
	sample[1][1] - sample[0][0] == 0%
	sample[1][2] - sample[1][1] == 0%

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 539

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +4 -0
+	if (flags & TEST_ISOLATION)
+		read_engine_cycles(new_fd, pceu2[1]);
+
+	check_results(pceu1[0], pceu2[0], hwe->engine_class, width, flags);

This check_results will check for TEST_BUSY case.

Quoted 3 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 542

Quoted 3 lines
tests/intel/xe_drm_fdinfo.c +3 -0
+
+	if (flags & TEST_ISOLATION) {
+		check_results(pceu1[1], pceu2[1], hwe->engine_class, width, 0);

This check_results will check for idle case (flags = 0).

Quoted 7 lines

tests/intel/xe_drm_fdinfo.c spin_ctx_destroy(int fd, struct spin_ctx *ctx) · line 547

Quoted 5 lines
tests/intel/xe_drm_fdinfo.c +5 -0
+		close(new_fd);
+	}
+
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);

if !BUSY you are not even calling spin_ctx_init()

:) yeah, sorry about the readability here. The spin_sync_end and spin_ctx_destroy bail out if the ctx is NULL which is the case for !BUSY. I can add and explicit check to not call these instead.

Thanks,
Umesh

Quoted 34 lines

Lucas De Marchi

Quoted 31 lines
tests/intel/xe_drm_fdinfo.c +3 -0
+	xe_vm_destroy(fd, vm);
+}
+

igt_main
{

tests/intel/xe_drm_fdinfo.c +1 -0
+	struct drm_xe_engine_class_instance *hwe;
	int xe;

	igt_fixture {
tests/intel/xe_drm_fdinfo.c +12 -0
@@ -484,6 +569,18 @@ igt_main
 	igt_subtest("basic-engine-utilization")
 		basic_engine_utilization(xe);
 
+	igt_subtest("drm-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, 0);
+
+	igt_subtest("drm-busy-idle")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-busy-idle-isolation")
+		xe_for_each_engine(xe, hwe)
+			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#23 lore raw

PATCH v2 6/10 tests/intel/xe_drm_fdinfo: Add tests to verify all class utilization

In reply to Umesh Nerlige Ramappa

Verify utilization for all classes with varying loads.

v2:
- Drop unused ISOLATION flag in some tests.
- s/measured_usleep/usleep
- remove xe_ prefix from helpers/structures
- Drop unused parameters to all_busy_check_all
- s/_class/class

---
 tests/intel/xe_drm_fdinfo.c | 130 ++++++++++++++++++++++++++++++++++++
 1 file changed, 130 insertions(+)
tests/intel/xe_drm_fdinfo.c +130 -0
index 410c885e7..f2051c422 100644
@@ -37,6 +37,15 @@
  * SUBTEST: drm-busy-idle-isolation
  * Description: Check that engine load does not spill over to other drm clients
  *
+ * SUBTEST: drm-busy-idle-check-all
+ * Description: Check that only the target engine shows load when idle after busy
+ *
+ * SUBTEST: drm-most-busy-idle-check-all
+ * Description: Check that only the target engine shows idle and all others are busy
+ *
+ * SUBTEST: drm-all-busy-idle-check-all
+ * Description: Check that all engines show busy when all are loaded
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -548,6 +557,116 @@ single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
 	xe_vm_destroy(fd, vm);
 }
 
+static void
+busy_check_all(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+	       unsigned int flags)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+	int class;
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		ctx = spin_ctx_init(fd, hwe, vm, width, count);
+		spin_sync_start(fd, ctx);
+	}
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE)
+		spin_sync_end(fd, ctx);
+	read_engine_cycles(fd, pceu2);
+
+	xe_for_each_engine_class(class)
+		check_results(pceu1, pceu2, class, width,
+			      hwe->engine_class == class ? flags : 0);
+
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);
+	xe_vm_destroy(fd, vm);
+}
+
+static void
+most_busy_check_all(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
+		    unsigned int flags)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx[DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
+	struct drm_xe_engine_class_instance *_hwe;
+	uint32_t vm;
+	int class;
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		/* spin on one hwe per class except the target class hwes */
+		xe_for_each_engine(fd, _hwe) {
+			int _class = _hwe->engine_class;
+
+			if (_class == hwe->engine_class || ctx[_class])
+				continue;
+
+			ctx[_class] = spin_ctx_init(fd, _hwe, vm, width, count);
+			spin_sync_start(fd, ctx[_class]);
+		}
+	}
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE)
+		xe_for_each_engine_class(class)
+			spin_sync_end(fd, ctx[class]);
+	read_engine_cycles(fd, pceu2);
+
+	xe_for_each_engine_class(class) {
+		check_results(pceu1, pceu2, class, width,
+			      hwe->engine_class == class ? 0 : flags);
+		spin_sync_end(fd, ctx[class]);
+		spin_ctx_destroy(fd, ctx[class]);
+	}
+	xe_vm_destroy(fd, vm);
+}
+
+static void
+all_busy_check_all(int fd, unsigned int flags)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx[DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
+	struct drm_xe_engine_class_instance *hwe;
+	uint32_t vm;
+	int class;
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		/* spin on one hwe per class */
+		xe_for_each_engine(fd, hwe) {
+			class = hwe->engine_class;
+
+			if (ctx[class])
+				continue;
+
+			ctx[class] = spin_ctx_init(fd, hwe, vm, 1, 1);
+			spin_sync_start(fd, ctx[class]);
+		}
+	}
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE)
+		xe_for_each_engine_class(class)
+			spin_sync_end(fd, ctx[class]);
+	read_engine_cycles(fd, pceu2);
+
+	xe_for_each_engine_class(class) {
+		check_results(pceu1, pceu2, class, 1, flags);
+		spin_sync_end(fd, ctx[class]);
+		spin_ctx_destroy(fd, ctx[class]);
+	}
+	xe_vm_destroy(fd, vm);
+}
 igt_main
 {
 	struct drm_xe_engine_class_instance *hwe;
@@ -581,6 +700,17 @@ igt_main
 		xe_for_each_engine(xe, hwe)
 			single(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
 
+	igt_subtest("drm-busy-idle-check-all")
+		xe_for_each_engine(xe, hwe)
+			busy_check_all(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-most-busy-idle-check-all")
+		xe_for_each_engine(xe, hwe)
+			most_busy_check_all(xe, hwe, 1, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-all-busy-idle-check-all")
+		all_busy_check_all(xe, TEST_BUSY | TEST_TRAILING_IDLE);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#24 lore raw

PATCH v2 7/10 tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

In reply to Umesh Nerlige Ramappa

Add a helper iterator for virtual engines.

---
 tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++
 1 file changed, 29 insertions(+)
tests/intel/xe_drm_fdinfo.c +29 -0
index f2051c422..9d3d1b285 100644
@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
 	0x1d0000,
 	0x1e0000,
 };
+
+#define MAX_GTS 2
+#define MAX_INSTANCE 9
+struct virtual_hwe {
+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
+	int count;
+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
+
+static void list_virtual_engines(int fd)
+{
+	struct drm_xe_engine_class_instance *hwe;
+
+	xe_for_each_engine(fd, hwe) {
+		struct virtual_hwe *v;
+
+		igt_assert(hwe->gt_id < MAX_GTS);
+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
+		v = &vhwe[hwe->gt_id][hwe->engine_class];
+
+		igt_assert(v->count < MAX_INSTANCE);
+		v->eci[v->count++] = *hwe;
+	}
+}
+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
+
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -678,6 +706,7 @@ igt_main
 		xe = drm_open_driver(DRIVER_XE);
 		igt_require_xe(xe);
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
+		list_virtual_engines(xe);
 	}
 
 	igt_describe("Check if basic fdinfo content is present for memory");
Signature · 1 line
2.38.1
#25 lore raw

Re: [PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

Quoted 12 lines · Umesh Nerlige Ramappa

Add a helper iterator for virtual engines.

---
tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++

1 file changed, 29 insertions(+)
tests/intel/xe_drm_fdinfo.c +0 -0
index f2051c422..9d3d1b285 100644

tests/intel/xe_drm_fdinfo.c static const uint64_t batch_addr[] = { · line 99

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +7 -0
@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
 	0x1d0000,
 	0x1e0000,
 };
+
+#define MAX_GTS 2
+#define MAX_INSTANCE 9
+struct virtual_hwe {
+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
+	int count;
+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};

there's no concept of "virtual engine". I think this is something brought over from i915? Is it related to the width when submitting?

Please add a comment for the confused reader and it's probably worth this abstraction to be outside this test. +Matt Brost

tests/intel/xe_drm_fdinfo.c static const uint64_t batch_addr[] = { · line 101

Quoted 2 lines
tests/intel/xe_drm_fdinfo.c +2 -0
+
+static void list_virtual_engines(int fd)

list? shouldn't it be virtual_engines_init()?

Lucas De Marchi

Quoted 33 lines
tests/intel/xe_drm_fdinfo.c +20 -0
+{
+	struct drm_xe_engine_class_instance *hwe;
+
+	xe_for_each_engine(fd, hwe) {
+		struct virtual_hwe *v;
+
+		igt_assert(hwe->gt_id < MAX_GTS);
+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
+		v = &vhwe[hwe->gt_id][hwe->engine_class];
+
+		igt_assert(v->count < MAX_INSTANCE);
+		v->eci[v->count++] = *hwe;
+	}
+}
+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
+
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -678,6 +706,7 @@ igt_main
 		xe = drm_open_driver(DRIVER_XE);
 		igt_require_xe(xe);
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
+		list_virtual_engines(xe);
 	}
 
 	igt_describe("Check if basic fdinfo content is present for memory");
Signature · 1 line
2.38.1
#26 lore raw

Re: [PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

Quoted 27 lines · Lucas De Marchi
Quoted 12 lines · Umesh Nerlige Ramappa

Add a helper iterator for virtual engines.

---
tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++

1 file changed, 29 insertions(+)
tests/intel/xe_drm_fdinfo.c +0 -0
index f2051c422..9d3d1b285 100644

tests/intel/xe_drm_fdinfo.c static const uint64_t batch_addr[] = { · line 99

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +7 -0
@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
 	0x1d0000,
 	0x1e0000,
 };
+
+#define MAX_GTS 2
+#define MAX_INSTANCE 9
+struct virtual_hwe {
+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
+	int count;
+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};

there's no concept of "virtual engine". I think this is something brought over from i915? Is it related to the width when submitting?

Quoted 2 lines

Please add a comment for the confused reader and it's probably worth this abstraction to be outside this test. +Matt Brost

width is for parallel submission and num_placements is for virtual.

Quoted 5 lines

tests/intel/xe_drm_fdinfo.c static const uint64_t batch_addr[] = { · line 101

Quoted 3 lines
tests/intel/xe_drm_fdinfo.c +2 -0
+
+static void list_virtual_engines(int fd)

list? shouldn't it be virtual_engines_init()?

Right, I will change that.

Thanks,
Umesh

Quoted 36 lines

Lucas De Marchi

Quoted 33 lines
tests/intel/xe_drm_fdinfo.c +20 -0
+{
+	struct drm_xe_engine_class_instance *hwe;
+
+	xe_for_each_engine(fd, hwe) {
+		struct virtual_hwe *v;
+
+		igt_assert(hwe->gt_id < MAX_GTS);
+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
+		v = &vhwe[hwe->gt_id][hwe->engine_class];
+
+		igt_assert(v->count < MAX_INSTANCE);
+		v->eci[v->count++] = *hwe;
+	}
+}
+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
+
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -678,6 +706,7 @@ igt_main
 		xe = drm_open_driver(DRIVER_XE);
 		igt_require_xe(xe);
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
+		list_virtual_engines(xe);
 	}
 
 	igt_describe("Check if basic fdinfo content is present for memory");
Signature · 1 line
2.38.1
#27 lore raw

Re: [PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

Quoted 28 lines · Umesh Nerlige Ramappa
Quoted 27 lines · Lucas De Marchi
Quoted 12 lines · Umesh Nerlige Ramappa

Add a helper iterator for virtual engines.

---
tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++

1 file changed, 29 insertions(+)
tests/intel/xe_drm_fdinfo.c +0 -0
index f2051c422..9d3d1b285 100644

tests/intel/xe_drm_fdinfo.c static const uint64_t batch_addr[] = { · line 99

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +7 -0
@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
 	0x1d0000,
 	0x1e0000,
 };
+
+#define MAX_GTS 2
+#define MAX_INSTANCE 9
+struct virtual_hwe {
+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
+	int count;
+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};

there's no concept of "virtual engine". I think this is something brought over from i915? Is it related to the width when submitting?

Quoted 4 lines
Quoted 2 lines

Please add a comment for the confused reader and it's probably worth this abstraction to be outside this test. +Matt Brost

width is for parallel submission and num_placements is for virtual.

ok... let's also s/virtual engines/virtual exec_queues/ where appropriate. Like we have in xe_exec_balancer.c

can we also use the helper in xe_exec_balancer?

Lucas De Marchi

Quoted 48 lines
Quoted 6 lines

tests/intel/xe_drm_fdinfo.c static const uint64_t batch_addr[] = { · line 101

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +2 -0
+
+static void list_virtual_engines(int fd)

list? shouldn't it be virtual_engines_init()?

Right, I will change that.

Thanks,
Umesh

Quoted 36 lines

Lucas De Marchi

Quoted 33 lines
tests/intel/xe_drm_fdinfo.c +20 -0
+{
+	struct drm_xe_engine_class_instance *hwe;
+
+	xe_for_each_engine(fd, hwe) {
+		struct virtual_hwe *v;
+
+		igt_assert(hwe->gt_id < MAX_GTS);
+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
+		v = &vhwe[hwe->gt_id][hwe->engine_class];
+
+		igt_assert(v->count < MAX_INSTANCE);
+		v->eci[v->count++] = *hwe;
+	}
+}
+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
+
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -678,6 +706,7 @@ igt_main
 		xe = drm_open_driver(DRIVER_XE);
 		igt_require_xe(xe);
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
+		list_virtual_engines(xe);
 	}
 
 	igt_describe("Check if basic fdinfo content is present for memory");
Signature · 1 line
2.38.1
#28 lore raw

Re: [PATCH i-g-t v2 07/10] tests/intel/xe_drm_fdinfo: Add an iterator for virtual engines

Quoted 32 lines · Lucas De Marchi
Quoted 31 lines · Umesh Nerlige Ramappa
Quoted 29 lines · Lucas De Marchi
Quoted 12 lines · Umesh Nerlige Ramappa

Add a helper iterator for virtual engines.

---
tests/intel/xe_drm_fdinfo.c | 29 +++++++++++++++++++++++++++++

1 file changed, 29 insertions(+)
tests/intel/xe_drm_fdinfo.c +0 -0
index f2051c422..9d3d1b285 100644

tests/intel/xe_drm_fdinfo.c static const uint64_t batch_addr[] = { · line 99

Quoted 10 lines
tests/intel/xe_drm_fdinfo.c +7 -0
@@ -90,6 +90,34 @@ static const uint64_t batch_addr[] = {
 	0x1d0000,
 	0x1e0000,
 };
+
+#define MAX_GTS 2
+#define MAX_INSTANCE 9
+struct virtual_hwe {
+	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
+	int count;
+} vhwe[MAX_GTS][DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};

there's no concept of "virtual engine". I think this is something brought over from i915? Is it related to the width when submitting?

Please add a comment for the confused reader and it's probably worth this abstraction to be outside this test. +Matt Brost

Quoted 6 lines
Quoted 1 line

width is for parallel submission and num_placements is for virtual.

ok... let's also s/virtual engines/virtual exec_queues/ where appropriate. Like we have in xe_exec_balancer.c

can we also use the helper in xe_exec_balancer?

I'm with Lucas that this looks overly complicated.

Perhaps this code from xe_exec_balancer.c can be moved to helper somewhere?

194 xe_for_each_engine(fd, hwe) {
195 if (hwe->engine_class != class || hwe->gt_id != gt)
196 continue;
197
198 eci[num_placements++] = *hwe;
199 }
200 if (num_placements < 2)
201 return;

This code collects the hwe instances for a class / gt + calculates the number of placements. Once you have that calling __xe_exec_queue_create is pretty simple.

So something like this:

int
xe_exec_queue_virtaul_setup(int fd, struct drm_xe_engine_class_instance *eci,
int class, int gt)

{
	struct drm_xe_engine_class_instance *hwe;
	int num_placements = 0;

        xe_for_each_engine(fd, hwe) {
                 if (hwe->engine_class != class || hwe->gt_id != gt)
                         continue;

                 eci[num_placements++] = *hwe;
        }

	return num_placements;
}

Then your test function is something like:

int foo_test(int fd, int class, int gt)

{
	struct drm_xe_engine_class_instance eci[MAX_INSTANCE];
	int num_placement;

	num_placement = xe_exec_queue_virtaul_setup(fd, eci, class, gt);
	if (num_placement < 2)
		return;

	/* Do test */
}

Then your main IGT loop is:

	igt_subtest("foo_test")
		xe_for_each_gt(fd, gt)
			xe_for_each_engine_class(class)
				foo_test(fd, class, gt);

This is the pattern in xe_exec_balancer.c. I don't think we should reinvent this (it is helpful if IGTs all look fairly similar so if you unstand one, you probably understand most).

Matt

Quoted 52 lines

Lucas De Marchi

Quoted 48 lines
Quoted 6 lines

tests/intel/xe_drm_fdinfo.c static const uint64_t batch_addr[] = { · line 101

Quoted 4 lines
tests/intel/xe_drm_fdinfo.c +2 -0
+
+static void list_virtual_engines(int fd)

list? shouldn't it be virtual_engines_init()?

Right, I will change that.

Thanks,
Umesh

Quoted 36 lines

Lucas De Marchi

Quoted 33 lines
tests/intel/xe_drm_fdinfo.c +20 -0
+{
+	struct drm_xe_engine_class_instance *hwe;
+
+	xe_for_each_engine(fd, hwe) {
+		struct virtual_hwe *v;
+
+		igt_assert(hwe->gt_id < MAX_GTS);
+		igt_assert(hwe->engine_class < DRM_XE_ENGINE_CLASS_COMPUTE + 1);
+		v = &vhwe[hwe->gt_id][hwe->engine_class];
+
+		igt_assert(v->count < MAX_INSTANCE);
+		v->eci[v->count++] = *hwe;
+	}
+}
+#define xe_for_each_multi_engine(__fd, __hwe, __count) \
+	for (int igt_unique(gt) = 0; igt_unique(gt) < MAX_GTS; igt_unique(gt)++) \
+		for (int igt_unique(c) = 0; igt_unique(c) < DRM_XE_ENGINE_CLASS_COMPUTE + 1; igt_unique(c)++) \
+			for_if((__hwe = &vhwe[igt_unique(gt)][igt_unique(c)].eci[0]) && ((__count = vhwe[igt_unique(gt)][igt_unique(c)].count) > 1))
+
 static void read_engine_cycles(int xe, struct pceu_cycles *pceu)
 {
 	struct drm_client_fdinfo info = { };
@@ -678,6 +706,7 @@ igt_main
 		xe = drm_open_driver(DRIVER_XE);
 		igt_require_xe(xe);
 		igt_require(igt_parse_drm_fdinfo(xe, &info, NULL, 0, NULL, 0));
+		list_virtual_engines(xe);
 	}
 
 	igt_describe("Check if basic fdinfo content is present for memory");
Signature · 1 line
2.38.1
#29 lore raw

PATCH v2 8/10 tests/intel/xe_drm_fdinfo: Add tests for virtual engines

In reply to Umesh Nerlige Ramappa

Add tests to verify utilization on virtual engines.

v2:
- s/measured_usleep/usleep
- Remove xe_ prefix from helpers/structures
- Remove ISOLATION case for multi_all_busy_check_all

---
 tests/intel/xe_drm_fdinfo.c | 71 ++++++++++++++++++++++++++++++++++++-
 1 file changed, 70 insertions(+), 1 deletion(-)
tests/intel/xe_drm_fdinfo.c +70 -1
index 9d3d1b285..5a3cd776c 100644
@@ -46,6 +46,18 @@
  * SUBTEST: drm-all-busy-idle-check-all
  * Description: Check that all engines show busy when all are loaded
  *
+ * SUBTEST: drm-virtual-idle
+ * Description: Check that virtual engines show no load when idle
+ *
+ * SUBTEST: drm-virtual-busy-idle
+ * Description: Check that virtual engines show load when idle after busy
+ *
+ * SUBTEST: drm-virtual-all-busy-idle-check-all
+ * Description: Check that all virtual engines show busy when loaded
+ *
+ * SUBTEST: drm-virtual-busy-idle-isolation
+ * Description: Check that virtual engine load does not spill over to other drm clients
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -695,10 +707,52 @@ all_busy_check_all(int fd, unsigned int flags)
 	}
 	xe_vm_destroy(fd, vm);
 }
+
+static void
+multi_all_busy_check_all(int fd, unsigned int flags)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx[DRM_XE_ENGINE_CLASS_COMPUTE + 1] = {};
+	struct drm_xe_engine_class_instance *hwe;
+	int count, class;
+	uint32_t vm;
+
+	vm = xe_vm_create(fd, 0, 0);
+	if (flags & TEST_BUSY) {
+		xe_for_each_multi_engine(fd, hwe, count) {
+			class = hwe->engine_class;
+			ctx[class] = spin_ctx_init(fd, hwe, vm,
+						      1,
+						      count);
+			spin_sync_start(fd, ctx[class]);
+		}
+	}
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+	if (flags & TEST_TRAILING_IDLE) {
+		xe_for_each_multi_engine(fd, hwe, count) {
+			spin_sync_end(fd, ctx[hwe->engine_class]);
+		}
+	}
+	read_engine_cycles(fd, pceu2);
+
+	xe_for_each_multi_engine(fd, hwe, count) {
+		class = hwe->engine_class;
+		check_results(pceu1, pceu2, class,
+			      1,
+			      flags);
+		spin_sync_end(fd, ctx[class]);
+		spin_ctx_destroy(fd, ctx[class]);
+	}
+	xe_vm_destroy(fd, vm);
+}
+
 igt_main
 {
 	struct drm_xe_engine_class_instance *hwe;
-	int xe;
+	int xe, count;
 
 	igt_fixture {
 		struct drm_client_fdinfo info = { };
@@ -740,6 +794,21 @@ igt_main
 	igt_subtest("drm-all-busy-idle-check-all")
 		all_busy_check_all(xe, TEST_BUSY | TEST_TRAILING_IDLE);
 
+	igt_subtest("drm-virtual-idle")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, 1, count, 0);
+
+	igt_subtest("drm-virtual-busy-idle")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, 1, count, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-virtual-all-busy-idle-check-all")
+		multi_all_busy_check_all(xe, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-virtual-busy-idle-isolation")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, 1, count, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#30 lore raw

PATCH v2 9/10 tests/intel/xe_drm_fdinfo: Add tests for parallel engines

In reply to Umesh Nerlige Ramappa

Add tests to verify utilization on parallel engines.

v2:
- s/measured_usleep/usleep
- remove xe_ prefix from helpers/structures

---
 tests/intel/xe_drm_fdinfo.c | 34 +++++++++++++++++++++++++++++++---
 1 file changed, 31 insertions(+), 3 deletions(-)
tests/intel/xe_drm_fdinfo.c +31 -3
index 5a3cd776c..f1e06991c 100644
@@ -58,6 +58,18 @@
  * SUBTEST: drm-virtual-busy-idle-isolation
  * Description: Check that virtual engine load does not spill over to other drm clients
  *
+ * SUBTEST: drm-parallel-idle
+ * Description: Check that parallel engines show no load when idle
+ *
+ * SUBTEST: drm-parallel-busy-idle
+ * Description: Check that parallel engines show load when idle after busy
+ *
+ * SUBTEST: drm-parallel-all-busy-idle-check-all
+ * Description: Check that all parallel engines show busy when loaded
+ *
+ * SUBTEST: drm-parallel-busy-idle-isolation
+ * Description: Check that parallel engine load does not spill over to other drm clients
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -76,6 +88,7 @@ IGT_TEST_DESCRIPTION("Read and verify drm client memory consumption and engine u
 #define TEST_BUSY		(1 << 0)
 #define TEST_TRAILING_IDLE	(1 << 1)
 #define TEST_ISOLATION		(1 << 2)
+#define EXEC_PARALLEL		(1 << 16)
 
 struct pceu_cycles {
 	uint64_t cycles;
@@ -723,8 +736,8 @@ multi_all_busy_check_all(int fd, unsigned int flags)
 		xe_for_each_multi_engine(fd, hwe, count) {
 			class = hwe->engine_class;
 			ctx[class] = spin_ctx_init(fd, hwe, vm,
-						      1,
-						      count);
+						   flags & EXEC_PARALLEL ? count : 1,
+						   flags & EXEC_PARALLEL ? 1 : count);
 			spin_sync_start(fd, ctx[class]);
 		}
 	}
@@ -741,7 +754,7 @@ multi_all_busy_check_all(int fd, unsigned int flags)
 	xe_for_each_multi_engine(fd, hwe, count) {
 		class = hwe->engine_class;
 		check_results(pceu1, pceu2, class,
-			      1,
+			      flags & EXEC_PARALLEL ? count : 1,
 			      flags);
 		spin_sync_end(fd, ctx[class]);
 		spin_ctx_destroy(fd, ctx[class]);
@@ -809,6 +822,21 @@ igt_main
 		xe_for_each_multi_engine(xe, hwe, count)
 			single(xe, hwe, 1, count, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
 
+	igt_subtest("drm-parallel-idle")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, count, 1, 0);
+
+	igt_subtest("drm-parallel-busy-idle")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, count, 1, TEST_BUSY | TEST_TRAILING_IDLE);
+
+	igt_subtest("drm-parallel-all-busy-idle-check-all")
+		multi_all_busy_check_all(xe, TEST_BUSY | TEST_TRAILING_IDLE | EXEC_PARALLEL);
+
+	igt_subtest("drm-parallel-busy-idle-isolation")
+		xe_for_each_multi_engine(xe, hwe, count)
+			single(xe, hwe, count, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#31 lore raw

PATCH v2 10/10 tests/intel/xe_drm_fdinfo: Ensure queue destroy records load correctly

In reply to Umesh Nerlige Ramappa

If the exec queue is destroyed before recording the utilization, make sure the load is correctly captured.

v2:
- s/measured_usleep/usleep
- remove xe_ prefix from helpers/structures

---
 tests/intel/xe_drm_fdinfo.c | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
tests/intel/xe_drm_fdinfo.c +33 -0
index f1e06991c..b3c865757 100644
@@ -70,6 +70,9 @@
  * SUBTEST: drm-parallel-busy-idle-isolation
  * Description: Check that parallel engine load does not spill over to other drm clients
  *
+ * SUBTEST: drm-busy-exec-queue-destroy-idle
+ * Description: Destroy exec queue before idle and ensure load is accurate
+ *
  * SUBTEST: drm-total-resident
  * Description: Create and compare total and resident memory consumption by client
  *
@@ -567,6 +570,32 @@ check_results(struct pceu_cycles *s1, struct pceu_cycles *s2,
 		igt_assert(!percent);
 }
 
+static void
+single_destroy_queue(int fd, struct drm_xe_engine_class_instance *hwe)
+{
+	struct pceu_cycles pceu1[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct pceu_cycles pceu2[DRM_XE_ENGINE_CLASS_COMPUTE + 1];
+	struct spin_ctx *ctx = NULL;
+	uint32_t vm;
+
+	vm = xe_vm_create(fd, 0, 0);
+	ctx = spin_ctx_init(fd, hwe, vm, 1, 1);
+	spin_sync_start(fd, ctx);
+
+	read_engine_cycles(fd, pceu1);
+	usleep(batch_duration_ns / 1000);
+
+	/* destroy queue before sampling again */
+	spin_sync_end(fd, ctx);
+	spin_ctx_destroy(fd, ctx);
+
+	read_engine_cycles(fd, pceu2);
+
+	xe_vm_destroy(fd, vm);
+
+	check_results(pceu1, pceu2, hwe->engine_class, 1, TEST_BUSY);
+}
+
 static void
 single(int fd, struct drm_xe_engine_class_instance *hwe, int width, int count,
        unsigned int flags)
@@ -837,6 +866,10 @@ igt_main
 		xe_for_each_multi_engine(xe, hwe, count)
 			single(xe, hwe, count, 1, TEST_BUSY | TEST_TRAILING_IDLE | TEST_ISOLATION);
 
+	igt_subtest("drm-busy-exec-queue-destroy-idle")
+		xe_for_each_engine(xe, hwe)
+			single_destroy_queue(xe, hwe);
+
 	igt_describe("Create and compare total and resident memory consumption by client");
 	igt_subtest("drm-total-resident")
 		test_total_resident(xe);
Signature · 1 line
2.38.1
#32 lore raw

✗ Fi.CI.BUILD: failure for Add per-client engine utilization tests (rev3)

In reply to Umesh Nerlige Ramappa
== Series Details ==

== Series Details ==

Series: Add per-client engine utilization tests (rev3)
URL : https://patchwork.freedesktop.org/series/135204/
State : failure

== Summary ==

IGT patchset build failed on latest successful build d0d71f374c95a89a3bdcd104c7d8c2043f79e37a tests/intel/xe_intel_bb: Reduce render subtest execution time

Tail of build.log:

#33 lore raw

✓ CI.xeBAT: success for Add per-client engine utilization tests (rev4)

In reply to Umesh Nerlige Ramappa
== Series Details ==

== Series Details ==

Series: Add per-client engine utilization tests (rev4)
URL : https://patchwork.freedesktop.org/series/135204/
State : success

== Summary ==

CI Bug Log - changes from XEIGT_7916_BAT -> XEIGTPW_11367_BAT
====================================================

Summary
-------

**SUCCESS**

No regressions found.

Participating hosts (5 -> 5)
------------------------------

No changes in participating hosts

Known issues
------------

Here are the changes found in XEIGTPW_11367_BAT that come from known issues:

### IGT changes ###

#### Possible fixes ####

* igt@xe_gt_freq@freq_basic_api:
- bat-adlp-7: [FAIL][1] -> [PASS][2]
[1]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/bat-adlp-7/igt@xe_gt_freq@freq_basic_api.html
[2]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/bat-adlp-7/igt@xe_gt_freq@freq_basic_api.html

Build changes
-------------

* IGT: IGT_7916 -> IGTPW_11367
* Linux: xe-1562-2bf4c5214910654a6a50082c90f0f4596ea6b0c6 -> xe-1565-55e18f9be7b75c5226a57eff4a3b3eb6110815c6

IGTPW_11367: 11367
IGT_7916: 971cfc9d6afc8242db25d1eabd4ae00890d9144a @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
xe-1562-2bf4c5214910654a6a50082c90f0f4596ea6b0c6: 2bf4c5214910654a6a50082c90f0f4596ea6b0c6 xe-1565-55e18f9be7b75c5226a57eff4a3b3eb6110815c6: 55e18f9be7b75c5226a57eff4a3b3eb6110815c6

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/index.html

#34 lore raw

✗ Fi.CI.BAT: failure for Add per-client engine utilization tests (rev4)

In reply to Umesh Nerlige Ramappa
== Series Details ==

== Series Details ==

Series: Add per-client engine utilization tests (rev4)
URL : https://patchwork.freedesktop.org/series/135204/
State : failure

== Summary ==

CI Bug Log - changes from CI_DRM_15034 -> IGTPW_11367
====================================================

Summary
-------

**FAILURE**

Serious unknown changes coming with IGTPW_11367 absolutely need to be verified manually.

If you think the reported changes have nothing to do with the changes
introduced in IGTPW_11367, please notify your bug team ([email protected]) to allow them to document this new failure mode, which will reduce false positives in CI.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/index.html

Participating hosts (42 -> 36)
------------------------------

Missing (6): fi-kbl-7567u bat-dg1-7 fi-snb-2520m fi-elk-e7500 bat-jsl-3 bat-arlh-2

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in IGTPW_11367:

### IGT changes ###

#### Possible regressions ####

* igt@i915_selftest@live@gt_lrc:
- fi-kbl-x1275: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/fi-kbl-x1275/igt@i915_selftest@live@gt_lrc.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/fi-kbl-x1275/igt@i915_selftest@live@gt_lrc.html

#### Suppressed ####

The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.

* igt@i915_selftest@live@gt_lrc:
- {bat-twl-2}: [PASS][3] -> [INCOMPLETE][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/bat-twl-2/igt@i915_selftest@live@gt_lrc.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/bat-twl-2/igt@i915_selftest@live@gt_lrc.html

Known issues
------------

Here are the changes found in IGTPW_11367 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@i915_selftest@live@execlists:
- fi-bsw-n3050: [PASS][5] -> [ABORT][6] ([i915#10594])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/fi-bsw-n3050/igt@i915_selftest@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/fi-bsw-n3050/igt@i915_selftest@[email protected]

* igt@i915_selftest@live@workarounds:
- bat-adlp-6: [PASS][7] -> [INCOMPLETE][8] ([i915#9413])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/bat-adlp-6/igt@i915_selftest@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/bat-adlp-6/igt@i915_selftest@[email protected]

* igt@kms_pipe_crc_basic@hang-read-crc@pipe-b-dp-1:
- bat-dg2-8: [PASS][9] -> [FAIL][10] ([i915#11379])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/bat-dg2-8/igt@kms_pipe_crc_basic@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/bat-dg2-8/igt@kms_pipe_crc_basic@[email protected]

#### Possible fixes ####

* igt@kms_frontbuffer_tracking@basic:
- bat-arls-2: [DMESG-WARN][11] ([i915#7507]) -> [PASS][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_15034/bat-arls-2/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/bat-arls-2/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[i915#10594]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/10594
[i915#11379]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/11379
[i915#180]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/180
[i915#1982]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/1982
[i915#7507]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/7507
[i915#9413]: https://gitlab.freedesktop.org/drm/i915/kernel/-/issues/9413

Build changes
-------------

* CI: CI-20190529 -> None
* IGT: IGT_7916 -> IGTPW_11367

CI-20190529: 20190529
CI_DRM_15034: 55e18f9be7b75c5226a57eff4a3b3eb6110815c6 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_11367: 11367
IGT_7916: 971cfc9d6afc8242db25d1eabd4ae00890d9144a @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_11367/index.html

#35 lore raw

✓ CI.xeFULL: success for Add per-client engine utilization tests (rev4)

In reply to Umesh Nerlige Ramappa
== Series Details ==

== Series Details ==

Series: Add per-client engine utilization tests (rev4)
URL : https://patchwork.freedesktop.org/series/135204/
State : success

== Summary ==

CI Bug Log - changes from XEIGT_7916_full -> XEIGTPW_11367_full
====================================================

Summary
-------

**SUCCESS**

No regressions found.

Participating hosts (3 -> 3)
------------------------------

No changes in participating hosts

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in XEIGTPW_11367_full:

### IGT changes ###

#### Possible regressions ####

* {igt@kms_getfb} (NEW):
- {shard-lnl}: NOTRUN -> [INCOMPLETE][1]
[1]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-1/igt@kms_getfb.html

* {igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle} (NEW):
- shard-dg2-set2: NOTRUN -> [FAIL][2] +1 other test fail
[2]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@[email protected]

* {igt@xe_drm_fdinfo@drm-most-busy-idle-check-all} (NEW):
- {shard-lnl}: NOTRUN -> [FAIL][3]
[3]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-8/igt@[email protected]

#### Suppressed ####

The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.

* igt@kms_chamelium_frames@hdmi-cmp-planes-random:
- {shard-lnl}: [SKIP][4] ([Intel XE#373]) -> [INCOMPLETE][5]
[4]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-5/igt@[email protected]
[5]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-7/igt@[email protected]

New tests
---------

New tests have been introduced between XEIGT_7916_full and XEIGTPW_11367_full:

### New IGT tests (18) ###

* igt@kms_getfb:
- Statuses : 1 incomplete(s)
- Exec time: [0.0] s

* igt@xe_drm_fdinfo@basic-engine-utilization:
- Statuses : 2 pass(s)
- Exec time: [0.0] s

* igt@xe_drm_fdinfo@basic-memory:
- Statuses : 2 pass(s)
- Exec time: [0.0] s

* igt@xe_drm_fdinfo@drm-all-busy-idle-check-all:
- Statuses : 2 pass(s)
- Exec time: [0.51] s

* igt@xe_drm_fdinfo@drm-busy-exec-queue-destroy-idle:
- Statuses : 1 fail(s)
- Exec time: [0.53] s

* igt@xe_drm_fdinfo@drm-busy-idle:
- Statuses : 2 pass(s)
- Exec time: [2.54, 3.54] s

* igt@xe_drm_fdinfo@drm-busy-idle-check-all:
- Statuses : 2 pass(s)
- Exec time: [2.55, 3.55] s

* igt@xe_drm_fdinfo@drm-busy-idle-isolation:
- Statuses : 1 pass(s)
- Exec time: [3.54] s

* igt@xe_drm_fdinfo@drm-idle:
- Statuses : 2 pass(s)
- Exec time: [2.51, 3.52] s

* igt@xe_drm_fdinfo@drm-most-busy-idle-check-all:
- Statuses : 2 fail(s)
- Exec time: [1.04, 1.56] s

* igt@xe_drm_fdinfo@drm-parallel-all-busy-idle-check-all:
- Statuses : 1 pass(s)
- Exec time: [0.51] s

* igt@xe_drm_fdinfo@drm-parallel-busy-idle:
- Statuses : 2 pass(s)
- Exec time: [0.0, 1.01] s

* igt@xe_drm_fdinfo@drm-parallel-busy-idle-isolation:
- Statuses : 1 pass(s)
- Exec time: [1.01] s

* igt@xe_drm_fdinfo@drm-parallel-idle:
- Statuses : 2 pass(s)
- Exec time: [0.0, 1.00] s

* igt@xe_drm_fdinfo@drm-virtual-all-busy-idle-check-all:
- Statuses : 2 pass(s)
- Exec time: [0.50, 0.51] s

* igt@xe_drm_fdinfo@drm-virtual-busy-idle:
- Statuses : 2 pass(s)
- Exec time: [0.0, 1.01] s

* igt@xe_drm_fdinfo@drm-virtual-busy-idle-isolation:
- Statuses : 2 pass(s)
- Exec time: [0.0, 1.01] s

* igt@xe_drm_fdinfo@drm-virtual-idle:
- Statuses : 2 pass(s)
- Exec time: [0.0, 1.00] s

Known issues
------------

Here are the changes found in XEIGTPW_11367_full that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@core_setmaster@master-drop-set-shared-fd:
- shard-dg2-set2: [PASS][6] -> [DMESG-WARN][7] ([Intel XE#1162] / [Intel XE#1214])
[6]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-434/igt@[email protected]
[7]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@[email protected]

* igt@kms_async_flips@async-flip-with-page-flip-events:
- shard-dg2-set2: [PASS][8] -> [INCOMPLETE][9] ([Intel XE#1150] / [Intel XE#1195])
[8]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-466/igt@[email protected]
[9]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@[email protected]

* igt@kms_async_flips@async-flip-with-page-flip-events@pipe-d-hdmi-a-6-x:
- shard-dg2-set2: [PASS][10] -> [INCOMPLETE][11] ([Intel XE#1195]) +2 other tests incomplete
[10]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-466/igt@kms_async_flips@[email protected]
[11]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@kms_async_flips@[email protected]

* igt@kms_big_fb@linear-8bpp-rotate-270:
- shard-dg2-set2: NOTRUN -> [SKIP][12] ([Intel XE#1201] / [Intel XE#316])
[12]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@[email protected]

* igt@kms_big_fb@yf-tiled-64bpp-rotate-90:
- shard-dg2-set2: NOTRUN -> [SKIP][13] ([Intel XE#1124] / [Intel XE#1201]) +2 other tests skip
[13]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-466/igt@[email protected]

* igt@kms_ccs@ccs-on-another-bo-y-tiled-gen12-rc-ccs:
- shard-dg2-set2: NOTRUN -> [SKIP][14] ([Intel XE#1201] / [Intel XE#455] / [Intel XE#787]) +9 other tests skip
[14]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@[email protected]

* igt@kms_ccs@crc-sprite-planes-basic-y-tiled-gen12-mc-ccs@pipe-b-hdmi-a-6:
- shard-dg2-set2: NOTRUN -> [SKIP][15] ([Intel XE#1201] / [Intel XE#787]) +34 other tests skip
[15]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@kms_ccs@crc-sprite-planes-basic-y-tiled-gen12-mc-ccs@pipe-b-hdmi-a-6.html

* igt@kms_chamelium_color@ctm-0-50:
- shard-dg2-set2: NOTRUN -> [SKIP][16] ([Intel XE#1201] / [Intel XE#306])
[16]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-434/igt@[email protected]

* igt@kms_chamelium_hpd@hdmi-hpd-fast:
- shard-dg2-set2: NOTRUN -> [SKIP][17] ([Intel XE#1201] / [Intel XE#373])
[17]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@[email protected]

* igt@kms_cursor_crc@cursor-offscreen-512x512:
- shard-dg2-set2: NOTRUN -> [SKIP][18] ([Intel XE#1201] / [Intel XE#308]) +1 other test skip
[18]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-434/igt@[email protected]

* igt@kms_cursor_legacy@torture-move:
- shard-dg2-set2: [PASS][19] -> [DMESG-WARN][20] ([Intel XE#1214] / [Intel XE#877]) +1 other test dmesg-warn
[19]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-466/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@[email protected]

* igt@kms_frontbuffer_tracking@drrs-2p-primscrn-cur-indfb-draw-mmap-wc:
- shard-dg2-set2: NOTRUN -> [SKIP][21] ([Intel XE#1201] / [Intel XE#651]) +11 other tests skip
[21]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@kms_frontbuffer_tracking@drrs-2p-primscrn-cur-indfb-draw-mmap-wc.html

* igt@kms_frontbuffer_tracking@psr-2p-primscrn-spr-indfb-draw-mmap-wc:
- shard-dg2-set2: NOTRUN -> [SKIP][22] ([Intel XE#1201] / [Intel XE#653]) +9 other tests skip
[22]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_frontbuffer_tracking@psr-2p-primscrn-spr-indfb-draw-mmap-wc.html

* igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25:
- shard-dg2-set2: NOTRUN -> [SKIP][23] ([Intel XE#1201] / [Intel XE#305] / [Intel XE#455]) +1 other test skip
[23]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25.html

* igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25@pipe-c-hdmi-a-6:
- shard-dg2-set2: NOTRUN -> [SKIP][24] ([Intel XE#1201] / [Intel XE#305]) +2 other tests skip
[24]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@kms_plane_scaling@planes-downscale-factor-0-25-upscale-factor-0-25@pipe-c-hdmi-a-6.html

* igt@kms_pm_backlight@bad-brightness:
- shard-dg2-set2: NOTRUN -> [SKIP][25] ([Intel XE#1201] / [Intel XE#870])
[25]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@[email protected]

* igt@kms_psr2_sf@overlay-plane-update-continuous-sf:
- shard-dg2-set2: NOTRUN -> [SKIP][26] ([Intel XE#1201] / [Intel XE#1489])
[26]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-466/igt@[email protected]

* igt@kms_psr@fbc-psr2-primary-blt:
- shard-dg2-set2: NOTRUN -> [SKIP][27] ([Intel XE#1201] / [Intel XE#929]) +3 other tests skip
[27]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@[email protected]

* igt@kms_rotation_crc@primary-rotation-270:
- shard-dg2-set2: NOTRUN -> [SKIP][28] ([Intel XE#1201] / [Intel XE#327])
[28]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@[email protected]

* igt@kms_setmode@basic-clone-single-crtc:
- shard-dg2-set2: NOTRUN -> [SKIP][29] ([Intel XE#1201] / [Intel XE#455]) +1 other test skip
[29]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@[email protected]

* igt@kms_writeback@writeback-fb-id-xrgb2101010:
- shard-dg2-set2: NOTRUN -> [SKIP][30] ([Intel XE#1201] / [Intel XE#756])
[30]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@[email protected]

* igt@xe_evict@evict-cm-threads-large:
- shard-dg2-set2: [PASS][31] -> [TIMEOUT][32] ([Intel XE#1473] / [Intel XE#392])
[31]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@[email protected]
[32]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-433/igt@[email protected]

* igt@xe_evict@evict-threads-large:
- shard-dg2-set2: [PASS][33] -> [INCOMPLETE][34] ([Intel XE#1195] / [Intel XE#1473] / [Intel XE#392]) +1 other test incomplete
[33]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-434/igt@[email protected]
[34]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@[email protected]

* igt@xe_exec_fault_mode@twice-bindexecqueue-rebind-imm:
- shard-dg2-set2: NOTRUN -> [SKIP][35] ([Intel XE#1201] / [Intel XE#288]) +8 other tests skip
[35]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@[email protected]

* igt@xe_live_ktest@xe_migrate:
- shard-dg2-set2: [PASS][36] -> [SKIP][37] ([Intel XE#1192] / [Intel XE#1201])
[36]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@xe_live_ktest@xe_migrate.html
[37]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-466/igt@xe_live_ktest@xe_migrate.html

* igt@xe_pm@s3-vm-bind-unbind-all:
- shard-dg2-set2: [PASS][38] -> [DMESG-WARN][39] ([Intel XE#1162] / [Intel XE#1214] / [Intel XE#1941])
[38]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-464/igt@[email protected]
[39]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@[email protected]

* igt@xe_pm@s3-vm-bind-userptr:
- shard-dg2-set2: [PASS][40] -> [DMESG-WARN][41] ([Intel XE#1214] / [Intel XE#1551] / [Intel XE#569])
[40]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-435/igt@[email protected]
[41]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@[email protected]

* igt@xe_vm@mixed-userptr-binds-1611661312:
- shard-dg2-set2: [PASS][42] -> [DMESG-WARN][43] ([Intel XE#1214]) +4 other tests dmesg-warn
[42]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-436/igt@[email protected]
[43]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@[email protected]

#### Possible fixes ####

* igt@kms_big_fb@4-tiled-max-hw-stride-64bpp-rotate-180:
- {shard-lnl}: [FAIL][44] ([Intel XE#1659]) -> [PASS][45]
[44]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-8/igt@[email protected]
[45]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-2/igt@[email protected]

* igt@kms_pipe_crc_basic@suspend-read-crc:
- shard-dg2-set2: [DMESG-WARN][46] ([Intel XE#1162] / [Intel XE#1214]) -> [PASS][47]
[46]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-434/igt@[email protected]
[47]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@[email protected]

* {igt@kms_plane@plane-position-covered@pipe-a-plane-1}:
- {shard-lnl}: [DMESG-FAIL][48] ([Intel XE#324]) -> [PASS][49] +1 other test pass
[48]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-5/igt@kms_plane@[email protected]
[49]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-2/igt@kms_plane@[email protected]

* igt@kms_universal_plane@cursor-fb-leak:
- shard-dg2-set2: [FAIL][50] ([Intel XE#771] / [Intel XE#899]) -> [PASS][51]
[50]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-433/igt@[email protected]
[51]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@[email protected]

* igt@kms_universal_plane@cursor-fb-leak@pipe-c-hdmi-a-6:
- shard-dg2-set2: [FAIL][52] ([Intel XE#899]) -> [PASS][53] +1 other test pass
[52]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-433/igt@kms_universal_plane@[email protected]
[53]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_universal_plane@[email protected]

* igt@xe_evict@evict-beng-threads-large:
- shard-dg2-set2: [TIMEOUT][54] ([Intel XE#1473]) -> [PASS][55]
[54]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-464/igt@[email protected]
[55]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-436/igt@[email protected]

* igt@xe_evict@evict-mixed-many-threads-small:
- shard-dg2-set2: [INCOMPLETE][56] ([Intel XE#1195] / [Intel XE#1473]) -> [PASS][57]
[56]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-466/igt@[email protected]
[57]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@[email protected]

* igt@xe_exec_basic@many-execqueues-bindexecqueue-rebind:
- {shard-lnl}: [FAIL][58] -> [PASS][59]
[58]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-5/igt@[email protected]
[59]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-4/igt@[email protected]

* igt@xe_live_ktest@xe_bo:
- {shard-lnl}: [SKIP][60] ([Intel XE#1192]) -> [PASS][61]
[60]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-lnl-4/igt@xe_live_ktest@xe_bo.html
[61]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-lnl-6/igt@xe_live_ktest@xe_bo.html

* igt@xe_module_load@reload:
- shard-dg2-set2: [DMESG-WARN][62] ([Intel XE#1214]) -> [PASS][63] +7 other tests pass
[62]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@[email protected]
[63]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-435/igt@[email protected]

* igt@xe_pm@s2idle-vm-bind-userptr:
- shard-dg2-set2: [INCOMPLETE][64] ([Intel XE#1195] / [Intel XE#1694]) -> [PASS][65]
[64]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-435/igt@[email protected]
[65]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@[email protected]

* igt@xe_pm@s3-vm-bind-prefetch:
- shard-dg2-set2: [DMESG-WARN][66] ([Intel XE#1214] / [Intel XE#569]) -> [PASS][67]
[66]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-434/igt@[email protected]
[67]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-434/igt@[email protected]

#### Warnings ####

* igt@kms_content_protection@lic-type-0:
- shard-dg2-set2: [INCOMPLETE][68] ([Intel XE#1195]) -> [FAIL][69] ([Intel XE#1204]) +1 other test fail
[68]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-435/igt@[email protected]
[69]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@[email protected]

* igt@kms_plane@plane-panning-bottom-right-suspend@pipe-b:
- shard-dg2-set2: [DMESG-FAIL][70] ([Intel XE#1551]) -> [FAIL][71] ([Intel XE#616]) +1 other test fail
[70]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-433/igt@kms_plane@[email protected]
[71]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-463/igt@kms_plane@[email protected]

* igt@xe_evict@evict-mixed-threads-large:
- shard-dg2-set2: [TIMEOUT][72] ([Intel XE#1473] / [Intel XE#392]) -> [DMESG-FAIL][73] ([Intel XE#482])
[72]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@[email protected]
[73]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-466/igt@[email protected]

* igt@xe_live_ktest@xe_mocs:
- shard-dg2-set2: [FAIL][74] ([Intel XE#1999]) -> [SKIP][75] ([Intel XE#1192] / [Intel XE#1201])
[74]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-463/igt@xe_live_ktest@xe_mocs.html
[75]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@xe_live_ktest@xe_mocs.html

* igt@xe_wedged@wedged-at-any-timeout:
- shard-dg2-set2: [DMESG-WARN][76] ([Intel XE#1214] / [Intel XE#1760]) -> [DMESG-FAIL][77] ([Intel XE#1760])
[76]: https://intel-gfx-ci.01.org/tree/intel-xe/IGT_7916/shard-dg2-433/igt@[email protected]
[77]: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/shard-dg2-464/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[Intel XE#1124]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1124
[Intel XE#1127]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1127
[Intel XE#1128]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1128
[Intel XE#1150]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1150
[Intel XE#1162]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1162
[Intel XE#1192]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1192
[Intel XE#1195]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1195
[Intel XE#1201]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1201
[Intel XE#1204]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1204
[Intel XE#1214]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1214
[Intel XE#1392]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1392
[Intel XE#1397]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1397
[Intel XE#1399]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1399
[Intel XE#1401]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1401
[Intel XE#1406]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1406
[Intel XE#1407]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1407
[Intel XE#1413]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1413
[Intel XE#1421]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1421
[Intel XE#1424]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1424
[Intel XE#1430]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1430
[Intel XE#1435]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1435
[Intel XE#1437]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1437
[Intel XE#1442]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1442
[Intel XE#1447]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1447
[Intel XE#1473]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1473
[Intel XE#1489]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1489
[Intel XE#1551]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1551
[Intel XE#1659]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1659
[Intel XE#1694]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1694
[Intel XE#1745]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1745
[Intel XE#1760]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1760
[Intel XE#1941]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1941
[Intel XE#1948]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1948
[Intel XE#1999]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/1999
[Intel XE#2191]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/2191
[Intel XE#288]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/288
[Intel XE#305]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/305
[Intel XE#306]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/306
[Intel XE#307]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/307
[Intel XE#308]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/308
[Intel XE#309]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/309
[Intel XE#316]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/316
[Intel XE#323]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/323
[Intel XE#324]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/324
[Intel XE#327]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/327
[Intel XE#330]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/330
[Intel XE#346]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/346
[Intel XE#352]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/352
[Intel XE#366]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/366
[Intel XE#367]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/367
[Intel XE#373]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/373
[Intel XE#392]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/392
[Intel XE#455]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/455
[Intel XE#482]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/482
[Intel XE#498]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/498
[Intel XE#569]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/569
[Intel XE#584]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/584
[Intel XE#599]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/599
[Intel XE#616]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/616
[Intel XE#651]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/651
[Intel XE#653]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/653
[Intel XE#656]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/656
[Intel XE#688]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/688
[Intel XE#756]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/756
[Intel XE#771]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/771
[Intel XE#787]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/787
[Intel XE#870]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/870
[Intel XE#877]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/877
[Intel XE#886]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/886
[Intel XE#899]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/899
[Intel XE#929]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/929
[Intel XE#944]: https://gitlab.freedesktop.org/drm/xe/kernel/issues/944

Build changes
-------------

* IGT: IGT_7916 -> IGTPW_11367
* Linux: xe-1562-2bf4c5214910654a6a50082c90f0f4596ea6b0c6 -> xe-1565-55e18f9be7b75c5226a57eff4a3b3eb6110815c6

IGTPW_11367: 11367
IGT_7916: 971cfc9d6afc8242db25d1eabd4ae00890d9144a @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
xe-1562-2bf4c5214910654a6a50082c90f0f4596ea6b0c6: 2bf4c5214910654a6a50082c90f0f4596ea6b0c6 xe-1565-55e18f9be7b75c5226a57eff4a3b3eb6110815c6: 55e18f9be7b75c5226a57eff4a3b3eb6110815c6

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_11367/index.html