Skip to contentSkip to conversation

Dynamic subtests, v3

PATCH Superseded Petri Latvala 5 participants 30 messages Oct 24, 2019 – Nov 14, 2019

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Dynamic subtests, v3

Rebased and renamed.

For examples of the usage of dynamic subtests, please see the latter 3 patches in the series. The changes to gem_exec_basic are especially simple.

Petri Latvala (8):
lib: Introduce dynamic subsubtests
lib/tests: Unit tests for dynamic subtests lib/tests: Test that igt_describe works with dynamic subtests
runner: Parse dynamic subtest outputs and results runner/json_tests: Test dynamic subtests
kms_plane_cursor: Use dynamic subtests
perf_pmu: Use dynamic subtests
i915/gem_exec_basic: Use dynamic subtests

 lib/igt_core.c                                | 148 ++++++-
 lib/igt_core.h                                | 123 ++++++
 lib/tests/igt_describe.c                      |  30 +-
 lib/tests/igt_dynamic_subtests.c              | 186 +++++++++
 lib/tests/meson.build                         |   1 +
 runner/job_list.c                             |  11 +
 runner/job_list.h                             |   3 +
 .../dynamic-subtests/0/dmesg.txt              |   7 +
 .../dynamic-subtests/0/err.txt                |  36 ++
 .../dynamic-subtests/0/journal.txt            |   2 +
 .../dynamic-subtests/0/out.txt                |  19 +
 .../dynamic-subtests/1/dmesg.txt              |   5 +
 .../dynamic-subtests/1/err.txt                |   2 +
 .../dynamic-subtests/1/journal.txt            |   2 +
 .../dynamic-subtests/1/out.txt                |   5 +
 .../dynamic-subtests/2/dmesg.txt              |  11 +
 .../dynamic-subtests/2/err.txt                |   8 +
 .../dynamic-subtests/2/journal.txt            |   4 +
 .../dynamic-subtests/2/out.txt                |  10 +
 .../dynamic-subtests/README.txt               |   2 +
 .../dynamic-subtests/endtime.txt              |   1 +
 .../dynamic-subtests/joblist.txt              |   3 +
 .../dynamic-subtests/metadata.txt             |  12 +
 .../dynamic-subtests/reference.json           | 168 ++++++++
 .../dynamic-subtests/starttime.txt            |   1 +
 .../dynamic-subtests/uname.txt                |   1 +
 runner/output_strings.h                       |  29 +-
 runner/resultgen.c                            | 377 ++++++++++++++----
 runner/runner_json_tests.c                    |   3 +-
 tests/i915/gem_exec_basic.c                   |  26 +-
 tests/kms_plane_cursor.c                      |  34 +-
 tests/perf_pmu.c                              | 244 +++++-------
 32 files changed, 1255 insertions(+), 259 deletions(-)
 create mode 100644 lib/tests/igt_dynamic_subtests.c
 create mode 100644 runner/json_tests_data/dynamic-subtests/0/dmesg.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/0/err.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/0/journal.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/0/out.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/1/dmesg.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/1/err.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/1/journal.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/1/out.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/2/dmesg.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/2/err.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/2/journal.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/2/out.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/README.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/endtime.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/joblist.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/metadata.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/reference.json
 create mode 100644 runner/json_tests_data/dynamic-subtests/starttime.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/uname.txt
Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#1 lore raw

PATCH 1/8 lib: Introduce dynamic subsubtests

Dynamic subsubtests, or subtests of subtests, are individual pieces of tests that are not statically available all the time.

A good example of a need for a dynamic subsubtest is i915 engine
listing: A normal subtest for each engine class ("bsd"), and a dynamic subsubtest for each instance ("bsd0", "bsd2", etc). Or a normal subtest for an operation with a dynamic subsubtest for every engine there is.

Another example is dynamic subsubtests for pipes: Instead of using foreach_pipe_static, make one subtest and use foreach_pipe with dynamic subsubtests for each pipe.

v2: Rebase and adapt to igt_describe changes

v3: Rename to igt_subtest_with_dynamic_subsubtests &
igt_dynamic_subsubtest, better docs, make igt_describe fail loudly if it's used in an impossible context.

---
 lib/igt_core.c | 148 ++++++++++++++++++++++++++++++++++++++++++-------
 lib/igt_core.h | 123 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 250 insertions(+), 21 deletions(-)
lib/igt_core.c +127 -21
index 7bd5afa5..9c5c698f 100644
@@ -263,9 +263,12 @@ bool igt_skip_crc_compare;
 static bool list_subtests = false;
 static bool describe_subtests = false;
 static char *run_single_subtest = NULL;
+static char *run_single_dynamic_subtest = NULL;
 static bool run_single_subtest_found = false;
 static const char *in_subtest = NULL;
+static const char *in_dynamic_subtest = NULL;
 static struct timespec subtest_time;
+static struct timespec dynamic_subtest_time;
 static clockid_t igt_clock = (clockid_t)-1;
 static bool in_fixture = false;
 static bool test_with_subtests = false;
@@ -300,6 +303,7 @@ enum {
 	OPT_LIST_SUBTESTS = 500,
 	OPT_DESCRIBE_SUBTESTS,
 	OPT_RUN_SUBTEST,
+	OPT_RUN_DYNAMIC_SUBTEST,
 	OPT_DESCRIPTION,
 	OPT_DEBUG,
 	OPT_INTERACTIVE_DEBUG,
@@ -323,6 +327,8 @@ char *igt_frame_dump_path;
 
 static bool stderr_needs_sentinel = false;
 
+static int _igt_dynamic_tests_executed = -1;
+
 const char *igt_test_name(void)
 {
 	return command_str;
@@ -354,7 +360,9 @@ static void _igt_log_buffer_dump(void)
 {
 	uint8_t i;
 
-	if (in_subtest)
+	if (in_dynamic_subtest)
+		fprintf(stderr, "Dynamic subtest %s failed.\n", in_dynamic_subtest);
+	else if (in_subtest)
 		fprintf(stderr, "Subtest %s failed.\n", in_subtest);
 	else
 		fprintf(stderr, "Test %s failed.\n", command_str);
@@ -639,6 +647,7 @@ static void print_usage(const char *help_str, bool output_on_stderr)
 	fprintf(f, "Usage: %s [OPTIONS]\n", command_str);
 	fprintf(f, "  --list-subtests\n"
 		   "  --run-subtest <pattern>\n"
+		   "  --dynamic-subtest <pattern>\n"
 		   "  --debug[=log-domain]\n"
 		   "  --interactive-debug[=domain]\n"
 		   "  --skip-crc-compare\n"
@@ -781,6 +790,7 @@ static int common_init(int *argc, char **argv,
 		{"list-subtests",     no_argument,       NULL, OPT_LIST_SUBTESTS},
 		{"describe",          optional_argument, NULL, OPT_DESCRIBE_SUBTESTS},
 		{"run-subtest",       required_argument, NULL, OPT_RUN_SUBTEST},
+		{"dynamic-subtest",   required_argument, NULL, OPT_RUN_DYNAMIC_SUBTEST},
 		{"help-description",  no_argument,       NULL, OPT_DESCRIPTION},
 		{"debug",             optional_argument, NULL, OPT_DEBUG},
 		{"interactive-debug", optional_argument, NULL, OPT_INTERACTIVE_DEBUG},
@@ -900,6 +910,11 @@ static int common_init(int *argc, char **argv,
 			if (!list_subtests)
 				run_single_subtest = strdup(optarg);
 			break;
+		case OPT_RUN_DYNAMIC_SUBTEST:
+			assert(optarg);
+			if (!list_subtests)
+				run_single_dynamic_subtest = strdup(optarg);
+			break;
 		case OPT_DESCRIPTION:
 			print_test_description();
 			ret = -1;
@@ -1084,6 +1099,19 @@ static void __igt_print_description(const char *subtest_name, const char *file,
 		printf("%sNO DOCUMENTATION!\n\n", indent);
 }
 
+static bool valid_name_for_subtest(const char *subtest_name)
+{
+	int i;
+
+	/* check the subtest name only contains a-z, A-Z, 0-9, '-' and '_' */
+	for (i = 0; subtest_name[i] != '\0'; i++)
+		if (subtest_name[i] != '_' && subtest_name[i] != '-'
+		    && !isalnum(subtest_name[i]))
+			return false;
+
+	return true;
+}
+
 /*
  * Note: Testcases which use these helpers MUST NOT output anything to stdout
  * outside of places protected by igt_run_subtest checks - the piglit
@@ -1091,18 +1119,13 @@ static void __igt_print_description(const char *subtest_name, const char *file,
  */
 bool __igt_run_subtest(const char *subtest_name, const char *file, const int line)
 {
-	int i;
-
 	assert(!igt_can_fail());
 
-	/* check the subtest name only contains a-z, A-Z, 0-9, '-' and '_' */
-	for (i = 0; subtest_name[i] != '\0'; i++)
-		if (subtest_name[i] != '_' && subtest_name[i] != '-'
-		    && !isalnum(subtest_name[i])) {
-			igt_critical("Invalid subtest name \"%s\".\n",
-				     subtest_name);
-			igt_exit();
-		}
+	if (!valid_name_for_subtest(subtest_name)) {
+		igt_critical("Invalid subtest name \"%s\".\n",
+			     subtest_name);
+		igt_exit();
+	}
 
 	if (run_single_subtest) {
 		if (uwildmat(subtest_name, run_single_subtest) == 0) {
@@ -1149,6 +1172,36 @@ bool __igt_run_subtest(const char *subtest_name, const char *file, const int lin
 	return (in_subtest = subtest_name);
 }
 
+bool __igt_run_dynamic_subtest(const char *dynamic_subtest_name)
+{
+	assert(in_subtest);
+	assert(_igt_dynamic_tests_executed >= 0);
+
+	if (!valid_name_for_subtest(dynamic_subtest_name)) {
+			igt_critical("Invalid dynamic subtest name \"%s\".\n",
+				     dynamic_subtest_name);
+			igt_exit();
+	}
+
+	if (run_single_dynamic_subtest &&
+	    uwildmat(dynamic_subtest_name, run_single_dynamic_subtest) == 0)
+		return false;
+
+	igt_kmsg(KMSG_INFO "%s: starting dynamic subtest %s\n",
+		 command_str, dynamic_subtest_name);
+	igt_info("Starting dynamic subtest: %s\n", dynamic_subtest_name);
+	fflush(stdout);
+	if (stderr_needs_sentinel)
+		fprintf(stderr, "Starting dynamic subtest: %s\n", dynamic_subtest_name);
+
+	_igt_log_buffer_reset();
+
+	_igt_dynamic_tests_executed++;
+
+	igt_gettime(&dynamic_subtest_time);
+	return (in_dynamic_subtest = dynamic_subtest_name);
+}
+
 /**
  * igt_subtest_name:
  *
@@ -1203,26 +1256,53 @@ void __igt_subtest_group_restore(int save, int desc)
 static bool skipped_one = false;
 static bool succeeded_one = false;
 static bool failed_one = false;
+static bool dynamic_failed_one = false;
+
+bool __igt_enter_dynamic_container(void)
+{
+	_igt_dynamic_tests_executed = 0;
+	dynamic_failed_one = false;
+
+	return true;
+}
 
 static void exit_subtest(const char *) __attribute__((noreturn));
 static void exit_subtest(const char *result)
 {
 	struct timespec now;
+	const char *subtest_text = in_dynamic_subtest ? "Dynamic subtest" : "Subtest";
+	const char **subtest_name = in_dynamic_subtest ? &in_dynamic_subtest : &in_subtest;
+	struct timespec *thentime = in_dynamic_subtest ? &dynamic_subtest_time : &subtest_time;
+	jmp_buf *jmptarget = in_dynamic_subtest ? &igt_dynamic_subsubtest_jmpbuf : &igt_subtest_jmpbuf;
 
 	igt_gettime(&now);
-	igt_info("%sSubtest %s: %s (%.3fs)%s\n",
+
+	igt_info("%s%s %s: %s (%.3fs)%s\n",
 		 (!__igt_plain_output) ? "\x1b[1m" : "",
-		 in_subtest, result, igt_time_elapsed(&subtest_time, &now),
+		 subtest_text, *subtest_name, result,
+		 igt_time_elapsed(thentime, &now),
 		 (!__igt_plain_output) ? "\x1b[0m" : "");
 	fflush(stdout);
 	if (stderr_needs_sentinel)
-		fprintf(stderr, "Subtest %s: %s (%.3fs)\n",
-			in_subtest, result, igt_time_elapsed(&subtest_time, &now));
+		fprintf(stderr, "%s %s: %s (%.3fs)\n",
+			subtest_text, *subtest_name,
+			result, igt_time_elapsed(thentime, &now));
 
 	igt_terminate_spins();
 
-	in_subtest = NULL;
-	siglongjmp(igt_subtest_jmpbuf, 1);
+	if (!in_dynamic_subtest)
+		_igt_dynamic_tests_executed = -1;
+
+	/*
+	 * Don't keep the above text in the log if exiting a dynamic
+	 * subsubtest, the subtest would print it again otherwise
+	 */
+	if (in_dynamic_subtest)
+		_igt_log_buffer_reset();
+
+	*subtest_name = NULL;
+
+	siglongjmp(*jmptarget, 1);
 }
 
 /**
@@ -1253,6 +1333,7 @@ void igt_skip(const char *f, ...)
 	}
 
 	if (in_subtest) {
+		/* Doing the same even if inside a dynamic subtest */
 		exit_subtest("SKIP");
 	} else if (test_with_subtests) {
 		skip_subtests_henceforth = SKIP;
@@ -1309,7 +1390,22 @@ void __igt_skip_check(const char *file, const int line,
  */
 void igt_success(void)
 {
-	succeeded_one = true;
+	if (in_subtest && !in_dynamic_subtest && _igt_dynamic_tests_executed >= 0) {
+		/*
+		 * We're exiting a dynamic container, yield a result
+		 * according to the dynamic tests that got
+		 * executed.
+		 */
+		if (dynamic_failed_one)
+			igt_fail(IGT_EXIT_FAILURE);
+
+		if (_igt_dynamic_tests_executed == 0)
+			igt_skip("No dynamic tests executed.\n");
+	}
+
+	if (!in_dynamic_subtest)
+		succeeded_one = true;
+
 	if (in_subtest)
 		exit_subtest("SUCCESS");
 }
@@ -1340,10 +1436,17 @@ void igt_fail(int exitcode)
 	if (in_atexit_handler)
 		_exit(IGT_EXIT_FAILURE);
 
-	if (!failed_one)
-		igt_exitcode = exitcode;
+	if (in_dynamic_subtest) {
+		dynamic_failed_one = true;
+	} else {
+		/* Dynamic subtest containers must not fail explicitly */
+		assert(_igt_dynamic_tests_executed < 0 || dynamic_failed_one);
 
-	failed_one = true;
+		if (!failed_one)
+			igt_exitcode = exitcode;
+
+		failed_one = true;
+	}
 
 	/* Silent exit, parent will do the yelling. */
 	if (test_child)
@@ -1419,6 +1522,9 @@ void igt_describe_f(const char *fmt, ...)
 	int ret;
 	va_list args;
 
+	if (in_subtest && _igt_dynamic_tests_executed >= 0)
+		assert(!"Documenting dynamic subsubtests is impossible. Document the subtest instead.");
+
 	if (!describe_subtests)
 		return;
 
lib/igt_core.h +123 -0

141 lines not on this page · raw message

Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Reviewed-by on v1 1/8
#2 lore raw

Re: [igt-dev] [PATCH i-g-t 1/8] lib: Introduce dynamic subsubtests

lib/igt_core.c

Quoted 29 lines · Petri Latvala

Dynamic subsubtests, or subtests of subtests, are individual pieces of tests that are not statically available all the time.

A good example of a need for a dynamic subsubtest is i915 engine
listing: A normal subtest for each engine class ("bsd"), and a dynamic subsubtest for each instance ("bsd0", "bsd2", etc). Or a normal subtest for an operation with a dynamic subsubtest for every engine there is.

Another example is dynamic subsubtests for pipes: Instead of using foreach_pipe_static, make one subtest and use foreach_pipe with dynamic subsubtests for each pipe.

v2: Rebase and adapt to igt_describe changes

v3: Rename to igt_subtest_with_dynamic_subsubtests &
igt_dynamic_subsubtest, better docs, make igt_describe fail loudly if it's used in an impossible context.

---
 lib/igt_core.c | 148 ++++++++++++++++++++++++++++++++++++++++++-------
 lib/igt_core.h | 123 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 250 insertions(+), 21 deletions(-)
lib/igt_core.c +0 -0
index 7bd5afa5..9c5c698f 100644

The section documentation in this file mentions magic control blocks giving a cautionary tale about how they are implemented and what not to do:

"This applies to #igt_fixture and #igt_subtest blocks and all the..."

https://drm.pages.freedesktop.org/igt-gpu-tools/igt-gpu-tools-Core.html#id-1.2.7.6.6

We can mention the new dynamic subsubtests by name there too.

With that:

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#3 lore raw

PATCH v4 1/8 lib: Introduce dynamic subsubtests

In reply to Petri Latvala

Dynamic subsubtests, or subtests of subtests, are individual pieces of tests that are not statically available all the time.

A good example of a need for a dynamic subsubtest is i915 engine
listing: A normal subtest for each engine class ("bsd"), and a dynamic subsubtest for each instance ("bsd0", "bsd2", etc). Or a normal subtest for an operation with a dynamic subsubtest for every engine there is.

Another example is dynamic subsubtests for pipes: Instead of using foreach_pipe_static, make one subtest and use foreach_pipe with dynamic subsubtests for each pipe.

v2: Rebase and adapt to igt_describe changes

v3: Rename to igt_subtest_with_dynamic_subsubtests &
igt_dynamic_subsubtest, better docs, make igt_describe fail loudly if it's used in an impossible context.

v4: Mention dynamic blocks in the warning for longjmp in core docs.

---
 lib/igt_core.c | 162 ++++++++++++++++++++++++++++++++++++++++---------
 lib/igt_core.h | 123 +++++++++++++++++++++++++++++++++++++
 2 files changed, 258 insertions(+), 27 deletions(-)
lib/igt_core.c +135 -27
index 7bd5afa5..1e1b8aa1 100644
@@ -125,12 +125,14 @@
  *   to interacting with the kernel driver must be reinitialized to avoid such
  *   issues.
  *
- * - Code blocks with magic control flow are implemented with setjmp() and
- *   longjmp(). This applies to #igt_fixture and #igt_subtest blocks and all the
- *   three variants to finish test: igt_success(), igt_skip() and igt_fail().
- *   Mostly this is of no concern, except when such a control block changes
- *   stack variables defined in the same function as the control block resides.
- *   Any store/load behaviour after a longjmp() is ill-defined for these
+ * - Code blocks with magic control flow are implemented with setjmp()
+ *   and longjmp(). This applies to #igt_fixture, #igt_subtest,
+ *   #igt_subtest_with_dynamic_subsubtests and #igt_dynamic_subsubtest
+ *   blocks and all the three variants to finish test: igt_success(),
+ *   igt_skip() and igt_fail(). Mostly this is of no concern, except
+ *   when such a control block changes stack variables defined in the
+ *   same function as the control block resides.  Any store/load
+ *   behaviour after a longjmp() is ill-defined for these
  *   variables. Avoid such code.
  *
  *   Quoting the man page for longjmp():
@@ -263,9 +265,12 @@ bool igt_skip_crc_compare;
 static bool list_subtests = false;
 static bool describe_subtests = false;
 static char *run_single_subtest = NULL;
+static char *run_single_dynamic_subtest = NULL;
 static bool run_single_subtest_found = false;
 static const char *in_subtest = NULL;
+static const char *in_dynamic_subtest = NULL;
 static struct timespec subtest_time;
+static struct timespec dynamic_subtest_time;
 static clockid_t igt_clock = (clockid_t)-1;
 static bool in_fixture = false;
 static bool test_with_subtests = false;
@@ -300,6 +305,7 @@ enum {
 	OPT_LIST_SUBTESTS = 500,
 	OPT_DESCRIBE_SUBTESTS,
 	OPT_RUN_SUBTEST,
+	OPT_RUN_DYNAMIC_SUBTEST,
 	OPT_DESCRIPTION,
 	OPT_DEBUG,
 	OPT_INTERACTIVE_DEBUG,
@@ -323,6 +329,8 @@ char *igt_frame_dump_path;
 
 static bool stderr_needs_sentinel = false;
 
+static int _igt_dynamic_tests_executed = -1;
+
 const char *igt_test_name(void)
 {
 	return command_str;
@@ -354,7 +362,9 @@ static void _igt_log_buffer_dump(void)
 {
 	uint8_t i;
 
-	if (in_subtest)
+	if (in_dynamic_subtest)
+		fprintf(stderr, "Dynamic subtest %s failed.\n", in_dynamic_subtest);
+	else if (in_subtest)
 		fprintf(stderr, "Subtest %s failed.\n", in_subtest);
 	else
 		fprintf(stderr, "Test %s failed.\n", command_str);
@@ -639,6 +649,7 @@ static void print_usage(const char *help_str, bool output_on_stderr)
 	fprintf(f, "Usage: %s [OPTIONS]\n", command_str);
 	fprintf(f, "  --list-subtests\n"
 		   "  --run-subtest <pattern>\n"
+		   "  --dynamic-subtest <pattern>\n"
 		   "  --debug[=log-domain]\n"
 		   "  --interactive-debug[=domain]\n"
 		   "  --skip-crc-compare\n"
@@ -781,6 +792,7 @@ static int common_init(int *argc, char **argv,
 		{"list-subtests",     no_argument,       NULL, OPT_LIST_SUBTESTS},
 		{"describe",          optional_argument, NULL, OPT_DESCRIBE_SUBTESTS},
 		{"run-subtest",       required_argument, NULL, OPT_RUN_SUBTEST},
+		{"dynamic-subtest",   required_argument, NULL, OPT_RUN_DYNAMIC_SUBTEST},
 		{"help-description",  no_argument,       NULL, OPT_DESCRIPTION},
 		{"debug",             optional_argument, NULL, OPT_DEBUG},
 		{"interactive-debug", optional_argument, NULL, OPT_INTERACTIVE_DEBUG},
@@ -900,6 +912,11 @@ static int common_init(int *argc, char **argv,
 			if (!list_subtests)
 				run_single_subtest = strdup(optarg);
 			break;
+		case OPT_RUN_DYNAMIC_SUBTEST:
+			assert(optarg);
+			if (!list_subtests)
+				run_single_dynamic_subtest = strdup(optarg);
+			break;
 		case OPT_DESCRIPTION:
 			print_test_description();
 			ret = -1;
@@ -1084,6 +1101,19 @@ static void __igt_print_description(const char *subtest_name, const char *file,
 		printf("%sNO DOCUMENTATION!\n\n", indent);
 }
 
+static bool valid_name_for_subtest(const char *subtest_name)
+{
+	int i;
+
+	/* check the subtest name only contains a-z, A-Z, 0-9, '-' and '_' */
+	for (i = 0; subtest_name[i] != '\0'; i++)
+		if (subtest_name[i] != '_' && subtest_name[i] != '-'
+		    && !isalnum(subtest_name[i]))
+			return false;
+
+	return true;
+}
+
 /*
  * Note: Testcases which use these helpers MUST NOT output anything to stdout
  * outside of places protected by igt_run_subtest checks - the piglit
@@ -1091,18 +1121,13 @@ static void __igt_print_description(const char *subtest_name, const char *file,
  */
 bool __igt_run_subtest(const char *subtest_name, const char *file, const int line)
 {
-	int i;
-
 	assert(!igt_can_fail());
 
-	/* check the subtest name only contains a-z, A-Z, 0-9, '-' and '_' */
-	for (i = 0; subtest_name[i] != '\0'; i++)
-		if (subtest_name[i] != '_' && subtest_name[i] != '-'
-		    && !isalnum(subtest_name[i])) {
-			igt_critical("Invalid subtest name \"%s\".\n",
-				     subtest_name);
-			igt_exit();
-		}
+	if (!valid_name_for_subtest(subtest_name)) {
+		igt_critical("Invalid subtest name \"%s\".\n",
+			     subtest_name);
+		igt_exit();
+	}
 
 	if (run_single_subtest) {
 		if (uwildmat(subtest_name, run_single_subtest) == 0) {
@@ -1149,6 +1174,36 @@ bool __igt_run_subtest(const char *subtest_name, const char *file, const int lin
 	return (in_subtest = subtest_name);
 }
 
+bool __igt_run_dynamic_subtest(const char *dynamic_subtest_name)
+{
+	assert(in_subtest);
+	assert(_igt_dynamic_tests_executed >= 0);
+
+	if (!valid_name_for_subtest(dynamic_subtest_name)) {
+			igt_critical("Invalid dynamic subtest name \"%s\".\n",
+				     dynamic_subtest_name);
+			igt_exit();
+	}
+
+	if (run_single_dynamic_subtest &&
+	    uwildmat(dynamic_subtest_name, run_single_dynamic_subtest) == 0)
+		return false;
+
+	igt_kmsg(KMSG_INFO "%s: starting dynamic subtest %s\n",
+		 command_str, dynamic_subtest_name);
+	igt_info("Starting dynamic subtest: %s\n", dynamic_subtest_name);
+	fflush(stdout);
+	if (stderr_needs_sentinel)
+		fprintf(stderr, "Starting dynamic subtest: %s\n", dynamic_subtest_name);
+
+	_igt_log_buffer_reset();
+
+	_igt_dynamic_tests_executed++;
+
+	igt_gettime(&dynamic_subtest_time);
+	return (in_dynamic_subtest = dynamic_subtest_name);
+}
+
 /**
  * igt_subtest_name:
  *
@@ -1203,26 +1258,53 @@ void __igt_subtest_group_restore(int save, int desc)
 static bool skipped_one = false;
 static bool succeeded_one = false;
 static bool failed_one = false;
+static bool dynamic_failed_one = false;
+
+bool __igt_enter_dynamic_container(void)
+{
+	_igt_dynamic_tests_executed = 0;
+	dynamic_failed_one = false;
+
+	return true;
+}
 
 static void exit_subtest(const char *) __attribute__((noreturn));
 static void exit_subtest(const char *result)
 {
 	struct timespec now;
+	const char *subtest_text = in_dynamic_subtest ? "Dynamic subtest" : "Subtest";
+	const char **subtest_name = in_dynamic_subtest ? &in_dynamic_subtest : &in_subtest;
+	struct timespec *thentime = in_dynamic_subtest ? &dynamic_subtest_time : &subtest_time;
+	jmp_buf *jmptarget = in_dynamic_subtest ? &igt_dynamic_subsubtest_jmpbuf : &igt_subtest_jmpbuf;
 
 	igt_gettime(&now);
-	igt_info("%sSubtest %s: %s (%.3fs)%s\n",
+
+	igt_info("%s%s %s: %s (%.3fs)%s\n",
 		 (!__igt_plain_output) ? "\x1b[1m" : "",
-		 in_subtest, result, igt_time_elapsed(&subtest_time, &now),
+		 subtest_text, *subtest_name, result,
+		 igt_time_elapsed(thentime, &now),
 		 (!__igt_plain_output) ? "\x1b[0m" : "");
 	fflush(stdout);
 	if (stderr_needs_sentinel)
-		fprintf(stderr, "Subtest %s: %s (%.3fs)\n",
-			in_subtest, result, igt_time_elapsed(&subtest_time, &now));
+		fprintf(stderr, "%s %s: %s (%.3fs)\n",
+			subtest_text, *subtest_name,
+			result, igt_time_elapsed(thentime, &now));
 
 	igt_terminate_spins();
 
-	in_subtest = NULL;
-	siglongjmp(igt_subtest_jmpbuf, 1);
+	if (!in_dynamic_subtest)
+		_igt_dynamic_tests_executed = -1;
+
+	/*
+	 * Don't keep the above text in the log if exiting a dynamic
+	 * subsubtest, the subtest would print it again otherwise
+	 */
+	if (in_dynamic_subtest)
+		_igt_log_buffer_reset();
+
+	*subtest_name = NULL;
+
+	siglongjmp(*jmptarget, 1);
 }
 
 /**
@@ -1253,6 +1335,7 @@ void igt_skip(const char *f, ...)
 	}
 
 	if (in_subtest) {
+		/* Doing the same even if inside a dynamic subtest */
 		exit_subtest("SKIP");
 	} else if (test_with_subtests) {
 		skip_subtests_henceforth = SKIP;
@@ -1309,7 +1392,22 @@ void __igt_skip_check(const char *file, const int line,
  */
 void igt_success(void)
 {
-	succeeded_one = true;
+	if (in_subtest && !in_dynamic_subtest && _igt_dynamic_tests_executed >= 0) {
+		/*
+		 * We're exiting a dynamic container, yield a result
+		 * according to the dynamic tests that got
+		 * executed.
+		 */
+		if (dynamic_failed_one)
+			igt_fail(IGT_EXIT_FAILURE);
+
+		if (_igt_dynamic_tests_executed == 0)
+			igt_skip("No dynamic tests executed.\n");
+	}
+
+	if (!in_dynamic_subtest)
+		succeeded_one = true;
+
 	if (in_subtest)
 		exit_subtest("SUCCESS");
 }
@@ -1340,10 +1438,17 @@ void igt_fail(int exitcode)
 	if (in_atexit_handler)
 		_exit(IGT_EXIT_FAILURE);
 
-	if (!failed_one)
-		igt_exitcode = exitcode;
+	if (in_dynamic_subtest) {
+		dynamic_failed_one = true;
+	} else {
+		/* Dynamic subtest containers must not fail explicitly */
+		assert(_igt_dynamic_tests_executed < 0 || dynamic_failed_one);
 
-	failed_one = true;
+		if (!failed_one)
+			igt_exitcode = exitcode;
+
+		failed_one = true;
+	}
 
 	/* Silent exit, parent will do the yelling. */
 	if (test_child)
@@ -1419,6 +1524,9 @@ void igt_describe_f(const char *fmt, ...)
 	int ret;
 	va_list args;
 
+	if (in_subtest && _igt_dynamic_tests_executed >= 0)
+		assert(!"Documenting dynamic subsubtests is impossible. Document the subtest instead.");
+
 	if (!describe_subtests)
 		return;
 
lib/igt_core.h +123 -0

141 lines not on this page · raw message

Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#4 lore raw

PATCH 2/8 lib/tests: Unit tests for dynamic subtests

In reply to Petri Latvala
---
 lib/tests/igt_dynamic_subtests.c | 186 +++++++++++++++++++++++++++++++
 lib/tests/meson.build            |   1 +
 2 files changed, 187 insertions(+)
 create mode 100644 lib/tests/igt_dynamic_subtests.c
lib/tests/igt_dynamic_subtests.c new file +186 -0
new file mode 100644
index 00000000..82d80a77
@@ -0,0 +1,186 @@
+/*
+ * Copyright © 2019 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ */
+
+#include <errno.h>
+#include <sys/wait.h>
+
+#include "igt_core.h"
+#include "drmtest.h"
+
+#include "igt_tests_common.h"
+
+static int do_fork(void (*test_to_run)(void))
+{
+	int pid, status;
+
+	switch (pid = fork()) {
+	case -1:
+		internal_assert(0);
+	case 0:
+		test_to_run();
+	default:
+		while (waitpid(pid, &status, 0) == -1 &&
+		       errno == EINTR)
+			;
+
+		return status;
+	}
+}
+
+static void dynamic_subtest_in_normal_subtest(void)
+{
+	char prog[] = "igt_no_exit";
+	char *fake_argv[] = {prog};
+	int fake_argc = ARRAY_SIZE(fake_argv);
+
+	igt_subtest_init(fake_argc, fake_argv);
+
+	igt_subtest("normal-subtest") {
+		igt_dynamic_subsubtest("dynamic") {
+			igt_info("Dynamic subtest in normal subtest\n");
+		}
+	}
+
+	igt_exit();
+}
+
+static void invalid_dynamic_subtest_name(void)
+{
+	char prog[] = "igt_no_exit";
+	char *fake_argv[] = {prog};
+	int fake_argc = ARRAY_SIZE(fake_argv);
+
+	igt_subtest_init(fake_argc, fake_argv);
+
+	igt_subtest_with_dynamic_subsubtests("subtest") {
+		igt_dynamic_subsubtest("# invalid name !") {
+			igt_info("Invalid dynamic subtest name test\n");
+		}
+	}
+
+	igt_exit();
+}
+
+static void dynamic_subtest_in_toplevel(void)
+{
+	char prog[] = "igt_no_exit";
+	char *fake_argv[] = {prog};
+	int fake_argc = ARRAY_SIZE(fake_argv);
+
+	igt_subtest_init(fake_argc, fake_argv);
+
+	igt_dynamic_subsubtest("dynamic-subtest-in-toplevel") {
+		igt_info("Dynamic subtests need to be in a subtest\n");
+	}
+
+	igt_exit();
+}
+
+static void subtest_itself_failing(void)
+{
+	char prog[] = "igt_no_exit";
+	char *fake_argv[] = {prog};
+	int fake_argc = ARRAY_SIZE(fake_argv);
+
+	igt_subtest_init(fake_argc, fake_argv);
+
+	igt_subtest_with_dynamic_subsubtests("subtest") {
+		igt_assert(false);
+	}
+
+	igt_exit();
+}
+
+static void subtest_itself_skipping(void)
+{
+	char prog[] = "igt_no_exit";
+	char *fake_argv[] = {prog};
+	int fake_argc = ARRAY_SIZE(fake_argv);
+
+	igt_subtest_init(fake_argc, fake_argv);
+
+	igt_subtest_with_dynamic_subsubtests("subtest") {
+		igt_skip("Skipping\n");
+	}
+
+	igt_exit();
+}
+
+static void dynamic_subtest_failure_leads_to_fail(void)
+{
+	char prog[] = "igt_no_exit";
+	char *fake_argv[] = {prog};
+	int fake_argc = ARRAY_SIZE(fake_argv);
+
+	igt_subtest_init(fake_argc, fake_argv);
+
+	igt_subtest_with_dynamic_subsubtests("subtest") {
+		igt_dynamic_subsubtest("dynamic") {
+			igt_assert(false);
+		}
+	}
+
+	igt_exit();
+}
+
+static void no_dynamic_subtests_entered_leads_to_skip(void)
+{
+	char prog[] = "igt_no_exit";
+	char *fake_argv[] = {prog};
+	int fake_argc = ARRAY_SIZE(fake_argv);
+
+	igt_subtest_init(fake_argc, fake_argv);
+
+	igt_subtest_with_dynamic_subsubtests("subtest") {
+	}
+
+	igt_exit();
+}
+
+int main(int argc, char **argv)
+{
+	int ret;
+
+	ret = do_fork(dynamic_subtest_in_normal_subtest);
+	internal_assert_wsignaled(ret, SIGABRT);
+
+	ret = do_fork(invalid_dynamic_subtest_name);
+	internal_assert_wsignaled(ret, SIGABRT);
+
+	ret = do_fork(dynamic_subtest_in_toplevel);
+	internal_assert_wsignaled(ret, SIGABRT);
+
+	ret = do_fork(subtest_itself_failing);
+	internal_assert_wsignaled(ret, SIGABRT);
+
+	ret = do_fork(subtest_itself_skipping);
+	internal_assert_wexited(ret, IGT_EXIT_SKIP);
+
+	ret = do_fork(dynamic_subtest_failure_leads_to_fail);
+	internal_assert_wexited(ret, IGT_EXIT_FAILURE);
+
+	ret = do_fork(no_dynamic_subtests_entered_leads_to_skip);
+	internal_assert_wexited(ret, IGT_EXIT_SKIP);
+
+	return 0;
+}
lib/tests/meson.build +1 -0
index 0c35da30..e5066665 100644
@@ -4,6 +4,7 @@ lib_tests = [
 	'igt_can_fail_simple',
 	'igt_conflicting_args',
 	'igt_describe',
+	'igt_dynamic_subtests',
 	'igt_edid',
 	'igt_exit_handler',
 	'igt_fork',
Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#6 lore raw

PATCH 3/8 lib/tests: Test that igt_describe works with dynamic subtests

In reply to Petri Latvala

v2: Also test that calling igt_describe in a dynamic context asserts

---
 lib/tests/igt_describe.c | 30 +++++++++++++++++++++++++++++-
 1 file changed, 29 insertions(+), 1 deletion(-)
lib/tests/igt_describe.c +29 -1
index 2ea47e9d..5e40569f 100644
@@ -87,6 +87,14 @@ static void fake_main(int argc, char **argv) {
 	igt_subtest("F")
 		;
 
+	igt_describe("Subtest with dynamic subsubtests");
+	igt_subtest_with_dynamic_subsubtests("G") {
+		printf("should not be executed!\n");
+		igt_describe("should assert on execution");
+		igt_dynamic_subsubtest("should-not-list")
+			printf("should not be executed!\n");
+	}
+
 	igt_exit();
 }
 
@@ -129,7 +137,10 @@ static const char DESCRIBE_ALL_OUTPUT[] = \
 	"\n"
 	"SUB F ../lib/tests/igt_describe.c:87:\n"
 	"  verylongwordthatshoudlbeprintedeventhoughitspastthewrppinglimitverylongwordthatshoudlbeprintedeventhoughitspastthewrappinglimit\n"
-	"  verylongwordthatshoudlbeprintedeventhoughitspastthewrappinglimitverylongwordthatshoudlbeprintedeventhoughitspastthewrappinglimit\n\n";
+	"  verylongwordthatshoudlbeprintedeventhoughitspastthewrappinglimitverylongwordthatshoudlbeprintedeventhoughitspastthewrappinglimit\n"
+	"\n"
+	"SUB G ../lib/tests/igt_describe.c:91:\n"
+	"  Subtest with dynamic subsubtests\n\n";
 
 static const char JUST_C_OUTPUT[] = \
 	"the top level description\n"
@@ -278,5 +289,22 @@ int main(int argc, char **argv)
 		close(errfd);
 	}
 
+	/* trying to igt_describe a dynamic subsubtest should assert */ {
+		static char err[4096];
+		char arg[] = "--run-subtest=G";
+		char *fake_argv[] = {prog, arg};
+		int fake_argc = ARRAY_SIZE(fake_argv);
+
+		pid_t pid = do_fork(fake_argc, fake_argv, &outfd, &errfd);
+
+		read_whole_pipe(errfd, err, sizeof(err));
+
+		internal_assert(_wait(pid, &status) != -1);
+		internal_assert_wsignaled(status, SIGABRT);
+
+		close(outfd);
+		close(errfd);
+	}
+
 	return 0;
 }
Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Reviewed-by on v1 3/8
#7 lore raw

Re: [igt-dev] [PATCH i-g-t 3/8] lib/tests: Test that igt_describe works with dynamic subtests

Quoted 3 lines · Petri Latvala

v2: Also test that calling igt_describe in a dynamic context asserts

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#8 lore raw

PATCH 4/8 runner: Parse dynamic subtest outputs and results

In reply to Petri Latvala

If binary 'bin' has a subtest 'sub', which has dynamic subtests 'foo' and 'bar', results.json will now have "subtests" by the names igt@bin@sub@foo and igt@bin@sub@bar, with data as expected of normal subtests.

---
 runner/job_list.c       |  11 ++
 runner/job_list.h       |   3 +
 runner/output_strings.h |  29 +++-
 runner/resultgen.c      | 377 ++++++++++++++++++++++++++++++++--------
 4 files changed, 346 insertions(+), 74 deletions(-)
runner/job_list.c +11 -0
index 5283fd72..93cede75 100644
@@ -357,6 +357,17 @@ void generate_piglit_name(const char *binary, const char *subtest,
 	free(lc_subtest);
 }
 
+void generate_piglit_name_for_dynamic(const char *base_piglit_name,
+				      const char *dynamic_subtest,
+				      char *namebuf, size_t namebuf_size)
+{
+	char *lc_dynamic = lowercase(dynamic_subtest);
+
+	snprintf(namebuf, namebuf_size, "%s@%s", base_piglit_name, lc_dynamic);
+
+	free(lc_dynamic);
+}
+
 void init_job_list(struct job_list *job_list)
 {
 	memset(job_list, 0, sizeof(*job_list));
runner/job_list.h +3 -0
index 39c9b863..30b4a252 100644
@@ -29,6 +29,9 @@ struct job_list
 
 void generate_piglit_name(const char *binary, const char *subtest,
 			  char *namebuf, size_t namebuf_size);
+void generate_piglit_name_for_dynamic(const char *base_piglit_name,
+				      const char *dynamic_subtest,
+				      char *namebuf, size_t namebuf_size);
 
 void init_job_list(struct job_list *job_list);
 void free_job_list(struct job_list *job_list);
runner/output_strings.h +28 -1
index 1e52c2ce..892895ea 100644
@@ -15,10 +15,28 @@ static const char STARTING_SUBTEST[] = "Starting subtest: ";
  *
  * Examples:
  * Subtest subtestname: SKIP
- * Subtest subtestname: PASS (0.003s)
+ * Subtest subtestname: SUCCESS (0.003s)
  */
 static const char SUBTEST_RESULT[] = "Subtest ";
 
+/*
+ * Output when a dynamic subtest has begun. Is followed by the subtest name.
+ *
+ * Example:
+ * Starting dynamic subtest: subtestname
+ */
+static const char STARTING_DYNAMIC_SUBTEST[] = "Starting dynamic subtest: ";
+
+/*
+ * Output when a dynamic subtest has ended. Is followed by the subtest name
+ * and optionally its runtime.
+ *
+ * Examples:
+ * Dynamic subtest subtestname: SKIP
+ * Dynamic subtest subtestname: SUCCESS (0.003s)
+ */
+static const char DYNAMIC_SUBTEST_RESULT[] = "Dynamic subtest ";
+
 /*
  * Output in dmesg when a subtest has begin. Is followed by the subtest name.
  *
@@ -27,6 +45,15 @@ static const char SUBTEST_RESULT[] = "Subtest ";
  */
 static const char STARTING_SUBTEST_DMESG[] = ": starting subtest ";
 
+/*
+ * Output in dmesg when a dynamic subtest has begin. Is followed by
+ * the subtest name.
+ *
+ * Example:
+ * [IGT] test-binary-name: starting dynamic subtest subtestname
+ */
+static const char STARTING_DYNAMIC_SUBTEST_DMESG[] = ": starting dynamic subtest ";
+
 /*
  * Output when a test process is executed.
  *
runner/resultgen.c +304 -73

616 lines not on this page · raw message

Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Reviewed-by on v1 4/8
#9 lore raw

Re: [igt-dev] [PATCH i-g-t 4/8] runner: Parse dynamic subtest outputs and results

Quoted 6 lines · Petri Latvala

If binary 'bin' has a subtest 'sub', which has dynamic subtests 'foo' and 'bar', results.json will now have "subtests" by the names igt@bin@sub@foo and igt@bin@sub@bar, with data as expected of normal subtests.

Those fill_from_ functions are monstrous... Each time I review changes there I have to read through the full code on the side first to get what's going on, and it's still easy to get lost.

Next time we are touching this area it should undergo a modernization...

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#10 lore raw

PATCH 5/8 runner/json_tests: Test dynamic subtests

In reply to Petri Latvala
---
 .../dynamic-subtests/0/dmesg.txt              |   7 +
 .../dynamic-subtests/0/err.txt                |  36 ++++
 .../dynamic-subtests/0/journal.txt            |   2 +
 .../dynamic-subtests/0/out.txt                |  19 ++
 .../dynamic-subtests/1/dmesg.txt              |   5 +
 .../dynamic-subtests/1/err.txt                |   2 +
 .../dynamic-subtests/1/journal.txt            |   2 +
 .../dynamic-subtests/1/out.txt                |   5 +
 .../dynamic-subtests/2/dmesg.txt              |  11 ++
 .../dynamic-subtests/2/err.txt                |   8 +
 .../dynamic-subtests/2/journal.txt            |   4 +
 .../dynamic-subtests/2/out.txt                |  10 ++
 .../dynamic-subtests/README.txt               |   2 +
 .../dynamic-subtests/endtime.txt              |   1 +
 .../dynamic-subtests/joblist.txt              |   3 +
 .../dynamic-subtests/metadata.txt             |  12 ++
 .../dynamic-subtests/reference.json           | 168 ++++++++++++++++++
 .../dynamic-subtests/starttime.txt            |   1 +
 .../dynamic-subtests/uname.txt                |   1 +
 runner/runner_json_tests.c                    |   3 +-
 20 files changed, 301 insertions(+), 1 deletion(-)
 create mode 100644 runner/json_tests_data/dynamic-subtests/0/dmesg.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/0/err.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/0/journal.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/0/out.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/1/dmesg.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/1/err.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/1/journal.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/1/out.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/2/dmesg.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/2/err.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/2/journal.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/2/out.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/README.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/endtime.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/joblist.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/metadata.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/reference.json
 create mode 100644 runner/json_tests_data/dynamic-subtests/starttime.txt
 create mode 100644 runner/json_tests_data/dynamic-subtests/uname.txt
runner/json_tests_data/dynamic-subtests/0/dmesg.txt new file +7 -0
new file mode 100644
index 00000000..c87b4271
@@ -0,0 +1,7 @@
+6,1157,23426155175691,-;Console: switching to colour dummy device 80x25
+14,1158,23426155175708,-;[IGT] dynamic: executing
+14,1159,23426155184875,-;[IGT] dynamic: starting subtest debug-log-checking
+14,1160,23426155184895,-;[IGT] dynamic: starting dynamic subtest this-is-dynamic-1
+14,1161,23426155240164,-;[IGT] dynamic: starting dynamic subtest this-is-dynamic-2
+14,1162,23426155293846,-;[IGT] dynamic: exiting, ret=98
+6,1163,23426155294003,-;Console: switching to colour frame buffer device 240x75
runner/json_tests_data/dynamic-subtests/0/err.txt new file +36 -0
new file mode 100644
index 00000000..3c280788
@@ -0,0 +1,36 @@
+Starting subtest: debug-log-checking
+Starting dynamic subtest: this-is-dynamic-1
+(dynamic:20904) CRITICAL: Test assertion failure function __real_main3, file ../runner/testdata/dynamic.c:8:
+(dynamic:20904) CRITICAL: Failed assertion: false
+Dynamic subtest this-is-dynamic-1 failed.
+**** DEBUG ****
+(dynamic:20904) DEBUG: This print is from 1
+(dynamic:20904) CRITICAL: Test assertion failure function __real_main3, file ../runner/testdata/dynamic.c:8:
+(dynamic:20904) CRITICAL: Failed assertion: false
+(dynamic:20904) igt_core-INFO: Stack trace:
+(dynamic:20904) igt_core-INFO:   #0 ../lib/igt_core.c:1607 __igt_fail_assert()
+(dynamic:20904) igt_core-INFO:   #1 ../runner/testdata/dynamic.c:11 __real_main3()
+(dynamic:20904) igt_core-INFO:   #2 ../runner/testdata/dynamic.c:3 main()
+(dynamic:20904) igt_core-INFO:   #3 ../csu/libc-start.c:342 __libc_start_main()
+(dynamic:20904) igt_core-INFO:   #4 [_start+0x2a]
+****  END  ****
+Dynamic subtest this-is-dynamic-1: FAIL (0.055s)
+Starting dynamic subtest: this-is-dynamic-2
+(dynamic:20904) CRITICAL: Test assertion failure function __real_main3, file ../runner/testdata/dynamic.c:13:
+(dynamic:20904) CRITICAL: Failed assertion: false
+Dynamic subtest this-is-dynamic-2 failed.
+**** DEBUG ****
+(dynamic:20904) DEBUG: This print is from 2
+(dynamic:20904) CRITICAL: Test assertion failure function __real_main3, file ../runner/testdata/dynamic.c:13:
+(dynamic:20904) CRITICAL: Failed assertion: false
+(dynamic:20904) igt_core-INFO: Stack trace:
+(dynamic:20904) igt_core-INFO:   #0 ../lib/igt_core.c:1607 __igt_fail_assert()
+(dynamic:20904) igt_core-INFO:   #1 ../runner/testdata/dynamic.c:5 __real_main3()
+(dynamic:20904) igt_core-INFO:   #2 ../runner/testdata/dynamic.c:3 main()
+(dynamic:20904) igt_core-INFO:   #3 ../csu/libc-start.c:342 __libc_start_main()
+(dynamic:20904) igt_core-INFO:   #4 [_start+0x2a]
+****  END  ****
+Dynamic subtest this-is-dynamic-2: FAIL (0.054s)
+Subtest debug-log-checking failed.
+No log.
+Subtest debug-log-checking: FAIL (0.109s)
runner/json_tests_data/dynamic-subtests/0/journal.txt new file +2 -0
new file mode 100644
index 00000000..aa8cef0c
@@ -0,0 +1,2 @@
+debug-log-checking
+exit:98 (0.130s)
runner/json_tests_data/dynamic-subtests/0/out.txt new file +19 -0
new file mode 100644
index 00000000..286da3f3
@@ -0,0 +1,19 @@
+IGT-Version: 1.23-g9e957acd (x86_64) (Linux: 4.18.0-1-amd64 x86_64)
+Starting subtest: debug-log-checking
+Starting dynamic subtest: this-is-dynamic-1
+Stack trace:
+  #0 ../lib/igt_core.c:1607 __igt_fail_assert()
+  #1 ../runner/testdata/dynamic.c:11 __real_main3()
+  #2 ../runner/testdata/dynamic.c:3 main()
+  #3 ../csu/libc-start.c:342 __libc_start_main()
+  #4 [_start+0x2a]
+Dynamic subtest this-is-dynamic-1: FAIL (0.055s)
+Starting dynamic subtest: this-is-dynamic-2
+Stack trace:
+  #0 ../lib/igt_core.c:1607 __igt_fail_assert()
+  #1 ../runner/testdata/dynamic.c:5 __real_main3()
+  #2 ../runner/testdata/dynamic.c:3 main()
+  #3 ../csu/libc-start.c:342 __libc_start_main()
+  #4 [_start+0x2a]
+Dynamic subtest this-is-dynamic-2: FAIL (0.054s)
+Subtest debug-log-checking: FAIL (0.109s)
runner/json_tests_data/dynamic-subtests/1/dmesg.txt new file +5 -0
new file mode 100644
index 00000000..d6e16a46
@@ -0,0 +1,5 @@
+6,1164,23426155304955,-;Console: switching to colour dummy device 80x25
+14,1165,23426155304968,-;[IGT] dynamic: executing
+14,1166,23426155308644,-;[IGT] dynamic: starting subtest empty-container
+14,1167,23426155308671,-;[IGT] dynamic: exiting, ret=77
+6,1168,23426155308822,-;Console: switching to colour frame buffer device 240x75
runner/json_tests_data/dynamic-subtests/1/err.txt new file +2 -0
new file mode 100644
index 00000000..6247e714
@@ -0,0 +1,2 @@
+Starting subtest: empty-container
+Subtest empty-container: SKIP (0.000s)
runner/json_tests_data/dynamic-subtests/1/journal.txt new file +2 -0
new file mode 100644
index 00000000..3e4ce5c4
@@ -0,0 +1,2 @@
+empty-container
+exit:77 (0.014s)
runner/json_tests_data/dynamic-subtests/1/out.txt new file +5 -0
new file mode 100644
index 00000000..704734dc
@@ -0,0 +1,5 @@
+IGT-Version: 1.23-g9e957acd (x86_64) (Linux: 4.18.0-1-amd64 x86_64)
+Starting subtest: empty-container
+This should skip
+No dynamic tests executed.
+Subtest empty-container: SKIP (0.000s)
runner/json_tests_data/dynamic-subtests/2/dmesg.txt new file +11 -0
new file mode 100644
index 00000000..046ef579
@@ -0,0 +1,11 @@
+6,1157,23426155175691,-;Console: switching to colour dummy device 80x25
+14,1158,23426155175708,-;[IGT] dynamic: executing
+14,1159,23426155184875,-;[IGT] dynamic: starting subtest normal
+6,1160,23426155184895,-;Dmesg output for normal
+6,1160,23426155184895,-;[IGT] dynamic: starting dynamic subtest normal-dynamic-subtest
+14,1159,23426155184875,-;[IGT] dynamic: starting subtest incomplete
+14,1160,23426155184895,-;[IGT] dynamic: starting dynamic subtest this-is-incomplete
+6,1160,23426155184895,-;Dmesg output for incomplete
+14,1161,23426155240164,-;[IGT] dynamic: starting subtest resume
+14,1162,23426155293846,-;[IGT] dynamic: exiting, ret=0
+6,1163,23426155294003,-;Console: switching to colour frame buffer device 240x75
runner/json_tests_data/dynamic-subtests/2/err.txt new file +8 -0
new file mode 100644
index 00000000..1eae6061
@@ -0,0 +1,8 @@
+Starting subtest: normal
+Starting dynamic subtest: normal-dynamic-subtest
+Dynamic subtest normal-dynamic-subtest: SUCCESS (0.055s)
+Subtest normal: SUCCESS (0.100s)
+Starting subtest: incomplete
+Starting dynamic subtest: this-is-incomplete
+Starting subtest: resume
+Subtest resume: SUCCESS (0.109s)
runner/json_tests_data/dynamic-subtests/2/journal.txt new file +4 -0
new file mode 100644
index 00000000..9469e227
@@ -0,0 +1,4 @@
+normal
+incomplete
+resume
+exit:0 (0.130s)
runner/json_tests_data/dynamic-subtests/2/out.txt new file +10 -0
new file mode 100644
index 00000000..3063844b
@@ -0,0 +1,10 @@
+IGT-Version: 1.23-g9e957acd (x86_64) (Linux: 4.18.0-1-amd64 x86_64)
+Starting subtest: normal
+Starting dynamic subtest: normal-dynamic-subtest
+Dynamic subtest normal-dynamic-subtest: SUCCESS (0.055s)
+Subtest normal: SUCCESS (0.100s)
+Starting subtest: incomplete
+Starting dynamic subtest: this-is-incomplete
+This is some output
+Starting subtest: resume
+Subtest resume: SUCCESS (0.109s)
runner/json_tests_data/dynamic-subtests/README.txt new file +2 -0
new file mode 100644
index 00000000..3a63ab17
@@ -0,0 +1,2 @@
+A test with dynamic subtests should generate separate subresults for
+the dynamic tests.
runner/json_tests_data/dynamic-subtests/endtime.txt new file +1 -0
new file mode 100644
index 00000000..5c7608b5
@@ -0,0 +1 @@
+1560163492.410489
runner/json_tests_data/dynamic-subtests/joblist.txt new file +3 -0
new file mode 100644
index 00000000..4fdcc5f1
@@ -0,0 +1,3 @@
+dynamic debug-log-checking
+dynamic empty-container
+dynamic normal,incomplete,resume
runner/json_tests_data/dynamic-subtests/metadata.txt new file +12 -0
new file mode 100644
index 00000000..8c77de01
@@ -0,0 +1,12 @@
+abort_mask : 0
+name : dynamic-subtests
+dry_run : 0
+sync : 0
+log_level : 0
+overwrite : 0
+multiple_mode : 1
+inactivity_timeout : 0
+use_watchdog : 0
+piglit_style_dmesg : 0
+test_root : /path/does/not/exist
+results_path : /path/does/not/exist
runner/json_tests_data/dynamic-subtests/reference.json new file +168 -0

168 lines not on this page · raw message

runner/json_tests_data/dynamic-subtests/starttime.txt new file +1 -0
new file mode 100644
index 00000000..b30b32ee
@@ -0,0 +1 @@
+1560163492.266377
runner/json_tests_data/dynamic-subtests/uname.txt new file +1 -0
new file mode 100644
index 00000000..a7aef6f7
@@ -0,0 +1 @@
+Linux hostname 4.18.0-1-amd64 #1 SMP Debian 4.18.6-1 (2018-09-06) x86_64
runner/runner_json_tests.c +2 -1
index 7c9540d1..3443bd2b 100644
@@ -161,7 +161,8 @@ static const char *dirnames[] = {
 	"notrun-results-multiple-mode",
 	"dmesg-warn-level",
 	"dmesg-warn-level-piglit-style",
-	"dmesg-warn-level-one-piglit-style"
+	"dmesg-warn-level-one-piglit-style",
+	"dynamic-subtests",
 };
 
 igt_main
Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#12 lore raw

PATCH 6/8 kms_plane_cursor: Use dynamic subtests

In reply to Petri Latvala

Instead of for_each_pipe_static, use for_each_pipe with a dynamic subtest per pipe.

---
 tests/kms_plane_cursor.c | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)
tests/kms_plane_cursor.c +18 -16
index cbad0041..9616430c 100644
@@ -318,22 +318,24 @@ igt_main
 		igt_display_require_output(&data.display);
 	}
 
-	for_each_pipe_static(pipe)
-		for (i = 0; i < ARRAY_SIZE(cursor_sizes); ++i) {
-			int size = cursor_sizes[i];
-
-			igt_subtest_f("pipe-%s-overlay-size-%d",
-				      kmstest_pipe_name(pipe), size)
-				test_cursor_overlay(&data, size, pipe);
-
-			igt_subtest_f("pipe-%s-primary-size-%d",
-				      kmstest_pipe_name(pipe), size)
-				test_cursor_primary(&data, size, pipe);
-
-			igt_subtest_f("pipe-%s-viewport-size-%d",
-				      kmstest_pipe_name(pipe), size)
-				test_cursor_viewport(&data, size, pipe);
-		}
+	for (i = 0; i < ARRAY_SIZE(cursor_sizes); ++i) {
+		int size = cursor_sizes[i];
+
+		igt_subtest_with_dynamic_subsubtests_f("overlay-size-%d", size)
+			for_each_pipe(&data.display, pipe)
+				igt_dynamic_subsubtest_f("pipe-%s", kmstest_pipe_name(pipe))
+					test_cursor_overlay(&data, size, pipe);
+
+		igt_subtest_with_dynamic_subsubtests_f("primary-size-%d", size)
+			for_each_pipe(&data.display, pipe)
+				igt_dynamic_subsubtest_f("pipe-%s", kmstest_pipe_name(pipe))
+					test_cursor_primary(&data, size, pipe);
+
+		igt_subtest_with_dynamic_subsubtests_f("viewport-size-%d", size)
+			for_each_pipe(&data.display, pipe)
+				igt_dynamic_subsubtest_f("pipe-%s", kmstest_pipe_name(pipe))
+					test_cursor_viewport(&data, size, pipe);
+	}
 
 	igt_fixture {
 		igt_display_fini(&data.display);
Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#13 lore raw

Re: [igt-dev] [PATCH i-g-t 6/8] kms_plane_cursor: Use dynamic subtests

Quoted 4 lines · Petri Latvala

Instead of for_each_pipe_static, use for_each_pipe with a dynamic subtest per pipe.

Hello,

I saw you are an author of these tests. Is it possible that you will review dynamic subtests usage in this binary?

Have a nice day!
Kasia :)

Quoted 52 lines
---
 tests/kms_plane_cursor.c | 34 ++++++++++++++++++----------------
 1 file changed, 18 insertions(+), 16 deletions(-)
tests/kms_plane_cursor.c +18 -16
index cbad0041..9616430c 100644
@@ -318,22 +318,24 @@ igt_main
 		igt_display_require_output(&data.display);
 	}
 
-	for_each_pipe_static(pipe)
-		for (i = 0; i < ARRAY_SIZE(cursor_sizes); ++i) {
-			int size = cursor_sizes[i];
-
-			igt_subtest_f("pipe-%s-overlay-size-%d",
-				      kmstest_pipe_name(pipe), size)
-				test_cursor_overlay(&data, size, pipe);
-
-			igt_subtest_f("pipe-%s-primary-size-%d",
-				      kmstest_pipe_name(pipe), size)
-				test_cursor_primary(&data, size, pipe);
-
-			igt_subtest_f("pipe-%s-viewport-size-%d",
-				      kmstest_pipe_name(pipe), size)
-				test_cursor_viewport(&data, size, pipe);
-		}
+	for (i = 0; i < ARRAY_SIZE(cursor_sizes); ++i) {
+		int size = cursor_sizes[i];
+
+		igt_subtest_with_dynamic_subsubtests_f("overlay-size-%d", size)
+			for_each_pipe(&data.display, pipe)
+				igt_dynamic_subsubtest_f("pipe-%s", kmstest_pipe_name(pipe))
+					test_cursor_overlay(&data, size, pipe);
+
+		igt_subtest_with_dynamic_subsubtests_f("primary-size-%d", size)
+			for_each_pipe(&data.display, pipe)
+				igt_dynamic_subsubtest_f("pipe-%s", kmstest_pipe_name(pipe))
+					test_cursor_primary(&data, size, pipe);
+
+		igt_subtest_with_dynamic_subsubtests_f("viewport-size-%d", size)
+			for_each_pipe(&data.display, pipe)
+				igt_dynamic_subsubtest_f("pipe-%s", kmstest_pipe_name(pipe))
+					test_cursor_viewport(&data, size, pipe);
+	}
 
 	igt_fixture {
 		igt_display_fini(&data.display);
Signature · 1 line
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#14 lore raw

Re: [igt-dev] [PATCH i-g-t 6/8] kms_plane_cursor: Use dynamic subtests

In reply to Petri Latvala
Quoted 4 lines · Petri Latvala

Instead of for_each_pipe_static, use for_each_pipe with a dynamic subtest per pipe.

As pointed out by Maarten, this kind of straightforward conversion makes us perform tons of more modesets, destroying the runtime gains when running multiple subtests in one binary execution.

Consider this patch returned to drawing table.

The rest of the lot will start trickling into git soon-ish, after some discussions with Lakshmi and Martin about the best process for cibuglog filtering.

Signature · 5 lines
Petri Latvala
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#15 lore raw

PATCH 7/8 perf_pmu: Use dynamic subtests

In reply to Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

In addition, convert comments to igt_describe calls and generally refactor the lot to a series of simpler loops.

---
 tests/perf_pmu.c | 244 ++++++++++++++++++++---------------------------
 1 file changed, 105 insertions(+), 139 deletions(-)
tests/perf_pmu.c +105 -139
index e2bd2cc5..6a4a600e 100644
@@ -270,7 +270,7 @@ static void end_spin(int fd, igt_spin_t *spin, unsigned int flags)
 }
 
 static void
-single(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
+single(int gem_fd, const struct intel_execution_engine2 *e, unsigned int num_engines, unsigned int flags)
 {
 	unsigned long slept;
 	igt_spin_t *spin;
@@ -606,7 +606,7 @@ all_busy_check_all(int gem_fd, const unsigned int num_engines,
 }
 
 static void
-no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
+no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int num_engines, unsigned int flags)
 {
 	igt_spin_t *spin;
 	uint64_t val[2][2];
@@ -646,7 +646,7 @@ no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
 
 static void
 sema_wait(int gem_fd, const struct intel_execution_engine2 *e,
-	  unsigned int flags)
+	  unsigned int num_engines, unsigned int flags)
 {
 	struct drm_i915_gem_relocation_entry reloc[2] = {};
 	struct drm_i915_gem_exec_object2 obj[2] = {};
@@ -809,6 +809,7 @@ __sema_busy(int gem_fd, int pmu,
 static void
 sema_busy(int gem_fd,
 	  const struct intel_execution_engine2 *e,
+	  unsigned int num_engines,
 	  unsigned int flags)
 {
 	const struct intel_execution_engine2 *signal;
@@ -1770,6 +1771,27 @@ igt_main
 	int fd = -1;
 	struct intel_execution_engine2 *e;
 	unsigned int i;
+	const unsigned int pct[] = { 2, 50, 98 };
+	struct {
+		void (*func)(int, const struct intel_execution_engine2 *, unsigned int, unsigned int);
+		const char *name;
+		unsigned int flags;
+		const char *desc;
+	} busyidle_tests[] = {
+		{ single, "idle", 0, "Test that engines show no load when idle." },
+		{ single, "busy", TEST_BUSY, "Test that a single engine reports load correctly." },
+		{ single, "busy-idle", TEST_BUSY | TEST_TRAILING_IDLE, "Test that a single engine reports load correctly." },
+		{ busy_check_all, "busy-check-all", TEST_BUSY, "Test that when one engine is loaded, others report no load." },
+		{ busy_check_all, "busy-idle-check-all", TEST_BUSY | TEST_TRAILING_IDLE, "Test that one one engine is loaded, others report no load." },
+		{ most_busy_check_all, "most-busy-check-all", TEST_BUSY, "Test that when all except one engine are loaded all loads are correctly reported." },
+		{ most_busy_check_all, "most-busy-idle-check-all", TEST_BUSY | TEST_TRAILING_IDLE, "Test that when all except one engine are loaded all loads are correctly reported." },
+		{ no_sema, "idle-no-semaphores", 0, "Test that semaphore counters report no activity on idle engines." },
+		{ no_sema, "busy-no-semaphores", TEST_BUSY, "Test that semaphore counters report no activity on busy engines." },
+		{ no_sema, "busy-idle-no-semaphores", TEST_BUSY | TEST_TRAILING_IDLE, "Test that semaphore counters report no activity on idle or busy engines." },
+		{ sema_wait, "semaphore-wait", TEST_BUSY, "Test that semaphore waits are correctly reported." },
+		{ sema_wait, "semaphore-wait-idle", TEST_BUSY | TEST_TRAILING_IDLE, "Test that semaphore waits are correctly reported." },
+		{ sema_busy, "semaphore-busy", 0, "Test that semaphore time and busy time difference is under a threshold." },
+	};
 
 	igt_fixture {
 		fd = drm_open_driver_master(DRIVER_INTEL);
@@ -1781,144 +1803,89 @@ igt_main
 			num_engines++;
 	}
 
-	/**
-	 * Test invalid access via perf API is rejected.
-	 */
+	igt_describe("Test that invalid access via perf API is rejected.");
 	igt_subtest("invalid-init")
 		invalid_init();
 
-	__for_each_physical_engine(fd, e) {
-		const unsigned int pct[] = { 2, 50, 98 };
-
-		/**
-		 * Test that a single engine metric can be initialized or it
-		 * is correctly rejected.
-		 */
-		igt_subtest_f("init-busy-%s", e->name)
-			init(fd, e, I915_SAMPLE_BUSY);
-
-		igt_subtest_f("init-wait-%s", e->name)
-			init(fd, e, I915_SAMPLE_WAIT);
-
-		igt_subtest_f("init-sema-%s", e->name)
-			init(fd, e, I915_SAMPLE_SEMA);
-
-		/**
-		 * Test that engines show no load when idle.
-		 */
-		igt_subtest_f("idle-%s", e->name)
-			single(fd, e, 0);
-
-		/**
-		 * Test that a single engine reports load correctly.
-		 */
-		igt_subtest_f("busy-%s", e->name)
-			single(fd, e, TEST_BUSY);
-		igt_subtest_f("busy-idle-%s", e->name)
-			single(fd, e, TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that when one engine is loaded other report no
-		 * load.
-		 */
-		igt_subtest_f("busy-check-all-%s", e->name)
-			busy_check_all(fd, e, num_engines, TEST_BUSY);
-		igt_subtest_f("busy-idle-check-all-%s", e->name)
-			busy_check_all(fd, e, num_engines,
-				       TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that when all except one engine are loaded all
-		 * loads are correctly reported.
-		 */
-		igt_subtest_f("most-busy-check-all-%s", e->name)
-			most_busy_check_all(fd, e, num_engines,
-					    TEST_BUSY);
-		igt_subtest_f("most-busy-idle-check-all-%s", e->name)
-			most_busy_check_all(fd, e, num_engines,
-					    TEST_BUSY |
-					    TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that semphore counters report no activity on
-		 * idle or busy engines.
-		 */
-		igt_subtest_f("idle-no-semaphores-%s", e->name)
-			no_sema(fd, e, 0);
-
-		igt_subtest_f("busy-no-semaphores-%s", e->name)
-			no_sema(fd, e, TEST_BUSY);
-
-		igt_subtest_f("busy-idle-no-semaphores-%s", e->name)
-			no_sema(fd, e, TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that semaphore waits are correctly reported.
-		 */
-		igt_subtest_f("semaphore-wait-%s", e->name)
-			sema_wait(fd, e, TEST_BUSY);
-
-		igt_subtest_f("semaphore-wait-idle-%s", e->name)
-			sema_wait(fd, e,
-				  TEST_BUSY | TEST_TRAILING_IDLE);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-busy")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_BUSY);
 
-		igt_subtest_f("semaphore-busy-%s", e->name)
-			sema_busy(fd, e, 0);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-wait")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_WAIT);
 
-		/**
-		 * Check that two perf clients do not influence each
-		 * others observations.
-		 */
-		igt_subtest_f("multi-client-%s", e->name)
-			multi_client(fd, e);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-sema")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_SEMA);
+
+	for (i = 0; i < ARRAY_SIZE(busyidle_tests); i++) {
+		igt_describe(busyidle_tests[i].desc);
+		igt_subtest_with_dynamic_subsubtests(busyidle_tests[i].name)
+			__for_each_physical_engine(fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					busyidle_tests[i].func(fd, e, num_engines, busyidle_tests[i].flags);
+	}
 
-		/**
-		 * Check that reported usage is correct when PMU is
-		 * enabled after the batch is running.
-		 */
-		igt_subtest_f("busy-start-%s", e->name)
-			busy_start(fd, e);
+	igt_describe("Check that two perf clients do not influence each other's observations.");
+	igt_subtest_with_dynamic_subsubtests("multi-client")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				multi_client(fd, e);
 
-		/**
-		 * Check that reported usage is correct when PMU is
-		 * enabled after two batches are running.
-		 */
-		igt_subtest_f("busy-double-start-%s", e->name) {
-			gem_require_contexts(fd);
-			busy_double_start(fd, e);
-		}
+	igt_describe("Check that reported usage is correct when PMU is enabled after the batch is running.");
+	igt_subtest_with_dynamic_subsubtests("busy-start")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				busy_start(fd, e);
 
-		/**
-		 * Check that the PMU can be safely enabled in face of
-		 * interrupt-heavy engine load.
-		 */
-		igt_subtest_f("enable-race-%s", e->name)
-			test_enable_race(fd, e);
+	igt_describe("Check that reported usage is correct when PMU is enabled after two batches are running.");
+	igt_subtest_with_dynamic_subsubtests("busy-double-start") {
+		gem_require_contexts(fd);
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				busy_double_start(fd, e);
+	}
 
-		/**
-		 * Check engine busyness accuracy is as expected.
-		 */
-		for (i = 0; i < ARRAY_SIZE(pct); i++) {
-			igt_subtest_f("busy-accuracy-%u-%s",
-				      pct[i], e->name)
-				accuracy(fd, e, pct[i], 10);
-		}
+	igt_describe("Check that the PMU can be safely enabled in face of interrupt-heavy engine load.");
+	igt_subtest_with_dynamic_subsubtests("enable-race")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				test_enable_race(fd, e);
+
+	for (i = 0; i < ARRAY_SIZE(pct); i++) {
+		igt_describe("Check engine busyness accuracy is as expected.");
+		igt_subtest_with_dynamic_subsubtests_f("busy-accuracy-%u", pct[i])
+			__for_each_physical_engine(fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					accuracy(fd, e, pct[i], 10);
+	}
 
-		igt_subtest_f("busy-hang-%s", e->name) {
-			igt_hang_t hang = igt_allow_hang(fd, 0, 0);
+	igt_subtest_with_dynamic_subsubtests("busy-hang")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name) {
+				igt_hang_t hang = igt_allow_hang(fd, 0, 0);
 
-			single(fd, e, TEST_BUSY | FLAG_HANG);
+				single(fd, e, 0, TEST_BUSY | FLAG_HANG);
 
-			igt_disallow_hang(fd, hang);
-		}
+				igt_disallow_hang(fd, hang);
+			}
 
-		/**
-		 * Test that event waits are correctly reported.
-		 */
-		if (e->class == I915_ENGINE_CLASS_RENDER)
-			igt_subtest_f("event-wait-%s", e->name)
-				event_wait(fd, e);
-	}
+	igt_describe("Test that event waits are correctly reported.");
+	igt_subtest_with_dynamic_subsubtests("event-wait")
+		__for_each_physical_engine(fd, e)
+			if (e->class == I915_ENGINE_CLASS_RENDER)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					event_wait(fd, e);
 
 	/**
 	 * Test that when all engines are loaded all loads are
@@ -1975,9 +1942,7 @@ igt_main
 	igt_subtest("rc6-runtime-pm-long")
 		test_rc6(fd, TEST_RUNTIME_PM | FLAG_LONG);
 
-	/**
-	 * Check render nodes are counted.
-	 */
+	igt_describe("Check that render nodes are counted.");
 	igt_subtest_group {
 		int render_fd = -1;
 
@@ -1988,14 +1953,15 @@ igt_main
 			gem_quiescent_gpu(fd);
 		}
 
-		__for_each_physical_engine(render_fd, e) {
-			igt_subtest_f("render-node-busy-%s", e->name)
-				single(render_fd, e, TEST_BUSY);
-			igt_subtest_f("render-node-busy-idle-%s",
-				      e->name)
-				single(render_fd, e,
-				       TEST_BUSY | TEST_TRAILING_IDLE);
-		}
+		igt_subtest_with_dynamic_subsubtests("render-node-busy")
+			__for_each_physical_engine(render_fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					single(render_fd, e, 0, TEST_BUSY);
+
+		igt_subtest_with_dynamic_subsubtests("render-node-busy-idle")
+			__for_each_physical_engine(render_fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					single(render_fd, e, 0, TEST_BUSY | TEST_TRAILING_IDLE);
 
 		igt_fixture {
 			close(render_fd);
Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#16 lore raw

Re: [igt-dev] [PATCH i-g-t 7/8] perf_pmu: Use dynamic subtests

Quoted 6 lines · Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

In addition, convert comments to igt_describe calls and generally refactor the lot to a series of simpler loops.

Tvrko, is there a chance you could look at dynamic engines usage here and give r-b? It will allow us to merge whole series.

Have a nice day!
Kasia :)

Quoted 326 lines
---
 tests/perf_pmu.c | 244 ++++++++++++++++++++---------------------------
 1 file changed, 105 insertions(+), 139 deletions(-)
tests/perf_pmu.c +105 -139
index e2bd2cc5..6a4a600e 100644
@@ -270,7 +270,7 @@ static void end_spin(int fd, igt_spin_t *spin, unsigned int flags)
 }
 
 static void
-single(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
+single(int gem_fd, const struct intel_execution_engine2 *e, unsigned int num_engines, unsigned int flags)
 {
 	unsigned long slept;
 	igt_spin_t *spin;
@@ -606,7 +606,7 @@ all_busy_check_all(int gem_fd, const unsigned int num_engines,
 }
 
 static void
-no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
+no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int num_engines, unsigned int flags)
 {
 	igt_spin_t *spin;
 	uint64_t val[2][2];
@@ -646,7 +646,7 @@ no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
 
 static void
 sema_wait(int gem_fd, const struct intel_execution_engine2 *e,
-	  unsigned int flags)
+	  unsigned int num_engines, unsigned int flags)
 {
 	struct drm_i915_gem_relocation_entry reloc[2] = {};
 	struct drm_i915_gem_exec_object2 obj[2] = {};
@@ -809,6 +809,7 @@ __sema_busy(int gem_fd, int pmu,
 static void
 sema_busy(int gem_fd,
 	  const struct intel_execution_engine2 *e,
+	  unsigned int num_engines,
 	  unsigned int flags)
 {
 	const struct intel_execution_engine2 *signal;
@@ -1770,6 +1771,27 @@ igt_main
 	int fd = -1;
 	struct intel_execution_engine2 *e;
 	unsigned int i;
+	const unsigned int pct[] = { 2, 50, 98 };
+	struct {
+		void (*func)(int, const struct intel_execution_engine2 *, unsigned int, unsigned int);
+		const char *name;
+		unsigned int flags;
+		const char *desc;
+	} busyidle_tests[] = {
+		{ single, "idle", 0, "Test that engines show no load when idle." },
+		{ single, "busy", TEST_BUSY, "Test that a single engine reports load correctly." },
+		{ single, "busy-idle", TEST_BUSY | TEST_TRAILING_IDLE, "Test that a single engine reports load correctly." },
+		{ busy_check_all, "busy-check-all", TEST_BUSY, "Test that when one engine is loaded, others report no load." },
+		{ busy_check_all, "busy-idle-check-all", TEST_BUSY | TEST_TRAILING_IDLE, "Test that one one engine is loaded, others report no load." },
+		{ most_busy_check_all, "most-busy-check-all", TEST_BUSY, "Test that when all except one engine are loaded all loads are correctly reported." },
+		{ most_busy_check_all, "most-busy-idle-check-all", TEST_BUSY | TEST_TRAILING_IDLE, "Test that when all except one engine are loaded all loads are correctly reported." },
+		{ no_sema, "idle-no-semaphores", 0, "Test that semaphore counters report no activity on idle engines." },
+		{ no_sema, "busy-no-semaphores", TEST_BUSY, "Test that semaphore counters report no activity on busy engines." },
+		{ no_sema, "busy-idle-no-semaphores", TEST_BUSY | TEST_TRAILING_IDLE, "Test that semaphore counters report no activity on idle or busy engines." },
+		{ sema_wait, "semaphore-wait", TEST_BUSY, "Test that semaphore waits are correctly reported." },
+		{ sema_wait, "semaphore-wait-idle", TEST_BUSY | TEST_TRAILING_IDLE, "Test that semaphore waits are correctly reported." },
+		{ sema_busy, "semaphore-busy", 0, "Test that semaphore time and busy time difference is under a threshold." },
+	};
 
 	igt_fixture {
 		fd = drm_open_driver_master(DRIVER_INTEL);
@@ -1781,144 +1803,89 @@ igt_main
 			num_engines++;
 	}
 
-	/**
-	 * Test invalid access via perf API is rejected.
-	 */
+	igt_describe("Test that invalid access via perf API is rejected.");
 	igt_subtest("invalid-init")
 		invalid_init();
 
-	__for_each_physical_engine(fd, e) {
-		const unsigned int pct[] = { 2, 50, 98 };
-
-		/**
-		 * Test that a single engine metric can be initialized or it
-		 * is correctly rejected.
-		 */
-		igt_subtest_f("init-busy-%s", e->name)
-			init(fd, e, I915_SAMPLE_BUSY);
-
-		igt_subtest_f("init-wait-%s", e->name)
-			init(fd, e, I915_SAMPLE_WAIT);
-
-		igt_subtest_f("init-sema-%s", e->name)
-			init(fd, e, I915_SAMPLE_SEMA);
-
-		/**
-		 * Test that engines show no load when idle.
-		 */
-		igt_subtest_f("idle-%s", e->name)
-			single(fd, e, 0);
-
-		/**
-		 * Test that a single engine reports load correctly.
-		 */
-		igt_subtest_f("busy-%s", e->name)
-			single(fd, e, TEST_BUSY);
-		igt_subtest_f("busy-idle-%s", e->name)
-			single(fd, e, TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that when one engine is loaded other report no
-		 * load.
-		 */
-		igt_subtest_f("busy-check-all-%s", e->name)
-			busy_check_all(fd, e, num_engines, TEST_BUSY);
-		igt_subtest_f("busy-idle-check-all-%s", e->name)
-			busy_check_all(fd, e, num_engines,
-				       TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that when all except one engine are loaded all
-		 * loads are correctly reported.
-		 */
-		igt_subtest_f("most-busy-check-all-%s", e->name)
-			most_busy_check_all(fd, e, num_engines,
-					    TEST_BUSY);
-		igt_subtest_f("most-busy-idle-check-all-%s", e->name)
-			most_busy_check_all(fd, e, num_engines,
-					    TEST_BUSY |
-					    TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that semphore counters report no activity on
-		 * idle or busy engines.
-		 */
-		igt_subtest_f("idle-no-semaphores-%s", e->name)
-			no_sema(fd, e, 0);
-
-		igt_subtest_f("busy-no-semaphores-%s", e->name)
-			no_sema(fd, e, TEST_BUSY);
-
-		igt_subtest_f("busy-idle-no-semaphores-%s", e->name)
-			no_sema(fd, e, TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that semaphore waits are correctly reported.
-		 */
-		igt_subtest_f("semaphore-wait-%s", e->name)
-			sema_wait(fd, e, TEST_BUSY);
-
-		igt_subtest_f("semaphore-wait-idle-%s", e->name)
-			sema_wait(fd, e,
-				  TEST_BUSY | TEST_TRAILING_IDLE);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-busy")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_BUSY);
 
-		igt_subtest_f("semaphore-busy-%s", e->name)
-			sema_busy(fd, e, 0);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-wait")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_WAIT);
 
-		/**
-		 * Check that two perf clients do not influence each
-		 * others observations.
-		 */
-		igt_subtest_f("multi-client-%s", e->name)
-			multi_client(fd, e);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-sema")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_SEMA);
+
+	for (i = 0; i < ARRAY_SIZE(busyidle_tests); i++) {
+		igt_describe(busyidle_tests[i].desc);
+		igt_subtest_with_dynamic_subsubtests(busyidle_tests[i].name)
+			__for_each_physical_engine(fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					busyidle_tests[i].func(fd, e, num_engines, busyidle_tests[i].flags);
+	}
 
-		/**
-		 * Check that reported usage is correct when PMU is
-		 * enabled after the batch is running.
-		 */
-		igt_subtest_f("busy-start-%s", e->name)
-			busy_start(fd, e);
+	igt_describe("Check that two perf clients do not influence each other's observations.");
+	igt_subtest_with_dynamic_subsubtests("multi-client")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				multi_client(fd, e);
 
-		/**
-		 * Check that reported usage is correct when PMU is
-		 * enabled after two batches are running.
-		 */
-		igt_subtest_f("busy-double-start-%s", e->name) {
-			gem_require_contexts(fd);
-			busy_double_start(fd, e);
-		}
+	igt_describe("Check that reported usage is correct when PMU is enabled after the batch is running.");
+	igt_subtest_with_dynamic_subsubtests("busy-start")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				busy_start(fd, e);
 
-		/**
-		 * Check that the PMU can be safely enabled in face of
-		 * interrupt-heavy engine load.
-		 */
-		igt_subtest_f("enable-race-%s", e->name)
-			test_enable_race(fd, e);
+	igt_describe("Check that reported usage is correct when PMU is enabled after two batches are running.");
+	igt_subtest_with_dynamic_subsubtests("busy-double-start") {
+		gem_require_contexts(fd);
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				busy_double_start(fd, e);
+	}
 
-		/**
-		 * Check engine busyness accuracy is as expected.
-		 */
-		for (i = 0; i < ARRAY_SIZE(pct); i++) {
-			igt_subtest_f("busy-accuracy-%u-%s",
-				      pct[i], e->name)
-				accuracy(fd, e, pct[i], 10);
-		}
+	igt_describe("Check that the PMU can be safely enabled in face of interrupt-heavy engine load.");
+	igt_subtest_with_dynamic_subsubtests("enable-race")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				test_enable_race(fd, e);
+
+	for (i = 0; i < ARRAY_SIZE(pct); i++) {
+		igt_describe("Check engine busyness accuracy is as expected.");
+		igt_subtest_with_dynamic_subsubtests_f("busy-accuracy-%u", pct[i])
+			__for_each_physical_engine(fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					accuracy(fd, e, pct[i], 10);
+	}
 
-		igt_subtest_f("busy-hang-%s", e->name) {
-			igt_hang_t hang = igt_allow_hang(fd, 0, 0);
+	igt_subtest_with_dynamic_subsubtests("busy-hang")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name) {
+				igt_hang_t hang = igt_allow_hang(fd, 0, 0);
 
-			single(fd, e, TEST_BUSY | FLAG_HANG);
+				single(fd, e, 0, TEST_BUSY | FLAG_HANG);
 
-			igt_disallow_hang(fd, hang);
-		}
+				igt_disallow_hang(fd, hang);
+			}
 
-		/**
-		 * Test that event waits are correctly reported.
-		 */
-		if (e->class == I915_ENGINE_CLASS_RENDER)
-			igt_subtest_f("event-wait-%s", e->name)
-				event_wait(fd, e);
-	}
+	igt_describe("Test that event waits are correctly reported.");
+	igt_subtest_with_dynamic_subsubtests("event-wait")
+		__for_each_physical_engine(fd, e)
+			if (e->class == I915_ENGINE_CLASS_RENDER)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					event_wait(fd, e);
 
 	/**
 	 * Test that when all engines are loaded all loads are
@@ -1975,9 +1942,7 @@ igt_main
 	igt_subtest("rc6-runtime-pm-long")
 		test_rc6(fd, TEST_RUNTIME_PM | FLAG_LONG);
 
-	/**
-	 * Check render nodes are counted.
-	 */
+	igt_describe("Check that render nodes are counted.");
 	igt_subtest_group {
 		int render_fd = -1;
 
@@ -1988,14 +1953,15 @@ igt_main
 			gem_quiescent_gpu(fd);
 		}
 
-		__for_each_physical_engine(render_fd, e) {
-			igt_subtest_f("render-node-busy-%s", e->name)
-				single(render_fd, e, TEST_BUSY);
-			igt_subtest_f("render-node-busy-idle-%s",
-				      e->name)
-				single(render_fd, e,
-				       TEST_BUSY | TEST_TRAILING_IDLE);
-		}
+		igt_subtest_with_dynamic_subsubtests("render-node-busy")
+			__for_each_physical_engine(render_fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					single(render_fd, e, 0, TEST_BUSY);
+
+		igt_subtest_with_dynamic_subsubtests("render-node-busy-idle")
+			__for_each_physical_engine(render_fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					single(render_fd, e, 0, TEST_BUSY | TEST_TRAILING_IDLE);
 
 		igt_fixture {
 			close(render_fd);
Signature · 1 line
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#17 lore raw

Re: [igt-dev] [PATCH i-g-t 7/8] perf_pmu: Use dynamic subtests

Quoted 10 lines · Katarzyna Dec
Quoted 6 lines · Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

In addition, convert comments to igt_describe calls and generally refactor the lot to a series of simpler loops.

Tvrko, is there a chance you could look at dynamic engines usage here and give r-b? It will allow us to merge whole series.

It slipped off my radar that there is actual recent series to look at. :I

Anyway, it looks cool and I think it improves things on more on one front. I also understand that the --dynamic-subtest allows me to isolate one engine so that's good. On this basis:

Regards,

Tvrtko

P.S. I trust Petri that the mechanical bits of the conversion are fine, or CI would have complained.

Quoted 324 lines

Have a nice day!
Kasia :)

Quoted 321 lines
---
  tests/perf_pmu.c | 244 ++++++++++++++++++++---------------------------
  1 file changed, 105 insertions(+), 139 deletions(-)
tests/perf_pmu.c +105 -139
index e2bd2cc5..6a4a600e 100644
@@ -270,7 +270,7 @@ static void end_spin(int fd, igt_spin_t *spin, unsigned int flags)
  }
 
  static void
-single(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
+single(int gem_fd, const struct intel_execution_engine2 *e, unsigned int num_engines, unsigned int flags)
  {
  	unsigned long slept;
  	igt_spin_t *spin;
@@ -606,7 +606,7 @@ all_busy_check_all(int gem_fd, const unsigned int num_engines,
  }
 
  static void
-no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
+no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int num_engines, unsigned int flags)
  {
  	igt_spin_t *spin;
  	uint64_t val[2][2];
@@ -646,7 +646,7 @@ no_sema(int gem_fd, const struct intel_execution_engine2 *e, unsigned int flags)
 
  static void
  sema_wait(int gem_fd, const struct intel_execution_engine2 *e,
-	  unsigned int flags)
+	  unsigned int num_engines, unsigned int flags)
  {
  	struct drm_i915_gem_relocation_entry reloc[2] = {};
  	struct drm_i915_gem_exec_object2 obj[2] = {};
@@ -809,6 +809,7 @@ __sema_busy(int gem_fd, int pmu,
  static void
  sema_busy(int gem_fd,
  	  const struct intel_execution_engine2 *e,
+	  unsigned int num_engines,
  	  unsigned int flags)
  {
  	const struct intel_execution_engine2 *signal;
@@ -1770,6 +1771,27 @@ igt_main
  	int fd = -1;
  	struct intel_execution_engine2 *e;
  	unsigned int i;
+	const unsigned int pct[] = { 2, 50, 98 };
+	struct {
+		void (*func)(int, const struct intel_execution_engine2 *, unsigned int, unsigned int);
+		const char *name;
+		unsigned int flags;
+		const char *desc;
+	} busyidle_tests[] = {
+		{ single, "idle", 0, "Test that engines show no load when idle." },
+		{ single, "busy", TEST_BUSY, "Test that a single engine reports load correctly." },
+		{ single, "busy-idle", TEST_BUSY | TEST_TRAILING_IDLE, "Test that a single engine reports load correctly." },
+		{ busy_check_all, "busy-check-all", TEST_BUSY, "Test that when one engine is loaded, others report no load." },
+		{ busy_check_all, "busy-idle-check-all", TEST_BUSY | TEST_TRAILING_IDLE, "Test that one one engine is loaded, others report no load." },
+		{ most_busy_check_all, "most-busy-check-all", TEST_BUSY, "Test that when all except one engine are loaded all loads are correctly reported." },
+		{ most_busy_check_all, "most-busy-idle-check-all", TEST_BUSY | TEST_TRAILING_IDLE, "Test that when all except one engine are loaded all loads are correctly reported." },
+		{ no_sema, "idle-no-semaphores", 0, "Test that semaphore counters report no activity on idle engines." },
+		{ no_sema, "busy-no-semaphores", TEST_BUSY, "Test that semaphore counters report no activity on busy engines." },
+		{ no_sema, "busy-idle-no-semaphores", TEST_BUSY | TEST_TRAILING_IDLE, "Test that semaphore counters report no activity on idle or busy engines." },
+		{ sema_wait, "semaphore-wait", TEST_BUSY, "Test that semaphore waits are correctly reported." },
+		{ sema_wait, "semaphore-wait-idle", TEST_BUSY | TEST_TRAILING_IDLE, "Test that semaphore waits are correctly reported." },
+		{ sema_busy, "semaphore-busy", 0, "Test that semaphore time and busy time difference is under a threshold." },
+	};
 
  	igt_fixture {
  		fd = drm_open_driver_master(DRIVER_INTEL);
@@ -1781,144 +1803,89 @@ igt_main
  			num_engines++;
  	}
 
-	/**
-	 * Test invalid access via perf API is rejected.
-	 */
+	igt_describe("Test that invalid access via perf API is rejected.");
  	igt_subtest("invalid-init")
  		invalid_init();
 
-	__for_each_physical_engine(fd, e) {
-		const unsigned int pct[] = { 2, 50, 98 };
-
-		/**
-		 * Test that a single engine metric can be initialized or it
-		 * is correctly rejected.
-		 */
-		igt_subtest_f("init-busy-%s", e->name)
-			init(fd, e, I915_SAMPLE_BUSY);
-
-		igt_subtest_f("init-wait-%s", e->name)
-			init(fd, e, I915_SAMPLE_WAIT);
-
-		igt_subtest_f("init-sema-%s", e->name)
-			init(fd, e, I915_SAMPLE_SEMA);
-
-		/**
-		 * Test that engines show no load when idle.
-		 */
-		igt_subtest_f("idle-%s", e->name)
-			single(fd, e, 0);
-
-		/**
-		 * Test that a single engine reports load correctly.
-		 */
-		igt_subtest_f("busy-%s", e->name)
-			single(fd, e, TEST_BUSY);
-		igt_subtest_f("busy-idle-%s", e->name)
-			single(fd, e, TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that when one engine is loaded other report no
-		 * load.
-		 */
-		igt_subtest_f("busy-check-all-%s", e->name)
-			busy_check_all(fd, e, num_engines, TEST_BUSY);
-		igt_subtest_f("busy-idle-check-all-%s", e->name)
-			busy_check_all(fd, e, num_engines,
-				       TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that when all except one engine are loaded all
-		 * loads are correctly reported.
-		 */
-		igt_subtest_f("most-busy-check-all-%s", e->name)
-			most_busy_check_all(fd, e, num_engines,
-					    TEST_BUSY);
-		igt_subtest_f("most-busy-idle-check-all-%s", e->name)
-			most_busy_check_all(fd, e, num_engines,
-					    TEST_BUSY |
-					    TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that semphore counters report no activity on
-		 * idle or busy engines.
-		 */
-		igt_subtest_f("idle-no-semaphores-%s", e->name)
-			no_sema(fd, e, 0);
-
-		igt_subtest_f("busy-no-semaphores-%s", e->name)
-			no_sema(fd, e, TEST_BUSY);
-
-		igt_subtest_f("busy-idle-no-semaphores-%s", e->name)
-			no_sema(fd, e, TEST_BUSY | TEST_TRAILING_IDLE);
-
-		/**
-		 * Test that semaphore waits are correctly reported.
-		 */
-		igt_subtest_f("semaphore-wait-%s", e->name)
-			sema_wait(fd, e, TEST_BUSY);
-
-		igt_subtest_f("semaphore-wait-idle-%s", e->name)
-			sema_wait(fd, e,
-				  TEST_BUSY | TEST_TRAILING_IDLE);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-busy")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_BUSY);
 
-		igt_subtest_f("semaphore-busy-%s", e->name)
-			sema_busy(fd, e, 0);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-wait")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_WAIT);
 
-		/**
-		 * Check that two perf clients do not influence each
-		 * others observations.
-		 */
-		igt_subtest_f("multi-client-%s", e->name)
-			multi_client(fd, e);
+	igt_describe("Test that a single engine metric can be initialized "
+		     "or it is correctly rejected.");
+	igt_subtest_with_dynamic_subsubtests("init-sema")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				init(fd, e, I915_SAMPLE_SEMA);
+
+	for (i = 0; i < ARRAY_SIZE(busyidle_tests); i++) {
+		igt_describe(busyidle_tests[i].desc);
+		igt_subtest_with_dynamic_subsubtests(busyidle_tests[i].name)
+			__for_each_physical_engine(fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					busyidle_tests[i].func(fd, e, num_engines, busyidle_tests[i].flags);
+	}
 
-		/**
-		 * Check that reported usage is correct when PMU is
-		 * enabled after the batch is running.
-		 */
-		igt_subtest_f("busy-start-%s", e->name)
-			busy_start(fd, e);
+	igt_describe("Check that two perf clients do not influence each other's observations.");
+	igt_subtest_with_dynamic_subsubtests("multi-client")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				multi_client(fd, e);
 
-		/**
-		 * Check that reported usage is correct when PMU is
-		 * enabled after two batches are running.
-		 */
-		igt_subtest_f("busy-double-start-%s", e->name) {
-			gem_require_contexts(fd);
-			busy_double_start(fd, e);
-		}
+	igt_describe("Check that reported usage is correct when PMU is enabled after the batch is running.");
+	igt_subtest_with_dynamic_subsubtests("busy-start")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				busy_start(fd, e);
 
-		/**
-		 * Check that the PMU can be safely enabled in face of
-		 * interrupt-heavy engine load.
-		 */
-		igt_subtest_f("enable-race-%s", e->name)
-			test_enable_race(fd, e);
+	igt_describe("Check that reported usage is correct when PMU is enabled after two batches are running.");
+	igt_subtest_with_dynamic_subsubtests("busy-double-start") {
+		gem_require_contexts(fd);
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				busy_double_start(fd, e);
+	}
 
-		/**
-		 * Check engine busyness accuracy is as expected.
-		 */
-		for (i = 0; i < ARRAY_SIZE(pct); i++) {
-			igt_subtest_f("busy-accuracy-%u-%s",
-				      pct[i], e->name)
-				accuracy(fd, e, pct[i], 10);
-		}
+	igt_describe("Check that the PMU can be safely enabled in face of interrupt-heavy engine load.");
+	igt_subtest_with_dynamic_subsubtests("enable-race")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name)
+				test_enable_race(fd, e);
+
+	for (i = 0; i < ARRAY_SIZE(pct); i++) {
+		igt_describe("Check engine busyness accuracy is as expected.");
+		igt_subtest_with_dynamic_subsubtests_f("busy-accuracy-%u", pct[i])
+			__for_each_physical_engine(fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					accuracy(fd, e, pct[i], 10);
+	}
 
-		igt_subtest_f("busy-hang-%s", e->name) {
-			igt_hang_t hang = igt_allow_hang(fd, 0, 0);
+	igt_subtest_with_dynamic_subsubtests("busy-hang")
+		__for_each_physical_engine(fd, e)
+			igt_dynamic_subsubtest_f("%s", e->name) {
+				igt_hang_t hang = igt_allow_hang(fd, 0, 0);
 
-			single(fd, e, TEST_BUSY | FLAG_HANG);
+				single(fd, e, 0, TEST_BUSY | FLAG_HANG);
 
-			igt_disallow_hang(fd, hang);
-		}
+				igt_disallow_hang(fd, hang);
+			}
 
-		/**
-		 * Test that event waits are correctly reported.
-		 */
-		if (e->class == I915_ENGINE_CLASS_RENDER)
-			igt_subtest_f("event-wait-%s", e->name)
-				event_wait(fd, e);
-	}
+	igt_describe("Test that event waits are correctly reported.");
+	igt_subtest_with_dynamic_subsubtests("event-wait")
+		__for_each_physical_engine(fd, e)
+			if (e->class == I915_ENGINE_CLASS_RENDER)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					event_wait(fd, e);
 
  	/**
  	 * Test that when all engines are loaded all loads are
@@ -1975,9 +1942,7 @@ igt_main
  	igt_subtest("rc6-runtime-pm-long")
  		test_rc6(fd, TEST_RUNTIME_PM | FLAG_LONG);
 
-	/**
-	 * Check render nodes are counted.
-	 */
+	igt_describe("Check that render nodes are counted.");
  	igt_subtest_group {
  		int render_fd = -1;
 
@@ -1988,14 +1953,15 @@ igt_main
  			gem_quiescent_gpu(fd);
  		}
 
-		__for_each_physical_engine(render_fd, e) {
-			igt_subtest_f("render-node-busy-%s", e->name)
-				single(render_fd, e, TEST_BUSY);
-			igt_subtest_f("render-node-busy-idle-%s",
-				      e->name)
-				single(render_fd, e,
-				       TEST_BUSY | TEST_TRAILING_IDLE);
-		}
+		igt_subtest_with_dynamic_subsubtests("render-node-busy")
+			__for_each_physical_engine(render_fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					single(render_fd, e, 0, TEST_BUSY);
+
+		igt_subtest_with_dynamic_subsubtests("render-node-busy-idle")
+			__for_each_physical_engine(render_fd, e)
+				igt_dynamic_subsubtest_f("%s", e->name)
+					single(render_fd, e, 0, TEST_BUSY | TEST_TRAILING_IDLE);
 
Quoted 10 lines
Quoted 5 lines
tests/perf_pmu.c +0 -0
  		igt_fixture {
  			close(render_fd);
Signature · 1 line
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#18 lore raw

PATCH 8/8 i915/gem_exec_basic: Use dynamic subtests

In reply to Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

---
 tests/i915/gem_exec_basic.c | 26 +++++++++++++++++++-------
 1 file changed, 19 insertions(+), 7 deletions(-)
tests/i915/gem_exec_basic.c +19 -7
index 1287860b..b2d6ddc2 100644
@@ -159,13 +159,25 @@ igt_main
 	igt_subtest("gtt-all")
 		gtt_all(fd);
 
-	__for_each_physical_engine(fd, e) {
-		igt_subtest_f("basic-%s", e->name)
-			noop(fd, e->flags);
-		igt_subtest_f("readonly-%s", e->name)
-			readonly(fd, e->flags);
-		igt_subtest_f("gtt-%s", e->name)
-			gtt(fd, e->flags);
+	igt_subtest_with_dynamic_subsubtests("basic") {
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				noop(fd, e->flags);
+		}
+	}
+
+	igt_subtest_with_dynamic_subsubtests("readonly") {
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				readonly(fd, e->flags);
+		}
+	}
+
+	igt_subtest_with_dynamic_subsubtests("gtt") {
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				gtt(fd, e->flags);
+		}
 	}
 
 	igt_fixture {
Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Reviewed-by on v1 8/8
#19 lore raw

Re: [igt-dev] [PATCH i-g-t 8/8] i915/gem_exec_basic: Use dynamic subtests

Quoted 49 lines · Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

---
 tests/i915/gem_exec_basic.c | 26 +++++++++++++++++++-------
 1 file changed, 19 insertions(+), 7 deletions(-)
tests/i915/gem_exec_basic.c +19 -7
index 1287860b..b2d6ddc2 100644
@@ -159,13 +159,25 @@ igt_main
 	igt_subtest("gtt-all")
 		gtt_all(fd);
 
-	__for_each_physical_engine(fd, e) {
-		igt_subtest_f("basic-%s", e->name)
-			noop(fd, e->flags);
-		igt_subtest_f("readonly-%s", e->name)
-			readonly(fd, e->flags);
-		igt_subtest_f("gtt-%s", e->name)
-			gtt(fd, e->flags);
+	igt_subtest_with_dynamic_subsubtests("basic") {
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				noop(fd, e->flags);
+		}
+	}
+
+	igt_subtest_with_dynamic_subsubtests("readonly") {
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				readonly(fd, e->flags);
+		}
+	}
+
+	igt_subtest_with_dynamic_subsubtests("gtt") {
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				gtt(fd, e->flags);
+		}
 	}
 
 	igt_fixture {
Signature · 1 line
2.19.1

LGTM :)

Kasia :)
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#20 lore raw

Re: [igt-dev] [PATCH i-g-t 8/8] i915/gem_exec_basic: Use dynamic subtests

In reply to Petri Latvala
Quoted 15 lines · Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

---
  tests/i915/gem_exec_basic.c | 26 +++++++++++++++++++-------
  1 file changed, 19 insertions(+), 7 deletions(-)
tests/i915/gem_exec_basic.c +0 -0
index 1287860b..b2d6ddc2 100644
@@ -159,13 +159,25 @@ igt_main
  	igt_subtest("gtt-all")

tests/i915/gem_exec_basic.c igt_main · line 162

Quoted 10 lines
tests/i915/gem_exec_basic.c +1 -7
  		gtt_all(fd);
 
-	__for_each_physical_engine(fd, e) {
-		igt_subtest_f("basic-%s", e->name)
-			noop(fd, e->flags);
-		igt_subtest_f("readonly-%s", e->name)
-			readonly(fd, e->flags);
-		igt_subtest_f("gtt-%s", e->name)
-			gtt(fd, e->flags);
+	igt_subtest_with_dynamic_subsubtests("basic") {

Bikeshed on the name maybe, since "subsub" reads a bit funny.

igt_dynamic_subtest_group ?

Regards,

Tvrtko

Quoted 12 lines
tests/i915/gem_exec_basic.c +11 -0
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				noop(fd, e->flags);
+		}
+	}
+
+	igt_subtest_with_dynamic_subsubtests("readonly") {
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				readonly(fd, e->flags);
+		}

tests/i915/gem_exec_basic.c igt_main · line 183

Quoted 10 lines
tests/i915/gem_exec_basic.c +7 -0
+	}
+
+	igt_subtest_with_dynamic_subsubtests("gtt") {
+		__for_each_physical_engine(fd, e) {
+			igt_dynamic_subsubtest_f("%s", e->name)
+				gtt(fd, e->flags);
+		}
  	}
 
  	igt_fixture {

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#21 lore raw

Re: [igt-dev] [PATCH i-g-t 8/8] i915/gem_exec_basic: Use dynamic subtests

Quoted 29 lines · Tvrtko Ursulin
Quoted 14 lines · Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

---
  tests/i915/gem_exec_basic.c | 26 +++++++++++++++++++-------

  1 file changed, 19 insertions(+), 7 deletions(-)
tests/i915/gem_exec_basic.c +0 -0
index 1287860b..b2d6ddc2 100644

tests/i915/gem_exec_basic.c igt_main · line 161

Quoted 10 lines
tests/i915/gem_exec_basic.c +1 -7
@@ -159,13 +159,25 @@ igt_main
       igt_subtest("gtt-all")
           gtt_all(fd);
-    __for_each_physical_engine(fd, e) {
-        igt_subtest_f("basic-%s", e->name)
-            noop(fd, e->flags);
-        igt_subtest_f("readonly-%s", e->name)
-            readonly(fd, e->flags);
-        igt_subtest_f("gtt-%s", e->name)
-            gtt(fd, e->flags);
+    igt_subtest_with_dynamic_subsubtests("basic") {

Bikeshed on the name maybe, since "subsub" reads a bit funny.

Quoted 1 line

igt_dynamic_subtest_group ?

Assuming at least it can both nest under igt_subtest_group and not nest ie. be at top-level? Do we need a nested version even?

Regards,

Tvrtko

Quoted 21 lines

Regards,

Tvrtko

Quoted 17 lines
tests/i915/gem_exec_basic.c +17 -0
+        __for_each_physical_engine(fd, e) {
+            igt_dynamic_subsubtest_f("%s", e->name)
+                noop(fd, e->flags);
+        }
+    }
+
+    igt_subtest_with_dynamic_subsubtests("readonly") {
+        __for_each_physical_engine(fd, e) {
+            igt_dynamic_subsubtest_f("%s", e->name)
+                readonly(fd, e->flags);
+        }
+    }
+
+    igt_subtest_with_dynamic_subsubtests("gtt") {
+        __for_each_physical_engine(fd, e) {
+            igt_dynamic_subsubtest_f("%s", e->name)
+                gtt(fd, e->flags);
Quoted 8 lines

tests/i915/gem_exec_basic.c igt_main

Quoted 4 lines
tests/i915/gem_exec_basic.c +1 -0
+        }
       }
       igt_fixture {
 

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#22 lore raw

Re: [igt-dev] [PATCH i-g-t 8/8] i915/gem_exec_basic: Use dynamic subtests

Quoted 31 lines · Tvrtko Ursulin
Quoted 29 lines · Tvrtko Ursulin
Quoted 14 lines · Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

---
  tests/i915/gem_exec_basic.c | 26 +++++++++++++++++++-------

  1 file changed, 19 insertions(+), 7 deletions(-)
tests/i915/gem_exec_basic.c +0 -0
index 1287860b..b2d6ddc2 100644

tests/i915/gem_exec_basic.c igt_main · line 161

Quoted 10 lines
tests/i915/gem_exec_basic.c +1 -7
@@ -159,13 +159,25 @@ igt_main
       igt_subtest("gtt-all")
           gtt_all(fd);
-    __for_each_physical_engine(fd, e) {
-        igt_subtest_f("basic-%s", e->name)
-            noop(fd, e->flags);
-        igt_subtest_f("readonly-%s", e->name)
-            readonly(fd, e->flags);
-        igt_subtest_f("gtt-%s", e->name)
-            gtt(fd, e->flags);
+    igt_subtest_with_dynamic_subsubtests("basic") {

Bikeshed on the name maybe, since "subsub" reads a bit funny.

Quoted 4 lines
Quoted 1 line

igt_dynamic_subtest_group ?

Assuming at least it can both nest under igt_subtest_group and not nest ie. be at top-level? Do we need a nested version even?

It's a brother of igt_subtest, not a brother of igt_subtest_group. igt_subtest_with_dynamic_subsubtests() is, like igt_subtest(),

* execution entry with --run-subtest
* enumerable with --list-subtests
* a block where you can touch the system inside it
* nests inside an igt_subtest_group and automatically produces a 'SKIP' if a fixture in that group triggered an igt_require

I'm all for bikeshedding on the name, but I want the closer association to igt_subtest to show. I like none of the names discussed so far all around the mailing list, I hate this current one the least so far. If it's all right, I'd like to land the series and start converting tests while the naming bikeshed (again, very welcome bikeshed) goes on simultaneously. It's a matter of a quick sed magickry later.

Signature · 5 lines
Petri Latvala
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#23 lore raw

Re: [igt-dev] [PATCH i-g-t 8/8] i915/gem_exec_basic: Use dynamic subtests

Quoted 47 lines · Petri Latvala
Quoted 31 lines · Tvrtko Ursulin
Quoted 29 lines · Tvrtko Ursulin
Quoted 14 lines · Petri Latvala

Instead of generating a subtest for each engine in a static list, convert to dynamic subtests, with one dynamic subtest per actually present physical engine.

---
   tests/i915/gem_exec_basic.c | 26 +++++++++++++++++++-------
   1 file changed, 19 insertions(+), 7 deletions(-)
tests/i915/gem_exec_basic.c +0 -0
index 1287860b..b2d6ddc2 100644

tests/i915/gem_exec_basic.c igt_main · line 161

Quoted 10 lines
tests/i915/gem_exec_basic.c +1 -7
@@ -159,13 +159,25 @@ igt_main
       igt_subtest("gtt-all")
           gtt_all(fd);
-    __for_each_physical_engine(fd, e) {
-        igt_subtest_f("basic-%s", e->name)
-            noop(fd, e->flags);
-        igt_subtest_f("readonly-%s", e->name)
-            readonly(fd, e->flags);
-        igt_subtest_f("gtt-%s", e->name)
-            gtt(fd, e->flags);
+    igt_subtest_with_dynamic_subsubtests("basic") {

Bikeshed on the name maybe, since "subsub" reads a bit funny.

Quoted 4 lines
Quoted 1 line

igt_dynamic_subtest_group ?

Assuming at least it can both nest under igt_subtest_group and not nest ie. be at top-level? Do we need a nested version even?

It's a brother of igt_subtest, not a brother of igt_subtest_group. igt_subtest_with_dynamic_subsubtests() is, like igt_subtest(),

* execution entry with --run-subtest
* enumerable with --list-subtests
* a block where you can touch the system inside it
* nests inside an igt_subtest_group and automatically produces a 'SKIP' if a fixture in that group triggered an igt_require

Quoted 7 lines

I'm all for bikeshedding on the name, but I want the closer association to igt_subtest to show. I like none of the names discussed so far all around the mailing list, I hate this current one the least so far. If it's all right, I'd like to land the series and start converting tests while the naming bikeshed (again, very welcome bikeshed) goes on simultaneously. It's a matter of a quick sed magickry later.

Fine with me.

Regards,

Tvrtko

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#24 lore raw

[igt-dev] ✗ GitLab.Pipeline: warning for Dynamic subtests (rev3)

In reply to Petri Latvala
== Series Details ==

== Series Details ==

Series: Dynamic subtests (rev3)
URL : https://patchwork.freedesktop.org/series/62384/
State : warning

== Summary ==

ERROR! This series introduces new undocumented tests:

gem_exec_basic@basic
gem_exec_basic@gtt
gem_exec_basic@readonly
kms_plane_cursor@overlay-size-128
kms_plane_cursor@overlay-size-256
kms_plane_cursor@overlay-size-64
kms_plane_cursor@primary-size-128
kms_plane_cursor@primary-size-256
kms_plane_cursor@primary-size-64
kms_plane_cursor@viewport-size-128
kms_plane_cursor@viewport-size-256
kms_plane_cursor@viewport-size-64
perf_pmu@busy-hang

Can you document them as per the requirement in the [CONTRIBUTING.md]?

[Documentation] has more details on how to do this.

Here are few examples:
https://gitlab.freedesktop.org/drm/igt-gpu-tools/commit/0316695d03aa46108296b27f3982ec93200c7a6e
https://gitlab.freedesktop.org/drm/igt-gpu-tools/commit/443cc658e1e6b492ee17bf4f4d891029eb7a205d

Thanks in advance!

[CONTRIBUTING.md]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/CONTRIBUTING.md#L19
[Documentation]: https://drm.pages.freedesktop.org/igt-gpu-tools/igt-gpu-tools-Core.html#igt-describe

Other than that, pipeline status: SUCCESS.

see https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/pipelines/73127 for more details

== Logs ==

For more details see: https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/pipelines/73127
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#25 lore raw

[igt-dev] ✓ Fi.CI.BAT: success for Dynamic subtests (rev3)

In reply to Petri Latvala
== Series Details ==

== Series Details ==

Series: Dynamic subtests (rev3)
URL : https://patchwork.freedesktop.org/series/62384/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7172 -> IGTPW_3605
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/index.html

Known issues
------------

Here are the changes found in IGTPW_3605 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_ctx_switch@legacy-render:
- fi-apl-guc: [PASS][1] -> [INCOMPLETE][2] ([fdo#103927] / [fdo#111381])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7172/fi-apl-guc/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/fi-apl-guc/igt@[email protected]
- fi-bxt-dsi: [PASS][3] -> [INCOMPLETE][4] ([fdo#103927] / [fdo#111381])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7172/fi-bxt-dsi/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/fi-bxt-dsi/igt@[email protected]

* igt@i915_pm_rpm@module-reload:
- fi-skl-6770hq: [PASS][5] -> [FAIL][6] ([fdo#108511])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7172/fi-skl-6770hq/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/fi-skl-6770hq/igt@[email protected]

#### Warnings ####

* igt@kms_chamelium@hdmi-hpd-fast:
- fi-kbl-7500u: [FAIL][7] ([fdo#111045] / [fdo#111096]) -> [FAIL][8] ([fdo#111407])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7172/fi-kbl-7500u/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/fi-kbl-7500u/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[fdo#103927]: https://bugs.freedesktop.org/show_bug.cgi?id=103927
[fdo#108511]: https://bugs.freedesktop.org/show_bug.cgi?id=108511
[fdo#111045]: https://bugs.freedesktop.org/show_bug.cgi?id=111045
[fdo#111096]: https://bugs.freedesktop.org/show_bug.cgi?id=111096
[fdo#111381]: https://bugs.freedesktop.org/show_bug.cgi?id=111381
[fdo#111407]: https://bugs.freedesktop.org/show_bug.cgi?id=111407
[fdo#112096]: https://bugs.freedesktop.org/show_bug.cgi?id=112096

Participating hosts (52 -> 43)
------------------------------

Missing (9): fi-ilk-m540 fi-hsw-4200u fi-tgl-u2 fi-byt-squawks fi-bsw-cyan fi-icl-y fi-skl-lmem fi-byt-clapper fi-bdw-samus

Build changes
-------------

* CI: CI-20190529 -> None
* IGT: IGT_5237 -> IGTPW_3605

CI-20190529: 20190529
CI_DRM_7172: c6e7ee3e8a2d06a8934ad5298208e3867994e6e8 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_3605: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/index.html
IGT_5237: 9a46404de7c42c8cc2d492176e956597ef28d7c4 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Testlist changes ==

+++ 39 lines
--- 229 lines

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/index.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#26 lore raw

[igt-dev] ✓ Fi.CI.IGT: success for Dynamic subtests (rev3)

In reply to Petri Latvala
== Series Details ==

== Series Details ==

Series: Dynamic subtests (rev3)
URL : https://patchwork.freedesktop.org/series/62384/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7172_full -> IGTPW_3605_full
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/index.html

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in IGTPW_3605_full:

### IGT changes ###

#### Possible regressions ####

* {igt@perf_pmu@busy} (NEW):
- {shard-tglb}: NOTRUN -> [WARN][1] +6 similar issues
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/shard-tglb2/igt@[email protected]

* {igt@perf_pmu@semaphore-wait} (NEW):
- shard-snb: NOTRUN -> [WARN][2]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/shard-snb2/igt@[email protected]

New tests
---------

New tests have been introduced between CI_DRM_7172_full and IGTPW_3605_full:

### New IGT tests (216) ###

* igt@gem_exec_basic@basic:
- Statuses : 6 pass(s)
- Exec time: [0.00, 0.01] s

* igt@gem_exec_basic@basic@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@basic@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@basic@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@basic@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@basic@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@gtt:
- Statuses : 7 pass(s)
- Exec time: [0.00, 0.01] s

* igt@gem_exec_basic@gtt@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@gtt@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@gtt@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@gtt@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@gtt@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@readonly:
- Statuses : 7 pass(s)
- Exec time: [0.00, 0.01] s

* igt@gem_exec_basic@readonly@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@readonly@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@readonly@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@readonly@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@readonly@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@kms_plane_cursor@overlay-size-128:
- Statuses : 6 pass(s)
- Exec time: [5.53, 9.62] s

* igt@kms_plane_cursor@overlay-size-128@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.18, 2.73] s

* igt@kms_plane_cursor@overlay-size-128@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.24, 3.58] s

* igt@kms_plane_cursor@overlay-size-128@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.23, 3.58] s

* igt@kms_plane_cursor@overlay-size-256:
- Statuses : 6 pass(s)
- Exec time: [5.53, 9.62] s

* igt@kms_plane_cursor@overlay-size-256@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.20, 2.73] s

* igt@kms_plane_cursor@overlay-size-256@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.23, 3.58] s

* igt@kms_plane_cursor@overlay-size-256@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.24, 3.58] s

* igt@kms_plane_cursor@overlay-size-64:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [5.50, 46.41] s

* igt@kms_plane_cursor@overlay-size-64@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.18, 2.71] s

* igt@kms_plane_cursor@overlay-size-64@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.22, 3.61] s

* igt@kms_plane_cursor@overlay-size-64@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.23, 3.58] s

* igt@kms_plane_cursor@primary-size-128:
- Statuses : 7 pass(s)
- Exec time: [4.96, 46.23] s

* igt@kms_plane_cursor@primary-size-128@pipe-a:
- Statuses : 7 pass(s)
- Exec time: [2.17, 37.92] s

* igt@kms_plane_cursor@primary-size-128@pipe-b:
- Statuses : 7 pass(s)
- Exec time: [2.20, 3.56] s

* igt@kms_plane_cursor@primary-size-128@pipe-c:
- Statuses : 6 pass(s)
- Exec time: [2.22, 3.58] s

* igt@kms_plane_cursor@primary-size-128@pipe-d:
- Statuses : 1 pass(s)
- Exec time: [2.75] s

* igt@kms_plane_cursor@primary-size-256:
- Statuses : 7 pass(s)
- Exec time: [4.95, 46.30] s

* igt@kms_plane_cursor@primary-size-256@pipe-a:
- Statuses : 7 pass(s)
- Exec time: [2.17, 38.00] s

* igt@kms_plane_cursor@primary-size-256@pipe-b:
- Statuses : 7 pass(s)
- Exec time: [2.20, 3.56] s

* igt@kms_plane_cursor@primary-size-256@pipe-c:
- Statuses : 6 pass(s)
- Exec time: [2.22, 3.58] s

* igt@kms_plane_cursor@primary-size-256@pipe-d:
- Statuses : 1 pass(s)
- Exec time: [2.75] s

* igt@kms_plane_cursor@primary-size-64:
- Statuses : 7 pass(s)
- Exec time: [4.96, 46.22] s

* igt@kms_plane_cursor@primary-size-64@pipe-a:
- Statuses : 7 pass(s)
- Exec time: [1.63, 37.91] s

* igt@kms_plane_cursor@primary-size-64@pipe-b:
- Statuses : 7 pass(s)
- Exec time: [2.20, 2.80] s

* igt@kms_plane_cursor@primary-size-64@pipe-c:
- Statuses : 6 pass(s)
- Exec time: [2.22, 2.76] s

* igt@kms_plane_cursor@primary-size-64@pipe-d:
- Statuses : 1 pass(s)
- Exec time: [2.75] s

* igt@kms_plane_cursor@viewport-size-128:
- Statuses : 6 pass(s)
- Exec time: [5.53, 9.60] s

* igt@kms_plane_cursor@viewport-size-128@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.19, 2.73] s

* igt@kms_plane_cursor@viewport-size-128@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.24, 3.57] s

* igt@kms_plane_cursor@viewport-size-128@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.23, 3.58] s

* igt@kms_plane_cursor@viewport-size-256:
- Statuses : 7 pass(s)
- Exec time: [5.53, 46.30] s

* igt@kms_plane_cursor@viewport-size-256@pipe-a:
- Statuses : 7 pass(s)
- Exec time: [2.20, 37.93] s

* igt@kms_plane_cursor@viewport-size-256@pipe-b:
- Statuses : 7 pass(s)
- Exec time: [2.24, 3.61] s

* igt@kms_plane_cursor@viewport-size-256@pipe-c:
- Statuses : 6 pass(s)
- Exec time: [2.24, 3.58] s

* igt@kms_plane_cursor@viewport-size-256@pipe-d:
- Statuses : 1 pass(s)
- Exec time: [2.81] s

* igt@kms_plane_cursor@viewport-size-64:
- Statuses : 6 pass(s)
- Exec time: [5.50, 9.67] s

* igt@kms_plane_cursor@viewport-size-64@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.19, 2.70] s

* igt@kms_plane_cursor@viewport-size-64@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.24, 3.62] s

* igt@kms_plane_cursor@viewport-size-64@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.23, 3.58] s

* igt@perf_pmu@busy:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [2.02, 2.57] s

* igt@perf_pmu@busy-accuracy-2:
- Statuses : 7 pass(s)
- Exec time: [0.00, 15.17] s

* igt@perf_pmu@busy-accuracy-2@bcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.05] s

* igt@perf_pmu@busy-accuracy-2@rcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.06] s

* igt@perf_pmu@busy-accuracy-2@vcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.05] s

* igt@perf_pmu@busy-accuracy-2@vcs1:
- Statuses : 3 pass(s)
- Exec time: [3.03] s

* igt@perf_pmu@busy-accuracy-2@vecs0:
- Statuses : 5 pass(s) 1 skip(s)
- Exec time: [0.0, 3.05] s

* igt@perf_pmu@busy-accuracy-50:
- Statuses : 6 pass(s)
- Exec time: [0.0, 7.62] s

* igt@perf_pmu@busy-accuracy-50@bcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 1.54] s

* igt@perf_pmu@busy-accuracy-50@rcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 1.54] s

* igt@perf_pmu@busy-accuracy-50@vcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 1.54] s

* igt@perf_pmu@busy-accuracy-50@vcs1:
- Statuses : 1 pass(s)
- Exec time: [1.52] s

* igt@perf_pmu@busy-accuracy-50@vecs0:
- Statuses : 4 pass(s) 1 skip(s)
- Exec time: [0.0, 1.54] s

* igt@perf_pmu@busy-accuracy-98:
- Statuses : 7 pass(s)
- Exec time: [0.0, 15.11] s

* igt@perf_pmu@busy-accuracy-98@bcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-98@rcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-98@vcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-98@vcs1:
- Statuses : 3 pass(s)
- Exec time: [3.02] s

* igt@perf_pmu@busy-accuracy-98@vecs0:
- Statuses : 5 pass(s) 1 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-check-all:
- Statuses : 7 pass(s)
- Exec time: [2.03, 2.58] s

* igt@perf_pmu@busy-check-all@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy-check-all@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy-check-all@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.51, 1.01] s

* igt@perf_pmu@busy-check-all@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.52] s

* igt@perf_pmu@busy-check-all@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy-double-start:
- Statuses : 6 pass(s)
- Exec time: [11.03, 17.59] s

* igt@perf_pmu@busy-double-start@bcs0:
- Statuses : 6 pass(s)
- Exec time: [3.51, 3.52] s

* igt@perf_pmu@busy-double-start@rcs0:
- Statuses : 6 pass(s)
- Exec time: [3.51, 3.52] s

* igt@perf_pmu@busy-double-start@vcs0:
- Statuses : 6 pass(s)
- Exec time: [3.51, 4.01] s

* igt@perf_pmu@busy-double-start@vcs1:
- Statuses : 1 pass(s)
- Exec time: [3.52] s

* igt@perf_pmu@busy-double-start@vecs0:
- Statuses : 5 pass(s)
- Exec time: [3.51, 3.52] s

* igt@perf_pmu@busy-hang:
- Statuses : 6 pass(s)
- Exec time: [3.53, 5.09] s

* igt@perf_pmu@busy-hang@bcs0:
- Statuses : 6 pass(s)
- Exec time: [1.01, 1.02] s

* igt@perf_pmu@busy-hang@rcs0:
- Statuses : 6 pass(s)
- Exec time: [1.01, 1.02] s

* igt@perf_pmu@busy-hang@vcs0:
- Statuses : 6 pass(s)
- Exec time: [1.01, 1.51] s

* igt@perf_pmu@busy-hang@vcs1:
- Statuses : 1 pass(s)
- Exec time: [1.02] s

* igt@perf_pmu@busy-hang@vecs0:
- Statuses : 5 pass(s)
- Exec time: [1.01, 1.02] s

* igt@perf_pmu@busy-idle:
- Statuses : 6 pass(s)
- Exec time: [2.34, 3.07] s

* igt@perf_pmu@busy-idle-check-all:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [2.33, 3.08] s

* igt@perf_pmu@busy-idle-check-all@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle-check-all@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle-check-all@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 1.11] s

* igt@perf_pmu@busy-idle-check-all@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.61] s

* igt@perf_pmu@busy-idle-check-all@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle-no-semaphores:
- Statuses : 6 pass(s)
- Exec time: [2.31, 3.03] s

* igt@perf_pmu@busy-idle-no-semaphores@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.60, 0.61] s

* igt@perf_pmu@busy-idle-no-semaphores@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.60, 0.61] s

* igt@perf_pmu@busy-idle-no-semaphores@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.60, 1.10] s

* igt@perf_pmu@busy-idle-no-semaphores@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.60, 0.61] s

* igt@perf_pmu@busy-idle-no-semaphores@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.60, 0.61] s

* igt@perf_pmu@busy-idle@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61] s

* igt@perf_pmu@busy-idle@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 1.11] s

* igt@perf_pmu@busy-idle@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.62] s

* igt@perf_pmu@busy-idle@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-no-semaphores:
- Statuses : 5 pass(s)
- Exec time: [2.00, 2.52] s

* igt@perf_pmu@busy-no-semaphores@bcs0:
- Statuses : 5 pass(s)
- Exec time: [0.50, 0.51] s

* igt@perf_pmu@busy-no-semaphores@rcs0:
- Statuses : 5 pass(s)
- Exec time: [0.50, 0.51] s

* igt@perf_pmu@busy-no-semaphores@vcs0:
- Statuses : 5 pass(s)
- Exec time: [0.50, 1.00] s

* igt@perf_pmu@busy-no-semaphores@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@busy-no-semaphores@vecs0:
- Statuses : 4 pass(s)
- Exec time: [0.50, 0.51] s

* igt@perf_pmu@busy-start:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [8.02, 12.58] s

* igt@perf_pmu@busy-start@bcs0:
- Statuses : 6 pass(s)
- Exec time: [2.51, 2.52] s

* igt@perf_pmu@busy-start@rcs0:
- Statuses : 6 pass(s)
- Exec time: [2.51, 2.52] s

* igt@perf_pmu@busy-start@vcs0:
- Statuses : 6 pass(s)
- Exec time: [2.51, 3.01] s

* igt@perf_pmu@busy-start@vcs1:
- Statuses : 1 pass(s)
- Exec time: [2.52] s

* igt@perf_pmu@busy-start@vecs0:
- Statuses : 5 pass(s)
- Exec time: [2.51, 2.52] s

* igt@perf_pmu@busy@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 1.01] s

* igt@perf_pmu@busy@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.51] s

* igt@perf_pmu@busy@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@enable-race:
- Statuses : 7 pass(s)
- Exec time: [0.0, 62.81] s

* igt@perf_pmu@enable-race@bcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 12.62] s

* igt@perf_pmu@enable-race@rcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 12.63] s

* igt@perf_pmu@enable-race@vcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 12.62] s

* igt@perf_pmu@enable-race@vcs1:
- Statuses : 2 pass(s)
- Exec time: [12.56] s

* igt@perf_pmu@enable-race@vecs0:
- Statuses : 5 pass(s) 1 skip(s)
- Exec time: [0.0, 12.63] s

* igt@perf_pmu@event-wait:
- Statuses : 6 pass(s)
- Exec time: [0.0, 2.97] s

* igt@perf_pmu@event-wait@rcs0:
- Statuses : 5 pass(s) 1 skip(s)
- Exec time: [0.0, 2.97] s

* igt@perf_pmu@idle:
- Statuses : 7 pass(s)
- Exec time: [1.50, 2.51] s

* igt@perf_pmu@idle-no-semaphores:
- Statuses : 7 pass(s)
- Exec time: [1.50, 2.51] s

* igt@perf_pmu@idle-no-semaphores@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle-no-semaphores@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50, 0.51] s

* igt@perf_pmu@idle-no-semaphores@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle-no-semaphores@vcs1:
- Statuses : 3 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle-no-semaphores@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@vcs1:
- Statuses : 3 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@init-busy:
- Statuses : 6 pass(s)
- Exec time: [0.00, 0.01] s

* igt@perf_pmu@init-busy@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-busy@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-busy@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-busy@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.0] s

* igt@perf_pmu@init-busy@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-sema:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [0.00, 0.01] s

* igt@perf_pmu@init-sema@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-sema@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-sema@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-sema@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.00] s

* igt@perf_pmu@init-sema@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait:
- Statuses : 7 pass(s)
- Exec time: [0.00, 0.01] s

* igt@perf_pmu@init-wait@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@most-busy-check-all:
- Statuses : 7 pass(s)
- Exec time: [1.52, 2.58] s

* igt@perf_pmu@most-busy-check-all@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-check-all@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-check-all@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-check-all@vcs1:
- Statuses : 3 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-check-all@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-idle-check-all:
- Statuses : 7 pass(s)
- Exec time: [1.83, 3.09] s

* igt@perf_pmu@most-busy-idle-check-all@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@most-busy-idle-check-all@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61] s

* igt@perf_pmu@most-busy-idle-check-all@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@most-busy-idle-check-all@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.62] s

* igt@perf_pmu@most-busy-idle-check-all@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.63] s

* igt@perf_pmu@multi-client:
- Statuses : 1 incomplete(s) 5 pass(s)
- Exec time: [0.0, 5.07] s

* igt@perf_pmu@multi-client@bcs0:
- Statuses : 5 pass(s)
- Exec time: [1.01, 1.02] s

* igt@perf_pmu@multi-client@rcs0:
- Statuses : 5 pass(s)
- Exec time: [1.00, 1.01] s

* igt@perf_pmu@multi-client@vcs0:
- Statuses : 5 pass(s)
- Exec time: [1.01, 1.51] s

* igt@perf_pmu@multi-client@vcs1:
- Statuses : 2 pass(s)
- Exec time: [1.01, 1.02] s

* igt@perf_pmu@multi-client@vecs0:
- Statuses : 4 pass(s)
- Exec time: [1.01] s

* igt@perf_pmu@render-node-busy:
- Statuses : 6 pass(s)
- Exec time: [2.02, 2.58] s

* igt@perf_pmu@render-node-busy-idle:
- Statuses : 7 pass(s)
- Exec time: [2.33, 3.08] s

* igt@perf_pmu@render-node-busy-idle@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@render-node-busy-idle@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@render-node-busy-idle@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61, 1.11] s

* igt@perf_pmu@render-node-busy-idle@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.61] s

* igt@perf_pmu@render-node

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3605/index.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#27 lore raw

[igt-dev] ✓ Fi.CI.BAT: success for Dynamic subtests (rev4)

In reply to Petri Latvala
== Series Details ==

== Series Details ==

Series: Dynamic subtests (rev4)
URL : https://patchwork.freedesktop.org/series/62384/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7303 -> IGTPW_3679
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/index.html

Known issues
------------

Here are the changes found in IGTPW_3679 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_exec_suspend@basic:
- fi-icl-u2: [PASS][1] -> [FAIL][2] ([fdo#111699])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7303/fi-icl-u2/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/fi-icl-u2/igt@[email protected]

* igt@gem_exec_suspend@basic-s3:
- fi-cml-s: [PASS][3] -> [DMESG-WARN][4] ([fdo#111764])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7303/fi-cml-s/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/fi-cml-s/igt@[email protected]

* igt@kms_chamelium@hdmi-hpd-fast:
- fi-kbl-7500u: [PASS][5] -> [FAIL][6] ([fdo#111407])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7303/fi-kbl-7500u/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/fi-kbl-7500u/igt@[email protected]

#### Possible fixes ####

* igt@i915_selftest@live_hangcheck:
- fi-bsw-nick: [INCOMPLETE][7] ([fdo#105876]) -> [PASS][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7303/fi-bsw-nick/igt@i915_selftest@live_hangcheck.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/fi-bsw-nick/igt@i915_selftest@live_hangcheck.html

* igt@kms_busy@basic-flip-c:
- fi-icl-u4: [DMESG-WARN][9] ([fdo#112233]) -> [PASS][10] +6 similar issues
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7303/fi-icl-u4/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/fi-icl-u4/igt@[email protected]

* igt@kms_chamelium@dp-edid-read:
- fi-icl-u2: [DMESG-WARN][11] ([fdo#106107]) -> [PASS][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7303/fi-icl-u2/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/fi-icl-u2/igt@[email protected]

#### Warnings ####

* igt@kms_chamelium@dp-edid-read:
- fi-icl-u4: [DMESG-FAIL][13] ([fdo#110503] / [fdo#111045]) -> [FAIL][14] ([fdo#111045])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7303/fi-icl-u4/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/fi-icl-u4/igt@[email protected]

* igt@kms_chamelium@dp-hpd-fast:
- fi-icl-u4: [DMESG-FAIL][15] ([fdo#112233]) -> [FAIL][16] ([fdo#111045])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7303/fi-icl-u4/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/fi-icl-u4/igt@[email protected]

[fdo#105876]: https://bugs.freedesktop.org/show_bug.cgi?id=105876
[fdo#106107]: https://bugs.freedesktop.org/show_bug.cgi?id=106107
[fdo#110503]: https://bugs.freedesktop.org/show_bug.cgi?id=110503
[fdo#111045]: https://bugs.freedesktop.org/show_bug.cgi?id=111045
[fdo#111407]: https://bugs.freedesktop.org/show_bug.cgi?id=111407
[fdo#111699]: https://bugs.freedesktop.org/show_bug.cgi?id=111699
[fdo#111764]: https://bugs.freedesktop.org/show_bug.cgi?id=111764
[fdo#112233]: https://bugs.freedesktop.org/show_bug.cgi?id=112233

Participating hosts (48 -> 43)
------------------------------

Missing (5): fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-byt-clapper fi-bdw-samus

Build changes
-------------

* CI: CI-20190529 -> None
* IGT: IGT_5268 -> IGTPW_3679

CI-20190529: 20190529
CI_DRM_7303: a3da3f4feaef89adf08c8357067965a9b56a8e66 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_3679: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/index.html
IGT_5268: c94958b8f7caefcda72392417ae6f3a98e36a48b @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Testlist changes ==

+++ 39 lines
--- 229 lines

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/index.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#28 lore raw

[igt-dev] ✗ GitLab.Pipeline: failure for Dynamic subtests (rev4)

In reply to Petri Latvala
== Series Details ==

== Series Details ==

Series: Dynamic subtests (rev4)
URL : https://patchwork.freedesktop.org/series/62384/
State : failure

== Summary ==

ERROR! This series introduces new undocumented tests:

gem_exec_basic@basic
gem_exec_basic@gtt
gem_exec_basic@readonly
kms_plane_cursor@overlay-size-128
kms_plane_cursor@overlay-size-256
kms_plane_cursor@overlay-size-64
kms_plane_cursor@primary-size-128
kms_plane_cursor@primary-size-256
kms_plane_cursor@primary-size-64
kms_plane_cursor@viewport-size-128
kms_plane_cursor@viewport-size-256
kms_plane_cursor@viewport-size-64
perf_pmu@busy-hang

Can you document them as per the requirement in the [CONTRIBUTING.md]?

[Documentation] has more details on how to do this.

Here are few examples:
https://gitlab.freedesktop.org/drm/igt-gpu-tools/commit/0316695d03aa46108296b27f3982ec93200c7a6e
https://gitlab.freedesktop.org/drm/igt-gpu-tools/commit/443cc658e1e6b492ee17bf4f4d891029eb7a205d

Thanks in advance!

[CONTRIBUTING.md]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/blob/master/CONTRIBUTING.md#L19
[Documentation]: https://drm.pages.freedesktop.org/igt-gpu-tools/igt-gpu-tools-Core.html#igt-describe

Other than that, pipeline status: SUCCESS.

see https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/pipelines/77876 for the overview.

== Logs ==

For more details see: https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/pipelines/77876
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#29 lore raw

[igt-dev] ✓ Fi.CI.IGT: success for Dynamic subtests (rev4)

In reply to Petri Latvala
== Series Details ==

== Series Details ==

Series: Dynamic subtests (rev4)
URL : https://patchwork.freedesktop.org/series/62384/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7303_full -> IGTPW_3679_full
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/index.html

Possible new issues
-------------------

Here are the unknown changes that may have been introduced in IGTPW_3679_full:

### IGT changes ###

#### Possible regressions ####

* {igt@kms_plane_cursor@overlay-size-128} (NEW):
- shard-tglb: NOTRUN -> [WARN][1] +6 similar issues
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/shard-tglb5/igt@[email protected]

New tests
---------

New tests have been introduced between CI_DRM_7303_full and IGTPW_3679_full:

### New IGT tests (215) ###

* igt@gem_exec_basic@basic:
- Statuses : 7 pass(s)
- Exec time: [0.00, 0.01] s

* igt@gem_exec_basic@basic@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@basic@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.00, 0.01] s

* igt@gem_exec_basic@basic@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@basic@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.0] s

* igt@gem_exec_basic@basic@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@gtt:
- Statuses : 6 pass(s)
- Exec time: [0.00, 0.01] s

* igt@gem_exec_basic@gtt@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@gtt@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@gtt@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@gtt@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@gtt@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@readonly:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [0.00, 0.01] s

* igt@gem_exec_basic@readonly@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@readonly@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.00] s

* igt@gem_exec_basic@readonly@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@readonly@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.0, 0.00] s

* igt@gem_exec_basic@readonly@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.0, 0.00] s

* igt@kms_plane_cursor@overlay-size-128:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [5.53, 47.52] s

* igt@kms_plane_cursor@overlay-size-128@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.20, 2.73] s

* igt@kms_plane_cursor@overlay-size-128@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.24, 3.57] s

* igt@kms_plane_cursor@overlay-size-128@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.24, 3.58] s

* igt@kms_plane_cursor@overlay-size-256:
- Statuses : 7 pass(s)
- Exec time: [5.53, 46.41] s

* igt@kms_plane_cursor@overlay-size-256@pipe-a:
- Statuses : 7 pass(s)
- Exec time: [2.20, 37.97] s

* igt@kms_plane_cursor@overlay-size-256@pipe-b:
- Statuses : 7 pass(s)
- Exec time: [2.24, 3.62] s

* igt@kms_plane_cursor@overlay-size-256@pipe-c:
- Statuses : 6 pass(s)
- Exec time: [2.25, 3.58] s

* igt@kms_plane_cursor@overlay-size-256@pipe-d:
- Statuses : 1 pass(s)
- Exec time: [2.81] s

* igt@kms_plane_cursor@overlay-size-64:
- Statuses : 7 pass(s)
- Exec time: [5.50, 46.32] s

* igt@kms_plane_cursor@overlay-size-64@pipe-a:
- Statuses : 7 pass(s)
- Exec time: [2.17, 37.95] s

* igt@kms_plane_cursor@overlay-size-64@pipe-b:
- Statuses : 7 pass(s)
- Exec time: [2.22, 3.61] s

* igt@kms_plane_cursor@overlay-size-64@pipe-c:
- Statuses : 6 pass(s)
- Exec time: [2.24, 3.58] s

* igt@kms_plane_cursor@overlay-size-64@pipe-d:
- Statuses : 1 pass(s)
- Exec time: [2.81] s

* igt@kms_plane_cursor@primary-size-128:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [4.96, 46.33] s

* igt@kms_plane_cursor@primary-size-128@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.18, 2.57] s

* igt@kms_plane_cursor@primary-size-128@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.22, 3.59] s

* igt@kms_plane_cursor@primary-size-128@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.22, 3.58] s

* igt@kms_plane_cursor@primary-size-256:
- Statuses : 6 pass(s)
- Exec time: [4.93, 9.59] s

* igt@kms_plane_cursor@primary-size-256@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.16, 2.58] s

* igt@kms_plane_cursor@primary-size-256@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.20, 3.56] s

* igt@kms_plane_cursor@primary-size-256@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.22, 3.58] s

* igt@kms_plane_cursor@primary-size-64:
- Statuses : 6 pass(s)
- Exec time: [4.96, 9.61] s

* igt@kms_plane_cursor@primary-size-64@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.19, 2.56] s

* igt@kms_plane_cursor@primary-size-64@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.22, 3.58] s

* igt@kms_plane_cursor@primary-size-64@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.22, 3.58] s

* igt@kms_plane_cursor@viewport-size-128:
- Statuses : 7 pass(s)
- Exec time: [5.54, 46.41] s

* igt@kms_plane_cursor@viewport-size-128@pipe-a:
- Statuses : 7 pass(s)
- Exec time: [2.25, 37.97] s

* igt@kms_plane_cursor@viewport-size-128@pipe-b:
- Statuses : 7 pass(s)
- Exec time: [2.23, 3.60] s

* igt@kms_plane_cursor@viewport-size-128@pipe-c:
- Statuses : 6 pass(s)
- Exec time: [2.23, 3.58] s

* igt@kms_plane_cursor@viewport-size-128@pipe-d:
- Statuses : 1 pass(s)
- Exec time: [2.81] s

* igt@kms_plane_cursor@viewport-size-256:
- Statuses : 6 pass(s)
- Exec time: [5.53, 9.60] s

* igt@kms_plane_cursor@viewport-size-256@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.19, 2.73] s

* igt@kms_plane_cursor@viewport-size-256@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.24, 3.56] s

* igt@kms_plane_cursor@viewport-size-256@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.24, 3.58] s

* igt@kms_plane_cursor@viewport-size-64:
- Statuses : 6 pass(s)
- Exec time: [5.52, 9.63] s

* igt@kms_plane_cursor@viewport-size-64@pipe-a:
- Statuses : 6 pass(s)
- Exec time: [2.19, 2.73] s

* igt@kms_plane_cursor@viewport-size-64@pipe-b:
- Statuses : 6 pass(s)
- Exec time: [2.22, 3.59] s

* igt@kms_plane_cursor@viewport-size-64@pipe-c:
- Statuses : 5 pass(s)
- Exec time: [2.24, 3.58] s

* igt@perf_pmu@busy:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [2.02, 2.57] s

* igt@perf_pmu@busy-accuracy-2:
- Statuses : 6 pass(s)
- Exec time: [0.00, 15.18] s

* igt@perf_pmu@busy-accuracy-2@bcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-2@rcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-2@vcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-2@vcs1:
- Statuses : 2 pass(s)
- Exec time: [3.04] s

* igt@perf_pmu@busy-accuracy-2@vecs0:
- Statuses : 4 pass(s) 1 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-50:
- Statuses : 7 pass(s)
- Exec time: [0.0, 7.62] s

* igt@perf_pmu@busy-accuracy-50@bcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 1.54] s

* igt@perf_pmu@busy-accuracy-50@rcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 1.54] s

* igt@perf_pmu@busy-accuracy-50@vcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 1.54] s

* igt@perf_pmu@busy-accuracy-50@vcs1:
- Statuses : 2 pass(s)
- Exec time: [1.52] s

* igt@perf_pmu@busy-accuracy-50@vecs0:
- Statuses : 5 pass(s) 1 skip(s)
- Exec time: [0.0, 1.54] s

* igt@perf_pmu@busy-accuracy-98:
- Statuses : 7 pass(s)
- Exec time: [0.00, 15.12] s

* igt@perf_pmu@busy-accuracy-98@bcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-98@rcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-98@vcs0:
- Statuses : 5 pass(s) 2 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-accuracy-98@vcs1:
- Statuses : 3 pass(s)
- Exec time: [3.02] s

* igt@perf_pmu@busy-accuracy-98@vecs0:
- Statuses : 5 pass(s) 1 skip(s)
- Exec time: [0.0, 3.04] s

* igt@perf_pmu@busy-check-all:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [2.03, 2.58] s

* igt@perf_pmu@busy-check-all@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy-check-all@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy-check-all@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 1.01] s

* igt@perf_pmu@busy-check-all@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.52] s

* igt@perf_pmu@busy-check-all@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy-double-start:
- Statuses : 6 pass(s)
- Exec time: [11.03, 17.59] s

* igt@perf_pmu@busy-double-start@bcs0:
- Statuses : 6 pass(s)
- Exec time: [3.51, 3.52] s

* igt@perf_pmu@busy-double-start@rcs0:
- Statuses : 6 pass(s)
- Exec time: [3.51, 3.52] s

* igt@perf_pmu@busy-double-start@vcs0:
- Statuses : 6 pass(s)
- Exec time: [3.51, 4.01] s

* igt@perf_pmu@busy-double-start@vcs1:
- Statuses : 1 pass(s)
- Exec time: [3.52] s

* igt@perf_pmu@busy-double-start@vecs0:
- Statuses : 5 pass(s)
- Exec time: [3.51, 3.52] s

* igt@perf_pmu@busy-hang:
- Statuses : 6 pass(s)
- Exec time: [3.53, 5.08] s

* igt@perf_pmu@busy-hang@bcs0:
- Statuses : 6 pass(s)
- Exec time: [1.01, 1.03] s

* igt@perf_pmu@busy-hang@rcs0:
- Statuses : 6 pass(s)
- Exec time: [1.01, 1.03] s

* igt@perf_pmu@busy-hang@vcs0:
- Statuses : 6 pass(s)
- Exec time: [1.01, 1.51] s

* igt@perf_pmu@busy-hang@vcs1:
- Statuses : 1 pass(s)
- Exec time: [1.02] s

* igt@perf_pmu@busy-hang@vecs0:
- Statuses : 5 pass(s)
- Exec time: [1.01, 1.03] s

* igt@perf_pmu@busy-idle:
- Statuses : 6 pass(s)
- Exec time: [2.33, 3.08] s

* igt@perf_pmu@busy-idle-check-all:
- Statuses : 7 pass(s)
- Exec time: [2.34, 3.08] s

* igt@perf_pmu@busy-idle-check-all@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle-check-all@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle-check-all@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.61, 1.11] s

* igt@perf_pmu@busy-idle-check-all@vcs1:
- Statuses : 3 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle-check-all@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle-no-semaphores:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [2.31, 3.03] s

* igt@perf_pmu@busy-idle-no-semaphores@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.60, 0.61] s

* igt@perf_pmu@busy-idle-no-semaphores@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.60] s

* igt@perf_pmu@busy-idle-no-semaphores@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.60, 1.10] s

* igt@perf_pmu@busy-idle-no-semaphores@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.61] s

* igt@perf_pmu@busy-idle-no-semaphores@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.60, 0.61] s

* igt@perf_pmu@busy-idle@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-idle@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 1.11] s

* igt@perf_pmu@busy-idle@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.62] s

* igt@perf_pmu@busy-idle@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@busy-no-semaphores:
- Statuses : 7 pass(s)
- Exec time: [2.00, 2.53] s

* igt@perf_pmu@busy-no-semaphores@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50, 0.51] s

* igt@perf_pmu@busy-no-semaphores@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@busy-no-semaphores@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50, 1.00] s

* igt@perf_pmu@busy-no-semaphores@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.50, 0.51] s

* igt@perf_pmu@busy-no-semaphores@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.50, 0.51] s

* igt@perf_pmu@busy-start:
- Statuses : 7 pass(s)
- Exec time: [8.02, 12.58] s

* igt@perf_pmu@busy-start@bcs0:
- Statuses : 7 pass(s)
- Exec time: [2.51, 2.52] s

* igt@perf_pmu@busy-start@rcs0:
- Statuses : 7 pass(s)
- Exec time: [2.51, 2.52] s

* igt@perf_pmu@busy-start@vcs0:
- Statuses : 7 pass(s)
- Exec time: [2.51, 3.01] s

* igt@perf_pmu@busy-start@vcs1:
- Statuses : 2 pass(s)
- Exec time: [2.51, 2.52] s

* igt@perf_pmu@busy-start@vecs0:
- Statuses : 6 pass(s)
- Exec time: [2.51, 2.52] s

* igt@perf_pmu@busy@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@busy@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51] s

* igt@perf_pmu@busy@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 1.01] s

* igt@perf_pmu@busy@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.51] s

* igt@perf_pmu@busy@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@enable-race:
- Statuses : 6 pass(s)
- Exec time: [0.00, 62.82] s

* igt@perf_pmu@enable-race@bcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 12.62] s

* igt@perf_pmu@enable-race@rcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 12.63] s

* igt@perf_pmu@enable-race@vcs0:
- Statuses : 4 pass(s) 2 skip(s)
- Exec time: [0.0, 12.62] s

* igt@perf_pmu@enable-race@vcs1:
- Statuses : 2 pass(s)
- Exec time: [12.56, 12.57] s

* igt@perf_pmu@enable-race@vecs0:
- Statuses : 4 pass(s) 1 skip(s)
- Exec time: [0.0, 12.62] s

* igt@perf_pmu@event-wait:
- Statuses : 7 pass(s)
- Exec time: [0.0, 3.41] s

* igt@perf_pmu@event-wait@rcs0:
- Statuses : 6 pass(s) 1 skip(s)
- Exec time: [0.0, 3.41] s

* igt@perf_pmu@idle:
- Statuses : 7 pass(s)
- Exec time: [1.50, 2.51] s

* igt@perf_pmu@idle-no-semaphores:
- Statuses : 7 pass(s)
- Exec time: [1.50, 2.51] s

* igt@perf_pmu@idle-no-semaphores@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle-no-semaphores@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle-no-semaphores@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle-no-semaphores@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle-no-semaphores@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@idle@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.50] s

* igt@perf_pmu@init-busy:
- Statuses : 7 pass(s)
- Exec time: [0.00, 0.01] s

* igt@perf_pmu@init-busy@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-busy@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.00, 0.01] s

* igt@perf_pmu@init-busy@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-busy@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.0] s

* igt@perf_pmu@init-busy@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-sema:
- Statuses : 6 pass(s)
- Exec time: [0.00, 0.01] s

* igt@perf_pmu@init-sema@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-sema@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-sema@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-sema@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.00] s

* igt@perf_pmu@init-sema@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait:
- Statuses : 7 pass(s)
- Exec time: [0.00, 0.01] s

* igt@perf_pmu@init-wait@bcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait@rcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait@vcs0:
- Statuses : 7 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait@vcs1:
- Statuses : 3 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@init-wait@vecs0:
- Statuses : 6 pass(s)
- Exec time: [0.0, 0.00] s

* igt@perf_pmu@most-busy-check-all:
- Statuses : 6 pass(s)
- Exec time: [1.53, 2.58] s

* igt@perf_pmu@most-busy-check-all@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-check-all@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-check-all@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-check-all@vcs1:
- Statuses : 1 pass(s)
- Exec time: [0.52] s

* igt@perf_pmu@most-busy-check-all@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@most-busy-idle-check-all:
- Statuses : 6 pass(s)
- Exec time: [1.84, 3.09] s

* igt@perf_pmu@most-busy-idle-check-all@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@most-busy-idle-check-all@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@most-busy-idle-check-all@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@most-busy-idle-check-all@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@most-busy-idle-check-all@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@multi-client:
- Statuses : 6 pass(s) 1 warn(s)
- Exec time: [3.52, 5.06] s

* igt@perf_pmu@multi-client@bcs0:
- Statuses : 6 pass(s)
- Exec time: [1.01, 1.02] s

* igt@perf_pmu@multi-client@rcs0:
- Statuses : 6 pass(s)
- Exec time: [1.00] s

* igt@perf_pmu@multi-client@vcs0:
- Statuses : 6 pass(s)
- Exec time: [1.01, 1.51] s

* igt@perf_pmu@multi-client@vcs1:
- Statuses : 1 pass(s)
- Exec time: [1.02] s

* igt@perf_pmu@multi-client@vecs0:
- Statuses : 5 pass(s)
- Exec time: [1.01, 1.02] s

* igt@perf_pmu@render-node-busy:
- Statuses : 6 pass(s)
- Exec time: [2.02, 2.58] s

* igt@perf_pmu@render-node-busy-idle:
- Statuses : 6 pass(s)
- Exec time: [2.33, 3.08] s

* igt@perf_pmu@render-node-busy-idle@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@render-node-busy-idle@rcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@render-node-busy-idle@vcs0:
- Statuses : 6 pass(s)
- Exec time: [0.61, 1.11] s

* igt@perf_pmu@render-node-busy-idle@vcs1:
- Statuses : 2 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@render-node-busy-idle@vecs0:
- Statuses : 5 pass(s)
- Exec time: [0.61, 0.62] s

* igt@perf_pmu@render-node-busy@bcs0:
- Statuses : 6 pass(s)
- Exec time: [0.51, 0.52] s

* igt@perf_pmu@render-node-busy@rcs0:

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3679/index.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev