Skip to contentSkip to conversation

Speedup IGT build time

PATCH New Mauro Carvalho Chehab 2 participants 7 messages Jul 5, 2023

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Speedup IGT build time

From: Mauro Carvalho Chehab <[email protected]>

Currently, even a single change on IGT makes the build time too slow.

This is because it will re-generate the check testlist on every single build that touches any file.

Change the logic to generate per-file testlists, with speeds up build time from 22 seconds to 4 seconds.

It can be improved even further after changing the build logic to maintain separate test_executables for i915, kms and xe.

As there's a pending patch series touching those tests, moving them to tests/intel directory, let's not do a conflicting change. Once such patch series gets merged, I'll write a patch with the pending optimization.

Mauro Carvalho Chehab (5):
lib/igt_core: use the macro when checking for list_subtests lib/igt_core: add an option to show the testlist tests/meson.build: create testlists for tests tests/meson.build: add gem_stress to the test_list logic scripts/test_list.py: speedup testlist check logic

 lib/igt_core.c       | 26 ++++++++++++++++++++------
 scripts/test_list.py | 30 ++++++------------------------
 tests/meson.build    | 25 ++++++++++++++++++-------
 3 files changed, 44 insertions(+), 37 deletions(-)
Signature · 1 line
2.40.1
#1 lore raw

PATCH 1/5 lib/igt_core: use the macro when checking for list_subtests

From: Mauro Carvalho Chehab <[email protected]>

We're implementing a new argument to display all tests. Before doing that, make the check for it more generic to prepare for the changes.

---
 lib/igt_core.c | 10 +++++-----
 1 file changed, 5 insertions(+), 5 deletions(-)
lib/igt_core.c +5 -5
index 3ee3a01c3668..9a0029d294a8 100644
@@ -870,7 +870,7 @@ static void print_version(void)
 {
 	struct utsname uts;
 
-	if (list_subtests)
+	if (igt_only_list_subtests())
 		return;
 
 	uname(&uts);
@@ -1198,12 +1198,12 @@ static int common_init(int *argc, char **argv,
 			break;
 		case OPT_RUN_SUBTEST:
 			assert(optarg);
-			if (!list_subtests)
+			if (!igt_only_list_subtests())
 				run_single_subtest = strdup(optarg);
 			break;
 		case OPT_RUN_DYNAMIC_SUBTEST:
 			assert(optarg);
-			if (!list_subtests)
+			if (!igt_only_list_subtests())
 				run_single_dynamic_subtest = strdup(optarg);
 			break;
 		case OPT_DESCRIPTION:
@@ -1265,7 +1265,7 @@ out:
 		/* exit with no error for -h/--help */
 		exit(ret == -1 ? 0 : IGT_EXIT_INVALID);
 
-	if (!list_subtests) {
+	if (!igt_only_list_subtests()) {
 		bind_fbcon(false);
 		igt_kmsg(KMSG_INFO "%s: executing\n", command_str);
 		print_version();
@@ -3098,7 +3098,7 @@ void igt_vlog(const char *domain, enum igt_log_level level, const char *format,
 	if (!thread_id)
 		return;
 
-	if (list_subtests && level <= IGT_LOG_WARN)
+	if (igt_only_list_subtests() && level <= IGT_LOG_WARN)
 		return;
 
 	if (vasprintf(&line, format, args) == -1)
Signature · 1 line
2.40.1
#2 lore raw

PATCH 2/5 lib/igt_core: add an option to show the testlist

In reply to Mauro Carvalho Chehab

From: Mauro Carvalho Chehab <[email protected]>

There are several cases where we need to list the tests the same way as igt_runner displays them.

That should not return errors on failures and should work also when igt_simple_main() macro is used.

Ideally, it should also show dynamic subtests, but this is more complex, specially for Kselftest/KUnit ones. So, for now, let's not handle dynamic ones.

Add an option to produce such testlist.

---
 lib/igt_core.c | 16 +++++++++++++++-
 1 file changed, 15 insertions(+), 1 deletion(-)
lib/igt_core.c +15 -1
index 9a0029d294a8..2ae2cb68835e 100644
@@ -274,6 +274,7 @@ const char *igt_interactive_debug;
 bool igt_skip_crc_compare;
 
 /* subtests helpers */
+static bool show_testlist = false;
 static bool list_subtests = false;
 static bool describe_subtests = false;
 static char *run_single_subtest = NULL;
@@ -326,6 +327,7 @@ enum {
 	 * conflict with core ones
 	 */
 	OPT_LIST_SUBTESTS = 500,
+	OPT_SHOW_TESTLIST,
 	OPT_DESCRIBE_SUBTESTS,
 	OPT_RUN_SUBTEST,
 	OPT_RUN_DYNAMIC_SUBTEST,
@@ -896,6 +898,7 @@ static void print_usage(const char *help_str, bool output_on_stderr)
 
 	fprintf(f, "Usage: %s [OPTIONS]\n", command_str);
 	fprintf(f, "  --list-subtests\n"
+		   "  --show-testlist\n"
 		   "  --run-subtest <pattern>\n"
 		   "  --dynamic-subtest <pattern>\n"
 		   "  --debug[=log-domain]\n"
@@ -1076,6 +1079,7 @@ static int common_init(int *argc, char **argv,
 	int c, option_index = 0, i, x;
 	static struct option long_options[] = {
 		{"list-subtests",     no_argument,       NULL, OPT_LIST_SUBTESTS},
+		{"show-testlist",     no_argument,       NULL, OPT_SHOW_TESTLIST},
 		{"describe",          optional_argument, NULL, OPT_DESCRIBE_SUBTESTS},
 		{"run-subtest",       required_argument, NULL, OPT_RUN_SUBTEST},
 		{"dynamic-subtest",   required_argument, NULL, OPT_RUN_DYNAMIC_SUBTEST},
@@ -1189,6 +1193,9 @@ static int common_init(int *argc, char **argv,
 			if (!run_single_subtest)
 				list_subtests = true;
 			break;
+		case OPT_SHOW_TESTLIST:
+			show_testlist = true;
+			break;
 		case OPT_DESCRIBE_SUBTESTS:
 			if (optarg)
 				run_single_subtest = strdup(optarg);
@@ -1257,6 +1264,10 @@ out:
 			igt_warn("Unknown subtest: %s\n", run_single_subtest);
 			exit(IGT_EXIT_INVALID);
 		}
+		if (show_testlist) {
+			printf("igt@%s\n", igt_test_name());
+			exit(0);
+		}
 		if (list_subtests)
 			exit(IGT_EXIT_INVALID);
 	}
@@ -1449,6 +1460,9 @@ bool __igt_run_subtest(const char *subtest_name, const char *file, const int lin
 		__igt_print_description(subtest_name, file, line);
 		_clear_current_description();
 		return false;
+	} else if (show_testlist) {
+		printf("igt@%s@%s\n", igt_test_name(), subtest_name);
+		return false;
 	} else if (list_subtests) {
 		printf("%s\n", subtest_name);
 		return false;
@@ -1522,7 +1536,7 @@ const char *igt_subtest_name(void)
  */
 bool igt_only_list_subtests(void)
 {
-	return list_subtests;
+	return list_subtests || show_testlist;
 }
 
 
Signature · 1 line
2.40.1
#3 lore raw

PATCH 3/5 tests/meson.build: create testlists for tests

In reply to Mauro Carvalho Chehab

From: Mauro Carvalho Chehab <[email protected]>

Let's dynamically create testlists for during build time, as this can speed up a lot the validation check for testplan.

---
 tests/meson.build | 12 +++++++++++-
 1 file changed, 11 insertions(+), 1 deletion(-)
tests/meson.build +11 -1
index ee066b84900b..246a585aa423 100644
@@ -545,7 +545,17 @@ test_list_full_target = custom_target('testlist-full',
 	      install_dir : libexecdir)
 
 test_script = find_program('igt_command_line.sh')
-foreach prog : test_list
+foreach testexe : test_executables
+	prog = testexe.name()
+
+	output = prog.split('/').get(-1) + '.testlist'
+	custom_target(output,
+		     build_by_default : true,
+		     command : [ testexe.full_path(), '--show-testlist'],
+		     capture : true,
+		     depends : testexe,
+		     output : output)
+
 	test('testcase check ' + prog, test_script, args : prog)
 endforeach
 
Signature · 1 line
2.40.1
#4 lore raw

PATCH 4/5 tests/meson.build: add gem_stress to the test_list logic

In reply to Mauro Carvalho Chehab

From: Mauro Carvalho Chehab <[email protected]>

This test seems to be somewhat abandoned at the build system, as it was not used to update the test lists and their dependencies.

---
 tests/meson.build | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
tests/meson.build +7 -6
index 246a585aa423..7620bd3598f0 100644
@@ -516,6 +516,13 @@ test_executables += executable('sw_sync', 'sw_sync.c',
 	   install : true)
 test_list += 'sw_sync'
 
+test_executables += executable('gem_stress', 'i915/gem_stress.c',
+	   install : true,
+	   install_dir : libexecdir,
+	   install_rpath : libexecdir_rpathdir,
+	   dependencies : igt_deps)
+test_list += 'gem_stress'
+
 subdir('amdgpu')
 
 subdir('v3d')
@@ -559,12 +566,6 @@ foreach testexe : test_executables
 	test('testcase check ' + prog, test_script, args : prog)
 endforeach
 
-executable('gem_stress', 'i915/gem_stress.c',
-	   install : true,
-	   install_dir : libexecdir,
-	   install_rpath : libexecdir_rpathdir,
-	   dependencies : igt_deps)
-
 image_files = [
   '1080p-left.png',
   '1080p-right.png',
Signature · 1 line
2.40.1
#5 lore raw

PATCH 5/5 scripts/test_list.py: speedup testlist check logic

In reply to Mauro Carvalho Chehab

From: Mauro Carvalho Chehab <[email protected]>

Now that the build system will generate the testlist for us, just read them when checking for missing symbols.

This should speed up a log the check logic.

After the change, touching on a single file and rebuilding the testplan with check enabled is a lot faster:

	$ touch tests/xe/xe_compute.c
	$ time make
	ninja -C build
	ninja: Entering directory `build'
	[10/10] Generating docs/testplan/i915_tests.html with a custom command
	real	0m4.510s
	user	0m5.944s
	sys	0m0.217s

This can be improved even further if we split the test_executables dependencies per driver.

---
 scripts/test_list.py | 30 ++++++------------------------
 1 file changed, 6 insertions(+), 24 deletions(-)
scripts/test_list.py +6 -24
index a55d2977c7c3..2c1a6f230fb5 100755
@@ -909,35 +909,17 @@ class TestList:
 
         return subtests
 
-    def __get_testlist(self, name):
-        match = re.match(r"(.*/)?(.*)\.c$", name)
-        if not match:
-            return []
-
-        basename = "igt@" + match.group(2)
-
-        fname = os.path.join(self.igt_build_path, "tests", match.group(2))
-        if not os.path.isfile(fname):
-            print(f"Error: file {fname} doesn't exist.")
-            sys.exit(1)
-        try:
-            result = subprocess.run([ fname, "--list-subtests" ],
-                                    check = True,
-                                    stdout = subprocess.PIPE,
-                                    universal_newlines=True)
-            subtests = result.stdout.splitlines()
-
-            return [basename  + "@" + i for i in subtests]
-        except subprocess.CalledProcessError:
-            # Handle it as a test using igt_simple_main
-            return [basename]
-
     def get_testlist(self):
 
         """ Return a list of tests as reported by --list-subtests """
         tests = []
         for name in self.filenames:
-            tests += self.__get_testlist(name)
+            fname = re.sub(r"\.c$", ".testlist", name.split('/')[-1])
+            fname = os.path.join(self.igt_build_path, "tests", fname)
+
+            with open(fname, 'r', encoding='utf8') as handle:
+                for line in handle:
+                    tests.append(line)
 
         return sorted(tests)
 
Signature · 1 line
2.40.1
#6 lore raw

[igt-dev] ✗ Fi.CI.BUILD: failure for Speedup IGT build time

In reply to Mauro Carvalho Chehab
== Series Details ==

== Series Details ==

Series: Speedup IGT build time
URL : https://patchwork.freedesktop.org/series/120233/
State : failure

== Summary ==

IGT patchset build failed on latest successful build
f8d05fd574fad1526cbf5e20672910df87b8839b intel-ci/xe-fast-feedback.testlist: Add xe_exec_store@basic-store subtest in fastfeedback

Compiler for C supports arguments -Werror=sequence-point: YES
Compiler for C supports arguments -Werror=trigraphs: YES
Compiler for C supports arguments -Werror=write-strings: YES
Compiler for C supports arguments -fno-builtin-malloc: YES
Compiler for C supports arguments -fno-builtin-calloc: YES
Found pkg-config: /usr/bin/pkg-config (0.29.1)
Run-time dependency libdrm found: YES 2.4.107
Run-time dependency libdrm_intel found: YES 2.4.107
Run-time dependency pciaccess found: YES 0.16
Run-time dependency libkmod found: YES 27
Run-time dependency libprocps found: YES 3.3.16
Found CMake: /usr/bin/cmake (3.16.3)
Run-time dependency libproc2 found: NO (tried pkgconfig and cmake)
Run-time dependency libunwind found: YES 1.21
Run-time dependency libdw found: YES 0.176
Run-time dependency pixman-1 found: YES 0.38.0
Run-time dependency valgrind found: YES 3.15.0
Run-time dependency cairo found: YES 1.17.2
Run-time dependency libudev found: YES 245
Run-time dependency glib-2.0 found: YES 2.64.6
Run-time dependency xmlrpc found: NO (tried pkgconfig and cmake)
Run-time dependency xmlrpc_util found: NO (tried pkgconfig and cmake) Run-time dependency xmlrpc_client found: NO (tried pkgconfig and cmake) Program xmlrpc-c-config found: YES (/usr/bin/xmlrpc-c-config)
Run-time dependency gsl found: YES 2.5
Run-time dependency alsa found: YES 1.2.2
Run-time dependency libcurl found: YES 7.68.0
Run-time dependency threads found: YES
Library m found: YES
Library rt found: YES
Library dl found: YES
Library z found: YES
Checking if "built-in atomics" links: YES
Has header "linux/kd.h" : YES
Has header "sys/kd.h" : YES
Has header "libgen.h" : YES
Has header "sys/io.h" : YES
Checking if "cpuid.h" links: YES
Header <unistd.h> has symbol "gettid" : YES
Checking whether type "struct sysinfo" has member "totalram" : YES
Checking for function "memfd_create" : YES
Configuring config.h using configuration
Program python3 found: YES (/usr/bin/python3)
Configuring i915-perf.pc using configuration
Program generate_testlist.sh found: YES (/usr/src/igt-gpu-tools/tests/generate_testlist.sh) Program igt_command_line.sh found: YES (/usr/src/igt-gpu-tools/tests/igt_command_line.sh)

tests/meson.build:556:1: ERROR: Unknown method "name" in object.

A full log can be found at /opt/igt/build/meson-logs/meson-log.txt