tests: Separate tests with lots of subtests
PATCH 1/1 tests: Separate tests with lots of subtests
Currently, when trying to run i-g-t tests with piglit, it takes up to 20s until the first test is executed.
The main reason is that gem_concurrent_all has ~500k subtests, overkilling piglit.
This patch separates gem_concurrent_* tests from the rest of the tests, creating two files test-list.txt and test-list-full.txt.
The piglit can now enumerate the i-g-t tests within a decent few seconds. The second list could be used when trying to execute specific these tests.
--- tests/Makefile.am | 6 ++++++ tests/Makefile.sources | 22 ++++++++++++++++++++-- 2 files changed, 26 insertions(+), 2 deletions(-)
tests/Makefile.am +6 -0
@@ -14,6 +14,12 @@ test-list.txt: Makefile.sources @echo ${single_kernel_tests} ${multi_kernel_tests} >> $@ @echo END TESTLIST >> $@ +test-list-full.txt: Makefile.sources + @echo TESTLIST > $@ + @echo ${single_kernel_tests} ${multi_kernel_tests} ${extra_multi_kernel_tests} >> $@ + @echo END TESTLIST >> $@ + + noinst_PROGRAMS = \ $(HANG) \ $(TESTS_testsuite) \
tests/Makefile.sources +20 -2
@@ -20,8 +20,6 @@ TESTS_progs_M = \ gem_busy \ gem_caching \ gem_close_race \ - gem_concurrent_blit \ - gem_concurrent_all \ gem_create \ gem_cs_tlb \ gem_ctx_param_basic \
@@ -117,6 +115,11 @@ TESTS_progs_M = \ template \ $(NULL) +TESTS_progs_XM = \ + gem_concurrent_blit \ + gem_concurrent_all \ + $(NULL) + TESTS_progs = \ core_auth \ core_getclient \
@@ -219,6 +222,15 @@ multi_kernel_tests = \ $(TESTS_scripts_M) \ $(NULL) + +# This target contains testcases with lots of subtests. +# It was split from multi_kernel_tests because of the overhead of listing all +# the subtests in piglit +extra_multi_kernel_tests = \ + $(TESTS_progs_XM) \ + $(NULL) + + # This target is for simple testcase which don't expose any subtest. # # Your testcase should probably use igt_simple_main, but none of the other magic
@@ -233,6 +245,12 @@ kernel_tests = \ $(multi_kernel_tests) \ $(NULL) +kernel_tests_full = \ + $(single_kernel_tests) \ + $(multi_kernel_tests) \ + $(extra_multi_kernel_tests) \ + $(NULL) + # Test that exercise specific asserts in the test framework library and are # hence expected to fail. XFAIL_TESTS = \
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 11 lines · Gabriel Feceoru
Currently, when trying to run i-g-t tests with piglit, it takes up to 20s until the first test is executed.
The main reason is thatgem_concurrent_allhas ~500k subtests, overkilling piglit.This patch separates gem_concurrent_* tests from the rest of the tests, creating two files test-list.txt and test-list-full.txt.
The piglit can now enumerate the i-g-t tests within a decent few seconds. The second list could be used when trying to execute specific these tests.Signed-off-by Gabriel Feceoru <[email protected]>
Imo just shuffle gem_concurrent_all into the EXTRA binaries target. With 500k tests it's way past the "just a few regression tests" phase fully into "full blown stress test suite that takes centuries to run".
With that change: Acked-by: Daniel Vetter <[email protected]>
So feel free to ask Marius to push right away.
-Daniel
Quoted 79 lines
--- tests/Makefile.am | 6 ++++++ tests/Makefile.sources | 22 ++++++++++++++++++++-- 2 files changed, 26 insertions(+), 2 deletions(-)tests/Makefile.am +6
-0@@ -14,6 +14,12 @@ test-list.txt: Makefile.sources @echo ${single_kernel_tests} ${multi_kernel_tests} >> $@ @echo END TESTLIST >> $@ +test-list-full.txt: Makefile.sources + @echo TESTLIST > $@ + @echo ${single_kernel_tests} ${multi_kernel_tests} ${extra_multi_kernel_tests} >> $@ + @echo END TESTLIST >> $@ + + noinst_PROGRAMS = \ $(HANG) \ $(TESTS_testsuite) \tests/Makefile.sources +20
-2@@ -20,8 +20,6 @@ TESTS_progs_M = \ gem_busy \ gem_caching \ gem_close_race \ - gem_concurrent_blit \ - gem_concurrent_all \ gem_create \ gem_cs_tlb \ gem_ctx_param_basic \@@ -117,6 +115,11 @@ TESTS_progs_M = \ template \ $(NULL) +TESTS_progs_XM = \ + gem_concurrent_blit \ + gem_concurrent_all \ + $(NULL) + TESTS_progs = \ core_auth \ core_getclient \@@ -219,6 +222,15 @@ multi_kernel_tests = \ $(TESTS_scripts_M) \ $(NULL) + +# This target contains testcases with lots of subtests. +# It was split from multi_kernel_tests because of the overhead of listing all +# the subtests in piglit +extra_multi_kernel_tests = \ + $(TESTS_progs_XM) \ + $(NULL) + + # This target is for simple testcase which don't expose any subtest. # # Your testcase should probably use igt_simple_main, but none of the other magic@@ -233,6 +245,12 @@ kernel_tests = \ $(multi_kernel_tests) \ $(NULL) +kernel_tests_full = \ + $(single_kernel_tests) \ + $(multi_kernel_tests) \ + $(extra_multi_kernel_tests) \ + $(NULL) + # Test that exercise specific asserts in the test framework library and are # hence expected to fail. XFAIL_TESTS = \Signature · 1 line
1.9.1
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH v2 1/1 tests: Separate tests with lots of subtests
In reply to Gabriel FeceoruCurrently, when trying to run i-g-t tests with piglit, it takes up to 20s until the first test is executed.
The main reason is that gem_concurrent_all has ~500k subtests, overkilling piglit.
This patch separates gem_concurrent_* tests from the rest of the tests, creating two files test-list.txt and test-list-full.txt.
The piglit can now enumerate the i-g-t tests within a decent few seconds. The second list could be used when trying to execute specific these tests.
v2: Rebased
--- tests/Makefile.am | 12 +++++++++--- tests/Makefile.sources | 22 ++++++++++++++++++++-- 2 files changed, 29 insertions(+), 5 deletions(-)
tests/Makefile.am +9 -3
@@ -14,6 +14,12 @@ test-list.txt: Makefile.sources @echo ${single_kernel_tests} ${multi_kernel_tests} >> $@ @echo END TESTLIST >> $@ +test-list-full.txt: Makefile.sources + @echo TESTLIST > $@ + @echo ${single_kernel_tests} ${multi_kernel_tests} ${extra_multi_kernel_tests} >> $@ + @echo END TESTLIST >> $@ + + noinst_PROGRAMS = \ $(HANG) \ $(TESTS_testsuite) \
@@ -38,14 +44,14 @@ dist_pkgdata_DATA = \ all-local: .gitignore .gitignore: Makefile.sources - @echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt .gitignore" | sed 's/\s\+/\n/g' | sort > $@ + @echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt test-list-full.txt .gitignore" | sed 's/\s\+/\n/g' | sort > $@ -pkgdata_DATA = test-list.txt +pkgdata_DATA = test-list.txt test-list-full.txt EXTRA_PROGRAMS = $(HANG) EXTRA_DIST = $(common_files) -CLEANFILES = $(EXTRA_PROGRAMS) test-list.txt .gitignore +CLEANFILES = $(EXTRA_PROGRAMS) test-list.txt test-list-full.txt .gitignore AM_CFLAGS = $(DRM_CFLAGS) $(CWARNFLAGS) $(DEBUG_CFLAGS)\ -I$(srcdir)/.. \
tests/Makefile.sources +20 -2
@@ -20,8 +20,6 @@ TESTS_progs_M = \ gem_busy \ gem_caching \ gem_close_race \ - gem_concurrent_blit \ - gem_concurrent_all \ gem_create \ gem_cs_tlb \ gem_ctx_bad_exec \
@@ -117,6 +115,11 @@ TESTS_progs_M = \ template \ $(NULL) +TESTS_progs_XM = \ + gem_concurrent_blit \ + gem_concurrent_all \ + $(NULL) + TESTS_progs = \ core_auth \ core_getclient \
@@ -219,6 +222,15 @@ multi_kernel_tests = \ $(TESTS_scripts_M) \ $(NULL) + +# This target contains testcases with lots of subtests. +# It was split from multi_kernel_tests because of the overhead of listing all +# the subtests in piglit +extra_multi_kernel_tests = \ + $(TESTS_progs_XM) \ + $(NULL) + + # This target is for simple testcase which don't expose any subtest. # # Your testcase should probably use igt_simple_main, but none of the other magic
@@ -233,6 +245,12 @@ kernel_tests = \ $(multi_kernel_tests) \ $(NULL) +kernel_tests_full = \ + $(single_kernel_tests) \ + $(multi_kernel_tests) \ + $(extra_multi_kernel_tests) \ + $(NULL) + # Test that exercise specific asserts in the test framework library and are # hence expected to fail. XFAIL_TESTS = \
Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 5 lines · Gabriel Feceoru
Currently, when trying to run i-g-t tests with piglit, it takes up to 20s until the first test is executed.
The main reason is thatgem_concurrent_allhas ~500k subtests, overkilling piglit.
Quoted 8 lines
This patch separates gem_concurrent_* tests from the rest of the tests, creating two files test-list.txt and test-list-full.txt.
The piglit can now enumerate the i-g-t tests within a decent few seconds. The second list could be used when trying to execute specific these tests.v2: Rebased
Signed-off-by Gabriel Feceoru <[email protected]>
You didn't take my review into account but still just pushed this patch. Please fix up.
Also I think we should keep gem_concurrent_blit in the normal set of testcases.
-Daniel
Quoted 101 lines
--- tests/Makefile.am | 12 +++++++++--- tests/Makefile.sources | 22 ++++++++++++++++++++-- 2 files changed, 29 insertions(+), 5 deletions(-)tests/Makefile.am +9
-3@@ -14,6 +14,12 @@ test-list.txt: Makefile.sources @echo ${single_kernel_tests} ${multi_kernel_tests} >> $@ @echo END TESTLIST >> $@ +test-list-full.txt: Makefile.sources + @echo TESTLIST > $@ + @echo ${single_kernel_tests} ${multi_kernel_tests} ${extra_multi_kernel_tests} >> $@ + @echo END TESTLIST >> $@ + + noinst_PROGRAMS = \ $(HANG) \ $(TESTS_testsuite) \@@ -38,14 +44,14 @@ dist_pkgdata_DATA = \ all-local: .gitignore .gitignore: Makefile.sources - @echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt .gitignore" | sed 's/\s\+/\n/g' | sort > $@ + @echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt test-list-full.txt .gitignore" | sed 's/\s\+/\n/g' | sort > $@ -pkgdata_DATA = test-list.txt +pkgdata_DATA = test-list.txt test-list-full.txt EXTRA_PROGRAMS = $(HANG) EXTRA_DIST = $(common_files) -CLEANFILES = $(EXTRA_PROGRAMS) test-list.txt .gitignore +CLEANFILES = $(EXTRA_PROGRAMS) test-list.txt test-list-full.txt .gitignore AM_CFLAGS = $(DRM_CFLAGS) $(CWARNFLAGS) $(DEBUG_CFLAGS)\ -I$(srcdir)/.. \tests/Makefile.sources +20
-2@@ -20,8 +20,6 @@ TESTS_progs_M = \ gem_busy \ gem_caching \ gem_close_race \ - gem_concurrent_blit \ - gem_concurrent_all \ gem_create \ gem_cs_tlb \ gem_ctx_bad_exec \@@ -117,6 +115,11 @@ TESTS_progs_M = \ template \ $(NULL) +TESTS_progs_XM = \ + gem_concurrent_blit \ + gem_concurrent_all \ + $(NULL) + TESTS_progs = \ core_auth \ core_getclient \@@ -219,6 +222,15 @@ multi_kernel_tests = \ $(TESTS_scripts_M) \ $(NULL) + +# This target contains testcases with lots of subtests. +# It was split from multi_kernel_tests because of the overhead of listing all +# the subtests in piglit +extra_multi_kernel_tests = \ + $(TESTS_progs_XM) \ + $(NULL) + + # This target is for simple testcase which don't expose any subtest. # # Your testcase should probably use igt_simple_main, but none of the other magic@@ -233,6 +245,12 @@ kernel_tests = \ $(multi_kernel_tests) \ $(NULL) +kernel_tests_full = \ + $(single_kernel_tests) \ + $(multi_kernel_tests) \ + $(extra_multi_kernel_tests) \ + $(NULL) + # Test that exercise specific asserts in the test framework library and are # hence expected to fail. XFAIL_TESTS = \Signature · 6 lines
1.9.1 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
Quoted 11 lines · Daniel Vetter
Quoted 10 lines · Gabriel Feceoru
Currently, when trying to run i-g-t tests with piglit, it takes up to 20s until the first test is executed.
The main reason is thatgem_concurrent_allhas ~500k subtests, overkilling piglit.This patch separates gem_concurrent_* tests from the rest of the tests, creating two files test-list.txt and test-list-full.txt.
The piglit can now enumerate the i-g-t tests within a decent few seconds. The second list could be used when trying to execute specific these tests.
Quoted 6 lines
Quoted 3 lines
v2: Rebased
Signed-off-by Gabriel Feceoru <[email protected]>You didn't take my review into account but still just pushed this patch. Please fix up.
My apologies, I didn't ignore it but didn't read it carefully.
Regards,
Gabriel
Quoted 102 lines
Also I think we should keep
gem_concurrent_blitin the normal set of testcases.
-DanielQuoted 97 lines
--- tests/Makefile.am | 12 +++++++++--- tests/Makefile.sources | 22 ++++++++++++++++++++-- 2 files changed, 29 insertions(+), 5 deletions(-)tests/Makefile.am +9
-3@@ -14,6 +14,12 @@ test-list.txt: Makefile.sources @echo ${single_kernel_tests} ${multi_kernel_tests} >> $@ @echo END TESTLIST >> $@ +test-list-full.txt: Makefile.sources + @echo TESTLIST > $@ + @echo ${single_kernel_tests} ${multi_kernel_tests} ${extra_multi_kernel_tests} >> $@ + @echo END TESTLIST >> $@ + + noinst_PROGRAMS = \ $(HANG) \ $(TESTS_testsuite) \@@ -38,14 +44,14 @@ dist_pkgdata_DATA = \ all-local: .gitignore .gitignore: Makefile.sources - @echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt .gitignore" | sed 's/\s\+/\n/g' | sort > $@ + @echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt test-list-full.txt .gitignore" | sed 's/\s\+/\n/g' | sort > $@ -pkgdata_DATA = test-list.txt +pkgdata_DATA = test-list.txt test-list-full.txt EXTRA_PROGRAMS = $(HANG) EXTRA_DIST = $(common_files) -CLEANFILES = $(EXTRA_PROGRAMS) test-list.txt .gitignore +CLEANFILES = $(EXTRA_PROGRAMS) test-list.txt test-list-full.txt .gitignore AM_CFLAGS = $(DRM_CFLAGS) $(CWARNFLAGS) $(DEBUG_CFLAGS)\ -I$(srcdir)/.. \tests/Makefile.sources +20
-3@@ -20,8 +20,6 @@ TESTS_progs_M = \ gem_busy \ gem_caching \ gem_close_race \ - gem_concurrent_blit \ - gem_concurrent_all \ gem_create \ gem_cs_tlb \ gem_ctx_bad_exec \@@ -117,6 +115,11 @@ TESTS_progs_M = \ template \ $(NULL) +TESTS_progs_XM = \ + gem_concurrent_blit \ + gem_concurrent_all \ + $(NULL) + TESTS_progs = \ core_auth \ core_getclient \@@ -219,6 +222,15 @@ multi_kernel_tests = \ $(TESTS_scripts_M) \ $(NULL) + +# This target contains testcases with lots of subtests. +# It was split from multi_kernel_tests because of the overhead of listing all +# the subtests in piglit +extra_multi_kernel_tests = \ + $(TESTS_progs_XM) \ + $(NULL) + + # This target is for simple testcase which don't expose any subtest. # # Your testcase should probably use igt_simple_main, but none of the other magic@@ -233,6 +245,12 @@ kernel_tests = \ $(multi_kernel_tests) \ $(NULL) +kernel_tests_full = \ + $(single_kernel_tests) \ + $(multi_kernel_tests) \ + $(extra_multi_kernel_tests) \ + $(NULL) + # Test that exercise specific asserts in the test framework library and are # hence expected to fail. XFAIL_TESTS = \ --1.9.1
Quoted 5 lines
Quoted 4 lines
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx