Testlist changes
cover Testlist changes
This series introduces a build-time checker for testlists, the test list for extended testing, and as an added bonus, introduces the ability to comment test lists.
Petri Latvala (4):
intel-ci: Add comments on test order to fast-feedback.testlist tests/Makefile.am: Only ignore generated gitignore in gitignore
intel-ci: Build-time check for testlists
intel-ci: Add extended.testlist for wider testing
configure.ac | 1 + tests/Makefile.am | 4 +- tests/intel-ci/.gitignore | 1 + tests/intel-ci/Makefile.am | 42 + tests/intel-ci/README | 6 + tests/intel-ci/check_testlists.sh | 65 ++ tests/intel-ci/checked_lists.txt | 2 + tests/intel-ci/extended.testlist | 1919 +++++++++++++++++++++++++++++++++ tests/intel-ci/fast-feedback.testlist | 7 + 9 files changed, 2046 insertions(+), 1 deletion(-) create mode 100644 tests/intel-ci/.gitignore create mode 100644 tests/intel-ci/Makefile.am create mode 100755 tests/intel-ci/check_testlists.sh create mode 100644 tests/intel-ci/checked_lists.txt create mode 100644 tests/intel-ci/extended.testlist
Signature · 6 lines
2.9.3 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH 1/4 intel-ci: Add comments on test order to fast-feedback.testlist
Document the test ordering choices in fast-feedback.testlist.
For comments in testlists, piglit commit
commit 0c535186d624071098c10003fdafe8f475ed9ae7
Author: Petri Latvala <[email protected]>
Date: Wed Feb 1 12:57:45 2017 +0200
framework/programs/run.py: Allow comments in test-list files.
is required.
--- tests/intel-ci/fast-feedback.testlist | 7 +++++++ 1 file changed, 7 insertions(+)
tests/intel-ci/fast-feedback.testlist +7 -0
@@ -1,3 +1,5 @@ +# Keep alphabetically sorted by default + igt@core_auth@basic-auth igt@core_prop_blob@basic igt@drv_getparams_basic@basic-eu-total
@@ -270,6 +272,11 @@ igt@vgem_basic@dmabuf-mmap igt@vgem_basic@mmap igt@vgem_basic@second-client igt@vgem_basic@sysfs + +# All tests that do module unloading and reloading are executed last. +# They will sometimes reveal issues of earlier tests leaving the +# driver in a broken state that is not otherwise noticed in that test. + igt@vgem_basic@unload igt@drv_module_reload@basic-reload igt@drv_module_reload@basic-no-display
Signature · 6 lines
2.9.3 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH 2/4 tests/Makefile.am: Only ignore generated gitignore in gitignore
In reply to Petri LatvalaWhen generating the .gitignore file, use /.gitignore instead of .gitignore to not have it apply to subdirectories.
--- tests/Makefile.am | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
tests/Makefile.am +1 -1
@@ -44,7 +44,7 @@ dist_pkgdata_DATA = \ all-local: .gitignore .gitignore: Makefile.sources - @echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt test-list-full.txt .gitignore" | sed 's/\s\+/\n/g' | sort > $@ + @echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt test-list-full.txt /.gitignore" | sed 's/\s\+/\n/g' | sort > $@ pkgdata_DATA = test-list.txt test-list-full.txt
Signature · 6 lines
2.9.3 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH 3/4 intel-ci: Build-time check for testlists
In reply to Petri LatvalaTest at check and distcheck that testlist files only contain tests that are still present.
--- configure.ac | 1 + tests/Makefile.am | 2 ++ tests/intel-ci/.gitignore | 1 + tests/intel-ci/Makefile.am | 42 +++++++++++++++++++++++++ tests/intel-ci/check_testlists.sh | 65 +++++++++++++++++++++++++++++++++++++++ tests/intel-ci/checked_lists.txt | 1 + 6 files changed, 112 insertions(+) create mode 100644 tests/intel-ci/.gitignore create mode 100644 tests/intel-ci/Makefile.am create mode 100755 tests/intel-ci/check_testlists.sh create mode 100644 tests/intel-ci/checked_lists.txt
configure.ac +1 -0
@@ -341,6 +341,7 @@ AC_CONFIG_FILES([ man/Makefile scripts/Makefile tests/Makefile + tests/intel-ci/Makefile tools/Makefile tools/null_state_gen/Makefile tools/registers/Makefile
tests/Makefile.am +2 -0
@@ -1,5 +1,7 @@ include Makefile.sources +SUBDIRS = . intel-ci + if HAVE_LIBDRM_NOUVEAU TESTS_progs_M += $(NOUVEAU_TESTS_M) endif
tests/intel-ci/.gitignore new file +1 -0
@@ -0,0 +1 @@ +all-tests.testlist
tests/intel-ci/Makefile.am new file +42 -0
42 lines not on this page · raw message
tests/intel-ci/check_testlists.sh new file +65 -0
65 lines not on this page · raw message
tests/intel-ci/checked_lists.txt new file +1 -0
@@ -0,0 +1 @@ +fast-feedback.testlist
Signature · 6 lines
2.9.3 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx
PATCH 4/4 intel-ci: Add extended.testlist for wider testing
In reply to Petri LatvalaTest list with wider coverage and longer full runtime.
---
This is a preliminary testlist for extended testing. Tomi has been running it semi-manually in CI already.
I think I have incorporated all the changes Tomi has been making for his runs. Tomi can double-check that the list matches.
tests/intel-ci/README | 6 + tests/intel-ci/checked_lists.txt | 1 + tests/intel-ci/extended.testlist | 1919 ++++++++++++++++++++++++++++++++++++++ 3 files changed, 1926 insertions(+) create mode 100644 tests/intel-ci/extended.testlist
tests/intel-ci/README +6 -0
@@ -32,3 +32,9 @@ test per feature. The string "basic" in a test name means the test probably belongs in this list. + +================= +extended.testlist +================= + +Test list for wider coverage testing.
tests/intel-ci/checked_lists.txt +1 -0
@@ -1 +1,2 @@ fast-feedback.testlist +extended.testlist
tests/intel-ci/extended.testlist new file +1919 -0
1,919 lines not on this page · raw message
Signature · 6 lines
2.9.3 _______________________________________________ Intel-gfx mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/intel-gfx