Skip to contentSkip to conversation

Testlist changes

PATCH New Petri Latvala 1 participant 5 messages Mar 17, 2017

lore
Share & tools
lore
#0 lore raw

cover Testlist changes

This series introduces a build-time checker for testlists, the test list for extended testing, and as an added bonus, introduces the ability to comment test lists.

Petri Latvala (4):
intel-ci: Add comments on test order to fast-feedback.testlist tests/Makefile.am: Only ignore generated gitignore in gitignore
intel-ci: Build-time check for testlists
intel-ci: Add extended.testlist for wider testing

 configure.ac                          |    1 +
 tests/Makefile.am                     |    4 +-
 tests/intel-ci/.gitignore             |    1 +
 tests/intel-ci/Makefile.am            |   42 +
 tests/intel-ci/README                 |    6 +
 tests/intel-ci/check_testlists.sh     |   65 ++
 tests/intel-ci/checked_lists.txt      |    2 +
 tests/intel-ci/extended.testlist      | 1919 +++++++++++++++++++++++++++++++++
 tests/intel-ci/fast-feedback.testlist |    7 +
 9 files changed, 2046 insertions(+), 1 deletion(-)
 create mode 100644 tests/intel-ci/.gitignore
 create mode 100644 tests/intel-ci/Makefile.am
 create mode 100755 tests/intel-ci/check_testlists.sh
 create mode 100644 tests/intel-ci/checked_lists.txt
 create mode 100644 tests/intel-ci/extended.testlist
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 1/4 intel-ci: Add comments on test order to fast-feedback.testlist

Document the test ordering choices in fast-feedback.testlist.

For comments in testlists, piglit commit

commit 0c535186d624071098c10003fdafe8f475ed9ae7
Author: Petri Latvala <[email protected]>
Date: Wed Feb 1 12:57:45 2017 +0200

framework/programs/run.py: Allow comments in test-list files.

is required.

---
 tests/intel-ci/fast-feedback.testlist | 7 +++++++
 1 file changed, 7 insertions(+)
tests/intel-ci/fast-feedback.testlist +7 -0
index 5ffa2ce..ce7a41f 100644
@@ -1,3 +1,5 @@
+# Keep alphabetically sorted by default
+
 igt@core_auth@basic-auth
 igt@core_prop_blob@basic
 igt@drv_getparams_basic@basic-eu-total
@@ -270,6 +272,11 @@ igt@vgem_basic@dmabuf-mmap
 igt@vgem_basic@mmap
 igt@vgem_basic@second-client
 igt@vgem_basic@sysfs
+
+# All tests that do module unloading and reloading are executed last.
+# They will sometimes reveal issues of earlier tests leaving the
+# driver in a broken state that is not otherwise noticed in that test.
+
 igt@vgem_basic@unload
 igt@drv_module_reload@basic-reload
 igt@drv_module_reload@basic-no-display
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

PATCH 2/4 tests/Makefile.am: Only ignore generated gitignore in gitignore

In reply to Petri Latvala

When generating the .gitignore file, use /.gitignore instead of .gitignore to not have it apply to subdirectories.

---
 tests/Makefile.am | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/Makefile.am +1 -1
index 8930c24..2080b3f 100644
@@ -44,7 +44,7 @@ dist_pkgdata_DATA = \
 
 all-local: .gitignore
 .gitignore: Makefile.sources
-	@echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt test-list-full.txt .gitignore" | sed 's/\s\+/\n/g' | sort > $@
+	@echo "$(pkglibexec_PROGRAMS) $(HANG) test-list.txt test-list-full.txt /.gitignore" | sed 's/\s\+/\n/g' | sort > $@
 
 pkgdata_DATA = test-list.txt test-list-full.txt
 
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH 3/4 intel-ci: Build-time check for testlists

In reply to Petri Latvala

Test at check and distcheck that testlist files only contain tests that are still present.

---
 configure.ac                      |  1 +
 tests/Makefile.am                 |  2 ++
 tests/intel-ci/.gitignore         |  1 +
 tests/intel-ci/Makefile.am        | 42 +++++++++++++++++++++++++
 tests/intel-ci/check_testlists.sh | 65 +++++++++++++++++++++++++++++++++++++++
 tests/intel-ci/checked_lists.txt  |  1 +
 6 files changed, 112 insertions(+)
 create mode 100644 tests/intel-ci/.gitignore
 create mode 100644 tests/intel-ci/Makefile.am
 create mode 100755 tests/intel-ci/check_testlists.sh
 create mode 100644 tests/intel-ci/checked_lists.txt
configure.ac +1 -0
index 12b0ab0..9aa6068 100644
@@ -341,6 +341,7 @@ AC_CONFIG_FILES([
 		 man/Makefile
 		 scripts/Makefile
 		 tests/Makefile
+		 tests/intel-ci/Makefile
 		 tools/Makefile
 		 tools/null_state_gen/Makefile
 		 tools/registers/Makefile
tests/Makefile.am +2 -0
index 2080b3f..001106f 100644
@@ -1,5 +1,7 @@
 include Makefile.sources
 
+SUBDIRS = . intel-ci
+
 if HAVE_LIBDRM_NOUVEAU
     TESTS_progs_M += $(NOUVEAU_TESTS_M)
 endif
tests/intel-ci/.gitignore new file +1 -0
new file mode 100644
index 0000000..1a8f133
@@ -0,0 +1 @@
+all-tests.testlist
tests/intel-ci/Makefile.am new file +42 -0

42 lines not on this page · raw message

tests/intel-ci/check_testlists.sh new file +65 -0

65 lines not on this page · raw message

tests/intel-ci/checked_lists.txt new file +1 -0
new file mode 100644
index 0000000..2f8a386
@@ -0,0 +1 @@
+fast-feedback.testlist
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

PATCH 4/4 intel-ci: Add extended.testlist for wider testing

In reply to Petri Latvala

Test list with wider coverage and longer full runtime.

---

This is a preliminary testlist for extended testing. Tomi has been running it semi-manually in CI already.

I think I have incorporated all the changes Tomi has been making for his runs. Tomi can double-check that the list matches.

 tests/intel-ci/README            |    6 +
 tests/intel-ci/checked_lists.txt |    1 +
 tests/intel-ci/extended.testlist | 1919 ++++++++++++++++++++++++++++++++++++++
 3 files changed, 1926 insertions(+)
 create mode 100644 tests/intel-ci/extended.testlist
tests/intel-ci/README +6 -0
index dc7e5c8..1bc940d 100644
@@ -32,3 +32,9 @@ test per feature.
 
 The string "basic" in a test name means the test probably belongs in
 this list.
+
+=================
+extended.testlist
+=================
+
+Test list for wider coverage testing.
tests/intel-ci/checked_lists.txt +1 -0
index 2f8a386..34ac85c 100644
@@ -1 +1,2 @@
 fast-feedback.testlist
+extended.testlist
tests/intel-ci/extended.testlist new file +1919 -0

1,919 lines not on this page · raw message

Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx