scripts/verify-blacklist: Script for checking blacklist files
PATCH v3 1/2 scripts/verify-blacklist: Script for checking blacklist files
tests/intel-ci/blacklist*.txt files can collect bitrot unless there's an easy way to check for lines that are no longer needed due to the tests being renamed or removed. Therefore, a script just for that.
v2: Use long options for readability, exit with 1 if something found
v3: Verify manually against the list of all tests instead of trying
with igt_runner's --include-tests
--- scripts/verify-blacklist.sh | 54 +++++++++++++++++++++++++++++++++++++ 1 file changed, 54 insertions(+) create mode 100755 scripts/verify-blacklist.sh
scripts/verify-blacklist.sh new file +54 -0
@@ -0,0 +1,54 @@ +#!/bin/sh + +# Verify that all entries in a blacklist file are still valid + +usage() { + echo "Usage: $0 <path-to-igt-runner> <test-binary-directory> <blacklist-file>" + echo + echo " path-to-igt-runner: For example build/runner/igt_runner" + echo " test-binary-directory: For example build/tests" + echo " blacklist-file: For example tests/intel-ci/blacklist.txt" + exit 2 +} + +if [ $# -ne 3 ]; then + usage +fi + +RUNNER="$1" +BINDIR="$2" +BLFILE="$3" + +if [ ! -x "$RUNNER" ]; then + echo "$RUNNER not found" + echo + usage +fi + +if [ ! -f "$BINDIR/test-list.txt" ]; then + echo "$BINDIR doesn't look like a test-binary directory" + echo + usage +fi + +if [ ! -f "$BLFILE" ]; then + echo "$BLFILE not found" + echo + usage +fi + +STATUS=0 + +TESTLIST="$("$RUNNER" --list-all "$BINDIR")" + +cat "$BLFILE" | while read line; do + blentry=$(echo "$line" | sed 's/#.*//' | tr -d '[:space:]') + if [ "$blentry" = "" ]; then continue; fi + + if ! (echo "$TESTLIST" | grep -Pq "$blentry") >/dev/null 2>/dev/null; then + echo Useless blacklist entry: "$blentry" + STATUS=1 + fi +done + +exit $STATUS
Signature · 6 lines
2.20.1 _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
PATCH 2/2 gitlab-ci: Verify blacklist files with verify-blacklist.sh
---
Now finished in 47 seconds in gitlab.
.gitlab-ci.yml | 6 ++++++ 1 file changed, 6 insertions(+)
.gitlab-ci.yml +6 -0
@@ -252,6 +252,12 @@ test:list-undocumented-tests: paths: - undocumented_tests.txt +test:verify-blacklists: + dependencies: + - build:tests-fedora + stage: test + script: for bl in tests/intel-ci/blacklist{,-pre-merge}.txt; do scripts/verify-blacklist.sh build/runner/igt_runner build/tests "$bl" || exit 1; done + ################### DEPLOY ######################### pages:
Signature · 6 lines
2.20.1 _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
Re: [igt-dev] [PATCH i-g-t 2/2] gitlab-ci: Verify blacklist files with verify-blacklist.sh
Quoted 29 lines · Petri Latvala
Signed-off-by Petri Latvala <[email protected]>Cc Arkadiusz Hiler <[email protected]>---
Now finished in 47 seconds in gitlab.
.gitlab-ci.yml | 6 ++++++ 1 file changed, 6 insertions(+).gitlab-ci.yml +6
-0@@ -252,6 +252,12 @@ test:list-undocumented-tests: paths: - undocumented_tests.txt +test:verify-blacklists: + dependencies: + - build:tests-fedora + stage: test + script: for bl in tests/intel-ci/blacklist{,-pre-merge}.txt; do scripts/verify-blacklist.sh build/runner/igt_runner build/tests "$bl" || exit 1; done + ################### DEPLOY ######################### pages:Signature · 1 line
2.20.1
For both patches,
We still should do something about the assume implicit ^$ that are not there.
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Patchwork
CI[igt-dev] ✗ Fi.CI.BAT: failure for series starting with [i-g-t,v3,1/2] scripts/verify-blacklist: Script for checking blacklist files
In reply to Petri Latvala== Series Details ==
== Series Details ==
Series: series starting with [i-g-t,v3,1/2] scripts/verify-blacklist: Script for checking blacklist files
URL : https://patchwork.freedesktop.org/series/78368/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_8626 -> IGTPW_4675
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with IGTPW_4675 absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in IGTPW_4675, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/index.html
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in IGTPW_4675:
### IGT changes ###
#### Possible regressions ####
* igt@gem_exec_parallel@engines@fds:
- fi-tgl-u2: [PASS][1] -> [INCOMPLETE][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-tgl-u2/igt@gem_exec_parallel@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-tgl-u2/igt@gem_exec_parallel@[email protected]
Known issues
------------
Here are the changes found in IGTPW_4675 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_exec_suspend@basic-s0:
- fi-apl-guc: [PASS][3] -> [INCOMPLETE][4] ([i915#1242])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-apl-guc/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-apl-guc/igt@[email protected]
* igt@i915_module_load@reload:
- fi-byt-j1900: [PASS][5] -> [DMESG-WARN][6] ([i915#1982])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-byt-j1900/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-byt-j1900/igt@[email protected]
- fi-byt-n2820: [PASS][7] -> [DMESG-WARN][8] ([i915#1982])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-byt-n2820/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-byt-n2820/igt@[email protected]
* igt@kms_cursor_legacy@basic-busy-flip-before-cursor-atomic:
- fi-bsw-kefka: [PASS][9] -> [DMESG-WARN][10] ([i915#1982]) +1 similar issue
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-bsw-kefka/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-bsw-kefka/igt@[email protected]
* igt@kms_flip@basic-flip-vs-wf_vblank@c-edp1:
- fi-icl-u2: [PASS][11] -> [DMESG-WARN][12] ([i915#1982]) +1 similar issue
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-icl-u2/igt@kms_flip@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-icl-u2/igt@kms_flip@[email protected]
#### Possible fixes ####
* igt@kms_cursor_legacy@basic-flip-before-cursor-atomic:
- fi-icl-guc: [DMESG-WARN][13] ([i915#1982]) -> [PASS][14]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-icl-guc/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-icl-guc/igt@[email protected]
* igt@kms_flip@basic-flip-vs-wf_vblank@b-edp1:
- fi-icl-u2: [DMESG-WARN][15] ([i915#1982]) -> [PASS][16] +2 similar issues
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-icl-u2/igt@kms_flip@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-icl-u2/igt@kms_flip@[email protected]
* igt@kms_flip@basic-plain-flip@d-dsi1:
- {fi-tgl-dsi}: [DMESG-WARN][17] ([i915#1982]) -> [PASS][18]
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-tgl-dsi/igt@kms_flip@[email protected]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-tgl-dsi/igt@kms_flip@[email protected]
#### Warnings ####
* igt@gem_exec_suspend@basic-s3:
- fi-kbl-x1275: [DMESG-WARN][19] ([i915#62] / [i915#92] / [i915#95]) -> [DMESG-WARN][20] ([i915#1982] / [i915#62] / [i915#92] / [i915#95])
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-kbl-x1275/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-kbl-x1275/igt@[email protected]
* igt@kms_flip@basic-flip-vs-modeset@a-dp1:
- fi-kbl-x1275: [DMESG-WARN][21] ([i915#62] / [i915#92]) -> [DMESG-WARN][22] ([i915#62] / [i915#92] / [i915#95]) +5 similar issues
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-kbl-x1275/igt@kms_flip@[email protected]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-kbl-x1275/igt@kms_flip@[email protected]
* igt@kms_pipe_crc_basic@suspend-read-crc-pipe-a:
- fi-kbl-x1275: [DMESG-WARN][23] ([i915#62] / [i915#92] / [i915#95]) -> [DMESG-WARN][24] ([i915#62] / [i915#92]) +4 similar issues
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_8626/fi-kbl-x1275/igt@[email protected]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/fi-kbl-x1275/igt@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[i915#1242]: https://gitlab.freedesktop.org/drm/intel/issues/1242
[i915#1982]: https://gitlab.freedesktop.org/drm/intel/issues/1982
[i915#62]: https://gitlab.freedesktop.org/drm/intel/issues/62
[i915#92]: https://gitlab.freedesktop.org/drm/intel/issues/92
[i915#95]: https://gitlab.freedesktop.org/drm/intel/issues/95
Participating hosts (44 -> 41)
------------------------------
Additional (3): fi-whl-u fi-bdw-5557u fi-skl-6600u
Missing (6): fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-byt-clapper fi-bdw-samus
Build changes
-------------
* CI: CI-20190529 -> None
* IGT: IGT_5710 -> IGTPW_4675
CI-20190529: 20190529
CI_DRM_8626: 563d95505a36489373782ca98d67897d4b338f94 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_4675: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/index.html
IGT_5710: f524eee47930601ad7b4cba9d40c26d68dc7d250 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4675/index.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Patchwork
CI[igt-dev] ✗ GitLab.Pipeline: warning for series starting with [i-g-t,v3,1/2] scripts/verify-blacklist: Script for checking blacklist files
In reply to Petri Latvala== Series Details ==
== Series Details ==
Series: series starting with [i-g-t,v3,1/2] scripts/verify-blacklist: Script for checking blacklist files
URL : https://patchwork.freedesktop.org/series/78368/
State : warning
== Summary ==
Did not get list of undocumented tests for this run, something is wrong!
Other than that, pipeline status: FAILED.
see https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/pipelines/160705 for the overview.
build-containers:build-debian-arm64 has failed (https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/jobs/3098121):
Using docker image sha256:594aa868d31ee3304dee8cae8a3433c89a6fcfcf6c7d420c04cce22f60147176 for registry.freedesktop.org/wayland/ci-templates/buildah:2019-08-13.0 ... section_end:1592218945:prepare_executor
section_start:1592218945:prepare_script
Preparing environment
Running on runner-j4Lrg1oF-project-3185-concurrent-1 via gst-htz-1...
section_end:1592218947:prepare_script
section_start:1592218947:get_sources
Getting source from Git repository
Fetching changes...
Initialized empty Git repository in /builds/gfx-ci/igt-ci-tags/.git/
Created fresh repository.
warning: redirecting to https://gitlab-ci-hetzner.freedesktop.org/gfx-ci/igt-ci-tags.git/
error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
fatal: the remote end hung up unexpectedly
section_end:1592219009:get_sources
section_start:1592219009:upload_artifacts_on_failure
Uploading artifacts for failed job
section_end:1592219011:upload_artifacts_on_failure
ERROR: Job failed: exit code 1
build-containers:build-debian-armhf has failed (https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/-/jobs/3098120):
Using docker image sha256:594aa868d31ee3304dee8cae8a3433c89a6fcfcf6c7d420c04cce22f60147176 for registry.freedesktop.org/wayland/ci-templates/buildah:2019-08-13.0 ... section_end:1592218945:prepare_executor
section_start:1592218945:prepare_script
Preparing environment
Running on runner-z2CifdYy-project-3185-concurrent-0 via gst-htz-3...
section_end:1592218946:prepare_script
section_start:1592218946:get_sources
Getting source from Git repository
Fetching changes...
Initialized empty Git repository in /builds/gfx-ci/igt-ci-tags/.git/
Created fresh repository.
warning: redirecting to https://gitlab-ci-hetzner.freedesktop.org/gfx-ci/igt-ci-tags.git/
error: RPC failed; curl 92 HTTP/2 stream 0 was not closed cleanly: PROTOCOL_ERROR (err 1)
fatal: the remote end hung up unexpectedly
section_end:1592219009:get_sources
section_start:1592219009:upload_artifacts_on_failure
Uploading artifacts for failed job
section_end:1592219010:upload_artifacts_on_failure
ERROR: Job failed: exit code 1
== Logs ==
For more details see: https://gitlab.freedesktop.org/gfx-ci/igt-ci-tags/pipelines/160705
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev