./scripts/igt_doc.py: speedup check logic
v2 cover ./scripts/igt_doc.py: speedup check logic
From: Mauro Carvalho Chehab <[email protected]>
The logic which checks for missing/obsolete documentation is too slow. Currently, on my notebook it takes 12 seconds:
$ time ./scripts/igt_doc.py --check --config tests/xe/xe_*json
real 0m12.085s user 0m10.468s sys 0m1.457s
Implent it on a different way, in order to speedup the build process.
After the change, it now takes less than 500ms:
$ time ./scripts/igt_doc.py --check --config tests/xe/xe_*json
real 0m0.460s user 0m0.357s sys 0m0.102s
It should be noticed that it is possible to speed it up even further by running:
build/tests/* --list-subtests
tasks in parallel. I did some tests: the improvements weren't so relevant on my notebook with 8 CPUs, and, on my tests with python 3.11 it seemed a little too fragile when using multithreading, as it was not hard to mess with it, not having threads or subprocess forks not happening, and without triggering an error.
As the benefit of such optimization was marginal, let's avoid the additional complexity of multithreading.
---
v2:
- remove a now uneeded loop
Mauro Carvalho Chehab (3):
scripts/igt_doc.py: cleanup some pylint warnings scripts/igt_doc.py: don't depend on igt_runner anymore scripts/test_list.py: use a compiled regex for check
Mauro Carvalho Chehab (3):
scripts/igt_doc.py: cleanup some pylint warnings scripts/igt_doc.py: don't depend on igt_runner anymore scripts/test_list.py: use a compiled regex for check
docs/testplan/meson.build | 7 ++-- scripts/igt_doc.py | 11 +++--- scripts/test_list.py | 73 ++++++++++++++++++++++++++------------- 3 files changed, 55 insertions(+), 36 deletions(-)
Signature · 1 line
2.39.2
PATCH v2 1/3 scripts/igt_doc.py: cleanup some pylint warnings
From: Mauro Carvalho Chehab <[email protected]>
Ensure that pylint won't report any issues on igt_doc.py.
--- scripts/igt_doc.py | 1 + scripts/test_list.py | 4 ++-- 2 files changed, 3 insertions(+), 2 deletions(-)
scripts/igt_doc.py +1 -0
@@ -11,6 +11,7 @@ """Maintain test plan and test implementation documentation on IGT.""" import argparse +import sys from test_list import TestList
scripts/test_list.py +2 -2
@@ -1,5 +1,5 @@ #!/usr/bin/env python3 -# pylint: disable=C0301,R0902,R0914,R0912,R0915,R1702,C0302 +# pylint: disable=C0301,R0902,R0914,R0912,R0913,R0915,R1702,C0302 # SPDX-License-Identifier: (GPL-2.0 OR MIT) ## Copyright (C) 2023 Intel Corporation ##
@@ -394,7 +394,7 @@ class TestList: for subtest in self.doc[test]["subtest"].keys(): summary = test_name if self.doc[test]["subtest"][subtest]["Summary"] != '': - summary += '@' + self.doc[test]["subtest"][subtest]["Summary"] + summary += '@' + self.doc[test]["subtest"][subtest]["Summary"] if not summary: continue
Signature · 1 line
2.39.2
PATCH v2 2/3 scripts/igt_doc.py: don't depend on igt_runner anymore
In reply to Mauro Carvalho ChehabFrom: Mauro Carvalho Chehab <[email protected]>
There's no need to actually call IGT runner to get test lists.
Remove such dependency, in order to speedup --check.
--- docs/testplan/meson.build | 7 ++---- scripts/igt_doc.py | 10 +++----- scripts/test_list.py | 53 ++++++++++++++++++++++++++------------- 3 files changed, 41 insertions(+), 29 deletions(-)
docs/testplan/meson.build +2 -5
@@ -11,11 +11,8 @@ xe_test_config = join_paths(source_root, 'tests', 'xe', 'xe_test_config.json') check_testlist = [] if build_tests doc_dependencies = test_executables - if jsonc.found() - # Check if documentation matches the actual tests - check_testlist = [ '--check-testlist', '--igt-build-path', build_root ] - doc_dependencies += runner - endif + # Check if documentation matches the actual tests + check_testlist = [ '--check-testlist', '--igt-build-path', build_root ] else doc_dependencies = [] endif
scripts/igt_doc.py +3 -7
@@ -16,7 +16,6 @@ import sys from test_list import TestList IGT_BUILD_PATH = 'build' -IGT_RUNNER = 'runner/igt_runner' parser = argparse.ArgumentParser(description = "Print formatted kernel documentation to stdout.", formatter_class = argparse.ArgumentDefaultsHelpFormatter,
@@ -36,24 +35,21 @@ parser.add_argument("--sort-field", parser.add_argument("--filter-field", help="modify --show-subtests to filter output based a regex given by FILTER_FIELD=~'regex'") parser.add_argument("--check-testlist", action="store_true", - help="Compare documentation against IGT runner testlist.") + help="Compare documentation against IGT built tests.") parser.add_argument("--include-plan", action="store_true", help="Include test plans, if any.") parser.add_argument("--igt-build-path", - help="Path where the IGT runner is sitting. Used by --check-testlist.", + help="Path to the IGT build directory. Used by --check-testlist.", default=IGT_BUILD_PATH) parser.add_argument("--gen-testlist", help="Generate documentation at the GEN_TESTLIST directory, using SORT_FIELD to split the tests. Requires --sort-field.") -parser.add_argument("--igt-runner", - help="Path where the IGT runner is sitting. Used by --check-testlist.", - default=IGT_RUNNER) parser.add_argument('--files', nargs='+', help="File name(s) to be processed") parse_args = parser.parse_args() tests = TestList(parse_args.config, parse_args.include_plan, parse_args.files, - parse_args.igt_build_path, parse_args.igt_runner) + parse_args.igt_build_path) RUN = 0 if parse_args.show_subtests:
scripts/test_list.py +36 -17
84 lines not on this page · raw message
Signature · 1 line
2.39.2
PATCH v2 3/3 scripts/test_list.py: use a compiled regex for check
In reply to Mauro Carvalho ChehabFrom: Mauro Carvalho Chehab <[email protected]>
Speed up even further the logic for --check-testlist by pre-compiling the regex that will be used to validate the results.
--- scripts/test_list.py | 16 +++++++++++----- 1 file changed, 11 insertions(+), 5 deletions(-)
scripts/test_list.py +11 -5
@@ -820,10 +820,18 @@ class TestList: run_missing = [] doc_uneeded = [] + test_regex = r"" + for doc_test in doc_subtests: + if test_regex != r"": + test_regex += r"|" + test_regex += r'^' + doc_test + r'$' + + test_regex = re.compile(test_regex) + for doc_test in doc_subtests: found = False for run_test in run_subtests: - if re.match(r'^' + doc_test + r'$', run_test): + if re.match(test_regex, run_test): found = True break if not found:
@@ -831,10 +839,8 @@ class TestList: for run_test in run_subtests: found = False - for doc_test in doc_subtests: - if re.match(r'^' + doc_test + r'$', run_test): - found = True - break + if re.match(test_regex, run_test): + found = True if not found: run_missing.append(run_test)
Signature · 1 line
2.39.2
Patchwork
CI[igt-dev] ✗ Fi.CI.BAT: failure for ./scripts/igt_doc.py: speedup check logic (rev2)
In reply to Mauro Carvalho Chehab== Series Details ==
== Series Details ==
Series: ./scripts/igt_doc.py: speedup check logic (rev2)
URL : https://patchwork.freedesktop.org/series/116379/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_12995 -> IGTPW_8790
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with IGTPW_8790 absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in IGTPW_8790, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/index.html
Participating hosts (37 -> 35)
------------------------------
Missing (2): fi-kbl-soraka fi-snb-2520m
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in IGTPW_8790:
### IGT changes ###
#### Possible regressions ####
* igt@i915_selftest@live@hangcheck:
- fi-hsw-4770: [PASS][1] -> [DMESG-WARN][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/fi-hsw-4770/igt@i915_selftest@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/fi-hsw-4770/igt@i915_selftest@[email protected]
Known issues
------------
Here are the changes found in IGTPW_8790 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@i915_pm_rps@basic-api:
- bat-dg2-11: [PASS][3] -> [FAIL][4] ([i915#8308])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/bat-dg2-11/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-dg2-11/igt@[email protected]
* igt@i915_selftest@live@gt_heartbeat:
- fi-glk-j4005: [PASS][5] -> [DMESG-FAIL][6] ([i915#5334])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/fi-glk-j4005/igt@i915_selftest@live@gt_heartbeat.html
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/fi-glk-j4005/igt@i915_selftest@live@gt_heartbeat.html
- fi-skl-guc: [PASS][7] -> [DMESG-FAIL][8] ([i915#5334])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/fi-skl-guc/igt@i915_selftest@live@gt_heartbeat.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/fi-skl-guc/igt@i915_selftest@live@gt_heartbeat.html
* igt@i915_selftest@live@hangcheck:
- bat-dg2-11: [PASS][9] -> [ABORT][10] ([i915#7913] / [i915#7979])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/bat-dg2-11/igt@i915_selftest@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-dg2-11/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@migrate:
- bat-dg2-11: [PASS][11] -> [DMESG-WARN][12] ([i915#7699])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/bat-dg2-11/igt@i915_selftest@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-dg2-11/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@reset:
- bat-rpls-2: [PASS][13] -> [ABORT][14] ([i915#4983] / [i915#7913] / [i915#7981])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/bat-rpls-2/igt@i915_selftest@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-rpls-2/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@slpc:
- bat-rpls-1: [PASS][15] -> [DMESG-FAIL][16] ([i915#6367] / [i915#7996])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/bat-rpls-1/igt@i915_selftest@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-rpls-1/igt@i915_selftest@[email protected]
- bat-adln-1: NOTRUN -> [DMESG-FAIL][17] ([i915#6997])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-adln-1/igt@i915_selftest@[email protected]
* igt@kms_chamelium_hpd@common-hpd-after-suspend:
- bat-adln-1: NOTRUN -> [SKIP][18] ([i915#7828])
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-adln-1/igt@[email protected]
- bat-rpls-1: NOTRUN -> [SKIP][19] ([i915#7828])
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-rpls-1/igt@[email protected]
* igt@kms_pipe_crc_basic@read-crc:
- bat-adlp-9: NOTRUN -> [SKIP][20] ([i915#3546]) +1 similar issue
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-adlp-9/igt@[email protected]
* igt@kms_pipe_crc_basic@suspend-read-crc:
- bat-rpls-1: NOTRUN -> [SKIP][21] ([i915#1845])
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-rpls-1/igt@[email protected]
#### Possible fixes ####
* igt@gem_exec_suspend@basic-s3@smem:
- bat-rpls-1: [ABORT][22] ([i915#6687] / [i915#7978]) -> [PASS][23]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/bat-rpls-1/igt@gem_exec_suspend@[email protected]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-rpls-1/igt@gem_exec_suspend@[email protected]
* igt@i915_module_load@load:
- fi-elk-e7500: [DMESG-WARN][24] ([i915#4391]) -> [PASS][25]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/fi-elk-e7500/igt@[email protected]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/fi-elk-e7500/igt@[email protected]
* igt@i915_selftest@live@gt_lrc:
- bat-adln-1: [INCOMPLETE][26] ([i915#7609]) -> [PASS][27]
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/bat-adln-1/igt@i915_selftest@live@gt_lrc.html
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-adln-1/igt@i915_selftest@live@gt_lrc.html
* igt@kms_pipe_crc_basic@nonblocking-crc-frame-sequence@pipe-d-dp-1:
- bat-dg2-8: [FAIL][28] ([i915#7932]) -> [PASS][29]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_12995/bat-dg2-8/igt@kms_pipe_crc_basic@[email protected]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/bat-dg2-8/igt@kms_pipe_crc_basic@[email protected]
[i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
[i915#3546]: https://gitlab.freedesktop.org/drm/intel/issues/3546
[i915#4391]: https://gitlab.freedesktop.org/drm/intel/issues/4391
[i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983
[i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
[i915#6367]: https://gitlab.freedesktop.org/drm/intel/issues/6367
[i915#6687]: https://gitlab.freedesktop.org/drm/intel/issues/6687
[i915#6997]: https://gitlab.freedesktop.org/drm/intel/issues/6997
[i915#7609]: https://gitlab.freedesktop.org/drm/intel/issues/7609
[i915#7699]: https://gitlab.freedesktop.org/drm/intel/issues/7699
[i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
[i915#7913]: https://gitlab.freedesktop.org/drm/intel/issues/7913
[i915#7932]: https://gitlab.freedesktop.org/drm/intel/issues/7932
[i915#7978]: https://gitlab.freedesktop.org/drm/intel/issues/7978
[i915#7979]: https://gitlab.freedesktop.org/drm/intel/issues/7979
[i915#7981]: https://gitlab.freedesktop.org/drm/intel/issues/7981
[i915#7996]: https://gitlab.freedesktop.org/drm/intel/issues/7996
[i915#8308]: https://gitlab.freedesktop.org/drm/intel/issues/8308
Build changes
-------------
* CI: CI-20190529 -> None
* IGT: IGT_7251 -> IGTPW_8790
CI-20190529: 20190529
CI_DRM_12995: 0c5be3027d7ac4a8b107bf1113867744aa3bb440 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_8790: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/index.html
IGT_7251: 55fa959aad79b3771350a801c1c2dbd4e5034102 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_8790/index.html