RFC: tests/kms_setmode: rename subtest to simplify test combinations
PATCH 1/1 RFC: tests/kms_setmode: rename subtest to simplify test combinations
Both 'basic-clone-single-crtc' and 'invalid-clone-single-crtc' tests aim to assess a scenario where two connectors are attempted to be run with a single CRTC, which is not feasible. Consequently, the 'basic-clone-single-crtc' test is marked as invalid and skipped due to the impractical scenario. On the other hand, the 'invalid-clone-single-crtc' test serves as a negative scenario, confirming the failure of the 'ioctl' and indicating a successful test.
similarly, the 'invalid-clone-exclusive-crtc' test attempts to run two connectors with two CRTCs, which is indeed possible, rendering the test nonsensical. Consequently, the 'clone-exclusive-crtc' test runs and passes as expected.
--- tests/kms_setmode.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)
tests/kms_setmode.c +1 -5
@@ -932,12 +932,8 @@ igt_main_args("det:", NULL, help_str, opt_handler, NULL)const char *name; } tests[] = { { TEST_TIMINGS, "basic" }, - { TEST_CLONE | TEST_SINGLE_CRTC_CLONE, - "basic-clone-single-crtc" }, { TEST_INVALID | TEST_CLONE | TEST_SINGLE_CRTC_CLONE, - "invalid-clone-single-crtc" }, - { TEST_INVALID | TEST_CLONE | TEST_EXCLUSIVE_CRTC_CLONE, - "invalid-clone-exclusive-crtc" }, + "clone-single-crtc-negative" }, { TEST_CLONE | TEST_EXCLUSIVE_CRTC_CLONE, "clone-exclusive-crtc" }, { TEST_INVALID | TEST_CLONE | TEST_SINGLE_CRTC_CLONE | TEST_STEALING,
Signature · 1 line
2.25.1
Patchwork
CI[igt-dev] ✗ Fi.CI.BAT: failure for RFC: tests/kms_setmode: rename subtest to simplify test combinations
== Series Details ==
== Series Details ==
Series: RFC: tests/kms_setmode: rename subtest to simplify test combinations
URL : https://patchwork.freedesktop.org/series/122409/
State : failure
== Summary ==
CI Bug Log - changes from IGT_7435 -> IGTPW_9583
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with IGTPW_9583 absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in IGTPW_9583, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/index.html
Participating hosts (41 -> 40)
------------------------------
Missing (1): fi-snb-2520m
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in IGTPW_9583:
### IGT changes ###
#### Possible regressions ####
* igt@i915_pm_rpm@module-reload:
- fi-apl-guc: [PASS][1] -> [DMESG-FAIL][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/fi-apl-guc/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/fi-apl-guc/igt@[email protected]
#### Warnings ####
* igt@kms_setmode@basic-clone-single-crtc:
- bat-adlp-9: [SKIP][3] ([i915#3555]) -> [SKIP][4]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-adlp-9/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-adlp-9/igt@[email protected]
- fi-rkl-11600: [SKIP][5] ([i915#3555] / [i915#4098]) -> [SKIP][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/fi-rkl-11600/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/fi-rkl-11600/igt@[email protected]
- bat-adls-5: [SKIP][7] ([i915#3555]) -> [SKIP][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-adls-5/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-adls-5/igt@[email protected]
- bat-dg1-5: [SKIP][9] ([i915#3555]) -> [SKIP][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-dg1-5/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-dg1-5/igt@[email protected]
- bat-dg2-9: [SKIP][11] ([i915#3555]) -> [SKIP][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-dg2-9/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-dg2-9/igt@[email protected]
- fi-tgl-1115g4: [SKIP][13] ([i915#3555]) -> [SKIP][14]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/fi-tgl-1115g4/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/fi-tgl-1115g4/igt@[email protected]
- bat-mtlp-6: [SKIP][15] ([i915#8809]) -> [SKIP][16]
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-mtlp-6/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-6/igt@[email protected]
- bat-rpls-2: [SKIP][17] ([i915#3555]) -> [SKIP][18]
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-rpls-2/igt@[email protected]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-rpls-2/igt@[email protected]
- bat-dg2-11: [SKIP][19] ([i915#3555]) -> [SKIP][20]
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-dg2-11/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-dg2-11/igt@[email protected]
- bat-mtlp-8: [SKIP][21] ([i915#8809]) -> [SKIP][22]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-mtlp-8/igt@[email protected]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-8/igt@[email protected]
- bat-dg2-8: [SKIP][23] ([i915#3555]) -> [SKIP][24]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-dg2-8/igt@[email protected]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-dg2-8/igt@[email protected]
- bat-adlm-1: [SKIP][25] ([i915#3555]) -> [SKIP][26]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-adlm-1/igt@[email protected]
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-adlm-1/igt@[email protected]
- bat-rpls-1: [SKIP][27] ([i915#3555]) -> [SKIP][28]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-rpls-1/igt@[email protected]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-rpls-1/igt@[email protected]
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.
* igt@kms_setmode@basic-clone-single-crtc:
- {bat-dg2-14}: [SKIP][29] ([i915#3555]) -> [SKIP][30]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-dg2-14/igt@[email protected]
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-dg2-14/igt@[email protected]
Known issues
------------
Here are the changes found in IGTPW_9583 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@i915_selftest@live@gt_mocs:
- bat-mtlp-8: [PASS][31] -> [DMESG-FAIL][32] ([i915#7059])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-mtlp-8/igt@i915_selftest@live@gt_mocs.html
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-8/igt@i915_selftest@live@gt_mocs.html
* igt@i915_selftest@live@gt_pm:
- bat-rpls-2: NOTRUN -> [DMESG-FAIL][33] ([i915#4258] / [i915#7913])
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-rpls-2/igt@i915_selftest@live@gt_pm.html
* igt@i915_selftest@live@migrate:
- bat-mtlp-8: NOTRUN -> [DMESG-FAIL][34] ([i915#7699])
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-8/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@requests:
- bat-mtlp-6: [PASS][35] -> [DMESG-FAIL][36] ([i915#8497])
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-mtlp-6/igt@i915_selftest@[email protected]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-6/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@slpc:
- bat-rpls-2: NOTRUN -> [DMESG-WARN][37] ([i915#6367])
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-rpls-2/igt@i915_selftest@[email protected]
- bat-mtlp-8: NOTRUN -> [DMESG-WARN][38] ([i915#6367])
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-8/igt@i915_selftest@[email protected]
* igt@i915_suspend@basic-s3-without-i915:
- bat-mtlp-8: NOTRUN -> [SKIP][39] ([i915#6645])
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-8/igt@[email protected]
* igt@kms_chamelium_hpd@common-hpd-after-suspend:
- bat-rpls-2: NOTRUN -> [SKIP][40] ([i915#7828])
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-rpls-2/igt@[email protected]
- bat-mtlp-8: NOTRUN -> [SKIP][41] ([i915#7828])
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-8/igt@[email protected]
* igt@kms_pipe_crc_basic@suspend-read-crc:
- bat-rpls-2: NOTRUN -> [SKIP][42] ([i915#1845])
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-rpls-2/igt@[email protected]
#### Possible fixes ####
* igt@i915_selftest@live@gt_heartbeat:
- fi-kbl-soraka: [DMESG-FAIL][43] ([i915#5334] / [i915#7872]) -> [PASS][44]
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/fi-kbl-soraka/igt@i915_selftest@live@gt_heartbeat.html
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/fi-kbl-soraka/igt@i915_selftest@live@gt_heartbeat.html
* igt@i915_selftest@live@gt_lrc:
- bat-rpls-2: [INCOMPLETE][45] ([i915#4983] / [i915#7913]) -> [PASS][46]
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-rpls-2/igt@i915_selftest@live@gt_lrc.html
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-rpls-2/igt@i915_selftest@live@gt_lrc.html
* igt@i915_selftest@live@guc:
- bat-rpls-1: [DMESG-WARN][47] ([i915#7852]) -> [PASS][48]
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-rpls-1/igt@i915_selftest@[email protected]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-rpls-1/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@slpc:
- bat-mtlp-6: [DMESG-WARN][49] ([i915#6367]) -> [PASS][50]
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-mtlp-6/igt@i915_selftest@[email protected]
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-6/igt@i915_selftest@[email protected]
#### Warnings ####
* igt@i915_selftest@live@requests:
- bat-mtlp-8: [ABORT][51] ([i915#7982] / [i915#8865]) -> [DMESG-FAIL][52] ([i915#8497])
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-mtlp-8/igt@i915_selftest@[email protected]
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-mtlp-8/igt@i915_selftest@[email protected]
* igt@kms_setmode@basic-clone-single-crtc:
- bat-jsl-3: [SKIP][53] ([i915#3555]) -> [SKIP][54] ([fdo#112080])
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-jsl-3/igt@[email protected]
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-jsl-3/igt@[email protected]
- bat-jsl-1: [SKIP][55] ([i915#3555]) -> [SKIP][56] ([fdo#112080])
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/IGT_7435/bat-jsl-1/igt@[email protected]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/bat-jsl-1/igt@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#112080]: https://bugs.freedesktop.org/show_bug.cgi?id=112080
[i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
[i915#3555]: https://gitlab.freedesktop.org/drm/intel/issues/3555
[i915#4098]: https://gitlab.freedesktop.org/drm/intel/issues/4098
[i915#4258]: https://gitlab.freedesktop.org/drm/intel/issues/4258
[i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983
[i915#5334]: https://gitlab.freedesktop.org/drm/intel/issues/5334
[i915#6367]: https://gitlab.freedesktop.org/drm/intel/issues/6367
[i915#6645]: https://gitlab.freedesktop.org/drm/intel/issues/6645
[i915#7059]: https://gitlab.freedesktop.org/drm/intel/issues/7059
[i915#7699]: https://gitlab.freedesktop.org/drm/intel/issues/7699
[i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
[i915#7852]: https://gitlab.freedesktop.org/drm/intel/issues/7852
[i915#7872]: https://gitlab.freedesktop.org/drm/intel/issues/7872
[i915#7913]: https://gitlab.freedesktop.org/drm/intel/issues/7913
[i915#7982]: https://gitlab.freedesktop.org/drm/intel/issues/7982
[i915#8497]: https://gitlab.freedesktop.org/drm/intel/issues/8497
[i915#8809]: https://gitlab.freedesktop.org/drm/intel/issues/8809
[i915#8865]: https://gitlab.freedesktop.org/drm/intel/issues/8865
Build changes
-------------
* CI: CI-20190529 -> None
* IGT: IGT_7435 -> IGTPW_9583
CI-20190529: 20190529
CI_DRM_13515: 8dffdb87e35b56a507daec2cbcf5814a7e44d15c @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_9583: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/index.html
IGT_7435: b6eaa6bfdc94c94b34ec80f437c4b6125eb19357 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
Testlist changes
----------------
+igt@kms_setmode@clone-single-crtc-negative
-igt@kms_setmode@basic-clone-single-crtc
-igt@kms_setmode@invalid-clone-exclusive-crtc
-igt@kms_setmode@invalid-clone-single-crtc
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9583/index.html
Patchwork
CI[igt-dev] ○ CI.xeBAT: info for RFC: tests/kms_setmode: rename subtest to simplify test combinations
In reply to Jeevan B== Series Details ==
== Series Details ==
Series: RFC: tests/kms_setmode: rename subtest to simplify test combinations
URL : https://patchwork.freedesktop.org/series/122409/
State : info
== Summary ==
Participating hosts:
bat-atsm-2
bat-dg2-oem2
bat-adlp-7
Missing hosts results[0]:
Results: [IGTPW_9583](https://intel-gfx-ci.01.org/tree/intel-xe/IGTPW_9583/index.html)
Quoted 22 lines · Jeevan B
Both 'basic-clone-single-crtc' and 'invalid-clone-single-crtc' tests aim to assess a scenario where two connectors are attempted to be run with a single CRTC, which is not feasible. Consequently, the 'basic-clone-single-crtc' test is marked as invalid and skipped due to the impractical scenario. On the other hand, the 'invalid-clone-single-crtc' test serves as a negative scenario, confirming the failure of the 'ioctl' and indicating a successful test.
similarly, the 'invalid-clone-exclusive-crtc' test attempts to run two connectors with two CRTCs, which is indeed possible, rendering the test nonsensical. Consequently, the 'clone-exclusive-crtc' test runs and passes as expected.
Signed-off-by Jeevan B <[email protected]>Cc Imre Deak <[email protected]>--- tests/kms_setmode.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)tests/kms_setmode.c +0
-0
tests/kms_setmode.c igt_main_args("det:", NULL, help_str, opt_handler, NULL) · line 936
Quoted 5 lines
tests/kms_setmode.c +0
-2@@ -932,12 +932,8 @@ igt_main_args("det:", NULL, help_str, opt_handler, NULL)const char *name; } tests[] = { { TEST_TIMINGS, "basic" }, - { TEST_CLONE | TEST_SINGLE_CRTC_CLONE, - "basic-clone-single-crtc" },
It is feasible to use a single CRTC with two connectors, for instance in case of a VGA and an HDMI connector; so this subtest can't be removed.
tests/kms_setmode.c igt_main_args("det:", NULL, help_str, opt_handler, NULL) · line 939
Quoted 3 lines
tests/kms_setmode.c +0
-2{ TEST_INVALID | TEST_CLONE | TEST_SINGLE_CRTC_CLONE, - "invalid-clone-single-crtc" }, - { TEST_INVALID | TEST_CLONE | TEST_EXCLUSIVE_CRTC_CLONE,
There are invalid configurations when each connector uses its own CRTC, for instance when a connector uses a CRTC not compatible with the connector. So this subtest can't be removed either.
I suppose you're fixing a problem where a subtest is failing if there are no dynamic tests defined for it, on some platforms. In that case the subtest should be made to pass explicitly, perhaps by ensuring that there is at least one dynamic test defined for each subtest.
Quoted 8 lines
tests/kms_setmode.c +1
-1- "invalid-clone-exclusive-crtc" }, + "clone-single-crtc-negative" }, {TEST_CLONE |TEST_EXCLUSIVE_CRTC_CLONE , "clone-exclusive-crtc" }, { TEST_INVALID | TEST_CLONE | TEST_SINGLE_CRTC_CLONE | TEST_STEALING,Signature · 1 line
2.25.1
Quoted 13 lines
-----Original Message-----
From: Deak, Imre <[email protected]>
Sent: Sunday, August 27, 2023 11:33 PM
To: B, Jeevan <[email protected]>
Cc: [email protected]
Subject: Re: [PATCH] RFC: tests/kms_setmode: rename subtest to simplify test combinationsQuoted 3 lines · Jeevan B
Both 'basic-clone-single-crtc' and 'invalid-clone-single-crtc' tests aim to assess a scenario where two connectors are attempted to be run with a single CRTC, which is not feasible. Consequently, the 'basic-clone-
single-crtc'
Quoted 10 lines
Quoted 3 lines
test is marked as invalid and skipped due to the impractical scenario. On the other hand, the 'invalid-clone-single-crtc' test serves as a negative scenario, confirming the failure of the 'ioctl' and indicating a
successful test.
Quoted 3 lines
similarly, the 'invalid-clone-exclusive-crtc' test attempts to run two connectors with two CRTCs, which is indeed possible, rendering the test
nonsensical.
Quoted 2 lines
Consequently, the 'clone-exclusive-crtc' test runs and passes as expected.
Thanks for the inputs. Will work on the comments given. .
Quoted 45 lines
tests/kms_setmode.c igt_main_args("det:", NULL, help_str,
Quoted 11 lines
Signed-off-by Jeevan B <[email protected]>Cc Imre Deak <[email protected]>--- tests/kms_setmode.c | 6 +----- 1 file changed, 1 insertion(+), 5 deletions(-)tests/kms_setmode.c index renamed from tests/kms_setmode.c +0
-0
035bbd0b0..0c65e3937100644tests/kms_setmode.c +0
-0@@ -932,12 +932,8 @@ igt_main_args("det:", NULL, help_str,
opt_handler, NULL)tests/kms_setmode.c igt_main_args("det:", NULL, help_str, opt_handler, NULL) · line 936
Quoted 5 lines
tests/kms_setmode.c +0
-2const char *name; } tests[] = { { TEST_TIMINGS, "basic" }, - { TEST_CLONE | TEST_SINGLE_CRTC_CLONE, - "basic-clone-single-crtc" },It is feasible to use a single CRTC with two connectors, for instance in case of a VGA and an HDMI connector; so this subtest can't be removed.
tests/kms_setmode.c igt_main_args("det:", NULL, help_str, opt_handler, NULL)
Quoted 3 lines
tests/kms_setmode.c +0
-2{ TEST_INVALID | TEST_CLONE | TEST_SINGLE_CRTC_CLONE, - "invalid-clone-single-crtc" }, - { TEST_INVALID | TEST_CLONE |
TEST_EXCLUSIVE_CRTC_CLONE,There are invalid configurations when each connector uses its own CRTC, for instance when a connector uses a CRTC not compatible with the connector. So this subtest can't be removed either.
I suppose you're fixing a problem where a subtest is failing if there are no dynamic tests defined for it, on some platforms. In that case the subtest should be made to pass explicitly, perhaps by ensuring that there is at least one dynamic test defined for each subtest.
tests/kms_setmode.c igt_main_args("det:", NULL, help_str, opt_handler, NULL)
Quoted 5 lines
tests/kms_setmode.c +1
-1- "invalid-clone-exclusive-crtc" }, + "clone-single-crtc-negative" }, {TEST_CLONE |TEST_EXCLUSIVE_CRTC_CLONE , "clone-exclusive-crtc" }, { TEST_INVALID | TEST_CLONE | TEST_SINGLE_CRTC_CLONE|
Quoted 4 lines
TEST_STEALING,
--
2.25.1