tests/kms_setmode: Fix dynamic subtests
PATCH 1/1 tests/kms_setmode: Fix dynamic subtests
basic-clone-single-crtc and invalid-clone-exclusive-crtc were not part of dynamic subtest so tests were getting skipped. fixed the test to execute skipping tests as expected.
v2: add missing call.
--- tests/kms_setmode.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)
tests/kms_setmode.c +13 -13
@@ -647,20 +647,19 @@ static void test_one_combination(const struct test_config *tconf,struct crtc_config crtcs[MAX_CRTCS ];int crtc_count;bool config_valid; +int i, pos =0 ; +char test_name[256 ];setup_crtcs (tconf, cconfs, connector_count, crtcs, &crtc_count, &config_valid); -if (config_valid == !(tconf->flags &TEST_INVALID )) { -int i, pos =0 ; -char test_name[256 ]; - -for (i =0 ; i < crtc_count; i++) { -if (i >0 ) - pos +=snprintf (&test_name[pos],ARRAY_SIZE (test_name) - pos,"-" ); - pos +=get_test_name_str (&crtcs[i], &test_name[pos],ARRAY_SIZE (test_name) - pos); - } +for (i =0 ; i < crtc_count; i++) { +if (i >0 ) + pos +=snprintf (&test_name[pos],ARRAY_SIZE (test_name) - pos,"-" ); + pos +=get_test_name_str (&crtcs[i], &test_name[pos],ARRAY_SIZE (test_name) - pos); + } +if (config_valid == !(tconf->flags &TEST_INVALID )) {for (i =0 ; i < crtc_count; i++) {struct crtc_config *crtc = &crtcs[i];
@@ -679,16 +678,17 @@ static void test_one_combination(const struct test_config *tconf, ((i > 0) && (crtc[i - 1].mode.hdisplay > MAX_HDISPLAY_PER_CRTC) && (abs(crtc->crtc_idx - crtcs[i - 1].crtc_idx) <= 1))) {igt_info ("Combo: %s is not possible with selected mode(s).\n" , test_name); - goto out; +cleanup_crtcs (crtcs, crtc_count); +return ; } } igt_dynamic_f("%s", test_name)test_crtc_config (tconf, crtcs, crtc_count); + }else { + igt_dynamic_f("%s", test_name) +cleanup_crtcs (crtcs, crtc_count); } - -out: -cleanup_crtcs (crtcs, crtc_count); } static int assign_crtc_to_connectors(const struct test_config *tconf,
Signature · 1 line
2.41.0
Patchwork
CI[igt-dev] ✗ Fi.CI.BAT: failure for tests/kms_setmode: Fix dynamic subtests (rev2)
== Series Details ==
== Series Details ==
Series: tests/kms_setmode: Fix dynamic subtests (rev2)
URL : https://patchwork.freedesktop.org/series/119631/
State : failure
== Summary ==
CI Bug Log - changes from CI_DRM_13306 -> IGTPW_9240
====================================================
Summary
-------
**FAILURE**
Serious unknown changes coming with IGTPW_9240 absolutely need to be verified manually.
If you think the reported changes have nothing to do with the changes introduced in IGTPW_9240, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.
External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/index.html
Participating hosts (42 -> 42)
------------------------------
Additional (1): bat-dg1-8
Missing (1): fi-kbl-soraka
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in IGTPW_9240:
### IGT changes ###
#### Possible regressions ####
* igt@gem_exec_fence@basic-await@ccs2:
- bat-dg2-8: [PASS][1] -> [FAIL][2]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13306/bat-dg2-8/igt@gem_exec_fence@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-dg2-8/igt@gem_exec_fence@[email protected]
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.
* {igt@xe_create@create-massive-size}:
- {bat-dg1-8}: NOTRUN -> [FAIL][3] +2 similar issues
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-dg1-8/igt@[email protected]
New tests
---------
New tests have been introduced between CI_DRM_13306 and IGTPW_9240:
### New IGT tests (4) ###
* igt@kms_setmode@basic-clone-single-crtc@pipe-a-dp-1-dp-2:
- Statuses : 1 pass(s)
- Exec time: [0.0] s
* igt@kms_setmode@basic-clone-single-crtc@pipe-a-hdmi-a-2-dp-2:
- Statuses : 1 pass(s)
- Exec time: [0.0] s
* igt@kms_setmode@basic-clone-single-crtc@pipe-b-dp-1-dp-2:
- Statuses : 1 pass(s)
- Exec time: [0.0] s
* igt@kms_setmode@basic-clone-single-crtc@pipe-b-hdmi-a-2-dp-2:
- Statuses : 1 pass(s)
- Exec time: [0.0] s
Known issues
------------
Here are the changes found in IGTPW_9240 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@i915_module_load@load:
- bat-adlp-11: [PASS][4] -> [ABORT][5] ([i915#4423])
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13306/bat-adlp-11/igt@[email protected]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-adlp-11/igt@[email protected]
* igt@i915_selftest@live@requests:
- bat-rpls-2: [PASS][6] -> [ABORT][7] ([i915#4983] / [i915#7913])
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13306/bat-rpls-2/igt@i915_selftest@[email protected]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-rpls-2/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@workarounds:
- bat-mtlp-6: [PASS][8] -> [DMESG-FAIL][9] ([i915#6763])
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13306/bat-mtlp-6/igt@i915_selftest@[email protected]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-mtlp-6/igt@i915_selftest@[email protected]
* igt@kms_pipe_crc_basic@read-crc-frame-sequence:
- bat-dg2-11: NOTRUN -> [SKIP][10] ([i915#1845] / [i915#5354])
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-dg2-11/igt@[email protected]
* igt@kms_setmode@basic-clone-single-crtc:
- bat-rplp-1: NOTRUN -> [ABORT][11] ([i915#4579] / [i915#8260])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-rplp-1/igt@[email protected]
#### Possible fixes ####
* igt@i915_selftest@live@migrate:
- bat-dg2-11: [DMESG-WARN][12] ([i915#7699]) -> [PASS][13]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13306/bat-dg2-11/igt@i915_selftest@[email protected]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-dg2-11/igt@i915_selftest@[email protected]
* igt@i915_selftest@live@requests:
- bat-mtlp-6: [DMESG-FAIL][14] ([i915#8497]) -> [PASS][15]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13306/bat-mtlp-6/igt@i915_selftest@[email protected]
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-mtlp-6/igt@i915_selftest@[email protected]
* igt@kms_pipe_crc_basic@nonblocking-crc-frame-sequence@pipe-d-dp-1:
- bat-dg2-8: [FAIL][16] ([i915#7932]) -> [PASS][17]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13306/bat-dg2-8/igt@kms_pipe_crc_basic@[email protected]
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-dg2-8/igt@kms_pipe_crc_basic@[email protected]
#### Warnings ####
* igt@kms_psr@primary_mmap_gtt:
- bat-rplp-1: [ABORT][18] ([i915#8442]) -> [SKIP][19] ([i915#1072])
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_13306/bat-rplp-1/igt@kms_psr@primary_mmap_gtt.html
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/bat-rplp-1/igt@kms_psr@primary_mmap_gtt.html
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[i915#1072]: https://gitlab.freedesktop.org/drm/intel/issues/1072
[i915#1845]: https://gitlab.freedesktop.org/drm/intel/issues/1845
[i915#3637]: https://gitlab.freedesktop.org/drm/intel/issues/3637
[i915#4078]: https://gitlab.freedesktop.org/drm/intel/issues/4078
[i915#4391]: https://gitlab.freedesktop.org/drm/intel/issues/4391
[i915#4423]: https://gitlab.freedesktop.org/drm/intel/issues/4423
[i915#4579]: https://gitlab.freedesktop.org/drm/intel/issues/4579
[i915#4983]: https://gitlab.freedesktop.org/drm/intel/issues/4983
[i915#5354]: https://gitlab.freedesktop.org/drm/intel/issues/5354
[i915#6763]: https://gitlab.freedesktop.org/drm/intel/issues/6763
[i915#7699]: https://gitlab.freedesktop.org/drm/intel/issues/7699
[i915#7828]: https://gitlab.freedesktop.org/drm/intel/issues/7828
[i915#7913]: https://gitlab.freedesktop.org/drm/intel/issues/7913
[i915#7932]: https://gitlab.freedesktop.org/drm/intel/issues/7932
[i915#8213]: https://gitlab.freedesktop.org/drm/intel/issues/8213
[i915#8260]: https://gitlab.freedesktop.org/drm/intel/issues/8260
[i915#8442]: https://gitlab.freedesktop.org/drm/intel/issues/8442
[i915#8497]: https://gitlab.freedesktop.org/drm/intel/issues/8497
[i915#8513]: https://gitlab.freedesktop.org/drm/intel/issues/8513
[i915#8676]: https://gitlab.freedesktop.org/drm/intel/issues/8676
[i915#8678]: https://gitlab.freedesktop.org/drm/intel/issues/8678
[i915#8679]: https://gitlab.freedesktop.org/drm/intel/issues/8679
[i915#8698]: https://gitlab.freedesktop.org/drm/intel/issues/8698
[i915#8699]: https://gitlab.freedesktop.org/drm/intel/issues/8699
[i915#8700]: https://gitlab.freedesktop.org/drm/intel/issues/8700
Build changes
-------------
* CI: CI-20190529 -> None
* IGT: IGT_7345 -> IGTPW_9240
CI-20190529: 20190529
CI_DRM_13306: fd802e7196f63720d52eb2bd6451ae9f61df6484 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_9240: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/index.html
IGT_7345: bedba6f82d70f8eb49c53cb2c45ee60a1eec04d1 @ https://gitlab.freedesktop.org/drm/igt-gpu-tools.git
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_9240/index.html
Hi Jeevan,
Quoted 49 lines · Jeevan B
basic-clone-single-crtc and invalid-clone-exclusive-crtc were not part of dynamic subtest so tests were getting skipped. fixed the test to execute skipping tests as expected.
v2: add missing call.
Signed-off-by Jeevan B <[email protected]>--- tests/kms_setmode.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)tests/kms_setmode.c +8
-10@@ -647,20 +647,19 @@ static void test_one_combination(const struct test_config *tconf,struct crtc_config crtcs[MAX_CRTCS ];int crtc_count;bool config_valid; +int i, pos =0 ; +char test_name[256 ];setup_crtcs (tconf, cconfs, connector_count, crtcs, &crtc_count, &config_valid); -if (config_valid == !(tconf->flags &TEST_INVALID )) { -int i, pos =0 ; -char test_name[256 ]; - -for (i =0 ; i < crtc_count; i++) { -if (i >0 ) - pos +=snprintf (&test_name[pos],ARRAY_SIZE (test_name) - pos,"-" ); - pos +=get_test_name_str (&crtcs[i], &test_name[pos],ARRAY_SIZE (test_name) - pos); - } +for (i =0 ; i < crtc_count; i++) { +if (i >0 ) + pos +=snprintf (&test_name[pos],ARRAY_SIZE (test_name) - pos,"-" ); + pos +=get_test_name_str (&crtcs[i], &test_name[pos],ARRAY_SIZE (test_name) - pos); + } +if (config_valid == !(tconf->flags &TEST_INVALID )) {for (i =0 ; i < crtc_count; i++) {struct crtc_config *crtc = &crtcs[i];@@ -679,16 +678,17 @@ static void test_one_combination(const struct test_config *tconf, ((i > 0) && (crtc[i - 1].mode.hdisplay > MAX_HDISPLAY_PER_CRTC) && (abs(crtc->crtc_idx - crtcs[i - 1].crtc_idx) <= 1))) {igt_info ("Combo: %s is not possible with selected mode(s).\n" , test_name); - goto out;
tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 690
Quoted 10 lines
tests/kms_setmode.c +5
-0+cleanup_crtcs (crtcs, crtc_count); +return ; } } igt_dynamic_f("%s", test_name)test_crtc_config (tconf, crtcs, crtc_count); + }else { + igt_dynamic_f("%s", test_name) +cleanup_crtcs (crtcs, crtc_count);
----------------------- ^
The name of function and its placement just before function end suggest it is a cleanup, not a test.
I would advise going with git log and finding when these two tests where executed with "normal", non-dynamic way and then adjusting this function, or adding them as separate subtests with proper configuration.
tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 691
Quoted 4 lines
tests/kms_setmode.c +0
-3} - -out: -cleanup_crtcs (crtcs, crtc_count);
Drop this change, looks dangereous.
Regards,
Kamil
Quoted 6 lines
tests/kms_setmode.c +0
-0} static int assign_crtc_to_connectors(const struct test_config *tconf,Signature · 1 line
2.41.0
Hi Kamil,
Quoted 82 lines
-----Original Message-----
From: Kamil Konieczny <[email protected]>
Sent: Thursday, June 22, 2023 6:26 PM
To: [email protected]
Cc: B, Jeevan <[email protected]>; Modem, Bhanuprakash <[email protected]>
Subject: Re: [PATCH i-g-t] tests/kms_setmode: Fix dynamic subtestsHi Jeevan,
tests/kms_setmode.c static void test_one_combination(const struct
Quoted 16 lines · Jeevan B
basic-clone-single-crtc and invalid-clone-exclusive-crtc were not part of dynamic subtest so tests were getting skipped. fixed the test to execute skipping tests as expected.
v2: add missing call.
Signed-off-by Jeevan B <[email protected]>--- tests/kms_setmode.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)tests/kms_setmode.c index renamed from tests/kms_setmode.c +0
-0
a611d938..96fb0b4b100644tests/kms_setmode.c +0
-0@@ -647,20 +647,19 @@ static void test_one_combination(const struct
test_config*tconf,Quoted 6 lines
tests/kms_setmode.c +2
-0struct crtc_config crtcs[MAX_CRTCS ];int crtc_count;bool config_valid; +int i, pos =0 ; +char test_name[256 ];tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf,
Quoted 10 lines
tests/kms_setmode.c +0
-7setup_crtcs (tconf, cconfs, connector_count, crtcs, &crtc_count, &config_valid); -if (config_valid == !(tconf->flags &TEST_INVALID )) { -int i, pos =0 ; -char test_name[256 ]; - -for (i =0 ; i < crtc_count; i++) { -if (i >0 ) - pos += snprintf(&test_name[pos],
ARRAY_SIZE(test_name)- pos, "-");tests/kms_setmode.c static void test_one_combination(const struct · line 661
Quoted 1 line
tests/kms_setmode.c +0
-1- pos += get_test_name_str(&crtcs[i],&test_name[pos],
ARRAY_SIZE(test_name)- pos);tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf,
Quoted 4 lines
tests/kms_setmode.c +3
-1- } +for (i =0 ; i < crtc_count; i++) { +if (i >0 ) + pos += snprintf(&test_name[pos],
ARRAY_SIZE(test_name)- pos, "-");tests/kms_setmode.c static void test_one_combination(const struct · line 659
Quoted 1 line
tests/kms_setmode.c +1
-0+ pos += get_test_name_str(&crtcs[i], &test_name[pos],
ARRAY_SIZE(test_name)- pos);tests/kms_setmode.c static void test_one_combination(const struct
Quoted 7 lines
tests/kms_setmode.c +2
-0+ } +if (config_valid == !(tconf->flags &TEST_INVALID )) {for (i =0 ; i < crtc_count; i++) {struct crtc_config *crtc = &crtcs[i];@@ -679,16 +678,17 @@ static void test_one_combination(const struct
test_config*tconf,tests/kms_setmode.c static void test_one_combination(const struct · line 678
Quoted 1 line
tests/kms_setmode.c +0
-0((i > 0) && (crtc[i - 1].mode.hdisplay >
MAX_HDISPLAY_PER_CRTC) &&tests/kms_setmode.c static void test_one_combination(const struct · line 680
Quoted 2 lines
tests/kms_setmode.c +0
-0(abs(crtc->crtc_idx - crtcs[i - 1].crtc_idx) <= 1))) { igt_info("Combo: %s is not possible withselected mode(s).\n",
test_name);tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 690
Quoted 11 lines
tests/kms_setmode.c +5
-1- goto out; +cleanup_crtcs (crtcs, crtc_count); +return ; } } igt_dynamic_f("%s", test_name)test_crtc_config (tconf, crtcs, crtc_count); + }else { + igt_dynamic_f("%s", test_name) +cleanup_crtcs (crtcs, crtc_count);----------------------- ^
The name of function and its placement just before function end suggest it is a cleanup, not a test.
Quoted 4 lines
I would advise going with git log and finding when these two tests where executed with "normal", non-dynamic way and then adjusting this function, or adding them as separate subtests with proper configuration.
This was before tests were converted to dynamic. (9f32d552afd7e7bb34b4cc3542e184b3be1a1dc8) @@ -664,8 +684,19 @@ static void test_one_combination(const struct test_config *tconf,
setup_crtcs(tconf, cconfs, connector_count, crtcs,
&crtc_count, &config_valid);
- if (config_valid == !(tconf->flags & TEST_INVALID))
- test_crtc_config(tconf, crtcs, crtc_count);
+ if (config_valid == !(tconf->flags & TEST_INVALID)) {
+ int i, pos = 0;
+ char test_name[256];
+
+ for (i = 0; i < crtc_count; i++) {
+ if (i > 0)
+ pos += snprintf(&test_name[pos], ARRAY_SIZE(test_name) - pos, "-");
+ pos += get_test_name_str(&crtcs[i], &test_name[pos], ARRAY_SIZE(test_name) - pos);
+ }
+
+ igt_dynamic_f("%s", test_name)
+ test_crtc_config(tconf, crtcs, crtc_count);
+ }
cleanup_crtcs(crtcs, crtc_count);
}
Quoted 17 lines
tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 691
Quoted 4 lines
tests/kms_setmode.c +0
-3} - -out: -cleanup_crtcs (crtcs, crtc_count);Drop this change, looks dangereous.
Regards,
KamilQuoted 6 lines
tests/kms_setmode.c +0
-1} static int assign_crtc_to_connectors(const struct test_config *tconf, --2.41.0
Hi Jeevan,
Quoted 3 lines · Jeevan B
basic-clone-single-crtc and invalid-clone-exclusive-crtc were not part of dynamic subtest so tests were getting skipped. fixed the test to execute skipping tests as expected.
What does it mean these tests are not part of dynamic subtests?
AFAIK, all basic-* tests are positive test cases & there are corresponding negative test cases (invalid-*)
If any positive test executed, the corresponding negative test is supposed to skip & vice versa.
Example:
https://intel-gfx-ci.01.org/tree/drm-tip/shards-all.html?testfilter=kms_setmode&hosts=shard-glk
- basic-close-single-crtc: SKIP
- invalid-clone-single-crtc: PASS
- clone-executive-crtc: PASS
- invalid-clone-executive-crtc: SKIP
Quoted 46 lines
v2: add missing call.
Signed-off-by Jeevan B <[email protected]>--- tests/kms_setmode.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)tests/kms_setmode.c +8
-10@@ -647,20 +647,19 @@ static void test_one_combination(const struct test_config *tconf,struct crtc_config crtcs[MAX_CRTCS ];int crtc_count;bool config_valid; +int i, pos =0 ; +char test_name[256 ];setup_crtcs (tconf, cconfs, connector_count, crtcs, &crtc_count, &config_valid); -if (config_valid == !(tconf->flags &TEST_INVALID )) { -int i, pos =0 ; -char test_name[256 ]; - -for (i =0 ; i < crtc_count; i++) { -if (i >0 ) - pos +=snprintf (&test_name[pos],ARRAY_SIZE (test_name) - pos,"-" ); - pos +=get_test_name_str (&crtcs[i], &test_name[pos],ARRAY_SIZE (test_name) - pos); - } +for (i =0 ; i < crtc_count; i++) { +if (i >0 ) + pos +=snprintf (&test_name[pos],ARRAY_SIZE (test_name) - pos,"-" ); + pos +=get_test_name_str (&crtcs[i], &test_name[pos],ARRAY_SIZE (test_name) - pos); + } +if (config_valid == !(tconf->flags &TEST_INVALID )) {for (i =0 ; i < crtc_count; i++) {struct crtc_config *crtc = &crtcs[i];@@ -679,16 +678,17 @@ static void test_one_combination(const struct test_config *tconf, ((i > 0) && (crtc[i - 1].mode.hdisplay > MAX_HDISPLAY_PER_CRTC) && (abs(crtc->crtc_idx - crtcs[i - 1].crtc_idx) <= 1))) {igt_info ("Combo: %s is not possible with selected mode(s).\n" , test_name); - goto out;
tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 690
Quoted 10 lines
tests/kms_setmode.c +5
-0+cleanup_crtcs (crtcs, crtc_count); +return ; } } igt_dynamic_f("%s", test_name)test_crtc_config (tconf, crtcs, crtc_count); + }else { + igt_dynamic_f("%s", test_name) +cleanup_crtcs (crtcs, crtc_count);
We are supposed to execute the test, not a cleanup. This will always throw the test result as PASS (which is bogus) without even testing anything.
- Bhanu
Quoted 7 lines
tests/kms_setmode.c +0
-3} - -out: -cleanup_crtcs (crtcs, crtc_count); } static int assign_crtc_to_connectors(const struct test_config *tconf,
Hi Jeevan,
Quoted 98 lines · Jeevan
Hi Kamil,
Quoted 82 lines
-----Original Message-----
From: Kamil Konieczny <[email protected]>
Sent: Thursday, June 22, 2023 6:26 PM
To: [email protected]
Cc: B, Jeevan <[email protected]>; Modem, Bhanuprakash <[email protected]>
Subject: Re: [PATCH i-g-t] tests/kms_setmode: Fix dynamic subtestsHi Jeevan,
tests/kms_setmode.c static void test_one_combination(const struct
Quoted 16 lines · Jeevan B
basic-clone-single-crtc and invalid-clone-exclusive-crtc were not part of dynamic subtest so tests were getting skipped. fixed the test to execute skipping tests as expected.
v2: add missing call.
Signed-off-by Jeevan B <[email protected]>--- tests/kms_setmode.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)tests/kms_setmode.c index renamed from tests/kms_setmode.c +0
-0
a611d938..96fb0b4b100644tests/kms_setmode.c +0
-0@@ -647,20 +647,19 @@ static void test_one_combination(const struct
test_config*tconf,Quoted 6 lines
tests/kms_setmode.c +2
-0struct crtc_config crtcs[MAX_CRTCS ];int crtc_count;bool config_valid; +int i, pos =0 ; +char test_name[256 ];tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf,
Quoted 10 lines
tests/kms_setmode.c +0
-7setup_crtcs (tconf, cconfs, connector_count, crtcs, &crtc_count, &config_valid); -if (config_valid == !(tconf->flags &TEST_INVALID )) { -int i, pos =0 ; -char test_name[256 ]; - -for (i =0 ; i < crtc_count; i++) { -if (i >0 ) - pos += snprintf(&test_name[pos],
ARRAY_SIZE(test_name)- pos, "-");tests/kms_setmode.c static void test_one_combination(const struct · line 661
Quoted 1 line
tests/kms_setmode.c +0
-1- pos += get_test_name_str(&crtcs[i],&test_name[pos],
ARRAY_SIZE(test_name)- pos);tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf,
Quoted 4 lines
tests/kms_setmode.c +3
-1- } +for (i =0 ; i < crtc_count; i++) { +if (i >0 ) + pos += snprintf(&test_name[pos],
ARRAY_SIZE(test_name)- pos, "-");tests/kms_setmode.c static void test_one_combination(const struct · line 659
Quoted 1 line
tests/kms_setmode.c +1
-0+ pos += get_test_name_str(&crtcs[i], &test_name[pos],
ARRAY_SIZE(test_name)- pos);tests/kms_setmode.c static void test_one_combination(const struct
Quoted 7 lines
tests/kms_setmode.c +2
-0+ } +if (config_valid == !(tconf->flags &TEST_INVALID )) {for (i =0 ; i < crtc_count; i++) {struct crtc_config *crtc = &crtcs[i];@@ -679,16 +678,17 @@ static void test_one_combination(const struct
test_config*tconf,tests/kms_setmode.c static void test_one_combination(const struct · line 678
Quoted 1 line
tests/kms_setmode.c +0
-0((i > 0) && (crtc[i - 1].mode.hdisplay >
MAX_HDISPLAY_PER_CRTC) &&tests/kms_setmode.c static void test_one_combination(const struct · line 680
Quoted 2 lines
tests/kms_setmode.c +0
-0(abs(crtc->crtc_idx - crtcs[i - 1].crtc_idx) <= 1))) { igt_info("Combo: %s is not possible withselected mode(s).\n",
test_name);tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 690
Quoted 11 lines
tests/kms_setmode.c +5
-1- goto out; +cleanup_crtcs (crtcs, crtc_count); +return ; } } igt_dynamic_f("%s", test_name)test_crtc_config (tconf, crtcs, crtc_count); + }else { + igt_dynamic_f("%s", test_name) +cleanup_crtcs (crtcs, crtc_count);----------------------- ^
The name of function and its placement just before function end suggest it is a cleanup, not a test.Quoted 4 lines
I would advise going with git log and finding when these two tests where executed with "normal", non-dynamic way and then adjusting this function, or adding them as separate subtests with proper configuration.
This was before tests were converted to dynamic. (
9f32d552afd7e7bb34b4cc3542e184b3be1a1dc8)(quoted diff) +3
-2@@ -664,8 +684,19 @@ static void test_one_combination(const struct test_config *tconf, setup_crtcs(tconf, cconfs, connector_count, crtcs, &crtc_count, &config_valid); - if (config_valid == !(tconf->flags & TEST_INVALID)) - test_crtc_config(tconf, crtcs, crtc_count); + if (config_valid == !(tconf->flags & TEST_INVALID)) { + int i, pos = 0; + char test_name[256];
Review static void test_one_combination(const struct test_config *tconf, · line 699
Quoted 10 lines
(quoted diff) +10
-0+ + for (i = 0; i < crtc_count; i++) { + if (i > 0) + pos += snprintf(&test_name[pos], ARRAY_SIZE(test_name) - pos, "-"); + pos += get_test_name_str(&crtcs[i], &test_name[pos], ARRAY_SIZE(test_name) - pos); + } + + igt_dynamic_f("%s", test_name) + test_crtc_config(tconf, crtcs, crtc_count); + }
What is wrong in this logic & how your patch fixes that?
- Bhanu
Quoted 21 lines
(quoted diff) +0
-0cleanup_crtcs(crtcs, crtc_count); }Quoted 17 lines
tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 691
Quoted 4 lines
tests/kms_setmode.c +0
-3} - -out: -cleanup_crtcs (crtcs, crtc_count);Drop this change, looks dangereous.
Regards,
KamilQuoted 6 lines
tests/kms_setmode.c +0
-1} static int assign_crtc_to_connectors(const struct test_config *tconf, --2.41.0
Hi Bhanu,
Quoted 92 lines
-----Original Message-----
From: Modem, Bhanuprakash <[email protected]>
Sent: Friday, June 23, 2023 9:20 AM
To: B, Jeevan <[email protected]>; [email protected]
Cc: [email protected]
Subject: Re: [PATCH i-g-t] tests/kms_setmode: Fix dynamic subtestsHi Jeevan,
Quoted 3 lines · Jeevan B
basic-clone-single-crtc and invalid-clone-exclusive-crtc were not part of dynamic subtest so tests were getting skipped. fixed the test to execute skipping tests as expected.
What does it mean these tests are not part of dynamic subtests?
AFAIK, all basic-* tests are positive test cases & there are corresponding negative test cases (invalid-*)
If any positive test executed, the corresponding negative test is supposed to skip & vice versa.
Example:
https://intel-gfx-ci.01.org/tree/drm-tip/shards-
all.html?testfilter=kms_setmode&hosts=shard-glk- basic-close-single-crtc: SKIP
- invalid-clone-single-crtc: PASS- clone-executive-crtc: PASS
- invalid-clone-executive-crtc: SKIPtests/kms_setmode.c static void test_one_combination(const struct
Quoted 13 lines
v2: add missing call.
Signed-off-by Jeevan B <[email protected]>--- tests/kms_setmode.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)tests/kms_setmode.c index renamed from tests/kms_setmode.c +0
-0
a611d938..96fb0b4b100644tests/kms_setmode.c +0
-0@@ -647,20 +647,19 @@ static void test_one_combination(const struct
test_config*tconf,Quoted 6 lines
tests/kms_setmode.c +2
-0struct crtc_config crtcs[MAX_CRTCS ];int crtc_count;bool config_valid; +int i, pos =0 ; +char test_name[256 ];tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf,
Quoted 10 lines
tests/kms_setmode.c +0
-7setup_crtcs (tconf, cconfs, connector_count, crtcs, &crtc_count, &config_valid); -if (config_valid == !(tconf->flags &TEST_INVALID )) { -int i, pos =0 ; -char test_name[256 ]; - -for (i =0 ; i < crtc_count; i++) { -if (i >0 ) - pos += snprintf(&test_name[pos],
ARRAY_SIZE(test_name)- pos, "-");tests/kms_setmode.c static void test_one_combination(const struct · line 661
Quoted 1 line
tests/kms_setmode.c +0
-1- pos += get_test_name_str(&crtcs[i],&test_name[pos],
ARRAY_SIZE(test_name)- pos);tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf,
Quoted 4 lines
tests/kms_setmode.c +3
-1- } +for (i =0 ; i < crtc_count; i++) { +if (i >0 ) + pos += snprintf(&test_name[pos],
ARRAY_SIZE(test_name)- pos, "-");tests/kms_setmode.c static void test_one_combination(const struct · line 659
Quoted 1 line
tests/kms_setmode.c +1
-0+ pos += get_test_name_str(&crtcs[i], &test_name[pos],
ARRAY_SIZE(test_name)- pos);tests/kms_setmode.c static void test_one_combination(const struct
Quoted 7 lines
tests/kms_setmode.c +2
-0+ } +if (config_valid == !(tconf->flags &TEST_INVALID )) {for (i =0 ; i < crtc_count; i++) {struct crtc_config *crtc = &crtcs[i];@@ -679,16 +678,17 @@ static void test_one_combination(const struct
test_config*tconf,tests/kms_setmode.c static void test_one_combination(const struct · line 678
Quoted 1 line
tests/kms_setmode.c +0
-0((i > 0) && (crtc[i - 1].mode.hdisplay >
MAX_HDISPLAY_PER_CRTC) &&tests/kms_setmode.c static void test_one_combination(const struct · line 680
Quoted 2 lines
tests/kms_setmode.c +0
-0(abs(crtc->crtc_idx - crtcs[i - 1].crtc_idx) <= 1))) { igt_info("Combo: %s is not possible withselected mode(s).\n",
test_name);Quoted 6 lines
tests/kms_setmode.c +2
-1- goto out; +cleanup_crtcs (crtcs, crtc_count); +return ; } }
Quoted 8 lines
tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 690
Quoted 5 lines
tests/kms_setmode.c +3
-0igt_dynamic_f("%s", test_name)test_crtc_config (tconf, crtcs, crtc_count); + }else { + igt_dynamic_f("%s", test_name) +cleanup_crtcs (crtcs, crtc_count);We are supposed to execute the test, not a cleanup. This will always throw the test result as PASS (which is bogus) without even testing anything.
Understood. I will make the change for this.
Quoted 11 lines
- Bhanu
Quoted 8 lines
tests/kms_setmode.c +0
-3} - -out: -cleanup_crtcs (crtcs, crtc_count); } static int assign_crtc_to_connectors(const struct test_config*tconf,
Quoted 16 lines
-----Original Message-----
From: Modem, Bhanuprakash <[email protected]>
Sent: Friday, June 23, 2023 9:20 AM
To: B, Jeevan <[email protected]>; [email protected]
Cc: [email protected]
Subject: Re: [PATCH i-g-t] tests/kms_setmode: Fix dynamic subtestsHi Jeevan,
Quoted 3 lines · Jeevan B
basic-clone-single-crtc and invalid-clone-exclusive-crtc were not part of dynamic subtest so tests were getting skipped. fixed the test to execute skipping tests as expected.
Quoted 7 lines
What does it mean these tests are not part of dynamic subtests?
AFAIK, all basic-* tests are positive test cases & there are corresponding negative test cases (invalid-*)
If any positive test executed, the corresponding negative test is supposed to skip & vice versa.
Hi Bhanu,
Got it. Tests before converting to dynamic.
Starting subtest: basic-clone-single-crtc
Testing: basic-clone-single-crtc 2 connector combinations Subtest basic-clone-single-crtc: SUCCESS (0.242s)
Starting subtest: invalid-clone-single-crtc
Testing: invalid-clone-single-crtc 2 connector combinations
Test id#1 CRTC count 1
CRTC[80] [Pipe A] Mode: 1920x1080@60Hz Connectors: DP-1[236], HDMI-A-2[255]
Test id#2 CRTC count 1
CRTC[131] [Pipe B] Mode: 1920x1080@60Hz Connectors: DP-1[236], HDMI-A-2[255]Subtest invalid-clone-single-crtc: SUCCESS (0.371s)
Starting subtest: invalid-clone-exclusive-crtc
Testing: invalid-clone-exclusive-crtc 2 connector combinations Subtest invalid-clone-exclusive-crtc: SUCCESS (0.330s)
Starting subtest: clone-exclusive-crtc
Testing: clone-exclusive-crtc 2 connector combinations
Test id#1 CRTC count 2
CRTC[80] [Pipe A] Mode: 3840x2160@60Hz Connectors: DP-1[236]
CRTC[131] [Pipe B] Mode: 1920x1200@60Hz Connectors: HDMI-A-2[255]
Test id#2 CRTC count 2
CRTC[131] [Pipe B] Mode: 3840x2160@60Hz Connectors: DP-1[236]
CRTC[80] [Pipe A] Mode: 1920x1200@60Hz Connectors: HDMI-A-2[255]Subtest clone-exclusive-crtc: SUCCESS (1.282s)
So, when tests now started to skip these are getting caught by filters. Should these skips be kept as expected behavior or should we modify tests ??
Thanks
Jeevan B
Quoted 90 lines
Example:
https://intel-gfx-ci.01.org/tree/drm-tip/shards-
all.html?testfilter=kms_setmode&hosts=shard-glk- basic-close-single-crtc: SKIP
- invalid-clone-single-crtc: PASS- clone-executive-crtc: PASS
- invalid-clone-executive-crtc: SKIPtests/kms_setmode.c static void test_one_combination(const struct
Quoted 13 lines
v2: add missing call.
Signed-off-by Jeevan B <[email protected]>--- tests/kms_setmode.c | 26 +++++++++++++------------- 1 file changed, 13 insertions(+), 13 deletions(-)tests/kms_setmode.c index renamed from tests/kms_setmode.c +0
-0
a611d938..96fb0b4b100644tests/kms_setmode.c +0
-0@@ -647,20 +647,19 @@ static void test_one_combination(const struct
test_config*tconf,Quoted 6 lines
tests/kms_setmode.c +2
-0struct crtc_config crtcs[MAX_CRTCS ];int crtc_count;bool config_valid; +int i, pos =0 ; +char test_name[256 ];tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf,
Quoted 10 lines
tests/kms_setmode.c +0
-7setup_crtcs (tconf, cconfs, connector_count, crtcs, &crtc_count, &config_valid); -if (config_valid == !(tconf->flags &TEST_INVALID )) { -int i, pos =0 ; -char test_name[256 ]; - -for (i =0 ; i < crtc_count; i++) { -if (i >0 ) - pos += snprintf(&test_name[pos],
ARRAY_SIZE(test_name)- pos, "-");tests/kms_setmode.c static void test_one_combination(const struct · line 661
Quoted 1 line
tests/kms_setmode.c +0
-1- pos += get_test_name_str(&crtcs[i],&test_name[pos],
ARRAY_SIZE(test_name)- pos);tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf,
Quoted 4 lines
tests/kms_setmode.c +3
-1- } +for (i =0 ; i < crtc_count; i++) { +if (i >0 ) + pos += snprintf(&test_name[pos],
ARRAY_SIZE(test_name)- pos, "-");tests/kms_setmode.c static void test_one_combination(const struct · line 659
Quoted 1 line
tests/kms_setmode.c +1
-0+ pos += get_test_name_str(&crtcs[i], &test_name[pos],
ARRAY_SIZE(test_name)- pos);tests/kms_setmode.c static void test_one_combination(const struct
Quoted 7 lines
tests/kms_setmode.c +2
-0+ } +if (config_valid == !(tconf->flags &TEST_INVALID )) {for (i =0 ; i < crtc_count; i++) {struct crtc_config *crtc = &crtcs[i];@@ -679,16 +678,17 @@ static void test_one_combination(const struct
test_config*tconf,tests/kms_setmode.c static void test_one_combination(const struct · line 678
Quoted 1 line
tests/kms_setmode.c +0
-0((i > 0) && (crtc[i - 1].mode.hdisplay >
MAX_HDISPLAY_PER_CRTC) &&tests/kms_setmode.c static void test_one_combination(const struct · line 680
Quoted 2 lines
tests/kms_setmode.c +0
-0(abs(crtc->crtc_idx - crtcs[i - 1].crtc_idx) <= 1))) { igt_info("Combo: %s is not possible withselected mode(s).\n",
test_name);tests/kms_setmode.c static void test_one_combination(const struct test_config *tconf, · line 690
Quoted 11 lines
tests/kms_setmode.c +5
-1- goto out; +cleanup_crtcs (crtcs, crtc_count); +return ; } } igt_dynamic_f("%s", test_name)test_crtc_config (tconf, crtcs, crtc_count); + }else { + igt_dynamic_f("%s", test_name) +cleanup_crtcs (crtcs, crtc_count);We are supposed to execute the test, not a cleanup. This will always throw the test result as PASS (which is bogus) without even testing anything.
- Bhanu
Quoted 8 lines
tests/kms_setmode.c +0
-3} - -out: -cleanup_crtcs (crtcs, crtc_count); } static int assign_crtc_to_connectors(const struct test_config*tconf,