Skip to contentSkip to conversation

runner: Cope with repeated subtests in journal

PATCH In review Petri Latvala 3 participants 4 messages May 31, 2019 – Jun 1, 2019

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 runner: Cope with repeated subtests in journal

As seen in fallout with the i915 engine discovery patchset, subtests can in very dire cases be entered multiple times, when the for loop they're in doesn't progress. That is of course a bug in IGT that needs to be fixed, but if (when) that happens again, don't make results take a million years to process.

This patch has been tested to produce good results from the corresponding bad runs.

Cc Tomi Sarvela <[email protected]>
Cc Arkadiusz Hiler <[email protected]>
---
 runner/resultgen.c | 6 ++++++
 1 file changed, 6 insertions(+)
runner/resultgen.c +6 -0
index 2b7d26d5..7b4cd519 100644
@@ -742,6 +742,7 @@ static const char *result_from_exitcode(int exitcode)
 static void add_subtest(struct subtests *subtests, char *subtest)
 {
 	size_t len = strlen(subtest);
+	size_t i;
 
 	if (len == 0)
 		return;
@@ -749,6 +750,11 @@ static void add_subtest(struct subtests *subtests, char *subtest)
 	if (subtest[len - 1] == '\n')
 		subtest[len - 1] = '\0';
 
+	/* Don't add if we already have this subtest */
+	for (i = 0; i < subtests->size; i++)
+		if (!strcmp(subtest, subtests->names[i]))
+			return;
+
 	subtests->size++;
 	subtests->names = realloc(subtests->names, sizeof(*subtests->names) * subtests->size);
 	subtests->names[subtests->size - 1] = subtest;
Signature · 6 lines
2.19.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Reviewed-by on v1 1/1
#1 lore raw

Quoting Petri Latvala (2019-05-31 10:56:19)

Quoted 29 lines

As seen in fallout with the i915 engine discovery patchset, subtests can in very dire cases be entered multiple times, when the for loop they're in doesn't progress. That is of course a bug in IGT that needs to be fixed, but if (when) that happens again, don't make results take a million years to process.

This patch has been tested to produce good results from the corresponding bad runs.

Cc Tomi Sarvela <[email protected]>
Cc Arkadiusz Hiler <[email protected]>
---
 runner/resultgen.c | 6 ++++++
 1 file changed, 6 insertions(+)
runner/resultgen.c +1 -0
index 2b7d26d5..7b4cd519 100644
@@ -742,6 +742,7 @@ static const char *result_from_exitcode(int exitcode)
 static void add_subtest(struct subtests *subtests, char *subtest)
 {
        size_t len = strlen(subtest);
+       size_t i;
 
        if (len == 0)
                return;

runner/resultgen.c static void add_subtest(struct subtests *subtests, char *subtest) · line 756

Quoted 7 lines
runner/resultgen.c +4 -0
@@ -749,6 +750,11 @@ static void add_subtest(struct subtests *subtests, char *subtest)
        if (subtest[len - 1] == '\n')
                subtest[len - 1] = '\0';
 
+       /* Don't add if we already have this subtest */
+       for (i = 0; i < subtests->size; i++)
+               if (!strcmp(subtest, subtests->names[i]))
+                       return;

Looks sane in the short term. I wonder how well it would scale, but until we want to spend weeks running tests, I'm fairly confident we will not reach 1M subtests.

-Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#2 lore raw

[igt-dev] ✓ Fi.CI.BAT: success for runner: Cope with repeated subtests in journal

In reply to Petri Latvala
== Series Details ==

== Series Details ==

Series: runner: Cope with repeated subtests in journal
URL : https://patchwork.freedesktop.org/series/61419/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_6174 -> IGTPW_3087
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://patchwork.freedesktop.org/api/1.0/series/61419/revisions/1/mbox/

Known issues
------------

Here are the changes found in IGTPW_3087 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@i915_selftest@live_contexts:
- fi-hsw-peppy: [PASS][1] -> [DMESG-FAIL][2] ([fdo#110235])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/fi-hsw-peppy/igt@i915_selftest@live_contexts.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/fi-hsw-peppy/igt@i915_selftest@live_contexts.html

* igt@i915_selftest@live_evict:
- fi-bsw-kefka: [PASS][3] -> [DMESG-WARN][4] ([fdo#107709])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/fi-bsw-kefka/igt@i915_selftest@live_evict.html
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/fi-bsw-kefka/igt@i915_selftest@live_evict.html

* igt@kms_frontbuffer_tracking@basic:
- fi-icl-dsi: [PASS][5] -> [FAIL][6] ([fdo#103167])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/fi-icl-dsi/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/fi-icl-dsi/igt@[email protected]

#### Possible fixes ####

* igt@gem_exec_suspend@basic-s4-devices:
- fi-blb-e6850: [INCOMPLETE][7] ([fdo#107718]) -> [PASS][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/fi-blb-e6850/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/fi-blb-e6850/igt@[email protected]

* igt@i915_selftest@live_contexts:
- fi-bdw-gvtdvm: [DMESG-FAIL][9] ([fdo#110235]) -> [PASS][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/fi-bdw-gvtdvm/igt@i915_selftest@live_contexts.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/fi-bdw-gvtdvm/igt@i915_selftest@live_contexts.html

* igt@kms_frontbuffer_tracking@basic:
- fi-icl-u2: [FAIL][11] ([fdo#103167]) -> [PASS][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/fi-icl-u2/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/fi-icl-u2/igt@[email protected]

* igt@prime_vgem@basic-fence-flip:
- fi-ilk-650: [DMESG-WARN][13] ([fdo#106387]) -> [PASS][14] +1 similar issue
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/fi-ilk-650/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/fi-ilk-650/igt@[email protected]

[fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
[fdo#106387]: https://bugs.freedesktop.org/show_bug.cgi?id=106387
[fdo#107709]: https://bugs.freedesktop.org/show_bug.cgi?id=107709
[fdo#107718]: https://bugs.freedesktop.org/show_bug.cgi?id=107718
[fdo#110235]: https://bugs.freedesktop.org/show_bug.cgi?id=110235

Participating hosts (51 -> 47)
------------------------------

Additional (2): fi-gdg-551 fi-cfl-8109u
Missing (6): fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-ctg-p8600 fi-byt-clapper fi-bdw-samus

Build changes
-------------

* IGT: IGT_5026 -> IGTPW_3087

CI_DRM_6174: 4c6b438dfd6d3a4cf8c4cf9ba66a8abba50702c0 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_3087: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/
IGT_5026: 4108c74c3b15460de25ab989f4e2031594559dfc @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#3 lore raw

[igt-dev] ✓ Fi.CI.IGT: success for runner: Cope with repeated subtests in journal

In reply to Petri Latvala
== Series Details ==

== Series Details ==

Series: runner: Cope with repeated subtests in journal
URL : https://patchwork.freedesktop.org/series/61419/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_6174_full -> IGTPW_3087_full
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://patchwork.freedesktop.org/api/1.0/series/61419/revisions/1/mbox/

Known issues
------------

Here are the changes found in IGTPW_3087_full that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_create@create-clear:
- shard-glk: [PASS][1] -> [INCOMPLETE][2] ([fdo#103359] / [k.org#198133])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-glk7/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-glk3/igt@[email protected]

* igt@gem_wait@await-vcs2:
- shard-iclb: [PASS][3] -> [INCOMPLETE][4] ([fdo#107713]) +2 similar issues
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb4/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb7/igt@[email protected]

* igt@kms_cursor_legacy@2x-long-flip-vs-cursor-legacy:
- shard-glk: [PASS][5] -> [FAIL][6] ([fdo#104873])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-glk3/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-glk3/igt@[email protected]

* igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-shrfb-draw-mmap-cpu:
- shard-glk: [PASS][7] -> [FAIL][8] ([fdo#103167])
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-glk2/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-shrfb-draw-mmap-cpu.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-glk1/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-shrfb-draw-mmap-cpu.html

* igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-shrfb-pgflip-blt:
- shard-iclb: [PASS][9] -> [INCOMPLETE][10] ([fdo#106978] / [fdo#107713])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb5/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-shrfb-pgflip-blt.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb7/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-shrfb-pgflip-blt.html

* igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-shrfb-plflip-blt:
- shard-iclb: [PASS][11] -> [FAIL][12] ([fdo#103167]) +4 similar issues
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb2/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-shrfb-plflip-blt.html
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb8/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-shrfb-plflip-blt.html

* igt@kms_psr@psr2_primary_blt:
- shard-iclb: [PASS][13] -> [SKIP][14] ([fdo#109441])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb2/igt@kms_psr@psr2_primary_blt.html
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb8/igt@kms_psr@psr2_primary_blt.html

* igt@kms_vblank@pipe-c-ts-continuation-suspend:
- shard-apl: [PASS][15] -> [DMESG-WARN][16] ([fdo#108566]) +6 similar issues
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-apl4/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-apl8/igt@[email protected]

* igt@perf_pmu@rc6:
- shard-kbl: [PASS][17] -> [SKIP][18] ([fdo#109271])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-kbl7/igt@[email protected]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-kbl1/igt@[email protected]

#### Possible fixes ####

* igt@gem_eio@in-flight-suspend:
- shard-kbl: [FAIL][19] ([fdo#110667]) -> [PASS][20]
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-kbl3/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-kbl3/igt@[email protected]

* igt@gem_mmap_gtt@forked-big-copy:
- shard-iclb: [INCOMPLETE][21] ([fdo#107713] / [fdo#109100]) -> [PASS][22]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb8/igt@[email protected]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb7/igt@[email protected]

* igt@gem_mmap_gtt@forked-medium-copy-xy:
- shard-glk: [INCOMPLETE][23] ([fdo#103359] / [k.org#198133]) -> [PASS][24]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-glk2/igt@[email protected]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-glk9/igt@[email protected]

* igt@gem_workarounds@suspend-resume-context:
- shard-apl: [DMESG-WARN][25] ([fdo#108566]) -> [PASS][26] +5 similar issues
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-apl4/igt@[email protected]
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-apl2/igt@[email protected]

* igt@i915_pm_rpm@gem-execbuf-stress:
- shard-iclb: [INCOMPLETE][27] ([fdo#107713] / [fdo#108840]) -> [PASS][28]
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb5/igt@[email protected]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb4/igt@[email protected]

* igt@kms_busy@extended-modeset-hang-newfb-with-reset-render-a:
- shard-hsw: [INCOMPLETE][29] ([fdo#103540]) -> [PASS][30]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-hsw2/igt@[email protected]
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-hsw2/igt@[email protected]

* igt@kms_cursor_legacy@2x-long-cursor-vs-flip-atomic:
- shard-hsw: [FAIL][31] ([fdo#105767]) -> [PASS][32]
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-hsw6/igt@[email protected]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-hsw4/igt@[email protected]

* igt@kms_cursor_legacy@cursora-vs-flipa-atomic-transitions:
- shard-kbl: [INCOMPLETE][33] ([fdo#103665]) -> [PASS][34]
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-kbl6/igt@[email protected]
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-kbl1/igt@[email protected]

* igt@kms_flip@2x-flip-vs-expired-vblank-interruptible:
- shard-glk: [FAIL][35] ([fdo#105363]) -> [PASS][36]
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-glk4/igt@[email protected]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-glk9/igt@[email protected]

* igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-indfb-draw-render:
- shard-iclb: [FAIL][37] ([fdo#103167]) -> [PASS][38] +2 similar issues
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb5/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-indfb-draw-render.html
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb8/igt@kms_frontbuffer_tracking@fbcpsr-1p-primscrn-pri-indfb-draw-render.html

* igt@kms_plane_lowres@pipe-a-tiling-x:
- shard-iclb: [FAIL][39] ([fdo#103166]) -> [PASS][40]
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb6/igt@[email protected]
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb2/igt@[email protected]

* igt@kms_psr@psr2_sprite_plane_move:
- shard-iclb: [SKIP][41] ([fdo#109441]) -> [PASS][42]
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb8/igt@kms_psr@psr2_sprite_plane_move.html
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb2/igt@kms_psr@psr2_sprite_plane_move.html

* igt@perf_pmu@rc6-runtime-pm-long:
- shard-glk: [FAIL][43] ([fdo#105010]) -> [PASS][44]
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-glk6/igt@[email protected]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-glk9/igt@[email protected]

#### Warnings ####

* igt@gem_mmap_gtt@forked-big-copy-odd:
- shard-iclb: [INCOMPLETE][45] ([fdo#107713] / [fdo#109100]) -> [TIMEOUT][46] ([fdo#109673]) +1 similar issue
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-iclb2/igt@[email protected]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-iclb1/igt@[email protected]

* igt@gem_tiled_swapping@non-threaded:
- shard-hsw: [FAIL][47] ([fdo#108686]) -> [INCOMPLETE][48] ([fdo#103540])
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6174/shard-hsw2/igt@[email protected]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/shard-hsw6/igt@[email protected]

[fdo#103166]: https://bugs.freedesktop.org/show_bug.cgi?id=103166
[fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
[fdo#103359]: https://bugs.freedesktop.org/show_bug.cgi?id=103359
[fdo#103540]: https://bugs.freedesktop.org/show_bug.cgi?id=103540
[fdo#103665]: https://bugs.freedesktop.org/show_bug.cgi?id=103665
[fdo#104873]: https://bugs.freedesktop.org/show_bug.cgi?id=104873
[fdo#105010]: https://bugs.freedesktop.org/show_bug.cgi?id=105010
[fdo#105363]: https://bugs.freedesktop.org/show_bug.cgi?id=105363
[fdo#105767]: https://bugs.freedesktop.org/show_bug.cgi?id=105767
[fdo#106978]: https://bugs.freedesktop.org/show_bug.cgi?id=106978
[fdo#107713]: https://bugs.freedesktop.org/show_bug.cgi?id=107713
[fdo#108566]: https://bugs.freedesktop.org/show_bug.cgi?id=108566
[fdo#108686]: https://bugs.freedesktop.org/show_bug.cgi?id=108686
[fdo#108840]: https://bugs.freedesktop.org/show_bug.cgi?id=108840
[fdo#109100]: https://bugs.freedesktop.org/show_bug.cgi?id=109100
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
[fdo#109673]: https://bugs.freedesktop.org/show_bug.cgi?id=109673
[fdo#110667]: https://bugs.freedesktop.org/show_bug.cgi?id=110667
[k.org#198133]: https://bugzilla.kernel.org/show_bug.cgi?id=198133

Participating hosts (10 -> 6)
------------------------------

Missing (4): pig-skl-6260u shard-skl pig-hsw-4770r pig-glk-j5005

Build changes
-------------

* IGT: IGT_5026 -> IGTPW_3087
* Piglit: piglit_4509 -> None

CI_DRM_6174: 4c6b438dfd6d3a4cf8c4cf9ba66a8abba50702c0 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_3087: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/
IGT_5026: 4108c74c3b15460de25ab989f4e2031594559dfc @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3087/
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev