runner: Also consider TAINT_MACHINE_CHECK as abortable taint
PATCH 1/1 runner: Also consider TAINT_MACHINE_CHECK as abortable taint
--- runner/executor.c | 1 + 1 file changed, 1 insertion(+)
runner/executor.c +1 -0
@@ -142,6 +142,7 @@ static const struct {unsigned long bit;const char *explanation; } abort_taints[] = { + {(1 << 4), "TAINT_MACHINE_CHECK: Processor reported a Machine Check Exception."}, {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."}, {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."}, {(1 << 9), "TAINT_WARN: WARN_ON has happened."},
Signature · 6 lines
2.19.1 _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
Quoted 2 lines · Petri Latvala
Signed-off-by Petri Latvala <[email protected]>Cc Mika Kuoppala <[email protected]>
I've seen lots of machines where these happen as normal side-effect of thermal throtlling. For some value of "normal".
Do we really want to reboot on these? It could be like the network thing I recently disabled, and then everyone started screaming because our machines where constantly rebooting due to network cards/drivers temporarily having a bad time (but usually recovering). -Daniel
Quoted 23 lines
--- runner/executor.c | 1 + 1 file changed, 1 insertion(+)runner/executor.c +1
-0@@ -142,6 +142,7 @@ static const struct {unsigned long bit;const char *explanation; } abort_taints[] = { + {(1 << 4), "TAINT_MACHINE_CHECK: Processor reported a Machine Check Exception."}, {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."}, {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."}, {(1 << 9), "TAINT_WARN: WARN_ON has happened."},Signature · 6 lines
2.19.1 _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
Quoted 7 lines · Daniel Vetter
Quoted 2 lines · Petri Latvala
Signed-off-by Petri Latvala <[email protected]>Cc Mika Kuoppala <[email protected]>I've seen lots of machines where these happen as normal side-effect of thermal throtlling. For some value of "normal".
Thermals could be ok and filtered if we can be somewhat certain that the state didn't suffer. Can we?
Quoted 4 lines
Do we really want to reboot on these? It could be like the network thing I recently disabled, and then everyone started screaming because our machines where constantly rebooting due to network cards/drivers temporarily having a bad time (but usually recovering).
Reboot? This won't get fixed by reboot as the hw is busted.
The question is that is it fruitful to weed out software failures with a broken hardware.
-Mika
Quoted 20 lines
-Daniel
Quoted 19 lines
--- runner/executor.c | 1 + 1 file changed, 1 insertion(+)runner/executor.c +1
-0@@ -142,6 +142,7 @@ static const struct {unsigned long bit;const char *explanation; } abort_taints[] = { + {(1 << 4), "TAINT_MACHINE_CHECK: Processor reported a Machine Check Exception."}, {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."}, {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."}, {(1 << 9), "TAINT_WARN: WARN_ON has happened."},Signature · 1 line
2.19.1
Quoted 13 lines
Quoted 4 lines
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-devSignature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Quoted 18 lines · Mika Kuoppala
Quoted 7 lines · Daniel Vetter
Quoted 2 lines · Petri Latvala
Signed-off-by Petri Latvala <[email protected]>Cc Mika Kuoppala <[email protected]>I've seen lots of machines where these happen as normal side-effect of thermal throtlling. For some value of "normal".
Thermals could be ok and filtered if we can be somewhat certain that the state didn't suffer. Can we?
Quoted 4 lines
Do we really want to reboot on these? It could be like the network thing I recently disabled, and then everyone started screaming because our machines where constantly rebooting due to network cards/drivers temporarily having a bad time (but usually recovering).
Quoted 4 lines
Reboot? This won't get fixed by reboot as the hw is busted.
The question is that is it fruitful to weed out software failures with a broken hardware.
Yeah, I forgot to mention this aspect.
We don't want to just reboot on this: We want to
1) never file a cibuglog filter on TAINT_MACHINE_CHECK. We want that to always be critical.
2) if we get that abort, replace the hw.
Whether that's possible depends on when/if we get that particular taint happening...
Signature · 5 lines
Petri Latvala _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
Quoted 11 lines · Daniel Vetter
Quoted 2 lines · Petri Latvala
Signed-off-by Petri Latvala <[email protected]>Cc Mika Kuoppala <[email protected]>I've seen lots of machines where these happen as normal side-effect of thermal throtlling. For some value of "normal".
Do we really want to reboot on these? It could be like the network thing I recently disabled, and then everyone started screaming because our machines where constantly rebooting due to network cards/drivers temporarily having a bad time (but usually recovering).
I've seen some MCE log messages on dmesgs, quite often on one of the BXTs for example. How often those MCE triggers caused taint is another question.
Reading the mce code, it seems to be thermal _failure_ that causes a taint. And all of these add_taint() calls also use LOCKDEP_NOW_UNRELIABLE so we're already deep under the bus if we get that taint.
Signature · 5 lines
Petri Latvala _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
Quoted 14 lines · Petri Latvala
Quoted 11 lines · Daniel Vetter
Quoted 2 lines · Petri Latvala
Signed-off-by Petri Latvala <[email protected]>Cc Mika Kuoppala <[email protected]>I've seen lots of machines where these happen as normal side-effect of thermal throtlling. For some value of "normal".
Do we really want to reboot on these? It could be like the network thing I recently disabled, and then everyone started screaming because our machines where constantly rebooting due to network cards/drivers temporarily having a bad time (but usually recovering).
Quoted 8 lines
I've seen some MCE log messages on dmesgs, quite often on one of the BXTs for example. How often those MCE triggers caused taint is another question.
Reading the mce code, it seems to be thermal _failure_ that causes a taint. And all of these
add_taint()calls also useLOCKDEP_NOW_UNRELIABLEso we're already deep under the bus if we get that taint.
Hm ok if lockdep is gone then we reboot anyway. I guess ack from me then.
Since the goal is to throw these machines out, shouldn't we have a special error for these? Instead of just aborted I mean.
-Daniel
Signature · 7 lines
Daniel Vetter Software Engineer, Intel Corporation http://blog.ffwll.ch _______________________________________________ igt-dev mailing list [email protected] https://lists.freedesktop.org/mailman/listinfo/igt-dev
Quoted 2 lines · Petri Latvala
Signed-off-by Petri Latvala <[email protected]>Cc Mika Kuoppala <[email protected]>
If we get one, the machine should be removed from the pool.
Quoted 18 lines
--- runner/executor.c | 1 + 1 file changed, 1 insertion(+)runner/executor.c +1
-0@@ -142,6 +142,7 @@ static const struct {unsigned long bit;const char *explanation; } abort_taints[] = { + {(1 << 4), "TAINT_MACHINE_CHECK: Processor reported a Machine Check Exception."}, {(1 << 5), "TAINT_BAD_PAGE: Bad page reference or an unexpected page flags."}, {(1 << 7), "TAINT_DIE: Kernel has died - BUG/OOPS."}, {(1 << 9), "TAINT_WARN: WARN_ON has happened."},Signature · 1 line
2.19.1
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Patchwork
CI[igt-dev] ✓ Fi.CI.BAT: success for runner: Also consider TAINT_MACHINE_CHECK as abortable taint
In reply to Petri Latvala== Series Details ==
== Series Details ==
Series: runner: Also consider TAINT_MACHINE_CHECK as abortable taint
URL : https://patchwork.freedesktop.org/series/61651/
State : success
== Summary ==
CI Bug Log - changes from CI_DRM_6195 -> IGTPW_3116
====================================================
Summary
-------
**SUCCESS**
No regressions found.
External URL: https://patchwork.freedesktop.org/api/1.0/series/61651/revisions/1/mbox/
Known issues
------------
Here are the changes found in IGTPW_3116 that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@i915_module_load@reload:
- fi-icl-u3: [PASS][1] -> [DMESG-WARN][2] ([fdo#106107])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/fi-icl-u3/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/fi-icl-u3/igt@[email protected]
* igt@kms_cursor_legacy@basic-flip-after-cursor-legacy:
- fi-icl-u3: [PASS][3] -> [DMESG-WARN][4] ([fdo#107724]) +2 similar issues
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/fi-icl-u3/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/fi-icl-u3/igt@[email protected]
#### Possible fixes ####
* igt@kms_frontbuffer_tracking@basic:
- fi-hsw-peppy: [DMESG-WARN][5] ([fdo#102614]) -> [PASS][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/fi-hsw-peppy/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/fi-hsw-peppy/igt@[email protected]
* igt@prime_vgem@basic-fence-flip:
- fi-icl-u3: [DMESG-WARN][7] ([fdo#107724]) -> [PASS][8] +1 similar issue
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/fi-icl-u3/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/fi-icl-u3/igt@[email protected]
[fdo#102614]: https://bugs.freedesktop.org/show_bug.cgi?id=102614
[fdo#106107]: https://bugs.freedesktop.org/show_bug.cgi?id=106107
[fdo#107724]: https://bugs.freedesktop.org/show_bug.cgi?id=107724
Participating hosts (52 -> 46)
------------------------------
Additional (1): fi-apl-guc
Missing (7): fi-ilk-m540 fi-hsw-4200u fi-byt-squawks fi-bsw-cyan fi-kbl-7560u fi-byt-clapper fi-bdw-samus
Build changes
-------------
* IGT: IGT_5039 -> IGTPW_3116
CI_DRM_6195: 06b71939f2477c76f9eecb1dd5e99dcb25cb8371 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_3116: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/
IGT_5039: 2d4f470bba1cb51ed116fb80b170f717c6294714 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Patchwork
CI[igt-dev] ✓ Fi.CI.IGT: success for runner: Also consider TAINT_MACHINE_CHECK as abortable taint
In reply to Petri Latvala== Series Details ==
== Series Details ==
Series: runner: Also consider TAINT_MACHINE_CHECK as abortable taint
URL : https://patchwork.freedesktop.org/series/61651/
State : success
== Summary ==
CI Bug Log - changes from CI_DRM_6195_full -> IGTPW_3116_full
====================================================
Summary
-------
**SUCCESS**
No regressions found.
External URL: https://patchwork.freedesktop.org/api/1.0/series/61651/revisions/1/mbox/
Possible new issues
-------------------
Here are the unknown changes that may have been introduced in IGTPW_3116_full:
### IGT changes ###
#### Suppressed ####
The following results come from untrusted machines, tests, or statuses. They do not affect the overall result.
* {igt@gem_ctx_shared@exec-shared-gtt-render}:
- shard-apl: NOTRUN -> [FAIL][1]
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-apl6/igt@[email protected]
* {igt@gem_exec_balancer@smoke}:
- shard-iclb: [PASS][2] -> [SKIP][3]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-iclb1/igt@[email protected]
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-iclb3/igt@[email protected]
Known issues
------------
Here are the changes found in IGTPW_3116_full that come from known issues:
### IGT changes ###
#### Issues hit ####
* igt@gem_eio@reset-stress:
- shard-snb: [PASS][4] -> [FAIL][5] ([fdo#109661])
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-snb1/igt@[email protected]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-snb6/igt@[email protected]
* igt@gem_mmap_gtt@forked-medium-copy-odd:
- shard-iclb: [PASS][6] -> [INCOMPLETE][7] ([fdo#107713])
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-iclb8/igt@[email protected]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-iclb8/igt@[email protected]
* igt@gem_softpin@noreloc-s3:
- shard-apl: [PASS][8] -> [DMESG-WARN][9] ([fdo#108566]) +3 similar issues
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-apl8/igt@[email protected]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-apl5/igt@[email protected]
* igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-blt:
- shard-hsw: [PASS][10] -> [SKIP][11] ([fdo#109271]) +7 similar issues
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-hsw2/igt@[email protected]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-hsw1/igt@[email protected]
* igt@kms_frontbuffer_tracking@fbc-badstride:
- shard-iclb: [PASS][12] -> [FAIL][13] ([fdo#103167]) +4 similar issues
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-iclb8/igt@[email protected]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-iclb7/igt@[email protected]
* igt@kms_setmode@basic:
- shard-kbl: [PASS][14] -> [FAIL][15] ([fdo#99912])
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-kbl3/igt@[email protected]
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-kbl1/igt@[email protected]
* igt@kms_sysfs_edid_timing:
- shard-hsw: [PASS][16] -> [FAIL][17] ([fdo#100047])
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-hsw4/igt@kms_sysfs_edid_timing.html
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-hsw1/igt@kms_sysfs_edid_timing.html
#### Possible fixes ####
* {igt@gem_exec_balancer@bonded-imm}:
- shard-iclb: [FAIL][18] -> [PASS][19]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-iclb4/igt@[email protected]
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-iclb3/igt@[email protected]
* igt@gem_tiled_swapping@non-threaded:
- shard-apl: [DMESG-WARN][20] ([fdo#108686]) -> [PASS][21]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-apl2/igt@[email protected]
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-apl5/igt@[email protected]
* igt@gem_workarounds@suspend-resume-context:
- shard-apl: [DMESG-WARN][22] ([fdo#108566]) -> [PASS][23] +4 similar issues
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-apl6/igt@[email protected]
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-apl8/igt@[email protected]
* igt@kms_flip@flip-vs-suspend:
- shard-kbl: [INCOMPLETE][24] ([fdo#103665]) -> [PASS][25]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-kbl6/igt@[email protected]
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-kbl1/igt@[email protected]
* igt@kms_frontbuffer_tracking@fbc-1p-primscrn-pri-indfb-draw-render:
- shard-iclb: [FAIL][26] ([fdo#103167]) -> [PASS][27] +1 similar issue
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-iclb7/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-pri-indfb-draw-render.html
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-iclb3/igt@kms_frontbuffer_tracking@fbc-1p-primscrn-pri-indfb-draw-render.html
* igt@kms_frontbuffer_tracking@fbc-1p-rte:
- shard-iclb: [FAIL][28] ([fdo#103167] / [fdo#110378]) -> [PASS][29]
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-iclb7/igt@[email protected]
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-iclb5/igt@[email protected]
* igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-pwrite:
- shard-hsw: [SKIP][30] ([fdo#109271]) -> [PASS][31] +16 similar issues
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-hsw1/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-pwrite.html
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-hsw2/igt@kms_frontbuffer_tracking@fbc-2p-scndscrn-cur-indfb-draw-pwrite.html
* igt@kms_psr2_su@page_flip:
- shard-iclb: [SKIP][32] ([fdo#109642]) -> [PASS][33]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-iclb1/igt@kms_psr2_su@page_flip.html
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-iclb2/igt@kms_psr2_su@page_flip.html
* igt@kms_psr@psr2_cursor_plane_move:
- shard-iclb: [SKIP][34] ([fdo#109441]) -> [PASS][35]
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-iclb1/igt@kms_psr@psr2_cursor_plane_move.html
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-iclb2/igt@kms_psr@psr2_cursor_plane_move.html
* igt@kms_setmode@basic:
- shard-apl: [FAIL][36] ([fdo#99912]) -> [PASS][37]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_6195/shard-apl8/igt@[email protected]
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/shard-apl4/igt@[email protected]
{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).
[fdo#100047]: https://bugs.freedesktop.org/show_bug.cgi?id=100047
[fdo#103167]: https://bugs.freedesktop.org/show_bug.cgi?id=103167
[fdo#103665]: https://bugs.freedesktop.org/show_bug.cgi?id=103665
[fdo#107713]: https://bugs.freedesktop.org/show_bug.cgi?id=107713
[fdo#108566]: https://bugs.freedesktop.org/show_bug.cgi?id=108566
[fdo#108686]: https://bugs.freedesktop.org/show_bug.cgi?id=108686
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
[fdo#109642]: https://bugs.freedesktop.org/show_bug.cgi?id=109642
[fdo#109661]: https://bugs.freedesktop.org/show_bug.cgi?id=109661
[fdo#110378]: https://bugs.freedesktop.org/show_bug.cgi?id=110378
[fdo#99912]: https://bugs.freedesktop.org/show_bug.cgi?id=99912
Participating hosts (10 -> 6)
------------------------------
Missing (4): pig-skl-6260u shard-skl pig-hsw-4770r pig-glk-j5005
Build changes
-------------
* IGT: IGT_5039 -> IGTPW_3116
* Piglit: piglit_4509 -> None
CI_DRM_6195: 06b71939f2477c76f9eecb1dd5e99dcb25cb8371 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_3116: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/
IGT_5039: 2d4f470bba1cb51ed116fb80b170f717c6294714 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit
== Logs ==
For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_3116/
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev