Skip to contentSkip to conversation

intel-ci: add a pre-merge blacklist to reduce the testing queue

PATCH Superseded Martin Peres 5 participants 21 messages Feb 20, 2020 – Feb 22, 2020 Arm/Amlogic Meson SoC support

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 intel-ci: add a pre-merge blacklist to reduce the testing queue

When arriving at the office on Monday morning, the reported queue size was ~100 hours. This defeats the point of pre-merge testing and vastly exceeds our target of ~6 hours.

We have a lot of work needed to reduce testing time, but this patches reduces the reported run time by 15-30% depending on the platforms:

- shard-skl: 23.9 -> 18.2 minutes (18.5%)
- shard-kbl: 21.2 -> 16.2 minutes (20%)
- shard-apl: 25.9 -> 18.5 minutes (24.3%)
- shard-glk: 24.7 -> 17.6 minutes (24.8%)
- shard-icl: 25.1 -> 16.7 minutes (28.7%)
- shard-tgl: 28.2 -> 19.6 minutes (26.4%)

The reason why the reported runtime is so low compared to the actual time is due to:

- Unaccounted time spent outside of the IGT subtests (exec(), fixtures)
- Unaccounted time spent in suspend (monotonic clock, 20s / suspend)
- Boot time / extra reboots between shards to workaround kernel failures
- Intel GFX CI shard scheduling overhead
- More?

Tomi and Petri are working on reducing these overheads by detecting the bad conditions and rebooting the machine only at this point rather than between every single shard, and increasing the size of the shard test lists to reduce the per-shard CI overhead.

Because of this, the actual savings are way smaller in percentage but still compound over the tens of executions we do per week:

- shard-skl: ~58 -> ~52 minutes
- shard-kbl: ~50 -> ~45 minutes
- shard-apl: ~53 -> ~46 minutes
- shard-glk: ~38 -> ~31 minutes
- shard-icl: ~47 -> ~39 minutes
- shard-tgl: ~60 -> ~51 minutes

More work needed, but we'll get there :)

---
 tests/intel-ci/README                  |   7 +
 tests/intel-ci/blacklist-pre-merge.txt | 221 +++++++++++++++++++++++++
 2 files changed, 228 insertions(+)
 create mode 100644 tests/intel-ci/blacklist-pre-merge.txt
tests/intel-ci/README +7 -0
index e3289933..07b32b54 100644
@@ -37,6 +37,13 @@ blacklist.txt
 This file contains regular expressions (one per line) for tests that
 are not to be executed in full suite test rounds.
 
+=======================
+blacklist-pre-merge.txt
+=======================
+
+This file contains regular expressions (one per line) for tests that
+are not to be executed in pre-merge full suite test rounds.
+
 =============
 meta.testlist
 =============
tests/intel-ci/blacklist-pre-merge.txt new file +221 -0

221 lines not on this page · raw message

Signature · 6 lines
2.25.0

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#1 lore raw

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 4 lines
tests/intel-ci/blacklist-pre-merge.txt new file +4 -0
+###############################################################################
+# These 8 tests are responsible for a significant portion of our execution time
+# despite them testing a feature which is only found in older userspaces:
+#

tests/intel-ci/blacklist-pre-merge.txt line 47

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-skl: 0.1% (~15 seconds)
+# - shard-kbl: 3.5% (~4.5 minutes)
+# - shard-apl: 10% (~18 minutes)
+# - shard-glk: 6.3% (~14 minutes)
+# - shard-icl: 1.7% (~3.5 minutes)
+# - shard-tgl: 1.6% (~3 minutes)
+#
+# Data acquired on 2020-02-19 by Martin Peres
+###############################################################################
+igt@gem_pwrite@big-.*

That is not true. They are testing the obj->mm.get_page cache which is used for all page lookups in the driver.

It's harder to predictably exercise from userspace through other interfaces.

So replace with a unittest for the cache :-p
-Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#2 lore raw
In reply to Martin Peres

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 6 lines
tests/intel-ci/blacklist-pre-merge.txt new file +6 -0
+###############################################################################
+# This test is reading one file at a time while being suspended, which makes
+# testing extremelly slow. This is somewhat of an edge case that is unlikely
+# to be hit, hence why it could be dropped from pre-merge testing. Here are the
+# execution time statistics:
+#

tests/intel-ci/blacklist-pre-merge.txt line 100

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-skl: 0.5% (~1 minute)
+# - shard-kbl: 0.1% (~2 seconds)
+# - shard-apl: 0.1% (~2 seconds)
+# - shard-glk: 0.1% (~2 seconds)
+# - shard-icl: 0.6% (~1.5 minutes)
+# - shard-tgl: 0.7% (~1.5 minutes)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@i915_pm_rpm@debugfs-read

You can say the debugfs-read is a developer-only feature, and it is the sysfs-read that must be kept working. However, the danger here is that we get random errors in other tests that would have been caught by this. -Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#3 lore raw

Quoting Chris Wilson (2020-02-20 15:43:09)

Quoted 13 lines

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 12 lines
tests/intel-ci/blacklist-pre-merge.txt new file +12 -0
+###############################################################################
+# This test is reading one file at a time while being suspended, which makes
+# testing extremelly slow. This is somewhat of an edge case that is unlikely
+# to be hit, hence why it could be dropped from pre-merge testing. Here are the
+# execution time statistics:
+#
+# - shard-skl: 0.5% (~1 minute)
+# - shard-kbl: 0.1% (~2 seconds)
+# - shard-apl: 0.1% (~2 seconds)
+# - shard-glk: 0.1% (~2 seconds)
+# - shard-icl: 0.6% (~1.5 minutes)
+# - shard-tgl: 0.7% (~1.5 minutes)
Quoted 8 lines

tests/intel-ci/blacklist-pre-merge.txt line 100

Quoted 4 lines
tests/intel-ci/blacklist-pre-merge.txt new file +4 -0
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@i915_pm_rpm@debugfs-read

You can say the debugfs-read is a developer-only feature, and it is the sysfs-read that must be kept working. However, the danger here is that we get random errors in other tests that would have been caught by this.

Actually, a 60x difference between systems => bug. -Chris
_______________________________________________
igt-dev mailing list [email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#4 lore raw
In reply to Martin Peres

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 6 lines
tests/intel-ci/blacklist-pre-merge.txt new file +6 -0
+###############################################################################
+# Perf tests are for people using performance counters to get details about
+# how the execution is performed (Observability Architecture). As such, the
+# audience is very limited (game developers, driver developers), and it does
+# not justify the overall execution time:
+#

tests/intel-ci/blacklist-pre-merge.txt line 118

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-skl: 0%
+# - shard-kbl: 0%
+# - shard-apl: 0%
+# - shard-glk: 0%
+# - shard-icl: 0%
+# - shard-tgl: 1.7% (~3.5 minutes)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@perf@gen12-mi-rpc

Nak. That is just a straightforward BUG. The test is doing it's job in reporting the interface is snafu. Then the test screws up by deadlocking due to unclean exit handling after detecting the error. -Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#5 lore raw
Quoted 13 lines · Chris Wilson

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 12 lines
tests/intel-ci/blacklist-pre-merge.txt new file +12 -0
+###############################################################################
+# Perf tests are for people using performance counters to get details about
+# how the execution is performed (Observability Architecture). As such, the
+# audience is very limited (game developers, driver developers), and it does
+# not justify the overall execution time:
+#
+# - shard-skl: 0%
+# - shard-kbl: 0%
+# - shard-apl: 0%
+# - shard-glk: 0%
+# - shard-icl: 0%
+# - shard-tgl: 1.7% (~3.5 minutes)
Quoted 8 lines

tests/intel-ci/blacklist-pre-merge.txt line 118

Quoted 4 lines
tests/intel-ci/blacklist-pre-merge.txt new file +4 -0
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@perf@gen12-mi-rpc

Nak. That is just a straightforward BUG. The test is doing it's job in reporting the interface is snafu. Then the test screws up by deadlocking due to unclean exit handling after detecting the error.

I raised the severity, but this has been opened for 3 weeks without much work being done on it.

That being said, I don't mind dropping this hunk.

Martin

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#6 lore raw
In reply to Martin Peres

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 5 lines
tests/intel-ci/blacklist-pre-merge.txt new file +5 -0
+###############################################################################
+# Modern userspace does not depend on the GTT anymore, so let's drop the
+# slowest tests from pre-merge testing:
+#
+# - shard-skl: 2.7% (~6.5 minutes)

tests/intel-ci/blacklist-pre-merge.txt line 135

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-kbl: 2% (~2.5 minutes)
+# - shard-apl: 4.7% (~8.5 minutes)
+# - shard-glk: 3.5% (~8 minutes)
+# - shard-icl: 4.2% (~8.5 minutes)
+# - shard-tgl: 2.5% (~4.5 minutes)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@gem_fence_thrash@bo-write-verify-threaded-[xy]
+igt@gem_tiled_(wc|(|blits|fence_blits)@(normal|interruptible))

So where's the pre-merge regression testing then?
Just speed the tests up.
-Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#7 lore raw
Quoted 10 lines · Chris Wilson

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 9 lines
tests/intel-ci/blacklist-pre-merge.txt new file +9 -0
+###############################################################################
+# Modern userspace does not depend on the GTT anymore, so let's drop the
+# slowest tests from pre-merge testing:
+#
+# - shard-skl: 2.7% (~6.5 minutes)
+# - shard-kbl: 2% (~2.5 minutes)
+# - shard-apl: 4.7% (~8.5 minutes)
+# - shard-glk: 3.5% (~8 minutes)
+# - shard-icl: 4.2% (~8.5 minutes)
Quoted 8 lines

tests/intel-ci/blacklist-pre-merge.txt line 135

Quoted 6 lines
tests/intel-ci/blacklist-pre-merge.txt new file +6 -0
+# - shard-tgl: 2.5% (~4.5 minutes)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@gem_fence_thrash@bo-write-verify-threaded-[xy]
+igt@gem_tiled_(wc|(|blits|fence_blits)@(normal|interruptible))

So where's the pre-merge regression testing then?

I assume that the problem are the tests' assumptions, so if someone wants to change them, they can just remove the tests from the blacklist.

Quoted 1 line

Just speed the tests up.

I'll be looking forward to reviewing them, in the mean time, it will be post-merge testing only for these. We have plenty of other GTT tests that are kept.

Martin

Quoted 2 lines

-Chris

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#8 lore raw
In reply to Martin Peres

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 5 lines
tests/intel-ci/blacklist-pre-merge.txt new file +5 -0
+###############################################################################
+# This tests modesetting-vs-wedged which is a useful thing to check, but it
+# seems like it mostly catches unrelated issues which are better caught by
+# other tests.
+#

tests/intel-ci/blacklist-pre-merge.txt line 152

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-skl: 0.5% (~1 minute)
+# - shard-kbl: 1% (~1 minute)
+# - shard-apl: 0.6% (~1 minute)
+# - shard-glk: 0.5% (~1 minute)
+# - shard-icl: 0.6% (~1.5 minutes)
+# - shard-tgl: 0.7% (~1.5 minutes)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@gem_eio@kms

Hah. It caught novel issues, which have not been split out into separate regression tests.

Making sure kms is robust is not a bad thing; you can easily speed this up by reducing the workload.
-Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#9 lore raw
Quoted 20 lines · Chris Wilson

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 5 lines
tests/intel-ci/blacklist-pre-merge.txt new file +5 -0
+###############################################################################
+# This tests modesetting-vs-wedged which is a useful thing to check, but it
+# seems like it mostly catches unrelated issues which are better caught by
+# other tests.
+#

tests/intel-ci/blacklist-pre-merge.txt line 152

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-skl: 0.5% (~1 minute)
+# - shard-kbl: 1% (~1 minute)
+# - shard-apl: 0.6% (~1 minute)
+# - shard-glk: 0.5% (~1 minute)
+# - shard-icl: 0.6% (~1.5 minutes)
+# - shard-tgl: 0.7% (~1.5 minutes)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@gem_eio@kms

Hah. It caught novel issues, which have not been split out into separate regression tests.

Quoted 2 lines

Making sure kms is robust is not a bad thing; you can easily speed this up by reducing the workload.

Thanks for your patch! I dropped this hunk :)

Quoted 2 lines

-Chris

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#10 lore raw
In reply to Martin Peres

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 4 lines
tests/intel-ci/blacklist-pre-merge.txt new file +4 -0
+###############################################################################
+# This test virtually never failed, yet is responsible for a relatively big
+# execution time on some platforms:
+#

tests/intel-ci/blacklist-pre-merge.txt line 204

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-skl: 1.3% (~3 minutes)
+# - shard-kbl: 0.3% (~0.3 minutes)
+# - shard-apl: 0.6% (~1 minute)
+# - shard-glk: 0.4% (50 seconds)
+# - shard-icl: 0.1% (15 seconds)
+# - shard-tgl: 0.1% (15 seconds)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@sw_sync@sync_expired_merge

Change from fixed workload to fixed runtime.
-Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#11 lore raw
Quoted 17 lines · Chris Wilson

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 4 lines
tests/intel-ci/blacklist-pre-merge.txt new file +4 -0
+###############################################################################
+# This test virtually never failed, yet is responsible for a relatively big
+# execution time on some platforms:
+#

tests/intel-ci/blacklist-pre-merge.txt line 204

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-skl: 1.3% (~3 minutes)
+# - shard-kbl: 0.3% (~0.3 minutes)
+# - shard-apl: 0.6% (~1 minute)
+# - shard-glk: 0.4% (50 seconds)
+# - shard-icl: 0.1% (15 seconds)
+# - shard-tgl: 0.1% (15 seconds)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@sw_sync@sync_expired_merge

Change from fixed workload to fixed runtime.

Thanks! I reviewed your patch and dropped this hunk :)

Quoted 2 lines

-Chris

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#12 lore raw
In reply to Martin Peres

Quoting Martin Peres (2020-02-20 15:32:09)

Quoted 5 lines
tests/intel-ci/blacklist-pre-merge.txt new file +5 -0
+###############################################################################
+# These 2 tests are stressing the re-usability of objects. It does not look
+# like we have had issues with this outside of the gen7 ppgtt issue, which
+# does not counterbalance its overall execution time.
+#

tests/intel-ci/blacklist-pre-merge.txt line 221

Quoted 10 lines
tests/intel-ci/blacklist-pre-merge.txt new file +10 -0
+# - shard-skl: 2% (~5 minutes)
+# - shard-kbl: 1% (~1.5 minutes)
+# - shard-apl: 1.7% (~3 minutes)
+# - shard-glk: 1% (2.5 minutes)
+# - shard-icl: 0.5% (1 minute)
+# - shard-tgl: 0.5% (1 minute)
+#
+# Data acquired on 2020-02-20 by Martin Peres
+###############################################################################
+igt@gem_exec_reuse@(baggage|contexts)

File a bug; there's no reason why this should be greatly exceeding it's 5s runtime. So presumably we waste too much time in the setup/teardown. -Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#13 lore raw

[igt-dev] ✓ Fi.CI.BAT: success for intel-ci: add a pre-merge blacklist to reduce the testing queue

In reply to Martin Peres
== Series Details ==

== Series Details ==

Series: intel-ci: add a pre-merge blacklist to reduce the testing queue
URL : https://patchwork.freedesktop.org/series/73720/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7973 -> IGTPW_4199
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/index.html

New tests
---------

New tests have been introduced between CI_DRM_7973 and IGTPW_4199:

### New IGT tests (4) ###

* igt@i915_pm_backlight@basic-brightness:
- Statuses : 1 dmesg-warn(s) 17 pass(s) 23 skip(s)
- Exec time: [0.0, 0.25] s

* igt@i915_pm_rpm@basic-pci-d3-state:
- Statuses : 1 dmesg-warn(s) 29 pass(s) 11 skip(s)
- Exec time: [0.0, 6.69] s

* igt@i915_pm_rpm@basic-rte:
- Statuses : 1 dmesg-warn(s) 29 pass(s) 11 skip(s)
- Exec time: [0.44, 27.44] s

* igt@i915_pm_rps@basic-api:
- Statuses : 36 pass(s) 5 skip(s)
- Exec time: [0.0, 0.02] s

Known issues
------------

Here are the changes found in IGTPW_4199 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_close_race@basic-threads:
- fi-byt-n2820: [PASS][1] -> [INCOMPLETE][2] ([i915#45])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/fi-byt-n2820/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/fi-byt-n2820/igt@[email protected]

* igt@gem_mmap_gtt@basic:
- fi-tgl-y: [PASS][3] -> [DMESG-WARN][4] ([CI#94] / [i915#402])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/fi-tgl-y/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/fi-tgl-y/igt@[email protected]

* igt@kms_chamelium@hdmi-edid-read:
- fi-icl-u2: [PASS][5] -> [FAIL][6] ([i915#217])
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/fi-icl-u2/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/fi-icl-u2/igt@[email protected]

#### Possible fixes ####

* igt@gem_mmap@basic:
- fi-tgl-y: [DMESG-WARN][7] ([CI#94] / [i915#402]) -> [PASS][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/fi-tgl-y/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/fi-tgl-y/igt@[email protected]

* igt@i915_selftest@live_gem_contexts:
- fi-cml-s: [DMESG-FAIL][9] ([i915#877]) -> [PASS][10]
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/fi-cml-s/igt@i915_selftest@live_gem_contexts.html
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/fi-cml-s/igt@i915_selftest@live_gem_contexts.html

* igt@kms_chamelium@hdmi-hpd-fast:
- fi-icl-u2: [FAIL][11] ([i915#217]) -> [PASS][12]
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/fi-icl-u2/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/fi-icl-u2/igt@[email protected]
- fi-kbl-7500u: [FAIL][13] ([fdo#111407]) -> [PASS][14]
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/fi-kbl-7500u/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/fi-kbl-7500u/igt@[email protected]

[CI#94]: https://gitlab.freedesktop.org/gfx-ci/i915-infra/issues/94
[fdo#111407]: https://bugs.freedesktop.org/show_bug.cgi?id=111407
[i915#217]: https://gitlab.freedesktop.org/drm/intel/issues/217
[i915#402]: https://gitlab.freedesktop.org/drm/intel/issues/402
[i915#45]: https://gitlab.freedesktop.org/drm/intel/issues/45
[i915#877]: https://gitlab.freedesktop.org/drm/intel/issues/877

Participating hosts (49 -> 45)
------------------------------

Additional (4): fi-kbl-soraka fi-skl-lmem fi-ivb-3770 fi-pnv-d510
Missing (8): fi-ilk-m540 fi-hsw-4200u fi-byt-j1900 fi-skl-6770hq fi-byt-squawks fi-bsw-cyan fi-ctg-p8600 fi-bdw-samus

Build changes
-------------

* CI: CI-20190529 -> None
* IGT: IGT_5453 -> IGTPW_4199

CI-20190529: 20190529
CI_DRM_7973: 07350317e4b2be54b1de7f1e73f77875df5e43f3 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_4199: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/index.html
IGT_5453: cae9a5881ed2c5be2c2518a255740b612a927f9a @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/index.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#14 lore raw

PATCH v2 1/1 intel-ci: add a pre-merge blacklist to reduce the testing queue

In reply to Martin Peres

When arriving at the office on Monday morning, the reported queue size was ~100 hours. This defeats the point of pre-merge testing and vastly exceeds our target of ~6 hours.

We have a lot of work needed to reduce testing time, but this patches reduces the reported run time by 15-30% depending on the platforms:

- shard-skl: 23.9 -> 18.2 minutes (18.5%)
- shard-kbl: 21.2 -> 16.2 minutes (20%)
- shard-apl: 25.9 -> 18.5 minutes (24.3%)
- shard-glk: 24.7 -> 17.6 minutes (24.8%)
- shard-icl: 25.1 -> 16.7 minutes (28.7%)
- shard-tgl: 28.2 -> 19.6 minutes (26.4%)

The reason why the reported runtime is so low compared to the actual time is due to:

- Unaccounted time spent outside of the IGT subtests (exec(), fixtures)
- Unaccounted time spent in suspend (monotonic clock, 20s / suspend)
- Boot time / extra reboots between shards to workaround kernel failures
- Intel GFX CI shard scheduling overhead
- More?

Tomi and Petri are working on reducing these overheads by detecting the bad conditions and rebooting the machine only at this point rather than between every single shard, and increasing the size of the shard test lists to reduce the per-shard CI overhead.

Because of this, the actual savings are way smaller in percentage but still compound over the tens of executions we do per week:

- shard-skl: ~58 -> ~52 minutes
- shard-kbl: ~50 -> ~45 minutes
- shard-apl: ~53 -> ~46 minutes
- shard-glk: ~38 -> ~31 minutes
- shard-icl: ~47 -> ~39 minutes
- shard-tgl: ~60 -> ~51 minutes

More work needed, but we'll get there :)

v2:
- Avoid using | in the regular expressions (Petri Latvala)
- Update the description for igt@gem_pwrite@big-.* (Chris Wilson)
- Drop igt@sw_sync@sync_expired_merge (fixed by Chris Wilson)
- Drop igt@gem_eio@kms (fixed by Chris Wilson)
- Drop igt@perf@gen12-mi-rpc as it is serious kernel bug (Chris Wilson)
- Add links to issues tracking this for all blacklisted item

NOTICE: The above numbers have not been edited for the v2 since
blacklisting or improving the runtime dramatically yields the same results, and only igt@perf@gen12-mi-rpc is back to being slow.

---
 tests/intel-ci/README                  |   7 +
 tests/intel-ci/blacklist-pre-merge.txt | 204 +++++++++++++++++++++++++
 2 files changed, 211 insertions(+)
 create mode 100644 tests/intel-ci/blacklist-pre-merge.txt
tests/intel-ci/README +7 -0
index e3289933..07b32b54 100644
@@ -37,6 +37,13 @@ blacklist.txt
 This file contains regular expressions (one per line) for tests that
 are not to be executed in full suite test rounds.
 
+=======================
+blacklist-pre-merge.txt
+=======================
+
+This file contains regular expressions (one per line) for tests that
+are not to be executed in pre-merge full suite test rounds.
+
 =============
 meta.testlist
 =============
tests/intel-ci/blacklist-pre-merge.txt new file +204 -0

204 lines not on this page · raw message

Signature · 6 lines
2.25.0

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#15 lore raw
Quoted 38 lines · Martin Peres

When arriving at the office on Monday morning, the reported queue size was ~100 hours. This defeats the point of pre-merge testing and vastly exceeds our target of ~6 hours.

We have a lot of work needed to reduce testing time, but this patches reduces the reported run time by 15-30% depending on the platforms:

- shard-skl: 23.9 -> 18.2 minutes (18.5%)
- shard-kbl: 21.2 -> 16.2 minutes (20%)
- shard-apl: 25.9 -> 18.5 minutes (24.3%)
- shard-glk: 24.7 -> 17.6 minutes (24.8%)
- shard-icl: 25.1 -> 16.7 minutes (28.7%)
- shard-tgl: 28.2 -> 19.6 minutes (26.4%)

The reason why the reported runtime is so low compared to the actual time is due to:

- Unaccounted time spent outside of the IGT subtests (exec(), fixtures)
- Unaccounted time spent in suspend (monotonic clock, 20s / suspend)
- Boot time / extra reboots between shards to workaround kernel failures
- Intel GFX CI shard scheduling overhead
- More?

Tomi and Petri are working on reducing these overheads by detecting the bad conditions and rebooting the machine only at this point rather than between every single shard, and increasing the size of the shard test lists to reduce the per-shard CI overhead.

Because of this, the actual savings are way smaller in percentage but still compound over the tens of executions we do per week:

- shard-skl: ~58 -> ~52 minutes
- shard-kbl: ~50 -> ~45 minutes
- shard-apl: ~53 -> ~46 minutes
- shard-glk: ~38 -> ~31 minutes
- shard-icl: ~47 -> ~39 minutes
- shard-tgl: ~60 -> ~51 minutes

Quoted 4 lines

More work needed, but we'll get there :)

v2:
- Avoid using | in the regular expressions (Petri Latvala)

For the record: I objected to using (|stuff) in regular expressions.

Signature · 5 lines
Petri Latvala
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Acked-by on v2 1/1
#16 lore raw
In reply to Martin Peres

Quoting Martin Peres (2020-02-21 09:00:47)

Quoted 48 lines

When arriving at the office on Monday morning, the reported queue size was ~100 hours. This defeats the point of pre-merge testing and vastly exceeds our target of ~6 hours.

We have a lot of work needed to reduce testing time, but this patches reduces the reported run time by 15-30% depending on the platforms:

- shard-skl: 23.9 -> 18.2 minutes (18.5%)
- shard-kbl: 21.2 -> 16.2 minutes (20%)
- shard-apl: 25.9 -> 18.5 minutes (24.3%)
- shard-glk: 24.7 -> 17.6 minutes (24.8%)
- shard-icl: 25.1 -> 16.7 minutes (28.7%)
- shard-tgl: 28.2 -> 19.6 minutes (26.4%)

The reason why the reported runtime is so low compared to the actual time is due to:

- Unaccounted time spent outside of the IGT subtests (exec(), fixtures)
- Unaccounted time spent in suspend (monotonic clock, 20s / suspend)
- Boot time / extra reboots between shards to workaround kernel failures
- Intel GFX CI shard scheduling overhead
- More?

Tomi and Petri are working on reducing these overheads by detecting the bad conditions and rebooting the machine only at this point rather than between every single shard, and increasing the size of the shard test lists to reduce the per-shard CI overhead.

Because of this, the actual savings are way smaller in percentage but still compound over the tens of executions we do per week:

- shard-skl: ~58 -> ~52 minutes
- shard-kbl: ~50 -> ~45 minutes
- shard-apl: ~53 -> ~46 minutes
- shard-glk: ~38 -> ~31 minutes
- shard-icl: ~47 -> ~39 minutes
- shard-tgl: ~60 -> ~51 minutes

More work needed, but we'll get there :)

v2:
- Avoid using | in the regular expressions (Petri Latvala)
- Update the description for igt@gem_pwrite@big-.* (Chris Wilson)
- Drop igt@sw_sync@sync_expired_merge (fixed by Chris Wilson)
- Drop igt@gem_eio@kms (fixed by Chris Wilson)
- Drop igt@perf@gen12-mi-rpc as it is serious kernel bug (Chris Wilson)
- Add links to issues tracking this for all blacklisted item

Quoted 6 lines

NOTICE: The above numbers have not been edited for the v2 since
blacklisting or improving the runtime dramatically yields the same results, and only igt@perf@gen12-mi-rpc is back to being slow.

I dream of a day where the test lists are autogenerated based on historical information on how effective each one is at rejecting patches, tuned for a particular test runtime. And with feedback from bugs reported after the fact (along with the new testcases we need to capture new code and user reported bugs). [Oh and fuzzing to generate new tests.]

Imagine if we can do 95% patch^W bug rejection within 10min and 99.9% rejection within 1hour. Then we might have enough free time for the extended tests on CI_DRM.
-Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#17 lore raw

[igt-dev] ✓ Fi.CI.BAT: success for intel-ci: add a pre-merge blacklist to reduce the testing queue (rev2)

In reply to Martin Peres
== Series Details ==

== Series Details ==

Series: intel-ci: add a pre-merge blacklist to reduce the testing queue (rev2)
URL : https://patchwork.freedesktop.org/series/73720/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7979 -> IGTPW_4205
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4205/index.html

Known issues
------------

Here are the changes found in IGTPW_4205 that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@i915_selftest@live_hangcheck:
- fi-icl-guc: [PASS][1] -> [INCOMPLETE][2] ([fdo#108569])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7979/fi-icl-guc/igt@i915_selftest@live_hangcheck.html
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4205/fi-icl-guc/igt@i915_selftest@live_hangcheck.html

* igt@kms_chamelium@hdmi-hpd-fast:
- fi-kbl-7500u: [PASS][3] -> [FAIL][4] ([fdo#111407])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7979/fi-kbl-7500u/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4205/fi-kbl-7500u/igt@[email protected]

#### Possible fixes ####

* igt@gem_exec_suspend@basic-s4-devices:
- fi-tgl-y: [INCOMPLETE][5] ([CI#94] / [i915#460]) -> [PASS][6]
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7979/fi-tgl-y/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4205/fi-tgl-y/igt@[email protected]

* igt@i915_selftest@live_gt_lrc:
- {fi-tgl-u}: [DMESG-FAIL][7] ([i915#1233]) -> [PASS][8]
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7979/fi-tgl-u/igt@i915_selftest@live_gt_lrc.html
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4205/fi-tgl-u/igt@i915_selftest@live_gt_lrc.html

#### Warnings ####

* igt@gem_exec_parallel@contexts:
- fi-byt-n2820: [TIMEOUT][9] ([fdo#112271] / [i915#1084]) -> [FAIL][10] ([i915#694])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7979/fi-byt-n2820/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4205/fi-byt-n2820/igt@[email protected]

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[CI#94]: https://gitlab.freedesktop.org/gfx-ci/i915-infra/issues/94
[fdo#108569]: https://bugs.freedesktop.org/show_bug.cgi?id=108569
[fdo#111407]: https://bugs.freedesktop.org/show_bug.cgi?id=111407
[fdo#112271]: https://bugs.freedesktop.org/show_bug.cgi?id=112271
[i915#1084]: https://gitlab.freedesktop.org/drm/intel/issues/1084
[i915#1233]: https://gitlab.freedesktop.org/drm/intel/issues/1233
[i915#460]: https://gitlab.freedesktop.org/drm/intel/issues/460
[i915#694]: https://gitlab.freedesktop.org/drm/intel/issues/694

Participating hosts (45 -> 40)
------------------------------

Additional (4): fi-ivb-3770 fi-glk-dsi fi-bwr-2160 fi-bsw-n3050
Missing (9): fi-ilk-m540 fi-hsw-4200u fi-hsw-peppy fi-ilk-650 fi-ctg-p8600 fi-gdg-551 fi-skl-lmem fi-byt-clapper fi-bdw-samus

Build changes
-------------

* CI: CI-20190529 -> None
* IGT: IGT_5453 -> IGTPW_4205

CI-20190529: 20190529
CI_DRM_7979: 45d61ea8faa5bdb50719bed2de3dd2ef8e6f5a12 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_4205: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4205/index.html
IGT_5453: cae9a5881ed2c5be2c2518a255740b612a927f9a @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4205/index.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#18 lore raw
In reply to Martin Peres
Quoted 38 lines · Martin Peres

When arriving at the office on Monday morning, the reported queue size was ~100 hours. This defeats the point of pre-merge testing and vastly exceeds our target of ~6 hours.

We have a lot of work needed to reduce testing time, but this patches reduces the reported run time by 15-30% depending on the platforms:

- shard-skl: 23.9 -> 18.2 minutes (18.5%)
- shard-kbl: 21.2 -> 16.2 minutes (20%)
- shard-apl: 25.9 -> 18.5 minutes (24.3%)
- shard-glk: 24.7 -> 17.6 minutes (24.8%)
- shard-icl: 25.1 -> 16.7 minutes (28.7%)
- shard-tgl: 28.2 -> 19.6 minutes (26.4%)

The reason why the reported runtime is so low compared to the actual time is due to:

- Unaccounted time spent outside of the IGT subtests (exec(), fixtures)
- Unaccounted time spent in suspend (monotonic clock, 20s / suspend)
- Boot time / extra reboots between shards to workaround kernel failures
- Intel GFX CI shard scheduling overhead
- More?

Tomi and Petri are working on reducing these overheads by detecting the bad conditions and rebooting the machine only at this point rather than between every single shard, and increasing the size of the shard test lists to reduce the per-shard CI overhead.

Because of this, the actual savings are way smaller in percentage but still compound over the tens of executions we do per week:

- shard-skl: ~58 -> ~52 minutes
- shard-kbl: ~50 -> ~45 minutes
- shard-apl: ~53 -> ~46 minutes
- shard-glk: ~38 -> ~31 minutes
- shard-icl: ~47 -> ~39 minutes
- shard-tgl: ~60 -> ~51 minutes

Quoted 6 lines

More work needed, but we'll get there :)

---
 tests/intel-ci/README                  |   7 +
 tests/intel-ci/blacklist-pre-merge.txt | 221 +++++++++++++++++++++++++

With a drive-by addition of a forgotten blacklist.txt, squash this in:

tests/intel-ci/meson.build +2 -0
index b3abb848..62811e92 100644
@@ -3,6 +3,8 @@ intelci_files = [
   'fast-feedback.testlist',
   'meta.testlist',
   'README',
+  'blacklist.txt',
+  'blacklist-pre-merge.txt',
 ]
 
 install_data(sources : intelci_files, install_dir : datadir)

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#19 lore raw
Quoted 54 lines · Petri Latvala
Quoted 38 lines · Martin Peres

When arriving at the office on Monday morning, the reported queue size was ~100 hours. This defeats the point of pre-merge testing and vastly exceeds our target of ~6 hours.

We have a lot of work needed to reduce testing time, but this patches reduces the reported run time by 15-30% depending on the platforms:

- shard-skl: 23.9 -> 18.2 minutes (18.5%)
- shard-kbl: 21.2 -> 16.2 minutes (20%)
- shard-apl: 25.9 -> 18.5 minutes (24.3%)
- shard-glk: 24.7 -> 17.6 minutes (24.8%)
- shard-icl: 25.1 -> 16.7 minutes (28.7%)
- shard-tgl: 28.2 -> 19.6 minutes (26.4%)

The reason why the reported runtime is so low compared to the actual time is due to:

- Unaccounted time spent outside of the IGT subtests (exec(), fixtures)
- Unaccounted time spent in suspend (monotonic clock, 20s / suspend)
- Boot time / extra reboots between shards to workaround kernel failures
- Intel GFX CI shard scheduling overhead
- More?

Tomi and Petri are working on reducing these overheads by detecting the bad conditions and rebooting the machine only at this point rather than between every single shard, and increasing the size of the shard test lists to reduce the per-shard CI overhead.

Because of this, the actual savings are way smaller in percentage but still compound over the tens of executions we do per week:

- shard-skl: ~58 -> ~52 minutes
- shard-kbl: ~50 -> ~45 minutes
- shard-apl: ~53 -> ~46 minutes
- shard-glk: ~38 -> ~31 minutes
- shard-icl: ~47 -> ~39 minutes
- shard-tgl: ~60 -> ~51 minutes

Quoted 6 lines

More work needed, but we'll get there :)

---
 tests/intel-ci/README                  |   7 +
 tests/intel-ci/blacklist-pre-merge.txt | 221 +++++++++++++++++++++++++

With a drive-by addition of a forgotten blacklist.txt, squash this in:

tests/intel-ci/meson.build +0 -0
index b3abb848..62811e92 100644

tests/intel-ci/meson.build intelci_files = [ · line 10

Quoted 8 lines
tests/intel-ci/meson.build +2 -0
@@ -3,6 +3,8 @@ intelci_files = [
   'fast-feedback.testlist',
   'meta.testlist',
   'README',
+  'blacklist.txt',
+  'blacklist-pre-merge.txt',
 ]
 
 install_data(sources : intelci_files, install_dir : datadir)

With Martin's IRL ack to squashing this, the patch is now merged.

Signature · 5 lines
Petri Latvala
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#20 lore raw

[igt-dev] ✓ Fi.CI.IGT: success for intel-ci: add a pre-merge blacklist to reduce the testing queue

In reply to Martin Peres
== Series Details ==

== Series Details ==

Series: intel-ci: add a pre-merge blacklist to reduce the testing queue
URL : https://patchwork.freedesktop.org/series/73720/
State : success

== Summary ==

CI Bug Log - changes from CI_DRM_7973_full -> IGTPW_4199_full
====================================================

Summary
-------

**SUCCESS**

No regressions found.

External URL: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/index.html

Known issues
------------

Here are the changes found in IGTPW_4199_full that come from known issues:

### IGT changes ###

#### Issues hit ####

* igt@gem_ctx_shared@exec-shared-gtt-bsd2:
- shard-tglb: [PASS][1] -> [FAIL][2] ([i915#616])
[1]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-tglb8/igt@[email protected]
[2]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-tglb8/igt@[email protected]

* igt@gem_exec_reuse@baggage:
- shard-apl: [PASS][3] -> [TIMEOUT][4] ([fdo#112271])
[3]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-apl6/igt@[email protected]
[4]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-apl4/igt@[email protected]

* igt@gem_exec_schedule@preempt-queue-bsd1:
- shard-iclb: [PASS][5] -> [SKIP][6] ([fdo#109276]) +20 similar issues
[5]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb4/igt@[email protected]
[6]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb5/igt@[email protected]

* igt@gem_exec_schedule@reorder-wide-bsd:
- shard-iclb: [PASS][7] -> [SKIP][8] ([fdo#112146]) +3 similar issues
[7]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb8/igt@[email protected]
[8]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb2/igt@[email protected]

* igt@gem_fence_thrash@bo-write-verify-y:
- shard-snb: [PASS][9] -> [DMESG-WARN][10] ([i915#478])
[9]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-snb2/igt@[email protected]
[10]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-snb6/igt@[email protected]

* igt@gem_partial_pwrite_pread@reads-display:
- shard-hsw: [PASS][11] -> [FAIL][12] ([i915#694])
[11]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-hsw6/igt@[email protected]
[12]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-hsw1/igt@[email protected]

* igt@gem_ppgtt@flink-and-close-vma-leak:
- shard-tglb: [PASS][13] -> [FAIL][14] ([i915#644])
[13]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-tglb1/igt@[email protected]
[14]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-tglb7/igt@[email protected]

* igt@gem_pwrite@big-cpu-fbr:
- shard-hsw: [PASS][15] -> [TIMEOUT][16] ([fdo#112271])
[15]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-hsw7/igt@[email protected]
[16]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-hsw5/igt@[email protected]

* igt@gem_tiled_partial_pwrite_pread@writes:
- shard-hsw: [PASS][17] -> [FAIL][18] ([i915#817])
[17]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-hsw1/igt@[email protected]
[18]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-hsw1/igt@[email protected]

* igt@kms_cursor_crc@pipe-a-cursor-suspend:
- shard-kbl: [PASS][19] -> [DMESG-WARN][20] ([i915#180]) +7 similar issues
[19]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-kbl2/igt@[email protected]
[20]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-kbl1/igt@[email protected]

* igt@kms_flip@flip-vs-modeset-vs-hang-interruptible:
- shard-glk: [PASS][21] -> [TIMEOUT][22] ([fdo#112271]) +2 similar issues
[21]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-glk6/igt@[email protected]
[22]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-glk7/igt@[email protected]

* igt@kms_flip@flip-vs-panning-vs-hang:
- shard-tglb: [PASS][23] -> [TIMEOUT][24] ([fdo#112271] / [i915#561]) +2 similar issues
[23]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-tglb8/igt@[email protected]
[24]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-tglb8/igt@[email protected]

* igt@kms_psr@no_drrs:
- shard-iclb: [PASS][25] -> [FAIL][26] ([i915#173])
[25]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb5/igt@kms_psr@no_drrs.html
[26]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb1/igt@kms_psr@no_drrs.html

* igt@kms_psr@psr2_primary_mmap_cpu:
- shard-iclb: [PASS][27] -> [SKIP][28] ([fdo#109441]) +3 similar issues
[27]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb2/igt@kms_psr@psr2_primary_mmap_cpu.html
[28]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb6/igt@kms_psr@psr2_primary_mmap_cpu.html

* igt@kms_vblank@pipe-a-ts-continuation-suspend:
- shard-apl: [PASS][29] -> [DMESG-WARN][30] ([i915#180]) +2 similar issues
[29]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-apl7/igt@[email protected]
[30]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-apl2/igt@[email protected]

* igt@perf@gen12-mi-rpc:
- shard-tglb: [PASS][31] -> [TIMEOUT][32] ([fdo#112271] / [i915#1085])
[31]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-tglb8/igt@[email protected]
[32]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-tglb6/igt@[email protected]

* igt@perf_pmu@busy-accuracy-2-vcs1:
- shard-iclb: [PASS][33] -> [SKIP][34] ([fdo#112080]) +12 similar issues
[33]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb4/igt@[email protected]
[34]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb6/igt@[email protected]

* igt@prime_mmap_coherency@ioctl-errors:
- shard-hsw: [PASS][35] -> [FAIL][36] ([i915#831])
[35]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-hsw1/igt@[email protected]
[36]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-hsw1/igt@[email protected]

#### Possible fixes ####

* igt@gem_busy@busy-vcs1:
- shard-iclb: [SKIP][37] ([fdo#112080]) -> [PASS][38] +12 similar issues
[37]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb6/igt@[email protected]
[38]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb1/igt@[email protected]

* {igt@gem_ctx_persistence@close-replace-race}:
- shard-kbl: [INCOMPLETE][39] ([fdo#103665]) -> [PASS][40] +1 similar issue
[39]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-kbl3/igt@[email protected]
[40]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-kbl6/igt@[email protected]

* igt@gem_exec_balancer@hang:
- shard-iclb: [FAIL][41] ([i915#1277]) -> [PASS][42]
[41]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb4/igt@[email protected]
[42]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb1/igt@[email protected]
- shard-tglb: [FAIL][43] ([i915#1277]) -> [PASS][44]
[43]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-tglb8/igt@[email protected]
[44]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-tglb8/igt@[email protected]

* {igt@gem_exec_schedule@implicit-both-bsd2}:
- shard-iclb: [SKIP][45] ([fdo#109276] / [i915#677]) -> [PASS][46] +1 similar issue
[45]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb5/igt@[email protected]
[46]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb4/igt@[email protected]

* igt@gem_exec_schedule@pi-distinct-iova-bsd:
- shard-iclb: [SKIP][47] ([i915#677]) -> [PASS][48] +2 similar issues
[47]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb4/igt@[email protected]
[48]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb6/igt@[email protected]

* igt@gem_exec_schedule@preemptive-hang-bsd:
- shard-iclb: [SKIP][49] ([fdo#112146]) -> [PASS][50] +4 similar issues
[49]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb1/igt@[email protected]
[50]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb7/igt@[email protected]

* igt@gem_ppgtt@flink-and-close-vma-leak:
- shard-apl: [FAIL][51] ([i915#644]) -> [PASS][52]
[51]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-apl7/igt@[email protected]
[52]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-apl2/igt@[email protected]
- shard-kbl: [FAIL][53] ([i915#644]) -> [PASS][54]
[53]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-kbl6/igt@[email protected]
[54]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-kbl1/igt@[email protected]

* igt@gem_render_copy_redux@normal:
- shard-hsw: [FAIL][55] ([i915#694]) -> [PASS][56] +2 similar issues
[55]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-hsw5/igt@[email protected]
[56]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-hsw5/igt@[email protected]

* igt@i915_pm_rpm@system-suspend-execbuf:
- shard-glk: [FAIL][57] ([fdo#103375]) -> [PASS][58]
[57]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-glk6/igt@[email protected]
[58]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-glk4/igt@[email protected]

* igt@kms_color@pipe-a-ctm-blue-to-red:
- shard-kbl: [FAIL][59] ([i915#129]) -> [PASS][60]
[59]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-kbl6/igt@[email protected]
[60]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-kbl6/igt@[email protected]
- shard-apl: [FAIL][61] ([i915#129]) -> [PASS][62]
[61]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-apl2/igt@[email protected]
[62]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-apl1/igt@[email protected]

* igt@kms_cursor_crc@pipe-a-cursor-alpha-opaque:
- shard-apl: [FAIL][63] ([i915#54]) -> [PASS][64] +1 similar issue
[63]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-apl2/igt@[email protected]
[64]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-apl7/igt@[email protected]
- shard-glk: [FAIL][65] ([i915#54]) -> [PASS][66]
[65]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-glk4/igt@[email protected]
[66]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-glk3/igt@[email protected]

* igt@kms_cursor_crc@pipe-c-cursor-256x85-sliding:
- shard-kbl: [FAIL][67] ([i915#54]) -> [PASS][68] +1 similar issue
[67]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-kbl2/igt@[email protected]
[68]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-kbl2/igt@[email protected]

* igt@kms_flip@2x-flip-vs-suspend:
- shard-hsw: [INCOMPLETE][69] ([i915#61]) -> [PASS][70]
[69]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-hsw5/igt@[email protected]
[70]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-hsw7/igt@[email protected]

* igt@kms_flip@flip-vs-panning:
- shard-kbl: [DMESG-WARN][71] -> [PASS][72]
[71]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-kbl4/igt@[email protected]
[72]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-kbl7/igt@[email protected]
- shard-apl: [DMESG-WARN][73] -> [PASS][74]
[73]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-apl3/igt@[email protected]
[74]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-apl7/igt@[email protected]

* igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-indfb-draw-mmap-wc:
- shard-glk: [FAIL][75] ([i915#49]) -> [PASS][76]
[75]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-glk4/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-indfb-draw-mmap-wc.html
[76]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-glk5/igt@kms_frontbuffer_tracking@fbc-2p-primscrn-pri-indfb-draw-mmap-wc.html

* igt@kms_frontbuffer_tracking@fbcpsr-slowdraw:
- shard-tglb: [SKIP][77] ([i915#668]) -> [PASS][78] +1 similar issue
[77]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-tglb1/igt@[email protected]
[78]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-tglb5/igt@[email protected]

* igt@kms_plane_lowres@pipe-a-tiling-y:
- shard-glk: [FAIL][79] ([i915#899]) -> [PASS][80]
[79]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-glk2/igt@[email protected]
[80]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-glk1/igt@[email protected]

* igt@kms_psr@psr2_primary_page_flip:
- shard-iclb: [SKIP][81] ([fdo#109441]) -> [PASS][82] +1 similar issue
[81]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb1/igt@kms_psr@psr2_primary_page_flip.html
[82]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb2/igt@kms_psr@psr2_primary_page_flip.html

* igt@kms_rotation_crc@primary-rotation-90:
- shard-glk: [TIMEOUT][83] ([fdo#112271]) -> [PASS][84]
[83]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-glk6/igt@[email protected]
[84]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-glk3/igt@[email protected]

* igt@kms_setmode@basic:
- shard-apl: [FAIL][85] ([i915#31]) -> [PASS][86]
[85]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-apl6/igt@[email protected]
[86]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-apl4/igt@[email protected]

* igt@kms_vblank@pipe-b-ts-continuation-suspend:
- shard-apl: [DMESG-WARN][87] ([i915#180]) -> [PASS][88] +4 similar issues
[87]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-apl6/igt@[email protected]
[88]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-apl2/igt@[email protected]

* igt@perf_pmu@busy-idle-check-all-vcs0:
- shard-glk: [FAIL][89] -> [PASS][90] +7 similar issues
[89]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-glk6/igt@[email protected]
[90]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-glk7/igt@[email protected]

* igt@perf_pmu@cpu-hotplug:
- shard-glk: [TIMEOUT][91] ([fdo#111561] / [fdo#112271]) -> [PASS][92]
[91]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-glk6/igt@[email protected]
[92]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-glk4/igt@[email protected]

* igt@prime_vgem@fence-wait-bsd2:
- shard-iclb: [SKIP][93] ([fdo#109276]) -> [PASS][94] +17 similar issues
[93]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb8/igt@[email protected]
[94]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb4/igt@[email protected]

#### Warnings ####

* igt@gem_ctx_isolation@vcs1-nonpriv-switch:
- shard-iclb: [SKIP][95] ([fdo#112080]) -> [FAIL][96] ([IGT#28])
[95]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-iclb8/igt@[email protected]
[96]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-iclb4/igt@[email protected]

* igt@i915_pm_rpm@pm-caching:
- shard-snb: [INCOMPLETE][97] ([i915#82]) -> [SKIP][98] ([fdo#109271])
[97]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-snb5/igt@[email protected]
[98]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-snb5/igt@[email protected]

* igt@i915_selftest@live_gt_lrc:
- shard-tglb: [DMESG-FAIL][99] ([i915#1233]) -> [INCOMPLETE][100] ([i915#1233])
[99]: https://intel-gfx-ci.01.org/tree/drm-tip/CI_DRM_7973/shard-tglb2/igt@i915_selftest@live_gt_lrc.html
[100]: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/shard-tglb8/igt@i915_selftest@live_gt_lrc.html

{name}: This element is suppressed. This means it is ignored when computing
the status of the difference (SUCCESS, WARNING, or FAILURE).

[IGT#28]: https://gitlab.freedesktop.org/drm/igt-gpu-tools/issues/28
[fdo#103375]: https://bugs.freedesktop.org/show_bug.cgi?id=103375
[fdo#103665]: https://bugs.freedesktop.org/show_bug.cgi?id=103665
[fdo#109271]: https://bugs.freedesktop.org/show_bug.cgi?id=109271
[fdo#109276]: https://bugs.freedesktop.org/show_bug.cgi?id=109276
[fdo#109441]: https://bugs.freedesktop.org/show_bug.cgi?id=109441
[fdo#111561]: https://bugs.freedesktop.org/show_bug.cgi?id=111561
[fdo#112080]: https://bugs.freedesktop.org/show_bug.cgi?id=112080
[fdo#112146]: https://bugs.freedesktop.org/show_bug.cgi?id=112146
[fdo#112271]: https://bugs.freedesktop.org/show_bug.cgi?id=112271
[i915#1085]: https://gitlab.freedesktop.org/drm/intel/issues/1085
[i915#1233]: https://gitlab.freedesktop.org/drm/intel/issues/1233
[i915#1277]: https://gitlab.freedesktop.org/drm/intel/issues/1277
[i915#129]: https://gitlab.freedesktop.org/drm/intel/issues/129
[i915#1291]: https://gitlab.freedesktop.org/drm/intel/issues/1291
[i915#173]: https://gitlab.freedesktop.org/drm/intel/issues/173
[i915#180]: https://gitlab.freedesktop.org/drm/intel/issues/180
[i915#31]: https://gitlab.freedesktop.org/drm/intel/issues/31
[i915#478]: https://gitlab.freedesktop.org/drm/intel/issues/478
[i915#49]: https://gitlab.freedesktop.org/drm/intel/issues/49
[i915#54]: https://gitlab.freedesktop.org/drm/intel/issues/54
[i915#561]: https://gitlab.freedesktop.org/drm/intel/issues/561
[i915#61]: https://gitlab.freedesktop.org/drm/intel/issues/61
[i915#616]: https://gitlab.freedesktop.org/drm/intel/issues/616
[i915#644]: https://gitlab.freedesktop.org/drm/intel/issues/644
[i915#668]: https://gitlab.freedesktop.org/drm/intel/issues/668
[i915#677]: https://gitlab.freedesktop.org/drm/intel/issues/677
[i915#694]: https://gitlab.freedesktop.org/drm/intel/issues/694
[i915#817]: https://gitlab.freedesktop.org/drm/intel/issues/817
[i915#82]: https://gitlab.freedesktop.org/drm/intel/issues/82
[i915#831]: https://gitlab.freedesktop.org/drm/intel/issues/831
[i915#899]: https://gitlab.freedesktop.org/drm/intel/issues/899

Participating hosts (10 -> 8)
------------------------------

Missing (2): pig-skl-6260u pig-glk-j5005

Build changes
-------------

* CI: CI-20190529 -> None
* IGT: IGT_5453 -> IGTPW_4199
* Piglit: piglit_4509 -> None

CI-20190529: 20190529
CI_DRM_7973: 07350317e4b2be54b1de7f1e73f77875df5e43f3 @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_4199: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/index.html
IGT_5453: cae9a5881ed2c5be2c2518a255740b612a927f9a @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools
piglit_4509: fdc5a4ca11124ab8413c7988896eec4c97336694 @ git://anongit.freedesktop.org/piglit

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_4199/index.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev