Skip to contentSkip to conversation

extended.testlist: Remove default and render engine test duplicates

RFC In review Tvrtko Ursulin 4 participants 9 messages Jun 16, 2017 – Jun 26, 2017

lore
Share & tools
lore
#0 lore raw

RFC 1/2 extended.testlist: Remove default and render engine test duplicates

From: Tvrtko Ursulin <[email protected]>

Where there is both default and render for the same test, remove the former to save some execution time.

---
 tests/intel-ci/extended.testlist | 21 ---------------------
 1 file changed, 21 deletions(-)
tests/intel-ci/extended.testlist +0 -21
index 24ec295faf66..e0926ff09bcd 100644
@@ -76,7 +76,6 @@ igt@gem_ringfill@blt-bomb
 igt@gem_ringfill@bsd-bomb
 igt@gem_ringfill@bsd1-bomb
 igt@gem_ringfill@bsd2-bomb
-igt@gem_ringfill@default-bomb
 igt@gem_ringfill@render-bomb
 igt@gem_ringfill@vebox-bomb
 igt@gem_userptr_blits@stress-mm
@@ -1154,12 +1153,10 @@ igt@gem_bad_reloc@negative-reloc-bltcopy
 igt@gem_bad_reloc@negative-reloc-bsd
 igt@gem_bad_reloc@negative-reloc-bsd1
 igt@gem_bad_reloc@negative-reloc-bsd2
-igt@gem_bad_reloc@negative-reloc-default
 igt@gem_bad_reloc@negative-reloc-lut-blt
 igt@gem_bad_reloc@negative-reloc-lut-bsd
 igt@gem_bad_reloc@negative-reloc-lut-bsd1
 igt@gem_bad_reloc@negative-reloc-lut-bsd2
-igt@gem_bad_reloc@negative-reloc-lut-default
 igt@gem_bad_reloc@negative-reloc-lut-render
 igt@gem_bad_reloc@negative-reloc-lut-vebox
 igt@gem_bad_reloc@negative-reloc-render
@@ -1203,7 +1200,6 @@ igt@gem_cs_prefetch@blt
 igt@gem_cs_prefetch@bsd
 igt@gem_cs_prefetch@bsd1
 igt@gem_cs_prefetch@bsd2
-igt@gem_cs_prefetch@default
 igt@gem_cs_prefetch@render
 igt@gem_cs_prefetch@vebox
 igt@gem_cs_tlb@blt
@@ -1220,7 +1216,6 @@ igt@gem_ctx_bad_exec@blt
 igt@gem_ctx_bad_exec@bsd
 igt@gem_ctx_bad_exec@bsd1
 igt@gem_ctx_bad_exec@bsd2
-igt@gem_ctx_bad_exec@default
 igt@gem_ctx_bad_exec@render
 igt@gem_ctx_bad_exec@vebox
 igt@gem_ctx_exec@lrc-lite-restore
@@ -1244,7 +1239,6 @@ igt@gem_ctx_switch@bsd1
 igt@gem_ctx_switch@bsd1-interruptible
 igt@gem_ctx_switch@bsd2
 igt@gem_ctx_switch@bsd2-interruptible
-igt@gem_ctx_switch@default-interruptible
 igt@gem_ctx_switch@render
 igt@gem_ctx_switch@render-interruptible
 igt@gem_ctx_switch@vebox
@@ -1322,7 +1316,6 @@ igt@gem_exec_nop@blt
 igt@gem_exec_nop@bsd
 igt@gem_exec_nop@bsd1
 igt@gem_exec_nop@bsd2
-igt@gem_exec_nop@default
 igt@gem_exec_nop@parallel
 igt@gem_exec_nop@render
 igt@gem_exec_nop@series
@@ -1340,9 +1333,6 @@ igt@gem_exec_parallel@bsd2
 igt@gem_exec_parallel@bsd2-contexts
 igt@gem_exec_parallel@bsd2-fds
 igt@gem_exec_parallel@contexts
-igt@gem_exec_parallel@default
-igt@gem_exec_parallel@default-contexts
-igt@gem_exec_parallel@default-fds
 igt@gem_exec_parallel@fds
 igt@gem_exec_parallel@render
 igt@gem_exec_parallel@render-contexts
@@ -1392,7 +1382,6 @@ igt@gem_exec_reloc@active-blt
 igt@gem_exec_reloc@active-bsd
 igt@gem_exec_reloc@active-bsd1
 igt@gem_exec_reloc@active-bsd2
-igt@gem_exec_reloc@active-default
 igt@gem_exec_reloc@active-render
 igt@gem_exec_reloc@active-vebox
 igt@gem_exec_schedule@deep-blt
@@ -1431,7 +1420,6 @@ igt@gem_exec_whisper@bsd1-normal
 igt@gem_exec_whisper@bsd2-normal
 igt@gem_exec_whisper@chain
 igt@gem_exec_whisper@contexts
-igt@gem_exec_whisper@default-normal
 igt@gem_exec_whisper@fds
 igt@gem_exec_whisper@forked
 igt@gem_exec_whisper@interruptible
@@ -1682,8 +1670,6 @@ igt@gem_ringfill@bsd2
 igt@gem_ringfill@bsd2-child
 igt@gem_ringfill@bsd2-interruptible
 igt@gem_ringfill@bsd2-s3
-igt@gem_ringfill@default-child
-igt@gem_ringfill@default-s3
 igt@gem_ringfill@render
 igt@gem_ringfill@render-child
 igt@gem_ringfill@render-interruptible
@@ -1728,26 +1714,22 @@ igt@gem_storedw_loop@long-blt
 igt@gem_storedw_loop@long-bsd
 igt@gem_storedw_loop@long-bsd1
 igt@gem_storedw_loop@long-bsd2
-igt@gem_storedw_loop@long-default
 igt@gem_storedw_loop@long-render
 igt@gem_storedw_loop@long-vebox
 igt@gem_storedw_loop@short-blt
 igt@gem_storedw_loop@short-bsd
 igt@gem_storedw_loop@short-bsd1
 igt@gem_storedw_loop@short-bsd2
-igt@gem_storedw_loop@short-default
 igt@gem_storedw_loop@short-render
 igt@gem_storedw_loop@short-vebox
 igt@gem_sync@blt
 igt@gem_sync@bsd
 igt@gem_sync@bsd1
 igt@gem_sync@bsd2
-igt@gem_sync@default
 igt@gem_sync@many-blt
 igt@gem_sync@many-bsd
 igt@gem_sync@many-bsd1
 igt@gem_sync@many-bsd2
-igt@gem_sync@many-default
 igt@gem_sync@many-render
 igt@gem_sync@many-vebox
 igt@gem_sync@render
@@ -1755,7 +1737,6 @@ igt@gem_sync@store-blt
 igt@gem_sync@store-bsd
 igt@gem_sync@store-bsd1
 igt@gem_sync@store-bsd2
-igt@gem_sync@store-default
 igt@gem_sync@store-render
 igt@gem_sync@store-vebox
 igt@gem_sync@vebox
@@ -1808,7 +1789,6 @@ igt@gem_wait@busy-blt
 igt@gem_wait@busy-bsd
 igt@gem_wait@busy-bsd1
 igt@gem_wait@busy-bsd2
-igt@gem_wait@busy-default
 igt@gem_wait@busy-render
 igt@gem_wait@busy-vebox
 igt@gem_wait@invalid-buf
@@ -1817,7 +1797,6 @@ igt@gem_wait@wait-blt
 igt@gem_wait@wait-bsd
 igt@gem_wait@wait-bsd1
 igt@gem_wait@wait-bsd2
-igt@gem_wait@wait-default
 igt@gem_wait@wait-render
 igt@gem_wait@wait-vebox
 igt@gem_workarounds@reset
Signature · 6 lines
2.9.4

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

RFC 2/2 extended.testlist: Remove some test-subtest combinations

From: Tvrtko Ursulin <[email protected]>

For tests with attempt to hit races and such by running for relatively long time, it seems that it might be possible to get by only testing some subtest-engine combinations as long as in total we still exercise all engines per test.

More precisely, I think that if we view our engines as fast (!rcs) and a little less fast (others), we group the race hitting tests in those two camps, we keep the same coverage by running each subtest against each engine group.

---
 tests/intel-ci/extended.testlist | 44 ----------------------------------------
 1 file changed, 44 deletions(-)
tests/intel-ci/extended.testlist +0 -44
index e0926ff09bcd..246a14f6e501 100644
@@ -73,11 +73,7 @@ igt@gem_pwrite@huge-gtt-fbr
 igt@gem_pwrite@huge-gtt-forwards
 igt@gem_pwrite@huge-gtt-random
 igt@gem_ringfill@blt-bomb
-igt@gem_ringfill@bsd-bomb
-igt@gem_ringfill@bsd1-bomb
-igt@gem_ringfill@bsd2-bomb
 igt@gem_ringfill@render-bomb
-igt@gem_ringfill@vebox-bomb
 igt@gem_userptr_blits@stress-mm
 igt@gem_userptr_blits@stress-mm-invalidate-close
 igt@gem_userptr_blits@stress-mm-invalidate-close-overlap
@@ -1654,30 +1650,14 @@ igt@gem_ring_sync_copy@sync-render-blitter-read-write
 igt@gem_ring_sync_copy@sync-render-blitter-write-read
 igt@gem_ring_sync_copy@sync-render-blitter-write-write
 igt@gem_ring_sync_loop
-igt@gem_ringfill@blt
-igt@gem_ringfill@blt-child
 igt@gem_ringfill@blt-interruptible
-igt@gem_ringfill@blt-s3
 igt@gem_ringfill@bsd
-igt@gem_ringfill@bsd-child
-igt@gem_ringfill@bsd-interruptible
 igt@gem_ringfill@bsd-s3
-igt@gem_ringfill@bsd1
-igt@gem_ringfill@bsd1-child
-igt@gem_ringfill@bsd1-interruptible
-igt@gem_ringfill@bsd1-s3
-igt@gem_ringfill@bsd2
-igt@gem_ringfill@bsd2-child
-igt@gem_ringfill@bsd2-interruptible
-igt@gem_ringfill@bsd2-s3
 igt@gem_ringfill@render
 igt@gem_ringfill@render-child
 igt@gem_ringfill@render-interruptible
 igt@gem_ringfill@render-s3
-igt@gem_ringfill@vebox
 igt@gem_ringfill@vebox-child
-igt@gem_ringfill@vebox-interruptible
-igt@gem_ringfill@vebox-s3
 igt@gem_set_tiling_vs_blt@tiled-to-tiled
 igt@gem_set_tiling_vs_blt@tiled-to-untiled
 igt@gem_set_tiling_vs_blt@untiled-to-tiled
@@ -1711,35 +1691,17 @@ igt@gem_storedw_batches_loop@normal
 igt@gem_storedw_batches_loop@secure-dispatch
 igt@gem_storedw_batches_loop@uncached-mapping
 igt@gem_storedw_loop@long-blt
-igt@gem_storedw_loop@long-bsd
-igt@gem_storedw_loop@long-bsd1
-igt@gem_storedw_loop@long-bsd2
 igt@gem_storedw_loop@long-render
 igt@gem_storedw_loop@long-vebox
-igt@gem_storedw_loop@short-blt
 igt@gem_storedw_loop@short-bsd
-igt@gem_storedw_loop@short-bsd1
-igt@gem_storedw_loop@short-bsd2
 igt@gem_storedw_loop@short-render
 igt@gem_storedw_loop@short-vebox
 igt@gem_sync@blt
-igt@gem_sync@bsd
-igt@gem_sync@bsd1
-igt@gem_sync@bsd2
-igt@gem_sync@many-blt
 igt@gem_sync@many-bsd
-igt@gem_sync@many-bsd1
-igt@gem_sync@many-bsd2
 igt@gem_sync@many-render
-igt@gem_sync@many-vebox
 igt@gem_sync@render
-igt@gem_sync@store-blt
-igt@gem_sync@store-bsd
-igt@gem_sync@store-bsd1
-igt@gem_sync@store-bsd2
 igt@gem_sync@store-render
 igt@gem_sync@store-vebox
-igt@gem_sync@vebox
 igt@gem_threaded_access_tiled
 igt@gem_tiled_blits@interruptible
 igt@gem_tiled_blits@normal
@@ -1786,17 +1748,11 @@ igt@gem_userptr_blits@unsync-unmap-after-close
 igt@gem_userptr_blits@unsync-unmap-cycles
 igt@gem_userptr_blits@usage-restrictions
 igt@gem_wait@busy-blt
-igt@gem_wait@busy-bsd
-igt@gem_wait@busy-bsd1
-igt@gem_wait@busy-bsd2
 igt@gem_wait@busy-render
 igt@gem_wait@busy-vebox
 igt@gem_wait@invalid-buf
 igt@gem_wait@invalid-flags
-igt@gem_wait@wait-blt
 igt@gem_wait@wait-bsd
-igt@gem_wait@wait-bsd1
-igt@gem_wait@wait-bsd2
 igt@gem_wait@wait-render
 igt@gem_wait@wait-vebox
 igt@gem_workarounds@reset
Signature · 6 lines
2.9.4

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

Re: [RFC i-g-t 2/2] extended.testlist: Remove some test-subtest combinations

Quoting Tvrtko Ursulin (2017-06-16 11:55:30)

Quoted 29 lines

From: Tvrtko Ursulin <[email protected]>

For tests with attempt to hit races and such by running for relatively long time, it seems that it might be possible to get by only testing some subtest-engine combinations as long as in total we still exercise all engines per test.

More precisely, I think that if we view our engines as fast (!rcs) and a little less fast (others), we group the race hitting tests in those two camps, we keep the same coverage by running each subtest against each engine group.

---
 tests/intel-ci/extended.testlist | 44 ----------------------------------------
 1 file changed, 44 deletions(-)
tests/intel-ci/extended.testlist +0 -6
index e0926ff09bcd..246a14f6e501 100644
 igt@gem_sync@blt
-igt@gem_sync@bsd
-igt@gem_sync@bsd1
-igt@gem_sync@bsd2
-igt@gem_sync@many-blt
 igt@gem_sync@many-bsd
-igt@gem_sync@many-bsd1
-igt@gem_sync@many-bsd2

tests/intel-ci/extended.testlist igt@gem_storedw_batches_loop@normal · line 1742

Quoted 10 lines
tests/intel-ci/extended.testlist +0 -6
 igt@gem_sync@many-render
-igt@gem_sync@many-vebox
 igt@gem_sync@render
-igt@gem_sync@store-blt
-igt@gem_sync@store-bsd
-igt@gem_sync@store-bsd1
-igt@gem_sync@store-bsd2
 igt@gem_sync@store-render
 igt@gem_sync@store-vebox
-igt@gem_sync@vebox

The selection should be igt@gem_sync@*all* then any smattering of other engines, if desired but coverage of races on individual engines should already be good enough.
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw
In reply to Tvrtko Ursulin

Quoting Tvrtko Ursulin (2017-06-16 11:55:29)

Quoted 4 lines

From: Tvrtko Ursulin <[email protected]>

Where there is both default and render for the same test, remove the former to save some execution time.

Fwiw, ack to using a subset of engines in each test. Though I'd prefer it if we engineered a better subtest for each test that gave coverage across all, the caveat being that we need clear detection of the error (e.g. if we have a dodgy flush on one engine).
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw
In reply to Tvrtko Ursulin
Quoted 4 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Where there is both default and render for the same test, remove the former to save some execution time.

If they are redundant, why do we even have them? Can we just remove the testcase itself? Accumulating unused tests of questionable use at best in igt is serious pain, because it means we never can get to a world where new testcases are auto-added to CI withou some manual review. And that's the world of pain we live in now and I really want to get out of it. That means reviewing and removing testcases, not massaging courated testlists forever, I don't think we have the time for that among all the other tasks.

</rant> :-)

Cheers, Daniel

Quoted 156 lines
---
 tests/intel-ci/extended.testlist | 21 ---------------------
 1 file changed, 21 deletions(-)
tests/intel-ci/extended.testlist +0 -22
index 24ec295faf66..e0926ff09bcd 100644
@@ -76,7 +76,6 @@ igt@gem_ringfill@blt-bomb
 igt@gem_ringfill@bsd-bomb
 igt@gem_ringfill@bsd1-bomb
 igt@gem_ringfill@bsd2-bomb
-igt@gem_ringfill@default-bomb
 igt@gem_ringfill@render-bomb
 igt@gem_ringfill@vebox-bomb
 igt@gem_userptr_blits@stress-mm
@@ -1154,12 +1153,10 @@ igt@gem_bad_reloc@negative-reloc-bltcopy
 igt@gem_bad_reloc@negative-reloc-bsd
 igt@gem_bad_reloc@negative-reloc-bsd1
 igt@gem_bad_reloc@negative-reloc-bsd2
-igt@gem_bad_reloc@negative-reloc-default
 igt@gem_bad_reloc@negative-reloc-lut-blt
 igt@gem_bad_reloc@negative-reloc-lut-bsd
 igt@gem_bad_reloc@negative-reloc-lut-bsd1
 igt@gem_bad_reloc@negative-reloc-lut-bsd2
-igt@gem_bad_reloc@negative-reloc-lut-default
 igt@gem_bad_reloc@negative-reloc-lut-render
 igt@gem_bad_reloc@negative-reloc-lut-vebox
 igt@gem_bad_reloc@negative-reloc-render
@@ -1203,7 +1200,6 @@ igt@gem_cs_prefetch@blt
 igt@gem_cs_prefetch@bsd
 igt@gem_cs_prefetch@bsd1
 igt@gem_cs_prefetch@bsd2
-igt@gem_cs_prefetch@default
 igt@gem_cs_prefetch@render
 igt@gem_cs_prefetch@vebox
 igt@gem_cs_tlb@blt
@@ -1220,7 +1216,6 @@ igt@gem_ctx_bad_exec@blt
 igt@gem_ctx_bad_exec@bsd
 igt@gem_ctx_bad_exec@bsd1
 igt@gem_ctx_bad_exec@bsd2
-igt@gem_ctx_bad_exec@default
 igt@gem_ctx_bad_exec@render
 igt@gem_ctx_bad_exec@vebox
 igt@gem_ctx_exec@lrc-lite-restore
@@ -1244,7 +1239,6 @@ igt@gem_ctx_switch@bsd1
 igt@gem_ctx_switch@bsd1-interruptible
 igt@gem_ctx_switch@bsd2
 igt@gem_ctx_switch@bsd2-interruptible
-igt@gem_ctx_switch@default-interruptible
 igt@gem_ctx_switch@render
 igt@gem_ctx_switch@render-interruptible
 igt@gem_ctx_switch@vebox
@@ -1322,7 +1316,6 @@ igt@gem_exec_nop@blt
 igt@gem_exec_nop@bsd
 igt@gem_exec_nop@bsd1
 igt@gem_exec_nop@bsd2
-igt@gem_exec_nop@default
 igt@gem_exec_nop@parallel
 igt@gem_exec_nop@render
 igt@gem_exec_nop@series
@@ -1340,9 +1333,6 @@ igt@gem_exec_parallel@bsd2
 igt@gem_exec_parallel@bsd2-contexts
 igt@gem_exec_parallel@bsd2-fds
 igt@gem_exec_parallel@contexts
-igt@gem_exec_parallel@default
-igt@gem_exec_parallel@default-contexts
-igt@gem_exec_parallel@default-fds
 igt@gem_exec_parallel@fds
 igt@gem_exec_parallel@render
 igt@gem_exec_parallel@render-contexts
@@ -1392,7 +1382,6 @@ igt@gem_exec_reloc@active-blt
 igt@gem_exec_reloc@active-bsd
 igt@gem_exec_reloc@active-bsd1
 igt@gem_exec_reloc@active-bsd2
-igt@gem_exec_reloc@active-default
 igt@gem_exec_reloc@active-render
 igt@gem_exec_reloc@active-vebox
 igt@gem_exec_schedule@deep-blt
@@ -1431,7 +1420,6 @@ igt@gem_exec_whisper@bsd1-normal
 igt@gem_exec_whisper@bsd2-normal
 igt@gem_exec_whisper@chain
 igt@gem_exec_whisper@contexts
-igt@gem_exec_whisper@default-normal
 igt@gem_exec_whisper@fds
 igt@gem_exec_whisper@forked
 igt@gem_exec_whisper@interruptible
@@ -1682,8 +1670,6 @@ igt@gem_ringfill@bsd2
 igt@gem_ringfill@bsd2-child
 igt@gem_ringfill@bsd2-interruptible
 igt@gem_ringfill@bsd2-s3
-igt@gem_ringfill@default-child
-igt@gem_ringfill@default-s3
 igt@gem_ringfill@render
 igt@gem_ringfill@render-child
 igt@gem_ringfill@render-interruptible
@@ -1728,26 +1714,22 @@ igt@gem_storedw_loop@long-blt
 igt@gem_storedw_loop@long-bsd
 igt@gem_storedw_loop@long-bsd1
 igt@gem_storedw_loop@long-bsd2
-igt@gem_storedw_loop@long-default
 igt@gem_storedw_loop@long-render
 igt@gem_storedw_loop@long-vebox
 igt@gem_storedw_loop@short-blt
 igt@gem_storedw_loop@short-bsd
 igt@gem_storedw_loop@short-bsd1
 igt@gem_storedw_loop@short-bsd2
-igt@gem_storedw_loop@short-default
 igt@gem_storedw_loop@short-render
 igt@gem_storedw_loop@short-vebox
 igt@gem_sync@blt
 igt@gem_sync@bsd
 igt@gem_sync@bsd1
 igt@gem_sync@bsd2
-igt@gem_sync@default
 igt@gem_sync@many-blt
 igt@gem_sync@many-bsd
 igt@gem_sync@many-bsd1
 igt@gem_sync@many-bsd2
-igt@gem_sync@many-default
 igt@gem_sync@many-render
 igt@gem_sync@many-vebox
 igt@gem_sync@render
@@ -1755,7 +1737,6 @@ igt@gem_sync@store-blt
 igt@gem_sync@store-bsd
 igt@gem_sync@store-bsd1
 igt@gem_sync@store-bsd2
-igt@gem_sync@store-default
 igt@gem_sync@store-render
 igt@gem_sync@store-vebox
 igt@gem_sync@vebox
@@ -1808,7 +1789,6 @@ igt@gem_wait@busy-blt
 igt@gem_wait@busy-bsd
 igt@gem_wait@busy-bsd1
 igt@gem_wait@busy-bsd2
-igt@gem_wait@busy-default
 igt@gem_wait@busy-render
 igt@gem_wait@busy-vebox
 igt@gem_wait@invalid-buf
@@ -1817,7 +1797,6 @@ igt@gem_wait@wait-blt
 igt@gem_wait@wait-bsd
 igt@gem_wait@wait-bsd1
 igt@gem_wait@wait-bsd2
-igt@gem_wait@wait-default
 igt@gem_wait@wait-render
 igt@gem_wait@wait-vebox
 igt@gem_workarounds@reset
--

2.9.4

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw
Quoted 7 lines · Daniel Vetter
Quoted 4 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Where there is both default and render for the same test, remove the former to save some execution time.

If they are redundant, why do we even have them? Can we just remove

Maybe we can remove the default entry from intel_execution_engines array? And just test that default == render explicitly in a few short tests.

Quoted 9 lines

the testcase itself? Accumulating unused tests of questionable use at best in igt is serious pain, because it means we never can get to a world where new testcases are auto-added to CI withou some manual review. And that's the world of pain we live in now and I really want to get out of it. That means reviewing and removing testcases, not massaging courated testlists forever, I don't think we have the time for that among all the other tasks.

</rant> :-)

I am not a fan of the testlist neither, but thought to avoid letting perfect being the enemy of good, and at least do something until the final solution crystallizes.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#6 lore raw
Quoted 12 lines · Tvrtko Ursulin
Quoted 10 lines · Daniel Vetter
Quoted 5 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Where there is both default and render for the same test, remove the former to save some execution time.

Quoted 5 lines
Quoted 1 line

If they are redundant, why do we even have them? Can we just remove

Maybe we can remove the default entry from intel_execution_engines array? And just test that default == render explicitly in a few short tests.

That has my ack, in case you have the patch ...

Quoted 8 lines
Quoted 8 lines

the testcase itself? Accumulating unused tests of questionable use at best in igt is serious pain, because it means we never can get to a world where new testcases are auto-added to CI withou some manual review. And that's the world of pain we live in now and I really want to get out of it. That means reviewing and removing testcases, not massaging courated testlists forever, I don't think we have the time for that among all the other tasks.

Quoted 6 lines
Quoted 1 line

</rant> :-)

I am not a fan of the testlist neither, but thought to avoid letting perfect being the enemy of good, and at least do something until the final solution crystallizes.

Yeah not going to block this here, just wanted to chime in with what I think we need to do in the longer-term. I really want CI to run all igt, and run new igt tests by default, to make sure both tests are high-quality (no noise), and the kernel doesn't randomly regress because we didn't run the igts :-/
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
+41 (0) 79 365 57 48 - http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#7 lore raw

Quoting Daniel Vetter (2017-06-23 12:14:49)

Quoted 14 lines
Quoted 12 lines · Tvrtko Ursulin
Quoted 10 lines · Daniel Vetter
Quoted 5 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Where there is both default and render for the same test, remove the former to save some execution time.

Quoted 7 lines
Quoted 5 lines
Quoted 1 line

If they are redundant, why do we even have them? Can we just remove

Maybe we can remove the default entry from intel_execution_engines array? And just test that default == render explicitly in a few short tests.

That has my ack, in case you have the patch ...

No. The ABI does allow for the switch in theory and it is a distinct enum so it does need testing.

Quoted 9 lines
Quoted 9 lines
Quoted 9 lines

the testcase itself? Accumulating unused tests of questionable use at best in igt is serious pain, because it means we never can get to a world where new testcases are auto-added to CI withou some manual review. And that's the world of pain we live in now and I really want to get out of it. That means reviewing and removing testcases, not massaging courated testlists forever, I don't think we have the time for that among all the other tasks.

</rant> :-)

It's the curated lists that are the problem, not the tests. The fact that we don't feel it is worthwhile to continuously test minor variations to catch weird bugs is our problem.
-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#8 lore raw
Quoted 18 lines · Chris Wilson

Quoting Daniel Vetter (2017-06-23 12:14:49)

Quoted 17 lines
Quoted 15 lines · Tvrtko Ursulin
Quoted 11 lines · Daniel Vetter
Quoted 5 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Where there is both default and render for the same test, remove the former to save some execution time.

If they are redundant, why do we even have them? Can we just remove

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

Maybe we can remove the default entry from intel_execution_engines array? And just test that default == render explicitly in a few short tests.

That has my ack, in case you have the patch ...

No. The ABI does allow for the switch in theory and it is a distinct enum so it does need testing.

One testcase to make sure the default engine is the render engine is I think ok. Testing all the combinatorics is over the top.

Quoted 8 lines
Quoted 8 lines
Quoted 8 lines
Quoted 8 lines

the testcase itself? Accumulating unused tests of questionable use at best in igt is serious pain, because it means we never can get to a world where new testcases are auto-added to CI withou some manual review. And that's the world of pain we live in now and I really want to get out of it. That means reviewing and removing testcases, not massaging courated testlists forever, I don't think we have the time for that among all the other tasks.

Quoted 5 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line

</rant> :-)

It's the curated lists that are the problem, not the tests. The fact that we don't feel it is worthwhile to continuously test minor variations to catch weird bugs is our problem.

So it's an different topic, but I disagree. gem_* testcases are massively over the top with how much machine time they blow through, to the point where gem takes 90%+ of all machine time.

The result of that is that we simply don't run anything at all, so gem is hurting kms_ and other test coverage. We really need to get this down to something reasonable, and I think reasonable = completes within 1 work-day on a fast machine. Stress-tests are ok to keep, and run on an ad-hoc basis, but we can't have them in the default list.

Atm gem_concurrent_blt alone blows through 24 days of machine time, and that's 24 days of not testing some other patch series. We have a too high patch count to afford that, so if you insist on keeping all these gem tests in the default set then we'll be able to merge about 1 patch series per week. That just doesn't work.

Even excluding gem_concurrent_blt gem_* tests take a few times more than everything else combined.

There's really imo just 2 options:
a) drastically reduce the time we spend running gem_* tests.
b) don't run gem_* tests, and don't merge gem_* patches.

Atm the result is that we can't even run the other igt tests, which means everyone (not just the gem team) loses.
-Daniel

Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx