Skip to contentSkip to conversation

tests/gem_exec_flush: Tune down BAT timeout to ~1sec.

PATCH In review Marius Vlad 4 participants 24 messages May 27, 2016

lore
Share & tools
lore
#0 lore raw

PATCH 1/8 tests/gem_exec_flush: Tune down BAT timeout to ~1sec.

---
 tests/gem_exec_flush.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/gem_exec_flush.c +1 -1
index d08b843..b608060 100644
@@ -523,7 +523,7 @@ igt_main
 
 	for (e = intel_execution_engines; e->name; e++) igt_subtest_group {
 		unsigned ring = e->exec_id | e->flags;
-		unsigned timeout = 5 + 120*!!e->exec_id;
+		unsigned timeout = 1 + 120*!!e->exec_id;
 
 		igt_fixture {
 			gem_require_ring(fd, ring);
Signature · 6 lines
2.5.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 2/8 tests/gem_ctx_switch: Tune down for BAT ~1s.

---
 tests/gem_ctx_switch.c | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
tests/gem_ctx_switch.c +2 -1
index 7b27336..066a5fc 100644
@@ -73,6 +73,7 @@ static void single(int fd, uint32_t handle,
 	struct drm_i915_gem_exec_object2 obj;
 	struct drm_i915_gem_relocation_entry reloc;
 	uint32_t contexts[64];
+	uint32_t timeout = 1;
 	int n;
 
 	gem_require_ring(fd, e->exec_id | e->flags);
@@ -125,7 +126,7 @@ static void single(int fd, uint32_t handle,
 				count += 1024;
 			}
 			clock_gettime(CLOCK_MONOTONIC, &now);
-		} while (elapsed(&start, &now) < 20.);
+		} while (elapsed(&start, &now) < timeout);
 		gem_sync(fd, handle);
 		clock_gettime(CLOCK_MONOTONIC, &now);
 
Signature · 6 lines
2.5.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

Re: [PATCH i-g-t 2/8] tests/gem_ctx_switch: Tune down for BAT ~1s.

Quoted 1 line · Marius Vlad

Nak. Please suggest how to improve reliablity first. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH 3/8 tests/gem_exec_nop: Tune down timeout for BAT ~1s.

In reply to Marius Vlad
---
 tests/gem_exec_nop.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/gem_exec_nop.c +1 -1
index e90d5eb..1b2d144 100644
@@ -213,7 +213,7 @@ igt_main
 	igt_fork_hang_detector(device);
 
 	igt_subtest("basic")
-		all(device, handle, 10);
+		all(device, handle, 1);
 
 	for (e = intel_execution_engines; e->name; e++)
 		igt_subtest_f("%s", e->name)
Signature · 6 lines
2.5.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

Re: [PATCH i-g-t 3/8] tests/gem_exec_nop: Tune down timeout for BAT ~1s.

On Fri, May 27, 2016 at 02:50:33PM +0300, Marius Vlad wrote:

You need some explanation here to explain how you believe that all the corner-cases from this minimal stress test are covered elsewhere. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

PATCH 4/8 tests/gem_close_race: Tune down for BAT ~1s.

In reply to Marius Vlad
---
 tests/gem_close_race.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/gem_close_race.c +1 -1
index 94fb905..45aa2cc 100644
@@ -232,7 +232,7 @@ igt_main
 	}
 
 	igt_subtest("basic-threads")
-		threads(10);
+		threads(1);
 
 	igt_subtest("process-exit") {
 		igt_fork(child, 768)
Signature · 6 lines
2.5.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#6 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#7 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

Quoted 5 lines · Chris Wilson
Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.

As a more or less well know TV personality would say - "it's better than nothing"! :))

Seriously, under the new rules this is all we can do. Full gem-close-race will (will it?) run in the nightly run so a little bit more pain while bisecting breakages but those are the rules. We can give them a go and see how it works out.

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#8 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

Quoted 8 lines · Tvrtko Ursulin
Quoted 5 lines · Chris Wilson
Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.

Quoted 7 lines

As a more or less well know TV personality would say - "it's better than nothing"! :))

Seriously, under the new rules this is all we can do. Full gem-close-race will (will it?) run in the nightly run so a little bit more pain while bisecting breakages but those are the rules. We can give them a go and see how it works out.

Wrong approach. Right approach would be to add a new test that reliably detected a checklist of the most common races in under 1s. Nerfing a test to make it useless makes BAT equally useless.

So we are giving up on BAT?
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#9 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

Quoted 17 lines · Chris Wilson
Quoted 8 lines · Tvrtko Ursulin
Quoted 5 lines · Chris Wilson
Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.

Quoted 7 lines

As a more or less well know TV personality would say - "it's better than nothing"! :))

Seriously, under the new rules this is all we can do. Full gem-close-race will (will it?) run in the nightly run so a little bit more pain while bisecting breakages but those are the rules. We can give them a go and see how it works out.

Quoted 5 lines

Wrong approach. Right approach would be to add a new test that reliably detected a checklist of the most common races in under 1s. Nerfing a test to make it useless makes BAT equally useless.

So we are giving up on BAT?

If it is not possible to stuff everything into the allocated budget, and in cases where it may not be possible to come up with a 1s race detector, it makes sense to move the test out of BAT and into the nightly runs.

Agreement was that the time limits are hard limits so thats pretty much it. We can do this now and hit the targets and them add more tests if and when someone manages to implement them.

I don't have a problem with that, B in BAT stands for basic anyway.

We just adjust the model that when nightly test run fails someone from QA gets tasked with bisection etc as a top priority.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#10 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

Quoted 25 lines · Tvrtko Ursulin
Quoted 17 lines · Chris Wilson
Quoted 8 lines · Tvrtko Ursulin
Quoted 5 lines · Chris Wilson
Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.

Quoted 7 lines

As a more or less well know TV personality would say - "it's better than nothing"! :))

Seriously, under the new rules this is all we can do. Full gem-close-race will (will it?) run in the nightly run so a little bit more pain while bisecting breakages but those are the rules. We can give them a go and see how it works out.

Quoted 5 lines

Wrong approach. Right approach would be to add a new test that reliably detected a checklist of the most common races in under 1s. Nerfing a test to make it useless makes BAT equally useless.

So we are giving up on BAT?

Quoted 8 lines

If it is not possible to stuff everything into the allocated budget, and in cases where it may not be possible to come up with a 1s race detector, it makes sense to move the test out of BAT and into the nightly runs.

Agreement was that the time limits are hard limits so thats pretty much it. We can do this now and hit the targets and them add more tests if and when someone manages to implement them.

1s is totally arbitrary, setting it as a hard limit is ridiculous.

Quoted 1 line

I don't have a problem with that, B in BAT stands for basic anyway.

The most basic userspace appears far more complex than igt. Quite often BAT passes, but a couple of seconds of dogfooding reveals an unusable mess. Basic Acceptance Testing is failing in its task, and we seem to heading down the path of making it fail harder.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#11 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

In reply to Tvrtko Ursulin
Quoted 24 lines · Tvrtko Ursulin

[ text/plain ]

Quoted 21 lines · Chris Wilson
Quoted 8 lines · Tvrtko Ursulin
Quoted 5 lines · Chris Wilson
Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.

Quoted 7 lines

As a more or less well know TV personality would say - "it's better than nothing"! :))

Seriously, under the new rules this is all we can do. Full gem-close-race will (will it?) run in the nightly run so a little bit more pain while bisecting breakages but those are the rules. We can give them a go and see how it works out.

Wrong approach. Right approach would be to add a new test that reliably detected a checklist of the most common races in under 1s. Nerfing a test to make it useless makes BAT equally useless.

Quoted 7 lines
Quoted 1 line

So we are giving up on BAT?

If it is not possible to stuff everything into the allocated budget, and in cases where it may not be possible to come up with a 1s race detector, it makes sense to move the test out of BAT and into the nightly runs.

With tests to poke out races, I think the best way is to take them out from BAT and move to nightly.

As A is for acceptance, it is not much value as some poor soul will hit the race 1 month after we regressed it.

- Mika

Quoted 4 lines

Agreement was that the time limits are hard limits so thats pretty much it. We can do this now and hit the targets and them add more tests if and when someone manages to implement them.

Quoted 8 lines

I don't have a problem with that, B in BAT stands for basic anyway.

We just adjust the model that when nightly test run fails someone from QA gets tasked with bisection etc as a top priority.

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#12 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

Quoted 28 lines · Mika Kuoppala
Quoted 26 lines · Tvrtko Ursulin

[ text/plain ]

Quoted 22 lines · Chris Wilson
Quoted 8 lines · Tvrtko Ursulin
Quoted 5 lines · Chris Wilson
Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.

Quoted 7 lines

As a more or less well know TV personality would say - "it's better than nothing"! :))

Seriously, under the new rules this is all we can do. Full gem-close-race will (will it?) run in the nightly run so a little bit more pain while bisecting breakages but those are the rules. We can give them a go and see how it works out.

Wrong approach. Right approach would be to add a new test that reliably detected a checklist of the most common races in under 1s. Nerfing a test to make it useless makes BAT equally useless.

So we are giving up on BAT?

Quoted 8 lines
Quoted 5 lines

If it is not possible to stuff everything into the allocated budget, and in cases where it may not be possible to come up with a 1s race detector, it makes sense to move the test out of BAT and into the nightly runs.

With tests to poke out races, I think the best way is to take them out from BAT and move to nightly.

Right, going to send another series with them removed if everyone agrees on this.

Quoted 19 lines

As A is for acceptance, it is not much value as some poor soul will hit the race 1 month after we regressed it.

- Mika

Quoted 12 lines

Agreement was that the time limits are hard limits so thats pretty much it. We can do this now and hit the targets and them add more tests if and when someone manages to implement them.

I don't have a problem with that, B in BAT stands for basic anyway.

We just adjust the model that when nightly test run fails someone from QA gets tasked with bisection etc as a top priority.

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#13 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

In reply to Chris Wilson
Quoted 17 lines · Chris Wilson
Quoted 8 lines · Tvrtko Ursulin
Quoted 5 lines · Chris Wilson
Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.

Quoted 7 lines

As a more or less well know TV personality would say - "it's better than nothing"! :))

Seriously, under the new rules this is all we can do. Full gem-close-race will (will it?) run in the nightly run so a little bit more pain while bisecting breakages but those are the rules. We can give them a go and see how it works out.

Quoted 5 lines

Wrong approach. Right approach would be to add a new test that reliably detected a checklist of the most common races in under 1s. Nerfing a test to make it useless makes BAT equally useless.

So we are giving up on BAT?

It is a compromise. Either this or nigthly.

Quoted 4 lines

-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#14 lore raw

Re: [PATCH i-g-t 4/8] tests/gem_close_race: Tune down for BAT ~1s.

In reply to Tvrtko Ursulin
Quoted 8 lines · Tvrtko Ursulin
Quoted 5 lines · Chris Wilson
Quoted 1 line · Marius Vlad

Nak. It's a race detector. Please suggest how to increase detection rates.

Quoted 7 lines

As a more or less well know TV personality would say - "it's better than nothing"! :))

Seriously, under the new rules this is all we can do. Full gem-close-race will (will it?) run in the nightly run so a little bit more pain while bisecting breakages but those are the rules. We can give them a go and see how it works out.

Combinatorial tests (gem_concurrent_all/blit) and gem_evict_everything are the only exceptions. We've been running nightly for over a week now.

You can have a look under /archive/results/Nightly/.

I'm not saying its perfect, but at least we have somewhere to start.

Quoted 5 lines

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#15 lore raw

PATCH 5/8 tests/gem_sync: Tune down for BAT to ~1s for basic-each and ~2s for basic-all.

In reply to Marius Vlad
---
 tests/gem_sync.c | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
tests/gem_sync.c +2 -2
index 320bce3..f824d1e 100644
@@ -126,7 +126,7 @@ sync_ring(int fd, unsigned ring, int num_children)
 				gem_execbuf(fd, &execbuf);
 				gem_sync(fd, object.handle);
 			} while (++cycles & 1023);
-		} while ((elapsed = gettime() - start) < SLOW_QUICK(10, 1));
+		} while ((elapsed = gettime() - start) < 1);
 		igt_info("%s%sompleted %ld cycles: %.3f us\n",
 			 names[child % num_engines] ?: "",
 			 names[child % num_engines] ? " c" : "C",
@@ -191,7 +191,7 @@ sync_all(int fd, int num_children)
 				}
 				gem_sync(fd, object.handle);
 			} while (++cycles & 1023);
-		} while ((elapsed = gettime() - start) < SLOW_QUICK(10, 1));
+		} while ((elapsed = gettime() - start) < SLOW_QUICK(2, 1));
 		igt_info("Completed %ld cycles: %.3f us\n",
 			 cycles, elapsed*1e6/cycles);
 
Signature · 6 lines
2.5.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#16 lore raw

Re: [PATCH i-g-t 5/8] tests/gem_sync: Tune down for BAT to ~1s for basic-each and ~2s for basic-all.

Quoted 1 line · Marius Vlad

NAK. gem_sync is not reliable enough, please suggest how to improve detection rates for *existing* issues.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#17 lore raw

PATCH 6/8 tests/gem_exec_fault: Decrease the counts to lower the time for BAT.

In reply to Marius Vlad
---
 tests/gem_storedw_loop.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/gem_storedw_loop.c +1 -1
index 317b8c6..89f0182 100644
@@ -187,7 +187,7 @@ igt_main
 	for (e = intel_execution_engines; e->name; e++) {
 		igt_subtest_f("basic-%s", e->name) {
 			check_test_requirements(fd, e->exec_id);
-			store_test(fd, e->exec_id | e->flags, 16*1024);
+			store_test(fd, e->exec_id | e->flags, 4*1024);
 		}
 
 		igt_subtest_f("long-%s", e->name) {
Signature · 6 lines
2.5.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#18 lore raw

Re: [PATCH i-g-t 6/8] tests/gem_exec_fault: Decrease the counts to lower the time for BAT.

Quoted 1 line · Marius Vlad

Summary and patch do not match.

But you could remove gem_storedw_loop as everything BAT about it is tested elsewhere.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#19 lore raw

PATCH 7/8 tests/gem_tiled_pread_basic: Rename test and BATify.

In reply to Marius Vlad

Add a extended subtest and reduce the number of iteration to qualify for BAT. Renamed to test to gem_tiled_pread.

---
 tests/Makefile.sources        |   2 +-
 tests/gem_tiled_pread.c       | 231 ++++++++++++++++++++++++++++++++++++++++++
 tests/gem_tiled_pread_basic.c | 215 ---------------------------------------
 3 files changed, 232 insertions(+), 216 deletions(-)
 create mode 100644 tests/gem_tiled_pread.c
 delete mode 100644 tests/gem_tiled_pread_basic.c
tests/Makefile.sources +1 -1
index c81eeeb..86328cb 100644
@@ -175,7 +175,7 @@ TESTS_progs = \
 	gem_sync \
 	gem_threaded_access_tiled \
 	gem_tiled_fence_blits \
-	gem_tiled_pread_basic \
+	gem_tiled_pread \
 	gem_tiled_pread_pwrite \
 	gem_tiled_swapping \
 	gem_tiled_wb \
tests/gem_tiled_pread.c new file +231 -0

231 lines not on this page · raw message

tests/gem_tiled_pread_basic.c deleted +0 -215

215 lines not on this page · raw message

Signature · 6 lines
2.5.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#20 lore raw

Re: [PATCH i-g-t 7/8] tests/gem_tiled_pread_basic: Rename test and BATify.

Quoted 2 lines · Marius Vlad

Add a extended subtest and reduce the number of iteration to qualify for BAT. Renamed to test to gem_tiled_pread.

No. This doesn't increase coverage.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#21 lore raw

PATCH 8/8 tests/gem_exec_suspend: Remove hibernate test from BAT.

In reply to Marius Vlad
---
 tests/gem_exec_suspend.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
tests/gem_exec_suspend.c +1 -1
index cd133cc..3d529bd 100644
@@ -247,7 +247,7 @@ igt_main
 		run_test(fd, -1, NOSLEEP);
 	igt_subtest("basic-S3")
 		run_test(fd, -1, SUSPEND);
-	igt_subtest("basic-S4")
+	igt_subtest("S4")
 		run_test(fd, -1, HIBERNATE);
 
 	for (e = intel_execution_engines; e->name; e++) {
Signature · 6 lines
2.5.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#22 lore raw

Re: [PATCH i-g-t 8/8] tests/gem_exec_suspend: Remove hibernate test from BAT.

Quoted 1 line · Marius Vlad

Nak. Please suggest how to fix the broken machines first. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx