Skip to contentSkip to conversation

Tracing & workload simulation misc patches

PATCH In review Tvrtko Ursulin 5 participants 24 messages Sep 5, 2018 – Sep 7, 2018

lore
Share & tools
lore
#0 lore raw

cover Tracing & workload simulation misc patches

From: Tvrtko Ursulin <[email protected]>

These are the small fixes and new features not depending on any unmerged patches.

Virtual engine and Dynamic SSEU support is built on top of these.

Each sub-series (trace.pl, gem_wsim, media-bench) starts with trivial patches and fixed, which are then followed by new features in cases of gem_wsim and media-bench.

Tvrtko Ursulin (13):
trace.pl: Fix frequency timeline
trace.pl: Use undocumented -o to perf record to allow tee
gem_wsim: Fix BCS usage under VCS2 remap warning
gem_wsim: Check sleep times
gem_wsim: Make workload commands case sensitive
gem_wsim: Context priority support
gem_wsim: Make batches preemptable by default
gem_wsim: Per context preemption point control
media-bench: Update for engine=class:instance tracepoints
media-bench: Protect against incorrect -b syntax
media-bench: Fix tracing of direct workloads
media-bench: Write out trace files directly.
media-bench: Add mixed mode evaluation

 benchmarks/gem_wsim.c  | 214 ++++++++++++++++++++++++++++++++++---
 benchmarks/wsim/README |  34 ++++++
 scripts/media-bench.pl | 235 +++++++++++++++++++++++++++++++----------
 scripts/trace.pl       |   5 +-
 4 files changed, 416 insertions(+), 72 deletions(-)
Signature · 6 lines
2.17.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 1/13 trace.pl: Fix frequency timeline

From: Tvrtko Ursulin <[email protected]>

Frequency timeline needs to be finished with an entry spanning to the end of known time so that the last known frequency is displayed.

---
 scripts/trace.pl | 2 ++
 1 file changed, 2 insertions(+)
scripts/trace.pl +2 -0
index 1924333e12b6..2976cfdf585a 100755
@@ -1201,6 +1201,8 @@ foreach my $key (sort sortQueue keys %db) {
 	last if $i > $max_items;
 }
 
+push @freqs, [$prev_freq_ts, $last_ts, $prev_freq] if $prev_freq;
+
 foreach my $item (@freqs) {
 	my ($start, $end, $freq) = @$item;
 	my $startend;
Signature · 6 lines
2.17.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

PATCH 2/13 trace.pl: Use undocumented -o to perf record to allow tee

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

---
 scripts/trace.pl | 3 ++-
 1 file changed, 2 insertions(+), 1 deletion(-)
scripts/trace.pl +2 -1
index 2976cfdf585a..18f9f3b18396 100755
@@ -170,8 +170,9 @@ sub arg_trace
 	if ($_[0] eq '--trace') {
 		shift @_;
 
+		unshift @_, '--';
 		unshift @_, join(',', @events);
-		unshift @_, ('perf', 'record', '-a', '-c', '1', '-q', '-e');
+		unshift @_, ('perf', 'record', '-a', '-c', '1', '-q', '-o', 'perf.data', '-e');
 
 		exec @_;
 	}
Signature · 6 lines
2.17.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH 3/13 gem_wsim: Fix BCS usage under VCS2 remap warning

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Need to check we actually are in VCS2 remapping mode!

---
 benchmarks/gem_wsim.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
benchmarks/gem_wsim.c +1 -1
index 80f180829241..25af4d678ba4 100644
@@ -592,7 +592,7 @@ add_step:
 		}
 	}
 
-	if (bcs_used && verbose)
+	if (bcs_used && (flags & VCS2REMAP) && verbose)
 		printf("BCS usage in workload with VCS2 remapping enabled!\n");
 
 	return wrk;
Signature · 6 lines
2.17.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

PATCH 4/13 gem_wsim: Check sleep times

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Notice in more places if we are running behind.

---
 benchmarks/gem_wsim.c | 52 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 46 insertions(+), 6 deletions(-)
benchmarks/gem_wsim.c +46 -6
index 25af4d678ba4..b05e9760f419 100644
@@ -1718,6 +1718,21 @@ static bool sync_deps(struct workload *wrk, struct w_step *w)
 	return synced;
 }
 
+static unsigned int measured_usleep(unsigned int usec)
+{
+	struct timespec ts = { };
+	unsigned int slept;
+
+	slept = igt_nsec_elapsed(&ts);
+	igt_assert(slept == 0);
+	do {
+		usleep(usec - slept);
+		slept = igt_nsec_elapsed(&ts) / 1000;
+	} while (slept < usec);
+
+	return igt_nsec_elapsed(&ts);
+}
+
 static void *run_workload(void *data)
 {
 	struct workload *wrk = (struct workload *)data;
@@ -1739,7 +1754,7 @@ static void *run_workload(void *data)
 	     count++) {
 		unsigned int cur_seqno = wrk->sync_seqno;
 
-		clock_gettime(CLOCK_MONOTONIC, &wrk->repeat_start);
+		igt_gettime(&wrk->repeat_start);
 
 		for (i = 0, w = wrk->steps; wrk->run && (i < wrk->nr_steps);
 		     i++, w++) {
@@ -1751,13 +1766,14 @@ static void *run_workload(void *data)
 			} else if (w->type == PERIOD) {
 				struct timespec now;
 
-				clock_gettime(CLOCK_MONOTONIC, &now);
+				igt_gettime(&now);
 				do_sleep = w->period -
 					   elapsed_us(&wrk->repeat_start, &now);
 				if (do_sleep < 0) {
-					if (verbose > 1)
-						printf("%u: Dropped period @ %u/%u (%dus late)!\n",
-						       wrk->id, count, i, do_sleep);
+					if (verbose > 1 &&
+					    -do_sleep > 10 * w->period / 100)
+						printf("%u: Missed period @ %u/%u (%dus late)!\n",
+						       wrk->id, count, i, -do_sleep);
 					continue;
 				}
 			} else if (w->type == SYNC) {
@@ -1793,7 +1809,31 @@ static void *run_workload(void *data)
 			}
 
 			if (do_sleep || w->type == PERIOD) {
-				usleep(do_sleep);
+				long slept = measured_usleep(do_sleep) / 1000;
+
+				if ( w->type == PERIOD) {
+					struct timespec now;
+
+					igt_gettime(&now);
+
+					slept = elapsed_us(&wrk->repeat_start,
+							   &now);
+
+					if (verbose > 1 &&
+					    slept > (110 * w->period / 100))
+						printf("%u: Overslept period by %ldus @ %u/%u! (%uus)\n",
+						       wrk->id,
+						       -slept, count,
+						       i, w->period);
+				} else {
+					if (verbose > 1 &&
+					    slept > (110 * do_sleep / 100))
+						printf("%u: Overslept by %ldus @ %u/%u! (%uus)\n",
+						       wrk->id,
+						       slept - do_sleep, count,
+						       i, do_sleep);
+				}
+
 				continue;
 			}
 
Signature · 6 lines
2.17.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

Re: [igt-dev] [PATH i-g-t 04/13] gem_wsim: Check sleep times

Quoted 18 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Notice in more places if we are running behind.

---
 benchmarks/gem_wsim.c | 52 ++++++++++++++++++++++++++++++++++++++-----
 1 file changed, 46 insertions(+), 6 deletions(-)
benchmarks/gem_wsim.c +1 -0
index 25af4d678ba4..b05e9760f419 100644
@@ -1718,6 +1718,21 @@ static bool sync_deps(struct workload *wrk, struct w_step *w)
 	return synced;
 }
 
+static unsigned int measured_usleep(unsigned int usec)

benchmarks/gem_wsim.c static bool sync_deps(struct workload *wrk, struct w_step *w) · line 1731

Quoted 10 lines
benchmarks/gem_wsim.c +10 -0
+{
+	struct timespec ts = { };
+	unsigned int slept;
+
+	slept = igt_nsec_elapsed(&ts);
+	igt_assert(slept == 0);
+	do {
+		usleep(usec - slept);
+		slept = igt_nsec_elapsed(&ts) / 1000;
+	} while (slept < usec);

clock_nanosleep(ABS)?

Quoted 75 lines
benchmarks/gem_wsim.c +35 -6
+
+	return igt_nsec_elapsed(&ts);
+}
+
 static void *run_workload(void *data)
 {
 	struct workload *wrk = (struct workload *)data;
@@ -1739,7 +1754,7 @@ static void *run_workload(void *data)
 	     count++) {
 		unsigned int cur_seqno = wrk->sync_seqno;
 
-		clock_gettime(CLOCK_MONOTONIC, &wrk->repeat_start);
+		igt_gettime(&wrk->repeat_start);
 
 		for (i = 0, w = wrk->steps; wrk->run && (i < wrk->nr_steps);
 		     i++, w++) {
@@ -1751,13 +1766,14 @@ static void *run_workload(void *data)
 			} else if (w->type == PERIOD) {
 				struct timespec now;
 
-				clock_gettime(CLOCK_MONOTONIC, &now);
+				igt_gettime(&now);
 				do_sleep = w->period -
 					   elapsed_us(&wrk->repeat_start, &now);
 				if (do_sleep < 0) {
-					if (verbose > 1)
-						printf("%u: Dropped period @ %u/%u (%dus late)!\n",
-						       wrk->id, count, i, do_sleep);
+					if (verbose > 1 &&
+					    -do_sleep > 10 * w->period / 100)
+						printf("%u: Missed period @ %u/%u (%dus late)!\n",
+						       wrk->id, count, i, -do_sleep);
 					continue;
 				}
 			} else if (w->type == SYNC) {
@@ -1793,7 +1809,31 @@ static void *run_workload(void *data)
 			}
 
 			if (do_sleep || w->type == PERIOD) {
-				usleep(do_sleep);
+				long slept = measured_usleep(do_sleep) / 1000;
+
+				if ( w->type == PERIOD) {
+					struct timespec now;
+
+					igt_gettime(&now);
+
+					slept = elapsed_us(&wrk->repeat_start,
+							   &now);
+
+					if (verbose > 1 &&
+					    slept > (110 * w->period / 100))
+						printf("%u: Overslept period by %ldus @ %u/%u! (%uus)\n",
+						       wrk->id,
+						       -slept, count,
+						       i, w->period);
+				} else {
+					if (verbose > 1 &&
+					    slept > (110 * do_sleep / 100))
+						printf("%u: Overslept by %ldus @ %u/%u! (%uus)\n",
+						       wrk->id,
+						       slept - do_sleep, count,
+						       i, do_sleep);
+				}
+
 				continue;
 			}
 
Signature · 6 lines
2.17.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
Signature · 6 lines
Ville Syrjälä
Intel
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#6 lore raw

Re: [igt-dev] [Intel-gfx] [PATH i-g-t 04/13] gem_wsim: Check sleep times

Quoted 18 lines · Ville Syrjälä
Quoted 17 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Notice in more places if we are running behind.

---
  benchmarks/gem_wsim.c | 52 ++++++++++++++++++++++++++++++++++++++-----
  1 file changed, 46 insertions(+), 6 deletions(-)
benchmarks/gem_wsim.c +0 -0
index 25af4d678ba4..b05e9760f419 100644
@@ -1718,6 +1718,21 @@ static bool sync_deps(struct workload *wrk, struct w_step *w)
  	return synced;
  }
 
Quoted 13 lines

benchmarks/gem_wsim.c static bool sync_deps(struct workload *wrk, struct w_step *w) · line 1731

Quoted 11 lines
benchmarks/gem_wsim.c +11 -0
+static unsigned int measured_usleep(unsigned int usec)
+{
+	struct timespec ts = { };
+	unsigned int slept;
+
+	slept = igt_nsec_elapsed(&ts);
+	igt_assert(slept == 0);
+	do {
+		usleep(usec - slept);
+		slept = igt_nsec_elapsed(&ts) / 1000;
+	} while (slept < usec);

clock_nanosleep(ABS)?

Hm I think I see what you mean. Rather than a relative sleep trying to hit the loop period, ask from the kernel (or glibc, I don't know who implements it) to sleep until an absolute target. This totally makes sense and would simplify the code from one angle, I am just not sure if absolute sleep can be relied upon any better to not oversleep. Well, actually for scheduling delays not to affect the caller. However maybe it doesn't matter since AFAIR my main problem were dropped period due GPU activity (the first pair of warning messages in the patch), and again AFAIR, it was quite hard to hit the second ones.

I'll revisit it to remind myself for sure.

Thanks for the drive-by, it was extremely surprising you looked into this obscure for most series. :)

Regards,

Tvrtko

Quoted 68 lines
Quoted 68 lines
benchmarks/gem_wsim.c +35 -6
+
+	return igt_nsec_elapsed(&ts);
+}
+
  static void *run_workload(void *data)
  {
  	struct workload *wrk = (struct workload *)data;
@@ -1739,7 +1754,7 @@ static void *run_workload(void *data)
  	     count++) {
  		unsigned int cur_seqno = wrk->sync_seqno;
 
-		clock_gettime(CLOCK_MONOTONIC, &wrk->repeat_start);
+		igt_gettime(&wrk->repeat_start);
 
  		for (i = 0, w = wrk->steps; wrk->run && (i < wrk->nr_steps);
  		     i++, w++) {
@@ -1751,13 +1766,14 @@ static void *run_workload(void *data)
  			} else if (w->type == PERIOD) {
  				struct timespec now;
 
-				clock_gettime(CLOCK_MONOTONIC, &now);
+				igt_gettime(&now);
  				do_sleep = w->period -
  					   elapsed_us(&wrk->repeat_start, &now);
  				if (do_sleep < 0) {
-					if (verbose > 1)
-						printf("%u: Dropped period @ %u/%u (%dus late)!\n",
-						       wrk->id, count, i, do_sleep);
+					if (verbose > 1 &&
+					    -do_sleep > 10 * w->period / 100)
+						printf("%u: Missed period @ %u/%u (%dus late)!\n",
+						       wrk->id, count, i, -do_sleep);
  					continue;
  				}
  			} else if (w->type == SYNC) {
@@ -1793,7 +1809,31 @@ static void *run_workload(void *data)
  			}
 
  			if (do_sleep || w->type == PERIOD) {
-				usleep(do_sleep);
+				long slept = measured_usleep(do_sleep) / 1000;
+
+				if ( w->type == PERIOD) {
+					struct timespec now;
+
+					igt_gettime(&now);
+
+					slept = elapsed_us(&wrk->repeat_start,
+							   &now);
+
+					if (verbose > 1 &&
+					    slept > (110 * w->period / 100))
+						printf("%u: Overslept period by %ldus @ %u/%u! (%uus)\n",
+						       wrk->id,
+						       -slept, count,
+						       i, w->period);
+				} else {
+					if (verbose > 1 &&
+					    slept > (110 * do_sleep / 100))
+						printf("%u: Overslept by %ldus @ %u/%u! (%uus)\n",
+						       wrk->id,
+						       slept - do_sleep, count,
+						       i, do_sleep);
+				}
+
  				continue;
  			}
 
Quoted 8 lines
Quoted 7 lines
Signature · 6 lines
2.17.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#7 lore raw

Re: [igt-dev] [Intel-gfx] [PATH i-g-t 04/13] gem_wsim: Check sleep times

Quoting Tvrtko Ursulin (2018-09-07 09:37:00)

Quoted 34 lines · Ville Syrjälä
Quoted 18 lines · Tvrtko Ursulin
Quoted 17 lines

From: Tvrtko Ursulin <[email protected]>

Notice in more places if we are running behind.

---
  benchmarks/gem_wsim.c | 52 ++++++++++++++++++++++++++++++++++++++-----
  1 file changed, 46 insertions(+), 6 deletions(-)
benchmarks/gem_wsim.c +0 -0
index 25af4d678ba4..b05e9760f419 100644
@@ -1718,6 +1718,21 @@ static bool sync_deps(struct workload *wrk, struct w_step *w)
     return synced;
  }
 
Quoted 13 lines

benchmarks/gem_wsim.c static bool sync_deps(struct workload *wrk, struct w_step *w) · line 1731

Quoted 11 lines
benchmarks/gem_wsim.c +11 -0
+static unsigned int measured_usleep(unsigned int usec)
+{
+    struct timespec ts = { };
+    unsigned int slept;
+
+    slept = igt_nsec_elapsed(&ts);
+    igt_assert(slept == 0);
+    do {
+            usleep(usec - slept);
+            slept = igt_nsec_elapsed(&ts) / 1000;
+    } while (slept < usec);

clock_nanosleep(ABS)?

Quoted 9 lines

Hm I think I see what you mean. Rather than a relative sleep trying to hit the loop period, ask from the kernel (or glibc, I don't know who implements it) to sleep until an absolute target. This totally makes sense and would simplify the code from one angle, I am just not sure if absolute sleep can be relied upon any better to not oversleep. Well, actually for scheduling delays not to affect the caller. However maybe it doesn't matter since AFAIR my main problem were dropped period due GPU activity (the first pair of warning messages in the patch), and again AFAIR, it was quite hard to hit the second ones.

Right, it removes the loop but we still want to keep the measurement. -Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#8 lore raw

Re: [Intel-gfx] [igt-dev] [PATH i-g-t 04/13] gem_wsim: Check sleep times

Quoted 45 lines · Chris Wilson

Quoting Tvrtko Ursulin (2018-09-07 09:37:00)

Quoted 34 lines · Ville Syrjälä
Quoted 18 lines · Tvrtko Ursulin
Quoted 17 lines

From: Tvrtko Ursulin <[email protected]>

Notice in more places if we are running behind.

---
  benchmarks/gem_wsim.c | 52 ++++++++++++++++++++++++++++++++++++++-----
  1 file changed, 46 insertions(+), 6 deletions(-)
benchmarks/gem_wsim.c +0 -0
index 25af4d678ba4..b05e9760f419 100644
@@ -1718,6 +1718,21 @@ static bool sync_deps(struct workload *wrk, struct w_step *w)
     return synced;
  }
 
Quoted 13 lines

benchmarks/gem_wsim.c static bool sync_deps(struct workload *wrk, struct w_step *w) · line 1731

Quoted 11 lines
benchmarks/gem_wsim.c +11 -0
+static unsigned int measured_usleep(unsigned int usec)
+{
+    struct timespec ts = { };
+    unsigned int slept;
+
+    slept = igt_nsec_elapsed(&ts);
+    igt_assert(slept == 0);
+    do {
+            usleep(usec - slept);
+            slept = igt_nsec_elapsed(&ts) / 1000;
+    } while (slept < usec);

clock_nanosleep(ABS)?

Quoted 9 lines

Hm I think I see what you mean. Rather than a relative sleep trying to hit the loop period, ask from the kernel (or glibc, I don't know who implements it) to sleep until an absolute target. This totally makes sense and would simplify the code from one angle, I am just not sure if absolute sleep can be relied upon any better to not oversleep. Well, actually for scheduling delays not to affect the caller. However maybe it doesn't matter since AFAIR my main problem were dropped period due GPU activity (the first pair of warning messages in the patch), and again AFAIR, it was quite hard to hit the second ones.

Quoted 1 line

Right, it removes the loop but we still want to keep the measurement.

I guess we still want a loop if we're worried about signals? Not sure why else we'd need a loop anyway. But the loop could be just something like 'while (clock_nanosleep()) ;'

Signature · 6 lines
Ville Syrjälä
Intel
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#9 lore raw

Re: [Intel-gfx] [igt-dev] [PATH i-g-t 04/13] gem_wsim: Check sleep times

Quoted 46 lines · Ville Syrjälä
Quoted 45 lines · Chris Wilson

Quoting Tvrtko Ursulin (2018-09-07 09:37:00)

Quoted 43 lines · Ville Syrjälä
Quoted 18 lines · Tvrtko Ursulin
Quoted 17 lines

From: Tvrtko Ursulin <[email protected]>

Notice in more places if we are running behind.

---
   benchmarks/gem_wsim.c | 52 ++++++++++++++++++++++++++++++++++++++-----
   1 file changed, 46 insertions(+), 6 deletions(-)
benchmarks/gem_wsim.c +0 -0
index 25af4d678ba4..b05e9760f419 100644
@@ -1718,6 +1718,21 @@ static bool sync_deps(struct workload *wrk, struct w_step *w)
      return synced;
   }
 
Quoted 13 lines

benchmarks/gem_wsim.c static bool sync_deps(struct workload *wrk, struct w_step *w) · line 1731

Quoted 11 lines
benchmarks/gem_wsim.c +11 -0
+static unsigned int measured_usleep(unsigned int usec)
+{
+    struct timespec ts = { };
+    unsigned int slept;
+
+    slept = igt_nsec_elapsed(&ts);
+    igt_assert(slept == 0);
+    do {
+            usleep(usec - slept);
+            slept = igt_nsec_elapsed(&ts) / 1000;
+    } while (slept < usec);

clock_nanosleep(ABS)?

Hm I think I see what you mean. Rather than a relative sleep trying to hit the loop period, ask from the kernel (or glibc, I don't know who implements it) to sleep until an absolute target. This totally makes sense and would simplify the code from one angle, I am just not sure if absolute sleep can be relied upon any better to not oversleep. Well, actually for scheduling delays not to affect the caller. However maybe it doesn't matter since AFAIR my main problem were dropped period due GPU activity (the first pair of warning messages in the patch), and again AFAIR, it was quite hard to hit the second ones.

Quoted 5 lines
Quoted 1 line

Right, it removes the loop but we still want to keep the measurement.

I guess we still want a loop if we're worried about signals? Not sure why else we'd need a loop anyway. But the loop could be just something like 'while (clock_nanosleep()) ;'

In this case just a case of copying measured_usleep verbatim from another test because it was easy. Otherwise I think it's fine what you suggest, especially since Chris also complained about using igt helpers from outside tests/. So I'll respin to that effect.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#10 lore raw

PATCH 5/13 gem_wsim: Make workload commands case sensitive

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Need namespace for new commands and I never documented they are case insensitive so it is fine.

---
 benchmarks/gem_wsim.c | 14 +++++++-------
 1 file changed, 7 insertions(+), 7 deletions(-)
benchmarks/gem_wsim.c +7 -7
index b05e9760f419..fe9535df75c6 100644
@@ -310,7 +310,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 		if ((field = strtok_r(fstart, ".", &fctx)) != NULL) {
 			fstart = NULL;
 
-			if (!strcasecmp(field, "d")) {
+			if (!strcmp(field, "d")) {
 				if ((field = strtok_r(fstart, ".", &fctx)) !=
 				    NULL) {
 					tmp = atoi(field);
@@ -326,7 +326,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 					step.delay = tmp;
 					goto add_step;
 				}
-			} else if (!strcasecmp(field, "p")) {
+			} else if (!strcmp(field, "p")) {
 				if ((field = strtok_r(fstart, ".", &fctx)) !=
 				    NULL) {
 					tmp = atoi(field);
@@ -342,7 +342,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 					step.period = tmp;
 					goto add_step;
 				}
-			} else if (!strcasecmp(field, "s")) {
+			} else if (!strcmp(field, "s")) {
 				if ((field = strtok_r(fstart, ".", &fctx)) !=
 				    NULL) {
 					tmp = atoi(field);
@@ -359,7 +359,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 					step.target = tmp;
 					goto add_step;
 				}
-			} else if (!strcasecmp(field, "t")) {
+			} else if (!strcmp(field, "t")) {
 				if ((field = strtok_r(fstart, ".", &fctx)) !=
 				    NULL) {
 					tmp = atoi(field);
@@ -375,7 +375,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 					step.throttle = tmp;
 					goto add_step;
 				}
-			} else if (!strcasecmp(field, "q")) {
+			} else if (!strcmp(field, "q")) {
 				if ((field = strtok_r(fstart, ".", &fctx)) !=
 				    NULL) {
 					tmp = atoi(field);
@@ -391,7 +391,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 					step.throttle = tmp;
 					goto add_step;
 				}
-			} else if (!strcasecmp(field, "a")) {
+			} else if (!strcmp(field, "a")) {
 				if ((field = strtok_r(fstart, ".", &fctx)) !=
 				    NULL) {
 					tmp = atoi(field);
@@ -407,7 +407,7 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 					step.target = tmp;
 					goto add_step;
 				}
-			} else if (!strcasecmp(field, "f")) {
+			} else if (!strcmp(field, "f")) {
 				step.type = SW_FENCE;
 				goto add_step;
 			}
Signature · 6 lines
2.17.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#11 lore raw

PATCH 6/13 gem_wsim: Context priority support

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

A new workload command ('P') is added which enables per context dynamic priority control.

---
 benchmarks/gem_wsim.c  | 48 +++++++++++++++++++++++++++++++++++++++++-
 benchmarks/wsim/README | 18 ++++++++++++++++
 2 files changed, 65 insertions(+), 1 deletion(-)
benchmarks/gem_wsim.c +47 -1
index fe9535df75c6..0c52ec567430 100644
@@ -82,7 +82,8 @@ enum w_type
 	THROTTLE,
 	QD_THROTTLE,
 	SW_FENCE,
-	SW_FENCE_SIGNAL
+	SW_FENCE_SIGNAL,
+	CTX_PRIORITY
 };
 
 struct deps
@@ -114,6 +115,7 @@ struct w_step
 		int target;
 		int throttle;
 		int fence_signal;
+		int priority;
 	};
 
 	/* Implementation details */
@@ -162,6 +164,7 @@ struct workload
 	unsigned int nr_ctxs;
 	struct {
 		uint32_t id;
+		int priority;
 		unsigned int static_vcs;
 	} *ctx_list;
 
@@ -342,6 +345,36 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 					step.period = tmp;
 					goto add_step;
 				}
+			} else if (!strcmp(field, "P")) {
+				unsigned int nr = 0;
+				while ((field = strtok_r(fstart, ".", &fctx)) !=
+				    NULL) {
+					tmp = atoi(field);
+					if (tmp <= 0 && nr == 0) {
+						if (verbose)
+							fprintf(stderr,
+								"Invalid context at step %u!\n",
+								nr_steps);
+						return NULL;
+					}
+
+					if (nr == 0) {
+						step.context = tmp;
+					} else if (nr == 1) {
+						step.priority = tmp;
+					} else {
+						if (verbose)
+							fprintf(stderr,
+								"Invalid priority format at step %u!\n",
+								nr_steps);
+						return NULL;
+					}
+
+					nr++;
+				}
+
+				step.type = CTX_PRIORITY;
+				goto add_step;
 			} else if (!strcmp(field, "s")) {
 				if ((field = strtok_r(fstart, ".", &fctx)) !=
 				    NULL) {
@@ -1806,6 +1839,19 @@ static void *run_workload(void *data)
 				inc = cur_seqno - wrk->sync_seqno;
 				sw_sync_timeline_inc(wrk->sync_timeline, inc);
 				continue;
+			} else if (w->type == CTX_PRIORITY) {
+				if (w->priority != wrk->ctx_list[w->context].priority) {
+					struct drm_i915_gem_context_param param = {
+						.ctx_id = wrk->ctx_list[w->context].id,
+						.param = I915_CONTEXT_PARAM_PRIORITY,
+						.value = w->priority,
+					};
+
+					gem_context_set_param(fd, &param);
+					wrk->ctx_list[w->context].priority =
+								    w->priority;
+				}
+				continue;
 			}
 
 			if (do_sleep || w->type == PERIOD) {
benchmarks/wsim/README +18 -0
index 01766d37564f..fd9bff3401af 100644
@@ -3,6 +3,7 @@ Workload descriptor format
 
 ctx.engine.duration_us.dependency.wait,...
 <uint>.<str>.<uint>[-<uint>].<int <= 0>[/<int <= 0>][...].<0|1>,...
+P.<uint>.<int>
 d|p|s|t|q|a.<int>,...
 f
 
@@ -22,6 +23,7 @@ Additional workload steps are also supported:
  'q' - Throttle to n max queue depth.
  'f' - Create a sync fence.
  'a' - Advance the previously created sync fence.
+ 'P' - Context priority.
 
 Engine ids: RCS, BCS, VCS, VCS1, VCS2, VECS
 
@@ -110,3 +112,19 @@ created at the second step. They are submitted ahead of time while still not
 runnable. When the second RCS batch completes the standalone fence is signaled
 which allows the two VCS batches to be executed. Finally we wait until the both
 VCS batches have completed before starting the (optional) next iteration.
+
+Context priority
+----------------
+
+  P.1.-1
+  1.RCS.1000.0.0
+  P.2.1
+  2.BCS.1000.-2.0
+
+Context 1 is marked as low priority (-1) and then a batch buffer is submitted
+against it. Context 2 is marked as high priority (1) and then a batch buffer
+is submitted against it which depends on the batch from context 1.
+
+Context priority command is executed at workload runtime and is valid until
+overriden by another (optional) same context priority change. Actual driver
+ioctls are executed only if the priority level has changed for the context.
Signature · 6 lines
2.17.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#12 lore raw

PATCH 7/13 gem_wsim: Make batches preemptable by default

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

MI_NOOP cannot be preempted which means up to now gem_wsim workloads were preemptable on batch buffer granularity only.

Add MI_ARB_CHK every 100us so the new default is mid-batch preemption.

---
 benchmarks/gem_wsim.c | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
benchmarks/gem_wsim.c +20 -0
index 0c52ec567430..7f2d0c83a0e7 100644
@@ -685,6 +685,25 @@ static unsigned long get_bb_sz(unsigned int duration)
 		     nop_calibration_us, sizeof(uint32_t));
 }
 
+static void
+init_bb(struct w_step *w, unsigned int flags)
+{
+	/* Preemption point every 100us. */
+	const unsigned int arb_period = get_bb_sz(100) / sizeof(uint32_t);
+	unsigned int i;
+	uint32_t *ptr;
+
+	gem_set_domain(fd, w->bb_handle,
+		       I915_GEM_DOMAIN_WC, I915_GEM_DOMAIN_WC);
+
+	ptr = gem_mmap__wc(fd, w->bb_handle, 0, w->bb_sz, PROT_WRITE);
+
+	for (i = arb_period; i < w->bb_sz / sizeof(uint32_t); i += arb_period)
+		ptr[i] = 0x5 << 23; /* MI_ARB_CHK */
+
+	munmap(ptr, w->bb_sz);
+}
+
 static void
 terminate_bb(struct w_step *w, unsigned int flags)
 {
@@ -839,6 +858,7 @@ alloc_step_batch(struct workload *wrk, struct w_step *w, unsigned int flags)
 
 	w->bb_sz = get_bb_sz(w->duration.max);
 	w->bb_handle = w->obj[j].handle = gem_create(fd, w->bb_sz);
+	init_bb(w, flags);
 	terminate_bb(w, flags);
 
 	if (flags & SEQNO) {
Signature · 6 lines
2.17.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#13 lore raw

Re: [igt-dev] [PATH i-g-t 07/13] gem_wsim: Make batches preemptable by default

Quoting Tvrtko Ursulin (2018-09-05 14:49:33)

Quoted 6 lines

From: Tvrtko Ursulin <[email protected]>

MI_NOOP cannot be preempted which means up to now gem_wsim workloads were preemptable on batch buffer granularity only.

Add MI_ARB_CHK every 100us so the new default is mid-batch preemption.

Sure you don't want this configurable to adjust between different worklods? I'm just thinking that some of the fixed function stuff may have vastly different preemption granularity.
-Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#14 lore raw

PATCH 8/13 gem_wsim: Per context preemption point control

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Allow workloads to specify frequency of preemption points per context.

New workload command ('X') is added to allow this.

---
 benchmarks/gem_wsim.c  | 84 ++++++++++++++++++++++++++++++++++++++++--
 benchmarks/wsim/README | 18 ++++++++-
 2 files changed, 98 insertions(+), 4 deletions(-)
benchmarks/gem_wsim.c +81 -3
index 7f2d0c83a0e7..6b128a1e9d57 100644
@@ -83,7 +83,8 @@ enum w_type
 	QD_THROTTLE,
 	SW_FENCE,
 	SW_FENCE_SIGNAL,
-	CTX_PRIORITY
+	CTX_PRIORITY,
+	PREEMPTION
 };
 
 struct deps
@@ -122,6 +123,7 @@ struct w_step
 	unsigned int idx;
 	struct igt_list rq_link;
 	unsigned int request;
+	unsigned int preempt_us;
 
 	struct drm_i915_gem_execbuffer2 eb;
 	struct drm_i915_gem_exec_object2 *obj;
@@ -443,6 +445,42 @@ parse_workload(struct w_arg *arg, unsigned int flags, struct workload *app_w)
 			} else if (!strcmp(field, "f")) {
 				step.type = SW_FENCE;
 				goto add_step;
+			} else if (!strcmp(field, "X")) {
+				unsigned int nr = 0;
+				while ((field = strtok_r(fstart, ".", &fctx)) !=
+				    NULL) {
+					tmp = atoi(field);
+					if (tmp <= 0 && nr == 0) {
+						if (verbose)
+							fprintf(stderr,
+								"Invalid context at step %u!\n",
+								nr_steps);
+						return NULL;
+					} else if (tmp < 0 && nr == 1) {
+						if (verbose)
+							fprintf(stderr,
+								"Invalid preemption period at step %u!\n",
+								nr_steps);
+						return NULL;
+					}
+
+					if (nr == 0) {
+						step.context = tmp;
+					} else if (nr == 1) {
+						step.period = tmp;
+					} else {
+						if (verbose)
+							fprintf(stderr,
+								"Invalid preemption format at step %u!\n",
+								nr_steps);
+						return NULL;
+					}
+
+					nr++;
+				}
+
+				step.type = PREEMPTION;
+				goto add_step;
 			}
 
 			tmp = atoi(field);
@@ -688,11 +726,14 @@ static unsigned long get_bb_sz(unsigned int duration)
 static void
 init_bb(struct w_step *w, unsigned int flags)
 {
-	/* Preemption point every 100us. */
-	const unsigned int arb_period = get_bb_sz(100) / sizeof(uint32_t);
+	const unsigned int arb_period =
+			get_bb_sz(w->preempt_us) / sizeof(uint32_t);
 	unsigned int i;
 	uint32_t *ptr;
 
+	if (!arb_period)
+		return;
+
 	gem_set_domain(fd, w->bb_handle,
 		       I915_GEM_DOMAIN_WC, I915_GEM_DOMAIN_WC);
 
@@ -968,6 +1009,41 @@ prepare_workload(unsigned int id, struct workload *wrk, unsigned int flags)
 		}
 	}
 
+	/* Record default preemption. */
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		if (w->type == BATCH)
+			w->preempt_us = 100;
+	}
+
+	/*
+	 * Scan for contexts with modified preemption config and record their
+	 * preemption period for the following steps belonging to the same
+	 * context.
+	 */
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		struct w_step *w2;
+		int j;
+
+		if (w->type != PREEMPTION)
+			continue;
+
+		for (j = i + 1; j < wrk->nr_steps; j++) {
+			w2 = &wrk->steps[j];
+
+			if (w2->context != w->context)
+				continue;
+			else if (w2->type == PREEMPTION)
+				break;
+			else if (w2->type != BATCH)
+				continue;
+
+			w2->preempt_us = w->period;
+		}
+	}
+
+	/*
+	 * Allocate batch buffers.
+	 */
 	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
 		unsigned int _flags = flags;
 		enum intel_engine_id engine = w->engine;
@@ -1872,6 +1948,8 @@ static void *run_workload(void *data)
 								    w->priority;
 				}
 				continue;
+			} else if (w->type == PREEMPTION) {
+				continue;
 			}
 
 			if (do_sleep || w->type == PERIOD) {
benchmarks/wsim/README +17 -1
index fd9bff3401af..205cd6c93afb 100644
@@ -3,7 +3,7 @@ Workload descriptor format
 
 ctx.engine.duration_us.dependency.wait,...
 <uint>.<str>.<uint>[-<uint>].<int <= 0>[/<int <= 0>][...].<0|1>,...
-P.<uint>.<int>
+P|X.<uint>.<int>
 d|p|s|t|q|a.<int>,...
 f
 
@@ -24,6 +24,7 @@ Additional workload steps are also supported:
  'f' - Create a sync fence.
  'a' - Advance the previously created sync fence.
  'P' - Context priority.
+ 'X' - Context preemption control.
 
 Engine ids: RCS, BCS, VCS, VCS1, VCS2, VECS
 
@@ -128,3 +129,18 @@ is submitted against it which depends on the batch from context 1.
 Context priority command is executed at workload runtime and is valid until
 overriden by another (optional) same context priority change. Actual driver
 ioctls are executed only if the priority level has changed for the context.
+
+Context preemption control
+--------------------------
+
+  X.1.0
+  1.RCS.1000.0.0
+  X.1.500
+  1.RCS.1000.0.0
+
+Context 1 is marked as non-preemptable batches and a batch is sent against 1.
+The same context is then marked to have batches which can be preempted every
+500us and another batch is submitted.
+
+Same as with context priority, context preemption commands are valid until
+optionally overriden by another preemption control change on the same context.
Signature · 6 lines
2.17.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#15 lore raw

Re: [igt-dev] [PATH i-g-t 08/13] gem_wsim: Per context preemption point control

Quoting Tvrtko Ursulin (2018-09-05 14:49:34)

Quoted 5 lines

From: Tvrtko Ursulin <[email protected]>

Allow workloads to specify frequency of preemption points per context.

New workload command ('X') is added to allow this.

Called it! ;)
-Chris
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

Reviewed-by Acked-by
#16 lore raw

Re: [Intel-gfx] [igt-dev] [PATH i-g-t 08/13] gem_wsim: Per context preemption point control

Quoting Chris Wilson (2018-09-07 09:49:09)

Quoted 8 lines

Quoting Tvrtko Ursulin (2018-09-05 14:49:34)

Quoted 5 lines

From: Tvrtko Ursulin <[email protected]>

Allow workloads to specify frequency of preemption points per context.

New workload command ('X') is added to allow this.

Called it! ;)

With muttering under my breath about .wsim, you can have a

for all the gem_wsim bits. All the .pl updates I read, look fine, but I haven't taken the time to contemplate if they are correct, so for the rest of the series,

-Chris
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#17 lore raw

PATCH 9/13 media-bench: Update for engine=class:instance tracepoints

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

---
 scripts/media-bench.pl | 8 ++++----
 1 file changed, 4 insertions(+), 4 deletions(-)
scripts/media-bench.pl +4 -4
index 375844d9cdf6..a3619ceb34cc 100755
@@ -187,7 +187,7 @@ sub trace_workload
 	open CMD, "$cmd |" or die;
 	while (<CMD>) {
 		chomp;
-		if (/Ring(\d+): (\d+) batches.*?(\d+\.?\d+)% idle,/) {
+		if (/Ring(\S+): (\d+) batches.*?(\d+\.?\d+)% idle,/) {
 			if ($2 >= $min_batches) {
 				$engines{$1} = $3;
 			} else {
@@ -542,7 +542,7 @@ foreach my $wrk (@workloads) {
 	my ($r, $error, $c, $wps, $swps);
 	my $saturated = 0;
 	my $result = 'Pass';
-	my $vcs2 = $gt2 ? '1' : '3';
+	my $vcs2 = $gt2 ? '1:0' : '2:1';
 	my %problem;
 	my $engines;
 
@@ -575,11 +575,11 @@ foreach my $wrk (@workloads) {
 	if ($saturated == 0) {
 		# Not a single saturated engine
 		$result = 'FAIL';
-	} elsif (not exists $engines->{'2'} or not exists $engines->{$vcs2}) {
+	} elsif (not exists $engines->{'2:0'} or not exists $engines->{$vcs2}) {
 		# VCS1 and VCS2 not present in a balancing workload
 		$result = 'FAIL';
 	} elsif ($saturated == 1 and
-		 ($engines->{'2'} < $idle_tolerance_pct or
+		 ($engines->{'2:0'} < $idle_tolerance_pct or
 		  $engines->{$vcs2} < $idle_tolerance_pct)) {
 		# Only one VCS saturated
 		$result = 'WARN';
Signature · 6 lines
2.17.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#18 lore raw

PATCH 10/13 media-bench: Protect against incorrect -b syntax

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

-b is to pass the command argument directly to gem_wsim so must include another -b.

---
 scripts/media-bench.pl | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
scripts/media-bench.pl +4 -1
index a3619ceb34cc..5070cae088f3 100755
@@ -341,7 +341,10 @@ ENDHELP
 $verbose = 1 if defined $opts{'v'};
 $gt2 = 1 if defined $opts{'2'};
 $show_cmds = 1 if defined $opts{'x'};
-$balancer = $opts{'b'} if defined $opts{'b'};
+if (defined $opts{'b'}) {
+	die unless substr($opts{'b'}, 0, 2) eq '-b';
+	$balancer = $opts{'b'};
+}
 if (defined $opts{'B'}) {
 	@balancers = split /,/, $opts{'B'};
 } else {
Signature · 6 lines
2.17.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#19 lore raw

PATCH 11/13 media-bench: Fix tracing of direct workloads

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Build argument list properly and check exit codes when executing sub-commands.

---
 scripts/media-bench.pl | 54 +++++++++++++++++++++++++++---------------
 1 file changed, 35 insertions(+), 19 deletions(-)
scripts/media-bench.pl +35 -19
index 5070cae088f3..9bf8c8789fe2 100755
@@ -131,7 +131,8 @@ sub add_wps_arg
 	return @args if $realtime_target <= 0;
 
 	$period = int(1000000 / $realtime_target);
-	push @args, "-a p.$period";
+	push @args, '-a';
+	push @args, 'p.$period';
 
 	return @args;
 }
@@ -144,7 +145,7 @@ sub run_workload
 	@args = add_wps_arg(@args);
 	push @args, '-2' if $gt2;
 
-	unshift @args, "$wsim";
+	unshift @args, $wsim;
 	$cmd = join ' ', @args;
 	show_cmd($cmd);
 
@@ -164,23 +165,31 @@ sub run_workload
 sub trace_workload
 {
 	my ($wrk, $b, $r, $c) = @_;
-	my @args = ( "-n $nop", "-r $r", "-c $c");
+	my @args = ($tracepl, '--trace', $wsim, '-q', '-n', $nop, '-r', $r, '-c', $c);
 	my $min_batches = 16 + $r * $c / 2;
 	my @skip_engine;
 	my %engines;
 	my ($cmd, $file);
-	my $warg = defined $w_direct ? $wrk : "-w $wrk_root/$wrk";
 
-	push @args, "$b -R" unless $b eq '<none>';
 	push @args, '-2' if $gt2;
-	push @args, $warg;
 
-	unshift @args, '-q';
-	unshift @args, "$tracepl --trace $wsim";
+	unless ($b eq '<none>') {
+		push @args, '-R';
+		push @args, split /\s+/, $b;
+	}
 
-	$cmd = join ' ', @args;
-	show_cmd($cmd);
-	system($cmd);
+	if (defined $w_direct) {
+		push @args, split /\s+/, $wrk;
+	} else {
+		push @args, '-w';
+		push @args, $wrk_root . '/' . $wrk;
+	}
+
+	show_cmd(join ' ', @args);
+	if (-e 'perf.data') {
+		unlink 'perf.data' or die;
+	}
+	system(@args) == 0 or die;
 
 	$cmd = "perf script | $tracepl";
 	show_cmd($cmd);
@@ -205,13 +214,13 @@ sub trace_workload
 
 	$cmd = "perf script > ${file}.trace";
 	show_cmd($cmd);
-	system($cmd);
+	system($cmd) == 0 or die;
 
 	$cmd = "perf script | $tracepl --html -x ctxsave -s -c ";
 	$cmd .= join ' ', map("-i $_", @skip_engine);
 	$cmd .= " > ${file}.html";
 	show_cmd($cmd);
-	system($cmd);
+	system($cmd) == 0 or die;
 
 	return \%engines;
 }
@@ -219,7 +228,6 @@ sub trace_workload
 sub calibrate_workload
 {
 	my ($wrk) = @_;
-	my $warg = defined $w_direct ? $wrk : "-w $wrk_root/$wrk";
 	my $tol = $tolerance;
 	my $loops = 0;
 	my $error;
@@ -227,9 +235,16 @@ sub calibrate_workload
 
 	$r = $realtime_target > 0 ? $realtime_target * $client_target_s : 23;
 	for (;;) {
-		my @args = ( "-n $nop", "-r $r", $warg);
+		my @args = ('-n', $nop, '-r', $r);
 		my ($time, $wps);
 
+		if (defined $w_direct) {
+			push @args, split /\s+/, $wrk;
+		} else {
+			push @args, '-w';
+			push @args, $wrk_root . '/' . $wrk;
+		}
+
 		($time, $wps) = run_workload(@args);
 
 		$error = abs($time - $client_target_s) / $client_target_s;
@@ -254,23 +269,24 @@ sub find_saturation_point
 	my ($last_wps, $c, $swps);
 	my $target = $realtime_target > 0 ? $realtime_target : $wps_target;
 	my $r = $rr;
-	my ($warg, $wcnt);
+	my $wcnt;
 	my $maxc;
 	my $max = 0;
 
 	if (defined $w_direct) {
-		$warg = $wrk;
+		push @args, split /\s+/, $wrk;
 		$wcnt = () = $wrk =~ /-[wW]/gi;
 
 	} else {
-		$warg = "-w $wrk_root/$wrk";
+		push @args, '-w';
+		push @args, $wrk_root . '/' . $wrk;
 		$wcnt = 1;
 	}
 
 	for ($c = 1; ; $c = $c + 1) {
 		my ($time, $wps);
 
-		($time, $wps) = run_workload((@args, ($warg, "-r $r", "-c $c")));
+		($time, $wps) = run_workload((@args, ('-r', $r, '-c', $c)));
 
 		say "        $c clients is $wps wps." if $verbose;
 
Signature · 6 lines
2.17.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#20 lore raw

PATCH 12/13 media-bench: Write out trace files directly.

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Instead of relying on shell redirection.

---
 scripts/media-bench.pl | 24 ++++++++++++++++++------
 1 file changed, 18 insertions(+), 6 deletions(-)
scripts/media-bench.pl +18 -6
index 9bf8c8789fe2..77e75c78b113 100755
@@ -162,6 +162,21 @@ sub run_workload
 	return ($time, $wps);
 }
 
+sub dump_cmd
+{
+	my ($cmd, $file) = @_;
+
+	show_cmd("$cmd > $file");
+
+	open FOUT, '>', $file or die;
+	open TIN, "$cmd |" or die;
+	while (<TIN>) {
+		print FOUT $_;
+	}
+	close TIN;
+	close FOUT;
+}
+
 sub trace_workload
 {
 	my ($wrk, $b, $r, $c) = @_;
@@ -212,15 +227,12 @@ sub trace_workload
 	$b =~ s/[ <>]/_/g;
 	$file = "${wrk}_${b}_-r${r}_-c${c}";
 
-	$cmd = "perf script > ${file}.trace";
-	show_cmd($cmd);
-	system($cmd) == 0 or die;
+	dump_cmd('perf script', "${file}.trace");
 
 	$cmd = "perf script | $tracepl --html -x ctxsave -s -c ";
 	$cmd .= join ' ', map("-i $_", @skip_engine);
-	$cmd .= " > ${file}.html";
-	show_cmd($cmd);
-	system($cmd) == 0 or die;
+
+	dump_cmd($cmd, "${file}.html");
 
 	return \%engines;
 }
Signature · 6 lines
2.17.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#21 lore raw

PATCH 13/13 media-bench: Add mixed mode evaluation

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Mixed mode (-m) enables evaluation of different workload sets against one or more load balancing strategies.

Contrary to the default mode which runs all selected workloads serialy, mixed mode runs a second stage where they are all run in parallel. The performance difference between the two passes is then used for the scoring metric.

First metric is the normalized aggregate throughput, and second is balancer "fairness" as approximated by throughput achieved in mixed mode, relative to the best individual balancer for each workload.

---
 scripts/media-bench.pl | 150 +++++++++++++++++++++++++++++++++--------
 1 file changed, 121 insertions(+), 29 deletions(-)
scripts/media-bench.pl +121 -29
index 77e75c78b113..066b542f95df 100755
@@ -41,6 +41,8 @@ my $show_cmds = 0;
 my $realtime_target = 0;
 my $wps_target = 0;
 my $wps_target_param = 0;
+my $multi_mode = 0;
+my @multi_workloads;
 my $w_direct;
 my $balancer;
 my $nop;
@@ -141,6 +143,7 @@ sub run_workload
 {
 	my (@args) = @_;
 	my ($time, $wps, $cmd);
+	my @ret;
 
 	@args = add_wps_arg(@args);
 	push @args, '-2' if $gt2;
@@ -155,11 +158,13 @@ sub run_workload
 		if (/^(\d+\.\d+)s elapsed \((\d+\.?\d+) workloads\/s\)$/) {
 			$time = $1;
 			$wps = $2;
+		} elsif (/(\d+)\: \d+\.\d+s elapsed \(\d+ cycles, (\d+\.?\d+) workloads\/s\)/) {
+			$ret[$1] = $2;
 		}
 	}
 	close WSIM;
 
-	return ($time, $wps);
+	return ($time, $wps, \@ret);
 }
 
 sub dump_cmd
@@ -223,7 +228,11 @@ sub trace_workload
 	}
 	close CMD;
 
-	$wrk =~ s/ /_/g;
+	$wrk =~ s/$wrk_root//g;
+	$wrk =~ s/\.wsim//g;
+	$wrk =~ s/-w/W/g;
+	$wrk =~ s/[ -]/_/g;
+	$wrk =~ s/\//-/g;
 	$b =~ s/[ <>]/_/g;
 	$file = "${wrk}_${b}_-r${r}_-c${c}";
 
@@ -259,6 +268,7 @@ sub calibrate_workload
 
 		($time, $wps) = run_workload(@args);
 
+		$wps = $r / $time if $w_direct;
 		$error = abs($time - $client_target_s) / $client_target_s;
 
 		last if $error <= $tol;
@@ -278,13 +288,15 @@ sub calibrate_workload
 sub find_saturation_point
 {
 	my ($wrk, $rr, $verbose, @args) = @_;
-	my ($last_wps, $c, $swps);
+	my ($last_wps, $c, $swps, $wwps);
 	my $target = $realtime_target > 0 ? $realtime_target : $wps_target;
 	my $r = $rr;
 	my $wcnt;
 	my $maxc;
 	my $max = 0;
 
+	push @args, '-v' if $multi_mode and $w_direct;
+
 	if (defined $w_direct) {
 		push @args, split /\s+/, $wrk;
 		$wcnt = () = $wrk =~ /-[wW]/gi;
@@ -297,8 +309,9 @@ sub find_saturation_point
 
 	for ($c = 1; ; $c = $c + 1) {
 		my ($time, $wps);
+		my @args_ = (@args, ('-r', $r, '-c', $c));
 
-		($time, $wps) = run_workload((@args, ('-r', $r, '-c', $c)));
+		($time, $wps, $wwps) = run_workload(@args_);
 
 		say "        $c clients is $wps wps." if $verbose;
 
@@ -324,21 +337,23 @@ sub find_saturation_point
 			$r = int($rr * ($client_target_s / $time));
 		} elsif ($c == 1) {
 			$swps = $wps;
-			return ($c, $wps, $swps) if $wcnt > 1 or
-						    ($wps_target_param < 0 and $wps_target == 0);
+			return ($c, $wps, $swps, $wwps) if $wcnt > 1 or
+							   $multi_mode or
+							   ($wps_target_param < 0 and
+							    $wps_target == 0);
 		}
 
 		$last_wps = $wps;
 	}
 
 	if ($target <= 0) {
-		return ($maxc, $max, $swps);
+		return ($maxc, $max, $swps, $wwps);
 	} else {
-		return ($c - 1, $last_wps, $swps);
+		return ($c - 1, $last_wps, $swps, $wwps);
 	}
 }
 
-getopts('hv2xn:b:W:B:r:t:i:R:T:w:', \%opts);
+getopts('hv2xmn:b:W:B:r:t:i:R:T:w:', \%opts);
 
 if (defined $opts{'h'}) {
 	print <<ENDHELP;
@@ -360,8 +375,11 @@ Supported options:
   -R wps      Run workloads in the real-time mode at wps rate.
   -T wps      Calibrate up to wps/client target instead of GPU saturation.
               Negative values set the target based on the single client
-	      performance where target = single-client-wps / -N.
+              performance where target = single-client-wps / -N.
   -w str      Pass-through to gem_wsim. Overrides normal workload selection.
+  -m          Multi-workload mode. All selected workloads will be run in
+              parallel and overal score will be relative to when run
+              individually.
 ENDHELP
 	exit 0;
 }
@@ -369,6 +387,7 @@ ENDHELP
 $verbose = 1 if defined $opts{'v'};
 $gt2 = 1 if defined $opts{'2'};
 $show_cmds = 1 if defined $opts{'x'};
+$multi_mode = 1 if defined $opts{'m'};
 if (defined $opts{'b'}) {
 	die unless substr($opts{'b'}, 0, 2) eq '-b';
 	$balancer = $opts{'b'};
@@ -387,7 +406,13 @@ $wps_target = $opts{'T'} if defined $opts{'T'};
 $wps_target_param = $wps_target;
 $w_direct = $opts{'w'} if defined $opts{'w'};
 
-@workloads =  ($w_direct ) if defined $w_direct;
+if ($multi_mode) {
+	die if $w_direct; # Not supported
+	@multi_workloads = @workloads;
+}
+
+@workloads = ($w_direct) if defined $w_direct;
+
 say "Workloads:";
 print map { "  $_\n" } @workloads;
 print "Balancers: ";
@@ -396,13 +421,14 @@ say "Target workload duration is ${client_target_s}s.";
 say "Calibration tolerance is $tolerance.";
 say "Real-time mode at ${realtime_target} wps." if $realtime_target > 0;
 say "Wps target is ${wps_target} wps." if $wps_target > 0;
+say "Multi-workload mode." if $multi_mode;
 $nop = $opts{'n'};
 $nop = calibrate_nop() unless $nop;
 say "Nop calibration is $nop.";
 
 goto VERIFY if defined $balancer;
 
-my %best_bal;
+my (%best_bal, %best_bid);
 my %results;
 my %scores;
 my %wscores;
@@ -438,13 +464,22 @@ sub add_points
 	}
 }
 
-foreach my $wrk (@workloads) {
+my @saturation_workloads = $multi_mode ? @multi_workloads : @workloads;
+my %allwps;
+my $widx = 0;
+
+push @saturation_workloads, '-w ' . join ' -w ', map("$wrk_root/$_", @workloads)
+     if $multi_mode;
+
+foreach my $wrk (@saturation_workloads) {
 	my @args = ( "-n $nop");
 	my ($r, $error, $should_b, $best);
 	my (%wps, %cwps, %mwps);
 	my @sorted;
 	my $range;
 
+	$w_direct = $wrk if $multi_mode and $widx == $#saturation_workloads;
+
 	$should_b = 1;
 	$should_b = can_balance_workload($wrk) unless defined $w_direct;
 
@@ -459,7 +494,7 @@ foreach my $wrk (@workloads) {
 		GBAL: foreach my $G ('', '-G', '-d', '-G -d') {
 			foreach my $H ('', '-H') {
 				my @xargs;
-				my ($w, $c, $s);
+				my ($w, $c, $s, $bwwps);
 				my $bid;
 
 				if ($bal ne '') {
@@ -476,21 +511,67 @@ foreach my $wrk (@workloads) {
 
 				$wps_target = 0 if $wps_target_param < 0;
 
-				($c, $w, $s) = find_saturation_point($wrk, $r,
-								     0,
-								     (@args,
-								      @xargs));
+				($c, $w, $s, $bwwps) =
+					find_saturation_point($wrk, $r, 0,
+							      (@args, @xargs));
 
 				if ($wps_target_param < 0) {
 					$wps_target = $s / -$wps_target_param;
 
-					($c, $w, $s) =
+					($c, $w, $s, $bwwps) =
 						find_saturation_point($wrk, $r,
 								      0,
 								      (@args,
 								       @xargs));
 				}
 
+				if ($multi_mode and $w_direct) {
+					my $widx;
+
+					die unless scalar(@multi_workloads) ==
+						   scalar(@{$bwwps});
+					die unless scalar(@multi_workloads) ==
+						   scalar(keys %allwps);
+
+					# Total of all workload wps from the
+					# mixed run.
+					$w = 0;
+					foreach $widx (0..$#{$bwwps}) {
+						$w += $bwwps->[$widx];
+					}
+
+					# Total of all workload wps from when
+					# ran individually with the best
+					# balancer.
+					my $tot = 0;
+					foreach my $wrk (@multi_workloads) {
+						$tot += $allwps{$wrk}->{$best_bid{$wrk}};
+					}
+
+					# Normalize mixed sum with sum of
+					# individual runs.
+					$w *= 100;
+					$w /= $tot;
+
+					# Second metric is average of each
+					# workload wps normalized by their
+					# individual run performance with the
+					# best balancer.
+					$s = 0;
+					$widx = 0;
+					foreach my $wrk (@multi_workloads) {
+						$s += 100 * $bwwps->[$widx] /
+						      $allwps{$wrk}->{$best_bid{$wrk}};
+						$widx++;
+					}
+					$s /= scalar(@multi_workloads);
+
+					say sprintf('Aggregate (normalized) %.2f%%; fairness %.2f%%',
+						    $w, $s);
+				} else {
+					$allwps{$wrk} = \%wps;
+				}
+
 				$wps{$bid} = $w;
 				$cwps{$bid} = $s;
 
@@ -500,7 +581,8 @@ foreach my $wrk (@workloads) {
 					$mwps{$bid} = $w + $s;
 				}
 
-				say "$c clients ($w wps, $s wps single client, score=$mwps{$bid}).";
+				say "$c clients ($w wps, $s wps single client, score=$mwps{$bid})."
+				    unless $multi_mode and $w_direct;
 
 				last BAL unless $should_b;
 				next BAL if $bal eq '';
@@ -509,19 +591,24 @@ foreach my $wrk (@workloads) {
 		}
 	}
 
+	$widx++;
+
 	@sorted = sort { $mwps{$b} <=> $mwps{$a} } keys %mwps;
-	$best = $sorted[0];
+	$best_bid{$wrk} = $sorted[0];
 	@sorted = sort { $b <=> $a } values %mwps;
 	$range = 1 - $sorted[-1] / $sorted[0];
 	$best_bal{$wrk} = $sorted[0];
-	say "  Best balancer is '$best' (range=$range).";
+
+	next if $multi_mode and not $w_direct;
+
+	say "  Best balancer is '$best_bid{$wrk}' (range=$range).";
 
 
 	$results{$wrk} = \%mwps;
 
 	add_points(\%wps, \%scores, \%wscores);
-	add_points(\%cwps, \%cscores, \%cwscores);
 	add_points(\%mwps, \%mscores, \%mwscores);
+	add_points(\%cwps, \%cscores, \%cwscores);
 }
 
 sub dump_scoreboard
@@ -552,12 +639,12 @@ sub dump_scoreboard
 	return $balancer;
 }
 
-dump_scoreboard('Total wps', \%scores);
-dump_scoreboard('Total weighted wps', \%wscores);
-dump_scoreboard('Per client wps', \%cscores);
-dump_scoreboard('Per client weighted wps', \%cwscores);
-dump_scoreboard('Combined wps', \%mscores);
-$balancer = dump_scoreboard('Combined weighted wps', \%mwscores);
+dump_scoreboard($multi_mode ? 'Throughput' : 'Total wps', \%scores);
+dump_scoreboard('Total weighted wps', \%wscores) unless $multi_mode;
+dump_scoreboard($multi_mode ? 'Fairness' : 'Per client wps', \%cscores);
+dump_scoreboard('Per client weighted wps', \%cwscores) unless $multi_mode;
+$balancer = dump_scoreboard($multi_mode ? 'Combined' : 'Combined wps', \%mscores);
+$balancer = dump_scoreboard('Combined weighted wps', \%mwscores) unless $multi_mode;
 
 VERIFY:
 
@@ -568,6 +655,11 @@ die unless defined $balancer;
 say "\nBalancer is '$balancer'.";
 say "Idleness tolerance is $idle_tolerance_pct%.";
 
+if ($multi_mode) {
+	$w_direct = '-w ' . join ' -w ', map("$wrk_root/$_", @workloads);
+	@workloads = ($w_direct);
+}
+
 foreach my $wrk (@workloads) {
 	my @args = ( "-n $nop" );
 	my ($r, $error, $c, $wps, $swps);
Signature · 6 lines
2.17.1

_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev
#22 lore raw

[igt-dev] ✓ Fi.CI.BAT: success for Tracing & workload simulation misc patches

In reply to Tvrtko Ursulin
== Series Details ==

== Series Details ==

Series: Tracing & workload simulation misc patches
URL : https://patchwork.freedesktop.org/series/49183/
State : success

== Summary ==

= CI Bug Log - changes from CI_DRM_4770 -> IGTPW_1792 =

== Summary - WARNING ==

Minor unknown changes coming with IGTPW_1792 need to be verified manually.

If you think the reported changes have nothing to do with the changes introduced in IGTPW_1792, please notify your bug team to allow them to document this new failure mode, which will reduce false positives in CI.

External URL: https://patchwork.freedesktop.org/api/1.0/series/49183/revisions/1/mbox/

== Possible new issues ==

Here are the unknown changes that may have been introduced in IGTPW_1792:

=== IGT changes ===

    ==== Warnings ====

    igt@pm_rpm@module-reload:
      fi-hsw-4770r:       SKIP -> PASS

== Known issues ==

Here are the changes found in IGTPW_1792 that come from known issues:

=== IGT changes ===

    ==== Issues hit ====

    igt@gem_exec_suspend@basic-s3:
      fi-skl-caroline:    NOTRUN -> INCOMPLETE (fdo#104108, fdo#107556)

    igt@kms_psr@primary_page_flip:
      fi-icl-u:           NOTRUN -> FAIL (fdo#107383) +3

    
    ==== Possible fixes ====

    igt@drv_module_reload@basic-reload-inject:
      fi-hsw-4770r:       DMESG-WARN (fdo#107425) -> PASS

    igt@gem_exec_suspend@basic-s4-devices:
      fi-kbl-7500u:       DMESG-WARN (fdo#105128, fdo#107139) -> PASS

    igt@kms_frontbuffer_tracking@basic:
      fi-byt-clapper:     FAIL (fdo#103167) -> PASS

    igt@prime_vgem@basic-fence-flip:
      fi-ilk-650:         FAIL (fdo#104008) -> PASS

fdo#103167 https://bugs.freedesktop.org/show_bug.cgi?id=103167
fdo#104008 https://bugs.freedesktop.org/show_bug.cgi?id=104008
fdo#104108 https://bugs.freedesktop.org/show_bug.cgi?id=104108
fdo#105128 https://bugs.freedesktop.org/show_bug.cgi?id=105128
fdo#107139 https://bugs.freedesktop.org/show_bug.cgi?id=107139
fdo#107383 https://bugs.freedesktop.org/show_bug.cgi?id=107383
fdo#107425 https://bugs.freedesktop.org/show_bug.cgi?id=107425
fdo#107556 https://bugs.freedesktop.org/show_bug.cgi?id=107556

== Participating hosts (50 -> 49) ==

Additional (4): fi-glk-j4005 fi-skl-caroline fi-icl-u fi-elk-e7500
Missing (5): fi-ctg-p8600 fi-ilk-m540 fi-byt-squawks fi-bsw-cyan fi-hsw-4200u

== Build changes ==

* IGT: IGT_4627 -> IGTPW_1792

CI_DRM_4770: 0c3535cf60140d017a5df73d84d06e8b1a5b5d3b @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_1792: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1792/
IGT_4627: e0c3033a57d85c0d2eb33af0451afa16edc79f10 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1792/issues.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev

#23 lore raw

[igt-dev] ✓ Fi.CI.IGT: success for Tracing & workload simulation misc patches

In reply to Tvrtko Ursulin
== Series Details ==

== Series Details ==

Series: Tracing & workload simulation misc patches
URL : https://patchwork.freedesktop.org/series/49183/
State : success

== Summary ==

= CI Bug Log - changes from IGT_4627_full -> IGTPW_1792_full =

== Summary - SUCCESS ==

No regressions found.

External URL: https://patchwork.freedesktop.org/api/1.0/series/49183/revisions/1/mbox/

== Known issues ==

Here are the changes found in IGTPW_1792_full that come from known issues:

=== IGT changes ===

    ==== Issues hit ====

    igt@gem_exec_bad_domains@double-write-domain:
      shard-snb:          PASS -> INCOMPLETE (fdo#105411)

    igt@gem_ppgtt@blt-vs-render-ctx0:
      shard-kbl:          PASS -> INCOMPLETE (fdo#103665, fdo#106023)

    igt@kms_cursor_legacy@2x-long-nonblocking-modeset-vs-cursor-atomic:
      shard-glk:          NOTRUN -> FAIL (fdo#105454, fdo#106509)

    igt@kms_rotation_crc@primary-rotation-180:
      shard-snb:          PASS -> FAIL (fdo#103925)

    igt@testdisplay:
      shard-glk:          PASS -> INCOMPLETE (k.org#198133, fdo#107093, fdo#103359)

    
    ==== Possible fixes ====

    igt@kms_plane@pixel-format-pipe-a-planes:
      shard-snb:          FAIL (fdo#107749) -> PASS

    igt@kms_plane_multiple@atomic-pipe-a-tiling-x:
      shard-snb:          FAIL (fdo#103166) -> PASS

    igt@kms_rotation_crc@sprite-rotation-180:
      shard-snb:          FAIL (fdo#103925) -> PASS

    igt@perf@buffer-fill:
      shard-kbl:          INCOMPLETE (fdo#103665) -> PASS

    igt@prime_vgem@basic-write:
      shard-snb:          INCOMPLETE (fdo#105411) -> PASS

fdo#103166 https://bugs.freedesktop.org/show_bug.cgi?id=103166
fdo#103359 https://bugs.freedesktop.org/show_bug.cgi?id=103359
fdo#103665 https://bugs.freedesktop.org/show_bug.cgi?id=103665
fdo#103925 https://bugs.freedesktop.org/show_bug.cgi?id=103925
fdo#105411 https://bugs.freedesktop.org/show_bug.cgi?id=105411
fdo#105454 https://bugs.freedesktop.org/show_bug.cgi?id=105454
fdo#106023 https://bugs.freedesktop.org/show_bug.cgi?id=106023
fdo#106509 https://bugs.freedesktop.org/show_bug.cgi?id=106509
fdo#107093 https://bugs.freedesktop.org/show_bug.cgi?id=107093
fdo#107749 https://bugs.freedesktop.org/show_bug.cgi?id=107749
k.org#198133 https://bugzilla.kernel.org/show_bug.cgi?id=198133

== Participating hosts (5 -> 5) ==

No changes in participating hosts

== Build changes ==

* IGT: IGT_4627 -> IGTPW_1792
* Linux: CI_DRM_4767 -> CI_DRM_4770

CI_DRM_4767: e9b69bafd3c2c13a8b9fa8e7a410f5d5ef32e328 @ git://anongit.freedesktop.org/gfx-ci/linux
CI_DRM_4770: 0c3535cf60140d017a5df73d84d06e8b1a5b5d3b @ git://anongit.freedesktop.org/gfx-ci/linux
IGTPW_1792: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1792/
IGT_4627: e0c3033a57d85c0d2eb33af0451afa16edc79f10 @ git://anongit.freedesktop.org/xorg/app/intel-gpu-tools

== Logs ==

For more details see: https://intel-gfx-ci.01.org/tree/drm-tip/IGTPW_1792/shards.html
_______________________________________________
igt-dev mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/igt-dev