Skip to contentSkip to conversation

Workload simulation and tracing

PATCH Superseded Tvrtko Ursulin 3 participants 26 messages Mar 31, 2017 – Apr 25, 2017

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Workload simulation and tracing

From: Tvrtko Ursulin <[email protected]>

Another WIP posting with some interopability improvements this time.

Example usage:

root@scnuc:~/intel-gpu-tools# benchmarks/gem_wsim
Calibrating nop delay with 1% tolerance...
Nop calibration for 1000us delay is 438660.
root@scnuc:~/intel-gpu-tools# scripts/trace.pl --trace benchmarks/gem_wsim -n 438660 -r 600 -w benchmarks/wsim/workload1 -c 2 -x
Using 438660 nop calibration for 1000us delay.
2 clients.
Swapping VCS rings between clients.
0: 10.222053s elapsed (58.696622 workloads/s)
1: 10.225807s elapsed (58.675078 workloads/s)
10.226307s elapsed (117.344411 workloads/s)
[ perf record: Woken up 16 times to write data ]
[ perf record: Captured and wrote 3.929 MB perf.data (44688 samples) ]
root@scnuc:~/intel-gpu-tools# perf script | scripts/trace.pl -i 1 -i 4
Ring0: 4832 batches, 4329.07 (4348.11) avg batch us, 0.15% idle, 138.73% busy, 23.88% runnable, 79.66% queued, 0.01% wait)
Ring2: 1808 batches, 1715.68 (1719.99) avg batch us, 69.68% idle, 30.38% busy, 0.12% runnable, 112.04% queued, 99.32% wait)
Ring3: 1810 batches, 1727.59 (1739.31) avg batch us, 69.28% idle, 30.75% busy, 0.11% runnable, 105.37% queued, 99.46% wait)

Most interesting metric here is the engine idle time which will come into play as I start adding the load balancing options to gem_wsim.

Secondary mode here would be:

root@scnuc:~/intel-gpu-tools# perf script | scripts/trace.pl -i 1 -i 4 --html >graph.html

This would enable a timeline of GPU request execution to be viewed with a little bit of local setup as described in the trace.pl help text.

Tvrtko Ursulin (2):
benchmarks/gem_wsim: Command submission workload simulator igt/scripts: trace.pl to parse the i915 tracepoints

 benchmarks/Makefile.sources |   1 +
 benchmarks/gem_wsim.c       | 593 +++++++++++++++++++++++++++
 benchmarks/wsim/workload1   |   7 +
 scripts/Makefile.am         |   2 +-
 scripts/trace.pl            | 946 ++++++++++++++++++++++++++++++++++++++++++++
 5 files changed, 1548 insertions(+), 1 deletion(-)
 create mode 100644 benchmarks/gem_wsim.c
 create mode 100644 benchmarks/wsim/workload1
 create mode 100755 scripts/trace.pl
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 1/2 benchmarks/gem_wsim: Command submission workload simulator

From: Tvrtko Ursulin <[email protected]>

Tool which emits batch buffers to engines with configurable sequences, durations, contexts, dependencies and userspace waits.

Unfinished but shows promise so sending out for early feedback.

v2:
* Load workload descriptors from files. (also -w)
* Help text.
* Calibration control if needed. (-t)
* NORELOC | LUT to eb flags.
* Added sample workload to wsim/workload1.

TODO list:

* Better error handling.
* Multi-context support for individual clients.
* Random/variable batch length.
* Load balancing plug-in.
* ... ?

Cc Chris Wilson <[email protected]>
Cc "Rogozhkin, Dmitry V" <[email protected]>

gem_wsim updates

---
 benchmarks/Makefile.sources |   1 +
 benchmarks/gem_wsim.c       | 593 ++++++++++++++++++++++++++++++++++++++++++++
 benchmarks/wsim/workload1   |   7 +
 3 files changed, 601 insertions(+)
 create mode 100644 benchmarks/gem_wsim.c
 create mode 100644 benchmarks/wsim/workload1
benchmarks/Makefile.sources +1 -0
index 3af54ebe36f2..3a941150abb3 100644
@@ -14,6 +14,7 @@ benchmarks_prog_list =			\
 	gem_prw				\
 	gem_set_domain			\
 	gem_syslatency			\
+	gem_wsim			\
 	kms_vblank			\
 	prime_lookup			\
 	vgem_mmap			\
benchmarks/gem_wsim.c new file +593 -0

593 lines not on this page · raw message

benchmarks/wsim/workload1 new file +7 -0
new file mode 100644
index 000000000000..5f533d8e168b
@@ -0,0 +1,7 @@
+1.VCS1.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS2.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS2.600.-1.1
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

Re: [PATCH i-g-t 1/2] benchmarks/gem_wsim: Command submission workload simulator

Quoted 14 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Tool which emits batch buffers to engines with configurable sequences, durations, contexts, dependencies and userspace waits.

Unfinished but shows promise so sending out for early feedback.

v2:
* Load workload descriptors from files. (also -w)
* Help text.
* Calibration control if needed. (-t)
* NORELOC | LUT to eb flags.
* Added sample workload to wsim/workload1.

Quoted 4 lines

TODO list:

* Better error handling.
* Multi-context support for individual clients.

I think that will also wants multiple dependencies.

Quoted 3 lines

* Random/variable batch length.
* Load balancing plug-in.
* ... ?

Waits and delayed execution cycles.

Multiple clients (as threads).

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH v3 1/1 benchmarks/gem_wsim: Command submission workload simulator

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Tool which emits batch buffers to engines with configurable sequences, durations, contexts, dependencies and userspace waits.

Unfinished but shows promise so sending out for early feedback.

v2:
* Load workload descriptors from files. (also -w)
* Help text.
* Calibration control if needed. (-t)
* NORELOC | LUT to eb flags.
* Added sample workload to wsim/workload1.

v3:
* Multiple parallel different workloads (-w -w ...).
* Multi-context workloads.
* Variable (random) batch length.
* Load balancing (round robin and queue depth estimation).
* Workloads delays and explicit sync steps.
* Workload frequency (period) control.

TODO list:

* Fence support.
* Move majority of help text to README.
* Better error handling.
* Less 1980's workload parsing.
* Proper workloads.
* Explicit waits?
* Threads?
* ... ?

Cc Chris Wilson <[email protected]>
Cc "Rogozhkin, Dmitry V" <[email protected]>
---
 benchmarks/Makefile.sources |    1 +
 benchmarks/gem_wsim.c       | 1053 +++++++++++++++++++++++++++++++++++++++++++
 benchmarks/wsim/workload1   |    7 +
 benchmarks/wsim/workload2   |    7 +
 benchmarks/wsim/workload3   |    7 +
 benchmarks/wsim/workload4   |    8 +
 benchmarks/wsim/workload5   |    8 +
 benchmarks/wsim/workload6   |    8 +
 8 files changed, 1099 insertions(+)
 create mode 100644 benchmarks/gem_wsim.c
 create mode 100644 benchmarks/wsim/workload1
 create mode 100644 benchmarks/wsim/workload2
 create mode 100644 benchmarks/wsim/workload3
 create mode 100644 benchmarks/wsim/workload4
 create mode 100644 benchmarks/wsim/workload5
 create mode 100644 benchmarks/wsim/workload6
benchmarks/Makefile.sources +1 -0
index 3af54ebe36f2..3a941150abb3 100644
@@ -14,6 +14,7 @@ benchmarks_prog_list =			\
 	gem_prw				\
 	gem_set_domain			\
 	gem_syslatency			\
+	gem_wsim			\
 	kms_vblank			\
 	prime_lookup			\
 	vgem_mmap			\
benchmarks/gem_wsim.c new file +1053 -0

1,053 lines not on this page · raw message

benchmarks/wsim/workload1 new file +7 -0
new file mode 100644
index 000000000000..5f533d8e168b
@@ -0,0 +1,7 @@
+1.VCS1.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS2.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS2.600.-1.1
benchmarks/wsim/workload2 new file +7 -0
new file mode 100644
index 000000000000..25a692032eae
@@ -0,0 +1,7 @@
+1.VCS.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS.600.-1.1
benchmarks/wsim/workload3 new file +7 -0
new file mode 100644
index 000000000000..bc9f6df52775
@@ -0,0 +1,7 @@
+1.VCS.3000.0.0
+1.RCS.500-1500.-1.0
+0.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS.2300.-2.0
+2.RCS.4700.-1.0
+1.VCS.600.-1.0
benchmarks/wsim/workload4 new file +8 -0
new file mode 100644
index 000000000000..3e4720a6949c
@@ -0,0 +1,8 @@
+1.VCS.3000.0.0
+1.RCS.500-1500.-1.0
+d.1000
+0.RCS.3700.0.0
+1.RCS.1000.-3.0
+1.VCS.2300.-2.0
+2.RCS.4700.-1.0
+1.VCS.600.-1.0
benchmarks/wsim/workload5 new file +8 -0
new file mode 100644
index 000000000000..65440a8264ef
@@ -0,0 +1,8 @@
+1.VCS.3000.0.0
+1.RCS.500-1500.-1.0
+0.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS.2300.-2.0
+2.RCS.4700.-1.0
+1.VCS.600.-1.0
+p.16000
benchmarks/wsim/workload6 new file +8 -0
new file mode 100644
index 000000000000..d5b7141dfdd0
@@ -0,0 +1,8 @@
+1.VCS.3000.0.0
+1.RCS.500-1500.-1.0
+s.-1
+0.RCS.3700.0.0
+1.RCS.1000.-3.0
+1.VCS.2300.-2.0
+2.RCS.4700.-1.0
+1.VCS.600.-1.0
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

Re: [PATCH i-g-t v3] benchmarks/gem_wsim: Command submission workload simulator

Quoted 50 lines · Tvrtko Ursulin
benchmarks/gem_wsim.c new file +50 -0
+static void
+__emit_bb_end(struct w_step *w, bool terminate, bool seqnos, uint32_t seqno)
+{
+	const uint32_t bbe = 0xa << 23;
+	unsigned long bb_sz = get_bb_sz(&w->duration);
+	unsigned long mmap_start, cmd_offset, mmap_len;
+	uint32_t *ptr, *cs;
+
+	mmap_len = (seqnos ? 5 : 1) * sizeof(uint32_t);
+	cmd_offset = bb_sz - mmap_len;
+	mmap_start = rounddown(cmd_offset, PAGE_SIZE);
+	mmap_len += cmd_offset - mmap_start;
+
+	gem_set_domain(fd, w->bb_handle,
+		       I915_GEM_DOMAIN_CPU, I915_GEM_DOMAIN_CPU);
+
+	ptr = gem_mmap__cpu(fd, w->bb_handle, mmap_start, mmap_len, PROT_WRITE);
+	cs = (uint32_t *)((char *)ptr + cmd_offset - mmap_start);
+
+	if (seqnos) {
+		const int gen = intel_gen(intel_get_drm_devid(fd));
+
+		igt_assert(gen >= 8);
+
+		w->reloc.offset = bb_sz - 4 * sizeof(uint32_t);
+		w->seqno_offset = bb_sz - 2 * sizeof(uint32_t);
+
+		*cs++ = terminate ? MI_STORE_DWORD_IMM : 0;
+		*cs++ = 0;
+		*cs++ = 0;
+		*cs++ = seqno;
+	}
+
+	*cs = terminate ? bbe : 0;
+
+	munmap(ptr, mmap_len);
+}
+
+static void terminate_bb(struct w_step *w, bool seqnos, uint32_t seqno)
+{
+	__emit_bb_end(w, true, seqnos, seqno);
+}
+
+static void unterminate_bb(struct w_step *w, bool seqnos)
+{
+	__emit_bb_end(w, false, seqnos, 0);
+}
+
+static void
+prepare_workload(struct workload *wrk, bool swap_vcs, bool seqnos)

benchmarks/gem_wsim.c line 444

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+{
+	int max_ctx = -1;
+	struct w_step *w;
+	int i;
+
+	if (seqnos) {
+		const unsigned int status_sz = sizeof(uint32_t);
+
+		for (i = 0; i < NUM_ENGINES; i++) {
+			wrk->status_page_handle[i] = gem_create(fd, status_sz);

Need to set_cache_level(CACHED) for llc.

You can use one page for all engines. Just use a different cacheline for each, for safety.

Quoted 33 lines
benchmarks/gem_wsim.c new file +33 -0
+			wrk->status_page[i] =
+				gem_mmap__cpu(fd, wrk->status_page_handle[i],
+					      0, status_sz, PROT_READ);
+		}
+	}
+
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		if ((int)w->context > max_ctx) {
+			int delta = w->context + 1 - wrk->nr_ctxs;
+
+			wrk->nr_ctxs += delta;
+			wrk->ctx_id = realloc(wrk->ctx_id,
+					      wrk->nr_ctxs * sizeof(uint32_t));
+			memset(&wrk->ctx_id[wrk->nr_ctxs - delta], 0,
+			       delta * sizeof(uint32_t));
+
+			max_ctx = w->context;
+		}
+
+		if (!wrk->ctx_id[w->context]) {
+			struct drm_i915_gem_context_create arg = {};
+
+			drmIoctl(fd, DRM_IOCTL_I915_GEM_CONTEXT_CREATE, &arg);
+			igt_assert(arg.ctx_id);
+
+			wrk->ctx_id[w->context] = arg.ctx_id;
+		}
+	}
+
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		enum intel_engine_id engine = w->engine;
+		unsigned int bb_i, j = 0;
+

benchmarks/gem_wsim.c line 487

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+		if (w->type != BATCH)
+			continue;
+
+		w->obj[j].handle = gem_create(fd, 4096);
+		w->obj[j].flags = EXEC_OBJECT_WRITE;
+		j++;
+
+		if (seqnos) {
+			w->obj[j].handle = wrk->status_page_handle[engine];
+			w->obj[j].flags = EXEC_OBJECT_WRITE;

The trick for sharing between engines is to not mark this as a WRITE. Fun little lies.

benchmarks/gem_wsim.c line 500

Quoted 13 lines
benchmarks/gem_wsim.c new file +13 -0
+			j++;
+		}
+
+		bb_i = j++;
+		w->duration.cur = w->duration.max;
+		w->bb_sz = get_bb_sz(&w->duration);
+		w->bb_handle = w->obj[bb_i].handle = gem_create(fd, w->bb_sz);
+		terminate_bb(w, seqnos, 0);
+		if (seqnos) {
+			w->reloc.presumed_offset = -1;
+			w->reloc.target_handle = 1;
+			w->reloc.read_domains = I915_GEM_DOMAIN_INSTRUCTION;
+			w->reloc.write_domain = I915_GEM_DOMAIN_INSTRUCTION;

Ugh. That's a magic w/a value for pipecontrols. Fortunately we don't want to set write_domain here anyway.

Quoted 23 lines
benchmarks/gem_wsim.c new file +23 -0
+		}
+
+		igt_assert(w->dependency <= 0);
+		if (w->dependency) {
+			int dep_idx = i + w->dependency;
+
+			igt_assert(dep_idx >= 0 && dep_idx < wrk->nr_steps);
+			igt_assert(wrk->steps[dep_idx].type == BATCH);
+
+			w->obj[j].handle = w->obj[bb_i].handle;
+			bb_i = j;
+			w->obj[j - 1].handle =
+					wrk->steps[dep_idx].obj[0].handle;
+			j++;
+		}
+
+		if (seqnos) {
+			w->obj[bb_i].relocs_ptr = to_user_pointer(&w->reloc);
+			w->obj[bb_i].relocation_count = 1;
+		}
+
+		w->eb.buffers_ptr = to_user_pointer(w->obj);
+		w->eb.buffer_count = j;

benchmarks/gem_wsim.c line 533

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+		w->eb.rsvd1 = wrk->ctx_id[w->context];
+
+		if (swap_vcs && engine == VCS1)
+			engine = VCS2;
+		else if (swap_vcs && engine == VCS2)
+			engine = VCS1;
+		w->eb.flags = eb_engine_map[engine];
+		w->eb.flags |= I915_EXEC_HANDLE_LUT;
+		if (!seqnos)
+			w->eb.flags |= I915_EXEC_NO_RELOC;

Doesn't look too hard to get the relocation right. Forcing relocations between batches is probably a good one to check (just to say don't do that)

Quoted 8 lines
benchmarks/gem_wsim.c new file +8 -0
+#ifdef DEBUG
+		printf("%u: %u:%x|%x|%x|%x %10lu flags=%llx bb=%x[%u] ctx[%u]=%u\n",
+		       i, w->eb.buffer_count, w->obj[0].handle,
+		       w->obj[1].handle, w->obj[2].handle, w->obj[3].handle,
+		       w->bb_sz, w->eb.flags, w->bb_handle, bb_i,
+		       w->context, wrk->ctx_id[w->context]);
+#endif
+	}

benchmarks/gem_wsim.c line 551

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+}
+
+static double elapsed(const struct timespec *start, const struct timespec *end)
+{
+	return (end->tv_sec - start->tv_sec) +
+	       (end->tv_nsec - start->tv_nsec) / 1e9;
+}
+
+static int elapsed_us(const struct timespec *start, const struct timespec *end)
+{

return 1e6 * elapsed(); might as well use gcc for something!

Quoted 39 lines
benchmarks/gem_wsim.c new file +39 -0
+	return (1e9 * (end->tv_sec - start->tv_sec) +
+	       (end->tv_nsec - start->tv_nsec)) / 1e3;
+}
+
+static enum intel_engine_id
+rr_balance(struct workload *wrk, struct w_step *w)
+{
+	unsigned int engine;
+
+	if (wrk->vcs_rr)
+		engine = VCS2;
+	else
+		engine = VCS1;
+
+	wrk->vcs_rr ^= 1;
+
+	return engine;
+}
+
+static enum intel_engine_id
+qd_balance(struct workload *wrk, struct w_step *w)
+{
+	unsigned long qd[NUM_ENGINES];
+	enum intel_engine_id engine = w->engine;
+
+	igt_assert(engine == VCS);
+
+	qd[VCS1] = wrk->seqno[VCS1] - wrk->status_page[VCS1][0];
+	wrk->qd_sum[VCS1] += qd[VCS1];
+
+	qd[VCS2] = wrk->seqno[VCS2] - wrk->status_page[VCS2][0];
+	wrk->qd_sum[VCS2] += qd[VCS2];
+
+	if (qd[VCS1] < qd[VCS2]) {
+		engine = VCS1;
+		wrk->vcs_rr = 0;
+	} else if (qd[VCS2] < qd[VCS1]) {
+		engine = VCS2;
+		wrk->vcs_rr = 1;

benchmarks/gem_wsim.c line 600

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+	} else {
+		unsigned int vcs = wrk->vcs_rr ^ 1;
+
+		wrk->vcs_rr = vcs;
+
+		if (vcs == 0)
+			engine = VCS1;
+		else
+			engine = VCS2;
+	}

Hmm. Just thinking we don't even need hw to simulate a load-balancer, but that would be boring!

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+// printf("qd_balance: 1:%lu 2:%lu rr:%u = %u\n", qd[VCS1], qd[VCS2], wrk->vcs_rr, engine);
+
+	return engine;
+}
+
+static void update_bb_seqno(struct w_step *w, uint32_t seqno)
+{
+	unsigned long mmap_start, mmap_offset, mmap_len;
+	void *ptr;
+

benchmarks/gem_wsim.c line 621

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+	mmap_start = rounddown(w->seqno_offset, PAGE_SIZE);
+	mmap_offset = w->seqno_offset - mmap_start;
+	mmap_len = sizeof(uint32_t) + mmap_offset;
+
+	gem_set_domain(fd, w->bb_handle,
+		       I915_GEM_DOMAIN_CPU, I915_GEM_DOMAIN_CPU);
+
+	ptr = gem_mmap__cpu(fd, w->bb_handle, mmap_start, mmap_len, PROT_WRITE);
+
+	*(uint32_t *)((char *)ptr + mmap_offset) = seqno;

Uh oh. I hope this isn't called inside any loop. Note this is unsynchronized to the gpu so I wonder what this is for.

Quoted 67 lines
benchmarks/gem_wsim.c new file +67 -0
+
+	munmap(ptr, mmap_len);
+}
+
+static void
+run_workload(unsigned int id, struct workload *wrk, unsigned int repeat,
+	     enum intel_engine_id (*balance)(struct workload *wrk,
+					     struct w_step *w), bool seqnos)
+{
+	struct timespec t_start, t_end;
+	struct w_step *w;
+	double t;
+	int i, j;
+
+	clock_gettime(CLOCK_MONOTONIC, &t_start);
+
+	srand(t_start.tv_nsec);
+
+	for (j = 0; j < repeat; j++) {
+		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+			enum intel_engine_id engine = w->engine;
+			uint32_t seqno;
+			bool seqno_updated = false;
+			int do_sleep = 0;
+
+			if (i == 0)
+				clock_gettime(CLOCK_MONOTONIC,
+					      &wrk->repeat_start);
+
+			if (w->type == DELAY) {
+				do_sleep = w->wait;
+			} else if (w->type == PERIOD) {
+				struct timespec now;
+
+				clock_gettime(CLOCK_MONOTONIC, &now);
+				do_sleep = w->wait -
+					   elapsed_us(&wrk->repeat_start, &now);
+				if (do_sleep < 0) {
+					if (!quiet) {
+						printf("%u: Dropped period @ %u/%u (%dus late)!\n",
+						       id, j, i, do_sleep);
+						continue;
+					}
+				}
+			} else if (w->type == SYNC) {
+				unsigned int s_idx = i + w->wait;
+
+				igt_assert(i > 0 && i < wrk->nr_steps);
+				igt_assert(wrk->steps[s_idx].type == BATCH);
+				gem_sync(fd, wrk->steps[s_idx].obj[0].handle);
+				continue;
+			}
+
+			if (do_sleep) {
+				usleep(do_sleep);
+				continue;
+			}
+
+			wrk->nr_bb[engine]++;
+
+			if (engine == VCS && balance) {
+				engine = balance(wrk, w);
+				wrk->nr_bb[engine]++;
+
+				w->obj[1].handle = wrk->status_page_handle[engine];
+
+				w->eb.flags = eb_engine_map[engine];

benchmarks/gem_wsim.c line 698

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+				w->eb.flags |= I915_EXEC_HANDLE_LUT;
+			}
+
+			seqno = ++wrk->seqno[engine];
+
+			if (w->duration.min != w->duration.max) {
+				unsigned int cur = get_duration(&w->duration);
+
+				if (cur != w->duration.cur) {
+					unterminate_bb(w, seqnos);

Ah, you said this was for adjusting runlength of the batches. I suggest using batch_start_offset to change the number of nops rather than rewrite the batch.

I need to study this a bit more...
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

Re: [PATCH i-g-t v3] benchmarks/gem_wsim: Command submission workload simulator

Quoted 105 lines · Chris Wilson
Quoted 50 lines · Tvrtko Ursulin
benchmarks/gem_wsim.c new file +50 -0
+static void
+__emit_bb_end(struct w_step *w, bool terminate, bool seqnos, uint32_t seqno)
+{
+	const uint32_t bbe = 0xa << 23;
+	unsigned long bb_sz = get_bb_sz(&w->duration);
+	unsigned long mmap_start, cmd_offset, mmap_len;
+	uint32_t *ptr, *cs;
+
+	mmap_len = (seqnos ? 5 : 1) * sizeof(uint32_t);
+	cmd_offset = bb_sz - mmap_len;
+	mmap_start = rounddown(cmd_offset, PAGE_SIZE);
+	mmap_len += cmd_offset - mmap_start;
+
+	gem_set_domain(fd, w->bb_handle,
+		       I915_GEM_DOMAIN_CPU, I915_GEM_DOMAIN_CPU);
+
+	ptr = gem_mmap__cpu(fd, w->bb_handle, mmap_start, mmap_len, PROT_WRITE);
+	cs = (uint32_t *)((char *)ptr + cmd_offset - mmap_start);
+
+	if (seqnos) {
+		const int gen = intel_gen(intel_get_drm_devid(fd));
+
+		igt_assert(gen >= 8);
+
+		w->reloc.offset = bb_sz - 4 * sizeof(uint32_t);
+		w->seqno_offset = bb_sz - 2 * sizeof(uint32_t);
+
+		*cs++ = terminate ? MI_STORE_DWORD_IMM : 0;
+		*cs++ = 0;
+		*cs++ = 0;
+		*cs++ = seqno;
+	}
+
+	*cs = terminate ? bbe : 0;
+
+	munmap(ptr, mmap_len);
+}
+
+static void terminate_bb(struct w_step *w, bool seqnos, uint32_t seqno)
+{
+	__emit_bb_end(w, true, seqnos, seqno);
+}
+
+static void unterminate_bb(struct w_step *w, bool seqnos)
+{
+	__emit_bb_end(w, false, seqnos, 0);
+}
+
+static void
+prepare_workload(struct workload *wrk, bool swap_vcs, bool seqnos)

benchmarks/gem_wsim.c line 444

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+{
+	int max_ctx = -1;
+	struct w_step *w;
+	int i;
+
+	if (seqnos) {
+		const unsigned int status_sz = sizeof(uint32_t);
+
+		for (i = 0; i < NUM_ENGINES; i++) {
+			wrk->status_page_handle[i] = gem_create(fd, status_sz);

Need to set_cache_level(CACHED) for llc.

You can use one page for all engines. Just use a different cacheline for each, for safety.

Quoted 38 lines
benchmarks/gem_wsim.c new file +38 -0
+			wrk->status_page[i] =
+				gem_mmap__cpu(fd, wrk->status_page_handle[i],
+					      0, status_sz, PROT_READ);
+		}
+	}
+
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		if ((int)w->context > max_ctx) {
+			int delta = w->context + 1 - wrk->nr_ctxs;
+
+			wrk->nr_ctxs += delta;
+			wrk->ctx_id = realloc(wrk->ctx_id,
+					      wrk->nr_ctxs * sizeof(uint32_t));
+			memset(&wrk->ctx_id[wrk->nr_ctxs - delta], 0,
+			       delta * sizeof(uint32_t));
+
+			max_ctx = w->context;
+		}
+
+		if (!wrk->ctx_id[w->context]) {
+			struct drm_i915_gem_context_create arg = {};
+
+			drmIoctl(fd, DRM_IOCTL_I915_GEM_CONTEXT_CREATE, &arg);
+			igt_assert(arg.ctx_id);
+
+			wrk->ctx_id[w->context] = arg.ctx_id;
+		}
+	}
+
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		enum intel_engine_id engine = w->engine;
+		unsigned int bb_i, j = 0;
+
+		if (w->type != BATCH)
+			continue;
+
+		w->obj[j].handle = gem_create(fd, 4096);
+		w->obj[j].flags = EXEC_OBJECT_WRITE;
Quoted 8 lines

benchmarks/gem_wsim.c line 487

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+		j++;
+
+		if (seqnos) {
+			w->obj[j].handle = wrk->status_page_handle[engine];
+			w->obj[j].flags = EXEC_OBJECT_WRITE;

The trick for sharing between engines is to not mark this as a WRITE. Fun little lies.

Yeah thats why I have per-engine objects. Which I don't mind since it is not like they are wasting any resources compared to everything else. But not admitting the write sounds still interesting. What would the repercussions of that be - limit us to llc platforms or something?

Quoted 8 lines
Quoted 8 lines
benchmarks/gem_wsim.c new file +8 -0
+			j++;
+		}
+
+		bb_i = j++;
+		w->duration.cur = w->duration.max;
+		w->bb_sz = get_bb_sz(&w->duration);
+		w->bb_handle = w->obj[bb_i].handle = gem_create(fd, w->bb_sz);
+		terminate_bb(w, seqnos, 0);
Quoted 8 lines

benchmarks/gem_wsim.c line 500

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+		if (seqnos) {
+			w->reloc.presumed_offset = -1;
+			w->reloc.target_handle = 1;
+			w->reloc.read_domains = I915_GEM_DOMAIN_INSTRUCTION;
+			w->reloc.write_domain = I915_GEM_DOMAIN_INSTRUCTION;

Ugh. That's a magic w/a value for pipecontrols. Fortunately we don't want to set write_domain here anyway.

I think I copy-pasted this from another IGT. So you say cheat here as well and set zero for both domains?

Quoted 30 lines
Quoted 30 lines
benchmarks/gem_wsim.c new file +29 -0
+		}
+
+		igt_assert(w->dependency <= 0);
+		if (w->dependency) {
+			int dep_idx = i + w->dependency;
+
+			igt_assert(dep_idx >= 0 && dep_idx < wrk->nr_steps);
+			igt_assert(wrk->steps[dep_idx].type == BATCH);
+
+			w->obj[j].handle = w->obj[bb_i].handle;
+			bb_i = j;
+			w->obj[j - 1].handle =
+					wrk->steps[dep_idx].obj[0].handle;
+			j++;
+		}
+
+		if (seqnos) {
+			w->obj[bb_i].relocs_ptr = to_user_pointer(&w->reloc);
+			w->obj[bb_i].relocation_count = 1;
+		}
+
+		w->eb.buffers_ptr = to_user_pointer(w->obj);
+		w->eb.buffer_count = j;
+		w->eb.rsvd1 = wrk->ctx_id[w->context];
+
+		if (swap_vcs && engine == VCS1)
+			engine = VCS2;
+		else if (swap_vcs && engine == VCS2)
+			engine = VCS1;
Quoted 8 lines

benchmarks/gem_wsim.c line 533

Quoted 4 lines
benchmarks/gem_wsim.c new file +4 -0
+		w->eb.flags = eb_engine_map[engine];
+		w->eb.flags |= I915_EXEC_HANDLE_LUT;
+		if (!seqnos)
+			w->eb.flags |= I915_EXEC_NO_RELOC;

Doesn't look too hard to get the relocation right. Forcing relocations between batches is probably a good one to check (just to say don't do that)

I am not following here? You are saying don't do relocations at all? How do I make sure things stay fixed and even how to find out where they are in the first pass?

Quoted 65 lines
Quoted 8 lines
benchmarks/gem_wsim.c new file +8 -0
+#ifdef DEBUG
+		printf("%u: %u:%x|%x|%x|%x %10lu flags=%llx bb=%x[%u] ctx[%u]=%u\n",
+		       i, w->eb.buffer_count, w->obj[0].handle,
+		       w->obj[1].handle, w->obj[2].handle, w->obj[3].handle,
+		       w->bb_sz, w->eb.flags, w->bb_handle, bb_i,
+		       w->context, wrk->ctx_id[w->context]);
+#endif
+	}

benchmarks/gem_wsim.c line 551

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+}
+
+static double elapsed(const struct timespec *start, const struct timespec *end)
+{
+	return (end->tv_sec - start->tv_sec) +
+	       (end->tv_nsec - start->tv_nsec) / 1e9;
+}
+
+static int elapsed_us(const struct timespec *start, const struct timespec *end)
+{

return 1e6 * elapsed(); might as well use gcc for something!

Quoted 44 lines
benchmarks/gem_wsim.c new file +44 -0
+	return (1e9 * (end->tv_sec - start->tv_sec) +
+	       (end->tv_nsec - start->tv_nsec)) / 1e3;
+}
+
+static enum intel_engine_id
+rr_balance(struct workload *wrk, struct w_step *w)
+{
+	unsigned int engine;
+
+	if (wrk->vcs_rr)
+		engine = VCS2;
+	else
+		engine = VCS1;
+
+	wrk->vcs_rr ^= 1;
+
+	return engine;
+}
+
+static enum intel_engine_id
+qd_balance(struct workload *wrk, struct w_step *w)
+{
+	unsigned long qd[NUM_ENGINES];
+	enum intel_engine_id engine = w->engine;
+
+	igt_assert(engine == VCS);
+
+	qd[VCS1] = wrk->seqno[VCS1] - wrk->status_page[VCS1][0];
+	wrk->qd_sum[VCS1] += qd[VCS1];
+
+	qd[VCS2] = wrk->seqno[VCS2] - wrk->status_page[VCS2][0];
+	wrk->qd_sum[VCS2] += qd[VCS2];
+
+	if (qd[VCS1] < qd[VCS2]) {
+		engine = VCS1;
+		wrk->vcs_rr = 0;
+	} else if (qd[VCS2] < qd[VCS1]) {
+		engine = VCS2;
+		wrk->vcs_rr = 1;
+	} else {
+		unsigned int vcs = wrk->vcs_rr ^ 1;
+
+		wrk->vcs_rr = vcs;
+
Quoted 8 lines

benchmarks/gem_wsim.c line 600

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+		if (vcs == 0)
+			engine = VCS1;
+		else
+			engine = VCS2;
+	}

Hmm. Just thinking we don't even need hw to simulate a load-balancer, but that would be boring!

Definitely not as exciting. :) Perhaps there will be other benefits from this tool than the load balancing, but we'll see.

Quoted 15 lines
Quoted 15 lines
benchmarks/gem_wsim.c new file +15 -0
+// printf("qd_balance: 1:%lu 2:%lu rr:%u = %u\n", qd[VCS1], qd[VCS2], wrk->vcs_rr, engine);
+
+	return engine;
+}
+
+static void update_bb_seqno(struct w_step *w, uint32_t seqno)
+{
+	unsigned long mmap_start, mmap_offset, mmap_len;
+	void *ptr;
+
+	mmap_start = rounddown(w->seqno_offset, PAGE_SIZE);
+	mmap_offset = w->seqno_offset - mmap_start;
+	mmap_len = sizeof(uint32_t) + mmap_offset;
+
+	gem_set_domain(fd, w->bb_handle,
Quoted 8 lines

benchmarks/gem_wsim.c line 621

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+		       I915_GEM_DOMAIN_CPU, I915_GEM_DOMAIN_CPU);
+
+	ptr = gem_mmap__cpu(fd, w->bb_handle, mmap_start, mmap_len, PROT_WRITE);
+
+	*(uint32_t *)((char *)ptr + mmap_offset) = seqno;

Uh oh. I hope this isn't called inside any loop. Note this is unsynchronized to the gpu so I wonder what this is for.

To update the seqno inside the store_dword_imm. It is called every time before a batch is executed so I was thinking whether a gem_sync should be preceding it. But then I was thinking it is problematic in general if we queue up multiple same batches before they get executed. :( Sounds like I would need a separate batch for every iteration for this to work correctly. But that sounds too costly. So I don't know at the moment.

Quoted 73 lines
Quoted 73 lines
benchmarks/gem_wsim.c new file +73 -0
+
+	munmap(ptr, mmap_len);
+}
+
+static void
+run_workload(unsigned int id, struct workload *wrk, unsigned int repeat,
+	     enum intel_engine_id (*balance)(struct workload *wrk,
+					     struct w_step *w), bool seqnos)
+{
+	struct timespec t_start, t_end;
+	struct w_step *w;
+	double t;
+	int i, j;
+
+	clock_gettime(CLOCK_MONOTONIC, &t_start);
+
+	srand(t_start.tv_nsec);
+
+	for (j = 0; j < repeat; j++) {
+		for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+			enum intel_engine_id engine = w->engine;
+			uint32_t seqno;
+			bool seqno_updated = false;
+			int do_sleep = 0;
+
+			if (i == 0)
+				clock_gettime(CLOCK_MONOTONIC,
+					      &wrk->repeat_start);
+
+			if (w->type == DELAY) {
+				do_sleep = w->wait;
+			} else if (w->type == PERIOD) {
+				struct timespec now;
+
+				clock_gettime(CLOCK_MONOTONIC, &now);
+				do_sleep = w->wait -
+					   elapsed_us(&wrk->repeat_start, &now);
+				if (do_sleep < 0) {
+					if (!quiet) {
+						printf("%u: Dropped period @ %u/%u (%dus late)!\n",
+						       id, j, i, do_sleep);
+						continue;
+					}
+				}
+			} else if (w->type == SYNC) {
+				unsigned int s_idx = i + w->wait;
+
+				igt_assert(i > 0 && i < wrk->nr_steps);
+				igt_assert(wrk->steps[s_idx].type == BATCH);
+				gem_sync(fd, wrk->steps[s_idx].obj[0].handle);
+				continue;
+			}
+
+			if (do_sleep) {
+				usleep(do_sleep);
+				continue;
+			}
+
+			wrk->nr_bb[engine]++;
+
+			if (engine == VCS && balance) {
+				engine = balance(wrk, w);
+				wrk->nr_bb[engine]++;
+
+				w->obj[1].handle = wrk->status_page_handle[engine];
+
+				w->eb.flags = eb_engine_map[engine];
+				w->eb.flags |= I915_EXEC_HANDLE_LUT;
+			}
+
+			seqno = ++wrk->seqno[engine];
+
+			if (w->duration.min != w->duration.max) {
Quoted 8 lines

benchmarks/gem_wsim.c line 698

Quoted 4 lines
benchmarks/gem_wsim.c new file +4 -0
+				unsigned int cur = get_duration(&w->duration);
+
+				if (cur != w->duration.cur) {
+					unterminate_bb(w, seqnos);

Ah, you said this was for adjusting runlength of the batches. I suggest using batch_start_offset to change the number of nops rather than rewrite the batch.

Yeah thanks for that suggestion, I did not think of that.

Quoted 1 line

I need to study this a bit more...

Yes please, especially the bit about how to get accurate seqnos written out in each step without needing separate execbuf batches.

I've heard recursive batches mentioned in the past so maybe each iteration could have it's own small batch which would jump to the nop/delay one (shared between all iterations) and write the unique seqno. No idea if that is possible/supported at the moment - I'll go and dig a bit.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#6 lore raw

Re: [PATCH i-g-t v3] benchmarks/gem_wsim: Command submission workload simulator

Quoted 116 lines · Tvrtko Ursulin
Quoted 105 lines · Chris Wilson
Quoted 50 lines · Tvrtko Ursulin
benchmarks/gem_wsim.c new file +50 -0
+static void
+__emit_bb_end(struct w_step *w, bool terminate, bool seqnos, uint32_t seqno)
+{
+	const uint32_t bbe = 0xa << 23;
+	unsigned long bb_sz = get_bb_sz(&w->duration);
+	unsigned long mmap_start, cmd_offset, mmap_len;
+	uint32_t *ptr, *cs;
+
+	mmap_len = (seqnos ? 5 : 1) * sizeof(uint32_t);
+	cmd_offset = bb_sz - mmap_len;
+	mmap_start = rounddown(cmd_offset, PAGE_SIZE);
+	mmap_len += cmd_offset - mmap_start;
+
+	gem_set_domain(fd, w->bb_handle,
+		       I915_GEM_DOMAIN_CPU, I915_GEM_DOMAIN_CPU);
+
+	ptr = gem_mmap__cpu(fd, w->bb_handle, mmap_start, mmap_len, PROT_WRITE);
+	cs = (uint32_t *)((char *)ptr + cmd_offset - mmap_start);
+
+	if (seqnos) {
+		const int gen = intel_gen(intel_get_drm_devid(fd));
+
+		igt_assert(gen >= 8);
+
+		w->reloc.offset = bb_sz - 4 * sizeof(uint32_t);
+		w->seqno_offset = bb_sz - 2 * sizeof(uint32_t);
+
+		*cs++ = terminate ? MI_STORE_DWORD_IMM : 0;
+		*cs++ = 0;
+		*cs++ = 0;
+		*cs++ = seqno;
+	}
+
+	*cs = terminate ? bbe : 0;
+
+	munmap(ptr, mmap_len);
+}
+
+static void terminate_bb(struct w_step *w, bool seqnos, uint32_t seqno)
+{
+	__emit_bb_end(w, true, seqnos, seqno);
+}
+
+static void unterminate_bb(struct w_step *w, bool seqnos)
+{
+	__emit_bb_end(w, false, seqnos, 0);
+}
+
+static void
+prepare_workload(struct workload *wrk, bool swap_vcs, bool seqnos)

benchmarks/gem_wsim.c line 444

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+{
+	int max_ctx = -1;
+	struct w_step *w;
+	int i;
+
+	if (seqnos) {
+		const unsigned int status_sz = sizeof(uint32_t);
+
+		for (i = 0; i < NUM_ENGINES; i++) {
+			wrk->status_page_handle[i] = gem_create(fd, status_sz);

Need to set_cache_level(CACHED) for llc.

You can use one page for all engines. Just use a different cacheline for each, for safety.

Quoted 38 lines
benchmarks/gem_wsim.c new file +38 -0
+			wrk->status_page[i] =
+				gem_mmap__cpu(fd, wrk->status_page_handle[i],
+					      0, status_sz, PROT_READ);
+		}
+	}
+
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		if ((int)w->context > max_ctx) {
+			int delta = w->context + 1 - wrk->nr_ctxs;
+
+			wrk->nr_ctxs += delta;
+			wrk->ctx_id = realloc(wrk->ctx_id,
+					      wrk->nr_ctxs * sizeof(uint32_t));
+			memset(&wrk->ctx_id[wrk->nr_ctxs - delta], 0,
+			       delta * sizeof(uint32_t));
+
+			max_ctx = w->context;
+		}
+
+		if (!wrk->ctx_id[w->context]) {
+			struct drm_i915_gem_context_create arg = {};
+
+			drmIoctl(fd, DRM_IOCTL_I915_GEM_CONTEXT_CREATE, &arg);
+			igt_assert(arg.ctx_id);
+
+			wrk->ctx_id[w->context] = arg.ctx_id;
+		}
+	}
+
+	for (i = 0, w = wrk->steps; i < wrk->nr_steps; i++, w++) {
+		enum intel_engine_id engine = w->engine;
+		unsigned int bb_i, j = 0;
+
+		if (w->type != BATCH)
+			continue;
+
+		w->obj[j].handle = gem_create(fd, 4096);
+		w->obj[j].flags = EXEC_OBJECT_WRITE;
Quoted 8 lines

benchmarks/gem_wsim.c line 487

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+		j++;
+
+		if (seqnos) {
+			w->obj[j].handle = wrk->status_page_handle[engine];
+			w->obj[j].flags = EXEC_OBJECT_WRITE;

The trick for sharing between engines is to not mark this as a WRITE. Fun little lies.

Quoted 5 lines

Yeah thats why I have per-engine objects. Which I don't mind since it is not like they are wasting any resources compared to everything else. But not admitting the write sounds still interesting. What would the repercussions of that be - limit us to llc platforms or something?

It used to be that if we evicted the object (e.g. mempressure/suspend), then we would not save the contents since it was never marked as dirty. However, between libva being buggy and Vk deliberately eskewing write hazards, we had to always mark GPU usage as dirtying the buffers. So nowadays, EXEC_OBJECT_WRITE only means to track the implicit write hazard.

Quoted 18 lines
Quoted 17 lines
Quoted 4 lines
benchmarks/gem_wsim.c new file +3 -0
+			j++;
+		}
+

benchmarks/gem_wsim.c line 500

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+		bb_i = j++;
+		w->duration.cur = w->duration.max;
+		w->bb_sz = get_bb_sz(&w->duration);
+		w->bb_handle = w->obj[bb_i].handle = gem_create(fd, w->bb_sz);
+		terminate_bb(w, seqnos, 0);
+		if (seqnos) {
+			w->reloc.presumed_offset = -1;
+			w->reloc.target_handle = 1;
+			w->reloc.read_domains = I915_GEM_DOMAIN_INSTRUCTION;
+			w->reloc.write_domain = I915_GEM_DOMAIN_INSTRUCTION;

Ugh. That's a magic w/a value for pipecontrols. Fortunately we don't want to set write_domain here anyway.

Quoted 2 lines

I think I copy-pasted this from another IGT. So you say cheat here as well and set zero for both domains?

Technically the MI is outside of all the GPU cache domains we have :) Which you pick is immaterial, aside from understanding that (INSTRUCTION, INSTRUCTION) is special ;)

If you were to drop EXEC_OBJECT_WRITE, you would also drop reloc.write_domain.

Quoted 38 lines
Quoted 29 lines
Quoted 29 lines
benchmarks/gem_wsim.c new file +29 -0
+		}
+
+		igt_assert(w->dependency <= 0);
+		if (w->dependency) {
+			int dep_idx = i + w->dependency;
+
+			igt_assert(dep_idx >= 0 && dep_idx < wrk->nr_steps);
+			igt_assert(wrk->steps[dep_idx].type == BATCH);
+
+			w->obj[j].handle = w->obj[bb_i].handle;
+			bb_i = j;
+			w->obj[j - 1].handle =
+					wrk->steps[dep_idx].obj[0].handle;
+			j++;
+		}
+
+		if (seqnos) {
+			w->obj[bb_i].relocs_ptr = to_user_pointer(&w->reloc);
+			w->obj[bb_i].relocation_count = 1;
+		}
+
+		w->eb.buffers_ptr = to_user_pointer(w->obj);
+		w->eb.buffer_count = j;
+		w->eb.rsvd1 = wrk->ctx_id[w->context];
+
+		if (swap_vcs && engine == VCS1)
+			engine = VCS2;
+		else if (swap_vcs && engine == VCS2)
+			engine = VCS1;
Quoted 8 lines

benchmarks/gem_wsim.c line 533

Quoted 4 lines
benchmarks/gem_wsim.c new file +4 -0
+		w->eb.flags = eb_engine_map[engine];
+		w->eb.flags |= I915_EXEC_HANDLE_LUT;
+		if (!seqnos)
+			w->eb.flags |= I915_EXEC_NO_RELOC;

Doesn't look too hard to get the relocation right. Forcing relocations between batches is probably a good one to check (just to say don't do that)

Quoted 3 lines

I am not following here? You are saying don't do relocations at all? How do I make sure things stay fixed and even how to find out where they are in the first pass?

Depending on the workload, it may be informative to also do comparisons between NORELOC and always RELOC. Personally I would make sure we were using NORELOC as this should be a simulator/example.

Quoted 19 lines
Quoted 10 lines
Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+static void update_bb_seqno(struct w_step *w, uint32_t seqno)
+{
+	unsigned long mmap_start, mmap_offset, mmap_len;
+	void *ptr;
+
+	mmap_start = rounddown(w->seqno_offset, PAGE_SIZE);
+	mmap_offset = w->seqno_offset - mmap_start;
+	mmap_len = sizeof(uint32_t) + mmap_offset;
+
+	gem_set_domain(fd, w->bb_handle,
Quoted 8 lines

benchmarks/gem_wsim.c line 621

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+		       I915_GEM_DOMAIN_CPU, I915_GEM_DOMAIN_CPU);
+
+	ptr = gem_mmap__cpu(fd, w->bb_handle, mmap_start, mmap_len, PROT_WRITE);
+
+	*(uint32_t *)((char *)ptr + mmap_offset) = seqno;

Uh oh. I hope this isn't called inside any loop. Note this is unsynchronized to the gpu so I wonder what this is for.

Quoted 7 lines

To update the seqno inside the store_dword_imm. It is called every time before a batch is executed so I was thinking whether a gem_sync should be preceding it. But then I was thinking it is problematic in general if we queue up multiple same batches before they get executed. :( Sounds like I would need a separate batch for every iteration for this to work correctly. But that sounds too costly. So I don't know at the moment.

mmap/munmap, especially munmap, is not free. The munmap will do a tlb_flush across all cores -- though maybe that's batched and the munmaps I do all tend to be large enough to trigger every time.

Since you are using a CPU write, on !llc this will be clflushing everytime. I would suggest stashing the gem_mmap__wc for updating the seqno between repeats.

[snip]

Quoted 10 lines
Quoted 1 line

I need to study this a bit more...

Yes please, especially the bit about how to get accurate seqnos written out in each step without needing separate execbuf batches.

I've heard recursive batches mentioned in the past so maybe each iteration could have it's own small batch which would jump to the nop/delay one (shared between all iterations) and write the unique seqno. No idea if that is possible/supported at the moment - I'll go and dig a bit.

You end up with the same problem of having the reloc change and need to update every cycle. You could use a fresh batch to rewrite the seqno values... However, now that you explained what you want, just keep the WC mmap.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#7 lore raw

Re: [PATCH i-g-t v3] benchmarks/gem_wsim: Command submission workload simulator

Quoted 1 line · Chris Wilson

On Thu, Apr 06, 2017 at 09:18:36AM +0100, Tvrtko Ursulin wrote:

[snip]

Quoted 20 lines
Quoted 17 lines
Quoted 8 lines
Quoted 8 lines
benchmarks/gem_wsim.c new file +8 -0
+			j++;
+		}
+
+		bb_i = j++;
+		w->duration.cur = w->duration.max;
+		w->bb_sz = get_bb_sz(&w->duration);
+		w->bb_handle = w->obj[bb_i].handle = gem_create(fd, w->bb_sz);
+		terminate_bb(w, seqnos, 0);
Quoted 8 lines

benchmarks/gem_wsim.c line 500

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+		if (seqnos) {
+			w->reloc.presumed_offset = -1;
+			w->reloc.target_handle = 1;
+			w->reloc.read_domains = I915_GEM_DOMAIN_INSTRUCTION;
+			w->reloc.write_domain = I915_GEM_DOMAIN_INSTRUCTION;

Ugh. That's a magic w/a value for pipecontrols. Fortunately we don't want to set write_domain here anyway.

Quoted 2 lines

I think I copy-pasted this from another IGT. So you say cheat here as well and set zero for both domains?

Quoted 6 lines

Technically the MI is outside of all the GPU cache domains we have :) Which you pick is immaterial, aside from understanding that (INSTRUCTION, INSTRUCTION) is special ;)

If you were to drop EXEC_OBJECT_WRITE, you would also drop reloc.write_domain.

Okay, I will try the cheating approach then.

Quoted 38 lines
Quoted 38 lines
Quoted 37 lines
Quoted 23 lines
benchmarks/gem_wsim.c new file +23 -0
+		}
+
+		igt_assert(w->dependency <= 0);
+		if (w->dependency) {
+			int dep_idx = i + w->dependency;
+
+			igt_assert(dep_idx >= 0 && dep_idx < wrk->nr_steps);
+			igt_assert(wrk->steps[dep_idx].type == BATCH);
+
+			w->obj[j].handle = w->obj[bb_i].handle;
+			bb_i = j;
+			w->obj[j - 1].handle =
+					wrk->steps[dep_idx].obj[0].handle;
+			j++;
+		}
+
+		if (seqnos) {
+			w->obj[bb_i].relocs_ptr = to_user_pointer(&w->reloc);
+			w->obj[bb_i].relocation_count = 1;
+		}
+
+		w->eb.buffers_ptr = to_user_pointer(w->obj);
+		w->eb.buffer_count = j;

benchmarks/gem_wsim.c line 533

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+		w->eb.rsvd1 = wrk->ctx_id[w->context];
+
+		if (swap_vcs && engine == VCS1)
+			engine = VCS2;
+		else if (swap_vcs && engine == VCS2)
+			engine = VCS1;
+		w->eb.flags = eb_engine_map[engine];
+		w->eb.flags |= I915_EXEC_HANDLE_LUT;
+		if (!seqnos)
+			w->eb.flags |= I915_EXEC_NO_RELOC;

Doesn't look too hard to get the relocation right. Forcing relocations between batches is probably a good one to check (just to say don't do that)

Quoted 7 lines
Quoted 3 lines

I am not following here? You are saying don't do relocations at all? How do I make sure things stay fixed and even how to find out where they are in the first pass?

Depending on the workload, it may be informative to also do comparisons between NORELOC and always RELOC. Personally I would make sure we were using NORELOC as this should be a simulator/example.

How do I use NORELOC? I mean, I have to know where to objects will be pinned, or be able to pin them first and know they will remain put. What am I not understanding here?

Quoted 27 lines
Quoted 19 lines
Quoted 10 lines
Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+static void update_bb_seqno(struct w_step *w, uint32_t seqno)
+{
+	unsigned long mmap_start, mmap_offset, mmap_len;
+	void *ptr;
+
+	mmap_start = rounddown(w->seqno_offset, PAGE_SIZE);
+	mmap_offset = w->seqno_offset - mmap_start;
+	mmap_len = sizeof(uint32_t) + mmap_offset;
+
+	gem_set_domain(fd, w->bb_handle,
Quoted 8 lines

benchmarks/gem_wsim.c line 621

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+		       I915_GEM_DOMAIN_CPU, I915_GEM_DOMAIN_CPU);
+
+	ptr = gem_mmap__cpu(fd, w->bb_handle, mmap_start, mmap_len, PROT_WRITE);
+
+	*(uint32_t *)((char *)ptr + mmap_offset) = seqno;

Uh oh. I hope this isn't called inside any loop. Note this is unsynchronized to the gpu so I wonder what this is for.

Quoted 7 lines

To update the seqno inside the store_dword_imm. It is called every time before a batch is executed so I was thinking whether a gem_sync should be preceding it. But then I was thinking it is problematic in general if we queue up multiple same batches before they get executed. :( Sounds like I would need a separate batch for every iteration for this to work correctly. But that sounds too costly. So I don't know at the moment.

Quoted 7 lines

mmap/munmap, especially munmap, is not free. The munmap will do a tlb_flush across all cores -- though maybe that's batched and the munmaps I do all tend to be large enough to trigger every time.

Since you are using a CPU write, on !llc this will be clflushing everytime. I would suggest stashing the gem_mmap__wc for updating the seqno between repeats.

Ok, I can try that approach.

Quoted 13 lines

[snip]

Quoted 10 lines
Quoted 1 line

I need to study this a bit more...

Yes please, especially the bit about how to get accurate seqnos written out in each step without needing separate execbuf batches.

I've heard recursive batches mentioned in the past so maybe each iteration could have it's own small batch which would jump to the nop/delay one (shared between all iterations) and write the unique seqno. No idea if that is possible/supported at the moment - I'll go and dig a bit.

Quoted 4 lines

You end up with the same problem of having the reloc change and need to update every cycle. You could use a fresh batch to rewrite the seqno values... However, now that you explained what you want, just keep the WC mmap.

Hm not sure without researching that approach first.

But in general is this correctly implementing your idea for queue depth estimation?

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#8 lore raw

Re: [PATCH i-g-t v3] benchmarks/gem_wsim: Command submission workload simulator

Quoted 5 lines · Tvrtko Ursulin
Quoted 1 line · Chris Wilson

On Thu, Apr 06, 2017 at 09:18:36AM +0100, Tvrtko Ursulin wrote:

[snip]

[snip]

Quoted 17 lines
Quoted 17 lines
Quoted 16 lines
Quoted 4 lines
Quoted 4 lines
benchmarks/gem_wsim.c new file +4 -0
+		if (swap_vcs && engine == VCS1)
+			engine = VCS2;
+		else if (swap_vcs && engine == VCS2)
+			engine = VCS1;
Quoted 8 lines

benchmarks/gem_wsim.c line 533

Quoted 4 lines
benchmarks/gem_wsim.c new file +4 -0
+		w->eb.flags = eb_engine_map[engine];
+		w->eb.flags |= I915_EXEC_HANDLE_LUT;
+		if (!seqnos)
+			w->eb.flags |= I915_EXEC_NO_RELOC;

Doesn't look too hard to get the relocation right. Forcing relocations between batches is probably a good one to check (just to say don't do that)

I am not following here? You are saying don't do relocations at all? How do I make sure things stay fixed and even how to find out where they are in the first pass?

Quoted 7 lines
Quoted 3 lines

Depending on the workload, it may be informative to also do comparisons between NORELOC and always RELOC. Personally I would make sure we were using NORELOC as this should be a simulator/example.

How do I use NORELOC? I mean, I have to know where to objects will be pinned, or be able to pin them first and know they will remain put. What am I not understanding here?

It will be assigned an address on first execution. Can I quote the spiel I wrote for i915_gem_execbuffer.c and see if that answers how to use NORELOC:

* Reserving resources for the execbuf is the most complicated phase. We
* neither want to have to migrate the object in the address space, nor do
* we want to have to update any relocations pointing to this object. Ideally,
* we want to leave the object where it is and for all the existing relocations
* to match. If the object is given a new address, or if userspace thinks the
* object is elsewhere, we have to parse all the relocation entries and update
* the addresses. Userspace can set the I915_EXEC_NORELOC flag to hint that
* all the target addresses in all of its objects match the value in the
* relocation entries and that they all match the presumed offsets given by the
* list of execbuffer objects. Using this knowledge, we know that if we haven't
* moved any buffers, all the relocation entries are valid and we can skip
* the update. (If userspace is wrong, the likely outcome is an impromptu GPU
* hang.) The requirement for using I915_EXEC_NO_RELOC are:
*

 *      The addresses written in the objects must match the corresponding
 *      reloc.presumed_offset which in turn must match the corresponding
 *      execobject.offset.

*
* Any render targets written to in the batch must be flagged with
* EXEC_OBJECT_WRITE.
*
* To avoid stalling, execobject.offset should match the current
* address of that object within the active context.
*

Does that make sense? How questions remain unanswered?

Hmm, I usually sum it up as

	batch[reloc.offset] == reloc.presumed_offset + reloc.delta;

and

	execobj.offset == reloc.presumed_offset

must be true at the time of execbuf. Note that upon relocation, batch[reloc.offset], reloc.presumed_offset and execobj.offset are updated. This is important to remember if you are prerecording the reloc/execobj arrays, and not feeding back the results of execbuf between phases.

Quoted 2 lines

But in general is this correctly implementing your idea for queue depth estimation?

From my rough checklist:

* writes engine->next_seqno++ after each op (in this case end of batch)
* qlen[engine] = engine->next_seqno - *engine->current_seqno;

Design looks right. Implementation requires checking... I'll be back. -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#9 lore raw

PATCH v4 1/1 benchmarks/gem_wsim: Command submission workload simulator

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Tool which emits batch buffers to engines with configurable sequences, durations, contexts, dependencies and userspace waits.

Unfinished but shows promise so sending out for early feedback.

v2:
* Load workload descriptors from files. (also -w)
* Help text.
* Calibration control if needed. (-t)
* NORELOC | LUT to eb flags.
* Added sample workload to wsim/workload1.

v3:
* Multiple parallel different workloads (-w -w ...).
* Multi-context workloads.
* Variable (random) batch length.
* Load balancing (round robin and queue depth estimation).
* Workloads delays and explicit sync steps.
* Workload frequency (period) control.

v4:
* Fixed queue-depth estimation by creating separate batches per engine when qd load balancing is on.
* Dropped separate -s cmd line option. It can turn itself on automatically when needed.
* Keep a single status page and lie about the write hazard as suggested by Chris.
* Use batch_start_offset for controlling the batch duration. (Chris)
* Set status page object cache level. (Chris)
* Moved workload description to a README.
* Tidied example workloads.
* Some other cleanups and refactorings.

TODO list:

* Fence support.
* Better error handling.
* Less 1980's workload parsing.
* Proper workloads.
* Threads?
* ... ?

Cc Chris Wilson <[email protected]>
Cc "Rogozhkin, Dmitry V" <[email protected]>

---

Comparing some test workloads under load balancing it seems that it is starting to work, but it still needs more thorough verification. For example, round- robin balancing:

# benchmarks/gem_wsim -n 585341 \
		      -w benchmarks/wsim/vcs1.wsim \
		      -w benchmarks/wsim/vcs_balanced.wsim \
		      -r 100 -b 0
Using 585341 nop calibration for 1000us delay.
2 clients.
1: 3.008s elapsed (33.243 workloads/s). 2500 (1250 + 1250) total VCS batches.
0: 4.455s elapsed (22.449 workloads/s). 0 (2500 + 0) total VCS batches.
4.455s elapsed (44.889 workloads/s)

Versus the queue-depth estimation:

# benchmarks/gem_wsim -n 585341 \
		      -w benchmarks/wsim/vcs1.wsim \
		      -w benchmarks/wsim/vcs_balanced.wsim \
		      -r 100 -b 1
Using 585341 nop calibration for 1000us delay.
2 clients.
1: 2.239s elapsed (44.659 workloads/s). 2500 (837 + 1663) total VCS batches. Average queue depths 27.575, 19.285.
0: 4.012s elapsed (24.928 workloads/s). 0 (2500 + 0) total VCS batches. Average queue depths -nan, -nan.
4.012s elapsed (49.845 workloads/s)

In both cases we run two workloads, one which only submits to VCS1 and one which can be load-balanced. The latter gets a ~33% boost with queue-depth estimation, and the non-balancing workload ~10%.

---
 benchmarks/Makefile.sources                  |    1 +
 benchmarks/gem_wsim.c                        | 1014 ++++++++++++++++++++++++++
 benchmarks/wsim/README                       |   54 ++
 benchmarks/wsim/media_17i7.wsim              |    7 +
 benchmarks/wsim/media_load_balance_17i7.wsim |    7 +
 benchmarks/wsim/vcs1.wsim                    |   25 +
 benchmarks/wsim/vcs_balanced.wsim            |   25 +
 7 files changed, 1133 insertions(+)
 create mode 100644 benchmarks/gem_wsim.c
 create mode 100644 benchmarks/wsim/README
 create mode 100644 benchmarks/wsim/media_17i7.wsim
 create mode 100644 benchmarks/wsim/media_load_balance_17i7.wsim
 create mode 100644 benchmarks/wsim/vcs1.wsim
 create mode 100644 benchmarks/wsim/vcs_balanced.wsim
benchmarks/Makefile.sources +1 -0
index 3af54ebe36f2..3a941150abb3 100644
@@ -14,6 +14,7 @@ benchmarks_prog_list =			\
 	gem_prw				\
 	gem_set_domain			\
 	gem_syslatency			\
+	gem_wsim			\
 	kms_vblank			\
 	prime_lookup			\
 	vgem_mmap			\
benchmarks/gem_wsim.c new file +1014 -0

1,014 lines not on this page · raw message

benchmarks/wsim/README new file +54 -0

54 lines not on this page · raw message

benchmarks/wsim/media_17i7.wsim new file +7 -0
new file mode 100644
index 000000000000..5f533d8e168b
@@ -0,0 +1,7 @@
+1.VCS1.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS2.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS2.600.-1.1
benchmarks/wsim/media_load_balance_17i7.wsim new file +7 -0
new file mode 100644
index 000000000000..25a692032eae
@@ -0,0 +1,7 @@
+1.VCS.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS.600.-1.1
benchmarks/wsim/vcs1.wsim new file +25 -0
new file mode 100644
index 000000000000..e1986aadd65c
@@ -0,0 +1,25 @@
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
benchmarks/wsim/vcs_balanced.wsim new file +25 -0
new file mode 100644
index 000000000000..9a4b3d785db1
@@ -0,0 +1,25 @@
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#10 lore raw

Re: [PATCH i-g-t v4] benchmarks/gem_wsim: Command submission workload simulator

Quoted 33 lines · Tvrtko Ursulin
benchmarks/gem_wsim.c new file +33 -0
+static void
+alloc_step_batch(struct workload *wrk, struct w_step *w, struct w_step_eb *b,
+		 enum intel_engine_id engine, unsigned int flags)
+{
+	unsigned int bb_i, j = 0;
+
+	b->obj[j].handle = gem_create(fd, 4096);
+	b->obj[j].flags = EXEC_OBJECT_WRITE;
+	j++;
+
+	if (flags & SEQNO) {
+		b->obj[j].handle = wrk->status_page_handle;
+		j++;
+	}
+
+	bb_i = j++;
+	b->bb_sz = get_bb_sz(w->duration.max);
+	b->bb_handle = b->obj[bb_i].handle = gem_create(fd, b->bb_sz);
+	terminate_bb(w, b, engine, flags);
+
+	igt_assert(w->dependency <= 0);
+	if (w->dependency) {
+		int dep_idx = w->idx + w->dependency;
+
+		igt_assert(dep_idx >= 0 && dep_idx < wrk->nr_steps);
+		igt_assert(wrk->steps[dep_idx].type == BATCH);
+
+		b->obj[j].handle = b->obj[bb_i].handle;
+		bb_i = j;
+		b->obj[j - 1].handle = wrk->steps[dep_idx].b[0].obj[0].handle;
+		j++;
+
+		if (wrk->steps[dep_idx].b[1].obj[0].handle) {

benchmarks/gem_wsim.c line 472

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+			b->obj[j].handle = b->obj[bb_i].handle;
+			bb_i = j;
+			b->obj[j - 1].handle =
+					wrk->steps[dep_idx].b[1].obj[0].handle;
+			j++;
+		}
+	}
+
+	if (flags & SEQNO) {
+		b->reloc.presumed_offset = -1;

So as I understand it, you are caching the execbuf/obj/reloc for the workload and then may reissue later with different seqno on different rings? In which case we have a problem as the kernel will write back the updated offsets to b->reloc.presumed_offset and b->obj[].offset and in future passes they will match and the seqno write will go into the wrong slot (if it swaps rings).

You either want to reset presumed_offset=-1 each time, or better for all concerned write the correct address alongside the seqno (which also enables NORELOC).

Delta incoming.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#11 lore raw

Re: [PATCH i-g-t v4] benchmarks/gem_wsim: Command submission workload simulator

Quoted 5 lines · Chris Wilson

You either want to reset presumed_offset=-1 each time, or better for all concerned write the correct address alongside the seqno (which also enables NORELOC).

Delta incoming.

See attached.

Next concern is that I have full rings which implies that we are not waiting on each batch before resubmitting with a new seqno?

If I throw a assert(!busy(batch_bo)) before the *b->mapped_seqno am I going to be upset?
-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre
Quoted 1 line

From 5bf424c2719e81f926b74f4136610cbdfd26a4d8 Mon Sep 17 00:00:00 2001

From: Chris Wilson <[email protected]>
Date: Thu, 20 Apr 2017 15:30:07 +0100
Subject: [PATCH] seqno-reloc

---
 benchmarks/gem_wsim.c | 17 +++++++++++------
 1 file changed, 11 insertions(+), 6 deletions(-)
benchmarks/gem_wsim.c +11 -6
index adf2d6de..e616335b 100644
@@ -45,6 +45,8 @@
 #include "drmtest.h"
 #include "intel_io.h"
 
+#define LOCAL_I915_GEM_DOMAIN_WC 0x80
+
 enum intel_engine_id {
 	RCS,
 	BCS,
@@ -86,7 +88,9 @@ struct w_step
 		struct drm_i915_gem_relocation_entry reloc;
 		unsigned long bb_sz;
 		uint32_t bb_handle;
-		uint32_t *mapped_batch, *mapped_seqno;
+		uint32_t *mapped_batch;
+		uint64_t *mapped_address;
+		uint32_t *mapped_seqno;
 		unsigned int mapped_len;
 	} b[2]; /* One for each VCS when load balancing */
 };
@@ -405,7 +409,8 @@ terminate_bb(struct w_step *w, struct w_step_eb *b, enum intel_engine_id engine,
 	mmap_len += cmd_offset - mmap_start;
 
 	gem_set_domain(fd, b->bb_handle,
-		       I915_GEM_DOMAIN_CPU, I915_GEM_DOMAIN_CPU);
+		       LOCAL_I915_GEM_DOMAIN_WC,
+		       LOCAL_I915_GEM_DOMAIN_WC);
 
 	ptr = gem_mmap__wc(fd, b->bb_handle, mmap_start, mmap_len, PROT_WRITE);
 	cs = (uint32_t *)((char *)ptr + cmd_offset - mmap_start);
@@ -415,6 +420,7 @@ terminate_bb(struct w_step *w, struct w_step_eb *b, enum intel_engine_id engine,
 		b->reloc.delta = (engine - VCS1) * sizeof(uint32_t);
 
 		*cs++ = MI_STORE_DWORD_IMM;
+		b->mapped_address = (uint64_t *)cs;
 		*cs++ = 0;
 		*cs++ = 0;
 		b->mapped_seqno = cs;
@@ -469,7 +475,6 @@ alloc_step_batch(struct workload *wrk, struct w_step *w, struct w_step_eb *b,
 	}
 
 	if (flags & SEQNO) {
-		b->reloc.presumed_offset = -1;
 		b->reloc.target_handle = 1;
 		b->obj[bb_i].relocs_ptr = to_user_pointer(&b->reloc);
 		b->obj[bb_i].relocation_count = 1;
@@ -485,8 +490,7 @@ alloc_step_batch(struct workload *wrk, struct w_step *w, struct w_step_eb *b,
 		engine = VCS1;
 	b->eb.flags = eb_engine_map[engine];
 	b->eb.flags |= I915_EXEC_HANDLE_LUT;
-	if (!(flags & SEQNO))
-		b->eb.flags |= I915_EXEC_NO_RELOC;
+	b->eb.flags |= I915_EXEC_NO_RELOC;
 #ifdef DEBUG
 	printf("%u: %u:%x|%x|%x|%x %10lu flags=%llx bb=%x[%u] ctx[%u]=%u\n",
 		w->idx, b->eb.buffer_count, b->obj[0].handle,
@@ -628,8 +632,9 @@ static void
 update_bb_seqno(struct w_step_eb *b, enum intel_engine_id engine,
 		uint32_t seqno)
 {
-	*b->mapped_seqno = seqno;
 	b->reloc.delta = (engine - VCS1) * sizeof(uint32_t);
+	*b->mapped_address = b->reloc.presumed_offset + b->reloc.delta;
+	*b->mapped_seqno = seqno;
 }
 
 static void
Signature · 1 line
2.11.0

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#12 lore raw

Re: [PATCH i-g-t v4] benchmarks/gem_wsim: Command submission workload simulator

Quoted 7 lines · Chris Wilson
Quoted 5 lines · Chris Wilson

You either want to reset presumed_offset=-1 each time, or better for all concerned write the correct address alongside the seqno (which also enables NORELOC).

Delta incoming.

Quoted 7 lines

See attached.

Next concern is that I have full rings which implies that we are not waiting on each batch before resubmitting with a new seqno?

If I throw a assert(!busy(batch_bo)) before the *b->mapped_seqno am I going to be upset?

Yes you would. :) I had a sync (as a move to cpu domain) before seqno update in the last version but it disappeared as I was fixing the whole area of seqno tracking. So the balancing results in the patch are bogus since the seqno can jump to latest ahead of the time...

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#13 lore raw

Re: [PATCH i-g-t v4] benchmarks/gem_wsim: Command submission workload simulator

In reply to Chris Wilson
Quoted 53 lines · Chris Wilson
Quoted 33 lines · Tvrtko Ursulin
benchmarks/gem_wsim.c new file +33 -0
+static void
+alloc_step_batch(struct workload *wrk, struct w_step *w, struct w_step_eb *b,
+		 enum intel_engine_id engine, unsigned int flags)
+{
+	unsigned int bb_i, j = 0;
+
+	b->obj[j].handle = gem_create(fd, 4096);
+	b->obj[j].flags = EXEC_OBJECT_WRITE;
+	j++;
+
+	if (flags & SEQNO) {
+		b->obj[j].handle = wrk->status_page_handle;
+		j++;
+	}
+
+	bb_i = j++;
+	b->bb_sz = get_bb_sz(w->duration.max);
+	b->bb_handle = b->obj[bb_i].handle = gem_create(fd, b->bb_sz);
+	terminate_bb(w, b, engine, flags);
+
+	igt_assert(w->dependency <= 0);
+	if (w->dependency) {
+		int dep_idx = w->idx + w->dependency;
+
+		igt_assert(dep_idx >= 0 && dep_idx < wrk->nr_steps);
+		igt_assert(wrk->steps[dep_idx].type == BATCH);
+
+		b->obj[j].handle = b->obj[bb_i].handle;
+		bb_i = j;
+		b->obj[j - 1].handle = wrk->steps[dep_idx].b[0].obj[0].handle;
+		j++;
+
+		if (wrk->steps[dep_idx].b[1].obj[0].handle) {

benchmarks/gem_wsim.c line 472

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+			b->obj[j].handle = b->obj[bb_i].handle;
+			bb_i = j;
+			b->obj[j - 1].handle =
+					wrk->steps[dep_idx].b[1].obj[0].handle;
+			j++;
+		}
+	}
+
+	if (flags & SEQNO) {
+		b->reloc.presumed_offset = -1;

So as I understand it, you are caching the execbuf/obj/reloc for the workload and then may reissue later with different seqno on different rings? In which case we have a problem as the kernel will write back the >

updated offsets to b->reloc.presumed_offset and b->obj[].offset and in future passes they will match and the seqno write will go into the wrong slot (if it swaps rings).

Quoted 5 lines

You either want to reset presumed_offset=-1 each time, or better for all concerned write the correct address alongside the seqno (which also enables NORELOC).

Delta incoming.

Only the seqno changes, but each engine has its own eb/obj/reloc. So I think there is no problem. Or is there still?

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#14 lore raw

Re: [PATCH i-g-t v4] benchmarks/gem_wsim: Command submission workload simulator

Quoted 55 lines · Tvrtko Ursulin
Quoted 53 lines · Chris Wilson
Quoted 33 lines · Tvrtko Ursulin
benchmarks/gem_wsim.c new file +33 -0
+static void
+alloc_step_batch(struct workload *wrk, struct w_step *w, struct w_step_eb *b,
+		 enum intel_engine_id engine, unsigned int flags)
+{
+	unsigned int bb_i, j = 0;
+
+	b->obj[j].handle = gem_create(fd, 4096);
+	b->obj[j].flags = EXEC_OBJECT_WRITE;
+	j++;
+
+	if (flags & SEQNO) {
+		b->obj[j].handle = wrk->status_page_handle;
+		j++;
+	}
+
+	bb_i = j++;
+	b->bb_sz = get_bb_sz(w->duration.max);
+	b->bb_handle = b->obj[bb_i].handle = gem_create(fd, b->bb_sz);
+	terminate_bb(w, b, engine, flags);
+
+	igt_assert(w->dependency <= 0);
+	if (w->dependency) {
+		int dep_idx = w->idx + w->dependency;
+
+		igt_assert(dep_idx >= 0 && dep_idx < wrk->nr_steps);
+		igt_assert(wrk->steps[dep_idx].type == BATCH);
+
+		b->obj[j].handle = b->obj[bb_i].handle;
+		bb_i = j;
+		b->obj[j - 1].handle = wrk->steps[dep_idx].b[0].obj[0].handle;
+		j++;
+
+		if (wrk->steps[dep_idx].b[1].obj[0].handle) {

benchmarks/gem_wsim.c line 472

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+			b->obj[j].handle = b->obj[bb_i].handle;
+			bb_i = j;
+			b->obj[j - 1].handle =
+					wrk->steps[dep_idx].b[1].obj[0].handle;
+			j++;
+		}
+	}
+
+	if (flags & SEQNO) {
+		b->reloc.presumed_offset = -1;

So as I understand it, you are caching the execbuf/obj/reloc for the workload and then may reissue later with different seqno on different rings? In which case we have a problem as the kernel will write back the >

updated offsets to b->reloc.presumed_offset and b->obj[].offset and in future passes they will match and the seqno write will go into the wrong slot (if it swaps rings).

Quoted 8 lines
Quoted 5 lines

You either want to reset presumed_offset=-1 each time, or better for all concerned write the correct address alongside the seqno (which also enables NORELOC).

Delta incoming.

Only the seqno changes, but each engine has its own eb/obj/reloc. So I think there is no problem. Or is there still?

bb_handle is per engine as well. Ugh. No, that seems like you are self-consistent, you just need to remove the -1 and your code is NORELOC correct.

I wouldn't go so far as having entirely separate batches for each engine though :)
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#15 lore raw

Re: [PATCH i-g-t v4] benchmarks/gem_wsim: Command submission workload simulator

In reply to Tvrtko Ursulin
Quoted 12 lines · Tvrtko Ursulin
benchmarks/gem_wsim.c new file +12 -0
+			wrk->nr_bb[engine]++;
+
+			if (engine == VCS && balance) {
+				engine = balance(wrk, w);
+				wrk->nr_bb[engine]++;
+				b = &w->b[engine - VCS1];
+
+				if (flags & SEQNO)
+					update_bb_seqno(b, engine,
+							++wrk->seqno[engine]);
+			}
+

benchmarks/gem_wsim.c line 712

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+			if (w->duration.min != w->duration.max) {
+				unsigned int d = get_duration(&w->duration);
+				unsigned long offset;
+
+				offset = ALIGN(b->bb_sz - get_bb_sz(d),
+					       2 * sizeof(uint32_t));
+				b->eb.batch_start_offset = offset;
+			}
+
+			gem_execbuf(fd, &b->eb);

Likely double counting wrk->nr_bb. I suggest placing it next to the gem_execbuf().
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#16 lore raw

Re: [PATCH i-g-t v4] benchmarks/gem_wsim: Command submission workload simulator

Quoted 18 lines · Chris Wilson
Quoted 17 lines · Tvrtko Ursulin
benchmarks/gem_wsim.c new file +17 -0
+			wrk->nr_bb[engine]++;
+
+			if (engine == VCS && balance) {
+				engine = balance(wrk, w);
+				wrk->nr_bb[engine]++;
+				b = &w->b[engine - VCS1];
+
+				if (flags & SEQNO)
+					update_bb_seqno(b, engine,
+							++wrk->seqno[engine]);
+			}
+
+			if (w->duration.min != w->duration.max) {
+				unsigned int d = get_duration(&w->duration);
+				unsigned long offset;
+
+				offset = ALIGN(b->bb_sz - get_bb_sz(d),
Quoted 8 lines

benchmarks/gem_wsim.c line 712

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+					       2 * sizeof(uint32_t));
+				b->eb.batch_start_offset = offset;
+			}
+
+			gem_execbuf(fd, &b->eb);

Likely double counting wrk->nr_bb. I suggest placing it next to the gem_execbuf().

Just convenience in balancing mode so that nr(VCS) = nr(VCS1) + nr(VCS2). Also from a different angle, if the sum does not hold, that means workload had auto-balancing and explicit VCS1/2 batches. It's only used to print out the stats at the end of the run.

Regards,

Tvrtko

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#17 lore raw

Re: [PATCH i-g-t v4] benchmarks/gem_wsim: Command submission workload simulator

In reply to Tvrtko Ursulin
Quoted 5 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

benchmarks/gem_wsim.c new file +4 -0
+static void
+run_workload(unsigned int id, struct workload *wrk, unsigned int repeat,
+	     enum intel_engine_id (*balance)(struct workload *wrk,
+					     struct w_step *w),

benchmarks/gem_wsim.c line 648

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+	     unsigned int flags)
+{
+	struct timespec t_start, t_end;
+	struct w_step *w;
+	double t;
+	int i, j;
+
+	clock_gettime(CLOCK_MONOTONIC, &t_start);
+
+	srand(t_start.tv_nsec);

Let's supply the seed with the workload specification. And use a portable prng so we can be sure we can reproduce results from one system to the next.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#18 lore raw

PATCH v5 1/1 benchmarks/gem_wsim: Command submission workload simulator

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Tool which emits batch buffers to engines with configurable sequences, durations, contexts, dependencies and userspace waits.

Unfinished but shows promise so sending out for early feedback.

v2:
* Load workload descriptors from files. (also -w)
* Help text.
* Calibration control if needed. (-t)
* NORELOC | LUT to eb flags.
* Added sample workload to wsim/workload1.

v3:
* Multiple parallel different workloads (-w -w ...).
* Multi-context workloads.
* Variable (random) batch length.
* Load balancing (round robin and queue depth estimation).
* Workloads delays and explicit sync steps.
* Workload frequency (period) control.

v4:
* Fixed queue-depth estimation by creating separate batches per engine when qd load balancing is on.
* Dropped separate -s cmd line option. It can turn itself on automatically when needed.
* Keep a single status page and lie about the write hazard as suggested by Chris.
* Use batch_start_offset for controlling the batch duration. (Chris)
* Set status page object cache level. (Chris)
* Moved workload description to a README.
* Tidied example workloads.
* Some other cleanups and refactorings.

v5:
* Master and background workloads (-W / -w).
* Single batch per step is enough even when balancing. (Chris)
* Use hars_petruska_f54_1_random IGT functions and see to zero at start. (Chris)
* Use WC cache domain when WC mapping. (Chris)
* Keep seqnos 64-bytes apart in the status page. (Chris)
* Add workload throttling and queue-depth throttling commands. (Chris)

TODO list:

* Fence support.
* Better error handling.
* Less 1980's workload parsing.
* Proper workloads.
* Threads?
* ... ?

Cc Chris Wilson <[email protected]>
Cc "Rogozhkin, Dmitry V" <[email protected]>
---
 benchmarks/Makefile.sources                  |    1 +
 benchmarks/gem_wsim.c                        | 1189 ++++++++++++++++++++++++++
 benchmarks/wsim/README                       |   56 ++
 benchmarks/wsim/media_17i7.wsim              |    7 +
 benchmarks/wsim/media_load_balance_17i7.wsim |    7 +
 benchmarks/wsim/vcs1.wsim                    |   26 +
 benchmarks/wsim/vcs_balanced.wsim            |   26 +
 lib/igt_core.c                               |   26 +
 lib/igt_core.h                               |    1 +
 9 files changed, 1339 insertions(+)
 create mode 100644 benchmarks/gem_wsim.c
 create mode 100644 benchmarks/wsim/README
 create mode 100644 benchmarks/wsim/media_17i7.wsim
 create mode 100644 benchmarks/wsim/media_load_balance_17i7.wsim
 create mode 100644 benchmarks/wsim/vcs1.wsim
 create mode 100644 benchmarks/wsim/vcs_balanced.wsim
benchmarks/Makefile.sources +1 -0
index 3af54ebe36f2..3a941150abb3 100644
@@ -14,6 +14,7 @@ benchmarks_prog_list =			\
 	gem_prw				\
 	gem_set_domain			\
 	gem_syslatency			\
+	gem_wsim			\
 	kms_vblank			\
 	prime_lookup			\
 	vgem_mmap			\
benchmarks/gem_wsim.c new file +1189 -0

1,189 lines not on this page · raw message

benchmarks/wsim/README new file +56 -0

56 lines not on this page · raw message

benchmarks/wsim/media_17i7.wsim new file +7 -0
new file mode 100644
index 000000000000..5f533d8e168b
@@ -0,0 +1,7 @@
+1.VCS1.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS2.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS2.600.-1.1
benchmarks/wsim/media_load_balance_17i7.wsim new file +7 -0
new file mode 100644
index 000000000000..25a692032eae
@@ -0,0 +1,7 @@
+1.VCS.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS.600.-1.1
benchmarks/wsim/vcs1.wsim new file +26 -0
new file mode 100644
index 000000000000..9d3e682b5ce8
@@ -0,0 +1,26 @@
+t.5
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
benchmarks/wsim/vcs_balanced.wsim new file +26 -0
new file mode 100644
index 000000000000..e8958b8f7f43
@@ -0,0 +1,26 @@
+q.5
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
lib/igt_core.c +26 -0
index 403b9423fa9f..9c3b37fe3d63 100644
@@ -1558,6 +1558,32 @@ bool __igt_fork(void)
 }
 
 /**
+ * igt_child_done:
+ *
+ * Lets the IGT core know that one of the children has exited.
+ */
+void igt_child_done(pid_t pid)
+{
+	int i = 0;
+	int found = -1;
+
+	igt_assert(num_test_children > 1);
+
+	for (i = 0; i < num_test_children; i++) {
+		if (pid == test_children[i]) {
+			found = i;
+			break;
+		}
+	}
+
+	igt_assert(found >= 0);
+
+	num_test_children--;
+	for (i = found; i < num_test_children; i++)
+		test_children[i] = test_children[i + 1];
+}
+
+/**
  * igt_waitchildren:
  *
  * Wait for all children forked with igt_fork.
lib/igt_core.h +1 -0
index 51b98d82ef7f..4a125af1d6a5 100644
@@ -688,6 +688,7 @@ bool __igt_fork(void);
 #define igt_fork(child, num_children) \
 	for (int child = 0; child < (num_children); child++) \
 		for (; __igt_fork(); exit(0))
+void igt_child_done(pid_t pid);
 void igt_waitchildren(void);
 void igt_waitchildren_timeout(int seconds, const char *reason);
 
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#19 lore raw

PATCH v6 1/1 benchmarks/gem_wsim: Command submission workload simulator

From: Tvrtko Ursulin <[email protected]>

Tool which emits batch buffers to engines with configurable sequences, durations, contexts, dependencies and userspace waits.

Unfinished but shows promise so sending out for early feedback.

v2:
* Load workload descriptors from files. (also -w)
* Help text.
* Calibration control if needed. (-t)
* NORELOC | LUT to eb flags.
* Added sample workload to wsim/workload1.

v3:
* Multiple parallel different workloads (-w -w ...).
* Multi-context workloads.
* Variable (random) batch length.
* Load balancing (round robin and queue depth estimation).
* Workloads delays and explicit sync steps.
* Workload frequency (period) control.

v4:
* Fixed queue-depth estimation by creating separate batches per engine when qd load balancing is on.
* Dropped separate -s cmd line option. It can turn itself on automatically when needed.
* Keep a single status page and lie about the write hazard as suggested by Chris.
* Use batch_start_offset for controlling the batch duration. (Chris)
* Set status page object cache level. (Chris)
* Moved workload description to a README.
* Tidied example workloads.
* Some other cleanups and refactorings.

v5:
* Master and background workloads (-W / -w).
* Single batch per step is enough even when balancing. (Chris)
* Use hars_petruska_f54_1_random IGT functions and see to zero at start. (Chris)
* Use WC cache domain when WC mapping. (Chris)
* Keep seqnos 64-bytes apart in the status page. (Chris)
* Add workload throttling and queue-depth throttling commands. (Chris)

v6:
* Added two more workloads.
* Merged RT balancer from Chris.

TODO list:

* Fence support.
* Better error handling.
* Less 1980's workload parsing.
* More workloads.
* Threads?
* ... ?

Cc Chris Wilson <[email protected]>
Cc "Rogozhkin, Dmitry V" <[email protected]>
---
 benchmarks/Makefile.sources                  |    1 +
 benchmarks/gem_wsim.c                        | 1307 ++++++++++++++++++++++++++
 benchmarks/wsim/README                       |   56 ++
 benchmarks/wsim/media_17i7.wsim              |    7 +
 benchmarks/wsim/media_19.wsim                |   10 +
 benchmarks/wsim/media_load_balance_17i7.wsim |    7 +
 benchmarks/wsim/media_load_balance_19.wsim   |   10 +
 benchmarks/wsim/vcs1.wsim                    |   26 +
 benchmarks/wsim/vcs_balanced.wsim            |   26 +
 lib/igt_core.c                               |   26 +
 lib/igt_core.h                               |    1 +
 11 files changed, 1477 insertions(+)
 create mode 100644 benchmarks/gem_wsim.c
 create mode 100644 benchmarks/wsim/README
 create mode 100644 benchmarks/wsim/media_17i7.wsim
 create mode 100644 benchmarks/wsim/media_19.wsim
 create mode 100644 benchmarks/wsim/media_load_balance_17i7.wsim
 create mode 100644 benchmarks/wsim/media_load_balance_19.wsim
 create mode 100644 benchmarks/wsim/vcs1.wsim
 create mode 100644 benchmarks/wsim/vcs_balanced.wsim
benchmarks/Makefile.sources +1 -0
index 3af54ebe36f2..3a941150abb3 100644
@@ -14,6 +14,7 @@ benchmarks_prog_list =			\
 	gem_prw				\
 	gem_set_domain			\
 	gem_syslatency			\
+	gem_wsim			\
 	kms_vblank			\
 	prime_lookup			\
 	vgem_mmap			\
benchmarks/gem_wsim.c new file +1307 -0

1,307 lines not on this page · raw message

benchmarks/wsim/README new file +56 -0

56 lines not on this page · raw message

benchmarks/wsim/media_17i7.wsim new file +7 -0
new file mode 100644
index 000000000000..5f533d8e168b
@@ -0,0 +1,7 @@
+1.VCS1.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS2.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS2.600.-1.1
benchmarks/wsim/media_19.wsim new file +10 -0
new file mode 100644
index 000000000000..f210d7940800
@@ -0,0 +1,10 @@
+0.VECS.1400-1500.0.0
+0.RCS.1000-1500.-1.0
+s.-2
+2.VCS2.50-350.0.1
+1.VCS1.1300-1400.0.1
+0.VECS.1400-1500.0.0
+0.RCS.100-300.-1.1
+2.RCS.1300-1500.0.0
+2.VCS2.100-300.-1.1
+1.VCS1.900-1400.0.1
benchmarks/wsim/media_load_balance_17i7.wsim new file +7 -0
new file mode 100644
index 000000000000..25a692032eae
@@ -0,0 +1,7 @@
+1.VCS.3000.0.1
+1.RCS.1000.-1.0
+1.RCS.3700.0.0
+1.RCS.1000.-2.0
+1.VCS.2300.-2.0
+1.RCS.4700.-1.0
+1.VCS.600.-1.1
benchmarks/wsim/media_load_balance_19.wsim new file +10 -0
new file mode 100644
index 000000000000..03890776fda3
@@ -0,0 +1,10 @@
+0.VECS.1400-1500.0.0
+0.RCS.1000-1500.-1.0
+s.-2
+1.VCS.50-350.0.1
+1.VCS.1300-1400.0.1
+0.VECS.1400-1500.0.0
+0.RCS.100-300.-1.1
+1.RCS.1300-1500.0.0
+1.VCS.100-300.-1.1
+1.VCS.900-1400.0.1
benchmarks/wsim/vcs1.wsim new file +26 -0
new file mode 100644
index 000000000000..9d3e682b5ce8
@@ -0,0 +1,26 @@
+t.5
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
+0.VCS1.500-2000.0.0
benchmarks/wsim/vcs_balanced.wsim new file +26 -0
new file mode 100644
index 000000000000..e8958b8f7f43
@@ -0,0 +1,26 @@
+q.5
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
+0.VCS.500-2000.0.0
lib/igt_core.c +26 -0
index 403b9423fa9f..9c3b37fe3d63 100644
@@ -1558,6 +1558,32 @@ bool __igt_fork(void)
 }
 
 /**
+ * igt_child_done:
+ *
+ * Lets the IGT core know that one of the children has exited.
+ */
+void igt_child_done(pid_t pid)
+{
+	int i = 0;
+	int found = -1;
+
+	igt_assert(num_test_children > 1);
+
+	for (i = 0; i < num_test_children; i++) {
+		if (pid == test_children[i]) {
+			found = i;
+			break;
+		}
+	}
+
+	igt_assert(found >= 0);
+
+	num_test_children--;
+	for (i = found; i < num_test_children; i++)
+		test_children[i] = test_children[i + 1];
+}
+
+/**
  * igt_waitchildren:
  *
  * Wait for all children forked with igt_fork.
lib/igt_core.h +1 -0
index 51b98d82ef7f..4a125af1d6a5 100644
@@ -688,6 +688,7 @@ bool __igt_fork(void);
 #define igt_fork(child, num_children) \
 	for (int child = 0; child < (num_children); child++) \
 		for (; __igt_fork(); exit(0))
+void igt_child_done(pid_t pid);
 void igt_waitchildren(void);
 void igt_waitchildren_timeout(int seconds, const char *reason);
 
Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#20 lore raw

Re: [PATCH i-g-t v6] benchmarks/gem_wsim: Command submission workload simulator

Quoted 46 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Tool which emits batch buffers to engines with configurable sequences, durations, contexts, dependencies and userspace waits.

Unfinished but shows promise so sending out for early feedback.

v2:
* Load workload descriptors from files. (also -w)
* Help text.
* Calibration control if needed. (-t)
* NORELOC | LUT to eb flags.
* Added sample workload to wsim/workload1.

v3:
* Multiple parallel different workloads (-w -w ...).
* Multi-context workloads.
* Variable (random) batch length.
* Load balancing (round robin and queue depth estimation).
* Workloads delays and explicit sync steps.
* Workload frequency (period) control.

v4:
* Fixed queue-depth estimation by creating separate batches per engine when qd load balancing is on.
* Dropped separate -s cmd line option. It can turn itself on automatically when needed.
* Keep a single status page and lie about the write hazard as suggested by Chris.
* Use batch_start_offset for controlling the batch duration. (Chris)
* Set status page object cache level. (Chris)
* Moved workload description to a README.
* Tidied example workloads.
* Some other cleanups and refactorings.

v5:
* Master and background workloads (-W / -w).
* Single batch per step is enough even when balancing. (Chris)
* Use hars_petruska_f54_1_random IGT functions and see to zero at start. (Chris)
* Use WC cache domain when WC mapping. (Chris)
* Keep seqnos 64-bytes apart in the status page. (Chris)
* Add workload throttling and queue-depth throttling commands. (Chris)

Quoted 5 lines

v6:
* Added two more workloads.
* Merged RT balancer from Chris.

TODO list:

* No reloc!
* bb caching/reuse

Quoted 18 lines

* Fence support.
* Better error handling.
* Less 1980's workload parsing.
* More workloads.
* Threads?
* ... ?

Cc Chris Wilson <[email protected]>
Cc "Rogozhkin, Dmitry V" <[email protected]>

---

benchmarks/gem_wsim.c new file +6 -0
+static enum intel_engine_id
+rt_balance(const struct workload_balancer *balancer,
+	   struct workload *wrk, struct w_step *w)
+{
+	enum intel_engine_id engine;
+	long qd[NUM_ENGINES];

benchmarks/gem_wsim.c line 734

Quoted 10 lines
benchmarks/gem_wsim.c new file +10 -0
+	unsigned int n;
+
+	igt_assert(w->engine == VCS);
+
+	/* Estimate the "speed" of the most recent batch
+	 *    (finish time - submit time)
+	 * and use that as an approximate for the total remaining time for
+	 * all batches on that engine. We try to keep the total remaining
+	 * balanced between the engines.
+	 */

Next steps for this would be to move from an instantaneous speed, to an average. I'm thinking something like a exponential decay moving average just to make the estimation more robust.

benchmarks/gem_wsim.c line 925

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+			if (qd_throttle > 0 && balancer && balancer->get_qd) {
+				unsigned int target;
+
+				for (target = wrk->nr_steps - 1; target > 0;
+				     target--) {

I think this should skip other engines.

if (target->engine != engine)

	continue;
Quoted 6 lines
benchmarks/gem_wsim.c new file +6 -0
+					if (balancer->get_qd(balancer, wrk,
+							     engine) <
+					    qd_throttle)
+						break;
+					w_sync_to(wrk, w, i - target);
+				}
Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#21 lore raw

Re: [PATCH i-g-t v6] benchmarks/gem_wsim: Command submission workload simulator

Quoted 47 lines · Chris Wilson
Quoted 46 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Tool which emits batch buffers to engines with configurable sequences, durations, contexts, dependencies and userspace waits.

Unfinished but shows promise so sending out for early feedback.

v2:
* Load workload descriptors from files. (also -w)
* Help text.
* Calibration control if needed. (-t)
* NORELOC | LUT to eb flags.
* Added sample workload to wsim/workload1.

v3:
* Multiple parallel different workloads (-w -w ...).
* Multi-context workloads.
* Variable (random) batch length.
* Load balancing (round robin and queue depth estimation).
* Workloads delays and explicit sync steps.
* Workload frequency (period) control.

v4:
* Fixed queue-depth estimation by creating separate batches per engine when qd load balancing is on.
* Dropped separate -s cmd line option. It can turn itself on automatically when needed.
* Keep a single status page and lie about the write hazard as suggested by Chris.
* Use batch_start_offset for controlling the batch duration. (Chris)
* Set status page object cache level. (Chris)
* Moved workload description to a README.
* Tidied example workloads.
* Some other cleanups and refactorings.

v5:
* Master and background workloads (-W / -w).
* Single batch per step is enough even when balancing. (Chris)
* Use hars_petruska_f54_1_random IGT functions and see to zero at start. (Chris)
* Use WC cache domain when WC mapping. (Chris)
* Keep seqnos 64-bytes apart in the status page. (Chris)
* Add workload throttling and queue-depth throttling commands. (Chris)

Quoted 8 lines
Quoted 5 lines

v6:
* Added two more workloads.
* Merged RT balancer from Chris.

TODO list:

* No reloc!
* bb caching/reuse

Yeah I know, but have to progress the overall case as well and I am thinking it is getting close to good enough now. So now is the time to think of interesting workloads, and workload combinations.

Quoted 24 lines
Quoted 24 lines

* Fence support.
* Better error handling.
* Less 1980's workload parsing.
* More workloads.
* Threads?
* ... ?

Cc Chris Wilson <[email protected]>
Cc "Rogozhkin, Dmitry V" <[email protected]>

---

benchmarks/gem_wsim.c new file +12 -0
+static enum intel_engine_id
+rt_balance(const struct workload_balancer *balancer,
+	   struct workload *wrk, struct w_step *w)
+{
+	enum intel_engine_id engine;
+	long qd[NUM_ENGINES];
+	unsigned int n;
+
+	igt_assert(w->engine == VCS);
+
+	/* Estimate the "speed" of the most recent batch
+	 *    (finish time - submit time)
Quoted 8 lines

benchmarks/gem_wsim.c line 734

Quoted 4 lines
benchmarks/gem_wsim.c new file +4 -0
+	 * and use that as an approximate for the total remaining time for
+	 * all batches on that engine. We try to keep the total remaining
+	 * balanced between the engines.
+	 */

Next steps for this would be to move from an instantaneous speed, to an average. I'm thinking something like a exponential decay moving average just to make the estimation more robust.

Do you think it would be OK to merge these two tools at this point and continue improving them in place?

Your balancer already looks a solid step up from the queue-depth one. I checked today myself and, what looks like a worst case of a VCS1 hog and a balancing workloads running together, it gets the VCS2 utilisation to impressive 85%.

As mentioned before those stats can now be collected easily with:

  trace.pl --trace gem_wsim ...; perf script | trace.pl

I need to start pining the relevant people for help with creating relevant workloads and am also entertaining the idea of trying balancing via exporting the stats from i915 directly. Just to see if true vs estimated numbers would make a difference here.

Quoted 10 lines

benchmarks/gem_wsim.c line 925

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+			if (qd_throttle > 0 && balancer && balancer->get_qd) {
+				unsigned int target;
+
+				for (target = wrk->nr_steps - 1; target > 0;
+				     target--) {

I think this should skip other engines.

if (target->engine != engine)

	continue;

If you say so. I don't have an opinion on it. Would it be useful to perhaps have both options - to throttle globally and per-engine? I could easily add two different workload commands for that.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#22 lore raw

Re: [PATCH i-g-t v6] benchmarks/gem_wsim: Command submission workload simulator

Quoted 4 lines · Tvrtko Ursulin
Quoted 2 lines · Chris Wilson
Quoted 1 line · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

[snip]

Quoted 21 lines
Quoted 12 lines
Quoted 12 lines
benchmarks/gem_wsim.c new file +12 -0
+static enum intel_engine_id
+rt_balance(const struct workload_balancer *balancer,
+	   struct workload *wrk, struct w_step *w)
+{
+	enum intel_engine_id engine;
+	long qd[NUM_ENGINES];
+	unsigned int n;
+
+	igt_assert(w->engine == VCS);
+
+	/* Estimate the "speed" of the most recent batch
+	 *    (finish time - submit time)
Quoted 8 lines

benchmarks/gem_wsim.c line 734

Quoted 4 lines
benchmarks/gem_wsim.c new file +4 -0
+	 * and use that as an approximate for the total remaining time for
+	 * all batches on that engine. We try to keep the total remaining
+	 * balanced between the engines.
+	 */

Next steps for this would be to move from an instantaneous speed, to an average. I'm thinking something like a exponential decay moving average just to make the estimation more robust.

Quoted 2 lines

Do you think it would be OK to merge these two tools at this point and continue improving them in place?

Yes. Although there's no excuse no to make this NO_RELOC from the start, especially if we want to demonstrate how it should be done! Hopefully attached the delta.

Quoted 4 lines

Your balancer already looks a solid step up from the queue-depth one. I checked today myself and, what looks like a worst case of a VCS1 hog and a balancing workloads running together, it gets the VCS2 utilisation to impressive 85%.

Yup. Just thinking of the danger in using an instantaneous value as our estimate and how to improve.

Quoted 8 lines

As mentioned before those stats can now be collected easily with:

  trace.pl --trace gem_wsim ...; perf script | trace.pl

I need to start pining the relevant people for help with creating relevant workloads and am also entertaining the idea of trying balancing via exporting the stats from i915 directly. Just to see if true vs estimated numbers would make a difference here.

Aye.

Quoted 8 lines
Quoted 8 lines

benchmarks/gem_wsim.c line 925

Quoted 5 lines
benchmarks/gem_wsim.c new file +5 -0
+			if (qd_throttle > 0 && balancer && balancer->get_qd) {
+				unsigned int target;
+
+				for (target = wrk->nr_steps - 1; target > 0;
+				     target--) {

I think this should skip other engines.

Quoted 6 lines
Quoted 2 lines

if (target->engine != engine)

	continue;

If you say so. I don't have an opinion on it. Would it be useful to perhaps have both options - to throttle globally and per-engine? I could easily add two different workload commands for that.

I'm leaning towards making it a balancer->callback(). -Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#23 lore raw

Re: [PATCH i-g-t v6] benchmarks/gem_wsim: Command submission workload simulator

Quoted 27 lines · Chris Wilson
Quoted 4 lines · Tvrtko Ursulin
Quoted 2 lines · Chris Wilson
Quoted 1 line · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

[snip]

Quoted 21 lines
Quoted 20 lines
Quoted 6 lines
(quoted diff) +6 -0
+static enum intel_engine_id
+rt_balance(const struct workload_balancer *balancer,
+	   struct workload *wrk, struct w_step *w)
+{
+	enum intel_engine_id engine;
+	long qd[NUM_ENGINES];

Review

Quoted 10 lines
(quoted diff) +10 -0
+	unsigned int n;
+
+	igt_assert(w->engine == VCS);
+
+	/* Estimate the "speed" of the most recent batch
+	 *    (finish time - submit time)
+	 * and use that as an approximate for the total remaining time for
+	 * all batches on that engine. We try to keep the total remaining
+	 * balanced between the engines.
+	 */

Next steps for this would be to move from an instantaneous speed, to an average. I'm thinking something like a exponential decay moving average just to make the estimation more robust.

Quoted 6 lines
Quoted 2 lines

Do you think it would be OK to merge these two tools at this point and continue improving them in place?

Yes. Although there's no excuse no to make this NO_RELOC from the start, especially if we want to demonstrate how it should be done! Hopefully attached the delta.

Which I forgot. Let's try again...
-Chris

Signature · 1 line
Chris Wilson, Intel Open Source Technology Centre
Quoted 1 line

From 985f873f1c9cdaec396c5410738910da04e8f95b Mon Sep 17 00:00:00 2001

From: Chris Wilson <[email protected]>
Date: Tue, 25 Apr 2017 13:22:39 +0100
Subject: [PATCH] no-reloc

---
 benchmarks/gem_wsim.c | 45 +++++++++++++++++++++++++++++----------------
 1 file changed, 29 insertions(+), 16 deletions(-)
benchmarks/gem_wsim.c +29 -16
index 1f491a5..f13477a 100644
@@ -90,9 +90,13 @@ struct w_step
 	struct drm_i915_gem_relocation_entry reloc[3];
 	unsigned long bb_sz;
 	uint32_t bb_handle;
-	uint32_t *mapped_batch, *mapped_seqno;
-	unsigned int mapped_len;
+	uint32_t *mapped_batch;
+	uint32_t *seqno_value;
+	uint32_t *seqno_address;
 	uint32_t *rt0_value;
+	uint32_t *rt0_address;
+	uint32_t *rt1_address;
+	unsigned int mapped_len;
 };
 
 struct workload
@@ -463,9 +467,10 @@ terminate_bb(struct w_step *w, unsigned int flags)
 		batch_start += 4 * sizeof(uint32_t);
 
 		*cs++ = MI_STORE_DWORD_IMM;
+		w->seqno_address = cs;
 		*cs++ = 0;
 		*cs++ = 0;
-		w->mapped_seqno = cs;
+		w->seqno_value = cs;
 		*cs++ = 0;
 	}
 
@@ -474,6 +479,7 @@ terminate_bb(struct w_step *w, unsigned int flags)
 		batch_start += 4 * sizeof(uint32_t);
 
 		*cs++ = MI_STORE_DWORD_IMM;
+		w->rt0_address = cs;
 		*cs++ = 0;
 		*cs++ = 0;
 		w->rt0_value = cs;
@@ -484,6 +490,7 @@ terminate_bb(struct w_step *w, unsigned int flags)
 
 		*cs++ = 0x24 << 23 | 2; /* MI_STORE_REG_MEM */
 		*cs++ = RCS_TIMESTAMP;
+		w->rt1_address = cs;
 		*cs++ = 0;
 		*cs++ = 0;
 	}
@@ -500,8 +507,7 @@ eb_update_flags(struct w_step *w, enum intel_engine_id engine,
 {
 	w->eb.flags = eb_engine_map[engine];
 	w->eb.flags |= I915_EXEC_HANDLE_LUT;
-	if (!(flags & SEQNO))
-		w->eb.flags |= I915_EXEC_NO_RELOC;
+	w->eb.flags |= I915_EXEC_NO_RELOC;
 }
 
 static void
@@ -543,10 +549,8 @@ alloc_step_batch(struct workload *wrk, struct w_step *w, unsigned int flags)
 			w->obj[bb_i].relocation_count = 3;
 		else
 			w->obj[bb_i].relocation_count = 1;
-		for (int i = 0; i < w->obj[bb_i].relocation_count; i++) {
-			w->reloc[i].presumed_offset = -1;
+		for (int i = 0; i < w->obj[bb_i].relocation_count; i++)
 			w->reloc[i].target_handle = 1;
-		}
 	}
 
 	w->eb.buffers_ptr = to_user_pointer(w->obj);
@@ -782,10 +786,14 @@ update_bb_seqno(struct w_step *w, enum intel_engine_id engine, uint32_t seqno)
 	gem_set_domain(fd, w->bb_handle,
 		       I915_GEM_DOMAIN_WC, I915_GEM_DOMAIN_WC);
 
-	*w->mapped_seqno = seqno;
-
-	w->reloc[0].presumed_offset = -1;
 	w->reloc[0].delta = VCS_SEQNO_OFFSET(engine);
+
+	*w->seqno_value = seqno;
+	*w->seqno_address = w->reloc[0].presumed_offset + w->reloc[0].delta;
+
+	/* If not using NO_RELOC, force the relocations */
+	if ((w->eb.flags & I915_EXEC_NO_RELOC))
+		w->reloc[0].presumed_offset = -1;
 }
 
 static void
@@ -796,13 +804,18 @@ update_bb_rt(struct w_step *w, enum intel_engine_id engine)
 	gem_set_domain(fd, w->bb_handle,
 		       I915_GEM_DOMAIN_WC, I915_GEM_DOMAIN_WC);
 
-	*w->rt0_value = *REG(RCS_TIMESTAMP);
-
-	w->reloc[1].presumed_offset = -1;
 	w->reloc[1].delta = VCS_SEQNO_OFFSET(engine) + sizeof(uint32_t);
-
-	w->reloc[2].presumed_offset = -1;
 	w->reloc[2].delta = VCS_SEQNO_OFFSET(engine) + 2 * sizeof(uint32_t);
+
+	*w->rt0_value = *REG(RCS_TIMESTAMP);
+	*w->rt0_address = w->reloc[1].presumed_offset + w->reloc[1].delta;
+	*w->rt1_address = w->reloc[1].presumed_offset + w->reloc[1].delta;
+
+	/* If not using NO_RELOC, force the relocations */
+	if ((w->eb.flags & I915_EXEC_NO_RELOC)) {
+		w->reloc[1].presumed_offset = -1;
+		w->reloc[2].presumed_offset = -1;
+	}
 }
 
 static void w_sync_to(struct workload *wrk, struct w_step *w, int target)
Signature · 1 line
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx

#24 lore raw

PATCH 2/2 igt/scripts: trace.pl to parse the i915 tracepoints

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Given a log file created via perf with some interesting trace events enabled, this tool can generate the timeline graph of requests getting queued, their dependencies resolved, sent to the GPU for executing and finally completed.

This can be useful when analyzing certain classes of performance issues. More help is available in the tool itself.

The tool will also calculate some overall per engine statistics, like total time engine was idle and similar.

v2:
* Address missing git add.
* Make html output optional (--html switch) and by default just output aggregated per engine stats to stdout.

v3:
* Added --trace option which invokes perf with the correct options automatically.
* Added --avg-delay-stats which prints averages for things like waiting on ready, waiting on GPU and context save duration.
* Fix warnings when no waits on an engine.
* Correct help text.

Cc Chris Wilson <[email protected]>
Cc Harri Syrja <[email protected]>
Cc Krzysztof E Olinski <[email protected]>
---
 scripts/Makefile.am |   2 +-
 scripts/trace.pl    | 946 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 947 insertions(+), 1 deletion(-)
 create mode 100755 scripts/trace.pl
scripts/Makefile.am +1 -1
index 85d4a5cf4e5c..641715294936 100644
@@ -1,2 +1,2 @@
-dist_noinst_SCRIPTS = intel-gfx-trybot who.sh run-tests.sh
+dist_noinst_SCRIPTS = intel-gfx-trybot who.sh run-tests.sh trace.pl
 noinst_PYTHON = throttle.py
scripts/trace.pl new file +946 -0

946 lines not on this page · raw message

Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx
#25 lore raw

PATCH v4 1/1 igt/scripts: trace.pl to parse the i915 tracepoints

In reply to Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Given a log file created via perf with some interesting trace events enabled, this tool can generate the timeline graph of requests getting queued, their dependencies resolved, sent to the GPU for executing and finally completed.

This can be useful when analyzing certain classes of performance issues. More help is available in the tool itself.

The tool will also calculate some overall per engine statistics, like total time engine was idle and similar.

v2:
* Address missing git add.
* Make html output optional (--html switch) and by default just output aggregated per engine stats to stdout.

v3:
* Added --trace option which invokes perf with the correct options automatically.
* Added --avg-delay-stats which prints averages for things like waiting on ready, waiting on GPU and context save duration.
* Fix warnings when no waits on an engine.
* Correct help text.

v4:
* Add --squash-ctx-id to substract engine id from ctx id when parsing to make it easier to identify which context is which with new i915 ctx id allocation scheme.
* Reconstruct request_out events where they are missing.

Cc Chris Wilson <[email protected]>
Cc Harri Syrja <[email protected]>
Cc Krzysztof E Olinski <[email protected]>
---
 scripts/Makefile.am |   2 +-
 scripts/trace.pl    | 990 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 2 files changed, 991 insertions(+), 1 deletion(-)
 create mode 100755 scripts/trace.pl
scripts/Makefile.am +1 -1
index 85d4a5cf4e5c..641715294936 100644
@@ -1,2 +1,2 @@
-dist_noinst_SCRIPTS = intel-gfx-trybot who.sh run-tests.sh
+dist_noinst_SCRIPTS = intel-gfx-trybot who.sh run-tests.sh trace.pl
 noinst_PYTHON = throttle.py
scripts/trace.pl new file +990 -0

990 lines not on this page · raw message

Signature · 6 lines
2.9.3

_______________________________________________
Intel-gfx mailing list
[email protected]
https://lists.freedesktop.org/mailman/listinfo/intel-gfx