Skip to contentSkip to conversation

Memory bandwidth tester

PATCH In review Antti Koskipaa 3 participants 6 messages Oct 5, 2015 – Oct 6, 2015 Library code

lore
Share & tools
lore
#0 lore raw

cover Memory bandwidth tester

This is a memory bandwidth tester based on the existing rendercopy tests in IGT. Its purpose is to find regressions in the kernel driver and to figure out where a bottleneck is. This test bypasses Mesa and goes directly to the kernel. Say there is a performance regression. Where is it, Mesa or kernel? Just compare the results of this test to the Mesa bandwidth results. If they are similar, the problem is in the kernel. If Mesa is much slower than this test, the problem is in Mesa.

Equivalent GLSL for the 'write' shader:

void main() {
	gl_FragColor = vec4(1.0, 0.99, 0.5, 0.1);
}

Equivalend GLSL for the 'read' shader:

void main() {
	gl_FragColor = texture2D(sourcebuf, gl_TexCoord);
	discard;
}

I'm not too sure about my understanding of the shaders, especially the one that does the discard operation (ps_kernel_read). Supposedly you just need to set the output masks to 0 in the header of the final 'send' and set some bits in 3DSTATE_PS_EXTRA. Halt instructions not needed since I'm not doing any complicated flow control.
Someone more knowledgeable may want to review those shaders very closely.

Coming up soon: alpha blending versions of the copy and write tests.

Antti Koskipaa (3):
gem_render_copy: Make render_copyfunc more generic
gem_render_copy: Add functions for performance testing
gem_memory_bandwidth: Add test

 lib/gen8_render.h            |   3 +
 lib/intel_batchbuffer.c      |  40 +++++++++
 lib/intel_batchbuffer.h      |   2 +
 lib/rendercopy.h             |  12 +++
 lib/rendercopy_gen8.c        |  68 ++++++++++++--
 shaders/ps/discard.g7a       |  73 +++++++++++++++
 shaders/ps/fill.g7a          |   6 ++
 tests/.gitignore             |   1 +
 tests/Makefile.sources       |   1 +
 tests/gem_memory_bandwidth.c | 209 +++++++++++++++++++++++++++++++++++++++++++
 10 files changed, 408 insertions(+), 7 deletions(-)
 create mode 100644 shaders/ps/discard.g7a
 create mode 100644 shaders/ps/fill.g7a
 create mode 100644 tests/gem_memory_bandwidth.c
Signature · 6 lines
2.3.6

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw

PATCH 1/3 gem_render_copy: Make render_copyfunc more generic

This will allow adding more tests that use this function without excessive copying and pasting.

---
 lib/rendercopy_gen8.c | 19 +++++++++++++++----
 1 file changed, 15 insertions(+), 4 deletions(-)
lib/rendercopy_gen8.c +15 -4
index a7fc2c4..4a9a283 100644
@@ -49,7 +49,7 @@ struct {
 } viewport;
 
 /* see shaders/ps/blit.g7a */
-static const uint32_t ps_kernel[][4] = {
+static const uint32_t ps_kernel_copy[][4] = {
 #if 1
    { 0x0060005a, 0x21403ae8, 0x3a0000c0, 0x008d0040 },
    { 0x0060005a, 0x21603ae8, 0x3a0000c0, 0x008d0080 },
@@ -920,11 +920,12 @@ static void gen8_emit_primitive(struct intel_batchbuffer *batch, uint32_t offset
 
 #define BATCH_STATE_SPLIT 2048
 
-void gen8_render_copyfunc(struct intel_batchbuffer *batch,
+static void _gen8_render_func(struct intel_batchbuffer *batch,
 			  drm_intel_context *context,
 			  struct igt_buf *src, unsigned src_x, unsigned src_y,
 			  unsigned width, unsigned height,
-			  struct igt_buf *dst, unsigned dst_x, unsigned dst_y)
+			  struct igt_buf *dst, unsigned dst_x, unsigned dst_y,
+			  const uint32_t ps_kernel[][4], int kernel_size)
 {
 	struct annotations_context aub_annotations;
 	uint32_t ps_sampler_state, ps_kernel_off, ps_binding_table;
@@ -944,7 +945,7 @@ void gen8_render_copyfunc(struct intel_batchbuffer *batch,
 					       src, dst);
 	ps_sampler_state  = gen8_create_sampler(batch, &aub_annotations);
 	ps_kernel_off = gen8_fill_ps(batch, &aub_annotations,
-				     ps_kernel, sizeof(ps_kernel));
+				     ps_kernel, kernel_size);
 	vertex_buffer = gen7_fill_vertex_buffer_data(batch, &aub_annotations,
 						     src,
 						     src_x, src_y,
@@ -1030,3 +1031,13 @@ void gen8_render_copyfunc(struct intel_batchbuffer *batch,
 	gen6_render_flush(batch, context, batch_end);
 	intel_batchbuffer_reset(batch);
 }
+
+void gen8_render_copyfunc(struct intel_batchbuffer *batch,
+				 drm_intel_context *context,
+				 struct igt_buf *src, unsigned src_x, unsigned src_y,
+				 unsigned width, unsigned height,
+				 struct igt_buf *dst, unsigned dst_x, unsigned dst_y)
+{
+	_gen8_render_func(batch, context, src, src_x, src_y, width, height,
+			  dst, dst_x, dst_y, ps_kernel_copy, sizeof(ps_kernel_copy));
+}
Signature · 6 lines
2.3.6

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

PATCH 2/3 gem_render_copy: Add functions for performance testing

In reply to Antti Koskipaa

gen8_render_writefunc does only constant writes to the framebuffer, no texture reads. Used for testing write bandwidth.

gen8_render_readfunc does only reads from the sampler and discards the result. Used for testing sampler read bandwidth.

Alpha blend tests and support for more Gens still to come.

---
 lib/gen8_render.h       |  3 ++
 lib/intel_batchbuffer.c | 40 +++++++++++++++++++++++++++
 lib/intel_batchbuffer.h |  2 ++
 lib/rendercopy.h        | 12 ++++++++
 lib/rendercopy_gen8.c   | 53 +++++++++++++++++++++++++++++++----
 shaders/ps/discard.g7a  | 73 +++++++++++++++++++++++++++++++++++++++++++++++++
 shaders/ps/fill.g7a     |  6 ++++
 7 files changed, 184 insertions(+), 5 deletions(-)
 create mode 100644 shaders/ps/discard.g7a
 create mode 100644 shaders/ps/fill.g7a
lib/gen8_render.h +3 -0
index ba3f9f2..610a457 100644
@@ -60,6 +60,9 @@
 #define GEN8_3DSTATE_WM_DEPTH_STENCIL		GEN6_3D(3, 0, 0x4e)
 #define GEN8_3DSTATE_PS_EXTRA			GEN6_3D(3,0, 0x4f)
 # define GEN8_PSX_PIXEL_SHADER_VALID			(1 << 31)
+# define GEN8_PSX_DONT_WRITE_RT				(1 << 30)
+# define GEN8_PSX_OMASK_PRESENT				(1 << 29)
+# define GEN8_PSX_KILLS_PIXEL				(1 << 28)
 # define GEN8_PSX_ATTRIBUTE_ENABLE			(1 << 8)
 
 #define GEN7_3DSTATE_VIEWPORT_STATE_POINTERS_CC	GEN6_3D(3, 0, 0x23)
lib/intel_batchbuffer.c +40 -0
index 692521f..e3cf622 100644
@@ -748,6 +748,46 @@ igt_render_copyfunc_t igt_get_render_copyfunc(int devid)
 }
 
 /**
+ * igt_get_render_writefunc:
+ * @devid: pci device id
+ *
+ * Returns:
+ *
+ * The platform-specific render write function pointer for the device
+ * specified with @devid. Will return NULL when no render copy function is
+ * implemented.
+ */
+igt_render_copyfunc_t igt_get_render_writefunc(int devid)
+{
+	igt_render_copyfunc_t copy = NULL;
+
+	if (IS_GEN8(devid))
+		copy = gen8_render_writefunc;
+
+	return copy;
+}
+
+/**
+ * igt_get_render_readfunc:
+ * @devid: pci device id
+ *
+ * Returns:
+ *
+ * The platform-specific render read function pointer for the device
+ * specified with @devid. Will return NULL when no render copy function is
+ * implemented.
+ */
+igt_render_copyfunc_t igt_get_render_readfunc(int devid)
+{
+	igt_render_copyfunc_t copy = NULL;
+
+	if (IS_GEN8(devid))
+		copy = gen8_render_readfunc;
+
+	return copy;
+}
+
+/**
  * igt_get_media_fillfunc:
  * @devid: pci device id
  *
lib/intel_batchbuffer.h +2 -0
index 869747d..7d8f990 100644
@@ -274,6 +274,8 @@ typedef void (*igt_render_copyfunc_t)(struct intel_batchbuffer *batch,
 				      struct igt_buf *dst, unsigned dst_x, unsigned dst_y);
 
 igt_render_copyfunc_t igt_get_render_copyfunc(int devid);
+igt_render_copyfunc_t igt_get_render_writefunc(int devid);
+igt_render_copyfunc_t igt_get_render_readfunc(int devid);
 
 /**
  * igt_fillfunc_t:
lib/rendercopy.h +12 -0
index fdc3cab..f4ec74b 100644
@@ -53,3 +53,15 @@ void gen2_render_copyfunc(struct intel_batchbuffer *batch,
 			  struct igt_buf *src, unsigned src_x, unsigned src_y,
 			  unsigned width, unsigned height,
 			  struct igt_buf *dst, unsigned dst_x, unsigned dst_y);
+
+void gen8_render_writefunc(struct intel_batchbuffer *batch,
+			   drm_intel_context *context,
+			   struct igt_buf *src, unsigned src_x, unsigned src_y,
+			   unsigned width, unsigned height,
+			   struct igt_buf *dst, unsigned dst_x, unsigned dst_y);
+
+void gen8_render_readfunc(struct intel_batchbuffer *batch,
+			   drm_intel_context *context,
+			   struct igt_buf *src, unsigned src_x, unsigned src_y,
+			   unsigned width, unsigned height,
+			   struct igt_buf *dst, unsigned dst_x, unsigned dst_y);
lib/rendercopy_gen8.c +48 -5

87 lines not on this page · raw message

shaders/ps/discard.g7a new file +73 -0

73 lines not on this page · raw message

shaders/ps/fill.g7a new file +6 -0
new file mode 100644
index 0000000..89f130f
@@ -0,0 +1,6 @@
+mov (16) g112<1>F 1.0F { align1 };
+mov (16) g114<1>F 0.99F { align1 };
+mov (16) g116<1>F 0.5F { align1 };
+mov (16) g118<1>F 0.1F { align1 };
+send (16) null g112 0x25 0x10031000 { align1, EOT };
+/* <8,8,1>F render RT write SIMD16 LastRT Surface = 0 mlen 8 rlen 0 { align1 1H EOT }; */
Signature · 6 lines
2.3.6

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#3 lore raw

PATCH 3/3 gem_memory_bandwidth: Add test

In reply to Antti Koskipaa

This is a benchmark for testing the GPU read and write bandwidth.

Issue: VIZ-5664

---
 tests/.gitignore             |   1 +
 tests/Makefile.sources       |   1 +
 tests/gem_memory_bandwidth.c | 209 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 211 insertions(+)
 create mode 100644 tests/gem_memory_bandwidth.c
tests/.gitignore +1 -0
index dc8bb53..2ea4107 100644
@@ -65,6 +65,7 @@ gem_linear_blits
 gem_lut_handle
 gem_madvise
 gem_media_fill
+gem_memory_bandwidth
 gem_mmap
 gem_mmap_gtt
 gem_mmap_offset_exhaustion
tests/Makefile.sources +1 -0
index 2e2e088..4429c29 100644
@@ -36,6 +36,7 @@ TESTS_progs_M = \
 	gem_flink_race \
 	gem_linear_blits \
 	gem_madvise \
+	gem_memory_bandwidth \
 	gem_mmap \
 	gem_mmap_gtt \
 	gem_mmap_wc \
tests/gem_memory_bandwidth.c new file +209 -0

209 lines not on this page · raw message

Signature · 1 line
2.3.6

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#4 lore raw

Re: [PATCH i-g-t 3/3] gem_memory_bandwidth: Add test

Quoted 208 lines · Antti Koskipaa

This is a benchmark for testing the GPU read and write bandwidth.

Issue: VIZ-5664

---
 tests/.gitignore             |   1 +
 tests/Makefile.sources       |   1 +
 tests/gem_memory_bandwidth.c | 209 +++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 211 insertions(+)
 create mode 100644 tests/gem_memory_bandwidth.c
tests/.gitignore +1 -0
index dc8bb53..2ea4107 100644
@@ -65,6 +65,7 @@ gem_linear_blits
 gem_lut_handle
 gem_madvise
 gem_media_fill
+gem_memory_bandwidth
 gem_mmap
 gem_mmap_gtt
 gem_mmap_offset_exhaustion
tests/Makefile.sources +1 -0
index 2e2e088..4429c29 100644
@@ -36,6 +36,7 @@ TESTS_progs_M = \
 	gem_flink_race \
 	gem_linear_blits \
 	gem_madvise \
+	gem_memory_bandwidth \
 	gem_mmap \
 	gem_mmap_gtt \
 	gem_mmap_wc \
tests/gem_memory_bandwidth.c new file +167 -0
new file mode 100644
index 0000000..a44987e
@@ -0,0 +1,209 @@
+/*
+ * Copyright © 2013-2014 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ *    Antti Koskip?? <[email protected]>
+ *    Damien Lespiau <[email protected]>
+ */
+
+/*
+ * We need a way to test memory bandwidth bottlenecks and understand better
+ * where they are. This test bypasses Mesa and uses the kernel GEM interface
+ * directly.
+ *
+ * Say there is a performance regression. Where is it, Mesa or kernel? Just
+ * compare the results of this test to the Mesa bandwidth results. If they are
+ * similar, the problem is in the kernel. If Mesa is much slower than this test,
+ * the problem is in Mesa.
+ */
+
+#include "igt.h"
+#include <stdbool.h>
+#include <unistd.h>
+#include <stdlib.h>
+#include <sys/ioctl.h>
+#include <stdio.h>
+#include <string.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <errno.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+
+#include <drm.h>
+
+#include "intel_bufmgr.h"
+
+IGT_TEST_DESCRIPTION("GPU memory bandwidth benchmark.");
+
+/* Each test block is 1 meg. */
+#define WIDTH 512
+#define STRIDE (WIDTH*4)
+#define HEIGHT 512
+#define SIZE (HEIGHT*STRIDE)
+/* Run this many times. 10240 = 10 gigabytes are copied. 1024 was too small,
+ * giving too much variance. */
+#define LOOPS 10240
+/* How many buffers to allocate for main memory speed testing.
+ * Must be large enough to thrash the caches.
+ */
+#define NBUFS 512
+
+#define SRC_COLOR	0xffff00ff
+#define DST_COLOR	0xfff0ff00
+
+typedef struct {
+	int fd;
+	uint32_t devid;
+	drm_intel_bufmgr *bufmgr;
+	struct intel_batchbuffer *batch;
+	igt_render_copyfunc_t render_copy;
+	igt_render_copyfunc_t render_read;
+	igt_render_copyfunc_t render_write;
+	uint32_t linear[WIDTH * HEIGHT];
+} data_t;
+
+static void data_init(data_t *data)
+{
+	data->fd = drm_open_driver(DRIVER_INTEL);
+	data->devid = intel_get_drm_devid(data->fd);
+
+	data->bufmgr = drm_intel_bufmgr_gem_init(data->fd, 4096);
+	igt_assert(data->bufmgr);
+
+	data->render_copy = igt_get_render_copyfunc(data->devid);
+	igt_require_f(data->render_copy,
+		      "no render-copy function\n");
+	data->render_write = igt_get_render_writefunc(data->devid);
+	igt_require_f(data->render_write,
+		      "no render-write function\n");
+	data->render_read = igt_get_render_readfunc(data->devid);
+	igt_require_f(data->render_read,
+		      "no render-read function\n");
+
+	data->batch = intel_batchbuffer_alloc(data->bufmgr, data->devid);
+	igt_assert(data->batch);
+
+	for (int i = 0; i < WIDTH * HEIGHT; i++)
+		data->linear[i] = i;
+
+}
+
+static void data_fini(data_t *data)
+{
+	intel_batchbuffer_free(data->batch);
+	drm_intel_bufmgr_destroy(data->bufmgr);
+	close(data->fd);
+}
+
+static int scratch_buf_init(data_t *data, struct igt_buf *buf,
+			    int width, int height, int stride, uint32_t color)
+{
+	drm_intel_bo *bo;
+
+	bo = drm_intel_bo_alloc(data->bufmgr, "", SIZE, 4096);
+	if (!bo)
+		return -1;
+	gem_write(data->fd, bo->handle, 0, data->linear,
+		  sizeof(data->linear));
+
+	buf->bo = bo;
+	buf->stride = stride;
+	buf->tiling = I915_TILING_NONE;
+	buf->size = SIZE;
+	return 0;
+}
+
+static void scratch_buf_fini(data_t *data, struct igt_buf *buf)
+{
+	dri_bo_unreference(buf->bo);
+	memset(buf, 0, sizeof(*buf));
+}
+
+static void print_bandwidth(const char *desc, struct timeval *start, struct timeval *end)
+{
+	struct timeval diff;
+	uint64_t usecs;
+	timersub(end, start, &diff);
+	usecs = diff.tv_sec * 1000000ULL + diff.tv_usec;
+	igt_assert(usecs != 0);
+	/* 1 byte/us = 1M/s */
+	printf("%s: %i MB in %f seconds, %f MB/s\n", desc,
+	       LOOPS, (float)usecs / 1.0e6,
+	       (float)(STRIDE*HEIGHT*(uint64_t)LOOPS) / (float)usecs);
+}
+
+static void test(data_t *data, struct igt_buf *bufs, igt_render_copyfunc_t func,
+		 const char *desc)
+{
+	int i;
+	struct timeval start, end;
+
+	gettimeofday(&start, NULL);
+
+	for (i = 0; i < LOOPS; i++)
+		func(data->batch, NULL, &bufs[i % NBUFS], 0, 0, WIDTH, HEIGHT,
+		     &bufs[(i + 1) % NBUFS], WIDTH / 2, HEIGHT / 2);
+
+	gettimeofday(&end, NULL);

tests/gem_memory_bandwidth.c line 177

Quoted 10 lines
tests/gem_memory_bandwidth.c new file +10 -0
+	print_bandwidth(desc, &start, &end);
+}
+
+int main(int argc, char **argv)
+{
+	data_t data = {0, };
+	struct igt_buf bufs[NBUFS];
+
+	igt_subtest_init(argc, argv);
+

igt_main {

Quoted 19 lines
tests/gem_memory_bandwidth.c new file +19 -0
+	igt_fixture {
+		data_init(&data);
+		for (int i = 0; i < NBUFS; i++)
+			if (scratch_buf_init(&data, &bufs[i], WIDTH, HEIGHT, STRIDE, SRC_COLOR)) {
+				printf("Not enough memory to allocate all scratch buffers. Need" \
+				       "%i megabytes more.\n", NBUFS - i);
+				for (i--; i >= 0; i--)
+					scratch_buf_fini(&data, &bufs[i]);
+				igt_fail(IGT_EXIT_FAILURE);
+			}
+	}
+
+	igt_subtest("copy") {
+		test(&data, bufs, data.render_copy, "Copy");
+	}
+
+	igt_subtest("write") {
+		test(&data, bufs, data.render_write, "Write");
+	}

tests/gem_memory_bandwidth.c line 206

Quoted 10 lines
tests/gem_memory_bandwidth.c new file +10 -0
+
+	igt_subtest("read") {
+		test(&data, bufs, data.render_read, "Read");
+	}
+
+	igt_fixture {
+		for (int i = 0; i < NBUFS; i++)
+			scratch_buf_fini(&data, &bufs[i]);
+		data_fini(&data);
+	}

}

And you can drop igt_exit and igt_subtest_init.
-Daniel

Quoted 11 lines
tests/gem_memory_bandwidth.c new file +3 -0
+
+	igt_exit();
+}
Signature · 7 lines
2.3.6


_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
Signature · 7 lines
Daniel Vetter
Software Engineer, Intel Corporation
http://blog.ffwll.ch
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#5 lore raw

Re: [PATCH i-g-t 3/3] gem_memory_bandwidth: Add test

In reply to Antti Koskipaa
Quoted 1 line · Antti Koskipaa

This is a benchmark for testing the GPU read and write bandwidth.

I don't see any pass/fail logic so why put in amongst the regression tests?

Please don't advertise this as gem_memory_bandwidth either, it is just a rendercopy speed test. Note that LOOPS is critical here, setting it to 1 and we measure dispatch overhead (also v. important), setting it to 10,000 we saturate the RCS ring, force RPS boosting etc. Similarly, measuring performance at min/max clocks is also interesting. There are a lot of implicit variables here at play that are not taken account of.

I would like to have a set of perf tests as regression tests. What I have in mind is basically doing a fixed workload inside igt and have the core automatically compare the runtime of a subtest against a local database of results and s/PASS/SLOW/ (or outright FAIL) if it exceeds some threshold.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx