Skip to contentSkip to conversation

igt/gem_request_retire: Provoke context destruction with active VMAs

PATCH Superseded Tvrtko Ursulin 2 participants 5 messages Nov 19, 2015 – Nov 20, 2015

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 igt/gem_request_retire: Provoke context destruction with active VMAs

From: Tvrtko Ursulin <[email protected]>

Test designed to trigger the
WARN_ON(!list_empty(&ppgtt->base.active_list))
in i915_gem_context_clean.

Cc Chris Wilson <[email protected]>
Cc Daniel Vetter <[email protected]>

---
Some potentially hairy batch buffer building code since I don't have a lot of experience in that area. :)

---
 tests/.gitignore           |   1 +
 tests/Makefile.sources     |   1 +
 tests/gem_request_retire.c | 279 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 281 insertions(+)
 create mode 100644 tests/gem_request_retire.c
tests/.gitignore +1 -0
index 80af9a718f06..85936ea45c9f 100644
@@ -91,6 +91,7 @@ gem_render_copy
 gem_render_copy_redux
 gem_render_linear_blits
 gem_render_tiled_blits
+gem_request_retire
 gem_reset_stats
 gem_ring_sync_copy
 gem_ring_sync_loop
tests/Makefile.sources +1 -0
index 8fb2de8b4665..ff178f7a2df4 100644
@@ -53,6 +53,7 @@ TESTS_progs_M = \
 	gem_reloc_overflow \
 	gem_reloc_vs_gpu \
 	gem_render_copy_redux \
+	gem_request_retire \
 	gem_reset_stats \
 	gem_ringfill \
 	gem_set_tiling_vs_blt \
tests/gem_request_retire.c new file +279 -0

279 lines not on this page · raw message

Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#1 lore raw
Quoted 9 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Test designed to trigger the
WARN_ON(!list_empty(&ppgtt->base.active_list))
in i915_gem_context_clean.

Cc Chris Wilson <[email protected]>
Cc Daniel Vetter <[email protected]>

Wouldn't this be a better fit in gem_ctx_somethingorother

gem_ctx/close-active

Quoted 153 lines

---
Some potentially hairy batch buffer building code since I don't have a lot of experience in that area. :)

---
 tests/.gitignore           |   1 +
 tests/Makefile.sources     |   1 +
 tests/gem_request_retire.c | 279 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 281 insertions(+)
 create mode 100644 tests/gem_request_retire.c
tests/.gitignore +1 -0
index 80af9a718f06..85936ea45c9f 100644
@@ -91,6 +91,7 @@ gem_render_copy
 gem_render_copy_redux
 gem_render_linear_blits
 gem_render_tiled_blits
+gem_request_retire
 gem_reset_stats
 gem_ring_sync_copy
 gem_ring_sync_loop
tests/Makefile.sources +1 -0
index 8fb2de8b4665..ff178f7a2df4 100644
@@ -53,6 +53,7 @@ TESTS_progs_M = \
 	gem_reloc_overflow \
 	gem_reloc_vs_gpu \
 	gem_render_copy_redux \
+	gem_request_retire \
 	gem_reset_stats \
 	gem_ringfill \
 	gem_set_tiling_vs_blt \
tests/gem_request_retire.c new file +113 -0
new file mode 100644
index 000000000000..710a95f7b6ed
@@ -0,0 +1,279 @@
+/*
+ * Copyright © 2015 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ *    Tvrtko Ursulin <[email protected]>
+ *
+ */
+
+/** @file gem_request_retire
+ *
+ * Collection of tests targeting request retirement code paths.
+ */
+
+#include "igt.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <errno.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/mman.h>
+#include <signal.h>
+#include <pthread.h>
+#include <time.h>
+
+#include "drm.h"
+#include "i915_drm.h"
+
+#include "intel_bufmgr.h"
+
+#define WIDTH 4096
+#define HEIGHT 4096
+#define BO_SIZE (WIDTH * HEIGHT * sizeof(uint32_t))
+
+static uint32_t
+blit(int fd, uint32_t dst, uint32_t src, uint32_t ctx_id)
+{
+	const unsigned int copies = 1000;
+	uint32_t batch[12 * copies + 2];
+	struct drm_i915_gem_relocation_entry reloc[2 * copies];
+	struct drm_i915_gem_exec_object2 obj[3];
+	struct drm_i915_gem_execbuffer2 exec;
+	uint32_t handle;
+	int ret;
+	unsigned int i = 0, j;
+	unsigned int blit_len = 0;
+
+	for (j = 0; j < copies; j++) {
+		batch[i++] = XY_SRC_COPY_BLT_CMD |
+			XY_SRC_COPY_BLT_WRITE_ALPHA |
+			XY_SRC_COPY_BLT_WRITE_RGB;
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			batch[i - 1] |= 8;
+		else
+			batch[i - 1] |= 6;
+
+		batch[i++] = (3 << 24) | /* 32 bits */
+			(0xcc << 16) | /* copy ROP */
+			WIDTH*4;
+		batch[i++] = 0; /* dst x1,y1 */
+		batch[i++] = (HEIGHT << 16) | WIDTH; /* dst x2,y2 */
+		batch[i++] = 0; /* dst reloc */
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			batch[i++] = 0;
+		batch[i++] = 0; /* src x1,y1 */
+		batch[i++] = WIDTH*4;
+		batch[i++] = 0; /* src reloc */
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			batch[i++] = 0;
+
+		while (i % 4)
+			batch[i++] = MI_NOOP;
+
+		if (blit_len == 0)
+			blit_len = i;
+	}
+
+	batch[i++] = MI_BATCH_BUFFER_END;
+	batch[i++] = MI_NOOP;
+
+	handle = gem_create(fd, sizeof(batch));
+	gem_write(fd, handle, 0, batch, sizeof(batch));
+
+	for (j = 0; j < copies; j++) {
+		unsigned int r0 = j * 2;
+		unsigned int r1 = r0 + 1;
+
+		reloc[r0].target_handle = dst;
+		reloc[r0].delta = 0;
+		reloc[r0].offset = j * blit_len + 4 * sizeof(uint32_t);
+		reloc[r0].presumed_offset = 0;
+		reloc[r0].read_domains = I915_GEM_DOMAIN_RENDER;

tests/gem_request_retire.c line 123

Quoted 10 lines
tests/gem_request_retire.c new file +10 -0
+		reloc[r0].write_domain = I915_GEM_DOMAIN_RENDER;
+
+		reloc[r1].target_handle = src;
+		reloc[r1].delta = 0;
+		reloc[r1].offset = j * blit_len + 7 * sizeof(uint32_t);
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			reloc[r1].offset += sizeof(uint32_t);
+		reloc[r1].presumed_offset = 0;
+		reloc[r1].read_domains = I915_GEM_DOMAIN_RENDER;
+		reloc[r1].write_domain = 0;

Just fill the relocation array as you generate the batch.

tests/gem_request_retire.c line 136

Quoted 13 lines
tests/gem_request_retire.c new file +13 -0
+	}
+
+	memset(obj, 0, sizeof(obj));
+	exec.buffer_count = 0;
+	obj[exec.buffer_count++].handle = dst;
+	if (src != dst)
+		obj[exec.buffer_count++].handle = src;
+	obj[exec.buffer_count].handle = handle;
+	obj[exec.buffer_count].relocation_count = 2 * copies;
+	obj[exec.buffer_count].relocs_ptr = (uintptr_t)reloc;
+	exec.buffer_count++;
+	exec.buffers_ptr = (uintptr_t)obj;
+

memset(&exec, 0, sizeof(exec));
instead of:

tests/gem_request_retire.c line 147

Quoted 11 lines
tests/gem_request_retire.c new file +11 -0
+	exec.batch_start_offset = 0;
+	exec.batch_len = i * sizeof(uint32_t);
+	exec.DR1 = exec.DR4 = 0;
+	exec.num_cliprects = 0;
+	exec.cliprects_ptr = 0;
+	exec.flags = I915_EXEC_BLT;
+	i915_execbuffer2_set_context_id(exec, ctx_id);
+	exec.rsvd2 = 0;
+
+	ret = drmIoctl(fd, DRM_IOCTL_I915_GEM_EXECBUFFER2, &exec);
+	igt_assert_eq(ret, 0);

gem_execbuf(fd, &exec);

Quoted 50 lines
tests/gem_request_retire.c new file +50 -0
+
+	return handle;
+}
+
+static uint32_t
+store(int fd, uint32_t dst, uint32_t src, uint32_t ctx_id)
+{
+	uint32_t batch[6];
+	struct drm_i915_gem_relocation_entry reloc[2];
+	struct drm_i915_gem_exec_object2 obj[3];
+	struct drm_i915_gem_execbuffer2 exec;
+	uint32_t handle;
+	int ret, i = 0;
+
+	batch[i++] = MI_STORE_DWORD_IMM;
+	batch[i++] = 0; /* addrhigh */
+	batch[i++] = 0; /* addrlow */
+	batch[i++] = 0; /* val */
+	batch[i++] = MI_BATCH_BUFFER_END;
+	batch[i++] = MI_BATCH_BUFFER_END;
+
+	handle = gem_create(fd, 4096);
+	gem_write(fd, handle, 0, batch, sizeof(batch));
+
+	reloc[0].target_handle = dst;
+	reloc[0].delta = 0;
+	reloc[0].offset = 2 * sizeof(batch[0]);
+	reloc[0].presumed_offset = 0;
+	reloc[0].read_domains = I915_GEM_DOMAIN_RENDER;
+	reloc[0].write_domain = I915_GEM_DOMAIN_RENDER;
+
+	reloc[1].target_handle = src;
+	reloc[1].delta = 0;
+	reloc[1].offset = 3 * sizeof(batch[0]);
+	reloc[1].presumed_offset = 0;
+	reloc[1].read_domains = I915_GEM_DOMAIN_RENDER;
+	reloc[1].write_domain = 0;
+
+	memset(obj, 0, sizeof(obj));
+	exec.buffer_count = 0;
+	obj[exec.buffer_count++].handle = dst;
+	if (src != dst)
+		obj[exec.buffer_count++].handle = src;
+	obj[exec.buffer_count].handle = handle;
+	obj[exec.buffer_count].relocation_count = 2;
+	obj[exec.buffer_count].relocs_ptr = (uintptr_t)reloc;
+	exec.buffer_count++;
+	exec.buffers_ptr = (uintptr_t)obj;
+
+	exec.batch_start_offset = 0;

tests/gem_request_retire.c line 207

Quoted 10 lines
tests/gem_request_retire.c new file +10 -0
+	exec.batch_len = i * sizeof(uint32_t);
+	exec.DR1 = exec.DR4 = 0;
+	exec.num_cliprects = 0;
+	exec.cliprects_ptr = 0;
+	exec.flags = I915_EXEC_RENDER;
+	i915_execbuffer2_set_context_id(exec, ctx_id);
+	exec.rsvd2 = 0;
+
+	ret = drmIoctl(fd, DRM_IOCTL_I915_GEM_EXECBUFFER2, &exec);
+	igt_assert_eq(ret, 0);

For the purposes of your test, you don't even need to execute anything, just associate the dst handle with an empty batch on the render ring. You don't even need a fake relocation since the test already requires a fairly recent kernel.

Quoted 30 lines
tests/gem_request_retire.c new file +30 -0
+
+	return handle;
+}
+
+/*
+ * A single bo is operated from batchbuffers submitted from two contexts and on
+ * different rings.
+ * One execbuf finishes way ahead of the other at which point the respective
+ * context is destroyed.
+ */
+static void
+test_retire_vma_not_inactive(int fd)
+{
+	uint32_t ctx_id;
+	uint32_t src, dst, dst2;
+	uint32_t blit_bb, store_bb;
+
+	igt_require(HAS_BLT_RING(intel_get_drm_devid(fd)));
+
+	ctx_id = gem_context_create(fd);
+	igt_assert(ctx_id);
+
+	/* Create some bos batch buffers will operate on. */
+	src = gem_create(fd, BO_SIZE);
+	igt_assert(src);
+	dst = gem_create(fd, BO_SIZE);
+	igt_assert(dst);
+	dst2 = gem_create(fd, 4096);
+	igt_assert(dst2);
+

tests/gem_request_retire.c line 246

Quoted 10 lines
tests/gem_request_retire.c new file +9 -0
+	/* Submit a long running batch. */
+	blit_bb = blit(fd, dst, src, 0);
+	/* Submit a quick batch referencing the same object. */
+	store_bb = store(fd, dst2, src, ctx_id);
+	/* Wait for the quick batch to complete. */
+	gem_sync(fd, store_bb);
+	gem_sync(fd, dst2);
+	gem_close(fd, store_bb);
 
+	/* Do it again to ensure in kernel retirement is triggered. */

Circumstantial!

tests/gem_request_retire.c line 254

Quoted 8 lines
tests/gem_request_retire.c new file +8 -0
+	store_bb = store(fd, dst2, src, ctx_id);
+	gem_sync(fd, store_bb);
+	gem_sync(fd, dst2);
+	gem_close(fd, store_bb);
+	gem_close(fd, dst2);
+
+	/* Now destroy the context in which the quick batch was submitted. */
+	gem_context_destroy(fd, ctx_id);

Feels very fragile. Isn't the bug something like

batch = big_render_copy(ctx);
context_close(ctx);
gem_sync(batch);
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#2 lore raw

Hi,

Quoted 7 lines · Chris Wilson
Quoted 6 lines · Tvrtko Ursulin

From: Tvrtko Ursulin <[email protected]>

Test designed to trigger the
WARN_ON(!list_empty(&ppgtt->base.active_list))
in i915_gem_context_clean.

Quoted 7 lines
Quoted 3 lines
Cc Chris Wilson <[email protected]>
Cc Daniel Vetter <[email protected]>

Wouldn't this be a better fit in gem_ctx_somethingorother

gem_ctx/close-active

Can move it, don't really mind either way.

Quoted 125 lines
Quoted 125 lines

---
Some potentially hairy batch buffer building code since I don't have a lot of experience in that area. :)

---
  tests/.gitignore           |   1 +
  tests/Makefile.sources     |   1 +
  tests/gem_request_retire.c | 279 +++++++++++++++++++++++++++++++++++++++++++++
  3 files changed, 281 insertions(+)

create mode 100644 tests/gem_request_retire.c

tests/.gitignore +1 -0
index 80af9a718f06..85936ea45c9f 100644
@@ -91,6 +91,7 @@ gem_render_copy
  gem_render_copy_redux
  gem_render_linear_blits
  gem_render_tiled_blits
+gem_request_retire
  gem_reset_stats
  gem_ring_sync_copy
  gem_ring_sync_loop
tests/Makefile.sources +1 -0
index 8fb2de8b4665..ff178f7a2df4 100644
@@ -53,6 +53,7 @@ TESTS_progs_M = \
  	gem_reloc_overflow \
  	gem_reloc_vs_gpu \
  	gem_render_copy_redux \
+	gem_request_retire \
  	gem_reset_stats \
  	gem_ringfill \
  	gem_set_tiling_vs_blt \
tests/gem_request_retire.c new file +85 -0
new file mode 100644
index 000000000000..710a95f7b6ed
@@ -0,0 +1,279 @@
+/*
+ * Copyright © 2015 Intel Corporation
+ *
+ * Permission is hereby granted, free of charge, to any person obtaining a
+ * copy of this software and associated documentation files (the "Software"),
+ * to deal in the Software without restriction, including without limitation
+ * the rights to use, copy, modify, merge, publish, distribute, sublicense,
+ * and/or sell copies of the Software, and to permit persons to whom the
+ * Software is furnished to do so, subject to the following conditions:
+ *
+ * The above copyright notice and this permission notice (including the next
+ * paragraph) shall be included in all copies or substantial portions of the
+ * Software.
+ *
+ * THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
+ * IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
+ * FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT.  IN NO EVENT SHALL
+ * THE AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
+ * LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING
+ * FROM, OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS
+ * IN THE SOFTWARE.
+ *
+ * Authors:
+ *    Tvrtko Ursulin <[email protected]>
+ *
+ */
+
+/** @file gem_request_retire
+ *
+ * Collection of tests targeting request retirement code paths.
+ */
+
+#include "igt.h"
+#include <stdlib.h>
+#include <stdio.h>
+#include <string.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <errno.h>
+#include <sys/stat.h>
+#include <sys/time.h>
+#include <sys/mman.h>
+#include <signal.h>
+#include <pthread.h>
+#include <time.h>
+
+#include "drm.h"
+#include "i915_drm.h"
+
+#include "intel_bufmgr.h"
+
+#define WIDTH 4096
+#define HEIGHT 4096
+#define BO_SIZE (WIDTH * HEIGHT * sizeof(uint32_t))
+
+static uint32_t
+blit(int fd, uint32_t dst, uint32_t src, uint32_t ctx_id)
+{
+	const unsigned int copies = 1000;
+	uint32_t batch[12 * copies + 2];
+	struct drm_i915_gem_relocation_entry reloc[2 * copies];
+	struct drm_i915_gem_exec_object2 obj[3];
+	struct drm_i915_gem_execbuffer2 exec;
+	uint32_t handle;
+	int ret;
+	unsigned int i = 0, j;
+	unsigned int blit_len = 0;
+
+	for (j = 0; j < copies; j++) {
+		batch[i++] = XY_SRC_COPY_BLT_CMD |
+			XY_SRC_COPY_BLT_WRITE_ALPHA |
+			XY_SRC_COPY_BLT_WRITE_RGB;
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			batch[i - 1] |= 8;
+		else
+			batch[i - 1] |= 6;
+
+		batch[i++] = (3 << 24) | /* 32 bits */
+			(0xcc << 16) | /* copy ROP */
+			WIDTH*4;
+		batch[i++] = 0; /* dst x1,y1 */
+		batch[i++] = (HEIGHT << 16) | WIDTH; /* dst x2,y2 */
+		batch[i++] = 0; /* dst reloc */
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			batch[i++] = 0;
Quoted 8 lines

tests/gem_request_retire.c line 93

Quoted 8 lines
tests/gem_request_retire.c new file +8 -0
+		batch[i++] = 0; /* src x1,y1 */
+		batch[i++] = WIDTH*4;
+		batch[i++] = 0; /* src reloc */
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			batch[i++] = 0;
+
+		while (i % 4)
+			batch[i++] = MI_NOOP;

Is this ok? I arrived at it by experimentation, kernel was complaining that relocations are not 4-byte aligned without it.

Although now I'm thinking offsets are in bytes and my calculation below is completely wrong.. but it did not crash the GPU. Hm.

Quoted 24 lines
Quoted 24 lines
tests/gem_request_retire.c new file +24 -0
+
+		if (blit_len == 0)
+			blit_len = i;
+	}
+
+	batch[i++] = MI_BATCH_BUFFER_END;
+	batch[i++] = MI_NOOP;
+
+	handle = gem_create(fd, sizeof(batch));
+	gem_write(fd, handle, 0, batch, sizeof(batch));
+
+	for (j = 0; j < copies; j++) {
+		unsigned int r0 = j * 2;
+		unsigned int r1 = r0 + 1;
+
+		reloc[r0].target_handle = dst;
+		reloc[r0].delta = 0;
+		reloc[r0].offset = j * blit_len + 4 * sizeof(uint32_t);
+		reloc[r0].presumed_offset = 0;
+		reloc[r0].read_domains = I915_GEM_DOMAIN_RENDER;
+		reloc[r0].write_domain = I915_GEM_DOMAIN_RENDER;
+
+		reloc[r1].target_handle = src;
+		reloc[r1].delta = 0;
Quoted 8 lines

tests/gem_request_retire.c line 123

Quoted 6 lines
tests/gem_request_retire.c new file +6 -0
+		reloc[r1].offset = j * blit_len + 7 * sizeof(uint32_t);
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			reloc[r1].offset += sizeof(uint32_t);
+		reloc[r1].presumed_offset = 0;
+		reloc[r1].read_domains = I915_GEM_DOMAIN_RENDER;
+		reloc[r1].write_domain = 0;

Just fill the relocation array as you generate the batch.

To save one loop? :)

Quoted 16 lines

tests/gem_request_retire.c line 136

Quoted 13 lines
tests/gem_request_retire.c new file +13 -0
+	}
+
+	memset(obj, 0, sizeof(obj));
+	exec.buffer_count = 0;
+	obj[exec.buffer_count++].handle = dst;
+	if (src != dst)
+		obj[exec.buffer_count++].handle = src;
+	obj[exec.buffer_count].handle = handle;
+	obj[exec.buffer_count].relocation_count = 2 * copies;
+	obj[exec.buffer_count].relocs_ptr = (uintptr_t)reloc;
+	exec.buffer_count++;
+	exec.buffers_ptr = (uintptr_t)obj;
+

memset(&exec, 0, sizeof(exec));
instead of:

Quoted 12 lines

tests/gem_request_retire.c line 147

Quoted 11 lines
tests/gem_request_retire.c new file +11 -0
+	exec.batch_start_offset = 0;
+	exec.batch_len = i * sizeof(uint32_t);
+	exec.DR1 = exec.DR4 = 0;
+	exec.num_cliprects = 0;
+	exec.cliprects_ptr = 0;
+	exec.flags = I915_EXEC_BLT;
+	i915_execbuffer2_set_context_id(exec, ctx_id);
+	exec.rsvd2 = 0;
+
+	ret = drmIoctl(fd, DRM_IOCTL_I915_GEM_EXECBUFFER2, &exec);
+	igt_assert_eq(ret, 0);

gem_execbuf(fd, &exec);

Okay, it was copy&paste. :)

Quoted 57 lines
Quoted 57 lines
tests/gem_request_retire.c new file +57 -0
+
+	return handle;
+}
+
+static uint32_t
+store(int fd, uint32_t dst, uint32_t src, uint32_t ctx_id)
+{
+	uint32_t batch[6];
+	struct drm_i915_gem_relocation_entry reloc[2];
+	struct drm_i915_gem_exec_object2 obj[3];
+	struct drm_i915_gem_execbuffer2 exec;
+	uint32_t handle;
+	int ret, i = 0;
+
+	batch[i++] = MI_STORE_DWORD_IMM;
+	batch[i++] = 0; /* addrhigh */
+	batch[i++] = 0; /* addrlow */
+	batch[i++] = 0; /* val */
+	batch[i++] = MI_BATCH_BUFFER_END;
+	batch[i++] = MI_BATCH_BUFFER_END;
+
+	handle = gem_create(fd, 4096);
+	gem_write(fd, handle, 0, batch, sizeof(batch));
+
+	reloc[0].target_handle = dst;
+	reloc[0].delta = 0;
+	reloc[0].offset = 2 * sizeof(batch[0]);
+	reloc[0].presumed_offset = 0;
+	reloc[0].read_domains = I915_GEM_DOMAIN_RENDER;
+	reloc[0].write_domain = I915_GEM_DOMAIN_RENDER;
+
+	reloc[1].target_handle = src;
+	reloc[1].delta = 0;
+	reloc[1].offset = 3 * sizeof(batch[0]);
+	reloc[1].presumed_offset = 0;
+	reloc[1].read_domains = I915_GEM_DOMAIN_RENDER;
+	reloc[1].write_domain = 0;
+
+	memset(obj, 0, sizeof(obj));
+	exec.buffer_count = 0;
+	obj[exec.buffer_count++].handle = dst;
+	if (src != dst)
+		obj[exec.buffer_count++].handle = src;
+	obj[exec.buffer_count].handle = handle;
+	obj[exec.buffer_count].relocation_count = 2;
+	obj[exec.buffer_count].relocs_ptr = (uintptr_t)reloc;
+	exec.buffer_count++;
+	exec.buffers_ptr = (uintptr_t)obj;
+
+	exec.batch_start_offset = 0;
+	exec.batch_len = i * sizeof(uint32_t);
+	exec.DR1 = exec.DR4 = 0;
+	exec.num_cliprects = 0;
+	exec.cliprects_ptr = 0;
+	exec.flags = I915_EXEC_RENDER;
+	i915_execbuffer2_set_context_id(exec, ctx_id);
+	exec.rsvd2 = 0;
Quoted 8 lines

tests/gem_request_retire.c line 147

Quoted 3 lines
tests/gem_request_retire.c new file +3 -0
+
+	ret = drmIoctl(fd, DRM_IOCTL_I915_GEM_EXECBUFFER2, &exec);
+	igt_assert_eq(ret, 0);

For the purposes of your test, you don't even need to execute anything, just associate the dst handle with an empty batch on the render ring. You don't even need a fake relocation since the test already requires a fairly recent kernel.

I thought so but couldn't get it to work. Kept getting EINVAL which is a pig to find the real reason with execbuf. I'll try some more.

Quoted 40 lines
Quoted 39 lines
tests/gem_request_retire.c new file +38 -0
+
+	return handle;
+}
+
+/*
+ * A single bo is operated from batchbuffers submitted from two contexts and on
+ * different rings.
+ * One execbuf finishes way ahead of the other at which point the respective
+ * context is destroyed.
+ */
+static void
+test_retire_vma_not_inactive(int fd)
+{
+	uint32_t ctx_id;
+	uint32_t src, dst, dst2;
+	uint32_t blit_bb, store_bb;
+
+	igt_require(HAS_BLT_RING(intel_get_drm_devid(fd)));
+
+	ctx_id = gem_context_create(fd);
+	igt_assert(ctx_id);
+
+	/* Create some bos batch buffers will operate on. */
+	src = gem_create(fd, BO_SIZE);
+	igt_assert(src);
+	dst = gem_create(fd, BO_SIZE);
+	igt_assert(dst);
+	dst2 = gem_create(fd, 4096);
+	igt_assert(dst2);
+
+	/* Submit a long running batch. */
+	blit_bb = blit(fd, dst, src, 0);
+	/* Submit a quick batch referencing the same object. */
+	store_bb = store(fd, dst2, src, ctx_id);
+	/* Wait for the quick batch to complete. */
+	gem_sync(fd, store_bb);
+	gem_sync(fd, dst2);
+	gem_close(fd, store_bb);
Quoted 2 lines

tests/gem_request_retire.c line 246

Quoted 1 line
tests/gem_request_retire.c new file +1 -0
+	/* Do it again to ensure in kernel retirement is triggered. */

Circumstantial!

Yes I don't think it is actually required but a leftover from development.

Key is that retire work handler runs before the context destruction, since execlist retire is only runs from there, and it is keeping references to requests which have been processed long ago.

So I was also thinking - should we call the execlist retire requests more often in general? Execbuf path calls the normal ring retirement so maybe add it there?

Quoted 10 lines

tests/gem_request_retire.c line 254

Quoted 8 lines
tests/gem_request_retire.c new file +8 -0
+	store_bb = store(fd, dst2, src, ctx_id);
+	gem_sync(fd, store_bb);
+	gem_sync(fd, dst2);
+	gem_close(fd, store_bb);
+	gem_close(fd, dst2);
+
+	/* Now destroy the context in which the quick batch was submitted. */
+	gem_context_destroy(fd, ctx_id);

Feels very fragile. Isn't the bug something like

Well my blit runs for 6-7 seconds on a fast SKL so only fragility is that retire work handler manages to get scheduled during that window. So I think it is fine.

Quoted 3 lines

batch = big_render_copy(ctx);
context_close(ctx);
gem_sync(batch);

I don't think so. It needs two contexts so that object can be active on two rings and we can destroy one context with another one still active.

Regards,

Tvrtko
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx

#3 lore raw
Quoted 5 lines · Tvrtko Ursulin
Quoted 2 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+		while (i % 4)
+			batch[i++] = MI_NOOP;

Is this ok? I arrived at it by experimentation, kernel was complaining that relocations are not 4-byte aligned without it.

batch is u32, so all is good. What the kernel would be complaining about would be that the batch must be quadword aligned in length I guess.

Quoted 37 lines

Although now I'm thinking offsets are in bytes and my calculation below is completely wrong.. but it did not crash the GPU. Hm.

Quoted 24 lines
Quoted 24 lines
tests/gem_request_retire.c new file +24 -0
+
+		if (blit_len == 0)
+			blit_len = i;
+	}
+
+	batch[i++] = MI_BATCH_BUFFER_END;
+	batch[i++] = MI_NOOP;
+
+	handle = gem_create(fd, sizeof(batch));
+	gem_write(fd, handle, 0, batch, sizeof(batch));
+
+	for (j = 0; j < copies; j++) {
+		unsigned int r0 = j * 2;
+		unsigned int r1 = r0 + 1;
+
+		reloc[r0].target_handle = dst;
+		reloc[r0].delta = 0;
+		reloc[r0].offset = j * blit_len + 4 * sizeof(uint32_t);
+		reloc[r0].presumed_offset = 0;
+		reloc[r0].read_domains = I915_GEM_DOMAIN_RENDER;
+		reloc[r0].write_domain = I915_GEM_DOMAIN_RENDER;
+
+		reloc[r1].target_handle = src;
+		reloc[r1].delta = 0;
Quoted 8 lines

tests/gem_request_retire.c line 123

Quoted 6 lines
tests/gem_request_retire.c new file +6 -0
+		reloc[r1].offset = j * blit_len + 7 * sizeof(uint32_t);
+		if (intel_gen(intel_get_drm_devid(fd)) >= 8)
+			reloc[r1].offset += sizeof(uint32_t);
+		reloc[r1].presumed_offset = 0;
+		reloc[r1].read_domains = I915_GEM_DOMAIN_RENDER;
+		reloc[r1].write_domain = 0;

Just fill the relocation array as you generate the batch.

Quoted 1 line

To save one loop? :)

It was more to do with getting the offsets correct. The blit_len looks very fragile and confusing.

Quoted 30 lines
Quoted 17 lines
Quoted 4 lines
tests/gem_request_retire.c new file +3 -0
+	}
+
+	memset(obj, 0, sizeof(obj));

tests/gem_request_retire.c line 136

Quoted 10 lines
tests/gem_request_retire.c new file +10 -0
+	exec.buffer_count = 0;
+	obj[exec.buffer_count++].handle = dst;
+	if (src != dst)
+		obj[exec.buffer_count++].handle = src;
+	obj[exec.buffer_count].handle = handle;
+	obj[exec.buffer_count].relocation_count = 2 * copies;
+	obj[exec.buffer_count].relocs_ptr = (uintptr_t)reloc;
+	exec.buffer_count++;
+	exec.buffers_ptr = (uintptr_t)obj;
+

memset(&exec, 0, sizeof(exec));
instead of:

Quoted 12 lines

tests/gem_request_retire.c line 147

Quoted 11 lines
tests/gem_request_retire.c new file +11 -0
+	exec.batch_start_offset = 0;
+	exec.batch_len = i * sizeof(uint32_t);
+	exec.DR1 = exec.DR4 = 0;
+	exec.num_cliprects = 0;
+	exec.cliprects_ptr = 0;
+	exec.flags = I915_EXEC_BLT;
+	i915_execbuffer2_set_context_id(exec, ctx_id);
+	exec.rsvd2 = 0;
+
+	ret = drmIoctl(fd, DRM_IOCTL_I915_GEM_EXECBUFFER2, &exec);
+	igt_assert_eq(ret, 0);

gem_execbuf(fd, &exec);

Quoted 1 line

Okay, it was copy&paste. :)

I haven't killed them all yet.

Quoted 53 lines
Quoted 4 lines

For the purposes of your test, you don't even need to execute anything, just associate the dst handle with an empty batch on the render ring. You don't even need a fake relocation since the test already requires a fairly recent kernel.

I thought so but couldn't get it to work. Kept getting EINVAL which is a pig to find the real reason with execbuf. I'll try some more.

Quoted 40 lines
Quoted 39 lines
tests/gem_request_retire.c new file +38 -0
+
+	return handle;
+}
+
+/*
+ * A single bo is operated from batchbuffers submitted from two contexts and on
+ * different rings.
+ * One execbuf finishes way ahead of the other at which point the respective
+ * context is destroyed.
+ */
+static void
+test_retire_vma_not_inactive(int fd)
+{
+	uint32_t ctx_id;
+	uint32_t src, dst, dst2;
+	uint32_t blit_bb, store_bb;
+
+	igt_require(HAS_BLT_RING(intel_get_drm_devid(fd)));
+
+	ctx_id = gem_context_create(fd);
+	igt_assert(ctx_id);
+
+	/* Create some bos batch buffers will operate on. */
+	src = gem_create(fd, BO_SIZE);
+	igt_assert(src);
+	dst = gem_create(fd, BO_SIZE);
+	igt_assert(dst);
+	dst2 = gem_create(fd, 4096);
+	igt_assert(dst2);
+
+	/* Submit a long running batch. */
+	blit_bb = blit(fd, dst, src, 0);
+	/* Submit a quick batch referencing the same object. */
+	store_bb = store(fd, dst2, src, ctx_id);
+	/* Wait for the quick batch to complete. */
+	gem_sync(fd, store_bb);
+	gem_sync(fd, dst2);
+	gem_close(fd, store_bb);
Quoted 2 lines

tests/gem_request_retire.c line 246

Quoted 1 line
tests/gem_request_retire.c new file +1 -0
+	/* Do it again to ensure in kernel retirement is triggered. */

Circumstantial!

Yes I don't think it is actually required but a leftover from development.

Quoted 7 lines

Key is that retire work handler runs before the context destruction, since execlist retire is only runs from there, and it is keeping references to requests which have been processed long ago.

So I was also thinking - should we call the execlist retire requests more often in general? Execbuf path calls the normal ring retirement so maybe add it there?

No. execbuf will not be calling ring retire itself in future. That is to handle an igt starvation issue, but it has interesting implications wrt overhead. Similarly, there are technical reasons why execlists retire has to be called infrequently atm (I guess it was just lucky bad placement during development), call execlists retire too often and you end up having to repin the contexts every batch.

The bug is not peculiar to execlists. You can hit it elsewhere as the last ref is dropped in i915_gem_object_retire__read before the object is not marked as inactive.

Quoted 11 lines
Quoted 10 lines

tests/gem_request_retire.c line 254

Quoted 8 lines
tests/gem_request_retire.c new file +8 -0
+	store_bb = store(fd, dst2, src, ctx_id);
+	gem_sync(fd, store_bb);
+	gem_sync(fd, dst2);
+	gem_close(fd, store_bb);
+	gem_close(fd, dst2);
+
+	/* Now destroy the context in which the quick batch was submitted. */
+	gem_context_destroy(fd, ctx_id);

Feels very fragile. Isn't the bug something like

Quoted 7 lines

Well my blit runs for 6-7 seconds on a fast SKL so only fragility is that retire work handler manages to get scheduled during that window. So I think it is fine.

Quoted 3 lines

batch = big_render_copy(ctx);
context_close(ctx);
gem_sync(batch);

That actually has to be an explicit gem_set_domain() not gem_sync().

Quoted 3 lines

I don't think so. It needs two contexts so that object can be active on two rings and we can destroy one context with another one still active.

That's not the same bug I see I guess.
-Chris

Signature · 5 lines
Chris Wilson, Intel Open Source Technology Centre
_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx
#4 lore raw

PATCH v2 1/1 igt/gem_request_retire: Provoke context destruction with active VMAs

From: Tvrtko Ursulin <[email protected]>

Test designed to trigger the
WARN_ON(!list_empty(&ppgtt->base.active_list))
in i915_gem_context_clean.

v2:
Simplify execbuf building and the test itself. Cleanup code. (Chris Wilson)

Cc Chris Wilson <[email protected]>
Cc Daniel Vetter <[email protected]>
---
 tests/.gitignore           |   1 +
 tests/Makefile.sources     |   1 +
 tests/gem_request_retire.c | 236 +++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 238 insertions(+)
 create mode 100644 tests/gem_request_retire.c
tests/.gitignore +1 -0
index 80af9a718f06..85936ea45c9f 100644
@@ -91,6 +91,7 @@ gem_render_copy
 gem_render_copy_redux
 gem_render_linear_blits
 gem_render_tiled_blits
+gem_request_retire
 gem_reset_stats
 gem_ring_sync_copy
 gem_ring_sync_loop
tests/Makefile.sources +1 -0
index 8fb2de8b4665..ff178f7a2df4 100644
@@ -53,6 +53,7 @@ TESTS_progs_M = \
 	gem_reloc_overflow \
 	gem_reloc_vs_gpu \
 	gem_render_copy_redux \
+	gem_request_retire \
 	gem_reset_stats \
 	gem_ringfill \
 	gem_set_tiling_vs_blt \
tests/gem_request_retire.c new file +236 -0

236 lines not on this page · raw message

Signature · 6 lines
1.9.1

_______________________________________________
Intel-gfx mailing list
[email protected]
http://lists.freedesktop.org/mailman/listinfo/intel-gfx