Skip to contentSkip to conversation

add cgroup_dmem test

PATCH In review Thadeu Lima de Souza Cascardo 2 participants 16 messages Sep 23 – Oct 22

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

v7 cover add cgroup_dmem test

v6 TLDR:
Change igt_dmem_driver API by moving handle array allocation out of it and into the test itself and including dmem.max new behavior test.

This patchset is based on the work of Thomas Hellström at [1] and a followup I have submitted at [2].

With this version, I reverted the order in which to apply the patchsets, as this one adds the infrastructure and the drivers mechanisms, and a test for dmem.current and its behavior around dmem.max that is currently supported in the kernel.

As for the new dmem.max eviction behavior, introduced by [3], it has been now included since it has been merged to Linux v7.3-rc1.

I have run the tests, including the ones that test dmem.max eviction behavior on a system with a 1GiB VRAM amdgpu card. Rodrigo Siqueira helped me run it on a Xe system. We have had applied the patchset at [3] on top of then current linux origin/master [5], with a fixup for the amdgpu driver.

This latest version has not been tested on Xe, except for the dmem.current behavior, which was tested with a 7.0 kernel.

Latest tests for amdgpu used v7.3-rc1.

Now, for the changes since v1 [2]:
- Changed the test name from drv_dmem_cgroups to dmem_cgroups.
- Have system includes before igt ones.
- Documented the current SUBTEST.
- Changed free_vram driver hook to only release a given BO. This allows for the eviction trigger by a new allocation added by write_eviction_nonblock subtest.
- Changed allocate_vram to keep more state necessary for releasing a single BO.
- Added a global free_vram helper to free all allocated VRAM.

Compared to v3 of [1]:
- I have kept wait_for_usage_drop as release can be lazy and we test for it on test_current SUBTEST, but had USAGE_DROP_TIMEOUT_MS be a little longer.
- Changed BO_SIZE and EVICT_STEP to allow for more than two eviction steps on a 1GiB VRAM system.
- Reduced USAGE_SLACK to avoid false positives, but still allow for some page table accounting.

Compared to v2 of this one [6]:

- Ignore -ENOMEM and -ENOSPC when trying to fill VRAM.
- When limiting max memory, allow for some slack, as there might be some overhead.
- On Xe, allocate GEM object with no deferred backing, and skip binding.

Compared to v3 of this one [7]:

- Merge dmem_cgroups tests into cgroup_dmem and making cgroup_dmem original test a subtest.
- Remove assertion when trying to verify allocations have failed as amdgpu will fallback to GTT.

Compared to v4 of this one [8]:
- Removed the creation of vm in Xe, that was leading to extra bytes being used and was no longer necessary.
- Removed the USAGE_SLACK that was necessary due to the extra bytes allocated due to the use of vm in Xe.

Compared to v5 of this one [10]:
- Documented igt_dmem_driver structure.
- Simplified igt_dmem_driver API, such that handle array allocation is not done, and handles are returned as part of allocation.
- Make get_region_name callback use the context, instead of fd, which avoids repeating the vram region lookup for Xe.
- Moved handle array allocation into the test.
- init/deinit the igt_dmem_driver per driver, not per subtest.

[1] https://patchwork.freedesktop.org/series/163935/#rev3
[2] https://patchwork.freedesktop.org/series/166496/
[3] https://lore.kernel.org/dri-devel/[email protected]/
[4] https://gitlab.freedesktop.org/cascardo/igt-gpu-tools/-/commits/dmem_max?ref_type=heads
[5] 502d801f0ab0 ("Merge tag 'erofs-for-7.2-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/xiang/erofs")
[6] https://lore.kernel.org/dri-devel/[email protected]/
[7] https://lore.kernel.org/dri-devel/[email protected]/
[8] https://lore.kernel.org/dri-devel/[email protected]/
[9] https://lore.kernel.org/dri-devel/[email protected]/
[10] https://lore.kernel.org/dri-devel/[email protected]/

To: [email protected]
Cc: [email protected]
Cc: Thadeu Lima de Souza Cascardo <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: Christian Koenig <[email protected]>
Cc: [email protected]
Cc: Thomas Hellström <[email protected]>
Cc: Kamil Konieczny <[email protected]>
Cc: Janusz Krzysztofik <[email protected]>
Cc: Vitaly Prosyak <[email protected]>
Cc: Natalie Vock <[email protected]>
Cc: Tvrtko Ursulin <[email protected]>
Cc: [email protected]

--
2.47.3

---
Changes in v7:
- igt_cgroup_free is now called by an exit handler and cannot be called directly.
- cgroup names are namespaced by subtest name so they do not conflict, both changes allow cgroups to be removed in case of test failures.
- igt_cgroup_exit_current is introduced to allow process to exit cgroup and go back to root cgroup.
- Driver is closed and reopened between tests, dropping any leftover usage in case of failures.
- Call igt_stop_signal_helper at test fixture to prevet child leak in case of failures.
- Update/fix copyright, comments and documentation as requested in reviews.
- Link to v6: https://patch.msgid.link/[email protected]

---
Thadeu Lima de Souza Cascardo (4):
lib/xe: Introduce dmem driver and implement Xe support lib/amdgpu: add amdgpu_cgroup_region_name lib/amdgpu: add amdgpu support to igt_dmem_driver tests/cgroup_dmem: add test for dmem.current

Thomas Hellström (5):
lib/igt_cgroup: add cgroup v2 and dmem controller helpers tests/cgroup_dmem: add dmem cgroup controller test lib/xe: add xe_cgroup_region_name() helper tests/cgroup_dmem: add dmem cgroup eviction test tests/cgroup_dmem: add write_eviction_nonblock subtest

 lib/amdgpu/amd_dmem.c   |  90 +++++++
 lib/amdgpu/amd_memory.c |  25 ++
 lib/amdgpu/amd_memory.h |   2 +
 lib/igt.h               |   1 +
 lib/igt_cgroup.c        | 685 ++++++++++++++++++++++++++++++++++++++++++++++++
 lib/igt_cgroup.h        |  56 ++++
 lib/igt_dmem_driver.h   |  36 +++
 lib/meson.build         |   3 +
 lib/xe/xe_dmem.c        |  98 +++++++
 lib/xe/xe_query.c       |  32 +++
 lib/xe/xe_query.h       |   2 +
 tests/cgroup_dmem.c     | 542 ++++++++++++++++++++++++++++++++++++++
 tests/meson.build       |   1 +
 13 files changed, 1573 insertions(+)

---
base-commit: 4aeabf104b136437d0ccc57caf32a752587d595c
change-id: 20260902-dmem_max-1a5e837347a8

Best regards,

Signature · 1 line
Thadeu Lima de Souza Cascardo <[email protected]>
#1 lore raw

PATCH v7 1/9 lib/igt_cgroup: add cgroup v2 and dmem controller helpers

From: Thomas Hellström <[email protected]>

Add igt_cgroup, a library module providing helpers to create and manage cgroup v2 sub-cgroups from IGT tests, with support for the dmem controller that governs device memory (e.g. GPU VRAM) limits.

The API covers:
- igt_cgroup_new() / igt_cgroup_free(): create and destroy a named sub-cgroup under the unified cgroupv2 hierarchy, enabling the dmem controller automatically.
- igt_cgroup_move_current(): move the calling process into a cgroup.
- igt_cgroup_dmem_set/get_max/min/low(): write and read dmem.max, dmem.min and dmem.low for a named device memory region.
- igt_cgroup_dmem_get_current(): read current per-cgroup device memory usage.
- igt_cgroup_dmem_get_system_current(): read system-wide device memory usage from the root cgroup.
- igt_cgroup_dmem_get_capacity(): read total region capacity from the root cgroup's dmem.capacity file.
- igt_cgroup_dmem_regions() / igt_cgroup_dmem_regions_free(): enumerate all registered device memory regions.

All public API functions that can fail use igt_assert internally rather than returning error codes, following the IGT convention.

cgroup names embed the subtest name, allowing an exit handler to be used to remove them all at the end of the test.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
 lib/igt.h        |   1 +
 lib/igt_cgroup.c | 685 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
 lib/igt_cgroup.h |  56 +++++
 lib/meson.build  |   1 +
 4 files changed, 743 insertions(+)
lib/igt.h +1 -0
index 173ca70bff28..d8e5de7dcd2f 100644
@@ -27,6 +27,7 @@
 #include "drmtest.h"
 #include "i915_3d.h"
 #include "igt_aux.h"
+#include "igt_cgroup.h"
 #include "igt_configfs.h"
 #include "igt_core.h"
 #include "igt_debugfs.h"
lib/igt_cgroup.c new file +685 -0

685 lines not on this page · raw message

lib/igt_cgroup.h new file +56 -0

56 lines not on this page · raw message

lib/meson.build +1 -0
index f3e6d96693a1..df5f13efac56 100644
@@ -19,6 +19,7 @@ lib_sources = [
 	'i915/i915_dp.c',
 	'igt_collection.c',
 	'igt_color_encoding.c',
+	'igt_cgroup.c',
 	'igt_configfs.c',
 	'igt_facts.c',
 	'igt_crc.c',
Signature · 1 line
2.47.3
#2 lore raw

Re: [PATCH i-g-t v7 1/9] lib/igt_cgroup: add cgroup v2 and dmem controller helpers

Quoted 22 lines · Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

Add igt_cgroup, a library module providing helpers to create and manage cgroup v2 sub-cgroups from IGT tests, with support for the dmem controller that governs device memory (e.g. GPU VRAM) limits.

The API covers:
- igt_cgroup_new() / igt_cgroup_free(): create and destroy a named sub-cgroup under the unified cgroupv2 hierarchy, enabling the dmem controller automatically.
- igt_cgroup_move_current(): move the calling process into a cgroup.
- igt_cgroup_dmem_set/get_max/min/low(): write and read dmem.max, dmem.min and dmem.low for a named device memory region.
- igt_cgroup_dmem_get_current(): read current per-cgroup device memory usage.
- igt_cgroup_dmem_get_system_current(): read system-wide device memory usage from the root cgroup.
- igt_cgroup_dmem_get_capacity(): read total region capacity from the root cgroup's dmem.capacity file.
- igt_cgroup_dmem_regions() / igt_cgroup_dmem_regions_free(): enumerate all registered device memory regions.

Quoted 5 lines

All public API functions that can fail use igt_assert internally rather than returning error codes, following the IGT convention.

cgroup names embed the subtest name, allowing an exit handler to be used to remove them all at the end of the test.

I do not see that subtest name is relevant for the exit handler so perhaps just clarify this statement here.

Quoted 115 lines
Assisted-by GitHub Copilot:claude-sonnet-4.6
---
  lib/igt.h        |   1 +
  lib/igt_cgroup.c | 685 +++++++++++++++++++++++++++++++++++++++++++++++++++++++
  lib/igt_cgroup.h |  56 +++++
  lib/meson.build  |   1 +
  4 files changed, 743 insertions(+)
lib/igt.h +1 -0
index 173ca70bff28..d8e5de7dcd2f 100644
@@ -27,6 +27,7 @@
  #include "drmtest.h"
  #include "i915_3d.h"
  #include "igt_aux.h"
+#include "igt_cgroup.h"
  #include "igt_configfs.h"
  #include "igt_core.h"
  #include "igt_debugfs.h"
lib/igt_cgroup.c new file +86 -0
new file mode 100644
index 000000000000..7ada1e2efed5
@@ -0,0 +1,685 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2025 Intel Corporation
+ * Copyright © 2026 Valve Corporation
+ */
+
+/**
+ * SECTION:igt_cgroup
+ * @short_description: cgroup v2 helpers for IGT tests
+ * @title: cgroup
+ * @include: igt_cgroup.h
+ *
+ * This library provides helpers for creating and managing cgroup v2
+ * sub-cgroups from IGT tests, including support for the dmem controller
+ * which governs device memory (e.g. GPU VRAM) limits.
+ */
+
+#include <errno.h>
+#include <fcntl.h>
+#include <inttypes.h>
+#include <limits.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/statfs.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include "igt_cgroup.h"
+#include "igt_core.h"
+#include "igt_fs.h"
+
+#ifndef CGROUP2_SUPER_MAGIC
+#define CGROUP2_SUPER_MAGIC 0x63677270
+#endif
+
+/**
+ * struct igt_cgroup - Opaque handle to a cgroup v2 sub-cgroup.
+ * @dirfd: File descriptor for the cgroup directory.
+ * @path: Absolute path to the cgroup directory.
+ * @parent_path: Absolute path to the parent cgroup directory.
+ *
+ * Allocated by igt_cgroup_new() and freed by igt_cgroup_free().
+ */
+struct igt_cgroup {
+	int dirfd;
+	char *path;
+	char *parent_path;
+};
+
+static const char *cgroupv2_mount(void)
+{
+	static const char *path;
+	static const char * const candidates[] = {
+		"/sys/fs/cgroup",
+		"/sys/fs/cgroup/unified",
+		NULL,
+	};
+	struct statfs st;
+	int i;
+
+	if (path)
+		return path;
+
+	for (i = 0; candidates[i]; i++) {
+		if (statfs(candidates[i], &st) == 0 &&
+		    (unsigned long)st.f_type == CGROUP2_SUPER_MAGIC) {
+			path = candidates[i];
+			return path;
+		}
+	}
+
+	return NULL;
+}
+
+/*
+ * Move every PID listed in @cgroup_path/cgroup.procs to
+ * @parent_path/cgroup.procs.  Silently ignores individual failures
+ * (a PID may have exited between reading and writing).
+ */
+static void drain_procs_to_parent(const char *cgroup_path,
+				  const char *parent_path)
+{
+	char proc_path[PATH_MAX];
+	char parent_procs[PATH_MAX];

lib/igt_cgroup.c line 96

Quoted 10 lines
lib/igt_cgroup.c new file +10 -0
+	int parent_fd;
+	FILE *f;
+	int pid;
+
+	snprintf(proc_path, sizeof(proc_path), "%s/cgroup.procs", cgroup_path);
+	snprintf(parent_procs, sizeof(parent_procs), "%s/cgroup.procs", parent_path);
+
+	parent_fd = open(parent_procs, O_WRONLY);
+	if (parent_fd < 0)
+		return;

FWIW it not only ignores the individual failures but the "complete" failure as well. Lets see later if that matters.

Quoted 22 lines
lib/igt_cgroup.c new file +22 -0
+
+	f = fopen(proc_path, "r");
+	if (f) {
+		while (fscanf(f, "%d", &pid) == 1) {
+			char pidbuf[32];
+			ssize_t len = snprintf(pidbuf, sizeof(pidbuf), "%d", pid);
+
+			write(parent_fd, pidbuf, len);
+		}
+		fclose(f);
+	}
+
+	close(parent_fd);
+}
+
+/**
+ * igt_cgroup_free() - Destroy a cgroup and release its resources.
+ * @cg: The cgroup to destroy.
+ *
+ * Moves any processes still running inside @cg back to the parent cgroup,
+ * removes the cgroup directory, and frees all associated memory.
+ * After this call @cg must not be used.

lib/igt_cgroup.c line 128

Quoted 10 lines
lib/igt_cgroup.c new file +10 -0
+ * Called at exit time by an exit handler installed by igt_cgroup_new.
+ * Should not be called by a subtest, otherwise, might lead to a double
+ * free.
+ */
+static void igt_cgroup_free(struct igt_cgroup *cg)
+{
+	if (!cg)
+		return;
+
+	drain_procs_to_parent(cg->path, cg->parent_path);

Could take cg as the argument but whatever.

Quoted 146 lines
lib/igt_cgroup.c new file +146 -0
+
+	close(cg->dirfd);
+
+	if (rmdir(cg->path) < 0)
+		igt_debug("Failed to remove cgroup %s: %m\n", cg->path);
+
+	free(cg->path);
+	free(cg->parent_path);
+	free(cg);
+}
+
+#define MAX_CGROUPS 32
+static unsigned int cgroup_cleanup_count;
+static struct igt_cgroup *cgroup_cleanup[MAX_CGROUPS];
+
+static void igt_cgroup_exit_handler(int sig)
+{
+	int i;
+
+	for (i = 0; i < cgroup_cleanup_count; i++)
+		igt_cgroup_free(cgroup_cleanup[i]);
+}
+
+static void igt_cgroup_install_exit_handler(struct igt_cgroup *cg)
+{
+	igt_assert(cgroup_cleanup_count < MAX_CGROUPS);
+	cgroup_cleanup[cgroup_cleanup_count++] = cg;
+	igt_install_exit_handler(igt_cgroup_exit_handler);
+}
+
+/*
+ * Write "+controller" to @cgroup_path/cgroup.subtree_control to enable
+ * the named controller for children of that cgroup.
+ */
+static int enable_controller(const char *cgroup_path, const char *controller)
+{
+	char path[PATH_MAX];
+	char cmd[64];
+	ssize_t ret;
+	int fd;
+
+	snprintf(path, sizeof(path), "%s/cgroup.subtree_control", cgroup_path);
+	snprintf(cmd, sizeof(cmd), "+%s", controller);
+
+	fd = open(path, O_WRONLY);
+	if (fd < 0)
+		return -errno;
+
+	ret = write(fd, cmd, strlen(cmd));
+	close(fd);
+
+	return (ret < 0) ? -errno : 0;
+}
+
+/**
+ * igt_cgroup_new() - Create a new cgroup v2 sub-cgroup.
+ * @name: Name for the new cgroup directory.
+ *
+ * Creates a sub-cgroup named @name under the system's unified cgroupv2
+ * hierarchy.  The dmem controller is enabled in the parent's
+ * subtree_control so that igt_cgroup_dmem_set_max() and friends take effect
+ * immediately.
+ *
+ * Return: Pointer to an &struct igt_cgroup on success, %NULL on failure.
+ */
+struct igt_cgroup *igt_cgroup_new(const char *name)
+{
+	struct igt_cgroup *cg;
+	const char *mount;
+	int ret;
+
+	mount = cgroupv2_mount();
+	if (!mount) {
+		igt_debug("cgroup v2 not found\n");
+		return NULL;
+	}
+
+	cg = calloc(1, sizeof(*cg));
+	if (!cg)
+		return NULL;
+
+	cg->parent_path = strdup(mount);
+	if (!cg->parent_path)
+		goto err_free;
+
+	if (asprintf(&cg->path, "%s/%s-%s", mount, igt_subtest_name(), name) < 0) {
+		cg->path = NULL;
+		goto err_parent;
+	}
+
+	/*
+	 * Try to enable the dmem controller in the parent's subtree_control.
+	 * Ignore EINVAL which the kernel returns when the controller is already
+	 * listed (i.e. already enabled).
+	 */
+	ret = enable_controller(mount, "dmem");
+	if (ret < 0 && ret != -EINVAL)
+		igt_debug("Failed to enable dmem controller in %s: %d\n",
+			  mount, ret);
+
+	if (mkdir(cg->path, 0755) < 0 && errno != EEXIST) {
+		igt_debug("Failed to create cgroup %s: %m\n", cg->path);
+		goto err_path;
+	}
+
+	cg->dirfd = open(cg->path, O_RDONLY | O_DIRECTORY);
+	if (cg->dirfd < 0) {
+		igt_debug("Failed to open cgroup dir %s: %m\n", cg->path);
+		goto err_rmdir;
+	}
+
+	igt_cgroup_install_exit_handler(cg);
+
+	return cg;
+
+err_rmdir:
+	rmdir(cg->path);
+err_path:
+	free(cg->path);
+err_parent:
+	free(cg->parent_path);
+err_free:
+	free(cg);
+	return NULL;
+}
+
+static void igt_cgroup_move_current_to_fd(int fd)
+{
+	char pidbuf[32];
+	ssize_t len;
+	int ret;
+
+	len = snprintf(pidbuf, sizeof(pidbuf), "%d", (int)getpid());
+
+	ret = write(fd, pidbuf, len);
+	close(fd);
+
+	igt_assert_f(ret == len, "Failed to write PID to cgroup.procs: %m\n");
+}
+
+
+/**
+ * igt_cgroup_move_current() - Move the calling process into a cgroup.
+ * @cg: Target cgroup.
+ *
+ * Writes the calling process's PID to @cg's cgroup.procs file, transferring

lib/igt_cgroup.c line 284

Quoted 10 lines
lib/igt_cgroup.c new file +10 -0
+ * it into the cgroup.  All threads of the process move together.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_move_current(struct igt_cgroup *cg)
+{
+	int fd;
+
+	fd = openat(cg->dirfd, "cgroup.procs", O_WRONLY);
+	igt_assert_f(fd >= 0, "Failed to open cgroup.procs: %m\n");
+	igt_cgroup_move_current_to_fd(fd);

A bit funky fd ownership convention and prefix of igt_cgroup_move_current_to_fd but nothing seems broken at least.

Quoted 21 lines
lib/igt_cgroup.c new file +21 -0
+}
+
+/**
+ * igt_cgroup_exit_current() - Move the calling process out of cgroup.
+ */
+void igt_cgroup_exit_current(void)
+{
+	int fd;
+	char parent_procs[PATH_MAX];
+
+	snprintf(parent_procs, sizeof(parent_procs), "%s/cgroup.procs", cgroupv2_mount());
+
+	fd = open(parent_procs, O_WRONLY);
+	igt_assert_f(fd >= 0, "Failed to open cgroup.procs: %m\n");
+	igt_cgroup_move_current_to_fd(fd);
+}
+
+/*
+ * Parse a single dmem interface file line of the form "region_name value\n"
+ * where value is either a decimal byte count or the string "max".
+ * Returns 0 and writes to *out on success, -EINVAL on parse error.

lib/igt_cgroup.c line 315

Quoted 10 lines
lib/igt_cgroup.c new file +10 -0
+ */
+static int dmem_parse_line(char *line, const char *region, uint64_t *out)
+{
+	char *space = strchr(line, ' ');
+
+	if (!space)
+		return -EINVAL;
+
+	*space = '\0';
+	if (strcmp(line, region) != 0)

Looking for a long region name on a short line can cause out of bounds read here I think. Not sure if out of bounds for the whole buffer, but definitely outside the current line.

lib/igt_cgroup.c line 318

Quoted 3 lines
lib/igt_cgroup.c new file +3 -0
+		return -ENOENT;
+
+	if (strcmp(space + 1, "max") == 0) {

On the unlikely case dmem controller is returning something unexpected
space + 1 could already be out of bounds I think. This one is probably not worth guarding against though.

Quoted 48 lines
lib/igt_cgroup.c new file +48 -0
+		*out = IGT_CGROUP_DMEM_MAX;
+		return 0;
+	}
+
+	errno = 0;
+	*out = strtoull(space + 1, &space, 10);
+	if (errno || *space != '\0')
+		return -EINVAL;
+
+	return 0;
+}
+
+/*
+ * Read a dmem interface file opened relative to @dirfd, searching for
+ * @region.  On success writes the region's value to @out and returns 0.
+ * Returns -ENOENT when @region is absent, or a negative errno otherwise.
+ */
+static int dmem_read_region(int dirfd, const char *file,
+			    const char *region, uint64_t *out)
+{
+	char buf[4096];
+	char *line, *saveptr;
+	ssize_t n;
+	int fd;
+
+	fd = openat(dirfd, file, O_RDONLY);
+	if (fd < 0)
+		return -errno;
+
+	n = igt_readn(fd, buf, sizeof(buf) - 1);
+	close(fd);
+	if (n < 0)
+		return (int)n;
+	buf[n] = '\0';
+
+	for (line = strtok_r(buf, "\n", &saveptr); line;
+	     line = strtok_r(NULL, "\n", &saveptr)) {
+		int ret = dmem_parse_line(line, region, out);
+
+		if (ret != -ENOENT)
+			return ret;
+	}
+
+	return -ENOENT;
+}
+
+/*
+ * Write "region_name value" (or "region_name max") to the dmem interface

lib/igt_cgroup.c line 376

Quoted 10 lines
lib/igt_cgroup.c new file +10 -0
+ * file @file opened relative to @dirfd.
+ * If @nonblock is true the file is opened with O_NONBLOCK, causing any
+ * eviction triggered by the limit change to be skipped rather than waited
+ * for; the write still succeeds (returns 0).
+ * Returns 0 on success, negative errno on failure.
+ */
+static int dmem_write_region(int dirfd, const char *file,
+			     const char *region, uint64_t bytes, bool nonblock)
+{
+	char buf[PATH_MAX + 64];

I don't see a connection to PATH_MAX, and then with +64 it's even more weird. The read has a 4k limit for the whole region list so perhaps 4k for one of the lines from that list is also enough.

Quoted 354 lines
lib/igt_cgroup.c new file +309 -0
+	ssize_t len;
+	int fd, ret;
+	int flags = O_WRONLY;
+
+	if (bytes == IGT_CGROUP_DMEM_MAX)
+		len = snprintf(buf, sizeof(buf), "%s max", region);
+	else
+		len = snprintf(buf, sizeof(buf), "%s %" PRIu64, region, bytes);
+
+	if (nonblock)
+		flags |= O_NONBLOCK;
+
+	fd = openat(dirfd, file, flags);
+	if (fd < 0)
+		return -errno;
+
+	do {
+		ret = write(fd, buf, len);
+		if (ret < 0 && errno == EINTR)
+			igt_debug("dmem cgroup write interrupted by signal, retrying\n");
+	} while (ret < 0 && errno == EINTR);
+	close(fd);
+
+	return (ret < 0) ? -errno : 0;
+}
+
+/**
+ * igt_cgroup_dmem_set_max() - Set the hard device memory limit for a region.
+ * @cg: Target cgroup.
+ * @region: Device memory region name (e.g. "drm/0000:03:00.0/vram0").
+ * @bytes: Hard limit in bytes.  Use %IGT_CGROUP_DMEM_MAX for no limit.
+ * @nonblock: If true, open the file with O_NONBLOCK so that eviction
+ *            triggered by the limit change is skipped rather than awaited.
+ *
+ * Writes @bytes to dmem.max for @region inside @cg.  Allocation attempts
+ * that would push usage past this limit fail with -EAGAIN in the kernel.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_set_max(struct igt_cgroup *cg, const char *region,
+			     uint64_t bytes, bool nonblock)
+{
+	igt_assert_f(dmem_write_region(cg->dirfd, "dmem.max", region, bytes,
+				       nonblock) == 0,
+		     "Failed to set dmem.max for region %s\n", region);
+}
+
+/**
+ * igt_cgroup_dmem_set_min() - Set the hard protection threshold for a region.
+ * @cg: Target cgroup.
+ * @region: Device memory region name.
+ * @bytes: Hard protection threshold in bytes.  Pass 0 to disable.
+ *
+ * Writes @bytes to dmem.min for @region inside @cg.  Device memory below
+ * this threshold is never reclaimed regardless of system pressure.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_set_min(struct igt_cgroup *cg, const char *region,
+			     uint64_t bytes)
+{
+	igt_assert_f(dmem_write_region(cg->dirfd, "dmem.min", region, bytes,
+				       false) == 0,
+		     "Failed to set dmem.min for region %s\n", region);
+}
+
+/**
+ * igt_cgroup_dmem_set_low() - Set the soft protection threshold for a region.
+ * @cg: Target cgroup.
+ * @region: Device memory region name.
+ * @bytes: Soft protection threshold in bytes.  Pass 0 to disable.
+ *
+ * Writes @bytes to dmem.low for @region inside @cg.  Device memory below
+ * this threshold is only reclaimed when no unprotected memory remains.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_set_low(struct igt_cgroup *cg, const char *region,
+			     uint64_t bytes)
+{
+	igt_assert_f(dmem_write_region(cg->dirfd, "dmem.low", region, bytes,
+				       false) == 0,
+		     "Failed to set dmem.low for region %s\n", region);
+}
+
+/**
+ * igt_cgroup_dmem_get_current() - Read current device memory usage for a region.
+ * @cg: Target cgroup.
+ * @region: Device memory region name.
+ * @out: Receives the current usage in bytes.
+ *
+ * Reads dmem.current from @cg and returns the usage for @region.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_get_current(struct igt_cgroup *cg, const char *region,
+				 uint64_t *out)
+{
+	igt_assert_f(dmem_read_region(cg->dirfd, "dmem.current", region, out) == 0,
+		     "Failed to read dmem.current for region %s\n", region);
+}
+
+/**
+ * igt_cgroup_dmem_get_max() - Read the configured hard limit for a region.
+ * @cg: Target cgroup.
+ * @region: Device memory region name.
+ * @out: Receives the limit in bytes, or %IGT_CGROUP_DMEM_MAX if unset.
+ *
+ * Reads dmem.max from @cg for @region.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_get_max(struct igt_cgroup *cg, const char *region,
+			     uint64_t *out)
+{
+	igt_assert_f(dmem_read_region(cg->dirfd, "dmem.max", region, out) == 0,
+		     "Failed to read dmem.max for region %s\n", region);
+}
+
+/**
+ * igt_cgroup_dmem_get_min() - Read the configured hard protection threshold for a region.
+ * @cg: Target cgroup.
+ * @region: Device memory region name.
+ * @out: Receives the threshold in bytes.
+ *
+ * Reads dmem.min from @cg for @region.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_get_min(struct igt_cgroup *cg, const char *region,
+			     uint64_t *out)
+{
+	igt_assert_f(dmem_read_region(cg->dirfd, "dmem.min", region, out) == 0,
+		     "Failed to read dmem.min for region %s\n", region);
+}
+
+/**
+ * igt_cgroup_dmem_get_low() - Read the configured soft protection threshold for a region.
+ * @cg: Target cgroup.
+ * @region: Device memory region name.
+ * @out: Receives the threshold in bytes.
+ *
+ * Reads dmem.low from @cg for @region.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_get_low(struct igt_cgroup *cg, const char *region,
+			     uint64_t *out)
+{
+	igt_assert_f(dmem_read_region(cg->dirfd, "dmem.low", region, out) == 0,
+		     "Failed to read dmem.low for region %s\n", region);
+}
+
+/**
+ * igt_cgroup_dmem_available() - Check if the dmem cgroup controller is available.
+ *
+ * Probes the cgroup v2 hierarchy for the presence of a dmem.capacity file at
+ * the root, indicating that the kernel dmem controller is compiled in and at
+ * least one device memory region has been registered.
+ *
+ * Return: %true if the dmem controller is available, %false otherwise.
+ */
+bool igt_cgroup_dmem_available(void)
+{
+	char **regions = igt_cgroup_dmem_regions();
+
+	if (!regions)
+		return false;
+
+	igt_cgroup_dmem_regions_free(regions);
+	return true;
+}
+
+/**
+ * igt_cgroup_dmem_regions() - Enumerate all registered device memory regions.
+ *
+ * Reads the root cgroup's dmem.capacity file and returns a NULL-terminated
+ * array of region name strings.  Each name can be passed directly to
+ * igt_cgroup_dmem_get_capacity(), igt_cgroup_dmem_get_current(), and the
+ * igt_cgroup_dmem_set_*() / igt_cgroup_dmem_get_*() family.
+ *
+ * Free the returned array with igt_cgroup_dmem_regions_free().
+ *
+ * Return: A NULL-terminated array of strings on success, %NULL if cgroupv2
+ * is unavailable or no regions are registered.
+ */
+char **igt_cgroup_dmem_regions(void)
+{
+	char buf[4096];
+	char *line, *saveptr, *space, *name;
+	char **regions = NULL, **tmp;
+	int count = 0;
+	const char *mount;
+	ssize_t n;
+	int dirfd, fd;
+
+	mount = cgroupv2_mount();
+	if (!mount)
+		return NULL;
+
+	dirfd = open(mount, O_RDONLY | O_DIRECTORY);
+	if (dirfd < 0)
+		return NULL;
+
+	fd = openat(dirfd, "dmem.capacity", O_RDONLY);
+	close(dirfd);
+	if (fd < 0)
+		return NULL;
+
+	n = igt_readn(fd, buf, sizeof(buf) - 1);
+	close(fd);
+	if (n <= 0)
+		return NULL;
+	buf[n] = '\0';
+
+	for (line = strtok_r(buf, "\n", &saveptr); line;
+	     line = strtok_r(NULL, "\n", &saveptr)) {
+		space = strchr(line, ' ');
+
+		if (!space)
+			continue;
+		*space = '\0';
+
+		name = strdup(line);
+		if (!name)
+			goto err;
+
+		tmp = realloc(regions, (count + 2) * sizeof(*regions));
+		if (!tmp) {
+			free(name);
+			goto err;
+		}
+		regions = tmp;
+		regions[count++] = name;
+		regions[count] = NULL;
+	}
+
+	return regions;
+
+err:
+	igt_cgroup_dmem_regions_free(regions);
+	return NULL;
+}
+
+/**
+ * igt_cgroup_dmem_regions_free() - Free a region list returned by igt_cgroup_dmem_regions().
+ * @regions: NULL-terminated array returned by igt_cgroup_dmem_regions().
+ *
+ * Frees each string in @regions and the array itself.  Safe to call with
+ * %NULL.
+ */
+void igt_cgroup_dmem_regions_free(char **regions)
+{
+	int i;
+
+	if (!regions)
+		return;
+
+	for (i = 0; regions[i]; i++)
+		free(regions[i]);
+
+	free(regions);
+}
+
+/**
+ * igt_cgroup_dmem_get_capacity() - Read total device memory capacity for a region.
+ * @region: Device memory region name.
+ * @out: Receives the total capacity in bytes.
+ *
+ * Reads dmem.capacity from the root cgroup and returns the capacity for
+ * @region.  This reflects the maximum allocatable bytes, excluding memory
+ * reserved by the kernel for internal use.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_get_capacity(const char *region, uint64_t *out)
+{
+	const char *mount;
+	int dirfd, ret;
+
+	mount = cgroupv2_mount();
+	igt_assert_f(mount, "cgroup v2 not available\n");
+
+	dirfd = open(mount, O_RDONLY | O_DIRECTORY);
+	igt_assert_f(dirfd >= 0, "Failed to open cgroup root: %m\n");
+
+	ret = dmem_read_region(dirfd, "dmem.capacity", region, out);
+	close(dirfd);
+
+	igt_assert_f(ret == 0, "Failed to read dmem.capacity for region %s\n", region);
+}
+
+/**
+ * igt_cgroup_dmem_get_system_current() - Read system-wide device memory usage for a region.
+ * @region: Device memory region name.
+ * @out: Receives the total system-wide usage in bytes.
+ *
+ * Reads dmem.current from the root cgroup for @region.  This reflects the
+ * aggregate device memory usage across all cgroups on the system.
+ * Fails the test via igt_assert on error.
+ */
+void igt_cgroup_dmem_get_system_current(const char *region, uint64_t *out)
+{
+	const char *mount;
+	int dirfd, ret;
+
+	mount = cgroupv2_mount();
+	igt_assert_f(mount, "cgroup v2 not available\n");
+
+	dirfd = open(mount, O_RDONLY | O_DIRECTORY);
+	igt_assert_f(dirfd >= 0, "Failed to open cgroup root: %m\n");
+
+	ret = dmem_read_region(dirfd, "dmem.current", region, out);
+	close(dirfd);
+
+	igt_assert_f(ret == 0, "Failed to read root dmem.current for region %s\n", region);
+}
lib/igt_cgroup.h new file +39 -0
new file mode 100644
index 000000000000..5115c60427da
@@ -0,0 +1,56 @@
+/* SPDX-License-Identifier: MIT */
+/*
+ * Copyright © 2025 Intel Corporation
+ */
+
+#ifndef __IGT_CGROUP_H__
+#define __IGT_CGROUP_H__
+
+#include <stdbool.h>
+#include <stdint.h>
+
+/**
+ * IGT_CGROUP_DMEM_MAX - Sentinel value meaning "no device memory limit".
+ *
+ * Pass this to igt_cgroup_dmem_set_max() to remove a previously set limit,
+ * equivalent to writing "max" to the dmem.max interface file.
+ */
+#define IGT_CGROUP_DMEM_MAX UINT64_MAX
+
+/**
+ * struct igt_cgroup - Opaque handle to a cgroup v2 sub-cgroup.
+ *
+ * Allocated by igt_cgroup_new() and freed at exit time.
+ * All other functions in this module take a pointer to this type.
+ */
+struct igt_cgroup;
+
+struct igt_cgroup *igt_cgroup_new(const char *name);
+
+void igt_cgroup_move_current(struct igt_cgroup *cg);
+void igt_cgroup_exit_current(void);
+
+void igt_cgroup_dmem_set_max(struct igt_cgroup *cg, const char *region,
+			     uint64_t bytes, bool nonblock);
+void igt_cgroup_dmem_set_min(struct igt_cgroup *cg, const char *region,
+			     uint64_t bytes);
+void igt_cgroup_dmem_set_low(struct igt_cgroup *cg, const char *region,
+			     uint64_t bytes);
+

lib/igt_cgroup.h line 49

Quoted 10 lines
lib/igt_cgroup.h new file +10 -0
+void igt_cgroup_dmem_get_max(struct igt_cgroup *cg, const char *region,
+			     uint64_t *out);
+void igt_cgroup_dmem_get_min(struct igt_cgroup *cg, const char *region,
+			     uint64_t *out);
+void igt_cgroup_dmem_get_low(struct igt_cgroup *cg, const char *region,
+			     uint64_t *out);
+
+void igt_cgroup_dmem_get_current(struct igt_cgroup *cg, const char *region,
+				 uint64_t *out);
+void igt_cgroup_dmem_get_capacity(const char *region, uint64_t *out);

All igt_cgroup_dmem_get apart from _current work on igt_cgroup. I get it.. capacity is system wide by definition. Just that it briefly threw me off together with the igt_cgroup_dmem_get_system_current below for which I thought is it system memory, surely not I thought. Would renaming it to igt_cgroup_dmem_get_system_capacity make sense? Or s/system/total/?

Slightly related question - how are multiple GPUs of a same model handled in dmem? Are the memory region lines prefixed with some sort of a GPU id?

Quoted 13 lines
lib/igt_cgroup.h new file +7 -0
+void igt_cgroup_dmem_get_system_current(const char *region, uint64_t *out);
+
+bool igt_cgroup_dmem_available(void);
+char **igt_cgroup_dmem_regions(void);
+void igt_cgroup_dmem_regions_free(char **regions);
+
+#endif /* __IGT_CGROUP_H__ */
lib/meson.build +0 -0
index f3e6d96693a1..df5f13efac56 100644

lib/meson.build lib_sources = [ · line 25

Quoted 7 lines
lib/meson.build +1 -0
@@ -19,6 +19,7 @@ lib_sources = [
  	'i915/i915_dp.c',
  	'igt_collection.c',
  	'igt_color_encoding.c',
+	'igt_cgroup.c',
  	'igt_configfs.c',
  	'igt_facts.c',
  	'igt_crc.c',

In summary, I think the only things which I would like to addresses are:

1. Commit text
2. PATH_MAX + 64
3. Potential out of bounds strcmp or whatever it was.

With those r-b.

Regards,

Tvrtko

#3 lore raw

PATCH v7 2/9 tests/cgroup_dmem: add dmem cgroup controller test

In reply to Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

Add a test that exercises the cgroup v2 dmem controller interface using the new igt_cgroup library.

The test uses igt_simple_main and:
- Skips if no dmem regions are registered (no cgroup v2 or no dmem-capable device).
- Creates a sub-cgroup and moves the test process into it.
- Enumerates all registered device memory regions and prints their capacity, system-wide current usage, per-cgroup current usage, and configured min, low and max limits.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
 tests/cgroup_dmem.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++++
 tests/meson.build   |  1 +
 2 files changed, 92 insertions(+)
tests/cgroup_dmem.c new file +91 -0
new file mode 100644
index 000000000000..2ccd8bdb00ae
@@ -0,0 +1,91 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2025 Intel Corporation
+ */
+
+/**
+ * TEST: cgroup dmem
+ * Description: Exercises the cgroup v2 dmem controller interface.  Creates a
+ *              cgroup, moves the process into it, enumerates all dmem regions,
+ *              prints their capacity, system-wide current usage, per-cgroup
+ *              current usage and configured limits, then destroys the cgroup.
+ * Category: Core
+ * Mega feature: General Core features
+ * Sub-category: uapi
+ * Functionality: cgroup
+ * Feature: dmem
+ * Test category: uapi
+ */
+
+#include <inttypes.h>
+
+#include "igt.h"
+#include "igt_cgroup.h"
+
+IGT_TEST_DESCRIPTION("Exercises the cgroup v2 dmem controller interface.");
+
+static void fmt_bytes(uint64_t v, char *buf, size_t len)
+{
+	if (v == IGT_CGROUP_DMEM_MAX)
+		snprintf(buf, len, "max");
+	else
+		snprintf(buf, len, "%" PRIu64, v);
+}
+
+int igt_simple_main()
+{
+	struct igt_cgroup *cg;
+	const char *region;
+	char **regions;
+	uint64_t capacity, sys_current, cg_current, min, low, max;
+	char cap_s[32], sys_s[32], cg_s[32];
+	char min_s[32], low_s[32], max_s[32];
+	int i;
+
+	igt_require_f(igt_cgroup_dmem_available(),
+		      "No dmem regions found; is cgroup v2 with the "
+		      "dmem controller available?\n");
+
+	cg = igt_cgroup_new("igt-cgroup-dmem-test");
+	igt_assert_f(cg, "Failed to create cgroup\n");
+
+	igt_cgroup_move_current(cg);
+
+	regions = igt_cgroup_dmem_regions();
+	igt_assert_f(regions, "Failed to enumerate dmem regions\n");
+
+	igt_info("%-40s %16s %16s %16s %16s %16s %16s\n",
+		 "region", "capacity", "system-current",
+		 "cgroup-current", "min", "low", "max");
+	igt_info("%-40s %16s %16s %16s %16s %16s %16s\n",
+		 "------", "--------", "--------------",
+		 "--------------", "---", "---", "---");
+
+	for (i = 0; regions[i]; i++) {
+		region = regions[i];
+
+		igt_cgroup_dmem_get_capacity(region, &capacity);
+		fmt_bytes(capacity, cap_s, sizeof(cap_s));
+
+		igt_cgroup_dmem_get_system_current(region, &sys_current);
+		fmt_bytes(sys_current, sys_s, sizeof(sys_s));
+
+		igt_cgroup_dmem_get_current(cg, region, &cg_current);
+		fmt_bytes(cg_current, cg_s, sizeof(cg_s));
+
+		igt_cgroup_dmem_get_min(cg, region, &min);
+		fmt_bytes(min, min_s, sizeof(min_s));
+
+		igt_cgroup_dmem_get_low(cg, region, &low);
+		fmt_bytes(low, low_s, sizeof(low_s));
+
+		igt_cgroup_dmem_get_max(cg, region, &max);
+		fmt_bytes(max, max_s, sizeof(max_s));
+
+		igt_info("%-40s %16s %16s %16s %16s %16s %16s\n",
+			 region, cap_s, sys_s, cg_s,
+			 min_s, low_s, max_s);
+	}
+
+	igt_cgroup_dmem_regions_free(regions);
+}
tests/meson.build +1 -0
index 4f5c9be7b458..65b9a230f935 100644
@@ -1,4 +1,5 @@
 test_progs = [
+	'cgroup_dmem',
 	'core_auth',
 	'core_debugfs',
 	'core_getclient',
Signature · 1 line
2.47.3
Reviewed-by on v7 2/9
#4 lore raw

Re: [PATCH i-g-t v7 2/9] tests/cgroup_dmem: add dmem cgroup controller test

Quoted 124 lines · Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

Add a test that exercises the cgroup v2 dmem controller interface using the new igt_cgroup library.

The test uses igt_simple_main and:
- Skips if no dmem regions are registered (no cgroup v2 or no dmem-capable device).
- Creates a sub-cgroup and moves the test process into it.
- Enumerates all registered device memory regions and prints their capacity, system-wide current usage, per-cgroup current usage, and configured min, low and max limits.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
  tests/cgroup_dmem.c | 91 +++++++++++++++++++++++++++++++++++++++++++++++++++++
  tests/meson.build   |  1 +
  2 files changed, 92 insertions(+)
tests/cgroup_dmem.c new file +91 -0
new file mode 100644
index 000000000000..2ccd8bdb00ae
@@ -0,0 +1,91 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2025 Intel Corporation
+ */
+
+/**
+ * TEST: cgroup dmem
+ * Description: Exercises the cgroup v2 dmem controller interface.  Creates a
+ *              cgroup, moves the process into it, enumerates all dmem regions,
+ *              prints their capacity, system-wide current usage, per-cgroup
+ *              current usage and configured limits, then destroys the cgroup.
+ * Category: Core
+ * Mega feature: General Core features
+ * Sub-category: uapi
+ * Functionality: cgroup
+ * Feature: dmem
+ * Test category: uapi
+ */
+
+#include <inttypes.h>
+
+#include "igt.h"
+#include "igt_cgroup.h"
+
+IGT_TEST_DESCRIPTION("Exercises the cgroup v2 dmem controller interface.");
+
+static void fmt_bytes(uint64_t v, char *buf, size_t len)
+{
+	if (v == IGT_CGROUP_DMEM_MAX)
+		snprintf(buf, len, "max");
+	else
+		snprintf(buf, len, "%" PRIu64, v);
+}
+
+int igt_simple_main()
+{
+	struct igt_cgroup *cg;
+	const char *region;
+	char **regions;
+	uint64_t capacity, sys_current, cg_current, min, low, max;
+	char cap_s[32], sys_s[32], cg_s[32];
+	char min_s[32], low_s[32], max_s[32];
+	int i;
+
+	igt_require_f(igt_cgroup_dmem_available(),
+		      "No dmem regions found; is cgroup v2 with the "
+		      "dmem controller available?\n");
+
+	cg = igt_cgroup_new("igt-cgroup-dmem-test");
+	igt_assert_f(cg, "Failed to create cgroup\n");
+
+	igt_cgroup_move_current(cg);
+
+	regions = igt_cgroup_dmem_regions();
+	igt_assert_f(regions, "Failed to enumerate dmem regions\n");
+
+	igt_info("%-40s %16s %16s %16s %16s %16s %16s\n",
+		 "region", "capacity", "system-current",
+		 "cgroup-current", "min", "low", "max");
+	igt_info("%-40s %16s %16s %16s %16s %16s %16s\n",
+		 "------", "--------", "--------------",
+		 "--------------", "---", "---", "---");
+
+	for (i = 0; regions[i]; i++) {
+		region = regions[i];
+
+		igt_cgroup_dmem_get_capacity(region, &capacity);
+		fmt_bytes(capacity, cap_s, sizeof(cap_s));
+
+		igt_cgroup_dmem_get_system_current(region, &sys_current);
+		fmt_bytes(sys_current, sys_s, sizeof(sys_s));
+
+		igt_cgroup_dmem_get_current(cg, region, &cg_current);
+		fmt_bytes(cg_current, cg_s, sizeof(cg_s));
+
+		igt_cgroup_dmem_get_min(cg, region, &min);
+		fmt_bytes(min, min_s, sizeof(min_s));
+
+		igt_cgroup_dmem_get_low(cg, region, &low);
+		fmt_bytes(low, low_s, sizeof(low_s));
+
+		igt_cgroup_dmem_get_max(cg, region, &max);
+		fmt_bytes(max, max_s, sizeof(max_s));
+
+		igt_info("%-40s %16s %16s %16s %16s %16s %16s\n",
+			 region, cap_s, sys_s, cg_s,
+			 min_s, low_s, max_s);
+	}
+
+	igt_cgroup_dmem_regions_free(regions);
+}
tests/meson.build +0 -0
index 4f5c9be7b458..65b9a230f935 100644

tests/meson.build line 5

Quoted 5 lines
tests/meson.build +1 -0
@@ -1,4 +1,5 @@
  test_progs = [
+	'cgroup_dmem',
  	'core_auth',
  	'core_debugfs',
  	'core_getclient',

Regards,

Tvrtko

#5 lore raw

PATCH v7 3/9 lib/xe: add xe_cgroup_region_name() helper

In reply to Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

Add xe_cgroup_region_name(fd, region) which constructs the dmem cgroup region path for a given xe memory region. The returned string has the form "drm/<pci-slot>/<region>" (e.g. "drm/0000:03:00.0/vram0"), matching the name registered by the kernel via drmm_cgroup_register_region().

Only VRAM regions are tracked by the dmem controller; system and stolen memory regions return NULL.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
 lib/xe/xe_query.c | 32 ++++++++++++++++++++++++++++++++
 lib/xe/xe_query.h |  2 ++
 2 files changed, 34 insertions(+)
lib/xe/xe_query.c +32 -0
index 91ffc36e6425..058f95ed1484 100644
@@ -7,6 +7,7 @@
  */
 
 #include <fcntl.h>
+#include <limits.h>
 #include <stdlib.h>
 #include <pthread.h>
 
@@ -21,6 +22,7 @@
 
 #include "drmtest.h"
 #include "igt_debugfs.h"
+#include "igt_device.h"
 #include "ioctl_wrappers.h"
 #include "igt_map.h"
 #include "intel_common.h"
@@ -1302,6 +1304,36 @@ int xe_query_eu_thread_count(int fd, int gt)
 		xe_hwconfig_lookup_value_u32(fd, INTEL_HWCONFIG_NUM_THREADS_PER_EU);
 }
 
+/**
+ * xe_cgroup_region_name() - Build the dmem cgroup region name for an xe memory region.
+ * @fd: xe device fd.
+ * @region: Region mask (as used by xe_mem_region(), xe_region_name(), etc.).
+ *
+ * Constructs the full dmem cgroup region path for @region on the device
+ * identified by @fd.  The returned string has the form
+ * ``drm/<pci-slot>/<region>`` (e.g. ``drm/0000:03:00.0/vram0``), matching
+ * the name registered by the kernel driver via drmm_cgroup_register_region().
+ *
+ * Only VRAM regions are registered with the dmem controller; passing a
+ * system-memory region returns %NULL.
+ *
+ * Return: A newly allocated string that the caller must free(), or %NULL if
+ * @region is not tracked by the dmem cgroup controller.
+ */
+char *xe_cgroup_region_name(int fd, uint64_t region)
+{
+	char pci_slot[NAME_MAX];
+	char *name;
+
+	if (xe_region_class(fd, region) != DRM_XE_MEM_REGION_CLASS_VRAM)
+		return NULL;
+
+	igt_device_get_pci_slot_name(fd, pci_slot);
+
+	igt_assert(asprintf(&name, "drm/%s/%s", pci_slot, xe_region_name(region)) > 0);
+	return name;
+}
+
 igt_constructor
 {
 	xe_device_cache_init();
lib/xe/xe_query.h +2 -0
index 59330d80fd1b..a6ef9cab9aa1 100644
@@ -206,4 +206,6 @@ void xe_device_put(int fd);
 int xe_query_eu_count(int fd, int gt);
 int xe_query_eu_thread_count(int fd, int gt);
 
+char *xe_cgroup_region_name(int fd, uint64_t region);
+
 #endif	/* XE_QUERY_H */
Signature · 1 line
2.47.3
Reviewed-by on v7 3/9
#6 lore raw

Re: [PATCH i-g-t v7 3/9] lib/xe: add xe_cgroup_region_name() helper

Quoted 80 lines · Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

Add xe_cgroup_region_name(fd, region) which constructs the dmem cgroup region path for a given xe memory region. The returned string has the form "drm/<pci-slot>/<region>" (e.g. "drm/0000:03:00.0/vram0"), matching the name registered by the kernel via drmm_cgroup_register_region().

Only VRAM regions are tracked by the dmem controller; system and stolen memory regions return NULL.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
  lib/xe/xe_query.c | 32 ++++++++++++++++++++++++++++++++
  lib/xe/xe_query.h |  2 ++
  2 files changed, 34 insertions(+)
lib/xe/xe_query.c +32 -0
index 91ffc36e6425..058f95ed1484 100644
@@ -7,6 +7,7 @@
   */
 
  #include <fcntl.h>
+#include <limits.h>
  #include <stdlib.h>
  #include <pthread.h>
 
@@ -21,6 +22,7 @@
 
  #include "drmtest.h"
  #include "igt_debugfs.h"
+#include "igt_device.h"
  #include "ioctl_wrappers.h"
  #include "igt_map.h"
  #include "intel_common.h"
@@ -1302,6 +1304,36 @@ int xe_query_eu_thread_count(int fd, int gt)
  		xe_hwconfig_lookup_value_u32(fd, INTEL_HWCONFIG_NUM_THREADS_PER_EU);
  }
 
+/**
+ * xe_cgroup_region_name() - Build the dmem cgroup region name for an xe memory region.
+ * @fd: xe device fd.
+ * @region: Region mask (as used by xe_mem_region(), xe_region_name(), etc.).
+ *
+ * Constructs the full dmem cgroup region path for @region on the device
+ * identified by @fd.  The returned string has the form
+ * ``drm/<pci-slot>/<region>`` (e.g. ``drm/0000:03:00.0/vram0``), matching
+ * the name registered by the kernel driver via drmm_cgroup_register_region().
+ *
+ * Only VRAM regions are registered with the dmem controller; passing a
+ * system-memory region returns %NULL.
+ *
+ * Return: A newly allocated string that the caller must free(), or %NULL if
+ * @region is not tracked by the dmem cgroup controller.
+ */
+char *xe_cgroup_region_name(int fd, uint64_t region)
+{
+	char pci_slot[NAME_MAX];
+	char *name;
+
+	if (xe_region_class(fd, region) != DRM_XE_MEM_REGION_CLASS_VRAM)
+		return NULL;
+
+	igt_device_get_pci_slot_name(fd, pci_slot);
+
+	igt_assert(asprintf(&name, "drm/%s/%s", pci_slot, xe_region_name(region)) > 0);
+	return name;
+}
+
  igt_constructor
  {
  	xe_device_cache_init();
lib/xe/xe_query.h +0 -0
index 59330d80fd1b..a6ef9cab9aa1 100644

lib/xe/xe_query.h void xe_device_put(int fd); · line 211

Quoted 6 lines
lib/xe/xe_query.h +2 -0
@@ -206,4 +206,6 @@ void xe_device_put(int fd);
  int xe_query_eu_count(int fd, int gt);
  int xe_query_eu_thread_count(int fd, int gt);
 
+char *xe_cgroup_region_name(int fd, uint64_t region);
+
  #endif	/* XE_QUERY_H */

Regards,

Tvrtko

#7 lore raw

PATCH v7 4/9 lib/xe: Introduce dmem driver and implement Xe support

In reply to Thadeu Lima de Souza Cascardo

In order to be reuse the same dmem tests with multiple drivers, we need to abstract a few operations. That includes getting the region name, and allocating and releasing VRAM. As there is some initialization also when multiple allocations are done, also provide init and deinit functions.

The Xe implementation was based on the original operations from xe_cgroups.c written by Thomas Hellström. However, instead of doing a deferred backing, followed by a bind, it does a simple non-deferred GEM object creation on the VRAM region.

---
 lib/igt_dmem_driver.h | 35 ++++++++++++++++++
 lib/meson.build       |  1 +
 lib/xe/xe_dmem.c      | 98 +++++++++++++++++++++++++++++++++++++++++++++++++++
 3 files changed, 134 insertions(+)
lib/igt_dmem_driver.h new file +35 -0
new file mode 100644
index 000000000000..821ccb7a29a9
@@ -0,0 +1,35 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright 2026 Valve Corporation
+ * Authors:
+ *  Thadeu Lima de Souza Cascardo <[email protected]>
+ */
+
+#ifndef __IGT_DMEM_DRIVER_H__
+#define __IGT_DMEM_DRIVER_H__
+
+/**
+ * struct igt_dmem_driver - vendor driver to allocate and free device memory
+ *
+ */
+struct igt_dmem_driver {
+	/** @name: Driver name */
+	const char *name;
+	/** @init: Initialize an opaque context given a DRM device fd */
+	int (*init)(void **ctx, int fd);
+	/** @deinit: Release resources associated with context */
+	void (*deinit)(void *ctx);
+	/**
+	 * @get_region_name: Return expected region name at dmem cgroup files.
+	 * Must be freed by caller after use.
+	 */
+	char * (*get_region_name)(void *ctx);
+	/** @allocate_vram: Allocate @len sized vram and return an opaque @handle */
+	int (*allocate_vram)(void *ctx, size_t len, void **handle);
+	/** @free_vram: Free vram associated with @handle */
+	void (*free_vram)(void *ctx, void *handle);
+};
+
+extern const struct igt_dmem_driver xe_dmem_driver;
+
+#endif
lib/meson.build +1 -0
index df5f13efac56..c5d35bc8e88e 100644
@@ -132,6 +132,7 @@ lib_sources = [
 	'igt_hook.c',
 	'imagination/pvr_device_info.c',
 	'xe/xe_device.c',
+	'xe/xe_dmem.c',
 	'xe/xe_ggtt.c',
 	'xe/xe_gt.c',
 	'xe/xe_ioctl.c',
lib/xe/xe_dmem.c new file +98 -0

98 lines not on this page · raw message

Signature · 1 line
2.47.3
#8 lore raw

PATCH v7 5/9 lib/amdgpu: add amdgpu_cgroup_region_name

In reply to Thadeu Lima de Souza Cascardo

The amdgpu dmem region name uses its PCI address, just like the one from Xe, but there is only a single VRAM region.

---
 lib/amdgpu/amd_memory.c | 25 +++++++++++++++++++++++++
 lib/amdgpu/amd_memory.h |  2 ++
 2 files changed, 27 insertions(+)
lib/amdgpu/amd_memory.c +25 -0
index 12fe23c65ab2..2da4a8a4baee 100644
@@ -30,9 +30,11 @@
 #include <amdgpu_drm.h>
 #include <stdio.h>
 #include <string.h>
+#include <limits.h>
 #include <unistd.h>
 #include <sys/mman.h>
 #include <inttypes.h>
+#include "igt_device.h"
 
 /**
  *
@@ -679,6 +681,29 @@ bool virtual_free_memory(void *address, unsigned int size)
 	}
 }
 
+/**
+ * amdgpu_cgroup_region_name() - Build the dmem cgroup region name for an amdgpu.
+ * @fd: amdgpu device fd.
+ *
+ * Constructs the full dmem cgroup region path for VRAM on the device
+ * identified by @fd.  The returned string has the form
+ * ``drm/<pci-slot>/vram`` (e.g. ``drm/0000:03:00.0/vram``), matching
+ * the name registered by the kernel driver via drmm_cgroup_register_region().
+ *
+ * Return: A newly allocated string that the caller must free(), or %NULL if
+ * @region is not tracked by the dmem cgroup controller.
+ */
+char *amdgpu_cgroup_region_name(int fd)
+{
+	char pci_slot[NAME_MAX];
+	char *name;
+
+	igt_device_get_pci_slot_name(fd, pci_slot);
+
+	igt_assert(asprintf(&name, "drm/%s/vram", pci_slot) > 0);
+	return name;
+}
+
 /**
  * Wait for specific value in memory with timeout
  */
lib/amdgpu/amd_memory.h +2 -0
index e26c85bc4b0a..de169e580c1b 100644
@@ -105,6 +105,8 @@ void
 bool
 virtual_free_memory(void *address, unsigned int size);
 
+char *amdgpu_cgroup_region_name(int fd);
+
 bool
 wait_on_value(unsigned int *ptr, unsigned int expected);
 #endif
Signature · 1 line
2.47.3
#9 lore raw

PATCH v7 6/9 lib/amdgpu: add amdgpu support to igt_dmem_driver

In reply to Thadeu Lima de Souza Cascardo

This allows dmem cgroups tests to run on top of amdgpu driver, adding support to allocate and release VRAM memory.

This does this by allocating a BO from VRAM domain, which will try to place BOs on VRAM, but may fallback to GTT.

---
 lib/amdgpu/amd_dmem.c | 90 +++++++++++++++++++++++++++++++++++++++++++++++++++
 lib/igt_dmem_driver.h |  1 +
 lib/meson.build       |  1 +
 3 files changed, 92 insertions(+)
lib/amdgpu/amd_dmem.c new file +90 -0
new file mode 100644
index 000000000000..dd1e9083170b
@@ -0,0 +1,90 @@
+// SPDX-License-Identifier: MIT
+/*
+ * Copyright © 2026 Valve Corporation
+ * Authors:
+ *  Thadeu Lima de Souza Cascardo <[email protected]>
+ */
+
+#include <errno.h>
+
+#include "igt.h"
+#include "igt_cgroup.h"
+#include "igt_dmem_driver.h"
+#include "lib/amdgpu/amd_memory.h"
+
+struct amdgpu_dmem_ctx {
+	int fd;
+	amdgpu_device_handle device;
+};
+
+static int amdgpu_dmem_init(void **ctx, int fd)
+{
+	struct amdgpu_dmem_ctx *actx;
+	uint32_t major, minor;
+	int err;
+
+	actx = malloc(sizeof(*actx));
+	if (!actx)
+		return -ENOMEM;
+
+	err = amdgpu_device_initialize(fd, &major, &minor, &actx->device);
+	if (err)
+		goto out;
+
+	actx->fd = fd;
+
+	*ctx = actx;
+
+	return 0;
+
+out:
+	free(actx);
+
+	return err;
+}
+
+static void amdgpu_dmem_deinit(void *ctx)
+{
+	struct amdgpu_dmem_ctx *actx = ctx;
+
+	amdgpu_device_deinitialize(actx->device);
+	free(actx);
+}
+
+static char * amdgpu_dmem_get_region_name(void *ctx)
+{
+	struct amdgpu_dmem_ctx *actx = ctx;
+
+	return amdgpu_cgroup_region_name(actx->fd);
+}
+
+static int amdgpu_dmem_allocate_vram(void *ctx, size_t len, void **ret_handle)
+{
+	struct amdgpu_dmem_ctx *actx = ctx;
+	amdgpu_bo_handle handle;
+	int err;
+
+	err = amdgpu_bo_alloc_wrap(actx->device, len, 4096,
+				   AMDGPU_GEM_DOMAIN_VRAM, 0, &handle);
+	if (err)
+		return err;
+
+	if (ret_handle)
+		*ret_handle = (void *) handle;
+
+	return 0;
+}
+
+static void amdgpu_dmem_free_vram(void *ctx, void *handle)
+{
+	amdgpu_bo_free(handle);
+}
+
+const struct igt_dmem_driver amdgpu_dmem_driver = {
+	.name = "amdgpu",
+	.get_region_name = amdgpu_dmem_get_region_name,
+	.init = amdgpu_dmem_init,
+	.deinit = amdgpu_dmem_deinit,
+	.allocate_vram = amdgpu_dmem_allocate_vram,
+	.free_vram = amdgpu_dmem_free_vram,
+};
lib/igt_dmem_driver.h +1 -0
index 821ccb7a29a9..ad739c536435 100644
@@ -31,5 +31,6 @@ struct igt_dmem_driver {
 };
 
 extern const struct igt_dmem_driver xe_dmem_driver;
+extern const struct igt_dmem_driver amdgpu_dmem_driver;
 
 #endif
lib/meson.build +1 -0
index c5d35bc8e88e..b9075ff37631 100644
@@ -207,6 +207,7 @@ if libdrm_amdgpu.found()
 		'amdgpu/amd_mmd_shared.c',
 		'amdgpu/amd_jpeg_shared.c',
 		'amdgpu/amd_utils.c',
+		'amdgpu/amd_dmem.c',
 		'amdgpu/amd_vcn_shared.c'
 	]
 	if libdrm_amdgpu.version().version_compare('> 2.4.99')
Signature · 1 line
2.47.3
#10 lore raw

PATCH v7 7/9 tests/cgroup_dmem: add dmem cgroup eviction test

In reply to Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

The write_eviction subtest:
- Creates a sub-cgroup and moves the test process into it.
- Sets a dmem.max limit on the first VRAM region (up to 4 GiB, or the full capacity if smaller).
- Fills VRAM by repeatedly creating BOs placed in VRAM, depending on card support.
- Verifies that cgroup current usage is within the expected range when the limit is hit.
- Lowers dmem.max in 128 MiB steps, waiting for usage to follow each reduction.

The write_eviction_interruptible subtest runs the same test with SIGCONT signals injected via igt_fork_signal_helper() and reports the number of signals received. When a signal interrupts kernel-side eviction, a small BO allocation is used to re-trigger it.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
 tests/cgroup_dmem.c | 285 ++++++++++++++++++++++++++++++++++++++++++++++++++--
 1 file changed, 279 insertions(+), 6 deletions(-)
tests/cgroup_dmem.c +279 -6
index 2ccd8bdb00ae..5cdb2f8830c8 100644
@@ -1,6 +1,7 @@
 // SPDX-License-Identifier: MIT
 /*
- * Copyright © 2025 Intel Corporation
+ * Copyright © 2025,2026 Intel Corporation
+ * Copyright © 2026 Valve Corporation
  */
 
 /**
@@ -17,10 +18,217 @@
  * Test category: uapi
  */
 
+#include <errno.h>
 #include <inttypes.h>
+#include <signal.h>
+#include <stdatomic.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
 
+#include "drmtest.h"
 #include "igt.h"
+#include "igt_aux.h"
 #include "igt_cgroup.h"
+#include "igt_dmem_driver.h"
+
+#define BO_SIZE			SZ_64M
+#define MAX_LIMIT		((uint64_t)4 * SZ_1G)
+#define EVICT_STEP		SZ_128M
+
+#define TEST_INTERRUPTIBLE	(1 << 0)
+
+/**
+ * SUBTEST: simple
+ * DESCRIPTION:
+ *  Creates a cgroup, moves the process into it, enumerates all dmem regions,
+ *  prints their capacity, system-wide current usage, per-cgroup current usage
+ *  and configured limits, then destroys the cgroup.
+ */
+
+/**
+ * SUBTEST: write_eviction
+ * DESCRIPTION:
+ *   Create a dmem cgroup, move the current process into it and set the max
+ *   device memory limit for the first VRAM region to 4 GiB.  Then fill VRAM
+ *   by creating BOs with %DRM_XE_GEM_CREATE_FLAG_DEFER_BACKING (so that the
+ *   physical allocation is deferred until VM_BIND) and binding them into an
+ *   LR VM until the cgroup limit is hit.  Verify that the reported cgroup
+ *   current usage is within the expected range when the error occurs.
+ *   Finally lower the max limit in 256 MiB steps and verify that the cgroup
+ *   usage follows.
+ *   This subtest requires GPU device with at least one VRAM region.
+ */
+
+/**
+ * SUBTEST: write_eviction_interruptible
+ * DESCRIPTION:
+ *   Same as write_eviction but with SIGCONT signals injected throughout via
+ *   igt_fork_signal_helper() to verify that the dmem.max write path handles
+ *   signal interruption correctly.  A signal handler counts received signals
+ *   and the count is reported as debug output at the end of the test.
+ *   A signal interrupts the set-time eviction, and further eviction can be
+ *   triggered by an explicit allocation.
+ *   This subtest requires GPU device with at least one VRAM region.
+ */
+
+static atomic_int signal_count;
+static struct sigaction sigcont_oldact;
+
+static void sigcont_handler(int sig)
+{
+	atomic_fetch_add(&signal_count, 1);
+
+	/* Chain to the previous handler (IGT's dummy sig_handler) */
+	if (sigcont_oldact.sa_handler &&
+	    sigcont_oldact.sa_handler != SIG_IGN &&
+	    sigcont_oldact.sa_handler != SIG_DFL)
+		sigcont_oldact.sa_handler(sig);
+}
+
+static void install_sigcont_counter(void)
+{
+	struct sigaction sa;
+
+	atomic_store(&signal_count, 0);
+	igt_fork_signal_helper();
+	/*
+	 * Install the counter after igt_fork_signal_helper() so our handler
+	 * is not overwritten.  Save the old handler so we can chain to it.
+	 */
+	memset(&sa, 0, sizeof(sa));
+	sa.sa_handler = sigcont_handler;
+	sigemptyset(&sa.sa_mask);
+	sigaction(SIGCONT, &sa, &sigcont_oldact);
+}
+
+static int allocate_vram(void **handles, const struct igt_dmem_driver *drv,
+			 void *ctx, int max_bo, size_t len)
+{
+	int i, err = 0;
+
+	for (i = 0; i < max_bo; i++) {
+		err = drv->allocate_vram(ctx, len, &handles[i]);
+		if (err)
+			break;
+	}
+	/* These are expected failures we can ignore. */
+	if (err == -ENOMEM || err == -ENOSPC)
+		err = 0;
+	if (!err)
+		return i;
+	for (i--; i >= 0; i--)
+		drv->free_vram(ctx, handles[i]);
+
+	return err;
+}
+
+static void test_write_eviction(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx)
+{
+	struct igt_cgroup *cg;
+	void **handles;
+	int max_bo;
+	uint64_t current, capacity, cg_max, limit, after;
+	int err;
+
+	igt_cgroup_dmem_get_capacity(cg_region, &capacity);
+	igt_require_f(capacity >= 4 * BO_SIZE,
+		      "VRAM capacity (%"PRIu64" MiB) too small to test\n",
+		      capacity / SZ_1M);
+
+	/*
+	 * Use up to 4 GiB, or the full capacity if the device has less.
+	 * Leave one BO_SIZE worth of headroom so the device isn't completely
+	 * exhausted before the cgroup limit is hit.
+	 */
+	cg_max = min(MAX_LIMIT, capacity - BO_SIZE);
+	cg_max = ALIGN_DOWN(cg_max, EVICT_STEP);
+
+	if (flags & TEST_INTERRUPTIBLE)
+		install_sigcont_counter();
+
+	/* Create cgroup and move into it */
+	cg = igt_cgroup_new("igt_cgroups_test");
+	igt_cgroup_move_current(cg);
+	igt_cgroup_dmem_set_max(cg, cg_region, cg_max, false);
+
+	max_bo = (cg_max / BO_SIZE) + 8; /* headroom for overcommit */
+
+	handles = calloc(max_bo, sizeof(handles[0]));
+	igt_assert_f(handles, "failed to allocate handles array");
+
+	allocate_vram(handles, drv, ctx, max_bo, BO_SIZE);
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After fill: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+
+	igt_assert_f(current <= cg_max,
+		     "Usage %"PRIu64" MiB exceeds max %"PRIu64" MiB\n",
+		     current / SZ_1M, cg_max / SZ_1M);
+
+	/* Phase 2: lower max in 128 MiB steps, verify usage follows */
+	limit = cg_max;
+	while (limit >= EVICT_STEP) {
+
+		limit -= EVICT_STEP;
+		igt_cgroup_dmem_set_max(cg, cg_region, limit, false);
+
+		igt_cgroup_dmem_get_current(cg, cg_region, &after);
+		igt_debug("Lowered max to %"PRIu64" MiB: usage = %"PRIu64" MiB\n",
+			  limit / SZ_1M, after / SZ_1M);
+
+		if (limit > EVICT_STEP) {
+			if ((flags & TEST_INTERRUPTIBLE) && after > limit) {
+				/* Let a new bo creation trigger eviction. */
+				void *handle;
+				err = drv->allocate_vram(ctx, BO_SIZE / 8, &handle);
+				igt_assert_f(err == 0,
+					"Error trying to allocate more VRAM to trigger eviction.");
+				drv->free_vram(ctx, handle);
+
+				igt_cgroup_dmem_get_current(cg, cg_region, &after);
+				igt_debug("Forced eviction max is %"PRIu64
+					  " MiB: usage = %"PRIu64" MiB\n",
+					  limit / SZ_1M, after / SZ_1M);
+			}
+
+			igt_assert_f(after <= limit,
+				     "Usage %"PRIu64" MiB did not follow max %"PRIu64" MiB\n",
+				     after / SZ_1M, limit / SZ_1M);
+		}
+	}
+
+	if (flags & TEST_INTERRUPTIBLE) {
+		igt_stop_signal_helper();
+		igt_info("Signals received during test: %d\n",
+			 atomic_load(&signal_count));
+	}
+
+	/* Cleanup */
+	igt_cgroup_dmem_set_max(cg, cg_region, IGT_CGROUP_DMEM_MAX, false);
+}
+
+static const struct {
+	const char *name;
+	void (*test_fn)(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx);
+	unsigned int flags;
+} subtests[] = {
+	{ "write_eviction", test_write_eviction, 0 },
+	{ "write_eviction_interruptible", test_write_eviction, TEST_INTERRUPTIBLE },
+	{ }
+};
+
+static const struct {
+	int driver_flag;
+	const struct igt_dmem_driver *driver;
+} drivers[] = {
+	{ DRIVER_XE, &xe_dmem_driver },
+	{ DRIVER_AMDGPU, &amdgpu_dmem_driver },
+	{ },
+};
 
 IGT_TEST_DESCRIPTION("Exercises the cgroup v2 dmem controller interface.");
 
@@ -32,7 +240,7 @@ static void fmt_bytes(uint64_t v, char *buf, size_t len)
 		snprintf(buf, len, "%" PRIu64, v);
 }
 
-int igt_simple_main()
+static void simple_cgroup(void)
 {
 	struct igt_cgroup *cg;
 	const char *region;
@@ -42,10 +250,6 @@ int igt_simple_main()
 	char min_s[32], low_s[32], max_s[32];
 	int i;
 
-	igt_require_f(igt_cgroup_dmem_available(),
-		      "No dmem regions found; is cgroup v2 with the "
-		      "dmem controller available?\n");
-
 	cg = igt_cgroup_new("igt-cgroup-dmem-test");
 	igt_assert_f(cg, "Failed to create cgroup\n");
 
@@ -89,3 +293,72 @@ int igt_simple_main()
 
 	igt_cgroup_dmem_regions_free(regions);
 }
+
+int igt_main()
+{
+	igt_fixture() {
+		igt_require_f(getuid() == 0, "Test requires root\n");
+		/* Check dmem cgroup controller is available before doing anything else */
+		igt_require_f(igt_cgroup_dmem_available(),
+			      "dmem cgroup controller not available (no cgroup v2 or no registered regions)\n");
+
+	}
+
+	igt_subtest("simple")
+		simple_cgroup();
+
+	for (int d = 0; drivers[d].driver; d++) {
+
+		int fd = -1;
+
+		for (int i = 0; subtests[i].name; i++) {
+			igt_subtest_group() {
+
+				int ret = -1;
+				char *cg_region = NULL;
+				void *ctx = NULL;
+
+				igt_fixture() {
+					igt_cgroup_exit_current();
+					if (fd >= 0) {
+						int new_fd;
+						new_fd = drm_reopen_driver(fd);
+						drm_close_driver(fd);
+						fd = new_fd;
+					} else {
+						fd = drm_open_driver(drivers[d].driver_flag);
+					}
+					igt_require_f(fd >= 0,
+						"No %s device found, skipping\n",
+						drivers[d].driver->name);
+					ret = drivers[d].driver->init(&ctx, fd);
+					igt_require_f(ret == 0,
+						"Failed to initialize %s device, skipping\n",
+						drivers[d].driver->name);
+					cg_region = drivers[d].driver->get_region_name(ctx);
+					igt_require_f(cg_region, "Region not tracked by dmem cgroup controller\n");
+				}
+
+				igt_subtest_f("%s-%s", drivers[d].driver->name, subtests[i].name)
+					subtests[i].test_fn(fd, cg_region, subtests[i].flags, drivers[d].driver, ctx);
+
+				igt_fixture() {
+
+					if (subtests[i].flags & TEST_INTERRUPTIBLE)
+						igt_stop_signal_helper();
+
+					if (!ret)
+						drivers[d].driver->deinit(ctx);
+					free(cg_region);
+				}
+
+			}
+		}
+
+		igt_fixture() {
+			if (fd >= 0)
+				drm_close_driver(fd);
+		}
+
+	}
+}
Signature · 1 line
2.47.3
#11 lore raw

Re: [PATCH i-g-t v7 7/9] tests/cgroup_dmem: add dmem cgroup eviction test

Quoted 161 lines · Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

The write_eviction subtest:
- Creates a sub-cgroup and moves the test process into it.
- Sets a dmem.max limit on the first VRAM region (up to 4 GiB, or the full capacity if smaller).
- Fills VRAM by repeatedly creating BOs placed in VRAM, depending on card support.
- Verifies that cgroup current usage is within the expected range when the limit is hit.
- Lowers dmem.max in 128 MiB steps, waiting for usage to follow each reduction.

The write_eviction_interruptible subtest runs the same test with SIGCONT signals injected via igt_fork_signal_helper() and reports the number of signals received. When a signal interrupts kernel-side eviction, a small BO allocation is used to re-trigger it.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
  tests/cgroup_dmem.c | 285 ++++++++++++++++++++++++++++++++++++++++++++++++++--
  1 file changed, 279 insertions(+), 6 deletions(-)
tests/cgroup_dmem.c +117 -1
index 2ccd8bdb00ae..5cdb2f8830c8 100644
@@ -1,6 +1,7 @@
  // SPDX-License-Identifier: MIT
  /*
- * Copyright © 2025 Intel Corporation
+ * Copyright © 2025,2026 Intel Corporation
+ * Copyright © 2026 Valve Corporation
   */
 
  /**
@@ -17,10 +18,217 @@
   * Test category: uapi
   */
 
+#include <errno.h>
  #include <inttypes.h>
+#include <signal.h>
+#include <stdatomic.h>
+#include <stdint.h>
+#include <stdlib.h>
+#include <string.h>
+#include <unistd.h>
 
+#include "drmtest.h"
  #include "igt.h"
+#include "igt_aux.h"
  #include "igt_cgroup.h"
+#include "igt_dmem_driver.h"
+
+#define BO_SIZE			SZ_64M
+#define MAX_LIMIT		((uint64_t)4 * SZ_1G)
+#define EVICT_STEP		SZ_128M
+
+#define TEST_INTERRUPTIBLE	(1 << 0)
+
+/**
+ * SUBTEST: simple
+ * DESCRIPTION:
+ *  Creates a cgroup, moves the process into it, enumerates all dmem regions,
+ *  prints their capacity, system-wide current usage, per-cgroup current usage
+ *  and configured limits, then destroys the cgroup.
+ */
+
+/**
+ * SUBTEST: write_eviction
+ * DESCRIPTION:
+ *   Create a dmem cgroup, move the current process into it and set the max
+ *   device memory limit for the first VRAM region to 4 GiB.  Then fill VRAM
+ *   by creating BOs with %DRM_XE_GEM_CREATE_FLAG_DEFER_BACKING (so that the
+ *   physical allocation is deferred until VM_BIND) and binding them into an
+ *   LR VM until the cgroup limit is hit.  Verify that the reported cgroup
+ *   current usage is within the expected range when the error occurs.
+ *   Finally lower the max limit in 256 MiB steps and verify that the cgroup
+ *   usage follows.
+ *   This subtest requires GPU device with at least one VRAM region.
+ */
+
+/**
+ * SUBTEST: write_eviction_interruptible
+ * DESCRIPTION:
+ *   Same as write_eviction but with SIGCONT signals injected throughout via
+ *   igt_fork_signal_helper() to verify that the dmem.max write path handles
+ *   signal interruption correctly.  A signal handler counts received signals
+ *   and the count is reported as debug output at the end of the test.
+ *   A signal interrupts the set-time eviction, and further eviction can be
+ *   triggered by an explicit allocation.
+ *   This subtest requires GPU device with at least one VRAM region.
+ */
+
+static atomic_int signal_count;
+static struct sigaction sigcont_oldact;
+
+static void sigcont_handler(int sig)
+{
+	atomic_fetch_add(&signal_count, 1);
+
+	/* Chain to the previous handler (IGT's dummy sig_handler) */
+	if (sigcont_oldact.sa_handler &&
+	    sigcont_oldact.sa_handler != SIG_IGN &&
+	    sigcont_oldact.sa_handler != SIG_DFL)
+		sigcont_oldact.sa_handler(sig);
+}
+
+static void install_sigcont_counter(void)
+{
+	struct sigaction sa;
+
+	atomic_store(&signal_count, 0);
+	igt_fork_signal_helper();
+	/*
+	 * Install the counter after igt_fork_signal_helper() so our handler
+	 * is not overwritten.  Save the old handler so we can chain to it.
+	 */
+	memset(&sa, 0, sizeof(sa));
+	sa.sa_handler = sigcont_handler;
+	sigemptyset(&sa.sa_mask);
+	sigaction(SIGCONT, &sa, &sigcont_oldact);
+}
+
+static int allocate_vram(void **handles, const struct igt_dmem_driver *drv,
+			 void *ctx, int max_bo, size_t len)
+{
+	int i, err = 0;
+
+	for (i = 0; i < max_bo; i++) {
+		err = drv->allocate_vram(ctx, len, &handles[i]);
+		if (err)
+			break;
+	}
+	/* These are expected failures we can ignore. */
+	if (err == -ENOMEM || err == -ENOSPC)
+		err = 0;
+	if (!err)
+		return i;
+	for (i--; i >= 0; i--)
+		drv->free_vram(ctx, handles[i]);
+
+	return err;
+}
+
+static void test_write_eviction(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx)
+{
+	struct igt_cgroup *cg;
+	void **handles;
+	int max_bo;
+	uint64_t current, capacity, cg_max, limit, after;
+	int err;
+
+	igt_cgroup_dmem_get_capacity(cg_region, &capacity);
+	igt_require_f(capacity >= 4 * BO_SIZE,
+		      "VRAM capacity (%"PRIu64" MiB) too small to test\n",
+		      capacity / SZ_1M);
+

tests/cgroup_dmem.c line 149

Quoted 10 lines
tests/cgroup_dmem.c +10 -0
+	/*
+	 * Use up to 4 GiB, or the full capacity if the device has less.
+	 * Leave one BO_SIZE worth of headroom so the device isn't completely
+	 * exhausted before the cgroup limit is hit.
+	 */
+	cg_max = min(MAX_LIMIT, capacity - BO_SIZE);
+	cg_max = ALIGN_DOWN(cg_max, EVICT_STEP);
+
+	if (flags & TEST_INTERRUPTIBLE)
+		install_sigcont_counter();

Is it okay to start the signal spam so early or it should be before the loop below? I guess if it works it works.. more stress testing the better.

Quoted 106 lines
tests/cgroup_dmem.c +83 -5
+
+	/* Create cgroup and move into it */
+	cg = igt_cgroup_new("igt_cgroups_test");
+	igt_cgroup_move_current(cg);
+	igt_cgroup_dmem_set_max(cg, cg_region, cg_max, false);
+
+	max_bo = (cg_max / BO_SIZE) + 8; /* headroom for overcommit */
+
+	handles = calloc(max_bo, sizeof(handles[0]));
+	igt_assert_f(handles, "failed to allocate handles array");
+
+	allocate_vram(handles, drv, ctx, max_bo, BO_SIZE);
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After fill: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+
+	igt_assert_f(current <= cg_max,
+		     "Usage %"PRIu64" MiB exceeds max %"PRIu64" MiB\n",
+		     current / SZ_1M, cg_max / SZ_1M);
+
+	/* Phase 2: lower max in 128 MiB steps, verify usage follows */
+	limit = cg_max;
+	while (limit >= EVICT_STEP) {
+
+		limit -= EVICT_STEP;
+		igt_cgroup_dmem_set_max(cg, cg_region, limit, false);
+
+		igt_cgroup_dmem_get_current(cg, cg_region, &after);
+		igt_debug("Lowered max to %"PRIu64" MiB: usage = %"PRIu64" MiB\n",
+			  limit / SZ_1M, after / SZ_1M);
+
+		if (limit > EVICT_STEP) {
+			if ((flags & TEST_INTERRUPTIBLE) && after > limit) {
+				/* Let a new bo creation trigger eviction. */
+				void *handle;
+				err = drv->allocate_vram(ctx, BO_SIZE / 8, &handle);
+				igt_assert_f(err == 0,
+					"Error trying to allocate more VRAM to trigger eviction.");
+				drv->free_vram(ctx, handle);
+
+				igt_cgroup_dmem_get_current(cg, cg_region, &after);
+				igt_debug("Forced eviction max is %"PRIu64
+					  " MiB: usage = %"PRIu64" MiB\n",
+					  limit / SZ_1M, after / SZ_1M);
+			}
+
+			igt_assert_f(after <= limit,
+				     "Usage %"PRIu64" MiB did not follow max %"PRIu64" MiB\n",
+				     after / SZ_1M, limit / SZ_1M);
+		}
+	}
+
+	if (flags & TEST_INTERRUPTIBLE) {
+		igt_stop_signal_helper();
+		igt_info("Signals received during test: %d\n",
+			 atomic_load(&signal_count));
+	}
+
+	/* Cleanup */
+	igt_cgroup_dmem_set_max(cg, cg_region, IGT_CGROUP_DMEM_MAX, false);
+}
+
+static const struct {
+	const char *name;
+	void (*test_fn)(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx);
+	unsigned int flags;
+} subtests[] = {
+	{ "write_eviction", test_write_eviction, 0 },
+	{ "write_eviction_interruptible", test_write_eviction, TEST_INTERRUPTIBLE },
+	{ }
+};
+
+static const struct {
+	int driver_flag;
+	const struct igt_dmem_driver *driver;
+} drivers[] = {
+	{ DRIVER_XE, &xe_dmem_driver },
+	{ DRIVER_AMDGPU, &amdgpu_dmem_driver },
+	{ },
+};
 
  IGT_TEST_DESCRIPTION("Exercises the cgroup v2 dmem controller interface.");
 
@@ -32,7 +240,7 @@ static void fmt_bytes(uint64_t v, char *buf, size_t len)
  		snprintf(buf, len, "%" PRIu64, v);
  }
 
-int igt_simple_main()
+static void simple_cgroup(void)
  {
  	struct igt_cgroup *cg;
  	const char *region;
@@ -42,10 +250,6 @@ int igt_simple_main()
  	char min_s[32], low_s[32], max_s[32];
  	int i;
 
-	igt_require_f(igt_cgroup_dmem_available(),
-		      "No dmem regions found; is cgroup v2 with the "
-		      "dmem controller available?\n");
-
  	cg = igt_cgroup_new("igt-cgroup-dmem-test");
  	igt_assert_f(cg, "Failed to create cgroup\n");
 

tests/cgroup_dmem.c int igt_simple_main() · line 300

Quoted 8 lines
tests/cgroup_dmem.c +5 -0
@@ -89,3 +293,72 @@ int igt_simple_main()
 
  	igt_cgroup_dmem_regions_free(regions);
  }
+
+int igt_main()
+{
+	igt_fixture() {
+		igt_require_f(getuid() == 0, "Test requires root\n");

It doesn't matter for IGT but geteuid would be more correct.

tests/cgroup_dmem.c int igt_simple_main() · line 308

Quoted 8 lines
tests/cgroup_dmem.c +8 -0
+		/* Check dmem cgroup controller is available before doing anything else */
+		igt_require_f(igt_cgroup_dmem_available(),
+			      "dmem cgroup controller not available (no cgroup v2 or no registered regions)\n");
+
+	}
+
+	igt_subtest("simple")
+		simple_cgroup();

FWIW "basic" would be more in line with how we used to do it in the past. IIRC.

tests/cgroup_dmem.c int igt_simple_main() · line 311

Quoted 3 lines
tests/cgroup_dmem.c +3 -0
+
+	for (int d = 0; drivers[d].driver; d++) {
+

Spurious blank line.

tests/cgroup_dmem.c int igt_simple_main() · line 316

Quoted 5 lines
tests/cgroup_dmem.c +5 -0
+		int fd = -1;
+
+		for (int i = 0; subtests[i].name; i++) {
+			igt_subtest_group() {
+

Hm perhaps it was deliberate? I don't see a pattern here and in some cases below.

tests/cgroup_dmem.c int igt_simple_main() · line 324

Quoted 8 lines
tests/cgroup_dmem.c +8 -0
+				int ret = -1;
+				char *cg_region = NULL;
+				void *ctx = NULL;
+
+				igt_fixture() {
+					igt_cgroup_exit_current();
+					if (fd >= 0) {
+						int new_fd;

Blank line.

tests/cgroup_dmem.c int igt_simple_main() · line 326

Quoted 2 lines
tests/cgroup_dmem.c +2 -0
+						new_fd = drm_reopen_driver(fd);
+						drm_close_driver(fd);

This I guess frees the bos belonging to the previous subtest?

tests/cgroup_dmem.c int igt_simple_main() · line 329

Quoted 3 lines
tests/cgroup_dmem.c +3 -0
+						fd = new_fd;
+					} else {
+						fd = drm_open_driver(drivers[d].driver_flag);

Strictly speaking it could use render node right? (Moot because it needs root to configure dmem controller.)

tests/cgroup_dmem.c int igt_simple_main() · line 339

Quoted 10 lines
tests/cgroup_dmem.c +10 -0
+					}
+					igt_require_f(fd >= 0,
+						"No %s device found, skipping\n",
+						drivers[d].driver->name);
+					ret = drivers[d].driver->init(&ctx, fd);
+					igt_require_f(ret == 0,
+						"Failed to initialize %s device, skipping\n",
+						drivers[d].driver->name);
+					cg_region = drivers[d].driver->get_region_name(ctx);
+					igt_require_f(cg_region, "Region not tracked by dmem cgroup controller\n");

"No regions tracked.." ?

tests/cgroup_dmem.c int igt_simple_main() · line 343

Quoted 4 lines
tests/cgroup_dmem.c +4 -0
+				}
+
+				igt_subtest_f("%s-%s", drivers[d].driver->name, subtests[i].name)
+					subtests[i].test_fn(fd, cg_region, subtests[i].flags, drivers[d].driver, ctx);

My IGT-fu is somewhat dimishied these days, but isn't one bad thing here that the list of subtests is not stable (as in cgroup_dmem --l)? Ie. it depends on the GPUs installed. In the past that was annoying for CI systems so we added igt_dynamic to deal with disappearing subtests. Maybe look into that, or perhaps nowadays people do not care.

Regards,

Tvrtko

Quoted 22 lines
tests/cgroup_dmem.c +21 -0
+
+				igt_fixture() {
+
+					if (subtests[i].flags & TEST_INTERRUPTIBLE)
+						igt_stop_signal_helper();
+
+					if (!ret)
+						drivers[d].driver->deinit(ctx);
+					free(cg_region);
+				}
+
+			}
+		}
+
+		igt_fixture() {
+			if (fd >= 0)
+				drm_close_driver(fd);
+		}
+
+	}
+}
#12 lore raw

PATCH v7 8/9 tests/cgroup_dmem: add write_eviction_nonblock subtest

In reply to Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

Add write_eviction_nonblock to exercise the O_NONBLOCK path of the dmem cgroup max interface. After filling VRAM to the cgroup limit, each limit-lowering step writes dmem.max with O_NONBLOCK so that synchronous eviction is skipped. The test then verifies that usage has not yet dropped below the new limit, allocates a small BO to trigger eviction explicitly, and finally confirms that usage falls within bounds.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
 tests/cgroup_dmem.c | 37 ++++++++++++++++++++++++++++++++++---
 1 file changed, 34 insertions(+), 3 deletions(-)
tests/cgroup_dmem.c +34 -3
index 5cdb2f8830c8..7f3cfe22db2c 100644
@@ -38,6 +38,7 @@
 #define EVICT_STEP		SZ_128M
 
 #define TEST_INTERRUPTIBLE	(1 << 0)
+#define TEST_NONBLOCK		(1 << 1)
 
 /**
  * SUBTEST: simple
@@ -73,6 +74,18 @@
  *   This subtest requires GPU device with at least one VRAM region.
  */
 
+/**
+ * SUBTEST: write_eviction_nonblock
+ * DESCRIPTION:
+ *   Same fill phase as write_eviction.  In the limit-lowering phase dmem.max
+ *   is written with O_NONBLOCK, which causes the kernel to skip synchronous
+ *   eviction.  After each nonblock write the test verifies that usage has not
+ *   yet dropped below the new limit, then triggers eviction explicitly by
+ *   allocating a small BO.  Finally verifies that usage falls within bounds
+ *   after the forced eviction.
+ *   This subtest requires GPU device with at least one VRAM region.
+ */
+
 static atomic_int signal_count;
 static struct sigaction sigcont_oldact;
 
@@ -129,7 +142,7 @@ static void test_write_eviction(int fd, char *cg_region, unsigned int flags, con
 	struct igt_cgroup *cg;
 	void **handles;
 	int max_bo;
-	uint64_t current, capacity, cg_max, limit, after;
+	uint64_t current, capacity, cg_max, limit, after, before;
 	int err;
 
 	igt_cgroup_dmem_get_capacity(cg_region, &capacity);
@@ -174,14 +187,31 @@ static void test_write_eviction(int fd, char *cg_region, unsigned int flags, con
 	while (limit >= EVICT_STEP) {
 
 		limit -= EVICT_STEP;
-		igt_cgroup_dmem_set_max(cg, cg_region, limit, false);
+
+		if (flags & TEST_NONBLOCK)
+			igt_cgroup_dmem_get_current(cg, cg_region, &before);
+
+		igt_cgroup_dmem_set_max(cg, cg_region, limit,
+					!!(flags & TEST_NONBLOCK));
 
 		igt_cgroup_dmem_get_current(cg, cg_region, &after);
 		igt_debug("Lowered max to %"PRIu64" MiB: usage = %"PRIu64" MiB\n",
 			  limit / SZ_1M, after / SZ_1M);
 
+		if (flags & TEST_NONBLOCK) {
+			/*
+			 * O_NONBLOCK skips eviction: verify usage has not
+			 * dropped below the new limit yet.
+			 */
+			igt_assert_f(after == before,
+				     "Expected no eviction with O_NONBLOCK, but "
+				     "usage dropped from %"PRIu64" MiB to %"PRIu64" MiB "
+				     "(limit %"PRIu64" MiB)\n",
+				     before / SZ_1M, after / SZ_1M, limit / SZ_1M);
+		}
+
 		if (limit > EVICT_STEP) {
-			if ((flags & TEST_INTERRUPTIBLE) && after > limit) {
+			if ((flags & (TEST_INTERRUPTIBLE | TEST_NONBLOCK)) && after > limit) {
 				/* Let a new bo creation trigger eviction. */
 				void *handle;
 				err = drv->allocate_vram(ctx, BO_SIZE / 8, &handle);
@@ -218,6 +248,7 @@ static const struct {
 } subtests[] = {
 	{ "write_eviction", test_write_eviction, 0 },
 	{ "write_eviction_interruptible", test_write_eviction, TEST_INTERRUPTIBLE },
+	{ "write_eviction_nonblock", test_write_eviction, TEST_NONBLOCK },
 	{ }
 };
 
Signature · 1 line
2.47.3
Reviewed-by on v7 8/9
#13 lore raw

Re: [PATCH i-g-t v7 8/9] tests/cgroup_dmem: add write_eviction_nonblock subtest

Quoted 92 lines · Thadeu Lima de Souza Cascardo

From: Thomas Hellström <[email protected]>

Add write_eviction_nonblock to exercise the O_NONBLOCK path of the dmem cgroup max interface. After filling VRAM to the cgroup limit, each limit-lowering step writes dmem.max with O_NONBLOCK so that synchronous eviction is skipped. The test then verifies that usage has not yet dropped below the new limit, allocates a small BO to trigger eviction explicitly, and finally confirms that usage falls within bounds.

Assisted-by GitHub Copilot:claude-sonnet-4.6
---
  tests/cgroup_dmem.c | 37 ++++++++++++++++++++++++++++++++++---
  1 file changed, 34 insertions(+), 3 deletions(-)
tests/cgroup_dmem.c +33 -3
index 5cdb2f8830c8..7f3cfe22db2c 100644
@@ -38,6 +38,7 @@
  #define EVICT_STEP		SZ_128M
 
  #define TEST_INTERRUPTIBLE	(1 << 0)
+#define TEST_NONBLOCK		(1 << 1)
 
  /**
   * SUBTEST: simple
@@ -73,6 +74,18 @@
   *   This subtest requires GPU device with at least one VRAM region.
   */
 
+/**
+ * SUBTEST: write_eviction_nonblock
+ * DESCRIPTION:
+ *   Same fill phase as write_eviction.  In the limit-lowering phase dmem.max
+ *   is written with O_NONBLOCK, which causes the kernel to skip synchronous
+ *   eviction.  After each nonblock write the test verifies that usage has not
+ *   yet dropped below the new limit, then triggers eviction explicitly by
+ *   allocating a small BO.  Finally verifies that usage falls within bounds
+ *   after the forced eviction.
+ *   This subtest requires GPU device with at least one VRAM region.
+ */
+
  static atomic_int signal_count;
  static struct sigaction sigcont_oldact;
 
@@ -129,7 +142,7 @@ static void test_write_eviction(int fd, char *cg_region, unsigned int flags, con
  	struct igt_cgroup *cg;
  	void **handles;
  	int max_bo;
-	uint64_t current, capacity, cg_max, limit, after;
+	uint64_t current, capacity, cg_max, limit, after, before;
  	int err;
 
  	igt_cgroup_dmem_get_capacity(cg_region, &capacity);
@@ -174,14 +187,31 @@ static void test_write_eviction(int fd, char *cg_region, unsigned int flags, con
  	while (limit >= EVICT_STEP) {
 
  		limit -= EVICT_STEP;
-		igt_cgroup_dmem_set_max(cg, cg_region, limit, false);
+
+		if (flags & TEST_NONBLOCK)
+			igt_cgroup_dmem_get_current(cg, cg_region, &before);
+
+		igt_cgroup_dmem_set_max(cg, cg_region, limit,
+					!!(flags & TEST_NONBLOCK));
 
  		igt_cgroup_dmem_get_current(cg, cg_region, &after);
  		igt_debug("Lowered max to %"PRIu64" MiB: usage = %"PRIu64" MiB\n",
  			  limit / SZ_1M, after / SZ_1M);
 
+		if (flags & TEST_NONBLOCK) {
+			/*
+			 * O_NONBLOCK skips eviction: verify usage has not
+			 * dropped below the new limit yet.
+			 */
+			igt_assert_f(after == before,
+				     "Expected no eviction with O_NONBLOCK, but "
+				     "usage dropped from %"PRIu64" MiB to %"PRIu64" MiB "
+				     "(limit %"PRIu64" MiB)\n",
+				     before / SZ_1M, after / SZ_1M, limit / SZ_1M);
+		}
+
  		if (limit > EVICT_STEP) {
-			if ((flags & TEST_INTERRUPTIBLE) && after > limit) {
+			if ((flags & (TEST_INTERRUPTIBLE | TEST_NONBLOCK)) && after > limit) {
  				/* Let a new bo creation trigger eviction. */
  				void *handle;
  				err = drv->allocate_vram(ctx, BO_SIZE / 8, &handle);

tests/cgroup_dmem.c static const struct { · line 254

Quoted 7 lines
tests/cgroup_dmem.c +1 -0
@@ -218,6 +248,7 @@ static const struct {
  } subtests[] = {
  	{ "write_eviction", test_write_eviction, 0 },
  	{ "write_eviction_interruptible", test_write_eviction, TEST_INTERRUPTIBLE },
+	{ "write_eviction_nonblock", test_write_eviction, TEST_NONBLOCK },
  	{ }
  };
 

Regards,

Tvrtko

#14 lore raw

PATCH v7 9/9 tests/cgroup_dmem: add test for dmem.current

In reply to Thadeu Lima de Souza Cascardo

Based on the work of Thomas Hellström to test dmem.max eviction, add a test for dmem.current usage after allocations and setting dmem.max.

Create a dmem cgroup, allocate close to capacity (or at most 4GiB), check current usage is within a small slack of the expected allocation. Then, set max to a small value and check allocations and current usage are limited to the max set. Set max to less than a single BO size, then check no allocations are allowed and current usage is also within the slack. After each allocation, release memory and check current usage has gone down.

---
 tests/cgroup_dmem.c | 147 ++++++++++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 147 insertions(+)
tests/cgroup_dmem.c +147 -0
index 7f3cfe22db2c..1f01037e1cf2 100644
@@ -36,6 +36,8 @@
 #define BO_SIZE			SZ_64M
 #define MAX_LIMIT		((uint64_t)4 * SZ_1G)
 #define EVICT_STEP		SZ_128M
+#define USAGE_POLL_MS		10
+#define USAGE_DROP_TIMEOUT_MS	1000
 
 #define TEST_INTERRUPTIBLE	(1 << 0)
 #define TEST_NONBLOCK		(1 << 1)
@@ -48,6 +50,20 @@
  *  and configured limits, then destroys the cgroup.
  */
 
+/**
+ * SUBTEST: current
+ * DESCRIPTION:
+ *  Create a dmem cgroup, allocate close to capacity (or at most 4GiB),
+ *  check current usage is within a small slack of the expected allocation.
+ *  Then, set max to a small value and check allocations and current usage
+ *  are limited to the max set.
+ *  Set max to less than a single BO size, then check no allocations are allowed
+ *  and current usage is also within the slack.
+ *  After each allocation, release memory and check current usage has gone
+ *  down.
+ *  This subtest requires GPU device with at least one VRAM region.
+ */
+
 /**
  * SUBTEST: write_eviction
  * DESCRIPTION:
@@ -116,6 +132,23 @@ static void install_sigcont_counter(void)
 	sigaction(SIGCONT, &sa, &sigcont_oldact);
 }
 
+static uint64_t wait_for_usage_drop(struct igt_cgroup *cg, const char *region,
+				    uint64_t limit)
+{
+	uint64_t current;
+	unsigned int elapsed = 0;
+
+	do {
+		igt_cgroup_dmem_get_current(cg, region, &current);
+		if (current <= limit)
+			return current;
+		usleep(USAGE_POLL_MS * 1000);
+		elapsed += USAGE_POLL_MS;
+	} while (elapsed < USAGE_DROP_TIMEOUT_MS);
+
+	return current;
+}
+
 static int allocate_vram(void **handles, const struct igt_dmem_driver *drv,
 			 void *ctx, int max_bo, size_t len)
 {
@@ -137,6 +170,119 @@ static int allocate_vram(void **handles, const struct igt_dmem_driver *drv,
 	return err;
 }
 
+static void free_vram(void **handles, const struct igt_dmem_driver *drv,
+		      void *ctx, int max_bo)
+{
+	int i;
+	for (i = 0; i < max_bo; i++)
+		drv->free_vram(ctx, handles[i]);
+}
+
+static void test_current(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx)
+{
+	struct igt_cgroup *cg;
+	void **handles;
+	uint64_t current, capacity, cg_max;
+	int n_bo = 0, max_bo;
+
+	igt_cgroup_dmem_get_capacity(cg_region, &capacity);
+	igt_require_f(capacity >= 4 * BO_SIZE,
+		      "VRAM capacity (%"PRIu64" MiB) too small to test\n",
+		      capacity / SZ_1M);
+
+	/*
+	 * Use up to 4 GiB, or the full capacity if the device has less.
+	 * Leave one BO_SIZE worth of headroom so the device isn't completely
+	 * exhausted before the cgroup limit is hit.
+	 */
+	cg_max = min(MAX_LIMIT, capacity - BO_SIZE);
+	cg_max = ALIGN_DOWN(cg_max, BO_SIZE);
+
+	/* Create cgroup and move into it */
+	cg = igt_cgroup_new("igt_cgroups_test");
+	igt_cgroup_move_current(cg);
+
+	max_bo = cg_max / BO_SIZE;
+
+	handles = calloc(max_bo, sizeof(handles[0]));
+	igt_assert_f(handles, "failed to allocate handles array");
+
+	n_bo = allocate_vram(handles, drv, ctx, max_bo, BO_SIZE);
+	igt_assert_f(n_bo > 0, "failed to allocate VRAM\n");
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After fill: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == cg_max,
+		     "current usage (%"PRIu64" MiB) is not requested allocation (%"PRIu64" MiB)\n",
+		     current / SZ_1M, cg_max / SZ_1M);
+
+	free_vram(handles, drv, ctx, n_bo);
+	wait_for_usage_drop(cg, cg_region, 0);
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After free: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 0,
+		     "current usage (%"PRIu64" MiB) is not zero\n",
+		     current / SZ_1M);
+
+	igt_cgroup_dmem_set_max(cg, cg_region, 2 * BO_SIZE, false);
+
+	n_bo = allocate_vram(handles, drv, ctx, max_bo, BO_SIZE);
+	igt_assert_f(n_bo > 0, "failed to allocate VRAM\n");
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After fill: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 2 * BO_SIZE,
+		     "current usage (%"PRIu64" MiB) is not requested max allocation (%"PRIu64" MiB)\n",
+		     current / SZ_1M, cg_max / SZ_1M);
+
+	free_vram(handles, drv, ctx, n_bo);
+	wait_for_usage_drop(cg, cg_region, 0);
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After free: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 0,
+		     "current usage (%"PRIu64" MiB) is not zero\n",
+		     current / SZ_1M);
+
+	igt_cgroup_dmem_set_max(cg, cg_region, 0, false);
+
+	n_bo = allocate_vram(handles, drv, ctx, max_bo, BO_SIZE);
+
+	/*
+	 * amdgpu may succeed the allocation, by falling back to GTT, so no assertion here.
+	 * Verify by reading current usage.
+	 */
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After fill: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 0,
+		     "current usage (%"PRIu64" MiB) is not zero\n",
+		     current / SZ_1M);
+
+	if (n_bo > 0)
+		free_vram(handles, drv, ctx, n_bo);
+	wait_for_usage_drop(cg, cg_region, 0);
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After free: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 0,
+		     "current usage (%"PRIu64" MiB) is not zero\n",
+		     current / SZ_1M);
+}
+
 static void test_write_eviction(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx)
 {
 	struct igt_cgroup *cg;
@@ -246,6 +392,7 @@ static const struct {
 	void (*test_fn)(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx);
 	unsigned int flags;
 } subtests[] = {
+	{ "current", test_current, 0 },
 	{ "write_eviction", test_write_eviction, 0 },
 	{ "write_eviction_interruptible", test_write_eviction, TEST_INTERRUPTIBLE },
 	{ "write_eviction_nonblock", test_write_eviction, TEST_NONBLOCK },
Signature · 1 line
2.47.3
#15 lore raw

Re: [PATCH i-g-t v7 9/9] tests/cgroup_dmem: add test for dmem.current

Quoted 2 lines · Thadeu Lima de Souza Cascardo

Based on the work of Thomas Hellström to test dmem.max eviction, add a test for dmem.current usage after allocations and setting dmem.max.

Isn't 7/9 testing current after setting max already? I went back and forth a bit and isn't the new thing in this test is looking at current after freeing objects? Unless I got confused maybe then rename the test and update the commit message. One more comment below.

Quoted 115 lines

Create a dmem cgroup, allocate close to capacity (or at most 4GiB), check current usage is within a small slack of the expected allocation. Then, set max to a small value and check allocations and current usage are limited to the max set. Set max to less than a single BO size, then check no allocations are allowed and current usage is also within the slack. After each allocation, release memory and check current usage has gone down.

---
  tests/cgroup_dmem.c | 147 ++++++++++++++++++++++++++++++++++++++++++++++++++++
  1 file changed, 147 insertions(+)
tests/cgroup_dmem.c +73 -0
index 7f3cfe22db2c..1f01037e1cf2 100644
@@ -36,6 +36,8 @@
  #define BO_SIZE			SZ_64M
  #define MAX_LIMIT		((uint64_t)4 * SZ_1G)
  #define EVICT_STEP		SZ_128M
+#define USAGE_POLL_MS		10
+#define USAGE_DROP_TIMEOUT_MS	1000
 
  #define TEST_INTERRUPTIBLE	(1 << 0)
  #define TEST_NONBLOCK		(1 << 1)
@@ -48,6 +50,20 @@
   *  and configured limits, then destroys the cgroup.
   */
 
+/**
+ * SUBTEST: current
+ * DESCRIPTION:
+ *  Create a dmem cgroup, allocate close to capacity (or at most 4GiB),
+ *  check current usage is within a small slack of the expected allocation.
+ *  Then, set max to a small value and check allocations and current usage
+ *  are limited to the max set.
+ *  Set max to less than a single BO size, then check no allocations are allowed
+ *  and current usage is also within the slack.
+ *  After each allocation, release memory and check current usage has gone
+ *  down.
+ *  This subtest requires GPU device with at least one VRAM region.
+ */
+
  /**
   * SUBTEST: write_eviction
   * DESCRIPTION:
@@ -116,6 +132,23 @@ static void install_sigcont_counter(void)
  	sigaction(SIGCONT, &sa, &sigcont_oldact);
  }
 
+static uint64_t wait_for_usage_drop(struct igt_cgroup *cg, const char *region,
+				    uint64_t limit)
+{
+	uint64_t current;
+	unsigned int elapsed = 0;
+
+	do {
+		igt_cgroup_dmem_get_current(cg, region, &current);
+		if (current <= limit)
+			return current;
+		usleep(USAGE_POLL_MS * 1000);
+		elapsed += USAGE_POLL_MS;
+	} while (elapsed < USAGE_DROP_TIMEOUT_MS);
+
+	return current;
+}
+
  static int allocate_vram(void **handles, const struct igt_dmem_driver *drv,
  			 void *ctx, int max_bo, size_t len)
  {
@@ -137,6 +170,119 @@ static int allocate_vram(void **handles, const struct igt_dmem_driver *drv,
  	return err;
  }
 
+static void free_vram(void **handles, const struct igt_dmem_driver *drv,
+		      void *ctx, int max_bo)
+{
+	int i;
+	for (i = 0; i < max_bo; i++)
+		drv->free_vram(ctx, handles[i]);
+}
+
+static void test_current(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx)
+{
+	struct igt_cgroup *cg;
+	void **handles;
+	uint64_t current, capacity, cg_max;
+	int n_bo = 0, max_bo;
+
+	igt_cgroup_dmem_get_capacity(cg_region, &capacity);
+	igt_require_f(capacity >= 4 * BO_SIZE,
+		      "VRAM capacity (%"PRIu64" MiB) too small to test\n",
+		      capacity / SZ_1M);
+
+	/*
+	 * Use up to 4 GiB, or the full capacity if the device has less.
+	 * Leave one BO_SIZE worth of headroom so the device isn't completely
+	 * exhausted before the cgroup limit is hit.
+	 */
+	cg_max = min(MAX_LIMIT, capacity - BO_SIZE);
+	cg_max = ALIGN_DOWN(cg_max, BO_SIZE);
+
+	/* Create cgroup and move into it */
+	cg = igt_cgroup_new("igt_cgroups_test");
+	igt_cgroup_move_current(cg);
+
+	max_bo = cg_max / BO_SIZE;
+
+	handles = calloc(max_bo, sizeof(handles[0]));
+	igt_assert_f(handles, "failed to allocate handles array");
+
+	n_bo = allocate_vram(handles, drv, ctx, max_bo, BO_SIZE);
+	igt_assert_f(n_bo > 0, "failed to allocate VRAM\n");
+

tests/cgroup_dmem.c static int allocate_vram(void **handles, const struct igt_dmem_driver *drv, · line 222

Quoted 10 lines
tests/cgroup_dmem.c +10 -0
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After fill: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == cg_max,
+		     "current usage (%"PRIu64" MiB) is not requested allocation (%"PRIu64" MiB)\n",
+		     current / SZ_1M, cg_max / SZ_1M);
+
+	free_vram(handles, drv, ctx, n_bo);
+	wait_for_usage_drop(cg, cg_region, 0);

The fact in there is a sleeping loop makes me think dmem cgroup contract is not for current to go down immediate when bos are freed from userspace, but when the driver feels like it?

In which case, to ask an annoying question, does it ever have to go down (in a generic case) if no one is touching neither the fd nor the device. Or maybe I am misunderstanding something.

Regards,

Tvrtko

Quoted 75 lines
tests/cgroup_dmem.c +64 -0
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After free: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 0,
+		     "current usage (%"PRIu64" MiB) is not zero\n",
+		     current / SZ_1M);
+
+	igt_cgroup_dmem_set_max(cg, cg_region, 2 * BO_SIZE, false);
+
+	n_bo = allocate_vram(handles, drv, ctx, max_bo, BO_SIZE);
+	igt_assert_f(n_bo > 0, "failed to allocate VRAM\n");
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After fill: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 2 * BO_SIZE,
+		     "current usage (%"PRIu64" MiB) is not requested max allocation (%"PRIu64" MiB)\n",
+		     current / SZ_1M, cg_max / SZ_1M);
+
+	free_vram(handles, drv, ctx, n_bo);
+	wait_for_usage_drop(cg, cg_region, 0);
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After free: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 0,
+		     "current usage (%"PRIu64" MiB) is not zero\n",
+		     current / SZ_1M);
+
+	igt_cgroup_dmem_set_max(cg, cg_region, 0, false);
+
+	n_bo = allocate_vram(handles, drv, ctx, max_bo, BO_SIZE);
+
+	/*
+	 * amdgpu may succeed the allocation, by falling back to GTT, so no assertion here.
+	 * Verify by reading current usage.
+	 */
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After fill: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 0,
+		     "current usage (%"PRIu64" MiB) is not zero\n",
+		     current / SZ_1M);
+
+	if (n_bo > 0)
+		free_vram(handles, drv, ctx, n_bo);
+	wait_for_usage_drop(cg, cg_region, 0);
+
+	igt_cgroup_dmem_get_current(cg, cg_region, &current);
+	igt_debug("After free: cgroup current = %"PRIu64" MiB, "
+		  "max = %"PRIu64" MiB\n",
+		  current / SZ_1M, cg_max / SZ_1M);
+	igt_assert_f(current == 0,
+		     "current usage (%"PRIu64" MiB) is not zero\n",
+		     current / SZ_1M);
+}
+
  static void test_write_eviction(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx)
  {
  	struct igt_cgroup *cg;
@@ -246,6 +392,7 @@ static const struct {
  	void (*test_fn)(int fd, char *cg_region, unsigned int flags, const struct igt_dmem_driver *drv, void *ctx);
  	unsigned int flags;
  } subtests[] = {
+	{ "current", test_current, 0 },
  	{ "write_eviction", test_write_eviction, 0 },
  	{ "write_eviction_interruptible", test_write_eviction, TEST_INTERRUPTIBLE },
  	{ "write_eviction_nonblock", test_write_eviction, TEST_NONBLOCK },