selftests/cgroup: memory controller self-tests

Cgroups are used for controlling the physical resource distribution (memory, CPU, io, etc) and often are used as basic building blocks for large distributed computing systems. Even small differences in the actual behavior may lead to significant incidents.

The codebase is under the active development, which will unlikely stop at any time soon. Also it's scattered over different kernel subsystems, which makes regressions more probable.

Given that, the lack of any tests is crying.

This patch implements some basic tests for the memory controller, as well as a minimal required framework.
It doesn't pretend for a very good coverage, but pretends to be a starting point.

Hopefully, any following significant changes will
include corresponding tests.

Tests for CPU and io controllers, as well as cgroup core are next in the todo list.

Quoted 19 lines · Roman Gushchin

Cgroups are used for controlling the physical resource distribution (memory, CPU, io, etc) and often are used as basic building blocks for large distributed computing systems. Even small differences in the actual behavior may lead to significant incidents.

The codebase is under the active development, which will unlikely stop at any time soon. Also it's scattered over different kernel subsystems, which makes regressions more probable.

Given that, the lack of any tests is crying.

This patch implements some basic tests for the memory controller, as well as a minimal required framework.
It doesn't pretend for a very good coverage, but pretends to be a starting point.

Hopefully, any following significant changes will
include corresponding tests.

Quoted 2 lines

Tests for CPU and io controllers, as well as cgroup core are next in the todo list.

This is great. Applying to cgroup/for-4.18.

Thanks a lot.

Don't pull them yet. If Roman can redo the patches on linux-kselftest next, he can pick up the SKIP changes.

Hello, Shuah.

Quoted 2 lines · Shuah Khan

Don't pull them yet. If Roman can redo the patches on linux-kselftest next, he can pick up the SKIP changes.

So, what we can do is either fix that up during / after the merge window or creating a separate branch which pulls from both cgroup and kselftest. Please let me know which way you wanna go and if the latter which branch I should pull (it gotta be a stable branch).

Thanks.

I think we don't need to create a special branch and all. The following should work:

linux-next already has the skip work. What we can do is:

Do the cleanup and test it against linux-next. In linux-next SKIP isn't PASS. If test is compiled on linux-next, you will see that SKIP is SKIP. If it is compiled on the mainline, it will be reported PASS, which will be a temporary state.

Hello, Shuah.

Quoted 5 lines · Shuah Khan

I think we don't need to create a special branch and all. The following should work:

linux-next already has the skip work. What we can do is:

Quoted 3 lines

Do the cleanup and test it against linux-next. In linux-next SKIP isn't PASS. If test is compiled on linux-next, you will see that SKIP is SKIP. If it is compiled on the mainline, it will be reported PASS, which will be a temporary state.

Hah, why not just create a branch and make sure what we see in the topic branch is what we'll push? That's how these things are done usually.

Quoted 8 lines · Tejun Heo

Hello, Shuah.

Quoted 5 lines · Shuah Khan

I think we don't need to create a special branch and all. The following should work:

linux-next already has the skip work. What we can do is:

Quoted 8 lines

Quoted 3 lines

Do the cleanup and test it against linux-next. In linux-next SKIP isn't PASS. If test is compiled on linux-next, you will see that SKIP is SKIP. If it is compiled on the mainline, it will be reported PASS, which will be a temporary state.

Hah, why not just create a branch and make sure what we see in the topic branch is what we'll push? That's how these things are done usually.

It probably doesn't need to be complex.

Unless there is a dependency with the cgroup tree and the cgroup test, the test can go through kselftest tree with you. That is usually how I handle kselftests.

If you think there is a dependency and it has to go through cgroup tree then, I can give you the Ack once this TEST_* gets cleaned up.

thanks,
-- Shuah

From 0cfd4f1ce368fd240575f1a087f93b743690bda3 Mon Sep 17 00:00:00 2001

Quoted 24 lines · Shuah Khan

Quoted 8 lines · Tejun Heo

Hello, Shuah.

Quoted 5 lines · Shuah Khan

I think we don't need to create a special branch and all. The following should work:

linux-next already has the skip work. What we can do is:

Quoted 8 lines

Quoted 3 lines

Do the cleanup and test it against linux-next. In linux-next SKIP isn't PASS. If test is compiled on linux-next, you will see that SKIP is SKIP. If it is compiled on the mainline, it will be reported PASS, which will be a temporary state.

Hah, why not just create a branch and make sure what we see in the topic branch is what we'll push? That's how these things are done usually.

It probably doesn't need to be complex.

Unless there is a dependency with the cgroup tree and the cgroup test, the test can go through kselftest tree with you. That is usually how I handle kselftests.

Quoted 6 lines

If you think there is a dependency and it has to go through cgroup tree then, I can give you the Ack once this TEST_* gets cleaned up.

thanks,
-- Shuah

Hello, Shuah!

A minor problem here is a version with KSFT_ constants won't even compile without your patch, which redefines KSFT_SKIP into a separate value, due to duplicate cases in the switch statement at the end of the file.

Anyway, not a big deal, we can handle it either way. An updated version below.

Thanks!

Yeah. I see that. You have a switch for the KSFT_ values. Since there is no dependency on the cgroup tree, I would recommend having this patch go through kselftest tree which is the normal process for tests anyway.

This version is good and I can apply this to linux-kselftest next. I ran a quick test and the Skip case looks good.

TAP version 13
selftests: cgroup: test_memcontrol
========================================
1..0 # Skipped: memory controller isn't available not ok 1..1 selftests: cgroup: test_memcontrol [SKIP]

Tejun! Please send me your Ack.

Quoted 14 lines · Shuah Khan

Yeah. I see that. You have a switch for the KSFT_ values. Since there is no dependency on the cgroup tree, I would recommend having this patch go through kselftest tree which is the normal process for tests anyway.

This version is good and I can apply this to linux-kselftest next. I ran a quick test and the Skip case looks good.

TAP version 13
selftests: cgroup: test_memcontrol
========================================
1..0 # Skipped: memory controller isn't available not ok 1..1 selftests: cgroup: test_memcontrol [SKIP]

Quoted 1 line

Tejun! Please send me your Ack.

Sure, please feel free to add my ack. I'll revert the original patch from cgroup tree.

Thanks.

Quoted 20 lines · Tejun Heo

Quoted 7 lines · Shuah Khan

Yeah. I see that. You have a switch for the KSFT_ values. Since there is no dependency on the cgroup tree, I would recommend having this patch go through kselftest tree which is the normal process for tests anyway.

This version is good and I can apply this to linux-kselftest next. I ran a quick test and the Skip case looks good.

Quoted 8 lines

TAP version 13
selftests: cgroup: test_memcontrol
========================================
1..0 # Skipped: memory controller isn't available not ok 1..1 selftests: cgroup: test_memcontrol [SKIP]

Tejun! Please send me your Ack.

Sure, please feel free to add my ack. I'll revert the original patch from cgroup tree.

Quoted 2 lines

Thanks.

Thanks. I will apply this to linux-kselftest next for 4.18-rc1 with you Ack.

thanks,
-- Shuah

Cgroups are used for controlling the physical resource distribution (memory, CPU, io, etc) and often are used as basic building blocks for large distributed computing systems. Even small differences in the actual behavior may lead to significant incidents.

The codebase is under the active development, which will unlikely stop at any time soon. Also it's scattered over different kernel subsystems, which makes regressions more probable.

Given that, the lack of any tests is crying.

This patch implements some basic tests for the memory controller, as well as a minimal required framework.
It doesn't pretend for a very good coverage, but pretends to be a starting point.

Hopefully, any following significant changes will
include corresponding tests.

Tests for CPU and io controllers, as well as cgroup core are next in the todo list.

Signed-off-by Roman Gushchin <guro at fb.com>

Cc Tejun Heo <tj at kernel.org>

Cc Shuah Khan <shuah at kernel.org>

Cc Johannes Weiner <hannes at cmpxchg.org>

Cc Michal Hocko <mhocko at suse.com>

Cc Mike Rapoport <rppt at linux.vnet.ibm.com>

Cc kernel-team at fb.com

Cc linux-kselftest at vger.kernel.org

Cc linux-kernel at vger.kernel.org

---
 tools/testing/selftests/Makefile                 |   1 +
 tools/testing/selftests/cgroup/Makefile          |  10 +
 tools/testing/selftests/cgroup/cgroup_util.c     | 317 ++++++++++
 tools/testing/selftests/cgroup/cgroup_util.h     |  40 ++
 tools/testing/selftests/cgroup/test_memcontrol.c | 741 +++++++++++++++++++++++
 5 files changed, 1109 insertions(+)
 create mode 100644 tools/testing/selftests/cgroup/Makefile
 create mode 100644 tools/testing/selftests/cgroup/cgroup_util.c
 create mode 100644 tools/testing/selftests/cgroup/cgroup_util.h
 create mode 100644 tools/testing/selftests/cgroup/test_memcontrol.c

tools/testing/selftests/Makefile +1 -0

index 32aafa92074c..24d0331d6c11 100644
@@ -3,6 +3,7 @@ TARGETS = android
 TARGETS += bpf
 TARGETS += breakpoints
 TARGETS += capabilities
+TARGETS += cgroup
 TARGETS += cpufreq
 TARGETS += cpu-hotplug
 TARGETS += efivarfs

tools/testing/selftests/cgroup/Makefile new file +10 -0

new file mode 100644
index 000000000000..f7a31392eb2f
@@ -0,0 +1,10 @@
+# SPDX-License-Identifier: GPL-2.0
+CFLAGS += -Wall
+
+all:
+
+TEST_GEN_PROGS = test_memcontrol
+
+include ../lib.mk
+
+$(OUTPUT)/test_memcontrol: cgroup_util.c

tools/testing/selftests/cgroup/cgroup_util.c new file +317 -0

new file mode 100644
index 000000000000..a938b6c8b55a
@@ -0,0 +1,317 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+
+#define _GNU_SOURCE
+
+#include <errno.h>
+#include <fcntl.h>
+#include <linux/limits.h>
+#include <signal.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <sys/wait.h>
+#include <unistd.h>
+
+#include "cgroup_util.h"
+
+static ssize_t read_text(const char *path, char *buf, size_t max_len)
+{
+	ssize_t len;
+	int fd;
+
+	fd = open(path, O_RDONLY);
+	if (fd < 0)
+		return fd;
+
+	len = read(fd, buf, max_len - 1);
+	if (len < 0)
+		goto out;
+
+	buf[len] = 0;
+out:
+	close(fd);
+	return len;
+}
+
+static ssize_t write_text(const char *path, char *buf, size_t len)
+{
+	int fd;
+
+	fd = open(path, O_WRONLY | O_APPEND);
+	if (fd < 0)
+		return fd;
+
+	len = write(fd, buf, len);
+	if (len < 0) {
+		close(fd);
+		return len;
+	}
+
+	close(fd);
+
+	return len;
+}
+
+char *cg_name(const char *root, const char *name)
+{
+	size_t len = strlen(root) + strlen(name) + 2;
+	char *ret = malloc(len);
+
+	if (name)
+		snprintf(ret, len, "%s/%s", root, name);
+
+	return ret;
+}
+
+char *cg_name_indexed(const char *root, const char *name, int index)
+{
+	size_t len = strlen(root) + strlen(name) + 10;
+	char *ret = malloc(len);
+
+	if (name)
+		snprintf(ret, len, "%s/%s_%d", root, name, index);
+
+	return ret;
+}
+
+int cg_read(const char *cgroup, const char *control, char *buf, size_t len)
+{
+	char path[PATH_MAX];
+
+	snprintf(path, sizeof(path), "%s/%s", cgroup, control);
+
+	if (read_text(path, buf, len) >= 0)
+		return 0;
+
+	return -1;
+}
+
+int cg_read_strcmp(const char *cgroup, const char *control,
+		   const char *expected)
+{
+	size_t size = strlen(expected) + 1;
+	char *buf;
+
+	buf = malloc(size);
+	if (!buf)
+		return -1;
+
+	if (cg_read(cgroup, control, buf, size))
+		return -1;
+
+	return strcmp(expected, buf);
+}
+
+int cg_read_strstr(const char *cgroup, const char *control, const char *needle)
+{
+	char buf[PAGE_SIZE];
+
+	if (cg_read(cgroup, control, buf, sizeof(buf)))
+		return -1;
+
+	return strstr(buf, needle) ? 0 : -1;
+}
+
+long cg_read_long(const char *cgroup, const char *control)
+{
+	char buf[128];
+
+	if (cg_read(cgroup, control, buf, sizeof(buf)))
+		return -1;
+
+	return atol(buf);
+}
+
+long cg_read_key_long(const char *cgroup, const char *control, const char *key)
+{
+	char buf[PAGE_SIZE];
+	char *ptr;
+
+	if (cg_read(cgroup, control, buf, sizeof(buf)))
+		return -1;
+
+	ptr = strstr(buf, key);
+	if (!ptr)
+		return -1;
+
+	return atol(ptr + strlen(key));
+}
+
+int cg_write(const char *cgroup, const char *control, char *buf)
+{
+	char path[PATH_MAX];
+	size_t len = strlen(buf);
+
+	snprintf(path, sizeof(path), "%s/%s", cgroup, control);
+
+	if (write_text(path, buf, len) == len)
+		return 0;
+
+	return -1;
+}
+
+int cg_find_unified_root(char *root, size_t len)
+{
+	char buf[10 * PAGE_SIZE];
+	char *fs, *mount, *type;
+	const char delim[] = "\n\t ";
+
+	if (read_text("/proc/self/mounts", buf, sizeof(buf)) <= 0)
+		return -1;
+
+	/*
+	 * Example:
+	 * cgroup /sys/fs/cgroup cgroup2 rw,seclabel,noexec,relatime 0 0
+	 */
+	for (fs = strtok(buf, delim); fs; fs = strtok(NULL, delim)) {
+		mount = strtok(NULL, delim);
+		type = strtok(NULL, delim);
+		strtok(NULL, delim);
+		strtok(NULL, delim);
+		strtok(NULL, delim);
+
+		if (strcmp(fs, "cgroup") == 0 &&
+		    strcmp(type, "cgroup2") == 0) {
+			strncpy(root, mount, len);
+			return 0;
+		}
+	}
+
+	return -1;
+}
+
+int cg_create(const char *cgroup)
+{
+	return mkdir(cgroup, 0644);
+}
+
+static int cg_killall(const char *cgroup)
+{
+	char buf[PAGE_SIZE];
+	char *ptr = buf;
+
+	if (cg_read(cgroup, "cgroup.procs", buf, sizeof(buf)))
+		return -1;
+
+	while (ptr < buf + sizeof(buf)) {
+		int pid = strtol(ptr, &ptr, 10);
+
+		if (pid == 0)
+			break;
+		if (*ptr)
+			ptr++;
+		else
+			break;
+		if (kill(pid, SIGKILL))
+			return -1;
+	}
+
+	return 0;
+}
+
+int cg_destroy(const char *cgroup)
+{
+	int ret;
+
+retry:
+	ret = rmdir(cgroup);
+	if (ret && errno == EBUSY) {
+		ret = cg_killall(cgroup);
+		if (ret)
+			return ret;
+		usleep(100);
+		goto retry;
+	}
+
+	if (ret && errno == ENOENT)
+		ret = 0;
+
+	return ret;
+}
+
+int cg_run(const char *cgroup,
+	   int (*fn)(const char *cgroup, void *arg),
+	   void *arg)
+{
+	int pid, retcode;
+
+	pid = fork();
+	if (pid < 0) {
+		return pid;
+	} else if (pid == 0) {
+		char buf[64];
+
+		snprintf(buf, sizeof(buf), "%d", getpid());
+		if (cg_write(cgroup, "cgroup.procs", buf))
+			exit(EXIT_FAILURE);
+		exit(fn(cgroup, arg));
+	} else {
+		waitpid(pid, &retcode, 0);
+		if (WIFEXITED(retcode))
+			return WEXITSTATUS(retcode);
+		else
+			return -1;
+	}
+}
+
+int cg_run_nowait(const char *cgroup,
+		  int (*fn)(const char *cgroup, void *arg),
+		  void *arg)
+{
+	int pid;
+
+	pid = fork();
+	if (pid == 0) {
+		char buf[64];
+
+		snprintf(buf, sizeof(buf), "%d", getpid());
+		if (cg_write(cgroup, "cgroup.procs", buf))
+			exit(EXIT_FAILURE);
+		exit(fn(cgroup, arg));
+	}
+
+	return pid;
+}
+
+int get_temp_fd(void)
+{
+	return open(".", O_TMPFILE | O_RDWR | O_EXCL);
+}
+
+int alloc_pagecache(int fd, size_t size)
+{
+	char buf[PAGE_SIZE];
+	struct stat st;
+	int i;
+
+	if (fstat(fd, &st))
+		goto cleanup;
+
+	size += st.st_size;
+
+	if (ftruncate(fd, size))
+		goto cleanup;
+
+	for (i = 0; i < size; i += sizeof(buf))
+		read(fd, buf, sizeof(buf));
+
+	return 0;
+
+cleanup:
+	return -1;
+}
+
+int alloc_anon(const char *cgroup, void *arg)
+{
+	size_t size = (unsigned long)arg;
+	char *buf, *ptr;
+
+	buf = malloc(size);
+	for (ptr = buf; ptr < buf + size; ptr += PAGE_SIZE)
+		*ptr = 0;
+
+	free(buf);
+	return 0;
+}

tools/testing/selftests/cgroup/cgroup_util.h new file +40 -0

new file mode 100644
index 000000000000..000de075d3d8
@@ -0,0 +1,40 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#include <stdlib.h>
+
+#define PAGE_SIZE 4096
+
+#define ARRAY_SIZE(arr) (sizeof(arr) / sizeof((arr)[0]))
+
+#define MB(x) (x << 20)
+
+/*
+ * Checks if two given values differ by less than err% of their sum.
+ */
+static inline int values_close(long a, long b, int err)
+{
+	return abs(a - b) <= (a + b) / 100 * err;
+}
+
+extern int cg_find_unified_root(char *root, size_t len);
+extern char *cg_name(const char *root, const char *name);
+extern char *cg_name_indexed(const char *root, const char *name, int index);
+extern int cg_create(const char *cgroup);
+extern int cg_destroy(const char *cgroup);
+extern int cg_read(const char *cgroup, const char *control,
+		   char *buf, size_t len);
+extern int cg_read_strcmp(const char *cgroup, const char *control,
+			  const char *expected);
+extern int cg_read_strstr(const char *cgroup, const char *control,
+			  const char *needle);
+extern long cg_read_long(const char *cgroup, const char *control);
+long cg_read_key_long(const char *cgroup, const char *control, const char *key);
+extern int cg_write(const char *cgroup, const char *control, char *buf);
+extern int cg_run(const char *cgroup,
+		  int (*fn)(const char *cgroup, void *arg),
+		  void *arg);
+extern int cg_run_nowait(const char *cgroup,
+			 int (*fn)(const char *cgroup, void *arg),
+			 void *arg);
+extern int get_temp_fd(void);
+extern int alloc_pagecache(int fd, size_t size);
+extern int alloc_anon(const char *cgroup, void *arg);

tools/testing/selftests/cgroup/test_memcontrol.c new file +15 -0

new file mode 100644
index 000000000000..b6bbc4c1adc3
@@ -0,0 +1,741 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#define _GNU_SOURCE
+
+#include <linux/limits.h>
+#include <fcntl.h>
+#include <stdio.h>
+#include <stdlib.h>
+#include <string.h>
+#include <sys/stat.h>
+#include <sys/types.h>
+#include <unistd.h>
+
+#include "../kselftest.h"
+#include "cgroup_util.h"
+