Skip to contentSkip to conversation

bpf: arena: handle memory.max on fault-in with reclaim/OOM

PATCH Superseded Jiayuan Chen 2 participants 7 messages Aug 24

Subsystems (5)
Kselftest BPF BPF selftests BPF core +1 more subsystems

v5 of 9 Superseded Latest is v9 →

lore
Share & tools
lore
#0 lore raw

v5 cover bpf: arena: handle memory.max on fault-in with reclaim/OOM

Since commit e66fe1bc6d25 ("bpf: arena: Reintroduce memcg accounting"), arena pages are charged to the memcg of the process that created the arena. That accounting exposes two problems in the arena user page fault path.

1. The fault-in allocation runs under arena->spinlock, so it can only use the non-blocking allocator, which never reclaims. Once memory.current is at memory.max the allocation simply fails. Reaching memory.max is completely normal for a healthy application - e.g. reading a large file fills memory.current with page cache - so the process ends up killed for no real reason.

2. That failure is turned into VM_FAULT_SIGSEGV, which is misleading: the faulting address is a perfectly valid arena address. The memcg OOM killer should run instead, and a genuine out-of-memory should surface as a non-recoverable fault, not a bogus segfault.

Preallocate the page outside the lock (patch 2), the way do_anonymous_page() does, so the allocation can sleep, reclaim and run the memcg OOM killer. On a genuine failure the fault is non-recoverable and returns VM_FAULT_SIGBUS; a task faulting its own arena is unaffected, the OOM killer picks it inside the allocation and it dies by SIGKILL. This needs a sleepable allocator (patch 1), because can_alloc_pages() is a conservative guess for BPF program context and always forces the non-blocking allocator under PREEMPT_RT. patch 3&4 adds a selftest that faults an arena in under a memory.max limit: without the fix the child gets SIGSEGV on a valid address, with it the child is killed by the memcg OOM killer.

v4 -> v5:
- arena: return VM_FAULT_SIGBUS instead of VM_FAULT_OOM when the fault-in allocation fails. The allocation already ran reclaim and the OOM killer, so the failure is non-recoverable; VM_FAULT_OOM would be retried by the fault path and can livelock a task faulting a shared arena whose memcg OOM killer cannot reach it. (reported by the Sashiko AI review)
- selftest: cap the arena at 50000 pages so it stays under the 4G arena limit on 64K-page kernels.
- selftest: ASSERT_OK setup_cgroup_environment(), print a reason on the no-memory-controller skip, and flush only stderr in the child.

v3 -> v4:
- rebase bpf-next and fix conflict
- add Reviewed-by tag from Emil Tsalapatis

v2 -> v3:
- selftest: check the memcg OOM via memory.events "oom_kill" instead of the exit signal; it only aims to pass on the fixed kernel, since the unfixed SIGSEGV is racy.

v1 -> v2:
- Rebase on the separate deadlock fix (found by the Sashiko AI review), now applied to bpf-next.
- Honor the map's NUMA node on fault-in.
- Return VM_FAULT_SIGBUS for the non-recoverable faults (lock, range-tree and page-table failures); a scratch-page hole stays VM_FAULT_SIGSEGV only under BPF_F_SEGV_ON_FAULT. (Kumar Kartikeya Dwivedi)
- Add read_cgroup_file() to cgroup_helpers instead of open-coding the /mnt/... path in the test. (Emil Tsalapatis)
- Dump the cgroup memory stats on test failure to ease debugging.

v4:
https://lore.kernel.org/bpf/[email protected]/T/#t
v2:
https://lore.kernel.org/bpf/[email protected]/
v1:
https://lore.kernel.org/bpf/[email protected]/

Jiayuan Chen (4):
bpf: Add a sleepable page allocator for map memory
bpf: arena: allocate the fault-in page outside the lock selftests/bpf: Add read_cgroup_file() to cgroup_helpers selftests/bpf: Add a test for arena fault-in under memory.max

 include/linux/bpf.h                           |   1 +
 kernel/bpf/arena.c                            |  90 +++++++---
 kernel/bpf/syscall.c                          |  21 ++-
 tools/testing/selftests/bpf/cgroup_helpers.c  |  67 ++++++++
 tools/testing/selftests/bpf/cgroup_helpers.h  |   4 +
 .../selftests/bpf/prog_tests/arena_memcg.c    | 158 ++++++++++++++++++
 .../testing/selftests/bpf/progs/arena_memcg.c |  24 +++
 7 files changed, 340 insertions(+), 25 deletions(-)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_memcg.c
 create mode 100644 tools/testing/selftests/bpf/progs/arena_memcg.c
Signature · 1 line
2.43.0
#1 lore raw

PATCH v5 1/4 bpf: Add a sleepable page allocator for map memory

bpf_map_alloc_pages() picks the allocator via can_alloc_pages(), a conservative guess for BPF program context that is always false under PREEMPT_RT. So even a caller that really is sleepable gets the non-blocking allocator, which never reclaims and never engages the OOM machinery.

Add bpf_map_alloc_page_sleepable() for callers that know they are sleepable. Like the other bpf map allocators it places the page on the map's numa_node and does not follow the faulting task's NUMA mempolicy; arena memory is shared, so the map's node is the right placement policy. The next patch uses it from the arena page fault handler.

---
To sashiko:
- The alloc_pages_nolock() fallback still zeroes: it forces __GFP_ZERO internally and only accepts __GFP_ACCOUNT.
- __GFP_ZERO is unchanged from the existing __bpf_alloc_page(), and no arena-capable arch has D-cache aliasing, so there is no dcache concern.

---
 include/linux/bpf.h  |  1 +
 kernel/bpf/syscall.c | 21 +++++++++++++++++----
 2 files changed, 18 insertions(+), 4 deletions(-)
include/linux/bpf.h +1 -0
index ffa5626411ac..18e84ec4692d 100644
@@ -2784,6 +2784,7 @@ struct bpf_prog *bpf_prog_get_curr_or_next(u32 *id);
 
 int bpf_map_alloc_pages(const struct bpf_map *map, int nid,
 			unsigned long nr_pages, struct page **page_array);
+struct page *bpf_map_alloc_page_sleepable(const struct bpf_map *map);
 #ifdef CONFIG_MEMCG
 void bpf_map_memcg_enter(const struct bpf_map *map, struct mem_cgroup **old_memcg,
 			 struct mem_cgroup **new_memcg);
kernel/bpf/syscall.c +17 -4
index 6874ba1424af..f9b81638e537 100644
@@ -602,15 +602,14 @@ static bool can_alloc_pages(void)
 		!IS_ENABLED(CONFIG_PREEMPT_RT);
 }
 
+#define BPF_PAGE_GFP (GFP_KERNEL | __GFP_ZERO | __GFP_ACCOUNT | __GFP_NOWARN)
+
 static struct page *__bpf_alloc_page(int nid)
 {
 	if (!can_alloc_pages())
 		return alloc_pages_nolock(__GFP_ACCOUNT, nid, 0);
 
-	return alloc_pages_node(nid,
-				GFP_KERNEL | __GFP_ZERO | __GFP_ACCOUNT
-				| __GFP_NOWARN,
-				0);
+	return alloc_pages_node(nid, BPF_PAGE_GFP, 0);
 }
 
 int bpf_map_alloc_pages(const struct bpf_map *map, int nid,
@@ -636,6 +635,20 @@ int bpf_map_alloc_pages(const struct bpf_map *map, int nid,
 	return ret;
 }
 
+/*
+ * For callers that know they run in a sleepable context, e.g. a user page
+ * fault handler. can_alloc_pages() is a conservative guess made for BPF
+ * program context - notably it is always false on PREEMPT_RT - so going
+ * through bpf_map_alloc_pages() there would needlessly pick the
+ * non-blocking allocator, which never reclaims and never engages the OOM
+ * machinery.
+ */
+struct page *bpf_map_alloc_page_sleepable(const struct bpf_map *map)
+{
+	might_sleep();
+	return alloc_pages_node(map->numa_node, BPF_PAGE_GFP, 0);
+}
+
 static int btf_field_cmp(const void *a, const void *b)
 {
 	const struct btf_field *f1 = a, *f2 = b;
Signature · 1 line
2.43.0
#2 lore raw

PATCH v5 2/4 bpf: arena: allocate the fault-in page outside the lock

In reply to Jiayuan Chen

arena_vm_fault() allocated the page while holding arena->spinlock, so it could only use the non-blocking allocator. Once the memcg is at memory.max that allocation just fails, the fault turns into VM_FAULT_SIGSEGV, and the process gets a SIGSEGV on a perfectly valid arena address. Hitting memory.max is routine (e.g. page cache from reading a big file), so this kills innocent processes.

Rework the fault handler:

- Preallocate the page before taking the lock, like do_anonymous_page() does, so it can sleep and go through reclaim and the memcg OOM killer, instead of turning a routine memory.max into a fake segfault.

- On allocation failure return VM_FAULT_SIGBUS. The allocation already ran reclaim and the OOM killer, so the failure is non-recoverable. For a task faulting its own arena this changes nothing: the OOM killer already picked it inside the allocation and it dies by SIGKILL, the SIGBUS is shadowed by the pending fatal signal, and the memcg OOM is still reported. VM_FAULT_OOM would instead be retried by the fault path and can livelock when the charged memcg is not the faulting task's (e.g. a shared arena) and its OOM killer cannot reach it.

- A lockless probe skips that preallocation when a page is already mapped (e.g. allocated by the bpf program), so the common case wastes no allocation. The rare race where such a page is freed before we take the lock falls back to the non-blocking allocator under the lock.

- Return VM_FAULT_SIGBUS for the other non-recoverable errors (lock failure, range-tree and page-table failures) instead of VM_FAULT_SIGSEGV; only BPF_F_SEGV_ON_FAULT, and a scratch-page hole under that flag, is a real user addressing error and keeps VM_FAULT_SIGSEGV.

- Tidy up the error labels.

---
 kernel/bpf/arena.c | 90 +++++++++++++++++++++++++++++++++++-----------
 1 file changed, 69 insertions(+), 21 deletions(-)
kernel/bpf/arena.c +69 -21
index 7b6847200b43..fa462a0ff1ad 100644
@@ -481,7 +481,8 @@ static vm_fault_t arena_vm_fault(struct vm_fault *vmf)
 	struct bpf_map *map = vmf->vma->vm_file->private_data;
 	struct bpf_arena *arena = container_of(map, struct bpf_arena, map);
 	struct mem_cgroup *new_memcg, *old_memcg;
-	struct page *page;
+	struct page *page, *new_page = NULL;
+	vm_fault_t fault_ret;
 	long kbase, kaddr;
 	unsigned long flags;
 	int ret;
@@ -489,59 +490,106 @@ static vm_fault_t arena_vm_fault(struct vm_fault *vmf)
 	kbase = bpf_arena_get_kern_vm_start(arena);
 	kaddr = kbase + (u32)(vmf->address);
 
-	if (raw_res_spin_lock_irqsave(&arena->spinlock, flags))
+	page = vmalloc_to_page((void *)kaddr);
+	if (!page && !(arena->map.map_flags & BPF_F_SEGV_ON_FAULT)) {
+		/*
+		 * Preallocate outside the lock with a sleepable allocator so it
+		 * can reclaim and run the memcg OOM killer, which the
+		 * non-blocking allocator under arena->spinlock cannot. A NULL
+		 * return is non-recoverable, so fail with VM_FAULT_SIGBUS;
+		 * VM_FAULT_OOM would be retried by the fault path and can
+		 * livelock when the charged memcg is not the faulting task's.
+		 */
+		bpf_map_memcg_enter(&arena->map, &old_memcg, &new_memcg);
+		new_page = bpf_map_alloc_page_sleepable(map);
+		bpf_map_memcg_exit(old_memcg, new_memcg);
+		if (!new_page)
+			return VM_FAULT_SIGBUS;
+	}
+
+	if (raw_res_spin_lock_irqsave(&arena->spinlock, flags)) {
 		/*
 		 * A failed lock means a possible deadlock was detected. Don't
 		 * return VM_FAULT_RETRY: this handler never took mmap_lock, but
 		 * the fault path would re-take it on retry and deadlock. Fail.
 		 */
+		if (new_page)
+			free_pages_nolock(new_page, 0);
 		return VM_FAULT_SIGBUS;
+	}
 
 	page = vmalloc_to_page((void *)kaddr);
 	if (page) {
-		if (page == arena->scratch_page)
-			/* BPF triggered scratch here; don't lazy-alloc over it */
-			goto out_sigsegv;
+		if (page == arena->scratch_page) {
+			/*
+			 * A scratch page marks a hole. Segfault only if the user
+			 * asked for it; otherwise we could lazy-allocate but
+			 * choose not to over a hole, so report a bus error.
+			 */
+			fault_ret = (arena->map.map_flags & BPF_F_SEGV_ON_FAULT) ?
+				    VM_FAULT_SIGSEGV : VM_FAULT_SIGBUS;
+			goto out_err_locked;
+		}
 		/* already have a page vmap-ed */
 		goto out;
 	}
 
+	if (arena->map.map_flags & BPF_F_SEGV_ON_FAULT) {
+		/* User space requested to segfault when page is not allocated by bpf prog */
+		fault_ret = VM_FAULT_SIGSEGV;
+		goto out_err_locked;
+	}
+
 	bpf_map_memcg_enter(&arena->map, &old_memcg, &new_memcg);
 
-	if (arena->map.map_flags & BPF_F_SEGV_ON_FAULT)
-		/* User space requested to segfault when page is not allocated by bpf prog */
-		goto out_sigsegv_memcg;
+	if (!new_page) {
+		/*
+		 * Very rare race: the bpf program had allocated a page here, so
+		 * the lockless probe saw it and we skipped preallocation, but it
+		 * freed the page before we took the lock. Now we do need one;
+		 * sleeping is not allowed here, so fall back to the non-blocking
+		 * allocator and give up if it fails.
+		 */
+		ret = bpf_map_alloc_pages(map, map->numa_node, 1, &new_page);
+		if (ret) {
+			fault_ret = VM_FAULT_SIGBUS;
+			goto out_err_locked_memcg;
+		}
+	}
 
 	ret = range_tree_clear(&arena->rt, vmf->pgoff, 1);
-	if (ret)
-		goto out_sigsegv_memcg;
-
-	struct apply_range_data data = { .arena = arena, .pages = &page, .i = 0 };
-	/* Account into memcg of the process that created bpf_arena */
-	ret = bpf_map_alloc_pages(map, NUMA_NO_NODE, 1, &page);
 	if (ret) {
-		range_tree_set(&arena->rt, vmf->pgoff, 1);
-		goto out_sigsegv_memcg;
+		fault_ret = VM_FAULT_SIGBUS;
+		goto out_err_locked_memcg;
 	}
+	struct apply_range_data data = { .arena = arena, .pages = &new_page, .i = 0 };
 
 	ret = apply_to_page_range(&init_mm, kaddr, PAGE_SIZE, apply_range_set_cb, &data);
 	if (ret) {
 		range_tree_set(&arena->rt, vmf->pgoff, 1);
-		free_pages_nolock(page, 0);
-		goto out_sigsegv_memcg;
+		fault_ret = VM_FAULT_SIGBUS;
+		goto out_err_locked_memcg;
 	}
 	flush_vmap_cache(kaddr, PAGE_SIZE);
 	bpf_map_memcg_exit(old_memcg, new_memcg);
+	/* new_page was consumed */
+	page = new_page;
+	new_page = NULL;
 out:
 	page_ref_add(page, 1);
 	raw_res_spin_unlock_irqrestore(&arena->spinlock, flags);
+	if (new_page)
+		free_pages_nolock(new_page, 0);
 	vmf->page = page;
 	return 0;
-out_sigsegv_memcg:
+
+out_err_locked_memcg:
 	bpf_map_memcg_exit(old_memcg, new_memcg);
-out_sigsegv:
+out_err_locked:
 	raw_res_spin_unlock_irqrestore(&arena->spinlock, flags);
-	return VM_FAULT_SIGSEGV;
+	if (new_page)
+		free_pages_nolock(new_page, 0);
+	return fault_ret;
 }
 
 static const struct vm_operations_struct arena_vm_ops = {
Signature · 1 line
2.43.0
#4 lore raw

PATCH v5 3/4 selftests/bpf: Add read_cgroup_file() to cgroup_helpers

In reply to Jiayuan Chen

cgroup_helpers has write_cgroup_file()/write_cgroup_file_parent() but no read counterpart. Add read_cgroup_file() and read_cgroup_file_parent() so a forked child can read a cgroup file (e.g. memory.current) from the work dir owned by the parent that set the environment up, without hand-building the /mnt/... path.

---
 tools/testing/selftests/bpf/cgroup_helpers.c | 67 ++++++++++++++++++++
 tools/testing/selftests/bpf/cgroup_helpers.h |  4 ++
 2 files changed, 71 insertions(+)
tools/testing/selftests/bpf/cgroup_helpers.c +67 -0
index 45cd0b479fe3..4183ff6150c2 100644
@@ -188,6 +188,73 @@ int write_cgroup_file_parent(const char *relative_path, const char *file,
 	return __write_cgroup_file(cgroup_path, file, buf);
 }
 
+static int __read_cgroup_file(const char *cgroup_path, const char *file,
+			      char *buf, size_t len)
+{
+	char file_path[PATH_MAX + 1];
+	ssize_t got;
+	int fd;
+
+	snprintf(file_path, sizeof(file_path), "%s/%s", cgroup_path, file);
+	fd = open(file_path, O_RDONLY);
+	if (fd < 0) {
+		log_err("Opening %s", file_path);
+		return 1;
+	}
+
+	got = read(fd, buf, len - 1);
+	if (got < 0) {
+		log_err("Reading %s", file_path);
+		close(fd);
+		return 1;
+	}
+	buf[got] = '\0';
+	close(fd);
+	return 0;
+}
+
+/**
+ * read_cgroup_file() - Read from a cgroup file
+ * @relative_path: The cgroup path, relative to the workdir
+ * @file: The name of the file in cgroupfs to read from
+ * @buf: Buffer to read into, NUL-terminated on success
+ * @len: Size of @buf
+ *
+ * Read from a file in the given cgroup's directory.
+ *
+ * If successful, 0 is returned.
+ */
+int read_cgroup_file(const char *relative_path, const char *file,
+		     char *buf, size_t len)
+{
+	char cgroup_path[PATH_MAX - 24];
+
+	format_cgroup_path(cgroup_path, relative_path);
+	return __read_cgroup_file(cgroup_path, file, buf, len);
+}
+
+/**
+ * read_cgroup_file_parent() - Read from a cgroup file in the parent process
+ *                             workdir
+ * @relative_path: The cgroup path, relative to the parent process workdir
+ * @file: The name of the file in cgroupfs to read from
+ * @buf: Buffer to read into, NUL-terminated on success
+ * @len: Size of @buf
+ *
+ * Read from a file in the given cgroup's directory under the parent process
+ * workdir.
+ *
+ * If successful, 0 is returned.
+ */
+int read_cgroup_file_parent(const char *relative_path, const char *file,
+			    char *buf, size_t len)
+{
+	char cgroup_path[PATH_MAX - 24];
+
+	format_parent_cgroup_path(cgroup_path, relative_path);
+	return __read_cgroup_file(cgroup_path, file, buf, len);
+}
+
 /**
  * setup_cgroup_environment() - Setup the cgroup environment
  *
tools/testing/selftests/bpf/cgroup_helpers.h +4 -0
index 3857304be874..d42d2e13044e 100644
@@ -15,6 +15,10 @@ int write_cgroup_file(const char *relative_path, const char *file,
 		      const char *buf);
 int write_cgroup_file_parent(const char *relative_path, const char *file,
 			     const char *buf);
+int read_cgroup_file(const char *relative_path, const char *file,
+		     char *buf, size_t len);
+int read_cgroup_file_parent(const char *relative_path, const char *file,
+			    char *buf, size_t len);
 int cgroup_setup_and_join(const char *relative_path);
 int get_root_cgroup(void);
 int create_and_get_cgroup(const char *relative_path);
Signature · 1 line
2.43.0
#5 lore raw

PATCH v5 4/4 selftests/bpf: Add a test for arena fault-in under memory.max

In reply to Jiayuan Chen

A child joins a memcg capped 64M above its post-load usage and faults an arena in until it runs out of that budget.

With the fix the arena page comes from the sleepable allocator, so hitting memory.max goes through the memcg OOM path and the child is OOM-killed, which the test checks via memory.events "oom_kill".

Without the fix the test may still pass, because a concurrent blocking allocation in the child (e.g. a COW fault on an inherited page) can hit memory.max and OOM-kill it first. The goal is only that the fixed kernel passes reliably.

  # test_progs -v -t arena_memcg
  serial_test_arena_memcg:PASS:child killed by signal
  serial_test_arena_memcg:PASS:memcg oom_kill
  #5       arena_memcg:OK
  # dmesg (the OOM comes from the arena sleepable allocation)
  test_progs invoked oom-killer: gfp_mask=GFP_KERNEL_ACCOUNT|__GFP_ZERO
   arena_vm_fault+0x4bc/0xad0
  Memory cgroup out of memory: Killed process 473 (test_progs)
---
 .../selftests/bpf/prog_tests/arena_memcg.c    | 158 ++++++++++++++++++
 .../testing/selftests/bpf/progs/arena_memcg.c |  24 +++
 2 files changed, 182 insertions(+)
 create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_memcg.c
 create mode 100644 tools/testing/selftests/bpf/progs/arena_memcg.c
tools/testing/selftests/bpf/prog_tests/arena_memcg.c new file +158 -0
new file mode 100644
index 000000000000..c57b98494cfa
@@ -0,0 +1,158 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>
+#include <fcntl.h>
+#include <signal.h>
+#include <sys/mman.h>
+#include <sys/wait.h>
+#include <unistd.h>
+#include <sys/user.h>
+#ifndef PAGE_SIZE /* on some archs it comes in sys/user.h */
+#include <unistd.h>
+#define PAGE_SIZE getpagesize()
+#endif
+
+#include "cgroup_helpers.h"
+#include "arena_memcg.skel.h"
+
+#define CG_PATH		"/arena_memcg"
+
+/* Budget the arena gets on top of whatever is already charged after load. */
+#define ARENA_BUDGET	(64 * 1024 * 1024)
+
+static void dump_memcg(int (*rd)(const char *, const char *, char *, size_t))
+{
+	char buf[512];
+
+	/*
+	 * memory.current reads 0 once the child has left the cgroup, so it only
+	 * carries information when dumped from the live child; memory.peak and
+	 * memory.events survive the child and tell the story either way.
+	 */
+	if (!rd(CG_PATH, "memory.current", buf, sizeof(buf)))
+		fprintf(stderr, "memory.current: %s", buf);
+	if (!rd(CG_PATH, "memory.max", buf, sizeof(buf)))
+		fprintf(stderr, "memory.max: %s", buf);
+	if (!rd(CG_PATH, "memory.peak", buf, sizeof(buf)))
+		fprintf(stderr, "memory.peak: %s", buf);
+	if (!rd(CG_PATH, "memory.events", buf, sizeof(buf)))
+		fprintf(stderr, "memory.events:\n%s", buf);
+	fflush(stderr);
+}
+
+/* Read one key from a flat keyed cgroup file, e.g. "oom_kill" in memory.events. */
+static long cg_read_key(const char *cg, const char *file, const char *key)
+{
+	char buf[512], *p;
+
+	if (read_cgroup_file(cg, file, buf, sizeof(buf)))
+		return -1;
+	p = strstr(buf, key);
+	if (!p)
+		return -1;
+	return strtol(p + strlen(key), NULL, 10);
+}
+
+void serial_test_arena_memcg(void)
+{
+	int cgroup_fd = -1, status, err;
+	const long ps = PAGE_SIZE;
+	char buf[64];
+	pid_t pid;
+
+	err = setup_cgroup_environment();
+	if (!ASSERT_OK(err, "setup_cgroup_environment"))
+		return;
+
+	cgroup_fd = create_and_get_cgroup(CG_PATH);
+	if (!ASSERT_OK_FD(cgroup_fd, "create_and_get_cgroup"))
+		goto out;
+
+	/* No memory controller -> nothing to test. */
+	if (read_cgroup_file(CG_PATH, "memory.current", buf, sizeof(buf))) {
+		fprintf(stderr, "%s:SKIP:no memory controller\n", __func__);
+		test__skip();
+		goto out;
+	}
+
+	pid = fork();
+	if (!ASSERT_GE(pid, 0, "fork"))
+		goto out;
+	if (pid == 0) {
+		struct arena_memcg *cskel;
+		__u32 i, npages;
+		char *base;
+		size_t sz;
+		long cur;
+
+		/*
+		 * Do everything from the child: the arena vma is VM_DONTCOPY so
+		 * it would not survive fork(), only the child should be under the
+		 * limit so that a memcg OOM cannot pick test_progs, and a map is
+		 * charged to the memcg of the task that creates it - so join
+		 * before load. The cgroup work dir belongs to the parent that set
+		 * the environment up, so reach it with the _parent() helpers.
+		 * Errors are reported to the parent through the exit code, since
+		 * ASSERT_* in a forked child does not reach it.
+		 */
+		snprintf(buf, sizeof(buf), "%d", getpid());
+		if (write_cgroup_file_parent(CG_PATH, "cgroup.procs", buf))
+			_exit(2);
+
+		cskel = arena_memcg__open_and_load();
+		if (!cskel)
+			_exit(3);
+
+		base = bpf_map__initial_value(cskel->maps.arena, &sz);
+		if (!base)
+			_exit(4);
+		npages = bpf_map__max_entries(cskel->maps.arena);
+
+		/*
+		 * Cap only now, after load: everything but the fault-in is
+		 * charged, so the arena gets a fixed budget regardless of what
+		 * the load itself cost, and the load can never hit the limit.
+		 */
+		if (read_cgroup_file_parent(CG_PATH, "memory.current", buf, sizeof(buf)))
+			_exit(5);
+		cur = strtol(buf, NULL, 10);
+		snprintf(buf, sizeof(buf), "%ld", cur + ARENA_BUDGET);
+		if (write_cgroup_file_parent(CG_PATH, "memory.max", buf))
+			_exit(6);
+
+		for (i = 0; i < npages; i++)
+			base[(size_t)i * ps] = 1;
+		/* Faulted everything without dying: dump why (only under -v). */
+		dump_memcg(read_cgroup_file_parent);
+		_exit(0);
+	}
+
+	if (!ASSERT_EQ(waitpid(pid, &status, 0), pid, "waitpid"))
+		goto out;
+
+	/* A non-zero exit means the child failed to set up; the code says where. */
+	if (WIFEXITED(status) && WEXITSTATUS(status)) {
+		ASSERT_OK(WEXITSTATUS(status), "child setup");
+		goto out;
+	}
+
+	/*
+	 * Faulting a valid arena address until memory.max is hit must not look
+	 * like an invalid access. Without the fix the fault path allocated with
+	 * the non-blocking allocator, turned its -ENOMEM into VM_FAULT_SIGSEGV,
+	 * and the child died with SIGSEGV on a valid address; now it is handled
+	 * by the memcg OOM path instead. A SIGKILL alone would not prove the
+	 * memcg OOM killer did it (a global OOM or an unrelated crash could also
+	 * kill the child), so check memory.events.oom_kill, which records the
+	 * memcg OOM and survives the child.
+	 */
+	if (!ASSERT_TRUE(WIFSIGNALED(status), "child killed by signal"))
+		goto out;
+	if (!ASSERT_GE(cg_read_key(CG_PATH, "memory.events", "oom_kill"), 1,
+		       "memcg oom_kill"))
+		dump_memcg(read_cgroup_file);
+out:
+	if (cgroup_fd >= 0)
+		close(cgroup_fd);
+	cleanup_cgroup_environment();
+}
tools/testing/selftests/bpf/progs/arena_memcg.c new file +24 -0
new file mode 100644
index 000000000000..88259cfea06b
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_arena_common.h"
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARENA);
+	__uint(map_flags, BPF_F_MMAPABLE);
+	__uint(max_entries, 50000); /* number of pages */
+#ifdef __TARGET_ARCH_arm64
+	__ulong(map_extra, 0x1ull << 32); /* start of mmap() region */
+#else
+	__ulong(map_extra, 0x1ull << 44); /* start of mmap() region */
+#endif
+} arena SEC(".maps");
+
+SEC("syscall")
+int noop(void *ctx)
+{
+	return 0;
+}
+
+char _license[] SEC("license") = "GPL";
Signature · 1 line
2.43.0
#6 lore raw

Re: [PATCH bpf-next v5 4/4] selftests/bpf: Add a test for arena fault-in under memory.max

This isn't a bug, but are fcntl.h, signal.h and sys/mman.h needed here? Nothing in the file appears to use open()/O_* flags, signal APIs (SIGSEGV and SIGKILL appear only in comments, and WIFSIGNALED…
Quoted 9 lines
tools/testing/selftests/bpf/prog_tests/arena_memcg.c new file +3 -0
new file mode 100644
index 000000000000..c57b98494cfa
@@ -0,0 +1,158 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <test_progs.h>

tools/testing/selftests/bpf/prog_tests/arena_memcg.c line 13

Quoted 10 lines
tools/testing/selftests/bpf/prog_tests/arena_memcg.c new file +10 -0
+#include <fcntl.h>
+#include <signal.h>
+#include <sys/mman.h>
+#include <sys/wait.h>
+#include <unistd.h>
+#include <sys/user.h>
+#ifndef PAGE_SIZE /* on some archs it comes in sys/user.h */
+#include <unistd.h>
+#define PAGE_SIZE getpagesize()
+#endif

This isn't a bug, but are fcntl.h, signal.h and sys/mman.h needed here? Nothing in the file appears to use open()/O_* flags, signal APIs (SIGSEGV and SIGKILL appear only in comments, and WIFSIGNALED comes from sys/wait.h), or mmap()/PROT_*/MAP_* constants.

Also, since unistd.h is already included above, could the copy inside the PAGE_SIZE block be dropped?

Quoted 3 lines
tools/testing/selftests/bpf/prog_tests/arena_memcg.c new file +3 -0
+
+#include "cgroup_helpers.h"
+#include "arena_memcg.skel.h"

[ ... ]

tools/testing/selftests/bpf/prog_tests/arena_memcg.c line 57

Quoted 2 lines
tools/testing/selftests/bpf/prog_tests/arena_memcg.c new file +2 -0
+void serial_test_arena_memcg(void)
+{

A subsystem pattern flags this as potentially concerning: a new standalone test file is added to prog_tests/ for user-space arena fault-in, while prog_tests/arena_mem_usage.c in the same directory already exercises that same interface. arena_mem_usage.c's 'User-space fault-in' block does exactly the same thing this test's inner loop does:

  base = bpf_map__initial_value(skel->maps.arena, &sz);
  ...
  for (i = 0; i < 8; i++)
          base[i * ps] = 1;

Both files are serial_ tests, both duplicate the same PAGE_SIZE preamble, the same page-size local, and the same bpf_map__initial_value() / base[i * ps] = 1 fault-in idiom, and progs/arena_memcg.c's map stanza is byte-identical to progs/arena_mem_usage.c's apart from max_entries. The question is whether this belongs as an additional case or subtest in the existing arena fault-in test rather than a new binary-level test with its own setup.

On the other hand, the two tests assert different behaviour, not a narrow variant of one behaviour: arena_mem_usage.c asserts the memlock accounting value after fault-in, while this test asserts that hitting memory.max during fault-in produces a memcg OOM kill rather than a SIGSEGV on a valid address. The new test also needs a harness the existing one does not have:
setup_cgroup_environment(), a cgroup created and joined before skeleton load so the map is charged to the right memcg, a forked child (the arena vma is VM_DONTCOPY and the parent must stay outside the limit so the memcg OOM cannot pick test_progs), and exit-code-based error reporting because ASSERT_* does not cross fork(). Folding that into arena_mem_usage.c would put every one of its existing assertions behind a fork and a memory limit.

Is the separate file the right call here, or should this be integrated into the existing test?

[ ... ]

Quoted 18 lines
tools/testing/selftests/bpf/progs/arena_memcg.c new file +12 -0
new file mode 100644
index 000000000000..88259cfea06b
@@ -0,0 +1,24 @@
+// SPDX-License-Identifier: GPL-2.0
+
+#include <vmlinux.h>
+#include <bpf/bpf_helpers.h>
+#include "bpf_arena_common.h"
+
+struct {
+	__uint(type, BPF_MAP_TYPE_ARENA);
+	__uint(map_flags, BPF_F_MMAPABLE);
+	__uint(max_entries, 50000); /* number of pages */
+#ifdef __TARGET_ARCH_arm64
+	__ulong(map_extra, 0x1ull << 32); /* start of mmap() region */

tools/testing/selftests/bpf/progs/arena_memcg.c line 22

Quoted 10 lines
tools/testing/selftests/bpf/progs/arena_memcg.c new file +10 -0
+#else
+	__ulong(map_extra, 0x1ull << 44); /* start of mmap() region */
+#endif
+} arena SEC(".maps");
+
+SEC("syscall")
+int noop(void *ctx)
+{
+	return 0;
+}

This isn't a bug, but is the noop program load-bearing here - does the object need at least one program for the skeleton/load path - or could progs/arena_memcg.c be just the arena map definition? The test only calls arena_memcg__open_and_load() and then works through bpf_map__initial_value() / bpf_map__max_entries() on maps.arena, so cskel->progs.noop is never referenced.

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32713810092

#3 lore raw

Re: [PATCH bpf-next v5 2/4] bpf: arena: allocate the fault-in page outside the lock

In reply to Jiayuan Chen
This isn't a bug, but could this comment spell out why a scratch-marked hole is not lazy-allocated even though an untouched hole is? A reader might wonder why the two cases are treated differently…
Quoted 4 lines
kernel/bpf/arena.c +0 -0
index 7b6847200b43..fa462a0ff1ad 100644

[ ... ]

Quoted 22 lines
kernel/bpf/arena.c +18 -0
@@ -489,59 +490,106 @@ static vm_fault_t arena_vm_fault(struct vm_fault *vmf)
 	kbase = bpf_arena_get_kern_vm_start(arena);
 	kaddr = kbase + (u32)(vmf->address);
 
+	page = vmalloc_to_page((void *)kaddr);
+	if (!page && !(arena->map.map_flags & BPF_F_SEGV_ON_FAULT)) {
+		/*
+		 * Preallocate outside the lock with a sleepable allocator so it
+		 * can reclaim and run the memcg OOM killer, which the
+		 * non-blocking allocator under arena->spinlock cannot. A NULL
+		 * return is non-recoverable, so fail with VM_FAULT_SIGBUS;
+		 * VM_FAULT_OOM would be retried by the fault path and can
+		 * livelock when the charged memcg is not the faulting task's.
+		 */
+		bpf_map_memcg_enter(&arena->map, &old_memcg, &new_memcg);
+		new_page = bpf_map_alloc_page_sleepable(map);
+		bpf_map_memcg_exit(old_memcg, new_memcg);
+		if (!new_page)
+			return VM_FAULT_SIGBUS;
+	}
+
+	if (raw_res_spin_lock_irqsave(&arena->spinlock, flags)) {

[ ... ]

kernel/bpf/arena.c static vm_fault_t arena_vm_fault(struct vm_fault *vmf) · line 532

Quoted 12 lines
kernel/bpf/arena.c +10 -0
 	page = vmalloc_to_page((void *)kaddr);
 	if (page) {
+		if (page == arena->scratch_page) {
+			/*
+			 * A scratch page marks a hole. Segfault only if the user
+			 * asked for it; otherwise we could lazy-allocate but
+			 * choose not to over a hole, so report a bus error.
+			 */
+			fault_ret = (arena->map.map_flags & BPF_F_SEGV_ON_FAULT) ?
+				    VM_FAULT_SIGSEGV : VM_FAULT_SIGBUS;
+			goto out_err_locked;
+		}

This isn't a bug, but could this comment spell out why a scratch-marked hole is not lazy-allocated even though an untouched hole is? A reader might wonder why the two cases are treated differently since both are equally free as far as the range tree is concerned.

Quoted 11 lines
kernel/bpf/arena.c +6 -0
 		/* already have a page vmap-ed */
 		goto out;
 	}
 
+	if (arena->map.map_flags & BPF_F_SEGV_ON_FAULT) {
+		/* User space requested to segfault when page is not allocated by bpf prog */
+		fault_ret = VM_FAULT_SIGSEGV;
+		goto out_err_locked;
+	}
+
 	bpf_map_memcg_enter(&arena->map, &old_memcg, &new_memcg);

kernel/bpf/arena.c static vm_fault_t arena_vm_fault(struct vm_fault *vmf) · line 553

Quoted 10 lines
kernel/bpf/arena.c +9 -0
 
+	if (!new_page) {
+		/*
+		 * Very rare race: the bpf program had allocated a page here, so
+		 * the lockless probe saw it and we skipped preallocation, but it
+		 * freed the page before we took the lock. Now we do need one;
+		 * sleeping is not allowed here, so fall back to the non-blocking
+		 * allocator and give up if it fails.
+		 */
+		ret = bpf_map_alloc_pages(map, map->numa_node, 1, &new_page);

This isn't a bug, but would it be clearer to describe the condition generically (the lockless probe's result was invalidated before the lock was taken) rather than naming only the bpf-alloc-then-free case? The fallback is also reached if the probe observed arena->scratch_page and the PTE was cleared by arena_free_pages() before the lock was taken, which is not a bpf program page allocation.

[ ... ]

---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md

CI run summary: https://github.com/kernel-patches/bpf/actions/runs/32713810092