bpf: arena: handle memory.max on fault-in with reclaim
Status from this message: This series was applied to bpf/bpf-next.git (master)
Get series with b4
b4 am [email protected]Versions newest first · linked by author and title
- v9 Latest3 patches 0 repliesApplied 3/3
- v83 patches 1 replySuperseded
- v74 patches 1 replySuperseded
Earlier versions · 5
| Version | Posted | Patches | Replies | Status |
|---|---|---|---|---|
| v5 | 4 patches | 0 replies | Superseded | |
| v4 | 4 patches | 6 replies | Superseded | |
| v3 | 4 patches | 3 replies | Superseded | |
| v2 | 4 patches | 2 replies | Superseded | |
| v1 | 3 patches | 8 replies | Superseded |
Changes in v9 from the cover letter
- Trim the two comments in arena_vm_fault() to a sentence or two; the details are in the commit log. (Alexei Starovoitov)
Patches in v9 All 3 applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/3 | bpf: arena: allocate the fault-in page outside the lockkernel/ | +63 | 0 replies | Applied |
| 2/3 | selftests/bpf: Add read_cgroup_file() to cgroup_helperstools/ tools/ | +71 | 0 replies | Applied |
| 3/3 | selftests/bpf: Add a test for arena fault-in under memory.maxtools/ tools/ | +219 | 0 replies | Applied |
Reviews trailers per patch
All 3 patches have a Reviewed-by or Acked-by.
| Patch | Emil Tsalapatis |
|---|---|
| 1/3 | R-b |
| 2/3 | R-b |
| 3/3 | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.
Cover letter [PATCH bpf-next v9 0/3] bpf: arena: handle memory.max on fault-in with reclaim
Since commit e66fe1bc6d25 ("bpf: arena: Reintroduce memcg accounting"), arena pages are charged to the memcg of the process that created the arena. That exposes a problem in the arena user page fault path: the fault-in allocation runs under arena->spinlock, so it can only use the non-blocking allocator, which never reclaims. Once memory.current is at memory.max the allocation simply fails, even when the memcg is full of page cache that could be dropped right away. Reaching memory.max is completely normal for a healthy application - e.g. reading a large file fills memory.current with page cache - and the process then gets SIGSEGV on a perfectly valid arena address.
Preallocate the page outside the lock (patch 1), the way do_anonymous_page() does, so the allocation can sleep and reclaim. It never invokes the OOM
killer: the page is charged to the map's memcg, which need not be the faulting task's, so an OOM there could kill unrelated tasks in the map's cgroup. On a genuine failure the fault returns VM_FAULT_SIGBUS. patch 2&3 adds a selftest that fills a memcg with reclaimable page cache and faults an arena in under memory.max: without the fix the child gets SIGSEGV on a valid address, with it the fault-in reclaims and succeeds.
v8 -> v9:
- Trim the two comments in arena_vm_fault() to a sentence or two; the details are in the commit log. (Alexei Starovoitov)
v7 -> v8:
- Merge patch 1 into patch 2. (Alexei Starovoitov)
v4 -> v7:
- Simplify the implementation: do not aim for OOM anymore. An arena is shared between processes and can be shared across cgroups, so the OOM killer is the wrong tool here - it would act on the map's memcg, which need not be the faulting task's. Only try to reclaim now, via __GFP_RETRY_MAYFAIL. Everything else is kept as before.
- selftest: changed accordingly - fill the memcg with reclaimable page cache and check that the arena fault-in succeeds by reclaiming it, instead of relying on an OOM kill.
v3 -> v4:
- rebase bpf-next and fix conflict
- add Reviewed-by tag from Emil Tsalapatis
v2 -> v3:
- selftest: check the memcg OOM via memory.events "oom_kill" instead of the exit signal; it only aims to pass on the fixed kernel, since the unfixed SIGSEGV is racy.
v1 -> v2:
- Rebase on the separate deadlock fix (found by the Sashiko AI review), now applied to bpf-next.
- Honor the map's NUMA node on fault-in.
- Return VM_FAULT_SIGBUS for the non-recoverable faults (lock, range-tree and page-table failures); a scratch-page hole stays VM_FAULT_SIGSEGV only under BPF_F_SEGV_ON_FAULT. (Kumar Kartikeya Dwivedi)
- Add read_cgroup_file() to cgroup_helpers instead of open-coding the /mnt/... path in the test. (Emil Tsalapatis)
- Dump the cgroup memory stats on test failure to ease debugging.
v8:
https://lore.kernel.org/bpf/[email protected]/
v7:
https://lore.kernel.org/bpf/[email protected]/#r
v4:
https://lore.kernel.org/bpf/[email protected]/T/#t
v2:
https://lore.kernel.org/bpf/[email protected]/
v1:
https://lore.kernel.org/bpf/[email protected]/
Jiayuan Chen (3):
bpf: arena: allocate the fault-in page outside the lock selftests/bpf: Add read_cgroup_file() to cgroup_helpers selftests/bpf: Add a test for arena fault-in under memory.max
kernel/bpf/arena.c | 83 ++++++-- tools/testing/selftests/bpf/cgroup_helpers.c | 67 ++++++ tools/testing/selftests/bpf/cgroup_helpers.h | 4 + .../selftests/bpf/prog_tests/arena_memcg.c | 196 ++++++++++++++++++ .../testing/selftests/bpf/progs/arena_memcg.c | 23 ++ 5 files changed, 353 insertions(+), 20 deletions(-) create mode 100644 tools/testing/selftests/bpf/prog_tests/arena_memcg.c create mode 100644 tools/testing/selftests/bpf/progs/arena_memcg.c
Signature · 1 line
2.43.0