Skip to content

mm/page_owner: add bpf_iter and kfuncs

Zhen Ni bpf linux-kselftest linux-mm lkml 3 versions – In review

Get series with b4
b4 am [email protected]

Versions newest first · linked by author and title

  1. v3 Latest7 patches 9 repliesIn review
  2. v26 patches 5 repliesSuperseded
  3. v16 patches 6 repliesSuperseded

Changes in v3 from the cover letter

=============
- Add a mem-hotplug read lock across the page scan.
- Drop the live struct page from the scan cursor and the bpf_iter context.
- Check the page_owner_inited static key in the iterator.
- Without scan->page, start at min_low_pfn - 1 and let next() pre-increment.
- Gate the kfuncs with KF_PERFMON.
- Enable BPF_ITER_RESCHED for the bpf_iter target.
- Check ENODATA/EOPNOTSUPP from <errno.h>.

Patches in v3 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/7mm/page_owner: extract page_owner_next_eligible() from read loopmm/page_owner.c+64 -370 replies
2/7mm/page_owner: protect page access with mem-hotplug read lockmm/page_owner.c+30 -50 replies
3/7mm/page_owner: add bpf_iter target "page_owner"mm/Makefile mm/page_owner.c mm/page_owner.h +1 more+175 -250 replies
4/7mm/page_owner: add open-coded page_owner iteratorkernel/bpf/helpers.c mm/page_owner.h mm/page_owner_iter.c+76 -00 replies
5/7mm/page_owner: record the NUMA node in the scan cursormm/page_owner.c mm/page_owner.h mm/page_owner_iter.c+21 -130 replies
6/7mm/page_owner: add bpf_page_owner_get_memcg_info() kfunckernel/bpf/helpers.c mm/page_owner.c mm/page_owner.h +1 more+124 -190 replies
7/7selftests/bpf: add tests for the page_owner bpf_iter targettools/testing/selftests/bpf/config tools/testing/selftests/bpf/prog_tests/page_owner_iter.c tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c+352 -00 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v3 yet.

Read the v3 thread

Cover letter [PATCH v3 0/7] mm/page_owner: add bpf_iter and kfuncs

Hi all,

Introduction
============

page_owner records who allocated every page and is a core memory-debug facility, but it currently reaches userspace as the debugfs flat dump:
slicing allocations by pid, comm, NUMA node, cgroup or stack, etc., means streaming and parsing the entire file in userspace, which is expensive on large hosts and hard to query live.

Exposing the same walk to eBPF lets a program filter and aggregate in the kernel and surface only the records it cares about. This series adds that access as a seq_file bpf_iter target and an open-coded iterator.

Implementation
==============

The page walk is factored into page_owner_next_eligible(), shared by the read path and the iterators so eligibility stays identical. Two kfuncs expose information that a BPF program cannot decode from the snapshot alone: per-page memcg info including the full cgroup path, and the stack trace behind a stack_depot handle. Each record is snapshotted under the page_ext RCU lock, so consumers read a consistent copy.

Testing
=======

The selftest exercises the seq_file and open-coded iterators, and validates the stack_snprint and get_memcg_info kfuncs.

Changes in v3
=============
- Add a mem-hotplug read lock across the page scan.
- Drop the live struct page from the scan cursor and the bpf_iter context.
- Check the page_owner_inited static key in the iterator.
- Without scan->page, start at min_low_pfn - 1 and let next() pre-increment.
- Gate the kfuncs with KF_PERFMON.
- Enable BPF_ITER_RESCHED for the bpf_iter target.
- Check ENODATA/EOPNOTSUPP from <errno.h>.

Changes in v2
=============
- Route the zero-handle case through ext_put_continue.
- Drop the scan->pfn++.
- Fill page/po/pfn only when v is non-NULL.
- Implement page_owner_seq_stop().
- bpf_page_owner_stack_snprint() now takes the scan cursor.
- Buffer args renamed to buf__uninit/buf__sz.
- Removes the v1 get_nid() kfunc. The node is now stored in scan->nid.
- bpf_page_owner_get_memcg_info() now resolves the page with pfn_to_online_page() for an offline or removed pfn.
- Buffer/info args renamed to info__uninit/buf__uninit/buf__sz.
- Drop the get_nid kfunc coverage.
- Enable CONFIG_PAGE_OWNER in the bpf selftests config.

v2: https://lore.kernel.org/linux-mm/[email protected]/
v1: https://lore.kernel.org/linux-mm/[email protected]/

Zhen Ni (7):
mm/page_owner: extract page_owner_next_eligible() from read loop mm/page_owner: protect page access with mem-hotplug read lock mm/page_owner: add bpf_iter target "page_owner" mm/page_owner: add open-coded page_owner iterator mm/page_owner: record the NUMA node in the scan cursor mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc selftests/bpf: add tests for the page_owner bpf_iter target

 kernel/bpf/helpers.c                          |  10 +
 mm/Makefile                                   |   3 +
 mm/page_owner.c                               | 252 ++++++++++++------
 mm/page_owner.h                               |  60 +++++
 mm/page_owner_iter.c                          | 232 ++++++++++++++++
 tools/testing/selftests/bpf/config            |   2 +
 .../bpf/prog_tests/page_owner_iter.c          | 193 ++++++++++++++
 .../selftests/bpf/progs/bpf_iter_page_owner.c | 157 +++++++++++
 8 files changed, 826 insertions(+), 83 deletions(-)
 create mode 100644 mm/page_owner.h
 create mode 100644 mm/page_owner_iter.c
 create mode 100644 tools/testing/selftests/bpf/prog_tests/page_owner_iter.c
 create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c
Signature · 1 line
2.20.1