mm/page_owner: add bpf_iter and kfuncs
Subsystems (5)
v2 cover mm/page_owner: add bpf_iter and kfuncs
Hi all,
Introduction
============
page_owner records who allocated every page and is a core memory-debug facility, but it currently reaches userspace as the debugfs flat dump:
slicing allocations by pid, comm, NUMA node, cgroup or stack, etc., means streaming and parsing the entire file in userspace, which is expensive on large hosts and hard to query live.
Exposing the same walk to eBPF lets a program filter and aggregate in the kernel and surface only the records it cares about. This series adds that access as a seq_file bpf_iter target and an open-coded iterator.
Implementation
==============
The page walk is factored into page_owner_next_eligible(), shared by the read path and the iterators so eligibility stays identical. Two kfuncs expose information that a BPF program cannot decode from the snapshot alone: per-page memcg info including the full cgroup path, and the stack trace behind a stack_depot handle. Each record is snapshotted under the page_ext RCU lock, so consumers read a consistent copy.
Testing
=======
The selftest exercises the seq_file and open-coded iterators, and validates the stack_snprint and get_memcg_info kfuncs.
Changes in v2
=============
- Route the zero-handle case through ext_put_continue.
- Drop the scan->pfn++.
- Fill page/po/pfn only when v is non-NULL.
- Implement page_owner_seq_stop().
- bpf_page_owner_stack_snprint() now takes the scan cursor.
- Buffer args renamed to buf__uninit/buf__sz.
- Removes the v1 get_nid() kfunc. The node is now stored in scan->nid.
- bpf_page_owner_get_memcg_info() now resolves the page with pfn_to_online_page() for an offline or removed pfn.
- Buffer/info args renamed to info__uninit/buf__uninit/buf__sz.
- Drop the get_nid kfunc coverage.
- Enable CONFIG_PAGE_OWNER in the bpf selftests config.
v1: https://lore.kernel.org/linux-mm/[email protected]/
Zhen Ni (6):
mm/page_owner: extract page_owner_next_eligible() from read loop mm/page_owner: add bpf_iter target "page_owner" mm/page_owner: add open-coded page_owner iterator mm/page_owner: record the NUMA node in the scan cursor mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc selftests/bpf: add tests for the page_owner bpf_iter target
kernel/bpf/helpers.c | 7 + mm/Makefile | 3 + mm/page_owner.c | 219 +++++++++++------- mm/page_owner.h | 58 +++++ mm/page_owner_iter.c | 197 ++++++++++++++++ tools/testing/selftests/bpf/config | 2 + .../bpf/prog_tests/page_owner_iter.c | 193 +++++++++++++++ .../selftests/bpf/progs/bpf_iter_page_owner.c | 159 +++++++++++++ 8 files changed, 758 insertions(+), 80 deletions(-) create mode 100644 mm/page_owner.h create mode 100644 mm/page_owner_iter.c create mode 100644 tools/testing/selftests/bpf/prog_tests/page_owner_iter.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c
Signature · 1 line
2.20.1
PATCH v2 1/6 mm/page_owner: extract page_owner_next_eligible() from read loop
Move the page scanning logic out of read_page_owner() into a dedicated helper, in preparation for using it in the page_owner bpf_iter target.
The helper advances a struct page_owner_scan cursor to the next page eligible for page_owner output and returns true on a hit. On a hit it takes a snapshot of the page_owner record inside the page_ext RCU window.
No change to the output is intended. handle is read from the snapshot, taken before the non-zero check, so the validated and reported handle are the same value.
---
Changes in v2:
- Route the zero-handle case through ext_put_continue.
v1: https://lore.kernel.org/linux-mm/[email protected]/
--- mm/page_owner.c | 103 ++++++++++++++++++++++++++++++------------------ 1 file changed, 65 insertions(+), 38 deletions(-)
mm/page_owner.c +65 -38
@@ -36,6 +36,16 @@ struct page_owner {pid_t free_tgid; }; +/* +* Cursor and per-page result of page_owner_next_eligible(). +*/ +struct page_owner_scan { +/* resume point */ +unsigned long pfn; +struct page *page; +struct page_owner po_snap; +}; +struct stack {struct stack_record *stack_record;struct stack *next;
@@ -725,37 +735,24 @@ void __dump_page_owner(const struct page *page)page_ext_put (page_ext); } -static ssize_t -read_page_owner (struct file *file,char __user *buf,size_t count,loff_t *ppos) +/* +* Advance @scan to the next page eligible for page_owner output. +* On a hit, fill scan->pfn/page/po_snap and return true; the caller +* must advance scan->pfn past the hit before calling again to resume +* the scan. +*/ +static bool page_owner_next_eligible (struct page_owner_scan *scan) { -unsigned long pfn; -struct page *page; -struct page_ext *page_ext; -struct page_owner *page_owner; -depot_stack_handle_t handle; -struct page_owner_filter_state *state = file->private_data; +unsigned long pfn = scan->pfn; -if (!static_branch_unlikely (&page_owner_inited)) -return -EINVAL ; - - page =NULL ; -if (*ppos ==0 ) - pfn = min_low_pfn; -else - pfn = *ppos;/* Find a valid PFN or the start of a MAX_ORDER_NR_PAGES area */ while (!pfn_valid (pfn) && (pfn & (MAX_ORDER_NR_PAGES -1 )) !=0 ) pfn++; -/* Find an allocated page */ for (; pfn < max_pfn; pfn++) { -/* -* This temporary page_owner is required so -* that we can avoid the context switches while holding -* the rcu lock and copying the page owner information to -* user through copy_to_user() or GFP_KERNEL allocations. -*/ -struct page_owner page_owner_tmp; +struct page_owner *page_owner; +struct page_ext *page_ext; +struct page *page;/* * If the new page is in a new MAX_ORDER_NR_PAGES area,
@@ -798,37 +795,67 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos) goto ext_put_continue;/* -* Access to page_ext->handle isn't synchronous so we should -* be careful to access it. +* Take a snapshot of the page_owner record inside the RCU +* window, then validate it, so that print_page_owner() +* reads the checked handle from the same snapshot. */ - handle =READ_ONCE (page_owner->handle); -if (!handle) + scan->pfn = pfn; + scan->page = page; + scan->po_snap = *page_owner; + +if (!scan->po_snap.handle) goto ext_put_continue; +page_ext_put (page_ext); +return true; + +ext_put_continue: +page_ext_put (page_ext); +cond_resched (); + } + +return false; +} + +static ssize_t +read_page_owner (struct file *file,char __user *buf,size_t count,loff_t *ppos) +{ +struct page_owner_scan scan; +struct page_owner_filter_state *state = file->private_data; + +if (!static_branch_unlikely (&page_owner_inited)) +return -EINVAL ; + +if (*ppos ==0 ) + scan.pfn = min_low_pfn; +else + scan.pfn = *ppos; + +/* Find an allocated page */ +while (page_owner_next_eligible (&scan)) {if (state->nid_filter_enabled) {int nid; -memdesc_flags_t page_flags =READ_ONCE (page->flags); +memdesc_flags_t page_flags = +READ_ONCE (scan.page->flags);/* * Bypass PF_POISONED_CHECK() in page_to_nid() to avoid * VM_BUG_ON when accessing poisoned pages. */ if (page_flags.f ==PAGE_POISON_PATTERN ) - goto ext_put_continue; + goto skip_continue; nid =memdesc_nid (&page_flags);if (!node_isset (nid, state->nid_filter)) - goto ext_put_continue; + goto skip_continue; }/* Record the next PFN to read in the file offset */ - *ppos = pfn +1 ; + *ppos = scan.pfn +1 ; - page_owner_tmp = *page_owner; -page_ext_put (page_ext); -return print_page_owner (buf, count, pfn, page, - &page_owner_tmp, handle, state); -ext_put_continue: -page_ext_put (page_ext); +return print_page_owner (buf, count, scan.pfn, scan.page, + &scan.po_snap, scan.po_snap.handle, state); +skip_continue: + scan.pfn++;cond_resched (); }
Signature · 1 line
2.20.1
PATCH v2 2/6 mm/page_owner: add bpf_iter target "page_owner"
In reply to Zhen NiRegister a bpf_iter target "page_owner" that walks the pages eligible for page_owner output, with the same eligibility rules as the /sys/kernel/debug/page_owner read path.
For each eligible page the attached BPF program receives the pfn, the page and the page_owner record, and can freely filter and format output.
The po_snap record in the scan cursor is taken inside the page_ext RCU window and handed to the BPF program as a copy, so no RCU window is needed at consumption time. Torn reads remain possible if the record is concurrently updated while it is copied, but this is the same best-effort behavior as the read path has always had.
The target is registered from pageowner_init() when both CONFIG_PAGE_OWNER and CONFIG_BPF_SYSCALL are enabled; a registration failure only disables the bpf_iter target.
Userspace tools such as bpftrace can attach to this target like any other bpf_iter. For example, to count pages owned by init (pid 1):
bpftrace -e 'iter:page_owner / ctx->po->pid == 1 / { @ = count(); }'---
Changes in v2:
- Drop the scan->pfn++.
- Fill page/po/pfn only when v is non-NULL.
- Implement page_owner_seq_stop().
v1: https://lore.kernel.org/linux-mm/[email protected]/
--- mm/Makefile | 3 ++ mm/page_owner.c | 33 +++--------- mm/page_owner.h | 48 +++++++++++++++++ mm/page_owner_iter.c | 124 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 182 insertions(+), 26 deletions(-) create mode 100644 mm/page_owner.h create mode 100644 mm/page_owner_iter.c
mm/Makefile +3 -0
@@ -115,6 +115,9 @@ obj-$(CONFIG_DEBUG_KMEMLEAK) += kmemleak.o obj-$(CONFIG_DEBUG_RODATA_TEST) += rodata_test.o obj-$(CONFIG_DEBUG_VM_PGTABLE) += debug_vm_pgtable.o obj-$(CONFIG_PAGE_OWNER) += page_owner.o +ifdef CONFIG_BPF_SYSCALL +obj-$(CONFIG_PAGE_OWNER) += page_owner_iter.o +endif obj-$(CONFIG_MEMORY_ISOLATION) += page_isolation.o obj-$(CONFIG_ZSMALLOC) += zsmalloc.o obj-$(CONFIG_GENERIC_EARLY_IOREMAP) += early_ioremap.o
mm/page_owner.c +7 -26
@@ -14,6 +14,7 @@#include <linux/sched/clock.h> #include "page_alloc.h" +#include "page_owner.h" /* * TODO: teach PAGE_OWNER_STACK_DEPTH (__dump_page_owner and save_stack)
@@ -21,31 +22,6 @@*/ #define PAGE_OWNER_STACK_DEPTH (16) -struct page_owner { -unsigned short order; -short last_migrate_reason; -gfp_t gfp_mask; -depot_stack_handle_t handle; -depot_stack_handle_t free_handle; -u64 ts_nsec; -u64 free_ts_nsec; -char comm[TASK_COMM_LEN ]; -pid_t pid; -pid_t tgid; -pid_t free_pid; -pid_t free_tgid; -}; - -/* -* Cursor and per-page result of page_owner_next_eligible(). -*/ -struct page_owner_scan { -/* resume point */ -unsigned long pfn; -struct page *page; -struct page_owner po_snap; -}; -struct stack {struct stack_record *stack_record;struct stack *next;
@@ -741,7 +717,7 @@ void __dump_page_owner(const struct page *page)* must advance scan->pfn past the hit before calling again to resume * the scan. */ -static bool page_owner_next_eligible (struct page_owner_scan *scan) +bool page_owner_next_eligible (struct page_owner_scan *scan) {unsigned long pfn = scan->pfn;
@@ -1196,6 +1172,11 @@ static int __init pageowner_init(void) &stack_fops);debugfs_create_file ("count_threshold" ,0600 , dir,NULL , &threshold_fops); + +#ifdef CONFIG_BPF_SYSCALL +if (page_owner_iter_register ()) +pr_warn ("page_owner: failed to register bpf_iter target\n" ); +#endif return 0 ; }late_initcall (pageowner_init)
mm/page_owner.h new file +48 -0
48 lines not on this page · raw message
mm/page_owner_iter.c new file +124 -0
124 lines not on this page · raw message
Signature · 1 line
2.20.1
[email protected]
AI reviewRe: [PATCH v2 2/6] mm/page_owner: add bpf_iter target "page_owner"
TL;DR: - Missing BPF_ITER_RESCHED may let one read() loop up to 1M objects without yielding, risking RCU stalls on !PREEMPT - Commit message bpftrace example counts allocations, not pages
Quoted 1 line
mm/page_owner: addbpf_itertarget "page_owner"
TL;DR:
- Missing BPF_ITER_RESCHED may let one read() loop up to 1M objects without yielding, risking RCU stalls on !PREEMPT
- Commit message bpftrace example counts allocations, not pages
Quoted 19 lines
mm/page_owner: addbpf_itertarget "page_owner"Register a
bpf_itertarget "page_owner" that walks the pages eligible forpage_owneroutput, with the same eligibility rules as the/sys/kernel/debug/page_ownerread path.For each eligible page the attached BPF program receives the pfn, the page and the
page_ownerrecord, and can freely filter and format output.The
po_snaprecord in the scan cursor is taken inside thepage_extRCU window and handed to the BPF program as a copy, so no RCU window is needed at consumption time. Torn reads remain possible if the record is concurrently updated while it is copied, but this is the same best-effort behavior as the read path has always had.The target is registered from
pageowner_init()when bothCONFIG_PAGE_OWNERandCONFIG_BPF_SYSCALLare enabled; a registration failure only disables thebpf_itertarget.
Quoted 6 lines
Userspace tools such as bpftrace can attach to this target like any other
bpf_iter. For example, to count pages owned by init (pid 1):bpftrace -e 'iter:page_owner /
ctx->po->pid== 1 / { @ =count(); }'Signed-off-by Zhen Ni <[email protected]>
The example in the commit message says it counts pages owned by init:
For example, to count pages owned by init (pid 1):
bpftrace -e 'iter:page_owner / ctx->po->pid == 1 / { @ = count(); }'
Is that accurate? The iterator produces one record per allocation, not one per page, because page_owner_next_eligible() skips the tail pages of high-order allocations:
/*
* Don't print "tail" pages of high-order allocations as that
* would inflate the stats.
*/
if (!IS_ALIGNED(pfn, 1 << page_owner->order))
goto ext_put_continue;So '@ = count()' counts allocations made by pid 1. An order-9 THP allocation is counted as 1 instead of 512 pages.
Could the example or its description be corrected so that it reports what it claims, for instance with '@ = sum(1 << ctx->po->order)' to count pages?
Quoted 6 lines
mm/page_owner_iter.c new file +0
-0@@ -0,0 +1,124 @@
[ ... ]
mm/page_owner_iter.c line 110
Quoted 11 lines
mm/page_owner_iter.c new file +11
-0+static struct bpf_iter_reg page_owner_iter_reg_info = { + .target ="page_owner" , + .ctx_arg_info_size =2 , + .ctx_arg_info = { + { offsetof(struct bpf_iter__page_owner , page), +PTR_TO_BTF_ID_OR_NULL }, + { offsetof(struct bpf_iter__page_owner , po), +PTR_TO_BTF_ID_OR_NULL }, + }, + .seq_info = &page_owner_iter_seq_info, +};
Should page_owner_iter_reg_info set .feature = BPF_ITER_RESCHED?
Without it, bpf_seq_read() never calls cond_resched() between objects for this target, because can_resched = bpf_iter_support_resched(seq) is false.
page_owner_next_eligible() has no scheduling point on its hit path either:
scan->po_snap = *page_owner;
if (!scan->po_snap.handle)
goto ext_put_continue;
page_ext_put(page_ext);
return true;cond_resched() only runs on the ext_put_continue miss path.
When hits are back to back (runs of allocated order-0 pages, which is the normal case for anon and page cache memory) and the BPF program writes less than the user buffer size (the commit message's own '@ = count()' example writes nothing to the seq_file), a single read() loops up to MAX_ITER_OBJECTS (1,000,000) times. Each pass does page_ext_get/put, a struct copy and a BPF program run, with nothing yielding in between. On !PREEMPT kernels that holds the CPU and delays RCU quiescent states for the whole loop.
This is the situation commit cf83b2d2e2b6 ("bpf: Permit cond_resched for some iterators") hit in the same bpf_seq_read() loop, an rcu_sched self-detected stall under load. The debugfs read path doesn't have this problem because read_page_owner() returns to userspace after every record.
The requirement for the flag looks satisfied, since nothing is held across next() -> show(): page_owner_next_eligible() drops the page_ext RCU read lock before returning, and already calls cond_resched() itself. The other iterators that can safely reschedule (task, task_file, task_vma, cgroup, kmem_cache, dmabuf, kallsyms) all set BPF_ITER_RESCHED.
Later commits in this series (54a9d79ddbf5..e52345a241ae) do not add it.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38403671988
PATCH v2 3/6 mm/page_owner: add open-coded page_owner iterator
In reply to Zhen NiAdd an open-coded bpf_iter over the pages eligible for page_owner output, reusing page_owner_next_eligible().
A sleepable BPF program drives it with bpf_iter_page_owner_new()/next()/destroy(); each next() returns the page_owner_scan. po_snap is copied inside the page_ext RCU window, so the program needs no RCU section when it reads it.
Also add bpf_page_owner_stack_snprint(), which renders the stack trace of the current page into a caller-provided buffer.
---
Changes in v2:
- bpf_page_owner_stack_snprint() now takes the scan cursor.
- Buffer args renamed to buf__uninit/buf__sz.
v1: https://lore.kernel.org/linux-mm/[email protected]/
--- kernel/bpf/helpers.c | 6 +++++ mm/page_owner_iter.c | 53 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 59 insertions(+)
kernel/bpf/helpers.c +6 -0
@@ -4946,6 +4946,12 @@ BTF_ID_FLAGS(func, bpf_iter_dmabuf_new, KF_ITER_NEW | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_iter_dmabuf_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_iter_dmabuf_destroy, KF_ITER_DESTROY | KF_SLEEPABLE)#endif +#ifdef CONFIG_PAGE_OWNER +BTF_ID_FLAGS(func, bpf_iter_page_owner_new, KF_ITER_NEW | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_iter_page_owner_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_iter_page_owner_destroy, KF_ITER_DESTROY | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_page_owner_stack_snprint, KF_SLEEPABLE) +#endif BTF_ID_FLAGS(func, __bpf_trap)BTF_ID_FLAGS (func, bpf_strcmp,KF_PERFMON );BTF_ID_FLAGS (func, bpf_strcasecmp,KF_PERFMON );
mm/page_owner_iter.c +53 -0
62 lines not on this page · raw message
Signature · 1 line
2.20.1
[email protected]
AI reviewRe: [PATCH v2 3/6] mm/page_owner: add open-coded page_owner iterator
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] mm: soft lockup in page_owner_next_eligible via BPF iter --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] mm: soft lockup in page_owner_next_eligible via BPF iter
--
commit b7f01516d35223b7a6800ce695c29a9203809699
Author: Zhen Ni <[email protected]>
mm/page_owner: add open-coded page_owner iterator
This patch adds an open-coded bpf_iter over the pages eligible for page_owner output, reusing page_owner_next_eligible(). It also provides a sleepable BPF program to drive the iterator and retrieve stack traces.
Quoted 4 lines
mm/page_owner_iter.c +0
-0
[ ... ]
Quoted 15 lines
mm/page_owner_iter.c +11
-0@@ -122,3 +123,55 @@ int page_owner_iter_register(void)return bpf_iter_reg_target (&page_owner_iter_reg_info); } + +/* opaque to BPF programs */ +struct bpf_iter_page_owner { +__u64 __opaque[16 ]; +} __attribute__((aligned (8 ))); + +struct bpf_iter_page_owner_kern { +struct page_owner_scan scan; +} __attribute__((aligned (8 ))); + +__bpf_kfunc_start_defs ();
[ ... ]
mm/page_owner_iter.c int page_owner_iter_register(void) · line 161
Quoted 12 lines
mm/page_owner_iter.c +12
-0+__bpf_kfunc struct page_owner_scan * +bpf_iter_page_owner_next (struct bpf_iter_page_owner *it) +{ +struct bpf_iter_page_owner_kern *kit = (void *)it; + +cond_resched (); + +if (kit->scan.page) + kit->scan.pfn++; + +if (!page_owner_next_eligible (&kit->scan)) +return NULL ;
[Severity: High]
Can this iterator cause a soft lockup or RCU stall?
If page_owner is disabled, page_ext_get() inside page_owner_next_eligible() constantly returns NULL, triggering a continue on every single page up to max_pfn:
mm/page_owner.c:page_owner_next_eligible() {
...
for (; pfn < max_pfn; pfn++) {
...
page_ext = page_ext_get(page);
if (unlikely(!page_ext))
continue;
...
ext_put_continue:
page_ext_put(page_ext);
cond_resched();
}
...
}Because the continue statement skips the ext_put_continue block, the loop never calls cond_resched(). When called from this new sleepable BPF iterator, won't it spin over the entire physical memory space in a tight loop without yielding?
Quoted 3 lines
mm/page_owner_iter.c +3
-0+ +return &kit->scan; +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
Re: [PATCH v2 3/6] mm/page_owner: add open-coded page_owner iterator
Quoted 69 lines · 在 2026/10/13 20:49, [email protected] 写道:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] mm: soft lockup inpage_owner_next_eligiblevia BPF iter
--commit
b7f01516d35223b7a6800ce695c29a9203809699
Author: Zhen Ni <[email protected]>
mm/page_owner: add open-codedpage_owneriteratorThis patch adds an open-coded
bpf_iterover the pages eligible forpage_owneroutput, reusingpage_owner_next_eligible(). It also provides a sleepable BPF program to drive the iterator and retrieve stack traces.Quoted 4 lines
mm/page_owner_iter.c +0
-0[ ... ]
Quoted 15 lines
mm/page_owner_iter.c +11
-0@@ -122,3 +123,55 @@ int page_owner_iter_register(void)return bpf_iter_reg_target (&page_owner_iter_reg_info); } + +/* opaque to BPF programs */ +struct bpf_iter_page_owner { +__u64 __opaque[16 ]; +} __attribute__((aligned (8 ))); + +struct bpf_iter_page_owner_kern { +struct page_owner_scan scan; +} __attribute__((aligned (8 ))); + +__bpf_kfunc_start_defs ();[ ... ]
mm/page_owner_iter.c int page_owner_iter_register(void) · line 161
Quoted 12 lines
mm/page_owner_iter.c +12
-0+__bpf_kfunc struct page_owner_scan * +bpf_iter_page_owner_next (struct bpf_iter_page_owner *it) +{ +struct bpf_iter_page_owner_kern *kit = (void *)it; + +cond_resched (); + +if (kit->scan.page) + kit->scan.pfn++; + +if (!page_owner_next_eligible (&kit->scan)) +return NULL ;[Severity: High]
Can this iterator cause a soft lockup or RCU stall?If
page_owneris disabled,page_ext_get()insidepage_owner_next_eligible()constantly returns NULL, triggering a continue on every single page up tomax_pfn:mm/page_owner.c:page_owner_next_eligible() { ... for (; pfn < max_pfn; pfn++) { ... page_ext = page_ext_get(page); if (unlikely(!page_ext)) continue; ... ext_put_continue: page_ext_put(page_ext); cond_resched(); } ... }
Quoted 9 lines
Because the continue statement skips the
ext_put_continueblock, the loop never callscond_resched(). When called from this new sleepable BPF iterator, won't it spin over the entire physical memory space in a tight loop without yielding?mm/page_owner_iter.c int page_owner_iter_register(void) · line 164
Quoted 3 lines
mm/page_owner_iter.c +3
-0+ +return &kit->scan; +}
The soft lockup / RCU stall doesn't seem critical, since the debugfs path behaves the same way, but I think it's worth disabling these kfuncs when page_owner is not enabled. I'll guard them with static_branch_unlikely(&page_owner_inited).
Thanks,
Zhen
[email protected]
AI reviewRe: [PATCH v2 3/6] mm/page_owner: add open-coded page_owner iterator
In reply to Zhen NiTL;DR: The new page_owner kfuncs may lack KF_PERFMON, so a CAP_BPF-only syscall program could read page_owner records and symbolized allocation stacks that are root-only in debugfs.
Quoted 1 line
mm/page_owner: add open-codedpage_owneriterator
TL;DR: The new page_owner kfuncs may lack KF_PERFMON, so a CAP_BPF-only syscall program could read page_owner records and symbolized allocation stacks that are root-only in debugfs.
Quoted 4 lines
kernel/bpf/helpers.c +0
-0
[ ... ]
kernel/bpf/helpers.c BTF_ID_FLAGS(func, bpf_iter_dmabuf_new, KF_ITER_NEW | KF_SLEEPABLE) · line 5188
Quoted 13 lines
kernel/bpf/helpers.c +6
-0@@ -5177,6 +5177,12 @@ BTF_ID_FLAGS(func, bpf_iter_dmabuf_new, KF_ITER_NEW | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_iter_dmabuf_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_iter_dmabuf_destroy, KF_ITER_DESTROY | KF_SLEEPABLE)#endif +#ifdef CONFIG_PAGE_OWNER +BTF_ID_FLAGS(func, bpf_iter_page_owner_new, KF_ITER_NEW | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_iter_page_owner_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_iter_page_owner_destroy, KF_ITER_DESTROY | KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_page_owner_stack_snprint, KF_SLEEPABLE) +#endif BTF_ID_FLAGS(func, __bpf_trap)BTF_ID_FLAGS (func, bpf_strcmp,KF_PERFMON );BTF_ID_FLAGS (func, bpf_strcasecmp,KF_PERFMON );
Should bpf_page_owner_stack_snprint(), and probably bpf_iter_page_owner_new(), carry KF_PERFMON here?
These kfuncs are added to common_btf_ids, which is registered for BPF_PROG_TYPE_UNSPEC, and none of them has KF_PERFMON.
BPF_PROG_TYPE_SYSCALL programs are always sleepable (check_attach_btf_id() requires it), and they only need CAP_BPF (or a delegated BPF token) since SYSCALL is not covered by is_perfmon_prog_type().
That means a CAP_BPF-only loader can walk all page_owner records and use bpf_page_owner_stack_snprint() to copy symbolized allocation stack traces into program memory. This skips the allow_ptr_leaks gate that check_ptr_to_btf_access() applies when the same data is read field by field.
The debugfs interface that exposes this information is root-only: page_owner is 0600 and the page_owner_stacks/* files are 0400.
This tree already has KF_PERFMON (commit 88ce88e933e4 "bpf: Add KF_PERFMON kfunc flag"), and comparable kfuncs use it, for example bpf_get_kmem_cache() and bpf_probe_read_kernel_dynptr() in this same table.
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/38403671988
PATCH v2 4/6 mm/page_owner: record the NUMA node in the scan cursor
In reply to Zhen NiCompute the page's NUMA node in page_owner_next_eligible() while the page is still valid, and store it in scan->nid.
The debugfs nid filter reads scan->nid instead of recomputing it, and a BPF program iterating the cursor reads the node straight from scan->nid.
Resolve nid at scan time to avoid dereferencing a possibly-unmapped page. Keep it in the transient scan cursor rather than struct page_owner, so nid does not add per-page memory overhead.
---
Changes in v2:
- Removes the v1 get_nid() kfunc. The node is now stored in scan->nid.
v1: https://lore.kernel.org/linux-mm/[email protected]/
--- mm/page_owner.c | 28 ++++++++++++++++------------ mm/page_owner.h | 1 + mm/page_owner_iter.c | 5 ++++- 3 files changed, 21 insertions(+), 13 deletions(-)
mm/page_owner.c +16 -12
@@ -711,6 +711,19 @@ void __dump_page_owner(const struct page *page)page_ext_put (page_ext); } +/* +* NUMA node of @page, bypassing PF_POISONED_CHECK() in page_to_nid() to +* avoid VM_BUG_ON on poisoned pages; -1 for poisoned pages. +*/ +static int page_owner_get_nid (struct page *page) +{ +memdesc_flags_t page_flags =READ_ONCE (page->flags); + +if (page_flags.f ==PAGE_POISON_PATTERN ) +return -1 ; +return memdesc_nid (&page_flags); +} +/* * Advance @scan to the next page eligible for page_owner output. * On a hit, fill scan->pfn/page/po_snap and return true; the caller
@@ -782,6 +795,7 @@ bool page_owner_next_eligible(struct page_owner_scan *scan)if (!scan->po_snap.handle) goto ext_put_continue; + scan->nid =page_owner_get_nid (page);page_ext_put (page_ext);return true;
@@ -810,18 +824,8 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)/* Find an allocated page */ while (page_owner_next_eligible (&scan)) {if (state->nid_filter_enabled) { -int nid; -memdesc_flags_t page_flags = -READ_ONCE (scan.page->flags); - -/* -* Bypass PF_POISONED_CHECK() in page_to_nid() to avoid -* VM_BUG_ON when accessing poisoned pages. -*/ -if (page_flags.f ==PAGE_POISON_PATTERN ) - goto skip_continue; - nid =memdesc_nid (&page_flags); -if (!node_isset (nid, state->nid_filter)) +if (scan.nid <0 || + !node_isset (scan.nid, state->nid_filter)) goto skip_continue; }
mm/page_owner.h +1 -0
@@ -36,6 +36,7 @@ struct page_owner_scan {unsigned long pfn;struct page *page;struct page_owner po_snap; +int nid; };bool page_owner_next_eligible (struct page_owner_scan *scan);
mm/page_owner_iter.c +4 -1
@@ -13,10 +13,12 @@ struct bpf_iter__page_owner { __bpf_md_ptr(struct page *, page); __bpf_md_ptr(struct page_owner *, po);u64 pfn; +int nid; }; DEFINE_BPF_ITER_FUNC(page_owner, struct bpf_iter_meta *meta, - struct page *page, struct page_owner *po, u64 pfn) + struct page *page, struct page_owner *po, u64 pfn, + int nid)static void *page_owner_seq_start (struct seq_file *seq,loff_t *posp) {
@@ -54,6 +56,7 @@ static int page_owner_prog_seq_show(struct bpf_prog *prog,/* @po: hit-time snapshot, no RCU window needed. */ ctx.po = &scan->po_snap; ctx.pfn = scan->pfn; + ctx.nid = scan->nid; }return bpf_iter_run_prog (prog, &ctx);
Signature · 1 line
2.20.1
[email protected]
AI reviewRe: [PATCH v2 4/6] mm/page_owner: record the NUMA node in the scan cursor
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [High] mm: incomplete fix leaves page_owner vulnerable to unmapped page dereference --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] mm: incomplete fix leaves page_owner vulnerable to unmapped page dereference --
commit 7b174b26adeea5712aaa10bd465514b5ce7dbe36
Author: Zhen Ni <[email protected]>
Subject: mm/page_owner: record the NUMA node in the scan cursor
This commit computes the page's NUMA node in page_owner_next_eligible() and stores it in the scan cursor to avoid dereferencing a potentially unmapped page later. It updates the debugfs nid filter and BPF program to use this stored node value.
Quoted 4 lines
mm/page_owner.c +0
-0
[ ... ]
Quoted 11 lines
mm/page_owner.c +0
-7@@ -810,18 +824,8 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)/* Find an allocated page */ while (page_owner_next_eligible (&scan)) {if (state->nid_filter_enabled) { -int nid; -memdesc_flags_t page_flags = -READ_ONCE (scan.page->flags); - -/* -* Bypass PF_POISONED_CHECK() in page_to_nid() to avoid -* VM_BUG_ON when accessing poisoned pages.
mm/page_owner.c read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos) · line 831
Quoted 10 lines
mm/page_owner.c +2
-5-*/ -if (page_flags.f ==PAGE_POISON_PATTERN ) - goto skip_continue; - nid =memdesc_nid (&page_flags); -if (!node_isset (nid, state->nid_filter)) +if (scan.nid <0 || + !node_isset (scan.nid, state->nid_filter)) goto skip_continue; }
[Severity: High]
Does this fix leave scan.page vulnerable to an unmapped page dereference shortly after this block?
Although this patch successfully avoids dereferencing scan.page when checking the node filter, page_owner_next_eligible() drops the RCU lock via page_ext_put() before returning.
Just below the quoted block in read_page_owner(), scan.page is still passed directly to print_page_owner():
read_page_owner() {
...
while (page_owner_next_eligible(&scan)) {
...
return print_page_owner(buf, count, scan.pfn, scan.page,
&scan.po_snap, scan.po_snap.handle, state);
...
}Inside print_page_owner(), the pointer is dereferenced by scnprintf using the %pGp formatter on &page->flags.f.
If a concurrent memory hot-remove operation frees the vmemmap backing this struct page after the RCU lock is dropped, will this %pGp format access trigger a kernel panic?
Additionally, a follow-up patch in this series (Patch 5) appears to introduce get_page_memcg_info() which also unconditionally dereferences scan.page outside the RCU lock from read_page_owner(). Does that face the same use-after-free risk?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=4
Re: [PATCH v2 4/6] mm/page_owner: record the NUMA node in the scan cursor
Quoted 63 lines · 在 2026/10/13 20:36, [email protected] 写道:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [High] mm: incomplete fix leavespage_ownervulnerable to unmapped page dereference --commit
7b174b26adeea5712aaa10bd465514b5ce7dbe36
Author: Zhen Ni <[email protected]>
Subject:mm/page_owner: record the NUMA node in the scan cursorThis commit computes the page's NUMA node in
page_owner_next_eligible()and stores it in the scan cursor to avoid dereferencing a potentially unmapped page later. It updates the debugfs nid filter and BPF program to use this stored node value.Quoted 4 lines
mm/page_owner.c +0
-0[ ... ]
Quoted 11 lines
mm/page_owner.c +0
-7@@ -810,18 +824,8 @@ read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos)/* Find an allocated page */ while (page_owner_next_eligible (&scan)) {if (state->nid_filter_enabled) { -int nid; -memdesc_flags_t page_flags = -READ_ONCE (scan.page->flags); - -/* -* Bypass PF_POISONED_CHECK() in page_to_nid() to avoid -* VM_BUG_ON when accessing poisoned pages. mm/page_owner.c read_page_owner(struct file *file, char __user *buf, size_t count, loff_t *ppos) · line 831
Quoted 10 lines
mm/page_owner.c +2
-5-*/ -if (page_flags.f ==PAGE_POISON_PATTERN ) - goto skip_continue; - nid =memdesc_nid (&page_flags); -if (!node_isset (nid, state->nid_filter)) +if (scan.nid <0 || + !node_isset (scan.nid, state->nid_filter)) goto skip_continue; }[Severity: High]
Does this fix leave scan.page vulnerable to an unmapped page dereference shortly after this block?Although this patch successfully avoids dereferencing scan.page when checking the node filter,
page_owner_next_eligible()drops the RCU lock viapage_ext_put()before returning.Just below the quoted block in
read_page_owner(), scan.page is still passed directly toprint_page_owner():read_page_owner() { ... while (page_owner_next_eligible(&scan)) { ... return print_page_owner(buf, count, scan.pfn, scan.page, &scan.po_snap, scan.po_snap.handle, state); ... }Inside
print_page_owner(), the pointer is dereferenced by scnprintf using the %pGp formatter on&page->flags.f.
Quoted 9 lines
If a concurrent memory hot-remove operation frees the vmemmap backing this struct page after the RCU lock is dropped, will this %pGp format access trigger a kernel panic?
Additionally, a follow-up patch in this series (Patch 5) appears to introduce
get_page_memcg_info()which also unconditionally dereferences scan.page outside the RCU lock fromread_page_owner(). Does that face the same use-after-free risk?
The page_ext RCU lock does not pin the underlying struct page. The dereference in print_page_owner() predates this series and is not introduced by these changes.
Thanks,
Zhen
PATCH v2 5/6 mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc
In reply to Zhen NiExpose per-page memcg information to BPF programs, so that a bpf_iter over page_owner can aggregate and filter allocations by memcg. Attributing an allocation to a specific cgroup needs its full cgroup path, not just the memcg name.
Split memcg info collection out of print_page_owner_memcg() into get_page_memcg_info(), which fills struct memcg_info and, when given a caller-provided buffer, writes the full cgroup path via cgroup_path(). print_page_owner_memcg() now consumes the struct; this introduces no functional change to the debugfs output.
The helper takes a single READ_ONCE snapshot within one RCU read-side critical section, so the flags and cgroup path reflect a consistent view of the page. cgroup_path() returns the path relative to the cgroup hierarchy root, so memcg attribution works on both cgroup v1 and v2.
The kfunc gets the page via pfn_to_online_page(), so an offline or removed pfn yields -ENODATA instead of a stale deref.
---
Changes in v2:
- bpf_page_owner_get_memcg_info() now resolves the page with pfn_to_online_page() for an offline or removed pfn.
- Buffer/info args renamed to info__uninit/buf__uninit/buf__sz.
v1: https://lore.kernel.org/linux-mm/[email protected]/
--- kernel/bpf/helpers.c | 1 + mm/page_owner.c | 85 ++++++++++++++++++++++++++++++++++---------- mm/page_owner.h | 9 +++++ mm/page_owner_iter.c | 17 +++++++++ 4 files changed, 93 insertions(+), 19 deletions(-)
kernel/bpf/helpers.c +1 -0
@@ -4951,6 +4951,7 @@ BTF_ID_FLAGS(func, bpf_iter_page_owner_new, KF_ITER_NEW | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_iter_page_owner_next, KF_ITER_NEXT | KF_RET_NULL | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_iter_page_owner_destroy, KF_ITER_DESTROY | KF_SLEEPABLE) BTF_ID_FLAGS(func, bpf_page_owner_stack_snprint, KF_SLEEPABLE) +BTF_ID_FLAGS(func, bpf_page_owner_get_memcg_info, KF_SLEEPABLE)#endif BTF_ID_FLAGS(func, __bpf_trap)BTF_ID_FLAGS (func, bpf_strcmp,KF_PERFMON );
mm/page_owner.c +66 -19
141 lines not on this page · raw message
mm/page_owner.h +9 -0
@@ -39,7 +39,16 @@ struct page_owner_scan {int nid; }; +struct memcg_info { +char name[80 ]; +bool is_slab; +bool is_kmem; +bool is_online; +}; +bool page_owner_next_eligible (struct page_owner_scan *scan); +int get_page_memcg_info (struct page *page,struct memcg_info *info, +char *buf,int size);/* * mm/page_owner_iter.c
mm/page_owner_iter.c +17 -0
@@ -177,4 +177,21 @@ __bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan, 0); } +__bpf_kfunc int bpf_page_owner_get_memcg_info(struct page_owner_scan *scan, +struct memcg_info *info__uninit, +char *buf__uninit,u32 buf__sz) +{ +/* Guard against the section being offlined after scan. */ +struct page *page =pfn_to_online_page (scan->pfn); + +if (!page) { + *info__uninit = (struct memcg_info ){}; +if (buf__uninit && buf__sz) + buf__uninit[0 ] ='\0' ; +return -ENODATA ; + } + +return get_page_memcg_info (page, info__uninit, buf__uninit, buf__sz); +} +__bpf_kfunc_end_defs ();
Signature · 1 line
2.20.1
[email protected]
AI reviewRe: [PATCH v2 5/6] mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider: - [High] bpf: TOCTOU race in bpf_page_owner_get_memcg_info() leads to page fault - [High] mm: wild pointer…
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] bpf: TOCTOU race in bpf_page_owner_get_memcg_info() leads to page fault
- [High] mm: wild pointer dereference in get_page_memcg_info() for tail pages --
commit ab0ccc33a4ab8ccdaacb5f6b2dca8722bf6d6123
Author: Zhen Ni <[email protected]>
mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc
This commit exposes per-page memcg information to BPF programs. It splits memcg info collection out of print_page_owner_memcg() into a new get_page_memcg_info() function and adds the bpf_page_owner_get_memcg_info() kfunc.
Quoted 5 lines
mm/page_owner.c +0
-0@@ -538,16 +538,22 @@ void pagetypeinfo_showmixedcount_print(struct seq_file *m,
[ ... ]
Quoted 4 lines
mm/page_owner.c +2
-0+int get_page_memcg_info (struct page *page,struct memcg_info *info, +char *buf,int size) {unsigned long memcg_data;
mm/page_owner.c void pagetypeinfo_showmixedcount_print(struct seq_file *m,
Quoted 10 lines
mm/page_owner.c +5
-0struct obj_cgroup *objcg;struct mem_cgroup *memcg; +int ret = -ENODATA ; + + *info = (struct memcg_info ){};rcu_read_lock (); memcg_data =READ_ONCE (page->memcg_data); +if (!memcg_data) + goto out_unlock;
[Severity: High]
Does removing the PageTail() guard here expose a wild pointer dereference?
In struct folio, the first tail page's _nr_pages field aliases exactly with the memcg_data offset when CONFIG_MEMCG is enabled. Due to a lack of memory barriers in __update_page_owner_handle(), page_owner_next_eligible() can observe PAGE_EXT_OWNER_ALLOCATED before page_owner->order is updated, yielding a tail page.
If a tail page is passed here, the non-zero _nr_pages value (e.g., 512 for a large folio) will bypass the !memcg_data check and be dereferenced as an obj_cgroup pointer later in the function, leading to a kernel panic.
[ ... ]
Quoted 14 lines
mm/page_owner_iter.c +6
-0@@ -177,4 +177,21 @@ __bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan, 0); } +__bpf_kfunc int bpf_page_owner_get_memcg_info(struct page_owner_scan *scan, +struct memcg_info *info__uninit, +char *buf__uninit,u32 buf__sz) +{ +/* Guard against the section being offlined after scan. */ +struct page *page =pfn_to_online_page (scan->pfn);
mm/page_owner_iter.c __bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan, · line 195
Quoted 10 lines
mm/page_owner_iter.c +10
-0+ +if (!page) { + *info__uninit = (struct memcg_info ){}; +if (buf__uninit && buf__sz) + buf__uninit[0 ] ='\0' ; +return -ENODATA ; + } + +return get_page_memcg_info (page, info__uninit, buf__uninit, buf__sz); +}
[Severity: High]
Is this vulnerable to a time-of-check to time-of-use race with memory hot-remove?
The pfn_to_online_page() check and subsequent get_page_memcg_info() dereference occur without holding get_online_mems(). Because this is a sleepable BPF kfunc (KF_SLEEPABLE), the thread can be preempted between the check and the dereference.
If memory hot-remove offlines and removes the memory section concurrently, it tears down the vmemmap backing the struct page. When the thread resumes and get_page_memcg_info() attempts to read page->memcg_data, it will fault on the unmapped address.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Re: [PATCH v2 5/6] mm/page_owner: add bpf_page_owner_get_memcg_info() kfunc
Quoted 49 lines · 在 2026/10/13 20:49, [email protected] 写道:
Thank you for your contribution! Sashiko AI review found 2 potential issue(s) to consider:
- [High] bpf: TOCTOU race inbpf_page_owner_get_memcg_info()leads to page fault
- [High] mm: wild pointer dereference inget_page_memcg_info()for tail pages --commit
ab0ccc33a4ab8ccdaacb5f6b2dca8722bf6d6123
Author: Zhen Ni <[email protected]>
mm/page_owner: addbpf_page_owner_get_memcg_info()kfuncThis commit exposes per-page memcg information to BPF programs. It splits memcg info collection out of
print_page_owner_memcg()into a newget_page_memcg_info()function and adds thebpf_page_owner_get_memcg_info()kfunc.Quoted 5 lines
mm/page_owner.c +0
-0@@ -538,16 +538,22 @@ void pagetypeinfo_showmixedcount_print(struct seq_file *m,[ ... ]
Quoted 4 lines
mm/page_owner.c +2
-0+int get_page_memcg_info (struct page *page,struct memcg_info *info, +char *buf,int size) {unsigned long memcg_data;mm/page_owner.c void pagetypeinfo_showmixedcount_print(struct seq_file *m,
Quoted 10 lines
mm/page_owner.c +5
-0struct obj_cgroup *objcg;struct mem_cgroup *memcg; +int ret = -ENODATA ; + + *info = (struct memcg_info ){};rcu_read_lock (); memcg_data =READ_ONCE (page->memcg_data); +if (!memcg_data) + goto out_unlock;[Severity: High]
Does removing thePageTail()guard here expose a wild pointer dereference?In struct folio, the first tail page's
_nr_pagesfield aliases exactly with thememcg_dataoffset whenCONFIG_MEMCGis enabled. Due to a lack of memory barriers in__update_page_owner_handle(),page_owner_next_eligible()can observePAGE_EXT_OWNER_ALLOCATEDbeforepage_owner->orderis updated, yielding a tail page.
Quoted 4 lines
If a tail page is passed here, the non-zero
_nr_pagesvalue (e.g., 512 for a large folio) will bypass the !memcg_data check and be dereferenced as anobj_cgrouppointer later in the function, leading to a kernel panic.
The PageTail(page) check is not removed. get_page_memcg_info() still reads:
memcg_data = READ_ONCE(page->memcg_data);
if (!memcg_data || PageTail(page))
goto out_unlock;Won't fix.
Quoted 36 lines
[ ... ]
Quoted 14 lines
mm/page_owner_iter.c +6
-0@@ -177,4 +177,21 @@ __bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan, 0); } +__bpf_kfunc int bpf_page_owner_get_memcg_info(struct page_owner_scan *scan, +struct memcg_info *info__uninit, +char *buf__uninit,u32 buf__sz) +{ +/* Guard against the section being offlined after scan. */ +struct page *page =pfn_to_online_page (scan->pfn);mm/page_owner_iter.c __bpf_kfunc int bpf_page_owner_stack_snprint(struct page_owner_scan *scan, · line 195
Quoted 10 lines
mm/page_owner_iter.c +10
-0+ +if (!page) { + *info__uninit = (struct memcg_info ){}; +if (buf__uninit && buf__sz) + buf__uninit[0 ] ='\0' ; +return -ENODATA ; + } + +return get_page_memcg_info (page, info__uninit, buf__uninit, buf__sz); +}[Severity: High]
Is this vulnerable to a time-of-check to time-of-use race with memory hot-remove?The
pfn_to_online_page()check and subsequentget_page_memcg_info()dereference occur without holdingget_online_mems(). Because this is a sleepable BPF kfunc (KF_SLEEPABLE), the thread can be preempted between the check and the dereference.
Quoted 5 lines
If memory hot-remove offlines and removes the memory section concurrently, it tears down the vmemmap backing the struct page. When the thread resumes and
get_page_memcg_info()attempts to readpage->memcg_data, it will fault on the unmapped address.
pfn_to_online_page() here is meant to close a window that the iterator would otherwise widen, not to fully serialize against memory hot-remove.
Thanks,
Zhen
PATCH v2 6/6 selftests/bpf: add tests for the page_owner bpf_iter target
In reply to Zhen NiAdd a sleepable BPF program and a userspace runner covering the seq_file and open-coded page_owner iterators. The test checks:
- seq_file walk, unfiltered and pid-filtered: records read back from the iterator fd match the lines the program printed.
- open-coded iterator parity with the seq path.
- the kfuncs' contracts: a non-empty stack trace, and memcg classification.
CONFIG_PAGE_OWNER is enabled in the bpf selftests config so the program's page_owner BTF types are available. Skipped when debugfs page_owner is unavailable.
---
Changes in v2:
- Drop the get_nid kfunc coverage.
- Enable CONFIG_PAGE_OWNER in the bpf selftests config.
v1: https://lore.kernel.org/linux-mm/[email protected]/
--- tools/testing/selftests/bpf/config | 2 + .../bpf/prog_tests/page_owner_iter.c | 193 ++++++++++++++++++ .../selftests/bpf/progs/bpf_iter_page_owner.c | 159 +++++++++++++++ 3 files changed, 354 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/page_owner_iter.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c
tools/testing/selftests/bpf/config +2 -0
@@ -136,3 +136,5 @@ CONFIG_SMC_HS_CTRL_BPF=y CONFIG_DIBS=y CONFIG_DIBS_LO=y CONFIG_PM_WAKELOCKS=y +CONFIG_DEBUG_KERNEL=y +CONFIG_PAGE_OWNER=y
tools/testing/selftests/bpf/prog_tests/page_owner_iter.c new file +193 -0
193 lines not on this page · raw message
tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c new file +159 -0
159 lines not on this page · raw message
Signature · 1 line
2.20.1
[email protected]
AI reviewRe: [PATCH v2 6/6] selftests/bpf: add tests for the page_owner bpf_iter target
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] bpf/selftests: hardcoded errno values break architecture portability --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] bpf/selftests: hardcoded errno values break architecture portability --
commit d63d4254b64f243f696e8d9b4ca8b07f371d8139
Author: Zhen Ni <[email protected]>
selftests/bpf: add tests for the page_owner bpf_iter target
This commit adds a sleepable BPF program and a userspace runner to test the seq_file and open-coded page_owner iterators. It verifies iterator parity, output matching, and kfunc contracts including memcg classification.
Quoted 4 lines
tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c new file +0
-0
[ ... ]
tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c line 37
Quoted 9 lines
tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c new file +9
-0+char memcg_pathbuf[4096 ]; +char memcg_first_path[4096 ]; +char memcg_first_name[80 ]; + +#define MEMCG_ENODATA 61 +#define MEMCG_EOPNOTSUPP 95 + +#define MEMCG_F_ONLINE (1 << 0) +#define MEMCG_F_OBJCG (1 << 1)
[Severity: Medium]
Will these hardcoded errno values cause a regression in the selftests on certain architectures?
The bpf_page_owner_get_memcg_info() kfunc returns standard kernel error codes like -ENODATA and -EOPNOTSUPP. However, these error codes vary by architecture. For example, ENODATA is 111 on SPARC and 86 on Alpha, while EOPNOTSUPP is 122 on MIPS.
When the test runs on these architectures, the kfunc will return the architecture-specific error code. This will fail the hardcoded comparison in check_memcg(), incorrectly incrementing memcg_bad and causing the selftest to fail.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Re: [PATCH v2 6/6] selftests/bpf: add tests for the page_owner bpf_iter target
Quoted 37 lines · 在 2026/10/13 20:38, [email protected] 写道:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] bpf/selftests: hardcoded errno values break architecture portability --commit
d63d4254b64f243f696e8d9b4ca8b07f371d8139
Author: Zhen Ni <[email protected]>selftests/bpf: add tests for the
page_ownerbpf_itertargetThis commit adds a sleepable BPF program and a userspace runner to test the
seq_fileand open-codedpage_owneriterators. It verifies iterator parity, output matching, and kfunc contracts including memcg classification.Quoted 4 lines
tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c new file +0
-0[ ... ]
tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c line 37
Quoted 9 lines
tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c new file +9
-0+char memcg_pathbuf[4096 ]; +char memcg_first_path[4096 ]; +char memcg_first_name[80 ]; + +#define MEMCG_ENODATA 61 +#define MEMCG_EOPNOTSUPP 95 + +#define MEMCG_F_ONLINE (1 << 0) +#define MEMCG_F_OBJCG (1 << 1)[Severity: Medium]
Will these hardcoded errno values cause a regression in the selftests on certain architectures?The
bpf_page_owner_get_memcg_info()kfunc returns standard kernel error codes like-ENODATAand-EOPNOTSUPP. However, these error codes vary by architecture. For example, ENODATA is 111 on SPARC and 86 on Alpha, while EOPNOTSUPP is 122 on MIPS.
Quoted 5 lines
When the test runs on these architectures, the kfunc will return the architecture-specific error code. This will fail the hardcoded comparison in
check_memcg(), incorrectly incrementingmemcg_badand causing the selftest to fail.
I'll use -ENODATA/-EOPNOTSUPP instead of the hardcoded. Will fix it.
Thanks,
Zhen
Zi Yan
reviewerQuoted 33 lines · Zhen Ni
Hi all,
Introduction
============
page_ownerrecords who allocated every page and is a core memory-debug facility, but it currently reaches userspace as the debugfs flat dump:
slicing allocations by pid, comm, NUMA node, cgroup or stack, etc., means streaming and parsing the entire file in userspace, which is expensive on large hosts and hard to query live.Exposing the same walk to eBPF lets a program filter and aggregate in the kernel and surface only the records it cares about. This series adds that access as a
seq_filebpf_itertarget and an open-coded iterator.Implementation
==============The page walk is factored into
page_owner_next_eligible(), shared by the read path and the iterators so eligibility stays identical. Two kfuncs expose information that a BPF program cannot decode from the snapshot alone: per-page memcg info including the full cgroup path, and the stack trace behind astack_depothandle. Each record is snapshotted under thepage_extRCU lock, so consumers read a consistent copy.Testing
=======The selftest exercises the
seq_fileand open-coded iterators, and validates thestack_snprintandget_memcg_infokfuncs.
Quoted 14 lines
Changes in v2
=============
- Route the zero-handle case throughext_put_continue.
- Drop the scan->pfn++.
- Fill page/po/pfn only when v is non-NULL.
- Implementpage_owner_seq_stop().
-bpf_page_owner_stack_snprint()now takes the scan cursor.
- Buffer args renamed to buf__uninit/buf__sz.
- Removes the v1get_nid()kfunc. The node is now stored inscan->nid.
-bpf_page_owner_get_memcg_info()now resolves the page withpfn_to_online_page()for an offline or removed pfn.
- Buffer/info args renamed to info__uninit/buf__uninit/buf__sz.
- Drop theget_nidkfunc coverage.
- EnableCONFIG_PAGE_OWNERin the bpf selftests config.
Hi Zhen,
Thank you for your patches. Review focus is currently on hotfixes and patches already in mm tree. Please ping us on this after the merge window is close, if no one still has not looked at it by then.
Thanks.
Quoted 24 lines
v1: https://lore.kernel.org/linux-mm/[email protected]/
Zhen Ni (6):
mm/page_owner: extractpage_owner_next_eligible()from read loopmm/page_owner: addbpf_itertarget "page_owner"mm/page_owner: add open-codedpage_owneriteratormm/page_owner: record the NUMA node in the scan cursormm/page_owner: addbpf_page_owner_get_memcg_info()kfunc selftests/bpf: add tests for thepage_ownerbpf_itertargetkernel/bpf/helpers.c | 7 + mm/Makefile | 3 + mm/page_owner.c | 219 +++++++++++------- mm/page_owner.h | 58 +++++ mm/page_owner_iter.c | 197 ++++++++++++++++ tools/testing/selftests/bpf/config | 2 + .../bpf/prog_tests/page_owner_iter.c | 193 +++++++++++++++ .../selftests/bpf/progs/bpf_iter_page_owner.c | 159 +++++++++++++ 8 files changed, 758 insertions(+), 80 deletions(-) create mode 100644 mm/page_owner.h create mode 100644 mm/page_owner_iter.c create mode 100644 tools/testing/selftests/bpf/prog_tests/page_owner_iter.c create mode 100644 tools/testing/selftests/bpf/progs/bpf_iter_page_owner.c
Signature · 2 lines
Best Regards, Yan, Zi