Add wakeup_source iterators
Subsystems (4)
v5 cover Add wakeup_source iterators
This patch series introduces BPF iterators for wakeup_source, enabling BPF programs to efficiently traverse a device's wakeup sources.
Currently, inspecting wakeup sources typically involves reading interfaces like /sys/class/wakeup/* or debugfs. The repeated syscalls to query the sysfs nodes is inefficient, as there can be hundreds of wakeup_sources, and each wakeup source have multiple stats, with one sysfs node per stat. debugfs is unstable and insecure.
The iterators utilize pre-existing wakeup_sources_walk_* functions to traverse over the SRCU that backs the list of wakeup_sources.
Changes in v5:
- Add CORE definitions for *__local variables to fix s390 build per bpf-ci
- v4 link: https://lore.kernel.org/all/[email protected]/
Changes in v4:
- Proper init of variables in ws_iter_check_sleep_times() test per bpf-ci
- Remove PM patch since it's already part of rebase
- v3 link: https://lore.kernel.org/all/[email protected]/
Changes in v3:
- Update wakeup_sources_walk_start() to handle an empty list per bpf-ci
- Simplify read of a struct's field in BPF program selftest per Andrii
- Drop open coded iterators for wakeup_sources
- Fix condition from !get_ws_iter_stream to get_ws_iter_stream in selftest
- Read event_count instead of wakeup_count in selftest
- v2 link: https://lore.kernel.org/all/[email protected]/
Changes in v2:
- Guard BPF Makefile with CONFIG_PM_SLEEP to fix build errors
- Update copyright from 2025 to 2026
- v1 link: https://lore.kernel.org/all/[email protected]/
Samuel Wu (2):
bpf: Add wakeup_source iterator
selftests/bpf: Add tests for wakeup_sources
kernel/bpf/Makefile | 3 + kernel/bpf/wakeup_source_iter.c | 103 ++++++ tools/testing/selftests/bpf/config | 1 + .../bpf/prog_tests/wakeup_source_iter.c | 300 ++++++++++++++++++ .../selftests/bpf/progs/wakeup_source_iter.c | 80 +++++ 5 files changed, 487 insertions(+) create mode 100644 kernel/bpf/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/prog_tests/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/progs/wakeup_source_iter.c
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
PATCH v5 1/2 bpf: Add wakeup_source iterator
Add a BPF iterator for traversing through wakeup_sources.
Setup iterators to traverse through a SRCUs of wakeup_sources. This is a more elegant and efficient traversal than going through the options today, such as at /sys/class/wakeup, or through debugfs.
--- kernel/bpf/Makefile | 3 + kernel/bpf/wakeup_source_iter.c | 103 ++++++++++++++++++++++++++++++++ 2 files changed, 106 insertions(+) create mode 100644 kernel/bpf/wakeup_source_iter.c
kernel/bpf/Makefile +3 -0
@@ -66,6 +66,9 @@ obj-$(CONFIG_BPF_SYSCALL) += kmem_cache_iter.o ifeq ($(CONFIG_DMA_SHARED_BUFFER),y) obj-$(CONFIG_BPF_SYSCALL) += dmabuf_iter.o endif +ifeq ($(CONFIG_PM_SLEEP),y) +obj-$(CONFIG_BPF_SYSCALL) += wakeup_source_iter.o +endif CFLAGS_REMOVE_percpu_freelist.o = $(CC_FLAGS_FTRACE) CFLAGS_REMOVE_bpf_lru_list.o = $(CC_FLAGS_FTRACE)
kernel/bpf/wakeup_source_iter.c new file +103 -0
103 lines not on this page · raw message
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
[email protected]
AI reviewRe: [PATCH bpf-next v5 1/2] bpf: Add wakeup_source iterator
Should wakeup_source_iter_seq_stop() call __wakeup_source_seq_show(seq, v, true) when v is NULL, before releasing the SRCU lock?
Quoted 3 lines
kernel/bpf/wakeup_source_iter.c new file +0
-0
[ ... ]
kernel/bpf/wakeup_source_iter.c line 45
Quoted 8 lines
kernel/bpf/wakeup_source_iter.c new file +8
-0+static void wakeup_source_iter_seq_stop (struct seq_file *seq,void *v) +{ +int *srcuidx = seq->private; + +if (*srcuidx >=0 ) +wakeup_sources_read_unlock (*srcuidx); + *srcuidx =-1 ; +}
Should wakeup_source_iter_seq_stop() call __wakeup_source_seq_show(seq, v, true) when v is NULL, before releasing the SRCU lock?
bpf_seq_read() in kernel/bpf/bpf_iter.c has this at the stop label:
/* bpf program called if !p */
seq->op->stop(seq, p);The other BPF iterators follow this contract:
bpf_map_seq_stop():
if (!v)
(void)__bpf_map_seq_show(seq, v, true);
task_seq_stop():
if (!v)
(void)__task_seq_show(seq, v, true);The same pattern appears in bpf_prog_seq_stop(),bpf_link_seq_stop(), task_file_seq_stop(), andtask_vma_seq_stop().
Without the in_stop callback, BPF programs attached to this iterator cannot detect when iteration has finished, which prevents finalization such as outputting aggregated results.
The __wakeup_source_seq_show() function already accepts the in_stop parameter but is never called with true:
Quoted 2 lines
kernel/bpf/wakeup_source_iter.c new file +2
-0+static int __wakeup_source_seq_show (struct seq_file *seq,void *v,bool in_stop) +{
[ ... ]
Quoted 1 line
kernel/bpf/wakeup_source_iter.c new file +1
-0+struct bpf_prog *prog =bpf_iter_get_info (&meta, in_stop);
[ ... ]
Quoted 1 line
+}
---
AI reviewed your patch. Please fix the bug or email reply why it's not a bug.
See: https://github.com/kernel-patches/vmtest/blob/master/ci/claude/README.md
CI run summary: https://github.com/kernel-patches/bpf/actions/runs/22416722500
AI-authorship-score: low
AI-authorship-explanation: The code follows standard kernel BPF iterator patterns with no unusual characteristics suggesting AI generation.
issues-found: 1
issue-severity-score: low
issue-severity-explanation: Missing end-of-iteration callback is an API consistency issue that prevents BPF programs from detecting iteration completion, but does not cause crashes or data corruption.
Re: [PATCH bpf-next v5 1/2] bpf: Add wakeup_source iterator
Quoted 7 lines · On Wed, Feb 25, 2026 at 2:03 PM <[email protected]> wrote:
Quoted 4 lines
kernel/bpf/wakeup_source_iter.c new file +0
-0[ ... ]
Quoted 12 lines
kernel/bpf/wakeup_source_iter.c line 45
Quoted 8 lines
kernel/bpf/wakeup_source_iter.c new file +8
-0+static void wakeup_source_iter_seq_stop (struct seq_file *seq,void *v) +{ +int *srcuidx = seq->private; + +if (*srcuidx >=0 ) +wakeup_sources_read_unlock (*srcuidx); + *srcuidx =-1 ; +}Should
wakeup_source_iter_seq_stop()call__wakeup_source_seq_show(seq, v, true)when v is NULL, before releasing the SRCU lock?
This is the same response as in v3 and v4, but I don't think it's necessary here since we don't need any additional output after finalization, such as printing a footer.
I understand it that bpf-next is now open for processing since the merge window is closed, so unless a human reviewer has a different opinion on this I'll ignore this AI feedback for now.
Quoted 1 line
[ ... ]
PATCH v5 2/2 selftests/bpf: Add tests for wakeup_sources
In reply to Samuel WuSets up the framework to test wakeup_sources iterators using BPF, and adds a few basic tests.
Adds several helper functions that for grabbing and releasing a wakelock, abstracting out key functions to setup a framework for testing wakeup_sources.
Additionally, adds 3 tests:
1. check_active_count: Checks that stats related to active_count are properly set after several lock/unlock cycles
2. check_sleep_times: Checks that time accounting related to sleep are properly calculated
3. check_no_infinite_reads: Checks that the iterator traversal returns NULL at the end
--- tools/testing/selftests/bpf/config | 1 + .../bpf/prog_tests/wakeup_source_iter.c | 300 ++++++++++++++++++ .../selftests/bpf/progs/wakeup_source_iter.c | 80 +++++ 3 files changed, 381 insertions(+) create mode 100644 tools/testing/selftests/bpf/prog_tests/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/progs/wakeup_source_iter.c
tools/testing/selftests/bpf/config +1 -0
@@ -111,6 +111,7 @@ CONFIG_IP6_NF_IPTABLES=y CONFIG_IP6_NF_FILTER=y CONFIG_NF_NAT=y CONFIG_PACKET=y +CONFIG_PM_WAKELOCKS=y CONFIG_RC_CORE=y CONFIG_SAMPLES=y CONFIG_SAMPLE_LIVEPATCH=m
tools/testing/selftests/bpf/prog_tests/wakeup_source_iter.c new file +300 -0
300 lines not on this page · raw message
tools/testing/selftests/bpf/progs/wakeup_source_iter.c new file +80 -0
80 lines not on this page · raw message
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
Quoted 50 lines · Samuel Wu
This patch series introduces BPF iterators for
wakeup_source, enabling BPF programs to efficiently traverse a device's wakeup sources.Currently, inspecting wakeup sources typically involves reading interfaces like
/sys/class/wakeup/*or debugfs. The repeated syscalls to query the sysfs nodes is inefficient, as there can be hundreds ofwakeup_sources, and each wakeup source have multiple stats, with one sysfs node per stat. debugfs is unstable and insecure.The iterators utilize pre-existing wakeup_sources_walk_* functions to traverse over the SRCU that backs the list of
wakeup_sources.Changes in v5:
- Add CORE definitions for*__localvariables to fix s390 build per bpf-ci
- v4 link: https://lore.kernel.org/all/[email protected]/Changes in v4:
- Proper init of variables inws_iter_check_sleep_times()test per bpf-ci
- Remove PM patch since it's already part of rebase
- v3 link: https://lore.kernel.org/all/[email protected]/Changes in v3:
- Updatewakeup_sources_walk_start()to handle an empty list per bpf-ci
- Simplify read of a struct's field in BPF program selftest per Andrii
- Drop open coded iterators forwakeup_sources
- Fix condition from !get_ws_iter_stream toget_ws_iter_streamin selftest
- Readevent_countinstead ofwakeup_countin selftest
- v2 link: https://lore.kernel.org/all/[email protected]/Changes in v2:
- Guard BPFMakefilewithCONFIG_PM_SLEEPto fix build errors
- Update copyright from 2025 to 2026
- v1 link: https://lore.kernel.org/all/[email protected]/Samuel Wu (2):
bpf: Addwakeup_sourceiterator
selftests/bpf: Add tests forwakeup_sourceskernel/bpf/Makefile | 3 + kernel/bpf/wakeup_source_iter.c | 103 ++++++ tools/testing/selftests/bpf/config | 1 + .../bpf/prog_tests/wakeup_source_iter.c | 300 ++++++++++++++++++ .../selftests/bpf/progs/wakeup_source_iter.c | 80 +++++ 5 files changed, 487 insertions(+) create mode 100644 kernel/bpf/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/prog_tests/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/progs/wakeup_source_iter.c
Quoted 3 lines
--
2.53.0.473.g4a7958ca14-goog
Hi Andrii, any thoughts on this patchset? I've addressed your suggestion from v2 of this patchset regarding selftest, but nothing fundamental has changed since then.
Thanks!
Sam
Kumar Kartikeya Dwivedi
maintainerQuoted 52 lines · Samuel Wu
Quoted 50 lines · Samuel Wu
This patch series introduces BPF iterators for
wakeup_source, enabling BPF programs to efficiently traverse a device's wakeup sources.Currently, inspecting wakeup sources typically involves reading interfaces like
/sys/class/wakeup/*or debugfs. The repeated syscalls to query the sysfs nodes is inefficient, as there can be hundreds ofwakeup_sources, and each wakeup source have multiple stats, with one sysfs node per stat. debugfs is unstable and insecure.The iterators utilize pre-existing wakeup_sources_walk_* functions to traverse over the SRCU that backs the list of
wakeup_sources.Changes in v5:
- Add CORE definitions for*__localvariables to fix s390 build per bpf-ci
- v4 link: https://lore.kernel.org/all/[email protected]/Changes in v4:
- Proper init of variables inws_iter_check_sleep_times()test per bpf-ci
- Remove PM patch since it's already part of rebase
- v3 link: https://lore.kernel.org/all/[email protected]/Changes in v3:
- Updatewakeup_sources_walk_start()to handle an empty list per bpf-ci
- Simplify read of a struct's field in BPF program selftest per Andrii
- Drop open coded iterators forwakeup_sources
- Fix condition from !get_ws_iter_stream toget_ws_iter_streamin selftest
- Readevent_countinstead ofwakeup_countin selftest
- v2 link: https://lore.kernel.org/all/[email protected]/Changes in v2:
- Guard BPFMakefilewithCONFIG_PM_SLEEPto fix build errors
- Update copyright from 2025 to 2026
- v1 link: https://lore.kernel.org/all/[email protected]/Samuel Wu (2):
bpf: Addwakeup_sourceiterator
selftests/bpf: Add tests forwakeup_sourceskernel/bpf/Makefile | 3 + kernel/bpf/wakeup_source_iter.c | 103 ++++++ tools/testing/selftests/bpf/config | 1 + .../bpf/prog_tests/wakeup_source_iter.c | 300 ++++++++++++++++++ .../selftests/bpf/progs/wakeup_source_iter.c | 80 +++++ 5 files changed, 487 insertions(+) create mode 100644 kernel/bpf/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/prog_tests/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/progs/wakeup_source_iter.c
Quoted 7 lines
Quoted 3 lines
--
2.53.0.473.g4a7958ca14-googHi Andrii, any thoughts on this patchset? I've addressed your suggestion from v2 of this patchset regarding selftest, but nothing fundamental has changed since then.
I think it's still unclear why this cannot be achieved by exploring what Andrii proposed initially in [0], i.e. by using existing primitives and adding extra kfuncs where necessary to solve bottlenecks, and I didn't see any indication where it was explored sufficiently.
I think all we need is the wakeup source read_lock()/unlock() APIs as KF_ACQUIRE/KF_RELEASE kfuncs. Just need to ensure we pair each lock with unlock by passing the index, which can be an opaque value returned from the kfunc (e.g. dummy empty struct pointer encoding the index). Then you can use bpf_for() to open-code the list iteration. The iterator would become unnecessary, and everything can be done from a syscall program's BPF_PROG_RUN invocation. I didn't
check how we can get the pointer to the wakeup_sources LIST_HEAD, but you could try several options (__ksym, bpf_kallsyms_lookup_name+bpf_core_cast and then walk it, or return pointer from a new kfunc and walk it using bpf_for()+bpf_core_cast()).
Also, how much speed up does this really provide? It is hard to understand the motivation from the current cover letter. Is the bottleneck due to opening multiple files in sysfs? You will still have to read text and parse text from the iterator's output in your current solution. Wouldn't it be faster if you just used a syscall program, and used binary output (using a ring buffer) to stream data to user space?
[0]: https://lore.kernel.org/all/CAEf4BzZv9kCAmX0Fo4kR8qh9mu5NB-wLDmNAK_R=VzcxL7VPpQ@mail.gmail.com/
Quoted 4 lines
Thanks!
Sam
Kumar Kartikeya Dwivedi
maintainerQuoted 68 lines · Kumar Kartikeya Dwivedi
Quoted 52 lines · Samuel Wu
Quoted 50 lines · Samuel Wu
This patch series introduces BPF iterators for
wakeup_source, enabling BPF programs to efficiently traverse a device's wakeup sources.Currently, inspecting wakeup sources typically involves reading interfaces like
/sys/class/wakeup/*or debugfs. The repeated syscalls to query the sysfs nodes is inefficient, as there can be hundreds ofwakeup_sources, and each wakeup source have multiple stats, with one sysfs node per stat. debugfs is unstable and insecure.The iterators utilize pre-existing wakeup_sources_walk_* functions to traverse over the SRCU that backs the list of
wakeup_sources.Changes in v5:
- Add CORE definitions for*__localvariables to fix s390 build per bpf-ci
- v4 link: https://lore.kernel.org/all/[email protected]/Changes in v4:
- Proper init of variables inws_iter_check_sleep_times()test per bpf-ci
- Remove PM patch since it's already part of rebase
- v3 link: https://lore.kernel.org/all/[email protected]/Changes in v3:
- Updatewakeup_sources_walk_start()to handle an empty list per bpf-ci
- Simplify read of a struct's field in BPF program selftest per Andrii
- Drop open coded iterators forwakeup_sources
- Fix condition from !get_ws_iter_stream toget_ws_iter_streamin selftest
- Readevent_countinstead ofwakeup_countin selftest
- v2 link: https://lore.kernel.org/all/[email protected]/Changes in v2:
- Guard BPFMakefilewithCONFIG_PM_SLEEPto fix build errors
- Update copyright from 2025 to 2026
- v1 link: https://lore.kernel.org/all/[email protected]/Samuel Wu (2):
bpf: Addwakeup_sourceiterator
selftests/bpf: Add tests forwakeup_sourceskernel/bpf/Makefile | 3 + kernel/bpf/wakeup_source_iter.c | 103 ++++++ tools/testing/selftests/bpf/config | 1 + .../bpf/prog_tests/wakeup_source_iter.c | 300 ++++++++++++++++++ .../selftests/bpf/progs/wakeup_source_iter.c | 80 +++++ 5 files changed, 487 insertions(+) create mode 100644 kernel/bpf/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/prog_tests/wakeup_source_iter.c create mode 100644 tools/testing/selftests/bpf/progs/wakeup_source_iter.cQuoted 7 lines
Quoted 3 lines
--
2.53.0.473.g4a7958ca14-googHi Andrii, any thoughts on this patchset? I've addressed your suggestion from v2 of this patchset regarding selftest, but nothing fundamental has changed since then.
I think it's still unclear why this cannot be achieved by exploring what Andrii proposed initially in [0], i.e. by using existing primitives and adding extra kfuncs where necessary to solve bottlenecks, and I didn't see any indication where it was explored sufficiently.
Quoted 5 lines
I think all we need is the wakeup source read_lock()/unlock() APIs as KF_ACQUIRE/KF_RELEASE kfuncs. Just need to ensure we pair each lock with unlock by passing the index, which can be an opaque value returned from the kfunc (e.g. dummy empty struct pointer encoding the index). Then you can use
bpf_for()to open-code the list iteration.
I had a longer discussion with Paul on this. When having multiple open SRCU sections, mixing nested calls doesn't flatten the protection scopes and leads to all kinds of issues, an example of which is in [0]. That said, in this case, the verifier isn't tracking what the underlying SRCU section protects or isn't even aware of it, all it needs to ensure is that section is closed eventually, out-of-order unlocks are ok, it would just exist to allow looking at the list in a stable fashion using safe bpf_core_cast() accesses.
So, I just wanted to clarify why I think this should be perfectly ok.
[0]: https://www.kernel.org/pub/linux/kernel/people/paulmck/Answers/RCU/SRCUhoha.html
(if you are like me, please follow the link at the bottom of [0] before feeling good about understanding the example).
Quoted 21 lines
The iterator would become unnecessary, and everything can be done from a syscall program's
BPF_PROG_RUNinvocation. I didn't
check how we can get the pointer to thewakeup_sourcesLIST_HEAD, but you could try several options (__ksym, bpf_kallsyms_lookup_name+bpf_core_cast and then walk it, or return pointer from a new kfunc and walk it using bpf_for()+bpf_core_cast()).Also, how much speed up does this really provide? It is hard to understand the motivation from the current cover letter. Is the bottleneck due to opening multiple files in sysfs? You will still have to read text and parse text from the iterator's output in your current solution. Wouldn't it be faster if you just used a syscall program, and used binary output (using a ring buffer) to stream data to user space?
[0]: https://lore.kernel.org/all/CAEf4BzZv9kCAmX0Fo4kR8qh9mu5NB-wLDmNAK_R=VzcxL7VPpQ@mail.gmail.com/
Quoted 4 lines
Thanks!
Sam
Quoted 2 lines · Kumar Kartikeya Dwivedi
On Sat, 7 Mar 2026 at 00:28, Kumar Kartikeya Dwivedi <[email protected]> wrote:
[...]
Quoted 6 lines
Quoted 6 lines
I think it's still unclear why this cannot be achieved by exploring what Andrii proposed initially in [0], i.e. by using existing primitives and adding extra kfuncs where necessary to solve bottlenecks, and I didn't see any indication where it was explored sufficiently.
This is my bad, I misinterpreted/misunderstood Andrii's original proposal to apply only to open-coded BPF iterators, hence the follow up patches with them removed. However, I don't disagree this can be achieved via open-coding. Both proposals are viable to traverse wakeup sources via BPF, however I see two distinct advantages of the BPF iterator:
1. Iterators provide a stable kernel interface that decouples the BPF code from the kernel's internal data structures and refactors
2. If wakeup sources are accessible via BPF, the next step should be to deprecate the existing interfaces for wakeup sources, such as the sysfs and debugfs stats. In that case, a text-based BPF iterator output will provide less friction during the transition.
Quoted 15 lines
Quoted 5 lines
I think all we need is the wakeup source read_lock()/unlock() APIs as KF_ACQUIRE/KF_RELEASE kfuncs. Just need to ensure we pair each lock with unlock by passing the index, which can be an opaque value returned from the kfunc (e.g. dummy empty struct pointer encoding the index). Then you can use
bpf_for()to open-code the list iteration.I had a longer discussion with Paul on this. When having multiple open SRCU sections, mixing nested calls doesn't flatten the protection scopes and leads to all kinds of issues, an example of which is in [0]. That said, in this case, the verifier isn't tracking what the underlying SRCU section protects or isn't even aware of it, all it needs to ensure is that section is closed eventually, out-of-order unlocks are ok, it would just exist to allow looking at the list in a stable fashion using safe
bpf_core_cast()accesses.
Quoted 6 lines
So, I just wanted to clarify why I think this should be perfectly ok.
[0]: https://www.kernel.org/pub/linux/kernel/people/paulmck/Answers/RCU/SRCUhoha.html
(if you are like me, please follow the link at the bottom of [0] before feeling good about understanding the example).
Appreciate the thoughtful feedback from you and Paul. The example was nuanced but well-explained by the post, however, a key difference I see is that the BPF iterator solution creates continuous locking segments during traversal, as opposed to hand-over-hand locking in the example provided. I could be missing something fundamental here- do you mind elaborating?
Quoted 8 lines
Quoted 8 lines
The iterator would become unnecessary, and everything can be done from a syscall program's
BPF_PROG_RUNinvocation. I didn't
check how we can get the pointer to thewakeup_sourcesLIST_HEAD, but you could try several options (__ksym, bpf_kallsyms_lookup_name+bpf_core_cast and then walk it, or return pointer from a new kfunc and walk it using bpf_for()+bpf_core_cast()).
Getting the static variable is key to the open-coded approach, but a clean way forward isn't obvious to me. Passing around raw addresses seems brittle and can raise security concerns. The wrapper kfunc to get the static address might be the best option, but it feels like it violates the power subsystem's encapsulation. The iterator approach abstracts all of this cleanly.
Quoted 7 lines
Quoted 7 lines
Also, how much speed up does this really provide? It is hard to understand the motivation from the current cover letter. Is the bottleneck due to opening multiple files in sysfs? You will still have to read text and parse text from the iterator's output in your current solution. Wouldn't it be faster if you just used a syscall program, and used binary output (using a ring buffer) to stream data to user space?
With the BPF iterator approach on a Pixel 6 in a quiescent state, I'm seeing a 10x speedup when traversing wakeup sources; the speedup would likely be even greater on a loaded system.
As alluded to earlier, the next step is to remove the /sys/class/wakeup node and debugfs nodes. My apologies for not explicitly listing this in the cover letter, but ultimately this would save memory by removing those nodes, eliminate work by not creating the nodes, and improve performance with a faster traversal, all while transparently replacing legacy interfaces.
Thanks!
Sam
[...]
Kumar Kartikeya Dwivedi
maintainerQuoted 21 lines · Samuel Wu
Quoted 2 lines · Kumar Kartikeya Dwivedi
On Sat, 7 Mar 2026 at 00:28, Kumar Kartikeya Dwivedi <[email protected]> wrote:
[...]
Quoted 6 lines
Quoted 6 lines
I think it's still unclear why this cannot be achieved by exploring what Andrii proposed initially in [0], i.e. by using existing primitives and adding extra kfuncs where necessary to solve bottlenecks, and I didn't see any indication where it was explored sufficiently.
This is my bad, I misinterpreted/misunderstood Andrii's original proposal to apply only to open-coded BPF iterators, hence the follow up patches with them removed. However, I don't disagree this can be achieved via open-coding. Both proposals are viable to traverse wakeup sources via BPF, however I see two distinct advantages of the BPF iterator:
Quoted 2 lines
1. Iterators provide a stable kernel interface that decouples the BPF code from the kernel's internal data structures and refactors
I think this is my concern; committing to a stable interface that we'll have to maintain forever.
It's a narrow use case that can be better served by dedicated kfuncs and rolling your own solution.
There will be no end to the number of iterators if we keep making one for every linked list in the kernel.
It would be different if the synchronization scheme for the wakeup_sources list was really complicated and hard to express inline in the verifier, but that is not the case.
Quoted 5 lines
2. If wakeup sources are accessible via BPF, the next step should be to deprecate the existing interfaces for wakeup sources, such as the sysfs and debugfs stats. In that case, a text-based BPF iterator output will provide less friction during the transition.
I would agree if this was a one-to-one conversion, where you just replace the directory for look up.
But you'll be reading from one file vs several, so it's already a non-trivial change in the consumer.
At that point, the conversion burden doesn't have the same weight in deciding whether to stick with textual representation.
Quoted 22 lines
Quoted 15 lines
Quoted 5 lines
I think all we need is the wakeup source read_lock()/unlock() APIs as KF_ACQUIRE/KF_RELEASE kfuncs. Just need to ensure we pair each lock with unlock by passing the index, which can be an opaque value returned from the kfunc (e.g. dummy empty struct pointer encoding the index). Then you can use
bpf_for()to open-code the list iteration.I had a longer discussion with Paul on this. When having multiple open SRCU sections, mixing nested calls doesn't flatten the protection scopes and leads to all kinds of issues, an example of which is in [0]. That said, in this case, the verifier isn't tracking what the underlying SRCU section protects or isn't even aware of it, all it needs to ensure is that section is closed eventually, out-of-order unlocks are ok, it would just exist to allow looking at the list in a stable fashion using safe
bpf_core_cast()accesses.Quoted 6 lines
So, I just wanted to clarify why I think this should be perfectly ok.
[0]: https://www.kernel.org/pub/linux/kernel/people/paulmck/Answers/RCU/SRCUhoha.html
(if you are like me, please follow the link at the bottom of [0] before feeling good about understanding the example).
Quoted 6 lines
Appreciate the thoughtful feedback from you and Paul. The example was nuanced but well-explained by the post, however, a key difference I see is that the BPF iterator solution creates continuous locking segments during traversal, as opposed to hand-over-hand locking in the example provided. I could be missing something fundamental here- do you mind elaborating?
It was just an illustration to identify what guarantees the verifier would need to enforce.
The most we need to do is just ensure SRCU indices are passed back during unlock, in whatever order.
Critical section will be using bpf_core_cast() anyway to do safe walking of the list.
With the KF_ACQUIRE/KF_RELEASE kfuncs you will still have a continuous locking segment.
Just enclose your list walking loop between lock() and unlock().
Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
The iterator would become unnecessary, and everything can be done from a syscall program's
BPF_PROG_RUNinvocation. I didn't
check how we can get the pointer to thewakeup_sourcesLIST_HEAD, but you could try several options (__ksym, bpf_kallsyms_lookup_name+bpf_core_cast and then walk it, or return pointer from a new kfunc and walk it using bpf_for()+bpf_core_cast()).
Quoted 3 lines
Getting the static variable is key to the open-coded approach, but a clean way forward isn't obvious to me. Passing around raw addresses seems brittle and can raise security concerns. The wrapper kfunc to
I don't understand the security concern. All of this happens inside the BPF program.
If you're writing these addresses to user space and leaking them, it's your bug. You can do that with iterators too. You have all the raw addresses available to you.
Quoted 12 lines
get the static address might be the best option, but it feels like it violates the power subsystem's encapsulation. The iterator approach abstracts all of this cleanly.
Quoted 7 lines
Quoted 7 lines
Also, how much speed up does this really provide? It is hard to understand the motivation from the current cover letter. Is the bottleneck due to opening multiple files in sysfs? You will still have to read text and parse text from the iterator's output in your current solution. Wouldn't it be faster if you just used a syscall program, and used binary output (using a ring buffer) to stream data to user space?
Quoted 4 lines
With the BPF iterator approach on a Pixel 6 in a quiescent state, I'm seeing a 10x speedup when traversing wakeup sources; the speedup would likely be even greater on a loaded system.
I'm sure you can replicate it with the approach described above. It might even be faster than parsing text from the file.
Quoted 6 lines
As alluded to earlier, the next step is to remove the
/sys/class/wakeupnode and debugfs nodes. My apologies for not explicitly listing this in the cover letter, but ultimately this would save memory by removing those nodes, eliminate work by not creating the nodes, and improve performance with a faster traversal, all while transparently replacing legacy interfaces.
And you can do that regardless of the type of solution chosen above.
Quoted 5 lines
Thanks!
Sam[...]
Quoted 33 lines · Kumar Kartikeya Dwivedi
Quoted 21 lines · Samuel Wu
Quoted 2 lines · Kumar Kartikeya Dwivedi
On Sat, 7 Mar 2026 at 00:28, Kumar Kartikeya Dwivedi <[email protected]> wrote:
[...]
Quoted 6 lines
Quoted 6 lines
I think it's still unclear why this cannot be achieved by exploring what Andrii proposed initially in [0], i.e. by using existing primitives and adding extra kfuncs where necessary to solve bottlenecks, and I didn't see any indication where it was explored sufficiently.
This is my bad, I misinterpreted/misunderstood Andrii's original proposal to apply only to open-coded BPF iterators, hence the follow up patches with them removed. However, I don't disagree this can be achieved via open-coding. Both proposals are viable to traverse wakeup sources via BPF, however I see two distinct advantages of the BPF iterator:
Quoted 2 lines
1. Iterators provide a stable kernel interface that decouples the BPF code from the kernel's internal data structures and refactors
I think this is my concern; committing to a stable interface that we'll have to maintain forever.
It's a narrow use case that can be better served by dedicated kfuncs and rolling your own solution.
There will be no end to the number of iterators if we keep making one for every linked list in the kernel.
Quoted 4 lines
It would be different if the synchronization scheme for the
wakeup_sourceslist was really complicated and hard to express inline in the verifier, but that is not the case.
I completely agree it's a viable approach, it seems to come down to where/who carries the maintenance burden. Regardless, let me explore how to prototype this cleanly to handle the details like the static address, and then we will have something concrete for comparisons. I appreciate the alternative suggestion and the feedback here.
Quoted 6 lines
Quoted 5 lines
2. If wakeup sources are accessible via BPF, the next step should be to deprecate the existing interfaces for wakeup sources, such as the sysfs and debugfs stats. In that case, a text-based BPF iterator output will provide less friction during the transition.
Quoted 6 lines
I would agree if this was a one-to-one conversion, where you just replace the directory for look up.
But you'll be reading from one file vs several, so it's already a non-trivial change in the consumer.
At that point, the conversion burden doesn't have the same weight in deciding whether to stick with textual representation.
My understanding is that most (all?) current consumers ultimately want the aggregate list of wakeup source stats, not a single node in isolation; the reason there are multiple nodes is to adhere to the precedent that sysfs nodes contain a singular stat. A single text-based output node with the aggregated stats would be a straightforward change that simplifies userspace code consuming these stats.
Quoted 93 lines
Quoted 22 lines
Quoted 15 lines
Quoted 5 lines
I think all we need is the wakeup source read_lock()/unlock() APIs as KF_ACQUIRE/KF_RELEASE kfuncs. Just need to ensure we pair each lock with unlock by passing the index, which can be an opaque value returned from the kfunc (e.g. dummy empty struct pointer encoding the index). Then you can use
bpf_for()to open-code the list iteration.I had a longer discussion with Paul on this. When having multiple open SRCU sections, mixing nested calls doesn't flatten the protection scopes and leads to all kinds of issues, an example of which is in [0]. That said, in this case, the verifier isn't tracking what the underlying SRCU section protects or isn't even aware of it, all it needs to ensure is that section is closed eventually, out-of-order unlocks are ok, it would just exist to allow looking at the list in a stable fashion using safe
bpf_core_cast()accesses.Quoted 6 lines
So, I just wanted to clarify why I think this should be perfectly ok.
[0]: https://www.kernel.org/pub/linux/kernel/people/paulmck/Answers/RCU/SRCUhoha.html
(if you are like me, please follow the link at the bottom of [0] before feeling good about understanding the example).
Quoted 6 lines
Appreciate the thoughtful feedback from you and Paul. The example was nuanced but well-explained by the post, however, a key difference I see is that the BPF iterator solution creates continuous locking segments during traversal, as opposed to hand-over-hand locking in the example provided. I could be missing something fundamental here- do you mind elaborating?
It was just an illustration to identify what guarantees the verifier would need to enforce.
The most we need to do is just ensure SRCU indices are passed back during unlock, in whatever order.
Critical section will be usingbpf_core_cast()anyway to do safe walking of the list.
With the KF_ACQUIRE/KF_RELEASE kfuncs you will still have a continuous locking segment.
Just enclose your list walking loop betweenlock()andunlock().Quoted 10 lines
Quoted 9 lines
Quoted 9 lines
The iterator would become unnecessary, and everything can be done from a syscall program's
BPF_PROG_RUNinvocation. I didn't
check how we can get the pointer to thewakeup_sourcesLIST_HEAD, but you could try several options (__ksym, bpf_kallsyms_lookup_name+bpf_core_cast and then walk it, or return pointer from a new kfunc and walk it using bpf_for()+bpf_core_cast()).Quoted 3 lines
Getting the static variable is key to the open-coded approach, but a clean way forward isn't obvious to me. Passing around raw addresses seems brittle and can raise security concerns. The wrapper kfunc to
I don't understand the security concern. All of this happens inside the BPF program.
If you're writing these addresses to user space and leaking them, it's your bug. You can do that with iterators too. You have all the raw addresses available to you.Quoted 12 lines
get the static address might be the best option, but it feels like it violates the power subsystem's encapsulation. The iterator approach abstracts all of this cleanly.
Quoted 7 lines
Quoted 7 lines
Also, how much speed up does this really provide? It is hard to understand the motivation from the current cover letter. Is the bottleneck due to opening multiple files in sysfs? You will still have to read text and parse text from the iterator's output in your current solution. Wouldn't it be faster if you just used a syscall program, and used binary output (using a ring buffer) to stream data to user space?
Quoted 4 lines
With the BPF iterator approach on a Pixel 6 in a quiescent state, I'm seeing a 10x speedup when traversing wakeup sources; the speedup would likely be even greater on a loaded system.
I'm sure you can replicate it with the approach described above. It might even be faster than parsing text from the file.
Quoted 6 lines
As alluded to earlier, the next step is to remove the
/sys/class/wakeupnode and debugfs nodes. My apologies for not explicitly listing this in the cover letter, but ultimately this would save memory by removing those nodes, eliminate work by not creating the nodes, and improve performance with a faster traversal, all while transparently replacing legacy interfaces.And you can do that regardless of the type of solution chosen above.
Quoted 5 lines
Thanks!
Sam[...]