bpf: use cond_resched_tasks_rcu_qs in bpf_fd_array_map_clear() loop
PATCH 1/1 bpf: use cond_resched_tasks_rcu_qs in bpf_fd_array_map_clear() loop
syzkaller creates a PROG_ARRAY with huge max_entries and triggers perf tracepoint open close in parallel. The hung task detector reports "INFO: task hung in perf_tp_event_init" with event_mutex held waiting for synchronize_rcu_tasks().
bpf_fd_array_map_clear walks max_entries under RCU read lock on a workqueue kworker. Commit 4406942e65ca added cond_resched for classic RCU, but under PREEMPT it is a no-op and a preempted kworker never reports RCU-tasks QS. The loop then blocks the grace period while event_mutex is held in ftrace_shutdown, stalling perf_event_open().
Use cond_resched_tasks_rcu_qs to report RCU-tasks QS independent of preemption model. The grace period completes and event_mutex is released normally.
---
QUESTION: should we change cond_resched() instead, so it unblocks RCU in any preempt configuration where cond_resched() is a noop?
That would take care of every bug of this shape with one change, and rcu_tasks_qs() looks cheap enough?
kernel/bpf/arraymap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
kernel/bpf/arraymap.c +2 -1
@@ -1015,7 +1015,8 @@ static void bpf_fd_array_map_clear(struct bpf_map *map, bool need_defer)for (i =0 ; i < array->map.max_entries; i++) {__fd_array_map_delete_elem (map, &i, need_defer); -cond_resched (); +/* cond_resched() is a noop with preempt; unblock RCU */ +cond_resched_tasks_rcu_qs (); } }
Signature · 1 line
2.53.0-Meta
[email protected]
AI reviewThank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 4 potential issue(s) to consider:
New issues:
- [Medium] The commit message incorrectly claims that bpf_fd_array_map_clear() executes 'under RCU read lock'.
Pre-existing issues:
- [High] perf_event_fd_array_release() iterates over the entire map under a single rcu_read_lock() region, causing Classic RCU stalls for large maps.
- [High] The patch only fixes the RCU-tasks stall in bpf_fd_array_map_clear(), leaving identical vulnerabilities in other BPF map teardown functions.
- [High] A Time-Of-Check to Time-Of-Use race in perf_event_fd_array_release can maliciously delete concurrently updated perf events. --
--- Patch [1]: [PATCH + QUESTION] bpf: use cond_resched_tasks_rcu_qs in bpf_fd_array_map_clear() loop --- commit c19063bda5ebcdaccc0217e960d2910817786a73
Author: Rik van Riel <[email protected]>
bpf: use cond_resched_tasks_rcu_qs in bpf_fd_array_map_clear() loop
syzkaller creates a PROG_ARRAY with huge max_entries and triggers perf tracepoint open close in parallel. The hung task detector reports "INFO: task hung in perf_tp_event_init" with event_mutex held waiting for synchronize_rcu_tasks().
Quoted 5 lines
bpf_fd_array_map_clearwalksmax_entriesunder RCU read lock on a workqueue kworker. Commit4406942e65caaddedcond_reschedfor classic RCU, but under PREEMPT it is a no-op and a preempted kworker never reports RCU-tasks QS. The loop then blocks the grace period whileevent_mutexis held inftrace_shutdown, stallingperf_event_open().
[Severity: Medium]
This isn't a bug introduced by this patch, but the commit message incorrectly claims that bpf_fd_array_map_clear() executes 'under RCU read lock'.
bpf_fd_array_map_clear() is executed via map destruction callbacks (e.g., prog_array_map_clear_deferred) on a sleepable workqueue context without holding rcu_read_lock(). If a classic RCU read lock were actually held, calling cond_resched_tasks_rcu_qs() (which invokes cond_resched()) would be an illegal sleeping operation and trigger might_sleep() lockdep splats.
Could the commit message be updated to reflect the actual context?
[ ... ]
kernel/bpf/arraymap.c static void bpf_fd_array_map_clear(struct bpf_map *map, bool need_defer) · line 1021
Quoted 13 lines
kernel/bpf/arraymap.c +2
-1@@ -1015,7 +1015,8 @@ static void bpf_fd_array_map_clear(struct bpf_map *map, bool need_defer)for (i =0 ; i < array->map.max_entries; i++) {__fd_array_map_delete_elem (map, &i, need_defer); -cond_resched (); +/* cond_resched() is a noop with preempt; unblock RCU */ +cond_resched_tasks_rcu_qs (); } }
[Severity: High]
This isn't a bug introduced by this patch, but the fix only addresses the RCU-tasks stall in bpf_fd_array_map_clear(), leaving similar issues in other BPF map teardown functions.
For instance, fd_htab_map_free() in kernel/bpf/hashtab.c loops over all buckets without any yield point:
for (i = 0; i < htab->n_buckets; i++) {
head = select_bucket(htab, i);
...
}Also, bpf_array_free_percpu() in kernel/bpf/arraymap.c loops over max_entries using only cond_resched():
for (i = 0; i < array->map.max_entries; i++) {
free_percpu(array->pptrs[i]);
cond_resched();
}Since cond_resched() is a noop under CONFIG_PREEMPT, wouldn't these loops similarly stall RCU Tasks when freeing huge hash or per-cpu array maps?
[Severity: High]
This is another pre-existing issue, but while we are looking at arraymap teardown, perf_event_fd_array_release() also has a potential issue.
It iterates over the entire map under a single rcu_read_lock() region:
rcu_read_lock();
for (i = 0; i < array->map.max_entries; i++) {
ee = READ_ONCE(array->ptrs[i]);
if (ee && ee->map_file == map_file)
__fd_array_map_delete_elem(map, &i, true);
}
rcu_read_unlock();Does this cause Classic RCU stalls for large maps, since it prevents grace periods from completing without any yield points?
[Severity: High]
Additionally, there seems to be a pre-existing Time-Of-Check to Time-Of-Use race in perf_event_fd_array_release() calling __fd_array_map_delete_elem().
If Thread A closes a map file, it triggers perf_event_fd_array_release(), which verifies ee->map_file == map_file.
If Thread B concurrently calls bpf_map_update_elem to replace the element, Thread A then calls __fd_array_map_delete_elem(), which unconditionally executes:
old_ptr = xchg(array->ptrs + index, NULL);
Does this mean Thread A could unconditionally nullify Thread B's newly inserted element, incorrectly deleting it from the map?
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/20260715215314.44423f47@fangorn?part=1
Quoted 20 lines · Rik van Riel
syzkaller creates a
PROG_ARRAYwith hugemax_entriesand triggers perf tracepoint open close in parallel. The hung task detector reports "INFO: task hung inperf_tp_event_init" withevent_mutexheld waiting forsynchronize_rcu_tasks().
bpf_fd_array_map_clearwalksmax_entriesunder RCU read lock on a workqueue kworker. Commit4406942e65caaddedcond_reschedfor classic RCU, but under PREEMPT it is a no-op and a preempted kworker never reports RCU-tasks QS. The loop then blocks the grace period whileevent_mutexis held inftrace_shutdown, stallingperf_event_open().Use
cond_resched_tasks_rcu_qsto report RCU-tasks QS independent of preemption model. The grace period completes andevent_mutexis released normally.Fixes da765a2f5993 ("bpf: Add poke dependency tracking for prog array maps")Fixes 4406942e65ca ("bpf: Fix RCU stall in bpf_fd_array_map_clear()")Signed-off-by Rik van Riel <[email protected]>Assisted-by Claude:claude-opus-4-8
Quoted 6 lines
---
QUESTION: should we changecond_resched()instead, so it unblocks RCU in any preempt configuration wherecond_resched()is a noop?That would take care of every bug of this shape with one change, and
rcu_tasks_qs()looks cheap enough?
Just confirming that rcu_tasks_qs() is quite cheap: Check a constant, check a flag in the task_struct structure, and, if set, set another flag in this same structure. If there is no RCU Tasks grace period in effect, the first flag will always be zero.
Thanx, Paul
Quoted 21 lines
kernel/bpf/arraymap.c | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)kernel/bpf/arraymap.c +2
-1@@ -1015,7 +1015,8 @@ static void bpf_fd_array_map_clear(struct bpf_map *map, bool need_defer)for (i =0 ; i < array->map.max_entries; i++) {__fd_array_map_delete_elem (map, &i, need_defer); -cond_resched (); +/* cond_resched() is a noop with preempt; unblock RCU */ +cond_resched_tasks_rcu_qs (); } }Signature · 1 line
2.53.0-Meta
Quoted 6 lines · Rik van Riel
syzkaller creates a
PROG_ARRAYwith hugemax_entriesand triggers perf
tracepoint open close in parallel. The hung task detector reports "INFO: task hung inperf_tp_event_init" withevent_mutexheld waiting forsynchronize_rcu_tasks().
OK, this is more interesting than it looked at first, and this patch does not look like the right approach, but I'm also not sure what other approach would work :(
The actual error being thrown is a hung task:
The syzkaller kernel failure is a tragedy in 3 parts.
First, we have perf_event_open() waiting on the event_mutex:
syz.7.35551 (pid 31354, state D): perf_event_open
-> perf_tp_event_init
-> perf_trace_init
-> __mutex_lock
-> blocked on event_mutex.Second, we have perf_trace_destroy() indirectly waiting in synchronize_rcu_tasks(), with the event_mutex held.
syz.5.35491 (state:I) do_exit
-> task_work_run
-> __fput
-> perf_release
-> perf_event_release_kernel
-> __free_event
-> perf_trace_destroy
-> perf_trace_event_close
-> reg(TRACE_REG_PERF_CLOSE)
-> perf_ftrace_event_register
-> perf_ftrace_function_unregister
-> unregister_ftrace_function
-> ftrace_shutdown
-> synchronize_rcu_tasks() <-- BLOCKS HERE, forever,holding event_mutex
Tasks RCU waits for every task to have gone through a voluntary reschedule, before the grace period can be advanced. A preemption does not count.
Third, we have a kworker looping for a very long time:
kworker/0:0 (pid 9, TASK_RUNNING, on CPU0): process_one_work -> prog_array_map_clear_deferred
-> bpf_fd_array_map_clear
-> __fd_array_map_delete_elem. Looping over a huge PROG_ARRAY
max_entries.
With CONFIG_PREEMPT_LAZY=y this gets preempted, but since involuntary preemptions do not count for tasks RCU, that does nothnig to help ftrace_shutdown() get unstuck.
How do we solve this?
Do we actually need to have cond_resched() points quiesce the task RCU state when running with lazy preempt, since those are points where we could preempt voluntarily?
Do we need to prohibit calling synchronize_rcu_tasks() while holding a mutex?
Do we need to do something else?
What is the best way forward here?
Signature · 1 line
All Rights Reversed.
Quoted 72 lines · Rik van Riel
Quoted 6 lines · Rik van Riel
syzkaller creates a
PROG_ARRAYwith hugemax_entriesand triggers perf
tracepoint open close in parallel. The hung task detector reports "INFO: task hung inperf_tp_event_init" withevent_mutexheld waiting forsynchronize_rcu_tasks().OK, this is more interesting than it looked at first, and this patch does not look like the right approach, but I'm also not sure what other approach would work :(
The actual error being thrown is a hung task:
The syzkaller kernel failure is a tragedy in 3 parts.
First, we have
perf_event_open()waiting on theevent_mutex:syz.7.35551 (pid 31354, state D): perf_event_open -> perf_tp_event_init -> perf_trace_init -> __mutex_lock -> blocked on event_mutex.Second, we have
perf_trace_destroy()indirectly waiting insynchronize_rcu_tasks(), with theevent_mutexheld.syz.5.35491 (state:I) do_exit -> task_work_run -> __fput -> perf_release -> perf_event_release_kernel -> __free_event -> perf_trace_destroy -> perf_trace_event_close -> reg(TRACE_REG_PERF_CLOSE) -> perf_ftrace_event_register -> perf_ftrace_function_unregister -> unregister_ftrace_function -> ftrace_shutdown -> synchronize_rcu_tasks() <-- BLOCKS HERE, forever,holding
event_mutexTasks RCU waits for every task to have gone through a voluntary reschedule, before the grace period can be advanced. A preemption does not count.
Third, we have a kworker looping for a very long time:
kworker/0:0 (pid 9,TASK_RUNNING, on CPU0):process_one_work->prog_array_map_clear_deferred
->bpf_fd_array_map_clear-> __fd_array_map_delete_elem. Looping over a huge PROG_ARRAY
max_entries.With
CONFIG_PREEMPT_LAZY=ythis gets preempted, but since involuntary preemptions do not count for tasks RCU, that does nothnig to helpftrace_shutdown()get unstuck.How do we solve this?
Do we actually need to have
cond_resched()points quiesce the task RCU state when running with lazy preempt, since those are points where we could preempt voluntarily?
Quoted 4 lines
Do we need to prohibit calling
synchronize_rcu_tasks()while holding a mutex?Do we need to do something else?
The traditional approach is cond_resched_tasks_rcu_qs(), which is to synchronize_rcu_tasks() as cond_resched() is to synchronize_rcu().
But if we can avoid holding mutexes across synchronize_rcu_tasks(), that would be a very good thing. The synchronize_rcu_tasks() latency can be many minutes, which isn't usually what you want when blocking on a mutex.
Or maybe we want a way of sleeping on a mutex at TASK_IDLE.
Quoted 1 line
What is the best way forward here?
Excellent question!!!
Thanx, Paul