bpf: add rcu_read_lock_trace to bpf_timer_cb
PATCH 1/1 bpf: add rcu_read_lock_trace to bpf_timer_cb
bpf_timer_cb() invokes the BPF callback without holding rcu_read_lock_trace(), unlike bpf_wq_work() which correctly wraps its callback in a trace RCU reader section. The free path in bpf_async_cb_rcu_tasks_trace_free() uses call_rcu_tasks_trace() to wait for running callbacks before freeing bpf_async_cb, but this synchronization is ineffective for timers since they never enter a trace RCU reader section.
Add rcu_read_lock_trace()/rcu_read_unlock_trace() around callback_fn() in bpf_timer_cb() to match the bpf_wq_work() pattern. migrate_disable() is not needed because the timer callback already runs in softirq context which prevents migration.
--- kernel/bpf/helpers.c | 2 ++ 1 file changed, 2 insertions(+)
kernel/bpf/helpers.c +2 -0
@@ -1204,7 +1204,9 @@ static enum hrtimer_restart bpf_timer_cb(struct hrtimer *hrtimer) key =map_key_from_value (map, value, &idx); +rcu_read_lock_trace ();callback_fn ((u64 )(long )map, (u64 )(long )key, (u64 )(long )value,0 ,0 ); +rcu_read_unlock_trace ();/* The verifier checked that return value is zero. */ this_cpu_write (hrtimer_running,NULL );
Signature · 1 line
2.50.1 (Apple Git-155)
Quoted 8 lines · Zhenghang Xiao
bpf_timer_cb()invokes the BPF callback without holdingrcu_read_lock_trace(), unlikebpf_wq_work()which correctly wraps its callback in a trace RCU reader section. The free path inbpf_async_cb_rcu_tasks_trace_free()usescall_rcu_tasks_trace()to wait for running callbacks before freeingbpf_async_cb, but this synchronization is ineffective for timers since they never enter a trace RCU reader section.
Quoted 8 lines
Add rcu_read_lock_trace()/rcu_read_unlock_trace() around
callback_fn()inbpf_timer_cb()to match thebpf_wq_work()pattern.migrate_disable()is not needed because the timer callback already runs in softirq context which prevents migration.Fixes 1bfbc267ec91 ("bpf: Enable bpf_timer and bpf_wq in any context")Signed-off-by Zhenghang Xiao <[email protected]>---
Is there any concrete issue you are running into?
If I remember correctly, there is no explicit rcu read lock, because
the timer callback bpf_timer_cb() assumes RCU lock already (see rcu_read_lock_bh_held()). similarly to migrate_disable(), as you mention in the commit message.
Quoted 17 lines
kernel/bpf/helpers.c | 2 ++ 1 file changed, 2 insertions(+)kernel/bpf/helpers.c +2
-0@@ -1204,7 +1204,9 @@ static enum hrtimer_restart bpf_timer_cb(struct hrtimer *hrtimer) key =map_key_from_value (map, value, &idx); +rcu_read_lock_trace ();callback_fn ((u64 )(long )map, (u64 )(long )key, (u64 )(long )value,0 ,0 ); +rcu_read_unlock_trace ();/* The verifier checked that return value is zero. */ this_cpu_write (hrtimer_running,NULL );