uprobes: Switch uretprobes_srcu to SRCU-fast-updown
Status from this message: The following commit has been merged into the perf/core branch of tip:
Get series with b4
b4 am [email protected]Versions newest first
Patches in v1 Applied
| # | Subject and files | Size | Replies | Applied |
|---|---|---|---|---|
| 1/2 | srcu: Add lock guard for srcu_fast_updown flavorinclude/ | +7 | 2 replies | |
| 2/2 | uprobes: Switch uretprobes_srcu to SRCU-fast-updowninclude/ kernel/ | +20 | 5 replies |
Reviews trailers per patch
All 2 patches have a Reviewed-by or Acked-by.
| Patch | Paul E. McKenney | Andrii Nakryiko | Oleg Nesterov |
|---|---|---|---|
| 1/2 | R-b | R-b | |
| 2/2 | A-b | R-b |
Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes. Scroll the table sideways for all 3 reviewers.
Cover letter [PATCH bpf-next 0/2] uprobes: Switch uretprobes_srcu to SRCU-fast-updown
uretprobes_srcu currently uses normal SRCU. Normal SRCU issues an smp_mb() on both srcu_read_lock() and srcu_read_unlock(), i.e. two full memory barriers per read-side critical section. For uretprobes this cost is paid on every uretprobe invocation: prepare_uretprobe() takes the read lock that is later dropped on the normal return path (hprobe_finalize()) or from ri_timer()/dup_utask() (hprobe_expire()).
Switch uretprobes_srcu to the SRCU-fast-updown flavor. SRCU-fast moves the read-side ordering off the reader and onto the (rare) grace-period
side: synchronize_srcu() rides on synchronize_rcu() instead of relying on reader-side smp_mb(). This is a good trade for uretprobes, where reader-side hits vastly outnumber grace periods (uprobe unregistration).
The updown variant (rather than plain SRCU-fast) is required because the read lock is acquired in prepare_uretprobe() on the way out to user space and is released only once the return instance is finalized -- from a different context than it was taken: the normal return path (uprobe_handle_trampoline() -> hprobe_finalize()), the timer callback, or the fork path (ri_timer()/dup_utask() -> hprobe_expire()). srcu_down_read_fast()/srcu_up_read_fast() are designed for this semaphore-like, cross-context pattern and, unlike the same-context srcu_read_lock_fast() variant, do not carry lockdep read-side tracking that would warn on it -- which is why the old code had to use the raw __srcu_read_lock() here. For the short, same-context sections in ri_timer() and dup_utask(), guard(srcu_fast_updown) is used instead, giving proper lockdep coverage.
Patch 1 adds the guard(srcu_fast_updown) definition, following the existing guard(srcu)/guard(srcu_fast) pattern.
Patch 2 does the uretprobes_srcu conversion.
Note
----
Only uretprobes_srcu is converted; the main uprobe readers (RB-tree lookup and consumer-list iteration) are deliberately left on RCU Tasks Trace. RCU Tasks Trace is already implemented on top of srcu_read_lock_fast(), so the reader-side cost is identical, and it has a nesting fast path that the uprobe -> sleepable-BPF-program call chain relies on (the BPF trampoline takes rcu_read_lock_trace() while uprobes already holds it; the nested acquire is just a counter bump). Converting those readers to a separate srcu_struct would turn one real + one nested lock into two real locks and lose that optimization for no reader-side gain. uretprobes_srcu is different: it uses normal SRCU (not Tasks Trace), its readers are long-lived and cross-context, and it genuinely benefits from dropping the per-reader barriers.
Puranjay Mohan (2):
srcu: Add lock guard for srcu_fast_updown flavor
uprobes: Switch uretprobes_srcu to SRCU-fast-updown
include/linux/srcu.h | 7 +++++++ include/linux/uprobes.h | 5 +++-- kernel/events/uprobes.c | 29 +++++++++++++++++------------ 3 files changed, 27 insertions(+), 14 deletions(-)
base-commit: 87bfe634b1193db90e5170e1ddbad04a63ef4501
Signature · 1 line
2.53.0-Meta