Skip to content

RFC RCU: Enable callbacks to benefit from expedited grace periods

Puranjay Mohan linux-trace-kernel lkml rcu 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 RFC Latest10 patches 8 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/10rcu/segcblist: Add SRCU and Tasks RCU wrapper functionskernel/rcu/rcu_segcblist.c kernel/rcu/rcu_segcblist.h kernel/rcu/srcutree.c +1 more+23 -110 replies
2/10rcu/segcblist: Factor out rcu_segcblist_advance_compact() helperkernel/rcu/rcu_segcblist.c+32 -180 replies
3/10rcu/segcblist: Change gp_seq to struct rcu_gp_oldstate gp_seq_fullinclude/linux/rcu_segcblist.h include/trace/events/rcu.h kernel/rcu/rcu_segcblist.c +3 more+53 -442 replies
4/10rcu: Add RCU_GET_STATE_NOT_TRACKED for subsystems without expedited GPskernel/rcu/rcu.h kernel/rcu/tree.c+13 -30 replies
5/10rcu: Enable RCU callbacks to benefit from expedited grace periodskernel/rcu/rcu_segcblist.c kernel/rcu/rcu_segcblist.h kernel/rcu/tree.c +1 more+51 -231 reply
6/10rcu: Update comments for gp_seq_full and expedited GP trackinginclude/linux/rcu_segcblist.h kernel/rcu/rcu_segcblist.c kernel/rcu/tree.c+39 -240 replies
7/10rcu: Wake NOCB rcuog kthreads on expedited grace period completionkernel/rcu/tree.h kernel/rcu/tree_exp.h kernel/rcu/tree_nocb.h+31 -02 replies
8/10rcu: Detect expedited grace period completion in rcu_pending()kernel/rcu/tree.c+7 -01 reply
9/10rcu: Advance callbacks for expedited GP completion in rcu_core()kernel/rcu/tree.c+16 -00 replies
10/10rcuscale: Add concurrent expedited GP threads for callback scaling testskernel/rcu/rcuscale.c+82 -20 replies

Reviews trailers per patch

All 10 patches have a Reviewed-by or Acked-by.

PatchPaul E. McKenney
1/10R-b
2/10R-b
3/10R-b
4/10R-b
5/10R-b
6/10R-b
7/10R-b
8/10R-b
9/10R-b
10/10R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [RFC PATCH 00/10] RCU: Enable callbacks to benefit from expedited grace periods

RCU callbacks only track normal grace period sequence numbers. This means callbacks must wait for normal GPs even when expedited GPs have already elapsed.

This series tracks both normal and expedited GP sequences in the callback infrastructure using struct rcu_gp_oldstate, so callbacks advance when either GP type completes.

Commits 1-4 prepare the callback infrastructure: SRCU and Tasks RCU are isolated behind wrapper functions, the segment compaction logic is factored out for reuse, gp_seq is widened to struct rcu_gp_oldstate, and RCU_GET_STATE_NOT_TRACKED is added for subsystems that do not track expedited GPs.

Commit 5 is the core change: rcu_segcblist_advance() checks both normal and expedited GP completion via poll_state_synchronize_rcu_full(), and rcu_accelerate_cbs() captures both GP sequences via get_state_synchronize_rcu_full(). SRCU wrappers become standalone implementations since they cannot use poll_state_synchronize_rcu_full() which reads RCU-specific globals.

Commits 7-9 fix three notification paths so that expedited GP completion triggers callback processing: NOCB rcuog kthreads are woken, rcu_pending() detects the completion, and rcu_core() advances the callbacks.

Commit 10 adds nexp and exp_interval parameters to rcuscale for testing callback drain under concurrent expedited GP load.

Puranjay Mohan (10):
rcu/segcblist: Add SRCU and Tasks RCU wrapper functions rcu/segcblist: Factor out rcu_segcblist_advance_compact() helper rcu/segcblist: Change gp_seq to struct rcu_gp_oldstate gp_seq_full
rcu: Add RCU_GET_STATE_NOT_TRACKED for subsystems without expedited
GPs
rcu: Enable RCU callbacks to benefit from expedited grace periods
rcu: Update comments for gp_seq_full and expedited GP tracking
rcu: Wake NOCB rcuog kthreads on expedited grace period completion
rcu: Detect expedited grace period completion in rcu_pending()
rcu: Advance callbacks for expedited GP completion in rcu_core()
rcuscale: Add concurrent expedited GP threads for callback scaling
tests

 include/linux/rcu_segcblist.h |  16 ++--
 include/trace/events/rcu.h    |   5 +-
 kernel/rcu/rcu.h              |  13 ++-
 kernel/rcu/rcu_segcblist.c    | 145 ++++++++++++++++++++++++----------
 kernel/rcu/rcu_segcblist.h    |   8 +-
 kernel/rcu/rcuscale.c         |  84 +++++++++++++++++++-
 kernel/rcu/srcutree.c         |  14 ++--
 kernel/rcu/tasks.h            |   8 +-
 kernel/rcu/tree.c             |  54 +++++++++----
 kernel/rcu/tree.h             |   1 +
 kernel/rcu/tree_exp.h         |   1 +
 kernel/rcu/tree_nocb.h        |  77 ++++++++++++++----
 12 files changed, 324 insertions(+), 102 deletions(-)

base-commit: 51289896149317e0b6a4abf5d300a569c3353dca

Signature · 1 line
2.52.0