Skip to content

No-CBs bypass addition for v5.4

Paul E. McKenney lkml rcu 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest14 patches 42 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/14rcu/nocb: Atomic ->len field in rcu_segcblist structureinclude/linux/rcu_segcblist.h kernel/rcu/rcu_segcblist.c kernel/rcu/rcu_segcblist.h+94 -84 replies
2/14rcu/nocb: Add bypass callback queueingkernel/rcu/rcu_segcblist.c kernel/rcu/rcu_segcblist.h kernel/rcu/tree.c +2 more+394 -417 replies
3/14rcu/nocb: EXP Check use and usefulness of ->nocb_lock_contendedkernel/rcu/tree_plugin.h+3 -10 replies
4/14rcu/nocb: Print no-CBs diagnostics when rcutorture writer unduly delayedkernel/rcu/rcutorture.c kernel/rcu/tree.h kernel/rcu/tree_plugin.h +1 more+94 -10 replies
5/14rcu/nocb: Avoid synchronous wakeup in __call_rcu_nocb_wake()kernel/rcu/tree_plugin.h+5 -140 replies
6/14rcu/nocb: Advance CBs after merge in rcutree_migrate_callbacks()kernel/rcu/tree.c+1 -00 replies
7/14rcu/nocb: Reduce nocb_cb_wait() leaf rcu_node ->lock contentionkernel/rcu/tree_plugin.h+4 -10 replies
8/14rcu/nocb: Reduce __call_rcu_nocb_wake() leaf rcu_node ->lock contentionkernel/rcu/tree_plugin.h+10 -30 replies
9/14rcu/nocb: Don't wake no-CBs GP kthread if timer posted under overloadkernel/rcu/tree_plugin.h+5 -20 replies
10/14rcu: Allow rcu_do_batch() to dynamically adjust batch sizeskernel/rcu/tree.c+19 -10 replies
11/14EXP nohz: Add TICK_DEP_BIT_RCUinclude/linux/tick.h kernel/time/tick-sched.c+8 -10 replies
12/14rcu/nohz: Force on tick when invoking lots of callbackskernel/rcu/tree.c+4 -00 replies
13/14rcutorture: Force on tick for readers and callback flooderskernel/rcu/rcutorture.c+18 -60 replies
14/14rcu/nohz: Make multi_cpu_stop() enable tick on all online CPUskernel/stop_machine.c+8 -131 replies

Reviews trailers per patch

No Reviewed-by, Acked-by or Tested-by trailers for v1 yet.

Read the v1 thread

Cover letter [PATCH tip/core/rcu 0/14] No-CBs bypass addition for v5.4

Hello!

This series is a sneak preview of additional work for the move of no-CBs CPUs to the ->cblist segmented RCU callback list. This work adds a ->nocb_bypass list with its own lock to further reduce contention. This series also includes some nascent work to turn the scheduling-clock interrupt back on for nohz_full CPUs doing heavy rcutorture work or RCU callback invocation, both of which can remain in the kernel for long time periods, which in turn can impede CPU hotplug removals. (On some systems "impede" means up to seven minutes for stop-machine to actually get things to stop, a problem that has not yet been observed on no-CBs CPUs that are not also nohz_full CPUs.)

1. Atomic ->len field in rcu_segcblist structure.

2. Add bypass callback queueing in ->nocb_bypass with its own

	->nocb_bypass_lock.

3. (Experimental) Check use and usefulness of ->nocb_lock_contended.

4. Print no-CBs diagnostics when rcutorture writer unduly delayed.

5. Avoid synchronous wakeup in __call_rcu_nocb_wake().

6. Advance CBs after merge in rcutree_migrate_callbacks() to
avoid unnecessary invocation delays.

7. Reduce nocb_cb_wait() leaf rcu_node ->lock contention.

8. Reduce __call_rcu_nocb_wake() leaf rcu_node ->lock contention.

9. Don't wake no-CBs GP kthread if timer posted under overload,
thus reducing overhead in the overload case.

10. Allow rcu_do_batch() to dynamically adjust batch sizes, courtesy
of Eric Dumazet.

11. (Experimental) Add TICK_DEP_BIT_RCU, courtesy of Frederic Weisbecker.

12. Force on tick when invoking lots of callbacks to reduce the
probability of long stop-machine delays.

13. Force on tick for readers and callback flooders, again to reduce
the probability of long stop-machine delays.

14. (Experimental and likely quite imperfect) Make multi_cpu_stop()
enable tick on all online CPUs, yet again to reduce the probability of long stop-machine delays.

Thanx, Paul

------------------------------------------------------------------------

 include/linux/rcu_segcblist.h |    4 
 include/linux/tick.h          |    7 
 kernel/rcu/rcu_segcblist.c    |  116 +++++++++-
 kernel/rcu/rcu_segcblist.h    |   17 +
 kernel/rcu/rcutorture.c       |   25 +-
 kernel/rcu/tree.c             |   41 +++
 kernel/rcu/tree.h             |   35 ++-
 kernel/rcu/tree_plugin.h      |  486 +++++++++++++++++++++++++++++++++++++-----
 kernel/rcu/tree_stall.h       |    5 
 kernel/stop_machine.c         |    9 
 kernel/time/tick-sched.c      |    2 
 11 files changed, 667 insertions(+), 80 deletions(-)