Skip to content

Improvements to rcu_barrier() and RT response on big systems

Paul E. McKenney lkml 1 version In review

Get series with b4
b4 am [email protected]

Versions newest first

  1. v1 Latest15 patches 34 repliesIn review

Patches in v1 No applies recorded yet

#Subject and filesSizeRepliesApplied
1/15rcu: Control RCU_FANOUT_LEAF from boot-time parameterDocumentation/kernel-parameters.txt kernel/rcutree.c kernel/rcutree.h +2 more+104 -262 replies
2/15rcu: Size rcu_node tree from nr_cpu_ids rather than NR_CPUSkernel/rcutree.c kernel/rcutree_plugin.h+3 -18 replies
3/15rcu: Prevent excessive line length in RCU_STATE_INITIALIZER()kernel/rcutree.c+7 -71 reply
4/15rcu: Place pointer to call_rcu() in rcu_data structurekernel/rcutree.c kernel/rcutree.h kernel/rcutree_plugin.h+17 -171 reply
5/15rcu: Move _rcu_barrier()'s rcu_head structures to rcu_data structureskernel/rcutree.c kernel/rcutree.h+5 -41 reply
6/15rcu: Move rcu_barrier_cpu_count to rcu_state structurekernel/rcutree.c kernel/rcutree.h+15 -111 reply
7/15rcu: Move rcu_barrier_completion to rcu_state structurekernel/rcutree.c kernel/rcutree.h+5 -51 reply
8/15rcu: Move rcu_barrier_mutex to rcu_state structurekernel/rcutree.c kernel/rcutree.h+4 -81 reply
9/15rcu: Increasing rcu_barrier() concurrencykernel/rcutree.c kernel/rcutree.h+28 -14 replies
10/15rcu: Add tracing for _rcu_barrier()include/trace/events/rcu.h kernel/rcutree.c+73 -11 reply
11/15rcu: Add rcu_barrier() statistics to debugfs tracingkernel/rcutree_trace.c+39 -01 reply
12/15rcu: Remove unneeded __rcu_process_callbacks() argumentkernel/rcutree.c kernel/rcutree_plugin.h+5 -61 reply
13/15rcu: Introduce for_each_rcu_flavor() and use itkernel/rcutree.c kernel/rcutree.h kernel/rcutree_plugin.h+36 -1454 replies
14/15rcu: Use for_each_rcu_flavor() in TREE_RCU tracingkernel/rcutree_trace.c+38 -574 replies
15/15rcu: RCU_SAVE_DYNTICK code no longer ever deadkernel/rcutree.c+0 -23 replies

Reviews trailers per patch

9 of 15 patches have a Reviewed-by or Acked-by.

PatchJosh Triplett
1/15
2/15
3/15R-b
4/15R-b
5/15R-b
6/15R-b
7/15R-b
8/15
9/15
10/15R-b
11/15R-b
12/15R-b
13/15
14/15
15/15R-b

Trailers given in replies are filled; those carried in the patches from earlier versions are outlined with dashes.

Read the v1 thread

Cover letter [PATCH tip/core/rcu 0/15] Improvements to rcu_barrier() and RT response on big systems

Hello!

This patch series contains improvements to the rcu_barrier() family of primitives and to latency for large systems. These are in a single series due to conflicts that would otherwise occur. The individual patches are as follows:

1. Allow the value for RCU_FANOUT_LEAF to be increased (but not
decreased!) via a boot-time parameter, in turn allowing a default kernel build to be adjusted for low RCU grace-period initialization latency on a large system.
2. Work around the new default NR_CPUS=4096 by checking the
boot-time-computed nr_cpu_ids, allowing this to override NR_CPUS. This again reduces RCU grace-period initialization latency for kernels built with large NR_CPUS running on small systems.
3. Shrink a macro argument to keep lines under 80 characters.
4. Add a pointer in the rcu_state structure to the corresponding
member of the call_rcu() family of functions in preparation for increasing rcu_barrier() concurrency.
5. Move _rcu_barrier()'s rcu_head structures to the per-CPU
per-RCU-flavor rcu_data structures so that different flavors of rcu_barrier() do not need to contend for the rcu_head structures.
6. Move rcu_barrier()'s rcu_barrier_cpu_count global variable to
a new ->barrier_cpu_count field in the rcu_state structure, so that different flavors of rcu_barrier() do not need to contend for this variable.
7. Move rcu_barrier()'s rcu_barrier_completion global variable to
a new ->barrier_completion field in the rcu_state structure, so that different flavors of rcu_barrier() do not need to contend for this variable.
8. Move rcu_barrier()'s rcu_barrier_mutex global variable to

	a new ->barrier_mutex field in the rcu_state structure, so that
	different flavors of rcu_barrier() do not need to contend for
	this variable.

9. Introduce counter scheme to allow multiple concurrent executions
of a given flavor of rcu_barrier() to share work.
10. Add event tracing for _rcu_barrier().
11. Add debugfs tracing for _rcu_barrier().
12. Remove unnecessary per-CPU variable argument from

	__rcu_process_callbacks().

13. Introduce for_each_rcu_flavor() iterator and use it. This provides
a nicer way to iterate through the RCU flavors to do per-flavor processing.
14. Apply the for_each_rcu_flavor() iterator to debugfs tracing.
15. Remove dead-code gcc helper from code that is no longer ever dead.

Thanx, Paul

 b/Documentation/kernel-parameters.txt |    4 
 b/include/trace/events/rcu.h          |   45 +++++++
 b/kernel/rcutree.c                    |   97 +++++++++++++--
 b/kernel/rcutree.h                    |   23 ++-
 b/kernel/rcutree_plugin.h             |    4 
 b/kernel/rcutree_trace.c              |    2 
 kernel/rcutree.c                      |  213 +++++++++++++++++++++-------------
 kernel/rcutree.h                      |   22 ++-
 kernel/rcutree_plugin.h               |  126 --------------------
 kernel/rcutree_trace.c                |  134 ++++++++++++---------
 10 files changed, 379 insertions(+), 291 deletions(-)