Real-time updates for 3.19
Subsystems (7)
cover Real-time updates for 3.19
Hello!
This series contains real-time-related updates:
1. Move RCU_NOCB_CPU to choice, courtesy of Stefan Hengelein.
2. Place RCU_BOOST-specific variables under pre-existing #ifdef.
3. Prevent synchronize_sched_expedited() from sending IPIs to
idle and to nohz_full CPUs.
4. Unify boost and kthread priorities, and allow this unified
priority to be set at boot time, courtesy of Clark Williams.
5. Remove redundant TREE_PREEMPT_RCU Kconfig option, courtesy of
Pranith Kumar.
6. Handle the possibility of a no-CBs CPU posting callbacks on
their last pass through the idle loop by waking up the leader kthread after the CPU has gone completely offline.
7. Fix rcuo online-time-creation reoganization bug, which seems
to be a latent bug due to systems (wisely) bringing CPUs online in numerical order.
Thanx, Paul
------------------------------------------------------------------------
b/Documentation/RCU/rcu.txt | 4 b/Documentation/RCU/stallwarn.txt | 8 - b/Documentation/RCU/trace.txt | 4 b/Documentation/RCU/whatisRCU.txt | 2 b/Documentation/kernel-parameters.txt | 6 + b/include/linux/init_task.h | 2 b/include/linux/rcupdate.h | 6 - b/include/linux/sched.h | 4 b/include/trace/events/rcu.h | 4 b/init/Kconfig | 49 +++------ b/kernel/rcu/Makefile | 2 b/kernel/rcu/tree.c | 44 +++++--- b/kernel/rcu/tree.h | 10 - b/kernel/rcu/tree_plugin.h | 53 ++++++---- b/kernel/rcu/update.c | 2 b/lib/Kconfig.debug | 2 b/tools/testing/selftests/rcutorture/configs/rcu/TREE01 | 2 b/tools/testing/selftests/rcutorture/configs/rcu/TREE02 | 2 b/tools/testing/selftests/rcutorture/configs/rcu/TREE02-T | 2 b/tools/testing/selftests/rcutorture/configs/rcu/TREE03 | 4 b/tools/testing/selftests/rcutorture/configs/rcu/TREE08 | 2 b/tools/testing/selftests/rcutorture/configs/rcu/TREE08-T | 2 b/tools/testing/selftests/rcutorture/configs/rcu/TREE09 | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v0.0/P1-S-T-NH-SD-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v0.0/P2-2-t-nh-sd-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v0.0/P3-3-T-nh-SD-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v0.0/P4-A-t-NH-sd-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v0.0/P5-U-T-NH-sd-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P1-S-T-NH-SD-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P2-2-t-nh-sd-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P3-3-T-nh-SD-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P4-A-t-NH-sd-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P5-U-T-NH-sd-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P6---t-nh-SD-smp-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP-all | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP-none | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.3/P1-S-T-NH-SD-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.3/P2-2-t-nh-sd-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.3/P3-3-T-nh-SD-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.3/P4-A-t-NH-sd-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.3/P5-U-T-NH-sd-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.5/P1-S-T-NH-SD-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.5/P2-2-t-nh-sd-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.5/P3-3-T-nh-SD-SMP-hp | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.5/P4-A-t-NH-sd-SMP-HP | 2 b/tools/testing/selftests/rcutorture/configs/rcu/v3.5/P5-U-T-NH-sd-SMP-hp | 2 b/tools/testing/selftests/rcutorture/doc/TINY_RCU.txt | 2 b/tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt | 14 +- 50 files changed, 157 insertions(+), 127 deletions(-)
PATCH 1/7 init/Kconfig: move RCU_NOCB_CPU dependencies to choice
From: Stefan Hengelein <[email protected]>
Every choice item of the "Build-forced no-CBs CPUs" choice had a dependency to RCU_NOCB_CPU. It's more comprehensible if the choice itself has the dependency instead of every choice item. The choice itself doesn't need to be visible if there are no items selectable (i.e. on arch/frv) or RCU_NOCB_CPU is not defined.
--- init/Kconfig | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
init/Kconfig +1 -3
@@ -739,6 +739,7 @@ config RCU_NOCB_CPU choice prompt "Build-forced no-CBs CPUs" default RCU_NOCB_CPU_NONE + depends on RCU_NOCB_CPU help This option allows no-CBs CPUs (whose RCU callbacks are invoked from kthreads rather than from softirq context) to be specified
@@ -747,7 +748,6 @@ choice config RCU_NOCB_CPU_NONE bool "No build_forced no-CBs CPUs" - depends on RCU_NOCB_CPU help This option does not force any of the CPUs to be no-CBs CPUs. Only CPUs designated by the rcu_nocbs= boot parameter will be
@@ -761,7 +761,6 @@ config RCU_NOCB_CPU_NONE config RCU_NOCB_CPU_ZERO bool "CPU 0 is a build_forced no-CBs CPU" - depends on RCU_NOCB_CPU help This option forces CPU 0 to be a no-CBs CPU, so that its RCU callbacks are invoked by a per-CPU kthread whose name begins
@@ -776,7 +775,6 @@ config RCU_NOCB_CPU_ZERO config RCU_NOCB_CPU_ALL bool "All CPUs are build_forced no-CBs CPUs" - depends on RCU_NOCB_CPU help This option forces all CPUs to be no-CBs CPUs. The rcu_nocbs= boot parameter will be ignored. All CPUs' RCU callbacks will
Signature · 1 line
1.8.1.5
PATCH 2/7 rcu: Move RCU_BOOST variable declarations, eliminating #ifdef
From: "Paul E. McKenney" <[email protected]>
There are some RCU_BOOST-specific per-CPU variable declarations that are needlessly defined under #ifdef in kernel/rcu/tree.c. This commit therefore moves these declarations into a pre-existing #ifdef in kernel/rcu/tree_plugin.h.
--- kernel/rcu/tree.c | 13 ------------- kernel/rcu/tree_plugin.h | 17 +++++++++++++++-- 2 files changed, 15 insertions(+), 15 deletions(-)
kernel/rcu/tree.c +0 -13
@@ -152,19 +152,6 @@ EXPORT_SYMBOL_GPL(rcu_scheduler_active);*/ static int rcu_scheduler_fully_active__read_mostly ; -#ifdef CONFIG_RCU_BOOST - -/* -* Control variables for per-CPU and per-rcu_node kthreads. These -* handle all flavors of RCU. -*/ -static DEFINE_PER_CPU(struct task_struct *, rcu_cpu_kthread_task); -DEFINE_PER_CPU (unsigned int , rcu_cpu_kthread_status); -DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_loops); -DEFINE_PER_CPU (char, rcu_cpu_has_work); - -#endif /* #ifdef CONFIG_RCU_BOOST */ -static void rcu_boost_kthread_setaffinity (struct rcu_node *rnp,int outgoingcpu);static void invoke_rcu_core (void );static void invoke_rcu_callbacks (struct rcu_state *rsp,struct rcu_data *rdp);
kernel/rcu/tree_plugin.h +15 -2
@@ -33,11 +33,24 @@#define RCU_KTHREAD_PRIO 1#ifdef CONFIG_RCU_BOOST +#include "../locking/rtmutex_common.h" #define RCU_BOOST_PRIO CONFIG_RCU_BOOST_PRIO -#else + +/* +* Control variables for per-CPU and per-rcu_node kthreads. These +* handle all flavors of RCU. +*/ +static DEFINE_PER_CPU(struct task_struct *, rcu_cpu_kthread_task); +DEFINE_PER_CPU (unsigned int , rcu_cpu_kthread_status); +DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_loops); +DEFINE_PER_CPU (char, rcu_cpu_has_work); + +#else /* #ifdef CONFIG_RCU_BOOST */ +#define RCU_BOOST_PRIO RCU_KTHREAD_PRIO -#endif + +#endif /* #else #ifdef CONFIG_RCU_BOOST */ #ifdef CONFIG_RCU_NOCB_CPU static cpumask_var_t rcu_nocb_mask;/* CPUs to have callbacks offloaded. */
Signature · 1 line
1.8.1.5
PATCH 3/7 rcu: Avoid IPIing idle CPUs from synchronize_sched_expedited()
In reply to Paul E. McKenneyFrom: "Paul E. McKenney" <[email protected]>
Currently, synchronize_sched_expedited() sends IPIs to all online CPUs, even those that are idle or executing in nohz_full= userspace. Because idle CPUs and nohz_full= userspace CPUs are in extended quiescent states, there is no need to IPI them in the first place. This commit therefore avoids IPIing CPUs that are already in extended quiescent states.
--- kernel/rcu/tree.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +26 -1
@@ -2950,6 +2950,9 @@ static int synchronize_sched_expedited_cpu_stop(void *data)*/ void synchronize_sched_expedited (void ) { +cpumask_var_t cm; +bool cma = false; +int cpu;long firstsnap, s, snap;int trycount =0 ;struct rcu_state *rsp = &rcu_sched_state;
@@ -2984,11 +2987,26 @@ void synchronize_sched_expedited(void) }WARN_ON_ONCE (cpu_is_offline (raw_smp_processor_id ())); +/* Offline CPUs, idle CPUs, and any CPU we run on are quiescent. */ + cma =zalloc_cpumask_var (&cm,GFP_KERNEL ); +if (cma) { +cpumask_copy (cm, cpu_online_mask); +cpumask_clear_cpu (raw_smp_processor_id (), cm); +for_each_cpu (cpu, cm) { +struct rcu_dynticks *rdtp = &per_cpu (rcu_dynticks, cpu); + +if (!(atomic_add_return (0 , &rdtp->dynticks) &0x1 )) +cpumask_clear_cpu (cpu, cm); + } +if (cpumask_weight (cm) ==0 ) + goto all_cpus_idle; + } +/* * Each pass through the following loop attempts to force a * context switch on each CPU. */ -while (try_stop_cpus (cpu_online_mask, +while (try_stop_cpus (cma ? cm : cpu_online_mask, synchronize_sched_expedited_cpu_stop,NULL ) == -EAGAIN ) {put_online_cpus ();
@@ -3000,6 +3018,7 @@ void synchronize_sched_expedited(void)/* ensure test happens before caller kfree */ smp_mb__before_atomic ();/* ^^^ */ atomic_long_inc (&rsp->expedited_workdone1); +free_cpumask_var (cm);return ; }
@@ -3009,6 +3028,7 @@ void synchronize_sched_expedited(void) }else {wait_rcu_gp (call_rcu_sched);atomic_long_inc (&rsp->expedited_normal); +free_cpumask_var (cm);return ; }
@@ -3018,6 +3038,7 @@ void synchronize_sched_expedited(void)/* ensure test happens before caller kfree */ smp_mb__before_atomic ();/* ^^^ */ atomic_long_inc (&rsp->expedited_workdone2); +free_cpumask_var (cm);return ; }
@@ -3032,6 +3053,7 @@ void synchronize_sched_expedited(void)/* CPU hotplug operation in flight, use normal GP. */ wait_rcu_gp (call_rcu_sched);atomic_long_inc (&rsp->expedited_normal); +free_cpumask_var (cm);return ; } snap =atomic_long_read (&rsp->expedited_start);
@@ -3039,6 +3061,9 @@ void synchronize_sched_expedited(void) }atomic_long_inc (&rsp->expedited_stoppedcpus); +all_cpus_idle: +free_cpumask_var (cm); +/* * Everyone up to our most recent fetch is covered by our grace * period. Update the counter, but only if our work is still
Signature · 1 line
1.8.1.5
Peter Zijlstra
maintainerRe: [PATCH tip/core/rcu 3/7] rcu: Avoid IPIing idle CPUs from synchronize_sched_expedited()
Quoted 35 lines · Paul E. McKenney
From: "Paul E. McKenney" <[email protected]>
Currently,
synchronize_sched_expedited()sends IPIs to all online CPUs, even those that are idle or executing in nohz_full= userspace. Because idle CPUs and nohz_full= userspace CPUs are in extended quiescent states, there is no need to IPI them in the first place. This commit therefore avoids IPIing CPUs that are already in extended quiescent states.Signed-off-by Paul E. McKenney <[email protected]>--- kernel/rcu/tree.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)kernel/rcu/tree.c +7
-0@@ -2950,6 +2950,9 @@ static int synchronize_sched_expedited_cpu_stop(void *data)*/ void synchronize_sched_expedited (void ) { +cpumask_var_t cm; +bool cma = false; +int cpu;long firstsnap, s, snap;int trycount =0 ;struct rcu_state *rsp = &rcu_sched_state;@@ -2984,11 +2987,26 @@ void synchronize_sched_expedited(void) }WARN_ON_ONCE (cpu_is_offline (raw_smp_processor_id ())); +/* Offline CPUs, idle CPUs, and any CPU we run on are quiescent. */ + cma =zalloc_cpumask_var (&cm,GFP_KERNEL ); +if (cma) { +cpumask_copy (cm, cpu_online_mask);
kernel/rcu/tree.c void synchronize_sched_expedited(void) · line 3003
Quoted 10 lines
kernel/rcu/tree.c +10
-0+cpumask_clear_cpu (raw_smp_processor_id (), cm); +for_each_cpu (cpu, cm) { +struct rcu_dynticks *rdtp = &per_cpu (rcu_dynticks, cpu); + +if (!(atomic_add_return (0 , &rdtp->dynticks) &0x1 )) +cpumask_clear_cpu (cpu, cm); + } +if (cpumask_weight (cm) ==0 ) + goto all_cpus_idle; + }
Is there a reason not to use on_each_cpu_cond()?
Re: [PATCH tip/core/rcu 3/7] rcu: Avoid IPIing idle CPUs from synchronize_sched_expedited()
Quoted 32 lines · Peter Zijlstra
Quoted 31 lines · Paul E. McKenney
From: "Paul E. McKenney" <[email protected]>
Currently,
synchronize_sched_expedited()sends IPIs to all online CPUs, even those that are idle or executing in nohz_full= userspace. Because idle CPUs and nohz_full= userspace CPUs are in extended quiescent states, there is no need to IPI them in the first place. This commit therefore avoids IPIing CPUs that are already in extended quiescent states.Signed-off-by Paul E. McKenney <[email protected]>--- kernel/rcu/tree.c | 27 ++++++++++++++++++++++++++- 1 file changed, 26 insertions(+), 1 deletion(-)kernel/rcu/tree.c +3
-0@@ -2950,6 +2950,9 @@ static int synchronize_sched_expedited_cpu_stop(void *data)*/ void synchronize_sched_expedited (void ) { +cpumask_var_t cm; +bool cma = false; +int cpu;long firstsnap, s, snap;int trycount =0 ;struct rcu_state *rsp = &rcu_sched_state;@@ -2984,11 +2987,26 @@ void synchronize_sched_expedited(void) }WARN_ON_ONCE (cpu_is_offline (raw_smp_processor_id ()));
Quoted 16 lines
Quoted 4 lines
kernel/rcu/tree.c +4
-0+/* Offline CPUs, idle CPUs, and any CPU we run on are quiescent. */ + cma =zalloc_cpumask_var (&cm,GFP_KERNEL ); +if (cma) { +cpumask_copy (cm, cpu_online_mask);kernel/rcu/tree.c void synchronize_sched_expedited(void) · line 3003
Quoted 10 lines
kernel/rcu/tree.c +10
-0+cpumask_clear_cpu (raw_smp_processor_id (), cm); +for_each_cpu (cpu, cm) { +struct rcu_dynticks *rdtp = &per_cpu (rcu_dynticks, cpu); + +if (!(atomic_add_return (0 , &rdtp->dynticks) &0x1 )) +cpumask_clear_cpu (cpu, cm); + } +if (cpumask_weight (cm) ==0 ) + goto all_cpus_idle; + }Is there a reason not to use
on_each_cpu_cond()?
Because I don't know how to write a function that returns a blooean value? (Sorry, couldn't resist, and yes I do know that "boolean" was meant.) If we had lambdas, I might be interested in making that transformation, but pulling the condition into a separate function doesn't seem like a win to me.
But even with lambdas, it looks to me like on_each_cpu_cond() just does an IPI, and I need the selected CPUs to do a context switch. Yes, I could make the IPI handler function call induce a context switch, but then I would have to add more mechanism to wait for the induced context switches to actually happen.
That said, I am considering switching synchronize_sched_expedited() from try_stop_cpus() to resched_cpu() if I need to parallelize synchronize_sched_expedited().
Thanx, Paul
PATCH 4/7 rcu: Unify boost and kthread priorities
In reply to Paul E. McKenneyFrom: Clark Williams <[email protected]>
Rename CONFIG_RCU_BOOST_PRIO to CONFIG_RCU_KTHREAD_PRIO and use this value for both the per-CPU kthreads (rcuc/N) and the rcu boosting threads (rcub/n).
Also, create the module_parameter rcutree.kthread_prio to be used on the kernel command line at boot to set a new value (rcutree.kthread_prio=N).
--- Documentation/kernel-parameters.txt | 6 ++++++ init/Kconfig | 23 +++++++++++----------- kernel/rcu/tree_plugin.h | 20 +++++++++---------- .../selftests/rcutorture/configs/rcu/TREE03 | 2 +- .../selftests/rcutorture/doc/TREE_RCU-kconfig.txt | 4 ++-- 5 files changed, 31 insertions(+), 24 deletions(-)
Documentation/kernel-parameters.txt +6 -0
@@ -2922,6 +2922,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. quiescent states. Units are jiffies, minimum value is one, and maximum value is HZ. + rcutree.kthread_prio= [KNL,BOOT] + Set the SCHED_FIFO priority of the RCU + per-CPU kthreads (rcuc/N). This value is also + used for the priority of the RCU boost threads + (rcub/N). Valid values are 1-99. + rcutree.rcu_nocb_leader_stride= [KNL] Set the number of NOCB kthread groups, which defaults to the square root of the number of
init/Kconfig +12 -11
@@ -672,30 +672,31 @@ config RCU_BOOST Say Y here if you are working with real-time apps or heavy loads Say N here if you are unsure. -config RCU_BOOST_PRIO - int "Real-time priority to boost RCU readers to" +config RCU_KTHREAD_PRIO + int "Real-time priority to use for RCU worker threads" range 1 99 depends on RCU_BOOST default 1 help - This option specifies the real-time priority to which long-term - preempted RCU readers are to be boosted. If you are working - with a real-time application that has one or more CPU-bound - threads running at a real-time priority level, you should set - RCU_BOOST_PRIO to a priority higher then the highest-priority - real-time CPU-bound thread. The default RCU_BOOST_PRIO value - of 1 is appropriate in the common case, which is real-time + This option specifies the SCHED_FIFO priority value that will be + assigned to the rcuc/n and rcub/n threads and is also the value + used for RCU_BOOST (if enabled). If you are working with a + real-time application that has one or more CPU-bound threads + running at a real-time priority level, you should set + RCU_KTHREAD_PRIO to a priority higher than the highest-priority + real-time CPU-bound application thread. The default RCU_KTHREAD_PRIO + value of 1 is appropriate in the common case, which is real-time applications that do not have any CPU-bound threads. Some real-time applications might not have a single real-time thread that saturates a given CPU, but instead might have multiple real-time threads that, taken together, fully utilize - that CPU. In this case, you should set RCU_BOOST_PRIO to + that CPU. In this case, you should set RCU_KTHREAD_PRIO to a priority higher than the lowest-priority thread that is conspiring to prevent the CPU from running any non-real-time tasks. For example, if one thread at priority 10 and another thread at priority 5 are between themselves fully consuming - the CPU time on a given CPU, then RCU_BOOST_PRIO should be + the CPU time on a given CPU, then RCU_KTHREAD_PRIO should be set to priority 6 or higher. Specify the real-time priority, or take the default if unsure.
kernel/rcu/tree_plugin.h +10 -10
@@ -30,12 +30,13 @@#include <linux/smpboot.h> #include "../time/tick-internal.h" -#define RCU_KTHREAD_PRIO 1 -#ifdef CONFIG_RCU_BOOST #include "../locking/rtmutex_common.h" -#define RCU_BOOST_PRIO CONFIG_RCU_BOOST_PRIO + +/* rcuc/rcub kthread realtime priority */ +static int kthread_prio =CONFIG_RCU_KTHREAD_PRIO ; +module_param (kthread_prio, int,0644 );/* * Control variables for per-CPU and per-rcu_node kthreads. These
@@ -46,11 +47,7 @@ DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_status); DEFINE_PER_CPU(unsigned int, rcu_cpu_kthread_loops);DEFINE_PER_CPU (char, rcu_cpu_has_work); -#else /* #ifdef CONFIG_RCU_BOOST */ - -#define RCU_BOOST_PRIO RCU_KTHREAD_PRIO - -#endif /* #else #ifdef CONFIG_RCU_BOOST */ +#endif /* #ifdef CONFIG_RCU_BOOST */ #ifdef CONFIG_RCU_NOCB_CPU static cpumask_var_t rcu_nocb_mask;/* CPUs to have callbacks offloaded. */
@@ -98,6 +95,9 @@ static void __init rcu_bootup_announce_oddness(void)pr_info ("\tBoot-time adjustment of leaf fanout to %d.\n" , rcu_fanout_leaf);if (nr_cpu_ids !=NR_CPUS )pr_info ("\tRCU restricting CPUs from NR_CPUS=%d to nr_cpu_ids=%d.\n" ,NR_CPUS , nr_cpu_ids); +#ifdef CONFIG_RCU_BOOST +pr_info ("\tRCU kthread priority: %d.\n" , kthread_prio); +#endif }#ifdef CONFIG_TREE_PREEMPT_RCU
@@ -1339,7 +1339,7 @@ static int rcu_spawn_one_boost_kthread(struct rcu_state *rsp,smp_mb__after_unlock_lock (); rnp->boost_kthread_task = t;raw_spin_unlock_irqrestore (&rnp->lock, flags); - sp.sched_priority =RCU_BOOST_PRIO ; + sp.sched_priority = kthread_prio;sched_setscheduler_nocheck (t,SCHED_FIFO , &sp);wake_up_process (t);/* get to TASK_INTERRUPTIBLE quickly. */ return 0 ;
@@ -1356,7 +1356,7 @@ static void rcu_cpu_kthread_setup(unsigned int cpu) {struct sched_param sp; - sp.sched_priority =RCU_KTHREAD_PRIO ; + sp.sched_priority = kthread_prio;sched_setscheduler_nocheck (current,SCHED_FIFO , &sp); }
tools/testing/selftests/rcutorture/configs/rcu/TREE03 +1 -1
@@ -17,5 +17,5 @@ CONFIG_DEBUG_LOCK_ALLOC=n CONFIG_RCU_CPU_STALL_INFO=n CONFIG_RCU_CPU_STALL_VERBOSE=n CONFIG_RCU_BOOST=y -CONFIG_RCU_BOOST_PRIO=2 +CONFIG_RCU_KTHREAD_PRIO=2 CONFIG_DEBUG_OBJECTS_RCU_HEAD=n
tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt +2 -2
@@ -15,8 +15,8 @@ CONFIG_PREEMPT -- Do half. (First three and #8.) CONFIG_PROVE_LOCKING -- Do all but two, covering CONFIG_PROVE_RCU and not. CONFIG_PROVE_RCU -- Do all but one under CONFIG_PROVE_LOCKING. CONFIG_RCU_BOOST -- one of TREE_PREEMPT_RCU. -CONFIG_RCU_BOOST_PRIO -- set to 2 for _BOOST testing. -CONFIG_RCU_CPU_STALL_INFO -- do one with and without _VERBOSE. +CONFIG_RCU_KTHREAD_PRIO -- set to 2 for _BOOST testing. +CONFIG_RCU_CPU_STALL_INFO -- Do one. CONFIG_RCU_CPU_STALL_VERBOSE -- do one with and without _INFO. CONFIG_RCU_FANOUT -- Cover hierarchy as currently, but overlap with others. CONFIG_RCU_FANOUT_EXACT -- Do one.
Signature · 1 line
1.8.1.5
Peter Zijlstra
maintainerRe: [PATCH tip/core/rcu 4/7] rcu: Unify boost and kthread priorities
Quoted 25 lines · Paul E. McKenney
From: Clark Williams <[email protected]>
Rename
CONFIG_RCU_BOOST_PRIOtoCONFIG_RCU_KTHREAD_PRIOand use this value for both the per-CPU kthreads (rcuc/N) and the rcu boosting threads (rcub/n).Also, create the
module_parameterrcutree.kthread_prio to be used on the kernel command line at boot to set a new value (rcutree.kthread_prio=N).Signed-off-by Clark Williams <[email protected]>[ paulmck: Ported to rcu/dev, applied Paul Bolle feedback. ]Signed-off-by Paul E. McKenney <[email protected]>--- Documentation/kernel-parameters.txt | 6 ++++++ init/Kconfig | 23 +++++++++++----------- kernel/rcu/tree_plugin.h | 20 +++++++++---------- .../selftests/rcutorture/configs/rcu/TREE03 | 2 +- .../selftests/rcutorture/doc/TREE_RCU-kconfig.txt | 4 ++-- 5 files changed, 31 insertions(+), 24 deletions(-)Documentation/kernel-parameters.txt +0
-0
Documentation/kernel-parameters.txt bytes respectively. Such letter suffixes can also be entirely omitted. · line 2930
Quoted 9 lines
Documentation/kernel-parameters.txt +6
-0@@ -2922,6 +2922,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. quiescent states. Units are jiffies, minimum value is one, and maximum value is HZ. + rcutree.kthread_prio= [KNL,BOOT] + Set the SCHED_FIFO priority of the RCU + per-CPU kthreads (rcuc/N). This value is also + used for the priority of the RCU boost threads + (rcub/N). Valid values are 1-99. +
It would maybe be useful to mention the default?
Also, should we look at running this stuff as deadline in order to provide interference guarantees etc.. ?
Re: [PATCH tip/core/rcu 4/7] rcu: Unify boost and kthread priorities
Quoted 29 lines · Peter Zijlstra
Quoted 28 lines · Paul E. McKenney
From: Clark Williams <[email protected]>
Rename
CONFIG_RCU_BOOST_PRIOtoCONFIG_RCU_KTHREAD_PRIOand use this value for both the per-CPU kthreads (rcuc/N) and the rcu boosting threads (rcub/n).Also, create the
module_parameterrcutree.kthread_prio to be used on the kernel command line at boot to set a new value (rcutree.kthread_prio=N).Signed-off-by Clark Williams <[email protected]>[ paulmck: Ported to rcu/dev, applied Paul Bolle feedback. ]Signed-off-by Paul E. McKenney <[email protected]>--- Documentation/kernel-parameters.txt | 6 ++++++ init/Kconfig | 23 +++++++++++----------- kernel/rcu/tree_plugin.h | 20 +++++++++---------- .../selftests/rcutorture/configs/rcu/TREE03 | 2 +- .../selftests/rcutorture/doc/TREE_RCU-kconfig.txt | 4 ++-- 5 files changed, 31 insertions(+), 24 deletions(-)Documentation/kernel-parameters.txt +0
-0@@ -2922,6 +2922,12 @@ bytes respectively. Such letter suffixes can also be entirely omitted. quiescent states. Units are jiffies, minimum value is one, and maximum value is HZ.
Quoted 8 lines
Documentation/kernel-parameters.txt bytes respectively. Such letter suffixes can also be entirely omitted. · line 2930
Quoted 6 lines
Documentation/kernel-parameters.txt +6
-0+ rcutree.kthread_prio= [KNL,BOOT] + Set the SCHED_FIFO priority of the RCU + per-CPU kthreads (rcuc/N). This value is also + used for the priority of the RCU boost threads + (rcub/N). Valid values are 1-99. +It would maybe be useful to mention the default?
Good point, fixed. It now reads as follows:
rcutree.kthread_prio= [KNL,BOOT]
Set the SCHED_FIFO priority of the RCU per-CPU kthreads (rcuc/N). This value is also used for the priority of the RCU boost threads (rcub/N). Valid values are 1-99 and the default is 1 (the least-favored priority).
Quoted 2 lines
Also, should we look at running this stuff as deadline in order to provide interference guarantees etc.. ?
Excellent question! I have absolutely no idea what the answer might be.
Taking the two sets of kthreads separately...
rcub/N: This is for RCU priority boosting. In the preferred common case,
these never wake up ever. When they do wake up, all they do is
cause blocked RCU readers to get priority boosted. I vaguely
recall something about inheritance of deadlines, which might work here. One concern is what happens if the deadline is violated, as this isn't really necessarily an error condition in this case -- we don't know how long the RCU read-side critical section will run once awakened.
rcuc/N: This is the softirq replacement in -rt, but in mainline all it
does is invoke RCU callbacks. It might make sense to give it a
deadline of something like a few milliseconds, but we would need to temper that if there were huge numbers of callbacks pending. Or perhaps have it claim that its "unit of work" was some fixed number of callbacks or emptying the list, whichever came first. Or maybe have its "unit of work" also depend on the number of callbacks pending.
Thoughts?
Thanx, Paul
Peter Zijlstra
maintainerRe: [PATCH tip/core/rcu 4/7] rcu: Unify boost and kthread priorities
Quoted 7 lines · Paul E. McKenney
Quoted 2 lines
Also, should we look at running this stuff as deadline in order to provide interference guarantees etc.. ?
Excellent question! I have absolutely no idea what the answer might be.
Taking the two sets of kthreads separately...
Quoted 8 lines
rcub/N: This is for RCU priority boosting. In the preferred common case,
these never wake up ever. When they do wake up, all they do is
cause blocked RCU readers to get priority boosted. I vaguely
recall something about inheritance of deadlines, which might work here. One concern is what happens if the deadline is violated, as this isn't really necessarily an error condition in this case -- we don't know how long the RCU read-side critical section will run once awakened.
Yea, this one is 'hard'. How is this used today? From the previous email we've learnt that the default is FIFO-1, iow. it will preempt SCHED_OTHER but not much more. How is this used in RT systems, what are the criteria for actually changing this?
Increase until RCU stops spilling stalled warns, but not so far that your workload fails?
Not quite sure how to translate that into dl speak :-), the problem of course is that if a DL task starts to trigger the stalls we need to do something.
Quoted 8 lines
rcuc/N: This is the softirq replacement in -rt, but in mainline all it
does is invoke RCU callbacks. It might make sense to give it a
deadline of something like a few milliseconds, but we would need to temper that if there were huge numbers of callbacks pending. Or perhaps have it claim that its "unit of work" was some fixed number of callbacks or emptying the list, whichever came first. Or maybe have its "unit of work" also depend on the number of callbacks pending.
Right, so the problem is if we give it insufficient time it will never catch up on running the callbacks, ie. more will come in than we can process and get out.
So if it works by splicing a callback list to a local list, then runs until completion and then either immediately starts again if there's new work, or goes to sleep waiting for more, _then_ we can already assign it DL parameters with the only caveat being the above issue.
The advantage being indeed that if there are 'many' callbacks pending, we'd only run a few, sleep, run a few more, etc.. due to the CBS until we're done. This smooths out peak interference at the 'cost' of additional delays in actually running the callbacks.
We should be able to detect the case where more and work piles on and the actual running does not appear to catch up, but I'm not sure what to do about it, seeing how system stability is at risk.
Certainly something to think about..
Re: [PATCH tip/core/rcu 4/7] rcu: Unify boost and kthread priorities
Quoted 18 lines · Peter Zijlstra
Quoted 7 lines · Paul E. McKenney
Quoted 2 lines
Also, should we look at running this stuff as deadline in order to provide interference guarantees etc.. ?
Excellent question! I have absolutely no idea what the answer might be.
Taking the two sets of kthreads separately...
Quoted 8 lines
rcub/N: This is for RCU priority boosting. In the preferred common case,
these never wake up ever. When they do wake up, all they do is
cause blocked RCU readers to get priority boosted. I vaguely
recall something about inheritance of deadlines, which might work here. One concern is what happens if the deadline is violated, as this isn't really necessarily an error condition in this case -- we don't know how long the RCU read-side critical section will run once awakened.
Quoted 4 lines
Yea, this one is 'hard'. How is this used today? From the previous email we've learnt that the default is FIFO-1, iow. it will preempt
SCHED_OTHERbut not much more. How is this used in RT systems, what are the criteria for actually changing this?
The old way is to update CONFIG_RCU_BOOST_PRIO and rebuild your kernel, but a recent commit from Clark Williams provides a boot parameter that allows this priority to be changed more conveniently.
Quoted 2 lines
Increase until RCU stops spilling stalled warns, but not so far that your workload fails?
Well, you are supposed to determine the highest RT priority at which your workload might run CPU-bound tasks, and set the boost priority at some level above that. My model of RCU priority boosting is that it should be used to make inadvertent high-priority infinite loops easier to debug, but others might have different approaches.
Quoted 3 lines
Not quite sure how to translate that into dl speak :-), the problem of course is that if a DL task starts to trigger the stalls we need to do something.
Indeed! ;-)
Quoted 9 lines
Quoted 8 lines
rcuc/N: This is the softirq replacement in -rt, but in mainline all it
does is invoke RCU callbacks. It might make sense to give it a
deadline of something like a few milliseconds, but we would need to temper that if there were huge numbers of callbacks pending. Or perhaps have it claim that its "unit of work" was some fixed number of callbacks or emptying the list, whichever came first. Or maybe have its "unit of work" also depend on the number of callbacks pending.
Quoted 3 lines
Right, so the problem is if we give it insufficient time it will never catch up on running the callbacks, ie. more will come in than we can process and get out.
Yep, which can result in OOM.
Quoted 5 lines
So if it works by splicing a callback list to a local list, then runs until completion and then either immediately starts again if there's new work, or goes to sleep waiting for more, _then_ we can already assign it DL parameters with the only caveat being the above issue.
Quoted 8 lines
The advantage being indeed that if there are 'many' callbacks pending, we'd only run a few, sleep, run a few more, etc.. due to the CBS until we're done. This smooths out peak interference at the 'cost' of additional delays in actually running the callbacks.
We should be able to detect the case where more and work piles on and the actual running does not appear to catch up, but I'm not sure what to do about it, seeing how system stability is at risk.
I could imagine having a backup SCHED_FIFO task that handled the case where callbacks were piling up, but synchronizing it with the SCHED_DEADLINE task while avoiding callback misordering could be a bit "interesting". (Recall that callback misordering messes up rcu_barrier().)
Quoted 1 line
Certainly something to think about..
No argument here! ;-)
Thanx, Paul
Peter Zijlstra
maintainerRe: [PATCH tip/core/rcu 4/7] rcu: Unify boost and kthread priorities
Quoted 5 lines · Paul E. McKenney
Well, you are supposed to determine the highest RT priority at which your workload might run CPU-bound tasks, and set the boost priority at some level above that. My model of RCU priority boosting is that it should be used to make inadvertent high-priority infinite loops easier to debug, but others might have different approaches.
Ah, so DL will never be CPU-bound -- and RR/FIFO _should_ never be, but I digress ;-)
Quoted 8 lines
Quoted 3 lines
We should be able to detect the case where more and work piles on and the actual running does not appear to catch up, but I'm not sure what to do about it, seeing how system stability is at risk.
I could imagine having a backup
SCHED_FIFOtask that handled the case where callbacks were piling up, but synchronizing it with theSCHED_DEADLINEtask while avoiding callback misordering could be a bit "interesting". (Recall that callback misordering messes uprcu_barrier().)
Ah, so there is talk of 'soft' CBS modes, which instead of hard throttle either reclaim 'unused' DL bandwidth, or continue running in lower scheduling classes.
Re: [PATCH tip/core/rcu 4/7] rcu: Unify boost and kthread priorities
Quoted 10 lines · Peter Zijlstra
Quoted 5 lines · Paul E. McKenney
Well, you are supposed to determine the highest RT priority at which your workload might run CPU-bound tasks, and set the boost priority at some level above that. My model of RCU priority boosting is that it should be used to make inadvertent high-priority infinite loops easier to debug, but others might have different approaches.
Ah, so DL will never be CPU-bound -- and RR/FIFO _should_ never be, but I digress ;-)
;-) ;-) ;-)
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
We should be able to detect the case where more and work piles on and the actual running does not appear to catch up, but I'm not sure what to do about it, seeing how system stability is at risk.
Quoted 8 lines
Quoted 4 lines
I could imagine having a backup
SCHED_FIFOtask that handled the case where callbacks were piling up, but synchronizing it with theSCHED_DEADLINEtask while avoiding callback misordering could be a bit "interesting". (Recall that callback misordering messes uprcu_barrier().)Ah, so there is talk of 'soft' CBS modes, which instead of hard throttle either reclaim 'unused' DL bandwidth, or continue running in lower scheduling classes.
OK, something like that might work in this case, at least making the dubious assumption that I actually understand what you are getting at. ;-)
You might be amused to hear that one of the solutions in my back pocket for the potential problem of callbacks piling up in the current RCU priority boosting scheme is to temporarily increase the priority. This is similar to how blimit gets increased if too many callbacks pile up.
Thus far, no need for this back-pocket solution, but I wanted to put it out there in case it sparks something in the SCHED_DEADLINE approach.
Thanx, Paul
PATCH 5/7 rcu: Remove redundant TREE_PREEMPT_RCU config option
In reply to Paul E. McKenneyFrom: Pranith Kumar <[email protected]>
PREEMPT_RCU and TREE_PREEMPT_RCU serve the same function after TINY_PREEMPT_RCU has been removed. This patch removes TREE_PREEMPT_RCU and uses PREEMPT_RCU config option in its place.
--- Documentation/RCU/rcu.txt | 4 ++-- Documentation/RCU/stallwarn.txt | 8 ++++---- Documentation/RCU/trace.txt | 4 ++-- Documentation/RCU/whatisRCU.txt | 2 +- include/linux/init_task.h | 2 +- include/linux/rcupdate.h | 6 +++--- include/linux/sched.h | 4 ++-- include/trace/events/rcu.h | 4 ++-- init/Kconfig | 22 ++++++++-------------- kernel/rcu/Makefile | 2 +- kernel/rcu/tree.h | 10 +++++----- kernel/rcu/tree_plugin.h | 6 +++--- kernel/rcu/update.c | 2 +- lib/Kconfig.debug | 2 +- .../selftests/rcutorture/configs/rcu/TREE01 | 2 +- .../selftests/rcutorture/configs/rcu/TREE02 | 2 +- .../selftests/rcutorture/configs/rcu/TREE02-T | 2 +- .../selftests/rcutorture/configs/rcu/TREE03 | 2 +- .../selftests/rcutorture/configs/rcu/TREE08 | 2 +- .../selftests/rcutorture/configs/rcu/TREE08-T | 2 +- .../selftests/rcutorture/configs/rcu/TREE09 | 2 +- .../configs/rcu/v0.0/P1-S-T-NH-SD-SMP-HP | 2 +- .../configs/rcu/v0.0/P2-2-t-nh-sd-SMP-hp | 2 +- .../configs/rcu/v0.0/P3-3-T-nh-SD-SMP-hp | 2 +- .../configs/rcu/v0.0/P4-A-t-NH-sd-SMP-HP | 2 +- .../configs/rcu/v0.0/P5-U-T-NH-sd-SMP-hp | 2 +- .../configs/rcu/v3.12/P1-S-T-NH-SD-SMP-HP | 2 +- .../configs/rcu/v3.12/P2-2-t-nh-sd-SMP-hp | 2 +- .../configs/rcu/v3.12/P3-3-T-nh-SD-SMP-hp | 2 +- .../configs/rcu/v3.12/P4-A-t-NH-sd-SMP-HP | 2 +- .../configs/rcu/v3.12/P5-U-T-NH-sd-SMP-hp | 2 +- .../configs/rcu/v3.12/P6---t-nh-SD-smp-hp | 2 +- .../configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP | 2 +- .../configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP-all | 2 +- .../configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP-none | 2 +- .../configs/rcu/v3.12/P7-4-T-NH-SD-SMP-hp | 2 +- .../configs/rcu/v3.3/P1-S-T-NH-SD-SMP-HP | 2 +- .../configs/rcu/v3.3/P2-2-t-nh-sd-SMP-hp | 2 +- .../configs/rcu/v3.3/P3-3-T-nh-SD-SMP-hp | 2 +- .../configs/rcu/v3.3/P4-A-t-NH-sd-SMP-HP | 2 +- .../configs/rcu/v3.3/P5-U-T-NH-sd-SMP-hp | 2 +- .../configs/rcu/v3.5/P1-S-T-NH-SD-SMP-HP | 2 +- .../configs/rcu/v3.5/P2-2-t-nh-sd-SMP-hp | 2 +- .../configs/rcu/v3.5/P3-3-T-nh-SD-SMP-hp | 2 +- .../configs/rcu/v3.5/P4-A-t-NH-sd-SMP-HP | 2 +- .../configs/rcu/v3.5/P5-U-T-NH-sd-SMP-hp | 2 +- .../testing/selftests/rcutorture/doc/TINY_RCU.txt | 2 +- .../selftests/rcutorture/doc/TREE_RCU-kconfig.txt | 10 +++++----- 48 files changed, 74 insertions(+), 80 deletions(-)
Documentation/RCU/rcu.txt +2 -2
@@ -36,7 +36,7 @@ o How can the updater tell when a grace period has completed executed in user mode, or executed in the idle loop, we can safely free up that item. - Preemptible variants of RCU (CONFIG_TREE_PREEMPT_RCU) get the + Preemptible variants of RCU (CONFIG_PREEMPT_RCU) get the same effect, but require that the readers manipulate CPU-local counters. These counters allow limited types of blocking within RCU read-side critical sections. SRCU also uses CPU-local
@@ -81,7 +81,7 @@ o I hear that RCU is patented? What is with that? o I hear that RCU needs work in order to support realtime kernels? This work is largely completed. Realtime-friendly RCU can be - enabled via the CONFIG_TREE_PREEMPT_RCU kernel configuration + enabled via the CONFIG_PREEMPT_RCU kernel configuration parameter. However, work is in progress for enabling priority boosting of preempted RCU read-side critical sections. This is needed if you have CPU-bound realtime threads.
Documentation/RCU/stallwarn.txt +4 -4
@@ -77,7 +77,7 @@ This message indicates that CPU 5 detected that it was causing a stall, and that the stall was affecting RCU-sched. This message will normally be followed by a stack dump of the offending CPU. On TREE_RCU kernel builds, RCU and RCU-sched are implemented by the same underlying mechanism, -while on TREE_PREEMPT_RCU kernel builds, RCU is instead implemented +while on PREEMPT_RCU kernel builds, RCU is instead implemented by rcu_preempt_state. On the other hand, if the offending CPU fails to print out a stall-warning
@@ -89,7 +89,7 @@ INFO: rcu_bh_state detected stalls on CPUs/tasks: { 3 5 } (detected by 2, 2502 j This message indicates that CPU 2 detected that CPUs 3 and 5 were both causing stalls, and that the stall was affecting RCU-bh. This message will normally be followed by stack dumps for each CPU. Please note that -TREE_PREEMPT_RCU builds can be stalled by tasks as well as by CPUs, +PREEMPT_RCU builds can be stalled by tasks as well as by CPUs, and that the tasks will be indicated by PID, for example, "P3421". It is even possible for a rcu_preempt_state stall to be caused by both CPUs -and- tasks, in which case the offending CPUs and tasks will all
@@ -205,10 +205,10 @@ o A CPU-bound real-time task in a CONFIG_PREEMPT kernel, which might o A CPU-bound real-time task in a CONFIG_PREEMPT_RT kernel that is running at a higher priority than the RCU softirq threads. This will prevent RCU callbacks from ever being invoked, - and in a CONFIG_TREE_PREEMPT_RCU kernel will further prevent + and in a CONFIG_PREEMPT_RCU kernel will further prevent RCU grace periods from ever completing. Either way, the system will eventually run out of memory and hang. In the - CONFIG_TREE_PREEMPT_RCU case, you might see stall-warning + CONFIG_PREEMPT_RCU case, you might see stall-warning messages. o A hardware or software issue shuts off the scheduler-clock
Documentation/RCU/trace.txt +2 -2
@@ -8,7 +8,7 @@ The following sections describe the debugfs files and formats, first for rcutree and next for rcutiny. -CONFIG_TREE_RCU and CONFIG_TREE_PREEMPT_RCU debugfs Files and Formats +CONFIG_TREE_RCU and CONFIG_PREEMPT_RCU debugfs Files and Formats These implementations of RCU provide several debugfs directories under the top-level directory "rcu":
@@ -18,7 +18,7 @@ rcu/rcu_preempt rcu/rcu_sched Each directory contains files for the corresponding flavor of RCU. -Note that rcu/rcu_preempt is only present for CONFIG_TREE_PREEMPT_RCU. +Note that rcu/rcu_preempt is only present for CONFIG_PREEMPT_RCU. For CONFIG_TREE_RCU, the RCU flavor maps onto the RCU-sched flavor, so that activity for both appears in rcu/rcu_sched.
Documentation/RCU/whatisRCU.txt +1 -1
@@ -137,7 +137,7 @@ rcu_read_lock() Used by a reader to inform the reclaimer that the reader is entering an RCU read-side critical section. It is illegal to block while in an RCU read-side critical section, though - kernels built with CONFIG_TREE_PREEMPT_RCU can preempt RCU + kernels built with CONFIG_PREEMPT_RCU can preempt RCU read-side critical sections. Any RCU-protected data structure accessed during an RCU read-side critical section is guaranteed to remain unreclaimed for the full duration of that critical section.
include/linux/init_task.h +1 -1
@@ -102,7 +102,7 @@ extern struct group_info init_groups;#define INIT_IDS #endif -#ifdef CONFIG_TREE_PREEMPT_RCU +#ifdef CONFIG_PREEMPT_RCU #define INIT_TASK_RCU_TREE_PREEMPT () \ .rcu_blocked_node = NULL,#else
include/linux/rcupdate.h +3 -3
@@ -57,7 +57,7 @@ enum rcutorture_type { INVALID_RCU_FLAVOR }; -#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_TREE_PREEMPT_RCU ) +#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_PREEMPT_RCU )void rcutorture_get_gp_data (enum rcutorture_type test_type,int *flags,unsigned long *gpnum,unsigned long *completed);void rcutorture_record_test_transition (void );
@@ -365,7 +365,7 @@ typedef void call_rcu_func_t(struct rcu_head *head, void (*func)(struct rcu_head *head));void wait_rcu_gp (call_rcu_func_t crf); -#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_TREE_PREEMPT_RCU ) +#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_PREEMPT_RCU )#include <linux/rcutree.h> #elif defined(CONFIG_TINY_RCU )#include <linux/rcutiny.h>
@@ -852,7 +852,7 @@ static inline void rcu_preempt_sleep_check(void)* * In non-preemptible RCU implementations (TREE_RCU and TINY_RCU), * it is illegal to block while in an RCU read-side critical section. -* In preemptible RCU implementations (TREE_PREEMPT_RCU) in CONFIG_PREEMPT +* In preemptible RCU implementations (PREEMPT_RCU) in CONFIG_PREEMPT * kernel builds, RCU read-side critical sections may be preempted, * but explicit blocking is illegal. Finally, in preemptible RCU * implementations in real-time (with -rt patchset) kernel builds, RCU
include/linux/sched.h +2 -2
@@ -1278,9 +1278,9 @@ struct task_struct {union rcu_special rcu_read_unlock_special;struct list_head rcu_node_entry;#endif /* #ifdef CONFIG_PREEMPT_RCU */ -#ifdef CONFIG_TREE_PREEMPT_RCU +#ifdef CONFIG_PREEMPT_RCU struct rcu_node *rcu_blocked_node; -#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ +#endif /* #ifdef CONFIG_PREEMPT_RCU */ #ifdef CONFIG_TASKS_RCU unsigned long rcu_tasks_nvcsw;bool rcu_tasks_holdout;
include/trace/events/rcu.h +2 -2
@@ -36,7 +36,7 @@ TRACE_EVENT(rcu_utilization,#ifdef CONFIG_RCU_TRACE -#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_TREE_PREEMPT_RCU ) +#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_PREEMPT_RCU )/* * Tracepoint for grace-period events. Takes a string identifying the
@@ -345,7 +345,7 @@ TRACE_EVENT(rcu_fqs, __entry->cpu, __entry->qsevent) ); -#endif /* #if defined(CONFIG_TREE_RCU) || defined(CONFIG_TREE_PREEMPT_RCU) */ +#endif /* #if defined(CONFIG_TREE_RCU) || defined(CONFIG_PREEMPT_RCU) */ /* * Tracepoint for dyntick-idle entry/exit events. These take a string
init/Kconfig +8 -14
@@ -477,7 +477,7 @@ config TREE_RCU thousands of CPUs. It also scales down nicely to smaller systems. -config TREE_PREEMPT_RCU +config PREEMPT_RCU bool "Preemptible tree-based hierarchical RCU" depends on PREEMPT select IRQ_WORK
@@ -501,12 +501,6 @@ config TINY_RCU endchoice -config PREEMPT_RCU - def_bool TREE_PREEMPT_RCU - help - This option enables preemptible-RCU code that is common between - TREE_PREEMPT_RCU and, in the old days, TINY_PREEMPT_RCU. - config TASKS_RCU bool "Task_based RCU implementation using voluntary context switch" default n
@@ -518,7 +512,7 @@ config TASKS_RCU If unsure, say N. config RCU_STALL_COMMON - def_bool ( TREE_RCU || TREE_PREEMPT_RCU || RCU_TRACE ) + def_bool ( TREE_RCU || PREEMPT_RCU || RCU_TRACE ) help This option enables RCU CPU stall code that is common between the TINY and TREE variants of RCU. The purpose is to allow
@@ -576,7 +570,7 @@ config RCU_FANOUT int "Tree-based hierarchical RCU fanout value" range 2 64 if 64BIT range 2 32 if !64BIT - depends on TREE_RCU || TREE_PREEMPT_RCU + depends on TREE_RCU || PREEMPT_RCU default 64 if 64BIT default 32 if !64BIT help
@@ -596,7 +590,7 @@ config RCU_FANOUT_LEAF int "Tree-based hierarchical RCU leaf-level fanout value" range 2 RCU_FANOUT if 64BIT range 2 RCU_FANOUT if !64BIT - depends on TREE_RCU || TREE_PREEMPT_RCU + depends on TREE_RCU || PREEMPT_RCU default 16 help This option controls the leaf-level fanout of hierarchical
@@ -621,7 +615,7 @@ config RCU_FANOUT_LEAF config RCU_FANOUT_EXACT bool "Disable tree-based hierarchical RCU auto-balancing" - depends on TREE_RCU || TREE_PREEMPT_RCU + depends on TREE_RCU || PREEMPT_RCU default n help This option forces use of the exact RCU_FANOUT value specified,
@@ -652,11 +646,11 @@ config RCU_FAST_NO_HZ Say N if you are unsure. config TREE_RCU_TRACE - def_bool RCU_TRACE && ( TREE_RCU || TREE_PREEMPT_RCU ) + def_bool RCU_TRACE && ( TREE_RCU || PREEMPT_RCU ) select DEBUG_FS help This option provides tracing for the TREE_RCU and - TREE_PREEMPT_RCU implementations, permitting Makefile to + PREEMPT_RCU implementations, permitting Makefile to trivially select kernel/rcutree_trace.c. config RCU_BOOST
@@ -716,7 +710,7 @@ config RCU_BOOST_DELAY config RCU_NOCB_CPU bool "Offload RCU callback processing from boot-selected CPUs" - depends on TREE_RCU || TREE_PREEMPT_RCU + depends on TREE_RCU || PREEMPT_RCU default n help Use this option to reduce OS jitter for aggressive HPC or
kernel/rcu/Makefile +1 -1
@@ -1,6 +1,6 @@ obj-y += update.o srcu.o obj-$(CONFIG_RCU_TORTURE_TEST) += rcutorture.o obj-$(CONFIG_TREE_RCU) += tree.o -obj-$(CONFIG_TREE_PREEMPT_RCU) += tree.o +obj-$(CONFIG_PREEMPT_RCU) += tree.o obj-$(CONFIG_TREE_RCU_TRACE) += tree_trace.o obj-$(CONFIG_TINY_RCU) += tiny.o
kernel/rcu/tree.h +5 -5
@@ -139,7 +139,7 @@ struct rcu_node {unsigned long expmask;/* Groups that have ->blkd_tasks */ /* elements that need to drain to allow the */ /* current expedited grace period to */ -/* complete (only for TREE_PREEMPT_RCU). */ +/* complete (only for PREEMPT_RCU). */ unsigned long qsmaskinit;/* Per-GP initial value for qsmask & expmask. */ unsigned long grpmask;/* Mask to apply to parent qsmask. */
@@ -530,10 +530,10 @@ DECLARE_PER_CPU(struct rcu_data, rcu_sched_data);extern struct rcu_state rcu_bh_state; DECLARE_PER_CPU(struct rcu_data, rcu_bh_data); -#ifdef CONFIG_TREE_PREEMPT_RCU +#ifdef CONFIG_PREEMPT_RCU extern struct rcu_state rcu_preempt_state; DECLARE_PER_CPU(struct rcu_data, rcu_preempt_data); -#endif /* #ifdef CONFIG_TREE_PREEMPT_RCU */ +#endif /* #ifdef CONFIG_PREEMPT_RCU */ #ifdef CONFIG_RCU_BOOST DECLARE_PER_CPU(unsigned int, rcu_cpu_kthread_status);
@@ -563,10 +563,10 @@ static int rcu_preempt_offline_tasks(struct rcu_state *rsp,#endif /* #ifdef CONFIG_HOTPLUG_CPU */ static void rcu_preempt_check_callbacks (int cpu);void call_rcu (struct rcu_head *head,void (*func)(struct rcu_head *rcu)); -#if defined(CONFIG_HOTPLUG_CPU ) || defined(CONFIG_TREE_PREEMPT_RCU ) +#if defined(CONFIG_HOTPLUG_CPU ) || defined(CONFIG_PREEMPT_RCU )static void rcu_report_exp_rnp (struct rcu_state *rsp,struct rcu_node *rnp,bool wake); -#endif /* #if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_TREE_PREEMPT_RCU) */ +#endif /* #if defined(CONFIG_HOTPLUG_CPU) || defined(CONFIG_PREEMPT_RCU) */ static void __init __rcu_init_preempt (void );static void rcu_initiate_boost (struct rcu_node *rnp,unsigned long flags);static void rcu_preempt_boost_start_gp (struct rcu_node *rnp);
kernel/rcu/tree_plugin.h +3 -3
@@ -100,7 +100,7 @@ static void __init rcu_bootup_announce_oddness(void)#endif } -#ifdef CONFIG_TREE_PREEMPT_RCU +#ifdef CONFIG_PREEMPT_RCU RCU_STATE_INITIALIZER (rcu_preempt,'p' , call_rcu);static struct rcu_state *rcu_state_p = &rcu_preempt_state;
@@ -932,7 +932,7 @@ void exit_rcu(void) __rcu_read_unlock(); } -#else /* #ifdef CONFIG_TREE_PREEMPT_RCU */ +#else /* #ifdef CONFIG_PREEMPT_RCU */ static struct rcu_state *rcu_state_p = &rcu_sched_state;
@@ -1083,7 +1083,7 @@ void exit_rcu(void) { } -#endif /* #else #ifdef CONFIG_TREE_PREEMPT_RCU */ +#endif /* #else #ifdef CONFIG_PREEMPT_RCU */ #ifdef CONFIG_RCU_BOOST
kernel/rcu/update.c +1 -1
@@ -306,7 +306,7 @@ struct debug_obj_descr rcuhead_debug_descr = {EXPORT_SYMBOL_GPL (rcuhead_debug_descr);#endif /* #ifdef CONFIG_DEBUG_OBJECTS_RCU_HEAD */ -#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_TREE_PREEMPT_RCU ) || defined(CONFIG_RCU_TRACE ) +#if defined(CONFIG_TREE_RCU ) || defined(CONFIG_PREEMPT_RCU ) || defined(CONFIG_RCU_TRACE )void do_trace_rcu_torture_read (const char *rcutorturename,struct rcu_head *rhp,unsigned long secs,unsigned long c_old,unsigned long c)
lib/Kconfig.debug +1 -1
@@ -1252,7 +1252,7 @@ config RCU_CPU_STALL_VERBOSE config RCU_CPU_STALL_INFO bool "Print additional diagnostics on RCU CPU stall" - depends on (TREE_RCU || TREE_PREEMPT_RCU) && DEBUG_KERNEL + depends on (TREE_RCU || PREEMPT_RCU) && DEBUG_KERNEL default n help For each stalled CPU that is aware of the current RCU grace
tools/testing/selftests/rcutorture/configs/rcu/TREE01 +1 -1
@@ -2,7 +2,7 @@ CONFIG_SMP=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y CONFIG_NO_HZ_FULL=n
tools/testing/selftests/rcutorture/configs/rcu/TREE02 +1 -1
@@ -3,7 +3,7 @@ CONFIG_NR_CPUS=8 CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y CONFIG_NO_HZ_FULL=n
tools/testing/selftests/rcutorture/configs/rcu/TREE02-T +1 -1
@@ -3,7 +3,7 @@ CONFIG_NR_CPUS=8 CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y CONFIG_NO_HZ_FULL=n
tools/testing/selftests/rcutorture/configs/rcu/TREE03 +1 -1
@@ -3,7 +3,7 @@ CONFIG_NR_CPUS=8 CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_HZ_PERIODIC=y CONFIG_NO_HZ_IDLE=n CONFIG_NO_HZ_FULL=n
tools/testing/selftests/rcutorture/configs/rcu/TREE08 +1 -1
@@ -3,7 +3,7 @@ CONFIG_NR_CPUS=16 CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y CONFIG_NO_HZ_FULL=n
tools/testing/selftests/rcutorture/configs/rcu/TREE08-T +1 -1
@@ -3,7 +3,7 @@ CONFIG_NR_CPUS=16 CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y CONFIG_NO_HZ_FULL=n
tools/testing/selftests/rcutorture/configs/rcu/TREE09 +1 -1
@@ -3,7 +3,7 @@ CONFIG_NR_CPUS=1 CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_HZ_PERIODIC=n CONFIG_NO_HZ_IDLE=y CONFIG_NO_HZ_FULL=n
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P1-S-T-NH-SD-SMP-HP +1 -1
@@ -9,7 +9,7 @@ CONFIG_HOTPLUG_CPU=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P2-2-t-nh-sd-SMP-hp +1 -1
@@ -10,7 +10,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P3-3-T-nh-SD-SMP-hp +1 -1
@@ -10,7 +10,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P4-A-t-NH-sd-SMP-HP +1 -1
@@ -8,7 +8,7 @@ CONFIG_HOTPLUG_CPU=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_RT_MUTEXES=y
tools/testing/selftests/rcutorture/configs/rcu/v0.0/P5-U-T-NH-sd-SMP-hp +1 -1
@@ -11,7 +11,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_OBJECTS=y CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P1-S-T-NH-SD-SMP-HP +1 -1
@@ -10,7 +10,7 @@ CONFIG_HOTPLUG_CPU=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P2-2-t-nh-sd-SMP-hp +1 -1
@@ -10,7 +10,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P3-3-T-nh-SD-SMP-hp +1 -1
@@ -10,7 +10,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P4-A-t-NH-sd-SMP-HP +1 -1
@@ -8,7 +8,7 @@ CONFIG_HOTPLUG_CPU=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_RT_MUTEXES=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P5-U-T-NH-sd-SMP-hp +1 -1
@@ -11,7 +11,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_OBJECTS=y CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P6---t-nh-SD-smp-hp +1 -1
@@ -8,7 +8,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -CONFIG_TREE_PREEMPT_RCU=y +CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP +1 -1
@@ -14,7 +14,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_PROVE_LOCKING=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP-all +1 -1
@@ -14,7 +14,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_PROVE_LOCKING=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-HP-none +1 -1
@@ -14,7 +14,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_PROVE_LOCKING=y
tools/testing/selftests/rcutorture/configs/rcu/v3.12/P7-4-T-NH-SD-SMP-hp +1 -1
@@ -14,7 +14,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_PROVE_LOCKING=y
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P1-S-T-NH-SD-SMP-HP +1 -1
@@ -10,7 +10,7 @@ CONFIG_HOTPLUG_CPU=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P2-2-t-nh-sd-SMP-hp +1 -1
@@ -10,7 +10,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P3-3-T-nh-SD-SMP-hp +1 -1
@@ -10,7 +10,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P4-A-t-NH-sd-SMP-HP +1 -1
@@ -8,7 +8,7 @@ CONFIG_HOTPLUG_CPU=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_RT_MUTEXES=y
tools/testing/selftests/rcutorture/configs/rcu/v3.3/P5-U-T-NH-sd-SMP-hp +1 -1
@@ -11,7 +11,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_OBJECTS=y CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P1-S-T-NH-SD-SMP-HP +1 -1
@@ -10,7 +10,7 @@ CONFIG_HOTPLUG_CPU=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P2-2-t-nh-sd-SMP-hp +1 -1
@@ -10,7 +10,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P3-3-T-nh-SD-SMP-hp +1 -1
@@ -10,7 +10,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_SYSFS_DEPRECATED_V2=y
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P4-A-t-NH-sd-SMP-HP +1 -1
@@ -8,7 +8,7 @@ CONFIG_HOTPLUG_CPU=y CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_RCU_TORTURE_TEST=m CONFIG_MODULE_UNLOAD=y CONFIG_RT_MUTEXES=y
tools/testing/selftests/rcutorture/configs/rcu/v3.5/P5-U-T-NH-sd-SMP-hp +1 -1
@@ -11,7 +11,7 @@ CONFIG_HIBERNATION=n CONFIG_PREEMPT_NONE=n CONFIG_PREEMPT_VOLUNTARY=n CONFIG_PREEMPT=y -#CHECK#CONFIG_TREE_PREEMPT_RCU=y +#CHECK#CONFIG_PREEMPT_RCU=y CONFIG_DEBUG_KERNEL=y CONFIG_DEBUG_OBJECTS=y CONFIG_DEBUG_OBJECTS_RCU_HEAD=y
tools/testing/selftests/rcutorture/doc/TINY_RCU.txt +1 -1
@@ -34,7 +34,7 @@ CONFIG_PREEMPT CONFIG_PREEMPT_RCU CONFIG_SMP CONFIG_TINY_RCU -CONFIG_TREE_PREEMPT_RCU +CONFIG_PREEMPT_RCU CONFIG_TREE_RCU All forced by CONFIG_TINY_RCU.
tools/testing/selftests/rcutorture/doc/TREE_RCU-kconfig.txt +5 -5
@@ -1,5 +1,5 @@ This document gives a brief rationale for the TREE_RCU-related test -cases, a group that includes TREE_PREEMPT_RCU. +cases, a group that includes PREEMPT_RCU. Kconfig Parameters:
@@ -14,7 +14,7 @@ CONFIG_NO_HZ_FULL_SYSIDLE -- Do one. CONFIG_PREEMPT -- Do half. (First three and #8.) CONFIG_PROVE_LOCKING -- Do all but two, covering CONFIG_PROVE_RCU and not. CONFIG_PROVE_RCU -- Do all but one under CONFIG_PROVE_LOCKING. -CONFIG_RCU_BOOST -- one of TREE_PREEMPT_RCU. +CONFIG_RCU_BOOST -- one of PREEMPT_RCU. CONFIG_RCU_KTHREAD_PRIO -- set to 2 for _BOOST testing. CONFIG_RCU_CPU_STALL_INFO -- Do one. CONFIG_RCU_CPU_STALL_VERBOSE -- do one with and without _INFO.
@@ -27,7 +27,7 @@ CONFIG_RCU_NOCB_CPU_ALL -- Do one. CONFIG_RCU_NOCB_CPU_NONE -- Do one. CONFIG_RCU_NOCB_CPU_ZERO -- Do one. CONFIG_RCU_TRACE -- Do half. -CONFIG_SMP -- Need one !SMP for TREE_PREEMPT_RCU. +CONFIG_SMP -- Need one !SMP for PREEMPT_RCU. RCU-bh: Do one with PREEMPT and one with !PREEMPT. RCU-sched: Do one with PREEMPT but not BOOST.
@@ -77,7 +77,7 @@ CONFIG_RCU_CPU_STALL_TIMEOUT CONFIG_RCU_STALL_COMMON - Implied by TREE_RCU and TREE_PREEMPT_RCU. + Implied by TREE_RCU and PREEMPT_RCU. CONFIG_RCU_TORTURE_TEST CONFIG_RCU_TORTURE_TEST_RUNNABLE
@@ -88,7 +88,7 @@ CONFIG_RCU_USER_QS Redundant with CONFIG_NO_HZ_FULL. -CONFIG_TREE_PREEMPT_RCU +CONFIG_PREEMPT_RCU CONFIG_TREE_RCU These are controlled by CONFIG_PREEMPT.
Signature · 1 line
1.8.1.5
PATCH 6/7 rcu: Kick rcuo kthreads after their CPU goes offline
In reply to Paul E. McKenneyFrom: "Paul E. McKenney" <[email protected]>
If a no-CBs CPU were to post an RCU callback with interrupts disabled after it entered the idle loop for the last time, there might be no deferred wakeup for the corresponding rcuo kthreads. This commit therefore adds a set of calls to do_nocb_deferred_wakeup() after the CPU has gone completely offline.
--- kernel/rcu/tree.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/rcu/tree.c +3 -1
@@ -3497,8 +3497,10 @@ static int rcu_cpu_notify(struct notifier_block *self,case CPU_DEAD_FROZEN :case CPU_UP_CANCELED :case CPU_UP_CANCELED_FROZEN : -for_each_rcu_flavor (rsp) +for_each_rcu_flavor (rsp) {rcu_cleanup_dead_cpu (cpu, rsp); +do_nocb_deferred_wakeup (per_cpu_ptr (rsp->rda, cpu)); + }break ;default :break ;
Signature · 1 line
1.8.1.5
PATCH 7/7 rcu: Fix for rcuo online-time-creation reorganization bug
In reply to Paul E. McKenneyFrom: "Paul E. McKenney" <[email protected]>
Commit 35ce7f29a44a (rcu: Create rcuo kthreads only for onlined CPUs) contains checks for the case where CPUs are brought online out of order, re-wiring the rcuo leader-follower relationships as needed. Unfortunately, this rewiring was broken. This apparently went undetected due to the tendency of systems to bring CPUs online in order. This commit nevertheless fixes the rewiring.
--- kernel/rcu/tree_plugin.h | 10 +++++++--- 1 file changed, 7 insertions(+), 3 deletions(-)
kernel/rcu/tree_plugin.h +7 -3
@@ -2586,9 +2586,13 @@ static void rcu_spawn_one_nocb_kthread(struct rcu_state *rsp, int cpu) rdp->nocb_leader = rdp_spawn;if (rdp_last && rdp != rdp_spawn) rdp_last->nocb_next_follower = rdp; - rdp_last = rdp; - rdp = rdp->nocb_next_follower; - rdp_last->nocb_next_follower =NULL ; +if (rdp == rdp_spawn) { + rdp = rdp->nocb_next_follower; + }else { + rdp_last = rdp; + rdp = rdp->nocb_next_follower; + rdp_last->nocb_next_follower =NULL ; + } }while (rdp); rdp_spawn->nocb_next_follower = rdp_old_leader; }
Signature · 1 line
1.8.1.5