Callback-offload (nocb) updates for v6.1
Paul E. McKenney
maintainercover Callback-offload (nocb) updates for v6.1
Hello!
This series provides NOCB-CPU updates:
1. Choose the right rcuog/rcuop kthreads to output, courtesy
of Zqiang.
2. Add CPU number to CPU-{,de}offload failure messages.
3. Simplify rcu_init_nohz() cpumask handling, courtesy of Zhen Lei.
Thanx, Paul
------------------------------------------------------------------------
b/kernel/rcu/tree_nocb.h | 6 +++--- kernel/rcu/tree_nocb.h | 36 +++++++++++------------------------- 2 files changed, 14 insertions(+), 28 deletions(-)
Paul E. McKenney
maintainerPATCH 1/3 rcu/nocb: Choose the right rcuog/rcuop kthreads to output
From: Zqiang <[email protected]>
The show_rcu_nocb_gp_state() function is supposed to dump out the rcuog kthread and the show_rcu_nocb_state() function is supposed to dump out the rcuo[ps] kthread. Currently, both do a mixture, which is not optimal for debugging, even though it does not affect functionality.
This commit therefore adjusts these two functions to focus on their respective kthreads.
--- kernel/rcu/tree_nocb.h | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/rcu/tree_nocb.h +3 -3
@@ -1452,8 +1452,8 @@ static void show_rcu_nocb_gp_state(struct rcu_data *rdp) (long )rdp->nocb_gp_seq, rnp->grplo, rnp->grphi,READ_ONCE (rdp->nocb_gp_loops), rdp->nocb_gp_kthread ?task_state_to_char (rdp->nocb_gp_kthread) :'.' , - rdp->nocb_cb_kthread ? (int )task_cpu (rdp->nocb_gp_kthread) :-1 , - show_rcu_should_be_on_cpu(rdp->nocb_cb_kthread)); + rdp->nocb_gp_kthread ? (int )task_cpu (rdp->nocb_gp_kthread) :-1 , + show_rcu_should_be_on_cpu(rdp->nocb_gp_kthread)); }/* Dump out nocb kthread state for the specified rcu_data structure. */
@@ -1497,7 +1497,7 @@ static void show_rcu_nocb_state(struct rcu_data *rdp)".B" [!!rcu_cblist_n_cbs (&rdp->nocb_bypass)],rcu_segcblist_n_cbs (&rdp->cblist), rdp->nocb_cb_kthread ?task_state_to_char (rdp->nocb_cb_kthread) :'.' , - rdp->nocb_cb_kthread ? (int )task_cpu (rdp->nocb_gp_kthread) :-1 , + rdp->nocb_cb_kthread ? (int )task_cpu (rdp->nocb_cb_kthread) :-1 , show_rcu_should_be_on_cpu(rdp->nocb_cb_kthread));/* It is OK for GP kthreads to have GP state. */
Signature · 1 line
2.31.1.189.g2e36527f23
Paul E. McKenney
maintainerPATCH 2/3 rcu/nocb: Add CPU number to CPU-{,de}offload failure messages
In reply to Paul E. McKenneyOffline CPUs cannot be offloaded or deoffloaded. Any attempt to offload or deoffload an offline CPU causes a message to be printed on the console, which is good, but this message does not contain the CPU number, which is bad. Such a CPU number can be helpful when debugging, as it gives a clear indication that the CPU in question is in fact offline. This commit therefore adds the CPU number to the CPU-{,de}offload failure messages.
--- kernel/rcu/tree_nocb.h | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/rcu/tree_nocb.h +2 -2
@@ -1111,7 +1111,7 @@ int rcu_nocb_cpu_deoffload(int cpu)if (!ret)cpumask_clear_cpu (cpu, rcu_nocb_mask); }else { -pr_info ("NOCB: Can't CB-deoffload an offline CPU\n" ); +pr_info ("NOCB: Cannot CB-deoffload offline CPU %d\n" , rdp->cpu); ret = -EINVAL ; } }
@@ -1196,7 +1196,7 @@ int rcu_nocb_cpu_offload(int cpu)if (!ret)cpumask_set_cpu (cpu, rcu_nocb_mask); }else { -pr_info ("NOCB: Can't CB-offload an offline CPU\n" ); +pr_info ("NOCB: Cannot CB-offload offline CPU %d\n" , rdp->cpu); ret = -EINVAL ; } }
Signature · 1 line
2.31.1.189.g2e36527f23
Paul E. McKenney
maintainerPATCH 3/3 rcu: Simplify rcu_init_nohz() cpumask handling
In reply to Paul E. McKenneyFrom: Zhen Lei <[email protected]>
In kernels built with either CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y or CONFIG_NO_HZ_FULL=y, additional CPUs must be added to rcu_nocb_mask. Except that kernels booted without the rcu_nocbs= will not have allocated rcu_nocb_mask. And the current rcu_init_nohz() function uses its need_rcu_nocb_mask and offload_all local variables to track the rcu_nocb and nohz_full state.
But there is a much simpler approach, namely creating a cpumask pointer to track the default and then using cpumask_available() to check the rcu_nocb_mask state. This commit takes this approach, thereby simplifying and shortening the rcu_init_nohz() function.
--- kernel/rcu/tree_nocb.h | 32 +++++++++----------------------- 1 file changed, 9 insertions(+), 23 deletions(-)
kernel/rcu/tree_nocb.h +9 -23
@@ -1210,45 +1210,31 @@ EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload);void __init rcu_init_nohz (void ) {int cpu; -bool need_rcu_nocb_mask = false; -bool offload_all = false;struct rcu_data *rdp; +const struct cpumask *cpumask =NULL ;#if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL ) -if (!rcu_state.nocb_is_setup) { - need_rcu_nocb_mask = true; - offload_all = true; - } -#endif /* #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL) */ - -#if defined(CONFIG_NO_HZ_FULL ) -if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) { - need_rcu_nocb_mask = true; - offload_all = false;/* NO_HZ_FULL has its own mask. */ - } -#endif /* #if defined(CONFIG_NO_HZ_FULL) */ + cpumask = cpu_possible_mask; +#elif defined(CONFIG_NO_HZ_FULL ) +if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) + cpumask = tick_nohz_full_mask; +#endif -if (need_rcu_nocb_mask) { +if (cpumask) {if (!cpumask_available (rcu_nocb_mask)) {if (!zalloc_cpumask_var (&rcu_nocb_mask,GFP_KERNEL )) {pr_info ("rcu_nocb_mask allocation failed, callback offloading disabled.\n" );return ; } } + +cpumask_or (rcu_nocb_mask, rcu_nocb_mask, cpumask); rcu_state.nocb_is_setup = true; }if (!rcu_state.nocb_is_setup)return ; -#if defined(CONFIG_NO_HZ_FULL ) -if (tick_nohz_full_running) -cpumask_or (rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask); -#endif /* #if defined(CONFIG_NO_HZ_FULL) */ - -if (offload_all) -cpumask_setall (rcu_nocb_mask); -if (!cpumask_subset (rcu_nocb_mask, cpu_possible_mask)) {pr_info ("\tNote: kernel parameter 'rcu_nocbs=', 'nohz_full', or 'isolcpus=' contains nonexistent CPUs.\n" ); cpumask_and(rcu_nocb_mask, cpu_possible_mask,
Signature · 1 line
2.31.1.189.g2e36527f23
Frederic Weisbecker
maintainerRe: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling
Quoted 38 lines · Paul E. McKenney
From: Zhen Lei <[email protected]>
In kernels built with either
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=yorCONFIG_NO_HZ_FULL=y, additional CPUs must be added torcu_nocb_mask. Except that kernels booted without the rcu_nocbs= will not have allocatedrcu_nocb_mask. And the currentrcu_init_nohz()function uses itsneed_rcu_nocb_maskandoffload_alllocal variables to track thercu_nocbandnohz_fullstate.But there is a much simpler approach, namely creating a cpumask pointer to track the default and then using
cpumask_available()to check thercu_nocb_maskstate. This commit takes this approach, thereby simplifying and shortening thercu_init_nohz()function.Signed-off-by Zhen Lei <[email protected]>Signed-off-by Paul E. McKenney <[email protected]>Reviewed-by Joel Fernandes (Google) <[email protected]>--- kernel/rcu/tree_nocb.h | 32 +++++++++----------------------- 1 file changed, 9 insertions(+), 23 deletions(-)kernel/rcu/tree_nocb.h +1
-5@@ -1210,45 +1210,31 @@ EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload);void __init rcu_init_nohz (void ) {int cpu; -bool need_rcu_nocb_mask = false; -bool offload_all = false;struct rcu_data *rdp; +const struct cpumask *cpumask =NULL ;#if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL ) -if (!rcu_state.nocb_is_setup) { - need_rcu_nocb_mask = true; - offload_all = true;
kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1217
Quoted 10 lines
kernel/rcu/tree_nocb.h +1
-9- } -#endif /* #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL) */ - -#if defined(CONFIG_NO_HZ_FULL ) -if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) { - need_rcu_nocb_mask = true; - offload_all = false;/* NO_HZ_FULL has its own mask. */ - } -#endif /* #if defined(CONFIG_NO_HZ_FULL) */ + cpumask = cpu_possible_mask;
You're missing the rcu_state.nocb_is_setup check, so CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now always override the rcu_nocbs= parameter (should be the other way around).
kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221
Quoted 4 lines
kernel/rcu/tree_nocb.h +4
-0+#elif defined(CONFIG_NO_HZ_FULL ) +if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) + cpumask = tick_nohz_full_mask; +#endif
A subtle behaviour difference here too: CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now override nohz_full=
I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.
Thanks.
Quoted 31 lines
kernel/rcu/tree_nocb.h +3
-9-if (need_rcu_nocb_mask) { +if (cpumask) {if (!cpumask_available (rcu_nocb_mask)) {if (!zalloc_cpumask_var (&rcu_nocb_mask,GFP_KERNEL )) {pr_info ("rcu_nocb_mask allocation failed, callback offloading disabled.\n" );return ; } } + +cpumask_or (rcu_nocb_mask, rcu_nocb_mask, cpumask); rcu_state.nocb_is_setup = true; }if (!rcu_state.nocb_is_setup)return ; -#if defined(CONFIG_NO_HZ_FULL ) -if (tick_nohz_full_running) -cpumask_or (rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask); -#endif /* #if defined(CONFIG_NO_HZ_FULL) */ - -if (offload_all) -cpumask_setall (rcu_nocb_mask); -if (!cpumask_subset (rcu_nocb_mask, cpu_possible_mask)) {pr_info ("\tNote: kernel parameter 'rcu_nocbs=', 'nohz_full', or 'isolcpus=' contains nonexistent CPUs.\n" ); cpumask_and(rcu_nocb_mask, cpu_possible_mask,Signature · 1 line
2.31.1.189.g2e36527f23
Paul E. McKenney
maintainerRe: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling
Quoted 45 lines · Frederic Weisbecker
Quoted 44 lines · Paul E. McKenney
From: Zhen Lei <[email protected]>
In kernels built with either
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=yorCONFIG_NO_HZ_FULL=y, additional CPUs must be added torcu_nocb_mask. Except that kernels booted without the rcu_nocbs= will not have allocatedrcu_nocb_mask. And the currentrcu_init_nohz()function uses itsneed_rcu_nocb_maskandoffload_alllocal variables to track thercu_nocbandnohz_fullstate.But there is a much simpler approach, namely creating a cpumask pointer to track the default and then using
cpumask_available()to check thercu_nocb_maskstate. This commit takes this approach, thereby simplifying and shortening thercu_init_nohz()function.Signed-off-by Zhen Lei <[email protected]>Signed-off-by Paul E. McKenney <[email protected]>Reviewed-by Joel Fernandes (Google) <[email protected]>--- kernel/rcu/tree_nocb.h | 32 +++++++++----------------------- 1 file changed, 9 insertions(+), 23 deletions(-)kernel/rcu/tree_nocb.h +1
-11@@ -1210,45 +1210,31 @@ EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload);void __init rcu_init_nohz (void ) {int cpu; -bool need_rcu_nocb_mask = false; -bool offload_all = false;struct rcu_data *rdp; +const struct cpumask *cpumask =NULL ;#if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL ) -if (!rcu_state.nocb_is_setup) { - need_rcu_nocb_mask = true; - offload_all = true; - } -#endif /* #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL) */ - -#if defined(CONFIG_NO_HZ_FULL ) -if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) { - need_rcu_nocb_mask = true;
Quoted 8 lines
kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1217
Quoted 4 lines
kernel/rcu/tree_nocb.h +1
-3- offload_all = false; /* NO_HZ_FULL has its own mask. */ - } -#endif /* #if defined(CONFIG_NO_HZ_FULL) */ + cpumask = cpu_possible_mask;You're missing the rcu_state.nocb_is_setup check, so
CONFIG_RCU_NOCB_CPU_DEFAULT_ALLwill now always override the rcu_nocbs= parameter (should be the other way around).
This is indeed a problem, so especially thank you for looking this over!
I will revert this patch.
Quoted 10 lines
kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221
Quoted 4 lines
kernel/rcu/tree_nocb.h +4
-0+#elif defined(CONFIG_NO_HZ_FULL ) +if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) + cpumask = tick_nohz_full_mask; +#endif A subtle behaviour difference here too:
CONFIG_RCU_NOCB_CPU_DEFAULT_ALLwill now override nohz_full=I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.
Good point. Perhaps the key point is that if there is nohz_full=, rcu_nocbs=, and CONFIG_RCU_NOCB_CPU_DEFAULT_ALL, we still need rcu_nocbs= to include at least those bits set by nohz_full=.
Are there any other constraints?
Zhen Lei, please feel free to submit an updated patch.
Thanx, Paul
Quoted 33 lines
Thanks.
Quoted 31 lines
kernel/rcu/tree_nocb.h +3
-9-if (need_rcu_nocb_mask) { +if (cpumask) {if (!cpumask_available (rcu_nocb_mask)) {if (!zalloc_cpumask_var (&rcu_nocb_mask,GFP_KERNEL )) {pr_info ("rcu_nocb_mask allocation failed, callback offloading disabled.\n" );return ; } } + +cpumask_or (rcu_nocb_mask, rcu_nocb_mask, cpumask); rcu_state.nocb_is_setup = true; }if (!rcu_state.nocb_is_setup)return ; -#if defined(CONFIG_NO_HZ_FULL ) -if (tick_nohz_full_running) -cpumask_or (rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask); -#endif /* #if defined(CONFIG_NO_HZ_FULL) */ - -if (offload_all) -cpumask_setall (rcu_nocb_mask); -if (!cpumask_subset (rcu_nocb_mask, cpu_possible_mask)) {pr_info ("\tNote: kernel parameter 'rcu_nocbs=', 'nohz_full', or 'isolcpus=' contains nonexistent CPUs.\n" ); cpumask_and(rcu_nocb_mask, cpu_possible_mask,Signature · 1 line
2.31.1.189.g2e36527f23
Frederic Weisbecker
maintainerRe: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling
Quoted 9 lines · Paul E. McKenney
Quoted 8 lines · Frederic Weisbecker
kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221
Quoted 4 lines
kernel/rcu/tree_nocb.h +4
-0+#elif defined(CONFIG_NO_HZ_FULL ) +if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) + cpumask = tick_nohz_full_mask; +#endif A subtle behaviour difference here too:
CONFIG_RCU_NOCB_CPU_DEFAULT_ALLwill now override nohz_full=
Quoted 6 lines
Quoted 2 lines
I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.
Good point. Perhaps the key point is that if there is nohz_full=, rcu_nocbs=, and
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL, we still need rcu_nocbs= to include at least those bits set by nohz_full=.
Not sure I get what you mean. nohz_full= should in any case always force rcu_nocbs at least on the nohz_full CPUs.
For example assuming the following combination: rcu_nocbs=6, nohz_full=7 AND CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result should be:
NOCB CPUs = 6,7NOHZ_FULL CPUs = 7
(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y is overriden by rcu_nocbs=6).
Now if we have nohz_full=7 AND CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result is expected to be either:
NOCB CPUs = 7 (upstream behaviour)NOHZ_FULL CPUs = 7
or
NOCB CPUs = allNOHZ_FULL CPUs = 7
The second makes more sense IMHO but that should be in a separate change.
Thanks.
Re: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling
Quoted 26 lines · Frederic Weisbecker
Quoted 9 lines · Paul E. McKenney
Quoted 8 lines · Frederic Weisbecker
kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221
Quoted 4 lines
kernel/rcu/tree_nocb.h +4
-0+#elif defined(CONFIG_NO_HZ_FULL ) +if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) + cpumask = tick_nohz_full_mask; +#endif A subtle behaviour difference here too:
CONFIG_RCU_NOCB_CPU_DEFAULT_ALLwill now override nohz_full=Quoted 6 lines
Quoted 2 lines
I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.
Good point. Perhaps the key point is that if there is nohz_full=, rcu_nocbs=, and
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL, we still need rcu_nocbs= to include at least those bits set by nohz_full=.Not sure I get what you mean. nohz_full= should in any case always force
rcu_nocbsat least on thenohz_fullCPUs.For example assuming the following combination: rcu_nocbs=6, nohz_full=7 AND
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result should be:NOCB CPUs = 6,7
NOHZ_FULLCPUs = 7
Quoted 7 lines
(
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=yis overriden by rcu_nocbs=6).Now if we have nohz_full=7 AND
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result is expected to be either:NOCB CPUs = 7 (upstream behaviour)
NOHZ_FULLCPUs = 7
OK, I got it. CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y takes effect only when there are no 'rcu_nocbs=' and 'nohz_full='.
Quoted 7 lines
or
NOCB CPUs = all
NOHZ_FULLCPUs = 7The second makes more sense IMHO but that should be in a separate change.
OK, in this case, CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y takes effect only when there is no 'rcu_nocbs='.
Quoted 4 lines
Thanks.
.
Signature · 2 lines
Regards, Zhen Lei
Paul E. McKenney
maintainerRe: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling
In reply to Frederic WeisbeckerQuoted 34 lines · Frederic Weisbecker
Quoted 9 lines · Paul E. McKenney
Quoted 8 lines · Frederic Weisbecker
kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221
Quoted 4 lines
kernel/rcu/tree_nocb.h +4
-0+#elif defined(CONFIG_NO_HZ_FULL ) +if (tick_nohz_full_running && !cpumask_empty (tick_nohz_full_mask)) + cpumask = tick_nohz_full_mask; +#endif A subtle behaviour difference here too:
CONFIG_RCU_NOCB_CPU_DEFAULT_ALLwill now override nohz_full=Quoted 6 lines
Quoted 2 lines
I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.
Good point. Perhaps the key point is that if there is nohz_full=, rcu_nocbs=, and
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL, we still need rcu_nocbs= to include at least those bits set by nohz_full=.Not sure I get what you mean. nohz_full= should in any case always force
rcu_nocbsat least on thenohz_fullCPUs.For example assuming the following combination: rcu_nocbs=6, nohz_full=7 AND
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result should be:NOCB CPUs = 6,7
NOHZ_FULLCPUs = 7(
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=yis overriden by rcu_nocbs=6).Now if we have nohz_full=7 AND
CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result is expected to be either:NOCB CPUs = 7 (upstream behaviour)
NOHZ_FULLCPUs = 7
Quoted 6 lines
or
NOCB CPUs = all
NOHZ_FULLCPUs = 7The second makes more sense IMHO but that should be in a separate change.
Your examples are consistent with what I was trying to say. ;-)
Thanx, Paul