Skip to contentSkip to conversation

Callback-offload (nocb) updates for v6.1

PATCH New Paul E. McKenney 3 participants 9 messages Aug 31, 2022 – Sep 1, 2022 RCU

lore
Share & tools
lore

Paul E. McKenney

author maintainer
#0 lore raw

cover Callback-offload (nocb) updates for v6.1

Hello!

This series provides NOCB-CPU updates:

1. Choose the right rcuog/rcuop kthreads to output, courtesy
of Zqiang.

2. Add CPU number to CPU-{,de}offload failure messages.

3. Simplify rcu_init_nohz() cpumask handling, courtesy of Zhen Lei.

Thanx, Paul

------------------------------------------------------------------------

 b/kernel/rcu/tree_nocb.h |    6 +++---
 kernel/rcu/tree_nocb.h   |   36 +++++++++++-------------------------
 2 files changed, 14 insertions(+), 28 deletions(-)
author maintainer
#1 lore raw

PATCH 1/3 rcu/nocb: Choose the right rcuog/rcuop kthreads to output

From: Zqiang <[email protected]>

The show_rcu_nocb_gp_state() function is supposed to dump out the rcuog kthread and the show_rcu_nocb_state() function is supposed to dump out the rcuo[ps] kthread. Currently, both do a mixture, which is not optimal for debugging, even though it does not affect functionality.

This commit therefore adjusts these two functions to focus on their respective kthreads.

---
 kernel/rcu/tree_nocb.h | 6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
kernel/rcu/tree_nocb.h +3 -3
index a8f574d8850d2..f20aec4f4394b 100644
@@ -1452,8 +1452,8 @@ static void show_rcu_nocb_gp_state(struct rcu_data *rdp)
 		(long)rdp->nocb_gp_seq,
 		rnp->grplo, rnp->grphi, READ_ONCE(rdp->nocb_gp_loops),
 		rdp->nocb_gp_kthread ? task_state_to_char(rdp->nocb_gp_kthread) : '.',
-		rdp->nocb_cb_kthread ? (int)task_cpu(rdp->nocb_gp_kthread) : -1,
-		show_rcu_should_be_on_cpu(rdp->nocb_cb_kthread));
+		rdp->nocb_gp_kthread ? (int)task_cpu(rdp->nocb_gp_kthread) : -1,
+		show_rcu_should_be_on_cpu(rdp->nocb_gp_kthread));
 }
 
 /* Dump out nocb kthread state for the specified rcu_data structure. */
@@ -1497,7 +1497,7 @@ static void show_rcu_nocb_state(struct rcu_data *rdp)
 		".B"[!!rcu_cblist_n_cbs(&rdp->nocb_bypass)],
 		rcu_segcblist_n_cbs(&rdp->cblist),
 		rdp->nocb_cb_kthread ? task_state_to_char(rdp->nocb_cb_kthread) : '.',
-		rdp->nocb_cb_kthread ? (int)task_cpu(rdp->nocb_gp_kthread) : -1,
+		rdp->nocb_cb_kthread ? (int)task_cpu(rdp->nocb_cb_kthread) : -1,
 		show_rcu_should_be_on_cpu(rdp->nocb_cb_kthread));
 
 	/* It is OK for GP kthreads to have GP state. */
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#2 lore raw

PATCH 2/3 rcu/nocb: Add CPU number to CPU-{,de}offload failure messages

In reply to Paul E. McKenney

Offline CPUs cannot be offloaded or deoffloaded. Any attempt to offload or deoffload an offline CPU causes a message to be printed on the console, which is good, but this message does not contain the CPU number, which is bad. Such a CPU number can be helpful when debugging, as it gives a clear indication that the CPU in question is in fact offline. This commit therefore adds the CPU number to the CPU-{,de}offload failure messages.

Cc Frederic Weisbecker <[email protected]>
---
 kernel/rcu/tree_nocb.h | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
kernel/rcu/tree_nocb.h +2 -2
index f20aec4f4394b..0a5f0ef414845 100644
@@ -1111,7 +1111,7 @@ int rcu_nocb_cpu_deoffload(int cpu)
 			if (!ret)
 				cpumask_clear_cpu(cpu, rcu_nocb_mask);
 		} else {
-			pr_info("NOCB: Can't CB-deoffload an offline CPU\n");
+			pr_info("NOCB: Cannot CB-deoffload offline CPU %d\n", rdp->cpu);
 			ret = -EINVAL;
 		}
 	}
@@ -1196,7 +1196,7 @@ int rcu_nocb_cpu_offload(int cpu)
 			if (!ret)
 				cpumask_set_cpu(cpu, rcu_nocb_mask);
 		} else {
-			pr_info("NOCB: Can't CB-offload an offline CPU\n");
+			pr_info("NOCB: Cannot CB-offload offline CPU %d\n", rdp->cpu);
 			ret = -EINVAL;
 		}
 	}
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#3 lore raw

PATCH 3/3 rcu: Simplify rcu_init_nohz() cpumask handling

In reply to Paul E. McKenney

From: Zhen Lei <[email protected]>

In kernels built with either CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y or CONFIG_NO_HZ_FULL=y, additional CPUs must be added to rcu_nocb_mask. Except that kernels booted without the rcu_nocbs= will not have allocated rcu_nocb_mask. And the current rcu_init_nohz() function uses its need_rcu_nocb_mask and offload_all local variables to track the rcu_nocb and nohz_full state.

But there is a much simpler approach, namely creating a cpumask pointer to track the default and then using cpumask_available() to check the rcu_nocb_mask state. This commit takes this approach, thereby simplifying and shortening the rcu_init_nohz() function.

---
 kernel/rcu/tree_nocb.h | 32 +++++++++-----------------------
 1 file changed, 9 insertions(+), 23 deletions(-)
kernel/rcu/tree_nocb.h +9 -23
index 0a5f0ef414845..c8167be2288fa 100644
@@ -1210,45 +1210,31 @@ EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload);
 void __init rcu_init_nohz(void)
 {
 	int cpu;
-	bool need_rcu_nocb_mask = false;
-	bool offload_all = false;
 	struct rcu_data *rdp;
+	const struct cpumask *cpumask = NULL;
 
 #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL)
-	if (!rcu_state.nocb_is_setup) {
-		need_rcu_nocb_mask = true;
-		offload_all = true;
-	}
-#endif /* #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL) */
-
-#if defined(CONFIG_NO_HZ_FULL)
-	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask)) {
-		need_rcu_nocb_mask = true;
-		offload_all = false; /* NO_HZ_FULL has its own mask. */
-	}
-#endif /* #if defined(CONFIG_NO_HZ_FULL) */
+	cpumask = cpu_possible_mask;
+#elif defined(CONFIG_NO_HZ_FULL)
+	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask))
+		cpumask = tick_nohz_full_mask;
+#endif
 
-	if (need_rcu_nocb_mask) {
+	if (cpumask) {
 		if (!cpumask_available(rcu_nocb_mask)) {
 			if (!zalloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL)) {
 				pr_info("rcu_nocb_mask allocation failed, callback offloading disabled.\n");
 				return;
 			}
 		}
+
+		cpumask_or(rcu_nocb_mask, rcu_nocb_mask, cpumask);
 		rcu_state.nocb_is_setup = true;
 	}
 
 	if (!rcu_state.nocb_is_setup)
 		return;
 
-#if defined(CONFIG_NO_HZ_FULL)
-	if (tick_nohz_full_running)
-		cpumask_or(rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask);
-#endif /* #if defined(CONFIG_NO_HZ_FULL) */
-
-	if (offload_all)
-		cpumask_setall(rcu_nocb_mask);
-
 	if (!cpumask_subset(rcu_nocb_mask, cpu_possible_mask)) {
 		pr_info("\tNote: kernel parameter 'rcu_nocbs=', 'nohz_full', or 'isolcpus=' contains nonexistent CPUs.\n");
 		cpumask_and(rcu_nocb_mask, cpu_possible_mask,
Signature · 1 line
2.31.1.189.g2e36527f23
#4 lore raw

Re: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling

Quoted 38 lines · Paul E. McKenney

From: Zhen Lei <[email protected]>

In kernels built with either CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y or CONFIG_NO_HZ_FULL=y, additional CPUs must be added to rcu_nocb_mask. Except that kernels booted without the rcu_nocbs= will not have allocated rcu_nocb_mask. And the current rcu_init_nohz() function uses its need_rcu_nocb_mask and offload_all local variables to track the rcu_nocb and nohz_full state.

But there is a much simpler approach, namely creating a cpumask pointer to track the default and then using cpumask_available() to check the rcu_nocb_mask state. This commit takes this approach, thereby simplifying and shortening the rcu_init_nohz() function.

---
 kernel/rcu/tree_nocb.h | 32 +++++++++-----------------------
 1 file changed, 9 insertions(+), 23 deletions(-)
kernel/rcu/tree_nocb.h +1 -5
index 0a5f0ef414845..c8167be2288fa 100644
@@ -1210,45 +1210,31 @@ EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload);
 void __init rcu_init_nohz(void)
 {
 	int cpu;
-	bool need_rcu_nocb_mask = false;
-	bool offload_all = false;
 	struct rcu_data *rdp;
+	const struct cpumask *cpumask = NULL;
 
 #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL)
-	if (!rcu_state.nocb_is_setup) {
-		need_rcu_nocb_mask = true;
-		offload_all = true;

kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1217

Quoted 10 lines
kernel/rcu/tree_nocb.h +1 -9
-	}
-#endif /* #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL) */
-
-#if defined(CONFIG_NO_HZ_FULL)
-	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask)) {
-		need_rcu_nocb_mask = true;
-		offload_all = false; /* NO_HZ_FULL has its own mask. */
-	}
-#endif /* #if defined(CONFIG_NO_HZ_FULL) */
+	cpumask = cpu_possible_mask;

You're missing the rcu_state.nocb_is_setup check, so CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now always override the rcu_nocbs= parameter (should be the other way around).

kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221

Quoted 4 lines
kernel/rcu/tree_nocb.h +4 -0
+#elif defined(CONFIG_NO_HZ_FULL)
+	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask))
+		cpumask = tick_nohz_full_mask;
+#endif

A subtle behaviour difference here too: CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now override nohz_full=

I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.

Thanks.

Quoted 31 lines
kernel/rcu/tree_nocb.h +3 -9
-	if (need_rcu_nocb_mask) {
+	if (cpumask) {
 		if (!cpumask_available(rcu_nocb_mask)) {
 			if (!zalloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL)) {
 				pr_info("rcu_nocb_mask allocation failed, callback offloading disabled.\n");
 				return;
 			}
 		}
+
+		cpumask_or(rcu_nocb_mask, rcu_nocb_mask, cpumask);
 		rcu_state.nocb_is_setup = true;
 	}
 
 	if (!rcu_state.nocb_is_setup)
 		return;
 
-#if defined(CONFIG_NO_HZ_FULL)
-	if (tick_nohz_full_running)
-		cpumask_or(rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask);
-#endif /* #if defined(CONFIG_NO_HZ_FULL) */
-
-	if (offload_all)
-		cpumask_setall(rcu_nocb_mask);
-
 	if (!cpumask_subset(rcu_nocb_mask, cpu_possible_mask)) {
 		pr_info("\tNote: kernel parameter 'rcu_nocbs=', 'nohz_full', or 'isolcpus=' contains nonexistent CPUs.\n");
 		cpumask_and(rcu_nocb_mask, cpu_possible_mask,
Signature · 1 line
2.31.1.189.g2e36527f23
author maintainer
#5 lore raw

Re: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling

Quoted 45 lines · Frederic Weisbecker
Quoted 44 lines · Paul E. McKenney

From: Zhen Lei <[email protected]>

In kernels built with either CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y or CONFIG_NO_HZ_FULL=y, additional CPUs must be added to rcu_nocb_mask. Except that kernels booted without the rcu_nocbs= will not have allocated rcu_nocb_mask. And the current rcu_init_nohz() function uses its need_rcu_nocb_mask and offload_all local variables to track the rcu_nocb and nohz_full state.

But there is a much simpler approach, namely creating a cpumask pointer to track the default and then using cpumask_available() to check the rcu_nocb_mask state. This commit takes this approach, thereby simplifying and shortening the rcu_init_nohz() function.

---
 kernel/rcu/tree_nocb.h | 32 +++++++++-----------------------
 1 file changed, 9 insertions(+), 23 deletions(-)
kernel/rcu/tree_nocb.h +1 -11
index 0a5f0ef414845..c8167be2288fa 100644
@@ -1210,45 +1210,31 @@ EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload);
 void __init rcu_init_nohz(void)
 {
 	int cpu;
-	bool need_rcu_nocb_mask = false;
-	bool offload_all = false;
 	struct rcu_data *rdp;
+	const struct cpumask *cpumask = NULL;
 
 #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL)
-	if (!rcu_state.nocb_is_setup) {
-		need_rcu_nocb_mask = true;
-		offload_all = true;
-	}
-#endif /* #if defined(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL) */
-
-#if defined(CONFIG_NO_HZ_FULL)
-	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask)) {
-		need_rcu_nocb_mask = true;
Quoted 8 lines

kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1217

Quoted 4 lines
kernel/rcu/tree_nocb.h +1 -3
-		offload_all = false; /* NO_HZ_FULL has its own mask. */
-	}
-#endif /* #if defined(CONFIG_NO_HZ_FULL) */
+	cpumask = cpu_possible_mask;

You're missing the rcu_state.nocb_is_setup check, so CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now always override the rcu_nocbs= parameter (should be the other way around).

This is indeed a problem, so especially thank you for looking this over!

I will revert this patch.

Quoted 10 lines

kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221

Quoted 4 lines
kernel/rcu/tree_nocb.h +4 -0
+#elif defined(CONFIG_NO_HZ_FULL)
+	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask))
+		cpumask = tick_nohz_full_mask;
+#endif

A subtle behaviour difference here too: CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now override nohz_full=

I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.

Good point. Perhaps the key point is that if there is nohz_full=, rcu_nocbs=, and CONFIG_RCU_NOCB_CPU_DEFAULT_ALL, we still need rcu_nocbs= to include at least those bits set by nohz_full=.

Are there any other constraints?

Zhen Lei, please feel free to submit an updated patch.

Thanx, Paul

Quoted 33 lines

Thanks.

Quoted 31 lines
kernel/rcu/tree_nocb.h +3 -9
-	if (need_rcu_nocb_mask) {
+	if (cpumask) {
 		if (!cpumask_available(rcu_nocb_mask)) {
 			if (!zalloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL)) {
 				pr_info("rcu_nocb_mask allocation failed, callback offloading disabled.\n");
 				return;
 			}
 		}
+
+		cpumask_or(rcu_nocb_mask, rcu_nocb_mask, cpumask);
 		rcu_state.nocb_is_setup = true;
 	}
 
 	if (!rcu_state.nocb_is_setup)
 		return;
 
-#if defined(CONFIG_NO_HZ_FULL)
-	if (tick_nohz_full_running)
-		cpumask_or(rcu_nocb_mask, rcu_nocb_mask, tick_nohz_full_mask);
-#endif /* #if defined(CONFIG_NO_HZ_FULL) */
-
-	if (offload_all)
-		cpumask_setall(rcu_nocb_mask);
-
 	if (!cpumask_subset(rcu_nocb_mask, cpu_possible_mask)) {
 		pr_info("\tNote: kernel parameter 'rcu_nocbs=', 'nohz_full', or 'isolcpus=' contains nonexistent CPUs.\n");
 		cpumask_and(rcu_nocb_mask, cpu_possible_mask,
Signature · 1 line
2.31.1.189.g2e36527f23
#6 lore raw

Re: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling

Quoted 9 lines · Paul E. McKenney
Quoted 8 lines · Frederic Weisbecker

kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221

Quoted 4 lines
kernel/rcu/tree_nocb.h +4 -0
+#elif defined(CONFIG_NO_HZ_FULL)
+	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask))
+		cpumask = tick_nohz_full_mask;
+#endif

A subtle behaviour difference here too: CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now override nohz_full=

Quoted 6 lines
Quoted 2 lines

I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.

Good point. Perhaps the key point is that if there is nohz_full=, rcu_nocbs=, and CONFIG_RCU_NOCB_CPU_DEFAULT_ALL, we still need rcu_nocbs= to include at least those bits set by nohz_full=.

Not sure I get what you mean. nohz_full= should in any case always force rcu_nocbs at least on the nohz_full CPUs.

For example assuming the following combination: rcu_nocbs=6, nohz_full=7 AND CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result should be:

NOCB CPUs = 6,7
NOHZ_FULL CPUs = 7

(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y is overriden by rcu_nocbs=6).

Now if we have nohz_full=7 AND CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result is expected to be either:

NOCB CPUs = 7 (upstream behaviour)
NOHZ_FULL CPUs = 7

or

NOCB CPUs = all
NOHZ_FULL CPUs = 7

The second makes more sense IMHO but that should be in a separate change.

Thanks.

#7 lore raw

Re: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling

Quoted 26 lines · Frederic Weisbecker
Quoted 9 lines · Paul E. McKenney
Quoted 8 lines · Frederic Weisbecker

kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221

Quoted 4 lines
kernel/rcu/tree_nocb.h +4 -0
+#elif defined(CONFIG_NO_HZ_FULL)
+	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask))
+		cpumask = tick_nohz_full_mask;
+#endif

A subtle behaviour difference here too: CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now override nohz_full=

Quoted 6 lines
Quoted 2 lines

I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.

Good point. Perhaps the key point is that if there is nohz_full=, rcu_nocbs=, and CONFIG_RCU_NOCB_CPU_DEFAULT_ALL, we still need rcu_nocbs= to include at least those bits set by nohz_full=.

Not sure I get what you mean. nohz_full= should in any case always force rcu_nocbs at least on the nohz_full CPUs.

For example assuming the following combination: rcu_nocbs=6, nohz_full=7 AND CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result should be:

NOCB CPUs = 6,7
NOHZ_FULL CPUs = 7

Quoted 7 lines

(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y is overriden by rcu_nocbs=6).

Now if we have nohz_full=7 AND CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result is expected to be either:

NOCB CPUs = 7 (upstream behaviour)
NOHZ_FULL CPUs = 7

OK, I got it. CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y takes effect only when there are no 'rcu_nocbs=' and 'nohz_full='.

Quoted 7 lines

or

NOCB CPUs = all
NOHZ_FULL CPUs = 7

The second makes more sense IMHO but that should be in a separate change.

OK, in this case, CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y takes effect only when there is no 'rcu_nocbs='.

Quoted 4 lines

Thanks.
.

Signature · 2 lines
Regards,
  Zhen Lei
author maintainer
#8 lore raw

Re: [PATCH rcu 3/3] rcu: Simplify rcu_init_nohz() cpumask handling

In reply to Frederic Weisbecker
Quoted 34 lines · Frederic Weisbecker
Quoted 9 lines · Paul E. McKenney
Quoted 8 lines · Frederic Weisbecker

kernel/rcu/tree_nocb.h EXPORT_SYMBOL_GPL(rcu_nocb_cpu_offload); · line 1221

Quoted 4 lines
kernel/rcu/tree_nocb.h +4 -0
+#elif defined(CONFIG_NO_HZ_FULL)
+	if (tick_nohz_full_running && !cpumask_empty(tick_nohz_full_mask))
+		cpumask = tick_nohz_full_mask;
+#endif

A subtle behaviour difference here too: CONFIG_RCU_NOCB_CPU_DEFAULT_ALL will now override nohz_full=

Quoted 6 lines
Quoted 2 lines

I don't mind, it's probably what we want in the end, but the changelog should tell about it, or even better, this should be a separate change.

Good point. Perhaps the key point is that if there is nohz_full=, rcu_nocbs=, and CONFIG_RCU_NOCB_CPU_DEFAULT_ALL, we still need rcu_nocbs= to include at least those bits set by nohz_full=.

Not sure I get what you mean. nohz_full= should in any case always force rcu_nocbs at least on the nohz_full CPUs.

For example assuming the following combination: rcu_nocbs=6, nohz_full=7 AND CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result should be:

NOCB CPUs = 6,7
NOHZ_FULL CPUs = 7

(CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y is overriden by rcu_nocbs=6).

Now if we have nohz_full=7 AND CONFIG_RCU_NOCB_CPU_DEFAULT_ALL=y, then the result is expected to be either:

NOCB CPUs = 7 (upstream behaviour)
NOHZ_FULL CPUs = 7

Quoted 6 lines

or

NOCB CPUs = all
NOHZ_FULL CPUs = 7

The second makes more sense IMHO but that should be in a separate change.

Your examples are consistent with what I was trying to say. ;-)

Thanx, Paul