Skip to contentSkip to conversation

rcu: Allow user to ovveride RCU_NOCB_CPU_ALL at boot time

RFC PATCH In review Pranith Kumar 2 participants 3 messages Jul 16, 2014 – Jul 17, 2014 RCU

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 rcu: Allow user to ovveride RCU_NOCB_CPU_ALL at boot time

A kernel built with RCU_NOCB_CPU_ALL build time option will offload callbacks from all CPUs. The user cannot override this behavior without recompiling the kernel with the RCU_NOCB_CPU_ALL option turned off.

This commit allows the user to override the build-time option by using the rcu_nocbs= boot time option without needing to recompile the kernel.

Please note that this is how NO_HZ_FULL_ALL build time option works and this commit makes it work similar to that.

---
 init/Kconfig             | 14 +++++++-------
 kernel/rcu/tree_plugin.h |  8 ++++----
 2 files changed, 11 insertions(+), 11 deletions(-)
init/Kconfig +7 -7
index 41066e4..7d363a4 100644
@@ -768,13 +768,13 @@ config RCU_NOCB_CPU_ALL
 	bool "All CPUs are build_forced no-CBs CPUs"
 	depends on RCU_NOCB_CPU
 	help
-	  This option forces all CPUs to be no-CBs CPUs.  The rcu_nocbs=
-	  boot parameter will be ignored.  All CPUs' RCU callbacks will
-	  be executed in the context of per-CPU rcuo kthreads created for
-	  this purpose.  Assuming that the kthreads whose names start with
-	  "rcuo" are bound to "housekeeping" CPUs, this reduces OS jitter
-	  on the remaining CPUs, but might decrease memory locality during
-	  RCU-callback invocation, thus potentially degrading throughput.
+	  If the user doesn't pass the rcu_nocbs= boot option, force all CPUs
+	  to be no-CBs CPUs.  All CPUs' RCU callbacks will be executed in the
+	  context of per-CPU rcuo kthreads created for this purpose. Assuming
+	  that the kthreads whose names start with "rcuo" are bound to
+	  "housekeeping" CPUs, this reduces OS jitter on the remaining CPUs, but
+	  might decrease memory locality during RCU-callback invocation, thus
+	  potentially degrading throughput.
 
 	  Select this if all CPUs need to be no-CBs CPUs for real-time
 	  or energy-efficiency reasons.
kernel/rcu/tree_plugin.h +4 -4
index a2113f6..b97a939 100644
@@ -92,15 +92,15 @@ static void __init rcu_bootup_announce_oddness(void)
 	if (!have_rcu_nocb_mask) {
 		zalloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL);
 		have_rcu_nocb_mask = true;
+#ifdef CONFIG_RCU_NOCB_CPU_ALL
+		pr_info("\tOffload RCU callbacks from all CPUs\n");
+		cpumask_copy(rcu_nocb_mask, cpu_possible_mask);
+#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */
 	}
 #ifdef CONFIG_RCU_NOCB_CPU_ZERO
 	pr_info("\tOffload RCU callbacks from CPU 0\n");
 	cpumask_set_cpu(0, rcu_nocb_mask);
 #endif /* #ifdef CONFIG_RCU_NOCB_CPU_ZERO */
-#ifdef CONFIG_RCU_NOCB_CPU_ALL
-	pr_info("\tOffload RCU callbacks from all CPUs\n");
-	cpumask_copy(rcu_nocb_mask, cpu_possible_mask);
-#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */
 #endif /* #ifndef CONFIG_RCU_NOCB_CPU_NONE */
 	if (have_rcu_nocb_mask) {
 		if (!cpumask_subset(rcu_nocb_mask, cpu_possible_mask)) {
Signature · 1 line
1.9.1
#1 lore raw
Quoted 11 lines · Pranith Kumar

A kernel built with RCU_NOCB_CPU_ALL build time option will offload callbacks from all CPUs. The user cannot override this behavior without recompiling the kernel with the RCU_NOCB_CPU_ALL option turned off.

This commit allows the user to override the build-time option by using the rcu_nocbs= boot time option without needing to recompile the kernel.

Please note that this is how NO_HZ_FULL_ALL build time option works and this commit makes it work similar to that.

I cannot accept this patch. For one thing, tick_nohz_init_all() looks a bit on the unconditional side when CONFIG_NO_HZ_FULL_ALL=y. For another thing, we really do not want to be handing the user a tool that allows CPUs that are nohz_full to not be no-CBs CPUs. For another thing, if we add this and it turns out to be a bad idea, it will be difficult to take it back -- someone somewhere will no doubt have scripted the boot parameter.

Thanx, Paul

Quoted 57 lines
---
 init/Kconfig             | 14 +++++++-------
 kernel/rcu/tree_plugin.h |  8 ++++----
 2 files changed, 11 insertions(+), 11 deletions(-)
init/Kconfig +7 -7
index 41066e4..7d363a4 100644
@@ -768,13 +768,13 @@ config RCU_NOCB_CPU_ALL
 	bool "All CPUs are build_forced no-CBs CPUs"
 	depends on RCU_NOCB_CPU
 	help
-	  This option forces all CPUs to be no-CBs CPUs.  The rcu_nocbs=
-	  boot parameter will be ignored.  All CPUs' RCU callbacks will
-	  be executed in the context of per-CPU rcuo kthreads created for
-	  this purpose.  Assuming that the kthreads whose names start with
-	  "rcuo" are bound to "housekeeping" CPUs, this reduces OS jitter
-	  on the remaining CPUs, but might decrease memory locality during
-	  RCU-callback invocation, thus potentially degrading throughput.
+	  If the user doesn't pass the rcu_nocbs= boot option, force all CPUs
+	  to be no-CBs CPUs.  All CPUs' RCU callbacks will be executed in the
+	  context of per-CPU rcuo kthreads created for this purpose. Assuming
+	  that the kthreads whose names start with "rcuo" are bound to
+	  "housekeeping" CPUs, this reduces OS jitter on the remaining CPUs, but
+	  might decrease memory locality during RCU-callback invocation, thus
+	  potentially degrading throughput.
 
 	  Select this if all CPUs need to be no-CBs CPUs for real-time
 	  or energy-efficiency reasons.
kernel/rcu/tree_plugin.h +4 -4
index a2113f6..b97a939 100644
@@ -92,15 +92,15 @@ static void __init rcu_bootup_announce_oddness(void)
 	if (!have_rcu_nocb_mask) {
 		zalloc_cpumask_var(&rcu_nocb_mask, GFP_KERNEL);
 		have_rcu_nocb_mask = true;
+#ifdef CONFIG_RCU_NOCB_CPU_ALL
+		pr_info("\tOffload RCU callbacks from all CPUs\n");
+		cpumask_copy(rcu_nocb_mask, cpu_possible_mask);
+#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */
 	}
 #ifdef CONFIG_RCU_NOCB_CPU_ZERO
 	pr_info("\tOffload RCU callbacks from CPU 0\n");
 	cpumask_set_cpu(0, rcu_nocb_mask);
 #endif /* #ifdef CONFIG_RCU_NOCB_CPU_ZERO */
-#ifdef CONFIG_RCU_NOCB_CPU_ALL
-	pr_info("\tOffload RCU callbacks from all CPUs\n");
-	cpumask_copy(rcu_nocb_mask, cpu_possible_mask);
-#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */
 #endif /* #ifndef CONFIG_RCU_NOCB_CPU_NONE */
 	if (have_rcu_nocb_mask) {
 		if (!cpumask_subset(rcu_nocb_mask, cpu_possible_mask)) {
Signature · 1 line
1.9.1
#2 lore raw
Quoted 5 lines · Paul E. McKenney
Quoted 4 lines · Pranith Kumar

A kernel built with RCU_NOCB_CPU_ALL build time option will offload callbacks from all CPUs. The user cannot override this behavior without recompiling the kernel with the RCU_NOCB_CPU_ALL option turned off.

Quoted 7 lines
Quoted 7 lines

This commit allows the user to override the build-time option by using the rcu_nocbs= boot time option without needing to recompile the kernel.

Please note that this is how NO_HZ_FULL_ALL build time option works and this commit makes it work similar to that.

Hi Paul,

Quoted 4 lines

I cannot accept this patch. For one thing, tick_nohz_init_all() looks a bit on the unconditional side when CONFIG_NO_HZ_FULL_ALL=y. For another thing, we really do not want to be handing the user a tool that allows CPUs that are nohz_full to not be no-CBs CPUs. For another thing,

I thought the latest patch does not allow that by ORing the nohz_full and rcu_nocbs mask. Doesn't it? All nohz_full CPUs will be nocb CPUS. We can mention this explicitly in the kernel-parameters.txt file.

Quoted 3 lines

if we add this and it turns out to be a bad idea, it will be difficult to take it back -- someone somewhere will no doubt have scripted the boot parameter.

This option already exists in the kernel when RCU_NOCB_CPU_ALL is not set. The user can pass in rcu_nocbs= at boot time. I am not sure we are adding anything new with this patch.

Most of the distro kernels set RCU_NOCB_CPU_ALL and rightly so, as it is the suggested and most appropriate option. This patch will make it easier for users who want to specify nocb CPUs for their needs, without having to recompile the kernel.

Quoted 4 lines

Thanx, Paul

--
Pranith