rcu: Allow user to ovveride RCU_NOCB_CPU_ALL at boot time
RFC PATCH 1/1 rcu: Allow user to ovveride RCU_NOCB_CPU_ALL at boot time
A kernel built with RCU_NOCB_CPU_ALL build time option will offload callbacks from all CPUs. The user cannot override this behavior without recompiling the kernel with the RCU_NOCB_CPU_ALL option turned off.
This commit allows the user to override the build-time option by using the rcu_nocbs= boot time option without needing to recompile the kernel.
Please note that this is how NO_HZ_FULL_ALL build time option works and this commit makes it work similar to that.
--- init/Kconfig | 14 +++++++------- kernel/rcu/tree_plugin.h | 8 ++++---- 2 files changed, 11 insertions(+), 11 deletions(-)
init/Kconfig +7 -7
@@ -768,13 +768,13 @@ config RCU_NOCB_CPU_ALL bool "All CPUs are build_forced no-CBs CPUs" depends on RCU_NOCB_CPU help - This option forces all CPUs to be no-CBs CPUs. The rcu_nocbs= - boot parameter will be ignored. All CPUs' RCU callbacks will - be executed in the context of per-CPU rcuo kthreads created for - this purpose. Assuming that the kthreads whose names start with - "rcuo" are bound to "housekeeping" CPUs, this reduces OS jitter - on the remaining CPUs, but might decrease memory locality during - RCU-callback invocation, thus potentially degrading throughput. + If the user doesn't pass the rcu_nocbs= boot option, force all CPUs + to be no-CBs CPUs. All CPUs' RCU callbacks will be executed in the + context of per-CPU rcuo kthreads created for this purpose. Assuming + that the kthreads whose names start with "rcuo" are bound to + "housekeeping" CPUs, this reduces OS jitter on the remaining CPUs, but + might decrease memory locality during RCU-callback invocation, thus + potentially degrading throughput. Select this if all CPUs need to be no-CBs CPUs for real-time or energy-efficiency reasons.
kernel/rcu/tree_plugin.h +4 -4
@@ -92,15 +92,15 @@ static void __init rcu_bootup_announce_oddness(void)if (!have_rcu_nocb_mask) {zalloc_cpumask_var (&rcu_nocb_mask,GFP_KERNEL ); have_rcu_nocb_mask = true; +#ifdef CONFIG_RCU_NOCB_CPU_ALL +pr_info ("\tOffload RCU callbacks from all CPUs\n" ); +cpumask_copy (rcu_nocb_mask, cpu_possible_mask); +#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */ }#ifdef CONFIG_RCU_NOCB_CPU_ZERO pr_info ("\tOffload RCU callbacks from CPU 0\n" );cpumask_set_cpu (0 , rcu_nocb_mask);#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ZERO */ -#ifdef CONFIG_RCU_NOCB_CPU_ALL -pr_info ("\tOffload RCU callbacks from all CPUs\n" ); -cpumask_copy (rcu_nocb_mask, cpu_possible_mask); -#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */ #endif /* #ifndef CONFIG_RCU_NOCB_CPU_NONE */ if (have_rcu_nocb_mask) {if (!cpumask_subset (rcu_nocb_mask, cpu_possible_mask)) {
Signature · 1 line
1.9.1
Quoted 11 lines · Pranith Kumar
A kernel built with
RCU_NOCB_CPU_ALLbuild time option will offload callbacks from all CPUs. The user cannot override this behavior without recompiling the kernel with theRCU_NOCB_CPU_ALLoption turned off.This commit allows the user to override the build-time option by using the rcu_nocbs= boot time option without needing to recompile the kernel.
Please note that this is how
NO_HZ_FULL_ALLbuild time option works and this commit makes it work similar to that.Signed-off-by Pranith Kumar <[email protected]>
I cannot accept this patch. For one thing, tick_nohz_init_all() looks a bit on the unconditional side when CONFIG_NO_HZ_FULL_ALL=y. For another thing, we really do not want to be handing the user a tool that allows CPUs that are nohz_full to not be no-CBs CPUs. For another thing, if we add this and it turns out to be a bad idea, it will be difficult to take it back -- someone somewhere will no doubt have scripted the boot parameter.
Thanx, Paul
Quoted 57 lines
--- init/Kconfig | 14 +++++++------- kernel/rcu/tree_plugin.h | 8 ++++---- 2 files changed, 11 insertions(+), 11 deletions(-)init/Kconfig +7
-7@@ -768,13 +768,13 @@ config RCU_NOCB_CPU_ALL bool "All CPUs are build_forced no-CBs CPUs" depends on RCU_NOCB_CPU help - This option forces all CPUs to be no-CBs CPUs. The rcu_nocbs= - boot parameter will be ignored. All CPUs' RCU callbacks will - be executed in the context of per-CPU rcuo kthreads created for - this purpose. Assuming that the kthreads whose names start with - "rcuo" are bound to "housekeeping" CPUs, this reduces OS jitter - on the remaining CPUs, but might decrease memory locality during - RCU-callback invocation, thus potentially degrading throughput. + If the user doesn't pass the rcu_nocbs= boot option, force all CPUs + to be no-CBs CPUs. All CPUs' RCU callbacks will be executed in the + context of per-CPU rcuo kthreads created for this purpose. Assuming + that the kthreads whose names start with "rcuo" are bound to + "housekeeping" CPUs, this reduces OS jitter on the remaining CPUs, but + might decrease memory locality during RCU-callback invocation, thus + potentially degrading throughput. Select this if all CPUs need to be no-CBs CPUs for real-time or energy-efficiency reasons.kernel/rcu/tree_plugin.h +4
-4@@ -92,15 +92,15 @@ static void __init rcu_bootup_announce_oddness(void)if (!have_rcu_nocb_mask) {zalloc_cpumask_var (&rcu_nocb_mask,GFP_KERNEL ); have_rcu_nocb_mask = true; +#ifdef CONFIG_RCU_NOCB_CPU_ALL +pr_info ("\tOffload RCU callbacks from all CPUs\n" ); +cpumask_copy (rcu_nocb_mask, cpu_possible_mask); +#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */ }#ifdef CONFIG_RCU_NOCB_CPU_ZERO pr_info ("\tOffload RCU callbacks from CPU 0\n" );cpumask_set_cpu (0 , rcu_nocb_mask);#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ZERO */ -#ifdef CONFIG_RCU_NOCB_CPU_ALL -pr_info ("\tOffload RCU callbacks from all CPUs\n" ); -cpumask_copy (rcu_nocb_mask, cpu_possible_mask); -#endif /* #ifdef CONFIG_RCU_NOCB_CPU_ALL */ #endif /* #ifndef CONFIG_RCU_NOCB_CPU_NONE */ if (have_rcu_nocb_mask) {if (!cpumask_subset (rcu_nocb_mask, cpu_possible_mask)) {Signature · 1 line
1.9.1
Quoted 5 lines · Paul E. McKenney
Quoted 4 lines · Pranith Kumar
A kernel built with
RCU_NOCB_CPU_ALLbuild time option will offload callbacks from all CPUs. The user cannot override this behavior without recompiling the kernel with theRCU_NOCB_CPU_ALLoption turned off.
Quoted 7 lines
Quoted 7 lines
This commit allows the user to override the build-time option by using the rcu_nocbs= boot time option without needing to recompile the kernel.
Please note that this is how
NO_HZ_FULL_ALLbuild time option works and this commit makes it work similar to that.Signed-off-by Pranith Kumar <[email protected]>
Hi Paul,
Quoted 4 lines
I cannot accept this patch. For one thing,
tick_nohz_init_all()looks a bit on the unconditional side whenCONFIG_NO_HZ_FULL_ALL=y. For another thing, we really do not want to be handing the user a tool that allows CPUs that arenohz_fullto not be no-CBs CPUs. For another thing,
I thought the latest patch does not allow that by ORing the nohz_full and rcu_nocbs mask. Doesn't it? All nohz_full CPUs will be nocb CPUS. We can mention this explicitly in the kernel-parameters.txt file.
Quoted 3 lines
if we add this and it turns out to be a bad idea, it will be difficult to take it back -- someone somewhere will no doubt have scripted the boot parameter.
This option already exists in the kernel when RCU_NOCB_CPU_ALL is not set. The user can pass in rcu_nocbs= at boot time. I am not sure we are adding anything new with this patch.
Most of the distro kernels set RCU_NOCB_CPU_ALL and rightly so, as it is the suggested and most appropriate option. This patch will make it easier for users who want to specify nocb CPUs for their needs, without having to recompile the kernel.
Quoted 4 lines
Thanx, Paul
--
Pranith