rcu: Make rcu_normal_after_boot writable on RT
PATCH 1/1 rcu: Make rcu_normal_after_boot writable on RT
Certain configurations (e.g., systems that make heavy use of netns) need to use synchronize_rcu_expedited() to service RCU grace periods even after boot.
Even though synchronize_rcu_expedited() has been traditionally considered harmful for RT for the heavy use of IPIs, it is perfectly usable under certain conditions (e.g. nohz_full).
Make rcupdate.rcu_normal_after_boot= again writeable on RT, but keep its default value to 1 (enabled) to avoid regressions. Users who need synchronize_rcu_expedited() will boot with rcupdate.rcu_normal_after_ boot=0 in the kernel cmdline.
--- kernel/rcu/update.c | 2 -- 1 file changed, 2 deletions(-)
kernel/rcu/update.c +0 -2
@@ -57,9 +57,7 @@module_param (rcu_expedited, int,0 );module_param (rcu_normal, int,0 );static int rcu_normal_after_boot =IS_ENABLED (CONFIG_PREEMPT_RT ); -#ifndef CONFIG_PREEMPT_RT module_param (rcu_normal_after_boot, int,0 ); -#endif #endif /* #ifndef CONFIG_TINY_RCU */ #ifdef CONFIG_DEBUG_LOCK_ALLOC
Signature · 1 line
2.31.1
Paul E. McKenney
maintainerQuoted 8 lines · Juri Lelli
Certain configurations (e.g., systems that make heavy use of netns) need to use
synchronize_rcu_expedited()to service RCU grace periods even after boot.Even though
synchronize_rcu_expedited()has been traditionally considered harmful for RT for the heavy use of IPIs, it is perfectly usable under certain conditions (e.g.nohz_full).
Quoted 6 lines
Make rcupdate.rcu_normal_after_boot= again writeable on RT, but keep its default value to 1 (enabled) to avoid regressions. Users who need
synchronize_rcu_expedited()will boot with rcupdate.rcu_normal_after_ boot=0 in the kernel cmdline.Signed-off-by Juri Lelli <[email protected]>
Makes sense to me!
But would another of the -rt people be willing to give an Acked-by? For example, maybe they would prefer this kernel boot parameter to be exposed only if (!PREEMPT_RT || NO_HZ_FULL). Or are there !NO_HZ_FULL situations where rcu_normal_after_boot makes sense?
Thanx, Paul
Quoted 21 lines
--- kernel/rcu/update.c | 2 -- 1 file changed, 2 deletions(-)kernel/rcu/update.c +0
-2@@ -57,9 +57,7 @@module_param (rcu_expedited, int,0 );module_param (rcu_normal, int,0 );static int rcu_normal_after_boot =IS_ENABLED (CONFIG_PREEMPT_RT ); -#ifndef CONFIG_PREEMPT_RT module_param (rcu_normal_after_boot, int,0 ); -#endif #endif /* #ifndef CONFIG_TINY_RCU */ #ifdef CONFIG_DEBUG_LOCK_ALLOC Signature · 1 line
2.31.1
Paul E. McKenney
maintainerQuoted 14 lines · Paul E. McKenney
Quoted 13 lines · Juri Lelli
Certain configurations (e.g., systems that make heavy use of netns) need to use
synchronize_rcu_expedited()to service RCU grace periods even after boot.Even though
synchronize_rcu_expedited()has been traditionally considered harmful for RT for the heavy use of IPIs, it is perfectly usable under certain conditions (e.g.nohz_full).Make rcupdate.rcu_normal_after_boot= again writeable on RT, but keep its default value to 1 (enabled) to avoid regressions. Users who need
synchronize_rcu_expedited()will boot with rcupdate.rcu_normal_after_ boot=0 in the kernel cmdline.
Quoted 8 lines
Quoted 1 line
Signed-off-by Juri Lelli <[email protected]>Makes sense to me!
But would another of the -rt people be willing to give an Acked-by? For example, maybe they would prefer this kernel boot parameter to be exposed only if (!PREEMPT_RT ||
NO_HZ_FULL). Or are there !NO_HZ_FULL situations wherercu_normal_after_bootmakes sense?
Ah, and this will also need to be reflected in the WARN_ON_ONCE() in synchronize_rcu_expedited_wait() in kernel/rcu/tree_exp.h.
Thanx, Paul
Quoted 21 lines
Quoted 21 lines
--- kernel/rcu/update.c | 2 -- 1 file changed, 2 deletions(-)kernel/rcu/update.c +0
-2@@ -57,9 +57,7 @@module_param (rcu_expedited, int,0 );module_param (rcu_normal, int,0 );static int rcu_normal_after_boot =IS_ENABLED (CONFIG_PREEMPT_RT ); -#ifndef CONFIG_PREEMPT_RT module_param (rcu_normal_after_boot, int,0 ); -#endif #endif /* #ifndef CONFIG_TINY_RCU */ #ifdef CONFIG_DEBUG_LOCK_ALLOC Signature · 1 line
2.31.1
Hi,
Thanks for reviewing Paul.
Quoted 19 lines · Paul E. McKenney
Quoted 18 lines · Paul E. McKenney
Quoted 8 lines · Juri Lelli
Certain configurations (e.g., systems that make heavy use of netns) need to use
synchronize_rcu_expedited()to service RCU grace periods even after boot.Even though
synchronize_rcu_expedited()has been traditionally considered harmful for RT for the heavy use of IPIs, it is perfectly usable under certain conditions (e.g.nohz_full).Quoted 6 lines
Make rcupdate.rcu_normal_after_boot= again writeable on RT, but keep its default value to 1 (enabled) to avoid regressions. Users who need
synchronize_rcu_expedited()will boot with rcupdate.rcu_normal_after_ boot=0 in the kernel cmdline.Signed-off-by Juri Lelli <[email protected]>Makes sense to me!
Quoted 7 lines
Quoted 4 lines
But would another of the -rt people be willing to give an Acked-by? For example, maybe they would prefer this kernel boot parameter to be exposed only if (!PREEMPT_RT ||
NO_HZ_FULL). Or are there !NO_HZ_FULL situations wherercu_normal_after_bootmakes sense?Ah, and this will also need to be reflected in the
WARN_ON_ONCE()insynchronize_rcu_expedited_wait()inkernel/rcu/tree_exp.h.
Indeed. Will add the change as soon as I receive indication about your first point.
Best,
Juri
Quoted 6 lines · Paul E. McKenney
Makes sense to me!
But would another of the -rt people be willing to give an Acked-by? For example, maybe they would prefer this kernel boot parameter to be exposed only if (!PREEMPT_RT ||
NO_HZ_FULL). Or are there !NO_HZ_FULL situations wherercu_normal_after_bootmakes sense?
Julia crafted that "rcu_normal_after_boot = 1" for RT after we had more and more synchronize_rcu_expedited() users popping up. I would like to keep that part (default value) since it good to have for most users.
I don't mind removing CONFIG_PREEMPT_RT part here if there are legitimate use cases for using "rcu_normal_after_boot = 0".
Paul suggested initially to restrict that option for PREEMPT_RT and I would follow here Paul's guidance to either remove it or restrict it to NO_HZ_FULL in RT's case (as suggested).
Quoted 1 line
Thanx, Paul
Sebastian
Paul E. McKenney
maintainerQuoted 12 lines · Sebastian Andrzej Siewior
Quoted 6 lines · Paul E. McKenney
Makes sense to me!
But would another of the -rt people be willing to give an Acked-by? For example, maybe they would prefer this kernel boot parameter to be exposed only if (!PREEMPT_RT ||
NO_HZ_FULL). Or are there !NO_HZ_FULL situations wherercu_normal_after_bootmakes sense?Julia crafted that "
rcu_normal_after_boot= 1" for RT after we had more and moresynchronize_rcu_expedited()users popping up. I would like to keep that part (default value) since it good to have for most users.
Quoted 5 lines
I don't mind removing
CONFIG_PREEMPT_RTpart here if there are legitimate use cases for using "rcu_normal_after_boot= 0".
Paul suggested initially to restrict that option forPREEMPT_RTand I would follow here Paul's guidance to either remove it or restrict it toNO_HZ_FULLin RT's case (as suggested).
Given what I know now, I suggest the following:
o Restrict the option to !PREEMPT_RT unless NO_HZ_FULL.
Maybe "!defined(CONFIG_PREEMPT_RT) || defined(CONFIG_NO_HZ_FULL)".
If there is some non-NO_HZ_FULL PREEMPT_RT configuration that tolerates expedited grace periods, this would need to change.
o Change the permissions from "0" to "0444", if desired. If you
would rather not, I can do this in a follow-up patch. (No idea why I let such an ugly serviceability issue through, but the previous pair of module_param() instances have the same problem.)
Anything I am missing?
Thanx, Paul
Quoted 27 lines · Paul E. McKenney
Quoted 12 lines · Sebastian Andrzej Siewior
Quoted 6 lines · Paul E. McKenney
Makes sense to me!
But would another of the -rt people be willing to give an Acked-by? For example, maybe they would prefer this kernel boot parameter to be exposed only if (!PREEMPT_RT ||
NO_HZ_FULL). Or are there !NO_HZ_FULL situations wherercu_normal_after_bootmakes sense?Julia crafted that "
rcu_normal_after_boot= 1" for RT after we had more and moresynchronize_rcu_expedited()users popping up. I would like to keep that part (default value) since it good to have for most users.Quoted 5 lines
I don't mind removing
CONFIG_PREEMPT_RTpart here if there are legitimate use cases for using "rcu_normal_after_boot= 0".
Paul suggested initially to restrict that option forPREEMPT_RTand I would follow here Paul's guidance to either remove it or restrict it toNO_HZ_FULLin RT's case (as suggested).Given what I know now, I suggest the following:
o Restrict the option to !PREEMPT_RT unless
NO_HZ_FULL.
Maybe "!defined(CONFIG_PREEMPT_RT) ||defined(CONFIG_NO_HZ_FULL)".If there is some non-NO_HZ_FULL
PREEMPT_RTconfiguration that tolerates expedited grace periods, this would need to change.
Quoted 6 lines
o Change the permissions from "0" to "0444", if desired. If you
would rather not, I can do this in a follow-up patch. (No idea why I let such an ugly serviceability issue through, but the previous pair ofmodule_param()instances have the same problem.)Anything I am missing?
Not that I can think of right now. :)
Will implement your suggestions and submit v2 soon. Thank again to you and Sebastian for the review!
Best,
Juri