rcu: synchronize_rcu[_expedited]() related fixes
Frederic Weisbecker
maintainerPATCH 1/3 rcu: Fix expedited GP polling against UP/no-preempt environment
In reply to Frederic Weisbeckersynchronize_rcu_expedited() has an early return condition: if the current CPU is the only one online and the kernel doesn't run in preemption mode, the current assumed quiescent state is worth a grace period.
However the expedited grace period polling caller of synchronize_rcu_expedited() takes a GP sequence snapshot and expects it to complete by the end of the synchronize_rcu_expedited() call. Yet if synchronize_rcu_expedited() relies on the above described UP/no-preempt early return, the grace period sequence won't move and may cause an expedited grace period polling stall.
Fix this with treating polling differently while calling synchronize_rcu_expedited() and ignore the UP-no-preempt optimization in this case.
--- kernel/rcu/tree_exp.h | 57 ++++++++++++++++++++++++++----------------- 1 file changed, 35 insertions(+), 22 deletions(-)
kernel/rcu/tree_exp.h +35 -22
@@ -794,27 +794,14 @@ static int rcu_print_task_exp_stall(struct rcu_node *rnp)#endif /* #else #ifdef CONFIG_PREEMPT_RCU */ -/** -* synchronize_rcu_expedited - Brute-force RCU grace period -* -* Wait for an RCU grace period, but expedite it. The basic idea is to -* IPI all non-idle non-nohz online CPUs. The IPI handler checks whether -* the CPU is in an RCU critical section, and if so, it sets a flag that -* causes the outermost rcu_read_unlock() to report the quiescent state -* for RCU-preempt or asks the scheduler for help for RCU-sched. On the -* other hand, if the CPU is not in an RCU read-side critical section, -* the IPI handler reports the quiescent state immediately. -* -* Although this is a great improvement over previous expedited -* implementations, it is still unfriendly to real-time workloads, so is -* thus not recommended for any sort of common-case code. In fact, if -* you are using synchronize_rcu_expedited() in a loop, please restructure -* your code to batch your updates, and then use a single synchronize_rcu() -* instead. -* -* This has the same semantics as (but is more brutal than) synchronize_rcu(). +/* +* Start and wait for an expedited grace period completion. +* If it happens to be called by polling functions (@polling = true), +* there is no possible early return in UP no-preempt mode because +* the callers are waiting for an actual given sequence snapshot to start +* and end. */ -void synchronize_rcu_expedited (void ) +static void __synchronize_rcu_expedited (bool polling) {bool boottime = (rcu_scheduler_active ==RCU_SCHEDULER_INIT );struct rcu_exp_work rew;
@@ -827,7 +814,7 @@ void synchronize_rcu_expedited(void) "Illegal synchronize_rcu_expedited() in RCU read-side critical section");/* Is the state is such that the call is a grace period? */ -if (rcu_blocking_is_gp ()) +if (rcu_blocking_is_gp () && !polling)return ;/* If expedited grace periods are prohibited, fall back to normal. */
@@ -863,6 +850,32 @@ void synchronize_rcu_expedited(void)if (likely (!boottime))destroy_work_on_stack (&rew.rew_work); + +} + +/** +* synchronize_rcu_expedited - Brute-force RCU grace period +* +* Wait for an RCU grace period, but expedite it. The basic idea is to +* IPI all non-idle non-nohz online CPUs. The IPI handler checks whether +* the CPU is in an RCU critical section, and if so, it sets a flag that +* causes the outermost rcu_read_unlock() to report the quiescent state +* for RCU-preempt or asks the scheduler for help for RCU-sched. On the +* other hand, if the CPU is not in an RCU read-side critical section, +* the IPI handler reports the quiescent state immediately. +* +* Although this is a great improvement over previous expedited +* implementations, it is still unfriendly to real-time workloads, so is +* thus not recommended for any sort of common-case code. In fact, if +* you are using synchronize_rcu_expedited() in a loop, please restructure +* your code to batch your updates, and then use a single synchronize_rcu() +* instead. +* +* This has the same semantics as (but is more brutal than) synchronize_rcu(). +*/ +void synchronize_rcu_expedited (void ) +{ +__synchronize_rcu_expedited (false); }EXPORT_SYMBOL_GPL (synchronize_rcu_expedited);
@@ -903,7 +916,7 @@ static void sync_rcu_do_polled_gp(struct work_struct *wp)if (s &0x1 )return ;while (!sync_exp_work_done (s)) -synchronize_rcu_expedited (); +__synchronize_rcu_expedited (true);raw_spin_lock_irqsave (&rnp->exp_poll_lock, flags); s = rnp->exp_seq_poll_rq;if (!(s &0x1 ) && !sync_exp_work_done (s))
Signature · 1 line
2.25.1
Frederic Weisbecker
maintainercover rcu: synchronize_rcu[_expedited]() related fixes
A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.
We may still want to optimize start_poll_synchronize_rcu_expedited() on UP-no-preempt but I think Paul may be implying this while doing other fixes.
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/dev
HEAD: 6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096
Thanks,
Frederic
---
Frederic Weisbecker (2):
rcu: Fix expedited GP polling against UP/no-preempt environment
rcu: Fix preemption mode check on synchronize_rcu[_expedited]()
Valentin Schneider (1):
preempt/dynamic: Introduce preempt mode accessors
include/linux/sched.h | 16 +++++++++++++++ kernel/rcu/tree.c | 2 +- kernel/rcu/tree_exp.h | 57 +++++++++++++++++++++++++++++++-------------------- kernel/sched/core.c | 11 ++++++++++ 4 files changed, 63 insertions(+), 23 deletions(-)
Frederic Weisbecker
maintainerPATCH 2/3 preempt/dynamic: Introduce preempt mode accessors
From: Valentin Schneider <[email protected]>
CONFIG_PREEMPT{_NONE, _VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model when PREEMPT_DYNAMIC
IOW, using those on PREEMPT_DYNAMIC kernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.
Introduce a set of helpers to determine the actual preemption mode used by the live kernel.
--- include/linux/sched.h | 16 ++++++++++++++++ kernel/sched/core.c | 11 +++++++++++ 2 files changed, 27 insertions(+)
include/linux/sched.h +16 -0
@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)#endif } +#ifdef CONFIG_PREEMPT_DYNAMIC + +extern bool preempt_mode_none (void ); +extern bool preempt_mode_voluntary (void ); +extern bool preempt_mode_full (void ); + +#else + +#define preempt_mode_none () IS_ENABLED(CONFIG_PREEMPT_NONE) +#define preempt_mode_voluntary () IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY) +#define preempt_mode_full () IS_ENABLED(CONFIG_PREEMPT) + +#endif + +#define preempt_mode_rt () IS_ENABLED(CONFIG_PREEMPT_RT) +/* * Does a critical section need to be broken due to another * task waiting?: (technically does not depend on CONFIG_PREEMPTION,
kernel/sched/core.c +11 -0
@@ -6684,6 +6684,17 @@ static void __init preempt_dynamic_init(void) } } +#define PREEMPT_MODE_ACCESSOR (mode) \ + bool preempt_mode_##mode(void) \ + { \ + WARN_ON_ONCE(preempt_dynamic_mode == preempt_dynamic_undefined); \ + return preempt_dynamic_mode == preempt_dynamic_##mode; \ + } + +PREEMPT_MODE_ACCESSOR(none) +PREEMPT_MODE_ACCESSOR(voluntary) +PREEMPT_MODE_ACCESSOR(full) + #else /* !CONFIG_PREEMPT_DYNAMIC */ static inline void preempt_dynamic_init(void) { }
Signature · 1 line
2.25.1
Frederic Weisbecker
maintainerPATCH 3/3 rcu: Fix preemption mode check on synchronize_rcu[_expedited]()
In reply to Frederic WeisbeckerAn early check on synchronize_rcu[_expedited]() tries to determine if the current CPU is in UP mode on an SMP no-preempt kernel, in which case there is no need to start a grace period since the current assumed quiescent state is all we need.
However the preemption mode doesn't take into account the boot selected preemption mode under CONFIG_PREEMPT_DYNAMIC=y, missing a possible early return if the running flavour is "none" or "voluntary".
Use the shiny new preempt mode accessors to fix this.
--- kernel/rcu/tree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/tree.c +1 -1
@@ -3769,7 +3769,7 @@ static int rcu_blocking_is_gp(void) {int ret; -if (IS_ENABLED (CONFIG_PREEMPTION )) +if (preempt_mode_full () ||preempt_mode_rt ())return rcu_scheduler_active ==RCU_SCHEDULER_INACTIVE ;might_sleep ();/* Check for RCU read-side critical section. */ preempt_disable ();
Signature · 1 line
2.25.1
Re: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors
In reply to Frederic WeisbeckerQuoted 37 lines · Frederic Weisbecker
From: Valentin Schneider <[email protected]>
CONFIG_PREEMPT{_NONE,
_VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model whenPREEMPT_DYNAMICIOW, using those on
PREEMPT_DYNAMICkernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.Introduce a set of helpers to determine the actual preemption mode used by the live kernel.
Suggested-by Marco Elver <[email protected]>Signed-off-by Valentin Schneider <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>Cc Uladzislau Rezki <[email protected]>Cc Joel Fernandes <[email protected]>Cc Boqun Feng <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Neeraj Upadhyay <[email protected]>--- include/linux/sched.h | 16 ++++++++++++++++ kernel/sched/core.c | 11 +++++++++++ 2 files changed, 27 insertions(+)include/linux/sched.h +2
-0@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)#endif } +#ifdef CONFIG_PREEMPT_DYNAMIC +
include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110
Quoted 10 lines
include/linux/sched.h +10
-0+extern bool preempt_mode_none (void ); +extern bool preempt_mode_voluntary (void ); +extern bool preempt_mode_full (void ); + +#else + +#define preempt_mode_none () IS_ENABLED(CONFIG_PREEMPT_NONE) +#define preempt_mode_voluntary () IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY) +#define preempt_mode_full () IS_ENABLED(CONFIG_PREEMPT) +
Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).
https://lore.kernel.org/lkml/[email protected]/
Is it also possible to take all the rest of that series (all 4 patches) from Valentin?
Thanks,
-- Marco
Paul E. McKenney
maintainerRe: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors
Quoted 54 lines · Marco Elver
Quoted 37 lines · Frederic Weisbecker
From: Valentin Schneider <[email protected]>
CONFIG_PREEMPT{_NONE,
_VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model whenPREEMPT_DYNAMICIOW, using those on
PREEMPT_DYNAMICkernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.Introduce a set of helpers to determine the actual preemption mode used by the live kernel.
Suggested-by Marco Elver <[email protected]>Signed-off-by Valentin Schneider <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>Cc Uladzislau Rezki <[email protected]>Cc Joel Fernandes <[email protected]>Cc Boqun Feng <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Neeraj Upadhyay <[email protected]>--- include/linux/sched.h | 16 ++++++++++++++++ kernel/sched/core.c | 11 +++++++++++ 2 files changed, 27 insertions(+)include/linux/sched.h +2
-0@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)#endif } +#ifdef CONFIG_PREEMPT_DYNAMIC +include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110
Quoted 10 lines
include/linux/sched.h +10
-0+extern bool preempt_mode_none (void ); +extern bool preempt_mode_voluntary (void ); +extern bool preempt_mode_full (void ); + +#else + +#define preempt_mode_none () IS_ENABLED(CONFIG_PREEMPT_NONE) +#define preempt_mode_voluntary () IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY) +#define preempt_mode_full () IS_ENABLED(CONFIG_PREEMPT) +Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).
Quoted 4 lines
https://lore.kernel.org/lkml/[email protected]/
Is it also possible to take all the rest of that series (all 4 patches) from Valentin?
Me, I am assuming that #2/3 is an experimental test so that I am able to easily whack this series over the head with rcutorture. ;-)
Thanx, Paul
Paul E. McKenney
maintainerQuoted 8 lines · Frederic Weisbecker
A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.
We may still want to optimize
start_poll_synchronize_rcu_expedited()on UP-no-preempt but I think Paul may be implying this while doing other fixes.
Quoted 7 lines
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/devHEAD:
6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096Thanks,
Frederic
I have pulled these in for review and testing, thank you!!! I have started a ~90-minute test and will let you know how that goes.
Quoted 4 lines
---
Frederic Weisbecker (2):
rcu: Fix expedited GP polling against UP/no-preempt environment
I have some concerns with this one due to the fact that it acquires locks in cases where the old code would not. (I did have problems with this in both recent SRCU changes and the normal-grace-period counterpart to this series.)
But let's see what rcutorture and kbuild test robot think about it.
Quoted 1 line
rcu: Fix preemption mode check on synchronize_rcu[_expedited]()
This one looks good.
Quoted 2 lines
Valentin Schneider (1):
preempt/dynamic: Introduce preempt mode accessors
I am guessing that this one is a compact placeholder for my convenience (in which case thank you!). I will be marking it "EXP" on my next rebase.
Thanx, Paul
Quoted 5 lines
include/linux/sched.h | 16 +++++++++++++++ kernel/rcu/tree.c | 2 +- kernel/rcu/tree_exp.h | 57 +++++++++++++++++++++++++++++++-------------------- kernel/sched/core.c | 11 ++++++++++ 4 files changed, 63 insertions(+), 23 deletions(-)
Re: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors
In reply to Paul E. McKenneyQuoted 56 lines · Paul E. McKenney
Quoted 54 lines · Marco Elver
Quoted 37 lines · Frederic Weisbecker
From: Valentin Schneider <[email protected]>
CONFIG_PREEMPT{_NONE,
_VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model whenPREEMPT_DYNAMICIOW, using those on
PREEMPT_DYNAMICkernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.Introduce a set of helpers to determine the actual preemption mode used by the live kernel.
Suggested-by Marco Elver <[email protected]>Signed-off-by Valentin Schneider <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>Cc Uladzislau Rezki <[email protected]>Cc Joel Fernandes <[email protected]>Cc Boqun Feng <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Neeraj Upadhyay <[email protected]>--- include/linux/sched.h | 16 ++++++++++++++++ kernel/sched/core.c | 11 +++++++++++ 2 files changed, 27 insertions(+)include/linux/sched.h +2
-0@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)#endif } +#ifdef CONFIG_PREEMPT_DYNAMIC +include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110
Quoted 10 lines
include/linux/sched.h +10
-0+extern bool preempt_mode_none (void ); +extern bool preempt_mode_voluntary (void ); +extern bool preempt_mode_full (void ); + +#else + +#define preempt_mode_none () IS_ENABLED(CONFIG_PREEMPT_NONE) +#define preempt_mode_voluntary () IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY) +#define preempt_mode_full () IS_ENABLED(CONFIG_PREEMPT) +Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).
Quoted 7 lines
Quoted 4 lines
https://lore.kernel.org/lkml/[email protected]/
Is it also possible to take all the rest of that series (all 4 patches) from Valentin?
Me, I am assuming that #2/3 is an experimental test so that I am able to easily whack this series over the head with rcutorture. ;-)
I might be out of the loop here. All I can add is that any issues that are a consequence of the preempt mode accessors are only testable if the preemption model is actually changed at runtime and AFAIK rcutorture doesn't do that. But as noted, this patch wasn't the latest version and there were issues with it fixed by Valentin's latest v3 (from November, but had never been picked up anywhere).
Thanks,
-- Marco
Paul E. McKenney
maintainerRe: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors
Quoted 65 lines · Marco Elver
Quoted 56 lines · Paul E. McKenney
Quoted 54 lines · Marco Elver
Quoted 37 lines · Frederic Weisbecker
From: Valentin Schneider <[email protected]>
CONFIG_PREEMPT{_NONE,
_VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model whenPREEMPT_DYNAMICIOW, using those on
PREEMPT_DYNAMICkernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.Introduce a set of helpers to determine the actual preemption mode used by the live kernel.
Suggested-by Marco Elver <[email protected]>Signed-off-by Valentin Schneider <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>Cc Uladzislau Rezki <[email protected]>Cc Joel Fernandes <[email protected]>Cc Boqun Feng <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Neeraj Upadhyay <[email protected]>--- include/linux/sched.h | 16 ++++++++++++++++ kernel/sched/core.c | 11 +++++++++++ 2 files changed, 27 insertions(+)include/linux/sched.h +2
-0@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)#endif } +#ifdef CONFIG_PREEMPT_DYNAMIC +include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110
Quoted 10 lines
include/linux/sched.h +10
-0+extern bool preempt_mode_none (void ); +extern bool preempt_mode_voluntary (void ); +extern bool preempt_mode_full (void ); + +#else + +#define preempt_mode_none () IS_ENABLED(CONFIG_PREEMPT_NONE) +#define preempt_mode_voluntary () IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY) +#define preempt_mode_full () IS_ENABLED(CONFIG_PREEMPT) +Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).Quoted 7 lines
Quoted 4 lines
https://lore.kernel.org/lkml/[email protected]/
Is it also possible to take all the rest of that series (all 4 patches) from Valentin?
Me, I am assuming that #2/3 is an experimental test so that I am able to easily whack this series over the head with rcutorture. ;-)
Quoted 6 lines
I might be out of the loop here. All I can add is that any issues that are a consequence of the preempt mode accessors are only testable if the preemption model is actually changed at runtime and AFAIK rcutorture doesn't do that. But as noted, this patch wasn't the latest version and there were issues with it fixed by Valentin's latest v3 (from November, but had never been picked up anywhere).
I will be marking 2/3 "EXP" to prevent myself from accidentally sending it upstream.
But longer term, maybe I should pick up Valentin's series.
This stuff is v5.19 at the earliest, so not (yet) urgent.
Thanx, Paul
Paul E. McKenney
maintainerQuoted 12 lines · Paul E. McKenney
Quoted 11 lines · Frederic Weisbecker
A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.
We may still want to optimize
start_poll_synchronize_rcu_expedited()on UP-no-preempt but I think Paul may be implying this while doing other fixes.
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/dev
Quoted 7 lines
Quoted 4 lines
HEAD:
6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096Thanks,
FredericI have pulled these in for review and testing, thank you!!! I have started a ~90-minute test and will let you know how that goes.
And TREE05 doesn't like this much. I get too-short grace periods. TREE05 is unusual in being the one with the kernel build with CONFIG_PREEMPT_NONE=y but also with CONFIG_PREEMPT_DYNAMIC=n.
Running tests of the commits individually.
Thanx, Paul
Quoted 29 lines
Quoted 4 lines
---
Frederic Weisbecker (2):
rcu: Fix expedited GP polling against UP/no-preempt environmentI have some concerns with this one due to the fact that it acquires locks in cases where the old code would not. (I did have problems with this in both recent SRCU changes and the normal-grace-period counterpart to this series.)
But let's see what rcutorture and kbuild test robot think about it.
Quoted 1 line
rcu: Fix preemption mode check on synchronize_rcu[_expedited]()
This one looks good.
Quoted 2 lines
Valentin Schneider (1):
preempt/dynamic: Introduce preempt mode accessorsI am guessing that this one is a compact placeholder for my convenience (in which case thank you!). I will be marking it "EXP" on my next rebase.
Thanx, Paul
Quoted 5 lines
include/linux/sched.h | 16 +++++++++++++++ kernel/rcu/tree.c | 2 +- kernel/rcu/tree_exp.h | 57 +++++++++++++++++++++++++++++++-------------------- kernel/sched/core.c | 11 ++++++++++ 4 files changed, 63 insertions(+), 23 deletions(-)
Frederic Weisbecker
maintainerRe: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors
In reply to Marco ElverQuoted 65 lines · Marco Elver
Quoted 56 lines · Paul E. McKenney
Quoted 54 lines · Marco Elver
Quoted 37 lines · Frederic Weisbecker
From: Valentin Schneider <[email protected]>
CONFIG_PREEMPT{_NONE,
_VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model whenPREEMPT_DYNAMICIOW, using those on
PREEMPT_DYNAMICkernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.Introduce a set of helpers to determine the actual preemption mode used by the live kernel.
Suggested-by Marco Elver <[email protected]>Signed-off-by Valentin Schneider <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>Cc Uladzislau Rezki <[email protected]>Cc Joel Fernandes <[email protected]>Cc Boqun Feng <[email protected]>Cc Peter Zijlstra <[email protected]>Cc Neeraj Upadhyay <[email protected]>--- include/linux/sched.h | 16 ++++++++++++++++ kernel/sched/core.c | 11 +++++++++++ 2 files changed, 27 insertions(+)include/linux/sched.h +2
-0@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)#endif } +#ifdef CONFIG_PREEMPT_DYNAMIC +include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110
Quoted 10 lines
include/linux/sched.h +10
-0+extern bool preempt_mode_none (void ); +extern bool preempt_mode_voluntary (void ); +extern bool preempt_mode_full (void ); + +#else + +#define preempt_mode_none () IS_ENABLED(CONFIG_PREEMPT_NONE) +#define preempt_mode_voluntary () IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY) +#define preempt_mode_full () IS_ENABLED(CONFIG_PREEMPT) +Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).Quoted 7 lines
Quoted 4 lines
https://lore.kernel.org/lkml/[email protected]/
Is it also possible to take all the rest of that series (all 4 patches) from Valentin?
Me, I am assuming that #2/3 is an experimental test so that I am able to easily whack this series over the head with rcutorture. ;-)
Quoted 6 lines
I might be out of the loop here. All I can add is that any issues that are a consequence of the preempt mode accessors are only testable if the preemption model is actually changed at runtime and AFAIK rcutorture doesn't do that. But as noted, this patch wasn't the latest version and there were issues with it fixed by Valentin's latest v3 (from November, but had never been picked up anywhere).
Thanks for the reminder, the patchset indeed got forgotten somewhow. If necessary I'll resend it.
Peter Zijlstra
maintainerRe: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors
In reply to Marco ElverQuoted 4 lines · Marco Elver
https://lore.kernel.org/lkml/[email protected]/
Is it also possible to take all the rest of that series (all 4 patches) from Valentin?
I'll go stick the remaining 3 patches from Valentin in sched/core, they seem to still apply without issue.
Paul E. McKenney
maintainerRe: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Marco Elver
https://lore.kernel.org/lkml/[email protected]/
Is it also possible to take all the rest of that series (all 4 patches) from Valentin?
I'll go stick the remaining 3 patches from Valentin in sched/core, they seem to still apply without issue.
If testing goes well, are you planning to push into the upcoming merge window? (I won't be pushing until the v5.19 merge window at the earliest.)
Either way, just please let me know!
Thanx, Paul
Frederic Weisbecker
maintainerQuoted 18 lines · Paul E. McKenney
Quoted 17 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker
A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.
We may still want to optimize
start_poll_synchronize_rcu_expedited()on UP-no-preempt but I think Paul may be implying this while doing other fixes.
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/devHEAD:
6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096Thanks,
Frederic
Quoted 8 lines
Quoted 2 lines
I have pulled these in for review and testing, thank you!!! I have started a ~90-minute test and will let you know how that goes.
And TREE05 doesn't like this much. I get too-short grace periods. TREE05 is unusual in being the one with the kernel build with
CONFIG_PREEMPT_NONE=ybut also with CONFIG_PREEMPT_DYNAMIC=n.Running tests of the commits individually.
Ouch, please test the following:
---
From f180dd5809d2c3a6343cbd13f244b7b7f110a506 Mon Sep 17 00:00:00 2001
From: Frederic Weisbecker <[email protected]>
Date: Tue, 15 Mar 2022 16:33:38 +0100
Subject: [PATCH] rcutorture: Call preempt_schedule() through static call/key
rcutorture sometimes want to trigger a random scheduler preemption call while simulating a read delay. However a direct call to preempt_schedule() is not desirable because it bypasses the static call/key filter used by CONFIG_PREEMPT_DYNAMIC. This breaks the no-preempt assumption in case the dynamic preemption mode is "none".
For example rcu_blocking_is_gp() is fooled and abbreviates grace periods when the CPU runs in no-preempt UP mode.
Fix this with making torture_preempt_schedule() to call through preempt dynamic static call/key.
--- include/linux/torture.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/torture.h +1 -1
@@ -118,7 +118,7 @@ void _torture_stop_kthread(char *m, struct task_struct **tp); _torture_stop_kthread("Stopping " #n " task", &(tp))#ifdef CONFIG_PREEMPTION -#define torture_preempt_schedule () preempt_schedule() +#define torture_preempt_schedule () __preempt_schedule()#else #define torture_preempt_schedule () do { } while (0)#endif
Signature · 1 line
2.25.1
Paul E. McKenney
maintainerQuoted 28 lines · Frederic Weisbecker
Quoted 18 lines · Paul E. McKenney
Quoted 17 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker
A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.
We may still want to optimize
start_poll_synchronize_rcu_expedited()on UP-no-preempt but I think Paul may be implying this while doing other fixes.
git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/devHEAD:
6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096Thanks,
FredericQuoted 8 lines
Quoted 2 lines
I have pulled these in for review and testing, thank you!!! I have started a ~90-minute test and will let you know how that goes.
And TREE05 doesn't like this much. I get too-short grace periods. TREE05 is unusual in being the one with the kernel build with
CONFIG_PREEMPT_NONE=ybut also with CONFIG_PREEMPT_DYNAMIC=n.Running tests of the commits individually.
Quoted 7 lines
Ouch, please test the following:
---
Quoted 1 line
From
f180dd5809d2c3a6343cbd13f244b7b7f110a506Mon Sep 17 00:00:00 2001From: Frederic Weisbecker <[email protected]>
Date: Tue, 15 Mar 2022 16:33:38 +0100
Subject: [PATCH] rcutorture: Callpreempt_schedule()through static call/key
You know, it would have been quite some time before I would have thought to suspect that code, so thank you very much for chasing this down!
I have pulled it in, reverted my reversion of your patch 3/3, and started a moderate set of tests.
Thanx, Paul
Quoted 34 lines
rcutorture sometimes want to trigger a random scheduler preemption call while simulating a read delay. However a direct call to
preempt_schedule()is not desirable because it bypasses the static call/key filter used byCONFIG_PREEMPT_DYNAMIC. This breaks the no-preempt assumption in case the dynamic preemption mode is "none".For example
rcu_blocking_is_gp()is fooled and abbreviates grace periods when the CPU runs in no-preempt UP mode.Fix this with making
torture_preempt_schedule()to call through preempt dynamic static call/key.Reported-by Paul E. McKenney <[email protected]>Signed-off-by Frederic Weisbecker <[email protected]>--- include/linux/torture.h | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)include/linux/torture.h +1
-1@@ -118,7 +118,7 @@ void _torture_stop_kthread(char *m, struct task_struct **tp); _torture_stop_kthread("Stopping " #n " task", &(tp))#ifdef CONFIG_PREEMPTION -#define torture_preempt_schedule () preempt_schedule() +#define torture_preempt_schedule () __preempt_schedule()#else #define torture_preempt_schedule () do { } while (0)#endif Signature · 1 line
2.25.1