Skip to contentSkip to conversation

rcu: synchronize_rcu[_expedited]() related fixes

PATCH In review Frederic Weisbecker 4 participants 15 messages Mar 14, 2022 – Mar 15, 2022

Subsystems (2)
lore
Share & tools
lore

Frederic Weisbecker

author maintainer
#0 lore raw

cover rcu: synchronize_rcu[_expedited]() related fixes

A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.

We may still want to optimize start_poll_synchronize_rcu_expedited() on UP-no-preempt but I think Paul may be implying this while doing other fixes.

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/dev

HEAD: 6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096

Thanks,
Frederic
---

Frederic Weisbecker (2):
rcu: Fix expedited GP polling against UP/no-preempt environment
rcu: Fix preemption mode check on synchronize_rcu[_expedited]()

Valentin Schneider (1):
preempt/dynamic: Introduce preempt mode accessors

 include/linux/sched.h | 16 +++++++++++++++
 kernel/rcu/tree.c     |  2 +-
 kernel/rcu/tree_exp.h | 57 +++++++++++++++++++++++++++++++--------------------
 kernel/sched/core.c   | 11 ++++++++++
 4 files changed, 63 insertions(+), 23 deletions(-)
author maintainer
#1 lore raw

PATCH 1/3 rcu: Fix expedited GP polling against UP/no-preempt environment

synchronize_rcu_expedited() has an early return condition: if the current CPU is the only one online and the kernel doesn't run in preemption mode, the current assumed quiescent state is worth a grace period.

However the expedited grace period polling caller of synchronize_rcu_expedited() takes a GP sequence snapshot and expects it to complete by the end of the synchronize_rcu_expedited() call. Yet if synchronize_rcu_expedited() relies on the above described UP/no-preempt early return, the grace period sequence won't move and may cause an expedited grace period polling stall.

Fix this with treating polling differently while calling synchronize_rcu_expedited() and ignore the UP-no-preempt optimization in this case.

Cc Uladzislau Rezki <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Neeraj Upadhyay <[email protected]>
Cc Valentin Schneider <[email protected]>
---
 kernel/rcu/tree_exp.h | 57 ++++++++++++++++++++++++++-----------------
 1 file changed, 35 insertions(+), 22 deletions(-)
kernel/rcu/tree_exp.h +35 -22
index d5f30085b0cf..3d8216ced93e 100644
@@ -794,27 +794,14 @@ static int rcu_print_task_exp_stall(struct rcu_node *rnp)
 
 #endif /* #else #ifdef CONFIG_PREEMPT_RCU */
 
-/**
- * synchronize_rcu_expedited - Brute-force RCU grace period
- *
- * Wait for an RCU grace period, but expedite it.  The basic idea is to
- * IPI all non-idle non-nohz online CPUs.  The IPI handler checks whether
- * the CPU is in an RCU critical section, and if so, it sets a flag that
- * causes the outermost rcu_read_unlock() to report the quiescent state
- * for RCU-preempt or asks the scheduler for help for RCU-sched.  On the
- * other hand, if the CPU is not in an RCU read-side critical section,
- * the IPI handler reports the quiescent state immediately.
- *
- * Although this is a great improvement over previous expedited
- * implementations, it is still unfriendly to real-time workloads, so is
- * thus not recommended for any sort of common-case code.  In fact, if
- * you are using synchronize_rcu_expedited() in a loop, please restructure
- * your code to batch your updates, and then use a single synchronize_rcu()
- * instead.
- *
- * This has the same semantics as (but is more brutal than) synchronize_rcu().
+/*
+ * Start and wait for an expedited grace period completion.
+ * If it happens to be called by polling functions (@polling = true),
+ * there is no possible early return in UP no-preempt mode because
+ * the callers are waiting for an actual given sequence snapshot to start
+ * and end.
  */
-void synchronize_rcu_expedited(void)
+static void __synchronize_rcu_expedited(bool polling)
 {
 	bool boottime = (rcu_scheduler_active == RCU_SCHEDULER_INIT);
 	struct rcu_exp_work rew;
@@ -827,7 +814,7 @@ void synchronize_rcu_expedited(void)
 			 "Illegal synchronize_rcu_expedited() in RCU read-side critical section");
 
 	/* Is the state is such that the call is a grace period? */
-	if (rcu_blocking_is_gp())
+	if (rcu_blocking_is_gp() && !polling)
 		return;
 
 	/* If expedited grace periods are prohibited, fall back to normal. */
@@ -863,6 +850,32 @@ void synchronize_rcu_expedited(void)
 
 	if (likely(!boottime))
 		destroy_work_on_stack(&rew.rew_work);
+
+}
+
+/**
+ * synchronize_rcu_expedited - Brute-force RCU grace period
+ *
+ * Wait for an RCU grace period, but expedite it.  The basic idea is to
+ * IPI all non-idle non-nohz online CPUs.  The IPI handler checks whether
+ * the CPU is in an RCU critical section, and if so, it sets a flag that
+ * causes the outermost rcu_read_unlock() to report the quiescent state
+ * for RCU-preempt or asks the scheduler for help for RCU-sched.  On the
+ * other hand, if the CPU is not in an RCU read-side critical section,
+ * the IPI handler reports the quiescent state immediately.
+ *
+ * Although this is a great improvement over previous expedited
+ * implementations, it is still unfriendly to real-time workloads, so is
+ * thus not recommended for any sort of common-case code.  In fact, if
+ * you are using synchronize_rcu_expedited() in a loop, please restructure
+ * your code to batch your updates, and then use a single synchronize_rcu()
+ * instead.
+ *
+ * This has the same semantics as (but is more brutal than) synchronize_rcu().
+ */
+void synchronize_rcu_expedited(void)
+{
+	__synchronize_rcu_expedited(false);
 }
 EXPORT_SYMBOL_GPL(synchronize_rcu_expedited);
 
@@ -903,7 +916,7 @@ static void sync_rcu_do_polled_gp(struct work_struct *wp)
 	if (s & 0x1)
 		return;
 	while (!sync_exp_work_done(s))
-		synchronize_rcu_expedited();
+		__synchronize_rcu_expedited(true);
 	raw_spin_lock_irqsave(&rnp->exp_poll_lock, flags);
 	s = rnp->exp_seq_poll_rq;
 	if (!(s & 0x1) && !sync_exp_work_done(s))
Signature · 1 line
2.25.1
author maintainer
#2 lore raw

PATCH 2/3 preempt/dynamic: Introduce preempt mode accessors

In reply to Frederic Weisbecker

From: Valentin Schneider <[email protected]>

CONFIG_PREEMPT{_NONE, _VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model when PREEMPT_DYNAMIC

IOW, using those on PREEMPT_DYNAMIC kernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.

Introduce a set of helpers to determine the actual preemption mode used by the live kernel.

Cc Uladzislau Rezki <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Neeraj Upadhyay <[email protected]>
---
 include/linux/sched.h | 16 ++++++++++++++++
 kernel/sched/core.c   | 11 +++++++++++
 2 files changed, 27 insertions(+)
include/linux/sched.h +16 -0
index 508b91d57470..d348e886e4d0 100644
@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)
 #endif
 }
 
+#ifdef CONFIG_PREEMPT_DYNAMIC
+
+extern bool preempt_mode_none(void);
+extern bool preempt_mode_voluntary(void);
+extern bool preempt_mode_full(void);
+
+#else
+
+#define preempt_mode_none() IS_ENABLED(CONFIG_PREEMPT_NONE)
+#define preempt_mode_voluntary() IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)
+#define preempt_mode_full() IS_ENABLED(CONFIG_PREEMPT)
+
+#endif
+
+#define preempt_mode_rt() IS_ENABLED(CONFIG_PREEMPT_RT)
+
 /*
  * Does a critical section need to be broken due to another
  * task waiting?: (technically does not depend on CONFIG_PREEMPTION,
kernel/sched/core.c +11 -0
index 2e4ae00e52d1..6e5c8f0f874a 100644
@@ -6684,6 +6684,17 @@ static void __init preempt_dynamic_init(void)
 	}
 }
 
+#define PREEMPT_MODE_ACCESSOR(mode) \
+	bool preempt_mode_##mode(void)						 \
+	{									 \
+		WARN_ON_ONCE(preempt_dynamic_mode == preempt_dynamic_undefined); \
+		return preempt_dynamic_mode == preempt_dynamic_##mode;		 \
+	}
+
+PREEMPT_MODE_ACCESSOR(none)
+PREEMPT_MODE_ACCESSOR(voluntary)
+PREEMPT_MODE_ACCESSOR(full)
+
 #else /* !CONFIG_PREEMPT_DYNAMIC */
 
 static inline void preempt_dynamic_init(void) { }
Signature · 1 line
2.25.1
#3 lore raw

Re: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors

Quoted 37 lines · Frederic Weisbecker

From: Valentin Schneider <[email protected]>

CONFIG_PREEMPT{_NONE, _VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model when PREEMPT_DYNAMIC

IOW, using those on PREEMPT_DYNAMIC kernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.

Introduce a set of helpers to determine the actual preemption mode used by the live kernel.

Cc Uladzislau Rezki <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Neeraj Upadhyay <[email protected]>
---
 include/linux/sched.h | 16 ++++++++++++++++
 kernel/sched/core.c   | 11 +++++++++++
 2 files changed, 27 insertions(+)
include/linux/sched.h +2 -0
index 508b91d57470..d348e886e4d0 100644
@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)
 #endif
 }
 
+#ifdef CONFIG_PREEMPT_DYNAMIC
+

include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110

Quoted 10 lines
include/linux/sched.h +10 -0
+extern bool preempt_mode_none(void);
+extern bool preempt_mode_voluntary(void);
+extern bool preempt_mode_full(void);
+
+#else
+
+#define preempt_mode_none() IS_ENABLED(CONFIG_PREEMPT_NONE)
+#define preempt_mode_voluntary() IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)
+#define preempt_mode_full() IS_ENABLED(CONFIG_PREEMPT)
+

Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).

https://lore.kernel.org/lkml/[email protected]/

Is it also possible to take all the rest of that series (all 4 patches) from Valentin?

Thanks,
-- Marco

maintainer
#4 lore raw

Re: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors

Quoted 54 lines · Marco Elver
Quoted 37 lines · Frederic Weisbecker

From: Valentin Schneider <[email protected]>

CONFIG_PREEMPT{_NONE, _VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model when PREEMPT_DYNAMIC

IOW, using those on PREEMPT_DYNAMIC kernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.

Introduce a set of helpers to determine the actual preemption mode used by the live kernel.

Cc Uladzislau Rezki <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Neeraj Upadhyay <[email protected]>
---
 include/linux/sched.h | 16 ++++++++++++++++
 kernel/sched/core.c   | 11 +++++++++++
 2 files changed, 27 insertions(+)
include/linux/sched.h +2 -0
index 508b91d57470..d348e886e4d0 100644
@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)
 #endif
 }
 
+#ifdef CONFIG_PREEMPT_DYNAMIC
+

include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110

Quoted 10 lines
include/linux/sched.h +10 -0
+extern bool preempt_mode_none(void);
+extern bool preempt_mode_voluntary(void);
+extern bool preempt_mode_full(void);
+
+#else
+
+#define preempt_mode_none() IS_ENABLED(CONFIG_PREEMPT_NONE)
+#define preempt_mode_voluntary() IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)
+#define preempt_mode_full() IS_ENABLED(CONFIG_PREEMPT)
+

Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).

Quoted 4 lines

https://lore.kernel.org/lkml/[email protected]/

Is it also possible to take all the rest of that series (all 4 patches) from Valentin?

Me, I am assuming that #2/3 is an experimental test so that I am able to easily whack this series over the head with rcutorture. ;-)

Thanx, Paul

#5 lore raw

Re: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors

Quoted 56 lines · Paul E. McKenney
Quoted 54 lines · Marco Elver
Quoted 37 lines · Frederic Weisbecker

From: Valentin Schneider <[email protected]>

CONFIG_PREEMPT{_NONE, _VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model when PREEMPT_DYNAMIC

IOW, using those on PREEMPT_DYNAMIC kernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.

Introduce a set of helpers to determine the actual preemption mode used by the live kernel.

Cc Uladzislau Rezki <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Neeraj Upadhyay <[email protected]>
---
 include/linux/sched.h | 16 ++++++++++++++++
 kernel/sched/core.c   | 11 +++++++++++
 2 files changed, 27 insertions(+)
include/linux/sched.h +2 -0
index 508b91d57470..d348e886e4d0 100644
@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)
 #endif
 }
 
+#ifdef CONFIG_PREEMPT_DYNAMIC
+

include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110

Quoted 10 lines
include/linux/sched.h +10 -0
+extern bool preempt_mode_none(void);
+extern bool preempt_mode_voluntary(void);
+extern bool preempt_mode_full(void);
+
+#else
+
+#define preempt_mode_none() IS_ENABLED(CONFIG_PREEMPT_NONE)
+#define preempt_mode_voluntary() IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)
+#define preempt_mode_full() IS_ENABLED(CONFIG_PREEMPT)
+

Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).

Quoted 7 lines
Quoted 4 lines

https://lore.kernel.org/lkml/[email protected]/

Is it also possible to take all the rest of that series (all 4 patches) from Valentin?

Me, I am assuming that #2/3 is an experimental test so that I am able to easily whack this series over the head with rcutorture. ;-)

I might be out of the loop here. All I can add is that any issues that are a consequence of the preempt mode accessors are only testable if the preemption model is actually changed at runtime and AFAIK rcutorture doesn't do that. But as noted, this patch wasn't the latest version and there were issues with it fixed by Valentin's latest v3 (from November, but had never been picked up anywhere).

Thanks,
-- Marco

maintainer
#6 lore raw

Re: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors

Quoted 65 lines · Marco Elver
Quoted 56 lines · Paul E. McKenney
Quoted 54 lines · Marco Elver
Quoted 37 lines · Frederic Weisbecker

From: Valentin Schneider <[email protected]>

CONFIG_PREEMPT{_NONE, _VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model when PREEMPT_DYNAMIC

IOW, using those on PREEMPT_DYNAMIC kernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.

Introduce a set of helpers to determine the actual preemption mode used by the live kernel.

Cc Uladzislau Rezki <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Neeraj Upadhyay <[email protected]>
---
 include/linux/sched.h | 16 ++++++++++++++++
 kernel/sched/core.c   | 11 +++++++++++
 2 files changed, 27 insertions(+)
include/linux/sched.h +2 -0
index 508b91d57470..d348e886e4d0 100644
@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)
 #endif
 }
 
+#ifdef CONFIG_PREEMPT_DYNAMIC
+

include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110

Quoted 10 lines
include/linux/sched.h +10 -0
+extern bool preempt_mode_none(void);
+extern bool preempt_mode_voluntary(void);
+extern bool preempt_mode_full(void);
+
+#else
+
+#define preempt_mode_none() IS_ENABLED(CONFIG_PREEMPT_NONE)
+#define preempt_mode_voluntary() IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)
+#define preempt_mode_full() IS_ENABLED(CONFIG_PREEMPT)
+

Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).

Quoted 7 lines
Quoted 4 lines

https://lore.kernel.org/lkml/[email protected]/

Is it also possible to take all the rest of that series (all 4 patches) from Valentin?

Me, I am assuming that #2/3 is an experimental test so that I am able to easily whack this series over the head with rcutorture. ;-)

Quoted 6 lines

I might be out of the loop here. All I can add is that any issues that are a consequence of the preempt mode accessors are only testable if the preemption model is actually changed at runtime and AFAIK rcutorture doesn't do that. But as noted, this patch wasn't the latest version and there were issues with it fixed by Valentin's latest v3 (from November, but had never been picked up anywhere).

I will be marking 2/3 "EXP" to prevent myself from accidentally sending it upstream.

But longer term, maybe I should pick up Valentin's series.

This stuff is v5.19 at the earliest, so not (yet) urgent.

Thanx, Paul

author maintainer
#7 lore raw

Re: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors

In reply to Marco Elver
Quoted 65 lines · Marco Elver
Quoted 56 lines · Paul E. McKenney
Quoted 54 lines · Marco Elver
Quoted 37 lines · Frederic Weisbecker

From: Valentin Schneider <[email protected]>

CONFIG_PREEMPT{_NONE, _VOLUNTARY} designate either:
o The build-time preemption model when !PREEMPT_DYNAMIC
o The default boot-time preemption model when PREEMPT_DYNAMIC

IOW, using those on PREEMPT_DYNAMIC kernels is meaningless - the actual model could have been set to something else by the "preempt=foo" cmdline parameter.

Introduce a set of helpers to determine the actual preemption mode used by the live kernel.

Cc Uladzislau Rezki <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Neeraj Upadhyay <[email protected]>
---
 include/linux/sched.h | 16 ++++++++++++++++
 kernel/sched/core.c   | 11 +++++++++++
 2 files changed, 27 insertions(+)
include/linux/sched.h +2 -0
index 508b91d57470..d348e886e4d0 100644
@@ -2096,6 +2096,22 @@ static inline void cond_resched_rcu(void)
 #endif
 }
 
+#ifdef CONFIG_PREEMPT_DYNAMIC
+

include/linux/sched.h static inline void cond_resched_rcu(void) · line 2110

Quoted 10 lines
include/linux/sched.h +10 -0
+extern bool preempt_mode_none(void);
+extern bool preempt_mode_voluntary(void);
+extern bool preempt_mode_full(void);
+
+#else
+
+#define preempt_mode_none() IS_ENABLED(CONFIG_PREEMPT_NONE)
+#define preempt_mode_voluntary() IS_ENABLED(CONFIG_PREEMPT_VOLUNTARY)
+#define preempt_mode_full() IS_ENABLED(CONFIG_PREEMPT)
+

Shame this was somehow forgotten.
There was a v3 of this patch that fixed a bunch of things (e.g. making these proper functions so all builds error if accidentally used in #if).

Quoted 7 lines
Quoted 4 lines

https://lore.kernel.org/lkml/[email protected]/

Is it also possible to take all the rest of that series (all 4 patches) from Valentin?

Me, I am assuming that #2/3 is an experimental test so that I am able to easily whack this series over the head with rcutorture. ;-)

Quoted 6 lines

I might be out of the loop here. All I can add is that any issues that are a consequence of the preempt mode accessors are only testable if the preemption model is actually changed at runtime and AFAIK rcutorture doesn't do that. But as noted, this patch wasn't the latest version and there were issues with it fixed by Valentin's latest v3 (from November, but had never been picked up anywhere).

Thanks for the reminder, the patchset indeed got forgotten somewhow. If necessary I'll resend it.

maintainer
#9 lore raw

Re: [PATCH 2/3] preempt/dynamic: Introduce preempt mode accessors

Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Marco Elver

https://lore.kernel.org/lkml/[email protected]/

Is it also possible to take all the rest of that series (all 4 patches) from Valentin?

I'll go stick the remaining 3 patches from Valentin in sched/core, they seem to still apply without issue.

If testing goes well, are you planning to push into the upcoming merge window? (I won't be pushing until the v5.19 merge window at the earliest.)

Either way, just please let me know!

Thanx, Paul

author maintainer
#10 lore raw

PATCH 3/3 rcu: Fix preemption mode check on synchronize_rcu[_expedited]()

In reply to Frederic Weisbecker

An early check on synchronize_rcu[_expedited]() tries to determine if the current CPU is in UP mode on an SMP no-preempt kernel, in which case there is no need to start a grace period since the current assumed quiescent state is all we need.

However the preemption mode doesn't take into account the boot selected preemption mode under CONFIG_PREEMPT_DYNAMIC=y, missing a possible early return if the running flavour is "none" or "voluntary".

Use the shiny new preempt mode accessors to fix this.

Cc Uladzislau Rezki <[email protected]>
Cc Joel Fernandes <[email protected]>
Cc Boqun Feng <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Neeraj Upadhyay <[email protected]>
Cc Valentin Schneider <[email protected]>
---
 kernel/rcu/tree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/tree.c +1 -1
index d2db709f83d9..8fe2f9843139 100644
@@ -3769,7 +3769,7 @@ static int rcu_blocking_is_gp(void)
 {
 	int ret;
 
-	if (IS_ENABLED(CONFIG_PREEMPTION))
+	if (preempt_mode_full() || preempt_mode_rt())
 		return rcu_scheduler_active == RCU_SCHEDULER_INACTIVE;
 	might_sleep();  /* Check for RCU read-side critical section. */
 	preempt_disable();
Signature · 1 line
2.25.1
maintainer
#11 lore raw
In reply to Frederic Weisbecker
Quoted 8 lines · Frederic Weisbecker

A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.

We may still want to optimize start_poll_synchronize_rcu_expedited() on UP-no-preempt but I think Paul may be implying this while doing other fixes.

Quoted 7 lines

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/dev

HEAD: 6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096

Thanks,
Frederic

I have pulled these in for review and testing, thank you!!! I have started a ~90-minute test and will let you know how that goes.

Quoted 4 lines

---

Frederic Weisbecker (2):
rcu: Fix expedited GP polling against UP/no-preempt environment

I have some concerns with this one due to the fact that it acquires locks in cases where the old code would not. (I did have problems with this in both recent SRCU changes and the normal-grace-period counterpart to this series.)

But let's see what rcutorture and kbuild test robot think about it.

Quoted 1 line

rcu: Fix preemption mode check on synchronize_rcu[_expedited]()

This one looks good.

Quoted 2 lines

Valentin Schneider (1):
preempt/dynamic: Introduce preempt mode accessors

I am guessing that this one is a compact placeholder for my convenience (in which case thank you!). I will be marking it "EXP" on my next rebase.

Thanx, Paul

Quoted 5 lines
 include/linux/sched.h | 16 +++++++++++++++
 kernel/rcu/tree.c     |  2 +-
 kernel/rcu/tree_exp.h | 57 +++++++++++++++++++++++++++++++--------------------
 kernel/sched/core.c   | 11 ++++++++++
 4 files changed, 63 insertions(+), 23 deletions(-)
maintainer
#12 lore raw
Quoted 12 lines · Paul E. McKenney
Quoted 11 lines · Frederic Weisbecker

A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.

We may still want to optimize start_poll_synchronize_rcu_expedited() on UP-no-preempt but I think Paul may be implying this while doing other fixes.

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/dev

Quoted 7 lines
Quoted 4 lines

HEAD: 6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096

Thanks,
Frederic

I have pulled these in for review and testing, thank you!!! I have started a ~90-minute test and will let you know how that goes.

And TREE05 doesn't like this much. I get too-short grace periods. TREE05 is unusual in being the one with the kernel build with CONFIG_PREEMPT_NONE=y but also with CONFIG_PREEMPT_DYNAMIC=n.

Running tests of the commits individually.

Thanx, Paul

Quoted 29 lines
Quoted 4 lines

---

Frederic Weisbecker (2):
rcu: Fix expedited GP polling against UP/no-preempt environment

I have some concerns with this one due to the fact that it acquires locks in cases where the old code would not. (I did have problems with this in both recent SRCU changes and the normal-grace-period counterpart to this series.)

But let's see what rcutorture and kbuild test robot think about it.

Quoted 1 line

rcu: Fix preemption mode check on synchronize_rcu[_expedited]()

This one looks good.

Quoted 2 lines

Valentin Schneider (1):
preempt/dynamic: Introduce preempt mode accessors

I am guessing that this one is a compact placeholder for my convenience (in which case thank you!). I will be marking it "EXP" on my next rebase.

Thanx, Paul

Quoted 5 lines
 include/linux/sched.h | 16 +++++++++++++++
 kernel/rcu/tree.c     |  2 +-
 kernel/rcu/tree_exp.h | 57 +++++++++++++++++++++++++++++++--------------------
 kernel/sched/core.c   | 11 ++++++++++
 4 files changed, 63 insertions(+), 23 deletions(-)
author maintainer
#13 lore raw
Quoted 18 lines · Paul E. McKenney
Quoted 17 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.

We may still want to optimize start_poll_synchronize_rcu_expedited() on UP-no-preempt but I think Paul may be implying this while doing other fixes.

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/dev

HEAD: 6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096

Thanks,
Frederic

Quoted 8 lines
Quoted 2 lines

I have pulled these in for review and testing, thank you!!! I have started a ~90-minute test and will let you know how that goes.

And TREE05 doesn't like this much. I get too-short grace periods. TREE05 is unusual in being the one with the kernel build with CONFIG_PREEMPT_NONE=y but also with CONFIG_PREEMPT_DYNAMIC=n.

Running tests of the commits individually.

Ouch, please test the following:

---
From f180dd5809d2c3a6343cbd13f244b7b7f110a506 Mon Sep 17 00:00:00 2001
From: Frederic Weisbecker <[email protected]>
Date: Tue, 15 Mar 2022 16:33:38 +0100
Subject: [PATCH] rcutorture: Call preempt_schedule() through static call/key

rcutorture sometimes want to trigger a random scheduler preemption call while simulating a read delay. However a direct call to preempt_schedule() is not desirable because it bypasses the static call/key filter used by CONFIG_PREEMPT_DYNAMIC. This breaks the no-preempt assumption in case the dynamic preemption mode is "none".

For example rcu_blocking_is_gp() is fooled and abbreviates grace periods when the CPU runs in no-preempt UP mode.

Fix this with making torture_preempt_schedule() to call through preempt dynamic static call/key.

---
 include/linux/torture.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/torture.h +1 -1
index 63fa4196e51c..7038104463e4 100644
@@ -118,7 +118,7 @@ void _torture_stop_kthread(char *m, struct task_struct **tp);
 	_torture_stop_kthread("Stopping " #n " task", &(tp))
 
 #ifdef CONFIG_PREEMPTION
-#define torture_preempt_schedule() preempt_schedule()
+#define torture_preempt_schedule() __preempt_schedule()
 #else
 #define torture_preempt_schedule()	do { } while (0)
 #endif
Signature · 1 line
2.25.1
maintainer
#14 lore raw
Quoted 28 lines · Frederic Weisbecker
Quoted 18 lines · Paul E. McKenney
Quoted 17 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

A few fixes especially for expedited GP polling causing a stall on TREE07, as reported by Paul.

We may still want to optimize start_poll_synchronize_rcu_expedited() on UP-no-preempt but I think Paul may be implying this while doing other fixes.

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
rcu/dev

HEAD: 6e5fd7e614fd5c8f0fffeaa140b7ea697bfeb096

Thanks,
Frederic

Quoted 8 lines
Quoted 2 lines

I have pulled these in for review and testing, thank you!!! I have started a ~90-minute test and will let you know how that goes.

And TREE05 doesn't like this much. I get too-short grace periods. TREE05 is unusual in being the one with the kernel build with CONFIG_PREEMPT_NONE=y but also with CONFIG_PREEMPT_DYNAMIC=n.

Running tests of the commits individually.

Quoted 7 lines

Ouch, please test the following:

---

Quoted 1 line

From f180dd5809d2c3a6343cbd13f244b7b7f110a506 Mon Sep 17 00:00:00 2001

From: Frederic Weisbecker <[email protected]>
Date: Tue, 15 Mar 2022 16:33:38 +0100
Subject: [PATCH] rcutorture: Call preempt_schedule() through static call/key

You know, it would have been quite some time before I would have thought to suspect that code, so thank you very much for chasing this down!

I have pulled it in, reverted my reversion of your patch 3/3, and started a moderate set of tests.

Thanx, Paul

Quoted 34 lines

rcutorture sometimes want to trigger a random scheduler preemption call while simulating a read delay. However a direct call to preempt_schedule() is not desirable because it bypasses the static call/key filter used by CONFIG_PREEMPT_DYNAMIC. This breaks the no-preempt assumption in case the dynamic preemption mode is "none".

For example rcu_blocking_is_gp() is fooled and abbreviates grace periods when the CPU runs in no-preempt UP mode.

Fix this with making torture_preempt_schedule() to call through preempt dynamic static call/key.

---
 include/linux/torture.h | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
include/linux/torture.h +1 -1
index 63fa4196e51c..7038104463e4 100644
@@ -118,7 +118,7 @@ void _torture_stop_kthread(char *m, struct task_struct **tp);
 	_torture_stop_kthread("Stopping " #n " task", &(tp))
 
 #ifdef CONFIG_PREEMPTION
-#define torture_preempt_schedule() preempt_schedule()
+#define torture_preempt_schedule() __preempt_schedule()
 #else
 #define torture_preempt_schedule()	do { } while (0)
 #endif
Signature · 1 line
2.25.1