Skip to contentSkip to conversation

rcu: Fix series of spurious RCU softirqs

PATCH In review Frederic Weisbecker 2 participants 16 messages Dec 10, 2010 – Dec 11, 2010 1 message not in the archive

lore
Share & tools
lore
#0 lore raw

v2 cover rcu: Fix series of spurious RCU softirqs

Hi,

Following Lai's idea.

An example of such series of spurious softirqs:

http://tglx.de/~fweisbec/trace_rcu_softirq.txt

In that example, the rcu softirq is raised at every tick during 20 secs (was perhaps more, but the trace snapshot happened during 20 secs). It happens randomly.

Ah and it survived several hours of rcutorture (with rcu cpu stall detection).

Thanks.

Frederic Weisbecker (2):
rcu: Stop chasing QS if another CPU did it for us
rcu: Keep gpnum and completed fields synchronized

 kernel/rcutree.c |   17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)
Signature · 1 line
1.7.3.2
#1 lore raw

PATCH v2 1/2 rcu: Stop chasing QS if another CPU did it for us

When a CPU is idle and others CPUs handled its extended quiescent state to complete grace periods on its behalf, it will catch up with completed grace periods numbers when it wakes up.

But at this point there might be no more grace period to complete, but still the woken CPU always keeps its stale qs_pending value and will then continue to chase quiescent states even if its not needed anymore.

This results in clusters of spurious softirqs until a new real grace period is started. Because if we continue to chase quiescent states but we have completed every grace periods, rcu_report_qs_rdp() is puzzled and makes that state run into infinite loops.

As suggested by Lai Jiangshan, just reset qs_pending if someone completed every grace periods on our behalf.

Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]>
---
 kernel/rcutree.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)
kernel/rcutree.c +8 -0
index ccdc04c..8c4ed60 100644
@@ -681,6 +681,14 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/* Remember that we saw this grace-period completion. */
 		rdp->completed = rnp->completed;
+
+		/*
+		 * If another CPU handled our extended quiescent states and
+		 * we have no more grace period to complete yet, then stop
+		 * chasing quiescent states.
+		 */
+		if (rdp->completed == rnp->gpnum)
+			rdp->qs_pending = 0;
 	}
 }
 
Signature · 1 line
1.7.3.2
#2 lore raw

Re: [PATCH 1/2] rcu: Stop chasing QS if another CPU did it for us

Quoted 10 lines · Frederic Weisbecker

When a CPU is idle and others CPUs handled its extended quiescent state to complete grace periods on its behalf, it will catch up with completed grace periods numbers when it wakes up.

But at this point there might be no more grace period to complete, but still the woken CPU always keeps its stale qs_pending value and will then continue to chase quiescent states even if its not needed anymore.

Quoted 8 lines

This results in clusters of spurious softirqs until a new real grace period is started. Because if we continue to chase quiescent states but we have completed every grace periods, rcu_report_qs_rdp() is puzzled and makes that state run into infinite loops.

As suggested by Lai Jiangshan, just reset qs_pending if someone completed every grace periods on our behalf.

Nice!!!

I have queued this patch, and followed it up with a patch that changes the condition to "rnp->qsmask & rdp->grpmask", which indicates that RCU needs a quiescent state from the CPU, and is valid regardless of how messed up the CPU is about which grace period is which.

I am making a similar change to the check in __note_new_gpnum().

Seem reasonable?

Thanx, Paul

Quoted 33 lines
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]>
---
 kernel/rcutree.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)
kernel/rcutree.c +8 -0
index ccdc04c..8c4ed60 100644
@@ -681,6 +681,14 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/* Remember that we saw this grace-period completion. */
 		rdp->completed = rnp->completed;
+
+		/*
+		 * If another CPU handled our extended quiescent states and
+		 * we have no more grace period to complete yet, then stop
+		 * chasing quiescent states.
+		 */
+		if (rdp->completed == rnp->gpnum)
+			rdp->qs_pending = 0;
 	}
 }
 
Signature · 1 line
1.7.3.2
#3 lore raw

Re: [PATCH 1/2] rcu: Stop chasing QS if another CPU did it for us

Quoted 22 lines · Paul E. McKenney
Quoted 10 lines · Frederic Weisbecker

When a CPU is idle and others CPUs handled its extended quiescent state to complete grace periods on its behalf, it will catch up with completed grace periods numbers when it wakes up.

But at this point there might be no more grace period to complete, but still the woken CPU always keeps its stale qs_pending value and will then continue to chase quiescent states even if its not needed anymore.

Quoted 8 lines

This results in clusters of spurious softirqs until a new real grace period is started. Because if we continue to chase quiescent states but we have completed every grace periods, rcu_report_qs_rdp() is puzzled and makes that state run into infinite loops.

As suggested by Lai Jiangshan, just reset qs_pending if someone completed every grace periods on our behalf.

Nice!!!

Quoted 8 lines

I have queued this patch, and followed it up with a patch that changes the condition to "rnp->qsmask & rdp->grpmask", which indicates that RCU needs a quiescent state from the CPU, and is valid regardless of how messed up the CPU is about which grace period is which.

I am making a similar change to the check in __note_new_gpnum().

Seem reasonable?

Look good yeah.

Thanks!

Quoted 36 lines

Thanx, Paul

Quoted 33 lines
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]>
---
 kernel/rcutree.c |    8 ++++++++
 1 files changed, 8 insertions(+), 0 deletions(-)
kernel/rcutree.c +8 -0
index ccdc04c..8c4ed60 100644
@@ -681,6 +681,14 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/* Remember that we saw this grace-period completion. */
 		rdp->completed = rnp->completed;
+
+		/*
+		 * If another CPU handled our extended quiescent states and
+		 * we have no more grace period to complete yet, then stop
+		 * chasing quiescent states.
+		 */
+		if (rdp->completed == rnp->gpnum)
+			rdp->qs_pending = 0;
 	}
 }
 
Signature · 1 line
1.7.3.2
#4 lore raw
In reply to Frederic Weisbecker

(Adding Lai in Cc, I forgot my script doesn't handle Suggested-by: tags :)

Quoted 27 lines · Frederic Weisbecker

Hi,

Following Lai's idea.

An example of such series of spurious softirqs:

http://tglx.de/~fweisbec/trace_rcu_softirq.txt

In that example, the rcu softirq is raised at every tick during 20 secs (was perhaps more, but the trace snapshot happened during 20 secs). It happens randomly.

Ah and it survived several hours of rcutorture (with rcu cpu stall detection).

Thanks.

Frederic Weisbecker (2):
rcu: Stop chasing QS if another CPU did it for us
rcu: Keep gpnum and completed fields synchronized

 kernel/rcutree.c |   17 +++++++++++++++++
 1 files changed, 17 insertions(+), 0 deletions(-)
Signature · 1 line
1.7.3.2

A message that is not in the archive (sent off-list or not yet received): [email protected]

#6 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And I clearly need to make RCU defend itself against the scenario where a CPU stays in dyntick-idle mode long enough for the grace-period number to wrap halfway around its range of possible values. Not a problem at the moment, and never will be for 64-bit systems, but...

I will fix that up.

Thanx, Paul

Quoted 34 lines
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]
---
 kernel/rcutree.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)
kernel/rcutree.c +9 -0
index 8c4ed60..2e16da3 100644
@@ -683,6 +683,15 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 		rdp->completed = rnp->completed;
 
 		/*
+		 * If we were in an extended quiescent state, we may have
+		 * missed some grace periods that others CPUs took care on
+		 * our behalf. Catch up with this state to avoid noting
+		 * spurious new grace periods.
+		 */
+		if (rdp->completed > rdp->gpnum)
+			rdp->gpnum = rdp->completed;
+
+		/*
 		 * If another CPU handled our extended quiescent states and
 		 * we have no more grace period to complete yet, then stop
 		 * chasing quiescent states.
Signature · 1 line
1.7.3.2
#7 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 21 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Quoted 6 lines

Also good, queued!

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And here is the follow-up patch, FWIW.

Thanx, Paul

------------------------------------------------------------------------

commit d864b245030645e3465b3bd7e253b7ccf76e9d35

Author: Paul E. McKenney <[email protected]>
Date:   Fri Dec 10 15:02:47 2010 -0800

rcu: fine-tune grace-period begin/end checks

Use the CPU's bit in rnp->qsmask to determine whether or not the CPU should try to report a quiescent state. Handle overflow in the check for rdp->gpnum having fallen behind.

kernel/rcutree.c +15 -18
index f8e4ee7..6103017 100644
@@ -618,20 +618,16 @@ static void __note_new_gpnum(struct rcu_state *rsp, struct rcu_node *rnp, struct
 {
 	if (rdp->gpnum != rnp->gpnum) {
 		/*
-		 * Because RCU checks for the prior grace period ending
-		 * before checking for a new grace period starting, it
-		 * is possible for rdp->gpnum to be set to the old grace
-		 * period and rdp->completed to be set to the new grace
-		 * period.  So don't bother checking for a quiescent state
-		 * for the rnp->gpnum grace period unless it really is
-		 * waiting for this CPU.
+		 * If the current grace period is waiting for this CPU,
+		 * set up to detect a quiescent state, otherwise don't
+		 * go looking for one.
 		 */
-		if (rdp->completed != rnp->gpnum) {
+		rdp->gpnum = rnp->gpnum;
+		if (rnp->qsmask & rdp->grpmask) {
 			rdp->qs_pending = 1;
 			rdp->passed_quiesc = 0;
-		}
-
-		rdp->gpnum = rnp->gpnum;
+		} else
+			rdp->qs_pending = 0;
 	}
 }
 
@@ -693,19 +689,20 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/*
 		 * If we were in an extended quiescent state, we may have
-		 * missed some grace periods that others CPUs took care on
+		 * missed some grace periods that others CPUs handled on
 		 * our behalf. Catch up with this state to avoid noting
-		 * spurious new grace periods.
+		 * spurious new grace periods.  If another grace period
+		 * has started, then rnp->gpnum will have advanced, so
+		 * we will detect this later on.
 		 */
-		if (rdp->completed > rdp->gpnum)
+		if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))
 			rdp->gpnum = rdp->completed;
 
 		/*
-		 * If another CPU handled our extended quiescent states and
-		 * we have no more grace period to complete yet, then stop
-		 * chasing quiescent states.
+		 * If RCU does not need a quiescent state from this CPU,
+		 * then make sure that this CPU doesn't go looking for one.
 		 */
-		if (rdp->completed == rnp->gpnum)
+		if (rnp->qsmask & rdp->grpmask)
 			rdp->qs_pending = 0;
 	}
 }
#8 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 24 lines · Paul E. McKenney
Quoted 23 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

Quoted 8 lines
Quoted 4 lines

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And here is the follow-up patch, FWIW.

Thanx, Paul

Hmm, it doesn't apply on top of my two patches. It seems you have kept my two previous patches, which makes it fail as it lacks them as a base.

Did you intend to keep them? I hope they are quite useless now, otherwise it means there is other cases I forgot.

Thanks.

Quoted 75 lines

------------------------------------------------------------------------

commit d864b245030645e3465b3bd7e253b7ccf76e9d35

Author: Paul E. McKenney <[email protected]>
Date:   Fri Dec 10 15:02:47 2010 -0800

rcu: fine-tune grace-period begin/end checks

Use the CPU's bit in rnp->qsmask to determine whether or not the CPU should try to report a quiescent state. Handle overflow in the check for rdp->gpnum having fallen behind.

kernel/rcutree.c +15 -18
index f8e4ee7..6103017 100644
@@ -618,20 +618,16 @@ static void __note_new_gpnum(struct rcu_state *rsp, struct rcu_node *rnp, struct
 {
 	if (rdp->gpnum != rnp->gpnum) {
 		/*
-		 * Because RCU checks for the prior grace period ending
-		 * before checking for a new grace period starting, it
-		 * is possible for rdp->gpnum to be set to the old grace
-		 * period and rdp->completed to be set to the new grace
-		 * period.  So don't bother checking for a quiescent state
-		 * for the rnp->gpnum grace period unless it really is
-		 * waiting for this CPU.
+		 * If the current grace period is waiting for this CPU,
+		 * set up to detect a quiescent state, otherwise don't
+		 * go looking for one.
 		 */
-		if (rdp->completed != rnp->gpnum) {
+		rdp->gpnum = rnp->gpnum;
+		if (rnp->qsmask & rdp->grpmask) {
 			rdp->qs_pending = 1;
 			rdp->passed_quiesc = 0;
-		}
-
-		rdp->gpnum = rnp->gpnum;
+		} else
+			rdp->qs_pending = 0;
 	}
 }
 
@@ -693,19 +689,20 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/*
 		 * If we were in an extended quiescent state, we may have
-		 * missed some grace periods that others CPUs took care on
+		 * missed some grace periods that others CPUs handled on
 		 * our behalf. Catch up with this state to avoid noting
-		 * spurious new grace periods.
+		 * spurious new grace periods.  If another grace period
+		 * has started, then rnp->gpnum will have advanced, so
+		 * we will detect this later on.
 		 */
-		if (rdp->completed > rdp->gpnum)
+		if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))
 			rdp->gpnum = rdp->completed;
 
 		/*
-		 * If another CPU handled our extended quiescent states and
-		 * we have no more grace period to complete yet, then stop
-		 * chasing quiescent states.
+		 * If RCU does not need a quiescent state from this CPU,
+		 * then make sure that this CPU doesn't go looking for one.
 		 */
-		if (rdp->completed == rnp->gpnum)
+		if (rnp->qsmask & rdp->grpmask)
 			rdp->qs_pending = 0;
 	}
 }
#9 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 32 lines · Frederic Weisbecker
Quoted 31 lines · Paul E. McKenney
Quoted 21 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Quoted 6 lines

Also good, queued!

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And here is the follow-up patch, FWIW.

Quoted 8 lines
Quoted 1 line

Thanx, Paul

Hmm, it doesn't apply on top of my two patches. It seems you have kept my two previous patches, which makes it fail as it lacks them as a base.

Did you intend to keep them? I hope they are quite useless now, otherwise it means there is other cases I forgot.

One is indeed useless, while the other is useful in combinations of dyntick-idle and force_quiescent_state(). I rebased your earlier two out and reworked mine, please see below. Work better?

Thanx, Paul

------------------------------------------------------------------------

commit c808bedd1b1d7c720546a6682fca44c66703af4e

Author: Paul E. McKenney <[email protected]>
Date:   Fri Dec 10 15:02:47 2010 -0800

rcu: fine-tune grace-period begin/end checks

Use the CPU's bit in rnp->qsmask to determine whether or not the CPU should try to report a quiescent state. Handle overflow in the check for rdp->gpnum having fallen behind.

kernel/rcutree.c +18 -9
index 368be76..530cdcd 100644
@@ -616,9 +616,17 @@ static void __init check_cpu_stall_init(void)
 static void __note_new_gpnum(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_data *rdp)
 {
 	if (rdp->gpnum != rnp->gpnum) {
-		rdp->qs_pending = 1;
-		rdp->passed_quiesc = 0;
+		/*
+		 * If the current grace period is waiting for this CPU,
+		 * set up to detect a quiescent state, otherwise don't
+		 * go looking for one.
+		 */
 		rdp->gpnum = rnp->gpnum;
+		if (rnp->qsmask & rdp->grpmask) {
+			rdp->qs_pending = 1;
+			rdp->passed_quiesc = 0;
+		} else
+			rdp->qs_pending = 0;
 	}
 }
 
@@ -680,19 +688,20 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/*
 		 * If we were in an extended quiescent state, we may have
-		 * missed some grace periods that others CPUs took care on
+		 * missed some grace periods that others CPUs handled on
 		 * our behalf. Catch up with this state to avoid noting
-		 * spurious new grace periods.
+		 * spurious new grace periods.  If another grace period
+		 * has started, then rnp->gpnum will have advanced, so
+		 * we will detect this later on.
 		 */
-		if (rdp->completed > rdp->gpnum)
+		if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))
 			rdp->gpnum = rdp->completed;
 
 		/*
-		 * If another CPU handled our extended quiescent states and
-		 * we have no more grace period to complete yet, then stop
-		 * chasing quiescent states.
+		 * If RCU does not need a quiescent state from this CPU,
+		 * then make sure that this CPU doesn't go looking for one.
 		 */
-		if (rdp->completed == rnp->gpnum)
+		if (rnp->qsmask & rdp->grpmask)
 			rdp->qs_pending = 0;
 	}
 }
#10 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 39 lines · Paul E. McKenney
Quoted 38 lines · Frederic Weisbecker
Quoted 24 lines · Paul E. McKenney
Quoted 23 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

Quoted 8 lines
Quoted 4 lines

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And here is the follow-up patch, FWIW.

Thanx, Paul

Hmm, it doesn't apply on top of my two patches. It seems you have kept my two previous patches, which makes it fail as it lacks them as a base.

Quoted 5 lines
Quoted 2 lines

Did you intend to keep them? I hope they are quite useless now, otherwise it means there is other cases I forgot.

One is indeed useless, while the other is useful in combinations of dyntick-idle and force_quiescent_state().

I don't see how.

Before we call __note_new_gpnum(), we always have the opportunity to resync gpnum and completed as __rcu_process_gp_end() is called before.

Am I missing something?

Thanks.

Quoted 71 lines

I rebased your earlier two
out and reworked mine, please see below. Work better?

Thanx, Paul

------------------------------------------------------------------------

commit c808bedd1b1d7c720546a6682fca44c66703af4e

Author: Paul E. McKenney <[email protected]>
Date:   Fri Dec 10 15:02:47 2010 -0800

rcu: fine-tune grace-period begin/end checks

Use the CPU's bit in rnp->qsmask to determine whether or not the CPU should try to report a quiescent state. Handle overflow in the check for rdp->gpnum having fallen behind.

kernel/rcutree.c +18 -9
index 368be76..530cdcd 100644
@@ -616,9 +616,17 @@ static void __init check_cpu_stall_init(void)
 static void __note_new_gpnum(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_data *rdp)
 {
 	if (rdp->gpnum != rnp->gpnum) {
-		rdp->qs_pending = 1;
-		rdp->passed_quiesc = 0;
+		/*
+		 * If the current grace period is waiting for this CPU,
+		 * set up to detect a quiescent state, otherwise don't
+		 * go looking for one.
+		 */
 		rdp->gpnum = rnp->gpnum;
+		if (rnp->qsmask & rdp->grpmask) {
+			rdp->qs_pending = 1;
+			rdp->passed_quiesc = 0;
+		} else
+			rdp->qs_pending = 0;
 	}
 }
 
@@ -680,19 +688,20 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/*
 		 * If we were in an extended quiescent state, we may have
-		 * missed some grace periods that others CPUs took care on
+		 * missed some grace periods that others CPUs handled on
 		 * our behalf. Catch up with this state to avoid noting
-		 * spurious new grace periods.
+		 * spurious new grace periods.  If another grace period
+		 * has started, then rnp->gpnum will have advanced, so
+		 * we will detect this later on.
 		 */
-		if (rdp->completed > rdp->gpnum)
+		if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))
 			rdp->gpnum = rdp->completed;
 
 		/*
-		 * If another CPU handled our extended quiescent states and
-		 * we have no more grace period to complete yet, then stop
-		 * chasing quiescent states.
+		 * If RCU does not need a quiescent state from this CPU,
+		 * then make sure that this CPU doesn't go looking for one.
 		 */
-		if (rdp->completed == rnp->gpnum)
+		if (rnp->qsmask & rdp->grpmask)
 			rdp->qs_pending = 0;
 	}
 }
#11 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 46 lines · Frederic Weisbecker
Quoted 39 lines · Paul E. McKenney
Quoted 38 lines · Frederic Weisbecker
Quoted 24 lines · Paul E. McKenney
Quoted 23 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

Quoted 8 lines
Quoted 4 lines

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And here is the follow-up patch, FWIW.

Thanx, Paul

Hmm, it doesn't apply on top of my two patches. It seems you have kept my two previous patches, which makes it fail as it lacks them as a base.

Quoted 5 lines
Quoted 2 lines

Did you intend to keep them? I hope they are quite useless now, otherwise it means there is other cases I forgot.

One is indeed useless, while the other is useful in combinations of dyntick-idle and force_quiescent_state().

Quoted 7 lines

I don't see how.

Before we call __note_new_gpnum(), we always have the opportunity to resync gpnum and completed as __rcu_process_gp_end() is called before.

Am I missing something?

If the CPU is already aware of the end of the previous grace period, then __rcu_process_gp_end() will return without doing anything. But if force_quiescent_state() already took care of this CPU, there is no point in its looking for another quiescent state. This can happen as follows:

o CPU 0 notes the end of the previous grace period and then
enters dyntick-idle mode.

o CPU 2 enters a very long RCU read-side critical section.

o CPU 1 starts a new grace period.

o CPU 0 does not check in because it is in dyntick-idle mode.

o CPU 1 eventually calls force_quiescent_state() a few times,
and sees that CPU 0 is in dyntick-idle mode, so tells RCU that CPU 0 is in an extended quiescent state. But the grace period cannot end because CPU 2 is still in its RCU read-side critical section.

o CPU 0 comes out of dyntick-idle mode, and sees the new
grace period. The old code would nevertheless look for a quiescent state, and the new code would avoid doing so.

Unless I am missing something, of course...

Thanx, Paul

Quoted 73 lines

Thanks.

Quoted 71 lines

I rebased your earlier two
out and reworked mine, please see below. Work better?

Thanx, Paul

------------------------------------------------------------------------

commit c808bedd1b1d7c720546a6682fca44c66703af4e

Author: Paul E. McKenney <[email protected]>
Date:   Fri Dec 10 15:02:47 2010 -0800

rcu: fine-tune grace-period begin/end checks

Use the CPU's bit in rnp->qsmask to determine whether or not the CPU should try to report a quiescent state. Handle overflow in the check for rdp->gpnum having fallen behind.

kernel/rcutree.c +18 -9
index 368be76..530cdcd 100644
@@ -616,9 +616,17 @@ static void __init check_cpu_stall_init(void)
 static void __note_new_gpnum(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_data *rdp)
 {
 	if (rdp->gpnum != rnp->gpnum) {
-		rdp->qs_pending = 1;
-		rdp->passed_quiesc = 0;
+		/*
+		 * If the current grace period is waiting for this CPU,
+		 * set up to detect a quiescent state, otherwise don't
+		 * go looking for one.
+		 */
 		rdp->gpnum = rnp->gpnum;
+		if (rnp->qsmask & rdp->grpmask) {
+			rdp->qs_pending = 1;
+			rdp->passed_quiesc = 0;
+		} else
+			rdp->qs_pending = 0;
 	}
 }
 
@@ -680,19 +688,20 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/*
 		 * If we were in an extended quiescent state, we may have
-		 * missed some grace periods that others CPUs took care on
+		 * missed some grace periods that others CPUs handled on
 		 * our behalf. Catch up with this state to avoid noting
-		 * spurious new grace periods.
+		 * spurious new grace periods.  If another grace period
+		 * has started, then rnp->gpnum will have advanced, so
+		 * we will detect this later on.
 		 */
-		if (rdp->completed > rdp->gpnum)
+		if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))
 			rdp->gpnum = rdp->completed;
 
 		/*
-		 * If another CPU handled our extended quiescent states and
-		 * we have no more grace period to complete yet, then stop
-		 * chasing quiescent states.
+		 * If RCU does not need a quiescent state from this CPU,
+		 * then make sure that this CPU doesn't go looking for one.
 		 */
-		if (rdp->completed == rnp->gpnum)
+		if (rnp->qsmask & rdp->grpmask)
 			rdp->qs_pending = 0;
 	}
 }
#12 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 75 lines · Paul E. McKenney
Quoted 46 lines · Frederic Weisbecker
Quoted 39 lines · Paul E. McKenney
Quoted 38 lines · Frederic Weisbecker
Quoted 24 lines · Paul E. McKenney
Quoted 23 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

Quoted 8 lines
Quoted 4 lines

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And here is the follow-up patch, FWIW.

Thanx, Paul

Hmm, it doesn't apply on top of my two patches. It seems you have kept my two previous patches, which makes it fail as it lacks them as a base.

Quoted 5 lines
Quoted 2 lines

Did you intend to keep them? I hope they are quite useless now, otherwise it means there is other cases I forgot.

One is indeed useless, while the other is useful in combinations of dyntick-idle and force_quiescent_state().

Quoted 7 lines

I don't see how.

Before we call __note_new_gpnum(), we always have the opportunity to resync gpnum and completed as __rcu_process_gp_end() is called before.

Am I missing something?

If the CPU is already aware of the end of the previous grace period, then __rcu_process_gp_end() will return without doing anything. But if force_quiescent_state() already took care of this CPU, there is no point in its looking for another quiescent state. This can happen as follows:

o CPU 0 notes the end of the previous grace period and then
enters dyntick-idle mode.

o CPU 2 enters a very long RCU read-side critical section.

o CPU 1 starts a new grace period.

o CPU 0 does not check in because it is in dyntick-idle mode.

o CPU 1 eventually calls force_quiescent_state() a few times,
and sees that CPU 0 is in dyntick-idle mode, so tells RCU that CPU 0 is in an extended quiescent state. But the grace period cannot end because CPU 2 is still in its RCU read-side critical section.

Quoted 7 lines

o CPU 0 comes out of dyntick-idle mode, and sees the new
grace period. The old code would nevertheless look for a quiescent state, and the new code would avoid doing so.

Unless I am missing something, of course...

Thanx, Paul

Aah, so in your scenario, CPU 0, 1 et 2 are the same node (rnp), we have not updated rnp->completed because we still wait for CPU 2.

Then __rcu_process_gp_end() won't increase the gpnum either because rnp->completed is still equal to rdp->completed.

And later on we call note_new_gpnum() that thinks it has a new gp to handle but it's wrong.

Hence the need to look at the mask level there.

This makes all sense!

Thanks!

Quoted 74 lines
Quoted 74 lines

Thanks.

Quoted 71 lines

I rebased your earlier two
out and reworked mine, please see below. Work better?

Thanx, Paul

------------------------------------------------------------------------

commit c808bedd1b1d7c720546a6682fca44c66703af4e

Author: Paul E. McKenney <[email protected]>
Date:   Fri Dec 10 15:02:47 2010 -0800

rcu: fine-tune grace-period begin/end checks

Use the CPU's bit in rnp->qsmask to determine whether or not the CPU should try to report a quiescent state. Handle overflow in the check for rdp->gpnum having fallen behind.

kernel/rcutree.c +18 -9
index 368be76..530cdcd 100644
@@ -616,9 +616,17 @@ static void __init check_cpu_stall_init(void)
 static void __note_new_gpnum(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_data *rdp)
 {
 	if (rdp->gpnum != rnp->gpnum) {
-		rdp->qs_pending = 1;
-		rdp->passed_quiesc = 0;
+		/*
+		 * If the current grace period is waiting for this CPU,
+		 * set up to detect a quiescent state, otherwise don't
+		 * go looking for one.
+		 */
 		rdp->gpnum = rnp->gpnum;
+		if (rnp->qsmask & rdp->grpmask) {
+			rdp->qs_pending = 1;
+			rdp->passed_quiesc = 0;
+		} else
+			rdp->qs_pending = 0;
 	}
 }
 
@@ -680,19 +688,20 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/*
 		 * If we were in an extended quiescent state, we may have
-		 * missed some grace periods that others CPUs took care on
+		 * missed some grace periods that others CPUs handled on
 		 * our behalf. Catch up with this state to avoid noting
-		 * spurious new grace periods.
+		 * spurious new grace periods.  If another grace period
+		 * has started, then rnp->gpnum will have advanced, so
+		 * we will detect this later on.
 		 */
-		if (rdp->completed > rdp->gpnum)
+		if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))
 			rdp->gpnum = rdp->completed;
 
 		/*
-		 * If another CPU handled our extended quiescent states and
-		 * we have no more grace period to complete yet, then stop
-		 * chasing quiescent states.
+		 * If RCU does not need a quiescent state from this CPU,
+		 * then make sure that this CPU doesn't go looking for one.
 		 */
-		if (rdp->completed == rnp->gpnum)
+		if (rnp->qsmask & rdp->grpmask)
 			rdp->qs_pending = 0;
 	}
 }
#13 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 87 lines · Frederic Weisbecker
Quoted 75 lines · Paul E. McKenney
Quoted 46 lines · Frederic Weisbecker
Quoted 39 lines · Paul E. McKenney
Quoted 38 lines · Frederic Weisbecker
Quoted 24 lines · Paul E. McKenney
Quoted 23 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

Quoted 8 lines
Quoted 4 lines

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And here is the follow-up patch, FWIW.

Thanx, Paul

Hmm, it doesn't apply on top of my two patches. It seems you have kept my two previous patches, which makes it fail as it lacks them as a base.

Quoted 5 lines
Quoted 2 lines

Did you intend to keep them? I hope they are quite useless now, otherwise it means there is other cases I forgot.

One is indeed useless, while the other is useful in combinations of dyntick-idle and force_quiescent_state().

Quoted 7 lines

I don't see how.

Before we call __note_new_gpnum(), we always have the opportunity to resync gpnum and completed as __rcu_process_gp_end() is called before.

Am I missing something?

If the CPU is already aware of the end of the previous grace period, then __rcu_process_gp_end() will return without doing anything. But if force_quiescent_state() already took care of this CPU, there is no point in its looking for another quiescent state. This can happen as follows:

o CPU 0 notes the end of the previous grace period and then
enters dyntick-idle mode.

o CPU 2 enters a very long RCU read-side critical section.

o CPU 1 starts a new grace period.

o CPU 0 does not check in because it is in dyntick-idle mode.

o CPU 1 eventually calls force_quiescent_state() a few times,
and sees that CPU 0 is in dyntick-idle mode, so tells RCU that CPU 0 is in an extended quiescent state. But the grace period cannot end because CPU 2 is still in its RCU read-side critical section.

Quoted 7 lines

o CPU 0 comes out of dyntick-idle mode, and sees the new
grace period. The old code would nevertheless look for a quiescent state, and the new code would avoid doing so.

Unless I am missing something, of course...

Thanx, Paul

Aah, so in your scenario, CPU 0, 1 et 2 are the same node (rnp), we have not updated rnp->completed because we still wait for CPU 2.

Quoted 7 lines

Then __rcu_process_gp_end() won't increase the gpnum either because rnp->completed is still equal to rdp->completed.

And later on we call note_new_gpnum() that thinks it has a new gp to handle but it's wrong.

Hence the need to look at the mask level there.

CPUs 0, 1, et 2 are not necessarily on the same node, but other than that, you have it exactly. The trick is that force_quiescent_state() takes global action, so CPU 1 does not need to be on the same node as CPU 0. Furthermore, an RCU read-side critical section anywhere in the system will prevent any subsequent grace period from completing, so CPU 2 can be on yet another node.

Quoted 1 line

This makes all sense!

Thank you for the confirmation, will try testing it out more thoroughly.

Thanx, Paul

Quoted 76 lines

Thanks!

Quoted 74 lines
Quoted 74 lines

Thanks.

Quoted 71 lines

I rebased your earlier two
out and reworked mine, please see below. Work better?

Thanx, Paul

------------------------------------------------------------------------

commit c808bedd1b1d7c720546a6682fca44c66703af4e

Author: Paul E. McKenney <[email protected]>
Date:   Fri Dec 10 15:02:47 2010 -0800

rcu: fine-tune grace-period begin/end checks

Use the CPU's bit in rnp->qsmask to determine whether or not the CPU should try to report a quiescent state. Handle overflow in the check for rdp->gpnum having fallen behind.

kernel/rcutree.c +18 -9
index 368be76..530cdcd 100644
@@ -616,9 +616,17 @@ static void __init check_cpu_stall_init(void)
 static void __note_new_gpnum(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_data *rdp)
 {
 	if (rdp->gpnum != rnp->gpnum) {
-		rdp->qs_pending = 1;
-		rdp->passed_quiesc = 0;
+		/*
+		 * If the current grace period is waiting for this CPU,
+		 * set up to detect a quiescent state, otherwise don't
+		 * go looking for one.
+		 */
 		rdp->gpnum = rnp->gpnum;
+		if (rnp->qsmask & rdp->grpmask) {
+			rdp->qs_pending = 1;
+			rdp->passed_quiesc = 0;
+		} else
+			rdp->qs_pending = 0;
 	}
 }
 
@@ -680,19 +688,20 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 
 		/*
 		 * If we were in an extended quiescent state, we may have
-		 * missed some grace periods that others CPUs took care on
+		 * missed some grace periods that others CPUs handled on
 		 * our behalf. Catch up with this state to avoid noting
-		 * spurious new grace periods.
+		 * spurious new grace periods.  If another grace period
+		 * has started, then rnp->gpnum will have advanced, so
+		 * we will detect this later on.
 		 */
-		if (rdp->completed > rdp->gpnum)
+		if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))
 			rdp->gpnum = rdp->completed;
 
 		/*
-		 * If another CPU handled our extended quiescent states and
-		 * we have no more grace period to complete yet, then stop
-		 * chasing quiescent states.
+		 * If RCU does not need a quiescent state from this CPU,
+		 * then make sure that this CPU doesn't go looking for one.
 		 */
-		if (rdp->completed == rnp->gpnum)
+		if (rnp->qsmask & rdp->grpmask)
 			rdp->qs_pending = 0;
 	}
 }
#14 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

In reply to Paul E. McKenney
Quoted 26 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

Quoted 8 lines
	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And I clearly need to make RCU defend itself against the scenario where a CPU stays in dyntick-idle mode long enough for the grace-period number to wrap halfway around its range of possible values. Not a problem at the moment, and never will be for 64-bit systems, but...

I will fix that up.

Oh you're right of course. I did not think about possible overflows.

Now looking at ULONG_CMP_LT() definition, if it wraps more than halfways we are screwed anyway. I suspect it won't ever happen, but it can. Perhaps we need some watchguard code in note_new_gpnum() to fixup that corner case.

Quoted 37 lines

Thanx, Paul

Quoted 34 lines
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]
---
 kernel/rcutree.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)
kernel/rcutree.c +9 -0
index 8c4ed60..2e16da3 100644
@@ -683,6 +683,15 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 		rdp->completed = rnp->completed;
 
 		/*
+		 * If we were in an extended quiescent state, we may have
+		 * missed some grace periods that others CPUs took care on
+		 * our behalf. Catch up with this state to avoid noting
+		 * spurious new grace periods.
+		 */
+		if (rdp->completed > rdp->gpnum)
+			rdp->gpnum = rdp->completed;
+
+		/*
 		 * If another CPU handled our extended quiescent states and
 		 * we have no more grace period to complete yet, then stop
 		 * chasing quiescent states.
Signature · 1 line
1.7.3.2
#15 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 34 lines · Frederic Weisbecker
Quoted 33 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And I clearly need to make RCU defend itself against the scenario where a CPU stays in dyntick-idle mode long enough for the grace-period number to wrap halfway around its range of possible values. Not a problem at the moment, and never will be for 64-bit systems, but...

Quoted 7 lines
Quoted 1 line

I will fix that up.

Oh you're right of course. I did not think about possible overflows.

Now looking at ULONG_CMP_LT() definition, if it wraps more than halfways we are screwed anyway. I suspect it won't ever happen, but it can. Perhaps we need some watchguard code in note_new_gpnum() to fixup that corner case.

We still have to guard against a full wrap, though there are lots of other things that break if you stay in dyntick-idle mode that long. My plan is to have a counter in the rcu_state structure that cycles through the CPUs. Check the current CPU at the start of each grace period. If a given CPU is more than one-quarter of the way behind, force it to wake up and catch up. This gets easier to do given my in-progress changes to convert from softirq to kthread, so I will combine it with those changes.

So I will cover this.

Thanx, Paul

Quoted 39 lines
Quoted 18 lines
Quoted 18 lines
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]
---
 kernel/rcutree.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)
kernel/rcutree.c +0 -0
index 8c4ed60..2e16da3 100644
@@ -683,6 +683,15 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 		rdp->completed = rnp->completed;
 
Quoted 16 lines
Quoted 16 lines
kernel/rcutree.c +9 -0
 		/*
+		 * If we were in an extended quiescent state, we may have
+		 * missed some grace periods that others CPUs took care on
+		 * our behalf. Catch up with this state to avoid noting
+		 * spurious new grace periods.
+		 */
+		if (rdp->completed > rdp->gpnum)
+			rdp->gpnum = rdp->completed;
+
+		/*
 		 * If another CPU handled our extended quiescent states and
 		 * we have no more grace period to complete yet, then stop
 		 * chasing quiescent states.
Signature · 1 line
1.7.3.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/

#16 lore raw

Re: [PATCH 2/2] rcu: Keep gpnum and completed fields synchronized

Quoted 51 lines · Paul E. McKenney
Quoted 34 lines · Frederic Weisbecker
Quoted 33 lines · Paul E. McKenney
Quoted 15 lines · Frederic Weisbecker

When a CPU that was in an extended quiescent state wakes up and catches up with grace periods that remote CPUs completed on its behalf, we update the completed field but not the gpnum that keeps a stale value of a backward grace period ID.

Later, note_new_gpnum() will interpret the shift between the local CPU and the node grace period ID as some new grace period to handle and will then start to hunt quiescent state.

But if every grace periods have already been completed, this interpretation becomes broken. And we'll be stuck in clusters of spurious softirqs because rcu_report_qs_rdp() will make this broken state run into infinite loop.

Quoted 4 lines

The solution, as suggested by Lai Jiangshan, is to ensure that the gpnum and completed fields are well synchronized when we catch up with completed grace periods on their behalf by other cpus. This way we won't start noting spurious new grace periods.

Also good, queued!

One issue -- this approach is vulnerable to overflow. I therefore followed up with a patch that changes the condition to

	if (ULONG_CMP_LT(rdp->gpnum, rdp->completed))

And I clearly need to make RCU defend itself against the scenario where a CPU stays in dyntick-idle mode long enough for the grace-period number to wrap halfway around its range of possible values. Not a problem at the moment, and never will be for 64-bit systems, but...

Quoted 7 lines
Quoted 1 line

I will fix that up.

Oh you're right of course. I did not think about possible overflows.

Now looking at ULONG_CMP_LT() definition, if it wraps more than halfways we are screwed anyway. I suspect it won't ever happen, but it can. Perhaps we need some watchguard code in note_new_gpnum() to fixup that corner case.

We still have to guard against a full wrap, though there are lots of other things that break if you stay in dyntick-idle mode that long. My plan is to have a counter in the rcu_state structure that cycles through the CPUs. Check the current CPU at the start of each grace period. If a given CPU is more than one-quarter of the way behind, force it to wake up and catch up. This gets easier to do given my in-progress changes to convert from softirq to kthread, so I will combine it with those changes.

Quoted 1 line

So I will cover this.

Fine!

Thanks.

Quoted 42 lines

Thanx, Paul

Quoted 39 lines
Quoted 18 lines
Quoted 18 lines
Cc Paul E. McKenney <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Thomas Gleixner <[email protected]>
Cc Peter Zijlstra <[email protected]>
Cc Steven Rostedt <[email protected]
---
 kernel/rcutree.c |    9 +++++++++
 1 files changed, 9 insertions(+), 0 deletions(-)
kernel/rcutree.c +0 -0
index 8c4ed60..2e16da3 100644
@@ -683,6 +683,15 @@ __rcu_process_gp_end(struct rcu_state *rsp, struct rcu_node *rnp, struct rcu_dat
 		rdp->completed = rnp->completed;
 
Quoted 16 lines
Quoted 16 lines
kernel/rcutree.c +9 -0
 		/*
+		 * If we were in an extended quiescent state, we may have
+		 * missed some grace periods that others CPUs took care on
+		 * our behalf. Catch up with this state to avoid noting
+		 * spurious new grace periods.
+		 */
+		if (rdp->completed > rdp->gpnum)
+			rdp->gpnum = rdp->completed;
+
+		/*
 		 * If another CPU handled our extended quiescent states and
 		 * we have no more grace period to complete yet, then stop
 		 * chasing quiescent states.
Signature · 1 line
1.7.3.2

--
To unsubscribe from this list: send the line "unsubscribe linux-kernel" in the body of a message to [email protected]
More majordomo info at http://vger.kernel.org/majordomo-info.html
Please read the FAQ at http://www.tux.org/lkml/