Skip to contentSkip to conversation

rcu: Use rcu_gp_kthread_wake() to wake up grace period kthreads

PATCH Superseded Pranith Kumar 2 participants 4 messages Jul 27, 2014 – Jul 28, 2014 RCU

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 rcu: Use rcu_gp_kthread_wake() to wake up grace period kthreads

The rcu_gp_kthread_wake() function checks for three conditions before waking up grace period kthreads:

*  Is the thread we are trying to wake up the current thread?
*  Are the gp_flags zero? (all threads wait on non-zero gp_flags condition)
*  Is there no thread created for this flavour, hence nothing to wake up?

If any one of these condition is true, we do not call wake_up().

It was found that there are quite a few avoidable wake ups both during idle time and under stress induced by rcutorture.

Idle:

Total:66000, unnecessary:66000, case1:61827, case2:66000, case3:0 Total:68000, unnecessary:68000, case1:63696, case2:68000, case3:0

rcutorture:

Total:254000, unnecessary:254000, case1:199913, case2:254000, case3:0 Total:256000, unnecessary:256000, case1:201784, case2:256000, case3:0

Here case{1-3} are the cases listed above. We can avoid these wake ups by using rcu_gp_kthread_wake() to conditionally wake up the grace period kthreads.

Hence this commit tries to avoid calling wake_up() whenever we can by using rcu_gp_kthread_wake() function.

---
 kernel/rcu/tree.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
kernel/rcu/tree.c +8 -2
index b63517c..36911ee 100644
@@ -1938,7 +1938,10 @@ static void rcu_report_qs_rsp(struct rcu_state *rsp, unsigned long flags)
 {
 	WARN_ON_ONCE(!rcu_gp_in_progress(rsp));
 	raw_spin_unlock_irqrestore(&rcu_get_root(rsp)->lock, flags);
-	wake_up(&rsp->gp_wq);  /* Memory barrier implied by wake_up() path. */
+	/* ->gp_flags is properly protected by locks, so a memory barrier
+	 * is not necessary here
+	 */
+	rcu_gp_kthread_wake(rsp);
 }
 
 /*
@@ -2516,7 +2519,10 @@ static void force_quiescent_state(struct rcu_state *rsp)
 	ACCESS_ONCE(rsp->gp_flags) =
 		ACCESS_ONCE(rsp->gp_flags) | RCU_GP_FLAG_FQS;
 	raw_spin_unlock_irqrestore(&rnp_old->lock, flags);
-	wake_up(&rsp->gp_wq);  /* Memory barrier implied by wake_up() path. */
+	/* ->gp_flags is properly protected by locks, so a memory barrier
+	 * is not necessary here
+	 */
+	rcu_gp_kthread_wake(rsp);
 }
 
 /*
Signature · 1 line
1.9.1
#1 lore raw

----- Original Message -----

Quoted 47 lines

From: "Pranith Kumar" <[email protected]>
To: "Paul E. McKenney" <[email protected]>, "Josh Triplett" <[email protected]>, "Steven Rostedt" <[email protected]>, "Mathieu Desnoyers" <[email protected]>, "Lai Jiangshan" <[email protected]>, "open list:READ-COPY UPDATE..." <[email protected]>
Sent: Sunday, July 27, 2014 7:37:29 PM
Subject: [PATCH 1/1] rcu: Use rcu_gp_kthread_wake() to wake up grace period kthreads

The rcu_gp_kthread_wake() function checks for three conditions before waking up
grace period kthreads:

*  Is the thread we are trying to wake up the current thread?
*  Are the gp_flags zero? (all threads wait on non-zero gp_flags condition)
*  Is there no thread created for this flavour, hence nothing to wake up?

If any one of these condition is true, we do not call wake_up().

It was found that there are quite a few avoidable wake ups both during idle time and under stress induced by rcutorture.

Idle:

Total:66000, unnecessary:66000, case1:61827, case2:66000, case3:0 Total:68000, unnecessary:68000, case1:63696, case2:68000, case3:0

rcutorture:

Total:254000, unnecessary:254000, case1:199913, case2:254000, case3:0 Total:256000, unnecessary:256000, case1:201784, case2:256000, case3:0

Here case{1-3} are the cases listed above. We can avoid these wake ups by using
rcu_gp_kthread_wake() to conditionally wake up the grace period kthreads.

Hence this commit tries to avoid calling wake_up() whenever we can by using rcu_gp_kthread_wake() function.

---
 kernel/rcu/tree.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
kernel/rcu/tree.c +0 -0
index b63517c..36911ee 100644

kernel/rcu/tree.c static void rcu_report_qs_rsp(struct rcu_state *rsp, · line 1943

Quoted 7 lines
kernel/rcu/tree.c +2 -1
@@ -1938,7 +1938,10 @@ static void rcu_report_qs_rsp(struct rcu_state *rsp,
 unsigned long flags)
 {
 	WARN_ON_ONCE(!rcu_gp_in_progress(rsp));
 	raw_spin_unlock_irqrestore(&rcu_get_root(rsp)->lock, flags);
-	wake_up(&rsp->gp_wq);  /* Memory barrier implied by wake_up() path. */
+	/* ->gp_flags is properly protected by locks, so a memory barrier
+	 * is not necessary here

Two point:

1- The format of this comment is odd, and should be:

/*
* Text...
*/

2- Since when can a memory barrier be replaced by a lock ? More explanation
appears to be needed on what this barrier matches exactly.

Thanks,

Mathieu

Quoted 22 lines
kernel/rcu/tree.c +2 -0
+	 */
+	rcu_gp_kthread_wake(rsp);
 }
 

/*

kernel/rcu/tree.c +4 -1
@@ -2516,7 +2519,10 @@ static void force_quiescent_state(struct rcu_state
 *rsp)
 	ACCESS_ONCE(rsp->gp_flags) =
 		ACCESS_ONCE(rsp->gp_flags) | RCU_GP_FLAG_FQS;
 	raw_spin_unlock_irqrestore(&rnp_old->lock, flags);
-	wake_up(&rsp->gp_wq);  /* Memory barrier implied by wake_up() path. */
+	/* ->gp_flags is properly protected by locks, so a memory barrier
+	 * is not necessary here
+	 */
+	rcu_gp_kthread_wake(rsp);
 }
 

/*
--
1.9.1

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com
#2 lore raw
Quoted 58 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 47 lines

From: "Pranith Kumar" <[email protected]>
To: "Paul E. McKenney" <[email protected]>, "Josh Triplett" <[email protected]>, "Steven Rostedt" <[email protected]>, "Mathieu Desnoyers" <[email protected]>, "Lai Jiangshan" <[email protected]>, "open list:READ-COPY UPDATE..." <[email protected]>
Sent: Sunday, July 27, 2014 7:37:29 PM
Subject: [PATCH 1/1] rcu: Use rcu_gp_kthread_wake() to wake up grace period kthreads

The rcu_gp_kthread_wake() function checks for three conditions before waking up
grace period kthreads:

*  Is the thread we are trying to wake up the current thread?
*  Are the gp_flags zero? (all threads wait on non-zero gp_flags condition)
*  Is there no thread created for this flavour, hence nothing to wake up?

If any one of these condition is true, we do not call wake_up().

It was found that there are quite a few avoidable wake ups both during idle time and under stress induced by rcutorture.

Idle:

Total:66000, unnecessary:66000, case1:61827, case2:66000, case3:0 Total:68000, unnecessary:68000, case1:63696, case2:68000, case3:0

rcutorture:

Total:254000, unnecessary:254000, case1:199913, case2:254000, case3:0 Total:256000, unnecessary:256000, case1:201784, case2:256000, case3:0

Here case{1-3} are the cases listed above. We can avoid these wake ups by using
rcu_gp_kthread_wake() to conditionally wake up the grace period kthreads.

Hence this commit tries to avoid calling wake_up() whenever we can by using rcu_gp_kthread_wake() function.

---
 kernel/rcu/tree.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
kernel/rcu/tree.c +0 -0
index b63517c..36911ee 100644

kernel/rcu/tree.c static void rcu_report_qs_rsp(struct rcu_state *rsp, · line 1943

Quoted 7 lines
kernel/rcu/tree.c +2 -1
@@ -1938,7 +1938,10 @@ static void rcu_report_qs_rsp(struct rcu_state *rsp,
 unsigned long flags)
 {
      WARN_ON_ONCE(!rcu_gp_in_progress(rsp));
      raw_spin_unlock_irqrestore(&rcu_get_root(rsp)->lock, flags);
-     wake_up(&rsp->gp_wq);  /* Memory barrier implied by wake_up() path. */
+     /* ->gp_flags is properly protected by locks, so a memory barrier
+      * is not necessary here

Two point:

Quoted 5 lines

1- The format of this comment is odd, and should be:

/*
* Text...
*/

OK, I will update it according to this format.

Quoted 3 lines

2- Since when can a memory barrier be replaced by a lock ? More explanation
appears to be needed on what this barrier matches exactly.

On re-reading I realize that this comment is very vague and introduces more doubts than it clears.

The context here is that in rcu_gp_kthread_wake() we are accessing ->gp_flags to determine whether we need to wake up the gp kthreads. We don't need a barrier here since we are accessing it using ACCESS_ONCE() and all other accesses are properly protected by using ACCESS_ONCE() and taking the root rcu_node lock.

So how about this:

/*
* ->gp_flags is being accessed using ACCESS_ONCE() because of
* which a memory barrier is not required here.
*/

Quoted 28 lines
Quoted 18 lines
kernel/rcu/tree.c +2 -0
+      */
+     rcu_gp_kthread_wake(rsp);
 }
 

/*

kernel/rcu/tree.c +4 -1
@@ -2516,7 +2519,10 @@ static void force_quiescent_state(struct rcu_state
 *rsp)
      ACCESS_ONCE(rsp->gp_flags) =
              ACCESS_ONCE(rsp->gp_flags) | RCU_GP_FLAG_FQS;
      raw_spin_unlock_irqrestore(&rnp_old->lock, flags);
-     wake_up(&rsp->gp_wq);  /* Memory barrier implied by wake_up() path. */
+     /* ->gp_flags is properly protected by locks, so a memory barrier
+      * is not necessary here
+      */
+     rcu_gp_kthread_wake(rsp);
 }
 
Quoted 5 lines

/*
--
1.9.1

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

Signature · 1 line
Pranith
#3 lore raw

----- Original Message -----

Quoted 98 lines

From: "Pranith Kumar" <[email protected]>
To: "Mathieu Desnoyers" <[email protected]>
Cc: "Paul E. McKenney" <[email protected]>, "Josh Triplett" <[email protected]>, "Steven Rostedt" <[email protected]>, "Lai Jiangshan" <[email protected]>, "open list:READ-COPY UPDATE..." <[email protected]>
Sent: Sunday, July 27, 2014 7:58:40 PM
Subject: Re: [PATCH 1/1] rcu: Use rcu_gp_kthread_wake() to wake up grace period kthreads

Quoted 67 lines · Mathieu Desnoyers

----- Original Message -----

Quoted 54 lines

From: "Pranith Kumar" <[email protected]>
To: "Paul E. McKenney" <[email protected]>, "Josh Triplett" <[email protected]>, "Steven Rostedt" <[email protected]>, "Mathieu Desnoyers" <[email protected]>, "Lai Jiangshan" <[email protected]>, "open list:READ-COPY UPDATE..." <[email protected]>
Sent: Sunday, July 27, 2014 7:37:29 PM
Subject: [PATCH 1/1] rcu: Use rcu_gp_kthread_wake() to wake up grace period kthreads

The rcu_gp_kthread_wake() function checks for three conditions before waking
up
grace period kthreads:

* Is the thread we are trying to wake up the current thread?
* Are the gp_flags zero? (all threads wait on non-zero gp_flags condition)
* Is there no thread created for this flavour, hence nothing to wake up?

If any one of these condition is true, we do not call wake_up().

It was found that there are quite a few avoidable wake ups both during idle
time and under stress induced by rcutorture.

Idle:

Total:66000, unnecessary:66000, case1:61827, case2:66000, case3:0 Total:68000, unnecessary:68000, case1:63696, case2:68000, case3:0

rcutorture:

Total:254000, unnecessary:254000, case1:199913, case2:254000, case3:0 Total:256000, unnecessary:256000, case1:201784, case2:256000, case3:0

Here case{1-3} are the cases listed above. We can avoid these wake ups by using
rcu_gp_kthread_wake() to conditionally wake up the grace period kthreads.

Hence this commit tries to avoid calling wake_up() whenever we can by using
rcu_gp_kthread_wake() function.

---
 kernel/rcu/tree.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
kernel/rcu/tree.c +0 -0
index b63517c..36911ee 100644

kernel/rcu/tree.c static void rcu_report_qs_rsp(struct rcu_state · line 1945

Quoted 9 lines
kernel/rcu/tree.c +2 -1
@@ -1938,7 +1938,10 @@ static void rcu_report_qs_rsp(struct rcu_state
 *rsp,
 unsigned long flags)
 {
      WARN_ON_ONCE(!rcu_gp_in_progress(rsp));
      raw_spin_unlock_irqrestore(&rcu_get_root(rsp)->lock, flags);
-     wake_up(&rsp->gp_wq);  /* Memory barrier implied by wake_up() path.
 */
+     /* ->gp_flags is properly protected by locks, so a memory barrier
+      * is not necessary here

Two point:

Quoted 5 lines

1- The format of this comment is odd, and should be:

/*
* Text...
*/

OK, I will update it according to this format.

Quoted 3 lines

2- Since when can a memory barrier be replaced by a lock ? More explanation
appears to be needed on what this barrier matches exactly.

On re-reading I realize that this comment is very vague and introduces more doubts than it clears.

The context here is that in rcu_gp_kthread_wake() we are accessing ->gp_flags to determine whether we need to wake up the gp kthreads. We don't need a barrier here since we are accessing it using ACCESS_ONCE() and all other accesses are properly protected by using ACCESS_ONCE() and taking the root rcu_node lock.

Quoted 7 lines

So how about this:

/*
* ->gp_flags is being accessed using ACCESS_ONCE() because of
* which a memory barrier is not required here.
*/

A memory barrier is typically not there to interact with a single variable and a single memory access. I'm concerned that this memory barrier might be ordering other things besides gp_flags.

Thanks,

Mathieu

Quoted 35 lines
Quoted 19 lines
Quoted 19 lines
kernel/rcu/tree.c +2 -0
+      */
+     rcu_gp_kthread_wake(rsp);

}

/*

kernel/rcu/tree.c +4 -1
@@ -2516,7 +2519,10 @@ static void force_quiescent_state(struct rcu_state
 *rsp)
      ACCESS_ONCE(rsp->gp_flags) =
              ACCESS_ONCE(rsp->gp_flags) | RCU_GP_FLAG_FQS;
      raw_spin_unlock_irqrestore(&rnp_old->lock, flags);
-     wake_up(&rsp->gp_wq);  /* Memory barrier implied by wake_up() path.
 */
+     /* ->gp_flags is properly protected by locks, so a memory barrier
+      * is not necessary here
+      */
+     rcu_gp_kthread_wake(rsp);
 }
Quoted 10 lines
Quoted 5 lines

/*
--
1.9.1

--
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com

--
Pranith

Signature · 3 lines
Mathieu Desnoyers
EfficiOS Inc.
http://www.efficios.com