Skip to contentSkip to conversation

RCU: introduce rcu_soon_pending() interface

PATCH In review Heiko Carstens 4 participants 14 messages Apr 24, 2006 – May 2, 2006 1 message not in the archive

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 RCU: introduce rcu_soon_pending() interface

From: Heiko Carstens <[email protected]>

Introduce rcu_soon_pending() interface. This can be used to tell if there will be a new rcu batch on a cpu soon by looking at the curlist pointer. This can be used to avoid to enter a tickless idle state where the cpu would miss that a new batch is ready when rcu_start_batch would be called on a different cpu.

---

See also here: http://www.ussg.iu.edu/hypermail/linux/kernel/0604.3/0057.html
Better solutions welcome :)

 include/linux/rcupdate.h |    1 +
 kernel/rcupdate.c        |    8 ++++++++
 2 files changed, 9 insertions(+)
include/linux/rcupdate.h +1 -0
index 5673008..80bcbce 100644
@@ -132,6 +132,7 @@ static inline void rcu_bh_qsctr_inc(int 
 }
 
 extern int rcu_pending(int cpu);
+extern int rcu_soon_pending(int cpu);
 
 /**
  * rcu_read_lock - mark the beginning of an RCU read-side critical section.
kernel/rcupdate.c +8 -0
index 13458bb..e8cf09f 100644
@@ -485,6 +485,14 @@ int rcu_pending(int cpu)
 		__rcu_pending(&rcu_bh_ctrlblk, &per_cpu(rcu_bh_data, cpu));
 }
 
+int rcu_soon_pending(int cpu)
+{
+	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);
+
+	return (!!rdp->curlist || !!rdp_bh->curlist);
+}
+
 void rcu_check_callbacks(int cpu, int user)
 {
 	if (user || 
#1 lore raw

kernel/rcupdate.c int rcu_pending(int cpu) · line 494

Quoted 12 lines · Heiko Carstens
kernel/rcupdate.c +7 -0
@@ -485,6 +485,14 @@ int rcu_pending(int cpu)
 		__rcu_pending(&rcu_bh_ctrlblk, &per_cpu(rcu_bh_data, cpu));
 }
 
+int rcu_soon_pending(int cpu)
+{
+	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);
+
+	return (!!rdp->curlist || !!rdp_bh->curlist);
+}

This patch sets my nerves a-jangling.

What are the units of soonness? It's awfully waffly. Can we specify this more tightly?

Neither rcu_pending() nor rcu_soon_pending() are commented or documented. Pity the poor user trying to work out what they do, and how they differ. They're global symbols and they form part of the RCU API - they should be kernel docified, please.

There's probably a reason why neither of these symbols are exported to modules. Once they're actually documented I mught be able to work out what that reason is ;)

#2 lore raw
Quoted 17 lines

kernel/rcupdate.c int rcu_pending(int cpu) · line 494

Quoted 11 lines
kernel/rcupdate.c +7 -0
@@ -485,6 +485,14 @@ int rcu_pending(int cpu)
 		__rcu_pending(&rcu_bh_ctrlblk, &per_cpu(rcu_bh_data, cpu));
 }
 
+int rcu_soon_pending(int cpu)
+{
+	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);
+
+	return (!!rdp->curlist || !!rdp_bh->curlist);
+}

This patch sets my nerves a-jangling.

What are the units of soonness? It's awfully waffly. Can we specify this more tightly?

Quoted 8 lines

Neither rcu_pending() nor rcu_soon_pending() are commented or documented. Pity the poor user trying to work out what they do, and how they differ. They're global symbols and they form part of the RCU API - they should be kernel docified, please.

There's probably a reason why neither of these symbols are exported to modules. Once they're actually documented I mught be able to work out what that reason is ;)

Maybe rcu_batch_pending() would be a better name for rcu_soon_pending(). Also rcu_batch_in_work() would be a more descriptive name for rcu_pending() as far as I can tell.
Actually I was hoping for a better solution from the rcu experts, since I don't like this too, but couldn't find something better.

#3 lore raw
Quoted 26 lines · Heiko Carstens
Quoted 17 lines

kernel/rcupdate.c int rcu_pending(int cpu) · line 494

Quoted 11 lines
kernel/rcupdate.c +7 -0
@@ -485,6 +485,14 @@ int rcu_pending(int cpu)
 		__rcu_pending(&rcu_bh_ctrlblk, &per_cpu(rcu_bh_data, cpu));
 }
 
+int rcu_soon_pending(int cpu)
+{
+	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);
+
+	return (!!rdp->curlist || !!rdp_bh->curlist);
+}

This patch sets my nerves a-jangling.

What are the units of soonness? It's awfully waffly. Can we specify this more tightly?

Quoted 8 lines

Neither rcu_pending() nor rcu_soon_pending() are commented or documented. Pity the poor user trying to work out what they do, and how they differ. They're global symbols and they form part of the RCU API - they should be kernel docified, please.

There's probably a reason why neither of these symbols are exported to modules. Once they're actually documented I mught be able to work out what that reason is ;)

Quoted 5 lines

Maybe rcu_batch_pending() would be a better name for rcu_soon_pending(). Also rcu_batch_in_work() would be a more descriptive name for rcu_pending() as far as I can tell.
Actually I was hoping for a better solution from the rcu experts, since I don't like this too, but couldn't find something better.

OK, got a look at your patch.

You are using this internally, as part of the RCU -implementation-. You are determining whether this CPU will still be needed by RCU, or whether it can be turned off. So how 'bout calling the (internal) API something like rcu_needs_cpu()?

int rcu_needs_cpu(int cpu)

{
	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);

	return (!!rdp->curlist || !!rdp_bh->curlist || rcu_pending(cpu));
}

Then you can drop the rcu_pending() check from your 390 patch.

Seem reasonable?

The meaning of rcu_pending() is "Does RCU have some work pending on this CPU, so that there is a need to invoke rcu_check_callbacks() on this CPU?"

Thanx, Paul

#4 lore raw
Quoted 15 lines

OK, got a look at your patch.

You are using this internally, as part of the RCU -implementation-. You are determining whether this CPU will still be needed by RCU, or whether it can be turned off. So how 'bout calling the (internal) API something like rcu_needs_cpu()?

int rcu_needs_cpu(int cpu)

{
	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);

	return (!!rdp->curlist || !!rdp_bh->curlist || rcu_pending(cpu));
}
Quoted 3 lines

Then you can drop the rcu_pending() check from your 390 patch.

Seem reasonable?

Looks fine to me! Will you post a patch or should I?

#5 lore raw
Quoted 12 lines · Heiko Carstens
Quoted 12 lines

OK, got a look at your patch.

You are using this internally, as part of the RCU -implementation-. You are determining whether this CPU will still be needed by RCU, or whether it can be turned off. So how 'bout calling the (internal) API something like rcu_needs_cpu()?

int rcu_needs_cpu(int cpu)

{
	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);
Quoted 8 lines
Quoted 6 lines
	return (!!rdp->curlist || !!rdp_bh->curlist || rcu_pending(cpu));
}

Then you can drop the rcu_pending() check from your 390 patch.

Seem reasonable?

Looks fine to me! Will you post a patch or should I?

Given that I am getting 24Kbps right now, could you please post it?

Thanx, Paul

#6 lore raw

PATCH 1/1 RCU: introduce rcu_needs_cpu() interface

From: Heiko Carstens <[email protected]>
From: "Paul E. McKenney" <[email protected]>

Introduce rcu_needs_cpu() interface. This can be used to tell if there will be a new rcu batch on a cpu soon by looking at the curlist pointer. This can be used to avoid to enter a tickless idle state where the cpu would miss that a new batch is ready when rcu_start_batch would be called on a different cpu.

---

This one replaces rcu-introduce-rcu_soon_pending-interface.patch .

 include/linux/rcupdate.h |    1 +
 kernel/rcupdate.c        |    8 ++++++++
 2 files changed, 9 insertions(+)
include/linux/rcupdate.h +1 -0
index 5673008..970284f 100644
@@ -132,6 +132,7 @@ static inline void rcu_bh_qsctr_inc(int 
 }
 
 extern int rcu_pending(int cpu);
+extern int rcu_needs_cpu(int cpu);
 
 /**
  * rcu_read_lock - mark the beginning of an RCU read-side critical section.
kernel/rcupdate.c +8 -0
index 13458bb..651ac14 100644
@@ -485,6 +485,14 @@ int rcu_pending(int cpu)
 		__rcu_pending(&rcu_bh_ctrlblk, &per_cpu(rcu_bh_data, cpu));
 }
 
+int rcu_needs_cpu(int cpu)
+{
+	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);
+
+	return (!!rdp->curlist || !!rdp_bh->curlist || rcu_pending(cpu));
+}
+
 void rcu_check_callbacks(int cpu, int user)
 {
 	if (user || 

A message that is not in the archive (sent off-list or not yet received): [email protected]

#8 lore raw

PATCH 1/1 RCU: add comments to rcu_pending/rcu_needs_cpu

From: Heiko Carstens <[email protected]>

Add some comments to rcu_pending() and rcu_needs_cpu().

---

Wording might be poor, but probably better than no comments at all.

 kernel/rcupdate.c |   18 ++++++++++++++++++
 1 file changed, 18 insertions(+)
kernel/rcupdate.c +18 -0
diff -purN a/kernel/rcupdate.c b/kernel/rcupdate.c
@@ -479,12 +479,30 @@ static int __rcu_pending(struct rcu_ctrl
 	return 0;
 }
 
+/**
+ * rcu_pending - Check for pending RCU work on cpu.
+ * @cpu: cpu to check.
+ *
+ * Does RCU have some work pending on the specified cpu, so that there is a
+ * need to invoke rcu_check_callbacks() on the cpu?
+ */
 int rcu_pending(int cpu)
 {
 	return __rcu_pending(&rcu_ctrlblk, &per_cpu(rcu_data, cpu)) ||
 		__rcu_pending(&rcu_bh_ctrlblk, &per_cpu(rcu_bh_data, cpu));
 }
 
+/**
+ * rcu_needs_cpu - Determine if cpu will still be needed by RCU.
+ * @cpu: cpu to check.
+ *
+ * Determine whether the specified cpu will still be needed by RCU, or whether
+ * it can be turned off (e.g. by entering a tickless idle state).
+ * Note the difference to rcu_pending() which checks if there is some work to
+ * do that can be done immediately. While this function in addition checks if
+ * there would be some work to do if e.g. a different cpu finished working on
+ * the current batch.
+ */
 int rcu_needs_cpu(int cpu)
 {
 	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
#9 lore raw

Re: [patch] RCU: add comments to rcu_pending/rcu_needs_cpu

Quoted 8 lines · Heiko Carstens

From: Heiko Carstens <[email protected]>

Add some comments to rcu_pending() and rcu_needs_cpu().

---

Wording might be poor, but probably better than no comments at all.

But these are internal interfaces for RCU, so doesn't seem like they should go into docbook. Quite different than (say) rcu_read_lock() or call_rcu().

How about something like the following instead?

Thanx, Paul

Signed-off-by <[email protected]>

---

kernel/rcupdate.c +13 -0
diff -urpNa -X dontdiff linux-2.6.17-rc3-rnc/kernel/rcupdate.c linux-2.6.17-rc3-rnc.pem/kernel/rcupdate.c
@@ -479,12 +479,25 @@ static int __rcu_pending(struct rcu_ctrl
 	return 0;
 }
 
+/*
+ * Check to see if there is any immediate RCU-related work to be done
+ * by the current CPU, returning 1 if so.  This function is part of the
+ * RCU implementation; it is -not- an exported member of the RCU API.
+ */
+
 int rcu_pending(int cpu)
 {
 	return __rcu_pending(&rcu_ctrlblk, &per_cpu(rcu_data, cpu)) ||
 		__rcu_pending(&rcu_bh_ctrlblk, &per_cpu(rcu_bh_data, cpu));
 }
 
+/* 
+ * Check to see if any future RCU-related work will need to be done
+ * by the current CPU, even if none need be done immediately, returning
+ * 1 if so.  This function is part of the RCU implementation; it is -not-
+ * an exported member of the RCU API.
+ */
+
 int rcu_needs_cpu(int cpu)
 {
 	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
#10 lore raw

Re: [patch] RCU: add comments to rcu_pending/rcu_needs_cpu

Quoted 7 lines
Quoted 1 line

Add some comments to rcu_pending() and rcu_needs_cpu().

But these are internal interfaces for RCU, so doesn't seem like they should go into docbook. Quite different than (say) rcu_read_lock() or call_rcu().

How about something like the following instead?

Fine with me :)

#11 lore raw

PATCH 1/1 s390: exploit rcu_needs_cpu() interface

In reply to Paul E. McKenney

From: Heiko Carstens <[email protected]>

Exploit rcu_needs_cpu() interface to keep the cpu 'ticking' if necessary.

---

This one replaces s390-exploit-rcu_soon_pending-interface.patch .

 arch/s390/kernel/time.c |    7 ++++---
 1 file changed, 4 insertions(+), 3 deletions(-)
arch/s390/kernel/time.c +4 -3
index fea043b..029f099 100644
@@ -249,18 +249,19 @@ static inline void stop_hz_timer(void)
 	unsigned long flags;
 	unsigned long seq, next;
 	__u64 timer, todval;
+	int cpu = smp_processor_id();
 
 	if (sysctl_hz_timer != 0)
 		return;
 
-	cpu_set(smp_processor_id(), nohz_cpu_mask);
+	cpu_set(cpu, nohz_cpu_mask);
 
 	/*
 	 * Leave the clock comparator set up for the next timer
 	 * tick if either rcu or a softirq is pending.
 	 */
-	if (rcu_pending(smp_processor_id()) || local_softirq_pending()) {
-		cpu_clear(smp_processor_id(), nohz_cpu_mask);
+	if (rcu_needs_cpu(cpu) || local_softirq_pending()) {
+		cpu_clear(cpu, nohz_cpu_mask);
 		return;
 	}
 
#12 lore raw
In reply to Andrew Morton
Quoted 16 lines · Andrew Morton

kernel/rcupdate.c int rcu_pending(int cpu) · line 494

Quoted 12 lines · Heiko Carstens
kernel/rcupdate.c +7 -0
@@ -485,6 +485,14 @@ int rcu_pending(int cpu)
 		__rcu_pending(&rcu_bh_ctrlblk, &per_cpu(rcu_bh_data, cpu));
 }
 
+int rcu_soon_pending(int cpu)
+{
+	struct rcu_data *rdp = &per_cpu(rcu_data, cpu);
+	struct rcu_data *rdp_bh = &per_cpu(rcu_bh_data, cpu);
+
+	return (!!rdp->curlist || !!rdp_bh->curlist);
+}

This patch sets my nerves a-jangling.

Quoted 7 lines

What are the units of soonness? It's awfully waffly. Can we specify this more tightly?

Neither rcu_pending() nor rcu_soon_pending() are commented or documented. Pity the poor user trying to work out what they do, and how they differ. They're global symbols and they form part of the RCU API - they should be kernel docified, please.

Please note that the rcu_pending() interface was never intended for external use -- it is purely internal to the RCU infrastructure. If there is a new external use for rcu_pending(), then it would need to be documented. But I would rather this one stay internal -- different RCU implementations might need different things.

So, what are we trying to do here?

Quoted 3 lines

There's probably a reason why neither of these symbols are exported to modules. Once they're actually documented I mught be able to work out what that reason is ;)

The reason for rcu_pending() was that it is a private interface.

Thanx, Paul

#13 lore raw
Quoted 5 lines · Paul E. McKenney
Quoted 4 lines

Neither rcu_pending() nor rcu_soon_pending() are commented or documented. Pity the poor user trying to work out what they do, and how they differ. They're global symbols and they form part of the RCU API - they should be kernel docified, please.

Quoted 7 lines

Please note that the rcu_pending() interface was never intended for external use -- it is purely internal to the RCU infrastructure. If there is a new external use for rcu_pending(), then it would need to be documented. But I would rather this one stay internal -- different RCU implementations might need different things.

So, what are we trying to do here?

We are trying to find out if an idle cpu needs to continue to tick every HZ or if we can put it into deep sleep. If there is any kind of rcu work pending that requires the idle cpu to call in later via a timer interrupt, we need to know.

Signature · 8 lines
blue skies,
  Martin.

Martin Schwidefsky
Linux for zSeries Development & Services
IBM Deutschland Entwicklung GmbH

"Reality continues to ruin my life." - Calvin.
#14 lore raw
Quoted 12 lines · Martin Schwidefsky
Quoted 11 lines · Paul E. McKenney
Quoted 4 lines

Neither rcu_pending() nor rcu_soon_pending() are commented or documented. Pity the poor user trying to work out what they do, and how they differ. They're global symbols and they form part of the RCU API - they should be kernel docified, please.

Please note that the rcu_pending() interface was never intended for external use -- it is purely internal to the RCU infrastructure. If there is a new external use for rcu_pending(), then it would need to be documented. But I would rather this one stay internal -- different RCU implementations might need different things.

Quoted 6 lines
Quoted 1 line

So, what are we trying to do here?

We are trying to find out if an idle cpu needs to continue to tick every HZ or if we can put it into deep sleep. If there is any kind of rcu work pending that requires the idle cpu to call in later via a timer interrupt, we need to know.

OK, this confirms my guess that you are acting as part of the RCU implementation rather than as a normal user of the RCU API.

Thanx, Paul