Skip to contentSkip to conversation

SRCU updates

PATCH New Paul E. McKenney 1 participant 6 messages Feb 26, 2018 Sleepable read-copy update (SRCU)

lore
Share & tools
lore
#0 lore raw

cover SRCU updates

Hello!

This series provides minor SRCU updates:

1. Use __func__ instead of hard-coded function names.

2. Prevent sdp->srcu_gp_seq_needed_exp counter wrap, courtesy of
Ildar Ismagilov.

3. Reduce scans of srcu_data in counter wrap check, courtesy of
Ildar Ismagilov.

4. Remove dead code in srcu_gp_end(), courtesy of Byungchul Park.

5. Remove SRCU throttling.

Thanx, Paul

------------------------------------------------------------------------

 srcutree.c |   17 +++++++++--------
 1 file changed, 9 insertions(+), 8 deletions(-)
#2 lore raw

PATCH 2/5 srcu: Prevent sdp->srcu_gp_seq_needed_exp counter wrap

From: Ildar Ismagilov <[email protected]>

SRCU checks each srcu_data structure's grace-period number for counter wrap four times per cycle by default. This frequency guarantees that normal comparisons will detect potential wrap. However, the expedited grace-period number is not checked. The consquences are not too horrible (a failure to expedite a grace period when requested), but it would be good to avoid such things. This commit therefore adds this check to the expedited grace-period number.

---
 kernel/rcu/srcutree.c | 3 +++
 1 file changed, 3 insertions(+)
kernel/rcu/srcutree.c +3 -0
index 01f4a8bf0e03..bb9607baad9b 100644
@@ -579,6 +579,9 @@ static void srcu_gp_end(struct srcu_struct *sp)
 				if (ULONG_CMP_GE(gpseq,
 						 sdp->srcu_gp_seq_needed + 100))
 					sdp->srcu_gp_seq_needed = gpseq;
+				if (ULONG_CMP_GE(gpseq,
+						 sdp->srcu_gp_seq_needed_exp + 100))
+					sdp->srcu_gp_seq_needed_exp = gpseq;
 				spin_unlock_irqrestore_rcu_node(sdp, flags);
 			}
 	}
Signature · 1 line
2.5.2
#5 lore raw

PATCH 5/5 rcu: Remove SRCU throttling

In reply to Paul E. McKenney

The code in srcu_gp_end() inserts a delay every 0x3ff grace periods in order to prevent SRCU grace-period work from consuming an entire CPU when there is a long sequence of expedited SRCU grace-period requests. However, all of SRCU's grace-period work is carried out in workqueues, which are in turn within kthreads, which are automatically throttled as needed by the scheduler. In particular, if there is plenty of idle time, there is no point in throttling.

This commit therefore removes the expedited SRCU grace-period throttling.

---
 kernel/rcu/srcutree.c | 4 +---
 1 file changed, 1 insertion(+), 3 deletions(-)
kernel/rcu/srcutree.c +1 -3
index f962f4428c39..e2796f79a597 100644
@@ -596,9 +596,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
 	    ULONG_CMP_LT(gpseq, sp->srcu_gp_seq_needed)) {
 		srcu_gp_start(sp);
 		spin_unlock_irq_rcu_node(sp);
-		/* Throttle expedited grace periods: Should be rare! */
-		srcu_reschedule(sp, rcu_seq_ctr(gpseq) & 0x3ff
-				    ? 0 : SRCU_INTERVAL);
+		srcu_reschedule(sp, 0);
 	} else {
 		spin_unlock_irq_rcu_node(sp);
 	}
Signature · 1 line
2.5.2
#3 lore raw

PATCH 3/5 srcu: Reduce scans of srcu_data in counter wrap check

In reply to Paul E. McKenney

From: Ildar Ismagilov <[email protected]>

Currently, given a multi-level srcu_node tree, SRCU can scan the full set of srcu_data structures at each level when cleaning up after a grace period. This, though harmless otherwise, represents pointless overhead. This commit therefore eliminates this overhead by scanning the srcu_data structures only when traversing the leaf srcu_node structures.

---
 kernel/rcu/srcutree.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
kernel/rcu/srcutree.c +4 -2
index bb9607baad9b..d582a2352cdb 100644
@@ -527,6 +527,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
 {
 	unsigned long cbdelay;
 	bool cbs;
+	bool last_lvl;
 	int cpu;
 	unsigned long flags;
 	unsigned long gpseq;
@@ -559,7 +560,8 @@ static void srcu_gp_end(struct srcu_struct *sp)
 	rcu_for_each_node_breadth_first(sp, snp) {
 		spin_lock_irq_rcu_node(snp);
 		cbs = false;
-		if (snp >= sp->level[rcu_num_lvls - 1])
+		last_lvl = snp >= sp->level[rcu_num_lvls - 1];
+		if (last_lvl)
 			cbs = snp->srcu_have_cbs[idx] == gpseq;
 		snp->srcu_have_cbs[idx] = gpseq;
 		rcu_seq_set_state(&snp->srcu_have_cbs[idx], 1);
@@ -572,7 +574,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
 			srcu_schedule_cbs_snp(sp, snp, mask, cbdelay);
 
 		/* Occasionally prevent srcu_data counter wrap. */
-		if (!(gpseq & counter_wrap_check))
+		if (!(gpseq & counter_wrap_check) && last_lvl)
 			for (cpu = snp->grplo; cpu <= snp->grphi; cpu++) {
 				sdp = per_cpu_ptr(sp->sda, cpu);
 				spin_lock_irqsave_rcu_node(sdp, flags);
Signature · 1 line
2.5.2
#1 lore raw

PATCH 1/5 srcu: Abstract function name

In reply to Paul E. McKenney

This commit moves to __func__ for function names in the name of better resilience to change.

---
 kernel/rcu/srcutree.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
index d5cea81378cc..01f4a8bf0e03 100644
@@ -386,7 +386,7 @@ void cleanup_srcu_struct(struct srcu_struct *sp)
 		flush_delayed_work(&per_cpu_ptr(sp->sda, cpu)->work);
 	if (WARN_ON(rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)) != SRCU_STATE_IDLE) ||
 	    WARN_ON(srcu_readers_active(sp))) {
-		pr_info("cleanup_srcu_struct: Active srcu_struct %p state: %d\n", sp, rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)));
+		pr_info("%s: Active srcu_struct %p state: %d\n", __func__, sp, rcu_seq_state(READ_ONCE(sp->srcu_gp_seq)));
 		return; /* Caller forgot to stop doing call_srcu()? */
 	}
 	free_percpu(sp->sda);
Signature · 1 line
2.5.2
#4 lore raw

PATCH 4/5 srcu: Remove dead code in srcu_gp_end()

In reply to Paul E. McKenney

From: Byungchul Park <[email protected]>

Of course, compilers will optimize out a dead code. Anyway, remove any dead code for better readibility.

---
 kernel/rcu/srcutree.c | 2 --
 1 file changed, 2 deletions(-)
kernel/rcu/srcutree.c +0 -2
index d582a2352cdb..f962f4428c39 100644
@@ -532,7 +532,6 @@ static void srcu_gp_end(struct srcu_struct *sp)
 	unsigned long flags;
 	unsigned long gpseq;
 	int idx;
-	int idxnext;
 	unsigned long mask;
 	struct srcu_data *sdp;
 	struct srcu_node *snp;
@@ -556,7 +555,6 @@ static void srcu_gp_end(struct srcu_struct *sp)
 
 	/* Initiate callback invocation as needed. */
 	idx = rcu_seq_ctr(gpseq) % ARRAY_SIZE(snp->srcu_have_cbs);
-	idxnext = (idx + 1) % ARRAY_SIZE(snp->srcu_have_cbs);
 	rcu_for_each_node_breadth_first(sp, snp) {
 		spin_lock_irq_rcu_node(snp);
 		cbs = false;
Signature · 1 line
2.5.2