Skip to contentSkip to conversation

srcu: reduce scans of srcu_data in counter wrap check

PATCH In review Ildar Ismagilov 2 participants 2 messages Jan 31, 2018 – Feb 3, 2018 Sleepable read-copy update (SRCU)

lore
Share & tools
lore

Ildar Ismagilov

author
#0 lore raw

PATCH 2/2 srcu: reduce scans of srcu_data in counter wrap check

This patch reduce scans of srcu_data in case of a srcu_struct tree have more than one levels

Signed-off-by Ildar Ismagilov <[email protected]>
---
 kernel/rcu/srcutree.c | 6 ++++--
 1 file changed, 4 insertions(+), 2 deletions(-)
kernel/rcu/srcutree.c +4 -2
index c54617280245..2268381706d2 100644
@@ -500,6 +500,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
 {
 	unsigned long cbdelay;
 	bool cbs;
+	bool last_lvl;
 	int cpu;
 	unsigned long flags;
 	unsigned long gpseq;

@@ -532,7 +533,8 @@ static void srcu_gp_end(struct srcu_struct *sp)
 	rcu_for_each_node_breadth_first(sp, snp) {
 		raw_spin_lock_irq_rcu_node(snp);
 		cbs = false;
-		if (snp >= sp->level[rcu_num_lvls - 1])
+		last_lvl = snp >= sp->level[rcu_num_lvls - 1];
+		if (last_lvl)
 			cbs = snp->srcu_have_cbs[idx] == gpseq;
 		snp->srcu_have_cbs[idx] = gpseq;
 		rcu_seq_set_state(&snp->srcu_have_cbs[idx], 1);

@@ -545,7 +547,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
 			srcu_schedule_cbs_snp(sp, snp, mask, cbdelay);
 
 		/* Occasionally prevent srcu_data counter wrap. */
-		if (!(gpseq & counter_wrap_check))
+		if (!(gpseq & counter_wrap_check) && last_lvl)
 			for (cpu = snp->grplo; cpu <= snp->grphi; cpu++) {
 				sdp = per_cpu_ptr(sp->sda, cpu);
 				raw_spin_lock_irqsave_rcu_node(sdp, flags);
Signature · 1 line
2.11.0
#1 lore raw
Quoted 2 lines · Ildar Ismagilov

This patch reduce scans of srcu_data in case of a srcu_struct tree have more than one levels

Good catch!!! Applied for testing and review.

I had to hand-apply this patch as well, and also fill out the commit log. In addition, in the future, please capitalize the word in the Subject line following the colon (":"). Please see below for the updated commit and again please let me know if I messed something up.

Thanx, Paul

------------------------------------------------------------------------

commit bcec03ee2e43f2a24c61a99709d4dd1aa90438c2

Author: Ildar Ismagilov <[email protected]>
Date:   Wed Jan 31 22:42:21 2018 +0300

srcu: Reduce scans of srcu_data in counter wrap check

Currently, given a multi-level srcu_node tree, SRCU can scan the full set of srcu_data structures at each level when cleaning up after a grace period. This, though harmless otherwise, represents pointless overhead. This commit therefore eliminates this overhead by scanning the srcu_data structures only when traversing the leaf srcu_node structures.

Signed-off-by Ildar Ismagilov <[email protected]>
Signed-off-by Paul E. McKenney <[email protected]>
kernel/rcu/srcutree.c +4 -2
index ad3e1aa5e6ea..72dc6e39be14 100644
@@ -527,6 +527,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
 {
 	unsigned long cbdelay;
 	bool cbs;
+	bool last_lvl;
 	int cpu;
 	unsigned long flags;
 	unsigned long gpseq;

@@ -559,7 +560,8 @@ static void srcu_gp_end(struct srcu_struct *sp)
 	rcu_for_each_node_breadth_first(sp, snp) {
 		spin_lock_irq_rcu_node(snp);
 		cbs = false;
-		if (snp >= sp->level[rcu_num_lvls - 1])
+		last_lvl = snp >= sp->level[rcu_num_lvls - 1];
+		if (last_lvl)
 			cbs = snp->srcu_have_cbs[idx] == gpseq;
 		snp->srcu_have_cbs[idx] = gpseq;
 		rcu_seq_set_state(&snp->srcu_have_cbs[idx], 1);

@@ -572,7 +574,7 @@ static void srcu_gp_end(struct srcu_struct *sp)
 			srcu_schedule_cbs_snp(sp, snp, mask, cbdelay);
 
 		/* Occasionally prevent srcu_data counter wrap. */
-		if (!(gpseq & counter_wrap_check))
+		if (!(gpseq & counter_wrap_check) && last_lvl)
 			for (cpu = snp->grplo; cpu <= snp->grphi; cpu++) {
 				sdp = per_cpu_ptr(sp->sda, cpu);
 				spin_lock_irqsave_rcu_node(sdp, flags);