SRCU updates
cover SRCU updates
Hello!
This series provides minor SRCU updates:
1. Use __func__ instead of hard-coded function names.
2. Prevent sdp->srcu_gp_seq_needed_exp counter wrap, courtesy of
Ildar Ismagilov.
3. Reduce scans of srcu_data in counter wrap check, courtesy of
Ildar Ismagilov.
4. Remove dead code in srcu_gp_end(), courtesy of Byungchul Park.
5. Remove SRCU throttling.
Thanx, Paul
------------------------------------------------------------------------
srcutree.c | 17 +++++++++-------- 1 file changed, 9 insertions(+), 8 deletions(-)
PATCH 2/5 srcu: Prevent sdp->srcu_gp_seq_needed_exp counter wrap
From: Ildar Ismagilov <[email protected]>
SRCU checks each srcu_data structure's grace-period number for counter wrap four times per cycle by default. This frequency guarantees that normal comparisons will detect potential wrap. However, the expedited grace-period number is not checked. The consquences are not too horrible (a failure to expedite a grace period when requested), but it would be good to avoid such things. This commit therefore adds this check to the expedited grace-period number.
--- kernel/rcu/srcutree.c | 3 +++ 1 file changed, 3 insertions(+)
kernel/rcu/srcutree.c +3 -0
@@ -579,6 +579,9 @@ static void srcu_gp_end(struct srcu_struct *sp)if (ULONG_CMP_GE (gpseq, sdp->srcu_gp_seq_needed +100 )) sdp->srcu_gp_seq_needed = gpseq; +if (ULONG_CMP_GE (gpseq, + sdp->srcu_gp_seq_needed_exp +100 )) + sdp->srcu_gp_seq_needed_exp = gpseq;spin_unlock_irqrestore_rcu_node (sdp, flags); } }
Signature · 1 line
2.5.2
PATCH 5/5 rcu: Remove SRCU throttling
In reply to Paul E. McKenneyThe code in srcu_gp_end() inserts a delay every 0x3ff grace periods in order to prevent SRCU grace-period work from consuming an entire CPU when there is a long sequence of expedited SRCU grace-period requests. However, all of SRCU's grace-period work is carried out in workqueues, which are in turn within kthreads, which are automatically throttled as needed by the scheduler. In particular, if there is plenty of idle time, there is no point in throttling.
This commit therefore removes the expedited SRCU grace-period throttling.
--- kernel/rcu/srcutree.c | 4 +--- 1 file changed, 1 insertion(+), 3 deletions(-)
kernel/rcu/srcutree.c +1 -3
@@ -596,9 +596,7 @@ static void srcu_gp_end(struct srcu_struct *sp) ULONG_CMP_LT(gpseq, sp->srcu_gp_seq_needed)) { srcu_gp_start(sp); spin_unlock_irq_rcu_node(sp); - /* Throttle expedited grace periods: Should be rare! */ - srcu_reschedule(sp, rcu_seq_ctr(gpseq) & 0x3ff - ? 0 : SRCU_INTERVAL); + srcu_reschedule(sp, 0); } else {spin_unlock_irq_rcu_node (sp); }
Signature · 1 line
2.5.2
PATCH 3/5 srcu: Reduce scans of srcu_data in counter wrap check
In reply to Paul E. McKenneyFrom: Ildar Ismagilov <[email protected]>
Currently, given a multi-level srcu_node tree, SRCU can scan the full set of srcu_data structures at each level when cleaning up after a grace period. This, though harmless otherwise, represents pointless overhead. This commit therefore eliminates this overhead by scanning the srcu_data structures only when traversing the leaf srcu_node structures.
--- kernel/rcu/srcutree.c | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
kernel/rcu/srcutree.c +4 -2
@@ -527,6 +527,7 @@ static void srcu_gp_end(struct srcu_struct *sp) {unsigned long cbdelay;bool cbs; +bool last_lvl;int cpu;unsigned long flags;unsigned long gpseq;
@@ -559,7 +560,8 @@ static void srcu_gp_end(struct srcu_struct *sp)rcu_for_each_node_breadth_first (sp, snp) {spin_lock_irq_rcu_node (snp); cbs = false; -if (snp >= sp->level[rcu_num_lvls -1 ]) + last_lvl = snp >= sp->level[rcu_num_lvls -1 ]; +if (last_lvl) cbs = snp->srcu_have_cbs[idx] == gpseq; snp->srcu_have_cbs[idx] = gpseq;rcu_seq_set_state (&snp->srcu_have_cbs[idx],1 );
@@ -572,7 +574,7 @@ static void srcu_gp_end(struct srcu_struct *sp)srcu_schedule_cbs_snp (sp, snp, mask, cbdelay);/* Occasionally prevent srcu_data counter wrap. */ -if (!(gpseq & counter_wrap_check)) +if (!(gpseq & counter_wrap_check) && last_lvl)for (cpu = snp->grplo; cpu <= snp->grphi; cpu++) { sdp =per_cpu_ptr (sp->sda, cpu);spin_lock_irqsave_rcu_node (sdp, flags);
Signature · 1 line
2.5.2
PATCH 1/5 srcu: Abstract function name
In reply to Paul E. McKenneyThis commit moves to __func__ for function names in the name of better resilience to change.
--- kernel/rcu/srcutree.c | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/rcu/srcutree.c +1 -1
@@ -386,7 +386,7 @@ void cleanup_srcu_struct(struct srcu_struct *sp)flush_delayed_work (&per_cpu_ptr (sp->sda, cpu)->work);if (WARN_ON (rcu_seq_state (READ_ONCE (sp->srcu_gp_seq)) !=SRCU_STATE_IDLE ) ||WARN_ON (srcu_readers_active (sp))) { -pr_info ("cleanup_srcu_struct: Active srcu_struct %p state: %d\n" , sp,rcu_seq_state (READ_ONCE (sp->srcu_gp_seq))); +pr_info ("%s: Active srcu_struct %p state: %d\n" , __func__, sp,rcu_seq_state (READ_ONCE (sp->srcu_gp_seq)));return ;/* Caller forgot to stop doing call_srcu()? */ }free_percpu (sp->sda);
Signature · 1 line
2.5.2
PATCH 4/5 srcu: Remove dead code in srcu_gp_end()
In reply to Paul E. McKenneyFrom: Byungchul Park <[email protected]>
Of course, compilers will optimize out a dead code. Anyway, remove any dead code for better readibility.
--- kernel/rcu/srcutree.c | 2 -- 1 file changed, 2 deletions(-)
kernel/rcu/srcutree.c +0 -2
@@ -532,7 +532,6 @@ static void srcu_gp_end(struct srcu_struct *sp)unsigned long flags;unsigned long gpseq;int idx; -int idxnext;unsigned long mask;struct srcu_data *sdp;struct srcu_node *snp;
@@ -556,7 +555,6 @@ static void srcu_gp_end(struct srcu_struct *sp)/* Initiate callback invocation as needed. */ idx =rcu_seq_ctr (gpseq) %ARRAY_SIZE (snp->srcu_have_cbs); - idxnext = (idx +1 ) %ARRAY_SIZE (snp->srcu_have_cbs);rcu_for_each_node_breadth_first (sp, snp) {spin_lock_irq_rcu_node (snp); cbs = false;
Signature · 1 line
2.5.2