rcu: Tweak tiny RCU
cover rcu: Tweak tiny RCU
Hi Paul,
These are few code improvements to the tiny RCU.
Alexander Gordeev (3):
rcu: Remove unnecessary condition check in rcu_qsctr_help()
rcu: Remove fast path from __rcu_process_callbacks()
rcu: Call trace_rcu_batch_start() with enabled interrupts
kernel/rcu/tiny.c | 18 ++++-------------- 1 file changed, 4 insertions(+), 14 deletions(-)
Signature · 1 line
1.8.3.1
PATCH 1/3 rcu: Remove unnecessary condition check in rcu_qsctr_help()
In cases ->curtail and ->donetail pointers differ ->rcucblist always points to the beginning of the current list and thus can not be NULL. Therefore, the check ->rcucblist != NULL is redundant and could be removed.
--- kernel/rcu/tiny.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)
kernel/rcu/tiny.c +1 -2
@@ -103,8 +103,7 @@ EXPORT_SYMBOL(__rcu_is_watching);static int rcu_qsctr_help (struct rcu_ctrlblk *rcp) {RCU_TRACE (reset_cpu_stall_ticks (rcp)); -if (rcp->rcucblist !=NULL && - rcp->donetail != rcp->curtail) { +if (rcp->donetail != rcp->curtail) { rcp->donetail = rcp->curtail;return 1 ; }
Signature · 1 line
1.8.3.1
Re: [PATCH 1/3] rcu: Remove unnecessary condition check in rcu_qsctr_help()
Quoted 7 lines · Alexander Gordeev
In cases ->curtail and ->donetail pointers differ ->rcucblist always points to the beginning of the current list and thus can not be NULL. Therefore, the check ->rcucblist != NULL is redundant and could be removed.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Alexander Gordeev <[email protected]>
Good point, queued for 3.21.
Thanx, Paul
Quoted 21 lines
--- kernel/rcu/tiny.c | 3 +-- 1 file changed, 1 insertion(+), 2 deletions(-)kernel/rcu/tiny.c +1
-2@@ -103,8 +103,7 @@ EXPORT_SYMBOL(__rcu_is_watching);static int rcu_qsctr_help (struct rcu_ctrlblk *rcp) {RCU_TRACE (reset_cpu_stall_ticks (rcp)); -if (rcp->rcucblist !=NULL && - rcp->donetail != rcp->curtail) { +if (rcp->donetail != rcp->curtail) { rcp->donetail = rcp->curtail;return 1 ; }Signature · 1 line
1.8.3.1
PATCH 2/3 rcu: Remove fast path from __rcu_process_callbacks()
In reply to Alexander GordeevThe standard code path accommodates a condition when no RCU callbacks are ready to invoke. Since size of the code is a priority for tiny RCU, remove the fast path.
--- kernel/rcu/tiny.c | 11 ----------- 1 file changed, 11 deletions(-)
kernel/rcu/tiny.c +0 -11
@@ -168,17 +168,6 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)unsigned long flags; RCU_TRACE(int cb_count = 0); -/* If no RCU callbacks ready to invoke, just return. */ -if (&rcp->rcucblist == rcp->donetail) { -RCU_TRACE (trace_rcu_batch_start (rcp->name,0 ,0 ,-1 )); -RCU_TRACE (trace_rcu_batch_end (rcp->name,0 , - !!ACCESS_ONCE (rcp->rcucblist), -need_resched (), -is_idle_task (current), - false)); -return ; - } -/* Move the ready-to-invoke callbacks to a local list. */ local_irq_save (flags);RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , rcp->qlen,-1 ));
Signature · 1 line
1.8.3.1
Re: [PATCH 2/3] rcu: Remove fast path from __rcu_process_callbacks()
Quoted 6 lines · Alexander Gordeev
The standard code path accommodates a condition when no RCU callbacks are ready to invoke. Since size of the code is a priority for tiny RCU, remove the fast path.
Cc "Paul E. McKenney" <[email protected]>Signed-off-by Alexander Gordeev <[email protected]>
Also a good point. The savings is small in production builds because the RCU_TRACE() statements are not compiled, but every little bit helps, and the simplification is good as well.
Queued for 3.21.
Thanx, Paul
Quoted 29 lines
--- kernel/rcu/tiny.c | 11 ----------- 1 file changed, 11 deletions(-)kernel/rcu/tiny.c +0
-11@@ -168,17 +168,6 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)unsigned long flags; RCU_TRACE(int cb_count = 0); -/* If no RCU callbacks ready to invoke, just return. */ -if (&rcp->rcucblist == rcp->donetail) { -RCU_TRACE (trace_rcu_batch_start (rcp->name,0 ,0 ,-1 )); -RCU_TRACE (trace_rcu_batch_end (rcp->name,0 , - !!ACCESS_ONCE (rcp->rcucblist), -need_resched (), -is_idle_task (current), - false)); -return ; - } -/* Move the ready-to-invoke callbacks to a local list. */ local_irq_save (flags);RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , rcp->qlen,-1 ));Signature · 1 line
1.8.3.1
PATCH 3/3 rcu: Call trace_rcu_batch_start() with enabled interrupts
In reply to Alexander GordeevCurrently trace_rcu_batch_start() is called with local interrupts disabled. Yet, there is no reason to do so.
--- kernel/rcu/tiny.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/rcu/tiny.c +3 -1
@@ -166,11 +166,12 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)const char *rn =NULL ;struct rcu_head *next, *list;unsigned long flags; +RCU_TRACE (long qlen ); RCU_TRACE(int cb_count = 0);/* Move the ready-to-invoke callbacks to a local list. */ local_irq_save (flags); -RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , rcp->qlen,-1 )); +RCU_TRACE (qlen = rcp->qlen); list = rcp->rcucblist; rcp->rcucblist = *rcp->donetail; *rcp->donetail =NULL ;
@@ -180,6 +181,7 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)local_irq_restore (flags);/* Invoke the callbacks on the local list. */ +RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , qlen,-1 ));RCU_TRACE (rn = rcp->name);while (list) { next = list->next;
Signature · 1 line
1.8.3.1
Re: [PATCH 3/3] rcu: Call trace_rcu_batch_start() with enabled interrupts
Quoted 5 lines · Alexander Gordeev
Currently
trace_rcu_batch_start()is called with local interrupts disabled. Yet, there is no reason to do so.Cc "Paul E. McKenney" <[email protected]>Signed-off-by Alexander Gordeev <[email protected]>
Hmmm... I am not seeing this one. As you noted in the commit log for your earlier patch, the purpose of Tiny RCU is to be tiny, not to be all that fast. This commit increases the size a bit (admittedly only when CONFIG_RCU_TRACE=y), and also increases complexity a bit.
So it does not look to me to be something we want for Tiny RCU.
So what am I missing here?
Thanx, Paul
Quoted 33 lines
--- kernel/rcu/tiny.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)kernel/rcu/tiny.c +3
-1@@ -166,11 +166,12 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)const char *rn =NULL ;struct rcu_head *next, *list;unsigned long flags; +RCU_TRACE (long qlen ); RCU_TRACE(int cb_count = 0);/* Move the ready-to-invoke callbacks to a local list. */ local_irq_save (flags); -RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , rcp->qlen,-1 )); +RCU_TRACE (qlen = rcp->qlen); list = rcp->rcucblist; rcp->rcucblist = *rcp->donetail; *rcp->donetail =NULL ;@@ -180,6 +181,7 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)local_irq_restore (flags);/* Invoke the callbacks on the local list. */ +RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , qlen,-1 ));RCU_TRACE (rn = rcp->name);while (list) { next = list->next;Signature · 1 line
1.8.3.1
Re: [PATCH 3/3] rcu: Call trace_rcu_batch_start() with enabled interrupts
Quoted 7 lines · Paul E. McKenney
Quoted 5 lines · Alexander Gordeev
Currently
trace_rcu_batch_start()is called with local interrupts disabled. Yet, there is no reason to do so.Cc "Paul E. McKenney" <[email protected]>Signed-off-by Alexander Gordeev <[email protected]>
Quoted 8 lines
Hmmm... I am not seeing this one. As you noted in the commit log for your earlier patch, the purpose of Tiny RCU is to be tiny, not to be all that fast. This commit increases the size a bit (admittedly only when
CONFIG_RCU_TRACE=y), and also increases complexity a bit.So it does not look to me to be something we want for Tiny RCU.
So what am I missing here?
The benefit - "heavy" trace_rcu_batch_start() is called while interrupts are enabled. Which is normally a priority, but in this case - still a good tradeoff IMHO.
And I do not agree :) The code reads better with the loop tightly "enclosed" with trace_rcu_batch_start()/trace_rcu_batch_end().
Quoted 36 lines
Thanx, Paul
Quoted 33 lines
--- kernel/rcu/tiny.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)kernel/rcu/tiny.c +3
-1@@ -166,11 +166,12 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)const char *rn =NULL ;struct rcu_head *next, *list;unsigned long flags; +RCU_TRACE (long qlen ); RCU_TRACE(int cb_count = 0);/* Move the ready-to-invoke callbacks to a local list. */ local_irq_save (flags); -RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , rcp->qlen,-1 )); +RCU_TRACE (qlen = rcp->qlen); list = rcp->rcucblist; rcp->rcucblist = *rcp->donetail; *rcp->donetail =NULL ;@@ -180,6 +181,7 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)local_irq_restore (flags);/* Invoke the callbacks on the local list. */ +RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , qlen,-1 ));RCU_TRACE (rn = rcp->name);while (list) { next = list->next;Signature · 1 line
1.8.3.1
Signature · 3 lines
Regards, Alexander Gordeev [email protected]
Re: [PATCH 3/3] rcu: Call trace_rcu_batch_start() with enabled interrupts
Quoted 15 lines · Alexander Gordeev
Quoted 14 lines · Paul E. McKenney
Quoted 5 lines · Alexander Gordeev
Currently
trace_rcu_batch_start()is called with local interrupts disabled. Yet, there is no reason to do so.Cc "Paul E. McKenney" <[email protected]>Signed-off-by Alexander Gordeev <[email protected]>Hmmm... I am not seeing this one. As you noted in the commit log for your earlier patch, the purpose of Tiny RCU is to be tiny, not to be all that fast. This commit increases the size a bit (admittedly only when
CONFIG_RCU_TRACE=y), and also increases complexity a bit.So it does not look to me to be something we want for Tiny RCU.
Quoted 8 lines
Quoted 1 line
So what am I missing here?
The benefit - "heavy"
trace_rcu_batch_start()is called while interrupts are enabled. Which is normally a priority, but in this case - still a good tradeoff IMHO.And I do not agree :) The code reads better with the loop tightly "enclosed" with trace_rcu_batch_start()/trace_rcu_batch_end().
Sorry, but I am still not seeing this one as being worth the change. I did take the other two, and they are passing light rcutorture testing, so we are good on that front, at least.
Thanx, Paul
Quoted 40 lines
Quoted 34 lines
Quoted 33 lines
--- kernel/rcu/tiny.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)kernel/rcu/tiny.c +3
-1@@ -166,11 +166,12 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)const char *rn =NULL ;struct rcu_head *next, *list;unsigned long flags; +RCU_TRACE (long qlen ); RCU_TRACE(int cb_count = 0);/* Move the ready-to-invoke callbacks to a local list. */ local_irq_save (flags); -RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , rcp->qlen,-1 )); +RCU_TRACE (qlen = rcp->qlen); list = rcp->rcucblist; rcp->rcucblist = *rcp->donetail; *rcp->donetail =NULL ;@@ -180,6 +181,7 @@ static void __rcu_process_callbacks(struct rcu_ctrlblk *rcp)local_irq_restore (flags);/* Invoke the callbacks on the local list. */ +RCU_TRACE (trace_rcu_batch_start (rcp->name,0 , qlen,-1 ));RCU_TRACE (rn = rcp->name);while (list) { next = list->next;Signature · 1 line
1.8.3.1Signature · 3 lines
Regards, Alexander Gordeev [email protected]