RCU recursion? (code inspection)
Hi Paul,
While looking at synchronize_rcu(), I noticed that synchronize_rcu_expedited() calls synchronize_sched_expedited(), which can call synchronize_sched() when it is worried about the counter wrapping, which can call synchronize_sched_expedited()
The code is sufficiently convoluted that I am unsure whether this recursion can actually happen in practice, but I also did not spot anything that would stop it.
Signature · 1 line
All rights reversed
Quoted 10 lines · Rik van Riel
Hi Paul,
While looking at
synchronize_rcu(), I noticed thatsynchronize_rcu_expedited()callssynchronize_sched_expedited(), which can callsynchronize_sched()when it is worried about the counter wrapping, which can callsynchronize_sched_expedited()The code is sufficiently convoluted that I am unsure whether this recursion can actually happen in practice, but I also did not spot anything that would stop it.
Hmmm... Sounds like I should take a look!
Thanx, Paul
Quoted 8 lines · Paul E. McKenney
Quoted 7 lines · Rik van Riel
Hi Paul,
While looking at
synchronize_rcu(), I noticed thatsynchronize_rcu_expedited()callssynchronize_sched_expedited(), which can callsynchronize_sched()when it is worried about the counter wrapping, which can callsynchronize_sched_expedited()
Quoted 5 lines
Quoted 3 lines
The code is sufficiently convoluted that I am unsure whether this recursion can actually happen in practice, but I also did not spot anything that would stop it.
Hmmm... Sounds like I should take a look!
And good catch! The following patch should fix this. Bad one on me, given that all the other places in synchronize_sched_expedited() that you would expect to invoke synchronize_sched() instead invoke wait_rcu_gp(call_rcu_sched)...
Thanx, Paul
------------------------------------------------------------------------
rcu: Make synchronize_sched_expedited() call wait_rcu_gp()
Currently, synchronize_sched_expedited() will call synchronize_sched() if there is danger of counter wrap. But if configuration says to always do expedited grace periods, synchronize_sched() will just call synchronize_sched_expedited() right back again. In theory, the old expedited operations will complete, the counters will get back in synch, and the recursion will end. But we could easily run out of stack long before that time. This commit therefore makes synchronize_sched_expedited() invoke the underlying wait_rcu_gp(call_rcu_sched) instead of synchronize_sched(), the same as all the other calls out from synchronize_sched_expedited().
This bug was introduced by commit 1924bcb02597 (Avoid counter wrap in synchronize_sched_expedited()).
kernel/rcu/tree.c +1 -1
@@ -3310,7 +3310,7 @@ void synchronize_sched_expedited(void)if (ULONG_CMP_GE ((ulong )atomic_long_read (&rsp->expedited_start), (ulong )atomic_long_read (&rsp->expedited_done) +ULONG_MAX /8 )) { -synchronize_sched (); +wait_rcu_gp (call_rcu_sched);atomic_long_inc (&rsp->expedited_wrap);return ; }