RFC Refactor RCU's interactions with CPU hotplug

Hello!

As promised/threatened for some years now...

This series, perhaps for v3.21, contains changes that allow RCU to more precisely determine when a CPU goes offline. This is a nice improvement over the current state, in which a CPU takes a final pass through the scheduler after the CPU_DYING notifier. RCU currently gets around this imprecision by assuming that it will never take more than one jiffy for the CPU to make its final pass through the scheduler, which is clearly an accident waiting to happen, especially in virtualized environments. This series of changes removes this vulnerability, although the corresponding window of vulnerability for CPUs coming online remains and will be addressed at a later date.

The patches in this series are as follows:

1. Remove the need for the ->onoff_mutex sleeplock by buffering
CPU online and offline events in a pair of new rcu_node bitmasks, ->oflmask and ->onlmask. These are protected by the rcu_node structure's ->lock, which may be acquired from preemption-disabled environments such as the idle loop. CPU-hotplug races with grace-period initialization are eliminated by applying ->oflmask and ->onlmask during the early phases of grace-period initialization. The various lockdep-RCU checks for using RCU from an offline CPU are updated to allow for the new ->oflmask and ->onlmask bitmasks.

2. Remove the ->onoff_mutex sleeplock.

3. Replace the current use of idle_cpu() with a per-CPU cpu_dead_idle
variable so that powering off the CPU will be deferred until just before the outgoing CPU invokes arch_cpu_idle_dead().

4. Cause the idle loop to invoke rcu_cpu_notify() just before
the cpu_dead_idle per-CPU variable is set. Should a more organized method of invoking CPU-hotplug notifiers from the idle loop appear, this code should clearly be modified to use that more organized method.

This passes light rcutorture testing, but probably has a bug or three remaining. Interestingly enough, patch #1 above allows RCU to tolerate some new torture methods that otherwise break it.

Thanx, Paul

------------------------------------------------------------------------

 b/include/linux/cpu.h      |    2 
 b/include/linux/rcupdate.h |    2 
 b/kernel/cpu.c             |    6 +
 b/kernel/rcu/tree.c        |  209 ++++++++++++++++++++++++++++++++-------------
 b/kernel/rcu/tree.h        |    9 +
 b/kernel/rcu/tree_plugin.h |   24 +----
 b/kernel/rcu/tree_trace.c  |    3 
 b/kernel/sched/idle.c      |    9 +
 8 files changed, 184 insertions(+), 80 deletions(-)