sched/nohz: Make kcpustat vtime aware (Fix kcpustat on nohz_full)

Kcpustat (the stats you see for each CPU on /proc/stat) is partly maintained by the tick, updated by TICK_NSEC every jiffy, the same way we account the cputime for tasks.

Now in the case of nohz_full, kcpustat doesn't get accounted anymore while the tick is stopped. Vtime maintains the task cputime but not kcpustat.

This issue was hidden as long as we had the 1Hz remaining tick, then Yauheni Kaliuta made me remember that problem.

I scratched my head a lot on this, due to all the possible races. The solution here is to fetch the task running on a CPU with RCU, read its vtime delta (like we do for cputime) and add it to the relevant kcpustat field. There have been several subtleties on the way (concurrent task nice changes, earliest RCU delayed put_task_struct(), ordering with vtime) and I couldn't resist a few cleanups so the patchset isn't too small, sorry about that...

git://git.kernel.org/pub/scm/linux/kernel/git/frederic/linux-dynticks.git
nohz/kcpustat

HEAD: c7c45c06334346f62dbbf7bb12e2a8ab954532e5

Thanks,
Frederic
---

Frederic Weisbecker (25):
sched/vtime: Fix guest/system mis-accounting on task switch
sched/vtime: Protect idle accounting under vtime seqcount
vtime: Rename vtime_account_system() to vtime_account_kernel()
vtime: Spare a seqcount lock/unlock cycle on context switch
sched/vtime: Record CPU under seqcount for kcpustat needs
sched/cputime: Add vtime idle task state
sched/cputime: Add vtime guest task state
vtime: Exit vtime before exit_notify()
kcpustat: Track running task following vtime sequences
context_tracking: Remove context_tracking_active()
context_tracking: s/context_tracking_is_enabled/context_tracking_enabled()
context_tracking: Rename context_tracking_is_cpu_enabled() to context_tracking_enabled_this_cpu()
context_tracking: Introduce context_tracking_enabled_cpu()
sched/vtime: Rename vtime_accounting_cpu_enabled() to vtime_accounting_enabled_this_cpu() sched/vtime: Introduce vtime_accounting_enabled_cpu()
sched/cputime: Allow to pass cputime index on user/guest accounting sched/cputime: Standardize the kcpustat index based accounting functions
vtime: Track nice-ness on top of context switch
sched/vite: Handle nice updates under vtime
sched/kcpustat: Introduce vtime-aware kcpustat accessor
procfs: Use vtime aware kcpustat accessor
cpufreq: Use vtime aware kcpustat accessor
leds: Use vtime aware kcpustat accessors
rackmeter: Use vtime aware kcpustat accessors
sched/vtime: Clarify vtime_task_switch() argument layout

 arch/ia64/include/asm/cputime.h         |   3 +-
 arch/ia64/kernel/time.c                 |  15 +-
 arch/powerpc/include/asm/cputime.h      |   8 +-
 arch/powerpc/kernel/time.c              |  12 +-
 arch/s390/kernel/vtime.c                |  19 +-
 arch/x86/entry/calling.h                |   2 +-
 drivers/cpufreq/cpufreq.c               |  18 +-
 drivers/cpufreq/cpufreq_governor.c      |  27 ++-
 drivers/leds/trigger/ledtrig-activity.c |   9 +-
 drivers/macintosh/rack-meter.c          |  14 +-
 fs/proc/stat.c                          |  21 +-
 include/linux/context_tracking.h        |  30 +--
 include/linux/context_tracking_state.h  |  19 +-
 include/linux/kernel_stat.h             |  28 ++-
 include/linux/sched.h                   |  12 +-
 include/linux/tick.h                    |   2 +-
 include/linux/vtime.h                   |  72 ++++---
 kernel/context_tracking.c               |   6 +-
 kernel/exit.c                           |   1 +
 kernel/sched/core.c                     |   6 +-
 kernel/sched/cputime.c                  | 372 +++++++++++++++++++++++++-------
 kernel/sched/sched.h                    |  39 ++++
 kernel/time/tick-sched.c                |   2 +-
 23 files changed, 548 insertions(+), 189 deletions(-)