sched/nohz: Fix rq-> cpu_load calculations some more

Commit-ID:  5aaa0b7a2ed5b12692c9ffb5222182bd558d3146
Gitweb:     http://git.kernel.org/tip/5aaa0b7a2ed5b12692c9ffb5222182bd558d3146
Author:     Peter Zijlstra <[email protected]>
AuthorDate: Thu, 17 May 2012 17:15:29 +0200
Committer:  Ingo Molnar <[email protected]>
CommitDate: Wed, 30 May 2012 14:02:16 +0200

sched/nohz: Fix rq->cpu_load calculations some more

Follow up on commit 556061b00 ("sched/nohz: Fix rq->cpu_load[] calculations") since while that fixed the busy case it regressed the mostly idle case.

Add a callback from the nohz exit to also age the rq->cpu_load[] array. This closes the hole where either there was no nohz load balance pass during the nohz, or there was a 'significant' amount of idle time between the last nohz balance and the nohz exit.

So we'll update unconditionally from the tick to not insert any accidental 0 load periods while busy, and we try and catch up from nohz idle balance and nohz exit. Both these are still prone to missing a jiffy, but that has always been the case.

---
 include/linux/sched.h    |    1 +
 kernel/sched/core.c      |   53 +++++++++++++++++++++++++++++++++++++--------
 kernel/time/tick-sched.c |    1 +
 3 files changed, 45 insertions(+), 10 deletions(-)

index f45c0b2..d61e597 100644
@@ -145,6 +145,7 @@ extern unsigned long this_cpu_load(void);
 
 
 extern void calc_global_load(unsigned long ticks);
+extern void update_cpu_load_nohz(void);
 
 extern unsigned long get_parent_ip(unsigned long addr);
 

index 6a3a5b9..0c927cd 100644
@@ -576,6 +576,7 @@ void tick_nohz_idle_exit(void)
 	/* Update jiffies first */
 	select_nohz_load_balancer(0);
 	tick_do_update_jiffies64(now);
+	update_cpu_load_nohz();
 
 #ifndef CONFIG_VIRT_CPU_ACCOUNTING
 	/*