nohz: Remove update_ts_time_stat from tick_nohz_start_idle
PATCH 1/3 nohz: Remove update_ts_time_stat from tick_nohz_start_idle
From: Michal Hocko <[email protected]>
There is no reason to call update_ts_time_stat from tick_nohz_start_idle anymore (after e0e37c20 sched: Eliminate the ts->idle_lastupdate field) when we updated idle_lastupdate unconditionally.
We haven't set idle_active yet and do not provide last_update_time so the whole call end up being just 2 wasted branches.
--- kernel/time/tick-sched.c | 2 -- 1 files changed, 0 insertions(+), 2 deletions(-)
kernel/time/tick-sched.c +0 -2
@@ -186,8 +186,6 @@ static ktime_t tick_nohz_start_idle(int cpu, struct tick_sched *ts) now =ktime_get (); -update_ts_time_stats (cpu, ts, now,NULL ); - ts->idle_entrytime = now; ts->idle_active =1 ;sched_clock_idle_sleep_event ();
Signature · 1 line
1.7.5.4
PATCH 2/3 nohz: Remove ts->inidle checks before restarting the tick
ts->inidle is set by tick_nohz_idle_enter() and unset by tick_nohz_idle_exit(). However These two calls are assumed to be always paired. This means that by the time we call tick_nohz_idle_exit(), ts->inidle is supposed to be always set to 1.
Remove the checks for ts->inidle in tick_nohz_idle_exit(). This simplifies a bit the code and improves its debuggability (ie: ensure the call is paired with a tick_nohz_idle_enter() call).
--- kernel/time/tick-sched.c | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-)
kernel/time/tick-sched.c +6 -5
@@ -560,20 +560,21 @@ void tick_nohz_idle_exit(void)local_irq_disable (); -if (ts->idle_active || (ts->inidle && ts->tick_stopped)) +WARN_ON_ONCE (!ts->inidle); + + ts->inidle =0 ; + +if (ts->idle_active || ts->tick_stopped) now =ktime_get ();if (ts->idle_active)tick_nohz_stop_idle (cpu, now); -if (!ts->inidle || !ts->tick_stopped) { - ts->inidle =0 ; +if (!ts->tick_stopped) {local_irq_enable ();return ; } - ts->inidle =0 ; -/* Update jiffies first */ select_nohz_load_balancer (0 );tick_do_update_jiffies64 (now);
Signature · 1 line
1.7.5.4
[tip:timers/core] nohz: Remove ts-> Einidle checks before restarting the tick
Commit-ID: 15f827be93928890bba965bc175caee50c4406d2 Gitweb: http://git.kernel.org/tip/15f827be93928890bba965bc175caee50c4406d2 Author: Frederic Weisbecker <[email protected]> AuthorDate: Tue, 24 Jan 2012 18:59:43 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Wed, 15 Feb 2012 15:23:09 +0100
nohz: Remove ts->Einidle checks before restarting the tick
ts->inidle is set by tick_nohz_idle_enter() and unset by tick_nohz_idle_exit(). However these two calls are assumed to be always paired. This means that by the time we call tick_nohz_idle_exit(), ts->inidle is supposed to be always set to 1.
Remove the checks for ts->inidle in tick_nohz_idle_exit(). This simplifies a bit the code and improves its debuggability (ie: ensure the call is paired with a tick_nohz_idle_enter() call).
--- kernel/time/tick-sched.c | 11 ++++++----- 1 files changed, 6 insertions(+), 5 deletions(-)
kernel/time/tick-sched.c +6 -5
@@ -558,20 +558,21 @@ void tick_nohz_idle_exit(void)local_irq_disable (); -if (ts->idle_active || (ts->inidle && ts->tick_stopped)) +WARN_ON_ONCE (!ts->inidle); + + ts->inidle =0 ; + +if (ts->idle_active || ts->tick_stopped) now =ktime_get ();if (ts->idle_active)tick_nohz_stop_idle (cpu, now); -if (!ts->inidle || !ts->tick_stopped) { - ts->inidle =0 ; +if (!ts->tick_stopped) {local_irq_enable ();return ; } - ts->inidle =0 ; -/* Update jiffies first */ select_nohz_load_balancer (0 );tick_do_update_jiffies64 (now);
PATCH 3/3 timer: Fix bad idle check on irq entry
In reply to Frederic Weisbeckeridle_cpu() is called on irq entry to guess if we need to call tick_check_idle(). This way we can catch up with jiffies if the tick was stopped, stop accounting idle time during the interrupt and maintain the sched clock if it is unstable.
But if we are going to exit the idle loop to schedule a new task (ie: if we have a task in the runqueue or a remotely enqueued ttwu to perform), the idle_cpu() check will return 0 such that we miss the call to tick_check_idle() for all interrupts happening before we schedule the new task.
As a result these interrupts and the softirqs coming along may deal with stale jiffies values, bad sched clock values, and won't substract their time from the idle time accounting.
Fix this with using is_idle_task() instead that strictly checks that we are running the idle task, without caring about the fact we are going to schedule a task soon.
--- kernel/softirq.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/softirq.c +1 -1
@@ -297,7 +297,7 @@ void irq_enter(void)int cpu =smp_processor_id ();rcu_irq_enter (); -if (idle_cpu (cpu) && !in_interrupt ()) { +if (is_idle_task (current) && !in_interrupt ()) {/* * Prevent raise_softirq from needlessly waking up ksoftirqd * here, as softirq will be serviced on return from interrupt.
Signature · 1 line
1.7.5.4
[tip:timers/core] timer: Fix bad idle check on irq entry
Commit-ID: 0a8a2e78b7eece7c65884fcff9f98dc0fce89ee4 Gitweb: http://git.kernel.org/tip/0a8a2e78b7eece7c65884fcff9f98dc0fce89ee4 Author: Frederic Weisbecker <[email protected]> AuthorDate: Tue, 24 Jan 2012 18:59:44 +0100 Committer: Thomas Gleixner <[email protected]> CommitDate: Wed, 15 Feb 2012 15:23:09 +0100
timer: Fix bad idle check on irq entry
idle_cpu() is called on irq entry to guess if we need to call tick_check_idle(). This way we can catch up with jiffies if the tick was stopped, stop accounting idle time during the interrupt and maintain the sched clock if it is unstable.
But if we are going to exit the idle loop to schedule a new task (ie:
if we have a task in the runqueue or a remotely enqueued ttwu to perform), the idle_cpu() check will return 0 such that we miss the call to tick_check_idle() for all interrupts happening before we schedule the new task.
As a result these interrupts and the softirqs coming along may deal with stale jiffies values, bad sched clock values, and won't substract their time from the idle time accounting.
Fix this with using is_idle_task() instead that strictly checks that we are running the idle task, without caring about the fact we are going to schedule a task soon.
--- kernel/softirq.c | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
kernel/softirq.c +1 -1
@@ -297,7 +297,7 @@ void irq_enter(void)int cpu =smp_processor_id ();rcu_irq_enter (); -if (idle_cpu (cpu) && !in_interrupt ()) { +if (is_idle_task (current) && !in_interrupt ()) {/* * Prevent raise_softirq from needlessly waking up ksoftirqd * here, as softirq will be serviced on return from interrupt.