timers/nohz: Update nohz load even if tick already stopped
Subsystems (2)
Frederic Weisbecker
maintainerPATCH 1/1 timers/nohz: Update nohz load even if tick already stopped
From: Scott Wood <[email protected]>
The way loadavg is tracked during nohz only pays attention to the load upon entering nohz. This can be particularly noticeable if nohz is entered while non-idle, and then the cpu goes idle and stays that way for a long time. We've had reports of a loadavg near 150 on a mostly idle system.
Calling calc_load_nohz_start() regardless of whether the tick is already stopped addresses the issue when going idle. Tracking load changes when not going idle (e.g. multiple SCHED_FIFO tasks coming and going) is not addressed by this patch.
--- kernel/time/tick-sched.c | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/time/tick-sched.c +3 -1
@@ -763,6 +763,9 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu) ts->do_timer_last =0 ; } +/* Even if the tick was already stopped, load may have changed */ +calc_load_nohz_start (); +/* Skip reprogram of event if its not changed */ if (ts->tick_stopped && (expires == ts->next_tick)) {/* Sanity check: make sure clockevent is actually programmed */
@@ -783,7 +786,6 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu)* the scheduler tick in nohz_restart_sched_tick. */ if (!ts->tick_stopped) { -calc_load_nohz_start ();quiet_vmstat (); ts->last_tick =hrtimer_get_expires (&ts->sched_timer);
Signature · 1 line
2.23.0
Peter Zijlstra
maintainerQuoted 8 lines · Frederic Weisbecker
From: Scott Wood <[email protected]>
The way loadavg is tracked during nohz only pays attention to the load upon entering nohz. This can be particularly noticeable if nohz is entered while non-idle, and then the cpu goes idle and stays that way for a long time. We've had reports of a loadavg near 150 on a mostly idle system.
Quoted 4 lines
Calling
calc_load_nohz_start()regardless of whether the tick is already stopped addresses the issue when going idle. Tracking load changes when not going idle (e.g. multipleSCHED_FIFOtasks coming and going) is not addressed by this patch.
Hurph, is that phenomena you describe NOHZ or NOHZ_FULL? Because that second thing you talk about, multiple SCHED_FIFO tasks running without a tick is definitely NOHZ_FULL.
I'm thinking all of this is NOHZ_FULL because IIRC we always start the tick when there is a runnable task. So your example of going idle in NOHZ already cannot happen for regular NOHZ.
And for loadavg vs NOHZ_FULL I don't much like this patch. It ductapes one symptom but doesn't address the actual issue. Let me go rediscover how loadavg-nohz works again, I so hate all this crap :/
Bah, I can't get me head straight, but I'm thinking something like the below ought to do. Does it actually work?
--- include/linux/sched/nohz.h | 2 ++ kernel/sched/core.c | 3 +++ kernel/sched/loadavg.c | 26 +++++++++++++++++++------- 3 files changed, 24 insertions(+), 7 deletions(-)
include/linux/sched/nohz.h +2 -0
@@ -16,9 +16,11 @@ static inline void nohz_balance_enter_idle(int cpu) { }#ifdef CONFIG_NO_HZ_COMMON void calc_load_nohz_start (void );void calc_load_nohz_stop (void ); +void calc_load_nohz_remote (int cpu);#else static inline void calc_load_nohz_start (void ) { }static inline void calc_load_nohz_stop (void ) { } +static inline void calc_load_nohz_remote (int cpu) { }#endif /* CONFIG_NO_HZ_COMMON */ #if defined(CONFIG_NO_HZ_COMMON ) && defined(CONFIG_SMP )
kernel/sched/core.c +3 -0
@@ -3666,6 +3666,8 @@ static void sched_tick_remote(struct work_struct *work)* having one too much is no big deal because the scheduler tick updates * statistics and checks timeslices in a time-independent way, regardless * of when exactly it is running. +* +* XXX should we be checking tick_nohz_tick_stopped_cpu() under rq->lock ? */ if (idle_cpu (cpu) || !tick_nohz_tick_stopped_cpu (cpu)) goto out_requeue;
@@ -3686,6 +3688,7 @@ static void sched_tick_remote(struct work_struct *work) curr->sched_class->task_tick (rq, curr,0 ); out_unlock: +calc_load_nohz_remote (cpu);rq_unlock_irq (rq, &rf); out_requeue:
kernel/sched/loadavg.c +19 -7
@@ -231,16 +231,11 @@ static inline int calc_load_read_idx(void)return calc_load_idx &1 ; } -void calc_load_nohz_start (void ) +static void calc_load_nohz_fold (struct rq *rq) { -struct rq *this_rq =this_rq ();long delta; -/* -* We're going into NO_HZ mode, if there's any pending delta, fold it -* into the pending NO_HZ delta. -*/ - delta =calc_load_fold_active (this_rq,0 ); + delta =calc_load_fold_active (rq,0 );if (delta) {int idx =calc_load_write_idx ();
@@ -248,6 +243,23 @@ void calc_load_nohz_start(void) } } +void calc_load_nohz_start (void ) +{ +/* +* We're going into NO_HZ mode, if there's any pending delta, fold it +* into the pending NO_HZ delta. +*/ +calc_load_nohz_fold (this_rq ()); +} + +void calc_load_nohz_remote (int cpu) +{ +/* +* XXX comment goes here. +*/ +calc_load_nohz_fold (cpu_rq (cpu)); +} +void calc_load_nohz_stop (void ) {struct rq *this_rq =this_rq ();
Quoted 15 lines · Peter Zijlstra
Quoted 9 lines · Frederic Weisbecker
From: Scott Wood <[email protected]>
The way loadavg is tracked during nohz only pays attention to the load upon entering nohz. This can be particularly noticeable if nohz is entered while non-idle, and then the cpu goes idle and stays that way for
a long time. We've had reports of a loadavg near 150 on a mostly idle system.Quoted 4 lines
Calling
calc_load_nohz_start()regardless of whether the tick is already stopped addresses the issue when going idle. Tracking load changes when not going idle (e.g. multipleSCHED_FIFOtasks coming and going) is not addressed by this patch.
Quoted 7 lines
Hurph, is that phenomena you describe NOHZ or
NOHZ_FULL? Because that second thing you talk about, multipleSCHED_FIFOtasks running without a tick is definitelyNOHZ_FULL.I'm thinking all of this is
NOHZ_FULLbecause IIRC we always start the tick when there is a runnable task. So your example of going idle in NOHZ already cannot happen for regular NOHZ.
Yes, NOHZ_FULL (sorry for not stating that clearly).
Quoted 18 lines
kernel/sched/core.c +0
-0@@ -3666,6 +3666,8 @@ static void sched_tick_remote(struct work_struct *work) * having one too much is no big deal because the scheduler tickupdates
* statistics and checks timeslices in a time-independent way,
regardlesskernel/sched/core.c +2
-0* of when exactly it is running. +* +* XXX should we be checking tick_nohz_tick_stopped_cpu() under rq- Quoted 1 line
lock ?
kernel/sched/core.c +0
-0*/ if (idle_cpu (cpu) || !tick_nohz_tick_stopped_cpu (cpu)) goto out_requeue;
kernel/sched/core.c static void sched_tick_remote(struct work_struct · line 3693
Quoted 6 lines
kernel/sched/core.c +1
-0@@ -3686,6 +3688,7 @@ static void sched_tick_remote(struct work_struct *work) curr->sched_class->task_tick (rq, curr,0 ); out_unlock: +calc_load_nohz_remote (cpu);rq_unlock_irq (rq, &rf);
This gets skipped when the cpu is idle, so it still misses the update.
-Scott
Peter Zijlstra
maintainerQuoted 11 lines · Scott Wood
Review static void sched_tick_remote(struct work_struct · line 3693
Quoted 7 lines · Peter Zijlstra
(quoted diff) +1
-0@@ -3686,6 +3688,7 @@ static void sched_tick_remote(struct work_struct *work) curr->sched_class->task_tick(rq, curr, 0); out_unlock: + calc_load_nohz_remote(cpu); rq_unlock_irq(rq, &rf);This gets skipped when the cpu is idle, so it still misses the update.
Oh argh! that's a bit radical of the remote tick. The normal tick runs just fine on idle CPUs, so lets mirror that.
How's this then?
---
include/linux/sched/nohz.h +2 -0
@@ -15,9 +15,11 @@ static inline void nohz_balance_enter_idle(int cpu) { }#ifdef CONFIG_NO_HZ_COMMON void calc_load_nohz_start (void ); +void calc_load_nohz_remote (struct rq *rq);void calc_load_nohz_stop (void );#else static inline void calc_load_nohz_start (void ) { } +static inline void calc_load_nohz_remote (struct rq *rq) { }static inline void calc_load_nohz_stop (void ) { }#endif /* CONFIG_NO_HZ_COMMON */
kernel/sched/core.c +8 -11
@@ -3660,21 +3660,17 @@ static void sched_tick_remote(struct work_struct *work)u64 delta;int os; -/* -* Handle the tick only if it appears the remote CPU is running in full -* dynticks mode. The check is racy by nature, but missing a tick or -* having one too much is no big deal because the scheduler tick updates -* statistics and checks timeslices in a time-independent way, regardless -* of when exactly it is running. -*/ -if (idle_cpu (cpu) || !tick_nohz_tick_stopped_cpu (cpu)) +if (!tick_nohz_tick_stopped_cpu (cpu)) goto out_requeue;rq_lock_irq (rq, &rf); - curr = rq->curr; -if (is_idle_task (curr) ||cpu_is_offline (cpu)) +/* +* We must not call calc_load_nohz_remote() when not in NOHZ mode. +*/ +if (cpu_is_offline (cpu) || !tick_nohz_tick_stopped (cpu)) goto out_unlock; + curr = rq->curr;update_rq_clock (rq); delta =rq_clock_task (rq) - curr->se.exec_start;
@@ -3685,10 +3681,11 @@ static void sched_tick_remote(struct work_struct *work)WARN_ON_ONCE (delta > (u64 )NSEC_PER_SEC *3 ); curr->sched_class->task_tick (rq, curr,0 ); +calc_load_nohz_remote (rq); out_unlock:rq_unlock_irq (rq, &rf); - out_requeue: +/* * Run the remote tick once per second (1Hz). This arbitrary * frequency is large enough to avoid overload but short enough
kernel/sched/loadavg.c +23 -10
@@ -231,16 +231,11 @@ static inline int calc_load_read_idx(void)return calc_load_idx &1 ; } -void calc_load_nohz_start (void ) +static void calc_load_nohz_fold (struct rq *rq) { -struct rq *this_rq =this_rq ();long delta; -/* -* We're going into NO_HZ mode, if there's any pending delta, fold it -* into the pending NO_HZ delta. -*/ - delta =calc_load_fold_active (this_rq,0 ); + delta =calc_load_fold_active (rq,0 );if (delta) {int idx =calc_load_write_idx ();
@@ -248,6 +243,24 @@ void calc_load_nohz_start(void) } } +void calc_load_nohz_start (void ) +{ +/* +* We're going into NO_HZ mode, if there's any pending delta, fold it +* into the pending NO_HZ delta. +*/ +calc_load_nohz_fold (this_rq ()); +} + +/* +* Keep track of the load for NOHZ_FULL, must be called between +* calc_load_nohz_{start,stop}(). +*/ +void calc_load_nohz_remote (struct rq *rq) +{ +calc_load_nohz_fold (rq); +} +void calc_load_nohz_stop (void ) {struct rq *this_rq =this_rq ();
@@ -268,7 +281,7 @@ void calc_load_nohz_stop(void) this_rq->calc_load_update +=LOAD_FREQ ; } -static long calc_load_nohz_fold (void ) +static long calc_load_nohz_read (void ) {int idx =calc_load_read_idx ();long delta =0 ;
@@ -323,7 +336,7 @@ static void calc_global_nohz(void) }#else /* !CONFIG_NO_HZ_COMMON */ -static inline long calc_load_nohz_fold (void ) {return 0 ; } +static inline long calc_load_nohz_read (void ) {return 0 ; }static inline void calc_global_nohz (void ) { }#endif /* CONFIG_NO_HZ_COMMON */
@@ -346,7 +359,7 @@ void calc_global_load(unsigned long ticks)/* * Fold the 'old' NO_HZ-delta to include all NO_HZ CPUs. */ - delta =calc_load_nohz_fold (); + delta =calc_load_nohz_read ();if (delta)atomic_long_add (delta, &calc_load_tasks);
Quoted 55 lines · Peter Zijlstra
Quoted 10 lines · Scott Wood
kernel/sched/core.c static void sched_tick_remote(struct work_struct · line 3693
Quoted 7 lines · Peter Zijlstra
kernel/sched/core.c +1
-0@@ -3686,6 +3688,7 @@ static void sched_tick_remote(struct work_struct *work) curr->sched_class->task_tick (rq, curr,0 ); out_unlock: +calc_load_nohz_remote (cpu);rq_unlock_irq (rq, &rf);This gets skipped when the cpu is idle, so it still misses the update.
Oh argh! that's a bit radical of the remote tick. The normal tick runs just fine on idle CPUs, so lets mirror that.
How's this then?
---
include/linux/sched/nohz.h +2
-0@@ -15,9 +15,11 @@ static inline void nohz_balance_enter_idle(int cpu) { }#ifdef CONFIG_NO_HZ_COMMON void calc_load_nohz_start (void ); +void calc_load_nohz_remote (struct rq *rq);void calc_load_nohz_stop (void );#else static inline void calc_load_nohz_start (void ) { } +static inline void calc_load_nohz_remote (struct rq *rq) { }static inline void calc_load_nohz_stop (void ) { }#endif /* CONFIG_NO_HZ_COMMON */ kernel/sched/core.c +0
-2@@ -3660,21 +3660,17 @@ static void sched_tick_remote(struct work_struct *work)u64 delta;int os; -/* -* Handle the tick only if it appears the remote CPU is running in full
kernel/sched/core.c +0
-2-* dynticks mode. The check is racy by nature, but missing a tick or -* having one too much is no big deal because the scheduler tick updates
kernel/sched/core.c +0
-1-* statistics and checks timeslices in a time-independent way, regardless
kernel/sched/core.c +1
-3-* of when exactly it is running. -*/ -if (idle_cpu (cpu) || !tick_nohz_tick_stopped_cpu (cpu)) +if (!tick_nohz_tick_stopped_cpu (cpu))
kernel/sched/core.c static void sched_tick_remote(struct work_struct *work) · line 3671
Quoted 10 lines
kernel/sched/core.c +4
-2goto out_requeue;rq_lock_irq (rq, &rf); - curr = rq->curr; -if (is_idle_task (curr) ||cpu_is_offline (cpu)) +/* +* We must not call calc_load_nohz_remote() when not in NOHZ mode. +*/ +if (cpu_is_offline (cpu) || !tick_nohz_tick_stopped (cpu)) goto out_unlock;
Needs to be tick_nohz_tick_stopped_cpu(cpu)
After fixing that, I get:
[ 7.439068] WARNING: CPU: 20 PID: 7 at /home/root/linux/kernel/sched/core.c:3681 sched_tick_remote+0x132/0x150 [ 7.439068] Modules linked in: [ 7.439068] CPU: 20 PID: 7 Comm: kworker/u209:0 Not tainted 5.4.0-rc5.std+ #15 [ 7.439068] Hardware name: Intel Corporation S2600BT/S2600BT, BIOS SE5C620.86B.01.00.0763.022420181017 02/24/2018 [ 7.439068] Workqueue: events_unbound sched_tick_remote [ 7.446308] pci_bus 0000:9f: resource 1 [mem 0xe6a00000-0xe6bfffff] [ 7.455068] RIP: 0010:sched_tick_remote+0x132/0x150 [ 7.455068] Code: 00 e9 b2 fd fe ff 0f 0b e9 46 ff ff ff 83 f8 02 89 c2 74 d3 8d 4a ff 89 d0 f0 0f b1 0e 0f 94 c1 84 c9 0f 85 23 ff ff ff eb e3 <0f> 0b eb 9a 80 3d 9c d6 2c 01 00 0f 1f 00 0f 85 71 ff ff ff e8 05 [ 7.455068] RSP: 0000:ffffc9000c683e58 EFLAGS: 00010002 [ 7.455068] RAX: 00000000e7061da1 RBX: ffff8897e026e688 RCX: 0000000181f93295 [ 7.455068] RDX: 00000000b2d05e00 RSI: ffff8897e0269e50 RDI: 0000000000000004 [ 7.455068] RBP: ffff8881004c0000 R08: ffff8e8191a2b423 R09: 0000000000000000 [ 7.455068] R10: 0000000000000010 R11: 0000000000000018 R12: ffff8897e0269240 [ 7.455068] R13: ffff8897e0240000 R14: 0000000000000000 R15: ffff888107edc2e8 [ 7.455068] FS: 0000000000000000(0000) GS:ffff8897e0700000(0000) knlGS:0000000000000000 [ 7.455068] CS: 0010 DS: 0000 ES: 0000 CR0: 0000000080050033 [ 7.455068] CR2: 0000000000000000 CR3: 000000303e60a001 CR4: 00000000007606e0 [ 7.455068] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000 [ 7.459338] pci_bus 0000:9f: resource 2 [mem 0x3a0000000000-0x3a00001fffff 64bit pref] [ 7.465068] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400 [ 7.465068] PKRU: 55555554 [ 7.465068] Call Trace: [ 7.465068] process_one_work+0x165/0x3c0 [ 7.465068] worker_thread+0x46/0x3d0 [ 7.465068] kthread+0xf8/0x130 [ 7.465068] ? process_one_work+0x3c0/0x3c0 [ 7.476788] pci_bus 0000:a0: resource 1 [mem 0xe6c00000-0xe6dfffff] [ 7.465068] ? kthread_bind+0x10/0x10 [ 7.465068] ret_from_fork+0x35/0x40
-Scott
Quoted 5 lines · Scott Wood
Quoted 4 lines · Peter Zijlstra
Oh argh! that's a bit radical of the remote tick. The normal tick runs just fine on idle CPUs, so lets mirror that.
How's this then?
....
Quoted 6 lines
Needs to be
tick_nohz_tick_stopped_cpu(cpu)After fixing that, I get:
[ 7.439068] WARNING: CPU: 20 PID: 7 at /home/root/linux/kernel/sched/core.c:3681 sched_tick_remote+0x132/0x150
So I'm going to apply Scotts patch if nobody comes up with a better idea until tomorrow.
Thanks,
tglx
Quoted 10 lines · Thomas Gleixner
Quoted 5 lines · Scott Wood
Quoted 4 lines · Peter Zijlstra
Oh argh! that's a bit radical of the remote tick. The normal tick runs just fine on idle CPUs, so lets mirror that.
How's this then?
....
Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Needs to be
tick_nohz_tick_stopped_cpu(cpu)After fixing that, I get:
[ 7.439068] WARNING: CPU: 20 PID: 7 at /home/root/linux/kernel/sched/core.c:3681 sched_tick_remote+0x132/0x150So I'm going to apply Scotts patch if nobody comes up with a better idea until tomorrow.
As Peter pointed out to me privately we should rather go and analyze the real thing instead of just applying duct tape.
/me drops the patch again.
Quoted 19 lines · Thomas Gleixner
Quoted 18 lines · Thomas Gleixner
Quoted 6 lines · Scott Wood
Quoted 5 lines · Peter Zijlstra
Oh argh! that's a bit radical of the remote tick. The normal tick runs
just fine on idle CPUs, so lets mirror that.How's this then?
....
Quoted 7 lines
Needs to be
tick_nohz_tick_stopped_cpu(cpu)After fixing that, I get:
[ 7.439068] WARNING: CPU: 20 PID: 7 at
/home/root/linux/kernel/sched/core.c:3681sched_tick_remote+0x132/0x150
Quoted 7 lines
Quoted 2 lines
So I'm going to apply Scotts patch if nobody comes up with a better idea until tomorrow.
As Peter pointed out to me privately we should rather go and analyze the real thing instead of just applying duct tape.
/me drops the patch again.
The warning is due to kernel/sched/idle.c not updating curr->se.exec_start.
While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice tick_period. When the tick is started, the timer expiration is more than tick_period in the past, so hrtimer_forward() tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running. sched_tick_remote() does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are calling calc_load_nohz_remote().
Is there a reason to not just remove the hrtimer_forward() from tick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doing hrtimer_forward()?
As for the warning in sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying on curr->se.exec_start, in order to detect things like this.
-Scott
Quoted 19 lines · Scott Wood
Quoted 4 lines · Thomas Gleixner
As Peter pointed out to me privately we should rather go and analyze the real thing instead of just applying duct tape.
/me drops the patch again.
The warning is due to
kernel/sched/idle.cnot updatingcurr->se.exec_start.While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice
tick_period. When the tick is started, the timer expiration is more thantick_periodin the past, sohrtimer_forward()tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running.sched_tick_remote()does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are callingcalc_load_nohz_remote().
Quoted 3 lines
Is there a reason to not just remove the
hrtimer_forward()fromtick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doinghrtimer_forward()?
Well, no. tick_nohz_restart() can be invoked in a situation where the timer is armed for something in the far future (or completelt disabled) due to previously entering an estimated long idle (or user space execution on NOHZ_FULL) period.
That means if the timer is not canceled, realigned to the current tick and forwarded to the next due tick, the tick will not fire on time causing another sort of trouble.
Quoted 3 lines
As for the warning in
sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying oncurr->se.exec_start, in order to detect things like this.
Care to give that a shot?
Thanks,
tglx
Peter Zijlstra
maintainerQuoted 1 line · Scott Wood
The warning is due to
kernel/sched/idle.cnot updatingcurr->se.exec_start.
Ah, indeed so.
Quoted 11 lines
While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice
tick_period. When the tick is started, the timer expiration is more thantick_periodin the past, sohrtimer_forward()tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running.sched_tick_remote()does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are callingcalc_load_nohz_remote().
Quoted 3 lines
Is there a reason to not just remove the
hrtimer_forward()fromtick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doinghrtimer_forward()?
I'll have to look into that. I always get confused by all that nohz code :/
Quoted 3 lines
As for the warning in
sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying oncurr->se.exec_start, in order to detect things like this.
I don't think we have a timestamp that is shared between the remote and local tick. Also, there is a reason this warning uses the task time accounting, there used to be (as in, I can't find it in a hurry) code that could not deal with >u32 (~4s) clock updates.
The below should have idle keep the timestamp up-to-date. Keeping accurate idle->se.sum_exec_runtime doesn't seem too interesting, the idle code already keeps track of total idle times.
---
kernel/sched/idle.c +2 -0
@@ -381,6 +381,7 @@ static void put_prev_task_idle(struct rqstatic void set_next_task_idle (struct rq *rq,struct task_struct *next) { + curr->se.exec_start =rq_clock_task (rq);update_idle_core (rq);schedstat_inc (rq->sched_goidle); }
@@ -417,6 +418,7 @@ dequeue_task_idle(struct rq *rq, struct*/ static void task_tick_idle (struct rq *rq,struct task_struct *curr,int queued) { + curr->se.exec_start =rq_clock_task (rq); }static void switched_to_idle (struct rq *rq,struct task_struct *p)
Peter Zijlstra
maintainerQuoted 32 lines · Peter Zijlstra
Quoted 1 line · Scott Wood
The warning is due to
kernel/sched/idle.cnot updatingcurr->se.exec_start.Ah, indeed so.
Quoted 11 lines
While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice
tick_period. When the tick is started, the timer expiration is more thantick_periodin the past, sohrtimer_forward()tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running.sched_tick_remote()does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are callingcalc_load_nohz_remote().Quoted 3 lines
Is there a reason to not just remove the
hrtimer_forward()fromtick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doinghrtimer_forward()?I'll have to look into that. I always get confused by all that nohz code :/
Quoted 3 lines
As for the warning in
sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying oncurr->se.exec_start, in order to detect things like this.I don't think we have a timestamp that is shared between the remote and local tick. Also, there is a reason this warning uses the task time accounting, there used to be (as in, I can't find it in a hurry) code that could not deal with >u32 (~4s) clock updates.
Quoted 5 lines
The below should have idle keep the timestamp up-to-date. Keeping accurate
idle->se.sum_exec_runtimedoesn't seem too interesting, the idle code already keeps track of total idle times.---
The obvious alternative is something like:
if (rq->curr != rq->idle) {
s64 delta = rq_clock_task(rq) - curr->se.exec_start;
WARN_ON_ONCE(delta > 3ULL * NSEC_PER_SEC);
}Which would avoid polluting the idle path with that extra assignment.
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Scott Wood
As for the warning in
sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than
relying oncurr->se.exec_start, in order to detect things like this.I don't think we have a timestamp that is shared between the remote and local tick.
Why wouldn't rq_clock_task() work on the local tick? It's what ->task_tick() itself uses.
Quoted 3 lines
Also, there is a reason this warning uses the task time accounting, there used to be (as in, I can't find it in a hurry) code that could not deal with >u32 (~4s) clock updates.
Detecting a 3 second interval between ticks for a given cpu should assert in a superset of the situations the current check asserts in -- it just avoids the false negative of exec_runtime getting updated by something other than the tick.
-Scott
Quoted 41 lines · Thomas Gleixner
Quoted 29 lines · Scott Wood
Quoted 5 lines · Thomas Gleixner
As Peter pointed out to me privately we should rather go and analyze the
real thing instead of just applying duct tape./me drops the patch again.
The warning is due to
kernel/sched/idle.cnot updating curr-Quoted 1 line
se.exec_start.
While debugging I noticed an issue with a particular load pattern. The CPU
goes non-nohz for a brief time at an interval very close to twicetick_period. When the tick is started, the timer expiration is more thantick_periodin the past, sohrtimer_forward()tries to catch up by adding
2*tick_period to the expiration. Then the tick is stopped before that new
expiration, and when the tick is woken up the expiry is again advanced by
2*tick_period with the timer never actually
running.sched_tick_remote()
does fire every second, but there are streaks of several seconds where it
keeps catching the CPU in a non-nohz state, so neither the normal nor remote
ticks are callingcalc_load_nohz_remote().Quoted 4 lines
Is there a reason to not just remove the
hrtimer_forward()fromtick_nohz_restart(), letting the timer fire if it's in the past, which will
take care of doinghrtimer_forward()?Well, no.
tick_nohz_restart()can be invoked in a situation where the timer
is armed for something in the far future (or completelt disabled) due to previously entering an estimated long idle (or user space execution onNOHZ_FULL) period.
Quoted 3 lines
That means if the timer is not canceled, realigned to the current tick and forwarded to the next due tick, the tick will not fire on time causing another sort of trouble.
That might be true of the expiry on entering tick_nohz_restart(), but it shouldn't be true of ts->last_tick which the expiry is set to before calling hrtimer_forward() -- and if it were, hrtimer_forward() is a no-op when the expiry is in the future.
BTW, the name "last_tick" seems misleading as it's actually the saved expiry, not the last time the tick ran.
-Scott
Quoted 14 lines · Scott Wood
Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Scott Wood
As for the warning in
sched_tick_remote(), it seems like a test for time
since the last tick on this cpu (remote or otherwise) would be better than
relying oncurr->se.exec_start, in order to detect things like this.I don't think we have a timestamp that is shared between the remote and local tick.
Why wouldn't
rq_clock_task()work on the local tick? It's what ->task_tick() itself uses.
Quoted 8 lines
Quoted 3 lines
Also, there is a reason this warning uses the task time accounting, there used to be (as in, I can't find it in a hurry) code that could not deal with >u32 (~4s) clock updates.
Detecting a 3 second interval between ticks for a given cpu should assert in a superset of the situations the current check asserts in -- it just avoids the false negative of
exec_runtimegetting updated by something other than the tick.
The main difficulty with such a check is that when we're not on a full nohz cpu, there's no remote tick, and so we can legitimately go more than 3 seconds between ticks when idle.
-Scott
Quoted 21 lines · Peter Zijlstra
kernel/sched/core.c +0
-2@@ -3660,21 +3660,17 @@ static void sched_tick_remote(struct work_struct *work)u64 delta;int os; -/* -* Handle the tick only if it appears the remote CPU is running in full
kernel/sched/core.c +0
-2-* dynticks mode. The check is racy by nature, but missing a tick or -* having one too much is no big deal because the scheduler tick updates
kernel/sched/core.c +0
-1-* statistics and checks timeslices in a time-independent way, regardless
kernel/sched/core.c +1
-3-* of when exactly it is running. -*/ -if (idle_cpu (cpu) || !tick_nohz_tick_stopped_cpu (cpu)) +if (!tick_nohz_tick_stopped_cpu (cpu))
kernel/sched/core.c static void sched_tick_remote(struct work_struct *work) · line 3671
Quoted 10 lines
kernel/sched/core.c +4
-2goto out_requeue;rq_lock_irq (rq, &rf); - curr = rq->curr; -if (is_idle_task (curr) ||cpu_is_offline (cpu)) +/* +* We must not call calc_load_nohz_remote() when not in NOHZ mode. +*/ +if (cpu_is_offline (cpu) || !tick_nohz_tick_stopped (cpu)) goto out_unlock;
Is it really a problem if calc_load_nohz_remote() gets called in non-NOHZ? It won't race due to rq lock -- and we're already mixing remote and non-remote updates because the normal tick timer can still be run while "stopped".
-Scott