Skip to contentSkip to conversation

timers/nohz: Update nohz load even if tick already stopped

PATCH In review Frederic Weisbecker 4 participants 15 messages Oct 28, 2019 – Dec 11, 2019

Subsystems (2)
lore
Share & tools
lore

Frederic Weisbecker

author maintainer
#0 lore raw

PATCH 1/1 timers/nohz: Update nohz load even if tick already stopped

From: Scott Wood <[email protected]>

The way loadavg is tracked during nohz only pays attention to the load upon entering nohz. This can be particularly noticeable if nohz is entered while non-idle, and then the cpu goes idle and stays that way for a long time. We've had reports of a loadavg near 150 on a mostly idle system.

Calling calc_load_nohz_start() regardless of whether the tick is already stopped addresses the issue when going idle. Tracking load changes when not going idle (e.g. multiple SCHED_FIFO tasks coming and going) is not addressed by this patch.

Cc Thomas Gleixner <[email protected]>
Cc Ingo Molnar <[email protected]>
Cc Peter Zijlstra <[email protected]>
---
 kernel/time/tick-sched.c | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
kernel/time/tick-sched.c +3 -1
index c2748232f607..f55ddeb652a3 100644
@@ -763,6 +763,9 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu)
 		ts->do_timer_last = 0;
 	}
 
+	/* Even if the tick was already stopped, load may have changed */
+	calc_load_nohz_start();
+
 	/* Skip reprogram of event if its not changed */
 	if (ts->tick_stopped && (expires == ts->next_tick)) {
 		/* Sanity check: make sure clockevent is actually programmed */
@@ -783,7 +786,6 @@ static void tick_nohz_stop_tick(struct tick_sched *ts, int cpu)
 	 * the scheduler tick in nohz_restart_sched_tick.
 	 */
 	if (!ts->tick_stopped) {
-		calc_load_nohz_start();
 		quiet_vmstat();
 
 		ts->last_tick = hrtimer_get_expires(&ts->sched_timer);
Signature · 1 line
2.23.0
maintainer
#1 lore raw
Quoted 8 lines · Frederic Weisbecker

From: Scott Wood <[email protected]>

The way loadavg is tracked during nohz only pays attention to the load upon entering nohz. This can be particularly noticeable if nohz is entered while non-idle, and then the cpu goes idle and stays that way for a long time. We've had reports of a loadavg near 150 on a mostly idle system.

Quoted 4 lines

Calling calc_load_nohz_start() regardless of whether the tick is already stopped addresses the issue when going idle. Tracking load changes when not going idle (e.g. multiple SCHED_FIFO tasks coming and going) is not addressed by this patch.

Hurph, is that phenomena you describe NOHZ or NOHZ_FULL? Because that second thing you talk about, multiple SCHED_FIFO tasks running without a tick is definitely NOHZ_FULL.

I'm thinking all of this is NOHZ_FULL because IIRC we always start the tick when there is a runnable task. So your example of going idle in NOHZ already cannot happen for regular NOHZ.

And for loadavg vs NOHZ_FULL I don't much like this patch. It ductapes one symptom but doesn't address the actual issue. Let me go rediscover how loadavg-nohz works again, I so hate all this crap :/

Bah, I can't get me head straight, but I'm thinking something like the below ought to do. Does it actually work?

---
 include/linux/sched/nohz.h |  2 ++
 kernel/sched/core.c        |  3 +++
 kernel/sched/loadavg.c     | 26 +++++++++++++++++++-------
 3 files changed, 24 insertions(+), 7 deletions(-)
include/linux/sched/nohz.h +2 -0
index 1abe91ff6e4a..a2296004351f 100644
@@ -16,9 +16,11 @@ static inline void nohz_balance_enter_idle(int cpu) { }
 #ifdef CONFIG_NO_HZ_COMMON
 void calc_load_nohz_start(void);
 void calc_load_nohz_stop(void);
+void calc_load_nohz_remote(int cpu);
 #else
 static inline void calc_load_nohz_start(void) { }
 static inline void calc_load_nohz_stop(void) { }
+static inline void calc_load_nohz_remote(int cpu) { }
 #endif /* CONFIG_NO_HZ_COMMON */
 
 #if defined(CONFIG_NO_HZ_COMMON) && defined(CONFIG_SMP)
kernel/sched/core.c +3 -0
index eb42b71faab9..209e50d48f80 100644
@@ -3666,6 +3666,8 @@ static void sched_tick_remote(struct work_struct *work)
 	 * having one too much is no big deal because the scheduler tick updates
 	 * statistics and checks timeslices in a time-independent way, regardless
 	 * of when exactly it is running.
+	 *
+	 * XXX should we be checking tick_nohz_tick_stopped_cpu() under rq->lock ?
 	 */
 	if (idle_cpu(cpu) || !tick_nohz_tick_stopped_cpu(cpu))
 		goto out_requeue;
@@ -3686,6 +3688,7 @@ static void sched_tick_remote(struct work_struct *work)
 	curr->sched_class->task_tick(rq, curr, 0);
 
 out_unlock:
+	calc_load_nohz_remote(cpu);
 	rq_unlock_irq(rq, &rf);
 
 out_requeue:
kernel/sched/loadavg.c +19 -7
index 28a516575c18..7549bd9b6853 100644
@@ -231,16 +231,11 @@ static inline int calc_load_read_idx(void)
 	return calc_load_idx & 1;
 }
 
-void calc_load_nohz_start(void)
+static void calc_load_nohz_fold(struct rq *rq)
 {
-	struct rq *this_rq = this_rq();
 	long delta;
 
-	/*
-	 * We're going into NO_HZ mode, if there's any pending delta, fold it
-	 * into the pending NO_HZ delta.
-	 */
-	delta = calc_load_fold_active(this_rq, 0);
+	delta = calc_load_fold_active(rq, 0);
 	if (delta) {
 		int idx = calc_load_write_idx();
 
@@ -248,6 +243,23 @@ void calc_load_nohz_start(void)
 	}
 }
 
+void calc_load_nohz_start(void)
+{
+	/*
+	 * We're going into NO_HZ mode, if there's any pending delta, fold it
+	 * into the pending NO_HZ delta.
+	 */
+	calc_load_nohz_fold(this_rq());
+}
+
+void calc_load_nohz_remote(int cpu)
+{
+	/*
+	 * XXX comment goes here.
+	 */
+	calc_load_nohz_fold(cpu_rq(cpu));
+}
+
 void calc_load_nohz_stop(void)
 {
 	struct rq *this_rq = this_rq();
#2 lore raw
Quoted 15 lines · Peter Zijlstra
Quoted 9 lines · Frederic Weisbecker

From: Scott Wood <[email protected]>

The way loadavg is tracked during nohz only pays attention to the load upon entering nohz. This can be particularly noticeable if nohz is entered while non-idle, and then the cpu goes idle and stays that way for
a long time. We've had reports of a loadavg near 150 on a mostly idle system.

Quoted 4 lines

Calling calc_load_nohz_start() regardless of whether the tick is already stopped addresses the issue when going idle. Tracking load changes when not going idle (e.g. multiple SCHED_FIFO tasks coming and going) is not addressed by this patch.

Quoted 7 lines

Hurph, is that phenomena you describe NOHZ or NOHZ_FULL? Because that second thing you talk about, multiple SCHED_FIFO tasks running without a tick is definitely NOHZ_FULL.

I'm thinking all of this is NOHZ_FULL because IIRC we always start the tick when there is a runnable task. So your example of going idle in NOHZ already cannot happen for regular NOHZ.

Yes, NOHZ_FULL (sorry for not stating that clearly).

Quoted 18 lines
kernel/sched/core.c +0 -0
index eb42b71faab9..209e50d48f80 100644
@@ -3666,6 +3666,8 @@ static void sched_tick_remote(struct work_struct
 *work)
 	 * having one too much is no big deal because the scheduler tick

updates
* statistics and checks timeslices in a time-independent way,
regardless

kernel/sched/core.c +2 -0
 	 * of when exactly it is running.
+	 *
+	 * XXX should we be checking tick_nohz_tick_stopped_cpu() under rq-
Quoted 1 line

lock ?

kernel/sched/core.c +0 -0
 	 */
 	if (idle_cpu(cpu) || !tick_nohz_tick_stopped_cpu(cpu))
 		goto out_requeue;

kernel/sched/core.c static void sched_tick_remote(struct work_struct · line 3693

Quoted 6 lines
kernel/sched/core.c +1 -0
@@ -3686,6 +3688,7 @@ static void sched_tick_remote(struct work_struct
 *work)
 	curr->sched_class->task_tick(rq, curr, 0);
 
 out_unlock:
+	calc_load_nohz_remote(cpu);
 	rq_unlock_irq(rq, &rf);

This gets skipped when the cpu is idle, so it still misses the update.

-Scott

maintainer
#3 lore raw
Quoted 11 lines · Scott Wood

Review static void sched_tick_remote(struct work_struct · line 3693

Quoted 7 lines · Peter Zijlstra
(quoted diff) +1 -0
@@ -3686,6 +3688,7 @@ static void sched_tick_remote(struct work_struct
 *work)
 	curr->sched_class->task_tick(rq, curr, 0);
 
 out_unlock:
+	calc_load_nohz_remote(cpu);
 	rq_unlock_irq(rq, &rf);

This gets skipped when the cpu is idle, so it still misses the update.

Oh argh! that's a bit radical of the remote tick. The normal tick runs just fine on idle CPUs, so lets mirror that.

How's this then?

---

include/linux/sched/nohz.h +2 -0
index 1abe91ff6e4a..6d67e9a5af6b 100644
@@ -15,9 +15,11 @@ static inline void nohz_balance_enter_idle(int cpu) { }
 
 #ifdef CONFIG_NO_HZ_COMMON
 void calc_load_nohz_start(void);
+void calc_load_nohz_remote(struct rq *rq);
 void calc_load_nohz_stop(void);
 #else
 static inline void calc_load_nohz_start(void) { }
+static inline void calc_load_nohz_remote(struct rq *rq) { }
 static inline void calc_load_nohz_stop(void) { }
 #endif /* CONFIG_NO_HZ_COMMON */
 
kernel/sched/core.c +8 -11
index eb42b71faab9..d02d1b8f40af 100644
@@ -3660,21 +3660,17 @@ static void sched_tick_remote(struct work_struct *work)
 	u64 delta;
 	int os;
 
-	/*
-	 * Handle the tick only if it appears the remote CPU is running in full
-	 * dynticks mode. The check is racy by nature, but missing a tick or
-	 * having one too much is no big deal because the scheduler tick updates
-	 * statistics and checks timeslices in a time-independent way, regardless
-	 * of when exactly it is running.
-	 */
-	if (idle_cpu(cpu) || !tick_nohz_tick_stopped_cpu(cpu))
+	if (!tick_nohz_tick_stopped_cpu(cpu))
 		goto out_requeue;
 
 	rq_lock_irq(rq, &rf);
-	curr = rq->curr;
-	if (is_idle_task(curr) || cpu_is_offline(cpu))
+	/*
+	 * We must not call calc_load_nohz_remote() when not in NOHZ mode.
+	 */
+	if (cpu_is_offline(cpu) || !tick_nohz_tick_stopped(cpu))
 		goto out_unlock;
 
+	curr = rq->curr;
 	update_rq_clock(rq);
 	delta = rq_clock_task(rq) - curr->se.exec_start;
 
@@ -3685,10 +3681,11 @@ static void sched_tick_remote(struct work_struct *work)
 	WARN_ON_ONCE(delta > (u64)NSEC_PER_SEC * 3);
 	curr->sched_class->task_tick(rq, curr, 0);
 
+	calc_load_nohz_remote(rq);
 out_unlock:
 	rq_unlock_irq(rq, &rf);
-
 out_requeue:
+
 	/*
 	 * Run the remote tick once per second (1Hz). This arbitrary
 	 * frequency is large enough to avoid overload but short enough
kernel/sched/loadavg.c +23 -10
index 28a516575c18..de22da666ac7 100644
@@ -231,16 +231,11 @@ static inline int calc_load_read_idx(void)
 	return calc_load_idx & 1;
 }
 
-void calc_load_nohz_start(void)
+static void calc_load_nohz_fold(struct rq *rq)
 {
-	struct rq *this_rq = this_rq();
 	long delta;
 
-	/*
-	 * We're going into NO_HZ mode, if there's any pending delta, fold it
-	 * into the pending NO_HZ delta.
-	 */
-	delta = calc_load_fold_active(this_rq, 0);
+	delta = calc_load_fold_active(rq, 0);
 	if (delta) {
 		int idx = calc_load_write_idx();
 
@@ -248,6 +243,24 @@ void calc_load_nohz_start(void)
 	}
 }
 
+void calc_load_nohz_start(void)
+{
+	/*
+	 * We're going into NO_HZ mode, if there's any pending delta, fold it
+	 * into the pending NO_HZ delta.
+	 */
+	calc_load_nohz_fold(this_rq());
+}
+
+/*
+ * Keep track of the load for NOHZ_FULL, must be called between
+ * calc_load_nohz_{start,stop}().
+ */
+void calc_load_nohz_remote(struct rq *rq)
+{
+	calc_load_nohz_fold(rq);
+}
+
 void calc_load_nohz_stop(void)
 {
 	struct rq *this_rq = this_rq();
@@ -268,7 +281,7 @@ void calc_load_nohz_stop(void)
 		this_rq->calc_load_update += LOAD_FREQ;
 }
 
-static long calc_load_nohz_fold(void)
+static long calc_load_nohz_read(void)
 {
 	int idx = calc_load_read_idx();
 	long delta = 0;
@@ -323,7 +336,7 @@ static void calc_global_nohz(void)
 }
 #else /* !CONFIG_NO_HZ_COMMON */
 
-static inline long calc_load_nohz_fold(void) { return 0; }
+static inline long calc_load_nohz_read(void) { return 0; }
 static inline void calc_global_nohz(void) { }
 
 #endif /* CONFIG_NO_HZ_COMMON */
@@ -346,7 +359,7 @@ void calc_global_load(unsigned long ticks)
 	/*
 	 * Fold the 'old' NO_HZ-delta to include all NO_HZ CPUs.
 	 */
-	delta = calc_load_nohz_fold();
+	delta = calc_load_nohz_read();
 	if (delta)
 		atomic_long_add(delta, &calc_load_tasks);
 
#4 lore raw
Quoted 55 lines · Peter Zijlstra
Quoted 10 lines · Scott Wood

kernel/sched/core.c static void sched_tick_remote(struct work_struct · line 3693

Quoted 7 lines · Peter Zijlstra
kernel/sched/core.c +1 -0
@@ -3686,6 +3688,7 @@ static void sched_tick_remote(struct work_struct
 *work)
 	curr->sched_class->task_tick(rq, curr, 0);
 
 out_unlock:
+	calc_load_nohz_remote(cpu);
 	rq_unlock_irq(rq, &rf);

This gets skipped when the cpu is idle, so it still misses the update.

Oh argh! that's a bit radical of the remote tick. The normal tick runs just fine on idle CPUs, so lets mirror that.

How's this then?

---

include/linux/sched/nohz.h +2 -0
index 1abe91ff6e4a..6d67e9a5af6b 100644
@@ -15,9 +15,11 @@ static inline void nohz_balance_enter_idle(int cpu) { }
 
 #ifdef CONFIG_NO_HZ_COMMON
 void calc_load_nohz_start(void);
+void calc_load_nohz_remote(struct rq *rq);
 void calc_load_nohz_stop(void);
 #else
 static inline void calc_load_nohz_start(void) { }
+static inline void calc_load_nohz_remote(struct rq *rq) { }
 static inline void calc_load_nohz_stop(void) { }
 #endif /* CONFIG_NO_HZ_COMMON */
 
kernel/sched/core.c +0 -2
index eb42b71faab9..d02d1b8f40af 100644
@@ -3660,21 +3660,17 @@ static void sched_tick_remote(struct work_struct
 *work)
 	u64 delta;
 	int os;
 
-	/*
-	 * Handle the tick only if it appears the remote CPU is running in

full

kernel/sched/core.c +0 -2
-	 * dynticks mode. The check is racy by nature, but missing a tick or
-	 * having one too much is no big deal because the scheduler tick

updates

kernel/sched/core.c +0 -1
-	 * statistics and checks timeslices in a time-independent way,

regardless

kernel/sched/core.c +1 -3
-	 * of when exactly it is running.
-	 */
-	if (idle_cpu(cpu) || !tick_nohz_tick_stopped_cpu(cpu))
+	if (!tick_nohz_tick_stopped_cpu(cpu))

kernel/sched/core.c static void sched_tick_remote(struct work_struct *work) · line 3671

Quoted 10 lines
kernel/sched/core.c +4 -2
 		goto out_requeue;
 
 	rq_lock_irq(rq, &rf);
-	curr = rq->curr;
-	if (is_idle_task(curr) || cpu_is_offline(cpu))
+	/*
+	 * We must not call calc_load_nohz_remote() when not in NOHZ mode.
+	 */
+	if (cpu_is_offline(cpu) || !tick_nohz_tick_stopped(cpu))
 		goto out_unlock;

Needs to be tick_nohz_tick_stopped_cpu(cpu)

After fixing that, I get:

[    7.439068] WARNING: CPU: 20 PID: 7 at /home/root/linux/kernel/sched/core.c:3681 sched_tick_remote+0x132/0x150
[    7.439068] Modules linked in:
[    7.439068] CPU: 20 PID: 7 Comm: kworker/u209:0 Not tainted 5.4.0-rc5.std+ #15
[    7.439068] Hardware name: Intel Corporation S2600BT/S2600BT, BIOS SE5C620.86B.01.00.0763.022420181017 02/24/2018
[    7.439068] Workqueue: events_unbound sched_tick_remote
[    7.446308] pci_bus 0000:9f: resource 1 [mem 0xe6a00000-0xe6bfffff]
[    7.455068] RIP: 0010:sched_tick_remote+0x132/0x150
[    7.455068] Code: 00 e9 b2 fd fe ff 0f 0b e9 46 ff ff ff 83 f8 02 89 c2 74 d3 8d 4a ff 89 d0 f0 0f b1 0e 0f 94 c1 84 c9 0f 85 23 ff ff ff eb e3 <0f> 0b eb 9a 80 3d 9c d6 2c 01 00 0f 1f 00 0f 85 71 ff ff ff e8 05
[    7.455068] RSP: 0000:ffffc9000c683e58 EFLAGS: 00010002
[    7.455068] RAX: 00000000e7061da1 RBX: ffff8897e026e688 RCX: 0000000181f93295
[    7.455068] RDX: 00000000b2d05e00 RSI: ffff8897e0269e50 RDI: 0000000000000004
[    7.455068] RBP: ffff8881004c0000 R08: ffff8e8191a2b423 R09: 0000000000000000
[    7.455068] R10: 0000000000000010 R11: 0000000000000018 R12: ffff8897e0269240
[    7.455068] R13: ffff8897e0240000 R14: 0000000000000000 R15: ffff888107edc2e8
[    7.455068] FS:  0000000000000000(0000) GS:ffff8897e0700000(0000) knlGS:0000000000000000
[    7.455068] CS:  0010 DS: 0000 ES: 0000 CR0: 0000000080050033
[    7.455068] CR2: 0000000000000000 CR3: 000000303e60a001 CR4: 00000000007606e0
[    7.455068] DR0: 0000000000000000 DR1: 0000000000000000 DR2: 0000000000000000
[    7.459338] pci_bus 0000:9f: resource 2 [mem 0x3a0000000000-0x3a00001fffff 64bit pref]
[    7.465068] DR3: 0000000000000000 DR6: 00000000fffe0ff0 DR7: 0000000000000400
[    7.465068] PKRU: 55555554
[    7.465068] Call Trace:
[    7.465068]  process_one_work+0x165/0x3c0
[    7.465068]  worker_thread+0x46/0x3d0
[    7.465068]  kthread+0xf8/0x130
[    7.465068]  ? process_one_work+0x3c0/0x3c0
[    7.476788] pci_bus 0000:a0: resource 1 [mem 0xe6c00000-0xe6dfffff]
[    7.465068]  ? kthread_bind+0x10/0x10
[    7.465068]  ret_from_fork+0x35/0x40

-Scott

#5 lore raw
Quoted 5 lines · Scott Wood
Quoted 4 lines · Peter Zijlstra

Oh argh! that's a bit radical of the remote tick. The normal tick runs just fine on idle CPUs, so lets mirror that.

How's this then?

....

Quoted 6 lines

Needs to be tick_nohz_tick_stopped_cpu(cpu)

After fixing that, I get:

[    7.439068] WARNING: CPU: 20 PID: 7 at /home/root/linux/kernel/sched/core.c:3681 sched_tick_remote+0x132/0x150

So I'm going to apply Scotts patch if nobody comes up with a better idea until tomorrow.

Thanks,

tglx

#6 lore raw
Quoted 10 lines · Thomas Gleixner
Quoted 5 lines · Scott Wood
Quoted 4 lines · Peter Zijlstra

Oh argh! that's a bit radical of the remote tick. The normal tick runs just fine on idle CPUs, so lets mirror that.

How's this then?

....

Quoted 1 line
Quoted 8 lines
Quoted 5 lines

Needs to be tick_nohz_tick_stopped_cpu(cpu)

After fixing that, I get:

[    7.439068] WARNING: CPU: 20 PID: 7 at /home/root/linux/kernel/sched/core.c:3681 sched_tick_remote+0x132/0x150

So I'm going to apply Scotts patch if nobody comes up with a better idea until tomorrow.

As Peter pointed out to me privately we should rather go and analyze the real thing instead of just applying duct tape.

/me drops the patch again.

#7 lore raw
Quoted 19 lines · Thomas Gleixner
Quoted 18 lines · Thomas Gleixner
Quoted 6 lines · Scott Wood
Quoted 5 lines · Peter Zijlstra

Oh argh! that's a bit radical of the remote tick. The normal tick runs
just fine on idle CPUs, so lets mirror that.

How's this then?

....

Quoted 7 lines

Needs to be tick_nohz_tick_stopped_cpu(cpu)

After fixing that, I get:

[    7.439068] WARNING: CPU: 20 PID: 7 at

/home/root/linux/kernel/sched/core.c:3681
sched_tick_remote+0x132/0x150

Quoted 7 lines
Quoted 2 lines

So I'm going to apply Scotts patch if nobody comes up with a better idea until tomorrow.

As Peter pointed out to me privately we should rather go and analyze the real thing instead of just applying duct tape.

/me drops the patch again.

The warning is due to kernel/sched/idle.c not updating curr->se.exec_start.

While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice tick_period. When the tick is started, the timer expiration is more than tick_period in the past, so hrtimer_forward() tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running. sched_tick_remote() does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are calling calc_load_nohz_remote().

Is there a reason to not just remove the hrtimer_forward() from tick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doing hrtimer_forward()?

As for the warning in sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying on curr->se.exec_start, in order to detect things like this.

-Scott

#8 lore raw
Quoted 19 lines · Scott Wood
Quoted 4 lines · Thomas Gleixner

As Peter pointed out to me privately we should rather go and analyze the real thing instead of just applying duct tape.

/me drops the patch again.

The warning is due to kernel/sched/idle.c not updating curr->se.exec_start.

While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice tick_period. When the tick is started, the timer expiration is more than tick_period in the past, so hrtimer_forward() tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running. sched_tick_remote() does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are calling calc_load_nohz_remote().

Quoted 3 lines

Is there a reason to not just remove the hrtimer_forward() from tick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doing hrtimer_forward()?

Well, no. tick_nohz_restart() can be invoked in a situation where the timer is armed for something in the far future (or completelt disabled) due to previously entering an estimated long idle (or user space execution on NOHZ_FULL) period.

That means if the timer is not canceled, realigned to the current tick and forwarded to the next due tick, the tick will not fire on time causing another sort of trouble.

Quoted 3 lines

As for the warning in sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying on curr->se.exec_start, in order to detect things like this.

Care to give that a shot?

Thanks,

tglx

#9 lore raw
Quoted 41 lines · Thomas Gleixner
Quoted 29 lines · Scott Wood
Quoted 5 lines · Thomas Gleixner

As Peter pointed out to me privately we should rather go and analyze the
real thing instead of just applying duct tape.

/me drops the patch again.

The warning is due to kernel/sched/idle.c not updating curr-

Quoted 1 line

se.exec_start.

While debugging I noticed an issue with a particular load pattern. The CPU
goes non-nohz for a brief time at an interval very close to twice tick_period. When the tick is started, the timer expiration is more than
tick_period in the past, so hrtimer_forward() tries to catch up by adding
2*tick_period to the expiration. Then the tick is stopped before that new
expiration, and when the tick is woken up the expiry is again advanced by
2*tick_period with the timer never actually
running. sched_tick_remote()
does fire every second, but there are streaks of several seconds where it
keeps catching the CPU in a non-nohz state, so neither the normal nor remote
ticks are calling calc_load_nohz_remote().

Quoted 4 lines

Is there a reason to not just remove the hrtimer_forward() from tick_nohz_restart(), letting the timer fire if it's in the past, which will
take care of doing hrtimer_forward()?

Well, no. tick_nohz_restart() can be invoked in a situation where the timer
is armed for something in the far future (or completelt disabled) due to previously entering an estimated long idle (or user space execution on NOHZ_FULL) period.

Quoted 3 lines

That means if the timer is not canceled, realigned to the current tick and forwarded to the next due tick, the tick will not fire on time causing another sort of trouble.

That might be true of the expiry on entering tick_nohz_restart(), but it shouldn't be true of ts->last_tick which the expiry is set to before calling hrtimer_forward() -- and if it were, hrtimer_forward() is a no-op when the expiry is in the future.

BTW, the name "last_tick" seems misleading as it's actually the saved expiry, not the last time the tick ran.

-Scott

maintainer
#10 lore raw
In reply to Scott Wood
Quoted 1 line · Scott Wood

The warning is due to kernel/sched/idle.c not updating curr->se.exec_start.

Ah, indeed so.

Quoted 11 lines

While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice tick_period. When the tick is started, the timer expiration is more than tick_period in the past, so hrtimer_forward() tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running. sched_tick_remote() does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are calling calc_load_nohz_remote().

Quoted 3 lines

Is there a reason to not just remove the hrtimer_forward() from tick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doing hrtimer_forward()?

I'll have to look into that. I always get confused by all that nohz code :/

Quoted 3 lines

As for the warning in sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying on curr->se.exec_start, in order to detect things like this.

I don't think we have a timestamp that is shared between the remote and local tick. Also, there is a reason this warning uses the task time accounting, there used to be (as in, I can't find it in a hurry) code that could not deal with >u32 (~4s) clock updates.

The below should have idle keep the timestamp up-to-date. Keeping accurate idle->se.sum_exec_runtime doesn't seem too interesting, the idle code already keeps track of total idle times.

---

kernel/sched/idle.c +2 -0
@@ -381,6 +381,7 @@ static void put_prev_task_idle(struct rq
 
 static void set_next_task_idle(struct rq *rq, struct task_struct *next)
 {
+	curr->se.exec_start = rq_clock_task(rq);
 	update_idle_core(rq);
 	schedstat_inc(rq->sched_goidle);
 }
@@ -417,6 +418,7 @@ dequeue_task_idle(struct rq *rq, struct
  */
 static void task_tick_idle(struct rq *rq, struct task_struct *curr, int queued)
 {
+	curr->se.exec_start = rq_clock_task(rq);
 }
 
 static void switched_to_idle(struct rq *rq, struct task_struct *p)
maintainer
#11 lore raw
Quoted 32 lines · Peter Zijlstra
Quoted 1 line · Scott Wood

The warning is due to kernel/sched/idle.c not updating curr->se.exec_start.

Ah, indeed so.

Quoted 11 lines

While debugging I noticed an issue with a particular load pattern. The CPU goes non-nohz for a brief time at an interval very close to twice tick_period. When the tick is started, the timer expiration is more than tick_period in the past, so hrtimer_forward() tries to catch up by adding 2*tick_period to the expiration. Then the tick is stopped before that new expiration, and when the tick is woken up the expiry is again advanced by 2*tick_period with the timer never actually running. sched_tick_remote() does fire every second, but there are streaks of several seconds where it keeps catching the CPU in a non-nohz state, so neither the normal nor remote ticks are calling calc_load_nohz_remote().

Quoted 3 lines

Is there a reason to not just remove the hrtimer_forward() from tick_nohz_restart(), letting the timer fire if it's in the past, which will take care of doing hrtimer_forward()?

I'll have to look into that. I always get confused by all that nohz code :/

Quoted 3 lines

As for the warning in sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than relying on curr->se.exec_start, in order to detect things like this.

I don't think we have a timestamp that is shared between the remote and local tick. Also, there is a reason this warning uses the task time accounting, there used to be (as in, I can't find it in a hurry) code that could not deal with >u32 (~4s) clock updates.

Quoted 5 lines

The below should have idle keep the timestamp up-to-date. Keeping accurate idle->se.sum_exec_runtime doesn't seem too interesting, the idle code already keeps track of total idle times.

---

The obvious alternative is something like:

	if (rq->curr != rq->idle) {
		s64 delta = rq_clock_task(rq) - curr->se.exec_start;
		WARN_ON_ONCE(delta > 3ULL * NSEC_PER_SEC);
	}

Which would avoid polluting the idle path with that extra assignment.

#12 lore raw
In reply to Peter Zijlstra
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Scott Wood

As for the warning in sched_tick_remote(), it seems like a test for time since the last tick on this cpu (remote or otherwise) would be better than
relying on curr->se.exec_start, in order to detect things like this.

I don't think we have a timestamp that is shared between the remote and local tick.

Why wouldn't rq_clock_task() work on the local tick? It's what ->task_tick() itself uses.

Quoted 3 lines

Also, there is a reason this warning uses the task time accounting, there used to be (as in, I can't find it in a hurry) code that could not deal with >u32 (~4s) clock updates.

Detecting a 3 second interval between ticks for a given cpu should assert in a superset of the situations the current check asserts in -- it just avoids the false negative of exec_runtime getting updated by something other than the tick.

-Scott

#13 lore raw
Quoted 14 lines · Scott Wood
Quoted 9 lines · Peter Zijlstra
Quoted 5 lines · Scott Wood

As for the warning in sched_tick_remote(), it seems like a test for time
since the last tick on this cpu (remote or otherwise) would be better than
relying on curr->se.exec_start, in order to detect things like this.

I don't think we have a timestamp that is shared between the remote and local tick.

Why wouldn't rq_clock_task() work on the local tick? It's what ->task_tick() itself uses.

Quoted 8 lines
Quoted 3 lines

Also, there is a reason this warning uses the task time accounting, there used to be (as in, I can't find it in a hurry) code that could not deal with >u32 (~4s) clock updates.

Detecting a 3 second interval between ticks for a given cpu should assert in a superset of the situations the current check asserts in -- it just avoids the false negative of exec_runtime getting updated by something other than the tick.

The main difficulty with such a check is that when we're not on a full nohz cpu, there's no remote tick, and so we can legitimately go more than 3 seconds between ticks when idle.

-Scott

#14 lore raw
In reply to Peter Zijlstra
Quoted 21 lines · Peter Zijlstra
kernel/sched/core.c +0 -2
index eb42b71faab9..d02d1b8f40af 100644
@@ -3660,21 +3660,17 @@ static void sched_tick_remote(struct work_struct
 *work)
 	u64 delta;
 	int os;
 
-	/*
-	 * Handle the tick only if it appears the remote CPU is running in

full

kernel/sched/core.c +0 -2
-	 * dynticks mode. The check is racy by nature, but missing a tick or
-	 * having one too much is no big deal because the scheduler tick

updates

kernel/sched/core.c +0 -1
-	 * statistics and checks timeslices in a time-independent way,

regardless

kernel/sched/core.c +1 -3
-	 * of when exactly it is running.
-	 */
-	if (idle_cpu(cpu) || !tick_nohz_tick_stopped_cpu(cpu))
+	if (!tick_nohz_tick_stopped_cpu(cpu))

kernel/sched/core.c static void sched_tick_remote(struct work_struct *work) · line 3671

Quoted 10 lines
kernel/sched/core.c +4 -2
 		goto out_requeue;
 
 	rq_lock_irq(rq, &rf);
-	curr = rq->curr;
-	if (is_idle_task(curr) || cpu_is_offline(cpu))
+	/*
+	 * We must not call calc_load_nohz_remote() when not in NOHZ mode.
+	 */
+	if (cpu_is_offline(cpu) || !tick_nohz_tick_stopped(cpu))
 		goto out_unlock;

Is it really a problem if calc_load_nohz_remote() gets called in non-NOHZ? It won't race due to rq lock -- and we're already mixing remote and non-remote updates because the normal tick timer can still be run while "stopped".

-Scott