Skip to contentSkip to conversation

sched: Initialize rq->age_stamp on processor start

PATCH In review [email protected] 4 participants 7 messages May 6, 2014 – May 22, 2014 Scheduler

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched: Initialize rq->age_stamp on processor start

From: Corey Minyard <[email protected]>

If the sched_clock time starts at a large value, the kernel will spin in sched_avg_update for a long time while rq->age_stamp catches up with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise that it starts at zero. So initialize rq->age_stamp when a cpu starts up to avoid this.

I was seeing long delays on a simulator that didn't start the clock at zero. This might also be an issue on reboots on processors that don't re-initialize the timer to zero on reset, and when using kexec.

---
 kernel/sched/core.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/sched/core.c +10 -0
index 268a45e..6e9570b 100644
@@ -5039,11 +5039,20 @@ static struct notifier_block migration_notifier = {
 	.priority = CPU_PRI_MIGRATION,
 };
 
+static void __cpuinit set_cpu_rq_start_time(void)
+{
+	int cpu = smp_processor_id();
+	struct rq *rq = cpu_rq(cpu);
+	rq->age_stamp = sched_clock_cpu(cpu);
+}
+
 static int sched_cpu_active(struct notifier_block *nfb,
 				      unsigned long action, void *hcpu)
 {
 	switch (action & ~CPU_TASKS_FROZEN) {
 	case CPU_STARTING:
+		set_cpu_rq_start_time();
+		/* fall through */
 	case CPU_DOWN_FAILED:
 		set_cpu_active((long)hcpu, true);
 		return NOTIFY_OK;
@@ -6923,6 +6932,7 @@ void __init sched_init(void)
 	init_sched_fair_class();
 
 	scheduler_running = 1;
+	set_cpu_rq_start_time();
 }
 
 #ifdef CONFIG_DEBUG_ATOMIC_SLEEP
Signature · 1 line
1.8.3.1
maintainer
#1 lore raw
Quoted 10 lines · On Tue, May 06, 2014 at 02:50:57PM -0500, [email protected] wrote:

From: Corey Minyard <[email protected]>

If the sched_clock time starts at a large value, the kernel will spin in sched_avg_update for a long time while rq->age_stamp catches up with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise that it starts at zero. So initialize rq->age_stamp when a cpu starts up to avoid this.

Quoted 5 lines

I was seeing long delays on a simulator that didn't start the clock at zero. This might also be an issue on reboots on processors that don't re-initialize the timer to zero on reset, and when using kexec.

This patch made my cris-defconfig build fail.

#2 lore raw
Quoted 11 lines · Peter Zijlstra
Quoted 10 lines · On Tue, May 06, 2014 at 02:50:57PM -0500, [email protected] wrote:

From: Corey Minyard <[email protected]>

If the sched_clock time starts at a large value, the kernel will spin in sched_avg_update for a long time while rq->age_stamp catches up with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise that it starts at zero. So initialize rq->age_stamp when a cpu starts up to avoid this.

Quoted 6 lines
Quoted 5 lines

I was seeing long delays on a simulator that didn't start the clock at zero. This might also be an issue on reboots on processors that don't re-initialize the timer to zero on reset, and when using kexec.

This patch made my cris-defconfig build fail.

Dang, I missed propagating a change. Yes, it's broken. One more time...

-corey

#3 lore raw

PATCH 1/1 sched: Initialize rq->age_stamp on processor start

In reply to Peter Zijlstra

From: Corey Minyard <[email protected]>

If the sched_clock time starts at a large value, the kernel will spin in sched_avg_update for a long time while rq->age_stamp catches up with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise that it starts at zero. So initialize rq->age_stamp when a cpu starts up to avoid this.

I was seeing long delays on a simulator that didn't start the clock at zero. This might also be an issue on reboots on processors that don't re-initialize the timer to zero on reset, and when using kexec.

---
 kernel/sched/core.c | 10 ++++++++++
 1 file changed, 10 insertions(+)
kernel/sched/core.c +10 -0
index 268a45e..80725c9 100644
@@ -5039,11 +5039,20 @@ static struct notifier_block migration_notifier = {
 	.priority = CPU_PRI_MIGRATION,
 };
 
+static void __cpuinit set_cpu_rq_start_time(void)
+{
+	int cpu = smp_processor_id();
+	struct rq *rq = cpu_rq(cpu);
+	rq->age_stamp = sched_clock_cpu(cpu);
+}
+
 static int sched_cpu_active(struct notifier_block *nfb,
 				      unsigned long action, void *hcpu)
 {
 	switch (action & ~CPU_TASKS_FROZEN) {
 	case CPU_STARTING:
+		set_cpu_rq_start_time();
+		/* fall through */
 	case CPU_DOWN_FAILED:
 		set_cpu_active((long)hcpu, true);
 		return NOTIFY_OK;
@@ -6919,6 +6928,7 @@ void __init sched_init(void)
 	if (cpu_isolated_map == NULL)
 		zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
 	idle_thread_set_boot_cpu();
+	set_cpu_rq_start_time();
 #endif
 	init_sched_fair_class();
 
Signature · 1 line
1.8.3.1
maintainer
#4 lore raw
Quoted 10 lines · On Thu, May 08, 2014 at 01:47:39PM -0500, [email protected] wrote:

From: Corey Minyard <[email protected]>

If the sched_clock time starts at a large value, the kernel will spin in sched_avg_update for a long time while rq->age_stamp catches up with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise that it starts at zero. So initialize rq->age_stamp when a cpu starts up to avoid this.

Quoted 5 lines

I was seeing long delays on a simulator that didn't start the clock at zero. This might also be an issue on reboots on processors that don't re-initialize the timer to zero on reset, and when using kexec.

Thanks

#5 lore raw

[tip:sched/core] sched: Initialize rq-> age_stamp on processor start

In reply to [email protected]
sched: Initialize rq->age_stamp on processor start
Commit-ID:  6cfa0c323ca96b7f619932725bfc0de1e4a484fc
Gitweb:     http://git.kernel.org/tip/6cfa0c323ca96b7f619932725bfc0de1e4a484fc
Author:     Corey Minyard <[email protected]>
AuthorDate: Thu, 8 May 2014 13:47:39 -0500
Committer:  Thomas Gleixner <[email protected]>
CommitDate: Mon, 19 May 2014 22:02:41 +0900

sched: Initialize rq->age_stamp on processor start

If the sched_clock time starts at a large value, the kernel will spin in sched_avg_update for a long time while rq->age_stamp catches up with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise that it starts at zero. So initialize rq->age_stamp when a cpu starts up to avoid this.

I was seeing long delays on a simulator that didn't start the clock at zero. This might also be an issue on reboots on processors that don't re-initialize the timer to zero on reset, and when using kexec.

---
 kernel/sched/core.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
kernel/sched/core.c +11 -0
index 10d9b50..c134eb0 100644
@@ -5089,10 +5089,20 @@ static struct notifier_block migration_notifier = {
 	.priority = CPU_PRI_MIGRATION,
 };
 
+static void __cpuinit set_cpu_rq_start_time(void)
+{
+	int cpu = smp_processor_id();
+	struct rq *rq = cpu_rq(cpu);
+	rq->age_stamp = sched_clock_cpu(cpu);
+}
+
 static int sched_cpu_active(struct notifier_block *nfb,
 				      unsigned long action, void *hcpu)
 {
 	switch (action & ~CPU_TASKS_FROZEN) {
+	case CPU_STARTING:
+		set_cpu_rq_start_time();
+		return NOTIFY_OK;
 	case CPU_DOWN_FAILED:
 		set_cpu_active((long)hcpu, true);
 		return NOTIFY_OK;
@@ -6970,6 +6980,7 @@ void __init sched_init(void)
 	if (cpu_isolated_map == NULL)
 		zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
 	idle_thread_set_boot_cpu();
+	set_cpu_rq_start_time();
 #endif
 	init_sched_fair_class();
 
#6 lore raw

[tip:sched/core] sched: Initialize rq-> age_stamp on processor start

In reply to [email protected]
sched: Initialize rq->age_stamp on processor start
Commit-ID:  a803f0261bb2bb57aab5542af3174db43b2a3887
Gitweb:     http://git.kernel.org/tip/a803f0261bb2bb57aab5542af3174db43b2a3887
Author:     Corey Minyard <[email protected]>
AuthorDate: Thu, 8 May 2014 13:47:39 -0500
Committer:  Ingo Molnar <[email protected]>
CommitDate: Thu, 22 May 2014 11:16:35 +0200

sched: Initialize rq->age_stamp on processor start

If the sched_clock time starts at a large value, the kernel will spin in sched_avg_update for a long time while rq->age_stamp catches up with rq->clock.

The comment in kernel/sched/clock.c says that there is no strict promise that it starts at zero. So initialize rq->age_stamp when a cpu starts up to avoid this.

I was seeing long delays on a simulator that didn't start the clock at zero. This might also be an issue on reboots on processors that don't re-initialize the timer to zero on reset, and when using kexec.

---
 kernel/sched/core.c | 11 +++++++++++
 1 file changed, 11 insertions(+)
kernel/sched/core.c +11 -0
index f5605b6..da302ca 100644
@@ -5089,10 +5089,20 @@ static struct notifier_block migration_notifier = {
 	.priority = CPU_PRI_MIGRATION,
 };
 
+static void __cpuinit set_cpu_rq_start_time(void)
+{
+	int cpu = smp_processor_id();
+	struct rq *rq = cpu_rq(cpu);
+	rq->age_stamp = sched_clock_cpu(cpu);
+}
+
 static int sched_cpu_active(struct notifier_block *nfb,
 				      unsigned long action, void *hcpu)
 {
 	switch (action & ~CPU_TASKS_FROZEN) {
+	case CPU_STARTING:
+		set_cpu_rq_start_time();
+		return NOTIFY_OK;
 	case CPU_DOWN_FAILED:
 		set_cpu_active((long)hcpu, true);
 		return NOTIFY_OK;
@@ -6970,6 +6980,7 @@ void __init sched_init(void)
 	if (cpu_isolated_map == NULL)
 		zalloc_cpumask_var(&cpu_isolated_map, GFP_NOWAIT);
 	idle_thread_set_boot_cpu();
+	set_cpu_rq_start_time();
 #endif
 	init_sched_fair_class();