Skip to contentSkip to conversation

2.4.21-pre5 correct scheduling of idle tasks [ all arch ]

PATCH New Keith Owens 2 participants 4 messages Mar 26, 2003 – Mar 28, 2003

Subsystems (6)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 2.4.21-pre5 correct scheduling of idle tasks [ all arch ]

There are several inconsistencies in the scheduling of idle tasks and, for UP, tracking which task is on the cpu. This patch standardizes idle task scheduling across all architectures and corrects the UP error, it is just a bug fix.

For each task, the cpus_runnable, cpus_allowed and processor fields should be set. They are set correctly for normal tasks, but not for the idle tasks. Each architecture manually sets these fields as each idle task is created, but no architecture gets it exactly right. x86_64 comes close but it is using 1<<cpu instead of 1UL<<cpu so will break past 32 cpus.

To make it worse, on UP a task is assigned to a cpu but never released. Very quickly, all tasks are marked as currently running on cpu 0 :(.

Add inline function task_set_cpu_only() to set cpus_runnable, cpus_allowed and processor. Call that function from every arch setup of the idle task, plus from start_kernel for the boot idle task. Release the cpu in the UP case.

Tested on i386 2.4.21-pre5 and ia64 2.4.20. Eyeballed for other architectures but not compiled, please verify.

arch/alpha/kernel/smp.c +1 -2
diff -ur 2.4.20-pristine/arch/alpha/kernel/smp.c 2.4.20-sched/arch/alpha/kernel/smp.c
@@ -505,8 +505,7 @@
 	if (idle == &init_task)
 		panic("idle process is init_task for CPU %d", cpuid);
 
-	idle->processor = cpuid;
-	idle->cpus_runnable = 1 << cpuid; /* we schedule the first task manually */
+	task_set_cpu_only(idle, cpuid);	/* we schedule the first task manually */
 	__cpu_logical_map[cpunum] = cpuid;
 	__cpu_number_map[cpuid] = cpunum;
  
arch/i386/kernel/smpboot.c +1 -2
diff -ur 2.4.20-pristine/arch/i386/kernel/smpboot.c 2.4.20-sched/arch/i386/kernel/smpboot.c
@@ -803,8 +803,7 @@
 	if (!idle)
 		panic("No idle process for CPU %d", cpu);
 
-	idle->processor = cpu;
-	idle->cpus_runnable = 1 << cpu; /* we schedule the first task manually */
+	task_set_cpu_only(idle, cpu);	/* we schedule the first task manually */
 
 	map_cpu_to_boot_apicid(cpu, apicid);
 
arch/ia64/kernel/smpboot.c +1 -1
diff -ur 2.4.20-pristine/arch/ia64/kernel/smpboot.c 2.4.20-sched/arch/ia64/kernel/smpboot.c
@@ -422,7 +422,7 @@
 	if (!idle)
 		panic("No idle process for CPU %d", cpu);
 
-	task_set_cpu(idle, cpu);	/* we schedule the first task manually */
+	task_set_cpu_only(idle, cpu);	/* we schedule the first task manually */
 
 	ia64_cpu_to_sapicid[cpu] = sapicid;
 
arch/mips/sgi-ip27/ip27-init.c +1 -2
diff -ur 2.4.20-pristine/arch/mips/sgi-ip27/ip27-init.c 2.4.20-sched/arch/mips/sgi-ip27/ip27-init.c
@@ -479,8 +479,7 @@
 			init_tasks[num_cpus] = p;
 			alloc_cpupda(cpu, num_cpus);
 			del_from_runqueue(p);
-			p->processor = num_cpus;
-			p->cpus_runnable = 1 << num_cpus; /* we schedule the first task manually */
+			task_set_cpu_only(p, num_cpus);	/* we schedule the first task manually */
 			unhash_process(p);
 			/* Attach to the address space of init_task. */
 			atomic_inc(&init_mm.mm_count);
arch/mips/sibyte/sb1250/smp.c +1 -3
diff -ur 2.4.20-pristine/arch/mips/sibyte/sb1250/smp.c 2.4.20-sched/arch/mips/sibyte/sb1250/smp.c
@@ -124,9 +124,7 @@
 		do_fork(CLONE_VM|CLONE_PID, 0, &regs, 0);
 		p = init_task.prev_task;
 
-		/* Schedule the first task manually */
-		p->processor = i;
-		p->cpus_runnable = 1 << i; /* we schedule the first task manually */
+		task_set_cpu_only(p, i);	/* we schedule the first task manually */
 
 		/* Attach to the address space of init_task. */
 		atomic_inc(&init_mm.mm_count);
arch/parisc/kernel/smp.c +1 -1
diff -ur 2.4.20-pristine/arch/parisc/kernel/smp.c 2.4.20-sched/arch/parisc/kernel/smp.c
@@ -542,7 +542,7 @@
 	if (!idle)
 		panic("SMP: No idle process for CPU:%d", cpuid);
 
-	task_set_cpu(idle, cpunum);	/* manually schedule idle task */
+	task_set_cpu_only(idle, cpunum);	/* manually schedule idle task */
 	del_from_runqueue(idle);
 	unhash_process(idle);
 	init_tasks[cpunum] = idle;
arch/ppc/kernel/smp.c +1 -2
diff -ur 2.4.20-pristine/arch/ppc/kernel/smp.c 2.4.20-sched/arch/ppc/kernel/smp.c
@@ -352,8 +352,7 @@
 		unhash_process(p);
 		init_tasks[i] = p;
 
-		p->processor = i;
-		p->cpus_runnable = 1 << i; /* we schedule the first task manually */
+		task_set_cpu_only(p, i);	/* we schedule the first task manually */
 		current_set[i] = p;
 
 		/*
arch/ppc64/kernel/smp.c +1 -2
diff -ur 2.4.20-pristine/arch/ppc64/kernel/smp.c 2.4.20-sched/arch/ppc64/kernel/smp.c
@@ -688,8 +688,7 @@
 		unhash_process(p);
 		init_tasks[i] = p;
 
-		p->processor = i;
-		p->cpus_runnable = 1 << i; /* we schedule the first task manually */
+		task_set_cpu_only(p, i);	/* we schedule the first task manually */
 		current_set[i].task = p;
 		sp = ((unsigned long)p) + sizeof(union task_union)
 			- STACK_FRAME_OVERHEAD;
arch/s390/kernel/smp.c +1 -2
diff -ur 2.4.20-pristine/arch/s390/kernel/smp.c 2.4.20-sched/arch/s390/kernel/smp.c
@@ -547,8 +547,7 @@
         idle = init_task.prev_task;
         if (!idle)
                 panic("No idle process for CPU %d",cpu);
-        idle->processor = cpu;
-	idle->cpus_runnable = 1 << cpu; /* we schedule the first task manually */
+	task_set_cpu_only(idle, cpu);	/* we schedule the first task manually */
 
         del_from_runqueue(idle);
         unhash_process(idle);
arch/s390x/kernel/smp.c +1 -2
diff -ur 2.4.20-pristine/arch/s390x/kernel/smp.c 2.4.20-sched/arch/s390x/kernel/smp.c
@@ -533,8 +533,7 @@
         idle = init_task.prev_task;
         if (!idle)
                 panic("No idle process for CPU %d",cpu);
-        idle->processor = cpu;
-	idle->cpus_runnable = 1 << cpu; /* we schedule the first task manually */
+	task_set_cpu_only(idle, cpu);	/* we schedule the first task manually */
 
         del_from_runqueue(idle);
         unhash_process(idle);
arch/sparc/kernel/sun4d_smp.c +1 -2
diff -ur 2.4.20-pristine/arch/sparc/kernel/sun4d_smp.c 2.4.20-sched/arch/sparc/kernel/sun4d_smp.c
@@ -224,8 +224,7 @@
 			p = init_task.prev_task;
 			init_tasks[i] = p;
 
-			p->processor = i;
-			p->cpus_runnable = 1 << i; /* we schedule the first task manually */
+			task_set_cpu_only(p, i);	/* we schedule the first task manually */
 
 			current_set[i] = p;
 
arch/sparc/kernel/sun4m_smp.c +1 -2
diff -ur 2.4.20-pristine/arch/sparc/kernel/sun4m_smp.c 2.4.20-sched/arch/sparc/kernel/sun4m_smp.c
@@ -197,8 +197,7 @@
 			p = init_task.prev_task;
 			init_tasks[i] = p;
 
-			p->processor = i;
-			p->cpus_runnable = 1 << i; /* we schedule the first task manually */
+			task_set_cpu_only(p, i);	/* we schedule the first task manually */
 
 			current_set[i] = p;
 
arch/sparc64/kernel/smp.c +1 -2
diff -ur 2.4.20-pristine/arch/sparc64/kernel/smp.c 2.4.20-sched/arch/sparc64/kernel/smp.c
@@ -284,8 +284,7 @@
 			p = init_task.prev_task;
 			init_tasks[cpucount] = p;
 
-			p->processor = i;
-			p->cpus_runnable = 1UL << i; /* we schedule the first task manually */
+			task_set_cpu_only(p, i);	/* we schedule the first task manually */
 
 			del_from_runqueue(p);
 			unhash_process(p);
arch/x86_64/kernel/smpboot.c +1 -3
diff -ur 2.4.20-pristine/arch/x86_64/kernel/smpboot.c 2.4.20-sched/arch/x86_64/kernel/smpboot.c
@@ -540,11 +540,9 @@
 	if (!idle)
 		panic("No idle process for CPU %d", cpu);
 
-	idle->processor = cpu;
 	x86_cpu_to_apicid[cpu] = apicid;
 	x86_apicid_to_cpu[apicid] = cpu;
-	idle->cpus_runnable = 1<<cpu; 
-	idle->cpus_allowed = 1<<cpu;
+	task_set_cpu_only(idle, cpu);	/* we schedule the first task manually */
 	idle->thread.rip = (unsigned long)start_secondary;
 	idle->thread.rsp = (unsigned long)idle + THREAD_SIZE - 8;
 
include/linux/sched.h +6 -0
diff -ur 2.4.20-pristine/include/linux/sched.h 2.4.20-sched/include/linux/sched.h
@@ -568,6 +568,12 @@
 	tsk->cpus_runnable = 1UL << cpu;
 }
 
+static inline void task_set_cpu_only(struct task_struct *tsk, unsigned int cpu)
+{
+	task_set_cpu(tsk, cpu);
+	tsk->cpus_allowed = 1UL << cpu;
+}
+
 static inline void task_release_cpu(struct task_struct *tsk)
 {
 	tsk->cpus_runnable = ~0UL;
init/main.c +1 -0
diff -ur 2.4.20-pristine/init/main.c 2.4.20-sched/init/main.c
@@ -354,6 +354,7 @@
  * enable them
  */
 	lock_kernel();
+	task_set_cpu_only(current, 0);
 	printk(linux_banner);
 	setup_arch(&command_line);
 	printk("Kernel command line: %s\n", saved_command_line);
kernel/sched.c +1 -0
diff -ur 2.4.20-pristine/kernel/sched.c 2.4.20-sched/kernel/sched.c
@@ -525,6 +525,7 @@
 		goto out_unlock;
 	}
 #else
+	task_release_cpu(prev);
 	prev->policy &= ~SCHED_YIELD;
 #endif /* CONFIG_SMP */
 }
#1 lore raw

PATCH 1/1 2.4.21-pre5 correct scheduling of idle tasks [ all arch ]

Quoted 4 lines · Keith Owens

There are several inconsistencies in the scheduling of idle tasks and, for UP, tracking which task is on the cpu. This patch standardizes idle task scheduling across all architectures and corrects the UP error, it is just a bug fix.

...

Quoted 2 lines

To make it worse, on UP a task is assigned to a cpu but never released. Very quickly, all tasks are marked as currently running on cpu 0 :(.

->cpus_runnable and task_has_cpu() are SMP-only, as a quick grep through 2.4.20 will tell you. There is no UP bug here to fix.

/Mikael

#2 lore raw
Quoted 11 lines · On Thu, 27 Mar 2003 16:54:47 +0100, [email protected] wrote:
Quoted 4 lines · Keith Owens

There are several inconsistencies in the scheduling of idle tasks and, for UP, tracking which task is on the cpu. This patch standardizes idle task scheduling across all architectures and corrects the UP error, it is just a bug fix.

...

Quoted 2 lines

To make it worse, on UP a task is assigned to a cpu but never released. Very quickly, all tasks are marked as currently running on cpu 0 :(.

->cpus_runnable and task_has_cpu() are SMP-only, as a quick grep through 2.4.20 will tell you. There is no UP bug here to fix.

cpus_runnable has task_has_cpu are not guarded by CONFIG_SMP. task_set_cpu() is called for UP as well as SMP. UP is missing the corresponding call to task_release_cpu().

#3 lore raw
Quoted 11 lines · Keith Owens
Quoted 9 lines · On Thu, 27 Mar 2003 16:54:47 +0100, [email protected] wrote:
Quoted 4 lines · Keith Owens

There are several inconsistencies in the scheduling of idle tasks and, for UP, tracking which task is on the cpu. This patch standardizes idle task scheduling across all architectures and corrects the UP error, it is just a bug fix.

...

Quoted 2 lines

To make it worse, on UP a task is assigned to a cpu but never released. Very quickly, all tasks are marked as currently running on cpu 0 :(.

Quoted 6 lines
Quoted 2 lines

->cpus_runnable and task_has_cpu() are SMP-only, as a quick grep through 2.4.20 will tell you. There is no UP bug here to fix.

cpus_runnable has task_has_cpu are not guarded by CONFIG_SMP. task_set_cpu() is called for UP as well as SMP. UP is missing the corresponding call to task_release_cpu().

No generic kernel code _uses_ ->cpus_runnable on UP. arch/s390{,x}/kernel/traps.c appears to use task_has_cpu() on UP, but that's their bug and not an argument for slowing down UP kernels.

Hence, kernel/sched.c not calling task_release_cpu() to reset ->cpus_runnable to ~0 is not a bug.

The only bug (apart from s390's trap.c) is that task_set_cpu() performs an unnecessary assignment to ->cpus_runnable on UP.

/Mikael