Skip to contentSkip to conversation

fix perf. bug in wake-up load balancing for aim7 and db workload

PATCH In review Chen, Kenneth W 3 participants 3 messages Feb 14, 2006 – Feb 16, 2006 Scheduler

lore
Share & tools
lore
#0 lore raw

PATCH 1/2 fix perf. bug in wake-up load balancing for aim7 and db workload

Revert commit d7102e95b7b9c00277562c29aad421d2d521c5f6, which causes more than 10% performance regression with aim7.

include/linux/sched.h renamed from include/linux/sched.h.orig +1 -4
@@ -697,12 +697,9 @@ struct task_struct {
 
 	int lock_depth;		/* BKL lock depth */
 
-#if defined(CONFIG_SMP)
-	int last_waker_cpu;	/* CPU that last woke this task up */
-#if defined(__ARCH_WANT_UNLOCKED_CTXSW)
+#if defined(CONFIG_SMP) && defined(__ARCH_WANT_UNLOCKED_CTXSW)
 	int oncpu;
 #endif
-#endif
 	int prio, static_prio;
 	struct list_head run_list;
 	prio_array_t *array;
kernel/sched.c renamed from kernel/sched.c.orig +1 -9
@@ -1294,9 +1294,6 @@ static int try_to_wake_up(task_t *p, uns
 		}
 	}
 
-	if (p->last_waker_cpu != this_cpu)
-		goto out_set_cpu;
-
 	if (unlikely(!cpu_isset(this_cpu, p->cpus_allowed)))
 		goto out_set_cpu;
 
@@ -1367,8 +1364,6 @@ out_set_cpu:
 		cpu = task_cpu(p);
 	}
 
-	p->last_waker_cpu = this_cpu;
-
 out_activate:
 #endif /* CONFIG_SMP */
 	if (old_state == TASK_UNINTERRUPTIBLE) {
@@ -1450,12 +1445,9 @@ void fastcall sched_fork(task_t *p, int 
 #ifdef CONFIG_SCHEDSTATS
 	memset(&p->sched_info, 0, sizeof(p->sched_info));
 #endif
-#if defined(CONFIG_SMP)
-	p->last_waker_cpu = cpu;
-#if defined(__ARCH_WANT_UNLOCKED_CTXSW)
+#if defined(CONFIG_SMP) && defined(__ARCH_WANT_UNLOCKED_CTXSW)
 	p->oncpu = 0;
 #endif
-#endif
 #ifdef CONFIG_PREEMPT
 	/* Want to start with kernel preemption disabled. */
 	task_thread_info(p)->preempt_count = 1;
#1 lore raw
Quoted 3 lines · Kenneth W

Revert commit d7102e95b7b9c00277562c29aad421d2d521c5f6, which causes more than 10% performance regression with aim7.

Just to be sure, what kernel did you test with? In particular, did it have the smpnice patch reverted (as -rc3 does).

Signature · 2 lines
SUSE Labs, Novell Inc.
Send instant messages to your online friends http://au.messenger.yahoo.com 
#2 lore raw
Quoted 9 lines · Nick Piggin
Quoted 3 lines · Kenneth W

Revert commit d7102e95b7b9c00277562c29aad421d2d521c5f6, which causes more than 10% performance regression with aim7.

Just to be sure, what kernel did you test with? In particular, did it have the smpnice patch reverted (as -rc3 does).

Analysis of the smpnice code indicates that it could cause anomalous cpu selection decisions in try_to_wake_up() if there is a skew in the distribution of nice among the tasks on the cpus under consideration. Attached for review is a proposed patch to address problem. In particular, I request comments on the following issues:

1. Is this potential problem worth worrying about?
2. Do you agree with my decision to replace SCHED_LOAD_SCALE with the average load per task for this_cpu in the if statement in try_to_wake_up() or should I be using the average load per task for the tasks current cpu in one or both places?

Peter

Signature · 4 lines
Peter Williams                                   [email protected]

"Learning, n. The kind of ignorance distinguishing the studious."
 -- Ambrose Bierce
kernel/sched.c +15 -2
Index: MM-2.6.X/kernel/sched.c
===================================================================
@@ -1061,6 +1061,18 @@ static inline unsigned long target_load(
 }
 
 /*
+ * Return the average load per task on the cpu's run queue
+ */
+static inline unsigned long cpu_avg_load_per_task(int cpu)
+{
+	runqueue_t *rq = cpu_rq(cpu);
+	unsigned long n = rq->nr_running;
+	unsigned long load = weighted_load(rq->prio_bias);
+
+	return n ? load / n : load;
+}
+
+/*
  * find_idlest_group finds and returns the least busy CPU group within the
  * domain.
  */
@@ -1309,6 +1321,7 @@ static int try_to_wake_up(task_t *p, uns
 
 		if (this_sd->flags & SD_WAKE_AFFINE) {
 			unsigned long tl = this_load;
+			unsigned long tl_per_task = cpu_avg_load_per_task(this_cpu);
 			/*
 			 * If sync wakeup then subtract the (maximum possible)
 			 * effect of the currently running task from the load
@@ -1318,8 +1331,8 @@ static int try_to_wake_up(task_t *p, uns
 				tl -= weighted_load(p->bias_prio);
 
 			if ((tl <= load &&
-				tl + target_load(cpu, idx) <= SCHED_LOAD_SCALE) ||
-				100*(tl + SCHED_LOAD_SCALE) <= imbalance*load) {
+				tl + target_load(cpu, idx) <= tl_per_task) ||
+				100*(tl + tl_per_task) <= imbalance*load) {
 				/*
 				 * This domain has SD_WAKE_AFFINE and
 				 * p is cache cold in this domain, and