Skip to contentSkip to conversation

sched: aggressive idle balance

RFC PATCH In review Andrew Theurer 2 participants 4 messages Nov 2, 2004 – Nov 3, 2004

lore
Share & tools
lore
#0 lore raw

RFC PATCH 1/1 sched: aggressive idle balance

Hello all,

This patch allows more aggressive idle balances, reducing idle time in scenarios where should not be any, where nr_running > nr_cpus. We have seen this in a couple of online transaction workloads. Three areas are targeted:

1) In try_to_wake_up(), wake_idle() is called to move the task to a sibling if the task->cpu is busy and the sibling is idle. This has been expanded to any idle cpu, but the closest idle cpu is picked first by starting with cpu->sd, then going up the domains as necessary.

2) can_migrate() is modified. Honestly part of it looked kind of backwards. We checked for task_hot() if the cpu was idle. I would think we would do that if is was -not- idle. Changes to can migrate are:
a) same policies for running tasks and cpus_allowed, return 0
b) We omit task_hot check for -whole- idle cpus (not just a sibling), when task_cpu(p) and this_cpu are member siblings in the same core, or when nr_balance_failed > cache_nice_tries
c) finally, if this_cpu is busy we still do the task_hot check

3) Add SD_BALANCE_NEWIDLE to SD_NODE_INIT. All this does is allow a newly idle cpu a broader span of cpus, only if necessary, to pull a task. We still target cpus closest to the this_cpu, starting with this_cpu->sd, then as needed moving up the domains. **This is vital for Opteron, where each node only has one cpu. idle_balance() never works because it would never consider looking beyond the cpu->sd, because cpu->sd->sd did not have SD_BALANCE_NEWIDLE set.

So far we have seen 3-5% with these patches on online transaction workolads and no regressions on SDET. Kenneth, I am particularly interested in using this with your increased cache_hot_time value, where you got your best throughput:

Quoted 3 lines

cache_hot_time | workload throughput
--------------------------------------
25ms - 111.1 (5% idle)

...but still had idle time. Do you think you could try these patches with your 25ms cache_hot_time? I think your workload could benefit from both the longer cache_hot_time for busy cpus, but more aggressive idle balances, hopefully driving your workload to 100% cpu utilization.

Any feedback greatly appreciated, thanks.

-Andrew Theurer

include/asm-i386/topology.h +2 -0
diff -Naurp linux-2.6.10-rc1-bk9/include/asm-i386/topology.h linux-2.6.10-rc1-bk9-idle/include/asm-i386/topology.h
@@ -83,6 +83,8 @@ static inline cpumask_t pcibus_to_cpumas
 	.per_cpu_gain		= 100,			\
 	.flags			= SD_LOAD_BALANCE	\
 				| SD_BALANCE_EXEC	\
+				| SD_BALANCE_NEWIDLE	\
+				| SD_WAKE_IDLE		\
 				| SD_WAKE_BALANCE,	\
 	.last_balance		= jiffies,		\
 	.balance_interval	= 1,			\
include/asm-ia64/topology.h +2 -0
diff -Naurp linux-2.6.10-rc1-bk9/include/asm-ia64/topology.h linux-2.6.10-rc1-bk9-idle/include/asm-ia64/topology.h
@@ -56,6 +56,8 @@ void build_cpu_to_node_map(void);
 	.per_cpu_gain		= 100,			\
 	.flags			= SD_LOAD_BALANCE	\
 				| SD_BALANCE_EXEC	\
+				| SD_BALANCE_NEWIDLE	\
+				| SD_WAKE_IDLE		\
 				| SD_WAKE_BALANCE,	\
 	.last_balance		= jiffies,		\
 	.balance_interval	= 1,			\
include/asm-ppc64/topology.h +2 -0
diff -Naurp linux-2.6.10-rc1-bk9/include/asm-ppc64/topology.h linux-2.6.10-rc1-bk9-idle/include/asm-ppc64/topology.h
@@ -51,6 +51,8 @@ static inline int node_to_first_cpu(int 
 	.per_cpu_gain		= 100,			\
 	.flags			= SD_LOAD_BALANCE	\
 				| SD_BALANCE_EXEC	\
+				| SD_BALANCE_NEWIDLE	\
+				| SD_WAKE_IDLE		\
 				| SD_WAKE_BALANCE,	\
 	.last_balance		= jiffies,		\
 	.balance_interval	= 1,			\
include/asm-x86_64/topology.h +2 -0
diff -Naurp linux-2.6.10-rc1-bk9/include/asm-x86_64/topology.h linux-2.6.10-rc1-bk9-idle/include/asm-x86_64/topology.h
@@ -46,7 +46,9 @@ static inline cpumask_t __pcibus_to_cpum
 	.cache_nice_tries	= 1,			\
 	.per_cpu_gain		= 100,			\
 	.flags			= SD_LOAD_BALANCE	\
+				| SD_BALANCE_NEWIDLE	\
 				| SD_BALANCE_EXEC	\
+				| SD_WAKE_IDLE		\
 				| SD_WAKE_BALANCE,	\
 	.last_balance		= jiffies,		\
 	.balance_interval	= 1,			\
include/linux/topology.h +1 -0
diff -Naurp linux-2.6.10-rc1-bk9/include/linux/topology.h linux-2.6.10-rc1-bk9-idle/include/linux/topology.h
@@ -120,6 +120,7 @@ static inline int __next_node_with_cpus(
 				| SD_BALANCE_NEWIDLE	\
 				| SD_BALANCE_EXEC	\
 				| SD_WAKE_AFFINE	\
+				| SD_WAKE_IDLE		\
 				| SD_WAKE_BALANCE,	\
 	.last_balance		= jiffies,		\
 	.balance_interval	= 1,			\
kernel/sched.c +53 -38

135 lines not on this page · raw message

#1 lore raw

Andrew Theurer wrote on Tuesday, November 02, 2004 12:17 PM

Quoted 10 lines

So far we have seen 3-5% with these patches on online transaction workolads and no regressions on SDET. Kenneth, I am particularly interested in using this with your increased cache_hot_time value, where you got your best throughput:

...but still had idle time. Do you think you could try these patches with your 25ms cache_hot_time? I think your workload could benefit from both the longer cache_hot_time for busy cpus, but more aggressive idle balances, hopefully driving your workload to 100% cpu utilization.

Looks interesting, I will queue this up on our benchmark setup.

- Ken

#2 lore raw
In reply to Andrew Theurer

Andrew Theurer wrote on Tuesday, November 02, 2004 12:17 PM

Quoted 9 lines

This patch allows more aggressive idle balances, reducing idle time in scenarios where should not be any, where nr_running > nr_cpus. We have seen this in a couple of online transaction workloads. Three areas are targeted:

1) In try_to_wake_up(), wake_idle() is called to move the task to a sibling if the task->cpu is busy and the sibling is idle. This has been expanded to any idle cpu, but the closest idle cpu is picked first by starting with cpu->sd, then going up the domains as necessary.

It occurs to me that half of the patch only applicable to HT, like the change in wake_idle(). And also, do you really want to put that functionality in wake_idle()? Seems defeating the original intention of that function, which only tries to wake up sibling cpu as far as how I understand the code.

My setup is 4-way SMP, no HT (4-way itanium2 processor), sorry, I won't be able to tell you how this portion of the change affect online transaction workload.

- Ken

#3 lore raw
Quoted 14 lines · Kenneth W

Andrew Theurer wrote on Tuesday, November 02, 2004 12:17 PM

Quoted 10 lines

This patch allows more aggressive idle balances, reducing idle time in scenarios where should not be any, where nr_running > nr_cpus. We have seen this in a couple of online transaction workloads. Three areas are targeted:

1) In try_to_wake_up(), wake_idle() is called to move the task to a sibling if the task->cpu is busy and the sibling is idle. This has been expanded to any idle cpu, but the closest idle cpu is picked first by starting with cpu->sd, then going up the domains as necessary.

Quoted 6 lines

It occurs to me that half of the patch only applicable to HT, like the change in wake_idle(). And also, do you really want to put that functionality in wake_idle()? Seems defeating the original intention of that function, which only tries to wake up sibling cpu as far as how I understand the code.

The patch (wake_idle()) should still make a difference with nonHT systems. It is true that in the mainline code, only HT systems benefited from wake_idle(). In fact, wake_idle() probably did nothing at all for Itanium, but with this patch it will, since we scan for an idle cpu up the domains that have flag SD_WAKE_IDLE (and we addedd that flag to SD_CPU_INIT and SD_NODE_INIT).

The patch's intention is to make sure we don't leave idle cpus idle. This can happen because the rate at which tasks are activated and deactivated is much higher than the idle balance interval, and because the task_hot prevents migrations even when we do encounter an idle balance tick.

Honsetly, I think you should be able to see the benefits form all seciotns of this patch, even without HT.

-Andrew Theurer