Skip to contentSkip to conversation

sched: fix sched domain degenerate

PATCH In review Siddha, Suresh B 2 participants 4 messages Apr 14, 2005 – Apr 16, 2005

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 sched: fix sched domain degenerate

Appended patch makes sched domain degenerate really work.

For example, now NUMA domain really gets removed on a non-NUMA system.

kernel/sched.c +1 -1
@@ -4777,7 +4777,7 @@ static int __devinit sd_parent_degenerat
 	/* WAKE_BALANCE is a subset of WAKE_AFFINE */
 	if (cflags & SD_WAKE_AFFINE)
 		pflags &= ~SD_WAKE_BALANCE;
-	if ((~sd->flags) & parent->flags)
+	if (~cflags & pflags)
 		return 0;
 
 	return 1;
include/linux/topology.h +2 -0
@@ -97,6 +97,7 @@
 	.flags			= SD_LOAD_BALANCE	\
 				| SD_BALANCE_NEWIDLE	\
 				| SD_BALANCE_EXEC	\
+				| SD_BALANCE_FORK	\
 				| SD_WAKE_AFFINE	\
 				| SD_WAKE_IDLE		\
 				| SD_SHARE_CPUPOWER,	\
@@ -128,6 +129,7 @@
 	.flags			= SD_LOAD_BALANCE	\
 				| SD_BALANCE_NEWIDLE	\
 				| SD_BALANCE_EXEC	\
+				| SD_BALANCE_FORK	\
 				| SD_WAKE_AFFINE,	\
 	.last_balance		= jiffies,		\
 	.balance_interval	= 1,			\
#1 lore raw
Quoted 10 lines · Suresh B

Appended patch makes sched domain degenerate really work.

For example, now NUMA domain really gets removed on a non-NUMA system.

kernel/sched.c static int __devinit sd_parent_degenerat · line 4783

Quoted 8 lines
kernel/sched.c +1 -1
@@ -4777,7 +4777,7 @@ static int __devinit sd_parent_degenerat
 	/* WAKE_BALANCE is a subset of WAKE_AFFINE */
 	if (cflags & SD_WAKE_AFFINE)
 		pflags &= ~SD_WAKE_BALANCE;
-	if ((~sd->flags) & parent->flags)
+	if (~cflags & pflags)
 		return 0;
 
 	return 1;

Thanks, I need a brown paper bag.

I think instead of the other 2 hunks in your patch I would like to do it the following way (I hope I get this right, finally).

Signature · 1 line
SUSE Labs, Novell Inc.

Make sched-remove-degenerate-domains.patch actually work.

Catch more (hopefully all) cases.

kernel/sched.c +8 -1
Index: linux-2.6/kernel/sched.c
===================================================================
@@ -4844,7 +4844,14 @@ static int __devinit sd_parent_degenerat
 	/* WAKE_BALANCE is a subset of WAKE_AFFINE */
 	if (cflags & SD_WAKE_AFFINE)
 		pflags &= ~SD_WAKE_BALANCE;
-	if ((~sd->flags) & parent->flags)
+	/* Flags needing groups don't count if only 1 group in parent */
+	if (parent->groups == parent->groups->next) {
+		pflags &= ~(SD_LOAD_BALANCE |
+				SD_BALANCE_NEWIDLE |
+				SD_BALANCE_FORK |
+				SD_BALANCE_EXEC);
+	}
+	if (~cflags & pflags)
 		return 0;
 
 	return 1;
#2 lore raw
Quoted 6 lines · Nick Piggin

Index: linux-2.6/kernel/sched.c
===================================================================

kernel/sched.c +0 -0
@@ -4844,7 +4844,14 @@ static int __devinit sd_parent_degenerat
 	/* WAKE_BALANCE is a subset of WAKE_AFFINE */

kernel/sched.c static int __devinit sd_parent_degenerat · line 4853

Quoted 10 lines
kernel/sched.c +7 -1
 	if (cflags & SD_WAKE_AFFINE)
 		pflags &= ~SD_WAKE_BALANCE;
-	if ((~sd->flags) & parent->flags)
+	/* Flags needing groups don't count if only 1 group in parent */
+	if (parent->groups == parent->groups->next) {
+		pflags &= ~(SD_LOAD_BALANCE |
+				SD_BALANCE_NEWIDLE |
+				SD_BALANCE_FORK |
+				SD_BALANCE_EXEC);
+	}

This patch works fine and I like this fix. But should n't we be adding SD_WAKE_AFFINE and SD_WAKE_BALANCE to this list?

And about SD_BALANCE_FORK, now that we have multi level sbe/sbf, we should add this flag to SD_CPU/SIBLING_INIT too..

thanks,
suresh

#3 lore raw
Quoted 13 lines · Suresh B
Quoted 11 lines · Nick Piggin

Index: linux-2.6/kernel/sched.c
===================================================================

kernel/sched.c +2 -1
@@ -4844,7 +4844,14 @@ static int __devinit sd_parent_degenerat
 	/* WAKE_BALANCE is a subset of WAKE_AFFINE */
 	if (cflags & SD_WAKE_AFFINE)
 		pflags &= ~SD_WAKE_BALANCE;
-	if ((~sd->flags) & parent->flags)
+	/* Flags needing groups don't count if only 1 group in parent */
+	if (parent->groups == parent->groups->next) {
Quoted 9 lines

kernel/sched.c static int __devinit sd_parent_degenerat · line 4853

Quoted 5 lines
kernel/sched.c +5 -0
+		pflags &= ~(SD_LOAD_BALANCE |
+				SD_BALANCE_NEWIDLE |
+				SD_BALANCE_FORK |
+				SD_BALANCE_EXEC);
+	}

This patch works fine and I like this fix. But should n't we be adding SD_WAKE_AFFINE and SD_WAKE_BALANCE to this list?

Hmm, well they don't use groups, but I guess they can be excluded, because if the parent span is the same as the child span (and that is true at this point), then SD_WAKE_AFFFINE/BALANCE in the parent will never be executed. Good point.

wake_idle should be doing a similar thing too, but that needs a bit of work.

Quoted 2 lines

And about SD_BALANCE_FORK, now that we have multi level sbe/sbf, we should add this flag to SD_CPU/SIBLING_INIT too..

I guess we should think about it. It depends - does SD_BALANCE_FORK make sense on a plain SMP machine? If so, then it probably makes sense to be in the 'SMP' domain on a NUMA system, otherwise not.

I suspect that for BALANCE_FORK, the answer may be no. On an SMP, it is far less disastrous to misplace tasks and have them picked up by the periodic rebalancer. What's more, BALANCE_FORK does add a non trivial overhead when moving tasks to other CPUs.

But it's open for debate. I haven't done comprehensive tests.

Signature · 1 line
SUSE Labs, Novell Inc.