Skip to contentSkip to conversation

per-arch load balancing

PATCH In review Robert Love 3 participants 4 messages Aug 26, 2002 Generic include/asm header files

lore
Share & tools
lore

Robert Love

author
#0 lore raw

PATCH 1/1 per-arch load balancing

Linus,

The attached patch implements (optional) per-architecture load balancing so we can cleanly implement specialized load balancing behavior for NUMA, hyperthreading, etc.

The new method is "arch_load_balance()" and is defined (if available) in asm/smp_balance.h - otherwise it defines away. Currently, we call it from "find_busiest_queue()".

This patch, against current BK, only implements the infrastructure and not any particular new logic. This is a similar implementation as found in 2.4-ac.

Please, apply.

Robert Love

include/linux/smp_balance.h +14 -0
diff -urN linux-2.5.31/include/linux/smp_balance.h linux/include/linux/smp_balance.h
@@ -0,0 +1,14 @@
+#ifndef _LINUX_SMP_BALANCE_H
+#define _LINUX_SMP_BALANCE_H
+
+/*
+ * per-architecture load balancing logic, e.g. for hyperthreading
+ */
+
+#ifdef ARCH_HAS_SMP_BALANCE
+#include <asm/smp_balance.h>
+#else
+#define arch_load_balance(x, y)		(0)
+#endif
+
+#endif /* _LINUX_SMP_BALANCE_H */
kernel/sched.c +8 -0
diff -urN linux-2.5.31/kernel/sched.c linux/kernel/sched.c
@@ -630,6 +630,8 @@
 	return nr_running;
 }
 
+#include <linux/smp_balance.h>
+
 /*
  * find_busiest_queue - find the busiest runqueue.
  */

@@ -639,6 +641,12 @@
 	runqueue_t *busiest, *rq_src;
 
 	/*
+	 * Handle architecture-specific balancing, such as hyperthreading.
+	 */
+	if (arch_load_balance(this_cpu, idle))
+		return NULL;
+
+	/*
 	 * We search all runqueues to find the most busy one.
 	 * We do this lockless to reduce cache-bouncing overhead,
 	 * we re-check the 'best' source CPU later on again, with
#1 lore raw
Quoted 9 lines · Robert Love

Linus,

The attached patch implements (optional) per-architecture load balancing so we can cleanly implement specialized load balancing behavior for NUMA, hyperthreading, etc.

The new method is "arch_load_balance()" and is defined (if available) in asm/smp_balance.h - otherwise it defines away. Currently, we call it from "find_busiest_queue()".

Can we have a asm/sched.h instead? especially if we might add additional per-arch scheduler bits. Also I think a asm-generic version is better than linux/smp_balance.h + the ARCH_HAS_SMP_BALANCE hack. I'd prefer if you would move the #include ontop ot sched.c, too - includes in the middle of a file are really messy.

author
#2 lore raw
Quoted 5 lines · Christoph Hellwig

Can we have a asm/sched.h instead? especially if we might add additional per-arch scheduler bits. Also I think a asm-generic version is better than linux/smp_balance.h + the ARCH_HAS_SMP_BALANCE hack. I'd prefer if you would move the #include ontop ot sched.c, too - includes in the middle of a file are really messy.

These are all good ideas. Here we go again:

This patch implements per-arch scheduler support with specific support (currently) for per-arch load balancing. We implement an asm-generic/sched.h with default methods. Each architecture needs to define its own asm/sched.h but by default it would just include asm-generic/sched.h.

Better?

Robert Love

include/asm-generic/sched.h +14 -0
diff -urN linux-2.5.31/include/asm-generic/sched.h linux/include/asm-generic/sched.h
@@ -0,0 +1,14 @@
+#ifndef _LINUX_SCHED_H
+#define _LINUX_SCHED_H
+
+/*
+ * include/asm-generic/sched.h - generic and default versions of per-arch
+ * scheduler bits
+ */
+
+/*
+ * per-architecture load balancing logic, e.g. for hyperthreading
+ */
+#define arch_load_balance(x, y)		(0)
+
+#endif /* _LINUX_SCHED_H */
include/asm-i386/sched.h +7 -0
diff -urN linux-2.5.31/include/asm-i386/sched.h linux/include/asm-i386/sched.h
@@ -0,0 +1,7 @@
+#ifndef _I386_SCHED_H
+#define _I386_SCHED_H
+
+/* nothing to see here, move along */
+#include <asm-generic/sched.h>
+
+#endif /* _I386_SCHED_H */
kernel/sched.c +7 -0
diff -urN linux-2.5.31/kernel/sched.c linux/kernel/sched.c
@@ -29,6 +29,7 @@
 #include <linux/security.h>
 #include <linux/notifier.h>
 #include <linux/delay.h>
+#include <asm/sched.h>
 
 /*
  * Convert user-nice values [ -20 ... 0 ... 19 ]

@@ -639,6 +640,12 @@
 	runqueue_t *busiest, *rq_src;
 
 	/*
+	 * Handle architecture-specific balancing, such as hyperthreading.
+	 */
+	if (arch_load_balance(this_cpu, idle))
+		return NULL;
+
+	/*
 	 * We search all runqueues to find the most busy one.
 	 * We do this lockless to reduce cache-bouncing overhead,
 	 * we re-check the 'best' source CPU later on again, with
#3 lore raw
In reply to Robert Love
Quoted 3 lines · Robert Love

The attached patch implements (optional) per-architecture load balancing so we can cleanly implement specialized load balancing behavior for NUMA, hyperthreading, etc.

nope, this is not the right approach, at least not for hyperthreading - i've got a generic approach in my tree, will post a patch soon (tm).

Quoted 1 line

Please, apply.

please dont ...

Ingo