scheduler fix / cross node balancer change

A new attempt to fix the 1 CPU per node problem of the current NUMA scheduler. The previous version was a bit too aggressive with cross node balancing.

This patch:
- links the frequency of cross-node balances to the number of failed local balance attempts while simplifying the code,
- fixes the 1 CPU/node issue, i.e. eliminates local balance attempts for the nodes which have only one CPU. Can happen on any NUMA platform, is a major issue on x86_64.
- makes the cross-node balance frequency tunable by the parameter NUMA_FACTOR_BONUS. Its default setting is such that the scheduler behaves like before: cross node balance every 5 local node balances on an idle CPU, every 2 local node balances on a busy CPU. This parameter should be tuned for each platform depending on its NUMA factor.

Regards,
Erich

diff -urNp 2.6.0-test3/include/linux/topology.h 2.6.0-test3-1cpuf/include/linux/topology.h
@@ -48,4 +48,13 @@ static inline int __next_node_with_cpus(
 #define for_each_node_with_cpus(node) \
 	for (node = 0; node < numnodes; node = __next_node_with_cpus(node))
 
+#ifndef NUMA_FACTOR_BONUS
+/*
+ * High NUMA_FACTOR_BONUS means rare cross-node load balancing. The default
+ * value of 3 means idle_node_rebalance after 5 (failed) local balances,
+ * busy_node_rebalance after 2 failed local balances. Should be tuned for
+ * each platform in asm/topology.h.
+ */
+#define NUMA_FACTOR_BONUS 3
+#endif
 #endif /* _LINUX_TOPOLOGY_H */