Skip to contentSkip to conversation

node capacity fixes for NUMA balancing

PATCH New [email protected] 2 participants 5 messages Aug 4, 2014 – Aug 12, 2014 Scheduler

lore
Share & tools
lore
#0 lore raw

cover node capacity fixes for NUMA balancing

The NUMA balancing code has a few issues with determining the capacity of nodes, and using it when doing a task move.

First the NUMA balancing code does not have the equivalent of c61037e9 to fix the "phantom cores" phenomenon in the presence of SMT.

Secondly, the NUMA balancing code will happily move a task from a node that is loaded to capacity, to another node that is loaded to capacity. This can leave the second node overloaded.

#1 lore raw

PATCH 1/2 sched,numa: fix off-by-one in capacity check

From: Rik van Riel <[email protected]>

Commit a43455a1d572daf7b730fe12eb747d1e17411365 ensures that task_numa_migrate will call task_numa_compare on the preferred node all the time, even when the preferred node has no free capacity.

This could lead to a performance regression if nr_running == capacity on both the source and the destination node. This can be avoided by also checking for nr_running == capacity on the source node, which is one stricter than checking .has_free_capacity.

---
 kernel/sched/fair.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/fair.c +1 -1
index bfa3c86..678ed03 100644
@@ -1206,7 +1206,7 @@ static void task_numa_compare(struct task_numa_env *env,
 
 	if (!cur) {
 		/* Is there capacity at our destination? */
-		if (env->src_stats.has_free_capacity &&
+		if (env->src_stats.nr_running <= env->src_stats.task_capacity &&
 		    !env->dst_stats.has_free_capacity)
 			goto unlock;
 
Signature · 1 line
1.9.3
#2 lore raw

[tip:sched/core] sched/numa: Fix off-by-one in capacity check

sched/numa: Fix off-by-one in capacity check
Commit-ID:  b932c03c34f3b03c7364c06aa8cae5b74609fc41
Gitweb:     http://git.kernel.org/tip/b932c03c34f3b03c7364c06aa8cae5b74609fc41
Author:     Rik van Riel <[email protected]>
AuthorDate: Mon, 4 Aug 2014 13:23:27 -0400
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 12 Aug 2014 12:48:22 +0200

sched/numa: Fix off-by-one in capacity check

Commit a43455a1d572daf7b730fe12eb747d1e17411365 ensures that task_numa_migrate will call task_numa_compare on the preferred node all the time, even when the preferred node has no free capacity.

This could lead to a performance regression if nr_running == capacity on both the source and the destination node. This can be avoided by also checking for nr_running == capacity on the source node, which is one stricter than checking .has_free_capacity.

---
 kernel/sched/fair.c | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
kernel/sched/fair.c +1 -1
index df1ed17..e1cf419 100644
@@ -1206,7 +1206,7 @@ static void task_numa_compare(struct task_numa_env *env,
 
 	if (!cur) {
 		/* Is there capacity at our destination? */
-		if (env->src_stats.has_free_capacity &&
+		if (env->src_stats.nr_running <= env->src_stats.task_capacity &&
 		    !env->dst_stats.has_free_capacity)
 			goto unlock;
 
#3 lore raw

PATCH 2/2 sched,numa: fix numa capacity computation

In reply to [email protected]

From: Rik van Riel <[email protected]>

Commit c61037e9 fixes the phenomenon of 'fantom' cores due to N*frac(smt_power) >= 1 by limiting the capacity to the actual number of cores in the load balancing code.

This patch applies the same correction to the NUMA balancing code.

---
 kernel/sched/fair.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
kernel/sched/fair.c +8 -3
index 678ed03..376bc07c 100644
@@ -1038,7 +1038,8 @@ struct numa_stats {
  */
 static void update_numa_stats(struct numa_stats *ns, int nid)
 {
-	int cpu, cpus = 0;
+	int smt, cpu, cpus = 0;
+	unsigned long capacity;
 
 	memset(ns, 0, sizeof(*ns));
 	for_each_cpu(cpu, cpumask_of_node(nid)) {
@@ -1062,8 +1063,12 @@ static void update_numa_stats(struct numa_stats *ns, int nid)
 	if (!cpus)
 		return;
 
-	ns->task_capacity =
-		DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_CAPACITY_SCALE);
+	/* smt := ceil(cpus / capacity), assumes: 1 < smt_power < 2 */
+	smt = DIV_ROUND_UP(SCHED_CAPACITY_SCALE * cpus, ns->compute_capacity);
+	capacity = cpus / smt; /* cores */
+
+	ns->task_capacity = min_t(unsigned, capacity,
+		DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_CAPACITY_SCALE));
 	ns->has_free_capacity = (ns->nr_running < ns->task_capacity);
 }
 
Signature · 1 line
1.9.3
#4 lore raw

[tip:sched/core] sched/numa: Fix numa capacity computation

sched/numa: Fix numa capacity computation
Commit-ID:  83d7f2424741c9dc76c21377c9d00d47abaf88df
Gitweb:     http://git.kernel.org/tip/83d7f2424741c9dc76c21377c9d00d47abaf88df
Author:     Rik van Riel <[email protected]>
AuthorDate: Mon, 4 Aug 2014 13:23:28 -0400
Committer:  Ingo Molnar <[email protected]>
CommitDate: Tue, 12 Aug 2014 12:48:23 +0200

sched/numa: Fix numa capacity computation

Commit c61037e9 fixes the phenomenon of 'fantom' cores due to N*frac(smt_power) >= 1 by limiting the capacity to the actual number of cores in the load balancing code.

This patch applies the same correction to the NUMA balancing code.

---
 kernel/sched/fair.c | 11 ++++++++---
 1 file changed, 8 insertions(+), 3 deletions(-)
kernel/sched/fair.c +8 -3
index e1cf419..1413c44 100644
@@ -1038,7 +1038,8 @@ struct numa_stats {
  */
 static void update_numa_stats(struct numa_stats *ns, int nid)
 {
-	int cpu, cpus = 0;
+	int smt, cpu, cpus = 0;
+	unsigned long capacity;
 
 	memset(ns, 0, sizeof(*ns));
 	for_each_cpu(cpu, cpumask_of_node(nid)) {
@@ -1062,8 +1063,12 @@ static void update_numa_stats(struct numa_stats *ns, int nid)
 	if (!cpus)
 		return;
 
-	ns->task_capacity =
-		DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_CAPACITY_SCALE);
+	/* smt := ceil(cpus / capacity), assumes: 1 < smt_power < 2 */
+	smt = DIV_ROUND_UP(SCHED_CAPACITY_SCALE * cpus, ns->compute_capacity);
+	capacity = cpus / smt; /* cores */
+
+	ns->task_capacity = min_t(unsigned, capacity,
+		DIV_ROUND_CLOSEST(ns->compute_capacity, SCHED_CAPACITY_SCALE));
 	ns->has_free_capacity = (ns->nr_running < ns->task_capacity);
 }