sched,numa: always try to migrate to preferred node at task_numa_placement time
PATCH 1/1 sched,numa: always try to migrate to preferred node at task_numa_placement time
It is possible that at task_numa_placement time, the task's numa_preferred_nid does not change, but the task is not actually running on the preferred node at the time.
In that case, we still want to attempt migration to the preferred node.
--- kernel/sched/fair.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
kernel/sched/fair.c +7 -5
@@ -1575,11 +1575,13 @@ static void task_numa_placement(struct task_struct *p)spin_unlock_irq (group_lock); } -/* Preferred node as the node with the most faults */ -if (max_faults && max_nid != p->numa_preferred_nid) { -/* Update the preferred nid and migrate task if possible */ -sched_setnuma (p, max_nid); -numa_migrate_preferred (p); +if (max_faults) { +/* Set the new preferred node */ +if (max_nid != p->numa_preferred_nid) +sched_setnuma (p, max_nid); + +if (task_node (p) != p->numa_preferred_nid) +numa_migrate_preferred (p); } }
Peter Zijlstra
maintainerQuoted 6 lines · Rik van Riel
It is possible that at
task_numa_placementtime, the task'snuma_preferred_niddoes not change, but the task is not actually running on the preferred node at the time.In that case, we still want to attempt migration to the preferred node.
So we have that numa_migrate_retry which was supposed to keep kicking the task until it got where it needed to go.
But now you continuously kick from task_numa_placement().
Clearly the retry thing didn't work, what happened? We got to the preferred nid, disabled the retry and got moved away again?
Do we want to remove the retry logic in favour of this more aggressive form?
Quoted 11 lines
kernel/sched/fair.c +7
-0@@ -1575,11 +1575,13 @@ static void task_numa_placement(struct task_struct *p) +if (max_faults) { +/* Set the new preferred node */ +if (max_nid != p->numa_preferred_nid) +sched_setnuma (p, max_nid); + +if (task_node (p) != p->numa_preferred_nid) +numa_migrate_preferred (p); }
Quoted 10 lines · Peter Zijlstra
Quoted 6 lines · Rik van Riel
It is possible that at
task_numa_placementtime, the task'snuma_preferred_niddoes not change, but the task is not actually running on the preferred node at the time.In that case, we still want to attempt migration to the preferred node.
So we have that
numa_migrate_retrywhich was supposed to keep kicking the task until it got where it needed to go.
It does, and it appears to work.
Quoted 1 line
But now you continuously kick from
task_numa_placement().
No, we only kick from task_numa_placement() if the task is not already running on its preferred nid.
Quoted 5 lines
Clearly the retry thing didn't work, what happened? We got to the preferred nid, disabled the retry and got moved away again?
Do we want to remove the retry logic in favour of this more aggressive form?
I think we want both. When we have fresh statistics, and we discover that we are not running on our preferred nid, is there any reason not to relocate to a better node?
Moving a task to another node is cheap, and moving it sooner means we can end up avoiding migrating memory around twice.
Quoted 13 lines
Quoted 11 lines
kernel/sched/fair.c +7
-0@@ -1575,11 +1575,13 @@ static void task_numa_placement(struct task_struct *p) +if (max_faults) { +/* Set the new preferred node */ +if (max_nid != p->numa_preferred_nid) +sched_setnuma (p, max_nid); + +if (task_node (p) != p->numa_preferred_nid) +numa_migrate_preferred (p); }
Signature · 1 line
All rights reversed
Peter Zijlstra
maintainerQuoted 11 lines · Rik van Riel
Quoted 5 lines · Peter Zijlstra
Clearly the retry thing didn't work, what happened? We got to the preferred nid, disabled the retry and got moved away again?
Do we want to remove the retry logic in favour of this more aggressive form?
I think we want both. When we have fresh statistics, and we discover that we are not running on our preferred nid, is there any reason not to relocate to a better node?
I got confused on the exact details, I thought we ended up calling task_numa_placement() far often than we actually do (well, we do call it as often, but bail early).
Yes this makes sense.
tip-bot for Rik van Riel
merge bot[tip:sched/core] sched/numa: Always try to migrate to preferred node at task_numa_placement() time
In reply to Rik van Rielsched/numa: Always try to migrate to preferred node at task_numa_placement() time
Commit-ID: bb97fc31647539f1f102eed646a95e200160a150 Gitweb: http://git.kernel.org/tip/bb97fc31647539f1f102eed646a95e200160a150 Author: Rik van Riel <[email protected]> AuthorDate: Wed, 4 Jun 2014 16:33:15 -0400 Committer: Ingo Molnar <[email protected]> CommitDate: Wed, 18 Jun 2014 18:29:58 +0200
sched/numa: Always try to migrate to preferred node at task_numa_placement() time
It is possible that at task_numa_placement() time, the task's numa_preferred_nid does not change, but the task is not actually running on the preferred node at the time.
In that case, we still want to attempt migration to the preferred node.
--- kernel/sched/fair.c | 12 +++++++----- 1 file changed, 7 insertions(+), 5 deletions(-)
kernel/sched/fair.c +7 -5
@@ -1613,11 +1613,13 @@ static void task_numa_placement(struct task_struct *p)spin_unlock_irq (group_lock); } -/* Preferred node as the node with the most faults */ -if (max_faults && max_nid != p->numa_preferred_nid) { -/* Update the preferred nid and migrate task if possible */ -sched_setnuma (p, max_nid); -numa_migrate_preferred (p); +if (max_faults) { +/* Set the new preferred node */ +if (max_nid != p->numa_preferred_nid) +sched_setnuma (p, max_nid); + +if (task_node (p) != p->numa_preferred_nid) +numa_migrate_preferred (p); } }