sched/numa: skip VMA scanning on memory pinned to one NUMA node via cpuset.mems
PATCH 1/1 sched/numa: skip VMA scanning on memory pinned to one NUMA node via cpuset.mems
When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.
--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)
kernel/sched/fair.c +7 -0
@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work)if (p->flags &PF_EXITING )return ; +/* +* Memory is pinned to only one NUMA node via cpuset.mems, naturally +* no page can be migrated. +*/ +if (nodes_weight (cpuset_current_mems_allowed) ==1 ) +return ; +if (!mm->numa_next_scan) { mm->numa_next_scan = now +msecs_to_jiffies (sysctl_numa_balancing_scan_delay);
Signature · 1 line
2.43.5
Quoted 18 lines · Libo Chen
When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.
Signed-off-by Libo Chen <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work)if (p->flags &PF_EXITING )return ;
kernel/sched/fair.c static void task_numa_work(struct callback_head *work) · line 3327
Quoted 10 lines
kernel/sched/fair.c +7
-0+/* +* Memory is pinned to only one NUMA node via cpuset.mems, naturally +* no page can be migrated. +*/ +if (nodes_weight (cpuset_current_mems_allowed) ==1 ) +return ; +if (!mm->numa_next_scan) { mm->numa_next_scan = now +msecs_to_jiffies (sysctl_numa_balancing_scan_delay);
Do you have any performance improvement data that can be included in the commit log?
Cheers,
Longman
Quoted 24 lines · Waiman Long
Quoted 23 lines · Libo Chen
When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.
Signed-off-by Libo Chen <[email protected]>---
kernel/sched/fair.c| 7 +++++++1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work) if (p->flags & PF_EXITING) return;+ /*
+ * Memory is pinned to only one NUMA node via cpuset.mems, naturally
+ * no page can be migrated.
+ */
Quoted 9 lines
Quoted 6 lines
+ if (
nodes_weight(cpuset_current_mems_allowed)== 1) + return;
+
if (!mm->numa_next_scan) {mm->numa_next_scan= now +msecs_to_jiffies(sysctl_numa_balancing_scan_delay);Do you have any performance improvement data that can be included in the commit log?
Yes, will put out some numbers in v2.
Thanks,
Libo
Quoted 3 lines
Cheers,
Longman
Peter Zijlstra
maintainerQuoted 4 lines · Libo Chen
When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.
Its been a while since I looked at all this, but if we don't scan these pages, then it will not account for these pages, and the pinned memory will not become an attractor for the tasks that use this memory, right?
Quoted 26 lines
Signed-off-by Libo Chen <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +7
-0@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work)if (p->flags &PF_EXITING )return ; +/* +* Memory is pinned to only one NUMA node via cpuset.mems, naturally +* no page can be migrated. +*/ +if (nodes_weight (cpuset_current_mems_allowed) ==1 ) +return ; +if (!mm->numa_next_scan) { mm->numa_next_scan = now +msecs_to_jiffies (sysctl_numa_balancing_scan_delay);Signature · 1 line
2.43.5
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Libo Chen
When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.
Its been a while since I looked at all this, but if we don't scan these pages, then it will not account for these pages, and the pinned memory will not become an attractor for the tasks that use this memory, right?
Hi Peter,
Yes, you are absolutely right. It will skip change_prot_numa() which marks an VMA to be inaccessible (not PROT_NONE though) by setting MM_CP_PROT_NUMA flag, without that there will be no numa faults on those pages, hence no task migrations towards those pages. But that's how similar cases are handled such as hugetlb or MPOL_BIND pages. If you look at the checks !vma_migratable() || !vma_policy_mof(), etc, the rest of the loop will skipped if one of those is true. I am not sure why it's handle that way, maybe there are some old discussions I am not aware of, I personally think task migrations should be allowed even if the pages aren't.
Consider in most cases if cpuset.mems is set to one node, the same node assuming it's not a memory-only node is probably set in cpuset.cpus as well. I think I can add an additional check on CPU affinity, probably needs to add an helper cpumask_to_nodemask(), so something like below?
+ cpumask_to_nodemask(¤t->cpus_mask, &allow_cpu_nodes);
+ if (nodes_weight(cpuset_current_mems_allowed) == 1 && nodes_weight(allow_cpu_nodes) == 1)
+ return;
Libo
Quoted 26 lines
Quoted 26 lines
Signed-off-by Libo Chen <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +7
-0@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work)if (p->flags &PF_EXITING )return ; +/* +* Memory is pinned to only one NUMA node via cpuset.mems, naturally +* no page can be migrated. +*/ +if (nodes_weight (cpuset_current_mems_allowed) ==1 ) +return ; +if (!mm->numa_next_scan) { mm->numa_next_scan = now +msecs_to_jiffies (sysctl_numa_balancing_scan_delay);Signature · 1 line
2.43.5