Skip to contentSkip to conversation

sched/numa: skip VMA scanning on memory pinned to one NUMA node via cpuset.mems

PATCH Superseded Libo Chen 3 participants 5 messages Mar 11, 2025 – Mar 12, 2025 Scheduler

v1 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Libo Chen

author
#0 lore raw

PATCH 1/1 sched/numa: skip VMA scanning on memory pinned to one NUMA node via cpuset.mems

When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.

---
 kernel/sched/fair.c | 7 +++++++
 1 file changed, 7 insertions(+)
kernel/sched/fair.c +7 -0
index c798d27952431..ec4749a7be33a 100644
@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work)
 	if (p->flags & PF_EXITING)
 		return;
 
+	/*
+	 * Memory is pinned to only one NUMA node via cpuset.mems, naturally
+	 * no page can be migrated.
+	 */
+	if (nodes_weight(cpuset_current_mems_allowed) == 1)
+		return;
+
 	if (!mm->numa_next_scan) {
 		mm->numa_next_scan = now +
 			msecs_to_jiffies(sysctl_numa_balancing_scan_delay);
Signature · 1 line
2.43.5
#1 lore raw
Quoted 18 lines · Libo Chen

When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.

---
  kernel/sched/fair.c | 7 +++++++
  1 file changed, 7 insertions(+)
kernel/sched/fair.c +0 -0
index c798d27952431..ec4749a7be33a 100644
@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work)
  	if (p->flags & PF_EXITING)
  		return;
 

kernel/sched/fair.c static void task_numa_work(struct callback_head *work) · line 3327

Quoted 10 lines
kernel/sched/fair.c +7 -0
+	/*
+	 * Memory is pinned to only one NUMA node via cpuset.mems, naturally
+	 * no page can be migrated.
+	 */
+	if (nodes_weight(cpuset_current_mems_allowed) == 1)
+		return;
+
  	if (!mm->numa_next_scan) {
  		mm->numa_next_scan = now +
  			msecs_to_jiffies(sysctl_numa_balancing_scan_delay);

Do you have any performance improvement data that can be included in the commit log?

Cheers,
Longman

author
#2 lore raw
Quoted 24 lines · Waiman Long
Quoted 23 lines · Libo Chen

When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.

---
  kernel/sched/fair.c | 7 +++++++

  1 file changed, 7 insertions(+)
kernel/sched/fair.c +0 -0
index c798d27952431..ec4749a7be33a 100644
@@ -3315,6 +3315,13 @@ static void task_numa_work(struct 
 callback_head *work)
       if (p->flags & PF_EXITING)
           return;

  +    /*
+     * Memory is pinned to only one NUMA node via cpuset.mems, naturally
+     * no page can be migrated.
+     */

Quoted 9 lines
Quoted 6 lines

+    if (nodes_weight(cpuset_current_mems_allowed) == 1) +        return;
+
      if (!mm->numa_next_scan) { mm->numa_next_scan = now + msecs_to_jiffies(sysctl_numa_balancing_scan_delay);

Do you have any performance improvement data that can be included in the commit log?

Yes, will put out some numbers in v2.

Thanks,
Libo

Quoted 3 lines

Cheers,
Longman

maintainer
#3 lore raw
In reply to Libo Chen
Quoted 4 lines · Libo Chen

When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.

Its been a while since I looked at all this, but if we don't scan these pages, then it will not account for these pages, and the pinned memory will not become an attractor for the tasks that use this memory, right?

Quoted 26 lines
---
 kernel/sched/fair.c | 7 +++++++
 1 file changed, 7 insertions(+)
kernel/sched/fair.c +7 -0
index c798d27952431..ec4749a7be33a 100644
@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work)
 	if (p->flags & PF_EXITING)
 		return;
 
+	/*
+	 * Memory is pinned to only one NUMA node via cpuset.mems, naturally
+	 * no page can be migrated.
+	 */
+	if (nodes_weight(cpuset_current_mems_allowed) == 1)
+		return;
+
 	if (!mm->numa_next_scan) {
 		mm->numa_next_scan = now +
 			msecs_to_jiffies(sysctl_numa_balancing_scan_delay);
Signature · 1 line
2.43.5
author
#4 lore raw
Quoted 8 lines · Peter Zijlstra
Quoted 4 lines · Libo Chen

When the memory of the current task is pinned to one NUMA node by cgroup, there is no point in continuing the rest of VMA scanning and hinting page faults as they will just be overhead. With this change, there will be no more unnecessary PTE updates or page faults in this scenario.

Its been a while since I looked at all this, but if we don't scan these pages, then it will not account for these pages, and the pinned memory will not become an attractor for the tasks that use this memory, right?

Hi Peter,

Yes, you are absolutely right. It will skip change_prot_numa() which marks an VMA to be inaccessible (not PROT_NONE though) by setting MM_CP_PROT_NUMA flag, without that there will be no numa faults on those pages, hence no task migrations towards those pages. But that's how similar cases are handled such as hugetlb or MPOL_BIND pages. If you look at the checks !vma_migratable() || !vma_policy_mof(), etc, the rest of the loop will skipped if one of those is true. I am not sure why it's handle that way, maybe there are some old discussions I am not aware of, I personally think task migrations should be allowed even if the pages aren't.

Consider in most cases if cpuset.mems is set to one node, the same node assuming it's not a memory-only node is probably set in cpuset.cpus as well. I think I can add an additional check on CPU affinity, probably needs to add an helper cpumask_to_nodemask(), so something like below?

+ cpumask_to_nodemask(&current->cpus_mask, &allow_cpu_nodes);
+ if (nodes_weight(cpuset_current_mems_allowed) == 1 && nodes_weight(allow_cpu_nodes) == 1)
+ return;

Libo

Quoted 26 lines
Quoted 26 lines
---
  kernel/sched/fair.c | 7 +++++++
  1 file changed, 7 insertions(+)
kernel/sched/fair.c +7 -0
index c798d27952431..ec4749a7be33a 100644
@@ -3315,6 +3315,13 @@ static void task_numa_work(struct callback_head *work)
  	if (p->flags & PF_EXITING)
  		return;
 
+	/*
+	 * Memory is pinned to only one NUMA node via cpuset.mems, naturally
+	 * no page can be migrated.
+	 */
+	if (nodes_weight(cpuset_current_mems_allowed) == 1)
+		return;
+
  	if (!mm->numa_next_scan) {
  		mm->numa_next_scan = now +
  			msecs_to_jiffies(sysctl_numa_balancing_scan_delay);
Signature · 1 line
2.43.5