memory tiering: Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled
Subsystems (2)
PATCH v2 1/1 memory tiering: Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled
In the current implementation, if NUMA_BALANCING_MEMORY_TIERING is disabled and the pages are on the lower tier, the pages may still be promoted.
This happens because task_numa_work() updates the last_cpupid field to record the last access time only when NUMA_BALANCING_MEMORY_TIERING is enabled and the folio is on the lower tier. If NUMA_BALANCING_MEMORY_TIERING is disabled, the last_cpupid field can retains a valid last CPU id.
In should_numa_migrate_memory(), the decision checks whether NUMA_BALANCING_MEMORY_TIERING is disabled, the folio is on the lower tier, and last_cpupid is invalid. However, the last_cpupid can be valid when NUMA_BALANCING_MEMORY_TIERING is disabled, the condition evaluates to false and migration is allowed.
This patch prevents promotion when NUMA_BALANCING_MEMORY_TIERING is disabled and the folio is on the lower tier.
Behavior before this change:
============================
- If NUMA_BALANCING_NORMAL is enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.
- If NUMA_BALANCING_MEMORY_TIERING is enabled, promotion from lower tier to higher tier nodes is allowed.
Behavior after this change:
===========================
- If NUMA_BALANCING_NORMAL is enabled, migration will occur only between nodes within the same memory tier.
- If NUMA_BALANCING_MEMORY_TIERING is enabled, promotion from lower tier to higher tier nodes will be allowed.
- If both NUMA_BALANCING_MEMORY_TIERING and NUMA_BALANCING_NORMAL are enabled, both migration (same tier) and promotion (cross tier) are allowed.
---
v1 -> v2
========
1. Dropped changes in task_numa_fault() since the original changes already handle runtime disabling of NUMA_BALANCING_MEMORY_TIERING.
v1 -> https://lore.kernel.org/all/[email protected]/
--- kernel/sched/fair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/sched/fair.c +5 -1
@@ -2024,8 +2024,12 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, this_cpupid =cpu_pid_to_cpupid (dst_cpu, current->pid); last_cpupid =folio_xchg_last_cpupid (folio, this_cpupid); +/* +* Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled +* and the pages are on the lower tier. +*/ if (!(sysctl_numa_balancing_mode &NUMA_BALANCING_MEMORY_TIERING ) && - !node_is_toptier (src_nid) && !cpupid_valid (last_cpupid)) + !node_is_toptier (src_nid))return false;/*
Signature · 1 line
2.47.1
Andrew Morton
maintainerQuoted 33 lines · On Mon, 23 Mar 2026 04:48:49 -0500 Donet Tom <[email protected]> wrote:
In the current implementation, if
NUMA_BALANCING_MEMORY_TIERINGis disabled and the pages are on the lower tier, the pages may still be promoted.This happens because
task_numa_work()updates thelast_cpupidfield to record the last access time only whenNUMA_BALANCING_MEMORY_TIERINGis enabled and the folio is on the lower tier. IfNUMA_BALANCING_MEMORY_TIERINGis disabled, thelast_cpupidfield can retains a valid last CPU id.In
should_numa_migrate_memory(), the decision checks whetherNUMA_BALANCING_MEMORY_TIERINGis disabled, the folio is on the lower tier, andlast_cpupidis invalid. However, thelast_cpupidcan be valid whenNUMA_BALANCING_MEMORY_TIERINGis disabled, the condition evaluates to false and migration is allowed.This patch prevents promotion when
NUMA_BALANCING_MEMORY_TIERINGis disabled and the folio is on the lower tier.Behavior before this change:
============================
- IfNUMA_BALANCING_NORMALis enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes is allowed.Behavior after this change:
===========================
- IfNUMA_BALANCING_NORMALis enabled, migration will occur only between nodes within the same memory tier.
Quoted 6 lines
- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes will be allowed.- If both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALare enabled, both migration (same tier) and promotion (cross tier) are allowed.
There was no feedback on this, nor on your v1.
Quoted 1 line
Fixes 33024536bafd ("memory tiering: hot page selection with hint page fault latency")
Ying Huang seems to have moved around a bit - let me add a couple more email addresses. Apologies if we have multiple Ying Huangs!
Rik, Mel? It's a bugfix.
Thanks.
From: Donet Tom <[email protected]>
Subject: memory tiering: do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled
Date: Mon, 23 Mar 2026 04:48:49 -0500
In the current implementation, if NUMA_BALANCING_MEMORY_TIERING is disabled and the pages are on the lower tier, the pages may still be promoted.
This happens because task_numa_work() updates the last_cpupid field to record the last access time only when NUMA_BALANCING_MEMORY_TIERING is enabled and the folio is on the lower tier. If NUMA_BALANCING_MEMORY_TIERING is disabled, the last_cpupid field can retains a valid last CPU id.
In should_numa_migrate_memory(), the decision checks whether NUMA_BALANCING_MEMORY_TIERING is disabled, the folio is on the lower tier, and last_cpupid is invalid. However, the last_cpupid can be valid when NUMA_BALANCING_MEMORY_TIERING is disabled, the condition evaluates to false and migration is allowed.
This patch prevents promotion when NUMA_BALANCING_MEMORY_TIERING is disabled and the folio is on the lower tier.
Behavior before this change:
============================
- If NUMA_BALANCING_NORMAL is enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.
- If NUMA_BALANCING_MEMORY_TIERING is enabled, promotion from lower tier to higher tier nodes is allowed.
Behavior after this change:
===========================
- If NUMA_BALANCING_NORMAL is enabled, migration will occur only between nodes within the same memory tier.
- If NUMA_BALANCING_MEMORY_TIERING is enabled, promotion from lower tier to higher tier nodes will be allowed.
- If both NUMA_BALANCING_MEMORY_TIERING and NUMA_BALANCING_NORMAL are enabled, both migration (same tier) and promotion (cross tier) are allowed.
---
kernel/sched/fair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)
kernel/sched/fair.c renamed from kernel/sched/fair.c~memory-tiering-do-not-allow-promotion-if-numa_balancing_memory_tiering-is-disabled +5 -1
@@ -2024,8 +2024,12 @@ bool should_numa_migrate_memory(struct t this_cpupid =cpu_pid_to_cpupid (dst_cpu, current->pid); last_cpupid =folio_xchg_last_cpupid (folio, this_cpupid); +/* +* Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled +* and the pages are on the lower tier. +*/ if (!(sysctl_numa_balancing_mode &NUMA_BALANCING_MEMORY_TIERING ) && - !node_is_toptier (src_nid) && !cpupid_valid (last_cpupid)) + !node_is_toptier (src_nid))return false;/*
_
Hi, Andrew,
Quoted 42 lines · Andrew Morton
Quoted 33 lines · On Mon, 23 Mar 2026 04:48:49 -0500 Donet Tom <[email protected]> wrote:
In the current implementation, if
NUMA_BALANCING_MEMORY_TIERINGis disabled and the pages are on the lower tier, the pages may still be promoted.This happens because
task_numa_work()updates thelast_cpupidfield to record the last access time only whenNUMA_BALANCING_MEMORY_TIERINGis enabled and the folio is on the lower tier. IfNUMA_BALANCING_MEMORY_TIERINGis disabled, thelast_cpupidfield can retains a valid last CPU id.In
should_numa_migrate_memory(), the decision checks whetherNUMA_BALANCING_MEMORY_TIERINGis disabled, the folio is on the lower tier, andlast_cpupidis invalid. However, thelast_cpupidcan be valid whenNUMA_BALANCING_MEMORY_TIERINGis disabled, the condition evaluates to false and migration is allowed.This patch prevents promotion when
NUMA_BALANCING_MEMORY_TIERINGis disabled and the folio is on the lower tier.Behavior before this change:
============================
- IfNUMA_BALANCING_NORMALis enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes is allowed.Behavior after this change:
===========================
- IfNUMA_BALANCING_NORMALis enabled, migration will occur only between nodes within the same memory tier.Quoted 6 lines
- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes will be allowed.- If both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALare enabled, both migration (same tier) and promotion (cross tier) are allowed.
Quoted 6 lines
There was no feedback on this, nor on your v1.
Quoted 1 line
Fixes 33024536bafd ("memory tiering: hot page selection with hint page fault latency")Ying Huang seems to have moved around a bit - let me add a couple more email addresses. Apologies if we have multiple Ying Huangs!
Thanks! I don't find other Ying Huang in mm community yet.
Now I use the following email address:
"Huang, Ying" <[email protected]>
Ying Huang <[email protected]>
and stop using the following email address:
Quoted 88 lines
Rik, Mel? It's a bugfix.
Thanks.
From: Donet Tom <[email protected]>
Subject: memory tiering: do not allow promotion ifNUMA_BALANCING_MEMORY_TIERINGis disabled
Date: Mon, 23 Mar 2026 04:48:49 -0500In the current implementation, if
NUMA_BALANCING_MEMORY_TIERINGis disabled and the pages are on the lower tier, the pages may still be promoted.This happens because
task_numa_work()updates thelast_cpupidfield to record the last access time only whenNUMA_BALANCING_MEMORY_TIERINGis enabled and the folio is on the lower tier. IfNUMA_BALANCING_MEMORY_TIERINGis disabled, thelast_cpupidfield can retains a valid last CPU id.In
should_numa_migrate_memory(), the decision checks whetherNUMA_BALANCING_MEMORY_TIERINGis disabled, the folio is on the lower tier, andlast_cpupidis invalid. However, thelast_cpupidcan be valid whenNUMA_BALANCING_MEMORY_TIERINGis disabled, the condition evaluates to false and migration is allowed.This patch prevents promotion when
NUMA_BALANCING_MEMORY_TIERINGis disabled and the folio is on the lower tier.Behavior before this change:
============================
- IfNUMA_BALANCING_NORMALis enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes is allowed.Behavior after this change:
===========================
- IfNUMA_BALANCING_NORMALis enabled, migration will occur only between nodes within the same memory tier.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes will be allowed.- If both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALare enabled, both migration (same tier) and promotion (cross tier) are allowed.Fixes 33024536bafd ("memory tiering: hot page selection with hint page fault latency")Signed-off-by Donet Tom <[email protected]>Cc Baolin Wang <[email protected]>Cc Ben Segall <[email protected]>Cc David Hildenbrand <[email protected]>Cc Dietmar Eggemann <[email protected]>Cc "Huang, Ying" <[email protected]>Cc Ingo Molnar <[email protected]>Cc Juri Lelli <[email protected]>Cc Mel Gorman <[email protected]>Cc Peter Zijlstra <[email protected]>Cc "Ritesh Harjani (IBM)" <[email protected]>Cc Steven Rostedt <[email protected]>Cc Valentin Schneider <[email protected]>Cc Vincent Guittot <[email protected]>Signed-off-by Andrew Morton <[email protected]>---
kernel/sched/fair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)kernel/sched/fair.c renamed from kernel/sched/fair.c~memory-tiering-do-not-allow-promotion-if-numa_balancing_memory_tiering-is-disabled +5
-1@@ -2024,8 +2024,12 @@ bool should_numa_migrate_memory(struct t this_cpupid =cpu_pid_to_cpupid (dst_cpu, current->pid); last_cpupid =folio_xchg_last_cpupid (folio, this_cpupid); +/* +* Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled +* and the pages are on the lower tier. +*/ if (!(sysctl_numa_balancing_mode &NUMA_BALANCING_MEMORY_TIERING ) && - !node_is_toptier (src_nid) && !cpupid_valid (last_cpupid)) + !node_is_toptier (src_nid))return false;/*
Quoted 1 line
_
---
Best Regards,
Huang, Ying
Quoted 61 lines · Donet Tom
In the current implementation, if
NUMA_BALANCING_MEMORY_TIERINGis disabled and the pages are on the lower tier, the pages may still be promoted.This happens because
task_numa_work()updates thelast_cpupidfield to record the last access time only whenNUMA_BALANCING_MEMORY_TIERINGis enabled and the folio is on the lower tier. IfNUMA_BALANCING_MEMORY_TIERINGis disabled, thelast_cpupidfield can retains a valid last CPU id.In
should_numa_migrate_memory(), the decision checks whetherNUMA_BALANCING_MEMORY_TIERINGis disabled, the folio is on the lower tier, andlast_cpupidis invalid. However, thelast_cpupidcan be valid whenNUMA_BALANCING_MEMORY_TIERINGis disabled, the condition evaluates to false and migration is allowed.This patch prevents promotion when
NUMA_BALANCING_MEMORY_TIERINGis disabled and the folio is on the lower tier.Behavior before this change:
============================
- IfNUMA_BALANCING_NORMALis enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes is allowed.Behavior after this change:
===========================
- IfNUMA_BALANCING_NORMALis enabled, migration will occur only between nodes within the same memory tier.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes will be allowed.- If both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALare enabled, both migration (same tier) and promotion (cross tier) are allowed.Fixes 33024536bafd ("memory tiering: hot page selection with hint page fault latency")Signed-off-by Donet Tom <[email protected]>---
v1 -> v2
========
1. Dropped changes intask_numa_fault()since the original changes already handle runtime disabling ofNUMA_BALANCING_MEMORY_TIERING.v1 -> https://lore.kernel.org/all/[email protected]/
--- kernel/sched/fair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)kernel/sched/fair.c +0
-0@@ -2024,8 +2024,12 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, this_cpupid =cpu_pid_to_cpupid (dst_cpu, current->pid); last_cpupid =folio_xchg_last_cpupid (folio, this_cpupid);
kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 2035
Quoted 10 lines
kernel/sched/fair.c +5
-1+/* +* Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled +* and the pages are on the lower tier. +*/ if (!(sysctl_numa_balancing_mode &NUMA_BALANCING_MEMORY_TIERING ) && - !node_is_toptier (src_nid) && !cpupid_valid (last_cpupid)) + !node_is_toptier (src_nid))return false;/*
No. Even if NUMA_BALANCING_MEMORY_TIERING is disabled, we should still allow migrate pages from lower tier to higher tier via NUMA_BALANCING_NORMAL. If we have precious DDR, why waste it? This follows the semantics of NUMA_BALANCING_NORMAL before introducing NUMA_BALANCING_MEMORY_TIERING.
---
Best Regards,
Huang, Ying
Hi
Quoted 72 lines · Ying
Quoted 70 lines · Donet Tom
In the current implementation, if
NUMA_BALANCING_MEMORY_TIERINGis disabled and the pages are on the lower tier, the pages may still be promoted.This happens because
task_numa_work()updates thelast_cpupidfield to record the last access time only whenNUMA_BALANCING_MEMORY_TIERINGis enabled and the folio is on the lower tier. IfNUMA_BALANCING_MEMORY_TIERINGis disabled, thelast_cpupidfield can retains a valid last CPU id.In
should_numa_migrate_memory(), the decision checks whetherNUMA_BALANCING_MEMORY_TIERINGis disabled, the folio is on the lower tier, andlast_cpupidis invalid. However, thelast_cpupidcan be valid whenNUMA_BALANCING_MEMORY_TIERINGis disabled, the condition evaluates to false and migration is allowed.This patch prevents promotion when
NUMA_BALANCING_MEMORY_TIERINGis disabled and the folio is on the lower tier.Behavior before this change:
============================
- IfNUMA_BALANCING_NORMALis enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes is allowed.Behavior after this change:
===========================
- IfNUMA_BALANCING_NORMALis enabled, migration will occur only between nodes within the same memory tier.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes will be allowed.- If both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALare enabled, both migration (same tier) and promotion (cross tier) are allowed.Fixes 33024536bafd ("memory tiering: hot page selection with hint page fault latency")Signed-off-by Donet Tom <[email protected]>---
v1 -> v2
========
1. Dropped changes intask_numa_fault()since the original changes already handle runtime disabling ofNUMA_BALANCING_MEMORY_TIERING.v1 -> https://lore.kernel.org/all/[email protected]/
--- kernel/sched/fair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)kernel/sched/fair.c +5
-1@@ -2024,8 +2024,12 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, this_cpupid =cpu_pid_to_cpupid (dst_cpu, current->pid); last_cpupid =folio_xchg_last_cpupid (folio, this_cpupid); +/* +* Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled +* and the pages are on the lower tier. +*/ if (!(sysctl_numa_balancing_mode &NUMA_BALANCING_MEMORY_TIERING ) && - !node_is_toptier (src_nid) && !cpupid_valid (last_cpupid)) + !node_is_toptier (src_nid))return false;
Quoted 6 lines
kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 2035
Quoted 1 line
kernel/sched/fair.c +0
-0/* No. Even if
NUMA_BALANCING_MEMORY_TIERINGis disabled, we should still allow migrate pages from lower tier to higher tier viaNUMA_BALANCING_NORMAL. If we have precious DDR, why waste it? This follows the semantics ofNUMA_BALANCING_NORMALbefore introducingNUMA_BALANCING_MEMORY_TIERING.
Thank you for the review comments.
One thing I am trying to understand is that page promotion appears to happen regardless of whether NUMA_BALANCING_MEMORY_TIERING is enabled or disabled. In that case, what is the specific role of NUMA_BALANCING_MEMORY_TIERING? Do we get better performance when it is enabled?
My initial understanding was that disabling NUMA_BALANCING_MEMORY_TIERING could be used to turn off promotion. However, it seems that currently we cannot control promotion independently. If NUMA_BALANCING_NORMAL is disabled, neither migration nor promotion happens, and if it is enabled, both migration and promotion can occur.
I was under the impression that:
- NUMA_BALANCING_NORMAL would handle migration within the same tier,
- NUMA_BALANCING_MEMORY_TIERING would handle promotion across tiers,
- and enabling both would allow both migration and promotion.
This would provide more fine-grained control. Is my understanding correct, or am I missing something here?
Quoted 4 lines
---
Best Regards,
Huang, Ying
Quoted 1 line · Donet Tom
Hi
Hi, Donet,
Quoted 78 lines · Ying
Quoted 68 lines
Quoted 66 lines · Donet Tom
In the current implementation, if
NUMA_BALANCING_MEMORY_TIERINGis disabled and the pages are on the lower tier, the pages may still be promoted.This happens because
task_numa_work()updates thelast_cpupidfield to record the last access time only whenNUMA_BALANCING_MEMORY_TIERINGis enabled and the folio is on the lower tier. IfNUMA_BALANCING_MEMORY_TIERINGis disabled, thelast_cpupidfield can retains a valid last CPU id.In
should_numa_migrate_memory(), the decision checks whetherNUMA_BALANCING_MEMORY_TIERINGis disabled, the folio is on the lower tier, andlast_cpupidis invalid. However, thelast_cpupidcan be valid whenNUMA_BALANCING_MEMORY_TIERINGis disabled, the condition evaluates to false and migration is allowed.This patch prevents promotion when
NUMA_BALANCING_MEMORY_TIERINGis disabled and the folio is on the lower tier.Behavior before this change:
============================
- IfNUMA_BALANCING_NORMALis enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes is allowed.Behavior after this change:
===========================
- IfNUMA_BALANCING_NORMALis enabled, migration will occur only between nodes within the same memory tier.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes will be allowed.- If both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALare enabled, both migration (same tier) and promotion (cross tier) are allowed.Fixes 33024536bafd ("memory tiering: hot page selection with hint page fault latency")Signed-off-by Donet Tom <[email protected]>---
v1 -> v2
========
1. Dropped changes intask_numa_fault()since the original changes already handle runtime disabling ofNUMA_BALANCING_MEMORY_TIERING.v1 -> https://lore.kernel.org/all/[email protected]/
--- kernel/sched/fair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)kernel/sched/fair.c +3
-1@@ -2024,8 +2024,12 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, this_cpupid =cpu_pid_to_cpupid (dst_cpu, current->pid); last_cpupid =folio_xchg_last_cpupid (folio, this_cpupid); +/* +* Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled +* and the pages are on the lower tier. +*/ if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && -!node_is_toptier(src_nid) && !cpupid_valid(last_cpupid)) Quoted 8 lines
kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 2035
Quoted 3 lines
kernel/sched/fair.c +1
-0+ !node_is_toptier(src_nid))return false;/* No. Even if
NUMA_BALANCING_MEMORY_TIERINGis disabled, we should still allow migrate pages from lower tier to higher tier viaNUMA_BALANCING_NORMAL. If we have precious DDR, why waste it? This follows the semantics ofNUMA_BALANCING_NORMALbefore introducingNUMA_BALANCING_MEMORY_TIERING.
Quoted 8 lines
Thank you for the review comments.
One thing I am trying to understand is that page promotion appears to happen regardless of whether
NUMA_BALANCING_MEMORY_TIERINGis enabled or disabled. In that case, what is the specific role ofNUMA_BALANCING_MEMORY_TIERING? Do we get better performance when it is enabled?
You can search NUMA_BALANCING_MEMORY_TIERING to find out what it does. We can get better performance as the original commit message says.
When NUMA_BALANCING_MEMORY_TIERING is introduced, we didn't change the original behavior of NUMA_BALANCING_MEMORY_NORMAL because we had no good reason to do that. In fact, you change its behavior, so you should provide some supporting data or bug report to justify the change.
Quoted 7 lines
My initial understanding was that disabling
NUMA_BALANCING_MEMORY_TIERINGcould be used to turn off promotion. However, it seems that currently we cannot control promotion independently. IfNUMA_BALANCING_NORMALis disabled, neither migration nor promotion happens, and if it is enabled, both migration and promotion can occur.
Quoted 7 lines
I was under the impression that:
-NUMA_BALANCING_NORMALwould handle migration within the same tier,
-NUMA_BALANCING_MEMORY_TIERINGwould handle promotion across tiers,
- and enabling both would allow both migration and promotion.This would provide more fine-grained control. Is my understanding correct, or am I missing something here?
You can change this, if you have some supporting data or bug report.
---
Best Regards,
Huang, Ying
Quoted 105 lines · Ying
Quoted 1 line · Donet Tom
Hi
Hi, Donet,
Quoted 77 lines · Ying
Quoted 68 lines
Quoted 66 lines · Donet Tom
In the current implementation, if
NUMA_BALANCING_MEMORY_TIERINGis disabled and the pages are on the lower tier, the pages may still be promoted.This happens because
task_numa_work()updates thelast_cpupidfield to record the last access time only whenNUMA_BALANCING_MEMORY_TIERINGis enabled and the folio is on the lower tier. IfNUMA_BALANCING_MEMORY_TIERINGis disabled, thelast_cpupidfield can retains a valid last CPU id.In
should_numa_migrate_memory(), the decision checks whetherNUMA_BALANCING_MEMORY_TIERINGis disabled, the folio is on the lower tier, andlast_cpupidis invalid. However, thelast_cpupidcan be valid whenNUMA_BALANCING_MEMORY_TIERINGis disabled, the condition evaluates to false and migration is allowed.This patch prevents promotion when
NUMA_BALANCING_MEMORY_TIERINGis disabled and the folio is on the lower tier.Behavior before this change:
============================
- IfNUMA_BALANCING_NORMALis enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes is allowed.Behavior after this change:
===========================
- IfNUMA_BALANCING_NORMALis enabled, migration will occur only between nodes within the same memory tier.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes will be allowed.- If both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALare enabled, both migration (same tier) and promotion (cross tier) are allowed.Fixes 33024536bafd ("memory tiering: hot page selection with hint page fault latency")Signed-off-by Donet Tom <[email protected]>---
v1 -> v2
========
1. Dropped changes intask_numa_fault()since the original changes already handle runtime disabling ofNUMA_BALANCING_MEMORY_TIERING.v1 -> https://lore.kernel.org/all/[email protected]/
--- kernel/sched/fair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)kernel/sched/fair.c +3
-1@@ -2024,8 +2024,12 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, this_cpupid =cpu_pid_to_cpupid (dst_cpu, current->pid); last_cpupid =folio_xchg_last_cpupid (folio, this_cpupid); +/* +* Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled +* and the pages are on the lower tier. +*/ if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && -!node_is_toptier(src_nid) && !cpupid_valid(last_cpupid)) Quoted 8 lines
kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 2035
Quoted 3 lines
kernel/sched/fair.c +1
-0+ !node_is_toptier(src_nid))return false;/* No. Even if
NUMA_BALANCING_MEMORY_TIERINGis disabled, we should still allow migrate pages from lower tier to higher tier viaNUMA_BALANCING_NORMAL. If we have precious DDR, why waste it? This follows the semantics ofNUMA_BALANCING_NORMALbefore introducingNUMA_BALANCING_MEMORY_TIERING.Quoted 8 lines
Thank you for the review comments.
One thing I am trying to understand is that page promotion appears to happen regardless of whether
NUMA_BALANCING_MEMORY_TIERINGis enabled or disabled. In that case, what is the specific role ofNUMA_BALANCING_MEMORY_TIERING? Do we get better performance when it is enabled?You can search
NUMA_BALANCING_MEMORY_TIERINGto find out what it does. We can get better performance as the original commit message says.When
NUMA_BALANCING_MEMORY_TIERINGis introduced, we didn't change the original behavior ofNUMA_BALANCING_MEMORY_NORMALbecause we had no good reason to do that. In fact, you change its behavior, so you should provide some supporting data or bug report to justify the change.Quoted 7 lines
My initial understanding was that disabling
NUMA_BALANCING_MEMORY_TIERINGcould be used to turn off promotion. However, it seems that currently we cannot control promotion independently. IfNUMA_BALANCING_NORMALis disabled, neither migration nor promotion happens, and if it is enabled, both migration and promotion can occur.
Quoted 8 lines
Quoted 7 lines
I was under the impression that:
-NUMA_BALANCING_NORMALwould handle migration within the same tier,
-NUMA_BALANCING_MEMORY_TIERINGwould handle promotion across tiers,
- and enabling both would allow both migration and promotion.This would provide more fine-grained control. Is my understanding correct, or am I missing something here?
You can change this, if you have some supporting data or bug report.
Thanks for the clarification. I was running some experiments where I only required migration, not promotion. However, I observed that promotion was still occurring even when NUMA_BALANCING_MEMORY_TIERING was disabled, which led me to believe it might be a bug, so I reported it.
As I understand it, enabling both NUMA_BALANCING_MEMORY_TIERING and NUMA_BALANCING_NORMAL results in both promotion and migration. Given this, do you see any concerns with modifying the behavior of NUMA_BALANCING_NORMAL?
With this patch, we would have better control over enabling and disabling promotion independently. I would appreciate your thoughts on this.
-Donet
Quoted 4 lines
---
Best Regards,
Huang, Ying
Quoted 122 lines · Donet Tom
Quoted 105 lines · Ying
Quoted 1 line · Donet Tom
Hi
Hi, Donet,
Quoted 77 lines · Ying
Quoted 68 lines
Quoted 66 lines · Donet Tom
In the current implementation, if
NUMA_BALANCING_MEMORY_TIERINGis disabled and the pages are on the lower tier, the pages may still be promoted.This happens because
task_numa_work()updates thelast_cpupidfield to record the last access time only whenNUMA_BALANCING_MEMORY_TIERINGis enabled and the folio is on the lower tier. IfNUMA_BALANCING_MEMORY_TIERINGis disabled, thelast_cpupidfield can retains a valid last CPU id.In
should_numa_migrate_memory(), the decision checks whetherNUMA_BALANCING_MEMORY_TIERINGis disabled, the folio is on the lower tier, andlast_cpupidis invalid. However, thelast_cpupidcan be valid whenNUMA_BALANCING_MEMORY_TIERINGis disabled, the condition evaluates to false and migration is allowed.This patch prevents promotion when
NUMA_BALANCING_MEMORY_TIERINGis disabled and the folio is on the lower tier.Behavior before this change:
============================
- IfNUMA_BALANCING_NORMALis enabled, migration occurs between nodes within the same memory tier, and promotion from lower tier to higher tier may also happen.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes is allowed.Behavior after this change:
===========================
- IfNUMA_BALANCING_NORMALis enabled, migration will occur only between nodes within the same memory tier.- If
NUMA_BALANCING_MEMORY_TIERINGis enabled, promotion from lower tier to higher tier nodes will be allowed.- If both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALare enabled, both migration (same tier) and promotion (cross tier) are allowed.Fixes 33024536bafd ("memory tiering: hot page selection with hint page fault latency")Signed-off-by Donet Tom <[email protected]>---
v1 -> v2
========
1. Dropped changes intask_numa_fault()since the original changes already handle runtime disabling ofNUMA_BALANCING_MEMORY_TIERING.v1 -> https://lore.kernel.org/all/[email protected]/
--- kernel/sched/fair.c | 6 +++++- 1 file changed, 5 insertions(+), 1 deletion(-)kernel/sched/fair.c +3
-1@@ -2024,8 +2024,12 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, this_cpupid =cpu_pid_to_cpupid (dst_cpu, current->pid); last_cpupid =folio_xchg_last_cpupid (folio, this_cpupid); +/* +* Do not allow promotion if NUMA_BALANCING_MEMORY_TIERING is disabled +* and the pages are on the lower tier. +*/ if (!(sysctl_numa_balancing_mode & NUMA_BALANCING_MEMORY_TIERING) && -!node_is_toptier(src_nid) && !cpupid_valid(last_cpupid)) Quoted 8 lines
kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 2035
Quoted 3 lines
kernel/sched/fair.c +1
-0+ !node_is_toptier(src_nid))return false;/* No. Even if
NUMA_BALANCING_MEMORY_TIERINGis disabled, we should still allow migrate pages from lower tier to higher tier viaNUMA_BALANCING_NORMAL. If we have precious DDR, why waste it? This follows the semantics ofNUMA_BALANCING_NORMALbefore introducingNUMA_BALANCING_MEMORY_TIERING.Quoted 8 lines
Thank you for the review comments.
One thing I am trying to understand is that page promotion appears to happen regardless of whether
NUMA_BALANCING_MEMORY_TIERINGis enabled or disabled. In that case, what is the specific role ofNUMA_BALANCING_MEMORY_TIERING? Do we get better performance when it is enabled?You can search
NUMA_BALANCING_MEMORY_TIERINGto find out what it does. We can get better performance as the original commit message says.When
NUMA_BALANCING_MEMORY_TIERINGis introduced, we didn't change the original behavior ofNUMA_BALANCING_MEMORY_NORMALbecause we had no good reason to do that. In fact, you change its behavior, so you should provide some supporting data or bug report to justify the change.Quoted 7 lines
My initial understanding was that disabling
NUMA_BALANCING_MEMORY_TIERINGcould be used to turn off promotion. However, it seems that currently we cannot control promotion independently. IfNUMA_BALANCING_NORMALis disabled, neither migration nor promotion happens, and if it is enabled, both migration and promotion can occur.Quoted 8 lines
Quoted 7 lines
I was under the impression that:
-NUMA_BALANCING_NORMALwould handle migration within the same tier,
-NUMA_BALANCING_MEMORY_TIERINGwould handle promotion across tiers,
- and enabling both would allow both migration and promotion.This would provide more fine-grained control. Is my understanding correct, or am I missing something here?
You can change this, if you have some supporting data or bug report.
Thanks for the clarification. I was running some experiments where I only required migration, not promotion. However, I observed that promotion was still occurring even when
NUMA_BALANCING_MEMORY_TIERINGwas disabled, which led me to believe it might be a bug, so I reported it.
Quoted 8 lines
As I understand it, enabling both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALresults in both promotion and migration. Given this, do you see any concerns with modifying the behavior ofNUMA_BALANCING_NORMAL?With this patch, we would have better control over enabling and disabling promotion independently. I would appreciate your thoughts on this.
IIUC, we change the existing user visible behavior only with strong enough practical reason. If so, making something conceptually better isn't enough for that.
---
Best Regards,
Huang, Ying
Quoted 14 lines · Ying
Quoted 14 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Donet Tom <[email protected]> writes:
Thanks for the clarification. I was running some experiments where I only required migration, not promotion. However, I observed that promotion was still occurring even when
NUMA_BALANCING_MEMORY_TIERINGwas disabled, which led me to believe it might be a bug, so I reported it.As I understand it, enabling both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALresults in both promotion and migration. Given this, do you see any concerns with modifying the behavior ofNUMA_BALANCING_NORMAL?
Quoted 6 lines
Quoted 3 lines
With this patch, we would have better control over enabling and disabling promotion independently. I would appreciate your thoughts on this.
IIUC, we change the existing user visible behavior only with strong enough practical reason.
So what I understood from this discussion so far is, we don't have any mechanism to do auto-numa base page migration between DRAM -to- DRAM w/o triggering promotions too from a lower tiers to higher tiers.
... This to me sounds more like a broken interface.
Quoted 2 lines
If so, making something conceptually better isn't enough for that.
I think Donet's approach was more towards fixing the problem, then making it conceptually better. So, as of now most of us may not see this as a problem, since not many systems have different memory tiers attached. But with more widespread CXL adoption and more memory tiers in the system, we might require more finer control over auto-numa based page migration.
But hey, I just wanted to voice out my opinion here. If we think changing user visible behavior is going to break existing applications and we don't want that - then in that case the reasoning sounds ok to me.
Quoted 3 lines
---
Best Regards,
Huang, Ying
Thanks for your feedback!
-ritesh
Quoted 27 lines · Ritesh Harjani (IBM)
Quoted 14 lines · Ying
Quoted 14 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Donet Tom <[email protected]> writes:
Thanks for the clarification. I was running some experiments where I only required migration, not promotion. However, I observed that promotion was still occurring even when
NUMA_BALANCING_MEMORY_TIERINGwas disabled, which led me to believe it might be a bug, so I reported it.As I understand it, enabling both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALresults in both promotion and migration. Given this, do you see any concerns with modifying the behavior ofNUMA_BALANCING_NORMAL?Quoted 6 lines
Quoted 3 lines
With this patch, we would have better control over enabling and disabling promotion independently. I would appreciate your thoughts on this.
IIUC, we change the existing user visible behavior only with strong enough practical reason.
So what I understood from this discussion so far is, we don't have any mechanism to do auto-numa base page migration between DRAM -to- DRAM w/o triggering promotions too from a lower tiers to higher tiers.
Quoted 7 lines
... This to me sounds more like a broken interface.
Quoted 2 lines
If so, making something conceptually better isn't enough for that.
I think Donet's approach was more towards fixing the problem, then making it conceptually better.
To fix a theoretical problem instead of a practical problem?
Quoted 5 lines
So, as of now most of us may not see this
as a problem, since not many systems have different memory tiers attached. But with more widespread CXL adoption and more memory tiers in the system, we might require more finer control over auto-numa based page migration.
By design, normal NUMA balancing (not memory tiering) should migrate pages between tiers too. Because it migrates pages to the node near a CPU regardless of the memory tiers to optimize NUMA locality.
Quoted 4 lines
But hey, I just wanted to voice out my opinion here. If we think changing user visible behavior is going to break existing applications and we don't want that - then in that case the reasoning sounds ok to me.
---
Best Regards,
Huang, Ying
Quoted 20 lines · Ritesh Harjani
Quoted 18 lines · Ying
Quoted 17 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Donet Tom <[email protected]> writes:
Thanks for the clarification. I was running some experiments where I only required migration, not promotion. However, I observed that promotion was still occurring even when
NUMA_BALANCING_MEMORY_TIERINGwas disabled, which led me to believe it might be a bug, so I reported it.As I understand it, enabling both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALresults in both promotion and migration. Given this, do you see any concerns with modifying the behavior ofNUMA_BALANCING_NORMAL?With this patch, we would have better control over enabling and disabling promotion independently. I would appreciate your thoughts on this.
Quoted 9 lines
Quoted 2 lines
IIUC, we change the existing user visible behavior only with strong enough practical reason.
So what I understood from this discussion so far is, we don't have any mechanism to do auto-numa base page migration between DRAM -to- DRAM w/o triggering promotions too from a lower tiers to higher tiers.
... This to me sounds more like a broken interface.
It only seems that way because the naming suggests tiering did not exist prior to _TIERING - instead _NORMAL just operates in a suboptimal manner when multiple tiers exist.
_NORMAL migrates a page when it detects the node it's on is not the local node of the task doing the work.
_TIERING takes into account the liveliness of the pages with a timestamp.
Going to agree with Ying here - this change should be dropped without additional data. If you can show _NORMAL would be better off not moving low-tier pages for at least a handful of common benchmarks, or that _NORMAL is causes incorrect placement - then I think this change is warranted.
But as it is, this would just be a behavioral change without supporting data to justify it.
~Gregory
Quoted 35 lines · Gregory Price
Quoted 20 lines · Ritesh Harjani
Quoted 18 lines · Ying
Quoted 17 lines
Quoted 1 line
Quoted 1 line
Quoted 1 line
Donet Tom <[email protected]> writes:
Thanks for the clarification. I was running some experiments where I only required migration, not promotion. However, I observed that promotion was still occurring even when
NUMA_BALANCING_MEMORY_TIERINGwas disabled, which led me to believe it might be a bug, so I reported it.As I understand it, enabling both
NUMA_BALANCING_MEMORY_TIERINGandNUMA_BALANCING_NORMALresults in both promotion and migration. Given this, do you see any concerns with modifying the behavior ofNUMA_BALANCING_NORMAL?With this patch, we would have better control over enabling and disabling promotion independently. I would appreciate your thoughts on this.
Quoted 9 lines
Quoted 2 lines
IIUC, we change the existing user visible behavior only with strong enough practical reason.
So what I understood from this discussion so far is, we don't have any mechanism to do auto-numa base page migration between DRAM -to- DRAM w/o triggering promotions too from a lower tiers to higher tiers.
... This to me sounds more like a broken interface.
It only seems that way because the naming suggests tiering did not exist prior to
_TIERING- instead_NORMALjust operates in a suboptimal manner when multiple tiers exist.
Quoted 6 lines
_NORMALmigrates a page when it detects the node it's on is not the local node of the task doing the work.
_TIERINGtakes into account the liveliness of the pages with a timestamp.
Sure Gregory and Ying, got the point.
Thanks for your comments and feedback. Appreciate it!
For now let's live with that, in case if we come across a real world usecase where we would want, only-migration w/o tiering behavior, then will share it.
-ritesh