sched/numa, mm: do not promote folios to nodes not set N_MEMORY
Subsystems (2)
PATCH v3 1/1 sched/numa, mm: do not promote folios to nodes not set N_MEMORY
Changes from v2:
1. Rewrite the comment in code and the commit message becasue it turns out that this patch is not the real fix for the oops descriped. The real fix goes in another patch below:
https://lore.kernel.org/lkml/[email protected]/
Changes from v1:
1. Trim the verbose oops in the commit message. (feedbacked by Phil Auld)
2. Rewrite a comment in code. (feedbacked by Phil Auld)
--->8---
From 150af2f78e19217a1d03e47e3ee5279684590fb4 Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 20:18:10 +0900
Subject: [PATCH v3] sched/numa, mm: do not promote folios to nodes not set N_MEMORY
A numa node might not have its local memory but CPUs. Promoting a folio to the node's local memory is nonsense. So avoid nodes not set N_MEMORY from getting promoted.
--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)
kernel/sched/fair.c +7 -0
@@ -1828,6 +1828,13 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,int dst_nid =cpu_to_node (dst_cpu);int last_cpupid, this_cpupid; +/* +* A node of dst_nid might not have its local memory. Promoting +* a folio to the node is meaningless. +*/ +if (!node_state (dst_nid,N_MEMORY )) +return false; +/* * The pages in slow memory node should be migrated according * to hot/cold instead of private/shared.
Signature · 1 line
2.17.1
Quoted 4 lines · Byungchul Park
From
150af2f78e19217a1d03e47e3ee5279684590fb4Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 20:18:10 +0900
Subject: [PATCH v3] sched/numa, mm: do not promote folios to nodes not setN_MEMORY
"do not try to promote folios to memoryless nodes"
because AFAICS we are just trying.
Even if should_numa_migrate_memory() returns true, I assume that we will fail somewhere down the chain e.g: migrate_pages() when we see that this node does not any memory, right?
Quoted 3 lines
A numa node might not have its local memory but CPUs. Promoting a folio to the node's local memory is nonsense. So avoid nodes not set
N_MEMORYfrom getting promoted.
If you talk about memoryless nodes everybody gets it better IMHO. "Memoryless nodes do not have any memory to migrate to, so stop trying it."
Quoted 10 lines
Signed-off-by Byungchul Park <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0
kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 1836
Quoted 9 lines
kernel/sched/fair.c +6
-0@@ -1828,6 +1828,13 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,int dst_nid =cpu_to_node (dst_cpu);int last_cpupid, this_cpupid; +/* +* A node of dst_nid might not have its local memory. Promoting +* a folio to the node is meaningless. +*/ +if (!node_state (dst_nid,N_MEMORY )) +return false;
"Cannot migrate to memoryless nodes"
seems shorter and more clear.
So, what happens when we return true here? will we fail at migrate_pages() I guess? That is quite down the road so I guess this check can save us some time.
Signature · 2 lines
Oscar Salvador SUSE Labs
Oscar Salvador
Reviewed-byQuoted 7 lines · Oscar Salvador
Quoted 4 lines · Byungchul Park
From
150af2f78e19217a1d03e47e3ee5279684590fb4Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 20:18:10 +0900
Subject: [PATCH v3] sched/numa, mm: do not promote folios to nodes not setN_MEMORY"do not try to promote folios to memoryless nodes"
Thinking some more, promote might be misleading, just something like "do not try to migrate memory to memoryless nodes".
As this is not a bug fix but an optimization, as we will fail anyways in migrate_misplaced_folio() when migrate_balanced_pgdat() notices that we do not have any memory on that code.
With the other comments addressed:
Quoted 47 lines
because AFAICS we are just trying.
Even ifshould_numa_migrate_memory()returns true, I assume that we will fail somewhere down the chain e.g:migrate_pages()when we see that this node does not any memory, right?Quoted 3 lines
A numa node might not have its local memory but CPUs. Promoting a folio to the node's local memory is nonsense. So avoid nodes not set
N_MEMORYfrom getting promoted.If you talk about memoryless nodes everybody gets it better IMHO. "Memoryless nodes do not have any memory to migrate to, so stop trying it."
Quoted 10 lines
Signed-off-by Byungchul Park <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 1836
Quoted 9 lines
kernel/sched/fair.c +6
-0@@ -1828,6 +1828,13 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,int dst_nid =cpu_to_node (dst_cpu);int last_cpupid, this_cpupid; +/* +* A node of dst_nid might not have its local memory. Promoting +* a folio to the node is meaningless. +*/ +if (!node_state (dst_nid,N_MEMORY )) +return false;"Cannot migrate to memoryless nodes"
seems shorter and more clear.
So, what happens when we return true here? will we fail at
migrate_pages()I guess? That is quite down the road so I guess this check can save us some time.Signature · 2 lines
Oscar Salvador SUSE Labs
Signature · 2 lines
Oscar Salvador SUSE Labs
Quoted 11 lines · Oscar Salvador
Quoted 7 lines · Oscar Salvador
Quoted 4 lines · Byungchul Park
From
150af2f78e19217a1d03e47e3ee5279684590fb4Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 20:18:10 +0900
Subject: [PATCH v3] sched/numa, mm: do not promote folios to nodes not setN_MEMORY"do not try to promote folios to memoryless nodes"
Thinking some more, promote might be misleading, just something like "do not try to migrate memory to memoryless nodes".
Thank you. I will.
Quoted 7 lines
As this is not a bug fix but an optimization, as we will fail anyways in
migrate_misplaced_folio()whenmigrate_balanced_pgdat()notices that we do not have any memory on that code.With the other comments addressed:
Reviewed-by Oscar Salvador <[email protected]>
Thank you for reviewing.
Byungchul
Quoted 50 lines
Quoted 46 lines
because AFAICS we are just trying.
Even ifshould_numa_migrate_memory()returns true, I assume that we will fail somewhere down the chain e.g:migrate_pages()when we see that this node does not any memory, right?Quoted 3 lines
A numa node might not have its local memory but CPUs. Promoting a folio to the node's local memory is nonsense. So avoid nodes not set
N_MEMORYfrom getting promoted.If you talk about memoryless nodes everybody gets it better IMHO. "Memoryless nodes do not have any memory to migrate to, so stop trying it."
Quoted 10 lines
Signed-off-by Byungchul Park <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 1836
Quoted 9 lines
kernel/sched/fair.c +6
-0@@ -1828,6 +1828,13 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,int dst_nid =cpu_to_node (dst_cpu);int last_cpupid, this_cpupid; +/* +* A node of dst_nid might not have its local memory. Promoting +* a folio to the node is meaningless. +*/ +if (!node_state (dst_nid,N_MEMORY )) +return false;"Cannot migrate to memoryless nodes"
seems shorter and more clear.
So, what happens when we return true here? will we fail at
migrate_pages()I guess? That is quite down the road so I guess this check can save us some time.Signature · 2 lines
Oscar Salvador SUSE LabsSignature · 2 lines
Oscar Salvador SUSE Labs
Davidlohr Bueso
Reviewed-byQuoted 11 lines · Oscar Salvador
Quoted 7 lines · Oscar Salvador
Quoted 4 lines · Byungchul Park
From
150af2f78e19217a1d03e47e3ee5279684590fb4Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 20:18:10 +0900
Subject: [PATCH v3] sched/numa, mm: do not promote folios to nodes not setN_MEMORY"do not try to promote folios to memoryless nodes"
Thinking some more, promote might be misleading, just something like "do not try to migrate memory to memoryless nodes".
Yes. Does this also want an unlikely()? Not that it would be measurable.
Quoted 3 lines
As this is not a bug fix but an optimization, as we will fail anyways in
migrate_misplaced_folio()whenmigrate_balanced_pgdat()notices that we do not have any memory on that code.
This should be in the changelog and the subject is misleading as well.
Quoted 57 lines
With the other comments addressed:
Reviewed-by Oscar Salvador <[email protected]>Quoted 47 lines
because AFAICS we are just trying.
Even ifshould_numa_migrate_memory()returns true, I assume that we will fail somewhere down the chain e.g:migrate_pages()when we see that this node does not any memory, right?Quoted 3 lines
A numa node might not have its local memory but CPUs. Promoting a folio to the node's local memory is nonsense. So avoid nodes not set
N_MEMORYfrom getting promoted.If you talk about memoryless nodes everybody gets it better IMHO. "Memoryless nodes do not have any memory to migrate to, so stop trying it."
Quoted 10 lines
Signed-off-by Byungchul Park <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 1836
Quoted 9 lines
kernel/sched/fair.c +6
-0@@ -1828,6 +1828,13 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,int dst_nid =cpu_to_node (dst_cpu);int last_cpupid, this_cpupid; +/* +* A node of dst_nid might not have its local memory. Promoting +* a folio to the node is meaningless. +*/ +if (!node_state (dst_nid,N_MEMORY )) +return false;"Cannot migrate to memoryless nodes"
seems shorter and more clear.
So, what happens when we return true here? will we fail at
migrate_pages()I guess? That is quite down the road so I guess this check can save us some time.--
Oscar Salvador
SUSE LabsSignature · 2 lines
Oscar Salvador SUSE Labs
Quoted 7 lines · Oscar Salvador
Quoted 4 lines · Byungchul Park
From
150af2f78e19217a1d03e47e3ee5279684590fb4Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 20:18:10 +0900
Subject: [PATCH v3] sched/numa, mm: do not promote folios to nodes not setN_MEMORY"do not try to promote folios to memoryless nodes"
Thank you. I will.
Quoted 4 lines
because AFAICS we are just trying.
Even ifshould_numa_migrate_memory()returns true, I assume that we will fail somewhere down the chain e.g:migrate_pages()when we see that this node does not any memory, right?
Yes.
Quoted 6 lines
Quoted 3 lines
A numa node might not have its local memory but CPUs. Promoting a folio to the node's local memory is nonsense. So avoid nodes not set
N_MEMORYfrom getting promoted.If you talk about memoryless nodes everybody gets it better IMHO. "Memoryless nodes do not have any memory to migrate to, so stop trying it."
Much better.
Quoted 22 lines
Quoted 10 lines
Signed-off-by Byungchul Park <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 1836
Quoted 9 lines
kernel/sched/fair.c +6
-0@@ -1828,6 +1828,13 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,int dst_nid =cpu_to_node (dst_cpu);int last_cpupid, this_cpupid; +/* +* A node of dst_nid might not have its local memory. Promoting +* a folio to the node is meaningless. +*/ +if (!node_state (dst_nid,N_MEMORY )) +return false;"Cannot migrate to memoryless nodes"
Quoted 1 line
seems shorter and more clear.
Agree.
Byungchul
Quoted 8 lines
So, what happens when we return true here? will we fail at
migrate_pages()I guess? That is quite down the road so I guess this check can save us some time.Signature · 2 lines
Oscar Salvador SUSE Labs
Phil Auld
Reviewed-byQuoted 49 lines · On Mon, Feb 19, 2024 at 11:08:54AM +0900 Byungchul Park wrote:
Quoted 7 lines · Oscar Salvador
Quoted 4 lines · Byungchul Park
From
150af2f78e19217a1d03e47e3ee5279684590fb4Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 20:18:10 +0900
Subject: [PATCH v3] sched/numa, mm: do not promote folios to nodes not setN_MEMORY"do not try to promote folios to memoryless nodes"
Thank you. I will.
Quoted 4 lines
because AFAICS we are just trying.
Even ifshould_numa_migrate_memory()returns true, I assume that we will fail somewhere down the chain e.g:migrate_pages()when we see that this node does not any memory, right?Yes.
Quoted 6 lines
Quoted 3 lines
A numa node might not have its local memory but CPUs. Promoting a folio to the node's local memory is nonsense. So avoid nodes not set
N_MEMORYfrom getting promoted.If you talk about memoryless nodes everybody gets it better IMHO. "Memoryless nodes do not have any memory to migrate to, so stop trying it."
Much better.
Quoted 22 lines
Quoted 10 lines
Signed-off-by Byungchul Park <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 1836
Quoted 9 lines
kernel/sched/fair.c +6
-0@@ -1828,6 +1828,13 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,int dst_nid =cpu_to_node (dst_cpu);int last_cpupid, this_cpupid; +/* +* A node of dst_nid might not have its local memory. Promoting +* a folio to the node is meaningless. +*/ +if (!node_state (dst_nid,N_MEMORY )) +return false;"Cannot migrate to memoryless nodes"
Quoted 4 lines
Quoted 1 line
seems shorter and more clear.
Agree.
We clearly have dst_cpu when we call this so I still think a check could be done farther up. But this one still looks reasonable to me.
Thanks,
Phil
Quoted 11 lines
Byungchul
Quoted 8 lines
So, what happens when we return true here? will we fail at
migrate_pages()I guess? That is quite down the road so I guess this check can save us some time.Signature · 2 lines
Oscar Salvador SUSE Labs
Quoted 18 lines · Byungchul Park
Changes from v2:
1. Rewrite the comment in code and the commit message becasue it turns out that this patch is not the real fix for the oops descriped. The real fix goes in another patch below:https://lore.kernel.org/lkml/[email protected]/
Changes from v1:
1. Trim the verbose oops in the commit message. (feedbacked by Phil Auld)
2. Rewrite a comment in code. (feedbacked by Phil Auld)--->8---
From150af2f78e19217a1d03e47e3ee5279684590fb4Mon Sep 17 00:00:00 2001
From: Byungchul Park <[email protected]>
Date: Fri, 16 Feb 2024 20:18:10 +0900
Subject: [PATCH v3] sched/numa, mm: do not promote folios to nodes not setN_MEMORY
Quoted 3 lines
A numa node might not have its local memory but CPUs. Promoting a folio to the node's local memory is nonsense. So avoid nodes not set
N_MEMORYfrom getting promoted.
So there is no bug/panic that can be triggered and this is not a "fix" but an optimization?
Quoted 14 lines
Signed-off-by Byungchul Park <[email protected]>--- kernel/sched/fair.c | 7 +++++++ 1 file changed, 7 insertions(+)kernel/sched/fair.c +0
-0@@ -1828,6 +1828,13 @@ bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio,int dst_nid =cpu_to_node (dst_cpu);int last_cpupid, this_cpupid;
kernel/sched/fair.c bool should_numa_migrate_memory(struct task_struct *p, struct folio *folio, · line 1840
Quoted 10 lines
kernel/sched/fair.c +7
-0+/* +* A node of dst_nid might not have its local memory. Promoting +* a folio to the node is meaningless. +*/ +if (!node_state (dst_nid,N_MEMORY )) +return false; +/* * The pages in slow memory node should be migrated according * to hot/cold instead of private/shared.
Signature · 3 lines
Cheers, David / dhildenb