mem-hotplug: shall we skip unmovable node when doing numa balance?
Subsystems (2)
RFC 1/1 mem-hotplug: shall we skip unmovable node when doing numa balance?
On mem-hotplug system, there is a problem, please see the following case.
memtester xxG, the memory will be alloced on a movable node. And after numa balancing, the memory may be migrated to the other node, it may be a unmovable node. This will reduce the free memory of the unmovable node, and may be oom later.
My question is that shall we skip unmovable node when doing numa balance? or just let the manager set some numa policies?
mm/mempolicy.c +7 -0
@@ -2334,6 +2334,13 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long out:mpol_cond_put (pol); +/* Skip unmovable nodes when do numa balancing */ +if (movable_node_enabled && ret !=-1 ) { + zone =NODE_DATA (ret)->node_zones +MAX_NR_ZONES -1 ; +if (!populated_zone (zone)) + ret =-1 ; + } +return ret; }
Thanks,
Xishi Qiu
Mel Gorman
NAKQuoted 7 lines · Xishi Qiu
On mem-hotplug system, there is a problem, please see the following case.
memtester xxG, the memory will be alloced on a movable node. And after numa balancing, the memory may be migrated to the other node, it may be a unmovable node. This will reduce the free memory of the unmovable node, and may be oom later.
How would it OOM later? It's movable memmory that is moving via automatic NUMA balancing so at the very least it can be reclaimed. If the memory is mlocked or unable to migrate then it's irrelevant if automatic balancing put it there.
Quoted 3 lines
My question is that shall we skip unmovable node when doing numa balance? or just let the manager set some numa policies?
If the unmovable node must be protected from automatic NUMA balancing then policies are the appropriate step to prevent the processes running on that node or from allocating memory on that node.
Either way, protecting unmovable nodes in the name of hotplug is pretty much guaranteed to be a performance black hole because at the very least, page table pages will always be remote accesses for processes running on the unmovable node.
Quoted 7 lines
mm/mempolicy.c +0
-0@@ -2334,6 +2334,13 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long out:mpol_cond_put (pol);
mm/mempolicy.c int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long · line 2345
Quoted 10 lines
mm/mempolicy.c +7
-0+/* Skip unmovable nodes when do numa balancing */ +if (movable_node_enabled && ret !=-1 ) { + zone =NODE_DATA (ret)->node_zones +MAX_NR_ZONES -1 ; +if (!populated_zone (zone)) + ret =-1 ; + } +return ret; }
Nak.
Signature · 8 lines
Mel Gorman SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>
Quoted 9 lines · Mel Gorman
Quoted 7 lines · Xishi Qiu
On mem-hotplug system, there is a problem, please see the following case.
memtester xxG, the memory will be alloced on a movable node. And after numa balancing, the memory may be migrated to the other node, it may be a unmovable node. This will reduce the free memory of the unmovable node, and may be oom later.
Quoted 5 lines
How would it OOM later? It's movable memmory that is moving via automatic NUMA balancing so at the very least it can be reclaimed. If the memory is mlocked or unable to migrate then it's irrelevant if automatic balancing put it there.
Hi Mel,
memtester will mlock the memory, so we can not reclaim, then maybe oom, right? So let the manager set some numa policies to prevent the above case, right?
Thanks,
Xishi Qiu
Quoted 33 lines
Quoted 3 lines
My question is that shall we skip unmovable node when doing numa balance? or just let the manager set some numa policies?
If the unmovable node must be protected from automatic NUMA balancing then policies are the appropriate step to prevent the processes running on that node or from allocating memory on that node.
Either way, protecting unmovable nodes in the name of hotplug is pretty much guaranteed to be a performance black hole because at the very least, page table pages will always be remote accesses for processes running on the unmovable node.
Quoted 7 lines
mm/mempolicy.c +0
-0@@ -2334,6 +2334,13 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long out:mpol_cond_put (pol);mm/mempolicy.c int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long · line 2345
Quoted 10 lines
mm/mempolicy.c +7
-0+/* Skip unmovable nodes when do numa balancing */ +if (movable_node_enabled && ret !=-1 ) { + zone =NODE_DATA (ret)->node_zones +MAX_NR_ZONES -1 ; +if (!populated_zone (zone)) + ret =-1 ; + } +return ret; }Nak.
Quoted 11 lines · Xishi Qiu
Quoted 9 lines · Mel Gorman
Quoted 7 lines · Xishi Qiu
On mem-hotplug system, there is a problem, please see the following case.
memtester xxG, the memory will be alloced on a movable node. And after numa balancing, the memory may be migrated to the other node, it may be a unmovable node. This will reduce the free memory of the unmovable node, and may be oom later.
Quoted 9 lines
Quoted 5 lines
How would it OOM later? It's movable memmory that is moving via automatic NUMA balancing so at the very least it can be reclaimed. If the memory is mlocked or unable to migrate then it's irrelevant if automatic balancing put it there.
memtester will mlock the memory, so we can not reclaim, then maybe oom, right? So let the manager set some numa policies to prevent the above case, right?
Deal with it using policies.
Signature · 8 lines
Mel Gorman SUSE Labs -- To unsubscribe, send a message with 'unsubscribe linux-mm' in the body to [email protected]. For more info on Linux MM, see: http://www.linux-mm.org/ . Don't email: <a href=mailto:"[email protected]"> [email protected] </a>