Introduce multi-preference mempolicy
Subsystems (6)
v6 cover Introduce multi-preference mempolicy
This patch series introduces the concept of the MPOL_PREFERRED_MANY mempolicy. This mempolicy mode can be used with either the set_mempolicy(2) or mbind(2) interfaces. Like the MPOL_PREFERRED interface, it allows an application to set a preference for nodes which will fulfil memory allocation requests. Unlike the MPOL_PREFERRED mode, it takes a set of nodes. Like the MPOL_BIND interface, it works over a set of nodes. Unlike MPOL_BIND, it will not cause a SIGSEGV or invoke the OOM killer if those preferred nodes are not available.
Along with these patches are patches for libnuma, numactl, numademo, and memhog. They still need some polish, but can be found here:
https://gitlab.com/bwidawsk/numactl/-/tree/prefer-many
It allows new usage: numactl -P 0,3,4
The goal of the new mode is to enable some use-cases when using tiered memory usage models which I've lovingly named.
1a. The Hare - The interconnect is fast enough to meet bandwidth and latency requirements allowing preference to be given to all nodes with "fast" memory. 1b. The Indiscriminate Hare - An application knows it wants fast memory (or perhaps slow memory), but doesn't care which node it runs on. The application can prefer a set of nodes and then xpu bind to the local node (cpu, accelerator, etc). This reverses the nodes are chosen today where the kernel attempts to use local memory to the CPU whenever possible. This will attempt to use the local accelerator to the memory.
2. The Tortoise - The administrator (or the application itself) is aware it only needs slow memory, and so can prefer that.
Much of this is almost achievable with the bind interface, but the bind interface suffers from an inability to fallback to another set of nodes if binding fails to all nodes in the nodemask.
Like MPOL_BIND a nodemask is given. Inherently this removes ordering from the preference.
Quoted 7 lines
/* Set first two nodes as preferred in an 8 node system. */ const unsigned long nodes =
0x3set_mempolicy(MPOL_PREFER_MANY, &nodes, 8);/* Mimic interleave policy, but have fallback *.
const unsigned long nodes =0xaaset_mempolicy(MPOL_PREFER_MANY, &nodes, 8);
Some internal discussion took place around the interface. There are two alternatives which we have discussed, plus one I stuck in:
1. Ordered list of nodes. Currently it's believed that the added complexity is nod needed for expected usecases.
2. A flag for bind to allow falling back to other nodes. This confuses the notion of binding and is less flexible than the current solution.
3. Create flags or new modes that helps with some ordering. This offers both a friendlier API as well as a solution for more customized usage. It's unknown if it's worth the complexity to support this. Here is sample code for how this might work:
Quoted 8 lines
// Prefer specific nodes for some something wacky
set_mempolicy(MPOL_PREFER_MANY, 0x17c, 1024);// Default
set_mempolicy(MPOL_PREFER_MANY | MPOL_F_PREFER_ORDER_SOCKET, NULL, 0); // which is the same asset_mempolicy(MPOL_DEFAULT, NULL, 0);
Quoted 9 lines
// The Hare
set_mempolicy(MPOL_PREFER_MANY | MPOL_F_PREFER_ORDER_TYPE, NULL, 0);// The Tortoise
set_mempolicy(MPOL_PREFER_MANY | MPOL_F_PREFER_ORDER_TYPE_REV, NULL, 0);// Prefer the fast memory of the first two sockets
set_mempolicy(MPOL_PREFER_MANY | MPOL_F_PREFER_ORDER_TYPE, -1, 2);
In v1, Andi Kleen brought up reusing MPOL_PREFERRED as the mode for the API. There wasn't consensus around this, so I've left the existing API as it was. I'm open to more feedback here, but my slight preference is to use a new API as it ensures if people are using it, they are entirely aware of what they're doing and not accidentally misusing the old interface. (In a similar way to how MPOL_LOCAL was introduced).
In v1, Michal also brought up renaming this MPOL_PREFERRED_MASK. I'm equally fine with that change, but I hadn't heard much emphatic support for one way or another, so I've left that too.
- Ben/Dave/Feng
---
Changelog:
Since v5:
* Rebased against 5.14-rc1.
Since v4:
* Rebased on latest -mm tree (v5.13-rc), whose mempolicy code has been refactored much since v4 submission
* add a dedicated alloc_page_preferred_many() (Michal Hocko)
* refactor and add fix to hugetlb supporting code (Michal Hocko)
Since v3:
* Rebased against v5.12-rc2
* Drop the v3/0013 patch of creating NO_SLOWPATH gfp_mask bit
* Skip direct reclaim for the first allocation try for MPOL_PREFERRED_MANY, which makes its semantics close to existing MPOL_PREFFERRED policy
Since v2:
* Rebased against v5.11
* Fix a stack overflow related panic, and a kernel warning (Feng)
* Some code clearup (Feng)
* One RFC patch to speedup mem alloc in some case (Feng)
Since v1:
* Dropped patch to replace numa_node_id in some places (mhocko)
* Dropped all the page allocation patches in favor of new mechanism to use fallbacks. (mhocko)
* Dropped the special snowflake preferred node algorithm (bwidawsk)
* If the preferred node fails, ALL nodes are rechecked instead of just the non-preferred nodes.
---
Ben Widawsky (3):
mm/mempolicy: enable page allocation for MPOL_PREFERRED_MANY for
general cases
mm/hugetlb: add support for mempolicy MPOL_PREFERRED_MANY mm/mempolicy: Advertise new MPOL_PREFERRED_MANY
Dave Hansen (1):
mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Feng Tang (2):
mm/memplicy: add page allocation function for MPOL_PREFERRED_MANY
policy
mm/mempolicy: unify the create() func for bind/interleave/prefer-many
policies
.../admin-guide/mm/numa_memory_policy.rst | 16 +++-- include/uapi/linux/mempolicy.h | 1 + mm/hugetlb.c | 25 ++++++++ mm/mempolicy.c | 75 +++++++++++++++++----- 4 files changed, 96 insertions(+), 21 deletions(-)
Signature · 1 line
2.7.4
PATCH v6 1/6 mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
From: Dave Hansen <[email protected]>
The NUMA APIs currently allow passing in a "preferred node" as a single bit set in a nodemask. If more than one bit it set, bits after the first are ignored.
This single node is generally OK for location-based NUMA where memory being allocated will eventually be operated on by a single CPU. However, in systems with multiple memory types, folks want to target a *type* of memory instead of a location. For instance, someone might want some high-bandwidth memory but do not care about the CPU next to which it is allocated. Or, they want a cheap, high capacity allocation and want to target all NUMA nodes which have persistent memory in volatile mode. In both of these cases, the application wants to target a *set* of nodes, but does not want strict MPOL_BIND behavior as that could lead to OOM killer or SIGSEGV.
So add MPOL_PREFERRED_MANY policy to support the multiple preferred nodes requirement. This is not a pie-in-the-sky dream for an API. This was a response to a specific ask of more than one group at Intel. Specifically:
1. There are existing libraries that target memory types such as
https://github.com/memkind/memkind. These are known to suffer from SIGSEGV's when memory is low on targeted memory "kinds" that span more than one node. The MCDRAM on a Xeon Phi in "Cluster on Die" mode is an example of this.
2. Volatile-use persistent memory users want to have a memory policy which is targeted at either "cheap and slow" (PMEM) or "expensive and fast" (DRAM). However, they do not want to experience allocation failures when the targeted type is unavailable.
3. Allocate-then-run. Generally, we let the process scheduler decide on which physical CPU to run a task. That location provides a default allocation policy, and memory availability is not generally considered when placing tasks. For situations where memory is valuable and constrained, some users want to allocate memory first, *then* allocate close compute resources to the allocation. This is the reverse of the normal (CPU) model. Accelerators such as GPUs that operate on core-mm-managed memory are interested in this model.
A check is added in sanitize_mpol_flags() to not permit 'prefer_many' policy to be used for now, and will be removed in later patch after all implementations for 'prefer_many' are ready, as suggested by Michal Hocko.
--- include/uapi/linux/mempolicy.h | 1 + mm/mempolicy.c | 44 +++++++++++++++++++++++++++++++++++++----- 2 files changed, 40 insertions(+), 5 deletions(-)
include/uapi/linux/mempolicy.h +1 -0
@@ -22,6 +22,7 @@ enum {MPOL_BIND ,MPOL_INTERLEAVE ,MPOL_LOCAL , +MPOL_PREFERRED_MANY , MPOL_MAX, /* always last member of enum */ };
mm/mempolicy.c +39 -5
129 lines not on this page · raw message
Signature · 1 line
2.7.4
Michal Hocko
reviewerRe: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
[Sorry for a late review]
On Mon 12-07-21 16:09:29, Feng Tang wrote:
[...]
mm/mempolicy.c nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy) · line 1916
Quoted 10 lines
mm/mempolicy.c +2
-1@@ -1887,7 +1909,8 @@ nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy)/* Return the node id preferred by the given mempolicy, or the given id */ static int policy_node (gfp_t gfp,struct mempolicy *policy,int nd) { -if (policy->mode ==MPOL_PREFERRED ) { +if (policy->mode ==MPOL_PREFERRED || + policy->mode ==MPOL_PREFERRED_MANY ) { nd =first_node (policy->nodes); }else {/*
Do we really want to have the preferred node to be always the first node in the node mask? Shouldn't that strive for a locality as well? Existing callers already prefer numa_node_id() - aka local node - and I belive we shouldn't just throw that away here.
mm/mempolicy.c unsigned int mempolicy_slab_node(void) · line 1958
Quoted 6 lines
mm/mempolicy.c +1
-0@@ -1931,6 +1954,7 @@ unsigned int mempolicy_slab_node(void)switch (policy->mode) {case MPOL_PREFERRED : +case MPOL_PREFERRED_MANY :return first_node (policy->nodes);
Similarly here but I am not really familiar with the slab numa code enough to have strong opinions here.
Quoted 4 lines
mm/mempolicy.c +0
-0@@ -2173,10 +2198,12 @@ struct page *alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,* node and don't fall back to other nodes, as the cost of * remote accesses would likely offset THP benefits. *
mm/mempolicy.c struct page *alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma, · line 2207
Quoted 10 lines
mm/mempolicy.c +5
-3-* If the policy is interleave, or does not allow the current -* node in its nodemask, we allocate the standard way. +* If the policy is interleave or multiple preferred nodes, or +* does not allow the current node in its nodemask, we allocate +* the standard way. */ -if (pol->mode ==MPOL_PREFERRED ) +if ((pol->mode ==MPOL_PREFERRED || + pol->mode ==MPOL_PREFERRED_MANY )) hpage_node =first_node (pol->nodes);
Same here.
mm/mempolicy.c int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long · line 2486
Quoted 9 lines
mm/mempolicy.c +3
-0@@ -2451,6 +2479,9 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned longbreak ;case MPOL_PREFERRED : +case MPOL_PREFERRED_MANY : +if (node_isset (curnid, pol->nodes)) + goto out; polnid =first_node (pol->nodes);break ;
I do not follow what is the point of using first_node here. Either the node is in the mask or it is misplaced. What are you trying to achieve here?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 1 line · Michal Hocko
[Sorry for a late review]
Not at all. Thank you for all your reviews and suggestions from v1 to v6!
Quoted 17 lines · Feng Tang
[...]
mm/mempolicy.c nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy) · line 1916
Quoted 10 lines
mm/mempolicy.c +2
-1@@ -1887,7 +1909,8 @@ nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy)/* Return the node id preferred by the given mempolicy, or the given id */ static int policy_node (gfp_t gfp,struct mempolicy *policy,int nd) { -if (policy->mode ==MPOL_PREFERRED ) { +if (policy->mode ==MPOL_PREFERRED || + policy->mode ==MPOL_PREFERRED_MANY ) { nd =first_node (policy->nodes); }else {/* Do we really want to have the preferred node to be always the first node in the node mask? Shouldn't that strive for a locality as well? Existing callers already prefer
numa_node_id()- aka local node - and I belive we shouldn't just throw that away here.
I think it's about the difference of 'local' and 'prefer/perfer-many' policy. There are different kinds of memory HW: HBM(High Bandwidth Memory), normal DRAM, PMEM (Persistent Memory), which have different price, bandwidth, speed etc. A platform may have two, or all three of these types, and there are real use case which want memory comes 'preferred' node/nodes than the local node.
And good point for 'local node', if the 'prefer-many' policy's nodemask has local node set, we should pick it han this 'first_node', and the same semantic also applies to the other several places you pointed out. Or do I misunderstand you point?
Thanks,
Feng
Quoted 43 lines
mm/mempolicy.c unsigned int mempolicy_slab_node(void) · line 1958
Quoted 6 lines
mm/mempolicy.c +1
-0@@ -1931,6 +1954,7 @@ unsigned int mempolicy_slab_node(void)switch (policy->mode) {case MPOL_PREFERRED : +case MPOL_PREFERRED_MANY :return first_node (policy->nodes);Similarly here but I am not really familiar with the slab numa code enough to have strong opinions here.
Quoted 4 lines
mm/mempolicy.c +0
-0@@ -2173,10 +2198,12 @@ struct page *alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma,* node and don't fall back to other nodes, as the cost of * remote accesses would likely offset THP benefits. * mm/mempolicy.c struct page *alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma, · line 2207
Quoted 10 lines
mm/mempolicy.c +5
-3-* If the policy is interleave, or does not allow the current -* node in its nodemask, we allocate the standard way. +* If the policy is interleave or multiple preferred nodes, or +* does not allow the current node in its nodemask, we allocate +* the standard way. */ -if (pol->mode ==MPOL_PREFERRED ) +if ((pol->mode ==MPOL_PREFERRED || + pol->mode ==MPOL_PREFERRED_MANY )) hpage_node =first_node (pol->nodes);Same here.
mm/mempolicy.c int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned long · line 2486
Quoted 9 lines
mm/mempolicy.c +3
-0@@ -2451,6 +2479,9 @@ int mpol_misplaced(struct page *page, struct vm_area_struct *vma, unsigned longbreak ;case MPOL_PREFERRED : +case MPOL_PREFERRED_MANY : +if (node_isset (curnid, pol->nodes)) + goto out; polnid =first_node (pol->nodes);break ;I do not follow what is the point of using
first_nodehere. Either the node is in the mask or it is misplaced. What are you trying to achieve here?Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
reviewerRe: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 31 lines · Feng Tang
Quoted 1 line · Michal Hocko
[Sorry for a late review]
Not at all. Thank you for all your reviews and suggestions from v1 to v6!
Quoted 17 lines · Feng Tang
[...]
mm/mempolicy.c nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy) · line 1916
Quoted 10 lines
mm/mempolicy.c +2
-1@@ -1887,7 +1909,8 @@ nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy)/* Return the node id preferred by the given mempolicy, or the given id */ static int policy_node (gfp_t gfp,struct mempolicy *policy,int nd) { -if (policy->mode ==MPOL_PREFERRED ) { +if (policy->mode ==MPOL_PREFERRED || + policy->mode ==MPOL_PREFERRED_MANY ) { nd =first_node (policy->nodes); }else {/* Do we really want to have the preferred node to be always the first node in the node mask? Shouldn't that strive for a locality as well? Existing callers already prefer
numa_node_id()- aka local node - and I belive we shouldn't just throw that away here.I think it's about the difference of 'local' and 'prefer/perfer-many' policy. There are different kinds of memory HW: HBM(High Bandwidth Memory), normal DRAM, PMEM (Persistent Memory), which have different price, bandwidth, speed etc. A platform may have two, or all three of these types, and there are real use case which want memory comes 'preferred' node/nodes than the local node.
Quoted 4 lines
And good point for 'local node', if the 'prefer-many' policy's nodemask has local node set, we should pick it han this '
first_node', and the same semantic also applies to the other several places you pointed out. Or do I misunderstand you point?
Yeah. Essentially what I am trying to tell is that for MPOL_PREFERRED_MANY you simply want to return the given node without any alternation. That node will be used for the fallback zonelist and the nodemask would make sure we won't get out of the policy.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 37 lines · Michal Hocko
Quoted 31 lines · Feng Tang
Quoted 1 line · Michal Hocko
[Sorry for a late review]
Not at all. Thank you for all your reviews and suggestions from v1 to v6!
Quoted 17 lines · Feng Tang
[...]
mm/mempolicy.c nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy) · line 1916
Quoted 10 lines
mm/mempolicy.c +2
-1@@ -1887,7 +1909,8 @@ nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy)/* Return the node id preferred by the given mempolicy, or the given id */ static int policy_node (gfp_t gfp,struct mempolicy *policy,int nd) { -if (policy->mode ==MPOL_PREFERRED ) { +if (policy->mode ==MPOL_PREFERRED || + policy->mode ==MPOL_PREFERRED_MANY ) { nd =first_node (policy->nodes); }else {/* Do we really want to have the preferred node to be always the first node in the node mask? Shouldn't that strive for a locality as well? Existing callers already prefer
numa_node_id()- aka local node - and I belive we shouldn't just throw that away here.I think it's about the difference of 'local' and 'prefer/perfer-many' policy. There are different kinds of memory HW: HBM(High Bandwidth Memory), normal DRAM, PMEM (Persistent Memory), which have different price, bandwidth, speed etc. A platform may have two, or all three of these types, and there are real use case which want memory comes 'preferred' node/nodes than the local node.
Quoted 4 lines
And good point for 'local node', if the 'prefer-many' policy's nodemask has local node set, we should pick it han this '
first_node', and the same semantic also applies to the other several places you pointed out. Or do I misunderstand you point?
Quoted 4 lines
Yeah. Essentially what I am trying to tell is that for
MPOL_PREFERRED_MANYyou simply want to return the given node without any alternation. That node will be used for the fallback zonelist and the nodemask would make sure we won't get out of the policy.
I think I got your point now :)
With current mainline code, the 'prefer' policy will return the preferred node.
For 'prefer-many', we would like to keep the similar semantic, that the preference of node is 'preferred' > 'local' > all other nodes. There is some customer use case, whose platform has both DRAM and cheaper, bigger and slower PMEM, and they anlayzed the hotness of their huge data, and they want to put huge cold data into the PMEM, and only fallback to DRAM as the last step. The HW topology could be simplified like this:
Socket 0: Node 0 (CPU + 64GB DRAM), Node 2 (512GB PMEM) Socket 1: Node 1 (CPU + 64GB DRAM), Node 3 (512GB PMEM)
E.g they want to allocate memory for colde application data with 'prefer-many' policy + 0xC nodemask (N2+N3 PMEM nodes), so no matter the application is running on Node 0 or Node 1, the 'local' node only has DRAM which is not their preference, and want a preferred-->local-->others order.
Thanks,
Feng
Quoted 3 lines
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
reviewerRe: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 43 lines · Feng Tang
Quoted 37 lines · Michal Hocko
Quoted 31 lines · Feng Tang
Quoted 1 line · Michal Hocko
[Sorry for a late review]
Not at all. Thank you for all your reviews and suggestions from v1 to v6!
Quoted 17 lines · Feng Tang
[...]
mm/mempolicy.c nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy) · line 1916
Quoted 10 lines
mm/mempolicy.c +2
-1@@ -1887,7 +1909,8 @@ nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy)/* Return the node id preferred by the given mempolicy, or the given id */ static int policy_node (gfp_t gfp,struct mempolicy *policy,int nd) { -if (policy->mode ==MPOL_PREFERRED ) { +if (policy->mode ==MPOL_PREFERRED || + policy->mode ==MPOL_PREFERRED_MANY ) { nd =first_node (policy->nodes); }else {/* Do we really want to have the preferred node to be always the first node in the node mask? Shouldn't that strive for a locality as well? Existing callers already prefer
numa_node_id()- aka local node - and I belive we shouldn't just throw that away here.I think it's about the difference of 'local' and 'prefer/perfer-many' policy. There are different kinds of memory HW: HBM(High Bandwidth Memory), normal DRAM, PMEM (Persistent Memory), which have different price, bandwidth, speed etc. A platform may have two, or all three of these types, and there are real use case which want memory comes 'preferred' node/nodes than the local node.
Quoted 4 lines
And good point for 'local node', if the 'prefer-many' policy's nodemask has local node set, we should pick it han this '
first_node', and the same semantic also applies to the other several places you pointed out. Or do I misunderstand you point?Quoted 4 lines
Yeah. Essentially what I am trying to tell is that for
MPOL_PREFERRED_MANYyou simply want to return the given node without any alternation. That node will be used for the fallback zonelist and the nodemask would make sure we won't get out of the policy.
Quoted 4 lines
I think I got your point now :)
With current mainline code, the 'prefer' policy will return the preferred node.
Yes this makes sense as there is only one node.
Quoted 2 lines
For 'prefer-many', we would like to keep the similar semantic, that the preference of node is 'preferred' > 'local' > all other nodes.
Yes but which of the preferred nodes you want to start with. Say your nodemask preferring nodes 0 and 2 with the following topology
0 1 2 3
0 10 30 20 30
1 30 10 20 30
2 20 30 10 30
3 30 30 30 10
And say you are running on cpu 1. I believe you want your allocation preferably from node 2 rathern than 0, right? With your approach you would start with node 0 which would be more distant from cpu 1. Also the semantic to give nodes some ordering based on their numbers sounds rather weird to me.
The semantic I am proposing is to allocate from prefered nodes in distance order starting from the local node.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 56 lines · Michal Hocko
Quoted 43 lines · Feng Tang
Quoted 37 lines · Michal Hocko
Quoted 31 lines · Feng Tang
Quoted 1 line · Michal Hocko
[Sorry for a late review]
Not at all. Thank you for all your reviews and suggestions from v1 to v6!
Quoted 17 lines · Feng Tang
[...]
mm/mempolicy.c nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy) · line 1916
Quoted 10 lines
mm/mempolicy.c +2
-1@@ -1887,7 +1909,8 @@ nodemask_t *policy_nodemask(gfp_t gfp, struct mempolicy *policy)/* Return the node id preferred by the given mempolicy, or the given id */ static int policy_node (gfp_t gfp,struct mempolicy *policy,int nd) { -if (policy->mode ==MPOL_PREFERRED ) { +if (policy->mode ==MPOL_PREFERRED || + policy->mode ==MPOL_PREFERRED_MANY ) { nd =first_node (policy->nodes); }else {/* Do we really want to have the preferred node to be always the first node in the node mask? Shouldn't that strive for a locality as well? Existing callers already prefer
numa_node_id()- aka local node - and I belive we shouldn't just throw that away here.I think it's about the difference of 'local' and 'prefer/perfer-many' policy. There are different kinds of memory HW: HBM(High Bandwidth Memory), normal DRAM, PMEM (Persistent Memory), which have different price, bandwidth, speed etc. A platform may have two, or all three of these types, and there are real use case which want memory comes 'preferred' node/nodes than the local node.
Quoted 4 lines
And good point for 'local node', if the 'prefer-many' policy's nodemask has local node set, we should pick it han this '
first_node', and the same semantic also applies to the other several places you pointed out. Or do I misunderstand you point?Quoted 4 lines
Yeah. Essentially what I am trying to tell is that for
MPOL_PREFERRED_MANYyou simply want to return the given node without any alternation. That node will be used for the fallback zonelist and the nodemask would make sure we won't get out of the policy.Quoted 4 lines
I think I got your point now :)
With current mainline code, the 'prefer' policy will return the preferred node.
Yes this makes sense as there is only one node.
Quoted 2 lines
For 'prefer-many', we would like to keep the similar semantic, that the preference of node is 'preferred' > 'local' > all other nodes.
Yes but which of the preferred nodes you want to start with. Say your nodemask preferring nodes 0 and 2 with the following topology
Quoted 8 lines
0 1 2 30 10 30 20 30
1 30 10 20 30
2 20 30 10 30
3 30 30 30 10And say you are running on cpu 1. I believe you want your allocation preferably from node 2 rathern than 0, right?
Yes, and in one earlier reply, I had a similar thought
https://lore.kernel.org/lkml/[email protected]/
"
One further thought is, if local node is not in the nodemask, should we compare the distance of all the nodes in nodemask to the local node and chose the shortest?
"
And we may add a new API if there is no existing one:
int cloest_node(int nid, nodemask_t *nmask);
to pick the best node from 'prefer-many' nodemsk.
Quoted 6 lines
With your approach you
would start with node 0 which would be more distant from cpu 1.Also the
semantic to give nodes some ordering based on their numbers sounds rather weird to me.
I agree, and as I admitted in the first reply, this need to be fixed.
Quoted 2 lines
The semantic I am proposing is to allocate from prefered nodes in distance order starting from the local node.
So the plan is:
* if the local node is set in 'prefer-many's nodemask, then chose
* otherwise chose the node with the shortest distance to local node ?
Thanks,
Feng
Quoted 3 lines
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
reviewerRe: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 1 line · Feng Tang
On Thu, Jul 29, 2021 at 03:38:44PM +0200, Michal Hocko wrote:
[...]
Quoted 5 lines
Quoted 3 lines
Also the
semantic to give nodes some ordering based on their numbers sounds rather weird to me.I agree, and as I admitted in the first reply, this need to be fixed.
OK. I was not really clear that we are on the same page here.
Quoted 7 lines
Quoted 2 lines
The semantic I am proposing is to allocate from prefered nodes in distance order starting from the local node.
So the plan is:
* if the local node is set in 'prefer-many's nodemask, then chose
* otherwise chose the node with the shortest distance to local node ?
Yes and what I am trying to say is that you will achieve that simply by doing the following in policy_node:
if (policy->mode == MPOL_PREFERRED_MANY) return nd;
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 19 lines · Michal Hocko
Quoted 1 line · Feng Tang
On Thu, Jul 29, 2021 at 03:38:44PM +0200, Michal Hocko wrote:
[...]
Quoted 5 lines
Quoted 3 lines
Also the
semantic to give nodes some ordering based on their numbers sounds rather weird to me.I agree, and as I admitted in the first reply, this need to be fixed.
OK. I was not really clear that we are on the same page here.
Quoted 7 lines
Quoted 2 lines
The semantic I am proposing is to allocate from prefered nodes in distance order starting from the local node.
So the plan is:
* if the local node is set in 'prefer-many's nodemask, then chose
* otherwise chose the node with the shortest distance to local node ?
Quoted 4 lines
Yes and what I am trying to say is that you will achieve that simply by doing the following in
policy_node:if (policy->mode == MPOL_PREFERRED_MANY) return nd;
One thing is, it's possible that 'nd' is not set in the preferred nodemask.
For policy_node(), most of its caller use the local node id as 'nd' parameter. For HBM and PMEM memory nodes, they are cpuless nodes, so they will not be a 'local node', but some use cases only prefer these nodes.
Thanks,
Feng
Quoted 3 lines
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
reviewerRe: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 20 lines · Feng Tang
Quoted 19 lines · Michal Hocko
Quoted 1 line · Feng Tang
On Thu, Jul 29, 2021 at 03:38:44PM +0200, Michal Hocko wrote:
[...]
Quoted 5 lines
Quoted 3 lines
Also the
semantic to give nodes some ordering based on their numbers sounds rather weird to me.I agree, and as I admitted in the first reply, this need to be fixed.
OK. I was not really clear that we are on the same page here.
Quoted 7 lines
Quoted 2 lines
The semantic I am proposing is to allocate from prefered nodes in distance order starting from the local node.
So the plan is:
* if the local node is set in 'prefer-many's nodemask, then chose
* otherwise chose the node with the shortest distance to local node ?
Quoted 7 lines
Quoted 4 lines
Yes and what I am trying to say is that you will achieve that simply by doing the following in
policy_node:if (policy->mode == MPOL_PREFERRED_MANY) return nd;One thing is, it's possible that 'nd' is not set in the preferred nodemask.
Yes, and there shouldn't be any problem with that. The given node is only used to get the respective zonelist (order distance ordered list of zones to try). get_page_from_freelist will then use the preferred node mask to filter this zone list. Is that more clear now?
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 26 lines · Michal Hocko
Quoted 25 lines · Feng Tang
Quoted 23 lines · Michal Hocko
Quoted 1 line · Feng Tang
On Thu, Jul 29, 2021 at 03:38:44PM +0200, Michal Hocko wrote:
[...]
Quoted 5 lines
Quoted 3 lines
Also the
semantic to give nodes some ordering based on their numbers sounds rather weird to me.I agree, and as I admitted in the first reply, this need to be fixed.
OK. I was not really clear that we are on the same page here.
Quoted 7 lines
Quoted 2 lines
The semantic I am proposing is to allocate from prefered nodes in distance order starting from the local node.
So the plan is:
* if the local node is set in 'prefer-many's nodemask, then chose
* otherwise chose the node with the shortest distance to local node ?Yes and what I am trying to say is that you will achieve that simply by doing the following in
policy_node:if (policy->mode == MPOL_PREFERRED_MANY) return nd;
Quoted 7 lines
Quoted 2 lines
One thing is, it's possible that 'nd' is not set in the preferred nodemask.
Yes, and there shouldn't be any problem with that. The given node is only used to get the respective zonelist (order distance ordered list of zones to try).
get_page_from_freelistwill then use the preferred node mask to filter this zone list. Is that more clear now?
Yes, from the code, the policy_node() is always coupled with policy_nodemask(), which secures the 'nodemask' limit. Thanks for the clarification!
And for the mempolicy_slab_node(), it seems to be a little different, and we may need to reuse its logic for 'bind' policy, which is similar to what we've discussed, pick a nearest node to the local node. And similar for mpol_misplaced(). Thoughts?
Thanks,
Feng
Quoted 3 lines
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
reviewerRe: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
On Fri 30-07-21 15:18:40, Feng Tang wrote:
[...]
Quoted 4 lines
And for the
mempolicy_slab_node(), it seems to be a little different, and we may need to reuse its logic for 'bind' policy, which is similar to what we've discussed, pick a nearest node to the local node. And similar formpol_misplaced(). Thoughts?
I have to admit, I haven't looked closer to what slab does. I am not familiar with internals and would have to study it some more. Maybe slab maintainers have an idea.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
In reply to Feng TangOn Fri, Jul 30, 2021 at 03:18:40PM +0800, Tang, Feng wrote:
[snip]
Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
One thing is, it's possible that 'nd' is not set in the preferred nodemask.
Yes, and there shouldn't be any problem with that. The given node is only used to get the respective zonelist (order distance ordered list of zones to try).
get_page_from_freelistwill then use the preferred node mask to filter this zone list. Is that more clear now?Yes, from the code, the
policy_node()is always coupled withpolicy_nodemask(), which secures the 'nodemask' limit. Thanks for the clarification!
Hi Michal,
To ensure the nodemask limit, the policy_nodemask() also needs some change to return the nodemask for 'prefer-many' policy, so here is a updated 1/6 patch, which mainly changes the node/nodemask selection for 'prefer-many' policy, could you review it? thanks!
- Feng
------8<-------------------------------------------------------
include/uapi/linux/mempolicy.h +1 -0
@@ -22,6 +22,7 @@ enum {MPOL_BIND ,MPOL_INTERLEAVE ,MPOL_LOCAL , +MPOL_PREFERRED_MANY , MPOL_MAX, /* always last member of enum */ };
mm/mempolicy.c +56 -11
184 lines not on this page · raw message
Michal Hocko
reviewerRe: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 14 lines · Feng Tang
On Fri, Jul 30, 2021 at 03:18:40PM +0800, Tang, Feng wrote:
[snip]Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
One thing is, it's possible that 'nd' is not set in the preferred nodemask.
Yes, and there shouldn't be any problem with that. The given node is only used to get the respective zonelist (order distance ordered list of zones to try).
get_page_from_freelistwill then use the preferred node mask to filter this zone list. Is that more clear now?Yes, from the code, the
policy_node()is always coupled withpolicy_nodemask(), which secures the 'nodemask' limit. Thanks for the clarification!
Quoted 6 lines
Hi Michal,
To ensure the nodemask limit, the
policy_nodemask()also needs some change to return the nodemask for 'prefer-many' policy, so here is a updated 1/6 patch, which mainly changes the node/nodemask selection for 'prefer-many' policy, could you review it? thanks!
right, I have mixed it with get_policy_nodemask
Quoted 6 lines
mm/mempolicy.c +0
-2@@ -1875,8 +1897,13 @@ static int apply_policy_zone(struct mempolicy *policy, enum zone_type zone)*/ nodemask_t *policy_nodemask (gfp_t gfp,struct mempolicy *policy) { -/* Lower zones don't get a nodemask applied for MPOL_BIND */ - if (unlikely(policy->mode == MPOL_BIND) &&
mm/mempolicy.c static int apply_policy_zone(struct mempolicy *policy, enum zone_type zone) · line 1909
Quoted 10 lines
mm/mempolicy.c +7
-0+int mode = policy->mode; + +/* +* Lower zones don't get a nodemask applied for 'bind' and +* 'prefer-many' policies +*/ +if (unlikely (mode ==MPOL_BIND || mode ==MPOL_PREFERRED_MANY ) &&apply_policy_zone (policy,gfp_zone (gfp)) &&cpuset_nodemask_valid_mems_allowed (&policy->nodes))return &policy->nodes;
Isn't this just too cryptic? Why didn't you simply
if (mode == MPOL_PREFERRED_MANY) return &policy->mode;
in addition to the existing code? I mean why would you even care about cpusets? Those are handled at the page allocator layer and will further filter the given nodemask.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
Quoted 42 lines · Michal Hocko
Quoted 14 lines · Feng Tang
On Fri, Jul 30, 2021 at 03:18:40PM +0800, Tang, Feng wrote:
[snip]Quoted 11 lines
Quoted 7 lines
Quoted 2 lines
One thing is, it's possible that 'nd' is not set in the preferred nodemask.
Yes, and there shouldn't be any problem with that. The given node is only used to get the respective zonelist (order distance ordered list of zones to try).
get_page_from_freelistwill then use the preferred node mask to filter this zone list. Is that more clear now?Yes, from the code, the
policy_node()is always coupled withpolicy_nodemask(), which secures the 'nodemask' limit. Thanks for the clarification!Quoted 6 lines
Hi Michal,
To ensure the nodemask limit, the
policy_nodemask()also needs some change to return the nodemask for 'prefer-many' policy, so here is a updated 1/6 patch, which mainly changes the node/nodemask selection for 'prefer-many' policy, could you review it? thanks!right, I have mixed it with
get_policy_nodemaskQuoted 6 lines
mm/mempolicy.c +0
-2@@ -1875,8 +1897,13 @@ static int apply_policy_zone(struct mempolicy *policy, enum zone_type zone)*/ nodemask_t *policy_nodemask (gfp_t gfp,struct mempolicy *policy) { -/* Lower zones don't get a nodemask applied for MPOL_BIND */ - if (unlikely(policy->mode == MPOL_BIND) &&mm/mempolicy.c static int apply_policy_zone(struct mempolicy *policy, enum zone_type zone) · line 1909
Quoted 10 lines
mm/mempolicy.c +7
-0+int mode = policy->mode; + +/* +* Lower zones don't get a nodemask applied for 'bind' and +* 'prefer-many' policies +*/ +if (unlikely (mode ==MPOL_BIND || mode ==MPOL_PREFERRED_MANY ) &&apply_policy_zone (policy,gfp_zone (gfp)) &&cpuset_nodemask_valid_mems_allowed (&policy->nodes))return &policy->nodes;Isn't this just too cryptic? Why didn't you simply
Quoted 6 lines
if (mode == MPOL_PREFERRED_MANY) return &policy->mode;in addition to the existing code? I mean why would you even care about cpusets? Those are handled at the page allocator layer and will further filter the given nodemask.
Ok, I will follow your suggestion and keep 'bind' handling unchanged.
And to be honest, I don't fully understand the current handling for 'bind' policy, will the returning NULL for 'bind' policy open a sideway for the strict 'bind' limit.
Thanks,
Feng
Quoted 3 lines
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
reviewerRe: [PATCH v6 1/6] mm/mempolicy: Add MPOL_PREFERRED_MANY for multiple preferred nodes
On Mon 02-08-21 19:33:26, Feng Tang wrote:
[...]
Quoted 3 lines
And to be honest, I don't fully understand the current handling for 'bind' policy, will the returning NULL for 'bind' policy open a sideway for the strict 'bind' limit.
I do not remember all the details but this is an old behavior that MBIND policy doesn't apply to kernel allocations in presnce of the movable zone. Detailed reasoning is not clear to me at the moment, maybe Mel remembers?
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v6 2/6 mm/memplicy: add page allocation function for MPOL_PREFERRED_MANY policy
In reply to Feng TangThe semantics of MPOL_PREFERRED_MANY is similar to MPOL_PREFERRED, that it will first try to allocate memory from the preferred node(s), and fallback to all nodes in system when first try fails.
Add a dedicated function for it just like 'interleave' policy.
--- mm/mempolicy.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)
mm/mempolicy.c +19 -0
@@ -2153,6 +2153,25 @@ static struct page *alloc_page_interleave(gfp_t gfp, unsigned order,return page; } +static struct page *alloc_page_preferred_many (gfp_t gfp,unsigned int order, +struct mempolicy *pol) +{ +struct page *page; + +/* +* This is a two pass approach. The first pass will only try the +* preferred nodes but skip the direct reclaim and allow the +* allocation to fail, while the second pass will try all the +* nodes in system. +*/ + page =__alloc_pages (((gfp | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM), + order,first_node (pol->nodes), &pol->nodes); +if (!page) + page =__alloc_pages (gfp, order,numa_node_id (),NULL ); + +return page; +} +/** * alloc_pages_vma - Allocate a page for a VMA. * @gfp: GFP flags.
Signature · 1 line
2.7.4
Michal Hocko
reviewerRe: [PATCH v6 2/6] mm/memplicy: add page allocation function for MPOL_PREFERRED_MANY policy
Quoted 11 lines · Feng Tang
The semantics of
MPOL_PREFERRED_MANYis similar toMPOL_PREFERRED, that it will first try to allocate memory from the preferred node(s), and fallback to all nodes in system when first try fails.Add a dedicated function for it just like 'interleave' policy.
Suggested-by Michal Hocko <[email protected]>Co-developed-by Ben Widawsky <[email protected]>Signed-off-by Ben Widawsky <[email protected]>Signed-off-by Feng Tang <[email protected]>
It would be better to squash this together with the actual user of the function added by the next patch.
Quoted 9 lines
--- mm/mempolicy.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)mm/mempolicy.c +0
-0
mm/mempolicy.c static struct page *alloc_page_interleave(gfp_t gfp, unsigned order, · line 2157
Quoted 5 lines
mm/mempolicy.c +2
-0@@ -2153,6 +2153,25 @@ static struct page *alloc_page_interleave(gfp_t gfp, unsigned order,return page; } +static struct page *alloc_page_preferred_many (gfp_t gfp,unsigned int order, +struct mempolicy *pol)
We likely want a node parameter to know which one we want to start with for locality. Callers should use policy_node for that.
mm/mempolicy.c static struct page *alloc_page_interleave(gfp_t gfp, unsigned order, · line 2168
Quoted 11 lines
mm/mempolicy.c +11
-0+{ +struct page *page; + +/* +* This is a two pass approach. The first pass will only try the +* preferred nodes but skip the direct reclaim and allow the +* allocation to fail, while the second pass will try all the +* nodes in system. +*/ + page =__alloc_pages (((gfp | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM), + order,first_node (pol->nodes), &pol->nodes);
Although most users will likely have some form of GFP_*USER* here and clearing __GFP_DIRECT_RECLAIM will put all other reclaim modifiers out of game I think it would be better to explicitly disable some of them to prevent from surprises. E.g. any potential __GFP_NOFAIL would be more than surprising here. We do not have any (hopefully) but this should be pretty cheap to exclude as we already have to modify already.
preferred_gfp = gfp | __GFP_NOWARN; preferred_gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL)
Quoted 11 lines
mm/mempolicy.c +6
-0+if (!page) + page =__alloc_pages (gfp, order,numa_node_id (),NULL ); + +return page; +} +/** * alloc_pages_vma - Allocate a page for a VMA. * @gfp: GFP flags. Signature · 1 line
2.7.4
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 2/6] mm/memplicy: add page allocation function for MPOL_PREFERRED_MANY policy
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Feng Tang
The semantics of
MPOL_PREFERRED_MANYis similar toMPOL_PREFERRED, that it will first try to allocate memory from the preferred node(s), and fallback to all nodes in system when first try fails.Add a dedicated function for it just like 'interleave' policy.
Quoted 8 lines
Quoted 5 lines
Suggested-by Michal Hocko <[email protected]>Co-developed-by Ben Widawsky <[email protected]>Signed-off-by Ben Widawsky <[email protected]>Signed-off-by Feng Tang <[email protected]>It would be better to squash this together with the actual user of the function added by the next patch.
Ok, will do
Quoted 12 lines
Quoted 12 lines
--- mm/mempolicy.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)mm/mempolicy.c +0
-0@@ -2153,6 +2153,25 @@ static struct page *alloc_page_interleave(gfp_t gfp, unsigned order,return page; }
Quoted 5 lines
mm/mempolicy.c static struct page *alloc_page_interleave(gfp_t gfp, unsigned order, · line 2157
Quoted 2 lines
mm/mempolicy.c +2
-0+static struct page *alloc_page_preferred_many (gfp_t gfp,unsigned int order, +struct mempolicy *pol)We likely want a node parameter to know which one we want to start with for locality. Callers should use
policy_nodefor that.
Yes, locality should be considered, something like this?
int pnid, lnid = numa_node_id(); if (is_nodeset(lnid, &pol->nodes)) pnid = local_nid; else pnid = first_node(pol->nodes); page = __alloc_pages(((gfp | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM), order, pnid, &pol->nodes); if (!page) page = __alloc_pages(gfp, order, lnid, NULL); return page;
Quoted 19 lines
mm/mempolicy.c static struct page *alloc_page_interleave(gfp_t gfp, unsigned order, · line 2168
Quoted 11 lines
mm/mempolicy.c +11
-0+{ +struct page *page; + +/* +* This is a two pass approach. The first pass will only try the +* preferred nodes but skip the direct reclaim and allow the +* allocation to fail, while the second pass will try all the +* nodes in system. +*/ + page =__alloc_pages (((gfp | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM), + order,first_node (pol->nodes), &pol->nodes);Although most users will likely have some form of GFP_*USER* here and clearing
__GFP_DIRECT_RECLAIMwill put all other reclaim modifiers out of game I think it would be better to explicitly disable some of them to prevent from surprises. E.g. any potential__GFP_NOFAILwould be more than surprising here. We do not have any (hopefully) but this should be pretty cheap to exclude as we already have to modify already.
Quoted 2 lines
preferred_gfp = gfp | __GFP_NOWARN; preferred_gfp &= ~(__GFP_DIRECT_RECLAIM | __GFP_NOFAIL)
OK, will add.
Thanks,
Feng
Quoted 16 lines
Quoted 12 lines
mm/mempolicy.c +6
-0+if (!page) + page =__alloc_pages (gfp, order,numa_node_id (),NULL ); + +return page; +} +/** * alloc_pages_vma - Allocate a page for a VMA. * @gfp: GFP flags. Signature · 1 line
2.7.4Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 2/6] mm/memplicy: add page allocation function for MPOL_PREFERRED_MANY policy
Quoted 37 lines · Feng
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Feng Tang
The semantics of
MPOL_PREFERRED_MANYis similar toMPOL_PREFERRED, that it will first try to allocate memory from the preferred node(s), and fallback to all nodes in system when first try fails.Add a dedicated function for it just like 'interleave' policy.
Quoted 8 lines
Quoted 5 lines
Suggested-by Michal Hocko <[email protected]>Co-developed-by Ben Widawsky <[email protected]>Signed-off-by Ben Widawsky <[email protected]>Signed-off-by Feng Tang <[email protected]>It would be better to squash this together with the actual user of the function added by the next patch.
Ok, will do
Quoted 12 lines
Quoted 12 lines
--- mm/mempolicy.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)mm/mempolicy.c +0
-0@@ -2153,6 +2153,25 @@ static struct page *alloc_page_interleave(gfp_t gfp, unsigned order,return page; }Quoted 5 lines
mm/mempolicy.c static struct page *alloc_page_interleave(gfp_t gfp, unsigned order, · line 2157
Quoted 2 lines
mm/mempolicy.c +2
-0+static struct page *alloc_page_preferred_many (gfp_t gfp,unsigned int order, +struct mempolicy *pol)We likely want a node parameter to know which one we want to start with for locality. Callers should use
policy_nodefor that.
Quoted 8 lines
Yes, locality should be considered, something like this?
int pnid, lnid = numa_node_id(); if (is_nodeset(lnid, &pol->nodes)) pnid = local_nid; else pnid = first_node(pol->nodes);
One further thought is, if local node is not in the nodemask, should we compare the distance of all the nodes in nodemask to the local node and chose the shortest?
Thanks,
Feng
Quoted 6 lines
page = __alloc_pages(((gfp | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM), order, pnid, &pol->nodes); if (!page) page = __alloc_pages(gfp, order, lnid, NULL); return page;
Michal Hocko
reviewerRe: [PATCH v6 2/6] mm/memplicy: add page allocation function for MPOL_PREFERRED_MANY policy
Quoted 42 lines · Feng Tang
Quoted 41 lines · Feng
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Feng Tang
The semantics of
MPOL_PREFERRED_MANYis similar toMPOL_PREFERRED, that it will first try to allocate memory from the preferred node(s), and fallback to all nodes in system when first try fails.Add a dedicated function for it just like 'interleave' policy.
Quoted 8 lines
Quoted 5 lines
Suggested-by Michal Hocko <[email protected]>Co-developed-by Ben Widawsky <[email protected]>Signed-off-by Ben Widawsky <[email protected]>Signed-off-by Feng Tang <[email protected]>It would be better to squash this together with the actual user of the function added by the next patch.
Ok, will do
Quoted 12 lines
Quoted 12 lines
--- mm/mempolicy.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)mm/mempolicy.c +0
-0@@ -2153,6 +2153,25 @@ static struct page *alloc_page_interleave(gfp_t gfp, unsigned order,return page; }Quoted 5 lines
mm/mempolicy.c static struct page *alloc_page_interleave(gfp_t gfp, unsigned order, · line 2157
Quoted 2 lines
mm/mempolicy.c +2
-0+static struct page *alloc_page_preferred_many (gfp_t gfp,unsigned int order, +struct mempolicy *pol)We likely want a node parameter to know which one we want to start with for locality. Callers should use
policy_nodefor that.Yes, locality should be considered, something like this?
int pnid, lnid = numa_node_id();
Quoted 8 lines
Quoted 4 lines
if (is_nodeset(lnid, &pol->nodes)) pnid = local_nid; else pnid = first_node(pol->nodes);One further thought is, if local node is not in the nodemask, should we compare the distance of all the nodes in nodemask to the local node and chose the shortest?
Nope, That is zonelist for. Nodemask will do the rest.
Signature · 2 lines
Michal Hocko SUSE Labs
Michal Hocko
reviewerRe: [PATCH v6 2/6] mm/memplicy: add page allocation function for MPOL_PREFERRED_MANY policy
In reply to Feng TangQuoted 39 lines · Feng Tang
Quoted 7 lines · Michal Hocko
Quoted 6 lines · Feng Tang
The semantics of
MPOL_PREFERRED_MANYis similar toMPOL_PREFERRED, that it will first try to allocate memory from the preferred node(s), and fallback to all nodes in system when first try fails.Add a dedicated function for it just like 'interleave' policy.
Quoted 8 lines
Quoted 5 lines
Suggested-by Michal Hocko <[email protected]>Co-developed-by Ben Widawsky <[email protected]>Signed-off-by Ben Widawsky <[email protected]>Signed-off-by Feng Tang <[email protected]>It would be better to squash this together with the actual user of the function added by the next patch.
Ok, will do
Quoted 12 lines
Quoted 12 lines
--- mm/mempolicy.c | 19 +++++++++++++++++++ 1 file changed, 19 insertions(+)mm/mempolicy.c +0
-0@@ -2153,6 +2153,25 @@ static struct page *alloc_page_interleave(gfp_t gfp, unsigned order,return page; }Quoted 5 lines
mm/mempolicy.c static struct page *alloc_page_interleave(gfp_t gfp, unsigned order, · line 2157
Quoted 2 lines
mm/mempolicy.c +2
-0+static struct page *alloc_page_preferred_many (gfp_t gfp,unsigned int order, +struct mempolicy *pol)We likely want a node parameter to know which one we want to start with for locality. Callers should use
policy_nodefor that.Yes, locality should be considered, something like this?
Quoted 12 lines
int pnid, lnid = numa_node_id(); if (is_nodeset(lnid, &pol->nodes)) pnid = local_nid; else pnid = first_node(pol->nodes); page = __alloc_pages(((gfp | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM), order, pnid, &pol->nodes); if (!page) page = __alloc_pages(gfp, order, lnid, NULL); return page;
No. I really meant to get a node argument and use it as it is. Your callers already have some node preferences. Usually a local node and as we have a nodemask here then we do not really need to have any special logic here as mentioned in other email. The preferred node will act only as a source for the zone list.
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v6 3/6 mm/mempolicy: enable page allocation for MPOL_PREFERRED_MANY for general cases
In reply to Feng TangFrom: Ben Widawsky <[email protected]>
In order to support MPOL_PREFERRED_MANY which is used by set_mempolicy(2), mbind(2), enable both alloc_pages() and alloc_pages_vma() by using alloc_page_preferred_many().
--- mm/mempolicy.c | 8 ++++++++ 1 file changed, 8 insertions(+)
mm/mempolicy.c +8 -0
@@ -2207,6 +2207,12 @@ struct page *alloc_pages_vma(gfp_t gfp, int order, struct vm_area_struct *vma, goto out; } +if (pol->mode ==MPOL_PREFERRED_MANY ) { + page =alloc_page_preferred_many (gfp, order, pol); +mpol_cond_put (pol); + goto out; + } +if (unlikely (IS_ENABLED (CONFIG_TRANSPARENT_HUGEPAGE ) && hugepage)) {int hpage_node = node;
@@ -2286,6 +2292,8 @@ struct page *alloc_pages(gfp_t gfp, unsigned order)*/ if (pol->mode ==MPOL_INTERLEAVE ) page =alloc_page_interleave (gfp, order,interleave_nodes (pol)); +else if (pol->mode ==MPOL_PREFERRED_MANY ) + page =alloc_page_preferred_many (gfp, order, pol);else page = __alloc_pages(gfp, order, policy_node(gfp, pol, numa_node_id()),
Signature · 1 line
2.7.4
PATCH v6 4/6 mm/hugetlb: add support for mempolicy MPOL_PREFERRED_MANY
In reply to Feng TangFrom: Ben Widawsky <[email protected]>
Implement the missing huge page allocation functionality while obeying the preferred node semantics. This is similar to the implementation for general page allocation, as it uses a fallback mechanism to try multiple preferred nodes first, and then all other nodes.
[Thanks to 0day bot for caching the missing #ifdef CONFIG_NUMA issue]
--- mm/hugetlb.c | 25 +++++++++++++++++++++++++ mm/mempolicy.c | 3 ++- 2 files changed, 27 insertions(+), 1 deletion(-)
mm/hugetlb.c +25 -0
@@ -1164,7 +1164,18 @@ static struct page *dequeue_huge_page_vma(struct hstate *h, gfp_mask =htlb_alloc_mask (h); nid =huge_node (vma, address, gfp_mask, &mpol, &nodemask); +#ifdef CONFIG_NUMA +if (mpol->mode ==MPOL_PREFERRED_MANY ) { + page =dequeue_huge_page_nodemask (h, gfp_mask, nid, nodemask); +if (page) + goto check_reserve; +/* Fallback to all nodes */ + nodemask =NULL ; + } +#endif page =dequeue_huge_page_nodemask (h, gfp_mask, nid, nodemask); + +check_reserve:if (page && !avoid_reserve &&vma_has_reserves (vma, chg)) {SetHPageRestoreReserve (page); h->resv_huge_pages--;
@@ -2095,6 +2106,20 @@ struct page *alloc_buddy_huge_page_with_mpol(struct hstate *h,nodemask_t *nodemask; nid =huge_node (vma, addr, gfp_mask, &mpol, &nodemask); +#ifdef CONFIG_NUMA +if (mpol->mode ==MPOL_PREFERRED_MANY ) { +gfp_t gfp = (gfp_mask | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM; + + page =alloc_surplus_huge_page (h, gfp, nid, nodemask); +if (page) { +mpol_cond_put (mpol); +return page; + } + +/* Fallback to all nodes */ + nodemask =NULL ; + } +#endif page =alloc_surplus_huge_page (h, gfp_mask, nid, nodemask);mpol_cond_put (mpol);
mm/mempolicy.c +2 -1
@@ -2054,7 +2054,8 @@ int huge_node(struct vm_area_struct *vma, unsigned long addr, gfp_t gfp_flags, huge_page_shift(hstate_vma(vma))); }else { nid =policy_node (gfp_flags, *mpol,numa_node_id ()); -if ((*mpol)->mode ==MPOL_BIND ) +if ((*mpol)->mode ==MPOL_BIND || + (*mpol)->mode ==MPOL_PREFERRED_MANY ) *nodemask = &(*mpol)->nodes; }return nid;
Signature · 1 line
2.7.4
Re: [PATCH v6 4/6] mm/hugetlb: add support for mempolicy MPOL_PREFERRED_MANY
Quoted 43 lines · Feng Tang
From: Ben Widawsky <[email protected]>
Implement the missing huge page allocation functionality while obeying the preferred node semantics. This is similar to the implementation for general page allocation, as it uses a fallback mechanism to try multiple preferred nodes first, and then all other nodes.
[Thanks to 0day bot for caching the missing #ifdef
CONFIG_NUMAissue]Suggested-by Michal Hocko <[email protected]>Signed-off-by Ben Widawsky <[email protected]>Co-developed-by Feng Tang <[email protected]>Signed-off-by Feng Tang <[email protected]>--- mm/hugetlb.c | 25 +++++++++++++++++++++++++ mm/mempolicy.c | 3 ++- 2 files changed, 27 insertions(+), 1 deletion(-)mm/hugetlb.c +11
-0@@ -1164,7 +1164,18 @@ static struct page *dequeue_huge_page_vma(struct hstate *h, gfp_mask =htlb_alloc_mask (h); nid =huge_node (vma, address, gfp_mask, &mpol, &nodemask); +#ifdef CONFIG_NUMA +if (mpol->mode ==MPOL_PREFERRED_MANY ) { + page =dequeue_huge_page_nodemask (h, gfp_mask, nid, nodemask); +if (page) + goto check_reserve; +/* Fallback to all nodes */ + nodemask =NULL ; + } +#endif page =dequeue_huge_page_nodemask (h, gfp_mask, nid, nodemask); + +check_reserve:if (page && !avoid_reserve &&vma_has_reserves (vma, chg)) {SetHPageRestoreReserve (page); h->resv_huge_pages--;
mm/hugetlb.c struct page *alloc_buddy_huge_page_with_mpol(struct hstate *h, · line 2111
Quoted 6 lines
mm/hugetlb.c +3
-0@@ -2095,6 +2106,20 @@ struct page *alloc_buddy_huge_page_with_mpol(struct hstate *h,nodemask_t *nodemask; nid =huge_node (vma, addr, gfp_mask, &mpol, &nodemask); +#ifdef CONFIG_NUMA +if (mpol->mode ==MPOL_PREFERRED_MANY ) { +gfp_t gfp = (gfp_mask | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM;
I believe __GFP_NOWARN will be added later in alloc_buddy_huge_page, so no need to add here?
Quoted 20 lines
mm/hugetlb.c +11
-0+ + page =alloc_surplus_huge_page (h, gfp, nid, nodemask); +if (page) { +mpol_cond_put (mpol); +return page; + } + +/* Fallback to all nodes */ + nodemask = NULL; + } +#endif page =alloc_surplus_huge_page (h, gfp_mask, nid, nodemask);mpol_cond_put (mpol);mm/mempolicy.c +0
-0
mm/mempolicy.c int huge_node(struct vm_area_struct *vma, unsigned long addr, gfp_t gfp_flags, · line 2061
Quoted 9 lines
mm/mempolicy.c +2
-1@@ -2054,7 +2054,8 @@ int huge_node(struct vm_area_struct *vma, unsigned long addr, gfp_t gfp_flags, huge_page_shift(hstate_vma(vma))); }else { nid =policy_node (gfp_flags, *mpol,numa_node_id ()); -if ((*mpol)->mode ==MPOL_BIND ) +if ((*mpol)->mode ==MPOL_BIND || + (*mpol)->mode ==MPOL_PREFERRED_MANY ) *nodemask = &(*mpol)->nodes; }return nid;
Other than the one nit above,
Signature · 1 line
Mike Kravetz
Re: [PATCH v6 4/6] mm/hugetlb: add support for mempolicy MPOL_PREFERRED_MANY
Mike,
Quoted 46 lines · Mike Kravetz
Quoted 45 lines · Feng Tang
From: Ben Widawsky <[email protected]>
Implement the missing huge page allocation functionality while obeying the preferred node semantics. This is similar to the implementation for general page allocation, as it uses a fallback mechanism to try multiple preferred nodes first, and then all other nodes.
[Thanks to 0day bot for caching the missing #ifdef
CONFIG_NUMAissue]Suggested-by Michal Hocko <[email protected]>Signed-off-by Ben Widawsky <[email protected]>Co-developed-by Feng Tang <[email protected]>Signed-off-by Feng Tang <[email protected]>--- mm/hugetlb.c | 25 +++++++++++++++++++++++++ mm/mempolicy.c | 3 ++- 2 files changed, 27 insertions(+), 1 deletion(-)mm/hugetlb.c +11
-0@@ -1164,7 +1164,18 @@ static struct page *dequeue_huge_page_vma(struct hstate *h, gfp_mask =htlb_alloc_mask (h); nid =huge_node (vma, address, gfp_mask, &mpol, &nodemask); +#ifdef CONFIG_NUMA +if (mpol->mode ==MPOL_PREFERRED_MANY ) { + page =dequeue_huge_page_nodemask (h, gfp_mask, nid, nodemask); +if (page) + goto check_reserve; +/* Fallback to all nodes */ + nodemask =NULL ; + } +#endif page =dequeue_huge_page_nodemask (h, gfp_mask, nid, nodemask); + +check_reserve:if (page && !avoid_reserve &&vma_has_reserves (vma, chg)) {SetHPageRestoreReserve (page); h->resv_huge_pages--;@@ -2095,6 +2106,20 @@ struct page *alloc_buddy_huge_page_with_mpol(struct hstate *h,nodemask_t *nodemask;
Quoted 7 lines
mm/hugetlb.c struct page *alloc_buddy_huge_page_with_mpol(struct hstate *h, · line 2111
Quoted 4 lines
mm/hugetlb.c +3
-0nid =huge_node (vma, addr, gfp_mask, &mpol, &nodemask); +#ifdef CONFIG_NUMA +if (mpol->mode ==MPOL_PREFERRED_MANY ) { +gfp_t gfp = (gfp_mask | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM;I believe
__GFP_NOWARNwill be added later inalloc_buddy_huge_page, so no need to add here?
Thanks for the suggestion, will remove it.
Quoted 32 lines
Quoted 20 lines
mm/hugetlb.c +11
-0+ + page =alloc_surplus_huge_page (h, gfp, nid, nodemask); +if (page) { +mpol_cond_put (mpol); +return page; + } + +/* Fallback to all nodes */ + nodemask = NULL; + } +#endif page =alloc_surplus_huge_page (h, gfp_mask, nid, nodemask);mpol_cond_put (mpol);mm/mempolicy.c +0
-0mm/mempolicy.c int huge_node(struct vm_area_struct *vma, unsigned long addr, gfp_t gfp_flags, · line 2061
Quoted 9 lines
mm/mempolicy.c +2
-1@@ -2054,7 +2054,8 @@ int huge_node(struct vm_area_struct *vma, unsigned long addr, gfp_t gfp_flags, huge_page_shift(hstate_vma(vma))); }else { nid =policy_node (gfp_flags, *mpol,numa_node_id ()); -if ((*mpol)->mode ==MPOL_BIND ) +if ((*mpol)->mode ==MPOL_BIND || + (*mpol)->mode ==MPOL_PREFERRED_MANY ) *nodemask = &(*mpol)->nodes; }return nid;Other than the one nit above,
Quoted 1 line
Reviewed-by Mike Kravetz <[email protected]>
Thanks!
Andrew,
I have to ask for your help again to fold this to the 4/6 patch, thanks!
- Feng
---------------------------8<--------------------------------------------
From de1cd29d8da96856a6d754a30a4c7585d87b8348 Mon Sep 17 00:00:00 2001
From: Feng Tang <[email protected]>
Date: Thu, 22 Jul 2021 16:00:49 +0800
Subject: [PATCH] mm/hugetlb: remove the unneeded __GFP_NOWARN flag setting
As the alloc_buddy_huge_page() will set it anyway.
--- mm/hugetlb.c | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
mm/hugetlb.c +3 -3
@@ -2162,9 +2162,9 @@ struct page *alloc_buddy_huge_page_with_mpol(struct hstate *h, nid =huge_node (vma, addr, gfp_mask, &mpol, &nodemask);#ifdef CONFIG_NUMA if (mpol->mode ==MPOL_PREFERRED_MANY ) { -gfp_t gfp = (gfp_mask | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM; - - page =alloc_surplus_huge_page (h, gfp, nid, nodemask, false); + page =alloc_surplus_huge_page (h, + gfp_mask & ~__GFP_DIRECT_RECLAIM, + nid, nodemask, false);if (page) {mpol_cond_put (mpol);return page;
Signature · 1 line
2.7.4
Michal Hocko
reviewerRe: [PATCH v6 4/6] mm/hugetlb: add support for mempolicy MPOL_PREFERRED_MANY
In reply to Mike KravetzQuoted 1 line · Mike Kravetz
On 7/12/21 1:09 AM, Feng Tang wrote:
[...]
Quoted 6 lines
mm/hugetlb.c struct page *alloc_buddy_huge_page_with_mpol(struct hstate *h, · line 2111
Quoted 3 lines
mm/hugetlb.c +3
-0+#ifdef CONFIG_NUMA +if (mpol->mode ==MPOL_PREFERRED_MANY ) { +gfp_t gfp = (gfp_mask | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM;I believe
__GFP_NOWARNwill be added later inalloc_buddy_huge_page, so no need to add here?
The mask is manipulated here anyway and the __GFP_NOWARN is really telling that there is no need to report the failure for _this_ allocation request. alloc_surplus_huge_page might alter that in whatever way in the future. So I would keep NOWARN here for the code clarity rather than rely on some implicit assumption down the path.
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 4/6] mm/hugetlb: add support for mempolicy MPOL_PREFERRED_MANY
Quoted 10 lines · Michal Hocko
Quoted 1 line · Mike Kravetz
On 7/12/21 1:09 AM, Feng Tang wrote:
[...]
Quoted 6 lines
mm/hugetlb.c struct page *alloc_buddy_huge_page_with_mpol(struct hstate *h, · line 2111
Quoted 3 lines
mm/hugetlb.c +3
-0+#ifdef CONFIG_NUMA +if (mpol->mode ==MPOL_PREFERRED_MANY ) { +gfp_t gfp = (gfp_mask | __GFP_NOWARN) & ~__GFP_DIRECT_RECLAIM;I believe
__GFP_NOWARNwill be added later inalloc_buddy_huge_page, so no need to add here?
Quoted 5 lines
The mask is manipulated here anyway and the
__GFP_NOWARNis really telling that there is no need to report the failure for _this_ allocation request.alloc_surplus_huge_pagemight alter that in whatever way in the future. So I would keep NOWARN here for the code clarity rather than rely on some implicit assumption down the path.
Makes sense. Better to leave the __GFP_NOWARN here for clarity.
Signature · 1 line
Mike Kravetz
PATCH v6 5/6 mm/mempolicy: Advertise new MPOL_PREFERRED_MANY
In reply to Feng TangFrom: Ben Widawsky <[email protected]>
Adds a new mode to the existing mempolicy modes, MPOL_PREFERRED_MANY.
MPOL_PREFERRED_MANY will be adequately documented in the internal admin-guide with this patch. Eventually, the man pages for mbind(2), get_mempolicy(2), set_mempolicy(2) and numactl(8) will also have text about this mode. Those shall contain the canonical reference.
NUMA systems continue to become more prevalent. New technologies like PMEM make finer grain control over memory access patterns increasingly desirable. MPOL_PREFERRED_MANY allows userspace to specify a set of nodes that will be tried first when performing allocations. If those allocations fail, all remaining nodes will be tried. It's a straight forward API which solves many of the presumptive needs of system administrators wanting to optimize workloads on such machines. The mode will work either per VMA, or per thread.
--- Documentation/admin-guide/mm/numa_memory_policy.rst | 16 ++++++++++++---- mm/mempolicy.c | 7 +------ 2 files changed, 13 insertions(+), 10 deletions(-)
Documentation/admin-guide/mm/numa_memory_policy.rst +12 -4
@@ -245,6 +245,14 @@ MPOL_INTERLEAVED address range or file. During system boot up, the temporary interleaved system default policy works in this mode. +MPOL_PREFERRED_MANY + This mode specifies that the allocation should be attempted from the + nodemask specified in the policy. If that allocation fails, the kernel + will search other nodes, in order of increasing distance from the first + set bit in the nodemask based on information provided by the platform + firmware. It is similar to MPOL_PREFERRED with the main exception that + is an error to have an empty nodemask. + NUMA memory policy supports the following optional mode flags: MPOL_F_STATIC_NODES
@@ -253,10 +261,10 @@ MPOL_F_STATIC_NODES nodes changes after the memory policy has been defined. Without this flag, any time a mempolicy is rebound because of a - change in the set of allowed nodes, the node (Preferred) or - nodemask (Bind, Interleave) is remapped to the new set of - allowed nodes. This may result in nodes being used that were - previously undesired. + change in the set of allowed nodes, the preferred nodemask (Preferred + Many), preferred node (Preferred) or nodemask (Bind, Interleave) is + remapped to the new set of allowed nodes. This may result in nodes + being used that were previously undesired. With this flag, if the user-specified nodes overlap with the nodes allowed by the task's cpuset, then the memory policy is
mm/mempolicy.c +1 -6
@@ -1463,12 +1463,7 @@ static inline int sanitize_mpol_flags(int *mode, unsigned short *flags) *flags = *mode &MPOL_MODE_FLAGS ; *mode &= ~MPOL_MODE_FLAGS ; -/* -* The check should be 'mode >= MPOL_MAX', but as 'prefer_many' -* is not fully implemented, don't permit it to be used for now, -* and the logic will be restored in following patch -*/ -if ((unsigned int )(*mode) >=MPOL_PREFERRED_MANY ) +if ((unsigned int )(*mode) >=MPOL_MAX )return -EINVAL ;if ((*flags &MPOL_F_STATIC_NODES ) && (*flags &MPOL_F_RELATIVE_NODES ))return -EINVAL ;
Signature · 1 line
2.7.4
Re: [PATCH v6 5/6] mm/mempolicy: Advertise new MPOL_PREFERRED_MANY
Quoted 31 lines · Feng Tang
From: Ben Widawsky <[email protected]>
Adds a new mode to the existing mempolicy modes,
MPOL_PREFERRED_MANY.
MPOL_PREFERRED_MANYwill be adequately documented in the internal admin-guide with this patch. Eventually, the man pages for mbind(2),get_mempolicy(2),set_mempolicy(2)and numactl(8) will also have text about this mode. Those shall contain the canonical reference.NUMA systems continue to become more prevalent. New technologies like PMEM make finer grain control over memory access patterns increasingly desirable.
MPOL_PREFERRED_MANYallows userspace to specify a set of nodes that will be tried first when performing allocations. If those allocations fail, all remaining nodes will be tried. It's a straight forward API which solves many of the presumptive needs of system administrators wanting to optimize workloads on such machines. The mode will work either per VMA, or per thread.Signed-off-by Ben Widawsky <[email protected]>Signed-off-by Feng Tang <[email protected]>--- Documentation/admin-guide/mm/numa_memory_policy.rst | 16 ++++++++++++---- mm/mempolicy.c | 7 +------ 2 files changed, 13 insertions(+), 10 deletions(-)Documentation/admin-guide/mm/numa_memory_policy.rst +0
-0
Documentation/admin-guide/mm/numa_memory_policy.rst MPOL_INTERLEAVED · line 254
Quoted 10 lines
Documentation/admin-guide/mm/numa_memory_policy.rst +7
-0@@ -245,6 +245,14 @@ MPOL_INTERLEAVED address range or file. During system boot up, the temporary interleaved system default policy works in this mode. +MPOL_PREFERRED_MANY + This mode specifies that the allocation should be attempted from the + nodemask specified in the policy. If that allocation fails, the kernel + will search other nodes, in order of increasing distance from the first + set bit in the nodemask based on information provided by the platform + firmware. It is similar to MPOL_PREFERRED with the main exception that + is an error to have an empty nodemask.
I believe the target audience of this documents are users rather than kernel developers and for those the wording might be rather cryptic. I would rephrase like this
This mode specifices that the allocation should be preferrably satisfied from the nodemask specified in the policy. If there is a memory pressure on all nodes in the nodemask the allocation can fall back to all existing numa nodes. This is effectively MPOL_PREFERRED allowed for a mask rather than a single node.
With that or similar feel free to add
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 5/6] mm/mempolicy: Advertise new MPOL_PREFERRED_MANY
Quoted 52 lines · Michal Hocko
Quoted 31 lines · Feng Tang
From: Ben Widawsky <[email protected]>
Adds a new mode to the existing mempolicy modes,
MPOL_PREFERRED_MANY.
MPOL_PREFERRED_MANYwill be adequately documented in the internal admin-guide with this patch. Eventually, the man pages for mbind(2),get_mempolicy(2),set_mempolicy(2)and numactl(8) will also have text about this mode. Those shall contain the canonical reference.NUMA systems continue to become more prevalent. New technologies like PMEM make finer grain control over memory access patterns increasingly desirable.
MPOL_PREFERRED_MANYallows userspace to specify a set of nodes that will be tried first when performing allocations. If those allocations fail, all remaining nodes will be tried. It's a straight forward API which solves many of the presumptive needs of system administrators wanting to optimize workloads on such machines. The mode will work either per VMA, or per thread.Signed-off-by Ben Widawsky <[email protected]>Signed-off-by Feng Tang <[email protected]>--- Documentation/admin-guide/mm/numa_memory_policy.rst | 16 ++++++++++++---- mm/mempolicy.c | 7 +------ 2 files changed, 13 insertions(+), 10 deletions(-)Documentation/admin-guide/mm/numa_memory_policy.rst +0
-0Documentation/admin-guide/mm/numa_memory_policy.rst MPOL_INTERLEAVED · line 254
Quoted 10 lines
Documentation/admin-guide/mm/numa_memory_policy.rst +7
-0@@ -245,6 +245,14 @@ MPOL_INTERLEAVED address range or file. During system boot up, the temporary interleaved system default policy works in this mode. +MPOL_PREFERRED_MANY + This mode specifies that the allocation should be attempted from the + nodemask specified in the policy. If that allocation fails, the kernel + will search other nodes, in order of increasing distance from the first + set bit in the nodemask based on information provided by the platform + firmware. It is similar to MPOL_PREFERRED with the main exception that + is an error to have an empty nodemask.I believe the target audience of this documents are users rather than kernel developers and for those the wording might be rather cryptic. I would rephrase like this
This mode specifices that the allocation should be preferrably satisfied from the nodemask specified in the policy. If there is a memory pressure on all nodes in the nodemask the allocation can fall back to all existing numa nodes. This is effectivelyMPOL_PREFERREDallowed for a mask rather than a single node.
Quoted 2 lines
With that or similar feel free to add
Acked-by Michal Hocko <[email protected]>
Thanks!
Will revise the test as suggested.
- Feng
Quoted 3 lines
Signature · 2 lines
Michal Hocko SUSE Labs
PATCH v6 6/6 mm/mempolicy: unify the create() func for bind/interleave/prefer-many policies
In reply to Feng TangAs they all do the same thing: sanity check and save nodemask info, create one mpol_new_nodemask() to reduce redundancy.
--- mm/mempolicy.c | 24 ++++-------------------- 1 file changed, 4 insertions(+), 20 deletions(-)
mm/mempolicy.c +4 -20
@@ -192,7 +192,7 @@ static void mpol_relative_nodemask(nodemask_t *ret, const nodemask_t *orig,nodes_onto (*ret, tmp, *rel); } -static int mpol_new_interleave (struct mempolicy *pol,const nodemask_t *nodes) +static int mpol_new_nodemask (struct mempolicy *pol,const nodemask_t *nodes) {if (nodes_empty (*nodes))return -EINVAL ;
@@ -210,22 +210,6 @@ static int mpol_new_preferred(struct mempolicy *pol, const nodemask_t *nodes)return 0 ; } -static int mpol_new_preferred_many (struct mempolicy *pol,const nodemask_t *nodes) -{ -if (nodes_empty (*nodes)) -return -EINVAL ; - pol->nodes = *nodes; -return 0 ; -} - -static int mpol_new_bind (struct mempolicy *pol,const nodemask_t *nodes) -{ -if (nodes_empty (*nodes)) -return -EINVAL ; - pol->nodes = *nodes; -return 0 ; -} -/* * mpol_set_nodemask is called after mpol_new() to set up the nodemask, if * any, for the new policy. mpol_new() has already validated the nodes
@@ -405,7 +389,7 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = { .rebind = mpol_rebind_default, }, [MPOL_INTERLEAVE] = { - .create = mpol_new_interleave, + .create = mpol_new_nodemask, .rebind = mpol_rebind_nodemask, }, [MPOL_PREFERRED] = {
@@ -413,14 +397,14 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = { .rebind = mpol_rebind_preferred, }, [MPOL_BIND] = { - .create = mpol_new_bind, + .create = mpol_new_nodemask, .rebind = mpol_rebind_nodemask, }, [MPOL_LOCAL] = { .rebind = mpol_rebind_default, }, [MPOL_PREFERRED_MANY] = { - .create = mpol_new_preferred_many, + .create = mpol_new_nodemask, .rebind = mpol_rebind_preferred, }, };
Signature · 1 line
2.7.4
Re: [PATCH v6 6/6] mm/mempolicy: unify the create() func for bind/interleave/prefer-many policies
Quoted 2 lines · Feng Tang
As they all do the same thing: sanity check and save nodemask info, create one
mpol_new_nodemask()to reduce redundancy.
Do we really need a create() callback these days?
Quoted 1 line
Signed-off-by Feng Tang <[email protected]>
Other than that LGTM
Quoted 68 lines
--- mm/mempolicy.c | 24 ++++-------------------- 1 file changed, 4 insertions(+), 20 deletions(-)mm/mempolicy.c +4
-20@@ -192,7 +192,7 @@ static void mpol_relative_nodemask(nodemask_t *ret, const nodemask_t *orig,nodes_onto (*ret, tmp, *rel); } -static int mpol_new_interleave (struct mempolicy *pol,const nodemask_t *nodes) +static int mpol_new_nodemask (struct mempolicy *pol,const nodemask_t *nodes) {if (nodes_empty (*nodes))return -EINVAL ;@@ -210,22 +210,6 @@ static int mpol_new_preferred(struct mempolicy *pol, const nodemask_t *nodes)return 0 ; } -static int mpol_new_preferred_many (struct mempolicy *pol,const nodemask_t *nodes) -{ -if (nodes_empty (*nodes)) -return -EINVAL ; - pol->nodes = *nodes; -return 0 ; -} - -static int mpol_new_bind (struct mempolicy *pol,const nodemask_t *nodes) -{ -if (nodes_empty (*nodes)) -return -EINVAL ; - pol->nodes = *nodes; -return 0 ; -} -/* * mpol_set_nodemask is called after mpol_new() to set up the nodemask, if * any, for the new policy. mpol_new() has already validated the nodes @@ -405,7 +389,7 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = { .rebind = mpol_rebind_default, }, [MPOL_INTERLEAVE] = { - .create = mpol_new_interleave, + .create = mpol_new_nodemask, .rebind = mpol_rebind_nodemask, }, [MPOL_PREFERRED] = {@@ -413,14 +397,14 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = { .rebind = mpol_rebind_preferred, }, [MPOL_BIND] = { - .create = mpol_new_bind, + .create = mpol_new_nodemask, .rebind = mpol_rebind_nodemask, }, [MPOL_LOCAL] = { .rebind = mpol_rebind_default, }, [MPOL_PREFERRED_MANY] = { - .create = mpol_new_preferred_many, + .create = mpol_new_nodemask, .rebind = mpol_rebind_preferred, }, };Signature · 1 line
2.7.4
Signature · 2 lines
Michal Hocko SUSE Labs
Re: [PATCH v6 6/6] mm/mempolicy: unify the create() func for bind/interleave/prefer-many policies
Quoted 5 lines · Michal Hocko
Quoted 2 lines · Feng Tang
As they all do the same thing: sanity check and save nodemask info, create one
mpol_new_nodemask()to reduce redundancy.Do we really need a
create()callback these days?
I think it tries to provide a per-policy sanity check (though it's the same for all existing ones), and a per-policy nodemask setting (current 'prefer' policy is different from others).
Quoted 4 lines
Quoted 1 line
Signed-off-by Feng Tang <[email protected]>Other than that LGTM
Acked-by Michal Hocko <[email protected]>
Thanks!
- Feng
Quoted 72 lines
Quoted 68 lines
--- mm/mempolicy.c | 24 ++++-------------------- 1 file changed, 4 insertions(+), 20 deletions(-)mm/mempolicy.c +4
-20@@ -192,7 +192,7 @@ static void mpol_relative_nodemask(nodemask_t *ret, const nodemask_t *orig,nodes_onto (*ret, tmp, *rel); } -static int mpol_new_interleave (struct mempolicy *pol,const nodemask_t *nodes) +static int mpol_new_nodemask (struct mempolicy *pol,const nodemask_t *nodes) {if (nodes_empty (*nodes))return -EINVAL ;@@ -210,22 +210,6 @@ static int mpol_new_preferred(struct mempolicy *pol, const nodemask_t *nodes)return 0 ; } -static int mpol_new_preferred_many (struct mempolicy *pol,const nodemask_t *nodes) -{ -if (nodes_empty (*nodes)) -return -EINVAL ; - pol->nodes = *nodes; -return 0 ; -} - -static int mpol_new_bind (struct mempolicy *pol,const nodemask_t *nodes) -{ -if (nodes_empty (*nodes)) -return -EINVAL ; - pol->nodes = *nodes; -return 0 ; -} -/* * mpol_set_nodemask is called after mpol_new() to set up the nodemask, if * any, for the new policy. mpol_new() has already validated the nodes @@ -405,7 +389,7 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = { .rebind = mpol_rebind_default, }, [MPOL_INTERLEAVE] = { - .create = mpol_new_interleave, + .create = mpol_new_nodemask, .rebind = mpol_rebind_nodemask, }, [MPOL_PREFERRED] = {@@ -413,14 +397,14 @@ static const struct mempolicy_operations mpol_ops[MPOL_MAX] = { .rebind = mpol_rebind_preferred, }, [MPOL_BIND] = { - .create = mpol_new_bind, + .create = mpol_new_nodemask, .rebind = mpol_rebind_nodemask, }, [MPOL_LOCAL] = { .rebind = mpol_rebind_default, }, [MPOL_PREFERRED_MANY] = { - .create = mpol_new_preferred_many, + .create = mpol_new_nodemask, .rebind = mpol_rebind_preferred, }, };Signature · 1 line
2.7.4Signature · 2 lines
Michal Hocko SUSE Labs
Andrew Morton
maintainerQuoted 7 lines · On Mon, 12 Jul 2021 16:09:28 +0800 Feng Tang <[email protected]> wrote:
This patch series introduces the concept of the
MPOL_PREFERRED_MANYmempolicy. This mempolicy mode can be used with either theset_mempolicy(2)or mbind(2) interfaces. Like theMPOL_PREFERREDinterface, it allows an application to set a preference for nodes which will fulfil memory allocation requests. Unlike theMPOL_PREFERREDmode, it takes a set of nodes. Like theMPOL_BINDinterface, it works over a set of nodes. UnlikeMPOL_BIND, it will not cause a SIGSEGV or invoke the OOM killer if those preferred nodes are not available.
Do we have any real-world testing which demonstrates the benefits of all of this?
Hi Andrew,
Thanks for reviewing!
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Mon, 12 Jul 2021 16:09:28 +0800 Feng Tang <[email protected]> wrote:
This patch series introduces the concept of the
MPOL_PREFERRED_MANYmempolicy. This mempolicy mode can be used with either theset_mempolicy(2)or mbind(2) interfaces. Like theMPOL_PREFERREDinterface, it allows an application to set a preference for nodes which will fulfil memory allocation requests. Unlike theMPOL_PREFERREDmode, it takes a set of nodes. Like theMPOL_BINDinterface, it works over a set of nodes. UnlikeMPOL_BIND, it will not cause a SIGSEGV or invoke the OOM killer if those preferred nodes are not available.
Quoted 2 lines
Do we have any real-world testing which demonstrates the benefits of all of this?
We have done some internal tests, and are actively working with some external customer on using this new 'prefer-many' policy, as they have different types of memory (fast DRAM and slower Persistent memory) in system, and their program wants to set clear preference for several NUMA nodes, to better deploy the huge application data before running the application.
We have met another issue that customer wanted to run a docker container while binding it to 2 persistent memory nodes, which always failed. At that time we tried 2 hack pachtes to solve it.
https://lore.kernel.org/lkml/[email protected]/
https://lore.kernel.org/lkml/[email protected]/
And that use case can be easily achieved with this new policy.
Thanks,
Feng
Quoted 10 lines · Andrew Morton
Quoted 7 lines · On Mon, 12 Jul 2021 16:09:28 +0800 Feng Tang <[email protected]> wrote:
This patch series introduces the concept of the
MPOL_PREFERRED_MANYmempolicy. This mempolicy mode can be used with either theset_mempolicy(2)or mbind(2) interfaces. Like theMPOL_PREFERREDinterface, it allows an application to set a preference for nodes which will fulfil memory allocation requests. Unlike theMPOL_PREFERREDmode, it takes a set of nodes. Like theMPOL_BINDinterface, it works over a set of nodes. UnlikeMPOL_BIND, it will not cause a SIGSEGV or invoke the OOM killer if those preferred nodes are not available.Do we have any real-world testing which demonstrates the benefits of all of this?
Yes, it's actually been quite useful in practice already.
If we take persistent memory media (PMEM) and hot-add/online it with the DAX kmem driver, we get NUMA nodes with lots of capacity (~6TB is typical) but weird performance; PMEM has good read speed, but low write speed.
That low write speed is *so* low that it dominates the performance more than the distance from the CPUs. Folks who want PMEM really don't care about locality. The discussions with the testers usually go something like this:
Tester: How do I make my test use PMEM on nodes 2 and 3? Kernel Guys: use 'numactl --membind=2-3'
Tester: I tried that, but I'm getting allocation failures once I fill up
PMEM. Shouldn't it fall back to DRAM?
Kernel Guys: Fine, use 'numactl --preferred=2-3'
Tester: That worked, but it started using DRAM after it exhausted node 2 Kernel Guys: Dang it. I forgot --preferred ignores everything after
the first node. Fine, we'll patch the kernel.
This has happened more than once. End users want to be able to specify a specific physical media, but don't want to have to deal with the sharp edges of strict binding.
This has happened both with slow media like PMEM and "faster" media like High-Bandwidth Memory.