Dynamic hugetlb pool resizing

In many places throughout the kernel, the expression (HPAGE_SIZE/PAGE_SIZE) is used to convert quantities in huge page units to a number of base pages. Reduce redundancy and make the code more readable by introducing a constant BASE_PAGES_PER_HPAGE whose name more clearly conveys the intended conversion.

Quoted 5 lines · On Fri, 13 Jul 2007 08:16:31 -0700 Adam Litke <[email protected]> wrote:

In many places throughout the kernel, the expression (HPAGE_SIZE/PAGE_SIZE) is used to convert quantities in huge page units to a number of base pages. Reduce redundancy and make the code more readable by introducing a constant BASE_PAGES_PER_HPAGE whose name more clearly conveys the intended conversion.

It may be better to put in a generic way of determining the pages of a compound page.

Usually

1 << compound_order(page) will do the trick.

mm/hugetlb.c int hugetlb_reserve_pages(struct inode *inode, long from, long to) · line 1006

Quoted 9 lines · Adam

mm/hugetlb.c +7 -0

+	/*
+	 * I haven't figured out how to incorporate this cpuset bodge into
+	 * the dynamic hugetlb pool yet.  Hopefully someone more familiar with
+	 * cpusets can weigh in on their desired semantics.  Maybe we can just
+	 * drop this check?
+	 *
 	if (chg > cpuset_mems_nr(free_huge_pages_node))
 		return -ENOMEM;
+	 */

I can't figure out the value of this check either -- Ken Chen added it, perhaps he can comment.

But the cpuset behaviour of this hugetlb stuff looks suspicious to me:
1) The code in alloc_fresh_huge_page() seems to round robin over the entire system, spreading the hugetlb pages uniformly on all nodes. If one a task in one small cpuset starts aggressively allocating hugetlb pages, do you think this will work, Adam -- looks to me like we will end up calling alloc_fresh_huge_page() many times, most of which will fail to alloc_pages_node() anything because the 'static nid' clock hand will be pointing at a node outside of the current tasks cpuset (not in that tasks mems_allowed). Inefficient, but I guess ok.

2) I don't see what keeps us from picking hugetlb pages off -any- node in the system, perhaps way outside the current cpuset. We shouldn't be looking for enough available (free_huge_pages - resv_huge_pages) pages in the whole system. Rather we should be looking for and reserving enough such pages that are in the current tasks cpuset (set in its mems_allowed, to be precise) Folks aren't going to want their hugetlb pages coming from outside their tasks cpuset.

3) If there is some code I missed (good chance) that enforces the rule that a task can only get a hugetlb page from a node in its cpuset, then this uniform global allocation of hugetlb pages, as noted in (1) above, can't be right. Either it will force all nodes, including many nodes outside of the current tasks cpuset, to bulk up on free hugetlb pages, just to get enough of them on nodes allowed by the current tasks cpuset, or else it will fail to get enough on nodes local to the current tasks cpuset. I don't understand the logic well enough to know which, but either way sucks.

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period.

Quoted 9 lines

But the cpuset behaviour of this hugetlb stuff looks suspicious to me:
1) The code in alloc_fresh_huge_page() seems to round robin over the entire system, spreading the hugetlb pages uniformly on all nodes. If one a task in one small cpuset starts aggressively allocating hugetlb pages, do you think this will work, Adam -- looks to me like we will end up calling alloc_fresh_huge_page() many times, most of which will fail to alloc_pages_node() anything because the 'static nid' clock hand will be pointing at a node outside of the current tasks cpuset (not in that tasks mems_allowed). Inefficient, but I guess ok.

Very good point. I guess we call alloc_fresh_huge_page in two scenarios now... 1) By echoing a number into /proc/sys/vm/nr_hugepages, and 2) by trying to dynamically increase the pool size for a particular process. Case 1 is not in the context of any process (per se) and so node_online_map makes sense. For case 2 we could teach the __alloc_fresh_huge_page() to take a nodemask. That could get nasty though since we'd have to move away from a static variable to get proper interleaving.

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period. I wonder if moving to the mempool interface and

Hmm, I see what you mean, but cpusets are already broken because we use the global resv_huge_pages counter. I realize that's what the cpuset_mems_nr() thing was meant to address but it's not correct.

Quoted 2 lines · Adam Litke

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period. I wonder if moving to the mempool interface and

Sorry no. We always had per node pools. There is no need to have per cpuset pools.

Quoted 3 lines

Hmm, I see what you mean, but cpusets are already broken because we use the global resv_huge_pages counter. I realize that's what the cpuset_mems_nr() thing was meant to address but it's not correct.

Well the global reserve counter causes a big reduction in performance since it requires the serialization of the hugetlb faults. Could we please get this straigthened out? This serialization somehow snuck in when I was not looking and it screws up multiple things.

Quoted 3 lines

since it requires the serialization of the hugetlb faults. Could we please get this straigthened out? This serialization somehow snuck in when I was not looking and it screws up multiple things.

Sadly, global serialization has some nice property. It is now used in three paths that I'm aware of:
(1) shared mapping reservation count
(2) linked list protection in unmap_hugepage_range
(3) shared page table on hugetlb mapping.

i suppose (2) and (3) can be moved into per-inode lock?

But we need per cpuset reservation to preserve current hugetlb semantics on shared mapping.

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period.

Quoted 2 lines · Adam

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period.

It's not an easy fit, that's for sure ;).

Quoted 5 lines · Paul Jackson

Quoted 2 lines · Adam

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period.

It's not an easy fit, that's for sure ;).

In the context of my patches to make the hugetlb pool's interleave work with memoryless nodes, I may have pseudo-solution for growing the pool while respecting cpusets.

Essentially, given that GFP_THISNODE allocations stay on the node requested (which is the case after Christoph's set of memoryless node patches go in), we invoke:

pol = mpol_new(MPOL_INTERLEAVE, &node_states[N_MEMORY])

in the two callers of alloc_fresh_huge_page(pol) in hugetlb.c. alloc_fresh_huge_page() in turn invokes interleave_nodes(pol) so that we request hugepages in an interleaved fashion over all nodes with memory.

Now, what I'm wondering is why interleave_nodes() is not cpuset aware? Or is it expected that the caller do the right thing with the policy beforehand? If so, I think I could just make those two callers do

pol = mpol_new(MPOL_INTERLEAVE, cpuset_mems_allowed(current))

?

Or am I way off here?

Quoted 26 lines · Nish Aravamudan

Quoted 5 lines · Paul Jackson

Quoted 2 lines · Adam

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period.

It's not an easy fit, that's for sure ;).

In the context of my patches to make the hugetlb pool's interleave work with memoryless nodes, I may have pseudo-solution for growing the pool while respecting cpusets.

Essentially, given that GFP_THISNODE allocations stay on the node requested (which is the case after Christoph's set of memoryless node patches go in), we invoke:

pol = mpol_new(MPOL_INTERLEAVE, &node_states[N_MEMORY])

in the two callers of alloc_fresh_huge_page(pol) in hugetlb.c. alloc_fresh_huge_page() in turn invokes interleave_nodes(pol) so that we request hugepages in an interleaved fashion over all nodes with memory.

Now, what I'm wondering is why interleave_nodes() is not cpuset aware? Or is it expected that the caller do the right thing with the policy beforehand? If so, I think I could just make those two callers do

Quoted 5 lines

pol = mpol_new(MPOL_INTERLEAVE, cpuset_mems_allowed(current))

?

Or am I way off here?

Nish:

I have always considered the huge page pool, as populated by alloc_fresh_huge_page() in response to changes in nr_hugepages, to be a system global resource. I think the system "does the right thing"--well, almost--with Christoph's memoryless patches and your hugetlb patches. Certaintly, the huge pages allocated at boot time, based on the command line parameter, are system-wide. cpusets have not been set up at that time.

It requires privilege to write to the nr_hugepages sysctl, so allowing it to spread pages across all available nodes [with memory], regardless of cpusets, makes sense to me. Altho' I don't expect many folks are currently changing nr_hugepages from within a constrained cpuset, I wouldn't want to see us change existing behavior, in this respect. Your per node attributes will provide the mechanism to allocate different numbers of hugepages for, e.g., nodes in cpusets that have applications that need them.

Re: the "well, almost": nr_hugepages is still "broken" for me on some of my platforms where the interleaved, dma-only pseudo-node contains sufficient memory to satisfy a hugepage request. I'll end up with a few hugepages consuming most of the dma memory. Consuming the dma isn't the issue--there should be enough remaining for any dma needs. I just want more control over what gets placed on the interleaved pseudo-node by default. I think that Paul Mundt [added to cc list] has similar concerns about default policies on the sh platforms. I have some ideas, but I'm waiting for the memoryless nodes and your patches to stabilize in the mm tree.

Quoted 36 lines · Lee Schermerhorn

Quoted 26 lines · Nish Aravamudan

Quoted 5 lines · Paul Jackson

Quoted 2 lines · Adam

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period.

It's not an easy fit, that's for sure ;).

In the context of my patches to make the hugetlb pool's interleave work with memoryless nodes, I may have pseudo-solution for growing the pool while respecting cpusets.

Essentially, given that GFP_THISNODE allocations stay on the node requested (which is the case after Christoph's set of memoryless node patches go in), we invoke:

pol = mpol_new(MPOL_INTERLEAVE, &node_states[N_MEMORY])

in the two callers of alloc_fresh_huge_page(pol) in hugetlb.c. alloc_fresh_huge_page() in turn invokes interleave_nodes(pol) so that we request hugepages in an interleaved fashion over all nodes with memory.

Now, what I'm wondering is why interleave_nodes() is not cpuset aware? Or is it expected that the caller do the right thing with the policy beforehand? If so, I think I could just make those two callers do

Quoted 5 lines

pol = mpol_new(MPOL_INTERLEAVE, cpuset_mems_allowed(current))

?

Or am I way off here?

Nish:

Quoted 7 lines

I have always considered the huge page pool, as populated by alloc_fresh_huge_page() in response to changes in nr_hugepages, to be a system global resource. I think the system "does the right thing"--well, almost--with Christoph's memoryless patches and your hugetlb patches. Certaintly, the huge pages allocated at boot time, based on the command line parameter, are system-wide. cpusets have not been set up at that time.

I fully agree that hugepages are a global resource.

Quoted 8 lines

It requires privilege to write to the nr_hugepages sysctl, so allowing it to spread pages across all available nodes [with memory], regardless of cpusets, makes sense to me. Altho' I don't expect many folks are currently changing nr_hugepages from within a constrained cpuset, I wouldn't want to see us change existing behavior, in this respect. Your per node attributes will provide the mechanism to allocate different numbers of hugepages for, e.g., nodes in cpusets that have applications that need them.

The issue is that with Adam's patches, the hugepage pool will grow on demand, presuming the process owner's mlock limit is sufficiently high. If said process were running within a constrained cpuset, it seems slightly out-of-whack to allow it grow the pool on other nodes to satisfy the demand.

Quoted 11 lines

Re: the "well, almost": nr_hugepages is still "broken" for me on some of my platforms where the interleaved, dma-only pseudo-node contains sufficient memory to satisfy a hugepage request. I'll end up with a few hugepages consuming most of the dma memory. Consuming the dma isn't the issue--there should be enough remaining for any dma needs. I just want more control over what gets placed on the interleaved pseudo-node by default. I think that Paul Mundt [added to cc list] has similar concerns about default policies on the sh platforms. I have some ideas, but I'm waiting for the memoryless nodes and your patches to stabilize in the mm tree.

And well, we're already 'broken' as far as I can tell with cpusets and the hugepage pool. I'm just trying to decide if it's fixable as is, or if we need extra cleverness. A simple hack would be to just modify the interleave call with a callback that uses the appropriate mask if CPUSETS is on or off (I don't want to always use cpuset_mems_allowed() unconditionally, becuase it returns node_possible_map if !CPUSETS.

Thanks for the feedback. If folks are ok with the way things are, then so be it. I was just hoping Paul might have some thoughts on how best to avoid violating cpuset constraints with Adam's patches in the context of my patches.

Quoted 56 lines · Nish Aravamudan

Quoted 36 lines · Lee Schermerhorn

Quoted 26 lines · Nish Aravamudan

Quoted 5 lines · Paul Jackson

Quoted 2 lines · Adam

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period.

It's not an easy fit, that's for sure ;).

In the context of my patches to make the hugetlb pool's interleave work with memoryless nodes, I may have pseudo-solution for growing the pool while respecting cpusets.

Essentially, given that GFP_THISNODE allocations stay on the node requested (which is the case after Christoph's set of memoryless node patches go in), we invoke:

pol = mpol_new(MPOL_INTERLEAVE, &node_states[N_MEMORY])

in the two callers of alloc_fresh_huge_page(pol) in hugetlb.c. alloc_fresh_huge_page() in turn invokes interleave_nodes(pol) so that we request hugepages in an interleaved fashion over all nodes with memory.

Now, what I'm wondering is why interleave_nodes() is not cpuset aware? Or is it expected that the caller do the right thing with the policy beforehand? If so, I think I could just make those two callers do

Quoted 5 lines

pol = mpol_new(MPOL_INTERLEAVE, cpuset_mems_allowed(current))

?

Or am I way off here?

Nish:

Quoted 7 lines

I have always considered the huge page pool, as populated by alloc_fresh_huge_page() in response to changes in nr_hugepages, to be a system global resource. I think the system "does the right thing"--well, almost--with Christoph's memoryless patches and your hugetlb patches. Certaintly, the huge pages allocated at boot time, based on the command line parameter, are system-wide. cpusets have not been set up at that time.

I fully agree that hugepages are a global resource.

Quoted 8 lines

It requires privilege to write to the nr_hugepages sysctl, so allowing it to spread pages across all available nodes [with memory], regardless of cpusets, makes sense to me. Altho' I don't expect many folks are currently changing nr_hugepages from within a constrained cpuset, I wouldn't want to see us change existing behavior, in this respect. Your per node attributes will provide the mechanism to allocate different numbers of hugepages for, e.g., nodes in cpusets that have applications that need them.

Quoted 5 lines

The issue is that with Adam's patches, the hugepage pool will grow on demand, presuming the process owner's mlock limit is sufficiently high. If said process were running within a constrained cpuset, it seems slightly out-of-whack to allow it grow the pool on other nodes to satisfy the demand.

Ah, I see. In that case, it might make sense to grow just for the cpuset. A couple of things come to mind tho':

1) we might want a per cpuset control to enable/disable hugetlb pool growth on demand, or to limit the max size of the pool--especially if the memories are not exclusively owned by the cpuset. Otherwise, non-privileged processes could grow the hugetlb pool in memories shared with other cpusets [maybe the root cpuset?], thereby reducing the amount of normal, managed pages available to the other cpusets. Probably want such a control in the absense of cpusets as well, if on-demand hugetlb pool growth is implemented.

2) per cpuset, on-demand hugetlb pool growth shouldn't affect the behavior of the nr_hugepages sysctl--IMO, anyway.

3) managed "superpages" keeps sounding better and better ;-)

Quoted 11 lines

Quoted 10 lines

Re: the "well, almost": nr_hugepages is still "broken" for me on some of my platforms where the interleaved, dma-only pseudo-node contains sufficient memory to satisfy a hugepage request. I'll end up with a few hugepages consuming most of the dma memory. Consuming the dma isn't the issue--there should be enough remaining for any dma needs. I just want more control over what gets placed on the interleaved pseudo-node by default. I think that Paul Mundt [added to cc list] has similar concerns about default policies on the sh platforms. I have some ideas, but I'm waiting for the memoryless nodes and your patches to stabilize in the mm tree.

Quoted 6 lines

And well, we're already 'broken' as far as I can tell with cpusets and the hugepage pool. I'm just trying to decide if it's fixable as is, or if we need extra cleverness. A simple hack would be to just modify the interleave call with a callback that uses the appropriate mask if CPUSETS is on or off (I don't want to always use cpuset_mems_allowed() unconditionally, becuase it returns node_possible_map if !CPUSETS.

Maybe you want/need a cpuset_hugemems_allowed() that does "the right thing" with and without cpusets?

Quoted 4 lines

Thanks for the feedback. If folks are ok with the way things are, then so be it. I was just hoping Paul might have some thoughts on how best to avoid violating cpuset constraints with Adam's patches in the context of my patches.

I'm not trying to discourage you, here. I agree that cpusets, as useful as I find them, do make things, uh, "interesting"--especially with shared resources.

Quoted 66 lines · Lee Schermerhorn

Quoted 56 lines · Nish Aravamudan

Quoted 36 lines · Lee Schermerhorn

Quoted 26 lines · Nish Aravamudan

Quoted 5 lines · Paul Jackson

Quoted 2 lines · Adam

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period.

It's not an easy fit, that's for sure ;).

In the context of my patches to make the hugetlb pool's interleave work with memoryless nodes, I may have pseudo-solution for growing the pool while respecting cpusets.

Essentially, given that GFP_THISNODE allocations stay on the node requested (which is the case after Christoph's set of memoryless node patches go in), we invoke:

pol = mpol_new(MPOL_INTERLEAVE, &node_states[N_MEMORY])

in the two callers of alloc_fresh_huge_page(pol) in hugetlb.c. alloc_fresh_huge_page() in turn invokes interleave_nodes(pol) so that we request hugepages in an interleaved fashion over all nodes with memory.

Now, what I'm wondering is why interleave_nodes() is not cpuset aware? Or is it expected that the caller do the right thing with the policy beforehand? If so, I think I could just make those two callers do

Quoted 5 lines

pol = mpol_new(MPOL_INTERLEAVE, cpuset_mems_allowed(current))

?

Or am I way off here?

Nish:

Quoted 7 lines

I have always considered the huge page pool, as populated by alloc_fresh_huge_page() in response to changes in nr_hugepages, to be a system global resource. I think the system "does the right thing"--well, almost--with Christoph's memoryless patches and your hugetlb patches. Certaintly, the huge pages allocated at boot time, based on the command line parameter, are system-wide. cpusets have not been set up at that time.

I fully agree that hugepages are a global resource.

Quoted 8 lines

It requires privilege to write to the nr_hugepages sysctl, so allowing it to spread pages across all available nodes [with memory], regardless of cpusets, makes sense to me. Altho' I don't expect many folks are currently changing nr_hugepages from within a constrained cpuset, I wouldn't want to see us change existing behavior, in this respect. Your per node attributes will provide the mechanism to allocate different numbers of hugepages for, e.g., nodes in cpusets that have applications that need them.

Quoted 5 lines

The issue is that with Adam's patches, the hugepage pool will grow on demand, presuming the process owner's mlock limit is sufficiently high. If said process were running within a constrained cpuset, it seems slightly out-of-whack to allow it grow the pool on other nodes to satisfy the demand.

Ah, I see. In that case, it might make sense to grow just for the cpuset. A couple of things come to mind tho':

Quoted 8 lines

1) we might want a per cpuset control to enable/disable hugetlb pool growth on demand, or to limit the max size of the pool--especially if the memories are not exclusively owned by the cpuset. Otherwise, non-privileged processes could grow the hugetlb pool in memories shared with other cpusets [maybe the root cpuset?], thereby reducing the amount of normal, managed pages available to the other cpusets. Probably want such a control in the absense of cpusets as well, if on-demand hugetlb pool growth is implemented.

Well, the current restriction is on a per-process basis for locked memory. But it might make sense to add a separate rlimit for hugepages and then just allow cpusets to restrict that rlimit for processes contained therein?

Similar would probably hold for the non-cpuset case?

But that seems like special casing for hugetlb pages where small pages don't have the same restriction. If two cpusets share the same node, can't one exhaust the node and thus starve the other cpuset? At that point you need more than cpusets (arguably) and want resource management at some level.

Quoted 20 lines · Nish Aravamudan

Quoted 7 lines · Lee Schermerhorn

I have always considered the huge page pool, as populated by alloc_fresh_huge_page() in response to changes in nr_hugepages, to be a system global resource. I think the system "does the right thing"--well, almost--with Christoph's memoryless patches and your hugetlb patches. Certaintly, the huge pages allocated at boot time, based on the command line parameter, are system-wide. cpusets have not been set up at that time.

I fully agree that hugepages are a global resource.

Quoted 8 lines

It requires privilege to write to the nr_hugepages sysctl, so allowing it to spread pages across all available nodes [with memory], regardless of cpusets, makes sense to me. Altho' I don't expect many folks are currently changing nr_hugepages from within a constrained cpuset, I wouldn't want to see us change existing behavior, in this respect. Your per node attributes will provide the mechanism to allocate different numbers of hugepages for, e.g., nodes in cpusets that have applications that need them.

Quoted 5 lines

The issue is that with Adam's patches, the hugepage pool will grow on demand, presuming the process owner's mlock limit is sufficiently high. If said process were running within a constrained cpuset, it seems slightly out-of-whack to allow it grow the pool on other nodes to satisfy the demand.

Ah, I see. In that case, it might make sense to grow just for the cpuset. A couple of things come to mind tho':

1) we might want a per cpuset control to enable/disable hugetlb pool growth on demand, or to limit the max size of the pool--especially if the memories are not exclusively owned by the cpuset. Otherwise, non-privileged processes could grow the hugetlb pool in memories shared with other cpusets [maybe the root cpuset?], thereby reducing the amount of normal, managed pages available to the other cpusets. Probably want such a control in the absense of cpusets as well, if on-demand hugetlb pool growth is implemented.

Quoted 30 lines · Lee Schermerhorn

Quoted 20 lines · Nish Aravamudan

Quoted 7 lines · Lee Schermerhorn

I have always considered the huge page pool, as populated by alloc_fresh_huge_page() in response to changes in nr_hugepages, to be a system global resource. I think the system "does the right thing"--well, almost--with Christoph's memoryless patches and your hugetlb patches. Certaintly, the huge pages allocated at boot time, based on the command line parameter, are system-wide. cpusets have not been set up at that time.

I fully agree that hugepages are a global resource.

Quoted 8 lines

It requires privilege to write to the nr_hugepages sysctl, so allowing it to spread pages across all available nodes [with memory], regardless of cpusets, makes sense to me. Altho' I don't expect many folks are currently changing nr_hugepages from within a constrained cpuset, I wouldn't want to see us change existing behavior, in this respect. Your per node attributes will provide the mechanism to allocate different numbers of hugepages for, e.g., nodes in cpusets that have applications that need them.

Quoted 5 lines

The issue is that with Adam's patches, the hugepage pool will grow on demand, presuming the process owner's mlock limit is sufficiently high. If said process were running within a constrained cpuset, it seems slightly out-of-whack to allow it grow the pool on other nodes to satisfy the demand.

Ah, I see. In that case, it might make sense to grow just for the cpuset. A couple of things come to mind tho':

Quoted 9 lines

1) we might want a per cpuset control to enable/disable hugetlb pool growth on demand, or to limit the max size of the pool--especially if the memories are not exclusively owned by the cpuset. Otherwise, non-privileged processes could grow the hugetlb pool in memories shared with other cpusets [maybe the root cpuset?], thereby reducing the amount of normal, managed pages available to the other cpusets. Probably want such a control in the absense of cpusets as well, if on-demand hugetlb pool growth is implemented.

I don't see that the two are mutually exclusive. Hugetlb pools have to be node-local anyways due to the varying distances, so perhaps the global resource thing is the wrong way to approach it. There are already hooks for spreading slab and page cache pages in cpusets, perhaps it makes sense to add a hugepage spread variant to balance across the constrained set?

nr_hugepages is likely something that should still be global, so the sum of the hugepages in the per-node pools don't exceed this value.

It would be quite nice to have some way to have nodes opt-in to the sort of behaviour they're willing to tolerate. Some nodes are never going to tolerate spreading of any sort, hugepages, and so forth. Perhaps it makes more sense to have some flags in the pgdat where we can more strongly type the sort of behaviour the node is willing to put up with (or capable of supporting), at least in this case the nodes that explicitly can't cope are factored out before we even get to cpuset constraints (plus this gives us a hook for setting up the interleave nodes in both the system init and default policies). Thoughts?

On 7/18/07, Paul Mundt <[email protected]> wrote:

Quoted 9 lines

It would be quite nice to have some way to have nodes opt-in to the sort of behaviour they're willing to tolerate. Some nodes are never going to tolerate spreading of any sort, hugepages, and so forth. Perhaps it makes more sense to have some flags in the pgdat where we can more strongly type the sort of behaviour the node is willing to put up with (or capable of supporting), at least in this case the nodes that explicitly can't cope are factored out before we even get to cpuset constraints (plus this gives us a hook for setting up the interleave nodes in both the system init and default policies). Thoughts?

I guess I don't understand which nodes you're talking about now? How do you spread across any particular single node (how I read "Some nodes are never going to tolerate spreading of any sort")? Or do you mean that some cpusets aren't going to want to spread (interleave?).

Oh, are you trying to say that some nodes should be dropped from interleave masks (explicitly excluded from all possible interleave masks)? What kind of nodes would these be? We're doing something similar to deal with memoryless nodes, perhaps it could be generalized?

Quoted 1 line · Nish Aravamudan

On 7/18/07, Paul Mundt <[email protected]> wrote:

<snip>

Quoted 15 lines

Quoted 9 lines

It would be quite nice to have some way to have nodes opt-in to the sort of behaviour they're willing to tolerate. Some nodes are never going to tolerate spreading of any sort, hugepages, and so forth. Perhaps it makes more sense to have some flags in the pgdat where we can more strongly type the sort of behaviour the node is willing to put up with (or capable of supporting), at least in this case the nodes that explicitly can't cope are factored out before we even get to cpuset constraints (plus this gives us a hook for setting up the interleave nodes in both the system init and default policies). Thoughts?

I guess I don't understand which nodes you're talking about now? How do you spread across any particular single node (how I read "Some nodes are never going to tolerate spreading of any sort")? Or do you mean that some cpusets aren't going to want to spread (interleave?).

Quoted 5 lines

Oh, are you trying to say that some nodes should be dropped from interleave masks (explicitly excluded from all possible interleave masks)? What kind of nodes would these be? We're doing something similar to deal with memoryless nodes, perhaps it could be generalized?

If that's what Paul means [and I think it is, based on a converstation at OLS], I have a similar requirement. I'd like to be able to specify, on the command line, at least [run time reconfig not a hard requirement] nodes to be excluded from interleave masks, including the hugetlb allocation mask [if this is different from the regular interleaving nodemask].

And, I agree, I think we can add another node_states[] entry or two to hold these nodes. I'll try to work up a patch next week if noone beats me to it.

Quoted 46 lines · Paul Mundt

Quoted 4 lines · Lee Schermerhorn

Quoted 3 lines · Nish Aravamudan

Quoted 2 lines · Lee Schermerhorn

I have always considered the huge page pool, as populated by alloc_fresh_huge_page() in response to changes in nr_hugepages, to

be a

Quoted 4 lines

Quoted 4 lines

Quoted 4 lines

system global resource. I think the system "does the right thing"--well, almost--with Christoph's memoryless patches and your hugetlb patches. Certaintly, the huge pages allocated at boot time, based on the command line parameter, are system-wide. cpusets have

not

Quoted 5 lines

Quoted 5 lines

Quoted 1 line

been set up at that time.

I fully agree that hugepages are a global resource.

Quoted 1 line

It requires privilege to write to the nr_hugepages sysctl, so

allowing

Quoted 1 line

Quoted 1 line

Quoted 1 line

it to spread pages across all available nodes [with memory],

regardless

Quoted 3 lines

Quoted 3 lines

Quoted 3 lines

of cpusets, makes sense to me. Altho' I don't expect many folks are currently changing nr_hugepages from within a constrained cpuset, I wouldn't want to see us change existing behavior, in this respect.

Your

Quoted 2 lines

Quoted 2 lines

Quoted 2 lines

per node attributes will provide the mechanism to allocate different numbers of hugepages for, e.g., nodes in cpusets that have

applications

Quoted 11 lines

Quoted 7 lines

Quoted 1 line

that need them.

The issue is that with Adam's patches, the hugepage pool will grow on demand, presuming the process owner's mlock limit is sufficiently high. If said process were running within a constrained cpuset, it seems slightly out-of-whack to allow it grow the pool on other nodes to satisfy the demand.

Ah, I see. In that case, it might make sense to grow just for the cpuset. A couple of things come to mind tho':

Quoted 9 lines

1) we might want a per cpuset control to enable/disable hugetlb pool growth on demand, or to limit the max size of the pool--especially if the memories are not exclusively owned by the cpuset. Otherwise, non-privileged processes could grow the hugetlb pool in memories shared with other cpusets [maybe the root cpuset?], thereby reducing the amount of normal, managed pages available to the other cpusets. Probably want such a control in the absense of cpusets as well, if on-demand hugetlb pool growth is implemented.

Quoted 6 lines

I don't see that the two are mutually exclusive. Hugetlb pools have to be node-local anyways due to the varying distances, so perhaps the global resource thing is the wrong way to approach it. There are already hooks for spreading slab and page cache pages in cpusets, perhaps it makes sense to add a hugepage spread variant to balance across the constrained set?

I'm not sure I understand why you say "hugetlb pools"? There is no plural in the kernel, there is only the global pool. Now, on NUMA machines, yes, the pool is spread across nodes, but, well, that's just because of where the memory is. We already spread out the allocation of hugepages across all NUMA nodes (or will, once my patches go in). And I think with my earlier suggestion (of just changing the interleave mask used for those allocations to be cpuset-aware), that we'd spread across the cpuset too, if there is one. Is that what you mean by "spread variant"?

Quoted 9 lines

It would be quite nice to have some way to have nodes opt-in to the sort of behaviour they're willing to tolerate. Some nodes are never going to tolerate spreading of any sort, hugepages, and so forth. Perhaps it makes more sense to have some flags in the pgdat where we can more strongly type the sort of behaviour the node is willing to put up with (or capable of supporting), at least in this case the nodes that explicitly can't cope are factored out before we even get to cpuset constraints (plus this gives us a hook for setting up the interleave nodes in both the system init and default policies). Thoughts?

I guess I don't understand which nodes you're talking about now? How do you spread across any particular single node (how I read "Some nodes are never going to tolerate spreading of any sort")? Or do you mean that some cpusets aren't going to want to spread (interleave?).

Oh, are you trying to say that some nodes should be dropped from interleave masks (explicitly excluded from all possible interleave masks)? What kind of nodes would these be? We're doing something similar to deal with memoryless nodes, perhaps it could be generalized?

Quoted 11 lines · Paul Jackson

mm/hugetlb.c int hugetlb_reserve_pages(struct inode *inode, long from, long to) · line 1006

Quoted 9 lines · Adam

mm/hugetlb.c +7 -0

+   /*
+    * I haven't figured out how to incorporate this cpuset bodge into
+    * the dynamic hugetlb pool yet.  Hopefully someone more familiar with
+    * cpusets can weigh in on their desired semantics.  Maybe we can just
+    * drop this check?
+    *
    if (chg > cpuset_mems_nr(free_huge_pages_node))
            return -ENOMEM;
+    */

Quoted 2 lines

I can't figure out the value of this check either -- Ken Chen added it, perhaps he can comment.

To be honest, I just don't think a global hugetlb pool and cpusets are compatible, period. I wonder if moving to the mempool interface and having dynamic adjustable per-cpuset hugetlb mempools (ick) could make things work saner. It's on my list to see if mempools could be used to replace the custom hugetlb pool code. Otherwise, Mel's zone_movable stuff could possibly remove the need for hugetlb pools as we know them.

Quoted 9 lines

But the cpuset behaviour of this hugetlb stuff looks suspicious to me:
1) The code in alloc_fresh_huge_page() seems to round robin over the entire system, spreading the hugetlb pages uniformly on all nodes. If one a task in one small cpuset starts aggressively allocating hugetlb pages, do you think this will work, Adam -- looks to me like we will end up calling alloc_fresh_huge_page() many times, most of which will fail to alloc_pages_node() anything because the 'static nid' clock hand will be pointing at a node outside of the current tasks cpuset (not in that tasks mems_allowed). Inefficient, but I guess ok.

Very good point. I guess we call alloc_fresh_huge_page in two scenarios now... 1) By echoing a number into /proc/sys/vm/nr_hugepages, and 2) by trying to dynamically increase the pool size for a particular process. Case 1 is not in the context of any process (per se) and so node_online_map makes sense. For case 2 we could teach the __alloc_fresh_huge_page() to take a nodemask. That could get nasty though since we'd have to move away from a static variable to get proper interleaving.

Perhaps if we make sure __alloc_fresh_huge_page() can be restricted to a nodemask then we can avoid stealing pages from other cpusets. But we'd still be stuck with the existing problem for shared mappings: cpusets + our strict_reservation algorithm cannot provide guarantees (like we can without cpusets).

I'll cook up a __alloc_fresh_huge_page(nodemask) patch and see if that makes things better. Thanks for your review and comments.

But the cpuset behaviour of this hugetlb stuff looks suspicious to me:
1) The code in alloc_fresh_huge_page() seems to round robin over the entire system, spreading the hugetlb pages uniformly on all nodes. If one a task in one small cpuset starts aggressively allocating hugetlb pages, do you think this will work,

2) I don't see what keeps us from picking hugetlb pages off -any- node in the system, perhaps way outside the current cpuset.