Fixes for NUMA allocations on memoryless nodes

Quoted 4 lines

* int node_online(node) Is some node online?
* int node_possible(node) Is some node possible?
+ * int node_memory(node) Does a node have memory?
*

This name doesn't make sense; wouldn't node_has_memory() be better?

Quoted 9 lines

* int any_online_node(mask) First online node in mask
*
* node_set_online(node) set bit 'node' in node_online_map
* node_set_offline(node) clear bit 'node' in node_online_map
*
+ * node_set_memory(node) set bit 'node' in node_memory_map
+ * node_clear_memoryd(node) clear bit 'node' in node_memory_map
+ *

Extra 'd' there in node_clear_memory().

Quoted 6 lines · David Rientjes

Quoted 4 lines

* int node_online(node) Is some node online?
* int node_possible(node) Is some node possible?
+ * int node_memory(node) Does a node have memory?
*

This name doesn't make sense; wouldn't node_has_memory() be better?

node_set_has_memory and node_clear_has_memory sounds a bit strange.

Quoted 7 lines · Christoph Lameter

Quoted 5 lines · David Rientjes

Quoted 4 lines

* int node_online(node) Is some node online?
* int node_possible(node) Is some node possible?
+ * int node_memory(node) Does a node have memory?
*

Quoted 4 lines

Quoted 1 line

This name doesn't make sense; wouldn't node_has_memory() be better?

node_set_has_memory and node_clear_has_memory sounds a bit strange.

This will probably be one of those things that people see in the source and have to look up everytime. node_has_memory() is straight-forward and to the point.

Quoted 7 lines

Quoted 7 lines · David Rientjes

Quoted 6 lines

Quoted 5 lines

* int node_online(node) Is some node online?
* int node_possible(node) Is some node possible?
+ * int node_memory(node) Does a node have memory?
*

Quoted 8 lines

Quoted 4 lines

Quoted 1 line

This name doesn't make sense; wouldn't node_has_memory() be better?

node_set_has_memory and node_clear_has_memory sounds a bit strange.

This will probably be one of those things that people see in the source and have to look up everytime. node_has_memory() is straight-forward and to the point.

But node_possible is similar to node_memory.

Would you also prefer node_is_possible over node_possible?

node_is_online?

I think the problem is that online and possible are adverbs and adjectives, respectively, and memory is a noun. That's why when it appears in source code, it doesn't make a lot of sense for node_memory to return a boolean value. I suspect it would return the memory, whatever that is.

A sentence such as "This is a memory node" in opposition to "This is a memoryless nide" would put memory to use as an adjective.

Quoted 7 lines · Christoph Lameter

Quoted 5 lines · David Rientjes

Quoted 4 lines

* int node_online(node) Is some node online?
* int node_possible(node) Is some node possible?
+ * int node_memory(node) Does a node have memory?
*

Quoted 4 lines

Quoted 1 line

This name doesn't make sense; wouldn't node_has_memory() be better?

node_set_has_memory and node_clear_has_memory sounds a bit strange.

This will probably be one of those things that people see in the source and have to look up everytime. node_has_memory() is straight-forward and to the point.

Indeed, I did and (I like to think) I helped write the patches :)

We can keep

node_set_memory()
node_clear_memory()

but change node_memory() to node_has_memory() ?

Quoted 2 lines

Indeed, I did and (I like to think) I helped write the patches :)

The patches contain your signoff because of your authorship...

Quoted 2 lines

We can keep

Quoted 4 lines

node_set_memory()
node_clear_memory()

but change node_memory() to node_has_memory() ?

Hmmm.... That deviates from how the other node_xxx() things are so it disturbed my sense of order. We have no three word node_is/has_xxx yet.

Yeah, I realize that -- but I also agree with David that

node_memory()

is not intuitive at all. And we've already admitted that a few of the macros in there are inconsistent already :)

is not intuitive at all. And we've already admitted that a few of the macros in there are inconsistent already :)

Quoted 2 lines · Nishanth Aravamudan

is not intuitive at all. And we've already admitted that a few of the macros in there are inconsistent already :)

I do not want to add another inconsistency.

if (node_memory(node))

is pretty clear as far as I can tell.

Some of the macros in include/linux/nodemask.h are inconsistent. How can we make those consistent. Could you come up with a consistent naming scheme? Add the explanation for that scheme.

But that should be a separate patch. And the patch would have to change all uses of those macros in the kernel source tree.

Quoted 8 lines · David Rientjes

Quoted 6 lines · Christoph Lameter

Quoted 4 lines · David Rientjes

Quoted 4 lines

* int node_online(node) Is some node online?
* int node_possible(node) Is some node possible?
+ * int node_memory(node) Does a node have memory?
*

Quoted 6 lines

Quoted 3 lines

Quoted 1 line

This name doesn't make sense; wouldn't node_has_memory() be better?

node_set_has_memory and node_clear_has_memory sounds a bit strange.

This will probably be one of those things that people see in the source and have to look up everytime. node_has_memory() is straight-forward and to the point.

Indeed, I did and (I like to think) I helped write the patches :)

Why not just make the boolean sensible?

We can keep

node_set_memory()
node_clear_memory()

but change node_memory() to node_has_memory() ?

Index: linux-2.6.22-rc4-mm2/include/linux/gfp.h
===================================================================

Quoted 7 lines

===================================================================

include/linux/gfp.h +0 -0

@@ -175,6 +175,13 @@ static inline struct page *alloc_pages_n
 	if (nid < 0)
 		nid = numa_node_id();
 

include/linux/gfp.h static inline struct page *alloc_pages_n · line 184

Quoted 7 lines

include/linux/gfp.h +7 -0

+	/*
+	 * Check for the special case that GFP_THISNODE is used on a
+	 * memoryless node
+	 */
+	if ((gfp_mask & __GFP_THISNODE) && !node_memory(nid))
+		return NULL;
+

unlikely()?

Quoted 7 lines · David Rientjes

Quoted 7 lines

===================================================================

include/linux/gfp.h +0 -0

@@ -175,6 +175,13 @@ static inline struct page *alloc_pages_n
 	if (nid < 0)
 		nid = numa_node_id();
 

Quoted 9 lines

include/linux/gfp.h static inline struct page *alloc_pages_n · line 184

Quoted 7 lines

include/linux/gfp.h +7 -0

+	/*
+	 * Check for the special case that GFP_THISNODE is used on a
+	 * memoryless node
+	 */
+	if ((gfp_mask & __GFP_THISNODE) && !node_memory(nid))
+		return NULL;
+

unlikely()?

The gfp_mask is typically constant. So the whole expression is folded by the compiler into either

if (!node_memory(nid))
return NULL

or

nothing.

That's the point. Isn't !node_memory(nid) unlikely?

Quoted 1 line · David Rientjes

That's the point. Isn't !node_memory(nid) unlikely?

Correct.

Use unlikely

Signed-off-cy: Christoph Lameter <[email protected]>

Quoted 4 lines · Christoph Lameter

Quoted 1 line · David Rientjes

That's the point. Isn't !node_memory(nid) unlikely?

Quoted 5 lines

Correct.

Use unlikely

Signed-off-cy: Christoph Lameter <[email protected]>

Acked-by David Rientjes <[email protected]>

GFP_THISNODE checks that the zone selected is within the pgdat (node) of the first zone of a nodelist. That only works if the node has memory. A memoryless node will have its first zone on another pgdat (node).

Thus GFP_THISNODE may be returning memory on other nodes. GFP_THISNODE should fail if there is no local memory on a node.

So we add a check to verify that the node specified has memory in alloc_pages_node(). If the node has no memory then return NULL.

The case of alloc_pages(GFP_THISNODE) is not changed. alloc_pages() (with no memory policies in effect) is understood to prefer the current node. If a process is running on a node with no memory then its default allocations come from the next neighboring node. GFP_THISNODE will then force the memory to come from that node.

Signed-off-by Christoph Lameter <[email protected]>

Signed-off-by Nishanth Aravamudan <[email protected]>

Index: linux-2.6.22-rc4-mm2/include/linux/gfp.h
===================================================================

include/linux/gfp.h +0 -0

@@ -175,6 +175,13 @@ static inline struct page *alloc_pages_n
 	if (nid < 0)
 		nid = numa_node_id();
 

Quoted 28 lines · On Tue, 2007-06-12 at 13:48 -0700, [email protected] wrote:

GFP_THISNODE checks that the zone selected is within the pgdat (node) of the first zone of a nodelist. That only works if the node has memory. A memoryless node will have its first zone on another pgdat (node).

Thus GFP_THISNODE may be returning memory on other nodes. GFP_THISNODE should fail if there is no local memory on a node.

So we add a check to verify that the node specified has memory in alloc_pages_node(). If the node has no memory then return NULL.

The case of alloc_pages(GFP_THISNODE) is not changed. alloc_pages() (with no memory policies in effect) is understood to prefer the current node. If a process is running on a node with no memory then its default allocations come from the next neighboring node. GFP_THISNODE will then force the memory to come from that node.

Signed-off-by Christoph Lameter <[email protected]>

Signed-off-by Nishanth Aravamudan <[email protected]>

Index: linux-2.6.22-rc4-mm2/include/linux/gfp.h
===================================================================

include/linux/gfp.h +0 -0

@@ -175,6 +175,13 @@ static inline struct page *alloc_pages_n
 	if (nid < 0)
 		nid = numa_node_id();
 

include/linux/gfp.h static inline struct page *alloc_pages_n · line 187

Quoted 10 lines

include/linux/gfp.h +7 -0

+	/*
+	 * Check for the special case that GFP_THISNODE is used on a
+	 * memoryless node
+	 */
+	if ((gfp_mask & __GFP_THISNODE) && !node_memory(nid))
+		return NULL;
+
 	return __alloc_pages(gfp_mask, order,
 		NODE_DATA(nid)->node_zonelists + gfp_zone(gfp_mask));
 }

Attached patch fixes alloc_pages_node() so that it never returns an off-node page when GFP_THISNODE is specified by. This requires a fix to SLUB early allocation, included in the patch. Works on HP ia64 platform with small DMA only node and "zone order" zonelists. Will test on x86_64 real soon now...

---

PATCH fix GFP_THISNODE for DMA only nodes and zone-order zonelists

The map of nodes with memory may include nodes with just DMA/DMA32 memory. Using this map/mask together with GFP_THISNODE will not guarantee on-node allocations at higher zones. Modify checks in alloc_pages_node() to ensure that the first zone in the selected zonelist is "on-node".

This change will result in alloc_pages_node() returning NULL when GFP_THISNODE is specified and the first zone in the zonelist selected by (nid, gfp_zone(gfp_mask) is not on node 'nid'. This, in turn, BUGs out in slub.c:early_kmem_cache_node_alloc() which apparently can't handle a NULL page from new_slab(). Fix SLUB to handle NULL page in early allocation.

Signed-off-by Lee Schermerhorn <[email protected]>

 include/linux/gfp.h |   11 ++++++++---
 mm/slub.c           |   22 ++++++++++++----------
 2 files changed, 20 insertions(+), 13 deletions(-)

Index: Linux/include/linux/gfp.h
===================================================================

include/linux/gfp.h +3 -0

@@ -168,6 +168,9 @@ FASTCALL(__alloc_pages(gfp_t, unsigned i
 static inline struct page *alloc_pages_node(int nid, gfp_t gfp_mask,
 						unsigned int order)
 {
+	pg_data_t *pgdat;
+	struct zonelist *zonelist;
+
 	if (unlikely(order >= MAX_ORDER))
 		return NULL;
 

SLUB early allocation, included in the patch. Works on HP ia64 platform with small DMA only node and "zone order" zonelists. Will test on x86_64 real soon now...

The map of nodes with memory may include nodes with just DMA/DMA32 memory. Using this map/mask together with GFP_THISNODE will not guarantee on-node allocations at higher zones. Modify checks in alloc_pages_node() to ensure that the first zone in the selected zonelist is "on-node".

This change will result in alloc_pages_node() returning NULL when GFP_THISNODE is specified and the first zone in the zonelist selected by (nid, gfp_zone(gfp_mask) is not on node 'nid'. This, in turn, BUGs out in slub.c:early_kmem_cache_node_alloc() which apparently can't handle a NULL page from new_slab(). Fix SLUB to handle NULL page in early allocation.

Index: Linux/mm/slub.c
===================================================================

mm/slub.c +4 -10

@@ -1870,16 +1870,18 @@ static struct kmem_cache_node * __init e
 	/* new_slab() disables interupts */
 	local_irq_enable();
 
-	BUG_ON(!page);
-	n = page->freelist;
-	BUG_ON(!n);
-	page->freelist = get_freepointer(kmalloc_caches, n);
-	page->inuse++;
-	kmalloc_caches->node[node] = n;
-	setup_object_debug(kmalloc_caches, page, n);
-	init_kmem_cache_node(n);
-	atomic_long_inc(&n->nr_slabs);
-	add_partial(n, page);
+	if (page) {
+		n = page->freelist;
+		BUG_ON(!n);
+		page->freelist = get_freepointer(kmalloc_caches, n);

Quoted 3 lines · Lee Schermerhorn

SLUB early allocation, included in the patch. Works on HP ia64 platform with small DMA only node and "zone order" zonelists. Will test on x86_64 real soon now...

I do not see the difference?? How does this work? node_memory(x) fails there?

Quoted 5 lines

The map of nodes with memory may include nodes with just DMA/DMA32 memory. Using this map/mask together with GFP_THISNODE will not guarantee on-node allocations at higher zones. Modify checks in alloc_pages_node() to ensure that the first zone in the selected zonelist is "on-node".

That check is already done by __alloc_pages.

Quoted 6 lines

This change will result in alloc_pages_node() returning NULL when GFP_THISNODE is specified and the first zone in the zonelist selected by (nid, gfp_zone(gfp_mask) is not on node 'nid'. This, in turn, BUGs out in slub.c:early_kmem_cache_node_alloc() which apparently can't handle a NULL page from new_slab(). Fix SLUB to handle NULL page in early allocation.

Ummm... Slub would need to consult node_memory_map instead I guess.

Quoted 28 lines

Index: Linux/mm/slub.c
===================================================================

mm/slub.c +10 -10

@@ -1870,16 +1870,18 @@ static struct kmem_cache_node * __init e
 	/* new_slab() disables interupts */
 	local_irq_enable();
 
-	BUG_ON(!page);
-	n = page->freelist;
-	BUG_ON(!n);
-	page->freelist = get_freepointer(kmalloc_caches, n);
-	page->inuse++;
-	kmalloc_caches->node[node] = n;
-	setup_object_debug(kmalloc_caches, page, n);
-	init_kmem_cache_node(n);
-	atomic_long_inc(&n->nr_slabs);
-	add_partial(n, page);
+	if (page) {
+		n = page->freelist;
+		BUG_ON(!n);
+		page->freelist = get_freepointer(kmalloc_caches, n);
+		page->inuse++;
+		kmalloc_caches->node[node] = n;
+		setup_object_debug(kmalloc_caches, page, n);
+		init_kmem_cache_node(n);
+		atomic_long_inc(&n->nr_slabs);
+		add_partial(n, page);

mm/slub.c static struct kmem_cache_node * __init e · line 1886

Quoted 4 lines

mm/slub.c +2 -0

+	} else
+		kmalloc_caches->node[node] = NULL;
 	return n;
 }

It would be easier to modify SLUB to loop over node_memory_map instead of node_online_map? Potentially we have to change all loops over online node in the slab allocators.

I would like to roll up the patches and small fixes into a set of 4 or 5 patches that Andrew can pick up, so once this is all stable, I'll post a fresh series. Sound good, Andrew?

Quoted 3 lines · Nishanth Aravamudan

I would like to roll up the patches and small fixes into a set of 4 or 5 patches that Andrew can pick up, so once this is all stable, I'll post a fresh series. Sound good, Andrew?

NACK. This patchset is not ready for any inclusion and nothing like that should go into 2.6.22. We first need to assess the breakage that results if GFP_THISNODE now returns NULL for memoryless nodes. So far GFP_THISNODE returns memory on the nearest node and that seems to make lots of things keep working.

Who the heck said anything about mainline?

Quoted 1 line · Nishanth Aravamudan

Who the heck said anything about mainline?

Well we do have discovered issues that are bugs. Handling of memoryless nodes is rather strange.

Sorry, the real reason for wrapping the patches and reposting, for me, is that we've had a lot of versions flying around, with small fixlets here and there. I wanted to start a new thread for the 3 or so patches I see that implement the core of dealing with memoryless nodes, and then keep the discussion going there, but that was purely for my own sanity.

Quoted 6 lines · Nishanth Aravamudan

Sorry, the real reason for wrapping the patches and reposting, for me, is that we've had a lot of versions flying around, with small fixlets here and there. I wanted to start a new thread for the 3 or so patches I see that implement the core of dealing with memoryless nodes, and then keep the discussion going there, but that was purely for my own sanity.

Ok. I will try to put all the fixes together. Basically that will be the three patches plus Lee's new suggestion for alloc_pages_node. Plus fixes to slab / slub / the uncached allocator etc where I see that the online map is used but what was really intended was the node_memory_map. There is also stuff in the oom killer, vmscan.c etc that seems to make that assumption. Sigh.

That still leaves the issue of the name.

node_memory

set_node_has_memory
set_node_no_memory

Quoted 3 lines · Lee Schermerhorn

SLUB early allocation, included in the patch. Works on HP ia64 platform with small DMA only node and "zone order" zonelists. Will test on x86_64 real soon now...

I do not see the difference?? How does this work? node_memory(x) fails there?

Quoted 6 lines

The map of nodes with memory may include nodes with just DMA/DMA32 memory. Using this map/mask together with GFP_THISNODE will not guarantee on-node allocations at higher zones. Modify checks in alloc_pages_node() to ensure that the first zone in the selected zonelist is "on-node".

That check is already done by __alloc_pages.

Quoted 7 lines

This change will result in alloc_pages_node() returning NULL when GFP_THISNODE is specified and the first zone in the zonelist selected by (nid, gfp_zone(gfp_mask) is not on node 'nid'. This, in turn, BUGs out in slub.c:early_kmem_cache_node_alloc() which apparently can't handle a NULL page from new_slab(). Fix SLUB to handle NULL page in early allocation.

Ummm... Slub would need to consult node_memory_map instead I guess.

Quoted 29 lines

Index: Linux/mm/slub.c
===================================================================

mm/slub.c +10 -10

@@ -1870,16 +1870,18 @@ static struct kmem_cache_node * __init e
 	/* new_slab() disables interupts */
 	local_irq_enable();
 
-	BUG_ON(!page);
-	n = page->freelist;
-	BUG_ON(!n);
-	page->freelist = get_freepointer(kmalloc_caches, n);
-	page->inuse++;
-	kmalloc_caches->node[node] = n;
-	setup_object_debug(kmalloc_caches, page, n);
-	init_kmem_cache_node(n);
-	atomic_long_inc(&n->nr_slabs);
-	add_partial(n, page);
+	if (page) {
+		n = page->freelist;
+		BUG_ON(!n);
+		page->freelist = get_freepointer(kmalloc_caches, n);
+		page->inuse++;
+		kmalloc_caches->node[node] = n;
+		setup_object_debug(kmalloc_caches, page, n);
+		init_kmem_cache_node(n);
+		atomic_long_inc(&n->nr_slabs);
+		add_partial(n, page);

mm/slub.c static struct kmem_cache_node * __init e · line 1886

Quoted 4 lines

mm/slub.c +2 -0

+	} else
+		kmalloc_caches->node[node] = NULL;
 	return n;
 }

It would be easier to modify SLUB to loop over node_memory_map instead of node_online_map? Potentially we have to change all loops over online node in the slab allocators.

---
 include/linux/nodemask.h |    1 +
 mm/slub.c                |    2 +-
 2 files changed, 2 insertions(+), 1 deletion(-)

Index: linux-2.6/include/linux/nodemask.h
===================================================================

include/linux/nodemask.h +1 -0

@@ -377,5 +377,6 @@ extern int nr_node_ids;
 
 #define for_each_node(node)	   for_each_node_mask((node), node_possible_map)
 #define for_each_online_node(node) for_each_node_mask((node), node_online_map)
+#define for_each_memory_node(node) for_each_node_mask((node), node_memory_map)
 
 #endif /* __LINUX_NODEMASK_H */

Index: linux-2.6/mm/slub.c
===================================================================

Quoted 1 line

That check is already done by __alloc_pages.

You mean in get_page_from_freelist()? No, it only checks that the zone under consideration is on the same node as the zone at the start of the list. This can be off-node if the node is populated only at lower zones; and the zonelists are in zone-order.

Quoted 1 line

Ummm... Slub would need to consult node_memory_map instead I guess.

Probably should check the node_memory_map to avoid attempting allocations from completely memoryless nodes. However, it should still be able to handle nulls from alloc_pages_nodes() because of the scenarios discussed above.

Again, node_memory_map can't detect the "first zone in zonelist off-node" situation. That's the one that alloc_pages_node() must guard
against. So, it can/should/must return NULL when attempting to
allocate from a higher zone that is off-node.

Quoted 6 lines · Lee Schermerhorn

Quoted 1 line

That check is already done by __alloc_pages.

You mean in get_page_from_freelist()? No, it only checks that the zone under consideration is on the same node as the zone at the start of the list. This can be off-node if the node is populated only at lower zones; and the zonelists are in zone-order.

See the later discussion. I did not see the use the nodes pgdat here that you only have in alloc_pages_node().

Quoted 6 lines

Quoted 1 line

Ummm... Slub would need to consult node_memory_map instead I guess.

Probably should check the node_memory_map to avoid attempting allocations from completely memoryless nodes. However, it should still be able to handle nulls from alloc_pages_nodes() because of the scenarios discussed above.

It is able to handle NULLs during usual operations but not during bootstrap.

Quoted 5 lines

Again, node_memory_map can't detect the "first zone in zonelist off-node" situation. That's the one that alloc_pages_node() must guard
against. So, it can/should/must return NULL when attempting to
allocate from a higher zone that is off-node.

I think GFP_THISNODE should not fail in that case.

Quoted 1 line

It is able to handle NULLs during usual operations but not during bootstrap.

??? it has to handle memoryless nodes, right? if so, why can't it handle alloc_pages_node('THISNODE,...) returning NULL? Too late in the process?

Or do you mean that GPF_THISNODE should return memory from the lower zones, if it satisfies the order requirement? Your custom 'thisnode' zonelist will enable this, right? That would work for my particular config of my platform, but on a larger platform, I'd have a larger DMA zone and wouldn't want hugetlb pages coming from there.

I guess that in the long run, I need to hope that Nish's per node huge page attribute goes in. Then, we can provide a fancy script to configure huge pages on a specific set of nodes, rather than relying on the kernel to distribute them evenly across the set of nodes that "make sense" for the given platform.

include/linux/gfp.h +3 -1

@@ -168,6 +168,9 @@ FASTCALL(__alloc_pages(gfp_t, unsigned i
 static inline struct page *alloc_pages_node(int nid, gfp_t gfp_mask,
 						unsigned int order)
 {
+	pg_data_t *pgdat;
+	struct zonelist *zonelist;
+
 	if (unlikely(order >= MAX_ORDER))
 		return NULL;
 

@@ -179,11 +182,13 @@ static inline struct page *alloc_pages_n
 	 * Check for the special case that GFP_THISNODE is used on a
 	 * memoryless node
 	 */
-	if ((gfp_mask & __GFP_THISNODE) && !node_memory(nid))

Quoted 17 lines · Lee Schermerhorn

include/linux/gfp.h +3 -1

@@ -168,6 +168,9 @@ FASTCALL(__alloc_pages(gfp_t, unsigned i
 static inline struct page *alloc_pages_node(int nid, gfp_t gfp_mask,
 						unsigned int order)
 {
+	pg_data_t *pgdat;
+	struct zonelist *zonelist;
+
 	if (unlikely(order >= MAX_ORDER))
 		return NULL;
 

@@ -179,11 +182,13 @@ static inline struct page *alloc_pages_n
 	 * Check for the special case that GFP_THISNODE is used on a
 	 * memoryless node
 	 */
-	if ((gfp_mask & __GFP_THISNODE) && !node_memory(nid))

include/linux/gfp.h static inline struct page *alloc_pages_n · line 192

Quoted 10 lines

include/linux/gfp.h +5 -2

+	pgdat = NODE_DATA(nid);
+	zonelist = pgdat->node_zonelists + gfp_zone(gfp_mask);
+	if ((gfp_mask & __GFP_THISNODE) &&
+		pgdat != zonelist->zones[0]->zone_pgdat)
 		return NULL;
 
-	return __alloc_pages(gfp_mask, order,
-		NODE_DATA(nid)->node_zonelists + gfp_zone(gfp_mask));
+	return __alloc_pages(gfp_mask, order, zonelist);
 }

Good idea but I think this does not address the case where the DMA zone of a node was moved to the end of the zonelist. In that case the first zone is not on the first pgdat but the node has memory. The memory of the node is listed elsewhere in the nodelist. I can probably modify __alloc_pages to make GFP_THISNODE to check all zones but we do not have the pgdat reference there. Sigh.

How about generating a special THISNODE zonelist in build_zonelist that only contains the zones of a single node. Then just use that one if GFP_THISNODE is set. Then we get rid of all the GFP_THISNODE crap that I added to __alloc_pages?