Skip to contentSkip to conversation

mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

PATCH Superseded Mel Gorman 3 participants 8 messages Apr 9, 2021 – Apr 10, 2021

Subsystems (4)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Mel Gorman

author
#0 lore raw

PATCH 1/1 mm/memory_hotplug: Make unpopulated zones PCP structures unreachable during hot remove

zone_pcp_reset allegedly protects against a race with drain_pages using local_irq_save but this is bogus. local_irq_save only operates on the local CPU. If memory hotplug is running on CPU A and drain_pages is running on CPU B, disabling IRQs on CPU A does not affect CPU B and offers no protection.

This patch reorders memory hotremove such that the PCP structures relevant to the zone are no longer reachable by the time the structures are freed. With this reordering, no protection is required to prevent a use-after-free and the IRQs can be left enabled. zone_pcp_reset is renamed to zone_pcp_destroy to make it clear that the per-cpu structures are deleted when the function returns.

---
 mm/internal.h       |  2 +-
 mm/memory_hotplug.c | 10 +++++++---
 mm/page_alloc.c     | 22 ++++++++++++++++------
 3 files changed, 24 insertions(+), 10 deletions(-)
mm/internal.h +1 -1
index 09adf152a10b..cc34ce4461b7 100644
@@ -203,7 +203,7 @@ extern void free_unref_page(struct page *page);
 extern void free_unref_page_list(struct list_head *list);
 
 extern void zone_pcp_update(struct zone *zone);
-extern void zone_pcp_reset(struct zone *zone);
+extern void zone_pcp_destroy(struct zone *zone);
 extern void zone_pcp_disable(struct zone *zone);
 extern void zone_pcp_enable(struct zone *zone);
 
mm/memory_hotplug.c +7 -3
index 0cdbbfbc5757..3d059c9f9c2d 100644
@@ -1687,12 +1687,16 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages)
 	zone->nr_isolate_pageblock -= nr_pages / pageblock_nr_pages;
 	spin_unlock_irqrestore(&zone->lock, flags);
 
-	zone_pcp_enable(zone);
-
 	/* removal success */
 	adjust_managed_page_count(pfn_to_page(start_pfn), -nr_pages);
 	zone->present_pages -= nr_pages;
 
+	/*
+	 * Restore PCP after managed pages has been updated. Unpopulated
+	 * zones PCP structures will remain unusable.
+	 */
+	zone_pcp_enable(zone);
+
 	pgdat_resize_lock(zone->zone_pgdat, &flags);
 	zone->zone_pgdat->node_present_pages -= nr_pages;
 	pgdat_resize_unlock(zone->zone_pgdat, &flags);
@@ -1700,8 +1704,8 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages)
 	init_per_zone_wmark_min();
 
 	if (!populated_zone(zone)) {
-		zone_pcp_reset(zone);
 		build_all_zonelists(NULL);
+		zone_pcp_destroy(zone);
 	} else
 		zone_pcp_update(zone);
 
mm/page_alloc.c +16 -6
index 5e8aedb64b57..d6c3db853552 100644
@@ -8946,18 +8946,29 @@ void zone_pcp_disable(struct zone *zone)
 
 void zone_pcp_enable(struct zone *zone)
 {
-	__zone_set_pageset_high_and_batch(zone, zone->pageset_high, zone->pageset_batch);
+	/*
+	 * If the zone is populated, restore the high and batch counts.
+	 * If unpopulated, leave the high and batch count as 0 and 1
+	 * respectively as done by zone_pcp_disable. The per-cpu
+	 * structures will later be freed by zone_pcp_destroy.
+	 */
+	if (populated_zone(zone))
+		__zone_set_pageset_high_and_batch(zone, zone->pageset_high, zone->pageset_batch);
+
 	mutex_unlock(&pcp_batch_high_lock);
 }
 
-void zone_pcp_reset(struct zone *zone)
+/*
+ * Called when a zone has been hot-removed. At this point, the PCP has been
+ * drained, disabled and the zone is removed from the zonelists so the
+ * structures are no longer in use. PCP was disabled/drained by
+ * zone_pcp_disable. This function will drain any remaining vmstat deltas.
+ */
+void zone_pcp_destroy(struct zone *zone)
 {
-	unsigned long flags;
 	int cpu;
 	struct per_cpu_pageset *pset;
 
-	/* avoid races with drain_pages()  */
-	local_irq_save(flags);
 	if (zone->pageset != &boot_pageset) {
 		for_each_online_cpu(cpu) {
 			pset = per_cpu_ptr(zone->pageset, cpu);
@@ -8966,7 +8977,6 @@ void zone_pcp_reset(struct zone *zone)
 		free_percpu(zone->pageset);
 		zone->pageset = &boot_pageset;
 	}
-	local_irq_restore(flags);
 }
 
 #ifdef CONFIG_MEMORY_HOTREMOVE
reviewer
#1 lore raw
Quoted 5 lines · Mel Gorman

zone_pcp_reset allegedly protects against a race with drain_pages using local_irq_save but this is bogus. local_irq_save only operates on the local CPU. If memory hotplug is running on CPU A and drain_pages is running on CPU B, disabling IRQs on CPU A does not affect CPU B and offers no protection.

Yes, the synchronization aspect is bogus indeed.

Quoted 6 lines

This patch reorders memory hotremove such that the PCP structures relevant to the zone are no longer reachable by the time the structures are freed. With this reordering, no protection is required to prevent a use-after-free and the IRQs can be left enabled. zone_pcp_reset is renamed to zone_pcp_destroy to make it clear that the per-cpu structures are deleted when the function returns.

Wouldn't it be much easier to simply not destroy/reset pcp of an empty zone at all? The whole point of this exercise seems to be described in 340175b7d14d5. setup_zone_pageset can check for an already allocated pcp and simply reinitialize it.

Signature · 2 lines
Michal Hocko
SUSE Labs
reviewer
#2 lore raw
Quoted 16 lines · Michal Hocko
Quoted 5 lines · Mel Gorman

zone_pcp_reset allegedly protects against a race with drain_pages using local_irq_save but this is bogus. local_irq_save only operates on the local CPU. If memory hotplug is running on CPU A and drain_pages is running on CPU B, disabling IRQs on CPU A does not affect CPU B and offers no protection.

Yes, the synchronization aspect is bogus indeed.

Quoted 6 lines

This patch reorders memory hotremove such that the PCP structures relevant to the zone are no longer reachable by the time the structures are freed. With this reordering, no protection is required to prevent a use-after-free and the IRQs can be left enabled. zone_pcp_reset is renamed to zone_pcp_destroy to make it clear that the per-cpu structures are deleted when the function returns.

Quoted 4 lines

Wouldn't it be much easier to simply not destroy/reset pcp of an empty zone at all? The whole point of this exercise seems to be described in 340175b7d14d5. setup_zone_pageset can check for an already allocated pcp and simply reinitialize it.

I meant this

mm/memory_hotplug.c +0 -1
index 754026a9164d..7169342f5474 100644
@@ -1810,7 +1818,6 @@ int __ref offline_pages(unsigned long start_pfn, unsigned long nr_pages)
 	init_per_zone_wmark_min();
 
 	if (!populated_zone(zone)) {
-		zone_pcp_reset(zone);
 		build_all_zonelists(NULL);
 	} else
 		zone_pcp_update(zone);
mm/page_alloc.c +7 -20
index e6a602e82860..b0fdda77e570 100644
@@ -6496,7 +6496,13 @@ void __meminit setup_zone_pageset(struct zone *zone)
 	struct per_cpu_pageset *p;
 	int cpu;
 
-	zone->pageset = alloc_percpu(struct per_cpu_pageset);
+	/*
+	 * zone could have gone completely offline during memory hotplug
+	 * when the pgdat is left behind for simplicity. On a next onlining
+	 * we do not need to reallocate pcp state.
+	 */
+	if (!zone->pageset)
+		zone->pageset = alloc_percpu(struct per_cpu_pageset);
 	for_each_possible_cpu(cpu) {
 		p = per_cpu_ptr(zone->pageset, cpu);
 		pageset_init(p);
@@ -8803,25 +8809,6 @@ void zone_pcp_enable(struct zone *zone)
 	mutex_unlock(&pcp_batch_high_lock);
 }
 
-void zone_pcp_reset(struct zone *zone)
-{
-	unsigned long flags;
-	int cpu;
-	struct per_cpu_pageset *pset;
-
-	/* avoid races with drain_pages()  */
-	local_irq_save(flags);
-	if (zone->pageset != &boot_pageset) {
-		for_each_online_cpu(cpu) {
-			pset = per_cpu_ptr(zone->pageset, cpu);
-			drain_zonestat(zone, pset);
-		}
-		free_percpu(zone->pageset);
-		zone->pageset = &boot_pageset;
-	}
-	local_irq_restore(flags);
-}
-
 #ifdef CONFIG_MEMORY_HOTREMOVE
 /*
  * All pages in the range must be in a single zone, must not contain holes,
Signature · 2 lines
Michal Hocko
SUSE Labs
author
#3 lore raw
Quoted 17 lines · Michal Hocko
Quoted 16 lines · Michal Hocko
Quoted 5 lines · Mel Gorman

zone_pcp_reset allegedly protects against a race with drain_pages using local_irq_save but this is bogus. local_irq_save only operates on the local CPU. If memory hotplug is running on CPU A and drain_pages is running on CPU B, disabling IRQs on CPU A does not affect CPU B and offers no protection.

Yes, the synchronization aspect is bogus indeed.

Quoted 6 lines

This patch reorders memory hotremove such that the PCP structures relevant to the zone are no longer reachable by the time the structures are freed. With this reordering, no protection is required to prevent a use-after-free and the IRQs can be left enabled. zone_pcp_reset is renamed to zone_pcp_destroy to make it clear that the per-cpu structures are deleted when the function returns.

Quoted 7 lines
Quoted 4 lines

Wouldn't it be much easier to simply not destroy/reset pcp of an empty zone at all? The whole point of this exercise seems to be described in 340175b7d14d5. setup_zone_pageset can check for an already allocated pcp and simply reinitialize it.

I meant this

It might be simplier but if the intention is to free as much memory as possible during hot-remove, it seems wasteful to leave the per-cpu structures behind if we do not have to. If a problem with my patch can be spotted then I'm happy to go with an alternative fix but there are two minor issues with your proposed fix.

Quoted 6 lines
mm/page_alloc.c +0 -0
index e6a602e82860..b0fdda77e570 100644
@@ -6496,7 +6496,13 @@ void __meminit setup_zone_pageset(struct zone *zone)
 	struct per_cpu_pageset *p;

mm/page_alloc.c void __meminit setup_zone_pageset(struct zone *zone) · line 6505

Quoted 10 lines
mm/page_alloc.c +7 -1
 	int cpu;
 
-	zone->pageset = alloc_percpu(struct per_cpu_pageset);
+	/*
+	 * zone could have gone completely offline during memory hotplug
+	 * when the pgdat is left behind for simplicity. On a next onlining
+	 * we do not need to reallocate pcp state.
+	 */
+	if (!zone->pageset)
+		zone->pageset = alloc_percpu(struct per_cpu_pageset);

Should be "if (zone->pageset != &boot_pageset)" ?

Quoted 16 lines
mm/page_alloc.c +0 -9
 	for_each_possible_cpu(cpu) {
 		p = per_cpu_ptr(zone->pageset, cpu);
 		pageset_init(p);
@@ -8803,25 +8809,6 @@ void zone_pcp_enable(struct zone *zone)
 	mutex_unlock(&pcp_batch_high_lock);
 }
 
-void zone_pcp_reset(struct zone *zone)
-{
-	unsigned long flags;
-	int cpu;
-	struct per_cpu_pageset *pset;
-
-	/* avoid races with drain_pages()  */
-	local_irq_save(flags);
-	if (zone->pageset != &boot_pageset) {

mm/page_alloc.c void zone_pcp_enable(struct zone *zone) · line 8824

Quoted 10 lines
mm/page_alloc.c +0 -10
-		for_each_online_cpu(cpu) {
-			pset = per_cpu_ptr(zone->pageset, cpu);
-			drain_zonestat(zone, pset);
-		}
-		free_percpu(zone->pageset);
-		zone->pageset = &boot_pageset;
-	}
-	local_irq_restore(flags);
-}
-

zone_pcp_reset still needs to exist to drain the remaining vmstats or it'll break 5a883813845a ("memory-hotplug: fix zone stat mismatch").

Signature · 2 lines
Mel Gorman
SUSE Labs
reviewer
#4 lore raw
Quoted 23 lines · Mel Gorman
Quoted 22 lines · Michal Hocko
Quoted 20 lines · Michal Hocko
Quoted 5 lines · Mel Gorman

zone_pcp_reset allegedly protects against a race with drain_pages using local_irq_save but this is bogus. local_irq_save only operates on the local CPU. If memory hotplug is running on CPU A and drain_pages is running on CPU B, disabling IRQs on CPU A does not affect CPU B and offers no protection.

Yes, the synchronization aspect is bogus indeed.

Quoted 6 lines

This patch reorders memory hotremove such that the PCP structures relevant to the zone are no longer reachable by the time the structures are freed. With this reordering, no protection is required to prevent a use-after-free and the IRQs can be left enabled. zone_pcp_reset is renamed to zone_pcp_destroy to make it clear that the per-cpu structures are deleted when the function returns.

Wouldn't it be much easier to simply not destroy/reset pcp of an empty zone at all? The whole point of this exercise seems to be described in 340175b7d14d5. setup_zone_pageset can check for an already allocated pcp and simply reinitialize it.

Quoted 6 lines
Quoted 2 lines

I meant this

It might be simplier but if the intention is to free as much memory as possible during hot-remove, it seems wasteful to leave the per-cpu structures behind if we do not have to.

We do leave the whole pgdat behind. I do not think pagesets really do matter.

Quoted 3 lines

If a problem with my patch can
be spotted then I'm happy to go with an alternative fix but there are two minor issues with your proposed fix.

I will not insist but this code has proven to bitrot and I just find it much simpler to drop it altogether rather than conserve it in some form. Not something I would insist though.

Quoted 8 lines
Quoted 8 lines
mm/page_alloc.c +0 -0
index e6a602e82860..b0fdda77e570 100644
@@ -6496,7 +6496,13 @@ void __meminit setup_zone_pageset(struct zone *zone)
 	struct per_cpu_pageset *p;
 	int cpu;
 
Quoted 10 lines

mm/page_alloc.c void __meminit setup_zone_pageset(struct zone *zone) · line 6505

Quoted 8 lines
mm/page_alloc.c +7 -1
-	zone->pageset = alloc_percpu(struct per_cpu_pageset);
+	/*
+	 * zone could have gone completely offline during memory hotplug
+	 * when the pgdat is left behind for simplicity. On a next onlining
+	 * we do not need to reallocate pcp state.
+	 */
+	if (!zone->pageset)
+		zone->pageset = alloc_percpu(struct per_cpu_pageset);

Should be "if (zone->pageset != &boot_pageset)" ?

Memory hotplug really wants the NULL
check. it doesn't use boot_pageset (if we drop rest to boot_pageset). But you are right that the boot time initialization first sets boot_pageset (zone_pcp_init) and initializes real pagesets later (setup_per_cpu_pageset). But this can be handled at the memory hotplug layer I believe

mm/memory_hotplug.c +2 -1
index 754026a9164d..1cadfec323fc 100644
@@ -883,7 +883,8 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages,
 	 */
 	if (!populated_zone(zone)) {
 		need_zonelists_rebuild = 1;
-		setup_zone_pageset(zone);
+		if (!zone->pageset)
+			setup_zone_pageset(zone);
 	}
 
Quoted 22 lines
Quoted 22 lines
mm/page_alloc.c +0 -15
 	for_each_possible_cpu(cpu) {
 		p = per_cpu_ptr(zone->pageset, cpu);
 		pageset_init(p);
@@ -8803,25 +8809,6 @@ void zone_pcp_enable(struct zone *zone)
 	mutex_unlock(&pcp_batch_high_lock);
 }
 
-void zone_pcp_reset(struct zone *zone)
-{
-	unsigned long flags;
-	int cpu;
-	struct per_cpu_pageset *pset;
-
-	/* avoid races with drain_pages()  */
-	local_irq_save(flags);
-	if (zone->pageset != &boot_pageset) {
-		for_each_online_cpu(cpu) {
-			pset = per_cpu_ptr(zone->pageset, cpu);
-			drain_zonestat(zone, pset);
-		}
-		free_percpu(zone->pageset);
-		zone->pageset = &boot_pageset;
Quoted 8 lines

mm/page_alloc.c void zone_pcp_enable(struct zone *zone) · line 8824

Quoted 4 lines
mm/page_alloc.c +0 -4
-	}
-	local_irq_restore(flags);
-}
-

zone_pcp_reset still needs to exist to drain the remaining vmstats or it'll break 5a883813845a ("memory-hotplug: fix zone stat mismatch").

Are you sure we are reseting vmstats in the hotremove. I do not see anything like that. Maybe this was needed at the time. I will double check.

Signature · 2 lines
Michal Hocko
SUSE Labs
author
#5 lore raw
Quoted 17 lines · Michal Hocko
Quoted 17 lines
Quoted 16 lines
Quoted 9 lines

Yes, the synchronization aspect is bogus indeed.

Quoted 6 lines

This patch reorders memory hotremove such that the PCP structures relevant to the zone are no longer reachable by the time the structures are freed. With this reordering, no protection is required to prevent a use-after-free and the IRQs can be left enabled. zone_pcp_reset is renamed to zone_pcp_destroy to make it clear that the per-cpu structures are deleted when the function returns.

Quoted 4 lines

Wouldn't it be much easier to simply not destroy/reset pcp of an empty zone at all? The whole point of this exercise seems to be described in 340175b7d14d5. setup_zone_pageset can check for an already allocated pcp and simply reinitialize it.

I meant this

Quoted 7 lines
Quoted 3 lines

It might be simplier but if the intention is to free as much memory as possible during hot-remove, it seems wasteful to leave the per-cpu structures behind if we do not have to.

We do leave the whole pgdat behind. I do not think pagesets really do matter.

Probably not given that zone and node hotplug is fragile to say the least with a low success rate. Still, it seems wasteful if we can preserve the freeing part and the alternative fix is more subtle than it appears.

Quoted 39 lines
Quoted 3 lines

If a problem with my patch can
be spotted then I'm happy to go with an alternative fix but there are two minor issues with your proposed fix.

I will not insist but this code has proven to bitrot and I just find it much simpler to drop it altogether rather than conserve it in some form. Not something I would insist though.

Quoted 8 lines
Quoted 8 lines
mm/page_alloc.c +0 -0
index e6a602e82860..b0fdda77e570 100644
@@ -6496,7 +6496,13 @@ void __meminit setup_zone_pageset(struct zone *zone)
 	struct per_cpu_pageset *p;
 	int cpu;
 
Quoted 10 lines

mm/page_alloc.c void __meminit setup_zone_pageset(struct zone *zone) · line 6505

Quoted 8 lines
mm/page_alloc.c +7 -1
-	zone->pageset = alloc_percpu(struct per_cpu_pageset);
+	/*
+	 * zone could have gone completely offline during memory hotplug
+	 * when the pgdat is left behind for simplicity. On a next onlining
+	 * we do not need to reallocate pcp state.
+	 */
+	if (!zone->pageset)
+		zone->pageset = alloc_percpu(struct per_cpu_pageset);

Should be "if (zone->pageset != &boot_pageset)" ?

Memory hotplug really wants the NULL
check. it doesn't use boot_pageset (if we drop rest to boot_pageset). But you are right that the boot time initialization first sets boot_pageset (zone_pcp_init) and initializes real pagesets later (setup_per_cpu_pageset). But this can be handled at the memory hotplug layer I believe

mm/memory_hotplug.c +0 -0
index 754026a9164d..1cadfec323fc 100644

mm/memory_hotplug.c int __ref online_pages(unsigned long pfn, unsigned long nr_pages, · line 889

Quoted 8 lines
mm/memory_hotplug.c +2 -1
@@ -883,7 +883,8 @@ int __ref online_pages(unsigned long pfn, unsigned long nr_pages,
 	 */
 	if (!populated_zone(zone)) {
 		need_zonelists_rebuild = 1;
-		setup_zone_pageset(zone);
+		if (!zone->pageset)
+			setup_zone_pageset(zone);
 	}
 

I think this will be fragile because it makes assumptions on how boot works -- specifically that zone->pageset is left alone for unpopulated zones. This will only work for zones in unpopulated *nodes* during boot because of this sequence

free_area_init_core(pgdat)

  for (j = 0; j < MAX_NR_ZONES; j++) {
        ...
        zone_init_internals(zone)
  }
  -> zone_init_internals(zone)
     -> zone_pcp_init(zone)
        zone->pageset = &boot_pageset;

I think a NULL check will mean that hotplugging a previously unpopulated zone will run the risk of using boot_pageset. That will likely explode because IRQs disabled does not protect boot_pageset, it only works early in boot or during the initial hotplug when it's not visible yet.

Quoted 31 lines
mm/memory_hotplug.c +0 -0
 > >  	for_each_possible_cpu(cpu) {
Quoted 21 lines
Quoted 21 lines
mm/page_alloc.c +0 -15
 		p = per_cpu_ptr(zone->pageset, cpu);
 		pageset_init(p);
@@ -8803,25 +8809,6 @@ void zone_pcp_enable(struct zone *zone)
 	mutex_unlock(&pcp_batch_high_lock);
 }
 
-void zone_pcp_reset(struct zone *zone)
-{
-	unsigned long flags;
-	int cpu;
-	struct per_cpu_pageset *pset;
-
-	/* avoid races with drain_pages()  */
-	local_irq_save(flags);
-	if (zone->pageset != &boot_pageset) {
-		for_each_online_cpu(cpu) {
-			pset = per_cpu_ptr(zone->pageset, cpu);
-			drain_zonestat(zone, pset);
-		}
-		free_percpu(zone->pageset);
-		zone->pageset = &boot_pageset;
Quoted 8 lines

mm/page_alloc.c void zone_pcp_enable(struct zone *zone) · line 8824

Quoted 4 lines
mm/page_alloc.c +0 -4
-	}
-	local_irq_restore(flags);
-}
-

zone_pcp_reset still needs to exist to drain the remaining vmstats or it'll break 5a883813845a ("memory-hotplug: fix zone stat mismatch").

Quoted 3 lines

Are you sure we are reseting vmstats in the hotremove. I do not see anything like that. Maybe this was needed at the time. I will double check.

zone_pcp_reset calls drain_zonestat to apply the per-cpu vmstat deltas to the atomic per-zone and global stats.

If anything, the minimal "fix" is to simply delete IRQ disable/enable on the grounds that IRQs protect nothing and assume the existing hotplug paths guarantees the PCP cannot be used after zone_pcp_enable(). That should be the case already because all the pages have been freed and there is nothing to even put into the PCPs but I worried that the PCP structure itself might still be reachable even if it's useless which is why I freed the structure once they could not be reached via zonelists.

Signature · 2 lines
Mel Gorman
SUSE Labs
#6 lore raw
Quoted 8 lines
Quoted 7 lines
Quoted 3 lines

zone_pcp_reset still needs to exist to drain the remaining vmstats or it'll break 5a883813845a ("memory-hotplug: fix zone stat mismatch").

Are you sure we are reseting vmstats in the hotremove. I do not see anything like that. Maybe this was needed at the time. I will double check.

Quoted 6 lines

zone_pcp_reset calls drain_zonestat to apply the per-cpu vmstat deltas to the atomic per-zone and global stats.

If anything, the minimal "fix" is to simply delete IRQ disable/enable on the grounds that IRQs protect nothing and assume the existing hotplug paths guarantees the PCP cannot be used after zone_pcp_enable(). That

^ that sounds sane to me

Quoted 5 lines

should be the case already because all the pages have been freed and there is nothing to even put into the PCPs but I worried that the PCP structure itself might still be reachable even if it's useless which is why I freed the structure once they could not be reached via zonelists.

Signature · 3 lines
Thanks,

David / dhildenb
reviewer
#7 lore raw
In reply to Mel Gorman

On Fri 09-04-21 16:12:59, Mel Gorman wrote:
[...]

Quoted 7 lines

If anything, the minimal "fix" is to simply delete IRQ disable/enable on the grounds that IRQs protect nothing and assume the existing hotplug paths guarantees the PCP cannot be used after zone_pcp_enable(). That should be the case already because all the pages have been freed and there is nothing to even put into the PCPs but I worried that the PCP structure itself might still be reachable even if it's useless which is why I freed the structure once they could not be reached via zonelists.

OK. Let's do that for now and I will put a follow up on my todo list.

Thanks!

Signature · 2 lines
Michal Hocko
SUSE Labs