Skip to contentSkip to conversation

fix two calls of unmap_poisoned_folio() for large folio

PATCH Superseded Jinjiang Tu 7 participants 36 messages Jun 27, 2025 – Aug 25, 2025

Subsystems (5)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

v2 cover fix two calls of unmap_poisoned_folio() for large folio

Fix calling unmap_poisoned_folio() for large folio in shrink_folio_list() and do_migrate_range().

Changelog:
v2:
1) add Closes tag for patch 1
2) add patch 2 to fix calling unmap_poisoned_folio() for large folio in

     do_migrate_range().

Jinjiang Tu (2):
mm/vmscan: fix hwpoisoned large folio handling in shrink_folio_list mm/memory_hotplug: fix hwpoisoned large folio handling in
do_migrate_range

 mm/memory_hotplug.c | 21 +++++++++++++--------
 mm/vmscan.c         |  8 ++++++++
 2 files changed, 21 insertions(+), 8 deletions(-)
Signature · 1 line
2.43.0
#1 lore raw

PATCH v2 1/2 mm/vmscan: fix hwpoisoned large folio handling in shrink_folio_list

In shrink_folio_list(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

Since UCE is rare in real world, and race with reclaimation is more rare, just skipping the hwpoisoned large folio is enough. memory_failure() will handle it if the UCE is triggered again.

Fixes 1b0449544c64 ("mm/vmscan: don't try to reclaim hwpoison folio")
Reported-by [email protected]
---
 mm/vmscan.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/vmscan.c +8 -0
index f8dfd2864bbf..424412680cfc 100644
@@ -1138,6 +1138,14 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 			goto keep;
 
 		if (folio_contain_hwpoisoned_page(folio)) {
+			/*
+			 * unmap_poisoned_folio() can't handle large
+			 * folio, just skip it. memory_failure() will
+			 * handle it if the UCE is triggered again.
+			 */
+			if (folio_test_large(folio))
+				goto keep_locked;
+
 			unmap_poisoned_folio(folio, folio_pfn(folio), false);
 			folio_unlock(folio);
 			folio_put(folio);
Signature · 1 line
2.43.0
Acked-by on v2 1/2
#2 lore raw

Re: [PATCH v2 1/2] mm/vmscan: fix hwpoisoned large folio handling in shrink_folio_list

Quoted 24 lines · Jinjiang Tu

In shrink_folio_list(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

Since UCE is rare in real world, and race with reclaimation is more rare, just skipping the hwpoisoned large folio is enough. memory_failure() will handle it if the UCE is triggered again.

Fixes 1b0449544c64 ("mm/vmscan: don't try to reclaim hwpoison folio")
Reported-by [email protected]
---
  mm/vmscan.c | 8 ++++++++
  1 file changed, 8 insertions(+)
mm/vmscan.c +1 -0
index f8dfd2864bbf..424412680cfc 100644
@@ -1138,6 +1138,14 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
  			goto keep;
 
  		if (folio_contain_hwpoisoned_page(folio)) {
+			/*

mm/vmscan.c static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1151

Quoted 10 lines
mm/vmscan.c +7 -0
+			 * unmap_poisoned_folio() can't handle large
+			 * folio, just skip it. memory_failure() will
+			 * handle it if the UCE is triggered again.
+			 */
+			if (folio_test_large(folio))
+				goto keep_locked;
+
  			unmap_poisoned_folio(folio, folio_pfn(folio), false);
  			folio_unlock(folio);
  			folio_put(folio);
Signature · 3 lines
Cheers,

David / dhildenb
maintainer
#3 lore raw

Re: [PATCH v2 1/2] mm/vmscan: fix hwpoisoned large folio handling in shrink_folio_list

In reply to Jinjiang Tu
Quoted 8 lines · On Fri, 27 Jun 2025 20:57:46 +0800 Jinjiang Tu <[email protected]> wrote:

In shrink_folio_list(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

Since UCE is rare in real world, and race with reclaimation is more rare, just skipping the hwpoisoned large folio is enough. memory_failure() will handle it if the UCE is triggered again.

Fixes 1b0449544c64 ("mm/vmscan: don't try to reclaim hwpoison folio")

Thanks. Can you please fully describe the userspace-visible effects of this bug?

Depending on that information, do you recommend that this fix be backported into -stable kernels?

#4 lore raw

Re: [PATCH v2 1/2] mm/vmscan: fix hwpoisoned large folio handling in shrink_folio_list

Quoted 5 lines · Andrew Morton
Quoted 3 lines · On Fri, 27 Jun 2025 20:57:46 +0800 Jinjiang Tu <[email protected]> wrote:

In shrink_folio_list(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

Quoted 7 lines
Quoted 5 lines

Since UCE is rare in real world, and race with reclaimation is more rare, just skipping the hwpoisoned large folio is enough. memory_failure() will handle it if the UCE is triggered again.

Fixes 1b0449544c64 ("mm/vmscan: don't try to reclaim hwpoison folio")

Thanks. Can you please fully describe the userspace-visible effects of this bug?

This happens when memory reclaimation for large folio races with memory_failure(), and will lead to kernel panic. The race is as following:

cpu0            cpu1
 shrink_folio_list memory_failure
                      TestSetPageHWPoison
    unmap_poisoned_folio
     --> trigger BUG_ON due to
           unmap_poisoned_folio couldn't
           handle large folio

Quoted 2 lines

Depending on that information, do you recommend that this fix be backported into -stable kernels?

The fixed commit 1b0449544c64 has been backported to -stable, so this fix should be backported too.

Quoted 2 lines
Reviewed-by on v2 1/2
#5 lore raw

Re: [PATCH v2 1/2] mm/vmscan: fix hwpoisoned large folio handling in shrink_folio_list

In reply to Jinjiang Tu
Quoted 11 lines · Jinjiang Tu

In shrink_folio_list(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

Since UCE is rare in real world, and race with reclaimation is more rare, just skipping the hwpoisoned large folio is enough. memory_failure() will handle it if the UCE is triggered again.

Fixes 1b0449544c64 ("mm/vmscan: don't try to reclaim hwpoison folio")
Reported-by [email protected]

Thanks for your patch.

Quoted 10 lines
---
 mm/vmscan.c | 8 ++++++++
 1 file changed, 8 insertions(+)
mm/vmscan.c +0 -0
index f8dfd2864bbf..424412680cfc 100644
@@ -1138,6 +1138,14 @@ static unsigned int shrink_folio_list(struct list_head *folio_list,
 			goto keep;

mm/vmscan.c static unsigned int shrink_folio_list(struct list_head *folio_list, · line 1148

Quoted 10 lines
mm/vmscan.c +8 -0
 
 		if (folio_contain_hwpoisoned_page(folio)) {
+			/*
+			 * unmap_poisoned_folio() can't handle large
+			 * folio, just skip it. memory_failure() will
+			 * handle it if the UCE is triggered again.
+			 */
+			if (folio_test_large(folio))
+				goto keep_locked;
+

I think this should work. The hwpoisoned folio is kept in the LRU list so it could be splitted later while the hwpoisoned flag for folio and sub-pages are still correct. And folio_mc_copy will handle re-accessing the hwpoisoned page when one of its caller, alloc_contig_range_noprof, calls migrate_pages.

Thanks.
.

maintainer Acked-by on v2 1/2
#6 lore raw

Re: [PATCH v2 1/2] mm/vmscan: fix hwpoisoned large folio handling in shrink_folio_list

In reply to Jinjiang Tu
Quoted 2 lines · Jinjiang Tu

In shrink_folio_list(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

Uhm, why can't be handled? I think we should spell it out, and maybe even put a comment above unmap_poisoned_folio.

Quoted 8 lines

Since UCE is rare in real world, and race with reclaimation is more rare, just skipping the hwpoisoned large folio is enough. memory_failure() will handle it if the UCE is triggered again.

Fixes 1b0449544c64 ("mm/vmscan: don't try to reclaim hwpoison folio")
Reported-by [email protected]

With the above addressed:

Signature · 2 lines
Oscar Salvador
SUSE Labs
#7 lore raw

Re: [PATCH v2 1/2] mm/vmscan: fix hwpoisoned large folio handling in shrink_folio_list

Quoted 5 lines · Oscar Salvador
Quoted 2 lines · Jinjiang Tu

In shrink_folio_list(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

Uhm, why can't be handled? I think we should spell it out, and maybe even put a comment above unmap_poisoned_folio.

This is disscussed in https://lore.kernel.org/all/[email protected]/
I will add more comments to explain it in the patch.

Thanks.

Quoted 13 lines
Quoted 8 lines

Since UCE is rare in real world, and race with reclaimation is more rare, just skipping the hwpoisoned large folio is enough. memory_failure() will handle it if the UCE is triggered again.

Fixes 1b0449544c64 ("mm/vmscan: don't try to reclaim hwpoison folio")
Reported-by [email protected]

With the above addressed:

#8 lore raw

PATCH v2 2/2 mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

In reply to Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
 <TASK>
 rmap_walk_anon+0xda/0x1f0
 try_to_unmap+0x78/0x80
 ? __pfx_try_to_unmap_one+0x10/0x10
 ? __pfx_folio_not_mapped+0x10/0x10
 ? __pfx_folio_lock_anon_vma_read+0x10/0x10
 unmap_poisoned_folio+0x60/0x140
 do_migrate_range+0x4d1/0x600
 ? slab_memory_callback+0x6a/0x190
 ? notifier_call_chain+0x56/0xb0
 offline_pages+0x3e6/0x460
 memory_subsys_offline+0x130/0x1f0
 device_offline+0xba/0x110
 acpi_bus_offline+0xb7/0x130
 acpi_scan_hot_remove+0x77/0x290
 acpi_device_hotplug+0x1e0/0x240
 acpi_hotplug_work_fn+0x1a/0x30
 process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")
---
 mm/memory_hotplug.c | 21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)
mm/memory_hotplug.c +13 -8
index b1caedbade5b..5fab212e3a2c 100644
@@ -1795,7 +1795,7 @@ static int scan_movable_pages(unsigned long start, unsigned long end,
 	return 0;
 }
 
-static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
+static int do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 {
 	struct folio *folio;
 	unsigned long pfn;
@@ -1819,8 +1819,10 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 			pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;
 
 		if (folio_contain_hwpoisoned_page(folio)) {
-			if (WARN_ON(folio_test_lru(folio)))
-				folio_isolate_lru(folio);
+			if (folio_test_large(folio) && !folio_test_hugetlb(folio))
+				goto err_out;
+			if (folio_test_lru(folio) && !folio_isolate_lru(folio))
+				goto err_out;
 			if (folio_mapped(folio)) {
 				folio_lock(folio);
 				unmap_poisoned_folio(folio, pfn, false);
@@ -1877,6 +1879,11 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 			putback_movable_pages(&source);
 		}
 	}
+	return 0;
+err_out:
+	folio_put(folio);
+	putback_movable_pages(&source);
+	return -EBUSY;
 }
 
 static int __init cmdline_parse_movable_node(char *p)
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 
 			ret = scan_movable_pages(pfn, end_pfn, &pfn);
 			if (!ret) {
-				/*
-				 * TODO: fatal migration failures should bail
-				 * out
-				 */
-				do_migrate_range(pfn, end_pfn);
+				ret = do_migrate_range(pfn, end_pfn);
+				if (ret)
+					break;
 			}
 		} while (!ret);
 
Signature · 1 line
2.43.0
maintainer
#9 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 36 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
 <TASK>
 rmap_walk_anon+0xda/0x1f0
 try_to_unmap+0x78/0x80
 ? __pfx_try_to_unmap_one+0x10/0x10
 ? __pfx_folio_not_mapped+0x10/0x10
 ? __pfx_folio_lock_anon_vma_read+0x10/0x10
 unmap_poisoned_folio+0x60/0x140
 do_migrate_range+0x4d1/0x600
 ? slab_memory_callback+0x6a/0x190
 ? notifier_call_chain+0x56/0xb0
 offline_pages+0x3e6/0x460
 memory_subsys_offline+0x130/0x1f0
 device_offline+0xba/0x110
 acpi_bus_offline+0xb7/0x130
 acpi_scan_hot_remove+0x77/0x290
 acpi_device_hotplug+0x1e0/0x240
 acpi_hotplug_work_fn+0x1a/0x30
 process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")

...

mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, unsigned long nr_pages, · line 2053

Quoted 12 lines
mm/memory_hotplug.c +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 
 			ret = scan_movable_pages(pfn, end_pfn, &pfn);
 			if (!ret) {
-				/*
-				 * TODO: fatal migration failures should bail
-				 * out
-				 */
-				do_migrate_range(pfn, end_pfn);
+				ret = do_migrate_range(pfn, end_pfn);
+				if (ret)
+					break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can also return "fatal" errors and we don't propagate that.

The moto has always been to migrate as much as possible, and this changes this behaviour.

Signature · 2 lines
Oscar Salvador
SUSE Labs
#10 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 54 lines · Oscar Salvador
Quoted 36 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
  <TASK>
  rmap_walk_anon+0xda/0x1f0
  try_to_unmap+0x78/0x80
  ? __pfx_try_to_unmap_one+0x10/0x10
  ? __pfx_folio_not_mapped+0x10/0x10
  ? __pfx_folio_lock_anon_vma_read+0x10/0x10
  unmap_poisoned_folio+0x60/0x140
  do_migrate_range+0x4d1/0x600
  ? slab_memory_callback+0x6a/0x190
  ? notifier_call_chain+0x56/0xb0
  offline_pages+0x3e6/0x460
  memory_subsys_offline+0x130/0x1f0
  device_offline+0xba/0x110
  acpi_bus_offline+0xb7/0x130
  acpi_scan_hot_remove+0x77/0x290
  acpi_device_hotplug+0x1e0/0x240
  acpi_hotplug_work_fn+0x1a/0x30
  process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")

...

mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, unsigned long nr_pages, · line 2053

Quoted 12 lines
mm/memory_hotplug.c +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 
  			ret = scan_movable_pages(pfn, end_pfn, &pfn);
  			if (!ret) {
-				/*
-				 * TODO: fatal migration failures should bail
-				 * out
-				 */
-				do_migrate_range(pfn, end_pfn);
+				ret = do_migrate_range(pfn, end_pfn);
+				if (ret)
+					break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can also return "fatal" errors and we don't propagate that.

Quoted 3 lines

The moto has always been to migrate as much as possible, and this changes this behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless util received signal. It seems there is no document to guarantee memory offline have to migrate as much as possible.

Quoted 1 line
#11 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 56 lines · Jinjiang Tu
Quoted 54 lines · Oscar Salvador
Quoted 36 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
   <TASK>
   rmap_walk_anon+0xda/0x1f0
   try_to_unmap+0x78/0x80
   ? __pfx_try_to_unmap_one+0x10/0x10
   ? __pfx_folio_not_mapped+0x10/0x10
   ? __pfx_folio_lock_anon_vma_read+0x10/0x10
   unmap_poisoned_folio+0x60/0x140
   do_migrate_range+0x4d1/0x600
   ? slab_memory_callback+0x6a/0x190
   ? notifier_call_chain+0x56/0xb0
   offline_pages+0x3e6/0x460
   memory_subsys_offline+0x130/0x1f0
   device_offline+0xba/0x110
   acpi_bus_offline+0xb7/0x130
   acpi_scan_hot_remove+0x77/0x290
   acpi_device_hotplug+0x1e0/0x240
   acpi_hotplug_work_fn+0x1a/0x30
   process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")

...

mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, unsigned long nr_pages, · line 2053

Quoted 12 lines
mm/memory_hotplug.c +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 
   			ret = scan_movable_pages(pfn, end_pfn, &pfn);
   			if (!ret) {
-				/*
-				 * TODO: fatal migration failures should bail
-				 * out
-				 */
-				do_migrate_range(pfn, end_pfn);
+				ret = do_migrate_range(pfn, end_pfn);
+				if (ret)
+					break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can also return "fatal" errors and we don't propagate that.

Quoted 5 lines
Quoted 3 lines

The moto has always been to migrate as much as possible, and this changes this behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless util received signal.

Yeah, that's also not good,

Quoted 2 lines

It seems there is no document to guarantee memory offline have to migrate as much as possible.

We should try offlining as good as possible. But if there is something we just cannot possibly migrate, there is no sense in retrying.

Now, could we run into this case here because we are racing with other code, and actually retrying again could make it work?

Remind me again: how exactly do we arrive at this point of having a large folio that is hwpoisoned but still mapped?

In memory_failure(), we do on a large folio

1) folio_set_has_hwpoisoned
2) try_to_split_thp_page
3) if splitting fails, kill_procs_now

So given that, couldn't we just retry the migration until the race is over and we are good?

Signature · 3 lines
Cheers,

David / dhildenb
#12 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 71 lines · David Hildenbrand
Quoted 56 lines · Jinjiang Tu
Quoted 54 lines · Oscar Salvador
Quoted 36 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
   <TASK>
   rmap_walk_anon+0xda/0x1f0
   try_to_unmap+0x78/0x80
   ? __pfx_try_to_unmap_one+0x10/0x10
   ? __pfx_folio_not_mapped+0x10/0x10
   ? __pfx_folio_lock_anon_vma_read+0x10/0x10
   unmap_poisoned_folio+0x60/0x140
   do_migrate_range+0x4d1/0x600
   ? slab_memory_callback+0x6a/0x190
   ? notifier_call_chain+0x56/0xb0
   offline_pages+0x3e6/0x460
   memory_subsys_offline+0x130/0x1f0
   device_offline+0xba/0x110
   acpi_bus_offline+0xb7/0x130
   acpi_scan_hot_remove+0x77/0x290
   acpi_device_hotplug+0x1e0/0x240
   acpi_hotplug_work_fn+0x1a/0x30
   process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")

...

mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, · line 2053

Quoted 12 lines
mm/memory_hotplug.c +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn, 
 unsigned long nr_pages,
                   ret = scan_movable_pages(pfn, end_pfn, &pfn);
                if (!ret) {
-                /*
-                 * TODO: fatal migration failures should bail
-                 * out
-                 */
-                do_migrate_range(pfn, end_pfn);
+                ret = do_migrate_range(pfn, end_pfn);
+                if (ret)
+                    break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can also return "fatal" errors and we don't propagate that.

Quoted 4 lines
Quoted 2 lines

The moto has always been to migrate as much as possible, and this changes this behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless util received signal.

Yeah, that's also not good,

Quoted 2 lines

It seems there is no document to guarantee memory offline have to migrate as much as possible.

We should try offlining as good as possible. But if there is something we just cannot possibly migrate, there is no sense in retrying.

Now, could we run into this case here because we are racing with other code, and actually retrying again could make it work?

Quoted 7 lines

Remind me again: how exactly do we arrive at this point of having a large folio that is hwpoisoned but still mapped?

In memory_failure(), we do on a  large folio

1) folio_set_has_hwpoisoned
2) try_to_split_thp_page
3) if splitting fails, kill_procs_now

If 2) is executed when do_migrate_range() increment the refcount of the folio, the split fails, and retry is meaningless. Since we don't know if memory_failure() finishes, we couldn't know when to abort retry.

Quoted 3 lines

So given that, couldn't we just retry the migration until the race is over and we are good?

#13 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 84 lines · Jinjiang Tu
Quoted 82 lines · David Hildenbrand
Quoted 61 lines · Jinjiang Tu
Quoted 59 lines · Oscar Salvador
Quoted 40 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
    <TASK>
    rmap_walk_anon+0xda/0x1f0
    try_to_unmap+0x78/0x80
    ? __pfx_try_to_unmap_one+0x10/0x10
    ? __pfx_folio_not_mapped+0x10/0x10
    ? __pfx_folio_lock_anon_vma_read+0x10/0x10
    unmap_poisoned_folio+0x60/0x140
    do_migrate_range+0x4d1/0x600
    ? slab_memory_callback+0x6a/0x190
    ? notifier_call_chain+0x56/0xb0
    offline_pages+0x3e6/0x460
    memory_subsys_offline+0x130/0x1f0
    device_offline+0xba/0x110
    acpi_bus_offline+0xb7/0x130
    acpi_scan_hot_remove+0x77/0x290
    acpi_device_hotplug+0x1e0/0x240
    acpi_hotplug_work_fn+0x1a/0x30
    process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other
places, unmap_poisoned_folio() is called when the folio is isolated, obey
it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned

pages to be offlined")

...

mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, · line 2053

Quoted 12 lines
mm/memory_hotplug.c +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn,
 unsigned long nr_pages,
                   ret = scan_movable_pages(pfn, end_pfn, &pfn);
                if (!ret) {
-                /*
-                 * TODO: fatal migration failures should bail
-                 * out
-                 */
-                do_migrate_range(pfn, end_pfn);
+                ret = do_migrate_range(pfn, end_pfn);
+                if (ret)
+                    break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can also return
"fatal" errors and we don't propagate that.

Quoted 5 lines
Quoted 3 lines

The moto has always been to migrate as much as possible, and this changes this
behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless util received signal.

Yeah, that's also not good,

Quoted 2 lines

It seems there is no document to guarantee memory offline have to migrate as much as possible.

We should try offlining as good as possible. But if there is something we just cannot possibly migrate, there is no sense in retrying.

Now, could we run into this case here because we are racing with other code, and actually retrying again could make it work?

Remind me again: how exactly do we arrive at this point of having a large folio that is hwpoisoned but still mapped?

Quoted 7 lines
Quoted 5 lines

In memory_failure(), we do on a  large folio

1) folio_set_has_hwpoisoned
2) try_to_split_thp_page
3) if splitting fails, kill_procs_now

If 2) is executed when do_migrate_range() increment the refcount of the folio, the split fails, and retry is meaningless.

kill_procs_now will kill all processes, effectively unmapping the folio in that case?

So retrying would later just ... get us an unmapped folio and we can make progress?

Signature · 3 lines
Cheers,

David / dhildenb
#14 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 90 lines · David Hildenbrand
Quoted 89 lines · Jinjiang Tu
Quoted 87 lines · David Hildenbrand
Quoted 61 lines · Jinjiang Tu
Quoted 59 lines · Oscar Salvador
Quoted 40 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
    <TASK>
    rmap_walk_anon+0xda/0x1f0
    try_to_unmap+0x78/0x80
    ? __pfx_try_to_unmap_one+0x10/0x10
    ? __pfx_folio_not_mapped+0x10/0x10
    ? __pfx_folio_lock_anon_vma_read+0x10/0x10
    unmap_poisoned_folio+0x60/0x140
    do_migrate_range+0x4d1/0x600
    ? slab_memory_callback+0x6a/0x190
    ? notifier_call_chain+0x56/0xb0
    offline_pages+0x3e6/0x460
    memory_subsys_offline+0x130/0x1f0
    device_offline+0xba/0x110
    acpi_bus_offline+0xb7/0x130
    acpi_scan_hot_remove+0x77/0x290
    acpi_device_hotplug+0x1e0/0x240
    acpi_hotplug_work_fn+0x1a/0x30
    process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other
places, unmap_poisoned_folio() is called when the folio is isolated, obey
it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned

pages to be offlined")

...

mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, · line 2053

Quoted 12 lines
mm/memory_hotplug.c +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn,
 unsigned long nr_pages,
                    ret = scan_movable_pages(pfn, end_pfn, &pfn);
                 if (!ret) {
-                /*
-                 * TODO: fatal migration failures should bail
-                 * out
-                 */
-                do_migrate_range(pfn, end_pfn);
+                ret = do_migrate_range(pfn, end_pfn);
+                if (ret)
+                    break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can also return
"fatal" errors and we don't propagate that.

Quoted 5 lines
Quoted 3 lines

The moto has always been to migrate as much as possible, and this changes this
behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless util received signal.

Yeah, that's also not good,

Quoted 2 lines

It seems there is no document to guarantee memory offline have to migrate as much as possible.

We should try offlining as good as possible. But if there is something we just cannot possibly migrate, there is no sense in retrying.

Now, could we run into this case here because we are racing with other code, and actually retrying again could make it work?

Remind me again: how exactly do we arrive at this point of having a large folio that is hwpoisoned but still mapped?

In memory_failure(), we do on a  large folio

1) folio_set_has_hwpoisoned
2) try_to_split_thp_page
3) if splitting fails, kill_procs_now

Quoted 7 lines
Quoted 2 lines

If 2) is executed when do_migrate_range() increment the refcount of the folio, the split fails, and retry is meaningless.

kill_procs_now will kill all processes, effectively unmapping the folio in that case?

So retrying would later just ... get us an unmapped folio and we can make progress?

kill_procs_now()->collect_procs() collects the tasks to kill. But not all tasks that maps the folio will be collected, collect_procs_anon()->task_early_kill()->find_early_kill_thread() will not select the task (not current) if PF_MCE_PROCESS isn't set and sysctl_memory_failure_early_kill isn't enabled (this is the default behaviour).

#15 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 102 lines · Jinjiang Tu
Quoted 91 lines · David Hildenbrand
Quoted 90 lines · Jinjiang Tu
Quoted 88 lines · David Hildenbrand
Quoted 62 lines · Jinjiang Tu
Quoted 60 lines · Oscar Salvador
Quoted 41 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which
can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
     <TASK>
     rmap_walk_anon+0xda/0x1f0
     try_to_unmap+0x78/0x80
     ? __pfx_try_to_unmap_one+0x10/0x10
     ? __pfx_folio_not_mapped+0x10/0x10
     ? __pfx_folio_lock_anon_vma_read+0x10/0x10
     unmap_poisoned_folio+0x60/0x140
     do_migrate_range+0x4d1/0x600
     ? slab_memory_callback+0x6a/0x190
     ? notifier_call_chain+0x56/0xb0
     offline_pages+0x3e6/0x460
     memory_subsys_offline+0x130/0x1f0
     device_offline+0xba/0x110
     acpi_bus_offline+0xb7/0x130
     acpi_scan_hot_remove+0x77/0x290
     acpi_device_hotplug+0x1e0/0x240
     acpi_hotplug_work_fn+0x1a/0x30
     process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other
places, unmap_poisoned_folio() is called when the folio is isolated, obey
it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned

pages to be offlined")

...

Review int offline_pages(unsigned long start_pfn, · line 2053

Quoted 12 lines
(quoted diff) +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn,
 unsigned long nr_pages,
                    ret = scan_movable_pages(pfn, end_pfn, &pfn);
                 if (!ret) {
-                /*
-                 * TODO: fatal migration failures should bail
-                 * out
-                 */
-                do_migrate_range(pfn, end_pfn);
+                ret = do_migrate_range(pfn, end_pfn);
+                if (ret)
+                    break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can also return
"fatal" errors and we don't propagate that.

Quoted 5 lines
Quoted 3 lines

The moto has always been to migrate as much as possible, and this changes this
behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless util received signal.

Yeah, that's also not good,

Quoted 2 lines

It seems there is no document to guarantee memory offline have to migrate as much as possible.

We should try offlining as good as possible. But if there is something we just cannot possibly migrate, there is no sense in retrying.

Now, could we run into this case here because we are racing with other code, and actually retrying again could make it work?

Remind me again: how exactly do we arrive at this point of having a large folio that is hwpoisoned but still mapped?

In memory_failure(), we do on a  large folio

1) folio_set_has_hwpoisoned
2) try_to_split_thp_page
3) if splitting fails, kill_procs_now

Quoted 9 lines
Quoted 2 lines

If 2) is executed when do_migrate_range() increment the refcount of the folio, the split fails, and retry is meaningless.

kill_procs_now will kill all processes, effectively unmapping the folio in that case?

So retrying would later just ... get us an unmapped folio and we can make progress?

Quoted 7 lines

kill_procs_now()->collect_procs() collects the tasks to kill. But not all tasks that maps the folio
will be collected, collect_procs_anon()->task_early_kill()->find_early_kill_thread() will not select the task (not current) if PF_MCE_PROCESS isn't set and sysctl_memory_failure_early_kill
isn't enabled (this is the default behaviour).

I think you're right, that's rather nasty.

We fail to split, but keep the folio mapped into some processes.

And we can't unmap it because unmap_poisoned_folio() does not properly support large folios yet.

We really should unmap the folio when splitting fail. :(

Signature · 3 lines
Cheers,

David / dhildenb
#16 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 111 lines · David Hildenbrand
Quoted 102 lines · Jinjiang Tu
Quoted 91 lines · David Hildenbrand
Quoted 90 lines · Jinjiang Tu
Quoted 88 lines · David Hildenbrand
Quoted 62 lines · Jinjiang Tu
Quoted 60 lines · Oscar Salvador
Quoted 41 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which
can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
     <TASK>
     rmap_walk_anon+0xda/0x1f0
     try_to_unmap+0x78/0x80
     ? __pfx_try_to_unmap_one+0x10/0x10
     ? __pfx_folio_not_mapped+0x10/0x10
     ? __pfx_folio_lock_anon_vma_read+0x10/0x10
     unmap_poisoned_folio+0x60/0x140
     do_migrate_range+0x4d1/0x600
     ? slab_memory_callback+0x6a/0x190
     ? notifier_call_chain+0x56/0xb0
     offline_pages+0x3e6/0x460
     memory_subsys_offline+0x130/0x1f0
     device_offline+0xba/0x110
     acpi_bus_offline+0xb7/0x130
     acpi_scan_hot_remove+0x77/0x290
     acpi_device_hotplug+0x1e0/0x240
     acpi_hotplug_work_fn+0x1a/0x30
     process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other
places, unmap_poisoned_folio() is called when the folio is isolated, obey
it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned

pages to be offlined")

...

Review int offline_pages(unsigned long start_pfn, · line 2053

Quoted 12 lines
(quoted diff) +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn,
 unsigned long nr_pages,
                     ret = scan_movable_pages(pfn, end_pfn, &pfn);
                  if (!ret) {
-                /*
-                 * TODO: fatal migration failures should bail
-                 * out
-                 */
-                do_migrate_range(pfn, end_pfn);
+                ret = do_migrate_range(pfn, end_pfn);
+                if (ret)
+                    break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can also return
"fatal" errors and we don't propagate that.

Quoted 5 lines
Quoted 3 lines

The moto has always been to migrate as much as possible, and this changes this
behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless util received signal.

Yeah, that's also not good,

Quoted 2 lines

It seems there is no document to guarantee memory offline have to migrate as much as possible.

We should try offlining as good as possible. But if there is something we just cannot possibly migrate, there is no sense in retrying.

Now, could we run into this case here because we are racing with other code, and actually retrying again could make it work?

Remind me again: how exactly do we arrive at this point of having a large folio that is hwpoisoned but still mapped?

In memory_failure(), we do on a  large folio

1) folio_set_has_hwpoisoned
2) try_to_split_thp_page
3) if splitting fails, kill_procs_now

Quoted 9 lines
Quoted 2 lines

If 2) is executed when do_migrate_range() increment the refcount of the folio, the split fails, and retry is meaningless.

kill_procs_now will kill all processes, effectively unmapping the folio in that case?

So retrying would later just ... get us an unmapped folio and we can make progress?

Quoted 7 lines

kill_procs_now()->collect_procs() collects the tasks to kill. But not all tasks that maps the folio
will be collected, collect_procs_anon()->task_early_kill()->find_early_kill_thread() will not select the task (not current) if PF_MCE_PROCESS isn't set and sysctl_memory_failure_early_kill
isn't enabled (this is the default behaviour).

Quoted 8 lines

I think you're right, that's rather nasty.

We fail to split, but keep the folio mapped into some processes.

And we can't unmap it because unmap_poisoned_folio() does not properly support large folios yet.

We really should unmap the folio when splitting fail. :(

unmap_poisoned_folio() doesn't guarantee the folio is unmapped successfully, according to the return val. Although I don't know in which case we will fail to unmap.

#17 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 124 lines · Jinjiang Tu
Quoted 122 lines · David Hildenbrand
Quoted 108 lines · Jinjiang Tu
Quoted 100 lines · David Hildenbrand
Quoted 90 lines · Jinjiang Tu
Quoted 88 lines · David Hildenbrand
Quoted 64 lines · Jinjiang Tu
Quoted 62 lines · Oscar Salvador
Quoted 42 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which
can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
      <TASK>
      rmap_walk_anon+0xda/0x1f0
      try_to_unmap+0x78/0x80
      ? __pfx_try_to_unmap_one+0x10/0x10
      ? __pfx_folio_not_mapped+0x10/0x10
      ? __pfx_folio_lock_anon_vma_read+0x10/0x10
      unmap_poisoned_folio+0x60/0x140
      do_migrate_range+0x4d1/0x600
      ? slab_memory_callback+0x6a/0x190
      ? notifier_call_chain+0x56/0xb0
      offline_pages+0x3e6/0x460
      memory_subsys_offline+0x130/0x1f0
      device_offline+0xba/0x110
      acpi_bus_offline+0xb7/0x130
      acpi_scan_hot_remove+0x77/0x290
      acpi_device_hotplug+0x1e0/0x240
      acpi_hotplug_work_fn+0x1a/0x30
      process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set
hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other
places, unmap_poisoned_folio() is called when the folio is isolated, obey
it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned

pages to be offlined")

...

Review int offline_pages(unsigned long start_pfn, · line 2053

Quoted 12 lines
(quoted diff) +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn,
 unsigned long nr_pages,
                     ret = scan_movable_pages(pfn, end_pfn, &pfn);
                  if (!ret) {
-                /*
-                 * TODO: fatal migration failures should bail
-                 * out
-                 */
-                do_migrate_range(pfn, end_pfn);
+                ret = do_migrate_range(pfn, end_pfn);
+                if (ret)
+                    break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can
also return
"fatal" errors and we don't propagate that.

Quoted 6 lines
Quoted 3 lines

The moto has always been to migrate as much as possible, and this changes this
behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless
util received signal.

Yeah, that's also not good,

Quoted 2 lines

It seems there is no document to guarantee memory offline have to migrate as much as possible.

We should try offlining as good as possible. But if there is something
we just cannot possibly migrate, there is no sense in retrying.

Now, could we run into this case here because we are racing with other
code, and actually retrying again could make it work?

Remind me again: how exactly do we arrive at this point of having a large folio that is hwpoisoned but still mapped?

Quoted 8 lines
Quoted 5 lines

In memory_failure(), we do on a  large folio

1) folio_set_has_hwpoisoned
2) try_to_split_thp_page
3) if splitting fails, kill_procs_now

If 2) is executed when do_migrate_range() increment the refcount of the
folio, the split fails, and retry is meaningless.

Quoted 6 lines

kill_procs_now will kill all processes, effectively unmapping the folio in that case?

So retrying would later just ... get us an unmapped folio and we can make progress?

Quoted 8 lines

kill_procs_now()->collect_procs() collects the tasks to kill. But not all tasks that maps the folio
will be collected, collect_procs_anon()->task_early_kill()->find_early_kill_thread() will not
select the task (not current) if PF_MCE_PROCESS isn't set and sysctl_memory_failure_early_kill
isn't enabled (this is the default behaviour).

I think you're right, that's rather nasty.

We fail to split, but keep the folio mapped into some processes.

Quoted 8 lines
Quoted 4 lines

And we can't unmap it because unmap_poisoned_folio() does not properly support large folios yet.

We really should unmap the folio when splitting fail. :(

unmap_poisoned_folio() doesn't guarantee the folio is unmapped successfully, according
to the return val. Although I don't know in which case we will fail to unmap.

I think there are only cases for anon folios, when the folio is in the swapcache or it's lazyfree (!swapbacked) and we run into the walk_abort cases. Retrying will likely make it work in many cases I assume.

This is all really rather suboptimal and, I'm afraid, requires much bigger rework to improve it.

Failing memory offlining is also not nice.

I was wondering whether we can just keep splitting the folio until it works:

mm/memory_hotplug.c +13 -0
index b1caedbade5b1..991b095ac7e78 100644
@@ -1819,6 +1819,19 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
                        pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;
 
                if (folio_contain_hwpoisoned_page(folio)) {
+                       /*
+                        * unmap_poisoned_folio() cannot handle THPs, so
+                        * keep trying to split first.
+                        */
+                       if (folio_test_large(folio) && !folio_test_hugetlb(folio)) {
+                               folio_lock(folio);
+                               split_huge_page_to_list_to_order(&folio->page,
+                                                                NULL,
+                                                                min_order_for_split(folio));
+                               folio_unlock(folio);
+                               if (folio_test_large(folio))
+                                       goto put_folio;
+                       }
                        if (WARN_ON(folio_test_lru(folio)))
                                folio_isolate_lru(folio);
                        if (folio_mapped(folio)) {

Probably the WARN_ON can indeed trigger now.

@Zi Yan, on a related note ...

in memory_failure(), we call try_to_split_thp_page(). If it works, we assume that we have a small folio.

But that is not the case if split_huge_page() cannot split it to order-0 ... min_order_for_split().

I'm afraid we havbe more such code that does not expect that if split_huge_page() succeeds that we still have a large folio ...

Signature · 3 lines
Cheers,

David / dhildenb
#18 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 172 lines · David Hildenbrand
Quoted 124 lines · Jinjiang Tu
Quoted 122 lines · David Hildenbrand
Quoted 108 lines · Jinjiang Tu
Quoted 100 lines · David Hildenbrand
Quoted 90 lines · Jinjiang Tu
Quoted 88 lines · David Hildenbrand
Quoted 64 lines · Jinjiang Tu
Quoted 62 lines · Oscar Salvador
Quoted 42 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which
can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
      <TASK>
      rmap_walk_anon+0xda/0x1f0
      try_to_unmap+0x78/0x80
      ? __pfx_try_to_unmap_one+0x10/0x10
      ? __pfx_folio_not_mapped+0x10/0x10
      ? __pfx_folio_lock_anon_vma_read+0x10/0x10
      unmap_poisoned_folio+0x60/0x140
      do_migrate_range+0x4d1/0x600
      ? slab_memory_callback+0x6a/0x190
      ? notifier_call_chain+0x56/0xb0
      offline_pages+0x3e6/0x460
      memory_subsys_offline+0x130/0x1f0
      device_offline+0xba/0x110
      acpi_bus_offline+0xb7/0x130
      acpi_scan_hot_remove+0x77/0x290
      acpi_device_hotplug+0x1e0/0x240
      acpi_hotplug_work_fn+0x1a/0x30
      process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set
hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other
places, unmap_poisoned_folio() is called when the folio is isolated, obey
it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned

pages to be offlined")

...

Review int offline_pages(unsigned long start_pfn, · line 2053

Quoted 12 lines
(quoted diff) +3 -5
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn,
 unsigned long nr_pages,
                     ret = scan_movable_pages(pfn, end_pfn, &pfn);
                  if (!ret) {
-                /*
-                 * TODO: fatal migration failures should bail
-                 * out
-                 */
-                do_migrate_range(pfn, end_pfn);
+                ret = do_migrate_range(pfn, end_pfn);
+                if (ret)
+                    break;

I am not really sure about this one.
I get the reason you're adding it, but note that migrate_pages() can
also return
"fatal" errors and we don't propagate that.

Quoted 6 lines
Quoted 3 lines

The moto has always been to migrate as much as possible, and this changes this
behaviour.

If we just skip to next pfn, offline_pages() will deadloop meaningless
util received signal.

Yeah, that's also not good,

Quoted 2 lines

It seems there is no document to guarantee memory offline have to migrate as much as possible.

We should try offlining as good as possible. But if there is something
we just cannot possibly migrate, there is no sense in retrying.

Now, could we run into this case here because we are racing with other
code, and actually retrying again could make it work?

Remind me again: how exactly do we arrive at this point of having a large folio that is hwpoisoned but still mapped?

Quoted 8 lines
Quoted 5 lines

In memory_failure(), we do on a  large folio

1) folio_set_has_hwpoisoned
2) try_to_split_thp_page
3) if splitting fails, kill_procs_now

If 2) is executed when do_migrate_range() increment the refcount of the
folio, the split fails, and retry is meaningless.

Quoted 6 lines

kill_procs_now will kill all processes, effectively unmapping the folio in that case?

So retrying would later just ... get us an unmapped folio and we can make progress?

Quoted 8 lines

kill_procs_now()->collect_procs() collects the tasks to kill. But not all tasks that maps the folio
will be collected, collect_procs_anon()->task_early_kill()->find_early_kill_thread() will not
select the task (not current) if PF_MCE_PROCESS isn't set and sysctl_memory_failure_early_kill
isn't enabled (this is the default behaviour).

I think you're right, that's rather nasty.

We fail to split, but keep the folio mapped into some processes.

Quoted 8 lines
Quoted 4 lines

And we can't unmap it because unmap_poisoned_folio() does not properly support large folios yet.

We really should unmap the folio when splitting fail. :(

unmap_poisoned_folio() doesn't guarantee the folio is unmapped successfully, according
to the return val. Although I don't know in which case we will fail to unmap.

I think there are only cases for anon folios, when the folio is in the swapcache or it's lazyfree (!swapbacked) and we run into the walk_abort cases. Retrying will likely make it work in many cases I assume.

This is all really rather suboptimal and, I'm afraid, requires much bigger rework to improve it.

Failing memory offlining is also not nice.

I was wondering whether we can just keep splitting the folio until it works:

mm/memory_hotplug.c +13 -0
index b1caedbade5b1..991b095ac7e78 100644
@@ -1819,6 +1819,19 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
                        pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;
                 if (folio_contain_hwpoisoned_page(folio)) {
+                       /*
+                        * unmap_poisoned_folio() cannot handle THPs, so
+                        * keep trying to split first.
+                        */
+                       if (folio_test_large(folio) && !folio_test_hugetlb(folio)) {
+                               folio_lock(folio);
+                               split_huge_page_to_list_to_order(&folio->page,
+                                                                NULL,
+                                                                min_order_for_split(folio));
+                               folio_unlock(folio);
+                               if (folio_test_large(folio))
+                                       goto put_folio;
+                       }
                        if (WARN_ON(folio_test_lru(folio)))
                                folio_isolate_lru(folio);
                        if (folio_mapped(folio)) {
 

Probably the WARN_ON can indeed trigger now.

Quoted 7 lines

@Zi Yan, on a related note ...

in memory_failure(), we call try_to_split_thp_page(). If it works, we assume that we have a small folio.

But that is not the case if split_huge_page() cannot split it to order-0 ... min_order_for_split().

Right. memory failure needs to learn about this. Either poison every subpage or write back if necessary and drop the page cache folio.

Quoted 3 lines

I'm afraid we havbe more such code that does not expect that if split_huge_page() succeeds that we still have a large folio ...

I did some search, here are the users of split_huge_page*():

1. ksm: since it is anonymous only, so split always goes to order-0;
2. userfaultfd: it is also anonymous;
3. madvise cold or pageout: a large pagecache folio will be split if it is partially mapped. And it will retry. It might cause a deadlock if the folio has a min order.
4. shmem: split always goes to order-0;
5. memory-failure: see above.

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my understanding of madvise_cold_or_pageout_pte_range(), it seems that it will try to split the folio and expects a order-0 folio after a successful split. But splitting a min-order folio is a nop. It could lead to a deadlock in the code. Or I just get it wrong?

Best Regards,
Yan, Zi

#19 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Hi Zi Yan,

Quoted 28 lines
Quoted 11 lines

Probably the WARN_ON can indeed trigger now.

@Zi Yan, on a related note ...

in memory_failure(), we call try_to_split_thp_page(). If it works, we assume that we have a small folio.

But that is not the case if split_huge_page() cannot split it to order-0 ... min_order_for_split().

Right. memory failure needs to learn about this. Either poison every subpage or write back if necessary and drop the page cache folio.

Quoted 3 lines

I'm afraid we havbe more such code that does not expect that if split_huge_page() succeeds that we still have a large folio ...

I did some search, here are the users of split_huge_page*():

1. ksm: since it is anonymous only, so split always goes to order-0;
2. userfaultfd: it is also anonymous;
3. madvise cold or pageout: a large pagecache folio will be split if it is partially mapped. And it will retry. It might cause a deadlock if the folio has a min order.
4. shmem: split always goes to order-0;
5. memory-failure: see above.

Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

Quoted 4 lines

understanding of madvise_cold_or_pageout_pte_range(), it seems that it will try to split the folio and expects a order-0 folio after a successful split. But splitting a min-order folio is a nop. It could lead to a deadlock in the code. Or I just get it wrong?

Yeah, we have checks to make sure we never split a folio < min-order.

I hope it answers your question :)

--
Pankaj

#20 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 42 lines · Pankaj Raghav

Hi Zi Yan,

Quoted 28 lines
Quoted 11 lines

Probably the WARN_ON can indeed trigger now.

@Zi Yan, on a related note ...

in memory_failure(), we call try_to_split_thp_page(). If it works, we assume that we have a small folio.

But that is not the case if split_huge_page() cannot split it to order-0 ... min_order_for_split().

Right. memory failure needs to learn about this. Either poison every subpage or write back if necessary and drop the page cache folio.

Quoted 3 lines

I'm afraid we havbe more such code that does not expect that if split_huge_page() succeeds that we still have a large folio ...

I did some search, here are the users of split_huge_page*():

1. ksm: since it is anonymous only, so split always goes to order-0;
2. userfaultfd: it is also anonymous;
3. madvise cold or pageout: a large pagecache folio will be split if it is partially mapped. And it will retry. It might cause a deadlock if the folio has a min order.
4. shmem: split always goes to order-0;
5. memory-failure: see above.

Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

Quoted 8 lines
Quoted 4 lines

understanding of madvise_cold_or_pageout_pte_range(), it seems that it will try to split the folio and expects a order-0 folio after a successful split. But splitting a min-order folio is a nop. It could lead to a deadlock in the code. Or I just get it wrong?

Yeah, we have checks to make sure we never split a folio < min-order.

I hope it answers your question :)

Thank you for the explanation. This means madvise cold or pageout case is good.

Best Regards,
Yan, Zi

#21 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 36 lines · Zi Yan
Quoted 34 lines · Pankaj Raghav

Hi Zi Yan,

Quoted 32 lines
Quoted 11 lines

Probably the WARN_ON can indeed trigger now.

@Zi Yan, on a related note ...

in memory_failure(), we call try_to_split_thp_page(). If it works, we assume that we have a small folio.

But that is not the case if split_huge_page() cannot split it to order-0 ... min_order_for_split().

Right. memory failure needs to learn about this. Either poison every subpage or write back if necessary and drop the page cache folio.

Quoted 3 lines

I'm afraid we havbe more such code that does not expect that if split_huge_page() succeeds that we still have a large folio ...

I did some search, here are the users of split_huge_page*():

1. ksm: since it is anonymous only, so split always goes to order-0;
2. userfaultfd: it is also anonymous;
3. madvise cold or pageout: a large pagecache folio will be split if it is partially mapped. And it will retry. It might cause a deadlock if the folio has a min order.
4. shmem: split always goes to order-0;
5. memory-failure: see above.

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Quoted 7 lines
Quoted 7 lines
Quoted 1 line

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

I'm confused. Shouldn't this be trivially possible?

E.g., just mmap() a single page of such a file? Who would make that fail?

And if it doesn't fail, who would stop us from munmap()'ing everything except a single page.

Signature · 3 lines
Cheers,

David / dhildenb
#22 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

I'm confused. Shouldn't this be trivially possible?

Hmm, maybe I misunderstood the question?

Quoted 2 lines

E.g., just mmap() a single page of such a file? Who would make that fail?

My point was, even if you try to mmap a single page of a file, page cache will read the whole block (that corresponds to min order folio).

Technically we can mmap a single page of file, but FS will always read and write **at least** in min folio order chunks.

Quoted 9 lines

And if it doesn't fail, who would stop us from munmap()'ing everything except a single page.

Signature · 3 lines
Cheers,

David / dhildenb

--
Pankaj

#23 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 16 lines · Pankaj Raghav (Samsung)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

I'm confused. Shouldn't this be trivially possible?

Hmm, maybe I misunderstood the question?

Quoted 8 lines
Quoted 2 lines

E.g., just mmap() a single page of such a file? Who would make that fail?

My point was, even if you try to mmap a single page of a file, page cache will read the whole block (that corresponds to min order folio).

Technically we can mmap a single page of file, but FS will always read and write **at least** in min folio order chunks.

Okay, so it can be partially mapped into page tables :) What happens in the background (page cache management) is a different story

Signature · 3 lines
Cheers,

David / dhildenb
#24 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 20 lines · David Hildenbrand
Quoted 19 lines · Pankaj Raghav (Samsung)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

I'm confused. Shouldn't this be trivially possible?

Hmm, maybe I misunderstood the question?

Quoted 2 lines

E.g., just mmap() a single page of such a file? Who would make that fail?

Quoted 7 lines
Quoted 5 lines

My point was, even if you try to mmap a single page of a file, page cache will read the whole block (that corresponds to min order folio).

Technically we can mmap a single page of file, but FS will always read and write **at least** in min folio order chunks.

Okay, so it can be partially mapped into page tables :) What happens in the background (page cache management) is a different story

David, thanks for getting to the bottom of this.

OK. So we will see deadlock looping in madvise cold or pageout case. I wonder how to proceed with this. Since the folio is seen as a whole by fs, it should be marked cold/paged out as a whole. Maybe we should skip the partially mapped region?

Best Regards,
Yan, Zi

#25 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 28 lines · Zi Yan
Quoted 26 lines · David Hildenbrand
Quoted 24 lines · Pankaj Raghav (Samsung)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

I'm confused. Shouldn't this be trivially possible?

Hmm, maybe I misunderstood the question?

Quoted 2 lines

E.g., just mmap() a single page of such a file? Who would make that fail?

My point was, even if you try to mmap a single page of a file, page cache will read the whole block (that corresponds to min order folio).

Technically we can mmap a single page of file, but FS will always read and write **at least** in min folio order chunks.

Quoted 8 lines
Quoted 1 line

Okay, so it can be partially mapped into page tables :) What happens in the background (page cache management) is a different story

David, thanks for getting to the bottom of this.

OK. So we will see deadlock looping in madvise cold or pageout case. I wonder how to proceed with this. Since the folio is seen as a whole by fs, it should be marked cold/paged out as a whole. Maybe we should skip the partially mapped region?

Actually, it is skipped, since split_folio() bumps new_order to the min order, and if the folio order is already at min order, split code return -EINVAL. This makes the madvise cold or pageout code move to the next address.

Best Regards,
Yan, Zi

#26 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 39 lines · Zi Yan
Quoted 28 lines · Zi Yan
Quoted 26 lines · David Hildenbrand
Quoted 24 lines · Pankaj Raghav (Samsung)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

I'm confused. Shouldn't this be trivially possible?

Hmm, maybe I misunderstood the question?

Quoted 2 lines

E.g., just mmap() a single page of such a file? Who would make that fail?

My point was, even if you try to mmap a single page of a file, page cache will read the whole block (that corresponds to min order folio).

Technically we can mmap a single page of file, but FS will always read and write **at least** in min folio order chunks.

Quoted 8 lines
Quoted 1 line

Okay, so it can be partially mapped into page tables :) What happens in the background (page cache management) is a different story

David, thanks for getting to the bottom of this.

OK. So we will see deadlock looping in madvise cold or pageout case. I wonder how to proceed with this. Since the folio is seen as a whole by fs, it should be marked cold/paged out as a whole. Maybe we should skip the partially mapped region?

Quoted 4 lines

Actually, it is skipped, since split_folio() bumps new_order to the min order, and if the folio order is already at min order, split code return -EINVAL. This makes the madvise cold or pageout code move to the next address.

But what if the folio order is 2x min_order etc?

Signature · 3 lines
Cheers,

David / dhildenb
#27 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 40 lines · David Hildenbrand
Quoted 39 lines · Zi Yan
Quoted 36 lines · Zi Yan
Quoted 20 lines · David Hildenbrand
Quoted 19 lines · Pankaj Raghav (Samsung)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

I'm confused. Shouldn't this be trivially possible?

Hmm, maybe I misunderstood the question?

Quoted 2 lines

E.g., just mmap() a single page of such a file? Who would make that fail?

Quoted 7 lines
Quoted 5 lines

My point was, even if you try to mmap a single page of a file, page cache will read the whole block (that corresponds to min order folio).

Technically we can mmap a single page of file, but FS will always read and write **at least** in min folio order chunks.

Okay, so it can be partially mapped into page tables :) What happens in the background (page cache management) is a different story

David, thanks for getting to the bottom of this.

OK. So we will see deadlock looping in madvise cold or pageout case. I wonder how to proceed with this. Since the folio is seen as a whole by fs, it should be marked cold/paged out as a whole. Maybe we should skip the partially mapped region?

Quoted 6 lines
Quoted 4 lines

Actually, it is skipped, since split_folio() bumps new_order to the min order, and if the folio order is already at min order, split code return -EINVAL. This makes the madvise cold or pageout code move to the next address.

But what if the folio order is 2x min_order etc?

If folio order is greater than min_order, the code is suboptimal. It will first split the original folio to min_order, then loop through after-split folios. If the after-split ones are fully mapped, madvise ops will be performed. Otherwise, the code will try to split after-split ones again, fail, and end up with skipping the partially mapped range.

An improvement can be done by skipping the folio if it is partially mapped and its order is already min_order. Something like:

mm/madvise.c +7 -0
index e61e32b2cd91..545ab7920a81 100644
@@ -499,6 +499,13 @@ static int madvise_cold_or_pageout_pte_range(pmd_t *pmd,
 			if (nr < folio_nr_pages(folio)) {
 				int err;
 
+				/*
+				 * Skip partially mapped folios that are
+				 * already at their min order
+				 */
+				if (folio_order(folio) ==
+				    min_order_for_split(folio))
+					continue;
 				if (folio_maybe_mapped_shared(folio))
 					continue;
 				if (pageout_anon_only_filter && !folio_test_anon(folio))

Best Regards,
Yan, Zi

#28 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 47 lines · Zi Yan
Quoted 45 lines · David Hildenbrand
Quoted 43 lines · Zi Yan
Quoted 28 lines · Zi Yan
Quoted 26 lines · David Hildenbrand
Quoted 24 lines · Pankaj Raghav (Samsung)
Quoted 6 lines
Quoted 6 lines
Quoted 6 lines
Quoted 5 lines

So we will need to take care of madvise cold or pageout case?

Hi Matthew, Pankaj, and Luis,

Is it possible to partially map a min-order folio in a fs with LBS? Based on my

Quoted 8 lines
Quoted 5 lines
Quoted 5 lines

Typically, FSs match the min order with the blocksize of the filesystem. As a filesystem block is the smallest unit of data that the filesystem uses to store file data on the disk, we cannot partially map them.

So if I understand your question correctly, the answer is no.

I'm confused. Shouldn't this be trivially possible?

Hmm, maybe I misunderstood the question?

Quoted 2 lines

E.g., just mmap() a single page of such a file? Who would make that fail?

My point was, even if you try to mmap a single page of a file, page cache will read the whole block (that corresponds to min order folio).

Technically we can mmap a single page of file, but FS will always read and write **at least** in min folio order chunks.

Quoted 8 lines
Quoted 1 line

Okay, so it can be partially mapped into page tables :) What happens in the background (page cache management) is a different story

David, thanks for getting to the bottom of this.

OK. So we will see deadlock looping in madvise cold or pageout case. I wonder how to proceed with this. Since the folio is seen as a whole by fs, it should be marked cold/paged out as a whole. Maybe we should skip the partially mapped region?

Actually, it is skipped, since split_folio() bumps new_order to the min order, and if the folio order is already at min order, split code return -EINVAL. This makes the madvise cold or pageout code move to the next address.

Quoted 8 lines
Quoted 1 line

But what if the folio order is 2x min_order etc?

If folio order is greater than min_order, the code is suboptimal. It will first split the original folio to min_order, then loop through after-split folios. If the after-split ones are fully mapped, madvise ops will be performed. Otherwise, the code will try to split after-split ones again, fail, and end up with skipping the partially mapped range.

Indeed, thanks for checking, madvise() is fine then!

Signature · 3 lines
Cheers,

David / dhildenb
maintainer
#29 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

I continue to retain the original patch in mm-hotfixes as part of akpm's lame bug-tracking system. 3 weeks in -next.

And I just added a cc:stable to it because December 2018.

I don't expect many real-world users will be putting fake delays in memory_failure(), but it's there.

So what do we do here? Add a TODO, merge it under the better-than-it-was-before theory and move on?

From: Jinjiang Tu <[email protected]>
Subject: mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range
Date: Fri, 27 Jun 2025 20:57:47 +0800

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
 <TASK>
 rmap_walk_anon+0xda/0x1f0
 try_to_unmap+0x78/0x80
 ? __pfx_try_to_unmap_one+0x10/0x10
 ? __pfx_folio_not_mapped+0x10/0x10
 ? __pfx_folio_lock_anon_vma_read+0x10/0x10
 unmap_poisoned_folio+0x60/0x140
 do_migrate_range+0x4d1/0x600
 ? slab_memory_callback+0x6a/0x190
 ? notifier_call_chain+0x56/0xb0
 offline_pages+0x3e6/0x460
 memory_subsys_offline+0x130/0x1f0
 device_offline+0xba/0x110
 acpi_bus_offline+0xb7/0x130
 acpi_scan_hot_remove+0x77/0x290
 acpi_device_hotplug+0x1e0/0x240
 acpi_hotplug_work_fn+0x1a/0x30
 process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Also, do_migrate_range() may be called between memory_failure() setting the hwposion flag and isolation of the folio from the lru, so remove WARN_ON().

Also, in other places unmap_poisoned_folio() is called when the folio is isolated, so obey that in do_migrate_range().

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")
Cc David Hildenbrand <[email protected]>
Cc Kefeng Wang <[email protected]>
Cc Miaohe Lin <[email protected]>
Cc Michal Hocko <[email protected]>
Cc Oscar Salvador <[email protected]>

---

 mm/memory_hotplug.c |   21 +++++++++++++--------
 1 file changed, 13 insertions(+), 8 deletions(-)
mm/memory_hotplug.c renamed from mm/memory_hotplug.c~mm-memory_hotplug-fix-hwpoisoned-large-folio-handling-in-do_migrate_range +13 -8
@@ -1795,7 +1795,7 @@ found:
 	return 0;
 }
 
-static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
+static int do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 {
 	struct folio *folio;
 	unsigned long pfn;
@@ -1819,8 +1819,10 @@ static void do_migrate_range(unsigned lo
 			pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;
 
 		if (folio_contain_hwpoisoned_page(folio)) {
-			if (WARN_ON(folio_test_lru(folio)))
-				folio_isolate_lru(folio);
+			if (folio_test_large(folio) && !folio_test_hugetlb(folio))
+				goto err_out;
+			if (folio_test_lru(folio) && !folio_isolate_lru(folio))
+				goto err_out;
 			if (folio_mapped(folio)) {
 				folio_lock(folio);
 				unmap_poisoned_folio(folio, pfn, false);
@@ -1877,6 +1879,11 @@ put_folio:
 			putback_movable_pages(&source);
 		}
 	}
+	return 0;
+err_out:
+	folio_put(folio);
+	putback_movable_pages(&source);
+	return -EBUSY;
 }
 
 static int __init cmdline_parse_movable_node(char *p)
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pf
 
 			ret = scan_movable_pages(pfn, end_pfn, &pfn);
 			if (!ret) {
-				/*
-				 * TODO: fatal migration failures should bail
-				 * out
-				 */
-				do_migrate_range(pfn, end_pfn);
+				ret = do_migrate_range(pfn, end_pfn);
+				if (ret)
+					break;
 			}
 		} while (!ret);
 

_

#30 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 11 lines · Andrew Morton

I continue to retain the original patch in mm-hotfixes as part of akpm's lame bug-tracking system. 3 weeks in -next.

And I just added a cc:stable to it because December 2018.

I don't expect many real-world users will be putting fake delays in memory_failure(), but it's there.

So what do we do here? Add a TODO, merge it under the better-than-it-was-before theory and move on?

I would feel better if we could just not fail memory offlining. Memory offlining is documented to loop forever if something bad happens, and user space can cancel it.

Signature · 3 lines
Cheers,

David / dhildenb
maintainer
#31 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 10 lines · On Tue, 22 Jul 2025 17:30:06 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 9 lines · Andrew Morton

I continue to retain the original patch in mm-hotfixes as part of akpm's lame bug-tracking system. 3 weeks in -next.

And I just added a cc:stable to it because December 2018.

I don't expect many real-world users will be putting fake delays in memory_failure(), but it's there.

Quoted 7 lines
Quoted 2 lines

So what do we do here? Add a TODO, merge it under the better-than-it-was-before theory and move on?

I would feel better if we could just not fail memory offlining. Memory offlining is documented to loop forever if something bad happens, and user space can cancel it.

Pathetic monthly prod to keep this on people's radar.

#32 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 15 lines · Andrew Morton
Quoted 13 lines · On Tue, 22 Jul 2025 17:30:06 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 11 lines · Andrew Morton

I continue to retain the original patch in mm-hotfixes as part of akpm's lame bug-tracking system. 3 weeks in -next.

And I just added a cc:stable to it because December 2018.

I don't expect many real-world users will be putting fake delays in memory_failure(), but it's there.

So what do we do here? Add a TODO, merge it under the better-than-it-was-before theory and move on?

Quoted 7 lines
Quoted 4 lines

I would feel better if we could just not fail memory offlining. Memory offlining is documented to loop forever if something bad happens, and user space can cancel it.

Pathetic monthly prod to keep this on people's radar.

Let's do something minimal for now:

From 403b2a375a10c17fd6e2aeffbe0fdaf623faa621 Mon Sep 17 00:00:00 2001
From: Jinjiang Tu <[email protected]>
Date: Fri, 27 Jun 2025 20:57:47 +0800
Subject: [PATCH] mm/memory_hotplug: fix hwpoisoned large folio handling in
do_migrate_range

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
 <TASK>
 rmap_walk_anon+0xda/0x1f0
 try_to_unmap+0x78/0x80
 ? __pfx_try_to_unmap_one+0x10/0x10
 ? __pfx_folio_not_mapped+0x10/0x10
 ? __pfx_folio_lock_anon_vma_read+0x10/0x10
 unmap_poisoned_folio+0x60/0x140
 do_migrate_range+0x4d1/0x600
 ? slab_memory_callback+0x6a/0x190
 ? notifier_call_chain+0x56/0xb0
 offline_pages+0x3e6/0x460
 memory_subsys_offline+0x130/0x1f0
 device_offline+0xba/0x110
 acpi_bus_offline+0xb7/0x130
 acpi_scan_hot_remove+0x77/0x290
 acpi_device_hotplug+0x1e0/0x240
 acpi_hotplug_work_fn+0x1a/0x30
 process_one_work+0x186/0x340

Besides, do_migrate_range() may be called between memory_failure set hwpoison flag and isolate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")
[ David: don't abort offlining, fixed typo, added comment ]
---
 mm/memory_hotplug.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
mm/memory_hotplug.c +8 -2
index 1f15af712bc34..74318c7877156 100644
@@ -1815,8 +1815,14 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 			pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;
 
 		if (folio_contain_hwpoisoned_page(folio)) {
-			if (WARN_ON(folio_test_lru(folio)))
-				folio_isolate_lru(folio);
+			/*
+			 * unmap_poisoned_folio() cannot handle large folios
+			 * in all cases yet.
+			 */
+			if (folio_test_large(folio) && !folio_test_hugetlb(folio))
+				goto put_folio;
+			if (folio_test_lru(folio) && !folio_isolate_lru(folio))
+				goto put_folio;
 			if (folio_mapped(folio)) {
 				folio_lock(folio);
 				unmap_poisoned_folio(folio, pfn, false);
Signature · 9 lines
2.50.1


Man oh man, is hwpoison handling a mess.

-- 
Cheers

David / dhildenb
Acked-by
#33 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

Quoted 64 lines · David Hildenbrand
Quoted 15 lines · Andrew Morton
Quoted 13 lines · On Tue, 22 Jul 2025 17:30:06 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 11 lines · Andrew Morton

I continue to retain the original patch in mm-hotfixes as part of akpm's lame bug-tracking system. 3 weeks in -next.

And I just added a cc:stable to it because December 2018.

I don't expect many real-world users will be putting fake delays in memory_failure(), but it's there.

So what do we do here? Add a TODO, merge it under the better-than-it-was-before theory and move on?

Quoted 7 lines
Quoted 4 lines

I would feel better if we could just not fail memory offlining. Memory offlining is documented to loop forever if something bad happens, and user space can cancel it.

Pathetic monthly prod to keep this on people's radar.

Let's do something minimal for now:

From 403b2a375a10c17fd6e2aeffbe0fdaf623faa621 Mon Sep 17 00:00:00 2001
From: Jinjiang Tu <[email protected]>
Date: Fri, 27 Jun 2025 20:57:47 +0800
Subject: [PATCH] mm/memory_hotplug: fix hwpoisoned large folio handling in
do_migrate_range

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
 <TASK>
 rmap_walk_anon+0xda/0x1f0
 try_to_unmap+0x78/0x80
 ? __pfx_try_to_unmap_one+0x10/0x10
 ? __pfx_folio_not_mapped+0x10/0x10
 ? __pfx_folio_lock_anon_vma_read+0x10/0x10
 unmap_poisoned_folio+0x60/0x140
 do_migrate_range+0x4d1/0x600
 ? slab_memory_callback+0x6a/0x190
 ? notifier_call_chain+0x56/0xb0
 offline_pages+0x3e6/0x460
 memory_subsys_offline+0x130/0x1f0
 device_offline+0xba/0x110
 acpi_bus_offline+0xb7/0x130
 acpi_scan_hot_remove+0x77/0x290
 acpi_device_hotplug+0x1e0/0x240
 acpi_hotplug_work_fn+0x1a/0x30
 process_one_work+0x186/0x340

Besides, do_migrate_range() may be called between memory_failure set hwpoison flag and isolate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Quoted 7 lines
Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")
[ David: don't abort offlining, fixed typo, added comment ]
---
 mm/memory_hotplug.c | 10 ++++++++--
 1 file changed, 8 insertions(+), 2 deletions(-)
Quoted 27 lines
mm/memory_hotplug.c +8 -2
index 1f15af712bc34..74318c7877156 100644
@@ -1815,8 +1815,14 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
 			pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;
  		if (folio_contain_hwpoisoned_page(folio)) {
-			if (WARN_ON(folio_test_lru(folio)))
-				folio_isolate_lru(folio);
+			/*
+			 * unmap_poisoned_folio() cannot handle large folios
+			 * in all cases yet.
+			 */
+			if (folio_test_large(folio) && !folio_test_hugetlb(folio))
+				goto put_folio;
+			if (folio_test_lru(folio) && !folio_isolate_lru(folio))
+				goto put_folio;
 			if (folio_mapped(folio)) {
 				folio_lock(folio);
 				unmap_poisoned_folio(folio, pfn, false);
Signature · 4 lines
2.50.1


Man oh man, is hwpoison handling a mess.

--
Best Regards,
Yan, Zi

Reviewed-by
#34 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

In reply to David Hildenbrand
Quoted 85 lines · David Hildenbrand
Quoted 15 lines · Andrew Morton
Quoted 13 lines · On Tue, 22 Jul 2025 17:30:06 +0200 David Hildenbrand <[email protected]> wrote:
Quoted 11 lines · Andrew Morton

I continue to retain the original patch in mm-hotfixes as part of akpm's lame bug-tracking system.  3 weeks in -next.

And I just added a cc:stable to it because December 2018.

I don't expect many real-world users will be putting fake delays in memory_failure(), but it's there.

So what do we do here?  Add a TODO, merge it under the better-than-it-was-before theory and move on?

Quoted 7 lines
Quoted 4 lines

I would feel better if we could just not fail memory offlining. Memory offlining is documented to loop forever if something bad happens, and user space can cancel it.

Pathetic monthly prod to keep this on people's radar.

Let's do something minimal for now:

From 403b2a375a10c17fd6e2aeffbe0fdaf623faa621 Mon Sep 17 00:00:00 2001
From: Jinjiang Tu <[email protected]>
Date: Fri, 27 Jun 2025 20:57:47 +0800
Subject: [PATCH] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
 <TASK>
 rmap_walk_anon+0xda/0x1f0
 try_to_unmap+0x78/0x80
 ? __pfx_try_to_unmap_one+0x10/0x10
 ? __pfx_folio_not_mapped+0x10/0x10
 ? __pfx_folio_lock_anon_vma_read+0x10/0x10
 unmap_poisoned_folio+0x60/0x140
 do_migrate_range+0x4d1/0x600
 ? slab_memory_callback+0x6a/0x190
 ? notifier_call_chain+0x56/0xb0
 offline_pages+0x3e6/0x460
 memory_subsys_offline+0x130/0x1f0
 device_offline+0xba/0x110
 acpi_bus_offline+0xb7/0x130
 acpi_scan_hot_remove+0x77/0x290
 acpi_device_hotplug+0x1e0/0x240
 acpi_hotplug_work_fn+0x1a/0x30
 process_one_work+0x186/0x340

Besides, do_migrate_range() may be called between memory_failure set hwpoison flag and isolate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")
[ David: don't abort offlining, fixed typo, added comment ]

---
 mm/memory_hotplug.c | 10 ++++++++--

 1 file changed, 8 insertions(+), 2 deletions(-)
mm/memory_hotplug.c +0 -0
index 1f15af712bc34..74318c7877156 100644
@@ -1815,8 +1815,14 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
              pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;


         if (folio_contain_hwpoisoned_page(folio)) {
-            if (WARN_ON(folio_test_lru(folio)))
-                folio_isolate_lru(folio);
+            /*
+             * unmap_poisoned_folio() cannot handle large folios +             * in all cases yet.

Quoted 8 lines

+             */
+            if (folio_test_large(folio) && !folio_test_hugetlb(folio)) +                goto put_folio;
+            if (folio_test_lru(folio) && !folio_isolate_lru(folio)) +                goto put_folio;
             if (folio_mapped(folio)) {
                 folio_lock(folio); unmap_poisoned_folio(folio, pfn, false);

Thanks.
.

#35 lore raw

Re: [PATCH v2 2/2] mm/memory_hotplug: fix hwpoisoned large folio handling in do_migrate_range

In reply to Jinjiang Tu
Quoted 82 lines · Jinjiang Tu

In do_migrate_range(), the hwpoisoned folio may be large folio, which can't be handled by unmap_poisoned_folio().

I can reproduce this issue in qemu after adding delay in memory_failure()

BUG: kernel NULL pointer dereference, address: 0000000000000000
Workqueue: kacpi_hotplug acpi_hotplug_work_fn
RIP: 0010:try_to_unmap_one+0x16a/0xfc0
  <TASK>
  rmap_walk_anon+0xda/0x1f0
  try_to_unmap+0x78/0x80
  ? __pfx_try_to_unmap_one+0x10/0x10
  ? __pfx_folio_not_mapped+0x10/0x10
  ? __pfx_folio_lock_anon_vma_read+0x10/0x10
  unmap_poisoned_folio+0x60/0x140
  do_migrate_range+0x4d1/0x600
  ? slab_memory_callback+0x6a/0x190
  ? notifier_call_chain+0x56/0xb0
  offline_pages+0x3e6/0x460
  memory_subsys_offline+0x130/0x1f0
  device_offline+0xba/0x110
  acpi_bus_offline+0xb7/0x130
  acpi_scan_hot_remove+0x77/0x290
  acpi_device_hotplug+0x1e0/0x240
  acpi_hotplug_work_fn+0x1a/0x30
  process_one_work+0x186/0x340

In this case, just make offline_pages() fail.

Besides, do_migrate_range() may be called between memory_failure set hwposion flag and ioslate the folio from lru, so remove WARN_ON(). In other places, unmap_poisoned_folio() is called when the folio is isolated, obey it in do_migrate_range() too.

Fixes b15c87263a69 ("hwpoison, memory_hotplug: allow hwpoisoned pages to be offlined")
---
  mm/memory_hotplug.c | 21 +++++++++++++--------
  1 file changed, 13 insertions(+), 8 deletions(-)
mm/memory_hotplug.c +10 -4
index b1caedbade5b..5fab212e3a2c 100644
@@ -1795,7 +1795,7 @@ static int scan_movable_pages(unsigned long start, unsigned long end,
  	return 0;
  }
 
-static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
+static int do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
  {
  	struct folio *folio;
  	unsigned long pfn;
@@ -1819,8 +1819,10 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
  			pfn = folio_pfn(folio) + folio_nr_pages(folio) - 1;
 
  		if (folio_contain_hwpoisoned_page(folio)) {
-			if (WARN_ON(folio_test_lru(folio)))
-				folio_isolate_lru(folio);
+			if (folio_test_large(folio) && !folio_test_hugetlb(folio))
+				goto err_out;
+			if (folio_test_lru(folio) && !folio_isolate_lru(folio))
+				goto err_out;
  			if (folio_mapped(folio)) {
  				folio_lock(folio);
  				unmap_poisoned_folio(folio, pfn, false);
@@ -1877,6 +1879,11 @@ static void do_migrate_range(unsigned long start_pfn, unsigned long end_pfn)
  			putback_movable_pages(&source);
  		}
  	}
+	return 0;
+err_out:
+	folio_put(folio);
+	putback_movable_pages(&source);
+	return -EBUSY;
  }
  >   static int __init cmdline_parse_movable_node(char *p)
@@ -2041,11 +2048,9 @@ int offline_pages(unsigned long start_pfn, unsigned long nr_pages,
 
  			ret = scan_movable_pages(pfn, end_pfn, &pfn);
  			if (!ret) {
-				/*

mm/memory_hotplug.c int offline_pages(unsigned long start_pfn, unsigned long nr_pages, · line 2056

Quoted 10 lines
mm/memory_hotplug.c +3 -4
-				 * TODO: fatal migration failures should bail
-				 * out
-				 */
-				do_migrate_range(pfn, end_pfn);
+				ret = do_migrate_range(pfn, end_pfn);
+				if (ret)
+					break;
  			}
  		} while (!ret);
 

I think this could simply be

...

	ret = scan_movable_pages(pfn, end_pfn, &pfn);
	if (!ret)
		ret = do_migrate_range(pfn, end_pfn);
} while (!ret);

because the while (!ret) will take care of the "break".

Signature · 3 lines
Cheers,

David / dhildenb