mm/memory-failure: add panic option for unrecoverable pages
Subsystems (6)
v7 cover mm/memory-failure: add panic option for unrecoverable pages
A multi-bit ECC error on a kernel-owned page that the memory failure handler cannot recover is currently swallowed: PG_hwpoison is set, the event is logged, and the kernel keeps running. The corrupted memory remains accessible to the kernel and either drives silent data corruption or surfaces seconds-to-minutes later as an apparently unrelated crash. In a large fleet that delayed, unattributable crash turns into significant engineering effort to root-cause; in a kdump configuration, by the time the crash happens the original error context (faulting PFN, MCE/GHES record, page state) is long gone.
This series adds an opt-in sysctl, vm.panic_on_unrecoverable_memory_failure, that converts an unrecoverable kernel-page hwpoison event into an immediate panic with a clean dmesg/vmcore that still contains the original failure context. The default is disabled so existing workloads see no change.
---
Changes in v7:
- Move the PG_reserved / unhandlable-kernel-page classification into get_any_page() and surface it via -ENOTRECOVERABLE, per David Hildenbrand's and Lance Yang's review of v6. This drops the is_reserved snapshot in memory_failure() and the mf_get_page_status enum / out-parameter introduced in v6.
- Restructure the post-call branch in memory_failure() as a switch
over the get_hwpoison_page() return code (David).
- Drop the "reserved" qualifier from the MF_MSG_KERNEL label and the matching tracepoint string; the enum now covers both PG_reserved
pages and other unhandlable kernel pages.
- Squash the former patches 1/4 ("MF_MSG_KERNEL for reserved pages")
and 2/4 ("classify get_any_page() failures by reason") into a
single classification patch; the series is now 3 patches.
- Simplify panic_on_unrecoverable_mf() to a single return statement (David).
- Link to v6: https://patch.msgid.link/[email protected]
Changes in v6:
- Dropped the selftest given the value was not clear
- Get the status of the failure from get_any_page()
- Small nits from different people/AIs.
- Link to v5: https://patch.msgid.link/[email protected]
Changes in v5:
- Add vm.panic_on_unrecoverable_memory_failure sysctl to panic on unrecoverable kernel page hwpoison events (reserved pages, refcount-0 non-buddy pages, unknown state), with a recheck to avoid racing with concurrent buddy allocations. (Miaohe)
- Distinguish reserved pages as MF_MSG_KERNEL in memory_failure(), document the new sysctl in Documentation/admin-guide/sysctl/vm.rst, and add a selftest verifying SIGBUS recovery on userspace pages still works when the sysctl is enabled. (Miaohe)
- Added a selftest
- Link to v4:
https://patch.msgid.link/[email protected]
Changes in v4:
- Drop CONFIG_BOOTPARAM_MEMORY_FAILURE_PANIC kernel configuration option.
- Split the reserved page classification (MF_MSG_KERNEL) into its own patch, separate from the panic mechanism.
- Document why the buddy allocator TOCTOU race (between get_hwpoison_page() and is_free_buddy_page()) cannot cause false
positives: PG_hwpoison is set beforehand and check_new_page() in the page allocator rejects hwpoisoned pages.
- Document the narrow LRU isolation race window for MF_MSG_UNKNOWN and its mitigation via identify_page_state()'s two-pass design.
- Explicitly document why MF_MSG_GET_HWPOISON is excluded from the
panic conditions (shared path with transient races and non-reserved kernel memory).
- Link to v3: https://patch.msgid.link/[email protected]
Changes in v3:
- Rename is_unrecoverable_memory_failure() to panic_on_unrecoverable_mf() as suggested by maintainer.
- Add CONFIG_BOOTPARAM_MEMORY_FAILURE_PANIC kernel configuration option, similar to CONFIG_BOOTPARAM_HARDLOCKUP_PANIC.
- Add documentation for the sysctl and CONFIG option.
- Add code comments documenting the panic condition design rationale and how the retry mechanism mitigates false positives from buddy allocator races.
- Link to v2: https://patch.msgid.link/[email protected]
Changes in v2:
- Panic on MF_MSG_KERNEL, MF_MSG_KERNEL_HIGH_ORDER and MF_MSG_UNKNOWN instead of MF_MSG_GET_HWPOISON.
- Report MF_MSG_KERNEL for reserved pages when get_hwpoison_page() fails instead of MF_MSG_GET_HWPOISON.
- Link to v1: https://patch.msgid.link/[email protected]
To: Miaohe Lin <[email protected]>
To: Naoya Horiguchi <[email protected]>
To: Andrew Morton <[email protected]>
To: Steven Rostedt <[email protected]>
To: Masami Hiramatsu <[email protected]>
To: Mathieu Desnoyers <[email protected]>
To: Jonathan Corbet <[email protected]>
To: Shuah Khan <[email protected]>
To: David Hildenbrand <[email protected]>
To: Lorenzo Stoakes <[email protected]>
To: "Liam R. Howlett" <[email protected]>
To: Vlastimil Babka <[email protected]>
To: Mike Rapoport <[email protected]>
To: Suren Baghdasaryan <[email protected]>
To: Michal Hocko <[email protected]>
To: Shuah Khan <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
---
Breno Leitao (6):
mm/memory-failure: drop dead error_states[] entry for reserved pages mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page() mm/memory-failure: add panic option for unrecoverable pages
Documentation: document panic_on_unrecoverable_memory_failure sysctl
Documentation/admin-guide/sysctl/vm.rst | 80 +++++++++++++++++++++++++++++++ mm/memory-failure.c | 85 +++++++++++++++++++++++++-------- 2 files changed, 146 insertions(+), 19 deletions(-)
---
base-commit: e98d21c170b01ddef366f023bbfcf6b31509fa83
change-id: 20260323-ecc_panic-4e473b83087c
Best regards,
Signature · 1 line
Breno Leitao <[email protected]>
PATCH v7 1/6 mm/memory-failure: drop dead error_states[] entry for reserved pages
The first entry of error_states[],
{ reserved, reserved, MF_MSG_KERNEL, me_kernel },
is unreachable. identify_page_state() has two callers, and neither one can dispatch a PG_reserved page to me_kernel():
* memory_failure() reaches identify_page_state() only after get_hwpoison_page() returned 1. get_any_page() reaches that return only via __get_hwpoison_page(), which gates the refcount on HWPoisonHandlable(). HWPoisonHandlable() rejects PG_reserved pages, so they fail with -EBUSY/-EIO long before identify_page_state() runs.
* try_memory_failure_hugetlb() reaches identify_page_state() on the MF_HUGETLB_IN_USED branch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head, MF_MSG_HUGE, me_huge_page }, so they dispatch to me_huge_page() before the (now-removed) reserved entry would have matched, regardless of whether PG_reserved happens to be set on the head page.
me_kernel() never executes and the entry exists only to be matched against by code that cannot see it.
Drop the entry, the me_kernel() helper, and the now-unused "reserved" macro. Leave the MF_MSG_KERNEL enum value in place: it remains part of the tracepoint and pr_err() string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.
No functional change.
--- mm/memory-failure.c | 14 -------------- 1 file changed, 14 deletions(-)
mm/memory-failure.c +0 -14
@@ -992,17 +992,6 @@ static bool has_extra_refcount(struct page_state *ps, struct page *p,return false; } -/* -* Error hit kernel page. -* Do nothing, try to be lucky and not touch this instead. For a few cases we -* could be more sophisticated. -*/ -static int me_kernel (struct page_state *ps,struct page *p) -{ -unlock_page (p); -return MF_IGNORED ; -} -/* * Page in unknown state. Do nothing. * This is a catch-all in case we fail to make sense of the page state.
@@ -1211,10 +1200,8 @@ static int me_huge_page(struct page_state *ps, struct page *p)#define mlock (1UL << PG_mlocked)#define lru (1UL << PG_lru)#define head (1UL << PG_head) -#define reserved (1UL << PG_reserved)static struct page_state error_states[] = { - { reserved, reserved,MF_MSG_KERNEL , me_kernel },/* * free pages are specially detected outside this table: * PG_buddy pages only make a small fraction of all free pages.
@@ -1246,7 +1233,6 @@ static struct page_state error_states[] = {#undef mlock#undef lru#undef head -#undef reservedstatic void update_per_node_mf_stats (unsigned long pfn,enum mf_result result)
Signature · 1 line
2.53.0-Meta
Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages
Quoted 7 lines · Breno Leitao
The first entry of error_states[],
{ reserved, reserved,
MF_MSG_KERNEL,me_kernel},is unreachable.
identify_page_state()has two callers, and neither one can dispatch aPG_reservedpage tome_kernel():
Quoted 6 lines
*
memory_failure()reachesidentify_page_state()only afterget_hwpoison_page()returned 1.get_any_page()reaches that return only via__get_hwpoison_page(), which gates the refcount onHWPoisonHandlable().HWPoisonHandlable()rejectsPG_reservedpages, so they fail with -EBUSY/-EIO long beforeidentify_page_state()runs.
You should clarify why they are rejected. There is no explicit check for PG_reserved in there!
Quoted 8 lines
*
try_memory_failure_hugetlb()reachesidentify_page_state()on theMF_HUGETLB_IN_USEDbranch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head,MF_MSG_HUGE,me_huge_page}, so they dispatch tome_huge_page()before the (now-removed) reserved entry would have matched, regardless of whetherPG_reservedhappens to be set on the head page.
See hugetlb_folio_init_vmemmap(): we always clear PG_reserved for hugetlb folios allocated from memblock.
Quoted 53 lines
me_kernel()never executes and the entry exists only to be matched against by code that cannot see it.Drop the entry, the
me_kernel()helper, and the now-unused "reserved" macro. Leave theMF_MSG_KERNELenum value in place: it remains part of the tracepoint andpr_err()string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.No functional change.
Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>--- mm/memory-failure.c | 14 -------------- 1 file changed, 14 deletions(-)mm/memory-failure.c +0
-13@@ -992,17 +992,6 @@ static bool has_extra_refcount(struct page_state *ps, struct page *p,return false; } -/* -* Error hit kernel page. -* Do nothing, try to be lucky and not touch this instead. For a few cases we -* could be more sophisticated. -*/ -static int me_kernel (struct page_state *ps,struct page *p) -{ -unlock_page (p); -return MF_IGNORED ; -} -/* * Page in unknown state. Do nothing. * This is a catch-all in case we fail to make sense of the page state. @@ -1211,10 +1200,8 @@ static int me_huge_page(struct page_state *ps, struct page *p)#define mlock (1UL << PG_mlocked)#define lru (1UL << PG_lru)#define head (1UL << PG_head) -#define reserved (1UL << PG_reserved)static struct page_state error_states[] = { - { reserved, reserved,MF_MSG_KERNEL , me_kernel },/* * free pages are specially detected outside this table: * PG_buddy pages only make a small fraction of all free pages.
mm/memory-failure.c static struct page_state error_states[] = { · line 1238
Quoted 7 lines
mm/memory-failure.c +0
-1@@ -1246,7 +1233,6 @@ static struct page_state error_states[] = {#undef mlock#undef lru#undef head -#undef reservedstatic void update_per_node_mf_stats (unsigned long pfn,enum mf_result result)
Yes, I think this should work.
Signature · 3 lines
Cheers, David
Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages
Quoted 10 lines · David Hildenbrand (Arm)
Quoted 6 lines · Breno Leitao
*
memory_failure()reachesidentify_page_state()only afterget_hwpoison_page()returned 1.get_any_page()reaches that return only via__get_hwpoison_page(), which gates the refcount onHWPoisonHandlable().HWPoisonHandlable()rejectsPG_reservedpages, so they fail with -EBUSY/-EIO long beforeidentify_page_state()runs.You should clarify why they are rejected. There is no explicit check for
PG_reservedin there!
True, I meant that PG_reserved pages do not fit any of the criterias of HWPoisonHandlable().
I will rewrite it more explictly:
__get_hwpoison_page() only takes a refcount when the page is HWPoisonHandlable()'d, and HWPoisonHandlable() is an allowlist for LRU / free-buddy / (soft-offline) movable_ops pages.
is it any better?
Quoted 10 lines
Quoted 7 lines
*
try_memory_failure_hugetlb()reachesidentify_page_state()on theMF_HUGETLB_IN_USEDbranch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head,MF_MSG_HUGE,me_huge_page}, so they dispatch tome_huge_page()before the (now-removed) reserved entry would have matched, regardless of whetherPG_reservedhappens to be set on the head page.See
hugetlb_folio_init_vmemmap(): we always clearPG_reservedfor hugetlb folios allocated from memblock.
Thanks. I clearly see a call to __folio_clear_reserved(folio), so, huge pagetlb folios are never reserved.
A better summary would be:
try_memory_failure_hugetlb() reaches identify_page_state() only via the MF_HUGETLB_IN_USED branch, as hugetlb folios don't carry PG_reserved at that point (hugetlb_folio_init_vmemmap() clears it during init).
Quoted 3 lines
Yes, I think this should work.
Thanks for the review,
--breno
Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages
In reply to Breno LeitaoQuoted 7 lines · Breno Leitao
The first entry of error_states[],
{ reserved, reserved,
MF_MSG_KERNEL,me_kernel},is unreachable.
identify_page_state()has two callers, and neither one can dispatch aPG_reservedpage tome_kernel():
Quoted 6 lines
*
memory_failure()reachesidentify_page_state()only afterget_hwpoison_page()returned 1.get_any_page()reaches that return only via__get_hwpoison_page(), which gates the refcount onHWPoisonHandlable().HWPoisonHandlable()rejectsPG_reservedpages, so they fail with -EBUSY/-EIO long beforeidentify_page_state()runs.
HWPoisonHandlable() does not test PG_reserved directly; it only lets LRU or free buddy pages through:
return PageLRU(page) || is_free_buddy_page(page);
So this really relies on PG_reserved not being combined with either of those states. I would not expect that to happen, though.
Quoted 8 lines
*
try_memory_failure_hugetlb()reachesidentify_page_state()on theMF_HUGETLB_IN_USEDbranch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head,MF_MSG_HUGE,me_huge_page}, so they dispatch tome_huge_page()before the (now-removed) reserved entry would have matched, regardless of whetherPG_reservedhappens to be set on the head page.
As David pointed out, hugetlb setup clears PG_reserved before setting PG_head. See hugetlb_folio_init_vmemmap():
__folio_clear_reserved(folio); __folio_set_head(folio);
Quoted 3 lines
me_kernel()never executes and the entry exists only to be matched against by code that cannot see it.
identify_page_state() is reached only when get_hwpoison_page() returns 1, but a PG_reserved page would not get that far, IIUC :)
Quoted 7 lines
Drop the entry, the
me_kernel()helper, and the now-unused "reserved" macro. Leave theMF_MSG_KERNELenum value in place: it remains part of the tracepoint andpr_err()string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.
Quoted 5 lines
No functional change.
Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>---
With David's comments addressed, feel free to add:
Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages
In reply to Breno LeitaoQuoted 25 lines · Breno Leitao
The first entry of error_states[],
{ reserved, reserved,
MF_MSG_KERNEL,me_kernel},is unreachable.
identify_page_state()has two callers, and neither one can dispatch aPG_reservedpage tome_kernel():*
memory_failure()reachesidentify_page_state()only afterget_hwpoison_page()returned 1.get_any_page()reaches that return only via__get_hwpoison_page(), which gates the refcount onHWPoisonHandlable().HWPoisonHandlable()rejectsPG_reservedpages, so they fail with -EBUSY/-EIO long beforeidentify_page_state()runs.*
try_memory_failure_hugetlb()reachesidentify_page_state()on theMF_HUGETLB_IN_USEDbranch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head,MF_MSG_HUGE,me_huge_page}, so they dispatch tome_huge_page()before the (now-removed) reserved entry would have matched, regardless of whetherPG_reservedhappens to be set on the head page.
me_kernel()never executes and the entry exists only to be matched against by code that cannot see it.
Quoted 7 lines
Drop the entry, the
me_kernel()helper, and the now-unused "reserved" macro. Leave theMF_MSG_KERNELenum value in place: it remains part of the tracepoint andpr_err()string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.No functional change.
As the code evolves, this entry is no longer needed. Thanks for cleanup.
Quoted 3 lines
Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>
With David's comments addressed, this patch looks good to me:
Thanks.
.
PATCH v7 2/6 mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
In reply to Breno Leitaoget_any_page() collapses three different failure modes into a single -EIO return:
* the put_page race in the !count_increased path;
* the HWPoisonHandlable() rejection that bounces out of
__get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;
* the HWPoisonHandlable() rejection that goes through the count_increased / put_page / shake_page retry loop.
The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.
Distinguish them on the return path: keep -EIO for both the put_page race and the -EBUSY-after-retries branch (shake_page() cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep -EBUSY for the allocation race (unchanged), and return -ENOTRECOVERABLE only from the count_increased-true HWPoisonHandlable() rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.
Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc.
memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch is a no-op for users of memory_failure() and only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires the new return code through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases.
--- mm/memory-failure.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)
mm/memory-failure.c +15 -3
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)shake_page (p); goto try_again; } +/* +* Return -EIO rather than -ENOTRECOVERABLE: this +* branch is also reached for pages that are merely +* off-LRU transiently (e.g. a folio in the middle +* of migration or compaction), which shake_page() +* cannot drag back. The caller cannot prove the +* page is permanently kernel-owned from here, so +* keep it on the recoverable errno. +*/ ret = -EIO ; goto out; }
@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags) goto try_again; }put_page (p); - ret = -EIO ; + ret = -ENOTRECOVERABLE ; } out: -if (ret == -EIO ) +if (ret == -EIO || ret == -ENOTRECOVERABLE )pr_err ("%#lx: unhandlable page.\n" ,page_to_pfn (p));return ret;
@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)* -EIO for pages on which we can not handle memory errors, * -EBUSY when get_hwpoison_page() has raced with page lifecycle * operations like allocation and free, -* -EHWPOISON when the page is hwpoisoned and taken off from buddy. +* -EHWPOISON when the page is hwpoisoned and taken off from buddy, +* -ENOTRECOVERABLE for stable kernel-owned pages the handler +* cannot recover (PG_reserved, slab, vmalloc, page tables, +* kernel stacks, and similar non-LRU/non-buddy pages). */ static int get_hwpoison_page (struct page *p,unsigned long flags) {
Signature · 1 line
2.53.0-Meta
Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 74 lines · Breno Leitao
get_any_page()collapses three different failure modes into a single-EIOreturn:* the
put_pagerace in the !count_increased path;
* theHWPoisonHandlable()rejection that bounces out of__get_hwpoison_page()with-EBUSYand exhaustsshake_page()retries;
* theHWPoisonHandlable()rejection that goes through thecount_increased/put_page/shake_pageretry loop.The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.
Distinguish them on the return path: keep
-EIOfor both theput_pagerace and the -EBUSY-after-retries branch (shake_page()cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep-EBUSYfor the allocation race (unchanged), and return-ENOTRECOVERABLEonly from the count_increased-trueHWPoisonHandlable()rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.Extend the unhandlable-page
pr_err()to fire for either errno and update theget_hwpoison_page()kerneldoc.
memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISONvia its existing "else if (res < 0)" branch, so this patch is a no-op for users ofmemory_failure()and only changes the errno thatsoft_offline_page()can propagate to its callers. A follow-up wires the new return code throughmemory_failure()and reportsMF_MSG_KERNELfor the unrecoverable cases.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>---
mm/memory-failure.c| 18 +++++++++++++++---1 file changed, 15 insertions(+), 3 deletions(-)mm/memory-failure.c +10
-1@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)shake_page (p); goto try_again; } +/* +* Return -EIO rather than -ENOTRECOVERABLE: this +* branch is also reached for pages that are merely +* off-LRU transiently (e.g. a folio in the middle +* of migration or compaction), which shake_page() +* cannot drag back. The caller cannot prove the +* page is permanently kernel-owned from here, so +* keep it on the recoverable errno. +*/ ret = -EIO ; goto out; }@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags) goto try_again; }put_page (p); - ret = -EIO ; + ret = -ENOTRECOVERABLE ; }out:
mm/memory-failure.c +1
-1-if (ret == -EIO ) +if (ret == -EIO || ret == -ENOTRECOVERABLE )pr_err("%#lx: unhandlable page.\n", page_to_pfn(p)); return ret;
mm/memory-failure.c static int __get_unpoison_page(struct page *page) · line 1502
Quoted 8 lines
mm/memory-failure.c +4
-1@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)* -EIO for pages on which we can not handle memory errors, * -EBUSY when get_hwpoison_page() has raced with page lifecycle * operations like allocation and free, -* -EHWPOISON when the page is hwpoisoned and taken off from buddy. +* -EHWPOISON when the page is hwpoisoned and taken off from buddy, +* -ENOTRECOVERABLE for stable kernel-owned pages the handler +* cannot recover (PG_reserved, slab, vmalloc, page tables, +* kernel stacks, and similar non-LRU/non-buddy pages).
Did you test this patch series? I don't see how we ever get to -ENOTRECOVERABLE there ...
Even with MF_COUNT_INCREASED, the first pass does:
if (flags & MF_COUNT_INCREASED)
count_increased = true;
[...]
if (PageHuge(p) || HWPoisonHandlable(p, flags)) {
ret = 1;
} else {
if (pass++ < GET_PAGE_MAX_RETRY_NUM) { <-
put_page(p);
shake_page(p);
count_increased = false;
goto try_again; <-
}
put_page(p);
ret = -ENOTRECOVERABLE;
}Then we come back with count_increased=false:
try_again:
if (!count_increased) {
ret = __get_hwpoison_page(p, flags); <-
if (!ret) {
[...]
} else if (ret == -EBUSY) { <-
[...]
ret = -EIO;
goto out; <-
}
}For slab/vmalloc/page-table pages, __get_hwpoison_page() returns -EBUSY:
if (!HWPoisonHandlable(&folio->page, flags)) return -EBUSY;
so they still seem to end up as -EIO ... Am I missing something?
Quoted 8 lines
mm/memory-failure.c +0
-0*/ static int get_hwpoison_page (struct page *p,unsigned long flags) {Signature · 2 lines
-- 2.53.0-Meta
Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 74 lines · Lance Yang
Quoted 72 lines · Breno Leitao
get_any_page()collapses three different failure modes into a single-EIOreturn:* the
put_pagerace in the !count_increased path;
* theHWPoisonHandlable()rejection that bounces out of__get_hwpoison_page()with-EBUSYand exhaustsshake_page()retries;
* theHWPoisonHandlable()rejection that goes through thecount_increased/put_page/shake_pageretry loop.The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.
Distinguish them on the return path: keep
-EIOfor both theput_pagerace and the -EBUSY-after-retries branch (shake_page()cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep-EBUSYfor the allocation race (unchanged), and return-ENOTRECOVERABLEonly from the count_increased-trueHWPoisonHandlable()rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.Extend the unhandlable-page
pr_err()to fire for either errno and update theget_hwpoison_page()kerneldoc.
memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISONvia its existing "else if (res < 0)" branch, so this patch is a no-op for users ofmemory_failure()and only changes the errno thatsoft_offline_page()can propagate to its callers. A follow-up wires the new return code throughmemory_failure()and reportsMF_MSG_KERNELfor the unrecoverable cases.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>---
mm/memory-failure.c| 18 +++++++++++++++---1 file changed, 15 insertions(+), 3 deletions(-)mm/memory-failure.c +10
-1@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)shake_page (p); goto try_again; } +/* +* Return -EIO rather than -ENOTRECOVERABLE: this +* branch is also reached for pages that are merely +* off-LRU transiently (e.g. a folio in the middle +* of migration or compaction), which shake_page() +* cannot drag back. The caller cannot prove the +* page is permanently kernel-owned from here, so +* keep it on the recoverable errno. +*/ ret = -EIO ; goto out; }@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags) goto try_again; }put_page (p); - ret = -EIO ; + ret = -ENOTRECOVERABLE ; }out:
mm/memory-failure.c +1
-1-if (ret == -EIO ) +if (ret == -EIO || ret == -ENOTRECOVERABLE )pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));
Quoted 13 lines
mm/memory-failure.c static int __get_unpoison_page(struct page *page) · line 1502
Quoted 10 lines
return ret;mm/memory-failure.c +4
-1@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)* -EIO for pages on which we can not handle memory errors, * -EBUSY when get_hwpoison_page() has raced with page lifecycle * operations like allocation and free, -* -EHWPOISON when the page is hwpoisoned and taken off from buddy. +* -EHWPOISON when the page is hwpoisoned and taken off from buddy, +* -ENOTRECOVERABLE for stable kernel-owned pages the handler +* cannot recover (PG_reserved, slab, vmalloc, page tables, +* kernel stacks, and similar non-LRU/non-buddy pages). Did you test this patch series? I don't see how we ever get to
-ENOTRECOVERABLEthere ...
Yes, I did. I am using the following test case:
https://github.com/leitao/linux/commit/cfebe84ddeab5ac34ed456331db980d57e7025dc
# RUN_DESTRUCTIVE=1 tools/testing/selftests/mm/hwpoison-panic.sh # enabling /proc/sys/vm/panic_on_unrecoverable_memory_failure # injecting hwpoison at phys 0x2a00000 (Kernel rodata) # expecting kernel panic: 'Memory failure: <pfn>: unrecoverable page' [ 501.113256] Memory failure: 0x2a00: recovery action for reserved kernel page: Ignored [ 501.113956] Kernel panic - not syncing: Memory failure: 0x2a00: unrecoverable page
Quoted 34 lines
Even with
MF_COUNT_INCREASED, the first pass does:if (flags & MF_COUNT_INCREASED) count_increased = true; [...] if (PageHuge(p) || HWPoisonHandlable(p, flags)) { ret = 1; } else { if (pass++ < GET_PAGE_MAX_RETRY_NUM) { <- put_page(p); shake_page(p); count_increased = false; goto try_again; <- } put_page(p); ret = -ENOTRECOVERABLE; }Then we come back with count_increased=false:
try_again:if (!count_increased) { ret = __get_hwpoison_page(p, flags); <- if (!ret) { [...] } else if (ret == -EBUSY) { <- [...] ret = -EIO; goto out; <- } }
Quoted 6 lines
For slab/vmalloc/page-table pages,
__get_hwpoison_page()returns-EBUSY:if (!HWPoisonHandlable(&folio->page, flags)) return -EBUSY;so they still seem to end up as
-EIO... Am I missing something?
You are not, and thanks for catching this. I traced it again and the -ENOTRECOVERABLE branch is unreachable for slab/vmalloc/page-table pages exactly as you described. The __get_hwpoison_page() → -EBUSY → shake → retry loop catches them first and they exit as -EIO.
The selftest I am using (link above) only validated the PageReserved short-circuit added in patch 3, which lives in memory_failure() and never reaches get_any_page().
I even thought about this code path, and I was not convinced we should return -ENOTRECOVERABLE, thus I documented the following (as in this current patch)
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
shake_page(p);
goto try_again;
}
+ /*
+ * Return -EIO rather than -ENOTRECOVERABLE: this
+ * branch is also reached for pages that are merely
+ * off-LRU transiently (e.g. a folio in the middle
+ * of migration or compaction), which shake_page()
+ * cannot drag back. The caller cannot prove the
+ * page is permanently kernel-owned from here, so
+ * keep it on the recoverable errno.
+ */
ret = -EIO;
Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 89 lines · Breno Leitao
Quoted 74 lines · Lance Yang
Quoted 72 lines · Breno Leitao
get_any_page()collapses three different failure modes into a single-EIOreturn:* the
put_pagerace in the !count_increased path;
* theHWPoisonHandlable()rejection that bounces out of__get_hwpoison_page()with-EBUSYand exhaustsshake_page()retries;
* theHWPoisonHandlable()rejection that goes through thecount_increased/put_page/shake_pageretry loop.The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.
Distinguish them on the return path: keep
-EIOfor both theput_pagerace and the -EBUSY-after-retries branch (shake_page()cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep-EBUSYfor the allocation race (unchanged), and return-ENOTRECOVERABLEonly from the count_increased-trueHWPoisonHandlable()rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.Extend the unhandlable-page
pr_err()to fire for either errno and update theget_hwpoison_page()kerneldoc.
memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISONvia its existing "else if (res < 0)" branch, so this patch is a no-op for users ofmemory_failure()and only changes the errno thatsoft_offline_page()can propagate to its callers. A follow-up wires the new return code throughmemory_failure()and reportsMF_MSG_KERNELfor the unrecoverable cases.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>---
mm/memory-failure.c| 18 +++++++++++++++---1 file changed, 15 insertions(+), 3 deletions(-)mm/memory-failure.c +10
-1@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)shake_page (p); goto try_again; } +/* +* Return -EIO rather than -ENOTRECOVERABLE: this +* branch is also reached for pages that are merely +* off-LRU transiently (e.g. a folio in the middle +* of migration or compaction), which shake_page() +* cannot drag back. The caller cannot prove the +* page is permanently kernel-owned from here, so +* keep it on the recoverable errno. +*/ ret = -EIO ; goto out; }@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags) goto try_again; }put_page (p); - ret = -EIO ; + ret = -ENOTRECOVERABLE ; }out:
mm/memory-failure.c +1
-1-if (ret == -EIO ) +if (ret == -EIO || ret == -ENOTRECOVERABLE )pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));Quoted 13 lines
mm/memory-failure.c static int __get_unpoison_page(struct page *page) · line 1502
Quoted 10 lines
return ret;mm/memory-failure.c +4
-1@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)* -EIO for pages on which we can not handle memory errors, * -EBUSY when get_hwpoison_page() has raced with page lifecycle * operations like allocation and free, -* -EHWPOISON when the page is hwpoisoned and taken off from buddy. +* -EHWPOISON when the page is hwpoisoned and taken off from buddy, +* -ENOTRECOVERABLE for stable kernel-owned pages the handler +* cannot recover (PG_reserved, slab, vmalloc, page tables, +* kernel stacks, and similar non-LRU/non-buddy pages). Did you test this patch series? I don't see how we ever get to
-ENOTRECOVERABLEthere ...
Quoted 1 line
Yes, I did. I am using the following test case:
Okay.
Quoted 49 lines
https://github.com/leitao/linux/commit/cfebe84ddeab5ac34ed456331db980d57e7025dc
# RUN_DESTRUCTIVE=1 tools/testing/selftests/mm/hwpoison-panic.sh # enabling /proc/sys/vm/panic_on_unrecoverable_memory_failure # injecting hwpoison at phys 0x2a00000 (Kernel rodata) # expecting kernel panic: 'Memory failure: <pfn>: unrecoverable page' [ 501.113256] Memory failure: 0x2a00: recovery action for reserved kernel page: Ignored [ 501.113956] Kernel panic - not syncing: Memory failure: 0x2a00: unrecoverable pageQuoted 39 lines
Even with
MF_COUNT_INCREASED, the first pass does:if (flags & MF_COUNT_INCREASED) count_increased = true; [...] if (PageHuge(p) || HWPoisonHandlable(p, flags)) { ret = 1; } else { if (pass++ < GET_PAGE_MAX_RETRY_NUM) { <- put_page(p); shake_page(p); count_increased = false; goto try_again; <- } put_page(p); ret = -ENOTRECOVERABLE; }Then we come back with count_increased=false:
try_again:if (!count_increased) { ret = __get_hwpoison_page(p, flags); <- if (!ret) { [...] } else if (ret == -EBUSY) { <- [...] ret = -EIO; goto out; <- } }For slab/vmalloc/page-table pages,
__get_hwpoison_page()returns-EBUSY:if (!HWPoisonHandlable(&folio->page, flags)) return -EBUSY;
Quoted 6 lines
Quoted 1 line
so they still seem to end up as
-EIO... Am I missing something?You are not, and thanks for catching this. I traced it again and the
-ENOTRECOVERABLEbranch is unreachable for slab/vmalloc/page-table pages exactly as you described. The__get_hwpoison_page()→-EBUSY→ shake → retry loop catches them first and they exit as-EIO.
Wonder if it would be simpler to just do a positive check near the top of get_any_page() instead. Something like:
static bool hwpoison_unrecoverable_kernel_page(struct page *page,
unsigned long flags)
{
if ((flags & MF_SOFT_OFFLINE) && page_has_movable_ops(page))
return false;
return PageReserved(page) || PageSlab(page) ||
PageTable(page) || PageLargeKmalloc(page);
}static int get_any_page(struct page *p, unsigned long flags)
{
int ret = 0, pass = 0;
bool count_increased = false;
if (flags & MF_COUNT_INCREASED)
count_increased = true;
if (hwpoison_unrecoverable_kernel_page(p, flags)) {
if (count_increased)
put_page(p);
ret = -ENOTRECOVERABLE;
goto out;
}[...]
}
Then get_any_page() could return -ENOTRECOVERABLE only for page types we can positively identify as kernel-owned.
These types always fail HWPoisonHandlable(), so retrying does not really buy us anything for them.
Won't cover everything (vmalloc, kernel stacks, etc. have no page_type to key off), but that's fine - best effort, right?
Cheers, Lance
Quoted 23 lines
The selftest I am using (link above) only validated the PageReserved short-circuit added in patch 3, which lives in
memory_failure()and never reachesget_any_page().I even thought about this code path, and I was not convinced we should return
-ENOTRECOVERABLE, thus I documented the following (as in this current patch)@@ -1408,6 +1408,15 @@ static int
get_any_page(struct page *p, unsigned long flags)
shake_page(p);
gototry_again;
}
+ /*
+ * Return-EIOrather than-ENOTRECOVERABLE: this
+ * branch is also reached for pages that are merely
+ * off-LRU transiently (e.g. a folio in the middle
+ * of migration or compaction), whichshake_page()
+ * cannot drag back. The caller cannot prove the
+ * page is permanently kernel-owned from here, so
+ * keep it on the recoverable errno.
+ */
ret =-EIO;
Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 151 lines · Lance Yang
Quoted 89 lines · Breno Leitao
Quoted 74 lines · Lance Yang
Quoted 72 lines · Breno Leitao
get_any_page()collapses three different failure modes into a single-EIOreturn:* the
put_pagerace in the !count_increased path;
* theHWPoisonHandlable()rejection that bounces out of__get_hwpoison_page()with-EBUSYand exhaustsshake_page()retries;
* theHWPoisonHandlable()rejection that goes through thecount_increased/put_page/shake_pageretry loop.The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.
Distinguish them on the return path: keep
-EIOfor both theput_pagerace and the -EBUSY-after-retries branch (shake_page()cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep-EBUSYfor the allocation race (unchanged), and return-ENOTRECOVERABLEonly from the count_increased-trueHWPoisonHandlable()rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.Extend the unhandlable-page
pr_err()to fire for either errno and update theget_hwpoison_page()kerneldoc.
memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISONvia its existing "else if (res < 0)" branch, so this patch is a no-op for users ofmemory_failure()and only changes the errno thatsoft_offline_page()can propagate to its callers. A follow-up wires the new return code throughmemory_failure()and reportsMF_MSG_KERNELfor the unrecoverable cases.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>---
mm/memory-failure.c| 18 +++++++++++++++---1 file changed, 15 insertions(+), 3 deletions(-)mm/memory-failure.c +10
-1@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)shake_page (p); goto try_again; } +/* +* Return -EIO rather than -ENOTRECOVERABLE: this +* branch is also reached for pages that are merely +* off-LRU transiently (e.g. a folio in the middle +* of migration or compaction), which shake_page() +* cannot drag back. The caller cannot prove the +* page is permanently kernel-owned from here, so +* keep it on the recoverable errno. +*/ ret = -EIO ; goto out; }@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags) goto try_again; }put_page (p); - ret = -EIO ; + ret = -ENOTRECOVERABLE ; }out:
mm/memory-failure.c +1
-1-if (ret == -EIO ) +if (ret == -EIO || ret == -ENOTRECOVERABLE )pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));Quoted 13 lines
mm/memory-failure.c static int __get_unpoison_page(struct page *page) · line 1502
Quoted 10 lines
return ret;mm/memory-failure.c +4
-1@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)* -EIO for pages on which we can not handle memory errors, * -EBUSY when get_hwpoison_page() has raced with page lifecycle * operations like allocation and free, -* -EHWPOISON when the page is hwpoisoned and taken off from buddy. +* -EHWPOISON when the page is hwpoisoned and taken off from buddy, +* -ENOTRECOVERABLE for stable kernel-owned pages the handler +* cannot recover (PG_reserved, slab, vmalloc, page tables, +* kernel stacks, and similar non-LRU/non-buddy pages). Did you test this patch series? I don't see how we ever get to
-ENOTRECOVERABLEthere ...Quoted 1 line
Yes, I did. I am using the following test case:
Okay.
Quoted 49 lines
https://github.com/leitao/linux/commit/cfebe84ddeab5ac34ed456331db980d57e7025dc
# RUN_DESTRUCTIVE=1 tools/testing/selftests/mm/hwpoison-panic.sh # enabling /proc/sys/vm/panic_on_unrecoverable_memory_failure # injecting hwpoison at phys 0x2a00000 (Kernel rodata) # expecting kernel panic: 'Memory failure: <pfn>: unrecoverable page' [ 501.113256] Memory failure: 0x2a00: recovery action for reserved kernel page: Ignored [ 501.113956] Kernel panic - not syncing: Memory failure: 0x2a00: unrecoverable pageQuoted 39 lines
Even with
MF_COUNT_INCREASED, the first pass does:if (flags & MF_COUNT_INCREASED) count_increased = true; [...] if (PageHuge(p) || HWPoisonHandlable(p, flags)) { ret = 1; } else { if (pass++ < GET_PAGE_MAX_RETRY_NUM) { <- put_page(p); shake_page(p); count_increased = false; goto try_again; <- } put_page(p); ret = -ENOTRECOVERABLE; }Then we come back with count_increased=false:
try_again:if (!count_increased) { ret = __get_hwpoison_page(p, flags); <- if (!ret) { [...] } else if (ret == -EBUSY) { <- [...] ret = -EIO; goto out; <- } }For slab/vmalloc/page-table pages,
__get_hwpoison_page()returns-EBUSY:if (!HWPoisonHandlable(&folio->page, flags)) return -EBUSY;Quoted 6 lines
Quoted 1 line
so they still seem to end up as
-EIO... Am I missing something?You are not, and thanks for catching this. I traced it again and the
-ENOTRECOVERABLEbranch is unreachable for slab/vmalloc/page-table pages exactly as you described. The__get_hwpoison_page()→-EBUSY→ shake → retry loop catches them first and they exit as-EIO.
Quoted 5 lines
Wonder if it would be simpler to just do a positive check near the top of
get_any_page()instead. Something like:static bool hwpoison_unrecoverable_kernel_page(struct page *page,
unsigned long flags)
Ack. We probably want to call it something like HWPoisonKernelOwned() to follow the same naming sematics of these helpers, such as HWPoisonHandlable()
By the way, I will re-include the self test back to this patch series, In case they are not useful, we do not merge it.
Thanks for the review,
--breno
Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
On 2026/5/15 21:13, Breno Leitao wrote:
[...]
Quoted 4 lines
Quoted 4 lines
Wonder if it would be simpler to just do a positive check near the top of
get_any_page()instead. Something like:
Quoted 9 lines
Quoted 2 lines
static bool hwpoison_unrecoverable_kernel_page(struct page *page,
unsigned long flags)Ack. We probably want to call it something like
HWPoisonKernelOwned()to follow the same naming sematics of these helpers, such asHWPoisonHandlable()By the way, I will re-include the self test back to this patch series, In case they are not useful, we do not merge it.
Sounds good :)
Can you also test the relevant page types if possible, especially the ones the new helper is supposed to classify?
Cheers, Lance
Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 14 lines · Lance Yang
On 2026/5/15 21:13, Breno Leitao wrote:
[...]Quoted 10 lines
Quoted 6 lines
Wonder if it would be simpler to just do a positive check near the top of
get_any_page()instead. Something like:static bool hwpoison_unrecoverable_kernel_page(struct page *page,
unsigned long flags)Ack. We probably want to call it something like
HWPoisonKernelOwned()to follow the same naming sematics of these helpers, such asHWPoisonHandlable()
Quoted 8 lines
Quoted 3 lines
By the way, I will re-include the self test back to this patch series, In case they are not useful, we do not merge it.
Sounds good :)
Can you also test the relevant page types if possible, especially the ones the new helper is supposed to classify?
Ack. I will expand the test to cover different page types as well!
Thanks,
--breno
Miaohe Lin
maintainerRe: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
In reply to Breno LeitaoQuoted 61 lines · Breno Leitao
get_any_page()collapses three different failure modes into a single-EIOreturn:* the
put_pagerace in the !count_increased path;
* theHWPoisonHandlable()rejection that bounces out of__get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;* the
HWPoisonHandlable()rejection that goes through thecount_increased/put_page/shake_pageretry loop.The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.
Distinguish them on the return path: keep
-EIOfor both theput_pagerace and the -EBUSY-after-retries branch (shake_page()cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep-EBUSYfor the allocation race (unchanged), and return-ENOTRECOVERABLEonly from the count_increased-trueHWPoisonHandlable()rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.Extend the unhandlable-page
pr_err()to fire for either errno and update theget_hwpoison_page()kerneldoc.
memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISONvia its existing "else if (res < 0)" branch, so this patch is a no-op for users ofmemory_failure()and only changes the errno thatsoft_offline_page()can propagate to its callers. A follow-up wires the new return code throughmemory_failure()and reportsMF_MSG_KERNELfor the unrecoverable cases.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>--- mm/memory-failure.c | 18 +++++++++++++++--- 1 file changed, 15 insertions(+), 3 deletions(-)mm/memory-failure.c +9
-0@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)shake_page (p); goto try_again; } +/* +* Return -EIO rather than -ENOTRECOVERABLE: this +* branch is also reached for pages that are merely +* off-LRU transiently (e.g. a folio in the middle +* of migration or compaction), which shake_page() +* cannot drag back. The caller cannot prove the +* page is permanently kernel-owned from here, so +* keep it on the recoverable errno. +*/ ret = -EIO ; goto out; }
mm/memory-failure.c static int get_any_page(struct page *p, unsigned long flags) · line 1439
Quoted 5 lines
mm/memory-failure.c +1
-1@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags) goto try_again; }put_page (p); - ret = -EIO ; + ret = -ENOTRECOVERABLE ;
Theoretically, pages that are merely off-LRU transiently as you commented above could reach here too? Or am I miss something?
Thanks.
.
Quoted 20 lines
mm/memory-failure.c +5
-2} out: -if (ret == -EIO ) +if (ret == -EIO || ret == -ENOTRECOVERABLE )pr_err ("%#lx: unhandlable page.\n" ,page_to_pfn (p));return ret;@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)* -EIO for pages on which we can not handle memory errors, * -EBUSY when get_hwpoison_page() has raced with page lifecycle * operations like allocation and free, -* -EHWPOISON when the page is hwpoisoned and taken off from buddy. +* -EHWPOISON when the page is hwpoisoned and taken off from buddy, +* -ENOTRECOVERABLE for stable kernel-owned pages the handler +* cannot recover (PG_reserved, slab, vmalloc, page tables, +* kernel stacks, and similar non-LRU/non-buddy pages). */ static int get_hwpoison_page (struct page *p,unsigned long flags) {
PATCH v7 3/6 mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages
In reply to Breno LeitaoThe previous patch teaches get_any_page() to return -ENOTRECOVERABLE for stable unhandlable kernel pages (PG_reserved, slab, vmalloc, page tables, kernel stacks, ...). memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON, so callers that want to react to the unrecoverable cases (a panic option, smarter logging) cannot tell them apart from transient page-allocator races.
Turn the post-call branch into a switch over the get_hwpoison_page() return code: map -ENOTRECOVERABLE to MF_MSG_KERNEL and any other negative return to MF_MSG_GET_HWPOISON. case 0 keeps the existing free-buddy / kernel-high-order handling and case 1 falls through to the rest of memory_failure() unchanged.
The MF_MSG_KERNEL label and tracepoint string are kept as "reserved kernel page" to avoid breaking userspace tools that match on those literals; the enum value still adequately tags the failure even though it now also covers slab, vmalloc, page tables and kernel stack pages.
--- mm/memory-failure.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)
mm/memory-failure.c +15 -2
@@ -2410,7 +2410,8 @@ int memory_failure(unsigned long pfn, int flags)* that may make page_ref_freeze()/page_ref_unfreeze() mismatch. */ res =get_hwpoison_page (p, flags); -if (!res) { +switch (res) { +case 0 :if (is_free_buddy_page (p)) {if (take_page_off_buddy (p)) {page_ref_inc (p);
@@ -2429,7 +2430,19 @@ int memory_failure(unsigned long pfn, int flags) res =action_result (pfn,MF_MSG_KERNEL_HIGH_ORDER ,MF_IGNORED ); } goto unlock_mutex; - }else if (res <0 ) { +case 1 : +/* Got a refcount on a handlable page. */ +break ; +case -ENOTRECOVERABLE : +/* +* Stable unhandlable kernel-owned page (PG_reserved, +* slab, vmalloc, page tables, kernel stacks, ...). +* No recovery possible. +*/ + res =action_result (pfn,MF_MSG_KERNEL ,MF_IGNORED ); + goto unlock_mutex; +default : +/* Transient lifecycle race with the page allocator. */ res =action_result (pfn,MF_MSG_GET_HWPOISON ,MF_IGNORED ); goto unlock_mutex; }
Signature · 1 line
2.53.0-Meta
PATCH v7 4/6 mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()
In reply to Breno LeitaoThe previous patch already classifies PG_reserved pages as MF_MSG_KERNEL through the long path: get_hwpoison_page() calls __get_hwpoison_page() which fails HWPoisonHandlable(), get_any_page() exhausts its shake_page() retry budget, and the resulting -ENOTRECOVERABLE is mapped to MF_MSG_KERNEL by the switch. The outcome is correct but the work in between is wasted: shake_page() cannot turn a reserved page into a handlable one.
Detect PG_reserved up front in memory_failure() and report MF_MSG_KERNEL directly. put_ref_page() releases the caller's reference when MF_COUNT_INCREASED is set, which is important on the MADV_HWPOISON path where get_user_pages_fast() holds a reference across the call.
--- mm/memory-failure.c | 13 +++++++++++++ 1 file changed, 13 insertions(+)
mm/memory-failure.c +13 -0
@@ -2398,6 +2398,19 @@ int memory_failure(unsigned long pfn, int flags) goto unlock_mutex; } +/* +* PG_reserved pages are kernel-owned (memblock reservations, +* driver reservations, ...) and cannot be recovered. Skip the +* get_hwpoison_page() lifecycle dance and report MF_MSG_KERNEL +* straight away; HWPoisonHandlable() would just keep rejecting +* the page through the retry budget anyway. +*/ +if (PageReserved (p)) { +put_ref_page (pfn, flags); + res =action_result (pfn,MF_MSG_KERNEL ,MF_IGNORED ); + goto unlock_mutex; + } +/* * We need/can do nothing about count=0 pages. * 1) it's a free page, and therefore in safe hand:
Signature · 1 line
2.53.0-Meta
David Hildenbrand (Arm)
maintainerRe: [PATCH v7 4/6] mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()
Quoted 7 lines · Breno Leitao
The previous patch already classifies
PG_reservedpages asMF_MSG_KERNELthrough the long path:get_hwpoison_page()calls__get_hwpoison_page()which failsHWPoisonHandlable(),get_any_page()exhausts itsshake_page()retry budget, and the resulting-ENOTRECOVERABLEis mapped toMF_MSG_KERNELby the switch. The outcome is correct but the work in between is wasted:shake_page()cannot turn a reserved page into a handlable one.
If really required, can we just move the check right there, into get_any_page() etc?
Signature · 3 lines
Cheers, David
Re: [PATCH v7 4/6] mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()
Quoted 10 lines · David Hildenbrand (Arm)
Quoted 7 lines · Breno Leitao
The previous patch already classifies
PG_reservedpages asMF_MSG_KERNELthrough the long path:get_hwpoison_page()calls__get_hwpoison_page()which failsHWPoisonHandlable(),get_any_page()exhausts itsshake_page()retry budget, and the resulting-ENOTRECOVERABLEis mapped toMF_MSG_KERNELby the switch. The outcome is correct but the work in between is wasted:shake_page()cannot turn a reserved page into a handlable one.If really required, can we just move the check right there, into
get_any_page()etc?
Sure, we might move it to get_any_page(). I took this current approach based on the following facts:
1) Lance suggested it, and it sounded a good idea.
https://lore.kernel.org/all/[email protected]/
2) There is a _similar_ check close to this one in memory_failure(), just before this one:
if (TestSetPageHWPoison(p)) {
....
action_result() goto unlock_mutex;
}
and now
if (PageReserved(p)) {
...
action_result()
goto unlock_mutes;}
3) I wanted to give get it as real layering point, not handwaving.
That said, I will short-circuit reserved pages inside get_any_page(), in an updated version.
Again, thanks for the review and direction!
--breno
PATCH v7 5/6 mm/memory-failure: add panic option for unrecoverable pages
In reply to Breno LeitaoAdd a sysctl panic_on_unrecoverable_memory_failure that triggers a kernel panic when memory_failure() encounters pages that cannot be recovered. This provides a clean crash with useful debug information rather than allowing silent data corruption or a delayed crash at an unrelated code path.
Panic eligibility is intentionally narrow: only MF_MSG_KERNEL with result == MF_IGNORED panics. After the previous patch, MF_MSG_KERNEL covers PG_reserved pages and the kernel-owned pages promoted from get_hwpoison_page() via -ENOTRECOVERABLE (slab, vmalloc, page tables, kernel stacks, ...).
All other action types are excluded:
- MF_MSG_GET_HWPOISON and MF_MSG_KERNEL_HIGH_ORDER can be reached by transient refcount races with the page allocator (an in-flight buddy allocation has refcount 0 and is no longer on the buddy free list, briefly), and panicking on them would risk killing the box for what is actually a recoverable userspace page.
- MF_MSG_UNKNOWN means identify_page_state() could not classify the page; that is precisely the wrong basis for a panic decision.
--- mm/memory-failure.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
mm/memory-failure.c +23 -0
@@ -74,6 +74,8 @@ static int sysctl_memory_failure_recovery __read_mostly = 1;static int sysctl_enable_soft_offline__read_mostly =1 ; +static int sysctl_panic_on_unrecoverable_mf__read_mostly ; +atomic_long_t num_poisoned_pages__read_mostly =ATOMIC_LONG_INIT (0 );static bool hw_memory_failure__read_mostly = false;
@@ -155,6 +157,15 @@ static const struct ctl_table memory_failure_table[] = { .proc_handler = proc_dointvec_minmax, .extra1 = SYSCTL_ZERO, .extra2 = SYSCTL_ONE, + }, + { + .procname = "panic_on_unrecoverable_memory_failure", + .data = &sysctl_panic_on_unrecoverable_mf, + .maxlen = sizeof(sysctl_panic_on_unrecoverable_mf), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = SYSCTL_ZERO, + .extra2 = SYSCTL_ONE, } };
@@ -1267,6 +1278,15 @@ static void update_per_node_mf_stats(unsigned long pfn, ++mf_stats->total; } +static bool panic_on_unrecoverable_mf (enum mf_action_page_type type, +enum mf_result result) +{ +if (!sysctl_panic_on_unrecoverable_mf || result !=MF_IGNORED ) +return false; + +return type ==MF_MSG_KERNEL ; +} +/* * "Dirty/Clean" indication is not 100% accurate due to the possibility of * setting PG_dirty outside page lock. See also comment above set_page_dirty().
@@ -1284,6 +1304,9 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type,pr_err ("%#lx: recovery action for %s: %s\n" , pfn, action_page_types[type], action_name[result]); +if (panic_on_unrecoverable_mf (type, result)) +panic ("Memory failure: %#lx: unrecoverable page" , pfn); +return (result ==MF_RECOVERED || result ==MF_DELAYED ) ?0 : -EBUSY ; }
Signature · 1 line
2.53.0-Meta
PATCH v7 6/6 Documentation: document panic_on_unrecoverable_memory_failure sysctl
In reply to Breno LeitaoAdd documentation for the new vm.panic_on_unrecoverable_memory_failure sysctl, describing which failures trigger a panic (kernel-owned pages the handler cannot recover) and which are intentionally left out (transient allocator races and unclassified pages).
--- Documentation/admin-guide/sysctl/vm.rst | 80 +++++++++++++++++++++++++++++++++ 1 file changed, 80 insertions(+)
Documentation/admin-guide/sysctl/vm.rst +80 -0
@@ -67,6 +67,7 @@ Currently, these files are in /proc/sys/vm: - page-cluster - page_lock_unfairness - panic_on_oom +- panic_on_unrecoverable_memory_failure - percpu_pagelist_high_fraction - stat_interval - stat_refresh
@@ -925,6 +926,85 @@ panic_on_oom=2+kdump gives you very strong tool to investigate why oom happens. You can get snapshot. +panic_on_unrecoverable_memory_failure +====================================== + +When a hardware memory error (e.g. multi-bit ECC) hits a kernel page +that cannot be recovered by the memory failure handler, the default +behaviour is to ignore the error and continue operation. This is +dangerous because the corrupted data remains accessible to the kernel, +risking silent data corruption or a delayed crash when the poisoned +memory is next accessed. + +When enabled, this sysctl triggers a panic on memory failure events +hitting reserved (``PageReserved``) memory: firmware reservations, +the kernel image, vDSO, the zero page, and similar memblock-reserved +regions. These are owned by the kernel, are not managed by the page +allocator, and cannot be recovered by the memory failure handler. + +Other unrecoverable kernel-owned populations (slab, vmalloc, page +tables, kernel stacks, ...) are not currently covered by this +sysctl. The handler cannot reliably distinguish them from a +userspace folio temporarily off the LRU during migration or +compaction, and the cost of a false-positive panic on a recoverable +userspace page is too high. Such pages still go through the +standard MF_MSG_GET_HWPOISON path: ``PG_hwpoison`` is set on them +and a delayed crash on the next access remains possible. Coverage +may grow in the future as the handler gains stronger +kernel-ownership signals. + +Recoverable failure paths are also intentionally left out: in-flight +buddy allocations and other transient races with the page allocator +can reach the same diagnostic, and panicking on them would risk +killing the box for a page destined for userspace where the standard +SIGBUS recovery path applies. Pages whose state could not be +classified at all are not covered either, since an unknown state is +not a sound basis for a panic decision. + +For many environments it is preferable to panic immediately with a clean +crash dump that captures the original error context, rather than to +continue and face a random crash later whose cause is difficult to +diagnose. + +Use cases +--------- + +This option is most useful in environments where unattributed crashes +are expensive to debug or where data integrity must take precedence +over availability: + +* Large fleets, where multi-bit ECC errors on kernel pages are observed + regularly and post-mortem analysis of an unrelated downstream crash + (often seconds to minutes after the original error) consumes + significant engineering effort. + +* Systems configured with kdump, where panicking at the moment of the + hardware error produces a vmcore that still contains the faulting + address, the affected page state, and the originating MCE/GHES + record — context that is typically lost by the time a delayed crash + occurs. + +* High-availability clusters that rely on fast, deterministic node + failure for failover, and prefer an immediate panic over silent data + corruption propagating to replicas or persistent storage. + +* Kernel and platform developers reproducing hwpoison issues with + tools such as ``mce-inject`` or error-injection debugfs interfaces, + where panicking on the unrecoverable path makes regressions + immediately visible instead of surfacing as later, unrelated + failures. + += ===================================================================== +0 Try to continue operation (default). +1 Panic immediately. If the ``panic`` sysctl is also non-zero then the + machine will be rebooted. += ===================================================================== + +Example:: + + echo 1 > /proc/sys/vm/panic_on_unrecoverable_memory_failure + + percpu_pagelist_high_fraction =============================
Signature · 1 line
2.53.0-Meta