mm/memory-failure: add panic option for unrecoverable pages
Subsystems (7)
v8 cover mm/memory-failure: add panic option for unrecoverable pages
A multi-bit ECC error on a kernel-owned page that the memory failure handler cannot recover is currently swallowed: PG_hwpoison is set, the event is logged, and the kernel keeps running. The corrupted memory remains accessible to the kernel and either drives silent data corruption or surfaces seconds-to-minutes later as an apparently unrelated crash. In a large fleet that delayed, unattributable crash turns into significant engineering effort to root-cause; in a kdump configuration, by the time the crash happens the original error context (faulting PFN, MCE/GHES record, page state) is long gone.
This series adds an opt-in sysctl, vm.panic_on_unrecoverable_memory_failure, that converts an unrecoverable kernel-page hwpoison event into an immediate panic with a clean dmesg/vmcore that still contains the original failure context. The default is disabled so existing workloads see no change.
There is a selftest that test different cases, and I tested it using the following variants:
┌─────────┬──────────┬───────────────────────────────────────────────────────────┐ │ Variant │ PFN │ Result │ ├─────────┼──────────┼───────────────────────────────────────────────────────────┤ │ rodata │ 0x2600 │ Panic with "Memory failure: 0x2600: unrecoverable page" │ ├─────────┼──────────┼───────────────────────────────────────────────────────────┤ │ slab │ 0x100032 │ Panic with "Memory failure: 0x100032: unrecoverable page" │ ├─────────┼──────────┼───────────────────────────────────────────────────────────┤ │ pgtable │ 0x100000 │ Panic with "Memory failure: 0x100000: unrecoverable page" │ └─────────┴──────────┴───────────────────────────────────────────────────────────┘
Each one shows the same call trace, exactly the path the series builds:
hard_offline_page_store
→ memory_failure
→ action_result
→ panic("Memory failure: %#lx: unrecoverable page")
---
Changes in v8:
- Commit message rewording (David)
- Add HWPoisonKernelOwned() helper (Lance)
- Removed patch "mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()"
- Broaden the selftest (Lance)
- Link to v7: https://patch.msgid.link/[email protected]
Changes in v7:
- Move the PG_reserved / unhandlable-kernel-page classification into get_any_page() and surface it via -ENOTRECOVERABLE, per David Hildenbrand's and Lance Yang's review of v6. This drops the is_reserved snapshot in memory_failure() and the mf_get_page_status enum / out-parameter introduced in v6.
- Restructure the post-call branch in memory_failure() as a switch
over the get_hwpoison_page() return code (David).
- Drop the "reserved" qualifier from the MF_MSG_KERNEL label and the matching tracepoint string; the enum now covers both PG_reserved
pages and other unhandlable kernel pages.
- Squash the former patches 1/4 ("MF_MSG_KERNEL for reserved pages")
and 2/4 ("classify get_any_page() failures by reason") into a
single classification patch; the series is now 3 patches.
- Simplify panic_on_unrecoverable_mf() to a single return statement (David).
- Link to v6: https://patch.msgid.link/[email protected]
Changes in v6:
- Dropped the selftest given the value was not clear
- Get the status of the failure from get_any_page()
- Small nits from different people/AIs.
- Link to v5: https://patch.msgid.link/[email protected]
Changes in v5:
- Add vm.panic_on_unrecoverable_memory_failure sysctl to panic on unrecoverable kernel page hwpoison events (reserved pages, refcount-0 non-buddy pages, unknown state), with a recheck to avoid racing with concurrent buddy allocations. (Miaohe)
- Distinguish reserved pages as MF_MSG_KERNEL in memory_failure(), document the new sysctl in Documentation/admin-guide/sysctl/vm.rst, and add a selftest verifying SIGBUS recovery on userspace pages still works when the sysctl is enabled. (Miaohe)
- Added a selftest
- Link to v4:
https://patch.msgid.link/[email protected]
Changes in v4:
- Drop CONFIG_BOOTPARAM_MEMORY_FAILURE_PANIC kernel configuration option.
- Split the reserved page classification (MF_MSG_KERNEL) into its own patch, separate from the panic mechanism.
- Document why the buddy allocator TOCTOU race (between get_hwpoison_page() and is_free_buddy_page()) cannot cause false
positives: PG_hwpoison is set beforehand and check_new_page() in the page allocator rejects hwpoisoned pages.
- Document the narrow LRU isolation race window for MF_MSG_UNKNOWN and its mitigation via identify_page_state()'s two-pass design.
- Explicitly document why MF_MSG_GET_HWPOISON is excluded from the
panic conditions (shared path with transient races and non-reserved kernel memory).
- Link to v3: https://patch.msgid.link/[email protected]
Changes in v3:
- Rename is_unrecoverable_memory_failure() to panic_on_unrecoverable_mf() as suggested by maintainer.
- Add CONFIG_BOOTPARAM_MEMORY_FAILURE_PANIC kernel configuration option, similar to CONFIG_BOOTPARAM_HARDLOCKUP_PANIC.
- Add documentation for the sysctl and CONFIG option.
- Add code comments documenting the panic condition design rationale and how the retry mechanism mitigates false positives from buddy allocator races.
- Link to v2: https://patch.msgid.link/[email protected]
Changes in v2:
- Panic on MF_MSG_KERNEL, MF_MSG_KERNEL_HIGH_ORDER and MF_MSG_UNKNOWN instead of MF_MSG_GET_HWPOISON.
- Report MF_MSG_KERNEL for reserved pages when get_hwpoison_page() fails instead of MF_MSG_GET_HWPOISON.
- Link to v1: https://patch.msgid.link/[email protected]
To: Miaohe Lin <[email protected]>
To: Naoya Horiguchi <[email protected]>
To: Andrew Morton <[email protected]>
To: Steven Rostedt <[email protected]>
To: Masami Hiramatsu <[email protected]>
To: Mathieu Desnoyers <[email protected]>
To: Jonathan Corbet <[email protected]>
To: Shuah Khan <[email protected]>
To: David Hildenbrand <[email protected]>
To: Lorenzo Stoakes <[email protected]>
To: "Liam R. Howlett" <[email protected]>
To: Vlastimil Babka <[email protected]>
To: Mike Rapoport <[email protected]>
To: Suren Baghdasaryan <[email protected]>
To: Michal Hocko <[email protected]>
To: Shuah Khan <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
---
Breno Leitao (6):
mm/memory-failure: drop dead error_states[] entry for reserved pages mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages mm/memory-failure: add panic option for unrecoverable pages
Documentation: document panic_on_unrecoverable_memory_failure sysctl selftests/mm: add hwpoison-panic destructive test
Documentation/admin-guide/sysctl/vm.rst | 85 ++++++++++++ mm/memory-failure.c | 96 ++++++++++--- tools/testing/selftests/mm/Makefile | 1 + tools/testing/selftests/mm/hwpoison-panic.sh | 193 +++++++++++++++++++++++++++ 4 files changed, 357 insertions(+), 18 deletions(-)
---
base-commit: e7e28506af98ce4e1059e5ec59334b335c00a246
change-id: 20260323-ecc_panic-4e473b83087c
Best regards,
Signature · 1 line
Breno Leitao <[email protected]>
PATCH v8 1/6 mm/memory-failure: drop dead error_states[] entry for reserved pages
The first entry of error_states[],
{ reserved, reserved, MF_MSG_KERNEL, me_kernel },
is unreachable. identify_page_state() has two callers, and neither one can dispatch a PG_reserved page to me_kernel():
* memory_failure() reaches identify_page_state() only after get_hwpoison_page() returned 1. get_any_page() reaches that return only via __get_hwpoison_page(), which only takes a refcount when the page is HWPoisonHandlable(). HWPoisonHandlable() is an allowlist for LRU, free-buddy, and (for soft-offline) movable_ops pages -- PG_reserved pages do not satisfy any of these, so they fail with -EBUSY/-EIO long before identify_page_state() runs.
* try_memory_failure_hugetlb() reaches identify_page_state() only via the MF_HUGETLB_IN_USED branch, where the page is necessarily a hugetlb folio. hugetlb folios don't carry PG_reserved at that
point: hugetlb_folio_init_vmemmap() calls __folio_clear_reserved() during init, so the reserved entry would not match even if it were still present.
me_kernel() never executes and the entry exists only to be matched against by code that cannot see it.
Drop the entry, the me_kernel() helper, and the now-unused "reserved" macro. Leave the MF_MSG_KERNEL enum value in place: it remains part of the tracepoint and pr_err() string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.
No functional change.
--- mm/memory-failure.c | 14 -------------- 1 file changed, 14 deletions(-)
mm/memory-failure.c +0 -14
@@ -980,17 +980,6 @@ static bool has_extra_refcount(struct page_state *ps, struct page *p,return false; } -/* -* Error hit kernel page. -* Do nothing, try to be lucky and not touch this instead. For a few cases we -* could be more sophisticated. -*/ -static int me_kernel (struct page_state *ps,struct page *p) -{ -unlock_page (p); -return MF_IGNORED ; -} -/* * Page in unknown state. Do nothing. * This is a catch-all in case we fail to make sense of the page state.
@@ -1199,10 +1188,8 @@ static int me_huge_page(struct page_state *ps, struct page *p)#define mlock (1UL << PG_mlocked)#define lru (1UL << PG_lru)#define head (1UL << PG_head) -#define reserved (1UL << PG_reserved)static struct page_state error_states[] = { - { reserved, reserved,MF_MSG_KERNEL , me_kernel },/* * free pages are specially detected outside this table: * PG_buddy pages only make a small fraction of all free pages.
@@ -1234,7 +1221,6 @@ static struct page_state error_states[] = {#undef mlock#undef lru#undef head -#undef reservedstatic void update_per_node_mf_stats (unsigned long pfn,enum mf_result result)
Signature · 1 line
2.54.0
PATCH v8 2/6 mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
In reply to Breno Leitaoget_any_page() collapses every HWPoisonHandlable() rejection into a single -EIO via the __get_hwpoison_page() -> -EBUSY -> shake_page() -> retry path. That is correct for the transient case (a userspace folio briefly off LRU during migration or compaction, which a later shake can drag back), but wrong for stable kernel-owned pages: slab, page-table, large-kmalloc and PG_reserved pages will never become HWPoisonHandlable(), so the retry loop is wasted work and the final -EIO loses the "this is structurally unrecoverable" information. memory_failure() then maps -EIO into MF_MSG_GET_HWPOISON, which the panic-on-unrecoverable sysctl deliberately does not act on.
Introduce HWPoisonKernelOwned(), a small predicate that positively identifies pages the hwpoison handler cannot recover from:
HWPoisonKernelOwned(p, flags) :=
!(MF_SOFT_OFFLINE && page_has_movable_ops(p)) &&
(PageReserved(p) || PageSlab(p) ||
PageTable(p) || PageLargeKmalloc(p))The MF_SOFT_OFFLINE / page_has_movable_ops() opt-out mirrors the same exception in HWPoisonHandlable(): soft-offline is allowed to migrate movable_ops pages even though they are not on the LRU, and we must not pre-empt that with an unrecoverable verdict.
The list is intentionally not exhaustive. vmalloc and kernel-stack pages, for example, do not carry a page_type bit and would need a different oracle; they keep going through the existing retry path unchanged. This is the smallest set we can identify with certainty by page type.
Wire the helper into the top of get_any_page() to short-circuit those pages before the retry loop runs. On a hit, drop the caller's MF_COUNT_INCREASED reference (if any) and return -ENOTRECOVERABLE straight away. Pages outside the helper's positive list still take the existing retry path and return -EIO, leaving operator-visible behaviour for those cases unchanged.
Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc to document the new return.
memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch on its own only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires -ENOTRECOVERABLE through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases, which is what the panic_on_unrecoverable_memory_failure sysctl observes.
--- mm/memory-failure.c | 42 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-)
mm/memory-failure.c +40 -2
@@ -1325,6 +1325,28 @@ static inline bool HWPoisonHandlable(struct page *page, unsigned long flags)return PageLRU (page) ||is_free_buddy_page (page); } +/* +* Positive identification of pages the hwpoison handler cannot recover. +* These page types are owned by kernel internals (no userspace mapping +* to unmap, no file mapping to invalidate, no migration target), so the +* shake_page() / retry loop in get_any_page() can never turn them into +* something HWPoisonHandlable() will accept. Short-circuit them to +* -ENOTRECOVERABLE so callers can panic on operator request instead of +* spinning through retries that exit as a transient-looking -EIO. +* +* The MF_SOFT_OFFLINE / page_has_movable_ops() opt-out mirrors +* HWPoisonHandlable(): soft-offline is allowed to migrate movable_ops +* pages even though they are not on the LRU. +*/ +static inline bool HWPoisonKernelOwned (struct page *page,unsigned long flags) +{ +if ((flags &MF_SOFT_OFFLINE ) &&page_has_movable_ops (page)) +return false; + +return PageReserved (page) ||PageSlab (page) || +PageTable (page) ||PageLargeKmalloc (page); +} +static int __get_hwpoison_page (struct page *page,unsigned long flags) {struct folio *folio =page_folio (page);
@@ -1371,6 +1393,19 @@ static int get_any_page(struct page *p, unsigned long flags)if (flags &MF_COUNT_INCREASED ) count_increased = true; +/* +* Page types we know are kernel-owned and cannot be recovered. +* Short-circuit before the shake_page() / retry loop, which +* cannot turn any of these into something HWPoisonHandlable(). +* Drop the caller's reference if MF_COUNT_INCREASED took one. +*/ +if (HWPoisonKernelOwned (p, flags)) { +if (count_increased) +put_page (p); + ret = -ENOTRECOVERABLE ; + goto out; + } + try_again:if (!count_increased) { ret =__get_hwpoison_page (p, flags);
@@ -1418,7 +1453,7 @@ static int get_any_page(struct page *p, unsigned long flags) ret = -EIO ; } out: -if (ret == -EIO ) +if (ret == -EIO || ret == -ENOTRECOVERABLE )pr_err ("%#lx: unhandlable page.\n" ,page_to_pfn (p));return ret;
@@ -1475,7 +1510,10 @@ static int __get_unpoison_page(struct page *page)* -EIO for pages on which we can not handle memory errors, * -EBUSY when get_hwpoison_page() has raced with page lifecycle * operations like allocation and free, -* -EHWPOISON when the page is hwpoisoned and taken off from buddy. +* -EHWPOISON when the page is hwpoisoned and taken off from buddy, +* -ENOTRECOVERABLE for kernel-owned pages identified by +* HWPoisonKernelOwned() (PG_reserved, slab, +* page-table, large-kmalloc) that the handler cannot recover. */ static int get_hwpoison_page (struct page *p,unsigned long flags) {
Signature · 1 line
2.54.0
PATCH v8 3/6 mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages
In reply to Breno LeitaoThe previous patch teaches get_any_page() to return -ENOTRECOVERABLE for stable unhandlable kernel pages (PG_reserved, slab, page tables, large-kmalloc). memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON, so callers that want to react to the unrecoverable cases (a panic option, smarter logging) cannot tell them apart from transient page-allocator races.
Turn the post-call branch into a switch over the get_hwpoison_page() return code: map -ENOTRECOVERABLE to MF_MSG_KERNEL and any other negative return to MF_MSG_GET_HWPOISON. case 0 keeps the existing free-buddy / kernel-high-order handling and case 1 falls through to the rest of memory_failure() unchanged.
The MF_MSG_KERNEL label and tracepoint string are kept as "reserved kernel page" to avoid breaking userspace tools that match on those literals; the enum value still adequately tags the failure even though it now also covers slab, page tables and large-kmalloc pages.
--- mm/memory-failure.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)
mm/memory-failure.c +15 -2
@@ -2426,7 +2426,8 @@ int memory_failure(unsigned long pfn, int flags)* that may make page_ref_freeze()/page_ref_unfreeze() mismatch. */ res =get_hwpoison_page (p, flags); -if (!res) { +switch (res) { +case 0 :if (is_free_buddy_page (p)) {if (take_page_off_buddy (p)) {page_ref_inc (p);
@@ -2445,7 +2446,19 @@ int memory_failure(unsigned long pfn, int flags) res =action_result (pfn,MF_MSG_KERNEL_HIGH_ORDER ,MF_IGNORED ); } goto unlock_mutex; - }else if (res <0 ) { +case 1 : +/* Got a refcount on a handlable page. */ +break ; +case -ENOTRECOVERABLE : +/* +* Stable unhandlable kernel-owned page (PG_reserved, +* slab, page tables, large-kmalloc). +* No recovery possible. +*/ + res =action_result (pfn,MF_MSG_KERNEL ,MF_IGNORED ); + goto unlock_mutex; +default : +/* Transient lifecycle race with the page allocator. */ res =action_result (pfn,MF_MSG_GET_HWPOISON ,MF_IGNORED ); goto unlock_mutex; }
Signature · 1 line
2.54.0
PATCH v8 4/6 mm/memory-failure: add panic option for unrecoverable pages
In reply to Breno LeitaoAdd a sysctl panic_on_unrecoverable_memory_failure (disabled by default) that triggers a kernel panic when memory_failure() encounters pages that cannot be recovered. This provides a clean crash with useful debug information rather than allowing silent data corruption or a delayed crash at an unrelated code path.
Panic eligibility is intentionally narrow: only MF_MSG_KERNEL with result == MF_IGNORED panics. After the previous patch, MF_MSG_KERNEL covers PG_reserved pages and the kernel-owned pages promoted from get_hwpoison_page() via -ENOTRECOVERABLE (slab, page tables, large-kmalloc).
All other action types are excluded:
- MF_MSG_GET_HWPOISON and MF_MSG_KERNEL_HIGH_ORDER can be reached by transient refcount races with the page allocator (an in-flight buddy allocation has refcount 0 and is no longer on the buddy free list, briefly), and panicking on them would risk killing the box for what is actually a recoverable userspace page.
- MF_MSG_UNKNOWN means identify_page_state() could not classify the page; that is precisely the wrong basis for a panic decision.
--- mm/memory-failure.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
mm/memory-failure.c +23 -0
@@ -74,6 +74,8 @@ static int sysctl_memory_failure_recovery __read_mostly = 1;static int sysctl_enable_soft_offline__read_mostly =1 ; +static int sysctl_panic_on_unrecoverable_mf__read_mostly ; +atomic_long_t num_poisoned_pages__read_mostly =ATOMIC_LONG_INIT (0 );static bool hw_memory_failure__read_mostly = false;
@@ -155,6 +157,15 @@ static const struct ctl_table memory_failure_table[] = { .proc_handler = proc_dointvec_minmax, .extra1 = SYSCTL_ZERO, .extra2 = SYSCTL_ONE, + }, + { + .procname = "panic_on_unrecoverable_memory_failure", + .data = &sysctl_panic_on_unrecoverable_mf, + .maxlen = sizeof(sysctl_panic_on_unrecoverable_mf), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = SYSCTL_ZERO, + .extra2 = SYSCTL_ONE, } };
@@ -1255,6 +1266,15 @@ static void update_per_node_mf_stats(unsigned long pfn, ++mf_stats->total; } +static bool panic_on_unrecoverable_mf (enum mf_action_page_type type, +enum mf_result result) +{ +if (!sysctl_panic_on_unrecoverable_mf || result !=MF_IGNORED ) +return false; + +return type ==MF_MSG_KERNEL ; +} +/* * "Dirty/Clean" indication is not 100% accurate due to the possibility of * setting PG_dirty outside page lock. See also comment above set_page_dirty().
@@ -1272,6 +1292,9 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type,pr_err ("%#lx: recovery action for %s: %s\n" , pfn, action_page_types[type], action_name[result]); +if (panic_on_unrecoverable_mf (type, result)) +panic ("Memory failure: %#lx: unrecoverable page" , pfn); +return (result ==MF_RECOVERED || result ==MF_DELAYED ) ?0 : -EBUSY ; }
Signature · 1 line
2.54.0
PATCH v8 5/6 Documentation: document panic_on_unrecoverable_memory_failure sysctl
In reply to Breno LeitaoAdd documentation for the new vm.panic_on_unrecoverable_memory_failure sysctl, describing which failures trigger a panic (kernel-owned pages the handler cannot recover) and which are intentionally left out (transient allocator races and unclassified pages).
--- Documentation/admin-guide/sysctl/vm.rst | 85 +++++++++++++++++++++++++++++++++ 1 file changed, 85 insertions(+)
Documentation/admin-guide/sysctl/vm.rst +85 -0
@@ -67,6 +67,7 @@ Currently, these files are in /proc/sys/vm: - page-cluster - page_lock_unfairness - panic_on_oom +- panic_on_unrecoverable_memory_failure - percpu_pagelist_high_fraction - stat_interval - stat_refresh
@@ -925,6 +926,90 @@ panic_on_oom=2+kdump gives you very strong tool to investigate why oom happens. You can get snapshot. +panic_on_unrecoverable_memory_failure +====================================== + +When a hardware memory error (e.g. multi-bit ECC) hits a kernel page +that cannot be recovered by the memory failure handler, the default +behaviour is to ignore the error and continue operation. This is +dangerous because the corrupted data remains accessible to the kernel, +risking silent data corruption or a delayed crash when the poisoned +memory is next accessed. + +When enabled, this sysctl triggers a panic on memory failure events +hitting kernel-owned pages that the handler cannot recover: +``PageReserved`` (firmware reservations, kernel image, vDSO, zero +page, and similar memblock-reserved regions), ``PageSlab``, +``PageTable``, and ``PageLargeKmalloc``. These are owned by the +kernel and the memory failure handler cannot reliably evict their +contents. + +For soft offline (``madvise(MADV_SOFT_OFFLINE)``, +``/sys/devices/system/memory/soft_offline_page``), pages owned by +``movable_ops`` are exempted, since soft offline is allowed to +migrate them even though they are not on the LRU. + +Other unrecoverable kernel-owned populations (vmalloc allocations, +kernel stack pages, ...) are not currently covered because the +handler has no page-type signal that distinguishes them from a +userspace folio temporarily off the LRU during migration or +compaction. Such pages still go through the standard +MF_MSG_GET_HWPOISON path: ``PG_hwpoison`` is set on them and a +delayed crash on the next access remains possible. Coverage may +grow as the handler gains stronger kernel-ownership signals. + +Recoverable failure paths are also intentionally left out: in-flight +buddy allocations and other transient races with the page allocator +can reach the same diagnostic, and panicking on them would risk +killing the box for a page destined for userspace where the standard +SIGBUS recovery path applies. Pages whose state could not be +classified at all are not covered either, since an unknown state is +not a sound basis for a panic decision. + +For many environments it is preferable to panic immediately with a clean +crash dump that captures the original error context, rather than to +continue and face a random crash later whose cause is difficult to +diagnose. + +Use cases +--------- + +This option is most useful in environments where unattributed crashes +are expensive to debug or where data integrity must take precedence +over availability: + +* Large fleets, where multi-bit ECC errors on kernel pages are observed + regularly and post-mortem analysis of an unrelated downstream crash + (often seconds to minutes after the original error) consumes + significant engineering effort. + +* Systems configured with kdump, where panicking at the moment of the + hardware error produces a vmcore that still contains the faulting + address, the affected page state, and the originating MCE/GHES + record — context that is typically lost by the time a delayed crash + occurs. + +* High-availability clusters that rely on fast, deterministic node + failure for failover, and prefer an immediate panic over silent data + corruption propagating to replicas or persistent storage. + +* Kernel and platform developers reproducing hwpoison issues with + tools such as ``mce-inject`` or error-injection debugfs interfaces, + where panicking on the unrecoverable path makes regressions + immediately visible instead of surfacing as later, unrelated + failures. + += ===================================================================== +0 Try to continue operation (default). +1 Panic immediately. If the ``panic`` sysctl is also non-zero then the + machine will be rebooted. += ===================================================================== + +Example:: + + echo 1 > /proc/sys/vm/panic_on_unrecoverable_memory_failure + + percpu_pagelist_high_fraction =============================
Signature · 1 line
2.54.0
PATCH v8 6/6 selftests/mm: add hwpoison-panic destructive test
In reply to Breno LeitaoAdd a destructive selftest that verifies vm.panic_on_unrecoverable_memory_failure actually panics when a hwpoison error hits a kernel-owned page.
Three "kinds" of kernel-owned page can be targeted, selectable via the script's first positional argument (default: rodata):
rodata - a PG_reserved page in the kernel rodata range, sourced
from the "Kernel rodata" sub-resource of "System RAM" in /proc/iomem. That entry is reported on every major architecture and guarantees the chosen PFN is backed by struct page (an online System RAM range, not a firmware hole), is PG_reserved, and is read-only -- so even if the panic fails to fire for some reason, the resulting PG_hwpoison marker on rodata does not corrupt writable kernel state.
slab - a slab page found by walking /proc/kpageflags for the
first PFN with KPF_SLAB set (and KPF_HWPOISON / KPF_NOPAGE / KPF_COMPOUND_TAIL clear). Exercises the get_any_page() path on a non PG_reserved kernel-owned page and so catches regressions where get_any_page() collapses kernel-owned pages into a transient -EIO instead of -ENOTRECOVERABLE.
pgtable - same as slab, but the PFN is selected via KPF_PGTABLE.
PageLargeKmalloc, the fourth page type matched by HWPoisonKernelOwned(), is intentionally not covered: it is a PAGE_TYPE_OPS flag with no /proc/kpageflags bit, so selecting such a PFN from userspace is not feasible. The slab and pgtable variants already exercise the same get_any_page() positive-check branch.
The script enables the sysctl and writes the selected physical address to /sys/devices/system/memory/hard_offline_page. A successful run crashes the kernel with
Memory failure: <pfn>: unrecoverable page
A return from the inject means the panic did not fire and the test fails. Test outcome is therefore observed externally (serial console, kdump) rather than from the script's own exit code.
The script is intentionally NOT wired into run_vmtests.sh: every successful run panics the kernel, which is incompatible with the sequential "run each category in the same VM" model that run_vmtests.sh assumes. It is also not registered as a TEST_PROGS / ksft_* wrapper so a default kselftest run does not opt itself into a panic. The script is meant to be executed manually inside a disposable VM (e.g. virtme-ng), one variant per VM boot, and requires RUN_DESTRUCTIVE=1 in the environment as a safety net.
--- tools/testing/selftests/mm/Makefile | 1 + tools/testing/selftests/mm/hwpoison-panic.sh | 193 +++++++++++++++++++++++++++ 2 files changed, 194 insertions(+)
tools/testing/selftests/mm/Makefile +1 -0
@@ -181,6 +181,7 @@ TEST_FILES += charge_reserved_hugetlb.sh TEST_FILES += hugetlb_reparenting_test.sh TEST_FILES += test_page_frag.sh TEST_FILES += run_vmtests.sh +TEST_FILES += hwpoison-panic.sh # required by charge_reserved_hugetlb.sh TEST_FILES += write_hugetlb_memory.sh
tools/testing/selftests/mm/hwpoison-panic.sh new file +193 -0
193 lines not on this page · raw message
Signature · 1 line
2.54.0
Andrew Morton
maintainerQuoted 10 lines · On Wed, 27 May 2026 07:06:13 -0700 Breno Leitao <[email protected]> wrote:
A multi-bit ECC error on a kernel-owned page that the memory failure handler cannot recover is currently swallowed:
PG_hwpoisonis set, the event is logged, and the kernel keeps running. The corrupted memory remains accessible to the kernel and either drives silent data corruption or surfaces seconds-to-minutes later as an apparently unrelated crash. In a large fleet that delayed, unattributable crash turns into significant engineering effort to root-cause; in a kdump configuration, by the time the crash happens the original error context (faulting PFN, MCE/GHES record, page state) is long gone.
Quoted 6 lines
This series adds an opt-in sysctl, vm.panic_on_unrecoverable_memory_failure, that converts an unrecoverable kernel-page hwpoison event into an immediate panic with a clean dmesg/vmcore that still contains the original failure context. The default is disabled so existing workloads see no change.
Thanks. That does seem useful.
I'll pass at this time, due to -rc5 and not-very-reviewed.
AI review said a few things. It claims to have found one pre-existing issue.
https://sashiko.dev/#/patchset/[email protected]
Miaohe Lin
maintainerRe: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
In reply to Breno LeitaoQuoted 40 lines · Breno Leitao
get_any_page()collapses everyHWPoisonHandlable()rejection into a single-EIOvia the__get_hwpoison_page()->-EBUSY->shake_page()-> retry path. That is correct for the transient case (a userspace folio briefly off LRU during migration or compaction, which a later shake can drag back), but wrong for stable kernel-owned pages: slab, page-table, large-kmalloc andPG_reservedpages will never becomeHWPoisonHandlable(), so the retry loop is wasted work and the final-EIOloses the "this is structurally unrecoverable" information.memory_failure()then maps-EIOintoMF_MSG_GET_HWPOISON, which the panic-on-unrecoverable sysctl deliberately does not act on.Introduce
HWPoisonKernelOwned(), a small predicate that positively identifies pages the hwpoison handler cannot recover from:HWPoisonKernelOwned(p, flags) := !(MF_SOFT_OFFLINE && page_has_movable_ops(p)) && (PageReserved(p) || PageSlab(p) || PageTable(p) || PageLargeKmalloc(p))The
MF_SOFT_OFFLINE/page_has_movable_ops()opt-out mirrors the same exception inHWPoisonHandlable(): soft-offline is allowed to migratemovable_opspages even though they are not on the LRU, and we must not pre-empt that with an unrecoverable verdict.The list is intentionally not exhaustive. vmalloc and kernel-stack pages, for example, do not carry a
page_typebit and would need a different oracle; they keep going through the existing retry path unchanged. This is the smallest set we can identify with certainty by page type.Wire the helper into the top of
get_any_page()to short-circuit those pages before the retry loop runs. On a hit, drop the caller'sMF_COUNT_INCREASEDreference (if any) and return-ENOTRECOVERABLEstraight away. Pages outside the helper's positive list still take the existing retry path and return-EIO, leaving operator-visible behaviour for those cases unchanged.Extend the unhandlable-page
pr_err()to fire for either errno and update theget_hwpoison_page()kerneldoc to document the new return.
Quoted 7 lines
memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISONvia its existing "else if (res < 0)" branch, so this patch on its own only changes the errno thatsoft_offline_page()can propagate to its callers. A follow-up wires-ENOTRECOVERABLEthroughmemory_failure()and reportsMF_MSG_KERNELfor the unrecoverable cases, which is what thepanic_on_unrecoverable_memory_failuresysctl observes.
Thanks for your patch.
Quoted 25 lines
Suggested-by David Hildenbrand <[email protected]>Suggested-by Lance Yang <[email protected]>Signed-off-by Breno Leitao <[email protected]>--- mm/memory-failure.c | 42 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-)mm/memory-failure.c +9
-0@@ -1325,6 +1325,28 @@ static inline bool HWPoisonHandlable(struct page *page, unsigned long flags)return PageLRU (page) ||is_free_buddy_page (page); } +/* +* Positive identification of pages the hwpoison handler cannot recover. +* These page types are owned by kernel internals (no userspace mapping +* to unmap, no file mapping to invalidate, no migration target), so the +* shake_page() / retry loop in get_any_page() can never turn them into +* something HWPoisonHandlable() will accept. Short-circuit them to +* -ENOTRECOVERABLE so callers can panic on operator request instead of +* spinning through retries that exit as a transient-looking -EIO. +*
mm/memory-failure.c static inline bool HWPoisonHandlable(struct page *page, unsigned long flags) · line 1346
Quoted 10 lines
mm/memory-failure.c +10
-0+* The MF_SOFT_OFFLINE / page_has_movable_ops() opt-out mirrors +* HWPoisonHandlable(): soft-offline is allowed to migrate movable_ops +* pages even though they are not on the LRU. +*/ +static inline bool HWPoisonKernelOwned (struct page *page,unsigned long flags) +{ +if ((flags &MF_SOFT_OFFLINE ) &&page_has_movable_ops (page)) +return false; + + return PageReserved(page) || PageSlab(page) ||
Once shake_page finds a lightweight range-based way to shrink slab, slab pages could be freed into buddy and above PageSlab test should be removed then. Maybe add a TODO or XXX here?
mm/memory-failure.c static inline bool HWPoisonHandlable(struct page *page, unsigned long flags) · line 1347
Quoted 1 line
mm/memory-failure.c +1
-0+PageTable (page) ||PageLargeKmalloc (page);
I'm not sure but is it safe or a common way to test PageReserved, PageSlab, PageTable and PageLargeKmalloc without extra page refcnt?
Apart from the above nits, this patch looks good to me.
Thanks.
.
David Hildenbrand (Arm)
maintainerRe: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 90 lines · Miaohe Lin
Quoted 40 lines · Breno Leitao
get_any_page()collapses everyHWPoisonHandlable()rejection into a single-EIOvia the__get_hwpoison_page()->-EBUSY->shake_page()-> retry path. That is correct for the transient case (a userspace folio briefly off LRU during migration or compaction, which a later shake can drag back), but wrong for stable kernel-owned pages: slab, page-table, large-kmalloc andPG_reservedpages will never becomeHWPoisonHandlable(), so the retry loop is wasted work and the final-EIOloses the "this is structurally unrecoverable" information.memory_failure()then maps-EIOintoMF_MSG_GET_HWPOISON, which the panic-on-unrecoverable sysctl deliberately does not act on.Introduce
HWPoisonKernelOwned(), a small predicate that positively identifies pages the hwpoison handler cannot recover from:HWPoisonKernelOwned(p, flags) := !(MF_SOFT_OFFLINE && page_has_movable_ops(p)) && (PageReserved(p) || PageSlab(p) || PageTable(p) || PageLargeKmalloc(p))The
MF_SOFT_OFFLINE/page_has_movable_ops()opt-out mirrors the same exception inHWPoisonHandlable(): soft-offline is allowed to migratemovable_opspages even though they are not on the LRU, and we must not pre-empt that with an unrecoverable verdict.The list is intentionally not exhaustive. vmalloc and kernel-stack pages, for example, do not carry a
page_typebit and would need a different oracle; they keep going through the existing retry path unchanged. This is the smallest set we can identify with certainty by page type.Wire the helper into the top of
get_any_page()to short-circuit those pages before the retry loop runs. On a hit, drop the caller'sMF_COUNT_INCREASEDreference (if any) and return-ENOTRECOVERABLEstraight away. Pages outside the helper's positive list still take the existing retry path and return-EIO, leaving operator-visible behaviour for those cases unchanged.Extend the unhandlable-page
pr_err()to fire for either errno and update theget_hwpoison_page()kerneldoc to document the new return.Quoted 7 lines
memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISONvia its existing "else if (res < 0)" branch, so this patch on its own only changes the errno thatsoft_offline_page()can propagate to its callers. A follow-up wires-ENOTRECOVERABLEthroughmemory_failure()and reportsMF_MSG_KERNELfor the unrecoverable cases, which is what thepanic_on_unrecoverable_memory_failuresysctl observes.Thanks for your patch.
Quoted 25 lines
Suggested-by David Hildenbrand <[email protected]>Suggested-by Lance Yang <[email protected]>Signed-off-by Breno Leitao <[email protected]>--- mm/memory-failure.c | 42 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-)mm/memory-failure.c +9
-0@@ -1325,6 +1325,28 @@ static inline bool HWPoisonHandlable(struct page *page, unsigned long flags)return PageLRU (page) ||is_free_buddy_page (page); } +/* +* Positive identification of pages the hwpoison handler cannot recover. +* These page types are owned by kernel internals (no userspace mapping +* to unmap, no file mapping to invalidate, no migration target), so the +* shake_page() / retry loop in get_any_page() can never turn them into +* something HWPoisonHandlable() will accept. Short-circuit them to +* -ENOTRECOVERABLE so callers can panic on operator request instead of +* spinning through retries that exit as a transient-looking -EIO. +* mm/memory-failure.c static inline bool HWPoisonHandlable(struct page *page, unsigned long flags) · line 1346
Quoted 10 lines
mm/memory-failure.c +10
-0+* The MF_SOFT_OFFLINE / page_has_movable_ops() opt-out mirrors +* HWPoisonHandlable(): soft-offline is allowed to migrate movable_ops +* pages even though they are not on the LRU. +*/ +static inline bool HWPoisonKernelOwned (struct page *page,unsigned long flags) +{ +if ((flags &MF_SOFT_OFFLINE ) &&page_has_movable_ops (page)) +return false; + + return PageReserved(page) || PageSlab(page) ||Once
shake_pagefinds a lightweight range-based way to shrink slab, slab pages could be freed into buddy and above PageSlab test should be removed then. Maybe add a TODO or XXX here?
Quoted 4 lines
mm/memory-failure.c static inline bool HWPoisonHandlable(struct page *page, unsigned long flags) · line 1347
Quoted 1 line
mm/memory-failure.c +1
-0+PageTable (page) ||PageLargeKmalloc (page);I'm not sure but is it safe or a common way to test PageReserved, PageSlab, PageTable and PageLargeKmalloc without extra page refcnt?
Checking typed pages in a racy fashion is fine (PageSlab, PageTable, PageLargeKmalloc).
Checking PageReserved in a racy fashion is fine as well. TESTPAGEFLAG() will allow checking it on compound pages.
For PageLargeKmalloc, we would want to check the head page, though. The page type is only stored for the head page.
So maybe we want to lookup the compound head (if any) and perform the type checks against that?
Signature · 3 lines
Cheers, David
Re: [PATCH v8 3/6] mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages
In reply to Breno LeitaoQuoted 51 lines · Breno Leitao
The previous patch teaches
get_any_page()to return-ENOTRECOVERABLEfor stable unhandlable kernel pages (PG_reserved, slab, page tables, large-kmalloc).memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISON, so callers that want to react to the unrecoverable cases (a panic option, smarter logging) cannot tell them apart from transient page-allocator races.Turn the post-call branch into a switch over the
get_hwpoison_page()return code: map-ENOTRECOVERABLEtoMF_MSG_KERNELand any other negative return toMF_MSG_GET_HWPOISON. case 0 keeps the existing free-buddy / kernel-high-order handling and case 1 falls through to the rest ofmemory_failure()unchanged.The
MF_MSG_KERNELlabel and tracepoint string are kept as "reserved kernel page" to avoid breaking userspace tools that match on those literals; the enum value still adequately tags the failure even though it now also covers slab, page tables and large-kmalloc pages.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>--- mm/memory-failure.c | 17 +++++++++++++++-- 1 file changed, 15 insertions(+), 2 deletions(-)mm/memory-failure.c +8
-2@@ -2426,7 +2426,8 @@ int memory_failure(unsigned long pfn, int flags)* that may make page_ref_freeze()/page_ref_unfreeze() mismatch. */ res =get_hwpoison_page (p, flags); -if (!res) { +switch (res) { +case 0 :if (is_free_buddy_page (p)) {if (take_page_off_buddy (p)) {page_ref_inc (p);@@ -2445,7 +2446,19 @@ int memory_failure(unsigned long pfn, int flags) res =action_result (pfn,MF_MSG_KERNEL_HIGH_ORDER ,MF_IGNORED ); } goto unlock_mutex; - }else if (res <0 ) { +case 1 : +/* Got a refcount on a handlable page. */ +break ; +case -ENOTRECOVERABLE : +/* +* Stable unhandlable kernel-owned page (PG_reserved,
mm/memory-failure.c int memory_failure(unsigned long pfn, int flags) · line 2464
Quoted 10 lines
mm/memory-failure.c +7
-0+* slab, page tables, large-kmalloc). +* No recovery possible. +*/ + res =action_result (pfn,MF_MSG_KERNEL ,MF_IGNORED ); + goto unlock_mutex; +default : +/* Transient lifecycle race with the page allocator. */ res =action_result (pfn,MF_MSG_GET_HWPOISON ,MF_IGNORED ); goto unlock_mutex; }
Signature · 3 lines
Cheers, David
Miaohe Lin
maintainerRe: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
In reply to David Hildenbrand (Arm)Quoted 91 lines · David Hildenbrand (Arm)
Quoted 90 lines · Miaohe Lin
Quoted 40 lines · Breno Leitao
get_any_page()collapses everyHWPoisonHandlable()rejection into a single-EIOvia the__get_hwpoison_page()->-EBUSY->shake_page()-> retry path. That is correct for the transient case (a userspace folio briefly off LRU during migration or compaction, which a later shake can drag back), but wrong for stable kernel-owned pages: slab, page-table, large-kmalloc andPG_reservedpages will never becomeHWPoisonHandlable(), so the retry loop is wasted work and the final-EIOloses the "this is structurally unrecoverable" information.memory_failure()then maps-EIOintoMF_MSG_GET_HWPOISON, which the panic-on-unrecoverable sysctl deliberately does not act on.Introduce
HWPoisonKernelOwned(), a small predicate that positively identifies pages the hwpoison handler cannot recover from:HWPoisonKernelOwned(p, flags) := !(MF_SOFT_OFFLINE && page_has_movable_ops(p)) && (PageReserved(p) || PageSlab(p) || PageTable(p) || PageLargeKmalloc(p))The
MF_SOFT_OFFLINE/page_has_movable_ops()opt-out mirrors the same exception inHWPoisonHandlable(): soft-offline is allowed to migratemovable_opspages even though they are not on the LRU, and we must not pre-empt that with an unrecoverable verdict.The list is intentionally not exhaustive. vmalloc and kernel-stack pages, for example, do not carry a
page_typebit and would need a different oracle; they keep going through the existing retry path unchanged. This is the smallest set we can identify with certainty by page type.Wire the helper into the top of
get_any_page()to short-circuit those pages before the retry loop runs. On a hit, drop the caller'sMF_COUNT_INCREASEDreference (if any) and return-ENOTRECOVERABLEstraight away. Pages outside the helper's positive list still take the existing retry path and return-EIO, leaving operator-visible behaviour for those cases unchanged.Extend the unhandlable-page
pr_err()to fire for either errno and update theget_hwpoison_page()kerneldoc to document the new return.Quoted 7 lines
memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISONvia its existing "else if (res < 0)" branch, so this patch on its own only changes the errno thatsoft_offline_page()can propagate to its callers. A follow-up wires-ENOTRECOVERABLEthroughmemory_failure()and reportsMF_MSG_KERNELfor the unrecoverable cases, which is what thepanic_on_unrecoverable_memory_failuresysctl observes.Thanks for your patch.
Quoted 25 lines
Suggested-by David Hildenbrand <[email protected]>Suggested-by Lance Yang <[email protected]>Signed-off-by Breno Leitao <[email protected]>--- mm/memory-failure.c | 42 ++++++++++++++++++++++++++++++++++++++++-- 1 file changed, 40 insertions(+), 2 deletions(-)mm/memory-failure.c +9
-0@@ -1325,6 +1325,28 @@ static inline bool HWPoisonHandlable(struct page *page, unsigned long flags)return PageLRU (page) ||is_free_buddy_page (page); } +/* +* Positive identification of pages the hwpoison handler cannot recover. +* These page types are owned by kernel internals (no userspace mapping +* to unmap, no file mapping to invalidate, no migration target), so the +* shake_page() / retry loop in get_any_page() can never turn them into +* something HWPoisonHandlable() will accept. Short-circuit them to +* -ENOTRECOVERABLE so callers can panic on operator request instead of +* spinning through retries that exit as a transient-looking -EIO. +* mm/memory-failure.c static inline bool HWPoisonHandlable(struct page *page, unsigned long flags) · line 1346
Quoted 10 lines
mm/memory-failure.c +10
-0+* The MF_SOFT_OFFLINE / page_has_movable_ops() opt-out mirrors +* HWPoisonHandlable(): soft-offline is allowed to migrate movable_ops +* pages even though they are not on the LRU. +*/ +static inline bool HWPoisonKernelOwned (struct page *page,unsigned long flags) +{ +if ((flags &MF_SOFT_OFFLINE ) &&page_has_movable_ops (page)) +return false; + + return PageReserved(page) || PageSlab(page) ||Once
shake_pagefinds a lightweight range-based way to shrink slab, slab pages could be freed into buddy and above PageSlab test should be removed then. Maybe add a TODO or XXX here?
Quoted 7 lines
Quoted 4 lines
mm/memory-failure.c static inline bool HWPoisonHandlable(struct page *page, unsigned long flags) · line 1347
Quoted 1 line
mm/memory-failure.c +1
-0+PageTable (page) ||PageLargeKmalloc (page);I'm not sure but is it safe or a common way to test PageReserved, PageSlab, PageTable and PageLargeKmalloc without extra page refcnt?
Checking typed pages in a racy fashion is fine (PageSlab, PageTable, PageLargeKmalloc).
Got it. Thanks.
Quoted 2 lines
Checking PageReserved in a racy fashion is fine as well.
TESTPAGEFLAG()will allow checking it on compound pages.
It seems PageReserved is not intended to be set on compound pages. I see there are PF_NO_COMPOUND in its definition: PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND).
Quoted 3 lines
For PageLargeKmalloc, we would want to check the head page, though. The page type is only stored for the head page.
Maybe we should check the head page for PageSlab and PageTable too? alloc_slab_page only set PageSlab on the head page and __pagetable_ctor uses __folio_set_pgtable to set PageTable on folio.
Quoted 3 lines
So maybe we want to lookup the compound head (if any) and perform the type checks against that?
Maybe we should or we might miss some pages that could have been handled. And if compound head is required, should we hold an extra page refcnt to guard against possible folio split race?
Thanks.
.
Re: [PATCH v8 3/6] mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages
In reply to Breno LeitaoQuoted 13 lines · Breno Leitao
The previous patch teaches
get_any_page()to return-ENOTRECOVERABLEfor stable unhandlable kernel pages (PG_reserved, slab, page tables, large-kmalloc).memory_failure()still folds every negative return intoMF_MSG_GET_HWPOISON, so callers that want to react to the unrecoverable cases (a panic option, smarter logging) cannot tell them apart from transient page-allocator races.Turn the post-call branch into a switch over the
get_hwpoison_page()return code: map-ENOTRECOVERABLEtoMF_MSG_KERNELand any other negative return toMF_MSG_GET_HWPOISON. case 0 keeps the existing free-buddy / kernel-high-order handling and case 1 falls through to the rest ofmemory_failure()unchanged.
Quoted 8 lines
The
MF_MSG_KERNELlabel and tracepoint string are kept as "reserved kernel page" to avoid breaking userspace tools that match on those literals; the enum value still adequately tags the failure even though it now also covers slab, page tables and large-kmalloc pages.Suggested-by David Hildenbrand <[email protected]>Signed-off-by Breno Leitao <[email protected]>
Thanks.
.
Miaohe Lin
maintainerQuoted 58 lines · Breno Leitao
Add a sysctl
panic_on_unrecoverable_memory_failure(disabled by default) that triggers a kernel panic whenmemory_failure()encounters pages that cannot be recovered. This provides a clean crash with useful debug information rather than allowing silent data corruption or a delayed crash at an unrelated code path.Panic eligibility is intentionally narrow: only
MF_MSG_KERNELwith result ==MF_IGNOREDpanics. After the previous patch,MF_MSG_KERNELcoversPG_reservedpages and the kernel-owned pages promoted fromget_hwpoison_page()via-ENOTRECOVERABLE(slab, page tables, large-kmalloc).All other action types are excluded:
-
MF_MSG_GET_HWPOISONandMF_MSG_KERNEL_HIGH_ORDERcan be reached by transient refcount races with the page allocator (an in-flight buddy allocation has refcount 0 and is no longer on the buddy free list, briefly), and panicking on them would risk killing the box for what is actually a recoverable userspace page.-
MF_MSG_UNKNOWNmeansidentify_page_state()could not classify the page; that is precisely the wrong basis for a panic decision.Signed-off-by Breno Leitao <[email protected]>--- mm/memory-failure.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)mm/memory-failure.c +11
-0@@ -74,6 +74,8 @@ static int sysctl_memory_failure_recovery __read_mostly = 1;static int sysctl_enable_soft_offline__read_mostly =1 ; +static int sysctl_panic_on_unrecoverable_mf__read_mostly ; +atomic_long_t num_poisoned_pages__read_mostly =ATOMIC_LONG_INIT (0 );static bool hw_memory_failure__read_mostly = false;@@ -155,6 +157,15 @@ static const struct ctl_table memory_failure_table[] = { .proc_handler = proc_dointvec_minmax, .extra1 = SYSCTL_ZERO, .extra2 = SYSCTL_ONE, + }, + { + .procname = "panic_on_unrecoverable_memory_failure", + .data = &sysctl_panic_on_unrecoverable_mf, + .maxlen = sizeof(sysctl_panic_on_unrecoverable_mf), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = SYSCTL_ZERO, + .extra2 = SYSCTL_ONE, } };
mm/memory-failure.c static void update_per_node_mf_stats(unsigned long pfn, · line 1275
Quoted 10 lines
mm/memory-failure.c +7
-0@@ -1255,6 +1266,15 @@ static void update_per_node_mf_stats(unsigned long pfn, ++mf_stats->total; } +static bool panic_on_unrecoverable_mf (enum mf_action_page_type type, +enum mf_result result) +{ +if (!sysctl_panic_on_unrecoverable_mf || result !=MF_IGNORED ) +return false; + +return type ==MF_MSG_KERNEL ;
Would it be more straightforward to write as something like:
if (!sysctl_panic_on_unrecoverable_mf)
return false;
return (type == MF_MSG_KERNEL && result == MF_IGNORED);
Thanks.
.
David Hildenbrand (Arm)
maintainerRe: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
In reply to Miaohe LinQuoted 33 lines · Miaohe Lin
Quoted 9 lines · David Hildenbrand (Arm)
Quoted 8 lines · Miaohe Lin
Thanks for your patch.
Once
shake_pagefinds a lightweight range-based way to shrink slab, slab pages could be freed into buddy and above PageSlab test should be removed then. Maybe add a TODO or XXX here?Quoted 5 lines
Quoted 2 lines
I'm not sure but is it safe or a common way to test PageReserved, PageSlab, PageTable and PageLargeKmalloc without extra page refcnt?
Checking typed pages in a racy fashion is fine (PageSlab, PageTable, PageLargeKmalloc).
Got it. Thanks.
Quoted 2 lines
Checking PageReserved in a racy fashion is fine as well.
TESTPAGEFLAG()will allow checking it on compound pages.It seems PageReserved is not intended to be set on compound pages. I see there are
PF_NO_COMPOUNDin its definition:PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND).Quoted 3 lines
For PageLargeKmalloc, we would want to check the head page, though. The page type is only stored for the head page.
Maybe we should check the head page for PageSlab and PageTable too?
alloc_slab_pageonly set PageSlab on the head page and__pagetable_ctoruses__folio_set_pgtableto set PageTable on folio.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
So maybe we want to lookup the compound head (if any) and perform the type checks against that?
Maybe we should or we might miss some pages that could have been handled. And if compound head is required, should we hold an extra page refcnt to guard against possible folio split race?
Races are fine. We might miss some pages, but that can happen on races either way.
I'd just do something like
if (PageReserved(page))
return true;
head = compound_head(page);
return PageSlab(head) || ...;
Signature · 3 lines
Cheers, David
Miaohe Lin
maintainerRe: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 44 lines · David Hildenbrand (Arm)
Quoted 33 lines · Miaohe Lin
Quoted 9 lines · David Hildenbrand (Arm)
Quoted 8 lines · Miaohe Lin
Thanks for your patch.
Once
shake_pagefinds a lightweight range-based way to shrink slab, slab pages could be freed into buddy and above PageSlab test should be removed then. Maybe add a TODO or XXX here?Quoted 5 lines
Quoted 2 lines
I'm not sure but is it safe or a common way to test PageReserved, PageSlab, PageTable and PageLargeKmalloc without extra page refcnt?
Checking typed pages in a racy fashion is fine (PageSlab, PageTable, PageLargeKmalloc).
Got it. Thanks.
Quoted 2 lines
Checking PageReserved in a racy fashion is fine as well.
TESTPAGEFLAG()will allow checking it on compound pages.It seems PageReserved is not intended to be set on compound pages. I see there are
PF_NO_COMPOUNDin its definition:PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND).Quoted 3 lines
For PageLargeKmalloc, we would want to check the head page, though. The page type is only stored for the head page.
Maybe we should check the head page for PageSlab and PageTable too?
alloc_slab_pageonly set PageSlab on the head page and__pagetable_ctoruses__folio_set_pgtableto set PageTable on folio.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
So maybe we want to lookup the compound head (if any) and perform the type checks against that?
Maybe we should or we might miss some pages that could have been handled. And if compound head is required, should we hold an extra page refcnt to guard against possible folio split race?
Races are fine. We might miss some pages, but that can happen on races either way.
Quoted 6 lines
I'd just do something like
if (PageReserved(page))
return true;head =
compound_head(page);
If @head is split just after compound_head. And then @head is freed into buddy and re-allocated as slab page while @page is still in the buddy. We would panic on this scene as @head is PageSlab. But we were supposed to successfully handle @page. Or am I miss something?
Thanks.
.
Quoted 3 lines
return PageSlab(head) || ...;
Re: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 47 lines · Miaohe Lin
Quoted 46 lines · David Hildenbrand (Arm)
Quoted 33 lines · Miaohe Lin
Quoted 9 lines · David Hildenbrand (Arm)
Quoted 8 lines · Miaohe Lin
Thanks for your patch.
Once
shake_pagefinds a lightweight range-based way to shrink slab, slab pages could be freed into buddy and above PageSlab test should be removed then. Maybe add a TODO or XXX here?Quoted 5 lines
Quoted 2 lines
I'm not sure but is it safe or a common way to test PageReserved, PageSlab, PageTable and PageLargeKmalloc without extra page refcnt?
Checking typed pages in a racy fashion is fine (PageSlab, PageTable, PageLargeKmalloc).
Got it. Thanks.
Quoted 2 lines
Checking PageReserved in a racy fashion is fine as well.
TESTPAGEFLAG()will allow checking it on compound pages.It seems PageReserved is not intended to be set on compound pages. I see there are
PF_NO_COMPOUNDin its definition:PAGEFLAG(Reserved, reserved, PF_NO_COMPOUND).Quoted 3 lines
For PageLargeKmalloc, we would want to check the head page, though. The page type is only stored for the head page.
Maybe we should check the head page for PageSlab and PageTable too?
alloc_slab_pageonly set PageSlab on the head page and__pagetable_ctoruses__folio_set_pgtableto set PageTable on folio.Quoted 1 line
Quoted 6 lines
Quoted 2 lines
So maybe we want to lookup the compound head (if any) and perform the type checks against that?
Maybe we should or we might miss some pages that could have been handled. And if compound head is required, should we hold an extra page refcnt to guard against possible folio split race?
Races are fine. We might miss some pages, but that can happen on races either way.
I'd just do something like
Quoted 8 lines
Quoted 4 lines
if (PageReserved(page))
return true;head =
compound_head(page);If @head is split just after
compound_head. And then @head is freed into buddy and re-allocated as slab page while @page is still in the buddy. We would panic on this scene as @head is PageSlab. But we were supposed to successfully handle @page. Or am I miss something?
You're right that it is racy, but I think it is an acceptable race here.
For it to happen, the poisoned @page has to be a tail of a live compound page at the time of the fault, and then -- in the few instructions between compound_head() and the PageSlab(head) test -- that compound page has to be split, the old head freed to buddy, and that head re-allocated as a slab page, all while @page lands back in the buddy. It cannot happen without concurrent split/free/alloc activity in that exact window.
It is also worth noting the page in question genuinely took a unrecoverable ECC error, and panic_on_unrecoverable_memory_failure is opt-in -- an operator who enables it has explicitly chosen to crash rather than risk running on corrupted memory. Mis-attributing one such rare, genuinely-poisoned page as unrecoverable is within that contract.
Thanks for the review and discussions,
--breno
Quoted 72 lines · Miaohe Lin
Quoted 58 lines · Breno Leitao
Add a sysctl
panic_on_unrecoverable_memory_failure(disabled by default) that triggers a kernel panic whenmemory_failure()encounters pages that cannot be recovered. This provides a clean crash with useful debug information rather than allowing silent data corruption or a delayed crash at an unrelated code path.Panic eligibility is intentionally narrow: only
MF_MSG_KERNELwith result ==MF_IGNOREDpanics. After the previous patch,MF_MSG_KERNELcoversPG_reservedpages and the kernel-owned pages promoted fromget_hwpoison_page()via-ENOTRECOVERABLE(slab, page tables, large-kmalloc).All other action types are excluded:
-
MF_MSG_GET_HWPOISONandMF_MSG_KERNEL_HIGH_ORDERcan be reached by transient refcount races with the page allocator (an in-flight buddy allocation has refcount 0 and is no longer on the buddy free list, briefly), and panicking on them would risk killing the box for what is actually a recoverable userspace page.-
MF_MSG_UNKNOWNmeansidentify_page_state()could not classify the page; that is precisely the wrong basis for a panic decision.Signed-off-by Breno Leitao <[email protected]>--- mm/memory-failure.c | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)mm/memory-failure.c +11
-0@@ -74,6 +74,8 @@ static int sysctl_memory_failure_recovery __read_mostly = 1;static int sysctl_enable_soft_offline__read_mostly =1 ; +static int sysctl_panic_on_unrecoverable_mf__read_mostly ; +atomic_long_t num_poisoned_pages__read_mostly =ATOMIC_LONG_INIT (0 );static bool hw_memory_failure__read_mostly = false;@@ -155,6 +157,15 @@ static const struct ctl_table memory_failure_table[] = { .proc_handler = proc_dointvec_minmax, .extra1 = SYSCTL_ZERO, .extra2 = SYSCTL_ONE, + }, + { + .procname = "panic_on_unrecoverable_memory_failure", + .data = &sysctl_panic_on_unrecoverable_mf, + .maxlen = sizeof(sysctl_panic_on_unrecoverable_mf), + .mode = 0644, + .proc_handler = proc_dointvec_minmax, + .extra1 = SYSCTL_ZERO, + .extra2 = SYSCTL_ONE, } };mm/memory-failure.c static void update_per_node_mf_stats(unsigned long pfn, · line 1275
Quoted 10 lines
mm/memory-failure.c +7
-0@@ -1255,6 +1266,15 @@ static void update_per_node_mf_stats(unsigned long pfn, ++mf_stats->total; } +static bool panic_on_unrecoverable_mf (enum mf_action_page_type type, +enum mf_result result) +{ +if (!sysctl_panic_on_unrecoverable_mf || result !=MF_IGNORED ) +return false; + +return type ==MF_MSG_KERNEL ;Would it be more straightforward to write as something like:
Quoted 4 lines
if (!sysctl_panic_on_unrecoverable_mf)
return false;return (type ==
MF_MSG_KERNEL&& result ==MF_IGNORED);
Sure, that reads better. I'll fold the MF_IGNORED check into the return for the next revision.
static bool panic_on_unrecoverable_mf(enum mf_action_page_type type,
enum mf_result result)
{
if (!sysctl_panic_on_unrecoverable_mf)
return false;
return type == MF_MSG_KERNEL && result == MF_IGNORED;
}David Hildenbrand (Arm)
maintainerRe: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
In reply to Breno LeitaoQuoted 11 lines · Breno Leitao
Quoted 10 lines · Miaohe Lin
Quoted 9 lines · David Hildenbrand (Arm)
Races are fine. We might miss some pages, but that can happen on races either way.
I'd just do something like
if (PageReserved(page))
return true;
Quoted 8 lines
Quoted 5 lines
Quoted 1 line
head =
compound_head(page);If @head is split just after
compound_head. And then @head is freed into buddy and re-allocated as slab page while @page is still in the buddy. We would panic on this scene as @head is PageSlab. But we were supposed to successfully handle @page. Or am I miss something?You're right that it is racy, but I think it is an acceptable race here.
I mean, any such races can currently already happen one way or the other?
Really, the only way to not get races is to tryget the (compound)page, revalidate that the page is still part of the compound page.
I'm not sure if that's really a good idea.
But my memory is a bit vague in which scenarios we already hold a page reference here to prevent any concurrent freeing?
Signature · 3 lines
Cheers, David
Re: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 23 lines · David Hildenbrand (Arm)
Quoted 11 lines · Breno Leitao
Quoted 10 lines · Miaohe Lin
Quoted 9 lines · David Hildenbrand (Arm)
Races are fine. We might miss some pages, but that can happen on races either way.
I'd just do something like
if (PageReserved(page))
return true;Quoted 8 lines
Quoted 5 lines
Quoted 1 line
head =
compound_head(page);If @head is split just after
compound_head. And then @head is freed into buddy and re-allocated as slab page while @page is still in the buddy. We would panic on this scene as @head is PageSlab. But we were supposed to successfully handle @page. Or am I miss something?You're right that it is racy, but I think it is an acceptable race here.
I mean, any such races can currently already happen one way or the other?
Quoted 7 lines
Really, the only way to not get races is to tryget the (compound)page, revalidate that the page is still part of the compound page.
I'm not sure if that's really a good idea.
But my memory is a bit vague in which scenarios we already hold a page reference here to prevent any concurrent freeing?
No, we don't hold one here in the case that matters.
HWPoisonKernelOwned() runs at the very top of get_any_page(), beforetry_again: and before __get_hwpoison_page(). The first refcount taken in the whole path is the folio_try_get() inside __get_hwpoison_page(), which runs *after* the short-circuit.
So get_any_page() itself never holds a reference at the check -- the only way one exists is if the caller passed MF_COUNT_INCREASED (count_increased == true).
So on the MCE/GHES path -- the one this panic option exists for -- no reference is held when HWPoisonKernelOwned() does its compound_head() + PageSlab()/PageTable()/PageLargeKmalloc() checks.
Given that, I'd rather keep it racy and take no refcount than add a tryget + revalidate purely for this check. As I've said earleir, an operator who enabled it has chosen to crash rather than run on corrupted memory; mis-attributing one such rare, genuinely-poisoned page is within that contract.
Miaohe Lin
maintainerRe: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 43 lines · Breno Leitao
Quoted 23 lines · David Hildenbrand (Arm)
Quoted 11 lines · Breno Leitao
Quoted 10 lines · Miaohe Lin
Quoted 9 lines · David Hildenbrand (Arm)
Races are fine. We might miss some pages, but that can happen on races either way.
I'd just do something like
if (PageReserved(page))
return true;Quoted 8 lines
Quoted 5 lines
Quoted 1 line
head =
compound_head(page);If @head is split just after
compound_head. And then @head is freed into buddy and re-allocated as slab page while @page is still in the buddy. We would panic on this scene as @head is PageSlab. But we were supposed to successfully handle @page. Or am I miss something?You're right that it is racy, but I think it is an acceptable race here.
I mean, any such races can currently already happen one way or the other?
Quoted 7 lines
Really, the only way to not get races is to tryget the (compound)page, revalidate that the page is still part of the compound page.
I'm not sure if that's really a good idea.
But my memory is a bit vague in which scenarios we already hold a page reference here to prevent any concurrent freeing?
No, we don't hold one here in the case that matters.
HWPoisonKernelOwned()runs at the very top ofget_any_page(), beforetry_again: and before__get_hwpoison_page(). The first refcount taken in the whole path is thefolio_try_get()inside__get_hwpoison_page(), which runs *after* the short-circuit.So
get_any_page()itself never holds a reference at the check -- the only way one exists is if the caller passedMF_COUNT_INCREASED(count_increased== true).
Quoted 6 lines
So on the MCE/GHES path -- the one this panic option exists for -- no reference is held when
HWPoisonKernelOwned()does itscompound_head()+ PageSlab()/PageTable()/PageLargeKmalloc() checks.Given that, I'd rather keep it racy and take no refcount than add a tryget + revalidate purely for this check. As I've said earleir, an operator
Would it be acceptable to add a simple recheck? Something like below:
retry:
head = compound_head(page);
PageSlab()/PageTable()/PageLargeKmalloc() checks
if (head != compound_head(page))
goto retry
Thanks both.
.
Quoted 4 lines
who enabled it has chosen to crash rather than run on corrupted memory; mis-attributing one such rare, genuinely-poisoned page is within that contract. .
David Hildenbrand (Arm)
maintainerRe: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 31 lines · Miaohe Lin
Quoted 23 lines · Breno Leitao
Quoted 10 lines · David Hildenbrand (Arm)
I mean, any such races can currently already happen one way or the other?
Really, the only way to not get races is to tryget the (compound)page, revalidate that the page is still part of the compound page.
I'm not sure if that's really a good idea.
But my memory is a bit vague in which scenarios we already hold a page reference here to prevent any concurrent freeing?
No, we don't hold one here in the case that matters.
HWPoisonKernelOwned()runs at the very top ofget_any_page(), beforetry_again: and before__get_hwpoison_page(). The first refcount taken in the whole path is thefolio_try_get()inside__get_hwpoison_page(), which runs *after* the short-circuit.So
get_any_page()itself never holds a reference at the check -- the only way one exists is if the caller passedMF_COUNT_INCREASED(count_increased== true).Quoted 6 lines
So on the MCE/GHES path -- the one this panic option exists for -- no reference is held when
HWPoisonKernelOwned()does itscompound_head()+ PageSlab()/PageTable()/PageLargeKmalloc() checks.Given that, I'd rather keep it racy and take no refcount than add a tryget + revalidate purely for this check. As I've said earleir, an operator
Quoted 7 lines
Would it be acceptable to add a simple recheck? Something like below:
retry:
head =compound_head(page);
PageSlab()/PageTable()/PageLargeKmalloc() checks
if (head !=compound_head(page))
goto retry
Sure. I guess it could still be racy in some weird scenarios where we free+allocate+free in-between.
Signature · 3 lines
Cheers, David
Re: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 34 lines · David Hildenbrand (Arm)
Quoted 33 lines · Miaohe Lin
Quoted 23 lines · Breno Leitao
Quoted 10 lines · David Hildenbrand (Arm)
I mean, any such races can currently already happen one way or the other?
Really, the only way to not get races is to tryget the (compound)page, revalidate that the page is still part of the compound page.
I'm not sure if that's really a good idea.
But my memory is a bit vague in which scenarios we already hold a page reference here to prevent any concurrent freeing?
No, we don't hold one here in the case that matters.
HWPoisonKernelOwned()runs at the very top ofget_any_page(), beforetry_again: and before__get_hwpoison_page(). The first refcount taken in the whole path is thefolio_try_get()inside__get_hwpoison_page(), which runs *after* the short-circuit.So
get_any_page()itself never holds a reference at the check -- the only way one exists is if the caller passedMF_COUNT_INCREASED(count_increased== true).Quoted 6 lines
So on the MCE/GHES path -- the one this panic option exists for -- no reference is held when
HWPoisonKernelOwned()does itscompound_head()+ PageSlab()/PageTable()/PageLargeKmalloc() checks.Given that, I'd rather keep it racy and take no refcount than add a tryget + revalidate purely for this check. As I've said earleir, an operator
Would it be acceptable to add a simple recheck? Something like below:
Quoted 8 lines
Quoted 5 lines
retry:
head =compound_head(page);
PageSlab()/PageTable()/PageLargeKmalloc() checks
if (head !=compound_head(page))
goto retrySure. I guess it could still be racy in some weird scenarios where we free+allocate+free in-between.
+1, sounds reasonable to me. Still racy, but acceptable here I guess :D
Re: [PATCH v8 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE
Quoted 43 lines · Lance Yang
Quoted 40 lines · David Hildenbrand (Arm)
Quoted 38 lines · Miaohe Lin
Quoted 23 lines · Breno Leitao
Quoted 10 lines · David Hildenbrand (Arm)
I mean, any such races can currently already happen one way or the other?
Really, the only way to not get races is to tryget the (compound)page, revalidate that the page is still part of the compound page.
I'm not sure if that's really a good idea.
But my memory is a bit vague in which scenarios we already hold a page reference here to prevent any concurrent freeing?
No, we don't hold one here in the case that matters.
HWPoisonKernelOwned()runs at the very top ofget_any_page(), beforetry_again: and before__get_hwpoison_page(). The first refcount taken in the whole path is thefolio_try_get()inside__get_hwpoison_page(), which runs *after* the short-circuit.So
get_any_page()itself never holds a reference at the check -- the only way one exists is if the caller passedMF_COUNT_INCREASED(count_increased== true).Quoted 6 lines
So on the MCE/GHES path -- the one this panic option exists for -- no reference is held when
HWPoisonKernelOwned()does itscompound_head()+ PageSlab()/PageTable()/PageLargeKmalloc() checks.Given that, I'd rather keep it racy and take no refcount than add a tryget + revalidate purely for this check. As I've said earleir, an operator
Would it be acceptable to add a simple recheck? Something like below:
retry:
head =compound_head(page);
PageSlab()/PageTable()/PageLargeKmalloc() checks
if (head !=compound_head(page))
goto retry
Quoted 4 lines
Quoted 2 lines
Sure. I guess it could still be racy in some weird scenarios where we free+allocate+free in-between.
+1, sounds reasonable to me. Still racy, but acceptable here I guess :D
Ack. I will post v9 shortly with this plus a couple of selftest fixes Sashiko flagged.