Skip to contentSkip to conversation

mm/memory-failure: add panic option for unrecoverable pages

PATCH Superseded Breno Leitao 4 participants 20 messages May 13 – May 21

Subsystems (6)

v7 of 10 Superseded Latest is v10 →

lore
Share & tools
lore
#1 lore raw

PATCH v7 1/6 mm/memory-failure: drop dead error_states[] entry for reserved pages

In reply to Breno Leitao

The first entry of error_states[],

{ reserved, reserved, MF_MSG_KERNEL, me_kernel },

is unreachable. identify_page_state() has two callers, and neither one can dispatch a PG_reserved page to me_kernel():

* memory_failure() reaches identify_page_state() only after get_hwpoison_page() returned 1. get_any_page() reaches that return only via __get_hwpoison_page(), which gates the refcount on HWPoisonHandlable(). HWPoisonHandlable() rejects PG_reserved pages, so they fail with -EBUSY/-EIO long before identify_page_state() runs.

* try_memory_failure_hugetlb() reaches identify_page_state() on the MF_HUGETLB_IN_USED branch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head, MF_MSG_HUGE, me_huge_page }, so they dispatch to me_huge_page() before the (now-removed) reserved entry would have matched, regardless of whether PG_reserved happens to be set on the head page.

me_kernel() never executes and the entry exists only to be matched against by code that cannot see it.

Drop the entry, the me_kernel() helper, and the now-unused "reserved" macro. Leave the MF_MSG_KERNEL enum value in place: it remains part of the tracepoint and pr_err() string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.

No functional change.

---
 mm/memory-failure.c | 14 --------------
 1 file changed, 14 deletions(-)
mm/memory-failure.c +0 -14
index 866c4428ac7ef..49bcfbd04d213 100644
@@ -992,17 +992,6 @@ static bool has_extra_refcount(struct page_state *ps, struct page *p,
 	return false;
 }
 
-/*
- * Error hit kernel page.
- * Do nothing, try to be lucky and not touch this instead. For a few cases we
- * could be more sophisticated.
- */
-static int me_kernel(struct page_state *ps, struct page *p)
-{
-	unlock_page(p);
-	return MF_IGNORED;
-}
-
 /*
  * Page in unknown state. Do nothing.
  * This is a catch-all in case we fail to make sense of the page state.
@@ -1211,10 +1200,8 @@ static int me_huge_page(struct page_state *ps, struct page *p)
 #define mlock		(1UL << PG_mlocked)
 #define lru		(1UL << PG_lru)
 #define head		(1UL << PG_head)
-#define reserved	(1UL << PG_reserved)
 
 static struct page_state error_states[] = {
-	{ reserved,	reserved,	MF_MSG_KERNEL,	me_kernel },
 	/*
 	 * free pages are specially detected outside this table:
 	 * PG_buddy pages only make a small fraction of all free pages.
@@ -1246,7 +1233,6 @@ static struct page_state error_states[] = {
 #undef mlock
 #undef lru
 #undef head
-#undef reserved
 
 static void update_per_node_mf_stats(unsigned long pfn,
 				     enum mf_result result)
Signature · 1 line
2.53.0-Meta
#0 lore raw

v7 cover mm/memory-failure: add panic option for unrecoverable pages

A multi-bit ECC error on a kernel-owned page that the memory failure handler cannot recover is currently swallowed: PG_hwpoison is set, the event is logged, and the kernel keeps running. The corrupted memory remains accessible to the kernel and either drives silent data corruption or surfaces seconds-to-minutes later as an apparently unrelated crash. In a large fleet that delayed, unattributable crash turns into significant engineering effort to root-cause; in a kdump configuration, by the time the crash happens the original error context (faulting PFN, MCE/GHES record, page state) is long gone.

This series adds an opt-in sysctl, vm.panic_on_unrecoverable_memory_failure, that converts an unrecoverable kernel-page hwpoison event into an immediate panic with a clean dmesg/vmcore that still contains the original failure context. The default is disabled so existing workloads see no change.

---
Changes in v7:
- Move the PG_reserved / unhandlable-kernel-page classification into get_any_page() and surface it via -ENOTRECOVERABLE, per David Hildenbrand's and Lance Yang's review of v6. This drops the is_reserved snapshot in memory_failure() and the mf_get_page_status enum / out-parameter introduced in v6.
- Restructure the post-call branch in memory_failure() as a switch
over the get_hwpoison_page() return code (David).
- Drop the "reserved" qualifier from the MF_MSG_KERNEL label and the matching tracepoint string; the enum now covers both PG_reserved
pages and other unhandlable kernel pages.
- Squash the former patches 1/4 ("MF_MSG_KERNEL for reserved pages")
and 2/4 ("classify get_any_page() failures by reason") into a
single classification patch; the series is now 3 patches.
- Simplify panic_on_unrecoverable_mf() to a single return statement (David).
- Link to v6: https://patch.msgid.link/[email protected]

Changes in v6:
- Dropped the selftest given the value was not clear
- Get the status of the failure from get_any_page()
- Small nits from different people/AIs.
- Link to v5: https://patch.msgid.link/[email protected]

Changes in v5:
- Add vm.panic_on_unrecoverable_memory_failure sysctl to panic on unrecoverable kernel page hwpoison events (reserved pages, refcount-0 non-buddy pages, unknown state), with a recheck to avoid racing with concurrent buddy allocations. (Miaohe)
- Distinguish reserved pages as MF_MSG_KERNEL in memory_failure(), document the new sysctl in Documentation/admin-guide/sysctl/vm.rst, and add a selftest verifying SIGBUS recovery on userspace pages still works when the sysctl is enabled. (Miaohe)
- Added a selftest
- Link to v4:
https://patch.msgid.link/[email protected]

Changes in v4:
- Drop CONFIG_BOOTPARAM_MEMORY_FAILURE_PANIC kernel configuration option.
- Split the reserved page classification (MF_MSG_KERNEL) into its own patch, separate from the panic mechanism.
- Document why the buddy allocator TOCTOU race (between get_hwpoison_page() and is_free_buddy_page()) cannot cause false
positives: PG_hwpoison is set beforehand and check_new_page() in the page allocator rejects hwpoisoned pages.
- Document the narrow LRU isolation race window for MF_MSG_UNKNOWN and its mitigation via identify_page_state()'s two-pass design.
- Explicitly document why MF_MSG_GET_HWPOISON is excluded from the
panic conditions (shared path with transient races and non-reserved kernel memory).
- Link to v3: https://patch.msgid.link/[email protected]

Changes in v3:
- Rename is_unrecoverable_memory_failure() to panic_on_unrecoverable_mf() as suggested by maintainer.
- Add CONFIG_BOOTPARAM_MEMORY_FAILURE_PANIC kernel configuration option, similar to CONFIG_BOOTPARAM_HARDLOCKUP_PANIC.
- Add documentation for the sysctl and CONFIG option.
- Add code comments documenting the panic condition design rationale and how the retry mechanism mitigates false positives from buddy allocator races.
- Link to v2: https://patch.msgid.link/[email protected]

Changes in v2:
- Panic on MF_MSG_KERNEL, MF_MSG_KERNEL_HIGH_ORDER and MF_MSG_UNKNOWN instead of MF_MSG_GET_HWPOISON.
- Report MF_MSG_KERNEL for reserved pages when get_hwpoison_page() fails instead of MF_MSG_GET_HWPOISON.
- Link to v1: https://patch.msgid.link/[email protected]

To: Miaohe Lin <[email protected]>
To: Naoya Horiguchi <[email protected]>
To: Andrew Morton <[email protected]>
To: Steven Rostedt <[email protected]>
To: Masami Hiramatsu <[email protected]>
To: Mathieu Desnoyers <[email protected]>
To: Jonathan Corbet <[email protected]>
To: Shuah Khan <[email protected]>
To: David Hildenbrand <[email protected]>
To: Lorenzo Stoakes <[email protected]>
To: "Liam R. Howlett" <[email protected]>
To: Vlastimil Babka <[email protected]>
To: Mike Rapoport <[email protected]>
To: Suren Baghdasaryan <[email protected]>
To: Michal Hocko <[email protected]>
To: Shuah Khan <[email protected]>
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]
Cc: [email protected]

---
Breno Leitao (6):
mm/memory-failure: drop dead error_states[] entry for reserved pages mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page() mm/memory-failure: add panic option for unrecoverable pages
Documentation: document panic_on_unrecoverable_memory_failure sysctl

 Documentation/admin-guide/sysctl/vm.rst | 80 +++++++++++++++++++++++++++++++
 mm/memory-failure.c                     | 85 +++++++++++++++++++++++++--------
 2 files changed, 146 insertions(+), 19 deletions(-)

---
base-commit: e98d21c170b01ddef366f023bbfcf6b31509fa83
change-id: 20260323-ecc_panic-4e473b83087c

Best regards,

Signature · 1 line
Breno Leitao <[email protected]>
#6 lore raw

PATCH v7 2/6 mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

get_any_page() collapses three different failure modes into a single -EIO return:

* the put_page race in the !count_increased path;
* the HWPoisonHandlable() rejection that bounces out of

    __get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;

* the HWPoisonHandlable() rejection that goes through the count_increased / put_page / shake_page retry loop.

The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.

Distinguish them on the return path: keep -EIO for both the put_page race and the -EBUSY-after-retries branch (shake_page() cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep -EBUSY for the allocation race (unchanged), and return -ENOTRECOVERABLE only from the count_increased-true HWPoisonHandlable() rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.

Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc.

memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch is a no-op for users of memory_failure() and only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires the new return code through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases.

---
 mm/memory-failure.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
mm/memory-failure.c +15 -3
index 49bcfbd04d213..bae883df3ccb2 100644
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
 				shake_page(p);
 				goto try_again;
 			}
+			/*
+			 * Return -EIO rather than -ENOTRECOVERABLE: this
+			 * branch is also reached for pages that are merely
+			 * off-LRU transiently (e.g. a folio in the middle
+			 * of migration or compaction), which shake_page()
+			 * cannot drag back.  The caller cannot prove the
+			 * page is permanently kernel-owned from here, so
+			 * keep it on the recoverable errno.
+			 */
 			ret = -EIO;
 			goto out;
 		}
@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags)
 			goto try_again;
 		}
 		put_page(p);
-		ret = -EIO;
+		ret = -ENOTRECOVERABLE;
 	}
 out:
-	if (ret == -EIO)
+	if (ret == -EIO || ret == -ENOTRECOVERABLE)
 		pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));
 
 	return ret;
@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)
  *         -EIO for pages on which we can not handle memory errors,
  *         -EBUSY when get_hwpoison_page() has raced with page lifecycle
  *         operations like allocation and free,
- *         -EHWPOISON when the page is hwpoisoned and taken off from buddy.
+ *         -EHWPOISON when the page is hwpoisoned and taken off from buddy,
+ *         -ENOTRECOVERABLE for stable kernel-owned pages the handler
+ *         cannot recover (PG_reserved, slab, vmalloc, page tables,
+ *         kernel stacks, and similar non-LRU/non-buddy pages).
  */
 static int get_hwpoison_page(struct page *p, unsigned long flags)
 {
Signature · 1 line
2.53.0-Meta
#14 lore raw

PATCH v7 3/6 mm/memory-failure: report MF_MSG_KERNEL for unrecoverable kernel pages

In reply to Breno Leitao

The previous patch teaches get_any_page() to return -ENOTRECOVERABLE for stable unhandlable kernel pages (PG_reserved, slab, vmalloc, page tables, kernel stacks, ...). memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON, so callers that want to react to the unrecoverable cases (a panic option, smarter logging) cannot tell them apart from transient page-allocator races.

Turn the post-call branch into a switch over the get_hwpoison_page() return code: map -ENOTRECOVERABLE to MF_MSG_KERNEL and any other negative return to MF_MSG_GET_HWPOISON. case 0 keeps the existing free-buddy / kernel-high-order handling and case 1 falls through to the rest of memory_failure() unchanged.

The MF_MSG_KERNEL label and tracepoint string are kept as "reserved kernel page" to avoid breaking userspace tools that match on those literals; the enum value still adequately tags the failure even though it now also covers slab, vmalloc, page tables and kernel stack pages.

---
 mm/memory-failure.c | 17 +++++++++++++++--
 1 file changed, 15 insertions(+), 2 deletions(-)
mm/memory-failure.c +15 -2
index bae883df3ccb2..4b3a5d4190a07 100644
@@ -2410,7 +2410,8 @@ int memory_failure(unsigned long pfn, int flags)
 	 * that may make page_ref_freeze()/page_ref_unfreeze() mismatch.
 	 */
 	res = get_hwpoison_page(p, flags);
-	if (!res) {
+	switch (res) {
+	case 0:
 		if (is_free_buddy_page(p)) {
 			if (take_page_off_buddy(p)) {
 				page_ref_inc(p);
@@ -2429,7 +2430,19 @@ int memory_failure(unsigned long pfn, int flags)
 			res = action_result(pfn, MF_MSG_KERNEL_HIGH_ORDER, MF_IGNORED);
 		}
 		goto unlock_mutex;
-	} else if (res < 0) {
+	case 1:
+		/* Got a refcount on a handlable page. */
+		break;
+	case -ENOTRECOVERABLE:
+		/*
+		 * Stable unhandlable kernel-owned page (PG_reserved,
+		 * slab, vmalloc, page tables, kernel stacks, ...).
+		 * No recovery possible.
+		 */
+		res = action_result(pfn, MF_MSG_KERNEL, MF_IGNORED);
+		goto unlock_mutex;
+	default:
+		/* Transient lifecycle race with the page allocator. */
 		res = action_result(pfn, MF_MSG_GET_HWPOISON, MF_IGNORED);
 		goto unlock_mutex;
 	}
Signature · 1 line
2.53.0-Meta
#15 lore raw

PATCH v7 4/6 mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()

In reply to Breno Leitao

The previous patch already classifies PG_reserved pages as MF_MSG_KERNEL through the long path: get_hwpoison_page() calls __get_hwpoison_page() which fails HWPoisonHandlable(), get_any_page() exhausts its shake_page() retry budget, and the resulting -ENOTRECOVERABLE is mapped to MF_MSG_KERNEL by the switch. The outcome is correct but the work in between is wasted: shake_page() cannot turn a reserved page into a handlable one.

Detect PG_reserved up front in memory_failure() and report MF_MSG_KERNEL directly. put_ref_page() releases the caller's reference when MF_COUNT_INCREASED is set, which is important on the MADV_HWPOISON path where get_user_pages_fast() holds a reference across the call.

---
 mm/memory-failure.c | 13 +++++++++++++
 1 file changed, 13 insertions(+)
mm/memory-failure.c +13 -0
index 4b3a5d4190a07..8ba3df21d1270 100644
@@ -2398,6 +2398,19 @@ int memory_failure(unsigned long pfn, int flags)
 		goto unlock_mutex;
 	}
 
+	/*
+	 * PG_reserved pages are kernel-owned (memblock reservations,
+	 * driver reservations, ...) and cannot be recovered.  Skip the
+	 * get_hwpoison_page() lifecycle dance and report MF_MSG_KERNEL
+	 * straight away; HWPoisonHandlable() would just keep rejecting
+	 * the page through the retry budget anyway.
+	 */
+	if (PageReserved(p)) {
+		put_ref_page(pfn, flags);
+		res = action_result(pfn, MF_MSG_KERNEL, MF_IGNORED);
+		goto unlock_mutex;
+	}
+
 	/*
 	 * We need/can do nothing about count=0 pages.
 	 * 1) it's a free page, and therefore in safe hand:
Signature · 1 line
2.53.0-Meta
#18 lore raw

PATCH v7 5/6 mm/memory-failure: add panic option for unrecoverable pages

In reply to Breno Leitao

Add a sysctl panic_on_unrecoverable_memory_failure that triggers a kernel panic when memory_failure() encounters pages that cannot be recovered. This provides a clean crash with useful debug information rather than allowing silent data corruption or a delayed crash at an unrelated code path.

Panic eligibility is intentionally narrow: only MF_MSG_KERNEL with result == MF_IGNORED panics. After the previous patch, MF_MSG_KERNEL covers PG_reserved pages and the kernel-owned pages promoted from get_hwpoison_page() via -ENOTRECOVERABLE (slab, vmalloc, page tables, kernel stacks, ...).

All other action types are excluded:

- MF_MSG_GET_HWPOISON and MF_MSG_KERNEL_HIGH_ORDER can be reached by transient refcount races with the page allocator (an in-flight buddy allocation has refcount 0 and is no longer on the buddy free list, briefly), and panicking on them would risk killing the box for what is actually a recoverable userspace page.

- MF_MSG_UNKNOWN means identify_page_state() could not classify the page; that is precisely the wrong basis for a panic decision.

---
 mm/memory-failure.c | 23 +++++++++++++++++++++++
 1 file changed, 23 insertions(+)
mm/memory-failure.c +23 -0
index 8ba3df21d1270..cb2965c0ec0b4 100644
@@ -74,6 +74,8 @@ static int sysctl_memory_failure_recovery __read_mostly = 1;
 
 static int sysctl_enable_soft_offline __read_mostly = 1;
 
+static int sysctl_panic_on_unrecoverable_mf __read_mostly;
+
 atomic_long_t num_poisoned_pages __read_mostly = ATOMIC_LONG_INIT(0);
 
 static bool hw_memory_failure __read_mostly = false;
@@ -155,6 +157,15 @@ static const struct ctl_table memory_failure_table[] = {
 		.proc_handler	= proc_dointvec_minmax,
 		.extra1		= SYSCTL_ZERO,
 		.extra2		= SYSCTL_ONE,
+	},
+	{
+		.procname	= "panic_on_unrecoverable_memory_failure",
+		.data		= &sysctl_panic_on_unrecoverable_mf,
+		.maxlen		= sizeof(sysctl_panic_on_unrecoverable_mf),
+		.mode		= 0644,
+		.proc_handler	= proc_dointvec_minmax,
+		.extra1		= SYSCTL_ZERO,
+		.extra2		= SYSCTL_ONE,
 	}
 };
 
@@ -1267,6 +1278,15 @@ static void update_per_node_mf_stats(unsigned long pfn,
 	++mf_stats->total;
 }
 
+static bool panic_on_unrecoverable_mf(enum mf_action_page_type type,
+				      enum mf_result result)
+{
+	if (!sysctl_panic_on_unrecoverable_mf || result != MF_IGNORED)
+		return false;
+
+	return type == MF_MSG_KERNEL;
+}
+
 /*
  * "Dirty/Clean" indication is not 100% accurate due to the possibility of
  * setting PG_dirty outside page lock. See also comment above set_page_dirty().
@@ -1284,6 +1304,9 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type,
 	pr_err("%#lx: recovery action for %s: %s\n",
 		pfn, action_page_types[type], action_name[result]);
 
+	if (panic_on_unrecoverable_mf(type, result))
+		panic("Memory failure: %#lx: unrecoverable page", pfn);
+
 	return (result == MF_RECOVERED || result == MF_DELAYED) ? 0 : -EBUSY;
 }
 
Signature · 1 line
2.53.0-Meta
#19 lore raw

PATCH v7 6/6 Documentation: document panic_on_unrecoverable_memory_failure sysctl

In reply to Breno Leitao

Add documentation for the new vm.panic_on_unrecoverable_memory_failure sysctl, describing which failures trigger a panic (kernel-owned pages the handler cannot recover) and which are intentionally left out (transient allocator races and unclassified pages).

---
 Documentation/admin-guide/sysctl/vm.rst | 80 +++++++++++++++++++++++++++++++++
 1 file changed, 80 insertions(+)
Documentation/admin-guide/sysctl/vm.rst +80 -0
index 97e12359775c9..452c2ab25b35e 100644
@@ -67,6 +67,7 @@ Currently, these files are in /proc/sys/vm:
 - page-cluster
 - page_lock_unfairness
 - panic_on_oom
+- panic_on_unrecoverable_memory_failure
 - percpu_pagelist_high_fraction
 - stat_interval
 - stat_refresh
@@ -925,6 +926,85 @@ panic_on_oom=2+kdump gives you very strong tool to investigate
 why oom happens. You can get snapshot.
 
 
+panic_on_unrecoverable_memory_failure
+======================================
+
+When a hardware memory error (e.g. multi-bit ECC) hits a kernel page
+that cannot be recovered by the memory failure handler, the default
+behaviour is to ignore the error and continue operation.  This is
+dangerous because the corrupted data remains accessible to the kernel,
+risking silent data corruption or a delayed crash when the poisoned
+memory is next accessed.
+
+When enabled, this sysctl triggers a panic on memory failure events
+hitting reserved (``PageReserved``) memory: firmware reservations,
+the kernel image, vDSO, the zero page, and similar memblock-reserved
+regions.  These are owned by the kernel, are not managed by the page
+allocator, and cannot be recovered by the memory failure handler.
+
+Other unrecoverable kernel-owned populations (slab, vmalloc, page
+tables, kernel stacks, ...) are not currently covered by this
+sysctl.  The handler cannot reliably distinguish them from a
+userspace folio temporarily off the LRU during migration or
+compaction, and the cost of a false-positive panic on a recoverable
+userspace page is too high.  Such pages still go through the
+standard MF_MSG_GET_HWPOISON path: ``PG_hwpoison`` is set on them
+and a delayed crash on the next access remains possible.  Coverage
+may grow in the future as the handler gains stronger
+kernel-ownership signals.
+
+Recoverable failure paths are also intentionally left out: in-flight
+buddy allocations and other transient races with the page allocator
+can reach the same diagnostic, and panicking on them would risk
+killing the box for a page destined for userspace where the standard
+SIGBUS recovery path applies.  Pages whose state could not be
+classified at all are not covered either, since an unknown state is
+not a sound basis for a panic decision.
+
+For many environments it is preferable to panic immediately with a clean
+crash dump that captures the original error context, rather than to
+continue and face a random crash later whose cause is difficult to
+diagnose.
+
+Use cases
+---------
+
+This option is most useful in environments where unattributed crashes
+are expensive to debug or where data integrity must take precedence
+over availability:
+
+* Large fleets, where multi-bit ECC errors on kernel pages are observed
+  regularly and post-mortem analysis of an unrelated downstream crash
+  (often seconds to minutes after the original error) consumes
+  significant engineering effort.
+
+* Systems configured with kdump, where panicking at the moment of the
+  hardware error produces a vmcore that still contains the faulting
+  address, the affected page state, and the originating MCE/GHES
+  record — context that is typically lost by the time a delayed crash
+  occurs.
+
+* High-availability clusters that rely on fast, deterministic node
+  failure for failover, and prefer an immediate panic over silent data
+  corruption propagating to replicas or persistent storage.
+
+* Kernel and platform developers reproducing hwpoison issues with
+  tools such as ``mce-inject`` or error-injection debugfs interfaces,
+  where panicking on the unrecoverable path makes regressions
+  immediately visible instead of surfacing as later, unrelated
+  failures.
+
+= =====================================================================
+0 Try to continue operation (default).
+1 Panic immediately.  If the ``panic`` sysctl is also non-zero then the
+  machine will be rebooted.
+= =====================================================================
+
+Example::
+
+     echo 1 > /proc/sys/vm/panic_on_unrecoverable_memory_failure
+
+
 percpu_pagelist_high_fraction
 =============================
 
Signature · 1 line
2.53.0-Meta
#16 lore raw

Re: [PATCH v7 4/6] mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()

In reply to Breno Leitao
Quoted 7 lines · Breno Leitao

The previous patch already classifies PG_reserved pages as MF_MSG_KERNEL through the long path: get_hwpoison_page() calls __get_hwpoison_page() which fails HWPoisonHandlable(), get_any_page() exhausts its shake_page() retry budget, and the resulting -ENOTRECOVERABLE is mapped to MF_MSG_KERNEL by the switch. The outcome is correct but the work in between is wasted: shake_page() cannot turn a reserved page into a handlable one.

If really required, can we just move the check right there, into get_any_page() etc?

Signature · 3 lines
Cheers,

David
maintainer Acked-by on v7 1/6
#2 lore raw

Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages

In reply to Breno Leitao
Quoted 7 lines · Breno Leitao

The first entry of error_states[],

{ reserved, reserved, MF_MSG_KERNEL, me_kernel },

is unreachable. identify_page_state() has two callers, and neither one can dispatch a PG_reserved page to me_kernel():

Quoted 6 lines

* memory_failure() reaches identify_page_state() only after get_hwpoison_page() returned 1. get_any_page() reaches that return only via __get_hwpoison_page(), which gates the refcount on HWPoisonHandlable(). HWPoisonHandlable() rejects PG_reserved pages, so they fail with -EBUSY/-EIO long before identify_page_state() runs.

You should clarify why they are rejected. There is no explicit check for PG_reserved in there!

Quoted 8 lines

* try_memory_failure_hugetlb() reaches identify_page_state() on the MF_HUGETLB_IN_USED branch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head, MF_MSG_HUGE, me_huge_page }, so they dispatch to me_huge_page() before the (now-removed) reserved entry would have matched, regardless of whether PG_reserved happens to be set on the head page.

See hugetlb_folio_init_vmemmap(): we always clear PG_reserved for hugetlb folios allocated from memblock.

Quoted 53 lines

me_kernel() never executes and the entry exists only to be matched against by code that cannot see it.

Drop the entry, the me_kernel() helper, and the now-unused "reserved" macro. Leave the MF_MSG_KERNEL enum value in place: it remains part of the tracepoint and pr_err() string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.

No functional change.

---
 mm/memory-failure.c | 14 --------------
 1 file changed, 14 deletions(-)
mm/memory-failure.c +0 -13
index 866c4428ac7ef..49bcfbd04d213 100644
@@ -992,17 +992,6 @@ static bool has_extra_refcount(struct page_state *ps, struct page *p,
 	return false;
 }
 
-/*
- * Error hit kernel page.
- * Do nothing, try to be lucky and not touch this instead. For a few cases we
- * could be more sophisticated.
- */
-static int me_kernel(struct page_state *ps, struct page *p)
-{
-	unlock_page(p);
-	return MF_IGNORED;
-}
-
 /*
  * Page in unknown state. Do nothing.
  * This is a catch-all in case we fail to make sense of the page state.
@@ -1211,10 +1200,8 @@ static int me_huge_page(struct page_state *ps, struct page *p)
 #define mlock		(1UL << PG_mlocked)
 #define lru		(1UL << PG_lru)
 #define head		(1UL << PG_head)
-#define reserved	(1UL << PG_reserved)
 
 static struct page_state error_states[] = {
-	{ reserved,	reserved,	MF_MSG_KERNEL,	me_kernel },
 	/*
 	 * free pages are specially detected outside this table:
 	 * PG_buddy pages only make a small fraction of all free pages.

mm/memory-failure.c static struct page_state error_states[] = { · line 1238

Quoted 7 lines
mm/memory-failure.c +0 -1
@@ -1246,7 +1233,6 @@ static struct page_state error_states[] = {
 #undef mlock
 #undef lru
 #undef head
-#undef reserved
 
 static void update_per_node_mf_stats(unsigned long pfn,
 				     enum mf_result result)

Yes, I think this should work.

Signature · 3 lines
Cheers,

David
Reviewed-by on v7 1/6
#4 lore raw

Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages

In reply to Breno Leitao
Quoted 7 lines · Breno Leitao

The first entry of error_states[],

{ reserved, reserved, MF_MSG_KERNEL, me_kernel },

is unreachable. identify_page_state() has two callers, and neither one can dispatch a PG_reserved page to me_kernel():

Quoted 6 lines

* memory_failure() reaches identify_page_state() only after get_hwpoison_page() returned 1. get_any_page() reaches that return only via __get_hwpoison_page(), which gates the refcount on HWPoisonHandlable(). HWPoisonHandlable() rejects PG_reserved pages, so they fail with -EBUSY/-EIO long before identify_page_state() runs.

HWPoisonHandlable() does not test PG_reserved directly; it only lets LRU or free buddy pages through:

return PageLRU(page) || is_free_buddy_page(page);

So this really relies on PG_reserved not being combined with either of those states. I would not expect that to happen, though.

Quoted 8 lines

* try_memory_failure_hugetlb() reaches identify_page_state() on the MF_HUGETLB_IN_USED branch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head, MF_MSG_HUGE, me_huge_page }, so they dispatch to me_huge_page() before the (now-removed) reserved entry would have matched, regardless of whether PG_reserved happens to be set on the head page.

As David pointed out, hugetlb setup clears PG_reserved before setting PG_head. See hugetlb_folio_init_vmemmap():

	__folio_clear_reserved(folio);
	__folio_set_head(folio);
Quoted 3 lines

me_kernel() never executes and the entry exists only to be matched against by code that cannot see it.

identify_page_state() is reached only when get_hwpoison_page() returns 1, but a PG_reserved page would not get that far, IIUC :)

Quoted 7 lines

Drop the entry, the me_kernel() helper, and the now-unused "reserved" macro. Leave the MF_MSG_KERNEL enum value in place: it remains part of the tracepoint and pr_err() string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.

Quoted 5 lines

No functional change.

---

With David's comments addressed, feel free to add:

#3 lore raw

Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages

In reply to David Hildenbrand (Arm)
Quoted 10 lines · David Hildenbrand (Arm)
Quoted 6 lines · Breno Leitao

* memory_failure() reaches identify_page_state() only after get_hwpoison_page() returned 1. get_any_page() reaches that return only via __get_hwpoison_page(), which gates the refcount on HWPoisonHandlable(). HWPoisonHandlable() rejects PG_reserved pages, so they fail with -EBUSY/-EIO long before identify_page_state() runs.

You should clarify why they are rejected. There is no explicit check for PG_reserved in there!

True, I meant that PG_reserved pages do not fit any of the criterias of HWPoisonHandlable().

I will rewrite it more explictly:

__get_hwpoison_page() only takes a refcount when the page is HWPoisonHandlable()'d, and HWPoisonHandlable() is an allowlist for LRU / free-buddy / (soft-offline) movable_ops pages.

is it any better?

Quoted 10 lines
Quoted 7 lines

* try_memory_failure_hugetlb() reaches identify_page_state() on the MF_HUGETLB_IN_USED branch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head, MF_MSG_HUGE, me_huge_page }, so they dispatch to me_huge_page() before the (now-removed) reserved entry would have matched, regardless of whether PG_reserved happens to be set on the head page.

See hugetlb_folio_init_vmemmap(): we always clear PG_reserved for hugetlb folios allocated from memblock.

Thanks. I clearly see a call to __folio_clear_reserved(folio), so, huge pagetlb folios are never reserved.

A better summary would be:

try_memory_failure_hugetlb() reaches identify_page_state() only via the MF_HUGETLB_IN_USED branch, as hugetlb folios don't carry PG_reserved at that point (hugetlb_folio_init_vmemmap() clears it during init).

Quoted 3 lines

Yes, I think this should work.

Thanks for the review,
--breno

#17 lore raw

Re: [PATCH v7 4/6] mm/memory-failure: short-circuit PG_reserved before get_hwpoison_page()

In reply to David Hildenbrand (Arm)
Quoted 10 lines · David Hildenbrand (Arm)
Quoted 7 lines · Breno Leitao

The previous patch already classifies PG_reserved pages as MF_MSG_KERNEL through the long path: get_hwpoison_page() calls __get_hwpoison_page() which fails HWPoisonHandlable(), get_any_page() exhausts its shake_page() retry budget, and the resulting -ENOTRECOVERABLE is mapped to MF_MSG_KERNEL by the switch. The outcome is correct but the work in between is wasted: shake_page() cannot turn a reserved page into a handlable one.

If really required, can we just move the check right there, into get_any_page() etc?

Sure, we might move it to get_any_page(). I took this current approach based on the following facts:

1) Lance suggested it, and it sounded a good idea.

	https://lore.kernel.org/all/[email protected]/

2) There is a _similar_ check close to this one in memory_failure(), just before this one:

if (TestSetPageHWPoison(p)) {
....

	action_result()
	goto unlock_mutex;

}

and now

  if (PageReserved(p)) {
	...
  	action_result()
	goto unlock_mutes;

}

3) I wanted to give get it as real layering point, not handwaving.

That said, I will short-circuit reserved pages inside get_any_page(), in an updated version.

Again, thanks for the review and direction!
--breno

#7 lore raw

Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

In reply to Breno Leitao
Quoted 74 lines · Breno Leitao

get_any_page() collapses three different failure modes into a single -EIO return:

* the put_page race in the !count_increased path;
* the HWPoisonHandlable() rejection that bounces out of __get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;
* the HWPoisonHandlable() rejection that goes through the count_increased / put_page / shake_page retry loop.

The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.

Distinguish them on the return path: keep -EIO for both the put_page race and the -EBUSY-after-retries branch (shake_page() cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep -EBUSY for the allocation race (unchanged), and return -ENOTRECOVERABLE only from the count_increased-true HWPoisonHandlable() rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.

Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc.

memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch is a no-op for users of memory_failure() and only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires the new return code through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases.

---
mm/memory-failure.c | 18 +++++++++++++++---

1 file changed, 15 insertions(+), 3 deletions(-)
mm/memory-failure.c +10 -1
index 49bcfbd04d213..bae883df3ccb2 100644
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
 				shake_page(p);
 				goto try_again;
 			}
+			/*
+			 * Return -EIO rather than -ENOTRECOVERABLE: this
+			 * branch is also reached for pages that are merely
+			 * off-LRU transiently (e.g. a folio in the middle
+			 * of migration or compaction), which shake_page()
+			 * cannot drag back.  The caller cannot prove the
+			 * page is permanently kernel-owned from here, so
+			 * keep it on the recoverable errno.
+			 */
 			ret = -EIO;
 			goto out;
 		}
@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags)
 			goto try_again;
 		}
 		put_page(p);
-		ret = -EIO;
+		ret = -ENOTRECOVERABLE;
 	}

out:

mm/memory-failure.c +1 -1
-	if (ret == -EIO)
+	if (ret == -EIO || ret == -ENOTRECOVERABLE)
		pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));

	return ret;

mm/memory-failure.c static int __get_unpoison_page(struct page *page) · line 1502

Quoted 8 lines
mm/memory-failure.c +4 -1
@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)
 *         -EIO for pages on which we can not handle memory errors,
 *         -EBUSY when get_hwpoison_page() has raced with page lifecycle
 *         operations like allocation and free,
- *         -EHWPOISON when the page is hwpoisoned and taken off from buddy.
+ *         -EHWPOISON when the page is hwpoisoned and taken off from buddy,
+ *         -ENOTRECOVERABLE for stable kernel-owned pages the handler
+ *         cannot recover (PG_reserved, slab, vmalloc, page tables,
+ *         kernel stacks, and similar non-LRU/non-buddy pages).

Did you test this patch series? I don't see how we ever get to -ENOTRECOVERABLE there ...

Even with MF_COUNT_INCREASED, the first pass does:

	if (flags & MF_COUNT_INCREASED)
		count_increased = true;

	[...]

	if (PageHuge(p) || HWPoisonHandlable(p, flags)) {
		ret = 1;
	} else {
		if (pass++ < GET_PAGE_MAX_RETRY_NUM) { <-
			put_page(p);
			shake_page(p);
			count_increased = false;
			goto try_again; <-
		}
		put_page(p);
		ret = -ENOTRECOVERABLE;
	}

Then we come back with count_increased=false:

try_again:

	if (!count_increased) {
		ret = __get_hwpoison_page(p, flags); <-
		if (!ret) {
		[...]
		} else if (ret == -EBUSY) { <-
		[...]
			ret = -EIO;
			goto out; <-
		}
	}

For slab/vmalloc/page-table pages, __get_hwpoison_page() returns -EBUSY:

	if (!HWPoisonHandlable(&folio->page, flags))
		return -EBUSY;

so they still seem to end up as -EIO ... Am I missing something?

Quoted 8 lines
mm/memory-failure.c +0 -0
 */
 static int get_hwpoison_page(struct page *p, unsigned long flags)
 {
Signature · 2 lines
-- 
2.53.0-Meta
#8 lore raw

Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

Quoted 74 lines · Lance Yang
Quoted 72 lines · Breno Leitao

get_any_page() collapses three different failure modes into a single -EIO return:

* the put_page race in the !count_increased path;
* the HWPoisonHandlable() rejection that bounces out of __get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;
* the HWPoisonHandlable() rejection that goes through the count_increased / put_page / shake_page retry loop.

The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.

Distinguish them on the return path: keep -EIO for both the put_page race and the -EBUSY-after-retries branch (shake_page() cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep -EBUSY for the allocation race (unchanged), and return -ENOTRECOVERABLE only from the count_increased-true HWPoisonHandlable() rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.

Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc.

memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch is a no-op for users of memory_failure() and only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires the new return code through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases.

---
mm/memory-failure.c | 18 +++++++++++++++---

1 file changed, 15 insertions(+), 3 deletions(-)
mm/memory-failure.c +10 -1
index 49bcfbd04d213..bae883df3ccb2 100644
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
 				shake_page(p);
 				goto try_again;
 			}
+			/*
+			 * Return -EIO rather than -ENOTRECOVERABLE: this
+			 * branch is also reached for pages that are merely
+			 * off-LRU transiently (e.g. a folio in the middle
+			 * of migration or compaction), which shake_page()
+			 * cannot drag back.  The caller cannot prove the
+			 * page is permanently kernel-owned from here, so
+			 * keep it on the recoverable errno.
+			 */
 			ret = -EIO;
 			goto out;
 		}
@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags)
 			goto try_again;
 		}
 		put_page(p);
-		ret = -EIO;
+		ret = -ENOTRECOVERABLE;
 	}

out:

mm/memory-failure.c +1 -1
-	if (ret == -EIO)
+	if (ret == -EIO || ret == -ENOTRECOVERABLE)
		pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));
Quoted 13 lines

mm/memory-failure.c static int __get_unpoison_page(struct page *page) · line 1502

Quoted 10 lines
	return ret;
mm/memory-failure.c +4 -1
@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)
 *         -EIO for pages on which we can not handle memory errors,
 *         -EBUSY when get_hwpoison_page() has raced with page lifecycle
 *         operations like allocation and free,
- *         -EHWPOISON when the page is hwpoisoned and taken off from buddy.
+ *         -EHWPOISON when the page is hwpoisoned and taken off from buddy,
+ *         -ENOTRECOVERABLE for stable kernel-owned pages the handler
+ *         cannot recover (PG_reserved, slab, vmalloc, page tables,
+ *         kernel stacks, and similar non-LRU/non-buddy pages).

Did you test this patch series? I don't see how we ever get to -ENOTRECOVERABLE there ...

Yes, I did. I am using the following test case:

https://github.com/leitao/linux/commit/cfebe84ddeab5ac34ed456331db980d57e7025dc

	# RUN_DESTRUCTIVE=1 tools/testing/selftests/mm/hwpoison-panic.sh
	# enabling /proc/sys/vm/panic_on_unrecoverable_memory_failure
	# injecting hwpoison at phys 0x2a00000 (Kernel rodata)
	# expecting kernel panic: 'Memory failure: <pfn>: unrecoverable page'
	[  501.113256] Memory failure: 0x2a00: recovery action for reserved kernel page: Ignored
	[  501.113956] Kernel panic - not syncing: Memory failure: 0x2a00: unrecoverable page
Quoted 34 lines

Even with MF_COUNT_INCREASED, the first pass does:

	if (flags & MF_COUNT_INCREASED)
		count_increased = true;

	[...]

	if (PageHuge(p) || HWPoisonHandlable(p, flags)) {
		ret = 1;
	} else {
		if (pass++ < GET_PAGE_MAX_RETRY_NUM) { <-
			put_page(p);
			shake_page(p);
			count_increased = false;
			goto try_again; <-
		}
		put_page(p);
		ret = -ENOTRECOVERABLE;
	}

Then we come back with count_increased=false:

try_again:

	if (!count_increased) {
		ret = __get_hwpoison_page(p, flags); <-
		if (!ret) {
		[...]
		} else if (ret == -EBUSY) { <-
		[...]
			ret = -EIO;
			goto out; <-
		}
	}
Quoted 6 lines

For slab/vmalloc/page-table pages, __get_hwpoison_page() returns -EBUSY:

	if (!HWPoisonHandlable(&folio->page, flags))
		return -EBUSY;

so they still seem to end up as -EIO ... Am I missing something?

You are not, and thanks for catching this. I traced it again and the -ENOTRECOVERABLE branch is unreachable for slab/vmalloc/page-table pages exactly as you described. The __get_hwpoison_page() → -EBUSY → shake → retry loop catches them first and they exit as -EIO.

The selftest I am using (link above) only validated the PageReserved short-circuit added in patch 3, which lives in memory_failure() and never reaches get_any_page().

I even thought about this code path, and I was not convinced we should return -ENOTRECOVERABLE, thus I documented the following (as in this current patch)

@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
shake_page(p);
goto try_again;
}
+ /*
+ * Return -EIO rather than -ENOTRECOVERABLE: this
+ * branch is also reached for pages that are merely
+ * off-LRU transiently (e.g. a folio in the middle
+ * of migration or compaction), which shake_page()
+ * cannot drag back. The caller cannot prove the
+ * page is permanently kernel-owned from here, so
+ * keep it on the recoverable errno.
+ */
ret = -EIO;

maintainer Acked-by on v7 1/6
#5 lore raw

Re: [PATCH v7 1/6] mm/memory-failure: drop dead error_states[] entry for reserved pages

In reply to Breno Leitao
Quoted 25 lines · Breno Leitao

The first entry of error_states[],

{ reserved, reserved, MF_MSG_KERNEL, me_kernel },

is unreachable. identify_page_state() has two callers, and neither one can dispatch a PG_reserved page to me_kernel():

* memory_failure() reaches identify_page_state() only after get_hwpoison_page() returned 1. get_any_page() reaches that return only via __get_hwpoison_page(), which gates the refcount on HWPoisonHandlable(). HWPoisonHandlable() rejects PG_reserved pages, so they fail with -EBUSY/-EIO long before identify_page_state() runs.

* try_memory_failure_hugetlb() reaches identify_page_state() on the MF_HUGETLB_IN_USED branch, but the page is necessarily a hugetlb folio there. The first table entry that matches a hugetlb folio is { head, head, MF_MSG_HUGE, me_huge_page }, so they dispatch to me_huge_page() before the (now-removed) reserved entry would have matched, regardless of whether PG_reserved happens to be set on the head page.

me_kernel() never executes and the entry exists only to be matched against by code that cannot see it.

Quoted 7 lines

Drop the entry, the me_kernel() helper, and the now-unused "reserved" macro. Leave the MF_MSG_KERNEL enum value in place: it remains part of the tracepoint and pr_err() string tables, and follow-on work to classify unrecoverable kernel pages can reuse it without churning the user-visible enum.

No functional change.

As the code evolves, this entry is no longer needed. Thanks for cleanup.

Quoted 3 lines

With David's comments addressed, this patch looks good to me:

Thanks.
.

maintainer
#13 lore raw

Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

In reply to Breno Leitao
Quoted 61 lines · Breno Leitao

get_any_page() collapses three different failure modes into a single -EIO return:

* the put_page race in the !count_increased path;
* the HWPoisonHandlable() rejection that bounces out of

    __get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;

* the HWPoisonHandlable() rejection that goes through the count_increased / put_page / shake_page retry loop.

The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.

Distinguish them on the return path: keep -EIO for both the put_page race and the -EBUSY-after-retries branch (shake_page() cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep -EBUSY for the allocation race (unchanged), and return -ENOTRECOVERABLE only from the count_increased-true HWPoisonHandlable() rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.

Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc.

memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch is a no-op for users of memory_failure() and only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires the new return code through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases.

---
 mm/memory-failure.c | 18 +++++++++++++++---
 1 file changed, 15 insertions(+), 3 deletions(-)
mm/memory-failure.c +9 -0
index 49bcfbd04d213..bae883df3ccb2 100644
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
 				shake_page(p);
 				goto try_again;
 			}
+			/*
+			 * Return -EIO rather than -ENOTRECOVERABLE: this
+			 * branch is also reached for pages that are merely
+			 * off-LRU transiently (e.g. a folio in the middle
+			 * of migration or compaction), which shake_page()
+			 * cannot drag back.  The caller cannot prove the
+			 * page is permanently kernel-owned from here, so
+			 * keep it on the recoverable errno.
+			 */
 			ret = -EIO;
 			goto out;
 		}

mm/memory-failure.c static int get_any_page(struct page *p, unsigned long flags) · line 1439

Quoted 5 lines
mm/memory-failure.c +1 -1
@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags)
 			goto try_again;
 		}
 		put_page(p);
-		ret = -EIO;
+		ret = -ENOTRECOVERABLE;

Theoretically, pages that are merely off-LRU transiently as you commented above could reach here too? Or am I miss something?

Thanks.
.

Quoted 20 lines
mm/memory-failure.c +5 -2
 	}
 out:
-	if (ret == -EIO)
+	if (ret == -EIO || ret == -ENOTRECOVERABLE)
 		pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));
 
 	return ret;
@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)
  *         -EIO for pages on which we can not handle memory errors,
  *         -EBUSY when get_hwpoison_page() has raced with page lifecycle
  *         operations like allocation and free,
- *         -EHWPOISON when the page is hwpoisoned and taken off from buddy.
+ *         -EHWPOISON when the page is hwpoisoned and taken off from buddy,
+ *         -ENOTRECOVERABLE for stable kernel-owned pages the handler
+ *         cannot recover (PG_reserved, slab, vmalloc, page tables,
+ *         kernel stacks, and similar non-LRU/non-buddy pages).
  */
 static int get_hwpoison_page(struct page *p, unsigned long flags)
 {
#9 lore raw

Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

In reply to Breno Leitao
Quoted 89 lines · Breno Leitao
Quoted 74 lines · Lance Yang
Quoted 72 lines · Breno Leitao

get_any_page() collapses three different failure modes into a single -EIO return:

* the put_page race in the !count_increased path;
* the HWPoisonHandlable() rejection that bounces out of __get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;
* the HWPoisonHandlable() rejection that goes through the count_increased / put_page / shake_page retry loop.

The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.

Distinguish them on the return path: keep -EIO for both the put_page race and the -EBUSY-after-retries branch (shake_page() cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep -EBUSY for the allocation race (unchanged), and return -ENOTRECOVERABLE only from the count_increased-true HWPoisonHandlable() rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.

Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc.

memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch is a no-op for users of memory_failure() and only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires the new return code through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases.

---
mm/memory-failure.c | 18 +++++++++++++++---

1 file changed, 15 insertions(+), 3 deletions(-)
mm/memory-failure.c +10 -1
index 49bcfbd04d213..bae883df3ccb2 100644
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
 				shake_page(p);
 				goto try_again;
 			}
+			/*
+			 * Return -EIO rather than -ENOTRECOVERABLE: this
+			 * branch is also reached for pages that are merely
+			 * off-LRU transiently (e.g. a folio in the middle
+			 * of migration or compaction), which shake_page()
+			 * cannot drag back.  The caller cannot prove the
+			 * page is permanently kernel-owned from here, so
+			 * keep it on the recoverable errno.
+			 */
 			ret = -EIO;
 			goto out;
 		}
@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags)
 			goto try_again;
 		}
 		put_page(p);
-		ret = -EIO;
+		ret = -ENOTRECOVERABLE;
 	}

out:

mm/memory-failure.c +1 -1
-	if (ret == -EIO)
+	if (ret == -EIO || ret == -ENOTRECOVERABLE)
		pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));
Quoted 13 lines

mm/memory-failure.c static int __get_unpoison_page(struct page *page) · line 1502

Quoted 10 lines
	return ret;
mm/memory-failure.c +4 -1
@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)
 *         -EIO for pages on which we can not handle memory errors,
 *         -EBUSY when get_hwpoison_page() has raced with page lifecycle
 *         operations like allocation and free,
- *         -EHWPOISON when the page is hwpoisoned and taken off from buddy.
+ *         -EHWPOISON when the page is hwpoisoned and taken off from buddy,
+ *         -ENOTRECOVERABLE for stable kernel-owned pages the handler
+ *         cannot recover (PG_reserved, slab, vmalloc, page tables,
+ *         kernel stacks, and similar non-LRU/non-buddy pages).

Did you test this patch series? I don't see how we ever get to -ENOTRECOVERABLE there ...

Quoted 1 line

Yes, I did. I am using the following test case:

Okay.

Quoted 49 lines

https://github.com/leitao/linux/commit/cfebe84ddeab5ac34ed456331db980d57e7025dc

	# RUN_DESTRUCTIVE=1 tools/testing/selftests/mm/hwpoison-panic.sh
	# enabling /proc/sys/vm/panic_on_unrecoverable_memory_failure
	# injecting hwpoison at phys 0x2a00000 (Kernel rodata)
	# expecting kernel panic: 'Memory failure: <pfn>: unrecoverable page'
	[  501.113256] Memory failure: 0x2a00: recovery action for reserved kernel page: Ignored
	[  501.113956] Kernel panic - not syncing: Memory failure: 0x2a00: unrecoverable page
Quoted 39 lines

Even with MF_COUNT_INCREASED, the first pass does:

	if (flags & MF_COUNT_INCREASED)
		count_increased = true;

	[...]

	if (PageHuge(p) || HWPoisonHandlable(p, flags)) {
		ret = 1;
	} else {
		if (pass++ < GET_PAGE_MAX_RETRY_NUM) { <-
			put_page(p);
			shake_page(p);
			count_increased = false;
			goto try_again; <-
		}
		put_page(p);
		ret = -ENOTRECOVERABLE;
	}

Then we come back with count_increased=false:

try_again:

	if (!count_increased) {
		ret = __get_hwpoison_page(p, flags); <-
		if (!ret) {
		[...]
		} else if (ret == -EBUSY) { <-
		[...]
			ret = -EIO;
			goto out; <-
		}
	}

For slab/vmalloc/page-table pages, __get_hwpoison_page() returns -EBUSY:

	if (!HWPoisonHandlable(&folio->page, flags))
		return -EBUSY;
Quoted 6 lines
Quoted 1 line

so they still seem to end up as -EIO ... Am I missing something?

You are not, and thanks for catching this. I traced it again and the -ENOTRECOVERABLE branch is unreachable for slab/vmalloc/page-table pages exactly as you described. The __get_hwpoison_page() → -EBUSY → shake → retry loop catches them first and they exit as -EIO.

Wonder if it would be simpler to just do a positive check near the top of get_any_page() instead. Something like:

static bool hwpoison_unrecoverable_kernel_page(struct page *page,
unsigned long flags)

{
	if ((flags & MF_SOFT_OFFLINE) && page_has_movable_ops(page))
		return false;

	return PageReserved(page) || PageSlab(page) || 
		PageTable(page) || PageLargeKmalloc(page);
}

static int get_any_page(struct page *p, unsigned long flags)

{
	int ret = 0, pass = 0;
	bool count_increased = false;

	if (flags & MF_COUNT_INCREASED)
		count_increased = true;

	if (hwpoison_unrecoverable_kernel_page(p, flags)) {
		if (count_increased)
			put_page(p);
		ret = -ENOTRECOVERABLE;
		goto out;
	}

[...]
}

Then get_any_page() could return -ENOTRECOVERABLE only for page types we can positively identify as kernel-owned.

These types always fail HWPoisonHandlable(), so retrying does not really buy us anything for them.

Won't cover everything (vmalloc, kernel stacks, etc. have no page_type to key off), but that's fine - best effort, right?

Cheers, Lance

Quoted 23 lines

The selftest I am using (link above) only validated the PageReserved short-circuit added in patch 3, which lives in memory_failure() and never reaches get_any_page().

I even thought about this code path, and I was not convinced we should return -ENOTRECOVERABLE, thus I documented the following (as in this current patch)

@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
shake_page(p);
goto try_again;
}
+ /*
+ * Return -EIO rather than -ENOTRECOVERABLE: this
+ * branch is also reached for pages that are merely
+ * off-LRU transiently (e.g. a folio in the middle
+ * of migration or compaction), which shake_page()
+ * cannot drag back. The caller cannot prove the
+ * page is permanently kernel-owned from here, so
+ * keep it on the recoverable errno.
+ */
ret = -EIO;

#10 lore raw

Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

Quoted 151 lines · Lance Yang
Quoted 89 lines · Breno Leitao
Quoted 74 lines · Lance Yang
Quoted 72 lines · Breno Leitao

get_any_page() collapses three different failure modes into a single -EIO return:

* the put_page race in the !count_increased path;
* the HWPoisonHandlable() rejection that bounces out of __get_hwpoison_page() with -EBUSY and exhausts shake_page() retries;
* the HWPoisonHandlable() rejection that goes through the count_increased / put_page / shake_page retry loop.

The first is transient (the page is racing with the allocator). The second can be either transient (a userspace folio briefly off LRU during migration/compaction) or stable (slab/vmalloc/page-table/ kernel-stack pages). The third describes a stable kernel-owned page that the count_increased=true caller already held a reference on.

Distinguish them on the return path: keep -EIO for both the put_page race and the -EBUSY-after-retries branch (shake_page() cannot drag a folio back from active migration, so we cannot prove the page is permanently kernel-owned from there), keep -EBUSY for the allocation race (unchanged), and return -ENOTRECOVERABLE only from the count_increased-true HWPoisonHandlable() rejection that exhausts its retries -- the caller's reference is structural evidence that the page is owned by the kernel.

Extend the unhandlable-page pr_err() to fire for either errno and update the get_hwpoison_page() kerneldoc.

memory_failure() still folds every negative return into MF_MSG_GET_HWPOISON via its existing "else if (res < 0)" branch, so this patch is a no-op for users of memory_failure() and only changes the errno that soft_offline_page() can propagate to its callers. A follow-up wires the new return code through memory_failure() and reports MF_MSG_KERNEL for the unrecoverable cases.

---
mm/memory-failure.c | 18 +++++++++++++++---

1 file changed, 15 insertions(+), 3 deletions(-)
mm/memory-failure.c +10 -1
index 49bcfbd04d213..bae883df3ccb2 100644
@@ -1408,6 +1408,15 @@ static int get_any_page(struct page *p, unsigned long flags)
 				shake_page(p);
 				goto try_again;
 			}
+			/*
+			 * Return -EIO rather than -ENOTRECOVERABLE: this
+			 * branch is also reached for pages that are merely
+			 * off-LRU transiently (e.g. a folio in the middle
+			 * of migration or compaction), which shake_page()
+			 * cannot drag back.  The caller cannot prove the
+			 * page is permanently kernel-owned from here, so
+			 * keep it on the recoverable errno.
+			 */
 			ret = -EIO;
 			goto out;
 		}
@@ -1427,10 +1436,10 @@ static int get_any_page(struct page *p, unsigned long flags)
 			goto try_again;
 		}
 		put_page(p);
-		ret = -EIO;
+		ret = -ENOTRECOVERABLE;
 	}

out:

mm/memory-failure.c +1 -1
-	if (ret == -EIO)
+	if (ret == -EIO || ret == -ENOTRECOVERABLE)
		pr_err("%#lx: unhandlable page.\n", page_to_pfn(p));
Quoted 13 lines

mm/memory-failure.c static int __get_unpoison_page(struct page *page) · line 1502

Quoted 10 lines
	return ret;
mm/memory-failure.c +4 -1
@@ -1487,7 +1496,10 @@ static int __get_unpoison_page(struct page *page)
 *         -EIO for pages on which we can not handle memory errors,
 *         -EBUSY when get_hwpoison_page() has raced with page lifecycle
 *         operations like allocation and free,
- *         -EHWPOISON when the page is hwpoisoned and taken off from buddy.
+ *         -EHWPOISON when the page is hwpoisoned and taken off from buddy,
+ *         -ENOTRECOVERABLE for stable kernel-owned pages the handler
+ *         cannot recover (PG_reserved, slab, vmalloc, page tables,
+ *         kernel stacks, and similar non-LRU/non-buddy pages).

Did you test this patch series? I don't see how we ever get to -ENOTRECOVERABLE there ...

Quoted 1 line

Yes, I did. I am using the following test case:

Okay.

Quoted 49 lines

https://github.com/leitao/linux/commit/cfebe84ddeab5ac34ed456331db980d57e7025dc

	# RUN_DESTRUCTIVE=1 tools/testing/selftests/mm/hwpoison-panic.sh
	# enabling /proc/sys/vm/panic_on_unrecoverable_memory_failure
	# injecting hwpoison at phys 0x2a00000 (Kernel rodata)
	# expecting kernel panic: 'Memory failure: <pfn>: unrecoverable page'
	[  501.113256] Memory failure: 0x2a00: recovery action for reserved kernel page: Ignored
	[  501.113956] Kernel panic - not syncing: Memory failure: 0x2a00: unrecoverable page
Quoted 39 lines

Even with MF_COUNT_INCREASED, the first pass does:

	if (flags & MF_COUNT_INCREASED)
		count_increased = true;

	[...]

	if (PageHuge(p) || HWPoisonHandlable(p, flags)) {
		ret = 1;
	} else {
		if (pass++ < GET_PAGE_MAX_RETRY_NUM) { <-
			put_page(p);
			shake_page(p);
			count_increased = false;
			goto try_again; <-
		}
		put_page(p);
		ret = -ENOTRECOVERABLE;
	}

Then we come back with count_increased=false:

try_again:

	if (!count_increased) {
		ret = __get_hwpoison_page(p, flags); <-
		if (!ret) {
		[...]
		} else if (ret == -EBUSY) { <-
		[...]
			ret = -EIO;
			goto out; <-
		}
	}

For slab/vmalloc/page-table pages, __get_hwpoison_page() returns -EBUSY:

	if (!HWPoisonHandlable(&folio->page, flags))
		return -EBUSY;
Quoted 6 lines
Quoted 1 line

so they still seem to end up as -EIO ... Am I missing something?

You are not, and thanks for catching this. I traced it again and the -ENOTRECOVERABLE branch is unreachable for slab/vmalloc/page-table pages exactly as you described. The __get_hwpoison_page() → -EBUSY → shake → retry loop catches them first and they exit as -EIO.

Quoted 5 lines

Wonder if it would be simpler to just do a positive check near the top of get_any_page() instead. Something like:

static bool hwpoison_unrecoverable_kernel_page(struct page *page,
unsigned long flags)

Ack. We probably want to call it something like HWPoisonKernelOwned() to follow the same naming sematics of these helpers, such as HWPoisonHandlable()

By the way, I will re-include the self test back to this patch series, In case they are not useful, we do not merge it.

Thanks for the review,
--breno

#11 lore raw

Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

On 2026/5/15 21:13, Breno Leitao wrote:
[...]

Quoted 4 lines
Quoted 4 lines

Wonder if it would be simpler to just do a positive check near the top of get_any_page() instead. Something like:

Quoted 9 lines
Quoted 2 lines

static bool hwpoison_unrecoverable_kernel_page(struct page *page,
unsigned long flags)

Ack. We probably want to call it something like HWPoisonKernelOwned() to follow the same naming sematics of these helpers, such as HWPoisonHandlable()

By the way, I will re-include the self test back to this patch series, In case they are not useful, we do not merge it.

Sounds good :)

Can you also test the relevant page types if possible, especially the ones the new helper is supposed to classify?

Cheers, Lance

#12 lore raw

Re: [PATCH v7 2/6] mm/memory-failure: surface unhandlable kernel pages as -ENOTRECOVERABLE

Quoted 14 lines · Lance Yang

On 2026/5/15 21:13, Breno Leitao wrote:
[...]

Quoted 10 lines
Quoted 6 lines

Wonder if it would be simpler to just do a positive check near the top of get_any_page() instead. Something like:

static bool hwpoison_unrecoverable_kernel_page(struct page *page,
unsigned long flags)

Ack. We probably want to call it something like HWPoisonKernelOwned() to follow the same naming sematics of these helpers, such as HWPoisonHandlable()

Quoted 8 lines
Quoted 3 lines

By the way, I will re-include the self test back to this patch series, In case they are not useful, we do not merge it.

Sounds good :)

Can you also test the relevant page types if possible, especially the ones the new helper is supposed to classify?

Ack. I will expand the test to cover different page types as well!

Thanks,
--breno