Skip to contentSkip to conversation

mm: Implement ECC handling for pfn with no struct page

PATCH Superseded [email protected] 8 participants 21 messages Oct 21, 2025 – Oct 24, 2025

Subsystems (11)
Memory management KVM ACPI MM core +7 more subsystems

v3 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

[email protected]

author maintainer
#0 lore raw

v3 cover mm: Implement ECC handling for pfn with no struct page

From: Ankit Agrawal <[email protected]>

The kernel MM currently handles ECC errors / poison only on memory page backed by struct page. The handling is currently missing for the PFNMAP memory that does not have struct pages. The series adds such support.

Implement a new ECC handling for memory without struct pages. Kernel MM expose registration APIs to allow modules that are managing the device to register its device memory region. MM then tracks such regions using interval tree.

The mechanism is largely similar to that of ECC on pfn with struct pages. If there is an ECC error on a pfn, all the mapping to it are identified and a SIGBUS is sent to the user space processes owning those mappings. Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the poisoned PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map at PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This can have a major performance impact.

nvgrace-gpu-vfio-pci module maps the device memory to user VA (Qemu) using remap_pfn_range without being added to the kernel [2]. These device memory PFNs are not backed by struct page. So make nvgrace-gpu-vfio-pci module make use of the mechanism to get poison handling support on the device memory.

Patch rebased to v6.17-rc7.

---

v2 -> v3
- Rebased to v6.17-rc7.
- Skipped the unmapping of PFNMAP during reception of poison. Suggested by Jason Gunthorpe, Jiaqi Yan, Vikram Sethi (Thanks!)
- Updated the check to prevent multiple registration to the same PFN range using interval_tree_iter_first. Thanks Shameer Kolothum for the suggestion.
- Removed the callback function in the nvgrace-gpu requiring tracking of poisoned PFN as it isn't required anymore.
- Introduced seperate collect_procs_pfn function to collect the list of processes mapping to the poisoned PFN.

v1 -> v2
- Change poisoned page tracking from bitmap to hashtable.
- Addressed miscellaneous comments in v1.

Ankit Agrawal (3):
mm: handle poisoning of pfn without struct pages
mm: Change ghes code to allow poison of non-struct pfn vfio/nvgrace-gpu: register device memory for poison handling

 MAINTAINERS                         |   1 +
 drivers/acpi/apei/ghes.c            |   6 --
 drivers/vfio/pci/nvgrace-gpu/main.c |  45 +++++++++-
 include/linux/memory-failure.h      |  17 ++++
 include/linux/mm.h                  |   1 +
 include/ras/ras_event.h             |   1 +
 mm/Kconfig                          |   1 +
 mm/memory-failure.c                 | 128 +++++++++++++++++++++++++++-
 8 files changed, 192 insertions(+), 8 deletions(-)
 create mode 100644 include/linux/memory-failure.h
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v3 1/3 mm: handle poisoning of pfn without struct pages

From: Ankit Agrawal <[email protected]>

The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region mapped using remap_pfn_range() for example, but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory.

Make kernel MM expose a function to allow modules managing the device memory to register the device memory SPA and the address space associated it. MM maintains this information as an interval tree. On poison, MM can search for the range that the poisoned PFN belong and use the address_space to determine the mapping VMA.

In this implementation, kernel MM follows the following sequence that is largely similar to the memory_failure() handler for struct page backed memory:
1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed.
2. memory_failure_pfn() collects the processes mapped to the PFN.
3. memory_failure_pfn() sends SIGBUS to all the processes mapping the poisoned PFN using kill_procs().

Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map in either PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This will have a major performance impact.

---
 MAINTAINERS                    |   1 +
 include/linux/memory-failure.h |  17 +++++
 include/linux/mm.h             |   1 +
 include/ras/ras_event.h        |   1 +
 mm/Kconfig                     |   1 +
 mm/memory-failure.c            | 128 ++++++++++++++++++++++++++++++++-
 6 files changed, 148 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/memory-failure.h
MAINTAINERS +1 -0
index 520fb4e379a3..463d062d0386 100644
@@ -11359,6 +11359,7 @@ M:	Miaohe Lin <[email protected]>
 R:	Naoya Horiguchi <[email protected]>
 L:	[email protected]
 S:	Maintained
+F:	include/linux/memory-failure.h
 F:	mm/hwpoison-inject.c
 F:	mm/memory-failure.c
 
include/linux/memory-failure.h new file +17 -0
new file mode 100644
index 000000000000..bc326503d2d2
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_MEMORY_FAILURE_H
+#define _LINUX_MEMORY_FAILURE_H
+
+#include <linux/interval_tree.h>
+
+struct pfn_address_space;
+
+struct pfn_address_space {
+	struct interval_tree_node node;
+	struct address_space *mapping;
+};
+
+int register_pfn_address_space(struct pfn_address_space *pfn_space);
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space);
+
+#endif /* _LINUX_MEMORY_FAILURE_H */
include/linux/mm.h +1 -0
index 1ae97a0b8ec7..0ab4ea82ce9e 100644
@@ -4006,6 +4006,7 @@ enum mf_action_page_type {
 	MF_MSG_DAX,
 	MF_MSG_UNSPLIT_THP,
 	MF_MSG_ALREADY_POISONED,
+	MF_MSG_PFN_MAP,
 	MF_MSG_UNKNOWN,
 };
 
include/ras/ras_event.h +1 -0
index c8cd0f00c845..fecfeb7c8be7 100644
@@ -375,6 +375,7 @@ TRACE_EVENT(aer_event,
 	EM ( MF_MSG_DAX, "dax page" )					\
 	EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" )			\
 	EM ( MF_MSG_ALREADY_POISONED, "already poisoned" )		\
+	EM ( MF_MSG_PFN_MAP, "non struct page pfn" )                    \
 	EMe ( MF_MSG_UNKNOWN, "unknown page" )
 
 /*
mm/Kconfig +1 -0
index e443fe8cd6cf..0b07219390b9 100644
@@ -777,6 +777,7 @@ config MEMORY_FAILURE
 	depends on ARCH_SUPPORTS_MEMORY_FAILURE
 	bool "Enable recovery from hardware memory errors"
 	select MEMORY_ISOLATION
+	select INTERVAL_TREE
 	select RAS
 	help
 	  Enables code to recover from some memory failures on systems
mm/memory-failure.c +127 -1

164 lines not on this page · raw message

Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v3 1/3] mm: handle poisoning of pfn without struct pages

Quoted 33 lines · ankita@ wrote:

From: Ankit Agrawal <[email protected]>

The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region mapped using remap_pfn_range() for example, but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory.

Make kernel MM expose a function to allow modules managing the device memory to register the device memory SPA and the address space associated it. MM maintains this information as an interval tree. On poison, MM can search for the range that the poisoned PFN belong and use the address_space to determine the mapping VMA.

In this implementation, kernel MM follows the following sequence that is largely similar to the memory_failure() handler for struct page backed memory:
1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed.
2. memory_failure_pfn() collects the processes mapped to the PFN.
3. memory_failure_pfn() sends SIGBUS to all the processes mapping the poisoned PFN using kill_procs().

Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map in either PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This will have a major performance impact.

[snip]

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2228

Quoted 8 lines
mm/memory-failure.c +4 -0
@@ -2216,6 +2222,121 @@ static void kill_procs_now(struct page *p, unsigned long pfn, int flags,
 	kill_procs(&tokill, true, pfn, flags);
 }
 
+int register_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+	if (!pfn_space)
+		return -EINVAL;

Is there a reason callers may be passing NULL?

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2235

Quoted 7 lines
mm/memory-failure.c +7 -0
+
+	mutex_lock(&pfn_space_lock);
+
+	if (interval_tree_iter_first(&pfn_space_itree,
+				     pfn_space->node.start,
+				     pfn_space->node.last)) {
+		mutex_unlock(&pfn_space_lock);

Register and unregister are good places to use guard().

Quoted 29 lines
mm/memory-failure.c +29 -0
+		return -EBUSY;
+	}
+
+	interval_tree_insert(&pfn_space->node, &pfn_space_itree);
+	mutex_unlock(&pfn_space_lock);
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(register_pfn_address_space);
+
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+	if (!pfn_space)
+		return;
+
+	mutex_lock(&pfn_space_lock);
+	interval_tree_remove(&pfn_space->node, &pfn_space_itree);
+	mutex_unlock(&pfn_space_lock);
+}
+EXPORT_SYMBOL_GPL(unregister_pfn_address_space);
+
+static void add_to_kill_pfn(struct task_struct *tsk,
+			    struct vm_area_struct *vma,
+			    struct list_head *to_kill,
+			    unsigned long pfn)
+{
+	struct to_kill *tk;
+
+	tk = kmalloc(sizeof(*tk), GFP_ATOMIC);

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2274

Quoted 10 lines
mm/memory-failure.c +10 -0
+	if (!tk)
+		return;
+
+	/* Check for pgoff not backed by struct page */
+	tk->addr = vma_address(vma, pfn, 1);
+	tk->size_shift = PAGE_SHIFT;
+
+	if (tk->addr == -EFAULT)
+		pr_info("Unable to find address %lx in %s\n",
+			pfn, tsk->comm);

If the pfn is not in the process why is it added to the kill list?

Quoted 28 lines
mm/memory-failure.c +28 -0
+
+	get_task_struct(tsk);
+	tk->tsk = tsk;
+	list_add_tail(&tk->nd, to_kill);
+}
+
+/*
+ * Collect processes when the error hit a PFN not backed by struct page.
+ */
+static void collect_procs_pfn(struct address_space *mapping,
+			      unsigned long pfn, struct list_head *to_kill)
+{
+	struct vm_area_struct *vma;
+	struct task_struct *tsk;
+
+	i_mmap_lock_read(mapping);
+	rcu_read_lock();
+	for_each_process(tsk) {
+		struct task_struct *t = tsk;
+
+		t = task_early_kill(tsk, true);
+		if (!t)
+			continue;
+		vma_interval_tree_foreach(vma, &mapping->i_mmap, pfn, pfn) {
+			if (vma->vm_mm == t->mm)
+				add_to_kill_pfn(t, vma, to_kill, pfn);
+		}
+	}

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2312

Quoted 10 lines
mm/memory-failure.c +10 -0
+	rcu_read_unlock();
+	i_mmap_unlock_read(mapping);
+}
+
+static int memory_failure_pfn(unsigned long pfn, int flags)
+{
+	struct interval_tree_node *node;
+	LIST_HEAD(tokill);
+
+	mutex_lock(&pfn_space_lock);

scoped_guard() Or probably wrap this part in a guarded function.

Ira

Quoted 46 lines
mm/memory-failure.c +32 -0
+	/*
+	 * Modules registers with MM the address space mapping to the device memory they
+	 * manage. Iterate to identify exactly which address space has mapped to this
+	 * failing PFN.
+	 */
+	for (node = interval_tree_iter_first(&pfn_space_itree, pfn, pfn); node;
+	     node = interval_tree_iter_next(node, pfn, pfn)) {
+		struct pfn_address_space *pfn_space =
+			container_of(node, struct pfn_address_space, node);
+
+		collect_procs_pfn(pfn_space->mapping, pfn, &tokill);
+	}
+	mutex_unlock(&pfn_space_lock);
+
+	/*
+	 * Unlike System-RAM there is no possibility to swap in a different
+	 * physical page at a given virtual address, so all userspace
+	 * consumption of direct PFN memory necessitates SIGBUS (i.e.
+	 * MF_MUST_KILL)
+	 */
+	flags |= MF_ACTION_REQUIRED | MF_MUST_KILL;
+
+	kill_procs(&tokill, true, pfn, flags);
+
+	return action_result(pfn, MF_MSG_PFN_MAP, MF_RECOVERED);
+}
+
 /**
  * memory_failure - Handle memory failure of a page.
  * @pfn: Page Number of the corrupted page
@@ -2259,6 +2380,11 @@ int memory_failure(unsigned long pfn, int flags)
 	if (!(flags & MF_SW_SIMULATED))
 		hw_memory_failure = true;
 
+	if (!pfn_valid(pfn) && !arch_is_platform_page(PFN_PHYS(pfn))) {
+		res = memory_failure_pfn(pfn, flags);
+		goto unlock_mutex;
+	}
+
 	p = pfn_to_online_page(pfn);
 	if (!p) {
 		res = arch_memory_failure(pfn, flags);
Signature · 1 line
2.34.1
#3 lore raw

Re: [PATCH v3 1/3] mm: handle poisoning of pfn without struct pages

In reply to [email protected]
Quoted 178 lines · On Tue, Oct 21, 2025 at 3:23 AM <[email protected]> wrote:

From: Ankit Agrawal <[email protected]>

The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region mapped using remap_pfn_range() for example, but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory.

Make kernel MM expose a function to allow modules managing the device memory to register the device memory SPA and the address space associated it. MM maintains this information as an interval tree. On poison, MM can search for the range that the poisoned PFN belong and use the address_space to determine the mapping VMA.

In this implementation, kernel MM follows the following sequence that is largely similar to the memory_failure() handler for struct page backed memory:
1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed.
2. memory_failure_pfn() collects the processes mapped to the PFN.
3. memory_failure_pfn() sends SIGBUS to all the processes mapping the poisoned PFN using kill_procs().

Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map in either PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This will have a major performance impact.

---
 MAINTAINERS                    |   1 +
 include/linux/memory-failure.h |  17 +++++
 include/linux/mm.h             |   1 +
 include/ras/ras_event.h        |   1 +
 mm/Kconfig                     |   1 +
 mm/memory-failure.c            | 128 ++++++++++++++++++++++++++++++++-
 6 files changed, 148 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/memory-failure.h
MAINTAINERS +1 -0
index 520fb4e379a3..463d062d0386 100644
@@ -11359,6 +11359,7 @@ M:      Miaohe Lin <[email protected]>
 R:     Naoya Horiguchi <[email protected]>
 L:     [email protected]
 S:     Maintained
+F:     include/linux/memory-failure.h
 F:     mm/hwpoison-inject.c
 F:     mm/memory-failure.c
 
include/linux/memory-failure.h new file +17 -0
new file mode 100644
index 000000000000..bc326503d2d2
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_MEMORY_FAILURE_H
+#define _LINUX_MEMORY_FAILURE_H
+
+#include <linux/interval_tree.h>
+
+struct pfn_address_space;
+
+struct pfn_address_space {
+       struct interval_tree_node node;
+       struct address_space *mapping;
+};
+
+int register_pfn_address_space(struct pfn_address_space *pfn_space);
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space);
+
+#endif /* _LINUX_MEMORY_FAILURE_H */
include/linux/mm.h +1 -0
index 1ae97a0b8ec7..0ab4ea82ce9e 100644
@@ -4006,6 +4006,7 @@ enum mf_action_page_type {
        MF_MSG_DAX,
        MF_MSG_UNSPLIT_THP,
        MF_MSG_ALREADY_POISONED,
+       MF_MSG_PFN_MAP,
        MF_MSG_UNKNOWN,
 };
 
include/ras/ras_event.h +1 -0
index c8cd0f00c845..fecfeb7c8be7 100644
@@ -375,6 +375,7 @@ TRACE_EVENT(aer_event,
        EM ( MF_MSG_DAX, "dax page" )                                   \
        EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" )                        \
        EM ( MF_MSG_ALREADY_POISONED, "already poisoned" )              \
+       EM ( MF_MSG_PFN_MAP, "non struct page pfn" )                    \
        EMe ( MF_MSG_UNKNOWN, "unknown page" )
 
 /*
mm/Kconfig +1 -0
index e443fe8cd6cf..0b07219390b9 100644
@@ -777,6 +777,7 @@ config MEMORY_FAILURE
        depends on ARCH_SUPPORTS_MEMORY_FAILURE
        bool "Enable recovery from hardware memory errors"
        select MEMORY_ISOLATION
+       select INTERVAL_TREE
        select RAS
        help
          Enables code to recover from some memory failures on systems
mm/memory-failure.c +25 -1
index df6ee59527dd..acfe5a9bde1d 100644
@@ -38,6 +38,7 @@
 
 #include <linux/kernel.h>
 #include <linux/mm.h>
+#include <linux/memory-failure.h>
 #include <linux/page-flags.h>
 #include <linux/sched/signal.h>
 #include <linux/sched/task.h>
@@ -154,6 +155,10 @@ static const struct ctl_table memory_failure_table[] = {
        }
 };
 
+static struct rb_root_cached pfn_space_itree = RB_ROOT_CACHED;
+
+static DEFINE_MUTEX(pfn_space_lock);
+
 /*
  * Return values:
  *   1:   the page is dissolved (if needed) and taken off from buddy,
@@ -957,6 +962,7 @@ static const char * const action_page_types[] = {
        [MF_MSG_DAX]                    = "dax page",
        [MF_MSG_UNSPLIT_THP]            = "unsplit thp",
        [MF_MSG_ALREADY_POISONED]       = "already poisoned page",
+       [MF_MSG_PFN_MAP]                = "non struct page pfn",
        [MF_MSG_UNKNOWN]                = "unknown page",
 };
 
@@ -1349,7 +1355,7 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type,
 {
        trace_memory_failure_event(pfn, type, result);
 
-       if (type != MF_MSG_ALREADY_POISONED) {
+       if (type != MF_MSG_ALREADY_POISONED && type != MF_MSG_PFN_MAP) {
                num_poisoned_pages_inc(pfn);
                update_per_node_mf_stats(pfn, result);
        }
@@ -2216,6 +2222,121 @@ static void kill_procs_now(struct page *p, unsigned long pfn, int flags,
        kill_procs(&tokill, true, pfn, flags);
 }
 
+int register_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+       if (!pfn_space)
+               return -EINVAL;
+
+       mutex_lock(&pfn_space_lock);
+
+       if (interval_tree_iter_first(&pfn_space_itree,
+                                    pfn_space->node.start,
+                                    pfn_space->node.last)) {
+               mutex_unlock(&pfn_space_lock);
+               return -EBUSY;
+       }
+
+       interval_tree_insert(&pfn_space->node, &pfn_space_itree);
+       mutex_unlock(&pfn_space_lock);
+
+       return 0;

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2252

Quoted 10 lines
mm/memory-failure.c +10 -0
+}
+EXPORT_SYMBOL_GPL(register_pfn_address_space);
+
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+       if (!pfn_space)
+               return;
+
+       mutex_lock(&pfn_space_lock);
+       interval_tree_remove(&pfn_space->node, &pfn_space_itree);

IIRC removing something not in interval tree will panic kernel. If I am not mistaken, should here do something like interval_tree_iter_first before interval_tree_remove, to avoid driver's ill behavior crash the system?

Quoted 106 lines
mm/memory-failure.c +92 -1
+       mutex_unlock(&pfn_space_lock);
+}
+EXPORT_SYMBOL_GPL(unregister_pfn_address_space);
+
+static void add_to_kill_pfn(struct task_struct *tsk,
+                           struct vm_area_struct *vma,
+                           struct list_head *to_kill,
+                           unsigned long pfn)
+{
+       struct to_kill *tk;
+
+       tk = kmalloc(sizeof(*tk), GFP_ATOMIC);
+       if (!tk)
+               return;
+
+       /* Check for pgoff not backed by struct page */
+       tk->addr = vma_address(vma, pfn, 1);
+       tk->size_shift = PAGE_SHIFT;
+
+       if (tk->addr == -EFAULT)
+               pr_info("Unable to find address %lx in %s\n",
+                       pfn, tsk->comm);
+
+       get_task_struct(tsk);
+       tk->tsk = tsk;
+       list_add_tail(&tk->nd, to_kill);
+}
+
+/*
+ * Collect processes when the error hit a PFN not backed by struct page.
+ */
+static void collect_procs_pfn(struct address_space *mapping,
+                             unsigned long pfn, struct list_head *to_kill)
+{
+       struct vm_area_struct *vma;
+       struct task_struct *tsk;
+
+       i_mmap_lock_read(mapping);
+       rcu_read_lock();
+       for_each_process(tsk) {
+               struct task_struct *t = tsk;
+
+               t = task_early_kill(tsk, true);
+               if (!t)
+                       continue;
+               vma_interval_tree_foreach(vma, &mapping->i_mmap, pfn, pfn) {
+                       if (vma->vm_mm == t->mm)
+                               add_to_kill_pfn(t, vma, to_kill, pfn);
+               }
+       }
+       rcu_read_unlock();
+       i_mmap_unlock_read(mapping);
+}
+
+static int memory_failure_pfn(unsigned long pfn, int flags)
+{
+       struct interval_tree_node *node;
+       LIST_HEAD(tokill);
+
+       mutex_lock(&pfn_space_lock);
+       /*
+        * Modules registers with MM the address space mapping to the device memory they
+        * manage. Iterate to identify exactly which address space has mapped to this
+        * failing PFN.
+        */
+       for (node = interval_tree_iter_first(&pfn_space_itree, pfn, pfn); node;
+            node = interval_tree_iter_next(node, pfn, pfn)) {
+               struct pfn_address_space *pfn_space =
+                       container_of(node, struct pfn_address_space, node);
+
+               collect_procs_pfn(pfn_space->mapping, pfn, &tokill);
+       }
+       mutex_unlock(&pfn_space_lock);
+
+       /*
+        * Unlike System-RAM there is no possibility to swap in a different
+        * physical page at a given virtual address, so all userspace
+        * consumption of direct PFN memory necessitates SIGBUS (i.e.
+        * MF_MUST_KILL)
+        */
+       flags |= MF_ACTION_REQUIRED | MF_MUST_KILL;
+
+       kill_procs(&tokill, true, pfn, flags);
+
+       return action_result(pfn, MF_MSG_PFN_MAP, MF_RECOVERED);
+}
+
 /**
  * memory_failure - Handle memory failure of a page.
  * @pfn: Page Number of the corrupted page
@@ -2259,6 +2380,11 @@ int memory_failure(unsigned long pfn, int flags)
        if (!(flags & MF_SW_SIMULATED))
                hw_memory_failure = true;
 
+       if (!pfn_valid(pfn) && !arch_is_platform_page(PFN_PHYS(pfn))) {
+               res = memory_failure_pfn(pfn, flags);
+               goto unlock_mutex;
+       }
+
        p = pfn_to_online_page(pfn);
        if (!p) {
                res = arch_memory_failure(pfn, flags);
--

2.34.1

maintainer
#4 lore raw

Re: [PATCH v3 1/3] mm: handle poisoning of pfn without struct pages

In reply to [email protected]
Quoted 31 lines · On 2025/10/21 18:23, [email protected] wrote:

From: Ankit Agrawal <[email protected]>

The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region mapped using remap_pfn_range() for example, but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory.

Make kernel MM expose a function to allow modules managing the device memory to register the device memory SPA and the address space associated it. MM maintains this information as an interval tree. On poison, MM can search for the range that the poisoned PFN belong and use the address_space to determine the mapping VMA.

In this implementation, kernel MM follows the following sequence that is largely similar to the memory_failure() handler for struct page backed memory:
1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed.
2. memory_failure_pfn() collects the processes mapped to the PFN.
3. memory_failure_pfn() sends SIGBUS to all the processes mapping the poisoned PFN using kill_procs().

Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map in either PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This will have a major performance impact.

Quoted 3 lines

Thanks for your patch. Some comments below.

Quoted 28 lines
---
 MAINTAINERS                    |   1 +
 include/linux/memory-failure.h |  17 +++++
 include/linux/mm.h             |   1 +
 include/ras/ras_event.h        |   1 +
 mm/Kconfig                     |   1 +
 mm/memory-failure.c            | 128 ++++++++++++++++++++++++++++++++-
 6 files changed, 148 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/memory-failure.h
MAINTAINERS +1 -0
index 520fb4e379a3..463d062d0386 100644
@@ -11359,6 +11359,7 @@ M:	Miaohe Lin <[email protected]>
 R:	Naoya Horiguchi <[email protected]>
 L:	[email protected]
 S:	Maintained
+F:	include/linux/memory-failure.h
 F:	mm/hwpoison-inject.c
 F:	mm/memory-failure.c
 
include/linux/memory-failure.h new file +0 -0
new file mode 100644
index 000000000000..bc326503d2d2

include/linux/memory-failure.h line 7

Quoted 7 lines
include/linux/memory-failure.h new file +7 -0
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_MEMORY_FAILURE_H
+#define _LINUX_MEMORY_FAILURE_H
+
+#include <linux/interval_tree.h>
+
+struct pfn_address_space;

Do we need this declaration?

Quoted 6 lines
include/linux/memory-failure.h new file +6 -0
+
+struct pfn_address_space {
+	struct interval_tree_node node;
+	struct address_space *mapping;
+};
+

<snip>

Quoted 20 lines
include/linux/memory-failure.h new file +20 -0
+static int memory_failure_pfn(unsigned long pfn, int flags)
+{
+	struct interval_tree_node *node;
+	LIST_HEAD(tokill);
+
+	mutex_lock(&pfn_space_lock);
+	/*
+	 * Modules registers with MM the address space mapping to the device memory they
+	 * manage. Iterate to identify exactly which address space has mapped to this
+	 * failing PFN.
+	 */
+	for (node = interval_tree_iter_first(&pfn_space_itree, pfn, pfn); node;
+	     node = interval_tree_iter_next(node, pfn, pfn)) {
+		struct pfn_address_space *pfn_space =
+			container_of(node, struct pfn_address_space, node);
+
+		collect_procs_pfn(pfn_space->mapping, pfn, &tokill);
+	}
+	mutex_unlock(&pfn_space_lock);
+

include/linux/memory-failure.h

Quoted 10 lines
include/linux/memory-failure.h new file +10 -0
+	/*
+	 * Unlike System-RAM there is no possibility to swap in a different
+	 * physical page at a given virtual address, so all userspace
+	 * consumption of direct PFN memory necessitates SIGBUS (i.e.
+	 * MF_MUST_KILL)
+	 */
+	flags |= MF_ACTION_REQUIRED | MF_MUST_KILL;
+
+	kill_procs(&tokill, true, pfn, flags);
+

If pfn doesn't belong to any address space mapping, it's still counted as MF_RECOVERED?

include/linux/memory-failure.h int memory_failure(unsigned long pfn, int flags) · line 2383

Quoted 11 lines
include/linux/memory-failure.h new file +4 -0
+	return action_result(pfn, MF_MSG_PFN_MAP, MF_RECOVERED);
+}
+
 /**
  * memory_failure - Handle memory failure of a page.
  * @pfn: Page Number of the corrupted page
@@ -2259,6 +2380,11 @@ int memory_failure(unsigned long pfn, int flags)
 	if (!(flags & MF_SW_SIMULATED))
 		hw_memory_failure = true;
 
+	if (!pfn_valid(pfn) && !arch_is_platform_page(PFN_PHYS(pfn))) {

It's better to have some comments for this case.

include/linux/memory-failure.h int memory_failure(unsigned long pfn, int flags) · line 2390

Quoted 7 lines
include/linux/memory-failure.h new file +4 -0
+		res = memory_failure_pfn(pfn, flags);
+		goto unlock_mutex;
+	}
+
 	p = pfn_to_online_page(pfn);
 	if (!p) {
 		res = arch_memory_failure(pfn, flags);

Can we move above memory_failure_pfn block here? I'm worried that too many scenario branches might lead to confusion.

Thanks.
.

reviewer
#5 lore raw

Re: [PATCH v3 1/3] mm: handle poisoning of pfn without struct pages

In reply to [email protected]
Quoted 14 lines · 在 2025/10/21 18:23, [email protected] 写道:

From: Ankit Agrawal <[email protected]>

The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region mapped using remap_pfn_range() for example, but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory.

Make kernel MM expose a function to allow modules managing the device memory to register the device memory SPA and the address space associated it. MM maintains this information as an interval tree. On poison, MM can search for the range that the poisoned PFN belong and use the address_space to determine the mapping VMA.

Quoted 6 lines

In this implementation, kernel MM follows the following sequence that is largely similar to the memory_failure() handler for struct page backed memory:
1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed.

This step depends on PATCH 2. I suggest reordering the patches so that PATCH 2 comes first, which would make the series easier to review and understand.

Quoted 136 lines

2. memory_failure_pfn() collects the processes mapped to the PFN.
3. memory_failure_pfn() sends SIGBUS to all the processes mapping the poisoned PFN using kill_procs().

Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map in either PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This will have a major performance impact.

---
  MAINTAINERS                    |   1 +
  include/linux/memory-failure.h |  17 +++++
  include/linux/mm.h             |   1 +
  include/ras/ras_event.h        |   1 +
  mm/Kconfig                     |   1 +
  mm/memory-failure.c            | 128 ++++++++++++++++++++++++++++++++-
  6 files changed, 148 insertions(+), 1 deletion(-)

create mode 100644 include/linux/memory-failure.h

MAINTAINERS +1 -0
index 520fb4e379a3..463d062d0386 100644
@@ -11359,6 +11359,7 @@ M:	Miaohe Lin <[email protected]>
  R:	Naoya Horiguchi <[email protected]>
  L:	[email protected]
  S:	Maintained
+F:	include/linux/memory-failure.h
  F:	mm/hwpoison-inject.c
  F:	mm/memory-failure.c
 
include/linux/memory-failure.h new file +17 -0
new file mode 100644
index 000000000000..bc326503d2d2
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_MEMORY_FAILURE_H
+#define _LINUX_MEMORY_FAILURE_H
+
+#include <linux/interval_tree.h>
+
+struct pfn_address_space;
+
+struct pfn_address_space {
+	struct interval_tree_node node;
+	struct address_space *mapping;
+};
+
+int register_pfn_address_space(struct pfn_address_space *pfn_space);
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space);
+
+#endif /* _LINUX_MEMORY_FAILURE_H */
include/linux/mm.h +1 -0
index 1ae97a0b8ec7..0ab4ea82ce9e 100644
@@ -4006,6 +4006,7 @@ enum mf_action_page_type {
  	MF_MSG_DAX,
  	MF_MSG_UNSPLIT_THP,
  	MF_MSG_ALREADY_POISONED,
+	MF_MSG_PFN_MAP,
  	MF_MSG_UNKNOWN,
  };
 
include/ras/ras_event.h +1 -0
index c8cd0f00c845..fecfeb7c8be7 100644
@@ -375,6 +375,7 @@ TRACE_EVENT(aer_event,
  	EM ( MF_MSG_DAX, "dax page" )					\
  	EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" )			\
  	EM ( MF_MSG_ALREADY_POISONED, "already poisoned" )		\
+	EM ( MF_MSG_PFN_MAP, "non struct page pfn" )                    \
  	EMe ( MF_MSG_UNKNOWN, "unknown page" )
 
  /*
mm/Kconfig +1 -0
index e443fe8cd6cf..0b07219390b9 100644
@@ -777,6 +777,7 @@ config MEMORY_FAILURE
  	depends on ARCH_SUPPORTS_MEMORY_FAILURE
  	bool "Enable recovery from hardware memory errors"
  	select MEMORY_ISOLATION
+	select INTERVAL_TREE
  	select RAS
  	help
  	  Enables code to recover from some memory failures on systems
mm/memory-failure.c +7 -1
index df6ee59527dd..acfe5a9bde1d 100644
@@ -38,6 +38,7 @@
 
  #include <linux/kernel.h>
  #include <linux/mm.h>
+#include <linux/memory-failure.h>
  #include <linux/page-flags.h>
  #include <linux/sched/signal.h>
  #include <linux/sched/task.h>
@@ -154,6 +155,10 @@ static const struct ctl_table memory_failure_table[] = {
  	}
  };
 
+static struct rb_root_cached pfn_space_itree = RB_ROOT_CACHED;
+
+static DEFINE_MUTEX(pfn_space_lock);
+
  /*
   * Return values:
   *   1:   the page is dissolved (if needed) and taken off from buddy,
@@ -957,6 +962,7 @@ static const char * const action_page_types[] = {
  	[MF_MSG_DAX]			= "dax page",
  	[MF_MSG_UNSPLIT_THP]		= "unsplit thp",
  	[MF_MSG_ALREADY_POISONED]	= "already poisoned page",
+	[MF_MSG_PFN_MAP]                = "non struct page pfn",
  	[MF_MSG_UNKNOWN]		= "unknown page",
  };
 
@@ -1349,7 +1355,7 @@ static int action_result(unsigned long pfn, enum mf_action_page_type type,
  {
  	trace_memory_failure_event(pfn, type, result);
 
-	if (type != MF_MSG_ALREADY_POISONED) {
+	if (type != MF_MSG_ALREADY_POISONED && type != MF_MSG_PFN_MAP) {
  		num_poisoned_pages_inc(pfn);
  		update_per_node_mf_stats(pfn, result);
  	}

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2225

Quoted 4 lines
mm/memory-failure.c +1 -0
@@ -2216,6 +2222,121 @@ static void kill_procs_now(struct page *p, unsigned long pfn, int flags,
  	kill_procs(&tokill, true, pfn, flags);
  }
 
+int register_pfn_address_space(struct pfn_address_space *pfn_space)

I have a design consideration here. Non-struct page PFNs typically represent device memory managed by device drivers through their own memory allocators. These drivers are responsible for allocation and deallocation of this memory.

Rather than having MM maintain metadata about these PFNs, have you considered adding an operation callback similar to dev_pagemap_ops->memory_failure? This would allow device memory allocators to:

- Maintain their own metadata tracking poison status (similar to TestSetPageHWPoison())
- Handle device-specific requirements for memory failure
- Provide more flexibility for different types of device memory

Thanks.
Shuai

#6 lore raw

Re: [PATCH v3 1/3] mm: handle poisoning of pfn without struct pages

Quoted 3 lines · Shuai Xue

Rather than having MM maintain metadata about these PFNs, have you considered adding an operation callback similar to dev_pagemap_ops->memory_failure?

I think someone could come with such a proposal on top of this, it would not be hard to add some ops to pfn_address_space and have the code call them instead of using the address_space.

This version just needs to link into the existing VMA machinery (ie collect_procs_pfn), it doesn't make alot of sense to push that work into drivers.

Jason

author maintainer
#7 lore raw

Re: [PATCH v3 1/3] mm: handle poisoning of pfn without struct pages

Thank you so much for the feedbacks! Comments inline.

Quoted 6 lines

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2228

Quoted 4 lines
mm/memory-failure.c +4 -0
+int register_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+	if (!pfn_space)
+		return -EINVAL;

Is there a reason callers may be passing NULL?

No, that would be an invalid use case for such.

Quoted 1 line

Register and unregister are good places to use guard().

Thanks for the suggestion Ira. Will update it.

Quoted 1 line

If the pfn is not in the process why is it added to the kill list?

I kept it as it is still a process with VMA mapped to the problematic PFN. This would ultimately result in SIGKILL to be sent to the process. in kill_procs(). This is very similar to the __add_to_kill() implementation.

Quoted 8 lines

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2312

Quoted 6 lines
mm/memory-failure.c +6 -0
+static int memory_failure_pfn(unsigned long pfn, int flags)
+{
+	struct interval_tree_node *node;
+	LIST_HEAD(tokill);
+
+	mutex_lock(&pfn_space_lock);

scoped_guard() Or probably wrap this part in a guarded function.

Ack, thanks.

Quoted 4 lines
Quoted 4 lines
mm/memory-failure.c +4 -0
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+       if (!pfn_space)
+               return;
Quoted 8 lines

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2252

Quoted 3 lines
mm/memory-failure.c +3 -0
+
+       mutex_lock(&pfn_space_lock);
+       interval_tree_remove(&pfn_space->node, &pfn_space_itree);

IIRC removing something not in interval tree will panic kernel. If I am not mistaken, should here do something like interval_tree_iter_first before interval_tree_remove, to avoid driver's ill behavior crash the system?

Thanks Jiaqi for the suggestion. Yeah, I think we should add it. I'll fix that in the next version.

Quoted 2 lines

If pfn doesn't belong to any address space mapping, it's still counted as MF_RECOVERED?

Hi Miaohe, I wasn't sure how should we tag this. It seems you are suggesting MF_FAILED is more appropriate. I'll address that.

Quoted 6 lines

mm/memory-failure.c int memory_failure(unsigned long pfn, int flags) · line 2390

Quoted 3 lines
mm/memory-failure.c +0 -0
 	p = pfn_to_online_page(pfn);
 	if (!p) {
 		res = arch_memory_failure(pfn, flags);

Can we move above memory_failure_pfn block here? I'm worried that too many scenario branches might lead to confusion.

Sure if there isn't any objection, I'll update it.

Quoted 5 lines
Quoted 4 lines · Shuai Xue

Rather than having MM maintain metadata about these PFNs, have you considered adding an operation callback similar to dev_pagemap_ops->memory_failure?

Quoted 7 lines

I think someone could come with such a proposal on top of this, it would not be hard to add some ops to pfn_address_space and have the code call them instead of using the address_space.

This version just needs to link into the existing VMA machinery (ie collect_procs_pfn), it doesn't make alot of sense to push that work into drivers.

Thanks Shuai for the suggestion. However, I agree on this with Jason. It is preferable to keep that as separate.

author maintainer
#8 lore raw

PATCH v3 2/3 mm: Change ghes code to allow poison of non-struct pfn

In reply to [email protected]

From: Ankit Agrawal <[email protected]>

The GHES code allows calling of memory_failure() on the PFNs that pass the pfn_valid() check. This contract is broken for the remapped PFNs which fails the check and ghes_do_memory_failure() returns without triggering memory_failure().

Update code to allow memory_failure() call on PFNs failing pfn_valid().

---
 drivers/acpi/apei/ghes.c | 6 ------
 1 file changed, 6 deletions(-)
drivers/acpi/apei/ghes.c +0 -6
index a0d54993edb3..bc4d0f2b3e9d 100644
@@ -505,12 +505,6 @@ static bool ghes_do_memory_failure(u64 physical_addr, int flags)
 		return false;
 
 	pfn = PHYS_PFN(physical_addr);
-	if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {
-		pr_warn_ratelimited(FW_WARN GHES_PFX
-		"Invalid address in generic error data: %#llx\n",
-		physical_addr);
-		return false;
-	}
 
 	if (flags == MF_ACTION_REQUIRED && current->mm) {
 		twcb = (void *)gen_pool_alloc(ghes_estatus_pool, sizeof(*twcb));
Signature · 1 line
2.34.1
#9 lore raw

Re: [PATCH v3 2/3] mm: Change ghes code to allow poison of non-struct pfn

Quoted 19 lines · ankita@ wrote:

From: Ankit Agrawal <[email protected]>

The GHES code allows calling of memory_failure() on the PFNs that pass the pfn_valid() check. This contract is broken for the remapped PFNs which fails the check and ghes_do_memory_failure() returns without triggering memory_failure().

Update code to allow memory_failure() call on PFNs failing pfn_valid().

---
 drivers/acpi/apei/ghes.c | 6 ------
 1 file changed, 6 deletions(-)
drivers/acpi/apei/ghes.c +0 -0
index a0d54993edb3..bc4d0f2b3e9d 100644

drivers/acpi/apei/ghes.c static bool ghes_do_memory_failure(u64 physical_addr, int flags) · line 508

Quoted 4 lines
drivers/acpi/apei/ghes.c +0 -1
@@ -505,12 +505,6 @@ static bool ghes_do_memory_failure(u64 physical_addr, int flags)
 		return false;
 
 	pfn = PHYS_PFN(physical_addr);
-	if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {

Tony,

I'm not an SGX expert but does this break SGX by removing arch_is_platform_page()?

See:

40e0e7843e23 ("x86/sgx: Add infrastructure to identify SGX EPC pages")
Cc: Tony Luck <[email protected]>

Ira

maintainer
#10 lore raw

RE: [PATCH v3 2/3] mm: Change ghes code to allow poison of non-struct pfn

Quoted 5 lines
Quoted 2 lines
    pfn = PHYS_PFN(physical_addr);

- if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {

Tony,

Quoted 8 lines

I'm not an SGX expert but does this break SGX by removing arch_is_platform_page()?

See:

40e0e7843e23 ("x86/sgx: Add infrastructure to identify SGX EPC pages")
Cc: Tony Luck <[email protected]>

Ira,

I think this deletion makes the GHES code always call memory_failure() instead of bailing out here on "bad" page frame numbers.

That centralizes the checks for different types of memory into memory_failure().

-Tony

reviewer Reviewed-by
#11 lore raw

Re: [PATCH v3 2/3] mm: Change ghes code to allow poison of non-struct pfn

Quoted 15 lines · Tony
Quoted 5 lines
Quoted 2 lines
     pfn = PHYS_PFN(physical_addr);

- if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {

Tony,

Quoted 8 lines

I'm not an SGX expert but does this break SGX by removing arch_is_platform_page()?

See:

40e0e7843e23 ("x86/sgx: Add infrastructure to identify SGX EPC pages")
Cc: Tony Luck <[email protected]>

Ira,

Quoted 7 lines

I think this deletion makes the GHES code always call memory_failure() instead of bailing out here on "bad" page frame numbers.

That centralizes the checks for different types of memory into memory_failure().

-Tony

Hi, Tony, Ankit and Ira,

Finally, we're seeing other use cases that need to handle errors for non-struct page PFNs :)

IMHO, non-struct page PFNs are common in production environments. Besides NVIDIA Grace GPU device memory, we also use reserved DRAM memory managed by a separate VMEM allocator. This VMEM allocator is designed for virtual machine memory allocation, significantly reducing kernel memory management overhead by minimizing page table maintenance.

To enable hardware error isolation for these memory pages, we've already removed this sanity check internally. This change makes memory_failure() the central point for handling all memory types, which is a much cleaner architecture.

Thanks.
Shuai

#12 lore raw

Re: [PATCH v3 2/3] mm: Change ghes code to allow poison of non-struct pfn

Quoted 26 lines · Shuai Xue
Quoted 15 lines · Tony
Quoted 5 lines
Quoted 2 lines
     pfn = PHYS_PFN(physical_addr);

- if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {

Tony,

Quoted 8 lines

I'm not an SGX expert but does this break SGX by removing arch_is_platform_page()?

See:

40e0e7843e23 ("x86/sgx: Add infrastructure to identify SGX EPC pages")
Cc: Tony Luck <[email protected]>

Ira,

Quoted 7 lines

I think this deletion makes the GHES code always call memory_failure() instead of bailing out here on "bad" page frame numbers.

That centralizes the checks for different types of memory into memory_failure().

-Tony

Quoted 8 lines

Hi, Tony, Ankit and Ira,

Finally, we're seeing other use cases that need to handle errors for non-struct page PFNs :)

IMHO, non-struct page PFNs are common in production environments. Besides NVIDIA Grace GPU device memory, we also use reserved DRAM memory managed by a separate VMEM allocator.

Can you elaborate on this more?

Ira

Quoted 14 lines

This VMEM allocator is designed
for virtual machine memory allocation, significantly reducing kernel memory management overhead by minimizing page table maintenance.

To enable hardware error isolation for these memory pages, we've already removed this sanity check internally. This change makes memory_failure() the central point for handling all memory types, which is a much cleaner architecture.

Thanks.
Shuai

reviewer
#13 lore raw

Re: [PATCH v3 2/3] mm: Change ghes code to allow poison of non-struct pfn

Quoted 29 lines · Ira Weiny
Quoted 28 lines · Shuai Xue
Quoted 15 lines · Tony
Quoted 5 lines
Quoted 2 lines
      pfn = PHYS_PFN(physical_addr);

- if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {

Tony,

Quoted 8 lines

I'm not an SGX expert but does this break SGX by removing arch_is_platform_page()?

See:

40e0e7843e23 ("x86/sgx: Add infrastructure to identify SGX EPC pages")
Cc: Tony Luck <[email protected]>

Ira,

Quoted 7 lines

I think this deletion makes the GHES code always call memory_failure() instead of bailing out here on "bad" page frame numbers.

That centralizes the checks for different types of memory into memory_failure().

-Tony

Hi, Tony, Ankit and Ira,

Quoted 8 lines
Quoted 6 lines

Finally, we're seeing other use cases that need to handle errors for non-struct page PFNs :)

IMHO, non-struct page PFNs are common in production environments. Besides NVIDIA Grace GPU device memory, we also use reserved DRAM memory managed by a separate VMEM allocator.

Can you elaborate on this more?

We reserve a significant portion of DRAM memory at boot time using kernel command line parameters. This reserved memory is then managed by our internal VMEM allocator, which handles memory allocation and deallocation for virtual machines.

To minimize memory overhead, we intentionally avoid creating struct pages for this reserved memory region. Instead, we've implemented the following approach:

- Our VMEM allocator directly manages the physical memory without the overhead of struct page metadata.
- Error Handling: We register custom RAS operations (ras_ops) with the memory failure infrastructure. When poisoned memory is accessed within this region, our registered handler: Tags the affected memory area as poisoned Isolates the memory to prevent further access Terminates any tasks that were using the poisoned memory

This approach allows us to handle memory errors effectively while maintaining minimal memory overhead for large reserved regions. It's similar in concept to how device memory (like NVIDIA Grace GPU memory mentioned earlier) needs error handling without struct page backing.

Thanks.
Shuai

author maintainer
#14 lore raw

Re: [PATCH v3 2/3] mm: Change ghes code to allow poison of non-struct pfn

Quoted 6 lines
Quoted 5 lines
Quoted 3 lines

IMHO, non-struct page PFNs are common in production environments. Besides NVIDIA Grace GPU device memory, we also use reserved DRAM memory managed by a separate VMEM allocator.

Can you elaborate on this more?

Quoted 8 lines

We reserve a significant portion of DRAM memory at boot time using kernel command line parameters. This reserved memory is then managed by

our internal VMEM allocator, which handles memory allocation and deallocation for virtual machines.

To minimize memory overhead, we intentionally avoid creating struct pages for this reserved memory region.

Thanks Shuai for the explanation and providing the "Reviewed-by". We use the PFNMAP for the similar reason to save on the memory usage by the struct pages.

Also as mentioned in the 1/3 patch, I'll move this patch to the top.

author maintainer
#15 lore raw

PATCH v3 3/3 vfio/nvgrace-gpu: register device memory for poison handling

In reply to [email protected]

From: Ankit Agrawal <[email protected]>

The nvgrace-gpu-vfio-pci module [1] maps the device memory to the user VA (Qemu) using remap_pfn_range() without adding the memory to the kernel. The device memory pages are not backed by struct page. Patches 1-2 implements the mechanism to handle ECC/poison on memory page without struct page. This new mechanism is being used here.

The module registers its memory region and the address_space with the kernel MM for ECC handling using the register_pfn_address_space() registration API exposed by the kernel.

---
 drivers/vfio/pci/nvgrace-gpu/main.c | 45 ++++++++++++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)
drivers/vfio/pci/nvgrace-gpu/main.c +44 -1
index d95761dcdd58..80b3ed63c682 100644
@@ -8,6 +8,10 @@
 #include <linux/delay.h>
 #include <linux/jiffies.h>
 
+#ifdef CONFIG_MEMORY_FAILURE
+#include <linux/memory-failure.h>
+#endif
+
 /*
  * The device memory usable to the workloads running in the VM is cached
  * and showcased as a 64b device BAR (comprising of BAR4 and BAR5 region)
@@ -47,6 +51,9 @@ struct mem_region {
 		void *memaddr;
 		void __iomem *ioaddr;
 	};                      /* Base virtual address of the region */
+#ifdef CONFIG_MEMORY_FAILURE
+	struct pfn_address_space pfn_address_space;
+#endif
 };
 
 struct nvgrace_gpu_pci_core_device {
@@ -60,6 +67,28 @@ struct nvgrace_gpu_pci_core_device {
 	bool has_mig_hw_bug;
 };
 
+#ifdef CONFIG_MEMORY_FAILURE
+
+static int
+nvgrace_gpu_vfio_pci_register_pfn_range(struct mem_region *region,
+					struct vm_area_struct *vma)
+{
+	unsigned long nr_pages;
+	int ret = 0;
+
+	nr_pages = region->memlength >> PAGE_SHIFT;
+
+	region->pfn_address_space.node.start = vma->vm_pgoff;
+	region->pfn_address_space.node.last = vma->vm_pgoff + nr_pages - 1;
+	region->pfn_address_space.mapping = vma->vm_file->f_mapping;
+
+	ret = register_pfn_address_space(&region->pfn_address_space);
+
+	return ret;
+}
+
+#endif
+
 static void nvgrace_gpu_init_fake_bar_emu_regs(struct vfio_device *core_vdev)
 {
 	struct nvgrace_gpu_pci_core_device *nvdev =
@@ -127,6 +156,13 @@ static void nvgrace_gpu_close_device(struct vfio_device *core_vdev)
 
 	mutex_destroy(&nvdev->remap_lock);
 
+#ifdef CONFIG_MEMORY_FAILURE
+	if (nvdev->resmem.memlength)
+		unregister_pfn_address_space(&nvdev->resmem.pfn_address_space);
+
+	unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
+#endif
+
 	vfio_pci_core_close_device(core_vdev);
 }
 
@@ -202,7 +238,14 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
 
 	vma->vm_pgoff = start_pfn;
 
-	return 0;
+#ifdef CONFIG_MEMORY_FAILURE
+	if (nvdev->resmem.memlength && index == VFIO_PCI_BAR2_REGION_INDEX)
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(&nvdev->resmem, vma);
+	else if (index == VFIO_PCI_BAR4_REGION_INDEX)
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(&nvdev->usemem, vma);
+#endif
+
+	return ret;
 }
 
 static long
Signature · 1 line
2.34.1
#16 lore raw
In reply to [email protected]
Quoted 11 lines · * [email protected] <[email protected]> [251021 06:23]:

From: Ankit Agrawal <[email protected]>

The kernel MM currently handles ECC errors / poison only on memory page backed by struct page. The handling is currently missing for the PFNMAP memory that does not have struct pages. The series adds such support.

Implement a new ECC handling for memory without struct pages. Kernel MM expose registration APIs to allow modules that are managing the device to register its device memory region. MM then tracks such regions using interval tree.

Quoted 9 lines

The mechanism is largely similar to that of ECC on pfn with struct pages. If there is an ECC error on a pfn, all the mapping to it are identified and a SIGBUS is sent to the user space processes owning those mappings. Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the poisoned PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map at PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This can have a major performance impact.

Is the performance impact really a concern in the event of failed memory? Does this happen enough to warrant this special case?

Surely it's not failing hardware that may cause performance impacts, so is this triggered in some other way that I'm missing or a conversation pointer?

Quoted 52 lines

nvgrace-gpu-vfio-pci module maps the device memory to user VA (Qemu) using remap_pfn_range without being added to the kernel [2]. These device memory PFNs are not backed by struct page. So make nvgrace-gpu-vfio-pci module make use of the mechanism to get poison handling support on the device memory.

Patch rebased to v6.17-rc7.

---

v2 -> v3
- Rebased to v6.17-rc7.
- Skipped the unmapping of PFNMAP during reception of poison. Suggested by Jason Gunthorpe, Jiaqi Yan, Vikram Sethi (Thanks!)
- Updated the check to prevent multiple registration to the same PFN range using interval_tree_iter_first. Thanks Shameer Kolothum for the suggestion.
- Removed the callback function in the nvgrace-gpu requiring tracking of poisoned PFN as it isn't required anymore.
- Introduced seperate collect_procs_pfn function to collect the list of processes mapping to the poisoned PFN.

v1 -> v2
- Change poisoned page tracking from bitmap to hashtable.
- Addressed miscellaneous comments in v1.

Ankit Agrawal (3):
mm: handle poisoning of pfn without struct pages
mm: Change ghes code to allow poison of non-struct pfn vfio/nvgrace-gpu: register device memory for poison handling

 MAINTAINERS                         |   1 +
 drivers/acpi/apei/ghes.c            |   6 --
 drivers/vfio/pci/nvgrace-gpu/main.c |  45 +++++++++-
 include/linux/memory-failure.h      |  17 ++++
 include/linux/mm.h                  |   1 +
 include/ras/ras_event.h             |   1 +
 mm/Kconfig                          |   1 +
 mm/memory-failure.c                 | 128 +++++++++++++++++++++++++++-
 8 files changed, 192 insertions(+), 8 deletions(-)
 create mode 100644 include/linux/memory-failure.h
Signature · 1 line
2.34.1
#17 lore raw
Quoted 6 lines · Liam R. Howlett
Quoted 3 lines

enables VM_PFNMAP vmas to map at PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This can have a major performance impact.

Is the performance impact really a concern in the event of failed memory?

Yes, something like the KVM S2 is very sensitive to page size for TLB performace.

Quoted 1 line

Does this happen enough to warrant this special case?

If you have a 100k sized cluster it happens constantly :\

Quoted 3 lines

Surely it's not failing hardware that may cause performance impacts, so is this triggered in some other way that I'm missing or a conversation pointer?

It is the splitting of a pgd/pmd level into PTEs that gets mirrored into the S2 and then greatly increases the cost of table walks inside a guest. The HW caches are sized for 1G S2 PTEs, not 4k.

Jason

#18 lore raw
Quoted 15 lines · Jason Gunthorpe
Quoted 6 lines · Liam R. Howlett
Quoted 3 lines

enables VM_PFNMAP vmas to map at PMD level. Otherwise, a poison to a PFN would need breaking the PMD mapping into PTEs to unmap only the poisoned PFN. This can have a major performance impact.

Is the performance impact really a concern in the event of failed memory?

Yes, something like the KVM S2 is very sensitive to page size for TLB performace.

Quoted 1 line

Does this happen enough to warrant this special case?

If you have a 100k sized cluster it happens constantly :\

Quoted 7 lines
Quoted 3 lines

Surely it's not failing hardware that may cause performance impacts, so is this triggered in some other way that I'm missing or a conversation pointer?

It is the splitting of a pgd/pmd level into PTEs that gets mirrored into the S2 and then greatly increases the cost of table walks inside a guest. The HW caches are sized for 1G S2 PTEs, not 4k.

Ah, I see. Seems like a worthy addition to the commit message? I mean, this is really a choice of throwing away memory for the benefit of tlb performance. Seems like a valid choice in your usecase but less so for the average laptop.

Won't leaving the poisoned memory mapped cause migration issues? Even if the machine is migrated, my understanding is the poison follows through checkpoint restore.

Thanks,
Liam

#19 lore raw
Quoted 4 lines · Liam R. Howlett
Quoted 4 lines
Quoted 3 lines

Surely it's not failing hardware that may cause performance impacts, so is this triggered in some other way that I'm missing or a conversation pointer?

Quoted 8 lines
Quoted 3 lines

It is the splitting of a pgd/pmd level into PTEs that gets mirrored into the S2 and then greatly increases the cost of table walks inside a guest. The HW caches are sized for 1G S2 PTEs, not 4k.

Ah, I see. Seems like a worthy addition to the commit message? I mean, this is really a choice of throwing away memory for the benefit of tlb performance. Seems like a valid choice in your usecase but less so for the average laptop.

No memory is being thrown away, the choice is if the kernel will protect itself from loading via userspace issuing repeated reads to bad memory.

Ankit please include some of these details in the commit message

Quoted 3 lines

Won't leaving the poisoned memory mapped cause migration issues? Even if the machine is migrated, my understanding is the poison follows through checkpoint restore.

The VMM has to keep track of this and not try to read the bad memory during migration.

Jason

author maintainer
#20 lore raw
Quoted 11 lines
Quoted 5 lines

Ah, I see.  Seems like a worthy addition to the commit message?  I mean, this is really a choice of throwing away memory for the benefit of tlb performance.  Seems like a valid choice in your usecase but less so for the average laptop.

No memory is being thrown away, the choice is if the kernel will protect itself from loading via userspace issuing repeated reads to bad memory.

Ankit please include some of these details in the commit message

Thanks Jason, Liam for the suggestion. I'll update the commit message in the next version.