Skip to contentSkip to conversation

mm: Implement ECC handling for pfn with no struct page

PATCH Superseded [email protected] 3 participants 12 messages Oct 26, 2025 – Nov 4, 2025

Subsystems (11)
Memory management KVM ACPI MM core +7 more subsystems

v4 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

[email protected]

author maintainer
#0 lore raw

v4 cover mm: Implement ECC handling for pfn with no struct page

From: Ankit Agrawal <[email protected]>

Poison (or ECC) errors can be very common on a large size cluster. The kernel MM currently handles ECC errors / poison only on memory page backed by struct page. The handling is currently missing for the PFNMAP memory that does not have struct pages. The series adds such support.

Implement a new ECC handling for memory without struct pages. Kernel MM expose registration APIs to allow modules that are managing the device to register its device memory region. MM then tracks such regions using interval tree.

The mechanism is largely similar to that of ECC on pfn with struct pages. If there is an ECC error on a pfn, all the mapping to it are identified and a SIGBUS is sent to the user space processes owning those mappings. Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map at PMD or PUD level. A poison to a PFN mapped in such as way would need breaking the PMD/PUD mapping into PTEs that will get mirrored into the S2. This can greatly increase the cost of table walks and have a major performance impact.

nvgrace-gpu-vfio-pci module maps the device memory to user VA (Qemu) using remap_pfn_range without being added to the kernel [2]. These device memory PFNs are not backed by struct page. So make nvgrace-gpu-vfio-pci module make use of the mechanism to get poison handling support on the device memory.

Patch rebased to v6.17-rc7.

---

v3 -> v4
- Added guards in memory_failure_pfn, register, unregister function to simplify code. (Thanks Ira Weiny for suggestion).
- Collected reviewed-by from Shuai Xue (Thanks!) on the mm GHES patch. Also moved it to the front of the series.
- Added check for interval_tree_iter_first before removing the device memory region. (Thanks Jiaqi Yan for suggestion)
- If pfn doesn't belong to any address space mapping, returning MF_IGNORED (Thanks Miaohe Lin for suggestion).
- Updated patch commit to add more details on the perf impact on HUGE PFNMAP (Thanks Jason Gunthorpe, Tony Luck for suggestion).

v2 -> v3
- Rebased to v6.17-rc7.
- Skipped the unmapping of PFNMAP during reception of poison. Suggested by Jason Gunthorpe, Jiaqi Yan, Vikram Sethi (Thanks!)
- Updated the check to prevent multiple registration to the same PFN range using interval_tree_iter_first. Thanks Shameer Kolothum for the suggestion.
- Removed the callback function in the nvgrace-gpu requiring tracking of poisoned PFN as it isn't required anymore.
- Introduced seperate collect_procs_pfn function to collect the list of processes mapping to the poisoned PFN.

v1 -> v2
- Change poisoned page tracking from bitmap to hashtable.
- Addressed miscellaneous comments in v1.

Ankit Agrawal (3):
mm: Change ghes code to allow poison of non-struct pfn
mm: handle poisoning of pfn without struct pages vfio/nvgrace-gpu: register device memory for poison handling

 MAINTAINERS                         |   1 +
 drivers/acpi/apei/ghes.c            |   6 --
 drivers/vfio/pci/nvgrace-gpu/main.c |  45 ++++++++-
 include/linux/memory-failure.h      |  17 ++++
 include/linux/mm.h                  |   1 +
 include/ras/ras_event.h             |   1 +
 mm/Kconfig                          |   1 +
 mm/memory-failure.c                 | 146 +++++++++++++++++++++++++++-
 8 files changed, 210 insertions(+), 8 deletions(-)
 create mode 100644 include/linux/memory-failure.h
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v4 1/3 mm: Change ghes code to allow poison of non-struct pfn

From: Ankit Agrawal <[email protected]>

The GHES code allows calling of memory_failure() on the PFNs that pass the pfn_valid() check. This contract is broken for the remapped PFNs which fails the check and ghes_do_memory_failure() returns without triggering memory_failure().

Update code to allow memory_failure() call on PFNs failing pfn_valid().

---
 drivers/acpi/apei/ghes.c | 6 ------
 1 file changed, 6 deletions(-)
drivers/acpi/apei/ghes.c +0 -6
index a0d54993edb3..bc4d0f2b3e9d 100644
@@ -505,12 +505,6 @@ static bool ghes_do_memory_failure(u64 physical_addr, int flags)
 		return false;
 
 	pfn = PHYS_PFN(physical_addr);
-	if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {
-		pr_warn_ratelimited(FW_WARN GHES_PFX
-		"Invalid address in generic error data: %#llx\n",
-		physical_addr);
-		return false;
-	}
 
 	if (flags == MF_ACTION_REQUIRED && current->mm) {
 		twcb = (void *)gen_pool_alloc(ghes_estatus_pool, sizeof(*twcb));
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

PATCH v4 2/3 mm: handle poisoning of pfn without struct pages

In reply to [email protected]

From: Ankit Agrawal <[email protected]>

Poison (or ECC) errors can be very common on a large size cluster. The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region mapped using remap_pfn_range() for example, but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory.

Make kernel MM expose a function to allow modules managing the device memory to register the device memory SPA and the address space associated it. MM maintains this information as an interval tree. On poison, MM can search for the range that the poisoned PFN belong and use the address_space to determine the mapping VMA.

In this implementation, kernel MM follows the following sequence that is largely similar to the memory_failure() handler for struct page backed memory:
1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed.
2. memory_failure_pfn() collects the processes mapped to the PFN.
3. memory_failure_pfn() sends SIGBUS to all the processes mapping the faulty PFN using kill_procs().

Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map at PMD or PUD level. A poison to a PFN mapped in such as way would need breaking the PMD/PUD mapping into PTEs that will get mirrored into the S2. This can greatly increase the cost of table walks and have a major performance impact.

---
 MAINTAINERS                    |   1 +
 include/linux/memory-failure.h |  17 ++++
 include/linux/mm.h             |   1 +
 include/ras/ras_event.h        |   1 +
 mm/Kconfig                     |   1 +
 mm/memory-failure.c            | 146 ++++++++++++++++++++++++++++++++-
 6 files changed, 166 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/memory-failure.h
MAINTAINERS +1 -0
index 520fb4e379a3..463d062d0386 100644
@@ -11359,6 +11359,7 @@ M:	Miaohe Lin <[email protected]>
 R:	Naoya Horiguchi <[email protected]>
 L:	[email protected]
 S:	Maintained
+F:	include/linux/memory-failure.h
 F:	mm/hwpoison-inject.c
 F:	mm/memory-failure.c
 
include/linux/memory-failure.h new file +17 -0
new file mode 100644
index 000000000000..bc326503d2d2
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_MEMORY_FAILURE_H
+#define _LINUX_MEMORY_FAILURE_H
+
+#include <linux/interval_tree.h>
+
+struct pfn_address_space;
+
+struct pfn_address_space {
+	struct interval_tree_node node;
+	struct address_space *mapping;
+};
+
+int register_pfn_address_space(struct pfn_address_space *pfn_space);
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space);
+
+#endif /* _LINUX_MEMORY_FAILURE_H */
include/linux/mm.h +1 -0
index 1ae97a0b8ec7..0ab4ea82ce9e 100644
@@ -4006,6 +4006,7 @@ enum mf_action_page_type {
 	MF_MSG_DAX,
 	MF_MSG_UNSPLIT_THP,
 	MF_MSG_ALREADY_POISONED,
+	MF_MSG_PFN_MAP,
 	MF_MSG_UNKNOWN,
 };
 
include/ras/ras_event.h +1 -0
index c8cd0f00c845..fecfeb7c8be7 100644
@@ -375,6 +375,7 @@ TRACE_EVENT(aer_event,
 	EM ( MF_MSG_DAX, "dax page" )					\
 	EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" )			\
 	EM ( MF_MSG_ALREADY_POISONED, "already poisoned" )		\
+	EM ( MF_MSG_PFN_MAP, "non struct page pfn" )                    \
 	EMe ( MF_MSG_UNKNOWN, "unknown page" )
 
 /*
mm/Kconfig +1 -0
index e443fe8cd6cf..0b07219390b9 100644
@@ -777,6 +777,7 @@ config MEMORY_FAILURE
 	depends on ARCH_SUPPORTS_MEMORY_FAILURE
 	bool "Enable recovery from hardware memory errors"
 	select MEMORY_ISOLATION
+	select INTERVAL_TREE
 	select RAS
 	help
 	  Enables code to recover from some memory failures on systems
mm/memory-failure.c +145 -1

182 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#3 lore raw

Re: [PATCH v4 2/3] mm: handle poisoning of pfn without struct pages

Quoted 36 lines · On Sun, 26 Oct 2025 14:19:18 +0000 <[email protected]> wrote:

From: Ankit Agrawal <[email protected]>

Poison (or ECC) errors can be very common on a large size cluster. The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region mapped using remap_pfn_range() for example, but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory.

Make kernel MM expose a function to allow modules managing the device memory to register the device memory SPA and the address space associated it. MM maintains this information as an interval tree. On poison, MM can search for the range that the poisoned PFN belong and use the address_space to determine the mapping VMA.

In this implementation, kernel MM follows the following sequence that is largely similar to the memory_failure() handler for struct page backed memory:
1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed.
2. memory_failure_pfn() collects the processes mapped to the PFN.
3. memory_failure_pfn() sends SIGBUS to all the processes mapping the faulty PFN using kill_procs().

Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map at PMD or PUD level. A poison to a PFN mapped in such as way would need breaking the PMD/PUD mapping into PTEs that will get mirrored into the S2. This can greatly increase the cost of table walks and have a major performance impact.

...

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2228

Quoted 7 lines
mm/memory-failure.c +4 -0
@@ -2216,6 +2222,136 @@ static void kill_procs_now(struct page *p, unsigned long pfn, int flags,
 	kill_procs(&tokill, true, pfn, flags);
 }
 
+int register_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+	if (!pfn_space)
+		return -EINVAL;

I suggest this be removed - make register_pfn_address_space(NULL) illegal and let the punishment be an oops.

Quoted 25 lines
mm/memory-failure.c +25 -0
+	scoped_guard(mutex, &pfn_space_lock) {
+		if (interval_tree_iter_first(&pfn_space_itree,
+					     pfn_space->node.start,
+					     pfn_space->node.last))
+			return -EBUSY;
+
+		interval_tree_insert(&pfn_space->node, &pfn_space_itree);
+	}
+
+	return 0;
+}
+EXPORT_SYMBOL_GPL(register_pfn_address_space);
+
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+	guard(mutex)(&pfn_space_lock);
+
+	if (pfn_space &&
+	    interval_tree_iter_first(&pfn_space_itree,
+				     pfn_space->node.start,
+				     pfn_space->node.last))
+		interval_tree_remove(&pfn_space->node, &pfn_space_itree);
+}
+EXPORT_SYMBOL_GPL(unregister_pfn_address_space);
+

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2264

Quoted 10 lines
mm/memory-failure.c +10 -0
+static void add_to_kill_pfn(struct task_struct *tsk,
+			    struct vm_area_struct *vma,
+			    struct list_head *to_kill,
+			    unsigned long pfn)
+{
+	struct to_kill *tk;
+
+	tk = kmalloc(sizeof(*tk), GFP_ATOMIC);
+	if (!tk)
+		return;

This is unfortunate. GFP_ATOMIC is unreliable and we silently behave as if it worked OK.

Quoted 26 lines
mm/memory-failure.c +26 -0
+	/* Check for pgoff not backed by struct page */
+	tk->addr = vma_address(vma, pfn, 1);
+	tk->size_shift = PAGE_SHIFT;
+
+	if (tk->addr == -EFAULT)
+		pr_info("Unable to find address %lx in %s\n",
+			pfn, tsk->comm);
+
+	get_task_struct(tsk);
+	tk->tsk = tsk;
+	list_add_tail(&tk->nd, to_kill);
+}
+
+/*
+ * Collect processes when the error hit a PFN not backed by struct page.
+ */
+static void collect_procs_pfn(struct address_space *mapping,
+			      unsigned long pfn, struct list_head *to_kill)
+{
+	struct vm_area_struct *vma;
+	struct task_struct *tsk;
+
+	i_mmap_lock_read(mapping);
+	rcu_read_lock();
+	for_each_process(tsk) {
+		struct task_struct *t = tsk;

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2301

Quoted 10 lines
mm/memory-failure.c +10 -0
+
+		t = task_early_kill(tsk, true);
+		if (!t)
+			continue;
+		vma_interval_tree_foreach(vma, &mapping->i_mmap, pfn, pfn) {
+			if (vma->vm_mm == t->mm)
+				add_to_kill_pfn(t, vma, to_kill, pfn);
+		}
+	}
+	rcu_read_unlock();

We could play games here to make the GFP_ATOMIC allocation unnecessary, but nasty. Allocate the to_kill* outside the rcu_read_lock, pass that pointer into add_to_kill_pfn(). If add_to_kill_pfn()'s kmalloc(GFP_ATOMIC) failed, add_to_kill_pfn() can then consume the caller's to_kill*. Then the caller can drop the lock, allocate a new to_kill* then restart the scan. And teach add_to_kill_pfn() to not re-add tasks which are already on the list. Ugh.

At the very very least we should tell the user that the kernel goofed and that one of their processes won't be getting killed.

Quoted 15 lines
mm/memory-failure.c +15 -0
+	i_mmap_unlock_read(mapping);
+}
+
+/**
+ * memory_failure_pfn - Handle memory failure on a page not backed by
+ *                      struct page.
+ * @pfn: Page Number of the corrupted page
+ * @flags: fine tune action taken
+ *
+ * Return:
+ *   0             - success,
+ *   -EBUSY        - Page PFN does not belong to any address space mapping.
+ */
+static int memory_failure_pfn(unsigned long pfn, int flags)
+{

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2326

Quoted 10 lines
mm/memory-failure.c +10 -0
+	struct interval_tree_node *node;
+	LIST_HEAD(tokill);
+
+	scoped_guard(mutex, &pfn_space_lock) {
+		bool mf_handled = false;
+
+		/*
+		 * Modules registers with MM the address space mapping to the device memory they
+		 * manage. Iterate to identify exactly which address space has mapped to this
+		 * failing PFN.

We're quite lenient about >80 columns nowadays, but overflowing 80 for a block comment is rather needless.

Quoted 27 lines
mm/memory-failure.c +26 -0
+		for (node = interval_tree_iter_first(&pfn_space_itree, pfn, pfn); node;
+		     node = interval_tree_iter_next(node, pfn, pfn)) {
+			struct pfn_address_space *pfn_space =
+				container_of(node, struct pfn_address_space, node);
 
+			collect_procs_pfn(pfn_space->mapping, pfn, &tokill);
+
+			mf_handled = true;
+		}
+
+		if (!mf_handled)
+			return action_result(pfn, MF_MSG_PFN_MAP, MF_IGNORED);
+	}
+
+	/*
+	 * Unlike System-RAM there is no possibility to swap in a different
+	 * physical page at a given virtual address, so all userspace
+	 * consumption of direct PFN memory necessitates SIGBUS (i.e.
+	 * MF_MUST_KILL)
+	 */
+	flags |= MF_ACTION_REQUIRED | MF_MUST_KILL;
+
+	kill_procs(&tokill, true, pfn, flags);
+
+	return action_result(pfn, MF_MSG_PFN_MAP, MF_RECOVERED);
+}
+
author maintainer
#4 lore raw

Re: [PATCH v4 2/3] mm: handle poisoning of pfn without struct pages

Thanks Andrew for the comments.

Quoted 7 lines

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2228

Quoted 4 lines
mm/memory-failure.c +4 -0
+int register_pfn_address_space(struct pfn_address_space *pfn_space)
+{
+     if (!pfn_space)
+             return -EINVAL;

I suggest this be removed - make register_pfn_address_space(NULL) illegal and let the punishment be an oops.

Yes, will remove it.

Quoted 5 lines
Quoted 5 lines
mm/memory-failure.c +5 -0
+static void add_to_kill_pfn(struct task_struct *tsk,
+                         struct vm_area_struct *vma,
+                         struct list_head *to_kill,
+                         unsigned long pfn)
+{
Quoted 8 lines
Quoted 5 lines

+     struct to_kill *tk;
+
+     tk = kmalloc(sizeof(*tk), GFP_ATOMIC);
+     if (!tk)
+             return;

This is unfortunate.  GFP_ATOMIC is unreliable and we silently behave as if it worked OK.

Got it. I'll mark this as a failure case.

Quoted 10 lines

We could play games here to make the GFP_ATOMIC allocation unnecessary, but nasty.  Allocate the to_kill* outside the rcu_read_lock, pass that pointer into add_to_kill_pfn().  If add_to_kill_pfn()'s kmalloc(GFP_ATOMIC) failed, add_to_kill_pfn() can then consume the caller's to_kill*.  Then the caller can drop the lock, allocate a new to_kill* then restart the scan.  And teach add_to_kill_pfn() to not re-add tasks which are already on the list.  Ugh.

At the very very least we should tell the user that the kernel goofed and that one of their processes won't be getting killed.

Thanks for the suggestion. As mentioned above I'll mark the kmalloc allocation error as a failure and can put a log message there.

Quoted 10 lines
Quoted 7 lines

+     scoped_guard(mutex, &pfn_space_lock) {
+             bool mf_handled = false;
+
+             /*
+              * Modules registers with MM the address space mapping to the device memory they +              * manage. Iterate to identify exactly which address space has mapped to this +              * failing PFN.

We're quite lenient about >80 columns nowadays, but overflowing 80 for a block comment is rather needless.

Yes. Since it passed through the strict checkpatch.pl check, I didn't notice. I'll fix it.

reviewer
#5 lore raw

Re: [PATCH v4 2/3] mm: handle poisoning of pfn without struct pages

Quoted 14 lines · Ankit Agrawal
Quoted 5 lines
Quoted 5 lines
mm/memory-failure.c +5 -0
+static void add_to_kill_pfn(struct task_struct *tsk,
+                         struct vm_area_struct *vma,
+                         struct list_head *to_kill,
+                         unsigned long pfn)
+{
Quoted 8 lines
Quoted 5 lines

+     struct to_kill *tk;
+
+     tk = kmalloc(sizeof(*tk), GFP_ATOMIC);
+     if (!tk)
+             return;

This is unfortunate.  GFP_ATOMIC is unreliable and we silently behave as if it worked OK.

Quoted 1 line

Got it. I'll mark this as a failure case.

why do you need to batch all processes and kill them at once? Can you just kill one by one?

Signature · 2 lines
Michal Hocko
SUSE Labs
author maintainer
#6 lore raw

Re: [PATCH v4 2/3] mm: handle poisoning of pfn without struct pages

Quoted 14 lines
Quoted 14 lines
Quoted 13 lines

mm/memory-failure.c static void kill_procs_now(struct page *p, unsigned long pfn, int flags, · line 2264

Quoted 10 lines
mm/memory-failure.c +10 -0
+static void add_to_kill_pfn(struct task_struct *tsk,
+                         struct vm_area_struct *vma,
+                         struct list_head *to_kill,
+                         unsigned long pfn)
+{
+     struct to_kill *tk;
+
+     tk = kmalloc(sizeof(*tk), GFP_ATOMIC);
+     if (!tk)
+             return;

This is unfortunate.  GFP_ATOMIC is unreliable and we silently behave as if it worked OK.

Quoted 4 lines
Quoted 1 line

Got it. I'll mark this as a failure case.

why do you need to batch all processes and kill them at once? Can you just kill one by one?

Hi Michal, I am trying to replicate what is being done today for non-PFNMAP memory failure in __add_to_kill
(https://github.com/torvalds/linux/blob/master/mm/memory-failure.c#L376).
For this series, I am inclined to keep it uniform.

reviewer
#7 lore raw

Re: [PATCH v4 2/3] mm: handle poisoning of pfn without struct pages

Quoted 16 lines · Ankit Agrawal
Quoted 16 lines
Quoted 15 lines
Quoted 5 lines
Quoted 5 lines
mm/memory-failure.c +5 -0
+static void add_to_kill_pfn(struct task_struct *tsk,
+                         struct vm_area_struct *vma,
+                         struct list_head *to_kill,
+                         unsigned long pfn)
+{
Quoted 8 lines
Quoted 5 lines

+     struct to_kill *tk;
+
+     tk = kmalloc(sizeof(*tk), GFP_ATOMIC);
+     if (!tk)
+             return;

This is unfortunate.  GFP_ATOMIC is unreliable and we silently behave as if it worked OK.

Got it. I'll mark this as a failure case.

Quoted 7 lines
Quoted 2 lines

why do you need to batch all processes and kill them at once? Can you just kill one by one?

Hi Michal, I am trying to replicate what is being done today for non-PFNMAP memory failure in __add_to_kill
(https://github.com/torvalds/linux/blob/master/mm/memory-failure.c#L376).
For this series, I am inclined to keep it uniform.

Unless there is a very good reason for this code then I would rather not rely on an atomic allocation. This just makes the behavior hard to predict

Signature · 2 lines
Michal Hocko
SUSE Labs
maintainer
#8 lore raw

Re: [PATCH v4 2/3] mm: handle poisoning of pfn without struct pages

Quoted 8 lines · On Mon, 3 Nov 2025 19:22:09 +0100 Michal Hocko <[email protected]> wrote:
Quoted 4 lines

Hi Michal, I am trying to replicate what is being done today for non-PFNMAP memory failure in __add_to_kill
(https://github.com/torvalds/linux/blob/master/mm/memory-failure.c#L376).
For this series, I am inclined to keep it uniform.

Unless there is a very good reason for this code then I would rather not rely on an atomic allocation. This just makes the behavior hard to predict

I don't think this was addressed in the v5 series.

Yes please, anything we can do to avoid GFP_ATOMIC makes the kernel more reliable.

reviewer
#9 lore raw

Re: [PATCH v4 2/3] mm: handle poisoning of pfn without struct pages

Quoted 7 lines · Andrew Morton
Quoted 5 lines · On Mon, 3 Nov 2025 19:22:09 +0100 Michal Hocko <[email protected]> wrote:
Quoted 4 lines

Hi Michal, I am trying to replicate what is being done today for non-PFNMAP memory failure in __add_to_kill
(https://github.com/torvalds/linux/blob/master/mm/memory-failure.c#L376).
For this series, I am inclined to keep it uniform.

Quoted 8 lines
Quoted 3 lines

Unless there is a very good reason for this code then I would rather not rely on an atomic allocation. This just makes the behavior hard to predict

I don't think this was addressed in the v5 series.

Yes please, anything we can do to avoid GFP_ATOMIC makes the kernel more reliable.

This could be done on top of the series because as such this is not a blocker but it would be really great if we can stop copying a bad code and rather get rid of it also in other poisoning code.

Signature · 2 lines
Michal Hocko
SUSE Labs
author maintainer
#10 lore raw

Re: [PATCH v4 2/3] mm: handle poisoning of pfn without struct pages

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines
Quoted 4 lines

Hi Michal, I am trying to replicate what is being done today for non-PFNMAP memory failure in __add_to_kill
(https://github.com/torvalds/linux/blob/master/mm/memory-failure.c#L376).
For this series, I am inclined to keep it uniform.

Unless there is a very good reason for this code then I would rather not rely on an atomic allocation. This just makes the behavior hard to predict

Quoted 8 lines
Quoted 4 lines

I don't think this was addressed in the v5 series.

Yes please, anything we can do to avoid GFP_ATOMIC makes the kernel more reliable.

This could be done on top of the series because as such this is not a blocker but it would be really great if we can stop copying a bad code and rather get rid of it also in other poisoning code.

Ok sure, I'll create a separate patch to cover that and do the one-by-one process kill. I think separation might also have an advantage to isolate regressions if any during verification.

author maintainer
#11 lore raw

PATCH v4 3/3 vfio/nvgrace-gpu: register device memory for poison handling

In reply to [email protected]

From: Ankit Agrawal <[email protected]>

The nvgrace-gpu-vfio-pci module [1] maps the device memory to the user VA (Qemu) using remap_pfn_range() without adding the memory to the kernel. The device memory pages are not backed by struct page. The previous patch implements the mechanism to handle ECC/poison on memory page without struct page. This new mechanism is being used here.

The module registers its memory region and the address_space with the kernel MM for ECC handling using the register_pfn_address_space() registration API exposed by the kernel.

---
 drivers/vfio/pci/nvgrace-gpu/main.c | 45 ++++++++++++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)
drivers/vfio/pci/nvgrace-gpu/main.c +44 -1
index d95761dcdd58..80b3ed63c682 100644
@@ -8,6 +8,10 @@
 #include <linux/delay.h>
 #include <linux/jiffies.h>
 
+#ifdef CONFIG_MEMORY_FAILURE
+#include <linux/memory-failure.h>
+#endif
+
 /*
  * The device memory usable to the workloads running in the VM is cached
  * and showcased as a 64b device BAR (comprising of BAR4 and BAR5 region)
@@ -47,6 +51,9 @@ struct mem_region {
 		void *memaddr;
 		void __iomem *ioaddr;
 	};                      /* Base virtual address of the region */
+#ifdef CONFIG_MEMORY_FAILURE
+	struct pfn_address_space pfn_address_space;
+#endif
 };
 
 struct nvgrace_gpu_pci_core_device {
@@ -60,6 +67,28 @@ struct nvgrace_gpu_pci_core_device {
 	bool has_mig_hw_bug;
 };
 
+#ifdef CONFIG_MEMORY_FAILURE
+
+static int
+nvgrace_gpu_vfio_pci_register_pfn_range(struct mem_region *region,
+					struct vm_area_struct *vma)
+{
+	unsigned long nr_pages;
+	int ret = 0;
+
+	nr_pages = region->memlength >> PAGE_SHIFT;
+
+	region->pfn_address_space.node.start = vma->vm_pgoff;
+	region->pfn_address_space.node.last = vma->vm_pgoff + nr_pages - 1;
+	region->pfn_address_space.mapping = vma->vm_file->f_mapping;
+
+	ret = register_pfn_address_space(&region->pfn_address_space);
+
+	return ret;
+}
+
+#endif
+
 static void nvgrace_gpu_init_fake_bar_emu_regs(struct vfio_device *core_vdev)
 {
 	struct nvgrace_gpu_pci_core_device *nvdev =
@@ -127,6 +156,13 @@ static void nvgrace_gpu_close_device(struct vfio_device *core_vdev)
 
 	mutex_destroy(&nvdev->remap_lock);
 
+#ifdef CONFIG_MEMORY_FAILURE
+	if (nvdev->resmem.memlength)
+		unregister_pfn_address_space(&nvdev->resmem.pfn_address_space);
+
+	unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
+#endif
+
 	vfio_pci_core_close_device(core_vdev);
 }
 
@@ -202,7 +238,14 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
 
 	vma->vm_pgoff = start_pfn;
 
-	return 0;
+#ifdef CONFIG_MEMORY_FAILURE
+	if (nvdev->resmem.memlength && index == VFIO_PCI_BAR2_REGION_INDEX)
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(&nvdev->resmem, vma);
+	else if (index == VFIO_PCI_BAR4_REGION_INDEX)
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(&nvdev->usemem, vma);
+#endif
+
+	return ret;
 }
 
 static long
Signature · 1 line
2.34.1