Skip to contentSkip to conversation

mm: Implement ECC handling for pfn with no struct page

PATCH Applied [email protected] 4 participants 13 messages Nov 2, 2025 – Jan 21

Subsystems (11)
Memory management KVM ACPI MM core +7 more subsystems
lore
Share & tools
lore

[email protected]

author maintainer
#0 lore raw

v5 cover mm: Implement ECC handling for pfn with no struct page

From: Ankit Agrawal <[email protected]>

Poison (or ECC) errors can be very common on a large size cluster. The kernel MM currently handles ECC errors / poison only on memory page backed by struct page. The handling is currently missing for the PFNMAP memory that does not have struct pages. The series adds such support.

Implement a new ECC handling for memory without struct pages. Kernel MM expose registration APIs to allow modules that are managing the device to register its device memory region. MM then tracks such regions using interval tree.

The mechanism is largely similar to that of ECC on pfn with struct pages. If there is an ECC error on a pfn, all the mapping to it are identified and a SIGBUS is sent to the user space processes owning those mappings. Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map at PMD or PUD level. A poison to a PFN mapped in such as way would need breaking the PMD/PUD mapping into PTEs that will get mirrored into the S2. This can greatly increase the cost of table walks and have a major performance impact.

nvgrace-gpu-vfio-pci module maps the device memory to user VA (Qemu) using remap_pfn_range without being added to the kernel [2]. These device memory PFNs are not backed by struct page. So make nvgrace-gpu-vfio-pci module make use of the mechanism to get poison handling support on the device memory.

Patch rebased to v6.17-rc7.

---

v4 -> v5
- Removed pfn_space NULL checks. Instead a wrong parameter would cause a panic. (Thanks Andrew Morton for suggestion)
- Log message to mention kmalloc allocation error and the failure to kill a process. (Thanks Andrew Morton)
- Comments with 80 chars.

v3 -> v4
- Added guards in memory_failure_pfn, register, unregister function to simplify code. (Thanks Ira Weiny for suggestion).
- Collected reviewed-by from Shuai Xue (Thanks!) on the mm GHES patch. Also moved it to the front of the series.
- Added check for interval_tree_iter_first before removing the device memory region. (Thanks Jiaqi Yan for suggestion)
- If pfn doesn't belong to any address space mapping, returning MF_IGNORED (Thanks Miaohe Lin for suggestion).
- Updated patch commit to add more details on the perf impact on HUGE PFNMAP (Thanks Jason Gunthorpe, Tony Luck for suggestion).

v2 -> v3
- Rebased to v6.17-rc7.
- Skipped the unmapping of PFNMAP during reception of poison. Suggested by Jason Gunthorpe, Jiaqi Yan, Vikram Sethi (Thanks!)
- Updated the check to prevent multiple registration to the same PFN range using interval_tree_iter_first. Thanks Shameer Kolothum for the suggestion.
- Removed the callback function in the nvgrace-gpu requiring tracking of poisoned PFN as it isn't required anymore.
- Introduced seperate collect_procs_pfn function to collect the list of processes mapping to the poisoned PFN.

v1 -> v2
- Change poisoned page tracking from bitmap to hashtable.
- Addressed miscellaneous comments in v1.

Ankit Agrawal (3):
mm: Change ghes code to allow poison of non-struct pfn
mm: handle poisoning of pfn without struct pages vfio/nvgrace-gpu: register device memory for poison handling

 MAINTAINERS                         |   1 +
 drivers/acpi/apei/ghes.c            |   6 --
 drivers/vfio/pci/nvgrace-gpu/main.c |  45 ++++++++-
 include/linux/memory-failure.h      |  17 ++++
 include/linux/mm.h                  |   1 +
 include/ras/ras_event.h             |   1 +
 mm/Kconfig                          |   1 +
 mm/memory-failure.c                 | 145 +++++++++++++++++++++++++++-
 8 files changed, 209 insertions(+), 8 deletions(-)
 create mode 100644 include/linux/memory-failure.h
Signature · 1 line
2.34.1
author maintainer
#1 lore raw

PATCH v5 1/3 mm: Change ghes code to allow poison of non-struct pfn

From: Ankit Agrawal <[email protected]>

The GHES code allows calling of memory_failure() on the PFNs that pass the pfn_valid() check. This contract is broken for the remapped PFNs which fails the check and ghes_do_memory_failure() returns without triggering memory_failure().

Update code to allow memory_failure() call on PFNs failing pfn_valid().

---
 drivers/acpi/apei/ghes.c | 6 ------
 1 file changed, 6 deletions(-)
drivers/acpi/apei/ghes.c +0 -6
index a0d54993edb3..bc4d0f2b3e9d 100644
@@ -505,12 +505,6 @@ static bool ghes_do_memory_failure(u64 physical_addr, int flags)
 		return false;
 
 	pfn = PHYS_PFN(physical_addr);
-	if (!pfn_valid(pfn) && !arch_is_platform_page(physical_addr)) {
-		pr_warn_ratelimited(FW_WARN GHES_PFX
-		"Invalid address in generic error data: %#llx\n",
-		physical_addr);
-		return false;
-	}
 
 	if (flags == MF_ACTION_REQUIRED && current->mm) {
 		twcb = (void *)gen_pool_alloc(ghes_estatus_pool, sizeof(*twcb));
Signature · 1 line
2.34.1
author maintainer
#2 lore raw

PATCH v5 2/3 mm: handle poisoning of pfn without struct pages

In reply to [email protected]

From: Ankit Agrawal <[email protected]>

Poison (or ECC) errors can be very common on a large size cluster. The kernel MM currently does not handle ECC errors / poison on a memory region that is not backed by struct pages. If a memory region mapped using remap_pfn_range() for example, but not added to the kernel, MM will not have associated struct pages. Add a new mechanism to handle memory failure on such memory.

Make kernel MM expose a function to allow modules managing the device memory to register the device memory SPA and the address space associated it. MM maintains this information as an interval tree. On poison, MM can search for the range that the poisoned PFN belong and use the address_space to determine the mapping VMA.

In this implementation, kernel MM follows the following sequence that is largely similar to the memory_failure() handler for struct page backed memory:
1. memory_failure() is triggered on reception of a poison error. An absence of struct page is detected and consequently memory_failure_pfn() is executed.
2. memory_failure_pfn() collects the processes mapped to the PFN.
3. memory_failure_pfn() sends SIGBUS to all the processes mapping the faulty PFN using kill_procs().

Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map at PMD or PUD level. A poison to a PFN mapped in such as way would need breaking the PMD/PUD mapping into PTEs that will get mirrored into the S2. This can greatly increase the cost of table walks and have a major performance impact.

---
 MAINTAINERS                    |   1 +
 include/linux/memory-failure.h |  17 ++++
 include/linux/mm.h             |   1 +
 include/ras/ras_event.h        |   1 +
 mm/Kconfig                     |   1 +
 mm/memory-failure.c            | 145 ++++++++++++++++++++++++++++++++-
 6 files changed, 165 insertions(+), 1 deletion(-)
 create mode 100644 include/linux/memory-failure.h
MAINTAINERS +1 -0
index 520fb4e379a3..463d062d0386 100644
@@ -11359,6 +11359,7 @@ M:	Miaohe Lin <[email protected]>
 R:	Naoya Horiguchi <[email protected]>
 L:	[email protected]
 S:	Maintained
+F:	include/linux/memory-failure.h
 F:	mm/hwpoison-inject.c
 F:	mm/memory-failure.c
 
include/linux/memory-failure.h new file +17 -0
new file mode 100644
index 000000000000..bc326503d2d2
@@ -0,0 +1,17 @@
+/* SPDX-License-Identifier: GPL-2.0 */
+#ifndef _LINUX_MEMORY_FAILURE_H
+#define _LINUX_MEMORY_FAILURE_H
+
+#include <linux/interval_tree.h>
+
+struct pfn_address_space;
+
+struct pfn_address_space {
+	struct interval_tree_node node;
+	struct address_space *mapping;
+};
+
+int register_pfn_address_space(struct pfn_address_space *pfn_space);
+void unregister_pfn_address_space(struct pfn_address_space *pfn_space);
+
+#endif /* _LINUX_MEMORY_FAILURE_H */
include/linux/mm.h +1 -0
index 1ae97a0b8ec7..0ab4ea82ce9e 100644
@@ -4006,6 +4006,7 @@ enum mf_action_page_type {
 	MF_MSG_DAX,
 	MF_MSG_UNSPLIT_THP,
 	MF_MSG_ALREADY_POISONED,
+	MF_MSG_PFN_MAP,
 	MF_MSG_UNKNOWN,
 };
 
include/ras/ras_event.h +1 -0
index c8cd0f00c845..fecfeb7c8be7 100644
@@ -375,6 +375,7 @@ TRACE_EVENT(aer_event,
 	EM ( MF_MSG_DAX, "dax page" )					\
 	EM ( MF_MSG_UNSPLIT_THP, "unsplit thp" )			\
 	EM ( MF_MSG_ALREADY_POISONED, "already poisoned" )		\
+	EM ( MF_MSG_PFN_MAP, "non struct page pfn" )                    \
 	EMe ( MF_MSG_UNKNOWN, "unknown page" )
 
 /*
mm/Kconfig +1 -0
index e443fe8cd6cf..0b07219390b9 100644
@@ -777,6 +777,7 @@ config MEMORY_FAILURE
 	depends on ARCH_SUPPORTS_MEMORY_FAILURE
 	bool "Enable recovery from hardware memory errors"
 	select MEMORY_ISOLATION
+	select INTERVAL_TREE
 	select RAS
 	help
 	  Enables code to recover from some memory failures on systems
mm/memory-failure.c +144 -1

181 lines not on this page · raw message

Signature · 1 line
2.34.1
maintainer
#3 lore raw

Re: [PATCH v5 2/3] mm: handle poisoning of pfn without struct pages

include/linux/memory-failure.h

Quoted 2 lines · On Sun, 2 Nov 2025 18:44:33 +0000 <[email protected]> wrote:
include/linux/memory-failure.h new file +0 -0
new file mode 100644

OK. I assume there is a bunch of memory-failure material which can now be moved into the new memory-failure.h.

author maintainer
#4 lore raw

PATCH v5 3/3 vfio/nvgrace-gpu: register device memory for poison handling

In reply to [email protected]

From: Ankit Agrawal <[email protected]>

The nvgrace-gpu-vfio-pci module [1] maps the device memory to the user VA (Qemu) using remap_pfn_range() without adding the memory to the kernel. The device memory pages are not backed by struct page. The previous patch implements the mechanism to handle ECC/poison on memory page without struct page. This new mechanism is being used here.

The module registers its memory region and the address_space with the kernel MM for ECC handling using the register_pfn_address_space() registration API exposed by the kernel.

---
 drivers/vfio/pci/nvgrace-gpu/main.c | 45 ++++++++++++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)
drivers/vfio/pci/nvgrace-gpu/main.c +44 -1
index d95761dcdd58..80b3ed63c682 100644
@@ -8,6 +8,10 @@
 #include <linux/delay.h>
 #include <linux/jiffies.h>
 
+#ifdef CONFIG_MEMORY_FAILURE
+#include <linux/memory-failure.h>
+#endif
+
 /*
  * The device memory usable to the workloads running in the VM is cached
  * and showcased as a 64b device BAR (comprising of BAR4 and BAR5 region)
@@ -47,6 +51,9 @@ struct mem_region {
 		void *memaddr;
 		void __iomem *ioaddr;
 	};                      /* Base virtual address of the region */
+#ifdef CONFIG_MEMORY_FAILURE
+	struct pfn_address_space pfn_address_space;
+#endif
 };
 
 struct nvgrace_gpu_pci_core_device {
@@ -60,6 +67,28 @@ struct nvgrace_gpu_pci_core_device {
 	bool has_mig_hw_bug;
 };
 
+#ifdef CONFIG_MEMORY_FAILURE
+
+static int
+nvgrace_gpu_vfio_pci_register_pfn_range(struct mem_region *region,
+					struct vm_area_struct *vma)
+{
+	unsigned long nr_pages;
+	int ret = 0;
+
+	nr_pages = region->memlength >> PAGE_SHIFT;
+
+	region->pfn_address_space.node.start = vma->vm_pgoff;
+	region->pfn_address_space.node.last = vma->vm_pgoff + nr_pages - 1;
+	region->pfn_address_space.mapping = vma->vm_file->f_mapping;
+
+	ret = register_pfn_address_space(&region->pfn_address_space);
+
+	return ret;
+}
+
+#endif
+
 static void nvgrace_gpu_init_fake_bar_emu_regs(struct vfio_device *core_vdev)
 {
 	struct nvgrace_gpu_pci_core_device *nvdev =
@@ -127,6 +156,13 @@ static void nvgrace_gpu_close_device(struct vfio_device *core_vdev)
 
 	mutex_destroy(&nvdev->remap_lock);
 
+#ifdef CONFIG_MEMORY_FAILURE
+	if (nvdev->resmem.memlength)
+		unregister_pfn_address_space(&nvdev->resmem.pfn_address_space);
+
+	unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
+#endif
+
 	vfio_pci_core_close_device(core_vdev);
 }
 
@@ -202,7 +238,14 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
 
 	vma->vm_pgoff = start_pfn;
 
-	return 0;
+#ifdef CONFIG_MEMORY_FAILURE
+	if (nvdev->resmem.memlength && index == VFIO_PCI_BAR2_REGION_INDEX)
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(&nvdev->resmem, vma);
+	else if (index == VFIO_PCI_BAR4_REGION_INDEX)
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(&nvdev->usemem, vma);
+#endif
+
+	return ret;
 }
 
 static long
Signature · 1 line
2.34.1
maintainer Acked-by on v5 3/3
#5 lore raw

Re: [PATCH v5 3/3] vfio/nvgrace-gpu: register device memory for poison handling

Quoted 12 lines · On Sun, 2 Nov 2025 18:44:34 +0000 <[email protected]> wrote:

From: Ankit Agrawal <[email protected]>

The nvgrace-gpu-vfio-pci module [1] maps the device memory to the user VA (Qemu) using remap_pfn_range() without adding the memory to the kernel. The device memory pages are not backed by struct page. The previous patch implements the mechanism to handle ECC/poison on memory page without struct page. This new mechanism is being used here.

The module registers its memory region and the address_space with the kernel MM for ECC handling using the register_pfn_address_space() registration API exposed by the kernel.

Quoted 6 lines
---
 drivers/vfio/pci/nvgrace-gpu/main.c | 45 ++++++++++++++++++++++++++++-
 1 file changed, 44 insertions(+), 1 deletion(-)

LGTM. I see Andrew has already picked this up in mm-new, if he refreshes, here's another ack.

Thanks,
Alex

Quoted 84 lines
drivers/vfio/pci/nvgrace-gpu/main.c +44 -1
index d95761dcdd58..80b3ed63c682 100644
@@ -8,6 +8,10 @@
 #include <linux/delay.h>
 #include <linux/jiffies.h>
 
+#ifdef CONFIG_MEMORY_FAILURE
+#include <linux/memory-failure.h>
+#endif
+
 /*
  * The device memory usable to the workloads running in the VM is cached
  * and showcased as a 64b device BAR (comprising of BAR4 and BAR5 region)
@@ -47,6 +51,9 @@ struct mem_region {
 		void *memaddr;
 		void __iomem *ioaddr;
 	};                      /* Base virtual address of the region */
+#ifdef CONFIG_MEMORY_FAILURE
+	struct pfn_address_space pfn_address_space;
+#endif
 };
 
 struct nvgrace_gpu_pci_core_device {
@@ -60,6 +67,28 @@ struct nvgrace_gpu_pci_core_device {
 	bool has_mig_hw_bug;
 };
 
+#ifdef CONFIG_MEMORY_FAILURE
+
+static int
+nvgrace_gpu_vfio_pci_register_pfn_range(struct mem_region *region,
+					struct vm_area_struct *vma)
+{
+	unsigned long nr_pages;
+	int ret = 0;
+
+	nr_pages = region->memlength >> PAGE_SHIFT;
+
+	region->pfn_address_space.node.start = vma->vm_pgoff;
+	region->pfn_address_space.node.last = vma->vm_pgoff + nr_pages - 1;
+	region->pfn_address_space.mapping = vma->vm_file->f_mapping;
+
+	ret = register_pfn_address_space(&region->pfn_address_space);
+
+	return ret;
+}
+
+#endif
+
 static void nvgrace_gpu_init_fake_bar_emu_regs(struct vfio_device *core_vdev)
 {
 	struct nvgrace_gpu_pci_core_device *nvdev =
@@ -127,6 +156,13 @@ static void nvgrace_gpu_close_device(struct vfio_device *core_vdev)
 
 	mutex_destroy(&nvdev->remap_lock);
 
+#ifdef CONFIG_MEMORY_FAILURE
+	if (nvdev->resmem.memlength)
+		unregister_pfn_address_space(&nvdev->resmem.pfn_address_space);
+
+	unregister_pfn_address_space(&nvdev->usemem.pfn_address_space);
+#endif
+
 	vfio_pci_core_close_device(core_vdev);
 }
 
@@ -202,7 +238,14 @@ static int nvgrace_gpu_mmap(struct vfio_device *core_vdev,
 
 	vma->vm_pgoff = start_pfn;
 
-	return 0;
+#ifdef CONFIG_MEMORY_FAILURE
+	if (nvdev->resmem.memlength && index == VFIO_PCI_BAR2_REGION_INDEX)
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(&nvdev->resmem, vma);
+	else if (index == VFIO_PCI_BAR4_REGION_INDEX)
+		ret = nvgrace_gpu_vfio_pci_register_pfn_range(&nvdev->usemem, vma);
+#endif
+
+	return ret;
 }
 
 static long
maintainer
#6 lore raw
In reply to [email protected]
Quoted 9 lines · On Sun, 2 Nov 2025 18:44:31 +0000 <[email protected]> wrote:

Poison (or ECC) errors can be very common on a large size cluster. The kernel MM currently handles ECC errors / poison only on memory page backed by struct page. The handling is currently missing for the PFNMAP memory that does not have struct pages. The series adds such support.

Implement a new ECC handling for memory without struct pages. Kernel MM expose registration APIs to allow modules that are managing the device to register its device memory region. MM then tracks such regions using interval tree.

Thanks. My knowledge of this material is weaker than usual :( But the series looks good to my eye so I'll toss it into mm.git's mm-new branch for some testing exposure. If that goes OK then I'll later move it into the mm-unstable branch where it will get linux-next esposure. At that point I'll monitor reviewer and tester feedback (please).

author maintainer
#7 lore raw
Quoted 6 lines

Thanks.  My knowledge of this material is weaker than usual :( But the series looks good to my eye so I'll toss it into mm.git's mm-new branch for some testing exposure.  If that goes OK then I'll later move it into the mm-unstable branch where it will get linux-next esposure.  At that point I'll monitor reviewer and tester feedback (please).

Thanks Andrew! Lets see how that goes.

#8 lore raw
In reply to [email protected]
Quoted 56 lines · On Sun, Nov 2, 2025 at 10:45 AM <[email protected]> wrote:

From: Ankit Agrawal <[email protected]>

Poison (or ECC) errors can be very common on a large size cluster. The kernel MM currently handles ECC errors / poison only on memory page backed by struct page. The handling is currently missing for the PFNMAP memory that does not have struct pages. The series adds such support.

Implement a new ECC handling for memory without struct pages. Kernel MM expose registration APIs to allow modules that are managing the device to register its device memory region. MM then tracks such regions using interval tree.

The mechanism is largely similar to that of ECC on pfn with struct pages. If there is an ECC error on a pfn, all the mapping to it are identified and a SIGBUS is sent to the user space processes owning those mappings. Note that there is one primary difference versus the handling of the poison on struct pages, which is to skip unmapping to the faulty PFN. This is done to handle the huge PFNMAP support added recently [1] that enables VM_PFNMAP vmas to map at PMD or PUD level. A poison to a PFN mapped in such as way would need breaking the PMD/PUD mapping into PTEs that will get mirrored into the S2. This can greatly increase the cost of table walks and have a major performance impact.

nvgrace-gpu-vfio-pci module maps the device memory to user VA (Qemu) using remap_pfn_range without being added to the kernel [2]. These device memory PFNs are not backed by struct page. So make nvgrace-gpu-vfio-pci module make use of the mechanism to get poison handling support on the device memory.

Patch rebased to v6.17-rc7.

---

v4 -> v5
- Removed pfn_space NULL checks. Instead a wrong parameter would cause a panic. (Thanks Andrew Morton for suggestion)
- Log message to mention kmalloc allocation error and the failure to kill a process. (Thanks Andrew Morton)
- Comments with 80 chars.

v3 -> v4
- Added guards in memory_failure_pfn, register, unregister function to simplify code. (Thanks Ira Weiny for suggestion).
- Collected reviewed-by from Shuai Xue (Thanks!) on the mm GHES patch. Also moved it to the front of the series.
- Added check for interval_tree_iter_first before removing the device memory region. (Thanks Jiaqi Yan for suggestion)
- If pfn doesn't belong to any address space mapping, returning MF_IGNORED (Thanks Miaohe Lin for suggestion).
- Updated patch commit to add more details on the perf impact on HUGE PFNMAP (Thanks Jason Gunthorpe, Tony Luck for suggestion).

Quoted 9 lines

v2 -> v3
- Rebased to v6.17-rc7.
- Skipped the unmapping of PFNMAP during reception of poison. Suggested by Jason Gunthorpe, Jiaqi Yan, Vikram Sethi (Thanks!)
- Updated the check to prevent multiple registration to the same PFN range using interval_tree_iter_first. Thanks Shameer Kolothum for the suggestion.
- Removed the callback function in the nvgrace-gpu requiring tracking of poisoned PFN as it isn't required anymore.

Hi Ankit,

I get that for nvgrace-gpu driver, you removed pfn_address_space_ops because there is no need to unmap poisoned HBM page.

What about the nvgrace-egm driver? Now that you removed the pfn_address_space_ops callback from pfn_address_space in [1], how can nvgrace-egm driver know the poisoned EGM pages at runtime?

I expect the functionality to return retired pages should also include runtime poisoned pages, which are not in the list queried from egm-retired-pages-data-base during initialization. Or maybe my expection is wrong/obsolete?

[1] https://lore.kernel.org/linux-mm/[email protected]
[2] https://lore.kernel.org/kvm/[email protected]

Quoted 30 lines

- Introduced seperate collect_procs_pfn function to collect the list of processes mapping to the poisoned PFN.

v1 -> v2
- Change poisoned page tracking from bitmap to hashtable.
- Addressed miscellaneous comments in v1.

Ankit Agrawal (3):
mm: Change ghes code to allow poison of non-struct pfn
mm: handle poisoning of pfn without struct pages vfio/nvgrace-gpu: register device memory for poison handling

 MAINTAINERS                         |   1 +
 drivers/acpi/apei/ghes.c            |   6 --
 drivers/vfio/pci/nvgrace-gpu/main.c |  45 ++++++++-
 include/linux/memory-failure.h      |  17 ++++
 include/linux/mm.h                  |   1 +
 include/ras/ras_event.h             |   1 +
 mm/Kconfig                          |   1 +
 mm/memory-failure.c                 | 145 +++++++++++++++++++++++++++-
 8 files changed, 209 insertions(+), 8 deletions(-)
 create mode 100644 include/linux/memory-failure.h

--
2.34.1

author maintainer
#9 lore raw
Quoted 17 lines
Quoted 10 lines

v2 -> v3
- Rebased to v6.17-rc7.
- Skipped the unmapping of PFNMAP during reception of poison. Suggested by Jason Gunthorpe, Jiaqi Yan, Vikram Sethi (Thanks!)
- Updated the check to prevent multiple registration to the same PFN range using interval_tree_iter_first. Thanks Shameer Kolothum for the suggestion.
- Removed the callback function in the nvgrace-gpu requiring tracking of poisoned PFN as it isn't required anymore.

Hi Ankit,

I get that for nvgrace-gpu driver, you removed pfn_address_space_ops because there is no need to unmap poisoned HBM page.

Quoted 8 lines

What about the nvgrace-egm driver? Now that you removed the pfn_address_space_ops callback from pfn_address_space in [1], how can nvgrace-egm driver know the poisoned EGM pages at runtime?

I expect the functionality to return retired pages should also include runtime poisoned pages, which are not in the list queried from egm-retired-pages-data-base during initialization. Or maybe my expection is wrong/obsolete?

Hi Jiaqi, yes the EGM code will include consideration for runtime
poisoned pages as well. It will now instead make use of the
pfn_to_vma_pgoff callback merged through https://github.com/torvalds/linux/commit/e6dbcb7c0e7b508d443a9aa6f77f63a2f83b1ae4

Quoted 2 lines

[1] https://lore.kernel.org/linux-mm/[email protected]
[2] https://lore.kernel.org/kvm/[email protected]

#10 lore raw
Quoted 22 lines · Ankit Agrawal
Quoted 22 lines
Quoted 11 lines

v2 -> v3
- Rebased to v6.17-rc7.
- Skipped the unmapping of PFNMAP during reception of poison. Suggested by Jason Gunthorpe, Jiaqi Yan, Vikram Sethi (Thanks!)
- Updated the check to prevent multiple registration to the same PFN range using interval_tree_iter_first. Thanks Shameer Kolothum for the suggestion.
- Removed the callback function in the nvgrace-gpu requiring tracking of poisoned PFN as it isn't required anymore.

Hi Ankit,

I get that for nvgrace-gpu driver, you removed pfn_address_space_ops because there is no need to unmap poisoned HBM page.

What about the nvgrace-egm driver? Now that you removed the pfn_address_space_ops callback from pfn_address_space in [1], how can nvgrace-egm driver know the poisoned EGM pages at runtime?

Quoted 8 lines
Quoted 4 lines

I expect the functionality to return retired pages should also include runtime poisoned pages, which are not in the list queried from egm-retired-pages-data-base during initialization. Or maybe my expection is wrong/obsolete?

Hi Jiaqi, yes the EGM code will include consideration for runtime
poisoned pages as well. It will now instead make use of the
pfn_to_vma_pgoff callback merged through https://github.com/torvalds/linux/commit/e6dbcb7c0e7b508d443a9aa6f77f63a2f83b1ae4

Thank you! Sorry I wasn't following that thread closely and missed it.

Quoted 4 lines
Quoted 3 lines

[1] https://lore.kernel.org/linux-mm/[email protected]
[2] https://lore.kernel.org/kvm/[email protected]

#11 lore raw
Quoted 29 lines · Jiaqi Yan
Quoted 27 lines · Ankit Agrawal
Quoted 26 lines
Quoted 11 lines

v2 -> v3
- Rebased to v6.17-rc7.
- Skipped the unmapping of PFNMAP during reception of poison. Suggested by Jason Gunthorpe, Jiaqi Yan, Vikram Sethi (Thanks!)
- Updated the check to prevent multiple registration to the same PFN range using interval_tree_iter_first. Thanks Shameer Kolothum for the suggestion.
- Removed the callback function in the nvgrace-gpu requiring tracking of poisoned PFN as it isn't required anymore.

Hi Ankit,

I get that for nvgrace-gpu driver, you removed pfn_address_space_ops because there is no need to unmap poisoned HBM page.

What about the nvgrace-egm driver? Now that you removed the pfn_address_space_ops callback from pfn_address_space in [1], how can nvgrace-egm driver know the poisoned EGM pages at runtime?

I expect the functionality to return retired pages should also include runtime poisoned pages, which are not in the list queried from egm-retired-pages-data-base during initialization. Or maybe my expection is wrong/obsolete?

Quoted 5 lines
Quoted 3 lines

Hi Jiaqi, yes the EGM code will include consideration for runtime
poisoned pages as well. It will now instead make use of the
pfn_to_vma_pgoff callback merged through https://github.com/torvalds/linux/commit/e6dbcb7c0e7b508d443a9aa6f77f63a2f83b1ae4

Thank you! Sorry I wasn't following that thread closely and missed it.

Sorry, one more quesiton. I saw [3] implemented pfn_to_vma_pgoff callback in nvgrace gpu driver. Is EGM driver's callback posted somewhere?

[3] https://lore.kernel.org/all/[email protected]/T/#mcc9ccec90b1ca755ad9af0a821f5ce524fed0ffc

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

[1] https://lore.kernel.org/linux-mm/[email protected]
[2] https://lore.kernel.org/kvm/[email protected]