Introduce nvgrace-egm driver for Extended GPU Memory
cover Introduce nvgrace-egm driver for Extended GPU Memory
Background
==========
NVIDIA Grace-based systems (Grace Hopper, Grace Blackwell Superchip) have a special bios-configurable mode called EGM (Extended GPU Memory) that allows a carveout to be created from the system memory. This carveout is a contiguous region of special range of system DRAM reserved by the UEFI boot firmware.
This range does not appear in the EFI memory map and is therefore invisible to the host kernel. i.e. there are no struct pages for it and it cannot be managed by the normal page allocator. This range is described in ACPI as vendor DSD properties on the GPU PCI device node (see ACPI Design below). It is intended to be used by a VMM to back an entire virtual machine's physical memory. The VMM maps the range into its address space via mmap() on a char device and passes it to KVM as the backing store for guest RAM similar to memfd or hugetlbfs. KVM builds Stage-2 page tables mapping the guest's GPA range linearly to the EGM SPAs i.e. a guest offset X maps to SPA (egm_base + X).
The range is special because the GPU on these systems has direct high-speed access to it that bypasses the host system IOMMU. Normal PCI DMA is gated by the IOMMU and requires an explicit IOMMU mapping. The GPU's path to this memory goes via NVLink/NVSwitch directly to the memory controller and does not involve the host IOMMU. Since KVM maps guest GPA to EGM SPA linearly, the GPU can reach guest memory at those same SPAs without needing any additional host IOMMU mapping. Normal system memory that the kernel sees is protected by the IOMMU in the usual way and this range is deliberately kept separate.
To make this work a driver is needed to parse the ACPI, capture the physical memory range, expose it as a char device and allow it to be mmap'd for KVM. A VMM like qemu can open this char dev and then use it much like a hugetlbfs or memfd to back the VM. Due to the security difference the memory has to be strictly separated from the rest of the hypervisor memory. Exposing the range as a DMABUF for iommufd is planned as future work and is not part of this series.
This hardware feature allows the system memory to be access across nodes on a multi-node system such as Grace Blackwell NVL72. It can also provide throughput improvements because the GPU can access VM system memory via its high-bandwidth fabric rather than going through the IOMMU-gated PCIe path. Moreover skipping IOMMU translations could also improve performance for any use cases targeted towards high performance.
Due to the security difference, the EGM range is strictly isolated from the rest of the hypervisor memory (which is covered through IOMMU).
A driver is needed to:
- Parse ACPI to locate the physical range (base SPA, size, proximity domain)
- Expose the range as a char device (/dev/egmX, X = NUMA proximity domain) that a VMM such as QEMU can open and mmap to back guest RAM
- Zero the region on first open to isolate successive VM uses
- Track and expose retired ECC pages to the VMM via ioctl
- Register the PFN range with the memory_failure infrastructure for runtime uncorrectable error handling
ACPI Design
===========
The EGM region properties are exposed as NVIDIA vendor DSD properties on the GPU's PCI ACPI device node:
nvidia,egm-pxm NUMA proximity domain of the region
nvidia,egm-base-pa Physical base address (SPA) of the
region
nvidia,egm-size Length of the region in bytes
nvidia,egm-retired-pages-data-base Physical address of the firmware-
provided retired ECC pages table
This ACPI construct helps with describing and discovering this physically present memory range that is intentionally absent from the EFI memory map and is tied to a specific device for its access properties.
On a multi-socket system there is one EGM region per socket. Multiple GPUs on the same socket advertise identical region properties (same proximity domain, base address and size). The driver deduplicates by proximity
domain: the first GPU probed for a given domain creates the char device and subsequent GPUs on the same socket are linked to it via sysfs.
Implementation
==============
Patch 1 introduces the driver skeleton: Kconfig/Makefile/MAINTAINERS, the egm device class, the char device minor number range (MAX_EGM_NODES=4 matching the largest 4-socket Grace configuration), ACPI property walking (nvidia,egm-pxm / nvidia,egm-base-pa / nvidia,egm-size), per-region state tracking with PXM-based deduplication, char device creation (/dev/egmX), GPU-to-EGM sysfs symlinks and the egm_size sysfs attribute.
Patch 2 implements the file operations: memory scrubbing on the first open() using memremap()/memset() in 1 GiB chunks with cond_resched() to avoid RCU stalls and mmap() via remap_pfn_range().
Patch 3 adds retired ECC page handling: reads the firmware-provided retired-pages table from the ACPI DSD property nvidia,egm-retired-pages-data-base at module init, stores the page offsets in a per-device xarray and exposes them to userspace via the EGM_RETIRED_PAGES_LIST ioctl (UAPI header include/uapi/linux/egm.h). Each firmware entry covers a 64 KiB retired region. The driver inserts one xarray entry per kernel page so a lookup at the running page size hits the right retired range on both 4K and 64K kernels.
Patch 4 registers the EGM PFN range with the memory_failure infrastructure on the first open() and unregisters it on the last close(). The pfn_to_vma_pgoff callback validates that a reported PFN falls within the EGM region, converts it to an in-region file offset, and inserts it into the retired-pages xarray so runtime uncorrectable errors are surfaced alongside ACPI-reported retired pages via the same ioctl.
Enablement
==========
EGM mode is enabled through a flag in the platform firmware (SBIOS). The size of the host-visible partition is separately configurable; all remaining system DRAM is reserved as the EGM region invisible to the hypervisor.
RFC Posting
===========
Verification
============
Tested on a 2-socket (4 GPU) Grace Blackwell platform by booting a VM with QEMU [1] and kernel branch [2] and confirming EGM capability is visible inside the guest and basic cuda workloads. Note that this has dependency on the generic dmabuf importer/exporter work in progress [3]. Until then, a bypass PFNMAP workaround is used [4] (also part of the kernel branch [2].
Ankit Agrawal (4):
platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM
regions
platform/nvidia: Implement mmap and memory scrubbing for EGM chardev platform/nvidia: Handle retired ECC pages and expose via ioctl platform/nvidia: Register EGM PFNMAP range with memory_failure
.../userspace-api/ioctl/ioctl-number.rst | 1 + MAINTAINERS | 6 + drivers/platform/Kconfig | 1 + drivers/platform/Makefile | 1 + drivers/platform/nvidia/Kconfig | 10 + drivers/platform/nvidia/Makefile | 3 + drivers/platform/nvidia/egm.c | 874 ++++++++++++++++++ include/uapi/linux/egm.h | 28 + 8 files changed, 924 insertions(+) create mode 100644 drivers/platform/nvidia/Kconfig create mode 100644 drivers/platform/nvidia/Makefile create mode 100644 drivers/platform/nvidia/egm.c create mode 100644 include/uapi/linux/egm.h
Signature · 1 line
2.34.1
PATCH 1/4 platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions
The Extended GPU Memory (EGM) feature enables the GPU to access system memory allocations within and across nodes through a high-bandwidth path on Grace-based systems. The GPU can utilize system memory on the same socket, a different socket or a different node in a multi-node setup.
When EGM mode is enabled through SBIOS, host system memory is partitioned into two parts: a Hypervisor region for the host OS and a Hypervisor-Invisible (HI) region for the VM. Only the hypervisor region is part of the host EFI map and visible to the host OS on bootup. The HI partition is interchangeably called the EGM region. Its base SPA and size are exposed through ACPI DSDT properties.
Whilst the EGM region is accessible on the host, it is not added/hot-plugged to the kernel. The HI region is assigned to a VM by mapping the QEMU VMA to the SPA using remap_pfn_range(). So it looks like the following:
|---- Sysmem ----| VM view | | |IPA <-> SPA map | | | |--- HI / EGM ---|-- Host Mem --| Host view
Introduce the nvgrace-egm platform driver module and implement full EGM region discovery, char device exposure and sysfs metadata.
Build infrastructure:
- Kconfig, Makefile entries, MAINTAINERS entry.
- An 'egm' device class and a range of char device minor numbers. MAX_EGM_NODES is set to 4 to matches the largest 4-socket Grace configuration.
Discovery:
- Walk all PCI devices at module init time. For each one advertising "nvidia,egm-pxm", read the EGM region properties (base SPA, size and PXM domain) from the ACPI DSD table via the GPU device handle, using "nvidia,egm-base-pa" and "nvidia,egm-size" properties.
- Maintain a module-level list (egm_chardevs) of discovered regions, de-duplicating by PXM so each socket gets exactly one EGM device regardless of how many GPUs it has (2 GPUs per EGM device in GB200).
Char device creation:
- Each EGM region is exposed as /dev/egmX, where X is the PXM number. Embed struct device and struct cdev inside nvgrace_egm_dev so the device lifecycle follows the standard kernel device model.
- Implement setup_egm_chardev() and del_egm_chardev() for setup and teardown. The devnode callback sets device permissions to 0600.
- Stub file operations (open, release and mmap) are registered.
GPU tracking and sysfs:
- Track which PCI devices are associated with each EGM region in a per-device GPU list (struct gpu_node).
- Expose sysfs symlinks from each EGM device to its associated GPU PCI devices to allow the userspace to determine GPU-to-socket affinity and replicate the host EGM topology in the VM. Helps admin understand GPU association.
- Expose the EGM region size via an egm_size sysfs attribute so QEMU can determine the memory-backend-file size. On a 2-socket Grace Blackwell setup this appears as:
Socket0: /sys/devices/virtual/egm/egm4/egm_size
Socket1: /sys/devices/virtual/egm/egm5/egm_size
Introduce nvgrace_egm_create_pci_egm_devs() and its counterpart nvgrace_egm_destroy_pci_devs() that are called from module init/exit.
--- MAINTAINERS | 5 + drivers/platform/Kconfig | 1 + drivers/platform/Makefile | 1 + drivers/platform/nvidia/Kconfig | 10 + drivers/platform/nvidia/Makefile | 3 + drivers/platform/nvidia/egm.c | 459 +++++++++++++++++++++++++++++++ 6 files changed, 479 insertions(+) create mode 100644 drivers/platform/nvidia/Kconfig create mode 100644 drivers/platform/nvidia/Makefile create mode 100644 drivers/platform/nvidia/egm.c
MAINTAINERS +5 -0
@@ -19344,6 +19344,11 @@ L: [email protected] S: Maintained F: drivers/acpi/apei/ghes-nvidia.c +NVIDIA GRACE EGM PLATFORM DRIVER +M: Ankit Agrawal <[email protected]> +S: Supported +F: drivers/platform/nvidia/ + NVIDIA VRS RTC DRIVER M: Shubhi Garg <[email protected]> L: [email protected]
drivers/platform/Kconfig +1 -0
@@ -22,3 +22,4 @@ source "drivers/platform/arm64/Kconfig" source "drivers/platform/raspberrypi/Kconfig" source "drivers/platform/wmi/Kconfig" +source "drivers/platform/nvidia/Kconfig"
drivers/platform/Makefile +1 -0
@@ -15,3 +15,4 @@ obj-$(CONFIG_SURFACE_PLATFORMS) += surface/ obj-$(CONFIG_ARM64_PLATFORM_DEVICES) += arm64/ obj-$(CONFIG_BCM2835_VCHIQ) += raspberrypi/ obj-$(CONFIG_ACPI_WMI) += wmi/ +obj-$(CONFIG_NVGRACE_EGM) += nvidia/
drivers/platform/nvidia/Kconfig new file +10 -0
@@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0-only +config NVGRACE_EGM + tristate "EGM driver for NVIDIA Grace Hopper and Blackwell Superchip" + depends on ARM64 || (COMPILE_TEST && 64BIT) + help + Extended GPU Memory (EGM) support for the GPU in the NVIDIA Grace + based chips required to avail the CPU memory as additional + cross-node/cross-socket memory for GPU using KVM/qemu. + + If you don't know what to do here, say N.
drivers/platform/nvidia/Makefile new file +3 -0
@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_NVGRACE_EGM) += nvgrace-egm.o +nvgrace-egm-y := egm.o
drivers/platform/nvidia/egm.c new file +459 -0
459 lines not on this page · raw message
Signature · 1 line
2.34.1
Re: [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions
Am 02.07.26 um 21:25 schrieb Ankit Agrawal:
Quoted 32 lines
The Extended GPU Memory (EGM) feature enables the GPU to access system memory allocations within and across nodes through a high-bandwidth path on Grace-based systems. The GPU can utilize system memory on the same socket, a different socket or a different node in a multi-node setup.
When EGM mode is enabled through SBIOS, host system memory is partitioned into two parts: a Hypervisor region for the host OS and a Hypervisor-Invisible (HI) region for the VM. Only the hypervisor region is part of the host EFI map and visible to the host OS on bootup. The HI partition is interchangeably called the EGM region. Its base SPA and size are exposed through ACPI DSDT properties.
Whilst the EGM region is accessible on the host, it is not added/hot-plugged to the kernel. The HI region is assigned to a VM by mapping the QEMU VMA to the SPA using
remap_pfn_range(). So it looks like the following:|---- Sysmem ----| VM view | | |IPA <-> SPA map | | | |--- HI / EGM ---|-- Host Mem --| Host viewIntroduce the nvgrace-egm platform driver module and implement full EGM region discovery, char device exposure and sysfs metadata.
Build infrastructure:
-Kconfig,Makefileentries,MAINTAINERSentry.
- An 'egm' device class and a range of char device minor numbers.MAX_EGM_NODESis set to 4 to matches the largest 4-socket Grace configuration.
Quoted 8 lines
Discovery:
- Walk all PCI devices at module init time. For each one advertising "nvidia,egm-pxm", read the EGM region properties (base SPA, size and PXM domain) from the ACPI DSD table via the GPU device handle, using "nvidia,egm-base-pa" and "nvidia,egm-size" properties.
- Maintain a module-level list (egm_chardevs) of discovered regions, de-duplicating by PXM so each socket gets exactly one EGM device regardless of how many GPUs it has (2 GPUs per EGM device in GB200).
I assume that the PCI device itself is being handled by the GPU driver, right? In this case i suggest that you do the discovery inside the GPU driver during probing, otherwise you might run intro problems once PCI hotplug comes into play.
Thanks,
Armin Wolf
Quoted 567 lines
Char device creation:
- Each EGM region is exposed as/dev/egmX, where X is the PXM number. Embed struct device and struct cdev insidenvgrace_egm_devso the device lifecycle follows the standard kernel device model.
- Implementsetup_egm_chardev()anddel_egm_chardev()for setup and teardown. The devnode callback sets device permissions to 0600.
- Stub file operations (open, release and mmap) are registered.GPU tracking and sysfs:
- Track which PCI devices are associated with each EGM region in a per-device GPU list (structgpu_node).
- Expose sysfs symlinks from each EGM device to its associated GPU PCI devices to allow the userspace to determine GPU-to-socket affinity and replicate the host EGM topology in the VM. Helps admin understand GPU association.
- Expose the EGM region size via anegm_sizesysfs attribute so QEMU can determine the memory-backend-file size. On a 2-socket Grace Blackwell setup this appears as:
Socket0:/sys/devices/virtual/egm/egm4/egm_size
Socket1:/sys/devices/virtual/egm/egm5/egm_sizeIntroduce
nvgrace_egm_create_pci_egm_devs()and its counterpartnvgrace_egm_destroy_pci_devs()that are called from module init/exit.Suggested-by Alex Williamson <[email protected]>Suggested-by Jason Gunthorpe <[email protected]>Suggested-by Aniket Agashe <[email protected]>Suggested-by Matthew R. Ochs <[email protected]>Assisted-by Claude:claude-opus-4.8Signed-off-by Ankit Agrawal <[email protected]>--- MAINTAINERS | 5 + drivers/platform/Kconfig | 1 + drivers/platform/Makefile | 1 + drivers/platform/nvidia/Kconfig | 10 + drivers/platform/nvidia/Makefile | 3 + drivers/platform/nvidia/egm.c | 459 +++++++++++++++++++++++++++++++ 6 files changed, 479 insertions(+)create mode 100644
drivers/platform/nvidia/Kconfigcreate mode 100644drivers/platform/nvidia/Makefilecreate mode 100644drivers/platform/nvidia/egm.cMAINTAINERS +5
-0@@ -19344,6 +19344,11 @@ L: [email protected] S: Maintained F: drivers/acpi/apei/ghes-nvidia.c +NVIDIA GRACE EGM PLATFORM DRIVER +M: Ankit Agrawal <[email protected]> +S: Supported +F: drivers/platform/nvidia/ + NVIDIA VRS RTC DRIVER M: Shubhi Garg <[email protected]> L: [email protected]drivers/platform/Kconfig +1
-0@@ -22,3 +22,4 @@ source "drivers/platform/arm64/Kconfig" source "drivers/platform/raspberrypi/Kconfig" source "drivers/platform/wmi/Kconfig" +source "drivers/platform/nvidia/Kconfig"drivers/platform/Makefile +1
-0@@ -15,3 +15,4 @@ obj-$(CONFIG_SURFACE_PLATFORMS) += surface/ obj-$(CONFIG_ARM64_PLATFORM_DEVICES) += arm64/ obj-$(CONFIG_BCM2835_VCHIQ) += raspberrypi/ obj-$(CONFIG_ACPI_WMI) += wmi/ +obj-$(CONFIG_NVGRACE_EGM) += nvidia/drivers/platform/nvidia/Kconfig new file +10
-0@@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0-only +config NVGRACE_EGM + tristate "EGM driver for NVIDIA Grace Hopper and Blackwell Superchip" + depends on ARM64 || (COMPILE_TEST && 64BIT) + help + Extended GPU Memory (EGM) support for the GPU in the NVIDIA Grace + based chips required to avail the CPU memory as additional + cross-node/cross-socket memory for GPU using KVM/qemu. + + If you don't know what to do here, say N.drivers/platform/nvidia/Makefile new file +3
-0@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_NVGRACE_EGM) += nvgrace-egm.o +nvgrace-egm-y := egm.odrivers/platform/nvidia/egm.c new file +459
-0@@ -0,0 +1,459 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved +*/ + +#include <linux/vfio_pci_core.h> + +#define NVGRACE_EGM_DEV_NAME "egm" +#define MAX_EGM_NODES 4 + +static dev_t dev; +static struct class *class; + +struct gpu_node { +struct list_head list; +struct pci_dev *pdev; +}; + +struct nvgrace_egm_dev { +struct device device; +struct cdev cdev; +phys_addr_t egmphys; +size_t egmlength; +u64 egmpxm; +struct list_head gpus; +}; + +struct nvgrace_egm_dev_entry { +struct list_head list; +struct nvgrace_egm_dev *egm_dev; +}; + +/* +* Track egm device lists. Note that there is one device per socket. +* All the GPUs belonging to the same sockets are associated with +* the EGM device for that socket. +*/ +static LIST_HEAD (egm_chardevs); + +static int nvgrace_egm_open (struct inode *inode,struct file *file) +{ +return 0 ; +} + +static int nvgrace_egm_release (struct inode *inode,struct file *file) +{ +return 0 ; +} + +static int nvgrace_egm_mmap (struct file *file,struct vm_area_struct *vma) +{ +/* +* Mapping the EGM region into userspace is implemented by a later +* patch. Until then refuse the mmap. +*/ +return -EOPNOTSUPP ; +} + +static const struct file_operations file_ops = { + .owner =THIS_MODULE , + .open = nvgrace_egm_open, + .release = nvgrace_egm_release, + .mmap = nvgrace_egm_mmap, +}; + +static int nvgrace_egm_create_gpu_links (struct nvgrace_egm_dev *egm_dev, +struct pci_dev *pdev) +{ +int ret; + + ret =sysfs_create_link (&egm_dev->device.kobj, + &pdev->dev.kobj, +dev_name (&pdev->dev)); + +if (ret && ret != -EEXIST ) +return ret; + +return 0 ; +} + +static void remove_egm_symlinks (struct nvgrace_egm_dev *egm_dev, +struct pci_dev *pdev) +{ +sysfs_remove_link (&egm_dev->device.kobj,dev_name (&pdev->dev)); +} + +static int add_gpu (struct nvgrace_egm_dev *egm_dev,struct pci_dev *pdev) +{ +struct gpu_node *node; + + node =kzalloc_obj (*node,GFP_KERNEL ); +if (!node) +return -ENOMEM ; + +/* +* The pdev is stored for the lifetime of the EGM device but it is +* obtained from a for_each_pci_dev() walk that drops its reference as +* it advances. Take a reference here so the stored pointer stays valid +* even if the GPU is later unbound; remove_gpus() drops it. +*/ + node->pdev =pci_dev_get (pdev); + +list_add_tail (&node->list, &egm_dev->gpus); + +return nvgrace_egm_create_gpu_links (egm_dev, pdev); +} + +static void remove_gpus (struct nvgrace_egm_dev *egm_dev) +{ +struct gpu_node *node, *tmp; + +list_for_each_entry_safe (node, tmp, &egm_dev->gpus, list) { +remove_egm_symlinks (egm_dev, node->pdev); +list_del (&node->list); +pci_dev_put (node->pdev); +kfree (node); + } +} + +static void egm_chardev_release (struct device *dev) +{ + struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device); + +remove_gpus (egm_chardev); +kfree (egm_chardev); +} + +static struct nvgrace_egm_dev *setup_egm_chardev (u64 egmphys,u64 egmlength, +u64 egmpxm) +{ +struct nvgrace_egm_dev *egm_chardev; +unsigned int baseminor =MINOR (dev); +int ret; + +/* +* Only MAX_EGM_NODES minors from baseminor were reserved. Reject a PXM +* outside that window. +*/ +if (egmpxm < baseminor || egmpxm - baseminor >=MAX_EGM_NODES ) { +pr_err ("nvgrace-egm: EGM proximity domain %llu outside reserved minor window [%u, %u)\n" , + egmpxm, baseminor, baseminor +MAX_EGM_NODES ); + goto create_err; + } + + egm_chardev =kzalloc_obj (*egm_chardev,GFP_KERNEL ); +if (!egm_chardev) + goto create_err; + +device_initialize (&egm_chardev->device); + +/* +* Use the proximity domain number as the device minor number. +* So the EGM corresponding to node X would be /dev/egmX. +*/ + egm_chardev->egmphys = egmphys; + egm_chardev->egmlength = egmlength; + egm_chardev->egmpxm = egmpxm; +INIT_LIST_HEAD (&egm_chardev->gpus); + + egm_chardev->device.devt =MKDEV (MAJOR (dev), egm_chardev->egmpxm); + egm_chardev->device.class = class; + egm_chardev->device.release = egm_chardev_release; +cdev_init (&egm_chardev->cdev, &file_ops); + egm_chardev->cdev.owner =THIS_MODULE ; + + ret =dev_set_name (&egm_chardev->device,"egm%llu" , egm_chardev->egmpxm); +if (ret) + goto error_exit; + + ret =cdev_device_add (&egm_chardev->cdev, &egm_chardev->device); +if (ret) + goto error_exit; + +return egm_chardev; + +error_exit: +put_device (&egm_chardev->device); +create_err: +return NULL ; +} + +static void del_egm_chardev (struct nvgrace_egm_dev *egm_chardev) +{ +cdev_device_del (&egm_chardev->cdev, &egm_chardev->device); +put_device (&egm_chardev->device); +} + +static char *egm_devnode (const struct device *device,umode_t *mode) +{ +if (mode) + *mode =0600 ; + +return NULL ; +} + +static ssize_t egm_size_show (struct device *dev,struct device_attribute *attr, +char *buf) +{ +struct nvgrace_egm_dev *egm_dev = + container_of(dev, struct nvgrace_egm_dev, device); + +return sysfs_emit (buf,"0x%zx\n" , egm_dev->egmlength); +} + +static DEVICE_ATTR_RO (egm_size); + +static struct attribute *attrs[] = { + &dev_attr_egm_size.attr, +NULL , +}; + +static const struct attribute_group attr_group = { + .attrs = attrs, +}; + +static const struct attribute_group *attr_groups[] = { + &attr_group, +NULL +}; + +/* +* Determine if the EGM feature is enabled. If disabled, there +* will be no EGM properties populated in the ACPI tables and this +* fetch would fail. +*/ +static int has_egm_property (struct pci_dev *pdev,u64 *pegmpxm) +{ +return device_property_read_u64 (&pdev->dev,"nvidia,egm-pxm" , + pegmpxm); +} + +static int fetch_egm_property (struct pci_dev *pdev,u64 *pegmphys, +u64 *pegmlength) +{ +int ret; + +/* +* The memory information is present in the system ACPI tables as DSD +* properties nvidia,egm-base-pa and nvidia,egm-size. +*/ + ret =device_property_read_u64 (&pdev->dev,"nvidia,egm-size" , + pegmlength); +if (ret) +return ret; + + ret =device_property_read_u64 (&pdev->dev,"nvidia,egm-base-pa" , + pegmphys); + +return ret; +} + +static struct nvgrace_egm_dev *get_egm_dev (u64 egmpxm) +{ +struct nvgrace_egm_dev_entry *egm_entry; + +list_for_each_entry (egm_entry, &egm_chardevs, list) { +/* +* A system could have multiple GPUs associated with an +* EGM region and will have the same set of EGM region +* information. Return the existing EGM device if already +* registered through a different GPU on the same socket. +*/ +if (egm_entry->egm_dev->egmpxm == egmpxm) +return egm_entry->egm_dev; + } + +return NULL ; +} + +static void nvgrace_egm_destroy_pci_devs (void ) +{ +struct nvgrace_egm_dev_entry *entry, *tmp; + +list_for_each_entry_safe (entry, tmp, &egm_chardevs, list) { +list_del (&entry->list); +del_egm_chardev (entry->egm_dev); +kfree (entry); + } +} + +/* +* Walk all PCI devices and for each one that has a companion ACPI object +* advertising EGM properties, create an auxiliary device for that EGM range. +* Multiple GPUs on the same socket share a single EGM region (same proximity +* domain). So de-duplicate by skipping a pxm that is already registered. +*/ +static int nvgrace_egm_create_pci_egm_devs (void ) +{ +struct nvgrace_egm_dev_entry *egm_entry; +struct nvgrace_egm_dev *egm_dev; +struct pci_dev *pdev =NULL ; +int ret; + +for_each_pci_dev (pdev) { +u64 egmphys, egmlength, egmpxm; + +if (has_egm_property (pdev, &egmpxm)) +continue ; + + ret =fetch_egm_property (pdev, &egmphys, &egmlength); +if (ret) +continue ; + + egm_dev =get_egm_dev (egmpxm); +if (egm_dev) { + ret =add_gpu (egm_dev, pdev); +if (ret) + goto free_dev; +continue ; + } + + egm_entry =kzalloc_obj (*egm_entry,GFP_KERNEL ); +if (!egm_entry) { +pci_dev_put (pdev); +return -ENOMEM ; + } + + egm_dev =setup_egm_chardev (egmphys, egmlength, egmpxm); +if (!egm_dev) { +kfree (egm_entry); +pci_dev_put (pdev); +return -EINVAL ; + } + + egm_entry->egm_dev = egm_dev; + + ret =add_gpu (egm_entry->egm_dev, pdev); +if (ret) { +del_egm_chardev (egm_dev); +kfree (egm_entry); + goto free_dev; + } + +list_add_tail (&egm_entry->list, &egm_chardevs); + } + +return 0 ; + +free_dev: +/* +* Reached only via goto from inside the loop where the pdev still +* holds the reference from the last pci_get_device(). So drop it here. +*/ +pci_dev_put (pdev); +nvgrace_egm_destroy_pci_devs (); +return ret; +} + +/* +* The char device minor is the EGM proximity domain number which is sparse +* and need not start at 0. Find the lowest proximity domain among the EGM +* devices so the minor range can be reserved starting there. Returns -ENODEV +* if the system has no EGM devices. +* +* EGM PXMs are assumed contiguous within [lowest, lowest + MAX_EGM_NODES), +* which holds for the supported Grace configurations. +*/ +static int nvgrace_egm_lowest_pxm (u64 *pmin_pxm) +{ +struct pci_dev *pdev =NULL ; +bool found = false; + +for_each_pci_dev (pdev) { +u64 egmphys, egmlength, egmpxm; + +if (has_egm_property (pdev, &egmpxm)) +continue ; + +if (fetch_egm_property (pdev, &egmphys, &egmlength)) +continue ; + +if (!found || egmpxm < *pmin_pxm) { + *pmin_pxm = egmpxm; + found = true; + } + } + +if (!found) +return -ENODEV ; + +/* +* The PXM is used verbatim as the minor (only MINORBITS wide). Validate +* the window would not overflow MINORMASK and be truncated. +*/ +if (*pmin_pxm + (MAX_EGM_NODES -1 ) >MINORMASK ) { +pr_err ("nvgrace-egm: lowest EGM proximity domain %llu\n" +"exceeds the minor range\n" , *pmin_pxm); +return -ERANGE ; + } + +return 0 ; +} + +static int __init nvgrace_egm_init (void ) +{ +u64 min_pxm; +int ret; + +/* +* Each EGM region is exposed as /dev/egmX with the proximity domain X +* as its minor number. Reserve MAX_EGM_NODES minors starting at +* the lowest proximity domain so that every egmX device falls within +* the reserved range. With no EGM devices present, load without +* reserving any resources. +*/ + ret =nvgrace_egm_lowest_pxm (&min_pxm); +if (ret == -ENODEV ) +return 0 ; +else if (ret) +return ret; + + ret =alloc_chrdev_region (&dev, min_pxm,MAX_EGM_NODES , +NVGRACE_EGM_DEV_NAME ); +if (ret <0 ) +return ret; + + class =class_create (NVGRACE_EGM_DEV_NAME ); +if (IS_ERR (class)) { +unregister_chrdev_region (dev,MAX_EGM_NODES ); +return PTR_ERR (class); + } + + class->devnode = egm_devnode; + class->dev_groups = attr_groups; + + ret =nvgrace_egm_create_pci_egm_devs (); +if (ret) + goto cleanup_pci_devs; + +return 0 ; + +cleanup_pci_devs: +nvgrace_egm_destroy_pci_devs (); +class_destroy (class); +unregister_chrdev_region (dev,MAX_EGM_NODES ); + +return ret; +} + +static void __exit nvgrace_egm_cleanup(void) +{ +/* +* When no EGM devices were present, the module loaded inertly and +* reserved nothing. @class stays NULL and there is nothing to undo. +*/ +if (!class) +return ; + +nvgrace_egm_destroy_pci_devs (); +class_destroy (class); +unregister_chrdev_region (dev,MAX_EGM_NODES ); +} + +module_init (nvgrace_egm_init); +module_exit (nvgrace_egm_cleanup); + +MODULE_LICENSE ("GPL" ); +MODULE_AUTHOR ("Ankit Agrawal <[email protected]>" ); +MODULE_DESCRIPTION ("NVGRACE EGM - Module to support Extended GPU Memory on NVIDIA Grace Based systems" );
Re: [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions
Quoted 11 lines · Armin Wolf
Quoted 7 lines
- Walk all PCI devices at module init time. For each one advertising "nvidia,egm-pxm", read the EGM region properties (base SPA, size and PXM domain) from the ACPI DSD table via the GPU device handle, using "nvidia,egm-base-pa" and "nvidia,egm-size" properties.
- Maintain a module-level list (egm_chardevs) of discovered regions, de-duplicating by PXM so each socket gets exactly one EGM device regardless of how many GPUs it has (2 GPUs per EGM device in GB200).I assume that the PCI device itself is being handled by the GPU driver, right? In this case i suggest that you do the discovery inside the GPU driver during probing, otherwise you might run intro problems once PCI hotplug comes into play.
IMHO the ACPI modeling placing the egm-base-pa under the PCI device is not really very good (but we are stuck with it). The EGM is an entirely distinct object from the PCI device. The ACPI is just saying that the PCI device has efficient access to the EGM.
It wouldn't make any sense to involve the GPU driver since the GPU driver doesn't even run on the hypervisor.
Jason
Re: [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions
In reply to Ankit AgrawalQuoted 148 lines · On Thu, 2 Jul 2026 19:25:29 +0000 Ankit Agrawal <[email protected]> wrote:
The Extended GPU Memory (EGM) feature enables the GPU to access system memory allocations within and across nodes through a high-bandwidth path on Grace-based systems. The GPU can utilize system memory on the same socket, a different socket or a different node in a multi-node setup.
When EGM mode is enabled through SBIOS, host system memory is partitioned into two parts: a Hypervisor region for the host OS and a Hypervisor-Invisible (HI) region for the VM. Only the hypervisor region is part of the host EFI map and visible to the host OS on bootup. The HI partition is interchangeably called the EGM region. Its base SPA and size are exposed through ACPI DSDT properties.
Whilst the EGM region is accessible on the host, it is not added/hot-plugged to the kernel. The HI region is assigned to a VM by mapping the QEMU VMA to the SPA using
remap_pfn_range(). So it looks like the following:|---- Sysmem ----| VM view | | |IPA <-> SPA map | | | |--- HI / EGM ---|-- Host Mem --| Host viewIntroduce the nvgrace-egm platform driver module and implement full EGM region discovery, char device exposure and sysfs metadata.
Build infrastructure:
-Kconfig,Makefileentries,MAINTAINERSentry.
- An 'egm' device class and a range of char device minor numbers.MAX_EGM_NODESis set to 4 to matches the largest 4-socket Grace configuration.Discovery:
- Walk all PCI devices at module init time. For each one advertising "nvidia,egm-pxm", read the EGM region properties (base SPA, size and PXM domain) from the ACPI DSD table via the GPU device handle, using "nvidia,egm-base-pa" and "nvidia,egm-size" properties.
- Maintain a module-level list (egm_chardevs) of discovered regions, de-duplicating by PXM so each socket gets exactly one EGM device regardless of how many GPUs it has (2 GPUs per EGM device in GB200).Char device creation:
- Each EGM region is exposed as/dev/egmX, where X is the PXM number. Embed struct device and struct cdev insidenvgrace_egm_devso the device lifecycle follows the standard kernel device model.
- Implementsetup_egm_chardev()anddel_egm_chardev()for setup and teardown. The devnode callback sets device permissions to 0600.
- Stub file operations (open, release and mmap) are registered.GPU tracking and sysfs:
- Track which PCI devices are associated with each EGM region in a per-device GPU list (structgpu_node).
- Expose sysfs symlinks from each EGM device to its associated GPU PCI devices to allow the userspace to determine GPU-to-socket affinity and replicate the host EGM topology in the VM. Helps admin understand GPU association.
- Expose the EGM region size via anegm_sizesysfs attribute so QEMU can determine the memory-backend-file size. On a 2-socket Grace Blackwell setup this appears as:
Socket0:/sys/devices/virtual/egm/egm4/egm_size
Socket1:/sys/devices/virtual/egm/egm5/egm_sizeIntroduce
nvgrace_egm_create_pci_egm_devs()and its counterpartnvgrace_egm_destroy_pci_devs()that are called from module init/exit.Suggested-by Alex Williamson <[email protected]>Suggested-by Jason Gunthorpe <[email protected]>Suggested-by Aniket Agashe <[email protected]>Suggested-by Matthew R. Ochs <[email protected]>Assisted-by Claude:claude-opus-4.8Signed-off-by Ankit Agrawal <[email protected]>--- MAINTAINERS | 5 + drivers/platform/Kconfig | 1 + drivers/platform/Makefile | 1 + drivers/platform/nvidia/Kconfig | 10 + drivers/platform/nvidia/Makefile | 3 + drivers/platform/nvidia/egm.c | 459 +++++++++++++++++++++++++++++++ 6 files changed, 479 insertions(+) create mode 100644 drivers/platform/nvidia/Kconfig create mode 100644 drivers/platform/nvidia/Makefile create mode 100644 drivers/platform/nvidia/egm.cMAINTAINERS +5
-0@@ -19344,6 +19344,11 @@ L: [email protected] S: Maintained F: drivers/acpi/apei/ghes-nvidia.c +NVIDIA GRACE EGM PLATFORM DRIVER +M: Ankit Agrawal <[email protected]> +S: Supported +F: drivers/platform/nvidia/ + NVIDIA VRS RTC DRIVER M: Shubhi Garg <[email protected]> L: [email protected]drivers/platform/Kconfig +1
-0@@ -22,3 +22,4 @@ source "drivers/platform/arm64/Kconfig" source "drivers/platform/raspberrypi/Kconfig" source "drivers/platform/wmi/Kconfig" +source "drivers/platform/nvidia/Kconfig"drivers/platform/Makefile +1
-0@@ -15,3 +15,4 @@ obj-$(CONFIG_SURFACE_PLATFORMS) += surface/ obj-$(CONFIG_ARM64_PLATFORM_DEVICES) += arm64/ obj-$(CONFIG_BCM2835_VCHIQ) += raspberrypi/ obj-$(CONFIG_ACPI_WMI) += wmi/ +obj-$(CONFIG_NVGRACE_EGM) += nvidia/drivers/platform/nvidia/Kconfig new file +10
-0@@ -0,0 +1,10 @@ +# SPDX-License-Identifier: GPL-2.0-only +config NVGRACE_EGM + tristate "EGM driver for NVIDIA Grace Hopper and Blackwell Superchip" + depends on ARM64 || (COMPILE_TEST && 64BIT) + help + Extended GPU Memory (EGM) support for the GPU in the NVIDIA Grace + based chips required to avail the CPU memory as additional + cross-node/cross-socket memory for GPU using KVM/qemu. + + If you don't know what to do here, say N.drivers/platform/nvidia/Makefile new file +3
-0@@ -0,0 +1,3 @@ +# SPDX-License-Identifier: GPL-2.0-only +obj-$(CONFIG_NVGRACE_EGM) += nvgrace-egm.o +nvgrace-egm-y := egm.odrivers/platform/nvidia/egm.c new file +0
-0
drivers/platform/nvidia/egm.c line 6
Quoted 6 lines
drivers/platform/nvidia/egm.c new file +6
-0@@ -0,0 +1,459 @@ +// SPDX-License-Identifier: GPL-2.0-only +/* +* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved +*/ + +#include <linux/vfio_pci_core.h>
Bogus include.
drivers/platform/nvidia/egm.c line 12
Quoted 6 lines
drivers/platform/nvidia/egm.c new file +6
-0+ +#define NVGRACE_EGM_DEV_NAME "egm" +#define MAX_EGM_NODES 4 + +static dev_t dev; +static struct class *class;
This should move to using class_register() rather than class_create() as the latter is being phased out[1]. This then becomes a static const struct class:
static const struct class egm_class = {
.name = NVGRACE_EGM_DEV_NAME,
.devnode = egm_devnode,
.dev_groups = attr_groups,
}; [1]https://lore.kernel.org/all/2023040244-duffel-pushpin-f738@gregkh/
Quoted 4 lines
drivers/platform/nvidia/egm.c new file +4
-0+ +struct gpu_node { +struct list_head list; +struct pci_dev *pdev;
drivers/platform/nvidia/egm.c line 26
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+}; + +struct nvgrace_egm_dev { +struct device device; +struct cdev cdev; +phys_addr_t egmphys; +size_t egmlength; +u64 egmpxm; +struct list_head gpus; +};
I can't spot any consistency in when nvgrace is prefixed to structures and function and when it's not.
Quoted 40 lines
drivers/platform/nvidia/egm.c new file +40
-0+ +struct nvgrace_egm_dev_entry { +struct list_head list; +struct nvgrace_egm_dev *egm_dev; +}; + +/* +* Track egm device lists. Note that there is one device per socket. +* All the GPUs belonging to the same sockets are associated with +* the EGM device for that socket. +*/ +static LIST_HEAD (egm_chardevs); + +static int nvgrace_egm_open (struct inode *inode,struct file *file) +{ +return 0 ; +} + +static int nvgrace_egm_release (struct inode *inode,struct file *file) +{ +return 0 ; +} + +static int nvgrace_egm_mmap (struct file *file,struct vm_area_struct *vma) +{ +/* +* Mapping the EGM region into userspace is implemented by a later +* patch. Until then refuse the mmap. +*/ +return -EOPNOTSUPP ; +} + +static const struct file_operations file_ops = { + .owner =THIS_MODULE , + .open = nvgrace_egm_open, + .release = nvgrace_egm_release, + .mmap = nvgrace_egm_mmap, +}; + +static int nvgrace_egm_create_gpu_links(struct nvgrace_egm_dev *egm_dev,
drivers/platform/nvidia/egm.c line 76
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+ struct pci_dev *pdev) +{ +int ret; + + ret =sysfs_create_link (&egm_dev->device.kobj, + &pdev->dev.kobj, +dev_name (&pdev->dev)); + +if (ret && ret != -EEXIST ) +return ret;
AIUI, there are 2 GPUs per egm, and we're creating symlinks from the single egm object to the individual, unique, GPUs. Why do we ignore -EEXIST?
drivers/platform/nvidia/egm.c line 85
Quoted 9 lines
drivers/platform/nvidia/egm.c new file +9
-0+ +return 0 ; +} + +static void remove_egm_symlinks (struct nvgrace_egm_dev *egm_dev, +struct pci_dev *pdev) +{ +sysfs_remove_link (&egm_dev->device.kobj,dev_name (&pdev->dev)); +}
Consistent naming between the symmetric functions would be nice.
Quoted 11 lines
drivers/platform/nvidia/egm.c new file +11
-0+ +static int add_gpu (struct nvgrace_egm_dev *egm_dev,struct pci_dev *pdev) +{ +struct gpu_node *node; + + node =kzalloc_obj (*node,GFP_KERNEL ); +if (!node) +return -ENOMEM ; + +/* +* The pdev is stored for the lifetime of the EGM device but it is
drivers/platform/nvidia/egm.c line 106
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+* obtained from a for_each_pci_dev() walk that drops its reference as +* it advances. Take a reference here so the stored pointer stays valid +* even if the GPU is later unbound; remove_gpus() drops it. +*/ + node->pdev =pci_dev_get (pdev); + +list_add_tail (&node->list, &egm_dev->gpus); + +return nvgrace_egm_create_gpu_links (egm_dev, pdev); +}
If the EEXIST issue above is fabricated and both symlink functions are just wrappers with a single caller, we should drop them, ex:
node = kzalloc_obj(...);
if (!node)
return -ENOMEM;
ret = sysfs_create_link(...);
if (ret) {
kfree(node);
return ret;
}
node->pdev = pci_dev_get(pdev);
list_add_tail(...);
return 0;drivers/platform/nvidia/egm.c line 118
Quoted 12 lines
drivers/platform/nvidia/egm.c new file +12
-0+ +static void remove_gpus (struct nvgrace_egm_dev *egm_dev) +{ +struct gpu_node *node, *tmp; + +list_for_each_entry_safe (node, tmp, &egm_dev->gpus, list) { +remove_egm_symlinks (egm_dev, node->pdev); +list_del (&node->list); +pci_dev_put (node->pdev); +kfree (node); + } +}
Style choice, but this could drain with:
struct gpu_node *gpu;
while ((gpu = list_first_entry_or_null(&egm_dev->gpus,...)) {
sysfs_remove_link(...)
...
}drivers/platform/nvidia/egm.c line 126
Quoted 8 lines
drivers/platform/nvidia/egm.c new file +8
-0+ +static void egm_chardev_release (struct device *dev) +{ + struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device); + +remove_gpus (egm_chardev); +kfree (egm_chardev); +}
Fix subject/verb consistency in naming throughout, egm_chardev_release() above (subject-verb), setup_egm_chardev() below (verb-subject).
Quoted 7 lines
drivers/platform/nvidia/egm.c new file +7
-0+ +static struct nvgrace_egm_dev *setup_egm_chardev (u64 egmphys,u64 egmlength, +u64 egmpxm) +{ +struct nvgrace_egm_dev *egm_chardev; +unsigned int baseminor =MINOR (dev); +int ret;
drivers/platform/nvidia/egm.c line 143
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+ +/* +* Only MAX_EGM_NODES minors from baseminor were reserved. Reject a PXM +* outside that window. +*/ +if (egmpxm < baseminor || egmpxm - baseminor >=MAX_EGM_NODES ) { +pr_err ("nvgrace-egm: EGM proximity domain %llu outside reserved minor window [%u, %u)\n" , + egmpxm, baseminor, baseminor +MAX_EGM_NODES ); + goto create_err; + }
See below, this is an artificial requirement.
drivers/platform/nvidia/egm.c line 154
Quoted 11 lines
drivers/platform/nvidia/egm.c new file +11
-0+ + egm_chardev =kzalloc_obj (*egm_chardev,GFP_KERNEL ); +if (!egm_chardev) + goto create_err; + +device_initialize (&egm_chardev->device); + +/* +* Use the proximity domain number as the device minor number. +* So the EGM corresponding to node X would be /dev/egmX. +*/
See below, what does this give us?
Quoted 66 lines
drivers/platform/nvidia/egm.c new file +66
-0+ egm_chardev->egmphys = egmphys; + egm_chardev->egmlength = egmlength; + egm_chardev->egmpxm = egmpxm; +INIT_LIST_HEAD (&egm_chardev->gpus); + + egm_chardev->device.devt =MKDEV (MAJOR (dev), egm_chardev->egmpxm); + egm_chardev->device.class = class; + egm_chardev->device.release = egm_chardev_release; +cdev_init (&egm_chardev->cdev, &file_ops); + egm_chardev->cdev.owner =THIS_MODULE ; + + ret =dev_set_name (&egm_chardev->device,"egm%llu" , egm_chardev->egmpxm); +if (ret) + goto error_exit; + + ret =cdev_device_add (&egm_chardev->cdev, &egm_chardev->device); +if (ret) + goto error_exit; + +return egm_chardev; + +error_exit: +put_device (&egm_chardev->device); +create_err: +return NULL ; +} + +static void del_egm_chardev (struct nvgrace_egm_dev *egm_chardev) +{ +cdev_device_del (&egm_chardev->cdev, &egm_chardev->device); +put_device (&egm_chardev->device); +} + +static char *egm_devnode (const struct device *device,umode_t *mode) +{ +if (mode) + *mode =0600 ; + +return NULL ; +} + +static ssize_t egm_size_show (struct device *dev,struct device_attribute *attr, +char *buf) +{ +struct nvgrace_egm_dev *egm_dev = + container_of(dev, struct nvgrace_egm_dev, device); + +return sysfs_emit (buf,"0x%zx\n" , egm_dev->egmlength); +} + +static DEVICE_ATTR_RO (egm_size); + +static struct attribute *attrs[] = { + &dev_attr_egm_size.attr, +NULL , +}; + +static const struct attribute_group attr_group = { + .attrs = attrs, +}; + +static const struct attribute_group *attr_groups[] = { + &attr_group, +NULL +}; +
drivers/platform/nvidia/egm.c line 230
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+/* +* Determine if the EGM feature is enabled. If disabled, there +* will be no EGM properties populated in the ACPI tables and this +* fetch would fail. +*/ +static int has_egm_property (struct pci_dev *pdev,u64 *pegmpxm) +{ +return device_property_read_u64 (&pdev->dev,"nvidia,egm-pxm" , + pegmpxm); +}
What's the value of splitting this out from the properties below? They're only used together. One function could fail on the first failed property. The function above has a non-intuitive convention, named as a bool test, but returns an errno. The "fetch" function below has the more intuitive convention.
Quoted 92 lines
drivers/platform/nvidia/egm.c new file +92
-0+ +static int fetch_egm_property (struct pci_dev *pdev,u64 *pegmphys, +u64 *pegmlength) +{ +int ret; + +/* +* The memory information is present in the system ACPI tables as DSD +* properties nvidia,egm-base-pa and nvidia,egm-size. +*/ + ret =device_property_read_u64 (&pdev->dev,"nvidia,egm-size" , + pegmlength); +if (ret) +return ret; + + ret =device_property_read_u64 (&pdev->dev,"nvidia,egm-base-pa" , + pegmphys); + +return ret; +} + +static struct nvgrace_egm_dev *get_egm_dev (u64 egmpxm) +{ +struct nvgrace_egm_dev_entry *egm_entry; + +list_for_each_entry (egm_entry, &egm_chardevs, list) { +/* +* A system could have multiple GPUs associated with an +* EGM region and will have the same set of EGM region +* information. Return the existing EGM device if already +* registered through a different GPU on the same socket. +*/ +if (egm_entry->egm_dev->egmpxm == egmpxm) +return egm_entry->egm_dev; + } + +return NULL ; +} + +static void nvgrace_egm_destroy_pci_devs (void ) +{ +struct nvgrace_egm_dev_entry *entry, *tmp; + +list_for_each_entry_safe (entry, tmp, &egm_chardevs, list) { +list_del (&entry->list); +del_egm_chardev (entry->egm_dev); +kfree (entry); + } +} + +/* +* Walk all PCI devices and for each one that has a companion ACPI object +* advertising EGM properties, create an auxiliary device for that EGM range. +* Multiple GPUs on the same socket share a single EGM region (same proximity +* domain). So de-duplicate by skipping a pxm that is already registered. +*/ +static int nvgrace_egm_create_pci_egm_devs (void ) +{ +struct nvgrace_egm_dev_entry *egm_entry; +struct nvgrace_egm_dev *egm_dev; +struct pci_dev *pdev =NULL ; +int ret; + +for_each_pci_dev (pdev) { +u64 egmphys, egmlength, egmpxm; + +if (has_egm_property (pdev, &egmpxm)) +continue ; + + ret =fetch_egm_property (pdev, &egmphys, &egmlength); +if (ret) +continue ; + + egm_dev =get_egm_dev (egmpxm); +if (egm_dev) { + ret =add_gpu (egm_dev, pdev); +if (ret) + goto free_dev; +continue ; + } + + egm_entry =kzalloc_obj (*egm_entry,GFP_KERNEL ); +if (!egm_entry) { +pci_dev_put (pdev); +return -ENOMEM ; + } + + egm_dev =setup_egm_chardev (egmphys, egmlength, egmpxm); +if (!egm_dev) { +kfree (egm_entry); +pci_dev_put (pdev); +return -EINVAL ;
drivers/platform/nvidia/egm.c line 332
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+ } + + egm_entry->egm_dev = egm_dev; + + ret =add_gpu (egm_entry->egm_dev, pdev); +if (ret) { +del_egm_chardev (egm_dev); +kfree (egm_entry); + goto free_dev; + }
Bare returns with pci_dev_put() mixed with partial unwinds and gotos is hard to review and hard to maintain. Make this consistent. Failed functions should cleanup any state they modified, drop the PCI reference and unwind based on the state of the egm_chardevs and egm_entry.
Quoted 71 lines
drivers/platform/nvidia/egm.c new file +71
-0+ +list_add_tail (&egm_entry->list, &egm_chardevs); + } + +return 0 ; + +free_dev: +/* +* Reached only via goto from inside the loop where the pdev still +* holds the reference from the last pci_get_device(). So drop it here. +*/ +pci_dev_put (pdev); +nvgrace_egm_destroy_pci_devs (); +return ret; +} + +/* +* The char device minor is the EGM proximity domain number which is sparse +* and need not start at 0. Find the lowest proximity domain among the EGM +* devices so the minor range can be reserved starting there. Returns -ENODEV +* if the system has no EGM devices. +* +* EGM PXMs are assumed contiguous within [lowest, lowest + MAX_EGM_NODES), +* which holds for the supported Grace configurations. +*/ +static int nvgrace_egm_lowest_pxm (u64 *pmin_pxm) +{ +struct pci_dev *pdev =NULL ; +bool found = false; + +for_each_pci_dev (pdev) { +u64 egmphys, egmlength, egmpxm; + +if (has_egm_property (pdev, &egmpxm)) +continue ; + +if (fetch_egm_property (pdev, &egmphys, &egmlength)) +continue ; + +if (!found || egmpxm < *pmin_pxm) { + *pmin_pxm = egmpxm; + found = true; + } + } + +if (!found) +return -ENODEV ; + +/* +* The PXM is used verbatim as the minor (only MINORBITS wide). Validate +* the window would not overflow MINORMASK and be truncated. +*/ +if (*pmin_pxm + (MAX_EGM_NODES -1 ) >MINORMASK ) { +pr_err ("nvgrace-egm: lowest EGM proximity domain %llu\n" +"exceeds the minor range\n" , *pmin_pxm); +return -ERANGE ; + } + +return 0 ; +} + +static int __init nvgrace_egm_init (void ) +{ +u64 min_pxm; +int ret; + +/* +* Each EGM region is exposed as /dev/egmX with the proximity domain X +* as its minor number. Reserve MAX_EGM_NODES minors starting at +* the lowest proximity domain so that every egmX device falls within +* the reserved range. With no EGM devices present, load without
drivers/platform/nvidia/egm.c line 413
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+* reserving any resources. +*/ + ret =nvgrace_egm_lowest_pxm (&min_pxm); +if (ret == -ENODEV ) +return 0 ; +else if (ret) +return ret; + + ret =alloc_chrdev_region (&dev, min_pxm,MAX_EGM_NODES , +NVGRACE_EGM_DEV_NAME );
It's not clear to me what it's buying us to require the minor numbers match the PXM. OTOH, it's making us walk all the PCI devices twice and introduces built in fragility to our minor allocation. Just get any MAX_EGM_NODES and use an IDA to track allocated minors. The chardev still exposes the PXM in the name and the lowest_pxm function goes away.
Quoted 24 lines
drivers/platform/nvidia/egm.c new file +24
-0+if (ret <0 ) +return ret; + + class =class_create (NVGRACE_EGM_DEV_NAME ); +if (IS_ERR (class)) { +unregister_chrdev_region (dev,MAX_EGM_NODES ); +return PTR_ERR (class); + } + + class->devnode = egm_devnode; + class->dev_groups = attr_groups; + + ret =nvgrace_egm_create_pci_egm_devs (); +if (ret) + goto cleanup_pci_devs; + +return 0 ; + +cleanup_pci_devs: +nvgrace_egm_destroy_pci_devs (); +class_destroy (class); +unregister_chrdev_region (dev,MAX_EGM_NODES ); + +return ret;
drivers/platform/nvidia/egm.c line 447
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+} + +static void __exit nvgrace_egm_cleanup(void) +{ +/* +* When no EGM devices were present, the module loaded inertly and +* reserved nothing. @class stays NULL and there is nothing to undo. +*/ +if (!class) +return ;
We should instead make nvgrace_egm_create_pci_egm_devs() return -ENODEV if no devices are found, remove the inertly loaded with nothing to do state, then we don't need to handle this case. We also won't have the class pointer once we move to class_register(). Thanks,
Alex
Quoted 12 lines
drivers/platform/nvidia/egm.c new file +12
-0+ +nvgrace_egm_destroy_pci_devs (); +class_destroy (class); +unregister_chrdev_region (dev,MAX_EGM_NODES ); +} + +module_init (nvgrace_egm_init); +module_exit (nvgrace_egm_cleanup); + +MODULE_LICENSE ("GPL" ); +MODULE_AUTHOR ("Ankit Agrawal <[email protected]>" ); +MODULE_DESCRIPTION ("NVGRACE EGM - Module to support Extended GPU Memory on NVIDIA Grace Based systems" );
Re: [PATCH 1/4] platform/nvidia: Introduce nvgrace-egm driver and enumerate EGM regions
In reply to Ankit Agrawaldrivers/platform/nvidia/egm.c line 48
Quoted 9 lines
drivers/platform/nvidia/egm.c new file +9
-0+static int nvgrace_egm_open (struct inode *inode,struct file *file) +{ +return 0 ; +} + +static int nvgrace_egm_release (struct inode *inode,struct file *file) +{ +return 0 ; +}
You don't have anything here yet, but based on how this is working later you should technically hold a device reference while the file is open to prevent the file->private_data from becoming freed while the file is still open.
That is impossible for this particular design since the freeing only happens on module unload which the open file directly prevents, but it is a nice thing to have to be tidy.
Quoted 35 lines
drivers/platform/nvidia/egm.c new file +35
-0+static struct nvgrace_egm_dev *setup_egm_chardev (u64 egmphys,u64 egmlength, +u64 egmpxm) +{ +struct nvgrace_egm_dev *egm_chardev; +unsigned int baseminor =MINOR (dev); +int ret; + +/* +* Only MAX_EGM_NODES minors from baseminor were reserved. Reject a PXM +* outside that window. +*/ +if (egmpxm < baseminor || egmpxm - baseminor >=MAX_EGM_NODES ) { +pr_err ("nvgrace-egm: EGM proximity domain %llu outside reserved minor window [%u, %u)\n" , + egmpxm, baseminor, baseminor +MAX_EGM_NODES ); + goto create_err; + } + + egm_chardev =kzalloc_obj (*egm_chardev,GFP_KERNEL ); +if (!egm_chardev) + goto create_err; + +device_initialize (&egm_chardev->device); + +/* +* Use the proximity domain number as the device minor number. +* So the EGM corresponding to node X would be /dev/egmX. +*/ + egm_chardev->egmphys = egmphys; + egm_chardev->egmlength = egmlength; + egm_chardev->egmpxm = egmpxm; +INIT_LIST_HEAD (&egm_chardev->gpus); + + egm_chardev->device.devt =MKDEV (MAJOR (dev), egm_chardev->egmpxm); + egm_chardev->device.class = class; + egm_chardev->device.release = egm_chardev_release;
drivers/platform/nvidia/egm.c line 172
Quoted 10 lines
drivers/platform/nvidia/egm.c new file +10
-0+cdev_init (&egm_chardev->cdev, &file_ops); + egm_chardev->cdev.owner =THIS_MODULE ; + + ret =dev_set_name (&egm_chardev->device,"egm%llu" , egm_chardev->egmpxm); +if (ret) + goto error_exit; + + ret =cdev_device_add (&egm_chardev->cdev, &egm_chardev->device); +if (ret) + goto error_exit;
This use of cdev_device_add() looks broadly right to me.
Though I usually put the attrs on the device not the class, I guess the effect is the same?
Signature · 1 line
Jason
PATCH 2/4 platform/nvidia: Implement mmap and memory scrubbing for EGM chardev
In reply to Ankit AgrawalSetup the file operations (open, close and mmap) for the EGM char device.
Memory scrubbing:
- The EGM region is invisible to the host Linux kernel and is not managed by the page allocator. So the driver is responsible for zeroing it before handing it to a VM.
- Clear the entire EGM region on the first open() call using memremap() and memset() in 1 GiB chunks with cond_resched() between iterations to avoid RCU stalls on large regions.
- The region is handed to a single VM at a time and must be scrubbed before each handover. Serialise openers with a mutex guarding open_count and refuse a second opener with -EBUSY while the region is still held.
mmap:
- Implement nvgrace_egm_mmap() using remap_pfn_range(). The EGM region is not part of the host EFI map and has no struct pages making remap_pfn_range() the correct interface for mapping it directly into QEMU's virtual address space.
- Validate that the requested range fits within the EGM region. Reject a vm_pgoff that already lies outside the region before it is shifted into a byte count so that the page offset cannot overflow and slip past the range check.
--- drivers/platform/nvidia/egm.c | 125 ++++++++++++++++++++++++++++++++-- 1 file changed, 121 insertions(+), 4 deletions(-)
drivers/platform/nvidia/egm.c +121 -4
@@ -3,6 +3,9 @@* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved */ +#include <linux/mutex.h> +#include <linux/sched/signal.h> +#include <linux/sizes.h> #include <linux/vfio_pci_core.h> #define NVGRACE_EGM_DEV_NAME "egm"
@@ -19,6 +22,10 @@ struct gpu_node {struct nvgrace_egm_dev {struct device device;struct cdev cdev; +/* serialises the first-open scrub */ +struct mutex open_lock; +/* protected by open_lock */ +unsigned int open_count;phys_addr_t egmphys;size_t egmlength;u64 egmpxm;
@@ -39,21 +46,129 @@ static LIST_HEAD(egm_chardevs);static int nvgrace_egm_open (struct inode *inode,struct file *file) { -return 0 ; +struct nvgrace_egm_dev *egm_dev = + container_of(inode->i_cdev, struct nvgrace_egm_dev, cdev); +phys_addr_t phys; +size_t remaining, chunk_size; +void *chunk_addr; +int ret; + + file->private_data = egm_dev; + +/* +* The EGM region is a physical carveout handed to a single VM at a +* time and must be scrubbed before each handover. Take open_lock +* killably around the scrub which can run for long time, so that +* the waiters stay killable. +*/ +if (mutex_lock_killable (&egm_dev->open_lock)) +return -EINTR ; + +/* +* Refuse a second opener while the region is still held. release() +* runs only when the last reference to the struct file drops but an +* mmap keeps that reference (and hence open_count) alive past +* close for the lifetime of the VMA. So open_count returns to zero +* only when every mapping is gone. A concurrent opener cannot be +* handed the region: it cannot be re-scrubbed without destroying the +* current consumer's live data. +*/ +if (egm_dev->open_count) { + ret = -EBUSY ; + goto unlock; + } + +/* +* nvgrace-egm module is responsible to manage the EGM memory as +* the host kernel has no knowledge of it. Clear the region before +* handing over to userspace. +* +* The EGM region can be very large (hundreds of GiB). So Map and zero +* one chunk at a time rather than mapping the whole region at once. +*/ + phys = egm_dev->egmphys; + remaining = egm_dev->egmlength; + +while (remaining >0 ) { +/* +* The scrub holds the lock for a long time; let it be +* SIGKILL'd. +*/ +if (fatal_signal_pending (current)) { + ret = -EINTR ; + goto unlock; + } + + chunk_size =min (remaining,SZ_1G ); + + chunk_addr =memremap (phys, chunk_size,MEMREMAP_WB ); +if (!chunk_addr) { + ret = -ENOMEM ; + goto unlock; + } + +memset (chunk_addr,0 , chunk_size); +memunmap (chunk_addr); +cond_resched (); + + phys += chunk_size; + remaining -= chunk_size; + } + +/* +* Mark the device open only after the scrub completes so that a +* concurrent opener cannot observe a non-zero count and proceed +* before the region has been cleared. +*/ + egm_dev->open_count =1 ; + ret =0 ; + +unlock: +mutex_unlock (&egm_dev->open_lock); +return ret; }static int nvgrace_egm_release (struct inode *inode,struct file *file) { +struct nvgrace_egm_dev *egm_dev = + container_of(inode->i_cdev, struct nvgrace_egm_dev, cdev); + +guard (mutex)(&egm_dev->open_lock); + +if (!--egm_dev->open_count) + file->private_data =NULL ; +return 0 ; }static int nvgrace_egm_mmap (struct file *file,struct vm_area_struct *vma) { +struct nvgrace_egm_dev *egm_dev = file->private_data; +u64 req_len, pgoff, end; +unsigned long start_pfn, num_pages; + + pgoff = vma->vm_pgoff; + num_pages = egm_dev->egmlength >>PAGE_SHIFT ; + +/* Reject a page offset that already lies outside the EGM region. */ +if (pgoff >= num_pages) +return -EINVAL ; + +if (check_sub_overflow (vma->vm_end, vma->vm_start, &req_len) || +check_add_overflow (PHYS_PFN (egm_dev->egmphys), pgoff, &start_pfn) || +check_add_overflow (PFN_PHYS (pgoff), req_len, &end)) +return -EOVERFLOW ; + +if (end > egm_dev->egmlength) +return -EINVAL ; +/* -* Mapping the EGM region into userspace is implemented by a later -* patch. Until then refuse the mmap. +* EGM memory is invisible to the host kernel and is not managed +* by it. Map the usermode VMA to the EGM region. */ -return -EOPNOTSUPP ; +return remap_pfn_range (vma, vma->vm_start, + start_pfn, req_len, + vma->vm_page_prot); }static const struct file_operations file_ops = {
@@ -122,6 +237,7 @@ static void egm_chardev_release(struct device *dev) struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device);remove_gpus (egm_chardev); +mutex_destroy (&egm_chardev->open_lock);kfree (egm_chardev); }
@@ -155,6 +271,7 @@ static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength, egm_chardev->egmphys = egmphys; egm_chardev->egmlength = egmlength; egm_chardev->egmpxm = egmpxm; +mutex_init (&egm_chardev->open_lock);INIT_LIST_HEAD (&egm_chardev->gpus); egm_chardev->device.devt =MKDEV (MAJOR (dev), egm_chardev->egmpxm);
Signature · 1 line
2.34.1
Re: [PATCH 2/4] platform/nvidia: Implement mmap and memory scrubbing for EGM chardev
Quoted 57 lines · On Thu, 2 Jul 2026 19:25:30 +0000 Ankit Agrawal <[email protected]> wrote:
Setup the file operations (open, close and mmap) for the EGM char device.
Memory scrubbing:
- The EGM region is invisible to the host Linux kernel and is not managed by the page allocator. So the driver is responsible for zeroing it before handing it to a VM.
- Clear the entire EGM region on the firstopen()call usingmemremap()andmemset()in 1 GiB chunks withcond_resched()between iterations to avoid RCU stalls on large regions.
- The region is handed to a single VM at a time and must be scrubbed before each handover. Serialise openers with a mutex guardingopen_countand refuse a second opener with-EBUSYwhile the region is still held.mmap:
- Implementnvgrace_egm_mmap()usingremap_pfn_range(). The EGM region is not part of the host EFI map and has no struct pages makingremap_pfn_range()the correct interface for mapping it directly into QEMU's virtual address space.
- Validate that the requested range fits within the EGM region. Reject avm_pgoffthat already lies outside the region before it is shifted into a byte count so that the page offset cannot overflow and slip past the range check.Suggested-by Aniket Agashe <[email protected]>Suggested-by Vikram Sethi <[email protected]>Assisted-by Claude:claude-opus-4.8Signed-off-by Ankit Agrawal <[email protected]>--- drivers/platform/nvidia/egm.c | 125 ++++++++++++++++++++++++++++++++-- 1 file changed, 121 insertions(+), 4 deletions(-)drivers/platform/nvidia/egm.c +7
-0@@ -3,6 +3,9 @@* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved */ +#include <linux/mutex.h> +#include <linux/sched/signal.h> +#include <linux/sizes.h> #include <linux/vfio_pci_core.h> #define NVGRACE_EGM_DEV_NAME "egm"@@ -19,6 +22,10 @@ struct gpu_node {struct nvgrace_egm_dev {struct device device;struct cdev cdev; +/* serialises the first-open scrub */ +struct mutex open_lock; +/* protected by open_lock */ +unsigned int open_count;phys_addr_t egmphys;size_t egmlength;u64 egmpxm;
drivers/platform/nvidia/egm.c static LIST_HEAD(egm_chardevs); · line 54
Quoted 10 lines
drivers/platform/nvidia/egm.c +6
-1@@ -39,21 +46,129 @@ static LIST_HEAD(egm_chardevs);static int nvgrace_egm_open (struct inode *inode,struct file *file) { -return 0 ; +struct nvgrace_egm_dev *egm_dev = + container_of(inode->i_cdev, struct nvgrace_egm_dev, cdev); +phys_addr_t phys; +size_t remaining, chunk_size; +void *chunk_addr; +int ret;
Nit, reverse christmas tree ordering.
Quoted 15 lines
drivers/platform/nvidia/egm.c +15
-0+ + file->private_data = egm_dev; + +/* +* The EGM region is a physical carveout handed to a single VM at a +* time and must be scrubbed before each handover. Take open_lock +* killably around the scrub which can run for long time, so that +* the waiters stay killable. +*/ +if (mutex_lock_killable (&egm_dev->open_lock)) +return -EINTR ; + +/* +* Refuse a second opener while the region is still held. release() +* runs only when the last reference to the struct file drops but an
drivers/platform/nvidia/egm.c static LIST_HEAD(egm_chardevs); · line 79
Quoted 10 lines
drivers/platform/nvidia/egm.c +10
-0+* mmap keeps that reference (and hence open_count) alive past +* close for the lifetime of the VMA. So open_count returns to zero +* only when every mapping is gone. A concurrent opener cannot be +* handed the region: it cannot be re-scrubbed without destroying the +* current consumer's live data. +*/ +if (egm_dev->open_count) { + ret = -EBUSY ; + goto unlock; + }
Do we need to hold the lock for the extent of the scrub?
Currently a 2nd open is blocked, only to fail once the scrub completes successfully. Can't we increment open_count here, let the 2nd open fail fast, scrub w/o lock, re-acquire lock and decrement open_count if scrub fails.
Ideally though, we probably want to kick off a scrub thread on init/close, where open does a killable wait for completion (completion signaled by the workqueue thread), and exit does a cancel_work_sync() to force the workqueue flush. Scrubbing is then asynchronous and open can be instant if the background thread has already completed. Just watch out for the ordering of exposing the chardev vs setting the completion state.
(Note that I'm including scrub on exit, not just open, it seems like a coverage gap if the memory isn't also scrubbed before being released to another driver, just like vfio-pci resets devices on close.)
Quoted 38 lines
drivers/platform/nvidia/egm.c +38
-0+ +/* +* nvgrace-egm module is responsible to manage the EGM memory as +* the host kernel has no knowledge of it. Clear the region before +* handing over to userspace. +* +* The EGM region can be very large (hundreds of GiB). So Map and zero +* one chunk at a time rather than mapping the whole region at once. +*/ + phys = egm_dev->egmphys; + remaining = egm_dev->egmlength; + +while (remaining >0 ) { +/* +* The scrub holds the lock for a long time; let it be +* SIGKILL'd. +*/ +if (fatal_signal_pending (current)) { + ret = -EINTR ; + goto unlock; + } + + chunk_size =min (remaining,SZ_1G ); + + chunk_addr =memremap (phys, chunk_size,MEMREMAP_WB ); +if (!chunk_addr) { + ret = -ENOMEM ; + goto unlock; + } + +memset (chunk_addr,0 , chunk_size); +memunmap (chunk_addr); +cond_resched (); + + phys += chunk_size; + remaining -= chunk_size; + } +
drivers/platform/nvidia/egm.c static LIST_HEAD(egm_chardevs); · line 127
Quoted 10 lines
drivers/platform/nvidia/egm.c +10
-0+/* +* Mark the device open only after the scrub completes so that a +* concurrent opener cannot observe a non-zero count and proceed +* before the region has been cleared. +*/ + egm_dev->open_count =1 ; + ret =0 ; + +unlock: +mutex_unlock (&egm_dev->open_lock);
With the above, we don't need the killable mutex acquire and this can just be a scoped guard mutex for open_count.
drivers/platform/nvidia/egm.c static LIST_HEAD(egm_chardevs); · line 139
Quoted 12 lines
drivers/platform/nvidia/egm.c +8
-0+return ret; }static int nvgrace_egm_release (struct inode *inode,struct file *file) { +struct nvgrace_egm_dev *egm_dev = + container_of(inode->i_cdev, struct nvgrace_egm_dev, cdev); + +guard (mutex)(&egm_dev->open_lock); + +if (!--egm_dev->open_count) + file->private_data =NULL ;
open_count is only ever 0/1, there's no increment in this thread, so the decrement and test is out of place here.
Quoted 14 lines
drivers/platform/nvidia/egm.c +9
-0+return 0 ; }static int nvgrace_egm_mmap (struct file *file,struct vm_area_struct *vma) { +struct nvgrace_egm_dev *egm_dev = file->private_data; +u64 req_len, pgoff, end; +unsigned long start_pfn, num_pages; + + pgoff = vma->vm_pgoff; + num_pages = egm_dev->egmlength >>PAGE_SHIFT ; + +/* Reject a page offset that already lies outside the EGM region. */
drivers/platform/nvidia/egm.c static LIST_HEAD(egm_chardevs); · line 163
Quoted 10 lines
drivers/platform/nvidia/egm.c +10
-0+if (pgoff >= num_pages) +return -EINVAL ; + +if (check_sub_overflow (vma->vm_end, vma->vm_start, &req_len) || +check_add_overflow (PHYS_PFN (egm_dev->egmphys), pgoff, &start_pfn) || +check_add_overflow (PFN_PHYS (pgoff), req_len, &end)) +return -EOVERFLOW ; + +if (end > egm_dev->egmlength) +return -EINVAL ;
Shouldn't we validate MAP_SHARED as well? Is this also a gap in current nvgrace-gpu?
drivers/platform/nvidia/egm.c static LIST_HEAD(egm_chardevs); · line 172
Quoted 12 lines
drivers/platform/nvidia/egm.c +6
-3+/* -* Mapping the EGM region into userspace is implemented by a later -* patch. Until then refuse the mmap. +* EGM memory is invisible to the host kernel and is not managed +* by it. Map the usermode VMA to the EGM region. */ -return -EOPNOTSUPP ; +return remap_pfn_range (vma, vma->vm_start, + start_pfn, req_len, + vma->vm_page_prot); }
Is huge_fault an option for an iteration after this gets merged to make use of pud/pmd mappings given the size of this range? Thanks,
Alex
Quoted 18 lines
drivers/platform/nvidia/egm.c +2
-0static const struct file_operations file_ops = {@@ -122,6 +237,7 @@ static void egm_chardev_release(struct device *dev) struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device);remove_gpus (egm_chardev); +mutex_destroy (&egm_chardev->open_lock);kfree (egm_chardev); }@@ -155,6 +271,7 @@ static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength, egm_chardev->egmphys = egmphys; egm_chardev->egmlength = egmlength; egm_chardev->egmpxm = egmpxm; +mutex_init (&egm_chardev->open_lock);INIT_LIST_HEAD (&egm_chardev->gpus); egm_chardev->device.devt =MKDEV (MAJOR (dev), egm_chardev->egmpxm);
Re: [PATCH 2/4] platform/nvidia: Implement mmap and memory scrubbing for EGM chardev
In reply to Ankit Agrawaldrivers/platform/nvidia/egm.c static LIST_HEAD(egm_chardevs); · line 139
Quoted 9 lines
static int
nvgrace_egm_release(struct inode *inode, struct file *file){drivers/platform/nvidia/egm.c +7
-0+struct nvgrace_egm_dev *egm_dev = + container_of(inode->i_cdev, struct nvgrace_egm_dev, cdev); + +guard (mutex)(&egm_dev->open_lock); + +if (!--egm_dev->open_count) + file->private_data =NULL ;
You do not need to zero the private data.
open_count is only zero or one, so if the only write is =1 then then only store should be =0 and it doesn't make sense to call it open *count*, it is just 'bool opened'
Quoted 18 lines
static int
nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma){drivers/platform/nvidia/egm.c +16
-0+struct nvgrace_egm_dev *egm_dev = file->private_data; +u64 req_len, pgoff, end; +unsigned long start_pfn, num_pages; + + pgoff = vma->vm_pgoff; + num_pages = egm_dev->egmlength >>PAGE_SHIFT ; + +/* Reject a page offset that already lies outside the EGM region. */ +if (pgoff >= num_pages) +return -EINVAL ; + +if (check_sub_overflow (vma->vm_end, vma->vm_start, &req_len) || +check_add_overflow (PHYS_PFN (egm_dev->egmphys), pgoff, &start_pfn) || +check_add_overflow (PFN_PHYS (pgoff), req_len, &end)) +return -EOVERFLOW ; +
drivers/platform/nvidia/egm.c static LIST_HEAD(egm_chardevs); · line 56
Quoted 10 lines
drivers/platform/nvidia/egm.c +5
-3+if (end > egm_dev->egmlength) +return -EINVAL ; +/* -* Mapping the EGM region into userspace is implemented by a later -* patch. Until then refuse the mmap. +* EGM memory is invisible to the host kernel and is not managed +* by it. Map the usermode VMA to the EGM region. */ -return -EOPNOTSUPP ;
Don't add an empty stub in the prior patch for an optional op fully implemented in a later patch
The logic looks fine otherwise
Signature · 1 line
Jason
PATCH 3/4 platform/nvidia: Handle retired ECC pages and expose via ioctl
In reply to Ankit AgrawalEGM regions may contain pages retired due to ECC/poison errors that are known to the system firmware beforehand. SBIOS records the list in a dedicated carveout region whose SPA is advertised via the ACPI DSD property "nvidia,egm-retired-pages-data-base". The carveout begins with a u64 count followed by that many u64 system physical addresses each the base SPA of a retired 64K page within the EGM region.
The EGM region is linear, so the SPA - EGM_base calculation can be used to determine the offset within the EGM region. Moreover, the region is assigned linearly to the VM. So offset within EGM region on the host would be same as on the VM. Hence the retired pages offset is calculated and stored by the nvgrace-egm. An xarray is used to track the retired page offsets. These are exposed to the userspace (Qemu) which may use the information to skip the retired pages while forming the memory mapping for the VM.
Note that a retired page SPA SBIOS entry is for 64k size, but the xarray list tracks offsets in PAGE_SIZE. So SZ_64K/PAGE_SIZE entries are added to the xarray to ensure comaptibility with 4k OS.
Patch implements the following:
Retired page tracking:
- Map the carveout region with memremap() and validate the count.
- convert each physical address to an in-EGM-region offset and populate a per-EGM-device hashtable (keyed by that offset).
Userspace ioctl:
- Introduce EGM_RETIRED_PAGES_LIST ioctl and add a new UAPI header Userspace (QEMU) calls this ioctl to retrieve the list of retired page offsets. QEMU then communicates the list to the VM so the guest can take appropriate action.
Update MAINTAINERS to list include/uapi/linux/egm.h
--- .../userspace-api/ioctl/ioctl-number.rst | 1 + MAINTAINERS | 1 + drivers/platform/nvidia/egm.c | 211 +++++++++++++++++- include/uapi/linux/egm.h | 28 +++ 4 files changed, 231 insertions(+), 10 deletions(-) create mode 100644 include/uapi/linux/egm.h
Documentation/userspace-api/ioctl/ioctl-number.rst +1 -0
@@ -306,6 +306,7 @@ Code Seq# Include File Comments 'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.h VAS API 'v' C0-FF linux/meye.h conflict! 'w' all CERN SCI driver +'x' 00-1F include/uapi/linux/egm.h NVIDIA nvgrace-egm driver 'y' 00-1F packet based user level communications <mailto:[email protected]> 'z' 00-3F CAN bus card conflict!
MAINTAINERS +1 -0
@@ -19348,6 +19348,7 @@ NVIDIA GRACE EGM PLATFORM DRIVER M: Ankit Agrawal <[email protected]> S: Supported F: drivers/platform/nvidia/ +F: include/uapi/linux/egm.h NVIDIA VRS RTC DRIVER M: Shubhi Garg <[email protected]>
drivers/platform/nvidia/egm.c +201 -10
321 lines not on this page · raw message
include/uapi/linux/egm.h new file +28 -0
@@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* +* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved +*/ + +#ifndef _UAPI_LINUX_EGM_H +#define _UAPI_LINUX_EGM_H + +#include <linux/types.h> + +#define EGM_TYPE ('x') + +struct egm_retired_pages_info { +__aligned_u64 offset; +__aligned_u64 size; +}; + +struct egm_retired_pages_list { +__u32 argsz; +/* out */ +__u32 count; +/* out */ +struct egm_retired_pages_info retired_pages[]; +}; + +#define EGM_RETIRED_PAGES_LIST _IOWR(EGM_TYPE, 0x01, struct egm_retired_pages_list) + +#endif /* _UAPI_LINUX_EGM_H */
Signature · 1 line
2.34.1
Re: [PATCH 3/4] platform/nvidia: Handle retired ECC pages and expose via ioctl
Quoted 132 lines · On Thu, 2 Jul 2026 19:25:31 +0000 Ankit Agrawal <[email protected]> wrote:
EGM regions may contain pages retired due to ECC/poison errors that are known to the system firmware beforehand. SBIOS records the list in a dedicated carveout region whose SPA is advertised via the ACPI DSD property "nvidia,egm-retired-pages-data-base". The carveout begins with a u64 count followed by that many u64 system physical addresses each the base SPA of a retired 64K page within the EGM region.
The EGM region is linear, so the SPA -
EGM_basecalculation can be used to determine the offset within the EGM region. Moreover, the region is assigned linearly to the VM. So offset within EGM region on the host would be same as on the VM. Hence the retired pages offset is calculated and stored by the nvgrace-egm. An xarray is used to track the retired page offsets. These are exposed to the userspace (Qemu) which may use the information to skip the retired pages while forming the memory mapping for the VM.Note that a retired page SPA SBIOS entry is for 64k size, but the xarray list tracks offsets in
PAGE_SIZE. So SZ_64K/PAGE_SIZE entries are added to the xarray to ensure comaptibility with 4k OS.Patch implements the following:
Retired page tracking:
- Map the carveout region withmemremap()and validate the count.
- convert each physical address to an in-EGM-region offset and populate a per-EGM-device hashtable (keyed by that offset).Userspace ioctl:
- IntroduceEGM_RETIRED_PAGES_LISTioctl and add a new UAPI header Userspace (QEMU) calls this ioctl to retrieve the list of retired page offsets. QEMU then communicates the list to the VM so the guest can take appropriate action.Update
MAINTAINERSto listinclude/uapi/linux/egm.hAssisted-by Claude:claude-opus-4.8Signed-off-by Ankit Agrawal <[email protected]>--- .../userspace-api/ioctl/ioctl-number.rst | 1 + MAINTAINERS | 1 + drivers/platform/nvidia/egm.c | 211 +++++++++++++++++- include/uapi/linux/egm.h | 28 +++ 4 files changed, 231 insertions(+), 10 deletions(-) create mode 100644 include/uapi/linux/egm.hDocumentation/userspace-api/ioctl/ioctl-number.rst +1
-0@@ -306,6 +306,7 @@ Code Seq# Include File Comments 'v' 20-27 arch/powerpc/include/uapi/asm/vas-api.h VAS API 'v' C0-FF linux/meye.h conflict! 'w' all CERN SCI driver +'x' 00-1F include/uapi/linux/egm.h NVIDIA nvgrace-egm driver 'y' 00-1F packet based user level communications <mailto:[email protected]> 'z' 00-3F CAN bus card conflict!MAINTAINERS +1
-0@@ -19348,6 +19348,7 @@ NVIDIA GRACE EGM PLATFORM DRIVER M: Ankit Agrawal <[email protected]> S: Supported F: drivers/platform/nvidia/ +F: include/uapi/linux/egm.h NVIDIA VRS RTC DRIVER M: Shubhi Garg <[email protected]>drivers/platform/nvidia/egm.c +29
-0@@ -7,10 +7,28 @@#include <linux/sched/signal.h> #include <linux/sizes.h> #include <linux/vfio_pci_core.h> +#include <linux/xarray.h> +#include <uapi/linux/egm.h> #define NVGRACE_EGM_DEV_NAME "egm"#define MAX_EGM_NODES 4 +/* +* Presence marker stored in nvgrace_egm_dev.retired_pages. Only the +* page offset within the region carries information and the value +* just needs to be a non-NULL xa entry. +*/ +#define EGM_RETIRED_MARK xa_mk_value(1) + +/* +* The structure match the format of the retired pages information populated +* by the system firmware. +*/ +struct fw_egm_retired_pages { +u64 num_retired_pages; +__u64 retired_page_addr[4096 ]; +}; +static dev_t dev;static struct class *class;@@ -26,8 +44,14 @@ struct nvgrace_egm_dev {struct mutex open_lock;/* protected by open_lock */ unsigned int open_count; +/* +* Set of retired page offsets within the +* EGM region and keyed by page index. +*/ +struct xarray retired_pages;phys_addr_t egmphys;size_t egmlength; +phys_addr_t retiredpagesphys;u64 egmpxm;struct list_head gpus; };@@ -44,6 +68,9 @@ struct nvgrace_egm_dev_entry {*/ static LIST_HEAD (egm_chardevs); +static void cleanup_retired_pages (struct nvgrace_egm_dev *egm_dev); +static int nvgrace_egm_fetch_retired_pages (struct nvgrace_egm_dev *egm_dev); +static int nvgrace_egm_open (struct inode *inode,struct file *file) { struct nvgrace_egm_dev *egm_dev =@@ -171,11 +198,68 @@ static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma) vma->vm_page_prot); } +static long nvgrace_egm_ioctl (struct file *file,unsigned int cmd,unsigned long arg) +{
drivers/platform/nvidia/egm.c static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma) · line 212
Quoted 10 lines
drivers/platform/nvidia/egm.c +10
-0+ unsigned long minsz = offsetofend(struct egm_retired_pages_list, count); +struct egm_retired_pages_list info; +void __user *uarg = (void __user *)arg; +struct nvgrace_egm_dev *egm_dev = file->private_data; + +if (copy_from_user (&info, uarg, minsz)) +return -EFAULT ; + +if (info.argsz < minsz || !egm_dev) +return -EINVAL ;
!egm_dev cannot happen, it's a dead test.
Quoted 7 lines
drivers/platform/nvidia/egm.c +7
-0+ +switch (cmd) { +case EGM_RETIRED_PAGES_LIST : { +unsigned long retired_page_struct_size =sizeof (struct egm_retired_pages_info ); +struct egm_retired_pages_info tmp; +unsigned long page_index; +void *entry;
drivers/platform/nvidia/egm.c static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma) · line 229
Quoted 10 lines
drivers/platform/nvidia/egm.c +10
-0+int count =0 , fill =0 ; + +xa_for_each (&egm_dev->retired_pages, page_index, entry) + count++; + +if (info.argsz < (minsz + count * retired_page_struct_size)) { + info.argsz = minsz + count * retired_page_struct_size; + info.count =0 ; + goto done; + }
Similar mechanics to vfio, but where is this uAPI defined other than deciphering the code. Also, this is a new uAPI, not a backwards compatible uAPI extension. We can return an appropriate errno to indicate space errors.
Quoted 110 lines
drivers/platform/nvidia/egm.c +69
-1+ +xa_for_each (&egm_dev->retired_pages, page_index, entry) { +if (fill >= count) +break ; + + tmp.offset = (u64 )page_index <<PAGE_SHIFT ; + tmp.size =PAGE_SIZE ; + +if (copy_to_user ((u8 __user *)uarg + minsz + + fill * retired_page_struct_size, + &tmp, retired_page_struct_size)) +return -EFAULT ; + + fill++; + } + + info.count = fill; +break ; + } +default : +return -EINVAL ; + } + +done: +return copy_to_user (uarg, &info, minsz) ? -EFAULT :0 ; +} +static const struct file_operations file_ops = { .owner =THIS_MODULE , .open = nvgrace_egm_open, .release = nvgrace_egm_release, .mmap = nvgrace_egm_mmap, + .unlocked_ioctl = nvgrace_egm_ioctl, }; static int nvgrace_egm_create_gpu_links(struct nvgrace_egm_dev *egm_dev,@@ -236,13 +320,15 @@ static void egm_chardev_release(struct device *dev) { struct nvgrace_egm_dev *egm_chardev = container_of(dev, struct nvgrace_egm_dev, device); +cleanup_retired_pages (egm_chardev);remove_gpus (egm_chardev);mutex_destroy (&egm_chardev->open_lock);kfree (egm_chardev); }static struct nvgrace_egm_dev *setup_egm_chardev (u64 egmphys,u64 egmlength, -u64 egmpxm) +u64 egmpxm, +u64 retiredpagesphys) {struct nvgrace_egm_dev *egm_chardev;unsigned int baseminor =MINOR (dev);@@ -271,7 +357,9 @@ static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength, egm_chardev->egmphys = egmphys; egm_chardev->egmlength = egmlength; egm_chardev->egmpxm = egmpxm; + egm_chardev->retiredpagesphys = retiredpagesphys;mutex_init (&egm_chardev->open_lock); +xa_init (&egm_chardev->retired_pages);INIT_LIST_HEAD (&egm_chardev->gpus); egm_chardev->device.devt =MKDEV (MAJOR (dev), egm_chardev->egmpxm);@@ -284,6 +372,10 @@ static struct nvgrace_egm_dev *setup_egm_chardev(u64 egmphys, u64 egmlength,if (ret) goto error_exit; + ret =nvgrace_egm_fetch_retired_pages (egm_chardev); +if (ret) + goto error_exit; + ret =cdev_device_add (&egm_chardev->cdev, &egm_chardev->device);if (ret) goto error_exit;@@ -302,6 +394,85 @@ static void del_egm_chardev(struct nvgrace_egm_dev *egm_chardev)put_device (&egm_chardev->device); } +static void cleanup_retired_pages (struct nvgrace_egm_dev *egm_dev) +{ +/* Entries are value marks, so xa_destroy() frees the table itself. */ +xa_destroy (&egm_dev->retired_pages); +} + +static int nvgrace_egm_fetch_retired_pages (struct nvgrace_egm_dev *egm_dev) +{ +struct fw_egm_retired_pages *egm_retired; +u64 count; +int index, ret =0 ; + +/* No retired-pages region was advertised; nothing to populate. */ +if (!egm_dev->retiredpagesphys) +return 0 ; + + egm_retired =memremap (egm_dev->retiredpagesphys, +sizeof (*egm_retired),MEMREMAP_WB ); +if (!egm_retired) +return -ENOMEM ; + + count = egm_retired->num_retired_pages; +if (count >ARRAY_SIZE (egm_retired->retired_page_addr)) { +memunmap (egm_retired); +return -EINVAL ; + } + +for (index =0 ; index < count && !ret; index++) { +phys_addr_t base = egm_retired->retired_page_addr[index]; +int sub; + +/*
drivers/platform/nvidia/egm.c static void del_egm_chardev(struct nvgrace_egm_dev *egm_chardev) · line 438
Quoted 10 lines
drivers/platform/nvidia/egm.c +10
-0+* Entries come from SBIOS. Skip any base outside the EGM +* region [egmphys, egmphys + egmlength). +*/ +if (base < egm_dev->egmphys || + base >= egm_dev->egmphys + egm_dev->egmlength) { +dev_warn_ratelimited (&egm_dev->device, +"Ignoring retired page %pa outside EGM region\n" , + &base); +continue ; + }
Don't we need to take the page itself into account, not just the base?
Quoted 150 lines
drivers/platform/nvidia/egm.c +66
-9+ +/* +* Since the EGM is linearly mapped, the offset in the +* carveout is the same offset in the VM system memory. +* +* Calculate the offset to communicate to the usermode +* apps. +* +* The retired page entry represent a retired region of +* size 64K. So on a 4K kernel, each entry spans 0x10 +* 4K sub-pages; add one entry per sub-page so that any +* lookup at 4K granularity hits the right retired range. +*/ +for (sub =0 ; sub <SZ_64K /PAGE_SIZE ; sub++) { +unsigned long offset = base + + (phys_addr_t )sub *PAGE_SIZE - egm_dev->egmphys; +void *old; + + old =xa_store (&egm_dev->retired_pages, + offset >>PAGE_SHIFT ,EGM_RETIRED_MARK , +GFP_KERNEL ); +if (xa_is_err (old)) { + ret =xa_err (old); +break ; + } + } + } + +memunmap (egm_retired); + +/* +* On failure the partially populated table is freed by +* egm_chardev_release() via the caller's error path. +*/ +return ret; +} +static char *egm_devnode (const struct device *device,umode_t *mode) {if (mode)@@ -347,22 +518,39 @@ static int has_egm_property(struct pci_dev *pdev, u64 *pegmpxm) }static int fetch_egm_property (struct pci_dev *pdev,u64 *pegmphys, -u64 *pegmlength) +u64 *pegmlength,u64 *pretiredpagesphys) {int ret;/* -* The memory information is present in the system ACPI tables as DSD -* properties nvidia,egm-base-pa and nvidia,egm-size. +* The EGM memory information is present in the system ACPI tables +* as DSD properties nvidia,egm-base-pa and nvidia,egm-size. */ ret =device_property_read_u64 (&pdev->dev,"nvidia,egm-size" , pegmlength);if (ret) -return ret; + goto error_exit; ret =device_property_read_u64 (&pdev->dev,"nvidia,egm-base-pa" , pegmphys); +if (ret) + goto error_exit; + +/* +* SBIOS puts the list of retired pages on a region exposed as +* "nvidia,egm-retired-pages-data-base". Older firmware may not expose +* it. Treat an absent or zero property as 0 retired pages so the +* EGM device is still created (with an empty retired-pages table). +*/ + ret =device_property_read_u64 (&pdev->dev, +"nvidia,egm-retired-pages-data-base" , + pretiredpagesphys); +if (ret) { + *pretiredpagesphys =0 ; + ret =0 ; + } +error_exit:return ret; }@@ -409,12 +597,13 @@ static int nvgrace_egm_create_pci_egm_devs(void)int ret;for_each_pci_dev (pdev) { -u64 egmphys, egmlength, egmpxm; +u64 egmphys, egmlength, egmpxm, retiredpagesphys;if (has_egm_property (pdev, &egmpxm))continue ; - ret =fetch_egm_property (pdev, &egmphys, &egmlength); + ret =fetch_egm_property (pdev, &egmphys, &egmlength, + &retiredpagesphys);if (ret)continue ;@@ -432,7 +621,8 @@ static int nvgrace_egm_create_pci_egm_devs(void)return -ENOMEM ; } - egm_dev =setup_egm_chardev (egmphys, egmlength, egmpxm); + egm_dev =setup_egm_chardev (egmphys, egmlength, egmpxm, + retiredpagesphys);if (!egm_dev) {kfree (egm_entry);pci_dev_put (pdev);@@ -478,12 +668,13 @@ static int nvgrace_egm_lowest_pxm(u64 *pmin_pxm)bool found = false;for_each_pci_dev (pdev) { -u64 egmphys, egmlength, egmpxm; +u64 egmphys, egmlength, egmpxm, retiredpagesphys;if (has_egm_property (pdev, &egmpxm))continue ; -if (fetch_egm_property (pdev, &egmphys, &egmlength)) +if (fetch_egm_property (pdev, &egmphys, &egmlength, + &retiredpagesphys))continue ;if (!found || egmpxm < *pmin_pxm) {include/uapi/linux/egm.h new file +18
-0@@ -0,0 +1,28 @@ +/* SPDX-License-Identifier: GPL-2.0 WITH Linux-syscall-note */ +/* +* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved +*/ + +#ifndef _UAPI_LINUX_EGM_H +#define _UAPI_LINUX_EGM_H + +#include <linux/types.h> + +#define EGM_TYPE ('x') + +struct egm_retired_pages_info { +__aligned_u64 offset; +__aligned_u64 size; +}; + +struct egm_retired_pages_list {
include/uapi/linux/egm.h line 28
Quoted 10 lines
include/uapi/linux/egm.h new file +10
-0+__u32 argsz; +/* out */ +__u32 count; +/* out */ + struct egm_retired_pages_info retired_pages[]; +}; + +#define EGM_RETIRED_PAGES_LIST _IOWR(EGM_TYPE, 0x01, struct egm_retired_pages_list) + +#endif /* _UAPI_LINUX_EGM_H */
It's generally a good idea to formalize the uAPI in comments here to define the expected behavior and content rather than necessarily letting the implementation define the uAPI. Thanks,
Alex
Re: [PATCH 3/4] platform/nvidia: Handle retired ECC pages and expose via ioctl
In reply to Ankit AgrawalQuoted 11 lines
Patch implements the following:
Retired page tracking:
- Map the carveout region withmemremap()and validate the count.
- convert each physical address to an in-EGM-region offset and populate a per-EGM-device hashtable (keyed by that offset).Userspace ioctl:
- IntroduceEGM_RETIRED_PAGES_LISTioctl and add a new UAPI header Userspace (QEMU) calls this ioctl to retrieve the list of retired page offsets. QEMU then communicates the list to the VM so the guest can take appropriate action.
This can probably be two patches, it will make it easier to see the uapi
drivers/platform/nvidia/egm.c static int nvgrace_egm_mmap(struct file *file, struct vm_area_struct *vma) · line 212
Quoted 12 lines
drivers/platform/nvidia/egm.c +12
-0+static long nvgrace_egm_ioctl (struct file *file,unsigned int cmd,unsigned long arg) +{ + unsigned long minsz = offsetofend(struct egm_retired_pages_list, count); +struct egm_retired_pages_list info; +void __user *uarg = (void __user *)arg; +struct nvgrace_egm_dev *egm_dev = file->private_data; + +if (copy_from_user (&info, uarg, minsz)) +return -EFAULT ; + +if (info.argsz < minsz || !egm_dev) +return -EINVAL ;
This is missing struct zero content checking for compatability.
include/uapi/linux/egm.h line 24
Quoted 12 lines
include/uapi/linux/egm.h new file +12
-0+struct egm_retired_pages_info { +__aligned_u64 offset; +__aligned_u64 size; +}; + +struct egm_retired_pages_list { +__u32 argsz; +/* out */ +__u32 count; +/* out */ +struct egm_retired_pages_info retired_pages[]; +};
I think you are better to follow the ioctl design from iommufd..
struct egm_retired_pages_list {
__u32 size;
__u32 flags;
__u32 out_count;
__aligned_u64 retired_pages; // Pointer to struct egm_retired_pages_list[]
}We have hit in vfio again and again that these trailing structs become a problem for extension..
You can cut and paste the simplified version from fwctl_fops_ioctl()
Signature · 1 line
Jason
PATCH 4/4 platform/nvidia: Register EGM PFNMAP range with memory_failure
In reply to Ankit AgrawalEGM carveout memory is mapped directly into userspace (QEMU) and is not added to the kernel. It is not managed by the kernel page allocator and has no struct pages. The module can thus utilize the Linux memory manager's memory_failure mechanism for regions with no struct pages. The Linux MM code exposes register/unregister APIs allowing modules to register such memory regions for memory_failure handling. On the occurrence of the memory_failure, Linux MM calls the registered function if the PFN is within the range.
These dynamically occurring memory_failure pages are tracked alongside the statically populated list by the system firmware in the xarray. So the userspace can get both the information through the common EGM_RETIRED_PAGES_LIST ioctl.
Register the EGM PFN range with the memory_failure infrastructure on first open() and unregister it on the last close(). Provide a pfn_to_vma_pgoff callback that:
- Validates that the reported PFN falls within the EGM region and the current VMA.
- Converts the PFN to a file offset within the EGM region.
- Records the poisoned offset in the existing retired-pages hashtable so it is reported to userspace via EGM_RETIRED_PAGES_LIST alongside statically configured retired pages from SBIOS.
--- drivers/platform/nvidia/egm.c | 123 +++++++++++++++++++++++++++++++--- 1 file changed, 115 insertions(+), 8 deletions(-)
drivers/platform/nvidia/egm.c +115 -8
@@ -3,6 +3,7 @@* Copyright (c) 2026, NVIDIA CORPORATION & AFFILIATES. All rights reserved */ +#include <linux/memory-failure.h> #include <linux/mutex.h> #include <linux/sched/signal.h> #include <linux/sizes.h>
@@ -40,7 +41,7 @@ struct gpu_node {struct nvgrace_egm_dev {struct device device;struct cdev cdev; -/* serialises the first-open scrub */ +/* serialises the first-open scrub + PFN registration */ struct mutex open_lock;/* protected by open_lock */ unsigned int open_count;
@@ -49,6 +50,7 @@ struct nvgrace_egm_dev {* EGM region and keyed by page index. */ struct xarray retired_pages; +struct pfn_address_space pfn_address_space;phys_addr_t egmphys;size_t egmlength;phys_addr_t retiredpagesphys;
@@ -71,6 +73,104 @@ static LIST_HEAD(egm_chardevs);static void cleanup_retired_pages (struct nvgrace_egm_dev *egm_dev);static int nvgrace_egm_fetch_retired_pages (struct nvgrace_egm_dev *egm_dev); +static int pfn_memregion_offset (struct nvgrace_egm_dev *egm_dev, +unsigned long pfn, +pgoff_t *pfn_offset_in_region) +{ +unsigned long start_pfn, num_pages; + + start_pfn =PHYS_PFN (egm_dev->egmphys); + num_pages = egm_dev->egmlength >>PAGE_SHIFT ; + +if (pfn < start_pfn || pfn >= start_pfn + num_pages) +return -EFAULT ; + + *pfn_offset_in_region = pfn - start_pfn; + +return 0 ; +} + +static int track_ecc_offset (struct nvgrace_egm_dev *egm_dev, +unsigned long mem_offset) +{ +void *old; + +/* +* This runs from the pfn_to_vma_pgoff callback, which the +* memory_failure path invokes under rcu_read_lock() (see +* collect_procs_pfn()). Sleeping is not allowed in that atomic +* context and so use GFP_ATOMIC rather than GFP_KERNEL. +* +* Storing by page index makes a repeated report at the same offset a +* harmless overwrite, so no explicit de-duplication is needed. +*/ + old =xa_store (&egm_dev->retired_pages, mem_offset >>PAGE_SHIFT , +EGM_RETIRED_MARK ,GFP_ATOMIC ); +if (xa_is_err (old)) +return xa_err (old); + +return 0 ; +} + +static int nvgrace_egm_pfn_to_vma_pgoff (struct vm_area_struct *vma, +unsigned long pfn, +pgoff_t *pgoff) +{ +struct nvgrace_egm_dev *egm_dev = vma->vm_file->private_data; +pgoff_t vma_offset_in_region = vma->vm_pgoff; +pgoff_t pfn_offset_in_region; +int ret; + + ret =pfn_memregion_offset (egm_dev, pfn, &pfn_offset_in_region); +if (ret) +return ret; + +/* Ensure PFN is not before VMA's start within the region */ +if (pfn_offset_in_region < vma_offset_in_region) +return -EFAULT ; + +/* Ensure PFN is not past the VMA's end within the region */ +if (pfn_offset_in_region >= vma_offset_in_region +vma_pages (vma)) +return -EFAULT ; + +/* +* The VMA's vm_pgoff is the page offset of its start within the EGM +* region, so the region offset of the pfn is already the pgoff to +* report. +*/ + *pgoff = pfn_offset_in_region; + +/* +* Record the poisoned offset for reporting via +* EGM_RETIRED_PAGES_LIST. This bookkeeping is best-effort: its +* failure must not be reported as "vma does not map pfn", or the +* owning process would not be signalled for the poisoned page. +*/ +track_ecc_offset (egm_dev, *pgoff <<PAGE_SHIFT ); + +return 0 ; +} + +static int +nvgrace_egm_register_pfn_range (struct inode *inode, +struct nvgrace_egm_dev *egm_dev) +{ +unsigned long pfn, nr_pages; +int ret; + + pfn =PHYS_PFN (egm_dev->egmphys); + nr_pages = egm_dev->egmlength >>PAGE_SHIFT ; + + egm_dev->pfn_address_space.node.start = pfn; + egm_dev->pfn_address_space.node.last = pfn + nr_pages -1 ; + egm_dev->pfn_address_space.mapping = inode->i_mapping; + egm_dev->pfn_address_space.pfn_to_vma_pgoff = nvgrace_egm_pfn_to_vma_pgoff; + + ret =register_pfn_address_space (&egm_dev->pfn_address_space); + +return ret; +} +static int nvgrace_egm_open (struct inode *inode,struct file *file) { struct nvgrace_egm_dev *egm_dev =
@@ -84,9 +184,9 @@ static int nvgrace_egm_open(struct inode *inode, struct file *file)/* * The EGM region is a physical carveout handed to a single VM at a -* time and must be scrubbed before each handover. Take open_lock -* killably around the scrub which can run for long time, so that -* the waiters stay killable. +* time and must be scrubbed and registered with memory_failure before +* each handover. Take open_lock killably around the scrub which can +* run for long time, so that the waiters stay killable. */ if (mutex_lock_killable (&egm_dev->open_lock))return -EINTR ;
@@ -142,10 +242,15 @@ static int nvgrace_egm_open(struct inode *inode, struct file *file) remaining -= chunk_size; } + ret =nvgrace_egm_register_pfn_range (inode, egm_dev); +if (ret && ret != -EOPNOTSUPP ) + goto unlock; +/* -* Mark the device open only after the scrub completes so that a -* concurrent opener cannot observe a non-zero count and proceed -* before the region has been cleared. +* Only mark the device open once the scrub and registration have +* succeeded. On failure open_count stays 0, so this opener returns +* an error (release() never runs for it) and the next opener +* retries the scrub and registration from a clean state. */ egm_dev->open_count =1 ; ret =0 ;
@@ -162,8 +267,10 @@ static int nvgrace_egm_release(struct inode *inode, struct file *file)guard (mutex)(&egm_dev->open_lock); -if (!--egm_dev->open_count) +if (!--egm_dev->open_count) { +unregister_pfn_address_space (&egm_dev->pfn_address_space); file->private_data =NULL ; + }return 0 ; }
Signature · 1 line
2.34.1
Hi
Gentle ping on this series. Happy to respin to address the feedbacks or concerns if needed.
Thanks
Ankit Agrawal
Quoted 4 lines · Ankit Agrawal
Hi
Gentle ping on this series. Happy to respin to address the feedbacks or concerns if needed.
There is no context here :(
Also, you are adding custom ioctls for memory and drm things, why is the mm and drm people not part of the review here as this seems to interact / go around those subsystems?
And where is the userspace code that will be using this new api?
thanks,
greg k-h
Quoted 11 lines · On Tue, Aug 04, 2026 at 09:24:47AM +0200, [email protected] wrote:
Quoted 4 lines · Ankit Agrawal
Hi
Gentle ping on this series. Happy to respin to address the feedbacks or concerns if needed.
There is no context here :(
Also, you are adding custom ioctls for memory and drm things, why is the mm and drm people not part of the review here as this seems to interact / go around those subsystems?
The cover letter does explain this, the memory is CPU DRAM, so there is nothing related to DRM. It is reserved by FW so it doesn't get a struct page or zone, so there is no direct mm connection.
We don't want to waste system memory on a useless struct page here, this could be 100GB's of memory potentially.
The call to register_pfn_address_space() follows how vfio was done and this was all gone over with mm a while ago.
The single ioctl is not "memory and drm things", it is reporting a table from the BIOS indicating portions of the FW reserved memory that has been recorded as unreliable that should be avoided.
Quoted 1 line
And where is the userspace code that will be using this new api?
The cover letter hints at this, it is usable via existing qemu features using the "-object memory-backend-file" or similar. Ankit you could probably make that section a bit more specific if you respin.
Jason
Quoted 6 lines
Quoted 1 line
And where is the userspace code that will be using this new api?
The cover letter hints at this, it is usable via existing qemu features using the "-object memory-backend-file" or similar. Ankit you could probably make that section a bit more specific if you respin.
Ack, will address that in the respin.