Skip to contentSkip to conversation

gpu: nova-core: Add memory management support

PATCH Superseded Joel Fernandes 4 participants 46 messages Feb 24 – Mar 6

Subsystems (8)

v8 of 12 Superseded Latest is v12 →

lore
Share & tools
lore
#0 lore raw

v8 cover gpu: nova-core: Add memory management support

This series adds support for nova-core memory management, including VRAM allocation, PRAMIN, VMM, page table walking, and BAR 1 read/writes. These are critical for channel management, vGPU, and all other memory management uses of nova-core.

Changes from v7 to v8:
- Incorporated "Select GPU_BUDDY for VRAM allocation" patch from the dependency series (Alex).
- Significant patch reordering for better logical flow (GSP/FB patches moved earlier, page table patches, Vmm, Bar1, tests) (Alex).
- Replaced several 'as' usages with into_safe_cast() (Danilo, Alex).
- Updated BAR 1 test cases to include exercising the block size API (Eliot, Danilo).

Changes from v6 to v7:
- Addressed DMA fence signalling usecase per Danilo's feedback.

Pre v6:
- Simplified PRAMIN code (John Hubbard, Alex Courbot).
- Handled different MMU versions: ver2 versus ver3 (John Hubbard).
- Added BAR1 usecase so we have user of DRM Buddy / VMM (John Hubbard).
- Iterating over clist/buddy bindings.

All patches, along with all the dependencies (including CList), can be found at:
https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git (branch nova/mm)

Link to v7: https://lore.kernel.org/all/[email protected]/

Joel Fernandes (25):
gpu: nova-core: Select GPU_BUDDY for VRAM allocation
gpu: nova-core: Kconfig: Sort select statements alphabetically
gpu: nova-core: gsp: Return GspStaticInfo and FbLayout from boot()
gpu: nova-core: gsp: Extract usable FB region from GSP
gpu: nova-core: fb: Add usable_vram field to FbLayout
gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
docs: gpu: nova-core: Document the PRAMIN aperture mechanism
gpu: nova-core: mm: Add common memory management types
gpu: nova-core: mm: Add TLB flush support
gpu: nova-core: mm: Add GpuMm centralized memory manager
gpu: nova-core: mm: Use usable VRAM region for buddy allocator
gpu: nova-core: mm: Add common types for all page table formats
gpu: nova-core: mm: Add MMU v2 page table types
gpu: nova-core: mm: Add MMU v3 page table types
gpu: nova-core: mm: Add unified page table entry wrapper enums
gpu: nova-core: mm: Add page table walker for MMU v2/v3
gpu: nova-core: mm: Add Virtual Memory Manager
gpu: nova-core: mm: Add virtual address range tracking to VMM
gpu: nova-core: mm: Add multi-page mapping API to VMM
gpu: nova-core: Add BAR1 aperture type and size constant
gpu: nova-core: gsp: Add BAR1 PDE base accessors
gpu: nova-core: mm: Add BAR1 user interface
gpu: nova-core: mm: Add BarUser to struct Gpu and create at boot
gpu: nova-core: mm: Add BAR1 memory management self-tests
gpu: nova-core: mm: Add PRAMIN aperture self-tests

 Documentation/gpu/nova/core/pramin.rst     | 125 ++++++
 Documentation/gpu/nova/index.rst           |   1 +
 drivers/gpu/nova-core/Kconfig              |  13 +-
 drivers/gpu/nova-core/driver.rs            |   9 +-
 drivers/gpu/nova-core/fb.rs                |  26 +-
 drivers/gpu/nova-core/gpu.rs               | 130 +++++-
 drivers/gpu/nova-core/gsp/boot.rs          |  22 +-
 drivers/gpu/nova-core/gsp/commands.rs      |  18 +-
 drivers/gpu/nova-core/gsp/fw/commands.rs   |  40 ++
 drivers/gpu/nova-core/mm.rs                | 245 ++++++++++
 drivers/gpu/nova-core/mm/bar_user.rs       | 406 +++++++++++++++++
 drivers/gpu/nova-core/mm/pagetable.rs      | 453 +++++++++++++++++++
 drivers/gpu/nova-core/mm/pagetable/ver2.rs | 199 +++++++++
 drivers/gpu/nova-core/mm/pagetable/ver3.rs | 302 +++++++++++++
 drivers/gpu/nova-core/mm/pagetable/walk.rs | 218 +++++++++
 drivers/gpu/nova-core/mm/pramin.rs         | 453 +++++++++++++++++++
 drivers/gpu/nova-core/mm/tlb.rs            |  90 ++++
 drivers/gpu/nova-core/mm/vmm.rs            | 494 +++++++++++++++++++++
 drivers/gpu/nova-core/nova_core.rs         |   1 +
 drivers/gpu/nova-core/regs.rs              |  38 ++
 20 files changed, 3273 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/gpu/nova/core/pramin.rst
 create mode 100644 drivers/gpu/nova-core/mm.rs
 create mode 100644 drivers/gpu/nova-core/mm/bar_user.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver3.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs
 create mode 100644 drivers/gpu/nova-core/mm/pramin.rs
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
 create mode 100644 drivers/gpu/nova-core/mm/vmm.rs

base-commit: 2961f841b025fb234860bac26dfb7fa7cb0fb122
prerequisite-patch-id: f8778e0b72697243cfe40a8bfcc3ca1d66160b12
prerequisite-patch-id: 763b30c69f9806de97eb58bc8e3406feb6bf61b4
prerequisite-patch-id: 2cded1a588520cf7d08f51bf5b3646d5b3dc26bd
prerequisite-patch-id: 4933d6b86caab3e4630c92292cfe8e9d3b8f3524
prerequisite-patch-id: 9f58738611f42a330b00bd976fbcafe2b7e6d75e
prerequisite-patch-id: 07f81850399af27b61dcba81ad1bb24bcf977ac1

Signature · 1 line
2.34.1
#1 lore raw

PATCH v8 1/25 gpu: nova-core: Select GPU_BUDDY for VRAM allocation

nova-core will use the GPU buddy allocator for physical VRAM management. Enable it in Kconfig.

---
 drivers/gpu/nova-core/Kconfig | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/nova-core/Kconfig +1 -0
index 527920f9c4d3..809485167aff 100644
@@ -5,6 +5,7 @@ config NOVA_CORE
 	depends on RUST
 	select RUST_FW_LOADER_ABSTRACTIONS
 	select AUXILIARY_BUS
+	select GPU_BUDDY
 	default n
 	help
 	  Choose this if you want to build the Nova Core driver for Nvidia
Signature · 1 line
2.34.1
#2 lore raw

Re: [PATCH v8 01/25] gpu: nova-core: Select GPU_BUDDY for VRAM allocation

Quoted 2 lines · Joel Fernandes

nova-core will use the GPU buddy allocator for physical VRAM management. Enable it in Kconfig.

Let's merge this one-liner with the first patch of this series that actually requires GPU_BUDDY.

#3 lore raw

PATCH v8 2/25 gpu: nova-core: Kconfig: Sort select statements alphabetically

In reply to Joel Fernandes

Reorder the select statements in NOVA_CORE Kconfig to be in alphabetical order.

---
 drivers/gpu/nova-core/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/nova-core/Kconfig +1 -1
index 809485167aff..6513007bf66f 100644
@@ -3,9 +3,9 @@ config NOVA_CORE
 	depends on 64BIT
 	depends on PCI
 	depends on RUST
-	select RUST_FW_LOADER_ABSTRACTIONS
 	select AUXILIARY_BUS
 	select GPU_BUDDY
+	select RUST_FW_LOADER_ABSTRACTIONS
 	default n
 	help
 	  Choose this if you want to build the Nova Core driver for Nvidia
Signature · 1 line
2.34.1
#4 lore raw

Re: [PATCH v8 02/25] gpu: nova-core: Kconfig: Sort select statements alphabetically

Quoted 2 lines · Joel Fernandes

Reorder the select statements in NOVA_CORE Kconfig to be in alphabetical order.

Nit: please do the sorting *before* adding a new dependency - it is more logical to fix the order before adding new stuff.

Not need to do it here, I'll apply this first as it is obviously correct - please just rebase the series on drm-rust-next and resolve the minor conflict in Kconfig.

#5 lore raw

PATCH v8 3/25 gpu: nova-core: gsp: Return GspStaticInfo and FbLayout from boot()

In reply to Joel Fernandes

Refactor the GSP boot function to return the GspStaticInfo and FbLayout.

This enables access required for memory management initialization to:
- bar1_pde_base: BAR1 page directory base.
- bar2_pde_base: BAR2 page directory base.
- usable memory regions in vidmem.

---
 drivers/gpu/nova-core/gpu.rs      |  9 +++++++--
 drivers/gpu/nova-core/gsp/boot.rs | 15 ++++++++++++---
 2 files changed, 19 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/gpu.rs +7 -2
index 9b042ef1a308..5e084ec7c926 100644
@@ -18,7 +18,10 @@
     },
     fb::SysmemFlush,
     gfw,
-    gsp::Gsp,
+    gsp::{
+        commands::GetGspStaticInfoReply,
+        Gsp, //
+    },
     regs,
 };
 
@@ -252,6 +255,8 @@ pub(crate) struct Gpu {
     /// GSP runtime data. Temporarily an empty placeholder.
     #[pin]
     gsp: Gsp,
+    /// Static GPU information from GSP.
+    gsp_static_info: GetGspStaticInfoReply,
 }
 
 impl Gpu {
@@ -283,7 +288,7 @@ pub(crate) fn new<'a>(
 
             gsp <- Gsp::new(pdev),
 
-            _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
+            gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?.0 },
 
             bar: devres_bar,
         })
drivers/gpu/nova-core/gsp/boot.rs +12 -3
index be427fe26a58..7a4a0c759267 100644
@@ -32,7 +32,10 @@
     },
     gpu::Chipset,
     gsp::{
-        commands,
+        commands::{
+            self,
+            GetGspStaticInfoReply, //
+        },
         sequencer::{
             GspSequencer,
             GspSequencerParams, //
@@ -127,6 +130,12 @@ fn run_fwsec_frts(
     /// structures that the GSP will use at runtime.
     ///
     /// Upon return, the GSP is up and running, and its runtime object given as return value.
+    ///
+    /// Returns a tuple containing:
+    /// - [`GetGspStaticInfoReply`]: Static GPU information from GSP, including the BAR1 page
+    ///   directory base address needed for memory management.
+    /// - [`FbLayout`]: Frame buffer layout computed during boot, containing memory regions
+    ///   required for [`GpuMm`] initialization.
     pub(crate) fn boot(
         mut self: Pin<&mut Self>,
         pdev: &pci::Device<device::Bound>,
@@ -134,7 +143,7 @@ pub(crate) fn boot(
         chipset: Chipset,
         gsp_falcon: &Falcon<Gsp>,
         sec2_falcon: &Falcon<Sec2>,
-    ) -> Result {
+    ) -> Result<(GetGspStaticInfoReply, FbLayout)> {
         let dev = pdev.as_ref();
 
         let bios = Vbios::new(dev, bar)?;
@@ -243,6 +252,6 @@ pub(crate) fn boot(
             Err(e) => dev_warn!(pdev.as_ref(), "GPU name unavailable: {:?}\n", e),
         }
 
-        Ok(())
+        Ok((info, fb_layout))
     }
 }
Signature · 1 line
2.34.1
#6 lore raw

PATCH v8 4/25 gpu: nova-core: gsp: Extract usable FB region from GSP

In reply to Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed via usable_fb_region() API for use by the memory subsystem.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 12 ++++++++-
 drivers/gpu/nova-core/gsp/fw/commands.rs | 32 ++++++++++++++++++++++++
 2 files changed, 43 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/gsp/commands.rs +11 -1
index 8f270eca33be..c31046df3acf 100644
@@ -186,9 +186,11 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
     }
 }
 
-/// The reply from the GSP to the [`GetGspInfo`] command.
+/// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    /// First usable FB region `(base, size)` for memory allocation.
+    usable_fb_region: Option<(u64, u64)>,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
@@ -202,6 +204,7 @@ fn read(
     ) -> Result<Self, Self::InitError> {
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
+            usable_fb_region: msg.first_usable_fb_region(),
         })
     }
 }
@@ -228,6 +231,13 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {
             .to_str()
             .map_err(GpuNameError::InvalidUtf8)
     }
+
+    /// Returns the usable FB region `(base, size)` for driver allocation which is
+    /// already retrieved from the GSP.
+    #[expect(dead_code)]
+    pub(crate) fn usable_fb_region(&self) -> Option<(u64, u64)> {
+        self.usable_fb_region
+    }
 }
 
 /// Send the [`GetGspInfo`] command and awaits for its reply.
drivers/gpu/nova-core/gsp/fw/commands.rs +32 -0
index 21be44199693..ff771a4aba4f 100644
@@ -5,6 +5,7 @@
 use kernel::{device, pci};
 
 use crate::gsp::GSP_PAGE_SIZE;
+use crate::num::IntoSafeCast;
 
 use super::bindings;
 
@@ -114,6 +115,37 @@ impl GspStaticConfigInfo {
     pub(crate) fn gpu_name_str(&self) -> [u8; 64] {
         self.0.gpuNameString
     }
+
+    /// Extract the first usable FB region from GSP firmware data.
+    ///
+    /// Returns the first region suitable for driver memory allocation as a `(base, size)` tuple.
+    /// Usable regions are those that:
+    /// - Are not reserved for firmware internal use.
+    /// - Are not protected (hardware-enforced access restrictions).
+    /// - Support compression (can use GPU memory compression for bandwidth).
+    /// - Support ISO (isochronous memory for display requiring guaranteed bandwidth).
+    pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
+        let fb_info = &self.0.fbRegionInfoParams;
+        for i in 0..fb_info.numFBRegions.into_safe_cast() {
+            if let Some(reg) = fb_info.fbRegion.get(i) {
+                // Skip malformed regions where limit < base.
+                if reg.limit < reg.base {
+                    continue;
+                }
+
+                // Filter: not reserved, not protected, supports compression and ISO.
+                if reg.reserved == 0
+                    && reg.bProtected == 0
+                    && reg.supportCompressed != 0
+                    && reg.supportISO != 0
+                {
+                    let size = reg.limit - reg.base + 1;
+                    return Some((reg.base, size));
+                }
+            }
+        }
+        None
+    }
 }
 
 // SAFETY: Padding is explicit and will not contain uninitialized data.
Signature · 1 line
2.34.1
#7 lore raw

Re: [PATCH v8 04/25] gpu: nova-core: gsp: Extract usable FB region from GSP

Quoted 20 lines · Joel Fernandes
drivers/gpu/nova-core/gsp/fw/commands.rs +20 -0
+    /// Extract the first usable FB region from GSP firmware data.
+    ///
+    /// Returns the first region suitable for driver memory allocation as a `(base, size)` tuple.
+    /// Usable regions are those that:
+    /// - Are not reserved for firmware internal use.
+    /// - Are not protected (hardware-enforced access restrictions).
+    /// - Support compression (can use GPU memory compression for bandwidth).
+    /// - Support ISO (isochronous memory for display requiring guaranteed bandwidth).
+    pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
+        let fb_info = &self.0.fbRegionInfoParams;
+        for i in 0..fb_info.numFBRegions.into_safe_cast() {
+            if let Some(reg) = fb_info.fbRegion.get(i) {
+                // Skip malformed regions where limit < base.
+                if reg.limit < reg.base {
+                    continue;
+                }
+
+                // Filter: not reserved, not protected, supports compression and ISO.
+                if reg.reserved == 0
+                    && reg.bProtected == 0

drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 148

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +10 -0
+                    && reg.supportCompressed != 0
+                    && reg.supportISO != 0
+                {
+                    let size = reg.limit - reg.base + 1;
+                    return Some((reg.base, size));
+                }
+            }
+        }
+        None
+    }

No need for an explicit index and two blocks here, and we have iterator methods designed just to do just this:

    fb_info
        .fbRegion
        .iter()
        .take(fb_info.numFBRegions.into_safe_cast())
        // Skip malformed regions where limit < base.
        .filter(|reg| reg.limit >= reg.base)
        .find_map(|reg| {
            // Filter: not reserved, not protected, supports compression and ISO.
            if reg.reserved == 0
                && reg.bProtected == 0
                && reg.supportCompressed != 0
                && reg.supportISO != 0
            {
                let size = reg.limit - reg.base + 1;
                Some(reg.base..reg.base.saturating_add(size))
            } else {
                None
            }
        })

Another thing, although not too important for now: we only take advantage of the first available region in this series. This is ok for now, but Nouveau for instance does collect all regions - so we should at least have a TODO to align Nova to at least the same capability.

But this doesn't need to be done for this series; actually I'd prefer if we start simple and get the buddy allocator in place before thinking about this.

#8 lore raw

PATCH v8 5/25 gpu: nova-core: fb: Add usable_vram field to FbLayout

In reply to Joel Fernandes

Add usable_vram field to FbLayout to store the usable VRAM region for driver allocations. This is populated after GSP boot with the region extracted from GSP's fbRegionInfoParams.

FbLayout is now a two-phase structure:
1. new() computes firmware layout from hardware
2. set_usable_vram() populates usable region from GSP

The new usable_vram field represents the actual usable VRAM region (~23.7GB on a 24GB GPU GA102 Ampere GPU).

---
 drivers/gpu/nova-core/fb.rs | 26 +++++++++++++++++++++++++-
 1 file changed, 25 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/fb.rs +25 -1
index c62abcaed547..d4d0f50d7afd 100644
@@ -97,6 +97,10 @@ pub(crate) fn unregister(&self, bar: &Bar0) {
 /// Layout of the GPU framebuffer memory.
 ///
 /// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process.
+///
+/// This structure is populated in 2 steps:
+/// 1. [`FbLayout::new()`] computes firmware layout from hardware.
+/// 2. [`FbLayout::set_usable_vram()`] populates usable region from GSP response.
 #[derive(Debug)]
 pub(crate) struct FbLayout {
     /// Range of the framebuffer. Starts at `0`.
@@ -111,10 +115,16 @@ pub(crate) struct FbLayout {
     pub(crate) elf: Range<u64>,
     /// WPR2 heap.
     pub(crate) wpr2_heap: Range<u64>,
-    /// WPR2 region range, starting with an instance of `GspFwWprMeta`.
+    /// WPR2 region range, starting with an instance of [`GspFwWprMeta`].
     pub(crate) wpr2: Range<u64>,
+    /// Non-WPR heap carved before WPR2, used by GSP firmware.
     pub(crate) heap: Range<u64>,
     pub(crate) vf_partition_count: u8,
+    /// Usable VRAM region for driver allocations (from GSP `fbRegionInfoParams`).
+    ///
+    /// Initially [`None`], populated after GSP boot with usable region info.
+    #[allow(dead_code)]
+    pub(crate) usable_vram: Option<Range<u64>>,
 }
 
 impl FbLayout {
@@ -212,6 +222,20 @@ pub(crate) fn new(chipset: Chipset, bar: &Bar0, gsp_fw: &GspFirmware) -> Result<
             wpr2,
             heap,
             vf_partition_count: 0,
+            usable_vram: None,
         })
     }
+
+    /// Set the usable VRAM region from GSP response.
+    ///
+    /// Called after GSP boot with the first usable region extracted from
+    /// GSP's `fbRegionInfoParams`. Usable regions are those that:
+    /// - Are not reserved for firmware internal use.
+    /// - Are not protected (hardware-enforced access restrictions).
+    /// - Support compression (can use GPU memory compression for bandwidth).
+    /// - Support ISO (isochronous memory for display requiring guaranteed bandwidth).
+    #[allow(dead_code)]
+    pub(crate) fn set_usable_vram(&mut self, base: u64, size: u64) {
+        self.usable_vram = Some(base..base.saturating_add(size));
+    }
 }
Signature · 1 line
2.34.1
#9 lore raw

PATCH v8 6/25 gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM

In reply to Joel Fernandes

PRAMIN apertures are a crucial mechanism to direct read/write to VRAM. Add support for the same.

---
 drivers/gpu/nova-core/mm.rs        |   5 +
 drivers/gpu/nova-core/mm/pramin.rs | 292 +++++++++++++++++++++++++++++
 drivers/gpu/nova-core/nova_core.rs |   1 +
 drivers/gpu/nova-core/regs.rs      |   5 +
 4 files changed, 303 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm.rs
 create mode 100644 drivers/gpu/nova-core/mm/pramin.rs
drivers/gpu/nova-core/mm.rs new file +5 -0
new file mode 100644
index 000000000000..7a5dd4220c67
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Memory management subsystems for nova-core.
+
+pub(crate) mod pramin;
drivers/gpu/nova-core/mm/pramin.rs new file +292 -0

292 lines not on this page · raw message

drivers/gpu/nova-core/nova_core.rs +1 -0
index c1121e7c64c5..3de00db3279e 100644
@@ -13,6 +13,7 @@
 mod gfw;
 mod gpu;
 mod gsp;
+mod mm;
 mod num;
 mod regs;
 mod sbuffer;
drivers/gpu/nova-core/regs.rs +5 -0
index ea0d32f5396c..d0982e346f74 100644
@@ -102,6 +102,11 @@ fn fmt(&self, f: &mut kernel::fmt::Formatter<'_>) -> kernel::fmt::Result {
     31:16   frts_err_code as u16;
 });
 
+register!(NV_PBUS_BAR0_WINDOW @ 0x00001700, "BAR0 window control for PRAMIN access" {
+    25:24   target as u8, "Target memory (0=VRAM, 1=SYS_MEM_COH, 2=SYS_MEM_NONCOH)";
+    23:0    window_base as u32, "Window base address (bits 39:16 of FB addr)";
+});
+
 // PFB
 
 // The following two registers together hold the physical system memory address that is used by the
Signature · 1 line
2.34.1
#10 lore raw

Re: [PATCH v8 06/25] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM

Hi Joel,

Quoted 5 lines · Joel Fernandes

PRAMIN apertures are a crucial mechanism to direct read/write to VRAM. Add support for the same.

I have two sets of comments for this patch - one that is immediately actionable, the other that depends on the availability of the new Io interface. Let's start with the actionable items, I'll do the discussion about Io in another email.

Quoted 28 lines
---
 drivers/gpu/nova-core/mm.rs        |   5 +
 drivers/gpu/nova-core/mm/pramin.rs | 292 +++++++++++++++++++++++++++++
 drivers/gpu/nova-core/nova_core.rs |   1 +
 drivers/gpu/nova-core/regs.rs      |   5 +
 4 files changed, 303 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm.rs
 create mode 100644 drivers/gpu/nova-core/mm/pramin.rs
drivers/gpu/nova-core/mm.rs new file +5 -0
new file mode 100644
index 000000000000..7a5dd4220c67
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Memory management subsystems for nova-core.
+
+pub(crate) mod pramin;
drivers/gpu/nova-core/mm/pramin.rs new file +2 -0
new file mode 100644
index 000000000000..04b652d3ee4f
@@ -0,0 +1,292 @@
+// SPDX-License-Identifier: GPL-2.0
+

drivers/gpu/nova-core/mm/pramin.rs line 12

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10 -0
+//! Direct VRAM access through the PRAMIN aperture.
+//!
+//! PRAMIN provides a 1MB sliding window into VRAM through BAR0, allowing the CPU to access
+//! video memory directly. Access is managed through a two-level API:
+//!
+//! - [`Pramin`]: The parent object that owns the BAR0 reference and synchronization lock.
+//! - [`PraminWindow`]: A guard object that holds exclusive PRAMIN access for its lifetime.
+//!
+//! The PRAMIN aperture is a 1MB region at BAR0 + 0x700000 for all GPUs. The window base is
+//! controlled by the `NV_PBUS_BAR0_WINDOW` register and must be 64KB aligned.

s/must be/is - it's not like that hardware is giving us a choice anyway since we cannot even express a non-aligned value in the window register.

Quoted 96 lines
drivers/gpu/nova-core/mm/pramin.rs new file +96 -0
+//!
+//! # Examples
+//!
+//! ## Basic read/write
+//!
+//! ```no_run
+//! use crate::driver::Bar0;
+//! use crate::mm::pramin;
+//! use kernel::devres::Devres;
+//! use kernel::prelude::*;
+//! use kernel::sync::Arc;
+//!
+//! fn example(devres_bar: Arc<Devres<Bar0>>) -> Result<()> {
+//!     let pramin = Arc::pin_init(pramin::Pramin::new(devres_bar)?, GFP_KERNEL)?;
+//!     let mut window = pramin.window()?;
+//!
+//!     // Write and read back.
+//!     window.try_write32(0x100, 0xDEADBEEF)?;
+//!     let val = window.try_read32(0x100)?;
+//!     assert_eq!(val, 0xDEADBEEF);
+//!
+//!     Ok(())
+//!     // Original window position restored on drop.
+//! }
+//! ```
+//!
+//! ## Auto-repositioning across VRAM regions
+//!
+//! ```no_run
+//! use crate::driver::Bar0;
+//! use crate::mm::pramin;
+//! use kernel::devres::Devres;
+//! use kernel::prelude::*;
+//! use kernel::sync::Arc;
+//!
+//! fn example(devres_bar: Arc<Devres<Bar0>>) -> Result<()> {
+//!     let pramin = Arc::pin_init(pramin::Pramin::new(devres_bar)?, GFP_KERNEL)?;
+//!     let mut window = pramin.window()?;
+//!
+//!     // Access first 1MB region.
+//!     window.try_write32(0x100, 0x11111111)?;
+//!
+//!     // Access at 2MB - window auto-repositions.
+//!     window.try_write32(0x200000, 0x22222222)?;
+//!
+//!     // Back to first region - window repositions again.
+//!     let val = window.try_read32(0x100)?;
+//!     assert_eq!(val, 0x11111111);
+//!
+//!     Ok(())
+//! }
+//! ```
+
+#![expect(unused)]
+
+use crate::{
+    driver::Bar0,
+    num::u64_as_usize,
+    regs, //
+};
+
+use kernel::bits::genmask_u64;
+use kernel::devres::Devres;
+use kernel::io::Io;
+use kernel::new_mutex;
+use kernel::prelude::*;
+use kernel::ptr::{
+    Alignable,
+    Alignment, //
+};
+use kernel::sizes::{
+    SZ_1M,
+    SZ_64K, //
+};
+use kernel::sync::{
+    lock::mutex::MutexGuard,
+    Arc,
+    Mutex, //
+};
+
+/// PRAMIN aperture base offset in BAR0.
+const PRAMIN_BASE: usize = 0x700000;
+
+/// PRAMIN aperture size (1MB).
+const PRAMIN_SIZE: usize = SZ_1M;
+
+/// 64KB alignment for window base.
+const WINDOW_ALIGN: Alignment = Alignment::new::<SZ_64K>();
+
+/// Maximum addressable VRAM offset (40-bit address space).
+///
+/// The `NV_PBUS_BAR0_WINDOW` register has a 24-bit `window_base` field (bits 23:0) that stores
+/// bits [39:16] of the target VRAM address. This limits the addressable space to 2^40 bytes.
+const MAX_VRAM_OFFSET: usize = u64_as_usize(genmask_u64(0..=39));
+
+/// Generate a PRAMIN read accessor.

drivers/gpu/nova-core/mm/pramin.rs line 118

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10 -0
+macro_rules! define_pramin_read {
+    ($name:ident, $ty:ty) => {
+        #[doc = concat!("Read a `", stringify!($ty), "` from VRAM at the given offset.")]
+        pub(crate) fn $name(&mut self, vram_offset: usize) -> Result<$ty> {
+            // Compute window parameters without bar reference.
+            let (bar_offset, new_base) =
+                self.compute_window(vram_offset, ::core::mem::size_of::<$ty>())?;
+
+            // Update window base if needed and perform read.
+            let bar = self.bar.try_access().ok_or(ENODEV)?;

Ouch, we are calling try_access for every single read or write operation. Thankfully we can do without this - see my comments on PraminWindow a bit later.

Quoted 23 lines
drivers/gpu/nova-core/mm/pramin.rs new file +23 -0
+            if let Some(base) = new_base {
+                Self::write_window_base(&bar, base);
+                self.state.current_base = base;
+            }
+            bar.$name(bar_offset)
+        }
+    };
+}
+
+/// Generate a PRAMIN write accessor.
+macro_rules! define_pramin_write {
+    ($name:ident, $ty:ty) => {
+        #[doc = concat!("Write a `", stringify!($ty), "` to VRAM at the given offset.")]
+        pub(crate) fn $name(&mut self, vram_offset: usize, value: $ty) -> Result {
+            // Compute window parameters without bar reference.
+            let (bar_offset, new_base) =
+                self.compute_window(vram_offset, ::core::mem::size_of::<$ty>())?;
+
+            // Update window base if needed and perform write.
+            let bar = self.bar.try_access().ok_or(ENODEV)?;
+            if let Some(base) = new_base {
+                Self::write_window_base(&bar, base);
+                self.state.current_base = base;

drivers/gpu/nova-core/mm/pramin.rs line 151

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10 -0
+            }
+            bar.$name(value, bar_offset)
+        }
+    };
+}
+
+/// PRAMIN state protected by mutex.
+struct PraminState {
+    current_base: usize,
+}

This has only one member and no impl block of its own - shall we inline it?

drivers/gpu/nova-core/mm/pramin.rs line 163

Quoted 12 lines
drivers/gpu/nova-core/mm/pramin.rs new file +12 -0
+
+/// PRAMIN aperture manager.
+///
+/// Call [`Pramin::window()`] to acquire exclusive PRAMIN access.
+#[pin_data]
+pub(crate) struct Pramin {
+    bar: Arc<Devres<Bar0>>,
+    /// PRAMIN aperture state, protected by a mutex.
+    ///
+    /// # Safety
+    ///
+    /// This lock is acquired during the DMA fence signalling critical path.

nit: s/signalling/signaling

Quoted 14 lines
drivers/gpu/nova-core/mm/pramin.rs new file +14 -0
+    /// It must NEVER be held across any reclaimable CPU memory / allocations
+    /// (`GFP_KERNEL`), because the memory reclaim path can call
+    /// `dma_fence_wait()`, which would deadlock with this lock held.
+    #[pin]
+    state: Mutex<PraminState>,
+}
+
+impl Pramin {
+    /// Create a pin-initializer for PRAMIN.
+    pub(crate) fn new(bar: Arc<Devres<Bar0>>) -> Result<impl PinInit<Self>> {
+        let bar_access = bar.try_access().ok_or(ENODEV)?;
+        let current_base = Self::try_read_window_base(&bar_access)?;
+
+        Ok(pin_init!(Self {

drivers/gpu/nova-core/mm/pramin.rs line 187

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10 -0
+            bar,
+            state <- new_mutex!(PraminState { current_base }, "pramin_state"),
+        }))
+    }
+
+    /// Acquire exclusive PRAMIN access.
+    ///
+    /// Returns a [`PraminWindow`] guard that provides VRAM read/write accessors.
+    /// The [`PraminWindow`] is exclusive and only one can exist at a time.
+    pub(crate) fn window(&self) -> Result<PraminWindow<'_>> {

The name of this method is strange - we don't pass the base of any window area. It looks more like it is actually acquiring the Pramin for exclusive access.

Also fun question: what happens if we try to access an area above (or below) the available VRAM?

Quoted 5 lines
drivers/gpu/nova-core/mm/pramin.rs new file +5 -0
+        let state = self.state.lock();
+        let saved_base = state.current_base;
+        Ok(PraminWindow {
+            bar: self.bar.clone(),
+            state,

drivers/gpu/nova-core/mm/pramin.rs line 202

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10 -0
+            saved_base,
+        })
+    }
+
+    /// Read the current window base from the BAR0_WINDOW register.
+    fn try_read_window_base(bar: &Bar0) -> Result<usize> {
+        let reg = regs::NV_PBUS_BAR0_WINDOW::read(bar);
+        let base = u64::from(reg.window_base());
+        let shifted = base.checked_shl(16).ok_or(EOVERFLOW)?;
+        shifted.try_into().map_err(|_| EOVERFLOW)

This function is actually infallible.

The checked_shl cannot fail because we are shifting a u32 by 16, which will always fit into a u64. There is some Bounded code incoming that will allow us to prove that safely, but it is not available in drm-rust-next yet. For now can you use a regular shift operation with a TODO to convert to Bounded?

The last line could use num::u64_as_size to do the conversion infallibly. But actually the window base is native to the GPU, not the host CPU architecture - so it should really be a u64 anyway.

drivers/gpu/nova-core/mm/pramin.rs line 215

Quoted 13 lines
drivers/gpu/nova-core/mm/pramin.rs new file +13 -0
+    }
+}
+
+/// PRAMIN window guard for direct VRAM access.
+///
+/// This guard holds exclusive access to the PRAMIN aperture. The window auto-repositions
+/// when accessing VRAM offsets outside the current 1MB range. Original window position
+/// is saved on creation and restored on drop.
+///
+/// Only one [`PraminWindow`] can exist at a time per [`Pramin`] instance (enforced by the
+/// internal `MutexGuard`).
+pub(crate) struct PraminWindow<'a> {
+    bar: Arc<Devres<Bar0>>,

Cloning the Arc is what forces us to do a try_access on every read/write operation. Needless to say this is overkill and not efficient.

The PraminWindow already holds a reference to its Pramin, so you can just call try_access in window and store that. This turns bar into this:

bar: RevocableGuard<'a, Bar0>,

And now you can perform operations on bar directly without needing to acquire it every time.

drivers/gpu/nova-core/mm/pramin.rs line 216

Quoted 1 line
drivers/gpu/nova-core/mm/pramin.rs new file +1 -0
+    state: MutexGuard<'a, PraminState>,

I'm wondering whether we should remove the Mutex from Pramin and make its methods take &mut self (and this would thus become a &mut PraminState).

The rationale for this is that Pramin will be owned by GpuMm, which can implement its own locking policy (including across several of the items it owns) as it needs. That way Pramin also stays "pure" in that it only needs to care about abstracting the hardware.

drivers/gpu/nova-core/mm/pramin.rs line 217

Quoted 1 line
drivers/gpu/nova-core/mm/pramin.rs new file +1 -0
+    saved_base: usize,

What is the point of saving and restoring the original position? Pramin gets exclusive access to the PRAMIN area, and whenever we make an access through the window the base address can be reprogrammed. So it's not as if the original value has some particular meaning or constitutes state that deserves to be preserved. Getting rid of this behavior also lets us remove the Drop implementation.

drivers/gpu/nova-core/mm/pramin.rs line 222

Quoted 5 lines
drivers/gpu/nova-core/mm/pramin.rs new file +5 -0
+}
+
+impl PraminWindow<'_> {
+    /// Write a new window base to the BAR0_WINDOW register.
+    fn write_window_base(bar: &Bar0, base: usize) {

base should be the native type of the GPU, i.e. u64.

drivers/gpu/nova-core/mm/pramin.rs line 225

Quoted 3 lines
drivers/gpu/nova-core/mm/pramin.rs new file +3 -0
+        // CAST:
+        // - We have guaranteed that the base is within the addressable range (40-bits).
+        // - After >> 16, a 40-bit aligned base becomes 24 bits, which fits in u32.

This function does not guarantee anything of the sort - even the caller doesn't, this actually relies on the behavior of compute_window, another method.

You can enforce this limit by using a Bounded<u64, 40> as the type for base addresses. There is even staging code (not yet in drm-rust-next unfortunately) that will lets you turn it into a Bounded<u32, 32> safely to be written into the register.

Quoted 8 lines
drivers/gpu/nova-core/mm/pramin.rs new file +8 -0
+        regs::NV_PBUS_BAR0_WINDOW::default()
+            .set_window_base((base >> 16) as u32)
+            .write(bar);
+    }
+
+    /// Compute window parameters for a VRAM access.
+    ///
+    /// Returns (`bar_offset`, `new_base`) where:

drivers/gpu/nova-core/mm/pramin.rs line 243

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10 -0
+    /// - `bar_offset`: The BAR0 offset to use for the access.
+    /// - `new_base`: `Some(base)` if window needs repositioning, `None` otherwise.
+    fn compute_window(
+        &self,
+        vram_offset: usize,
+        access_size: usize,
+    ) -> Result<(usize, Option<usize>)> {
+        // Validate VRAM offset is within addressable range (40-bit address space).
+        let end_offset = vram_offset.checked_add(access_size).ok_or(EINVAL)?;
+        if end_offset > MAX_VRAM_OFFSET + 1 {

If we are going to use MAX_VRAM_OFFSET like this, let's define it as 1 << 40 and turn this into if end_offset > MAX_VRAM_OFFSET.

drivers/gpu/nova-core/mm/pramin.rs line 256

Quoted 13 lines
drivers/gpu/nova-core/mm/pramin.rs new file +13 -0
+            return Err(EINVAL);
+        }
+
+        // Calculate which 64KB-aligned base we need.
+        let needed_base = vram_offset.align_down(WINDOW_ALIGN);
+
+        // Calculate offset within the window.
+        let offset_in_window = vram_offset - needed_base;
+
+        // Check if access fits in 1MB window from this base.
+        if offset_in_window + access_size > PRAMIN_SIZE {
+            return Err(EINVAL);
+        }

Here offset_in_window cannot be larger than 64K because of the line before - this check is effectively dead code.

drivers/gpu/nova-core/mm/pramin.rs line 263

Quoted 7 lines
drivers/gpu/nova-core/mm/pramin.rs new file +7 -0
+
+        // Return bar offset and whether window needs repositioning.
+        let new_base = if self.state.current_base != needed_base {
+            Some(needed_base)
+        } else {
+            None
+        };

So maybe I am missing something, but aren't we writing a new base if the requested offset doesn't fit within [current_base..current_base+64K], despite the fact that the PRAMIN window is 1MB? This looks like a bug.

Also what happens if we access PRAMIN with decreasing addresses? Won't we be doing more window resets than we need?

Quoted 19 lines
drivers/gpu/nova-core/mm/pramin.rs new file +19 -0
+
+        Ok((PRAMIN_BASE + offset_in_window, new_base))
+    }
+
+    define_pramin_read!(try_read8, u8);
+    define_pramin_read!(try_read16, u16);
+    define_pramin_read!(try_read32, u32);
+    define_pramin_read!(try_read64, u64);
+
+    define_pramin_write!(try_write8, u8);
+    define_pramin_write!(try_write16, u16);
+    define_pramin_write!(try_write32, u32);
+    define_pramin_write!(try_write64, u64);
+}
+
+impl Drop for PraminWindow<'_> {
+    fn drop(&mut self) {
+        // Restore the original window base if it changed.
+        if self.state.current_base != self.saved_base {

drivers/gpu/nova-core/mm/pramin.rs line 292

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10 -0
+            if let Some(bar) = self.bar.try_access() {
+                Self::write_window_base(&bar, self.saved_base);
+
+                // Update state to reflect the restored base.
+                self.state.current_base = self.saved_base;
+            }
+        }
+        // MutexGuard drops automatically, releasing the lock.
+    }
+}

Let's drop this alongside the original window base restoration, which doesn't serve any purpose.

Quoted 17 lines
drivers/gpu/nova-core/nova_core.rs +1 -0
index c1121e7c64c5..3de00db3279e 100644
@@ -13,6 +13,7 @@
 mod gfw;
 mod gpu;
 mod gsp;
+mod mm;
 mod num;
 mod regs;
 mod sbuffer;
drivers/gpu/nova-core/regs.rs +0 -0
index ea0d32f5396c..d0982e346f74 100644

drivers/gpu/nova-core/regs.rs fn fmt(&self, f: &mut kernel::fmt::Formatter<'_>) -> kernel::fmt::Result { · line 106

Quoted 5 lines
drivers/gpu/nova-core/regs.rs +2 -0
@@ -102,6 +102,11 @@ fn fmt(&self, f: &mut kernel::fmt::Formatter<'_>) -> kernel::fmt::Result {
     31:16   frts_err_code as u16;
 });
 
+register!(NV_PBUS_BAR0_WINDOW @ 0x00001700, "BAR0 window control for PRAMIN access" {
+    25:24   target as u8, "Target memory (0=VRAM, 1=SYS_MEM_COH, 2=SYS_MEM_NONCOH)";

This should be converted to an enum. I understand that we only ever want to use Vram as Hopper+ don't support the other types of memories - a single-variant enum will document that fact and force us to set the correct value.

#11 lore raw

Re: [PATCH v8 06/25] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM

Quoted 4 lines · Alexandre Courbot
Quoted 1 line

+//! controlled by the NV_PBUS_BAR0_WINDOW register and must be 64KB aligned.

s/must be/is - it's not like that hardware is giving us a choice anyway since we cannot even express a non-aligned value in the window register.

Done.

Quoted 5 lines
Quoted 1 line

+ let bar = self.bar.try_access().ok_or(ENODEV)?;

Ouch, we are calling try_access for every single read or write operation.

Quoted 7 lines
Quoted 1 line

+ bar: Arc<Devres<Bar0>>,

Cloning the Arc is what forces us to do a try_access on every read/write operation. [...] you can just call try_access in window and store that. This turns bar into:

bar: RevocableGuard<'a, Bar0>,

Agreed, this is a lot better and has fewer lines of code too. I have made the change.

Quoted 6 lines

drivers/gpu/nova-core/mm/pramin.rs line 151

Quoted 3 lines
drivers/gpu/nova-core/mm/pramin.rs new file +3 -0
+struct PraminState {
+    current_base: usize,
+}

This has only one member and no impl block of its own - shall we inline it?

Agreed. Inlined to Mutex<usize>.

Quoted 3 lines
Quoted 1 line

+ /// This lock is acquired during the DMA fence signalling critical path.

nit: s/signalling/signaling

Done.

Quoted 5 lines
Quoted 1 line

+ pub(crate) fn window(&self) -> Result<PraminWindow<'_>> {

The name of this method is strange - we don't pass the base of any window area. It looks more like it is actually acquiring the Pramin for exclusive access.

How about get_window()? Open to other suggestions too.

Quoted 2 lines

Also fun question: what happens if we try to access an area above (or below) the available VRAM?

Good catch, I added further hardening to check for that in next spin. And as a result, I can also drop this check now "Validate within hardware addressable range (40-bit address space)" since it will be covered by the VRAM region check.

Also I was doing unnecessary window repositioning: we were computing a new base for every 64KB boundary crossing, fixed now.

Quoted 6 lines

drivers/gpu/nova-core/mm/pramin.rs line 202

Quoted 3 lines
drivers/gpu/nova-core/mm/pramin.rs new file +3 -0
+    fn try_read_window_base(bar: &Bar0) -> Result<usize> {
+        let shifted = base.checked_shl(16).ok_or(EOVERFLOW)?;
+        shifted.try_into().map_err(|_| EOVERFLOW)

This function is actually infallible. [...] For now can you use a regular shift operation with a TODO to convert to Bounded?

Done. Renamed to read_window_base(), uses plain shift and u64_as_usize. Added TODO for Bounded.

Quoted 8 lines
Quoted 1 line

+ saved_base: usize,

What is the point of saving and restoring the original position?

Quoted 1 line

impl Drop for PraminWindow<'_> { ...

Let's drop this alongside the original window base restoration, which doesn't serve any purpose.

I was trying to avoid side effects for other users of PRAMIN, but you are right, that is too defensive with handling this. Will simplify.

Quoted 4 lines
Quoted 1 line

+ state: MutexGuard<'a, PraminState>,

I'm wondering whether we should remove the Mutex from Pramin and make its methods take &mut self.

Actually I would like not to do that because it complicates the design - &mut self on Pramin then requires &mut GpuMm everywhere, cascading through all callers. I had tried that path initially but ended up with the internal Mutex approach and concluded it is cleaner. I think we can start with this and refine as needed - let me know if you're okay with that. Similar reasoning for the buddy allocator, where I implemented internal locking there after trying the mutable reference path.

Quoted 3 lines
Quoted 1 line

+ fn write_window_base(bar: &Bar0, base: usize) {

base should be the native type of the GPU, i.e. u64.

Makes sense. I changed it throughout the file to be that way. Btw, I left the BAR0 offsets (bar_offset, offset_in_window) remain usize since those are host MMIO offsets. We could argue that is 'an offset into VRAM though which is GPU addresses' but I guess we have to draw the line somewhere :)

Quoted 5 lines

drivers/gpu/nova-core/mm/pramin.rs line 224

Quoted 2 lines
drivers/gpu/nova-core/mm/pramin.rs new file +2 -0
+        // CAST:
+        // - We have guaranteed that the base is within the addressable range (40-bits).

This function does not guarantee anything of the sort - even the caller doesn't, this actually relies on the behavior of compute_window.

Fixed. Reworded to: "The caller (compute_window) validates that base is within the VRAM region which is always <= 40 bits. After >> 16, a 40-bit base becomes 24 bits, which fits in u32."

Quoted 4 lines
Quoted 1 line

+ if end_offset > MAX_VRAM_OFFSET + 1 {

If we are going to use MAX_VRAM_OFFSET like this, let's define it as 1 << 40 and turn this into if end_offset > MAX_VRAM_OFFSET.

Removed entirely - the 40-bit hardware check is now redundant since compute_window validates against the actual vram_region (which is always <= 40 bits) as per above.

Quoted 4 lines
Quoted 1 line

+ if offset_in_window + access_size > PRAMIN_SIZE {

Here offset_in_window cannot be larger than 64K because of the line before - this check is effectively dead code.

This was true with the old code that aligned to 64KB on every access. With the repositioning fix (we now check if the access fits in the current 1MB window first), offset_in_window can be up to ~1MB, so this check is now live and needed.

Quoted 4 lines
Quoted 1 line

+ 25:24 target as u8, "Target memory (0=VRAM, 1=SYS_MEM_COH, 2=SYS_MEM_NONCOH)";

This should be converted to an enum. [...] a single-variant enum will document that fact and force us to set the correct value.

Done. Added Bar0WindowTarget enum with a single variant.

thanks,

--
Joel Fernandes

#12 lore raw

Re: [PATCH v8 06/25] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM

In reply to Joel Fernandes
Quoted 2 lines · Joel Fernandes

PRAMIN apertures are a crucial mechanism to direct read/write to VRAM. Add support for the same.

A few design thoughts that are not immediately actionable, but might become this cycle if we get a tag with the new Io work.

Basically this feature is a prime candidate for an Io implementation. It maps onto the BAR, has a fixed 1MB size, and needs to be accessed using various sizes. It is also used to fill structured values, which the I/O projection work will also allow us to do.

The current design doesn't allow the user to explicitly set the start of the sliding window - this results in a sub-optimal usage of the hardware and more complex code in this module. At this level, we just want something that exposes the hardware as it is, i.e. "give me a view of the 1MB of VRAM starting from this 64K-aligned address".

Then on top of that we can implement another type that handles the window automatically if we want, but I don't think we will actually need it. The page table code will most likely want to set the window to the start of its structure, project it, and access it using compile-time checked offsets.

If that turns out to be insufficient, we can always compose something more complex from this basic piece - but the base Pramin should stay simple and truthful to the underlying hardware IMHO.

#13 lore raw

Re: [PATCH v8 06/25] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM

Hi Alex,

Quoted 4 lines · Alexandre Courbot

Basically this feature is a prime candidate for an Io implementation. It maps onto the BAR, has a fixed 1MB size, and needs to be accessed using various sizes. It is also used to fill structured values, which the I/O projection work will also allow us to do.

Agreed, PRAMIN maps naturally onto the Io abstraction and I had thought about this.

Quoted 5 lines

The current design doesn't allow the user to explicitly set the start of the sliding window - this results in a sub-optimal usage of the hardware and more complex code in this module. At this level, we just want something that exposes the hardware as it is, i.e. "give me a view of the 1MB of VRAM starting from this 64K-aligned address".

I reduced the complexity quite a bit. I think we can further simplify, but for now I think it makes sense to keep it this way - I am happy with all the usage sites as well. It properly abstracts the internals without having the user worry about alignment or window size, so I think it simplifies things for the user quite a bit. I have tried the alternative before and the caller side became complicated. Not saying we should not simplify it further but we can do it piecemeal. The current code works and is easy to understand. Hope that's okay with you.

Quoted 9 lines

Then on top of that we can implement another type that handles the window automatically if we want, but I don't think we will actually need it. The page table code will most likely want to set the window to the start of its structure, project it, and access it using compile-time checked offsets.

If that turns out to be insufficient, we can always compose something more complex from this basic piece - but the base Pramin should stay simple and truthful to the underlying hardware IMHO.

This is a great future improvement if it makes sense. One issue I can see is for large page tables where the burden of moving the window would fall on the user. I don't want to box us into those constraints right now (especially since we also already know that MMU v3's PRAM mechanism is different and likely needs a HAL). If we want more APIs for fixed window sizes that can never be moved, we can always add that in the future, but for now I prefer to keep this as is (let us see if you like the v9, I think you will :)). Adding to this, auto repositioning does not really trigger if there is no need, and the current API removes the burden of managing the window from the user. I already took two passes to simplify the code since the posting last year so right now in my opinion it is quite simple and even further less lines of code. It should be acceptable as a first pass IMO.

One more point on this is, we don't yet have BAR2 support so PRAMIN is the main way to directly write to VRAM, and instance memory is much larger than the window size. So PRAMIN is our primary mechanism for bootstrap (our spoke to HW experts, and they said this is acceptables).

One other thing to note: the Io trait uses usize for offsets, while I just changed VRAM addresses to u64 per your feedback on the other PRAMIN thread. So implementing Io for PRAMIN would require u64-to-usize conversions at the interface boundary. Not a blocker since offsets within the 1MB window are always small, but worth considering. Also it is still not clear to me about how to deal with Io trait's "offsets" if they are VRAM addresses". Even if we convert it to usize to satisfy Io trait, does it make sense in the Io trait itself to have them as usize?

thanks,

--
Joel Fernandes

#14 lore raw

PATCH v8 7/25 docs: gpu: nova-core: Document the PRAMIN aperture mechanism

In reply to Joel Fernandes

Add documentation for the PRAMIN aperture mechanism used by nova-core for direct VRAM access.

Nova only uses TARGET=VID_MEM for VRAM access. The SYS_MEM target values are documented for completeness but not used by the driver.

---
 Documentation/gpu/nova/core/pramin.rst | 125 +++++++++++++++++++++++++
 Documentation/gpu/nova/index.rst       |   1 +
 2 files changed, 126 insertions(+)
 create mode 100644 Documentation/gpu/nova/core/pramin.rst
Documentation/gpu/nova/core/pramin.rst new file +125 -0
new file mode 100644
index 000000000000..55ec9d920629
@@ -0,0 +1,125 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+PRAMIN aperture mechanism
+=========================
+
+.. note::
+   The following description is approximate and current as of the Ampere family.
+   It may change for future generations and is intended to assist in understanding
+   the driver code.
+
+Introduction
+============
+
+PRAMIN is a hardware aperture mechanism that provides CPU access to GPU Video RAM (VRAM) before
+the GPU's Memory Management Unit (MMU) and page tables are initialized. This 1MB sliding window,
+located at a fixed offset within BAR0, is essential for setting up page tables and other critical
+GPU data structures without relying on the GPU's MMU.
+
+Architecture Overview
+=====================
+
+The PRAMIN aperture mechanism is logically implemented by the GPU's PBUS (PCIe Bus Controller Unit)
+and provides a CPU-accessible window into VRAM through the PCIe interface::
+
+    +-----------------+    PCIe     +------------------------------+
+    |      CPU        |<----------->|           GPU                |
+    +-----------------+             |                              |
+                                    |  +----------------------+    |
+                                    |  |       PBUS           |    |
+                                    |  |  (Bus Controller)    |    |
+                                    |  |                      |    |
+                                    |  |  +--------------+<------------ (window starts at
+                                    |  |  |   PRAMIN     |    |    |     BAR0 + 0x700000)
+                                    |  |  |   Window     |    |    |
+                                    |  |  |   (1MB)      |    |    |
+                                    |  |  +--------------+    |    |
+                                    |  |         |            |    |
+                                    |  +---------|------------+    |
+                                    |            |                 |
+                                    |            v                 |
+                                    |  +----------------------+<------------ (Program PRAMIN to any
+                                    |  |       VRAM           |    |    64KB-aligned VRAM boundary)
+                                    |  |    (Several GBs)     |    |
+                                    |  |                      |    |
+                                    |  |  FB[0x000000000000]  |    |
+                                    |  |          ...         |    |
+                                    |  |  FB[0x7FFFFFFFFFF]   |    |
+                                    |  +----------------------+    |
+                                    +------------------------------+
+
+PBUS (PCIe Bus Controller) is responsible for, among other things, handling MMIO
+accesses to the BAR registers.
+
+PRAMIN Window Operation
+=======================
+
+The PRAMIN window provides a 1MB sliding aperture that can be repositioned over
+the entire VRAM address space using the ``NV_PBUS_BAR0_WINDOW`` register.
+
+Window Control Mechanism
+-------------------------
+
+The window position is controlled via the PBUS ``BAR0_WINDOW`` register::
+
+    NV_PBUS_BAR0_WINDOW Register (0x1700):
+    +-------+--------+--------------------------------------+
+    | 31:26 | 25:24  |               23:0                   |
+    | RSVD  | TARGET |            BASE_ADDR                 |
+    |       |        |        (bits 39:16 of VRAM address)  |
+    +-------+--------+--------------------------------------+
+
+    BASE_ADDR field (bits 23:0):
+    - Contains bits [39:16] of the target VRAM address
+    - Provides 40-bit (1TB) address space coverage
+    - Must be programmed with 64KB-aligned addresses
+
+    TARGET field (bits 25:24):
+    - 0x0: VRAM (Video Memory)
+    - 0x1: SYS_MEM_COH (Coherent System Memory)
+    - 0x2: SYS_MEM_NONCOH (Non-coherent System Memory)
+    - 0x3: Reserved
+
+    .. note::
+       Nova only uses TARGET=VRAM (0x0) for video memory access. The SYS_MEM
+       target values are documented here for hardware completeness but are
+       not used by the driver.
+
+64KB Alignment Requirement
+---------------------------
+
+The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is enforced
+by the ``BASE_ADDR`` field representing bits [39:16] of the target address::
+
+    VRAM Address Calculation:
+    actual_vram_addr = (BASE_ADDR << 16) + pramin_offset
+    Where:
+    - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0]
+    - pramin_offset: 20-bit offset within the PRAMIN window [0x00000-0xFFFFF]
+
+    Example Window Positioning:
+    +---------------------------------------------------------+
+    |                    VRAM Space                           |
+    |                                                         |
+    |  0x000000000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x0000FFFFF  +-----------------+                       |
+    |                                                         |
+    |       |              ^                                  |
+    |       |              | Window can slide                 |
+    |       v              | to any 64KB-aligned boundary     |
+    |                                                         |
+    |  0x123400000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x1234FFFFF  +-----------------+                       |
+    |                                                         |
+    |                       ...                               |
+    |                                                         |
+    |  0x7FFFF0000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x7FFFFFFFF  +-----------------+                       |
+    +---------------------------------------------------------+
Documentation/gpu/nova/index.rst +1 -0
index e39cb3163581..b8254b1ffe2a 100644
@@ -32,3 +32,4 @@ vGPU manager VFIO driver and the nova-drm driver.
    core/devinit
    core/fwsec
    core/falcon
+   core/pramin
Signature · 1 line
2.34.1
#15 lore raw

Re: [PATCH v8 07/25] docs: gpu: nova-core: Document the PRAMIN aperture mechanism

Quoted 74 lines · Joel Fernandes

Add documentation for the PRAMIN aperture mechanism used by nova-core for direct VRAM access.

Nova only uses TARGET=VID_MEM for VRAM access. The SYS_MEM target values are documented for completeness but not used by the driver.

---
 Documentation/gpu/nova/core/pramin.rst | 125 +++++++++++++++++++++++++
 Documentation/gpu/nova/index.rst       |   1 +
 2 files changed, 126 insertions(+)
 create mode 100644 Documentation/gpu/nova/core/pramin.rst
Documentation/gpu/nova/core/pramin.rst new file +54 -0
new file mode 100644
index 000000000000..55ec9d920629
@@ -0,0 +1,125 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+PRAMIN aperture mechanism
+=========================
+
+.. note::
+   The following description is approximate and current as of the Ampere family.
+   It may change for future generations and is intended to assist in understanding
+   the driver code.
+
+Introduction
+============
+
+PRAMIN is a hardware aperture mechanism that provides CPU access to GPU Video RAM (VRAM) before
+the GPU's Memory Management Unit (MMU) and page tables are initialized. This 1MB sliding window,
+located at a fixed offset within BAR0, is essential for setting up page tables and other critical
+GPU data structures without relying on the GPU's MMU.
+
+Architecture Overview
+=====================
+
+The PRAMIN aperture mechanism is logically implemented by the GPU's PBUS (PCIe Bus Controller Unit)
+and provides a CPU-accessible window into VRAM through the PCIe interface::
+
+    +-----------------+    PCIe     +------------------------------+
+    |      CPU        |<----------->|           GPU                |
+    +-----------------+             |                              |
+                                    |  +----------------------+    |
+                                    |  |       PBUS           |    |
+                                    |  |  (Bus Controller)    |    |
+                                    |  |                      |    |
+                                    |  |  +--------------+<------------ (window starts at
+                                    |  |  |   PRAMIN     |    |    |     BAR0 + 0x700000)
+                                    |  |  |   Window     |    |    |
+                                    |  |  |   (1MB)      |    |    |
+                                    |  |  +--------------+    |    |
+                                    |  |         |            |    |
+                                    |  +---------|------------+    |
+                                    |            |                 |
+                                    |            v                 |
+                                    |  +----------------------+<------------ (Program PRAMIN to any
+                                    |  |       VRAM           |    |    64KB-aligned VRAM boundary)
+                                    |  |    (Several GBs)     |    |
+                                    |  |                      |    |
+                                    |  |  FB[0x000000000000]  |    |
+                                    |  |          ...         |    |
+                                    |  |  FB[0x7FFFFFFFFFF]   |    |
+                                    |  +----------------------+    |
+                                    +------------------------------+
+
+PBUS (PCIe Bus Controller) is responsible for, among other things, handling MMIO
+accesses to the BAR registers.
+

Documentation/gpu/nova/core/pramin.rst line 64

Quoted 10 lines
Documentation/gpu/nova/core/pramin.rst new file +10 -0
+PRAMIN Window Operation
+=======================
+
+The PRAMIN window provides a 1MB sliding aperture that can be repositioned over
+the entire VRAM address space using the ``NV_PBUS_BAR0_WINDOW`` register.
+
+Window Control Mechanism
+-------------------------
+
+The window position is controlled via the PBUS ``BAR0_WINDOW`` register::

This repeats the sentence of PRAMIN Window Operation. Let's remove that sentence.

Documentation/gpu/nova/core/pramin.rst line 76

Quoted 12 lines
Documentation/gpu/nova/core/pramin.rst new file +12 -0
+
+    NV_PBUS_BAR0_WINDOW Register (0x1700):
+    +-------+--------+--------------------------------------+
+    | 31:26 | 25:24  |               23:0                   |
+    | RSVD  | TARGET |            BASE_ADDR                 |
+    |       |        |        (bits 39:16 of VRAM address)  |
+    +-------+--------+--------------------------------------+
+
+    BASE_ADDR field (bits 23:0):
+    - Contains bits [39:16] of the target VRAM address
+    - Provides 40-bit (1TB) address space coverage
+    - Must be programmed with 64KB-aligned addresses

This reads a bit like filler - let's turn this into a single sentence, of just keep the first point - the other two are deducible from it.

#16 lore raw

PATCH v8 8/25 gpu: nova-core: mm: Add common memory management types

In reply to Joel Fernandes

Add foundational types for GPU memory management. These types are used throughout the nova memory management subsystem for page table operations, address translation, and memory allocation.

---
 drivers/gpu/nova-core/mm.rs | 174 ++++++++++++++++++++++++++++++++++++
 1 file changed, 174 insertions(+)
drivers/gpu/nova-core/mm.rs +174 -0
index 7a5dd4220c67..a8b2e1870566 100644
@@ -2,4 +2,178 @@
 
 //! Memory management subsystems for nova-core.
 
+#![expect(dead_code)]
+
 pub(crate) mod pramin;
+
+use kernel::sizes::SZ_4K;
+
+use crate::num::u64_as_usize;
+
+/// Page size in bytes (4 KiB).
+pub(crate) const PAGE_SIZE: usize = SZ_4K;
+
+bitfield! {
+    pub(crate) struct VramAddress(u64), "Physical VRAM address in GPU video memory" {
+        11:0    offset          as u64, "Offset within 4KB page";
+        63:12   frame_number    as u64 => Pfn, "Physical frame number";
+    }
+}
+
+impl VramAddress {
+    /// Create a new VRAM address from a raw value.
+    pub(crate) const fn new(addr: u64) -> Self {
+        Self(addr)
+    }
+
+    /// Get the raw address value as `usize` (useful for MMIO offsets).
+    pub(crate) const fn raw(&self) -> usize {
+        u64_as_usize(self.0)
+    }
+
+    /// Get the raw address value as `u64`.
+    pub(crate) const fn raw_u64(&self) -> u64 {
+        self.0
+    }
+}
+
+impl PartialEq for VramAddress {
+    fn eq(&self, other: &Self) -> bool {
+        self.0 == other.0
+    }
+}
+
+impl Eq for VramAddress {}
+
+impl PartialOrd for VramAddress {
+    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+        Some(self.cmp(other))
+    }
+}
+
+impl Ord for VramAddress {
+    fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+        self.0.cmp(&other.0)
+    }
+}
+
+impl From<Pfn> for VramAddress {
+    fn from(pfn: Pfn) -> Self {
+        Self::default().set_frame_number(pfn)
+    }
+}
+
+bitfield! {
+    pub(crate) struct VirtualAddress(u64), "Virtual address in GPU address space" {
+        11:0    offset          as u64, "Offset within 4KB page";
+        20:12   l4_index        as u64, "Level 4 index (PTE)";
+        29:21   l3_index        as u64, "Level 3 index (Dual PDE)";
+        38:30   l2_index        as u64, "Level 2 index";
+        47:39   l1_index        as u64, "Level 1 index";
+        56:48   l0_index        as u64, "Level 0 index (PDB)";
+        63:12   frame_number    as u64 => Vfn, "Virtual frame number";
+    }
+}
+
+impl VirtualAddress {
+    /// Create a new virtual address from a raw value.
+    #[expect(dead_code)]
+    pub(crate) const fn new(addr: u64) -> Self {
+        Self(addr)
+    }
+
+    /// Get the page table index for a given level (0-5).
+    pub(crate) fn level_index(&self, level: u64) -> u64 {
+        match level {
+            0 => self.l0_index(),
+            1 => self.l1_index(),
+            2 => self.l2_index(),
+            3 => self.l3_index(),
+            4 => self.l4_index(),
+
+            // L5 is only used by MMU v3 (PTE level).
+            5 => self.l4_index(),
+            _ => 0,
+        }
+    }
+}
+
+impl From<Vfn> for VirtualAddress {
+    fn from(vfn: Vfn) -> Self {
+        Self::default().set_frame_number(vfn)
+    }
+}
+
+/// Physical Frame Number.
+///
+/// Represents a physical page in VRAM.
+#[repr(transparent)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub(crate) struct Pfn(u64);
+
+impl Pfn {
+    /// Create a new PFN from a frame number.
+    pub(crate) const fn new(frame_number: u64) -> Self {
+        Self(frame_number)
+    }
+
+    /// Get the raw frame number.
+    pub(crate) const fn raw(self) -> u64 {
+        self.0
+    }
+}
+
+impl From<VramAddress> for Pfn {
+    fn from(addr: VramAddress) -> Self {
+        addr.frame_number()
+    }
+}
+
+impl From<u64> for Pfn {
+    fn from(val: u64) -> Self {
+        Self(val)
+    }
+}
+
+impl From<Pfn> for u64 {
+    fn from(pfn: Pfn) -> Self {
+        pfn.0
+    }
+}
+
+/// Virtual Frame Number.
+///
+/// Represents a virtual page in GPU address space.
+#[repr(transparent)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub(crate) struct Vfn(u64);
+
+impl Vfn {
+    /// Create a new VFN from a frame number.
+    pub(crate) const fn new(frame_number: u64) -> Self {
+        Self(frame_number)
+    }
+
+    /// Get the raw frame number.
+    pub(crate) const fn raw(self) -> u64 {
+        self.0
+    }
+}
+
+impl From<VirtualAddress> for Vfn {
+    fn from(addr: VirtualAddress) -> Self {
+        addr.frame_number()
+    }
+}
+
+impl From<u64> for Vfn {
+    fn from(val: u64) -> Self {
+        Self(val)
+    }
+}
+
+impl From<Vfn> for u64 {
+    fn from(vfn: Vfn) -> Self {
+        vfn.0
+    }
+}
Signature · 1 line
2.34.1
#17 lore raw

PATCH v8 9/25 gpu: nova-core: mm: Add TLB flush support

In reply to Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 90 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 33 ++++++++++++
 3 files changed, 124 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index a8b2e1870566..ca685b5a44f3 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +90 -0

90 lines not on this page · raw message

drivers/gpu/nova-core/regs.rs +33 -0
index d0982e346f74..c948f961f307 100644
@@ -454,3 +454,36 @@ pub(crate) mod ga100 {
         0:0     display_disabled as bool;
     });
 }
+
+// MMU TLB
+
+register!(NV_TLB_FLUSH_PDB_LO @ 0x00b830a0, "TLB flush register: PDB address bits [39:8]" {
+    31:0    pdb_lo as u32, "PDB address bits [39:8]";
+});
+
+impl NV_TLB_FLUSH_PDB_LO {
+    /// Create a register value from a PDB address.
+    ///
+    /// Extracts bits [39:8] of the address and shifts it right by 8 bits.
+    pub(crate) fn from_pdb_addr(addr: u64) -> Self {
+        Self::default().set_pdb_lo(((addr >> 8) & 0xFFFF_FFFF) as u32)
+    }
+}
+
+register!(NV_TLB_FLUSH_PDB_HI @ 0x00b830a4, "TLB flush register: PDB address bits [47:40]" {
+    7:0     pdb_hi as u8, "PDB address bits [47:40]";
+});
+
+impl NV_TLB_FLUSH_PDB_HI {
+    /// Create a register value from a PDB address.
+    ///
+    /// Extracts bits [47:40] of the address and shifts it right by 40 bits.
+    pub(crate) fn from_pdb_addr(addr: u64) -> Self {
+        Self::default().set_pdb_hi(((addr >> 40) & 0xFF) as u8)
+    }
+}
+
+register!(NV_TLB_FLUSH_CTRL @ 0x00b830b0, "TLB flush control register" {
+    0:0     page_all as bool, "Invalidate all pages";
+    31:31   enable as bool, "Enable/trigger flush (clears when flush completes)";
+});
Signature · 1 line
2.34.1
#18 lore raw

PATCH v8 10/25 gpu: nova-core: mm: Add GpuMm centralized memory manager

In reply to Joel Fernandes

Introduce GpuMm as the centralized GPU memory manager that owns:
- Buddy allocator for VRAM allocation.
- PRAMIN window for direct VRAM access.
- TLB manager for translation buffer operations.

This provides clean ownership model where GpuMm provides accessor methods for its components that can be used for memory management operations.

---
 drivers/gpu/nova-core/gpu.rs | 18 ++++++++++
 drivers/gpu/nova-core/mm.rs  | 66 ++++++++++++++++++++++++++++++++++--
 2 files changed, 82 insertions(+), 2 deletions(-)
drivers/gpu/nova-core/gpu.rs +18 -0
index 5e084ec7c926..ed6c5f63b249 100644
@@ -4,8 +4,13 @@
     device,
     devres::Devres,
     fmt,
+    gpu::buddy::GpuBuddyParams,
     pci,
     prelude::*,
+    sizes::{
+        SZ_1M,
+        SZ_4K, //
+    },
     sync::Arc, //
 };
 
@@ -22,6 +27,7 @@
         commands::GetGspStaticInfoReply,
         Gsp, //
     },
+    mm::GpuMm,
     regs,
 };
 
@@ -252,6 +258,9 @@ pub(crate) struct Gpu {
     gsp_falcon: Falcon<GspFalcon>,
     /// SEC2 falcon instance, used for GSP boot up and cleanup.
     sec2_falcon: Falcon<Sec2Falcon>,
+    /// GPU memory manager owning memory management resources.
+    #[pin]
+    mm: GpuMm,
     /// GSP runtime data. Temporarily an empty placeholder.
     #[pin]
     gsp: Gsp,
@@ -286,6 +295,15 @@ pub(crate) fn new<'a>(
 
             sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset)?,
 
+            // Create GPU memory manager owning memory management resources.
+            // This will be initialized with the usable VRAM region from GSP in a later
+            // patch. For now, we use a placeholder of 1MB.
+            mm <- GpuMm::new(devres_bar.clone(), GpuBuddyParams {
+                base_offset_bytes: 0,
+                physical_memory_size_bytes: SZ_1M as u64,
+                chunk_size_bytes: SZ_4K as u64,
+            })?,
+
             gsp <- Gsp::new(pdev),
 
             gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?.0 },
drivers/gpu/nova-core/mm.rs +64 -2

73 lines not on this page · raw message

Signature · 1 line
2.34.1
#19 lore raw

PATCH v8 11/25 gpu: nova-core: mm: Use usable VRAM region for buddy allocator

In reply to Joel Fernandes

The buddy allocator manages the actual usable VRAM. On my GA102 Ampere with 24GB video memory, that is ~23.7GB on a 24GB GPU enabling proper GPU memory allocation for driver use.

---
 drivers/gpu/nova-core/gpu.rs          | 66 +++++++++++++++++++++------
 drivers/gpu/nova-core/gsp/boot.rs     |  7 ++-
 drivers/gpu/nova-core/gsp/commands.rs |  1 -
 3 files changed, 58 insertions(+), 16 deletions(-)
drivers/gpu/nova-core/gpu.rs +52 -14
index ed6c5f63b249..ba769de2f984 100644
@@ -1,5 +1,7 @@
 // SPDX-License-Identifier: GPL-2.0
 
+use core::cell::Cell;
+
 use kernel::{
     device,
     devres::Devres,
@@ -7,10 +9,7 @@
     gpu::buddy::GpuBuddyParams,
     pci,
     prelude::*,
-    sizes::{
-        SZ_1M,
-        SZ_4K, //
-    },
+    sizes::SZ_4K,
     sync::Arc, //
 };
 
@@ -28,9 +27,17 @@
         Gsp, //
     },
     mm::GpuMm,
+    num::IntoSafeCast,
     regs,
 };
 
+/// Parameters extracted from GSP boot for initializing memory subsystems.
+#[derive(Clone, Copy)]
+struct BootParams {
+    usable_vram_start: u64,
+    usable_vram_size: u64,
+}
+
 macro_rules! define_chipset {
     ({ $($variant:ident = $value:expr),* $(,)* }) =>
     {
@@ -274,6 +281,13 @@ pub(crate) fn new<'a>(
         devres_bar: Arc<Devres<Bar0>>,
         bar: &'a Bar0,
     ) -> impl PinInit<Self, Error> + 'a {
+        // Cell to share boot parameters between GSP boot and subsequent initializations.
+        // Contains usable VRAM region from FbLayout for use by the buddy allocator.
+        let boot_params: Cell<BootParams> = Cell::new(BootParams {
+            usable_vram_start: 0,
+            usable_vram_size: 0,
+        });
+
         try_pin_init!(Self {
             spec: Spec::new(pdev.as_ref(), bar).inspect(|spec| {
                 dev_info!(pdev.as_ref(),"NVIDIA ({})\n", spec);
@@ -295,18 +309,42 @@ pub(crate) fn new<'a>(
 
             sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset)?,
 
-            // Create GPU memory manager owning memory management resources.
-            // This will be initialized with the usable VRAM region from GSP in a later
-            // patch. For now, we use a placeholder of 1MB.
-            mm <- GpuMm::new(devres_bar.clone(), GpuBuddyParams {
-                base_offset_bytes: 0,
-                physical_memory_size_bytes: SZ_1M as u64,
-                chunk_size_bytes: SZ_4K as u64,
-            })?,
-
             gsp <- Gsp::new(pdev),
 
-            gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?.0 },
+            // Boot GSP and extract usable VRAM region for buddy allocator.
+            gsp_static_info: {
+                let (info, fb_layout) = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?;
+
+                let usable_vram = fb_layout.usable_vram.as_ref().ok_or_else(|| {
+                    dev_err!(pdev.as_ref(), "No usable FB regions found from GSP\n");
+                    ENODEV
+                })?;
+
+                dev_info!(
+                    pdev.as_ref(),
+                    "Using FB region: {:#x}..{:#x}\n",
+                    usable_vram.start,
+                    usable_vram.end
+                );
+
+                boot_params.set(BootParams {
+                    usable_vram_start: usable_vram.start,
+                    usable_vram_size: usable_vram.end - usable_vram.start,
+                });
+
+                info
+            },
+
+            // Create GPU memory manager owning memory management resources.
+            // Uses the usable VRAM region from GSP for buddy allocator.
+            mm <- {
+                let params = boot_params.get();
+                GpuMm::new(devres_bar.clone(), GpuBuddyParams {
+                    base_offset_bytes: params.usable_vram_start,
+                    physical_memory_size_bytes: params.usable_vram_size,
+                    chunk_size_bytes: SZ_4K.into_safe_cast(),
+                })?
+            },
 
             bar: devres_bar,
         })
drivers/gpu/nova-core/gsp/boot.rs +6 -1
index 7a4a0c759267..bc4446282613 100644
@@ -150,7 +150,7 @@ pub(crate) fn boot(
 
         let gsp_fw = KBox::pin_init(GspFirmware::new(dev, chipset, FIRMWARE_VERSION), GFP_KERNEL)?;
 
-        let fb_layout = FbLayout::new(chipset, bar, &gsp_fw)?;
+        let mut fb_layout = FbLayout::new(chipset, bar, &gsp_fw)?;
         dev_dbg!(dev, "{:#x?}\n", fb_layout);
 
         Self::run_fwsec_frts(dev, gsp_falcon, bar, &bios, &fb_layout)?;
@@ -252,6 +252,11 @@ pub(crate) fn boot(
             Err(e) => dev_warn!(pdev.as_ref(), "GPU name unavailable: {:?}\n", e),
         }
 
+        // Populate usable VRAM from GSP response.
+        if let Some((base, size)) = info.usable_fb_region() {
+            fb_layout.set_usable_vram(base, size);
+        }
+
         Ok((info, fb_layout))
     }
 }
drivers/gpu/nova-core/gsp/commands.rs +0 -1
index c31046df3acf..fc9ba08f9f8d 100644
@@ -234,7 +234,6 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {
 
     /// Returns the usable FB region `(base, size)` for driver allocation which is
     /// already retrieved from the GSP.
-    #[expect(dead_code)]
     pub(crate) fn usable_fb_region(&self) -> Option<(u64, u64)> {
         self.usable_fb_region
     }
Signature · 1 line
2.34.1
#20 lore raw

Re: [PATCH v8 11/25] gpu: nova-core: mm: Use usable VRAM region for buddy allocator

On Wed Feb 25, 2026 at 7:53 AM JST, Joel Fernandes wrote:
<snip>

Quoted 14 lines
drivers/gpu/nova-core/gpu.rs +0 -9
@@ -295,18 +309,42 @@ pub(crate) fn new<'a>(
 
             sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset)?,
 
-            // Create GPU memory manager owning memory management resources.
-            // This will be initialized with the usable VRAM region from GSP in a later
-            // patch. For now, we use a placeholder of 1MB.
-            mm <- GpuMm::new(devres_bar.clone(), GpuBuddyParams {
-                base_offset_bytes: 0,
-                physical_memory_size_bytes: SZ_1M as u64,
-                chunk_size_bytes: SZ_4K as u64,
-            })?,
-
             gsp <- Gsp::new(pdev),

drivers/gpu/nova-core/gpu.rs pub(crate) fn new<'a>( · line 321

Quoted 10 lines
drivers/gpu/nova-core/gpu.rs +8 -1
 
-            gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?.0 },
+            // Boot GSP and extract usable VRAM region for buddy allocator.
+            gsp_static_info: {
+                let (info, fb_layout) = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?;
+
+                let usable_vram = fb_layout.usable_vram.as_ref().ok_or_else(|| {
+                    dev_err!(pdev.as_ref(), "No usable FB regions found from GSP\n");
+                    ENODEV
+                })?;

The chain through which we obtain usable_vram is very inefficient and much more complex than it needs to be.

fb_layout is used as a transport for usable_vram, but usable_vram is the only member of it that we ever use. In that case, why not return usable_vram directly?

This is all the better as usable_vram is added as an Option in FbLayout, not because None is a valid state but because FbLayout is already constructed by the time we obtain usable_vram. So None is just a value that tells the caller "please return an error". Now we can remove the option altogether, drop patch 5, and have boot return the error for us if usable_vram cannot be obtained.

But let's not stop here. usable_vram is first obtained as part of the GetGspStaticInfo reply. It starts its life as (u64, u64) before being morphed into a Range<u64> when stored in FbLayout. Since its tuple state is never useful, let's store it as a Range<u64> in GetGspStaticInfoReply to begin with and, since we cannot continue without it anyway, let's make commands::get_gsp_info() fail if it cannot build it. That way we don't even need to store it as an Option.

And now, since boot already returns the GetGspStaticInfoReply, we can directly access it just by making usable_vram public - no need for boot to return a tuple anymore.

Once you have that, something interesting happens: you don't need to change the gsp_static_info arm at all in this patch. All the new code can be moved to the mm arm, which is where it is actually useful anyway.

The subsequent patches that make use of boot_params in other arms can also access the exact same data using gsp_static_info:

- bar1_pde_base is also accessible from gsp_static_info (let's also make it public instead of adding an accessor method),
- The other boot_params used to build the GpuMM can be reconstructed from usable_vram.

Which means boot_params and BootParams can be removed (and the Cell import), and we have something simpler and more direct that takes ~60 less LoCs.

In other words, this arm remains as it was:

gsp_static_info: gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?,

drivers/gpu/nova-core/gpu.rs pub(crate) fn new<'a>( · line 340

Quoted 3 lines
drivers/gpu/nova-core/gpu.rs +3 -0
+            // Create GPU memory manager owning memory management resources.
+            // Uses the usable VRAM region from GSP for buddy allocator.
+            mm <- {

this arm can now be:

    let usable_vram = &gsp_static_info.usable_fb_region;

    dev_info!(
        pdev.as_ref(),
        "Using FB region: {:#x}..{:#x}\n",
        usable_vram.start,
        usable_vram.end
    );

    GpuMm::new(devres_bar.clone(), GpuBuddyParams {
        base_offset_bytes: usable_vram.start,
        physical_memory_size_bytes: usable_vram.end - usable_vram.start,
        chunk_size_bytes: SZ_4K.into_safe_cast(),
    })?
Quoted 24 lines
drivers/gpu/nova-core/gpu.rs +7 -0
+                let params = boot_params.get();
+                GpuMm::new(devres_bar.clone(), GpuBuddyParams {
+                    base_offset_bytes: params.usable_vram_start,
+                    physical_memory_size_bytes: params.usable_vram_size,
+                    chunk_size_bytes: SZ_4K.into_safe_cast(),
+                })?
+            },
 
             bar: devres_bar,
         })
drivers/gpu/nova-core/gsp/boot.rs +1 -1
index 7a4a0c759267..bc4446282613 100644
@@ -150,7 +150,7 @@ pub(crate) fn boot(
 
         let gsp_fw = KBox::pin_init(GspFirmware::new(dev, chipset, FIRMWARE_VERSION), GFP_KERNEL)?;
 
-        let fb_layout = FbLayout::new(chipset, bar, &gsp_fw)?;
+        let mut fb_layout = FbLayout::new(chipset, bar, &gsp_fw)?;
         dev_dbg!(dev, "{:#x?}\n", fb_layout);
 
         Self::run_fwsec_frts(dev, gsp_falcon, bar, &bios, &fb_layout)?;

drivers/gpu/nova-core/gsp/boot.rs pub(crate) fn boot( · line 259

Quoted 8 lines
drivers/gpu/nova-core/gsp/boot.rs +5 -0
@@ -252,6 +252,11 @@ pub(crate) fn boot(
             Err(e) => dev_warn!(pdev.as_ref(), "GPU name unavailable: {:?}\n", e),
         }
 
+        // Populate usable VRAM from GSP response.
+        if let Some((base, size)) = info.usable_fb_region() {
+            fb_layout.set_usable_vram(base, size);
+        }
+

And this change is not needed anymore.

#21 lore raw

Re: [PATCH v8 11/25] gpu: nova-core: mm: Use usable VRAM region for buddy allocator

On Sun Mar 1, 2026 at 9:56 PM JST, Alexandre Courbot wrote:
<snip>

Quoted 6 lines

This is all the better as usable_vram is added as an Option in FbLayout, not because None is a valid state but because FbLayout is already constructed by the time we obtain usable_vram. So None is just a value that tells the caller "please return an error". Now we can remove the option altogether, drop patch 5, and have boot return the error for us if usable_vram cannot be obtained.

Correction: patches 3 and 5 won't be needed anymore and can be dropped, it seems.

#22 lore raw

Re: [PATCH v8 11/25] gpu: nova-core: mm: Use usable VRAM region for buddy allocator

In reply to Alexandre Courbot

Hi Alex,

Quoted 6 lines · Alexandre Courbot

The chain through which we obtain usable_vram is very inefficient and much more complex than it needs to be.
[...]
Which means boot_params and BootParams can be removed (and the Cell import), and we have something simpler and more direct that takes ~60 less LoCs.

I like the simplification and I've made those changes! Currently testing further.

One concern though (both with the original code and after my changes):
FbLayout is ephemeral, it is computed during boot and then thrown away. I think the rest of nova-core should be aware of it. Other subsystems may need to know the FB layout in the future. Of course we can do that later, but maybe it will be less churn if we store FbLayout for longer term reference after boot, for usage say by future mm code etc. May also help debugging, say debugfs needs to dump the frame buffer layout etc.

Quoted 1 line

Correction: patches 3 and 5 won't be needed anymore and can be dropped,> it seems.

Ack, with the simplificiation the extra patches can be dropped.

thanks,

--
Joel Fernandes

#23 lore raw

PATCH v8 12/25 gpu: nova-core: mm: Add common types for all page table formats

In reply to Joel Fernandes

Add common page table types shared between MMU v2 and v3. These types are hardware-agnostic and used by both MMU versions.

---
 drivers/gpu/nova-core/mm.rs           |   1 +
 drivers/gpu/nova-core/mm/pagetable.rs | 149 ++++++++++++++++++++++++++
 2 files changed, 150 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable.rs
drivers/gpu/nova-core/mm.rs +1 -0
index a3c84738bac0..7a76af313d53 100644
@@ -4,6 +4,7 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
 
drivers/gpu/nova-core/mm/pagetable.rs new file +149 -0

149 lines not on this page · raw message

Signature · 1 line
2.34.1
#24 lore raw

PATCH v8 13/25 gpu: nova-core: mm: Add MMU v2 page table types

In reply to Joel Fernandes

Add page table entry and directory structures for MMU version 2 used by Turing/Ampere/Ada GPUs.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   1 +
 drivers/gpu/nova-core/mm/pagetable/ver2.rs | 199 +++++++++++++++++++++
 2 files changed, 200 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
index aea06e5da4ff..925063fde45d 100644
@@ -7,6 +7,7 @@
 //! - Memory aperture types for PDEs and PTEs
 
 #![expect(dead_code)]
+pub(crate) mod ver2;
 
 use crate::gpu::Architecture;
 
drivers/gpu/nova-core/mm/pagetable/ver2.rs new file +199 -0

199 lines not on this page · raw message

Signature · 1 line
2.34.1
#25 lore raw

PATCH v8 14/25 gpu: nova-core: mm: Add MMU v3 page table types

In reply to Joel Fernandes

Add page table entry and directory structures for MMU version 3 used by Hopper and later GPUs.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   1 +
 drivers/gpu/nova-core/mm/pagetable/ver3.rs | 302 +++++++++++++++++++++
 2 files changed, 303 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver3.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
index 925063fde45d..5263a8f56529 100644
@@ -8,6 +8,7 @@
 
 #![expect(dead_code)]
 pub(crate) mod ver2;
+pub(crate) mod ver3;
 
 use crate::gpu::Architecture;
 
drivers/gpu/nova-core/mm/pagetable/ver3.rs new file +302 -0

302 lines not on this page · raw message

Signature · 1 line
2.34.1
#26 lore raw

PATCH v8 15/25 gpu: nova-core: mm: Add unified page table entry wrapper enums

In reply to Joel Fernandes

Add unified Pte, Pde, and DualPde wrapper enums that abstract over MMU v2 and v3 page table entry formats. These enums allow the page table walker and VMM to work with both MMU versions.

Each unified type:
- Takes MmuVersion parameter in constructors
- Wraps both ver2 and ver3 variants
- Delegates method calls to the appropriate variant

This enables version-agnostic page table operations while keeping version-specific implementation details encapsulated in the ver2 and ver3 modules.

---
 drivers/gpu/nova-core/mm/pagetable.rs | 301 ++++++++++++++++++++++++++
 1 file changed, 301 insertions(+)
drivers/gpu/nova-core/mm/pagetable.rs +301 -0
index 5263a8f56529..33acb7053fbe 100644
@@ -10,6 +10,14 @@
 pub(crate) mod ver2;
 pub(crate) mod ver3;
 
+use kernel::prelude::*;
+
+use super::{
+    pramin,
+    Pfn,
+    VramAddress,
+    PAGE_SIZE, //
+};
 use crate::gpu::Architecture;
 
 /// MMU version enumeration.
@@ -77,6 +85,74 @@ pub(crate) const fn as_index(&self) -> u64 {
     }
 }
 
+impl MmuVersion {
+    /// Get the `PDE` levels (excluding PTE level) for page table walking.
+    pub(crate) fn pde_levels(&self) -> &'static [PageTableLevel] {
+        match self {
+            Self::V2 => ver2::PDE_LEVELS,
+            Self::V3 => ver3::PDE_LEVELS,
+        }
+    }
+
+    /// Get the PTE level for this MMU version.
+    pub(crate) fn pte_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::PTE_LEVEL,
+            Self::V3 => ver3::PTE_LEVEL,
+        }
+    }
+
+    /// Get the dual PDE level (128-bit entries) for this MMU version.
+    pub(crate) fn dual_pde_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::DUAL_PDE_LEVEL,
+            Self::V3 => ver3::DUAL_PDE_LEVEL,
+        }
+    }
+
+    /// Get the number of PDE levels for this MMU version.
+    pub(crate) fn pde_level_count(&self) -> usize {
+        self.pde_levels().len()
+    }
+
+    /// Get the entry size in bytes for a given level.
+    pub(crate) fn entry_size(&self, level: PageTableLevel) -> usize {
+        if level == self.dual_pde_level() {
+            16 // 128-bit dual PDE
+        } else {
+            8 // 64-bit PDE/PTE
+        }
+    }
+
+    /// Get the number of entries per page table page for a given level.
+    pub(crate) fn entries_per_page(&self, level: PageTableLevel) -> usize {
+        PAGE_SIZE / self.entry_size(level)
+    }
+
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
+    }
+}
+
 /// Memory aperture for Page Table Entries (`PTE`s).
 ///
 /// Determines which memory region the `PTE` points to.
@@ -149,3 +225,228 @@ fn from(val: AperturePde) -> Self {
         val as u8
     }
 }
+
+/// Unified Page Table Entry wrapper for both MMU v2 and v3 `PTE`
+/// types, allowing the walker to work with either format.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum Pte {
+    /// MMU v2 `PTE` (Turing/Ampere/Ada).
+    V2(ver2::Pte),
+    /// MMU v3 `PTE` (Hopper+).
+    V3(ver3::Pte),
+}
+
+impl Pte {
+    /// Create a `PTE` from a raw `u64` value for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, val: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::new(val)),
+            MmuVersion::V3 => Self::V3(ver3::Pte::new(val)),
+        }
+    }
+
+    /// Create an invalid `PTE` for the given MMU version.
+    pub(crate) fn invalid(version: MmuVersion) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::invalid()),
+            MmuVersion::V3 => Self::V3(ver3::Pte::invalid()),
+        }
+    }
+
+    /// Create a valid `PTE` for video memory.
+    pub(crate) fn new_vram(version: MmuVersion, pfn: Pfn, writable: bool) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::new_vram(pfn, writable)),
+            MmuVersion::V3 => Self::V3(ver3::Pte::new_vram(pfn, writable)),
+        }
+    }
+
+    /// Check if this `PTE` is valid.
+    pub(crate) fn is_valid(&self) -> bool {
+        match self {
+            Self::V2(p) => p.valid(),
+            Self::V3(p) => p.valid(),
+        }
+    }
+
+    /// Get the physical frame number.
+    pub(crate) fn frame_number(&self) -> Pfn {
+        match self {
+            Self::V2(p) => p.frame_number(),
+            Self::V3(p) => p.frame_number(),
+        }
+    }
+
+    /// Get the raw `u64` value.
+    pub(crate) fn raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(p) => p.raw_u64(),
+            Self::V3(p) => p.raw_u64(),
+        }
+    }
+
+    /// Read a `PTE` from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let val = window.try_read64(addr.raw())?;
+        Ok(Self::new(mmu_version, val))
+    }
+
+    /// Write this `PTE` to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.raw_u64())
+    }
+}
+
+/// Unified Page Directory Entry wrapper for both MMU v2 and v3 `PDE`.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum Pde {
+    /// MMU v2 `PDE` (Turing/Ampere/Ada).
+    V2(ver2::Pde),
+    /// MMU v3 `PDE` (Hopper+).
+    V3(ver3::Pde),
+}
+
+impl Pde {
+    /// Create a `PDE` from a raw `u64` value for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, val: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::new(val)),
+            MmuVersion::V3 => Self::V3(ver3::Pde::new(val)),
+        }
+    }
+
+    /// Create a valid `PDE` pointing to a page table in video memory.
+    pub(crate) fn new_vram(version: MmuVersion, table_pfn: Pfn) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::new_vram(table_pfn)),
+            MmuVersion::V3 => Self::V3(ver3::Pde::new_vram(table_pfn)),
+        }
+    }
+
+    /// Create an invalid `PDE` for the given MMU version.
+    pub(crate) fn invalid(version: MmuVersion) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::invalid()),
+            MmuVersion::V3 => Self::V3(ver3::Pde::invalid()),
+        }
+    }
+
+    /// Check if this `PDE` is valid.
+    pub(crate) fn is_valid(&self) -> bool {
+        match self {
+            Self::V2(p) => p.is_valid(),
+            Self::V3(p) => p.is_valid(),
+        }
+    }
+
+    /// Get the VRAM address of the page table.
+    pub(crate) fn table_vram_address(&self) -> VramAddress {
+        match self {
+            Self::V2(p) => p.table_vram_address(),
+            Self::V3(p) => p.table_vram_address(),
+        }
+    }
+
+    /// Get the raw `u64` value.
+    pub(crate) fn raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(p) => p.raw_u64(),
+            Self::V3(p) => p.raw_u64(),
+        }
+    }
+
+    /// Read a `PDE` from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let val = window.try_read64(addr.raw())?;
+        Ok(Self::new(mmu_version, val))
+    }
+
+    /// Write this `PDE` to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.raw_u64())
+    }
+}
+
+/// Unified Dual Page Directory Entry wrapper for both MMU v2 and v3 [`DualPde`].
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum DualPde {
+    /// MMU v2 [`DualPde`] (Turing/Ampere/Ada).
+    V2(ver2::DualPde),
+    /// MMU v3 [`DualPde`] (Hopper+).
+    V3(ver3::DualPde),
+}
+
+impl DualPde {
+    /// Create a [`DualPde`] from raw 128-bit value (two `u64`s) for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, big: u64, small: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::DualPde::new(big, small)),
+            MmuVersion::V3 => Self::V3(ver3::DualPde::new(big, small)),
+        }
+    }
+
+    /// Create a [`DualPde`] with only the small page table pointer set.
+    pub(crate) fn new_small(version: MmuVersion, table_pfn: Pfn) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::DualPde::new_small(table_pfn)),
+            MmuVersion::V3 => Self::V3(ver3::DualPde::new_small(table_pfn)),
+        }
+    }
+
+    /// Check if the small page table pointer is valid.
+    pub(crate) fn has_small(&self) -> bool {
+        match self {
+            Self::V2(d) => d.has_small(),
+            Self::V3(d) => d.has_small(),
+        }
+    }
+
+    /// Get the small page table VRAM address.
+    pub(crate) fn small_vram_address(&self) -> VramAddress {
+        match self {
+            Self::V2(d) => d.small.table_vram_address(),
+            Self::V3(d) => d.small.table_vram_address(),
+        }
+    }
+
+    /// Get the raw `u64` value of the big PDE.
+    pub(crate) fn big_raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(d) => d.big.raw_u64(),
+            Self::V3(d) => d.big.raw_u64(),
+        }
+    }
+
+    /// Get the raw `u64` value of the small PDE.
+    pub(crate) fn small_raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(d) => d.small.raw_u64(),
+            Self::V3(d) => d.small.raw_u64(),
+        }
+    }
+
+    /// Read a dual PDE (128-bit) from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let lo = window.try_read64(addr.raw())?;
+        let hi = window.try_read64(addr.raw() + 8)?;
+        Ok(Self::new(mmu_version, lo, hi))
+    }
+
+    /// Write this dual PDE (128-bit) to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.big_raw_u64())?;
+        window.try_write64(addr.raw() + 8, self.small_raw_u64())
+    }
+}
Signature · 1 line
2.34.1
#27 lore raw

PATCH v8 16/25 gpu: nova-core: mm: Add page table walker for MMU v2/v3

In reply to Joel Fernandes

Add the page table walker implementation that traverses the page table hierarchy for both MMU v2 (5-level) and MMU v3 (6-level) to resolve virtual addresses to physical addresses or find PTE locations.

Currently only v2 has been tested (nova-core currently boots pre-hopper) with some initial prepatory work done for v3.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   1 +
 drivers/gpu/nova-core/mm/pagetable/walk.rs | 218 +++++++++++++++++++++
 2 files changed, 219 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
index 33acb7053fbe..7ebea4cb8437 100644
@@ -9,6 +9,7 @@
 #![expect(dead_code)]
 pub(crate) mod ver2;
 pub(crate) mod ver3;
+pub(crate) mod walk;
 
 use kernel::prelude::*;
 
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +218 -0

218 lines not on this page · raw message

Signature · 1 line
2.34.1
reviewer
#28 lore raw

Re: [PATCH v8 16/25] gpu: nova-core: mm: Add page table walker for MMU v2/v3

Quoted 45 lines · Joel Fernandes

Add the page table walker implementation that traverses the page table hierarchy for both MMU v2 (5-level) and MMU v3 (6-level) to resolve virtual addresses to physical addresses or find PTE locations.

Currently only v2 has been tested (nova-core currently boots pre-hopper) with some initial prepatory work done for v3.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   1 +
 drivers/gpu/nova-core/mm/pagetable/walk.rs | 218 +++++++++++++++++++++
 2 files changed, 219 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
index 33acb7053fbe..7ebea4cb8437 100644
@@ -9,6 +9,7 @@
 #![expect(dead_code)]
 pub(crate) mod ver2;
 pub(crate) mod ver3;
+pub(crate) mod walk;
 
 use kernel::prelude::*;
 
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +12 -0
new file mode 100644
index 000000000000..023226af8816
@@ -0,0 +1,218 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Page table walker implementation for NVIDIA GPUs.
+//!
+//! This module provides page table walking functionality for MMU v2 and v3.
+//! The walker traverses the page table hierarchy to resolve virtual addresses
+//! to physical addresses or to find PTE locations.
+//!
+//! # Page Table Hierarchy
+//!
+//! ## MMU v2 (Turing/Ampere/Ada) - 5 levels
+//!

drivers/gpu/nova-core/mm/pagetable/walk.rs line 22

Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +10 -0
+//! ```text
+//!     +-------+     +-------+     +-------+     +---------+     +-------+
+//!     | PDB   |---->|  L1   |---->|  L2   |---->| L3 Dual |---->|  L4   |
+//!     | (L0)  |     |       |     |       |     | PDE     |     | (PTE) |
+//!     +-------+     +-------+     +-------+     +---------+     +-------+
+//!       64-bit        64-bit        64-bit        128-bit         64-bit
+//!        PDE           PDE           PDE        (big+small)        PTE
+//! ```
+//!
+//! ## MMU v3 (Hopper+) - 6 levels

I think this is called "4 levels" and "5 levels" in kernel MM rather than "5 levels" and "6 levels".

Best,
Gary

Quoted 13 lines
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +13 -0
+//!
+//! ```text
+//!     +-------+     +-------+     +-------+     +-------+     +---------+     +-------+
+//!     | PDB   |---->|  L1   |---->|  L2   |---->|  L3   |---->| L4 Dual |---->|  L5   |
+//!     | (L0)  |     |       |     |       |     |       |     | PDE     |     | (PTE) |
+//!     +-------+     +-------+     +-------+     +-------+     +---------+     +-------+
+//!       64-bit        64-bit        64-bit        64-bit        128-bit         64-bit
+//!        PDE           PDE           PDE           PDE        (big+small)        PTE
+//! ```
+//!
+//! # Result of a page table walk
+//!
+//! The walker returns a [`WalkResult`] indicating the outcome.
#29 lore raw

Re: [PATCH v8 16/25] gpu: nova-core: mm: Add page table walker for MMU v2/v3

Quoted 7 lines · Gary Guo

drivers/gpu/nova-core/mm/pagetable/walk.rs line 22

Quoted 3 lines · Joel Fernandes
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +2 -0
+//! ## MMU v2 (Turing/Ampere/Ada) - 5 levels
[...]
+//! ## MMU v3 (Hopper+) - 6 levels

I think this is called "4 levels" and "5 levels" in kernel MM rather than "5 levels" and "6 levels".

Actually, I think "5 levels" and "6 levels" is correct even by x86 kernel MM convention. In x86 "4-level paging", the 4 levels are PGD, PUD, PMD, PTE - the root page directory (PGD) IS counted as one of the 4 levels. Similarly, for the GPU MMU, counting the root PDB (L0) as a level gives us 5 levels for v2 (PDB/L0 through L4/PTE) and 6 levels for v3 (PDB/L0 through L5/PTE).

This is also consistent with NVIDIA's own hardware definitions in the OpenRM headers (dev_mmu.h for Turing and Hopper) which define the page table entries for each of these levels. The virtual address bitfield spans L0 (bits 56:48) through L4 (bits 20:12) for v2, giving 5 distinct page table levels.

FWIW, the existing nouveau driver also uses this convention - NVKM_VMM_LEVELS_MAX is defined as 6 in nvkm/subdev/mmu/vmm.c, and the GH100 page table descriptors in vmmgh100.c list all 6 levels.

Signature · 1 line
Joel Fernandes
reviewer
#30 lore raw

Re: [PATCH v8 16/25] gpu: nova-core: mm: Add page table walker for MMU v2/v3

Quoted 15 lines · Joel Fernandes
Quoted 7 lines · Gary Guo

drivers/gpu/nova-core/mm/pagetable/walk.rs line 22

Quoted 3 lines · Joel Fernandes
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +2 -0
+//! ## MMU v2 (Turing/Ampere/Ada) - 5 levels
[...]
+//! ## MMU v3 (Hopper+) - 6 levels

I think this is called "4 levels" and "5 levels" in kernel MM rather than "5 levels" and "6 levels".

Actually, I think "5 levels" and "6 levels" is correct even by x86 kernel MM convention. In x86 "4-level paging", the 4 levels are PGD, PUD, PMD, PTE - the root page directory (PGD) IS counted as one of the 4 levels. Similarly, for the GPU MMU, counting the root PDB (L0) as a level gives us 5 levels for v2 (PDB/L0 through L4/PTE) and 6 levels for v3 (PDB/L0 through L5/PTE).

Quoted 8 lines

This is also consistent with NVIDIA's own hardware definitions in the OpenRM headers (dev_mmu.h for Turing and Hopper) which define the page table entries for each of these levels. The virtual address bitfield spans L0 (bits 56:48) through L4 (bits 20:12) for v2, giving 5 distinct page table levels.

FWIW, the existing nouveau driver also uses this convention - NVKM_VMM_LEVELS_MAX is defined as 6 in nvkm/subdev/mmu/vmm.c, and the GH100 page table descriptors in vmmgh100.c list all 6 levels.

So PDB is not just a single address, but a list of page table entries? If that's the case, then the number of levels is indeed correct, but reading the code gives me an impression otherwise.

Best,
Gary

#31 lore raw

Re: [PATCH v8 16/25] gpu: nova-core: mm: Add page table walker for MMU v2/v3

Quoted 3 lines · Gary Guo

So PDB is not just a single address, but a list of page table entries? If that's the case, then the number of levels is indeed correct, but reading the code gives me an impression otherwise.

Correct - PDB (Page Directory Base) is not a single address/pointer but a full page table level containing a table of entries.

#32 lore raw

PATCH v8 17/25 gpu: nova-core: mm: Add Virtual Memory Manager

In reply to Joel Fernandes

Add the Virtual Memory Manager (VMM) infrastructure for GPU address space management. Each Vmm instance manages a single address space identified by its Page Directory Base (PDB) address, used for Channel, BAR1 and BAR2 mappings.

Mapping APIs and virtual address range tracking are added in later commits.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/vmm.rs | 63 +++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/vmm.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 7a76af313d53..594ac93497aa 100644
@@ -7,6 +7,7 @@
 pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
+pub(crate) mod vmm;
 
 use kernel::{
     devres::Devres,
drivers/gpu/nova-core/mm/vmm.rs new file +63 -0

63 lines not on this page · raw message

Signature · 1 line
2.34.1
#33 lore raw

PATCH v8 18/25 gpu: nova-core: mm: Add virtual address range tracking to VMM

In reply to Joel Fernandes

Add virtual address range tracking to the VMM using a buddy allocator. This enables contiguous virtual address range allocation for mappings.

---
 drivers/gpu/nova-core/mm/vmm.rs | 98 +++++++++++++++++++++++++++++----
 1 file changed, 87 insertions(+), 11 deletions(-)
drivers/gpu/nova-core/mm/vmm.rs +87 -11
index 0e1b0d668c57..d17571db2e2d 100644
@@ -9,19 +9,34 @@
 #![allow(dead_code)]
 
 use kernel::{
-    gpu::buddy::AllocatedBlocks,
-    prelude::*, //
+    gpu::buddy::{
+        AllocatedBlocks,
+        BuddyFlags,
+        GpuBuddy,
+        GpuBuddyAllocParams,
+        GpuBuddyParams, //
+    },
+    prelude::*,
+    sizes::SZ_4K, //
 };
 
-use crate::mm::{
-    pagetable::{
-        walk::{PtWalk, WalkResult},
-        MmuVersion, //
+use core::ops::Range;
+
+use crate::{
+    mm::{
+        pagetable::{
+            walk::{PtWalk, WalkResult},
+            MmuVersion, //
+        },
+        GpuMm,
+        Pfn,
+        Vfn,
+        VramAddress,
+        PAGE_SIZE, //
+    },
+    num::{
+        IntoSafeCast, //
     },
-    GpuMm,
-    Pfn,
-    Vfn,
-    VramAddress, //
 };
 
 /// Virtual Memory Manager for a GPU address space.
@@ -34,23 +49,84 @@ pub(crate) struct Vmm {
     pub(crate) mmu_version: MmuVersion,
     /// Page table allocations required for mappings.
     page_table_allocs: KVec<Pin<KBox<AllocatedBlocks>>>,
+    /// Buddy allocator for virtual address range tracking.
+    virt_buddy: GpuBuddy,
 }
 
 impl Vmm {
     /// Create a new [`Vmm`] for the given Page Directory Base address.
-    pub(crate) fn new(pdb_addr: VramAddress, mmu_version: MmuVersion) -> Result<Self> {
+    ///
+    /// The [`Vmm`] will manage a virtual address space of `va_size` bytes.
+    pub(crate) fn new(
+        pdb_addr: VramAddress,
+        mmu_version: MmuVersion,
+        va_size: u64,
+    ) -> Result<Self> {
         // Only MMU v2 is supported for now.
         if mmu_version != MmuVersion::V2 {
             return Err(ENOTSUPP);
         }
 
+        let virt_buddy = GpuBuddy::new(GpuBuddyParams {
+            base_offset_bytes: 0,
+            physical_memory_size_bytes: va_size,
+            chunk_size_bytes: SZ_4K.into_safe_cast(),
+        })?;
+
         Ok(Self {
             pdb_addr,
             mmu_version,
             page_table_allocs: KVec::new(),
+            virt_buddy,
         })
     }
 
+    /// Allocate a contiguous virtual frame number range.
+    ///
+    /// # Arguments
+    ///
+    /// - `num_pages`: Number of pages to allocate.
+    /// - `va_range`: `None` = allocate anywhere, `Some(range)` = constrain allocation to the given
+    ///   range.
+    pub(crate) fn alloc_vfn_range(
+        &self,
+        num_pages: usize,
+        va_range: Option<Range<u64>>,
+    ) -> Result<(Vfn, Pin<KBox<AllocatedBlocks>>)> {
+        let np: u64 = num_pages.into_safe_cast();
+        let size_bytes: u64 = np
+            .checked_mul(PAGE_SIZE.into_safe_cast())
+            .ok_or(EOVERFLOW)?;
+
+        let (start, end) = match va_range {
+            Some(r) => {
+                let range_size = r.end.checked_sub(r.start).ok_or(EOVERFLOW)?;
+                if range_size != size_bytes {
+                    return Err(EINVAL);
+                }
+                (r.start, r.end)
+            }
+            None => (0, 0),
+        };
+
+        let params = GpuBuddyAllocParams {
+            start_range_address: start,
+            end_range_address: end,
+            size_bytes,
+            min_block_size_bytes: SZ_4K.into_safe_cast(),
+            buddy_flags: BuddyFlags::try_new(BuddyFlags::CONTIGUOUS_ALLOCATION)?,
+        };
+
+        let alloc = KBox::pin_init(self.virt_buddy.alloc_blocks(&params), GFP_KERNEL)?;
+
+        // Get the starting offset of the first block (only block as range is contiguous).
+        let offset = alloc.iter().next().ok_or(ENOMEM)?.offset();
+        let page_size: u64 = PAGE_SIZE.into_safe_cast();
+        let vfn = Vfn::new(offset / page_size);
+
+        Ok((vfn, alloc))
+    }
+
     /// Read the [`Pfn`] for a mapped [`Vfn`] if one is mapped.
     pub(crate) fn read_mapping(&self, mm: &GpuMm, vfn: Vfn) -> Result<Option<Pfn>> {
         let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
Signature · 1 line
2.34.1
#34 lore raw

PATCH v8 19/25 gpu: nova-core: mm: Add multi-page mapping API to VMM

In reply to Joel Fernandes

Add the page table mapping and unmapping API to the Virtual Memory Manager, implementing a two-phase prepare/execute model suitable for use both inside and outside the DMA fence signalling critical path.

---
 drivers/gpu/nova-core/mm/vmm.rs | 359 +++++++++++++++++++++++++++++++-
 1 file changed, 357 insertions(+), 2 deletions(-)
drivers/gpu/nova-core/mm/vmm.rs +357 -2
index d17571db2e2d..c639e06c7d57 100644
@@ -17,16 +17,26 @@
         GpuBuddyParams, //
     },
     prelude::*,
+    rbtree::{RBTree, RBTreeNode},
     sizes::SZ_4K, //
 };
 
+use core::cell::Cell;
 use core::ops::Range;
 
 use crate::{
     mm::{
         pagetable::{
-            walk::{PtWalk, WalkResult},
-            MmuVersion, //
+            walk::{
+                PtWalk,
+                WalkPdeResult,
+                WalkResult, //
+            },
+            DualPde,
+            MmuVersion,
+            PageTableLevel,
+            Pde,
+            Pte, //
         },
         GpuMm,
         Pfn,
@@ -51,6 +61,74 @@ pub(crate) struct Vmm {
     page_table_allocs: KVec<Pin<KBox<AllocatedBlocks>>>,
     /// Buddy allocator for virtual address range tracking.
     virt_buddy: GpuBuddy,
+    /// Prepared PT pages pending PDE installation, keyed by `install_addr`.
+    ///
+    /// Populated by `Vmm` mapping prepare phase and drained in the execute phase.
+    /// Shared by all pending maps in the `Vmm`, thus preventing races where 2
+    /// maps might be trying to install the same page table/directory entry pointer.
+    pt_pages: RBTree<VramAddress, PreparedPtPage>,
+}
+
+/// A pre-allocated and zeroed page table page.
+///
+/// Created during the mapping prepare phase and consumed during the mapping execute phase.
+/// Stored in an [`RBTree`] keyed by the PDE slot address (`install_addr`).
+struct PreparedPtPage {
+    /// The allocated and zeroed page table page.
+    alloc: Pin<KBox<AllocatedBlocks>>,
+    /// Page table level -- needed to determine if this PT page is for a dual PDE.
+    level: PageTableLevel,
+}
+
+/// Multi-page prepared mapping -- VA range allocated, ready for execute.
+///
+/// Produced by [`Vmm::prepare_map()`], consumed by [`Vmm::execute_map()`].
+/// The struct owns the VA space allocation between prepare and execute phases.
+pub(crate) struct PreparedMapping {
+    vfn_start: Vfn,
+    num_pages: usize,
+    vfn_alloc: Pin<KBox<AllocatedBlocks>>,
+}
+
+/// Result of a mapping operation -- tracks the active mapped range.
+///
+/// Returned by [`Vmm::execute_map()`] and [`Vmm::map_pages()`].
+/// Owns the VA allocation; the VA range is freed when this is dropped.
+/// Callers must call [`Vmm::unmap_pages()`] before dropping to invalidate
+/// PTEs (dropping only frees the VA range, not the PTE entries).
+pub(crate) struct MappedRange {
+    pub(crate) vfn_start: Vfn,
+    pub(crate) num_pages: usize,
+    /// VA allocation -- freed when [`MappedRange`] is dropped.
+    _vfn_alloc: Pin<KBox<AllocatedBlocks>>,
+    /// Logs a warning if dropped without unmapping.
+    _drop_guard: MustUnmapGuard,
+}
+
+/// Guard that logs a warning once if a [`MappedRange`] is dropped without
+/// calling [`Vmm::unmap_pages()`].
+struct MustUnmapGuard {
+    armed: Cell<bool>,
+}
+
+impl MustUnmapGuard {
+    const fn new() -> Self {
+        Self {
+            armed: Cell::new(true),
+        }
+    }
+
+    fn disarm(&self) {
+        self.armed.set(false);
+    }
+}
+
+impl Drop for MustUnmapGuard {
+    fn drop(&mut self) {
+        if self.armed.get() {
+            kernel::pr_warn!("MappedRange dropped without calling unmap_pages()\n");
+        }
+    }
 }
 
 impl Vmm {
@@ -78,6 +156,7 @@ pub(crate) fn new(
             mmu_version,
             page_table_allocs: KVec::new(),
             virt_buddy,
+            pt_pages: RBTree::new(),
         })
     }
 
@@ -136,4 +215,280 @@ pub(crate) fn read_mapping(&self, mm: &GpuMm, vfn: Vfn) -> Result<Option<Pfn>> {
             WalkResult::Unmapped { .. } | WalkResult::PageTableMissing => Ok(None),
         }
     }
+
+    /// Allocate and zero a physical page table page for a specific PDE slot.
+    /// Called during the map prepare phase.
+    fn alloc_and_zero_page_table(
+        &mut self,
+        mm: &GpuMm,
+        level: PageTableLevel,
+    ) -> Result<PreparedPtPage> {
+        let params = GpuBuddyAllocParams {
+            start_range_address: 0,
+            end_range_address: 0,
+            size_bytes: SZ_4K.into_safe_cast(),
+            min_block_size_bytes: SZ_4K.into_safe_cast(),
+            buddy_flags: BuddyFlags::try_new(0)?,
+        };
+        let blocks = KBox::pin_init(mm.buddy().alloc_blocks(&params), GFP_KERNEL)?;
+
+        // Get page's VRAM address from the allocation.
+        let page_vram = VramAddress::new(blocks.iter().next().ok_or(ENOMEM)?.offset());
+
+        // Zero via PRAMIN.
+        let mut window = mm.pramin().window()?;
+        let base = page_vram.raw();
+        for off in (0..PAGE_SIZE).step_by(8) {
+            window.try_write64(base + off, 0)?;
+        }
+
+        Ok(PreparedPtPage {
+            alloc: blocks,
+            level,
+        })
+    }
+
+    /// Ensure all intermediate page table pages are prepared for a [`Vfn`]. Just
+    /// finds out which PDE pages are missing, allocates pages for them, and defers
+    /// installation to the execute phase.
+    ///
+    /// PRAMIN is released before each allocation and re-acquired after. Memory
+    /// allocations outside of holding this lock to prevent deadlocks with fence signalling
+    /// critical path.
+    fn ensure_pte_path(&mut self, mm: &GpuMm, vfn: Vfn) -> Result {
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let max_iter = 2 * self.mmu_version.pde_level_count();
+
+        // Keep looping until all PDE levels are resolved.
+        for _ in 0..max_iter {
+            let mut window = mm.pramin().window()?;
+
+            // Walk PDE levels. The closure checks self.pt_pages for prepared-but-uninstalled
+            // pages, letting the walker continue through them as if they were installed in HW.
+            // The walker keeps calling the closure to get these "prepared but not installed" pages.
+            let result = walker.walk_pde_levels(&mut window, vfn, |install_addr| {
+                self.pt_pages
+                    .get(&install_addr)
+                    .and_then(|p| Some(VramAddress::new(p.alloc.iter().next()?.offset())))
+            })?;
+
+            match result {
+                WalkPdeResult::Complete { .. } => {
+                    // All PDE levels resolved.
+                    return Ok(());
+                }
+                WalkPdeResult::Missing {
+                    install_addr,
+                    level,
+                } => {
+                    // Drop PRAMIN before allocation.
+                    drop(window);
+                    let page = self.alloc_and_zero_page_table(mm, level)?;
+                    let node = RBTreeNode::new(install_addr, page, GFP_KERNEL)?;
+                    let old = self.pt_pages.insert(node);
+                    if old.is_some() {
+                        kernel::pr_warn_once!(
+                            "VMM: duplicate install_addr in pt_pages (internal consistency error)\n"
+                        );
+                        return Err(EIO);
+                    }
+                    // Loop: re-acquire PRAMIN and re-walk from root.
+                }
+            }
+        }
+
+        kernel::pr_warn!(
+            "VMM: ensure_pte_path: loop exhausted after {} iters (VFN {:?})\n",
+            max_iter,
+            vfn
+        );
+        Err(EIO)
+    }
+
+    /// Prepare resources for mapping `num_pages` pages.
+    ///
+    /// Allocates a contiguous VA range, then walks the hierarchy per-VFN to prepare pages
+    /// for all missing PDEs. Returns a [`PreparedMapping`] with the VA allocation.
+    ///
+    /// If `va_range` is not `None`, the VA range is constrained to the given range. Safe
+    /// to call outside the fence signalling critical path.
+    pub(crate) fn prepare_map(
+        &mut self,
+        mm: &GpuMm,
+        num_pages: usize,
+        va_range: Option<Range<u64>>,
+    ) -> Result<PreparedMapping> {
+        if num_pages == 0 {
+            return Err(EINVAL);
+        }
+
+        // Pre-reserve so execute_map() can use push_within_capacity (no alloc in
+        // fence signalling critical path).
+        // Upper bound on page table pages needed for the full tree (PTE pages + PDE
+        // pages at all levels).
+        let pt_upper_bound = self.mmu_version.pt_pages_upper_bound(num_pages);
+        self.page_table_allocs.reserve(pt_upper_bound, GFP_KERNEL)?;
+
+        // Allocate contiguous VA range.
+        let (vfn_start, vfn_alloc) = self.alloc_vfn_range(num_pages, va_range)?;
+
+        // Walk the hierarchy per-VFN to prepare pages for all missing PDEs.
+        for i in 0..num_pages {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(vfn_start.raw() + i_u64);
+            self.ensure_pte_path(mm, vfn)?;
+        }
+
+        Ok(PreparedMapping {
+            vfn_start,
+            num_pages,
+            vfn_alloc,
+        })
+    }
+
+    /// Execute a prepared multi-page mapping.
+    ///
+    /// Drain prepared PT pages and install PDEs followed by single TLB flush.
+    pub(crate) fn execute_map(
+        &mut self,
+        mm: &GpuMm,
+        prepared: PreparedMapping,
+        pfns: &[Pfn],
+        writable: bool,
+    ) -> Result<MappedRange> {
+        if pfns.len() != prepared.num_pages {
+            return Err(EINVAL);
+        }
+
+        let PreparedMapping {
+            vfn_start,
+            num_pages,
+            vfn_alloc,
+        } = prepared;
+
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let mut window = mm.pramin().window()?;
+
+        // First, drain self.pt_pages, install all pending PDEs.
+        let mut cursor = self.pt_pages.cursor_front_mut();
+        while let Some(c) = cursor {
+            let (next, node) = c.remove_current();
+            let (install_addr, page) = node.to_key_value();
+            let page_vram = VramAddress::new(page.alloc.iter().next().ok_or(ENOMEM)?.offset());
+
+            if page.level == self.mmu_version.dual_pde_level() {
+                let new_dpde = DualPde::new_small(self.mmu_version, Pfn::from(page_vram));
+                new_dpde.write(&mut window, install_addr)?;
+            } else {
+                let new_pde = Pde::new_vram(self.mmu_version, Pfn::from(page_vram));
+                new_pde.write(&mut window, install_addr)?;
+            }
+
+            // Track the allocated pages in the `Vmm`.
+            self.page_table_allocs
+                .push_within_capacity(page.alloc)
+                .map_err(|_| ENOMEM)?;
+
+            cursor = next;
+        }
+
+        // Next, write PTEs (all PDEs now installed in HW).
+        for (i, &pfn) in pfns.iter().enumerate() {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(vfn_start.raw() + i_u64);
+            let result = walker.walk_to_pte_lookup_with_window(&mut window, vfn)?;
+
+            match result {
+                WalkResult::Unmapped { pte_addr } | WalkResult::Mapped { pte_addr, .. } => {
+                    let pte = Pte::new_vram(self.mmu_version, pfn, writable);
+                    pte.write(&mut window, pte_addr)?;
+                }
+                WalkResult::PageTableMissing => {
+                    kernel::pr_warn_once!("VMM: page table missing for VFN {vfn:?}\n");
+                    return Err(EIO);
+                }
+            }
+        }
+
+        drop(window);
+
+        // Finally, flush the TLB.
+        mm.tlb().flush(self.pdb_addr)?;
+
+        Ok(MappedRange {
+            vfn_start,
+            num_pages,
+            _vfn_alloc: vfn_alloc,
+            _drop_guard: MustUnmapGuard::new(),
+        })
+    }
+
+    /// Map pages doing prepare and execute in the same call.
+    ///
+    /// This is a convenience wrapper for callers outside the fence signalling critical
+    /// path (e.g., BAR mappings). For DRM usecases, [`Vmm::prepare_map()`] and
+    /// [`Vmm::execute_map()`] will be called separately.
+    pub(crate) fn map_pages(
+        &mut self,
+        mm: &GpuMm,
+        pfns: &[Pfn],
+        va_range: Option<Range<u64>>,
+        writable: bool,
+    ) -> Result<MappedRange> {
+        if pfns.is_empty() {
+            return Err(EINVAL);
+        }
+
+        // Check if provided VA range is sufficient (if provided).
+        if let Some(ref range) = va_range {
+            let required: u64 = pfns
+                .len()
+                .checked_mul(PAGE_SIZE)
+                .ok_or(EOVERFLOW)?
+                .into_safe_cast();
+            let available = range.end.checked_sub(range.start).ok_or(EINVAL)?;
+            if available < required {
+                return Err(EINVAL);
+            }
+        }
+
+        let prepared = self.prepare_map(mm, pfns.len(), va_range)?;
+        self.execute_map(mm, prepared, pfns, writable)
+    }
+
+    /// Unmap all pages in a [`MappedRange`] with a single TLB flush.
+    ///
+    /// Takes the range by value (consumes it), then invalidates PTEs for the range,
+    /// flushes the TLB, then drops the range (freeing the VA). PRAMIN lock is held.
+    pub(crate) fn unmap_pages(&mut self, mm: &GpuMm, range: MappedRange) -> Result {
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let invalid_pte = Pte::invalid(self.mmu_version);
+
+        let mut window = mm.pramin().window()?;
+        for i in 0..range.num_pages {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(range.vfn_start.raw() + i_u64);
+            let result = walker.walk_to_pte_lookup_with_window(&mut window, vfn)?;
+
+            match result {
+                WalkResult::Mapped { pte_addr, .. } | WalkResult::Unmapped { pte_addr } => {
+                    invalid_pte.write(&mut window, pte_addr)?;
+                }
+                WalkResult::PageTableMissing => {
+                    continue;
+                }
+            }
+        }
+        drop(window);
+
+        mm.tlb().flush(self.pdb_addr)?;
+
+        // TODO: Internal page table pages (PDE, PTE pages) are still kept around.
+        // This is by design as repeated maps/unmaps will be fast. As a future TODO,
+        // we can add a reclaimer here to reclaim if VRAM is short. For now, the PT
+        // pages are dropped once the `Vmm` is dropped.
+
+        range._drop_guard.disarm(); // Unmap complete, Ok to drop MappedRange.
+        Ok(())
+    }
 }
Signature · 1 line
2.34.1
#35 lore raw

PATCH v8 20/25 gpu: nova-core: Add BAR1 aperture type and size constant

In reply to Joel Fernandes

Add BAR1_SIZE constant and Bar1 type alias for the 256MB BAR1 aperture. These are prerequisites for BAR1 memory access functionality.

---
 drivers/gpu/nova-core/driver.rs | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/driver.rs +7 -1
index 5a4cc047bcfc..f30ffa45cf13 100644
@@ -13,7 +13,10 @@
         Vendor, //
     },
     prelude::*,
-    sizes::SZ_16M,
+    sizes::{
+        SZ_16M,
+        SZ_256M, //
+    },
     sync::Arc, //
 };
 
@@ -28,6 +31,7 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
+pub(crate) const BAR1_SIZE: usize = SZ_256M;
 
 // For now we only support Ampere which can use up to 47-bit DMA addresses.
 //
@@ -38,6 +42,8 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
+#[expect(dead_code)]
+pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
 
 kernel::pci_device_table!(
     PCI_TABLE,
Signature · 1 line
2.34.1
#36 lore raw

PATCH v8 21/25 gpu: nova-core: gsp: Add BAR1 PDE base accessors

In reply to Joel Fernandes

Add accessor methods to GspStaticConfigInfo for retrieving the BAR1 Page Directory Entry base addresses from GSP-RM firmware.

These addresses point to the root page tables for BAR1 virtual memory spaces. The page tables are set up by GSP-RM during GPU initialization.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 8 ++++++++
 drivers/gpu/nova-core/gsp/fw/commands.rs | 8 ++++++++
 2 files changed, 16 insertions(+)
drivers/gpu/nova-core/gsp/commands.rs +8 -0
index fc9ba08f9f8d..22bd61e2a67e 100644
@@ -189,6 +189,7 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
 /// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    bar1_pde_base: u64,
     /// First usable FB region `(base, size)` for memory allocation.
     usable_fb_region: Option<(u64, u64)>,
 }
@@ -204,6 +205,7 @@ fn read(
     ) -> Result<Self, Self::InitError> {
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
+            bar1_pde_base: msg.bar1_pde_base(),
             usable_fb_region: msg.first_usable_fb_region(),
         })
     }
@@ -232,6 +234,12 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {
             .map_err(GpuNameError::InvalidUtf8)
     }
 
+    /// Returns the BAR1 Page Directory Entry base address.
+    #[expect(dead_code)]
+    pub(crate) fn bar1_pde_base(&self) -> u64 {
+        self.bar1_pde_base
+    }
+
     /// Returns the usable FB region `(base, size)` for driver allocation which is
     /// already retrieved from the GSP.
     pub(crate) fn usable_fb_region(&self) -> Option<(u64, u64)> {
drivers/gpu/nova-core/gsp/fw/commands.rs +8 -0
index ff771a4aba4f..307f48670e6d 100644
@@ -116,6 +116,14 @@ impl GspStaticConfigInfo {
         self.0.gpuNameString
     }
 
+    /// Returns the BAR1 Page Directory Entry base address.
+    ///
+    /// This is the root page table address for BAR1 virtual memory,
+    /// set up by GSP-RM firmware.
+    pub(crate) fn bar1_pde_base(&self) -> u64 {
+        self.0.bar1PdeBase
+    }
+
     /// Extract the first usable FB region from GSP firmware data.
     ///
     /// Returns the first region suitable for driver memory allocation as a `(base, size)` tuple.
Signature · 1 line
2.34.1
#37 lore raw

PATCH v8 22/25 gpu: nova-core: mm: Add BAR1 user interface

In reply to Joel Fernandes

Add the BAR1 user interface for CPU access to GPU video memory through the BAR1 aperture.

---
 drivers/gpu/nova-core/driver.rs      |   1 -
 drivers/gpu/nova-core/mm.rs          |   1 +
 drivers/gpu/nova-core/mm/bar_user.rs | 156 +++++++++++++++++++++++++++
 3 files changed, 157 insertions(+), 1 deletion(-)
 create mode 100644 drivers/gpu/nova-core/mm/bar_user.rs
drivers/gpu/nova-core/driver.rs +0 -1
index f30ffa45cf13..d8b2e967ba4c 100644
@@ -42,7 +42,6 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
-#[expect(dead_code)]
 pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
 
 kernel::pci_device_table!(
drivers/gpu/nova-core/mm.rs +1 -0
index 594ac93497aa..42529e2339a1 100644
@@ -4,6 +4,7 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod bar_user;
 pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
drivers/gpu/nova-core/mm/bar_user.rs new file +156 -0

156 lines not on this page · raw message

Signature · 1 line
2.34.1
#38 lore raw

PATCH v8 23/25 gpu: nova-core: mm: Add BarUser to struct Gpu and create at boot

In reply to Joel Fernandes

Add a BarUser field to struct Gpu and eagerly create it during GPU initialization. The BarUser provides the BAR1 user interface for CPU access to GPU virtual memory through the GPU's MMU.

The BarUser is initialized using BAR1 PDE base address from GSP static info, MMU version and BAR1 size obtained from platform device.

---
 drivers/gpu/nova-core/gpu.rs          | 22 +++++++++++++++++++++-
 drivers/gpu/nova-core/gsp/commands.rs |  1 -
 2 files changed, 21 insertions(+), 2 deletions(-)
drivers/gpu/nova-core/gpu.rs +21 -1
index ba769de2f984..4281487ca52e 100644
@@ -26,7 +26,12 @@
         commands::GetGspStaticInfoReply,
         Gsp, //
     },
-    mm::GpuMm,
+    mm::{
+        bar_user::BarUser,
+        pagetable::MmuVersion,
+        GpuMm,
+        VramAddress, //
+    },
     num::IntoSafeCast,
     regs,
 };
@@ -36,6 +41,7 @@
 struct BootParams {
     usable_vram_start: u64,
     usable_vram_size: u64,
+    bar1_pde_base: u64,
 }
 
 macro_rules! define_chipset {
@@ -273,6 +279,8 @@ pub(crate) struct Gpu {
     gsp: Gsp,
     /// Static GPU information from GSP.
     gsp_static_info: GetGspStaticInfoReply,
+    /// BAR1 user interface for CPU access to GPU virtual memory.
+    bar_user: BarUser,
 }
 
 impl Gpu {
@@ -286,6 +294,7 @@ pub(crate) fn new<'a>(
         let boot_params: Cell<BootParams> = Cell::new(BootParams {
             usable_vram_start: 0,
             usable_vram_size: 0,
+            bar1_pde_base: 0,
         });
 
         try_pin_init!(Self {
@@ -330,6 +339,7 @@ pub(crate) fn new<'a>(
                 boot_params.set(BootParams {
                     usable_vram_start: usable_vram.start,
                     usable_vram_size: usable_vram.end - usable_vram.start,
+                    bar1_pde_base: info.bar1_pde_base(),
                 });
 
                 info
@@ -346,6 +356,16 @@ pub(crate) fn new<'a>(
                 })?
             },
 
+            // Create BAR1 user interface for CPU access to GPU virtual memory.
+            // Uses the BAR1 PDE base from GSP and full BAR1 size for VA space.
+            bar_user: {
+                let params = boot_params.get();
+                let pdb_addr = VramAddress::new(params.bar1_pde_base);
+                let mmu_version = MmuVersion::from(spec.chipset.arch());
+                let bar1_size = pdev.resource_len(1)?;
+                BarUser::new(pdb_addr, mmu_version, bar1_size)?
+            },
+
             bar: devres_bar,
         })
     }
drivers/gpu/nova-core/gsp/commands.rs +0 -1
index 22bd61e2a67e..8b3fa29c57f1 100644
@@ -235,7 +235,6 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> {
     }
 
     /// Returns the BAR1 Page Directory Entry base address.
-    #[expect(dead_code)]
     pub(crate) fn bar1_pde_base(&self) -> u64 {
         self.bar1_pde_base
     }
Signature · 1 line
2.34.1
#39 lore raw

PATCH v8 24/25 gpu: nova-core: mm: Add BAR1 memory management self-tests

In reply to Joel Fernandes

Add self-tests for BAR1 access during driver probe when CONFIG_NOVA_MM_SELFTESTS is enabled (default disabled). This results in testing the Vmm, GPU buddy allocator and BAR1 region all of which should function correctly for the tests to pass.

---
 drivers/gpu/nova-core/Kconfig        |  10 ++
 drivers/gpu/nova-core/driver.rs      |   2 +
 drivers/gpu/nova-core/gpu.rs         |  42 +++++
 drivers/gpu/nova-core/mm/bar_user.rs | 250 +++++++++++++++++++++++++++
 4 files changed, 304 insertions(+)
drivers/gpu/nova-core/Kconfig +10 -0
index 6513007bf66f..35de55aabcfc 100644
@@ -15,3 +15,13 @@ config NOVA_CORE
 	  This driver is work in progress and may not be functional.
 
 	  If M is selected, the module will be called nova_core.
+
+config NOVA_MM_SELFTESTS
+	bool "Memory management self-tests"
+	depends on NOVA_CORE
+	help
+	  Enable self-tests for the memory management subsystem. When enabled,
+	  tests are run during GPU probe to verify PRAMIN aperture access,
+	  page table walking, and BAR1 virtual memory mapping functionality.
+
+	  This is a testing option and is default-disabled.
drivers/gpu/nova-core/driver.rs +2 -0
index d8b2e967ba4c..7d0d09939835 100644
@@ -92,6 +92,8 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> impl PinInit<Self, E
 
             Ok(try_pin_init!(Self {
                 gpu <- Gpu::new(pdev, bar.clone(), bar.access(pdev.as_ref())?),
+                // Run optional GPU selftests.
+                _: { gpu.run_selftests(pdev)? },
                 _reg <- auxiliary::Registration::new(
                     pdev.as_ref(),
                     c"nova-drm",
drivers/gpu/nova-core/gpu.rs +42 -0

46 lines not on this page · raw message

drivers/gpu/nova-core/mm/bar_user.rs +250 -0

253 lines not on this page · raw message

Signature · 1 line
2.34.1
#40 lore raw

PATCH v8 25/25 gpu: nova-core: mm: Add PRAMIN aperture self-tests

In reply to Joel Fernandes

Add self-tests for the PRAMIN aperture mechanism to verify correct operation during GPU probe. The tests validate various alignment requirements and corner cases.

The tests are default disabled and behind CONFIG_NOVA_MM_SELFTESTS. When enabled, tests run after GSP boot during probe.

---
 drivers/gpu/nova-core/gpu.rs       |   3 +
 drivers/gpu/nova-core/mm/pramin.rs | 161 +++++++++++++++++++++++++++++
 2 files changed, 164 insertions(+)
drivers/gpu/nova-core/gpu.rs +3 -0
index fba6ddba6a3f..34827dc2afff 100644
@@ -396,6 +396,9 @@ fn run_mm_selftests(mut self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>)
 
         let mmu_version = MmuVersion::from(self.spec.chipset.arch());
 
+        // PRAMIN aperture self-tests.
+        crate::mm::pramin::run_self_test(pdev.as_ref(), self.bar.clone(), mmu_version)?;
+
         // BAR1 self-tests.
         let bar1 = Arc::pin_init(
             pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
drivers/gpu/nova-core/mm/pramin.rs +161 -0

164 lines not on this page · raw message

Signature · 1 line
2.34.1
#41 lore raw

Re: [PATCH v8 25/25] gpu: nova-core: mm: Add PRAMIN aperture self-tests

On Wed Feb 25, 2026 at 7:53 AM JST, Joel Fernandes wrote:
<snip>

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs +2 -0
index 04b652d3ee4f..30b1dba0c305 100644
@@ -290,3 +290,164 @@ fn drop(&mut self) {
         // MutexGuard drops automatically, releasing the lock.
     }
 }
+
+/// Run PRAMIN self-tests during boot if self-tests are enabled.

drivers/gpu/nova-core/mm/pramin.rs fn drop(&mut self) { · line 304

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs +10 -0
+#[cfg(CONFIG_NOVA_MM_SELFTESTS)]
+pub(crate) fn run_self_test(
+    dev: &kernel::device::Device,
+    bar: Arc<Devres<Bar0>>,
+    mmu_version: super::pagetable::MmuVersion,
+) -> Result {
+    use super::pagetable::MmuVersion;
+
+    // PRAMIN support is only for MMU v2 for now (Turing/Ampere/Ada).
+    if mmu_version != MmuVersion::V2 {

Why is that? I thought PRAMIN was also working on Hopper+. Isn't it orthogonal to the kind of MMU used?

Quoted 7 lines
drivers/gpu/nova-core/mm/pramin.rs +7 -0
+        dev_info!(
+            dev,
+            "PRAMIN: Skipping self-tests for MMU {:?} (only V2 supported)\n",
+            mmu_version
+        );
+        return Ok(());
+    }

drivers/gpu/nova-core/mm/pramin.rs fn drop(&mut self) { · line 321

Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs +10 -0
+
+    dev_info!(dev, "PRAMIN: Starting self-test...\n");
+
+    let pramin = Arc::pin_init(Pramin::new(bar)?, GFP_KERNEL)?;
+    let mut win = pramin.window()?;
+
+    // Use offset 0x1000 as test area.
+    let base: usize = 0x1000;
+
+    // Test 1: Read/write at byte-aligned locations.

Let's split each test into its own function for readability.

#42 lore raw

Re: [PATCH v8 25/25] gpu: nova-core: mm: Add PRAMIN aperture self-tests

Quoted 5 lines · Alexandre Courbot

drivers/gpu/nova-core/mm/pramin.rs fn drop(&mut self) { · line 304

Quoted 2 lines
drivers/gpu/nova-core/mm/pramin.rs +2 -0
+    // PRAMIN support is only for MMU v2 for now (Turing/Ampere/Ada).
+    if mmu_version != MmuVersion::V2 {

Why is that? I thought PRAMIN was also working on Hopper+. Isn't it orthogonal to the kind of MMU used?

The PRAMIN aperture window register changed on Hopper+: pre-Hopper uses NV_PBUS_BAR0_WINDOW while Hopper+ uses NV_XAL_EP_BAR0_WINDOW. Our current code only programs the former, so we'd need a HAL to support both. For now, V2-only is the safe choice. I've updated the comment to explain this.

Quoted 3 lines
Quoted 1 line

+ // Test 1: Read/write at byte-aligned locations.

Let's split each test into its own function for readability.

Done.

thanks,

--
Joel Fernandes

#43 lore raw

Re: [PATCH v8 25/25] gpu: nova-core: mm: Add PRAMIN aperture self-tests

Quoted 4 lines · Joel Fernandes
Quoted 3 lines · Alexandre Courbot
Quoted 2 lines
drivers/gpu/nova-core/mm/pramin.rs +2 -0
+    // PRAMIN support is only for MMU v2 for now (Turing/Ampere/Ada).
+    if mmu_version != MmuVersion::V2 {
Quoted 8 lines
Quoted 2 lines

Why is that? I thought PRAMIN was also working on Hopper+. Isn't it orthogonal to the kind of MMU used?

The PRAMIN aperture window register changed on Hopper+: pre-Hopper uses NV_PBUS_BAR0_WINDOW while Hopper+ uses NV_XAL_EP_BAR0_WINDOW. Our current code only programs the former, so we'd need a HAL to support both. For now, V2-only is the safe choice. I've updated the comment to explain this.

I see, but then it seems that the correct check would be against the Chipset, not the version of the page table format? I understand the two are linked transitively, but they are in practice different engines.

#44 lore raw

Re: [PATCH v8 25/25] gpu: nova-core: mm: Add PRAMIN aperture self-tests

Quoted 10 lines · Alexandre Courbot
Quoted 5 lines · Joel Fernandes

The PRAMIN aperture window register changed on Hopper+: pre-Hopper uses NV_PBUS_BAR0_WINDOW while Hopper+ uses NV_XAL_EP_BAR0_WINDOW. Our current code only programs the former, so we'd need a HAL to support both. For now, V2-only is the safe choice. I've updated the comment to explain this.

I see, but then it seems that the correct check would be against the Chipset, not the version of the page table format? I understand the two are linked transitively, but they are in practice different engines.

Alex, makes sense, so I'll change it to the following then (still testing but here is a preview):

@@ -439,17 +458,21 @@ pub(crate) fn run_self_test(

     dev: &kernel::device::Device,
     bar: Arc<Devres<Bar0>>,
     vram_region: Range<u64>,

- mmu_version: super::pagetable::MmuVersion,
+ chipset: crate::gpu::Chipset,
) -> Result {
- use super::pagetable::MmuVersion;
+ use crate::gpu::Architecture;

     // PRAMIN uses NV_PBUS_BAR0_WINDOW which is only available on pre-Hopper GPUs.
-    // Hopper+ uses a different register (NV_XAL_EP_BAR0_WINDOW), requiring a HAL.
-    if mmu_version != MmuVersion::V2 {
+    // Hopper+ uses NV_XAL_EP_BAR0_WINDOW instead, requiring a separate HAL that
+    // has not been implemented yet.
+    if !matches!(
+        chipset.arch(),
+        Architecture::Turing | Architecture::Ampere | Architecture::Ada
+    ) {
         dev_info!(
             dev,
-            "PRAMIN: Skipping self-tests for MMU {:?} (only V2 supported)\n",
-            mmu_version
+            "PRAMIN: Skipping self-tests for {:?} (only pre-Hopper supported)\n",
+            chipset
         );
         return Ok(());
     }
Signature · 1 line
Joel Fernandes
#45 lore raw

✗ Fi.CI.BUILD: failure for gpu: nova-core: Add memory management support

In reply to Joel Fernandes
== Series Details ==

== Series Details ==

Series: gpu: nova-core: Add memory management support
URL : https://patchwork.freedesktop.org/series/162102/
State : failure

== Summary ==

Error: patch https://patchwork.freedesktop.org/api/1.0/series/162102/revisions/1/mbox/ not applied
Applying: gpu: nova-core: Select GPU_BUDDY for VRAM allocation
Applying: gpu: nova-core: Kconfig: Sort select statements alphabetically
Applying: gpu: nova-core: gsp: Return GspStaticInfo and FbLayout from boot()
Using index info to reconstruct a base tree...
M drivers/gpu/nova-core/gpu.rs
M drivers/gpu/nova-core/gsp/boot.rs
Falling back to patching base and 3-way merge...
Auto-merging drivers/gpu/nova-core/gsp/boot.rs
Auto-merging drivers/gpu/nova-core/gpu.rs
Applying: gpu: nova-core: gsp: Extract usable FB region from GSP
Applying: gpu: nova-core: fb: Add usable_vram field to FbLayout
Applying: gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
Applying: docs: gpu: nova-core: Document the PRAMIN aperture mechanism
Applying: gpu: nova-core: mm: Add common memory management types
Applying: gpu: nova-core: mm: Add TLB flush support
Applying: gpu: nova-core: mm: Add GpuMm centralized memory manager
Applying: gpu: nova-core: mm: Use usable VRAM region for buddy allocator
error: sha1 information is lacking or useless (drivers/gpu/nova-core/gpu.rs).
error: could not build fake ancestor
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0011 gpu: nova-core: mm: Use usable VRAM region for buddy allocator When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Build failed, no error log produced