Skip to contentSkip to conversation

gpu: nova-core: Add memory management support

PATCH Superseded Joel Fernandes 8 participants 102 messages Mar 11 – Apr 15

Subsystems (8)

v9 of 12 Superseded Latest is v12 →

lore
Share & tools
lore
#0 lore raw

v9 cover gpu: nova-core: Add memory management support

This series adds support for nova-core memory management, including VRAM allocation, PRAMIN, VMM, page table walking, and BAR 1 read/writes. These are critical for channel management, vGPU, and all other memory management uses of nova-core.

It is based on linux -next and all patches, along with all the dependencies (such as buddy bindings, CList), can be found at:
git://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git (tag: nova-mm-v9-20260310)

Change log:

Changes from v8 to v9:
- Added fixes from Zhi Wang for bitfield position changes in virtual address and larger BAR1 size on some platforms. Tested and working for vGPU usecase!
- Refactored gsp: boot() to return only GspStaticInfo, removing FbLayout (Alex)
- bar1_pde_base and bar2_pde_base are now accessed via GspStaticInfo directly (Alex)
- Added new patch "gsp: Expose total physical VRAM end from FB region info" introducing total_fb_end() to expose VRAM extent. (Alex)
- Consolidated usable VRAM and BarUser setup; removed dedicated
"fb: Add usable_vram field to FbLayout", "mm: Use usable VRAM region for buddy allocator", and "mm: Add BarUser to struct Gpu and create at boot".

Changes from v7 to v8:
- Incorporated "Select GPU_BUDDY for VRAM allocation" patch from the dependency series (Alex).
- Significant patch reordering for better logical flow (GSP/FB patches moved earlier, page table patches, Vmm, Bar1, tests) (Alex).
- Replaced several 'as' usages with into_safe_cast() (Danilo, Alex).
- Updated BAR 1 test cases to include exercising the block size API (Eliot, Danilo).

Changes from v6 to v7:
- Addressed DMA fence signalling usecase per Danilo's feedback.

Pre v6:
- Simplified PRAMIN code (John Hubbard, Alex Courbot).
- Handled different MMU versions: ver2 versus ver3 (John Hubbard).
- Added BAR1 usecase so we have user of DRM Buddy / VMM (John Hubbard).
- Iterating over clist/buddy bindings.

Link to v8: https://lore.kernel.org/all/[email protected]/
Link to v7: https://lore.kernel.org/all/[email protected]/

Joel Fernandes (24):
gpu: nova-core: Select GPU_BUDDY for VRAM allocation
gpu: nova-core: Kconfig: Sort select statements alphabetically
gpu: nova-core: gsp: Return GspStaticInfo from boot()
gpu: nova-core: gsp: Extract usable FB region from GSP
gpu: nova-core: gsp: Expose total physical VRAM end from FB region
info
gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
docs: gpu: nova-core: Document the PRAMIN aperture mechanism
gpu: nova-core: mm: Add common memory management types
gpu: nova-core: mm: Add TLB flush support
gpu: nova-core: mm: Add GpuMm centralized memory manager
gpu: nova-core: mm: Add common types for all page table formats
gpu: nova-core: mm: Add MMU v2 page table types
gpu: nova-core: mm: Add MMU v3 page table types
gpu: nova-core: mm: Add unified page table entry wrapper enums
gpu: nova-core: mm: Add page table walker for MMU v2/v3
gpu: nova-core: mm: Add Virtual Memory Manager
gpu: nova-core: mm: Add virtual address range tracking to VMM
gpu: nova-core: mm: Add multi-page mapping API to VMM
gpu: nova-core: Add BAR1 aperture type and size constant
gpu: nova-core: mm: Add BAR1 user interface
gpu: nova-core: mm: Add BAR1 memory management self-tests
gpu: nova-core: mm: Add PRAMIN aperture self-tests

Zhi Wang (1):
gpu: nova-core: Use runtime BAR1 size instead of hardcoded 256MB

 Documentation/gpu/nova/core/pramin.rst     | 125 +++++
 Documentation/gpu/nova/index.rst           |   1 +
 MAINTAINERS                                |  14 +-
 drivers/gpu/nova-core/Kconfig              |  13 +-
 drivers/gpu/nova-core/driver.rs            |   3 +
 drivers/gpu/nova-core/gpu.rs               |  90 ++-
 drivers/gpu/nova-core/gsp/boot.rs          |  12 +-
 drivers/gpu/nova-core/gsp/commands.rs      |  18 +-
 drivers/gpu/nova-core/gsp/fw/commands.rs   |  59 ++
 drivers/gpu/nova-core/mm.rs                | 235 ++++++++
 drivers/gpu/nova-core/mm/bar_user.rs       | 412 ++++++++++++++
 drivers/gpu/nova-core/mm/pagetable.rs      | 481 ++++++++++++++++
 drivers/gpu/nova-core/mm/pagetable/ver2.rs | 232 ++++++++
 drivers/gpu/nova-core/mm/pagetable/ver3.rs | 337 ++++++++++++
 drivers/gpu/nova-core/mm/pagetable/walk.rs | 218 ++++++++
 drivers/gpu/nova-core/mm/pramin.rs         | 502 +++++++++++++++++
 drivers/gpu/nova-core/mm/tlb.rs            |  90 +++
 drivers/gpu/nova-core/mm/vmm.rs            | 498 +++++++++++++++++
 drivers/gpu/nova-core/nova_core.rs         |   1 +
 drivers/gpu/nova-core/regs.rs              |  39 ++
 29 files changed, 4390 insertions(+), 10 deletions(-)
 create mode 100644 Documentation/gpu/nova/core/pramin.rst
 create mode 100644 drivers/gpu/nova-core/mm.rs
 create mode 100644 drivers/gpu/nova-core/mm/bar_user.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver3.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs
 create mode 100644 drivers/gpu/nova-core/mm/pramin.rs
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
 create mode 100644 drivers/gpu/nova-core/mm/vmm.rs
Signature · 1 line
2.34.1
#1 lore raw

PATCH v9 1/23 gpu: nova-core: Select GPU_BUDDY for VRAM allocation

nova-core will use the GPU buddy allocator for physical VRAM management. Enable it in Kconfig.

---
 drivers/gpu/nova-core/Kconfig | 1 +
 1 file changed, 1 insertion(+)
drivers/gpu/nova-core/Kconfig +1 -0
index 527920f9c4d3..809485167aff 100644
@@ -5,6 +5,7 @@ config NOVA_CORE
 	depends on RUST
 	select RUST_FW_LOADER_ABSTRACTIONS
 	select AUXILIARY_BUS
+	select GPU_BUDDY
 	default n
 	help
 	  Choose this if you want to build the Nova Core driver for Nvidia
Signature · 1 line
2.34.1
#4 lore raw

Re: [PATCH v9 01/23] gpu: nova-core: Select GPU_BUDDY for VRAM allocation

Quoted 2 lines · Alexandre Courbot

As I said in v8, let's squash this change with the first commit that actually makes use of GPU_BUDDY.

Will do. The first commit actually using GPU_BUDDY is "Add GpuMm centralized memory manager", so I'll squash this Kconfig select into that commit in v10.

thanks,

--
Joel Fernandes

#5 lore raw

PATCH v9 2/23 gpu: nova-core: Kconfig: Sort select statements alphabetically

In reply to Joel Fernandes

Reorder the select statements in NOVA_CORE Kconfig to be in alphabetical order.

---
 drivers/gpu/nova-core/Kconfig | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/nova-core/Kconfig +1 -1
index 809485167aff..6513007bf66f 100644
@@ -3,9 +3,9 @@ config NOVA_CORE
 	depends on 64BIT
 	depends on PCI
 	depends on RUST
-	select RUST_FW_LOADER_ABSTRACTIONS
 	select AUXILIARY_BUS
 	select GPU_BUDDY
+	select RUST_FW_LOADER_ABSTRACTIONS
 	default n
 	help
 	  Choose this if you want to build the Nova Core driver for Nvidia
Signature · 1 line
2.34.1
#8 lore raw

Re: [PATCH v9 02/23] gpu: nova-core: Kconfig: Sort select statements alphabetically

Quoted 1 line · Alexandre Courbot

This one is already in drm-rust-next.

I'll rebase. thanks,

--
Joel Fernandes

#9 lore raw

PATCH v9 3/23 gpu: nova-core: gsp: Return GspStaticInfo from boot()

In reply to Joel Fernandes

Refactor the GSP boot function to return only the GspStaticInfo, removing the FbLayout from the return tuple.

This enables access required for memory management initialization to:
- bar1_pde_base: BAR1 page directory base.
- bar2_pde_base: BAR2 page directory base.
- usable memory regions in vidmem.

---
 drivers/gpu/nova-core/gpu.rs      |  9 +++++++--
 drivers/gpu/nova-core/gsp/boot.rs | 12 ++++++++----
 2 files changed, 15 insertions(+), 6 deletions(-)
drivers/gpu/nova-core/gpu.rs +7 -2
index 60c85fffaeaf..c324d96bd0c6 100644
@@ -18,7 +18,10 @@
     },
     fb::SysmemFlush,
     gfw,
-    gsp::Gsp,
+    gsp::{
+        commands::GetGspStaticInfoReply,
+        Gsp, //
+    },
     regs,
 };
 
@@ -252,6 +255,8 @@ pub(crate) struct Gpu {
     /// GSP runtime data. Temporarily an empty placeholder.
     #[pin]
     gsp: Gsp,
+    /// Static GPU information from GSP.
+    gsp_static_info: GetGspStaticInfoReply,
 }
 
 impl Gpu {
@@ -283,7 +288,7 @@ pub(crate) fn new<'a>(
 
             gsp <- Gsp::new(pdev),
 
-            _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
+            gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
 
             bar: devres_bar,
         })
drivers/gpu/nova-core/gsp/boot.rs +8 -4
index c56029f444cb..73a711f03044 100644
@@ -32,7 +32,10 @@
     },
     gpu::Chipset,
     gsp::{
-        commands,
+        commands::{
+            self,
+            GetGspStaticInfoReply, //
+        },
         sequencer::{
             GspSequencer,
             GspSequencerParams, //
@@ -126,7 +129,8 @@ fn run_fwsec_frts(
     /// user-space, patching them with signatures, and building firmware-specific intricate data
     /// structures that the GSP will use at runtime.
     ///
-    /// Upon return, the GSP is up and running, and its runtime object given as return value.
+    /// Upon return, the GSP is up and running, and static GPU information is returned.
+    ///
     pub(crate) fn boot(
         mut self: Pin<&mut Self>,
         pdev: &pci::Device<device::Bound>,
@@ -134,7 +138,7 @@ pub(crate) fn boot(
         chipset: Chipset,
         gsp_falcon: &Falcon<Gsp>,
         sec2_falcon: &Falcon<Sec2>,
-    ) -> Result {
+    ) -> Result<GetGspStaticInfoReply> {
         let dev = pdev.as_ref();
 
         let bios = Vbios::new(dev, bar)?;
@@ -225,6 +229,6 @@ pub(crate) fn boot(
             Err(e) => dev_warn!(pdev, "GPU name unavailable: {:?}\n", e),
         }
 
-        Ok(())
+        Ok(info)
     }
 }
Signature · 1 line
2.34.1
Reviewed-by on v9 3/23
#10 lore raw

Re: [PATCH v9 03/23] gpu: nova-core: gsp: Return GspStaticInfo from boot()

Quoted 2 lines · Joel Fernandes

Refactor the GSP boot function to return only the GspStaticInfo, removing the FbLayout from the return tuple.

I think the commit message may need updating - boot doesn't return FbLayout. And it returns GetGspStaticInfoReply, not GspStaticInfo.

Other than that,

Quoted 28 lines
drivers/gpu/nova-core/gsp/boot.rs +4 -3
@@ -126,7 +129,8 @@ fn run_fwsec_frts(
     /// user-space, patching them with signatures, and building firmware-specific intricate data
     /// structures that the GSP will use at runtime.
     ///
-    /// Upon return, the GSP is up and running, and its runtime object given as return value.
+    /// Upon return, the GSP is up and running, and static GPU information is returned.
+    ///
     pub(crate) fn boot(
         mut self: Pin<&mut Self>,
         pdev: &pci::Device<device::Bound>,
@@ -134,7 +138,7 @@ pub(crate) fn boot(
         chipset: Chipset,
         gsp_falcon: &Falcon<Gsp>,
         sec2_falcon: &Falcon<Sec2>,
-    ) -> Result {
+    ) -> Result<GetGspStaticInfoReply> {
         let dev = pdev.as_ref();
 
         let bios = Vbios::new(dev, bar)?;
@@ -225,6 +229,6 @@ pub(crate) fn boot(
             Err(e) => dev_warn!(pdev, "GPU name unavailable: {:?}\n", e),
         }
 
-        Ok(())
+        Ok(info)
     }
 }
#11 lore raw

PATCH v9 4/23 gpu: nova-core: gsp: Extract usable FB region from GSP

In reply to Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed via usable_fb_region() API for use by the memory subsystem.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 11 ++++++--
 drivers/gpu/nova-core/gsp/fw/commands.rs | 32 ++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 2 deletions(-)
drivers/gpu/nova-core/gsp/commands.rs +9 -2
index 8f270eca33be..8d5780d9cace 100644
@@ -4,6 +4,7 @@
     array,
     convert::Infallible,
     ffi::FromBytesUntilNulError,
+    ops::Range,
     str::Utf8Error, //
 };
 
@@ -186,22 +187,28 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
     }
 }
 
-/// The reply from the GSP to the [`GetGspInfo`] command.
+/// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    /// Usable FB (VRAM) region for driver memory allocation.
+    #[expect(dead_code)]
+    pub(crate) usable_fb_region: Range<u64>,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
     const FUNCTION: MsgFunction = MsgFunction::GetGspStaticInfo;
     type Message = GspStaticConfigInfo;
-    type InitError = Infallible;
+    type InitError = Error;
 
     fn read(
         msg: &Self::Message,
         _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>,
     ) -> Result<Self, Self::InitError> {
+        let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
+            usable_fb_region: base..base.saturating_add(size),
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +32 -0
index 67f44421fcc3..cef86cab8a12 100644
@@ -5,6 +5,7 @@
 use kernel::{device, pci};
 
 use crate::gsp::GSP_PAGE_SIZE;
+use crate::num::IntoSafeCast;
 
 use super::bindings;
 
@@ -115,6 +116,37 @@ impl GspStaticConfigInfo {
     pub(crate) fn gpu_name_str(&self) -> [u8; 64] {
         self.0.gpuNameString
     }
+
+    /// Extract the first usable FB region from GSP firmware data.
+    ///
+    /// Returns the first region suitable for driver memory allocation as a `(base, size)` tuple.
+    /// Usable regions are those that:
+    /// - Are not reserved for firmware internal use.
+    /// - Are not protected (hardware-enforced access restrictions).
+    /// - Support compression (can use GPU memory compression for bandwidth).
+    /// - Support ISO (isochronous memory for display requiring guaranteed bandwidth).
+    pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
+        let fb_info = &self.0.fbRegionInfoParams;
+        for i in 0..fb_info.numFBRegions.into_safe_cast() {
+            if let Some(reg) = fb_info.fbRegion.get(i) {
+                // Skip malformed regions where limit < base.
+                if reg.limit < reg.base {
+                    continue;
+                }
+
+                // Filter: not reserved, not protected, supports compression and ISO.
+                if reg.reserved == 0
+                    && reg.bProtected == 0
+                    && reg.supportCompressed != 0
+                    && reg.supportISO != 0
+                {
+                    let size = reg.limit - reg.base + 1;
+                    return Some((reg.base, size));
+                }
+            }
+        }
+        None
+    }
 }
 
 // SAFETY: Padding is explicit and will not contain uninitialized data.
Signature · 1 line
2.34.1
#12 lore raw

Re: [PATCH v9 04/23] gpu: nova-core: gsp: Extract usable FB region from GSP

Quoted 44 lines · Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed via usable_fb_region() API for use by the memory subsystem.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 11 ++++++--
 drivers/gpu/nova-core/gsp/fw/commands.rs | 32 ++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 2 deletions(-)
drivers/gpu/nova-core/gsp/commands.rs +6 -2
index 8f270eca33be..8d5780d9cace 100644
@@ -4,6 +4,7 @@
     array,
     convert::Infallible,
     ffi::FromBytesUntilNulError,
+    ops::Range,
     str::Utf8Error, //
 };
 
@@ -186,22 +187,28 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
     }
 }
 
-/// The reply from the GSP to the [`GetGspInfo`] command.
+/// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    /// Usable FB (VRAM) region for driver memory allocation.
+    #[expect(dead_code)]
+    pub(crate) usable_fb_region: Range<u64>,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
     const FUNCTION: MsgFunction = MsgFunction::GetGspStaticInfo;
     type Message = GspStaticConfigInfo;
-    type InitError = Infallible;
+    type InitError = Error;

drivers/gpu/nova-core/gsp/commands.rs fn init(&self) -> impl Init<Self::Command, Self::InitError> { · line 211

Quoted 10 lines
drivers/gpu/nova-core/gsp/commands.rs +3 -0
 
     fn read(
         msg: &Self::Message,
         _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>,
     ) -> Result<Self, Self::InitError> {
+        let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
+            usable_fb_region: base..base.saturating_add(size),

We already return a Result here, so why not use checked_add?:
base..base.checked_add(size).ok_or(EOVERFLOW)?

Quoted 18 lines
drivers/gpu/nova-core/gsp/commands.rs +0 -0
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +1 -0
index 67f44421fcc3..cef86cab8a12 100644
@@ -5,6 +5,7 @@
 use kernel::{device, pci};
 
 use crate::gsp::GSP_PAGE_SIZE;
+use crate::num::IntoSafeCast;
 
 use super::bindings;
 
@@ -115,6 +116,37 @@ impl GspStaticConfigInfo {
     pub(crate) fn gpu_name_str(&self) -> [u8; 64] {
         self.0.gpuNameString

drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 127

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +9 -0
     }
+
+    /// Extract the first usable FB region from GSP firmware data.
+    ///
+    /// Returns the first region suitable for driver memory allocation as a `(base, size)` tuple.
+    /// Usable regions are those that:
+    /// - Are not reserved for firmware internal use.
+    /// - Are not protected (hardware-enforced access restrictions).
+    /// - Support compression (can use GPU memory compression for bandwidth).
+    /// - Support ISO (isochronous memory for display requiring guaranteed bandwidth).

Are the above conditions all required (AND) or any required (OR)? Might be worth clarifying in the doc.

drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 132

Quoted 5 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +5 -0
+    pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
+        let fb_info = &self.0.fbRegionInfoParams;
+        for i in 0..fb_info.numFBRegions.into_safe_cast() {
+            if let Some(reg) = fb_info.fbRegion.get(i) {
+                // Skip malformed regions where limit < base.

Is it normal that it returns a bunch of broken regions?

drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 144

Quoted 12 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +12 -0
+                if reg.limit < reg.base {
+                    continue;
+                }
+
+                // Filter: not reserved, not protected, supports compression and ISO.
+                if reg.reserved == 0
+                    && reg.bProtected == 0
+                    && reg.supportCompressed != 0
+                    && reg.supportISO != 0
+                {
+                    let size = reg.limit - reg.base + 1;
+                    return Some((reg.base, size));

This is identifying a range, so how about returning Option<Range<u64>> instead? It gets immediately converted into a range anyway.

Quoted 8 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +5 -0
+                }
+            }
+        }
+        None
+    }
 }
 
 // SAFETY: Padding is explicit and will not contain uninitialized data.
#13 lore raw

Re: [PATCH v9 04/23] gpu: nova-core: gsp: Extract usable FB region from GSP

Quoted 46 lines · Eliot Courtney
Quoted 45 lines · Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed via usable_fb_region() API for use by the memory subsystem.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 11 ++++++--
 drivers/gpu/nova-core/gsp/fw/commands.rs | 32 ++++++++++++++++++++++++
 2 files changed, 41 insertions(+), 2 deletions(-)
drivers/gpu/nova-core/gsp/commands.rs +6 -2
index 8f270eca33be..8d5780d9cace 100644
@@ -4,6 +4,7 @@
     array,
     convert::Infallible,
     ffi::FromBytesUntilNulError,
+    ops::Range,
     str::Utf8Error, //
 };
 
@@ -186,22 +187,28 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
     }
 }
 
-/// The reply from the GSP to the [`GetGspInfo`] command.
+/// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    /// Usable FB (VRAM) region for driver memory allocation.
+    #[expect(dead_code)]
+    pub(crate) usable_fb_region: Range<u64>,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
     const FUNCTION: MsgFunction = MsgFunction::GetGspStaticInfo;
     type Message = GspStaticConfigInfo;
-    type InitError = Infallible;
+    type InitError = Error;
 
Quoted 12 lines

drivers/gpu/nova-core/gsp/commands.rs fn init(&self) -> impl Init<Self::Command, Self::InitError> { · line 211

Quoted 9 lines
drivers/gpu/nova-core/gsp/commands.rs +3 -0
     fn read(
         msg: &Self::Message,
         _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>,
     ) -> Result<Self, Self::InitError> {
+        let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
+            usable_fb_region: base..base.saturating_add(size),

We already return a Result here, so why not use checked_add?:
base..base.checked_add(size).ok_or(EOVERFLOW)?

Hmm, I think I was trying to play it safe and handle any situation of a corrupted size. But granted, it may be better to error out.

Quoted 16 lines
Quoted 16 lines
drivers/gpu/nova-core/gsp/commands.rs +0 -0
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +1 -0
index 67f44421fcc3..cef86cab8a12 100644
@@ -5,6 +5,7 @@
 use kernel::{device, pci};
 
 use crate::gsp::GSP_PAGE_SIZE;
+use crate::num::IntoSafeCast;
 
 use super::bindings;
 
Quoted 16 lines

drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 127

Quoted 13 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +9 -0
@@ -115,6 +116,37 @@ impl GspStaticConfigInfo {
     pub(crate) fn gpu_name_str(&self) -> [u8; 64] {
         self.0.gpuNameString
     }
+
+    /// Extract the first usable FB region from GSP firmware data.
+    ///
+    /// Returns the first region suitable for driver memory allocation as a `(base, size)` tuple.
+    /// Usable regions are those that:
+    /// - Are not reserved for firmware internal use.
+    /// - Are not protected (hardware-enforced access restrictions).
+    /// - Support compression (can use GPU memory compression for bandwidth).
+    /// - Support ISO (isochronous memory for display requiring guaranteed bandwidth).

Are the above conditions all required (AND) or any required (OR)? Might be worth clarifying in the doc.

I am not sure if any clarification is needed there but I will reword to 'Usable regions are those that satisfy all of the following:'.

Quoted 8 lines

drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 132

Quoted 6 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +5 -0
+    pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
+        let fb_info = &self.0.fbRegionInfoParams;
+        for i in 0..fb_info.numFBRegions.into_safe_cast() {
+            if let Some(reg) = fb_info.fbRegion.get(i) {
+                // Skip malformed regions where limit < base.

Is it normal that it returns a bunch of broken regions?

Not really. The aim was to make the code robust at a time when I was studying the FB regions. I will change the comment, and/or drop the check. Thanks for pointing it out.

Quoted 13 lines

drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 144

Quoted 12 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +12 -0
+                if reg.limit < reg.base {
+                    continue;
+                }
+
+                // Filter: not reserved, not protected, supports compression and ISO.
+                if reg.reserved == 0
+                    && reg.bProtected == 0
+                    && reg.supportCompressed != 0
+                    && reg.supportISO != 0
+                {
+                    let size = reg.limit - reg.base + 1;
+                    return Some((reg.base, size));
Quoted 2 lines

This is identifying a range, so how about returning Option<Range<u64>> instead? It gets immediately converted into a range anyway.

Sure, that works.

--
Joel Fernandes

#14 lore raw

Re: [PATCH v9 04/23] gpu: nova-core: gsp: Extract usable FB region from GSP

In reply to Joel Fernandes
Quoted 9 lines · Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed via usable_fb_region() API for use by the memory subsystem.

This doesn't take into account the feedback I gave in [1]. In particular, a TODO to handle the remaining regions looks important to me.

[1] https://lore.kernel.org/all/[email protected]/

#15 lore raw

Re: [PATCH v9 04/23] gpu: nova-core: gsp: Extract usable FB region from GSP

Quoted 3 lines · Alexandre Courbot

This doesn't take into account the feedback I gave in [1]. In particular, a TODO to handle the remaining regions looks important to me.

Sorry about missing the v8 feedback. Fixed in upcoming v10.

--
Joel Fernandes

#16 lore raw

PATCH v9 5/23 gpu: nova-core: gsp: Expose total physical VRAM end from FB region info

In reply to Joel Fernandes

Add total_fb_end() to GspStaticConfigInfo that computes the exclusive end address of the highest valid FB region covering both usable and GSP-reserved areas.

This allows callers to know the full physical VRAM extent, not just the allocatable portion.

---
 drivers/gpu/nova-core/gsp/commands.rs    |  6 ++++++
 drivers/gpu/nova-core/gsp/fw/commands.rs | 19 +++++++++++++++++++
 2 files changed, 25 insertions(+)
drivers/gpu/nova-core/gsp/commands.rs +6 -0
index 8d5780d9cace..389d215098c6 100644
@@ -193,6 +193,9 @@ pub(crate) struct GetGspStaticInfoReply {
     /// Usable FB (VRAM) region for driver memory allocation.
     #[expect(dead_code)]
     pub(crate) usable_fb_region: Range<u64>,
+    /// End of VRAM.
+    #[expect(dead_code)]
+    pub(crate) total_fb_end: u64,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
@@ -206,9 +209,12 @@ fn read(
     ) -> Result<Self, Self::InitError> {
         let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
 
+        let total_fb_end = msg.total_fb_end().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
             usable_fb_region: base..base.saturating_add(size),
+            total_fb_end,
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +19 -0
index cef86cab8a12..acaf92cd6735 100644
@@ -147,6 +147,25 @@ pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
         }
         None
     }
+
+    /// Compute the end of physical VRAM from all FB regions.
+    pub(crate) fn total_fb_end(&self) -> Option<u64> {
+        let fb_info = &self.0.fbRegionInfoParams;
+        let mut max_end: Option<u64> = None;
+        for i in 0..fb_info.numFBRegions.into_safe_cast() {
+            if let Some(reg) = fb_info.fbRegion.get(i) {
+                if reg.limit < reg.base {
+                    continue;
+                }
+                let end = reg.limit.saturating_add(1);
+                max_end = Some(match max_end {
+                    None => end,
+                    Some(prev) => prev.max(end),
+                });
+            }
+        }
+        max_end
+    }
 }
 
 // SAFETY: Padding is explicit and will not contain uninitialized data.
Signature · 1 line
2.34.1
#17 lore raw

Re: [PATCH v9 05/23] gpu: nova-core: gsp: Expose total physical VRAM end from FB region info

Quoted 48 lines · Joel Fernandes

Add total_fb_end() to GspStaticConfigInfo that computes the exclusive end address of the highest valid FB region covering both usable and GSP-reserved areas.

This allows callers to know the full physical VRAM extent, not just the allocatable portion.

---
 drivers/gpu/nova-core/gsp/commands.rs    |  6 ++++++
 drivers/gpu/nova-core/gsp/fw/commands.rs | 19 +++++++++++++++++++
 2 files changed, 25 insertions(+)
drivers/gpu/nova-core/gsp/commands.rs +6 -0
index 8d5780d9cace..389d215098c6 100644
@@ -193,6 +193,9 @@ pub(crate) struct GetGspStaticInfoReply {
     /// Usable FB (VRAM) region for driver memory allocation.
     #[expect(dead_code)]
     pub(crate) usable_fb_region: Range<u64>,
+    /// End of VRAM.
+    #[expect(dead_code)]
+    pub(crate) total_fb_end: u64,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
@@ -206,9 +209,12 @@ fn read(
     ) -> Result<Self, Self::InitError> {
         let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
 
+        let total_fb_end = msg.total_fb_end().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
             usable_fb_region: base..base.saturating_add(size),
+            total_fb_end,
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +0 -0
index cef86cab8a12..acaf92cd6735 100644
@@ -147,6 +147,25 @@ pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
         }
         None
     }

drivers/gpu/nova-core/gsp/fw/commands.rs pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> { · line 159

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +10 -0
+
+    /// Compute the end of physical VRAM from all FB regions.
+    pub(crate) fn total_fb_end(&self) -> Option<u64> {
+        let fb_info = &self.0.fbRegionInfoParams;
+        let mut max_end: Option<u64> = None;
+        for i in 0..fb_info.numFBRegions.into_safe_cast() {
+            if let Some(reg) = fb_info.fbRegion.get(i) {
+                if reg.limit < reg.base {
+                    continue;
+                }

This is basically a repeat of the code of the previous patch. Let's implement an iterator over the FB memory regions (that filters out invalid regions) that we can leverage in both places so we don't need to repeat ourselves.

#18 lore raw

Re: [PATCH v9 05/23] gpu: nova-core: gsp: Expose total physical VRAM end from FB region info

Quoted 4 lines · Alexandre Courbot

This is basically a repeat of the code of the previous patch. Let's implement an iterator over the FB memory regions (that filters out invalid regions) that we can leverage in both places so we don't need to repeat ourselves.

Good catch. Added fb_regions() to create an iterator which both this and previous patch use. Nice simplification:

    /// Compute the end of physical VRAM from all FB regions.
    pub(crate) fn total_fb_end(&self) -> Option<u64> {
        self.fb_regions().map(|reg| reg.limit.saturating_add(1)).max()
    }

--
Joel Fernandes

#19 lore raw

PATCH v9 6/23 gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM

In reply to Joel Fernandes

PRAMIN apertures are a crucial mechanism to direct read/write to VRAM. Add support for the same.

---
 drivers/gpu/nova-core/mm.rs        |   5 +
 drivers/gpu/nova-core/mm/pramin.rs | 293 +++++++++++++++++++++++++++++
 drivers/gpu/nova-core/nova_core.rs |   1 +
 drivers/gpu/nova-core/regs.rs      |   6 +
 4 files changed, 305 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm.rs
 create mode 100644 drivers/gpu/nova-core/mm/pramin.rs
drivers/gpu/nova-core/mm.rs new file +5 -0
new file mode 100644
index 000000000000..7a5dd4220c67
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Memory management subsystems for nova-core.
+
+pub(crate) mod pramin;
drivers/gpu/nova-core/mm/pramin.rs new file +293 -0

293 lines not on this page · raw message

drivers/gpu/nova-core/nova_core.rs +1 -0
index b5caf1044697..c5a78d6388e5 100644
@@ -13,6 +13,7 @@
 mod gfw;
 mod gpu;
 mod gsp;
+mod mm;
 mod num;
 mod regs;
 mod sbuffer;
drivers/gpu/nova-core/regs.rs +6 -0
index ea0d32f5396c..8ec35b8c4b28 100644
@@ -32,6 +32,7 @@
         Architecture,
         Chipset, //
     },
+    mm::pramin::Bar0WindowTarget,
     num::FromSafeCast,
 };
 
@@ -102,6 +103,11 @@ fn fmt(&self, f: &mut kernel::fmt::Formatter<'_>) -> kernel::fmt::Result {
     31:16   frts_err_code as u16;
 });
 
+register!(NV_PBUS_BAR0_WINDOW @ 0x00001700, "BAR0 window control for PRAMIN access" {
+    25:24   target as u8 ?=> Bar0WindowTarget;
+    23:0    window_base as u32, "Window base address (bits 39:16 of FB addr)";
+});
+
 // PFB
 
 // The following two registers together hold the physical system memory address that is used by the
Signature · 1 line
2.34.1
#20 lore raw

PATCH v9 7/23 docs: gpu: nova-core: Document the PRAMIN aperture mechanism

In reply to Joel Fernandes

Add documentation for the PRAMIN aperture mechanism used by nova-core for direct VRAM access.

Nova only uses TARGET=VID_MEM for VRAM access. The SYS_MEM target values are documented for completeness but not used by the driver.

---
 Documentation/gpu/nova/core/pramin.rst | 125 +++++++++++++++++++++++++
 Documentation/gpu/nova/index.rst       |   1 +
 2 files changed, 126 insertions(+)
 create mode 100644 Documentation/gpu/nova/core/pramin.rst
Documentation/gpu/nova/core/pramin.rst new file +125 -0
new file mode 100644
index 000000000000..55ec9d920629
@@ -0,0 +1,125 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+PRAMIN aperture mechanism
+=========================
+
+.. note::
+   The following description is approximate and current as of the Ampere family.
+   It may change for future generations and is intended to assist in understanding
+   the driver code.
+
+Introduction
+============
+
+PRAMIN is a hardware aperture mechanism that provides CPU access to GPU Video RAM (VRAM) before
+the GPU's Memory Management Unit (MMU) and page tables are initialized. This 1MB sliding window,
+located at a fixed offset within BAR0, is essential for setting up page tables and other critical
+GPU data structures without relying on the GPU's MMU.
+
+Architecture Overview
+=====================
+
+The PRAMIN aperture mechanism is logically implemented by the GPU's PBUS (PCIe Bus Controller Unit)
+and provides a CPU-accessible window into VRAM through the PCIe interface::
+
+    +-----------------+    PCIe     +------------------------------+
+    |      CPU        |<----------->|           GPU                |
+    +-----------------+             |                              |
+                                    |  +----------------------+    |
+                                    |  |       PBUS           |    |
+                                    |  |  (Bus Controller)    |    |
+                                    |  |                      |    |
+                                    |  |  +--------------+<------------ (window starts at
+                                    |  |  |   PRAMIN     |    |    |     BAR0 + 0x700000)
+                                    |  |  |   Window     |    |    |
+                                    |  |  |   (1MB)      |    |    |
+                                    |  |  +--------------+    |    |
+                                    |  |         |            |    |
+                                    |  +---------|------------+    |
+                                    |            |                 |
+                                    |            v                 |
+                                    |  +----------------------+<------------ (Program PRAMIN to any
+                                    |  |       VRAM           |    |    64KB-aligned VRAM boundary)
+                                    |  |    (Several GBs)     |    |
+                                    |  |                      |    |
+                                    |  |  FB[0x000000000000]  |    |
+                                    |  |          ...         |    |
+                                    |  |  FB[0x7FFFFFFFFFF]   |    |
+                                    |  +----------------------+    |
+                                    +------------------------------+
+
+PBUS (PCIe Bus Controller) is responsible for, among other things, handling MMIO
+accesses to the BAR registers.
+
+PRAMIN Window Operation
+=======================
+
+The PRAMIN window provides a 1MB sliding aperture that can be repositioned over
+the entire VRAM address space using the ``NV_PBUS_BAR0_WINDOW`` register.
+
+Window Control Mechanism
+-------------------------
+
+The window position is controlled via the PBUS ``BAR0_WINDOW`` register::
+
+    NV_PBUS_BAR0_WINDOW Register (0x1700):
+    +-------+--------+--------------------------------------+
+    | 31:26 | 25:24  |               23:0                   |
+    | RSVD  | TARGET |            BASE_ADDR                 |
+    |       |        |        (bits 39:16 of VRAM address)  |
+    +-------+--------+--------------------------------------+
+
+    BASE_ADDR field (bits 23:0):
+    - Contains bits [39:16] of the target VRAM address
+    - Provides 40-bit (1TB) address space coverage
+    - Must be programmed with 64KB-aligned addresses
+
+    TARGET field (bits 25:24):
+    - 0x0: VRAM (Video Memory)
+    - 0x1: SYS_MEM_COH (Coherent System Memory)
+    - 0x2: SYS_MEM_NONCOH (Non-coherent System Memory)
+    - 0x3: Reserved
+
+    .. note::
+       Nova only uses TARGET=VRAM (0x0) for video memory access. The SYS_MEM
+       target values are documented here for hardware completeness but are
+       not used by the driver.
+
+64KB Alignment Requirement
+---------------------------
+
+The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is enforced
+by the ``BASE_ADDR`` field representing bits [39:16] of the target address::
+
+    VRAM Address Calculation:
+    actual_vram_addr = (BASE_ADDR << 16) + pramin_offset
+    Where:
+    - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0]
+    - pramin_offset: 20-bit offset within the PRAMIN window [0x00000-0xFFFFF]
+
+    Example Window Positioning:
+    +---------------------------------------------------------+
+    |                    VRAM Space                           |
+    |                                                         |
+    |  0x000000000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x0000FFFFF  +-----------------+                       |
+    |                                                         |
+    |       |              ^                                  |
+    |       |              | Window can slide                 |
+    |       v              | to any 64KB-aligned boundary     |
+    |                                                         |
+    |  0x123400000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x1234FFFFF  +-----------------+                       |
+    |                                                         |
+    |                       ...                               |
+    |                                                         |
+    |  0x7FFFF0000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x7FFFFFFFF  +-----------------+                       |
+    +---------------------------------------------------------+
Documentation/gpu/nova/index.rst +1 -0
index e39cb3163581..b8254b1ffe2a 100644
@@ -32,3 +32,4 @@ vGPU manager VFIO driver and the nova-drm driver.
    core/devinit
    core/fwsec
    core/falcon
+   core/pramin
Signature · 1 line
2.34.1
#21 lore raw

PATCH v9 8/23 gpu: nova-core: mm: Add common memory management types

In reply to Joel Fernandes

Add foundational types for GPU memory management. These types are used throughout the nova memory management subsystem for page table operations, address translation, and memory allocation.

---
 drivers/gpu/nova-core/mm.rs | 160 ++++++++++++++++++++++++++++++++++++
 1 file changed, 160 insertions(+)
drivers/gpu/nova-core/mm.rs +160 -0
index 7a5dd4220c67..b2cb245b38b7 100644
@@ -2,4 +2,164 @@
 
 //! Memory management subsystems for nova-core.
 
+#![expect(dead_code)]
+
 pub(crate) mod pramin;
+
+use kernel::sizes::SZ_4K;
+
+use crate::num::u64_as_usize;
+
+/// Page size in bytes (4 KiB).
+pub(crate) const PAGE_SIZE: usize = SZ_4K;
+
+bitfield! {
+    pub(crate) struct VramAddress(u64), "Physical VRAM address in GPU video memory" {
+        11:0    offset          as u64, "Offset within 4KB page";
+        63:12   frame_number    as u64 => Pfn, "Physical frame number";
+    }
+}
+
+impl VramAddress {
+    /// Create a new VRAM address from a raw value.
+    pub(crate) const fn new(addr: u64) -> Self {
+        Self(addr)
+    }
+
+    /// Get the raw address value as `usize` (useful for MMIO offsets).
+    pub(crate) const fn raw(&self) -> usize {
+        u64_as_usize(self.0)
+    }
+
+    /// Get the raw address value as `u64`.
+    pub(crate) const fn raw_u64(&self) -> u64 {
+        self.0
+    }
+}
+
+impl PartialEq for VramAddress {
+    fn eq(&self, other: &Self) -> bool {
+        self.0 == other.0
+    }
+}
+
+impl Eq for VramAddress {}
+
+impl PartialOrd for VramAddress {
+    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+        Some(self.cmp(other))
+    }
+}
+
+impl Ord for VramAddress {
+    fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+        self.0.cmp(&other.0)
+    }
+}
+
+impl From<Pfn> for VramAddress {
+    fn from(pfn: Pfn) -> Self {
+        Self::default().set_frame_number(pfn)
+    }
+}
+
+// GPU virtual address.
+bitfield! {
+    pub(crate) struct VirtualAddress(u64), "Virtual address in GPU address space" {
+        11:0    offset          as u64, "Offset within 4KB page";
+        63:12   frame_number    as u64 => Vfn, "Virtual frame number";
+    }
+}
+
+impl VirtualAddress {
+    /// Create a new virtual address from a raw value.
+    #[expect(dead_code)]
+    pub(crate) const fn new(addr: u64) -> Self {
+        Self(addr)
+    }
+
+    /// Get the raw address value as `u64`.
+    pub(crate) const fn raw_u64(&self) -> u64 {
+        self.0
+    }
+}
+
+impl From<Vfn> for VirtualAddress {
+    fn from(vfn: Vfn) -> Self {
+        Self::default().set_frame_number(vfn)
+    }
+}
+
+/// Physical Frame Number.
+///
+/// Represents a physical page in VRAM.
+#[repr(transparent)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub(crate) struct Pfn(u64);
+
+impl Pfn {
+    /// Create a new PFN from a frame number.
+    pub(crate) const fn new(frame_number: u64) -> Self {
+        Self(frame_number)
+    }
+
+    /// Get the raw frame number.
+    pub(crate) const fn raw(self) -> u64 {
+        self.0
+    }
+}
+
+impl From<VramAddress> for Pfn {
+    fn from(addr: VramAddress) -> Self {
+        addr.frame_number()
+    }
+}
+
+impl From<u64> for Pfn {
+    fn from(val: u64) -> Self {
+        Self(val)
+    }
+}
+
+impl From<Pfn> for u64 {
+    fn from(pfn: Pfn) -> Self {
+        pfn.0
+    }
+}
+
+/// Virtual Frame Number.
+///
+/// Represents a virtual page in GPU address space.
+#[repr(transparent)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub(crate) struct Vfn(u64);
+
+impl Vfn {
+    /// Create a new VFN from a frame number.
+    pub(crate) const fn new(frame_number: u64) -> Self {
+        Self(frame_number)
+    }
+
+    /// Get the raw frame number.
+    pub(crate) const fn raw(self) -> u64 {
+        self.0
+    }
+}
+
+impl From<VirtualAddress> for Vfn {
+    fn from(addr: VirtualAddress) -> Self {
+        addr.frame_number()
+    }
+}
+
+impl From<u64> for Vfn {
+    fn from(val: u64) -> Self {
+        Self(val)
+    }
+}
+
+impl From<Vfn> for u64 {
+    fn from(vfn: Vfn) -> Self {
+        vfn.0
+    }
+}
Signature · 1 line
2.34.1
#22 lore raw

PATCH v9 9/23 gpu: nova-core: mm: Add TLB flush support

In reply to Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 90 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 33 ++++++++++++
 3 files changed, 124 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index b2cb245b38b7..b02dc265a2c8 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +90 -0

90 lines not on this page · raw message

drivers/gpu/nova-core/regs.rs +33 -0
index 8ec35b8c4b28..ff6faa9a7c5c 100644
@@ -455,3 +455,36 @@ pub(crate) mod ga100 {
         0:0     display_disabled as bool;
     });
 }
+
+// MMU TLB
+
+register!(NV_TLB_FLUSH_PDB_LO @ 0x00b830a0, "TLB flush register: PDB address bits [39:8]" {
+    31:0    pdb_lo as u32, "PDB address bits [39:8]";
+});
+
+impl NV_TLB_FLUSH_PDB_LO {
+    /// Create a register value from a PDB address.
+    ///
+    /// Extracts bits [39:8] of the address and shifts it right by 8 bits.
+    pub(crate) fn from_pdb_addr(addr: u64) -> Self {
+        Self::default().set_pdb_lo(((addr >> 8) & 0xFFFF_FFFF) as u32)
+    }
+}
+
+register!(NV_TLB_FLUSH_PDB_HI @ 0x00b830a4, "TLB flush register: PDB address bits [47:40]" {
+    7:0     pdb_hi as u8, "PDB address bits [47:40]";
+});
+
+impl NV_TLB_FLUSH_PDB_HI {
+    /// Create a register value from a PDB address.
+    ///
+    /// Extracts bits [47:40] of the address and shifts it right by 40 bits.
+    pub(crate) fn from_pdb_addr(addr: u64) -> Self {
+        Self::default().set_pdb_hi(((addr >> 40) & 0xFF) as u8)
+    }
+}
+
+register!(NV_TLB_FLUSH_CTRL @ 0x00b830b0, "TLB flush control register" {
+    0:0     page_all as bool, "Invalidate all pages";
+    31:31   enable as bool, "Enable/trigger flush (clears when flush completes)";
+});
Signature · 1 line
2.34.1
#23 lore raw

PATCH v9 10/23 gpu: nova-core: mm: Add GpuMm centralized memory manager

In reply to Joel Fernandes

Introduce GpuMm as the centralized GPU memory manager that owns:
- Buddy allocator for VRAM allocation.
- PRAMIN window for direct VRAM access.
- TLB manager for translation buffer operations.

This provides clean ownership model where GpuMm provides accessor methods for its components that can be used for memory management operations.

---
 drivers/gpu/nova-core/gpu.rs          | 32 +++++++++++-
 drivers/gpu/nova-core/gsp/commands.rs |  2 -
 drivers/gpu/nova-core/mm.rs           | 70 ++++++++++++++++++++++++++-
 3 files changed, 99 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/gpu.rs +31 -1
index c324d96bd0c6..32266480bb0f 100644
@@ -4,8 +4,10 @@
     device,
     devres::Devres,
     fmt,
+    gpu::buddy::GpuBuddyParams,
     pci,
     prelude::*,
+    sizes::SZ_4K,
     sync::Arc, //
 };
 
@@ -22,6 +24,7 @@
         commands::GetGspStaticInfoReply,
         Gsp, //
     },
+    mm::GpuMm,
     regs,
 };
 
@@ -252,6 +255,9 @@ pub(crate) struct Gpu {
     gsp_falcon: Falcon<GspFalcon>,
     /// SEC2 falcon instance, used for GSP boot up and cleanup.
     sec2_falcon: Falcon<Sec2Falcon>,
+    /// GPU memory manager owning memory management resources.
+    #[pin]
+    mm: GpuMm,
     /// GSP runtime data. Temporarily an empty placeholder.
     #[pin]
     gsp: Gsp,
@@ -288,7 +294,31 @@ pub(crate) fn new<'a>(
 
             gsp <- Gsp::new(pdev),
 
-            gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
+            gsp_static_info: {
+                let info = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?;
+
+                dev_info!(
+                    pdev.as_ref(),
+                    "Using FB region: {:#x}..{:#x}\n",
+                    info.usable_fb_region.start,
+                    info.usable_fb_region.end
+                );
+
+                info
+            },
+
+            // Create GPU memory manager owning memory management resources.
+            mm <- {
+                let usable_vram = &gsp_static_info.usable_fb_region;
+                // PRAMIN covers all physical VRAM (including GSP-reserved areas
+                // above the usable region, e.g. the BAR1 page directory).
+                let pramin_vram_region = 0..gsp_static_info.total_fb_end;
+                GpuMm::new(devres_bar.clone(), GpuBuddyParams {
+                    base_offset: usable_vram.start,
+                    physical_memory_size: usable_vram.end - usable_vram.start,
+                    chunk_size: SZ_4K,
+                }, pramin_vram_region)?
+            },
 
             bar: devres_bar,
         })
drivers/gpu/nova-core/gsp/commands.rs +0 -2
index 389d215098c6..18dd86a38d46 100644
@@ -191,10 +191,8 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
     /// Usable FB (VRAM) region for driver memory allocation.
-    #[expect(dead_code)]
     pub(crate) usable_fb_region: Range<u64>,
     /// End of VRAM.
-    #[expect(dead_code)]
     pub(crate) total_fb_end: u64,
 }
 
drivers/gpu/nova-core/mm.rs +68 -2

77 lines not on this page · raw message

Signature · 1 line
2.34.1
#24 lore raw

PATCH v9 11/23 gpu: nova-core: mm: Add common types for all page table formats

In reply to Joel Fernandes

Add common page table types shared between MMU v2 and v3. These types are hardware-agnostic and used by both MMU versions.

---
 drivers/gpu/nova-core/mm.rs           |   1 +
 drivers/gpu/nova-core/mm/pagetable.rs | 155 ++++++++++++++++++++++++++
 2 files changed, 156 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable.rs
drivers/gpu/nova-core/mm.rs +1 -0
index dd15175c841d..b0aad90e94bc 100644
@@ -4,6 +4,7 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
 
drivers/gpu/nova-core/mm/pagetable.rs new file +155 -0

155 lines not on this page · raw message

Signature · 1 line
2.34.1
#25 lore raw

PATCH v9 12/23 gpu: nova-core: mm: Add MMU v2 page table types

In reply to Joel Fernandes

Add page table entry and directory structures for MMU version 2 used by Turing/Ampere/Ada GPUs.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   2 +
 drivers/gpu/nova-core/mm/pagetable/ver2.rs | 232 +++++++++++++++++++++
 2 files changed, 234 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
drivers/gpu/nova-core/mm/pagetable.rs +2 -0
index 3946ce2f50a5..2dcd559cc692 100644
@@ -8,6 +8,8 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod ver2;
+
 use crate::gpu::Architecture;
 
 /// Extracts the page table index at a given level from a virtual address.
drivers/gpu/nova-core/mm/pagetable/ver2.rs new file +232 -0

232 lines not on this page · raw message

Signature · 1 line
2.34.1
#26 lore raw

PATCH v9 13/23 gpu: nova-core: mm: Add MMU v3 page table types

In reply to Joel Fernandes

Add page table entry and directory structures for MMU version 3 used by Hopper and later GPUs.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   1 +
 drivers/gpu/nova-core/mm/pagetable/ver3.rs | 337 +++++++++++++++++++++
 2 files changed, 338 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver3.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
index 2dcd559cc692..5c6ae78506af 100644
@@ -9,6 +9,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod ver2;
+pub(crate) mod ver3;
 
 use crate::gpu::Architecture;
 
drivers/gpu/nova-core/mm/pagetable/ver3.rs new file +337 -0

337 lines not on this page · raw message

Signature · 1 line
2.34.1
#27 lore raw

PATCH v9 14/23 gpu: nova-core: mm: Add unified page table entry wrapper enums

In reply to Joel Fernandes

Add unified Pte, Pde, and DualPde wrapper enums that abstract over MMU v2 and v3 page table entry formats. These enums allow the page table walker and VMM to work with both MMU versions.

Each unified type:
- Takes MmuVersion parameter in constructors
- Wraps both ver2 and ver3 variants
- Delegates method calls to the appropriate variant

This enables version-agnostic page table operations while keeping version-specific implementation details encapsulated in the ver2 and ver3 modules.

---
 drivers/gpu/nova-core/mm/pagetable.rs | 322 ++++++++++++++++++++++++++
 1 file changed, 322 insertions(+)
drivers/gpu/nova-core/mm/pagetable.rs +322 -0
index 5c6ae78506af..8cc5f72ead11 100644
@@ -12,6 +12,13 @@
 pub(crate) mod ver3;
 
 use crate::gpu::Architecture;
+use crate::mm::{
+    pramin,
+    Pfn,
+    VirtualAddress,
+    VramAddress, //
+};
+use kernel::prelude::*;
 
 /// Extracts the page table index at a given level from a virtual address.
 pub(crate) trait VaLevelIndex {
@@ -84,6 +91,96 @@ pub(crate) const fn as_index(&self) -> u64 {
     }
 }
 
+impl MmuVersion {
+    /// Get the `PDE` levels (excluding PTE level) for page table walking.
+    pub(crate) fn pde_levels(&self) -> &'static [PageTableLevel] {
+        match self {
+            Self::V2 => ver2::PDE_LEVELS,
+            Self::V3 => ver3::PDE_LEVELS,
+        }
+    }
+
+    /// Get the PTE level for this MMU version.
+    pub(crate) fn pte_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::PTE_LEVEL,
+            Self::V3 => ver3::PTE_LEVEL,
+        }
+    }
+
+    /// Get the dual PDE level (128-bit entries) for this MMU version.
+    pub(crate) fn dual_pde_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::DUAL_PDE_LEVEL,
+            Self::V3 => ver3::DUAL_PDE_LEVEL,
+        }
+    }
+
+    /// Get the number of PDE levels for this MMU version.
+    pub(crate) fn pde_level_count(&self) -> usize {
+        self.pde_levels().len()
+    }
+
+    /// Get the entry size in bytes for a given level.
+    pub(crate) fn entry_size(&self, level: PageTableLevel) -> usize {
+        if level == self.dual_pde_level() {
+            16 // 128-bit dual PDE
+        } else {
+            8 // 64-bit PDE/PTE
+        }
+    }
+
+    /// Get the number of entries per page table page for a given level.
+    pub(crate) fn entries_per_page(&self, level: PageTableLevel) -> usize {
+        match self {
+            Self::V2 => match level {
+                // TODO: Calculate these values from the bitfield dynamically
+                // instead of hardcoding them.
+                PageTableLevel::Pdb => 4,  // PD3 root: bits [48:47] = 2 bits
+                PageTableLevel::L3 => 256, // PD0 dual: bits [28:21] = 8 bits
+                _ => 512,                  // PD2, PD1, PT: 9 bits each
+            },
+            Self::V3 => match level {
+                PageTableLevel::Pdb => 2,  // PDE4 root: bit [56] = 1 bit, 2 entries
+                PageTableLevel::L4 => 256, // PDE0 dual: bits [28:21] = 8 bits
+                _ => 512,                  // PDE3, PDE2, PDE1, PT: 9 bits each
+            },
+        }
+    }
+
+    /// Extract the page table index at `level` from `va` for this MMU version.
+    pub(crate) fn level_index(&self, va: VirtualAddress, level: u64) -> u64 {
+        match self {
+            Self::V2 => ver2::VirtualAddressV2::new(va).level_index(level),
+            Self::V3 => ver3::VirtualAddressV3::new(va).level_index(level),
+        }
+    }
+
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
+    }
+}
+
 /// Memory aperture for Page Table Entries (`PTE`s).
 ///
 /// Determines which memory region the `PTE` points to.
@@ -156,3 +253,228 @@ fn from(val: AperturePde) -> Self {
         val as u8
     }
 }
+
+/// Unified Page Table Entry wrapper for both MMU v2 and v3 `PTE`
+/// types, allowing the walker to work with either format.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum Pte {
+    /// MMU v2 `PTE` (Turing/Ampere/Ada).
+    V2(ver2::Pte),
+    /// MMU v3 `PTE` (Hopper+).
+    V3(ver3::Pte),
+}
+
+impl Pte {
+    /// Create a `PTE` from a raw `u64` value for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, val: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::new(val)),
+            MmuVersion::V3 => Self::V3(ver3::Pte::new(val)),
+        }
+    }
+
+    /// Create an invalid `PTE` for the given MMU version.
+    pub(crate) fn invalid(version: MmuVersion) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::invalid()),
+            MmuVersion::V3 => Self::V3(ver3::Pte::invalid()),
+        }
+    }
+
+    /// Create a valid `PTE` for video memory.
+    pub(crate) fn new_vram(version: MmuVersion, pfn: Pfn, writable: bool) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::new_vram(pfn, writable)),
+            MmuVersion::V3 => Self::V3(ver3::Pte::new_vram(pfn, writable)),
+        }
+    }
+
+    /// Check if this `PTE` is valid.
+    pub(crate) fn is_valid(&self) -> bool {
+        match self {
+            Self::V2(p) => p.valid(),
+            Self::V3(p) => p.valid(),
+        }
+    }
+
+    /// Get the physical frame number.
+    pub(crate) fn frame_number(&self) -> Pfn {
+        match self {
+            Self::V2(p) => p.frame_number(),
+            Self::V3(p) => p.frame_number(),
+        }
+    }
+
+    /// Get the raw `u64` value.
+    pub(crate) fn raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(p) => p.raw_u64(),
+            Self::V3(p) => p.raw_u64(),
+        }
+    }
+
+    /// Read a `PTE` from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let val = window.try_read64(addr.raw())?;
+        Ok(Self::new(mmu_version, val))
+    }
+
+    /// Write this `PTE` to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.raw_u64())
+    }
+}
+
+/// Unified Page Directory Entry wrapper for both MMU v2 and v3 `PDE`.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum Pde {
+    /// MMU v2 `PDE` (Turing/Ampere/Ada).
+    V2(ver2::Pde),
+    /// MMU v3 `PDE` (Hopper+).
+    V3(ver3::Pde),
+}
+
+impl Pde {
+    /// Create a `PDE` from a raw `u64` value for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, val: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::new(val)),
+            MmuVersion::V3 => Self::V3(ver3::Pde::new(val)),
+        }
+    }
+
+    /// Create a valid `PDE` pointing to a page table in video memory.
+    pub(crate) fn new_vram(version: MmuVersion, table_pfn: Pfn) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::new_vram(table_pfn)),
+            MmuVersion::V3 => Self::V3(ver3::Pde::new_vram(table_pfn)),
+        }
+    }
+
+    /// Create an invalid `PDE` for the given MMU version.
+    pub(crate) fn invalid(version: MmuVersion) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::invalid()),
+            MmuVersion::V3 => Self::V3(ver3::Pde::invalid()),
+        }
+    }
+
+    /// Check if this `PDE` is valid.
+    pub(crate) fn is_valid(&self) -> bool {
+        match self {
+            Self::V2(p) => p.is_valid(),
+            Self::V3(p) => p.is_valid(),
+        }
+    }
+
+    /// Get the VRAM address of the page table.
+    pub(crate) fn table_vram_address(&self) -> VramAddress {
+        match self {
+            Self::V2(p) => p.table_vram_address(),
+            Self::V3(p) => p.table_vram_address(),
+        }
+    }
+
+    /// Get the raw `u64` value.
+    pub(crate) fn raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(p) => p.raw_u64(),
+            Self::V3(p) => p.raw_u64(),
+        }
+    }
+
+    /// Read a `PDE` from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let val = window.try_read64(addr.raw())?;
+        Ok(Self::new(mmu_version, val))
+    }
+
+    /// Write this `PDE` to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.raw_u64())
+    }
+}
+
+/// Unified Dual Page Directory Entry wrapper for both MMU v2 and v3 [`DualPde`].
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum DualPde {
+    /// MMU v2 [`DualPde`] (Turing/Ampere/Ada).
+    V2(ver2::DualPde),
+    /// MMU v3 [`DualPde`] (Hopper+).
+    V3(ver3::DualPde),
+}
+
+impl DualPde {
+    /// Create a [`DualPde`] from raw 128-bit value (two `u64`s) for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, big: u64, small: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::DualPde::new(big, small)),
+            MmuVersion::V3 => Self::V3(ver3::DualPde::new(big, small)),
+        }
+    }
+
+    /// Create a [`DualPde`] with only the small page table pointer set.
+    pub(crate) fn new_small(version: MmuVersion, table_pfn: Pfn) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::DualPde::new_small(table_pfn)),
+            MmuVersion::V3 => Self::V3(ver3::DualPde::new_small(table_pfn)),
+        }
+    }
+
+    /// Check if the small page table pointer is valid.
+    pub(crate) fn has_small(&self) -> bool {
+        match self {
+            Self::V2(d) => d.has_small(),
+            Self::V3(d) => d.has_small(),
+        }
+    }
+
+    /// Get the small page table VRAM address.
+    pub(crate) fn small_vram_address(&self) -> VramAddress {
+        match self {
+            Self::V2(d) => d.small.table_vram_address(),
+            Self::V3(d) => d.small.table_vram_address(),
+        }
+    }
+
+    /// Get the raw `u64` value of the big PDE.
+    pub(crate) fn big_raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(d) => d.big.raw_u64(),
+            Self::V3(d) => d.big.raw_u64(),
+        }
+    }
+
+    /// Get the raw `u64` value of the small PDE.
+    pub(crate) fn small_raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(d) => d.small.raw_u64(),
+            Self::V3(d) => d.small.raw_u64(),
+        }
+    }
+
+    /// Read a dual PDE (128-bit) from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let lo = window.try_read64(addr.raw())?;
+        let hi = window.try_read64(addr.raw() + 8)?;
+        Ok(Self::new(mmu_version, lo, hi))
+    }
+
+    /// Write this dual PDE (128-bit) to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.big_raw_u64())?;
+        window.try_write64(addr.raw() + 8, self.small_raw_u64())
+    }
+}
Signature · 1 line
2.34.1
#28 lore raw

PATCH v9 15/23 gpu: nova-core: mm: Add page table walker for MMU v2/v3

In reply to Joel Fernandes

Add the page table walker implementation that traverses the page table hierarchy for both MMU v2 (5-level) and MMU v3 (6-level) to resolve virtual addresses to physical addresses or find PTE locations.

Currently only v2 has been tested (nova-core currently boots pre-hopper) with some initial prepatory work done for v3.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   1 +
 drivers/gpu/nova-core/mm/pagetable/walk.rs | 218 +++++++++++++++++++++
 2 files changed, 219 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
index 8cc5f72ead11..f9996dcac7c6 100644
@@ -10,6 +10,7 @@
 
 pub(crate) mod ver2;
 pub(crate) mod ver3;
+pub(crate) mod walk;
 
 use crate::gpu::Architecture;
 use crate::mm::{
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +218 -0

218 lines not on this page · raw message

Signature · 1 line
2.34.1
#29 lore raw

PATCH v9 16/23 gpu: nova-core: mm: Add Virtual Memory Manager

In reply to Joel Fernandes

Add the Virtual Memory Manager (VMM) infrastructure for GPU address space management. Each Vmm instance manages a single address space identified by its Page Directory Base (PDB) address, used for Channel, BAR1 and BAR2 mappings.

Mapping APIs and virtual address range tracking are added in later commits.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/vmm.rs | 61 +++++++++++++++++++++++++++++++++
 2 files changed, 62 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/vmm.rs
drivers/gpu/nova-core/mm.rs +1 -0
index b0aad90e94bc..6e58f597fadd 100644
@@ -7,6 +7,7 @@
 pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
+pub(crate) mod vmm;
 
 use kernel::{
     devres::Devres,
drivers/gpu/nova-core/mm/vmm.rs new file +61 -0

61 lines not on this page · raw message

Signature · 1 line
2.34.1
#30 lore raw

PATCH v9 17/23 gpu: nova-core: mm: Add virtual address range tracking to VMM

In reply to Joel Fernandes

Add virtual address range tracking to the VMM using a buddy allocator. This enables contiguous virtual address range allocation for mappings.

---
 drivers/gpu/nova-core/mm/vmm.rs | 99 +++++++++++++++++++++++++++++----
 1 file changed, 88 insertions(+), 11 deletions(-)
drivers/gpu/nova-core/mm/vmm.rs +88 -11
index f0e6ffbe2b7a..78e614d8829d 100644
@@ -7,19 +7,35 @@
 //! and handles TLB flushing after modifications.
 
 use kernel::{
-    gpu::buddy::AllocatedBlocks,
-    prelude::*, //
+    gpu::buddy::{
+        AllocatedBlocks,
+        GpuBuddy,
+        GpuBuddyAllocFlag,
+        GpuBuddyAllocMode,
+        GpuBuddyParams, //
+    },
+    prelude::*,
+    ptr::Alignment,
+    sizes::SZ_4K, //
 };
 
-use crate::mm::{
-    pagetable::{
-        walk::{PtWalk, WalkResult},
-        MmuVersion, //
+use core::ops::Range;
+
+use crate::{
+    mm::{
+        pagetable::{
+            walk::{PtWalk, WalkResult},
+            MmuVersion, //
+        },
+        GpuMm,
+        Pfn,
+        Vfn,
+        VramAddress,
+        PAGE_SIZE, //
+    },
+    num::{
+        IntoSafeCast, //
     },
-    GpuMm,
-    Pfn,
-    Vfn,
-    VramAddress, //
 };
 
 /// Virtual Memory Manager for a GPU address space.
@@ -32,23 +48,84 @@ pub(crate) struct Vmm {
     pub(crate) mmu_version: MmuVersion,
     /// Page table allocations required for mappings.
     page_table_allocs: KVec<Pin<KBox<AllocatedBlocks>>>,
+    /// Buddy allocator for virtual address range tracking.
+    virt_buddy: GpuBuddy,
 }
 
 impl Vmm {
     /// Create a new [`Vmm`] for the given Page Directory Base address.
-    pub(crate) fn new(pdb_addr: VramAddress, mmu_version: MmuVersion) -> Result<Self> {
+    ///
+    /// The [`Vmm`] will manage a virtual address space of `va_size` bytes.
+    pub(crate) fn new(
+        pdb_addr: VramAddress,
+        mmu_version: MmuVersion,
+        va_size: u64,
+    ) -> Result<Self> {
         // Only MMU v2 is supported for now.
         if mmu_version != MmuVersion::V2 {
             return Err(ENOTSUPP);
         }
 
+        let virt_buddy = GpuBuddy::new(GpuBuddyParams {
+            base_offset: 0,
+            physical_memory_size: va_size,
+            chunk_size: SZ_4K,
+        })?;
+
         Ok(Self {
             pdb_addr,
             mmu_version,
             page_table_allocs: KVec::new(),
+            virt_buddy,
         })
     }
 
+    /// Allocate a contiguous virtual frame number range.
+    ///
+    /// # Arguments
+    ///
+    /// - `num_pages`: Number of pages to allocate.
+    /// - `va_range`: `None` = allocate anywhere, `Some(range)` = constrain allocation to the given
+    ///   range.
+    pub(crate) fn alloc_vfn_range(
+        &self,
+        num_pages: usize,
+        va_range: Option<Range<u64>>,
+    ) -> Result<(Vfn, Pin<KBox<AllocatedBlocks>>)> {
+        let size = num_pages.checked_mul(PAGE_SIZE).ok_or(EOVERFLOW)?;
+
+        let mode = match va_range {
+            Some(r) => {
+                let range_size = r.end.checked_sub(r.start).ok_or(EOVERFLOW)?;
+                if range_size != size.into_safe_cast() {
+                    return Err(EINVAL);
+                }
+                GpuBuddyAllocMode::Range {
+                    start: r.start,
+                    end: r.end,
+                }
+            }
+            None => GpuBuddyAllocMode::Simple,
+        };
+
+        let alloc = KBox::pin_init(
+            self.virt_buddy.alloc_blocks(
+                mode,
+                size,
+                Alignment::new::<SZ_4K>(),
+                GpuBuddyAllocFlag::Contiguous,
+            ),
+            GFP_KERNEL,
+        )?;
+
+        // Get the starting offset of the first block (only block as range is contiguous).
+        let offset = alloc.iter().next().ok_or(ENOMEM)?.offset();
+        let page_size: u64 = PAGE_SIZE.into_safe_cast();
+        let vfn = Vfn::new(offset / page_size);
+
+        Ok((vfn, alloc))
+    }
+
     /// Read the [`Pfn`] for a mapped [`Vfn`] if one is mapped.
     pub(crate) fn read_mapping(&self, mm: &GpuMm, vfn: Vfn) -> Result<Option<Pfn>> {
         let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
Signature · 1 line
2.34.1
#31 lore raw

PATCH v9 18/23 gpu: nova-core: mm: Add multi-page mapping API to VMM

In reply to Joel Fernandes

Add the page table mapping and unmapping API to the Virtual Memory Manager, implementing a two-phase prepare/execute model suitable for use both inside and outside the DMA fence signalling critical path.

---
 drivers/gpu/nova-core/mm/vmm.rs | 366 +++++++++++++++++++++++++++++++-
 1 file changed, 363 insertions(+), 3 deletions(-)
drivers/gpu/nova-core/mm/vmm.rs +363 -3
index 78e614d8829d..95ee3496e0a6 100644
@@ -11,21 +11,34 @@
         AllocatedBlocks,
         GpuBuddy,
         GpuBuddyAllocFlag,
+        GpuBuddyAllocFlags,
         GpuBuddyAllocMode,
         GpuBuddyParams, //
     },
     prelude::*,
     ptr::Alignment,
+    rbtree::{RBTree, RBTreeNode},
     sizes::SZ_4K, //
 };
 
-use core::ops::Range;
+use core::{
+    cell::Cell,
+    ops::Range, //
+};
 
 use crate::{
     mm::{
         pagetable::{
-            walk::{PtWalk, WalkResult},
-            MmuVersion, //
+            walk::{
+                PtWalk,
+                WalkPdeResult,
+                WalkResult, //
+            },
+            DualPde,
+            MmuVersion,
+            PageTableLevel,
+            Pde,
+            Pte, //
         },
         GpuMm,
         Pfn,
@@ -50,6 +63,74 @@ pub(crate) struct Vmm {
     page_table_allocs: KVec<Pin<KBox<AllocatedBlocks>>>,
     /// Buddy allocator for virtual address range tracking.
     virt_buddy: GpuBuddy,
+    /// Prepared PT pages pending PDE installation, keyed by `install_addr`.
+    ///
+    /// Populated by `Vmm` mapping prepare phase and drained in the execute phase.
+    /// Shared by all pending maps in the `Vmm`, thus preventing races where 2
+    /// maps might be trying to install the same page table/directory entry pointer.
+    pt_pages: RBTree<VramAddress, PreparedPtPage>,
+}
+
+/// A pre-allocated and zeroed page table page.
+///
+/// Created during the mapping prepare phase and consumed during the mapping execute phase.
+/// Stored in an [`RBTree`] keyed by the PDE slot address (`install_addr`).
+struct PreparedPtPage {
+    /// The allocated and zeroed page table page.
+    alloc: Pin<KBox<AllocatedBlocks>>,
+    /// Page table level -- needed to determine if this PT page is for a dual PDE.
+    level: PageTableLevel,
+}
+
+/// Multi-page prepared mapping -- VA range allocated, ready for execute.
+///
+/// Produced by [`Vmm::prepare_map()`], consumed by [`Vmm::execute_map()`].
+/// The struct owns the VA space allocation between prepare and execute phases.
+pub(crate) struct PreparedMapping {
+    vfn_start: Vfn,
+    num_pages: usize,
+    vfn_alloc: Pin<KBox<AllocatedBlocks>>,
+}
+
+/// Result of a mapping operation -- tracks the active mapped range.
+///
+/// Returned by [`Vmm::execute_map()`] and [`Vmm::map_pages()`].
+/// Owns the VA allocation; the VA range is freed when this is dropped.
+/// Callers must call [`Vmm::unmap_pages()`] before dropping to invalidate
+/// PTEs (dropping only frees the VA range, not the PTE entries).
+pub(crate) struct MappedRange {
+    pub(crate) vfn_start: Vfn,
+    pub(crate) num_pages: usize,
+    /// VA allocation -- freed when [`MappedRange`] is dropped.
+    _vfn_alloc: Pin<KBox<AllocatedBlocks>>,
+    /// Logs a warning if dropped without unmapping.
+    _drop_guard: MustUnmapGuard,
+}
+
+/// Guard that logs a warning once if a [`MappedRange`] is dropped without
+/// calling [`Vmm::unmap_pages()`].
+struct MustUnmapGuard {
+    armed: Cell<bool>,
+}
+
+impl MustUnmapGuard {
+    const fn new() -> Self {
+        Self {
+            armed: Cell::new(true),
+        }
+    }
+
+    fn disarm(&self) {
+        self.armed.set(false);
+    }
+}
+
+impl Drop for MustUnmapGuard {
+    fn drop(&mut self) {
+        if self.armed.get() {
+            kernel::pr_warn!("MappedRange dropped without calling unmap_pages()\n");
+        }
+    }
 }
 
 impl Vmm {
@@ -77,6 +158,7 @@ pub(crate) fn new(
             mmu_version,
             page_table_allocs: KVec::new(),
             virt_buddy,
+            pt_pages: RBTree::new(),
         })
     }
 
@@ -135,4 +217,282 @@ pub(crate) fn read_mapping(&self, mm: &GpuMm, vfn: Vfn) -> Result<Option<Pfn>> {
             WalkResult::Unmapped { .. } | WalkResult::PageTableMissing => Ok(None),
         }
     }
+
+    /// Allocate and zero a physical page table page for a specific PDE slot.
+    /// Called during the map prepare phase.
+    fn alloc_and_zero_page_table(
+        &mut self,
+        mm: &GpuMm,
+        level: PageTableLevel,
+    ) -> Result<PreparedPtPage> {
+        let blocks = KBox::pin_init(
+            mm.buddy().alloc_blocks(
+                GpuBuddyAllocMode::Simple,
+                SZ_4K,
+                Alignment::new::<SZ_4K>(),
+                GpuBuddyAllocFlags::default(),
+            ),
+            GFP_KERNEL,
+        )?;
+
+        // Get page's VRAM address from the allocation.
+        let page_vram = VramAddress::new(blocks.iter().next().ok_or(ENOMEM)?.offset());
+
+        // Zero via PRAMIN.
+        let mut window = mm.pramin().window()?;
+        let base = page_vram.raw();
+        for off in (0..PAGE_SIZE).step_by(8) {
+            window.try_write64(base + off, 0)?;
+        }
+
+        Ok(PreparedPtPage {
+            alloc: blocks,
+            level,
+        })
+    }
+
+    /// Ensure all intermediate page table pages are prepared for a [`Vfn`]. Just
+    /// finds out which PDE pages are missing, allocates pages for them, and defers
+    /// installation to the execute phase.
+    ///
+    /// PRAMIN is released before each allocation and re-acquired after. Memory
+    /// allocations are done outside of holding this lock to prevent deadlocks with
+    /// the fence signalling critical path.
+    fn ensure_pte_path(&mut self, mm: &GpuMm, vfn: Vfn) -> Result {
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let max_iter = 2 * self.mmu_version.pde_level_count();
+
+        // Keep looping until all PDE levels are resolved.
+        for _ in 0..max_iter {
+            let mut window = mm.pramin().window()?;
+
+            // Walk PDE levels. The closure checks self.pt_pages for prepared-but-uninstalled
+            // pages, letting the walker continue through them as if they were installed in HW.
+            // The walker keeps calling the closure to get these "prepared but not installed" pages.
+            let result = walker.walk_pde_levels(&mut window, vfn, |install_addr| {
+                self.pt_pages
+                    .get(&install_addr)
+                    .and_then(|p| Some(VramAddress::new(p.alloc.iter().next()?.offset())))
+            })?;
+
+            match result {
+                WalkPdeResult::Complete { .. } => {
+                    // All PDE levels resolved.
+                    return Ok(());
+                }
+                WalkPdeResult::Missing {
+                    install_addr,
+                    level,
+                } => {
+                    // Drop PRAMIN before allocation.
+                    drop(window);
+                    let page = self.alloc_and_zero_page_table(mm, level)?;
+                    let node = RBTreeNode::new(install_addr, page, GFP_KERNEL)?;
+                    let old = self.pt_pages.insert(node);
+                    if old.is_some() {
+                        kernel::pr_warn_once!(
+                            "VMM: duplicate install_addr in pt_pages (internal consistency error)\n"
+                        );
+                        return Err(EIO);
+                    }
+
+                    // Loop: re-acquire PRAMIN and re-walk from root.
+                }
+            }
+        }
+
+        kernel::pr_warn!(
+            "VMM: ensure_pte_path: loop exhausted after {} iters (VFN {:?})\n",
+            max_iter,
+            vfn
+        );
+        Err(EIO)
+    }
+
+    /// Prepare resources for mapping `num_pages` pages.
+    ///
+    /// Allocates a contiguous VA range, then walks the hierarchy per-VFN to prepare pages
+    /// for all missing PDEs. Returns a [`PreparedMapping`] with the VA allocation.
+    ///
+    /// If `va_range` is not `None`, the VA range is constrained to the given range. Safe
+    /// to call outside the fence signalling critical path.
+    pub(crate) fn prepare_map(
+        &mut self,
+        mm: &GpuMm,
+        num_pages: usize,
+        va_range: Option<Range<u64>>,
+    ) -> Result<PreparedMapping> {
+        if num_pages == 0 {
+            return Err(EINVAL);
+        }
+
+        // Pre-reserve so execute_map() can use push_within_capacity (no alloc in
+        // fence signalling critical path).
+        // Upper bound on page table pages needed for the full tree (PTE pages + PDE
+        // pages at all levels).
+        let pt_upper_bound = self.mmu_version.pt_pages_upper_bound(num_pages);
+        self.page_table_allocs.reserve(pt_upper_bound, GFP_KERNEL)?;
+
+        // Allocate contiguous VA range.
+        let (vfn_start, vfn_alloc) = self.alloc_vfn_range(num_pages, va_range)?;
+
+        // Walk the hierarchy per-VFN to prepare pages for all missing PDEs.
+        for i in 0..num_pages {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(vfn_start.raw() + i_u64);
+            self.ensure_pte_path(mm, vfn)?;
+        }
+
+        Ok(PreparedMapping {
+            vfn_start,
+            num_pages,
+            vfn_alloc,
+        })
+    }
+
+    /// Execute a prepared multi-page mapping.
+    ///
+    /// Drain prepared PT pages and install PDEs followed by single TLB flush.
+    pub(crate) fn execute_map(
+        &mut self,
+        mm: &GpuMm,
+        prepared: PreparedMapping,
+        pfns: &[Pfn],
+        writable: bool,
+    ) -> Result<MappedRange> {
+        if pfns.len() != prepared.num_pages {
+            return Err(EINVAL);
+        }
+
+        let PreparedMapping {
+            vfn_start,
+            num_pages,
+            vfn_alloc,
+        } = prepared;
+
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let mut window = mm.pramin().window()?;
+
+        // First, drain self.pt_pages, install all pending PDEs.
+        let mut cursor = self.pt_pages.cursor_front_mut();
+        while let Some(c) = cursor {
+            let (next, node) = c.remove_current();
+            let (install_addr, page) = node.to_key_value();
+            let page_vram = VramAddress::new(page.alloc.iter().next().ok_or(ENOMEM)?.offset());
+
+            if page.level == self.mmu_version.dual_pde_level() {
+                let new_dpde = DualPde::new_small(self.mmu_version, Pfn::from(page_vram));
+                new_dpde.write(&mut window, install_addr)?;
+            } else {
+                let new_pde = Pde::new_vram(self.mmu_version, Pfn::from(page_vram));
+                new_pde.write(&mut window, install_addr)?;
+            }
+
+            // Track the allocated pages in the `Vmm`.
+            self.page_table_allocs
+                .push_within_capacity(page.alloc)
+                .map_err(|_| ENOMEM)?;
+
+            cursor = next;
+        }
+
+        // Next, write PTEs (all PDEs now installed in HW).
+        for (i, &pfn) in pfns.iter().enumerate() {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(vfn_start.raw() + i_u64);
+            let result = walker.walk_to_pte_lookup_with_window(&mut window, vfn)?;
+
+            match result {
+                WalkResult::Unmapped { pte_addr } | WalkResult::Mapped { pte_addr, .. } => {
+                    let pte = Pte::new_vram(self.mmu_version, pfn, writable);
+                    pte.write(&mut window, pte_addr)?;
+                }
+                WalkResult::PageTableMissing => {
+                    kernel::pr_warn_once!("VMM: page table missing for VFN {vfn:?}\n");
+                    return Err(EIO);
+                }
+            }
+        }
+
+        drop(window);
+
+        // Finally, flush the TLB.
+        mm.tlb().flush(self.pdb_addr)?;
+
+        Ok(MappedRange {
+            vfn_start,
+            num_pages,
+            _vfn_alloc: vfn_alloc,
+            _drop_guard: MustUnmapGuard::new(),
+        })
+    }
+
+    /// Map pages doing prepare and execute in the same call.
+    ///
+    /// This is a convenience wrapper for callers outside the fence signalling critical
+    /// path (e.g., BAR mappings). For DRM usecases, [`Vmm::prepare_map()`] and
+    /// [`Vmm::execute_map()`] will be called separately.
+    pub(crate) fn map_pages(
+        &mut self,
+        mm: &GpuMm,
+        pfns: &[Pfn],
+        va_range: Option<Range<u64>>,
+        writable: bool,
+    ) -> Result<MappedRange> {
+        if pfns.is_empty() {
+            return Err(EINVAL);
+        }
+
+        // Check if provided VA range is sufficient (if provided).
+        if let Some(ref range) = va_range {
+            let required: u64 = pfns
+                .len()
+                .checked_mul(PAGE_SIZE)
+                .ok_or(EOVERFLOW)?
+                .into_safe_cast();
+            let available = range.end.checked_sub(range.start).ok_or(EINVAL)?;
+            if available < required {
+                return Err(EINVAL);
+            }
+        }
+
+        let prepared = self.prepare_map(mm, pfns.len(), va_range)?;
+        self.execute_map(mm, prepared, pfns, writable)
+    }
+
+    /// Unmap all pages in a [`MappedRange`] with a single TLB flush.
+    ///
+    /// Takes the range by value (consumes it), then invalidates PTEs for the range,
+    /// flushes the TLB, then drops the range (freeing the VA). PRAMIN lock is held.
+    pub(crate) fn unmap_pages(&mut self, mm: &GpuMm, range: MappedRange) -> Result {
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let invalid_pte = Pte::invalid(self.mmu_version);
+
+        let mut window = mm.pramin().window()?;
+        for i in 0..range.num_pages {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(range.vfn_start.raw() + i_u64);
+            let result = walker.walk_to_pte_lookup_with_window(&mut window, vfn)?;
+
+            match result {
+                WalkResult::Mapped { pte_addr, .. } | WalkResult::Unmapped { pte_addr } => {
+                    invalid_pte.write(&mut window, pte_addr)?;
+                }
+                WalkResult::PageTableMissing => {
+                    continue;
+                }
+            }
+        }
+        drop(window);
+
+        mm.tlb().flush(self.pdb_addr)?;
+
+        // TODO: Internal page table pages (PDE, PTE pages) are still kept around.
+        // This is by design as repeated maps/unmaps will be fast. As a future TODO,
+        // we can add a reclaimer here to reclaim if VRAM is short. For now, the PT
+        // pages are dropped once the `Vmm` is dropped.
+
+        range._drop_guard.disarm(); // Unmap complete, Ok to drop MappedRange.
+        Ok(())
+    }
 }
Signature · 1 line
2.34.1
#32 lore raw

PATCH v9 19/23 gpu: nova-core: Add BAR1 aperture type and size constant

In reply to Joel Fernandes

Add BAR1_SIZE constant and Bar1 type alias for the 256MB BAR1 aperture. These are prerequisites for BAR1 memory access functionality.

---
 drivers/gpu/nova-core/driver.rs          | 8 +++++++-
 drivers/gpu/nova-core/gsp/commands.rs    | 4 ++++
 drivers/gpu/nova-core/gsp/fw/commands.rs | 8 ++++++++
 3 files changed, 19 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/driver.rs +7 -1
index 84b0e1703150..b4311adf4cef 100644
@@ -13,7 +13,10 @@
         Vendor, //
     },
     prelude::*,
-    sizes::SZ_16M,
+    sizes::{
+        SZ_16M,
+        SZ_256M, //
+    },
     sync::{
         atomic::{
             Atomic,
@@ -37,6 +40,7 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
+pub(crate) const BAR1_SIZE: usize = SZ_256M;
 
 // For now we only support Ampere which can use up to 47-bit DMA addresses.
 //
@@ -47,6 +51,8 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
+#[expect(dead_code)]
+pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
 
 kernel::pci_device_table!(
     PCI_TABLE,
drivers/gpu/nova-core/gsp/commands.rs +4 -0
index 18dd86a38d46..1901c8928ab8 100644
@@ -190,6 +190,9 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
 /// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    /// BAR1 Page Directory Entry base address.
+    #[expect(dead_code)]
+    pub(crate) bar1_pde_base: u64,
     /// Usable FB (VRAM) region for driver memory allocation.
     pub(crate) usable_fb_region: Range<u64>,
     /// End of VRAM.
@@ -211,6 +214,7 @@ fn read(
 
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
+            bar1_pde_base: msg.bar1_pde_base(),
             usable_fb_region: base..base.saturating_add(size),
             total_fb_end,
         })
drivers/gpu/nova-core/gsp/fw/commands.rs +8 -0
index acaf92cd6735..75a3d602e6ce 100644
@@ -117,6 +117,14 @@ impl GspStaticConfigInfo {
         self.0.gpuNameString
     }
 
+    /// Returns the BAR1 Page Directory Entry base address.
+    ///
+    /// This is the root page table address for BAR1 virtual memory,
+    /// set up by GSP-RM firmware.
+    pub(crate) fn bar1_pde_base(&self) -> u64 {
+        self.0.bar1PdeBase
+    }
+
     /// Extract the first usable FB region from GSP firmware data.
     ///
     /// Returns the first region suitable for driver memory allocation as a `(base, size)` tuple.
Signature · 1 line
2.34.1
#33 lore raw

PATCH v9 20/23 gpu: nova-core: mm: Add BAR1 user interface

In reply to Joel Fernandes

Add the BAR1 user interface for CPU access to GPU video memory through the BAR1 aperture.

---
 drivers/gpu/nova-core/driver.rs       |   1 -
 drivers/gpu/nova-core/gpu.rs          |  17 ++-
 drivers/gpu/nova-core/gsp/commands.rs |   1 -
 drivers/gpu/nova-core/mm.rs           |   1 +
 drivers/gpu/nova-core/mm/bar_user.rs  | 156 ++++++++++++++++++++++++++
 5 files changed, 173 insertions(+), 3 deletions(-)
 create mode 100644 drivers/gpu/nova-core/mm/bar_user.rs
drivers/gpu/nova-core/driver.rs +0 -1
index b4311adf4cef..3bc264a099de 100644
@@ -51,7 +51,6 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
-#[expect(dead_code)]
 pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
 
 kernel::pci_device_table!(
drivers/gpu/nova-core/gpu.rs +16 -1
index 32266480bb0f..efff76313b89 100644
@@ -24,7 +24,12 @@
         commands::GetGspStaticInfoReply,
         Gsp, //
     },
-    mm::GpuMm,
+    mm::{
+        bar_user::BarUser,
+        pagetable::MmuVersion,
+        GpuMm,
+        VramAddress, //
+    },
     regs,
 };
 
@@ -263,6 +268,8 @@ pub(crate) struct Gpu {
     gsp: Gsp,
     /// Static GPU information from GSP.
     gsp_static_info: GetGspStaticInfoReply,
+    /// BAR1 user interface for CPU access to GPU virtual memory.
+    bar_user: BarUser,
 }
 
 impl Gpu {
@@ -320,6 +327,14 @@ pub(crate) fn new<'a>(
                 }, pramin_vram_region)?
             },
 
+            // Create BAR1 user interface for CPU access to GPU virtual memory.
+            bar_user: {
+                let pdb_addr = VramAddress::new(gsp_static_info.bar1_pde_base);
+                let mmu_version = MmuVersion::from(spec.chipset.arch());
+                let bar1_size = pdev.resource_len(1)?;
+                BarUser::new(pdb_addr, mmu_version, bar1_size)?
+            },
+
             bar: devres_bar,
         })
     }
drivers/gpu/nova-core/gsp/commands.rs +0 -1
index 1901c8928ab8..bfbe7bb05755 100644
@@ -191,7 +191,6 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
     /// BAR1 Page Directory Entry base address.
-    #[expect(dead_code)]
     pub(crate) bar1_pde_base: u64,
     /// Usable FB (VRAM) region for driver memory allocation.
     pub(crate) usable_fb_region: Range<u64>,
drivers/gpu/nova-core/mm.rs +1 -0
index 6e58f597fadd..c053d4f3b26c 100644
@@ -4,6 +4,7 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod bar_user;
 pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
drivers/gpu/nova-core/mm/bar_user.rs new file +156 -0

156 lines not on this page · raw message

Signature · 1 line
2.34.1
#34 lore raw

PATCH v9 21/23 gpu: nova-core: mm: Add BAR1 memory management self-tests

In reply to Joel Fernandes

Add self-tests for BAR1 access during driver probe when CONFIG_NOVA_MM_SELFTESTS is enabled (default disabled). This results in testing the Vmm, GPU buddy allocator and BAR1 region all of which should function correctly for the tests to pass.

---
 drivers/gpu/nova-core/Kconfig        |  10 ++
 drivers/gpu/nova-core/driver.rs      |   2 +
 drivers/gpu/nova-core/gpu.rs         |  38 ++++
 drivers/gpu/nova-core/mm/bar_user.rs | 256 +++++++++++++++++++++++++++
 4 files changed, 306 insertions(+)
drivers/gpu/nova-core/Kconfig +10 -0
index 6513007bf66f..35de55aabcfc 100644
@@ -15,3 +15,13 @@ config NOVA_CORE
 	  This driver is work in progress and may not be functional.
 
 	  If M is selected, the module will be called nova_core.
+
+config NOVA_MM_SELFTESTS
+	bool "Memory management self-tests"
+	depends on NOVA_CORE
+	help
+	  Enable self-tests for the memory management subsystem. When enabled,
+	  tests are run during GPU probe to verify PRAMIN aperture access,
+	  page table walking, and BAR1 virtual memory mapping functionality.
+
+	  This is a testing option and is default-disabled.
drivers/gpu/nova-core/driver.rs +2 -0
index 3bc264a099de..b1aafaff0cee 100644
@@ -101,6 +101,8 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> impl PinInit<Self, E
 
             Ok(try_pin_init!(Self {
                 gpu <- Gpu::new(pdev, bar.clone(), bar.access(pdev.as_ref())?),
+                // Run optional GPU selftests.
+                _: { gpu.run_selftests(pdev)? },
                 _reg <- auxiliary::Registration::new(
                     pdev.as_ref(),
                     c"nova-drm",
drivers/gpu/nova-core/gpu.rs +38 -0
index efff76313b89..022b156de0da 100644
@@ -349,4 +349,42 @@ pub(crate) fn unbind(&self, dev: &device::Device<device::Core>) {
             .inspect(|bar| self.sysmem_flush.unregister(bar))
             .is_err());
     }
+
+    /// Run selftests on the constructed [`Gpu`].
+    pub(crate) fn run_selftests(
+        mut self: Pin<&mut Self>,
+        pdev: &pci::Device<device::Bound>,
+    ) -> Result {
+        self.as_mut().run_mm_selftests(pdev)?;
+        Ok(())
+    }
+
+    #[cfg(CONFIG_NOVA_MM_SELFTESTS)]
+    fn run_mm_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) -> Result {
+        use crate::driver::BAR1_SIZE;
+
+        let mmu_version = MmuVersion::from(self.spec.chipset.arch());
+
+        // BAR1 self-tests.
+        let bar1 = Arc::pin_init(
+            pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
+            GFP_KERNEL,
+        )?;
+        let bar1_access = bar1.access(pdev.as_ref())?;
+
+        crate::mm::bar_user::run_self_test(
+            pdev.as_ref(),
+            &self.mm,
+            bar1_access,
+            self.gsp_static_info.bar1_pde_base,
+            mmu_version,
+        )?;
+
+        Ok(())
+    }
+
+    #[cfg(not(CONFIG_NOVA_MM_SELFTESTS))]
+    fn run_mm_selftests(self: Pin<&mut Self>, _pdev: &pci::Device<device::Bound>) -> Result {
+        Ok(())
+    }
 }
drivers/gpu/nova-core/mm/bar_user.rs +256 -0

259 lines not on this page · raw message

Signature · 1 line
2.34.1
#35 lore raw

PATCH v9 22/23 gpu: nova-core: mm: Add PRAMIN aperture self-tests

In reply to Joel Fernandes

Add self-tests for the PRAMIN aperture mechanism to verify correct operation during GPU probe. The tests validate various alignment requirements and corner cases.

The tests are default disabled and behind CONFIG_NOVA_MM_SELFTESTS. When enabled, tests run after GSP boot during probe.

---
 drivers/gpu/nova-core/gpu.rs       |   3 +
 drivers/gpu/nova-core/mm/pramin.rs | 209 +++++++++++++++++++++++++++++
 2 files changed, 212 insertions(+)
drivers/gpu/nova-core/gpu.rs +3 -0
index 022b156de0da..5f4199e41d16 100644
@@ -365,6 +365,9 @@ fn run_mm_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) ->
 
         let mmu_version = MmuVersion::from(self.spec.chipset.arch());
 
+        // PRAMIN aperture self-tests.
+        crate::mm::pramin::run_self_test(pdev.as_ref(), self.mm.pramin(), self.spec.chipset)?;
+
         // BAR1 self-tests.
         let bar1 = Arc::pin_init(
             pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
drivers/gpu/nova-core/mm/pramin.rs +209 -0

218 lines not on this page · raw message

Signature · 1 line
2.34.1
#36 lore raw

PATCH v9 23/23 gpu: nova-core: Use runtime BAR1 size instead of hardcoded 256MB

In reply to Joel Fernandes

From: Zhi Wang <[email protected]>

Remove the hardcoded BAR1_SIZE = SZ_256M constant. On GPUs like L40 the BAR1 aperture is larger than 256MB; using a hardcoded size prevents large BAR1 from working and mapping it would fail.

---
 drivers/gpu/nova-core/driver.rs | 8 ++------
 drivers/gpu/nova-core/gpu.rs    | 7 +------
 2 files changed, 3 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/driver.rs +2 -6
index b1aafaff0cee..6f95f8672158 100644
@@ -13,10 +13,7 @@
         Vendor, //
     },
     prelude::*,
-    sizes::{
-        SZ_16M,
-        SZ_256M, //
-    },
+    sizes::SZ_16M,
     sync::{
         atomic::{
             Atomic,
@@ -40,7 +37,6 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
-pub(crate) const BAR1_SIZE: usize = SZ_256M;
 
 // For now we only support Ampere which can use up to 47-bit DMA addresses.
 //
@@ -51,7 +47,7 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
-pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
+pub(crate) type Bar1 = pci::Bar;
 
 kernel::pci_device_table!(
     PCI_TABLE,
drivers/gpu/nova-core/gpu.rs +1 -6
index 5f4199e41d16..4d4040d56aba 100644
@@ -361,18 +361,13 @@ pub(crate) fn run_selftests(
 
     #[cfg(CONFIG_NOVA_MM_SELFTESTS)]
     fn run_mm_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) -> Result {
-        use crate::driver::BAR1_SIZE;
-
         let mmu_version = MmuVersion::from(self.spec.chipset.arch());
 
         // PRAMIN aperture self-tests.
         crate::mm::pramin::run_self_test(pdev.as_ref(), self.mm.pramin(), self.spec.chipset)?;
 
         // BAR1 self-tests.
-        let bar1 = Arc::pin_init(
-            pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
-            GFP_KERNEL,
-        )?;
+        let bar1 = Arc::pin_init(pdev.iomap_region(1, c"nova-core/bar1"), GFP_KERNEL)?;
         let bar1_access = bar1.access(pdev.as_ref())?;
 
         crate::mm::bar_user::run_self_test(
Signature · 1 line
2.34.1
#37 lore raw

✗ Fi.CI.BUILD: failure for gpu: nova-core: Add memory management support (rev2)

In reply to Joel Fernandes
== Series Details ==

== Series Details ==

Series: gpu: nova-core: Add memory management support (rev2)
URL : https://patchwork.freedesktop.org/series/162102/
State : failure

== Summary ==

Error: patch https://patchwork.freedesktop.org/api/1.0/series/162102/revisions/2/mbox/ not applied
Applying: gpu: nova-core: Select GPU_BUDDY for VRAM allocation
Using index info to reconstruct a base tree...
M drivers/gpu/nova-core/Kconfig
Falling back to patching base and 3-way merge...
Auto-merging drivers/gpu/nova-core/Kconfig
CONFLICT (content): Merge conflict in drivers/gpu/nova-core/Kconfig
error: Failed to merge in the changes.
hint: Use 'git am --show-current-patch=diff' to see the failed patch
Patch failed at 0001 gpu: nova-core: Select GPU_BUDDY for VRAM allocation
When you have resolved this problem, run "git am --continue".
If you prefer to skip this patch, run "git am --skip" instead.
To restore the original branch and stop patching, run "git am --abort".
Build failed, no error log produced

#38 lore raw

v10 cover gpu: nova-core: Add memory management support

In reply to Joel Fernandes

This series adds support for nova-core memory management, including VRAM allocation, PRAMIN, VMM, page table walking, and BAR 1 read/writes. These are critical for channel management, vGPU, and all other memory management uses of nova-core.

The patches are based on drm-rust-next and work on Ampere, and should "just work on Blackwell" once John's Blackwell patches are merged, however it does not depend on those patches and can independently go in.

Change log:

Changes from v9 to v10:
- Rebased and dropped patches already merged in to drm-rust-next.
- GPU_BUDDY select folded into GpuMm patch.
- updated code with new register macro API.
- Refactored fb_regions() to use iterator (Alex Courbot).
- Renamed Pramin::window() to get_window() to make it more clear it is 'acquiring a resource'.
- Converted Bar0WindowTarget to bounded_enum! macro, replacing TryFrom. Allows to use with_* instead of try_with_*.

Changes from v8 to v9:
- Added fixes from Zhi Wang for bitfield position changes in virtual address and larger BAR1 size on some platforms. Tested and working for vGPU usecase!
- Refactored gsp: boot() to return only GspStaticInfo, removing FbLayout (Alex).
- bar1_pde_base and bar2_pde_base are now accessed via GspStaticInfo directly (Alex).
- Added new patch "gsp: Expose total physical VRAM end from FB region info" introducing total_fb_end() to expose VRAM extent (Alex).
- Consolidated usable VRAM and BarUser setup; removed dedicated
"fb: Add usable_vram field to FbLayout", "mm: Use usable VRAM region for buddy allocator", and "mm: Add BarUser to struct Gpu and create at boot".

Changes from v7 to v8:
- Incorporated "Select GPU_BUDDY for VRAM allocation" patch from the dependency series (Alex).
- Significant patch reordering for better logical flow (GSP/FB patches moved earlier, page table patches, Vmm, Bar1, tests) (Alex).
- Replaced several 'as' usages with into_safe_cast() (Danilo, Alex).
- Updated BAR 1 test cases to include exercising the block size API (Eliot, Danilo).

Changes from v6 to v7:
- Addressed DMA fence signalling usecase per Danilo's feedback.

Pre v6:
- Simplified PRAMIN code (John Hubbard, Alex Courbot).
- Handled different MMU versions: ver2 versus ver3 (John Hubbard).
- Added BAR1 usecase so we have user of DRM Buddy / VMM (John Hubbard).
- Iterating over clist/buddy bindings.

Link to v9: https://lore.kernel.org/all/[email protected]/
Link to v8: https://lore.kernel.org/all/[email protected]/
Link to v7: https://lore.kernel.org/all/[email protected]/

Joel Fernandes (20):
gpu: nova-core: gsp: Return GspStaticInfo from boot()
gpu: nova-core: gsp: Extract usable FB region from GSP
gpu: nova-core: gsp: Expose total physical VRAM end from FB region
info
gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
docs: gpu: nova-core: Document the PRAMIN aperture mechanism
gpu: nova-core: mm: Add common memory management types
gpu: nova-core: mm: Add TLB flush support
gpu: nova-core: mm: Add GpuMm centralized memory manager
gpu: nova-core: mm: Add common types for all page table formats
gpu: nova-core: mm: Add MMU v2 page table types
gpu: nova-core: mm: Add MMU v3 page table types
gpu: nova-core: mm: Add unified page table entry wrapper enums
gpu: nova-core: mm: Add page table walker for MMU v2/v3
gpu: nova-core: mm: Add Virtual Memory Manager
gpu: nova-core: mm: Add virtual address range tracking to VMM
gpu: nova-core: mm: Add multi-page mapping API to VMM
gpu: nova-core: Add BAR1 aperture type and size constant
gpu: nova-core: mm: Add BAR1 user interface
gpu: nova-core: mm: Add BAR1 memory management self-tests
gpu: nova-core: mm: Add PRAMIN aperture self-tests

Zhi Wang (1):
gpu: nova-core: Use runtime BAR1 size instead of hardcoded 256MB

 Documentation/gpu/nova/core/pramin.rst     | 123 +++++
 Documentation/gpu/nova/index.rst           |   1 +
 drivers/gpu/nova-core/Kconfig              |  11 +
 drivers/gpu/nova-core/driver.rs            |   3 +
 drivers/gpu/nova-core/gpu.rs               |  94 +++-
 drivers/gpu/nova-core/gsp/boot.rs          |   9 +-
 drivers/gpu/nova-core/gsp/commands.rs      |  18 +-
 drivers/gpu/nova-core/gsp/fw/commands.rs   |  59 ++-
 drivers/gpu/nova-core/mm.rs                | 234 ++++++++++
 drivers/gpu/nova-core/mm/bar_user.rs       | 388 ++++++++++++++++
 drivers/gpu/nova-core/mm/pagetable.rs      | 489 ++++++++++++++++++++
 drivers/gpu/nova-core/mm/pagetable/ver2.rs | 232 ++++++++++
 drivers/gpu/nova-core/mm/pagetable/ver3.rs | 337 ++++++++++++++
 drivers/gpu/nova-core/mm/pagetable/walk.rs | 218 +++++++++
 drivers/gpu/nova-core/mm/pramin.rs         | 489 ++++++++++++++++++++
 drivers/gpu/nova-core/mm/tlb.rs            |  95 ++++
 drivers/gpu/nova-core/mm/vmm.rs            | 499 +++++++++++++++++++++
 drivers/gpu/nova-core/nova_core.rs         |   1 +
 drivers/gpu/nova-core/regs.rs              |  52 +++
 19 files changed, 3344 insertions(+), 8 deletions(-)
 create mode 100644 Documentation/gpu/nova/core/pramin.rst
 create mode 100644 drivers/gpu/nova-core/mm.rs
 create mode 100644 drivers/gpu/nova-core/mm/bar_user.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver3.rs
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs
 create mode 100644 drivers/gpu/nova-core/mm/pramin.rs
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
 create mode 100644 drivers/gpu/nova-core/mm/vmm.rs

base-commit: 76bce7ac51673640a4a46236ea723cf5543268d7

Signature · 1 line
2.34.1
#39 lore raw

PATCH v10 1/21 gpu: nova-core: gsp: Return GspStaticInfo from boot()

Refactor the GSP boot function to return only the GspStaticInfo, removing the FbLayout from the return tuple.

This enables access required for memory management initialization to:
- bar1_pde_base: BAR1 page directory base.
- bar2_pde_base: BAR2 page directory base.
- usable memory regions in vidmem.

---
 drivers/gpu/nova-core/gpu.rs      | 9 +++++++--
 drivers/gpu/nova-core/gsp/boot.rs | 9 ++++++---
 2 files changed, 13 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/gpu.rs +7 -2
index 0f6fe9a1b955..b4da4a1ae156 100644
@@ -21,7 +21,10 @@
     },
     fb::SysmemFlush,
     gfw,
-    gsp::Gsp,
+    gsp::{
+        commands::GetGspStaticInfoReply,
+        Gsp, //
+    },
     regs,
 };
 
@@ -238,6 +241,8 @@ pub(crate) struct Gpu {
     /// GSP runtime data. Temporarily an empty placeholder.
     #[pin]
     gsp: Gsp,
+    /// Static GPU information from GSP.
+    gsp_static_info: GetGspStaticInfoReply,
 }
 
 impl Gpu {
@@ -269,7 +274,7 @@ pub(crate) fn new<'a>(
 
             gsp <- Gsp::new(pdev),
 
-            _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
+            gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
 
             bar: devres_bar,
         })
drivers/gpu/nova-core/gsp/boot.rs +6 -3
index 6f707b3d1a54..d42637db06dd 100644
@@ -33,7 +33,10 @@
     },
     gpu::Chipset,
     gsp::{
-        commands,
+        commands::{
+            self,
+            GetGspStaticInfoReply, //
+        },
         sequencer::{
             GspSequencer,
             GspSequencerParams, //
@@ -145,7 +148,7 @@ pub(crate) fn boot(
         chipset: Chipset,
         gsp_falcon: &Falcon<Gsp>,
         sec2_falcon: &Falcon<Sec2>,
-    ) -> Result {
+    ) -> Result<GetGspStaticInfoReply> {
         let dev = pdev.as_ref();
 
         let bios = Vbios::new(dev, bar)?;
@@ -235,6 +238,6 @@ pub(crate) fn boot(
             Err(e) => dev_warn!(pdev, "GPU name unavailable: {:?}\n", e),
         }
 
-        Ok(())
+        Ok(info)
     }
 }
Signature · 1 line
2.34.1
#40 lore raw

Re: [PATCH v10 01/21] gpu: nova-core: gsp: Return GspStaticInfo from boot()

Quoted 11 lines · Joel Fernandes

Refactor the GSP boot function to return only the GspStaticInfo, removing the FbLayout from the return tuple.

This enables access required for memory management initialization to:
- bar1_pde_base: BAR1 page directory base.
- bar2_pde_base: BAR2 page directory base.
- usable memory regions in vidmem.

---

Please see my feedback from v9[1] which still applies.

[1]: https://lore.kernel.org/all/[email protected]/

#41 lore raw

Re: [PATCH v10 01/21] gpu: nova-core: gsp: Return GspStaticInfo from boot()

In reply to Joel Fernandes
Quoted 2 lines · Joel Fernandes

Refactor the GSP boot function to return only the GspStaticInfo, removing the FbLayout from the return tuple.

We are not returning the FbLayout - that bit was introduced from an earlier revision of this series and is not in the original code.

#42 lore raw

PATCH v10 2/21 gpu: nova-core: gsp: Extract usable FB region from GSP

In reply to Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed as usable_fb_region field for use by the memory subsystem.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 11 ++++--
 drivers/gpu/nova-core/gsp/fw/commands.rs | 44 +++++++++++++++++++++++-
 2 files changed, 52 insertions(+), 3 deletions(-)
drivers/gpu/nova-core/gsp/commands.rs +9 -2
index c89c7b57a751..41742c1633c8 100644
@@ -4,6 +4,7 @@
     array,
     convert::Infallible,
     ffi::FromBytesUntilNulError,
+    ops::Range,
     str::Utf8Error, //
 };
 
@@ -189,22 +190,28 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
     }
 }
 
-/// The reply from the GSP to the [`GetGspInfo`] command.
+/// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    /// Usable FB (VRAM) region for driver memory allocation.
+    #[expect(dead_code)]
+    pub(crate) usable_fb_region: Range<u64>,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
     const FUNCTION: MsgFunction = MsgFunction::GetGspStaticInfo;
     type Message = GspStaticConfigInfo;
-    type InitError = Infallible;
+    type InitError = Error;
 
     fn read(
         msg: &Self::Message,
         _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>,
     ) -> Result<Self, Self::InitError> {
+        let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
+            usable_fb_region: base..base.saturating_add(size),
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +43 -1

56 lines not on this page · raw message

Signature · 1 line
2.34.1
#43 lore raw

Re: [PATCH v10 02/21] gpu: nova-core: gsp: Extract usable FB region from GSP

Quoted 10 lines · Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed as usable_fb_region field for use by the memory subsystem.

---

Please see my feedback from v9[1] which still applies.

[1]: https://lore.kernel.org/all/[email protected]/

thanks

#44 lore raw

Re: [PATCH v10 02/21] gpu: nova-core: gsp: Extract usable FB region from GSP

Quoted 8 lines · Eliot Courtney
Quoted 7 lines · Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed as usable_fb_region field for use by the memory subsystem.

Quoted 7 lines
Quoted 3 lines

---

Please see my feedback from v9[1] which still applies.

[1]: https://lore.kernel.org/all/[email protected]/

Yeah, I am seeing it now. Amidst making the earlier 7.1 merge window for the DRM buddy and earlier patches in the series, I missed this. They seem to be simple nits and I will address them in the next revision. thanks,

--
Joel Fernandes

#45 lore raw

Re: [PATCH v10 02/21] gpu: nova-core: gsp: Extract usable FB region from GSP

Quoted 17 lines · Joel Fernandes
Quoted 14 lines · Eliot Courtney
Quoted 10 lines · Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed as usable_fb_region field for use by the memory subsystem.

---

Please see my feedback from v9[1] which still applies.

Quoted 8 lines
Quoted 1 line

[1]: https://lore.kernel.org/all/[email protected]/

Yeah, I am seeing it now. Amidst making the earlier 7.1 merge window for the DRM buddy and earlier patches in the series, I missed this. They seem to be simple nits and I will address them in the next revision. thanks,

--
Joel Fernandes

No worries. Sorry I have not gotten to more of the patches yet. Trying to get through some more now. Thanks!

#46 lore raw

Re: [PATCH v10 02/21] gpu: nova-core: gsp: Extract usable FB region from GSP

Quoted 24 lines · Eliot Courtney
Quoted 23 lines · Joel Fernandes
Quoted 8 lines · Eliot Courtney
Quoted 7 lines · Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed as usable_fb_region field for use by the memory subsystem.

Quoted 7 lines
Quoted 3 lines

---

Please see my feedback from v9[1] which still applies.

[1]: https://lore.kernel.org/all/[email protected]/

Yeah, I am seeing it now. Amidst making the earlier 7.1 merge window for the DRM buddy and earlier patches in the series, I missed this. They seem to be simple nits and I will address them in the next revision. thanks,

Quoted 5 lines
Quoted 2 lines

--
Joel Fernandes

No worries. Sorry I have not gotten to more of the patches yet. Trying to get through some more now. Thanks!

So far all the comments have been good ones, so thanks. ;-)

--
Joel Fernandes

#47 lore raw

Re: [PATCH v10 02/21] gpu: nova-core: gsp: Extract usable FB region from GSP

In reply to Joel Fernandes
Quoted 27 lines · Joel Fernandes

Add first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.

The extracted region is stored in GetGspStaticInfoReply and exposed as usable_fb_region field for use by the memory subsystem.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 11 ++++--
 drivers/gpu/nova-core/gsp/fw/commands.rs | 44 +++++++++++++++++++++++-
 2 files changed, 52 insertions(+), 3 deletions(-)
drivers/gpu/nova-core/gsp/commands.rs +1 -0
index c89c7b57a751..41742c1633c8 100644
@@ -4,6 +4,7 @@
     array,
     convert::Infallible,
     ffi::FromBytesUntilNulError,
+    ops::Range,
     str::Utf8Error, //
 };
 

drivers/gpu/nova-core/gsp/commands.rs fn init(&self) -> impl Init<Self::Command, Self::InitError> { · line 198

Quoted 10 lines
drivers/gpu/nova-core/gsp/commands.rs +4 -1
@@ -189,22 +190,28 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
     }
 }
 
-/// The reply from the GSP to the [`GetGspInfo`] command.
+/// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    /// Usable FB (VRAM) region for driver memory allocation.
+    #[expect(dead_code)]
+    pub(crate) usable_fb_region: Range<u64>,

Let's print the region when creating the GPU (using dev_info or dev_dbg) - this can be useful to the user, and lets us remove this dead_code.

#48 lore raw

PATCH v10 3/21 gpu: nova-core: gsp: Expose total physical VRAM end from FB region info

In reply to Joel Fernandes

Add total_fb_end() to GspStaticConfigInfo that computes the exclusive end address of the highest valid FB region covering both usable and GSP-reserved areas.

This allows callers to know the full physical VRAM extent, not just the allocatable portion.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 6 ++++++
 drivers/gpu/nova-core/gsp/fw/commands.rs | 7 +++++++
 2 files changed, 13 insertions(+)
drivers/gpu/nova-core/gsp/commands.rs +6 -0
index 41742c1633c8..5e0649024637 100644
@@ -196,6 +196,9 @@ pub(crate) struct GetGspStaticInfoReply {
     /// Usable FB (VRAM) region for driver memory allocation.
     #[expect(dead_code)]
     pub(crate) usable_fb_region: Range<u64>,
+    /// End of VRAM.
+    #[expect(dead_code)]
+    pub(crate) total_fb_end: u64,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
@@ -209,9 +212,12 @@ fn read(
     ) -> Result<Self, Self::InitError> {
         let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
 
+        let total_fb_end = msg.total_fb_end().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
             usable_fb_region: base..base.saturating_add(size),
+            total_fb_end,
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +7 -0
index 9fffa74d03f9..46932d5c8c1d 100644
@@ -163,6 +163,13 @@ pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
             }
         })
     }
+
+    /// Compute the end of physical VRAM from all FB regions.
+    pub(crate) fn total_fb_end(&self) -> Option<u64> {
+        self.fb_regions()
+            .map(|reg| reg.limit.saturating_add(1))
+            .max()
+    }
 }
 
 // SAFETY: Padding is explicit and will not contain uninitialized data.
Signature · 1 line
2.34.1
#49 lore raw

Re: [PATCH v10 03/21] gpu: nova-core: gsp: Expose total physical VRAM end from FB region info

Quoted 45 lines · Joel Fernandes

Add total_fb_end() to GspStaticConfigInfo that computes the exclusive end address of the highest valid FB region covering both usable and GSP-reserved areas.

This allows callers to know the full physical VRAM extent, not just the allocatable portion.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 6 ++++++
 drivers/gpu/nova-core/gsp/fw/commands.rs | 7 +++++++
 2 files changed, 13 insertions(+)
drivers/gpu/nova-core/gsp/commands.rs +6 -0
index 41742c1633c8..5e0649024637 100644
@@ -196,6 +196,9 @@ pub(crate) struct GetGspStaticInfoReply {
     /// Usable FB (VRAM) region for driver memory allocation.
     #[expect(dead_code)]
     pub(crate) usable_fb_region: Range<u64>,
+    /// End of VRAM.
+    #[expect(dead_code)]
+    pub(crate) total_fb_end: u64,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
@@ -209,9 +212,12 @@ fn read(
     ) -> Result<Self, Self::InitError> {
         let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
 
+        let total_fb_end = msg.total_fb_end().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
             usable_fb_region: base..base.saturating_add(size),
+            total_fb_end,
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +0 -0
index 9fffa74d03f9..46932d5c8c1d 100644

drivers/gpu/nova-core/gsp/fw/commands.rs pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> { · line 170

Quoted 8 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +5 -0
@@ -163,6 +163,13 @@ pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
             }
         })
     }
+
+    /// Compute the end of physical VRAM from all FB regions.
+    pub(crate) fn total_fb_end(&self) -> Option<u64> {
+        self.fb_regions()
+            .map(|reg| reg.limit.saturating_add(1))

I think it would be better to used checked_add here.

Quoted 5 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +2 -0
+            .max()
+    }
 }
 
 // SAFETY: Padding is explicit and will not contain uninitialized data.
#50 lore raw

Re: [PATCH v10 03/21] gpu: nova-core: gsp: Expose total physical VRAM end from FB region info

Quoted 6 lines · Eliot Courtney
drivers/gpu/nova-core/gsp/fw/commands.rs +4 -0
+    /// Compute the end of physical VRAM from all FB regions.
+    pub(crate) fn total_fb_end(&self) -> Option<u64> {
+        self.fb_regions()
+            .map(|reg| reg.limit.saturating_add(1))

I think it would be better to used checked_add here.

Fixed, thanks.

Signature · 1 line
Joel Fernandes
#51 lore raw

Re: [PATCH v10 03/21] gpu: nova-core: gsp: Expose total physical VRAM end from FB region info

In reply to Eliot Courtney
Quoted 48 lines · Eliot Courtney
Quoted 47 lines · Joel Fernandes

Add total_fb_end() to GspStaticConfigInfo that computes the exclusive end address of the highest valid FB region covering both usable and GSP-reserved areas.

This allows callers to know the full physical VRAM extent, not just the allocatable portion.

---
 drivers/gpu/nova-core/gsp/commands.rs    | 6 ++++++
 drivers/gpu/nova-core/gsp/fw/commands.rs | 7 +++++++
 2 files changed, 13 insertions(+)
drivers/gpu/nova-core/gsp/commands.rs +6 -0
index 41742c1633c8..5e0649024637 100644
@@ -196,6 +196,9 @@ pub(crate) struct GetGspStaticInfoReply {
     /// Usable FB (VRAM) region for driver memory allocation.
     #[expect(dead_code)]
     pub(crate) usable_fb_region: Range<u64>,
+    /// End of VRAM.
+    #[expect(dead_code)]
+    pub(crate) total_fb_end: u64,
 }
 
 impl MessageFromGsp for GetGspStaticInfoReply {
@@ -209,9 +212,12 @@ fn read(
     ) -> Result<Self, Self::InitError> {
         let (base, size) = msg.first_usable_fb_region().ok_or(ENODEV)?;
 
+        let total_fb_end = msg.total_fb_end().ok_or(ENODEV)?;
+
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
             usable_fb_region: base..base.saturating_add(size),
+            total_fb_end,
         })
     }
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +0 -0
index 9fffa74d03f9..46932d5c8c1d 100644
@@ -163,6 +163,13 @@ pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> {
             }
         })
Quoted 9 lines

drivers/gpu/nova-core/gsp/fw/commands.rs pub(crate) fn first_usable_fb_region(&self) -> Option<(u64, u64)> { · line 170

Quoted 6 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +5 -0
     }
+
+    /// Compute the end of physical VRAM from all FB regions.
+    pub(crate) fn total_fb_end(&self) -> Option<u64> {
+        self.fb_regions()
+            .map(|reg| reg.limit.saturating_add(1))

I think it would be better to used checked_add here.

Done, thanks.

--
Joel Fernandes

#52 lore raw

PATCH v10 4/21 gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM

In reply to Joel Fernandes

PRAMIN apertures are a crucial mechanism to direct read/write to VRAM. Add support for the same.

---
 drivers/gpu/nova-core/mm.rs        |   5 +
 drivers/gpu/nova-core/mm/pramin.rs | 280 +++++++++++++++++++++++++++++
 drivers/gpu/nova-core/nova_core.rs |   1 +
 drivers/gpu/nova-core/regs.rs      |  10 ++
 4 files changed, 296 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm.rs
 create mode 100644 drivers/gpu/nova-core/mm/pramin.rs
drivers/gpu/nova-core/mm.rs new file +5 -0
new file mode 100644
index 000000000000..7a5dd4220c67
@@ -0,0 +1,5 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Memory management subsystems for nova-core.
+
+pub(crate) mod pramin;
drivers/gpu/nova-core/mm/pramin.rs new file +280 -0

280 lines not on this page · raw message

drivers/gpu/nova-core/nova_core.rs +1 -0
index 04a1fa6b25f8..5f716d1b8c1c 100644
@@ -20,6 +20,7 @@
 mod gfw;
 mod gpu;
 mod gsp;
+mod mm;
 #[macro_use]
 mod num;
 mod regs;
drivers/gpu/nova-core/regs.rs +10 -0
index 2f171a4ff9ba..a3ca02345e20 100644
@@ -30,6 +30,7 @@
         Architecture,
         Chipset, //
     },
+    mm::pramin::Bar0WindowTarget,
     num::FromSafeCast,
 };
 
@@ -115,6 +116,15 @@ fn fmt(&self, f: &mut kernel::fmt::Formatter<'_>) -> kernel::fmt::Result {
     }
 }
 
+register! {
+    /// BAR0 window control for PRAMIN access.
+    pub(crate) NV_PBUS_BAR0_WINDOW(u32) @ 0x00001700 {
+        25:24   target ?=> Bar0WindowTarget;
+        /// Window base address (bits 39:16 of FB addr).
+        23:0    window_base;
+    }
+}
+
 // PFB
 
 register! {
Signature · 1 line
2.34.1
#53 lore raw

PATCH v10 5/21 docs: gpu: nova-core: Document the PRAMIN aperture mechanism

In reply to Joel Fernandes

Add documentation for the PRAMIN aperture mechanism used by nova-core for direct VRAM access.

Nova only uses TARGET=VRAM for VRAM access. The SYS_MEM target values are documented for completeness but not used by the driver.

---
 Documentation/gpu/nova/core/pramin.rst | 123 +++++++++++++++++++++++++
 Documentation/gpu/nova/index.rst       |   1 +
 2 files changed, 124 insertions(+)
 create mode 100644 Documentation/gpu/nova/core/pramin.rst
Documentation/gpu/nova/core/pramin.rst new file +123 -0
new file mode 100644
index 000000000000..bcedb6e06d33
@@ -0,0 +1,123 @@
+.. SPDX-License-Identifier: GPL-2.0
+
+=========================
+PRAMIN aperture mechanism
+=========================
+
+.. note::
+   The following description is approximate and current as of the Ampere family.
+   It may change for future generations and is intended to assist in understanding
+   the driver code.
+
+Introduction
+============
+
+PRAMIN is a hardware aperture mechanism that provides CPU access to GPU Video RAM (VRAM) before
+the GPU's Memory Management Unit (MMU) and page tables are initialized. This 1MB sliding window,
+located at a fixed offset within BAR0, is essential for setting up page tables and other critical
+GPU data structures without relying on the GPU's MMU.
+
+Architecture Overview
+=====================
+
+The PRAMIN aperture mechanism is logically implemented by the GPU's PBUS (PCIe Bus Controller Unit)
+and provides a CPU-accessible window into VRAM through the PCIe interface::
+
+    +-----------------+    PCIe     +------------------------------+
+    |      CPU        |<----------->|           GPU                |
+    +-----------------+             |                              |
+                                    |  +----------------------+    |
+                                    |  |       PBUS           |    |
+                                    |  |  (Bus Controller)    |    |
+                                    |  |                      |    |
+                                    |  |  +--------------+<------------ (window starts at
+                                    |  |  |   PRAMIN     |    |    |     BAR0 + 0x700000)
+                                    |  |  |   Window     |    |    |
+                                    |  |  |   (1MB)      |    |    |
+                                    |  |  +--------------+    |    |
+                                    |  |         |            |    |
+                                    |  +---------|------------+    |
+                                    |            |                 |
+                                    |            v                 |
+                                    |  +----------------------+<------------ (Program PRAMIN to any
+                                    |  |       VRAM           |    |    64KB-aligned VRAM boundary)
+                                    |  |    (Several GBs)     |    |
+                                    |  |                      |    |
+                                    |  |  FB[0x000000000000]  |    |
+                                    |  |          ...         |    |
+                                    |  |  FB[0x7FFFFFFFFFF]   |    |
+                                    |  +----------------------+    |
+                                    +------------------------------+
+
+PBUS (PCIe Bus Controller) is responsible for, among other things, handling MMIO
+accesses to the BAR registers.
+
+PRAMIN Window Operation
+=======================
+
+The PRAMIN window provides a 1MB sliding aperture that can be repositioned over
+the entire VRAM address space using the ``NV_PBUS_BAR0_WINDOW`` register.
+
+Window Control Mechanism
+-------------------------
+
+::
+
+    NV_PBUS_BAR0_WINDOW Register (0x1700):
+    +-------+--------+--------------------------------------+
+    | 31:26 | 25:24  |               23:0                   |
+    | RSVD  | TARGET |            BASE_ADDR                 |
+    |       |        |        (bits 39:16 of VRAM address)  |
+    +-------+--------+--------------------------------------+
+
+    The 24-bit BASE_ADDR field encodes bits [39:16] of the target VRAM address,
+    providing 40-bit (1TB) address space coverage with 64KB alignment.
+
+    TARGET field (bits 25:24):
+    - 0x0: VRAM (Video Memory)
+    - 0x1: SYS_MEM_COH (Coherent System Memory)
+    - 0x2: SYS_MEM_NONCOH (Non-coherent System Memory)
+    - 0x3: Reserved
+
+    .. note::
+       Nova only uses TARGET=VRAM (0x0) for video memory access. The SYS_MEM
+       target values are documented here for hardware completeness but are
+       not used by the driver.
+
+64KB Alignment Requirement
+---------------------------
+
+The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is enforced
+by the ``BASE_ADDR`` field representing bits [39:16] of the target address::
+
+    VRAM Address Calculation:
+    actual_vram_addr = (BASE_ADDR << 16) + pramin_offset
+    Where:
+    - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0]
+    - pramin_offset: 20-bit offset within the PRAMIN window [0x00000-0xFFFFF]
+
+    Example Window Positioning:
+    +---------------------------------------------------------+
+    |                    VRAM Space                           |
+    |                                                         |
+    |  0x000000000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x0000FFFFF  +-----------------+                       |
+    |                                                         |
+    |       |              ^                                  |
+    |       |              | Window can slide                 |
+    |       v              | to any 64KB-aligned boundary     |
+    |                                                         |
+    |  0x123400000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x1234FFFFF  +-----------------+                       |
+    |                                                         |
+    |                       ...                               |
+    |                                                         |
+    |  0x7FFFF0000  +-----------------+ <-- 64KB aligned      |
+    |               | PRAMIN Window   |                       |
+    |               |    (1MB)        |                       |
+    |  0x7FFFFFFFF  +-----------------+                       |
+    +---------------------------------------------------------+
Documentation/gpu/nova/index.rst +1 -0
index e39cb3163581..b8254b1ffe2a 100644
@@ -32,3 +32,4 @@ vGPU manager VFIO driver and the nova-drm driver.
    core/devinit
    core/fwsec
    core/falcon
+   core/pramin
Signature · 1 line
2.34.1
#54 lore raw

PATCH v10 6/21 gpu: nova-core: mm: Add common memory management types

In reply to Joel Fernandes

Add foundational types for GPU memory management. These types are used throughout the nova memory management subsystem for page table operations, address translation, and memory allocation.

---
 drivers/gpu/nova-core/mm.rs | 159 ++++++++++++++++++++++++++++++++++++
 1 file changed, 159 insertions(+)
drivers/gpu/nova-core/mm.rs +159 -0
index 7a5dd4220c67..8f3089a5fa88 100644
@@ -2,4 +2,163 @@
 
 //! Memory management subsystems for nova-core.
 
+#![expect(dead_code)]
+
 pub(crate) mod pramin;
+
+use kernel::sizes::SZ_4K;
+
+use crate::num::u64_as_usize;
+
+/// Page size in bytes (4 KiB).
+pub(crate) const PAGE_SIZE: usize = SZ_4K;
+
+bitfield! {
+    pub(crate) struct VramAddress(u64), "Physical VRAM address in GPU video memory" {
+        11:0    offset          as u64, "Offset within 4KB page";
+        63:12   frame_number    as u64 => Pfn, "Physical frame number";
+    }
+}
+
+impl VramAddress {
+    /// Create a new VRAM address from a raw value.
+    pub(crate) const fn new(addr: u64) -> Self {
+        Self(addr)
+    }
+
+    /// Get the raw address value as `usize` (useful for MMIO offsets).
+    pub(crate) const fn raw(&self) -> usize {
+        u64_as_usize(self.0)
+    }
+
+    /// Get the raw address value as `u64`.
+    pub(crate) const fn raw_u64(&self) -> u64 {
+        self.0
+    }
+}
+
+impl PartialEq for VramAddress {
+    fn eq(&self, other: &Self) -> bool {
+        self.0 == other.0
+    }
+}
+
+impl Eq for VramAddress {}
+
+impl PartialOrd for VramAddress {
+    fn partial_cmp(&self, other: &Self) -> Option<core::cmp::Ordering> {
+        Some(self.cmp(other))
+    }
+}
+
+impl Ord for VramAddress {
+    fn cmp(&self, other: &Self) -> core::cmp::Ordering {
+        self.0.cmp(&other.0)
+    }
+}
+
+impl From<Pfn> for VramAddress {
+    fn from(pfn: Pfn) -> Self {
+        Self::default().set_frame_number(pfn)
+    }
+}
+
+bitfield! {
+    pub(crate) struct VirtualAddress(u64), "Virtual address in GPU address space" {
+        11:0    offset          as u64, "Offset within 4KB page";
+        63:12   frame_number    as u64 => Vfn, "Virtual frame number";
+    }
+}
+
+impl VirtualAddress {
+    /// Create a new virtual address from a raw value.
+    #[expect(dead_code)]
+    pub(crate) const fn new(addr: u64) -> Self {
+        Self(addr)
+    }
+
+    /// Get the raw address value as `u64`.
+    pub(crate) const fn raw_u64(&self) -> u64 {
+        self.0
+    }
+}
+
+impl From<Vfn> for VirtualAddress {
+    fn from(vfn: Vfn) -> Self {
+        Self::default().set_frame_number(vfn)
+    }
+}
+
+/// Physical Frame Number.
+///
+/// Represents a physical page in VRAM.
+#[repr(transparent)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub(crate) struct Pfn(u64);
+
+impl Pfn {
+    /// Create a new PFN from a frame number.
+    pub(crate) const fn new(frame_number: u64) -> Self {
+        Self(frame_number)
+    }
+
+    /// Get the raw frame number.
+    pub(crate) const fn raw(self) -> u64 {
+        self.0
+    }
+}
+
+impl From<VramAddress> for Pfn {
+    fn from(addr: VramAddress) -> Self {
+        addr.frame_number()
+    }
+}
+
+impl From<u64> for Pfn {
+    fn from(val: u64) -> Self {
+        Self(val)
+    }
+}
+
+impl From<Pfn> for u64 {
+    fn from(pfn: Pfn) -> Self {
+        pfn.0
+    }
+}
+
+/// Virtual Frame Number.
+///
+/// Represents a virtual page in GPU address space.
+#[repr(transparent)]
+#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
+pub(crate) struct Vfn(u64);
+
+impl Vfn {
+    /// Create a new VFN from a frame number.
+    pub(crate) const fn new(frame_number: u64) -> Self {
+        Self(frame_number)
+    }
+
+    /// Get the raw frame number.
+    pub(crate) const fn raw(self) -> u64 {
+        self.0
+    }
+}
+
+impl From<VirtualAddress> for Vfn {
+    fn from(addr: VirtualAddress) -> Self {
+        addr.frame_number()
+    }
+}
+
+impl From<u64> for Vfn {
+    fn from(val: u64) -> Self {
+        Self(val)
+    }
+}
+
+impl From<Vfn> for u64 {
+    fn from(vfn: Vfn) -> Self {
+        vfn.0
+    }
+}
Signature · 1 line
2.34.1
#55 lore raw

PATCH v10 7/21 gpu: nova-core: mm: Add TLB flush support

In reply to Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 95 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 42 +++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 8f3089a5fa88..cfe9cbe11d57 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +95 -0

95 lines not on this page · raw message

drivers/gpu/nova-core/regs.rs +42 -0

45 lines not on this page · raw message

Signature · 1 line
2.34.1
#56 lore raw

Re: [PATCH v10 07/21] gpu: nova-core: mm: Add TLB flush support

Quoted 78 lines · Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 95 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 42 +++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 8f3089a5fa88..cfe9cbe11d57 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +41 -0
new file mode 100644
index 000000000000..cd3cbcf4c739
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! TLB (Translation Lookaside Buffer) flush support for GPU MMU.
+//!
+//! After modifying page table entries, the GPU's TLB must be flushed to
+//! ensure the new mappings take effect. This module provides TLB flush
+//! functionality for virtual memory managers.
+//!
+//! # Example
+//!
+//! ```ignore
+//! use crate::mm::tlb::Tlb;
+//!
+//! fn page_table_update(tlb: &Tlb, pdb_addr: VramAddress) -> Result<()> {
+//!     // ... modify page tables ...
+//!
+//!     // Flush TLB to make changes visible (polls for completion).
+//!     tlb.flush(pdb_addr)?;
+//!
+//!     Ok(())
+//! }
+//! ```
+
+use kernel::{
+    devres::Devres,
+    io::poll::read_poll_timeout,
+    io::Io,
+    new_mutex,
+    prelude::*,
+    sync::{
+        Arc,
+        Mutex, //
+    },
+    time::Delta, //
+};
+
+use crate::{
+    driver::Bar0,
+    mm::VramAddress,
+    regs, //
+};

drivers/gpu/nova-core/mm/tlb.rs line 51

Quoted 10 lines
drivers/gpu/nova-core/mm/tlb.rs new file +10 -0
+
+/// TLB manager for GPU translation buffer operations.
+#[pin_data]
+pub(crate) struct Tlb {
+    bar: Arc<Devres<Bar0>>,
+    /// TLB flush serialization lock: This lock is acquired during the
+    /// DMA fence signalling critical path. It must NEVER be held across any
+    /// reclaimable CPU memory allocations because the memory reclaim path can
+    /// call `dma_fence_wait()`, which would deadlock with this lock held.
+    #[pin]

This comment says that the lock is acquired during the DMA fence signalling critical path, but IIUC we don't have anything like that right now. Is this based on future yet to be done work? Can we reword this in a way so it makes sense in the current state?

Quoted 7 lines
drivers/gpu/nova-core/mm/tlb.rs new file +7 -0
+    lock: Mutex<()>,
+}
+
+impl Tlb {
+    /// Create a new TLB manager.
+    pub(super) fn new(bar: Arc<Devres<Bar0>>) -> impl PinInit<Self> {
+        pin_init!(Self {

drivers/gpu/nova-core/mm/tlb.rs line 68

Quoted 10 lines
drivers/gpu/nova-core/mm/tlb.rs new file +10 -0
+            bar,
+            lock <- new_mutex!((), "tlb_flush"),
+        })
+    }
+
+    /// Flush the GPU TLB for a specific page directory base.
+    ///
+    /// This invalidates all TLB entries associated with the given PDB address.
+    /// Must be called after modifying page table entries to ensure the GPU sees
+    /// the updated mappings.

If this must be called after every operation like that, I wonder if we can change the design to require a guard like pattern something to ensure flush is called. WDYT?

drivers/gpu/nova-core/mm/tlb.rs line 69

Quoted 1 line
drivers/gpu/nova-core/mm/tlb.rs new file +1 -0
+    pub(crate) fn flush(&self, pdb_addr: VramAddress) -> Result {

Hopefully we don't need to be calling flush() from anywhere in the entire crate. Can you tighten the visibility here and in other places? Many things seem to be pub(crate) that don't need to be.

Quoted 12 lines
drivers/gpu/nova-core/mm/tlb.rs new file +12 -0
+        let _guard = self.lock.lock();
+
+        let bar = self.bar.try_access().ok_or(ENODEV)?;
+
+        // Write PDB address.
+        bar.write_reg(regs::NV_TLB_FLUSH_PDB_LO::from_pdb_addr(pdb_addr.raw_u64()));
+        bar.write_reg(regs::NV_TLB_FLUSH_PDB_HI::from_pdb_addr(pdb_addr.raw_u64()));
+
+        // Trigger flush: invalidate all pages and enable.
+        bar.write_reg(
+            regs::NV_TLB_FLUSH_CTRL::zeroed()
+                .with_page_all(true)

drivers/gpu/nova-core/mm/tlb.rs line 91

Quoted 10 lines
drivers/gpu/nova-core/mm/tlb.rs new file +10 -0
+                .with_enable(true),
+        );
+
+        // Poll for completion - enable bit clears when flush is done.
+        read_poll_timeout(
+            || Ok(bar.read(regs::NV_TLB_FLUSH_CTRL)),
+            |ctrl: &regs::NV_TLB_FLUSH_CTRL| !ctrl.enable(),
+            Delta::ZERO,
+            Delta::from_secs(2),
+        )?;

This has zero delay on the read_poll_timeout - what about adding a small delay of a microsecond or so?

#57 lore raw

Re: [PATCH v10 07/21] gpu: nova-core: mm: Add TLB flush support

Quoted 10 lines · Eliot Courtney

drivers/gpu/nova-core/mm/tlb.rs line 51

Quoted 5 lines
drivers/gpu/nova-core/mm/tlb.rs new file +5 -0
+    /// TLB flush serialization lock: This lock is acquired during the
+    /// DMA fence signalling critical path. It must NEVER be held across any
+    /// reclaimable CPU memory allocations because the memory reclaim path can
+    /// call `dma_fence_wait()`, which would deadlock with this lock held.
+    #[pin]

This comment says that the lock is acquired during the DMA fence signalling critical path, but IIUC we don't have anything like that right now. Is this based on future yet to be done work? Can we reword this in a way so it makes sense in the current state?

Good point. Will reword. I'll still keep the references this design is specifically for that, but will refine to avoid making it look like a fence signalling usecase already exists. Thanks!

Quoted 7 lines

drivers/gpu/nova-core/mm/tlb.rs line 68

Quoted 3 lines
drivers/gpu/nova-core/mm/tlb.rs new file +3 -0
+    /// This invalidates all TLB entries associated with the given PDB address.
+    /// Must be called after modifying page table entries to ensure the GPU sees
+    /// the updated mappings.

If this must be called after every operation like that, I wonder if we can change the design to require a guard like pattern something to ensure flush is called. WDYT?

That's a good thought. I looked into this -- currently there are only 2 call sites (execute_map and unmap_pages in vmm.rs), and both follow the same pattern of dropping the PRAMIN window lock before flushing. A RAII guard would need careful lifetime management to maintain this lock ordering, and error propagation from drop is tricky. With just 2 call sites, the explicit flush calls are clearer and easier to audit. If more call sites emerge in the future, we can revisit adding a guard pattern then.

Quoted 5 lines
Quoted 1 line

+ pub(crate) fn flush(&self, pdb_addr: VramAddress) -> Result {

Hopefully we don't need to be calling flush() from anywhere in the entire crate. Can you tighten the visibility here and in other places? Many things seem to be pub(crate) that don't need to be.

Agreed, will tighten flush() to pub(super) and do a broader visibility audit across the series. Thanks!

Quoted 9 lines

drivers/gpu/nova-core/mm/tlb.rs line 91

Quoted 6 lines
drivers/gpu/nova-core/mm/tlb.rs new file +6 -0
+        read_poll_timeout(
+            || Ok(bar.read(regs::NV_TLB_FLUSH_CTRL)),
+            |ctrl: &regs::NV_TLB_FLUSH_CTRL| !ctrl.enable(),
+            Delta::ZERO,
+            Delta::from_secs(2),
+        )?;

This has zero delay on the read_poll_timeout - what about adding a small delay of a microsecond or so?

I think Delta::ZERO is fine here -- it still calls cpu_relax() on each iteration (not a pure busy-spin). This matches what other DRM drivers do such polls, e.g. lima_mmu.c and panfrost_gpu.c both use read_poll_timeout with sleep_us=0 in some places. I also measured it on GA102 and its about ~1-1.5us which I think is well within busy-loop territory. From my experience with the scheduler, if we sleep, such short sleeps will be much longer than 1us due to scheduling latency. I also checked OpenRM and even there it is a sleepless spin-loop.

thanks,

Signature · 1 line
Joel Fernandes
#58 lore raw

Re: [PATCH v10 07/21] gpu: nova-core: mm: Add TLB flush support

In reply to Joel Fernandes
Quoted 96 lines · Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 95 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 42 +++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 8f3089a5fa88..cfe9cbe11d57 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +59 -0
new file mode 100644
index 000000000000..cd3cbcf4c739
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! TLB (Translation Lookaside Buffer) flush support for GPU MMU.
+//!
+//! After modifying page table entries, the GPU's TLB must be flushed to
+//! ensure the new mappings take effect. This module provides TLB flush
+//! functionality for virtual memory managers.
+//!
+//! # Example
+//!
+//! ```ignore
+//! use crate::mm::tlb::Tlb;
+//!
+//! fn page_table_update(tlb: &Tlb, pdb_addr: VramAddress) -> Result<()> {
+//!     // ... modify page tables ...
+//!
+//!     // Flush TLB to make changes visible (polls for completion).
+//!     tlb.flush(pdb_addr)?;
+//!
+//!     Ok(())
+//! }
+//! ```
+
+use kernel::{
+    devres::Devres,
+    io::poll::read_poll_timeout,
+    io::Io,
+    new_mutex,
+    prelude::*,
+    sync::{
+        Arc,
+        Mutex, //
+    },
+    time::Delta, //
+};
+
+use crate::{
+    driver::Bar0,
+    mm::VramAddress,
+    regs, //
+};
+
+/// TLB manager for GPU translation buffer operations.
+#[pin_data]
+pub(crate) struct Tlb {
+    bar: Arc<Devres<Bar0>>,
+    /// TLB flush serialization lock: This lock is acquired during the
+    /// DMA fence signalling critical path. It must NEVER be held across any
+    /// reclaimable CPU memory allocations because the memory reclaim path can
+    /// call `dma_fence_wait()`, which would deadlock with this lock held.
+    #[pin]
+    lock: Mutex<()>,
+}
+
+impl Tlb {
+    /// Create a new TLB manager.
+    pub(super) fn new(bar: Arc<Devres<Bar0>>) -> impl PinInit<Self> {
+        pin_init!(Self {
+            bar,

drivers/gpu/nova-core/mm/tlb.rs line 69

Quoted 10 lines
drivers/gpu/nova-core/mm/tlb.rs new file +10 -0
+            lock <- new_mutex!((), "tlb_flush"),
+        })
+    }
+
+    /// Flush the GPU TLB for a specific page directory base.
+    ///
+    /// This invalidates all TLB entries associated with the given PDB address.
+    /// Must be called after modifying page table entries to ensure the GPU sees
+    /// the updated mappings.
+    pub(crate) fn flush(&self, pdb_addr: VramAddress) -> Result {

This landed on my list randomly, so I took a look.

Wouldn’t you want to virtualize the invalidation based on your device?

For example, what if you need to register interface changes on future hardware?

Or, if you’re a VF, can you even do MMIO?

I’d relayer this.

Matt

Quoted 79 lines
drivers/gpu/nova-core/mm/tlb.rs new file +26 -0
+        let _guard = self.lock.lock();
+
+        let bar = self.bar.try_access().ok_or(ENODEV)?;
+
+        // Write PDB address.
+        bar.write_reg(regs::NV_TLB_FLUSH_PDB_LO::from_pdb_addr(pdb_addr.raw_u64()));
+        bar.write_reg(regs::NV_TLB_FLUSH_PDB_HI::from_pdb_addr(pdb_addr.raw_u64()));
+
+        // Trigger flush: invalidate all pages and enable.
+        bar.write_reg(
+            regs::NV_TLB_FLUSH_CTRL::zeroed()
+                .with_page_all(true)
+                .with_enable(true),
+        );
+
+        // Poll for completion - enable bit clears when flush is done.
+        read_poll_timeout(
+            || Ok(bar.read(regs::NV_TLB_FLUSH_CTRL)),
+            |ctrl: &regs::NV_TLB_FLUSH_CTRL| !ctrl.enable(),
+            Delta::ZERO,
+            Delta::from_secs(2),
+        )?;
+
+        Ok(())
+    }
+}
drivers/gpu/nova-core/regs.rs +42 -0
index a3ca02345e20..5e3f5933a55c 100644
@@ -548,3 +548,45 @@ pub(crate) mod ga100 {
         }
     }
 }
+
+// MMU TLB
+
+register! {
+    /// TLB flush register: PDB address bits [39:8].
+    pub(crate) NV_TLB_FLUSH_PDB_LO(u32) @ 0x00b830a0 {
+        /// PDB address bits [39:8].
+        31:0    pdb_lo => u32;
+    }
+
+    /// TLB flush register: PDB address bits [47:40].
+    pub(crate) NV_TLB_FLUSH_PDB_HI(u32) @ 0x00b830a4 {
+        /// PDB address bits [47:40].
+        7:0     pdb_hi => u8;
+    }
+
+    /// TLB flush control register.
+    pub(crate) NV_TLB_FLUSH_CTRL(u32) @ 0x00b830b0 {
+        /// Invalidate all pages.
+        0:0     page_all => bool;
+        /// Enable/trigger flush (clears when flush completes).
+        31:31   enable => bool;
+    }
+}
+
+impl NV_TLB_FLUSH_PDB_LO {
+    /// Create a register value from a PDB address.
+    ///
+    /// Extracts bits [39:8] of the address and shifts it right by 8 bits.
+    pub(crate) fn from_pdb_addr(addr: u64) -> Self {
+        Self::zeroed().with_pdb_lo(((addr >> 8) & 0xFFFF_FFFF) as u32)
+    }
+}
+
+impl NV_TLB_FLUSH_PDB_HI {
+    /// Create a register value from a PDB address.
+    ///
+    /// Extracts bits [47:40] of the address and shifts it right by 40 bits.
+    pub(crate) fn from_pdb_addr(addr: u64) -> Self {
+        Self::zeroed().with_pdb_hi(((addr >> 40) & 0xFF) as u8)
+    }
+}
Signature · 1 line
2.34.1
#59 lore raw

Re: [PATCH v10 07/21] gpu: nova-core: mm: Add TLB flush support

Quoted 110 lines · Matthew Brost
Quoted 96 lines · Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 95 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 42 +++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 8f3089a5fa88..cfe9cbe11d57 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +59 -0
new file mode 100644
index 000000000000..cd3cbcf4c739
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! TLB (Translation Lookaside Buffer) flush support for GPU MMU.
+//!
+//! After modifying page table entries, the GPU's TLB must be flushed to
+//! ensure the new mappings take effect. This module provides TLB flush
+//! functionality for virtual memory managers.
+//!
+//! # Example
+//!
+//! ```ignore
+//! use crate::mm::tlb::Tlb;
+//!
+//! fn page_table_update(tlb: &Tlb, pdb_addr: VramAddress) -> Result<()> {
+//!     // ... modify page tables ...
+//!
+//!     // Flush TLB to make changes visible (polls for completion).
+//!     tlb.flush(pdb_addr)?;
+//!
+//!     Ok(())
+//! }
+//! ```
+
+use kernel::{
+    devres::Devres,
+    io::poll::read_poll_timeout,
+    io::Io,
+    new_mutex,
+    prelude::*,
+    sync::{
+        Arc,
+        Mutex, //
+    },
+    time::Delta, //
+};
+
+use crate::{
+    driver::Bar0,
+    mm::VramAddress,
+    regs, //
+};
+
+/// TLB manager for GPU translation buffer operations.
+#[pin_data]
+pub(crate) struct Tlb {
+    bar: Arc<Devres<Bar0>>,
+    /// TLB flush serialization lock: This lock is acquired during the
+    /// DMA fence signalling critical path. It must NEVER be held across any
+    /// reclaimable CPU memory allocations because the memory reclaim path can
+    /// call `dma_fence_wait()`, which would deadlock with this lock held.
+    #[pin]
+    lock: Mutex<()>,
+}
+
+impl Tlb {
+    /// Create a new TLB manager.
+    pub(super) fn new(bar: Arc<Devres<Bar0>>) -> impl PinInit<Self> {
+        pin_init!(Self {
+            bar,

drivers/gpu/nova-core/mm/tlb.rs line 69

Quoted 10 lines
drivers/gpu/nova-core/mm/tlb.rs new file +10 -0
+            lock <- new_mutex!((), "tlb_flush"),
+        })
+    }
+
+    /// Flush the GPU TLB for a specific page directory base.
+    ///
+    /// This invalidates all TLB entries associated with the given PDB address.
+    /// Must be called after modifying page table entries to ensure the GPU sees
+    /// the updated mappings.
+    pub(crate) fn flush(&self, pdb_addr: VramAddress) -> Result {

This landed on my list randomly, so I took a look.

Quoted 2 lines

Wouldn’t you want to virtualize the invalidation based on your device? For example, what if you need to register interface changes on future hardware?

Good point, for future hardware it indeed makes sense. I will do that.

Quoted 1 line

Or, if you’re a VF, can you even do MMIO?

For VFs, TLB flush is typically done directly via VF's BAR0 MMIO. If there is VF-specific handling for any reason, that should probably be a different interface than a per-chip interface (more specifically, checking if Nova is booted on a VF).

thanks,

--
Joel Fernandes

#60 lore raw

Re: [PATCH v10 07/21] gpu: nova-core: mm: Add TLB flush support

Quoted 113 lines · Joel Fernandes
Quoted 110 lines · Matthew Brost
Quoted 96 lines · Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 95 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 42 +++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 8f3089a5fa88..cfe9cbe11d57 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +59 -0
new file mode 100644
index 000000000000..cd3cbcf4c739
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! TLB (Translation Lookaside Buffer) flush support for GPU MMU.
+//!
+//! After modifying page table entries, the GPU's TLB must be flushed to
+//! ensure the new mappings take effect. This module provides TLB flush
+//! functionality for virtual memory managers.
+//!
+//! # Example
+//!
+//! ```ignore
+//! use crate::mm::tlb::Tlb;
+//!
+//! fn page_table_update(tlb: &Tlb, pdb_addr: VramAddress) -> Result<()> {
+//!     // ... modify page tables ...
+//!
+//!     // Flush TLB to make changes visible (polls for completion).
+//!     tlb.flush(pdb_addr)?;
+//!
+//!     Ok(())
+//! }
+//! ```
+
+use kernel::{
+    devres::Devres,
+    io::poll::read_poll_timeout,
+    io::Io,
+    new_mutex,
+    prelude::*,
+    sync::{
+        Arc,
+        Mutex, //
+    },
+    time::Delta, //
+};
+
+use crate::{
+    driver::Bar0,
+    mm::VramAddress,
+    regs, //
+};
+
+/// TLB manager for GPU translation buffer operations.
+#[pin_data]
+pub(crate) struct Tlb {
+    bar: Arc<Devres<Bar0>>,
+    /// TLB flush serialization lock: This lock is acquired during the
+    /// DMA fence signalling critical path. It must NEVER be held across any
+    /// reclaimable CPU memory allocations because the memory reclaim path can
+    /// call `dma_fence_wait()`, which would deadlock with this lock held.
+    #[pin]
+    lock: Mutex<()>,
+}
+
+impl Tlb {
+    /// Create a new TLB manager.
+    pub(super) fn new(bar: Arc<Devres<Bar0>>) -> impl PinInit<Self> {
+        pin_init!(Self {
+            bar,

drivers/gpu/nova-core/mm/tlb.rs line 69

Quoted 10 lines
drivers/gpu/nova-core/mm/tlb.rs new file +10 -0
+            lock <- new_mutex!((), "tlb_flush"),
+        })
+    }
+
+    /// Flush the GPU TLB for a specific page directory base.
+    ///
+    /// This invalidates all TLB entries associated with the given PDB address.
+    /// Must be called after modifying page table entries to ensure the GPU sees
+    /// the updated mappings.
+    pub(crate) fn flush(&self, pdb_addr: VramAddress) -> Result {

This landed on my list randomly, so I took a look.

Quoted 4 lines
Quoted 2 lines

Wouldn’t you want to virtualize the invalidation based on your device? For example, what if you need to register interface changes on future hardware?

Good point, for future hardware it indeed makes sense. I will do that.

Actually, at least in the future as far as I can see, the register definitions are the same for TLB invalidation are the same, so we are good and I will not be making any change in this regard.

But, thanks for raising the point and forcing me to double check!

--
Joel Fernandes

#61 lore raw

Re: [PATCH v10 07/21] gpu: nova-core: mm: Add TLB flush support

Quoted 119 lines · Joel Fernandes
Quoted 116 lines · Joel Fernandes
Quoted 112 lines · Matthew Brost
Quoted 96 lines · Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 95 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 42 +++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 8f3089a5fa88..cfe9cbe11d57 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +59 -0
new file mode 100644
index 000000000000..cd3cbcf4c739
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! TLB (Translation Lookaside Buffer) flush support for GPU MMU.
+//!
+//! After modifying page table entries, the GPU's TLB must be flushed to
+//! ensure the new mappings take effect. This module provides TLB flush
+//! functionality for virtual memory managers.
+//!
+//! # Example
+//!
+//! ```ignore
+//! use crate::mm::tlb::Tlb;
+//!
+//! fn page_table_update(tlb: &Tlb, pdb_addr: VramAddress) -> Result<()> {
+//!     // ... modify page tables ...
+//!
+//!     // Flush TLB to make changes visible (polls for completion).
+//!     tlb.flush(pdb_addr)?;
+//!
+//!     Ok(())
+//! }
+//! ```
+
+use kernel::{
+    devres::Devres,
+    io::poll::read_poll_timeout,
+    io::Io,
+    new_mutex,
+    prelude::*,
+    sync::{
+        Arc,
+        Mutex, //
+    },
+    time::Delta, //
+};
+
+use crate::{
+    driver::Bar0,
+    mm::VramAddress,
+    regs, //
+};
+
+/// TLB manager for GPU translation buffer operations.
+#[pin_data]
+pub(crate) struct Tlb {
+    bar: Arc<Devres<Bar0>>,
+    /// TLB flush serialization lock: This lock is acquired during the
+    /// DMA fence signalling critical path. It must NEVER be held across any
+    /// reclaimable CPU memory allocations because the memory reclaim path can
+    /// call `dma_fence_wait()`, which would deadlock with this lock held.
+    #[pin]
+    lock: Mutex<()>,
+}
+
+impl Tlb {
+    /// Create a new TLB manager.
+    pub(super) fn new(bar: Arc<Devres<Bar0>>) -> impl PinInit<Self> {
+        pin_init!(Self {
+            bar,

drivers/gpu/nova-core/mm/tlb.rs line 69

Quoted 10 lines
drivers/gpu/nova-core/mm/tlb.rs new file +10 -0
+            lock <- new_mutex!((), "tlb_flush"),
+        })
+    }
+
+    /// Flush the GPU TLB for a specific page directory base.
+    ///
+    /// This invalidates all TLB entries associated with the given PDB address.
+    /// Must be called after modifying page table entries to ensure the GPU sees
+    /// the updated mappings.
+    pub(crate) fn flush(&self, pdb_addr: VramAddress) -> Result {

This landed on my list randomly, so I took a look.

Wouldn’t you want to virtualize the invalidation based on your device? For example, what if you need to register interface changes on future hardware?

Quoted 7 lines
Quoted 1 line

Good point, for future hardware it indeed makes sense. I will do that.

Actually, at least in the future as far as I can see, the register definitions are the same for TLB invalidation are the same, so we are good and I will not be making any change in this regard.

But, thanks for raising the point and forcing me to double check!

Not my driver, but this looks like a classic “works now” change that may not hold up later, which is why I replied to something that isn’t really my business.

Again, not my area, but I’ve been through this before. Generally, getting the abstractions right up front pays off.

Matt

Quoted 3 lines

--
Joel Fernandes

#62 lore raw

Re: [PATCH v10 07/21] gpu: nova-core: mm: Add TLB flush support

Quoted 128 lines · Matthew Brost
Quoted 119 lines · Joel Fernandes
Quoted 116 lines · Joel Fernandes
Quoted 112 lines · Matthew Brost
Quoted 96 lines · Joel Fernandes

Add TLB (Translation Lookaside Buffer) flush support for GPU MMU.

After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.

The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout matching the Nouveau driver.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/tlb.rs | 95 +++++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs   | 42 +++++++++++++++
 3 files changed, 138 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 8f3089a5fa88..cfe9cbe11d57 100644
@@ -5,6 +5,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod pramin;
+pub(crate) mod tlb;
 
 use kernel::sizes::SZ_4K;
 
drivers/gpu/nova-core/mm/tlb.rs new file +59 -0
new file mode 100644
index 000000000000..cd3cbcf4c739
@@ -0,0 +1,95 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! TLB (Translation Lookaside Buffer) flush support for GPU MMU.
+//!
+//! After modifying page table entries, the GPU's TLB must be flushed to
+//! ensure the new mappings take effect. This module provides TLB flush
+//! functionality for virtual memory managers.
+//!
+//! # Example
+//!
+//! ```ignore
+//! use crate::mm::tlb::Tlb;
+//!
+//! fn page_table_update(tlb: &Tlb, pdb_addr: VramAddress) -> Result<()> {
+//!     // ... modify page tables ...
+//!
+//!     // Flush TLB to make changes visible (polls for completion).
+//!     tlb.flush(pdb_addr)?;
+//!
+//!     Ok(())
+//! }
+//! ```
+
+use kernel::{
+    devres::Devres,
+    io::poll::read_poll_timeout,
+    io::Io,
+    new_mutex,
+    prelude::*,
+    sync::{
+        Arc,
+        Mutex, //
+    },
+    time::Delta, //
+};
+
+use crate::{
+    driver::Bar0,
+    mm::VramAddress,
+    regs, //
+};
+
+/// TLB manager for GPU translation buffer operations.
+#[pin_data]
+pub(crate) struct Tlb {
+    bar: Arc<Devres<Bar0>>,
+    /// TLB flush serialization lock: This lock is acquired during the
+    /// DMA fence signalling critical path. It must NEVER be held across any
+    /// reclaimable CPU memory allocations because the memory reclaim path can
+    /// call `dma_fence_wait()`, which would deadlock with this lock held.
+    #[pin]
+    lock: Mutex<()>,
+}
+
+impl Tlb {
+    /// Create a new TLB manager.
+    pub(super) fn new(bar: Arc<Devres<Bar0>>) -> impl PinInit<Self> {
+        pin_init!(Self {
+            bar,

drivers/gpu/nova-core/mm/tlb.rs line 69

Quoted 10 lines
drivers/gpu/nova-core/mm/tlb.rs new file +10 -0
+            lock <- new_mutex!((), "tlb_flush"),
+        })
+    }
+
+    /// Flush the GPU TLB for a specific page directory base.
+    ///
+    /// This invalidates all TLB entries associated with the given PDB address.
+    /// Must be called after modifying page table entries to ensure the GPU sees
+    /// the updated mappings.
+    pub(crate) fn flush(&self, pdb_addr: VramAddress) -> Result {

This landed on my list randomly, so I took a look.

Wouldn’t you want to virtualize the invalidation based on your device? For example, what if you need to register interface changes on future hardware?

Quoted 7 lines
Quoted 1 line

Good point, for future hardware it indeed makes sense. I will do that.

Actually, at least in the future as far as I can see, the register definitions are the same for TLB invalidation are the same, so we are good and I will not be making any change in this regard.

But, thanks for raising the point and forcing me to double check!

Quoted 6 lines

Not my driver, but this looks like a classic “works now” change that may not hold up later, which is why I replied to something that isn’t really my business.

Again, not my area, but I’ve been through this before. Generally, getting the abstractions right up front pays off.

Our policy so far has been to introduce abstractions when there is a justifiable hardware difference within the (reduced) set of GPUs that we support. Adding HALs imply using virtual calls, which are harder to trace, or monomorphization using generic types, which complicate the code (and are hard to justify when there is only one implementation).

Tlb is already an abstraction of sorts; the register accesses are well-contained within this leaf module, so if the need arises to support a different scheme this can be done transparently to callers, which is what matter imho.

#63 lore raw

PATCH v10 8/21 gpu: nova-core: mm: Add GpuMm centralized memory manager

In reply to Joel Fernandes

Introduce GpuMm as the centralized GPU memory manager that owns:
- Buddy allocator for VRAM allocation.
- PRAMIN window for direct VRAM access.
- TLB manager for translation buffer operations.

This provides clean ownership model where GpuMm provides accessor methods for its components that can be used for memory management operations.

---
 drivers/gpu/nova-core/Kconfig         |  1 +
 drivers/gpu/nova-core/gpu.rs          | 34 ++++++++++++-
 drivers/gpu/nova-core/gsp/commands.rs |  2 -
 drivers/gpu/nova-core/mm.rs           | 70 ++++++++++++++++++++++++++-
 4 files changed, 102 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/Kconfig +1 -0
index a4f2380654e2..6513007bf66f 100644
@@ -4,6 +4,7 @@ config NOVA_CORE
 	depends on PCI
 	depends on RUST
 	select AUXILIARY_BUS
+	select GPU_BUDDY
 	select RUST_FW_LOADER_ABSTRACTIONS
 	default n
 	help
drivers/gpu/nova-core/gpu.rs +33 -1
index b4da4a1ae156..c49fa9c380b8 100644
@@ -4,10 +4,13 @@
     device,
     devres::Devres,
     fmt,
+    gpu::buddy::GpuBuddyParams,
     io::Io,
     num::Bounded,
     pci,
     prelude::*,
+    ptr::Alignment,
+    sizes::SZ_4K,
     sync::Arc, //
 };
 
@@ -25,6 +28,7 @@
         commands::GetGspStaticInfoReply,
         Gsp, //
     },
+    mm::GpuMm,
     regs,
 };
 
@@ -238,6 +242,9 @@ pub(crate) struct Gpu {
     gsp_falcon: Falcon<GspFalcon>,
     /// SEC2 falcon instance, used for GSP boot up and cleanup.
     sec2_falcon: Falcon<Sec2Falcon>,
+    /// GPU memory manager owning memory management resources.
+    #[pin]
+    mm: GpuMm,
     /// GSP runtime data. Temporarily an empty placeholder.
     #[pin]
     gsp: Gsp,
@@ -274,7 +281,32 @@ pub(crate) fn new<'a>(
 
             gsp <- Gsp::new(pdev),
 
-            gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
+            gsp_static_info: {
+                let info = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?;
+
+                dev_info!(
+                    pdev.as_ref(),
+                    "Using FB region: {:#x}..{:#x}\n",
+                    info.usable_fb_region.start,
+                    info.usable_fb_region.end
+                );
+
+                info
+            },
+
+            // Create GPU memory manager owning memory management resources.
+            mm <- {
+                let usable_vram = &gsp_static_info.usable_fb_region;
+
+                // PRAMIN covers all physical VRAM (including GSP-reserved areas
+                // above the usable region, e.g. the BAR1 page directory).
+                let pramin_vram_region = 0..gsp_static_info.total_fb_end;
+                GpuMm::new(devres_bar.clone(), GpuBuddyParams {
+                    base_offset: usable_vram.start,
+                    size: usable_vram.end - usable_vram.start,
+                    chunk_size: Alignment::new::<SZ_4K>(),
+                }, pramin_vram_region)?
+            },
 
             bar: devres_bar,
         })
drivers/gpu/nova-core/gsp/commands.rs +0 -2
index 5e0649024637..ec03bf94b34e 100644
@@ -194,10 +194,8 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
     /// Usable FB (VRAM) region for driver memory allocation.
-    #[expect(dead_code)]
     pub(crate) usable_fb_region: Range<u64>,
     /// End of VRAM.
-    #[expect(dead_code)]
     pub(crate) total_fb_end: u64,
 }
 
drivers/gpu/nova-core/mm.rs +68 -2

77 lines not on this page · raw message

Signature · 1 line
2.34.1
#64 lore raw

PATCH v10 9/21 gpu: nova-core: mm: Add common types for all page table formats

In reply to Joel Fernandes

Add common page table types shared between MMU v2 and v3. These types are hardware-agnostic and used by both MMU versions.

---
 drivers/gpu/nova-core/mm.rs           |   1 +
 drivers/gpu/nova-core/mm/pagetable.rs | 155 ++++++++++++++++++++++++++
 2 files changed, 156 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 3c34d7f05555..151b9add67d8 100644
@@ -4,6 +4,7 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
 
drivers/gpu/nova-core/mm/pagetable.rs new file +155 -0

155 lines not on this page · raw message

Signature · 1 line
2.34.1
#65 lore raw

PATCH v10 10/21 gpu: nova-core: mm: Add MMU v2 page table types

In reply to Joel Fernandes

Add page table entry and directory structures for MMU version 2 used by Turing/Ampere/Ada GPUs.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   2 +
 drivers/gpu/nova-core/mm/pagetable/ver2.rs | 232 +++++++++++++++++++++
 2 files changed, 234 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
drivers/gpu/nova-core/mm/pagetable.rs +2 -0
index 50b76d5e5aaf..38d88f8f09a9 100644
@@ -8,6 +8,8 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod ver2;
+
 use crate::gpu::Architecture;
 
 /// Extracts the page table index at a given level from a virtual address.
drivers/gpu/nova-core/mm/pagetable/ver2.rs new file +232 -0

232 lines not on this page · raw message

Signature · 1 line
2.34.1
#66 lore raw

Re: [PATCH v10 10/21] gpu: nova-core: mm: Add MMU v2 page table types

Quoted 16 lines · Joel Fernandes

Add page table entry and directory structures for MMU version 2 used by Turing/Ampere/Ada GPUs.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   2 +
 drivers/gpu/nova-core/mm/pagetable/ver2.rs | 232 +++++++++++++++++++++
 2 files changed, 234 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
drivers/gpu/nova-core/mm/pagetable.rs +0 -0
index 50b76d5e5aaf..38d88f8f09a9 100644

drivers/gpu/nova-core/mm/pagetable.rs line 12

Quoted 5 lines
drivers/gpu/nova-core/mm/pagetable.rs +2 -0
@@ -8,6 +8,8 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod ver2;
+

This looks like it has more visibility than necessary. And it seems incorrect for anyone in the crate to care about MMU version details. This can probably be just 'mod ver2'. There are a lot of other types / functions in this series that could have tighter visibility. Could you go through and see if you can reduce a bunch to private or pub(super)?

thanks!

#67 lore raw

Re: [PATCH v10 10/21] gpu: nova-core: mm: Add MMU v2 page table types

Quoted 20 lines · Eliot Courtney
Quoted 19 lines · Joel Fernandes

Add page table entry and directory structures for MMU version 2 used by Turing/Ampere/Ada GPUs.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   2 +
 drivers/gpu/nova-core/mm/pagetable/ver2.rs | 232 +++++++++++++++++++++
 2 files changed, 234 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
drivers/gpu/nova-core/mm/pagetable.rs +0 -0
index 50b76d5e5aaf..38d88f8f09a9 100644
@@ -8,6 +8,8 @@
 
 #![expect(dead_code)]
 
Quoted 9 lines

drivers/gpu/nova-core/mm/pagetable.rs line 12

Quoted 2 lines
drivers/gpu/nova-core/mm/pagetable.rs +2 -0
+pub(crate) mod ver2;
+

This looks like it has more visibility than necessary. And it seems incorrect for anyone in the crate to care about MMU version details. This can probably be just 'mod ver2'. There are a lot of other types / functions in this series that could have tighter visibility. Could you go through and see if you can reduce a bunch to private or pub(super)?

Yes, indeed. I am tightening it now, there are several. Thanks!

--
Joel Fernandes

#68 lore raw

PATCH v10 11/21 gpu: nova-core: mm: Add MMU v3 page table types

In reply to Joel Fernandes

Add page table entry and directory structures for MMU version 3 used by Hopper and later GPUs.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   1 +
 drivers/gpu/nova-core/mm/pagetable/ver3.rs | 337 +++++++++++++++++++++
 2 files changed, 338 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver3.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
index 38d88f8f09a9..6e01a1af5222 100644
@@ -9,6 +9,7 @@
 #![expect(dead_code)]
 
 pub(crate) mod ver2;
+pub(crate) mod ver3;
 
 use crate::gpu::Architecture;
 
drivers/gpu/nova-core/mm/pagetable/ver3.rs new file +337 -0

337 lines not on this page · raw message

Signature · 1 line
2.34.1
#69 lore raw

PATCH v10 12/21 gpu: nova-core: mm: Add unified page table entry wrapper enums

In reply to Joel Fernandes

Add unified Pte, Pde, and DualPde wrapper enums that abstract over MMU v2 and v3 page table entry formats. These enums allow the page table walker and VMM to work with both MMU versions.

Each unified type:
- Takes MmuVersion parameter in constructors
- Wraps both ver2 and ver3 variants
- Delegates method calls to the appropriate variant

This enables version-agnostic page table operations while keeping version-specific implementation details encapsulated in the ver2 and ver3 modules.

---
 drivers/gpu/nova-core/mm/pagetable.rs | 330 ++++++++++++++++++++++++++
 1 file changed, 330 insertions(+)
drivers/gpu/nova-core/mm/pagetable.rs +330 -0
index 6e01a1af5222..909df37c3ee8 100644
@@ -12,6 +12,13 @@
 pub(crate) mod ver3;
 
 use crate::gpu::Architecture;
+use crate::mm::{
+    pramin,
+    Pfn,
+    VirtualAddress,
+    VramAddress, //
+};
+use kernel::prelude::*;
 
 /// Extracts the page table index at a given level from a virtual address.
 pub(crate) trait VaLevelIndex {
@@ -84,6 +91,96 @@ pub(crate) const fn as_index(&self) -> u64 {
     }
 }
 
+impl MmuVersion {
+    /// Get the `PDE` levels (excluding PTE level) for page table walking.
+    pub(crate) fn pde_levels(&self) -> &'static [PageTableLevel] {
+        match self {
+            Self::V2 => ver2::PDE_LEVELS,
+            Self::V3 => ver3::PDE_LEVELS,
+        }
+    }
+
+    /// Get the PTE level for this MMU version.
+    pub(crate) fn pte_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::PTE_LEVEL,
+            Self::V3 => ver3::PTE_LEVEL,
+        }
+    }
+
+    /// Get the dual PDE level (128-bit entries) for this MMU version.
+    pub(crate) fn dual_pde_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::DUAL_PDE_LEVEL,
+            Self::V3 => ver3::DUAL_PDE_LEVEL,
+        }
+    }
+
+    /// Get the number of PDE levels for this MMU version.
+    pub(crate) fn pde_level_count(&self) -> usize {
+        self.pde_levels().len()
+    }
+
+    /// Get the entry size in bytes for a given level.
+    pub(crate) fn entry_size(&self, level: PageTableLevel) -> usize {
+        if level == self.dual_pde_level() {
+            16 // 128-bit dual PDE
+        } else {
+            8 // 64-bit PDE/PTE
+        }
+    }
+
+    /// Get the number of entries per page table page for a given level.
+    pub(crate) fn entries_per_page(&self, level: PageTableLevel) -> usize {
+        match self {
+            Self::V2 => match level {
+                // TODO: Calculate these values from the bitfield dynamically
+                // instead of hardcoding them.
+                PageTableLevel::Pdb => 4, // PD3 root: bits [48:47] = 2 bits
+                PageTableLevel::L3 => 256, // PD0 dual: bits [28:21] = 8 bits
+                _ => 512,                 // PD2, PD1, PT: 9 bits each
+            },
+            Self::V3 => match level {
+                PageTableLevel::Pdb => 2,  // PDE4 root: bit [56] = 1 bit, 2 entries
+                PageTableLevel::L4 => 256, // PDE0 dual: bits [28:21] = 8 bits
+                _ => 512,                  // PDE3, PDE2, PDE1, PT: 9 bits each
+            },
+        }
+    }
+
+    /// Extract the page table index at `level` from `va` for this MMU version.
+    pub(crate) fn level_index(&self, va: VirtualAddress, level: u64) -> u64 {
+        match self {
+            Self::V2 => ver2::VirtualAddressV2::new(va).level_index(level),
+            Self::V3 => ver3::VirtualAddressV3::new(va).level_index(level),
+        }
+    }
+
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
+    }
+}
+
 /// Memory aperture for Page Table Entries (`PTE`s).
 ///
 /// Determines which memory region the `PTE` points to.
@@ -156,3 +253,236 @@ fn from(val: AperturePde) -> Self {
         val as u8
     }
 }
+
+/// Unified Page Table Entry wrapper for both MMU v2 and v3 `PTE`
+/// types, allowing the walker to work with either format.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum Pte {
+    /// MMU v2 `PTE` (Turing/Ampere/Ada).
+    V2(ver2::Pte),
+    /// MMU v3 `PTE` (Hopper+).
+    V3(ver3::Pte),
+}
+
+impl Pte {
+    /// Create a `PTE` from a raw `u64` value for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, val: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::new(val)),
+            MmuVersion::V3 => Self::V3(ver3::Pte::new(val)),
+        }
+    }
+
+    /// Create an invalid `PTE` for the given MMU version.
+    pub(crate) fn invalid(version: MmuVersion) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::invalid()),
+            MmuVersion::V3 => Self::V3(ver3::Pte::invalid()),
+        }
+    }
+
+    /// Create a valid `PTE` for video memory.
+    pub(crate) fn new_vram(version: MmuVersion, pfn: Pfn, writable: bool) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pte::new_vram(pfn, writable)),
+            MmuVersion::V3 => Self::V3(ver3::Pte::new_vram(pfn, writable)),
+        }
+    }
+
+    /// Check if this `PTE` is valid.
+    pub(crate) fn is_valid(&self) -> bool {
+        match self {
+            Self::V2(p) => p.valid(),
+            Self::V3(p) => p.valid(),
+        }
+    }
+
+    /// Get the physical frame number.
+    pub(crate) fn frame_number(&self) -> Pfn {
+        match self {
+            Self::V2(p) => p.frame_number(),
+            Self::V3(p) => p.frame_number(),
+        }
+    }
+
+    /// Get the raw `u64` value.
+    pub(crate) fn raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(p) => p.raw_u64(),
+            Self::V3(p) => p.raw_u64(),
+        }
+    }
+
+    /// Read a `PTE` from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let val = window.try_read64(addr.raw())?;
+        Ok(Self::new(mmu_version, val))
+    }
+
+    /// Write this `PTE` to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.raw_u64())
+    }
+}
+
+/// Unified Page Directory Entry wrapper for both MMU v2 and v3 `PDE`.
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum Pde {
+    /// MMU v2 `PDE` (Turing/Ampere/Ada).
+    V2(ver2::Pde),
+    /// MMU v3 `PDE` (Hopper+).
+    V3(ver3::Pde),
+}
+
+impl Pde {
+    /// Create a `PDE` from a raw `u64` value for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, val: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::new(val)),
+            MmuVersion::V3 => Self::V3(ver3::Pde::new(val)),
+        }
+    }
+
+    /// Create a valid `PDE` pointing to a page table in video memory.
+    pub(crate) fn new_vram(version: MmuVersion, table_pfn: Pfn) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::new_vram(table_pfn)),
+            MmuVersion::V3 => Self::V3(ver3::Pde::new_vram(table_pfn)),
+        }
+    }
+
+    /// Create an invalid `PDE` for the given MMU version.
+    pub(crate) fn invalid(version: MmuVersion) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::Pde::invalid()),
+            MmuVersion::V3 => Self::V3(ver3::Pde::invalid()),
+        }
+    }
+
+    /// Check if this `PDE` is valid.
+    pub(crate) fn is_valid(&self) -> bool {
+        match self {
+            Self::V2(p) => p.is_valid(),
+            Self::V3(p) => p.is_valid(),
+        }
+    }
+
+    /// Get the memory aperture of this `PDE`.
+    pub(crate) fn aperture(&self) -> AperturePde {
+        match self {
+            Self::V2(p) => p.aperture(),
+            Self::V3(p) => p.aperture(),
+        }
+    }
+
+    /// Get the VRAM address of the page table.
+    pub(crate) fn table_vram_address(&self) -> VramAddress {
+        match self {
+            Self::V2(p) => p.table_vram_address(),
+            Self::V3(p) => p.table_vram_address(),
+        }
+    }
+
+    /// Get the raw `u64` value.
+    pub(crate) fn raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(p) => p.raw_u64(),
+            Self::V3(p) => p.raw_u64(),
+        }
+    }
+
+    /// Read a `PDE` from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let val = window.try_read64(addr.raw())?;
+        Ok(Self::new(mmu_version, val))
+    }
+
+    /// Write this `PDE` to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.raw_u64())
+    }
+}
+
+/// Unified Dual Page Directory Entry wrapper for both MMU v2 and v3 [`DualPde`].
+#[derive(Debug, Clone, Copy)]
+pub(crate) enum DualPde {
+    /// MMU v2 [`DualPde`] (Turing/Ampere/Ada).
+    V2(ver2::DualPde),
+    /// MMU v3 [`DualPde`] (Hopper+).
+    V3(ver3::DualPde),
+}
+
+impl DualPde {
+    /// Create a [`DualPde`] from raw 128-bit value (two `u64`s) for the given MMU version.
+    pub(crate) fn new(version: MmuVersion, big: u64, small: u64) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::DualPde::new(big, small)),
+            MmuVersion::V3 => Self::V3(ver3::DualPde::new(big, small)),
+        }
+    }
+
+    /// Create a [`DualPde`] with only the small page table pointer set.
+    pub(crate) fn new_small(version: MmuVersion, table_pfn: Pfn) -> Self {
+        match version {
+            MmuVersion::V2 => Self::V2(ver2::DualPde::new_small(table_pfn)),
+            MmuVersion::V3 => Self::V3(ver3::DualPde::new_small(table_pfn)),
+        }
+    }
+
+    /// Check if the small page table pointer is valid.
+    pub(crate) fn has_small(&self) -> bool {
+        match self {
+            Self::V2(d) => d.has_small(),
+            Self::V3(d) => d.has_small(),
+        }
+    }
+
+    /// Get the small page table VRAM address.
+    pub(crate) fn small_vram_address(&self) -> VramAddress {
+        match self {
+            Self::V2(d) => d.small.table_vram_address(),
+            Self::V3(d) => d.small.table_vram_address(),
+        }
+    }
+
+    /// Get the raw `u64` value of the big PDE.
+    pub(crate) fn big_raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(d) => d.big.raw_u64(),
+            Self::V3(d) => d.big.raw_u64(),
+        }
+    }
+
+    /// Get the raw `u64` value of the small PDE.
+    pub(crate) fn small_raw_u64(&self) -> u64 {
+        match self {
+            Self::V2(d) => d.small.raw_u64(),
+            Self::V3(d) => d.small.raw_u64(),
+        }
+    }
+
+    /// Read a dual PDE (128-bit) from VRAM.
+    pub(crate) fn read(
+        window: &mut pramin::PraminWindow<'_>,
+        addr: VramAddress,
+        mmu_version: MmuVersion,
+    ) -> Result<Self> {
+        let lo = window.try_read64(addr.raw())?;
+        let hi = window.try_read64(addr.raw() + 8)?;
+        Ok(Self::new(mmu_version, lo, hi))
+    }
+
+    /// Write this dual PDE (128-bit) to VRAM.
+    pub(crate) fn write(&self, window: &mut pramin::PraminWindow<'_>, addr: VramAddress) -> Result {
+        window.try_write64(addr.raw(), self.big_raw_u64())?;
+        window.try_write64(addr.raw() + 8, self.small_raw_u64())
+    }
+}
Signature · 1 line
2.34.1
#70 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 121 lines · Joel Fernandes

Add unified Pte, Pde, and DualPde wrapper enums that abstract over MMU v2 and v3 page table entry formats. These enums allow the page table walker and VMM to work with both MMU versions.

Each unified type:
- Takes MmuVersion parameter in constructors
- Wraps both ver2 and ver3 variants
- Delegates method calls to the appropriate variant

This enables version-agnostic page table operations while keeping version-specific implementation details encapsulated in the ver2 and ver3 modules.

---
 drivers/gpu/nova-core/mm/pagetable.rs | 330 ++++++++++++++++++++++++++
 1 file changed, 330 insertions(+)
drivers/gpu/nova-core/mm/pagetable.rs +87 -0
index 6e01a1af5222..909df37c3ee8 100644
@@ -12,6 +12,13 @@
 pub(crate) mod ver3;
 
 use crate::gpu::Architecture;
+use crate::mm::{
+    pramin,
+    Pfn,
+    VirtualAddress,
+    VramAddress, //
+};
+use kernel::prelude::*;
 
 /// Extracts the page table index at a given level from a virtual address.
 pub(crate) trait VaLevelIndex {
@@ -84,6 +91,96 @@ pub(crate) const fn as_index(&self) -> u64 {
     }
 }
 
+impl MmuVersion {
+    /// Get the `PDE` levels (excluding PTE level) for page table walking.
+    pub(crate) fn pde_levels(&self) -> &'static [PageTableLevel] {
+        match self {
+            Self::V2 => ver2::PDE_LEVELS,
+            Self::V3 => ver3::PDE_LEVELS,
+        }
+    }
+
+    /// Get the PTE level for this MMU version.
+    pub(crate) fn pte_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::PTE_LEVEL,
+            Self::V3 => ver3::PTE_LEVEL,
+        }
+    }
+
+    /// Get the dual PDE level (128-bit entries) for this MMU version.
+    pub(crate) fn dual_pde_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::DUAL_PDE_LEVEL,
+            Self::V3 => ver3::DUAL_PDE_LEVEL,
+        }
+    }
+
+    /// Get the number of PDE levels for this MMU version.
+    pub(crate) fn pde_level_count(&self) -> usize {
+        self.pde_levels().len()
+    }
+
+    /// Get the entry size in bytes for a given level.
+    pub(crate) fn entry_size(&self, level: PageTableLevel) -> usize {
+        if level == self.dual_pde_level() {
+            16 // 128-bit dual PDE
+        } else {
+            8 // 64-bit PDE/PTE
+        }
+    }
+
+    /// Get the number of entries per page table page for a given level.
+    pub(crate) fn entries_per_page(&self, level: PageTableLevel) -> usize {
+        match self {
+            Self::V2 => match level {
+                // TODO: Calculate these values from the bitfield dynamically
+                // instead of hardcoding them.
+                PageTableLevel::Pdb => 4, // PD3 root: bits [48:47] = 2 bits
+                PageTableLevel::L3 => 256, // PD0 dual: bits [28:21] = 8 bits
+                _ => 512,                 // PD2, PD1, PT: 9 bits each
+            },
+            Self::V3 => match level {
+                PageTableLevel::Pdb => 2,  // PDE4 root: bit [56] = 1 bit, 2 entries
+                PageTableLevel::L4 => 256, // PDE0 dual: bits [28:21] = 8 bits
+                _ => 512,                  // PDE3, PDE2, PDE1, PT: 9 bits each
+            },
+        }
+    }
+
+    /// Extract the page table index at `level` from `va` for this MMU version.
+    pub(crate) fn level_index(&self, va: VirtualAddress, level: u64) -> u64 {
+        match self {
+            Self::V2 => ver2::VirtualAddressV2::new(va).level_index(level),
+            Self::V3 => ver3::VirtualAddressV3::new(va).level_index(level),
+        }
+    }
+
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+

drivers/gpu/nova-core/mm/pagetable.rs pub(crate) const fn as_index(&self) -> u64 { · line 183

Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable.rs +10 -0
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
+    }
+}
+

We have a lot of matches on the MMU version here (and below in Pte, Pde, DualPde). What about making MmuVersion into a trait (e.g. Mmu) with associated types for Pte, Pde, DualPde which can implement traits defining their common operations too? Then you can parameterise Vmm/PtWalk on this type.

#71 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 129 lines · Eliot Courtney
Quoted 128 lines · Joel Fernandes

Add unified Pte, Pde, and DualPde wrapper enums that abstract over MMU v2 and v3 page table entry formats. These enums allow the page table walker and VMM to work with both MMU versions.

Each unified type:
- Takes MmuVersion parameter in constructors
- Wraps both ver2 and ver3 variants
- Delegates method calls to the appropriate variant

This enables version-agnostic page table operations while keeping version-specific implementation details encapsulated in the ver2 and ver3 modules.

---
 drivers/gpu/nova-core/mm/pagetable.rs | 330 ++++++++++++++++++++++++++
 1 file changed, 330 insertions(+)
drivers/gpu/nova-core/mm/pagetable.rs +94 -0
index 6e01a1af5222..909df37c3ee8 100644
@@ -12,6 +12,13 @@
 pub(crate) mod ver3;
 
 use crate::gpu::Architecture;
+use crate::mm::{
+    pramin,
+    Pfn,
+    VirtualAddress,
+    VramAddress, //
+};
+use kernel::prelude::*;
 
 /// Extracts the page table index at a given level from a virtual address.
 pub(crate) trait VaLevelIndex {
@@ -84,6 +91,96 @@ pub(crate) const fn as_index(&self) -> u64 {
     }
 }
 
+impl MmuVersion {
+    /// Get the `PDE` levels (excluding PTE level) for page table walking.
+    pub(crate) fn pde_levels(&self) -> &'static [PageTableLevel] {
+        match self {
+            Self::V2 => ver2::PDE_LEVELS,
+            Self::V3 => ver3::PDE_LEVELS,
+        }
+    }
+
+    /// Get the PTE level for this MMU version.
+    pub(crate) fn pte_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::PTE_LEVEL,
+            Self::V3 => ver3::PTE_LEVEL,
+        }
+    }
+
+    /// Get the dual PDE level (128-bit entries) for this MMU version.
+    pub(crate) fn dual_pde_level(&self) -> PageTableLevel {
+        match self {
+            Self::V2 => ver2::DUAL_PDE_LEVEL,
+            Self::V3 => ver3::DUAL_PDE_LEVEL,
+        }
+    }
+
+    /// Get the number of PDE levels for this MMU version.
+    pub(crate) fn pde_level_count(&self) -> usize {
+        self.pde_levels().len()
+    }
+
+    /// Get the entry size in bytes for a given level.
+    pub(crate) fn entry_size(&self, level: PageTableLevel) -> usize {
+        if level == self.dual_pde_level() {
+            16 // 128-bit dual PDE
+        } else {
+            8 // 64-bit PDE/PTE
+        }
+    }
+
+    /// Get the number of entries per page table page for a given level.
+    pub(crate) fn entries_per_page(&self, level: PageTableLevel) -> usize {
+        match self {
+            Self::V2 => match level {
+                // TODO: Calculate these values from the bitfield dynamically
+                // instead of hardcoding them.
+                PageTableLevel::Pdb => 4, // PD3 root: bits [48:47] = 2 bits
+                PageTableLevel::L3 => 256, // PD0 dual: bits [28:21] = 8 bits
+                _ => 512,                 // PD2, PD1, PT: 9 bits each
+            },
+            Self::V3 => match level {
+                PageTableLevel::Pdb => 2,  // PDE4 root: bit [56] = 1 bit, 2 entries
+                PageTableLevel::L4 => 256, // PDE0 dual: bits [28:21] = 8 bits
+                _ => 512,                  // PDE3, PDE2, PDE1, PT: 9 bits each
+            },
+        }
+    }
+
+    /// Extract the page table index at `level` from `va` for this MMU version.
+    pub(crate) fn level_index(&self, va: VirtualAddress, level: u64) -> u64 {
+        match self {
+            Self::V2 => ver2::VirtualAddressV2::new(va).level_index(level),
+            Self::V3 => ver3::VirtualAddressV3::new(va).level_index(level),
+        }
+    }
+
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
Quoted 8 lines

drivers/gpu/nova-core/mm/pagetable.rs pub(crate) const fn as_index(&self) -> u64 { · line 102

Quoted 3 lines
drivers/gpu/nova-core/mm/pagetable.rs +3 -0
+    }
+}
+

We have a lot of matches on the MMU version here (and below in Pte, Pde, DualPde). What about making MmuVersion into a trait (e.g. Mmu) with associated types for Pte, Pde, DualPde which can implement traits defining their common operations too?

I coded this up and it did not look pretty, there's not much LOC savings and the code becomes harder to read because of parametrization of several functions. Also:

Quoted 1 line

Then you can parameterise Vmm/PtWalk on this type.

The match still to be done somewhere, so you end up matching on chipset to call the correct parametrized functions versus just passing in the parameter or chipset down, in some cases.

For now I am inclined to leave it as is. Also there's a Rust pitfall we all learnt during the turing and other patch reviews, sometimes doing a bunch of matches is good especially if the number of variants are expected to be fixed (in the mm case, version 2 and version 3). Traits have some disadvantages too, example dyn traits have to heap-allocated, parametrizing can increase code size (due to monomorphization) etc.

thanks,

--
Joel Fernandes

#72 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 31 lines · Joel Fernandes
Quoted 22 lines
Quoted 22 lines
drivers/gpu/nova-core/mm/pagetable.rs +22 -0
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
Quoted 8 lines

drivers/gpu/nova-core/mm/pagetable.rs pub(crate) const fn as_index(&self) -> u64 { · line 102

Quoted 3 lines
drivers/gpu/nova-core/mm/pagetable.rs +3 -0
+    }
+}
+

We have a lot of matches on the MMU version here (and below in Pte, Pde, DualPde). What about making MmuVersion into a trait (e.g. Mmu) with associated types for Pte, Pde, DualPde which can implement traits defining their common operations too?

Quoted 2 lines

I coded this up and it did not look pretty, there's not much LOC savings and the code becomes harder to read because of parametrization of several functions. Also:

Thanks for looking into it. Sorry to be a bother, but would you have a branch around with the code? I'm curious what didn't look good about it.

Quoted 6 lines
Quoted 1 line

Then you can parameterise Vmm/PtWalk on this type.

The match still to be done somewhere, so you end up matching on chipset to call the correct parametrized functions versus just passing in the parameter or chipset down, in some cases.

Quoted 6 lines

For now I am inclined to leave it as is. Also there's a Rust pitfall we all learnt during the turing and other patch reviews, sometimes doing a bunch of matches is good especially if the number of variants are expected to be fixed (in the mm case, version 2 and version 3). Traits have some disadvantages too, example dyn traits have to heap-allocated, parametrizing can increase code size (due to monomorphization) etc.

Yeah, it's just this is a lot of matches in a lot of places. And we have ver2 / ver3 specific code leaking into the general pagetable.rs file. So it would be really nice if we could find a way to improve this specific aspect. We can reduce the match to happening in just one file. You can avoid heap allocation if you would like by making Vmm an enum, for example, and doing the match based dispatch there at the top of the API tree, rather than at the bottom where it fans out into a lot more locations.

Quoted 5 lines

thanks,

--
Joel Fernandes

#73 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Hi Eliot,

Quoted 32 lines · Eliot Courtney
Quoted 31 lines · Joel Fernandes
Quoted 30 lines
Quoted 15 lines
drivers/gpu/nova-core/mm/pagetable.rs +15 -0
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+

drivers/gpu/nova-core/mm/pagetable.rs pub(crate) const fn as_index(&self) -> u64 { · line 183

Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable.rs +10 -0
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
+    }
+}
+

We have a lot of matches on the MMU version here (and below in Pte, Pde, DualPde). What about making MmuVersion into a trait (e.g. Mmu) with associated types for Pte, Pde, DualPde which can implement traits defining their common operations too?

Quoted 5 lines
Quoted 2 lines

I coded this up and it did not look pretty, there's not much LOC savings and the code becomes harder to read because of parametrization of several functions. Also:

Thanks for looking into it. Sorry to be a bother, but would you have a branch around with the code? I'm curious what didn't look good about it.

Sorry but I already mentioned that above, the parameterizing of dozens of function call sites, 3-4 new traits (because each struct like Pte/Pde/DualPde etc each need their own trait which different MMU versions implement) etc. The code because hard to read and readability is the top critical criteria for me - I am personally strictly against "Lets use shiny features in language at the cost of making code unreadable". Because that translates into bugs and nightmare for maintainability.

I don't have the code at the moment, but if you still want to spend on time on this direction, feel free to share a tree. I am happy to take a look.

Quoted 13 lines
Quoted 6 lines
Quoted 1 line

Then you can parameterise Vmm/PtWalk on this type.

The match still to be done somewhere, so you end up matching on chipset to call the correct parametrized functions versus just passing in the parameter or chipset down, in some cases.

Quoted 6 lines

For now I am inclined to leave it as is. Also there's a Rust pitfall we all learnt during the turing and other patch reviews, sometimes doing a bunch of matches is good especially if the number of variants are expected to be fixed (in the mm case, version 2 and version 3). Traits have some disadvantages too, example dyn traits have to heap-allocated, parametrizing can increase code size (due to monomorphization) etc.

Quoted 2 lines

Yeah, it's just this is a lot of matches in a lot of places. And we have ver2 / ver3 specific code leaking into the general pagetable.rs file. So

That's not a leak, that's by design. pagetable.rs is where the matches are centralized, most of the code changes here on out should happen outside of this file.

31 out of 42 matches in the mm code are in pagetable.rs, so it is already centralized.

Quoted 2 lines

it would be really nice if we could find a way to improve this specific aspect. We can reduce the match to happening in just one file.

Assuming we know what we're improving. ;-)

Quoted 4 lines

You can> avoid heap allocation if you would like by making Vmm an enum, for example, and doing the match based dispatch there at the top of the API tree, rather than at the bottom where it fans out into a lot more locations.

heap allocation is not always free, this code sensitive to dynamic allocations in the kernel, due to MM reclaim and locking. I would like to keep it simple.

thanks,

--
Joel Fernandes

#74 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 41 lines · Joel Fernandes

Hi Eliot,

Quoted 32 lines · Eliot Courtney
Quoted 31 lines · Joel Fernandes
Quoted 30 lines
Quoted 15 lines
drivers/gpu/nova-core/mm/pagetable.rs +15 -0
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+

drivers/gpu/nova-core/mm/pagetable.rs pub(crate) const fn as_index(&self) -> u64 { · line 183

Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable.rs +10 -0
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
+    }
+}
+

We have a lot of matches on the MMU version here (and below in Pte, Pde, DualPde). What about making MmuVersion into a trait (e.g. Mmu) with associated types for Pte, Pde, DualPde which can implement traits defining their common operations too?

Quoted 5 lines
Quoted 2 lines

I coded this up and it did not look pretty, there's not much LOC savings and the code becomes harder to read because of parametrization of several functions. Also:

Thanks for looking into it. Sorry to be a bother, but would you have a branch around with the code? I'm curious what didn't look good about it.

Quoted 7 lines

Sorry but I already mentioned that above, the parameterizing of dozens of function call sites, 3-4 new traits (because each struct like Pte/Pde/DualPde etc each need their own trait which different MMU versions implement) etc. The code because hard to read and readability is the top critical criteria for me - I am personally strictly against "Lets use shiny features in language at the cost of making code unreadable". Because that translates into bugs and nightmare for maintainability.

After a quick look I'd say that having a trait here would actually be *good* for correctness and maintainability.

The current design implies that every operation on a page table (most likely using the walker) goes through a branching point. Just looking at PtWalk::read_pte_at_level, there are already at least 6 if version == 2 { } else { } branches that all resolve to the same result. Include walking down the PDEs and you have at least a dozen of these just to resolve a virtual address. I know CPUs are fast, but this is still wasted cycles for no good reason.

If you use a trait here, and make PtWalk generic against it, you can optimize this away. We had a similar situation when we introduced Turing support and the v2 ucode header, and tried both approaches: the trait-based one was slightly shorter, and arguably more readable.

But the main argument to use a trait here IMO is that it enables associated types and constants. That's particularly critical since some equivalent fields have different lengths between v2 and v3. An associated Bounded type for these would force the caller to validate the length of these fields before calling a non-fallible operation, which is exactly the level of caution that we want when dealing with page tables.

In order to fully benefit from it, we will need the bitfield macro from the kernel crate so the PDE/PTE fields can be Bounded, I will try to make it available quickly in a patch that you can depend on.

But long story short, and although I need to dive deeper into the code, this looks like a good candidate for using a trait and associated types.

#75 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Hi Alex, Eliot, Danilo,

Thanks for taking a look. Let me respond to the specific points below.

Quoted 10 lines · Alexandre Courbot

After a quick look I'd say that having a trait here would actually be *good* for correctness and maintainability.

The current design implies that every operation on a page table (most likely using the walker) goes through a branching point. Just looking at PtWalk::read_pte_at_level, there are already at least 6 if version == 2 { } else { } branches that all resolve to the same result. Include walking down the PDEs and you have at least a dozen of these just to resolve a virtual address. I know CPUs are fast, but this is still wasted cycles for no good reason.

I did some measurements and there is no notieceable difference in both approaches. I ran perf and loaded nova with self-tests running. The extra potential branching is lost in the noise. In both cases, loading nova and running the self-tests has ~119.7M branch instructions on my Ampere. The total instruction count is also identical (~615M).

I measured like this:
perf stat -e branches,branch-misses,cache-references,cache-misses,instructions,cycles -- modprobe nova_core

So I think the branching argument is not a strong one. I also did more measurements and the dominant time taken is MMIO. During the map prep and execute, page table walks are done. A TLB flush alone costs ~1.4 microseconds. And PRAMIN BAR0 writes to write the PTE is also about 1 microsecond. Considering this, I don't think the extra branching argument holds (even without branch prediction and speculation).

Also some branches cannot be eliminated even with parameterization:

    if level == self.mmu_version.dual_pde_level() {
        // 128-bit dual PDE read
    } else {
        // Regular 64-bit PDE read
    }

This isn't really a version branch -- it's a structural branch that distinguishes between 64-bit PDE and 128-bit dual PDE entries. Any MMU version with a dual PDE level would need this same distinction.

I also did code-generation size analysis (see diff of code used below):

Code generation analysis:

  Module .ko size:   Before: 511,792 bytes   After: 524,464 bytes  (+2.5%)
  .text section:     Before: 112,620 bytes   After: 116,628 bytes  (+4,008 bytes)

The +4K .text growth is the monomorphization cost: every generic function is compiled twice (once for MmuV2, once for MmuV3).

Quoted 4 lines

If you use a trait here, and make PtWalk generic against it, you can optimize this away. We had a similar situation when we introduced Turing support and the v2 ucode header, and tried both approaches: the trait-based one was slightly shorter, and arguably more readable.

Actually I was the one who suggested traits for Falcon ucode descriptor if you see this thread [1]. So basically you and Eliot are telling me to do what I suggested in [1]. :-) However, I disagree that it is the right choice for this code.

[1] https://lore.kernel.org/all/20251117231028.GA1095236@joelbox2/

I think the two cases are quite different in complexity:

The falcon ucode descriptor is essentially a set of flat field accessors and a few params (imem_sec_load_params, dmem_load_params).
The trait has ~10 simple getter methods. There's no multi-level hierarchy, no walker, and no generic propagation.

The MMU page table case is structurally different. Making PtWalk generic over an Mmu trait would require:

- PtWalk<M: Mmu> (the walker)
- Plus all the associated types: M::Pte, M::Pde, M::DualPde each needing their own trait bounds

And we would also need:
- Vmm<M: Mmu> (which creates PtWalk)
- BarUser<M: Mmu> (which creates Vmm)

I am also against making Vmm an enum as Eliot suggested:

       enum Vmm {
           V2(VmmInner<MmuV2>),
           V3(VmmInner<MmuV3>),
       }

That moves the version complexity up to the reader. Code complexity IMO should decrease as we go up abstractions, making it easier for users (Vmm/Bar).

If you look at the the changes in vmm.rs to handle version dispatch there [2]:
Added: +109
Removed: -28

[2]
https://github.com/Edgeworth/linux/commit/3627af550b61256184d589e7ec666c1108971f0e

The main benefit of my approach is version-specific dispatch complexity is completely isolated inside MmuVersion thus making the code outside of pagetable.rs much more readable, without having to parametrize anything, and without code size increase. I think that is worth considering.

Quoted 7 lines

But the main argument to use a trait here IMO is that it enables associated types and constants. That's particularly critical since some equivalent fields have different lengths between v2 and v3. An associated Bounded type for these would force the caller to validate the length of these fields before calling a non-fallible operation, which is exactly the level of caution that we want when dealing with page tables.

I think Bounded validation is orthogonal to the dispatch model. We can add Bounded to the current design without restructuring into traits. For example:

    // In ver2::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 25>, writable: bool) -> Self { ... }

    // In ver3::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 40>, writable: bool) -> Self { ... }

The unified Pte enum wrapper already dispatches to the correct version-specific constructor, which would enforce the correct Bounded constraint for that version.

Quoted 3 lines

In order to fully benefit from it, we will need the bitfield macro from the kernel crate so the PDE/PTE fields can be Bounded, I will try to make it available quickly in a patch that you can depend on.

That would be great, and I'd be happy to integrate Bounded validation once the macro is available. I just don't think we need to restructure the dispatch model in order to benefit from it.

Quoted 2 lines

But long story short, and although I need to dive deeper into the code, this looks like a good candidate for using a trait and associated types.

The walker code (walk.rs) is already version-agnostic and reads cleanly. The version dispatch is encapsulated behind method calls, not exposed as inline if/else blocks.

Generic propagation (or version-specific dispatch at higher levels) adds more complexity at higher layers.

Enclosed below [3] is the diff I used for my testing with the data, I don't really see a net readability win there (IMO, it is a net-loss in readability).

[3]
https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git/commit/?h=trait-pt-dispatch&id=5eb0e98af11ba608ff4d0f7a06065ee863f5066a

thanks,

--
Joel Fernandes

#76 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 16 lines · Joel Fernandes

Hi Alex, Eliot, Danilo,

Thanks for taking a look. Let me respond to the specific points below.

Quoted 10 lines · Alexandre Courbot

After a quick look I'd say that having a trait here would actually be *good* for correctness and maintainability.

The current design implies that every operation on a page table (most likely using the walker) goes through a branching point. Just looking at PtWalk::read_pte_at_level, there are already at least 6 if version == 2 { } else { } branches that all resolve to the same result. Include walking down the PDEs and you have at least a dozen of these just to resolve a virtual address. I know CPUs are fast, but this is still wasted cycles for no good reason.

Quoted 5 lines

I did some measurements and there is no notieceable difference in both approaches. I ran perf and loaded nova with self-tests running. The extra potential branching is lost in the noise. In both cases, loading nova and running the self-tests has ~119.7M branch instructions on my Ampere. The total instruction count is also identical (~615M).

That's expected - as I said, CPUs are fast - and that's also not my point. My issue is that we are doing countless tests that all resolve to the code path, a code path that is already known during probe time. That's a huge code smell.

When we create the GPU, we know whether we will be using v2 or v3 page tables. That we need to test that again 12 times per address resolution is a design issue, irrespective of performance. There are 24 version match sites in patch 12 alone.

And that's precisely a good justification for using monomorphization. v2 and v3 are technically two different page table implementations (they even have their own distinct module in your series), we just use generics to factorize the (source) code a bit.

Quoted 21 lines

I measured like this:
perf stat -e branches,branch-misses,cache-references,cache-misses,instructions,cycles -- modprobe nova_core

So I think the branching argument is not a strong one. I also did more measurements and the dominant time taken is MMIO. During the map prep and execute, page table walks are done. A TLB flush alone costs ~1.4 microseconds. And PRAMIN BAR0 writes to write the PTE is also about 1 microsecond. Considering this, I don't think the extra branching argument holds (even without branch prediction and speculation).

Also some branches cannot be eliminated even with parameterization:

    if level == self.mmu_version.dual_pde_level() {
        // 128-bit dual PDE read
    } else {
        // Regular 64-bit PDE read
    }
Quoted 3 lines

This isn't really a version branch -- it's a structural branch that distinguishes between 64-bit PDE and 128-bit dual PDE entries. Any MMU version with a dual PDE level would need this same distinction.

The dual PDE level should be an associated constant - you still need to do the test, but note that you would also do it if there was only a single page table version. It's orthogonal to whether we use a trait or not here.

Quoted 10 lines

I also did code-generation size analysis (see diff of code used below):

Code generation analysis:

  Module .ko size:   Before: 511,792 bytes   After: 524,464 bytes  (+2.5%)
  .text section:     Before: 112,620 bytes   After: 116,628 bytes  (+4,008 bytes)

The +4K .text growth is the monomorphization cost: every generic function is compiled twice (once for MmuV2, once for MmuV3).

I would say this is working as intended then.

Quoted 6 lines
Quoted 5 lines

If you use a trait here, and make PtWalk generic against it, you can optimize this away. We had a similar situation when we introduced Turing support and the v2 ucode header, and tried both approaches: the trait-based one was slightly shorter, and arguably more readable.

Quoted 7 lines

Actually I was the one who suggested traits for Falcon ucode descriptor if you see this thread [1]. So basically you and Eliot are telling me to do what I suggested in [1]. :-) However, I disagree that it is the right choice for this code.

[1] https://lore.kernel.org/all/20251117231028.GA1095236@joelbox2/

I think the two cases are quite different in complexity:

Exactly. The complexity is different (this one involves multiple traits and associated types) but the pattern is the same - and that's a pattern traits are designed to address. If we were supposed to stop applying it when things go beyond a certain level of complexity, the conceptors of Rust would not have bothered addings things like associated types.

These traits are nothing new, they simply formalize a reality that already exists in your code, which is that each version of the page table needs to implement a given set of methods. It's already there with the version doing dispatches, only it is not articulated clearly to the reader. So in that respect, having traits make the code *more* readable imho.

Quoted 56 lines

The falcon ucode descriptor is essentially a set of flat field accessors and a few params (imem_sec_load_params, dmem_load_params).
The trait has ~10 simple getter methods. There's no multi-level hierarchy, no walker, and no generic propagation.

The MMU page table case is structurally different. Making PtWalk generic over an Mmu trait would require:

- PtWalk<M: Mmu> (the walker)
- Plus all the associated types: M::Pte, M::Pde, M::DualPde each needing their own trait bounds

And we would also need:
- Vmm<M: Mmu> (which creates PtWalk)
- BarUser<M: Mmu> (which creates Vmm)

I am also against making Vmm an enum as Eliot suggested:

       enum Vmm {
           V2(VmmInner<MmuV2>),
           V3(VmmInner<MmuV3>),
       }

That moves the version complexity up to the reader. Code complexity IMO should decrease as we go up abstractions, making it easier for users (Vmm/Bar).

If you look at the the changes in vmm.rs to handle version dispatch there [2]:
Added: +109
Removed: -28

[2]
https://github.com/Edgeworth/linux/commit/3627af550b61256184d589e7ec666c1108971f0e

The main benefit of my approach is version-specific dispatch complexity is completely isolated inside MmuVersion thus making the code outside of pagetable.rs much more readable, without having to parametrize anything, and without code size increase. I think that is worth considering.

Quoted 7 lines

But the main argument to use a trait here IMO is that it enables associated types and constants. That's particularly critical since some equivalent fields have different lengths between v2 and v3. An associated Bounded type for these would force the caller to validate the length of these fields before calling a non-fallible operation, which is exactly the level of caution that we want when dealing with page tables.

I think Bounded validation is orthogonal to the dispatch model. We can add Bounded to the current design without restructuring into traits. For example:

    // In ver2::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 25>, writable: bool) -> Self { ... }

    // In ver3::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 40>, writable: bool) -> Self { ... }
Quoted 3 lines

The unified Pte enum wrapper already dispatches to the correct version-specific constructor, which would enforce the correct Bounded constraint for that version.

But then what type does the new_vram dispatch method take? Generic code lets us expose the expected Bounded type to the caller, which can do the proper validation. This is a small example, but I expect this pattern to come up in other parts of the code as well.

Quoted 8 lines
Quoted 4 lines

In order to fully benefit from it, we will need the bitfield macro from the kernel crate so the PDE/PTE fields can be Bounded, I will try to make it available quickly in a patch that you can depend on.

That would be great, and I'd be happy to integrate Bounded validation once the macro is available. I just don't think we need to restructure the dispatch model in order to benefit from it.

I'll finish the series and hopefully send it a bit later today. That's another significant rework for the series (sorry about that) but it should be worth the effort for the added correctness.

#77 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Hi Alex,

Quoted 33 lines · Alexandre Courbot
Quoted 16 lines · Joel Fernandes

Hi Alex, Eliot, Danilo,

Thanks for taking a look. Let me respond to the specific points below.

Quoted 10 lines · Alexandre Courbot

After a quick look I'd say that having a trait here would actually be *good* for correctness and maintainability.

The current design implies that every operation on a page table (most likely using the walker) goes through a branching point. Just looking at PtWalk::read_pte_at_level, there are already at least 6 if version == 2 { } else { } branches that all resolve to the same result. Include walking down the PDEs and you have at least a dozen of these just to resolve a virtual address. I know CPUs are fast, but this is still wasted cycles for no good reason.

Quoted 5 lines

I did some measurements and there is no notieceable difference in both approaches. I ran perf and loaded nova with self-tests running. The extra potential branching is lost in the noise. In both cases, loading nova and running the self-tests has ~119.7M branch instructions on my Ampere. The total instruction count is also identical (~615M).

That's expected - as I said, CPUs are fast - and that's also not my point. My issue is that we are doing countless tests that all resolve to the code path, a code path that is already known during probe time. That's a huge code smell.

When we create the GPU, we know whether we will be using v2 or v3 page tables. That we need to test that again 12 times per address resolution is a design issue, irrespective of performance. There are 24 version match sites in patch 12 alone.

Quoted 4 lines

And that's precisely a good justification for using monomorphization. v2 and v3 are technically two different page table implementations (they even have their own distinct module in your series), we just use generics to factorize the (source) code a bit.

To repeat my point though, the extra tests don't complicate the code or change performance. The bottleneck is here is MMIO, not CPU cycles (and arguably testing shows no change in CPU cycles either). So I tend to prefer cleaner code (which in my view is the few matches in pagetable.rs versus templatizing lots of code). I do see your point of view though and I Ok with (trying out) the alternative as well, and it seems to be what mostly everyone wants. It is a little more boiler plate in higher layers which are not exposed to version matching, but not the end of the world :).

Quoted 20 lines
Quoted 20 lines

I measured like this:
perf stat -e branches,branch-misses,cache-references,cache-misses,instructions,cycles -- modprobe nova_core

So I think the branching argument is not a strong one. I also did more measurements and the dominant time taken is MMIO. During the map prep and execute, page table walks are done. A TLB flush alone costs ~1.4 microseconds. And PRAMIN BAR0 writes to write the PTE is also about 1 microsecond. Considering this, I don't think the extra branching argument holds (even without branch prediction and speculation).

Also some branches cannot be eliminated even with parameterization:

    if level == self.mmu_version.dual_pde_level() {
        // 128-bit dual PDE read
    } else {
        // Regular 64-bit PDE read
    }
Quoted 8 lines
Quoted 3 lines

This isn't really a version branch -- it's a structural branch that distinguishes between 64-bit PDE and 128-bit dual PDE entries. Any MMU version with a dual PDE level would need this same distinction.

The dual PDE level should be an associated constant - you still need to do the test, but note that you would also do it if there was only a single page table version. It's orthogonal to whether we use a trait or not here.

Sure, fair enough.

Quoted 6 lines
Quoted 6 lines

I also did code-generation size analysis (see diff of code used below):

Code generation analysis:

Quoted 7 lines
Quoted 5 lines
  Module .ko size:   Before: 511,792 bytes   After: 524,464 bytes  (+2.5%)
  .text section:     Before: 112,620 bytes   After: 116,628 bytes  (+4,008 bytes)

The +4K .text growth is the monomorphization cost: every generic function is compiled twice (once for MmuV2, once for MmuV3).

I would say this is working as intended then.

2.5% is not much for 1 feature but overtime IMO it can add up and become significant, I think we should keep an eye on size.

Quoted 19 lines
Quoted 5 lines
Quoted 4 lines

If you use a trait here, and make PtWalk generic against it, you can optimize this away. We had a similar situation when we introduced Turing support and the v2 ucode header, and tried both approaches: the trait-based one was slightly shorter, and arguably more readable.

Quoted 7 lines

Actually I was the one who suggested traits for Falcon ucode descriptor if you see this thread [1]. So basically you and Eliot are telling me to do what I suggested in [1]. :-) However, I disagree that it is the right choice for this code.

[1] https://lore.kernel.org/all/20251117231028.GA1095236@joelbox2/

I think the two cases are quite different in complexity:

Exactly. The complexity is different (this one involves multiple traits and associated types) but the pattern is the same - and that's a pattern traits are designed to address. If we were supposed to stop applying it when things go beyond a certain level of complexity, the conceptors of Rust would not have bothered addings things like associated types.

Quoted 6 lines

These traits are nothing new, they simply formalize a reality that already exists in your code, which is that each version of the page table needs to implement a given set of methods. It's already there with the version doing dispatches, only it is not articulated clearly to the reader. So in that respect, having traits make the code *more* readable imho.

Makes the lower layers more readable and the higher layers more complex. Me, I'd have preferred to keep the version matching stuff to the lower layers / centralized. Maybe there is still a way to do that, while still using traits.

Quoted 56 lines
Quoted 56 lines

The falcon ucode descriptor is essentially a set of flat field accessors and a few params (imem_sec_load_params, dmem_load_params).
The trait has ~10 simple getter methods. There's no multi-level hierarchy, no walker, and no generic propagation.

The MMU page table case is structurally different. Making PtWalk generic over an Mmu trait would require:

- PtWalk<M: Mmu> (the walker)
- Plus all the associated types: M::Pte, M::Pde, M::DualPde each needing their own trait bounds

And we would also need:
- Vmm<M: Mmu> (which creates PtWalk)
- BarUser<M: Mmu> (which creates Vmm)

I am also against making Vmm an enum as Eliot suggested:

       enum Vmm {
           V2(VmmInner<MmuV2>),
           V3(VmmInner<MmuV3>),
       }

That moves the version complexity up to the reader. Code complexity IMO should decrease as we go up abstractions, making it easier for users (Vmm/Bar).

If you look at the the changes in vmm.rs to handle version dispatch there [2]:
Added: +109
Removed: -28

[2]
https://github.com/Edgeworth/linux/commit/3627af550b61256184d589e7ec666c1108971f0e

The main benefit of my approach is version-specific dispatch complexity is completely isolated inside MmuVersion thus making the code outside of pagetable.rs much more readable, without having to parametrize anything, and without code size increase. I think that is worth considering.

Quoted 7 lines

But the main argument to use a trait here IMO is that it enables associated types and constants. That's particularly critical since some equivalent fields have different lengths between v2 and v3. An associated Bounded type for these would force the caller to validate the length of these fields before calling a non-fallible operation, which is exactly the level of caution that we want when dealing with page tables.

I think Bounded validation is orthogonal to the dispatch model. We can add Bounded to the current design without restructuring into traits. For example:

    // In ver2::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 25>, writable: bool) -> Self { ... }

    // In ver3::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 40>, writable: bool) -> Self { ... }
Quoted 8 lines
Quoted 3 lines

The unified Pte enum wrapper already dispatches to the correct version-specific constructor, which would enforce the correct Bounded constraint for that version.

But then what type does the new_vram dispatch method take? Generic code lets us expose the expected Bounded type to the caller, which can do the proper validation. This is a small example, but I expect this pattern to come up in other parts of the code as well.

Maybe I didn't follow your point, but the caller would do a version match and pass the correct bounded number of bits, but granted that's uglier if/once we switched to bounded types here.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

In order to fully benefit from it, we will need the bitfield macro from the kernel crate so the PDE/PTE fields can be Bounded, I will try to make it available quickly in a patch that you can depend on.

Quoted 7 lines
Quoted 3 lines

That would be great, and I'd be happy to integrate Bounded validation once the macro is available. I just don't think we need to restructure the dispatch model in order to benefit from it.

I'll finish the series and hopefully send it a bit later today. That's another significant rework for the series (sorry about that) but it should be worth the effort for the added correctness.

Sure!

thanks,

--
Joel Fernandes

reviewer
#78 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

In reply to Joel Fernandes
Quoted 143 lines · Joel Fernandes

Hi Alex, Eliot, Danilo,

Thanks for taking a look. Let me respond to the specific points below.

Quoted 10 lines · Alexandre Courbot

After a quick look I'd say that having a trait here would actually be *good* for correctness and maintainability.

The current design implies that every operation on a page table (most likely using the walker) goes through a branching point. Just looking at PtWalk::read_pte_at_level, there are already at least 6 if version == 2 { } else { } branches that all resolve to the same result. Include walking down the PDEs and you have at least a dozen of these just to resolve a virtual address. I know CPUs are fast, but this is still wasted cycles for no good reason.

I did some measurements and there is no notieceable difference in both approaches. I ran perf and loaded nova with self-tests running. The extra potential branching is lost in the noise. In both cases, loading nova and running the self-tests has ~119.7M branch instructions on my Ampere. The total instruction count is also identical (~615M).

I measured like this:
perf stat -e branches,branch-misses,cache-references,cache-misses,instructions,cycles -- modprobe nova_core

So I think the branching argument is not a strong one. I also did more measurements and the dominant time taken is MMIO. During the map prep and execute, page table walks are done. A TLB flush alone costs ~1.4 microseconds. And PRAMIN BAR0 writes to write the PTE is also about 1 microsecond. Considering this, I don't think the extra branching argument holds (even without branch prediction and speculation).

Also some branches cannot be eliminated even with parameterization:

    if level == self.mmu_version.dual_pde_level() {
        // 128-bit dual PDE read
    } else {
        // Regular 64-bit PDE read
    }

This isn't really a version branch -- it's a structural branch that distinguishes between 64-bit PDE and 128-bit dual PDE entries. Any MMU version with a dual PDE level would need this same distinction.

I also did code-generation size analysis (see diff of code used below):

Code generation analysis:

  Module .ko size:   Before: 511,792 bytes   After: 524,464 bytes  (+2.5%)
  .text section:     Before: 112,620 bytes   After: 116,628 bytes  (+4,008 bytes)

The +4K .text growth is the monomorphization cost: every generic function is compiled twice (once for MmuV2, once for MmuV3).

Quoted 4 lines

If you use a trait here, and make PtWalk generic against it, you can optimize this away. We had a similar situation when we introduced Turing support and the v2 ucode header, and tried both approaches: the trait-based one was slightly shorter, and arguably more readable.

Actually I was the one who suggested traits for Falcon ucode descriptor if you see this thread [1]. So basically you and Eliot are telling me to do what I suggested in [1]. :-) However, I disagree that it is the right choice for this code.

[1] https://lore.kernel.org/all/20251117231028.GA1095236@joelbox2/

I think the two cases are quite different in complexity:

The falcon ucode descriptor is essentially a set of flat field accessors and a few params (imem_sec_load_params, dmem_load_params).
The trait has ~10 simple getter methods. There's no multi-level hierarchy, no walker, and no generic propagation.

The MMU page table case is structurally different. Making PtWalk generic over an Mmu trait would require:

- PtWalk<M: Mmu> (the walker)
- Plus all the associated types: M::Pte, M::Pde, M::DualPde each needing their own trait bounds

And we would also need:
- Vmm<M: Mmu> (which creates PtWalk)
- BarUser<M: Mmu> (which creates Vmm)

I am also against making Vmm an enum as Eliot suggested:

       enum Vmm {
           V2(VmmInner<MmuV2>),
           V3(VmmInner<MmuV3>),
       }

That moves the version complexity up to the reader. Code complexity IMO should decrease as we go up abstractions, making it easier for users (Vmm/Bar).

If you look at the the changes in vmm.rs to handle version dispatch there [2]:
Added: +109
Removed: -28

[2]
https://github.com/Edgeworth/linux/commit/3627af550b61256184d589e7ec666c1108971f0e

The main benefit of my approach is version-specific dispatch complexity is completely isolated inside MmuVersion thus making the code outside of pagetable.rs much more readable, without having to parametrize anything, and without code size increase. I think that is worth considering.

Quoted 7 lines

But the main argument to use a trait here IMO is that it enables associated types and constants. That's particularly critical since some equivalent fields have different lengths between v2 and v3. An associated Bounded type for these would force the caller to validate the length of these fields before calling a non-fallible operation, which is exactly the level of caution that we want when dealing with page tables.

I think Bounded validation is orthogonal to the dispatch model. We can add Bounded to the current design without restructuring into traits. For example:

    // In ver2::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 25>, writable: bool) -> Self { ... }

    // In ver3::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 40>, writable: bool) -> Self { ... }

The unified Pte enum wrapper already dispatches to the correct version-specific constructor, which would enforce the correct Bounded constraint for that version.

Quoted 3 lines

In order to fully benefit from it, we will need the bitfield macro from the kernel crate so the PDE/PTE fields can be Bounded, I will try to make it available quickly in a patch that you can depend on.

That would be great, and I'd be happy to integrate Bounded validation once the macro is available. I just don't think we need to restructure the dispatch model in order to benefit from it.

Quoted 2 lines

But long story short, and although I need to dive deeper into the code, this looks like a good candidate for using a trait and associated types.

The walker code (walk.rs) is already version-agnostic and reads cleanly. The version dispatch is encapsulated behind method calls, not exposed as inline if/else blocks.

Quoted 8 lines

Generic propagation (or version-specific dispatch at higher levels) adds more complexity at higher layers.

Enclosed below [3] is the diff I used for my testing with the data, I don't really see a net readability win there (IMO, it is a net-loss in readability).

[3]
https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git/commit/?h=trait-pt-dispatch&id=5eb0e98af11ba608ff4d0f7a06065ee863f5066a

IMO this diff is quite has got me quite in favour of trait approach.

I wanted about to purpose something similar (or maybe I had already?) trait approach some versions ago but didn't due to the eventual need of match like dispatch (like you had with vmm_dispatch), but your code made that looks not as bad as I thought it would be.

Best,
Gary

#79 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 153 lines · Gary Guo
Quoted 143 lines · Joel Fernandes

Hi Alex, Eliot, Danilo,

Thanks for taking a look. Let me respond to the specific points below.

Quoted 10 lines · Alexandre Courbot

After a quick look I'd say that having a trait here would actually be *good* for correctness and maintainability.

The current design implies that every operation on a page table (most likely using the walker) goes through a branching point. Just looking at PtWalk::read_pte_at_level, there are already at least 6 if version == 2 { } else { } branches that all resolve to the same result. Include walking down the PDEs and you have at least a dozen of these just to resolve a virtual address. I know CPUs are fast, but this is still wasted cycles for no good reason.

I did some measurements and there is no notieceable difference in both approaches. I ran perf and loaded nova with self-tests running. The extra potential branching is lost in the noise. In both cases, loading nova and running the self-tests has ~119.7M branch instructions on my Ampere. The total instruction count is also identical (~615M).

I measured like this:
perf stat -e branches,branch-misses,cache-references,cache-misses,instructions,cycles -- modprobe nova_core

So I think the branching argument is not a strong one. I also did more measurements and the dominant time taken is MMIO. During the map prep and execute, page table walks are done. A TLB flush alone costs ~1.4 microseconds. And PRAMIN BAR0 writes to write the PTE is also about 1 microsecond. Considering this, I don't think the extra branching argument holds (even without branch prediction and speculation).

Also some branches cannot be eliminated even with parameterization:

    if level == self.mmu_version.dual_pde_level() {
        // 128-bit dual PDE read
    } else {
        // Regular 64-bit PDE read
    }

This isn't really a version branch -- it's a structural branch that distinguishes between 64-bit PDE and 128-bit dual PDE entries. Any MMU version with a dual PDE level would need this same distinction.

I also did code-generation size analysis (see diff of code used below):

Code generation analysis:

  Module .ko size:   Before: 511,792 bytes   After: 524,464 bytes  (+2.5%)
  .text section:     Before: 112,620 bytes   After: 116,628 bytes  (+4,008 bytes)

The +4K .text growth is the monomorphization cost: every generic function is compiled twice (once for MmuV2, once for MmuV3).

Quoted 4 lines

If you use a trait here, and make PtWalk generic against it, you can optimize this away. We had a similar situation when we introduced Turing support and the v2 ucode header, and tried both approaches: the trait-based one was slightly shorter, and arguably more readable.

Actually I was the one who suggested traits for Falcon ucode descriptor if you see this thread [1]. So basically you and Eliot are telling me to do what I suggested in [1]. :-) However, I disagree that it is the right choice for this code.

[1] https://lore.kernel.org/all/20251117231028.GA1095236@joelbox2/

I think the two cases are quite different in complexity:

The falcon ucode descriptor is essentially a set of flat field accessors and a few params (imem_sec_load_params, dmem_load_params).
The trait has ~10 simple getter methods. There's no multi-level hierarchy, no walker, and no generic propagation.

The MMU page table case is structurally different. Making PtWalk generic over an Mmu trait would require:

- PtWalk<M: Mmu> (the walker)
- Plus all the associated types: M::Pte, M::Pde, M::DualPde each needing their own trait bounds

And we would also need:
- Vmm<M: Mmu> (which creates PtWalk)
- BarUser<M: Mmu> (which creates Vmm)

I am also against making Vmm an enum as Eliot suggested:

       enum Vmm {
           V2(VmmInner<MmuV2>),
           V3(VmmInner<MmuV3>),
       }

That moves the version complexity up to the reader. Code complexity IMO should decrease as we go up abstractions, making it easier for users (Vmm/Bar).

If you look at the the changes in vmm.rs to handle version dispatch there [2]:
Added: +109
Removed: -28

[2]
https://github.com/Edgeworth/linux/commit/3627af550b61256184d589e7ec666c1108971f0e

The main benefit of my approach is version-specific dispatch complexity is completely isolated inside MmuVersion thus making the code outside of pagetable.rs much more readable, without having to parametrize anything, and without code size increase. I think that is worth considering.

Quoted 7 lines

But the main argument to use a trait here IMO is that it enables associated types and constants. That's particularly critical since some equivalent fields have different lengths between v2 and v3. An associated Bounded type for these would force the caller to validate the length of these fields before calling a non-fallible operation, which is exactly the level of caution that we want when dealing with page tables.

I think Bounded validation is orthogonal to the dispatch model. We can add Bounded to the current design without restructuring into traits. For example:

    // In ver2::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 25>, writable: bool) -> Self { ... }

    // In ver3::Pte
    pub fn new_vram(pfn: Bounded<Pfn, 40>, writable: bool) -> Self { ... }

The unified Pte enum wrapper already dispatches to the correct version-specific constructor, which would enforce the correct Bounded constraint for that version.

Quoted 3 lines

In order to fully benefit from it, we will need the bitfield macro from the kernel crate so the PDE/PTE fields can be Bounded, I will try to make it available quickly in a patch that you can depend on.

That would be great, and I'd be happy to integrate Bounded validation once the macro is available. I just don't think we need to restructure the dispatch model in order to benefit from it.

Quoted 2 lines

But long story short, and although I need to dive deeper into the code, this looks like a good candidate for using a trait and associated types.

The walker code (walk.rs) is already version-agnostic and reads cleanly. The version dispatch is encapsulated behind method calls, not exposed as inline if/else blocks.

Quoted 8 lines

Generic propagation (or version-specific dispatch at higher levels) adds more complexity at higher layers.

Enclosed below [3] is the diff I used for my testing with the data, I don't really see a net readability win there (IMO, it is a net-loss in readability).

[3]
https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git/commit/?h=trait-pt-dispatch&id=5eb0e98af11ba608ff4d0f7a06065ee863f5066a

Quoted 7 lines

IMO this diff is quite has got me quite in favour of trait approach.

I wanted about to purpose something similar (or maybe I had already?) trait approach some versions ago but didn't due to the eventual need of match like dispatch (like you had with vmm_dispatch), but your code made that looks not as bad as I thought it would be.

That's the drawback right, now vmm_dispatch has to deal with version difference where as before, the lower layers would. Maybe we can keep the vmm layer the way it is now, but do the dispatch itself at the lower layers, while still using traits like in the diff. I'll try that as well. :)

thanks,

--
Joel Fernandes

#80 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

In reply to Joel Fernandes
Quoted 49 lines · Joel Fernandes

Hi Eliot,

Quoted 32 lines · Eliot Courtney
Quoted 31 lines · Joel Fernandes
Quoted 30 lines
Quoted 15 lines
drivers/gpu/nova-core/mm/pagetable.rs +15 -0
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+

drivers/gpu/nova-core/mm/pagetable.rs pub(crate) const fn as_index(&self) -> u64 { · line 183

Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable.rs +10 -0
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
+    }
+}
+

We have a lot of matches on the MMU version here (and below in Pte, Pde, DualPde). What about making MmuVersion into a trait (e.g. Mmu) with associated types for Pte, Pde, DualPde which can implement traits defining their common operations too?

Quoted 5 lines
Quoted 2 lines

I coded this up and it did not look pretty, there's not much LOC savings and the code becomes harder to read because of parametrization of several functions. Also:

Thanks for looking into it. Sorry to be a bother, but would you have a branch around with the code? I'm curious what didn't look good about it.

Sorry but I already mentioned that above, the parameterizing of dozens of function call sites, 3-4 new traits (because each struct like Pte/Pde/DualPde etc each need their own trait which different MMU versions implement) etc. The code because hard to read and readability is the top critical criteria for me - I am personally strictly against "Lets use shiny features in language at the cost of making code unreadable". Because that translates into bugs and nightmare for maintainability.

Quoted 2 lines

I don't have the code at the moment, but if you still want to spend on time on this direction, feel free to share a tree. I am happy to take a look.

I had a go at this, you can see the branch here [1] - it might not be perfect, but I think the shape is directionally good. It's structured so the HEAD commit has the diff from the current approach to the parametrised approach. The main decision is where to do the type erasure, I chose in Vmm since it looks like the main top level API for this code, but could do BarUser instead. I think it's overall better. I also think Alex's point about associated types making it easier to use the appropriate Bounded type is a good one.

[1]: https://github.com/Edgeworth/linux/commits/review/nova-mm-v10/

Quoted 28 lines
Quoted 13 lines
Quoted 6 lines
Quoted 1 line

Then you can parameterise Vmm/PtWalk on this type.

The match still to be done somewhere, so you end up matching on chipset to call the correct parametrized functions versus just passing in the parameter or chipset down, in some cases.

Quoted 6 lines

For now I am inclined to leave it as is. Also there's a Rust pitfall we all learnt during the turing and other patch reviews, sometimes doing a bunch of matches is good especially if the number of variants are expected to be fixed (in the mm case, version 2 and version 3). Traits have some disadvantages too, example dyn traits have to heap-allocated, parametrizing can increase code size (due to monomorphization) etc.

Quoted 2 lines

Yeah, it's just this is a lot of matches in a lot of places. And we have ver2 / ver3 specific code leaking into the general pagetable.rs file. So

That's not a leak, that's by design. pagetable.rs is where the matches are centralized, most of the code changes here on out should happen outside of this file.

31 out of 42 matches in the mm code are in pagetable.rs, so it is already centralized.

Quoted 2 lines

it would be really nice if we could find a way to improve this specific aspect. We can reduce the match to happening in just one file.

Assuming we know what we're improving. ;-)

Quoted 8 lines
Quoted 4 lines

You can> avoid heap allocation if you would like by making Vmm an enum, for example, and doing the match based dispatch there at the top of the API tree, rather than at the bottom where it fans out into a lot more locations.

heap allocation is not always free, this code sensitive to dynamic allocations in the kernel, due to MM reclaim and locking. I would like to keep it simple.

If you do the type erasure via enum in Vmm, you won't need to allocate it on the heap. The branch I posted above has an example on how to do this, although there might be a better way.

Quoted 5 lines

thanks,

--
Joel Fernandes

#81 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Hi Eliot,

Quoted 62 lines · Eliot Courtney
Quoted 49 lines · Joel Fernandes

Hi Eliot,

Quoted 32 lines · Eliot Courtney
Quoted 31 lines · Joel Fernandes
Quoted 30 lines
Quoted 15 lines
drivers/gpu/nova-core/mm/pagetable.rs +15 -0
+    /// Compute upper bound on page table pages needed for `num_virt_pages`.
+    ///
+    /// Walks from PTE level up through PDE levels, accumulating the tree.
+    pub(crate) fn pt_pages_upper_bound(&self, num_virt_pages: usize) -> usize {
+        let mut total = 0;
+
+        // PTE pages at the leaf level.
+        let pte_epp = self.entries_per_page(self.pte_level());
+        let mut pages_at_level = num_virt_pages.div_ceil(pte_epp);
+        total += pages_at_level;
+
+        // Walk PDE levels bottom-up (reverse of pde_levels()).
+        for &level in self.pde_levels().iter().rev() {
+            let epp = self.entries_per_page(level);
+

drivers/gpu/nova-core/mm/pagetable.rs pub(crate) const fn as_index(&self) -> u64 { · line 183

Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable.rs +10 -0
+            // How many pages at this level do we need to point to
+            // the previous pages_at_level?
+            pages_at_level = pages_at_level.div_ceil(epp);
+            total += pages_at_level;
+        }
+
+        total
+    }
+}
+

We have a lot of matches on the MMU version here (and below in Pte, Pde, DualPde). What about making MmuVersion into a trait (e.g. Mmu) with associated types for Pte, Pde, DualPde which can implement traits defining their common operations too?

Quoted 5 lines
Quoted 2 lines

I coded this up and it did not look pretty, there's not much LOC savings and the code becomes harder to read because of parametrization of several functions. Also:

Thanks for looking into it. Sorry to be a bother, but would you have a branch around with the code? I'm curious what didn't look good about it.

Sorry but I already mentioned that above, the parameterizing of dozens of function call sites, 3-4 new traits (because each struct like Pte/Pde/DualPde etc each need their own trait which different MMU versions implement) etc. The code because hard to read and readability is the top critical criteria for me - I am personally strictly against "Lets use shiny features in language at the cost of making code unreadable". Because that translates into bugs and nightmare for maintainability.

Quoted 2 lines

I don't have the code at the moment, but if you still want to spend on time on this direction, feel free to share a tree. I am happy to take a look.

I had a go at this, you can see the branch here [1] - it might not be perfect, but I think the shape is directionally good. It's structured so the HEAD commit has the diff from the current approach to the parametrised approach. The main decision is where to do the type erasure, I chose in Vmm since it looks like the main top level API for this code, but could do BarUser instead. I think it's overall better. I also think Alex's point about associated types making it easier to use the appropriate Bounded type is a good one.

Quoted 1 line

[1]: https://github.com/Edgeworth/linux/commits/review/nova-mm-v10/

First, thanks for the effort. I looked through this, its pretty much what I had before when I used traits. I don't think it is better to be honest. In fact your version is worse, it adds many new types and things like the following which I did not need before.

To put it mildly, the following suggestion should not be anywhere near my code:

/// Type-erased MMU-specific [Vmm] implementations.

enum VmmInner {
    /// `Vmm` implementation for MMU v2.
    V2(VmmImpl<MmuV2>),
    /// `Vmm` implementation for MMU v3.
    V3(VmmImpl<MmuV3>),
}

/// MMU-specific [Vmm] implementation.
struct VmmImpl<M: Mmu> {

Seriously, I have to pass on this. :-)

And, you unfortunately seem to have ignored my point about requiring 4 NEW traits (Mmu, PteOps, PdeOps, DualPdeOps etc), which I did not need before. So you're making the code much much worse than before actually. We don't new traits and types pointlessly.

The only positive thing I could take away from your diff is the following (I thought I had already done that, but I'll double check).

- fn level_index(&self, level: u64) -> u64 {
+ fn level_index(&self, level: PageTableLevel) -> u64 {

Also you're parametrizing VirtualAddress as well which I did not have before:

- let va = VirtualAddress::from(vfn);
+ let va = M::va(VirtualAddress::from(vfn));

This is another step back.

Quoted 2 lines

I also think Alex's point about associated types making it easier to use the appropriate Bounded type is a good one.

I will reply to Alex thread, separately. I have some good data that should hopefully convince you and Alex that my approach in this patch is better (Version struct based dispatch than monomorphization). I would emphasize, as we all know, that we should make optimizations and changes based on real data and proper technical arguments so in the spirit of that, I have collected data with both approaches and I will reply to Alex's email with all that in there.

Also, the bounded types usage is orthogonal to version-parameterization. That can be done regardless, we already use bitfield macro in this code and can use bounded types within that if needed to restrict type creation. So I don't think we should mix the 2 concepts "bounded types" and "parameterization".

thanks,

--
Joel Fernandes

maintainer
#82 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 2 lines · Joel Fernandes

So you're making the code much much worse than before actually. We don't new traits and types pointlessly.

I had a look at both approaches and yes, the traits can be considered boilerplate. But, they are not complex and they just list method signatures that each version's types already implement functionality wise and they get us rid of a lot of dispatch sites. The implementation turns out cleaner as there is less parameter threading throughout call chains, etc. Overall, it seems more scalable.

On the other hand, there are indeed more abstractions and type indirections to understand in Eliot's code. I.e. there are advantages and disadvantages to both approaches.

That said, please engage with Eliot's proposal, it is not as off as your reply implies and dismissing it right away is not what I'd like to see in this situation.

#83 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 11 lines · Danilo Krummrich
Quoted 2 lines · Joel Fernandes

So you're making the code much much worse than before actually. We don't new traits and types pointlessly.

I had a look at both approaches and yes, the traits can be considered boilerplate. But, they are not complex and they just list method signatures that each version's types already implement functionality wise and they get us rid of a lot of dispatch sites. The implementation turns out cleaner as there is less parameter threading throughout call chains, etc. Overall, it seems more scalable.

Quoted 7 lines

On the other hand, there are indeed more abstractions and type indirections to understand in Eliot's code. I.e. there are advantages and disadvantages to both approaches.

That said, please engage with Eliot's proposal, it is not as off as your reply implies and dismissing it right away is not what I'd like to see in this situation.

No one is “dismissing it right away”. If you read all the threads, you will see that I already came up with the same approach and have spent time coding it up myself to learn its potential advantages and drawbacks. :-) I would never dismiss anyone’s suggestion just for the sake of it, without evaluating. Hence most of my morning went in collecting data with both approaches (more on that in another upcoming reply).

Also yes of course we have to look at all the pros and cons and carefully make a choice collectively, there are tradeoffs in both approaches.

thanks,

--
Joel Fernandes

#84 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

In reply to Joel Fernandes
Quoted 4 lines · Joel Fernandes
Quoted 3 lines · Eliot Courtney
Quoted 2 lines · Joel Fernandes
Quoted 1 line · Eliot Courtney

On Tue Apr 7, 2026 at 6:55 AM JST, Joel Fernandes wrote:

...>> [1]: https://github.com/Edgeworth/linux/commits/review/nova-mm-v10/

Quoted 4 lines

First, thanks for the effort. I looked through this, its pretty much what I had before when I used traits. I don't think it is better to be honest. In fact your version is worse, it adds many new types and things like the following which I did not need before.

Hi Joel and all,

I also looked through Eliot's above attempt carefully, and actually liked it a lot (sorry! haha):

* It cleans up the code. The initial working version was readable, but also had lots of noise on the screen: match statements and pairs of v2/v3 statements.

And interestingly, the mmu_version was, in effect, sporadically implementing a Trait-based approach. But because it is custom, readers don't benefit as much as they would with Traits, which tell you immediately how things are structured.

Joel, I am passionately in agreement with your principles: code must be readable on the screen.

In this case, though, Traits make considerably more readable, especially if one makes the very reasonable assumption that readers are thoroughly accustomed to dealing with Rust traits.

Quoted 3 lines

To put it mildly, the following suggestion should not be anywhere near my code:

lol I understand, believe me. But this is short and not too bad, really.

Quoted 1 line

/// Type-erased MMU-specific [Vmm] implementations.

Type erasure remains a semi-exotic thing, IMHO. As such, another sentence to elaborate on this would be a nice touch.

Quoted 10 lines
enum VmmInner {
    /// `Vmm` implementation for MMU v2.
    V2(VmmImpl<MmuV2>),
    /// `Vmm` implementation for MMU v3.
    V3(VmmImpl<MmuV3>),
}

/// MMU-specific [Vmm] implementation.
struct VmmImpl<M: Mmu> {

Quoted 6 lines

Seriously, I have to pass on this. :-)

And, you unfortunately seem to have ignored my point about requiring 4 NEW traits (Mmu, PteOps, PdeOps, DualPdeOps etc), which I did not need before. So you're making the code much much worse than before actually. We don't new traits and types pointlessly.

They are not pointless.

However! What I think would be nice is: do a new v11 with approximately this approach, and then we can beat it into being as readable as possible.

thanks,

Signature · 1 line
John Hubbard
#85 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 62 lines · John Hubbard
Quoted 4 lines · Joel Fernandes
Quoted 4 lines · Eliot Courtney
Quoted 2 lines · Joel Fernandes
Quoted 1 line · Eliot Courtney

On Tue Apr 7, 2026 at 6:55 AM JST, Joel Fernandes wrote:

...>> [1]: https://github.com/Edgeworth/linux/commits/review/nova-mm-v10/

Quoted 4 lines

First, thanks for the effort. I looked through this, its pretty much what I had before when I used traits. I don't think it is better to be honest. In fact your version is worse, it adds many new types and things like the following which I did not need before.

Hi Joel and all,

I also looked through Eliot's above attempt carefully, and actually liked it a lot (sorry! haha):

* It cleans up the code. The initial working version was readable, but also had lots of noise on the screen: match statements and pairs of v2/v3 statements.

And interestingly, the mmu_version was, in effect, sporadically implementing a Trait-based approach. But because it is custom, readers don't benefit as much as they would with Traits, which tell you immediately how things are structured.

Joel, I am passionately in agreement with your principles: code must be readable on the screen.

In this case, though, Traits make considerably more readable, especially if one makes the very reasonable assumption that readers are thoroughly accustomed to dealing with Rust traits.

Quoted 3 lines

To put it mildly, the following suggestion should not be anywhere near my code:

lol I understand, believe me. But this is short and not too bad, really.

Quoted 1 line

/// Type-erased MMU-specific [Vmm] implementations.

Type erasure remains a semi-exotic thing, IMHO. As such, another sentence to elaborate on this would be a nice touch.

Quoted 10 lines
enum VmmInner {
   /// `Vmm` implementation for MMU v2.
   V2(VmmImpl<MmuV2>),
   /// `Vmm` implementation for MMU v3.
   V3(VmmImpl<MmuV3>),
}

/// MMU-specific [Vmm] implementation.
struct VmmImpl<M: Mmu> {

Quoted 6 lines

Seriously, I have to pass on this. :-)

And, you unfortunately seem to have ignored my point about requiring 4 NEW traits (Mmu, PteOps, PdeOps, DualPdeOps etc), which I did not need before. So you're making the code much much worse than before actually. We don't new traits and types pointlessly.

Quoted 5 lines

They are not pointless.

However! What I think would be nice is: do a new v11 with approximately this approach, and then we can beat it into being as readable as possible.

Since it is 3 against 1 here, I rest my case :-). I am still in disagreement since I do not see much benefit (that is why I said pointless above). Actually it is not even about readability, that is subjective (and I haven’t heard most people say parametrizing code for the sake of it makes it more readable anyway). It is that the code gen is worse, and the complexity is just moved to a higher level in the code, not removed. So what are we getting out of this really, other than more boiler plate in higher layers of the code that did not exist before? Not performance, not better generated code. Really nothing. See all the data points in my previous reply.

Note that if the mmu version threading bothers everyone so much, we can also pass down chipset instead and let the walker deal with determining versioning. Would that be better?

But otherwise and since you guys asked, here comes a parameterized v11... ;-).
(Coming next week since this week I’m working on IRQ handling).

thanks,

Signature · 1 line
Joel Fernandes
maintainer
#86 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 1 line · Joel Fernandes

Since it is 3 against 1 here, I rest my case :-).

That's not how I'd view it. :)

Anyways, in case I'm included in "3", that's not my position. My point was to ensure we keep discussing advantages and disadvantages on their merits, as I think you both have good points.

Quoted 2 lines

I am still in disagreement since I do not see much benefit (that is why I said pointless above).

That is fair -- in this case please explain why the advantages pointed out by others are not worth it, propose something that picks up the best of both worlds, etc.

You can also turn it around and ask people whether they can tweak their counter proposal to get rid of specific parts you dislike for a reason.

IOW, keep the ball rolling, so we can come up with the best possible solution.

#87 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Hi Danilo,

Quoted 8 lines · Danilo Krummrich
Quoted 1 line · Joel Fernandes

Since it is 3 against 1 here, I rest my case :-).

That's not how I'd view it. :)

Anyways, in case I'm included in "3", that's not my position. My point was to ensure we keep discussing advantages and disadvantages on their merits, as I think you both have good points.

Heh, yes I actually *did not* include you in the 3 since you sounded to be open to both. ;-)

Quoted 4 lines
Quoted 3 lines

I am still in disagreement since I do not see much benefit (that is why I said pointless above).

Quoted 8 lines

That is fair -- in this case please explain why the advantages pointed out by others are not worth it, propose something that picks up the best of both worlds, etc.

You can also turn it around and ask people whether they can tweak their counter proposal to get rid of specific parts you dislike for a reason.

IOW, keep the ball rolling, so we can come up with the best possible solution.

Good advice, thanks! I will try to come up with something that is acceptable to everyone and we can further debate pros/cons on v11.

There are some merits on the alternative proposal from Eliot/Alex that I'd like to explore while seeing if I can keep some of the merits in mine as well.

thanks,

Signature · 1 line
Joel Fernandes
#88 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 26 lines · Joel Fernandes

Hi Danilo,

Quoted 8 lines · Danilo Krummrich
Quoted 1 line · Joel Fernandes

Since it is 3 against 1 here, I rest my case :-).

That's not how I'd view it. :)

Anyways, in case I'm included in "3", that's not my position. My point was to ensure we keep discussing advantages and disadvantages on their merits, as I think you both have good points.

Heh, yes I actually *did not* include you in the 3 since you sounded to be open to both. ;-)

Quoted 11 lines
Quoted 3 lines

I am still in disagreement since I do not see much benefit (that is why I said pointless above).

That is fair -- in this case please explain why the advantages pointed out by others are not worth it, propose something that picks up the best of both worlds, etc.

You can also turn it around and ask people whether they can tweak their counter proposal to get rid of specific parts you dislike for a reason.

Quoted 7 lines
Quoted 1 line

IOW, keep the ball rolling, so we can come up with the best possible solution.

Good advice, thanks! I will try to come up with something that is acceptable to everyone and we can further debate pros/cons on v11.

There are some merits on the alternative proposal from Eliot/Alex that I'd like to explore while seeing if I can keep some of the merits in mine as well.

I think I found a nice approach. IMO the MMU version dispatch does not belong in Vmm/BarUser layers. Those are version-independent code. However I agree that doing version dispatch at every low-level page table operation is a bit heavy on matches (if we put the MMIO overhead counter-argument aside).

So how about the following approach?

PtWalk, PtMap and everything below it are monomorphized. Vmm and BarUser are not. Version dispatch is handled on PtWalk and PtMap entry points.

I think it makes it cleaner and splits the code up better too and the organizations makes sense because the version differences are related to page tables, not to generic concepts like Vmm and Bar.

Thoughts? Here is a preview:
https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git/commit/?h=pt-traits-v2&id=ff22ba64f729f9f73258777231763a7b9804123b

thanks,

--
Joel Fernandes

#89 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

Quoted 41 lines · Joel Fernandes
Quoted 26 lines · Joel Fernandes

Hi Danilo,

Quoted 8 lines · Danilo Krummrich
Quoted 1 line · Joel Fernandes

Since it is 3 against 1 here, I rest my case :-).

That's not how I'd view it. :)

Anyways, in case I'm included in "3", that's not my position. My point was to ensure we keep discussing advantages and disadvantages on their merits, as I think you both have good points.

Heh, yes I actually *did not* include you in the 3 since you sounded to be open to both. ;-)

Quoted 11 lines
Quoted 3 lines

I am still in disagreement since I do not see much benefit (that is why I said pointless above).

That is fair -- in this case please explain why the advantages pointed out by others are not worth it, propose something that picks up the best of both worlds, etc.

You can also turn it around and ask people whether they can tweak their counter proposal to get rid of specific parts you dislike for a reason.

Quoted 7 lines
Quoted 1 line

IOW, keep the ball rolling, so we can come up with the best possible solution.

Good advice, thanks! I will try to come up with something that is acceptable to everyone and we can further debate pros/cons on v11.

There are some merits on the alternative proposal from Eliot/Alex that I'd like to explore while seeing if I can keep some of the merits in mine as well.

I think I found a nice approach. IMO the MMU version dispatch does not belong in Vmm/BarUser layers. Those are version-independent code. However I agree that doing version dispatch at every low-level page table operation is a bit heavy on matches (if we put the MMIO overhead counter-argument aside).

Quoted 4 lines

So how about the following approach?

PtWalk, PtMap and everything below it are monomorphized. Vmm and BarUser are not. Version dispatch is handled on PtWalk and PtMap entry points.

Conceptually it sounds pretty good.

Quoted 8 lines

I think it makes it cleaner and splits the code up better too and the organizations makes sense because the version differences are related to page tables, not to generic concepts like Vmm and Bar.

Thoughts? Here is a preview:
https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git/commit/?h=pt-traits-v2&id=ff22ba64f729f9f73258777231763a7b9804123b

Probably impractical to review that here, so let's do the review in a v11 posting, I think.

thanks,

Signature · 1 line
John Hubbard
#90 lore raw

Re: [PATCH v10 12/21] gpu: nova-core: mm: Add unified page table entry wrapper enums

In reply to Joel Fernandes

On 4/9/26 3:33 AM, Joel Fernandes wrote:
...

Quoted 1 line

Since it is 3 against 1 here, I rest my case :-). I am still in

As Danilo points out, we want to foster healthy debate and land on whatever comes out of that. Not just by counting noses. :)

Quoted 9 lines

disagreement since I do not see much benefit (that is why I said pointless above). Actually it is not even about readability, that is subjective (and I haven’t heard most people say parametrizing code for the sake of it makes it more readable anyway). It is that the code gen is worse, and the complexity is just moved to a higher level in the code, not removed. So what are we getting out of this really, other than more boiler plate in higher layers of the code that did not exist before? Not performance, not better generated code. Really nothing. See all the data points in my previous reply.

Alex's latest response[1] does a *much* better job than mine, in explicitly highlighting what we get out of this. (It arrived a bit after your response here.) Please take a very close look at that response and see what you think.

The patterns in these page tables are not a new thing, and Rust has language features to help express them.

[1] https://lore.kernel.org/[email protected]

thanks,

Signature · 1 line
John Hubbard
#91 lore raw

PATCH v10 13/21 gpu: nova-core: mm: Add page table walker for MMU v2/v3

In reply to Joel Fernandes

Add the page table walker implementation that traverses the page table hierarchy for both MMU v2 (5-level) and MMU v3 (6-level) to resolve virtual addresses to physical addresses or find PTE locations.

Currently only v2 has been tested (nova-core currently boots pre-hopper) with some initial preparatory work done for v3.

---
 drivers/gpu/nova-core/mm/pagetable.rs      |   1 +
 drivers/gpu/nova-core/mm/pagetable/walk.rs | 218 +++++++++++++++++++++
 2 files changed, 219 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
index 909df37c3ee8..9acf100798d3 100644
@@ -10,6 +10,7 @@
 
 pub(crate) mod ver2;
 pub(crate) mod ver3;
+pub(crate) mod walk;
 
 use crate::gpu::Architecture;
 use crate::mm::{
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +218 -0

218 lines not on this page · raw message

Signature · 1 line
2.34.1
#92 lore raw

PATCH v10 14/21 gpu: nova-core: mm: Add Virtual Memory Manager

In reply to Joel Fernandes

Add the Virtual Memory Manager (VMM) infrastructure for GPU address space management. Each Vmm instance manages a single address space identified by its Page Directory Base (PDB) address, used for Channel, BAR1 and BAR2 mappings.

Mapping APIs and virtual address range tracking are added in later commits.

---
 drivers/gpu/nova-core/mm.rs     |  1 +
 drivers/gpu/nova-core/mm/vmm.rs | 63 +++++++++++++++++++++++++++++++++
 2 files changed, 64 insertions(+)
 create mode 100644 drivers/gpu/nova-core/mm/vmm.rs
drivers/gpu/nova-core/mm.rs +1 -0
index 151b9add67d8..1594279dea20 100644
@@ -7,6 +7,7 @@
 pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
+pub(crate) mod vmm;
 
 use kernel::{
     devres::Devres,
drivers/gpu/nova-core/mm/vmm.rs new file +63 -0

63 lines not on this page · raw message

Signature · 1 line
2.34.1
#93 lore raw

PATCH v10 15/21 gpu: nova-core: mm: Add virtual address range tracking to VMM

In reply to Joel Fernandes

Add virtual address range tracking to the VMM using a buddy allocator. This enables contiguous virtual address range allocation for mappings.

---
 drivers/gpu/nova-core/mm/vmm.rs | 98 +++++++++++++++++++++++++++++----
 1 file changed, 87 insertions(+), 11 deletions(-)
drivers/gpu/nova-core/mm/vmm.rs +87 -11
index a22d4c506ea6..2a65ffd73b0d 100644
@@ -7,19 +7,35 @@
 //! and handles TLB flushing after modifications.
 
 use kernel::{
-    gpu::buddy::AllocatedBlocks,
-    prelude::*, //
+    gpu::buddy::{
+        AllocatedBlocks,
+        GpuBuddy,
+        GpuBuddyAllocFlag,
+        GpuBuddyAllocMode,
+        GpuBuddyParams, //
+    },
+    prelude::*,
+    ptr::Alignment,
+    sizes::SZ_4K, //
 };
 
-use crate::mm::{
-    pagetable::{
-        walk::{PtWalk, WalkResult},
-        MmuVersion, //
+use core::ops::Range;
+
+use crate::{
+    mm::{
+        pagetable::{
+            walk::{PtWalk, WalkResult},
+            MmuVersion, //
+        },
+        GpuMm,
+        Pfn,
+        Vfn,
+        VramAddress,
+        PAGE_SIZE, //
+    },
+    num::{
+        IntoSafeCast, //
     },
-    GpuMm,
-    Pfn,
-    Vfn,
-    VramAddress, //
 };
 
 /// Virtual Memory Manager for a GPU address space.
@@ -34,23 +50,83 @@ pub(crate) struct Vmm {
     pub(crate) mmu_version: MmuVersion,
     /// Page table allocations required for mappings.
     page_table_allocs: KVec<Pin<KBox<AllocatedBlocks>>>,
+    /// Buddy allocator for virtual address range tracking.
+    virt_buddy: GpuBuddy,
 }
 
 impl Vmm {
     /// Create a new [`Vmm`] for the given Page Directory Base address.
-    pub(crate) fn new(pdb_addr: VramAddress, mmu_version: MmuVersion) -> Result<Self> {
+    ///
+    /// The [`Vmm`] will manage a virtual address space of `va_size` bytes.
+    pub(crate) fn new(
+        pdb_addr: VramAddress,
+        mmu_version: MmuVersion,
+        va_size: u64,
+    ) -> Result<Self> {
         // Only MMU v2 is supported for now.
         if mmu_version != MmuVersion::V2 {
             return Err(ENOTSUPP);
         }
 
+        let virt_buddy = GpuBuddy::new(GpuBuddyParams {
+            base_offset: 0,
+            size: va_size,
+            chunk_size: Alignment::new::<SZ_4K>(),
+        })?;
+
         Ok(Self {
             pdb_addr,
             mmu_version,
             page_table_allocs: KVec::new(),
+            virt_buddy,
         })
     }
 
+    /// Allocate a contiguous virtual frame number range.
+    ///
+    /// # Arguments
+    ///
+    /// - `num_pages`: Number of pages to allocate.
+    /// - `va_range`: `None` = allocate anywhere, `Some(range)` = constrain allocation to the given
+    ///   range.
+    pub(crate) fn alloc_vfn_range(
+        &self,
+        num_pages: usize,
+        va_range: Option<Range<u64>>,
+    ) -> Result<(Vfn, Pin<KBox<AllocatedBlocks>>)> {
+        let size: u64 = (num_pages as u64)
+            .checked_mul(PAGE_SIZE as u64)
+            .ok_or(EOVERFLOW)?;
+
+        let mode = match va_range {
+            Some(r) => {
+                let range_size = r.end.checked_sub(r.start).ok_or(EOVERFLOW)?;
+                if range_size != size {
+                    return Err(EINVAL);
+                }
+                GpuBuddyAllocMode::Range(r)
+            }
+            None => GpuBuddyAllocMode::Simple,
+        };
+
+        let alloc = KBox::pin_init(
+            self.virt_buddy.alloc_blocks(
+                mode,
+                size,
+                Alignment::new::<SZ_4K>(),
+                GpuBuddyAllocFlag::Contiguous,
+            ),
+            GFP_KERNEL,
+        )?;
+
+        // Get the starting offset of the first block (only block as range is contiguous).
+        let offset = alloc.iter().next().ok_or(ENOMEM)?.offset();
+        let page_size: u64 = PAGE_SIZE.into_safe_cast();
+        let vfn = Vfn::new(offset / page_size);
+
+        Ok((vfn, alloc))
+    }
+
     /// Read the [`Pfn`] for a mapped [`Vfn`] if one is mapped.
     pub(crate) fn read_mapping(&self, mm: &GpuMm, vfn: Vfn) -> Result<Option<Pfn>> {
         let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
Signature · 1 line
2.34.1
#94 lore raw

PATCH v10 16/21 gpu: nova-core: mm: Add multi-page mapping API to VMM

In reply to Joel Fernandes

Add the page table mapping and unmapping API to the Virtual Memory Manager, implementing a two-phase prepare/execute model suitable for use both inside and outside the DMA fence signalling critical path.

---
 drivers/gpu/nova-core/mm/vmm.rs | 366 +++++++++++++++++++++++++++++++-
 1 file changed, 363 insertions(+), 3 deletions(-)
drivers/gpu/nova-core/mm/vmm.rs +363 -3
index 2a65ffd73b0d..542bb0a760ba 100644
@@ -11,21 +11,34 @@
         AllocatedBlocks,
         GpuBuddy,
         GpuBuddyAllocFlag,
+        GpuBuddyAllocFlags,
         GpuBuddyAllocMode,
         GpuBuddyParams, //
     },
     prelude::*,
     ptr::Alignment,
+    rbtree::{RBTree, RBTreeNode},
     sizes::SZ_4K, //
 };
 
-use core::ops::Range;
+use core::{
+    cell::Cell,
+    ops::Range, //
+};
 
 use crate::{
     mm::{
         pagetable::{
-            walk::{PtWalk, WalkResult},
-            MmuVersion, //
+            walk::{
+                PtWalk,
+                WalkPdeResult,
+                WalkResult, //
+            },
+            DualPde,
+            MmuVersion,
+            PageTableLevel,
+            Pde,
+            Pte, //
         },
         GpuMm,
         Pfn,
@@ -52,6 +65,74 @@ pub(crate) struct Vmm {
     page_table_allocs: KVec<Pin<KBox<AllocatedBlocks>>>,
     /// Buddy allocator for virtual address range tracking.
     virt_buddy: GpuBuddy,
+    /// Prepared PT pages pending PDE installation, keyed by `install_addr`.
+    ///
+    /// Populated by `Vmm` mapping prepare phase and drained in the execute phase.
+    /// Shared by all pending maps in the `Vmm`, thus preventing races where 2
+    /// maps might be trying to install the same page table/directory entry pointer.
+    pt_pages: RBTree<VramAddress, PreparedPtPage>,
+}
+
+/// A pre-allocated and zeroed page table page.
+///
+/// Created during the mapping prepare phase and consumed during the mapping execute phase.
+/// Stored in an [`RBTree`] keyed by the PDE slot address (`install_addr`).
+struct PreparedPtPage {
+    /// The allocated and zeroed page table page.
+    alloc: Pin<KBox<AllocatedBlocks>>,
+    /// Page table level -- needed to determine if this PT page is for a dual PDE.
+    level: PageTableLevel,
+}
+
+/// Multi-page prepared mapping -- VA range allocated, ready for execute.
+///
+/// Produced by [`Vmm::prepare_map()`], consumed by [`Vmm::execute_map()`].
+/// The struct owns the VA space allocation between prepare and execute phases.
+pub(crate) struct PreparedMapping {
+    vfn_start: Vfn,
+    num_pages: usize,
+    vfn_alloc: Pin<KBox<AllocatedBlocks>>,
+}
+
+/// Result of a mapping operation -- tracks the active mapped range.
+///
+/// Returned by [`Vmm::execute_map()`] and [`Vmm::map_pages()`].
+/// Owns the VA allocation; the VA range is freed when this is dropped.
+/// Callers must call [`Vmm::unmap_pages()`] before dropping to invalidate
+/// PTEs (dropping only frees the VA range, not the PTE entries).
+pub(crate) struct MappedRange {
+    pub(crate) vfn_start: Vfn,
+    pub(crate) num_pages: usize,
+    /// VA allocation -- freed when [`MappedRange`] is dropped.
+    _vfn_alloc: Pin<KBox<AllocatedBlocks>>,
+    /// Logs a warning if dropped without unmapping.
+    _drop_guard: MustUnmapGuard,
+}
+
+/// Guard that logs a warning once if a [`MappedRange`] is dropped without
+/// calling [`Vmm::unmap_pages()`].
+struct MustUnmapGuard {
+    armed: Cell<bool>,
+}
+
+impl MustUnmapGuard {
+    const fn new() -> Self {
+        Self {
+            armed: Cell::new(true),
+        }
+    }
+
+    fn disarm(&self) {
+        self.armed.set(false);
+    }
+}
+
+impl Drop for MustUnmapGuard {
+    fn drop(&mut self) {
+        if self.armed.get() {
+            kernel::pr_warn!("MappedRange dropped without calling unmap_pages()\n");
+        }
+    }
 }
 
 impl Vmm {
@@ -79,6 +160,7 @@ pub(crate) fn new(
             mmu_version,
             page_table_allocs: KVec::new(),
             virt_buddy,
+            pt_pages: RBTree::new(),
         })
     }
 
@@ -136,4 +218,282 @@ pub(crate) fn read_mapping(&self, mm: &GpuMm, vfn: Vfn) -> Result<Option<Pfn>> {
             WalkResult::Unmapped { .. } | WalkResult::PageTableMissing => Ok(None),
         }
     }
+
+    /// Allocate and zero a physical page table page for a specific PDE slot.
+    /// Called during the map prepare phase.
+    fn alloc_and_zero_page_table(
+        &mut self,
+        mm: &GpuMm,
+        level: PageTableLevel,
+    ) -> Result<PreparedPtPage> {
+        let blocks = KBox::pin_init(
+            mm.buddy().alloc_blocks(
+                GpuBuddyAllocMode::Simple,
+                SZ_4K.into_safe_cast(),
+                Alignment::new::<SZ_4K>(),
+                GpuBuddyAllocFlags::default(),
+            ),
+            GFP_KERNEL,
+        )?;
+
+        // Get page's VRAM address from the allocation.
+        let page_vram = VramAddress::new(blocks.iter().next().ok_or(ENOMEM)?.offset());
+
+        // Zero via PRAMIN.
+        let mut window = mm.pramin().get_window()?;
+        let base = page_vram.raw();
+        for off in (0..PAGE_SIZE).step_by(8) {
+            window.try_write64(base + off, 0)?;
+        }
+
+        Ok(PreparedPtPage {
+            alloc: blocks,
+            level,
+        })
+    }
+
+    /// Ensure all intermediate page table pages are prepared for a [`Vfn`]. Just
+    /// finds out which PDE pages are missing, allocates pages for them, and defers
+    /// installation to the execute phase.
+    ///
+    /// PRAMIN is released before each allocation and re-acquired after. Memory
+    /// allocations are done outside of holding this lock to prevent deadlocks with
+    /// the fence signalling critical path.
+    fn ensure_pte_path(&mut self, mm: &GpuMm, vfn: Vfn) -> Result {
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let max_iter = 2 * self.mmu_version.pde_level_count();
+
+        // Keep looping until all PDE levels are resolved.
+        for _ in 0..max_iter {
+            let mut window = mm.pramin().get_window()?;
+
+            // Walk PDE levels. The closure checks self.pt_pages for prepared-but-uninstalled
+            // pages, letting the walker continue through them as if they were installed in HW.
+            // The walker keeps calling the closure to get these "prepared but not installed" pages.
+            let result = walker.walk_pde_levels(&mut window, vfn, |install_addr| {
+                self.pt_pages
+                    .get(&install_addr)
+                    .and_then(|p| Some(VramAddress::new(p.alloc.iter().next()?.offset())))
+            })?;
+
+            match result {
+                WalkPdeResult::Complete { .. } => {
+                    // All PDE levels resolved.
+                    return Ok(());
+                }
+                WalkPdeResult::Missing {
+                    install_addr,
+                    level,
+                } => {
+                    // Drop PRAMIN before allocation.
+                    drop(window);
+                    let page = self.alloc_and_zero_page_table(mm, level)?;
+                    let node = RBTreeNode::new(install_addr, page, GFP_KERNEL)?;
+                    let old = self.pt_pages.insert(node);
+                    if old.is_some() {
+                        kernel::pr_warn_once!(
+                            "VMM: duplicate install_addr in pt_pages (internal consistency error)\n"
+                        );
+                        return Err(EIO);
+                    }
+
+                    // Loop: re-acquire PRAMIN and re-walk from root.
+                }
+            }
+        }
+
+        kernel::pr_warn!(
+            "VMM: ensure_pte_path: loop exhausted after {} iters (VFN {:?})\n",
+            max_iter,
+            vfn
+        );
+        Err(EIO)
+    }
+
+    /// Prepare resources for mapping `num_pages` pages.
+    ///
+    /// Allocates a contiguous VA range, then walks the hierarchy per-VFN to prepare pages
+    /// for all missing PDEs. Returns a [`PreparedMapping`] with the VA allocation.
+    ///
+    /// If `va_range` is not `None`, the VA range is constrained to the given range. Safe
+    /// to call outside the fence signalling critical path.
+    pub(crate) fn prepare_map(
+        &mut self,
+        mm: &GpuMm,
+        num_pages: usize,
+        va_range: Option<Range<u64>>,
+    ) -> Result<PreparedMapping> {
+        if num_pages == 0 {
+            return Err(EINVAL);
+        }
+
+        // Pre-reserve so execute_map() can use push_within_capacity (no alloc in
+        // fence signalling critical path).
+        // Upper bound on page table pages needed for the full tree (PTE pages + PDE
+        // pages at all levels).
+        let pt_upper_bound = self.mmu_version.pt_pages_upper_bound(num_pages);
+        self.page_table_allocs.reserve(pt_upper_bound, GFP_KERNEL)?;
+
+        // Allocate contiguous VA range.
+        let (vfn_start, vfn_alloc) = self.alloc_vfn_range(num_pages, va_range)?;
+
+        // Walk the hierarchy per-VFN to prepare pages for all missing PDEs.
+        for i in 0..num_pages {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(vfn_start.raw() + i_u64);
+            self.ensure_pte_path(mm, vfn)?;
+        }
+
+        Ok(PreparedMapping {
+            vfn_start,
+            num_pages,
+            vfn_alloc,
+        })
+    }
+
+    /// Execute a prepared multi-page mapping.
+    ///
+    /// Drain prepared PT pages and install PDEs followed by single TLB flush.
+    pub(crate) fn execute_map(
+        &mut self,
+        mm: &GpuMm,
+        prepared: PreparedMapping,
+        pfns: &[Pfn],
+        writable: bool,
+    ) -> Result<MappedRange> {
+        if pfns.len() != prepared.num_pages {
+            return Err(EINVAL);
+        }
+
+        let PreparedMapping {
+            vfn_start,
+            num_pages,
+            vfn_alloc,
+        } = prepared;
+
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let mut window = mm.pramin().get_window()?;
+
+        // First, drain self.pt_pages, install all pending PDEs.
+        let mut cursor = self.pt_pages.cursor_front_mut();
+        while let Some(c) = cursor {
+            let (next, node) = c.remove_current();
+            let (install_addr, page) = node.to_key_value();
+            let page_vram = VramAddress::new(page.alloc.iter().next().ok_or(ENOMEM)?.offset());
+
+            if page.level == self.mmu_version.dual_pde_level() {
+                let new_dpde = DualPde::new_small(self.mmu_version, Pfn::from(page_vram));
+                new_dpde.write(&mut window, install_addr)?;
+            } else {
+                let new_pde = Pde::new_vram(self.mmu_version, Pfn::from(page_vram));
+                new_pde.write(&mut window, install_addr)?;
+            }
+
+            // Track the allocated pages in the `Vmm`.
+            self.page_table_allocs
+                .push_within_capacity(page.alloc)
+                .map_err(|_| ENOMEM)?;
+
+            cursor = next;
+        }
+
+        // Next, write PTEs (all PDEs now installed in HW).
+        for (i, &pfn) in pfns.iter().enumerate() {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(vfn_start.raw() + i_u64);
+            let result = walker.walk_to_pte_lookup_with_window(&mut window, vfn)?;
+
+            match result {
+                WalkResult::Unmapped { pte_addr } | WalkResult::Mapped { pte_addr, .. } => {
+                    let pte = Pte::new_vram(self.mmu_version, pfn, writable);
+                    pte.write(&mut window, pte_addr)?;
+                }
+                WalkResult::PageTableMissing => {
+                    kernel::pr_warn_once!("VMM: page table missing for VFN {vfn:?}\n");
+                    return Err(EIO);
+                }
+            }
+        }
+
+        drop(window);
+
+        // Finally, flush the TLB.
+        mm.tlb().flush(self.pdb_addr)?;
+
+        Ok(MappedRange {
+            vfn_start,
+            num_pages,
+            _vfn_alloc: vfn_alloc,
+            _drop_guard: MustUnmapGuard::new(),
+        })
+    }
+
+    /// Map pages doing prepare and execute in the same call.
+    ///
+    /// This is a convenience wrapper for callers outside the fence signalling critical
+    /// path (e.g., BAR mappings). For DRM usecases, [`Vmm::prepare_map()`] and
+    /// [`Vmm::execute_map()`] will be called separately.
+    pub(crate) fn map_pages(
+        &mut self,
+        mm: &GpuMm,
+        pfns: &[Pfn],
+        va_range: Option<Range<u64>>,
+        writable: bool,
+    ) -> Result<MappedRange> {
+        if pfns.is_empty() {
+            return Err(EINVAL);
+        }
+
+        // Check if provided VA range is sufficient (if provided).
+        if let Some(ref range) = va_range {
+            let required: u64 = pfns
+                .len()
+                .checked_mul(PAGE_SIZE)
+                .ok_or(EOVERFLOW)?
+                .into_safe_cast();
+            let available = range.end.checked_sub(range.start).ok_or(EINVAL)?;
+            if available < required {
+                return Err(EINVAL);
+            }
+        }
+
+        let prepared = self.prepare_map(mm, pfns.len(), va_range)?;
+        self.execute_map(mm, prepared, pfns, writable)
+    }
+
+    /// Unmap all pages in a [`MappedRange`] with a single TLB flush.
+    ///
+    /// Takes the range by value (consumes it), then invalidates PTEs for the range,
+    /// flushes the TLB, then drops the range (freeing the VA). PRAMIN lock is held.
+    pub(crate) fn unmap_pages(&mut self, mm: &GpuMm, range: MappedRange) -> Result {
+        let walker = PtWalk::new(self.pdb_addr, self.mmu_version);
+        let invalid_pte = Pte::invalid(self.mmu_version);
+
+        let mut window = mm.pramin().get_window()?;
+        for i in 0..range.num_pages {
+            let i_u64: u64 = i.into_safe_cast();
+            let vfn = Vfn::new(range.vfn_start.raw() + i_u64);
+            let result = walker.walk_to_pte_lookup_with_window(&mut window, vfn)?;
+
+            match result {
+                WalkResult::Mapped { pte_addr, .. } | WalkResult::Unmapped { pte_addr } => {
+                    invalid_pte.write(&mut window, pte_addr)?;
+                }
+                WalkResult::PageTableMissing => {
+                    continue;
+                }
+            }
+        }
+        drop(window);
+
+        mm.tlb().flush(self.pdb_addr)?;
+
+        // TODO: Internal page table pages (PDE, PTE pages) are still kept around.
+        // This is by design as repeated maps/unmaps will be fast. As a future TODO,
+        // we can add a reclaimer here to reclaim if VRAM is short. For now, the PT
+        // pages are dropped once the `Vmm` is dropped.
+
+        range._drop_guard.disarm(); // Unmap complete, Ok to drop MappedRange.
+        Ok(())
+    }
 }
Signature · 1 line
2.34.1
#95 lore raw

PATCH v10 17/21 gpu: nova-core: Add BAR1 aperture type and size constant

In reply to Joel Fernandes

Add BAR1_SIZE constant and Bar1 type alias for the 256MB BAR1 aperture. These are prerequisites for BAR1 memory access functionality.

---
 drivers/gpu/nova-core/driver.rs          | 8 +++++++-
 drivers/gpu/nova-core/gsp/commands.rs    | 4 ++++
 drivers/gpu/nova-core/gsp/fw/commands.rs | 8 ++++++++
 3 files changed, 19 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/driver.rs +7 -1
index 84b0e1703150..b4311adf4cef 100644
@@ -13,7 +13,10 @@
         Vendor, //
     },
     prelude::*,
-    sizes::SZ_16M,
+    sizes::{
+        SZ_16M,
+        SZ_256M, //
+    },
     sync::{
         atomic::{
             Atomic,
@@ -37,6 +40,7 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
+pub(crate) const BAR1_SIZE: usize = SZ_256M;
 
 // For now we only support Ampere which can use up to 47-bit DMA addresses.
 //
@@ -47,6 +51,8 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
+#[expect(dead_code)]
+pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
 
 kernel::pci_device_table!(
     PCI_TABLE,
drivers/gpu/nova-core/gsp/commands.rs +4 -0
index ec03bf94b34e..5a85478f2ba3 100644
@@ -193,6 +193,9 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
 /// The reply from the GSP to the [`GetGspStaticInfo`] command.
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
+    /// BAR1 Page Directory Entry base address.
+    #[expect(dead_code)]
+    pub(crate) bar1_pde_base: u64,
     /// Usable FB (VRAM) region for driver memory allocation.
     pub(crate) usable_fb_region: Range<u64>,
     /// End of VRAM.
@@ -214,6 +217,7 @@ fn read(
 
         Ok(GetGspStaticInfoReply {
             gpu_name: msg.gpu_name_str(),
+            bar1_pde_base: msg.bar1_pde_base(),
             usable_fb_region: base..base.saturating_add(size),
             total_fb_end,
         })
drivers/gpu/nova-core/gsp/fw/commands.rs +8 -0
index 46932d5c8c1d..25f230254d8f 100644
@@ -125,6 +125,14 @@ impl GspStaticConfigInfo {
         self.0.gpuNameString
     }
 
+    /// Returns the BAR1 Page Directory Entry base address.
+    ///
+    /// This is the root page table address for BAR1 virtual memory,
+    /// set up by GSP-RM firmware.
+    pub(crate) fn bar1_pde_base(&self) -> u64 {
+        self.0.bar1PdeBase
+    }
+
     /// Returns an iterator over valid FB regions from GSP firmware data.
     fn fb_regions(
         &self,
Signature · 1 line
2.34.1
#96 lore raw

PATCH v10 18/21 gpu: nova-core: mm: Add BAR1 user interface

In reply to Joel Fernandes

Add the BAR1 user interface for CPU access to GPU virtual memory through the BAR1 aperture.

---
 drivers/gpu/nova-core/driver.rs       |   1 -
 drivers/gpu/nova-core/gpu.rs          |  21 +++-
 drivers/gpu/nova-core/gsp/commands.rs |   1 -
 drivers/gpu/nova-core/mm.rs           |   1 +
 drivers/gpu/nova-core/mm/bar_user.rs  | 152 ++++++++++++++++++++++++++
 5 files changed, 173 insertions(+), 3 deletions(-)
 create mode 100644 drivers/gpu/nova-core/mm/bar_user.rs
drivers/gpu/nova-core/driver.rs +0 -1
index b4311adf4cef..3bc264a099de 100644
@@ -51,7 +51,6 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
-#[expect(dead_code)]
 pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
 
 kernel::pci_device_table!(
drivers/gpu/nova-core/gpu.rs +20 -1
index c49fa9c380b8..1cd0f147994b 100644
@@ -28,7 +28,12 @@
         commands::GetGspStaticInfoReply,
         Gsp, //
     },
-    mm::GpuMm,
+    mm::{
+        bar_user::BarUser,
+        pagetable::MmuVersion,
+        GpuMm,
+        VramAddress, //
+    },
     regs,
 };
 
@@ -122,6 +127,11 @@ pub(crate) const fn arch(self) -> Architecture {
     pub(crate) const fn needs_fwsec_bootloader(self) -> bool {
         matches!(self.arch(), Architecture::Turing) || matches!(self, Self::GA100)
     }
+
+    /// Returns the MMU version for this chipset.
+    pub(crate) fn mmu_version(self) -> MmuVersion {
+        MmuVersion::from(self.arch())
+    }
 }
 
 // TODO
@@ -250,6 +260,8 @@ pub(crate) struct Gpu {
     gsp: Gsp,
     /// Static GPU information from GSP.
     gsp_static_info: GetGspStaticInfoReply,
+    /// BAR1 user interface for CPU access to GPU virtual memory.
+    bar_user: BarUser,
 }
 
 impl Gpu {
@@ -308,6 +320,13 @@ pub(crate) fn new<'a>(
                 }, pramin_vram_region)?
             },
 
+            // Create BAR1 user interface for CPU access to GPU virtual memory.
+            bar_user: {
+                let pdb_addr = VramAddress::new(gsp_static_info.bar1_pde_base);
+                let bar1_size = pdev.resource_len(1)?;
+                BarUser::new(pdb_addr, spec.chipset, bar1_size)?
+            },
+
             bar: devres_bar,
         })
     }
drivers/gpu/nova-core/gsp/commands.rs +0 -1
index 5a85478f2ba3..f549c28172ab 100644
@@ -194,7 +194,6 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {
 pub(crate) struct GetGspStaticInfoReply {
     gpu_name: [u8; 64],
     /// BAR1 Page Directory Entry base address.
-    #[expect(dead_code)]
     pub(crate) bar1_pde_base: u64,
     /// Usable FB (VRAM) region for driver memory allocation.
     pub(crate) usable_fb_region: Range<u64>,
drivers/gpu/nova-core/mm.rs +1 -0
index 1594279dea20..9a38eeab53a6 100644
@@ -4,6 +4,7 @@
 
 #![expect(dead_code)]
 
+pub(crate) mod bar_user;
 pub(crate) mod pagetable;
 pub(crate) mod pramin;
 pub(crate) mod tlb;
drivers/gpu/nova-core/mm/bar_user.rs new file +152 -0

152 lines not on this page · raw message

Signature · 1 line
2.34.1
#97 lore raw

PATCH v10 19/21 gpu: nova-core: mm: Add BAR1 memory management self-tests

In reply to Joel Fernandes

Add self-tests for BAR1 access during driver probe when CONFIG_NOVA_MM_SELFTESTS is enabled (default disabled). This results in testing the Vmm, GPU buddy allocator and BAR1 region all of which should function correctly for the tests to pass.

---
 drivers/gpu/nova-core/Kconfig        |  10 ++
 drivers/gpu/nova-core/driver.rs      |   2 +
 drivers/gpu/nova-core/gpu.rs         |  36 ++++
 drivers/gpu/nova-core/mm/bar_user.rs | 236 +++++++++++++++++++++++++++
 4 files changed, 284 insertions(+)
drivers/gpu/nova-core/Kconfig +10 -0
index 6513007bf66f..35de55aabcfc 100644
@@ -15,3 +15,13 @@ config NOVA_CORE
 	  This driver is work in progress and may not be functional.
 
 	  If M is selected, the module will be called nova_core.
+
+config NOVA_MM_SELFTESTS
+	bool "Memory management self-tests"
+	depends on NOVA_CORE
+	help
+	  Enable self-tests for the memory management subsystem. When enabled,
+	  tests are run during GPU probe to verify PRAMIN aperture access,
+	  page table walking, and BAR1 virtual memory mapping functionality.
+
+	  This is a testing option and is default-disabled.
drivers/gpu/nova-core/driver.rs +2 -0
index 3bc264a099de..b1aafaff0cee 100644
@@ -101,6 +101,8 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> impl PinInit<Self, E
 
             Ok(try_pin_init!(Self {
                 gpu <- Gpu::new(pdev, bar.clone(), bar.access(pdev.as_ref())?),
+                // Run optional GPU selftests.
+                _: { gpu.run_selftests(pdev)? },
                 _reg <- auxiliary::Registration::new(
                     pdev.as_ref(),
                     c"nova-drm",
drivers/gpu/nova-core/gpu.rs +36 -0
index 1cd0f147994b..021fc7cc7247 100644
@@ -341,4 +341,40 @@ pub(crate) fn unbind(&self, dev: &device::Device<device::Core>) {
             .inspect(|bar| self.sysmem_flush.unregister(bar))
             .is_err());
     }
+
+    /// Run selftests on the constructed [`Gpu`].
+    pub(crate) fn run_selftests(
+        mut self: Pin<&mut Self>,
+        pdev: &pci::Device<device::Bound>,
+    ) -> Result {
+        self.as_mut().run_mm_selftests(pdev)?;
+        Ok(())
+    }
+
+    #[cfg(CONFIG_NOVA_MM_SELFTESTS)]
+    fn run_mm_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) -> Result {
+        use crate::driver::BAR1_SIZE;
+
+        // BAR1 self-tests.
+        let bar1 = Arc::pin_init(
+            pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
+            GFP_KERNEL,
+        )?;
+        let bar1_access = bar1.access(pdev.as_ref())?;
+
+        crate::mm::bar_user::run_self_test(
+            pdev.as_ref(),
+            &self.mm,
+            bar1_access,
+            self.gsp_static_info.bar1_pde_base,
+            self.spec.chipset,
+        )?;
+
+        Ok(())
+    }
+
+    #[cfg(not(CONFIG_NOVA_MM_SELFTESTS))]
+    fn run_mm_selftests(self: Pin<&mut Self>, _pdev: &pci::Device<device::Bound>) -> Result {
+        Ok(())
+    }
 }
drivers/gpu/nova-core/mm/bar_user.rs +236 -0

239 lines not on this page · raw message

Signature · 1 line
2.34.1
#98 lore raw

PATCH v10 20/21 gpu: nova-core: mm: Add PRAMIN aperture self-tests

In reply to Joel Fernandes

Add self-tests for the PRAMIN aperture mechanism to verify correct operation during GPU probe. The tests validate various alignment requirements and corner cases.

The tests are default disabled and behind CONFIG_NOVA_MM_SELFTESTS. When enabled, tests run after GSP boot during probe.

---
 drivers/gpu/nova-core/gpu.rs       |   3 +
 drivers/gpu/nova-core/mm/pramin.rs | 209 +++++++++++++++++++++++++++++
 2 files changed, 212 insertions(+)
drivers/gpu/nova-core/gpu.rs +3 -0
index 021fc7cc7247..8206ec015b26 100644
@@ -355,6 +355,9 @@ pub(crate) fn run_selftests(
     fn run_mm_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) -> Result {
         use crate::driver::BAR1_SIZE;
 
+        // PRAMIN aperture self-tests.
+        crate::mm::pramin::run_self_test(pdev.as_ref(), self.mm.pramin(), self.spec.chipset)?;
+
         // BAR1 self-tests.
         let bar1 = Arc::pin_init(
             pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
drivers/gpu/nova-core/mm/pramin.rs +209 -0

218 lines not on this page · raw message

Signature · 1 line
2.34.1
#99 lore raw

PATCH v10 21/21 gpu: nova-core: Use runtime BAR1 size instead of hardcoded 256MB

In reply to Joel Fernandes

From: Zhi Wang <[email protected]>

Remove the hardcoded BAR1_SIZE = SZ_256M constant. On GPUs like L40 the BAR1 aperture is larger than 256MB; using a hardcoded size prevents large BAR1 from working and mapping it would fail.

---
 drivers/gpu/nova-core/driver.rs | 8 ++------
 drivers/gpu/nova-core/gpu.rs    | 7 +------
 2 files changed, 3 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/driver.rs +2 -6
index b1aafaff0cee..6f95f8672158 100644
@@ -13,10 +13,7 @@
         Vendor, //
     },
     prelude::*,
-    sizes::{
-        SZ_16M,
-        SZ_256M, //
-    },
+    sizes::SZ_16M,
     sync::{
         atomic::{
             Atomic,
@@ -40,7 +37,6 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
-pub(crate) const BAR1_SIZE: usize = SZ_256M;
 
 // For now we only support Ampere which can use up to 47-bit DMA addresses.
 //
@@ -51,7 +47,7 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
-pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
+pub(crate) type Bar1 = pci::Bar;
 
 kernel::pci_device_table!(
     PCI_TABLE,
drivers/gpu/nova-core/gpu.rs +1 -6
index 8206ec015b26..ba6f1f6f0485 100644
@@ -353,16 +353,11 @@ pub(crate) fn run_selftests(
 
     #[cfg(CONFIG_NOVA_MM_SELFTESTS)]
     fn run_mm_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) -> Result {
-        use crate::driver::BAR1_SIZE;
-
         // PRAMIN aperture self-tests.
         crate::mm::pramin::run_self_test(pdev.as_ref(), self.mm.pramin(), self.spec.chipset)?;
 
         // BAR1 self-tests.
-        let bar1 = Arc::pin_init(
-            pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
-            GFP_KERNEL,
-        )?;
+        let bar1 = Arc::pin_init(pdev.iomap_region(1, c"nova-core/bar1"), GFP_KERNEL)?;
         let bar1_access = bar1.access(pdev.as_ref())?;
 
         crate::mm::bar_user::run_self_test(
Signature · 1 line
2.34.1
#100 lore raw

Re: [PATCH v10 21/21] gpu: nova-core: Use runtime BAR1 size instead of hardcoded 256MB

Quoted 58 lines · Joel Fernandes

From: Zhi Wang <[email protected]>

Remove the hardcoded BAR1_SIZE = SZ_256M constant. On GPUs like L40 the BAR1 aperture is larger than 256MB; using a hardcoded size prevents large BAR1 from working and mapping it would fail.

---
 drivers/gpu/nova-core/driver.rs | 8 ++------
 drivers/gpu/nova-core/gpu.rs    | 7 +------
 2 files changed, 3 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/driver.rs +2 -6
index b1aafaff0cee..6f95f8672158 100644
@@ -13,10 +13,7 @@
         Vendor, //
     },
     prelude::*,
-    sizes::{
-        SZ_16M,
-        SZ_256M, //
-    },
+    sizes::SZ_16M,
     sync::{
         atomic::{
             Atomic,
@@ -40,7 +37,6 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
-pub(crate) const BAR1_SIZE: usize = SZ_256M;
 
 // For now we only support Ampere which can use up to 47-bit DMA addresses.
 //
@@ -51,7 +47,7 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
-pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
+pub(crate) type Bar1 = pci::Bar;
 
 kernel::pci_device_table!(
     PCI_TABLE,
drivers/gpu/nova-core/gpu.rs +0 -2
index 8206ec015b26..ba6f1f6f0485 100644
@@ -353,16 +353,11 @@ pub(crate) fn run_selftests(
 
     #[cfg(CONFIG_NOVA_MM_SELFTESTS)]
     fn run_mm_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) -> Result {
-        use crate::driver::BAR1_SIZE;
-
         // PRAMIN aperture self-tests.
         crate::mm::pramin::run_self_test(pdev.as_ref(), self.mm.pramin(), self.spec.chipset)?;

drivers/gpu/nova-core/gpu.rs pub(crate) fn run_selftests( · line 363

Quoted 10 lines
drivers/gpu/nova-core/gpu.rs +1 -4
 
         // BAR1 self-tests.
-        let bar1 = Arc::pin_init(
-            pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
-            GFP_KERNEL,
-        )?;
+        let bar1 = Arc::pin_init(pdev.iomap_region(1, c"nova-core/bar1"), GFP_KERNEL)?;
         let bar1_access = bar1.access(pdev.as_ref())?;
 
         crate::mm::bar_user::run_self_test(

Can we move this directly after patch 17 which adds the fixed bar1? Or alternatively fold it in while preserving Zhi's attribution (I am not sure what the conventional method for this is).

#101 lore raw

Re: [PATCH v10 21/21] gpu: nova-core: Use runtime BAR1 size instead of hardcoded 256MB

Quoted 68 lines · Eliot Courtney
Quoted 67 lines · Joel Fernandes

From: Zhi Wang <[email protected]>

Remove the hardcoded BAR1_SIZE = SZ_256M constant. On GPUs like L40 the BAR1 aperture is larger than 256MB; using a hardcoded size prevents large BAR1 from working and mapping it would fail.

---
 drivers/gpu/nova-core/driver.rs | 8 ++------
 drivers/gpu/nova-core/gpu.rs    | 7 +------
 2 files changed, 3 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/driver.rs +2 -6
index b1aafaff0cee..6f95f8672158 100644
@@ -13,10 +13,7 @@
         Vendor, //
     },
     prelude::*,
-    sizes::{
-        SZ_16M,
-        SZ_256M, //
-    },
+    sizes::SZ_16M,
     sync::{
         atomic::{
             Atomic,
@@ -40,7 +37,6 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
-pub(crate) const BAR1_SIZE: usize = SZ_256M;
 
 // For now we only support Ampere which can use up to 47-bit DMA addresses.
 //
@@ -51,7 +47,7 @@ pub(crate) struct NovaCore {
 const GPU_DMA_BITS: u32 = 47;
 
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
-pub(crate) type Bar1 = pci::Bar<BAR1_SIZE>;
+pub(crate) type Bar1 = pci::Bar;
 
 kernel::pci_device_table!(
     PCI_TABLE,
drivers/gpu/nova-core/gpu.rs +1 -6
index 8206ec015b26..ba6f1f6f0485 100644
@@ -353,16 +353,11 @@ pub(crate) fn run_selftests(
 
     #[cfg(CONFIG_NOVA_MM_SELFTESTS)]
     fn run_mm_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bound>) -> Result {
-        use crate::driver::BAR1_SIZE;
-
         // PRAMIN aperture self-tests.
         crate::mm::pramin::run_self_test(pdev.as_ref(), self.mm.pramin(), self.spec.chipset)?;
 
         // BAR1 self-tests.
-        let bar1 = Arc::pin_init(
-            pdev.iomap_region_sized::<BAR1_SIZE>(1, c"nova-core/bar1"),
-            GFP_KERNEL,
-        )?;
+        let bar1 = Arc::pin_init(pdev.iomap_region(1, c"nova-core/bar1"), GFP_KERNEL)?;
         let bar1_access = bar1.access(pdev.as_ref())?;
 
Quoted 5 lines

drivers/gpu/nova-core/gpu.rs pub(crate) fn run_selftests( · line 363

Quoted 1 line
drivers/gpu/nova-core/gpu.rs +0 -0
         crate::mm::bar_user::run_self_test(

Can we move this directly after patch 17 which adds the fixed bar1? Or alternatively fold it in while preserving Zhi's attribution (I am not sure what the conventional method for this is).

Generally, squashing and attribution works. I will do that with attribution. (Zhi did tell me he would be Ok with that as well in this instance).

thanks,

--
Joel Fernandes