gpu: nova-core: Add memory management support
Subsystems (8)
v12 cover gpu: nova-core: Add memory management support
The patches are based on drm-rust-next and work on Ampere, and should work on Blackwell (with a few more patches) once John's Blackwell patches are merged, however it does not depend on those patches and can independently go in.
This series depends on Alex Courbot's bitfield series:
https://lore.kernel.org/all/[email protected]/
The git tree with all patches can be found at:git://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git (tag: nova-mm-v12-20260425)
Change log:
Changes from v11 to v12:
- Make VMM use maple tree for virtual address range management (Danilo)
- Added "rust: maple_tree: implement Send and Sync for MapleTree" sent out before.
- Refactored MMU-version dispatch from wrapper enums (Pte/Pde/DualPde) to trait-based generics (PteOps, PdeOps, DualPdeOps, MmuConfig). (Eliot, Alex)
- Replaces the previous try_access() patterns with access() (Danilo).
- Added Reviewed-by John Hubbard on some GSP patches.
- Removed the Static GPU information from GSP. comment that added no information beyond the field name (John Hubbard).
- Removed Nouveau references from one comment (John Hubbard).
- PRAMIN doc: widened address columns in the ASCII diagrams.
- Logged FB region and total VRAM at boot (Alex).
- Dropped a useless as_ref() in PRAMIN run_self_test.
Changes from v10 to v11:
- Rebased on new bitfield! macro infrastructure.
- Squashed Zhi Wang's "Use runtime BAR1 size" fix with Co-develop tag (Eliot)
- Refactored page table walker to generics (PtWalkInner<M: MmuConfig>) (Eliot, Alex)
- Changed first_usable_fb_region() to return Range<u64> with checked_add (Eliot)
- Tightened visibility from pub(crate) to pub(super) across mm submodules (Eliot)
- TLB flush: added ack_globally bit for global engine acknowledgment.
Changes from v9 to v10:
- Rebased and dropped patches already merged in to drm-rust-next.
- GPU_BUDDY select folded into GpuMm patch.
- updated code with new register macro API.
- Refactored fb_regions() to use iterator (Alex Courbot).
- Renamed Pramin::window() to get_window() to make it more clear it is 'acquiring a resource'.
- Converted Bar0WindowTarget to bounded_enum! macro, replacing TryFrom. Allows to use with_* instead of try_with_*.
Changes from v8 to v9:
- Added fixes from Zhi Wang for bitfield position changes in virtual address and larger BAR1 size on some platforms. Tested and working for vGPU usecase!
- Refactored gsp: boot() to return only GspStaticInfo, removing FbLayout (Alex).
- bar1_pde_base and bar2_pde_base are now accessed via GspStaticInfo directly (Alex).
- Added new patch "gsp: Expose total physical VRAM end from FB region info" introducing total_fb_end() to expose VRAM extent (Alex).
- Consolidated usable VRAM and BarUser setup; removed dedicated
"fb: Add usable_vram field to FbLayout", "mm: Use usable VRAM region for buddy allocator", and "mm: Add BarUser to struct Gpu and create at boot".
Changes from v7 to v8:
- Incorporated "Select GPU_BUDDY for VRAM allocation" patch from the dependency series (Alex).
- Significant patch reordering for better logical flow (GSP/FB patches moved earlier, page table patches, Vmm, Bar1, tests) (Alex).
- Replaced several 'as' usages with into_safe_cast() (Danilo, Alex).
- Updated BAR 1 test cases to include exercising the block size API (Eliot, Danilo).
Changes from v6 to v7:
- Addressed DMA fence signalling usecase per Danilo's feedback.
Pre v6:
- Simplified PRAMIN code (John Hubbard, Alex Courbot).
- Handled different MMU versions: ver2 versus ver3 (John Hubbard).
- Added BAR1 usecase so we have user of DRM Buddy / VMM (John Hubbard).
- Iterating over clist/buddy bindings.
Link to v11: https://lore.kernel.org/all/[email protected]/
Link to v10: https://lore.kernel.org/all/[email protected]/
Link to v9: https://lore.kernel.org/all/[email protected]/
Link to v8: https://lore.kernel.org/all/[email protected]/
Link to v7: https://lore.kernel.org/all/[email protected]/
Joel Fernandes (22):
gpu: nova-core: gsp: Return GspStaticInfo from boot()
gpu: nova-core: gsp: Extract usable FB region from GSP
gpu: nova-core: gsp: Expose total physical VRAM end from FB region
info
gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
docs: gpu: nova-core: Document the PRAMIN aperture mechanism
gpu: nova-core: mm: Add common memory management types
gpu: nova-core: mm: Add TLB flush support
gpu: nova-core: mm: Add GpuMm centralized memory manager
gpu: nova-core: mm: Add common types for all page table formats
gpu: nova-core: mm: Add MMU v2 page table types
gpu: nova-core: mm: Add MMU v3 page table types
gpu: nova-core: mm: Add page table entry operation traits
gpu: nova-core: mm: Add page table walker for MMU v2/v3
gpu: nova-core: mm: Add Virtual Memory Manager
gpu: nova-core: mm: Add virtual address range tracking to VMM
gpu: nova-core: mm: Add multi-page mapping API to VMM
gpu: nova-core: Add BAR1 aperture type and size constant
gpu: nova-core: mm: Add BAR1 user interface
gpu: nova-core: mm: Add BAR1 memory management self-tests
gpu: nova-core: mm: Add PRAMIN aperture self-tests
gpu: nova-core: mm: pramin: drop useless as_ref() in run_self_test
rust: maple_tree: implement Send and Sync for MapleTree
Documentation/gpu/nova/core/pramin.rst | 123 +++++ Documentation/gpu/nova/index.rst | 1 + drivers/gpu/nova-core/Kconfig | 11 + drivers/gpu/nova-core/driver.rs | 3 + drivers/gpu/nova-core/gpu.rs | 99 +++- drivers/gpu/nova-core/gsp/boot.rs | 12 +- drivers/gpu/nova-core/gsp/commands.rs | 18 +- drivers/gpu/nova-core/gsp/fw/commands.rs | 60 ++- drivers/gpu/nova-core/mm.rs | 267 +++++++++++ drivers/gpu/nova-core/mm/bar_user.rs | 400 ++++++++++++++++ drivers/gpu/nova-core/mm/pagetable.rs | 418 +++++++++++++++++ drivers/gpu/nova-core/mm/pagetable/map.rs | 355 ++++++++++++++ drivers/gpu/nova-core/mm/pagetable/ver2.rs | 306 ++++++++++++ drivers/gpu/nova-core/mm/pagetable/ver3.rs | 431 +++++++++++++++++ drivers/gpu/nova-core/mm/pagetable/walk.rs | 258 +++++++++++ drivers/gpu/nova-core/mm/pramin.rs | 515 +++++++++++++++++++++ drivers/gpu/nova-core/mm/tlb.rs | 130 ++++++ drivers/gpu/nova-core/mm/vmm.rs | 370 +++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 77 +++ rust/kernel/maple_tree.rs | 29 +- 21 files changed, 3868 insertions(+), 16 deletions(-) create mode 100644 Documentation/gpu/nova/core/pramin.rst create mode 100644 drivers/gpu/nova-core/mm.rs create mode 100644 drivers/gpu/nova-core/mm/bar_user.rs create mode 100644 drivers/gpu/nova-core/mm/pagetable.rs create mode 100644 drivers/gpu/nova-core/mm/pagetable/map.rs create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver3.rs create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs create mode 100644 drivers/gpu/nova-core/mm/pramin.rs create mode 100644 drivers/gpu/nova-core/mm/tlb.rs create mode 100644 drivers/gpu/nova-core/mm/vmm.rs
base-commit: 74a720e00dfbb3ab92934660b4692b90331623ac
Signature · 1 line
2.34.1
PATCH v12 1/22 gpu: nova-core: gsp: Return GspStaticInfo from boot()
Refactor the GSP boot function to return GetGspStaticInfoReply.
This enables access required for memory management initialization to:
- bar1_pde_base: BAR1 page directory base.
- bar2_pde_base: BAR2 page directory base.
- usable memory regions in video memory.
--- drivers/gpu/nova-core/gpu.rs | 8 ++++++-- drivers/gpu/nova-core/gsp/boot.rs | 12 ++++++++---- 2 files changed, 14 insertions(+), 6 deletions(-)
drivers/gpu/nova-core/gpu.rs +6 -2
@@ -21,7 +21,10 @@ }, fb::SysmemFlush, gfw, - gsp::Gsp, + gsp::{ + commands::GetGspStaticInfoReply, + Gsp, // + }, regs, };
@@ -238,6 +241,7 @@ pub(crate) struct Gpu {/// GSP runtime data. Temporarily an empty placeholder. #[pin] gsp: Gsp, + gsp_static_info: GetGspStaticInfoReply, }impl Gpu {
@@ -269,7 +273,7 @@ pub(crate) fn new<'a>( gsp <- Gsp::new(pdev), - _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? }, + gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? }, bar: devres_bar, })
drivers/gpu/nova-core/gsp/boot.rs +8 -4
@@ -33,7 +33,10 @@ }, gpu::Chipset, gsp::{ - commands, + commands::{ + self, + GetGspStaticInfoReply, // + }, sequencer::{GspSequencer ,GspSequencerParams ,//
@@ -145,7 +148,7 @@ pub(crate) fn boot( chipset: Chipset, gsp_falcon: &Falcon<Gsp>, sec2_falcon: &Falcon<Sec2>, - ) -> Result { + ) -> Result<GetGspStaticInfoReply> {let dev = pdev.as_ref ();let bios =Vbios ::new (dev, bar)?;
@@ -229,12 +232,13 @@ pub(crate) fn boot( commands::wait_gsp_init_done (&self.cmdq)?;// Obtain and display basic GPU information. -let info = commands::get_gsp_info (&self.cmdq, bar)?; +let info = commands::get_gsp_info (&self.cmdq, bar) + .inspect_err (|e|dev_err! (pdev,"Failed to obtain GSP static info ({:?})\n" , e))?;match info.gpu_name () {Ok (name) =>dev_info! (pdev,"GPU name: {}\n" , name),Err (e) =>dev_warn! (pdev,"GPU name unavailable: {:?}\n" , e), } -Ok (()) +Ok (info) } }
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 01/22] gpu: nova-core: gsp: Return GspStaticInfo from boot()
Quoted 33 lines · Joel Fernandes
Refactor the GSP boot function to return GetGspStaticInfoReply.
This enables access required for memory management initialization to:
-bar1_pde_base: BAR1 page directory base.
-bar2_pde_base: BAR2 page directory base.
- usable memory regions in video memory.Reviewed-by Eliot Courtney <[email protected]>Reviewed-by John Hubbard <[email protected]>Cc Nikola Djukic <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/gpu.rs | 8 ++++++-- drivers/gpu/nova-core/gsp/boot.rs | 12 ++++++++---- 2 files changed, 14 insertions(+), 6 deletions(-)drivers/gpu/nova-core/gpu.rs +4
-1@@ -21,7 +21,10 @@ }, fb::SysmemFlush, gfw, - gsp::Gsp, + gsp::{ + commands::GetGspStaticInfoReply, + Gsp, // + }, regs, };
drivers/gpu/nova-core/gpu.rs pub(crate) struct Gpu { · line 244
Quoted 4 lines
drivers/gpu/nova-core/gpu.rs +1
-0@@ -238,6 +241,7 @@ pub(crate) struct Gpu {/// GSP runtime data. Temporarily an empty placeholder. #[pin] gsp: Gsp, + gsp_static_info: GetGspStaticInfoReply,
A short doccomment would be nice, but otherwise this looks good.
Re: [PATCH v12 01/22] gpu: nova-core: gsp: Return GspStaticInfo from boot()
Quoted 33 lines · Alexandre Courbot
Quoted 32 lines · Joel Fernandes
Refactor the GSP boot function to return GetGspStaticInfoReply.
This enables access required for memory management initialization to:
-bar1_pde_base: BAR1 page directory base.
-bar2_pde_base: BAR2 page directory base.
- usable memory regions in video memory.Reviewed-by Eliot Courtney <[email protected]>Reviewed-by John Hubbard <[email protected]>Cc Nikola Djukic <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/gpu.rs | 8 ++++++-- drivers/gpu/nova-core/gsp/boot.rs | 12 ++++++++---- 2 files changed, 14 insertions(+), 6 deletions(-)drivers/gpu/nova-core/gpu.rs +4
-1@@ -21,7 +21,10 @@ }, fb::SysmemFlush, gfw, - gsp::Gsp, + gsp::{ + commands::GetGspStaticInfoReply, + Gsp, // + }, regs, };
Quoted 7 lines
drivers/gpu/nova-core/gpu.rs pub(crate) struct Gpu { · line 244
Quoted 5 lines
drivers/gpu/nova-core/gpu.rs +1
-0@@ -238,6 +241,7 @@ pub(crate) struct Gpu {/// GSP runtime data. Temporarily an empty placeholder. #[pin] gsp: Gsp, + gsp_static_info: GetGspStaticInfoReply,A short doccomment would be nice, but otherwise this looks good.
I had added one before, but John suggested to drop it. I could add it back again if all agree on what we want to do. I am Ok with either though I'd lean more to the fact that its not necessary since it will basically read like the variable.
thanks.
Alexandre Courbot
maintainerRe: [PATCH v12 01/22] gpu: nova-core: gsp: Return GspStaticInfo from boot()
Quoted 44 lines · Joel Fernandes
Quoted 33 lines · Alexandre Courbot
Quoted 32 lines · Joel Fernandes
Refactor the GSP boot function to return GetGspStaticInfoReply.
This enables access required for memory management initialization to:
-bar1_pde_base: BAR1 page directory base.
-bar2_pde_base: BAR2 page directory base.
- usable memory regions in video memory.Reviewed-by Eliot Courtney <[email protected]>Reviewed-by John Hubbard <[email protected]>Cc Nikola Djukic <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/gpu.rs | 8 ++++++-- drivers/gpu/nova-core/gsp/boot.rs | 12 ++++++++---- 2 files changed, 14 insertions(+), 6 deletions(-)drivers/gpu/nova-core/gpu.rs +4
-1@@ -21,7 +21,10 @@ }, fb::SysmemFlush, gfw, - gsp::Gsp, + gsp::{ + commands::GetGspStaticInfoReply, + Gsp, // + }, regs, };Quoted 7 lines
drivers/gpu/nova-core/gpu.rs pub(crate) struct Gpu { · line 244
Quoted 5 lines
drivers/gpu/nova-core/gpu.rs +1
-0@@ -238,6 +241,7 @@ pub(crate) struct Gpu {/// GSP runtime data. Temporarily an empty placeholder. #[pin] gsp: Gsp, + gsp_static_info: GetGspStaticInfoReply,A short doccomment would be nice, but otherwise this looks good.
Quoted 3 lines
I had added one before, but John suggested to drop it. I could add it back again if all agree on what we want to do. I am Ok with either though I'd lean more to the fact that its not necessary since it will basically read like the variable.
Without is fine, the name/type is indeed explicit enough. I was worried that Clippy would start complaining if we need to make this type public once nova-drm calls into nova-core. But we can fix that if it actually happens.
PATCH v12 2/22 gpu: nova-core: gsp: Extract usable FB region from GSP
In reply to Joel FernandesAdd first_usable_fb_region() to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.
The extracted region is stored in GetGspStaticInfoReply and exposed as usable_fb_region field for use by the memory subsystem.
--- drivers/gpu/nova-core/gsp/commands.rs | 11 ++++-- drivers/gpu/nova-core/gsp/fw/commands.rs | 45 +++++++++++++++++++++++- 2 files changed, 52 insertions(+), 4 deletions(-)
drivers/gpu/nova-core/gsp/commands.rs +8 -3
@@ -4,6 +4,7 @@ array, convert::Infallible, ffi::FromBytesUntilNulError, + ops::Range, str::Utf8Error, // };
@@ -189,15 +190,18 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> { } } -/// The reply from the GSP to the [`GetGspInfo`] command. +/// The reply from the GSP to the [`GetGspStaticInfo`] command. pub (crate )struct GetGspStaticInfoReply { gpu_name: [u8 ;64 ], +/// Usable FB (VRAM) region for driver memory allocation. +#[expect(dead_code)] +pub (crate ) usable_fb_region:Range <u64 >, }impl MessageFromGsp for GetGspStaticInfoReply {const FUNCTION :MsgFunction =MsgFunction ::GetGspStaticInfo ;type Message =GspStaticConfigInfo ; -type InitError =Infallible ; +type InitError =Error ;fn read ( msg: &Self::Message,
@@ -205,6 +209,7 @@ fn read( ) -> Result<Self, Self::InitError> {Ok (GetGspStaticInfoReply { gpu_name: msg.gpu_name_str (), + usable_fb_region: msg.first_usable_fb_region ().ok_or (ENODEV )?, }) } }
@@ -233,7 +238,7 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> { } } -/// Send the [`GetGspInfo`] command and awaits for its reply. +/// Send the [`GetGspStaticInfo`] command and awaits for its reply. pub (crate )fn get_gsp_info (cmdq: &Cmdq , bar: &Bar0 ) ->Result <GetGspStaticInfoReply > { cmdq.send_command (bar,GetGspStaticInfo ) }
drivers/gpu/nova-core/gsp/fw/commands.rs +44 -1
62 lines not on this page · raw message
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 02/22] gpu: nova-core: gsp: Extract usable FB region from GSP
Quoted 103 lines · Joel Fernandes
Add
first_usable_fb_region()to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.The extracted region is stored in GetGspStaticInfoReply and exposed as
usable_fb_regionfield for use by the memory subsystem.Cc Nikola Djukic <[email protected]>Reviewed-by John Hubbard <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/gsp/commands.rs | 11 ++++-- drivers/gpu/nova-core/gsp/fw/commands.rs | 45 +++++++++++++++++++++++- 2 files changed, 52 insertions(+), 4 deletions(-)drivers/gpu/nova-core/gsp/commands.rs +8
-3@@ -4,6 +4,7 @@ array, convert::Infallible, ffi::FromBytesUntilNulError, + ops::Range, str::Utf8Error, // };@@ -189,15 +190,18 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> { } } -/// The reply from the GSP to the [`GetGspInfo`] command. +/// The reply from the GSP to the [`GetGspStaticInfo`] command. pub (crate )struct GetGspStaticInfoReply { gpu_name: [u8 ;64 ], +/// Usable FB (VRAM) region for driver memory allocation. +#[expect(dead_code)] +pub (crate ) usable_fb_region:Range <u64 >, }impl MessageFromGsp for GetGspStaticInfoReply {const FUNCTION :MsgFunction =MsgFunction ::GetGspStaticInfo ;type Message =GspStaticConfigInfo ; -type InitError =Infallible ; +type InitError =Error ;fn read ( msg: &Self::Message,@@ -205,6 +209,7 @@ fn read( ) -> Result<Self, Self::InitError> {Ok (GetGspStaticInfoReply { gpu_name: msg.gpu_name_str (), + usable_fb_region: msg.first_usable_fb_region ().ok_or (ENODEV )?, }) } }@@ -233,7 +238,7 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> { } } -/// Send the [`GetGspInfo`] command and awaits for its reply. +/// Send the [`GetGspStaticInfo`] command and awaits for its reply. pub (crate )fn get_gsp_info (cmdq: &Cmdq , bar: &Bar0 ) ->Result <GetGspStaticInfoReply > { cmdq.send_command (bar,GetGspStaticInfo ) }drivers/gpu/nova-core/gsp/fw/commands.rs +16
-1@@ -1,5 +1,7 @@// SPDX-License-Identifier: GPL-2.0 +use core::ops::Range ; +use kernel::{ device, pci,@@ -10,7 +12,10 @@ }, // }; -use crate ::gsp::GSP_PAGE_SIZE ; +use crate ::{ + gsp::GSP_PAGE_SIZE , + num::IntoSafeCast ,// +};use super ::bindings;@@ -121,6 +126,44 @@ impl GspStaticConfigInfo {pub (crate )fn gpu_name_str (&self) -> [u8 ;64 ] { self.0 .gpuNameString } + +/// Returns an iterator over valid FB regions from GSP firmware data. +fn fb_regions ( + &self, + ) ->impl Iterator <Item = &bindings::NV2080_CTRL_CMD_FB_GET_FB_REGION_FB_REGION_INFO > { +let fb_info = &self.0 .fbRegionInfoParams; + fb_info + .fbRegion + .iter () + .take (fb_info.numFBRegions.into_safe_cast ())
drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 148
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +10
-0+ .filter(|reg| reg.limit >= reg.base) + } + +/// Extracts the first usable FB region from GSP firmware data. +/// +/// Returns the first region suitable for driver memory allocation as a [`Range<u64>`]. +/// Usable regions are those that satisfy all the following properties: +/// - Are not reserved for firmware internal use. +/// - Are not protected (hardware-enforced access restrictions). +/// - Support compression (can use GPU memory compression for bandwidth).
"can use GPU memory compression for saving bandwidth" maybe?
drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 153
Quoted 5 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +5
-0+/// - Support ISO (isochronous memory for display requiring guaranteed bandwidth). +/// +/// TODO: Multiple discontinuous usable regions of RAM are possible in +/// special cases. We need to support it. +pub (crate )fn first_usable_fb_region (&self) ->Option <Range <u64 >> {
Let's be forward-thinking, and turn this method into usable_fb_regions_iter, returning an iterator. It is trivial to do (just turn find_map into filter), we will need it later, and for now the caller can just do next() to get the first region.
Re: [PATCH v12 02/22] gpu: nova-core: gsp: Extract usable FB region from GSP
Quoted 108 lines · Alexandre Courbot
Quoted 107 lines · Joel Fernandes
Add
first_usable_fb_region()to GspStaticConfigInfo to extract the first usable FB region from GSP's fbRegionInfoParams. Usable regions are those that are not reserved or protected.The extracted region is stored in GetGspStaticInfoReply and exposed as
usable_fb_regionfield for use by the memory subsystem.Cc Nikola Djukic <[email protected]>Reviewed-by John Hubbard <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/gsp/commands.rs | 11 ++++-- drivers/gpu/nova-core/gsp/fw/commands.rs | 45 +++++++++++++++++++++++- 2 files changed, 52 insertions(+), 4 deletions(-)drivers/gpu/nova-core/gsp/commands.rs +8
-3@@ -4,6 +4,7 @@ array, convert::Infallible, ffi::FromBytesUntilNulError, + ops::Range, str::Utf8Error, // };@@ -189,15 +190,18 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> { } } -/// The reply from the GSP to the [`GetGspInfo`] command. +/// The reply from the GSP to the [`GetGspStaticInfo`] command. pub (crate )struct GetGspStaticInfoReply { gpu_name: [u8 ;64 ], +/// Usable FB (VRAM) region for driver memory allocation. +#[expect(dead_code)] +pub (crate ) usable_fb_region:Range <u64 >, }impl MessageFromGsp for GetGspStaticInfoReply {const FUNCTION :MsgFunction =MsgFunction ::GetGspStaticInfo ;type Message =GspStaticConfigInfo ; -type InitError =Infallible ; +type InitError =Error ;fn read ( msg: &Self::Message,@@ -205,6 +209,7 @@ fn read( ) -> Result<Self, Self::InitError> {Ok (GetGspStaticInfoReply { gpu_name: msg.gpu_name_str (), + usable_fb_region: msg.first_usable_fb_region ().ok_or (ENODEV )?, }) } }@@ -233,7 +238,7 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> { } } -/// Send the [`GetGspInfo`] command and awaits for its reply. +/// Send the [`GetGspStaticInfo`] command and awaits for its reply. pub (crate )fn get_gsp_info (cmdq: &Cmdq , bar: &Bar0 ) ->Result <GetGspStaticInfoReply > { cmdq.send_command (bar,GetGspStaticInfo ) }drivers/gpu/nova-core/gsp/fw/commands.rs +20
-1@@ -1,5 +1,7 @@// SPDX-License-Identifier: GPL-2.0 +use core::ops::Range ; +use kernel::{ device, pci,@@ -10,7 +12,10 @@ }, // }; -use crate ::gsp::GSP_PAGE_SIZE ; +use crate ::{ + gsp::GSP_PAGE_SIZE , + num::IntoSafeCast ,// +};use super ::bindings;@@ -121,6 +126,44 @@ impl GspStaticConfigInfo {pub (crate )fn gpu_name_str (&self) -> [u8 ;64 ] { self.0 .gpuNameString } + +/// Returns an iterator over valid FB regions from GSP firmware data. +fn fb_regions ( + &self, + ) ->impl Iterator <Item = &bindings::NV2080_CTRL_CMD_FB_GET_FB_REGION_FB_REGION_INFO > { +let fb_info = &self.0 .fbRegionInfoParams; + fb_info + .fbRegion + .iter () + .take (fb_info.numFBRegions.into_safe_cast ()) + .filter (|reg| reg.limit >= reg.base) + } + +/// Extracts the first usable FB region from GSP firmware data.
Quoted 8 lines
drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 148
Quoted 6 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +6
-0+/// +/// Returns the first region suitable for driver memory allocation as a [`Range<u64>`]. +/// Usable regions are those that satisfy all the following properties: +/// - Are not reserved for firmware internal use. +/// - Are not protected (hardware-enforced access restrictions). +/// - Support compression (can use GPU memory compression for bandwidth). "can use GPU memory compression for saving bandwidth" maybe?
Done.
Quoted 11 lines
drivers/gpu/nova-core/gsp/fw/commands.rs impl GspStaticConfigInfo { · line 153
Quoted 6 lines
drivers/gpu/nova-core/gsp/fw/commands.rs +5
-0+/// - Support ISO (isochronous memory for display requiring guaranteed bandwidth). +/// +/// TODO: Multiple discontinuous usable regions of RAM are possible in +/// special cases. We need to support it. +pub (crate )fn first_usable_fb_region (&self) ->Option <Range <u64 >> {Let's be forward-thinking, and turn this method into
usable_fb_regions_iter, returning an iterator. It is trivial to do (just turnfind_mapintofilter), we will need it later, and for now the caller can just donext()to get the first region.
Done. Indeed, it does not add more LOC while supporting multiple regions.
thanks,
Signature · 1 line
Joel Fernandes
PATCH v12 3/22 gpu: nova-core: gsp: Expose total physical VRAM end from FB region info
In reply to Joel FernandesAdd total_fb_end() to GspStaticConfigInfo that computes the exclusive end address of the highest valid FB region covering both usable and GSP-reserved areas.
This allows callers to know the full physical VRAM extent, not just the allocatable portion.
--- drivers/gpu/nova-core/gpu.rs | 12 +++++++++++- drivers/gpu/nova-core/gsp/commands.rs | 5 +++++ drivers/gpu/nova-core/gsp/fw/commands.rs | 7 +++++++ 3 files changed, 23 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/gpu.rs +11 -1
@@ -273,7 +273,17 @@ pub(crate) fn new<'a>( gsp <- Gsp::new(pdev), - gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? }, + gsp_static_info: { + let info = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?; + +dev_info! ( + pdev.as_ref (), +"Total physical VRAM: {} MiB\n" , + info.total_fb_end >>20 + ); + + info + }, bar: devres_bar, })
drivers/gpu/nova-core/gsp/commands.rs +5 -0
@@ -196,6 +196,8 @@ pub(crate) struct GetGspStaticInfoReply {/// Usable FB (VRAM) region for driver memory allocation. #[expect(dead_code)] pub(crate) usable_fb_region: Range<u64>, +/// End of VRAM. + pub(crate) total_fb_end: u64, }impl MessageFromGsp for GetGspStaticInfoReply {
@@ -207,9 +209,12 @@ fn read( msg: &Self::Message, _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>, ) -> Result<Self, Self::InitError> { +let total_fb_end = msg.total_fb_end ().ok_or (ENODEV )?; +Ok (GetGspStaticInfoReply { gpu_name: msg.gpu_name_str (), usable_fb_region: msg.first_usable_fb_region ().ok_or (ENODEV )?, + total_fb_end, }) } }
drivers/gpu/nova-core/gsp/fw/commands.rs +7 -0
@@ -164,6 +164,13 @@ pub(crate) fn first_usable_fb_region(&self) -> Option<Range<u64>> { } }) } + +/// Compute the end of physical VRAM from all FB regions. +pub (crate )fn total_fb_end (&self) ->Option <u64 > { + self.fb_regions () + .filter_map (|reg| reg.limit.checked_add (1 )) + .max () + } }// SAFETY: Padding is explicit and will not contain uninitialized data.
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 03/22] gpu: nova-core: gsp: Expose total physical VRAM end from FB region info
Quoted 20 lines · Joel Fernandes
Add
total_fb_end()toGspStaticConfigInfothat computes the exclusive end address of the highest valid FB region covering both usable and GSP-reserved areas.This allows callers to know the full physical VRAM extent, not just the allocatable portion.
Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/gpu.rs | 12 +++++++++++- drivers/gpu/nova-core/gsp/commands.rs | 5 +++++ drivers/gpu/nova-core/gsp/fw/commands.rs | 7 +++++++ 3 files changed, 23 insertions(+), 1 deletion(-)drivers/gpu/nova-core/gpu.rs +0
-0@@ -273,7 +273,17 @@ pub(crate) fn new<'a>(
drivers/gpu/nova-core/gpu.rs pub(crate) fn new<'a>( · line 282
Quoted 10 lines
drivers/gpu/nova-core/gpu.rs +7
-1gsp <- Gsp::new(pdev), - gsp_static_info: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? }, + gsp_static_info: { + let info = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?; + + dev_info!( + pdev.as_ref(), + "Total physical VRAM: {} MiB\n", + info.total_fb_end >> 20
info.total_fb_end / u64::SZ_1M (after importing kernel::sizes::SizeConstants) would carry the intent better.
drivers/gpu/nova-core/gpu.rs pub(crate) fn new<'a>( · line 286
Quoted 4 lines
drivers/gpu/nova-core/gpu.rs +4
-0+ ); + + info + },
Let's use inspect:
gsp_static_info: gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)
.inspect(|info|
dev_info!(
pdev.as_ref(),
"Total physical VRAM: {} MiB\n",
info.total_fb_end / u64::SZ_1M
)
)?,If we combine these 3 first patches with the series enabling calls from nova-drm to nova-core, we should be able to implement a TOTAL_VRAM argument to GET_PARAM that would be the first example of an actual Nova ioctl using real information from the GSP (granted, with an unstable interface). Looking forward to this.
Re: [PATCH v12 03/22] gpu: nova-core: gsp: Expose total physical VRAM end from FB region info
Quoted 1 line · Alexandre Courbot
On Sun Apr 26, 2026 at 6:14 AM JST, Joel Fernandes wrote:
[...]
Quoted 17 lines
drivers/gpu/nova-core/gpu.rs pub(crate) fn new<'a>( · line 282
Quoted 7 lines
drivers/gpu/nova-core/gpu.rs +7
-0+ gsp_static_info: { + let info = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?; + + dev_info!( + pdev.as_ref(), + "Total physical VRAM: {} MiB\n", + info.total_fb_end >> 20
info.total_fb_end / u64::SZ_1M(after importingkernel::sizes::SizeConstants) would carry the intent better.drivers/gpu/nova-core/gpu.rs pub(crate) fn new<'a>( · line 286
Quoted 4 lines
drivers/gpu/nova-core/gpu.rs +4
-0+ ); + + info + },Let's use
inspect:
Quoted 8 lines
gsp_static_info: gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)
.inspect(|info|
dev_info!(
pdev.as_ref(),
"Total physical VRAM: {} MiB\n",
info.total_fb_end / u64::SZ_1M
)
)?,Both suggestions applied.
Quoted 5 lines
If we combine these 3 first patches with the series enabling calls from nova-drm to nova-core, we should be able to implement a
TOTAL_VRAMargument toGET_PARAMthat would be the first example of an actual Nova ioctl using real information from the GSP (granted, with an unstable interface). Looking forward to this.
Sounds great.
thanks,
--
Joel Fernandes
PATCH v12 4/22 gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
In reply to Joel FernandesPRAMIN apertures are a crucial mechanism to direct read/write to VRAM. Add support for the same.
--- drivers/gpu/nova-core/mm.rs | 5 + drivers/gpu/nova-core/mm/pramin.rs | 300 +++++++++++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 10 + 4 files changed, 316 insertions(+) create mode 100644 drivers/gpu/nova-core/mm.rs create mode 100644 drivers/gpu/nova-core/mm/pramin.rs
drivers/gpu/nova-core/mm.rs new file +5 -0
@@ -0,0 +1,5 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Memory management subsystems for nova-core. + +pub (crate )mod pramin;
drivers/gpu/nova-core/mm/pramin.rs new file +300 -0
300 lines not on this page · raw message
drivers/gpu/nova-core/nova_core.rs +1 -0
@@ -17,6 +17,7 @@mod gfw;mod gpu;mod gsp; +mod mm;#[macro_use] mod num;mod regs;
drivers/gpu/nova-core/regs.rs +10 -0
@@ -30,6 +30,7 @@ Architecture, Chipset, // }, + mm::pramin::Bar0WindowTarget, num::FromSafeCast, };
@@ -115,6 +116,15 @@ fn fmt(&self, f: &mut kernel::fmt::Formatter<'_>) -> kernel::fmt::Result { } } +register! { +/// BAR0 window control for PRAMIN access. +pub (crate )NV_PBUS_BAR0_WINDOW (u32 ) @0x00001700 { +25 :24 target ?=>Bar0WindowTarget ; +/// Window base address (bits 39:16 of FB addr). +23 :0 window_base; + } +} +// PFB register! {
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 04/22] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
On Sun Apr 26, 2026 at 6:14 AM JST, Joel Fernandes wrote:
<snip>
Quoted 55 lines
PRAMIN apertures are a crucial mechanism to direct read/write to VRAM. Add support for the same.
Cc Nikola Djukic <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/mm.rs | 5 + drivers/gpu/nova-core/mm/pramin.rs | 300 +++++++++++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 10 + 4 files changed, 316 insertions(+) create mode 100644 drivers/gpu/nova-core/mm.rs create mode 100644 drivers/gpu/nova-core/mm/pramin.rsdrivers/gpu/nova-core/mm.rs new file +5
-0@@ -0,0 +1,5 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Memory management subsystems for nova-core. + +pub (crate )mod pramin;drivers/gpu/nova-core/mm/pramin.rs new file +24
-0@@ -0,0 +1,300 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Direct VRAM access through the PRAMIN aperture. +//! +//! PRAMIN provides a 1MB sliding window into VRAM through BAR0, allowing the CPU to access +//! video memory directly. Access is managed through a two-level API: +//! +//! - [`Pramin`]: The parent object that owns the BAR0 reference and synchronization lock. +//! - [`PraminWindow`]: A guard object that holds exclusive PRAMIN access for its lifetime. +//! +//! The PRAMIN aperture is a 1MB region at a fixed offset from BAR0. The window base is +//! controlled by an architecture-specific register and is 64KB aligned. +//! +//! # Examples +//! +//! ## Basic read/write +//! +//! ```no_run +//! use crate::driver::Bar0; +//! use crate::gpu::Chipset; +//! use crate::mm::pramin; +//! use kernel::device; +//! use kernel::devres::Devres; +//! use kernel::prelude::*;
drivers/gpu/nova-core/mm/pramin.rs line 34
Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10
-0+//! use kernel::sync::Arc; +//! +//! fn example( +//! devres_bar: Arc<Devres<Bar0>>, +//! dev: &device::Device<device::Bound>, +//! chipset: Chipset, +//! vram_region: core::ops::Range<u64>, +//! ) -> Result<()> { +//! let pramin = Arc::pin_init( +//! pramin::Pramin::new(devres_bar, dev, chipset, vram_region)?,
Pramin::new takes different arguments (also check the other examples).
Quoted 126 lines
drivers/gpu/nova-core/mm/pramin.rs new file +126
-0+//! GFP_KERNEL, +//! )?; +//! let mut window = pramin.get_window(dev)?; +//! +//! // Write and read back. +//! window.try_write32(0x100, 0xDEADBEEF)?; +//! let val = window.try_read32(0x100)?; +//! assert_eq!(val, 0xDEADBEEF); +//! +//! Ok(()) +//! } +//! ``` +//! +//! ## Auto-repositioning across VRAM regions +//! +//! ```no_run +//! use crate::driver::Bar0; +//! use crate::gpu::Chipset; +//! use crate::mm::pramin; +//! use kernel::device; +//! use kernel::devres::Devres; +//! use kernel::prelude::*; +//! use kernel::sync::Arc; +//! +//! fn example( +//! devres_bar: Arc<Devres<Bar0>>, +//! dev: &device::Device<device::Bound>, +//! chipset: Chipset, +//! vram_region: core::ops::Range<u64>, +//! ) -> Result<()> { +//! let pramin = Arc::pin_init( +//! pramin::Pramin::new(devres_bar, dev, chipset, vram_region)?, +//! GFP_KERNEL, +//! )?; +//! let mut window = pramin.get_window(dev)?; +//! +//! // Access first 1MB region. +//! window.try_write32(0x100, 0x11111111)?; +//! +//! // Access at 2MB - window auto-repositions. +//! window.try_write32(0x200000, 0x22222222)?; +//! +//! // Back to first region - window repositions again. +//! let val = window.try_read32(0x100)?; +//! assert_eq!(val, 0x11111111); +//! +//! Ok(()) +//! } +//! ``` + +#![expect(unused)] + +use core::ops::Range ; + +use crate ::{ + bounded_enum, + driver::Bar0 , + num::IntoSafeCast , + regs,// +}; + +use kernel::{ + devres::Devres , + io::Io , + new_mutex, + num::Bounded , + prelude::*, + revocable::RevocableGuard , + sizes::{ +SZ_1M , +SZ_64K ,// + }, + sync::{ + lock::mutex::MutexGuard , +Arc , +Mutex ,// + }, +}; + +bounded_enum! { +/// Target memory type for the BAR0 window register. +/// +/// Only VRAM is supported; Hopper+ GPUs do not support other targets. +#[derive( Debug )] +pub (crate )enum Bar0WindowTarget withTryFrom <Bounded <u32 ,2 >> { +/// Video RAM (GPU framebuffer memory). +Vram =0 , + } +} + +/// PRAMIN aperture base offset in BAR0. +const PRAMIN_BASE :usize =0x700000 ; + +/// PRAMIN aperture size (1MB). +const PRAMIN_SIZE :usize =SZ_1M ; + +/// Generate a PRAMIN read accessor. +macro_rules! define_pramin_read { + ($name:ident, $ty:ty) => { +#[doc = concat! ( "Read a `" , stringify! ($ty), "` from VRAM at the given offset." )] +pub (crate )fn $name(&mut self, vram_offset:usize ) ->Result <$ty> { +let (bar_offset, new_base) = + self.compute_window (vram_offset, ::core::mem::size_of ::<$ty>())?; + +if let Some (base) = new_base { +Self ::write_window_base (&self.bar, base)?; + *self.state = base; + } + self.bar.$name(bar_offset) + } + }; +} + +/// Generate a PRAMIN write accessor. +macro_rules! define_pramin_write { + ($name:ident, $ty:ty) => { +#[doc = concat! ( "Write a `" , stringify! ($ty), "` to VRAM at the given offset." )] +pub (crate )fn $name(&mut self, vram_offset:usize , value: $ty) ->Result { +let (bar_offset, new_base) = + self.compute_window (vram_offset, ::core::mem::size_of ::<$ty>())?; + +if let Some (base) = new_base { +Self ::write_window_base (&self.bar, base)?; + *self.state = base; + } + self.bar.$name(value, bar_offset)
drivers/gpu/nova-core/mm/pramin.rs line 170
Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10
-0+ } + }; +} + +/// PRAMIN aperture manager. +/// +/// Call [`Pramin::get_window()`] to acquire exclusive PRAMIN access. +#[pin_data] +pub (crate )struct Pramin { + bar:Arc <Devres <Bar0 >>,
The HRT series [1] will allow you to greatly simplify all this by storing a &'a Bar0 directly in this structure. It will most likely land this cycle, so I think it's a good idea to proactively depend on them. It should apply cleanly (modulo the Tyr patches IIRC, but you can skip them if you don't build the driver) on top of drm-rust-next - on top of which this should also be rebased anyway.
drivers/gpu/nova-core/mm/pramin.rs line 172
Quoted 2 lines
drivers/gpu/nova-core/mm/pramin.rs new file +2
-0+/// Valid VRAM region. Accesses outside this range are rejected. + vram_region: Range<u64>,
I don't think that checking the range in this type is particularly useful, because this abstraction cannot validate VRAM ownership
completely: VRAM can be sparse, reserved, or protected, and with the current bound access to these invalid areas is not prevented.
PRAMIN should only model the hardware windowing mechanism; address validity belongs to the higher layers that allocate or obtain those addresses. IIUC nothing will explode if we try to read or write into areas that are not VRAM.
drivers/gpu/nova-core/mm/pramin.rs line 182
Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10
-0+/// PRAMIN aperture state, protected by a mutex. +/// +/// # Invariants +/// +/// This lock is acquired during the DMA fence signaling critical path. +/// It must NEVER be held across any reclaimable CPU memory / allocations +/// (`GFP_KERNEL`), because the memory reclaim path can call +/// `dma_fence_wait()`, which would deadlock with this lock held. +#[pin] + state: Mutex<u64>,
The problem with this lock is that it is buried, alongside its usage guide, deep into this type, ensuring that users won't ever see it. It will create deadlocks rather than preventing them.
And when you look closer at it, you realize it is actually acquired for any use of Pramin, since the only method it exposes is get_window, which acquires the lock, and returns a PraminWindow which has exactly the same layout as Pramin, except that the lock is acquired.
So what's the point? Let's just make get_window require a &mut self and let the owner of Pramin decide how to manage concurrent accesses. At least the doccomment for the lock will be visible from a higher layer.
Currently Pramin mixes elements of hardware access, memory management, and synchronization into the same type, squashing what should be different layers into a single one. Now that the Io trait is merged and available in drm-rust-next, it is a good time to implement the feedback I gave on v8. Roughly:
- Pramin should just be the owner and arbiter of the NV_PBUS_BAR0_WINDOW register. Its `get_window(&mut self,
window_base: Bounded<u64, 24>) method simply returns a window to the 1MB area starting from base << 16`. It's the simplest possible abstraction.
- The returned window implements Io and IoKnownSize. This will allow users to do all the fancy stuff that comes with Io, including projections.
This API will also be perfect for cases like e.g. setting a memory page to zero: just give the start of the page to get_window, zap the first 4K of the window, done.
- Random accesses to VRAM, which are needed for walking the page tables and updating them, can be built on top of this simple API. Either by adding read/write ops directly to Pramin, or by defining another type that owns a &mut Pramin and moves the window automatically if the next access is out of bounds. This should result in window management code that is simpler than the current compute_window.
That's really all we need. Then GpuMm can manage concurrency similarly to the current model by wrapping Pramin inside a Mutex, and is also free to experiment with different locking strategies, something the current design doesn't allow.
Quoted 29 lines
drivers/gpu/nova-core/mm/pramin.rs new file +29
-0+} + +impl Pramin { +/// Create a pin-initializer for PRAMIN. +/// +/// `vram_region` specifies the valid VRAM address range. +pub (crate )fn new ( + bar:Arc <Devres <Bar0 >>, + vram_region:Range <u64 >, + ) ->Result <impl PinInit <Self >> { +let bar_access = bar.try_access ().ok_or (ENODEV )?; +let current_base =Self ::read_window_base (&bar_access); + +Ok (pin_init! (Self { + bar, + vram_region, + state <-new_mutex! (current_base,"pramin_state" ), + })) + } + +/// Acquire exclusive PRAMIN access. +/// +/// Returns a [`PraminWindow`] guard that provides VRAM read/write accessors. +/// The [`PraminWindow`] is exclusive and only one can exist at a time. +pub (crate )fn get_window (&self) ->Result <PraminWindow <'_>> { +let bar = self.bar.try_access ().ok_or (ENODEV )?; +let state = self.state.lock (); +Ok (PraminWindow { + bar,
drivers/gpu/nova-core/mm/pramin.rs line 221
Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10
-0+ vram_region: self.vram_region.clone(), + state, + }) + } + +/// Read the current window base from the BAR0_WINDOW register. +fn read_window_base (bar: &Bar0 ) ->u64 { +let reg = bar.read (regs::NV_PBUS_BAR0_WINDOW ); + +// TODO: Convert to Bounded<u64, 40> when available.
It's available. :)
drivers/gpu/nova-core/mm/pramin.rs line 223
Quoted 2 lines
drivers/gpu/nova-core/mm/pramin.rs new file +2
-0+ u64::from (reg.window_base ()) <<16 + }
I think this method (and write_window_base) should be methods of the NV_PBUS_BAR0_WINDOW register (window_base and set_window_base)
Quoted 26 lines
drivers/gpu/nova-core/mm/pramin.rs new file +26
-0+} + +/// PRAMIN window guard for direct VRAM access. +/// +/// This guard holds exclusive access to the PRAMIN aperture. The window auto-repositions +/// when accessing VRAM offsets outside the current 1MB range. +/// +/// Only one [`PraminWindow`] can exist at a time per [`Pramin`] instance (enforced by the +/// internal `MutexGuard`). +pub (crate )struct PraminWindow <'a> { + bar:RevocableGuard <'a,Bar0 >, + vram_region:Range <u64 >, + state:MutexGuard <'a,u64 >, +} + +impl PraminWindow <'_> { +/// Write a new window base to the BAR0_WINDOW register. +fn write_window_base (bar: &Bar0 , base:u64 ) ->Result { +// CAST: After >> 16, a VRAM address fits in u32. +let window_base = (base >>16 )as u32 ; + bar.write_reg ( + regs::NV_PBUS_BAR0_WINDOW ::zeroed () + .with_target (Bar0WindowTarget ::Vram ) + .try_with_window_base (window_base)?, + ); +Ok (())
drivers/gpu/nova-core/mm/pramin.rs line 259
Quoted 10 lines
drivers/gpu/nova-core/mm/pramin.rs new file +10
-0+ } + +/// Compute window parameters for a VRAM access. +/// +/// Returns (`bar_offset`, `new_base`) where: +/// - `bar_offset`: The BAR0 offset to use for the access. +/// - `new_base`: `Some(base)` if window needs repositioning, `None` otherwise. +fn compute_window ( + &self, + vram_offset: usize,
This should be a u64, since it is a VRAM address.
Actually, I noticed that you introduced a VramAddress type in a latter patch. It might be worth introducing it earlier and using it everywhere a VRAM address is involved, that way we cannot make any mistake - not critical for now, but feel free to do it if you think it helps.
Re: [PATCH v12 04/22] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
Quoted 19 lines
On May 2, 2026, at 11:42 AM, Alexandre Courbot <[email protected]>
drivers/gpu/nova-core/mm/pramin.rs line 170
Quoted 7 lines
drivers/gpu/nova-core/mm/pramin.rs new file +7
-0+ +/// PRAMIN aperture manager. +/// +/// Call [`Pramin::get_window()`] to acquire exclusive PRAMIN access. +#[pin_data] +pub (crate )struct Pramin { + bar:Arc <Devres <Bar0 >>,The HRT series [1] will allow you to greatly simplify all this by storing a
&'a Bar0directly in this structure. It will most likely land this cycle, so I think it's a good idea to proactively depend on them. It should apply cleanly (modulo the Tyr patches IIRC, but you can skip them if you don't build the driver) on top ofdrm-rust-next- on top of which this should also be rebased anyway.
Quoted 2 lines
I tried to apply it but had a lot of conflict.
Danilo, do you have a tree of these patches somewhere that applies on drm-rust-next?
thanks,
-
Joel Fernandes
Alexandre Courbot
maintainerRe: [PATCH v12 04/22] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
Quoted 19 lines · Joel Fernandes
Quoted 19 lines
On May 2, 2026, at 11:42 AM, Alexandre Courbot <[email protected]>
drivers/gpu/nova-core/mm/pramin.rs line 170
Quoted 7 lines
drivers/gpu/nova-core/mm/pramin.rs new file +7
-0+ +/// PRAMIN aperture manager. +/// +/// Call [`Pramin::get_window()`] to acquire exclusive PRAMIN access. +#[pin_data] +pub (crate )struct Pramin { + bar:Arc <Devres <Bar0 >>,The HRT series [1] will allow you to greatly simplify all this by storing a
&'a Bar0directly in this structure. It will most likely land this cycle, so I think it's a good idea to proactively depend on them. It should apply cleanly (modulo the Tyr patches IIRC, but you can skip them if you don't build the driver) on top ofdrm-rust-next- on top of which this should also be rebased anyway.
Quoted 6 lines
Quoted 2 lines
I tried to apply it but had a lot of conflict.
Danilo, do you have a tree of these patches somewhere that applies on drm-rust-next?
I have a tree here, it should build fine as long as Tyr is not enabled:
Re: [PATCH v12 04/22] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
Quoted 23 lines · Alexandre Courbot
Quoted 22 lines · Joel Fernandes
Quoted 21 lines
On May 2, 2026, at 11:42 AM, Alexandre Courbot <[email protected]>
drivers/gpu/nova-core/mm/pramin.rs line 170
Quoted 7 lines
drivers/gpu/nova-core/mm/pramin.rs new file +7
-0+ +/// PRAMIN aperture manager. +/// +/// Call [`Pramin::get_window()`] to acquire exclusive PRAMIN access. +#[pin_data] +pub (crate )struct Pramin { + bar:Arc <Devres <Bar0 >>,The HRT series [1] will allow you to greatly simplify all this by storing a
&'a Bar0directly in this structure. It will most likely land this cycle, so I think it's a good idea to proactively depend on them. It should apply cleanly (modulo the Tyr patches IIRC, but you can skip them if you don't build the driver) on top ofdrm-rust-next- on top of which this should also be rebased anyway.
Quoted 7 lines
Quoted 3 lines
I tried to apply it but had a lot of conflict.
Danilo, do you have a tree of these patches somewhere that applies on drm-rust-next?
I have a tree here, it should build fine as long as Tyr is not enabled:
Thank you Alex, I will give this a try.
Re: [PATCH v12 04/22] gpu: nova-core: mm: Add support to use PRAMIN windows to write to VRAM
In reply to Alexandre CourbotHi Alex,
Quoted 1 line · Alexandre Courbot
Pramin::newtakes different arguments (also check the other examples).
Fixed.
Quoted 8 lines
The HRT series [1] will allow you to greatly simplify all this by storing a
&'a Bar0directly in this structure. It will most likely land this cycle, so I think it's a good idea to proactively depend on them. It should apply cleanly (modulo the Tyr patches IIRC, but you can skip them if you don't build the driver) on top ofdrm-rust-next- on top of which this should also be rebased anyway.
Actually, I was going in the direction of making Bar0, GpuMm and Bar1 use Arc instead of lifetimes initially. The reason is, some of these are long lived references. Directly using &'a Bar0 means the lifetimes also gets threaded to Pramin and then threads to GpuMm which is long lived in some of my later patches. Some of the cases where we will have long lived mappings are also vGPU and channels. I spoke to Danilo about this, and we discussed that an initial approach could just use Arc since refcounts are cheap and it should be just as performant, then once we have the design ironed out, we can migrate to using lifetimes in this code where appropriate. Even before HRT, I already used lifetimes and analyzed the tradeoffs.
Quoted 4 lines
I don't think that checking the range in this type is particularly useful, because this abstraction cannot validate VRAM ownership
completely: VRAM can be sparse, reserved, or protected, and with the current bound access to these invalid areas is not prevented.
I think there's value in keeping this. The range check is similar in spirit to what Bar0 accessors do -- bounds are checked, but a passing check doesn't guarantee the IO will succeed at the hardware level (an address in a valid range could still cause issues). That's not a reason to remove the guard; it still catches obvious bugs early. Having a sanity check that says "this address is at least within the declared VRAM region" is more useful than having no check at all, even if ownership semantics aren't enforced at this layer.
Quoted 8 lines
PRAMIN should only model the hardware windowing mechanism; address validity belongs to the higher layers that allocate or obtain those addresses. IIUC nothing will explode if we try to read or write into areas that are not VRAM.
The problem with this lock is that it is buried, alongside its usage guide, deep into this type, ensuring that users won't ever see it. It will create deadlocks rather than preventing them.
Having an internal lock isn't inherently problematic -- the GPU buddy allocator bindings follow the same pattern, for example. Let us go by usecase, lets not complicate design based on a hypothetical. I wouldn't move the mutex out of Pramin just based on guessing and complicate/thread the mutable reference to layers that don't need to handle that complexity.
Further, what concrete design or use case requires mutex to be external? I wouldn't add that complexity based on hypotheticals -- we go by actual use cases, and there are actual users of this code in the patches that follow. For what it's worth, I did try the outer-mutex approach myself before you mentioned it and decided against it for this and the buddy code based on the reasons mentioned.
Quoted 9 lines
And when you look closer at it, you realize it is actually acquired for any use of
Pramin, since the only method it exposes isget_window, which acquires the lock, and returns aPraminWindowwhich has exactly the same layout asPramin, except that the lock is acquired.So what's the point? Let's just make
get_windowrequire a&mut selfand let the owner ofPramindecide how to manage concurrent accesses. At least the doccomment for the lock will be visible from a higher layer.
The point is to keep it simple. What you're suggesting requires the caller to do: lock + get_window + access window. My design is get_window
+ access window -- you need the latter two anyway, so the locking is absorbed without any extra burden. Propagating lock outward adds complexity at every layer that calls into Pramin; handling the lock internally, as done elsewhere in the codebase, avoids that.
Quoted 21 lines
Currently
Praminmixes elements of hardware access, memory management, and synchronization into the same type, squashing what should be different layers into a single one. Now that theIotrait is merged and available indrm-rust-next, it is a good time to implement the feedback I gave on v8. Roughly:-
Praminshould just be the owner and arbiter of theNV_PBUS_BAR0_WINDOWregister. Its `get_window(&mut self,
window_base: Bounded<u64, 24>)method simply returns a window to the 1MB area starting frombase << 16`. It's the simplest possible abstraction.
- The returned window implementsIoandIoKnownSize. This will allow users to do all the fancy stuff that comes withIo, including projections.
- Random accesses to VRAM, which are needed for walking the page tables and updating them, can be built on top of this simple API. Either by adding read/write ops directly toPramin, or by defining another type that owns a&mut Praminand moves the window automatically if the next access is out of bounds. This should result in window management code that is simpler than the currentcompute_window.
Quoted 4 lines
That's really all we need. Then
GpuMmcan manage concurrency similarly to the current model by wrappingPramininside aMutex, and is also free to experiment with different locking strategies, something the current design doesn't allow.
I am looking into implementing Io/IoKnownSize on PraminWindow -- that part makes sense independently. The Io impl would operate on the current fixed 1MB window view; the auto-repositioning methods stay as a there is a usecase (more later). None of this requires the locking changes.
I'm familiar with the v8 feedback, but I disagree with parts of it. The current design isn't complex -- it's the external locking + split into multiple types approach that I think would make it more complex. I'm adopting the parts of your suggestions that I agreed with.
The auto-repositioning is actively used and not hypothetical. install_mappings (mm/pagetable/map.rs) holds a single PraminWindow while writing PDEs at scattered buddy-allocated VRAM addresses and then walking up to 4 PDE levels per VFN -- each level a separately-allocated 4KB page that can be many MB apart. invalidate_ptes similarly holds one window across all VFNs: for a range mapping more than ~256 PT pages the PTE region alone spans > 1MB. Removing auto-repositioning would push that window arithmetic onto every caller.
Quoted 3 lines
Quoted 1 line
+ // TODO: Convert to Bounded<u64, 40> when available.
It's available. :)
Changed to use Bounded, thanks for that primitive.
Quoted 2 lines
I think this method (and
write_window_base) should be methods of theNV_PBUS_BAR0_WINDOWregister (window_baseandset_window_base)
Agreed, I will move it to the registers.
Quoted 8 lines
Quoted 1 line
+
vram_offset: usize,This should be a
u64, since it is a VRAM address.Actually, I noticed that you introduced a
VramAddresstype in a latter patch. It might be worth introducing it earlier and using it everywhere a VRAM address is involved, that way we cannot make any mistake - not critical for now, but feel free to do it if you think it helps.
Cool, I will look into it. Sounds reasonable to do.
thanks,
Signature · 1 line
Joel Fernandes
PATCH v12 5/22 docs: gpu: nova-core: Document the PRAMIN aperture mechanism
In reply to Joel FernandesAdd documentation for the PRAMIN aperture mechanism used by nova-core for direct VRAM access.
Nova only uses TARGET=VRAM for VRAM access. The SYS_MEM target values are documented for completeness but not used by the driver.
--- Documentation/gpu/nova/core/pramin.rst | 123 +++++++++++++++++++++++++ Documentation/gpu/nova/index.rst | 1 + 2 files changed, 124 insertions(+) create mode 100644 Documentation/gpu/nova/core/pramin.rst
Documentation/gpu/nova/core/pramin.rst new file +123 -0
@@ -0,0 +1,123 @@ +.. SPDX-License-Identifier: GPL-2.0 + +========================= +PRAMIN aperture mechanism +========================= + +.. note:: + The following description is approximate and current as of the Ampere family. + It may change for future generations and is intended to assist in understanding + the driver code. + +Introduction +============ + +PRAMIN is a hardware aperture mechanism that provides CPU access to GPU Video RAM (VRAM) before +the GPU's Memory Management Unit (MMU) and page tables are initialized. This 1MB sliding window, +located at a fixed offset within BAR0, is essential for setting up page tables and other critical +GPU data structures without relying on the GPU's MMU. + +Architecture Overview +===================== + +The PRAMIN aperture mechanism is logically implemented by the GPU's PBUS (PCIe Bus Controller Unit) +and provides a CPU-accessible window into VRAM through the PCIe interface:: + + +-----------------+ PCIe +------------------------------+ + | CPU |<----------->| GPU | + +-----------------+ | | + | +----------------------+ | + | | PBUS | | + | | (Bus Controller) | | + | | | | + | | +--------------+<------------ (window starts at + | | | PRAMIN | | | BAR0 + 0x700000) + | | | Window | | | + | | | (1MB) | | | + | | +--------------+ | | + | | | | | + | +---------|------------+ | + | | | + | v | + | +----------------------+<------------ (Program PRAMIN to any + | | VRAM | | 64KB-aligned VRAM boundary) + | | (Several GBs) | | + | | | | + | | FB[0x0000000000] | | + | | ... | | + | | FB[0xFFFFFFFFFF] | | + | +----------------------+ | + +------------------------------+ + +PBUS (PCIe Bus Controller) is responsible for, among other things, handling MMIO +accesses to the BAR registers. + +PRAMIN Window Operation +======================= + +The PRAMIN window provides a 1MB sliding aperture that can be repositioned over +the entire VRAM address space using the ``NV_PBUS_BAR0_WINDOW`` register. + +Window Control Mechanism +------------------------- + +:: + + NV_PBUS_BAR0_WINDOW Register (0x1700): + +-------+--------+--------------------------------------+ + | 31:26 | 25:24 | 23:0 | + | RSVD | TARGET | BASE_ADDR | + | | | (bits 39:16 of VRAM address) | + +-------+--------+--------------------------------------+ + + The 24-bit BASE_ADDR field encodes bits [39:16] of the target VRAM address, + providing 40-bit (1TB) address space coverage with 64KB alignment. + + TARGET field (bits 25:24): + - 0x0: VRAM (Video Memory) + - 0x1: SYS_MEM_COH (Coherent System Memory) + - 0x2: SYS_MEM_NONCOH (Non-coherent System Memory) + - 0x3: Reserved + +.. note:: + Nova only uses TARGET=VRAM (0x0) for video memory access. The SYS_MEM + target values are documented here for hardware completeness but are + not used by the driver. + +64KB Alignment Requirement +--------------------------- + +The PRAMIN window must be aligned to 64KB boundaries in VRAM. This is enforced +by the ``BASE_ADDR`` field representing bits [39:16] of the target address:: + + VRAM Address Calculation: + actual_vram_addr = (BASE_ADDR << 16) + pramin_offset + Where: + - BASE_ADDR: 24-bit value from NV_PBUS_BAR0_WINDOW[23:0] + - pramin_offset: 20-bit offset within the PRAMIN window [0x00000-0xFFFFF] + + Example Window Positioning: + +---------------------------------------------------------+ + | VRAM Space | + | | + | 0x0000000000 +-----------------+ <-- 64KB aligned | + | | PRAMIN Window | | + | | (1MB) | | + | 0x00000FFFFF +-----------------+ | + | | + | | ^ | + | | | Window can slide | + | v | to any 64KB-aligned boundary | + | | + | 0x0123400000 +-----------------+ <-- 64KB aligned | + | | PRAMIN Window | | + | | (1MB) | | + | 0x01234FFFFF +-----------------+ | + | | + | ... | + | | + | 0xFFFFF00000 +-----------------+ <-- 64KB aligned | + | | PRAMIN Window | | + | | (1MB) | | + | 0xFFFFFFFFFF +-----------------+ | + +---------------------------------------------------------+
Documentation/gpu/nova/index.rst +1 -0
@@ -32,3 +32,4 @@ vGPU manager VFIO driver and the nova-drm driver. core/devinit core/fwsec core/falcon + core/pramin
Signature · 1 line
2.34.1
PATCH v12 6/22 gpu: nova-core: mm: Add common memory management types
In reply to Joel FernandesAdd foundational types for GPU memory management. These types are used throughout the nova memory management subsystem for page table operations, address translation, and memory allocation.
--- drivers/gpu/nova-core/mm.rs | 193 ++++++++++++++++++++++++++++++++++++ 1 file changed, 193 insertions(+)
drivers/gpu/nova-core/mm.rs +193 -0
@@ -2,4 +2,197 @@//! Memory management subsystems for nova-core. +#![expect(dead_code)] + +/// Implements `From` conversions between a frame-number type ([`Pfn`] or [`Vfn`]) +/// and `Bounded<u64, N>` for bitfield interop. +/// +/// Each MMU version module should invoke this for the specific bit widths used by that version's +/// PTE/PDE bitfield definitions. +macro_rules! impl_frame_number_bounded { + ($type:ty, $bits:literal) => { +impl From <Bounded <u64 , $bits>>for $type { +fn from (val:Bounded <u64 , $bits>) ->Self { +Self ::new (val.get ()) + } + } + +impl From <$type>for Bounded <u64 , $bits> { +fn from (v: $type) ->Self { +Bounded ::from_expr (v.raw () & ::kernel::bits::genmask_u64 (0 ..=($bits -1 ))) + } + } + }; +} + +/// Implements `From` conversions between [`Pfn`] and `Bounded<u64, N>` for bitfield interop. +macro_rules! impl_pfn_bounded { + ($bits:literal) => { +impl_frame_number_bounded! (Pfn , $bits); + }; +} +pub (crate )mod pramin; + +use kernel::{ + bitfield, + num::Bounded , + prelude::*, + sizes::SZ_4K ,// +}; + +use crate ::num::u64_as_usize; + +/// Page size in bytes (4 KiB). +pub (crate )const PAGE_SIZE :usize =SZ_4K ; + +bitfield! { +/// Physical VRAM address in GPU video memory. +pub (crate )struct VramAddress (u64 ) { +/// Offset within 4KB page. +11 :0 offset; +/// Physical frame number. +63 :12 frame_number =>Pfn ; + } +} + +impl VramAddress { +/// Create a new VRAM address from a raw value. +pub (crate )const fn new (addr:u64 ) ->Self { +Self ::from_raw (addr) + } + +/// Get the raw address value as `usize` (useful for MMIO offsets). +pub (crate )const fn raw (&self) ->usize { +u64_as_usize (self.into_raw ()) + } + +/// Get the raw address value as `u64`. +pub (crate )const fn raw_u64 (&self) ->u64 { + self.into_raw () + } +} + +impl PartialOrd for VramAddress { +fn partial_cmp (&self, other: &Self ) ->Option <core::cmp::Ordering > { +Some (self.cmp (other)) + } +} + +impl Ord for VramAddress { +fn cmp (&self, other: &Self ) -> core::cmp::Ordering { + self.into_raw ().cmp (&other.into_raw ()) + } +} + +impl From <Pfn >for VramAddress { +fn from (pfn:Pfn ) ->Self { +Self ::zeroed ().with_frame_number (pfn) + } +} + +bitfield! { +/// Virtual address in GPU address space. +pub (crate )struct VirtualAddress (u64 ) { +/// Offset within 4KB page. +11 :0 offset; +/// Virtual frame number. +63 :12 frame_number =>Vfn ; + } +} + +impl VirtualAddress { +/// Create a new virtual address from a raw value. +pub (crate )const fn new (addr:u64 ) ->Self { +Self ::from_raw (addr) + } + +/// Get the raw address value as `u64`. +pub (crate )const fn raw_u64 (&self) ->u64 { + self.into_raw () + } +} + +impl From <Vfn >for VirtualAddress { +fn from (vfn:Vfn ) ->Self { +Self ::zeroed ().with_frame_number (vfn) + } +} + +/// Physical Frame Number. +/// +/// Represents a physical page in VRAM. +#[repr(transparent)] +#[derive( Debug , Clone , Copy , PartialEq , Eq , Default )] +pub (crate )struct Pfn (u64 ); + +impl Pfn { +/// Create a new PFN from a frame number. +pub (crate )const fn new (frame_number:u64 ) ->Self { +Self (frame_number) + } + +/// Get the raw frame number. +pub (crate )const fn raw (self) ->u64 { + self.0 + } +} + +impl From <VramAddress >for Pfn { +fn from (addr:VramAddress ) ->Self { + addr.frame_number () + } +} + +impl From <u64 >for Pfn { +fn from (val:u64 ) ->Self { +Self (val) + } +} + +impl From <Pfn >for u64 { +fn from (pfn:Pfn ) ->Self { + pfn.0 + } +} + +impl_pfn_bounded! (52 ); + +/// Virtual Frame Number. +/// +/// Represents a virtual page in GPU address space. +#[repr(transparent)] +#[derive( Debug , Clone , Copy , PartialEq , Eq , Default )] +pub (crate )struct Vfn (u64 ); + +impl Vfn { +/// Create a new VFN from a frame number. +pub (crate )const fn new (frame_number:u64 ) ->Self { +Self (frame_number) + } + +/// Get the raw frame number. +pub (crate )const fn raw (self) ->u64 { + self.0 + } +} + +impl From <VirtualAddress >for Vfn { +fn from (addr:VirtualAddress ) ->Self { + addr.frame_number () + } +} + +impl From <u64 >for Vfn { +fn from (val:u64 ) ->Self { +Self (val) + } +} + +impl From <Vfn >for u64 { +fn from (vfn:Vfn ) ->Self { + vfn.0 + } +} + +impl_frame_number_bounded! (Vfn ,52 );
Signature · 1 line
2.34.1
PATCH v12 7/22 gpu: nova-core: mm: Add TLB flush support
In reply to Joel FernandesAdd TLB (Translation Lookaside Buffer) flush support for GPU MMU.
After modifying page table entries, the GPU's TLB must be invalidated to ensure the new mappings take effect. The Tlb struct provides flush functionality through BAR0 registers.
The flush operation writes the page directory base address and triggers an invalidation, polling for completion with a 2 second timeout.
--- drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/tlb.rs | 130 ++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/regs.rs | 69 ++++++++++++++++- 3 files changed, 199 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/nova-core/mm/tlb.rs
drivers/gpu/nova-core/mm.rs +1 -0
@@ -33,6 +33,7 @@ macro_rules! impl_pfn_bounded { }pub (crate )mod pramin; +pub (super )mod tlb;use kernel::{ bitfield,
drivers/gpu/nova-core/mm/tlb.rs new file +130 -0
130 lines not on this page · raw message
drivers/gpu/nova-core/regs.rs +68 -1
78 lines not on this page · raw message
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 07/22] gpu: nova-core: mm: Add TLB flush support
On Sun Apr 26, 2026 at 6:14 AM JST, Joel Fernandes wrote:
<snip>
Quoted 20 lines
drivers/gpu/nova-core/regs.rs +20
-0+/// TLB flush control register. + pub(crate) NV_TLB_FLUSH_CTRL(u32) @ 0x00b830b0 { +/// Invalidate every VA in the PDB selected by `NV_TLB_FLUSH_PDB_LO/HI`. + 0:0 all_va => bool; +/// Invalidate TLBs for all PDBs (ignores `NV_TLB_FLUSH_PDB_LO/HI`). + 1:1 all_pdb => bool; +/// Restrict the flush to the HUB MMU's TLBs; skip broadcasting to the +/// per-GPC L2 TLBs. +/// +/// The GPU MMU has a two-level TLB hierarchy: +/// 1. The *HUB MMU* sits at the top and serves memory requests from +/// "host-side" engines: the host/channel interface, copy engines, +/// display, and BAR1/BAR2 accesses. +/// 2. Each GPC (Graphics Processing Cluster — the block that houses +/// shader cores / SMs) has its own L2 TLB that serves requests from +/// the compute and graphics engines inside the cluster. +/// +/// When set, only the HUB TLBs are invalidated. This is a performance +/// optimization for flushes that only affect HUB-side mappings (e.g. +/// BAR1/BAR2 windows), where fanning the invalidation out to every
drivers/gpu/nova-core/regs.rs pub(crate) mod ga100 { · line 599
Quoted 10 lines
drivers/gpu/nova-core/regs.rs +10
-0+/// GPC's L2 TLB would be wasted work. Must be false when flushing +/// mappings that may be cached by compute/graphics engines. + 2:2 hubtlb_only => bool; +/// Invalidation acknowledgment scope. See [`TlbAckMode`] for details. + 8:7 ack ?=> TlbAckMode; +/// Write 1 to kick off the flush. Hardware clears this bit when the +/// flush completes; reads as 1 while the flush is in progress. + 31:31 trigger => bool; + } +}
Registers should be defined with their upper fields first (i.e. 31:31 on top).
PATCH v12 8/22 gpu: nova-core: mm: Add GpuMm centralized memory manager
In reply to Joel FernandesIntroduce GpuMm as the centralized GPU memory manager that owns:
- Buddy allocator for VRAM allocation.
- PRAMIN window for direct VRAM access.
- TLB manager for translation buffer operations.
This provides clean ownership model where GpuMm provides accessor methods for its components that can be used for memory management operations.
--- drivers/gpu/nova-core/Kconfig | 1 + drivers/gpu/nova-core/gpu.rs | 28 +++++++++++ drivers/gpu/nova-core/gsp/commands.rs | 1 - drivers/gpu/nova-core/mm.rs | 69 ++++++++++++++++++++++++++- drivers/gpu/nova-core/mm/pramin.rs | 20 ++++---- 5 files changed, 108 insertions(+), 11 deletions(-)
drivers/gpu/nova-core/Kconfig +1 -0
@@ -4,6 +4,7 @@ config NOVA_CORE depends on PCI depends on RUST select AUXILIARY_BUS + select GPU_BUDDY select RUST_FW_LOADER_ABSTRACTIONS default n help
drivers/gpu/nova-core/gpu.rs +28 -0
@@ -4,10 +4,13 @@ device, devres::Devres, fmt, + gpu::buddy::GpuBuddyParams, io::Io, num::Bounded, pci, prelude::*, + ptr::Alignment, + sizes::SZ_4K, sync::Arc, // };
@@ -25,6 +28,7 @@ commands::GetGspStaticInfoReply, Gsp, // }, + mm::GpuMm, regs, };
@@ -238,6 +242,9 @@ pub(crate) struct Gpu { gsp_falcon: Falcon<GspFalcon>, /// SEC2 falcon instance, used for GSP boot up and cleanup. sec2_falcon: Falcon<Sec2Falcon>, +/// GPU memory manager owning memory management resources. +#[pin] + mm: GpuMm,/// GSP runtime data. Temporarily an empty placeholder. #[pin] gsp: Gsp,
@@ -276,6 +283,13 @@ pub(crate) fn new<'a>( gsp_static_info: { let info = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)?; +dev_info! ( + pdev.as_ref (), +"Using FB region: {:#x}..{:#x}\n" , + info.usable_fb_region.start, + info.usable_fb_region.end + ); + dev_info!( pdev.as_ref(), "Total physical VRAM: {} MiB\n",
@@ -285,6 +299,20 @@ pub(crate) fn new<'a>( info }, +// Create GPU memory manager owning memory management resources. + mm <- { +let usable_vram = &gsp_static_info.usable_fb_region; + +// PRAMIN covers all physical VRAM (including GSP-reserved areas +// above the usable region, e.g. the BAR1 page directory). +let pramin_vram_region =0 ..gsp_static_info.total_fb_end; +GpuMm ::new (devres_bar.clone (), pdev.as_ref (),GpuBuddyParams { + base_offset: usable_vram.start, + size: usable_vram.end - usable_vram.start, + chunk_size:Alignment ::new ::<SZ_4K >(), + }, pramin_vram_region)? + }, + bar: devres_bar, }) }
drivers/gpu/nova-core/gsp/commands.rs +0 -1
@@ -194,7 +194,6 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {pub (crate )struct GetGspStaticInfoReply { gpu_name: [u8 ;64 ],/// Usable FB (VRAM) region for driver memory allocation. -#[expect(dead_code)] pub (crate ) usable_fb_region:Range <u64 >,/// End of VRAM. pub (crate ) total_fb_end:u64 ,
drivers/gpu/nova-core/mm.rs +67 -2
79 lines not on this page · raw message
drivers/gpu/nova-core/mm/pramin.rs +12 -8
@@ -94,12 +94,12 @@ };use kernel::{ + device, devres::Devres , io::Io , new_mutex, num::Bounded , prelude::*, - revocable::RevocableGuard , sizes::{SZ_1M ,SZ_64K ,//
@@ -137,7 +137,7 @@ pub(crate) fn $name(&mut self, vram_offset: usize) -> Result<$ty> { self.compute_window (vram_offset, ::core::mem::size_of ::<$ty>())?;if let Some (base) = new_base { -Self ::write_window_base (&self.bar, base)?; +Self ::write_window_base (self.bar, base)?; *self.state = base; } self.bar.$name(bar_offset)
@@ -154,7 +154,7 @@ pub(crate) fn $name(&mut self, vram_offset: usize, value: $ty) -> Result { self.compute_window (vram_offset, ::core::mem::size_of ::<$ty>())?;if let Some (base) = new_base { -Self ::write_window_base (&self.bar, base)?; +Self ::write_window_base (self.bar, base)?; *self.state = base; } self.bar.$name(value, bar_offset)
@@ -188,10 +188,11 @@ impl Pramin {/// `vram_region` specifies the valid VRAM address range. pub (crate )fn new ( bar:Arc <Devres <Bar0 >>, + dev: &device::Device <device::Bound >, vram_region:Range <u64 >, ) ->Result <impl PinInit <Self >> { -let bar_access = bar.try_access ().ok_or (ENODEV )?; -let current_base =Self ::read_window_base (&bar_access); +let bar_access = bar.access (dev)?; +let current_base =Self ::read_window_base (bar_access); Ok(pin_init!(Self { bar,
@@ -204,8 +205,11 @@ pub(crate) fn new(/// /// Returns a [`PraminWindow`] guard that provides VRAM read/write accessors. /// The [`PraminWindow`] is exclusive and only one can exist at a time. -pub (crate )fn get_window (&self) ->Result <PraminWindow <'_>> { -let bar = self.bar.try_access ().ok_or (ENODEV )?; +pub (crate )fn get_window <'a>( + &'a self, + dev: &'a device::Device <device::Bound >, + ) ->Result <PraminWindow <'a>> { +let bar = self.bar.access (dev)?;let state = self.state.lock ();Ok (PraminWindow { bar,
@@ -231,7 +235,7 @@ fn read_window_base(bar: &Bar0) -> u64 {/// Only one [`PraminWindow`] can exist at a time per [`Pramin`] instance (enforced by the /// internal `MutexGuard`). pub (crate )struct PraminWindow <'a> { - bar:RevocableGuard <'a,Bar0 >, + bar: &'aBar0 , vram_region:Range <u64 >, state:MutexGuard <'a,u64 >, }
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 08/22] gpu: nova-core: mm: Add GpuMm centralized memory manager
Quoted 9 lines · Joel Fernandes
Introduce GpuMm as the centralized GPU memory manager that owns:
- Buddy allocator for VRAM allocation.
- PRAMIN window for direct VRAM access.
- TLB manager for translation buffer operations.This provides clean ownership model where GpuMm provides accessor methods for its components that can be used for memory management operations.
Quoted 8 lines
Cc Nikola Djukic <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/Kconfig | 1 + drivers/gpu/nova-core/gpu.rs | 28 +++++++++++ drivers/gpu/nova-core/gsp/commands.rs | 1 - drivers/gpu/nova-core/mm.rs | 69 ++++++++++++++++++++++++++- drivers/gpu/nova-core/mm/pramin.rs | 20 ++++----
Why does this need to edit pramin.rs? This is the first user, so can't we give pramin its useful form from the start? These changes on pramin.rs apply perfectly fine on top of patch 4, so there is no reason not to squash them there.
Reviewers read patch 4, build a mental model of it and write feedback, and this model is immediately altered without even being used. Please don't do that.
PATCH v12 9/22 gpu: nova-core: mm: Add common types for all page table formats
In reply to Joel FernandesAdd common page table types shared between MMU v2 and v3. These types are hardware-agnostic and used by both MMU versions.
--- drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/pagetable.rs | 157 ++++++++++++++++++++++++++ 2 files changed, 158 insertions(+) create mode 100644 drivers/gpu/nova-core/mm/pagetable.rs
drivers/gpu/nova-core/mm.rs +1 -0
@@ -32,6 +32,7 @@ macro_rules! impl_pfn_bounded { }; } +pub (super )mod pagetable;pub (crate )mod pramin;pub (super )mod tlb;
drivers/gpu/nova-core/mm/pagetable.rs new file +157 -0
157 lines not on this page · raw message
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 09/22] gpu: nova-core: mm: Add common types for all page table formats
Quoted 54 lines · Joel Fernandes
Add common page table types shared between MMU v2 and v3. These types are hardware-agnostic and used by both MMU versions.
Cc Nikola Djukic <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/pagetable.rs | 157 ++++++++++++++++++++++++++ 2 files changed, 158 insertions(+) create mode 100644 drivers/gpu/nova-core/mm/pagetable.rsdrivers/gpu/nova-core/mm.rs +1
-0@@ -32,6 +32,7 @@ macro_rules! impl_pfn_bounded { }; } +pub (super )mod pagetable;pub (crate )mod pramin;pub (super )mod tlb;drivers/gpu/nova-core/mm/pagetable.rs new file +25
-0@@ -0,0 +1,157 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Common page table types shared between MMU v2 and v3. +//! +//! This module provides foundational types used by both MMU versions: +//! - Page table level hierarchy +//! - Memory aperture types for PDEs and PTEs + +#![expect(dead_code)] + +use kernel::num::Bounded ; + +use crate ::gpu::Architecture ; + +/// Extracts the page table index at a given level from a virtual address. +pub (super )trait VaLevelIndex { +/// Return the page table index at `level` for this virtual address. +fn level_index (&self, level:u64 ) ->u64 ; +} + +/// MMU version enumeration. +#[derive( Debug , Clone , Copy , PartialEq , Eq )] +pub (crate )enum MmuVersion { +/// MMU v2 for Turing/Ampere/Ada. +V2 ,
drivers/gpu/nova-core/mm/pagetable.rs line 35
Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable.rs new file +10
-0+/// MMU v3 for Hopper and later. + V3, +} + +impl From <Architecture >for MmuVersion { +fn from (arch:Architecture ) ->Self { +match arch { +Architecture ::Turing |Architecture ::Ampere |Architecture ::Ada =>Self ::V2 , +// In the future, uncomment the following to support V3. +// _ => Self::V3,
The architecture definitions for Blackwell are now in drm-rust-next, so I think the next iteration can handle this.
Which reminds me: is V3 working? I remember some fixup by Eliot wandering around, have you integrated it to the series?
Re: [PATCH v12 09/22] gpu: nova-core: mm: Add common types for all page table formats
Quoted 69 lines · Alexandre Courbot
Quoted 54 lines · Joel Fernandes
Add common page table types shared between MMU v2 and v3. These types are hardware-agnostic and used by both MMU versions.
Cc Nikola Djukic <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/pagetable.rs | 157 ++++++++++++++++++++++++++ 2 files changed, 158 insertions(+) create mode 100644 drivers/gpu/nova-core/mm/pagetable.rsdrivers/gpu/nova-core/mm.rs +1
-0@@ -32,6 +32,7 @@ macro_rules! impl_pfn_bounded { }; } +pub (super )mod pagetable;pub (crate )mod pramin;pub (super )mod tlb;drivers/gpu/nova-core/mm/pagetable.rs new file +25
-0@@ -0,0 +1,157 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Common page table types shared between MMU v2 and v3. +//! +//! This module provides foundational types used by both MMU versions: +//! - Page table level hierarchy +//! - Memory aperture types for PDEs and PTEs + +#![expect(dead_code)] + +use kernel::num::Bounded ; + +use crate ::gpu::Architecture ; + +/// Extracts the page table index at a given level from a virtual address. +pub (super )trait VaLevelIndex { +/// Return the page table index at `level` for this virtual address. +fn level_index (&self, level:u64 ) ->u64 ; +} + +/// MMU version enumeration. +#[derive( Debug , Clone , Copy , PartialEq , Eq )] +pub (crate )enum MmuVersion { +/// MMU v2 for Turing/Ampere/Ada. +V2 ,drivers/gpu/nova-core/mm/pagetable.rs line 35
Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable.rs new file +10
-0+/// MMU v3 for Hopper and later. + V3, +} + +impl From <Architecture >for MmuVersion { +fn from (arch:Architecture ) ->Self { +match arch { +Architecture ::Turing |Architecture ::Ampere |Architecture ::Ada =>Self ::V2 , +// In the future, uncomment the following to support V3. +// _ => Self::V3, The architecture definitions for Blackwell are now in
drm-rust-next, so I think the next iteration can handle this.
Quoted 2 lines
Which reminds me: is V3 working? I remember some fixup by Eliot wandering around, have you integrated it to the series?
MMU v3 is working. But this series needs more patches on top of it for Blackwell, I am carrying those extras in a different tree based on this series. The fixes from have all been integrated (either in this series or that other tree of extras).
Have all the base blackwell patches made it into drm-rust-next? If yes, I can rebase this series on it, pull in the extras, and continue posting the next mm iteration with blackwell support.
Thanks.
Alexandre Courbot
maintainerRe: [PATCH v12 09/22] gpu: nova-core: mm: Add common types for all page table formats
Quoted 75 lines · Joel Fernandes
Quoted 69 lines · Alexandre Courbot
Quoted 54 lines · Joel Fernandes
Add common page table types shared between MMU v2 and v3. These types are hardware-agnostic and used by both MMU versions.
Cc Nikola Djukic <[email protected]>Signed-off-by Joel Fernandes <[email protected]>--- drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/pagetable.rs | 157 ++++++++++++++++++++++++++ 2 files changed, 158 insertions(+) create mode 100644 drivers/gpu/nova-core/mm/pagetable.rsdrivers/gpu/nova-core/mm.rs +1
-0@@ -32,6 +32,7 @@ macro_rules! impl_pfn_bounded { }; } +pub (super )mod pagetable;pub (crate )mod pramin;pub (super )mod tlb;drivers/gpu/nova-core/mm/pagetable.rs new file +25
-0@@ -0,0 +1,157 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Common page table types shared between MMU v2 and v3. +//! +//! This module provides foundational types used by both MMU versions: +//! - Page table level hierarchy +//! - Memory aperture types for PDEs and PTEs + +#![expect(dead_code)] + +use kernel::num::Bounded ; + +use crate ::gpu::Architecture ; + +/// Extracts the page table index at a given level from a virtual address. +pub (super )trait VaLevelIndex { +/// Return the page table index at `level` for this virtual address. +fn level_index (&self, level:u64 ) ->u64 ; +} + +/// MMU version enumeration. +#[derive( Debug , Clone , Copy , PartialEq , Eq )] +pub (crate )enum MmuVersion { +/// MMU v2 for Turing/Ampere/Ada. +V2 ,drivers/gpu/nova-core/mm/pagetable.rs line 35
Quoted 10 lines
drivers/gpu/nova-core/mm/pagetable.rs new file +10
-0+/// MMU v3 for Hopper and later. + V3, +} + +impl From <Architecture >for MmuVersion { +fn from (arch:Architecture ) ->Self { +match arch { +Architecture ::Turing |Architecture ::Ampere |Architecture ::Ada =>Self ::V2 , +// In the future, uncomment the following to support V3. +// _ => Self::V3, The architecture definitions for Blackwell are now in
drm-rust-next, so I think the next iteration can handle this.Quoted 2 lines
Which reminds me: is V3 working? I remember some fixup by Eliot wandering around, have you integrated it to the series?
Quoted 8 lines
MMU v3 is working. But this series needs more patches on top of it for Blackwell, I am carrying those extras in a different tree based on this series. The fixes from have all been integrated (either in this series or that other tree of extras).
Have all the base blackwell patches made it into drm-rust-next? If yes, I can rebase this series on it, pull in the extras, and continue posting the next mm iteration with blackwell support.
The architecture definitions are there yes. The full support (as in GSP booting), not yet - Gsp::boot returns ENOTSUPP at the moment. If practical, including support would be nice as it would allow the self-tests to run on Blackwell as soon as the series is merged.
PATCH v12 10/22 gpu: nova-core: mm: Add MMU v2 page table types
In reply to Joel FernandesAdd page table entry and directory structures for MMU version 2 used by Turing/Ampere/Ada GPUs.
--- drivers/gpu/nova-core/mm/pagetable.rs | 2 + drivers/gpu/nova-core/mm/pagetable/ver2.rs | 270 +++++++++++++++++++++ 2 files changed, 272 insertions(+) create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver2.rs
drivers/gpu/nova-core/mm/pagetable.rs +2 -0
@@ -8,6 +8,8 @@#![expect(dead_code)] +pub (super )mod ver2; +use kernel::num::Bounded ;use crate ::gpu::Architecture ;
drivers/gpu/nova-core/mm/pagetable/ver2.rs new file +270 -0
270 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 11/22 gpu: nova-core: mm: Add MMU v3 page table types
In reply to Joel FernandesAdd page table entry and directory structures for MMU version 3 used by Hopper and later GPUs.
--- drivers/gpu/nova-core/mm/pagetable.rs | 1 + drivers/gpu/nova-core/mm/pagetable/ver2.rs | 10 +- drivers/gpu/nova-core/mm/pagetable/ver3.rs | 391 +++++++++++++++++++++ 3 files changed, 397 insertions(+), 5 deletions(-) create mode 100644 drivers/gpu/nova-core/mm/pagetable/ver3.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
@@ -9,6 +9,7 @@#![expect(dead_code)] pub (super )mod ver2; +pub (super )mod ver3;use kernel::num::Bounded ;
drivers/gpu/nova-core/mm/pagetable/ver2.rs +5 -5
@@ -59,11 +59,11 @@ pub(super) fn new(va: VirtualAddress) -> Self {impl VaLevelIndex for VirtualAddressV2 {fn level_index (&self, level:u64 ) ->u64 {match level { -0 => self.pde3_idx (), -1 => self.pde2_idx (), -2 => self.pde1_idx (), -3 => self.pde0_idx (), -4 => self.pt_idx (), +0 => self.pde3_idx ().get (), +1 => self.pde2_idx ().get (), +2 => self.pde1_idx ().get (), +3 => self.pde0_idx ().get (), +4 => self.pt_idx ().get (), _ =>0 , } }
drivers/gpu/nova-core/mm/pagetable/ver3.rs new file +391 -0
391 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 12/22 gpu: nova-core: mm: Add page table entry operation traits
In reply to Joel FernandesIntroduce trait-based abstractions for GPU page table entries: PteOps, PdeOps, and DualPdeOps, along with the MmuConfig trait that ties them together with version-specific constants.
Refactor the ver2 and ver3 page-table modules to implement these traits and expose the shared entry/PDE/PTE operations uniformly.
Dispatch happens at compile time through the MmuV2 and MmuV3 marker structs, so version-specific code is selected without runtime overhead and without wrapper enums.
This enables version-agnostic page table operations while keeping version-specific implementation details encapsulated in the ver2 and ver3 modules.
--- drivers/gpu/nova-core/mm/pagetable.rs | 223 +++++++++++++++++++++ drivers/gpu/nova-core/mm/pagetable/ver2.rs | 150 ++++++++------ drivers/gpu/nova-core/mm/pagetable/ver3.rs | 120 +++++++---- 3 files changed, 396 insertions(+), 97 deletions(-)
drivers/gpu/nova-core/mm/pagetable.rs +223 -0
@@ -14,6 +14,13 @@use kernel::num::Bounded ;use crate ::gpu::Architecture ; +use crate ::mm::{ + pramin, +Pfn , +VirtualAddress , +VramAddress ,// +}; +use kernel::prelude::*;/// Extracts the page table index at a given level from a virtual address. pub (super )trait VaLevelIndex {
@@ -86,6 +93,222 @@ pub(super) const fn as_index(&self) -> u64 { } } +// Trait abstractions for page table operations. + +/// Operations on Page Table Entries (`PTE`s). +pub (super )trait PteOps :Copy + core::fmt::Debug { +/// Create a `PTE` from a raw `u64` value. +fn new (val:u64 ) ->Self ; + +/// Create an invalid `PTE`. +fn invalid () ->Self ; + +/// Create a valid `PTE` for video memory. +fn new_vram (pfn:Pfn , writable:bool ) ->Self ; + +/// Check if this `PTE` is valid. +fn is_valid (&self) ->bool ; + +/// Get the physical frame number. +fn frame_number (&self) ->Pfn ; + +/// Get the raw `u64` value. +fn raw_u64 (&self) ->u64 ; + +/// Read a `PTE` from VRAM. +fn read (window: &mut pramin::PraminWindow <'_>, addr:VramAddress ) ->Result <Self > { +let val = window.try_read64 (addr.raw ())?; +Ok (Self ::new (val)) + } + +/// Write this `PTE` to VRAM. +fn write (&self, window: &mut pramin::PraminWindow <'_>, addr:VramAddress ) ->Result { + window.try_write64 (addr.raw (), self.raw_u64 ()) + } +} + +/// Operations on Page Directory Entries (`PDE`s). +pub (super )trait PdeOps :Copy + core::fmt::Debug { +/// Create a `PDE` from a raw `u64` value. +fn new (val:u64 ) ->Self ; + +/// Create a valid `PDE` pointing to a page table in video memory. +fn new_vram (table_pfn:Pfn ) ->Self ; + +/// Create an invalid `PDE`. +fn invalid () ->Self ; + +/// Check if this `PDE` is valid. +fn is_valid (&self) ->bool ; + +/// Get the memory aperture of this `PDE`. +fn aperture (&self) ->AperturePde ; + +/// Get the VRAM address of the page table. +fn table_vram_address (&self) ->VramAddress ; + +/// Get the raw `u64` value. +fn raw_u64 (&self) ->u64 ; + +/// Read a `PDE` from VRAM. +fn read (window: &mut pramin::PraminWindow <'_>, addr:VramAddress ) ->Result <Self > { +let val = window.try_read64 (addr.raw ())?; +Ok (Self ::new (val)) + } + +/// Write this `PDE` to VRAM. +fn write (&self, window: &mut pramin::PraminWindow <'_>, addr:VramAddress ) ->Result { + window.try_write64 (addr.raw (), self.raw_u64 ()) + } + +/// Check if this `PDE` is valid and points to video memory. +fn is_valid_vram (&self) ->bool { + self.is_valid () && self.aperture () ==AperturePde ::VideoMemory + } +} + +/// Operations on Dual Page Directory Entries (128-bit `DualPde`s). +pub (super )trait DualPdeOps :Copy + core::fmt::Debug { +/// Create a `DualPde` from raw 128-bit value (two `u64`s). +fn new (big:u64 , small:u64 ) ->Self ; + +/// Create a `DualPde` with only the small page table pointer set. +fn new_small (table_pfn:Pfn ) ->Self ; + +/// Check if the small page table pointer is valid. +fn has_small (&self) ->bool ; + +/// Get the small page table VRAM address. +fn small_vram_address (&self) ->VramAddress ; + +/// Get the raw `u64` value of the big PDE. +fn big_raw_u64 (&self) ->u64 ; + +/// Get the raw `u64` value of the small PDE. +fn small_raw_u64 (&self) ->u64 ; + +/// Read a dual PDE (128-bit) from VRAM. +fn read (window: &mut pramin::PraminWindow <'_>, addr:VramAddress ) ->Result <Self > { +let lo = window.try_read64 (addr.raw ())?; +let hi = window.try_read64 (addr.raw () +8 )?; +Ok (Self ::new (lo, hi)) + } + +/// Write this dual PDE (128-bit) to VRAM. +fn write (&self, window: &mut pramin::PraminWindow <'_>, addr:VramAddress ) ->Result { + window.try_write64 (addr.raw (), self.big_raw_u64 ())?; + window.try_write64 (addr.raw () +8 , self.small_raw_u64 ()) + } +} + +/// MMU configuration trait -- encodes version-specific constants and types. +pub (super )trait MmuConfig : 'static { +/// Page Table Entry type. +type Pte :PteOps ; +/// Page Directory Entry type. +type Pde :PdeOps ; +/// Dual Page Directory Entry type (128-bit). +type DualPde :DualPdeOps ; + +/// PDE levels (excluding PTE level) for page table walking. +const PDE_LEVELS : &'static [PageTableLevel ]; +/// PTE level for this MMU version. +const PTE_LEVEL :PageTableLevel ; +/// Dual PDE level (128-bit entries) for this MMU version. +const DUAL_PDE_LEVEL :PageTableLevel ; + +/// Get the number of entries per page table page for a given level. +fn entries_per_page (level:PageTableLevel ) ->usize ; + +/// Extract the page table index at `level` from `va`. +fn level_index (va:VirtualAddress , level:u64 ) ->u64 ; + +/// Get the entry size in bytes for a given level. +fn entry_size (level:PageTableLevel ) ->usize { +if level ==Self ::DUAL_PDE_LEVEL { +16 // 128-bit dual PDE + }else { +8 // 64-bit PDE/PTE + } + } + +/// Compute upper bound on page table pages needed for `num_virt_pages`. +/// +/// Walks from PTE level up through PDE levels, accumulating the tree. +fn pt_pages_upper_bound (num_virt_pages:usize ) ->usize { +let mut total =0 ; + +// PTE pages at the leaf level. +let pte_epp =Self ::entries_per_page (Self ::PTE_LEVEL ); +let mut pages_at_level = num_virt_pages.div_ceil (pte_epp); + total += pages_at_level; + +// Walk PDE levels bottom-up (reverse of PDE_LEVELS). +for &levelin Self ::PDE_LEVELS .iter ().rev () { +let epp =Self ::entries_per_page (level); + +// How many pages at this level do we need to point to +// the previous pages_at_level? + pages_at_level = pages_at_level.div_ceil (epp); + total += pages_at_level; + } + + total + } +} + +/// Marker struct for MMU v2 (Turing/Ampere/Ada). +pub (super )struct MmuV2 ; + +impl MmuConfig for MmuV2 { +type Pte = ver2::Pte ; +type Pde = ver2::Pde ; +type DualPde = ver2::DualPde ; + +const PDE_LEVELS : &'static [PageTableLevel ] = ver2::PDE_LEVELS ; +const PTE_LEVEL :PageTableLevel = ver2::PTE_LEVEL ; +const DUAL_PDE_LEVEL :PageTableLevel = ver2::DUAL_PDE_LEVEL ; + +fn entries_per_page (level:PageTableLevel ) ->usize { +// TODO: Calculate these values from the bitfield dynamically +// instead of hardcoding them. +match level { +PageTableLevel ::Pdb =>4 ,// PD3 root: bits [48:47] = 2 bits +PageTableLevel ::L3 =>256 ,// PD0 dual: bits [28:21] = 8 bits + _ =>512 ,// PD2, PD1, PT: 9 bits each + } + } + +fn level_index (va:VirtualAddress , level:u64 ) ->u64 { + ver2::VirtualAddressV2 ::new (va).level_index (level) + } +} + +/// Marker struct for MMU v3 (Hopper and later). +pub (super )struct MmuV3 ; + +impl MmuConfig for MmuV3 { +type Pte = ver3::Pte ; +type Pde = ver3::Pde ; +type DualPde = ver3::DualPde ; + +const PDE_LEVELS : &'static [PageTableLevel ] = ver3::PDE_LEVELS ; +const PTE_LEVEL :PageTableLevel = ver3::PTE_LEVEL ; +const DUAL_PDE_LEVEL :PageTableLevel = ver3::DUAL_PDE_LEVEL ; + +fn entries_per_page (level:PageTableLevel ) ->usize { +match level { +PageTableLevel ::Pdb =>2 ,// PDE4 root: bit [56] = 1 bit, 2 entries +PageTableLevel ::L4 =>256 ,// PDE0 dual: bits [28:21] = 8 bits + _ =>512 ,// PDE3, PDE2, PDE1, PT: 9 bits each + } + } + +fn level_index (va:VirtualAddress , level:u64 ) ->u64 { + ver3::VirtualAddressV3 ::new (va).level_index (level) + } +} +/// Memory aperture for Page Table Entries (`PTE`s). /// /// Determines which memory region the `PTE` points to.
drivers/gpu/nova-core/mm/pagetable/ver2.rs +93 -57
220 lines not on this page · raw message
drivers/gpu/nova-core/mm/pagetable/ver3.rs +80 -40
182 lines not on this page · raw message
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 12/22] gpu: nova-core: mm: Add page table entry operation traits
Quoted 6 lines · Joel Fernandes
Introduce trait-based abstractions for GPU page table entries: PteOps, PdeOps, and DualPdeOps, along with the MmuConfig trait that ties them together with version-specific constants.
Refactor the ver2 and ver3 page-table modules to implement these traits and expose the shared entry/PDE/PTE operations uniformly.
Please, no. We don't introduce code that gets refactored the very next commit. This patch supersedes 270 lines of diff that reviewers will have processed for nothing.
Please reorder things so they land, as much as possible, in their final form. In this case this probably means defining the trait *before* the V2 and V3 page table definitions, so they can implement it from the get-go.
Re: [PATCH v12 12/22] gpu: nova-core: mm: Add page table entry operation traits
Quoted 11 lines · Alexandre Courbot
Quoted 6 lines · Joel Fernandes
Introduce trait-based abstractions for GPU page table entries: PteOps, PdeOps, and DualPdeOps, along with the MmuConfig trait that ties them together with version-specific constants.
Refactor the ver2 and ver3 page-table modules to implement these traits and expose the shared entry/PDE/PTE operations uniformly.
Please, no. We don't introduce code that gets refactored the very next commit. This patch supersedes 270 lines of diff that reviewers will have processed for nothing.
I really don't know what you mean by '270 lines of diff for nothing' because this patch only adds a negative delta of 90 lines or so, before adding yet another 300+ new lines. So it really is building on the previous patch, which is how we do things upstream. It is iterative. My sense is you reviewed this in a hurry and missed that fact while getting hung up on the words 'refactor'.
Keep in mind also that the addition of PteOps set of traits refactor was done much later in the iterations.
Quoted 3 lines
Please reorder things so they land, as much as possible, in their final form. In this case this probably means defining the trait *before* the V2 and V3 page table definitions, so they can implement it from the get-go.
That is a reasonable approach too, I can try to do that, but it is misleading to say '270 lines of diff that reviewers will have processed for nothing' which is nothing but fiction. Please look more carefully, the patch is iterative on the series.
Thanks.
Alexandre Courbot
maintainerRe: [PATCH v12 12/22] gpu: nova-core: mm: Add page table entry operation traits
Quoted 8 lines · Joel Fernandes
Quoted 3 lines
Please reorder things so they land, as much as possible, in their final form. In this case this probably means defining the trait *before* the V2 and V3 page table definitions, so they can implement it from the get-go.
That is a reasonable approach too, I can try to do that, but it is misleading to say '270 lines of diff that reviewers will have processed for nothing' which is nothing but fiction. Please look more carefully, the patch is iterative on the series.
For context, here is where the 270 lines of diff come from:
drivers/gpu/nova-core/mm/pagetable/ver2.rs | 150 ++++++++------ drivers/gpu/nova-core/mm/pagetable/ver3.rs | 120 +++++++----
But the number is not important. My feedback was that refactoring code right after you introduce it is a strong indicator that something should be reordered/squashed to prevent unneeded cognitive load on reviewers. This is particularly important on long series like this one.
Re: [PATCH v12 12/22] gpu: nova-core: mm: Add page table entry operation traits
Quoted 10 lines · Alexandre Courbot
Quoted 8 lines · Joel Fernandes
Quoted 3 lines
Please reorder things so they land, as much as possible, in their final form. In this case this probably means defining the trait *before* the V2 and V3 page table definitions, so they can implement it from the get-go.
That is a reasonable approach too, I can try to do that, but it is misleading to say '270 lines of diff that reviewers will have processed for nothing' which is nothing but fiction. Please look more carefully, the patch is iterative on the series.
Quoted 6 lines
For context, here is where the 270 lines of diff come from:
drivers/gpu/nova-core/mm/pagetable/ver2.rs | 150 ++++++++------ drivers/gpu/nova-core/mm/pagetable/ver3.rs | 120 +++++++----But the number is not important.
It is important, numbers and accuracy are really important things especially on the Linux kernel mailing list. Sorry if you feel that is inconvenient. And even quoting the 270 is a falsehood, the 270 lines were not refactored, only 90 lines or so was.
Quoted 1 line
My feedback was that refactoring code> right after you introduce it is a
strong indicator that something should
Quoted 2 lines
be reordered/squashed to prevent unneeded cognitive load on reviewers. This is particularly important on long series like this one.
Again you are misleading in the email, because the scale of 'refactor' matters. Only 90 lines were 'refactored' with 300 more lines added so it is a perfectly reasonable approach to refactor a small part of an earlier patch when you're trying to introduce a new concept (in this case, we introduced new traits). Do you want me to show you examples of that in the git tree or are you convinced?
Considering the amount of change is so small, I would prefer to leave it in its current state since I don't see the benefit of further restructuring. We are already at v12 now and the patches were posted and reposted (and tested) now for months. Keep in mind also that I already re-ordered the patches once per your request, and I don't think I would do that yet again. We are getting close to merging at this point.
thanks,
--
Joel Fernandes
Danilo Krummrich
maintainerRe: [PATCH v12 12/22] gpu: nova-core: mm: Add page table entry operation traits
Quoted 1 line · Joel Fernandes
We are already at v12 now
To be fair, the series is actually at v6, as you initially added those patches to another series that was at v6 already back then, and then you decided to just keep going with it. So, nothing too crazy going on that front. :-)
Re: [PATCH v12 12/22] gpu: nova-core: mm: Add page table entry operation traits
Quoted 6 lines · Danilo Krummrich
Quoted 1 line · Joel Fernandes
We are already at v12 now
To be fair, the series is actually at v6, as you initially added those patches to another series that was at v6 already back then, and then you decided to just keep going with it. So, nothing too crazy going on that front. :-)
I was thinking of resetting it to v1 and starting by breaking it up and starting over from what's left instead of a v13. I think at this point, probably breaking it down into multiple series with new version numbers makes sense. And I can point to the old patch series that way for reference.
Could you clarify how much time we have left to submit to drm-rust-next? I can then plan the next series accordingly. I did make some changes to GpuMm to be more suitable for channels, but haven't posted them yet.
Re: [PATCH v12 12/22] gpu: nova-core: mm: Add page table entry operation traits
Quoted 10 lines · Joel Fernandes
Quoted 6 lines · Danilo Krummrich
Quoted 1 line · Joel Fernandes
We are already at v12 now
To be fair, the series is actually at v6, as you initially added those patches to another series that was at v6 already back then, and then you decided to just keep going with it. So, nothing too crazy going on that front. :-)
Quoted 9 lines
I was thinking of resetting it to v1 and starting by breaking it up and starting over from what's left instead of a v13. I think at this point, probably breaking it down into multiple series with new version numbers makes sense. And I can point to the old patch series that way for reference.
Could you clarify how much time we have left to submit to drm-rust-next? I can then plan the next series accordingly. I did make some changes to GpuMm to be more suitable for channels, but haven't posted them yet.
Answering my own question, and correct me if I'm off but I think we have 4 weeks more left since the tree is expected to close on -rc6. So I'm going to curate a new drm-rust-next series tomorrow with patches that I think are 7.2 ready (with tags, or those that got lots of reviews already).
Alexandre Courbot
maintainerRe: [PATCH v12 12/22] gpu: nova-core: mm: Add page table entry operation traits
In reply to Joel FernandesQuoted 18 lines · Joel Fernandes
Quoted 15 lines · Alexandre Courbot
Quoted 8 lines · Joel Fernandes
Quoted 3 lines
Please reorder things so they land, as much as possible, in their final form. In this case this probably means defining the trait *before* the V2 and V3 page table definitions, so they can implement it from the get-go.
That is a reasonable approach too, I can try to do that, but it is misleading to say '270 lines of diff that reviewers will have processed for nothing' which is nothing but fiction. Please look more carefully, the patch is iterative on the series.
For context, here is where the 270 lines of diff come from:
drivers/gpu/nova-core/mm/pagetable/ver2.rs | 150 ++++++++------ drivers/gpu/nova-core/mm/pagetable/ver3.rs | 120 +++++++----
Quoted 6 lines
Quoted 1 line
But the number is not important.
It is important, numbers and accuracy are really important things especially on the Linux kernel mailing list. Sorry if you feel that is inconvenient. And even quoting the 270 is a falsehood, the 270 lines were not refactored, only 90 lines or so was.
I am not particularly attached to this 270 number. It represents the diff that I believe shouldn't be in this commit. Counting the number of changed lines is also perfectly valid.
But again, that's not the point. Let's set my metric aside: we still have, by your own account, 90 lines of churn that could be avoided by the following 3 steps, each of which takes one minute to perform:
- Move the trait definitions of pagetable.rs into their own commit.
- Move that new commit before the ones introducing ver2.rs and ver3.rs.
- Squash the relevant parts of the remainder into the commit introducing ver2.rs or ver3.rs.
By doing that, on top of removing 90 lines of immediate follow-up changes, you have also moved the public interface of the page tables before their implementation, making all 3 patches easier to process as reviewers are now introduced to how that code will be used *before* the implementation details.
That's my closing argument on this topic and I won't insist if you are not convinced this is worth doing; please act on it, or not, as you see fit.
PATCH v12 13/22 gpu: nova-core: mm: Add page table walker for MMU v2/v3
In reply to Joel FernandesAdd the page table walker implementation that traverses the page table hierarchy for both MMU v2 (5-level) and MMU v3 (6-level) to resolve virtual addresses to physical addresses or find PTE locations.
Currently only v2 has been tested (nova-core currently boots pre-hopper) with some initial preparatory work done for v3.
--- drivers/gpu/nova-core/mm/pagetable.rs | 1 + drivers/gpu/nova-core/mm/pagetable/walk.rs | 258 +++++++++++++++++++++ 2 files changed, 259 insertions(+) create mode 100644 drivers/gpu/nova-core/mm/pagetable/walk.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
@@ -10,6 +10,7 @@pub (super )mod ver2;pub (super )mod ver3; +pub (super )mod walk;use kernel::num::Bounded ;
drivers/gpu/nova-core/mm/pagetable/walk.rs new file +258 -0
258 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 14/22 gpu: nova-core: mm: Add Virtual Memory Manager
In reply to Joel FernandesAdd the Virtual Memory Manager (VMM) infrastructure for GPU address space management. Each Vmm instance manages a single address space identified by its Page Directory Base (PDB) address, used for Channel, BAR1 and BAR2 mappings.
Mapping APIs and virtual address range tracking are added in later commits.
--- drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/vmm.rs | 69 +++++++++++++++++++++++++++++++++ 2 files changed, 70 insertions(+) create mode 100644 drivers/gpu/nova-core/mm/vmm.rs
drivers/gpu/nova-core/mm.rs +1 -0
@@ -35,6 +35,7 @@ macro_rules! impl_pfn_bounded {pub (super )mod pagetable;pub (crate )mod pramin;pub (super )mod tlb; +pub (super )mod vmm;use kernel::{ bitfield,
drivers/gpu/nova-core/mm/vmm.rs new file +69 -0
69 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 15/22 gpu: nova-core: mm: Add virtual address range tracking to VMM
In reply to Joel FernandesAdd virtual address range tracking to the VMM using a buddy allocator. This enables contiguous virtual address range allocation for mappings.
--- drivers/gpu/nova-core/mm/vmm.rs | 89 +++++++++++++++++++++++++++++---- 1 file changed, 79 insertions(+), 10 deletions(-)
drivers/gpu/nova-core/mm/vmm.rs +79 -10
@@ -9,18 +9,27 @@use kernel::{ device, gpu::buddy::AllocatedBlocks , - prelude::*,// + maple_tree::MapleTreeAlloc , + prelude::*, }; -use crate ::mm::{ - pagetable::{ - walk::{PtWalk ,WalkResult }, -MmuVersion ,// +use core::ops::Range ; + +use crate ::{ + mm::{ + pagetable::{ + walk::{PtWalk ,WalkResult }, +MmuVersion ,// + }, +GpuMm , +Pfn , +Vfn , +VramAddress , +PAGE_SIZE ,// + }, + num::{ +IntoSafeCast ,// }, -GpuMm , -Pfn , -Vfn , -VramAddress ,// };/// Virtual Memory Manager for a GPU address space.
@@ -35,23 +44,83 @@ pub(crate) struct Vmm { mmu_version: MmuVersion,/// Page table allocations required for mappings. page_table_allocs: KVec<Pin<KBox<AllocatedBlocks>>>, +/// Maple tree allocator for virtual address range tracking. + virt_alloc: Pin<KBox<MapleTreeAlloc<()>>>, +/// Total number of pages in the virtual address space. + va_pages: usize, }impl Vmm {/// Create a new [`Vmm`] for the given Page Directory Base address. -pub (crate )fn new (pdb_addr:VramAddress , mmu_version:MmuVersion ) ->Result <Self > { +/// +/// The [`Vmm`] will manage a virtual address space of `va_size` bytes. +pub (crate )fn new ( + pdb_addr:VramAddress , + mmu_version:MmuVersion , + va_size:u64 , + ) ->Result <Self > {// Only MMU v2 is supported for now. if mmu_version !=MmuVersion ::V2 {return Err (ENOTSUPP ); } +let page_size:u64 =PAGE_SIZE .into_safe_cast (); +let va_pages:usize = (va_size / page_size).into_safe_cast (); +let virt_alloc =KBox ::pin_init (MapleTreeAlloc ::<()>::new (),GFP_KERNEL )?; +Ok (Self { pdb_addr, mmu_version, page_table_allocs:KVec ::new (), + virt_alloc, + va_pages, }) } +/// Allocate a contiguous virtual frame number range. +/// +/// # Arguments +/// +/// - `num_pages`: Number of pages to allocate. +/// - `va_range`: `None` = allocate anywhere, `Some(range)` = constrain allocation to the given +/// range. +fn alloc_vfn_range ( + &self, + num_pages:usize , + va_range:Option <Range <u64 >>, + ) ->Result <Vfn > { +let page_size:u64 =PAGE_SIZE .into_safe_cast (); + +let start_vfn =match va_range { +Some (r) => { +let num_pages_u64:u64 = num_pages.into_safe_cast (); +let size = num_pages_u64.checked_mul (page_size).ok_or (EOVERFLOW )?; +let range_size = r.end.checked_sub (r.start).ok_or (EOVERFLOW )?; +if range_size != size { +return Err (EINVAL ); + } +let start_vfn:usize = (r.start / page_size).into_safe_cast (); +let end_vfn:usize = (r.end / page_size).into_safe_cast (); + self.virt_alloc + .insert_range (start_vfn..end_vfn, (),GFP_KERNEL )?; + start_vfn + } +None => self + .virt_alloc + .alloc_range (num_pages, (), ..self.va_pages,GFP_KERNEL )?, + }; + +Ok (Vfn ::new (start_vfn.into_safe_cast ())) + } + +/// Free a virtual frame number range back to the maple tree. +fn free_vfn (&self, vfn:Vfn ) { +let vfn_index:usize = vfn.raw ().into_safe_cast (); +if self.virt_alloc.erase (vfn_index).is_none () { + kernel::pr_warn!("free_vfn: VFN {} not found in maple tree\n" , vfn_index); + } + } +/// Read the [`Pfn`] for a mapped [`Vfn`] if one is mapped. pub (super )fn read_mapping ( &self,
Signature · 1 line
2.34.1
PATCH v12 16/22 gpu: nova-core: mm: Add multi-page mapping API to VMM
In reply to Joel FernandesAdd the page table mapping and unmapping API to the Virtual Memory Manager, implementing a two-phase prepare/execute model suitable for use both inside and outside the DMA fence signalling critical path.
--- drivers/gpu/nova-core/mm/pagetable.rs | 1 + drivers/gpu/nova-core/mm/pagetable/map.rs | 355 ++++++++++++++++++++++ drivers/gpu/nova-core/mm/vmm.rs | 268 ++++++++++++++-- 3 files changed, 606 insertions(+), 18 deletions(-) create mode 100644 drivers/gpu/nova-core/mm/pagetable/map.rs
drivers/gpu/nova-core/mm/pagetable.rs +1 -0
@@ -8,6 +8,7 @@#![expect(dead_code)] +pub (super )mod map;pub (super )mod ver2;pub (super )mod ver3;pub (super )mod walk;
drivers/gpu/nova-core/mm/pagetable/map.rs new file +355 -0
355 lines not on this page · raw message
drivers/gpu/nova-core/mm/vmm.rs +250 -18
325 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 17/22 gpu: nova-core: Add BAR1 aperture type and size constant
In reply to Joel FernandesAdd BAR1_SIZE constant and Bar1 type alias for the 256MB BAR1 aperture. These are prerequisites for BAR1 memory access functionality.
--- drivers/gpu/nova-core/driver.rs | 2 ++ drivers/gpu/nova-core/gsp/commands.rs | 4 ++++ drivers/gpu/nova-core/gsp/fw/commands.rs | 8 ++++++++ 3 files changed, 14 insertions(+)
drivers/gpu/nova-core/driver.rs +2 -0
@@ -47,6 +47,8 @@ pub(crate) struct NovaCore {const GPU_DMA_BITS :u32 =47 ;pub (crate )type Bar0 = pci::Bar <BAR0_SIZE >; +#[expect(dead_code)] +pub (crate )type Bar1 = pci::Bar ; kernel::pci_device_table!(PCI_TABLE ,
drivers/gpu/nova-core/gsp/commands.rs +4 -0
@@ -193,6 +193,9 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {/// The reply from the GSP to the [`GetGspStaticInfo`] command. pub (crate )struct GetGspStaticInfoReply { gpu_name: [u8 ;64 ], +/// BAR1 Page Directory Entry base address. +#[expect(dead_code)] +pub (crate ) bar1_pde_base:u64 ,/// Usable FB (VRAM) region for driver memory allocation. pub (crate ) usable_fb_region:Range <u64 >,/// End of VRAM.
@@ -212,6 +215,7 @@ fn read(Ok (GetGspStaticInfoReply { gpu_name: msg.gpu_name_str (), + bar1_pde_base: msg.bar1_pde_base (), usable_fb_region: msg.first_usable_fb_region ().ok_or (ENODEV )?, total_fb_end, })
drivers/gpu/nova-core/gsp/fw/commands.rs +8 -0
@@ -127,6 +127,14 @@ impl GspStaticConfigInfo { self.0 .gpuNameString } +/// Returns the BAR1 Page Directory Entry base address. +/// +/// This is the root page table address for BAR1 virtual memory, +/// set up by GSP-RM firmware. +pub (crate )fn bar1_pde_base (&self) ->u64 { + self.0 .bar1PdeBase + } +/// Returns an iterator over valid FB regions from GSP firmware data. fn fb_regions ( &self,
Signature · 1 line
2.34.1
PATCH v12 18/22 gpu: nova-core: mm: Add BAR1 user interface
In reply to Joel FernandesAdd the BAR1 user interface for CPU access to GPU virtual memory through the BAR1 aperture.
--- drivers/gpu/nova-core/driver.rs | 1 - drivers/gpu/nova-core/gpu.rs | 21 +++- drivers/gpu/nova-core/gsp/commands.rs | 1 - drivers/gpu/nova-core/mm.rs | 1 + drivers/gpu/nova-core/mm/bar_user.rs | 156 ++++++++++++++++++++++++++ 5 files changed, 177 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/nova-core/mm/bar_user.rs
drivers/gpu/nova-core/driver.rs +0 -1
@@ -47,7 +47,6 @@ pub(crate) struct NovaCore {const GPU_DMA_BITS :u32 =47 ;pub (crate )type Bar0 = pci::Bar <BAR0_SIZE >; -#[expect(dead_code)] pub (crate )type Bar1 = pci::Bar ; kernel::pci_device_table!(
drivers/gpu/nova-core/gpu.rs +20 -1
@@ -28,7 +28,12 @@ commands::GetGspStaticInfoReply, Gsp, // }, - mm::GpuMm, + mm::{ + bar_user::BarUser, + pagetable::MmuVersion, + GpuMm, + VramAddress, // + }, regs, };
@@ -122,6 +127,11 @@ pub(crate) const fn arch(self) -> Architecture {pub (crate )const fn needs_fwsec_bootloader (self) ->bool {matches! (self.arch (),Architecture ::Turing ) ||matches! (self,Self ::GA100 ) } + +/// Returns the MMU version for this chipset. +pub (crate )fn mmu_version (self) ->MmuVersion { +MmuVersion ::from (self.arch ()) + } }// TODO
@@ -249,6 +259,8 @@ pub(crate) struct Gpu {#[pin] gsp: Gsp, gsp_static_info: GetGspStaticInfoReply, +/// BAR1 user interface for CPU access to GPU virtual memory. + bar_user: BarUser, }impl Gpu {
@@ -313,6 +325,13 @@ pub(crate) fn new<'a>( }, pramin_vram_region)? }, +// Create BAR1 user interface for CPU access to GPU virtual memory. + bar_user: { + let pdb_addr = VramAddress::new(gsp_static_info.bar1_pde_base); +let bar1_size = pdev.resource_len (1 )?; +BarUser ::new (pdb_addr, spec.chipset, bar1_size)? + }, + bar: devres_bar, }) }
drivers/gpu/nova-core/gsp/commands.rs +0 -1
@@ -194,7 +194,6 @@ fn init(&self) -> impl Init<Self::Command, Self::InitError> {pub (crate )struct GetGspStaticInfoReply { gpu_name: [u8 ;64 ],/// BAR1 Page Directory Entry base address. -#[expect(dead_code)] pub (crate ) bar1_pde_base:u64 ,/// Usable FB (VRAM) region for driver memory allocation. pub (crate ) usable_fb_region:Range <u64 >,
drivers/gpu/nova-core/mm.rs +1 -0
@@ -32,6 +32,7 @@ macro_rules! impl_pfn_bounded { }; } +pub (crate )mod bar_user;pub (super )mod pagetable;pub (crate )mod pramin;pub (super )mod tlb;
drivers/gpu/nova-core/mm/bar_user.rs new file +156 -0
156 lines not on this page · raw message
Signature · 1 line
2.34.1
PATCH v12 19/22 gpu: nova-core: mm: Add BAR1 memory management self-tests
In reply to Joel FernandesAdd self-tests for BAR1 access during driver probe when CONFIG_NOVA_MM_SELFTESTS is enabled (default disabled). This results in testing the Vmm, GPU buddy allocator and BAR1 region all of which should function correctly for the tests to pass.
--- drivers/gpu/nova-core/Kconfig | 10 ++ drivers/gpu/nova-core/driver.rs | 2 + drivers/gpu/nova-core/gpu.rs | 31 ++++ drivers/gpu/nova-core/mm/bar_user.rs | 244 ++++++++++++++++++++++++++ drivers/gpu/nova-core/mm/pagetable.rs | 35 +++- 5 files changed, 321 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/Kconfig +10 -0
@@ -15,3 +15,13 @@ config NOVA_CORE This driver is work in progress and may not be functional. If M is selected, the module will be called nova_core. + +config NOVA_MM_SELFTESTS + bool "Memory management self-tests" + depends on NOVA_CORE + help + Enable self-tests for the memory management subsystem. When enabled, + tests are run during GPU probe to verify PRAMIN aperture access, + page table walking, and BAR1 virtual memory mapping functionality. + + This is a testing option and is default-disabled.
drivers/gpu/nova-core/driver.rs +2 -0
@@ -97,6 +97,8 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> impl PinInit<Self, E Ok(try_pin_init!(Self { gpu <- Gpu::new(pdev, bar.clone(), bar.access(pdev.as_ref())?), + // Run optional GPU selftests. + _: { gpu.run_selftests(pdev)? }, _reg <- auxiliary::Registration::new( pdev.as_ref(), c"nova-drm",
drivers/gpu/nova-core/gpu.rs +31 -0
@@ -346,4 +346,35 @@ pub(crate) fn unbind(&self, dev: &device::Device<device::Core>) { .inspect(|bar| self.sysmem_flush.unregister(bar)) .is_err()); } + +/// Run selftests on the constructed [`Gpu`]. +pub (crate )fn run_selftests ( +mut self:Pin <&mut Self >, + pdev: &pci::Device <device::Bound >, + ) ->Result { + self.as_mut ().run_mm_selftests (pdev)?; +Ok (()) + } + +#[cfg( CONFIG_NOVA_MM_SELFTESTS )] +fn run_mm_selftests (self:Pin <&mut Self >, pdev: &pci::Device <device::Bound >) ->Result { +// BAR1 self-tests. +let bar1 =Arc ::pin_init (pdev.iomap_region (1 ,c"nova-core/bar1" ),GFP_KERNEL )?; +let bar1_access = bar1.access (pdev.as_ref ())?; + +crate ::mm::bar_user::run_self_test ( + pdev.as_ref (), + &self.mm, + bar1_access, + self.gsp_static_info.bar1_pde_base, + self.spec.chipset, + )?; + +Ok (()) + } + +#[cfg(not( CONFIG_NOVA_MM_SELFTESTS ))] +fn run_mm_selftests (self:Pin <&mut Self >, _pdev: &pci::Device <device::Bound >) ->Result { +Ok (()) + } }
drivers/gpu/nova-core/mm/bar_user.rs +244 -0
247 lines not on this page · raw message
drivers/gpu/nova-core/mm/pagetable.rs +34 -1
@@ -22,7 +22,10 @@ VirtualAddress, VramAddress, // }; -use kernel::prelude::*; +use kernel::{ + device, + prelude::*,// +};/// Extracts the page table index at a given level from a virtual address. pub (super )trait VaLevelIndex {
@@ -383,3 +386,33 @@ fn from(val: AperturePde) -> Self {Bounded ::from_expr (valas u64 &0x3 ) } } + +/// Check if the PDB has valid, VRAM-backed page tables. +#[cfg( CONFIG_NOVA_MM_SELFTESTS )] +fn check_pdb_inner <M :MmuConfig >( + dev: &device::Device <device::Bound >, + pramin: &pramin::Pramin , + pdb_addr:VramAddress , +) ->Result { +let mut window = pramin.get_window (dev)?; +let raw = window.try_read64 (pdb_addr.raw ())?; + +if !M ::Pde ::new (raw).is_valid_vram () { +return Err (ENOENT ); + } +Ok (()) +} + +/// Check if the PDB has valid, VRAM-backed page tables, dispatching by MMU version. +#[cfg( CONFIG_NOVA_MM_SELFTESTS )] +pub (super )fn check_pdb_valid ( + dev: &device::Device <device::Bound >, + pramin: &pramin::Pramin , + pdb_addr:VramAddress , + chipset:crate ::gpu::Chipset , +) ->Result { +match MmuVersion ::from (chipset.arch ()) { +MmuVersion ::V2 =>check_pdb_inner ::<MmuV2 >(dev, pramin, pdb_addr), +MmuVersion ::V3 =>check_pdb_inner ::<MmuV3 >(dev, pramin, pdb_addr), + } +}
Signature · 1 line
2.34.1
PATCH v12 20/22 gpu: nova-core: mm: Add PRAMIN aperture self-tests
In reply to Joel FernandesAdd self-tests for the PRAMIN aperture mechanism to verify correct operation during GPU probe. The tests validate various alignment requirements and corner cases.
The tests are default disabled and behind CONFIG_NOVA_MM_SELFTESTS. When enabled, tests run after GSP boot during probe.
--- drivers/gpu/nova-core/gpu.rs | 3 + drivers/gpu/nova-core/mm/pramin.rs | 211 +++++++++++++++++++++++++++++ 2 files changed, 214 insertions(+)
drivers/gpu/nova-core/gpu.rs +3 -0
@@ -358,6 +358,9 @@ pub(crate) fn run_selftests(#[cfg( CONFIG_NOVA_MM_SELFTESTS )] fn run_mm_selftests (self:Pin <&mut Self >, pdev: &pci::Device <device::Bound >) ->Result { +// PRAMIN aperture self-tests. +crate ::mm::pramin::run_self_test (pdev.as_ref (), self.mm.pramin (), self.spec.chipset)?; +// BAR1 self-tests. let bar1 =Arc ::pin_init (pdev.iomap_region (1 ,c"nova-core/bar1" ),GFP_KERNEL )?;let bar1_access = bar1.access (pdev.as_ref ())?;
drivers/gpu/nova-core/mm/pramin.rs +211 -0
220 lines not on this page · raw message
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 20/22] gpu: nova-core: mm: Add PRAMIN aperture self-tests
Quoted 6 lines · Joel Fernandes
Add self-tests for the PRAMIN aperture mechanism to verify correct operation during GPU probe. The tests validate various alignment requirements and corner cases.
The tests are default disabled and behind
CONFIG_NOVA_MM_SELFTESTS. When enabled, tests run after GSP boot during probe.
It makes sense for the PRAMIN self-test to be merged alongside PRAMIN support (and also gives us a reason to merge PRAMIN as soon as it is ready even if the rest of the series is still under review, as it exercises the code).
Can you move this patch right after patch 4?
Re: [PATCH v12 20/22] gpu: nova-core: mm: Add PRAMIN aperture self-tests
Quoted 8 lines · Alexandre Courbot
Quoted 6 lines · Joel Fernandes
Add self-tests for the PRAMIN aperture mechanism to verify correct operation during GPU probe. The tests validate various alignment requirements and corner cases.
The tests are default disabled and behind
CONFIG_NOVA_MM_SELFTESTS. When enabled, tests run after GSP boot during probe.
Quoted 6 lines
It makes sense for the PRAMIN self-test to be merged alongside PRAMIN support (and also gives us a reason to merge PRAMIN as soon as it is ready even if the rest of the series is still under review, as it exercises the code).
Can you move this patch right after patch 4?
Sure, I will do that.
PATCH v12 21/22 gpu: nova-core: mm: pramin: drop useless as_ref() in run_self_test
In reply to Joel Fernandespdev is already &device::Device<device::Bound>, so calling .as_ref() on it just returns the same reference. Bind dev to pdev directly to silence the clippy::useless_asref warning.
--- drivers/gpu/nova-core/mm/pramin.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/nova-core/mm/pramin.rs +1 -1
@@ -479,7 +479,7 @@ pub(crate) fn run_self_test( ) -> Result {use crate ::gpu::Architecture ; -let dev = pdev.as_ref (); +let dev = pdev;// PRAMIN uses NV_PBUS_BAR0_WINDOW which is only available on pre-Hopper GPUs. // Hopper+ uses NV_XAL_EP_BAR0_WINDOW instead, requiring a separate HAL that
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 21/22] gpu: nova-core: mm: pramin: drop useless as_ref() in run_self_test
Quoted 3 lines · Joel Fernandes
pdevis already&device::Device<device::Bound>, so calling.as_ref()on it just returns the same reference. Binddevtopdevdirectly to silence theclippy::useless_asrefwarning.
Why is this a separate commit? It can, and should, be squashed into the previous patch.
PATCH v12 22/22 rust: maple_tree: implement Send and Sync for MapleTree
In reply to Joel FernandesThe C maple_tree struct contains a *mut c_void, which prevents Rust from auto-deriving Send/Sync. Following is an example error message when using MapleTree in nova-core's Vmm.
This propagates up through MapleTreeAlloc to Vmm, BarUser, Gpu, and NovaCore, causing NovaCore to fail the Send bound required by pci::Driver:
error[E0277]: *mut c_void cannot be sent between threads safely
--> drivers/gpu/nova-core/driver.rs:77:22
|
77 | impl pci::Driver for NovaCore {
| ^^^^^^^^ `*mut c_void` cannot be sent between threads safely
|
= help: within `MapleTreeAlloc<()>`, the trait `Send` is not implemented for `*mut c_void`
note: required because it appears within the type `kernel::bindings::maple_tree`
note: required because it appears within the type `Opaque<kernel::bindings::maple_tree>`
note: required because it appears within the type `MapleTree<()>`
note: required because it appears within the type `MapleTreeAlloc<()>`
= note: required for `Box<MapleTreeAlloc<()>, Kmalloc>` to implement `Send`
note: required because it appears within the type `core::pin::Pin<Box<MapleTreeAlloc<()>, Kmalloc>>`
note: required because it appears within the type `Vmm`
note: required because it appears within the type `BarUser`
note: required because it appears within the type `Gpu`
note: required because it appears within the type `NovaCore`
note: required by a bound in `kernel::pci::Driver`
--> rust/kernel/pci.rs:294:19Implement Send and Sync for MapleTree. The tree contains no thread-local state, and all shared access goes through the internal ma_lock spinlock.
--- rust/kernel/maple_tree.rs | 29 +++++++++++++++++++++++------ 1 file changed, 23 insertions(+), 6 deletions(-)
rust/kernel/maple_tree.rs +23 -6
@@ -16,7 +16,11 @@ alloc::Flags, error::to_result, prelude::*, - types::{ForeignOwnable, Opaque}, + types::{ +ForeignOwnable , +NotThreadSafe , +Opaque ,// + }, };/// A maple tree optimized for storing non-overlapping ranges.
@@ -240,7 +244,10 @@ pub fn lock(&self) -> MapleGuard<'_, T> {unsafe { bindings::spin_lock (self.ma_lock ()) };// INVARIANT: We just took the spinlock. -MapleGuard (self) +MapleGuard { + tree: self, + _not_send:NotThreadSafe , + } }#[inline]
@@ -302,19 +309,29 @@ fn drop(mut self: Pin<&mut Self>) { } } +// SAFETY: `MapleTree<T>` is `Send` if `T` is `Send` because `MapleTree` owns its elements. +unsafe impl <T :ForeignOwnable +Send >Send for MapleTree <T > {} +// SAFETY: `&MapleTree<T>` never hands out `&T`; all entry access is serialized +// by `ma_lock` or `&mut Guard`, so `T: Send` suffices (`T: Sync` not required). +unsafe impl <T :ForeignOwnable +Send >Sync for MapleTree <T > {} +/// A reference to a [`MapleTree`] that owns the inner lock. /// /// # Invariants /// /// This guard owns the inner spinlock. #[must_use = "if unused, the lock will be immediately unlocked" ] -pub struct MapleGuard <'tree,T :ForeignOwnable >(&'treeMapleTree <T >); +pub struct MapleGuard <'tree,T :ForeignOwnable > { + tree: &'treeMapleTree <T >, +// A held spinlock must be released on the same CPU that acquired it. + _not_send:NotThreadSafe , +}impl <'tree,T :ForeignOwnable >Drop for MapleGuard <'tree,T > {#[inline] fn drop (&mut self) {// SAFETY: By the type invariants, we hold this spinlock. -unsafe { bindings::spin_unlock (self.0 .ma_lock ()) }; +unsafe { bindings::spin_unlock (self.tree.ma_lock ()) }; } }
@@ -323,7 +340,7 @@ impl<'tree, T: ForeignOwnable> MapleGuard<'tree, T> {pub fn ma_state (&mut self, first:usize , end:usize ) ->MaState <'_,T > {// SAFETY: The `MaState` borrows this `MapleGuard`, so it can also borrow the `MapleGuard`s // read/write permissions to the maple tree. -unsafe {MaState ::new_raw (self.0 , first, end) } +unsafe {MaState ::new_raw (self.tree, first, end) } }/// Load the value at the given index.
@@ -375,7 +392,7 @@ pub fn ma_state(&mut self, first: usize, end: usize) -> MaState<'_, T> {#[inline] pub fn load (&mut self, index:usize ) ->Option <T ::BorrowedMut <'_>> {// SAFETY: `self.tree` contains a valid maple tree. -let ret =unsafe { bindings::mtree_load (self.0 .tree.get (), index) }; +let ret =unsafe { bindings::mtree_load (self.tree.tree.get (), index) };if ret.is_null () {return None ; }
Signature · 1 line
2.34.1
Alexandre Courbot
maintainerRe: [PATCH v12 22/22] rust: maple_tree: implement Send and Sync for MapleTree
Quoted 6 lines · Joel Fernandes
The C
maple_treestruct contains a *mutc_void, which prevents Rust from auto-deriving Send/Sync. Following is an example error message when using MapleTree in nova-core's Vmm.This propagates up through MapleTreeAlloc to Vmm, BarUser, Gpu, and NovaCore, causing NovaCore to fail the Send bound required by
pci::Driver:
This patch is not at the right place - the last few patches of the series won't build without it.
Besides you have sent it separately before this series, so just mention it as a dependency in the cover letter, so reviewers can pick it up before applying the series and not witness it breaking in the course of reviewing it. The cover letter already mentions the bitfield series as a dependency, so the same could have been done for this one.
Re: [PATCH v12 22/22] rust: maple_tree: implement Send and Sync for MapleTree
Quoted 10 lines · Alexandre Courbot
Quoted 6 lines · Joel Fernandes
The C
maple_treestruct contains a *mutc_void, which prevents Rust from auto-deriving Send/Sync. Following is an example error message when using MapleTree in nova-core's Vmm.This propagates up through MapleTreeAlloc to Vmm, BarUser, Gpu, and NovaCore, causing NovaCore to fail the Send bound required by
pci::Driver:This patch is not at the right place - the last few patches of the series won't build without it.
Right, the maple tree change in an earlier patch makes this one required to go before it.
Quoted 5 lines
Besides you have sent it separately before this series, so just mention it as a dependency in the cover letter, so reviewers can pick it up before applying the series and not witness it breaking in the course of reviewing it. The cover letter already mentions the bitfield series as a dependency, so the same could have been done for this one.
The sending separately bit was actually an early RFC I wanted to get feedback on. But indeed, now that I have got that I can just post it independently. I'll post it separately once I am back to work. Thanks,
--
Joel Fernandes