gpu: nova-core: Boot GSP to RISC-V active
Subsystems (6)
v5 cover gpu: nova-core: Boot GSP to RISC-V active
Changes since v4:
The main change for this revision is to derive the Zeroable trait for most of our bindings. The rest of the changes just address relatively minor review comments made for v4 of the series.
This series is still based on a merge of drm-rust-next and drivers-core-testing made by Alex. A complete copy of the tree with these patches applied will be available at
https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream-v5
Changes since v3:
The main change for v4 is to switch to using the init! macros to ensure all fields on in-place initialised structs get initialised. This will require our bindings to derive the Zeroable trait, however for now I have left this as a TODO with manual implementations for each trait. That is because rebasing the binding changes is a bit of a pain, so I want to give reviewers a change to see if deriving Zeroable for all bindings makes sense or not.
Other changes include addressing most of the outstanding TODOs left in v3 and addressing review comments from v2 and v3. In particular some of the comments by Timur that had not been picked up.
Changes since v2:
The main change since v2 has been to make all firmware bindings completely opaque. It has been made clear this is a pre-requisite for this series to progress upstream as it should make supporting different firmware versions easier in future.
Overall the extra constructors and accessors add a couple of hundred lines of code and a few extra unsafe statements.
Other changes include addressing a bunch of other comments - see the individual patches for further details. There are also still some outstanding comments and TODO's to address which I have not gotten to yet - these will be done in the next version of this series.
Changes since v1:
- Based on feed back from Alex the GSP command queue logic was reworked extensively. This involved creating a new data struct (DmaGspMem) to manage the shared memory areas between CPU and GSP.
- This data structure helps ensure the safety constraints are meet when the CPU is reading/writing the shared memory queues.
- Several other minor comments were addressed, as noted in the individual patches.
This series builds on top of Alex's series[1], most of which has been merged into drm-rust-next, to continue initialising the GSP into a state where it becomes active and it starts communicating with the host. A tree including these patches with the prerequisite patches is available at [2].
It includes patches to initialise several important data structures required to boot the GSP. The biggest change is the implementation of the command/message circular queue used to establish communication between GSP and host in patch 6. Admittedly this patch is rather large - if necessary it could be split into send and receive patches if people prefer.
This is required to configure and boot the GSP. However this series does not get the GSP to a fully active state. Instead it gets it to a state where the GSP sends a message to the host with a sequence of instructions which need running to get to the active state. A subsequent series will implement processing of this message and allow the GSP to get to the fully active state.
A full tree including the prerequisites for this patch series is available at https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream.
[1] - https://lore.kernel.org/rust-for-linux/[email protected]/
[2] - https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream-v2
To: [email protected]
To: [email protected]
To: Danilo Krummrich <[email protected]>
To: Alexandre Courbot <[email protected]>
Cc: Miguel Ojeda <[email protected]>
Cc: Alex Gaynor <[email protected]>
Cc: Boqun Feng <[email protected]>
Cc: Gary Guo <[email protected]>
Cc: Björn Roy Baron <[email protected]>
Cc: Benno Lossin <[email protected]>
Cc: Andreas Hindborg <[email protected]>
Cc: Alice Ryhl <[email protected]>
Cc: Trevor Gross <[email protected]>
Cc: David Airlie <[email protected]>
Cc: Simona Vetter <[email protected]>
Cc: Maarten Lankhorst <[email protected]>
Cc: Maxime Ripard <[email protected]>
Cc: Thomas Zimmermann <[email protected]>
Cc: John Hubbard <[email protected]>
Cc: Joel Fernandes <[email protected]>
Cc: Timur Tabi <[email protected]>
Cc: [email protected]
Cc: [email protected]
Alistair Popple (11):
gpu: nova-core: Set correct DMA mask
gpu: nova-core: Create initial Gsp
gpu: nova-core: gsp: Create wpr metadata
gpu: nova-core: Add zeroable trait to bindings
gpu: nova-core: Add GSP command queue bindings
gpu: nova-core: gsp: Add GSP command queue handling
gpu: nova-core: gsp: Create rmargs
gpu: nova-core: Add bindings and accessors for GspSystemInfo
gpu: nova-core: Add bindings for the GSP RM registry tables
gpu: nova-core: gsp: Create RM registry and sysinfo commands
nova-core: gsp: Boot GSP
Joel Fernandes (3):
gpu: nova-core: Add a slice-buffer (sbuffer) datastructure
nova-core: falcon: Add support to check if RISC-V is active
nova-core: falcon: Add support to write firmware version
drivers/gpu/nova-core/driver.rs | 16 + drivers/gpu/nova-core/falcon.rs | 15 + drivers/gpu/nova-core/fb.rs | 1 - drivers/gpu/nova-core/firmware/gsp.rs | 3 +- drivers/gpu/nova-core/firmware/riscv.rs | 9 +- drivers/gpu/nova-core/gpu.rs | 2 +- drivers/gpu/nova-core/gsp.rs | 134 +++- drivers/gpu/nova-core/gsp/boot.rs | 76 ++- drivers/gpu/nova-core/gsp/cmdq.rs | 509 +++++++++++++++ drivers/gpu/nova-core/gsp/commands.rs | 115 ++++ drivers/gpu/nova-core/gsp/fw.rs | 455 +++++++++++++- drivers/gpu/nova-core/gsp/fw/commands.rs | 100 +++ drivers/gpu/nova-core/gsp/fw/r570_144.rs | 1 + .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 592 +++++++++++++++++- drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 17 +- drivers/gpu/nova-core/sbuffer.rs | 215 +++++++ scripts/Makefile.build | 2 +- 18 files changed, 2231 insertions(+), 32 deletions(-) create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs create mode 100644 drivers/gpu/nova-core/gsp/commands.rs create mode 100644 drivers/gpu/nova-core/gsp/fw/commands.rs create mode 100644 drivers/gpu/nova-core/sbuffer.rs
Signature · 1 line
2.50.1
PATCH v5 1/14 gpu: nova-core: Set correct DMA mask
Set the correct DMA mask. Without this DMA will fail on some setups.
---
Changes for v5:
- Update SAFETY comment for dma_set_mask_and_coherent()
- Add TODO for using different masks when we support more GPU models
Changes for v4:
- Use a const (GPU_DMA_BITS) instead of a magic number
Changes for v2:
- Update DMA mask to correct value for Ampere/Turing (47 bits)
--- drivers/gpu/nova-core/driver.rs | 16 ++++++++++++++++ 1 file changed, 16 insertions(+)
drivers/gpu/nova-core/driver.rs +16 -0
@@ -3,6 +3,8 @@use kernel::{ auxiliary, c_str, device::Core , + dma::Device , + dma::DmaMask , pci, pci::{Class ,ClassMask ,Vendor }, prelude::*,
@@ -20,6 +22,15 @@ pub(crate) struct NovaCore { }const BAR0_SIZE :usize =SZ_16M ; + +// For now we only support Ampere which can use up to 47-bit DMA addresses. +// +// TODO: Add an abstraction for this to support newer GPUs which may support +// larger DMA addresses. Limiting these GPUs to smaller address widths won't +// have any adverse affects, unless installed on systems which require larger +// DMA addresses. These systems should be quite rare. +const GPU_DMA_BITS :u32 =47 ; +pub (crate )type Bar0 = pci::Bar <BAR0_SIZE >; kernel::pci_device_table!(
@@ -57,6 +68,11 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self pdev.enable_device_mem ()?; pdev.set_master (); +// SAFETY: No concurrent DMA allocations or mappings can be made because +// the device is still being probed and therefore isn't being used by +// other threads of execution. +unsafe { pdev.dma_set_mask_and_coherent (DmaMask ::new ::<GPU_DMA_BITS >())? }; +let devres_bar =Arc ::pin_init ( pdev.iomap_region_sized ::<BAR0_SIZE >(0 ,c_str! ("nova-core/bar0" )), GFP_KERNEL,
Signature · 1 line
2.50.1
PATCH v5 2/14 gpu: nova-core: Create initial Gsp
In reply to Alistair PoppleThe GSP requires several areas of memory to operate. Each of these have their own simple embedded page tables. Set these up and map them for DMA to/from GSP using CoherentAllocation's. Return the DMA handle describing where each of these regions are for future use when booting GSP.
---
Changes for v5:
- Move GSP_HEAP_ALIGNMENT to gsp/fw.rs and add a comment.
- Create a LogBuffer type.
- Use checked_add to ensure PTE values don't overflow.
- Added some type documentation (shamelessly stolen from Nouveau)
Change for v3:
- Clean up the PTE array creation, with much thanks to Alex for doing most it (please let me know if I should put you as co-developer!)
Changes for v2:
- Renamed GspMemOjbects to Gsp as that is what they are
- Rebased on Alex's latest series
--- drivers/gpu/nova-core/gpu.rs | 2 +- drivers/gpu/nova-core/gsp.rs | 106 ++++++++++++++++-- drivers/gpu/nova-core/gsp/fw.rs | 64 ++++++++++- .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 19 ++++ 4 files changed, 179 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/gpu.rs +1 -1
@@ -197,7 +197,7 @@ pub(crate) fn new<'a>( sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset, bar, true)?, - gsp <- Gsp::new(), + gsp <- Gsp::new(pdev)?, _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
drivers/gpu/nova-core/gsp.rs +99 -7
124 lines not on this page · raw message
drivers/gpu/nova-core/gsp/fw.rs +60 -4
92 lines not on this page · raw message
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +19 -0
@@ -124,3 +124,22 @@ fn default() -> Self { } } } +pub type LibosAddress =u64_ ; +pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_NONE :LibosMemoryRegionKind =0 ; +pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS :LibosMemoryRegionKind =1 ; +pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_RADIX3 :LibosMemoryRegionKind =2 ; +pub type LibosMemoryRegionKind = ffi::c_uint ; +pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_NONE :LibosMemoryRegionLoc =0 ; +pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM :LibosMemoryRegionLoc =1 ; +pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_FB :LibosMemoryRegionLoc =2 ; +pub type LibosMemoryRegionLoc = ffi::c_uint ; +#[repr( C )] +#[derive( Debug , Default , Copy , Clone )] +pub struct LibosMemoryRegionInitArgument { +pub id8:LibosAddress , +pub pa:LibosAddress , +pub size:LibosAddress , +pub kind:u8_ , +pub loc:u8_ , +pub __bindgen_padding_0: [u8 ;6usize ], +}
Signature · 1 line
2.50.1
Alexandre Courbot
maintainerRe: [PATCH v5 02/14] gpu: nova-core: Create initial Gsp
Quoted 76 lines · Alistair Popple
The GSP requires several areas of memory to operate. Each of these have their own simple embedded page tables. Set these up and map them for DMA to/from GSP using CoherentAllocation's. Return the DMA handle describing where each of these regions are for future use when booting GSP.
Signed-off-by Alistair Popple <[email protected]>---
Changes for v5:
- MoveGSP_HEAP_ALIGNMENTtogsp/fw.rsand add a comment.
- Create a LogBuffer type.
- Usechecked_addto ensure PTE values don't overflow.
- Added some type documentation (shamelessly stolen from Nouveau)Change for v3:
- Clean up the PTE array creation, with much thanks to Alex for doing most it (please let me know if I should put you as co-developer!)Changes for v2:
- Renamed GspMemOjbects to Gsp as that is what they are
- Rebased on Alex's latest series--- drivers/gpu/nova-core/gpu.rs | 2 +- drivers/gpu/nova-core/gsp.rs | 106 ++++++++++++++++-- drivers/gpu/nova-core/gsp/fw.rs | 64 ++++++++++- .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 19 ++++ 4 files changed, 179 insertions(+), 12 deletions(-)drivers/gpu/nova-core/gpu.rs +1
-1@@ -197,7 +197,7 @@ pub(crate) fn new<'a>( sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset, bar, true)?, - gsp <- Gsp::new(), + gsp <- Gsp::new(pdev)?, _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },drivers/gpu/nova-core/gsp.rs +11
-4@@ -2,25 +2,117 @@mod boot; +use kernel::device; +use kernel::dma::CoherentAllocation ; +use kernel::dma::DmaAddress ; +use kernel::dma_write; +use kernel::pci;use kernel::prelude::*; -use kernel::ptr::Alignment ; +use kernel::transmute::AsBytes ;pub (crate )use fw::{GspFwWprMeta ,LibosParams };mod fw; +use fw::LibosMemoryRegionInitArgument ; +pub (crate )const GSP_PAGE_SHIFT :usize =12 ;pub (crate )const GSP_PAGE_SIZE :usize =1 <<GSP_PAGE_SHIFT ; -pub (crate )const GSP_HEAP_ALIGNMENT :Alignment =Alignment ::new ::<{1 <<20 }>(); + +/// Number of GSP pages to use in a RM log buffer. +const RM_LOG_BUFFER_NUM_PAGES :usize =0x10 ;/// GSP runtime data. -/// -/// This is an empty pinned placeholder for now. #[pin_data]
drivers/gpu/nova-core/gsp.rs line 35
Quoted 10 lines
drivers/gpu/nova-core/gsp.rs +9
-1-pub (crate )struct Gsp {} +pub (crate )struct Gsp { +pub (crate ) libos:CoherentAllocation <LibosMemoryRegionInitArgument >, + loginit:LogBuffer , + logintr:LogBuffer , + logrm:LogBuffer , +} + +#[repr( C )] +struct PteArray <const NUM_ENTRIES :usize >([u64 ;NUM_ENTRIES ]);
I'd just document this structure a bit as it is not obvious what it does from the name alone.
Quoted 21 lines
drivers/gpu/nova-core/gsp.rs +21
-0+ +/// SAFETY: arrays of `u64` implement `AsBytes` and we are but a wrapper around it. +unsafe impl <const NUM_ENTRIES :usize >AsBytes for PteArray <NUM_ENTRIES > {} + +impl <const NUM_PAGES :usize >PteArray <NUM_PAGES > { +fn new (handle:DmaAddress ) ->Result <Self > { +let mut ptes = [0u64 ;NUM_PAGES ]; +for (i, pte)in ptes.iter_mut ().enumerate () { + *pte = handle + .checked_add ((ias u64 ) <<GSP_PAGE_SHIFT ) + .ok_or (EOVERFLOW )?; + } + +Ok (Self (ptes)) + } +} + +/// The logging buffers are byte queues that contain encoded printf-like +/// messages from GSP-RM. They need to be decoded by a special application +/// that can parse the buffers. +///
drivers/gpu/nova-core/gsp.rs line 66
Quoted 10 lines
drivers/gpu/nova-core/gsp.rs +10
-0+/// The 'loginit' buffer contains logs from early GSP-RM init and +/// exception dumps. The 'logrm' buffer contains the subsequent logs. Both are +/// written to directly by GSP-RM and can be any multiple of GSP_PAGE_SIZE. +/// +/// The physical address map for the log buffer is stored in the buffer +/// itself, starting with offset 1. Offset 0 contains the "put" pointer (pp). +/// Initially, pp is equal to 0. If the buffer has valid logging data in it, +/// then pp points to index into the buffer where the next logging entry will +/// be written. Therefore, the logging data is valid if: +/// 1 <= pp < sizeof(buffer)/sizeof(u64)
Maybe we should mention what happens to the address map, namely that it gets overwritten by the buffer data and is only used for the initial setup.
Quoted 35 lines
drivers/gpu/nova-core/gsp.rs +31
-2+struct LogBuffer (CoherentAllocation <u8 >); + +impl LogBuffer { +fn new (dev: &device::Device <device::Bound >) ->Result <Self > { +const NUM_PAGES :usize =RM_LOG_BUFFER_NUM_PAGES ; + +let mut obj =Self (CoherentAllocation ::<u8 >::alloc_coherent ( + dev, +NUM_PAGES *GSP_PAGE_SIZE , +GFP_KERNEL | __GFP_ZERO, + )?); +let ptes =PteArray ::<NUM_PAGES >::new (obj.0 .dma_handle ())?; + +// SAFETY: `obj` has just been created and we are its sole user. +unsafe { +// Copy the self-mapping PTE at the expected location. + obj.0 + .as_slice_mut (size_of ::<u64 >(),size_of_val (&ptes))? + .copy_from_slice (ptes.as_bytes ()) + }; + +Ok (obj) + } +}impl Gsp { -pub (crate )fn new () ->impl PinInit <Self > { -pin_init! (Self {}) +pub (crate )fn new (pdev: &pci::Device <device::Bound >) ->Result <impl PinInit <Self ,Error >> { +let dev = pdev.as_ref (); +let libos =CoherentAllocation ::<LibosMemoryRegionInitArgument >::alloc_coherent ( + dev, +GSP_PAGE_SIZE /size_of ::<LibosMemoryRegionInitArgument >(), +GFP_KERNEL | __GFP_ZERO, + )?;
drivers/gpu/nova-core/gsp.rs line 109
Quoted 10 lines
drivers/gpu/nova-core/gsp.rs +10
-0+ +// Initialise the logging structures. The OpenRM equivalents are in: +// _kgspInitLibosLoggingStructures (allocates memory for buffers) +// kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array) +let loginit =LogBuffer ::new (dev)?; +dma_write! (libos[0 ] =LibosMemoryRegionInitArgument ::new ("LOGINIT" , &loginit.0 )?)?; +let logintr =LogBuffer ::new (dev)?; +dma_write! (libos[1 ] =LibosMemoryRegionInitArgument ::new ("LOGINTR" , &logintr.0 )?)?; +let logrm =LogBuffer ::new (dev)?; +dma_write! (libos[2 ] =LibosMemoryRegionInitArgument ::new ("LOGRM" , &logrm.0 )?)?;
Let's maybe add a space before each "let" statement.
Quoted 95 lines
drivers/gpu/nova-core/gsp.rs +7
-0+ +Ok (try_pin_init! (Self { + libos, + loginit, + logintr, + logrm, + })) } }drivers/gpu/nova-core/gsp/fw.rs +46
-4@@ -7,15 +7,20 @@use core::ops::Range ; -use kernel::ptr::Alignable ; +use kernel::dma::CoherentAllocation ; +use kernel::prelude::*; +use kernel::ptr::{Alignable ,Alignment };use kernel::sizes::SZ_1M ; +use kernel::transmute::{AsBytes ,FromBytes };use crate ::gpu::Chipset ; -use crate ::gsp;/// Dummy type to group methods related to heap parameters for running the GSP firmware. pub (crate )struct GspFwHeapParams (()); +/// Minimum required alignment for the GSP heap. +const GSP_HEAP_ALIGNMENT :Alignment =Alignment ::new ::<{1 <<20 }>(); +impl GspFwHeapParams {/// Returns the amount of GSP-RM heap memory used during GSP-RM boot and initialization (up to /// and including the first client subdevice allocation). @@ -29,7 +34,7 @@ fn base_rm_size(_chipset: Chipset) -> u64 {/// Returns the amount of heap memory required to support a single channel allocation. fn client_alloc_size () ->u64 { u64::from (bindings::GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE ) - .align_up (gsp::GSP_HEAP_ALIGNMENT ) + .align_up (GSP_HEAP_ALIGNMENT ) .unwrap_or (u64::MAX ) }@@ -40,7 +45,7 @@ fn management_overhead(fb_size: u64) -> u64 { u64::from (bindings::GSP_FW_HEAP_PARAM_SIZE_PER_GB_FB ) .saturating_mul (fb_size_gb) - .align_up (gsp::GSP_HEAP_ALIGNMENT ) + .align_up (GSP_HEAP_ALIGNMENT ) .unwrap_or (u64::MAX ) } }@@ -99,3 +104,54 @@ pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 {/// addresses of the GSP bootloader and firmware. #[repr(transparent)] pub (crate )struct GspFwWprMeta (bindings::GspFwWprMeta ); + +/// Struct containing the arguments required to pass a memory buffer to the GSP +/// for use during initialisation. +/// +/// The GSP only understands 4K pages (GSP_PAGE_SIZE), so even if the kernel is +/// configured for a larger page size (e.g. 64K pages), we need to give +/// the GSP an array of 4K pages. Since we only create physically contiguous +/// buffers the math to calculate the addresses is simple. +/// +/// The buffers must be a multiple of GSP_PAGE_SIZE. GSP-RM also currently +/// ignores the @kind field for LOGINIT, LOGINTR, and LOGRM, but expects the +/// buffers to be physically contiguous anyway. +/// +/// The memory allocated for the arguments must remain until the GSP sends the +/// init_done RPC. +#[repr(transparent)] +pub (crate )struct LibosMemoryRegionInitArgument (bindings::LibosMemoryRegionInitArgument ); + +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for LibosMemoryRegionInitArgument {} + +// SAFETY: This struct only contains integer types for which all bit patterns +// are valid. +unsafe impl FromBytes for LibosMemoryRegionInitArgument {} + +impl LibosMemoryRegionInitArgument { +pub (crate )fn new <A :AsBytes +FromBytes >( + name: &'staticstr , + obj: &CoherentAllocation <A >, + ) ->Result <Self > { +/// Generates the `ID8` identifier required for some GSP objects. +fn id8 (name: &str ) ->u64 { +let mut bytes = [0u8 ; core::mem::size_of ::<u64 >()]; + +for (c, b)in name.bytes ().rev ().zip (&mut bytes) { + *b = c; + }
drivers/gpu/nova-core/gsp/fw.rs pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 { · line 153
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ + u64::from_ne_bytes (bytes) + } + + Ok(Self(bindings::LibosMemoryRegionInitArgument { + id8: id8(name), + pa: obj.dma_handle(), + size: obj.size() as u64, + kind: bindings::LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS.try_into()?, + loc: bindings::LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM.try_into()?,
The unneeded runtime check is a bit unfortunate, and its removal would allow us to make this method non-fallible, but I cannot find a good alternative that also doesn't clutter the code. Can't wait for const traits methods! :)
Re: [PATCH v5 02/14] gpu: nova-core: Create initial Gsp
On 2025-10-16 at 17:22 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 82 lines · Alistair Popple
Quoted 81 lines
The GSP requires several areas of memory to operate. Each of these have their own simple embedded page tables. Set these up and map them for DMA to/from GSP using CoherentAllocation's. Return the DMA handle describing where each of these regions are for future use when booting GSP.
Signed-off-by Alistair Popple <[email protected]>---
Changes for v5:
- MoveGSP_HEAP_ALIGNMENTtogsp/fw.rsand add a comment.
- Create a LogBuffer type.
- Usechecked_addto ensure PTE values don't overflow.
- Added some type documentation (shamelessly stolen from Nouveau)Change for v3:
- Clean up the PTE array creation, with much thanks to Alex for doing most it (please let me know if I should put you as co-developer!)Changes for v2:
- Renamed GspMemOjbects to Gsp as that is what they are
- Rebased on Alex's latest series--- drivers/gpu/nova-core/gpu.rs | 2 +- drivers/gpu/nova-core/gsp.rs | 106 ++++++++++++++++-- drivers/gpu/nova-core/gsp/fw.rs | 64 ++++++++++- .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 19 ++++ 4 files changed, 179 insertions(+), 12 deletions(-)drivers/gpu/nova-core/gpu.rs +1
-1@@ -197,7 +197,7 @@ pub(crate) fn new<'a>( sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset, bar, true)?, - gsp <- Gsp::new(), + gsp <- Gsp::new(pdev)?, _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },drivers/gpu/nova-core/gsp.rs +15
-5@@ -2,25 +2,117 @@mod boot; +use kernel::device; +use kernel::dma::CoherentAllocation ; +use kernel::dma::DmaAddress ; +use kernel::dma_write; +use kernel::pci;use kernel::prelude::*; -use kernel::ptr::Alignment ; +use kernel::transmute::AsBytes ;pub (crate )use fw::{GspFwWprMeta ,LibosParams };mod fw; +use fw::LibosMemoryRegionInitArgument ; +pub (crate )const GSP_PAGE_SHIFT :usize =12 ;pub (crate )const GSP_PAGE_SIZE :usize =1 <<GSP_PAGE_SHIFT ; -pub (crate )const GSP_HEAP_ALIGNMENT :Alignment =Alignment ::new ::<{1 <<20 }>(); + +/// Number of GSP pages to use in a RM log buffer. +const RM_LOG_BUFFER_NUM_PAGES :usize =0x10 ;/// GSP runtime data. -/// -/// This is an empty pinned placeholder for now. #[pin_data] -pub (crate )struct Gsp {} +pub (crate )struct Gsp { +pub (crate ) libos:CoherentAllocation <LibosMemoryRegionInitArgument >, + loginit:LogBuffer , + logintr:LogBuffer ,
Quoted 8 lines
drivers/gpu/nova-core/gsp.rs line 35
Quoted 5 lines
drivers/gpu/nova-core/gsp.rs +5
-0+ logrm: LogBuffer, +} + +#[repr( C )] +struct PteArray <const NUM_ENTRIES :usize >([u64 ;NUM_ENTRIES ]);I'd just document this structure a bit as it is not obvious what it does from the name alone.
Ok. As discussed I'm going to put your Co-developed-by on v6 and leave this one to you given I think you're more familiar with it :-)
Quoted 27 lines
Quoted 27 lines
drivers/gpu/nova-core/gsp.rs +27
-0+ +/// SAFETY: arrays of `u64` implement `AsBytes` and we are but a wrapper around it. +unsafe impl <const NUM_ENTRIES :usize >AsBytes for PteArray <NUM_ENTRIES > {} + +impl <const NUM_PAGES :usize >PteArray <NUM_PAGES > { +fn new (handle:DmaAddress ) ->Result <Self > { +let mut ptes = [0u64 ;NUM_PAGES ]; +for (i, pte)in ptes.iter_mut ().enumerate () { + *pte = handle + .checked_add ((ias u64 ) <<GSP_PAGE_SHIFT ) + .ok_or (EOVERFLOW )?; + } + +Ok (Self (ptes)) + } +} + +/// The logging buffers are byte queues that contain encoded printf-like +/// messages from GSP-RM. They need to be decoded by a special application +/// that can parse the buffers. +/// +/// The 'loginit' buffer contains logs from early GSP-RM init and +/// exception dumps. The 'logrm' buffer contains the subsequent logs. Both are +/// written to directly by GSP-RM and can be any multiple of GSP_PAGE_SIZE. +/// +/// The physical address map for the log buffer is stored in the buffer +/// itself, starting with offset 1. Offset 0 contains the "put" pointer (pp).
Quoted 8 lines
drivers/gpu/nova-core/gsp.rs line 66
Quoted 4 lines
drivers/gpu/nova-core/gsp.rs +4
-0+/// Initially, pp is equal to 0. If the buffer has valid logging data in it, +/// then pp points to index into the buffer where the next logging entry will +/// be written. Therefore, the logging data is valid if: +/// 1 <= pp < sizeof(buffer)/sizeof(u64) Maybe we should mention what happens to the address map, namely that it gets overwritten by the buffer data and is only used for the initial setup.
Ditto.
Quoted 39 lines
Quoted 39 lines
drivers/gpu/nova-core/gsp.rs +35
-2+struct LogBuffer (CoherentAllocation <u8 >); + +impl LogBuffer { +fn new (dev: &device::Device <device::Bound >) ->Result <Self > { +const NUM_PAGES :usize =RM_LOG_BUFFER_NUM_PAGES ; + +let mut obj =Self (CoherentAllocation ::<u8 >::alloc_coherent ( + dev, +NUM_PAGES *GSP_PAGE_SIZE , +GFP_KERNEL | __GFP_ZERO, + )?); +let ptes =PteArray ::<NUM_PAGES >::new (obj.0 .dma_handle ())?; + +// SAFETY: `obj` has just been created and we are its sole user. +unsafe { +// Copy the self-mapping PTE at the expected location. + obj.0 + .as_slice_mut (size_of ::<u64 >(),size_of_val (&ptes))? + .copy_from_slice (ptes.as_bytes ()) + }; + +Ok (obj) + } +}impl Gsp { -pub (crate )fn new () ->impl PinInit <Self > { -pin_init! (Self {}) +pub (crate )fn new (pdev: &pci::Device <device::Bound >) ->Result <impl PinInit <Self ,Error >> { +let dev = pdev.as_ref (); +let libos =CoherentAllocation ::<LibosMemoryRegionInitArgument >::alloc_coherent ( + dev, +GSP_PAGE_SIZE /size_of ::<LibosMemoryRegionInitArgument >(), +GFP_KERNEL | __GFP_ZERO, + )?; + +// Initialise the logging structures. The OpenRM equivalents are in: +// _kgspInitLibosLoggingStructures (allocates memory for buffers) +// kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array)
Quoted 8 lines
drivers/gpu/nova-core/gsp.rs line 109
Quoted 6 lines
drivers/gpu/nova-core/gsp.rs +6
-0+let loginit =LogBuffer ::new (dev)?; +dma_write! (libos[0 ] =LibosMemoryRegionInitArgument ::new ("LOGINIT" , &loginit.0 )?)?; +let logintr =LogBuffer ::new (dev)?; +dma_write! (libos[1 ] =LibosMemoryRegionInitArgument ::new ("LOGINTR" , &logintr.0 )?)?; +let logrm =LogBuffer ::new (dev)?; +dma_write! (libos[2 ] =LibosMemoryRegionInitArgument ::new ("LOGRM" , &logrm.0 )?)?;Let's maybe add a space before each "let" statement.
Ok.
Quoted 110 lines
Quoted 95 lines
drivers/gpu/nova-core/gsp.rs +7
-0+ +Ok (try_pin_init! (Self { + libos, + loginit, + logintr, + logrm, + })) } }drivers/gpu/nova-core/gsp/fw.rs +46
-4@@ -7,15 +7,20 @@use core::ops::Range ; -use kernel::ptr::Alignable ; +use kernel::dma::CoherentAllocation ; +use kernel::prelude::*; +use kernel::ptr::{Alignable ,Alignment };use kernel::sizes::SZ_1M ; +use kernel::transmute::{AsBytes ,FromBytes };use crate ::gpu::Chipset ; -use crate ::gsp;/// Dummy type to group methods related to heap parameters for running the GSP firmware. pub (crate )struct GspFwHeapParams (()); +/// Minimum required alignment for the GSP heap. +const GSP_HEAP_ALIGNMENT :Alignment =Alignment ::new ::<{1 <<20 }>(); +impl GspFwHeapParams {/// Returns the amount of GSP-RM heap memory used during GSP-RM boot and initialization (up to /// and including the first client subdevice allocation). @@ -29,7 +34,7 @@ fn base_rm_size(_chipset: Chipset) -> u64 {/// Returns the amount of heap memory required to support a single channel allocation. fn client_alloc_size () ->u64 { u64::from (bindings::GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE ) - .align_up (gsp::GSP_HEAP_ALIGNMENT ) + .align_up (GSP_HEAP_ALIGNMENT ) .unwrap_or (u64::MAX ) }@@ -40,7 +45,7 @@ fn management_overhead(fb_size: u64) -> u64 { u64::from (bindings::GSP_FW_HEAP_PARAM_SIZE_PER_GB_FB ) .saturating_mul (fb_size_gb) - .align_up (gsp::GSP_HEAP_ALIGNMENT ) + .align_up (GSP_HEAP_ALIGNMENT ) .unwrap_or (u64::MAX ) } }@@ -99,3 +104,54 @@ pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 {/// addresses of the GSP bootloader and firmware. #[repr(transparent)] pub (crate )struct GspFwWprMeta (bindings::GspFwWprMeta ); + +/// Struct containing the arguments required to pass a memory buffer to the GSP +/// for use during initialisation. +/// +/// The GSP only understands 4K pages (GSP_PAGE_SIZE), so even if the kernel is +/// configured for a larger page size (e.g. 64K pages), we need to give +/// the GSP an array of 4K pages. Since we only create physically contiguous +/// buffers the math to calculate the addresses is simple. +/// +/// The buffers must be a multiple of GSP_PAGE_SIZE. GSP-RM also currently +/// ignores the @kind field for LOGINIT, LOGINTR, and LOGRM, but expects the +/// buffers to be physically contiguous anyway. +/// +/// The memory allocated for the arguments must remain until the GSP sends the +/// init_done RPC. +#[repr(transparent)] +pub (crate )struct LibosMemoryRegionInitArgument (bindings::LibosMemoryRegionInitArgument ); + +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for LibosMemoryRegionInitArgument {} + +// SAFETY: This struct only contains integer types for which all bit patterns +// are valid. +unsafe impl FromBytes for LibosMemoryRegionInitArgument {} + +impl LibosMemoryRegionInitArgument { +pub (crate )fn new <A :AsBytes +FromBytes >( + name: &'staticstr , + obj: &CoherentAllocation <A >, + ) ->Result <Self > { +/// Generates the `ID8` identifier required for some GSP objects. +fn id8 (name: &str ) ->u64 { +let mut bytes = [0u8 ; core::mem::size_of ::<u64 >()]; + +for (c, b)in name.bytes ().rev ().zip (&mut bytes) { + *b = c; + }drivers/gpu/nova-core/gsp/fw.rs pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 { · line 153
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ + u64::from_ne_bytes (bytes) + } + + Ok(Self(bindings::LibosMemoryRegionInitArgument { + id8: id8(name), + pa: obj.dma_handle(), + size: obj.size() as u64, + kind: bindings::LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS.try_into()?, + loc: bindings::LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM.try_into()?,The unneeded runtime check is a bit unfortunate, and its removal would allow us to make this method non-fallible, but I cannot find a good alternative that also doesn't clutter the code. Can't wait for const traits methods! :)
PATCH v5 3/14 gpu: nova-core: gsp: Create wpr metadata
In reply to Alistair PoppleThe GSP requires some pieces of metadata to boot. These are passed in a struct which the GSP transfers via DMA. Create this struct and get a handle to it for future use when booting the GSP.
---
Changes for v5:
- Make member visibility match the struct visibility (thanks Danilo)
Changes for v3:
- Don't re-export WPR constants (thanks Alex)
Changes for v2:
- Rebased on Alex's latest version
--- drivers/gpu/nova-core/fb.rs | 1 - drivers/gpu/nova-core/firmware/gsp.rs | 3 +- drivers/gpu/nova-core/firmware/riscv.rs | 6 +- drivers/gpu/nova-core/gsp.rs | 2 + drivers/gpu/nova-core/gsp/boot.rs | 7 +++ drivers/gpu/nova-core/gsp/fw.rs | 55 ++++++++++++++++++- .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 2 + 7 files changed, 69 insertions(+), 7 deletions(-)
drivers/gpu/nova-core/fb.rs +0 -1
@@ -87,7 +87,6 @@ pub(crate) fn unregister(&self, bar: &Bar0) {/// /// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process. #[derive( Debug )] -#[expect(dead_code)] pub (crate )struct FbLayout {/// Range of the framebuffer. Starts at `0`. pub (crate ) fb:Range <u64 >,
drivers/gpu/nova-core/firmware/gsp.rs +1 -2
@@ -131,7 +131,7 @@ pub(crate) struct GspFirmware {/// Size in bytes of the firmware contained in [`Self::fw`]. pub size: usize,/// Device-mapped GSP signatures matching the GPU's [`Chipset`]. - signatures: DmaObject, + pub signatures: DmaObject,/// GSP bootloader, verifies the GSP firmware before loading and running it. pub bootloader: RiscvFirmware, }
@@ -216,7 +216,6 @@ pub(crate) fn new<'a, 'b>( })) } -#[expect(unused)] /// Returns the DMA handle of the radix3 level 0 page table. pub (crate )fn radix3_dma_handle (&self) ->DmaAddress { self.level0.dma_handle ()
drivers/gpu/nova-core/firmware/riscv.rs +3 -3
@@ -55,11 +55,11 @@ fn new(bin_fw: &BinFirmware<'_>) -> Result<Self> {#[expect(unused)] pub (crate )struct RiscvFirmware {/// Offset at which the code starts in the firmware image. - code_offset:u32 , +pub (crate ) code_offset:u32 ,/// Offset at which the data starts in the firmware image. - data_offset:u32 , +pub (crate ) data_offset:u32 ,/// Offset at which the manifest starts in the firmware image. - manifest_offset:u32 , +pub (crate ) manifest_offset:u32 ,/// Application version. app_version:u32 ,/// Device-mapped firmware image.
drivers/gpu/nova-core/gsp.rs +2 -0
@@ -10,6 +10,8 @@use kernel::prelude::*;use kernel::transmute::AsBytes ; +use crate ::fb::FbLayout ; +pub (crate )use fw::{GspFwWprMeta ,LibosParams };mod fw;
drivers/gpu/nova-core/gsp/boot.rs +7 -0
@@ -1,6 +1,8 @@// SPDX-License-Identifier: GPL-2.0 use kernel::device; +use kernel::dma::CoherentAllocation ; +use kernel::dma_write;use kernel::pci;use kernel::prelude::*;
@@ -14,6 +16,7 @@ FIRMWARE_VERSION, };use crate ::gpu::Chipset ; +use crate ::gsp::GspFwWprMeta ;use crate ::regs;use crate ::vbios::Vbios ;
@@ -132,6 +135,10 @@ pub(crate) fn boot( bar, )?; +let wpr_meta = +CoherentAllocation ::<GspFwWprMeta >::alloc_coherent (dev,1 ,GFP_KERNEL | __GFP_ZERO)?; +dma_write! (wpr_meta[0 ] =GspFwWprMeta ::new (&gsp_fw, &fb_layout))?; +Ok (()) } }
drivers/gpu/nova-core/gsp/fw.rs +54 -1
70 lines not on this page · raw message
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +2 -0
@@ -9,6 +9,8 @@pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB :u32 =256 ;pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB :u32 =88 ;pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB :u32 =280 ; +pub const GSP_FW_WPR_META_REVISION :u32 =1 ; +pub const GSP_FW_WPR_META_MAGIC :i64 = -2577556379034558285 ;pub type __u8 = ffi::c_uchar ;pub type __u16 = ffi::c_ushort ;pub type __u32 = ffi::c_uint ;
Signature · 1 line
2.50.1
Alexandre Courbot
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
Quoted 43 lines · Alistair Popple
The GSP requires some pieces of metadata to boot. These are passed in a struct which the GSP transfers via DMA. Create this struct and get a handle to it for future use when booting the GSP.
Signed-off-by Alistair Popple <[email protected]>---
Changes for v5:
- Make member visibility match the struct visibility (thanks Danilo)Changes for v3:
- Don't re-export WPR constants (thanks Alex)Changes for v2:
- Rebased on Alex's latest version--- drivers/gpu/nova-core/fb.rs | 1 - drivers/gpu/nova-core/firmware/gsp.rs | 3 +- drivers/gpu/nova-core/firmware/riscv.rs | 6 +- drivers/gpu/nova-core/gsp.rs | 2 + drivers/gpu/nova-core/gsp/boot.rs | 7 +++ drivers/gpu/nova-core/gsp/fw.rs | 55 ++++++++++++++++++- .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 2 + 7 files changed, 69 insertions(+), 7 deletions(-)drivers/gpu/nova-core/fb.rs +0
-1@@ -87,7 +87,6 @@ pub(crate) fn unregister(&self, bar: &Bar0) {/// /// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process. #[derive( Debug )] -#[expect(dead_code)] pub (crate )struct FbLayout {/// Range of the framebuffer. Starts at `0`. pub (crate ) fb:Range <u64 >,drivers/gpu/nova-core/firmware/gsp.rs +0
-0
drivers/gpu/nova-core/firmware/gsp.rs pub(crate) struct GspFirmware { · line 134
Quoted 5 lines
drivers/gpu/nova-core/firmware/gsp.rs +1
-1@@ -131,7 +131,7 @@ pub(crate) struct GspFirmware {/// Size in bytes of the firmware contained in [`Self::fw`]. pub size: usize,/// Device-mapped GSP signatures matching the GPU's [`Chipset`]. - signatures: DmaObject, + pub signatures: DmaObject,
This needs to be pub(crate) or rustc 1.78 emits a warning (I'll also have to fix the surrounding ones in my own patch).
<snip>
drivers/gpu/nova-core/firmware/gsp.rs pub(crate) struct GspFirmware {
Quoted 7 lines
drivers/gpu/nova-core/firmware/gsp.rs +7
-0+impl GspFwWprMeta { +pub (crate )fn new (gsp_firmware: &GspFirmware , fb_layout: &FbLayout ) ->Self { +Self (bindings::GspFwWprMeta { + magic: r570_144::GSP_FW_WPR_META_MAGIC as u64 , + revision: u64::from (r570_144::GSP_FW_WPR_META_REVISION ), + sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle (), + sizeOfRadix3Elf: gsp_firmware.sizeas u64 ,
Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.
Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 44 lines · Alistair Popple
Quoted 43 lines
The GSP requires some pieces of metadata to boot. These are passed in a struct which the GSP transfers via DMA. Create this struct and get a handle to it for future use when booting the GSP.
Signed-off-by Alistair Popple <[email protected]>---
Changes for v5:
- Make member visibility match the struct visibility (thanks Danilo)Changes for v3:
- Don't re-export WPR constants (thanks Alex)Changes for v2:
- Rebased on Alex's latest version--- drivers/gpu/nova-core/fb.rs | 1 - drivers/gpu/nova-core/firmware/gsp.rs | 3 +- drivers/gpu/nova-core/firmware/riscv.rs | 6 +- drivers/gpu/nova-core/gsp.rs | 2 + drivers/gpu/nova-core/gsp/boot.rs | 7 +++ drivers/gpu/nova-core/gsp/fw.rs | 55 ++++++++++++++++++- .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 2 + 7 files changed, 69 insertions(+), 7 deletions(-)drivers/gpu/nova-core/fb.rs +0
-1@@ -87,7 +87,6 @@ pub(crate) fn unregister(&self, bar: &Bar0) {/// /// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process. #[derive( Debug )] -#[expect(dead_code)] pub (crate )struct FbLayout {/// Range of the framebuffer. Starts at `0`. pub (crate ) fb:Range <u64 >,drivers/gpu/nova-core/firmware/gsp.rs +0
-0@@ -131,7 +131,7 @@ pub(crate) struct GspFirmware {
Quoted 8 lines
drivers/gpu/nova-core/firmware/gsp.rs pub(crate) struct GspFirmware { · line 134
Quoted 5 lines
drivers/gpu/nova-core/firmware/gsp.rs +1
-1/// Size in bytes of the firmware contained in [`Self::fw`]. pub size: usize,/// Device-mapped GSP signatures matching the GPU's [`Chipset`]. - signatures: DmaObject, + pub signatures: DmaObject,This needs to be
pub(crate)or rustc 1.78 emits a warning (I'll also have to fix the surrounding ones in my own patch).
Thanks, missed that one.
Quoted 11 lines
<snip>
drivers/gpu/nova-core/firmware/gsp.rs
Quoted 7 lines
drivers/gpu/nova-core/firmware/gsp.rs +7
-0+impl GspFwWprMeta { +pub (crate )fn new (gsp_firmware: &GspFirmware , fb_layout: &FbLayout ) ->Self { +Self (bindings::GspFwWprMeta { + magic: r570_144::GSP_FW_WPR_META_MAGIC as u64 , + revision: u64::from (r570_144::GSP_FW_WPR_META_REVISION ), + sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle (), + sizeOfRadix3Elf: gsp_firmware.sizeas u64 ,Very unfortunately I'm afraid we will need to replace the
asin this method withu64::try_fromand make it return aResultfor now.
And presumably most of the other as keywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?
Danilo Krummrich
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
Quoted 10 lines · Alistair Popple
On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7
-0+impl GspFwWprMeta { +pub (crate )fn new (gsp_firmware: &GspFirmware , fb_layout: &FbLayout ) ->Self { +Self (bindings::GspFwWprMeta { + magic: r570_144::GSP_FW_WPR_META_MAGIC as u64 , + revision: u64::from (r570_144::GSP_FW_WPR_META_REVISION ), + sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle (), + sizeOfRadix3Elf: gsp_firmware.sizeas u64 ,
Quoted 7 lines
Quoted 2 lines
Very unfortunately I'm afraid we will need to replace the
asin this method withu64::try_fromand make it return aResultfor now.And presumably most of the other
askeywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?
Since nova-core depends on CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.
This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.
We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.
Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
Quoted 10 lines · Danilo Krummrich
Since nova-core depends on
CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.
We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.
Indeed.
Cheers,
Miguel
Alexandre Courbot
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
In reply to Danilo KrummrichQuoted 11 lines · Danilo Krummrich
Quoted 10 lines · Alistair Popple
On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7
-0+impl GspFwWprMeta { +pub (crate )fn new (gsp_firmware: &GspFirmware , fb_layout: &FbLayout ) ->Self { +Self (bindings::GspFwWprMeta { + magic: r570_144::GSP_FW_WPR_META_MAGIC as u64 , + revision: u64::from (r570_144::GSP_FW_WPR_META_REVISION ), + sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle (), + sizeOfRadix3Elf: gsp_firmware.sizeas u64 ,
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Very unfortunately I'm afraid we will need to replace the
asin this method withu64::try_fromand make it return aResultfor now.And presumably most of the other
askeywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?
Indeed, the concern is that as performs a lossy conversion without warning, which could catch us off-guard if e.g. some type changes in the bindings.
Quoted 10 lines
Since nova-core depends on
CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.
We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.
That would definitely be the correct way to address this.
Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
On 2025-10-17 at 11:43 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 28 lines · Danilo Krummrich
Quoted 11 lines · Alistair Popple
Quoted 10 lines
On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7
-0+impl GspFwWprMeta { +pub (crate )fn new (gsp_firmware: &GspFirmware , fb_layout: &FbLayout ) ->Self { +Self (bindings::GspFwWprMeta { + magic: r570_144::GSP_FW_WPR_META_MAGIC as u64 , + revision: u64::from (r570_144::GSP_FW_WPR_META_REVISION ), + sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle (), + sizeOfRadix3Elf: gsp_firmware.sizeas u64 ,Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Very unfortunately I'm afraid we will need to replace the
asin this method withu64::try_fromand make it return aResultfor now.And presumably most of the other
askeywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?Indeed, the concern is that
asperforms a lossy conversion without warning, which could catch us off-guard if e.g. some type changes in the bindings.Quoted 4 lines
Since nova-core depends on
CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.
Quoted 8 lines
Quoted 6 lines
This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.
We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.
That would definitely be the correct way to address this.
Sure. Although I don't really have a problem with the binding constructors being fallible as plenty of the others are anyway.
Alexandre Courbot
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
Quoted 32 lines · Alistair Popple
On 2025-10-17 at 11:43 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 31 lines · Danilo Krummrich
Quoted 11 lines · Alistair Popple
Quoted 10 lines
On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7
-0+impl GspFwWprMeta { +pub (crate )fn new (gsp_firmware: &GspFirmware , fb_layout: &FbLayout ) ->Self { +Self (bindings::GspFwWprMeta { + magic: r570_144::GSP_FW_WPR_META_MAGIC as u64 , + revision: u64::from (r570_144::GSP_FW_WPR_META_REVISION ), + sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle (), + sizeOfRadix3Elf: gsp_firmware.sizeas u64 ,Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Very unfortunately I'm afraid we will need to replace the
asin this method withu64::try_fromand make it return aResultfor now.And presumably most of the other
askeywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?Indeed, the concern is that
asperforms a lossy conversion without warning, which could catch us off-guard if e.g. some type changes in the bindings.Quoted 7 lines
Since nova-core depends on
CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.
Quoted 8 lines
Quoted 5 lines
Quoted 3 lines
We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.
That would definitely be the correct way to address this.
Sure. Although I don't really have a problem with the binding constructors being fallible as plenty of the others are anyway.
I think we can address the non-fallible conversions as a separate task (as there are many sites that could be similarly optimized in the nova-core code), so fallible constructors are acceptable imho.
Danilo Krummrich
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
Quoted 37 lines · Alexandre Courbot
Quoted 36 lines · Alistair Popple
On 2025-10-17 at 11:43 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 35 lines · Danilo Krummrich
Quoted 11 lines · Alistair Popple
Quoted 10 lines
On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 9 lines · Alistair Popple
Quoted 7 lines
(quoted diff) +7
-0+impl GspFwWprMeta { + pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self { + Self(bindings::GspFwWprMeta { + magic: r570_144::GSP_FW_WPR_META_MAGIC as u64, + revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION), + sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(), + sizeOfRadix3Elf: gsp_firmware.size as u64,Quoted 7 lines
Quoted 7 lines
Quoted 2 lines
Very unfortunately I'm afraid we will need to replace the
asin this method withu64::try_fromand make it return aResultfor now.And presumably most of the other
askeywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?Indeed, the concern is that
asperforms a lossy conversion without warning, which could catch us off-guard if e.g. some type changes in the bindings.Quoted 10 lines
Since nova-core depends on
CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.
We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
That would definitely be the correct way to address this.
Sure. Although I don't really have a problem with the binding constructors being fallible as plenty of the others are anyway.
I think we can address the non-fallible conversions as a separate task (as there are many sites that could be similarly optimized in the nova-core code), so fallible constructors are acceptable imho.
The infallible conversion function is trivial to implement.
If you prefer, we can also add it in nova-core first and subsequently move it to generic infrastructure.
I prefer not to introduce more as-casts or fallible conversions we have to clean up subsequently.
Alexandre Courbot
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
In reply to Danilo KrummrichQuoted 22 lines · Danilo Krummrich
Quoted 10 lines · Alistair Popple
On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7
-0+impl GspFwWprMeta { +pub (crate )fn new (gsp_firmware: &GspFirmware , fb_layout: &FbLayout ) ->Self { +Self (bindings::GspFwWprMeta { + magic: r570_144::GSP_FW_WPR_META_MAGIC as u64 , + revision: u64::from (r570_144::GSP_FW_WPR_META_REVISION ), + sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle (), + sizeOfRadix3Elf: gsp_firmware.sizeas u64 ,Quoted 7 lines
Quoted 2 lines
Very unfortunately I'm afraid we will need to replace the
asin this method withu64::try_fromand make it return aResultfor now.And presumably most of the other
askeywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?Since nova-core depends on
CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.
Quoted 6 lines
This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.
We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.
How do you see this taking shape concretely? I have tried writing extention traits (e.g. IntoU64), but unfortunately we also need to use these in const contexts so that won't until const trait methods are a thing.
The alternative would be to have const functions like usize_to_u64. It doesn't look as smooth as the extention trait, but can be used in const contexts.
I can try and provide an implementation to apply before this patch series if given guidance on which direction we want to go.
Danilo Krummrich
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
Quoted 3 lines · Alexandre Courbot
The alternative would be to have const functions like
usize_to_u64. It doesn't look as smooth as the extention trait, but can be used in const contexts.
That's what I thought of, exactly for the reason of being usable in const contexts (at least for a quick fix in nova-core).
Whether we want an extention trait and a separate temporary const_usize_to_u64() etc. can be discussed in the context of making it common infrastructure.
Alexandre Courbot
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
Quoted 10 lines · Danilo Krummrich
Quoted 3 lines · Alexandre Courbot
The alternative would be to have const functions like
usize_to_u64. It doesn't look as smooth as the extention trait, but can be used in const contexts.That's what I thought of, exactly for the reason of being usable in const contexts (at least for a quick fix in nova-core).
Whether we want an extention trait and a separate temporary
const_usize_to_u64()etc. can be discussed in the context of making it common infrastructure.
Thanks - so IIUC the idea would be to keep this local to nova-core in a first time? If so I guess I can produce this fast (and convert our many uses of as in the driver so far).
Danilo Krummrich
maintainerRe: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata
Quoted 9 lines · Alexandre Courbot
Quoted 8 lines · Danilo Krummrich
Quoted 3 lines · Alexandre Courbot
The alternative would be to have const functions like
usize_to_u64. It doesn't look as smooth as the extention trait, but can be used in const contexts.That's what I thought of, exactly for the reason of being usable in const contexts (at least for a quick fix in nova-core).
Quoted 6 lines
Quoted 2 lines
Whether we want an extention trait and a separate temporary
const_usize_to_u64()etc. can be discussed in the context of making it common infrastructure.Thanks - so IIUC the idea would be to keep this local to nova-core in a first time? If so I guess I can produce this fast (and convert our many uses of
asin the driver so far).
Yeah, I think that's reasonable. We can do both in parallel.
PATCH v5 4/14 gpu: nova-core: Add a slice-buffer (sbuffer) datastructure
In reply to Alistair PoppleFrom: Joel Fernandes <[email protected]>
A data structure that can be used to write across multiple slices which may be out of order in memory. This lets SBuffer user correctly and safely write out of memory order, without error-prone tracking of pointers/offsets.
let mut buf1 = [0u8; 3];
let mut buf2 = [0u8; 5];
let mut sbuffer = SBuffer::new([&mut buf1[..], &mut buf2[..]]);
let data = b"hello";
let result = sbuffer.write(data);
An internal conversion of gsp.rs to use this resulted in a nice -ve delta:gsp.rs: 37 insertions(+), 99 deletions(-)
---
Changes for v5:
- Typos
- s/ETOOSMALL/EINVAL/
- Add documentation
- Fix up examples
Changes for v3:
- Addressed minor review comment from Lyude
--- drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/sbuffer.rs | 218 +++++++++++++++++++++++++++++ 2 files changed, 219 insertions(+) create mode 100644 drivers/gpu/nova-core/sbuffer.rs
drivers/gpu/nova-core/nova_core.rs +1 -0
@@ -11,6 +11,7 @@mod gpu;mod gsp;mod regs; +mod sbuffer;mod util;mod vbios;
drivers/gpu/nova-core/sbuffer.rs new file +218 -0
218 lines not on this page · raw message
Signature · 1 line
2.50.1
Alexandre Courbot
maintainerRe: [PATCH v5 04/14] gpu: nova-core: Add a slice-buffer (sbuffer) datastructure
Quoted 69 lines · Alistair Popple
From: Joel Fernandes <[email protected]>
A data structure that can be used to write across multiple slices which may be out of order in memory. This lets SBuffer user correctly and safely write out of memory order, without error-prone tracking of pointers/offsets.
let mut buf1 = [0u8; 3];
let mut buf2 = [0u8; 5];
let mut sbuffer =SBuffer::new([&mut buf1[..], &mut buf2[..]]);let data = b"hello";
let result =sbuffer.write(data);An internal conversion of
gsp.rsto use this resulted in a nice -ve delta:gsp.rs: 37 insertions(+), 99 deletions(-)Co-developed-by Alistair Popple <[email protected]>Signed-off-by Alistair Popple <[email protected]>Signed-off-by Joel Fernandes <[email protected]>Reviewed-by Lyude Paul <[email protected]>---
Changes for v5:
- Typos
- s/ETOOSMALL/EINVAL/
- Add documentation
- Fix up examplesChanges for v3:
- Addressed minor review comment from Lyude--- drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/sbuffer.rs | 218 +++++++++++++++++++++++++++++ 2 files changed, 219 insertions(+) create mode 100644 drivers/gpu/nova-core/sbuffer.rsdrivers/gpu/nova-core/nova_core.rs +1
-0@@ -11,6 +11,7 @@mod gpu;mod gsp;mod regs; +mod sbuffer;mod util;mod vbios;drivers/gpu/nova-core/sbuffer.rs new file +13
-0@@ -0,0 +1,218 @@ +// SPDX-License-Identifier: GPL-2.0 + +use core::ops::Deref ; + +use kernel::alloc::KVec ; +use kernel::error::code::*; +use kernel::prelude::*; + +/// A buffer abstraction for discontiguous byte slices. +/// +/// This allows you to treat multiple non-contiguous `&mut [u8]` slices +/// of the same length as a single stream-like read/write buffer. +///
drivers/gpu/nova-core/sbuffer.rs line 23
Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+/// # Example: +/// +/// ``` +/// let mut buf1 = [0u8; 5]; +/// let mut buf2 = [0u8; 5]; +/// let mut sbuffer = SBufferIter::new_writer([&buf1, &buf2]); +/// +/// let data = b"hello"; +/// let result = sbuffer.write_all(data); +/// ```
This example doesn't build - there are several things wrong with it. It is also missing statements to confirm and show the expected result. Here is a fixed and slightly improved version:
/// let mut buf1 = [0u8; 5];
/// let mut buf2 = [0u8; 5];
/// let mut sbuffer = SBufferIter::new_writer([&mut buf1[..], &mut buf2[..]]); ///
/// let data = b"hi world!";
/// sbuffer.write_all(data)?;
/// drop(sbuffer);
///
/// assert_eq!(buf1, *b"hi wo");
/// assert_eq!(buf2, *b"rld!\0");
///
/// # Ok::<(), Error>(())
drivers/gpu/nova-core/sbuffer.rs line 30
Quoted 7 lines
drivers/gpu/nova-core/sbuffer.rs new file +7
-0+/// +/// A sliding window of slices to process. +/// +/// Both read and write buffers are implemented in terms of operating on slices of a requested +/// size. This base class implements logic that can be shared between the two to support that. +/// +/// `S` is a slice type, `I` is an iterator yielding `S`.
Why is there another doccomment after the example section? It looks like this should be merged with the first doccomment before the example? There is also no S generic parameter.
Quoted 50 lines
drivers/gpu/nova-core/sbuffer.rs new file +50
-0+pub (crate )struct SBufferIter <I :Iterator > { +/// `Some` if we are not at the end of the data yet. + cur_slice:Option <I ::Item >, +/// All the slices remaining after `cur_slice`. + slices:I , +} + +impl <'a,I >SBufferIter <I > +where +I :Iterator , +{ +/// Creates a reader buffer for a discontiguous set of byte slices. +/// +/// # Example: +/// +/// ``` +/// let buf1: [u8; 5] = [0, 1, 2, 3, 4]; +/// let buf2: [u8; 5] = [5, 6, 7, 8, 9]; +/// let sbuffer = SBufferIter::new_reader([&buf1[..], &buf2[..]]); +/// let sum: u8 = sbuffer.sum(); +/// assert_eq!(sum, 45); +/// ``` +#[expect(unused)] +pub (crate )fn new_reader (slices:impl IntoIterator <IntoIter =I >) ->Self +where +I :Iterator <Item = &'a [u8 ]>, + { +Self ::new (slices) + } + +/// Creates a writeable buffer for a discontiguous set of byte slices. +/// +/// # Example: +/// +/// ``` +/// let mut buf1 = [0u8; 5]; +/// let mut buf2 = [0u8; 5]; +/// let mut sbuffer = SBufferIter::new_writer([&mut buf1[..], &mut buf2[..]]); +/// sbuffer.write_all(&[0u8, 1, 2, 3, 4, 5, 6, 7, 8, 9][..])?; +/// drop(sbuffer); +/// assert_eq!(buf1, [0, 1, 2, 3, 4]); +/// assert_eq!(buf2, [5, 6, 7, 8, 9]); +/// +/// ``` +#[expect(unused)] +pub (crate )fn new_writer (slices:impl IntoIterator <IntoIter =I >) ->Self +where +I :Iterator <Item = &'amut [u8 ]>, + { +Self ::new (slices)
drivers/gpu/nova-core/sbuffer.rs line 90
Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+ } + +fn new (slices:impl IntoIterator <IntoIter =I >) ->Self +where +I ::Item :Deref <Target = [u8 ]>, + { +let mut slices = slices.into_iter (); + +Self { +// Skip empty slices to avoid trouble down the road.
I guess "Skip empty slices" is enough as it is part of the algorithm. :)
drivers/gpu/nova-core/sbuffer.rs line 100
Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+ cur_slice: slices.find(|s| !s.deref().is_empty()), + slices, + } + } + +fn get_slice_internal ( + &mut self, + len:usize , +mut f:impl FnMut (I ::Item ,usize ) -> (I ::Item ,I ::Item ), + ) ->Option <I ::Item >
Let's document this a bit as its purpose is not immediately clear. We can take the documentation from the get_slice methods, with a short explanation that the closure is supposed to split the slice received as first argument at the position given as the second.
Quoted 31 lines
drivers/gpu/nova-core/sbuffer.rs new file +31
-0+ where + I::Item: Deref<Target = [u8]>, + { +match self.cur_slice.take () { +None =>None , +Some (cur_slice) => { +if len >= cur_slice.len () { +// Caller requested more data than is in the current slice, return it entirely +// and prepare the following slice for being used. Skip empty slices to avoid +// trouble. + self.cur_slice = self.slices.find (|s| !s.is_empty ()); + +Some (cur_slice) + }else { +// The current slice can satisfy the request, split it and return a slice of +// the requested size. +let (ret, next) =f (cur_slice, len); + self.cur_slice =Some (next); + +Some (ret) + } + } + } + } +} + +/// Provides a way to get non-mutable slices of data to read from. +impl <'a,I >SBufferIter <I > +where +I :Iterator <Item = &'a [u8 ]>, +{
drivers/gpu/nova-core/sbuffer.rs line 141
Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+/// Returns a slice of at most `len` bytes, or `None` if we are at the end of the data. +/// +/// If a slice shorter than `len` bytes has been returned, the caller can call this method +/// again until it returns `None` to try and obtain the remainder of the data. +fn get_slice (&mut self, len:usize ) ->Option <&'a [u8 ]> { + self.get_slice_internal (len, |s, pos| s.split_at (pos)) + } + +/// Ideally we would implement `Read`, but it is not available in `core`. +/// So mimic `std::io::Read::read_exact`.
That's a useful side-comment, but we also need small sentence describing the method, e.g. "Fill dst with the next bytes from this SBufferIter, or return EINVAL if there isn't enough data available."
Quoted 11 lines
drivers/gpu/nova-core/sbuffer.rs new file +11
-0+#[expect(unused)] +pub (crate )fn read_exact (&mut self,mut dst: &mut [u8 ]) ->Result { +while !dst.is_empty () { +match self.get_slice (dst.len ()) { +None =>return Err (EINVAL ), +Some (src) => { +let dst_slice; + (dst_slice, dst) = dst.split_at_mut (src.len ()); + dst_slice.copy_from_slice (src); + } + }
drivers/gpu/nova-core/sbuffer.rs line 162
Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+ } + +Ok (()) + } + +/// Read all the remaining data into a `KVec`. +/// +/// `self` will be empty after this operation. +#[expect(unused)] +pub (crate )fn read_into_kvec (&mut self, flags: kernel::alloc::Flags ) ->Result <KVec <u8 >> {
nit: flush_into_kvec is probably a more descriptive name, as read is already used for the other method.
Quoted 18 lines
drivers/gpu/nova-core/sbuffer.rs new file +18
-0+let mut buf =KVec ::<u8 >::new (); + +if let Some (slice) = core::mem::take (&mut self.cur_slice) { + buf.extend_from_slice (slice, flags)?; + } +for slicein &mut self.slices { + buf.extend_from_slice (slice, flags)?; + } + +Ok (buf) + } +} + +/// Provides a way to get mutable slices of data to write into. +impl <'a,I >SBufferIter <I > +where +I :Iterator <Item = &'amut [u8 ]>, +{
drivers/gpu/nova-core/sbuffer.rs line 190
Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+/// Returns a mutable slice of at most `len` bytes, or `None` if we are at the end of the data. +/// +/// If a slice shorter than `len` bytes has been returned, the caller can call this method +/// again until it returns `None` to try and obtain the remainder of the data. +fn get_slice_mut (&mut self, len:usize ) ->Option <&'amut [u8 ]> { + self.get_slice_internal (len, |s, pos| s.split_at_mut (pos)) + } + +/// Ideally we would implement `Write`, but it is not available in `core`. +/// So mimic `std::io::Write::write_all`.
Same comment as read_all.
Re: [PATCH v5 04/14] gpu: nova-core: Add a slice-buffer (sbuffer) datastructure
Quoted 17 lines · Alexandre Courbot
drivers/gpu/nova-core/sbuffer.rs line 23
Quoted 11 lines · Alistair Popple
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+/// # Example: +/// +/// ``` +/// let mut buf1 = [0u8; 5]; +/// let mut buf2 = [0u8; 5]; +/// let mut sbuffer = SBufferIter::new_writer([&buf1, &buf2]); +/// +/// let data = b"hello"; +/// let result = sbuffer.write_all(data); +/// ``` This example doesn't build - there are several things wrong with it. It is also missing statements to confirm and show the expected result. Here is a fixed and slightly improved version:
Yeah, I mentioned this one in a previous version -- the section header is also still wrong too.
Alistair, please check the link I gave:
https://docs.kernel.org/rust/coding-guidelines.html#code-documentation
or other code in the kernel crate for examples on how it is usually done.
It is not critical today, of course, but the further it is from what will be needed in a few months, the harder it will become to start building the docs and running the examples as KUnit tests.
Thanks!
Cheers,
Miguel
Re: [PATCH v5 04/14] gpu: nova-core: Add a slice-buffer (sbuffer) datastructure
On 2025-10-17 at 06:18 +1100, Miguel Ojeda <[email protected]> wrote...
Quoted 18 lines · Alexandre Courbot
Quoted 17 lines · Alistair Popple
drivers/gpu/nova-core/sbuffer.rs line 23
Quoted 11 lines
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+/// # Example: +/// +/// ``` +/// let mut buf1 = [0u8; 5]; +/// let mut buf2 = [0u8; 5]; +/// let mut sbuffer = SBufferIter::new_writer([&buf1, &buf2]); +/// +/// let data = b"hello"; +/// let result = sbuffer.write_all(data); +/// ``` This example doesn't build - there are several things wrong with it. It is also missing statements to confirm and show the expected result. Here is a fixed and slightly improved version:
Argh, you're right. I cut and pasted then edited the wrong thing from my test build. How are you building these? The rustdoc target seems to ignore Nova (or I'm doing something wrong).
Quoted 4 lines
Yeah, I mentioned this one in a previous version -- the section header is also still wrong too.
Alistair, please check the link I gave:
Will do. I thought Joel had addressed your comments in the fix patch I pulled in from him (he wrote most of this originally) but I can see the /// # Example: heading is wrong.
Quoted 7 lines
https://docs.kernel.org/rust/coding-guidelines.html#code-documentation
or other code in the
kernelcrate for examples on how it is usually done.It is not critical today, of course, but the further it is from what will be needed in a few months, the harder it will become to start building the docs and running the examples as KUnit tests.
No, I think it *is* critical :-) Much easier just to get this right from the beginning than deal with heaps of errors/warnings later. It's just my fingers that are still getting used to the subtle differences between C kernel code and Rust kernel code, so thanks for the guidance.
- Alistair
Quoted 5 lines
Thanks!
Cheers,
Miguel
Alexandre Courbot
maintainerRe: [PATCH v5 04/14] gpu: nova-core: Add a slice-buffer (sbuffer) datastructure
Quoted 20 lines · Alistair Popple
On 2025-10-17 at 06:18 +1100, Miguel Ojeda <[email protected]> wrote...
Quoted 18 lines · Alexandre Courbot
Quoted 17 lines · Alistair Popple
drivers/gpu/nova-core/sbuffer.rs line 23
Quoted 11 lines
drivers/gpu/nova-core/sbuffer.rs new file +10
-0+/// # Example: +/// +/// ``` +/// let mut buf1 = [0u8; 5]; +/// let mut buf2 = [0u8; 5]; +/// let mut sbuffer = SBufferIter::new_writer([&buf1, &buf2]); +/// +/// let data = b"hello"; +/// let result = sbuffer.write_all(data); +/// ``` This example doesn't build - there are several things wrong with it. It is also missing statements to confirm and show the expected result. Here is a fixed and slightly improved version:
Quoted 3 lines
Argh, you're right. I cut and pasted then edited the wrong thing from my test build. How are you building these? The
rustdoctarget seems to ignore Nova (or I'm doing something wrong).
Indeed, rustdoc doesn't consider anything outside of the kernel crate for now, although this is scheduled to change so we need to be ready for it.
For now, I just copy/paste the test code into some function elsewhere and try to build. ^_^;
PATCH v5 5/14 gpu: nova-core: Add zeroable trait to bindings
In reply to Alistair PoppleDerive the Zeroable trait for existing bindgen generated bindings. This is safe because all bindgen generated types are simple integer types for which any bit pattern, including all zeros, is valid.
---
Changes for v5:
- New for v5
--- drivers/gpu/nova-core/gsp/fw/r570_144.rs | 1 + .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 16 ++++++++-------- 2 files changed, 9 insertions(+), 8 deletions(-)
drivers/gpu/nova-core/gsp/fw/r570_144.rs +1 -0
@@ -25,4 +25,5 @@ unsafe_op_in_unsafe_fn )]use kernel::ffi; +use kernel::prelude::Zeroable ;include! ("r570_144/bindings.rs" );
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +8 -8
@@ -20,7 +20,7 @@pub type u32_ =__u32 ;pub type u64_ =__u64 ;#[repr( C )] -#[derive( Copy , Clone )] +#[derive( Copy , Clone , Zeroable )] pub struct GspFwWprMeta {pub magic:u64_ ,pub revision:u64_ ,
@@ -55,19 +55,19 @@ pub struct GspFwWprMeta { pub verified: u64_, }#[repr( C )] -#[derive( Copy , Clone )] +#[derive( Copy , Clone , Zeroable )] pub union GspFwWprMeta__bindgen_ty_1 {pub __bindgen_anon_1:GspFwWprMeta__bindgen_ty_1__bindgen_ty_1 ,pub __bindgen_anon_2:GspFwWprMeta__bindgen_ty_1__bindgen_ty_2 , }#[repr( C )] -#[derive( Debug , Default , Copy , Clone )] +#[derive( Debug , Default , Copy , Clone , Zeroable )] pub struct GspFwWprMeta__bindgen_ty_1__bindgen_ty_1 {pub sysmemAddrOfSignature:u64_ ,pub sizeOfSignature:u64_ , }#[repr( C )] -#[derive( Debug , Default , Copy , Clone )] +#[derive( Debug , Default , Copy , Clone , Zeroable )] pub struct GspFwWprMeta__bindgen_ty_1__bindgen_ty_2 {pub gspFwHeapFreeListWprOffset:u32_ ,pub unused0:u32_ ,
@@ -83,13 +83,13 @@ fn default() -> Self { } }#[repr( C )] -#[derive( Copy , Clone )] +#[derive( Copy , Clone , Zeroable )] pub union GspFwWprMeta__bindgen_ty_2 {pub __bindgen_anon_1:GspFwWprMeta__bindgen_ty_2__bindgen_ty_1 ,pub __bindgen_anon_2:GspFwWprMeta__bindgen_ty_2__bindgen_ty_2 , }#[repr( C )] -#[derive( Debug , Default , Copy , Clone )] +#[derive( Debug , Default , Copy , Clone , Zeroable )] pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_1 {pub partitionRpcAddr:u64_ ,pub partitionRpcRequestOffset:u16_ ,
@@ -101,7 +101,7 @@ pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_1 { pub lsUcodeVersion: u32_, }#[repr( C )] -#[derive( Debug , Default , Copy , Clone )] +#[derive( Debug , Default , Copy , Clone , Zeroable )] pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_2 {pub partitionRpcPadding: [u32_ ;4usize ],pub sysmemAddrOfCrashReportQueue:u64_ ,
@@ -136,7 +136,7 @@ fn default() -> Self {pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_FB :LibosMemoryRegionLoc =2 ;pub type LibosMemoryRegionLoc = ffi::c_uint ;#[repr( C )] -#[derive( Debug , Default , Copy , Clone )] +#[derive( Debug , Default , Copy , Clone , Zeroable )] pub struct LibosMemoryRegionInitArgument {pub id8:LibosAddress ,pub pa:LibosAddress ,
Signature · 1 line
2.50.1
PATCH v5 6/14 gpu: nova-core: Add GSP command queue bindings
In reply to Alistair PoppleAdd bindings and accessors used for the GSP command queue.
---
Changes for v5:
- Derive the Zeroable trait for structs and enums
Changes for v4:
- Don't panic the kernel if trying to initialise a large (> 4GB) message header.
- Use init! to provide safe and complete initialisers.
- Take an enum type instead of a u32 for the function.
Changes for v3:
- New for v3
--- drivers/gpu/nova-core/gsp/fw.rs | 275 +++++++++ .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 541 ++++++++++++++++++ 2 files changed, 816 insertions(+)
drivers/gpu/nova-core/gsp/fw.rs +275 -0
@@ -5,6 +5,7 @@// Alias to avoid repeating the version number with every use. use r570_144as bindings; +use core::fmt;use core::ops::Range ;use kernel::dma::CoherentAllocation ;
@@ -16,6 +17,7 @@use crate ::firmware::gsp::GspFirmware ;use crate ::gpu::Chipset ;use crate ::gsp::FbLayout ; +use crate ::gsp::GSP_PAGE_SIZE ;/// Dummy type to group methods related to heap parameters for running the GSP firmware. pub (crate )struct GspFwHeapParams (());
@@ -158,6 +160,120 @@ pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self { } } +#[derive( PartialEq )] +pub (crate )enum MsgFunction { +// Common function codes +Nop = bindings::NV_VGPU_MSG_FUNCTION_NOP as isize , +SetGuestSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO as isize , +AllocRoot = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT as isize , +AllocDevice = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE as isize , +AllocMemory = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY as isize , +AllocCtxDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA as isize , +AllocChannelDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA as isize , +MapMemory = bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY as isize , +BindCtxDma = bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA as isize , +AllocObject = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT as isize , +Free = bindings::NV_VGPU_MSG_FUNCTION_FREE as isize , +Log = bindings::NV_VGPU_MSG_FUNCTION_LOG as isize , +GetGspStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO as isize , +SetRegistry = bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY as isize , +GspSetSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO as isize , +GspInitPostObjGpu = bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU as isize , +GspRmControl = bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL as isize , +GetStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO as isize , + +// Event codes +GspInitDone = bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE as isize , +GspRunCpuSequencer = bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER as isize , +PostEvent = bindings::NV_VGPU_MSG_EVENT_POST_EVENT as isize , +RcTriggered = bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED as isize , +MmuFaultQueued = bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED as isize , +OsErrorLog = bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG as isize , +GspPostNoCat = bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD as isize , +GspLockdownNotice = bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE as isize , +UcodeLibOsPrint = bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT as isize , +} + +impl fmt::Display for MsgFunction { +fn fmt (&self, f: &mut fmt::Formatter <'_>) -> fmt::Result { +match self { +// Common function codes +MsgFunction ::Nop =>write! (f,"NOP" ), +MsgFunction ::SetGuestSystemInfo =>write! (f,"SET_GUEST_SYSTEM_INFO" ), +MsgFunction ::AllocRoot =>write! (f,"ALLOC_ROOT" ), +MsgFunction ::AllocDevice =>write! (f,"ALLOC_DEVICE" ), +MsgFunction ::AllocMemory =>write! (f,"ALLOC_MEMORY" ), +MsgFunction ::AllocCtxDma =>write! (f,"ALLOC_CTX_DMA" ), +MsgFunction ::AllocChannelDma =>write! (f,"ALLOC_CHANNEL_DMA" ), +MsgFunction ::MapMemory =>write! (f,"MAP_MEMORY" ), +MsgFunction ::BindCtxDma =>write! (f,"BIND_CTX_DMA" ), +MsgFunction ::AllocObject =>write! (f,"ALLOC_OBJECT" ), +MsgFunction ::Free =>write! (f,"FREE" ), +MsgFunction ::Log =>write! (f,"LOG" ), +MsgFunction ::GetGspStaticInfo =>write! (f,"GET_GSP_STATIC_INFO" ), +MsgFunction ::SetRegistry =>write! (f,"SET_REGISTRY" ), +MsgFunction ::GspSetSystemInfo =>write! (f,"GSP_SET_SYSTEM_INFO" ), +MsgFunction ::GspInitPostObjGpu =>write! (f,"GSP_INIT_POST_OBJGPU" ), +MsgFunction ::GspRmControl =>write! (f,"GSP_RM_CONTROL" ), +MsgFunction ::GetStaticInfo =>write! (f,"GET_STATIC_INFO" ), + +// Event codes +MsgFunction ::GspInitDone =>write! (f,"INIT_DONE" ), +MsgFunction ::GspRunCpuSequencer =>write! (f,"RUN_CPU_SEQUENCER" ), +MsgFunction ::PostEvent =>write! (f,"POST_EVENT" ), +MsgFunction ::RcTriggered =>write! (f,"RC_TRIGGERED" ), +MsgFunction ::MmuFaultQueued =>write! (f,"MMU_FAULT_QUEUED" ), +MsgFunction ::OsErrorLog =>write! (f,"OS_ERROR_LOG" ), +MsgFunction ::GspPostNoCat =>write! (f,"NOCAT" ), +MsgFunction ::GspLockdownNotice =>write! (f,"LOCKDOWN_NOTICE" ), +MsgFunction ::UcodeLibOsPrint =>write! (f,"LIBOS_PRINT" ), + } + } +} + +impl TryFrom <u32 >for MsgFunction { +type Error = kernel::error::Error ; + +fn try_from (value:u32 ) ->Result <MsgFunction > { +match value { + bindings::NV_VGPU_MSG_FUNCTION_NOP =>Ok (MsgFunction ::Nop ), + bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO => { +Ok (MsgFunction ::SetGuestSystemInfo ) + } + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT =>Ok (MsgFunction ::AllocRoot ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE =>Ok (MsgFunction ::AllocDevice ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY =>Ok (MsgFunction ::AllocMemory ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA =>Ok (MsgFunction ::AllocCtxDma ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA =>Ok (MsgFunction ::AllocChannelDma ), + bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY =>Ok (MsgFunction ::MapMemory ), + bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA =>Ok (MsgFunction ::BindCtxDma ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT =>Ok (MsgFunction ::AllocObject ), + bindings::NV_VGPU_MSG_FUNCTION_FREE =>Ok (MsgFunction ::Free ), + bindings::NV_VGPU_MSG_FUNCTION_LOG =>Ok (MsgFunction ::Log ), + bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO =>Ok (MsgFunction ::GetGspStaticInfo ), + bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY =>Ok (MsgFunction ::SetRegistry ), + bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO =>Ok (MsgFunction ::GspSetSystemInfo ), + bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU => { +Ok (MsgFunction ::GspInitPostObjGpu ) + } + bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL =>Ok (MsgFunction ::GspRmControl ), + bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO =>Ok (MsgFunction ::GetStaticInfo ), + bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE =>Ok (MsgFunction ::GspInitDone ), + bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER => { +Ok (MsgFunction ::GspRunCpuSequencer ) + } + bindings::NV_VGPU_MSG_EVENT_POST_EVENT =>Ok (MsgFunction ::PostEvent ), + bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED =>Ok (MsgFunction ::RcTriggered ), + bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED =>Ok (MsgFunction ::MmuFaultQueued ), + bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG =>Ok (MsgFunction ::OsErrorLog ), + bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD =>Ok (MsgFunction ::GspPostNoCat ), + bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE =>Ok (MsgFunction ::GspLockdownNotice ), + bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT =>Ok (MsgFunction ::UcodeLibOsPrint ), + _ =>Err (EINVAL ), + } + } +} +/// Struct containing the arguments required to pass a memory buffer to the GSP /// for use during initialisation. ///
@@ -208,3 +324,162 @@ fn id8(name: &str) -> u64 { })) } } + +#[repr(transparent)] +pub (crate )struct MsgqTxHeader (bindings::msgqTxHeader ); + +impl MsgqTxHeader { +pub (crate )fn new (msgq_size:u32 , rx_hdr_offset:u32 , msg_count:u32 ) ->Self { +Self (bindings::msgqTxHeader { + version:0 , + size: msgq_size, + msgSize:GSP_PAGE_SIZE as u32 , + msgCount: msg_count, + writePtr:0 , + flags:1 , + rxHdrOff: rx_hdr_offset, + entryOff:GSP_PAGE_SIZE as u32 , + }) + } + +pub (crate )fn write_ptr (&self) ->u32 { +let ptr = (&self.0 .writePtr)as *const u32 ; + +// SAFETY: This is part of a CoherentAllocation and implements the +// equivalent as what the dma_read! macro would and is therefore safe +// for the same reasons. +unsafe { ptr.read_volatile () } + } + +pub (crate )fn set_write_ptr (&mut self, val:u32 ) { +let ptr = (&mut self.0 .writePtr)as *mut u32 ; + +// SAFETY: This is part of a CoherentAllocation and implements the +// equivalent as what the dma_write! macro would and is therefore safe +// for the same reasons. +unsafe { ptr.write_volatile (val) } + } +} + +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for MsgqTxHeader {} + +/// RX header for setting up a message queue with the GSP. +#[repr(transparent)] +pub (crate )struct MsgqRxHeader (bindings::msgqRxHeader ); + +impl MsgqRxHeader { +pub (crate )fn new () ->Self { +Self (Default ::default ()) + } + +pub (crate )fn read_ptr (&self) ->u32 { +let ptr = (&self.0 .readPtr)as *const u32 ; + +// SAFETY: This is part of a CoherentAllocation and implements the +// equivalent as what the dma_read! macro would and is therefore safe +// for the same reasons. +unsafe { ptr.read_volatile () } + } + +pub (crate )fn set_read_ptr (&mut self, val:u32 ) { +let ptr = (&mut self.0 .readPtr)as *mut u32 ; + +// SAFETY: This is part of a CoherentAllocation and implements the +// equivalent as what the dma_write! macro would and is therefore safe +// for the same reasons. +unsafe { ptr.write_volatile (val) } + } +} + +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for MsgqRxHeader {} + +impl bindings::rpc_message_header_v { +pub (crate )fn init (cmd_size:u32 , function:MsgFunction ) ->impl Init <Self ,Error > { +type RpcMessageHeader = bindings::rpc_message_header_v ; +try_init! (RpcMessageHeader { +// TODO: magic number + header_version:0x03000000 , + signature: bindings::NV_VGPU_MSG_SIGNATURE_VALID , + function: functionas u32 , + length: (size_of ::<Self >()as u32 ) + .checked_add (cmd_size) + .ok_or (EOVERFLOW )?, + rpc_result:0xffffffff , + rpc_result_private:0xffffffff , + ..Zeroable ::init_zeroed () + }) + } +} + +// SAFETY: We can't derive the Zeroable trait for this binding because the +// procedural macro doesn't support the syntax used by bindgen to create the +// __IncompleteArrayField types. So instead we implement it here, which is safe +// because these are explicitly padded structures only containing types for +// which any bit pattern, including all zeros, is valid. +unsafe impl Zeroable for bindings::rpc_message_header_v {} + +#[repr(transparent)] +pub (crate )struct GspMsgElement { + inner: bindings::GSP_MSG_QUEUE_ELEMENT , +} + +impl GspMsgElement { +#[allow(non_snake_case)] +pub (crate )fn init ( + sequence:u32 , + cmd_size:usize , + function:MsgFunction , + ) ->impl Init <Self ,Error > { +type RpcMessageHeader = bindings::rpc_message_header_v ; +type InnerGspMsgElement = bindings::GSP_MSG_QUEUE_ELEMENT ; +let init_inner =try_init! (InnerGspMsgElement { + seqNum: sequence, + elemCount:size_of ::<Self >() + .checked_add (cmd_size) + .ok_or (EOVERFLOW )? + .div_ceil (GSP_PAGE_SIZE )as u32 , + rpc <-RpcMessageHeader ::init (cmd_sizeas u32 , function), + ..Zeroable ::init_zeroed () + }); + +try_init! (GspMsgElement { + inner <- init_inner, + }) + } + +pub (crate )fn set_checksum (&mut self, checksum:u32 ) { + self.inner.checkSum = checksum; + } + +// Return the total length of the message, noting that rpc.length includes +// the length of the GspRpcHeader but not the message header. +pub (crate )fn length (&self) ->u32 { +size_of ::<Self >()as u32 -size_of ::<bindings::rpc_message_header_v >()as u32 + + self.inner.rpc.length + } + +pub (crate )fn sequence (&self) ->u32 { + self.inner.rpc.sequence + } + +pub (crate )fn function_number (&self) ->u32 { + self.inner.rpc.function + } + +pub (crate )fn function (&self) ->Result <MsgFunction > { + self.inner.rpc.function.try_into () + } + +pub (crate )fn element_count (&self) ->u32 { + self.inner.elemCount + } +} + +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for GspMsgElement {} + +// SAFETY: This struct only contains integer types for which all bit patterns +// are valid. +unsafe impl FromBytes for GspMsgElement {}
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +541 -0
561 lines not on this page · raw message
Signature · 1 line
2.50.1
Alexandre Courbot
maintainerRe: [PATCH v5 06/14] gpu: nova-core: Add GSP command queue bindings
Quoted 3 lines · Alistair Popple
Add bindings and accessors used for the GSP command queue.
Signed-off-by Alistair Popple <[email protected]>
There are lots of unused warnings when building on this commit. I know we don't like big patches, but since this is semantically related to the next one and the two touch different files anyway, how about merging them? IMHO this is preferable to adding lots of #[expect(unused)] that we are going to remove right after.
If you have been told to split in a previous version, let's just add a the #[expect(unused)] where needed.
Quoted 66 lines
---
Changes for v5:
- Derive the Zeroable trait for structs and enumsChanges for v4:
- Don't panic the kernel if trying to initialise a large (> 4GB) message header.
- Useinit!to provide safe and complete initialisers.
- Take an enum type instead of a u32 for the function.Changes for v3:
- New for v3--- drivers/gpu/nova-core/gsp/fw.rs | 275 +++++++++ .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 541 ++++++++++++++++++ 2 files changed, 816 insertions(+)drivers/gpu/nova-core/gsp/fw.rs +25
-0@@ -5,6 +5,7 @@// Alias to avoid repeating the version number with every use. use r570_144as bindings; +use core::fmt;use core::ops::Range ;use kernel::dma::CoherentAllocation ;@@ -16,6 +17,7 @@use crate ::firmware::gsp::GspFirmware ;use crate ::gpu::Chipset ;use crate ::gsp::FbLayout ; +use crate ::gsp::GSP_PAGE_SIZE ;/// Dummy type to group methods related to heap parameters for running the GSP firmware. pub (crate )struct GspFwHeapParams (());@@ -158,6 +160,120 @@ pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self { } } +#[derive( PartialEq )] +pub (crate )enum MsgFunction { +// Common function codes +Nop = bindings::NV_VGPU_MSG_FUNCTION_NOP as isize , +SetGuestSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO as isize , +AllocRoot = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT as isize , +AllocDevice = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE as isize , +AllocMemory = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY as isize , +AllocCtxDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA as isize , +AllocChannelDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA as isize , +MapMemory = bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY as isize , +BindCtxDma = bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA as isize , +AllocObject = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT as isize , +Free = bindings::NV_VGPU_MSG_FUNCTION_FREE as isize , +Log = bindings::NV_VGPU_MSG_FUNCTION_LOG as isize , +GetGspStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO as isize , +SetRegistry = bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY as isize , +GspSetSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO as isize , +GspInitPostObjGpu = bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU as isize , +GspRmControl = bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL as isize , +GetStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO as isize , + +// Event codes
drivers/gpu/nova-core/gsp/fw.rs pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self { · line 195
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ GspInitDone = bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE as isize, + GspRunCpuSequencer = bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER as isize, + PostEvent = bindings::NV_VGPU_MSG_EVENT_POST_EVENT as isize, + RcTriggered = bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED as isize, + MmuFaultQueued = bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED as isize, + OsErrorLog = bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG as isize, + GspPostNoCat = bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD as isize, + GspLockdownNotice = bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE as isize, + UcodeLibOsPrint = bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT as isize, +}
Can we split functions and events into distinct types? Their bindings even have different prefixes.
Also, rather than casting these all as isize, I'd suggest making the type #[repr(u32)]. We don't need negative values here.
Quoted 70 lines
drivers/gpu/nova-core/gsp/fw.rs +70
-0+ +impl fmt::Display for MsgFunction { +fn fmt (&self, f: &mut fmt::Formatter <'_>) -> fmt::Result { +match self { +// Common function codes +MsgFunction ::Nop =>write! (f,"NOP" ), +MsgFunction ::SetGuestSystemInfo =>write! (f,"SET_GUEST_SYSTEM_INFO" ), +MsgFunction ::AllocRoot =>write! (f,"ALLOC_ROOT" ), +MsgFunction ::AllocDevice =>write! (f,"ALLOC_DEVICE" ), +MsgFunction ::AllocMemory =>write! (f,"ALLOC_MEMORY" ), +MsgFunction ::AllocCtxDma =>write! (f,"ALLOC_CTX_DMA" ), +MsgFunction ::AllocChannelDma =>write! (f,"ALLOC_CHANNEL_DMA" ), +MsgFunction ::MapMemory =>write! (f,"MAP_MEMORY" ), +MsgFunction ::BindCtxDma =>write! (f,"BIND_CTX_DMA" ), +MsgFunction ::AllocObject =>write! (f,"ALLOC_OBJECT" ), +MsgFunction ::Free =>write! (f,"FREE" ), +MsgFunction ::Log =>write! (f,"LOG" ), +MsgFunction ::GetGspStaticInfo =>write! (f,"GET_GSP_STATIC_INFO" ), +MsgFunction ::SetRegistry =>write! (f,"SET_REGISTRY" ), +MsgFunction ::GspSetSystemInfo =>write! (f,"GSP_SET_SYSTEM_INFO" ), +MsgFunction ::GspInitPostObjGpu =>write! (f,"GSP_INIT_POST_OBJGPU" ), +MsgFunction ::GspRmControl =>write! (f,"GSP_RM_CONTROL" ), +MsgFunction ::GetStaticInfo =>write! (f,"GET_STATIC_INFO" ), + +// Event codes +MsgFunction ::GspInitDone =>write! (f,"INIT_DONE" ), +MsgFunction ::GspRunCpuSequencer =>write! (f,"RUN_CPU_SEQUENCER" ), +MsgFunction ::PostEvent =>write! (f,"POST_EVENT" ), +MsgFunction ::RcTriggered =>write! (f,"RC_TRIGGERED" ), +MsgFunction ::MmuFaultQueued =>write! (f,"MMU_FAULT_QUEUED" ), +MsgFunction ::OsErrorLog =>write! (f,"OS_ERROR_LOG" ), +MsgFunction ::GspPostNoCat =>write! (f,"NOCAT" ), +MsgFunction ::GspLockdownNotice =>write! (f,"LOCKDOWN_NOTICE" ), +MsgFunction ::UcodeLibOsPrint =>write! (f,"LIBOS_PRINT" ), + } + } +} + +impl TryFrom <u32 >for MsgFunction { +type Error = kernel::error::Error ; + +fn try_from (value:u32 ) ->Result <MsgFunction > { +match value { + bindings::NV_VGPU_MSG_FUNCTION_NOP =>Ok (MsgFunction ::Nop ), + bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO => { +Ok (MsgFunction ::SetGuestSystemInfo ) + } + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT =>Ok (MsgFunction ::AllocRoot ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE =>Ok (MsgFunction ::AllocDevice ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY =>Ok (MsgFunction ::AllocMemory ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA =>Ok (MsgFunction ::AllocCtxDma ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA =>Ok (MsgFunction ::AllocChannelDma ), + bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY =>Ok (MsgFunction ::MapMemory ), + bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA =>Ok (MsgFunction ::BindCtxDma ), + bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT =>Ok (MsgFunction ::AllocObject ), + bindings::NV_VGPU_MSG_FUNCTION_FREE =>Ok (MsgFunction ::Free ), + bindings::NV_VGPU_MSG_FUNCTION_LOG =>Ok (MsgFunction ::Log ), + bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO =>Ok (MsgFunction ::GetGspStaticInfo ), + bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY =>Ok (MsgFunction ::SetRegistry ), + bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO =>Ok (MsgFunction ::GspSetSystemInfo ), + bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU => { +Ok (MsgFunction ::GspInitPostObjGpu ) + } + bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL =>Ok (MsgFunction ::GspRmControl ), + bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO =>Ok (MsgFunction ::GetStaticInfo ), + bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE =>Ok (MsgFunction ::GspInitDone ), + bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER => { +Ok (MsgFunction ::GspRunCpuSequencer ) + } + bindings::NV_VGPU_MSG_EVENT_POST_EVENT =>Ok (MsgFunction ::PostEvent ),
drivers/gpu/nova-core/gsp/fw.rs pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self { · line 275
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED => Ok(MsgFunction::RcTriggered), + bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED => Ok(MsgFunction::MmuFaultQueued), + bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG => Ok(MsgFunction::OsErrorLog), + bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD => Ok(MsgFunction::GspPostNoCat), + bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE => Ok(MsgFunction::GspLockdownNotice), + bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT => Ok(MsgFunction::UcodeLibOsPrint), + _ => Err(EINVAL), + } + } +}
This is another place where the TryFrom macro will shine. :)
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 329
Quoted 11 lines
drivers/gpu/nova-core/gsp/fw.rs +4
-0+/// Struct containing the arguments required to pass a memory buffer to the GSP /// for use during initialisation. /// @@ -208,3 +324,162 @@ fn id8(name: &str) -> u64 { })) } } + +#[repr(transparent)] +pub (crate )struct MsgqTxHeader (bindings::msgqTxHeader );
Short documentation please (MsgqRxHeader provides a template).
Quoted 23 lines
drivers/gpu/nova-core/gsp/fw.rs +23
-0+ +impl MsgqTxHeader { +pub (crate )fn new (msgq_size:u32 , rx_hdr_offset:u32 , msg_count:u32 ) ->Self { +Self (bindings::msgqTxHeader { + version:0 , + size: msgq_size, + msgSize:GSP_PAGE_SIZE as u32 , + msgCount: msg_count, + writePtr:0 , + flags:1 , + rxHdrOff: rx_hdr_offset, + entryOff:GSP_PAGE_SIZE as u32 , + }) + } + +pub (crate )fn write_ptr (&self) ->u32 { +let ptr = (&self.0 .writePtr)as *const u32 ; + +// SAFETY: This is part of a CoherentAllocation and implements the +// equivalent as what the dma_read! macro would and is therefore safe +// for the same reasons. +unsafe { ptr.read_volatile () } + }
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 362
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ +pub (crate )fn set_write_ptr (&mut self, val:u32 ) { +let ptr = (&mut self.0 .writePtr)as *mut u32 ; + +// SAFETY: This is part of a CoherentAllocation and implements the +// equivalent as what the dma_write! macro would and is therefore safe +// for the same reasons. +unsafe { ptr.write_volatile (val) } + } +}
These methods should also be succintly documented imho.
Quoted 21 lines
drivers/gpu/nova-core/gsp/fw.rs +21
-0+ +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for MsgqTxHeader {} + +/// RX header for setting up a message queue with the GSP. +#[repr(transparent)] +pub (crate )struct MsgqRxHeader (bindings::msgqRxHeader ); + +impl MsgqRxHeader { +pub (crate )fn new () ->Self { +Self (Default ::default ()) + } + +pub (crate )fn read_ptr (&self) ->u32 { +let ptr = (&self.0 .readPtr)as *const u32 ; + +// SAFETY: This is part of a CoherentAllocation and implements the +// equivalent as what the dma_read! macro would and is therefore safe +// for the same reasons. +unsafe { ptr.read_volatile () } + }
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 393
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ +pub (crate )fn set_read_ptr (&mut self, val:u32 ) { +let ptr = (&mut self.0 .readPtr)as *mut u32 ; + +// SAFETY: This is part of a CoherentAllocation and implements the +// equivalent as what the dma_write! macro would and is therefore safe +// for the same reasons. +unsafe { ptr.write_volatile (val) } + } +}
Documentation here as well.
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 399
Quoted 6 lines
drivers/gpu/nova-core/gsp/fw.rs +6
-0+ +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for MsgqRxHeader {} + +impl bindings::rpc_message_header_v { +pub (crate )fn init (cmd_size:u32 , function:MsgFunction ) ->impl Init <Self ,Error > {
This method can be kept private.
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 403
Quoted 4 lines
drivers/gpu/nova-core/gsp/fw.rs +4
-0+type RpcMessageHeader = bindings::rpc_message_header_v ; +try_init! (RpcMessageHeader { +// TODO: magic number + header_version:0x03000000 ,
Let's resolve this TODO by declaring a constant or even better, adding the relevant bindings symbol if there is one.
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 405
Quoted 2 lines
drivers/gpu/nova-core/gsp/fw.rs +2
-0+ signature: bindings::NV_VGPU_MSG_SIGNATURE_VALID, + function: function as u32,
This as is ok once we set MsgFunction to have a u32 representation, but...
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 406
Quoted 1 line
drivers/gpu/nova-core/gsp/fw.rs +1
-0+ length: (size_of::<Self>() as u32)
... at the risk of sounding too pedantic, we probably want to do the arithmetic on the usize, and then do a try_into u32 to remove this use of as here:
length: size_of::<Self>()
.checked_add(cmd_size)
.ok_or(EOVERFLOW)
.and_then(|v| v.try_into().map_err(|_| EINVAL))?,This also requires changing the cmd_size to usize, which is actually better as that's also the type used by the caller anyway, so this removes another conversion.
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ .checked_add(cmd_size) + .ok_or(EOVERFLOW)?, + rpc_result: 0xffffffff, + rpc_result_private: 0xffffffff, + ..Zeroable ::init_zeroed () + }) + } +} + +// SAFETY: We can't derive the Zeroable trait for this binding because the
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 426
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+// procedural macro doesn't support the syntax used by bindgen to create the +// __IncompleteArrayField types. So instead we implement it here, which is safe +// because these are explicitly padded structures only containing types for +// which any bit pattern, including all zeros, is valid. +unsafe impl Zeroable for bindings::rpc_message_header_v {} + +#[repr(transparent)] +pub (crate )struct GspMsgElement { + inner: bindings::GSP_MSG_QUEUE_ELEMENT , +}
Small documentation needed.
Quoted 6 lines
drivers/gpu/nova-core/gsp/fw.rs +6
-0+ +impl GspMsgElement { +#[allow(non_snake_case)] +pub (crate )fn init ( + sequence:u32 , + cmd_size: usize,
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 442
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ function: MsgFunction, + ) -> impl Init<Self, Error> { +type RpcMessageHeader = bindings::rpc_message_header_v ; +type InnerGspMsgElement = bindings::GSP_MSG_QUEUE_ELEMENT ; + let init_inner = try_init!(InnerGspMsgElement { + seqNum: sequence, + elemCount: size_of::<Self>() + .checked_add(cmd_size) + .ok_or(EOVERFLOW)? + .div_ceil(GSP_PAGE_SIZE) as u32,
Let's chain a call to try_into here as well.
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 443
Quoted 1 line
drivers/gpu/nova-core/gsp/fw.rs +1
-0+ rpc <- RpcMessageHeader::init(cmd_size as u32, function),
... and thanks to the change suggested 3 blocks above, this as is no longer needed.
Quoted 8 lines
drivers/gpu/nova-core/gsp/fw.rs +8
-0+ ..Zeroable ::init_zeroed () + }); + +try_init! (GspMsgElement { + inner <- init_inner, + }) + } +
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 461
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+pub (crate )fn set_checksum (&mut self, checksum:u32 ) { + self.inner.checkSum = checksum; + } + +// Return the total length of the message, noting that rpc.length includes +// the length of the GspRpcHeader but not the message header. +pub (crate )fn length (&self) ->u32 { +size_of ::<Self >()as u32 -size_of ::<bindings::rpc_message_header_v >()as u32 + + self.inner.rpc.length + }
We can save ourselves a bunch of as conversions in the following patch if we make this method just return usize. The only one needed will be for self.inner.rpc.length to usize, we really ought to have non-fallible converters for things like this that obviously cannot fail.
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7
-0+ +pub (crate )fn sequence (&self) ->u32 { + self.inner.rpc.sequence + } + +pub (crate )fn function_number (&self) ->u32 { + self.inner.rpc.function
drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 478
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ } + +pub (crate )fn function (&self) ->Result <MsgFunction > { + self.inner.rpc.function.try_into () + } + +pub (crate )fn element_count (&self) ->u32 { + self.inner.elemCount + } +}
Documentation needed for these too.
Re: [PATCH v5 06/14] gpu: nova-core: Add GSP command queue bindings
Quoted 9 lines · Alexandre Courbot
There are lots of unused warnings when building on this commit. I know we don't like big patches, but since this is semantically related to the next one and the two touch different files anyway, how about merging them? IMHO this is preferable to adding lots of
#[expect(unused)]that we are going to remove right after.If you have been told to split in a previous version, let's just add a the
#[expect(unused)]where needed.
A possible middle-ground is to add the expect in a "more global" position, e.g. at the top, so that it is a single line change. I think that is fine as long as it is removed right after.
Thanks!
Cheers,
Miguel
Re: [PATCH v5 06/14] gpu: nova-core: Add GSP command queue bindings
On 2025-10-17 at 06:22 +1100, Miguel Ojeda <[email protected]> wrote...
Quoted 10 lines · Alexandre Courbot
Quoted 9 lines
There are lots of unused warnings when building on this commit. I know we don't like big patches, but since this is semantically related to the next one and the two touch different files anyway, how about merging them? IMHO this is preferable to adding lots of
#[expect(unused)]that we are going to remove right after.If you have been told to split in a previous version, let's just add a the
#[expect(unused)]where needed.
No, I just split it here because it seemed like a nice self-contained thing. That and I'm conditioned to linux-mm patches where 20 changes is often considered a big patch :-)
So I will probably just squash this as suggested.
Quoted 4 lines
A possible middle-ground is to add the
expectin a "more global" position, e.g. at the top, so that it is a single line change. I think that is fine as long as it is removed right after.
That could work, thanks.
Quoted 4 lines
Thanks!
Cheers,
Miguel
PATCH v5 7/14 gpu: nova-core: gsp: Add GSP command queue handling
In reply to Alistair PoppleThis commit introduces core infrastructure for handling GSP command and message queues in the nova-core driver. The command queue system enables bidirectional communication between the host driver and GSP firmware through a remote message passing interface.
The interface is based on passing serialised data structures over a ring buffer with separate transmit and receive queues. Commands are sent by writing to the CPU transmit queue and waiting for completion via the receive queue.
To ensure safety mutable or immutable (depending on whether it is a send or receive operation) references are taken on the command queue when allocating the message to write/read to. This ensures message memory remains valid and the command queue can't be mutated whilst an operation is in progress.
Currently this is only used by the probe() routine and therefore can only used by a single thread of execution. Locking to enable safe access from multiple threads will be introduced in a future series when that becomes necessary.
---
Changes for v4:
- Use read_poll_timeout() instead of wait_on()
- Switch to using init! (Thanks Alex for figuring out the required workarounds)
- Pass the enum type into the RPC bindings instead of a raw u32
- Fixup the TODOs for extracting/allocating the send command regions
- Split the sending functions into one taking just a command struct and another taking a command struct with extra payload
Changes for v3:
- Reduce the receive payloads to the correct size
- Use opaque bindings
- Clean up of the command queue PTE creation
- Add an enum for the GSP functions
- Rename GspCommandToGsp and GspMessageFromGsp
- Rename GspCmdq
- Add function to notify GSP of updated queue pointers
- Inline driver area access functions
- Fixup receive area calculations
Changes for v2:
- Rebased on Alex's latest series
--- drivers/gpu/nova-core/gsp.rs | 9 + drivers/gpu/nova-core/gsp/cmdq.rs | 493 ++++++++++++++++++++++++++++++ drivers/gpu/nova-core/regs.rs | 4 + drivers/gpu/nova-core/sbuffer.rs | 2 - scripts/Makefile.build | 2 +- 5 files changed, 507 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs
drivers/gpu/nova-core/gsp.rs +9 -0
@@ -2,6 +2,7 @@mod boot; +use kernel::alloc::flags::GFP_KERNEL ;use kernel::device;use kernel::dma::CoherentAllocation ;use kernel::dma::DmaAddress ;
@@ -11,6 +12,7 @@use kernel::transmute::AsBytes ;use crate ::fb::FbLayout ; +use crate ::gsp::cmdq::Cmdq ;pub (crate )use fw::{GspFwWprMeta ,LibosParams };
@@ -18,6 +20,8 @@use fw::LibosMemoryRegionInitArgument ; +pub (crate )mod cmdq; +pub (crate )const GSP_PAGE_SHIFT :usize =12 ;pub (crate )const GSP_PAGE_SIZE :usize =1 <<GSP_PAGE_SHIFT ;
@@ -31,6 +35,7 @@ pub(crate) struct Gsp { loginit: LogBuffer, logintr: LogBuffer, logrm: LogBuffer, + pub(crate) cmdq: Cmdq, }#[repr( C )]
@@ -110,11 +115,15 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Selflet logrm =LogBuffer ::new (dev)?;dma_write! (libos[2 ] =LibosMemoryRegionInitArgument ::new ("LOGRM" , &logrm.0 )?)?; +// Creates its own PTE array. +let cmdq =Cmdq ::new (dev)?; +Ok (try_pin_init! (Self { libos, loginit, logintr, logrm, + cmdq, })) } }
drivers/gpu/nova-core/gsp/cmdq.rs new file +493 -0
493 lines not on this page · raw message
drivers/gpu/nova-core/regs.rs +4 -0
@@ -71,6 +71,10 @@ pub(crate) fn chipset(self) -> Result<Chipset> { 30:30 ecc_mode_enabled as bool; }); +register! (NV_PGSP_QUEUE_HEAD @0x00110c00 { +31 :0 addressas u32 ; +}); +impl NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE {/// Returns the usable framebuffer size, in bytes. pub (crate )fn usable_fb_size (self) ->u64 {
drivers/gpu/nova-core/sbuffer.rs +0 -2
@@ -50,7 +50,6 @@ impl<'a, I> SBufferIter<I>/// let sum: u8 = sbuffer.sum(); /// assert_eq!(sum, 45); /// ``` -#[expect(unused)] pub (crate )fn new_reader (slices:impl IntoIterator <IntoIter =I >) ->Self where I :Iterator <Item = &'a [u8 ]>,
@@ -72,7 +71,6 @@ pub(crate) fn new_reader(slices: impl IntoIterator<IntoIter = I>) -> Self/// assert_eq!(buf2, [5, 6, 7, 8, 9]); /// /// ``` -#[expect(unused)] pub (crate )fn new_writer (slices:impl IntoIterator <IntoIter =I >) ->Self where I :Iterator <Item = &'amut [u8 ]>,
scripts/Makefile.build +1 -1
@@ -317,7 +317,7 @@ $(obj)/%.lst: $(obj)/%.c FORCE # # Please see https://github.com/Rust-for-Linux/linux/issues/2 for details on # the unstable features in use. -rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg +rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg,slice_flatten # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree
Signature · 1 line
2.50.1
Alexandre Courbot
maintainerRe: [PATCH v5 07/14] gpu: nova-core: gsp: Add GSP command queue handling
Quoted 61 lines · Alistair Popple
This commit introduces core infrastructure for handling GSP command and message queues in the nova-core driver. The command queue system enables bidirectional communication between the host driver and GSP firmware through a remote message passing interface.
The interface is based on passing serialised data structures over a ring buffer with separate transmit and receive queues. Commands are sent by writing to the CPU transmit queue and waiting for completion via the receive queue.
To ensure safety mutable or immutable (depending on whether it is a send or receive operation) references are taken on the command queue when allocating the message to write/read to. This ensures message memory remains valid and the command queue can't be mutated whilst an operation is in progress.
Currently this is only used by the
probe()routine and therefore can only used by a single thread of execution. Locking to enable safe access from multiple threads will be introduced in a future series when that becomes necessary.Signed-off-by Alistair Popple <[email protected]>---
Changes for v4:
- Useread_poll_timeout()instead ofwait_on()
- Switch to usinginit!(Thanks Alex for figuring out the required workarounds)
- Pass the enum type into the RPC bindings instead of a raw u32
- Fixup the TODOs for extracting/allocating the send command regions
- Split the sending functions into one taking just a command struct and another taking a command struct with extra payloadChanges for v3:
- Reduce the receive payloads to the correct size
- Use opaque bindings
- Clean up of the command queue PTE creation
- Add an enum for the GSP functions
- Rename GspCommandToGsp and GspMessageFromGsp
- Rename GspCmdq
- Add function to notify GSP of updated queue pointers
- Inline driver area access functions
- Fixup receive area calculationsChanges for v2:
- Rebased on Alex's latest series--- drivers/gpu/nova-core/gsp.rs | 9 + drivers/gpu/nova-core/gsp/cmdq.rs | 493 ++++++++++++++++++++++++++++++ drivers/gpu/nova-core/regs.rs | 4 + drivers/gpu/nova-core/sbuffer.rs | 2 - scripts/Makefile.build | 2 +- 5 files changed, 507 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rsdrivers/gpu/nova-core/gsp.rs +0
-0
drivers/gpu/nova-core/gsp.rs line 5
Quoted 4 lines
drivers/gpu/nova-core/gsp.rs +1
-0@@ -2,6 +2,7 @@mod boot; +use kernel::alloc::flags::GFP_KERNEL ;
This import should not be needed, GFP_KERNEL being part of the prelude.
Quoted 29 lines
drivers/gpu/nova-core/gsp.rs +4
-0use kernel::device;use kernel::dma::CoherentAllocation ;use kernel::dma::DmaAddress ;@@ -11,6 +12,7 @@use kernel::transmute::AsBytes ;use crate ::fb::FbLayout ; +use crate ::gsp::cmdq::Cmdq ;pub (crate )use fw::{GspFwWprMeta ,LibosParams };@@ -18,6 +20,8 @@use fw::LibosMemoryRegionInitArgument ; +pub (crate )mod cmdq; +pub (crate )const GSP_PAGE_SHIFT :usize =12 ;pub (crate )const GSP_PAGE_SIZE :usize =1 <<GSP_PAGE_SHIFT ;@@ -31,6 +35,7 @@ pub(crate) struct Gsp { loginit: LogBuffer, logintr: LogBuffer, logrm: LogBuffer, + pub(crate) cmdq: Cmdq, }#[repr( C )]
drivers/gpu/nova-core/gsp.rs pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self · line 119
Quoted 5 lines
drivers/gpu/nova-core/gsp.rs +2
-0@@ -110,11 +115,15 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Selflet logrm =LogBuffer ::new (dev)?;dma_write! (libos[2 ] =LibosMemoryRegionInitArgument ::new ("LOGRM" , &logrm.0 )?)?; +// Creates its own PTE array. +let cmdq =Cmdq ::new (dev)?;
I believe this comment is a leftover.
Quoted 16 lines
drivers/gpu/nova-core/gsp.rs +2
-0+Ok (try_pin_init! (Self { libos, loginit, logintr, logrm, + cmdq, })) } }drivers/gpu/nova-core/gsp/cmdq.rs new file +0
-0
drivers/gpu/nova-core/gsp/cmdq.rs line 7
Quoted 7 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +7
-0@@ -0,0 +1,493 @@ +// SPDX-License-Identifier: GPL-2.0 + +use core::mem::offset_of; +use core::sync::atomic::fence; +use core::sync::atomic::Ordering ; + +use kernel::alloc::flags::GFP_KERNEL ;
This is one not needed either.
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +4
-0+use kernel::device; +use kernel::dma::CoherentAllocation ; +use kernel::dma_write; +use kernel::io::poll::read_poll_timeout;
drivers/gpu/nova-core/gsp/cmdq.rs line 21
Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10
-0+use kernel::prelude::*; +use kernel::sync::aref::ARef ; +use kernel::time::Delta ; +use kernel::transmute::{AsBytes ,FromBytes }; + +use crate ::driver::Bar0 ; +use crate ::gsp::fw::{GspMsgElement ,MsgFunction ,MsgqRxHeader ,MsgqTxHeader }; +use crate ::gsp::PteArray ; +use crate ::gsp::{GSP_PAGE_SHIFT ,GSP_PAGE_SIZE }; +use crate ::regs::NV_PGSP_QUEUE_HEAD ;
The custom with registers to just import the regs module and prefix all registers with regs::.
drivers/gpu/nova-core/gsp/cmdq.rs line 28
Quoted 7 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +7
-0+use crate ::sbuffer::SBufferIter ; + +pub (crate )trait CommandToGsp :Sized +FromBytes +AsBytes { +const FUNCTION :MsgFunction ; +} + +pub (crate )trait CommandToGspWithPayload :CommandToGsp {}
Unfortunately this hierarchy won't work, because this way a command that is expected to take a payload can be sent without any.
I'd suggest instead having a CommandToGspBase trait that sets the FUNCTION, and have CommandToGsp and CommandToGspWithPayload require it.
drivers/gpu/nova-core/gsp/cmdq.rs line 32
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +4
-0+ +pub (crate )trait MessageFromGsp :Sized +FromBytes +AsBytes { +const FUNCTION :MsgFunction ; +}
Just a bit of documentation for these new traits please. :)
(also applies to all undocumented types/methods in this file)
<snip>
Quoted 6 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +6
-0+impl DmaGspMem { +fn new (dev: &device::Device <device::Bound >) ->Result <Self > { +const MSGQ_SIZE :u32 =size_of ::<Msgq >()as u32 ; +const RX_HDR_OFF :u32 =offset_of! (Msgq , rx)as u32 ; + + let gsp_mem =
drivers/gpu/nova-core/gsp/cmdq.rs line 90
Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10
-0+CoherentAllocation ::<GspMem >::alloc_coherent (dev,1 ,GFP_KERNEL | __GFP_ZERO)?; +dma_write! (gsp_mem[0 ].ptes =PteArray ::new (gsp_mem.dma_handle ())?)?; +dma_write! (gsp_mem[0 ].cpuq.tx =MsgqTxHeader ::new (MSGQ_SIZE ,RX_HDR_OFF ,MSGQ_NUM_PAGES ))?; +dma_write! (gsp_mem[0 ].cpuq.rx =MsgqRxHeader ::new ())?; + +Ok (Self (gsp_mem)) + } + +// Allocates the various regions for the command and reduces the payload size +// to match what is needed for the command.
Let's also explain what the elements in the returned tuple are.
Quoted 11 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +11
-0+// +// # Errors +// +// Returns `Err(EAGAIN)` if the driver area is too small to hold the +// requested command. +fn allocate_command_regions <'a,M :CommandToGsp >( + &'amut self, + payload_size:usize , + ) ->Result <(&'amut GspMsgElement , &'amut M , &'amut [u8 ], &'amut [u8 ])> { +let driver_area = self.driver_write_area (); +let msg_size =size_of ::<GspMsgElement >() +size_of ::<M >() + payload_size;
drivers/gpu/nova-core/gsp/cmdq.rs line 111
Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10
-0+let driver_area_size = (driver_area.0 .len () + driver_area.1 .len ()) <<GSP_PAGE_SHIFT ; + +if msg_size > driver_area_size { +return Err (EAGAIN ); + } + +#[allow(clippy::incompatible_msrv)] +let (msg_header_slice, slice_1) = driver_area + .0 + .as_flattened_mut ()
We are going to have a slight problem here. as_flattened_mut is only available since Rust 1.80... and we need to support 1.78.
I suppose we could work this around by defining a custom function tht just calls as_flattened_mut in Rust versions that support it and provides an alternative implementation for those that don't, but I wonder whether there is a better way.
Quoted 9 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +9
-0+ .split_at_mut(size_of::<GspMsgElement>()); +let msg_header =GspMsgElement ::from_bytes_mut (msg_header_slice).ok_or (EINVAL )?; +let (cmd_slice, payload_1) = slice_1.split_at_mut (size_of ::<M >()); +let cmd =M ::from_bytes_mut (cmd_slice).ok_or (EINVAL )?; + +#[allow(clippy::incompatible_msrv)] +let payload_2 = driver_area.1 .as_flattened_mut (); + +let (payload_1, payload_2) =if payload_1.len () > payload_size {
drivers/gpu/nova-core/gsp/cmdq.rs line 130
Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10
-0+// Payload fits entirely in payload_1 + (&mut payload_1[..payload_size], &mut payload_2[0 ..0 ]) + }else { +// Need all of payload_1 and some of payload_2 +let payload_2_len = payload_size - payload_1.len (); + (payload_1, &mut payload_2[..payload_2_len]) + }; + +Ok ((msg_header, cmd, payload_1, payload_2)) + }
We should probably add a few comments in this function to explain the steps, otherwise it can be a bit difficult to parse.
drivers/gpu/nova-core/gsp/cmdq.rs line 279
Quoted 13 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +13
-0+fn calculate_checksum <T :Iterator <Item =u8 >>(it:T ) ->u32 { +let sum64 = it + .enumerate () + .map (|(idx, byte)| (((idx %8 ) *8 )as u32 , byte)) + .fold (0 , |acc, (rol, byte)| acc ^ u64::from (byte).rotate_left (rol)); + + ((sum64 >>32 )as u32 ) ^ (sum64as u32 ) + } + +// Notify GSP that we have updated the command queue pointers. +fn notify_gsp (bar: &Bar0 ) { +NV_PGSP_QUEUE_HEAD ::default ().set_address (0 ).write (bar); + }
Can we move this method to regs.rs, as an impl block of NV_PGSP_QUEUE_HEAD? It makes more sense to have it there imho.
Quoted 109 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +109
-0+ +#[expect(unused)] +pub (crate )fn send_gsp_command <M ,E >(&mut self, bar: &Bar0 , init:impl Init <M ,E >) ->Result +where +M :CommandToGsp , +// This allows all error types, including `Infallible`, to be used with `init`. Without +// this we cannot use regular stack objects as `init` since their `Init` implementation +// does not return any error. +Error :From <E >, + { +#[repr( C )] +struct FullCommand <M > { + hdr:GspMsgElement , + cmd:M , + } +let (msg_header, cmd, _, _) = self.gsp_mem.allocate_command_regions ::<M >(0 )?; + +let seq = self.seq; +let initializer =try_init! (FullCommand { + hdr <-GspMsgElement ::init (seq,size_of ::<M >(),M ::FUNCTION ), + cmd <- init, + }); + +// Fill the header and command in-place. +// SAFETY: +// - allocate_command_regions guarantees msg_header points to enough +// space in the command queue to contain FullCommand +// - __init ensures the command header and struct a fully initialized +unsafe { + initializer.__init (msg_header.as_bytes_mut ().as_mut_ptr ().cast ())?; + } + + msg_header.set_checksum (Cmdq ::calculate_checksum (SBufferIter ::new_reader ([ + msg_header.as_bytes (), + cmd.as_bytes (), + ]))); + +dev_info! ( + &self.dev, +"GSP RPC: send: seq# {}, function=0x{:x} ({}), length=0x{:x}\n" , + self.seq, + msg_header.function_number (), + msg_header.function ()?, + msg_header.length (), + ); + +let elem_count = msg_header.element_count (); + self.seq +=1 ; + self.gsp_mem.advance_cpu_write_ptr (elem_count); +Cmdq ::notify_gsp (bar); + +Ok (()) + } + +#[expect(unused)] +pub (crate )fn send_gsp_command_with_payload <M ,E >( + &mut self, + bar: &Bar0 , + payload_size:usize , + init:impl Init <M ,E >, + init_payload:impl FnOnce (SBufferIter <core::array::IntoIter <&mut [u8 ],2 >>) ->Result , + ) ->Result +where +M :CommandToGspWithPayload , +// This allows all error types, including `Infallible`, to be used with `init`. Without +// this we cannot use regular stack objects as `init` since their `Init` implementation +// does not return any error. +Error :From <E >, + { +#[repr( C )] +struct FullCommand <M > { + hdr:GspMsgElement , + cmd:M , + } +let (msg_header, cmd, payload_1, payload_2) = + self.gsp_mem.allocate_command_regions ::<M >(payload_size)?; + +let seq = self.seq; +let initializer =try_init! (FullCommand { + hdr <-GspMsgElement ::init (seq,size_of ::<M >() + payload_size,M ::FUNCTION ), + cmd <- init, + }); + +// Fill the header and command in-place. +// SAFETY: +// - allocate_command_regions guarantees msg_header points to enough +// space in the command queue to contain FullCommand +// - __init ensures the command header and struct a fully initialized +unsafe { + initializer.__init (msg_header.as_bytes_mut ().as_mut_ptr ().cast ())?; + } + +// Fill the payload +let sbuffer =SBufferIter ::new_writer ([&mut payload_1[..], &mut payload_2[..]]); +init_payload (sbuffer)?; + + msg_header.set_checksum (Cmdq ::calculate_checksum (SBufferIter ::new_reader ([ + msg_header.as_bytes (), + cmd.as_bytes (), + payload_1, + payload_2, + ]))); + +dev_info! ( + &self.dev, +"GSP RPC: send: seq# {}, function=0x{:x} ({}), length=0x{:x}\n" , + self.seq, + msg_header.function_number (), + msg_header.function ()?,
drivers/gpu/nova-core/gsp/cmdq.rs line 398
Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10
-0+ msg_header.length(), + ); + +let elem_count = msg_header.element_count (); + self.seq +=1 ; + self.gsp_mem.advance_cpu_write_ptr (elem_count); +Cmdq ::notify_gsp (bar); + +Ok (()) + }
I think we can factorize most of send_gsp_command and send_gsp_command_with_payload into a private method that requires a CommandToGspBase and takes the same parameters as send_gsp_command_with_payload. All it would need to do when called from send_gsp_command is take a closure that doesn't write any payload.
Quoted 84 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +84
-0+ +#[expect(unused)] +pub (crate )fn receive_msg_from_gsp <M :MessageFromGsp ,R >( + &mut self, + timeout:Delta , + init:impl FnOnce (&M ,SBufferIter <core::array::IntoIter <&[u8 ],2 >>) ->Result <R >, + ) ->Result <R > { +let driver_area =read_poll_timeout ( + ||Ok (self.gsp_mem.driver_read_area ()), + |driver_area: &(&[[u8 ;4096 ]], &[[u8 ;4096 ]])| !driver_area.0 .is_empty (), +Delta ::from_millis (10 ), + timeout, + )?; + +#[allow(clippy::incompatible_msrv)] +let (msg_header_slice, slice_1) = driver_area + .0 + .as_flattened () + .split_at (size_of ::<GspMsgElement >()); +let msg_header =GspMsgElement ::from_bytes (msg_header_slice).ok_or (EIO )?; +if msg_header.length () <size_of ::<M >()as u32 { +return Err (EIO ); + } + +let function:MsgFunction = msg_header.function ().map_err (|_| { +dev_info! ( + self.dev, +"GSP RPC: receive: seq# {}, bad function=0x{:x}, length=0x{:x}\n" , + msg_header.sequence (), + msg_header.function_number (), + msg_header.length (), + ); +EIO + })?; + +// Log RPC receive with message type decoding +dev_info! ( + self.dev, +"GSP RPC: receive: seq# {}, function=0x{:x} ({}), length=0x{:x}\n" , + msg_header.sequence (), + msg_header.function_number (), + function, + msg_header.length (), + ); + +let (cmd_slice, payload_1) = slice_1.split_at (size_of ::<M >()); +#[allow(clippy::incompatible_msrv)] +let payload_2 = driver_area.1 .as_flattened (); + +// Cut the payload slice(s) down to the actual length of the payload. +let (cmd_payload_1, cmd_payload_2) = +if payload_1.len () > msg_header.length ()as usize -size_of ::<M >() { + ( + payload_1 + .split_at (msg_header.length ()as usize -size_of ::<M >()) + .0 , + &payload_2[0 ..0 ], + ) + }else { + ( + payload_1, + payload_2 + .split_at (msg_header.length ()as usize -size_of ::<M >() - payload_1.len ()) + .0 , + ) + }; + +if Cmdq ::calculate_checksum (SBufferIter ::new_reader ([ + msg_header.as_bytes (), + cmd_slice, + cmd_payload_1, + cmd_payload_2, + ])) !=0 + { +dev_err! ( + self.dev, +"GSP RPC: receive: Call {} - bad checksum" , + msg_header.sequence () + ); +return Err (EIO ); + } + +let result =if function ==M ::FUNCTION { +let cmd =M ::from_bytes (cmd_slice).ok_or (EINVAL )?;
drivers/gpu/nova-core/gsp/cmdq.rs line 492
Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10
-0+let sbuffer =SBufferIter ::new_reader ([cmd_payload_1, cmd_payload_2]); +init (cmd, sbuffer) + }else { +Err (ERANGE ) + }; + + self.gsp_mem + .advance_cpu_read_ptr (msg_header.length ().div_ceil (GSP_PAGE_SIZE as u32 )); + result + }
Here we probably also want to comment the different steps a bit.
Re: [PATCH v5 07/14] gpu: nova-core: gsp: Add GSP command queue handling
On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 57 lines · Alistair Popple
Quoted 56 lines
This commit introduces core infrastructure for handling GSP command and message queues in the nova-core driver. The command queue system enables bidirectional communication between the host driver and GSP firmware through a remote message passing interface.
The interface is based on passing serialised data structures over a ring buffer with separate transmit and receive queues. Commands are sent by writing to the CPU transmit queue and waiting for completion via the receive queue.
To ensure safety mutable or immutable (depending on whether it is a send or receive operation) references are taken on the command queue when allocating the message to write/read to. This ensures message memory remains valid and the command queue can't be mutated whilst an operation is in progress.
Currently this is only used by the
probe()routine and therefore can only used by a single thread of execution. Locking to enable safe access from multiple threads will be introduced in a future series when that becomes necessary.Signed-off-by Alistair Popple <[email protected]>---
Changes for v4:
- Useread_poll_timeout()instead ofwait_on()
- Switch to usinginit!(Thanks Alex for figuring out the required workarounds)
- Pass the enum type into the RPC bindings instead of a raw u32
- Fixup the TODOs for extracting/allocating the send command regions
- Split the sending functions into one taking just a command struct and another taking a command struct with extra payloadChanges for v3:
- Reduce the receive payloads to the correct size
- Use opaque bindings
- Clean up of the command queue PTE creation
- Add an enum for the GSP functions
- Rename GspCommandToGsp and GspMessageFromGsp
- Rename GspCmdq
- Add function to notify GSP of updated queue pointers
- Inline driver area access functions
- Fixup receive area calculationsChanges for v2:
- Rebased on Alex's latest series--- drivers/gpu/nova-core/gsp.rs | 9 + drivers/gpu/nova-core/gsp/cmdq.rs | 493 ++++++++++++++++++++++++++++++ drivers/gpu/nova-core/regs.rs | 4 + drivers/gpu/nova-core/sbuffer.rs | 2 - scripts/Makefile.build | 2 +- 5 files changed, 507 insertions(+), 3 deletions(-) create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs
[...]
Quoted 11 lines
drivers/gpu/nova-core/gsp/cmdq.rs line 28
Quoted 7 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +7
-0+use crate ::sbuffer::SBufferIter ; + +pub (crate )trait CommandToGsp :Sized +FromBytes +AsBytes { +const FUNCTION :MsgFunction ; +} + +pub (crate )trait CommandToGspWithPayload :CommandToGsp {}Unfortunately this hierarchy won't work, because this way a command that is expected to take a payload can be sent without any.
Quoted 3 lines
I'd suggest instead having a
CommandToGspBasetrait that sets the FUNCTION, and haveCommandToGspandCommandToGspWithPayloadrequire it.
Ok.
Quoted 8 lines
drivers/gpu/nova-core/gsp/cmdq.rs line 32
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +4
-0+ +pub (crate )trait MessageFromGsp :Sized +FromBytes +AsBytes { +const FUNCTION :MsgFunction ; +}Just a bit of documentation for these new traits please. :)
(also applies to all undocumented types/methods in this file)
Sure.
Quoted 45 lines
<snip>
Quoted 6 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +6
-0+impl DmaGspMem { +fn new (dev: &device::Device <device::Bound >) ->Result <Self > { +const MSGQ_SIZE :u32 =size_of ::<Msgq >()as u32 ; +const RX_HDR_OFF :u32 =offset_of! (Msgq , rx)as u32 ; + + let gsp_mem =drivers/gpu/nova-core/gsp/cmdq.rs line 90
Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10
-0+CoherentAllocation ::<GspMem >::alloc_coherent (dev,1 ,GFP_KERNEL | __GFP_ZERO)?; +dma_write! (gsp_mem[0 ].ptes =PteArray ::new (gsp_mem.dma_handle ())?)?; +dma_write! (gsp_mem[0 ].cpuq.tx =MsgqTxHeader ::new (MSGQ_SIZE ,RX_HDR_OFF ,MSGQ_NUM_PAGES ))?; +dma_write! (gsp_mem[0 ].cpuq.rx =MsgqRxHeader ::new ())?; + +Ok (Self (gsp_mem)) + } + +// Allocates the various regions for the command and reduces the payload size +// to match what is needed for the command. Let's also explain what the elements in the returned tuple are.
Quoted 11 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +11
-0+// +// # Errors +// +// Returns `Err(EAGAIN)` if the driver area is too small to hold the +// requested command. +fn allocate_command_regions <'a,M :CommandToGsp >( + &'amut self, + payload_size:usize , + ) ->Result <(&'amut GspMsgElement , &'amut M , &'amut [u8 ], &'amut [u8 ])> { +let driver_area = self.driver_write_area (); +let msg_size =size_of ::<GspMsgElement >() +size_of ::<M >() + payload_size;drivers/gpu/nova-core/gsp/cmdq.rs line 111
Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10
-0+let driver_area_size = (driver_area.0 .len () + driver_area.1 .len ()) <<GSP_PAGE_SHIFT ; + +if msg_size > driver_area_size { +return Err (EAGAIN ); + } + +#[allow(clippy::incompatible_msrv)] +let (msg_header_slice, slice_1) = driver_area + .0 + .as_flattened_mut ()We are going to have a slight problem here.
as_flattened_mutis only available since Rust 1.80... and we need to support 1.78.
Quoted 4 lines
I suppose we could work this around by defining a custom function tht just calls
as_flattened_mutin Rust versions that support it and provides an alternative implementation for those that don't, but I wonder whether there is a better way.
Oh I didn't realise the function name had changed, I thought we just needed to allow the experimental feature in older versions. I probably wont get to this this week so have left it as a TODO for now.
Quoted 14 lines
Quoted 14 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +14
-0+ .split_at_mut(size_of::<GspMsgElement>()); +let msg_header =GspMsgElement ::from_bytes_mut (msg_header_slice).ok_or (EINVAL )?; +let (cmd_slice, payload_1) = slice_1.split_at_mut (size_of ::<M >()); +let cmd =M ::from_bytes_mut (cmd_slice).ok_or (EINVAL )?; + +#[allow(clippy::incompatible_msrv)] +let payload_2 = driver_area.1 .as_flattened_mut (); + +let (payload_1, payload_2) =if payload_1.len () > payload_size { +// Payload fits entirely in payload_1 + (&mut payload_1[..payload_size], &mut payload_2[0 ..0 ]) + }else { +// Need all of payload_1 and some of payload_2 +let payload_2_len = payload_size - payload_1.len ();
Quoted 8 lines
drivers/gpu/nova-core/gsp/cmdq.rs line 130
Quoted 5 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +5
-0+ (payload_1, &mut payload_2[..payload_2_len]) + }; + +Ok ((msg_header, cmd, payload_1, payload_2)) + }We should probably add a few comments in this function to explain the steps, otherwise it can be a bit difficult to parse.
Ok.
Quoted 8 lines
Quoted 8 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +8
-0+fn calculate_checksum <T :Iterator <Item =u8 >>(it:T ) ->u32 { +let sum64 = it + .enumerate () + .map (|(idx, byte)| (((idx %8 ) *8 )as u32 , byte)) + .fold (0 , |acc, (rol, byte)| acc ^ u64::from (byte).rotate_left (rol)); + + ((sum64 >>32 )as u32 ) ^ (sum64as u32 ) + }
Quoted 8 lines
drivers/gpu/nova-core/gsp/cmdq.rs line 279
Quoted 5 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +5
-0+ +// Notify GSP that we have updated the command queue pointers. +fn notify_gsp (bar: &Bar0 ) { +NV_PGSP_QUEUE_HEAD ::default ().set_address (0 ).write (bar); + }Can we move this method to
regs.rs, as an impl block ofNV_PGSP_QUEUE_HEAD? It makes more sense to have it there imho.
We could, but I think it belongs here. If I'm understanding things correctly the value of this register is FW/SW defined. For now GSP doesn't look at the value so it doesn't really matter what we write, but it's possible that could change which means we want it here in the code that talks to GSP.
Quoted 118 lines
Quoted 118 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +118
-0+ +#[expect(unused)] +pub (crate )fn send_gsp_command <M ,E >(&mut self, bar: &Bar0 , init:impl Init <M ,E >) ->Result +where +M :CommandToGsp , +// This allows all error types, including `Infallible`, to be used with `init`. Without +// this we cannot use regular stack objects as `init` since their `Init` implementation +// does not return any error. +Error :From <E >, + { +#[repr( C )] +struct FullCommand <M > { + hdr:GspMsgElement , + cmd:M , + } +let (msg_header, cmd, _, _) = self.gsp_mem.allocate_command_regions ::<M >(0 )?; + +let seq = self.seq; +let initializer =try_init! (FullCommand { + hdr <-GspMsgElement ::init (seq,size_of ::<M >(),M ::FUNCTION ), + cmd <- init, + }); + +// Fill the header and command in-place. +// SAFETY: +// - allocate_command_regions guarantees msg_header points to enough +// space in the command queue to contain FullCommand +// - __init ensures the command header and struct a fully initialized +unsafe { + initializer.__init (msg_header.as_bytes_mut ().as_mut_ptr ().cast ())?; + } + + msg_header.set_checksum (Cmdq ::calculate_checksum (SBufferIter ::new_reader ([ + msg_header.as_bytes (), + cmd.as_bytes (), + ]))); + +dev_info! ( + &self.dev, +"GSP RPC: send: seq# {}, function=0x{:x} ({}), length=0x{:x}\n" , + self.seq, + msg_header.function_number (), + msg_header.function ()?, + msg_header.length (), + ); + +let elem_count = msg_header.element_count (); + self.seq +=1 ; + self.gsp_mem.advance_cpu_write_ptr (elem_count); +Cmdq ::notify_gsp (bar); + +Ok (()) + } + +#[expect(unused)] +pub (crate )fn send_gsp_command_with_payload <M ,E >( + &mut self, + bar: &Bar0 , + payload_size:usize , + init:impl Init <M ,E >, + init_payload:impl FnOnce (SBufferIter <core::array::IntoIter <&mut [u8 ],2 >>) ->Result , + ) ->Result +where +M :CommandToGspWithPayload , +// This allows all error types, including `Infallible`, to be used with `init`. Without +// this we cannot use regular stack objects as `init` since their `Init` implementation +// does not return any error. +Error :From <E >, + { +#[repr( C )] +struct FullCommand <M > { + hdr:GspMsgElement , + cmd:M , + } +let (msg_header, cmd, payload_1, payload_2) = + self.gsp_mem.allocate_command_regions ::<M >(payload_size)?; + +let seq = self.seq; +let initializer =try_init! (FullCommand { + hdr <-GspMsgElement ::init (seq,size_of ::<M >() + payload_size,M ::FUNCTION ), + cmd <- init, + }); + +// Fill the header and command in-place. +// SAFETY: +// - allocate_command_regions guarantees msg_header points to enough +// space in the command queue to contain FullCommand +// - __init ensures the command header and struct a fully initialized +unsafe { + initializer.__init (msg_header.as_bytes_mut ().as_mut_ptr ().cast ())?; + } + +// Fill the payload +let sbuffer =SBufferIter ::new_writer ([&mut payload_1[..], &mut payload_2[..]]); +init_payload (sbuffer)?; + + msg_header.set_checksum (Cmdq ::calculate_checksum (SBufferIter ::new_reader ([ + msg_header.as_bytes (), + cmd.as_bytes (), + payload_1, + payload_2, + ]))); + +dev_info! ( + &self.dev, +"GSP RPC: send: seq# {}, function=0x{:x} ({}), length=0x{:x}\n" , + self.seq, + msg_header.function_number (), + msg_header.function ()?, + msg_header.length (), + ); + +let elem_count = msg_header.element_count (); + self.seq +=1 ; + self.gsp_mem.advance_cpu_write_ptr (elem_count); +Cmdq ::notify_gsp (bar); + +Ok (())
Quoted 8 lines
Quoted 1 line
+ }
I think we can factorize most of
send_gsp_commandandsend_gsp_command_with_payloadinto a private method that requires aCommandToGspBaseand takes the same parameters assend_gsp_command_with_payload. All it would need to do when called fromsend_gsp_commandis take a closure that doesn't write any payload.
You're right, that is nicer. I was thinking it would be good to have a simple function that ignored the payloads, but it didn't really end up any simpler.
Quoted 88 lines
Quoted 88 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +88
-0+ +#[expect(unused)] +pub (crate )fn receive_msg_from_gsp <M :MessageFromGsp ,R >( + &mut self, + timeout:Delta , + init:impl FnOnce (&M ,SBufferIter <core::array::IntoIter <&[u8 ],2 >>) ->Result <R >, + ) ->Result <R > { +let driver_area =read_poll_timeout ( + ||Ok (self.gsp_mem.driver_read_area ()), + |driver_area: &(&[[u8 ;4096 ]], &[[u8 ;4096 ]])| !driver_area.0 .is_empty (), +Delta ::from_millis (10 ), + timeout, + )?; + +#[allow(clippy::incompatible_msrv)] +let (msg_header_slice, slice_1) = driver_area + .0 + .as_flattened () + .split_at (size_of ::<GspMsgElement >()); +let msg_header =GspMsgElement ::from_bytes (msg_header_slice).ok_or (EIO )?; +if msg_header.length () <size_of ::<M >()as u32 { +return Err (EIO ); + } + +let function:MsgFunction = msg_header.function ().map_err (|_| { +dev_info! ( + self.dev, +"GSP RPC: receive: seq# {}, bad function=0x{:x}, length=0x{:x}\n" , + msg_header.sequence (), + msg_header.function_number (), + msg_header.length (), + ); +EIO + })?; + +// Log RPC receive with message type decoding +dev_info! ( + self.dev, +"GSP RPC: receive: seq# {}, function=0x{:x} ({}), length=0x{:x}\n" , + msg_header.sequence (), + msg_header.function_number (), + function, + msg_header.length (), + ); + +let (cmd_slice, payload_1) = slice_1.split_at (size_of ::<M >()); +#[allow(clippy::incompatible_msrv)] +let payload_2 = driver_area.1 .as_flattened (); + +// Cut the payload slice(s) down to the actual length of the payload. +let (cmd_payload_1, cmd_payload_2) = +if payload_1.len () > msg_header.length ()as usize -size_of ::<M >() { + ( + payload_1 + .split_at (msg_header.length ()as usize -size_of ::<M >()) + .0 , + &payload_2[0 ..0 ], + ) + }else { + ( + payload_1, + payload_2 + .split_at (msg_header.length ()as usize -size_of ::<M >() - payload_1.len ()) + .0 , + ) + }; + +if Cmdq ::calculate_checksum (SBufferIter ::new_reader ([ + msg_header.as_bytes (), + cmd_slice, + cmd_payload_1, + cmd_payload_2, + ])) !=0 + { +dev_err! ( + self.dev, +"GSP RPC: receive: Call {} - bad checksum" , + msg_header.sequence () + ); +return Err (EIO ); + } + +let result =if function ==M ::FUNCTION { +let cmd =M ::from_bytes (cmd_slice).ok_or (EINVAL )?; +let sbuffer =SBufferIter ::new_reader ([cmd_payload_1, cmd_payload_2]); +init (cmd, sbuffer) + }else { +Err (ERANGE )
Quoted 8 lines
drivers/gpu/nova-core/gsp/cmdq.rs line 492
Quoted 6 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +6
-0+ }; + + self.gsp_mem + .advance_cpu_read_ptr (msg_header.length ().div_ceil (GSP_PAGE_SIZE as u32 )); + result + }Here we probably also want to comment the different steps a bit.
Ok.
Re: [PATCH v5 07/14] gpu: nova-core: gsp: Add GSP command queue handling
In reply to Alistair Popplescripts/Makefile.build $(obj)/%.lst: $(obj)/%.c FORCE · line 320
Quoted 3 lines · Alistair Popple
scripts/Makefile.build +1
-1-rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg +rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg,slice_flatten
This is not sorted, and it is not mentioned in the comment right above it -- in general, please try to clearly mention this sort of addition in the commit message too.
As Alexandre mentions, it will not work to use it as-is. However, what happened in 1.80 was just a rename, but the feature was available since 1.67 as flatten.
If you want to still use it through an indirection or similar, then please move it to a new commit that explains the addition of the feature and the indirection to use it.
See e.g. the file() function I added in e.g. rust/macros/helpers.rs in commit 36174d16f3ec ("rust: kunit: support KUnit-mapped assert! macros in #[test]s") to see how to set it up.
Essentially it is:
config RUSTC_HAS_SPAN_FILE
def_bool RUSTC_VERSION >= 108800And then:
pub(crate) fn file() -> String {
#[cfg(not(CONFIG_RUSTC_HAS_SPAN_FILE))]
{
...
}
#[cfg(CONFIG_RUSTC_HAS_SPAN_FILE)]
#[allow(clippy::incompatible_msrv)]
{
...
}
}I hope that helps.
Thanks!
Cheers,
Miguel
Re: [PATCH v5 07/14] gpu: nova-core: gsp: Add GSP command queue handling
On 2025-10-17 at 05:44 +1100, Miguel Ojeda <[email protected]> wrote...
Quoted 9 lines · Alistair Popple
scripts/Makefile.build $(obj)/%.lst: $(obj)/%.c FORCE · line 320
Quoted 3 lines
scripts/Makefile.build +1
-1-rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg +rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg,slice_flattenThis is not sorted, and it is not mentioned in the comment right above it -- in general, please try to clearly mention this sort of addition in the commit message too.
Quoted 3 lines
As Alexandre mentions, it will not work to use it as-is. However, what happened in 1.80 was just a rename, but the feature was available since 1.67 as
flatten.
Argh! Thanks, I wasn't aware the function had been renamed.
Quoted 3 lines
If you want to still use it through an indirection or similar, then please move it to a new commit that explains the addition of the feature and the indirection to use it.
That makes sense, thanks for the explaination.
Quoted 30 lines
See e.g. the
file()function I added in e.g.rust/macros/helpers.rsin commit36174d16f3ec("rust: kunit: support KUnit-mappedassert!macros in#[test]s") to see how to set it up.Essentially it is:
config RUSTC_HAS_SPAN_FILE def_bool RUSTC_VERSION >= 108800And then:
pub(crate) fn file() -> String { #[cfg(not(CONFIG_RUSTC_HAS_SPAN_FILE))] { ... } #[cfg(CONFIG_RUSTC_HAS_SPAN_FILE)] #[allow(clippy::incompatible_msrv)] { ... } }I hope that helps.
Thanks!
Cheers,
Miguel
PATCH v5 8/14 gpu: nova-core: gsp: Create rmargs
In reply to Alistair PoppleInitialise the GSP resource manager arguments (rmargs) which provide initialisation parameters to the GSP firmware during boot. The rmargs structure contains arguments to configure the GSP message/command queue location.
These are mapped for coherent DMA and added to the libos data structure for access when booting GSP.
---
Changes for v5:
- Derive Zeroable trait
Changes for v2:
- Rebased on Alex's latest series
--- drivers/gpu/nova-core/gsp.rs | 16 +++ drivers/gpu/nova-core/gsp/cmdq.rs | 24 +++- drivers/gpu/nova-core/gsp/fw.rs | 60 ++++++++ .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ------------------ 4 files changed, 97 insertions(+), 135 deletions(-)
drivers/gpu/nova-core/gsp.rs +16 -0
@@ -19,6 +19,7 @@mod fw;use fw::LibosMemoryRegionInitArgument ; +use fw::GspArgumentsCached ;pub (crate )mod cmdq;
@@ -36,6 +37,7 @@ pub(crate) struct Gsp { logintr: LogBuffer, logrm: LogBuffer, pub(crate) cmdq: Cmdq, + rmargs: CoherentAllocation<GspArgumentsCached>, }#[repr( C )]
@@ -117,12 +119,26 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self// Creates its own PTE array. let cmdq =Cmdq ::new (dev)?; +let rmargs =CoherentAllocation ::<GspArgumentsCached >::alloc_coherent ( + dev, +1 , +GFP_KERNEL | __GFP_ZERO, + )?; +dma_write! (libos[3 ] =LibosMemoryRegionInitArgument ::new ("RMARGS" , &rmargs)?)?; + +dma_write! ( + rmargs[0 ] = fw::GspArgumentsCached ::new ( + fw::MessageQueueInitArguments ::new (&cmdq), + fw::GspSrInitArguments ::new () + ) + )?;Ok (try_pin_init! (Self { libos, loginit, logintr, logrm, + rmargs, cmdq, })) }
drivers/gpu/nova-core/gsp/cmdq.rs +21 -3
@@ -6,7 +6,7 @@use kernel::alloc::flags::GFP_KERNEL ;use kernel::device; -use kernel::dma::CoherentAllocation ; +use kernel::dma::{CoherentAllocation ,DmaAddress };use kernel::dma_write;use kernel::io::poll::read_poll_timeout;use kernel::prelude::*;
@@ -247,10 +247,25 @@ pub(crate) struct Cmdq { dev: ARef<device::Device>, seq: u32, gsp_mem: DmaGspMem, - pub _nr_ptes: u32, }impl Cmdq { +/// Offset of the data after the PTEs. +const POST_PTE_OFFSET :usize = core::mem::offset_of!(GspMem , cpuq); + +/// Offset of command queue ring buffer. +pub (crate )const CMDQ_OFFSET :usize = core::mem::offset_of!(GspMem , cpuq) + + core::mem::offset_of!(Msgq , msgq) + -Self ::POST_PTE_OFFSET ; + +/// Offset of message queue ring buffer. +pub (crate )const STATQ_OFFSET :usize = core::mem::offset_of!(GspMem , gspq) + + core::mem::offset_of!(Msgq , msgq) + -Self ::POST_PTE_OFFSET ; + +/// Number of page table entries for the GSP shared region. +pub (crate )const NUM_PTES :usize =size_of ::<GspMem >() >>GSP_PAGE_SHIFT ; +pub (crate )fn new (dev: &device::Device <device::Bound >) ->Result <Cmdq > {let gsp_mem =DmaGspMem ::new (dev)?;let nr_ptes =size_of ::<GspMem >() >>GSP_PAGE_SHIFT ;
@@ -260,7 +275,6 @@ pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> { dev: dev.into(), seq: 0, gsp_mem, - _nr_ptes: nr_ptes as u32, }) }
@@ -490,4 +504,8 @@ pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>( .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32)); result } + +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.gsp_mem.0 .dma_handle () + } }
drivers/gpu/nova-core/gsp/fw.rs +60 -0
69 lines not on this page · raw message
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +0 -132
138 lines not on this page · raw message
Signature · 1 line
2.50.1
Alexandre Courbot
maintainerRe: [PATCH v5 08/14] gpu: nova-core: gsp: Create rmargs
Quoted 15 lines · Alistair Popple
Initialise the GSP resource manager arguments (rmargs) which provide initialisation parameters to the GSP firmware during boot. The rmargs structure contains arguments to configure the GSP message/command queue location.
These are mapped for coherent DMA and added to the libos data structure for access when booting GSP.
Signed-off-by Alistair Popple <[email protected]>---
Changes for v5:
- Derive Zeroable trait
Quoted 7 lines
Changes for v2:
- Rebased on Alex's latest series--- drivers/gpu/nova-core/gsp.rs | 16 +++ drivers/gpu/nova-core/gsp/cmdq.rs | 24 +++- drivers/gpu/nova-core/gsp/fw.rs | 60 ++++++++ .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ------------------
Mmm, looks like we are removing bindings. Can we not add them in the first place? :)
Quoted 23 lines
4 files changed, 97 insertions(+), 135 deletions(-)drivers/gpu/nova-core/gsp.rs +2
-0@@ -19,6 +19,7 @@mod fw;use fw::LibosMemoryRegionInitArgument ; +use fw::GspArgumentsCached ;pub (crate )mod cmdq;@@ -36,6 +37,7 @@ pub(crate) struct Gsp { logintr: LogBuffer, logrm: LogBuffer, pub(crate) cmdq: Cmdq, + rmargs: CoherentAllocation<GspArgumentsCached>, }#[repr( C )]
drivers/gpu/nova-core/gsp.rs pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self · line 122
Quoted 4 lines
drivers/gpu/nova-core/gsp.rs +1
-0@@ -117,12 +119,26 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self// Creates its own PTE array. let cmdq =Cmdq ::new (dev)?; + let rmargs = CoherentAllocation::<GspArgumentsCached>::alloc_coherent(
Let's add a space between the declaration of cmdq and rmargs.
Quoted 36 lines
drivers/gpu/nova-core/gsp.rs +13
-0+ dev, + 1, + GFP_KERNEL | __GFP_ZERO, + )?; +dma_write! (libos[3 ] =LibosMemoryRegionInitArgument ::new ("RMARGS" , &rmargs)?)?; + +dma_write! ( + rmargs[0 ] = fw::GspArgumentsCached ::new ( + fw::MessageQueueInitArguments ::new (&cmdq), + fw::GspSrInitArguments ::new () + ) + )?;Ok (try_pin_init! (Self { libos, loginit, logintr, logrm, + rmargs, cmdq, })) }drivers/gpu/nova-core/gsp/cmdq.rs +1
-1@@ -6,7 +6,7 @@use kernel::alloc::flags::GFP_KERNEL ;use kernel::device; -use kernel::dma::CoherentAllocation ; +use kernel::dma::{CoherentAllocation ,DmaAddress };use kernel::dma_write;use kernel::io::poll::read_poll_timeout;use kernel::prelude::*;
drivers/gpu/nova-core/gsp/cmdq.rs pub(crate) struct Cmdq { · line 250
Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs +0
-1@@ -247,10 +247,25 @@ pub(crate) struct Cmdq { dev: ARef<device::Device>, seq: u32, gsp_mem: DmaGspMem, - pub _nr_ptes: u32,
We probably shouldn't have introduced this unused member in the first place.
Quoted 68 lines
drivers/gpu/nova-core/gsp/cmdq.rs +20
-1}impl Cmdq { +/// Offset of the data after the PTEs. +const POST_PTE_OFFSET :usize = core::mem::offset_of!(GspMem , cpuq); + +/// Offset of command queue ring buffer. +pub (crate )const CMDQ_OFFSET :usize = core::mem::offset_of!(GspMem , cpuq) + + core::mem::offset_of!(Msgq , msgq) + -Self ::POST_PTE_OFFSET ; + +/// Offset of message queue ring buffer. +pub (crate )const STATQ_OFFSET :usize = core::mem::offset_of!(GspMem , gspq) + + core::mem::offset_of!(Msgq , msgq) + -Self ::POST_PTE_OFFSET ; + +/// Number of page table entries for the GSP shared region. +pub (crate )const NUM_PTES :usize =size_of ::<GspMem >() >>GSP_PAGE_SHIFT ; +pub (crate )fn new (dev: &device::Device <device::Bound >) ->Result <Cmdq > {let gsp_mem =DmaGspMem ::new (dev)?;let nr_ptes =size_of ::<GspMem >() >>GSP_PAGE_SHIFT ;@@ -260,7 +275,6 @@ pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> { dev: dev.into(), seq: 0, gsp_mem, - _nr_ptes: nr_ptes as u32, }) }@@ -490,4 +504,8 @@ pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>( .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32)); result } + +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.gsp_mem.0 .dma_handle () + } }drivers/gpu/nova-core/gsp/fw.rs +14
-0@@ -16,6 +16,7 @@use crate ::firmware::gsp::GspFirmware ;use crate ::gpu::Chipset ; +use crate ::gsp::cmdq::Cmdq ;use crate ::gsp::FbLayout ;use crate ::gsp::GSP_PAGE_SIZE ;@@ -483,3 +484,62 @@ unsafe impl AsBytes for GspMsgElement {}// SAFETY: This struct only contains integer types for which all bit patterns // are valid. unsafe impl FromBytes for GspMsgElement {} + +#[repr(transparent)] +pub (crate )struct GspArgumentsCached (bindings::GSP_ARGUMENTS_CACHED ); + +impl GspArgumentsCached { +pub (crate )fn new ( + queue_arguments:MessageQueueInitArguments , + sr_arguments:GspSrInitArguments , + ) ->Self { +Self (bindings::GSP_ARGUMENTS_CACHED { + messageQueueInitArguments: queue_arguments.0 , + srInitArguments: sr_arguments.0 , + bDmemStack:1 ,
drivers/gpu/nova-core/gsp/fw.rs unsafe impl AsBytes for GspMsgElement {} · line 509
Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10
-0+ ..Default ::default () + }) + } +} + +impl From <GspArgumentsCached >for bindings::GSP_ARGUMENTS_CACHED { +fn from (value:GspArgumentsCached ) ->Self { + value.0 + } +}
This From impl seems unneeded?
Re: [PATCH v5 08/14] gpu: nova-core: gsp: Create rmargs
On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 24 lines · Alistair Popple
Quoted 15 lines
Initialise the GSP resource manager arguments (rmargs) which provide initialisation parameters to the GSP firmware during boot. The rmargs structure contains arguments to configure the GSP message/command queue location.
These are mapped for coherent DMA and added to the libos data structure for access when booting GSP.
Signed-off-by Alistair Popple <[email protected]>---
Changes for v5:
- Derive Zeroable traitQuoted 7 lines
Changes for v2:
- Rebased on Alex's latest series--- drivers/gpu/nova-core/gsp.rs | 16 +++ drivers/gpu/nova-core/gsp/cmdq.rs | 24 +++- drivers/gpu/nova-core/gsp/fw.rs | 60 ++++++++ .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ------------------
Quoted 2 lines
Mmm, looks like we are removing bindings. Can we not add them in the first place? :)
Bah. That's just bad rebasing - updating the bindings to add Zeroable to them was as much of a pain as I thought it would be :-)
Quoted 21 lines
Quoted 21 lines
4 files changed, 97 insertions(+), 135 deletions(-)drivers/gpu/nova-core/gsp.rs +2
-0@@ -19,6 +19,7 @@mod fw;use fw::LibosMemoryRegionInitArgument ; +use fw::GspArgumentsCached ;pub (crate )mod cmdq;@@ -36,6 +37,7 @@ pub(crate) struct Gsp { logintr: LogBuffer, logrm: LogBuffer, pub(crate) cmdq: Cmdq, + rmargs: CoherentAllocation<GspArgumentsCached>, }
Quoted 8 lines
drivers/gpu/nova-core/gsp.rs pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self · line 122
Quoted 6 lines
drivers/gpu/nova-core/gsp.rs +1
-0#[repr( C )] @@ -117,12 +119,26 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self// Creates its own PTE array. let cmdq =Cmdq ::new (dev)?; + let rmargs = CoherentAllocation::<GspArgumentsCached>::alloc_coherent(Let's add a space between the declaration of
cmdqandrmargs.
Ok.
Quoted 29 lines
Quoted 29 lines
drivers/gpu/nova-core/gsp.rs +13
-0+ dev, + 1, + GFP_KERNEL | __GFP_ZERO, + )?; +dma_write! (libos[3 ] =LibosMemoryRegionInitArgument ::new ("RMARGS" , &rmargs)?)?; + +dma_write! ( + rmargs[0 ] = fw::GspArgumentsCached ::new ( + fw::MessageQueueInitArguments ::new (&cmdq), + fw::GspSrInitArguments ::new () + ) + )?;Ok (try_pin_init! (Self { libos, loginit, logintr, logrm, + rmargs, cmdq, })) }drivers/gpu/nova-core/gsp/cmdq.rs +0
-0@@ -6,7 +6,7 @@
Quoted 14 lines
drivers/gpu/nova-core/gsp/cmdq.rs pub(crate) struct Cmdq { · line 250
Quoted 12 lines
drivers/gpu/nova-core/gsp/cmdq.rs +1
-2use kernel::alloc::flags::GFP_KERNEL ;use kernel::device; -use kernel::dma::CoherentAllocation ; +use kernel::dma::{CoherentAllocation ,DmaAddress };use kernel::dma_write;use kernel::io::poll::read_poll_timeout;use kernel::prelude::*;@@ -247,10 +247,25 @@ pub(crate) struct Cmdq { dev: ARef<device::Device>, seq: u32, gsp_mem: DmaGspMem, - pub _nr_ptes: u32,We probably shouldn't have introduced this unused member in the first place.
Good point, it's a hangover from the implementations in previous versions.
Quoted 73 lines
Quoted 73 lines
drivers/gpu/nova-core/gsp/cmdq.rs +20
-1}impl Cmdq { +/// Offset of the data after the PTEs. +const POST_PTE_OFFSET :usize = core::mem::offset_of!(GspMem , cpuq); + +/// Offset of command queue ring buffer. +pub (crate )const CMDQ_OFFSET :usize = core::mem::offset_of!(GspMem , cpuq) + + core::mem::offset_of!(Msgq , msgq) + -Self ::POST_PTE_OFFSET ; + +/// Offset of message queue ring buffer. +pub (crate )const STATQ_OFFSET :usize = core::mem::offset_of!(GspMem , gspq) + + core::mem::offset_of!(Msgq , msgq) + -Self ::POST_PTE_OFFSET ; + +/// Number of page table entries for the GSP shared region. +pub (crate )const NUM_PTES :usize =size_of ::<GspMem >() >>GSP_PAGE_SHIFT ; +pub (crate )fn new (dev: &device::Device <device::Bound >) ->Result <Cmdq > {let gsp_mem =DmaGspMem ::new (dev)?;let nr_ptes =size_of ::<GspMem >() >>GSP_PAGE_SHIFT ;@@ -260,7 +275,6 @@ pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> { dev: dev.into(), seq: 0, gsp_mem, - _nr_ptes: nr_ptes as u32, }) }@@ -490,4 +504,8 @@ pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>( .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32)); result } + +pub (crate )fn dma_handle (&self) ->DmaAddress { + self.gsp_mem.0 .dma_handle () + } }drivers/gpu/nova-core/gsp/fw.rs +18
-0@@ -16,6 +16,7 @@use crate ::firmware::gsp::GspFirmware ;use crate ::gpu::Chipset ; +use crate ::gsp::cmdq::Cmdq ;use crate ::gsp::FbLayout ;use crate ::gsp::GSP_PAGE_SIZE ;@@ -483,3 +484,62 @@ unsafe impl AsBytes for GspMsgElement {}// SAFETY: This struct only contains integer types for which all bit patterns // are valid. unsafe impl FromBytes for GspMsgElement {} + +#[repr(transparent)] +pub (crate )struct GspArgumentsCached (bindings::GSP_ARGUMENTS_CACHED ); + +impl GspArgumentsCached { +pub (crate )fn new ( + queue_arguments:MessageQueueInitArguments , + sr_arguments:GspSrInitArguments , + ) ->Self { +Self (bindings::GSP_ARGUMENTS_CACHED { + messageQueueInitArguments: queue_arguments.0 , + srInitArguments: sr_arguments.0 , + bDmemStack:1 , + ..Default ::default () + }) + } +}
Quoted 8 lines
drivers/gpu/nova-core/gsp/fw.rs unsafe impl AsBytes for GspMsgElement {} · line 509
Quoted 6 lines
drivers/gpu/nova-core/gsp/fw.rs +6
-0+ +impl From <GspArgumentsCached >for bindings::GSP_ARGUMENTS_CACHED { +fn from (value:GspArgumentsCached ) ->Self { + value.0 + } +}This
Fromimpl seems unneeded?
Indeed. I don't remember why I added it, must have been needed in an earlier version. I'm suprised clippy doesn't complain.
PATCH v5 9/14 gpu: nova-core: Add bindings and accessors for GspSystemInfo
In reply to Alistair PoppleAdds bindings and an in-place initialiser for the GspSystemInfo struct.
---
Changes for v5:
- Derive Zeroable trait
Changes for v4:
- Use init! macros
Changes for v3:
- New for v3
--- drivers/gpu/nova-core/gsp/fw.rs | 1 + drivers/gpu/nova-core/gsp/fw/commands.rs | 51 +++++++ .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ++++++++++++++++++ 3 files changed, 184 insertions(+) create mode 100644 drivers/gpu/nova-core/gsp/fw/commands.rs
drivers/gpu/nova-core/gsp/fw.rs +1 -0
@@ -1,5 +1,6 @@// SPDX-License-Identifier: GPL-2.0 +pub (crate )mod commands;mod r570_144;// Alias to avoid repeating the version number with every use.
drivers/gpu/nova-core/gsp/fw/commands.rs new file +51 -0
51 lines not on this page · raw message
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +132 -0
138 lines not on this page · raw message
Signature · 1 line
2.50.1
Alexandre Courbot
maintainerRe: [PATCH v5 09/14] gpu: nova-core: Add bindings and accessors for GspSystemInfo
Quoted 3 lines · Alistair Popple
Adds bindings and an in-place initialiser for the GspSystemInfo struct.
Signed-off-by Alistair Popple <[email protected]>
I am getting a new unused warnings on this patch - we should either annotate or merge.
Re: [PATCH v5 09/14] gpu: nova-core: Add bindings and accessors for GspSystemInfo
On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 7 lines · Alistair Popple
Quoted 3 lines
Adds bindings and an in-place initialiser for the GspSystemInfo struct.
Signed-off-by Alistair Popple <[email protected]>I am getting a new unused warnings on this patch - we should either annotate or merge.
Yeah there are quite a few unused warnings. They're a pain to deal with during patch revisions and rebasing/rewriting so I was waiting until all the comments settled down before dealing with them as it's pretty tedious work adding lint statements in one patch and removing them in the next.
As you say the easiest solution is just to merge patches, but that just promotes larger patches. I wish we didn't have to make that trade off, but I guess if we want clippy bisection to be clean we have to.
I will probably just merge all the bindings into the patches that use them.
Alexandre Courbot
maintainerRe: [PATCH v5 09/14] gpu: nova-core: Add bindings and accessors for GspSystemInfo
Quoted 14 lines · Alistair Popple
On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 7 lines · Alistair Popple
Quoted 3 lines
Adds bindings and an in-place initialiser for the GspSystemInfo struct.
Signed-off-by Alistair Popple <[email protected]>I am getting a new unused warnings on this patch - we should either annotate or merge.
Yeah there are quite a few unused warnings. They're a pain to deal with during patch revisions and rebasing/rewriting so I was waiting until all the comments settled down before dealing with them as it's pretty tedious work adding lint statements in one patch and removing them in the next.
Quoted 5 lines
As you say the easiest solution is just to merge patches, but that just promotes larger patches. I wish we didn't have to make that trade off, but I guess if we want clippy bisection to be clean we have to.
I will probably just merge all the bindings into the patches that use them.
Miguel's suggestion of adding a temporary, file-global expect(unused) directive would also work very well if you want to preserve the patch sequence.
Re: [PATCH v5 09/14] gpu: nova-core: Add bindings and accessors for GspSystemInfo
On 2025-10-17 at 12:41 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 19 lines · Alistair Popple
Quoted 18 lines
On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...
Quoted 7 lines · Alistair Popple
Quoted 3 lines
Adds bindings and an in-place initialiser for the GspSystemInfo struct.
Signed-off-by Alistair Popple <[email protected]>I am getting a new unused warnings on this patch - we should either annotate or merge.
Yeah there are quite a few unused warnings. They're a pain to deal with during patch revisions and rebasing/rewriting so I was waiting until all the comments settled down before dealing with them as it's pretty tedious work adding lint statements in one patch and removing them in the next.
As you say the easiest solution is just to merge patches, but that just promotes larger patches. I wish we didn't have to make that trade off, but I guess if we want clippy bisection to be clean we have to.
Quoted 5 lines
Quoted 1 line
I will probably just merge all the bindings into the patches that use them.
Miguel's suggestion of adding a temporary, file-global
expect(unused)directive would also work very well if you want to preserve the patch sequence.
Yeah, I quite liked that suggestion as a way of alleviating some of the tedium. But given the bindings are pretty self-contained anyway lets just squash them.
PATCH v5 10/14 gpu: nova-core: Add bindings for the GSP RM registry tables
In reply to Alistair PoppleAdds bindings and constructors for PACKED_REGISTRY_TABLE and PACKED_REGISTRY_ENTRY structures.
---
Changes for v5:
- Derive Zeroable trait
Changes for v4:
- Use init! macros
- Add comments around only supporting DWORD entry types
Changes for v3:
- New for v3
--- drivers/gpu/nova-core/gsp/fw/commands.rs | 49 +++++++++++++++++++ .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 16 ++++++ 2 files changed, 65 insertions(+)
drivers/gpu/nova-core/gsp/fw/commands.rs +49 -0
@@ -49,3 +49,52 @@ unsafe impl AsBytes for GspSystemInfo {}// SAFETY: These structs don't meet the no-padding requirements of FromBytes but // that is not a problem because they are not used outside the kernel. unsafe impl FromBytes for GspSystemInfo {} + +#[repr(transparent)] +pub (crate )struct PackedRegistryEntry (bindings::PACKED_REGISTRY_ENTRY ); + +impl PackedRegistryEntry { +pub (crate )fn new (offset:u32 , value:u32 ) ->Self { +Self ({ + bindings::PACKED_REGISTRY_ENTRY { + nameOffset: offset, + +// We only support DWORD types for now. Support for other types +// will come later if required. + type_: bindings::REGISTRY_TABLE_ENTRY_TYPE_DWORD as u8 , + __bindgen_padding_0:Default ::default (), + data: value, + length:0 , + } + }) + } +} + +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for PackedRegistryEntry {} + +#[repr(transparent)] +pub (crate )struct PackedRegistryTable { + inner: bindings::PACKED_REGISTRY_TABLE , +} + +impl PackedRegistryTable { +#[allow(non_snake_case)] +pub (crate )fn init (num_entries:u32 , size:u32 ) ->impl Init <Self > { +type InnerPackedRegistryTable = bindings::PACKED_REGISTRY_TABLE ; +let init_inner =init! (InnerPackedRegistryTable { + numEntries: num_entries, + size, + entries:Default ::default () + }); + +init! (PackedRegistryTable { inner <- init_inner }) + } +} + +// SAFETY: Padding is explicit and will not contain uninitialized data. +unsafe impl AsBytes for PackedRegistryTable {} + +// SAFETY: This struct only contains integer types for which all bit patterns +// are valid. +unsafe impl FromBytes for PackedRegistryTable {}
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +16 -0
@@ -649,6 +649,22 @@ pub struct LibosMemoryRegionInitArgument { pub __bindgen_padding_0: [u8; 6usize], }#[repr( C )] +#[derive( Debug , Default , Copy , Clone )] +pub struct PACKED_REGISTRY_ENTRY { +pub nameOffset:u32_ , +pub type_:u8_ , +pub __bindgen_padding_0: [u8 ;3usize ], +pub data:u32_ , +pub length:u32_ , +} +#[repr( C )] +#[derive( Debug , Default )] +pub struct PACKED_REGISTRY_TABLE { +pub size:u32_ , +pub numEntries:u32_ , +pub entries:__IncompleteArrayField <PACKED_REGISTRY_ENTRY >, +} +#[repr( C )] #[derive( Debug , Default , Copy , Clone , Zeroable )] pub struct msgqTxHeader {pub version:u32_ ,
Signature · 1 line
2.50.1
PATCH v5 11/14 gpu: nova-core: gsp: Create RM registry and sysinfo commands
In reply to Alistair PoppleAdd the RM registry and system information commands that enable the host driver to configure GSP firmware parameters during initialization.
The RM registry is serialized into a packed format and sent via the command queue. For now only two parameters which are required to boot GSP are hardcoded. In future a kernel module parameter will be added to enable other parameters to be added.
Also add the system info command, which provides required hardware information to the GSP. These commands use the GSP command queue infrastructure to issue commands to the GSP which is read during GSP boot.
---
Changes for v4:
- Use init! macros
- Update to use send_gsp_command_with_payload() for the registry
- Add RMDevidCheckIgnore registry setting (thanks Timur)
Changes for v3:
- Use MsgFunction enum
- Rename GspCmdq to Cmdq
- Rename GspCommandToGsp to CommandToGsp
- Rename GspMessageFromGsp to MessageFromGsp
- Split bindings into separate patch
Changes for v2:
- Rebased on Alex's latest tree
--- drivers/gpu/nova-core/gsp.rs | 1 + drivers/gpu/nova-core/gsp/boot.rs | 6 +- drivers/gpu/nova-core/gsp/cmdq.rs | 2 - drivers/gpu/nova-core/gsp/commands.rs | 115 ++++++++++++++++++++++++++ drivers/gpu/nova-core/sbuffer.rs | 1 - 5 files changed, 121 insertions(+), 4 deletions(-) create mode 100644 drivers/gpu/nova-core/gsp/commands.rs
drivers/gpu/nova-core/gsp.rs +1 -0
@@ -22,6 +22,7 @@use fw::GspArgumentsCached ;pub (crate )mod cmdq; +pub (crate )mod commands;pub (crate )const GSP_PAGE_SHIFT :usize =12 ;pub (crate )const GSP_PAGE_SIZE :usize =1 <<GSP_PAGE_SHIFT ;
drivers/gpu/nova-core/gsp/boot.rs +5 -1
@@ -16,6 +16,7 @@ FIRMWARE_VERSION, };use crate ::gpu::Chipset ; +use crate ::gsp::commands::{build_registry, set_system_info};use crate ::gsp::GspFwWprMeta ;use crate ::regs;use crate ::vbios::Vbios ;
@@ -105,7 +106,7 @@ fn run_fwsec_frts(/// /// Upon return, the GSP is up and running, and its runtime object given as return value. pub (crate )fn boot ( - self:Pin <&mut Self >, +mut self:Pin <&mut Self >, pdev: &pci::Device <device::Bound >, bar: &Bar0 , chipset: Chipset,
@@ -139,6 +140,9 @@ pub(crate) fn boot(CoherentAllocation ::<GspFwWprMeta >::alloc_coherent (dev,1 ,GFP_KERNEL | __GFP_ZERO)?;dma_write! (wpr_meta[0 ] =GspFwWprMeta ::new (&gsp_fw, &fb_layout))?; +set_system_info (&mut self.cmdq, pdev, bar)?; +build_registry (&mut self.cmdq, bar)?; +Ok (()) } }
drivers/gpu/nova-core/gsp/cmdq.rs +0 -2
@@ -292,7 +292,6 @@ fn notify_gsp(bar: &Bar0) {NV_PGSP_QUEUE_HEAD ::default ().set_address (0 ).write (bar); } -#[expect(unused)] pub (crate )fn send_gsp_command <M ,E >(&mut self, bar: &Bar0 , init:impl Init <M ,E >) ->Result where M :CommandToGsp ,
@@ -345,7 +344,6 @@ struct FullCommand<M> {Ok (()) } -#[expect(unused)] pub (crate )fn send_gsp_command_with_payload <M ,E >( &mut self, bar: &Bar0,
drivers/gpu/nova-core/gsp/commands.rs new file +115 -0
115 lines not on this page · raw message
drivers/gpu/nova-core/sbuffer.rs +0 -1
@@ -186,7 +186,6 @@ fn get_slice_mut(&mut self, len: usize) -> Option<&'a mut [u8]> {/// Ideally we would implement `Write`, but it is not available in `core`. /// So mimic `std::io::Write::write_all`. -#[expect(unused)] pub (crate )fn write_all (&mut self,mut src: &[u8 ]) ->Result {while !src.is_empty () {match self.get_slice_mut (src.len ()) {
Signature · 1 line
2.50.1
PATCH v5 12/14 nova-core: falcon: Add support to check if RISC-V is active
In reply to Alistair PoppleFrom: Joel Fernandes <[email protected]>
Add definition for RISCV_CPUCTL register and use it in a new falcon API to check if the RISC-V core of a Falcon is active. It is required by the sequencer to know if the GSP's RISCV processor is active.
---
Changes for v4:
- Return bool instead of Result<bool> from is_riscv_active() as it can't fail (thanks Timur).
- Update register definitions to correct Falcon
- Switch register definition order
--- drivers/gpu/nova-core/falcon.rs | 9 +++++++++ drivers/gpu/nova-core/regs.rs | 7 ++++++- 2 files changed, 15 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/falcon.rs +9 -0
@@ -506,4 +506,13 @@ pub(crate) fn signature_reg_fuse_version( self.hal .signature_reg_fuse_version (self, bar, engine_id_mask, ucode_id) } + +/// Check if the RISC-V core is active. +/// +/// Returns `true` if the RISC-V core is active, `false` otherwise. +#[expect(unused)] +pub (crate )fn is_riscv_active (&self, bar: &Bar0 ) ->bool { +let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL ::read (bar, &E ::ID ); + cpuctl.active_stat () + } }
drivers/gpu/nova-core/regs.rs +6 -1
@@ -324,7 +324,12 @@ pub(crate) fn mem_scrubbing_done(self) -> bool {// PRISCV -register! (NV_PRISCV_RISCV_BCR_CTRL @PFalconBase [0x00001668 ] { +register! (NV_PRISCV_RISCV_CPUCTL @PFalcon2Base [0x00000388 ] { +0 :0 haltedas bool ; +7 :7 active_statas bool ; +}); + +register! (NV_PRISCV_RISCV_BCR_CTRL @PFalcon2Base [0x00000668 ] {0 :0 validas bool ;4 :4 core_selectas bool =>PeregrineCoreSelect ;8 :8 br_fetchas bool ;
Signature · 1 line
2.50.1
PATCH v5 13/14 nova-core: falcon: Add support to write firmware version
In reply to Alistair PoppleFrom: Joel Fernandes <[email protected]>
This will be needed by both the GSP boot code as well as GSP resume code in the sequencer.
---
Changes for v5:
- Make it infallible
--- drivers/gpu/nova-core/falcon.rs | 8 ++++++++ drivers/gpu/nova-core/regs.rs | 6 ++++++ 2 files changed, 14 insertions(+)
drivers/gpu/nova-core/falcon.rs +8 -0
@@ -515,4 +515,12 @@ pub(crate) fn is_riscv_active(&self, bar: &Bar0) -> bool {let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL ::read (bar, &E ::ID ); cpuctl.active_stat () } + +/// Write the application version to the OS register. +#[expect(dead_code)] +pub (crate )fn write_os_version (&self, bar: &Bar0 , app_version:u32 ) { + regs::NV_PFALCON_FALCON_OS ::default () + .set_value (app_version) + .write (bar, &E ::ID ); + } }
drivers/gpu/nova-core/regs.rs +6 -0
@@ -215,6 +215,12 @@ pub(crate) fn vga_workspace_addr(self) -> Option<u64> { 31:0 value as u32; }); +// Used to store version information about the firmware running +// on the Falcon processor. +register! (NV_PFALCON_FALCON_OS @PFalconBase [0x00000080 ] { +31 :0 valueas u32 ; +}); +register! (NV_PFALCON_FALCON_RM @PFalconBase [0x00000084 ] {31 :0 valueas u32 ; });
Signature · 1 line
2.50.1
PATCH v5 14/14 nova-core: gsp: Boot GSP
In reply to Alistair PoppleBoot the GSP to the RISC-V active state. Completing the boot requires running the CPU sequencer which will be added in a future commit.
---
Changes for v4:
- Switch wait_on to read_poll_timeout
Changes for v3:
- Fixed minor nit from John
- Added booter load error thanks to Timur's suggestion
Changes for v2:
- Rebased on Alex's latest tree
--- drivers/gpu/nova-core/falcon.rs | 2 - drivers/gpu/nova-core/firmware/riscv.rs | 3 +- drivers/gpu/nova-core/gsp/boot.rs | 63 ++++++++++++++++++++++++- 3 files changed, 63 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/falcon.rs +0 -2
@@ -510,14 +510,12 @@ pub(crate) fn signature_reg_fuse_version(/// Check if the RISC-V core is active. /// /// Returns `true` if the RISC-V core is active, `false` otherwise. -#[expect(unused)] pub (crate )fn is_riscv_active (&self, bar: &Bar0 ) ->bool {let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL ::read (bar, &E ::ID ); cpuctl.active_stat () }/// Write the application version to the OS register. -#[expect(dead_code)] pub (crate )fn write_os_version (&self, bar: &Bar0 , app_version:u32 ) { regs::NV_PFALCON_FALCON_OS ::default () .set_value (app_version)
drivers/gpu/nova-core/firmware/riscv.rs +1 -2
@@ -52,7 +52,6 @@ fn new(bin_fw: &BinFirmware<'_>) -> Result<Self> { }/// A parsed firmware for a RISC-V core, ready to be loaded and run. -#[expect(unused)] pub (crate )struct RiscvFirmware {/// Offset at which the code starts in the firmware image. pub (crate ) code_offset:u32 ,
@@ -61,7 +60,7 @@ pub(crate) struct RiscvFirmware {/// Offset at which the manifest starts in the firmware image. pub(crate) manifest_offset: u32,/// Application version. - app_version: u32, + pub app_version: u32,/// Device-mapped firmware image. pub ucode: DmaObject, }
drivers/gpu/nova-core/gsp/boot.rs +62 -1
83 lines not on this page · raw message
Signature · 1 line
2.50.1