Skip to contentSkip to conversation

gpu: nova-core: Boot GSP to RISC-V active

PATCH Superseded Alistair Popple 4 participants 46 messages Oct 13, 2025 – Oct 20, 2025

Subsystems (6)
DRM drivers Kbuild RISC-V Rust +2 more subsystems

v5 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v5 cover gpu: nova-core: Boot GSP to RISC-V active

Changes since v4:

The main change for this revision is to derive the Zeroable trait for most of our bindings. The rest of the changes just address relatively minor review comments made for v4 of the series.

This series is still based on a merge of drm-rust-next and drivers-core-testing made by Alex. A complete copy of the tree with these patches applied will be available at
https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream-v5

Changes since v3:

The main change for v4 is to switch to using the init! macros to ensure all fields on in-place initialised structs get initialised. This will require our bindings to derive the Zeroable trait, however for now I have left this as a TODO with manual implementations for each trait. That is because rebasing the binding changes is a bit of a pain, so I want to give reviewers a change to see if deriving Zeroable for all bindings makes sense or not.

Other changes include addressing most of the outstanding TODOs left in v3 and addressing review comments from v2 and v3. In particular some of the comments by Timur that had not been picked up.

Changes since v2:

The main change since v2 has been to make all firmware bindings completely opaque. It has been made clear this is a pre-requisite for this series to progress upstream as it should make supporting different firmware versions easier in future.

Overall the extra constructors and accessors add a couple of hundred lines of code and a few extra unsafe statements.

Other changes include addressing a bunch of other comments - see the individual patches for further details. There are also still some outstanding comments and TODO's to address which I have not gotten to yet - these will be done in the next version of this series.

Changes since v1:

- Based on feed back from Alex the GSP command queue logic was reworked extensively. This involved creating a new data struct (DmaGspMem) to manage the shared memory areas between CPU and GSP.

- This data structure helps ensure the safety constraints are meet when the CPU is reading/writing the shared memory queues.

- Several other minor comments were addressed, as noted in the individual patches.

This series builds on top of Alex's series[1], most of which has been merged into drm-rust-next, to continue initialising the GSP into a state where it becomes active and it starts communicating with the host. A tree including these patches with the prerequisite patches is available at [2].

It includes patches to initialise several important data structures required to boot the GSP. The biggest change is the implementation of the command/message circular queue used to establish communication between GSP and host in patch 6. Admittedly this patch is rather large - if necessary it could be split into send and receive patches if people prefer.

This is required to configure and boot the GSP. However this series does not get the GSP to a fully active state. Instead it gets it to a state where the GSP sends a message to the host with a sequence of instructions which need running to get to the active state. A subsequent series will implement processing of this message and allow the GSP to get to the fully active state.

A full tree including the prerequisites for this patch series is available at https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream.

[1] - https://lore.kernel.org/rust-for-linux/[email protected]/
[2] - https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream-v2

To: [email protected]
To: [email protected]
To: Danilo Krummrich <[email protected]>
To: Alexandre Courbot <[email protected]>
Cc: Miguel Ojeda <[email protected]>
Cc: Alex Gaynor <[email protected]>
Cc: Boqun Feng <[email protected]>
Cc: Gary Guo <[email protected]>
Cc: Björn Roy Baron <[email protected]>
Cc: Benno Lossin <[email protected]>
Cc: Andreas Hindborg <[email protected]>
Cc: Alice Ryhl <[email protected]>
Cc: Trevor Gross <[email protected]>
Cc: David Airlie <[email protected]>
Cc: Simona Vetter <[email protected]>
Cc: Maarten Lankhorst <[email protected]>
Cc: Maxime Ripard <[email protected]>
Cc: Thomas Zimmermann <[email protected]>
Cc: John Hubbard <[email protected]>
Cc: Joel Fernandes <[email protected]>
Cc: Timur Tabi <[email protected]>
Cc: [email protected]
Cc: [email protected]

Alistair Popple (11):
gpu: nova-core: Set correct DMA mask
gpu: nova-core: Create initial Gsp
gpu: nova-core: gsp: Create wpr metadata
gpu: nova-core: Add zeroable trait to bindings
gpu: nova-core: Add GSP command queue bindings
gpu: nova-core: gsp: Add GSP command queue handling
gpu: nova-core: gsp: Create rmargs
gpu: nova-core: Add bindings and accessors for GspSystemInfo
gpu: nova-core: Add bindings for the GSP RM registry tables
gpu: nova-core: gsp: Create RM registry and sysinfo commands
nova-core: gsp: Boot GSP

Joel Fernandes (3):
gpu: nova-core: Add a slice-buffer (sbuffer) datastructure
nova-core: falcon: Add support to check if RISC-V is active
nova-core: falcon: Add support to write firmware version

 drivers/gpu/nova-core/driver.rs               |  16 +
 drivers/gpu/nova-core/falcon.rs               |  15 +
 drivers/gpu/nova-core/fb.rs                   |   1 -
 drivers/gpu/nova-core/firmware/gsp.rs         |   3 +-
 drivers/gpu/nova-core/firmware/riscv.rs       |   9 +-
 drivers/gpu/nova-core/gpu.rs                  |   2 +-
 drivers/gpu/nova-core/gsp.rs                  | 134 +++-
 drivers/gpu/nova-core/gsp/boot.rs             |  76 ++-
 drivers/gpu/nova-core/gsp/cmdq.rs             | 509 +++++++++++++++
 drivers/gpu/nova-core/gsp/commands.rs         | 115 ++++
 drivers/gpu/nova-core/gsp/fw.rs               | 455 +++++++++++++-
 drivers/gpu/nova-core/gsp/fw/commands.rs      | 100 +++
 drivers/gpu/nova-core/gsp/fw/r570_144.rs      |   1 +
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 592 +++++++++++++++++-
 drivers/gpu/nova-core/nova_core.rs            |   1 +
 drivers/gpu/nova-core/regs.rs                 |  17 +-
 drivers/gpu/nova-core/sbuffer.rs              | 215 +++++++
 scripts/Makefile.build                        |   2 +-
 18 files changed, 2231 insertions(+), 32 deletions(-)
 create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs
 create mode 100644 drivers/gpu/nova-core/gsp/commands.rs
 create mode 100644 drivers/gpu/nova-core/gsp/fw/commands.rs
 create mode 100644 drivers/gpu/nova-core/sbuffer.rs
Signature · 1 line
2.50.1
#1 lore raw

PATCH v5 1/14 gpu: nova-core: Set correct DMA mask

Set the correct DMA mask. Without this DMA will fail on some setups.

---

Changes for v5:
- Update SAFETY comment for dma_set_mask_and_coherent()
- Add TODO for using different masks when we support more GPU models

Changes for v4:
- Use a const (GPU_DMA_BITS) instead of a magic number

Changes for v2:
- Update DMA mask to correct value for Ampere/Turing (47 bits)

---
 drivers/gpu/nova-core/driver.rs | 16 ++++++++++++++++
 1 file changed, 16 insertions(+)
drivers/gpu/nova-core/driver.rs +16 -0
index edc72052e27a..2407d0ab15e2 100644
@@ -3,6 +3,8 @@
 use kernel::{
     auxiliary, c_str,
     device::Core,
+    dma::Device,
+    dma::DmaMask,
     pci,
     pci::{Class, ClassMask, Vendor},
     prelude::*,
@@ -20,6 +22,15 @@ pub(crate) struct NovaCore {
 }
 
 const BAR0_SIZE: usize = SZ_16M;
+
+// For now we only support Ampere which can use up to 47-bit DMA addresses.
+//
+// TODO: Add an abstraction for this to support newer GPUs which may support
+// larger DMA addresses. Limiting these GPUs to smaller address widths won't
+// have any adverse affects, unless installed on systems which require larger
+// DMA addresses. These systems should be quite rare.
+const GPU_DMA_BITS: u32 = 47;
+
 pub(crate) type Bar0 = pci::Bar<BAR0_SIZE>;
 
 kernel::pci_device_table!(
@@ -57,6 +68,11 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self
         pdev.enable_device_mem()?;
         pdev.set_master();
 
+        // SAFETY: No concurrent DMA allocations or mappings can be made because
+        // the device is still being probed and therefore isn't being used by
+        // other threads of execution.
+        unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<GPU_DMA_BITS>())? };
+
         let devres_bar = Arc::pin_init(
             pdev.iomap_region_sized::<BAR0_SIZE>(0, c_str!("nova-core/bar0")),
             GFP_KERNEL,
Signature · 1 line
2.50.1
#2 lore raw

PATCH v5 2/14 gpu: nova-core: Create initial Gsp

In reply to Alistair Popple

The GSP requires several areas of memory to operate. Each of these have their own simple embedded page tables. Set these up and map them for DMA to/from GSP using CoherentAllocation's. Return the DMA handle describing where each of these regions are for future use when booting GSP.

---

Changes for v5:
- Move GSP_HEAP_ALIGNMENT to gsp/fw.rs and add a comment.
- Create a LogBuffer type.
- Use checked_add to ensure PTE values don't overflow.
- Added some type documentation (shamelessly stolen from Nouveau)

Change for v3:
- Clean up the PTE array creation, with much thanks to Alex for doing most it (please let me know if I should put you as co-developer!)

Changes for v2:
- Renamed GspMemOjbects to Gsp as that is what they are
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gpu.rs                  |   2 +-
 drivers/gpu/nova-core/gsp.rs                  | 106 ++++++++++++++++--
 drivers/gpu/nova-core/gsp/fw.rs               |  64 ++++++++++-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  19 ++++
 4 files changed, 179 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/gpu.rs +1 -1
index ea124d1912e7..c1396775e9b6 100644
@@ -197,7 +197,7 @@ pub(crate) fn new<'a>(
 
             sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset, bar, true)?,
 
-            gsp <- Gsp::new(),
+            gsp <- Gsp::new(pdev)?,
 
             _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
 
drivers/gpu/nova-core/gsp.rs +99 -7

124 lines not on this page · raw message

drivers/gpu/nova-core/gsp/fw.rs +60 -4

92 lines not on this page · raw message

drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +19 -0
index 0407000cca22..6a14cc324391 100644
@@ -124,3 +124,22 @@ fn default() -> Self {
         }
     }
 }
+pub type LibosAddress = u64_;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_NONE: LibosMemoryRegionKind = 0;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS: LibosMemoryRegionKind = 1;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_RADIX3: LibosMemoryRegionKind = 2;
+pub type LibosMemoryRegionKind = ffi::c_uint;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_NONE: LibosMemoryRegionLoc = 0;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM: LibosMemoryRegionLoc = 1;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_FB: LibosMemoryRegionLoc = 2;
+pub type LibosMemoryRegionLoc = ffi::c_uint;
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone)]
+pub struct LibosMemoryRegionInitArgument {
+    pub id8: LibosAddress,
+    pub pa: LibosAddress,
+    pub size: LibosAddress,
+    pub kind: u8_,
+    pub loc: u8_,
+    pub __bindgen_padding_0: [u8; 6usize],
+}
Signature · 1 line
2.50.1
#3 lore raw

Re: [PATCH v5 02/14] gpu: nova-core: Create initial Gsp

Quoted 76 lines · Alistair Popple

The GSP requires several areas of memory to operate. Each of these have their own simple embedded page tables. Set these up and map them for DMA to/from GSP using CoherentAllocation's. Return the DMA handle describing where each of these regions are for future use when booting GSP.

---

Changes for v5:
- Move GSP_HEAP_ALIGNMENT to gsp/fw.rs and add a comment.
- Create a LogBuffer type.
- Use checked_add to ensure PTE values don't overflow.
- Added some type documentation (shamelessly stolen from Nouveau)

Change for v3:
- Clean up the PTE array creation, with much thanks to Alex for doing most it (please let me know if I should put you as co-developer!)

Changes for v2:
- Renamed GspMemOjbects to Gsp as that is what they are
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gpu.rs                  |   2 +-
 drivers/gpu/nova-core/gsp.rs                  | 106 ++++++++++++++++--
 drivers/gpu/nova-core/gsp/fw.rs               |  64 ++++++++++-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  19 ++++
 4 files changed, 179 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/gpu.rs +1 -1
index ea124d1912e7..c1396775e9b6 100644
@@ -197,7 +197,7 @@ pub(crate) fn new<'a>(
 
             sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset, bar, true)?,
 
-            gsp <- Gsp::new(),
+            gsp <- Gsp::new(pdev)?,
 
             _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
 
drivers/gpu/nova-core/gsp.rs +11 -4
index 221281da1a45..f1727173bd42 100644
@@ -2,25 +2,117 @@
 
 mod boot;
 
+use kernel::device;
+use kernel::dma::CoherentAllocation;
+use kernel::dma::DmaAddress;
+use kernel::dma_write;
+use kernel::pci;
 use kernel::prelude::*;
-use kernel::ptr::Alignment;
+use kernel::transmute::AsBytes;
 
 pub(crate) use fw::{GspFwWprMeta, LibosParams};
 
 mod fw;
 
+use fw::LibosMemoryRegionInitArgument;
+
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
 pub(crate) const GSP_PAGE_SIZE: usize = 1 << GSP_PAGE_SHIFT;
-pub(crate) const GSP_HEAP_ALIGNMENT: Alignment = Alignment::new::<{ 1 << 20 }>();
+
+/// Number of GSP pages to use in a RM log buffer.
+const RM_LOG_BUFFER_NUM_PAGES: usize = 0x10;
 
 /// GSP runtime data.
-///
-/// This is an empty pinned placeholder for now.
 #[pin_data]

drivers/gpu/nova-core/gsp.rs line 35

Quoted 10 lines
drivers/gpu/nova-core/gsp.rs +9 -1
-pub(crate) struct Gsp {}
+pub(crate) struct Gsp {
+    pub(crate) libos: CoherentAllocation<LibosMemoryRegionInitArgument>,
+    loginit: LogBuffer,
+    logintr: LogBuffer,
+    logrm: LogBuffer,
+}
+
+#[repr(C)]
+struct PteArray<const NUM_ENTRIES: usize>([u64; NUM_ENTRIES]);

I'd just document this structure a bit as it is not obvious what it does from the name alone.

Quoted 21 lines
drivers/gpu/nova-core/gsp.rs +21 -0
+
+/// SAFETY: arrays of `u64` implement `AsBytes` and we are but a wrapper around it.
+unsafe impl<const NUM_ENTRIES: usize> AsBytes for PteArray<NUM_ENTRIES> {}
+
+impl<const NUM_PAGES: usize> PteArray<NUM_PAGES> {
+    fn new(handle: DmaAddress) -> Result<Self> {
+        let mut ptes = [0u64; NUM_PAGES];
+        for (i, pte) in ptes.iter_mut().enumerate() {
+            *pte = handle
+                .checked_add((i as u64) << GSP_PAGE_SHIFT)
+                .ok_or(EOVERFLOW)?;
+        }
+
+        Ok(Self(ptes))
+    }
+}
+
+/// The logging buffers are byte queues that contain encoded printf-like
+/// messages from GSP-RM.  They need to be decoded by a special application
+/// that can parse the buffers.
+///

drivers/gpu/nova-core/gsp.rs line 66

Quoted 10 lines
drivers/gpu/nova-core/gsp.rs +10 -0
+/// The 'loginit' buffer contains logs from early GSP-RM init and
+/// exception dumps.  The 'logrm' buffer contains the subsequent logs. Both are
+/// written to directly by GSP-RM and can be any multiple of GSP_PAGE_SIZE.
+///
+/// The physical address map for the log buffer is stored in the buffer
+/// itself, starting with offset 1. Offset 0 contains the "put" pointer (pp).
+/// Initially, pp is equal to 0. If the buffer has valid logging data in it,
+/// then pp points to index into the buffer where the next logging entry will
+/// be written. Therefore, the logging data is valid if:
+///   1 <= pp < sizeof(buffer)/sizeof(u64)

Maybe we should mention what happens to the address map, namely that it gets overwritten by the buffer data and is only used for the initial setup.

Quoted 35 lines
drivers/gpu/nova-core/gsp.rs +31 -2
+struct LogBuffer(CoherentAllocation<u8>);
+
+impl LogBuffer {
+    fn new(dev: &device::Device<device::Bound>) -> Result<Self> {
+        const NUM_PAGES: usize = RM_LOG_BUFFER_NUM_PAGES;
+
+        let mut obj = Self(CoherentAllocation::<u8>::alloc_coherent(
+            dev,
+            NUM_PAGES * GSP_PAGE_SIZE,
+            GFP_KERNEL | __GFP_ZERO,
+        )?);
+        let ptes = PteArray::<NUM_PAGES>::new(obj.0.dma_handle())?;
+
+        // SAFETY: `obj` has just been created and we are its sole user.
+        unsafe {
+            // Copy the self-mapping PTE at the expected location.
+            obj.0
+                .as_slice_mut(size_of::<u64>(), size_of_val(&ptes))?
+                .copy_from_slice(ptes.as_bytes())
+        };
+
+        Ok(obj)
+    }
+}
 
 impl Gsp {
-    pub(crate) fn new() -> impl PinInit<Self> {
-        pin_init!(Self {})
+    pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self, Error>> {
+        let dev = pdev.as_ref();
+        let libos = CoherentAllocation::<LibosMemoryRegionInitArgument>::alloc_coherent(
+            dev,
+            GSP_PAGE_SIZE / size_of::<LibosMemoryRegionInitArgument>(),
+            GFP_KERNEL | __GFP_ZERO,
+        )?;

drivers/gpu/nova-core/gsp.rs line 109

Quoted 10 lines
drivers/gpu/nova-core/gsp.rs +10 -0
+
+        // Initialise the logging structures. The OpenRM equivalents are in:
+        // _kgspInitLibosLoggingStructures (allocates memory for buffers)
+        // kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array)
+        let loginit = LogBuffer::new(dev)?;
+        dma_write!(libos[0] = LibosMemoryRegionInitArgument::new("LOGINIT", &loginit.0)?)?;
+        let logintr = LogBuffer::new(dev)?;
+        dma_write!(libos[1] = LibosMemoryRegionInitArgument::new("LOGINTR", &logintr.0)?)?;
+        let logrm = LogBuffer::new(dev)?;
+        dma_write!(libos[2] = LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0)?)?;

Let's maybe add a space before each "let" statement.

Quoted 95 lines
drivers/gpu/nova-core/gsp.rs +7 -0
+
+        Ok(try_pin_init!(Self {
+            libos,
+            loginit,
+            logintr,
+            logrm,
+        }))
     }
 }
drivers/gpu/nova-core/gsp/fw.rs +46 -4
index 181baa401770..c3bececc29cd 100644
@@ -7,15 +7,20 @@
 
 use core::ops::Range;
 
-use kernel::ptr::Alignable;
+use kernel::dma::CoherentAllocation;
+use kernel::prelude::*;
+use kernel::ptr::{Alignable, Alignment};
 use kernel::sizes::SZ_1M;
+use kernel::transmute::{AsBytes, FromBytes};
 
 use crate::gpu::Chipset;
-use crate::gsp;
 
 /// Dummy type to group methods related to heap parameters for running the GSP firmware.
 pub(crate) struct GspFwHeapParams(());
 
+/// Minimum required alignment for the GSP heap.
+const GSP_HEAP_ALIGNMENT: Alignment = Alignment::new::<{ 1 << 20 }>();
+
 impl GspFwHeapParams {
     /// Returns the amount of GSP-RM heap memory used during GSP-RM boot and initialization (up to
     /// and including the first client subdevice allocation).
@@ -29,7 +34,7 @@ fn base_rm_size(_chipset: Chipset) -> u64 {
     /// Returns the amount of heap memory required to support a single channel allocation.
     fn client_alloc_size() -> u64 {
         u64::from(bindings::GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE)
-            .align_up(gsp::GSP_HEAP_ALIGNMENT)
+            .align_up(GSP_HEAP_ALIGNMENT)
             .unwrap_or(u64::MAX)
     }
 
@@ -40,7 +45,7 @@ fn management_overhead(fb_size: u64) -> u64 {
 
         u64::from(bindings::GSP_FW_HEAP_PARAM_SIZE_PER_GB_FB)
             .saturating_mul(fb_size_gb)
-            .align_up(gsp::GSP_HEAP_ALIGNMENT)
+            .align_up(GSP_HEAP_ALIGNMENT)
             .unwrap_or(u64::MAX)
     }
 }
@@ -99,3 +104,54 @@ pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 {
 /// addresses of the GSP bootloader and firmware.
 #[repr(transparent)]
 pub(crate) struct GspFwWprMeta(bindings::GspFwWprMeta);
+
+/// Struct containing the arguments required to pass a memory buffer to the GSP
+/// for use during initialisation.
+///
+/// The GSP only understands 4K pages (GSP_PAGE_SIZE), so even if the kernel is
+/// configured for a larger page size (e.g. 64K pages), we need to give
+/// the GSP an array of 4K pages. Since we only create physically contiguous
+/// buffers the math to calculate the addresses is simple.
+///
+/// The buffers must be a multiple of GSP_PAGE_SIZE.  GSP-RM also currently
+/// ignores the @kind field for LOGINIT, LOGINTR, and LOGRM, but expects the
+/// buffers to be physically contiguous anyway.
+///
+/// The memory allocated for the arguments must remain until the GSP sends the
+/// init_done RPC.
+#[repr(transparent)]
+pub(crate) struct LibosMemoryRegionInitArgument(bindings::LibosMemoryRegionInitArgument);
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for LibosMemoryRegionInitArgument {}
+
+// SAFETY: This struct only contains integer types for which all bit patterns
+// are valid.
+unsafe impl FromBytes for LibosMemoryRegionInitArgument {}
+
+impl LibosMemoryRegionInitArgument {
+    pub(crate) fn new<A: AsBytes + FromBytes>(
+        name: &'static str,
+        obj: &CoherentAllocation<A>,
+    ) -> Result<Self> {
+        /// Generates the `ID8` identifier required for some GSP objects.
+        fn id8(name: &str) -> u64 {
+            let mut bytes = [0u8; core::mem::size_of::<u64>()];
+
+            for (c, b) in name.bytes().rev().zip(&mut bytes) {
+                *b = c;
+            }

drivers/gpu/nova-core/gsp/fw.rs pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 { · line 153

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+
+            u64::from_ne_bytes(bytes)
+        }
+
+        Ok(Self(bindings::LibosMemoryRegionInitArgument {
+            id8: id8(name),
+            pa: obj.dma_handle(),
+            size: obj.size() as u64,
+            kind: bindings::LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS.try_into()?,
+            loc: bindings::LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM.try_into()?,

The unneeded runtime check is a bit unfortunate, and its removal would allow us to make this method non-fallible, but I cannot find a good alternative that also doesn't clutter the code. Can't wait for const traits methods! :)

#4 lore raw

Re: [PATCH v5 02/14] gpu: nova-core: Create initial Gsp

On 2025-10-16 at 17:22 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 82 lines · Alistair Popple
Quoted 81 lines

The GSP requires several areas of memory to operate. Each of these have their own simple embedded page tables. Set these up and map them for DMA to/from GSP using CoherentAllocation's. Return the DMA handle describing where each of these regions are for future use when booting GSP.

---

Changes for v5:
- Move GSP_HEAP_ALIGNMENT to gsp/fw.rs and add a comment.
- Create a LogBuffer type.
- Use checked_add to ensure PTE values don't overflow.
- Added some type documentation (shamelessly stolen from Nouveau)

Change for v3:
- Clean up the PTE array creation, with much thanks to Alex for doing most it (please let me know if I should put you as co-developer!)

Changes for v2:
- Renamed GspMemOjbects to Gsp as that is what they are
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gpu.rs                  |   2 +-
 drivers/gpu/nova-core/gsp.rs                  | 106 ++++++++++++++++--
 drivers/gpu/nova-core/gsp/fw.rs               |  64 ++++++++++-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  19 ++++
 4 files changed, 179 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/gpu.rs +1 -1
index ea124d1912e7..c1396775e9b6 100644
@@ -197,7 +197,7 @@ pub(crate) fn new<'a>(
 
             sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset, bar, true)?,
 
-            gsp <- Gsp::new(),
+            gsp <- Gsp::new(pdev)?,
 
             _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
 
drivers/gpu/nova-core/gsp.rs +15 -5
index 221281da1a45..f1727173bd42 100644
@@ -2,25 +2,117 @@
 
 mod boot;
 
+use kernel::device;
+use kernel::dma::CoherentAllocation;
+use kernel::dma::DmaAddress;
+use kernel::dma_write;
+use kernel::pci;
 use kernel::prelude::*;
-use kernel::ptr::Alignment;
+use kernel::transmute::AsBytes;
 
 pub(crate) use fw::{GspFwWprMeta, LibosParams};
 
 mod fw;
 
+use fw::LibosMemoryRegionInitArgument;
+
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
 pub(crate) const GSP_PAGE_SIZE: usize = 1 << GSP_PAGE_SHIFT;
-pub(crate) const GSP_HEAP_ALIGNMENT: Alignment = Alignment::new::<{ 1 << 20 }>();
+
+/// Number of GSP pages to use in a RM log buffer.
+const RM_LOG_BUFFER_NUM_PAGES: usize = 0x10;
 
 /// GSP runtime data.
-///
-/// This is an empty pinned placeholder for now.
 #[pin_data]
-pub(crate) struct Gsp {}
+pub(crate) struct Gsp {
+    pub(crate) libos: CoherentAllocation<LibosMemoryRegionInitArgument>,
+    loginit: LogBuffer,
+    logintr: LogBuffer,
Quoted 8 lines

drivers/gpu/nova-core/gsp.rs line 35

Quoted 5 lines
drivers/gpu/nova-core/gsp.rs +5 -0
+    logrm: LogBuffer,
+}
+
+#[repr(C)]
+struct PteArray<const NUM_ENTRIES: usize>([u64; NUM_ENTRIES]);

I'd just document this structure a bit as it is not obvious what it does from the name alone.

Ok. As discussed I'm going to put your Co-developed-by on v6 and leave this one to you given I think you're more familiar with it :-)

Quoted 27 lines
Quoted 27 lines
drivers/gpu/nova-core/gsp.rs +27 -0
+
+/// SAFETY: arrays of `u64` implement `AsBytes` and we are but a wrapper around it.
+unsafe impl<const NUM_ENTRIES: usize> AsBytes for PteArray<NUM_ENTRIES> {}
+
+impl<const NUM_PAGES: usize> PteArray<NUM_PAGES> {
+    fn new(handle: DmaAddress) -> Result<Self> {
+        let mut ptes = [0u64; NUM_PAGES];
+        for (i, pte) in ptes.iter_mut().enumerate() {
+            *pte = handle
+                .checked_add((i as u64) << GSP_PAGE_SHIFT)
+                .ok_or(EOVERFLOW)?;
+        }
+
+        Ok(Self(ptes))
+    }
+}
+
+/// The logging buffers are byte queues that contain encoded printf-like
+/// messages from GSP-RM.  They need to be decoded by a special application
+/// that can parse the buffers.
+///
+/// The 'loginit' buffer contains logs from early GSP-RM init and
+/// exception dumps.  The 'logrm' buffer contains the subsequent logs. Both are
+/// written to directly by GSP-RM and can be any multiple of GSP_PAGE_SIZE.
+///
+/// The physical address map for the log buffer is stored in the buffer
+/// itself, starting with offset 1. Offset 0 contains the "put" pointer (pp).
Quoted 8 lines

drivers/gpu/nova-core/gsp.rs line 66

Quoted 4 lines
drivers/gpu/nova-core/gsp.rs +4 -0
+/// Initially, pp is equal to 0. If the buffer has valid logging data in it,
+/// then pp points to index into the buffer where the next logging entry will
+/// be written. Therefore, the logging data is valid if:
+///   1 <= pp < sizeof(buffer)/sizeof(u64)

Maybe we should mention what happens to the address map, namely that it gets overwritten by the buffer data and is only used for the initial setup.

Ditto.

Quoted 39 lines
Quoted 39 lines
drivers/gpu/nova-core/gsp.rs +35 -2
+struct LogBuffer(CoherentAllocation<u8>);
+
+impl LogBuffer {
+    fn new(dev: &device::Device<device::Bound>) -> Result<Self> {
+        const NUM_PAGES: usize = RM_LOG_BUFFER_NUM_PAGES;
+
+        let mut obj = Self(CoherentAllocation::<u8>::alloc_coherent(
+            dev,
+            NUM_PAGES * GSP_PAGE_SIZE,
+            GFP_KERNEL | __GFP_ZERO,
+        )?);
+        let ptes = PteArray::<NUM_PAGES>::new(obj.0.dma_handle())?;
+
+        // SAFETY: `obj` has just been created and we are its sole user.
+        unsafe {
+            // Copy the self-mapping PTE at the expected location.
+            obj.0
+                .as_slice_mut(size_of::<u64>(), size_of_val(&ptes))?
+                .copy_from_slice(ptes.as_bytes())
+        };
+
+        Ok(obj)
+    }
+}
 
 impl Gsp {
-    pub(crate) fn new() -> impl PinInit<Self> {
-        pin_init!(Self {})
+    pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self, Error>> {
+        let dev = pdev.as_ref();
+        let libos = CoherentAllocation::<LibosMemoryRegionInitArgument>::alloc_coherent(
+            dev,
+            GSP_PAGE_SIZE / size_of::<LibosMemoryRegionInitArgument>(),
+            GFP_KERNEL | __GFP_ZERO,
+        )?;
+
+        // Initialise the logging structures. The OpenRM equivalents are in:
+        // _kgspInitLibosLoggingStructures (allocates memory for buffers)
+        // kgspSetupLibosInitArgs_IMPL (creates pLibosInitArgs[] array)
Quoted 8 lines

drivers/gpu/nova-core/gsp.rs line 109

Quoted 6 lines
drivers/gpu/nova-core/gsp.rs +6 -0
+        let loginit = LogBuffer::new(dev)?;
+        dma_write!(libos[0] = LibosMemoryRegionInitArgument::new("LOGINIT", &loginit.0)?)?;
+        let logintr = LogBuffer::new(dev)?;
+        dma_write!(libos[1] = LibosMemoryRegionInitArgument::new("LOGINTR", &logintr.0)?)?;
+        let logrm = LogBuffer::new(dev)?;
+        dma_write!(libos[2] = LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0)?)?;

Let's maybe add a space before each "let" statement.

Ok.

Quoted 110 lines
Quoted 95 lines
drivers/gpu/nova-core/gsp.rs +7 -0
+
+        Ok(try_pin_init!(Self {
+            libos,
+            loginit,
+            logintr,
+            logrm,
+        }))
     }
 }
drivers/gpu/nova-core/gsp/fw.rs +46 -4
index 181baa401770..c3bececc29cd 100644
@@ -7,15 +7,20 @@
 
 use core::ops::Range;
 
-use kernel::ptr::Alignable;
+use kernel::dma::CoherentAllocation;
+use kernel::prelude::*;
+use kernel::ptr::{Alignable, Alignment};
 use kernel::sizes::SZ_1M;
+use kernel::transmute::{AsBytes, FromBytes};
 
 use crate::gpu::Chipset;
-use crate::gsp;
 
 /// Dummy type to group methods related to heap parameters for running the GSP firmware.
 pub(crate) struct GspFwHeapParams(());
 
+/// Minimum required alignment for the GSP heap.
+const GSP_HEAP_ALIGNMENT: Alignment = Alignment::new::<{ 1 << 20 }>();
+
 impl GspFwHeapParams {
     /// Returns the amount of GSP-RM heap memory used during GSP-RM boot and initialization (up to
     /// and including the first client subdevice allocation).
@@ -29,7 +34,7 @@ fn base_rm_size(_chipset: Chipset) -> u64 {
     /// Returns the amount of heap memory required to support a single channel allocation.
     fn client_alloc_size() -> u64 {
         u64::from(bindings::GSP_FW_HEAP_PARAM_CLIENT_ALLOC_SIZE)
-            .align_up(gsp::GSP_HEAP_ALIGNMENT)
+            .align_up(GSP_HEAP_ALIGNMENT)
             .unwrap_or(u64::MAX)
     }
 
@@ -40,7 +45,7 @@ fn management_overhead(fb_size: u64) -> u64 {
 
         u64::from(bindings::GSP_FW_HEAP_PARAM_SIZE_PER_GB_FB)
             .saturating_mul(fb_size_gb)
-            .align_up(gsp::GSP_HEAP_ALIGNMENT)
+            .align_up(GSP_HEAP_ALIGNMENT)
             .unwrap_or(u64::MAX)
     }
 }
@@ -99,3 +104,54 @@ pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 {
 /// addresses of the GSP bootloader and firmware.
 #[repr(transparent)]
 pub(crate) struct GspFwWprMeta(bindings::GspFwWprMeta);
+
+/// Struct containing the arguments required to pass a memory buffer to the GSP
+/// for use during initialisation.
+///
+/// The GSP only understands 4K pages (GSP_PAGE_SIZE), so even if the kernel is
+/// configured for a larger page size (e.g. 64K pages), we need to give
+/// the GSP an array of 4K pages. Since we only create physically contiguous
+/// buffers the math to calculate the addresses is simple.
+///
+/// The buffers must be a multiple of GSP_PAGE_SIZE.  GSP-RM also currently
+/// ignores the @kind field for LOGINIT, LOGINTR, and LOGRM, but expects the
+/// buffers to be physically contiguous anyway.
+///
+/// The memory allocated for the arguments must remain until the GSP sends the
+/// init_done RPC.
+#[repr(transparent)]
+pub(crate) struct LibosMemoryRegionInitArgument(bindings::LibosMemoryRegionInitArgument);
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for LibosMemoryRegionInitArgument {}
+
+// SAFETY: This struct only contains integer types for which all bit patterns
+// are valid.
+unsafe impl FromBytes for LibosMemoryRegionInitArgument {}
+
+impl LibosMemoryRegionInitArgument {
+    pub(crate) fn new<A: AsBytes + FromBytes>(
+        name: &'static str,
+        obj: &CoherentAllocation<A>,
+    ) -> Result<Self> {
+        /// Generates the `ID8` identifier required for some GSP objects.
+        fn id8(name: &str) -> u64 {
+            let mut bytes = [0u8; core::mem::size_of::<u64>()];
+
+            for (c, b) in name.bytes().rev().zip(&mut bytes) {
+                *b = c;
+            }

drivers/gpu/nova-core/gsp/fw.rs pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 { · line 153

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+
+            u64::from_ne_bytes(bytes)
+        }
+
+        Ok(Self(bindings::LibosMemoryRegionInitArgument {
+            id8: id8(name),
+            pa: obj.dma_handle(),
+            size: obj.size() as u64,
+            kind: bindings::LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS.try_into()?,
+            loc: bindings::LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM.try_into()?,

The unneeded runtime check is a bit unfortunate, and its removal would allow us to make this method non-fallible, but I cannot find a good alternative that also doesn't clutter the code. Can't wait for const traits methods! :)

#5 lore raw

PATCH v5 3/14 gpu: nova-core: gsp: Create wpr metadata

In reply to Alistair Popple

The GSP requires some pieces of metadata to boot. These are passed in a struct which the GSP transfers via DMA. Create this struct and get a handle to it for future use when booting the GSP.

---

Changes for v5:
- Make member visibility match the struct visibility (thanks Danilo)

Changes for v3:
- Don't re-export WPR constants (thanks Alex)

Changes for v2:
- Rebased on Alex's latest version

---
 drivers/gpu/nova-core/fb.rs                   |  1 -
 drivers/gpu/nova-core/firmware/gsp.rs         |  3 +-
 drivers/gpu/nova-core/firmware/riscv.rs       |  6 +-
 drivers/gpu/nova-core/gsp.rs                  |  2 +
 drivers/gpu/nova-core/gsp/boot.rs             |  7 +++
 drivers/gpu/nova-core/gsp/fw.rs               | 55 ++++++++++++++++++-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  2 +
 7 files changed, 69 insertions(+), 7 deletions(-)
drivers/gpu/nova-core/fb.rs +0 -1
index 4d6a1f452183..5580498ba2fb 100644
@@ -87,7 +87,6 @@ pub(crate) fn unregister(&self, bar: &Bar0) {
 ///
 /// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process.
 #[derive(Debug)]
-#[expect(dead_code)]
 pub(crate) struct FbLayout {
     /// Range of the framebuffer. Starts at `0`.
     pub(crate) fb: Range<u64>,
drivers/gpu/nova-core/firmware/gsp.rs +1 -2
index 3a1cf0607de7..c9ad912a3150 100644
@@ -131,7 +131,7 @@ pub(crate) struct GspFirmware {
     /// Size in bytes of the firmware contained in [`Self::fw`].
     pub size: usize,
     /// Device-mapped GSP signatures matching the GPU's [`Chipset`].
-    signatures: DmaObject,
+    pub signatures: DmaObject,
     /// GSP bootloader, verifies the GSP firmware before loading and running it.
     pub bootloader: RiscvFirmware,
 }
@@ -216,7 +216,6 @@ pub(crate) fn new<'a, 'b>(
         }))
     }
 
-    #[expect(unused)]
     /// Returns the DMA handle of the radix3 level 0 page table.
     pub(crate) fn radix3_dma_handle(&self) -> DmaAddress {
         self.level0.dma_handle()
drivers/gpu/nova-core/firmware/riscv.rs +3 -3
index 04f1283abb72..115b5f5355a1 100644
@@ -55,11 +55,11 @@ fn new(bin_fw: &BinFirmware<'_>) -> Result<Self> {
 #[expect(unused)]
 pub(crate) struct RiscvFirmware {
     /// Offset at which the code starts in the firmware image.
-    code_offset: u32,
+    pub(crate) code_offset: u32,
     /// Offset at which the data starts in the firmware image.
-    data_offset: u32,
+    pub(crate) data_offset: u32,
     /// Offset at which the manifest starts in the firmware image.
-    manifest_offset: u32,
+    pub(crate) manifest_offset: u32,
     /// Application version.
     app_version: u32,
     /// Device-mapped firmware image.
drivers/gpu/nova-core/gsp.rs +2 -0
index f1727173bd42..554eb1a34ee7 100644
@@ -10,6 +10,8 @@
 use kernel::prelude::*;
 use kernel::transmute::AsBytes;
 
+use crate::fb::FbLayout;
+
 pub(crate) use fw::{GspFwWprMeta, LibosParams};
 
 mod fw;
drivers/gpu/nova-core/gsp/boot.rs +7 -0
index fb22508128c4..1d2448331d7a 100644
@@ -1,6 +1,8 @@
 // SPDX-License-Identifier: GPL-2.0
 
 use kernel::device;
+use kernel::dma::CoherentAllocation;
+use kernel::dma_write;
 use kernel::pci;
 use kernel::prelude::*;
 
@@ -14,6 +16,7 @@
     FIRMWARE_VERSION,
 };
 use crate::gpu::Chipset;
+use crate::gsp::GspFwWprMeta;
 use crate::regs;
 use crate::vbios::Vbios;
 
@@ -132,6 +135,10 @@ pub(crate) fn boot(
             bar,
         )?;
 
+        let wpr_meta =
+            CoherentAllocation::<GspFwWprMeta>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?;
+        dma_write!(wpr_meta[0] = GspFwWprMeta::new(&gsp_fw, &fb_layout))?;
+
         Ok(())
     }
 }
drivers/gpu/nova-core/gsp/fw.rs +54 -1

70 lines not on this page · raw message

drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +2 -0
index 6a14cc324391..392b25dc6991 100644
@@ -9,6 +9,8 @@
 pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB: u32 = 256;
 pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB: u32 = 88;
 pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB: u32 = 280;
+pub const GSP_FW_WPR_META_REVISION: u32 = 1;
+pub const GSP_FW_WPR_META_MAGIC: i64 = -2577556379034558285;
 pub type __u8 = ffi::c_uchar;
 pub type __u16 = ffi::c_ushort;
 pub type __u32 = ffi::c_uint;
Signature · 1 line
2.50.1
#6 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

Quoted 43 lines · Alistair Popple

The GSP requires some pieces of metadata to boot. These are passed in a struct which the GSP transfers via DMA. Create this struct and get a handle to it for future use when booting the GSP.

---

Changes for v5:
- Make member visibility match the struct visibility (thanks Danilo)

Changes for v3:
- Don't re-export WPR constants (thanks Alex)

Changes for v2:
- Rebased on Alex's latest version

---
 drivers/gpu/nova-core/fb.rs                   |  1 -
 drivers/gpu/nova-core/firmware/gsp.rs         |  3 +-
 drivers/gpu/nova-core/firmware/riscv.rs       |  6 +-
 drivers/gpu/nova-core/gsp.rs                  |  2 +
 drivers/gpu/nova-core/gsp/boot.rs             |  7 +++
 drivers/gpu/nova-core/gsp/fw.rs               | 55 ++++++++++++++++++-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  2 +
 7 files changed, 69 insertions(+), 7 deletions(-)
drivers/gpu/nova-core/fb.rs +0 -1
index 4d6a1f452183..5580498ba2fb 100644
@@ -87,7 +87,6 @@ pub(crate) fn unregister(&self, bar: &Bar0) {
 ///
 /// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process.
 #[derive(Debug)]
-#[expect(dead_code)]
 pub(crate) struct FbLayout {
     /// Range of the framebuffer. Starts at `0`.
     pub(crate) fb: Range<u64>,
drivers/gpu/nova-core/firmware/gsp.rs +0 -0
index 3a1cf0607de7..c9ad912a3150 100644

drivers/gpu/nova-core/firmware/gsp.rs pub(crate) struct GspFirmware { · line 134

Quoted 5 lines
drivers/gpu/nova-core/firmware/gsp.rs +1 -1
@@ -131,7 +131,7 @@ pub(crate) struct GspFirmware {
     /// Size in bytes of the firmware contained in [`Self::fw`].
     pub size: usize,
     /// Device-mapped GSP signatures matching the GPU's [`Chipset`].
-    signatures: DmaObject,
+    pub signatures: DmaObject,

This needs to be pub(crate) or rustc 1.78 emits a warning (I'll also have to fix the surrounding ones in my own patch).

<snip>

drivers/gpu/nova-core/firmware/gsp.rs pub(crate) struct GspFirmware {

Quoted 7 lines
drivers/gpu/nova-core/firmware/gsp.rs +7 -0
+impl GspFwWprMeta {
+    pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
+        Self(bindings::GspFwWprMeta {
+            magic: r570_144::GSP_FW_WPR_META_MAGIC as u64,
+            revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION),
+            sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(),
+            sizeOfRadix3Elf: gsp_firmware.size as u64,

Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.

#7 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 44 lines · Alistair Popple
Quoted 43 lines

The GSP requires some pieces of metadata to boot. These are passed in a struct which the GSP transfers via DMA. Create this struct and get a handle to it for future use when booting the GSP.

---

Changes for v5:
- Make member visibility match the struct visibility (thanks Danilo)

Changes for v3:
- Don't re-export WPR constants (thanks Alex)

Changes for v2:
- Rebased on Alex's latest version

---
 drivers/gpu/nova-core/fb.rs                   |  1 -
 drivers/gpu/nova-core/firmware/gsp.rs         |  3 +-
 drivers/gpu/nova-core/firmware/riscv.rs       |  6 +-
 drivers/gpu/nova-core/gsp.rs                  |  2 +
 drivers/gpu/nova-core/gsp/boot.rs             |  7 +++
 drivers/gpu/nova-core/gsp/fw.rs               | 55 ++++++++++++++++++-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  2 +
 7 files changed, 69 insertions(+), 7 deletions(-)
drivers/gpu/nova-core/fb.rs +0 -1
index 4d6a1f452183..5580498ba2fb 100644
@@ -87,7 +87,6 @@ pub(crate) fn unregister(&self, bar: &Bar0) {
 ///
 /// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process.
 #[derive(Debug)]
-#[expect(dead_code)]
 pub(crate) struct FbLayout {
     /// Range of the framebuffer. Starts at `0`.
     pub(crate) fb: Range<u64>,
drivers/gpu/nova-core/firmware/gsp.rs +0 -0
index 3a1cf0607de7..c9ad912a3150 100644
@@ -131,7 +131,7 @@ pub(crate) struct GspFirmware {
Quoted 8 lines

drivers/gpu/nova-core/firmware/gsp.rs pub(crate) struct GspFirmware { · line 134

Quoted 5 lines
drivers/gpu/nova-core/firmware/gsp.rs +1 -1
     /// Size in bytes of the firmware contained in [`Self::fw`].
     pub size: usize,
     /// Device-mapped GSP signatures matching the GPU's [`Chipset`].
-    signatures: DmaObject,
+    pub signatures: DmaObject,

This needs to be pub(crate) or rustc 1.78 emits a warning (I'll also have to fix the surrounding ones in my own patch).

Thanks, missed that one.

Quoted 11 lines

<snip>

drivers/gpu/nova-core/firmware/gsp.rs

Quoted 7 lines
drivers/gpu/nova-core/firmware/gsp.rs +7 -0
+impl GspFwWprMeta {
+    pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
+        Self(bindings::GspFwWprMeta {
+            magic: r570_144::GSP_FW_WPR_META_MAGIC as u64,
+            revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION),
+            sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(),
+            sizeOfRadix3Elf: gsp_firmware.size as u64,

Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.

And presumably most of the other as keywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?

maintainer
#8 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

Quoted 10 lines · Alistair Popple

On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7 -0
+impl GspFwWprMeta {
+    pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
+        Self(bindings::GspFwWprMeta {
+            magic: r570_144::GSP_FW_WPR_META_MAGIC as u64,
+            revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION),
+            sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(),
+            sizeOfRadix3Elf: gsp_firmware.size as u64,
Quoted 7 lines
Quoted 2 lines

Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.

And presumably most of the other as keywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?

Since nova-core depends on CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.

This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.

We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.

#9 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

Quoted 10 lines · Danilo Krummrich

Since nova-core depends on CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.

This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.

We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.

Indeed.

Cheers,
Miguel

#10 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

In reply to Danilo Krummrich
Quoted 11 lines · Danilo Krummrich
Quoted 10 lines · Alistair Popple

On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7 -0
+impl GspFwWprMeta {
+    pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
+        Self(bindings::GspFwWprMeta {
+            magic: r570_144::GSP_FW_WPR_META_MAGIC as u64,
+            revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION),
+            sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(),
+            sizeOfRadix3Elf: gsp_firmware.size as u64,
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.

And presumably most of the other as keywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?

Indeed, the concern is that as performs a lossy conversion without warning, which could catch us off-guard if e.g. some type changes in the bindings.

Quoted 10 lines

Since nova-core depends on CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.

This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.

We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.

That would definitely be the correct way to address this.

#11 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

On 2025-10-17 at 11:43 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 28 lines · Danilo Krummrich
Quoted 11 lines · Alistair Popple
Quoted 10 lines

On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7 -0
+impl GspFwWprMeta {
+    pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
+        Self(bindings::GspFwWprMeta {
+            magic: r570_144::GSP_FW_WPR_META_MAGIC as u64,
+            revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION),
+            sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(),
+            sizeOfRadix3Elf: gsp_firmware.size as u64,
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.

And presumably most of the other as keywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?

Indeed, the concern is that as performs a lossy conversion without warning, which could catch us off-guard if e.g. some type changes in the bindings.

Quoted 4 lines

Since nova-core depends on CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.

Quoted 8 lines
Quoted 6 lines

This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.

We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.

That would definitely be the correct way to address this.

Sure. Although I don't really have a problem with the binding constructors being fallible as plenty of the others are anyway.

#12 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

Quoted 32 lines · Alistair Popple

On 2025-10-17 at 11:43 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 31 lines · Danilo Krummrich
Quoted 11 lines · Alistair Popple
Quoted 10 lines

On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7 -0
+impl GspFwWprMeta {
+    pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
+        Self(bindings::GspFwWprMeta {
+            magic: r570_144::GSP_FW_WPR_META_MAGIC as u64,
+            revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION),
+            sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(),
+            sizeOfRadix3Elf: gsp_firmware.size as u64,
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.

And presumably most of the other as keywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?

Indeed, the concern is that as performs a lossy conversion without warning, which could catch us off-guard if e.g. some type changes in the bindings.

Quoted 7 lines

Since nova-core depends on CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.

This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.

That would definitely be the correct way to address this.

Sure. Although I don't really have a problem with the binding constructors being fallible as plenty of the others are anyway.

I think we can address the non-fallible conversions as a separate task (as there are many sites that could be similarly optimized in the nova-core code), so fallible constructors are acceptable imho.

maintainer
#13 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

Quoted 37 lines · Alexandre Courbot
Quoted 36 lines · Alistair Popple

On 2025-10-17 at 11:43 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 35 lines · Danilo Krummrich
Quoted 11 lines · Alistair Popple
Quoted 10 lines

On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 9 lines · Alistair Popple
Quoted 7 lines
(quoted diff) +7 -0
+impl GspFwWprMeta {
+    pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
+        Self(bindings::GspFwWprMeta {
+            magic: r570_144::GSP_FW_WPR_META_MAGIC as u64,
+            revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION),
+            sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(),
+            sizeOfRadix3Elf: gsp_firmware.size as u64,
Quoted 7 lines
Quoted 7 lines
Quoted 2 lines

Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.

And presumably most of the other as keywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?

Indeed, the concern is that as performs a lossy conversion without warning, which could catch us off-guard if e.g. some type changes in the bindings.

Quoted 10 lines

Since nova-core depends on CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.

This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.

We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.

Quoted 8 lines
Quoted 4 lines
Quoted 1 line

That would definitely be the correct way to address this.

Sure. Although I don't really have a problem with the binding constructors being fallible as plenty of the others are anyway.

I think we can address the non-fallible conversions as a separate task (as there are many sites that could be similarly optimized in the nova-core code), so fallible constructors are acceptable imho.

The infallible conversion function is trivial to implement.

If you prefer, we can also add it in nova-core first and subsequently move it to generic infrastructure.

I prefer not to introduce more as-casts or fallible conversions we have to clean up subsequently.

#14 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

In reply to Danilo Krummrich
Quoted 22 lines · Danilo Krummrich
Quoted 10 lines · Alistair Popple

On 2025-10-16 at 17:23 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 9 lines · Alistair Popple
Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7 -0
+impl GspFwWprMeta {
+    pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
+        Self(bindings::GspFwWprMeta {
+            magic: r570_144::GSP_FW_WPR_META_MAGIC as u64,
+            revision: u64::from(r570_144::GSP_FW_WPR_META_REVISION),
+            sysmemAddrOfRadix3Elf: gsp_firmware.radix3_dma_handle(),
+            sizeOfRadix3Elf: gsp_firmware.size as u64,
Quoted 7 lines
Quoted 2 lines

Very unfortunately I'm afraid we will need to replace the as in this method with u64::try_from and make it return a Result for now.

And presumably most of the other as keywords in this function dealing with usize as well? Have made the change but would you mind quickly explaining why this is needed? Is the concern that usize might be more than 64 bits or something?

Since nova-core depends on CONFIG_64BIT, I think we want a helper function that converts usize to u64 infallibly.

Quoted 6 lines

This helper function can simply generate a compile time error, when !CONFIG_64BIT, etc.

We can do this locally in nova-core, but it could also find it's place in the generic infrastructure. nova-core clearly won't be the last driver running into this inconvinience.

How do you see this taking shape concretely? I have tried writing extention traits (e.g. IntoU64), but unfortunately we also need to use these in const contexts so that won't until const trait methods are a thing.

The alternative would be to have const functions like usize_to_u64. It doesn't look as smooth as the extention trait, but can be used in const contexts.

I can try and provide an implementation to apply before this patch series if given guidance on which direction we want to go.

maintainer
#15 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

Quoted 3 lines · Alexandre Courbot

The alternative would be to have const functions like usize_to_u64. It doesn't look as smooth as the extention trait, but can be used in const contexts.

That's what I thought of, exactly for the reason of being usable in const contexts (at least for a quick fix in nova-core).

Whether we want an extention trait and a separate temporary const_usize_to_u64() etc. can be discussed in the context of making it common infrastructure.

#16 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

Quoted 10 lines · Danilo Krummrich
Quoted 3 lines · Alexandre Courbot

The alternative would be to have const functions like usize_to_u64. It doesn't look as smooth as the extention trait, but can be used in const contexts.

That's what I thought of, exactly for the reason of being usable in const contexts (at least for a quick fix in nova-core).

Whether we want an extention trait and a separate temporary const_usize_to_u64() etc. can be discussed in the context of making it common infrastructure.

Thanks - so IIUC the idea would be to keep this local to nova-core in a first time? If so I guess I can produce this fast (and convert our many uses of as in the driver so far).

maintainer
#17 lore raw

Re: [PATCH v5 03/14] gpu: nova-core: gsp: Create wpr metadata

Quoted 9 lines · Alexandre Courbot
Quoted 8 lines · Danilo Krummrich
Quoted 3 lines · Alexandre Courbot

The alternative would be to have const functions like usize_to_u64. It doesn't look as smooth as the extention trait, but can be used in const contexts.

That's what I thought of, exactly for the reason of being usable in const contexts (at least for a quick fix in nova-core).

Quoted 6 lines
Quoted 2 lines

Whether we want an extention trait and a separate temporary const_usize_to_u64() etc. can be discussed in the context of making it common infrastructure.

Thanks - so IIUC the idea would be to keep this local to nova-core in a first time? If so I guess I can produce this fast (and convert our many uses of as in the driver so far).

Yeah, I think that's reasonable. We can do both in parallel.

#18 lore raw

PATCH v5 4/14 gpu: nova-core: Add a slice-buffer (sbuffer) datastructure

In reply to Alistair Popple

From: Joel Fernandes <[email protected]>

A data structure that can be used to write across multiple slices which may be out of order in memory. This lets SBuffer user correctly and safely write out of memory order, without error-prone tracking of pointers/offsets.

let mut buf1 = [0u8; 3];
let mut buf2 = [0u8; 5];
let mut sbuffer = SBuffer::new([&mut buf1[..], &mut buf2[..]]);

let data = b"hello";
let result = sbuffer.write(data);

An internal conversion of gsp.rs to use this resulted in a nice -ve delta:
gsp.rs: 37 insertions(+), 99 deletions(-)

---

Changes for v5:
- Typos
- s/ETOOSMALL/EINVAL/
- Add documentation
- Fix up examples

Changes for v3:
- Addressed minor review comment from Lyude

---
 drivers/gpu/nova-core/nova_core.rs |   1 +
 drivers/gpu/nova-core/sbuffer.rs   | 218 +++++++++++++++++++++++++++++
 2 files changed, 219 insertions(+)
 create mode 100644 drivers/gpu/nova-core/sbuffer.rs
drivers/gpu/nova-core/nova_core.rs +1 -0
index fffcaee2249f..a6feeba6254c 100644
@@ -11,6 +11,7 @@
 mod gpu;
 mod gsp;
 mod regs;
+mod sbuffer;
 mod util;
 mod vbios;
 
drivers/gpu/nova-core/sbuffer.rs new file +218 -0

218 lines not on this page · raw message

Signature · 1 line
2.50.1
#19 lore raw

Re: [PATCH v5 04/14] gpu: nova-core: Add a slice-buffer (sbuffer) datastructure

Quoted 69 lines · Alistair Popple

From: Joel Fernandes <[email protected]>

A data structure that can be used to write across multiple slices which may be out of order in memory. This lets SBuffer user correctly and safely write out of memory order, without error-prone tracking of pointers/offsets.

let mut buf1 = [0u8; 3];
let mut buf2 = [0u8; 5];
let mut sbuffer = SBuffer::new([&mut buf1[..], &mut buf2[..]]);

let data = b"hello";
let result = sbuffer.write(data);

An internal conversion of gsp.rs to use this resulted in a nice -ve delta:
gsp.rs: 37 insertions(+), 99 deletions(-)

---

Changes for v5:
- Typos
- s/ETOOSMALL/EINVAL/
- Add documentation
- Fix up examples

Changes for v3:
- Addressed minor review comment from Lyude

---
 drivers/gpu/nova-core/nova_core.rs |   1 +
 drivers/gpu/nova-core/sbuffer.rs   | 218 +++++++++++++++++++++++++++++
 2 files changed, 219 insertions(+)
 create mode 100644 drivers/gpu/nova-core/sbuffer.rs
drivers/gpu/nova-core/nova_core.rs +1 -0
index fffcaee2249f..a6feeba6254c 100644
@@ -11,6 +11,7 @@
 mod gpu;
 mod gsp;
 mod regs;
+mod sbuffer;
 mod util;
 mod vbios;
 
drivers/gpu/nova-core/sbuffer.rs new file +13 -0
new file mode 100644
index 000000000000..d9c412a68bd8
@@ -0,0 +1,218 @@
+// SPDX-License-Identifier: GPL-2.0
+
+use core::ops::Deref;
+
+use kernel::alloc::KVec;
+use kernel::error::code::*;
+use kernel::prelude::*;
+
+/// A buffer abstraction for discontiguous byte slices.
+///
+/// This allows you to treat multiple non-contiguous `&mut [u8]` slices
+/// of the same length as a single stream-like read/write buffer.
+///

drivers/gpu/nova-core/sbuffer.rs line 23

Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+/// # Example:
+///
+/// ```
+/// let mut buf1 = [0u8; 5];
+/// let mut buf2 = [0u8; 5];
+/// let mut sbuffer = SBufferIter::new_writer([&buf1, &buf2]);
+///
+/// let data = b"hello";
+/// let result = sbuffer.write_all(data);
+/// ```

This example doesn't build - there are several things wrong with it. It is also missing statements to confirm and show the expected result. Here is a fixed and slightly improved version:

/// let mut buf1 = [0u8; 5];
/// let mut buf2 = [0u8; 5];
/// let mut sbuffer = SBufferIter::new_writer([&mut buf1[..], &mut buf2[..]]); ///
/// let data = b"hi world!";
/// sbuffer.write_all(data)?;
/// drop(sbuffer);
///
/// assert_eq!(buf1, *b"hi wo");
/// assert_eq!(buf2, *b"rld!\0");
///
/// # Ok::<(), Error>(())

drivers/gpu/nova-core/sbuffer.rs line 30

Quoted 7 lines
drivers/gpu/nova-core/sbuffer.rs new file +7 -0
+///
+/// A sliding window of slices to process.
+///
+/// Both read and write buffers are implemented in terms of operating on slices of a requested
+/// size. This base class implements logic that can be shared between the two to support that.
+///
+/// `S` is a slice type, `I` is an iterator yielding `S`.

Why is there another doccomment after the example section? It looks like this should be merged with the first doccomment before the example? There is also no S generic parameter.

Quoted 50 lines
drivers/gpu/nova-core/sbuffer.rs new file +50 -0
+pub(crate) struct SBufferIter<I: Iterator> {
+    /// `Some` if we are not at the end of the data yet.
+    cur_slice: Option<I::Item>,
+    /// All the slices remaining after `cur_slice`.
+    slices: I,
+}
+
+impl<'a, I> SBufferIter<I>
+where
+    I: Iterator,
+{
+    /// Creates a reader buffer for a discontiguous set of byte slices.
+    ///
+    /// # Example:
+    ///
+    /// ```
+    /// let buf1: [u8; 5] = [0, 1, 2, 3, 4];
+    /// let buf2: [u8; 5] = [5, 6, 7, 8, 9];
+    /// let sbuffer = SBufferIter::new_reader([&buf1[..], &buf2[..]]);
+    /// let sum: u8 = sbuffer.sum();
+    /// assert_eq!(sum, 45);
+    /// ```
+    #[expect(unused)]
+    pub(crate) fn new_reader(slices: impl IntoIterator<IntoIter = I>) -> Self
+    where
+        I: Iterator<Item = &'a [u8]>,
+    {
+        Self::new(slices)
+    }
+
+    /// Creates a writeable buffer for a discontiguous set of byte slices.
+    ///
+    /// # Example:
+    ///
+    /// ```
+    /// let mut buf1 = [0u8; 5];
+    /// let mut buf2 = [0u8; 5];
+    /// let mut sbuffer = SBufferIter::new_writer([&mut buf1[..], &mut buf2[..]]);
+    /// sbuffer.write_all(&[0u8, 1, 2, 3, 4, 5, 6, 7, 8, 9][..])?;
+    /// drop(sbuffer);
+    /// assert_eq!(buf1, [0, 1, 2, 3, 4]);
+    /// assert_eq!(buf2, [5, 6, 7, 8, 9]);
+    ///
+    /// ```
+    #[expect(unused)]
+    pub(crate) fn new_writer(slices: impl IntoIterator<IntoIter = I>) -> Self
+    where
+        I: Iterator<Item = &'a mut [u8]>,
+    {
+        Self::new(slices)

drivers/gpu/nova-core/sbuffer.rs line 90

Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+    }
+
+    fn new(slices: impl IntoIterator<IntoIter = I>) -> Self
+    where
+        I::Item: Deref<Target = [u8]>,
+    {
+        let mut slices = slices.into_iter();
+
+        Self {
+            // Skip empty slices to avoid trouble down the road.

I guess "Skip empty slices" is enough as it is part of the algorithm. :)

drivers/gpu/nova-core/sbuffer.rs line 100

Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+            cur_slice: slices.find(|s| !s.deref().is_empty()),
+            slices,
+        }
+    }
+
+    fn get_slice_internal(
+        &mut self,
+        len: usize,
+        mut f: impl FnMut(I::Item, usize) -> (I::Item, I::Item),
+    ) -> Option<I::Item>

Let's document this a bit as its purpose is not immediately clear. We can take the documentation from the get_slice methods, with a short explanation that the closure is supposed to split the slice received as first argument at the position given as the second.

Quoted 31 lines
drivers/gpu/nova-core/sbuffer.rs new file +31 -0
+    where
+        I::Item: Deref<Target = [u8]>,
+    {
+        match self.cur_slice.take() {
+            None => None,
+            Some(cur_slice) => {
+                if len >= cur_slice.len() {
+                    // Caller requested more data than is in the current slice, return it entirely
+                    // and prepare the following slice for being used. Skip empty slices to avoid
+                    // trouble.
+                    self.cur_slice = self.slices.find(|s| !s.is_empty());
+
+                    Some(cur_slice)
+                } else {
+                    // The current slice can satisfy the request, split it and return a slice of
+                    // the requested size.
+                    let (ret, next) = f(cur_slice, len);
+                    self.cur_slice = Some(next);
+
+                    Some(ret)
+                }
+            }
+        }
+    }
+}
+
+/// Provides a way to get non-mutable slices of data to read from.
+impl<'a, I> SBufferIter<I>
+where
+    I: Iterator<Item = &'a [u8]>,
+{

drivers/gpu/nova-core/sbuffer.rs line 141

Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+    /// Returns a slice of at most `len` bytes, or `None` if we are at the end of the data.
+    ///
+    /// If a slice shorter than `len` bytes has been returned, the caller can call this method
+    /// again until it returns `None` to try and obtain the remainder of the data.
+    fn get_slice(&mut self, len: usize) -> Option<&'a [u8]> {
+        self.get_slice_internal(len, |s, pos| s.split_at(pos))
+    }
+
+    /// Ideally we would implement `Read`, but it is not available in `core`.
+    /// So mimic `std::io::Read::read_exact`.

That's a useful side-comment, but we also need small sentence describing the method, e.g. "Fill dst with the next bytes from this SBufferIter, or return EINVAL if there isn't enough data available."

Quoted 11 lines
drivers/gpu/nova-core/sbuffer.rs new file +11 -0
+    #[expect(unused)]
+    pub(crate) fn read_exact(&mut self, mut dst: &mut [u8]) -> Result {
+        while !dst.is_empty() {
+            match self.get_slice(dst.len()) {
+                None => return Err(EINVAL),
+                Some(src) => {
+                    let dst_slice;
+                    (dst_slice, dst) = dst.split_at_mut(src.len());
+                    dst_slice.copy_from_slice(src);
+                }
+            }

drivers/gpu/nova-core/sbuffer.rs line 162

Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+        }
+
+        Ok(())
+    }
+
+    /// Read all the remaining data into a `KVec`.
+    ///
+    /// `self` will be empty after this operation.
+    #[expect(unused)]
+    pub(crate) fn read_into_kvec(&mut self, flags: kernel::alloc::Flags) -> Result<KVec<u8>> {

nit: flush_into_kvec is probably a more descriptive name, as read is already used for the other method.

Quoted 18 lines
drivers/gpu/nova-core/sbuffer.rs new file +18 -0
+        let mut buf = KVec::<u8>::new();
+
+        if let Some(slice) = core::mem::take(&mut self.cur_slice) {
+            buf.extend_from_slice(slice, flags)?;
+        }
+        for slice in &mut self.slices {
+            buf.extend_from_slice(slice, flags)?;
+        }
+
+        Ok(buf)
+    }
+}
+
+/// Provides a way to get mutable slices of data to write into.
+impl<'a, I> SBufferIter<I>
+where
+    I: Iterator<Item = &'a mut [u8]>,
+{

drivers/gpu/nova-core/sbuffer.rs line 190

Quoted 10 lines
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+    /// Returns a mutable slice of at most `len` bytes, or `None` if we are at the end of the data.
+    ///
+    /// If a slice shorter than `len` bytes has been returned, the caller can call this method
+    /// again until it returns `None` to try and obtain the remainder of the data.
+    fn get_slice_mut(&mut self, len: usize) -> Option<&'a mut [u8]> {
+        self.get_slice_internal(len, |s, pos| s.split_at_mut(pos))
+    }
+
+    /// Ideally we would implement `Write`, but it is not available in `core`.
+    /// So mimic `std::io::Write::write_all`.

Same comment as read_all.

#20 lore raw

Re: [PATCH v5 04/14] gpu: nova-core: Add a slice-buffer (sbuffer) datastructure

Quoted 17 lines · Alexandre Courbot

drivers/gpu/nova-core/sbuffer.rs line 23

Quoted 11 lines · Alistair Popple
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+/// # Example:
+///
+/// ```
+/// let mut buf1 = [0u8; 5];
+/// let mut buf2 = [0u8; 5];
+/// let mut sbuffer = SBufferIter::new_writer([&buf1, &buf2]);
+///
+/// let data = b"hello";
+/// let result = sbuffer.write_all(data);
+/// ```

This example doesn't build - there are several things wrong with it. It is also missing statements to confirm and show the expected result. Here is a fixed and slightly improved version:

Yeah, I mentioned this one in a previous version -- the section header is also still wrong too.

Alistair, please check the link I gave:

https://docs.kernel.org/rust/coding-guidelines.html#code-documentation

or other code in the kernel crate for examples on how it is usually done.

It is not critical today, of course, but the further it is from what will be needed in a few months, the harder it will become to start building the docs and running the examples as KUnit tests.

Thanks!

Cheers,
Miguel

#21 lore raw

Re: [PATCH v5 04/14] gpu: nova-core: Add a slice-buffer (sbuffer) datastructure

On 2025-10-17 at 06:18 +1100, Miguel Ojeda <[email protected]> wrote...

Quoted 18 lines · Alexandre Courbot
Quoted 17 lines · Alistair Popple

drivers/gpu/nova-core/sbuffer.rs line 23

Quoted 11 lines
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+/// # Example:
+///
+/// ```
+/// let mut buf1 = [0u8; 5];
+/// let mut buf2 = [0u8; 5];
+/// let mut sbuffer = SBufferIter::new_writer([&buf1, &buf2]);
+///
+/// let data = b"hello";
+/// let result = sbuffer.write_all(data);
+/// ```

This example doesn't build - there are several things wrong with it. It is also missing statements to confirm and show the expected result. Here is a fixed and slightly improved version:

Argh, you're right. I cut and pasted then edited the wrong thing from my test build. How are you building these? The rustdoc target seems to ignore Nova (or I'm doing something wrong).

Quoted 4 lines

Yeah, I mentioned this one in a previous version -- the section header is also still wrong too.

Alistair, please check the link I gave:

Will do. I thought Joel had addressed your comments in the fix patch I pulled in from him (he wrote most of this originally) but I can see the /// # Example: heading is wrong.

Quoted 7 lines

https://docs.kernel.org/rust/coding-guidelines.html#code-documentation

or other code in the kernel crate for examples on how it is usually done.

It is not critical today, of course, but the further it is from what will be needed in a few months, the harder it will become to start building the docs and running the examples as KUnit tests.

No, I think it *is* critical :-) Much easier just to get this right from the beginning than deal with heaps of errors/warnings later. It's just my fingers that are still getting used to the subtle differences between C kernel code and Rust kernel code, so thanks for the guidance.

- Alistair

Quoted 5 lines

Thanks!

Cheers,
Miguel

#22 lore raw

Re: [PATCH v5 04/14] gpu: nova-core: Add a slice-buffer (sbuffer) datastructure

Quoted 20 lines · Alistair Popple

On 2025-10-17 at 06:18 +1100, Miguel Ojeda <[email protected]> wrote...

Quoted 18 lines · Alexandre Courbot
Quoted 17 lines · Alistair Popple

drivers/gpu/nova-core/sbuffer.rs line 23

Quoted 11 lines
drivers/gpu/nova-core/sbuffer.rs new file +10 -0
+/// # Example:
+///
+/// ```
+/// let mut buf1 = [0u8; 5];
+/// let mut buf2 = [0u8; 5];
+/// let mut sbuffer = SBufferIter::new_writer([&buf1, &buf2]);
+///
+/// let data = b"hello";
+/// let result = sbuffer.write_all(data);
+/// ```

This example doesn't build - there are several things wrong with it. It is also missing statements to confirm and show the expected result. Here is a fixed and slightly improved version:

Quoted 3 lines

Argh, you're right. I cut and pasted then edited the wrong thing from my test build. How are you building these? The rustdoc target seems to ignore Nova (or I'm doing something wrong).

Indeed, rustdoc doesn't consider anything outside of the kernel crate for now, although this is scheduled to change so we need to be ready for it.

For now, I just copy/paste the test code into some function elsewhere and try to build. ^_^;

#23 lore raw

PATCH v5 5/14 gpu: nova-core: Add zeroable trait to bindings

In reply to Alistair Popple

Derive the Zeroable trait for existing bindgen generated bindings. This is safe because all bindgen generated types are simple integer types for which any bit pattern, including all zeros, is valid.

---

Changes for v5:
- New for v5

---
 drivers/gpu/nova-core/gsp/fw/r570_144.rs         |  1 +
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs    | 16 ++++++++--------
 2 files changed, 9 insertions(+), 8 deletions(-)
drivers/gpu/nova-core/gsp/fw/r570_144.rs +1 -0
index 82a973cd99c3..4f5c65ac1eb9 100644
@@ -25,4 +25,5 @@
     unsafe_op_in_unsafe_fn
 )]
 use kernel::ffi;
+use kernel::prelude::Zeroable;
 include!("r570_144/bindings.rs");
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +8 -8
index 392b25dc6991..f7b38978c5f8 100644
@@ -20,7 +20,7 @@
 pub type u32_ = __u32;
 pub type u64_ = __u64;
 #[repr(C)]
-#[derive(Copy, Clone)]
+#[derive(Copy, Clone, Zeroable)]
 pub struct GspFwWprMeta {
     pub magic: u64_,
     pub revision: u64_,
@@ -55,19 +55,19 @@ pub struct GspFwWprMeta {
     pub verified: u64_,
 }
 #[repr(C)]
-#[derive(Copy, Clone)]
+#[derive(Copy, Clone, Zeroable)]
 pub union GspFwWprMeta__bindgen_ty_1 {
     pub __bindgen_anon_1: GspFwWprMeta__bindgen_ty_1__bindgen_ty_1,
     pub __bindgen_anon_2: GspFwWprMeta__bindgen_ty_1__bindgen_ty_2,
 }
 #[repr(C)]
-#[derive(Debug, Default, Copy, Clone)]
+#[derive(Debug, Default, Copy, Clone, Zeroable)]
 pub struct GspFwWprMeta__bindgen_ty_1__bindgen_ty_1 {
     pub sysmemAddrOfSignature: u64_,
     pub sizeOfSignature: u64_,
 }
 #[repr(C)]
-#[derive(Debug, Default, Copy, Clone)]
+#[derive(Debug, Default, Copy, Clone, Zeroable)]
 pub struct GspFwWprMeta__bindgen_ty_1__bindgen_ty_2 {
     pub gspFwHeapFreeListWprOffset: u32_,
     pub unused0: u32_,
@@ -83,13 +83,13 @@ fn default() -> Self {
     }
 }
 #[repr(C)]
-#[derive(Copy, Clone)]
+#[derive(Copy, Clone, Zeroable)]
 pub union GspFwWprMeta__bindgen_ty_2 {
     pub __bindgen_anon_1: GspFwWprMeta__bindgen_ty_2__bindgen_ty_1,
     pub __bindgen_anon_2: GspFwWprMeta__bindgen_ty_2__bindgen_ty_2,
 }
 #[repr(C)]
-#[derive(Debug, Default, Copy, Clone)]
+#[derive(Debug, Default, Copy, Clone, Zeroable)]
 pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_1 {
     pub partitionRpcAddr: u64_,
     pub partitionRpcRequestOffset: u16_,
@@ -101,7 +101,7 @@ pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_1 {
     pub lsUcodeVersion: u32_,
 }
 #[repr(C)]
-#[derive(Debug, Default, Copy, Clone)]
+#[derive(Debug, Default, Copy, Clone, Zeroable)]
 pub struct GspFwWprMeta__bindgen_ty_2__bindgen_ty_2 {
     pub partitionRpcPadding: [u32_; 4usize],
     pub sysmemAddrOfCrashReportQueue: u64_,
@@ -136,7 +136,7 @@ fn default() -> Self {
 pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_FB: LibosMemoryRegionLoc = 2;
 pub type LibosMemoryRegionLoc = ffi::c_uint;
 #[repr(C)]
-#[derive(Debug, Default, Copy, Clone)]
+#[derive(Debug, Default, Copy, Clone, Zeroable)]
 pub struct LibosMemoryRegionInitArgument {
     pub id8: LibosAddress,
     pub pa: LibosAddress,
Signature · 1 line
2.50.1
#24 lore raw

PATCH v5 6/14 gpu: nova-core: Add GSP command queue bindings

In reply to Alistair Popple

Add bindings and accessors used for the GSP command queue.

---

Changes for v5:
- Derive the Zeroable trait for structs and enums

Changes for v4:
- Don't panic the kernel if trying to initialise a large (> 4GB) message header.
- Use init! to provide safe and complete initialisers.
- Take an enum type instead of a u32 for the function.

Changes for v3:
- New for v3

---
 drivers/gpu/nova-core/gsp/fw.rs               | 275 +++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 541 ++++++++++++++++++
 2 files changed, 816 insertions(+)
drivers/gpu/nova-core/gsp/fw.rs +275 -0
index 1cc992ca492c..a2ce570ddfaf 100644
@@ -5,6 +5,7 @@
 // Alias to avoid repeating the version number with every use.
 use r570_144 as bindings;
 
+use core::fmt;
 use core::ops::Range;
 
 use kernel::dma::CoherentAllocation;
@@ -16,6 +17,7 @@
 use crate::firmware::gsp::GspFirmware;
 use crate::gpu::Chipset;
 use crate::gsp::FbLayout;
+use crate::gsp::GSP_PAGE_SIZE;
 
 /// Dummy type to group methods related to heap parameters for running the GSP firmware.
 pub(crate) struct GspFwHeapParams(());
@@ -158,6 +160,120 @@ pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
     }
 }
 
+#[derive(PartialEq)]
+pub(crate) enum MsgFunction {
+    // Common function codes
+    Nop = bindings::NV_VGPU_MSG_FUNCTION_NOP as isize,
+    SetGuestSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO as isize,
+    AllocRoot = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT as isize,
+    AllocDevice = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE as isize,
+    AllocMemory = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY as isize,
+    AllocCtxDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA as isize,
+    AllocChannelDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA as isize,
+    MapMemory = bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY as isize,
+    BindCtxDma = bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA as isize,
+    AllocObject = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT as isize,
+    Free = bindings::NV_VGPU_MSG_FUNCTION_FREE as isize,
+    Log = bindings::NV_VGPU_MSG_FUNCTION_LOG as isize,
+    GetGspStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO as isize,
+    SetRegistry = bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY as isize,
+    GspSetSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO as isize,
+    GspInitPostObjGpu = bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU as isize,
+    GspRmControl = bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL as isize,
+    GetStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO as isize,
+
+    // Event codes
+    GspInitDone = bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE as isize,
+    GspRunCpuSequencer = bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER as isize,
+    PostEvent = bindings::NV_VGPU_MSG_EVENT_POST_EVENT as isize,
+    RcTriggered = bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED as isize,
+    MmuFaultQueued = bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED as isize,
+    OsErrorLog = bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG as isize,
+    GspPostNoCat = bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD as isize,
+    GspLockdownNotice = bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE as isize,
+    UcodeLibOsPrint = bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT as isize,
+}
+
+impl fmt::Display for MsgFunction {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            // Common function codes
+            MsgFunction::Nop => write!(f, "NOP"),
+            MsgFunction::SetGuestSystemInfo => write!(f, "SET_GUEST_SYSTEM_INFO"),
+            MsgFunction::AllocRoot => write!(f, "ALLOC_ROOT"),
+            MsgFunction::AllocDevice => write!(f, "ALLOC_DEVICE"),
+            MsgFunction::AllocMemory => write!(f, "ALLOC_MEMORY"),
+            MsgFunction::AllocCtxDma => write!(f, "ALLOC_CTX_DMA"),
+            MsgFunction::AllocChannelDma => write!(f, "ALLOC_CHANNEL_DMA"),
+            MsgFunction::MapMemory => write!(f, "MAP_MEMORY"),
+            MsgFunction::BindCtxDma => write!(f, "BIND_CTX_DMA"),
+            MsgFunction::AllocObject => write!(f, "ALLOC_OBJECT"),
+            MsgFunction::Free => write!(f, "FREE"),
+            MsgFunction::Log => write!(f, "LOG"),
+            MsgFunction::GetGspStaticInfo => write!(f, "GET_GSP_STATIC_INFO"),
+            MsgFunction::SetRegistry => write!(f, "SET_REGISTRY"),
+            MsgFunction::GspSetSystemInfo => write!(f, "GSP_SET_SYSTEM_INFO"),
+            MsgFunction::GspInitPostObjGpu => write!(f, "GSP_INIT_POST_OBJGPU"),
+            MsgFunction::GspRmControl => write!(f, "GSP_RM_CONTROL"),
+            MsgFunction::GetStaticInfo => write!(f, "GET_STATIC_INFO"),
+
+            // Event codes
+            MsgFunction::GspInitDone => write!(f, "INIT_DONE"),
+            MsgFunction::GspRunCpuSequencer => write!(f, "RUN_CPU_SEQUENCER"),
+            MsgFunction::PostEvent => write!(f, "POST_EVENT"),
+            MsgFunction::RcTriggered => write!(f, "RC_TRIGGERED"),
+            MsgFunction::MmuFaultQueued => write!(f, "MMU_FAULT_QUEUED"),
+            MsgFunction::OsErrorLog => write!(f, "OS_ERROR_LOG"),
+            MsgFunction::GspPostNoCat => write!(f, "NOCAT"),
+            MsgFunction::GspLockdownNotice => write!(f, "LOCKDOWN_NOTICE"),
+            MsgFunction::UcodeLibOsPrint => write!(f, "LIBOS_PRINT"),
+        }
+    }
+}
+
+impl TryFrom<u32> for MsgFunction {
+    type Error = kernel::error::Error;
+
+    fn try_from(value: u32) -> Result<MsgFunction> {
+        match value {
+            bindings::NV_VGPU_MSG_FUNCTION_NOP => Ok(MsgFunction::Nop),
+            bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO => {
+                Ok(MsgFunction::SetGuestSystemInfo)
+            }
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT => Ok(MsgFunction::AllocRoot),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE => Ok(MsgFunction::AllocDevice),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY => Ok(MsgFunction::AllocMemory),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA => Ok(MsgFunction::AllocCtxDma),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA => Ok(MsgFunction::AllocChannelDma),
+            bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY => Ok(MsgFunction::MapMemory),
+            bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA => Ok(MsgFunction::BindCtxDma),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT => Ok(MsgFunction::AllocObject),
+            bindings::NV_VGPU_MSG_FUNCTION_FREE => Ok(MsgFunction::Free),
+            bindings::NV_VGPU_MSG_FUNCTION_LOG => Ok(MsgFunction::Log),
+            bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO => Ok(MsgFunction::GetGspStaticInfo),
+            bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY => Ok(MsgFunction::SetRegistry),
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO => Ok(MsgFunction::GspSetSystemInfo),
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU => {
+                Ok(MsgFunction::GspInitPostObjGpu)
+            }
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL => Ok(MsgFunction::GspRmControl),
+            bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO => Ok(MsgFunction::GetStaticInfo),
+            bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE => Ok(MsgFunction::GspInitDone),
+            bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER => {
+                Ok(MsgFunction::GspRunCpuSequencer)
+            }
+            bindings::NV_VGPU_MSG_EVENT_POST_EVENT => Ok(MsgFunction::PostEvent),
+            bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED => Ok(MsgFunction::RcTriggered),
+            bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED => Ok(MsgFunction::MmuFaultQueued),
+            bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG => Ok(MsgFunction::OsErrorLog),
+            bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD => Ok(MsgFunction::GspPostNoCat),
+            bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE => Ok(MsgFunction::GspLockdownNotice),
+            bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT => Ok(MsgFunction::UcodeLibOsPrint),
+            _ => Err(EINVAL),
+        }
+    }
+}
+
 /// Struct containing the arguments required to pass a memory buffer to the GSP
 /// for use during initialisation.
 ///
@@ -208,3 +324,162 @@ fn id8(name: &str) -> u64 {
         }))
     }
 }
+
+#[repr(transparent)]
+pub(crate) struct MsgqTxHeader(bindings::msgqTxHeader);
+
+impl MsgqTxHeader {
+    pub(crate) fn new(msgq_size: u32, rx_hdr_offset: u32, msg_count: u32) -> Self {
+        Self(bindings::msgqTxHeader {
+            version: 0,
+            size: msgq_size,
+            msgSize: GSP_PAGE_SIZE as u32,
+            msgCount: msg_count,
+            writePtr: 0,
+            flags: 1,
+            rxHdrOff: rx_hdr_offset,
+            entryOff: GSP_PAGE_SIZE as u32,
+        })
+    }
+
+    pub(crate) fn write_ptr(&self) -> u32 {
+        let ptr = (&self.0.writePtr) as *const u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_read! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.read_volatile() }
+    }
+
+    pub(crate) fn set_write_ptr(&mut self, val: u32) {
+        let ptr = (&mut self.0.writePtr) as *mut u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_write! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.write_volatile(val) }
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for MsgqTxHeader {}
+
+/// RX header for setting up a message queue with the GSP.
+#[repr(transparent)]
+pub(crate) struct MsgqRxHeader(bindings::msgqRxHeader);
+
+impl MsgqRxHeader {
+    pub(crate) fn new() -> Self {
+        Self(Default::default())
+    }
+
+    pub(crate) fn read_ptr(&self) -> u32 {
+        let ptr = (&self.0.readPtr) as *const u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_read! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.read_volatile() }
+    }
+
+    pub(crate) fn set_read_ptr(&mut self, val: u32) {
+        let ptr = (&mut self.0.readPtr) as *mut u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_write! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.write_volatile(val) }
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for MsgqRxHeader {}
+
+impl bindings::rpc_message_header_v {
+    pub(crate) fn init(cmd_size: u32, function: MsgFunction) -> impl Init<Self, Error> {
+        type RpcMessageHeader = bindings::rpc_message_header_v;
+        try_init!(RpcMessageHeader {
+            // TODO: magic number
+            header_version: 0x03000000,
+            signature: bindings::NV_VGPU_MSG_SIGNATURE_VALID,
+            function: function as u32,
+            length: (size_of::<Self>() as u32)
+                .checked_add(cmd_size)
+                .ok_or(EOVERFLOW)?,
+            rpc_result: 0xffffffff,
+            rpc_result_private: 0xffffffff,
+            ..Zeroable::init_zeroed()
+        })
+    }
+}
+
+// SAFETY: We can't derive the Zeroable trait for this binding because the
+// procedural macro doesn't support the syntax used by bindgen to create the
+// __IncompleteArrayField types. So instead we implement it here, which is safe
+// because these are explicitly padded structures only containing types for
+// which any bit pattern, including all zeros, is valid.
+unsafe impl Zeroable for bindings::rpc_message_header_v {}
+
+#[repr(transparent)]
+pub(crate) struct GspMsgElement {
+    inner: bindings::GSP_MSG_QUEUE_ELEMENT,
+}
+
+impl GspMsgElement {
+    #[allow(non_snake_case)]
+    pub(crate) fn init(
+        sequence: u32,
+        cmd_size: usize,
+        function: MsgFunction,
+    ) -> impl Init<Self, Error> {
+        type RpcMessageHeader = bindings::rpc_message_header_v;
+        type InnerGspMsgElement = bindings::GSP_MSG_QUEUE_ELEMENT;
+        let init_inner = try_init!(InnerGspMsgElement {
+            seqNum: sequence,
+            elemCount: size_of::<Self>()
+                .checked_add(cmd_size)
+                .ok_or(EOVERFLOW)?
+                .div_ceil(GSP_PAGE_SIZE) as u32,
+            rpc <- RpcMessageHeader::init(cmd_size as u32, function),
+            ..Zeroable::init_zeroed()
+        });
+
+        try_init!(GspMsgElement {
+            inner <- init_inner,
+        })
+    }
+
+    pub(crate) fn set_checksum(&mut self, checksum: u32) {
+        self.inner.checkSum = checksum;
+    }
+
+    // Return the total length of the message, noting that rpc.length includes
+    // the length of the GspRpcHeader but not the message header.
+    pub(crate) fn length(&self) -> u32 {
+        size_of::<Self>() as u32 - size_of::<bindings::rpc_message_header_v>() as u32
+            + self.inner.rpc.length
+    }
+
+    pub(crate) fn sequence(&self) -> u32 {
+        self.inner.rpc.sequence
+    }
+
+    pub(crate) fn function_number(&self) -> u32 {
+        self.inner.rpc.function
+    }
+
+    pub(crate) fn function(&self) -> Result<MsgFunction> {
+        self.inner.rpc.function.try_into()
+    }
+
+    pub(crate) fn element_count(&self) -> u32 {
+        self.inner.elemCount
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for GspMsgElement {}
+
+// SAFETY: This struct only contains integer types for which all bit patterns
+// are valid.
+unsafe impl FromBytes for GspMsgElement {}
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +541 -0

561 lines not on this page · raw message

Signature · 1 line
2.50.1
#25 lore raw

Re: [PATCH v5 06/14] gpu: nova-core: Add GSP command queue bindings

Quoted 3 lines · Alistair Popple

Add bindings and accessors used for the GSP command queue.

There are lots of unused warnings when building on this commit. I know we don't like big patches, but since this is semantically related to the next one and the two touch different files anyway, how about merging them? IMHO this is preferable to adding lots of #[expect(unused)] that we are going to remove right after.

If you have been told to split in a previous version, let's just add a the #[expect(unused)] where needed.

Quoted 66 lines

---

Changes for v5:
- Derive the Zeroable trait for structs and enums

Changes for v4:
- Don't panic the kernel if trying to initialise a large (> 4GB) message header.
- Use init! to provide safe and complete initialisers.
- Take an enum type instead of a u32 for the function.

Changes for v3:
- New for v3

---
 drivers/gpu/nova-core/gsp/fw.rs               | 275 +++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 541 ++++++++++++++++++
 2 files changed, 816 insertions(+)
drivers/gpu/nova-core/gsp/fw.rs +25 -0
index 1cc992ca492c..a2ce570ddfaf 100644
@@ -5,6 +5,7 @@
 // Alias to avoid repeating the version number with every use.
 use r570_144 as bindings;
 
+use core::fmt;
 use core::ops::Range;
 
 use kernel::dma::CoherentAllocation;
@@ -16,6 +17,7 @@
 use crate::firmware::gsp::GspFirmware;
 use crate::gpu::Chipset;
 use crate::gsp::FbLayout;
+use crate::gsp::GSP_PAGE_SIZE;
 
 /// Dummy type to group methods related to heap parameters for running the GSP firmware.
 pub(crate) struct GspFwHeapParams(());
@@ -158,6 +160,120 @@ pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
     }
 }
 
+#[derive(PartialEq)]
+pub(crate) enum MsgFunction {
+    // Common function codes
+    Nop = bindings::NV_VGPU_MSG_FUNCTION_NOP as isize,
+    SetGuestSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO as isize,
+    AllocRoot = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT as isize,
+    AllocDevice = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE as isize,
+    AllocMemory = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY as isize,
+    AllocCtxDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA as isize,
+    AllocChannelDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA as isize,
+    MapMemory = bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY as isize,
+    BindCtxDma = bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA as isize,
+    AllocObject = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT as isize,
+    Free = bindings::NV_VGPU_MSG_FUNCTION_FREE as isize,
+    Log = bindings::NV_VGPU_MSG_FUNCTION_LOG as isize,
+    GetGspStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO as isize,
+    SetRegistry = bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY as isize,
+    GspSetSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO as isize,
+    GspInitPostObjGpu = bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU as isize,
+    GspRmControl = bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL as isize,
+    GetStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO as isize,
+
+    // Event codes

drivers/gpu/nova-core/gsp/fw.rs pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self { · line 195

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+    GspInitDone = bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE as isize,
+    GspRunCpuSequencer = bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER as isize,
+    PostEvent = bindings::NV_VGPU_MSG_EVENT_POST_EVENT as isize,
+    RcTriggered = bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED as isize,
+    MmuFaultQueued = bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED as isize,
+    OsErrorLog = bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG as isize,
+    GspPostNoCat = bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD as isize,
+    GspLockdownNotice = bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE as isize,
+    UcodeLibOsPrint = bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT as isize,
+}

Can we split functions and events into distinct types? Their bindings even have different prefixes.

Also, rather than casting these all as isize, I'd suggest making the type #[repr(u32)]. We don't need negative values here.

Quoted 70 lines
drivers/gpu/nova-core/gsp/fw.rs +70 -0
+
+impl fmt::Display for MsgFunction {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            // Common function codes
+            MsgFunction::Nop => write!(f, "NOP"),
+            MsgFunction::SetGuestSystemInfo => write!(f, "SET_GUEST_SYSTEM_INFO"),
+            MsgFunction::AllocRoot => write!(f, "ALLOC_ROOT"),
+            MsgFunction::AllocDevice => write!(f, "ALLOC_DEVICE"),
+            MsgFunction::AllocMemory => write!(f, "ALLOC_MEMORY"),
+            MsgFunction::AllocCtxDma => write!(f, "ALLOC_CTX_DMA"),
+            MsgFunction::AllocChannelDma => write!(f, "ALLOC_CHANNEL_DMA"),
+            MsgFunction::MapMemory => write!(f, "MAP_MEMORY"),
+            MsgFunction::BindCtxDma => write!(f, "BIND_CTX_DMA"),
+            MsgFunction::AllocObject => write!(f, "ALLOC_OBJECT"),
+            MsgFunction::Free => write!(f, "FREE"),
+            MsgFunction::Log => write!(f, "LOG"),
+            MsgFunction::GetGspStaticInfo => write!(f, "GET_GSP_STATIC_INFO"),
+            MsgFunction::SetRegistry => write!(f, "SET_REGISTRY"),
+            MsgFunction::GspSetSystemInfo => write!(f, "GSP_SET_SYSTEM_INFO"),
+            MsgFunction::GspInitPostObjGpu => write!(f, "GSP_INIT_POST_OBJGPU"),
+            MsgFunction::GspRmControl => write!(f, "GSP_RM_CONTROL"),
+            MsgFunction::GetStaticInfo => write!(f, "GET_STATIC_INFO"),
+
+            // Event codes
+            MsgFunction::GspInitDone => write!(f, "INIT_DONE"),
+            MsgFunction::GspRunCpuSequencer => write!(f, "RUN_CPU_SEQUENCER"),
+            MsgFunction::PostEvent => write!(f, "POST_EVENT"),
+            MsgFunction::RcTriggered => write!(f, "RC_TRIGGERED"),
+            MsgFunction::MmuFaultQueued => write!(f, "MMU_FAULT_QUEUED"),
+            MsgFunction::OsErrorLog => write!(f, "OS_ERROR_LOG"),
+            MsgFunction::GspPostNoCat => write!(f, "NOCAT"),
+            MsgFunction::GspLockdownNotice => write!(f, "LOCKDOWN_NOTICE"),
+            MsgFunction::UcodeLibOsPrint => write!(f, "LIBOS_PRINT"),
+        }
+    }
+}
+
+impl TryFrom<u32> for MsgFunction {
+    type Error = kernel::error::Error;
+
+    fn try_from(value: u32) -> Result<MsgFunction> {
+        match value {
+            bindings::NV_VGPU_MSG_FUNCTION_NOP => Ok(MsgFunction::Nop),
+            bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO => {
+                Ok(MsgFunction::SetGuestSystemInfo)
+            }
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT => Ok(MsgFunction::AllocRoot),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE => Ok(MsgFunction::AllocDevice),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY => Ok(MsgFunction::AllocMemory),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA => Ok(MsgFunction::AllocCtxDma),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA => Ok(MsgFunction::AllocChannelDma),
+            bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY => Ok(MsgFunction::MapMemory),
+            bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA => Ok(MsgFunction::BindCtxDma),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT => Ok(MsgFunction::AllocObject),
+            bindings::NV_VGPU_MSG_FUNCTION_FREE => Ok(MsgFunction::Free),
+            bindings::NV_VGPU_MSG_FUNCTION_LOG => Ok(MsgFunction::Log),
+            bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO => Ok(MsgFunction::GetGspStaticInfo),
+            bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY => Ok(MsgFunction::SetRegistry),
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO => Ok(MsgFunction::GspSetSystemInfo),
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU => {
+                Ok(MsgFunction::GspInitPostObjGpu)
+            }
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL => Ok(MsgFunction::GspRmControl),
+            bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO => Ok(MsgFunction::GetStaticInfo),
+            bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE => Ok(MsgFunction::GspInitDone),
+            bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER => {
+                Ok(MsgFunction::GspRunCpuSequencer)
+            }
+            bindings::NV_VGPU_MSG_EVENT_POST_EVENT => Ok(MsgFunction::PostEvent),

drivers/gpu/nova-core/gsp/fw.rs pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self { · line 275

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+            bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED => Ok(MsgFunction::RcTriggered),
+            bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED => Ok(MsgFunction::MmuFaultQueued),
+            bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG => Ok(MsgFunction::OsErrorLog),
+            bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD => Ok(MsgFunction::GspPostNoCat),
+            bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE => Ok(MsgFunction::GspLockdownNotice),
+            bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT => Ok(MsgFunction::UcodeLibOsPrint),
+            _ => Err(EINVAL),
+        }
+    }
+}

This is another place where the TryFrom macro will shine. :)

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 329

Quoted 11 lines
drivers/gpu/nova-core/gsp/fw.rs +4 -0
+
 /// Struct containing the arguments required to pass a memory buffer to the GSP
 /// for use during initialisation.
 ///
@@ -208,3 +324,162 @@ fn id8(name: &str) -> u64 {
         }))
     }
 }
+
+#[repr(transparent)]
+pub(crate) struct MsgqTxHeader(bindings::msgqTxHeader);

Short documentation please (MsgqRxHeader provides a template).

Quoted 23 lines
drivers/gpu/nova-core/gsp/fw.rs +23 -0
+
+impl MsgqTxHeader {
+    pub(crate) fn new(msgq_size: u32, rx_hdr_offset: u32, msg_count: u32) -> Self {
+        Self(bindings::msgqTxHeader {
+            version: 0,
+            size: msgq_size,
+            msgSize: GSP_PAGE_SIZE as u32,
+            msgCount: msg_count,
+            writePtr: 0,
+            flags: 1,
+            rxHdrOff: rx_hdr_offset,
+            entryOff: GSP_PAGE_SIZE as u32,
+        })
+    }
+
+    pub(crate) fn write_ptr(&self) -> u32 {
+        let ptr = (&self.0.writePtr) as *const u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_read! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.read_volatile() }
+    }

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 362

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+
+    pub(crate) fn set_write_ptr(&mut self, val: u32) {
+        let ptr = (&mut self.0.writePtr) as *mut u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_write! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.write_volatile(val) }
+    }
+}

These methods should also be succintly documented imho.

Quoted 21 lines
drivers/gpu/nova-core/gsp/fw.rs +21 -0
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for MsgqTxHeader {}
+
+/// RX header for setting up a message queue with the GSP.
+#[repr(transparent)]
+pub(crate) struct MsgqRxHeader(bindings::msgqRxHeader);
+
+impl MsgqRxHeader {
+    pub(crate) fn new() -> Self {
+        Self(Default::default())
+    }
+
+    pub(crate) fn read_ptr(&self) -> u32 {
+        let ptr = (&self.0.readPtr) as *const u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_read! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.read_volatile() }
+    }

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 393

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+
+    pub(crate) fn set_read_ptr(&mut self, val: u32) {
+        let ptr = (&mut self.0.readPtr) as *mut u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_write! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.write_volatile(val) }
+    }
+}

Documentation here as well.

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 399

Quoted 6 lines
drivers/gpu/nova-core/gsp/fw.rs +6 -0
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for MsgqRxHeader {}
+
+impl bindings::rpc_message_header_v {
+    pub(crate) fn init(cmd_size: u32, function: MsgFunction) -> impl Init<Self, Error> {

This method can be kept private.

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 403

Quoted 4 lines
drivers/gpu/nova-core/gsp/fw.rs +4 -0
+        type RpcMessageHeader = bindings::rpc_message_header_v;
+        try_init!(RpcMessageHeader {
+            // TODO: magic number
+            header_version: 0x03000000,

Let's resolve this TODO by declaring a constant or even better, adding the relevant bindings symbol if there is one.

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 405

Quoted 2 lines
drivers/gpu/nova-core/gsp/fw.rs +2 -0
+            signature: bindings::NV_VGPU_MSG_SIGNATURE_VALID,
+            function: function as u32,

This as is ok once we set MsgFunction to have a u32 representation, but...

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 406

Quoted 1 line
drivers/gpu/nova-core/gsp/fw.rs +1 -0
+            length: (size_of::<Self>() as u32)

... at the risk of sounding too pedantic, we probably want to do the arithmetic on the usize, and then do a try_into u32 to remove this use of as here:

    length: size_of::<Self>()
        .checked_add(cmd_size)
        .ok_or(EOVERFLOW)
        .and_then(|v| v.try_into().map_err(|_| EINVAL))?,

This also requires changing the cmd_size to usize, which is actually better as that's also the type used by the caller anyway, so this removes another conversion.

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+                .checked_add(cmd_size)
+                .ok_or(EOVERFLOW)?,
+            rpc_result: 0xffffffff,
+            rpc_result_private: 0xffffffff,
+            ..Zeroable::init_zeroed()
+        })
+    }
+}
+
+// SAFETY: We can't derive the Zeroable trait for this binding because the

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 426

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+// procedural macro doesn't support the syntax used by bindgen to create the
+// __IncompleteArrayField types. So instead we implement it here, which is safe
+// because these are explicitly padded structures only containing types for
+// which any bit pattern, including all zeros, is valid.
+unsafe impl Zeroable for bindings::rpc_message_header_v {}
+
+#[repr(transparent)]
+pub(crate) struct GspMsgElement {
+    inner: bindings::GSP_MSG_QUEUE_ELEMENT,
+}

Small documentation needed.

Quoted 6 lines
drivers/gpu/nova-core/gsp/fw.rs +6 -0
+
+impl GspMsgElement {
+    #[allow(non_snake_case)]
+    pub(crate) fn init(
+        sequence: u32,
+        cmd_size: usize,

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 442

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+        function: MsgFunction,
+    ) -> impl Init<Self, Error> {
+        type RpcMessageHeader = bindings::rpc_message_header_v;
+        type InnerGspMsgElement = bindings::GSP_MSG_QUEUE_ELEMENT;
+        let init_inner = try_init!(InnerGspMsgElement {
+            seqNum: sequence,
+            elemCount: size_of::<Self>()
+                .checked_add(cmd_size)
+                .ok_or(EOVERFLOW)?
+                .div_ceil(GSP_PAGE_SIZE) as u32,

Let's chain a call to try_into here as well.

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 443

Quoted 1 line
drivers/gpu/nova-core/gsp/fw.rs +1 -0
+            rpc <- RpcMessageHeader::init(cmd_size as u32, function),

... and thanks to the change suggested 3 blocks above, this as is no longer needed.

Quoted 8 lines
drivers/gpu/nova-core/gsp/fw.rs +8 -0
+            ..Zeroable::init_zeroed()
+        });
+
+        try_init!(GspMsgElement {
+            inner <- init_inner,
+        })
+    }
+

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 461

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+    pub(crate) fn set_checksum(&mut self, checksum: u32) {
+        self.inner.checkSum = checksum;
+    }
+
+    // Return the total length of the message, noting that rpc.length includes
+    // the length of the GspRpcHeader but not the message header.
+    pub(crate) fn length(&self) -> u32 {
+        size_of::<Self>() as u32 - size_of::<bindings::rpc_message_header_v>() as u32
+            + self.inner.rpc.length
+    }

We can save ourselves a bunch of as conversions in the following patch if we make this method just return usize. The only one needed will be for self.inner.rpc.length to usize, we really ought to have non-fallible converters for things like this that obviously cannot fail.

Quoted 7 lines
drivers/gpu/nova-core/gsp/fw.rs +7 -0
+
+    pub(crate) fn sequence(&self) -> u32 {
+        self.inner.rpc.sequence
+    }
+
+    pub(crate) fn function_number(&self) -> u32 {
+        self.inner.rpc.function

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 478

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+    }
+
+    pub(crate) fn function(&self) -> Result<MsgFunction> {
+        self.inner.rpc.function.try_into()
+    }
+
+    pub(crate) fn element_count(&self) -> u32 {
+        self.inner.elemCount
+    }
+}

Documentation needed for these too.

#26 lore raw

Re: [PATCH v5 06/14] gpu: nova-core: Add GSP command queue bindings

Quoted 9 lines · Alexandre Courbot

There are lots of unused warnings when building on this commit. I know we don't like big patches, but since this is semantically related to the next one and the two touch different files anyway, how about merging them? IMHO this is preferable to adding lots of #[expect(unused)] that we are going to remove right after.

If you have been told to split in a previous version, let's just add a the #[expect(unused)] where needed.

A possible middle-ground is to add the expect in a "more global" position, e.g. at the top, so that it is a single line change. I think that is fine as long as it is removed right after.

Thanks!

Cheers,
Miguel

#27 lore raw

Re: [PATCH v5 06/14] gpu: nova-core: Add GSP command queue bindings

On 2025-10-17 at 06:22 +1100, Miguel Ojeda <[email protected]> wrote...

Quoted 10 lines · Alexandre Courbot
Quoted 9 lines

There are lots of unused warnings when building on this commit. I know we don't like big patches, but since this is semantically related to the next one and the two touch different files anyway, how about merging them? IMHO this is preferable to adding lots of #[expect(unused)] that we are going to remove right after.

If you have been told to split in a previous version, let's just add a the #[expect(unused)] where needed.

No, I just split it here because it seemed like a nice self-contained thing. That and I'm conditioned to linux-mm patches where 20 changes is often considered a big patch :-)

So I will probably just squash this as suggested.

Quoted 4 lines

A possible middle-ground is to add the expect in a "more global" position, e.g. at the top, so that it is a single line change. I think that is fine as long as it is removed right after.

That could work, thanks.

Quoted 4 lines

Thanks!

Cheers,
Miguel

#28 lore raw

PATCH v5 7/14 gpu: nova-core: gsp: Add GSP command queue handling

In reply to Alistair Popple

This commit introduces core infrastructure for handling GSP command and message queues in the nova-core driver. The command queue system enables bidirectional communication between the host driver and GSP firmware through a remote message passing interface.

The interface is based on passing serialised data structures over a ring buffer with separate transmit and receive queues. Commands are sent by writing to the CPU transmit queue and waiting for completion via the receive queue.

To ensure safety mutable or immutable (depending on whether it is a send or receive operation) references are taken on the command queue when allocating the message to write/read to. This ensures message memory remains valid and the command queue can't be mutated whilst an operation is in progress.

Currently this is only used by the probe() routine and therefore can only used by a single thread of execution. Locking to enable safe access from multiple threads will be introduced in a future series when that becomes necessary.

---

Changes for v4:
- Use read_poll_timeout() instead of wait_on()
- Switch to using init! (Thanks Alex for figuring out the required workarounds)
- Pass the enum type into the RPC bindings instead of a raw u32
- Fixup the TODOs for extracting/allocating the send command regions
- Split the sending functions into one taking just a command struct and another taking a command struct with extra payload

Changes for v3:
- Reduce the receive payloads to the correct size
- Use opaque bindings
- Clean up of the command queue PTE creation
- Add an enum for the GSP functions
- Rename GspCommandToGsp and GspMessageFromGsp
- Rename GspCmdq
- Add function to notify GSP of updated queue pointers
- Inline driver area access functions
- Fixup receive area calculations

Changes for v2:
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gsp.rs      |   9 +
 drivers/gpu/nova-core/gsp/cmdq.rs | 493 ++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs     |   4 +
 drivers/gpu/nova-core/sbuffer.rs  |   2 -
 scripts/Makefile.build            |   2 +-
 5 files changed, 507 insertions(+), 3 deletions(-)
 create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs
drivers/gpu/nova-core/gsp.rs +9 -0
index 554eb1a34ee7..1d472c5fad7a 100644
@@ -2,6 +2,7 @@
 
 mod boot;
 
+use kernel::alloc::flags::GFP_KERNEL;
 use kernel::device;
 use kernel::dma::CoherentAllocation;
 use kernel::dma::DmaAddress;
@@ -11,6 +12,7 @@
 use kernel::transmute::AsBytes;
 
 use crate::fb::FbLayout;
+use crate::gsp::cmdq::Cmdq;
 
 pub(crate) use fw::{GspFwWprMeta, LibosParams};
 
@@ -18,6 +20,8 @@
 
 use fw::LibosMemoryRegionInitArgument;
 
+pub(crate) mod cmdq;
+
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
 pub(crate) const GSP_PAGE_SIZE: usize = 1 << GSP_PAGE_SHIFT;
 
@@ -31,6 +35,7 @@ pub(crate) struct Gsp {
     loginit: LogBuffer,
     logintr: LogBuffer,
     logrm: LogBuffer,
+    pub(crate) cmdq: Cmdq,
 }
 
 #[repr(C)]
@@ -110,11 +115,15 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self
         let logrm = LogBuffer::new(dev)?;
         dma_write!(libos[2] = LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0)?)?;
 
+        // Creates its own PTE array.
+        let cmdq = Cmdq::new(dev)?;
+
         Ok(try_pin_init!(Self {
             libos,
             loginit,
             logintr,
             logrm,
+            cmdq,
         }))
     }
 }
drivers/gpu/nova-core/gsp/cmdq.rs new file +493 -0

493 lines not on this page · raw message

drivers/gpu/nova-core/regs.rs +4 -0
index 206dab2e1335..0585699ae951 100644
@@ -71,6 +71,10 @@ pub(crate) fn chipset(self) -> Result<Chipset> {
     30:30   ecc_mode_enabled as bool;
 });
 
+register!(NV_PGSP_QUEUE_HEAD @ 0x00110c00 {
+    31:0    address as u32;
+});
+
 impl NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE {
     /// Returns the usable framebuffer size, in bytes.
     pub(crate) fn usable_fb_size(self) -> u64 {
drivers/gpu/nova-core/sbuffer.rs +0 -2
index d9c412a68bd8..1a27226b65d8 100644
@@ -50,7 +50,6 @@ impl<'a, I> SBufferIter<I>
     /// let sum: u8 = sbuffer.sum();
     /// assert_eq!(sum, 45);
     /// ```
-    #[expect(unused)]
     pub(crate) fn new_reader(slices: impl IntoIterator<IntoIter = I>) -> Self
     where
         I: Iterator<Item = &'a [u8]>,
@@ -72,7 +71,6 @@ pub(crate) fn new_reader(slices: impl IntoIterator<IntoIter = I>) -> Self
     /// assert_eq!(buf2, [5, 6, 7, 8, 9]);
     ///
     /// ```
-    #[expect(unused)]
     pub(crate) fn new_writer(slices: impl IntoIterator<IntoIter = I>) -> Self
     where
         I: Iterator<Item = &'a mut [u8]>,
scripts/Makefile.build +1 -1
index d0ee33a487be..4ac6304332b6 100644
@@ -317,7 +317,7 @@ $(obj)/%.lst: $(obj)/%.c FORCE
 #
 # Please see https://github.com/Rust-for-Linux/linux/issues/2 for details on
 # the unstable features in use.
-rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg
+rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg,slice_flatten
 
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
Signature · 1 line
2.50.1
#29 lore raw

Re: [PATCH v5 07/14] gpu: nova-core: gsp: Add GSP command queue handling

Quoted 61 lines · Alistair Popple

This commit introduces core infrastructure for handling GSP command and message queues in the nova-core driver. The command queue system enables bidirectional communication between the host driver and GSP firmware through a remote message passing interface.

The interface is based on passing serialised data structures over a ring buffer with separate transmit and receive queues. Commands are sent by writing to the CPU transmit queue and waiting for completion via the receive queue.

To ensure safety mutable or immutable (depending on whether it is a send or receive operation) references are taken on the command queue when allocating the message to write/read to. This ensures message memory remains valid and the command queue can't be mutated whilst an operation is in progress.

Currently this is only used by the probe() routine and therefore can only used by a single thread of execution. Locking to enable safe access from multiple threads will be introduced in a future series when that becomes necessary.

---

Changes for v4:
- Use read_poll_timeout() instead of wait_on()
- Switch to using init! (Thanks Alex for figuring out the required workarounds)
- Pass the enum type into the RPC bindings instead of a raw u32
- Fixup the TODOs for extracting/allocating the send command regions
- Split the sending functions into one taking just a command struct and another taking a command struct with extra payload

Changes for v3:
- Reduce the receive payloads to the correct size
- Use opaque bindings
- Clean up of the command queue PTE creation
- Add an enum for the GSP functions
- Rename GspCommandToGsp and GspMessageFromGsp
- Rename GspCmdq
- Add function to notify GSP of updated queue pointers
- Inline driver area access functions
- Fixup receive area calculations

Changes for v2:
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gsp.rs      |   9 +
 drivers/gpu/nova-core/gsp/cmdq.rs | 493 ++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs     |   4 +
 drivers/gpu/nova-core/sbuffer.rs  |   2 -
 scripts/Makefile.build            |   2 +-
 5 files changed, 507 insertions(+), 3 deletions(-)
 create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs
drivers/gpu/nova-core/gsp.rs +0 -0
index 554eb1a34ee7..1d472c5fad7a 100644

drivers/gpu/nova-core/gsp.rs line 5

Quoted 4 lines
drivers/gpu/nova-core/gsp.rs +1 -0
@@ -2,6 +2,7 @@
 
 mod boot;
 
+use kernel::alloc::flags::GFP_KERNEL;

This import should not be needed, GFP_KERNEL being part of the prelude.

Quoted 29 lines
drivers/gpu/nova-core/gsp.rs +4 -0
 use kernel::device;
 use kernel::dma::CoherentAllocation;
 use kernel::dma::DmaAddress;
@@ -11,6 +12,7 @@
 use kernel::transmute::AsBytes;
 
 use crate::fb::FbLayout;
+use crate::gsp::cmdq::Cmdq;
 
 pub(crate) use fw::{GspFwWprMeta, LibosParams};
 
@@ -18,6 +20,8 @@
 
 use fw::LibosMemoryRegionInitArgument;
 
+pub(crate) mod cmdq;
+
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
 pub(crate) const GSP_PAGE_SIZE: usize = 1 << GSP_PAGE_SHIFT;
 
@@ -31,6 +35,7 @@ pub(crate) struct Gsp {
     loginit: LogBuffer,
     logintr: LogBuffer,
     logrm: LogBuffer,
+    pub(crate) cmdq: Cmdq,
 }
 
 #[repr(C)]

drivers/gpu/nova-core/gsp.rs pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self · line 119

Quoted 5 lines
drivers/gpu/nova-core/gsp.rs +2 -0
@@ -110,11 +115,15 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self
         let logrm = LogBuffer::new(dev)?;
         dma_write!(libos[2] = LibosMemoryRegionInitArgument::new("LOGRM", &logrm.0)?)?;
 
+        // Creates its own PTE array.
+        let cmdq = Cmdq::new(dev)?;

I believe this comment is a leftover.

Quoted 16 lines
drivers/gpu/nova-core/gsp.rs +2 -0
+
         Ok(try_pin_init!(Self {
             libos,
             loginit,
             logintr,
             logrm,
+            cmdq,
         }))
     }
 }
drivers/gpu/nova-core/gsp/cmdq.rs new file +0 -0
new file mode 100644
index 000000000000..3f8cb7a35922

drivers/gpu/nova-core/gsp/cmdq.rs line 7

Quoted 7 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +7 -0
@@ -0,0 +1,493 @@
+// SPDX-License-Identifier: GPL-2.0
+
+use core::mem::offset_of;
+use core::sync::atomic::fence;
+use core::sync::atomic::Ordering;
+
+use kernel::alloc::flags::GFP_KERNEL;

This is one not needed either.

Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +4 -0
+use kernel::device;
+use kernel::dma::CoherentAllocation;
+use kernel::dma_write;
+use kernel::io::poll::read_poll_timeout;

drivers/gpu/nova-core/gsp/cmdq.rs line 21

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10 -0
+use kernel::prelude::*;
+use kernel::sync::aref::ARef;
+use kernel::time::Delta;
+use kernel::transmute::{AsBytes, FromBytes};
+
+use crate::driver::Bar0;
+use crate::gsp::fw::{GspMsgElement, MsgFunction, MsgqRxHeader, MsgqTxHeader};
+use crate::gsp::PteArray;
+use crate::gsp::{GSP_PAGE_SHIFT, GSP_PAGE_SIZE};
+use crate::regs::NV_PGSP_QUEUE_HEAD;

The custom with registers to just import the regs module and prefix all registers with regs::.

drivers/gpu/nova-core/gsp/cmdq.rs line 28

Quoted 7 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +7 -0
+use crate::sbuffer::SBufferIter;
+
+pub(crate) trait CommandToGsp: Sized + FromBytes + AsBytes {
+    const FUNCTION: MsgFunction;
+}
+
+pub(crate) trait CommandToGspWithPayload: CommandToGsp {}

Unfortunately this hierarchy won't work, because this way a command that is expected to take a payload can be sent without any.

I'd suggest instead having a CommandToGspBase trait that sets the FUNCTION, and have CommandToGsp and CommandToGspWithPayload require it.

drivers/gpu/nova-core/gsp/cmdq.rs line 32

Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +4 -0
+
+pub(crate) trait MessageFromGsp: Sized + FromBytes + AsBytes {
+    const FUNCTION: MsgFunction;
+}

Just a bit of documentation for these new traits please. :)

(also applies to all undocumented types/methods in this file)

<snip>

Quoted 6 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +6 -0
+impl DmaGspMem {
+    fn new(dev: &device::Device<device::Bound>) -> Result<Self> {
+        const MSGQ_SIZE: u32 = size_of::<Msgq>() as u32;
+        const RX_HDR_OFF: u32 = offset_of!(Msgq, rx) as u32;
+
+        let gsp_mem =

drivers/gpu/nova-core/gsp/cmdq.rs line 90

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10 -0
+            CoherentAllocation::<GspMem>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?;
+        dma_write!(gsp_mem[0].ptes = PteArray::new(gsp_mem.dma_handle())?)?;
+        dma_write!(gsp_mem[0].cpuq.tx = MsgqTxHeader::new(MSGQ_SIZE, RX_HDR_OFF, MSGQ_NUM_PAGES))?;
+        dma_write!(gsp_mem[0].cpuq.rx = MsgqRxHeader::new())?;
+
+        Ok(Self(gsp_mem))
+    }
+
+    // Allocates the various regions for the command and reduces the payload size
+    // to match what is needed for the command.

Let's also explain what the elements in the returned tuple are.

Quoted 11 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +11 -0
+    //
+    // # Errors
+    //
+    // Returns `Err(EAGAIN)` if the driver area is too small to hold the
+    // requested command.
+    fn allocate_command_regions<'a, M: CommandToGsp>(
+        &'a mut self,
+        payload_size: usize,
+    ) -> Result<(&'a mut GspMsgElement, &'a mut M, &'a mut [u8], &'a mut [u8])> {
+        let driver_area = self.driver_write_area();
+        let msg_size = size_of::<GspMsgElement>() + size_of::<M>() + payload_size;

drivers/gpu/nova-core/gsp/cmdq.rs line 111

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10 -0
+        let driver_area_size = (driver_area.0.len() + driver_area.1.len()) << GSP_PAGE_SHIFT;
+
+        if msg_size > driver_area_size {
+            return Err(EAGAIN);
+        }
+
+        #[allow(clippy::incompatible_msrv)]
+        let (msg_header_slice, slice_1) = driver_area
+            .0
+            .as_flattened_mut()

We are going to have a slight problem here. as_flattened_mut is only available since Rust 1.80... and we need to support 1.78.

I suppose we could work this around by defining a custom function tht just calls as_flattened_mut in Rust versions that support it and provides an alternative implementation for those that don't, but I wonder whether there is a better way.

Quoted 9 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +9 -0
+            .split_at_mut(size_of::<GspMsgElement>());
+        let msg_header = GspMsgElement::from_bytes_mut(msg_header_slice).ok_or(EINVAL)?;
+        let (cmd_slice, payload_1) = slice_1.split_at_mut(size_of::<M>());
+        let cmd = M::from_bytes_mut(cmd_slice).ok_or(EINVAL)?;
+
+        #[allow(clippy::incompatible_msrv)]
+        let payload_2 = driver_area.1.as_flattened_mut();
+
+        let (payload_1, payload_2) = if payload_1.len() > payload_size {

drivers/gpu/nova-core/gsp/cmdq.rs line 130

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10 -0
+            // Payload fits entirely in payload_1
+            (&mut payload_1[..payload_size], &mut payload_2[0..0])
+        } else {
+            // Need all of payload_1 and some of payload_2
+            let payload_2_len = payload_size - payload_1.len();
+            (payload_1, &mut payload_2[..payload_2_len])
+        };
+
+        Ok((msg_header, cmd, payload_1, payload_2))
+    }

We should probably add a few comments in this function to explain the steps, otherwise it can be a bit difficult to parse.

drivers/gpu/nova-core/gsp/cmdq.rs line 279

Quoted 13 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +13 -0
+    fn calculate_checksum<T: Iterator<Item = u8>>(it: T) -> u32 {
+        let sum64 = it
+            .enumerate()
+            .map(|(idx, byte)| (((idx % 8) * 8) as u32, byte))
+            .fold(0, |acc, (rol, byte)| acc ^ u64::from(byte).rotate_left(rol));
+
+        ((sum64 >> 32) as u32) ^ (sum64 as u32)
+    }
+
+    // Notify GSP that we have updated the command queue pointers.
+    fn notify_gsp(bar: &Bar0) {
+        NV_PGSP_QUEUE_HEAD::default().set_address(0).write(bar);
+    }

Can we move this method to regs.rs, as an impl block of NV_PGSP_QUEUE_HEAD? It makes more sense to have it there imho.

Quoted 109 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +109 -0
+
+    #[expect(unused)]
+    pub(crate) fn send_gsp_command<M, E>(&mut self, bar: &Bar0, init: impl Init<M, E>) -> Result
+    where
+        M: CommandToGsp,
+        // This allows all error types, including `Infallible`, to be used with `init`. Without
+        // this we cannot use regular stack objects as `init` since their `Init` implementation
+        // does not return any error.
+        Error: From<E>,
+    {
+        #[repr(C)]
+        struct FullCommand<M> {
+            hdr: GspMsgElement,
+            cmd: M,
+        }
+        let (msg_header, cmd, _, _) = self.gsp_mem.allocate_command_regions::<M>(0)?;
+
+        let seq = self.seq;
+        let initializer = try_init!(FullCommand {
+            hdr <- GspMsgElement::init(seq, size_of::<M>(), M::FUNCTION),
+            cmd <- init,
+        });
+
+        // Fill the header and command in-place.
+        // SAFETY:
+        //  - allocate_command_regions guarantees msg_header points to enough
+        //    space in the command queue to contain FullCommand
+        //  - __init ensures the command header and struct a fully initialized
+        unsafe {
+            initializer.__init(msg_header.as_bytes_mut().as_mut_ptr().cast())?;
+        }
+
+        msg_header.set_checksum(Cmdq::calculate_checksum(SBufferIter::new_reader([
+            msg_header.as_bytes(),
+            cmd.as_bytes(),
+        ])));
+
+        dev_info!(
+            &self.dev,
+            "GSP RPC: send: seq# {}, function=0x{:x} ({}), length=0x{:x}\n",
+            self.seq,
+            msg_header.function_number(),
+            msg_header.function()?,
+            msg_header.length(),
+        );
+
+        let elem_count = msg_header.element_count();
+        self.seq += 1;
+        self.gsp_mem.advance_cpu_write_ptr(elem_count);
+        Cmdq::notify_gsp(bar);
+
+        Ok(())
+    }
+
+    #[expect(unused)]
+    pub(crate) fn send_gsp_command_with_payload<M, E>(
+        &mut self,
+        bar: &Bar0,
+        payload_size: usize,
+        init: impl Init<M, E>,
+        init_payload: impl FnOnce(SBufferIter<core::array::IntoIter<&mut [u8], 2>>) -> Result,
+    ) -> Result
+    where
+        M: CommandToGspWithPayload,
+        // This allows all error types, including `Infallible`, to be used with `init`. Without
+        // this we cannot use regular stack objects as `init` since their `Init` implementation
+        // does not return any error.
+        Error: From<E>,
+    {
+        #[repr(C)]
+        struct FullCommand<M> {
+            hdr: GspMsgElement,
+            cmd: M,
+        }
+        let (msg_header, cmd, payload_1, payload_2) =
+            self.gsp_mem.allocate_command_regions::<M>(payload_size)?;
+
+        let seq = self.seq;
+        let initializer = try_init!(FullCommand {
+            hdr <- GspMsgElement::init(seq, size_of::<M>() + payload_size, M::FUNCTION),
+            cmd <- init,
+        });
+
+        // Fill the header and command in-place.
+        // SAFETY:
+        //  - allocate_command_regions guarantees msg_header points to enough
+        //    space in the command queue to contain FullCommand
+        //  - __init ensures the command header and struct a fully initialized
+        unsafe {
+            initializer.__init(msg_header.as_bytes_mut().as_mut_ptr().cast())?;
+        }
+
+        // Fill the payload
+        let sbuffer = SBufferIter::new_writer([&mut payload_1[..], &mut payload_2[..]]);
+        init_payload(sbuffer)?;
+
+        msg_header.set_checksum(Cmdq::calculate_checksum(SBufferIter::new_reader([
+            msg_header.as_bytes(),
+            cmd.as_bytes(),
+            payload_1,
+            payload_2,
+        ])));
+
+        dev_info!(
+            &self.dev,
+            "GSP RPC: send: seq# {}, function=0x{:x} ({}), length=0x{:x}\n",
+            self.seq,
+            msg_header.function_number(),
+            msg_header.function()?,

drivers/gpu/nova-core/gsp/cmdq.rs line 398

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10 -0
+            msg_header.length(),
+        );
+
+        let elem_count = msg_header.element_count();
+        self.seq += 1;
+        self.gsp_mem.advance_cpu_write_ptr(elem_count);
+        Cmdq::notify_gsp(bar);
+
+        Ok(())
+    }

I think we can factorize most of send_gsp_command and send_gsp_command_with_payload into a private method that requires a CommandToGspBase and takes the same parameters as send_gsp_command_with_payload. All it would need to do when called from send_gsp_command is take a closure that doesn't write any payload.

Quoted 84 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +84 -0
+
+    #[expect(unused)]
+    pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>(
+        &mut self,
+        timeout: Delta,
+        init: impl FnOnce(&M, SBufferIter<core::array::IntoIter<&[u8], 2>>) -> Result<R>,
+    ) -> Result<R> {
+        let driver_area = read_poll_timeout(
+            || Ok(self.gsp_mem.driver_read_area()),
+            |driver_area: &(&[[u8; 4096]], &[[u8; 4096]])| !driver_area.0.is_empty(),
+            Delta::from_millis(10),
+            timeout,
+        )?;
+
+        #[allow(clippy::incompatible_msrv)]
+        let (msg_header_slice, slice_1) = driver_area
+            .0
+            .as_flattened()
+            .split_at(size_of::<GspMsgElement>());
+        let msg_header = GspMsgElement::from_bytes(msg_header_slice).ok_or(EIO)?;
+        if msg_header.length() < size_of::<M>() as u32 {
+            return Err(EIO);
+        }
+
+        let function: MsgFunction = msg_header.function().map_err(|_| {
+            dev_info!(
+                self.dev,
+                "GSP RPC: receive: seq# {}, bad function=0x{:x}, length=0x{:x}\n",
+                msg_header.sequence(),
+                msg_header.function_number(),
+                msg_header.length(),
+            );
+            EIO
+        })?;
+
+        // Log RPC receive with message type decoding
+        dev_info!(
+            self.dev,
+            "GSP RPC: receive: seq# {}, function=0x{:x} ({}), length=0x{:x}\n",
+            msg_header.sequence(),
+            msg_header.function_number(),
+            function,
+            msg_header.length(),
+        );
+
+        let (cmd_slice, payload_1) = slice_1.split_at(size_of::<M>());
+        #[allow(clippy::incompatible_msrv)]
+        let payload_2 = driver_area.1.as_flattened();
+
+        // Cut the payload slice(s) down to the actual length of the payload.
+        let (cmd_payload_1, cmd_payload_2) =
+            if payload_1.len() > msg_header.length() as usize - size_of::<M>() {
+                (
+                    payload_1
+                        .split_at(msg_header.length() as usize - size_of::<M>())
+                        .0,
+                    &payload_2[0..0],
+                )
+            } else {
+                (
+                    payload_1,
+                    payload_2
+                        .split_at(msg_header.length() as usize - size_of::<M>() - payload_1.len())
+                        .0,
+                )
+            };
+
+        if Cmdq::calculate_checksum(SBufferIter::new_reader([
+            msg_header.as_bytes(),
+            cmd_slice,
+            cmd_payload_1,
+            cmd_payload_2,
+        ])) != 0
+        {
+            dev_err!(
+                self.dev,
+                "GSP RPC: receive: Call {} - bad checksum",
+                msg_header.sequence()
+            );
+            return Err(EIO);
+        }
+
+        let result = if function == M::FUNCTION {
+            let cmd = M::from_bytes(cmd_slice).ok_or(EINVAL)?;

drivers/gpu/nova-core/gsp/cmdq.rs line 492

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10 -0
+            let sbuffer = SBufferIter::new_reader([cmd_payload_1, cmd_payload_2]);
+            init(cmd, sbuffer)
+        } else {
+            Err(ERANGE)
+        };
+
+        self.gsp_mem
+            .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32));
+        result
+    }

Here we probably also want to comment the different steps a bit.

#30 lore raw

Re: [PATCH v5 07/14] gpu: nova-core: gsp: Add GSP command queue handling

On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 57 lines · Alistair Popple
Quoted 56 lines

This commit introduces core infrastructure for handling GSP command and message queues in the nova-core driver. The command queue system enables bidirectional communication between the host driver and GSP firmware through a remote message passing interface.

The interface is based on passing serialised data structures over a ring buffer with separate transmit and receive queues. Commands are sent by writing to the CPU transmit queue and waiting for completion via the receive queue.

To ensure safety mutable or immutable (depending on whether it is a send or receive operation) references are taken on the command queue when allocating the message to write/read to. This ensures message memory remains valid and the command queue can't be mutated whilst an operation is in progress.

Currently this is only used by the probe() routine and therefore can only used by a single thread of execution. Locking to enable safe access from multiple threads will be introduced in a future series when that becomes necessary.

---

Changes for v4:
- Use read_poll_timeout() instead of wait_on()
- Switch to using init! (Thanks Alex for figuring out the required workarounds)
- Pass the enum type into the RPC bindings instead of a raw u32
- Fixup the TODOs for extracting/allocating the send command regions
- Split the sending functions into one taking just a command struct and another taking a command struct with extra payload

Changes for v3:
- Reduce the receive payloads to the correct size
- Use opaque bindings
- Clean up of the command queue PTE creation
- Add an enum for the GSP functions
- Rename GspCommandToGsp and GspMessageFromGsp
- Rename GspCmdq
- Add function to notify GSP of updated queue pointers
- Inline driver area access functions
- Fixup receive area calculations

Changes for v2:
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gsp.rs      |   9 +
 drivers/gpu/nova-core/gsp/cmdq.rs | 493 ++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs     |   4 +
 drivers/gpu/nova-core/sbuffer.rs  |   2 -
 scripts/Makefile.build            |   2 +-
 5 files changed, 507 insertions(+), 3 deletions(-)
 create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs

[...]

Quoted 11 lines

drivers/gpu/nova-core/gsp/cmdq.rs line 28

Quoted 7 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +7 -0
+use crate::sbuffer::SBufferIter;
+
+pub(crate) trait CommandToGsp: Sized + FromBytes + AsBytes {
+    const FUNCTION: MsgFunction;
+}
+
+pub(crate) trait CommandToGspWithPayload: CommandToGsp {}

Unfortunately this hierarchy won't work, because this way a command that is expected to take a payload can be sent without any.

Quoted 3 lines

I'd suggest instead having a CommandToGspBase trait that sets the FUNCTION, and have CommandToGsp and CommandToGspWithPayload require it.

Ok.

Quoted 8 lines

drivers/gpu/nova-core/gsp/cmdq.rs line 32

Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +4 -0
+
+pub(crate) trait MessageFromGsp: Sized + FromBytes + AsBytes {
+    const FUNCTION: MsgFunction;
+}

Just a bit of documentation for these new traits please. :)

(also applies to all undocumented types/methods in this file)

Sure.

Quoted 45 lines

<snip>

Quoted 6 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +6 -0
+impl DmaGspMem {
+    fn new(dev: &device::Device<device::Bound>) -> Result<Self> {
+        const MSGQ_SIZE: u32 = size_of::<Msgq>() as u32;
+        const RX_HDR_OFF: u32 = offset_of!(Msgq, rx) as u32;
+
+        let gsp_mem =

drivers/gpu/nova-core/gsp/cmdq.rs line 90

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10 -0
+            CoherentAllocation::<GspMem>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?;
+        dma_write!(gsp_mem[0].ptes = PteArray::new(gsp_mem.dma_handle())?)?;
+        dma_write!(gsp_mem[0].cpuq.tx = MsgqTxHeader::new(MSGQ_SIZE, RX_HDR_OFF, MSGQ_NUM_PAGES))?;
+        dma_write!(gsp_mem[0].cpuq.rx = MsgqRxHeader::new())?;
+
+        Ok(Self(gsp_mem))
+    }
+
+    // Allocates the various regions for the command and reduces the payload size
+    // to match what is needed for the command.

Let's also explain what the elements in the returned tuple are.

Quoted 11 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +11 -0
+    //
+    // # Errors
+    //
+    // Returns `Err(EAGAIN)` if the driver area is too small to hold the
+    // requested command.
+    fn allocate_command_regions<'a, M: CommandToGsp>(
+        &'a mut self,
+        payload_size: usize,
+    ) -> Result<(&'a mut GspMsgElement, &'a mut M, &'a mut [u8], &'a mut [u8])> {
+        let driver_area = self.driver_write_area();
+        let msg_size = size_of::<GspMsgElement>() + size_of::<M>() + payload_size;

drivers/gpu/nova-core/gsp/cmdq.rs line 111

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +10 -0
+        let driver_area_size = (driver_area.0.len() + driver_area.1.len()) << GSP_PAGE_SHIFT;
+
+        if msg_size > driver_area_size {
+            return Err(EAGAIN);
+        }
+
+        #[allow(clippy::incompatible_msrv)]
+        let (msg_header_slice, slice_1) = driver_area
+            .0
+            .as_flattened_mut()

We are going to have a slight problem here. as_flattened_mut is only available since Rust 1.80... and we need to support 1.78.

Quoted 4 lines

I suppose we could work this around by defining a custom function tht just calls as_flattened_mut in Rust versions that support it and provides an alternative implementation for those that don't, but I wonder whether there is a better way.

Oh I didn't realise the function name had changed, I thought we just needed to allow the experimental feature in older versions. I probably wont get to this this week so have left it as a TODO for now.

Quoted 14 lines
Quoted 14 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +14 -0
+            .split_at_mut(size_of::<GspMsgElement>());
+        let msg_header = GspMsgElement::from_bytes_mut(msg_header_slice).ok_or(EINVAL)?;
+        let (cmd_slice, payload_1) = slice_1.split_at_mut(size_of::<M>());
+        let cmd = M::from_bytes_mut(cmd_slice).ok_or(EINVAL)?;
+
+        #[allow(clippy::incompatible_msrv)]
+        let payload_2 = driver_area.1.as_flattened_mut();
+
+        let (payload_1, payload_2) = if payload_1.len() > payload_size {
+            // Payload fits entirely in payload_1
+            (&mut payload_1[..payload_size], &mut payload_2[0..0])
+        } else {
+            // Need all of payload_1 and some of payload_2
+            let payload_2_len = payload_size - payload_1.len();
Quoted 8 lines

drivers/gpu/nova-core/gsp/cmdq.rs line 130

Quoted 5 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +5 -0
+            (payload_1, &mut payload_2[..payload_2_len])
+        };
+
+        Ok((msg_header, cmd, payload_1, payload_2))
+    }

We should probably add a few comments in this function to explain the steps, otherwise it can be a bit difficult to parse.

Ok.

Quoted 8 lines
Quoted 8 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +8 -0
+    fn calculate_checksum<T: Iterator<Item = u8>>(it: T) -> u32 {
+        let sum64 = it
+            .enumerate()
+            .map(|(idx, byte)| (((idx % 8) * 8) as u32, byte))
+            .fold(0, |acc, (rol, byte)| acc ^ u64::from(byte).rotate_left(rol));
+
+        ((sum64 >> 32) as u32) ^ (sum64 as u32)
+    }
Quoted 8 lines

drivers/gpu/nova-core/gsp/cmdq.rs line 279

Quoted 5 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +5 -0
+
+    // Notify GSP that we have updated the command queue pointers.
+    fn notify_gsp(bar: &Bar0) {
+        NV_PGSP_QUEUE_HEAD::default().set_address(0).write(bar);
+    }

Can we move this method to regs.rs, as an impl block of NV_PGSP_QUEUE_HEAD? It makes more sense to have it there imho.

We could, but I think it belongs here. If I'm understanding things correctly the value of this register is FW/SW defined. For now GSP doesn't look at the value so it doesn't really matter what we write, but it's possible that could change which means we want it here in the code that talks to GSP.

Quoted 118 lines
Quoted 118 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +118 -0
+
+    #[expect(unused)]
+    pub(crate) fn send_gsp_command<M, E>(&mut self, bar: &Bar0, init: impl Init<M, E>) -> Result
+    where
+        M: CommandToGsp,
+        // This allows all error types, including `Infallible`, to be used with `init`. Without
+        // this we cannot use regular stack objects as `init` since their `Init` implementation
+        // does not return any error.
+        Error: From<E>,
+    {
+        #[repr(C)]
+        struct FullCommand<M> {
+            hdr: GspMsgElement,
+            cmd: M,
+        }
+        let (msg_header, cmd, _, _) = self.gsp_mem.allocate_command_regions::<M>(0)?;
+
+        let seq = self.seq;
+        let initializer = try_init!(FullCommand {
+            hdr <- GspMsgElement::init(seq, size_of::<M>(), M::FUNCTION),
+            cmd <- init,
+        });
+
+        // Fill the header and command in-place.
+        // SAFETY:
+        //  - allocate_command_regions guarantees msg_header points to enough
+        //    space in the command queue to contain FullCommand
+        //  - __init ensures the command header and struct a fully initialized
+        unsafe {
+            initializer.__init(msg_header.as_bytes_mut().as_mut_ptr().cast())?;
+        }
+
+        msg_header.set_checksum(Cmdq::calculate_checksum(SBufferIter::new_reader([
+            msg_header.as_bytes(),
+            cmd.as_bytes(),
+        ])));
+
+        dev_info!(
+            &self.dev,
+            "GSP RPC: send: seq# {}, function=0x{:x} ({}), length=0x{:x}\n",
+            self.seq,
+            msg_header.function_number(),
+            msg_header.function()?,
+            msg_header.length(),
+        );
+
+        let elem_count = msg_header.element_count();
+        self.seq += 1;
+        self.gsp_mem.advance_cpu_write_ptr(elem_count);
+        Cmdq::notify_gsp(bar);
+
+        Ok(())
+    }
+
+    #[expect(unused)]
+    pub(crate) fn send_gsp_command_with_payload<M, E>(
+        &mut self,
+        bar: &Bar0,
+        payload_size: usize,
+        init: impl Init<M, E>,
+        init_payload: impl FnOnce(SBufferIter<core::array::IntoIter<&mut [u8], 2>>) -> Result,
+    ) -> Result
+    where
+        M: CommandToGspWithPayload,
+        // This allows all error types, including `Infallible`, to be used with `init`. Without
+        // this we cannot use regular stack objects as `init` since their `Init` implementation
+        // does not return any error.
+        Error: From<E>,
+    {
+        #[repr(C)]
+        struct FullCommand<M> {
+            hdr: GspMsgElement,
+            cmd: M,
+        }
+        let (msg_header, cmd, payload_1, payload_2) =
+            self.gsp_mem.allocate_command_regions::<M>(payload_size)?;
+
+        let seq = self.seq;
+        let initializer = try_init!(FullCommand {
+            hdr <- GspMsgElement::init(seq, size_of::<M>() + payload_size, M::FUNCTION),
+            cmd <- init,
+        });
+
+        // Fill the header and command in-place.
+        // SAFETY:
+        //  - allocate_command_regions guarantees msg_header points to enough
+        //    space in the command queue to contain FullCommand
+        //  - __init ensures the command header and struct a fully initialized
+        unsafe {
+            initializer.__init(msg_header.as_bytes_mut().as_mut_ptr().cast())?;
+        }
+
+        // Fill the payload
+        let sbuffer = SBufferIter::new_writer([&mut payload_1[..], &mut payload_2[..]]);
+        init_payload(sbuffer)?;
+
+        msg_header.set_checksum(Cmdq::calculate_checksum(SBufferIter::new_reader([
+            msg_header.as_bytes(),
+            cmd.as_bytes(),
+            payload_1,
+            payload_2,
+        ])));
+
+        dev_info!(
+            &self.dev,
+            "GSP RPC: send: seq# {}, function=0x{:x} ({}), length=0x{:x}\n",
+            self.seq,
+            msg_header.function_number(),
+            msg_header.function()?,
+            msg_header.length(),
+        );
+
+        let elem_count = msg_header.element_count();
+        self.seq += 1;
+        self.gsp_mem.advance_cpu_write_ptr(elem_count);
+        Cmdq::notify_gsp(bar);
+
+        Ok(())
Quoted 8 lines
Quoted 1 line

+ }

I think we can factorize most of send_gsp_command and send_gsp_command_with_payload into a private method that requires a CommandToGspBase and takes the same parameters as send_gsp_command_with_payload. All it would need to do when called from send_gsp_command is take a closure that doesn't write any payload.

You're right, that is nicer. I was thinking it would be good to have a simple function that ignored the payloads, but it didn't really end up any simpler.

Quoted 88 lines
Quoted 88 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +88 -0
+
+    #[expect(unused)]
+    pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>(
+        &mut self,
+        timeout: Delta,
+        init: impl FnOnce(&M, SBufferIter<core::array::IntoIter<&[u8], 2>>) -> Result<R>,
+    ) -> Result<R> {
+        let driver_area = read_poll_timeout(
+            || Ok(self.gsp_mem.driver_read_area()),
+            |driver_area: &(&[[u8; 4096]], &[[u8; 4096]])| !driver_area.0.is_empty(),
+            Delta::from_millis(10),
+            timeout,
+        )?;
+
+        #[allow(clippy::incompatible_msrv)]
+        let (msg_header_slice, slice_1) = driver_area
+            .0
+            .as_flattened()
+            .split_at(size_of::<GspMsgElement>());
+        let msg_header = GspMsgElement::from_bytes(msg_header_slice).ok_or(EIO)?;
+        if msg_header.length() < size_of::<M>() as u32 {
+            return Err(EIO);
+        }
+
+        let function: MsgFunction = msg_header.function().map_err(|_| {
+            dev_info!(
+                self.dev,
+                "GSP RPC: receive: seq# {}, bad function=0x{:x}, length=0x{:x}\n",
+                msg_header.sequence(),
+                msg_header.function_number(),
+                msg_header.length(),
+            );
+            EIO
+        })?;
+
+        // Log RPC receive with message type decoding
+        dev_info!(
+            self.dev,
+            "GSP RPC: receive: seq# {}, function=0x{:x} ({}), length=0x{:x}\n",
+            msg_header.sequence(),
+            msg_header.function_number(),
+            function,
+            msg_header.length(),
+        );
+
+        let (cmd_slice, payload_1) = slice_1.split_at(size_of::<M>());
+        #[allow(clippy::incompatible_msrv)]
+        let payload_2 = driver_area.1.as_flattened();
+
+        // Cut the payload slice(s) down to the actual length of the payload.
+        let (cmd_payload_1, cmd_payload_2) =
+            if payload_1.len() > msg_header.length() as usize - size_of::<M>() {
+                (
+                    payload_1
+                        .split_at(msg_header.length() as usize - size_of::<M>())
+                        .0,
+                    &payload_2[0..0],
+                )
+            } else {
+                (
+                    payload_1,
+                    payload_2
+                        .split_at(msg_header.length() as usize - size_of::<M>() - payload_1.len())
+                        .0,
+                )
+            };
+
+        if Cmdq::calculate_checksum(SBufferIter::new_reader([
+            msg_header.as_bytes(),
+            cmd_slice,
+            cmd_payload_1,
+            cmd_payload_2,
+        ])) != 0
+        {
+            dev_err!(
+                self.dev,
+                "GSP RPC: receive: Call {} - bad checksum",
+                msg_header.sequence()
+            );
+            return Err(EIO);
+        }
+
+        let result = if function == M::FUNCTION {
+            let cmd = M::from_bytes(cmd_slice).ok_or(EINVAL)?;
+            let sbuffer = SBufferIter::new_reader([cmd_payload_1, cmd_payload_2]);
+            init(cmd, sbuffer)
+        } else {
+            Err(ERANGE)
Quoted 8 lines

drivers/gpu/nova-core/gsp/cmdq.rs line 492

Quoted 6 lines
drivers/gpu/nova-core/gsp/cmdq.rs new file +6 -0
+        };
+
+        self.gsp_mem
+            .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32));
+        result
+    }

Here we probably also want to comment the different steps a bit.

Ok.

#31 lore raw

Re: [PATCH v5 07/14] gpu: nova-core: gsp: Add GSP command queue handling

In reply to Alistair Popple

scripts/Makefile.build $(obj)/%.lst: $(obj)/%.c FORCE · line 320

Quoted 3 lines · Alistair Popple
scripts/Makefile.build +1 -1
-rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg
+rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg,slice_flatten

This is not sorted, and it is not mentioned in the comment right above it -- in general, please try to clearly mention this sort of addition in the commit message too.

As Alexandre mentions, it will not work to use it as-is. However, what happened in 1.80 was just a rename, but the feature was available since 1.67 as flatten.

If you want to still use it through an indirection or similar, then please move it to a new commit that explains the addition of the feature and the indirection to use it.

See e.g. the file() function I added in e.g. rust/macros/helpers.rs in commit 36174d16f3ec ("rust: kunit: support KUnit-mapped assert! macros in #[test]s") to see how to set it up.

Essentially it is:

    config RUSTC_HAS_SPAN_FILE
           def_bool RUSTC_VERSION >= 108800

And then:

    pub(crate) fn file() -> String {
        #[cfg(not(CONFIG_RUSTC_HAS_SPAN_FILE))]
        {
            ...
        }

        #[cfg(CONFIG_RUSTC_HAS_SPAN_FILE)]
        #[allow(clippy::incompatible_msrv)]
        {
            ...
        }
    }

I hope that helps.

Thanks!

Cheers,
Miguel

#32 lore raw

Re: [PATCH v5 07/14] gpu: nova-core: gsp: Add GSP command queue handling

On 2025-10-17 at 05:44 +1100, Miguel Ojeda <[email protected]> wrote...

Quoted 9 lines · Alistair Popple

scripts/Makefile.build $(obj)/%.lst: $(obj)/%.c FORCE · line 320

Quoted 3 lines
scripts/Makefile.build +1 -1
-rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg
+rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg,slice_flatten

This is not sorted, and it is not mentioned in the comment right above it -- in general, please try to clearly mention this sort of addition in the commit message too.

Quoted 3 lines

As Alexandre mentions, it will not work to use it as-is. However, what happened in 1.80 was just a rename, but the feature was available since 1.67 as flatten.

Argh! Thanks, I wasn't aware the function had been renamed.

Quoted 3 lines

If you want to still use it through an indirection or similar, then please move it to a new commit that explains the addition of the feature and the indirection to use it.

That makes sense, thanks for the explaination.

Quoted 30 lines

See e.g. the file() function I added in e.g. rust/macros/helpers.rs in commit 36174d16f3ec ("rust: kunit: support KUnit-mapped assert! macros in #[test]s") to see how to set it up.

Essentially it is:

    config RUSTC_HAS_SPAN_FILE
           def_bool RUSTC_VERSION >= 108800

And then:

    pub(crate) fn file() -> String {
        #[cfg(not(CONFIG_RUSTC_HAS_SPAN_FILE))]
        {
            ...
        }

        #[cfg(CONFIG_RUSTC_HAS_SPAN_FILE)]
        #[allow(clippy::incompatible_msrv)]
        {
            ...
        }
    }

I hope that helps.

Thanks!

Cheers,
Miguel

#33 lore raw

PATCH v5 8/14 gpu: nova-core: gsp: Create rmargs

In reply to Alistair Popple

Initialise the GSP resource manager arguments (rmargs) which provide initialisation parameters to the GSP firmware during boot. The rmargs structure contains arguments to configure the GSP message/command queue location.

These are mapped for coherent DMA and added to the libos data structure for access when booting GSP.

---

Changes for v5:
- Derive Zeroable trait

Changes for v2:
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gsp.rs                  |  16 +++
 drivers/gpu/nova-core/gsp/cmdq.rs             |  24 +++-
 drivers/gpu/nova-core/gsp/fw.rs               |  60 ++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ------------------
 4 files changed, 97 insertions(+), 135 deletions(-)
drivers/gpu/nova-core/gsp.rs +16 -0
index 1d472c5fad7a..58b595b8badd 100644
@@ -19,6 +19,7 @@
 mod fw;
 
 use fw::LibosMemoryRegionInitArgument;
+use fw::GspArgumentsCached;
 
 pub(crate) mod cmdq;
 
@@ -36,6 +37,7 @@ pub(crate) struct Gsp {
     logintr: LogBuffer,
     logrm: LogBuffer,
     pub(crate) cmdq: Cmdq,
+    rmargs: CoherentAllocation<GspArgumentsCached>,
 }
 
 #[repr(C)]
@@ -117,12 +119,26 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self
 
         // Creates its own PTE array.
         let cmdq = Cmdq::new(dev)?;
+        let rmargs = CoherentAllocation::<GspArgumentsCached>::alloc_coherent(
+            dev,
+            1,
+            GFP_KERNEL | __GFP_ZERO,
+        )?;
+        dma_write!(libos[3] = LibosMemoryRegionInitArgument::new("RMARGS", &rmargs)?)?;
+
+        dma_write!(
+            rmargs[0] = fw::GspArgumentsCached::new(
+                fw::MessageQueueInitArguments::new(&cmdq),
+                fw::GspSrInitArguments::new()
+            )
+        )?;
 
         Ok(try_pin_init!(Self {
             libos,
             loginit,
             logintr,
             logrm,
+            rmargs,
             cmdq,
         }))
     }
drivers/gpu/nova-core/gsp/cmdq.rs +21 -3
index 3f8cb7a35922..da074a2ed0d9 100644
@@ -6,7 +6,7 @@
 
 use kernel::alloc::flags::GFP_KERNEL;
 use kernel::device;
-use kernel::dma::CoherentAllocation;
+use kernel::dma::{CoherentAllocation, DmaAddress};
 use kernel::dma_write;
 use kernel::io::poll::read_poll_timeout;
 use kernel::prelude::*;
@@ -247,10 +247,25 @@ pub(crate) struct Cmdq {
     dev: ARef<device::Device>,
     seq: u32,
     gsp_mem: DmaGspMem,
-    pub _nr_ptes: u32,
 }
 
 impl Cmdq {
+    /// Offset of the data after the PTEs.
+    const POST_PTE_OFFSET: usize = core::mem::offset_of!(GspMem, cpuq);
+
+    /// Offset of command queue ring buffer.
+    pub(crate) const CMDQ_OFFSET: usize = core::mem::offset_of!(GspMem, cpuq)
+        + core::mem::offset_of!(Msgq, msgq)
+        - Self::POST_PTE_OFFSET;
+
+    /// Offset of message queue ring buffer.
+    pub(crate) const STATQ_OFFSET: usize = core::mem::offset_of!(GspMem, gspq)
+        + core::mem::offset_of!(Msgq, msgq)
+        - Self::POST_PTE_OFFSET;
+
+    /// Number of page table entries for the GSP shared region.
+    pub(crate) const NUM_PTES: usize = size_of::<GspMem>() >> GSP_PAGE_SHIFT;
+
     pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> {
         let gsp_mem = DmaGspMem::new(dev)?;
         let nr_ptes = size_of::<GspMem>() >> GSP_PAGE_SHIFT;
@@ -260,7 +275,6 @@ pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> {
             dev: dev.into(),
             seq: 0,
             gsp_mem,
-            _nr_ptes: nr_ptes as u32,
         })
     }
 
@@ -490,4 +504,8 @@ pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>(
             .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32));
         result
     }
+
+    pub(crate) fn dma_handle(&self) -> DmaAddress {
+        self.gsp_mem.0.dma_handle()
+    }
 }
drivers/gpu/nova-core/gsp/fw.rs +60 -0

69 lines not on this page · raw message

drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +0 -132

138 lines not on this page · raw message

Signature · 1 line
2.50.1
#34 lore raw

Re: [PATCH v5 08/14] gpu: nova-core: gsp: Create rmargs

Quoted 15 lines · Alistair Popple

Initialise the GSP resource manager arguments (rmargs) which provide initialisation parameters to the GSP firmware during boot. The rmargs structure contains arguments to configure the GSP message/command queue location.

These are mapped for coherent DMA and added to the libos data structure for access when booting GSP.

---

Changes for v5:
- Derive Zeroable trait

Quoted 7 lines

Changes for v2:
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gsp.rs                  |  16 +++
 drivers/gpu/nova-core/gsp/cmdq.rs             |  24 +++-
 drivers/gpu/nova-core/gsp/fw.rs               |  60 ++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ------------------

Mmm, looks like we are removing bindings. Can we not add them in the first place? :)

Quoted 23 lines
 4 files changed, 97 insertions(+), 135 deletions(-)
drivers/gpu/nova-core/gsp.rs +2 -0
index 1d472c5fad7a..58b595b8badd 100644
@@ -19,6 +19,7 @@
 mod fw;
 
 use fw::LibosMemoryRegionInitArgument;
+use fw::GspArgumentsCached;
 
 pub(crate) mod cmdq;
 
@@ -36,6 +37,7 @@ pub(crate) struct Gsp {
     logintr: LogBuffer,
     logrm: LogBuffer,
     pub(crate) cmdq: Cmdq,
+    rmargs: CoherentAllocation<GspArgumentsCached>,
 }
 
 #[repr(C)]

drivers/gpu/nova-core/gsp.rs pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self · line 122

Quoted 4 lines
drivers/gpu/nova-core/gsp.rs +1 -0
@@ -117,12 +119,26 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self
 
         // Creates its own PTE array.
         let cmdq = Cmdq::new(dev)?;
+        let rmargs = CoherentAllocation::<GspArgumentsCached>::alloc_coherent(

Let's add a space between the declaration of cmdq and rmargs.

Quoted 36 lines
drivers/gpu/nova-core/gsp.rs +13 -0
+            dev,
+            1,
+            GFP_KERNEL | __GFP_ZERO,
+        )?;
+        dma_write!(libos[3] = LibosMemoryRegionInitArgument::new("RMARGS", &rmargs)?)?;
+
+        dma_write!(
+            rmargs[0] = fw::GspArgumentsCached::new(
+                fw::MessageQueueInitArguments::new(&cmdq),
+                fw::GspSrInitArguments::new()
+            )
+        )?;
 
         Ok(try_pin_init!(Self {
             libos,
             loginit,
             logintr,
             logrm,
+            rmargs,
             cmdq,
         }))
     }
drivers/gpu/nova-core/gsp/cmdq.rs +1 -1
index 3f8cb7a35922..da074a2ed0d9 100644
@@ -6,7 +6,7 @@
 
 use kernel::alloc::flags::GFP_KERNEL;
 use kernel::device;
-use kernel::dma::CoherentAllocation;
+use kernel::dma::{CoherentAllocation, DmaAddress};
 use kernel::dma_write;
 use kernel::io::poll::read_poll_timeout;
 use kernel::prelude::*;

drivers/gpu/nova-core/gsp/cmdq.rs pub(crate) struct Cmdq { · line 250

Quoted 4 lines
drivers/gpu/nova-core/gsp/cmdq.rs +0 -1
@@ -247,10 +247,25 @@ pub(crate) struct Cmdq {
     dev: ARef<device::Device>,
     seq: u32,
     gsp_mem: DmaGspMem,
-    pub _nr_ptes: u32,

We probably shouldn't have introduced this unused member in the first place.

Quoted 68 lines
drivers/gpu/nova-core/gsp/cmdq.rs +20 -1
 }
 
 impl Cmdq {
+    /// Offset of the data after the PTEs.
+    const POST_PTE_OFFSET: usize = core::mem::offset_of!(GspMem, cpuq);
+
+    /// Offset of command queue ring buffer.
+    pub(crate) const CMDQ_OFFSET: usize = core::mem::offset_of!(GspMem, cpuq)
+        + core::mem::offset_of!(Msgq, msgq)
+        - Self::POST_PTE_OFFSET;
+
+    /// Offset of message queue ring buffer.
+    pub(crate) const STATQ_OFFSET: usize = core::mem::offset_of!(GspMem, gspq)
+        + core::mem::offset_of!(Msgq, msgq)
+        - Self::POST_PTE_OFFSET;
+
+    /// Number of page table entries for the GSP shared region.
+    pub(crate) const NUM_PTES: usize = size_of::<GspMem>() >> GSP_PAGE_SHIFT;
+
     pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> {
         let gsp_mem = DmaGspMem::new(dev)?;
         let nr_ptes = size_of::<GspMem>() >> GSP_PAGE_SHIFT;
@@ -260,7 +275,6 @@ pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> {
             dev: dev.into(),
             seq: 0,
             gsp_mem,
-            _nr_ptes: nr_ptes as u32,
         })
     }
 
@@ -490,4 +504,8 @@ pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>(
             .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32));
         result
     }
+
+    pub(crate) fn dma_handle(&self) -> DmaAddress {
+        self.gsp_mem.0.dma_handle()
+    }
 }
drivers/gpu/nova-core/gsp/fw.rs +14 -0
index a2ce570ddfaf..70abda1c2af8 100644
@@ -16,6 +16,7 @@
 
 use crate::firmware::gsp::GspFirmware;
 use crate::gpu::Chipset;
+use crate::gsp::cmdq::Cmdq;
 use crate::gsp::FbLayout;
 use crate::gsp::GSP_PAGE_SIZE;
 
@@ -483,3 +484,62 @@ unsafe impl AsBytes for GspMsgElement {}
 // SAFETY: This struct only contains integer types for which all bit patterns
 // are valid.
 unsafe impl FromBytes for GspMsgElement {}
+
+#[repr(transparent)]
+pub(crate) struct GspArgumentsCached(bindings::GSP_ARGUMENTS_CACHED);
+
+impl GspArgumentsCached {
+    pub(crate) fn new(
+        queue_arguments: MessageQueueInitArguments,
+        sr_arguments: GspSrInitArguments,
+    ) -> Self {
+        Self(bindings::GSP_ARGUMENTS_CACHED {
+            messageQueueInitArguments: queue_arguments.0,
+            srInitArguments: sr_arguments.0,
+            bDmemStack: 1,

drivers/gpu/nova-core/gsp/fw.rs unsafe impl AsBytes for GspMsgElement {} · line 509

Quoted 10 lines
drivers/gpu/nova-core/gsp/fw.rs +10 -0
+            ..Default::default()
+        })
+    }
+}
+
+impl From<GspArgumentsCached> for bindings::GSP_ARGUMENTS_CACHED {
+    fn from(value: GspArgumentsCached) -> Self {
+        value.0
+    }
+}

This From impl seems unneeded?

#35 lore raw

Re: [PATCH v5 08/14] gpu: nova-core: gsp: Create rmargs

On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 24 lines · Alistair Popple
Quoted 15 lines

Initialise the GSP resource manager arguments (rmargs) which provide initialisation parameters to the GSP firmware during boot. The rmargs structure contains arguments to configure the GSP message/command queue location.

These are mapped for coherent DMA and added to the libos data structure for access when booting GSP.

---

Changes for v5:
- Derive Zeroable trait

Quoted 7 lines

Changes for v2:
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gsp.rs                  |  16 +++
 drivers/gpu/nova-core/gsp/cmdq.rs             |  24 +++-
 drivers/gpu/nova-core/gsp/fw.rs               |  60 ++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ------------------
Quoted 2 lines

Mmm, looks like we are removing bindings. Can we not add them in the first place? :)

Bah. That's just bad rebasing - updating the bindings to add Zeroable to them was as much of a pain as I thought it would be :-)

Quoted 21 lines
Quoted 21 lines
 4 files changed, 97 insertions(+), 135 deletions(-)
drivers/gpu/nova-core/gsp.rs +2 -0
index 1d472c5fad7a..58b595b8badd 100644
@@ -19,6 +19,7 @@
 mod fw;
 
 use fw::LibosMemoryRegionInitArgument;
+use fw::GspArgumentsCached;
 
 pub(crate) mod cmdq;
 
@@ -36,6 +37,7 @@ pub(crate) struct Gsp {
     logintr: LogBuffer,
     logrm: LogBuffer,
     pub(crate) cmdq: Cmdq,
+    rmargs: CoherentAllocation<GspArgumentsCached>,
 }
 
Quoted 8 lines

drivers/gpu/nova-core/gsp.rs pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self · line 122

Quoted 6 lines
drivers/gpu/nova-core/gsp.rs +1 -0
 #[repr(C)]
@@ -117,12 +119,26 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self
 
         // Creates its own PTE array.
         let cmdq = Cmdq::new(dev)?;
+        let rmargs = CoherentAllocation::<GspArgumentsCached>::alloc_coherent(

Let's add a space between the declaration of cmdq and rmargs.

Ok.

Quoted 29 lines
Quoted 29 lines
drivers/gpu/nova-core/gsp.rs +13 -0
+            dev,
+            1,
+            GFP_KERNEL | __GFP_ZERO,
+        )?;
+        dma_write!(libos[3] = LibosMemoryRegionInitArgument::new("RMARGS", &rmargs)?)?;
+
+        dma_write!(
+            rmargs[0] = fw::GspArgumentsCached::new(
+                fw::MessageQueueInitArguments::new(&cmdq),
+                fw::GspSrInitArguments::new()
+            )
+        )?;
 
         Ok(try_pin_init!(Self {
             libos,
             loginit,
             logintr,
             logrm,
+            rmargs,
             cmdq,
         }))
     }
drivers/gpu/nova-core/gsp/cmdq.rs +0 -0
index 3f8cb7a35922..da074a2ed0d9 100644
@@ -6,7 +6,7 @@
 
Quoted 14 lines

drivers/gpu/nova-core/gsp/cmdq.rs pub(crate) struct Cmdq { · line 250

Quoted 12 lines
drivers/gpu/nova-core/gsp/cmdq.rs +1 -2
 use kernel::alloc::flags::GFP_KERNEL;
 use kernel::device;
-use kernel::dma::CoherentAllocation;
+use kernel::dma::{CoherentAllocation, DmaAddress};
 use kernel::dma_write;
 use kernel::io::poll::read_poll_timeout;
 use kernel::prelude::*;
@@ -247,10 +247,25 @@ pub(crate) struct Cmdq {
     dev: ARef<device::Device>,
     seq: u32,
     gsp_mem: DmaGspMem,
-    pub _nr_ptes: u32,

We probably shouldn't have introduced this unused member in the first place.

Good point, it's a hangover from the implementations in previous versions.

Quoted 73 lines
Quoted 73 lines
drivers/gpu/nova-core/gsp/cmdq.rs +20 -1
 }
 
 impl Cmdq {
+    /// Offset of the data after the PTEs.
+    const POST_PTE_OFFSET: usize = core::mem::offset_of!(GspMem, cpuq);
+
+    /// Offset of command queue ring buffer.
+    pub(crate) const CMDQ_OFFSET: usize = core::mem::offset_of!(GspMem, cpuq)
+        + core::mem::offset_of!(Msgq, msgq)
+        - Self::POST_PTE_OFFSET;
+
+    /// Offset of message queue ring buffer.
+    pub(crate) const STATQ_OFFSET: usize = core::mem::offset_of!(GspMem, gspq)
+        + core::mem::offset_of!(Msgq, msgq)
+        - Self::POST_PTE_OFFSET;
+
+    /// Number of page table entries for the GSP shared region.
+    pub(crate) const NUM_PTES: usize = size_of::<GspMem>() >> GSP_PAGE_SHIFT;
+
     pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> {
         let gsp_mem = DmaGspMem::new(dev)?;
         let nr_ptes = size_of::<GspMem>() >> GSP_PAGE_SHIFT;
@@ -260,7 +275,6 @@ pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> {
             dev: dev.into(),
             seq: 0,
             gsp_mem,
-            _nr_ptes: nr_ptes as u32,
         })
     }
 
@@ -490,4 +504,8 @@ pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>(
             .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32));
         result
     }
+
+    pub(crate) fn dma_handle(&self) -> DmaAddress {
+        self.gsp_mem.0.dma_handle()
+    }
 }
drivers/gpu/nova-core/gsp/fw.rs +18 -0
index a2ce570ddfaf..70abda1c2af8 100644
@@ -16,6 +16,7 @@
 
 use crate::firmware::gsp::GspFirmware;
 use crate::gpu::Chipset;
+use crate::gsp::cmdq::Cmdq;
 use crate::gsp::FbLayout;
 use crate::gsp::GSP_PAGE_SIZE;
 
@@ -483,3 +484,62 @@ unsafe impl AsBytes for GspMsgElement {}
 // SAFETY: This struct only contains integer types for which all bit patterns
 // are valid.
 unsafe impl FromBytes for GspMsgElement {}
+
+#[repr(transparent)]
+pub(crate) struct GspArgumentsCached(bindings::GSP_ARGUMENTS_CACHED);
+
+impl GspArgumentsCached {
+    pub(crate) fn new(
+        queue_arguments: MessageQueueInitArguments,
+        sr_arguments: GspSrInitArguments,
+    ) -> Self {
+        Self(bindings::GSP_ARGUMENTS_CACHED {
+            messageQueueInitArguments: queue_arguments.0,
+            srInitArguments: sr_arguments.0,
+            bDmemStack: 1,
+            ..Default::default()
+        })
+    }
+}
Quoted 8 lines

drivers/gpu/nova-core/gsp/fw.rs unsafe impl AsBytes for GspMsgElement {} · line 509

Quoted 6 lines
drivers/gpu/nova-core/gsp/fw.rs +6 -0
+
+impl From<GspArgumentsCached> for bindings::GSP_ARGUMENTS_CACHED {
+    fn from(value: GspArgumentsCached) -> Self {
+        value.0
+    }
+}

This From impl seems unneeded?

Indeed. I don't remember why I added it, must have been needed in an earlier version. I'm suprised clippy doesn't complain.

#36 lore raw

PATCH v5 9/14 gpu: nova-core: Add bindings and accessors for GspSystemInfo

In reply to Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

---

Changes for v5:
- Derive Zeroable trait

Changes for v4:
- Use init! macros

Changes for v3:
- New for v3

---
 drivers/gpu/nova-core/gsp/fw.rs               |   1 +
 drivers/gpu/nova-core/gsp/fw/commands.rs      |  51 +++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ++++++++++++++++++
 3 files changed, 184 insertions(+)
 create mode 100644 drivers/gpu/nova-core/gsp/fw/commands.rs
drivers/gpu/nova-core/gsp/fw.rs +1 -0
index 70abda1c2af8..4563e33e0859 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
+pub(crate) mod commands;
 mod r570_144;
 
 // Alias to avoid repeating the version number with every use.
drivers/gpu/nova-core/gsp/fw/commands.rs new file +51 -0

51 lines not on this page · raw message

drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +132 -0

138 lines not on this page · raw message

Signature · 1 line
2.50.1
#37 lore raw

Re: [PATCH v5 09/14] gpu: nova-core: Add bindings and accessors for GspSystemInfo

Quoted 3 lines · Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

I am getting a new unused warnings on this patch - we should either annotate or merge.

#38 lore raw

Re: [PATCH v5 09/14] gpu: nova-core: Add bindings and accessors for GspSystemInfo

On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 7 lines · Alistair Popple
Quoted 3 lines

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

I am getting a new unused warnings on this patch - we should either annotate or merge.

Yeah there are quite a few unused warnings. They're a pain to deal with during patch revisions and rebasing/rewriting so I was waiting until all the comments settled down before dealing with them as it's pretty tedious work adding lint statements in one patch and removing them in the next.

As you say the easiest solution is just to merge patches, but that just promotes larger patches. I wish we didn't have to make that trade off, but I guess if we want clippy bisection to be clean we have to.

I will probably just merge all the bindings into the patches that use them.

#39 lore raw

Re: [PATCH v5 09/14] gpu: nova-core: Add bindings and accessors for GspSystemInfo

Quoted 14 lines · Alistair Popple

On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 7 lines · Alistair Popple
Quoted 3 lines

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

I am getting a new unused warnings on this patch - we should either annotate or merge.

Yeah there are quite a few unused warnings. They're a pain to deal with during patch revisions and rebasing/rewriting so I was waiting until all the comments settled down before dealing with them as it's pretty tedious work adding lint statements in one patch and removing them in the next.

Quoted 5 lines

As you say the easiest solution is just to merge patches, but that just promotes larger patches. I wish we didn't have to make that trade off, but I guess if we want clippy bisection to be clean we have to.

I will probably just merge all the bindings into the patches that use them.

Miguel's suggestion of adding a temporary, file-global expect(unused) directive would also work very well if you want to preserve the patch sequence.

#40 lore raw

Re: [PATCH v5 09/14] gpu: nova-core: Add bindings and accessors for GspSystemInfo

On 2025-10-17 at 12:41 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 19 lines · Alistair Popple
Quoted 18 lines

On 2025-10-16 at 17:24 +1100, Alexandre Courbot <[email protected]> wrote...

Quoted 7 lines · Alistair Popple
Quoted 3 lines

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

I am getting a new unused warnings on this patch - we should either annotate or merge.

Yeah there are quite a few unused warnings. They're a pain to deal with during patch revisions and rebasing/rewriting so I was waiting until all the comments settled down before dealing with them as it's pretty tedious work adding lint statements in one patch and removing them in the next.

As you say the easiest solution is just to merge patches, but that just promotes larger patches. I wish we didn't have to make that trade off, but I guess if we want clippy bisection to be clean we have to.

Quoted 5 lines
Quoted 1 line

I will probably just merge all the bindings into the patches that use them.

Miguel's suggestion of adding a temporary, file-global expect(unused) directive would also work very well if you want to preserve the patch sequence.

Yeah, I quite liked that suggestion as a way of alleviating some of the tedium. But given the bindings are pretty self-contained anyway lets just squash them.

#41 lore raw

PATCH v5 10/14 gpu: nova-core: Add bindings for the GSP RM registry tables

In reply to Alistair Popple

Adds bindings and constructors for PACKED_REGISTRY_TABLE and PACKED_REGISTRY_ENTRY structures.

---

Changes for v5:
- Derive Zeroable trait

Changes for v4:
- Use init! macros
- Add comments around only supporting DWORD entry types

Changes for v3:
- New for v3

---
 drivers/gpu/nova-core/gsp/fw/commands.rs      | 49 +++++++++++++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 16 ++++++
 2 files changed, 65 insertions(+)
drivers/gpu/nova-core/gsp/fw/commands.rs +49 -0
index 9a524bba1ac4..79a69c6279e8 100644
@@ -49,3 +49,52 @@ unsafe impl AsBytes for GspSystemInfo {}
 // SAFETY: These structs don't meet the no-padding requirements of FromBytes but
 //         that is not a problem because they are not used outside the kernel.
 unsafe impl FromBytes for GspSystemInfo {}
+
+#[repr(transparent)]
+pub(crate) struct PackedRegistryEntry(bindings::PACKED_REGISTRY_ENTRY);
+
+impl PackedRegistryEntry {
+    pub(crate) fn new(offset: u32, value: u32) -> Self {
+        Self({
+            bindings::PACKED_REGISTRY_ENTRY {
+                nameOffset: offset,
+
+                // We only support DWORD types for now. Support for other types
+                // will come later if required.
+                type_: bindings::REGISTRY_TABLE_ENTRY_TYPE_DWORD as u8,
+                __bindgen_padding_0: Default::default(),
+                data: value,
+                length: 0,
+            }
+        })
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for PackedRegistryEntry {}
+
+#[repr(transparent)]
+pub(crate) struct PackedRegistryTable {
+    inner: bindings::PACKED_REGISTRY_TABLE,
+}
+
+impl PackedRegistryTable {
+    #[allow(non_snake_case)]
+    pub(crate) fn init(num_entries: u32, size: u32) -> impl Init<Self> {
+        type InnerPackedRegistryTable = bindings::PACKED_REGISTRY_TABLE;
+        let init_inner = init!(InnerPackedRegistryTable {
+            numEntries: num_entries,
+            size,
+            entries: Default::default()
+        });
+
+        init!(PackedRegistryTable { inner <- init_inner })
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for PackedRegistryTable {}
+
+// SAFETY: This struct only contains integer types for which all bit patterns
+// are valid.
+unsafe impl FromBytes for PackedRegistryTable {}
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +16 -0
index 1251b0c313ce..32933874ff97 100644
@@ -649,6 +649,22 @@ pub struct LibosMemoryRegionInitArgument {
     pub __bindgen_padding_0: [u8; 6usize],
 }
 #[repr(C)]
+#[derive(Debug, Default, Copy, Clone)]
+pub struct PACKED_REGISTRY_ENTRY {
+    pub nameOffset: u32_,
+    pub type_: u8_,
+    pub __bindgen_padding_0: [u8; 3usize],
+    pub data: u32_,
+    pub length: u32_,
+}
+#[repr(C)]
+#[derive(Debug, Default)]
+pub struct PACKED_REGISTRY_TABLE {
+    pub size: u32_,
+    pub numEntries: u32_,
+    pub entries: __IncompleteArrayField<PACKED_REGISTRY_ENTRY>,
+}
+#[repr(C)]
 #[derive(Debug, Default, Copy, Clone, Zeroable)]
 pub struct msgqTxHeader {
     pub version: u32_,
Signature · 1 line
2.50.1
#42 lore raw

PATCH v5 11/14 gpu: nova-core: gsp: Create RM registry and sysinfo commands

In reply to Alistair Popple

Add the RM registry and system information commands that enable the host driver to configure GSP firmware parameters during initialization.

The RM registry is serialized into a packed format and sent via the command queue. For now only two parameters which are required to boot GSP are hardcoded. In future a kernel module parameter will be added to enable other parameters to be added.

Also add the system info command, which provides required hardware information to the GSP. These commands use the GSP command queue infrastructure to issue commands to the GSP which is read during GSP boot.

---

Changes for v4:
- Use init! macros
- Update to use send_gsp_command_with_payload() for the registry
- Add RMDevidCheckIgnore registry setting (thanks Timur)

Changes for v3:
- Use MsgFunction enum
- Rename GspCmdq to Cmdq
- Rename GspCommandToGsp to CommandToGsp
- Rename GspMessageFromGsp to MessageFromGsp
- Split bindings into separate patch

Changes for v2:
- Rebased on Alex's latest tree

---
 drivers/gpu/nova-core/gsp.rs          |   1 +
 drivers/gpu/nova-core/gsp/boot.rs     |   6 +-
 drivers/gpu/nova-core/gsp/cmdq.rs     |   2 -
 drivers/gpu/nova-core/gsp/commands.rs | 115 ++++++++++++++++++++++++++
 drivers/gpu/nova-core/sbuffer.rs      |   1 -
 5 files changed, 121 insertions(+), 4 deletions(-)
 create mode 100644 drivers/gpu/nova-core/gsp/commands.rs
drivers/gpu/nova-core/gsp.rs +1 -0
index 58b595b8badd..0f88725266bb 100644
@@ -22,6 +22,7 @@
 use fw::GspArgumentsCached;
 
 pub(crate) mod cmdq;
+pub(crate) mod commands;
 
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
 pub(crate) const GSP_PAGE_SIZE: usize = 1 << GSP_PAGE_SHIFT;
drivers/gpu/nova-core/gsp/boot.rs +5 -1
index 1d2448331d7a..0b306313ec53 100644
@@ -16,6 +16,7 @@
     FIRMWARE_VERSION,
 };
 use crate::gpu::Chipset;
+use crate::gsp::commands::{build_registry, set_system_info};
 use crate::gsp::GspFwWprMeta;
 use crate::regs;
 use crate::vbios::Vbios;
@@ -105,7 +106,7 @@ fn run_fwsec_frts(
     ///
     /// Upon return, the GSP is up and running, and its runtime object given as return value.
     pub(crate) fn boot(
-        self: Pin<&mut Self>,
+        mut self: Pin<&mut Self>,
         pdev: &pci::Device<device::Bound>,
         bar: &Bar0,
         chipset: Chipset,
@@ -139,6 +140,9 @@ pub(crate) fn boot(
             CoherentAllocation::<GspFwWprMeta>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?;
         dma_write!(wpr_meta[0] = GspFwWprMeta::new(&gsp_fw, &fb_layout))?;
 
+        set_system_info(&mut self.cmdq, pdev, bar)?;
+        build_registry(&mut self.cmdq, bar)?;
+
         Ok(())
     }
 }
drivers/gpu/nova-core/gsp/cmdq.rs +0 -2
index da074a2ed0d9..0cace0dacf13 100644
@@ -292,7 +292,6 @@ fn notify_gsp(bar: &Bar0) {
         NV_PGSP_QUEUE_HEAD::default().set_address(0).write(bar);
     }
 
-    #[expect(unused)]
     pub(crate) fn send_gsp_command<M, E>(&mut self, bar: &Bar0, init: impl Init<M, E>) -> Result
     where
         M: CommandToGsp,
@@ -345,7 +344,6 @@ struct FullCommand<M> {
         Ok(())
     }
 
-    #[expect(unused)]
     pub(crate) fn send_gsp_command_with_payload<M, E>(
         &mut self,
         bar: &Bar0,
drivers/gpu/nova-core/gsp/commands.rs new file +115 -0

115 lines not on this page · raw message

drivers/gpu/nova-core/sbuffer.rs +0 -1
index 1a27226b65d8..e88fdab990b1 100644
@@ -186,7 +186,6 @@ fn get_slice_mut(&mut self, len: usize) -> Option<&'a mut [u8]> {
 
     /// Ideally we would implement `Write`, but it is not available in `core`.
     /// So mimic `std::io::Write::write_all`.
-    #[expect(unused)]
     pub(crate) fn write_all(&mut self, mut src: &[u8]) -> Result {
         while !src.is_empty() {
             match self.get_slice_mut(src.len()) {
Signature · 1 line
2.50.1
#43 lore raw

PATCH v5 12/14 nova-core: falcon: Add support to check if RISC-V is active

In reply to Alistair Popple

From: Joel Fernandes <[email protected]>

Add definition for RISCV_CPUCTL register and use it in a new falcon API to check if the RISC-V core of a Falcon is active. It is required by the sequencer to know if the GSP's RISCV processor is active.

---

Changes for v4:
- Return bool instead of Result<bool> from is_riscv_active() as it can't fail (thanks Timur).
- Update register definitions to correct Falcon
- Switch register definition order

---
 drivers/gpu/nova-core/falcon.rs | 9 +++++++++
 drivers/gpu/nova-core/regs.rs   | 7 ++++++-
 2 files changed, 15 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/falcon.rs +9 -0
index 734ac0fbfb49..185ed6d1cfb8 100644
@@ -506,4 +506,13 @@ pub(crate) fn signature_reg_fuse_version(
         self.hal
             .signature_reg_fuse_version(self, bar, engine_id_mask, ucode_id)
     }
+
+    /// Check if the RISC-V core is active.
+    ///
+    /// Returns `true` if the RISC-V core is active, `false` otherwise.
+    #[expect(unused)]
+    pub(crate) fn is_riscv_active(&self, bar: &Bar0) -> bool {
+        let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL::read(bar, &E::ID);
+        cpuctl.active_stat()
+    }
 }
drivers/gpu/nova-core/regs.rs +6 -1
index 0585699ae951..3bd1bddb16bb 100644
@@ -324,7 +324,12 @@ pub(crate) fn mem_scrubbing_done(self) -> bool {
 
 // PRISCV
 
-register!(NV_PRISCV_RISCV_BCR_CTRL @ PFalconBase[0x00001668] {
+register!(NV_PRISCV_RISCV_CPUCTL @ PFalcon2Base[0x00000388] {
+    0:0     halted as bool;
+    7:7     active_stat as bool;
+});
+
+register!(NV_PRISCV_RISCV_BCR_CTRL @ PFalcon2Base[0x00000668] {
     0:0     valid as bool;
     4:4     core_select as bool => PeregrineCoreSelect;
     8:8     br_fetch as bool;
Signature · 1 line
2.50.1
#44 lore raw

PATCH v5 13/14 nova-core: falcon: Add support to write firmware version

In reply to Alistair Popple

From: Joel Fernandes <[email protected]>

This will be needed by both the GSP boot code as well as GSP resume code in the sequencer.

---

Changes for v5:
- Make it infallible

---
 drivers/gpu/nova-core/falcon.rs | 8 ++++++++
 drivers/gpu/nova-core/regs.rs   | 6 ++++++
 2 files changed, 14 insertions(+)
drivers/gpu/nova-core/falcon.rs +8 -0
index 185ed6d1cfb8..c871fd061987 100644
@@ -515,4 +515,12 @@ pub(crate) fn is_riscv_active(&self, bar: &Bar0) -> bool {
         let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL::read(bar, &E::ID);
         cpuctl.active_stat()
     }
+
+    /// Write the application version to the OS register.
+    #[expect(dead_code)]
+    pub(crate) fn write_os_version(&self, bar: &Bar0, app_version: u32) {
+        regs::NV_PFALCON_FALCON_OS::default()
+            .set_value(app_version)
+            .write(bar, &E::ID);
+    }
 }
drivers/gpu/nova-core/regs.rs +6 -0
index 3bd1bddb16bb..6eda5c44c599 100644
@@ -215,6 +215,12 @@ pub(crate) fn vga_workspace_addr(self) -> Option<u64> {
     31:0    value as u32;
 });
 
+// Used to store version information about the firmware running
+// on the Falcon processor.
+register!(NV_PFALCON_FALCON_OS @ PFalconBase[0x00000080] {
+    31:0    value as u32;
+});
+
 register!(NV_PFALCON_FALCON_RM @ PFalconBase[0x00000084] {
     31:0    value as u32;
 });
Signature · 1 line
2.50.1
#45 lore raw

PATCH v5 14/14 nova-core: gsp: Boot GSP

In reply to Alistair Popple

Boot the GSP to the RISC-V active state. Completing the boot requires running the CPU sequencer which will be added in a future commit.

---

Changes for v4:
- Switch wait_on to read_poll_timeout

Changes for v3:
- Fixed minor nit from John
- Added booter load error thanks to Timur's suggestion

Changes for v2:
- Rebased on Alex's latest tree

---
 drivers/gpu/nova-core/falcon.rs         |  2 -
 drivers/gpu/nova-core/firmware/riscv.rs |  3 +-
 drivers/gpu/nova-core/gsp/boot.rs       | 63 ++++++++++++++++++++++++-
 3 files changed, 63 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/falcon.rs +0 -2
index c871fd061987..98ad75b93ea2 100644
@@ -510,14 +510,12 @@ pub(crate) fn signature_reg_fuse_version(
     /// Check if the RISC-V core is active.
     ///
     /// Returns `true` if the RISC-V core is active, `false` otherwise.
-    #[expect(unused)]
     pub(crate) fn is_riscv_active(&self, bar: &Bar0) -> bool {
         let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL::read(bar, &E::ID);
         cpuctl.active_stat()
     }
 
     /// Write the application version to the OS register.
-    #[expect(dead_code)]
     pub(crate) fn write_os_version(&self, bar: &Bar0, app_version: u32) {
         regs::NV_PFALCON_FALCON_OS::default()
             .set_value(app_version)
drivers/gpu/nova-core/firmware/riscv.rs +1 -2
index 115b5f5355a1..98be14263366 100644
@@ -52,7 +52,6 @@ fn new(bin_fw: &BinFirmware<'_>) -> Result<Self> {
 }
 
 /// A parsed firmware for a RISC-V core, ready to be loaded and run.
-#[expect(unused)]
 pub(crate) struct RiscvFirmware {
     /// Offset at which the code starts in the firmware image.
     pub(crate) code_offset: u32,
@@ -61,7 +60,7 @@ pub(crate) struct RiscvFirmware {
     /// Offset at which the manifest starts in the firmware image.
     pub(crate) manifest_offset: u32,
     /// Application version.
-    app_version: u32,
+    pub app_version: u32,
     /// Device-mapped firmware image.
     pub ucode: DmaObject,
 }
drivers/gpu/nova-core/gsp/boot.rs +62 -1

83 lines not on this page · raw message

Signature · 1 line
2.50.1