Skip to contentSkip to conversation

gpu: nova-core: Boot GSP to RISC-V active

PATCH Superseded Alistair Popple 7 participants 31 messages Sep 30, 2025 – Oct 3, 2025

Subsystems (6)
DRM drivers Kbuild RISC-V Rust +2 more subsystems

v3 of 6 Superseded Latest is v6 →

lore
Share & tools
lore
#0 lore raw

v3 cover gpu: nova-core: Boot GSP to RISC-V active

Changes since v2:

The main change since v2 has been to make all firmware bindings completely opaque. It has been made clear this is a pre-requisite for this series to progress upstream as it should make supporting different firmware versions easier in future.

Overall the extra constructors and accessors add a couple of hundred lines of code and a few extra unsafe statements.

Other changes include addressing a bunch of other comments - see the individual patches for further details. There are also still some outstanding comments and TODO's to address which I have not gotten to yet - these will be done in the next version of this series.

Changes since v1:

- Based on feed back from Alex the GSP command queue logic was reworked extensively. This involved creating a new data struct (DmaGspMem) to manage the shared memory areas between CPU and GSP.

- This data structure helps ensure the safety constraints are meet when the CPU is reading/writing the shared memory queues.

- Several other minor comments were addressed, as noted in the individual patches.

This series builds on top of Alex's series[1], most of which has been merged into drm-rust-next, to continue initialising the GSP into a state where it becomes active and it starts communicating with the host. A tree including these patches with the prerequisite patches is available at [2].

It includes patches to initialise several important data structures required to boot the GSP. The biggest change is the implementation of the command/message circular queue used to establish communication between GSP and host in patch 6. Admittedly this patch is rather large - if necessary it could be split into send and receive patches if people prefer.

This is required to configure and boot the GSP. However this series does not get the GSP to a fully active state. Instead it gets it to a state where the GSP sends a message to the host with a sequence of instructions which need running to get to the active state. A subsequent series will implement processing of this message and allow the GSP to get to the fully active state.

A full tree including the prerequisites for this patch series is available at https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream.

[1] - https://lore.kernel.org/rust-for-linux/[email protected]/
[2] - https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream-v2

To: [email protected]
To: [email protected]
To: Danilo Krummrich <[email protected]>
To: Alexandre Courbot <[email protected]>
Cc: Miguel Ojeda <[email protected]>
Cc: Alex Gaynor <[email protected]>
Cc: Boqun Feng <[email protected]>
Cc: Gary Guo <[email protected]>
Cc: Björn Roy Baron <[email protected]>
Cc: Benno Lossin <[email protected]>
Cc: Andreas Hindborg <[email protected]>
Cc: Alice Ryhl <[email protected]>
Cc: Trevor Gross <[email protected]>
Cc: David Airlie <[email protected]>
Cc: Simona Vetter <[email protected]>
Cc: Maarten Lankhorst <[email protected]>
Cc: Maxime Ripard <[email protected]>
Cc: Thomas Zimmermann <[email protected]>
Cc: John Hubbard <[email protected]>
Cc: Joel Fernandes <[email protected]>
Cc: Timur Tabi <[email protected]>
Cc: [email protected]
Cc: [email protected]

Alistair Popple (10):
gpu: nova-core: Set correct DMA mask
gpu: nova-core: Create initial Gsp
gpu: nova-core: gsp: Create wpr metadata
gpu: nova-core: Add GSP command queue bindings
gpu: nova-core: gsp: Add GSP command queue handling
gpu: nova-core: gsp: Create rmargs
gpu: nova-core: Add bindings and accessors for GspSystemInfo
gpu: nova-core: Add bindings for the GSP RM registry tables
gpu: nova-core: gsp: Create RM registry and sysinfo commands
nova-core: gsp: Boot GSP

Joel Fernandes (3):
gpu: nova-core: Add a slice-buffer (sbuffer) datastructure
nova-core: falcon: Add support to check if RISC-V is active
nova-core: falcon: Add support to write firmware version

 drivers/gpu/nova-core/driver.rs               |   8 +-
 drivers/gpu/nova-core/falcon.rs               |  16 +
 drivers/gpu/nova-core/fb.rs                   |   1 -
 drivers/gpu/nova-core/firmware/gsp.rs         |   3 +-
 drivers/gpu/nova-core/firmware/riscv.rs       |   9 +-
 drivers/gpu/nova-core/gpu.rs                  |   2 +-
 drivers/gpu/nova-core/gsp.rs                  | 111 +++-
 drivers/gpu/nova-core/gsp/boot.rs             |  77 ++-
 drivers/gpu/nova-core/gsp/cmdq.rs             | 419 +++++++++++++
 drivers/gpu/nova-core/gsp/commands.rs         | 101 +++
 drivers/gpu/nova-core/gsp/fw.rs               | 419 ++++++++++++-
 drivers/gpu/nova-core/gsp/fw/commands.rs      |  80 +++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 577 ++++++++++++++++++
 drivers/gpu/nova-core/nova_core.rs            |   1 +
 drivers/gpu/nova-core/regs.rs                 |  15 +
 drivers/gpu/nova-core/sbuffer.rs              | 188 ++++++
 scripts/Makefile.build                        |   2 +-
 17 files changed, 2006 insertions(+), 23 deletions(-)
 create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs
 create mode 100644 drivers/gpu/nova-core/gsp/commands.rs
 create mode 100644 drivers/gpu/nova-core/gsp/fw/commands.rs
 create mode 100644 drivers/gpu/nova-core/sbuffer.rs
Signature · 1 line
2.50.1
#1 lore raw

PATCH v3 1/13 gpu: nova-core: Set correct DMA mask

Set the correct DMA mask. Without this DMA will fail on some setups.

---

Changes for v2:

- Update DMA mask to correct value for Ampere/Turing (47 bits)

---
 drivers/gpu/nova-core/driver.rs | 8 +++++++-
 1 file changed, 7 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/driver.rs +7 -1
index 1380b47617f7..ccc97340206e 100644
@@ -1,6 +1,9 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use kernel::{auxiliary, bindings, c_str, device::Core, pci, prelude::*, sizes::SZ_16M, sync::Arc};
+use kernel::{
+    auxiliary, bindings, c_str, device::Core, dma::Device, dma::DmaMask, pci, prelude::*,
+    sizes::SZ_16M, sync::Arc,
+};
 
 use crate::gpu::Gpu;
 
@@ -34,6 +37,9 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self
         pdev.enable_device_mem()?;
         pdev.set_master();
 
+        // SAFETY: No DMA allocations have been made yet
+        unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
+
         let devres_bar = Arc::pin_init(
             pdev.iomap_region_sized::<BAR0_SIZE>(0, c_str!("nova-core/bar0")),
             GFP_KERNEL,
Signature · 1 line
2.50.1
maintainer
#2 lore raw

Re: [PATCH v3 01/13] gpu: nova-core: Set correct DMA mask

drivers/gpu/nova-core/driver.rs fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self · line 41

Quoted 2 lines · Alistair Popple
drivers/gpu/nova-core/driver.rs +2 -0
+        // SAFETY: No DMA allocations have been made yet
+        unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };

I think you forgot to derive the value from the relevant sources, i.e. physical bus, DMA controller and MMU capabilities.

I assume not all GPU architectures / generations have the exact same capabilities?

#3 lore raw

Re: [PATCH v3 01/13] gpu: nova-core: Set correct DMA mask

On 2025-09-30 at 23:29 +1000, Danilo Krummrich <[email protected]> wrote...

Quoted 8 lines · Alistair Popple

drivers/gpu/nova-core/driver.rs fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self · line 41

Quoted 2 lines
drivers/gpu/nova-core/driver.rs +2 -0
+        // SAFETY: No DMA allocations have been made yet
+        unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };

I think you forgot to derive the value from the relevant sources, i.e. physical bus, DMA controller and MMU capabilities.

I assume not all GPU architectures / generations have the exact same capabilities?

Right. Long term we need a HAL for this, and I believe John was going to look at that. In the short term everything we care about supports the same 47-bit address width. And when I say "everything" I mean Turing and Ampere, although I'm pretty sure 47-bits goes back to at least Pascal.

Newer GPU architectures (Hopper+) support greater widths (52-bits), but there's no real impact to constraining these for now until a proper HAL is in place.

And that sounds like an excellent addition that I should make to the commit logs and/or the comment on the constant definition, which fell off my radar when reworking the rest of this series but will fix for v4.

#4 lore raw

Re: [PATCH v3 01/13] gpu: nova-core: Set correct DMA mask

Quoted 5 lines · Alistair Popple

On 2025-09-30 at 23:29 +1000, Danilo Krummrich <[email protected]> wrote...

Quoted 4 lines · Alistair Popple
Quoted 2 lines
drivers/gpu/nova-core/driver.rs +2 -0
+        // SAFETY: No DMA allocations have been made yet
+        unsafe { pdev.dma_set_mask_and_coherent(DmaMask::new::<47>())? };
Quoted 7 lines
Quoted 4 lines

I think you forgot to derive the value from the relevant sources, i.e. physical bus, DMA controller and MMU capabilities.

I assume not all GPU architectures / generations have the exact same capabilities?

Right. Long term we need a HAL for this, and I believe John was going to look at that. In the short term everything we care about supports the same 47-bit

Yes. This will arrive with the Hopper/Blackwell support patchset.

Quoted 5 lines

address width. And when I say "everything" I mean Turing and Ampere, although I'm pretty sure 47-bits goes back to at least Pascal.

Newer GPU architectures (Hopper+) support greater widths (52-bits), but there's no real impact to constraining these for now until a proper HAL is in place.

Yes. The "soul" of these various patchsets so far is, "get GSP responsive for Ampere/Ada". To that end, it's OK to temporarily act as if the only thing that matters in this driver is Ampere/Ada. :)

Quoted 4 lines

And that sounds like an excellent addition that I should make to the commit logs and/or the comment on the constant definition, which fell off my radar when reworking the rest of this series but will fix for v4.

thanks,

Signature · 1 line
John Hubbard
#5 lore raw

PATCH v3 2/13 gpu: nova-core: Create initial Gsp

In reply to Alistair Popple

The GSP requires several areas of memory to operate. Each of these have their own simple embedded page tables. Set these up and map them for DMA to/from GSP using CoherentAllocation's. Return the DMA handle describing where each of these regions are for future use when booting GSP.

---

Change for v3:
- Clean up the PTE array creation, with much thanks to Alex for doing most it (please let me know if I should put you as co-developer!)

Changes for v2:
- Renamed GspMemOjbects to Gsp as that is what they are
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gpu.rs                  |  2 +-
 drivers/gpu/nova-core/gsp.rs                  | 83 +++++++++++++++++--
 drivers/gpu/nova-core/gsp/fw.rs               | 39 +++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 19 +++++
 4 files changed, 134 insertions(+), 9 deletions(-)
drivers/gpu/nova-core/gpu.rs +1 -1
index 5da9ad726483..c939b3868271 100644
@@ -221,7 +221,7 @@ pub(crate) fn new<'a>(
 
             sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset, bar, true)?,
 
-            gsp <- Gsp::new(),
+            gsp <- Gsp::new(pdev)?,
 
             _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? },
 
drivers/gpu/nova-core/gsp.rs +75 -8

102 lines not on this page · raw message

drivers/gpu/nova-core/gsp/fw.rs +39 -0
index 181baa401770..dd1e7fc85d85 100644
@@ -7,8 +7,10 @@
 
 use core::ops::Range;
 
+use kernel::dma::CoherentAllocation;
 use kernel::ptr::Alignable;
 use kernel::sizes::SZ_1M;
+use kernel::transmute::{AsBytes, FromBytes};
 
 use crate::gpu::Chipset;
 use crate::gsp;
@@ -99,3 +101,40 @@ pub(crate) fn wpr_heap_size(&self, chipset: Chipset, fb_size: u64) -> u64 {
 /// addresses of the GSP bootloader and firmware.
 #[repr(transparent)]
 pub(crate) struct GspFwWprMeta(bindings::GspFwWprMeta);
+
+#[repr(transparent)]
+pub(crate) struct LibosMemoryRegionInitArgument(bindings::LibosMemoryRegionInitArgument);
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for LibosMemoryRegionInitArgument {}
+
+// SAFETY: This struct only contains integer types for which all bit patterns
+// are valid.
+unsafe impl FromBytes for LibosMemoryRegionInitArgument {}
+
+impl LibosMemoryRegionInitArgument {
+    pub(crate) fn new<A: AsBytes + FromBytes>(
+        name: &'static str,
+        obj: &CoherentAllocation<A>,
+    ) -> Self {
+        /// Generates the `ID8` identifier required for some GSP objects.
+        fn id8(name: &str) -> u64 {
+            let mut bytes = [0u8; core::mem::size_of::<u64>()];
+
+            for (c, b) in name.bytes().rev().zip(&mut bytes) {
+                *b = c;
+            }
+
+            u64::from_ne_bytes(bytes)
+        }
+
+        Self(bindings::LibosMemoryRegionInitArgument {
+            id8: id8(name),
+            pa: obj.dma_handle(),
+            size: obj.size() as u64,
+            kind: bindings::LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS as u8,
+            loc: bindings::LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM as u8,
+            ..Default::default()
+        })
+    }
+}
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +19 -0
index 0407000cca22..6a14cc324391 100644
@@ -124,3 +124,22 @@ fn default() -> Self {
         }
     }
 }
+pub type LibosAddress = u64_;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_NONE: LibosMemoryRegionKind = 0;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_CONTIGUOUS: LibosMemoryRegionKind = 1;
+pub const LibosMemoryRegionKind_LIBOS_MEMORY_REGION_RADIX3: LibosMemoryRegionKind = 2;
+pub type LibosMemoryRegionKind = ffi::c_uint;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_NONE: LibosMemoryRegionLoc = 0;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_SYSMEM: LibosMemoryRegionLoc = 1;
+pub const LibosMemoryRegionLoc_LIBOS_MEMORY_REGION_LOC_FB: LibosMemoryRegionLoc = 2;
+pub type LibosMemoryRegionLoc = ffi::c_uint;
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone)]
+pub struct LibosMemoryRegionInitArgument {
+    pub id8: LibosAddress,
+    pub pa: LibosAddress,
+    pub size: LibosAddress,
+    pub kind: u8_,
+    pub loc: u8_,
+    pub __bindgen_padding_0: [u8; 6usize],
+}
Signature · 1 line
2.50.1
#6 lore raw

PATCH v3 3/13 gpu: nova-core: gsp: Create wpr metadata

In reply to Alistair Popple

The GSP requires some pieces of metadata to boot. These are passed in a struct which the GSP transfers via DMA. Create this struct and get a handle to it for future use when booting the GSP.

---

Changes for v3:
- Don't re-export WPR constants (thanks Alex)

Changes for v2:
- Rebased on Alex's latest version

---
 drivers/gpu/nova-core/fb.rs                   |  1 -
 drivers/gpu/nova-core/firmware/gsp.rs         |  3 +-
 drivers/gpu/nova-core/firmware/riscv.rs       |  6 +-
 drivers/gpu/nova-core/gsp.rs                  |  1 +
 drivers/gpu/nova-core/gsp/boot.rs             |  7 +++
 drivers/gpu/nova-core/gsp/fw.rs               | 57 ++++++++++++++++++-
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs |  2 +
 7 files changed, 69 insertions(+), 8 deletions(-)
drivers/gpu/nova-core/fb.rs +0 -1
index 4d6a1f452183..5580498ba2fb 100644
@@ -87,7 +87,6 @@ pub(crate) fn unregister(&self, bar: &Bar0) {
 ///
 /// Contains ranges of GPU memory reserved for a given purpose during the GSP boot process.
 #[derive(Debug)]
-#[expect(dead_code)]
 pub(crate) struct FbLayout {
     /// Range of the framebuffer. Starts at `0`.
     pub(crate) fb: Range<u64>,
drivers/gpu/nova-core/firmware/gsp.rs +1 -2
index 9654810834d9..67b85e1db27d 100644
@@ -127,7 +127,7 @@ pub(crate) struct GspFirmware {
     /// Size in bytes of the firmware contained in [`Self::fw`].
     pub size: usize,
     /// Device-mapped GSP signatures matching the GPU's [`Chipset`].
-    signatures: DmaObject,
+    pub signatures: DmaObject,
     /// GSP bootloader, verifies the GSP firmware before loading and running it.
     pub bootloader: RiscvFirmware,
 }
@@ -212,7 +212,6 @@ pub(crate) fn new<'a, 'b>(
         }))
     }
 
-    #[expect(unused)]
     /// Returns the DMA handle of the radix3 level 0 page table.
     pub(crate) fn radix3_dma_handle(&self) -> DmaAddress {
         self.level0.dma_handle()
drivers/gpu/nova-core/firmware/riscv.rs +3 -3
index b90acfc81e78..dec33d2b631a 100644
@@ -53,11 +53,11 @@ fn new(bin_fw: &BinFirmware<'_>) -> Result<Self> {
 #[expect(unused)]
 pub(crate) struct RiscvFirmware {
     /// Offset at which the code starts in the firmware image.
-    code_offset: u32,
+    pub code_offset: u32,
     /// Offset at which the data starts in the firmware image.
-    data_offset: u32,
+    pub data_offset: u32,
     /// Offset at which the manifest starts in the firmware image.
-    manifest_offset: u32,
+    pub manifest_offset: u32,
     /// Application version.
     app_version: u32,
     /// Device-mapped firmware image.
drivers/gpu/nova-core/gsp.rs +1 -0
index 91aa9ce17c57..537a226f98d0 100644
@@ -14,6 +14,7 @@
 use kernel::ptr::Alignment;
 use kernel::transmute::AsBytes;
 
+use crate::fb::FbLayout;
 use fw::LibosMemoryRegionInitArgument;
 
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
drivers/gpu/nova-core/gsp/boot.rs +7 -0
index fb22508128c4..1d2448331d7a 100644
@@ -1,6 +1,8 @@
 // SPDX-License-Identifier: GPL-2.0
 
 use kernel::device;
+use kernel::dma::CoherentAllocation;
+use kernel::dma_write;
 use kernel::pci;
 use kernel::prelude::*;
 
@@ -14,6 +16,7 @@
     FIRMWARE_VERSION,
 };
 use crate::gpu::Chipset;
+use crate::gsp::GspFwWprMeta;
 use crate::regs;
 use crate::vbios::Vbios;
 
@@ -132,6 +135,10 @@ pub(crate) fn boot(
             bar,
         )?;
 
+        let wpr_meta =
+            CoherentAllocation::<GspFwWprMeta>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?;
+        dma_write!(wpr_meta[0] = GspFwWprMeta::new(&gsp_fw, &fb_layout))?;
+
         Ok(())
     }
 }
drivers/gpu/nova-core/gsp/fw.rs +55 -2

73 lines not on this page · raw message

drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +2 -0
index 6a14cc324391..392b25dc6991 100644
@@ -9,6 +9,8 @@
 pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS2_MAX_MB: u32 = 256;
 pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MIN_MB: u32 = 88;
 pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB: u32 = 280;
+pub const GSP_FW_WPR_META_REVISION: u32 = 1;
+pub const GSP_FW_WPR_META_MAGIC: i64 = -2577556379034558285;
 pub type __u8 = ffi::c_uchar;
 pub type __u16 = ffi::c_ushort;
 pub type __u32 = ffi::c_uint;
Signature · 1 line
2.50.1
#7 lore raw

PATCH v3 4/13 gpu: nova-core: Add a slice-buffer (sbuffer) datastructure

In reply to Alistair Popple

From: Joel Fernandes <[email protected]>

A data structure that can be used to write across multiple slices which may be out of order in memory. This lets SBuffer user correctly and safely write out of memory order, without error-prone tracking of pointers/offsets.

let mut buf1 = [0u8; 3];
let mut buf2 = [0u8; 5];
let mut sbuffer = SBuffer::new([&mut buf1[..], &mut buf2[..]]);

let data = b"hello";
let result = sbuffer.write(data);

An internal conversion of gsp.rs to use this resulted in a nice -ve delta:
gsp.rs: 37 insertions(+), 99 deletions(-)

---

Changes for v3:
- Addressed minor review comment from Lyude

---
 drivers/gpu/nova-core/nova_core.rs |   1 +
 drivers/gpu/nova-core/sbuffer.rs   | 191 +++++++++++++++++++++++++++++
 2 files changed, 192 insertions(+)
 create mode 100644 drivers/gpu/nova-core/sbuffer.rs
drivers/gpu/nova-core/nova_core.rs +1 -0
index fffcaee2249f..a6feeba6254c 100644
@@ -11,6 +11,7 @@
 mod gpu;
 mod gsp;
 mod regs;
+mod sbuffer;
 mod util;
 mod vbios;
 
drivers/gpu/nova-core/sbuffer.rs new file +191 -0

191 lines not on this page · raw message

Signature · 1 line
2.50.1
#8 lore raw

PATCH v3 5/13 gpu: nova-core: Add GSP command queue bindings

In reply to Alistair Popple

Add bindings and accessors used for the GSP command queue.

---

Changes for v3:
- New for v3

---
 drivers/gpu/nova-core/gsp/fw.rs               | 262 ++++++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 374 ++++++++++++++++++
 2 files changed, 636 insertions(+)
drivers/gpu/nova-core/gsp/fw.rs +262 -0
index 68a7059bc965..ee86abe7ea10 100644
@@ -5,9 +5,11 @@
 // Alias to avoid repeating the version number with every use.
 use r570_144 as bindings;
 
+use core::fmt;
 use core::ops::Range;
 
 use kernel::dma::CoherentAllocation;
+use kernel::prelude::*;
 use kernel::ptr::{Alignable, Alignment};
 use kernel::sizes::{SZ_128K, SZ_1M};
 use kernel::transmute::{AsBytes, FromBytes};
@@ -16,6 +18,7 @@
 use crate::gpu::Chipset;
 use crate::gsp;
 use crate::gsp::FbLayout;
+use crate::gsp::GSP_PAGE_SIZE;
 
 /// Dummy type to group methods related to heap parameters for running the GSP firmware.
 pub(crate) struct GspFwHeapParams(());
@@ -155,6 +158,120 @@ pub(crate) fn new(gsp_firmware: &GspFirmware, fb_layout: &FbLayout) -> Self {
     }
 }
 
+#[derive(PartialEq)]
+pub(crate) enum MsgFunction {
+    // Common function codes
+    Nop = bindings::NV_VGPU_MSG_FUNCTION_NOP as isize,
+    SetGuestSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO as isize,
+    AllocRoot = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT as isize,
+    AllocDevice = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE as isize,
+    AllocMemory = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY as isize,
+    AllocCtxDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA as isize,
+    AllocChannelDma = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA as isize,
+    MapMemory = bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY as isize,
+    BindCtxDma = bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA as isize,
+    AllocObject = bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT as isize,
+    Free = bindings::NV_VGPU_MSG_FUNCTION_FREE as isize,
+    Log = bindings::NV_VGPU_MSG_FUNCTION_LOG as isize,
+    GetGspStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO as isize,
+    SetRegistry = bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY as isize,
+    GspSetSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO as isize,
+    GspInitPostObjGpu = bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU as isize,
+    GspRmControl = bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL as isize,
+    GetStaticInfo = bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO as isize,
+
+    // Event codes
+    GspInitDone = bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE as isize,
+    GspRunCpuSequencer = bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER as isize,
+    PostEvent = bindings::NV_VGPU_MSG_EVENT_POST_EVENT as isize,
+    RcTriggered = bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED as isize,
+    MmuFaultQueued = bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED as isize,
+    OsErrorLog = bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG as isize,
+    GspPostNoCat = bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD as isize,
+    GspLockdownNotice = bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE as isize,
+    UcodeLibOsPrint = bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT as isize,
+}
+
+impl fmt::Display for MsgFunction {
+    fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {
+        match self {
+            // Common function codes
+            MsgFunction::Nop => write!(f, "NOP"),
+            MsgFunction::SetGuestSystemInfo => write!(f, "SET_GUEST_SYSTEM_INFO"),
+            MsgFunction::AllocRoot => write!(f, "ALLOC_ROOT"),
+            MsgFunction::AllocDevice => write!(f, "ALLOC_DEVICE"),
+            MsgFunction::AllocMemory => write!(f, "ALLOC_MEMORY"),
+            MsgFunction::AllocCtxDma => write!(f, "ALLOC_CTX_DMA"),
+            MsgFunction::AllocChannelDma => write!(f, "ALLOC_CHANNEL_DMA"),
+            MsgFunction::MapMemory => write!(f, "MAP_MEMORY"),
+            MsgFunction::BindCtxDma => write!(f, "BIND_CTX_DMA"),
+            MsgFunction::AllocObject => write!(f, "ALLOC_OBJECT"),
+            MsgFunction::Free => write!(f, "FREE"),
+            MsgFunction::Log => write!(f, "LOG"),
+            MsgFunction::GetGspStaticInfo => write!(f, "GET_GSP_STATIC_INFO"),
+            MsgFunction::SetRegistry => write!(f, "SET_REGISTRY"),
+            MsgFunction::GspSetSystemInfo => write!(f, "GSP_SET_SYSTEM_INFO"),
+            MsgFunction::GspInitPostObjGpu => write!(f, "GSP_INIT_POST_OBJGPU"),
+            MsgFunction::GspRmControl => write!(f, "GSP_RM_CONTROL"),
+            MsgFunction::GetStaticInfo => write!(f, "GET_STATIC_INFO"),
+
+            // Event codes
+            MsgFunction::GspInitDone => write!(f, "INIT_DONE"),
+            MsgFunction::GspRunCpuSequencer => write!(f, "RUN_CPU_SEQUENCER"),
+            MsgFunction::PostEvent => write!(f, "POST_EVENT"),
+            MsgFunction::RcTriggered => write!(f, "RC_TRIGGERED"),
+            MsgFunction::MmuFaultQueued => write!(f, "MMU_FAULT_QUEUED"),
+            MsgFunction::OsErrorLog => write!(f, "OS_ERROR_LOG"),
+            MsgFunction::GspPostNoCat => write!(f, "NOCAT"),
+            MsgFunction::GspLockdownNotice => write!(f, "LOCKDOWN_NOTICE"),
+            MsgFunction::UcodeLibOsPrint => write!(f, "LIBOS_PRINT"),
+        }
+    }
+}
+
+impl TryFrom<u32> for MsgFunction {
+    type Error = kernel::error::Error;
+
+    fn try_from(value: u32) -> Result<MsgFunction> {
+        match value {
+            bindings::NV_VGPU_MSG_FUNCTION_NOP => Ok(MsgFunction::Nop),
+            bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO => {
+                Ok(MsgFunction::SetGuestSystemInfo)
+            }
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_ROOT => Ok(MsgFunction::AllocRoot),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_DEVICE => Ok(MsgFunction::AllocDevice),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_MEMORY => Ok(MsgFunction::AllocMemory),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CTX_DMA => Ok(MsgFunction::AllocCtxDma),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_CHANNEL_DMA => Ok(MsgFunction::AllocChannelDma),
+            bindings::NV_VGPU_MSG_FUNCTION_MAP_MEMORY => Ok(MsgFunction::MapMemory),
+            bindings::NV_VGPU_MSG_FUNCTION_BIND_CTX_DMA => Ok(MsgFunction::BindCtxDma),
+            bindings::NV_VGPU_MSG_FUNCTION_ALLOC_OBJECT => Ok(MsgFunction::AllocObject),
+            bindings::NV_VGPU_MSG_FUNCTION_FREE => Ok(MsgFunction::Free),
+            bindings::NV_VGPU_MSG_FUNCTION_LOG => Ok(MsgFunction::Log),
+            bindings::NV_VGPU_MSG_FUNCTION_GET_GSP_STATIC_INFO => Ok(MsgFunction::GetGspStaticInfo),
+            bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY => Ok(MsgFunction::SetRegistry),
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_SET_SYSTEM_INFO => Ok(MsgFunction::GspSetSystemInfo),
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_INIT_POST_OBJGPU => {
+                Ok(MsgFunction::GspInitPostObjGpu)
+            }
+            bindings::NV_VGPU_MSG_FUNCTION_GSP_RM_CONTROL => Ok(MsgFunction::GspRmControl),
+            bindings::NV_VGPU_MSG_FUNCTION_GET_STATIC_INFO => Ok(MsgFunction::GetStaticInfo),
+            bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE => Ok(MsgFunction::GspInitDone),
+            bindings::NV_VGPU_MSG_EVENT_GSP_RUN_CPU_SEQUENCER => {
+                Ok(MsgFunction::GspRunCpuSequencer)
+            }
+            bindings::NV_VGPU_MSG_EVENT_POST_EVENT => Ok(MsgFunction::PostEvent),
+            bindings::NV_VGPU_MSG_EVENT_RC_TRIGGERED => Ok(MsgFunction::RcTriggered),
+            bindings::NV_VGPU_MSG_EVENT_MMU_FAULT_QUEUED => Ok(MsgFunction::MmuFaultQueued),
+            bindings::NV_VGPU_MSG_EVENT_OS_ERROR_LOG => Ok(MsgFunction::OsErrorLog),
+            bindings::NV_VGPU_MSG_EVENT_GSP_POST_NOCAT_RECORD => Ok(MsgFunction::GspPostNoCat),
+            bindings::NV_VGPU_MSG_EVENT_GSP_LOCKDOWN_NOTICE => Ok(MsgFunction::GspLockdownNotice),
+            bindings::NV_VGPU_MSG_EVENT_UCODE_LIBOS_PRINT => Ok(MsgFunction::UcodeLibOsPrint),
+            _ => Err(EINVAL),
+        }
+    }
+}
+
 #[repr(transparent)]
 pub(crate) struct LibosMemoryRegionInitArgument(bindings::LibosMemoryRegionInitArgument);
 
@@ -191,3 +308,148 @@ fn id8(name: &str) -> u64 {
         })
     }
 }
+
+#[repr(transparent)]
+pub(crate) struct MsgqTxHeader(bindings::msgqTxHeader);
+
+impl MsgqTxHeader {
+    pub(crate) fn new(msgq_size: u32, rx_hdr_offset: u32, msg_count: u32) -> Self {
+        Self(bindings::msgqTxHeader {
+            version: 0,
+            size: msgq_size,
+            msgSize: GSP_PAGE_SIZE as u32,
+            msgCount: msg_count,
+            writePtr: 0,
+            flags: 1,
+            rxHdrOff: rx_hdr_offset,
+            entryOff: GSP_PAGE_SIZE as u32,
+        })
+    }
+
+    pub(crate) fn write_ptr(&self) -> u32 {
+        let ptr = (&self.0.writePtr) as *const u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_read! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.read_volatile() }
+    }
+
+    pub(crate) fn set_write_ptr(&mut self, val: u32) {
+        let ptr = (&mut self.0.writePtr) as *mut u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_write! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.write_volatile(val) }
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for MsgqTxHeader {}
+
+/// RX header for setting up a message queue with the GSP.
+#[repr(transparent)]
+pub(crate) struct MsgqRxHeader(bindings::msgqRxHeader);
+
+impl MsgqRxHeader {
+    pub(crate) fn new() -> Self {
+        Self(Default::default())
+    }
+
+    pub(crate) fn read_ptr(&self) -> u32 {
+        let ptr = (&self.0.readPtr) as *const u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_read! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.read_volatile() }
+    }
+
+    pub(crate) fn set_read_ptr(&mut self, val: u32) {
+        let ptr = (&mut self.0.readPtr) as *mut u32;
+
+        // SAFETY: This is part of a CoherentAllocation and implements the
+        // equivalent as what the dma_write! macro would and is therefore safe
+        // for the same reasons.
+        unsafe { ptr.write_volatile(val) }
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for MsgqRxHeader {}
+
+#[repr(transparent)]
+pub(crate) struct GspRpcHeader(bindings::rpc_message_header_v);
+
+impl GspRpcHeader {
+    pub(crate) fn new(cmd_size: u32, function: u32) -> Self {
+        Self(bindings::rpc_message_header_v {
+            // TODO: magic number
+            header_version: 0x03000000,
+            signature: bindings::NV_VGPU_MSG_SIGNATURE_VALID,
+            function,
+            // We don't ever expect to send a command large enough to overflow.
+            length: (size_of::<Self>() as u32).checked_add(cmd_size).unwrap(),
+            rpc_result: 0xffffffff,
+            rpc_result_private: 0xffffffff,
+            ..Default::default()
+        })
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for GspRpcHeader {}
+
+// SAFETY: This struct only contains integer types for which all bit patterns
+// are valid.
+unsafe impl FromBytes for GspRpcHeader {}
+
+#[repr(transparent)]
+pub(crate) struct GspMsgElement(bindings::GSP_MSG_QUEUE_ELEMENT);
+
+impl GspMsgElement {
+    pub(crate) fn new(sequence: u32, cmd_size: usize, function: u32) -> Self {
+        Self(bindings::GSP_MSG_QUEUE_ELEMENT {
+            seqNum: sequence,
+            // TODO: overflow check and fallible div?
+            elemCount: (size_of::<Self>() + cmd_size).div_ceil(GSP_PAGE_SIZE) as u32,
+            // TODO: fallible conversion.
+            rpc: GspRpcHeader::new(cmd_size as u32, function).0,
+            ..Default::default()
+        })
+    }
+
+    pub(crate) fn set_checksum(&mut self, checksum: u32) {
+        self.0.checkSum = checksum;
+    }
+
+    // Return the total length of the message, noting that rpc.length includes
+    // the length of the GspRpcHeader but not the message header.
+    pub(crate) fn length(&self) -> u32 {
+        size_of::<Self>() as u32 - size_of::<GspRpcHeader>() as u32 + self.0.rpc.length
+    }
+
+    pub(crate) fn sequence(&self) -> u32 {
+        self.0.rpc.sequence
+    }
+
+    pub(crate) fn function_number(&self) -> u32 {
+        self.0.rpc.function
+    }
+
+    pub(crate) fn function(&self) -> Result<MsgFunction> {
+        self.0.rpc.function.try_into()
+    }
+
+    pub(crate) fn element_count(&self) -> u32 {
+        self.0.elemCount
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for GspMsgElement {}
+
+// SAFETY: This struct only contains integer types for which all bit patterns
+// are valid.
+unsafe impl FromBytes for GspMsgElement {}
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +374 -0

388 lines not on this page · raw message

Signature · 1 line
2.50.1
maintainer
#9 lore raw

Re: [PATCH v3 05/13] gpu: nova-core: Add GSP command queue bindings

drivers/gpu/nova-core/gsp/fw.rs fn id8(name: &str) -> u64 { · line 393

Quoted 9 lines · Alistair Popple
drivers/gpu/nova-core/gsp/fw.rs +9 -0
+impl GspRpcHeader {
+    pub(crate) fn new(cmd_size: u32, function: u32) -> Self {
+        Self(bindings::rpc_message_header_v {
+            // TODO: magic number
+            header_version: 0x03000000,
+            signature: bindings::NV_VGPU_MSG_SIGNATURE_VALID,
+            function,
+            // We don't ever expect to send a command large enough to overflow.
+            length: (size_of::<Self>() as u32).checked_add(cmd_size).unwrap(),

This is pretty much equivalent to "we never expect to make any mistakes, hence just crash the kernel". Please handle errors gracefully.

#10 lore raw

PATCH v3 6/13 gpu: nova-core: gsp: Add GSP command queue handling

In reply to Alistair Popple

This commit introduces core infrastructure for handling GSP command and message queues in the nova-core driver. The command queue system enables bidirectional communication between the host driver and GSP firmware through a remote message passing interface.

The interface is based on passing serialised data structures over a ring buffer with separate transmit and receive queues. Commands are sent by writing to the CPU transmit queue and waiting for completion via the receive queue.

To ensure safety mutable or immutable (depending on whether it is a send or receive operation) references are taken on the command queue when allocating the message to write/read to. This ensures message memory remains valid and the command queue can't be mutated whilst an operation is in progress.

Currently this is only used by the probe() routine and therefore can only used by a single thread of execution. Locking to enable safe access from multiple threads will be introduced in a future series when that becomes necessary.

---

There are still a couple of TODOs left to address here, in particular switching to using init! (see "gpu: nova-core: Add bindings and accessors for GspSystemInfo" for why that hasn't been done yet) and a couple of other minor refactors of the send/receive functions which have been pointed out in v2 (eg. methods to extract a command region, etc.)

Changes for v3:
- Reduce the receive payloads to the correct size
- Use opaque bindings
- Clean up of the command queue PTE creation
- Add an enum for the GSP functions
- Rename GspCommandToGsp and GspMessageFromGsp
- Rename GspCmdq
- Add function to notify GSP of updated queue pointers
- Inline driver area access functions
- Fixup receive area calculations

Changes for v2:
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gsp.rs      |  10 +
 drivers/gpu/nova-core/gsp/cmdq.rs | 402 ++++++++++++++++++++++++++++++
 drivers/gpu/nova-core/regs.rs     |   4 +
 drivers/gpu/nova-core/sbuffer.rs  |   2 -
 scripts/Makefile.build            |   2 +-
 5 files changed, 417 insertions(+), 3 deletions(-)
 create mode 100644 drivers/gpu/nova-core/gsp/cmdq.rs
drivers/gpu/nova-core/gsp.rs +10 -0
index 537a226f98d0..3132f1009897 100644
@@ -5,6 +5,7 @@
 
 pub(crate) use fw::{GspFwWprMeta, LibosParams};
 
+use kernel::alloc::flags::GFP_KERNEL;
 use kernel::device;
 use kernel::dma::CoherentAllocation;
 use kernel::dma::DmaAddress;
@@ -15,8 +16,12 @@
 use kernel::transmute::AsBytes;
 
 use crate::fb::FbLayout;
+use crate::gsp::cmdq::Cmdq;
+
 use fw::LibosMemoryRegionInitArgument;
 
+pub(crate) mod cmdq;
+
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
 pub(crate) const GSP_PAGE_SIZE: usize = 1 << GSP_PAGE_SHIFT;
 pub(crate) const GSP_HEAP_ALIGNMENT: Alignment = Alignment::new::<{ 1 << 20 }>();
@@ -31,6 +36,7 @@ pub(crate) struct Gsp {
     pub loginit: CoherentAllocation<u8>,
     pub logintr: CoherentAllocation<u8>,
     pub logrm: CoherentAllocation<u8>,
+    pub cmdq: Cmdq,
 }
 
 #[repr(C)]
@@ -85,11 +91,15 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self
         let logrm = create_logbuffer_dma_object(dev)?;
         dma_write!(libos[2] = LibosMemoryRegionInitArgument::new("LOGRM", &logrm))?;
 
+        // Creates its own PTE array
+        let cmdq = Cmdq::new(dev)?;
+
         Ok(try_pin_init!(Self {
             libos,
             loginit,
             logintr,
             logrm,
+            cmdq,
         }))
     }
 }
drivers/gpu/nova-core/gsp/cmdq.rs new file +402 -0

402 lines not on this page · raw message

drivers/gpu/nova-core/regs.rs +4 -0
index 206dab2e1335..0585699ae951 100644
@@ -71,6 +71,10 @@ pub(crate) fn chipset(self) -> Result<Chipset> {
     30:30   ecc_mode_enabled as bool;
 });
 
+register!(NV_PGSP_QUEUE_HEAD @ 0x00110c00 {
+    31:0    address as u32;
+});
+
 impl NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE {
     /// Returns the usable framebuffer size, in bytes.
     pub(crate) fn usable_fb_size(self) -> u64 {
drivers/gpu/nova-core/sbuffer.rs +0 -2
index e82f9d97ad21..bde80cc3fa63 100644
@@ -37,7 +37,6 @@ impl<'a, I> SBuffer<I>
 where
     I: Iterator,
 {
-    #[expect(unused)]
     pub(crate) fn new_reader(slices: impl IntoIterator<IntoIter = I>) -> Self
     where
         I: Iterator<Item = &'a [u8]>,
@@ -45,7 +44,6 @@ pub(crate) fn new_reader(slices: impl IntoIterator<IntoIter = I>) -> Self
         Self::new(slices)
     }
 
-    #[expect(unused)]
     pub(crate) fn new_writer(slices: impl IntoIterator<IntoIter = I>) -> Self
     where
         I: Iterator<Item = &'a mut [u8]>,
scripts/Makefile.build +1 -1
index d0ee33a487be..4ac6304332b6 100644
@@ -317,7 +317,7 @@ $(obj)/%.lst: $(obj)/%.c FORCE
 #
 # Please see https://github.com/Rust-for-Linux/linux/issues/2 for details on
 # the unstable features in use.
-rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg
+rust_allowed_features := asm_const,asm_goto,arbitrary_self_types,lint_reasons,offset_of_nested,raw_ref_op,used_with_arg,slice_flatten
 
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
Signature · 1 line
2.50.1
#11 lore raw

PATCH v3 7/13 gpu: nova-core: gsp: Create rmargs

In reply to Alistair Popple

Initialise the GSP resource manager arguments (rmargs) which provide initialisation parameters to the GSP firmware during boot. The rmargs structure contains arguments to configure the GSP message/command queue location.

These are mapped for coherent DMA and added to the libos data structure for access when booting GSP.

---

Changes for v2:
- Rebased on Alex's latest series

---
 drivers/gpu/nova-core/gsp.rs                  | 16 +++++
 drivers/gpu/nova-core/gsp/cmdq.rs             | 24 +++++++-
 drivers/gpu/nova-core/gsp/fw.rs               | 60 +++++++++++++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 33 ++++++++++
 4 files changed, 130 insertions(+), 3 deletions(-)
drivers/gpu/nova-core/gsp.rs +16 -0
index 3132f1009897..9e5dd9e5a316 100644
@@ -3,6 +3,7 @@
 mod boot;
 mod fw;
 
+use fw::GspArgumentsCached;
 pub(crate) use fw::{GspFwWprMeta, LibosParams};
 
 use kernel::alloc::flags::GFP_KERNEL;
@@ -37,6 +38,7 @@ pub(crate) struct Gsp {
     pub logintr: CoherentAllocation<u8>,
     pub logrm: CoherentAllocation<u8>,
     pub cmdq: Cmdq,
+    rmargs: CoherentAllocation<GspArgumentsCached>,
 }
 
 #[repr(C)]
@@ -93,12 +95,26 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> Result<impl PinInit<Self
 
         // Creates its own PTE array
         let cmdq = Cmdq::new(dev)?;
+        let rmargs = CoherentAllocation::<GspArgumentsCached>::alloc_coherent(
+            dev,
+            1,
+            GFP_KERNEL | __GFP_ZERO,
+        )?;
+        dma_write!(libos[3] = LibosMemoryRegionInitArgument::new("RMARGS", &rmargs))?;
+
+        dma_write!(
+            rmargs[0] = fw::GspArgumentsCached::new(
+                fw::MessageQueueInitArguments::new(&cmdq),
+                fw::GspSrInitArguments::new()
+            )
+        )?;
 
         Ok(try_pin_init!(Self {
             libos,
             loginit,
             logintr,
             logrm,
+            rmargs,
             cmdq,
         }))
     }
drivers/gpu/nova-core/gsp/cmdq.rs +21 -3
index 7d43dc987ba2..2fd6b31db9e9 100644
@@ -6,7 +6,7 @@
 
 use kernel::alloc::flags::GFP_KERNEL;
 use kernel::device;
-use kernel::dma::CoherentAllocation;
+use kernel::dma::{CoherentAllocation, DmaAddress};
 use kernel::dma_write;
 use kernel::prelude::*;
 use kernel::sync::aref::ARef;
@@ -202,10 +202,25 @@ pub(crate) struct Cmdq {
     dev: ARef<device::Device>,
     seq: u32,
     gsp_mem: DmaGspMem,
-    pub _nr_ptes: u32,
 }
 
 impl Cmdq {
+    /// Offset of the data after the PTEs.
+    const POST_PTE_OFFSET: usize = core::mem::offset_of!(GspMem, cpuq);
+
+    /// Offset of command queue ring buffer.
+    pub(crate) const CMDQ_OFFSET: usize = core::mem::offset_of!(GspMem, cpuq)
+        + core::mem::offset_of!(Msgq, msgq)
+        - Self::POST_PTE_OFFSET;
+
+    /// Offset of message queue ring buffer.
+    pub(crate) const STATQ_OFFSET: usize = core::mem::offset_of!(GspMem, gspq)
+        + core::mem::offset_of!(Msgq, msgq)
+        - Self::POST_PTE_OFFSET;
+
+    /// Number of page table entries for the GSP shared region.
+    pub(crate) const NUM_PTES: usize = size_of::<GspMem>() >> GSP_PAGE_SHIFT;
+
     pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> {
         let gsp_mem = DmaGspMem::new(dev)?;
         let nr_ptes = size_of::<GspMem>() >> GSP_PAGE_SHIFT;
@@ -215,7 +230,6 @@ pub(crate) fn new(dev: &device::Device<device::Bound>) -> Result<Cmdq> {
             dev: dev.into(),
             seq: 0,
             gsp_mem,
-            _nr_ptes: nr_ptes as u32,
         })
     }
 
@@ -399,4 +413,8 @@ pub(crate) fn receive_msg_from_gsp<M: MessageFromGsp, R>(
             .advance_cpu_read_ptr(msg_header.length().div_ceil(GSP_PAGE_SIZE as u32));
         result
     }
+
+    pub(crate) fn dma_handle(&self) -> DmaAddress {
+        self.gsp_mem.0.dma_handle()
+    }
 }
drivers/gpu/nova-core/gsp/fw.rs +60 -0

69 lines not on this page · raw message

drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +33 -0
index 3d96d91e5b12..b87c4e6cb857 100644
@@ -319,6 +319,39 @@ fn fmt(&self, fmt: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result {
 pub const NV_VGPU_MSG_EVENT_NUM_EVENTS: _bindgen_ty_3 = 4131;
 pub type _bindgen_ty_3 = ffi::c_uint;
 #[repr(C)]
+#[derive(Debug, Default, Copy, Clone)]
+pub struct MESSAGE_QUEUE_INIT_ARGUMENTS {
+    pub sharedMemPhysAddr: u64_,
+    pub pageTableEntryCount: u32_,
+    pub __bindgen_padding_0: [u8; 4usize],
+    pub cmdQueueOffset: u64_,
+    pub statQueueOffset: u64_,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone)]
+pub struct GSP_SR_INIT_ARGUMENTS {
+    pub oldLevel: u32_,
+    pub flags: u32_,
+    pub bInPMTransition: u8_,
+    pub __bindgen_padding_0: [u8; 3usize],
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone)]
+pub struct GSP_ARGUMENTS_CACHED {
+    pub messageQueueInitArguments: MESSAGE_QUEUE_INIT_ARGUMENTS,
+    pub srInitArguments: GSP_SR_INIT_ARGUMENTS,
+    pub gpuInstance: u32_,
+    pub bDmemStack: u8_,
+    pub __bindgen_padding_0: [u8; 7usize],
+    pub profilerArgs: GSP_ARGUMENTS_CACHED__bindgen_ty_1,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone)]
+pub struct GSP_ARGUMENTS_CACHED__bindgen_ty_1 {
+    pub pa: u64_,
+    pub size: u64_,
+}
+#[repr(C)]
 #[derive(Copy, Clone)]
 pub union rpc_message_rpc_union_field_v03_00 {
     pub spare: u32_,
Signature · 1 line
2.50.1
#12 lore raw

PATCH v3 8/13 gpu: nova-core: Add bindings and accessors for GspSystemInfo

In reply to Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

---

It would be good to move to using the init! macros at some point, but I couldn't figure out how to make that work to initialise an enum rather than a struct as is required for the transparent representation.

Changes for v3:
- New for v3

---
 drivers/gpu/nova-core/gsp/fw.rs               |   1 +
 drivers/gpu/nova-core/gsp/fw/commands.rs      |  40 ++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 132 ++++++++++++++++++
 3 files changed, 173 insertions(+)
 create mode 100644 drivers/gpu/nova-core/gsp/fw/commands.rs
drivers/gpu/nova-core/gsp/fw.rs +1 -0
index aec0db50adea..2ef9d4acd6f9 100644
@@ -1,5 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
+pub(crate) mod commands;
 mod r570_144;
 
 // Alias to avoid repeating the version number with every use.
drivers/gpu/nova-core/gsp/fw/commands.rs new file +40 -0
new file mode 100644
index 000000000000..f28779af2620
@@ -0,0 +1,40 @@
+use super::bindings;
+
+use kernel::prelude::*;
+use kernel::transmute::{AsBytes, FromBytes};
+use kernel::{device, pci};
+
+#[repr(transparent)]
+pub(crate) struct GspSystemInfo(bindings::GspSystemInfo);
+
+impl GspSystemInfo {
+    pub(crate) fn init(&mut self, dev: &pci::Device<device::Bound>) -> Result {
+        self.0.gpuPhysAddr = dev.resource_start(0)?;
+        self.0.gpuPhysFbAddr = dev.resource_start(1)?;
+        self.0.gpuPhysInstAddr = dev.resource_start(3)?;
+        self.0.nvDomainBusDeviceFunc = u64::from(dev.dev_id());
+
+        // Using TASK_SIZE in r535_gsp_rpc_set_system_info() seems wrong because
+        // TASK_SIZE is per-task. That's probably a design issue in GSP-RM though.
+        self.0.maxUserVa = (1 << 47) - 4096;
+        self.0.pciConfigMirrorBase = 0x088000;
+        self.0.pciConfigMirrorSize = 0x001000;
+
+        self.0.PCIDeviceID = (u32::from(dev.device_id()) << 16) | u32::from(dev.vendor_id());
+        self.0.PCISubDeviceID =
+            (u32::from(dev.subsystem_device_id()) << 16) | u32::from(dev.subsystem_vendor_id());
+        self.0.PCIRevisionID = u32::from(dev.revision_id());
+        self.0.bIsPrimary = 0;
+        self.0.bPreserveVideoMemoryAllocations = 0;
+
+        Ok(())
+    }
+}
+
+// SAFETY: These structs don't meet the no-padding requirements of AsBytes but
+//         that is not a problem because they are not used outside the kernel.
+unsafe impl AsBytes for GspSystemInfo {}
+
+// SAFETY: These structs don't meet the no-padding requirements of FromBytes but
+//         that is not a problem because they are not used outside the kernel.
+unsafe impl FromBytes for GspSystemInfo {}
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +132 -0

138 lines not on this page · raw message

Signature · 1 line
2.50.1
#13 lore raw

Re: [PATCH v3 08/13] gpu: nova-core: Add bindings and accessors for GspSystemInfo

Hi Alistair, (+Benno as this concerns the init! macros)

Quoted 9 lines · Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

---

It would be good to move to using the init! macros at some point, but I couldn't figure out how to make that work to initialise an enum rather than a struct as is required for the transparent representation.

Indeed we have to jump through a few (minor) hoops.

First the init! macros do not seem to support tuple structs. They match a { after the type name, which is not present in GspSystemInfo. By turning it into a regular struct with a single field, we can overcome this, and it doesn't affect the layout the #[repr(transparent)] can still be used.

Then, due to a limitation with declarative macros, init! interprets :: as a separator for generic arguments, so bindings::GspSystemInfo also doesn't parse. Here the trick is to use a local type alias.

After overcoming these two, I have been able to make GspSystemInfo::init return an in-place initializer. It is then a matter of changing send_gsp_command to accept it - please apply the following patch on top of your series for an illustration of how it can be done.

Note that I have added a new generic argument for the error returned by the Init - this is to let us also use infallible initializers transparently. The cool thing is that callers don't need to specify any generic argument since they can be inferred automatically.

drivers/gpu/nova-core/gsp/cmdq.rs +25 -7
index 5580eaf52f7b..0709153f9dc9 100644
@@ -247,12 +247,20 @@ fn notify_gsp(bar: &Bar0) {
         NV_PGSP_QUEUE_HEAD::default().set_address(0).write(bar);
     }
 
-    pub(crate) fn send_gsp_command<M: CommandToGsp>(
+    pub(crate) fn send_gsp_command<M, E>(
         &mut self,
         bar: &Bar0,
         payload_size: usize,
-        init: impl FnOnce(&mut M, SBuffer<core::array::IntoIter<&mut [u8], 2>>) -> Result,
-    ) -> Result {
+        init: impl Init<M, E>,
+        init_sbuffer: impl FnOnce(SBuffer<core::array::IntoIter<&mut [u8], 2>>) -> Result,
+    ) -> Result
+    where
+        M: CommandToGsp,
+        // This allows all error types, including `Infallible`, to be used with `init`. Without
+        // this we cannot use regular stack objects as `init` since their `Init` implementation
+        // does not return any error.
+        Error: From<E>,
+    {
         // TODO: a method that extracts the regions for a given command?
         // ... and another that reduces the region to a given number of bytes!
         let driver_area = self.gsp_mem.driver_write_area();
@@ -264,7 +272,7 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
             return Err(EAGAIN);
         }
 
-        let (msg_header, cmd, payload_1, payload_2) = {
+        let (msg_header, cmd_ptr, payload_1, payload_2) = {
             #[allow(clippy::incompatible_msrv)]
             let (msg_header_slice, slice_1) = driver_area
                 .0
@@ -272,7 +280,6 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
                 .split_at_mut(size_of::<GspMsgElement>());
             let msg_header = GspMsgElement::from_bytes_mut(msg_header_slice).ok_or(EINVAL)?;
             let (cmd_slice, payload_1) = slice_1.split_at_mut(size_of::<M>());
-            let cmd = M::from_bytes_mut(cmd_slice).ok_or(EINVAL)?;
             #[allow(clippy::incompatible_msrv)]
             let payload_2 = driver_area.1.as_flattened_mut();
             // TODO: Replace this workaround to cut the payload size.
@@ -283,11 +290,22 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
                 None => (&mut payload_1[..payload_size], payload_2),
             };
 
-            (msg_header, cmd, payload_1, payload_2)
+            (
+                msg_header,
+                cmd_slice.as_mut_ptr().cast(),
+                payload_1,
+                payload_2,
+            )
+        };
+
+        let cmd = unsafe {
+            init.__init(cmd_ptr)?;
+            // Convert the pointer backto a reference for checksum.
+            &mut *cmd_ptr
         };
 
         let sbuffer = SBuffer::new_writer([&mut payload_1[..], &mut payload_2[..]]);
-        init(cmd, sbuffer)?;
+        init_sbuffer(sbuffer)?;
 
         *msg_header =
             GspMsgElement::new(self.seq, size_of::<M>() + payload_size, M::FUNCTION as u32);
drivers/gpu/nova-core/gsp/commands.rs +7 -5
index 69df8d4be353..6f1be9078853 100644
@@ -79,10 +79,12 @@ pub(crate) fn build_registry(cmdq: &mut Cmdq, bar: &Bar0) -> Result {
         ],
     };
 
-    cmdq.send_gsp_command::<PackedRegistryTable>(bar, registry.size(), |table, sbuffer| {
-        *table = PackedRegistryTable::new(GSP_REGISTRY_NUM_ENTRIES as u32, registry.size() as u32);
-        registry.write_payload(sbuffer)
-    })
+    cmdq.send_gsp_command(
+        bar,
+        registry.size(),
+        PackedRegistryTable::new(GSP_REGISTRY_NUM_ENTRIES as u32, registry.size() as u32),
+        |sbuffer| registry.write_payload(sbuffer),
+    )
 }
 
 impl CommandToGsp for GspSystemInfo {
@@ -95,7 +97,7 @@ pub(crate) fn set_system_info(
     bar: &Bar0,
 ) -> Result {
     build_assert!(size_of::<GspSystemInfo>() < GSP_PAGE_SIZE);
-    cmdq.send_gsp_command::<GspSystemInfo>(bar, 0, |info, _| GspSystemInfo::init(info, dev))?;
+    cmdq.send_gsp_command(bar, 0, GspSystemInfo::init(dev), |_| Ok(()))?;
 
     Ok(())
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +38 -19

68 lines not on this page · raw message

#14 lore raw

Re: [PATCH v3 08/13] gpu: nova-core: Add bindings and accessors for GspSystemInfo

On 2025-10-02 at 23:49 +1000, Alexandre Courbot <[email protected]> wrote...

Quoted 13 lines

Hi Alistair, (+Benno as this concerns the init! macros)

Quoted 9 lines · Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

---

It would be good to move to using the init! macros at some point, but I couldn't figure out how to make that work to initialise an enum rather than a struct as is required for the transparent representation.

Quoted 7 lines

Indeed we have to jump through a few (minor) hoops.

First the init! macros do not seem to support tuple structs. They match a { after the type name, which is not present in GspSystemInfo. By turning it into a regular struct with a single field, we can overcome this, and it doesn't affect the layout the #[repr(transparent)] can still be used.

I was thinking we should fix the init! macro to support tuple structs. Is there some fundamental reason init! couldn't be modified to support tuple structs? It seems like it would be nicer to fix that limitation rather than work around it here.

Quoted 188 lines

Then, due to a limitation with declarative macros, init! interprets :: as a separator for generic arguments, so bindings::GspSystemInfo also doesn't parse. Here the trick is to use a local type alias.

After overcoming these two, I have been able to make GspSystemInfo::init return an in-place initializer. It is then a matter of changing send_gsp_command to accept it - please apply the following patch on top of your series for an illustration of how it can be done.

Note that I have added a new generic argument for the error returned by the Init - this is to let us also use infallible initializers transparently. The cool thing is that callers don't need to specify any generic argument since they can be inferred automatically.

drivers/gpu/nova-core/gsp/cmdq.rs +25 -7
index 5580eaf52f7b..0709153f9dc9 100644
@@ -247,12 +247,20 @@ fn notify_gsp(bar: &Bar0) {
         NV_PGSP_QUEUE_HEAD::default().set_address(0).write(bar);
     }
 
-    pub(crate) fn send_gsp_command<M: CommandToGsp>(
+    pub(crate) fn send_gsp_command<M, E>(
         &mut self,
         bar: &Bar0,
         payload_size: usize,
-        init: impl FnOnce(&mut M, SBuffer<core::array::IntoIter<&mut [u8], 2>>) -> Result,
-    ) -> Result {
+        init: impl Init<M, E>,
+        init_sbuffer: impl FnOnce(SBuffer<core::array::IntoIter<&mut [u8], 2>>) -> Result,
+    ) -> Result
+    where
+        M: CommandToGsp,
+        // This allows all error types, including `Infallible`, to be used with `init`. Without
+        // this we cannot use regular stack objects as `init` since their `Init` implementation
+        // does not return any error.
+        Error: From<E>,
+    {
         // TODO: a method that extracts the regions for a given command?
         // ... and another that reduces the region to a given number of bytes!
         let driver_area = self.gsp_mem.driver_write_area();
@@ -264,7 +272,7 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
             return Err(EAGAIN);
         }
 
-        let (msg_header, cmd, payload_1, payload_2) = {
+        let (msg_header, cmd_ptr, payload_1, payload_2) = {
             #[allow(clippy::incompatible_msrv)]
             let (msg_header_slice, slice_1) = driver_area
                 .0
@@ -272,7 +280,6 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
                 .split_at_mut(size_of::<GspMsgElement>());
             let msg_header = GspMsgElement::from_bytes_mut(msg_header_slice).ok_or(EINVAL)?;
             let (cmd_slice, payload_1) = slice_1.split_at_mut(size_of::<M>());
-            let cmd = M::from_bytes_mut(cmd_slice).ok_or(EINVAL)?;
             #[allow(clippy::incompatible_msrv)]
             let payload_2 = driver_area.1.as_flattened_mut();
             // TODO: Replace this workaround to cut the payload size.
@@ -283,11 +290,22 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
                 None => (&mut payload_1[..payload_size], payload_2),
             };
 
-            (msg_header, cmd, payload_1, payload_2)
+            (
+                msg_header,
+                cmd_slice.as_mut_ptr().cast(),
+                payload_1,
+                payload_2,
+            )
+        };
+
+        let cmd = unsafe {
+            init.__init(cmd_ptr)?;
+            // Convert the pointer backto a reference for checksum.
+            &mut *cmd_ptr
         };
 
         let sbuffer = SBuffer::new_writer([&mut payload_1[..], &mut payload_2[..]]);
-        init(cmd, sbuffer)?;
+        init_sbuffer(sbuffer)?;
 
         *msg_header =
             GspMsgElement::new(self.seq, size_of::<M>() + payload_size, M::FUNCTION as u32);
drivers/gpu/nova-core/gsp/commands.rs +7 -5
index 69df8d4be353..6f1be9078853 100644
@@ -79,10 +79,12 @@ pub(crate) fn build_registry(cmdq: &mut Cmdq, bar: &Bar0) -> Result {
         ],
     };
 
-    cmdq.send_gsp_command::<PackedRegistryTable>(bar, registry.size(), |table, sbuffer| {
-        *table = PackedRegistryTable::new(GSP_REGISTRY_NUM_ENTRIES as u32, registry.size() as u32);
-        registry.write_payload(sbuffer)
-    })
+    cmdq.send_gsp_command(
+        bar,
+        registry.size(),
+        PackedRegistryTable::new(GSP_REGISTRY_NUM_ENTRIES as u32, registry.size() as u32),
+        |sbuffer| registry.write_payload(sbuffer),
+    )
 }
 
 impl CommandToGsp for GspSystemInfo {
@@ -95,7 +97,7 @@ pub(crate) fn set_system_info(
     bar: &Bar0,
 ) -> Result {
     build_assert!(size_of::<GspSystemInfo>() < GSP_PAGE_SIZE);
-    cmdq.send_gsp_command::<GspSystemInfo>(bar, 0, |info, _| GspSystemInfo::init(info, dev))?;
+    cmdq.send_gsp_command(bar, 0, GspSystemInfo::init(dev), |_| Ok(()))?;
 
     Ok(())
 }
drivers/gpu/nova-core/gsp/fw/commands.rs +38 -19
index 83c2b017c4cb..e69be2f422f2 100644
@@ -4,31 +4,50 @@
 use kernel::transmute::{AsBytes, FromBytes};
 use kernel::{device, pci};
 
+// Ideally we would derive this for all our bindings, using the same technique as
+// https://lore.kernel.org/rust-for-linux/[email protected]/
+unsafe impl Zeroable for bindings::GspSystemInfo {}
+
 #[repr(transparent)]
-pub(crate) struct GspSystemInfo(bindings::GspSystemInfo);
+pub(crate) struct GspSystemInfo {
+    // `try_init!` doesn't seem to work with tuple structs. Work around this by declaring a regular
+    // field, which comes down to exactly the same.
+    inner: bindings::GspSystemInfo,
+}
 
 impl GspSystemInfo {
-    pub(crate) fn init(&mut self, dev: &pci::Device<device::Bound>) -> Result {
-        self.0.gpuPhysAddr = dev.resource_start(0)?;
-        self.0.gpuPhysFbAddr = dev.resource_start(1)?;
-        self.0.gpuPhysInstAddr = dev.resource_start(3)?;
-        self.0.nvDomainBusDeviceFunc = u64::from(dev.dev_id());
+    #[allow(non_snake_case)]
+    pub(crate) fn init<'a>(dev: &'a pci::Device<device::Bound>) -> impl Init<Self, Error> + 'a {
+        // `try_init!` interprets `::` as a separator for generics, use a type alias to remove
+        // them.
+        type InnerGspSystemInfo = bindings::GspSystemInfo;
 
-        // Using TASK_SIZE in r535_gsp_rpc_set_system_info() seems wrong because
-        // TASK_SIZE is per-task. That's probably a design issue in GSP-RM though.
-        self.0.maxUserVa = (1 << 47) - 4096;
-        self.0.pciConfigMirrorBase = 0x088000;
-        self.0.pciConfigMirrorSize = 0x001000;
+        // Initializer for the bindings type.
+        let init_inner = try_init!(InnerGspSystemInfo {
+            gpuPhysAddr: dev.resource_start(0)?,
+            gpuPhysFbAddr: dev.resource_start(1)?,
+            gpuPhysInstAddr: dev.resource_start(3)?,
+            nvDomainBusDeviceFunc: u64::from(dev.dev_id()),
 
-        self.0.PCIDeviceID =
-            (u32::from(dev.device_id()) << 16) | u32::from(dev.vendor_id().as_raw());
-        self.0.PCISubDeviceID =
-            (u32::from(dev.subsystem_device_id()) << 16) | u32::from(dev.subsystem_vendor_id());
-        self.0.PCIRevisionID = u32::from(dev.revision_id());
-        self.0.bIsPrimary = 0;
-        self.0.bPreserveVideoMemoryAllocations = 0;
+            // Using TASK_SIZE in r535_gsp_rpc_set_system_info() seems wrong because
+            // TASK_SIZE is per-task. That's probably a design issue in GSP-RM though.
+            maxUserVa: (1 << 47) - 4096,
+            pciConfigMirrorBase: 0x088000,
+            pciConfigMirrorSize: 0x001000,
 
-        Ok(())
+            PCIDeviceID: (u32::from(dev.device_id()) << 16) | u32::from(dev.vendor_id().as_raw()),
+            PCISubDeviceID: (u32::from(dev.subsystem_device_id()) << 16)
+                | u32::from(dev.subsystem_vendor_id()),
+            PCIRevisionID: u32::from(dev.revision_id()),
+            bIsPrimary: 0,
+            bPreserveVideoMemoryAllocations: 0,
+            ..Zeroable::init_zeroed()
+        });
+
+        // Final initializer for our type.
+        try_init!(GspSystemInfo {
+            inner <- init_inner,
+        })
     }
 }
#15 lore raw

Re: [PATCH v3 08/13] gpu: nova-core: Add bindings and accessors for GspSystemInfo

Quoted 22 lines · Alistair Popple

On 2025-10-02 at 23:49 +1000, Alexandre Courbot <[email protected]> wrote...

Quoted 13 lines

Hi Alistair, (+Benno as this concerns the init! macros)

Quoted 9 lines · Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

---

It would be good to move to using the init! macros at some point, but I couldn't figure out how to make that work to initialise an enum rather than a struct as is required for the transparent representation.

Quoted 7 lines

Indeed we have to jump through a few (minor) hoops.

First the init! macros do not seem to support tuple structs. They match a { after the type name, which is not present in GspSystemInfo. By turning it into a regular struct with a single field, we can overcome this, and it doesn't affect the layout the #[repr(transparent)] can still be used.

Quoted 4 lines

I was thinking we should fix the init! macro to support tuple structs. Is there some fundamental reason init! couldn't be modified to support tuple structs? It seems like it would be nicer to fix that limitation rather than work around it here.

I took a look at it and quickly got lost in the macros's internals. :) Let's see what Benno has to say about this. In the meantime, using a single member is just as valid a constructs as a tuple struct for us.

reviewer
#16 lore raw

Re: [PATCH v3 08/13] gpu: nova-core: Add bindings and accessors for GspSystemInfo

In reply to Alexandre Courbot
Quoted 13 lines · Alexandre Courbot

Hi Alistair, (+Benno as this concerns the init! macros)

Quoted 9 lines · Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

---

It would be good to move to using the init! macros at some point, but I couldn't figure out how to make that work to initialise an enum rather than a struct as is required for the transparent representation.

Quoted 7 lines

Indeed we have to jump through a few (minor) hoops.

First the init! macros do not seem to support tuple structs. They match a { after the type name, which is not present in GspSystemInfo. By turning it into a regular struct with a single field, we can overcome this, and it doesn't affect the layout the #[repr(transparent)] can still be used.

Yeah that's the correct workaround at the moment. I'm tracking support for tuple structs in [1]. Essentially the problem is that it requires lots of effort to parse tuple structs using declarative macros. We will get syn this cycle, which will enable me to support several things, including tuple structs.

[1]: https://github.com/Rust-for-Linux/pin-init/issues/85

Quoted 3 lines

Then, due to a limitation with declarative macros, init! interprets :: as a separator for generic arguments, so bindings::GspSystemInfo also doesn't parse. Here the trick is to use a local type alias.

This one will also be solved when we switch to syn.

Quoted 61 lines

After overcoming these two, I have been able to make GspSystemInfo::init return an in-place initializer. It is then a matter of changing send_gsp_command to accept it - please apply the following patch on top of your series for an illustration of how it can be done.

Note that I have added a new generic argument for the error returned by the Init - this is to let us also use infallible initializers transparently. The cool thing is that callers don't need to specify any generic argument since they can be inferred automatically.

drivers/gpu/nova-core/gsp/cmdq.rs +12 -6
index 5580eaf52f7b..0709153f9dc9 100644
@@ -247,12 +247,20 @@ fn notify_gsp(bar: &Bar0) {
         NV_PGSP_QUEUE_HEAD::default().set_address(0).write(bar);
     }
 
-    pub(crate) fn send_gsp_command<M: CommandToGsp>(
+    pub(crate) fn send_gsp_command<M, E>(
         &mut self,
         bar: &Bar0,
         payload_size: usize,
-        init: impl FnOnce(&mut M, SBuffer<core::array::IntoIter<&mut [u8], 2>>) -> Result,
-    ) -> Result {
+        init: impl Init<M, E>,
+        init_sbuffer: impl FnOnce(SBuffer<core::array::IntoIter<&mut [u8], 2>>) -> Result,
+    ) -> Result
+    where
+        M: CommandToGsp,
+        // This allows all error types, including `Infallible`, to be used with `init`. Without
+        // this we cannot use regular stack objects as `init` since their `Init` implementation
+        // does not return any error.
+        Error: From<E>,
+    {
         // TODO: a method that extracts the regions for a given command?
         // ... and another that reduces the region to a given number of bytes!
         let driver_area = self.gsp_mem.driver_write_area();
@@ -264,7 +272,7 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
             return Err(EAGAIN);
         }
 
-        let (msg_header, cmd, payload_1, payload_2) = {
+        let (msg_header, cmd_ptr, payload_1, payload_2) = {
             #[allow(clippy::incompatible_msrv)]
             let (msg_header_slice, slice_1) = driver_area
                 .0
@@ -272,7 +280,6 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
                 .split_at_mut(size_of::<GspMsgElement>());
             let msg_header = GspMsgElement::from_bytes_mut(msg_header_slice).ok_or(EINVAL)?;
             let (cmd_slice, payload_1) = slice_1.split_at_mut(size_of::<M>());
-            let cmd = M::from_bytes_mut(cmd_slice).ok_or(EINVAL)?;
             #[allow(clippy::incompatible_msrv)]
             let payload_2 = driver_area.1.as_flattened_mut();
             // TODO: Replace this workaround to cut the payload size.
@@ -283,11 +290,22 @@ pub(crate) fn send_gsp_command<M: CommandToGsp>(
                 None => (&mut payload_1[..payload_size], payload_2),
             };
 
-            (msg_header, cmd, payload_1, payload_2)

drivers/gpu/nova-core/gsp/cmdq.rs pub(crate) fn send_gsp_command<M: CommandToGsp>( · line 302

Quoted 10 lines
drivers/gpu/nova-core/gsp/cmdq.rs +10 -0
+            (
+                msg_header,
+                cmd_slice.as_mut_ptr().cast(),
+                payload_1,
+                payload_2,
+            )
+        };
+
+        let cmd = unsafe {
+            init.__init(cmd_ptr)?;

This is missing a safety comment. I haven't looked at this locally, so I don't know what is happening in the 10-20 lines that aren't shown, so I don't know if this is correct (if you're only assuming its initialized after this line completes then it's fine). The rest of the patch looks normal.

Hope this helps!

---
Cheers,
Benno

Quoted 3 lines
drivers/gpu/nova-core/gsp/cmdq.rs +2 -0
+            // Convert the pointer backto a reference for checksum.
+            &mut *cmd_ptr
         };
#17 lore raw

Re: [PATCH v3 08/13] gpu: nova-core: Add bindings and accessors for GspSystemInfo

Quoted 28 lines · Benno Lossin
Quoted 13 lines · Alexandre Courbot

Hi Alistair, (+Benno as this concerns the init! macros)

Quoted 9 lines · Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

---

It would be good to move to using the init! macros at some point, but I couldn't figure out how to make that work to initialise an enum rather than a struct as is required for the transparent representation.

Quoted 7 lines

Indeed we have to jump through a few (minor) hoops.

First the init! macros do not seem to support tuple structs. They match a { after the type name, which is not present in GspSystemInfo. By turning it into a regular struct with a single field, we can overcome this, and it doesn't affect the layout the #[repr(transparent)] can still be used.

Yeah that's the correct workaround at the moment. I'm tracking support for tuple structs in [1]. Essentially the problem is that it requires lots of effort to parse tuple structs using declarative macros. We will get syn this cycle, which will enable me to support several things, including tuple structs.

Quoted 7 lines

[1]: https://github.com/Rust-for-Linux/pin-init/issues/85

Quoted 3 lines

Then, due to a limitation with declarative macros, init! interprets :: as a separator for generic arguments, so bindings::GspSystemInfo also doesn't parse. Here the trick is to use a local type alias.

This one will also be solved when we switch to syn.

I was planning to submit
https://github.com/AsahiLinux/linux/commit/2d95fd3b6c359634a0976f27f7a3c667826256da
https://github.com/AsahiLinux/linux/commit/515638cb47cf0ebdac378686fcbbdc6a8364096a
from the asahi downstream tree after 6.18-rc1. Does that still make sense timing wise?

Types with type paths are used extensively in the asahi driver but I can initially work around that.

Janne

reviewer
#18 lore raw

Re: [PATCH v3 08/13] gpu: nova-core: Add bindings and accessors for GspSystemInfo

Quoted 7 lines · Janne Grunau
Quoted 6 lines · Benno Lossin
Quoted 5 lines · Alexandre Courbot

Hi Alistair, (+Benno as this concerns the init! macros)

Quoted 2 lines · Alistair Popple

Adds bindings and an in-place initialiser for the GspSystemInfo struct.

Quoted 7 lines
Quoted 7 lines
Quoted 7 lines
Quoted 7 lines

---

It would be good to move to using the init! macros at some point, but I couldn't figure out how to make that work to initialise an enum rather than a struct as is required for the transparent representation.

Oh by the way, enums are not supported due to a language limitation, see:

https://github.com/Rust-for-Linux/pin-init/issues/59

Quoted 20 lines
Quoted 20 lines
Quoted 7 lines

Indeed we have to jump through a few (minor) hoops.

First the init! macros do not seem to support tuple structs. They match a { after the type name, which is not present in GspSystemInfo. By turning it into a regular struct with a single field, we can overcome this, and it doesn't affect the layout the #[repr(transparent)] can still be used.

Yeah that's the correct workaround at the moment. I'm tracking support for tuple structs in [1]. Essentially the problem is that it requires lots of effort to parse tuple structs using declarative macros. We will get syn this cycle, which will enable me to support several things, including tuple structs.

[1]: https://github.com/Rust-for-Linux/pin-init/issues/85

Quoted 3 lines

Then, due to a limitation with declarative macros, init! interprets :: as a separator for generic arguments, so bindings::GspSystemInfo also doesn't parse. Here the trick is to use a local type alias.

Quoted 7 lines
Quoted 1 line

This one will also be solved when we switch to syn.

I was planning to submit
https://github.com/AsahiLinux/linux/commit/2d95fd3b6c359634a0976f27f7a3c667826256da
https://github.com/AsahiLinux/linux/commit/515638cb47cf0ebdac378686fcbbdc6a8364096a
from the asahi downstream tree after 6.18-rc1. Does that still make sense timing wise?

Probably not, since I'll depend on the syn patches this cycle which will mean that pin-init supports tuples in 6.19.

Quoted 2 lines

Types with type paths are used extensively in the asahi driver but I can initially work around that.

Yeah they should be supported simply by moving to syn, hope it doesn't introduce too much pain in the next cycle.

---
Cheers,
Benno

#19 lore raw

PATCH v3 9/13 gpu: nova-core: Add bindings for the GSP RM registry tables

In reply to Alistair Popple

Adds bindings and constructors for PACKED_REGISTRY_TABLE and PACKED_REGISTRY_ENTRY structures.

---

Changes for v3:
- New for v3

---
 drivers/gpu/nova-core/gsp/fw/commands.rs      | 40 +++++++++++++++++++
 .../gpu/nova-core/gsp/fw/r570_144/bindings.rs | 17 ++++++++
 2 files changed, 57 insertions(+)
drivers/gpu/nova-core/gsp/fw/commands.rs +40 -0
index f28779af2620..3037425902f7 100644
@@ -38,3 +38,43 @@ unsafe impl AsBytes for GspSystemInfo {}
 // SAFETY: These structs don't meet the no-padding requirements of FromBytes but
 //         that is not a problem because they are not used outside the kernel.
 unsafe impl FromBytes for GspSystemInfo {}
+
+#[repr(transparent)]
+pub(crate) struct PackedRegistryEntry(bindings::PACKED_REGISTRY_ENTRY);
+
+impl PackedRegistryEntry {
+    pub(crate) fn new(offset: u32, value: u32) -> Self {
+        Self({
+            bindings::PACKED_REGISTRY_ENTRY {
+                nameOffset: offset,
+                type_: bindings::REGISTRY_TABLE_ENTRY_TYPE_DWORD as u8,
+                __bindgen_padding_0: Default::default(),
+                data: value,
+                length: 0,
+            }
+        })
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for PackedRegistryEntry {}
+
+#[repr(transparent)]
+pub(crate) struct PackedRegistryTable(bindings::PACKED_REGISTRY_TABLE);
+
+impl PackedRegistryTable {
+    pub(crate) fn new(num_entries: u32, size: u32) -> Self {
+        Self(bindings::PACKED_REGISTRY_TABLE {
+            numEntries: num_entries,
+            size,
+            entries: Default::default(),
+        })
+    }
+}
+
+// SAFETY: Padding is explicit and will not contain uninitialized data.
+unsafe impl AsBytes for PackedRegistryTable {}
+
+// SAFETY: This struct only contains integer types for which all bit patterns
+// are valid.
+unsafe impl FromBytes for PackedRegistryTable {}
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +17 -0
index 427fff82f7c1..7ad1981e471c 100644
@@ -42,6 +42,7 @@ fn fmt(&self, fmt: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result {
 pub const GSP_FW_HEAP_SIZE_OVERRIDE_LIBOS3_BAREMETAL_MAX_MB: u32 = 280;
 pub const GSP_FW_WPR_META_REVISION: u32 = 1;
 pub const GSP_FW_WPR_META_MAGIC: i64 = -2577556379034558285;
+pub const REGISTRY_TABLE_ENTRY_TYPE_DWORD: u32 = 1;
 pub type __u8 = ffi::c_uchar;
 pub type __u16 = ffi::c_ushort;
 pub type __u32 = ffi::c_uint;
@@ -649,6 +650,22 @@ pub struct LibosMemoryRegionInitArgument {
 }
 #[repr(C)]
 #[derive(Debug, Default, Copy, Clone)]
+pub struct PACKED_REGISTRY_ENTRY {
+    pub nameOffset: u32_,
+    pub type_: u8_,
+    pub __bindgen_padding_0: [u8; 3usize],
+    pub data: u32_,
+    pub length: u32_,
+}
+#[repr(C)]
+#[derive(Debug, Default)]
+pub struct PACKED_REGISTRY_TABLE {
+    pub size: u32_,
+    pub numEntries: u32_,
+    pub entries: __IncompleteArrayField<PACKED_REGISTRY_ENTRY>,
+}
+#[repr(C)]
+#[derive(Debug, Default, Copy, Clone)]
 pub struct msgqTxHeader {
     pub version: u32_,
     pub size: u32_,
Signature · 1 line
2.50.1
#20 lore raw

PATCH v3 10/13 gpu: nova-core: gsp: Create RM registry and sysinfo commands

In reply to Alistair Popple

Add the RM registry and system information commands that enable the host driver to configure GSP firmware parameters during initialization.

The RM registry is serialized into a packed format and sent via the command queue. For now only two parameters which are required to boot GSP are hardcoded. In future a kernel module parameter will be added to enable other parameters to be added.

Also add the system info command, which provides required hardware information to the GSP. These commands use the GSP command queue infrastructure to issue commands to the GSP which is read during GSP boot.

---

Changes for v3:
- Use MsgFunction enum
- Rename GspCmdq to Cmdq
- Rename GspCommandToGsp to CommandToGsp
- Rename GspMessageFromGsp to MessageFromGsp
- Split bindings into separate patch

Changes for v2:
- Rebased on Alex's latest tree

---
 drivers/gpu/nova-core/gsp.rs          |   1 +
 drivers/gpu/nova-core/gsp/boot.rs     |   6 +-
 drivers/gpu/nova-core/gsp/cmdq.rs     |   1 -
 drivers/gpu/nova-core/gsp/commands.rs | 101 ++++++++++++++++++++++++++
 drivers/gpu/nova-core/sbuffer.rs      |   1 -
 5 files changed, 107 insertions(+), 3 deletions(-)
 create mode 100644 drivers/gpu/nova-core/gsp/commands.rs
drivers/gpu/nova-core/gsp.rs +1 -0
index 9e5dd9e5a316..1d5544d9dfb4 100644
@@ -22,6 +22,7 @@
 use fw::LibosMemoryRegionInitArgument;
 
 pub(crate) mod cmdq;
+pub(crate) mod commands;
 
 pub(crate) const GSP_PAGE_SHIFT: usize = 12;
 pub(crate) const GSP_PAGE_SIZE: usize = 1 << GSP_PAGE_SHIFT;
drivers/gpu/nova-core/gsp/boot.rs +5 -1
index 1d2448331d7a..0b306313ec53 100644
@@ -16,6 +16,7 @@
     FIRMWARE_VERSION,
 };
 use crate::gpu::Chipset;
+use crate::gsp::commands::{build_registry, set_system_info};
 use crate::gsp::GspFwWprMeta;
 use crate::regs;
 use crate::vbios::Vbios;
@@ -105,7 +106,7 @@ fn run_fwsec_frts(
     ///
     /// Upon return, the GSP is up and running, and its runtime object given as return value.
     pub(crate) fn boot(
-        self: Pin<&mut Self>,
+        mut self: Pin<&mut Self>,
         pdev: &pci::Device<device::Bound>,
         bar: &Bar0,
         chipset: Chipset,
@@ -139,6 +140,9 @@ pub(crate) fn boot(
             CoherentAllocation::<GspFwWprMeta>::alloc_coherent(dev, 1, GFP_KERNEL | __GFP_ZERO)?;
         dma_write!(wpr_meta[0] = GspFwWprMeta::new(&gsp_fw, &fb_layout))?;
 
+        set_system_info(&mut self.cmdq, pdev, bar)?;
+        build_registry(&mut self.cmdq, bar)?;
+
         Ok(())
     }
 }
drivers/gpu/nova-core/gsp/cmdq.rs +0 -1
index 2fd6b31db9e9..5580eaf52f7b 100644
@@ -247,7 +247,6 @@ fn notify_gsp(bar: &Bar0) {
         NV_PGSP_QUEUE_HEAD::default().set_address(0).write(bar);
     }
 
-    #[expect(unused)]
     pub(crate) fn send_gsp_command<M: CommandToGsp>(
         &mut self,
         bar: &Bar0,
drivers/gpu/nova-core/gsp/commands.rs new file +101 -0

101 lines not on this page · raw message

drivers/gpu/nova-core/sbuffer.rs +0 -1
index bde80cc3fa63..5acfd005a86b 100644
@@ -159,7 +159,6 @@ fn get_slice_mut(&mut self, len: usize) -> Option<&'a mut [u8]> {
 
     /// Ideally we would implement `Write`, but it is not available in `core`.
     /// So mimic `std::io::Write::write_all`.
-    #[expect(unused)]
     pub(crate) fn write_all(&mut self, mut src: &[u8]) -> Result {
         while !src.is_empty() {
             match self.get_slice_mut(src.len()) {
Signature · 1 line
2.50.1
#21 lore raw

Re: [PATCH v3 10/13] gpu: nova-core: gsp: Create RM registry and sysinfo commands

Quoted 5 lines · Alistair Popple

+pub(crate) fn build_registry(cmdq: &mut Cmdq, bar: &Bar0) -> Result { +    let registry = RegistryTable {
+        entries: [
+            RegistryEntry {
+                key: "RMSecBusResetEnable",

Quoted 8 lines

+                value: 1,
+            },
+            RegistryEntry {
+                key: "RMForcePcieConfigSave",
+                value: 1,
+            },
+        ],
+    };

You forgot to include any of my comments from v2:

https://lore.kernel.org/rust-for-linux/[email protected]/

#22 lore raw

PATCH v3 11/13 nova-core: falcon: Add support to check if RISC-V is active

In reply to Alistair Popple

From: Joel Fernandes <[email protected]>

Add definition for RISCV_CPUCTL register and use it in a new falcon API to check if the RISC-V core of a Falcon is active. It is required by the sequencer to know if the GSP's RISCV processor is active.

---
 drivers/gpu/nova-core/falcon.rs | 9 +++++++++
 drivers/gpu/nova-core/regs.rs   | 5 +++++
 2 files changed, 14 insertions(+)
drivers/gpu/nova-core/falcon.rs +9 -0
index 37e6298195e4..c7907f16bcf4 100644
@@ -610,4 +610,13 @@ pub(crate) fn signature_reg_fuse_version(
         self.hal
             .signature_reg_fuse_version(self, bar, engine_id_mask, ucode_id)
     }
+
+    /// Check if the RISC-V core is active.
+    ///
+    /// Returns `true` if the RISC-V core is active, `false` otherwise.
+    #[expect(unused)]
+    pub(crate) fn is_riscv_active(&self, bar: &Bar0) -> Result<bool> {
+        let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL::read(bar, &E::ID);
+        Ok(cpuctl.active_stat())
+    }
 }
drivers/gpu/nova-core/regs.rs +5 -0
index 0585699ae951..5df6a2bf42ad 100644
@@ -324,6 +324,11 @@ pub(crate) fn mem_scrubbing_done(self) -> bool {
 
 // PRISCV
 
+register!(NV_PRISCV_RISCV_CPUCTL @ PFalconBase[0x00001388] {
+    7:7     active_stat as bool;
+    0:0     halted as bool;
+});
+
 register!(NV_PRISCV_RISCV_BCR_CTRL @ PFalconBase[0x00001668] {
     0:0     valid as bool;
     4:4     core_select as bool => PeregrineCoreSelect;
Signature · 1 line
2.50.1
#23 lore raw

Re: [PATCH v3 11/13] nova-core: falcon: Add support to check if RISC-V is active

Quoted 8 lines · Alistair Popple

From: Joel Fernandes <[email protected]>

Add definition for RISCV_CPUCTL register and use it in a new falcon API to check if the RISC-V core of a Falcon is active. It is required by the sequencer to know if the GSP's RISCV processor is active.

You didn't make any of my previously suggested changes.

Quoted 14 lines

---
 drivers/gpu/nova-core/falcon.rs | 9 +++++++++
 drivers/gpu/nova-core/regs.rs   | 5 +++++

 2 files changed, 14 insertions(+)
drivers/gpu/nova-core/falcon.rs +0 -0
index 37e6298195e4..c7907f16bcf4 100644
@@ -610,4 +610,13 @@ pub(crate) fn signature_reg_fuse_version(

         self.hal
             .signature_reg_fuse_version(self, bar, engine_id_mask, ucode_id) }
+

Quoted 8 lines

+    /// Check if the RISC-V core is active.
+    ///
+    /// Returns true if the RISC-V core is active, false otherwise. +    #[expect(unused)]
+    pub(crate) fn is_riscv_active(&self, bar: &Bar0) -> Result<bool> { +        let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL::read(bar, &E::ID); +        Ok(cpuctl.active_stat())
+    }

This should return just bool, not Result<bool>. Either it's active or it's not. There is no third option.

drivers/gpu/nova-core/regs.rs pub(crate) fn mem_scrubbing_done(self) -> bool { · line 330

Quoted 13 lines

 }

drivers/gpu/nova-core/regs.rs +0 -0
index 0585699ae951..5df6a2bf42ad 100644
@@ -324,6 +324,11 @@ pub(crate) fn mem_scrubbing_done(self) -> bool {


 // PRISCV

drivers/gpu/nova-core/regs.rs +4 -0
+register!(NV_PRISCV_RISCV_CPUCTL @ PFalconBase[0x00001388] {
+    7:7     active_stat as bool;
+    0:0     halted as bool;
+});

I think the convention is to list the bits in increase position. That is, 'active_stat' should be on the line below 'halted''

Also I think that this should actually be PFalcon2Base[0x00000388]

Quoted 4 lines

+
 register!(NV_PRISCV_RISCV_BCR_CTRL @ PFalconBase[0x00001668] { 0:0     valid as bool;
     4:4     core_select as bool => PeregrineCoreSelect;

And this should be PFalcon2Base[0x00000668]

#24 lore raw

PATCH v3 12/13 nova-core: falcon: Add support to write firmware version

In reply to Alistair Popple

From: Joel Fernandes <[email protected]>

This will be needed by both the GSP boot code as well as GSP resume code in the sequencer.

---
 drivers/gpu/nova-core/falcon.rs | 9 +++++++++
 drivers/gpu/nova-core/regs.rs   | 6 ++++++
 2 files changed, 15 insertions(+)
drivers/gpu/nova-core/falcon.rs +9 -0
index c7907f16bcf4..0cb7821341ed 100644
@@ -619,4 +619,13 @@ pub(crate) fn is_riscv_active(&self, bar: &Bar0) -> Result<bool> {
         let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL::read(bar, &E::ID);
         Ok(cpuctl.active_stat())
     }
+
+    /// Write the application version to the OS register.
+    #[expect(dead_code)]
+    pub(crate) fn write_os_version(&self, bar: &Bar0, app_version: u32) -> Result<()> {
+        regs::NV_PFALCON_FALCON_OS::default()
+            .set_value(app_version)
+            .write(bar, &E::ID);
+        Ok(())
+    }
 }
drivers/gpu/nova-core/regs.rs +6 -0
index 5df6a2bf42ad..d9212fa50197 100644
@@ -215,6 +215,12 @@ pub(crate) fn vga_workspace_addr(self) -> Option<u64> {
     31:0    value as u32;
 });
 
+// Used to store version information about the firmware running
+// on the Falcon processor.
+register!(NV_PFALCON_FALCON_OS @ PFalconBase[0x00000080] {
+    31:0    value as u32;
+});
+
 register!(NV_PFALCON_FALCON_RM @ PFalconBase[0x00000084] {
     31:0    value as u32;
 });
Signature · 1 line
2.50.1
#25 lore raw

PATCH v3 13/13 nova-core: gsp: Boot GSP

In reply to Alistair Popple

Boot the GSP to the RISC-V active state. Completing the boot requires running the CPU sequencer which will be added in a future commit.

---

Changes for v3:
- Fixed minor nit from John
- Added booter load error thanks to Timur's suggestion

Changes for v2:
- Rebased on Alex's latest tree

---
 drivers/gpu/nova-core/falcon.rs         |  2 -
 drivers/gpu/nova-core/firmware/riscv.rs |  3 +-
 drivers/gpu/nova-core/gsp.rs            |  2 +-
 drivers/gpu/nova-core/gsp/boot.rs       | 64 ++++++++++++++++++++++++-
 4 files changed, 65 insertions(+), 6 deletions(-)
drivers/gpu/nova-core/falcon.rs +0 -2
index 0cb7821341ed..960801f74bf1 100644
@@ -614,14 +614,12 @@ pub(crate) fn signature_reg_fuse_version(
     /// Check if the RISC-V core is active.
     ///
     /// Returns `true` if the RISC-V core is active, `false` otherwise.
-    #[expect(unused)]
     pub(crate) fn is_riscv_active(&self, bar: &Bar0) -> Result<bool> {
         let cpuctl = regs::NV_PRISCV_RISCV_CPUCTL::read(bar, &E::ID);
         Ok(cpuctl.active_stat())
     }
 
     /// Write the application version to the OS register.
-    #[expect(dead_code)]
     pub(crate) fn write_os_version(&self, bar: &Bar0, app_version: u32) -> Result<()> {
         regs::NV_PFALCON_FALCON_OS::default()
             .set_value(app_version)
drivers/gpu/nova-core/firmware/riscv.rs +1 -2
index dec33d2b631a..d1a9e027bac3 100644
@@ -50,7 +50,6 @@ fn new(bin_fw: &BinFirmware<'_>) -> Result<Self> {
 }
 
 /// A parsed firmware for a RISC-V core, ready to be loaded and run.
-#[expect(unused)]
 pub(crate) struct RiscvFirmware {
     /// Offset at which the code starts in the firmware image.
     pub code_offset: u32,
@@ -59,7 +58,7 @@ pub(crate) struct RiscvFirmware {
     /// Offset at which the manifest starts in the firmware image.
     pub manifest_offset: u32,
     /// Application version.
-    app_version: u32,
+    pub app_version: u32,
     /// Device-mapped firmware image.
     pub ucode: DmaObject,
 }
drivers/gpu/nova-core/gsp.rs +1 -1
index 1d5544d9dfb4..d5a33fbf8fb6 100644
@@ -34,7 +34,7 @@
 /// GSP runtime data.
 #[pin_data]
 pub(crate) struct Gsp {
-    libos: CoherentAllocation<LibosMemoryRegionInitArgument>,
+    pub libos: CoherentAllocation<LibosMemoryRegionInitArgument>,
     pub loginit: CoherentAllocation<u8>,
     pub logintr: CoherentAllocation<u8>,
     pub logrm: CoherentAllocation<u8>,
drivers/gpu/nova-core/gsp/boot.rs +63 -1

88 lines not on this page · raw message

Signature · 1 line
2.50.1
#26 lore raw

Re: [PATCH v3 13/13] nova-core: gsp: Boot GSP

Quoted 5 lines · Alistair Popple

+        let (mbox0, mbox1) = gsp_falcon.boot(
+            bar,
+            Some(libos_handle as u32),
+            Some((libos_handle >> 32) as u32),
+        )?;

The only time .boot() is called with None for mbox1 is with fwsec, and I'm pretty sure it doesn't matter if we set mbox1 to 0 in that call. So I suggest that we change boot() to take two u32s instead of two Option<u32> for the mbox registers. It's just unnecessary complication.

maintainer
#27 lore raw
In reply to Alistair Popple
Quoted 9 lines · Alistair Popple

Changes since v2:

The main change since v2 has been to make all firmware bindings completely opaque. It has been made clear this is a pre-requisite for this series to progress upstream as it should make supporting different firmware versions easier in future.

Overall the extra constructors and accessors add a couple of hundred lines of code and a few extra unsafe statements.

A bit of additional code is expected, but it's not clear to me why this would require additional unsafe blocks.

Can you please elaborate?

#28 lore raw

On 2025-09-30 at 23:26 +1000, Danilo Krummrich <[email protected]> wrote...

Quoted 8 lines · Alistair Popple
Quoted 7 lines

Changes since v2:

The main change since v2 has been to make all firmware bindings completely opaque. It has been made clear this is a pre-requisite for this series to progress upstream as it should make supporting different firmware versions easier in future.

Quoted 7 lines
Quoted 2 lines

Overall the extra constructors and accessors add a couple of hundred lines of code and a few extra unsafe statements.

A bit of additional code is expected, but it's not clear to me why this would require additional unsafe blocks.

Can you please elaborate?

See patch 5 and 6 and specifically cpu_read_ptr(), etc. The message headers are embedded in a structure within a DMA coherent mapping, struct DmaGspMem(CoherentAllocation<GspMem>), in cmdq.rs. Obviously we can't call dma_write!() on them because we can't get to the opaque fields, and even getting to the methods requires accessing the GspMem within the CoherentAllocation which requires some unsafe code to get the CPU pointer.

I'm open to suggestions though, maybe I missed something as it was late when I was trying to pull this all togeather.

#29 lore raw
In reply to Alistair Popple
Quoted 4 lines · Alistair Popple

Changes since v2:

The main change since v2 has been to make all firmware bindings completely opaque. It has been made clear this is a pre-requisite for

Any hints about where to see this aspect would be welcome.

...

Quoted 5 lines

A full tree including the prerequisites for this patch series is available at https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream.

[1] - https://lore.kernel.org/rust-for-linux/[email protected]/
[2] - https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream-v2

Could you maybe push up a nova-core-for-upstream-v3 branch please?

thanks,

Signature · 1 line
John Hubbard
#30 lore raw

On 2025-10-01 at 12:02 +1000, John Hubbard <[email protected]> wrote...

Quoted 7 lines · Alistair Popple
Quoted 4 lines

Changes since v2:

The main change since v2 has been to make all firmware bindings completely opaque. It has been made clear this is a pre-requisite for

Any hints about where to see this aspect would be welcome.

The changes to fw.rs in patch 5 for example.

Quoted 9 lines

...

Quoted 5 lines

A full tree including the prerequisites for this patch series is available at https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream.

[1] - https://lore.kernel.org/rust-for-linux/[email protected]/
[2] - https://github.com/apopple-nvidia/linux/tree/nova-core-for-upstream-v2

Could you maybe push up a nova-core-for-upstream-v3 branch please?

Done.

Quoted 4 lines

thanks,

Signature · 1 line
John Hubbard