gpu: nova-core: add two prerequisites for the r000 GSP firmware
Subsystems (7)
cover gpu: nova-core: add two prerequisites for the r000 GSP firmware
Here are two preparatory changes for (or more accurately, from) the r000 GSP firmware series [1], which enables nova-core to work with the GSP firmware for r615+ releases.
Both of these patches stand on their own, so posting them now helps avoid reviewing a huge-er r000 series. :)
* For the Rust PCI abstraction, add an accessor for the PCI domain number.
* Factor out the shutdown path's check for a suspended GSP. The r000 boot sequence needs the same check, so it gets factored out and cleaned up a bit at the same time, by using register!() for the bit 31 access.
This applies to today's drm-rust-next: commit a7cdb17ca0d7.
An LLM coding assistant (Claude Code) helped write both patches, but I have fussed over every line of both the code and the commit messages. The LLM also drafted the first version of this letter, which I've completely rewritten, because it totally missed the point. All of the points. :)
Testing: booted nova-core on my 3-GPU test system (Turing, Ampere and Blackwell GPUs) with all KUnit and built-in Nova tests enabled, as usual.
[1] https://lore.kernel.org/all/[email protected]/
John Hubbard (2):
rust: pci: add domain_nr() accessor
gpu: nova-core: factor out and expose Falcon::is_processor_suspended()
drivers/gpu/nova-core/falcon/gsp.rs | 7 +++++++ drivers/gpu/nova-core/gsp/boot.rs | 6 ++---- drivers/gpu/nova-core/regs.rs | 5 +++++ rust/helpers/pci.c | 5 +++++ rust/kernel/pci.rs | 14 ++++++++++++++ 5 files changed, 33 insertions(+), 4 deletions(-)
base-commit: a7cdb17ca0d7ce67e9f3a52d39edd60f03223729
Signature · 1 line
2.56.0
PATCH 1/2 rust: pci: add domain_nr() accessor
The nova-core driver has to send the GPU's PCI location to the GPU's firmware as one word that holds the domain, the bus and the device number.
The PCI abstraction has dev_id(), which packs the bus, the device and the function into one value. It has no accessor that reports the domain.
Add an accessor for the domain number.
--- rust/helpers/pci.c | 5 +++++ rust/kernel/pci.rs | 14 ++++++++++++++ 2 files changed, 19 insertions(+)
rust/helpers/pci.c +5 -0
@@ -7,6 +7,11 @@ __rust_helper u16 rust_helper_pci_dev_id(struct pci_dev *dev)return PCI_DEVID (dev->bus->number, dev->devfn); } +__rust_helper int rust_helper_pci_domain_nr(struct pci_dev *dev) +{ +return pci_domain_nr (dev->bus); +} + __rust_helper resource_size_trust_helper_pci_resource_start (struct pci_dev *pdev,int bar) {
rust/kernel/pci.rs +14 -0
@@ -408,6 +408,20 @@ pub fn dev_id(&self) -> u16 {unsafe { bindings::pci_dev_id (self.as_raw ()) } } +/// Returns the PCI domain number of the bus that this device is on. +#[inline] +pub fn domain_nr (&self) ->u32 { +// SAFETY: By its type invariant `self.as_raw` is always a valid pointer to a +// `struct pci_dev`. +let domain_nr =unsafe { bindings::pci_domain_nr (self.as_raw ()) }; + +// CAST: `pci_domain_nr()` returns an `int`, but a registered bus never has a negative +// domain, because `pci_register_host_bridge()` fails when the domain allocation fails. +// A negative value here is a kernel bug rather than a condition for a caller to handle. +debug_assert! (domain_nr >=0 ); + domain_nras u32 + } +/// Returns the PCI subsystem vendor ID. #[inline] pub fn subsystem_vendor_id (&self) ->u16 {
Signature · 1 line
2.56.0
[email protected]
AI reviewRe: [PATCH 1/2] rust: pci: add domain_nr() accessor
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider: - [Medium] rust: pci: signature mismatch in rust_helper_pci_domain_nr() on ppc and s390 --
Thank you for your contribution! Sashiko AI review found 1 potential issue to consider:
- [Medium] rust: pci: signature mismatch in rust_helper_pci_domain_nr() on ppc and s390
--
commit 40ebd2410b826b9813de5dae1bbe6075723a08bc
Author: John Hubbard <[email protected]>
rust: pci: add domain_nr() accessor
This commit adds a domain_nr() accessor to pci::Device and a corresponding rust_helper_pci_domain_nr() C helper to retrieve the PCI domain number.
Quoted 21 lines
rust/helpers/pci.c +5
-0@@ -7,6 +7,11 @@ __rust_helper u16 rust_helper_pci_dev_id(struct pci_dev *dev)return PCI_DEVID (dev->bus->number, dev->devfn); } +__rust_helper int rust_helper_pci_domain_nr(struct pci_dev *dev) +{ +return pci_domain_nr (dev->bus); +} + __rust_helper resource_size_trust_helper_pci_resource_start (struct pci_dev *pdev,int bar) {rust/kernel/pci.rs +0
-0
rust/kernel/pci.rs pub fn dev_id(&self) -> u16 { · line 416
Quoted 9 lines
rust/kernel/pci.rs +6
-0@@ -408,6 +408,20 @@ pub fn dev_id(&self) -> u16 {unsafe { bindings::pci_dev_id (self.as_raw ()) } } +/// Returns the PCI domain number of the bus that this device is on. +#[inline] +pub fn domain_nr (&self) ->u32 { +// SAFETY: By its type invariant `self.as_raw` is always a valid pointer to a +// `struct pci_dev`. +let domain_nr =unsafe { bindings::pci_domain_nr (self.as_raw ()) };
[Severity: Medium]
Does bindings::pci_domain_nr(self.as_raw()) fail to compile on PowerPC and s390 due to a pointer type mismatch?
In rust/bindings/lib.rs, bindings_generated.rs is included after pub use super::bindings_helper::*, so any non-inline C function emitted by bindgen shadows a rust_helper_* wrapper of the same stripped name. On PowerPC (arch/powerpc/include/asm/pci.h) and s390 (arch/s390/include/asm/pci.h), pci_domain_nr() is declared as a non-inline extern function taking struct pci_bus * rather than a static inline function:
arch/powerpc/include/asm/pci.h:
extern int pci_domain_nr(struct pci_bus *bus);
On those architectures, bindings::pci_domain_nr resolves to the C declaration expecting *mut bindings::pci_bus instead of the helper's
*mut bindings::pci_dev, breaking the build when CONFIG_RUST and CONFIG_PCI are enabled. Either rust_helper_pci_domain_nr() should take struct pci_bus *bus to match the C signature (with domain_nr() passing (*self.as_raw()).bus), or the helper should use a distinct name that does not collide with C pci_domain_nr().
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
Danilo Krummrich
maintainerRe: [PATCH 1/2] rust: pci: add domain_nr() accessor
In reply to John HubbardQuoted 10 lines · John Hubbard
The nova-core driver has to send the GPU's PCI location to the GPU's firmware as one word that holds the domain, the bus and the device number.
The PCI abstraction has
dev_id(), which packs the bus, the device and the function into one value. It has no accessor that reports the domain.Add an accessor for the domain number.
Suggested-by Miguel Ojeda <[email protected]>
Is this tag intentional? (Just asking since my tooling flagged it as it wasn't present in the first submission of this patch and some other criteria.)
Quoted 11 lines
Signed-off-by John Hubbard <[email protected]>--- rust/helpers/pci.c | 5 +++++ rust/kernel/pci.rs | 14 ++++++++++++++ 2 files changed, 19 insertions(+)rust/helpers/pci.c +0
-0
rust/helpers/pci.c __rust_helper u16 rust_helper_pci_dev_id(struct pci_dev *dev) · line 12
Quoted 6 lines
rust/helpers/pci.c +3
-0@@ -7,6 +7,11 @@ __rust_helper u16 rust_helper_pci_dev_id(struct pci_dev *dev)return PCI_DEVID (dev->bus->number, dev->devfn); } +__rust_helper int rust_helper_pci_domain_nr(struct pci_dev *dev) +{ +return pci_domain_nr (dev->bus);
Rust helpers should generally be transparent wrappers and not provide any semantics, even if it is just to resolve a pointer.
Quoted 15 lines
rust/helpers/pci.c +2
-0+} + __rust_helper resource_size_trust_helper_pci_resource_start (struct pci_dev *pdev,int bar) {rust/kernel/pci.rs +2
-0@@ -408,6 +408,20 @@ pub fn dev_id(&self) -> u16 {unsafe { bindings::pci_dev_id (self.as_raw ()) } } +/// Returns the PCI domain number of the bus that this device is on. +#[inline]
rust/kernel/pci.rs pub fn dev_id(&self) -> u16 { · line 422
Quoted 10 lines
rust/kernel/pci.rs +10
-0+pub fn domain_nr (&self) ->u32 { +// SAFETY: By its type invariant `self.as_raw` is always a valid pointer to a +// `struct pci_dev`. +let domain_nr =unsafe { bindings::pci_domain_nr (self.as_raw ()) }; + +// CAST: `pci_domain_nr()` returns an `int`, but a registered bus never has a negative +// domain, because `pci_register_host_bridge()` fails when the domain allocation fails. +// A negative value here is a kernel bug rather than a condition for a caller to handle. +debug_assert! (domain_nr >=0 ); + domain_nras u32
NIT: Please move the comment directly above the actual cast statement.
Quoted 1 line
rust/kernel/pci.rs +1
-0+ }
PATCH 2/2 gpu: nova-core: factor out and expose Falcon::is_processor_suspended()
In reply to John HubbardThe r000 boot protocol requires the driver to wait for the GSP suspend bit after each load-and-execute event. The shutdown path already does this.
The boot sequence will soon need this same functionality, so factor it out into a method, and put it in it's logical location as well, as a new method of Falcon<Gsp>.
Also, use register!() for this, instead of manually masking off bit 31. As usual, that results in much more readable code.
No functional changes.
--- drivers/gpu/nova-core/falcon/gsp.rs | 7 +++++++ drivers/gpu/nova-core/gsp/boot.rs | 6 ++---- drivers/gpu/nova-core/regs.rs | 5 +++++ 3 files changed, 14 insertions(+), 4 deletions(-)
drivers/gpu/nova-core/falcon/gsp.rs +7 -0
@@ -47,6 +47,13 @@ fn pfalcon2(io: Bar0<'_>) -> Mmio<'_, super::PFalcon2Registers> { }impl <'a>Falcon <'a,Gsp > { +/// Returns `true` if GSP-RM has suspended the RISC-V core. +pub (crate )fn is_processor_suspended (&self) ->bool { + self.pfalcon + .read (regs::NV_PFALCON_FALCON_MAILBOX0_LIBOS ) + .processor_suspended () + } +/// Clears the SWGEN0 latch in the GSP falcon. /// /// While the latch is set, no later message signals the tree, so a caller that consumed a
drivers/gpu/nova-core/gsp/boot.rs +2 -4
@@ -2,7 +2,6 @@// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. use kernel::{ - bits, io::poll::read_poll_timeout, prelude::*, time::Delta ,
@@ -98,10 +97,9 @@ fn shutdown_gsp( cmdq.send_rpc_command (commands::rpc::UnloadingGuestDriver ::new (mode))?;// Wait until GSP signals it is suspended. -const LIBOS_INTERRUPT_PROCESSOR_SUSPENDED :u32 = bits::bit_u32 (31 );read_poll_timeout ( - ||Ok (gsp_falcon.read_mailbox0 ()), - |&mb0| mb0 &LIBOS_INTERRUPT_PROCESSOR_SUSPENDED !=0 , + ||Ok (gsp_falcon.is_processor_suspended ()), + |suspended| *suspended,Delta ::from_millis (10 ),Delta ::from_secs (5 ), )
drivers/gpu/nova-core/regs.rs +5 -0
@@ -147,6 +147,11 @@ pub(crate) fn usable_fb_size(self) -> u64 { 31:0 value => u32; } +/// `MAILBOX0` as LibOS, the kernel that GSP-RM runs on, writes it. + pub(crate) NV_PFALCON_FALCON_MAILBOX0_LIBOS(u32) => NV_PFALCON_FALCON_MAILBOX0 { + 31:31 processor_suspended => bool; + } + pub(crate) NV_PFALCON_FALCON_MAILBOX1(u32) @ 0x00000044 { 31:0 value => u32; }
Signature · 1 line
2.56.0
[email protected]
AI reviewRe: [PATCH 2/2] gpu: nova-core: factor out and expose Falcon::is_processor_suspended()
Sashiko has reviewed this patch and found no issues.
Quoted 8 lines
The r000 boot protocol requires the driver to wait for the GSP suspend bit after each load-and-execute event. The shutdown path already does this.
The boot sequence will soon need this same functionality, so factor it out into a method, and put it in it's logical location as well, as a new method of Falcon<Gsp>.
Quoted 8 lines
Also, use register!() for this, instead of manually masking off bit 31. As usual, that results in much more readable code.
No functional changes.
Co-developed-by Alexandre Courbot <[email protected]>Signed-off-by Alexandre Courbot <[email protected]>Signed-off-by John Hubbard <[email protected]>
Sashiko has reviewed this patch and found no issues.
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Re: [PATCH 2/2] gpu: nova-core: factor out and expose Falcon::is_processor_suspended()
In reply to John HubbardQuoted 11 lines · John Hubbard
The r000 boot protocol requires the driver to wait for the GSP suspend bit after each load-and-execute event. The shutdown path already does this.
The boot sequence will soon need this same functionality, so factor it out into a method, and put it in it's logical location as well, as a new method of Falcon<Gsp>.
Also, use register!() for this, instead of manually masking off bit 31. As usual, that results in much more readable code.
Quoted 6 lines
No functional changes.
Co-developed-by Alexandre Courbot <[email protected]>Signed-off-by Alexandre Courbot <[email protected]>Signed-off-by John Hubbard <[email protected]>---
Re: (subset) [PATCH 0/2] gpu: nova-core: add two prerequisites for the r000 GSP firmware
In reply to John HubbardQuoted 1 line · John Hubbard
[PATCH 0/2] gpu: nova-core: add two prerequisites for the r000 GSP firmware
Applied, thanks!
Branch: drm-rust-next
Tree: https://gitlab.freedesktop.org/drm/rust/kernel.git
[2/2] gpu: nova-core: factor out and expose Falcon::is_processor_suspended()
commit: 6e557330b874
The patch will appear in the next linux-next integration (typically within 24 hours on weekdays).
The patch is queued up for the upcoming merge window for the next major kernel release.