nova-core: GPU interrupt support and GSP event delivery
Subsystems (9)
v5 cover nova-core: GPU interrupt support and GSP event delivery
This series adds GPU interrupt support to nova-core, so that GSP events reach the driver as interrupts instead of only when the driver polls. v5 applies Alex's review of v4, and makes three of the design changes that he had set aside for after the merge. The series is 15 patches, down from 17. The two rust/ patches are unchanged.
GIN, the GPU Interrupt and Notification unit, is the GPU's interrupt controller, and it latches pending interrupts in a two-level register tree. TOP, the tree's top-level enable register, holds one enable bit per subtree.
The design uses a threaded IRQ handler. The top half reads and writes only GPU registers, while the threaded bottom half drains the message queue.
Still TODO after this: retrieve the dynamic interrupt routing tables from GSP, and use those, instead of the simple, hard-coded approach that was done here for these first two interrupts.
The series is based on drm-rust-next at commit 658c5f0042e6 ("gpu:
nova-core: use a single try_pin_init!() block in probe()"), and it carries no prerequisite commits.
The patches, as applied to drm-rust-next, are on a git branch:
https://github.com/johnhubbard/linux/tree/nova-core-gin-interrupt-tree-v5/
Changes in v5:
* Dropped the patch that poisoned the message queue after a framing or checksum failure. (Alex)
* The GSP interrupt handler borrows the GSP falcon and the command queue instead of holding a BAR and a chipset. (Alex)
* The per-chipset falcon interrupt operations are methods of the existing falcon HAL rather than of a separate falcon interrupt HAL. (Alex)
* The PCI vector allocation and the interrupt tree are one type, which the Gpu owns and the GSP handler borrows. (Alex)
* The tree reset is a method of the tree, and it disables the serviced subtrees at TOP last. (Alex)
* The Gpu holds the TOP enable of the GSP subtree, and the GSP handler's registration holds only the leaf enable of its own vector. (Alex)
* The interrupt tree's trigger and the message queue's drain are added by the patches that first call them, so the separate message queue drain patch is dropped. (Alex)
* The two KUnit suites that tested the HAL dispatchers are removed, and the tree suite's chipset check covers every chipset. (Alex)
* Smaller items from the same review (Alex):
* repr(u32) on the leaf count, and an if/else in the vector check.
* The interrupt HAL performs the rearm, and the tree holds the HAL instead of a leaf count and a rearm method.
* The GFW comment restored at the wait in probe, where GFW is the GPU's boot firmware, and one import group in probe.
* The self-test handler's clear and rearm grouped ahead of the count, and the superfluous dev local removed.
* A comment at the MSI fallback in the vector allocation.
* "chipset" for "part", and "OpenRM" for "Open RM".
* The falcon HAL patch's message no longer claims that a HAL call allocates, since there is no separate HAL left to justify.
TESTING: Tested on Turing (TU117), Ampere (GA104), and Blackwell (GB202), covering probe and chipset identification, the interrupt self-test, a GSP name query over the command queue, driver unload and reload, and the KUnit suites. No test produced a GSP fault on any of the three chips, so the fault path is untested.
Joel Fernandes (2):
rust: sync: completion: add wait_for_completion_timeout()
gpu: nova-core: add the GIN interrupt tree and allocate its vectors
John Hubbard (13):
rust: pci: declare IrqType and IrqTypes with impl_flags
gpu: nova-core: add the GIN vector, leaf and subtree types
gpu: nova-core: add the GIN CPU interrupt tree and MSI EOI registers
gpu: nova-core: add the per-architecture GIN CPU interrupt HAL
gpu: nova-core: wait for GFW boot in probe, not in the Gpu constructor
gpu: nova-core: add an interrupt delivery self-test
gpu: nova-core: log GSP events instead of discarding them
gpu: nova-core: return ENOMSG for an unmatched GSP message
gpu: nova-core: bound a GSP wait by a single deadline
gpu: nova-core: add the falcon interrupt registers and HAL methods
gpu: nova-core: service GSP events from the SWGEN0 interrupt
gpu: nova-core: add KUnit tests for the interrupt tree
gpu: nova-core: document the GIN interrupt controller and GSP events
Documentation/gpu/nova/core/interrupts.rst | 679 +++++++++++++++++++ Documentation/gpu/nova/index.rst | 1 + drivers/gpu/nova-core/Kconfig | 5 + drivers/gpu/nova-core/driver.rs | 19 +- drivers/gpu/nova-core/falcon/gsp.rs | 47 +- drivers/gpu/nova-core/falcon/hal.rs | 19 +- drivers/gpu/nova-core/falcon/hal/ga102.rs | 16 + drivers/gpu/nova-core/falcon/hal/tu102.rs | 49 +- drivers/gpu/nova-core/gpu.rs | 122 +++- drivers/gpu/nova-core/gsp/cmdq.rs | 166 ++++- drivers/gpu/nova-core/gsp/commands.rs | 8 +- drivers/gpu/nova-core/gsp/sequencer.rs | 8 +- drivers/gpu/nova-core/irq.rs | 29 + drivers/gpu/nova-core/irq/doorbell_test.rs | 257 ++++++++ drivers/gpu/nova-core/irq/gsp.rs | 196 ++++++ drivers/gpu/nova-core/irq/hal.rs | 67 ++ drivers/gpu/nova-core/irq/hal/gh100.rs | 40 ++ drivers/gpu/nova-core/irq/hal/tu102.rs | 43 ++ drivers/gpu/nova-core/irq/interrupt_tree.rs | 682 ++++++++++++++++++++ drivers/gpu/nova-core/irq/regs.rs | 88 +++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 69 ++ rust/kernel/pci/irq.rs | 68 +- rust/kernel/sync/completion.rs | 23 +- 24 files changed, 2594 insertions(+), 108 deletions(-) create mode 100644 Documentation/gpu/nova/core/interrupts.rst create mode 100644 drivers/gpu/nova-core/irq.rs create mode 100644 drivers/gpu/nova-core/irq/doorbell_test.rs create mode 100644 drivers/gpu/nova-core/irq/gsp.rs create mode 100644 drivers/gpu/nova-core/irq/hal.rs create mode 100644 drivers/gpu/nova-core/irq/hal/gh100.rs create mode 100644 drivers/gpu/nova-core/irq/hal/tu102.rs create mode 100644 drivers/gpu/nova-core/irq/interrupt_tree.rs create mode 100644 drivers/gpu/nova-core/irq/regs.rs
base-commit: 658c5f0042e6e897338dd04330a363aa7022c85d
Signature · 1 line
2.55.0
PATCH v5 1/15 rust: pci: declare IrqType and IrqTypes with impl_flags
The kernel provides impl_flags! for declaring a bitmask type alongside the enum of its individual flags, generating the bit operators and the containment queries.
IrqTypes open-coded that pattern with a with() builder, so a caller naming two interrupt types chained two calls onto IrqTypes::default().
Declare both types through impl_flags!, so the same set reads as IrqType::Msi | IrqType::MsiX.
--- rust/kernel/pci/irq.rs | 68 +++++++++++++----------------------------- 1 file changed, 21 insertions(+), 47 deletions(-)
rust/kernel/pci/irq.rs +21 -47
@@ -13,27 +13,26 @@ };use core::num::NonZero ; -/// IRQ type flags for PCI interrupt allocation. -#[derive( Debug , Clone , Copy )] -pub enum IrqType { -/// INTx interrupts. -Intx , -/// Message Signaled Interrupts (MSI). -Msi , -/// Extended Message Signaled Interrupts (MSI-X). -MsiX , -} - -impl IrqType { -/// Convert to the corresponding kernel flags. -const fn as_raw (self) ->u32 { -match self { -IrqType ::Intx => bindings::PCI_IRQ_INTX , -IrqType ::Msi => bindings::PCI_IRQ_MSI , -IrqType ::MsiX => bindings::PCI_IRQ_MSIX , - } +crate ::impl_flags!( +/// Set of IRQ types that can be used for PCI interrupt allocation. +#[derive( Debug , Clone , Copy , Default )] +pub struct IrqTypes (u32 ); + +/// IRQ type flags for PCI interrupt allocation. +#[derive( Debug , Clone , Copy )] +pub enum IrqType { +/// INTx interrupts. +Intx = bindings::PCI_IRQ_INTX , + +/// Message Signaled Interrupts (MSI). +Msi = bindings::PCI_IRQ_MSI , + +/// Extended Message Signaled Interrupts (MSI-X). +MsiX = bindings::PCI_IRQ_MSIX , } +); +impl IrqType {/// Construct from raw value. #[inline] const fn from_raw (raw:u32 ) ->Self {
@@ -45,33 +44,10 @@ const fn from_raw(raw: u32) -> Self { } } -/// Set of IRQ types that can be used for PCI interrupt allocation. -#[derive( Debug , Clone , Copy , Default )] -pub struct IrqTypes (u32 ); -impl IrqTypes {/// Create a set containing all IRQ types (MSI-X, MSI, and INTx). pub const fn all () ->Self { -Self (bindings::PCI_IRQ_ALL_TYPES ) - } - -/// Build a set of IRQ types. -/// -/// # Examples -/// -/// ```ignore -/// // Create a set with only MSI and MSI-X (no INTx interrupts). -/// let msi_only = IrqTypes::default() -/// .with(IrqType::Msi) -/// .with(IrqType::MsiX); -/// ``` -pub const fn with (self, irq_type:IrqType ) ->Self { -Self (self.0 | irq_type.as_raw ()) - } - -/// Get the raw flags value. -const fn as_raw (self) ->u32 { - self.0 +Self (Self ::all_bits ()) } }
@@ -203,9 +179,7 @@ impl Device<device::Bound> {/// let vectors = dev.alloc_irq_vectors(1, 32, pci::IrqTypes::all())?; /// /// // Allocate MSI or MSI-X only (no INTx interrupts). -/// let msi_only = pci::IrqTypes::default() -/// .with(pci::IrqType::Msi) -/// .with(pci::IrqType::MsiX); +/// let msi_only = pci::IrqType::Msi | pci::IrqType::MsiX; /// let vectors = dev.alloc_irq_vectors(4, 16, msi_only)?; /// # Ok(()) /// # }
@@ -222,7 +196,7 @@ pub fn alloc_irq_vectors(// - `pci_alloc_irq_vectors` internally validates all other parameters // and returns error codes. let ret =unsafe { - bindings::pci_alloc_irq_vectors (self.as_raw (), min_vecs, max_vecs, irq_types.as_raw ()) + bindings::pci_alloc_irq_vectors (self.as_raw (), min_vecs, max_vecs, u32::from (irq_types)) };to_result (ret)?;
Signature · 1 line
2.55.0
Alexandre Courbot
maintainerRe: [PATCH v5 01/15] rust: pci: declare IrqType and IrqTypes with impl_flags
Quoted 7 lines · John Hubbard
The kernel provides
impl_flags! for declaring a bitmask type alongside the enum of its individual flags, generating the bit operators and the containment queries.IrqTypes open-coded that pattern with a
with()builder, so a caller naming two interrupt types chained two calls ontoIrqTypes::default().
Quoted 6 lines
Declare both types through
impl_flags!, so the same set reads asIrqType::Msi|IrqType::MsiX.Suggested-by Gary Guo <[email protected]>Reviewed-by Alexandre Courbot <[email protected]>Signed-off-by John Hubbard <[email protected]>
Danilo, is this ok if we take this patch through drm-rust-next?
PATCH v5 2/15 rust: sync: completion: add wait_for_completion_timeout()
In reply to John HubbardFrom: Joel Fernandes <[email protected]>
A driver that runs an interrupt self-test during probe waits for the handler to fire. wait_for_completion() has no timeout, so a broken interrupt path stalls probe indefinitely. Add a timeout variant of wait_for_completion().
--- rust/kernel/sync/completion.rs | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
rust/kernel/sync/completion.rs +22 -1
@@ -6,7 +6,12 @@//! //! C header: [`include/linux/completion.h`](srctree/include/linux/completion.h) -use crate ::{bindings, prelude::*, types::Opaque }; +use crate ::{ + bindings, + prelude::*, + time::Jiffies , + types::Opaque ,// +};/// Synchronization primitive to signal when a certain task has been completed. ///
@@ -111,4 +116,20 @@ pub fn wait_for_completion(&self) {// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. unsafe { bindings::wait_for_completion (self.as_raw ()) }; } + +/// Wait for completion of a task, with a timeout. +/// +/// This method waits for the completion of a task, or until `timeout` elapses. It is not +/// interruptible. Returns the number of jiffies left when the task completed, or [`None`] if +/// `timeout` elapsed first. +/// +/// See also [`Completion::complete_all`]. +#[inline] +pub fn wait_for_completion_timeout (&self, timeout:Jiffies ) ->Option <Jiffies > { +// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. +match unsafe { bindings::wait_for_completion_timeout (self.as_raw (), timeout) } { +0 =>None , + remaining =>Some (remaining), + } + } }
Signature · 1 line
2.55.0
[email protected]
AI reviewRe: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] rust: sync: wait_for_completion_timeout stack dump on large timeouts --
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] rust: sync: wait_for_completion_timeout stack dump on large timeouts --
commit f89eb4d12071b7e3c61766cd1bbbd5c2cfee1ea5
Author: Joel Fernandes <[email protected]>
rust: sync: completion: add wait_for_completion_timeout()
This commit adds wait_for_completion_timeout() to the Rust sync completion primitives. It provides a way to wait for a completion with a specified timeout in jiffies, preventing indefinite stalls in drivers.
Quoted 3 lines
rust/kernel/sync/completion.rs +0
-0
[ ... ]
Quoted 6 lines
rust/kernel/sync/completion.rs +2
-0@@ -111,4 +116,20 @@ pub fn wait_for_completion(&self) {// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. unsafe { bindings::wait_for_completion (self.as_raw ()) }; } + +/// Wait for completion of a task, with a timeout.
[ ... ]
rust/kernel/sync/completion.rs pub fn wait_for_completion(&self) { · line 130
Quoted 4 lines
rust/kernel/sync/completion.rs +4
-0+#[inline] +pub fn wait_for_completion_timeout (&self, timeout:Jiffies ) ->Option <Jiffies > { +// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. +match unsafe { bindings::wait_for_completion_timeout (self.as_raw (), timeout) } {
[Severity: Medium]
Does this code overflow the timeout parameter if large values are provided?
The timeout parameter is passed directly to the C API without bounds checking. Because C implicitly casts it to a signed long downstream, values greater than LONG_MAX can lead to an immediate unhandled timeout and a kernel stack dump.
For example, passing Jiffies::MAX causes the unsigned timeout to overflow into a negative long inside the C wait_for_common() and schedule_timeout() functions. This triggers a kernel stack dump (printk(KERN_ERR ...); dump_stack();) and causes the function to return immediately without waiting, violating the safe function's contract.
Could this be clamped to MAX_SCHEDULE_TIMEOUT to prevent unexpected behavior from excessively large timeouts?
Quoted 5 lines
rust/kernel/sync/completion.rs +4
-0+ 0 => None, + remaining => Some(remaining), + } + } }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Alexandre Courbot
maintainerRe: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
In reply to John HubbardQuoted 11 lines · John Hubbard
From: Joel Fernandes <[email protected]>
A driver that runs an interrupt self-test during probe waits for the handler to fire.
wait_for_completion()has no timeout, so a broken interrupt path stalls probe indefinitely. Add a timeout variant ofwait_for_completion().Reviewed-by Alexandre Courbot <[email protected]>Signed-off-by Joel Fernandes <[email protected]>[jhubbard: return the remaining jiffies]Signed-off-by John Hubbard <[email protected]>
Sync maintainers, is this ok if we push this patch through drm-rust-next?
Re: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
In reply to John HubbardQuoted 8 lines · John Hubbard
From: Joel Fernandes <[email protected]>
A driver that runs an interrupt self-test during probe waits for the handler to fire.
wait_for_completion()has no timeout, so a broken interrupt path stalls probe indefinitely. Add a timeout variant ofwait_for_completion().
Quoted 4 lines
Reviewed-by Alexandre Courbot <[email protected]>Signed-off-by Joel Fernandes <[email protected]>[jhubbard: return the remaining jiffies]Signed-off-by John Hubbard <[email protected]>
With the timeout clamped to LONG_MAX as sashiko says:
Feel free to take through drm-rust-next.
Re: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
Quoted 9 lines · Alice Ryhl
Quoted 8 lines · John Hubbard
From: Joel Fernandes <[email protected]>
A driver that runs an interrupt self-test during probe waits for the handler to fire.
wait_for_completion()has no timeout, so a broken interrupt path stalls probe indefinitely. Add a timeout variant ofwait_for_completion().
Quoted 6 lines
Quoted 4 lines
Reviewed-by Alexandre Courbot <[email protected]>Signed-off-by Joel Fernandes <[email protected]>[jhubbard: return the remaining jiffies]Signed-off-by John Hubbard <[email protected]>With the timeout clamped to
LONG_MAXas sashiko says:
Clearly Sashiko is improving its Rust-for-Linux review capabilities. It missed this on the first four revisions of the series, but now it spotted it. Progress!>
Quoted 3 lines
Acked-by Alice Ryhl <[email protected]>Feel free to take through drm-rust-next.
Thanks for the review, Alice!
thanks,
Signature · 1 line
John Hubbard
Gary Guo
maintainerRe: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
In reply to John HubbardQuoted 11 lines · John Hubbard
From: Joel Fernandes <[email protected]>
A driver that runs an interrupt self-test during probe waits for the handler to fire.
wait_for_completion()has no timeout, so a broken interrupt path stalls probe indefinitely. Add a timeout variant ofwait_for_completion().Reviewed-by Alexandre Courbot <[email protected]>Signed-off-by Joel Fernandes <[email protected]>[jhubbard: return the remaining jiffies]Signed-off-by John Hubbard <[email protected]>
This email should have RfL list CC'ed.
Quoted 26 lines
--- rust/kernel/sync/completion.rs | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)rust/kernel/sync/completion.rs +6
-1@@ -6,7 +6,12 @@//! //! C header: [`include/linux/completion.h`](srctree/include/linux/completion.h) -use crate ::{bindings, prelude::*, types::Opaque }; +use crate ::{ + bindings, + prelude::*, + time::Jiffies , + types::Opaque ,// +};/// Synchronization primitive to signal when a certain task has been completed. /// @@ -111,4 +116,20 @@ pub fn wait_for_completion(&self) {// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. unsafe { bindings::wait_for_completion (self.as_raw ()) }; }
rust/kernel/sync/completion.rs pub fn wait_for_completion(&self) { · line 128
Quoted 10 lines
rust/kernel/sync/completion.rs +10
-0+ +/// Wait for completion of a task, with a timeout. +/// +/// This method waits for the completion of a task, or until `timeout` elapses. It is not +/// interruptible. Returns the number of jiffies left when the task completed, or [`None`] if +/// `timeout` elapsed first. +/// +/// See also [`Completion::complete_all`]. +#[inline] +pub fn wait_for_completion_timeout (&self, timeout:Jiffies ) ->Option <Jiffies > {
NACK. Please use the typed Delta<Jiffy>, not this old type alias for c_ulong. I asked Miguel to take that late last cycle because I know there'll be new users that want to use Jiffies this cycle!
This would avoid you having to do clamping on the max side (but you still need to clamp the value to be at least 0). There'll be a helper method for this, but it isn't available yet.
https://lore.kernel.org/rust-for-linux/[email protected]/
Best,
Gary
Quoted 7 lines
rust/kernel/sync/completion.rs +6
-0+// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. +match unsafe { bindings::wait_for_completion_timeout (self.as_raw (), timeout) } { +0 =>None , + remaining =>Some (remaining), + } + } }
Alexandre Courbot
maintainerRe: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
Quoted 56 lines · Gary Guo
Quoted 11 lines · John Hubbard
From: Joel Fernandes <[email protected]>
A driver that runs an interrupt self-test during probe waits for the handler to fire.
wait_for_completion()has no timeout, so a broken interrupt path stalls probe indefinitely. Add a timeout variant ofwait_for_completion().Reviewed-by Alexandre Courbot <[email protected]>Signed-off-by Joel Fernandes <[email protected]>[jhubbard: return the remaining jiffies]Signed-off-by John Hubbard <[email protected]>This email should have RfL list CC'ed.
Quoted 26 lines
--- rust/kernel/sync/completion.rs | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)rust/kernel/sync/completion.rs +6
-1@@ -6,7 +6,12 @@//! //! C header: [`include/linux/completion.h`](srctree/include/linux/completion.h) -use crate ::{bindings, prelude::*, types::Opaque }; +use crate ::{ + bindings, + prelude::*, + time::Jiffies , + types::Opaque ,// +};/// Synchronization primitive to signal when a certain task has been completed. /// @@ -111,4 +116,20 @@ pub fn wait_for_completion(&self) {// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. unsafe { bindings::wait_for_completion (self.as_raw ()) }; }rust/kernel/sync/completion.rs pub fn wait_for_completion(&self) { · line 128
Quoted 10 lines
rust/kernel/sync/completion.rs +10
-0+ +/// Wait for completion of a task, with a timeout. +/// +/// This method waits for the completion of a task, or until `timeout` elapses. It is not +/// interruptible. Returns the number of jiffies left when the task completed, or [`None`] if +/// `timeout` elapsed first. +/// +/// See also [`Completion::complete_all`]. +#[inline] +pub fn wait_for_completion_timeout (&self, timeout:Jiffies ) ->Option <Jiffies > {NACK. Please use the typed
Delta<Jiffy>, not this old type alias forc_ulong. I asked Miguel to take that late last cycle because I know there'll be new users that want to use Jiffies this cycle!
Quoted 4 lines
This would avoid you having to do clamping on the max side (but you still need to clamp the value to be at least 0). There'll be a helper method for this, but it isn't available yet.
https://lore.kernel.org/rust-for-linux/[email protected]/
I will drop this patch and replace the code using this new method (which is just an optional probe-time self-test) by a polling loop on an atomic. That way we can wait for the helper method to be available and revisit this with the proper type.
Re: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
Quoted 9 lines · Alexandre Courbot
Quoted 8 lines · Gary Guo
Quoted 7 lines · John Hubbard
From: Joel Fernandes <[email protected]>
A driver that runs an interrupt self-test during probe waits for the handler to fire.
wait_for_completion()has no timeout, so a broken interrupt path stalls probe indefinitely. Add a timeout variant ofwait_for_completion().
Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Reviewed-by Alexandre Courbot <[email protected]>Signed-off-by Joel Fernandes <[email protected]>[jhubbard: return the remaining jiffies]Signed-off-by John Hubbard <[email protected]>This email should have RfL list CC'ed.
Hi Gary! Apologies, my send-email scripts had drifted into a nova-centric direction. Will fix.
Quoted 47 lines
Quoted 42 lines
Quoted 27 lines
--- rust/kernel/sync/completion.rs | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)rust/kernel/sync/completion.rs +6
-1@@ -6,7 +6,12 @@//! //! C header: [`include/linux/completion.h`](srctree/include/linux/completion.h) -use crate ::{bindings, prelude::*, types::Opaque }; +use crate ::{ + bindings, + prelude::*, + time::Jiffies , + types::Opaque ,// +};/// Synchronization primitive to signal when a certain task has been completed. /// @@ -111,4 +116,20 @@ pub fn wait_for_completion(&self) {// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. unsafe { bindings::wait_for_completion (self.as_raw ()) }; }rust/kernel/sync/completion.rs pub fn wait_for_completion(&self) { · line 128
Quoted 10 lines
rust/kernel/sync/completion.rs +10
-0+ +/// Wait for completion of a task, with a timeout. +/// +/// This method waits for the completion of a task, or until `timeout` elapses. It is not +/// interruptible. Returns the number of jiffies left when the task completed, or [`None`] if +/// `timeout` elapsed first. +/// +/// See also [`Completion::complete_all`]. +#[inline] +pub fn wait_for_completion_timeout (&self, timeout:Jiffies ) ->Option <Jiffies > {NACK. Please use the typed
Delta<Jiffy>, not this old type alias forc_ulong. I asked Miguel to take that late last cycle because I know there'll be new users that want to use Jiffies this cycle!Quoted 4 lines
This would avoid you having to do clamping on the max side (but you still need to clamp the value to be at least 0). There'll be a helper method for this, but it isn't available yet.
https://lore.kernel.org/rust-for-linux/[email protected]/
Quoted 4 lines
I will drop this patch and replace the code using this new method (which is just an optional probe-time self-test) by a polling loop on an atomic. That way we can wait for the helper method to be available and revisit this with the proper type.
OK, whatever works best. Just in case, I did stage a fixed-up version of the patch locally (in case I need to send a v6), as per Gary's recommended Delta approach. It looks like this:
rust/kernel/sync/completion.rs +31 -1
@@ -6,7 +6,15 @@//! //! C header: [`include/linux/completion.h`](srctree/include/linux/completion.h) -use crate ::{bindings, prelude::*, types::Opaque }; +use crate ::{ + bindings, + prelude::*, + time::{ +Delta , +Jiffy ,// + }, + types::Opaque ,// +};/// Synchronization primitive to signal when a certain task has been completed. ///
@@ -111,4 +119,26 @@ pub fn wait_for_completion(&self) {// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. unsafe { bindings::wait_for_completion (self.as_raw ()) }; } + +/// Waits for the completion of a task, or until `timeout` elapses. +/// +/// This method is not interruptible. It clamps a negative `timeout` to zero. +/// +/// Returns the time remaining, at least one jiffy, if the task completes before `timeout` +/// elapses, and [`None`] otherwise. +/// +/// See also [`Completion::complete_all`]. +#[inline] +pub fn wait_for_completion_timeout (&self, timeout:Delta <Jiffy >) ->Option <Delta <Jiffy >> { +// CAST: the value is non-negative after `max`, so the cast to unsigned is lossless. +let timeout = timeout.as_jiffies ().max (0 )as c_ulong ; + +// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. +match unsafe { bindings::wait_for_completion_timeout (self.as_raw (), timeout) } { +0 =>None , +// CAST: `remaining` is at most `timeout`, or `1` when `timeout` is zero, so it fits +// an `isize`. + remaining =>Some (Delta ::from_jiffies (remainingas isize )), + } + } }
thanks,
Signature · 1 line
John Hubbard
Alexandre Courbot
maintainerRe: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
Quoted 68 lines · John Hubbard
Quoted 9 lines · Alexandre Courbot
Quoted 8 lines · Gary Guo
Quoted 7 lines · John Hubbard
From: Joel Fernandes <[email protected]>
A driver that runs an interrupt self-test during probe waits for the handler to fire.
wait_for_completion()has no timeout, so a broken interrupt path stalls probe indefinitely. Add a timeout variant ofwait_for_completion().Quoted 6 lines
Quoted 6 lines
Quoted 4 lines
Reviewed-by Alexandre Courbot <[email protected]>Signed-off-by Joel Fernandes <[email protected]>[jhubbard: return the remaining jiffies]Signed-off-by John Hubbard <[email protected]>This email should have RfL list CC'ed.
Hi Gary! Apologies, my send-email scripts had drifted into a nova-centric direction. Will fix.
Quoted 47 lines
Quoted 46 lines
Quoted 27 lines
--- rust/kernel/sync/completion.rs | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)rust/kernel/sync/completion.rs +6
-1@@ -6,7 +6,12 @@//! //! C header: [`include/linux/completion.h`](srctree/include/linux/completion.h) -use crate ::{bindings, prelude::*, types::Opaque }; +use crate ::{ + bindings, + prelude::*, + time::Jiffies , + types::Opaque ,// +};/// Synchronization primitive to signal when a certain task has been completed. /// @@ -111,4 +116,20 @@ pub fn wait_for_completion(&self) {// SAFETY: `self.as_raw()` is a pointer to a valid `struct completion`. unsafe { bindings::wait_for_completion (self.as_raw ()) }; }rust/kernel/sync/completion.rs pub fn wait_for_completion(&self) { · line 128
Quoted 10 lines
rust/kernel/sync/completion.rs +10
-0+ +/// Wait for completion of a task, with a timeout. +/// +/// This method waits for the completion of a task, or until `timeout` elapses. It is not +/// interruptible. Returns the number of jiffies left when the task completed, or [`None`] if +/// `timeout` elapsed first. +/// +/// See also [`Completion::complete_all`]. +#[inline] +pub fn wait_for_completion_timeout (&self, timeout:Jiffies ) ->Option <Jiffies > {NACK. Please use the typed
Delta<Jiffy>, not this old type alias forc_ulong. I asked Miguel to take that late last cycle because I know there'll be new users that want to use Jiffies this cycle!This would avoid you having to do clamping on the max side (but you still need to clamp the value to be at least 0). There'll be a helper method for this, but it isn't available yet.
https://lore.kernel.org/rust-for-linux/[email protected]/
Quoted 8 lines
Quoted 4 lines
I will drop this patch and replace the code using this new method (which is just an optional probe-time self-test) by a polling loop on an atomic. That way we can wait for the helper method to be available and revisit this with the proper type.
OK, whatever works best. Just in case, I did stage a fixed-up version of the patch locally (in case I need to send a v6), as per Gary's recommended Delta approach. It looks like this:
It's your series so your call, but since the elegant solution will involve using [1] which is still under review, I would suggest waiting for it to introduce wait_for_completion_timeout, moving forward with the rest of the series, and revisit using completions once the base code is ready to limit churn.
Otherwise we are at risk of missing the -rc6 cutoff for this series and that means an even busier next cycle. The polling code is also simpler overall than using completions, and for a probe-time selftest polling is really not a problem. Nova is not losing anything by not using completions for this.
[1] https://lore.kernel.org/rust-for-linux/[email protected]/
Gary Guo
maintainerRe: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
Quoted 18 lines · Alexandre Courbot
Quoted 10 lines · John Hubbard
Quoted 4 lines · Alexandre Courbot
I will drop this patch and replace the code using this new method (which is just an optional probe-time self-test) by a polling loop on an atomic. That way we can wait for the helper method to be available and revisit this with the proper type.
OK, whatever works best. Just in case, I did stage a fixed-up version of the patch locally (in case I need to send a v6), as per Gary's recommended Delta approach. It looks like this:
It's your series so your call, but since the elegant solution will involve using [1] which is still under review, I would suggest waiting for it to introduce
wait_for_completion_timeout, moving forward with the rest of the series, and revisit using completions once the base code is ready to limit churn.
Quoted 5 lines
Otherwise we are at risk of missing the -rc6 cutoff for this series and that means an even busier next cycle. The polling code is also simpler overall than using completions, and for a probe-time selftest polling is really not a problem. Nova is not losing anything by not using completions for this.
I think putting it in without .as_jiffies_unsigned is fine. We can cleanup later.
But the code shouldn't be using the postfix .max() for clamping and should be using isize::max or core::cmp::max instead to avoid confusion, see
https://lore.kernel.org/rust-for-linux/[email protected]/.
Best,
Gary
Danilo Krummrich
maintainerRe: [PATCH v5 02/15] rust: sync: completion: add wait_for_completion_timeout()
In reply to Alexandre CourbotQuoted 4 lines · Alexandre Courbot
I will drop this patch and replace the code using this new method (which is just an optional probe-time self-test) by a polling loop on an atomic. That way we can wait for the helper method to be available and revisit this with the proper type.
Sounds good, let's do this.
PATCH v5 3/15 gpu: nova-core: add the GIN vector, leaf and subtree types
In reply to John HubbardGIN, the GPU Interrupt and Notification unit, is the GPU's interrupt controller. Each interrupt source has a GIN vector number, and the controller latches a pending vector in a two-level tree: one bit of a LEAF register, summarized two leaves at a time by one bit of the TOP register. A vector's number fixes its position in that tree:
leaf = vector / 32
bit = vector % 32
subtree = leaf / 2
A tree implements either 8 or 16 leaves, depending on the GPU family. The leaf count sets both the number of subtrees and the highest vector that the tree carries.
Without distinct types, a vector, a leaf index, a set of vectors within one leaf, a subtree and a set of subtrees are all plain integers. A caller can pass one where another belongs, and a register field can accept the wrong one.
Add a type for each of those, and for the leaf count. A vector converts to its own leaf, bit and subtree. Its constructor rejects, at build time, a number beyond the widest supported tree, and a validation method rejects, at run time, a number beyond the leaves that the current tree implements. A leaf count yields the set of subtrees that it implements.
The module has no user yet.
--- drivers/gpu/nova-core/irq.rs | 12 + drivers/gpu/nova-core/irq/interrupt_tree.rs | 238 ++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 2 + 3 files changed, 252 insertions(+) create mode 100644 drivers/gpu/nova-core/irq.rs create mode 100644 drivers/gpu/nova-core/irq/interrupt_tree.rs
drivers/gpu/nova-core/irq.rs new file +12 -0
@@ -0,0 +1,12 @@ +// SPDX-License-Identifier: GPL-2.0 +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + +//! GPU interrupt support. +//! +//! GIN, the GPU Interrupt and Notification unit, is the GPU's interrupt controller. It latches +//! every interrupt source in a two-level register tree and delivers the tree to the CPU as a +//! message-signaled PCI interrupt. +//! +//! See `Documentation/gpu/nova/core/interrupts.rst`. + +mod interrupt_tree;
drivers/gpu/nova-core/irq/interrupt_tree.rs new file +238 -0
238 lines not on this page · raw message
drivers/gpu/nova-core/nova_core.rs +2 -0
@@ -18,6 +18,8 @@mod fsp;mod gpu;mod gsp; +#[expect(dead_code)] +mod irq;mod mctp;mod mm;#[macro_use]
Signature · 1 line
2.55.0
PATCH v5 4/15 gpu: nova-core: add the GIN CPU interrupt tree and MSI EOI registers
In reply to John HubbardGIN is the GPU's interrupt controller. It latches each interrupt source in a two-level tree of LEAF registers summarized by TOP, and raises the PCI interrupt when an enabled vector in an enabled subtree becomes pending. A message-signaled interrupt is delivered once per edge, and pre-Hopper MSI rearms delivery by writing the end-of-interrupt register in the BAR0 mirror of PCI configuration space.
Add the CPU tree registers that receiving GSP interrupts and running the software-triggered self-test need: the leaf pending and enable arrays, the TOP enables, and the leaf trigger. Add the end-of-interrupt register, NV_XVE_CYA_2, alongside them.
Use the NV_VIRTUAL_FUNCTION_PRIV_CPU_INTR_* names on every chipset. The pre-Hopper headers call the same tree NV_CTRL, but each function reaches its own tree through this aperture on both families. Declare the leaf arrays at 16 entries, the widest tree that any supported chipset implements.
Leave the read-only TOP summary undeclared. A vector that latched while disabled does not appear in TOP, so nova-core never descends from it and reads every implemented leaf instead.
Declare each leaf field as a set of vectors within one leaf and each TOP field as a set of subtrees, so that a set of subtrees cannot be written to a leaf register, nor the reverse. The trigger register's vector field is 12 bits wide, wider than any GIN vector, so a vector converts into it infallibly.
--- drivers/gpu/nova-core/irq.rs | 1 + drivers/gpu/nova-core/irq/interrupt_tree.rs | 14 ++++ drivers/gpu/nova-core/irq/regs.rs | 88 +++++++++++++++++++++ 3 files changed, 103 insertions(+) create mode 100644 drivers/gpu/nova-core/irq/regs.rs
drivers/gpu/nova-core/irq.rs +1 -0
@@ -10,3 +10,4 @@//! See `Documentation/gpu/nova/core/interrupts.rst`. mod interrupt_tree; +mod regs;
drivers/gpu/nova-core/irq/interrupt_tree.rs +14 -0
@@ -16,6 +16,8 @@use crate ::num; +use super ::regs::*; +/// Number of vectors one leaf register carries, one per bit. const VECTORS_PER_LEAF :u32 = u32::BITS ;
@@ -31,6 +33,12 @@/// Number of bits needed to address every vector in the widest supported tree. const VECTOR_BITS :u32 = (MAX_NUM_LEAVES *VECTORS_PER_LEAF ).ilog2 (); +/// Width of the vector field in the leaf trigger register. +const TRIGGER_VECTOR_BITS :u32 = { +let range =NV_VIRTUAL_FUNCTION_PRIV_CPU_INTR_LEAF_TRIGGER ::VECTOR_RANGE ; + num::u8_as_u32 (*range.end () - *range.start () +1 ) +}; +/// Index of a leaf register within the widest supported tree. An 8-leaf tree implements only the /// lower half of the range. pub (super )type LeafIndex =Bounded <usize , {MAX_NUM_LEAVES .ilog2 () }>;
@@ -236,3 +244,9 @@ pub(super) const fn validate(self, leaves: LeafCount) -> Result { } } } + +impl From <GinVector >for Bounded <u32 ,TRIGGER_VECTOR_BITS > { +fn from (vector:GinVector ) ->Self { + vector.0 .extend () + } +}
drivers/gpu/nova-core/irq/regs.rs new file +88 -0
88 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH v5 5/15 gpu: nova-core: add the per-architecture GIN CPU interrupt HAL
In reply to John HubbardThe GIN CPU interrupt tree differs by GPU family in two ways:
* The size of the tree. Turing, Ampere and Ada implement 8 leaves. Hopper and later families implement 16.
* The write that rearms delivery. A message-signaled interrupt is delivered once per edge, and the PCI side delivers nothing more until the CPU rearms it. Before Hopper, MSI rearms by writing the end-of-interrupt register in the BAR0 mirror of PCI configuration space. On Hopper and later, MSI rearms by clearing and then setting the TOP enables of every serviced subtree, which produces a new edge. MSI-X rearms the same way on every family, but for the handler's own subtree only, since each subtree has its own table entry.
Add an interrupt HAL that provides the leaf count and performs the rearm for each family. Name the interrupt type with two variants, MSI and MSI-X, since nova-core never allocates the level-triggered INTx that the kernel's PCI interrupt type also names.
The HAL has no user yet.
--- drivers/gpu/nova-core/irq.rs | 13 +++++ drivers/gpu/nova-core/irq/hal.rs | 67 ++++++++++++++++++++++++++ drivers/gpu/nova-core/irq/hal/gh100.rs | 40 +++++++++++++++ drivers/gpu/nova-core/irq/hal/tu102.rs | 43 +++++++++++++++++ 4 files changed, 163 insertions(+) create mode 100644 drivers/gpu/nova-core/irq/hal.rs create mode 100644 drivers/gpu/nova-core/irq/hal/gh100.rs create mode 100644 drivers/gpu/nova-core/irq/hal/tu102.rs
drivers/gpu/nova-core/irq.rs +13 -0
@@ -9,5 +9,18 @@//! //! See `Documentation/gpu/nova/core/interrupts.rst`. +mod hal;mod interrupt_tree;mod regs; + +/// The message-signaled interrupt type that Linux granted. +/// +/// nova-core never requests INTx, so this has no variant for it, unlike [`kernel::pci::IrqType`]. +#[derive( Clone , Copy , Debug , Eq , PartialEq )] +enum MsiType { +/// A single message, which every subtree raises. +Msi , + +/// One table entry per subtree. +MsiX , +}
drivers/gpu/nova-core/irq/hal.rs new file +67 -0
67 lines not on this page · raw message
drivers/gpu/nova-core/irq/hal/gh100.rs new file +40 -0
@@ -0,0 +1,40 @@ +// SPDX-License-Identifier: GPL-2.0 +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + +use crate ::driver::Bar0 ; + +use super ::{ + cycle_top_enables, +CpuInterruptHal , +LeafCount , +MsiType , +Subtree , +SubtreeSet ,// +}; + +/// The CPU interrupt tree properties of Hopper and Blackwell. +struct Gh100 ; + +impl CpuInterruptHal for Gh100 { +fn leaf_count (&self) ->LeafCount { +LeafCount ::Sixteen + } + +fn rearm_pci_irq ( + &self, + bar:Bar0 <'_>, + msi_type:MsiType , + serviced:SubtreeSet , + subtree:Subtree , + ) { +let subtrees =match msi_type { +MsiType ::Msi => serviced, +MsiType ::MsiX => subtree.into (), + }; + +cycle_top_enables (bar, subtrees); + } +} + +const GH100 :Gh100 =Gh100 ; +pub (super )const GH100_HAL : &dyn CpuInterruptHal = &GH100 ;
drivers/gpu/nova-core/irq/hal/tu102.rs new file +43 -0
43 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH v5 6/15 gpu: nova-core: add the GIN interrupt tree and allocate its vectors
In reply to John HubbardFrom: Joel Fernandes <[email protected]>
Servicing a GIN leaf has a required order: read its pending bits, then clear them. Clearing a leaf first discards every vector latched in it, and the hardware keeps no record of what was discarded. The interrupt never arrives, and no register shows that it was ever pending.
Every subtree enabled at TOP also needs an allocated PCI vector with a handler registered on it. Under MSI-X each subtree has its own table entry. Linux masks every entry until a driver requests its IRQ, and a masked entry sends no message. An enabled subtree whose entry was never requested raises interrupts that never reach a handler, while the leaf and TOP registers show them pending and enabled. Under MSI the whole tree raises a single message, so one entry serves every subtree.
Add the CPU interrupt tree of one PCIe function. The tree allocates the PCI vectors for the subtrees that it services and hands out the request for each, so that the chipset is given once and the vectors live as long as the tree. Reading a leaf yields the handle that clears it, so that the wrong order does not compile. Building a tree fails if it names a subtree that the GPU does not implement. A reset disables every vector, clears every pending bit, rearms delivery, and leaves the serviced subtrees disabled at TOP, so that a handler registered afterwards receives no interrupt left over from boot.
Request MSI-X entries 0 through the highest serviced subtree, since an MSI-X allocation cannot be sparse, and fall back to a single MSI message, never to INTx.
[jhubbard: reworked on top of the GIN vector types: a leaf read yields
the handle that clears it, enables are guarded, the leaf count and the rearm come from the interrupt HAL, the drain reads every implemented leaf rather than descending from TOP, the tree owns its PCI vectors, and a reset returns the tree to a known state before a handler is registered]
--- drivers/gpu/nova-core/irq.rs | 2 +- drivers/gpu/nova-core/irq/interrupt_tree.rs | 298 +++++++++++++++++++- 2 files changed, 293 insertions(+), 7 deletions(-)
drivers/gpu/nova-core/irq.rs +1 -1
@@ -10,7 +10,7 @@//! See `Documentation/gpu/nova/core/interrupts.rst`. mod hal; -mod interrupt_tree; +pub (crate )mod interrupt_tree;mod regs;/// The message-signaled interrupt type that Linux granted.
drivers/gpu/nova-core/irq/interrupt_tree.rs +292 -6
344 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH v5 7/15 gpu: nova-core: wait for GFW boot in probe, not in the Gpu constructor
In reply to John HubbardThe GPU boots its own firmware, GFW, out of reset, and the driver must not program the GPU until GFW reports completion.
nova-core waited for GFW inside the Gpu constructor, which also boots the GSP. Code that has to run after GFW and before GSP boot, such as a probe-time hardware self-test, had nowhere to go.
Move the wait into probe, ahead of the Gpu constructor, and read the chipset there from a Spec that probe builds itself. Leave the DMA mask in the constructor, since it programs the host rather than the GPU.
--- drivers/gpu/nova-core/driver.rs | 14 +++++++++++++- drivers/gpu/nova-core/gpu.rs | 26 +++++++++++++++++++------- 2 files changed, 32 insertions(+), 8 deletions(-)
drivers/gpu/nova-core/driver.rs +13 -1
@@ -24,7 +24,11 @@use crate ::{ api::NovaCoreApi , - gpu::Gpu ,// + gpu::{ + self, +Gpu , +Spec ,// + },// };/// Counter for generating unique auxiliary device IDs.
@@ -113,6 +117,14 @@ fn probe<'bound>( pdev.iomap_region (bar1_idx,c"nova-core/bar1" )? }, + _: { +let spec =Spec ::new (pdev.as_ref (), bar)?; + +// We must wait for GFW_BOOT completion before doing any significant setup on +// the GPU. + gpu::wait_gfw_boot_completion (pdev.as_ref (), bar, spec.chipset)?; + }, +// TODO: Use self-referential pin-init syntax once available. gpu <-Gpu ::new ( pdev,
drivers/gpu/nova-core/gpu.rs +19 -7
@@ -251,7 +251,7 @@ pub struct Spec { }impl Spec { -fn new (dev: &device::Device , bar:Bar0 <'_>) ->Result <Spec > { +pub (crate )fn new (dev: &device::Device , bar:Bar0 <'_>) ->Result <Spec > {// Some brief notes about boot0 and boot42, in chronological order: // // NV04 through NV50:
@@ -397,10 +397,8 @@ pub(crate) fn new<'a>(dev_info! (dev,"NVIDIA ({})\n" , spec); })?, - // We must wait for GFW_BOOT completion before doing any significant setup on the GPU. _: { -let hal = hal::gpu_hal (spec.chipset); -let dma_mask = hal.dma_mask (); +let dma_mask = hal::gpu_hal (spec.chipset).dma_mask ();// SAFETY: `Gpu` owns all DMA allocations for this device, and we are // still constructing it, so no concurrent DMA allocations can exist.
@@ -412,9 +410,6 @@ pub(crate) fn new<'a>(// SAFETY: `Gpu` owns all DMA allocations for this device, and we are // still constructing it, so no concurrent DMA allocations can exist. unsafe { pdev.dma_set_max_seg_size (u32::MAX ) }; - - hal.wait_gfw_boot_completion (bar) - .inspect_err (|_|dev_err! (dev,"GFW boot did not complete\n" ))?; },// Initialize this early because `gsp_resources` depends on it.
@@ -533,6 +528,23 @@ pub(crate) fn run_selftests(self: Pin<&mut Self>, pdev: &pci::Device<device::Bou } } +/// Waits for GFW, the GPU's boot firmware, to report completion. +/// +/// The driver must not program the GPU before then. +/// +/// # Errors +/// +/// `ETIMEDOUT` if GFW does not report completion in time. +pub (crate )fn wait_gfw_boot_completion ( + dev: &device::Device <device::Bound >, + bar:Bar0 <'_>, + chipset:Chipset , +) ->Result { + hal::gpu_hal (chipset) + .wait_gfw_boot_completion (bar) + .inspect_err (|_|dev_err! (dev,"GFW boot did not complete\n" )) +} +/// Reads the boot0 register and returns its raw value. pub (crate )fn boot_0_raw (bar:Bar0 <'_>) ->u32 { bar.read (regs::NV_PMC_BOOT_0 ).into_raw ()
Signature · 1 line
2.55.0
Danilo Krummrich
maintainerRe: [PATCH v5 07/15] gpu: nova-core: wait for GFW boot in probe, not in the Gpu constructor
drivers/gpu/nova-core/driver.rs fn probe<'bound>( · line 126
Quoted 11 lines · John Hubbard
drivers/gpu/nova-core/driver.rs +7
-0@@ -113,6 +117,14 @@ fn probe<'bound>( pdev.iomap_region (bar1_idx,c"nova-core/bar1" )? }, + _: { +let spec =Spec ::new (pdev.as_ref (), bar)?; + +// We must wait for GFW_BOOT completion before doing any significant setup on +// the GPU. + gpu::wait_gfw_boot_completion (pdev.as_ref (), bar, spec.chipset)?; + },
So, apparently this is done in order to be able to add this
#[cfg(CONFIG_NOVA_CORE_SELFTESTS)] crate::irq::doorbell_test::run_selftest(pdev, bar, spec.chipset)?;
in a subsequent patch.
I prefer to make wait_gfw_boot_completion() an associated function of Gpu, keep it called from Gpu::new() and guard the whole block with #[cfg(CONFIG_NOVA_CORE_SELFTESTS)], such that eventually it becomes:
#[cfg(CONFIG_NOVA_CORE_SELFTESTS)]
_: {
let spec = Spec::new(pdev.as_ref(), bar)?;
Gpu::wait_gfw_boot_completion(pdev.as_ref(), bar, spec.chipset)?;
irq::doorbell_test::run_selftest(pdev, bar, spec.chipset)?;
},With this Gpu::new() remains self-contained and we don't build up corss initialization dependencies.
Alexandre Courbot
maintainerRe: [PATCH v5 07/15] gpu: nova-core: wait for GFW boot in probe, not in the Gpu constructor
Quoted 33 lines · Danilo Krummrich
drivers/gpu/nova-core/driver.rs fn probe<'bound>( · line 126
Quoted 11 lines · John Hubbard
drivers/gpu/nova-core/driver.rs +7
-0@@ -113,6 +117,14 @@ fn probe<'bound>( pdev.iomap_region (bar1_idx,c"nova-core/bar1" )? }, + _: { +let spec =Spec ::new (pdev.as_ref (), bar)?; + +// We must wait for GFW_BOOT completion before doing any significant setup on +// the GPU. + gpu::wait_gfw_boot_completion (pdev.as_ref (), bar, spec.chipset)?; + },So, apparently this is done in order to be able to add this
#[cfg(CONFIG_NOVA_CORE_SELFTESTS)] crate::irq::doorbell_test::run_selftest(pdev, bar, spec.chipset)?;in a subsequent patch.
I prefer to make
wait_gfw_boot_completion()an associated function of Gpu, keep it called fromGpu::new()and guard the whole block with #[cfg(CONFIG_NOVA_CORE_SELFTESTS)], such that eventually it becomes:#[cfg(CONFIG_NOVA_CORE_SELFTESTS)] _: { let spec = Spec::new(pdev.as_ref(), bar)?; Gpu::wait_gfw_boot_completion(pdev.as_ref(), bar, spec.chipset)?; irq::doorbell_test::run_selftest(pdev, bar, spec.chipset)?; },
Quoted 2 lines
With this
Gpu::new()remains self-contained and we don't build up corss initialization dependencies.
Indeed, this is architecturally more solid. I can apply the change upon pushing.
PATCH v5 8/15 gpu: nova-core: add an interrupt delivery self-test
In reply to John HubbardA GPU interrupt can be lost in the MSI or MSI-X allocation, in the GIN tree's enables, or in the rearm, and every one of those failures looks the same: no interrupt arrives, and no register or log line says which one broke.
Add a probe-time self-test, built under NOVA_CORE_SELFTESTS, that latches the CPU doorbell vector through the GIN software trigger and waits for a registered handler to service it. One delivery would pass with a broken rearm, because the first message-signaled interrupt arrives whether or not the driver rearms, so the test triggers twice and waits for the first handler to finish before the second trigger. It runs after GFW boot and before GSP boot, on a quiesced tree, and fails probe unless both deliveries arrive, each finds only the doorbell pending, and the leaf ends clear.
The doorbell has the same vector on every supported GPU, so the test names it without asking GSP-RM. It allocates the PCI vectors for the doorbell's subtree and releases them before returning, so under MSI-X the delivery also exercises that subtree's table entry.
--- drivers/gpu/nova-core/Kconfig | 5 + drivers/gpu/nova-core/driver.rs | 5 + drivers/gpu/nova-core/irq.rs | 2 + drivers/gpu/nova-core/irq/doorbell_test.rs | 257 ++++++++++++++++++++ drivers/gpu/nova-core/irq/interrupt_tree.rs | 15 ++ drivers/gpu/nova-core/nova_core.rs | 2 +- 6 files changed, 285 insertions(+), 1 deletion(-) create mode 100644 drivers/gpu/nova-core/irq/doorbell_test.rs
drivers/gpu/nova-core/Kconfig +5 -0
@@ -24,4 +24,9 @@ config NOVA_CORE_SELFTESTS help Build the driver self-tests and run them when the GPU is probed. + If the interrupt delivery test fails, the probe fails and the driver + does not bind to the GPU. A broken interrupt path would otherwise + show up later as a hang, far from its cause. Every other self-test + logs its failure and lets the probe continue. + If unsure, say N.
drivers/gpu/nova-core/driver.rs +5 -0
@@ -123,6 +123,11 @@ fn probe<'bound>(// We must wait for GFW_BOOT completion before doing any significant setup on // the GPU. gpu::wait_gfw_boot_completion (pdev.as_ref (), bar, spec.chipset)?; + +// The self-test disables and drains the whole tree, so it has to run before +// `Gpu::new` boots the GSP. +#[cfg( CONFIG_NOVA_CORE_SELFTESTS )] +crate ::irq::doorbell_test::run_selftest (pdev, bar, spec.chipset)?; },// TODO: Use self-referential pin-init syntax once available.
drivers/gpu/nova-core/irq.rs +2 -0
@@ -9,6 +9,8 @@//! //! See `Documentation/gpu/nova/core/interrupts.rst`. +#[cfg( CONFIG_NOVA_CORE_SELFTESTS )] +pub (crate )mod doorbell_test;mod hal;pub (crate )mod interrupt_tree;mod regs;
drivers/gpu/nova-core/irq/doorbell_test.rs new file +257 -0
257 lines not on this page · raw message
drivers/gpu/nova-core/irq/interrupt_tree.rs +15 -0
@@ -444,6 +444,21 @@ pub(super) fn read_pending(&self, leaf: LeafIndex) -> LeafPending<'a> { } } +/// Sets the pending bit of `vector` in its leaf, as if the vector's source had raised it. +/// +/// # Errors +/// +/// `EINVAL` if this tree does not implement `vector`. +#[cfg_attr(not( CONFIG_NOVA_CORE_SELFTESTS ), expect(dead_code))] +pub (super )fn trigger (&self, vector:GinVector ) ->Result { + vector.validate (self.hal.leaf_count ())?; + self.bar.write_reg ( +NV_VIRTUAL_FUNCTION_PRIV_CPU_INTR_LEAF_TRIGGER ::zeroed ().with_vector (vector), + ); + +Ok (()) + } +/// Disables every vector in every implemented leaf, including the subtrees that nova-core does /// not service. Call this only during probe. pub (super )fn disable_all_leaves (&self) {
drivers/gpu/nova-core/nova_core.rs +1 -1
@@ -18,7 +18,7 @@mod fsp;mod gpu;mod gsp; -#[expect(dead_code)] +#[cfg_attr(not( CONFIG_NOVA_CORE_SELFTESTS ), expect(dead_code))] mod irq;mod mctp;mod mm;
Signature · 1 line
2.55.0
PATCH v5 9/15 gpu: nova-core: log GSP events instead of discarding them
In reply to John HubbardThe GSP posts unsolicited messages on the same queue that carries command replies: log records, OS error and robust-channel records, and lifecycle notices.
nova-core discarded every message that was not the reply that a caller was waiting for, and an unrecognized function code aborted the in-flight command. The GSP's error reports never reached the kernel log.
Log every non-reply message according to its function code, on the receive path that already reads it, and leave the in-flight command waiting for its reply. Event payloads, such as XID numbers and log contents, are not decoded.
--- drivers/gpu/nova-core/gsp/cmdq.rs | 52 ++++++++++++++++++++++++------- 1 file changed, 41 insertions(+), 11 deletions(-)
drivers/gpu/nova-core/gsp/cmdq.rs +41 -11
@@ -559,8 +559,7 @@ fn notify_gsp(bar: Bar0<'_>) {/// Sends `command` to the GSP and waits for the reply. /// -/// Messages with non-matching function codes are silently consumed until the expected reply -/// arrives. +/// Events that arrive before the reply are logged and consumed. /// /// The queue is locked for the entire send+receive cycle to ensure that no other command can /// be interleaved.
@@ -819,8 +818,8 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {/// Receive a message from the GSP. /// -/// The expected message type is specified using the `M` generic parameter. If the pending -/// message has a different function code, `ERANGE` is returned and the message is consumed. +/// A message whose function code is `M::FUNCTION` is decoded and returned. Any other message +/// is logged as an event. /// /// The read pointer is always advanced past the message, regardless of whether it matched. ///
@@ -829,8 +828,7 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {/// - `ETIMEDOUT` if `timeout` has elapsed before any message becomes available. /// - `EIO` if there was some inconsistency (e.g. message shorter than advertised) on the /// message queue. -/// - `EINVAL` if the function code of the message was not recognized. -/// - `ERANGE` if the message had a recognized but non-matching function code. +/// - `ERANGE` if the message was not the awaited reply. /// /// Error codes returned by [`MessageFromGsp::read`] are propagated as-is. fn receive_msg <M :MessageFromGsp >(&mut self, timeout:Delta ) ->Result <M >
@@ -839,11 +837,11 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M> Error: From<M::InitError>, {let message = self.wait_for_msg (timeout)?; -let function = message.header.function ().map_err (|_|EINVAL )?; +let function = message.header.function (); +let seq = message.header.sequence (); -// Extract the message. Store the result as we want to advance the read pointer even in -// case of failure. -let result =if function ==M ::FUNCTION { +// An early return here would leave the read pointer on this message. +let result =if matches! (function,Ok (f)if f ==M ::FUNCTION ) {let (cmd, contents_1) =M ::Message ::from_bytes_prefix (message.contents.0 ).ok_or (EIO )?;let mut sbuffer =SBufferIter ::new_reader ([contents_1, message.contents.1 ]);
@@ -854,11 +852,13 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M>dev_warn! ( &self.dev,"GSP message {:?} has unprocessed data\n" , - function +M ::FUNCTION ); } }) }else { + self.log_event (function, seq); +Err (ERANGE ) };
@@ -869,4 +869,34 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M> result } + +/// Logs an event, meaning a message that no caller was waiting for. +/// +/// An OS error or robust-channel record is logged at error level and an unknown function code +/// at warning level. Every other event is recorded only by the receive trace in +/// [`Self::wait_for_msg`]. +fn log_event (&self, function:Result <MsgFunction ,u32 >, seq:u32 ) { +match function { +Ok (MsgFunction ::OsErrorLog ) => { +dev_err! (&self.dev,"GSP reported an OS error (seq {})\n" , seq); + } +Ok (MsgFunction ::RcTriggered ) => { +dev_err! ( + &self.dev, +"GSP triggered robust-channel recovery (seq {})\n" , + seq + ); + } +// Nothing to do for the remaining known function codes. +Ok (_) => {} +Err (raw) => { +dev_warn! ( + &self.dev, +"unknown GSP message function {:#x} (seq {})\n" , + raw, + seq + ); + } + } + } }
Signature · 1 line
2.55.0
PATCH v5 10/15 gpu: nova-core: return ENOMSG for an unmatched GSP message
In reply to John HubbardThe GSP posts unsolicited events on the same queue as command replies, so a receive that asks for one message type has to report that the message at the queue head was a different one.
That case returned ERANGE, which means a value outside a valid range and says nothing about a message.
Return ENOMSG, no message of the desired type, instead.
--- drivers/gpu/nova-core/gsp/cmdq.rs | 6 +++--- drivers/gpu/nova-core/gsp/commands.rs | 2 +- drivers/gpu/nova-core/gsp/sequencer.rs | 2 +- 3 files changed, 5 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/gsp/cmdq.rs +3 -3
@@ -585,7 +585,7 @@ pub(crate) fn send_command<M>(&self, command: M) -> Result<M::Reply>loop {match inner.receive_msg ::<M ::Reply >(Self ::RECEIVE_TIMEOUT ) {Ok (reply) =>break Ok (reply), -Err (ERANGE ) =>continue , +Err (ENOMSG ) =>continue ,Err (e) =>break Err (e), } }
@@ -828,7 +828,7 @@ fn wait_for_msg(&self, timeout: Delta) -> Result<GspMessage<'_>> {/// - `ETIMEDOUT` if `timeout` has elapsed before any message becomes available. /// - `EIO` if there was some inconsistency (e.g. message shorter than advertised) on the /// message queue. -/// - `ERANGE` if the message was not the awaited reply. +/// - `ENOMSG` if the message was not the awaited reply. /// /// Error codes returned by [`MessageFromGsp::read`] are propagated as-is. fn receive_msg <M :MessageFromGsp >(&mut self, timeout:Delta ) ->Result <M >
@@ -859,7 +859,7 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M> }else { self.log_event (function, seq); -Err (ERANGE ) +Err (ENOMSG ) };// Advance the read pointer past this message.
drivers/gpu/nova-core/gsp/commands.rs +1 -1
@@ -191,7 +191,7 @@ pub(crate) fn wait_gsp_init_done(cmdq: &Cmdq<'_>) -> Result {loop {match cmdq.receive_msg ::<GspInitDone >(Cmdq ::RECEIVE_TIMEOUT ) {Ok (_) =>break Ok (()), -Err (ERANGE ) =>continue , +Err (ENOMSG ) =>continue ,Err (e) =>break Err (e), } }
drivers/gpu/nova-core/gsp/sequencer.rs +1 -1
@@ -346,7 +346,7 @@ pub(crate) fn run(let seq_info =loop {match cmdq.receive_msg ::<GspSequence >(Cmdq ::RECEIVE_TIMEOUT ) {Ok (seq_info) =>break seq_info, -Err (ERANGE ) =>continue , +Err (ENOMSG ) =>continue ,Err (e) =>return Err (e), } };
Signature · 1 line
2.55.0
PATCH v5 11/15 gpu: nova-core: bound a GSP wait by a single deadline
In reply to John HubbardThe GSP posts unsolicited events on the same queue as command replies, so a caller waiting for one message consumes whatever arrives first and reads again.
Every read started a fresh five-second timeout, so a steady stream of events extended the wait without bound. The two boot-time waits for an unsolicited event also released the queue mutex between reads, so a command sent from another thread could consume the event and leave the waiter to time out.
Compute one deadline when the wait begins and pass the time remaining to each read, and hold the queue mutex across the whole wait. Put the loop in one helper that the command reply wait and both boot-time event waits share.
--- drivers/gpu/nova-core/gsp/cmdq.rs | 72 ++++++++++++++++++++------ drivers/gpu/nova-core/gsp/commands.rs | 8 +-- drivers/gpu/nova-core/gsp/sequencer.rs | 8 +-- 3 files changed, 59 insertions(+), 29 deletions(-)
drivers/gpu/nova-core/gsp/cmdq.rs +57 -15
@@ -28,7 +28,11 @@ }, Mutex, // }, - time::Delta, + time::{ + Delta, + Instant, + Monotonic, // + }, transmute::{AsBytes ,FromBytes ,//
@@ -130,7 +134,9 @@ fn size(&self) -> usize {/// Trait representing messages received from the GSP. /// -/// This trait tells [`Cmdq::receive_msg`] how it can receive a given type of message. +/// A reply that [`Cmdq::send_command`] waits for, or an event that [`Cmdq::await_msg`] waits for. +/// The receiver matches a message's function code against [`Self::FUNCTION`] and decodes the +/// message with [`Self::read`]. pub (crate )trait MessageFromGsp :Sized {/// Function identifying this message from the GSP. const FUNCTION :MsgFunction ;
@@ -566,8 +572,9 @@ fn notify_gsp(bar: Bar0<'_>) {/// /// # Errors /// -/// - `ETIMEDOUT` if space does not become available to send the command, or if the reply is -/// not received within the timeout. +/// - `ETIMEDOUT` if space does not become available to send the command, or if the reply does +/// not arrive within [`Self::RECEIVE_TIMEOUT`] of the send, however many events arrive +/// while waiting. /// - `EIO` if the variable payload requested by the command has not been entirely /// written to by its [`CommandToGsp::init_variable_payload`] method. ///
@@ -582,13 +589,7 @@ pub(crate) fn send_command<M>(&self, command: M) -> Result<M::Reply>let mut inner = self.inner.lock (); inner.send_command (command)?; -loop { -match inner.receive_msg ::<M ::Reply >(Self ::RECEIVE_TIMEOUT ) { -Ok (reply) =>break Ok (reply), -Err (ENOMSG ) =>continue , -Err (e) =>break Err (e), - } - } + inner.await_msg () }/// Sends `command` to the GSP without waiting for a reply.
@@ -608,15 +609,25 @@ pub(crate) fn send_command_no_wait<M>(&self, command: M) -> Result self.inner.lock ().send_command (command) } -/// Receive a message from the GSP. +/// Waits for an unsolicited GSP event of type `M`. Events that arrive before it are logged and +/// consumed. +/// +/// The queue mutex is held for the whole wait, up to [`Self::RECEIVE_TIMEOUT`], so no other +/// caller can send a command or consume an event meanwhile. /// -/// See [`CmdqInner::receive_msg`] for details. -pub (crate )fn receive_msg <M :MessageFromGsp >(&self, timeout:Delta ) ->Result <M > +/// # Errors +/// +/// - `ETIMEDOUT` if the event does not arrive within [`Self::RECEIVE_TIMEOUT`] of the call, +/// however many other events arrive while waiting. +/// - `EIO` if a message fails framing or checksum validation. +/// +/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is. +pub (crate )fn await_msg <M :MessageFromGsp >(&self) ->Result <M >where // This allows all error types, including `Infallible`, to be used for `M::InitError`. Error :From <M ::InitError >, { - self.inner.lock ().receive_msg (timeout) + self.inner.lock ().await_msg () } }
@@ -870,6 +881,37 @@ fn receive_msg<M: MessageFromGsp>(&mut self, timeout: Delta) -> Result<M> result } +/// Receives a message of type `M`, waiting up to [`Cmdq::RECEIVE_TIMEOUT`] from the call. +/// +/// Any other message that arrives first is logged as an event and does not extend the +/// deadline. +/// +/// # Errors +/// +/// - `ETIMEDOUT` if no message of type `M` arrives before the deadline, however many other +/// messages arrive while waiting. +/// - `EIO` if a message fails framing or checksum validation (see [`Self::wait_for_msg`]). +/// +/// Error codes returned by [`MessageFromGsp::read`] are propagated as-is. +fn await_msg <M :MessageFromGsp >(&mut self) ->Result <M > +where +// This allows all error types, including `Infallible`, to be used for `M::InitError`. +Error :From <M ::InitError >, + { +let deadline =Instant ::<Monotonic >::now () +Cmdq ::RECEIVE_TIMEOUT ; +loop { +let remaining = deadline -Instant ::<Monotonic >::now (); +if remaining.is_negative () { +break Err (ETIMEDOUT ); + } +match self.receive_msg ::<M >(remaining) { +Ok (msg) =>break Ok (msg), +Err (ENOMSG ) =>continue , +Err (e) =>break Err (e), + } + } + } +/// Logs an event, meaning a message that no caller was waiting for. /// /// An OS error or robust-channel record is logged at error level and an unknown function code
drivers/gpu/nova-core/gsp/commands.rs +1 -7
@@ -188,13 +188,7 @@ fn read(/// Waits for GSP initialization to complete. pub (crate )fn wait_gsp_init_done (cmdq: &Cmdq <'_>) ->Result { -loop { -match cmdq.receive_msg ::<GspInitDone >(Cmdq ::RECEIVE_TIMEOUT ) { -Ok (_) =>break Ok (()), -Err (ENOMSG ) =>continue , -Err (e) =>break Err (e), - } - } + cmdq.await_msg ::<GspInitDone >().map (|_| ()) }/// The `GetGspStaticInfo` command.
drivers/gpu/nova-core/gsp/sequencer.rs +1 -7
@@ -343,13 +343,7 @@ pub(crate) fn run( libos: &'a Coherent<'a, [LibosMemoryRegionInitArgument]>, bootloader_app_version: u32, ) -> Result { -let seq_info =loop { -match cmdq.receive_msg ::<GspSequence >(Cmdq ::RECEIVE_TIMEOUT ) { -Ok (seq_info) =>break seq_info, -Err (ENOMSG ) =>continue , -Err (e) =>return Err (e), - } - }; +let seq_info = cmdq.await_msg ::<GspSequence >()?; let sequencer = GspSequencer { bar: ctx.bar,
Signature · 1 line
2.55.0
PATCH v5 12/15 gpu: nova-core: add the falcon interrupt registers and HAL methods
In reply to John HubbardA falcon latches each interrupt cause that is raised in its IRQSTAT register. On a RISC-V falcon, each cause is routed either to the host, meaning the CPU, or to the falcon's own core, and two routing registers hold that routing. The falcon signals the interrupt tree only when its set of host-routed causes goes from empty to non-empty, so a handler that clears the tree leaf while a cause is still latched has to write INTR_RETRIGGER, which makes the falcon re-emit its causes. Turing falcons have no INTR_RETRIGGER, and the routing registers have different offsets from GA102 on.
Add IRQSTAT, INTR_RETRIGGER, and the two routing registers, the last at both sets of offsets, and add two falcon HAL methods: one intersects an IRQSTAT value with the routing registers, and one retriggers the falcon, or does nothing on Turing. GA100 keeps the Turing HAL, as it does for boot, and the retrigger register is a property of that HAL's instance.
The two methods have no user yet.
--- drivers/gpu/nova-core/falcon/hal.rs | 21 ++++++- drivers/gpu/nova-core/falcon/hal/ga102.rs | 16 ++++++ drivers/gpu/nova-core/falcon/hal/tu102.rs | 51 ++++++++++++++++- drivers/gpu/nova-core/regs.rs | 69 +++++++++++++++++++++++ 4 files changed, 154 insertions(+), 3 deletions(-)
drivers/gpu/nova-core/falcon/hal.rs +20 -1
@@ -12,6 +12,7 @@ Architecture, Chipset, // }, + regs, };mod ga102;
@@ -70,6 +71,24 @@ fn signature_reg_fuse_version(/// these. For anything above, the PIO registers appear to be masked to the CPU, so DMA is the /// only usable method. fn load_method (&self) ->LoadMethod ; + +/// Returns the causes in `latched` that are routed to the host, meaning the CPU, rather than +/// to the falcon's own RISC-V core. +/// +/// The causes routed to the core belong to the firmware running on it, and the host does not +/// service them. +#[expect(dead_code)] +fn host_routed_causes ( + &self, + falcon: &Falcon <'_,E >, + latched: regs::NV_PFALCON_FALCON_IRQSTAT , + ) -> regs::NV_PFALCON_FALCON_IRQSTAT ; + +/// Retriggers the falcon, which then re-emits its host-routed causes into the interrupt tree. +/// +/// Turing falcons have no retrigger register, so on Turing this does nothing. +#[expect(dead_code)] +fn retrigger (&self, falcon: &Falcon <'_,E >); }/// Returns a boxed falcon HAL adequate for `chipset`.
@@ -86,7 +105,7 @@ pub(super) fn falcon_hal<E: FalconEngine + 'static>( }// GA100 boots like Turing so use Turing HAL Architecture::Ampere if chipset == Chipset::GA100 => { - KBox::new(tu102::Tu102::<E>::new(), GFP_KERNEL)? as KBox<dyn FalconHal<E>> + KBox::new(tu102::Tu102::<E>::ga100(), GFP_KERNEL)? as KBox<dyn FalconHal<E>> } Architecture::Ampere | Architecture::Ada
drivers/gpu/nova-core/falcon/hal/ga102.rs +16 -0
@@ -169,4 +169,20 @@ fn reset_eng(&self, falcon: &Falcon<'_, E>) -> Result {fn load_method (&self) ->LoadMethod {LoadMethod ::Dma } + +fn host_routed_causes ( + &self, + falcon: &Falcon <'_,E >, + latched: regs::NV_PFALCON_FALCON_IRQSTAT , + ) -> regs::NV_PFALCON_FALCON_IRQSTAT { +let pfalcon2 = falcon.pfalcon2; +let mask = pfalcon2.read (regs::ga102::NV_PRISCV_RISCV_IRQMASK ).value (); +let dest = pfalcon2.read (regs::ga102::NV_PRISCV_RISCV_IRQDEST ).value (); + + regs::NV_PFALCON_FALCON_IRQSTAT ::from (latched.into_raw () & mask & dest) + } + +fn retrigger (&self, falcon: &Falcon <'_,E >) { +super ::tu102::retrigger_ga100 (falcon); + } }
drivers/gpu/nova-core/falcon/hal/tu102.rs +49 -2
73 lines not on this page · raw message
drivers/gpu/nova-core/regs.rs +69 -0
98 lines not on this page · raw message
Signature · 1 line
2.55.0
PATCH v5 13/15 gpu: nova-core: service GSP events from the SWGEN0 interrupt
In reply to John HubbardThe GSP posts each message for the CPU to the GSP-to-CPU queue and then raises SWGEN0, a software-generated interrupt cause of its falcon. GIN delivers SWGEN0 to the CPU on vector 155.
A thread waiting for a command reply was the only reader of the queue, so an event posted between commands stayed unread until the next command was sent.
Register a threaded handler on the GSP vector. Draining the queue takes the command-queue mutex, which can sleep, so the top half reads and writes only registers and wakes the IRQ thread to drain the queue. Every existing read of the queue asks for one message type and waits on a deadline, so add a drain that logs and consumes whatever the GSP has already posted and returns at once.
When a falcon cause other than SWGEN0 stays set after its latch is cleared, as the Blackwell fault-containment and ECC causes do, disable the GSP vector rather than retrigger the falcon. A retrigger would re-emit the cause at once, and the CPU would take the same interrupt again and again.
--- drivers/gpu/nova-core/falcon/gsp.rs | 47 ++++- drivers/gpu/nova-core/falcon/hal.rs | 2 - drivers/gpu/nova-core/falcon/hal/tu102.rs | 2 - drivers/gpu/nova-core/gpu.rs | 96 +++++++++- drivers/gpu/nova-core/gsp/cmdq.rs | 40 ++++ drivers/gpu/nova-core/irq.rs | 1 + drivers/gpu/nova-core/irq/gsp.rs | 196 ++++++++++++++++++++ drivers/gpu/nova-core/irq/interrupt_tree.rs | 3 + drivers/gpu/nova-core/nova_core.rs | 1 - 9 files changed, 375 insertions(+), 13 deletions(-) create mode 100644 drivers/gpu/nova-core/irq/gsp.rs
drivers/gpu/nova-core/falcon/gsp.rs +45 -2
@@ -47,13 +47,56 @@ fn pfalcon2(io: Bar0<'_>) -> Mmio<'_, super::PFalcon2Registers> { }impl <'a>Falcon <'a,Gsp > { -/// Clears the SWGEN0 bit in the Falcon's IRQ status clear register to -/// allow GSP to signal CPU for processing new messages in message queue. +/// Clears the SWGEN0 latch in the GSP falcon. +/// +/// While the latch is set, no later message signals the tree, so a caller that consumed a +/// notification by polling must clear it. pub (crate )fn clear_swgen0_intr (&self) { self.pfalcon .write_reg (regs::NV_PFALCON_FALCON_IRQSCLR ::zeroed ().with_swgen0 (true )); } +/// Reads the GSP falcon causes that are routed to the host, without clearing any latch. +/// +/// Every one of them other than SWGEN0 reports a GSP fault. +pub (crate )fn read_host_intr (&self) -> regs::NV_PFALCON_FALCON_IRQSTAT { +let latched = self.pfalcon.read (regs::NV_PFALCON_FALCON_IRQSTAT ); + + self.hal.host_routed_causes (self, latched) + } + +/// Reads the host-routed causes and clears the SWGEN0 latch if it was set. +/// +/// Returns the causes as read, before the clear. No other latch changes. +pub (crate )fn take_host_intr (&self) -> regs::NV_PFALCON_FALCON_IRQSTAT { +let status = self.read_host_intr (); + +if status.swgen0 () { + self.clear_swgen0_intr (); + } + + status + } + +/// Clears the latch of every interrupt cause set in `status`. +/// +/// A cause driven from outside the falcon is still set on return, and +/// [`Self::read_host_intr`] reports the causes that remain. +pub (crate )fn clear_intr (&self, status: regs::NV_PFALCON_FALCON_IRQSTAT ) { + self.pfalcon + .write_reg (regs::NV_PFALCON_FALCON_IRQSCLR ::from (status.into_raw ())); + } + +/// Retriggers the GSP falcon, which then re-emits its host-routed causes into the tree. +/// +/// Call this only once every host cause is clear. A cause still set is re-emitted at once, and +/// its vector arrives again as soon as delivery is rearmed. +/// +/// Does nothing on Turing, whose falcons have no retrigger register. +pub (crate )fn retrigger_intr (&self) { + self.hal.retrigger (self); + } +/// Checks if GSP reload/resume has completed during the boot process. pub (crate )fn check_reload_completed (&self, timeout:Delta ) ->Result <bool > { read_poll_timeout(
drivers/gpu/nova-core/falcon/hal.rs +0 -2
@@ -77,7 +77,6 @@ fn signature_reg_fuse_version(/// /// The causes routed to the core belong to the firmware running on it, and the host does not /// service them. -#[expect(dead_code)] fn host_routed_causes ( &self, falcon: &Falcon<'_, E>,
@@ -87,7 +86,6 @@ fn host_routed_causes(/// Retriggers the falcon, which then re-emits its host-routed causes into the interrupt tree. /// /// Turing falcons have no retrigger register, so on Turing this does nothing. -#[expect(dead_code)] fn retrigger (&self, falcon: &Falcon <'_,E >); }
drivers/gpu/nova-core/falcon/hal/tu102.rs +0 -2
@@ -28,7 +28,6 @@/// retrigger register. pub (super )struct Tu102 <E :FalconEngine > {/// If `true`, the falcons have `NV_PFALCON_FALCON_INTR_RETRIGGER`. -#[expect(dead_code)] has_intr_retrigger:bool , _engine:PhantomData <E >, }
@@ -52,7 +51,6 @@ pub(super) fn ga100() -> Self { }/// Writes `NV_PFALCON_FALCON_INTR_RETRIGGER`. -#[expect(dead_code)] pub (super )fn retrigger_ga100 <E :FalconEngine >(falcon: &Falcon <'_,E >) { falcon.pfalcon.write ( Array::at(0),
drivers/gpu/nova-core/gpu.rs +90 -6
140 lines not on this page · raw message
drivers/gpu/nova-core/gsp/cmdq.rs +40 -0
@@ -629,6 +629,20 @@ pub(crate) fn await_msg<M: MessageFromGsp>(&self) -> Result<M> { self.inner.lock ().await_msg () } + +/// Logs and consumes every message the GSP has already posted, and returns without waiting for +/// more. +/// +/// No caller is waiting for a reply while this holds the queue mutex, so every message is +/// logged as an event. See "Draining the GSP-to-CPU queue" in +/// `Documentation/gpu/nova/core/interrupts.rst`. +/// +/// # Errors +/// +/// `EIO` if a message fails framing or checksum validation. +pub (crate )fn drain (&self) ->Result { + self.inner.lock ().drain () + } }/// Inner mutex protected state of [`Cmdq`].
@@ -941,4 +955,30 @@ fn log_event(&self, function: Result<MsgFunction, u32>, seq: u32) { } } } + +/// Logs and consumes every message the queue holds. +/// +/// # Errors +/// +/// `EIO` if a message fails framing or checksum validation, or if a message's page count +/// overflows a `u32`. +fn drain (&mut self) ->Result { +while !self.gsp_mem.driver_read_area ().0 .is_empty () { +// A message is available, so this returns without waiting. +let msg = self.wait_for_msg (Delta ::ZERO )?; + +let pages = + u32::try_from (msg.header.length ().div_ceil (GSP_PAGE_SIZE )).map_err (|_| { +dev_err! (&self.dev,"GSP drain: message length overflow\n" ); +EIO + })?; +let function = msg.header.function (); +let seq = msg.header.sequence (); + + self.gsp_mem.advance_cpu_read_ptr (pages); + self.log_event (function, seq); + } + +Ok (()) + } }
drivers/gpu/nova-core/irq.rs +1 -0
@@ -11,6 +11,7 @@#[cfg( CONFIG_NOVA_CORE_SELFTESTS )] pub (crate )mod doorbell_test; +pub (crate )mod gsp;mod hal;pub (crate )mod interrupt_tree;mod regs;
drivers/gpu/nova-core/irq/gsp.rs new file +196 -0
196 lines not on this page · raw message
drivers/gpu/nova-core/irq/interrupt_tree.rs +3 -0
@@ -124,10 +124,12 @@ pub(super) const fn all() -> Self {Self (u32::MAX ) } +#[cfg_attr(not( CONFIG_NOVA_CORE_SELFTESTS ), expect(dead_code))] pub (super )const fn from_raw (raw:u32 ) ->Self {Self (raw) } +#[cfg_attr(not( CONFIG_NOVA_CORE_SELFTESTS ), expect(dead_code))] pub (super )const fn into_raw (self) ->u32 { self.0 }
@@ -240,6 +242,7 @@ pub(super) const fn new<const VECTOR: u32>() -> Self {Self (Bounded ::<u32 ,VECTOR_BITS >::new ::<VECTOR >()) } +#[cfg_attr(not( CONFIG_NOVA_CORE_SELFTESTS ), expect(dead_code))] pub (super )const fn into_raw (self) ->u32 { self.0 .get () }
drivers/gpu/nova-core/nova_core.rs +0 -1
@@ -18,7 +18,6 @@mod fsp;mod gpu;mod gsp; -#[cfg_attr(not( CONFIG_NOVA_CORE_SELFTESTS ), expect(dead_code))] mod irq;mod mctp;mod mm;
Signature · 1 line
2.55.0
PATCH v5 14/15 gpu: nova-core: add KUnit tests for the interrupt tree
In reply to John HubbardThe vector arithmetic and the leaf and subtree sets touch no hardware, so KUnit can cover them without a GPU. A wrong leaf, bit or subtree for a vector would otherwise show up only as a lost interrupt on hardware.
Add the nova_core_gin_tree suite. It checks that a leaf index stops at the widest supported tree, that a leaf count implies the right number of subtrees and vectors, that a leaf count enumerates every leaf in order, that a vector maps to the right leaf, bit and subtree, and that a vector beyond an 8-leaf tree is rejected there and accepted in a 16-leaf tree. It also exercises the subtree set operations and checks that every supported chipset implements the subtree that carries the GSP event.
--- drivers/gpu/nova-core/irq/interrupt_tree.rs | 128 +++++++++++++++++++- 1 file changed, 127 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/irq/interrupt_tree.rs +127 -1
@@ -129,7 +129,10 @@ pub(super) const fn from_raw(raw: u32) -> Self {Self (raw) } -#[cfg_attr(not( CONFIG_NOVA_CORE_SELFTESTS ), expect(dead_code))] +#[cfg_attr( +not(any( CONFIG_NOVA_CORE_SELFTESTS , CONFIG_KUNIT = "y" )), +expect(dead_code) +)] pub (super )const fn into_raw (self) ->u32 { self.0 }
@@ -554,3 +557,126 @@ fn drop(&mut self) {clear_top_enables (self.bar, self.serviced); } } + +#[kunit_tests(nova_core_gin_tree)] +mod tests { +use super ::*; + +/// A leaf index cannot name a leaf beyond the widest supported tree. +#[test] +fn leaf_index_bounds () { +assert! (LeafIndex ::try_new (0 ).is_some ()); +assert! (LeafIndex ::try_new (15 ).is_some ()); +assert! (LeafIndex ::try_new (16 ).is_none ()); + } + +/// The subtree count, the implemented-subtree set, and the vector count follow the leaf count. +#[test] +fn leaf_count_derives_subtrees_and_vectors () { +assert_eq! (LeafCount ::Eight .subtree_count (),4 ); +assert_eq! ( +Bounded ::<u32 ,32 >::from (LeafCount ::Eight .subtree_set ()).get (), +0x0f + ); +assert_eq! (LeafCount ::Eight .vector_count (),256 ); + +assert_eq! (LeafCount ::Sixteen .subtree_count (),8 ); +assert_eq! ( +Bounded ::<u32 ,32 >::from (LeafCount ::Sixteen .subtree_set ()).get (), +0xff + ); +assert_eq! (LeafCount ::Sixteen .vector_count (),512 ); + } + +/// A tree enumerates every leaf that it implements, in order, and no more. +#[test] +fn leaf_count_iter_covers_the_tree () { +for (count, expected)in [(LeafCount ::Eight ,8usize ), (LeafCount ::Sixteen ,16 )] { +let mut seen =0 ; + +for (index, leaf)in count.iter ().enumerate () { +assert_eq! (leaf.get (), index); + seen +=1 ; + } + +assert_eq! (seen, expected); + } + } + +/// A vector maps to its leaf, its bit within that leaf, and its subtree. The doorbell (129) +/// and the GSP event (155) share a subtree. +#[test] +fn vector_maps_to_leaf_bit_and_subtree () { +let doorbell =GinVector ::new ::<129 >(); +let gsp =GinVector ::new ::<155 >(); + +assert_eq! (doorbell.leaf_index ().get (),4 ); +assert_eq! (doorbell.leaf_mask ().into_raw (),1 <<1 ); +assert_eq! (doorbell.subtree ().index (),2 ); + +assert_eq! (gsp.leaf_index ().get (),4 ); +assert_eq! (gsp.leaf_mask ().into_raw (),1 <<27 ); +assert_eq! (gsp.subtree ().index (),2 ); + +assert_eq! (doorbell.subtree (), gsp.subtree ()); + } + +/// Both fixed vectors are within the 8-leaf tree, so every supported chipset implements them. +#[test] +fn fixed_vectors_fit_the_narrowest_tree () { +assert! (GinVector ::new ::<129 >().validate (LeafCount ::Eight ).is_ok ()); +assert! (GinVector ::new ::<155 >().validate (LeafCount ::Eight ).is_ok ()); + +// The first vector beyond an 8-leaf tree. +assert! (GinVector ::new ::<256 >().validate (LeafCount ::Eight ).is_err ()); +assert! (GinVector ::new ::<256 >().validate (LeafCount ::Sixteen ).is_ok ()); + } + +/// A subtree set reports membership, intersection, and its span from subtree 0. +#[test] +fn subtree_set_operations () { +let gsp =GinVector ::new ::<155 >().subtree (); + +assert! (LeafCount ::Eight .subtree_set ().contains (gsp)); +assert! (!LeafCount ::Eight .subtree_set ().is_empty ()); + +// The GSP needs no subtree above 2, so an MSI-X request covers entries 0 through 2. +assert_eq! (SubtreeSet ::from (gsp).span (),3 ); + +// A 16-leaf tree implements every subtree that an 8-leaf tree does. +assert_eq! ( +LeafCount ::Sixteen + .subtree_set () + .intersection (LeafCount ::Eight .subtree_set ()), +LeafCount ::Eight .subtree_set () + ); + } + +/// Iterating a subtree set yields each subtree once, lowest index first, and nothing for an +/// empty set. +#[test] +fn subtree_set_iterates_its_members () { +assert! (LeafCount ::Eight + .subtree_set () + .iter () + .map (Subtree ::index) + .eq ([0u32 ,1 ,2 ,3 ])); + +let gsp =SubtreeSet ::from (GinVector ::new ::<155 >().subtree ()); +assert! (gsp.iter ().map (Subtree ::index).eq ([2u32 ])); + +let empty =SubtreeSet ::from (Bounded ::<u32 ,32 >::new ::<0 >()); +assert_eq! (empty.iter ().count (),0 ); + } + +/// Every supported chipset implements the subtree that carries the GSP event. +#[test] +fn gsp_subtree_is_implemented_everywhere () { +for &chipsetin Chipset ::ALL { +assert! (cpu_interrupt_hal (chipset) + .leaf_count () + .subtree_set () + .contains (crate ::irq::gsp::GSP_SUBTREE )); + } + } +}
Signature · 1 line
2.55.0
PATCH v5 15/15 gpu: nova-core: document the GIN interrupt controller and GSP events
In reply to John HubbardThe interrupt code rests on hardware behavior that the code cannot show on its own: how GIN, the GPU's interrupt controller, records and delivers interrupts, what edge-triggered delivery requires of a handler, and how the GSP signals the CPU. Some of those requirements come from OpenRM rather than from the hardware manuals.
Add a design document that records that behavior, the rules that nova-core follows because of it, and the terms that the code uses for it. The code comments cite the document by section rather than repeating it.
--- Documentation/gpu/nova/core/interrupts.rst | 679 +++++++++++++++++++++ Documentation/gpu/nova/index.rst | 1 + 2 files changed, 680 insertions(+) create mode 100644 Documentation/gpu/nova/core/interrupts.rst
Documentation/gpu/nova/core/interrupts.rst new file +679 -0
@@ -0,0 +1,679 @@ +.. SPDX-License-Identifier: GPL-2.0 +.. SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + +============================================= +GPU interrupt handling: GIN and the GSP event +============================================= + +This document describes how nova-core receives interrupts from the GPU on Turing +and later chipsets. It covers the GPU Interrupt and Notification unit (GIN), +which is the GPU's interrupt controller, and the GSP event, the interrupt that +nova-core services in normal operation. + +Throughout, *CPU* means the CPU and the nova-core driver running on it. The GPU +also has on-chip processors that run their own firmware and receive their own +interrupts. The GSP (GPU System Processor) is one of them. + +Register names are the names from the GPU hardware reference headers. The +pre-Hopper headers call the controller ``NV_CTRL`` and the Hopper-plus headers +call it ``NV_GIN``. This document calls it GIN throughout, because the tree that +nova-core services is the same on every supported chipset. "Register naming" +at the end says how the names map onto the headers. OpenRM, NVIDIA's +open-source GPU kernel driver, is cited wherever nova-core follows it. + +Terminology +=========== + +The three levels of the controller, innermost first: + +leaf + One ``LEAF`` register. Each of its 32 bits is the pending bit of one + interrupt source. A Turing, Ampere, or Ada tree has 8 leaves. A Hopper or + Blackwell tree has 16. + +subtree + Two consecutive leaves, summarized by one bit of ``TOP``. A subtree is the + unit of enabling at ``TOP``, and under MSI-X it is the unit of delivery: + every interrupt from one subtree arrives on one MSI-X entry. + +tree + One ``TOP`` register and the leaves under it. Every PCIe function has its + own tree, and nova-core services the CPU tree of one function. + +The hardware headers, OpenRM, and the Linux PCI API all use the word "vector", +each for a different number. This document gives each one its own name, and a +bare "vector" always means a GIN vector. + +GIN vector + The GPU-internal interrupt source number. It addresses one bit of one leaf. + A 16-leaf tree holds vectors 0 through 511, and an 8-leaf tree holds 0 + through 255. The CPU doorbell is vector 129 and the GSP event is vector + 155. + +MSI-X entry + An index into the device's MSI-X table. One entry serves one subtree. + +PCI vector + One of the interrupts that ``pci_alloc_irq_vectors()`` allocates: an MSI-X + entry, or the single MSI message. + +Linux IRQ number + What ``request_irq()`` takes, obtained from ``pci_irq_vector()`` for a PCI + vector. Linux's ``struct msix_entry`` calls this number ``.vector`` as + well. + +The remaining terms, each named for the register or the specification that +defines it: + +enable, disable a vector + Writes to ``LEAF_EN_SET`` and ``LEAF_EN_CLEAR``. + +enable, disable a subtree + Writes to ``TOP_EN_SET`` and ``TOP_EN_CLEAR``. + +serviced subtree + A subtree that nova-core enables and has a handler for. + +rearm + Restoring PCI interrupt delivery after servicing an interrupt. See + "Rearming PCI interrupt delivery". + +mask + Reserved for the two places where hardware and the PCI specification use + the word: the MSI-X per-entry Vector Control mask bit, which Linux + controls, and the falcon interrupt masks. It never names a GIN enable. + +latched, pending + Two names for one state, a set ``LEAF`` bit. The vector's source sets the + bit whether or not the vector is enabled. + +clear a vector + Write a 1 to the vector's bit in ``LEAF``. OpenRM calls the same operation + ``intrClearLeafVector_HAL``. + +pending bits + The plain 32-bit value read from a ``LEAF`` register. + +notification + An interrupt whose only content is that something happened, such as a + posted message. Servicing a notification means reading what it announces. + The unit that raised it needs no attention. The GSP event is one. + +unit + Any block that raises an interrupt. "Engine" is reserved for the blocks + that do user work: GR, CE, NVDEC, and the like. + +falcon + One of the GPU's microcontrollers (see + Documentation/gpu/nova/core/falcon.rst). The GSP runs on the RISC-V core + inside its falcon. A falcon latches each of its interrupt causes and routes + it either to the host, meaning the CPU, or to its own core. + +The GIN controller +================== + +A GPU has many interrupt sources: the GSP, the copy engines, the graphics +engine, video decode and encode, the MMU fault path, timers, and others. GIN +records which of them are pending and raises the PCI interrupt to the CPU. + +Trees +----- + +GIN keeps one tree for each destination it can deliver an interrupt to. The CPU +has one tree per PCIe function, so the physical function and each virtual +function have their own. The GSP has a tree, and so do the other on-chip +processors that receive interrupts. Every tree has the same two-level layout, +and a function reaches its own tree through the per-function register aperture. + +nova-core services the CPU tree of one function. A virtual function's tree +belongs to that function's driver, and a processor's tree belongs to the +firmware running on that processor. + +The two-level tree +------------------ + +A tree is a set of ``LEAF`` registers and one ``TOP`` register. + +* ``LEAF(i)`` is a 32-bit register that holds the pending bits of vectors + ``32i`` through ``32i + 31``. A set bit is a pending vector. +* ``TOP`` is a 32-bit read-only register. Bit ``N`` summarizes subtree ``N``, + which is ``LEAF(2N)`` and ``LEAF(2N + 1)``. The bit is set when an enabled + vector is pending in either leaf. + +A tree with L leaves has L / 2 subtrees and uses TOP bits 0 through L / 2 - 1. +The other TOP bits read 0. The leaves and subtrees that a chipset has are its +implemented leaves and subtrees, and "Per-architecture differences" gives the +counts. For an 8-leaf tree:: + + TOP bit 0 -> subtree 0 -> LEAF(0), LEAF(1) vectors 0..63 + TOP bit 1 -> subtree 1 -> LEAF(2), LEAF(3) vectors 64..127 + TOP bit 2 -> subtree 2 -> LEAF(4), LEAF(5) vectors 128..191 + TOP bit 3 -> subtree 3 -> LEAF(6), LEAF(7) vectors 192..255 + + LEAF(4), one bit per vector, holds vectors 128..159: + + bit 1 = vector 129 (CPU doorbell) + bit 27 = vector 155 (GSP event) + +A vector's number fixes its place in the tree:: + + leaf = vector / 32 + bit = vector % 32 + subtree = leaf / 2 + +Registers +--------- + +nova-core defines the tree's registers in the ``irq`` module's ``regs.rs``. The +leaf registers are arrays indexed by leaf number. + +* ``LEAF(i)`` reads as the pending bits of leaf ``i``. Writing a 1 to a bit + clears that vector, and a 0 leaves the bit as it was. +* ``LEAF_EN_SET(i)`` and ``LEAF_EN_CLEAR(i)`` enable and disable the vectors + of leaf ``i``, one bit per vector. +* ``TOP_EN_SET`` and ``TOP_EN_CLEAR`` enable and disable subtrees, one bit per + subtree. +* ``LEAF_TRIGGER`` takes a vector number and latches that vector, exactly as + the vector's own source would. It is write-only. The self-test uses it. + +nova-core does not read ``TOP``. "Servicing the tree" says why. + +Every set and clear register acts per bit: a 1 performs the action for that +bit, and a 0 leaves the bit alone. No register needs a read-modify-write. + +GIN delivers a vector to the CPU only when its leaf enable bit and its +subtree's TOP enable bit are both set. The enables do not affect the latch. The +source of a disabled vector still sets its ``LEAF`` bit. ``TOP`` does not show +that bit, so reading the leaf is the only way to see it. + +How a unit interrupt reaches the CPU +------------------------------------ + +A unit does not write a ``LEAF`` register. Each unit has an interrupt control +register that GSP firmware programs. The control register holds the unit's +vector, the GFID that identifies the PCIe function whose tree receives the +interrupt, and one enable bit per destination: the CPU, the GSP, and the other +on-chip processors. When the unit has an event:: + + 1. The unit sends GIN an interrupt message carrying the vector, the GFID, + and the destination enables from its control register. + 2. In the tree of each destination that the message selects, GIN sets bit + (vector % 32) of LEAF(vector / 32). + 3. If the vector and its subtree are enabled in the CPU tree, GIN raises + the PCI interrupt. + +Because firmware assigns the vectors, nova-core does not hardcode which vector +belongs to which unit, with two exceptions. The hardware headers of every +supported chipset define the GSP event as vector 155, and GSP firmware's own +interrupt table uses that definition. The CPU doorbell is vector 129. Pre-Hopper +hardware fixes that number, and GSP firmware keeps it on Hopper and later. +nova-core names both by number. A driver can fetch the full unit-to-vector +table from the GSP by RPC, and nouveau does. nova-core does not, because a +fixed vector needs no lookup. + +Edge-triggered delivery +----------------------- + +A ``LEAF`` bit is a latch. Its source sets it on a rising edge, and it stays set +until the CPU clears it. A source that stays high does not set the bit again. + +GIN raises the PCI interrupt for a subtree when the subtree's enabled pending +state goes from low to high:: + + Per vector, in leaf i at bit b: + LEAF(i)[b] AND LEAF_EN(i)[b] + + Per subtree N, across leaves 2N and 2N + 1: + OR of every enabled pending bit -> TOP[N] + + Delivery for subtree N: + TOP[N] AND TOP_EN[N] -> rising edge -> PCI interrupt + +``TOP_EN`` applies after the summary, so disabling a subtree stops delivery +without changing what ``TOP`` reports. + +Three consequences: + +* Code that must find every pending vector reads the leaves. A vector that + latched while disabled is not in ``TOP``. +* Writing ``TOP_EN_SET`` for a subtree with an enabled pending bit produces a + new edge. GIN delivers an interrupt for a pending bit left uncleared as soon + as its subtree is enabled again. +* A source that holds its signal high produces no new edge after the CPU + clears the leaf bit. Such a source has to re-emit its interrupt. The falcons + do that through ``INTR_RETRIGGER`` (see "Retriggering a falcon"). + +Delivery over PCI +================= + +GIN delivers the tree's interrupts to the CPU as MSI or MSI-X, whichever Linux +grants. nova-core requests MSI-X first and falls back to MSI, and never uses +INTx. + +MSI has a single message, and every subtree raises that one message, so one +Linux IRQ serves the whole tree. + +MSI-X gives each subtree its own table entry, at the index equal to the subtree +number. Linux masks every entry until a driver requests its Linux IRQ number, +and a masked entry sends no message: the GPU records the interrupt in the MSI-X +pending bit array, where it stays until Linux unmasks the entry. An entry that +the driver never requests is never unmasked. A driver that enables a subtree +without requesting that subtree's entry loses every interrupt from that +subtree, with nothing reported: the leaf and TOP registers show the vector +pending and enabled while no handler runs. + +The serviced-subtree invariant +------------------------------ + +Every subtree enabled at ``TOP`` has an allocated PCI vector with a registered +handler. + +MSI satisfies this with its single message. MSI-X needs one allocated entry per +serviced subtree, and a PCI allocation cannot be sparse, so nova-core requests +entries 0 through the highest serviced subtree:: + + MSI-X, with subtree 2 serviced: + + subtree 0 -> entry 0 allocated, no handler, stays masked + subtree 1 -> entry 1 allocated, no handler, stays masked + subtree 2 -> entry 2 handler here, and its rearm covers subtree 2 + + MSI, with any serviced set: + + every serviced subtree -> the one allocated PCI vector, whose + handler's rearm covers the whole serviced set + +An allocated entry whose subtree nova-core does not service costs nothing. The +entry stays masked, and a disabled subtree raises no interrupt. + +nova-core services one subtree. The GSP event, vector 155, is in leaf 4, which +is in subtree 2. OpenRM's headers place its UVM_SHARED interrupt category in +subtree 2 on every chipset that nova-core supports. The self-test doorbell, +vector 129, is in the same leaf, and the test allocates its own vectors for it +(see "Self-test"). + +Rearming PCI interrupt delivery +------------------------------- + +A message-signaled interrupt is delivered once per edge, and the PCI side +delivers no further interrupt until the CPU rearms it. The rearm operation +depends on the GPU family and on the interrupt type that Linux granted: + +================== ===== =========================================== +Architecture Type Rearm operation +================== ===== =========================================== +Turing through Ada MSI write the configuration-mirror EOI register +Hopper and later MSI clear then set the serviced TOP enables +Any MSI-X clear then set the handler's own TOP enable +================== ===== =========================================== + +The end-of-interrupt register is ``NV_XVE_CYA_2`` in the BAR0 mirror of PCI +configuration space, and the value written does not matter. The ``TOP_EN`` +cycle produces a new delivery edge. The MSI forms cover every serviced subtree, +because one message serves all of them. The MSI-X form covers one subtree, +because each serviced subtree has its own entry and its own handler. + +A handler rearms once per delivered interrupt, on every path, including the +path where it finds its vector not pending. A handler that skips the rearm +receives no further interrupts. + +OpenRM makes the same split. It writes the configuration-space EOI for MSI on +pre-Hopper chipsets, and cycles the TOP enables of the subtrees that it +services for Hopper-plus MSI and for MSI-X. + +Servicing the tree +================== + +Servicing a leaf has a required order: read its pending bits, then clear them. +Clearing a leaf before reading it discards every vector latched in it, and no +register reports the loss. In nova-core, reading a leaf produces the handle that +clears it, so that the wrong order does not compile. The handle clears exactly +the bits that it read, so a vector that latched after the read stays pending. + +A handler clears its bit before it services the vector. Clearing afterwards +would discard an interrupt that the source raised while the handler ran. + +nova-core services the tree in two ways. + +The notification path services one vector. It reads the vector's leaf, clears +only the vector's bit, and rearms. The subtree stays enabled, and a vector +pending beside it in the same leaf keeps its bit set for the code that services +that vector. The GSP event handler takes this path, and so does the self-test +handler. + +The startup drain walks the whole tree, because it must clear whatever is +pending across every subtree rather than one known vector. It disables the +serviced subtrees at ``TOP``, reads and clears every implemented leaf, and +leaves the subtrees disabled for its caller to enable once the caller is ready +for deliveries. The drain reads every leaf rather than descending from ``TOP``, +because sources latch vectors during boot while those vectors are disabled, and +``TOP`` does not show them. OpenRM's stall-interrupt path reads every leaf for +the same reason. + +The two paths as register operations:: + + Startup drain, run once during probe: + write TOP_EN_CLEAR = serviced stop new deliveries + for each implemented leaf i: + pending = read LEAF(i) + write LEAF(i) = pending clear what was read + (returns with TOP_EN still clear) + + Notification, the subtree stays enabled: + pending = read LEAF(leaf) is the handler's bit set? + write LEAF(leaf) = bit clear that one bit + rearm PCI interrupt delivery + +The drain clears every pending bit, including bits that nova-core never +services. An uncleared bit holds its subtree in the pending state, and enabling +that subtree again would deliver an interrupt for a vector that no handler +services. + +The drain's ``TOP_EN_CLEAR`` is not a rearm, and pre-Hopper MSI rearms through +the configuration mirror, which the drain never writes. An interrupt delivered +before probe had no handler to rearm it, so the tree reset rearms explicitly +after the drain. A TOP-enable rearm leaves the serviced subtrees enabled, so +the reset then disables them again. + +nova-core does not serialize access to the tree. The GSP event handler +touches only its own leaf, and the drain runs during probe, before that handler +is registered. + +Per-architecture differences +============================ + +The tree is the same on every supported GPU except for its size, which changes +at Hopper: + +=================== ====== ======== ==================== +GPUs Leaves Subtrees Implemented subtrees +=================== ====== ======== ==================== +Turing, Ampere, Ada 8 4 ``0x0f`` +Hopper, Blackwell 16 8 ``0xff`` +=================== ====== ======== ==================== + +The interrupt HAL provides the leaf count, and the subtree count and the +implemented-subtree set derive from it. A subtree that the chipset does not +implement has no TOP bit, so building a tree that services one fails with +``EINVAL``. Vectors 129 and 155 are in the 8-leaf tree, so every supported +chipset has them. + +OpenRM's headers assign every interrupt category of a 16-leaf tree to leaves 0 +through 11. The drain reads all 16, because a vector can be latched in any +implemented leaf. + +The HAL performs the rearm as well (see "Rearming PCI interrupt delivery"). Two +falcon properties also differ by family, and the falcon HAL carries them: +Turing falcons have no ``INTR_RETRIGGER``, and the RISC-V routing registers +moved at GA102 (see "Retriggering a falcon"). + +The GSP event +============= + +When the GSP has output for the CPU, it writes messages into the GSP-to-CPU +queue in shared memory and raises SWGEN0, one of the software-generated +interrupt causes of the GSP falcon. SWGEN0 is routed to the host, at vector +155, leaf 4 bit 27, in subtree 2. + +The queue carries notifications (log records, error records, lifecycle events) +and command replies. A thread waiting for a reply reads the queue itself, so +the interrupt is only the trigger to drain the queue (see "Draining the +GSP-to-CPU queue"). + +The falcon latches every cause that it raises, SWGEN0 among them, in its +``IRQSTAT`` register. The handler services the host-routed causes and clears +their latches, and then it writes ``INTR_RETRIGGER`` so that the falcon re-emits +any cause that latched in the meantime. "Retriggering a falcon" has the details. + +Draining the queue takes the command-queue mutex and walks shared memory, so it +cannot run in hard interrupt context. nova-core registers a threaded handler, +under the name ``nova-core`` in ``/proc/interrupts``. The top half runs in hard +interrupt context and reads and writes only registers, and it wakes the IRQ +thread to drain the queue:: + + GSP writes messages into the GSP-to-CPU queue + GSP raises SWGEN0 + GIN sets bit 27 of LEAF(4), and subtree 2 becomes pending + PCI interrupt -> Linux IRQ -> top half, in hard interrupt context: + read LEAF(4), and if bit 27 is clear, rearm and return + clear bit 27 (the subtree stays enabled) + read the falcon causes routed to the host, clearing SWGEN0 if set + for every other host cause: log it, clear its latch, and read the + host causes back + if the clear ended all of them: retrigger the falcon + otherwise: disable vector 155 at its leaf and skip the retrigger + rearm PCI interrupt delivery + wake the IRQ thread if SWGEN0 was set + IRQ thread, which may sleep: + take the command-queue mutex and drain the GSP-to-CPU queue + +A halt and a posted message can be pending together, so the top half services +every cause that the status reports. + +A drain fails when a message's framing or checksum is bad (see "Draining the +GSP-to-CPU queue"). The message stays at the queue head, so every later drain +would fail the same way, and the IRQ thread disables vector 155 and logs the +failure, which leaves the queue unserviced until the device is reset. + +The handler, the self-test, and the rest of the driver read BAR0 through one +shared mapping. nova-core unregisters an interrupt handler when the device +unbinds, so a handler runs only while the mapping exists. + +Retriggering a falcon +--------------------- + +A falcon signals the tree when its set of host-routed causes goes from empty to +non-empty. A cause left latched keeps the set non-empty, so no later cause +signals the tree, and the vector is lost. For a cause that stays latched, the +handler can clear the tree leaf first or the falcon latch first, and the loss +is the same. + +``IRQSTAT`` latches every cause in the falcon, including the causes routed to +the falcon's own RISC-V core and owned by the firmware running on it. A host +handler owns only the causes that ``PRISCV_RISCV_IRQMASK`` and +``PRISCV_RISCV_IRQDEST`` both select, so it intersects ``IRQSTAT`` with both +before it reads or clears a cause. OpenRM computes the same intersection in +``kflcnRiscvReadIntrStatus``. GA100 keeps the Turing offsets of the two routing +registers and GA102 moves them, so the offsets change at GA102 rather than at +the Ampere boundary. The handler masks no cause: ``PRISCV_RISCV_IRQMASK`` is +read-only to the host, and ``FALCON_IRQMASK`` has no effect on host routing on +a RISC-V falcon. + +``INTR_RETRIGGER`` makes the falcon re-emit its host-routed causes into the +tree, which supplies the transition that clearing the leaf lost. The handler +writes ``INTR_RETRIGGER`` only on a path where it ended every cause that it +read, because a re-emitted cause that stays set arrives again at once and +on every pass after that. + +``IRQSCLR`` ends a latch and does not end the source behind it, so a cause +driven from outside the falcon stays set after the write. On Blackwell the +fault-containment and ECC causes are driven that way: they appear in +``IRQSTAT`` but come from ``PRISCV_RISCV_FAULT_CONTAINMENT_SRCSTAT`` and +``PGSP_ECC_INTR_STATUS``, and only a device reset ends them. So the handler +clears the latch of every host cause other than SWGEN0, reads the host causes +back, and retriggers only when the read-back is empty. When a cause is still +set, the handler disables vector 155 instead and reports that the device needs +a reset. Disabling loses no notification: the cause that is still set holds the +host-routed set non-empty, so the falcon would signal nothing further either +way. OpenRM makes the same choice, and ``kgspService_TU102`` skips +``kflcnIntrRetrigger`` once it has recorded a fatal error. + +The handler cannot distinguish a fault cause that arrives after the clear from +one that the clear failed to end, so it disables the vector in that case too. +Both mean that the GSP has faulted. + +Turing falcons have no ``INTR_RETRIGGER``, so a Turing handler cannot re-create +a transition that it has lost. It must leave no host cause latched: it reads the +host-routed status once and takes every cause that the status reports, rather +than stopping at the first one that it recognizes. One window stays open. A +cause that arrives after the handler has read the status is not in the value +that the handler clears, so it stays latched after the leaf has been cleared, +and no later cause from that falcon signals the tree. OpenRM has the same window +on Turing, where ``kflcnIntrRetrigger`` does nothing. + +Enabling the GSP event +---------------------- + +SWGEN0 is a latch, and the GSP drives no new edge into the tree while it stays +set. nova-core's GSP boot code consumes the GSP's notifications by polling the +queue, which leaves the latch set and leaves pending bits in the tree. The +handoff from polling to interrupts has a required order:: + + disable every implemented vector drop enables left by boot, or by a + driver that ran before this one + drain the tree clear stale pending bits + rearm PCI interrupt delivery required under pre-Hopper MSI, which + no later step rearms + disable subtree 2 at TOP a TOP-enable rearm enabled it again + clear the SWGEN0 latch so the next message makes an edge + register the threaded handler no delivery can reach it yet + enable vector 155 at LEAF(4) the subtree is still disabled + enable subtree 2 at TOP deliveries become possible here + drain the GSP-to-CPU queue messages posted before the clear + +The first four steps are the tree reset. nova-core runs them, and clears the +latch, before it registers the handler. Registering unmasks the PCI interrupt, +and GIN would then deliver a vector that boot left enabled to a handler that +services one vector and has no way to service any other. OpenRM clears every +leaf enable at the same point for the same reason. + +nova-core clears the latch after the drain. If nova-core cleared the latch +first, a message posted before the drain could set it again, along with bit 27 +of LEAF(4). The drain would then clear the leaf bit while the latch stays set, +and no later message would signal the tree. Clearing after the drain can +instead leave the leaf bit pending with the latch already clear. Enabling the +subtree then delivers one interrupt whose ``IRQSTAT`` reads zero. The top half +clears the leaf bit, rearms, and does not wake the IRQ thread, and the queue +drain that follows reads the message. + +Clearing the latch makes the first interrupt possible. A message that the GSP +posted before that clear produces no interrupt, so the sequence ends by +draining the queue. + +The subtree is enabled at ``TOP`` only once the handler is registered. A +TOP-enable rearm sets the enables that it cycles, so the reset disables the +serviced subtrees again after its rearm, and the tree reaches the registration +with subtree 2 disabled under every rearm method. On teardown nova-core disables +the vector at its leaf, so that no vector in the subtree can be delivered, then +calls ``free_irq()``, and disables the subtree last. Disabling the subtree +earlier would let a handler still in flight enable it again through the +``TOP_EN`` cycle of its rearm, which would leave the subtree enabled with no +handler registered. The driver's ``Gpu`` object, which owns the tree, holds the +subtree enable, and a handler's registration holds only the enable of its own +vector, so that tearing down one handler does not disable a subtree that another +handler shares. nova-core tears down the registration before it frees the queue +that the handler drains and the falcon that the handler reads, and before it +unloads the GSP. + +Draining the GSP-to-CPU queue +----------------------------- + +The queue carries command replies and unsolicited events, and a message's +function code says which it is. + +* A function code that matches the awaited reply: the message is decoded and + returned to the caller that sent the command. +* Anything else is an event. An OS error record and a robust-channel record + are logged at error level, and an unrecognized function code at warning + level. The other known events (GSP logs, libos prints, assertion records, + lifecycle notices) need no action and get no line of their own, because the + receive trace at debug level already records every message's arrival with + its sequence number, function code, and length. + +The sequence number takes no part in the match, because the GSP does not echo +the command's sequence number on every reply. On r570 the reply to +``UnloadingGuestDriver`` carries sequence 0. + +The read pointer advances past every message that passes framing and checksum +validation, whether it matched or was an event. A matched message that is too +short to decode is the exception, and it stays at the queue head. + +A message's length is inside the region that the checksum covers, so once the +framing or the checksum fails there is no trustworthy length with which to skip +the message. Such a message also stays at the queue head, and every later +receive fails on it with ``EIO`` until the device is reset. + +The polling path and the IRQ thread both read the queue under the command-queue +mutex. Replies and events share one queue and one read pointer, so one lock is +held across the whole drain. A thread waiting for a reply logs each event that +arrives before the reply and keeps waiting. One deadline of 5 seconds applies to +the whole wait, rather than a fresh timeout after each message, and the thread +holds the mutex for the whole wait, so that no other caller consumes the message +that it waits for. + +With one lock, a drain waits for an in-flight command's receive to finish or +time out. For log and error records that delay does not matter. + +Self-test +========= + +The self-test confirms that an interrupt injected at the GPU is delivered to a +registered handler. It runs during probe, after the GPU's boot firmware, GFW, +has completed, and before nova-core boots the GSP, because the test disables +and drains the whole tree, including the GSP's leaf. The test is built only +under ``CONFIG_NOVA_CORE_SELFTESTS``, like the other probe-time hardware tests. +Those other tests log a failure and let probe continue. A failed delivery test +fails probe, because an interrupt path that does not work leaves the driver +unable to make progress later, in a place that says nothing about the cause. + +The test writes ``LEAF_TRIGGER`` with vector 129, the CPU doorbell, at leaf 4 +bit 1. The vector then takes the ordinary path to the CPU under the ordinary +enables. The doorbell keeps the same number on every supported chipset, so the +test names it without asking the GSP, which is not running yet. + +The test allocates PCI vectors for subtree 2 and resets the tree, which +disables every vector, drains the tree, and rearms delivery. It checks that the +doorbell bit starts out clear. It registers a non-threaded handler with a +completion, under the name ``nova-core-selftest`` in ``/proc/interrupts``, and +enables the vector and the subtree. It triggers the doorbell, waits up to 1000 +ms for the first delivery, triggers it again, and waits up to 1000 ms for the +second. The handler takes the notification path: it clears only its own bit and +rearms, and never walks the tree. + +The test triggers twice, and the second trigger waits for the first handler to +finish. One delivery would prove nothing about the rearm, because the first +message-signaled interrupt arrives whether the driver rearms or not, and two +triggers in a row could coalesce into one delivery. A handler that walked the +tree would prove nothing either: on every configuration except pre-Hopper MSI +the rearm is a ``TOP_EN`` cycle, so a walk that enabled ``TOP`` again would +rearm delivery whether the handler asked for it or not. + +The test passes only if both deliveries arrive, each finds the doorbell bit and +nothing else pending in leaf 4, and the bit is clear once the vector is +disabled. Anything else fails probe. Requiring the exact pending bits on the +second delivery shows that the first handler's clear reached the hardware. No +other vector in leaf 4 can be pending, because the test disabled every vector +and runs before GSP boot. + +The test releases its PCI vectors and its handler before returning, so the +driver's own allocation covers the GSP subtree and nothing else. Sharing the +driver's allocation would have let the test pass only because the doorbell and +the GSP event happen to be in the same subtree. + +The test exercises the path from the GPU to the handler without GSP firmware, +which helps when bringing up PCI, MSI, MSI-X, and passthrough setups. Under +MSI-X a pass also shows that the delivery arrived on the entry belonging to the +serviced subtree. + +The vector arithmetic and the leaf and subtree sets depend on no hardware, and +KUnit tests cover them instead. + +Register naming +=============== + +nova-core uses the ``NV_VIRTUAL_FUNCTION_PRIV_CPU_INTR_*`` names for the CPU +tree on every supported chipset. That is the per-function aperture: each PCIe +function reaches its own tree through it, at the same offsets. The controller +also has a central aperture that exposes every function's tree. The pre-Hopper +headers name it ``NV_CTRL_CPU_INTR_*`` and the Hopper-plus headers +``NV_GIN_CPU_INTR_*``. nova-core does not use it. OpenRM's kernel-side code +for the controller is the ``Intr`` object. + +Scope +===== + +nova-core services the CPU tree of one function and nothing else. It implements +no virtual-function tree management and no GFID routing, which belong to the +physical function's driver or to firmware in a virtualized setup, and no MIG +(multi-instance GPU) support. It services none of the subtrees that the +hardware headers reserve for the stall interrupts of the host-driven engines.
Documentation/gpu/nova/index.rst +1 -0
@@ -35,3 +35,4 @@ vGPU manager VFIO driver and the nova-drm driver. core/falcon core/tlv core/pramin + core/interrupts
Signature · 1 line
2.55.0
Alexandre Courbot
maintainerRe: [PATCH v5 15/15] gpu: nova-core: document the GIN interrupt controller and GSP events
Quoted 11 lines · John Hubbard
The interrupt code rests on hardware behavior that the code cannot show on its own: how GIN, the GPU's interrupt controller, records and delivers interrupts, what edge-triggered delivery requires of a handler, and how the GSP signals the CPU. Some of those requirements come from OpenRM rather than from the hardware manuals.
Add a design document that records that behavior, the rules that nova-core follows because of it, and the terms that the code uses for it. The code comments cite the document by section rather than repeating it.
Quoted 6 lines
Assisted-by LLMSigned-off-by John Hubbard <[email protected]>--- Documentation/gpu/nova/core/interrupts.rst | 679 +++++++++++++++++++++ Documentation/gpu/nova/index.rst | 1 + 2 files changed, 680 insertions(+)
I took the freedom to remove the last 3 paragraphs because they read like filler and didn't provide any useful information. The self-test in particular should be self-contained (and mostly is) and describing it at length here carries the risk that the code will eventually diverge from the doc.
But the rest was informative. Maybe we can give it another pass later to smoothen it further, but even as-is the document is helpful imho.
Quoted 5 lines · John Hubbard
This series adds GPU interrupt support to nova-core, so that GSP events reach the driver as interrupts instead of only when the driver polls. v5 applies Alex's review of v4, and makes three of the design changes that he had set aside for after the merge. The series is 15 patches, down from 17. The two
rust/patches are unchanged.
With patch 7 and 8 adjusted accordingly,
Alexandre Courbot
maintainerQuoted 19 lines · John Hubbard
This series adds GPU interrupt support to nova-core, so that GSP events reach the driver as interrupts instead of only when the driver polls. v5 applies Alex's review of v4, and makes three of the design changes that he had set aside for after the merge. The series is 15 patches, down from 17. The two
rust/patches are unchanged.GIN, the GPU Interrupt and Notification unit, is the GPU's interrupt controller, and it latches pending interrupts in a two-level register tree. TOP, the tree's top-level enable register, holds one enable bit per subtree.
The design uses a threaded IRQ handler. The top half reads and writes only GPU registers, while the threaded bottom half drains the message queue.
Still TODO after this: retrieve the dynamic interrupt routing tables from GSP, and use those, instead of the simple, hard-coded approach that was done here for these first two interrupts.
Quoted 7 lines
The series is based on drm-rust-next at commit
658c5f0042e6("gpu:
nova-core: use a single try_pin_init!() block inprobe()"), and it carries no prerequisite commits.The patches, as applied to drm-rust-next, are on a git branch:
https://github.com/johnhubbard/linux/tree/nova-core-gin-interrupt-tree-v5/
This revision is a quite significant improvement, and although there are still a few things I want to fix before applying (notably on - guess what - the HAL :)), this is in the drive-by-upon-apply range. I'll leave a few review comments for the sake of completeness, but there should be no need to send a new revision, unless I missed something important on my first pass.
Quoted 6 lines · John Hubbard
This series adds GPU interrupt support to nova-core, so that GSP events reach the driver as interrupts instead of only when the driver polls. v5 applies Alex's review of v4, and makes three of the design changes that he had set aside for after the merge. The series is 15 patches, down from 17. The two
rust/patches are unchanged.
Quoted 6 lines
GIN, the GPU Interrupt and Notification unit, is the GPU's interrupt controller, and it latches pending interrupts in a two-level register tree. TOP, the tree's top-level enable register, holds one enable bit per subtree.
[...]
Applied to drm-rust-next, thanks!
[01/15] rust: pci: declare IrqType and IrqTypes with impl_flags
commit: 38c9764d330dfe1d59009179904e313b0923f7b4
[03/15] gpu: nova-core: add the GIN vector, leaf and subtree types
commit: e6c579b496f2740c447d215ef868dc5f9cc7dc93[04/15] gpu: nova-core: add the GIN CPU interrupt tree and MSI EOI registers
[acourbot: keep item that doesn't need to be exported private.]
commit: 2ef55e142d29970641e9be04897457bb98a8eaca
[05/15] gpu: nova-core: add the per-architecture GIN CPU interrupt HAL
commit: 4022bcaa34e2115e175f6a64022ec19a186b0d72
[06/15] gpu: nova-core: add the GIN interrupt tree and allocate its vectors
commit: 939b40877cb883de1fa335e950ff1a3f3bf113ed
[07/15] gpu: nova-core: wait for GFW boot in probe, not in the Gpu constructor
[acourbot: move code to associated function, move relevant hunks to next
patch.]
commit: 94ddc4b4843da8b1858535b13318346cb950800d[08/15] gpu: nova-core: add an interrupt delivery self-test
[acourbot: move relevant bits of previous patch into this one.] [acourbot: remove reference to self-test section of documentation.] [acourbot: replace completions with atomic polling code.]
commit: 9cb3f5d56e81fd97a82ef361b210bc2322daaaa7
[09/15] gpu: nova-core: log GSP events instead of discarding them
commit: 0c551cdc465b9c99da33a21d6ee3b274cfda7006
[10/15] gpu: nova-core: return ENOMSG for an unmatched GSP message
commit: d3cb3ed1a2ddd7cb77d9d270d82d3177ee744d1f
[11/15] gpu: nova-core: bound a GSP wait by a single deadline
commit: 621b455c56376f5469d896b58c2316236a3f939d[12/15] gpu: nova-core: add the falcon interrupt registers and HAL methods
[acourbot: remove implementation-specific comments on interface doc.] [acourbot: make constructor take boolean instead of having two methods.] [acourbot: add NVIDIA copyright headers to modified files.]
commit: e727d1c382a0cb2e10fd7aabef9217d1a37c80c7
[13/15] gpu: nova-core: service GSP events from the SWGEN0 interrupt
[acourbot: trim comment leaking unneeded implementation details.] [acourbot: make Subtree crate-visible here instead of in earlier patch.]
commit: 476abf9f669d87f993429ebb42668169464c907c
[14/15] gpu: nova-core: add KUnit tests for the interrupt tree
commit: 306279ce974d396136544aa88027ee921ac09540
[15/15] gpu: nova-core: document the GIN interrupt controller and GSP events
[acourbot: cut the last 3 sections.]
commit: df718311a84188e26d499301c3921e4c392d557e