gpu: nova-core: consolidate and streamline GSP boot process
Subsystems (5)
Alexandre Courbot
maintainerv2 cover gpu: nova-core: consolidate and streamline GSP boot process
The recently introduced GspBootContext is useful in at least two other places:
- As a replacement for the sequencer's GspSequencerParams,
- As the main parameter of Gsp::unload, for symmetry with Gsp::boot.
This symmetry can be further exploited to make Gsp::boot and Gsp::unload generic over the exact boot context required for the current chip. Doing so requires some more preliminary work that goes beyond this series, but this groundwork is needed regardless.
Another cleanup that can be performed as a consequence is the removal of the BootUnloadGuard. This type is cumbersome as it holds extra references to the boot context, making it difficult to make some of these references mutable (as we want to do with the Falcons that should require exclusive access to load and run firmware). Furthermore, it is only needed a single time in Gsp::boot.
By running the boot process in a local closure, we can catch errors and call Gsp::unload on the unload bundle just as efficiently, and with less code.
Building on this, the series also introduces a proper type for identifying the GSP boot method, and moves the Fsp instance to Gpu to make it available outside of the GSP boot process. This will be notably useful for vGPU support [1].
This series applies on drm-rust-next.
[1] https://lore.kernel.org/all/[email protected]/
---
Changes in v2:
- Replace nonexistent ENODATA with the more appropriate EAGAIN.
- Add patches introducing GspBootMethod.
- Add patches moving Fsp to Gpu.
- Link to v1: https://patch.msgid.link/[email protected]
---
Alexandre Courbot (13):
gpu: nova-core: gsp: sequencer: use GspBootContext
gpu: nova-core: gsp: sequencer: do not store sequence into GspSequencer
gpu: nova-core: gsp: move boot code into local closure
gpu: nova-core: gsp: replace BootUnloadGuard with local handler
gpu: nova-core: gsp: move unload bundle error handling to Gsp::boot
gpu: nova-core: gsp: make unload take GspBootContext
gpu: nova-core: gsp: fold TU102 unload bundle construction into HAL method
gpu: nova-core: gsp: turn FWSEC execution into HAL method
gpu: nova-core: gsp: make use of FWSEC bootloader a property of the TU102 HAL
gpu: nova-core: introduce GspBootMethod
gpu: nova-core: avoid repeated calls to pci::Device::as_ref
gpu: nova-core: gsp: pass GspBootContext mutably
gpu: nova-core: store Fsp instance in Gpu
drivers/gpu/nova-core/firmware.rs | 25 ++- drivers/gpu/nova-core/firmware/fwsec.rs | 5 +- drivers/gpu/nova-core/gpu.rs | 68 +++--- drivers/gpu/nova-core/gsp.rs | 44 +++- drivers/gpu/nova-core/gsp/boot.rs | 142 +++++------- drivers/gpu/nova-core/gsp/hal.rs | 63 +++--- drivers/gpu/nova-core/gsp/hal/ga102.rs | 14 ++ drivers/gpu/nova-core/gsp/hal/gh100.rs | 62 +++-- drivers/gpu/nova-core/gsp/hal/tu102.rs | 385 +++++++++++++++----------------- drivers/gpu/nova-core/gsp/sequencer.rs | 71 +++--- 10 files changed, 436 insertions(+), 443 deletions(-)
---
base-commit: f0c1bb8ead8a9790e7d0339354598b50a9c6789a
change-id: 20260619-nova-bootcontext-401d6107e8fb
Best regards,
Signature · 1 line
Alexandre Courbot <[email protected]>
Alexandre Courbot
maintainerPATCH v2 1/13 gpu: nova-core: gsp: sequencer: use GspBootContext
GspBootContext contains all the resources currently carried by GspSequencerParams, so replace the latter with the former for better integration with the boot process and less code.
--- drivers/gpu/nova-core/gsp/hal/tu102.rs | 21 +++++++------------- drivers/gpu/nova-core/gsp/sequencer.rs | 36 ++++++++++++---------------------- 2 files changed, 20 insertions(+), 37 deletions(-)
drivers/gpu/nova-core/gsp/hal/tu102.rs +7 -14
@@ -37,10 +37,7 @@ GspHal, UnloadBundle, // }, - sequencer::{ - GspSequencer, - GspSequencerParams, // - }, + sequencer::GspSequencer, Gsp, GspBootContext, GspFwWprMeta, //
@@ -336,16 +333,12 @@ fn boot<'a>( }fn post_boot (&self, gsp: &Gsp , ctx: &GspBootContext <'_>, gsp_fw: &GspFirmware ) ->Result { -// Create and run the GSP sequencer. -let seq_params =GspSequencerParams { - bootloader_app_version: gsp_fw.bootloader.app_version, - libos_dma_handle: gsp.libos.dma_handle (), - gsp_falcon: ctx.gsp_falcon, - sec2_falcon: ctx.sec2_falcon, - dev: ctx.dev (), - bar: ctx.bar, - }; -GspSequencer ::run (&gsp.cmdq, seq_params)?; +GspSequencer ::run ( + &gsp.cmdq, + ctx, + gsp.libos.dma_handle (), + gsp_fw.bootloader.app_version, + )?;Ok (()) }
drivers/gpu/nova-core/gsp/sequencer.rs +13 -23
@@ -31,6 +31,7 @@ MessageFromGsp, // }, fw, + GspBootContext, // }, num::FromSafeCast, sbuffer::SBufferIter,
@@ -338,24 +339,13 @@ fn iter(&self) -> GspSeqIter<'_> { } } -/// Parameters for running the GSP sequencer. -pub (crate )struct GspSequencerParams <'a> { -/// Bootloader application version. -pub (crate ) bootloader_app_version:u32 , -/// LibOS DMA handle address. -pub (crate ) libos_dma_handle:u64 , -/// GSP falcon for core operations. -pub (crate ) gsp_falcon: &'aFalcon <Gsp >, -/// SEC2 falcon for core operations. -pub (crate ) sec2_falcon: &'aFalcon <Sec2 >, -/// Device for logging. -pub (crate ) dev: &'a device::Device , -/// BAR0 for register access. -pub (crate ) bar:Bar0 <'a>, -} -impl <'a>GspSequencer <'a> { -pub (crate )fn run (cmdq: &Cmdq , params:GspSequencerParams <'a>) ->Result { +pub (crate )fn run ( + cmdq: &Cmdq , + ctx: &'aGspBootContext <'_>, + libos_dma_handle:u64 , + bootloader_app_version:u32 , + ) ->Result {let seq_info =loop {match cmdq.receive_msg ::<GspSequence >(Cmdq ::RECEIVE_TIMEOUT ) {Ok (seq_info) =>break seq_info,
@@ -366,12 +356,12 @@ pub(crate) fn run(cmdq: &Cmdq, params: GspSequencerParams<'a>) -> Result {let sequencer =GspSequencer { seq_info, - bar: params.bar, - sec2_falcon: params.sec2_falcon, - gsp_falcon: params.gsp_falcon, - libos_dma_handle: params.libos_dma_handle, - bootloader_app_version: params.bootloader_app_version, - dev: params.dev, + bar: ctx.bar, + sec2_falcon: ctx.sec2_falcon, + gsp_falcon: ctx.gsp_falcon, + libos_dma_handle, + bootloader_app_version, + dev: ctx.dev (), };dev_dbg! (sequencer.dev,"Running CPU Sequencer commands\n" );
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v2 2/13 gpu: nova-core: gsp: sequencer: do not store sequence into GspSequencer
In reply to Alexandre CourbotThe sequence is currently stored in the GspSequencer even though its lifetime is limited to GspSequencer::run. This object-oriented design does not play well with the borrow-checker, as GspSequencer::iter borrows a reference to the GspSequencer, which makes it difficult to introduce mutable references in GspBootContext, as we want to do in order to make the Falcon references mutable.
Thus, store the sequence locally in GspSequencer::run, and move iterator creation to GspSeqIter::new so it no longer needs to borrow the whole GspSequencer.
--- drivers/gpu/nova-core/gsp/sequencer.rs | 35 +++++++++++++++------------------- 1 file changed, 15 insertions(+), 20 deletions(-)
drivers/gpu/nova-core/gsp/sequencer.rs +15 -20
@@ -129,8 +129,6 @@ pub(crate) fn new(data: &[u8], dev: &device::Device) -> Result<(Self, usize)> {/// GSP Sequencer for executing firmware commands during boot. pub (crate )struct GspSequencer <'a> { -/// Sequencer information with command data. - seq_info:GspSequence ,/// `Bar0` for register access. bar:Bar0 <'a>,/// SEC2 falcon for core operations.
@@ -271,7 +269,7 @@ fn run(&self, seq: &GspSequencer<'_>) -> Result { }/// Iterator over GSP sequencer commands. -pub (crate )struct GspSeqIter <'a> { +struct GspSeqIter <'a> {/// Command data buffer. cmd_data: &'a [u8 ],/// Current position in the buffer.
@@ -284,6 +282,18 @@ pub(crate) struct GspSeqIter<'a> { dev: &'a device::Device, } +impl <'a>GspSeqIter <'a> { +fn new (seq: &'aGspSequence , dev: &'a device::Device ) ->Self { +Self { + cmd_data: &seq.cmd_data, + current_offset:0 , + total_cmds: seq.cmd_index, + cmds_processed:0 , + dev, + } + } +} +impl <'a>Iterator for GspSeqIter <'a> {type Item =Result <GspSeqCmd >;
@@ -325,20 +335,6 @@ fn next(&mut self) -> Option<Self::Item> { } } -impl <'a>GspSequencer <'a> { -fn iter (&self) ->GspSeqIter <'_> { -let cmd_data = &self.seq_info.cmd_data[..]; - -GspSeqIter { - cmd_data, - current_offset:0 , - total_cmds: self.seq_info.cmd_index, - cmds_processed:0 , - dev: self.dev, - } - } -} -impl <'a>GspSequencer <'a> {pub (crate )fn run ( cmdq: &Cmdq,
@@ -355,7 +351,6 @@ pub(crate) fn run( };let sequencer =GspSequencer { - seq_info, bar: ctx.bar, sec2_falcon: ctx.sec2_falcon, gsp_falcon: ctx.gsp_falcon,
@@ -366,14 +361,14 @@ pub(crate) fn run(dev_dbg! (sequencer.dev,"Running CPU Sequencer commands\n" ); -for cmd_resultin sequencer.iter () { +for cmd_resultin GspSeqIter ::new (&seq_info, sequencer.dev) {match cmd_result {Ok (cmd) => cmd.run (&sequencer)?,Err (e) => {dev_err! ( sequencer.dev,"Error running command at index {}\n" , - sequencer.seq_info.cmd_index + seq_info.cmd_index );return Err (e); }
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 02/13] gpu: nova-core: gsp: sequencer: do not store sequence into GspSequencer
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
Pre-existing issues:
- [High] This problem wasn't introduced by this patch, but the iterator silently terminates on command parsing errors, causing the GSP boot sequence to incorrectly report success instead of aborting. --
commit 67a42d6a50b9bd7a1977aebced3e9453d53bb90e
Author: Alexandre Courbot <[email protected]>
gpu: nova-core: gsp: sequencer: do not store sequence into GspSequencer
This commit refactors the GSP sequencer by moving the command sequence data out of the GspSequencer struct and instead passing it directly when creating the iterator. This resolves borrow-checker limitations and allows for future mutable borrows within the boot context.
Quoted 4 lines
drivers/gpu/nova-core/gsp/sequencer.rs +0
-0
[ ... ]
Quoted 9 lines
drivers/gpu/nova-core/gsp/sequencer.rs +0
-1@@ -355,7 +351,6 @@ pub(crate) fn run( };let sequencer =GspSequencer { - seq_info, bar: ctx.bar, sec2_falcon: ctx.sec2_falcon, gsp_falcon: ctx.gsp_falcon,
drivers/gpu/nova-core/gsp/sequencer.rs pub(crate) fn run( · line 364
Quoted 5 lines
drivers/gpu/nova-core/gsp/sequencer.rs +1
-1@@ -366,14 +361,14 @@ pub(crate) fn run(dev_dbg! (sequencer.dev,"Running CPU Sequencer commands\n" ); -for cmd_resultin sequencer.iter () { +for cmd_resultin GspSeqIter ::new (&seq_info, sequencer.dev) {
[Severity: High]
This isn't a bug introduced by this patch, but doesn't the iterator silently terminate on command parsing errors, causing the GSP boot sequence to incorrectly report success instead of aborting?
Looking at GspSeqIter::next(), when a command fails to parse, the error is swallowed and translated to None:
drivers/gpu/nova-core/gsp/sequencer.rs:GspSeqIter::next() {
...
cmd_result.map_or_else(
|_err| {
dev_err!(self.dev, "Error parsing command at offset {}\n", offset);
None
},
...
}Returning None signals normal exhaustion of the iterator. This causes the loop here in GspSequencer::run() to cleanly exit without ever hitting the Err(e) match arm below.
Consequently, GspSequencer::run() will incorrectly log success and return Ok(()), completely masking the hardware initialization failure and leaving the hardware in a malformed state.
Quoted 11 lines
drivers/gpu/nova-core/gsp/sequencer.rs +1
-1match cmd_result {Ok (cmd) => cmd.run (&sequencer)?,Err (e) => {dev_err! ( sequencer.dev,"Error running command at index {}\n" , - sequencer.seq_info.cmd_index + seq_info.cmd_index );return Err (e); }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2
Alexandre Courbot
maintainerPATCH v2 3/13 gpu: nova-core: gsp: move boot code into local closure
In reply to Alexandre CourbotThe next patch aims at replacing the cumbersome BootUnloadGuard with a more local and less intrusive mechanism to run the GSP unload sequence upon GSP boot failure. Doing so requires running the boot code in a local closure, which changes its indentation and would make other changes difficult to track in the diff. Thus, this preparatory patch moves said boot code into a local closure that is run upon construction, so the next patch does not need to re-indent code that changes.
This is a mechanical preparatory patch to make the next patch easier to read. No functional change intended.
--- drivers/gpu/nova-core/gsp/boot.rs | 42 ++++++++------- drivers/gpu/nova-core/gsp/hal/gh100.rs | 38 ++++++++------ drivers/gpu/nova-core/gsp/hal/tu102.rs | 96 ++++++++++++++++++---------------- 3 files changed, 95 insertions(+), 81 deletions(-)
drivers/gpu/nova-core/gsp/boot.rs +24 -18
@@ -119,30 +119,36 @@ pub(crate) fn boot(// Perform the chipset-specific boot sequence, and retrieve the unload bundle. let unload_guard = hal.boot (&self, &ctx, &fb_layout, &wpr_meta)?; +// Run from a closure so we can retrieve the result, and run the unload sequence of the GSP +// in case of error. +let res = (|| { + gsp_falcon.write_os_version (bar, gsp_fw.bootloader.app_version); - gsp_falcon.write_os_version (bar, gsp_fw.bootloader.app_version); +// Poll for RISC-V to become active before continuing. +read_poll_timeout ( + ||Ok (gsp_falcon.is_riscv_active (bar)), + |val: &bool | *val, +Delta ::from_millis (10 ), +Delta ::from_secs (5 ), + )?; -// Poll for RISC-V to become active before continuing. -read_poll_timeout ( - ||Ok (gsp_falcon.is_riscv_active (bar)), - |val: &bool | *val, -Delta ::from_millis (10 ), -Delta ::from_secs (5 ), - )?; +dev_dbg! (pdev,"RISC-V active? {}\n" , gsp_falcon.is_riscv_active (bar),); -dev_dbg! (pdev,"RISC-V active? {}\n" , gsp_falcon.is_riscv_active (bar),); + self.cmdq + .send_command_no_wait (bar, commands::SetSystemInfo ::new (pdev, chipset))?; + self.cmdq + .send_command_no_wait (bar, commands::SetRegistry ::new ())?; - self.cmdq - .send_command_no_wait (bar, commands::SetSystemInfo ::new (pdev, chipset))?; - self.cmdq - .send_command_no_wait (bar, commands::SetRegistry ::new ())?; + hal.post_boot (&self, &ctx, &gsp_fw)?; - hal.post_boot (&self, &ctx, &gsp_fw)?; +// Wait until GSP is fully initialized. + commands::wait_gsp_init_done (&self.cmdq) + })(); -// Wait until GSP is fully initialized. - commands::wait_gsp_init_done (&self.cmdq)?; - -Ok (unload_guard.dismiss ()) +match res { +Err (e) =>Err (e), +Ok (()) =>Ok (unload_guard.dismiss ()), + } }/// Shut down the GSP and wait until it is offline.
drivers/gpu/nova-core/gsp/hal/gh100.rs +21 -17
@@ -158,29 +158,33 @@ fn boot<'a>(let gsp_falcon = ctx.gsp_falcon;let sec2_falcon = ctx.sec2_falcon; -let unload_bundle =crate ::gsp::UnloadBundle ( -KBox ::new (FspUnloadBundle ,GFP_KERNEL )?as KBox <dyn UnloadBundle > - ); +let res = (|| { +let unload_bundle =crate ::gsp::UnloadBundle ( +KBox ::new (FspUnloadBundle ,GFP_KERNEL )?as KBox <dyn UnloadBundle > + ); -// Wrap the unload bundle into a drop guard so it is automatically run upon failure. -let unload_guard = -BootUnloadGuard ::new (gsp, dev, bar, gsp_falcon, sec2_falcon,Some (unload_bundle)); +// Wrap the unload bundle into a drop guard so it is automatically run upon failure. +let unload_guard = +BootUnloadGuard ::new (gsp, dev, bar, gsp_falcon, sec2_falcon,Some (unload_bundle)); -let mut fsp =Fsp ::wait_secure_boot (dev, bar, chipset)?; +let mut fsp =Fsp ::wait_secure_boot (dev, bar, chipset)?; -let args =FmcBootArgs ::new ( - dev, - chipset, - wpr_meta.dma_handle (), - gsp.libos.dma_handle (), -false , - )?; +let args =FmcBootArgs ::new ( + dev, + chipset, + wpr_meta.dma_handle (), + gsp.libos.dma_handle (), +false , + )?; - fsp.boot_fmc (dev, bar, fb_layout, &args)?; + fsp.boot_fmc (dev, bar, fb_layout, &args)?; -wait_for_gsp_lockdown_release (dev, bar, gsp_falcon, args.boot_params_dma_handle ())?; +wait_for_gsp_lockdown_release (dev, bar, gsp_falcon, args.boot_params_dma_handle ())?; -Ok (unload_guard) +Ok (unload_guard) + })(); + + res } }
drivers/gpu/nova-core/gsp/hal/tu102.rs +50 -46
109 lines not on this page · raw message
Signature · 1 line
2.54.0
Re: [PATCH v2 03/13] gpu: nova-core: gsp: move boot code into local closure
Quoted 8 lines · Alexandre Courbot
The next patch aims at replacing the cumbersome
BootUnloadGuardwith a more local and less intrusive mechanism to run the GSP unload sequence upon GSP boot failure. Doing so requires running the boot code in a local closure, which changes its indentation and would make other changes difficult to track in the diff. Thus, this preparatory patch moves said boot code into a local closure that is run upon construction, so the next patch does not need to re-indent code that changes.
Quoted 5 lines
This is a mechanical preparatory patch to make the next patch easier to read. No functional change intended.
Signed-off-by Alexandre Courbot <[email protected]>---
See my response at [1].
Re: [PATCH v2 03/13] gpu: nova-core: gsp: move boot code into local closure
Quoted 12 lines · Eliot Courtney
Quoted 11 lines · Alexandre Courbot
The next patch aims at replacing the cumbersome
BootUnloadGuardwith a more local and less intrusive mechanism to run the GSP unload sequence upon GSP boot failure. Doing so requires running the boot code in a local closure, which changes its indentation and would make other changes difficult to track in the diff. Thus, this preparatory patch moves said boot code into a local closure that is run upon construction, so the next patch does not need to re-indent code that changes.This is a mechanical preparatory patch to make the next patch easier to read. No functional change intended.
Quoted 6 lines
Quoted 2 lines
Signed-off-by Alexandre Courbot <[email protected]>---
See my response at [1].
I'm relieved to see us going with RAII scope guards instead of a pile of closures. But now I'm tempted to wait for a v3 that does that, before reviewing, on the assumption that the rest of the series changes quite a bit as a result.
thanks,
Signature · 1 line
John Hubbard
Alexandre Courbot
maintainerPATCH v2 4/13 gpu: nova-core: gsp: replace BootUnloadGuard with local handler
In reply to Alexandre CourbotWhen adding the GSP unload capability, we introduced new types to support what is effectively an ad-hoc behavior: that Gsp::unload must be run if any error occurs during the boot sequence.
Furthermore, BootUnloadGuard is problematic because it holds additional references to the boot context, notably the Falcons. These extra references stand in the way of making some of the Falcon's methods mutable, since those methods would require exclusive access. As this behavior is only needed in one place, introducing dedicated types for it is distracting and unnecessary.
Thus, take advantage of the local closures introduced in the preceding patch to run the unload sequence in Gsp::boot if an error occurred at any step of the boot process.
This slightly broadens the failure cleanup path, as Gsp::boot now attempts a best-effort Gsp::unload even when the failure happened before the unload bundle was prepared.
--- drivers/gpu/nova-core/gsp/boot.rs | 81 ++++++---------------------------- drivers/gpu/nova-core/gsp/hal.rs | 20 ++++++--- drivers/gpu/nova-core/gsp/hal/gh100.rs | 28 +++++------- drivers/gpu/nova-core/gsp/hal/tu102.rs | 23 ++++------ 4 files changed, 47 insertions(+), 105 deletions(-)
drivers/gpu/nova-core/gsp/boot.rs +14 -67
@@ -7,8 +7,7 @@ dma::Coherent, io::poll::read_poll_timeout, prelude::*, - time::Delta, - types::ScopeGuard, // + time::Delta, // };use crate ::{
@@ -30,66 +29,6 @@ }, }; -/// Arguments required to call [`Gsp::unload`](super::Gsp::unload). -/// -/// Stored as their own type to avoid repeating a long and tedious list in [`BootUnloadGuard`]. -pub (super )struct BootUnloadArgs <'a> { - gsp: &'asuper ::Gsp , - dev: &'a device::Device <device::Bound >, - bar:Bar0 <'a>, - gsp_falcon: &'aFalcon <Gsp >, - sec2_falcon: &'aFalcon <Sec2 >, - unload_bundle:Option <super ::UnloadBundle >, -} - -/// Guard that calls [`Gsp::unload`](super::Gsp::unload) with a -/// [`UnloadBundle`](super::UnloadBundle) when dropped. -/// -/// Used to ensure the `UnloadBundle` is run during failure paths. -pub (super )struct BootUnloadGuard <'a> { - guard:ScopeGuard <BootUnloadArgs <'a>,fn (BootUnloadArgs <'a>)>, -} - -impl <'a>BootUnloadGuard <'a> { -/// Wraps `unload_bundle` into a guard that executes it when dropped. -pub (super )fn new ( - gsp: &'asuper ::Gsp , - dev: &'a device::Device <device::Bound >, - bar:Bar0 <'a>, - gsp_falcon: &'aFalcon <Gsp >, - sec2_falcon: &'aFalcon <Sec2 >, - unload_bundle:Option <super ::UnloadBundle >, - ) ->Self { -Self { - guard:ScopeGuard ::new_with_data ( -BootUnloadArgs { - gsp, - dev, - bar, - gsp_falcon, - sec2_falcon, - unload_bundle, - }, - |args| { -let _ =super ::Gsp ::unload ( - args.gsp, - args.dev, - args.bar, - args.gsp_falcon, - args.sec2_falcon, - args.unload_bundle, - ); - }, - ), - } - } - -/// Disarms the guard and returns the [`UnloadBundle`](super::UnloadBundle) it contains. -pub (super )fn dismiss (self) ->Option <super ::UnloadBundle > { - self.guard.dismiss ().unload_bundle - } -} -impl super ::Gsp {/// Attempt to boot the GSP. ///
@@ -118,10 +57,11 @@ pub(crate) fn boot(let wpr_meta =Coherent ::init (dev,GFP_KERNEL ,GspFwWprMeta ::new (&gsp_fw, &fb_layout))?;// Perform the chipset-specific boot sequence, and retrieve the unload bundle. -let unload_guard = hal.boot (&self, &ctx, &fb_layout, &wpr_meta)?; +let (res, unload_bundle) = hal.boot (&self, &ctx, &fb_layout, &wpr_meta); +// Run from a closure so we can retrieve the result, and run the unload sequence of the GSP // in case of error. - let res = (|| { + let res = res.and_then(|()| { gsp_falcon.write_os_version(bar, gsp_fw.bootloader.app_version); // Poll for RISC-V to become active before continuing.
@@ -143,11 +83,18 @@ pub(crate) fn boot(// Wait until GSP is fully initialized. commands::wait_gsp_init_done (&self.cmdq) - })(); + });match res { -Err (e) =>Err (e), -Ok (()) =>Ok (unload_guard.dismiss ()), +Err (e) => { +dev_err! (dev,"GSP boot failed with error {:?}\n" , e); + +// Ignore errors during unload; we will return the error that happened during boot. +let _ = self.unload (dev, bar, gsp_falcon, ctx.sec2_falcon, unload_bundle); + +Err (e) + } +Ok (()) =>Ok (unload_bundle), } }
drivers/gpu/nova-core/gsp/hal.rs +13 -7
@@ -24,7 +24,6 @@ Chipset, // }, gsp::{ - boot::BootUnloadGuard,Gsp ,GspBootContext ,GspFwWprMeta ,//
@@ -51,15 +50,22 @@ fn run(pub (super )trait GspHal :Send {/// Performs the GSP boot process, loading and running the required firmwares as needed. /// -/// Upon success, returns a guard that runs the GSP unload sequence if GSP boot does not -/// complete. -fn boot <'a>( +/// Returns two things: +/// +/// - The `Result` of the boot process itself, +/// - The `UnloadBundle` to use with [`Gsp::unload`], or `None` if the bundle could not be +/// created. +/// +/// Note that the two returned values are independent: it is possible for the boot process to +/// succeed while the unload bundle couldn't be created. In this case, the GSP won't be able to +/// unload properly and a full GPU reset is required before the GSP can be booted again. +fn boot ( &self, - gsp: &'aGsp , - ctx: &GspBootContext <'a>, + gsp: &Gsp , + ctx: &GspBootContext <'_>, fb_layout: &FbLayout , wpr_meta: &Coherent <GspFwWprMeta >, - ) ->Result <BootUnloadGuard <'a>>; + ) -> (Result ,Option <crate ::gsp::UnloadBundle >);/// Performs HAL-specific post-GSP boot tasks. ///
drivers/gpu/nova-core/gsp/hal/gh100.rs +11 -17
@@ -23,7 +23,6 @@ Fsp, // }, gsp::{ - boot::BootUnloadGuard, hal::{GspHal ,UnloadBundle ,//
@@ -145,27 +144,24 @@ impl GspHal for Gh100 {/// /// This path uses FSP to establish a chain of trust and boot GSP-FMC. FSP handles /// the GSP boot internally - no manual GSP reset/boot is needed. -fn boot <'a>( +fn boot ( &self, - gsp: &'aGsp , - ctx: &GspBootContext <'a>, + gsp: &Gsp , + ctx: &GspBootContext <'_>, fb_layout: &FbLayout , wpr_meta: &Coherent <GspFwWprMeta >, - ) ->Result <BootUnloadGuard <'a>> { + ) -> (Result ,Option <crate ::gsp::UnloadBundle >) {let dev = ctx.dev ();let bar = ctx.bar;let chipset = ctx.chipset;let gsp_falcon = ctx.gsp_falcon; -let sec2_falcon = ctx.sec2_falcon; + +let mut unload_bundle =None ; let res = (|| { - let unload_bundle = crate::gsp::UnloadBundle( - KBox::new(FspUnloadBundle, GFP_KERNEL)? as KBox<dyn UnloadBundle> - ); - - // Wrap the unload bundle into a drop guard so it is automatically run upon failure. - let unload_guard = - BootUnloadGuard::new(gsp, dev, bar, gsp_falcon, sec2_falcon, Some(unload_bundle)); + unload_bundle = Some(crate::gsp::UnloadBundle( + KBox::new(FspUnloadBundle, GFP_KERNEL)? as KBox<dyn UnloadBundle>, + )); let mut fsp = Fsp::wait_secure_boot(dev, bar, chipset)?;
@@ -179,12 +175,10 @@ fn boot<'a>( fsp.boot_fmc (dev, bar, fb_layout, &args)?; -wait_for_gsp_lockdown_release (dev, bar, gsp_falcon, args.boot_params_dma_handle ())?; - -Ok (unload_guard) +wait_for_gsp_lockdown_release (dev, bar, gsp_falcon, args.boot_params_dma_handle ()) })(); - res + (res, unload_bundle) } }
drivers/gpu/nova-core/gsp/hal/tu102.rs +9 -14
@@ -32,7 +32,6 @@ }, gpu::Chipset, gsp::{ - boot::BootUnloadGuard, hal::{GspHal ,UnloadBundle ,//
@@ -264,19 +263,21 @@ fn run_fwsec_frts(struct Tu102 ;impl GspHal for Tu102 { -fn boot <'a>( +fn boot ( &self, - gsp: &'aGsp , - ctx: &GspBootContext <'a>, + gsp: &Gsp , + ctx: &GspBootContext <'_>, fb_layout: &FbLayout , wpr_meta: &Coherent <GspFwWprMeta >, - ) ->Result <BootUnloadGuard <'a>> { + ) -> (Result ,Option <crate ::gsp::UnloadBundle >) {let dev = ctx.dev ();let bar = ctx.bar;let chipset = ctx.chipset;let gsp_falcon = ctx.gsp_falcon;let sec2_falcon = ctx.sec2_falcon; +let mut unload_bundle =None ; + let res = (|| { let bios = Vbios::new(dev, bar)?;
@@ -284,7 +285,7 @@ fn boot<'a>(// // If the unload bundle creation fails, the GPU will need to be reset before the driver // can be probed again. -let unload_bundle = + unload_bundle =Sec2UnloadBundle ::build (dev, bar, chipset, &bios, gsp_falcon, sec2_falcon) .inspect_err (|e| {dev_warn! (dev,"Failed to prepare unload firmware: {:?}\n" , e);
@@ -297,10 +298,6 @@ fn boot<'a>( .ok() .map (crate ::gsp::UnloadBundle ); -// Wrap the unload bundle into a drop guard so it is automatically run upon failure. -let unload_guard = -BootUnloadGuard ::new (gsp, dev, bar, gsp_falcon, sec2_falcon, unload_bundle); -// FWSEC-FRTS is not executed on chips where the FRTS region size is 0 (e.g. GA100). if !fb_layout.frts.is_empty () {run_fwsec_frts (dev, chipset, gsp_falcon, bar, &bios, fb_layout)?;
@@ -328,12 +325,10 @@ fn boot<'a>( sec2_falcon, bar, )? - .run (dev, bar, sec2_falcon, wpr_meta)?; - -Ok (unload_guard) + .run (dev, bar, sec2_falcon, wpr_meta) })(); - res + (res, unload_bundle) }fn post_boot (&self, gsp: &Gsp , ctx: &GspBootContext <'_>, gsp_fw: &GspFirmware ) ->Result {
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v2 5/13 gpu: nova-core: gsp: move unload bundle error handling to Gsp::boot
In reply to Alexandre CourbotThe warning emitted when the unload bundle cannot be constructed is valid regardless of the boot method, but it was local to Tu102. Move it to Gsp::boot so it applies to all boot methods.
This requires the HAL's boot method to return the Result for the unload bundle's creation, so Gsp::boot can display the warning.
--- drivers/gpu/nova-core/gsp/boot.rs | 12 ++++++++++++ drivers/gpu/nova-core/gsp/hal.rs | 4 ++-- drivers/gpu/nova-core/gsp/hal/gh100.rs | 6 +++--- drivers/gpu/nova-core/gsp/hal/tu102.rs | 13 ++----------- 4 files changed, 19 insertions(+), 16 deletions(-)
drivers/gpu/nova-core/gsp/boot.rs +12 -0
@@ -59,6 +59,18 @@ pub(crate) fn boot(// Perform the chipset-specific boot sequence, and retrieve the unload bundle. let (res, unload_bundle) = hal.boot (&self, &ctx, &fb_layout, &wpr_meta); +// Display error for unload bundle if any, and convert to `Option`. +let unload_bundle = unload_bundle + .inspect_err (|e| { +dev_warn! (dev,"Failed to prepare unload firmware: {:?}\n" , e); +dev_warn! (dev,"The GSP won't be able to unload properly on unbind.\n" ); +dev_warn! ( + dev, +"The GPU will need to be reset before the driver can bind again.\n" + ); + }) + .ok (); +// Run from a closure so we can retrieve the result, and run the unload sequence of the GSP // in case of error. let res = res.and_then(|()| {
drivers/gpu/nova-core/gsp/hal.rs +2 -2
@@ -53,7 +53,7 @@ pub(super) trait GspHal: Send {/// Returns two things: /// /// - The `Result` of the boot process itself, -/// - The `UnloadBundle` to use with [`Gsp::unload`], or `None` if the bundle could not be +/// - The `UnloadBundle` to use with [`Gsp::unload`], or `Err` if the bundle could not be /// created. /// /// Note that the two returned values are independent: it is possible for the boot process to
@@ -65,7 +65,7 @@ fn boot( ctx: &GspBootContext<'_>, fb_layout: &FbLayout, wpr_meta: &Coherent<GspFwWprMeta>, - ) -> (Result, Option<crate::gsp::UnloadBundle>); + ) -> (Result, Result<crate::gsp::UnloadBundle>);/// Performs HAL-specific post-GSP boot tasks. ///
drivers/gpu/nova-core/gsp/hal/gh100.rs +3 -3
@@ -150,16 +150,16 @@ fn boot( ctx: &GspBootContext<'_>, fb_layout: &FbLayout, wpr_meta: &Coherent<GspFwWprMeta>, - ) -> (Result, Option<crate::gsp::UnloadBundle>) { + ) -> (Result, Result<crate::gsp::UnloadBundle>) { let dev = ctx.dev(); let bar = ctx.bar; let chipset = ctx.chipset; let gsp_falcon = ctx.gsp_falcon; - let mut unload_bundle = None; + let mut unload_bundle = Err(EAGAIN); let res = (|| { - unload_bundle = Some(crate::gsp::UnloadBundle( + unload_bundle = Ok(crate::gsp::UnloadBundle( KBox::new(FspUnloadBundle, GFP_KERNEL)? as KBox<dyn UnloadBundle>, ));
drivers/gpu/nova-core/gsp/hal/tu102.rs +2 -11
@@ -269,14 +269,14 @@ fn boot( ctx: &GspBootContext<'_>, fb_layout: &FbLayout, wpr_meta: &Coherent<GspFwWprMeta>, - ) -> (Result, Option<crate::gsp::UnloadBundle>) { + ) -> (Result, Result<crate::gsp::UnloadBundle>) { let dev = ctx.dev(); let bar = ctx.bar; let chipset = ctx.chipset; let gsp_falcon = ctx.gsp_falcon; let sec2_falcon = ctx.sec2_falcon; - let mut unload_bundle = None; + let mut unload_bundle = Err(EAGAIN); let res = (|| { let bios = Vbios::new(dev, bar)?;
@@ -287,15 +287,6 @@ fn boot(// can be probed again. unload_bundle =Sec2UnloadBundle ::build (dev, bar, chipset, &bios, gsp_falcon, sec2_falcon) - .inspect_err (|e| { -dev_warn! (dev,"Failed to prepare unload firmware: {:?}\n" , e); -dev_warn! (dev,"The GSP won't be able to unload properly on unbind.\n" ); -dev_warn! ( - dev, -"The GPU will need to be reset before the driver can bind again.\n" - ); - }) - .ok () .map (crate ::gsp::UnloadBundle );// FWSEC-FRTS is not executed on chips where the FRTS region size is 0 (e.g. GA100).
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 05/13] gpu: nova-core: gsp: move unload bundle error handling to Gsp::boot
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Medium] Early boot failures unconditionally trigger false warnings about unload firmware preparation and…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Medium] Early boot failures unconditionally trigger false warnings about unload firmware preparation and incorrectly state a GPU reset is required. --
commit 273d5dbf9e1e0ce85069f0026ccecdadd0a2d4fd
Author: Alexandre Courbot <[email protected]>
gpu: nova-core: gsp: move unload bundle error handling to Gsp::boot
This commit moves the warning for unload bundle creation failure from the Tu102 specific implementation to Gsp::boot so it applies to all boot methods. It changes the HAL boot method to return a Result for the unload bundle.
Quoted 3 lines
drivers/gpu/nova-core/gsp/boot.rs +0
-0
[ ... ]
drivers/gpu/nova-core/gsp/boot.rs pub(crate) fn boot( · line 64
Quoted 7 lines
drivers/gpu/nova-core/gsp/boot.rs +3
-0@@ -59,6 +59,18 @@ pub(crate) fn boot(// Perform the chipset-specific boot sequence, and retrieve the unload bundle. let (res, unload_bundle) = hal.boot (&self, &ctx, &fb_layout, &wpr_meta); +// Display error for unload bundle if any, and convert to Option. +let unload_bundle = unload_bundle + .inspect_err(|e| {
[Severity: Medium]
If the hal.boot() implementation fails early (for example, if Vbios::new() fails in Tu102::boot()), unload_bundle will retain its initial dummy error value of Err(EAGAIN).
Won't inspect_err() then unconditionally print these warnings about firmware preparation and the need for a GPU reset, even though the actual failure occurred before bundle preparation was attempted?
Quoted 12 lines
drivers/gpu/nova-core/gsp/boot.rs +9
-0+dev_warn! (dev,"Failed to prepare unload firmware: {:?}\n" , e); +dev_warn! (dev,"The GSP won't be able to unload properly on unbind.\n" ); +dev_warn! ( + dev, +"The GPU will need to be reset before the driver can bind again.\n" + ); + }) + .ok (); +// Run from a closure so we can retrieve the result, and run the unload sequence of the GSP // in case of error. let res = res.and_then(|()| {
[ ... ]
Quoted 3 lines
drivers/gpu/nova-core/gsp/hal/tu102.rs +0
-0
[ ... ]
Quoted 7 lines
drivers/gpu/nova-core/gsp/hal/tu102.rs +1
-1@@ -269,14 +269,14 @@ fn boot( ctx: &GspBootContext<'_>, fb_layout: &FbLayout, wpr_meta: &Coherent<GspFwWprMeta>, - ) -> (Result, Option<crate::gsp::UnloadBundle>) { + ) -> (Result, Result<crate::gsp::UnloadBundle>) {let dev = ctx.dev ();
drivers/gpu/nova-core/gsp/hal/tu102.rs fn boot( · line 282
Quoted 10 lines
drivers/gpu/nova-core/gsp/hal/tu102.rs +1
-1let bar = ctx.bar;let chipset = ctx.chipset;let gsp_falcon = ctx.gsp_falcon;let sec2_falcon = ctx.sec2_falcon; -let mut unload_bundle =None ; +let mut unload_bundle =Err (EAGAIN ); let res = (|| { let bios = Vbios::new(dev, bar)?;
[Severity: Medium]
Here is where unload_bundle defaults to Err(EAGAIN). Is there a way to differentiate between an early boot failure where bundle creation wasn't reached, versus an actual failure during bundle creation, so that Gsp::boot() doesn't emit false warnings?
Quoted 1 line
drivers/gpu/nova-core/gsp/hal/tu102.rs +0
-0@@ -287,15 +287,6 @@ fn boot(
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=5
Alexandre Courbot
maintainerPATCH v2 6/13 gpu: nova-core: gsp: make unload take GspBootContext
In reply to Alexandre CourbotGspBootContext contains the resources required to boot the GSP. As it turns out, this is also the context required for unloading it.
Reflect that fact by replacing the arguments of Gsp::unload with the GspBootContext. This symmetry between Gsp::boot and Gsp::unload will also be convenient when we want to make these methods generic over the boot context corresponding to the boot method used.
--- drivers/gpu/nova-core/gpu.rs | 19 ++++++++++++++++--- drivers/gpu/nova-core/gsp/boot.rs | 17 +++++++---------- drivers/gpu/nova-core/gsp/hal.rs | 15 +-------------- drivers/gpu/nova-core/gsp/hal/gh100.rs | 13 +++---------- drivers/gpu/nova-core/gsp/hal/tu102.rs | 20 +++++++++----------- 5 files changed, 36 insertions(+), 48 deletions(-)
drivers/gpu/nova-core/gpu.rs +16 -3
@@ -268,7 +268,9 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {#[pin_data( PinnedDrop )] struct GspResources <'gpu> {/// Device owning the GPU. - device: &'gpu device::Device <device::Bound >, + device: &'gpu pci::Device <device::Bound >, +/// Details about the chipset. + spec:Spec ,/// MMIO mapping of PCI BAR 0. bar:Bar0 <'gpu>,/// GSP falcon instance, used for GSP boot up and cleanup.
@@ -311,7 +313,16 @@ fn drop(self: Pin<&mut Self>) { .gsp .as_ref () .get_ref () - .unload (device, bar, &*this.gsp_falcon, &*this.sec2_falcon, bundle) + .unload ( +GspBootContext { + pdev: device, + bar, + chipset: this.spec.chipset, + gsp_falcon: &*this.gsp_falcon, + sec2_falcon: &*this.sec2_falcon, + }, + bundle, + ) .inspect_err (|e|dev_err! (device,"failed to unload GSP: {:?}\n" , e)); } }
@@ -343,7 +354,9 @@ pub(crate) fn new( sysmem_flush: SysmemFlush::register(pdev.as_ref(), bar, spec.chipset)?, gsp_resources <-try_pin_init! (GspResources { - device: pdev.as_ref (), + device: pdev, + + spec: *spec, bar,
drivers/gpu/nova-core/gsp/boot.rs +7 -10
@@ -3,7 +3,6 @@use kernel::{ bits, - device, dma::Coherent , io::poll::read_poll_timeout, prelude::*,
@@ -14,7 +13,6 @@ driver::Bar0, falcon::{ gsp::Gsp, - sec2::Sec2, Falcon, // }, fb::FbLayout,
@@ -102,7 +100,7 @@ pub(crate) fn boot(dev_err! (dev,"GSP boot failed with error {:?}\n" , e);// Ignore errors during unload; we will return the error that happened during boot. -let _ = self.unload (dev, bar, gsp_falcon, ctx.sec2_falcon, unload_bundle); +let _ = self.unload (ctx, unload_bundle);Err (e) }
@@ -136,17 +134,16 @@ fn shutdown_gsp(/// This stops all activity on the GSP. pub (crate )fn unload ( &self, - dev: &device::Device <device::Bound >, - bar:Bar0 <'_>, - gsp_falcon: &Falcon <Gsp >, - sec2_falcon: &Falcon <Sec2 >, + ctx:super ::GspBootContext <'_>, unload_bundle:Option <super ::UnloadBundle >, ) ->Result { +let dev = ctx.dev (); +// Shut down the GSP. Keep going even in case of error. let mut res =Self ::shutdown_gsp ( &self.cmdq, - bar, - gsp_falcon, + ctx.bar, + ctx.gsp_falcon, commands::PowerStateLevel ::Level0 , ) .inspect_err (|e|dev_err! (dev,"GSP shutdown failed: {:?}\n" , e));
@@ -156,7 +153,7 @@ pub(crate) fn unload( res = res.and ( unload_bundle .0 - .run (dev, bar, gsp_falcon, sec2_falcon) + .run (&ctx) .inspect_err (|e|dev_err! (dev,"Unload bundle failed: {:?}\n" , e)), ); }else {
drivers/gpu/nova-core/gsp/hal.rs +1 -14
@@ -5,18 +5,11 @@mod tu102;use kernel::{ - device, dma::Coherent , prelude::*,// };use crate ::{ - driver::Bar0 , - falcon::{ - gsp::Gsp as GspEngine , - sec2::Sec2 , -Falcon ,// - }, fb::FbLayout , firmware::gsp::GspFirmware , gpu::{
@@ -37,13 +30,7 @@/// required for unloading is prepared at load time, and stored here until it needs to be run. pub (super )trait UnloadBundle :Send {/// Performs the steps required to properly reset the GSP after it has been stopped. -fn run ( - &self, - dev: &device::Device <device::Bound >, - bar:Bar0 <'_>, - gsp_falcon: &Falcon <GspEngine >, - sec2_falcon: &Falcon <Sec2 >, - ) ->Result ; +fn run (&self, ctx: &GspBootContext <'_>) ->Result ; }/// Trait implemented by GSP HALs.
drivers/gpu/nova-core/gsp/hal/gh100.rs +3 -10
@@ -14,7 +14,6 @@ driver::Bar0, falcon::{ gsp::Gsp as GspEngine, - sec2::Sec2, Falcon, // }, fb::FbLayout,
@@ -118,22 +117,16 @@ fn wait_for_gsp_lockdown_release(struct FspUnloadBundle ;impl UnloadBundle for FspUnloadBundle { -fn run ( - &self, - dev: &device::Device <device::Bound >, - bar:Bar0 <'_>, - gsp_falcon: &Falcon <GspEngine >, - _sec2_falcon: &Falcon <Sec2 >, - ) ->Result { +fn run (&self, ctx: &GspBootContext <'_>) ->Result {// GSP falcon does most of the work of resetting, so just wait for it to finish. read_poll_timeout ( - ||Ok (gsp_falcon.is_riscv_active (bar)), + ||Ok (ctx.gsp_falcon.is_riscv_active (ctx.bar)), |&active| !active,Delta ::from_millis (10 ),Delta ::from_secs (5 ), ) .map (|_| ()) - .inspect_err (|_|dev_err! (dev,"GSP falcon failed to halt\n" )) + .inspect_err (|_|dev_err! (ctx.dev (),"GSP falcon failed to halt\n" )) } }
drivers/gpu/nova-core/gsp/hal/tu102.rs +9 -11
@@ -123,18 +123,15 @@ fn build( }impl UnloadBundle for Sec2UnloadBundle { -fn run ( - &self, - dev: &device::Device <device::Bound >, - bar:Bar0 <'_>, - gsp_falcon: &Falcon <GspEngine >, - sec2_falcon: &Falcon <Sec2 >, - ) ->Result { +fn run (&self, ctx: &GspBootContext <'_>) ->Result { +let dev = ctx.dev (); +let bar = ctx.bar; +// Run FWSEC-SB to reset the GSP falcon to its pre-libos state. // Log errors but keep going if it fails. let fwsec_sb_res = self .fwsec_sb - .run (dev, bar, gsp_falcon) + .run (dev, bar, ctx.gsp_falcon) .inspect_err (|e|dev_err! (dev,"FWSEC-SB failed to run: {:?}\n" , e));// Remove WPR2 region if set.
@@ -144,13 +141,14 @@ fn run(return Ok (()); } - sec2_falcon.reset (bar)?; - sec2_falcon.load (dev, bar, &self.booter_unloader)?; + ctx.sec2_falcon.reset (bar)?; + ctx.sec2_falcon.load (dev, bar, &self.booter_unloader)?;// Sentinel value to confirm that Booter Unloader has run. const MAILBOX_SENTINEL :u32 =0xff ;let (mbox0, _) = - sec2_falcon.boot (bar,Some (MAILBOX_SENTINEL ),Some (MAILBOX_SENTINEL ))?; + ctx.sec2_falcon + .boot (bar,Some (MAILBOX_SENTINEL ),Some (MAILBOX_SENTINEL ))?;if mbox0 !=0 {dev_err! (dev,"Booter Unloader returned error 0x{:x}\n" , mbox0);return Err (EINVAL );
Signature · 1 line
2.54.0
[email protected]
AI reviewRe: [PATCH v2 06/13] gpu: nova-core: gsp: make unload take GspBootContext
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider: - [Low] Early failures in `GspHal::boot` return a dummy `EAGAIN` error for the unload bundle, triggering…
Thank you for your contribution! Sashiko AI review found 1 potential issue(s) to consider:
- [Low] Early failures in GspHal::boot return a dummy EAGAIN error for the unload bundle, triggering misleading warnings in Gsp::boot. --
commit 59b7c0f9ed708bf2f09f8548faf948cfa1adfc96
Author: Alexandre Courbot <[email protected]>
gpu: nova-core: gsp: make unload take GspBootContext
This patch refactors Gsp::unload to accept GspBootContext instead of individual resource arguments. This reflects that the resources required to boot the GSP are also needed for unloading it.
Quoted 4 lines
drivers/gpu/nova-core/gsp/boot.rs +0
-0
[ ... ]
drivers/gpu/nova-core/gsp/boot.rs pub(crate) fn boot( · line 103
Quoted 6 lines
drivers/gpu/nova-core/gsp/boot.rs +1
-1@@ -102,7 +100,7 @@ pub(crate) fn boot(dev_err! (dev,"GSP boot failed with error {:?}\n" , e);// Ignore errors during unload; we will return the error that happened during boot. -let _ = self.unload (dev, bar, gsp_falcon, ctx.sec2_falcon, unload_bundle); +let _ = self.unload (ctx, unload_bundle);
[Severity: Low]
If an early operation in the HAL boot paths fails before unload_bundle is properly initialized (such as Vbios::new() failing in Tu102::boot(), or missing fsp in Gh100::boot()), does unload_bundle retain a dummy Err(EAGAIN) value?
If so, does this cause the inspect_err() on unload_bundle to unconditionally trigger and log misleading warnings that falsely claim the GPU will need a reset even for these early initialization failures?
Quoted 3 lines
drivers/gpu/nova-core/gsp/boot.rs +0
-0Err (e) }
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=6
Alexandre Courbot
maintainerPATCH v2 7/13 gpu: nova-core: gsp: fold TU102 unload bundle construction into HAL method
In reply to Alexandre CourbotThe construction of the unload bundle is currently a bit convoluted and could be done in one function instead of two.
Additionally, turn that function into a method of Tu102. A following patch will turn the "use FWSEC bootloader" property into a flag of the TU102 HAL itself, and making this a method will allow the code to access it instead of querying Chipset.
--- drivers/gpu/nova-core/gsp/hal/tu102.rs | 90 ++++++++++++++++------------------ 1 file changed, 42 insertions(+), 48 deletions(-)
drivers/gpu/nova-core/gsp/hal/tu102.rs +42 -48
@@ -55,23 +55,6 @@ enum FwsecUnloadFirmware { }impl FwsecUnloadFirmware { -/// Loads the FWSEC SB firmware, as well as its bootloader if `chipset` requires it. -fn new ( - dev: &device::Device <device::Bound >, - bar:Bar0 <'_>, - chipset:Chipset , - bios: &Vbios , - gsp_falcon: &Falcon <GspEngine >, - ) ->Result <Self > { -let fwsec_sb =FwsecFirmware ::new (dev, gsp_falcon, bar, bios,FwsecCommand ::Sb )?; - -Ok (if chipset.needs_fwsec_bootloader () { -Self ::WithBl (FwsecFirmwareWithBl ::new (fwsec_sb, dev, chipset)?) - }else { -Self ::WithoutBl (fwsec_sb) - }) - } -/// Runs the FWSEC SB firmware. fn run ( &self,
@@ -93,35 +76,6 @@ struct Sec2UnloadBundle { booter_unloader: BooterFirmware, } -impl Sec2UnloadBundle { -/// Load and prepare the resources required to properly reset the GSP after it has been stopped. -fn build ( - dev: &device::Device <device::Bound >, - bar:Bar0 <'_>, - chipset:Chipset , - bios: &Vbios , - gsp_falcon: &Falcon <GspEngine >, - sec2_falcon: &Falcon <Sec2 >, - ) ->Result <KBox <dyn UnloadBundle >> { -KBox ::new ( -Self { - fwsec_sb:FwsecUnloadFirmware ::new (dev, bar, chipset, bios, gsp_falcon)?, - booter_unloader:BooterFirmware ::new ( - dev, -BooterKind ::Unloader , - chipset, -FIRMWARE_VERSION , - sec2_falcon, - bar, - )?, - }, -GFP_KERNEL , - ) - .map (|b| bas KBox <dyn UnloadBundle >) - .map_err (Into ::into) - } -} -impl UnloadBundle for Sec2UnloadBundle {fn run (&self, ctx: &GspBootContext <'_>) ->Result {let dev = ctx.dev ();
@@ -260,6 +214,47 @@ fn run_fwsec_frts(struct Tu102 ; +impl Tu102 { +/// Load and prepare the resources required to properly reset the GSP after it has been stopped. +fn build_unload_bundle ( + &self, + dev: &device::Device <device::Bound >, + bar:Bar0 <'_>, + chipset:Chipset , + bios: &Vbios , + gsp_falcon: &Falcon <GspEngine >, + sec2_falcon: &Falcon <Sec2 >, + ) ->Result <crate ::gsp::UnloadBundle > { +// Load the FWSEC SB firmware, as well as its bootloader if required. +let fwsec_sb =FwsecFirmware ::new (dev, gsp_falcon, bar, bios,FwsecCommand ::Sb ).and_then ( + |fwsec_sb| { +Ok (if chipset.needs_fwsec_bootloader () { +FwsecUnloadFirmware ::WithBl (FwsecFirmwareWithBl ::new (fwsec_sb, dev, chipset)?) + }else { +FwsecUnloadFirmware ::WithoutBl (fwsec_sb) + }) + }, + )?; + +KBox ::new ( +Sec2UnloadBundle { + fwsec_sb, + booter_unloader:BooterFirmware ::new ( + dev, +BooterKind ::Unloader , + chipset, +FIRMWARE_VERSION , + sec2_falcon, + bar, + )?, + }, +GFP_KERNEL , + ) + .map (|b|crate ::gsp::UnloadBundle (b)) + .map_err (Into ::into) + } +} +impl GspHal for Tu102 {fn boot ( &self,
@@ -284,8 +279,7 @@ fn boot(// If the unload bundle creation fails, the GPU will need to be reset before the driver // can be probed again. unload_bundle = -Sec2UnloadBundle ::build (dev, bar, chipset, &bios, gsp_falcon, sec2_falcon) - .map (crate ::gsp::UnloadBundle ); + self.build_unload_bundle (dev, bar, chipset, &bios, gsp_falcon, sec2_falcon);// FWSEC-FRTS is not executed on chips where the FRTS region size is 0 (e.g. GA100). if !fb_layout.frts.is_empty () {
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v2 8/13 gpu: nova-core: gsp: turn FWSEC execution into HAL method
In reply to Alexandre CourbotTurn the run_fwsec_frts function into a method of Tu102. A following patch will turn the "use FWSEC bootloader" property into a flag of the TU102 HAL itself, and making this a method will allow the code to access it instead of querying Chipset.
--- drivers/gpu/nova-core/gsp/hal/tu102.rs | 175 +++++++++++++++++---------------- 1 file changed, 88 insertions(+), 87 deletions(-)
drivers/gpu/nova-core/gsp/hal/tu102.rs +88 -87
@@ -126,95 +126,96 @@ fn run(&self, ctx: &GspBootContext<'_>) -> Result { } } -/// Helper function to load and run the FWSEC-FRTS firmware and confirm that it has properly -/// created the WPR2 region. -fn run_fwsec_frts ( - dev: &device::Device <device::Bound >, - chipset:Chipset , - falcon: &Falcon <GspEngine >, - bar:Bar0 <'_>, - bios: &Vbios , - fb_layout: &FbLayout , -) ->Result { -// Check that the WPR2 region does not already exist - if it does, we cannot run -// FWSEC-FRTS until the GPU is reset. -if bar.read (regs::NV_PFB_PRI_MMU_WPR2_ADDR_HI ).higher_bound () !=0 { -dev_err! ( - dev, -"WPR2 region already exists - GPU needs to be reset to proceed\n" - ); -return Err (EBUSY ); - } - -// FWSEC-FRTS will create the WPR2 region. -let fwsec_frts =FwsecFirmware ::new ( - dev, - falcon, - bar, - bios, -FwsecCommand ::Frts { - frts_addr: fb_layout.frts.start, - frts_size: fb_layout.frts.len (), - }, - )?; - -if chipset.needs_fwsec_bootloader () { -let fwsec_frts_bl =FwsecFirmwareWithBl ::new (fwsec_frts, dev, chipset)?; -// Load and run the bootloader, which will load FWSEC-FRTS and run it. - fwsec_frts_bl.run (dev, falcon, bar)?; - }else { -// Load and run FWSEC-FRTS directly. - fwsec_frts.run (dev, falcon, bar)?; - } - -// SCRATCH_E contains the error code for FWSEC-FRTS. -let frts_status = bar - .read (regs::NV_PBUS_SW_SCRATCH_0E_FRTS_ERR ) - .frts_err_code (); -if frts_status !=0 { -dev_err! ( - dev, -"FWSEC-FRTS returned with error code {:#x}\n" , - frts_status - ); - -return Err (EIO ); - } - -// Check that the WPR2 region has been created as we requested. -let (wpr2_lo, wpr2_hi) = ( - bar.read (regs::NV_PFB_PRI_MMU_WPR2_ADDR_LO ).lower_bound (), - bar.read (regs::NV_PFB_PRI_MMU_WPR2_ADDR_HI ).higher_bound (), - ); - -match (wpr2_lo, wpr2_hi) { - (_,0 ) => { -dev_err! (dev,"WPR2 region not created after running FWSEC-FRTS\n" ); - -Err (EIO ) - } - (wpr2_lo, _)if wpr2_lo != fb_layout.frts.start => { -dev_err! ( - dev, -"WPR2 region created at unexpected address {:#x}; expected {:#x}\n" , - wpr2_lo, - fb_layout.frts.start, - ); - -Err (EIO ) - } - (wpr2_lo, wpr2_hi) => { -dev_dbg! (dev,"WPR2: {:#x}-{:#x}\n" , wpr2_lo, wpr2_hi); -dev_dbg! (dev,"GPU instance built\n" ); - -Ok (()) - } - } -} -struct Tu102 ;impl Tu102 { +/// Helper method to load and run the FWSEC-FRTS firmware and confirm that it has properly +/// created the WPR2 region. +fn run_fwsec_frts ( + &self, + dev: &device::Device <device::Bound >, + chipset:Chipset , + falcon: &Falcon <GspEngine >, + bar:Bar0 <'_>, + bios: &Vbios , + fb_layout: &FbLayout , + ) ->Result { +// Check that the WPR2 region does not already exist - if it does, we cannot run +// FWSEC-FRTS until the GPU is reset. +if bar.read (regs::NV_PFB_PRI_MMU_WPR2_ADDR_HI ).higher_bound () !=0 { +dev_err! ( + dev, +"WPR2 region already exists - GPU needs to be reset to proceed\n" + ); +return Err (EBUSY ); + } + +// FWSEC-FRTS will create the WPR2 region. +let fwsec_frts =FwsecFirmware ::new ( + dev, + falcon, + bar, + bios, +FwsecCommand ::Frts { + frts_addr: fb_layout.frts.start, + frts_size: fb_layout.frts.len (), + }, + )?; + +if chipset.needs_fwsec_bootloader () { +let fwsec_frts_bl =FwsecFirmwareWithBl ::new (fwsec_frts, dev, chipset)?; +// Load and run the bootloader, which will load FWSEC-FRTS and run it. + fwsec_frts_bl.run (dev, falcon, bar)?; + }else { +// Load and run FWSEC-FRTS directly. + fwsec_frts.run (dev, falcon, bar)?; + } + +// SCRATCH_E contains the error code for FWSEC-FRTS. +let frts_status = bar + .read (regs::NV_PBUS_SW_SCRATCH_0E_FRTS_ERR ) + .frts_err_code (); +if frts_status !=0 { +dev_err! ( + dev, +"FWSEC-FRTS returned with error code {:#x}\n" , + frts_status + ); + +return Err (EIO ); + } + +// Check that the WPR2 region has been created as we requested. +let (wpr2_lo, wpr2_hi) = ( + bar.read (regs::NV_PFB_PRI_MMU_WPR2_ADDR_LO ).lower_bound (), + bar.read (regs::NV_PFB_PRI_MMU_WPR2_ADDR_HI ).higher_bound (), + ); + +match (wpr2_lo, wpr2_hi) { + (_,0 ) => { +dev_err! (dev,"WPR2 region not created after running FWSEC-FRTS\n" ); + +Err (EIO ) + } + (wpr2_lo, _)if wpr2_lo != fb_layout.frts.start => { +dev_err! ( + dev, +"WPR2 region created at unexpected address {:#x}; expected {:#x}\n" , + wpr2_lo, + fb_layout.frts.start, + ); + +Err (EIO ) + } + (wpr2_lo, wpr2_hi) => { +dev_dbg! (dev,"WPR2: {:#x}-{:#x}\n" , wpr2_lo, wpr2_hi); +dev_dbg! (dev,"GPU instance built\n" ); + +Ok (()) + } + } + } +/// Load and prepare the resources required to properly reset the GSP after it has been stopped. fn build_unload_bundle ( &self,
@@ -283,7 +284,7 @@ fn boot(// FWSEC-FRTS is not executed on chips where the FRTS region size is 0 (e.g. GA100). if !fb_layout.frts.is_empty () { -run_fwsec_frts (dev, chipset, gsp_falcon, bar, &bios, fb_layout)?; + self.run_fwsec_frts (dev, chipset, gsp_falcon, bar, &bios, fb_layout)?; } gsp_falcon.reset (bar)?;
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v2 9/13 gpu: nova-core: gsp: make use of FWSEC bootloader a property of the TU102 HAL
In reply to Alexandre CourbotBy being in the TU102 HAL, we already know that the GSP boot method is the SEC2 Booter, so the only variable is whether the FWSEC bootloader is used or not. Since Chipset also includes the variants that boot FSP, querying it for that information introduces a potential code path where the chipset actually boots via FSP that the current code doesn't handle.
Turn the use of FWSEC bootloader into a property of the Tu102 HAL, and give Ampere chipsets their own instance with that property set to false. This removes the invalid code path and the only use of Chipset is now to load the correct firmware files.
This also removes some uses of the Chipset::needs_fwsec_bootloader method and prepares the ground for removing it.
--- drivers/gpu/nova-core/gsp/hal.rs | 5 ++++- drivers/gpu/nova-core/gsp/hal/ga102.rs | 14 ++++++++++++++ drivers/gpu/nova-core/gsp/hal/tu102.rs | 15 +++++++++++---- 3 files changed, 29 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/gsp/hal.rs +4 -1
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 // SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. +mod ga102;mod gh100;mod tu102;
@@ -66,7 +67,9 @@ fn post_boot(&self, _gsp: &Gsp, _ctx: &GspBootContext<'_>, _gsp_fw: &GspFirmware/// Returns the GSP HAL to be used for `chipset`. pub (super )fn gsp_hal (chipset:Chipset ) -> &'staticdyn GspHal {match chipset.arch () { -Architecture ::Turing |Architecture ::Ampere |Architecture ::Ada => tu102::TU102_HAL , +Architecture ::Turing => tu102::TU102_HAL , +Architecture ::Ampere if matches! (chipset,Chipset ::GA100 ) => tu102::TU102_HAL , +Architecture ::Ampere |Architecture ::Ada => ga102::GA102_HAL ,Architecture ::Hopper |Architecture ::BlackwellGB10x |Architecture ::BlackwellGB20x => { gh100::GH100_HAL }
drivers/gpu/nova-core/gsp/hal/ga102.rs new file +14 -0
@@ -0,0 +1,14 @@ +// SPDX-License-Identifier: GPL-2.0 +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + +use crate ::gsp::hal::{ + tu102::Tu102 , +GspHal ,// +}; + +/// The GA102 HAL is like the TU102 one, except it doesn't use the bootloader. +const GA102 :Tu102 =Tu102 { + needs_fwsec_bootloader:false , +}; + +pub (super )const GA102_HAL : &dyn GspHal = &GA102 ;
drivers/gpu/nova-core/gsp/hal/tu102.rs +11 -4
@@ -126,7 +126,10 @@ fn run(&self, ctx: &GspBootContext<'_>) -> Result { } } -struct Tu102 ; +pub (super )struct Tu102 { +/// If `true`, then the FWSEC-FRTS bootloader will be used to load the actual firmware. +pub (super ) needs_fwsec_bootloader:bool , +}impl Tu102 {/// Helper method to load and run the FWSEC-FRTS firmware and confirm that it has properly
@@ -162,7 +165,7 @@ fn run_fwsec_frts( }, )?; -if chipset.needs_fwsec_bootloader () { +if self.needs_fwsec_bootloader {let fwsec_frts_bl =FwsecFirmwareWithBl ::new (fwsec_frts, dev, chipset)?;// Load and run the bootloader, which will load FWSEC-FRTS and run it. fwsec_frts_bl.run (dev, falcon, bar)?;
@@ -229,7 +232,7 @@ fn build_unload_bundle(// Load the FWSEC SB firmware, as well as its bootloader if required. let fwsec_sb = FwsecFirmware::new(dev, gsp_falcon, bar, bios, FwsecCommand::Sb).and_then( |fwsec_sb| { - Ok(if chipset.needs_fwsec_bootloader() { + Ok(if self.needs_fwsec_bootloader { FwsecUnloadFirmware::WithBl(FwsecFirmwareWithBl::new(fwsec_sb, dev, chipset)?) } else { FwsecUnloadFirmware::WithoutBl(fwsec_sb)
@@ -327,5 +330,9 @@ fn post_boot(&self, gsp: &Gsp, ctx: &GspBootContext<'_>, gsp_fw: &GspFirmware) - } } -const TU102 :Tu102 =Tu102 ; +/// The TU102 HAL requires the use of the FWSEC bootloader. +const TU102 :Tu102 =Tu102 { + needs_fwsec_bootloader:true , +}; +pub (super )const TU102_HAL : &dyn GspHal = &TU102 ;
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v2 10/13 gpu: nova-core: introduce GspBootMethod
In reply to Alexandre CourbotThe GSP boot method is currently determined by two ad-hoc methods of Chipset: uses_fsp (a boolean telling whether to use the FSP boot path or the Sec2 Booter one) and needs_fwsec_bootloader (another boolean valid only for the Sec2 Booter method that tells whether the FWSEC bootloader must be used).
This is neither extensible nor sound: the combination uses_fsp && needs_fwsec_bootloader is invalid, but can still be expressed.
Thus, unify these two predicates into a single gsp_boot_method method that returns an enum type unambiguously describing the boot method to use. This ensures that no invalid combination can be expressed, which makes matching sounder.
--- drivers/gpu/nova-core/firmware.rs | 25 ++++++++++++-------- drivers/gpu/nova-core/firmware/fwsec.rs | 5 ++-- drivers/gpu/nova-core/gpu.rs | 16 ------------- drivers/gpu/nova-core/gsp.rs | 42 ++++++++++++++++++++++++++++++++- drivers/gpu/nova-core/gsp/hal.rs | 22 ++++++++--------- 5 files changed, 70 insertions(+), 40 deletions(-)
drivers/gpu/nova-core/firmware.rs +15 -10
@@ -21,6 +21,7 @@ FalconFirmware, // }, gpu, + gsp::GspBootMethod, num::{FromSafeCast ,IntoSafeCast ,//
@@ -432,17 +433,21 @@ const fn make_entry_chipset(self, chipset: gpu::Chipset) -> Self {// FSP-based chipsets (Hopper, Blackwell and later) boot the GSP via the FMC image loaded by // FSP. Older chipsets use the SEC2 booter instead. -let this =if chipset.uses_fsp () { - this.make_entry_file (name,"fmc" ) - }else { - this.make_entry_file (name,"booter_load" ) - .make_entry_file (name,"booter_unload" ) - }; +match chipset.gsp_boot_method () { +GspBootMethod ::Sec2 { + needs_fwsec_bootloader, + } => { +let mut this = this + .make_entry_file (name,"booter_load" ) + .make_entry_file (name,"booter_unload" ); -if chipset.needs_fwsec_bootloader () { - this.make_entry_file (name,"gen_bootloader" ) - }else { - this +if needs_fwsec_bootloader { + this = this.make_entry_file (name,"gen_bootloader" ) + } + + this + } +GspBootMethod ::Fsp => this.make_entry_file (name,"fmc" ), } }
drivers/gpu/nova-core/firmware/fwsec.rs +3 -2
@@ -388,8 +388,9 @@ pub(crate) fn new(/// Loads the FWSEC firmware into `falcon` and execute it. /// /// This must only be called on chipsets that do not need the FWSEC bootloader (i.e., where -/// [`Chipset::needs_fwsec_bootloader()`](crate::gpu::Chipset::needs_fwsec_bootloader) returns -/// `false`). On chipsets that do, use [`bootloader::FwsecFirmwareWithBl`] instead. +/// [`Chipset::gsp_boot_method()`](crate::gpu::Chipset::gsp_boot_method) returns +/// `GspBootMethod::Sec2 { needs_fwsec_bootloader: false }`). On chipsets where +/// `needs_fwsec_bootloader` is `true` do, use [`bootloader::FwsecFirmwareWithBl`] instead. pub (crate )fn run ( &self, dev: &Device<device::Bound>,
drivers/gpu/nova-core/gpu.rs +0 -16
@@ -132,22 +132,6 @@ pub(crate) const fn arch(self) -> Architecture { } } -/// Returns `true` if this chipset requires the PIO-loaded bootloader in order to boot FWSEC. -/// -/// This includes all chipsets < GA102. -pub (crate )const fn needs_fwsec_bootloader (self) ->bool { -matches! (self.arch (),Architecture ::Turing ) ||matches! (self,Self ::GA100 ) - } - -/// Returns `true` if this chipset boots via FSP (Hopper and later), which requires the FMC -/// firmware image. -pub (crate )const fn uses_fsp (self) ->bool { -matches! ( - self.arch (), -Architecture ::Hopper |Architecture ::BlackwellGB10x |Architecture ::BlackwellGB20x - ) - } -/// Returns the address range of the PCI config mirror space. pub (crate )fn pci_config_mirror_range (self) ->Range <u32 > { hal::gpu_hal (self).pci_config_mirror_range ()
drivers/gpu/nova-core/gsp.rs +41 -1
53 lines not on this page · raw message
drivers/gpu/nova-core/gsp/hal.rs +11 -11
@@ -13,13 +13,11 @@use crate ::{ fb::FbLayout , firmware::gsp::GspFirmware , - gpu::{ -Architecture , -Chipset ,// - }, + gpu::Chipset , gsp::{Gsp ,GspBootContext , +GspBootMethod ,GspFwWprMeta ,// }, };
@@ -66,12 +64,14 @@ fn post_boot(&self, _gsp: &Gsp, _ctx: &GspBootContext<'_>, _gsp_fw: &GspFirmware/// Returns the GSP HAL to be used for `chipset`. pub (super )fn gsp_hal (chipset:Chipset ) -> &'staticdyn GspHal { -match chipset.arch () { -Architecture ::Turing => tu102::TU102_HAL , -Architecture ::Ampere if matches! (chipset,Chipset ::GA100 ) => tu102::TU102_HAL , -Architecture ::Ampere |Architecture ::Ada => ga102::GA102_HAL , -Architecture ::Hopper |Architecture ::BlackwellGB10x |Architecture ::BlackwellGB20x => { - gh100::GH100_HAL - } +// The GSP HAL is entirely determined by the boot method. +match chipset.gsp_boot_method () { +GspBootMethod ::Sec2 { + needs_fwsec_bootloader:true , + } => tu102::TU102_HAL , +GspBootMethod ::Sec2 { + needs_fwsec_bootloader:false , + } => ga102::GA102_HAL , +GspBootMethod ::Fsp => gh100::GH100_HAL , } }
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v2 11/13 gpu: nova-core: avoid repeated calls to pci::Device::as_ref
In reply to Alexandre CourbotAdd a local Device reference created from the pci::Device in the Gpu constructor to avoid repeatedly calling as_ref.
--- drivers/gpu/nova-core/gpu.rs | 18 ++++++++++-------- 1 file changed, 10 insertions(+), 8 deletions(-)
drivers/gpu/nova-core/gpu.rs +10 -8
@@ -316,9 +316,11 @@ pub(crate) fn new( pdev: &'gpu pci::Device<device::Core<'_>>, bar: Bar0<'gpu>, ) -> impl PinInit<Self, Error> + 'gpu { +let dev = pdev.as_ref (); +try_pin_init! (Self { - spec:Spec ::new (pdev.as_ref (), bar).inspect (|spec| { -dev_info! (pdev,"NVIDIA ({})\n" , spec); + spec:Spec ::new (dev, bar).inspect (|spec| { +dev_info! (dev,"NVIDIA ({})\n" , spec); })?,// We must wait for GFW_BOOT completion before doing any significant setup on the GPU.
@@ -331,11 +333,11 @@ pub(crate) fn new(unsafe { pdev.dma_set_mask_and_coherent (dma_mask)? }; hal.wait_gfw_boot_completion (bar) - .inspect_err (|_|dev_err! (pdev,"GFW boot did not complete\n" ))?; + .inspect_err (|_|dev_err! (dev,"GFW boot did not complete\n" ))?; },// Initialize this early because `gsp_resources` depends on it. - sysmem_flush: SysmemFlush::register(pdev.as_ref(), bar, spec.chipset)?, + sysmem_flush: SysmemFlush::register(dev, bar, spec.chipset)?, gsp_resources <-try_pin_init! (GspResources { device: pdev,
@@ -345,12 +347,12 @@ pub(crate) fn new( bar, gsp_falcon: Falcon::new( - pdev.as_ref(), + dev, spec.chipset, ) .inspect(|falcon| falcon.clear_swgen0_intr(bar))?, - sec2_falcon: Falcon::new(pdev.as_ref(), spec.chipset)?, + sec2_falcon: Falcon::new(dev, spec.chipset)?, gsp <- Gsp::new(pdev),
@@ -370,8 +372,8 @@ pub(crate) fn new(// Obtain and display basic GPU information. let info = gsp_resources.gsp.get_static_info (bar)?;match info.gpu_name () { -Ok (name) =>dev_info! (pdev,"GPU name: {}\n" , name), -Err (e) =>dev_warn! (pdev,"GPU name unavailable: {:?}\n" , e), +Ok (name) =>dev_info! (dev,"GPU name: {}\n" , name), +Err (e) =>dev_warn! (dev,"GPU name unavailable: {:?}\n" , e), } info
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v2 12/13 gpu: nova-core: gsp: pass GspBootContext mutably
In reply to Alexandre CourbotWe want to move the Fsp instance into Gpu, which will require passing it as part of the GspBootContext as Fsp::boot_fmc is a mutable method.
Additionally, we will also want to make some methods of the Falcons mutable, which will also require passing them as mutable references.
To prepare for this, make the passed GspBootContext mutable, and pass mutable references to it to the GSP boot HAL methods.
--- drivers/gpu/nova-core/gsp/boot.rs | 6 +++--- drivers/gpu/nova-core/gsp/hal.rs | 9 +++++++-- drivers/gpu/nova-core/gsp/hal/gh100.rs | 2 +- drivers/gpu/nova-core/gsp/hal/tu102.rs | 4 ++-- 4 files changed, 13 insertions(+), 8 deletions(-)
drivers/gpu/nova-core/gsp/boot.rs +3 -3
@@ -38,7 +38,7 @@ impl super::Gsp {/// [`Self::unload`]) returned. pub (crate )fn boot ( self:Pin <&mut Self >, - ctx:super ::GspBootContext <'_>, +mut ctx:super ::GspBootContext <'_>, ) ->Result <Option <super ::UnloadBundle >> {let pdev = ctx.pdev;let bar = ctx.bar;
@@ -55,7 +55,7 @@ pub(crate) fn boot(let wpr_meta =Coherent ::init (dev,GFP_KERNEL ,GspFwWprMeta ::new (&gsp_fw, &fb_layout))?;// Perform the chipset-specific boot sequence, and retrieve the unload bundle. -let (res, unload_bundle) = hal.boot (&self, &ctx, &fb_layout, &wpr_meta); +let (res, unload_bundle) = hal.boot (&self, &mut ctx, &fb_layout, &wpr_meta);// Display error for unload bundle if any, and convert to `Option`. let unload_bundle = unload_bundle
@@ -89,7 +89,7 @@ pub(crate) fn boot( self.cmdq .send_command_no_wait (bar, commands::SetRegistry ::new ())?; - hal.post_boot (&self, &ctx, &gsp_fw)?; + hal.post_boot (&self, &mut ctx, &gsp_fw)?;// Wait until GSP is fully initialized. commands::wait_gsp_init_done (&self.cmdq)
drivers/gpu/nova-core/gsp/hal.rs +7 -2
@@ -48,7 +48,7 @@ pub(super) trait GspHal: Send {fn boot ( &self, gsp: &Gsp , - ctx: &GspBootContext <'_>, + ctx: &mut GspBootContext <'_>, fb_layout: &FbLayout , wpr_meta: &Coherent <GspFwWprMeta >, ) -> (Result ,Result <crate ::gsp::UnloadBundle >);
@@ -57,7 +57,12 @@ fn boot(/// /// This method is called by the GSP boot code after the GSP is confirmed to be running, and /// after the initialization commands have been pushed onto its queue. -fn post_boot (&self, _gsp: &Gsp , _ctx: &GspBootContext <'_>, _gsp_fw: &GspFirmware ) ->Result { +fn post_boot ( + &self, + _gsp: &Gsp , + _ctx: &mut GspBootContext <'_>, + _gsp_fw: &GspFirmware , + ) ->Result {Ok (()) } }
drivers/gpu/nova-core/gsp/hal/gh100.rs +1 -1
@@ -140,7 +140,7 @@ impl GspHal for Gh100 {fn boot ( &self, gsp: &Gsp , - ctx: &GspBootContext <'_>, + ctx: &mut GspBootContext <'_>, fb_layout: &FbLayout , wpr_meta: &Coherent <GspFwWprMeta >, ) -> (Result ,Result <crate ::gsp::UnloadBundle >) {
drivers/gpu/nova-core/gsp/hal/tu102.rs +2 -2
@@ -263,7 +263,7 @@ impl GspHal for Tu102 {fn boot ( &self, gsp: &Gsp , - ctx: &GspBootContext <'_>, + ctx: &mut GspBootContext <'_>, fb_layout: &FbLayout , wpr_meta: &Coherent <GspFwWprMeta >, ) -> (Result ,Result <crate ::gsp::UnloadBundle >) {
@@ -318,7 +318,7 @@ fn boot( (res, unload_bundle) } -fn post_boot (&self, gsp: &Gsp , ctx: &GspBootContext <'_>, gsp_fw: &GspFirmware ) ->Result { +fn post_boot (&self, gsp: &Gsp , ctx: &mut GspBootContext <'_>, gsp_fw: &GspFirmware ) ->Result {GspSequencer ::run ( &gsp.cmdq, ctx,
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v2 13/13 gpu: nova-core: store Fsp instance in Gpu
In reply to Alexandre CourbotThe Fsp instance was only used in the Hopper+ boot path, and consequently built locally (and immediately dropped) in it.
This worked well as a temporary measure, but the FSP is also needed in other parts of the driver, for instance vGPU.
Thus, create the Fsp instance in the Gpu constructor and store it there, passing it to the GSP boot as a mutable reference using GspBootContext. This makes the Fsp available even after the GSP is booted.
--- drivers/gpu/nova-core/gpu.rs | 15 ++++++++++++++- drivers/gpu/nova-core/gsp.rs | 2 ++ drivers/gpu/nova-core/gsp/hal/gh100.rs | 9 +++------ 3 files changed, 19 insertions(+), 7 deletions(-)
drivers/gpu/nova-core/gpu.rs +14 -1
@@ -21,11 +21,13 @@ Falcon, // }, fb::SysmemFlush, + fsp::Fsp, gsp::{ self, commands::GetGspStaticInfoReply, Gsp, - GspBootContext, // + GspBootContext, + GspBootMethod, // }, regs, };
@@ -261,6 +263,10 @@ struct GspResources<'gpu> { gsp_falcon: Falcon<GspFalcon>, /// SEC2 falcon instance, used for GSP boot up and cleanup. sec2_falcon: Falcon<Sec2Falcon>, + /// FSP instance, if on an arch that supports it. + // TODO: use different resource types for each boot method, and make the relevant Gsp methods + // generic against them. + fsp: Option<Fsp>,/// GSP runtime data. #[pin] gsp: Gsp,
@@ -304,6 +310,7 @@ fn drop(self: Pin<&mut Self>) { chipset: this.spec.chipset, gsp_falcon: &*this.gsp_falcon, sec2_falcon: &*this.sec2_falcon, + fsp: this.fsp.as_mut(), }, bundle, )
@@ -354,6 +361,11 @@ pub(crate) fn new( sec2_falcon: Falcon::new(dev, spec.chipset)?, + fsp: match spec.chipset.gsp_boot_method() { +GspBootMethod ::Sec2 { .. } =>None , +GspBootMethod ::Fsp =>Some (Fsp ::wait_secure_boot (dev, bar, spec.chipset)?), + }, + gsp <- Gsp::new(pdev),// This member must be initialized last, so the `UnloadBundle` can never be dropped
@@ -365,6 +377,7 @@ pub(crate) fn new( chipset: spec.chipset, gsp_falcon, sec2_falcon, + fsp: fsp.as_mut(), })?, }),
drivers/gpu/nova-core/gsp.rs +2 -0
@@ -38,6 +38,7 @@ sec2::Sec2 as Sec2Falcon, Falcon, // }, + fsp::Fsp, gpu::{Architecture ,Chipset ,//
@@ -62,6 +63,7 @@ pub(crate) struct GspBootContext<'a> { pub(crate) chipset: Chipset, pub(crate) gsp_falcon: &'a Falcon<GspFalcon>, pub(crate) sec2_falcon: &'a Falcon<Sec2Falcon>, + pub(crate) fsp: Option<&'a mut Fsp>, }impl <'a>GspBootContext <'a> {
drivers/gpu/nova-core/gsp/hal/gh100.rs +3 -6
@@ -17,10 +17,7 @@ Falcon, // }, fb::FbLayout, - fsp::{ - FmcBootArgs, - Fsp, // - }, + fsp::FmcBootArgs, gsp::{ hal::{GspHal ,
@@ -152,12 +149,12 @@ fn boot(let mut unload_bundle =Err (EAGAIN ); let res = (|| { + let fsp = ctx.fsp.as_mut().ok_or(ENODEV)?; + unload_bundle = Ok(crate::gsp::UnloadBundle( KBox::new(FspUnloadBundle, GFP_KERNEL)? as KBox<dyn UnloadBundle>, )); - let mut fsp = Fsp::wait_secure_boot(dev, bar, chipset)?; - let args = FmcBootArgs::new( dev, chipset,
Signature · 1 line
2.54.0
Re: [PATCH v2 13/13] gpu: nova-core: store Fsp instance in Gpu
Quoted 5 lines · Alexandre Courbot
The
Fspinstance was only used in the Hopper+ boot path, and consequently built locally (and immediately dropped) in it.This worked well as a temporary measure, but the FSP is also needed in other parts of the driver, for instance vGPU.
Or: "will be needed, in future patchsets", but not in this one. May I suggest that we put this patch in one of those future patchsets, instead of trying to do it here, in advance of anything using it?
thanks,
Signature · 1 line
John Hubbard
Quoted 123 lines
Thus, create the
Fspinstance in theGpuconstructor and store it there, passing it to the GSP boot as a mutable reference usingGspBootContext. This makes theFspavailable even after the GSP is booted.Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/gpu.rs | 15 ++++++++++++++- drivers/gpu/nova-core/gsp.rs | 2 ++ drivers/gpu/nova-core/gsp/hal/gh100.rs | 9 +++------ 3 files changed, 19 insertions(+), 7 deletions(-)drivers/gpu/nova-core/gpu.rs +14
-1@@ -21,11 +21,13 @@ Falcon, // }, fb::SysmemFlush, + fsp::Fsp, gsp::{ self, commands::GetGspStaticInfoReply, Gsp, - GspBootContext, // + GspBootContext, + GspBootMethod, // }, regs, };@@ -261,6 +263,10 @@ struct GspResources<'gpu> { gsp_falcon: Falcon<GspFalcon>, /// SEC2 falcon instance, used for GSP boot up and cleanup. sec2_falcon: Falcon<Sec2Falcon>, + /// FSP instance, if on an arch that supports it. + // TODO: use different resource types for each boot method, and make the relevant Gsp methods + // generic against them. + fsp: Option<Fsp>,/// GSP runtime data. #[pin] gsp: Gsp,@@ -304,6 +310,7 @@ fn drop(self: Pin<&mut Self>) { chipset: this.spec.chipset, gsp_falcon: &*this.gsp_falcon, sec2_falcon: &*this.sec2_falcon, + fsp: this.fsp.as_mut(), }, bundle, )@@ -354,6 +361,11 @@ pub(crate) fn new( sec2_falcon: Falcon::new(dev, spec.chipset)?, + fsp: match spec.chipset.gsp_boot_method() { +GspBootMethod ::Sec2 { .. } =>None , +GspBootMethod ::Fsp =>Some (Fsp ::wait_secure_boot (dev, bar, spec.chipset)?), + }, + gsp <- Gsp::new(pdev),// This member must be initialized last, so the `UnloadBundle` can never be dropped @@ -365,6 +377,7 @@ pub(crate) fn new( chipset: spec.chipset, gsp_falcon, sec2_falcon, + fsp: fsp.as_mut(), })?, }),drivers/gpu/nova-core/gsp.rs +2
-0@@ -38,6 +38,7 @@ sec2::Sec2 as Sec2Falcon, Falcon, // }, + fsp::Fsp, gpu::{Architecture ,Chipset ,// @@ -62,6 +63,7 @@ pub(crate) struct GspBootContext<'a> { pub(crate) chipset: Chipset, pub(crate) gsp_falcon: &'a Falcon<GspFalcon>, pub(crate) sec2_falcon: &'a Falcon<Sec2Falcon>, + pub(crate) fsp: Option<&'a mut Fsp>, }impl <'a>GspBootContext <'a> {drivers/gpu/nova-core/gsp/hal/gh100.rs +3
-6@@ -17,10 +17,7 @@ Falcon, // }, fb::FbLayout, - fsp::{ - FmcBootArgs, - Fsp, // - }, + fsp::FmcBootArgs, gsp::{ hal::{GspHal ,@@ -152,12 +149,12 @@ fn boot(let mut unload_bundle =Err (EAGAIN ); let res = (|| { + let fsp = ctx.fsp.as_mut().ok_or(ENODEV)?; + unload_bundle = Ok(crate::gsp::UnloadBundle( KBox::new(FspUnloadBundle, GFP_KERNEL)? as KBox<dyn UnloadBundle>, )); - let mut fsp = Fsp::wait_secure_boot(dev, bar, chipset)?; - let args = FmcBootArgs::new( dev, chipset,
Alexandre Courbot
maintainerRe: [PATCH v2 13/13] gpu: nova-core: store Fsp instance in Gpu
Quoted 11 lines · John Hubbard
Quoted 5 lines · Alexandre Courbot
The
Fspinstance was only used in the Hopper+ boot path, and consequently built locally (and immediately dropped) in it.This worked well as a temporary measure, but the FSP is also needed in other parts of the driver, for instance vGPU.
Or: "will be needed, in future patchsets", but not in this one. May I suggest that we put this patch in one of those future patchsets, instead of trying to do it here, in advance of anything using it?
So we have two patchsets (from Zhi and Eliot) that eventually depend on this, and I am not sure which one would take this patch. :)
Besides, I'd argue that this change also stands on its own; FSP is a sub-device of the GPU, we ephemerally instanciated it in the GH100 HAL because that's where we needed it, but from an architectural perspective its scope should be larger.