gpu: nova-core: run unload sequence upon unbinding
Subsystems (5)
Alexandre Courbot
maintainerv6 cover gpu: nova-core: run unload sequence upon unbinding
Currently the GSP is left running and the WPR2 memory region untouched when the driver is unbound. This is obviously not ideal for at least two reasons:
- Probing requires setting up the WPR2 region, which cannot be done if there is already one in place. Hence the current requirement to reset the GPU (using e.g. echo 1 >/sys/bus/pci/devices/.../reset) before the driver can be probed again after removal.
- The running GSP may still attempt to access shared memory regions which the kernel might recycle.
On top of that, there is a nasty bug in the Blackwell VBIOS that sometimes borks the GPU upon PCI reset, requiring a reboot. So relying on the PCI reset to unload/reload Nova is really not practical here.
This series does what is needed to leave the GPU in a clean state after unbind, for all currently supported GPUs. Blackwell support is basic and will be added alongside the Blackwell series if this can be merged first.
This revision rebases on top of the Device HRT series [1] and moves the unload bundle from Gsp into NovaCore. This makes GSP unload initiation only possible at the driver module level, which is the only place that can consume the unload bundle.
A branch with the series and its required dependencies is available at [2].
[1] https://lore.kernel.org/[email protected]
[2] https://github.com/Gnurou/linux/tree/b4/nova-unload
---
Changes in v6:
- Inline TU102 local run_booter method in its unique call site.
- Rename unload bundle field to unload_bundle.
- Make Sec2UnloadBundle private.
- Continue GSP teardown upon partial failure.
- Store the unload bundle into NovaCore.
- Take the unload bundle by value to make it one-shot.
- Link to v5: https://patch.msgid.link/[email protected]
Changes in v5:
- Rebase on top of the Device HRT series.
- Drop the now unneeded "gpu: nova-core: split BAR acquisition in unbind()".
- Link to v4: https://patch.msgid.link/[email protected]
Changes in v4:
- Remove warn_on_err macro as it isn't performing as expected and distracts from the goal of the series.
- Add John's patch from the Blackwell series refactoring the Booter
Loader runner code.
- Add a GSP HAL and move the existing TU102/SEC2 boot sequence into it
in preparation for the Hopper/Blackwell FSP boot path.
- Prepare the firmware required for unloading at probe time and save it into an unload bundle, as we cannot guarantee filesystem access at unload time.
- Constrain UNLOADING_GUEST_DRIVER's visibility to the parent module.
- Also write the sentinel value 0xff into mbox1 when running Booter Unloader to align with OpenRM.
- Link to v3: https://patch.msgid.link/[email protected]
Changes in v3:
- Disambiguate doccomment for warn_on_err.
- Test the correct bit instead of the whole register value to determine that the GSP has stopped.
- Use an enum instead of a boolean to encode the power level when
shutting down the GSP.
- Add missing newline to dev_err.
- Add missing doccomments for new types.
- Use values from bindings instead of magic numbers.
- Remove the redundant get_gsp_info function.
- Better document Booter Unloader mailbox sentinel value, and check the value of mbox0 upon return.
- Link to v2: https://patch.msgid.link/[email protected]
Changes in v2:
- Rebase on top of master and remove unneeded/obsolete preparatory patches.
- Tidy up the imports of commands from the fw module in the gsp module.
- Link to v1: https://patch.msgid.link/[email protected]
---
Alexandre Courbot (6):
gpu: nova-core: remove unneeded get_gsp_info proxy function
gpu: nova-core: do not import firmware commands into GSP command module
gpu: nova-core: send UNLOADING_GUEST_DRIVER GSP command upon unloading
gpu: nova-core: gsp: shuffle boot code a bit to keep chipset-specific parts close
gpu: nova-core: gsp: move chipset-specific parts of the boot process into a HAL
gpu: nova-core: run Booter Unloader and FWSEC-SB upon unbinding
John Hubbard (1):
gpu: nova-core: refactor SEC2 booter loading into BooterFirmware::run()
drivers/gpu/nova-core/driver.rs | 23 +- drivers/gpu/nova-core/firmware/booter.rs | 32 +- drivers/gpu/nova-core/firmware/fwsec.rs | 1 - drivers/gpu/nova-core/gpu.rs | 38 ++- drivers/gpu/nova-core/gsp.rs | 4 + drivers/gpu/nova-core/gsp/boot.rs | 262 ++++++----------- drivers/gpu/nova-core/gsp/commands.rs | 72 +++-- drivers/gpu/nova-core/gsp/fw.rs | 4 + drivers/gpu/nova-core/gsp/fw/commands.rs | 45 +++ drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs | 11 + drivers/gpu/nova-core/gsp/hal.rs | 93 ++++++ drivers/gpu/nova-core/gsp/hal/gh100.rs | 53 ++++ drivers/gpu/nova-core/gsp/hal/tu102.rs | 341 ++++++++++++++++++++++ drivers/gpu/nova-core/regs.rs | 5 + 14 files changed, 783 insertions(+), 201 deletions(-)
---
base-commit: 293c8393b49c9fc017168ddb46aa2012d508c921
change-id: 20251216-nova-unload-4029b3b76950
Best regards,
Signature · 1 line
Alexandre Courbot <[email protected]>
Alexandre Courbot
maintainerPATCH v6 1/7 gpu: nova-core: remove unneeded get_gsp_info proxy function
This function was useful before the generic command-queue send methods got merged, but it is just boilerplate now. Replace it with the correct sequence to queue the GetGspStaticInfo command directly.
--- drivers/gpu/nova-core/gsp/boot.rs | 2 +- drivers/gpu/nova-core/gsp/commands.rs | 8 +------- 2 files changed, 2 insertions(+), 8 deletions(-)
drivers/gpu/nova-core/gsp/boot.rs +1 -1
@@ -243,7 +243,7 @@ pub(crate) fn boot( commands::wait_gsp_init_done (&self.cmdq)?;// Obtain and display basic GPU information. -let info = commands::get_gsp_info (&self.cmdq, bar)?; +let info = self.cmdq.send_command (bar, commands::GetGspStaticInfo )?;match info.gpu_name () {Ok (name) =>dev_info! (pdev,"GPU name: {}\n" , name),Err (e) =>dev_warn! (pdev,"GPU name unavailable: {:?}\n" , e),
drivers/gpu/nova-core/gsp/commands.rs +1 -7
@@ -18,7 +18,6 @@ };use crate ::{ - driver::Bar0 , gsp::{ cmdq::{Cmdq ,
@@ -176,7 +175,7 @@ pub(crate) fn wait_gsp_init_done(cmdq: &Cmdq) -> Result { }/// The `GetGspStaticInfo` command. -struct GetGspStaticInfo ; +pub (crate )struct GetGspStaticInfo ;impl CommandToGsp for GetGspStaticInfo {const FUNCTION :MsgFunction =MsgFunction ::GetGspStaticInfo ;
@@ -232,8 +231,3 @@ pub(crate) fn gpu_name(&self) -> core::result::Result<&str, GpuNameError> { .map_err(GpuNameError::InvalidUtf8) } } - -/// Send the [`GetGspInfo`] command and awaits for its reply. -pub (crate )fn get_gsp_info (cmdq: &Cmdq , bar: &Bar0 ) ->Result <GetGspStaticInfoReply > { - cmdq.send_command (bar,GetGspStaticInfo ) -}
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v6 2/7 gpu: nova-core: do not import firmware commands into GSP command module
In reply to Alexandre CourbotImporting all the firmware commands like we did is a bit confusing, as the layer of a command type (fw or GSP) cannot be inferred from looking at its name alone. Furthermore it makes it impossible to create commands that have the same name as their firmware command.
Thus, stop importing all commands and refer to them from the fw module instead.
--- drivers/gpu/nova-core/gsp/commands.rs | 25 +++++++++++++------------ 1 file changed, 13 insertions(+), 12 deletions(-)
drivers/gpu/nova-core/gsp/commands.rs +13 -12
@@ -1,4 +1,5 @@// SPDX-License-Identifier: GPL-2.0 +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. use core::{ array,
@@ -26,7 +27,7 @@ NoReply, // }, fw::{ - commands::*, + self,MsgFunction ,// }, },
@@ -47,12 +48,12 @@ pub(crate) fn new(pdev: &'bound pci::Device<device::Bound>) -> Self {impl <'bound>CommandToGsp for SetSystemInfo <'bound> {const FUNCTION :MsgFunction =MsgFunction ::GspSetSystemInfo ; -type Command =GspSetSystemInfo ; +type Command = fw::commands::GspSetSystemInfo ;type Reply =NoReply ;type InitError =Error ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > { -GspSetSystemInfo ::init (self.pdev) +Self ::Command ::init (self.pdev) } }
@@ -99,12 +100,12 @@ pub(crate) fn new() -> Self {impl CommandToGsp for SetRegistry {const FUNCTION :MsgFunction =MsgFunction ::SetRegistry ; -type Command =PackedRegistryTable ; +type Command = fw::commands::PackedRegistryTable ;type Reply =NoReply ;type InitError =Infallible ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > { -PackedRegistryTable ::init (Self ::NUM_ENTRIES as u32 , self.variable_payload_len ()as u32 ) +Self ::Command ::init (Self ::NUM_ENTRIES as u32 , self.variable_payload_len ()as u32 ) }fn variable_payload_len (&self) ->usize {
@@ -112,22 +113,22 @@ fn variable_payload_len(&self) -> usize {for iin 0 ..Self ::NUM_ENTRIES { key_size += self.entries[i].key.len () +1 ;// +1 for NULL terminator } -Self ::NUM_ENTRIES *size_of ::<PackedRegistryEntry >() + key_size +Self ::NUM_ENTRIES *size_of ::<fw::commands::PackedRegistryEntry >() + key_size }fn init_variable_payload ( &self, dst: &mut SBufferIter <core::array::IntoIter <&mut [u8 ],2 >>, ) ->Result { -let string_data_start_offset = -size_of ::<PackedRegistryTable >() +Self ::NUM_ENTRIES *size_of ::<PackedRegistryEntry >(); +let string_data_start_offset =size_of ::<Self ::Command >() + +Self ::NUM_ENTRIES *size_of ::<fw::commands::PackedRegistryEntry >();// Array for string data. let mut string_data =KVec ::new ();for entryin self.entries.iter ().take (Self ::NUM_ENTRIES ) { dst.write_all ( -PackedRegistryEntry ::new ( + fw::commands::PackedRegistryEntry ::new ( (string_data_start_offset + string_data.len ())as u32 , entry.value, )
@@ -179,12 +180,12 @@ pub(crate) fn wait_gsp_init_done(cmdq: &Cmdq) -> Result {impl CommandToGsp for GetGspStaticInfo {const FUNCTION :MsgFunction =MsgFunction ::GetGspStaticInfo ; -type Command =GspStaticConfigInfo ; +type Command = fw::commands::GspStaticConfigInfo ;type Reply =GetGspStaticInfoReply ;type InitError =Infallible ;fn init (&self) ->impl Init <Self ::Command ,Self ::InitError > { -GspStaticConfigInfo ::init_zeroed () +Self ::Command ::init_zeroed () } }
@@ -195,7 +196,7 @@ pub(crate) struct GetGspStaticInfoReply {impl MessageFromGsp for GetGspStaticInfoReply {const FUNCTION :MsgFunction =MsgFunction ::GetGspStaticInfo ; -type Message =GspStaticConfigInfo ; +type Message = fw::commands::GspStaticConfigInfo ;type InitError =Infallible ; fn read(
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v6 3/7 gpu: nova-core: send UNLOADING_GUEST_DRIVER GSP command upon unloading
In reply to Alexandre CourbotCurrently, the GSP is left running after the driver is unbound. This is not great for several reasons, notably that it can still access shared memory areas that the kernel will now reclaim (especially problematic on setups without an IOMMU).
Fix this by sending the UNLOADING_GUEST_DRIVER GSP command when unbinding. This stops the GSP and lets us proceed with the rest of the unbind sequence in a later patch.
We make use of the pci::Driver::unbind() hook as we need access to the device in order to properly unbind.
--- drivers/gpu/nova-core/driver.rs | 7 ++++ drivers/gpu/nova-core/gpu.rs | 7 ++++ drivers/gpu/nova-core/gsp/boot.rs | 45 +++++++++++++++++++++++ drivers/gpu/nova-core/gsp/commands.rs | 43 ++++++++++++++++++++++ drivers/gpu/nova-core/gsp/fw.rs | 4 ++ drivers/gpu/nova-core/gsp/fw/commands.rs | 45 +++++++++++++++++++++++ drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs | 11 ++++++ 7 files changed, 162 insertions(+)
drivers/gpu/nova-core/driver.rs +7 -0
@@ -114,4 +114,11 @@ fn probe<'bound>( })) }) } + +fn unbind <'bound>( + dev: &'bound pci::Device <kernel::device::Core >, + this:Pin <&'boundSelf ::Data <'bound>>, + ) { + this.gpu.unbind (dev) + } }
drivers/gpu/nova-core/gpu.rs +7 -0
@@ -293,4 +293,11 @@ pub(crate) fn new( _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? }, }) } + +pub (crate )fn unbind (&self, pdev: &'bound pci::Device <device::Core >) { +let _ = self + .gsp + .unload (pdev.as_ref (), self.bar, &self.gsp_falcon) + .inspect_err (|e|dev_err! (pdev,"failed to unload GSP: {:?}\n" , e)); + } }
drivers/gpu/nova-core/gsp/boot.rs +45 -0
61 lines not on this page · raw message
drivers/gpu/nova-core/gsp/commands.rs +43 -0
46 lines not on this page · raw message
drivers/gpu/nova-core/gsp/fw.rs +4 -0
@@ -279,6 +279,7 @@ pub(crate) enum MsgFunction { Nop = bindings::NV_VGPU_MSG_FUNCTION_NOP, SetGuestSystemInfo = bindings::NV_VGPU_MSG_FUNCTION_SET_GUEST_SYSTEM_INFO, SetRegistry = bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY, + UnloadingGuestDriver = bindings::NV_VGPU_MSG_FUNCTION_UNLOADING_GUEST_DRIVER,// Event codes GspInitDone = bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE,
@@ -323,6 +324,9 @@ fn try_from(value: u32) -> Result<MsgFunction> {Ok (MsgFunction ::SetGuestSystemInfo ) } bindings::NV_VGPU_MSG_FUNCTION_SET_REGISTRY => Ok(MsgFunction::SetRegistry), + bindings::NV_VGPU_MSG_FUNCTION_UNLOADING_GUEST_DRIVER => { + Ok(MsgFunction::UnloadingGuestDriver) + }// Event codes bindings::NV_VGPU_MSG_EVENT_GSP_INIT_DONE => Ok(MsgFunction::GspInitDone),
drivers/gpu/nova-core/gsp/fw/commands.rs +45 -0
52 lines not on this page · raw message
drivers/gpu/nova-core/gsp/fw/r570_144/bindings.rs +11 -0
@@ -30,6 +30,9 @@ fn fmt(&self, fmt: &mut ::core::fmt::Formatter<'_>) -> ::core::fmt::Result { fmt.write_str ("__IncompleteArrayField" ) } } +pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_0 :u32 =0 ; +pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_3 :u32 =3 ; +pub const NV2080_CTRL_GPU_SET_POWER_STATE_GPU_LEVEL_7 :u32 =7 ;pub const NV_VGPU_MSG_SIGNATURE_VALID :u32 =1129337430 ;pub const GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS2 :u32 =0 ;pub const GSP_FW_HEAP_PARAM_OS_SIZE_LIBOS3_BAREMETAL :u32 =23068672 ;
@@ -880,6 +883,14 @@ fn default() -> Self { } }#[repr( C )] +#[derive( Debug , Default , Copy , Clone , MaybeZeroable )] +pub struct rpc_unloading_guest_driver_v1F_07 { +pub bInPMTransition:u8_ , +pub bGc6Entering:u8_ , +pub __bindgen_padding_0: [u8 ;2usize ], +pub newLevel:u32_ , +} +#[repr( C )] #[derive( Debug , Default , MaybeZeroable )] pub struct rpc_run_cpu_sequencer_v17_00 {pub bufferSizeDWord:u32_ ,
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v6 4/7 gpu: nova-core: refactor SEC2 booter loading into BooterFirmware::run()
In reply to Alexandre CourbotFrom: John Hubbard <[email protected]>
Move the SEC2 reset/load/boot sequence into a BooterFirmware::run() method This is mostly refactoring, with no significant behavior change, done in preparation for adding an alternative FSP boot path.
--- drivers/gpu/nova-core/firmware/booter.rs | 31 +++++++++++++++++++++++++++++++ drivers/gpu/nova-core/gsp/boot.rs | 30 ++++++++---------------------- 2 files changed, 39 insertions(+), 22 deletions(-)
drivers/gpu/nova-core/firmware/booter.rs +31 -0
@@ -1,4 +1,5 @@// SPDX-License-Identifier: GPL-2.0 +// SPDX-FileCopyrightText: Copyright (c) 2025-2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. //! Support for loading and patching the `Booter` firmware. `Booter` is a Heavy Secured firmware //! running on [`Sec2`], that is used on Turing/Ampere to load the GSP firmware into the GSP falcon
@@ -8,6 +9,7 @@use kernel::{ device, + dma::Coherent , prelude::*, transmute::FromBytes ,// };
@@ -396,6 +398,35 @@ pub(crate) fn new( ucode: ucode_signed, }) } + +/// Load and run the booter firmware on SEC2. +/// +/// Resets SEC2, loads this firmware image, then boots with the WPR metadata +/// address passed via the SEC2 mailboxes. +pub (crate )fn run <T >( + &self, + dev: &device::Device <device::Bound >, + bar: &Bar0 , + sec2_falcon: &Falcon <Sec2 >, + wpr_meta: &Coherent <T >, + ) ->Result { + sec2_falcon.reset (bar)?; + sec2_falcon.load (dev, bar, self)?; +let wpr_handle = wpr_meta.dma_handle (); +let (mbox0, mbox1) = sec2_falcon.boot ( + bar, +Some (wpr_handleas u32 ), +Some ((wpr_handle >>32 )as u32 ), + )?; +dev_dbg! (dev,"SEC2 MBOX0: {:#x}, MBOX1: {:#x}\n" , mbox0, mbox1); + +if mbox0 !=0 { +dev_err! (dev,"Booter-load failed with error {:#x}\n" , mbox0); +return Err (ENODEV ); + } + +Ok (()) + } }impl FalconDmaLoadable for BooterFirmware {
drivers/gpu/nova-core/gsp/boot.rs +8 -22
@@ -174,15 +174,6 @@ pub(crate) fn boot(Self ::run_fwsec_frts (dev, chipset, gsp_falcon, bar, &bios, &fb_layout)?; } -let booter_loader =BooterFirmware ::new ( - dev, -BooterKind ::Loader , - chipset, -FIRMWARE_VERSION , - sec2_falcon, - bar, - )?; -let wpr_meta =Coherent ::init (dev,GFP_KERNEL ,GspFwWprMeta ::new (&gsp_fw, &fb_layout))?; self.cmdq
@@ -204,20 +195,15 @@ pub(crate) fn boot("Using SEC2 to load and run the booter_load firmware...\n" ); - sec2_falcon.reset (bar)?; - sec2_falcon.load (dev, bar, &booter_loader)?; -let wpr_handle = wpr_meta.dma_handle (); -let (mbox0, mbox1) = sec2_falcon.boot ( +BooterFirmware ::new ( + dev, +BooterKind ::Loader , + chipset, +FIRMWARE_VERSION , + sec2_falcon, bar, -Some (wpr_handleas u32 ), -Some ((wpr_handle >>32 )as u32 ), - )?; -dev_dbg! (pdev,"SEC2 MBOX0: {:#x}, MBOX1: {:#x}\n" , mbox0, mbox1); - -if mbox0 !=0 { -dev_err! (pdev,"Booter-load failed with error {:#x}\n" , mbox0); -return Err (ENODEV ); - } + )? + .run (dev, bar, sec2_falcon, &wpr_meta)?; gsp_falcon.write_os_version (bar, gsp_fw.bootloader.app_version);
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v6 5/7 gpu: nova-core: gsp: shuffle boot code a bit to keep chipset-specific parts close
In reply to Alexandre CourbotSome parts of the GSP boot process are chip-specific actions, whereas others (like sending the initial post-boot messages) deal directly with the working GSP.
Reorganize the boot code a bit so the chipset-specific parts are clumped together, which will make their extraction into a HAL easier.
This has no effect on the GSP boot process.
--- drivers/gpu/nova-core/gsp/boot.rs | 14 +++++++------- 1 file changed, 7 insertions(+), 7 deletions(-)
drivers/gpu/nova-core/gsp/boot.rs +7 -7
@@ -169,18 +169,13 @@ pub(crate) fn boot(let fb_layout =FbLayout ::new (chipset, bar, &gsp_fw)?;dev_dbg! (dev,"{:#x?}\n" , fb_layout); +let wpr_meta =Coherent ::init (dev,GFP_KERNEL ,GspFwWprMeta ::new (&gsp_fw, &fb_layout))?; +// FWSEC-FRTS is not executed on chips where the FRTS region size is 0 (e.g. GA100). if !fb_layout.frts.is_empty () {Self ::run_fwsec_frts (dev, chipset, gsp_falcon, bar, &bios, &fb_layout)?; } -let wpr_meta =Coherent ::init (dev,GFP_KERNEL ,GspFwWprMeta ::new (&gsp_fw, &fb_layout))?; - - self.cmdq - .send_command_no_wait (bar, commands::SetSystemInfo ::new (pdev))?; - self.cmdq - .send_command_no_wait (bar, commands::SetRegistry ::new ())?; - gsp_falcon.reset (bar)?;let libos_handle = self.libos.dma_handle (); let (mbox0, mbox1) = gsp_falcon.boot(
@@ -217,6 +212,11 @@ pub(crate) fn boot(dev_dbg! (pdev,"RISC-V active? {}\n" , gsp_falcon.is_riscv_active (bar),); + self.cmdq + .send_command_no_wait (bar, commands::SetSystemInfo ::new (pdev))?; + self.cmdq + .send_command_no_wait (bar, commands::SetRegistry ::new ())?; +// Create and run the GSP sequencer. let seq_params =GspSequencerParams { bootloader_app_version: gsp_fw.bootloader.app_version,
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v6 6/7 gpu: nova-core: gsp: move chipset-specific parts of the boot process into a HAL
In reply to Alexandre CourbotBooting the GSP is done differently depending on the architecture. Move the parts that are chipset-specific under a HAL.
This does not change much at the moment, since the differences between Turing and Ampere are rather benign, but will become critical to properly support the FSP boot process used by Hopper and Blackwell.
The Hopper/Blackwell support is not merged yet, so their HAL is a stub for now.
This patch is intended to be a mechanical code extraction with no behavioral changes.
--- drivers/gpu/nova-core/gsp.rs | 1 + drivers/gpu/nova-core/gsp/boot.rs | 168 +++------------------------ drivers/gpu/nova-core/gsp/hal.rs | 74 ++++++++++++ drivers/gpu/nova-core/gsp/hal/gh100.rs | 50 ++++++++ drivers/gpu/nova-core/gsp/hal/tu102.rs | 206 +++++++++++++++++++++++++++++++++ 5 files changed, 345 insertions(+), 154 deletions(-)
drivers/gpu/nova-core/gsp.rs +1 -0
@@ -1,6 +1,7 @@// SPDX-License-Identifier: GPL-2.0 mod boot; +mod hal;use kernel::{ debugfs,
drivers/gpu/nova-core/gsp/boot.rs +14 -154
221 lines not on this page · raw message
drivers/gpu/nova-core/gsp/hal.rs new file +74 -0
74 lines not on this page · raw message
drivers/gpu/nova-core/gsp/hal/gh100.rs new file +50 -0
50 lines not on this page · raw message
drivers/gpu/nova-core/gsp/hal/tu102.rs new file +206 -0
206 lines not on this page · raw message
Signature · 1 line
2.54.0
Alexandre Courbot
maintainerPATCH v6 7/7 gpu: nova-core: run Booter Unloader and FWSEC-SB upon unbinding
In reply to Alexandre CourbotWhen probing the driver, the FWSEC-FRTS firmware creates a WPR2 secure memory region to store the GSP firmware, and the Booter Loader loads and starts that firmware into the GSP, making it run in RISC-V mode.
These operations need to be reverted upon unloading, particularly the WPR2 secure region creation, as its presence prevents the driver from subsequently probing.
Thus, prepare the Booter Unloader and FWSEC-SB firmwares when booting the GSP, so they can be executed at unbind time to put the GPU into a state where it can be probed again.
--- drivers/gpu/nova-core/driver.rs | 18 +++- drivers/gpu/nova-core/firmware/booter.rs | 1 - drivers/gpu/nova-core/firmware/fwsec.rs | 1 - drivers/gpu/nova-core/gpu.rs | 35 ++++++-- drivers/gpu/nova-core/gsp.rs | 3 + drivers/gpu/nova-core/gsp/boot.rs | 35 ++++++-- drivers/gpu/nova-core/gsp/hal.rs | 21 ++++- drivers/gpu/nova-core/gsp/hal/gh100.rs | 7 +- drivers/gpu/nova-core/gsp/hal/tu102.rs | 141 ++++++++++++++++++++++++++++++- drivers/gpu/nova-core/regs.rs | 5 ++ 10 files changed, 242 insertions(+), 25 deletions(-)
drivers/gpu/nova-core/driver.rs +15 -3
@@ -1,5 +1,7 @@// SPDX-License-Identifier: GPL-2.0 +use core::cell::Cell ; +use kernel::{ auxiliary, device::Core ,
@@ -21,7 +23,10 @@ types::ForLt, }; -use crate ::gpu::Gpu ; +use crate ::{ + gpu::Gpu , + gsp,// +};/// Counter for generating unique auxiliary device IDs. static AUXILIARY_ID_COUNTER :Atomic <u32 > =Atomic ::new (0 );
@@ -33,6 +38,10 @@ pub(crate) struct NovaCore<'bound> { bar: pci::Bar<'bound, BAR0_SIZE>,#[allow(clippy::type_complexity)] _reg: Devres<auxiliary::Registration<ForLt!(())>>, +/// GSP unload bundle, if any. +/// +/// Stored into a `Cell` so it can be [taken](Cell::take) without a mutable reference. + unload_bundle: Cell<Option<gsp::UnloadBundle>>, }const BAR0_SIZE :usize =SZ_16M ;
@@ -94,6 +103,8 @@ fn probe<'bound>(// other threads of execution. unsafe { pdev.dma_set_mask_and_coherent (DmaMask ::new ::<GPU_DMA_BITS >())? }; +let mut unload_bundle =None ; + Ok(try_pin_init!(NovaCore { bar: pdev.iomap_region_sized::<BAR0_SIZE>(0, c"nova-core/bar0")?, // TODO: Use `&bar` self-referential pin-init syntax once available.
@@ -101,7 +112,7 @@ fn probe<'bound>(// SAFETY: `bar` is initialized before this expression is evaluated // (`try_pin_init!()` initializes fields in declaration order), lives at a pinned // stable address, and is dropped after `gpu` (struct field drop order). - gpu <- Gpu::new(pdev, unsafe { &*core::ptr::from_ref(bar) }), + gpu <- Gpu::new(pdev, unsafe { &*core::ptr::from_ref(bar) }, &mut unload_bundle), _reg: auxiliary::Registration::new( pdev.as_ref (), c"nova-drm",
@@ -111,6 +122,7 @@ fn probe<'bound>( crate::MODULE_NAME, (), )?, + unload_bundle: Cell::new(unload_bundle), })) }) }
@@ -119,6 +131,6 @@ fn unbind<'bound>( dev: &'bound pci::Device<kernel::device::Core>, this: Pin<&'bound Self::Data<'bound>>, ) { - this.gpu.unbind (dev) + this.gpu.unbind (dev, this.unload_bundle.take ()) } }
drivers/gpu/nova-core/firmware/booter.rs +0 -1
@@ -282,7 +282,6 @@ fn new_booter(data: &[u8]) -> Result<Self> {#[derive( Copy , Clone , Debug , PartialEq )] pub (crate )enum BooterKind {Loader , -#[expect(unused)] Unloader , }
drivers/gpu/nova-core/firmware/fwsec.rs +0 -1
@@ -144,7 +144,6 @@ pub(crate) enum FwsecCommand {/// image into it. Frts { frts_addr: u64, frts_size: u64 },/// Asks [`FwsecFirmware`] to load pre-OS apps on the PMU. -#[expect(dead_code)] Sb, }
drivers/gpu/nova-core/gpu.rs +29 -6
@@ -18,7 +18,10 @@ Falcon, // }, fb::SysmemFlush, - gsp::Gsp, + gsp::{ + self, + Gsp, // + }, regs, };
@@ -261,10 +264,20 @@ pub(crate) struct Gpu<'bound> { } impl<'bound> Gpu<'bound> { - pub(crate) fn new( + /// Create a new [`Gpu`] instance. + /// + /// `pdev` is the PCI device for the GPU, `bar` is its `Bar0` mapping. + /// + /// `unload_bundle` is an output parameter, where the [GSP unload bundle](gsp::UnloadBundle) is + /// to be written. The driver layer will pass the written value back to [`Gpu::unbind`]. + pub(crate) fn new<'init>( pdev: &'bound pci::Device<device::Bound>, bar: &'bound Bar0, - ) -> impl PinInit<Self, Error> + 'bound { + unload_bundle: &'init mut Option<gsp::UnloadBundle>, + ) -> impl PinInit<Self, Error> + 'init + where + 'bound: 'init, + { try_pin_init!(Self { spec: Spec::new(pdev.as_ref(), bar).inspect(|spec| { dev_info!(pdev,"NVIDIA ({})\n", spec);
@@ -290,14 +303,24 @@ pub(crate) fn new( gsp <- Gsp::new(pdev), - _: { gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? }, + _: { *unload_bundle = gsp.boot(pdev, bar, spec.chipset, gsp_falcon, sec2_falcon)? }, }) } -pub (crate )fn unbind (&self, pdev: &'bound pci::Device <device::Core >) { +pub (crate )fn unbind ( + &self, + pdev: &'bound pci::Device <device::Core >, + unload_bundle:Option <gsp::UnloadBundle >, + ) {let _ = self .gsp - .unload (pdev.as_ref (), self.bar, &self.gsp_falcon) + .unload ( + pdev.as_ref (), + self.bar, + &self.gsp_falcon, + &self.sec2_falcon, + unload_bundle, + ) .inspect_err (|e|dev_err! (pdev,"failed to unload GSP: {:?}\n" , e)); } }
drivers/gpu/nova-core/gsp.rs +3 -0
@@ -185,3 +185,6 @@ pub(crate) fn new(pdev: &pci::Device<device::Bound>) -> impl PinInit<Self, Error }) } } + +/// Opaque bundle required to unload the GSP. Created by [`Gsp::boot`], consumed by [`Gsp::unload`]. +pub (crate )struct UnloadBundle (KBox <dyn hal::UnloadBundle >);
drivers/gpu/nova-core/gsp/boot.rs +27 -8
@@ -46,7 +46,7 @@ pub(crate) fn boot( chipset: Chipset, gsp_falcon: &Falcon<Gsp>, sec2_falcon: &Falcon<Sec2>, - ) -> Result { + ) -> Result<Option<super::UnloadBundle>> {let dev = pdev.as_ref ();let hal =super ::hal::gsp_hal (chipset);
@@ -57,8 +57,8 @@ pub(crate) fn boot(let wpr_meta =Coherent ::init (dev,GFP_KERNEL ,GspFwWprMeta ::new (&gsp_fw, &fb_layout))?; -// Perform the chipset-specific boot sequence. - hal.boot ( +// Perform the chipset-specific boot sequence, and retrieve the unload bundle. +let unload_bundle = hal.boot ( self.as_mut (), dev, bar,
@@ -98,7 +98,7 @@ pub(crate) fn boot( Err(e) => dev_warn!(pdev, "GPU name unavailable: {:?}\n", e), } -Ok (()) +Ok (unload_bundle.map (super ::UnloadBundle )) }/// Shut down the GSP and wait until it is offline.
@@ -130,16 +130,35 @@ pub(crate) fn unload( dev: &device::Device<device::Bound>, bar: &Bar0, gsp_falcon: &Falcon<Gsp>, + sec2_falcon: &Falcon<Sec2>, + unload_bundle: Option<super::UnloadBundle>, ) -> Result { -// Shut down the GSP. - Self::shutdown_gsp( + // Shut down the GSP. Keep going even in case of error. + let mut res = Self::shutdown_gsp( &self.cmdq, bar, gsp_falcon, commands::PowerStateLevel ::Level0 , ) - .inspect_err(|e| dev_err!(dev, "Unload guest driver failed: {:?}\n", e))?; + .inspect_err (|e|dev_err! (dev,"GSP shutdown failed: {:?}\n" , e)); -Ok (()) +// Run the unload bundle to reset the GSP so it can be booted again. +if let Some (unload_bundle) = unload_bundle { + res = res.and ( + unload_bundle + .0 + .run (dev, bar, gsp_falcon, sec2_falcon) + .inspect_err (|e|dev_err! (dev,"Unload bundle failed: {:?}\n" , e)), + ); + }else { +dev_warn! ( + dev, +"Unload bundle is missing, GSP won't be properly reset.\n" + ); + + res =Err (EAGAIN ); + } + + res.inspect (|()|dev_info! (dev,"GSP successfully unloaded\n" )) } }
drivers/gpu/nova-core/gsp/hal.rs +20 -1
@@ -30,9 +30,28 @@ }, }; +/// Trait for types containing the resources and code required to fully reset the GSP. +/// +/// The GSP unload code might run in a situation where we cannot load firmware dynamically (e.g. +/// because we are in shutdown and the file system is not accessible anymore). Thus, the firmware +/// required for unloading is prepared at load time, and stored here until it needs to be run. +pub (super )trait UnloadBundle :Send { +/// Performs the steps required to properly reset the GSP after it has been stopped. +fn run ( + &self, + dev: &device::Device <device::Bound >, + bar: &Bar0 , + gsp_falcon: &Falcon <GspEngine >, + sec2_falcon: &Falcon <Sec2 >, + ) ->Result ; +} +/// Trait implemented by GSP HALs. pub (super )trait GspHal :Send {/// Performs the GSP boot process, loading and running the required firmwares as needed. +/// +/// Upon success, returns the [`UnloadBundle`] to be run (if any) in order to properly reset the +/// GSP after it has been stopped. #[allow(clippy::too_many_arguments)] fn boot ( &self,
@@ -44,7 +63,7 @@ fn boot( wpr_meta: &Coherent<GspFwWprMeta>, gsp_falcon: &Falcon<GspEngine>, sec2_falcon: &Falcon<Sec2>, - ) -> Result; + ) -> Result<Option<KBox<dyn UnloadBundle>>>;/// Performs HAL-specific post-GSP boot tasks. ///
drivers/gpu/nova-core/gsp/hal/gh100.rs +5 -2
@@ -18,7 +18,10 @@ fb::FbLayout, gpu::Chipset, gsp::{ - hal::GspHal, + hal::{ + GspHal, + UnloadBundle, // + }, Gsp, GspFwWprMeta, // },
@@ -41,7 +44,7 @@ fn boot( _wpr_meta: &Coherent<GspFwWprMeta>, _gsp_falcon: &Falcon<GspEngine>, _sec2_falcon: &Falcon<Sec2>, - ) -> Result { + ) -> Result<Option<KBox<dyn UnloadBundle>>> {Err (ENOTSUPP ) } }
drivers/gpu/nova-core/gsp/hal/tu102.rs +138 -3
165 lines not on this page · raw message
drivers/gpu/nova-core/regs.rs +5 -0
@@ -175,6 +175,11 @@ impl NV_PFB_PRI_MMU_WPR2_ADDR_HI {pub (crate )fn higher_bound (self) ->u64 { u64::from (self.hi_val ()) <<12 } + +/// Returns whether the WPR2 region is currently set. +pub (crate )fn is_wpr2_set (self) ->bool { + self.hi_val () !=0 + } }// PGSP
Signature · 1 line
2.54.0
Re: [PATCH v6 7/7] gpu: nova-core: run Booter Unloader and FWSEC-SB upon unbinding
Quoted 8 lines · Alexandre Courbot
When probing the driver, the FWSEC-FRTS firmware creates a WPR2 secure memory region to store the GSP firmware, and the Booter Loader loads and starts that firmware into the GSP, making it run in RISC-V mode.
These operations need to be reverted upon unloading, particularly the WPR2 secure region creation, as its presence prevents the driver from subsequently probing.
Quoted 6 lines
Thus, prepare the Booter Unloader and FWSEC-SB firmwares when booting the GSP, so they can be executed at unbind time to put the GPU into a state where it can be probed again.
Signed-off-by Alexandre Courbot <[email protected]>---
After seeing the bundle moved outwards, I realised that it has the same issue that SysmemFlush does, i.e. if probe fails it does not reset the GSP. A lot of the time during development I will break things badly enough that probe fails, so it would be nice if this is supported. OTOH, this gets the probe suceed and unload case working which is important and this is a definite improvement, so for this version and the previous version of the patch:
I also had a brief go at making this work on Drop, here is the diff on top of this series. I can send this as a follow up if you would like after cleaning it up, or lmk wdyt:
drivers/gpu/nova-core/driver.rs +2 -21
@@ -1,7 +1,5 @@// SPDX-License-Identifier: GPL-2.0 -use core::cell::Cell ; -use kernel::{ auxiliary, device::Core ,
@@ -23,10 +21,7 @@ types::ForLt, }; -use crate ::{ - gpu::Gpu , - gsp,// -}; +use crate ::gpu::Gpu ;/// Counter for generating unique auxiliary device IDs. static AUXILIARY_ID_COUNTER :Atomic <u32 > =Atomic ::new (0 );
@@ -38,10 +33,6 @@ pub(crate) struct NovaCore<'bound> { bar: pci::Bar<'bound, BAR0_SIZE>,#[allow(clippy::type_complexity)] _reg: Devres<auxiliary::Registration<ForLt!(())>>, -/// GSP unload bundle, if any. -/// -/// Stored into a `Cell` so it can be [taken](Cell::take) without a mutable reference. - unload_bundle: Cell<Option<gsp::UnloadBundle>>, }const BAR0_SIZE :usize =SZ_16M ;
@@ -103,8 +94,6 @@ fn probe<'bound>(// other threads of execution. unsafe { pdev.dma_set_mask_and_coherent (DmaMask ::new ::<GPU_DMA_BITS >())? }; -let mut unload_bundle =None ; - Ok(try_pin_init!(NovaCore { bar: pdev.iomap_region_sized::<BAR0_SIZE>(0, c"nova-core/bar0")?, // TODO: Use `&bar` self-referential pin-init syntax once available.
@@ -112,7 +101,7 @@ fn probe<'bound>(// SAFETY: `bar` is initialized before this expression is evaluated // (`try_pin_init!()` initializes fields in declaration order), lives at a pinned // stable address, and is dropped after `gpu` (struct field drop order). - gpu <- Gpu::new(pdev, unsafe { &*core::ptr::from_ref(bar) }, &mut unload_bundle), + gpu <- Gpu::new(pdev, unsafe { &*core::ptr::from_ref(bar) }), _reg: auxiliary::Registration::new( pdev.as_ref (), c"nova-drm",
@@ -122,15 +111,7 @@ fn probe<'bound>( crate::MODULE_NAME, (), )?, - unload_bundle: Cell::new(unload_bundle), })) }) } - -fn unbind <'bound>( - dev: &'bound pci::Device <kernel::device::Core >, - this:Pin <&'boundSelf ::Data <'bound>>, - ) { - this.gpu.unbind (dev, this.unload_bundle.take ()) - } }
drivers/gpu/nova-core/gpu.rs +21 -20
77 lines not on this page · raw message
Alexandre Courbot
maintainerRe: [PATCH v6 7/7] gpu: nova-core: run Booter Unloader and FWSEC-SB upon unbinding
Quoted 24 lines · Eliot Courtney
Quoted 8 lines · Alexandre Courbot
When probing the driver, the FWSEC-FRTS firmware creates a WPR2 secure memory region to store the GSP firmware, and the Booter Loader loads and starts that firmware into the GSP, making it run in RISC-V mode.
These operations need to be reverted upon unloading, particularly the WPR2 secure region creation, as its presence prevents the driver from subsequently probing.
Quoted 6 lines
Thus, prepare the Booter Unloader and FWSEC-SB firmwares when booting the GSP, so they can be executed at unbind time to put the GPU into a state where it can be probed again.
Signed-off-by Alexandre Courbot <[email protected]>---
After seeing the bundle moved outwards, I realised that it has the same issue that SysmemFlush does, i.e. if probe fails it does not reset the GSP. A lot of the time during development I will break things badly enough that probe fails, so it would be nice if this is supported. OTOH, this gets the probe suceed and unload case working which is important and this is a definite improvement, so for this version and the previous version of the patch:
Quoted 5 lines
Reviewed-by Eliot Courtney <[email protected]>I also had a brief go at making this work on Drop, here is the diff on top of this series. I can send this as a follow up if you would like after cleaning it up, or lmk wdyt:
This is clearly better. It guarantees that the unload sequence is run when the Gpu is dropped, while preserving its one-shot nature. Also, no Cell and no awkward output parameter to Gpu::new.
The only blind spot remaining would be to also cover the case where a failure occurs during Gsp::boot, but that's for Gsp::boot to handle itself imho.
Would you be ok if I folded this into this patch for v7, with your Co-developed-by and Signed-off-by? Then I'll also try to tackle the Gsp::boot failure scenario using a drop wrapper or something similar.
Re: [PATCH v6 7/7] gpu: nova-core: run Booter Unloader and FWSEC-SB upon unbinding
Quoted 35 lines · Alexandre Courbot
Quoted 24 lines · Eliot Courtney
Quoted 8 lines · Alexandre Courbot
When probing the driver, the FWSEC-FRTS firmware creates a WPR2 secure memory region to store the GSP firmware, and the Booter Loader loads and starts that firmware into the GSP, making it run in RISC-V mode.
These operations need to be reverted upon unloading, particularly the WPR2 secure region creation, as its presence prevents the driver from subsequently probing.
Quoted 6 lines
Thus, prepare the Booter Unloader and FWSEC-SB firmwares when booting the GSP, so they can be executed at unbind time to put the GPU into a state where it can be probed again.
Signed-off-by Alexandre Courbot <[email protected]>---
After seeing the bundle moved outwards, I realised that it has the same issue that SysmemFlush does, i.e. if probe fails it does not reset the GSP. A lot of the time during development I will break things badly enough that probe fails, so it would be nice if this is supported. OTOH, this gets the probe suceed and unload case working which is important and this is a definite improvement, so for this version and the previous version of the patch:
Quoted 5 lines
Reviewed-by Eliot Courtney <[email protected]>I also had a brief go at making this work on Drop, here is the diff on top of this series. I can send this as a follow up if you would like after cleaning it up, or lmk wdyt:
This is clearly better. It guarantees that the unload sequence is run when the
Gpuis dropped, while preserving its one-shot nature. Also, noCelland no awkward output parameter toGpu::new.
Quoted 7 lines
The only blind spot remaining would be to also cover the case where a failure occurs during
Gsp::boot, but that's forGsp::bootto handle itself imho.Would you be ok if I folded this into this patch for v7, with your
Co-developed-byandSigned-off-by? Then I'll also try to tackle theGsp::bootfailure scenario using a drop wrapper or something similar.
Yerp that is fine of course. For Gsp::boot it should handle unwinding if there is a failure but I agree it is orthogonal to this case, so addressing in a follow-up SGTM.
Alexandre Courbot
maintainerRe: [PATCH v6 7/7] gpu: nova-core: run Booter Unloader and FWSEC-SB upon unbinding
Quoted 40 lines · Eliot Courtney
Quoted 39 lines · Alexandre Courbot
Quoted 24 lines · Eliot Courtney
Quoted 8 lines · Alexandre Courbot
When probing the driver, the FWSEC-FRTS firmware creates a WPR2 secure memory region to store the GSP firmware, and the Booter Loader loads and starts that firmware into the GSP, making it run in RISC-V mode.
These operations need to be reverted upon unloading, particularly the WPR2 secure region creation, as its presence prevents the driver from subsequently probing.
Quoted 6 lines
Thus, prepare the Booter Unloader and FWSEC-SB firmwares when booting the GSP, so they can be executed at unbind time to put the GPU into a state where it can be probed again.
Signed-off-by Alexandre Courbot <[email protected]>---
After seeing the bundle moved outwards, I realised that it has the same issue that SysmemFlush does, i.e. if probe fails it does not reset the GSP. A lot of the time during development I will break things badly enough that probe fails, so it would be nice if this is supported. OTOH, this gets the probe suceed and unload case working which is important and this is a definite improvement, so for this version and the previous version of the patch:
Quoted 5 lines
Reviewed-by Eliot Courtney <[email protected]>I also had a brief go at making this work on Drop, here is the diff on top of this series. I can send this as a follow up if you would like after cleaning it up, or lmk wdyt:
This is clearly better. It guarantees that the unload sequence is run when the
Gpuis dropped, while preserving its one-shot nature. Also, noCelland no awkward output parameter toGpu::new.The only blind spot remaining would be to also cover the case where a failure occurs during
Gsp::boot, but that's forGsp::bootto handle itself imho.
Quoted 7 lines
Quoted 3 lines
Would you be ok if I folded this into this patch for v7, with your
Co-developed-byandSigned-off-by? Then I'll also try to tackle theGsp::bootfailure scenario using a drop wrapper or something similar.Yerp that is fine of course. For
Gsp::bootit should handle unwinding if there is a failure but I agree it is orthogonal to this case, so addressing in a follow-up SGTM.
I will add a patch that takes care of that using a drop guard in v7.
While doing so, I noticed that with the current state of the code we could also perfectly store that drop guard into Gpu itself, and get the behavior we want without implementing PinnedDrop. However, this is only possible because Falcon::run and Falcon::load do not take mutable references, which they should do eventually for soundness - once we do the switch, storing the drop guard won't be possible anymore as that would mean keeping mutable references for the full life of the GPU. So PinnedDrop is necessary.
I mention this here preemptively lest it is raised in v7. :)
These pre-requisite patches:
Quoted 4 lines
gpu: nova-core: remove unneeded
get_gsp_infoproxy function
gpu: nova-core: do not import firmware commands into GSP command module
gpu: nova-core: refactor SEC2 booter loading intoBooterFirmware::run()
gpu: nova-core: gsp: shuffle boot code a bit to keep chipset-specific parts close
Pushed to drm-rust-next, thanks!
I will make a quick respin for the remaining patches, with the intent of merging v7 quickly if no new issue is raised.