nova-core: run FWSEC-FRTS to perform first stage of GSP initialization
Subsystems (9)
Alexandre Courbot
maintainerv3 cover nova-core: run FWSEC-FRTS to perform first stage of GSP initialization
Hi everyone,
I have respinned this series due to the introduction of Devres::access() in nova-next - this addition seriously simplifies the way we access the BAR and justifies sending a new version, although there are also more minor changes.
While this series is still far from bringing the GPU into a state where it can do anything useful, it sets up the basic layout of the driver upon which we can build in order to continue with the next steps of GSP booting, as well as supporting more chipsets.
Upon successful probe, the driver will display the range of the WPR2 region constructed by FWSEC-FRTS:
[ 95.436000] NovaCore 0000:01:00.0: WPR2: 0xffc00000-0xffce0000 [ 95.436002] NovaCore 0000:01:00.0: GPU instance built
This code is based on nova-next.
There are bits of documentation still missing, these are addressed by Joel in his own documentation patch series [1]. I'll also double-check and send follow-up patches if anything is still missing after that.
I have also tried to look at ways to split the patch adding falcon support, but couldn't find any that would not be awkward. Starting review from falcon.rs and going down to the HAL should be the logical order for a smooth review.
[1] https://lore.kernel.org/rust-for-linux/[email protected]/
---
Changes in v3:
- Rebased on top of latest nova-next.
- Use the new Devres::access() and remove the now unneeded with_bar!() macro.
- Dropped rust: devres: allow to borrow a reference to the resource's Device as it is not needed anymore.
- Fixed more erroneous uses of ERANGE error.
- Optimized alignment computations of the FB layout a bit.
- Link to v2: https://lore.kernel.org/r/[email protected]
Changes in v2:
- Rebased on latest nova-next.
- Fixed all clippy warnings.
- Added count and size methods to CoherentAllocation.
- Added method to obtain a reference to the Device from a Devres (this is super convenient).
- Split DmaObject into its own patch and added Deref implementation.
- Squashed field names from [3] into "extract FWSEC from BIOS".
- Fixed erroneous use of ERANGE error.
- Reworked register!() macro towards a more intuitive syntax, moved
its helper macros into internal rules to avoid polluting the macro namespace.
- Renamed all registers to capital snake case to better match OpenRM.
- Removed declarations for registers that are not used yet.
- Added more documentation for items not covered by Joel's documentation patches.
- Removed timer device and replaced it with a helper function using Ktime. This also made [4] unneeded so it is dropped.
- Unregister the sysmem flush page upon device destruction.
- ... probably more that I forgot. >_<
- Link to v1: https://lore.kernel.org/r/[email protected]
[3] https://lore.kernel.org/all/[email protected]/
[4] https://lore.kernel.org/lkml/[email protected]/
---
Alexandre Courbot (17):
rust: dma: expose the count and size of CoherentAllocation
gpu: nova-core: derive useful traits for Chipset
gpu: nova-core: add missing GA100 definition
gpu: nova-core: take bound device in Gpu::new
gpu: nova-core: define registers layout using helper macro
gpu: nova-core: fix layout of NV_PMC_BOOT_0
gpu: nova-core: move Firmware to firmware module
rust: make ETIMEDOUT error available
gpu: nova-core: wait for GFW_BOOT completion
gpu: nova-core: add DMA object struct
gpu: nova-core: register sysmem flush page
gpu: nova-core: add helper function to wait on condition
gpu: nova-core: add falcon register definitions and base code
gpu: nova-core: firmware: add ucode descriptor used by FWSEC-FRTS
gpu: nova-core: compute layout of the FRTS region
gpu: nova-core: extract FWSEC from BIOS and patch it to run FWSEC-FRTS
gpu: nova-core: load and run FWSEC-FRTS
Joel Fernandes (2):
rust: num: Add an upward alignment helper for usize
nova-core: Add support for VBIOS ucode extraction for boot
Documentation/gpu/nova/core/todo.rst | 6 + drivers/gpu/nova-core/devinit.rs | 38 + drivers/gpu/nova-core/dma.rs | 57 ++ drivers/gpu/nova-core/driver.rs | 2 +- drivers/gpu/nova-core/falcon.rs | 537 ++++++++++++++ drivers/gpu/nova-core/falcon/gsp.rs | 22 + drivers/gpu/nova-core/falcon/hal.rs | 56 ++ drivers/gpu/nova-core/falcon/hal/ga102.rs | 120 +++ drivers/gpu/nova-core/falcon/sec2.rs | 8 + drivers/gpu/nova-core/firmware.rs | 105 ++- drivers/gpu/nova-core/firmware/fwsec.rs | 359 +++++++++ drivers/gpu/nova-core/gpu.rs | 223 ++++-- drivers/gpu/nova-core/gsp.rs | 3 + drivers/gpu/nova-core/gsp/fb.rs | 108 +++ drivers/gpu/nova-core/nova_core.rs | 5 + drivers/gpu/nova-core/regs.rs | 259 +++++-- drivers/gpu/nova-core/regs/macros.rs | 380 ++++++++++ drivers/gpu/nova-core/util.rs | 29 + drivers/gpu/nova-core/vbios.rs | 1144 +++++++++++++++++++++++++++++ rust/kernel/dma.rs | 14 + rust/kernel/error.rs | 1 + rust/kernel/lib.rs | 1 + rust/kernel/num.rs | 44 ++ 23 files changed, 3427 insertions(+), 94 deletions(-)
---
base-commit: b75a99e1077b12c5631fef7ac36970a89f6021f7
change-id: 20250417-nova-frts-96ef299abe2c
Best regards,
Signature · 1 line
Alexandre Courbot <[email protected]>
Alexandre Courbot
maintainerPATCH v3 1/19 rust: dma: expose the count and size of CoherentAllocation
These properties are very useful to have and should be accessible.
--- rust/kernel/dma.rs | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
rust/kernel/dma.rs +14 -0
@@ -201,6 +201,20 @@ pub fn alloc_coherent(CoherentAllocation ::alloc_attrs (dev, count, gfp_flags,Attrs (0 )) } +/// Returns the number of elements `T` in this allocation. +/// +/// Note that this is not the size of the allocation in bytes, which is provided by +/// [`Self::size`]. +pub fn count (&self) ->usize { + self.count + } + +/// Returns the size in bytes of this allocation. +pub fn size (&self) ->usize { +// This is guaranteed not to overflow as the same operation has been done in `alloc_attrs`. + self.count * core::mem::size_of ::<T >() + } +/// Returns the base address to the allocated region in the CPU's virtual address space. pub fn start_ptr (&self) -> *const T { self.cpu_addr
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 01/19] rust: dma: expose the count and size of CoherentAllocation
Quoted 16 lines · Alexandre Courbot
These properties are very useful to have and should be accessible.
Signed-off-by Alexandre Courbot <[email protected]>--- rust/kernel/dma.rs | 14 ++++++++++++++ 1 file changed, 14 insertions(+)rust/kernel/dma.rs +1
-0@@ -201,6 +201,20 @@ pub fn alloc_coherent(CoherentAllocation ::alloc_attrs (dev, count, gfp_flags,Attrs (0 )) } +/// Returns the number of elements `T` in this allocation.
rust/kernel/dma.rs pub fn alloc_coherent( · line 214
Quoted 10 lines
rust/kernel/dma.rs +10
-0+/// +/// Note that this is not the size of the allocation in bytes, which is provided by +/// [`Self::size`]. +pub fn count (&self) ->usize { + self.count + } + +/// Returns the size in bytes of this allocation. +pub fn size (&self) ->usize { +// This is guaranteed not to overflow as the same operation has been done in `alloc_attrs`.
Please make this an invariant of CoherentAllocation and refer to that instead.
Quoted 10 lines
rust/kernel/dma.rs +3
-0+ self.count * core::mem::size_of ::<T >() + } +/// Returns the base address to the allocated region in the CPU's virtual address space. pub fn start_ptr (&self) -> *const T { self.cpu_addrSignature · 2 lines
-- 2.49.0
Alexandre Courbot
maintainerPATCH v3 2/19 gpu: nova-core: derive useful traits for Chipset
In reply to Alexandre CourbotWe will commonly need to compare chipset versions, so derive the ordering traits to make that possible. Also derive Copy and Clone since passing Chipset by value will be more efficient than by reference.
--- drivers/gpu/nova-core/gpu.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/nova-core/gpu.rs +1 -1
@@ -13,7 +13,7 @@ macro_rules! define_chipset { ({ $($variant:ident = $value:expr),* $(,)* }) => {/// Enum representation of the GPU chipset. -#[derive(fmt:: Debug )] +#[derive(fmt:: Debug , Copy , Clone , PartialOrd , Ord , PartialEq , Eq )] pub(crate) enum Chipset { $($variant = $value),*, }
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 3/19 gpu: nova-core: add missing GA100 definition
In reply to Alexandre Courbotlinux-firmware contains a directory for GA100, and it is a defined chipset in Nouveau.
--- drivers/gpu/nova-core/gpu.rs | 3 ++- 1 file changed, 2 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/gpu.rs +2 -1
@@ -54,6 +54,7 @@ fn try_from(value: u32) -> Result<Self, Self::Error> { TU117 = 0x167, TU116 = 0x168,// Ampere + GA100 = 0x170, GA102 = 0x172, GA103 = 0x173, GA104 = 0x174,
@@ -73,7 +74,7 @@ pub(crate) fn arch(&self) -> Architecture { Self::TU102 | Self::TU104 | Self::TU106 | Self::TU117 | Self::TU116 => { Architecture::Turing } - Self::GA102 | Self::GA103 | Self::GA104 | Self::GA106 | Self::GA107 => { + Self::GA100 | Self::GA102 | Self::GA103 | Self::GA104 | Self::GA106 | Self::GA107 => { Architecture::Ampere } Self::AD102 | Self::AD103 | Self::AD104 | Self::AD106 | Self::AD107 => {
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 4/19 gpu: nova-core: take bound device in Gpu::new
In reply to Alexandre CourbotWe will need to perform things like allocating DMA memory during device creation, so make sure to take the device context that will allow us to perform these actions. This also allows us to use Devres::access to obtain the BAR without holding a RCU lock.
--- drivers/gpu/nova-core/gpu.rs | 19 +++++++++++++------ 1 file changed, 13 insertions(+), 6 deletions(-)
drivers/gpu/nova-core/gpu.rs +13 -6
@@ -134,9 +134,8 @@ pub(crate) struct Spec { }impl Spec { -fn new (bar: &Devres <Bar0 >) ->Result <Spec > { -let bar = bar.try_access ().ok_or (ENXIO )?; -let boot0 = regs::Boot0 ::read (&bar); +fn new (bar: &Bar0 ) ->Result <Spec > { +let boot0 = regs::Boot0 ::read (bar);Ok (Self { chipset: boot0.chipset ().try_into ()?,
@@ -183,8 +182,12 @@ pub(crate) struct Gpu { }impl Gpu { -pub (crate )fn new (pdev: &pci::Device , bar:Devres <Bar0 >) ->Result <impl PinInit <Self >> { -let spec =Spec ::new (&bar)?; +pub (crate )fn new ( + pdev: &pci::Device <device::Bound >, + devres_bar:Devres <Bar0 >, + ) ->Result <impl PinInit <Self >> { +let bar = devres_bar.access (pdev.as_ref ())?; +let spec =Spec ::new (bar)?;let fw =Firmware ::new (pdev.as_ref (), &spec,"535.113.01" )?;dev_info! (
@@ -195,6 +198,10 @@ pub(crate) fn new(pdev: &pci::Device, bar: Devres<Bar0>) -> Result<impl PinInit< spec.revision ); -Ok (pin_init! (Self { spec, bar, fw })) +Ok (pin_init! (Self { + spec, + bar: devres_bar, + fw + })) } }
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 5/19 gpu: nova-core: define registers layout using helper macro
In reply to Alexandre CourbotAdd the register!() macro, which defines a given register's layout and provide bit-field accessors with a way to convert them to a given type. This macro will allow us to make clear definitions of the registers and manipulate their fields safely.
The long-term goal is to eventually move it to the kernel crate so it can be used my other drivers as well, but it was agreed to first land it into nova-core and make it mature there.
To illustrate its usage, use it to define the layout for the Boot0 (renamed to NV_PMC_BOOT_0 to match OpenRM's naming scheme) and take advantage of its accessors.
--- Documentation/gpu/nova/core/todo.rst | 6 + drivers/gpu/nova-core/gpu.rs | 10 +- drivers/gpu/nova-core/regs.rs | 61 ++---- drivers/gpu/nova-core/regs/macros.rs | 380 +++++++++++++++++++++++++++++++++++ 4 files changed, 403 insertions(+), 54 deletions(-)
Documentation/gpu/nova/core/todo.rst +6 -0
@@ -102,7 +102,13 @@ Usage: let boot0 = Boot0::read(&bar); pr_info!("Revision: {}\n", boot0.revision()); +Note: a work-in-progress implementation currently resides in +`drivers/gpu/nova-core/regs/macros.rs` and is used in nova-core. It would be +nice to improve it (possibly using proc macros) and move it to the `kernel` +crate so it can be used by other components as well. + | Complexity: Advanced +| Contact: Alexandre Courbot Delay / Sleep abstractions --------------------------
drivers/gpu/nova-core/gpu.rs +5 -5
@@ -112,10 +112,10 @@ pub(crate) struct Revision { }impl Revision { -fn from_boot0 (boot0: regs::Boot0 ) ->Self { +fn from_boot0 (boot0: regs::NV_PMC_BOOT_0 ) ->Self {Self { - major: boot0.major_rev (), - minor: boot0.minor_rev (), + major: boot0.major_revision (), + minor: boot0.minor_revision (), } } }
@@ -135,10 +135,10 @@ pub(crate) struct Spec {impl Spec {fn new (bar: &Bar0 ) ->Result <Spec > { -let boot0 = regs::Boot0 ::read (bar); +let boot0 = regs::NV_PMC_BOOT_0 ::read (bar);Ok (Self { - chipset: boot0.chipset ().try_into ()?, + chipset: boot0.chipset ()?, revision:Revision ::from_boot0 (boot0), }) }
drivers/gpu/nova-core/regs.rs +12 -49
67 lines not on this page · raw message
drivers/gpu/nova-core/regs/macros.rs new file +380 -0
380 lines not on this page · raw message
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 6/19 gpu: nova-core: fix layout of NV_PMC_BOOT_0
In reply to Alexandre CourbotThe layout of NV_PMC_BOOT_0 has two small issues:
- The "chipset" field, while useful to identify a chip, is actually an aggregate of two distinct fields named "architecture" and "implementation".
- The "architecture" field is split, with its MSB being at a different location than the rest of its bits.
Redefine the register layout to match its actual definition as provided by OpenRM and expose the fully-constructed "architecture" field through our own "Architecture" type. The "chipset" pseudo-field is also useful to have, so keep providing it.
--- drivers/gpu/nova-core/gpu.rs | 19 ++++++++++++++++--- drivers/gpu/nova-core/regs.rs | 26 ++++++++++++++++++++++++-- 2 files changed, 40 insertions(+), 5 deletions(-)
drivers/gpu/nova-core/gpu.rs +16 -3
@@ -101,9 +101,22 @@ fn fmt(&self, f: &mut fmt::Formatter<'_>) -> fmt::Result {/// Enum representation of the GPU generation. #[derive(fmt:: Debug )] pub (crate )enum Architecture { -Turing , -Ampere , -Ada , +Turing =0x16 , +Ampere =0x17 , +Ada =0x19 , +} + +impl TryFrom <u8 >for Architecture { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > { +match value { +0x16 =>Ok (Self ::Turing ), +0x17 =>Ok (Self ::Ampere ), +0x19 =>Ok (Self ::Ada ), + _ =>Err (ENODEV ), + } + } }pub (crate )struct Revision {
drivers/gpu/nova-core/regs.rs +24 -2
@@ -4,15 +4,37 @@// but are mapped to types. #![allow(non_camel_case_types)] +use kernel::error::Error ; +#[macro_use] mod macros; -use crate ::gpu::Chipset ; +use crate ::gpu::{Architecture ,Chipset };/* PMC */ register! (NV_PMC_BOOT_0 @0x00000000 ,"Basic revision information about the GPU" {3 :0 minor_revisionas u8 ,"Minor revision of the chip" ;7 :4 major_revisionas u8 ,"Major revision of the chip" ; -28 :20 chipsetas u32 ?=>Chipset ,"Chipset model" ; +8 :8 architecture_1as u8 ,"MSB of the architecture" ; +23 :20 implementationas u8 ,"Implementation version of the architecture" ; +28 :24 architecture_0as u8 ,"Lower bits of the architecture" ; }); + +impl NV_PMC_BOOT_0 { +/// Combines `architecture_0` and `architecture_1` to obtain the architecture of the chip. +pub (crate )fn architecture (self) ->Result <Architecture ,Error > { +Architecture ::try_from ( + self.architecture_0 () | (self.architecture_1 () <<Self ::ARCHITECTURE_0 .len ()), + ) + } + +/// Combines `architecture` and `implementation` to obtain a code unique to the chipset. +pub (crate )fn chipset (self) ->Result <Chipset ,Error > { + self.architecture () + .map (|arch| { + ((archas u32 ) <<Self ::IMPLEMENTATION .len ()) | self.implementation ()as u32 + }) + .and_then (Chipset ::try_from) + } +}
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 7/19 gpu: nova-core: move Firmware to firmware module
In reply to Alexandre CourbotWe will extend the firmware methods, so move it to its own module instead to keep gpu.rs focused.
--- drivers/gpu/nova-core/firmware.rs | 46 +++++++++++++++++++++++++++++++++++++-- drivers/gpu/nova-core/gpu.rs | 35 +++-------------------------- 2 files changed, 47 insertions(+), 34 deletions(-)
drivers/gpu/nova-core/firmware.rs +44 -2
@@ -1,12 +1,54 @@// SPDX-License-Identifier: GPL-2.0 -use crate ::gpu; +//! Contains structures and functions dedicated to the parsing, building and patching of firmwares +//! to be loaded into a given execution unit. + +use kernel::device;use kernel::firmware; +use kernel::prelude::*; +use kernel::str::CString ; + +use crate ::gpu; +use crate ::gpu::Chipset ; + +pub (crate )const FIRMWARE_VERSION : &str ="535.113.01" ; + +/// Structure encapsulating the firmware blobs required for the GPU to operate. +#[expect(dead_code)] +pub (crate )struct Firmware { +pub booter_load: firmware::Firmware , +pub booter_unload: firmware::Firmware , +pub bootloader: firmware::Firmware , +pub gsp: firmware::Firmware , +} + +impl Firmware { +pub (crate )fn new ( + dev: &device::Device <device::Bound >, + chipset:Chipset , + ver: &str , + ) ->Result <Firmware > { +let mut chip_name =CString ::try_from_fmt (fmt! ("{}" , chipset))?; + chip_name.make_ascii_lowercase (); + +let request = |name_| { +CString ::try_from_fmt (fmt! ("nvidia/{}/gsp/{}-{}.bin" , &*chip_name, name_, ver)) + .and_then (|path| firmware::Firmware ::request (&path, dev)) + }; + +Ok (Firmware { + booter_load:request ("booter_load" )?, + booter_unload:request ("booter_unload" )?, + bootloader:request ("bootloader" )?, + gsp:request ("gsp" )?, + }) + } +}pub (crate )struct ModInfoBuilder <const N :usize >(firmware::ModInfoBuilder <N >);impl <const N :usize >ModInfoBuilder <N > { -const VERSION : &'staticstr ="535.113.01" ; +const VERSION : &'staticstr =FIRMWARE_VERSION ;const fn make_entry_file (self, chipset: &str , fw: &str ) ->Self { ModInfoBuilder(
drivers/gpu/nova-core/gpu.rs +3 -32
@@ -1,10 +1,9 @@// SPDX-License-Identifier: GPL-2.0 -use kernel::{ - device, devres::Devres , error::code::*, firmware, fmt, pci, prelude::*, str::CString , -}; +use kernel::{device, devres::Devres , error::code::*, pci, prelude::*};use crate ::driver::Bar0 ; +use crate ::firmware::Firmware ;use crate ::regs;use crate ::util;use core::fmt;
@@ -157,34 +156,6 @@ fn new(bar: &Bar0) -> Result<Spec> { } } -/// Structure encapsulating the firmware blobs required for the GPU to operate. -#[expect(dead_code)] -pub (crate )struct Firmware { - booter_load: firmware::Firmware , - booter_unload: firmware::Firmware , - bootloader: firmware::Firmware , - gsp: firmware::Firmware , -} - -impl Firmware { -fn new (dev: &device::Device , spec: &Spec , ver: &str ) ->Result <Firmware > { -let mut chip_name =CString ::try_from_fmt (fmt! ("{}" , spec.chipset))?; - chip_name.make_ascii_lowercase (); - -let request = |name_| { -CString ::try_from_fmt (fmt! ("nvidia/{}/gsp/{}-{}.bin" , &*chip_name, name_, ver)) - .and_then (|path| firmware::Firmware ::request (&path, dev)) - }; - -Ok (Firmware { - booter_load:request ("booter_load" )?, - booter_unload:request ("booter_unload" )?, - bootloader:request ("bootloader" )?, - gsp:request ("gsp" )?, - }) - } -} -/// Structure holding the resources required to operate the GPU. #[pin_data] pub (crate )struct Gpu {
@@ -201,7 +172,7 @@ pub(crate) fn new( ) -> Result<impl PinInit<Self>> {let bar = devres_bar.access (pdev.as_ref ())?;let spec =Spec ::new (bar)?; -let fw =Firmware ::new (pdev.as_ref (), &spec,"535.113.01" )?; +let fw =Firmware ::new (pdev.as_ref (), spec.chipset,"535.113.01" )?; dev_info!( pdev.as_ref(),
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 8/19 rust: make ETIMEDOUT error available
In reply to Alexandre CourbotWe will use this error in the nova-core driver.
--- rust/kernel/error.rs | 1 + 1 file changed, 1 insertion(+)
rust/kernel/error.rs +1 -0
@@ -65,6 +65,7 @@ macro_rules! declare_err {declare_err! (EDOM ,"Math argument out of domain of func." );declare_err! (ERANGE ,"Math result not representable." );declare_err! (EOVERFLOW ,"Value too large for defined data type." ); +declare_err! (ETIMEDOUT ,"Connection timed out." );declare_err! (ERESTARTSYS ,"Restart the system call." );declare_err! (ERESTARTNOINTR ,"System call was interrupted by a signal and will be restarted." );declare_err! (ERESTARTNOHAND ,"Restart if no handler." );
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 9/19 gpu: nova-core: wait for GFW_BOOT completion
In reply to Alexandre CourbotUpon reset, the GPU executes the GFW_BOOT firmware in order to initialize its base parameters such as clocks. The driver must ensure that this step is completed before using the hardware.
--- drivers/gpu/nova-core/devinit.rs | 38 ++++++++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/driver.rs | 2 +- drivers/gpu/nova-core/gpu.rs | 5 +++++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 11 +++++++++++ 5 files changed, 56 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/devinit.rs new file +38 -0
@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Methods for device initialization. + +use kernel::bindings; +use kernel::prelude::*; + +use crate ::driver::Bar0 ; +use crate ::regs; + +/// Wait for devinit FW completion. +/// +/// Upon reset, the GPU runs some firmware code to setup its core parameters. Most of the GPU is +/// considered unusable until this step is completed, so it must be waited on very early during +/// driver initialization. +pub (crate )fn wait_gfw_boot_completion (bar: &Bar0 ) ->Result <()> { +let mut timeout =2000 ; + +loop { +let gfw_booted = regs::NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK ::read (bar) + .read_protection_level0 () + && (regs::NV_PGC6_AON_SECURE_SCRATCH_GROUP_05 ::read (bar).value () &0xff ) ==0xff ; + +if gfw_booted { +return Ok (()); + } + +if timeout ==0 { +return Err (ETIMEDOUT ); + } + timeout -=1 ; + +// TODO: use `read_poll_timeout` once it is available. +// (https://lore.kernel.org/lkml/[email protected]/) +// SAFETY: msleep should be safe to call with any parameter. +unsafe { bindings::msleep (2 ) }; + } +}
drivers/gpu/nova-core/driver.rs +1 -1
@@ -10,7 +10,7 @@ pub(crate) struct NovaCore { pub(crate) gpu: Gpu, } -const BAR0_SIZE :usize =8 ; +const BAR0_SIZE :usize =0x1000000 ;pub (crate )type Bar0 = pci::Bar <BAR0_SIZE >; kernel::pci_device_table!(
drivers/gpu/nova-core/gpu.rs +5 -0
@@ -2,6 +2,7 @@use kernel::{device, devres::Devres , error::code::*, pci, prelude::*}; +use crate ::devinit;use crate ::driver::Bar0 ;use crate ::firmware::Firmware ;use crate ::regs;
@@ -182,6 +183,10 @@ pub(crate) fn new( spec.revision ); +// We must wait for GFW_BOOT completion before doing any significant setup on the GPU. + devinit::wait_gfw_boot_completion (bar) + .inspect_err (|_|dev_err! (pdev.as_ref (),"GFW boot did not complete" ))?; + Ok(pin_init!(Self { spec, bar: devres_bar,
drivers/gpu/nova-core/nova_core.rs +1 -0
@@ -2,6 +2,7 @@//! Nova Core GPU Driver +mod devinit;mod driver;mod firmware;mod gpu;
drivers/gpu/nova-core/regs.rs +11 -0
@@ -38,3 +38,14 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> { .and_then(Chipset::try_from) } } + +/* PGC6 */ + +register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK @0x00118128 { +0 :0 read_protection_level0as bool ; +}); + +// TODO: This is an array of registers. +register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_05 @0x00118234 { +31 :0 valueas u32 ; +});
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 09/19] gpu: nova-core: wait for GFW_BOOT completion
Quoted 31 lines · Alexandre Courbot
Upon reset, the GPU executes the
GFW_BOOTfirmware in order to initialize its base parameters such as clocks. The driver must ensure that this step is completed before using the hardware.Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/devinit.rs | 38 ++++++++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/driver.rs | 2 +- drivers/gpu/nova-core/gpu.rs | 5 +++++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 11 +++++++++++ 5 files changed, 56 insertions(+), 1 deletion(-)drivers/gpu/nova-core/devinit.rs new file +12
-0@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Methods for device initialization. + +use kernel::bindings; +use kernel::prelude::*; + +use crate ::driver::Bar0 ; +use crate ::regs; + +/// Wait for devinit FW completion. +///
drivers/gpu/nova-core/devinit.rs line 22
Quoted 10 lines
drivers/gpu/nova-core/devinit.rs new file +10
-0+/// Upon reset, the GPU runs some firmware code to setup its core parameters. Most of the GPU is +/// considered unusable until this step is completed, so it must be waited on very early during +/// driver initialization. +pub (crate )fn wait_gfw_boot_completion (bar: &Bar0 ) ->Result <()> { +let mut timeout =2000 ; + +loop { +let gfw_booted = regs::NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK ::read (bar) + .read_protection_level0 () + && (regs::NV_PGC6_AON_SECURE_SCRATCH_GROUP_05 ::read (bar).value () &0xff ) ==0xff ;
What does it mean when the first 8 bits are set? Why don't we have a mask and value to compare for that?
drivers/gpu/nova-core/devinit.rs line 31
Quoted 9 lines
drivers/gpu/nova-core/devinit.rs new file +9
-0+ +if gfw_booted { +return Ok (()); + } + +if timeout ==0 { +return Err (ETIMEDOUT ); + } + timeout -=1 ;
NIT: This means the timeout is ~4s; can we start with timeout == 4000 and decrement with the number of ms passed to msleep()?
Anyways, this should go away with read_poll_timeout() anyways.
Quoted 12 lines
drivers/gpu/nova-core/devinit.rs new file +7
-0+ +// TODO: use `read_poll_timeout` once it is available. +// (https://lore.kernel.org/lkml/[email protected]/) +// SAFETY: msleep should be safe to call with any parameter. +unsafe { bindings::msleep (2 ) }; + } +}drivers/gpu/nova-core/driver.rs +0
-0
drivers/gpu/nova-core/driver.rs pub(crate) struct NovaCore { · line 13
Quoted 5 lines
drivers/gpu/nova-core/driver.rs +1
-1@@ -10,7 +10,7 @@ pub(crate) struct NovaCore { pub(crate) gpu: Gpu, } -const BAR0_SIZE :usize =8 ; +const BAR0_SIZE :usize =0x1000000 ;
This means that we'll fail probing the card if BAR0 is not at least 16MiB. AFAIK, that should be fine. However, can you make this a separate patch please?
Alexandre Courbot
maintainerRe: [PATCH v3 09/19] gpu: nova-core: wait for GFW_BOOT completion
Quoted 37 lines · Danilo Krummrich
Quoted 36 lines · Alexandre Courbot
Upon reset, the GPU executes the
GFW_BOOTfirmware in order to initialize its base parameters such as clocks. The driver must ensure that this step is completed before using the hardware.Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/devinit.rs | 38 ++++++++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/driver.rs | 2 +- drivers/gpu/nova-core/gpu.rs | 5 +++++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 11 +++++++++++ 5 files changed, 56 insertions(+), 1 deletion(-)drivers/gpu/nova-core/devinit.rs new file +17
-0@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Methods for device initialization. + +use kernel::bindings; +use kernel::prelude::*; + +use crate ::driver::Bar0 ; +use crate ::regs; + +/// Wait for devinit FW completion. +/// +/// Upon reset, the GPU runs some firmware code to setup its core parameters. Most of the GPU is +/// considered unusable until this step is completed, so it must be waited on very early during +/// driver initialization. +pub (crate )fn wait_gfw_boot_completion (bar: &Bar0 ) ->Result <()> { +let mut timeout =2000 ;
Quoted 8 lines
drivers/gpu/nova-core/devinit.rs line 22
Quoted 5 lines
drivers/gpu/nova-core/devinit.rs new file +5
-0+ +loop { +let gfw_booted = regs::NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK ::read (bar) + .read_protection_level0 () + && (regs::NV_PGC6_AON_SECURE_SCRATCH_GROUP_05 ::read (bar).value () &0xff ) ==0xff ;What does it mean when the first 8 bits are set? Why don't we have a mask and value to compare for that?
Since this is a scratch register, the particular behavior of the bits was documented in a separate OpenRM file - I finally found the explanation after looking it up.
This appears to be a GFW boot progress counter, with 0xff meaninig that boot is completed. I have added a (documented) specialization of this register as well as a helper function to make this clear.
Quoted 14 lines
drivers/gpu/nova-core/devinit.rs line 31
Quoted 10 lines
drivers/gpu/nova-core/devinit.rs new file +9
-0+ +if gfw_booted { +return Ok (()); + } + +if timeout ==0 { +return Err (ETIMEDOUT ); + } + timeout -=1 ;NIT: This means the timeout is ~4s; can we start with timeout == 4000 and decrement with the number of ms passed to
msleep()?
Quoted 1 line
Anyways, this should go away with
read_poll_timeout()anyways.
Yes, the timeout code was a bit weird. Actually as you pointed out, we can also use the wait_on helper introduced later in this series.
Quoted 16 lines
Quoted 16 lines
drivers/gpu/nova-core/devinit.rs new file +7
-0+ +// TODO: use `read_poll_timeout` once it is available. +// (https://lore.kernel.org/lkml/[email protected]/) +// SAFETY: msleep should be safe to call with any parameter. +unsafe { bindings::msleep (2 ) }; + } +}drivers/gpu/nova-core/driver.rs +0
-0@@ -10,7 +10,7 @@ pub(crate) struct NovaCore { pub(crate) gpu: Gpu, }
Quoted 5 lines
drivers/gpu/nova-core/driver.rs pub(crate) struct NovaCore { · line 13
Quoted 2 lines
drivers/gpu/nova-core/driver.rs +1
-1-const BAR0_SIZE :usize =8 ; +const BAR0_SIZE :usize =0x1000000 ;This means that we'll fail probing the card if BAR0 is not at least 16MiB. AFAIK, that should be fine. However, can you make this a separate patch please?
Sure!
Alexandre Courbot
maintainerPATCH v3 10/19 gpu: nova-core: add DMA object struct
In reply to Alexandre CourbotSince we will need to allocate lots of distinct memory chunks to be shared between GPU and CPU, introduce a type dedicated to that. It is a light wrapper around CoherentAllocation.
--- drivers/gpu/nova-core/dma.rs | 60 ++++++++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 1 + 2 files changed, 61 insertions(+)
drivers/gpu/nova-core/dma.rs new file +60 -0
@@ -0,0 +1,60 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Simple DMA object wrapper. + +// To be removed when all code is used. +#![expect(dead_code)] + +use core::ops::{Deref ,DerefMut }; + +use kernel::device; +use kernel::dma::CoherentAllocation ; +use kernel::page::PAGE_SIZE ; +use kernel::prelude::*; + +pub (crate )struct DmaObject { + dma:CoherentAllocation <u8 >, +} + +impl DmaObject { +pub (crate )fn new (dev: &device::Device <device::Bound >, len:usize ) ->Result <Self > { +let len = core::alloc::Layout ::from_size_align (len,PAGE_SIZE ) + .map_err (|_|EINVAL )? + .pad_to_align () + .size (); +let dma =CoherentAllocation ::alloc_coherent (dev, len,GFP_KERNEL | __GFP_ZERO)?; + +Ok (Self { dma }) + } + +pub (crate )fn from_data (dev: &device::Device <device::Bound >, data: &[u8 ]) ->Result <Self > { +Self ::new (dev, data.len ()).map (|mut dma_obj| { +// SAFETY: +// - The copied data fits within the size of the allocated object. +// - We have just created this object and there is no other user at this stage. +unsafe { + core::ptr::copy_nonoverlapping ( + data.as_ptr (), + dma_obj.dma.start_ptr_mut (), + data.len (), + ); + } + + dma_obj + }) + } +} + +impl Deref for DmaObject { +type Target =CoherentAllocation <u8 >; + +fn deref (&self) -> &Self ::Target { + &self.dma + } +} + +impl DerefMut for DmaObject { +fn deref_mut (&mut self) -> &mut Self ::Target { + &mut self.dma + } +}
drivers/gpu/nova-core/nova_core.rs +1 -0
@@ -3,6 +3,7 @@//! Nova Core GPU Driver mod devinit; +mod dma;mod driver;mod firmware;mod gpu;
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 10/19] gpu: nova-core: add DMA object struct
Quoted 47 lines · Alexandre Courbot
Since we will need to allocate lots of distinct memory chunks to be shared between GPU and CPU, introduce a type dedicated to that. It is a light wrapper around CoherentAllocation.
Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/dma.rs | 60 ++++++++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 1 + 2 files changed, 61 insertions(+)drivers/gpu/nova-core/dma.rs new file +31
-0@@ -0,0 +1,60 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Simple DMA object wrapper. + +// To be removed when all code is used. +#![expect(dead_code)] + +use core::ops::{Deref ,DerefMut }; + +use kernel::device; +use kernel::dma::CoherentAllocation ; +use kernel::page::PAGE_SIZE ; +use kernel::prelude::*; + +pub (crate )struct DmaObject { + dma:CoherentAllocation <u8 >, +} + +impl DmaObject { +pub (crate )fn new (dev: &device::Device <device::Bound >, len:usize ) ->Result <Self > { +let len = core::alloc::Layout ::from_size_align (len,PAGE_SIZE ) + .map_err (|_|EINVAL )? + .pad_to_align () + .size (); +let dma =CoherentAllocation ::alloc_coherent (dev, len,GFP_KERNEL | __GFP_ZERO)?; + +Ok (Self { dma }) + } + +pub (crate )fn from_data (dev: &device::Device <device::Bound >, data: &[u8 ]) ->Result <Self > { +Self ::new (dev, data.len ()).map (|mut dma_obj| {
drivers/gpu/nova-core/dma.rs line 41
Quoted 10 lines
drivers/gpu/nova-core/dma.rs new file +10
-0+// SAFETY: +// - The copied data fits within the size of the allocated object. +// - We have just created this object and there is no other user at this stage. +unsafe { + core::ptr::copy_nonoverlapping ( + data.as_ptr (), + dma_obj.dma.start_ptr_mut (), + data.len (), + ); + }
This will be replaced with CoherentAllocation::write() I suppose? Please add a corresponding TODO.
Alexandre Courbot
maintainerPATCH v3 11/19 gpu: nova-core: register sysmem flush page
In reply to Alexandre CourbotA page of system memory is reserved so sysmembar can perform a read on it if a system write occurred since the last flush. Do this early as it can be required to e.g. reset the GPU falcons.
--- drivers/gpu/nova-core/gpu.rs | 45 +++++++++++++++++++++++++++++++++++++++++-- drivers/gpu/nova-core/regs.rs | 10 ++++++++++ 2 files changed, 53 insertions(+), 2 deletions(-)
drivers/gpu/nova-core/gpu.rs +43 -2
@@ -3,6 +3,7 @@use kernel::{device, devres::Devres , error::code::*, pci, prelude::*};use crate ::devinit; +use crate ::dma::DmaObject ;use crate ::driver::Bar0 ;use crate ::firmware::Firmware ;use crate ::regs;
@@ -158,12 +159,32 @@ fn new(bar: &Bar0) -> Result<Spec> { }/// Structure holding the resources required to operate the GPU. -#[pin_data] +#[pin_data( PinnedDrop )] pub (crate )struct Gpu { spec:Spec ,/// MMIO mapping of PCI BAR 0 bar:Devres <Bar0 >, fw:Firmware , +// System memory page required for flushing all pending GPU-side memory writes done through +// PCIE into system memory. + sysmem_flush:DmaObject , +} + +#[pinned_drop] +impl PinnedDrop for Gpu { +fn drop (self:Pin <&mut Self >) { +// Unregister the sysmem flush page before we release it. +let _ = self.bar.try_access_with (|b| { + regs::NV_PFB_NISO_FLUSH_SYSMEM_ADDR ::default () + .set_adr_39_08 (0 ) + .write (b); +if self.spec.chipset >=Chipset ::GA102 { + regs::NV_PFB_NISO_FLUSH_SYSMEM_ADDR_HI ::default () + .set_adr_63_40 (0 ) + .write (b); + } + }); + } }impl Gpu {
@@ -187,10 +208,30 @@ pub(crate) fn new( devinit::wait_gfw_boot_completion (bar) .inspect_err (|_|dev_err! (pdev.as_ref (),"GFW boot did not complete" ))?; +// System memory page required for sysmembar to properly flush into system memory. +let sysmem_flush = { +let page =DmaObject ::new (pdev.as_ref (), kernel::bindings::PAGE_SIZE )?; + +// Register the sysmem flush page. +let handle = page.dma_handle (); + + regs::NV_PFB_NISO_FLUSH_SYSMEM_ADDR ::default () + .set_adr_39_08 ((handle >>8 )as u32 ) + .write (bar); +if spec.chipset >=Chipset ::GA102 { + regs::NV_PFB_NISO_FLUSH_SYSMEM_ADDR_HI ::default () + .set_adr_63_40 ((handle >>40 )as u32 ) + .write (bar); + } + + page + }; +Ok (pin_init! (Self { spec, bar: devres_bar, - fw + fw, + sysmem_flush, })) } }
drivers/gpu/nova-core/regs.rs +10 -0
@@ -39,6 +39,16 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> { } } +/* PFB */ + +register! (NV_PFB_NISO_FLUSH_SYSMEM_ADDR @0x00100c10 { +31 :0 adr_39_08as u32 ; +}); + +register! (NV_PFB_NISO_FLUSH_SYSMEM_ADDR_HI @0x00100c40 { +23 :0 adr_63_40as u32 ; +}); +/* PGC6 */ register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK @0x00118128 {
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 11/19] gpu: nova-core: register sysmem flush page
Quoted 1 line · Alexandre Courbot
A page of system memory is reserved so sysmembar can perform a read on
NIT: Use imperative mood.
Quoted 24 lines
it if a system write occurred since the last flush. Do this early as it can be required to e.g. reset the GPU falcons.
Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/gpu.rs | 45 +++++++++++++++++++++++++++++++++++++++++-- drivers/gpu/nova-core/regs.rs | 10 ++++++++++ 2 files changed, 53 insertions(+), 2 deletions(-)drivers/gpu/nova-core/gpu.rs +1
-0@@ -3,6 +3,7 @@use kernel::{device, devres::Devres , error::code::*, pci, prelude::*};use crate ::devinit; +use crate ::dma::DmaObject ;use crate ::driver::Bar0 ;use crate ::firmware::Firmware ;use crate ::regs;@@ -158,12 +159,32 @@ fn new(bar: &Bar0) -> Result<Spec> { }
drivers/gpu/nova-core/gpu.rs fn new(bar: &Bar0) -> Result<Spec> { · line 169
Quoted 10 lines
drivers/gpu/nova-core/gpu.rs +3
-1/// Structure holding the resources required to operate the GPU. -#[pin_data] +#[pin_data( PinnedDrop )] pub (crate )struct Gpu { spec:Spec ,/// MMIO mapping of PCI BAR 0 bar:Devres <Bar0 >, fw:Firmware , +// System memory page required for flushing all pending GPU-side memory writes done through +// PCIE into system memory.
Please make this a doc-comment.
Alexandre Courbot
maintainerPATCH v3 12/19 gpu: nova-core: add helper function to wait on condition
In reply to Alexandre CourbotWhile programming the hardware, we frequently need to busy-wait until a condition (like a given bit of a register to switch value) happens.
Add a basic wait_on helper function to wait on such conditions expressed as a closure, with a timeout argument.
This is temporary as we will switch to read_poll_timeout [1] once it is available.
[1] https://lore.kernel.org/lkml/[email protected]/
--- drivers/gpu/nova-core/util.rs | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)
drivers/gpu/nova-core/util.rs +30 -0
@@ -1,5 +1,10 @@// SPDX-License-Identifier: GPL-2.0 +use core::time::Duration ; + +use kernel::prelude::*; +use kernel::time::Ktime ; +pub (crate )const fn to_lowercase_bytes <const N :usize >(s: &str ) -> [u8 ;N ] {let src = s.as_bytes ();let mut dst = [0 ;N ];
@@ -19,3 +24,28 @@ pub(crate) const fn const_bytes_to_str(bytes: &[u8]) -> &str { Err(_) => kernel::build_error!("Bytes are not valid UTF-8."), } } + +/// Wait until `cond` is true or `timeout` elapsed. +/// +/// When `cond` evaluates to `Some`, its return value is returned. +/// +/// `Err(ETIMEDOUT)` is returned if `timeout` has been reached without `cond` evaluating to +/// `Some`. +/// +/// TODO: replace with `read_poll_timeout` once it is available. +/// (https://lore.kernel.org/lkml/[email protected]/) +#[expect(dead_code)] +pub (crate )fn wait_on <R ,F :Fn () ->Option <R >>(timeout:Duration , cond:F ) ->Result <R > { +let start_time =Ktime ::ktime_get (); + +loop { +if let Some (ret) =cond () { +return Ok (ret); + } + +let cur_time =Ktime ::ktime_get (); +if (cur_time - start_time).to_ns () > timeout.as_nanos ()as i64 { +return Err (ETIMEDOUT ); + } + } +}
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 12/19] gpu: nova-core: add helper function to wait on condition
Quoted 50 lines · Alexandre Courbot
While programming the hardware, we frequently need to busy-wait until a condition (like a given bit of a register to switch value) happens.
Add a basic
wait_onhelper function to wait on such conditions expressed as a closure, with a timeout argument.This is temporary as we will switch to
read_poll_timeout[1] once it is available.[1] https://lore.kernel.org/lkml/[email protected]/
Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/util.rs | 30 ++++++++++++++++++++++++++++++ 1 file changed, 30 insertions(+)drivers/gpu/nova-core/util.rs +20
-0@@ -1,5 +1,10 @@// SPDX-License-Identifier: GPL-2.0 +use core::time::Duration ; + +use kernel::prelude::*; +use kernel::time::Ktime ; +pub (crate )const fn to_lowercase_bytes <const N :usize >(s: &str ) -> [u8 ;N ] {let src = s.as_bytes ();let mut dst = [0 ;N ];@@ -19,3 +24,28 @@ pub(crate) const fn const_bytes_to_str(bytes: &[u8]) -> &str { Err(_) => kernel::build_error!("Bytes are not valid UTF-8."), } } + +/// Wait until `cond` is true or `timeout` elapsed. +/// +/// When `cond` evaluates to `Some`, its return value is returned. +/// +/// `Err(ETIMEDOUT)` is returned if `timeout` has been reached without `cond` evaluating to +/// `Some`. +/// +/// TODO: replace with `read_poll_timeout` once it is available. +/// (https://lore.kernel.org/lkml/[email protected]/) +#[expect(dead_code)] +pub (crate )fn wait_on <R ,F :Fn () ->Option <R >>(timeout:Duration , cond:F ) ->Result <R > { +let start_time =Ktime ::ktime_get (); + +loop {
drivers/gpu/nova-core/util.rs pub(crate) const fn const_bytes_to_str(bytes: &[u8]) -> &str { · line 51
Quoted 10 lines
drivers/gpu/nova-core/util.rs +10
-0+if let Some (ret) =cond () { +return Ok (ret); + } + +let cur_time =Ktime ::ktime_get (); +if (cur_time - start_time).to_ns () > timeout.as_nanos ()as i64 { +return Err (ETIMEDOUT ); + } + } +}
NIT: Can't you use this for wait_gfw_boot_completion() too?
Alexandre Courbot
maintainerPATCH v3 13/19 gpu: nova-core: add falcon register definitions and base code
In reply to Alexandre CourbotAdd the common Falcon code and HAL for Ampere GPUs, and instantiate the GSP and SEC2 Falcons that will be required to boot the GSP.
--- drivers/gpu/nova-core/falcon.rs | 540 ++++++++++++++++++++++++++++++ drivers/gpu/nova-core/falcon/gsp.rs | 22 ++ drivers/gpu/nova-core/falcon/hal.rs | 56 ++++ drivers/gpu/nova-core/falcon/hal/ga102.rs | 120 +++++++ drivers/gpu/nova-core/falcon/sec2.rs | 8 + drivers/gpu/nova-core/gpu.rs | 11 + drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 125 +++++++ drivers/gpu/nova-core/util.rs | 1 - 9 files changed, 883 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/falcon.rs new file +540 -0
@@ -0,0 +1,540 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Falcon microprocessor base support + +// To be removed when all code is used. +#![expect(dead_code)] + +use core::time::Duration ; +use hal::FalconHal ; +use kernel::bindings; +use kernel::device; +use kernel::prelude::*; +use kernel::sync::Arc ; +use kernel::types::ARef ; + +use crate ::driver::Bar0 ; +use crate ::gpu::Chipset ; +use crate ::regs; +use crate ::util; + +pub (crate )mod gsp; +mod hal; +pub (crate )mod sec2; + +/// Revision number of a falcon core, used in the [`crate::regs::NV_PFALCON_FALCON_HWCFG1`] +/// register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone , PartialEq , Eq , PartialOrd , Ord )] +pub (crate )enum FalconCoreRev { +#[default] +Rev1 =1 , +Rev2 =2 , +Rev3 =3 , +Rev4 =4 , +Rev5 =5 , +Rev6 =6 , +Rev7 =7 , +} + +impl TryFrom <u8 >for FalconCoreRev { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > { +use FalconCoreRev ::*; + +let rev =match value { +1 =>Rev1 , +2 =>Rev2 , +3 =>Rev3 , +4 =>Rev4 , +5 =>Rev5 , +6 =>Rev6 , +7 =>Rev7 , + _ =>return Err (EINVAL ), + }; + +Ok (rev) + } +} + +/// Revision subversion number of a falcon core, used in the +/// [`crate::regs::NV_PFALCON_FALCON_HWCFG1`] register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone , PartialEq , Eq , PartialOrd , Ord )] +pub (crate )enum FalconCoreRevSubversion { +#[default] +Subversion0 =0 , +Subversion1 =1 , +Subversion2 =2 , +Subversion3 =3 , +} + +impl TryFrom <u8 >for FalconCoreRevSubversion { +type Error =Error ; + +fn try_from (value:u8 ) ->Result <Self > { +use FalconCoreRevSubversion ::*; + +let sub_version =match value &0b11 { +0 =>Subversion0 , +1 =>Subversion1 , +2 =>Subversion2 , +3 =>Subversion3 , + _ =>return Err (EINVAL ), + }; + +Ok (sub_version) + } +} + +/// Security model of a falcon core, used in the [`crate::regs::NV_PFALCON_FALCON_HWCFG1`] +/// register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone )] +pub (crate )enum FalconSecurityModel { +/// Non-Secure: runs unsigned code without privileges. +#[default] +None =0 , +/// Low-secure: runs unsigned code with some privileges. Can only be entered from `Heavy` mode. +Light =2 , +/// High-Secure: runs signed code with full privileges. +Heavy =3 , +} + +impl TryFrom <u8 >for FalconSecurityModel { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > { +use FalconSecurityModel ::*; + +let sec_model =match value { +0 =>None , +2 =>Light , +3 =>Heavy , + _ =>return Err (EINVAL ), + }; + +Ok (sec_model) + } +} + +/// Signing algorithm for a given firmware, used in the [`crate::regs::NV_PFALCON2_FALCON_MOD_SEL`] +/// register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone , PartialEq , Eq )] +pub (crate )enum FalconModSelAlgo { +/// RSA3K. +#[default] +Rsa3k =1 , +} + +impl TryFrom <u8 >for FalconModSelAlgo { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > { +match value { +1 =>Ok (FalconModSelAlgo ::Rsa3k ), + _ =>Err (EINVAL ), + } + } +} + +/// Valid values for the `size` field of the [`crate::regs::NV_PFALCON_FALCON_DMATRFCMD`] register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone , PartialEq , Eq )] +pub (crate )enum DmaTrfCmdSize { +/// 256 bytes transfer. +#[default] +Size256B =0x6 , +} + +impl TryFrom <u8 >for DmaTrfCmdSize { +type Error =Error ; + +fn try_from (value:u8 ) ->Result <Self > { +match value { +0x6 =>Ok (Self ::Size256B ), + _ =>Err (EINVAL ), + } + } +} + +/// Currently active core on a dual falcon/riscv (Peregrine) controller. +#[derive( Debug , Clone , Copy , PartialEq , Eq )] +pub (crate )enum PeregrineCoreSelect { +/// Falcon core is active. +Falcon =0 , +/// RISC-V core is active. +Riscv =1 , +} + +impl From <bool >for PeregrineCoreSelect { +fn from (value:bool ) ->Self { +match value { +false =>PeregrineCoreSelect ::Falcon , +true =>PeregrineCoreSelect ::Riscv , + } + } +} + +/// Different types of memory present in a falcon core. +#[derive( Debug , Clone , Copy , PartialEq , Eq )] +pub (crate )enum FalconMem { +/// Instruction Memory. +Imem , +/// Data Memory. +Dmem , +} + +/// Target/source of a DMA transfer to/from falcon memory. +#[derive( Debug , Clone , Default )] +pub (crate )enum FalconFbifTarget { +/// VRAM. +#[default] +LocalFb =0 , +/// Coherent system memory. +CoherentSysmem =1 , +/// Non-coherent system memory. +NoncoherentSysmem =2 , +} + +impl TryFrom <u8 >for FalconFbifTarget { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > { +let res =match value { +0 =>Self ::LocalFb , +1 =>Self ::CoherentSysmem , +2 =>Self ::NoncoherentSysmem , + _ =>return Err (EINVAL ), + }; + +Ok (res) + } +} + +/// Type of memory addresses to use. +#[derive( Debug , Clone , Default )] +pub (crate )enum FalconFbifMemType { +/// Physical memory addresses. +#[default] +Virtual =0 , +/// Virtual memory addresses. +Physical =1 , +} + +impl From <bool >for FalconFbifMemType { +fn from (value:bool ) ->Self { +match value { +false =>Self ::Virtual , +true =>Self ::Physical , + } + } +} + +/// Trait defining the parameters of a given Falcon instance. +pub (crate )trait FalconEngine :Sync { +/// Base I/O address for the falcon, relative from which its registers are accessed. +const BASE :usize ; +} + +/// Represents a portion of the firmware to be loaded into a particular memory (e.g. IMEM or DMEM). +#[derive( Debug )] +pub (crate )struct FalconLoadTarget { +/// Offset from the start of the source object to copy from. +pub (crate ) src_start:u32 , +/// Offset from the start of the destination memory to copy into. +pub (crate ) dst_start:u32 , +/// Number of bytes to copy. +pub (crate ) len:u32 , +} + +/// Parameters for the falcon boot ROM. +#[derive( Debug )] +pub (crate )struct FalconBromParams { +/// Offset in `DMEM`` of the firmware's signature. +pub (crate ) pkc_data_offset:u32 , +/// Mask of engines valid for this firmware. +pub (crate ) engine_id_mask:u16 , +/// ID of the ucode used to infer a fuse register to validate the signature. +pub (crate ) ucode_id:u8 , +} + +/// Trait for a falcon firmware. +pub (crate )trait FalconFirmware { +/// Engine on which this firmware is to be loaded. +type Target :FalconEngine ; + +/// Returns the DMA handle of the object containing the firmware. +fn dma_handle (&self) -> bindings::dma_addr_t ; + +/// Returns the load parameters for `IMEM`. +fn imem_load (&self) ->FalconLoadTarget ; + +/// Returns the load parameters for `DMEM`. +fn dmem_load (&self) ->FalconLoadTarget ; + +/// Returns the parameters to write into the BROM registers. +fn brom_params (&self) ->FalconBromParams ; + +/// Returns the start address of the firmware. +fn boot_addr (&self) ->u32 ; +} + +/// Contains the base parameters common to all Falcon instances. +pub (crate )struct Falcon <E :FalconEngine > { + hal:Arc <dyn FalconHal <E >>, + dev:ARef <device::Device >, +} + +impl <E :FalconEngine + 'static>Falcon <E > { +/// Create a new falcon instance. +/// +/// `need_riscv` is set to `true` if the caller expects the falcon to be a dual falcon/riscv +/// controller. +pub (crate )fn new ( + dev: &device::Device , + chipset:Chipset , + bar: &Bar0 , + need_riscv:bool , + ) ->Result <Self > { +let hwcfg1 = regs::NV_PFALCON_FALCON_HWCFG1 ::read (bar,E ::BASE ); +// Ensure that the revision and security model contain valid values. +let _rev = hwcfg1.core_rev ()?; +let _sec_model = hwcfg1.security_model ()?; + +if need_riscv { +let hwcfg2 = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if !hwcfg2.riscv () { +dev_err! ( + dev, +"riscv support requested on a controller that does not support it\n" + ); +return Err (EINVAL ); + } + } + +Ok (Self { + hal: hal::create_falcon_hal (chipset)?, + dev: dev.into (), + }) + } + +/// Wait for memory scrubbing to complete. +fn reset_wait_mem_scrubbing (&self, bar: &Bar0 ) ->Result <()> { + util::wait_on (Duration ::from_millis (20 ), || { +let r = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if r.mem_scrubbing () { +Some (()) + }else { +None + } + }) + } + +/// Reset the falcon engine. +fn reset_eng (&self, bar: &Bar0 ) ->Result <()> { +let _ = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); + +// According to OpenRM's `kflcnPreResetWait_GA102` documentation, HW sometimes does not set +// RESET_READY so a non-failing timeout is used. +let _ = util::wait_on (Duration ::from_micros (150 ), || { +let r = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if r.reset_ready () { +Some (()) + }else { +None + } + }); + + regs::NV_PFALCON_FALCON_ENGINE ::alter (bar,E ::BASE , |v| v.set_reset (true )); + +let _:Result <()> = util::wait_on (Duration ::from_micros (10 ), ||None ); + + regs::NV_PFALCON_FALCON_ENGINE ::alter (bar,E ::BASE , |v| v.set_reset (false )); + + self.reset_wait_mem_scrubbing (bar)?; + +Ok (()) + } + +/// Reset the controller, select the falcon core, and wait for memory scrubbing to complete. +pub (crate )fn reset (&self, bar: &Bar0 ) ->Result <()> { + self.reset_eng (bar)?; + self.hal.select_core (self, bar)?; + self.reset_wait_mem_scrubbing (bar)?; + + regs::NV_PFALCON_FALCON_RM ::default () + .set_value (regs::NV_PMC_BOOT_0 ::read (bar).into ()) + .write (bar,E ::BASE ); + +Ok (()) + } + +/// Perform a DMA write according to `load_offsets` from `dma_handle` into the falcon's +/// `target_mem`. +/// +/// `sec` is set if the loaded firmware is expected to run in secure mode. +fn dma_wr ( + &self, + bar: &Bar0 , + dma_handle: bindings::dma_addr_t , + target_mem:FalconMem , + load_offsets:FalconLoadTarget , + sec:bool , + ) ->Result <()> { +const DMA_LEN :u32 =256 ; + +// For IMEM, we want to use the start offset as a virtual address tag for each page, since +// code addresses in the firmware (and the boot vector) are virtual. +// +// For DMEM we can fold the start offset into the DMA handle. +let (src_start, dma_start) =match target_mem { +FalconMem ::Imem => (load_offsets.src_start, dma_handle), +FalconMem ::Dmem => ( +0 , + dma_handle + load_offsets.src_startas bindings::dma_addr_t , + ), + }; +if dma_start %DMA_LEN as bindings::dma_addr_t >0 { +dev_err! ( + self.dev, +"DMA transfer start addresses must be a multiple of {}" , +DMA_LEN + ); +return Err (EINVAL ); + } +if load_offsets.len %DMA_LEN >0 { +dev_err! ( + self.dev, +"DMA transfer length must be a multiple of {}" , +DMA_LEN + ); +return Err (EINVAL ); + } + +// Set up the base source DMA address. + + regs::NV_PFALCON_FALCON_DMATRFBASE ::default () + .set_base ((dma_start >>8 )as u32 ) + .write (bar,E ::BASE ); + regs::NV_PFALCON_FALCON_DMATRFBASE1 ::default () + .set_base ((dma_start >>40 )as u16 ) + .write (bar,E ::BASE ); + +let cmd = regs::NV_PFALCON_FALCON_DMATRFCMD ::default () + .set_size (DmaTrfCmdSize ::Size256B ) + .set_imem (target_mem ==FalconMem ::Imem ) + .set_sec (if sec {1 }else {0 }); + +for posin (0 ..load_offsets.len).step_by (DMA_LEN as usize ) { +// Perform a transfer of size `DMA_LEN`. + regs::NV_PFALCON_FALCON_DMATRFMOFFS ::default () + .set_offs (load_offsets.dst_start + pos) + .write (bar,E ::BASE ); + regs::NV_PFALCON_FALCON_DMATRFFBOFFS ::default () + .set_offs (src_start + pos) + .write (bar,E ::BASE ); + cmd.write (bar,E ::BASE ); + +// Wait for the transfer to complete. + util::wait_on (Duration ::from_millis (2000 ), || { +let r = regs::NV_PFALCON_FALCON_DMATRFCMD ::read (bar,E ::BASE ); +if r.idle () { +Some (()) + }else { +None + } + })?; + } + +Ok (()) + } + +/// Perform a DMA load into `IMEM` and `DMEM` of `fw`, and prepare the falcon to run it. +pub (crate )fn dma_load <F :FalconFirmware <Target =E >>(&self, bar: &Bar0 , fw: &F ) ->Result <()> { +let dma_handle = fw.dma_handle (); + + regs::NV_PFALCON_FBIF_CTL ::alter (bar,E ::BASE , |v| v.set_allow_phys_no_ctx (true )); + regs::NV_PFALCON_FALCON_DMACTL ::default ().write (bar,E ::BASE ); + regs::NV_PFALCON_FBIF_TRANSCFG ::alter (bar,E ::BASE , |v| { + v.set_target (FalconFbifTarget ::CoherentSysmem ) + .set_mem_type (FalconFbifMemType ::Physical ) + }); + + self.dma_wr (bar, dma_handle,FalconMem ::Imem , fw.imem_load (),true )?; + self.dma_wr (bar, dma_handle,FalconMem ::Dmem , fw.dmem_load (),true )?; + + self.hal.program_brom (self, bar, &fw.brom_params ())?; + +// Set `BootVec` to start of non-secure code. + regs::NV_PFALCON_FALCON_BOOTVEC ::default () + .set_value (fw.boot_addr ()) + .write (bar,E ::BASE ); + +Ok (()) + } + +/// Start running the loaded firmware. +/// +/// `mbox0` and `mbox1` are optional parameters to write into the `MBOX0` and `MBOX1` registers +/// prior to running. +/// +/// Returns `MBOX0` and `MBOX1` after the firmware has stopped running. +pub (crate )fn boot ( + &self, + bar: &Bar0 , + mbox0:Option <u32 >, + mbox1:Option <u32 >, + ) ->Result <(u32 ,u32 )> { +if let Some (mbox0) = mbox0 { + regs::NV_PFALCON_FALCON_MAILBOX0 ::default () + .set_value (mbox0) + .write (bar,E ::BASE ); + } + +if let Some (mbox1) = mbox1 { + regs::NV_PFALCON_FALCON_MAILBOX1 ::default () + .set_value (mbox1) + .write (bar,E ::BASE ); + } + +match regs::NV_PFALCON_FALCON_CPUCTL ::read (bar,E ::BASE ).alias_en () { +true => regs::NV_PFALCON_FALCON_CPUCTL_ALIAS ::default () + .set_startcpu (true ) + .write (bar,E ::BASE ), +false => regs::NV_PFALCON_FALCON_CPUCTL ::default () + .set_startcpu (true ) + .write (bar,E ::BASE ), + } + + util::wait_on (Duration ::from_secs (2 ), || { +let r = regs::NV_PFALCON_FALCON_CPUCTL ::read (bar,E ::BASE ); +if r.halted () { +Some (()) + }else { +None + } + })?; + +let (mbox0, mbox1) = ( + regs::NV_PFALCON_FALCON_MAILBOX0 ::read (bar,E ::BASE ).value (), + regs::NV_PFALCON_FALCON_MAILBOX1 ::read (bar,E ::BASE ).value (), + ); + +Ok ((mbox0, mbox1)) + } + +/// Returns the fused version of the signature to use in order to run a HS firmware on this +/// falcon instance. `engine_id_mask` and `ucode_id` are obtained from the firmware header. +pub (crate )fn get_signature_reg_fuse_version ( + &self, + bar: &Bar0 , + engine_id_mask:u16 , + ucode_id:u8 , + ) ->Result <u32 > { + self.hal + .get_signature_reg_fuse_version (self, bar, engine_id_mask, ucode_id) + } +}
drivers/gpu/nova-core/falcon/gsp.rs new file +22 -0
@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0 + +use crate ::{ + driver::Bar0 , + falcon::{Falcon ,FalconEngine }, + regs, +}; + +pub (crate )struct Gsp ; +impl FalconEngine for Gsp { +const BASE :usize =0x00110000 ; +} + +impl Falcon <Gsp > { +/// Clears the SWGEN0 bit in the Falcon's IRQ status clear register to +/// allow GSP to signal CPU for processing new messages in message queue. +pub (crate )fn clear_swgen0_intr (&self, bar: &Bar0 ) { + regs::NV_PFALCON_FALCON_IRQSCLR ::default () + .set_swgen0 (true ) + .write (bar,Gsp ::BASE ); + } +}
drivers/gpu/nova-core/falcon/hal.rs new file +56 -0
56 lines not on this page · raw message
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +120 -0
120 lines not on this page · raw message
drivers/gpu/nova-core/falcon/sec2.rs new file +8 -0
@@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0 + +use crate ::falcon::FalconEngine ; + +pub (crate )struct Sec2 ; +impl FalconEngine for Sec2 { +const BASE :usize =0x00840000 ; +}
drivers/gpu/nova-core/gpu.rs +11 -0
@@ -5,6 +5,7 @@use crate ::devinit;use crate ::dma::DmaObject ;use crate ::driver::Bar0 ; +use crate ::falcon::{gsp::Gsp , sec2::Sec2 ,Falcon };use crate ::firmware::Firmware ;use crate ::regs;use crate ::util;
@@ -227,6 +228,16 @@ pub(crate) fn new( page }; +let gsp_falcon =Falcon ::<Gsp >::new ( + pdev.as_ref (), + spec.chipset, + bar, + spec.chipset >Chipset ::GA100 , + )?; + gsp_falcon.clear_swgen0_intr (bar); + +let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?; + Ok(pin_init!(Self { spec, bar: devres_bar,
drivers/gpu/nova-core/nova_core.rs +1 -0
@@ -5,6 +5,7 @@mod devinit;mod dma;mod driver; +mod falcon;mod firmware;mod gpu;mod regs;
drivers/gpu/nova-core/regs.rs +125 -0
134 lines not on this page · raw message
drivers/gpu/nova-core/util.rs +0 -1
@@ -34,7 +34,6 @@ pub(crate) const fn const_bytes_to_str(bytes: &[u8]) -> &str {/// /// TODO: replace with `read_poll_timeout` once it is available. /// (https://lore.kernel.org/lkml/[email protected]/) -#[expect(dead_code)] pub (crate )fn wait_on <R ,F :Fn () ->Option <R >>(timeout:Duration , cond:F ) ->Result <R > {let start_time =Ktime ::ktime_get ();
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 13/19] gpu: nova-core: add falcon register definitions and base code
Quoted 55 lines · Alexandre Courbot
Add the common Falcon code and HAL for Ampere GPUs, and instantiate the GSP and SEC2 Falcons that will be required to boot the GSP.
Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/falcon.rs | 540 ++++++++++++++++++++++++++++++ drivers/gpu/nova-core/falcon/gsp.rs | 22 ++ drivers/gpu/nova-core/falcon/hal.rs | 56 ++++ drivers/gpu/nova-core/falcon/hal/ga102.rs | 120 +++++++ drivers/gpu/nova-core/falcon/sec2.rs | 8 + drivers/gpu/nova-core/gpu.rs | 11 + drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 125 +++++++ drivers/gpu/nova-core/util.rs | 1 - 9 files changed, 883 insertions(+), 1 deletion(-)drivers/gpu/nova-core/falcon.rs new file +33
-0@@ -0,0 +1,540 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Falcon microprocessor base support + +// To be removed when all code is used. +#![expect(dead_code)] + +use core::time::Duration ; +use hal::FalconHal ; +use kernel::bindings; +use kernel::device; +use kernel::prelude::*; +use kernel::sync::Arc ; +use kernel::types::ARef ; + +use crate ::driver::Bar0 ; +use crate ::gpu::Chipset ; +use crate ::regs; +use crate ::util; + +pub (crate )mod gsp; +mod hal; +pub (crate )mod sec2; + +/// Revision number of a falcon core, used in the [`crate::regs::NV_PFALCON_FALCON_HWCFG1`] +/// register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone , PartialEq , Eq , PartialOrd , Ord )] +pub (crate )enum FalconCoreRev { +#[default] +Rev1 =1 , +Rev2 =2 , +Rev3 =3 ,
drivers/gpu/nova-core/falcon.rs line 43
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+ Rev4 = 4, + Rev5 = 5, + Rev6 = 6, + Rev7 = 7, +} + +impl TryFrom <u8 >for FalconCoreRev { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > {
Here and below, please use Result<T>, which comes from kernel::prelude.
Quoted 171 lines
drivers/gpu/nova-core/falcon.rs new file +171
-0+use FalconCoreRev ::*; + +let rev =match value { +1 =>Rev1 , +2 =>Rev2 , +3 =>Rev3 , +4 =>Rev4 , +5 =>Rev5 , +6 =>Rev6 , +7 =>Rev7 , + _ =>return Err (EINVAL ), + }; + +Ok (rev) + } +} + +/// Revision subversion number of a falcon core, used in the +/// [`crate::regs::NV_PFALCON_FALCON_HWCFG1`] register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone , PartialEq , Eq , PartialOrd , Ord )] +pub (crate )enum FalconCoreRevSubversion { +#[default] +Subversion0 =0 , +Subversion1 =1 , +Subversion2 =2 , +Subversion3 =3 , +} + +impl TryFrom <u8 >for FalconCoreRevSubversion { +type Error =Error ; + +fn try_from (value:u8 ) ->Result <Self > { +use FalconCoreRevSubversion ::*; + +let sub_version =match value &0b11 { +0 =>Subversion0 , +1 =>Subversion1 , +2 =>Subversion2 , +3 =>Subversion3 , + _ =>return Err (EINVAL ), + }; + +Ok (sub_version) + } +} + +/// Security model of a falcon core, used in the [`crate::regs::NV_PFALCON_FALCON_HWCFG1`] +/// register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone )] +pub (crate )enum FalconSecurityModel { +/// Non-Secure: runs unsigned code without privileges. +#[default] +None =0 , +/// Low-secure: runs unsigned code with some privileges. Can only be entered from `Heavy` mode. +Light =2 , +/// High-Secure: runs signed code with full privileges. +Heavy =3 , +} + +impl TryFrom <u8 >for FalconSecurityModel { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > { +use FalconSecurityModel ::*; + +let sec_model =match value { +0 =>None , +2 =>Light , +3 =>Heavy , + _ =>return Err (EINVAL ), + }; + +Ok (sec_model) + } +} + +/// Signing algorithm for a given firmware, used in the [`crate::regs::NV_PFALCON2_FALCON_MOD_SEL`] +/// register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone , PartialEq , Eq )] +pub (crate )enum FalconModSelAlgo { +/// RSA3K. +#[default] +Rsa3k =1 , +} + +impl TryFrom <u8 >for FalconModSelAlgo { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > { +match value { +1 =>Ok (FalconModSelAlgo ::Rsa3k ), + _ =>Err (EINVAL ), + } + } +} + +/// Valid values for the `size` field of the [`crate::regs::NV_PFALCON_FALCON_DMATRFCMD`] register. +#[repr( u8 )] +#[derive( Debug , Default , Copy , Clone , PartialEq , Eq )] +pub (crate )enum DmaTrfCmdSize { +/// 256 bytes transfer. +#[default] +Size256B =0x6 , +} + +impl TryFrom <u8 >for DmaTrfCmdSize { +type Error =Error ; + +fn try_from (value:u8 ) ->Result <Self > { +match value { +0x6 =>Ok (Self ::Size256B ), + _ =>Err (EINVAL ), + } + } +} + +/// Currently active core on a dual falcon/riscv (Peregrine) controller. +#[derive( Debug , Clone , Copy , PartialEq , Eq )] +pub (crate )enum PeregrineCoreSelect { +/// Falcon core is active. +Falcon =0 , +/// RISC-V core is active. +Riscv =1 , +} + +impl From <bool >for PeregrineCoreSelect { +fn from (value:bool ) ->Self { +match value { +false =>PeregrineCoreSelect ::Falcon , +true =>PeregrineCoreSelect ::Riscv , + } + } +} + +/// Different types of memory present in a falcon core. +#[derive( Debug , Clone , Copy , PartialEq , Eq )] +pub (crate )enum FalconMem { +/// Instruction Memory. +Imem , +/// Data Memory. +Dmem , +} + +/// Target/source of a DMA transfer to/from falcon memory. +#[derive( Debug , Clone , Default )] +pub (crate )enum FalconFbifTarget { +/// VRAM. +#[default] +LocalFb =0 , +/// Coherent system memory. +CoherentSysmem =1 , +/// Non-coherent system memory. +NoncoherentSysmem =2 , +} + +impl TryFrom <u8 >for FalconFbifTarget { +type Error =Error ; + +fn try_from (value:u8 ) -> core::result::Result <Self ,Self ::Error > { +let res =match value { +0 =>Self ::LocalFb , +1 =>Self ::CoherentSysmem , +2 =>Self ::NoncoherentSysmem , + _ =>return Err (EINVAL ), + }; + +Ok (res) + }
drivers/gpu/nova-core/falcon.rs line 224
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+} + +/// Type of memory addresses to use. +#[derive( Debug , Clone , Default )] +pub (crate )enum FalconFbifMemType { +/// Physical memory addresses. +#[default] +Virtual =0 , +/// Virtual memory addresses. +Physical =1 ,
NIT: Looks like the doc-comments are the wrong way around.
drivers/gpu/nova-core/falcon.rs line 234
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+} + +impl From <bool >for FalconFbifMemType { +fn from (value:bool ) ->Self { +match value { +false =>Self ::Virtual , +true =>Self ::Physical , + } + } +}
This conversion seems a bit odd; how is false a canonical value to convert to FalconFbifMemType::Virtual and true for FalconFbifMemType:::Physical?
Oh, I see. It comes from a register field of a single bit. Maybe it's worth adding a brief comment.
Quoted 43 lines
drivers/gpu/nova-core/falcon.rs new file +43
-0+ +/// Trait defining the parameters of a given Falcon instance. +pub (crate )trait FalconEngine :Sync { +/// Base I/O address for the falcon, relative from which its registers are accessed. +const BASE :usize ; +} + +/// Represents a portion of the firmware to be loaded into a particular memory (e.g. IMEM or DMEM). +#[derive( Debug )] +pub (crate )struct FalconLoadTarget { +/// Offset from the start of the source object to copy from. +pub (crate ) src_start:u32 , +/// Offset from the start of the destination memory to copy into. +pub (crate ) dst_start:u32 , +/// Number of bytes to copy. +pub (crate ) len:u32 , +} + +/// Parameters for the falcon boot ROM. +#[derive( Debug )] +pub (crate )struct FalconBromParams { +/// Offset in `DMEM`` of the firmware's signature. +pub (crate ) pkc_data_offset:u32 , +/// Mask of engines valid for this firmware. +pub (crate ) engine_id_mask:u16 , +/// ID of the ucode used to infer a fuse register to validate the signature. +pub (crate ) ucode_id:u8 , +} + +/// Trait for a falcon firmware. +pub (crate )trait FalconFirmware { +/// Engine on which this firmware is to be loaded. +type Target :FalconEngine ; + +/// Returns the DMA handle of the object containing the firmware. +fn dma_handle (&self) -> bindings::dma_addr_t ; + +/// Returns the load parameters for `IMEM`. +fn imem_load (&self) ->FalconLoadTarget ; + +/// Returns the load parameters for `DMEM`. +fn dmem_load (&self) ->FalconLoadTarget ; +
drivers/gpu/nova-core/falcon.rs line 287
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+/// Returns the parameters to write into the BROM registers. +fn brom_params (&self) ->FalconBromParams ; + +/// Returns the start address of the firmware. +fn boot_addr (&self) ->u32 ; +} + +/// Contains the base parameters common to all Falcon instances. +pub (crate )struct Falcon <E :FalconEngine > { + hal:Arc <dyn FalconHal <E >>,
TODO: Convert to Box, once Box supports type coercion?
Nevermind, you have the comment in create_falcon_hal().
Quoted 8 lines
drivers/gpu/nova-core/falcon.rs new file +8
-0+ dev: ARef<device::Device>, +} + +impl <E :FalconEngine + 'static>Falcon <E > { +/// Create a new falcon instance. +/// +/// `need_riscv` is set to `true` if the caller expects the falcon to be a dual falcon/riscv +/// controller.
drivers/gpu/nova-core/falcon.rs line 305
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+pub (crate )fn new ( + dev: &device::Device , + chipset:Chipset , + bar: &Bar0 , + need_riscv:bool , + ) ->Result <Self > { +let hwcfg1 = regs::NV_PFALCON_FALCON_HWCFG1 ::read (bar,E ::BASE ); +// Ensure that the revision and security model contain valid values. +let _rev = hwcfg1.core_rev ()?; +let _sec_model = hwcfg1.security_model ()?;
Why not just _?
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+ +if need_riscv { +let hwcfg2 = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if !hwcfg2.riscv () { +dev_err! ( + dev, +"riscv support requested on a controller that does not support it\n" + ); +return Err (EINVAL ); + }
drivers/gpu/nova-core/falcon.rs line 325
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+ } + +Ok (Self { + hal: hal::create_falcon_hal (chipset)?, + dev: dev.into (), + }) + } + +/// Wait for memory scrubbing to complete. +fn reset_wait_mem_scrubbing (&self, bar: &Bar0 ) ->Result <()> {
Here and below, please use Result instead of Result<()>.
Quoted 6 lines
drivers/gpu/nova-core/falcon.rs new file +6
-0+ util::wait_on (Duration ::from_millis (20 ), || { +let r = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if r.mem_scrubbing () { +Some (()) + }else { +None
drivers/gpu/nova-core/falcon.rs line 341
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+ } + }) + } + +/// Reset the falcon engine. +fn reset_eng (&self, bar: &Bar0 ) ->Result <()> { +let _ = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); + +// According to OpenRM's `kflcnPreResetWait_GA102` documentation, HW sometimes does not set +// RESET_READY so a non-failing timeout is used.
Should we still warn about it?
drivers/gpu/nova-core/falcon.rs line 342
Quoted 1 line
drivers/gpu/nova-core/falcon.rs new file +1
-0+ let _ = util::wait_on(Duration::from_micros(150), || {
Do we know for sure that if RESET_READY is not set after 150us, it won't ever be set? If the answer to that is yes, and we also do not want to warn about RESET_READY not being set, why even bother trying to read it in the first place?
drivers/gpu/nova-core/falcon.rs line 353
Quoted 11 lines
drivers/gpu/nova-core/falcon.rs new file +11
-0+let r = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if r.reset_ready () { +Some (()) + }else { +None + } + }); + + regs::NV_PFALCON_FALCON_ENGINE ::alter (bar,E ::BASE , |v| v.set_reset (true )); + +let _:Result <()> = util::wait_on (Duration ::from_micros (10 ), ||None );
Can we please get an abstraction for udelay() for this?
Quoted 322 lines
drivers/gpu/nova-core/falcon.rs new file +187
-0+ + regs::NV_PFALCON_FALCON_ENGINE ::alter (bar,E ::BASE , |v| v.set_reset (false )); + + self.reset_wait_mem_scrubbing (bar)?; + +Ok (()) + } + +/// Reset the controller, select the falcon core, and wait for memory scrubbing to complete. +pub (crate )fn reset (&self, bar: &Bar0 ) ->Result <()> { + self.reset_eng (bar)?; + self.hal.select_core (self, bar)?; + self.reset_wait_mem_scrubbing (bar)?; + + regs::NV_PFALCON_FALCON_RM ::default () + .set_value (regs::NV_PMC_BOOT_0 ::read (bar).into ()) + .write (bar,E ::BASE ); + +Ok (()) + } + +/// Perform a DMA write according to `load_offsets` from `dma_handle` into the falcon's +/// `target_mem`. +/// +/// `sec` is set if the loaded firmware is expected to run in secure mode. +fn dma_wr ( + &self, + bar: &Bar0 , + dma_handle: bindings::dma_addr_t , + target_mem:FalconMem , + load_offsets:FalconLoadTarget , + sec:bool , + ) ->Result <()> { +const DMA_LEN :u32 =256 ; + +// For IMEM, we want to use the start offset as a virtual address tag for each page, since +// code addresses in the firmware (and the boot vector) are virtual. +// +// For DMEM we can fold the start offset into the DMA handle. +let (src_start, dma_start) =match target_mem { +FalconMem ::Imem => (load_offsets.src_start, dma_handle), +FalconMem ::Dmem => ( +0 , + dma_handle + load_offsets.src_startas bindings::dma_addr_t , + ), + }; +if dma_start %DMA_LEN as bindings::dma_addr_t >0 { +dev_err! ( + self.dev, +"DMA transfer start addresses must be a multiple of {}" , +DMA_LEN + ); +return Err (EINVAL ); + } +if load_offsets.len %DMA_LEN >0 { +dev_err! ( + self.dev, +"DMA transfer length must be a multiple of {}" , +DMA_LEN + ); +return Err (EINVAL ); + } + +// Set up the base source DMA address. + + regs::NV_PFALCON_FALCON_DMATRFBASE ::default () + .set_base ((dma_start >>8 )as u32 ) + .write (bar,E ::BASE ); + regs::NV_PFALCON_FALCON_DMATRFBASE1 ::default () + .set_base ((dma_start >>40 )as u16 ) + .write (bar,E ::BASE ); + +let cmd = regs::NV_PFALCON_FALCON_DMATRFCMD ::default () + .set_size (DmaTrfCmdSize ::Size256B ) + .set_imem (target_mem ==FalconMem ::Imem ) + .set_sec (if sec {1 }else {0 }); + +for posin (0 ..load_offsets.len).step_by (DMA_LEN as usize ) { +// Perform a transfer of size `DMA_LEN`. + regs::NV_PFALCON_FALCON_DMATRFMOFFS ::default () + .set_offs (load_offsets.dst_start + pos) + .write (bar,E ::BASE ); + regs::NV_PFALCON_FALCON_DMATRFFBOFFS ::default () + .set_offs (src_start + pos) + .write (bar,E ::BASE ); + cmd.write (bar,E ::BASE ); + +// Wait for the transfer to complete. + util::wait_on (Duration ::from_millis (2000 ), || { +let r = regs::NV_PFALCON_FALCON_DMATRFCMD ::read (bar,E ::BASE ); +if r.idle () { +Some (()) + }else { +None + } + })?; + } + +Ok (()) + } + +/// Perform a DMA load into `IMEM` and `DMEM` of `fw`, and prepare the falcon to run it. +pub (crate )fn dma_load <F :FalconFirmware <Target =E >>(&self, bar: &Bar0 , fw: &F ) ->Result <()> { +let dma_handle = fw.dma_handle (); + + regs::NV_PFALCON_FBIF_CTL ::alter (bar,E ::BASE , |v| v.set_allow_phys_no_ctx (true )); + regs::NV_PFALCON_FALCON_DMACTL ::default ().write (bar,E ::BASE ); + regs::NV_PFALCON_FBIF_TRANSCFG ::alter (bar,E ::BASE , |v| { + v.set_target (FalconFbifTarget ::CoherentSysmem ) + .set_mem_type (FalconFbifMemType ::Physical ) + }); + + self.dma_wr (bar, dma_handle,FalconMem ::Imem , fw.imem_load (),true )?; + self.dma_wr (bar, dma_handle,FalconMem ::Dmem , fw.dmem_load (),true )?; + + self.hal.program_brom (self, bar, &fw.brom_params ())?; + +// Set `BootVec` to start of non-secure code. + regs::NV_PFALCON_FALCON_BOOTVEC ::default () + .set_value (fw.boot_addr ()) + .write (bar,E ::BASE ); + +Ok (()) + } + +/// Start running the loaded firmware. +/// +/// `mbox0` and `mbox1` are optional parameters to write into the `MBOX0` and `MBOX1` registers +/// prior to running. +/// +/// Returns `MBOX0` and `MBOX1` after the firmware has stopped running. +pub (crate )fn boot ( + &self, + bar: &Bar0 , + mbox0:Option <u32 >, + mbox1:Option <u32 >, + ) ->Result <(u32 ,u32 )> { +if let Some (mbox0) = mbox0 { + regs::NV_PFALCON_FALCON_MAILBOX0 ::default () + .set_value (mbox0) + .write (bar,E ::BASE ); + } + +if let Some (mbox1) = mbox1 { + regs::NV_PFALCON_FALCON_MAILBOX1 ::default () + .set_value (mbox1) + .write (bar,E ::BASE ); + } + +match regs::NV_PFALCON_FALCON_CPUCTL ::read (bar,E ::BASE ).alias_en () { +true => regs::NV_PFALCON_FALCON_CPUCTL_ALIAS ::default () + .set_startcpu (true ) + .write (bar,E ::BASE ), +false => regs::NV_PFALCON_FALCON_CPUCTL ::default () + .set_startcpu (true ) + .write (bar,E ::BASE ), + } + + util::wait_on (Duration ::from_secs (2 ), || { +let r = regs::NV_PFALCON_FALCON_CPUCTL ::read (bar,E ::BASE ); +if r.halted () { +Some (()) + }else { +None + } + })?; + +let (mbox0, mbox1) = ( + regs::NV_PFALCON_FALCON_MAILBOX0 ::read (bar,E ::BASE ).value (), + regs::NV_PFALCON_FALCON_MAILBOX1 ::read (bar,E ::BASE ).value (), + ); + +Ok ((mbox0, mbox1)) + } + +/// Returns the fused version of the signature to use in order to run a HS firmware on this +/// falcon instance. `engine_id_mask` and `ucode_id` are obtained from the firmware header. +pub (crate )fn get_signature_reg_fuse_version ( + &self, + bar: &Bar0 , + engine_id_mask:u16 , + ucode_id:u8 , + ) ->Result <u32 > { + self.hal + .get_signature_reg_fuse_version (self, bar, engine_id_mask, ucode_id) + } +}drivers/gpu/nova-core/falcon/gsp.rs new file +22
-0@@ -0,0 +1,22 @@ +// SPDX-License-Identifier: GPL-2.0 + +use crate ::{ + driver::Bar0 , + falcon::{Falcon ,FalconEngine }, + regs, +}; + +pub (crate )struct Gsp ; +impl FalconEngine for Gsp { +const BASE :usize =0x00110000 ; +} + +impl Falcon <Gsp > { +/// Clears the SWGEN0 bit in the Falcon's IRQ status clear register to +/// allow GSP to signal CPU for processing new messages in message queue. +pub (crate )fn clear_swgen0_intr (&self, bar: &Bar0 ) { + regs::NV_PFALCON_FALCON_IRQSCLR ::default () + .set_swgen0 (true ) + .write (bar,Gsp ::BASE ); + } +}drivers/gpu/nova-core/falcon/hal.rs new file +56
-0@@ -0,0 +1,56 @@ +// SPDX-License-Identifier: GPL-2.0 + +use kernel::prelude::*; +use kernel::sync::Arc ; + +use crate ::driver::Bar0 ; +use crate ::falcon::{Falcon ,FalconBromParams ,FalconEngine }; +use crate ::gpu::Chipset ; + +mod ga102; + +/// Hardware Abstraction Layer for Falcon cores. +/// +/// Implements chipset-specific low-level operations. The trait is generic against [`FalconEngine`] +/// so its `BASE` parameter can be used in order to avoid runtime bound checks when accessing +/// registers. +pub (crate )trait FalconHal <E :FalconEngine >:Sync { +// Activates the Falcon core if the engine is a risvc/falcon dual engine. +fn select_core (&self, _falcon: &Falcon <E >, _bar: &Bar0 ) ->Result <()> { +Ok (()) + } + +/// Returns the fused version of the signature to use in order to run a HS firmware on this +/// falcon instance. `engine_id_mask` and `ucode_id` are obtained from the firmware header. +fn get_signature_reg_fuse_version ( + &self, + falcon: &Falcon <E >, + bar: &Bar0 , + engine_id_mask:u16 , + ucode_id:u8 , + ) ->Result <u32 >; + +// Program the boot ROM registers prior to starting a secure firmware. +fn program_brom (&self, falcon: &Falcon <E >, bar: &Bar0 , params: &FalconBromParams ) + ->Result <()>; +} + +/// Returns a boxed falcon HAL adequate for the passed `chipset`. +/// +/// We use this function and a heap-allocated trait object instead of statically defined trait +/// objects because of the two-dimensional (Chipset, Engine) lookup required to return the +/// requested HAL. +/// +/// TODO: replace the return type with `KBox` once it gains the ability to host trait objects. +pub (crate )fn create_falcon_hal <E :FalconEngine + 'static>( + chipset:Chipset , +) ->Result <Arc <dyn FalconHal <E >>> { +let hal =match chipset { +Chipset ::GA102 |Chipset ::GA103 |Chipset ::GA104 |Chipset ::GA106 |Chipset ::GA107 => { +Arc ::new (ga102::Ga102 ::<E >::new (),GFP_KERNEL )?as Arc <dyn FalconHal <E >> + } + _ =>return Err (ENOTSUPP ), + }; + +Ok (hal) +}drivers/gpu/nova-core/falcon/hal/ga102.rs new file +39
-0@@ -0,0 +1,120 @@ +// SPDX-License-Identifier: GPL-2.0 + +use core::marker::PhantomData ; +use core::time::Duration ; + +use kernel::{device, prelude::*}; + +use crate ::driver::Bar0 ; +use crate ::falcon::{ +Falcon ,FalconBromParams ,FalconEngine ,FalconModSelAlgo ,PeregrineCoreSelect , +}; +use crate ::regs; +use crate ::util; + +use super ::FalconHal ; + +fn select_core_ga102 <E :FalconEngine >(bar: &Bar0 ) ->Result <()> { +let bcr_ctrl = regs::NV_PRISCV_RISCV_BCR_CTRL ::read (bar,E ::BASE ); +if bcr_ctrl.core_select () !=PeregrineCoreSelect ::Falcon { + regs::NV_PRISCV_RISCV_BCR_CTRL ::default () + .set_core_select (PeregrineCoreSelect ::Falcon ) + .write (bar,E ::BASE ); + + util::wait_on (Duration ::from_millis (10 ), || { +let r = regs::NV_PRISCV_RISCV_BCR_CTRL ::read (bar,E ::BASE ); +if r.valid () { +Some (()) + }else { +None + } + })?; + } + +Ok (()) +} + +fn get_signature_reg_fuse_version_ga102 ( + dev: &device::Device , + bar: &Bar0,
drivers/gpu/nova-core/falcon/hal/ga102.rs line 49
Quoted 10 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +10
-0+ engine_id_mask: u16, + ucode_id: u8, +) -> Result<u32> { +// The ucode fuse versions are contained in the FUSE_OPT_FPF_<ENGINE>_UCODE<X>_VERSION +// registers, which are an array. Our register definition macros do not allow us to manage them +// properly, so we need to hardcode their addresses for now. + +// Each engine has 16 ucode version registers numbered from 1 to 16. +if ucode_id ==0 || ucode_id >16 { +dev_warn! (dev,"invalid ucode id {:#x}" , ucode_id);
Given that this is an error condition, this should be dev_err!() I suppose.
drivers/gpu/nova-core/falcon/hal/ga102.rs line 62
Quoted 13 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +13
-0+return Err (EINVAL ); + } +let reg_fuse =if engine_id_mask &0x0001 !=0 { +// NV_FUSE_OPT_FPF_SEC2_UCODE1_VERSION +0x824140 + }else if engine_id_mask &0x0004 !=0 { +// NV_FUSE_OPT_FPF_NVDEC_UCODE1_VERSION +0x824100 + }else if engine_id_mask &0x0400 !=0 { +// NV_FUSE_OPT_FPF_GSP_UCODE1_VERSION +0x8241c0 + }else { +dev_warn! (dev,"unexpected engine_id_mask {:#x}" , engine_id_mask);
s/dev_warn/dev_err/
drivers/gpu/nova-core/falcon/hal/ga102.rs line 64
Quoted 2 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +2
-0+return Err (EINVAL ); + } + ((ucode_id -1 )as usize * core::mem::size_of ::<u32 >());
I feel like this calculation deserves a comment.
drivers/gpu/nova-core/falcon/hal/ga102.rs line 66
Quoted 2 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +2
-0+ +let reg_fuse_version = bar.read32 (reg_fuse);
I feel like the calculation of reg_fuse should be abstracted with a dedicated type in regs.rs. that takes the magic number derived from the engine_id_mask (which I assume is chip specific) and the ucode_id.
drivers/gpu/nova-core/falcon/hal/ga102.rs line 69
Quoted 3 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +3
-0+ +// Equivalent of Find Last Set bit. +Ok (u32::BITS - reg_fuse_version.leading_zeros ())
Maybe we should create a generic helper for that?
Quoted 80 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +51
-0+} + +fn program_brom_ga102 <E :FalconEngine >(bar: &Bar0 , params: &FalconBromParams ) ->Result <()> { + regs::NV_PFALCON2_FALCON_BROM_PARAADDR ::default () + .set_value (params.pkc_data_offset) + .write (bar,E ::BASE ); + regs::NV_PFALCON2_FALCON_BROM_ENGIDMASK ::default () + .set_value (params.engine_id_maskas u32 ) + .write (bar,E ::BASE ); + regs::NV_PFALCON2_FALCON_BROM_CURR_UCODE_ID ::default () + .set_ucode_id (params.ucode_id) + .write (bar,E ::BASE ); + regs::NV_PFALCON2_FALCON_MOD_SEL ::default () + .set_algo (FalconModSelAlgo ::Rsa3k ) + .write (bar,E ::BASE ); + +Ok (()) +} + +pub (super )struct Ga102 <E :FalconEngine >(PhantomData <E >); + +impl <E :FalconEngine >Ga102 <E > { +pub (super )fn new () ->Self { +Self (PhantomData ) + } +} + +impl <E :FalconEngine >FalconHal <E >for Ga102 <E > { +fn select_core (&self, _falcon: &Falcon <E >, bar: &Bar0 ) ->Result <()> { +select_core_ga102 ::<E >(bar) + } + +fn get_signature_reg_fuse_version ( + &self, + falcon: &Falcon <E >, + bar: &Bar0 , + engine_id_mask:u16 , + ucode_id:u8 , + ) ->Result <u32 > { +get_signature_reg_fuse_version_ga102 (&falcon.dev, bar, engine_id_mask, ucode_id) + } + +fn program_brom ( + &self, + _falcon: &Falcon <E >, + bar: &Bar0 , + params: &FalconBromParams , + ) ->Result <()> { +program_brom_ga102 ::<E >(bar, params) + } +}drivers/gpu/nova-core/falcon/sec2.rs new file +8
-0@@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0 + +use crate ::falcon::FalconEngine ; + +pub (crate )struct Sec2 ; +impl FalconEngine for Sec2 { +const BASE :usize =0x00840000 ; +}drivers/gpu/nova-core/gpu.rs +1
-0@@ -5,6 +5,7 @@use crate ::devinit;use crate ::dma::DmaObject ;use crate ::driver::Bar0 ; +use crate ::falcon::{gsp::Gsp , sec2::Sec2 ,Falcon };use crate ::firmware::Firmware ;use crate ::regs;use crate ::util;@@ -227,6 +228,16 @@ pub(crate) fn new( page };
drivers/gpu/nova-core/gpu.rs pub(crate) fn new( · line 239
Quoted 10 lines
drivers/gpu/nova-core/gpu.rs +9
-0+let gsp_falcon =Falcon ::<Gsp >::new ( + pdev.as_ref (), + spec.chipset, + bar, + spec.chipset >Chipset ::GA100 , + )?; + gsp_falcon.clear_swgen0_intr (bar); + +let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?;
Just _ instead? Also, please add a comment why it is important to create this instance even though it's never used.
Quoted 172 lines
drivers/gpu/nova-core/gpu.rs +1
-0+ Ok(pin_init!(Self { spec, bar: devres_bar,drivers/gpu/nova-core/nova_core.rs +1
-0@@ -5,6 +5,7 @@mod devinit;mod dma;mod driver; +mod falcon;mod firmware;mod gpu;mod regs;drivers/gpu/nova-core/regs.rs +125
-0@@ -9,6 +9,10 @@#[macro_use] mod macros; +use crate ::falcon::{ +DmaTrfCmdSize ,FalconCoreRev ,FalconCoreRevSubversion ,FalconFbifMemType ,FalconFbifTarget , +FalconModSelAlgo ,FalconSecurityModel ,PeregrineCoreSelect , +};use crate ::gpu::{Architecture ,Chipset };/* PMC */ @@ -59,3 +63,124 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> {register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_05 @0x00118234 {31 :0 valueas u32 ; }); + +/* PFALCON */ + +register! (NV_PFALCON_FALCON_IRQSCLR @ +0x00000004 { +4 :4 haltas bool ; +6 :6 swgen0as bool ; +}); + +register! (NV_PFALCON_FALCON_MAILBOX0 @ +0x00000040 { +31 :0 valueas u32 ; +}); + +register! (NV_PFALCON_FALCON_MAILBOX1 @ +0x00000044 { +31 :0 valueas u32 ; +}); + +register! (NV_PFALCON_FALCON_RM @ +0x00000084 { +31 :0 valueas u32 ; +}); + +register! (NV_PFALCON_FALCON_HWCFG2 @ +0x000000f4 { +10 :10 riscvas bool ; +12 :12 mem_scrubbingas bool ; +31 :31 reset_readyas bool ,"Signal indicating that reset is completed (GA102+)" ; +}); + +register! (NV_PFALCON_FALCON_CPUCTL @ +0x00000100 { +1 :1 startcpuas bool ; +4 :4 haltedas bool ; +6 :6 alias_enas bool ; +}); + +register! (NV_PFALCON_FALCON_BOOTVEC @ +0x00000104 { +31 :0 valueas u32 ; +}); + +register! (NV_PFALCON_FALCON_DMACTL @ +0x0000010c { +0 :0 require_ctxas bool ; +1 :1 dmem_scrubbingas bool ; +2 :2 imem_scrubbingas bool ; +6 :3 dmaq_numas u8 ; +7 :7 secure_statas bool ; +}); + +register! (NV_PFALCON_FALCON_DMATRFBASE @ +0x00000110 { +31 :0 baseas u32 ; +}); + +register! (NV_PFALCON_FALCON_DMATRFMOFFS @ +0x00000114 { +23 :0 offsas u32 ; +}); + +register! (NV_PFALCON_FALCON_DMATRFCMD @ +0x00000118 { +0 :0 fullas bool ; +1 :1 idleas bool ; +3 :2 secas u8 ; +4 :4 imemas bool ; +5 :5 is_writeas bool ; +10 :8 sizeas u8 ?=>DmaTrfCmdSize ; +14 :12 ctxdmaas u8 ; +16 :16 set_dmtagas u8 ; +}); + +register! (NV_PFALCON_FALCON_DMATRFFBOFFS @ +0x0000011c { +31 :0 offsas u32 ; +}); + +register! (NV_PFALCON_FALCON_DMATRFBASE1 @ +0x00000128 { +8 :0 baseas u16 ; +}); + +register! (NV_PFALCON_FALCON_HWCFG1 @ +0x0000012c { +3 :0 core_revas u8 ?=>FalconCoreRev ,"Core revision" ; +5 :4 security_modelas u8 ?=>FalconSecurityModel ,"Security model" ; +7 :6 core_rev_subversionas u8 ?=>FalconCoreRevSubversion ,"Core revision subversion" ; +}); + +register! (NV_PFALCON_FALCON_CPUCTL_ALIAS @ +0x00000130 { +1 :1 startcpuas bool ; +}); + +// Actually known as `NV_PSEC_FALCON_ENGINE` and `NV_PGSP_FALCON_ENGINE` depending on the falcon +// instance. +register! (NV_PFALCON_FALCON_ENGINE @ +0x000003c0 { +0 :0 resetas bool ; +}); + +// TODO: this is an array of registers. +register! (NV_PFALCON_FBIF_TRANSCFG @ +0x00000600 { +1 :0 targetas u8 ?=>FalconFbifTarget ; +2 :2 mem_typeas bool =>FalconFbifMemType ; +}); + +register! (NV_PFALCON_FBIF_CTL @ +0x00000624 { +7 :7 allow_phys_no_ctxas bool ; +}); + +register! (NV_PFALCON2_FALCON_MOD_SEL @ +0x00001180 { +7 :0 algoas u8 ?=>FalconModSelAlgo ; +}); + +register! (NV_PFALCON2_FALCON_BROM_CURR_UCODE_ID @ +0x00001198 { +7 :0 ucode_idas u8 ; +}); + +register! (NV_PFALCON2_FALCON_BROM_ENGIDMASK @ +0x0000119c { +31 :0 valueas u32 ; +}); + +// TODO: this is an array of registers. +register! (NV_PFALCON2_FALCON_BROM_PARAADDR @ +0x00001210 { +31 :0 valueas u32 ; +}); + +/* PRISCV */ + +register! (NV_PRISCV_RISCV_BCR_CTRL @ +0x00001668 { +0 :0 validas bool ; +4 :4 core_selectas bool =>PeregrineCoreSelect ; +8 :8 br_fetchas bool ; +});drivers/gpu/nova-core/util.rs +0
-1@@ -34,7 +34,6 @@ pub(crate) const fn const_bytes_to_str(bytes: &[u8]) -> &str {/// /// TODO: replace with `read_poll_timeout` once it is available. /// (https://lore.kernel.org/lkml/[email protected]/) -#[expect(dead_code)] pub (crate )fn wait_on <R ,F :Fn () ->Option <R >>(timeout:Duration , cond:F ) ->Result <R > {let start_time =Ktime ::ktime_get ();Signature · 2 lines
-- 2.49.0
Alexandre Courbot
maintainerRe: [PATCH v3 13/19] gpu: nova-core: add falcon register definitions and base code
On Wed May 14, 2025 at 1:19 AM JST, Danilo Krummrich wrote:
<snip>
Quoted 10 lines
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+ util::wait_on (Duration ::from_millis (20 ), || { +let r = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if r.mem_scrubbing () { +Some (()) + }else { +None + } + }) + } +
Quoted 8 lines
drivers/gpu/nova-core/falcon.rs line 341
Quoted 6 lines
drivers/gpu/nova-core/falcon.rs new file +6
-0+/// Reset the falcon engine. +fn reset_eng (&self, bar: &Bar0 ) ->Result <()> { +let _ = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); + +// According to OpenRM's `kflcnPreResetWait_GA102` documentation, HW sometimes does not set +// RESET_READY so a non-failing timeout is used. Should we still warn about it?
OpenRM does not (as this is apparently a workaround to a HW bug?) so I don't think we need to.
Quoted 6 lines
Quoted 2 lines
+ let _ = util::wait_on(Duration::from_micros(150), || {
Do we know for sure that if
RESET_READYis not set after 150us, it won't ever be set? If the answer to that is yes, and we also do not want to warn aboutRESET_READYnot being set, why even bother trying to read it in the first place?
My guess is because this would the expected behavior if the bug wasn't there. My GPU (Ampere) does wait until the timeout, but we can expect newer GPUs to not have this problem and return earlier.
Quoted 14 lines
drivers/gpu/nova-core/falcon.rs line 353
Quoted 12 lines
drivers/gpu/nova-core/falcon.rs new file +11
-0+let r = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if r.reset_ready () { +Some (()) + }else { +None + } + }); + + regs::NV_PFALCON_FALCON_ENGINE ::alter (bar,E ::BASE , |v| v.set_reset (true )); + +let _:Result <()> = util::wait_on (Duration ::from_micros (10 ), ||None );Can we please get an abstraction for
udelay()for this?
Should it be local to nova-core, or be generally available? I refrained from doing this because there is work going on regarding timer and I thought it would cover things like udelay() as well. I'll add a TODO item for now but please let me know if you have something different in mind.
<snip>
Quoted 30 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs line 49
Quoted 13 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +13
-0+fn get_signature_reg_fuse_version_ga102 ( + dev: &device::Device , + bar: &Bar0 , + engine_id_mask:u16 , + ucode_id:u8 , +) ->Result <u32 > { +// The ucode fuse versions are contained in the FUSE_OPT_FPF_<ENGINE>_UCODE<X>_VERSION +// registers, which are an array. Our register definition macros do not allow us to manage them +// properly, so we need to hardcode their addresses for now. + +// Each engine has 16 ucode version registers numbered from 1 to 16. +if ucode_id ==0 || ucode_id >16 { +dev_warn! (dev,"invalid ucode id {:#x}" , ucode_id);Given that this is an error condition, this should be dev_err!() I suppose.
drivers/gpu/nova-core/falcon/hal/ga102.rs line 62
Quoted 13 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +13
-0+return Err (EINVAL ); + } +let reg_fuse =if engine_id_mask &0x0001 !=0 { +// NV_FUSE_OPT_FPF_SEC2_UCODE1_VERSION +0x824140 + }else if engine_id_mask &0x0004 !=0 { +// NV_FUSE_OPT_FPF_NVDEC_UCODE1_VERSION +0x824100 + }else if engine_id_mask &0x0400 !=0 { +// NV_FUSE_OPT_FPF_GSP_UCODE1_VERSION +0x8241c0 + }else { +dev_warn! (dev,"unexpected engine_id_mask {:#x}" , engine_id_mask);
Quoted 6 lines
s/dev_warn/dev_err/
Review
Quoted 2 lines
(quoted diff) +2
-0+ return Err(EINVAL); + } + ((ucode_id - 1) as usize * core::mem::size_of::<u32>());I feel like this calculation deserves a comment.
Agreed, added one. Also reorganized the code to make the calculation a bit more obvious.
Quoted 7 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs line 66
Quoted 3 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +2
-0+ +let reg_fuse_version = bar.read32 (reg_fuse);I feel like the calculation of
reg_fuseshould be abstracted with a dedicated type inregs.rs. that takes the magic number derived from theengine_id_mask(which I assume is chip specific) and theucode_id.
We would need proper support for register arrays to manage the ucode_id offset, so I'm afraid this one will be hard to get rid of. What kind of type did you have in mind?
One thing we can do though, is expose the offset of each register as a register type constant, and use that instead of the hardcoded values currently in this code - that part at least will be cleaner.
Quoted 6 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs line 69
Quoted 4 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +3
-0+ +// Equivalent of Find Last Set bit. +Ok (u32::BITS - reg_fuse_version.leading_zeros ())Maybe we should create a generic helper for that?
Good idea.
Quoted 82 lines
Quoted 82 lines
drivers/gpu/nova-core/falcon/hal/ga102.rs new file +51
-0+} + +fn program_brom_ga102 <E :FalconEngine >(bar: &Bar0 , params: &FalconBromParams ) ->Result <()> { + regs::NV_PFALCON2_FALCON_BROM_PARAADDR ::default () + .set_value (params.pkc_data_offset) + .write (bar,E ::BASE ); + regs::NV_PFALCON2_FALCON_BROM_ENGIDMASK ::default () + .set_value (params.engine_id_maskas u32 ) + .write (bar,E ::BASE ); + regs::NV_PFALCON2_FALCON_BROM_CURR_UCODE_ID ::default () + .set_ucode_id (params.ucode_id) + .write (bar,E ::BASE ); + regs::NV_PFALCON2_FALCON_MOD_SEL ::default () + .set_algo (FalconModSelAlgo ::Rsa3k ) + .write (bar,E ::BASE ); + +Ok (()) +} + +pub (super )struct Ga102 <E :FalconEngine >(PhantomData <E >); + +impl <E :FalconEngine >Ga102 <E > { +pub (super )fn new () ->Self { +Self (PhantomData ) + } +} + +impl <E :FalconEngine >FalconHal <E >for Ga102 <E > { +fn select_core (&self, _falcon: &Falcon <E >, bar: &Bar0 ) ->Result <()> { +select_core_ga102 ::<E >(bar) + } + +fn get_signature_reg_fuse_version ( + &self, + falcon: &Falcon <E >, + bar: &Bar0 , + engine_id_mask:u16 , + ucode_id:u8 , + ) ->Result <u32 > { +get_signature_reg_fuse_version_ga102 (&falcon.dev, bar, engine_id_mask, ucode_id) + } + +fn program_brom ( + &self, + _falcon: &Falcon <E >, + bar: &Bar0 , + params: &FalconBromParams , + ) ->Result <()> { +program_brom_ga102 ::<E >(bar, params) + } +}drivers/gpu/nova-core/falcon/sec2.rs new file +8
-0@@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0 + +use crate ::falcon::FalconEngine ; + +pub (crate )struct Sec2 ; +impl FalconEngine for Sec2 { +const BASE :usize =0x00840000 ; +}drivers/gpu/nova-core/gpu.rs +1
-0@@ -5,6 +5,7 @@use crate ::devinit;use crate ::dma::DmaObject ;use crate ::driver::Bar0 ; +use crate ::falcon::{gsp::Gsp , sec2::Sec2 ,Falcon };use crate ::firmware::Firmware ;use crate ::regs;use crate ::util;@@ -227,6 +228,16 @@ pub(crate) fn new( page };
Quoted 12 lines
drivers/gpu/nova-core/gpu.rs pub(crate) fn new( · line 239
Quoted 9 lines
drivers/gpu/nova-core/gpu.rs +9
-0+let gsp_falcon =Falcon ::<Gsp >::new ( + pdev.as_ref (), + spec.chipset, + bar, + spec.chipset >Chipset ::GA100 , + )?; + gsp_falcon.clear_swgen0_intr (bar); + +let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?;Just
_instead? Also, please add a comment why it is important to create this instance even though it's never used.
It is not really important now, more a way to exercise the code until we need to run Booter. The variable will be renamed to sec2_falcon eventually, so I'd like to keep that name in the placeholder.
Danilo Krummrich
maintainerRe: [PATCH v3 13/19] gpu: nova-core: add falcon register definitions and base code
Quoted 27 lines · Alexandre Courbot
On Wed May 14, 2025 at 1:19 AM JST, Danilo Krummrich wrote:
<snip>Quoted 10 lines
Quoted 10 lines
drivers/gpu/nova-core/falcon.rs new file +10
-0+ util::wait_on (Duration ::from_millis (20 ), || { +let r = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if r.mem_scrubbing () { +Some (()) + }else { +None + } + }) + } +Quoted 8 lines
drivers/gpu/nova-core/falcon.rs line 341
Quoted 6 lines
drivers/gpu/nova-core/falcon.rs new file +6
-0+/// Reset the falcon engine. +fn reset_eng (&self, bar: &Bar0 ) ->Result <()> { +let _ = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); + +// According to OpenRM's `kflcnPreResetWait_GA102` documentation, HW sometimes does not set +// RESET_READY so a non-failing timeout is used. Should we still warn about it?
OpenRM does not (as this is apparently a workaround to a HW bug?) so I don't think we need to.
Quoted 3 lines
Quoted 2 lines
+ let _ = util::wait_on(Duration::from_micros(150), || {
Quoted 7 lines
Quoted 3 lines
Do we know for sure that if
RESET_READYis not set after 150us, it won't ever be set? If the answer to that is yes, and we also do not want to warn aboutRESET_READYnot being set, why even bother trying to read it in the first place?My guess is because this would the expected behavior if the bug wasn't there. My GPU (Ampere) does wait until the timeout, but we can expect newer GPUs to not have this problem and return earlier.
Ok, let's keep it then.
Quoted 15 lines
Quoted 14 lines
drivers/gpu/nova-core/falcon.rs line 353
Quoted 12 lines
drivers/gpu/nova-core/falcon.rs new file +11
-0+let r = regs::NV_PFALCON_FALCON_HWCFG2 ::read (bar,E ::BASE ); +if r.reset_ready () { +Some (()) + }else { +None + } + }); + + regs::NV_PFALCON_FALCON_ENGINE ::alter (bar,E ::BASE , |v| v.set_reset (true )); + +let _:Result <()> = util::wait_on (Duration ::from_micros (10 ), ||None );Can we please get an abstraction for
udelay()for this?
Quoted 5 lines
Should it be local to nova-core, or be generally available? I refrained from doing this because there is work going on regarding timer and I thought it would cover things like
udelay()as well. I'll add a TODO item for now but please let me know if you have something different in mind.
Not local to nova-core, but in the generic abstraction. I don't think the generic abstraction posted on the mailing list contains udelay(). Should be trivial to add it with a subsequent patch though.
A TODO should be fine for now.
Quoted 6 lines
Quoted 5 lines
Quoted 1 line
+ let
reg_fuse_version=bar.read32(reg_fuse);I feel like the calculation of
reg_fuseshould be abstracted with a dedicated type inregs.rs. that takes the magic number derived from theengine_id_mask(which I assume is chip specific) and theucode_id.
Quoted 7 lines
We would need proper support for register arrays to manage the
ucode_idoffset, so I'm afraid this one will be hard to get rid of. What kind of type did you have in mind?One thing we can do though, is expose the offset of each register as a register type constant, and use that instead of the hardcoded values currently in this code - that part at least will be cleaner.
Let's do that then for now.
Quoted 8 lines
Quoted 4 lines
Quoted 1 line
+ let
_sec2_falcon= Falcon::<Sec2>::new(pdev.as_ref(), spec.chipset, bar, true)?;Just
_instead? Also, please add a comment why it is important to create this instance even though it's never used.It is not really important now, more a way to exercise the code until we need to run Booter. The variable will be renamed to
sec2_falconeventually, so I'd like to keep that name in the placeholder.
Ok, seems reasonable.
Alexandre Courbot
maintainerPATCH v3 14/19 gpu: nova-core: firmware: add ucode descriptor used by FWSEC-FRTS
In reply to Alexandre CourbotFWSEC-FRTS is the first firmware we need to run on the GSP falcon in order to initiate the GSP boot process. Introduce the structure that describes it.
--- drivers/gpu/nova-core/firmware.rs | 43 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 43 insertions(+)
drivers/gpu/nova-core/firmware.rs +43 -0
@@ -45,6 +45,49 @@ pub(crate) fn new( } } +/// Structure used to describe some firmwares, notably FWSEC-FRTS. +#[repr( C )] +#[derive( Debug , Clone )] +pub (crate )struct FalconUCodeDescV3 { +/// Header defined by `NV_BIT_FALCON_UCODE_DESC_HEADER_VDESC*` in OpenRM. +/// +/// Bits `31:16` contain the size of the header, after which the actual ucode data starts. + hdr:u32 , +/// Stored size of the ucode after the header. + stored_size:u32 , +/// Offset in `DMEM` at which the signature is expected to be found. +pub (crate ) pkc_data_offset:u32 , +/// Offset after the code segment at which the app headers are located. +pub (crate ) interface_offset:u32 , +/// Base address at which to load the code segment into `IMEM`. +pub (crate ) imem_phys_base:u32 , +/// Size in bytes of the code to copy into `IMEM`. +pub (crate ) imem_load_size:u32 , +/// Virtual `IMEM` address (i.e. `tag`) at which the code should start. +pub (crate ) imem_virt_base:u32 , +/// Base address at which to load the data segment into `DMEM`. +pub (crate ) dmem_phys_base:u32 , +/// Size in bytes of the data to copy into `DMEM`. +pub (crate ) dmem_load_size:u32 , +/// Mask of the falcon engines on which this firmware can run. +pub (crate ) engine_id_mask:u16 , +/// ID of the ucode used to infer a fuse register to validate the signature. +pub (crate ) ucode_id:u8 , +/// Number of signatures in this firmware. +pub (crate ) signature_count:u8 , +/// Versions of the signatures, used to infer a valid signature to use. +pub (crate ) signature_versions:u16 , + _reserved:u16 , +} + +// To be removed once that code is used. +#[expect(dead_code)] +impl FalconUCodeDescV3 { +pub (crate )fn size (&self) ->usize { + ((self.hdr &0xffff0000 ) >>16 )as usize + } +} +pub (crate )struct ModInfoBuilder <const N :usize >(firmware::ModInfoBuilder <N >);impl <const N :usize >ModInfoBuilder <N > {
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 15/19 rust: num: Add an upward alignment helper for usize
In reply to Alexandre CourbotFrom: Joel Fernandes <[email protected]>
This will be used in the nova-core driver where we need to upward-align the image size to get to the next image in the VBIOS ROM.
[[email protected]: handled conflicts due to removal of patch creating num.rs]
--- rust/kernel/lib.rs | 1 + rust/kernel/num.rs | 44 ++++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+)
rust/kernel/lib.rs +1 -0
@@ -67,6 +67,7 @@pub mod miscdevice;#[cfg( CONFIG_NET )] pub mod net; +pub mod num;pub mod of;pub mod page;#[cfg( CONFIG_PCI )]
rust/kernel/num.rs new file +44 -0
44 lines not on this page · raw message
Signature · 1 line
2.49.0
Alexandre Courbot
maintainerPATCH v3 16/19 nova-core: Add support for VBIOS ucode extraction for boot
In reply to Alexandre CourbotFrom: Joel Fernandes <[email protected]>
Add support for navigating and setting up vBIOS ucode data required for GSP to boot. The main data extracted from the vBIOS is the FWSEC-FRTS firmware which runs on the GSP processor. This firmware runs in high secure mode, and sets up the WPR2 (Write protected region) before the Booter runs on the SEC2 processor.
Also add log messages to show the BIOS images.
[102141.013287] NovaCore: Found BIOS image at offset 0x0, size: 0xfe00, type: PciAt [102141.080692] NovaCore: Found BIOS image at offset 0xfe00, size: 0x14800, type: Efi [102141.098443] NovaCore: Found BIOS image at offset 0x24600, size: 0x5600, type: FwSec [102141.415095] NovaCore: Found BIOS image at offset 0x29c00, size: 0x60800, type: FwSec
Tested on my Ampere GA102 and boot is successful.
[applied changes by Alex Courbot for fwsec signatures] [applied feedback from Alex Courbot and Timur Tabi]
[applied changes related to code reorg, prints etc from Danilo Krummrich] [[email protected]: fix clippy warnings] [[email protected]: remove now-unneeded Devres acquisition] [[email protected]: fix read_more to read len bytes, not u32s]
--- drivers/gpu/nova-core/firmware.rs | 2 - drivers/gpu/nova-core/gpu.rs | 3 + drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/vbios.rs | 1147 ++++++++++++++++++++++++++++++++++++ 4 files changed, 1151 insertions(+), 2 deletions(-)
drivers/gpu/nova-core/firmware.rs +0 -2
@@ -80,8 +80,6 @@ pub(crate) struct FalconUCodeDescV3 { _reserved: u16, } -// To be removed once that code is used. -#[expect(dead_code)] impl FalconUCodeDescV3 {pub (crate )fn size (&self) ->usize { ((self.hdr &0xffff0000 ) >>16 )as usize
drivers/gpu/nova-core/gpu.rs +3 -0
@@ -9,6 +9,7 @@use crate ::firmware::Firmware ;use crate ::regs;use crate ::util; +use crate ::vbios::Vbios ;use core::fmt;macro_rules! define_chipset {
@@ -238,6 +239,8 @@ pub(crate) fn new(let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?; +let _bios =Vbios ::new (pdev, bar)?; + Ok(pin_init!(Self { spec, bar: devres_bar,
drivers/gpu/nova-core/nova_core.rs +1 -0
@@ -10,6 +10,7 @@mod gpu;mod regs;mod util; +mod vbios; kernel::module_pci_driver! {type :driver ::NovaCore ,
drivers/gpu/nova-core/vbios.rs new file +1147 -0
1,147 lines not on this page · raw message
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 65 lines · Alexandre Courbot
From: Joel Fernandes <[email protected]>
Add support for navigating and setting up vBIOS ucode data required for GSP to boot. The main data extracted from the vBIOS is the FWSEC-FRTS firmware which runs on the GSP processor. This firmware runs in high secure mode, and sets up the WPR2 (Write protected region) before the Booter runs on the SEC2 processor.
Also add log messages to show the BIOS images.
[102141.013287] NovaCore: Found BIOS image at offset 0x0, size: 0xfe00, type: PciAt [102141.080692] NovaCore: Found BIOS image at offset 0xfe00, size: 0x14800, type: Efi [102141.098443] NovaCore: Found BIOS image at offset 0x24600, size: 0x5600, type: FwSec [102141.415095] NovaCore: Found BIOS image at offset 0x29c00, size: 0x60800, type: FwSecTested on my Ampere GA102 and boot is successful.
[applied changes by Alex Courbot for fwsec signatures] [applied feedback from Alex Courbot and Timur Tabi]
[applied changes related to code reorg, prints etc from Danilo Krummrich] [[email protected]: fix clippy warnings] [[email protected]: remove now-unneeded Devres acquisition] [[email protected]: fixread_moreto readlenbytes, not u32s]Cc Alexandre Courbot <[email protected]>Cc John Hubbard <[email protected]>Cc Shirish Baskaran <[email protected]>Cc Alistair Popple <[email protected]>Cc Timur Tabi <[email protected]>Cc Ben Skeggs <[email protected]>Signed-off-by Joel Fernandes <[email protected]>Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/firmware.rs | 2 - drivers/gpu/nova-core/gpu.rs | 3 + drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/vbios.rs | 1147 ++++++++++++++++++++++++++++++++++++ 4 files changed, 1151 insertions(+), 2 deletions(-)drivers/gpu/nova-core/firmware.rs +0
-2@@ -80,8 +80,6 @@ pub(crate) struct FalconUCodeDescV3 { _reserved: u16, } -// To be removed once that code is used. -#[expect(dead_code)] impl FalconUCodeDescV3 {pub (crate )fn size (&self) ->usize { ((self.hdr &0xffff0000 ) >>16 )as usize drivers/gpu/nova-core/gpu.rs +1
-0@@ -9,6 +9,7 @@use crate ::firmware::Firmware ;use crate ::regs;use crate ::util; +use crate ::vbios::Vbios ;use core::fmt;macro_rules! define_chipset {
drivers/gpu/nova-core/gpu.rs pub(crate) fn new( · line 242
Quoted 4 lines
drivers/gpu/nova-core/gpu.rs +1
-0@@ -238,6 +239,8 @@ pub(crate) fn new(let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?; +let _bios =Vbios ::new (pdev, bar)?;
Please add a comment why, even though unused, it is important to create this instance.
Also, please use _ if it's not intended to ever be used.
Quoted 203 lines
drivers/gpu/nova-core/gpu.rs +1
-0+ Ok(pin_init!(Self { spec, bar: devres_bar,drivers/gpu/nova-core/nova_core.rs +1
-0@@ -10,6 +10,7 @@mod gpu;mod regs;mod util; +mod vbios; kernel::module_pci_driver! {type :driver ::NovaCore ,drivers/gpu/nova-core/vbios.rs new file +181
-0@@ -0,0 +1,1147 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! VBIOS extraction and parsing. + +// To be removed when all code is used. +#![expect(dead_code)] + +use crate ::driver::Bar0 ; +use crate ::firmware::FalconUCodeDescV3 ; +use core::convert::TryFrom ; +use kernel::device; +use kernel::error::Result ; +use kernel::num::NumAlign ; +use kernel::pci; +use kernel::prelude::*; + +/// The offset of the VBIOS ROM in the BAR0 space. +const ROM_OFFSET :usize =0x300000 ; +/// The maximum length of the VBIOS ROM to scan into. +const BIOS_MAX_SCAN_LEN :usize =0x100000 ; +/// The size to read ahead when parsing initial BIOS image headers. +const BIOS_READ_AHEAD_SIZE :usize =1024 ; +/// The bit in the last image indicator byte for the PCI Data Structure that +/// indicates the last image. Bit 0-6 are reserved, bit 7 is last image bit. +const LAST_IMAGE_BIT_MASK :u8 =0x80 ; + +// PMU lookup table entry types. Used to locate PMU table entries +// in the Fwsec image, corresponding to falcon ucodes. +#[expect(dead_code)] +const FALCON_UCODE_ENTRY_APPID_FIRMWARE_SEC_LIC :u8 =0x05 ; +#[expect(dead_code)] +const FALCON_UCODE_ENTRY_APPID_FWSEC_DBG :u8 =0x45 ; +const FALCON_UCODE_ENTRY_APPID_FWSEC_PROD :u8 =0x85 ; + +/// Vbios Reader for constructing the VBIOS data +struct VbiosIterator <'a> { + pdev: &'a pci::Device , + bar0: &'aBar0 , +// VBIOS data vector: As BIOS images are scanned, they are added to this vector +// for reference or copying into other data structures. It is the entire +// scanned contents of the VBIOS which progressively extends. It is used +// so that we do not re-read any contents that are already read as we use +// the cumulative length read so far, and re-read any gaps as we extend +// the length. + data:KVec <u8 >, + current_offset:usize ,// Current offset for iterator + last_found:bool ,// Whether the last image has been found +} + +impl <'a>VbiosIterator <'a> { +fn new (pdev: &'a pci::Device , bar0: &'aBar0 ) ->Result <Self > { +Ok (Self { + pdev, + bar0, + data:KVec ::new (), + current_offset:0 , + last_found:false , + }) + } + +/// Read bytes from the ROM at the current end of the data vector +fn read_more (&mut self, len:usize ) ->Result { +let current_len = self.data.len (); +let start =ROM_OFFSET + current_len; + +// Ensure length is a multiple of 4 for 32-bit reads +if len % core::mem::size_of ::<u32 >() !=0 { +dev_err! ( + self.pdev.as_ref (), +"VBIOS read length {} is not a multiple of 4\n" , + len + ); +return Err (EINVAL ); + } + + self.data.reserve (len,GFP_KERNEL )?; +// Read ROM data bytes and push directly to vector +for iin (0 ..len).step_by (core::mem::size_of ::<u32 >()) { +// Read 32-bit word from the VBIOS ROM +let word = self.bar0.try_read32 (start + i)?; + +// Convert the u32 to a 4 byte array and push each byte + word.to_ne_bytes () + .iter () + .try_for_each (|&b| self.data.push (b,GFP_KERNEL ))?; + } + +Ok (()) + } + +/// Read bytes at a specific offset, filling any gap +fn read_more_at_offset (&mut self, offset:usize , len:usize ) ->Result { +if offset >BIOS_MAX_SCAN_LEN { +dev_err! (self.pdev.as_ref (),"Error: exceeded BIOS scan limit.\n" ); +return Err (EINVAL ); + } + +// If offset is beyond current data size, fill the gap first +let current_len = self.data.len (); +let gap_bytes = offset.saturating_sub (current_len); + +// Now read the requested bytes at the offset + self.read_more (gap_bytes + len) + } + +/// Read a BIOS image at a specific offset and create a BiosImage from it. +/// self.data is extended as needed and a new BiosImage is returned. +/// @context is a string describing the operation for error reporting +fn read_bios_image_at_offset ( + &mut self, + offset:usize , + len:usize , + context: &str , + ) ->Result <BiosImage > { +let data_len = self.data.len (); +if offset + len > data_len { + self.read_more_at_offset (offset, len).inspect_err (|e| { +dev_err! ( + self.pdev.as_ref (), +"Failed to read more at offset {:#x}: {:?}\n" , + offset, + e + ) + })?; + } + +BiosImage ::new (self.pdev, &self.data[offset..offset + len]).inspect_err (|err| { +dev_err! ( + self.pdev.as_ref (), +"Failed to {} at offset {:#x}: {:?}\n" , + context, + offset, + err + ) + }) + } +} + +impl <'a>Iterator for VbiosIterator <'a> { +type Item =Result <BiosImage >; + +/// Iterate over all VBIOS images until the last image is detected or offset +/// exceeds scan limit. +fn next (&mut self) ->Option <Self ::Item > { +if self.last_found { +return None ; + } + +if self.current_offset >BIOS_MAX_SCAN_LEN { +dev_err! ( + self.pdev.as_ref (), +"Error: exceeded BIOS scan limit, stopping scan\n" + ); +return None ; + } + +// Parse image headers first to get image size +let image_size =match self + .read_bios_image_at_offset ( + self.current_offset, +BIOS_READ_AHEAD_SIZE , +"parse initial BIOS image headers" , + ) + .and_then (|image| image.image_size_bytes ()) + { +Ok (size) => size, +Err (e) =>return Some (Err (e)), + }; + +// Now create a new BiosImage with the full image data +let full_image =match self.read_bios_image_at_offset ( + self.current_offset, + image_size, +"parse full BIOS image" , + ) { +Ok (image) => image, +Err (e) =>return Some (Err (e)), + }; + + self.last_found = full_image.is_last (); +
drivers/gpu/nova-core/vbios.rs line 191
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+// Advance to next image (aligned to 512 bytes) + self.current_offset += image_size; + self.current_offset = self.current_offset.align_up (512 ); + +Some (Ok (full_image)) + } +} + +pub (crate )struct Vbios { +pub fwsec_image:Option <FwSecBiosImage >,
Please use pub(crate) instead or provide an accessor.
Also, this shouldn't be an Option, see below comment in Vbios::new().
Quoted 6 lines
drivers/gpu/nova-core/vbios.rs new file +6
-0+} + +impl Vbios { +/// Probe for VBIOS extraction +/// Once the VBIOS object is built, bar0 is not read for vbios purposes anymore. +pub (crate )fn new (pdev: &pci::Device , bar0: &Bar0 ) ->Result <Vbios > {
drivers/gpu/nova-core/vbios.rs line 207
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+// Images to extract from iteration +let mut pci_at_image:Option <PciAtBiosImage > =None ; +let mut first_fwsec_image:Option <FwSecBiosImage > =None ; +let mut second_fwsec_image:Option <FwSecBiosImage > =None ; + +// Parse all VBIOS images in the ROM +for image_resultin VbiosIterator ::new (pdev, bar0)? { +let full_image = image_result?; + +dev_info! (
Let's use dev_dbg!() instaed.
Quoted 35 lines
drivers/gpu/nova-core/vbios.rs new file +35
-0+ pdev.as_ref(), + "Found BIOS image: size: {:#x}, type: {}, last: {}\n", + full_image.image_size_bytes()?, + full_image.image_type_str(), + full_image.is_last () + ); + +// Get references to images we will need after the loop, in order to +// setup the falcon data offset. +match full_image { +BiosImage ::PciAt (image) => { + pci_at_image =Some (image); + } +BiosImage ::FwSec (image) => { +if first_fwsec_image.is_none () { + first_fwsec_image =Some (image); + }else { + second_fwsec_image =Some (image); + } + } +// For now we don't need to handle these +BiosImage ::Efi (_image) => {} +BiosImage ::Nbsi (_image) => {} + } + } + +// Using all the images, setup the falcon data pointer in Fwsec. +// We need mutable access here, so we handle the Option manually. +let final_fwsec_image = { +let mut second = second_fwsec_image;// Take ownership of the option + +if let (Some (second),Some (first),Some (pci_at)) = + (second.as_mut (), first_fwsec_image, pci_at_image) + { + second
drivers/gpu/nova-core/vbios.rs line 252
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ .setup_falcon_data(pdev, &pci_at, &first) + .inspect_err (|e| { +dev_err! (pdev.as_ref (),"Falcon data setup failed: {:?}\n" , e) + })?; + }else { +dev_err! ( + pdev.as_ref (), +"Missing required images for falcon data setup, skipping\n" + ); +return Err (EINVAL );
This means that if second == None we fail, which makes sense, so why store an Option in Vbios? All methods of Vbios fail if fwsec_image == None.
drivers/gpu/nova-core/vbios.rs line 255
Quoted 3 lines
drivers/gpu/nova-core/vbios.rs new file +3
-0+ } + second + };
I think this should be:
let mut second = second_fwsec_image;
if let (Some(second), Some(first), Some(pci_at)) =
(second.as_mut(), first_fwsec_image, pci_at_image)
{
second
.setup_falcon_data(pdev, &pci_at, &first)
.inspect_err(|e| {
dev_err!(pdev.as_ref(), "Falcon data setup failed: {:?}\n", e)
})?;
Ok(Vbios(second)
} else {
dev_err!(
pdev.as_ref(),
"Missing required images for falcon data setup, skipping\n"
);
Err(EINVAL)
}where Vbios can just be
pub(crate) struct Vbios(FwSecBiosImage);
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ +Ok (Vbios { + fwsec_image: final_fwsec_image, + }) + } + +pub (crate )fn fwsec_header (&self, pdev: &device::Device ) ->Result <&FalconUCodeDescV3 > { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_header (pdev) + }
drivers/gpu/nova-core/vbios.rs line 275
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ +pub (crate )fn fwsec_ucode (&self, pdev: &device::Device ) ->Result <&[u8 ]> { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_ucode (pdev, image.fwsec_header (pdev)?) + } + +pub (crate )fn fwsec_sigs (&self, pdev: &device::Device ) ->Result <&[u8 ]> { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_sigs (pdev, image.fwsec_header (pdev)?) + }
Those then become infallible, e.g.
pub(crate) fn fwsec_sigs(&self, pdev: &device::Device) -> &[u8] {
self.0.fwsec_sigs(pdev, self.fwsec_header(pdev))
}drivers/gpu/nova-core/vbios.rs line 276
Quoted 1 line
drivers/gpu/nova-core/vbios.rs new file +1
-0+}
<snip>
I have to continue with the rest of this patch later on.
- Danilo
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Hi Danilo,
Quoted 67 lines · Danilo Krummrich
Quoted 66 lines · Alexandre Courbot
From: Joel Fernandes <[email protected]>
Add support for navigating and setting up vBIOS ucode data required for GSP to boot. The main data extracted from the vBIOS is the FWSEC-FRTS firmware which runs on the GSP processor. This firmware runs in high secure mode, and sets up the WPR2 (Write protected region) before the Booter runs on the SEC2 processor.
Also add log messages to show the BIOS images.
[102141.013287] NovaCore: Found BIOS image at offset 0x0, size: 0xfe00, type: PciAt [102141.080692] NovaCore: Found BIOS image at offset 0xfe00, size: 0x14800, type: Efi [102141.098443] NovaCore: Found BIOS image at offset 0x24600, size: 0x5600, type: FwSec [102141.415095] NovaCore: Found BIOS image at offset 0x29c00, size: 0x60800, type: FwSecTested on my Ampere GA102 and boot is successful.
[applied changes by Alex Courbot for fwsec signatures] [applied feedback from Alex Courbot and Timur Tabi]
[applied changes related to code reorg, prints etc from Danilo Krummrich] [[email protected]: fix clippy warnings] [[email protected]: remove now-unneeded Devres acquisition] [[email protected]: fixread_moreto readlenbytes, not u32s]Cc Alexandre Courbot <[email protected]>Cc John Hubbard <[email protected]>Cc Shirish Baskaran <[email protected]>Cc Alistair Popple <[email protected]>Cc Timur Tabi <[email protected]>Cc Ben Skeggs <[email protected]>Signed-off-by Joel Fernandes <[email protected]>Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/firmware.rs | 2 - drivers/gpu/nova-core/gpu.rs | 3 + drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/vbios.rs | 1147 ++++++++++++++++++++++++++++++++++++ 4 files changed, 1151 insertions(+), 2 deletions(-)drivers/gpu/nova-core/firmware.rs +0
-2@@ -80,8 +80,6 @@ pub(crate) struct FalconUCodeDescV3 { _reserved: u16, } -// To be removed once that code is used. -#[expect(dead_code)] impl FalconUCodeDescV3 {pub (crate )fn size (&self) ->usize { ((self.hdr &0xffff0000 ) >>16 )as usize drivers/gpu/nova-core/gpu.rs +1
-0@@ -9,6 +9,7 @@use crate ::firmware::Firmware ;use crate ::regs;use crate ::util; +use crate ::vbios::Vbios ;use core::fmt;macro_rules! define_chipset {@@ -238,6 +239,8 @@ pub(crate) fn new(
Quoted 8 lines
drivers/gpu/nova-core/gpu.rs pub(crate) fn new( · line 242
Quoted 3 lines
drivers/gpu/nova-core/gpu.rs +1
-0let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?; +let _bios =Vbios ::new (pdev, bar)?;Please add a comment why, even though unused, it is important to create this instance.
Also, please use
_if it's not intended to ever be used.
If I add a comment, it will simply be removed by the next patch. I can add that though so it makes it more clear.
[...]
Quoted 56 lines
Quoted 43 lines
drivers/gpu/nova-core/gpu.rs +43
-0+impl <'a>Iterator for VbiosIterator <'a> { +type Item =Result <BiosImage >; + +/// Iterate over all VBIOS images until the last image is detected or offset +/// exceeds scan limit. +fn next (&mut self) ->Option <Self ::Item > { +if self.last_found { +return None ; + } + +if self.current_offset >BIOS_MAX_SCAN_LEN { +dev_err! ( + self.pdev.as_ref (), +"Error: exceeded BIOS scan limit, stopping scan\n" + ); +return None ; + } + +// Parse image headers first to get image size +let image_size =match self + .read_bios_image_at_offset ( + self.current_offset, +BIOS_READ_AHEAD_SIZE , +"parse initial BIOS image headers" , + ) + .and_then (|image| image.image_size_bytes ()) + { +Ok (size) => size, +Err (e) =>return Some (Err (e)), + }; + +// Now create a new BiosImage with the full image data +let full_image =match self.read_bios_image_at_offset ( + self.current_offset, + image_size, +"parse full BIOS image" , + ) { +Ok (image) => image, +Err (e) =>return Some (Err (e)), + }; + + self.last_found = full_image.is_last (); +drivers/gpu/nova-core/gpu.rs pub(crate) fn new(
Quoted 10 lines
drivers/gpu/nova-core/gpu.rs +10
-0+// Advance to next image (aligned to 512 bytes) + self.current_offset += image_size; + self.current_offset = self.current_offset.align_up (512 ); + +Some (Ok (full_image)) + } +} + +pub (crate )struct Vbios { +pub fwsec_image:Option <FwSecBiosImage >,Please use pub(crate) instead or provide an accessor.
Quoted 1 line
Also, this shouldn't be an Option, see below comment in
Vbios::new().
Ok, I just removed pub altogether, since the users all within this module.
Quoted 10 lines
Quoted 10 lines
drivers/gpu/nova-core/gpu.rs +10
-0+} + +impl Vbios { +/// Probe for VBIOS extraction +/// Once the VBIOS object is built, bar0 is not read for vbios purposes anymore. +pub (crate )fn new (pdev: &pci::Device , bar0: &Bar0 ) ->Result <Vbios > { +// Images to extract from iteration +let mut pci_at_image:Option <PciAtBiosImage > =None ; +let mut first_fwsec_image:Option <FwSecBiosImage > =None ; +let mut second_fwsec_image:Option <FwSecBiosImage > =None ;
Quoted 8 lines
drivers/gpu/nova-core/gpu.rs
Quoted 6 lines
drivers/gpu/nova-core/gpu.rs +6
-0+ +// Parse all VBIOS images in the ROM +for image_resultin VbiosIterator ::new (pdev, bar0)? { +let full_image = image_result?; + +dev_info! (Let's use dev_dbg!() instaed.
Done.
Quoted 41 lines
Quoted 41 lines
drivers/gpu/nova-core/gpu.rs +40
-0+ pdev.as_ref(), + "Found BIOS image: size: {:#x}, type: {}, last: {}\n", + full_image.image_size_bytes()?, + full_image.image_type_str(), + full_image.is_last () + ); + +// Get references to images we will need after the loop, in order to +// setup the falcon data offset. +match full_image { +BiosImage ::PciAt (image) => { + pci_at_image =Some (image); + } +BiosImage ::FwSec (image) => { +if first_fwsec_image.is_none () { + first_fwsec_image =Some (image); + }else { + second_fwsec_image =Some (image); + } + } +// For now we don't need to handle these +BiosImage ::Efi (_image) => {} +BiosImage ::Nbsi (_image) => {} + } + } + +// Using all the images, setup the falcon data pointer in Fwsec. +// We need mutable access here, so we handle the Option manually. +let final_fwsec_image = { +let mut second = second_fwsec_image;// Take ownership of the option + +if let (Some (second),Some (first),Some (pci_at)) = + (second.as_mut (), first_fwsec_image, pci_at_image) + { + second + .setup_falcon_data (pdev, &pci_at, &first) + .inspect_err (|e| { +dev_err! (pdev.as_ref (),"Falcon data setup failed: {:?}\n" , e) + })?; + }else {
Quoted 9 lines
drivers/gpu/nova-core/gpu.rs
Quoted 5 lines
drivers/gpu/nova-core/gpu.rs +5
-0+dev_err! ( + pdev.as_ref (), +"Missing required images for falcon data setup, skipping\n" + ); +return Err (EINVAL );This means that if second == None we fail, which makes sense, so why store an Option in Vbios? All methods of Vbios fail if
fwsec_image== None.
Well, if first and pci_at are None, we will fail as well. Not just second. But we don't know until we finish parsing all the images in the prior loop, if we found all the images. So we store it as Option during the prior loop, and check it later. Right?
Quoted 27 lines
drivers/gpu/nova-core/gpu.rs
Quoted 3 lines
drivers/gpu/nova-core/gpu.rs +3
-0+ } + second + };I think this should be:
let mut second = second_fwsec_image; if let (Some(second), Some(first), Some(pci_at)) = (second.as_mut(), first_fwsec_image, pci_at_image) { second .setup_falcon_data(pdev, &pci_at, &first) .inspect_err(|e| { dev_err!(pdev.as_ref(), "Falcon data setup failed: {:?}\n", e) })?; Ok(Vbios(second) } else { dev_err!( pdev.as_ref(), "Missing required images for falcon data setup, skipping\n" ); Err(EINVAL) }
Quoted 3 lines
where Vbios can just be
pub(crate) struct Vbios(FwSecBiosImage);
But your suggestion here still considers second as an Option? That's why you wrote 'Some(second)' ?
Quoted 24 lines
Quoted 11 lines
drivers/gpu/nova-core/gpu.rs +10
-0+ +Ok (Vbios { + fwsec_image: final_fwsec_image, + }) + } + +pub (crate )fn fwsec_header (&self, pdev: &device::Device ) ->Result <&FalconUCodeDescV3 > { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_header (pdev) + }drivers/gpu/nova-core/gpu.rs
Quoted 10 lines
drivers/gpu/nova-core/gpu.rs +10
-0+ +pub (crate )fn fwsec_ucode (&self, pdev: &device::Device ) ->Result <&[u8 ]> { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_ucode (pdev, image.fwsec_header (pdev)?) + } + +pub (crate )fn fwsec_sigs (&self, pdev: &device::Device ) ->Result <&[u8 ]> { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_sigs (pdev, image.fwsec_header (pdev)?) + }Those then become infallible, e.g.
Quoted 4 lines
pub(crate) fn fwsec_sigs(&self, pdev: &device::Device) -> &[u8] {
self.0.fwsec_sigs(pdev, self.fwsec_header(pdev))
}Nope, I think you are wrong there. fwsec_sigs() of the underlying .0 returns a Result.
Also in Vbios::new(), I extract the Option when returning:
Ok(Vbios {
fwsec_image: final_fwsec_image.ok_or(EINVAL)?,
})But fwsec_header() still has to return a Result:
pub(crate) fn fwsec_header(&self, pdev: &device::Device) ->
Result<FalconUCodeDescV3> {
self.fwsec_image.fwsec_header(pdev)
}thanks,
- Joel
Danilo Krummrich
maintainerRe: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 11 lines · Joel Fernandes
Quoted 10 lines · Danilo Krummrich
drivers/gpu/nova-core/gpu.rs pub(crate) fn new( · line 242
Quoted 5 lines · Alexandre Courbot
drivers/gpu/nova-core/gpu.rs +1
-0@@ -238,6 +239,8 @@ pub(crate) fn new(let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?; +let _bios =Vbios ::new (pdev, bar)?;Please add a comment why, even though unused, it is important to create this instance.
Quoted 4 lines
Quoted 1 line
Also, please use
_if it's not intended to ever be used.If I add a comment, it will simply be removed by the next patch. I can add that though so it makes it more clear.
I recommend to add such comments, because then reviewers don't stumble over it. :-)
Quoted 70 lines
Quoted 8 lines
Quoted 8 lines
drivers/gpu/nova-core/vbios.rs new file +8
-0+impl Vbios { +/// Probe for VBIOS extraction +/// Once the VBIOS object is built, bar0 is not read for vbios purposes anymore. +pub (crate )fn new (pdev: &pci::Device , bar0: &Bar0 ) ->Result <Vbios > { +// Images to extract from iteration +let mut pci_at_image:Option <PciAtBiosImage > =None ; +let mut first_fwsec_image:Option <FwSecBiosImage > =None ; +let mut second_fwsec_image:Option <FwSecBiosImage > =None ;Quoted 8 lines
drivers/gpu/nova-core/vbios.rs line 207
Quoted 6 lines
drivers/gpu/nova-core/vbios.rs new file +6
-0+ +// Parse all VBIOS images in the ROM +for image_resultin VbiosIterator ::new (pdev, bar0)? { +let full_image = image_result?; + +dev_info! (Let's use dev_dbg!() instaed.
Done.
Quoted 41 lines
Quoted 41 lines
drivers/gpu/nova-core/vbios.rs new file +40
-0+ pdev.as_ref(), + "Found BIOS image: size: {:#x}, type: {}, last: {}\n", + full_image.image_size_bytes()?, + full_image.image_type_str(), + full_image.is_last () + ); + +// Get references to images we will need after the loop, in order to +// setup the falcon data offset. +match full_image { +BiosImage ::PciAt (image) => { + pci_at_image =Some (image); + } +BiosImage ::FwSec (image) => { +if first_fwsec_image.is_none () { + first_fwsec_image =Some (image); + }else { + second_fwsec_image =Some (image); + } + } +// For now we don't need to handle these +BiosImage ::Efi (_image) => {} +BiosImage ::Nbsi (_image) => {} + } + } + +// Using all the images, setup the falcon data pointer in Fwsec. +// We need mutable access here, so we handle the Option manually. +let final_fwsec_image = { +let mut second = second_fwsec_image;// Take ownership of the option + +if let (Some (second),Some (first),Some (pci_at)) = + (second.as_mut (), first_fwsec_image, pci_at_image) + { + second + .setup_falcon_data (pdev, &pci_at, &first) + .inspect_err (|e| { +dev_err! (pdev.as_ref (),"Falcon data setup failed: {:?}\n" , e) + })?; + }else {Quoted 9 lines
drivers/gpu/nova-core/vbios.rs line 252
Quoted 5 lines
drivers/gpu/nova-core/vbios.rs new file +5
-0+dev_err! ( + pdev.as_ref (), +"Missing required images for falcon data setup, skipping\n" + ); +return Err (EINVAL );This means that if second == None we fail, which makes sense, so why store an Option in Vbios? All methods of Vbios fail if
fwsec_image== None.
Quoted 4 lines
Well, if first and
pci_atare None, we will fail as well. Not just second. But we don't know until we finish parsing all the images in the prior loop, if we found all the images. So we store it as Option during the prior loop, and check it later. Right?
My point is not that second is an option within this function -- that's fine. I don't want the Vbios type to store an Option, because that doesn't make sense. I.e. it should be
struct Vbios {
fwsec_image: FwSecBiosImage,
}or just
struct Vbios(FwSecBiosImage);
which is the same, rather than
struct Vbios {
fwsec_image: Option<FwSecBiosImage>,
}because Vbios::new() fails anyways if any of the images is None, i.e. vbios.fwsec_image can't ever be None.
The code below does that for you, i.e. it returns an instance of Vbios without the inner Option.
Quoted 27 lines
Quoted 27 lines
drivers/gpu/nova-core/vbios.rs line 255
Quoted 3 lines
drivers/gpu/nova-core/vbios.rs new file +3
-0+ } + second + };I think this should be:
let mut second = second_fwsec_image; if let (Some(second), Some(first), Some(pci_at)) = (second.as_mut(), first_fwsec_image, pci_at_image) { second .setup_falcon_data(pdev, &pci_at, &first) .inspect_err(|e| { dev_err!(pdev.as_ref(), "Falcon data setup failed: {:?}\n", e) })?; Ok(Vbios(second) } else { dev_err!( pdev.as_ref(), "Missing required images for falcon data setup, skipping\n" ); Err(EINVAL) }
Quoted 6 lines
Quoted 3 lines
where Vbios can just be
pub(crate) struct Vbios(FwSecBiosImage);But your suggestion here still considers second as an Option? That's why you wrote 'Some(second)' ?
Yes, that's fine, see above. The difference is that the code returns you an instance of
struct Vbios(FwSecBiosImage);
rather than
struct Vbios {
fwsec_image: Option<FwSecBiosImage>,
}which is unnecessary.
Quoted 25 lines
Quoted 25 lines
Quoted 12 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ +Ok (Vbios { + fwsec_image: final_fwsec_image, + }) + } + +pub (crate )fn fwsec_header (&self, pdev: &device::Device ) ->Result <&FalconUCodeDescV3 > { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_header (pdev) + }drivers/gpu/nova-core/vbios.rs line 275
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ +pub (crate )fn fwsec_ucode (&self, pdev: &device::Device ) ->Result <&[u8 ]> { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_ucode (pdev, image.fwsec_header (pdev)?) + } + +pub (crate )fn fwsec_sigs (&self, pdev: &device::Device ) ->Result <&[u8 ]> { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_sigs (pdev, image.fwsec_header (pdev)?) + }Those then become infallible, e.g.
Quoted 7 lines
Quoted 4 lines
pub(crate) fn fwsec_sigs(&self, pdev: &device::Device) -> &[u8] { self.0.fwsec_sigs(pdev, self.fwsec_header(pdev)) }Nope, I think you are wrong there.
fwsec_sigs()of the underlying .0 returns a Result.
That's true, I confused self.fwsec_sigs() with self.0.fwsec_sigs(). It seems that you may want to implement Deref for Vbios.
Also, can you please double check the Options in FwSecBiosImage (in case we didn't talk about them yet)? They look quite suspicious too.
In general, I feel like a lot of those Option come from a programming pattern that is very common in C, i.e. allocate a structure (stack or heap) and then initialize its fields.
In Rust you should aim to initialize all the fields of a structure when you create the instance. Option as a return type of a function is common, but it's always a bit suspicious when there is an Option field in a struct.
I understand that there are cases where we can't omit it, and for obvious reasons the Vbios code is probably a perfect example for that.
However, I recommend looking at this from top to bottom: Do the "final" structures that we expose to the driver from the Vbios module have fields that are *really* optional? Or is the Option type just a result from the parsing process?
If it's the latter, we should get rid of it and work with a different type during the parsing process and then create the final instance that is exposed to the driver at the end.
For instance FwSecBiosImage is defined as:
pub(crate) struct FwSecBiosImage {
base: BiosImageBase,
falcon_data_offset: Option<usize>,
pmu_lookup_table: Option<PmuLookupTable>,
falcon_ucode_offset: Option<usize>,
}Do only *some* FwSecBiosImage instances have a falcon_ucode_offset?
If the answer is 'no' then it shouldn't be an Option. If the answer is 'yes', then this indicates that FwSecBiosImage is probably too generic and should be split into more specific types of a FwSecBiosImage which instead share a common trait in order to treat the different types generically.
Quoted 5 lines
Also in
Vbios::new(), I extract the Option when returning:Ok(Vbios { fwsec_image: final_fwsec_image.ok_or(EINVAL)?, })
Maybe you do so in your tree? v3 of the patch series has:
pub(crate) struct Vbios {
pub fwsec_image: Option<FwSecBiosImage>,
}and
Ok(Vbios {
fwsec_image: final_fwsec_image,
})Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 12 lines · Danilo Krummrich
Quoted 11 lines · Joel Fernandes
Quoted 10 lines · Danilo Krummrich
drivers/gpu/nova-core/gpu.rs pub(crate) fn new( · line 242
Quoted 5 lines · Alexandre Courbot
drivers/gpu/nova-core/gpu.rs +1
-0@@ -238,6 +239,8 @@ pub(crate) fn new(let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?; +let _bios =Vbios ::new (pdev, bar)?;Please add a comment why, even though unused, it is important to create this instance.
Quoted 7 lines
Quoted 4 lines
Quoted 1 line
Also, please use
_if it's not intended to ever be used.If I add a comment, it will simply be removed by the next patch. I can add that though so it makes it more clear.
I recommend to add such comments, because then reviewers don't stumble over it. :-)
Point taken and fixed! ;-)
Quoted 74 lines
Quoted 51 lines
Quoted 41 lines
Quoted 41 lines
drivers/gpu/nova-core/vbios.rs new file +40
-0+ pdev.as_ref(), + "Found BIOS image: size: {:#x}, type: {}, last: {}\n", + full_image.image_size_bytes()?, + full_image.image_type_str(), + full_image.is_last () + ); + +// Get references to images we will need after the loop, in order to +// setup the falcon data offset. +match full_image { +BiosImage ::PciAt (image) => { + pci_at_image =Some (image); + } +BiosImage ::FwSec (image) => { +if first_fwsec_image.is_none () { + first_fwsec_image =Some (image); + }else { + second_fwsec_image =Some (image); + } + } +// For now we don't need to handle these +BiosImage ::Efi (_image) => {} +BiosImage ::Nbsi (_image) => {} + } + } + +// Using all the images, setup the falcon data pointer in Fwsec. +// We need mutable access here, so we handle the Option manually. +let final_fwsec_image = { +let mut second = second_fwsec_image;// Take ownership of the option + +if let (Some (second),Some (first),Some (pci_at)) = + (second.as_mut (), first_fwsec_image, pci_at_image) + { + second + .setup_falcon_data (pdev, &pci_at, &first) + .inspect_err (|e| { +dev_err! (pdev.as_ref (),"Falcon data setup failed: {:?}\n" , e) + })?; + }else {Quoted 9 lines
drivers/gpu/nova-core/vbios.rs line 252
Quoted 5 lines
drivers/gpu/nova-core/vbios.rs new file +5
-0+dev_err! ( + pdev.as_ref (), +"Missing required images for falcon data setup, skipping\n" + ); +return Err (EINVAL );This means that if second == None we fail, which makes sense, so why store an Option in Vbios? All methods of Vbios fail if
fwsec_image== None.Quoted 4 lines
Well, if first and
pci_atare None, we will fail as well. Not just second. But we don't know until we finish parsing all the images in the prior loop, if we found all the images. So we store it as Option during the prior loop, and check it later. Right?My point is not that second is an option within this function -- that's fine. I don't want the Vbios type to store an Option, because that doesn't make sense. I.e. it should be
struct Vbios { fwsec_image: FwSecBiosImage, }or just
struct Vbios(FwSecBiosImage);which is the same, rather than
struct Vbios { fwsec_image: Option<FwSecBiosImage>, }
Quoted 5 lines
because
Vbios::new()fails anyways if any of the images is None, i.e. vbios.fwsec_image can't ever be None.The code below does that for you, i.e. it returns an instance of Vbios without the inner Option.
But your code below does Vbios(second) where Vbios is an option..
Quoted 18 lines
Quoted 18 lines
Quoted 18 lines
drivers/gpu/nova-core/vbios.rs line 255
Quoted 4 lines
drivers/gpu/nova-core/vbios.rs new file +3
-0+ } + second + };I think this should be:
let mut second = second_fwsec_image; if let (Some(second), Some(first), Some(pci_at)) = (second.as_mut(), first_fwsec_image, pci_at_image) { second .setup_falcon_data(pdev, &pci_at, &first) .inspect_err(|e| { dev_err!(pdev.as_ref(), "Falcon data setup failed: {:?}\n", e) })?;
Quoted 1 line
Quoted 1 line
Quoted 1 line
Ok(Vbios(second)
I can't do that become second is a mutable reference in the above snippet.
But this works:
Ok(Vbios { fwsec_image: second.take().ok_or(EINVAL)? })(This did require changing 'Some(second)' to 'Some(second_ref)', see below.)
Quoted 21 lines
Quoted 9 lines
Quoted 9 lines
} else { dev_err!( pdev.as_ref(), "Missing required images for falcon data setup, skipping\n" ); Err(EINVAL) }Quoted 6 lines
Quoted 3 lines
where Vbios can just be
pub(crate) struct Vbios(FwSecBiosImage);But your suggestion here still considers second as an Option? That's why you wrote 'Some(second)' ?
Yes, that's fine, see above. The difference is that the code returns you an instance of
struct Vbios(FwSecBiosImage);
Quoted 7 lines
rather than
struct Vbios { fwsec_image: Option<FwSecBiosImage>, }which is unnecessary.
Sure, ok, yeah I made this change in another thread we are discussing so we are good.
So the code here now looks like the below, definitely better, thanks! :
if let (Some(second_ref), Some(first), Some(pci_at)) =
(second.as_mut(), first_fwsec_image, pci_at_image)
{
second_ref
.setup_falcon_data(pdev, &pci_at, &first)
.inspect_err(|e| {
dev_err!(..)
})?;
Ok(Vbios { fwsec_image: second.take().ok_or(EINVAL)? })
} else {
dev_err!(
pdev.as_ref(),
"Missing required images for falcon data setup, skipping\n"
);
Err(EINVAL)
}Quoted 38 lines
Quoted 23 lines
Quoted 23 lines
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ +Ok (Vbios { + fwsec_image: final_fwsec_image, + }) + } + +pub (crate )fn fwsec_header (&self, pdev: &device::Device ) ->Result <&FalconUCodeDescV3 > { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_header (pdev) + }drivers/gpu/nova-core/vbios.rs line 275
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ +pub (crate )fn fwsec_ucode (&self, pdev: &device::Device ) ->Result <&[u8 ]> { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_ucode (pdev, image.fwsec_header (pdev)?) + } + +pub (crate )fn fwsec_sigs (&self, pdev: &device::Device ) ->Result <&[u8 ]> { +let image = self.fwsec_image.as_ref ().ok_or (EINVAL )?; + image.fwsec_sigs (pdev, image.fwsec_header (pdev)?) + }Those then become infallible, e.g.
Quoted 7 lines
Quoted 4 lines
pub(crate) fn fwsec_sigs(&self, pdev: &device::Device) -> &[u8] { self.0.fwsec_sigs(pdev, self.fwsec_header(pdev)) }Nope, I think you are wrong there.
fwsec_sigs()of the underlying .0 returns a Result.That's true, I confused
self.fwsec_sigs()with self.0.fwsec_sigs(). It seems that you may want to implement Deref for Vbios.Also, can you please double check the Options in FwSecBiosImage (in case we didn't talk about them yet)? They look quite suspicious too.
Quoted 7 lines
In general, I feel like a lot of those Option come from a programming pattern that is very common in C, i.e. allocate a structure (stack or heap) and then initialize its fields.
In Rust you should aim to initialize all the fields of a structure when you create the instance. Option as a return type of a function is common, but it's always a bit suspicious when there is an Option field in a struct.
I looked into it, I could not git rid of those ones because we need to initialize in the "impl TryFrom<BiosImageBase> for BiosImage {"
0xE0 => Ok(BiosImage::FwSec(FwSecBiosImage {
base,
falcon_data_offset: None,
pmu_lookup_table: None,
falcon_ucode_offset: None,
})),And these fields will not be determined until much later, because as is the case with the earlier example, these fields cannot be determined until all the images are parsed.
Quoted 21 lines
I understand that there are cases where we can't omit it, and for obvious reasons the Vbios code is probably a perfect example for that.
However, I recommend looking at this from top to bottom: Do the "final" structures that we expose to the driver from the Vbios module have fields that are *really* optional? Or is the Option type just a result from the parsing process?
If it's the latter, we should get rid of it and work with a different type during the parsing process and then create the final instance that is exposed to the driver at the end.
For instance FwSecBiosImage is defined as:
pub(crate) struct FwSecBiosImage { base: BiosImageBase, falcon_data_offset: Option<usize>, pmu_lookup_table: Option<PmuLookupTable>, falcon_ucode_offset: Option<usize>, }
Quoted 6 lines
Do only *some* FwSecBiosImage instances have a
falcon_ucode_offset?If the answer is 'no' then it shouldn't be an Option. If the answer is 'yes', then this indicates that FwSecBiosImage is probably too generic and should be split into more specific types of a FwSecBiosImage which instead share a common trait in order to treat the different types generically.
Understood, thanks.
Quoted 12 lines
Quoted 5 lines
Also in
Vbios::new(), I extract the Option when returning:Ok(Vbios { fwsec_image: final_fwsec_image.ok_or(EINVAL)?, })Maybe you do so in your tree? v3 of the patch series has:
pub(crate) struct Vbios { pub fwsec_image: Option<FwSecBiosImage>, }
Quoted 5 lines
and
Ok(Vbios { fwsec_image: final_fwsec_image, })
Yes, I made the change during our review on the other thread and will be posted in the next posting. Sorry for any confusion.
thanks,
- Joel
Danilo Krummrich
maintainerRe: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 7 lines · Joel Fernandes
Quoted 3 lines · Danilo Krummrich
Quoted 2 lines · Joel Fernandes
Quoted 1 line · Danilo Krummrich
On Wed, May 07, 2025 at 10:52:43PM +0900, Alexandre Courbot wrote:
So the code here now looks like the below, definitely better, thanks! :
Quoted 16 lines
if let (Some(second_ref), Some(first), Some(pci_at)) =
(second.as_mut(), first_fwsec_image, pci_at_image)
{
second_ref
.setup_falcon_data(pdev, &pci_at, &first)
.inspect_err(|e| {
dev_err!(..)
})?;
Ok(Vbios { fwsec_image: second.take().ok_or(EINVAL)? })
} else {
dev_err!(
pdev.as_ref(),
"Missing required images for falcon data setup, skipping\n"
);
Err(EINVAL)
}Sorry, my code-snipped was incorrect indeed. Let me paste what I actually intended (and this time properly compile checked) and should be even better:
if let (Some(mut second), Some(first), Some(pci_at)) =
(second_fwsec_image, first_fwsec_image, pci_at_image)
{
second
.setup_falcon_data(pdev, &pci_at, &first)
.inspect_err(|e| {
dev_err!(pdev.as_ref(), "Falcon data setup failed: {:?}\n", e)
})?;
Ok(Vbios(second))
} else {
dev_err!(
pdev.as_ref(),
"Missing required images for falcon data setup, skipping\n"
);
Err(EINVAL)
}So, with this second is the actual value and not just a reference. :)
And the methods can become:
pub(crate) fn fwsec_header(&self, pdev: &device::Device) -> Result<&FalconUCodeDescV3> {
self.0.fwsec_header(pdev)
}
pub(crate) fn fwsec_ucode(&self, pdev: &device::Device) -> Result<&[u8]> {
self.0.fwsec_ucode(pdev, self.fwsec_header(pdev)?)
}
pub(crate) fn fwsec_sigs(&self, pdev: &device::Device) -> Result<&[u8]> {
self.0.fwsec_sigs(pdev, self.fwsec_header(pdev)?)
}However, I don't understand why they're not just implemented for FwSecBiosImage itself this way. You can just implement Deref for Vbios then.
Quoted 18 lines
Quoted 7 lines
In general, I feel like a lot of those Option come from a programming pattern that is very common in C, i.e. allocate a structure (stack or heap) and then initialize its fields.
In Rust you should aim to initialize all the fields of a structure when you create the instance. Option as a return type of a function is common, but it's always a bit suspicious when there is an Option field in a struct.
I looked into it, I could not git rid of those ones because we need to initialize in the "impl TryFrom<BiosImageBase> for BiosImage {"
0xE0 => Ok(BiosImage::FwSec(FwSecBiosImage { base, falcon_data_offset: None, pmu_lookup_table: None, falcon_ucode_offset: None, })),
Quoted 3 lines
And these fields will not be determined until much later, because as is the case with the earlier example, these fields cannot be determined until all the images are parsed.
You should not use TryFrom, but instead use a normal constructor, such as
BiosImage::new(base_bios_image)
and do the parsing within this constructor.
If you want a helper type with Options while parsing that's totally fine, but the final result can clearly be without Options. For instance:
struct Data {
image: KVec<u8>,
}
impl Data {
fn new() -> Result<Self> {
let parser = DataParser::new();
Self { image: parser.parse()? }
}
fn load_image(&self) {
...
}
}
struct DataParser {
// Only some images have a checksum.
checksum: Option<u64>,
// Some images have an extra offset.
offset: Option<u64>,
// Some images need to be patched.
patch: Option<KVec<u8>>,
image: KVec<u8>,
}
impl DataParser {
fn new() -> Self {
Self {
checksum: None,
offset: None,
patch: None,
bytes: KVec::new(),
}
}
fn parse(self) -> Result<KVec<u8>> {
// Fetch all the required data.
self.fetch_checksum()?;
self.fetch_offset()?;
self.fetch_patch()?;
self.fetch_byes()?;
// Doesn't do anything if `checksum == None`.
self.validate_checksum()?;
// Doesn't do anything if `offset == None`.
self.apply_offset()?;
// Doesn't do anything if `patch == None`.
self.apply_patch()?;
// Return the final image.
self.image
}
}I think the pattern here is the same, but in this example you keep working with the DataParser, instead of a new instance of Data.
Quoted 29 lines
Quoted 21 lines
I understand that there are cases where we can't omit it, and for obvious reasons the Vbios code is probably a perfect example for that.
However, I recommend looking at this from top to bottom: Do the "final" structures that we expose to the driver from the Vbios module have fields that are *really* optional? Or is the Option type just a result from the parsing process?
If it's the latter, we should get rid of it and work with a different type during the parsing process and then create the final instance that is exposed to the driver at the end.
For instance FwSecBiosImage is defined as:
pub(crate) struct FwSecBiosImage { base: BiosImageBase, falcon_data_offset: Option<usize>, pmu_lookup_table: Option<PmuLookupTable>, falcon_ucode_offset: Option<usize>, }Quoted 6 lines
Do only *some* FwSecBiosImage instances have a
falcon_ucode_offset?If the answer is 'no' then it shouldn't be an Option. If the answer is 'yes', then this indicates that FwSecBiosImage is probably too generic and should be split into more specific types of a FwSecBiosImage which instead share a common trait in order to treat the different types generically.
Understood, thanks.
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 51 lines · Danilo Krummrich
Quoted 7 lines · Joel Fernandes
Quoted 3 lines · Danilo Krummrich
Quoted 2 lines · Joel Fernandes
Quoted 1 line · Danilo Krummrich
On Wed, May 07, 2025 at 10:52:43PM +0900, Alexandre Courbot wrote:
So the code here now looks like the below, definitely better, thanks! :
Quoted 16 lines
if let (Some(second_ref), Some(first), Some(pci_at)) = (second.as_mut(), first_fwsec_image, pci_at_image) { second_ref .setup_falcon_data(pdev, &pci_at, &first) .inspect_err(|e| { dev_err!(..) })?; Ok(Vbios { fwsec_image: second.take().ok_or(EINVAL)? }) } else { dev_err!( pdev.as_ref(), "Missing required images for falcon data setup, skipping\n" ); Err(EINVAL) }Sorry, my code-snipped was incorrect indeed. Let me paste what I actually intended (and this time properly compile checked) and should be even better:
if let (Some(mut second), Some(first), Some(pci_at)) = (second_fwsec_image, first_fwsec_image, pci_at_image) { second .setup_falcon_data(pdev, &pci_at, &first) .inspect_err(|e| { dev_err!(pdev.as_ref(), "Falcon data setup failed: {:?}\n", e) })?; Ok(Vbios(second)) } else { dev_err!( pdev.as_ref(), "Missing required images for falcon data setup, skipping\n" ); Err(EINVAL) }So, with this second is the actual value and not just a reference. :)
And the methods can become:
Quoted 11 lines
pub(crate) fn fwsec_header(&self, pdev: &device::Device) -> Result<&FalconUCodeDescV3> {
self.0.fwsec_header(pdev)
}
pub(crate) fn fwsec_ucode(&self, pdev: &device::Device) -> Result<&[u8]> {
self.0.fwsec_ucode(pdev, self.fwsec_header(pdev)?)
}
pub(crate) fn fwsec_sigs(&self, pdev: &device::Device) -> Result<&[u8]> {
self.0.fwsec_sigs(pdev, self.fwsec_header(pdev)?)
}I made this change and it LGTM. Thanks! I did not do the '.0' though since I want to keep the readability, lets see in the next revision if that looks good.
Quoted 88 lines
Quoted 18 lines
Quoted 7 lines
In general, I feel like a lot of those Option come from a programming pattern that is very common in C, i.e. allocate a structure (stack or heap) and then initialize its fields.
In Rust you should aim to initialize all the fields of a structure when you create the instance. Option as a return type of a function is common, but it's always a bit suspicious when there is an Option field in a struct.
I looked into it, I could not git rid of those ones because we need to initialize in the "impl TryFrom<BiosImageBase> for BiosImage {"
0xE0 => Ok(BiosImage::FwSec(FwSecBiosImage { base, falcon_data_offset: None, pmu_lookup_table: None, falcon_ucode_offset: None, })),Quoted 3 lines
And these fields will not be determined until much later, because as is the case with the earlier example, these fields cannot be determined until all the images are parsed.
You should not use TryFrom, but instead use a normal constructor, such as
BiosImage::new(base_bios_image)and do the parsing within this constructor.
If you want a helper type with Options while parsing that's totally fine, but the final result can clearly be without Options. For instance:
struct Data { image: KVec<u8>, } impl Data { fn new() -> Result<Self> { let parser = DataParser::new(); Self { image: parser.parse()? } } fn load_image(&self) { ... } } struct DataParser { // Only some images have a checksum. checksum: Option<u64>, // Some images have an extra offset. offset: Option<u64>, // Some images need to be patched. patch: Option<KVec<u8>>, image: KVec<u8>, } impl DataParser { fn new() -> Self { Self { checksum: None, offset: None, patch: None, bytes: KVec::new(), } } fn parse(self) -> Result<KVec<u8>> { // Fetch all the required data. self.fetch_checksum()?; self.fetch_offset()?; self.fetch_patch()?; self.fetch_byes()?; // Doesn't do anything if `checksum == None`. self.validate_checksum()?; // Doesn't do anything if `offset == None`. self.apply_offset()?; // Doesn't do anything if `patch == None`. self.apply_patch()?; // Return the final image. self.image } }
Quoted 2 lines
I think the pattern here is the same, but in this example you keep working with the DataParser, instead of a new instance of Data.
I think this would be a fundamental rewrite of the patch. I am Ok with looking into it as a future item, but right now I am not sure if it justifies not using Option for these few. There's a lot of immediate work we have to do for boot, lets please not block the patch on just this if that's Ok with you. If you want, I could add a TODO here.
thanks,
- Joel
Danilo Krummrich
maintainerRe: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 4 lines · Joel Fernandes
On 5/20/2025 11:01 AM, Danilo Krummrich wrote:
I made this change and it LGTM. Thanks! I did not do the '.0' though since I want to keep the readability, lets see in the next revision if that looks good.
I think readability, is just as good with .0, but I'm fine with either.
Quoted 88 lines
Quoted 88 lines
Quoted 18 lines
Quoted 7 lines
In general, I feel like a lot of those Option come from a programming pattern that is very common in C, i.e. allocate a structure (stack or heap) and then initialize its fields.
In Rust you should aim to initialize all the fields of a structure when you create the instance. Option as a return type of a function is common, but it's always a bit suspicious when there is an Option field in a struct.
I looked into it, I could not git rid of those ones because we need to initialize in the "impl TryFrom<BiosImageBase> for BiosImage {"
0xE0 => Ok(BiosImage::FwSec(FwSecBiosImage { base, falcon_data_offset: None, pmu_lookup_table: None, falcon_ucode_offset: None, })),Quoted 3 lines
And these fields will not be determined until much later, because as is the case with the earlier example, these fields cannot be determined until all the images are parsed.
You should not use TryFrom, but instead use a normal constructor, such as
BiosImage::new(base_bios_image)and do the parsing within this constructor.
If you want a helper type with Options while parsing that's totally fine, but the final result can clearly be without Options. For instance:
struct Data { image: KVec<u8>, } impl Data { fn new() -> Result<Self> { let parser = DataParser::new(); Self { image: parser.parse()? } } fn load_image(&self) { ... } } struct DataParser { // Only some images have a checksum. checksum: Option<u64>, // Some images have an extra offset. offset: Option<u64>, // Some images need to be patched. patch: Option<KVec<u8>>, image: KVec<u8>, } impl DataParser { fn new() -> Self { Self { checksum: None, offset: None, patch: None, bytes: KVec::new(), } } fn parse(self) -> Result<KVec<u8>> { // Fetch all the required data. self.fetch_checksum()?; self.fetch_offset()?; self.fetch_patch()?; self.fetch_byes()?; // Doesn't do anything if `checksum == None`. self.validate_checksum()?; // Doesn't do anything if `offset == None`. self.apply_offset()?; // Doesn't do anything if `patch == None`. self.apply_patch()?; // Return the final image. self.image } }
Quoted 8 lines
Quoted 2 lines
I think the pattern here is the same, but in this example you keep working with the DataParser, instead of a new instance of Data.
I think this would be a fundamental rewrite of the patch. I am Ok with looking into it as a future item, but right now I am not sure if it justifies not using Option for these few. There's a lot of immediate work we have to do for boot, lets please not block the patch on just this if that's Ok with you. If you want, I could add a TODO here.
Honestly, I don't think it'd be too bad to fix this up. It's "just" a bit of juggling fields and moving code around. The actual code should not change much.
Having Option<T> where the corresponding value T isn't actually optional is extremely confusing and makes it hard for everyone, but especially new contributors, to understand the code and can easily trick people into taking wrong assumptions.
Making the code reasonably accessible for (new) contributors is one of the objectives of nova and one of the learnings from nouveau.
Hence, let's get this right from the get-go please.
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 11 lines · Danilo Krummrich
Quoted 4 lines · Joel Fernandes
On 5/20/2025 11:01 AM, Danilo Krummrich wrote:
I made this change and it LGTM. Thanks! I did not do the '.0' though since I want to keep the readability, lets see in the next revision if that looks good.
I think readability, is just as good with
.0, but I'm fine with either.
Cool.
Quoted 106 lines
Quoted 89 lines
Quoted 89 lines
Quoted 19 lines
Quoted 8 lines
In general, I feel like a lot of those Option come from a programming pattern that is very common in C, i.e. allocate a structure (stack or heap) and then initialize its fields.
In Rust you should aim to initialize all the fields of a structure when you create the instance. Option as a return type of a function is common, but it's always a bit suspicious when there is an Option field in a struct.
I looked into it, I could not git rid of those ones because we need to initialize in the "impl TryFrom<BiosImageBase> for BiosImage {"
0xE0 => Ok(BiosImage::FwSec(FwSecBiosImage { base, falcon_data_offset: None, pmu_lookup_table: None, falcon_ucode_offset: None, })),Quoted 3 lines
And these fields will not be determined until much later, because as is the case with the earlier example, these fields cannot be determined until all the images are parsed.
You should not use TryFrom, but instead use a normal constructor, such as
BiosImage::new(base_bios_image)and do the parsing within this constructor.
If you want a helper type with Options while parsing that's totally fine, but the final result can clearly be without Options. For instance:
struct Data { image: KVec<u8>, } impl Data { fn new() -> Result<Self> { let parser = DataParser::new();Self { image: parser.parse()? } } fn load_image(&self) { ... }}
struct DataParser { // Only some images have a checksum. checksum: Option<u64>, // Some images have an extra offset. offset: Option<u64>, // Some images need to be patched. patch: Option<KVec<u8>>, image: KVec<u8>, } impl DataParser { fn new() -> Self { Self { checksum: None, offset: None, patch: None, bytes: KVec::new(), } } fn parse(self) -> Result<KVec<u8>> { // Fetch all the required data. self.fetch_checksum()?; self.fetch_offset()?; self.fetch_patch()?; self.fetch_byes()?; // Doesn't do anything if `checksum == None`. self.validate_checksum()?; // Doesn't do anything if `offset == None`. self.apply_offset()?; // Doesn't do anything if `patch == None`. self.apply_patch()?; // Return the final image. self.image } }Quoted 8 lines
Quoted 2 lines
I think the pattern here is the same, but in this example you keep working with the DataParser, instead of a new instance of Data.
I think this would be a fundamental rewrite of the patch. I am Ok with looking into it as a future item, but right now I am not sure if it justifies not using Option for these few. There's a lot of immediate work we have to do for boot, lets please not block the patch on just this if that's Ok with you. If you want, I could add a TODO here.
Honestly, I don't think it'd be too bad to fix this up. It's "just" a bit of juggling fields and moving code around. The actual code should not change much.
Having Option<T> where the corresponding value T isn't actually optional is extremely confusing and makes it hard for everyone, but especially new contributors, to understand the code and can easily trick people into taking wrong assumptions.
Quoted 4 lines
Making the code reasonably accessible for (new) contributors is one of the objectives of nova and one of the learnings from nouveau.
Hence, let's get this right from the get-go please.
Ok, I will look into making this change. :-)
thanks,
- Joel
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
In reply to Danilo KrummrichQuoted 71 lines · Danilo Krummrich
Quoted 60 lines
Quoted 60 lines
If you want a helper type with Options while parsing that's totally fine, but the final result can clearly be without Options. For instance:
struct Data { image: KVec<u8>, } impl Data { fn new() -> Result<Self> { let parser = DataParser::new(); Self { image: parser.parse()? } } fn load_image(&self) { ... } } struct DataParser { // Only some images have a checksum. checksum: Option<u64>, // Some images have an extra offset. offset: Option<u64>, // Some images need to be patched. patch: Option<KVec<u8>>, image: KVec<u8>, } impl DataParser { fn new() -> Self { Self { checksum: None, offset: None, patch: None, bytes: KVec::new(), } } fn parse(self) -> Result<KVec<u8>> { // Fetch all the required data. self.fetch_checksum()?; self.fetch_offset()?; self.fetch_patch()?; self.fetch_byes()?; // Doesn't do anything if `checksum == None`. self.validate_checksum()?; // Doesn't do anything if `offset == None`. self.apply_offset()?; // Doesn't do anything if `patch == None`. self.apply_patch()?; // Return the final image. self.image } }Quoted 7 lines
Quoted 2 lines
I think the pattern here is the same, but in this example you keep working with the DataParser, instead of a new instance of Data.
I think this would be a fundamental rewrite of the patch. I am Ok with looking into it as a future item, but right now I am not sure if it justifies not using Option for these few. There's a lot of immediate work we have to do for boot, lets please not block the patch on just this if that's Ok with you. If you want, I could add a TODO here.
Honestly, I don't think it'd be too bad to fix this up. It's "just" a bit of juggling fields and moving code around. The actual code should not change much.
Quoted 7 lines
Having Option<T> where the corresponding value T isn't actually optional is extremely confusing and makes it hard for everyone, but especially new contributors, to understand the code and can easily trick people into taking wrong assumptions.
Making the code reasonably accessible for (new) contributors is one of the objectives of nova and one of the learnings from nouveau.
I implemented the Data parsing pattern like the following, the final FwSecBiosImage will not have optional fields as you suggested. It does get rid of 2 additional fields as well which are not needed after vbios parsing completes.
Hope that looks Ok now! I am open to naming FwSecBiosPartial as FwSecBiosData if that's better.
The full file after the change:
https://git.kernel.org/pub/scm/linux/kernel/git/jfern/linux.git/tree/drivers/gpu/nova-core/vbios.rs?h=nova/vbios&id=8cc852fe5573890596a91a2a935b3af24dcb9f04
thanks,
- Joel
Dave Airlie
maintainerRe: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 75 lines · Joel Fernandes
Quoted 71 lines · Danilo Krummrich
Quoted 60 lines
Quoted 60 lines
If you want a helper type with Options while parsing that's totally fine, but the final result can clearly be without Options. For instance:
struct Data { image: KVec<u8>, } impl Data { fn new() -> Result<Self> { let parser = DataParser::new(); Self { image: parser.parse()? } } fn load_image(&self) { ... } } struct DataParser { // Only some images have a checksum. checksum: Option<u64>, // Some images have an extra offset. offset: Option<u64>, // Some images need to be patched. patch: Option<KVec<u8>>, image: KVec<u8>, } impl DataParser { fn new() -> Self { Self { checksum: None, offset: None, patch: None, bytes: KVec::new(), } } fn parse(self) -> Result<KVec<u8>> { // Fetch all the required data. self.fetch_checksum()?; self.fetch_offset()?; self.fetch_patch()?; self.fetch_byes()?; // Doesn't do anything if `checksum == None`. self.validate_checksum()?; // Doesn't do anything if `offset == None`. self.apply_offset()?; // Doesn't do anything if `patch == None`. self.apply_patch()?; // Return the final image. self.image } }Quoted 7 lines
Quoted 2 lines
I think the pattern here is the same, but in this example you keep working with the DataParser, instead of a new instance of Data.
I think this would be a fundamental rewrite of the patch. I am Ok with looking into it as a future item, but right now I am not sure if it justifies not using Option for these few. There's a lot of immediate work we have to do for boot, lets please not block the patch on just this if that's Ok with you. If you want, I could add a TODO here.
Honestly, I don't think it'd be too bad to fix this up. It's "just" a bit of juggling fields and moving code around. The actual code should not change much.
Quoted 7 lines
Quoted 7 lines
Having Option<T> where the corresponding value T isn't actually optional is extremely confusing and makes it hard for everyone, but especially new contributors, to understand the code and can easily trick people into taking wrong assumptions.
Making the code reasonably accessible for (new) contributors is one of the objectives of nova and one of the learnings from nouveau.
I just want to back Danilo up on this concept as well.
When I did the experiments code, I faced the not fully constructed object problem a lot, and I tried to resist the C pattern of Option<> all the things, it's a very C based thing where we create an object then initialise it as we go, and it's not a great pattern to have for rust code.
I'm not a huge fan of constructor/builder objects either if they can be avoided, please do, and I tried to also avoid proliferating them, but I think for most things we can build the pieces and then the final object as we go, it just requires doing so from the start, and not giving into the Option<> pattern.
Dave.
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 92 lines · Dave Airlie
Quoted 75 lines · Joel Fernandes
Quoted 71 lines · Danilo Krummrich
Quoted 60 lines
Quoted 60 lines
If you want a helper type with Options while parsing that's totally fine, but the final result can clearly be without Options. For instance:
struct Data { image: KVec<u8>, } impl Data { fn new() -> Result<Self> { let parser = DataParser::new(); Self { image: parser.parse()? } } fn load_image(&self) { ... } } struct DataParser { // Only some images have a checksum. checksum: Option<u64>, // Some images have an extra offset. offset: Option<u64>, // Some images need to be patched. patch: Option<KVec<u8>>, image: KVec<u8>, } impl DataParser { fn new() -> Self { Self { checksum: None, offset: None, patch: None, bytes: KVec::new(), } } fn parse(self) -> Result<KVec<u8>> { // Fetch all the required data. self.fetch_checksum()?; self.fetch_offset()?; self.fetch_patch()?; self.fetch_byes()?; // Doesn't do anything if `checksum == None`. self.validate_checksum()?; // Doesn't do anything if `offset == None`. self.apply_offset()?; // Doesn't do anything if `patch == None`. self.apply_patch()?; // Return the final image. self.image } }Quoted 7 lines
Quoted 2 lines
I think the pattern here is the same, but in this example you keep working with the DataParser, instead of a new instance of Data.
I think this would be a fundamental rewrite of the patch. I am Ok with looking into it as a future item, but right now I am not sure if it justifies not using Option for these few. There's a lot of immediate work we have to do for boot, lets please not block the patch on just this if that's Ok with you. If you want, I could add a TODO here.
Honestly, I don't think it'd be too bad to fix this up. It's "just" a bit of juggling fields and moving code around. The actual code should not change much.
Quoted 7 lines
Quoted 7 lines
Having Option<T> where the corresponding value T isn't actually optional is extremely confusing and makes it hard for everyone, but especially new contributors, to understand the code and can easily trick people into taking wrong assumptions.
Making the code reasonably accessible for (new) contributors is one of the objectives of nova and one of the learnings from nouveau.
I just want to back Danilo up on this concept as well.
When I did the experiments code, I faced the not fully constructed object problem a lot, and I tried to resist the C pattern of Option<> all the things, it's a very C based thing where we create an object then initialise it as we go, and it's not a great pattern to have for rust code.
Quoted 5 lines
I'm not a huge fan of constructor/builder objects either if they can be avoided, please do, and I tried to also avoid proliferating them, but I think for most things we can build the pieces and then the final object as we go, it just requires doing so from the start, and not giving into the Option<> pattern.
Sure, I am on the same page there as well. For next revision of this patch, struct Vbios will contain a struct FwsecBiosImage without any Option in either struct Vbios or struct FwsecBiosImage. This is only logical, because there is nothing optional about it (in what Vbios::new() returns).
thanks,
- Joel
Danilo Krummrich
maintainerRe: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
In reply to Alexandre CourbotQuoted 19 lines · Alexandre Courbot
drivers/gpu/nova-core/vbios.rs new file +19
-0+/// PCI Data Structure as defined in PCI Firmware Specification +#[derive( Debug , Clone )] +#[repr( C )] +struct PcirStruct { +/// PCI Data Structure signature ("PCIR" or "NPDS") +pub signature: [u8 ;4 ], +/// PCI Vendor ID (e.g., 0x10DE for NVIDIA) +pub vendor_id:u16 , +/// PCI Device ID +pub device_id:u16 , +/// Device List Pointer +pub device_list_ptr:u16 , +/// PCI Data Structure Length +pub pci_data_struct_len:u16 , +/// PCI Data Structure Revision +pub pci_data_struct_rev:u8 , +/// Class code (3 bytes, 0x03 for display controller) +pub class_code: [u8 ;3 ], +/// Size of this image in 512-byte blocks
drivers/gpu/nova-core/vbios.rs line 306
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ pub image_len: u16, +/// Revision Level of the Vendor's ROM + pub vendor_rom_rev: u16, +/// ROM image type (0x00 = PC-AT compatible, 0x03 = EFI, 0x70 = NBSI) + pub code_type: u8, +/// Last image indicator (0x00 = Not last image, 0x80 = Last image) + pub last_image: u8, +/// Maximum Run-time Image Length (units of 512 bytes) + pub max_runtime_image_len: u16, +}
Here and in a couple more cases below, please don't use pub for fields of private structures.
Quoted 28 lines
drivers/gpu/nova-core/vbios.rs new file +28
-0+ +impl PcirStruct { +fn new (pdev: &pci::Device , data: &[u8 ]) ->Result <Self > { +if data.len () < core::mem::size_of ::<PcirStruct >() { +dev_err! (pdev.as_ref (),"Not enough data for PcirStruct\n" ); +return Err (EINVAL ); + } + +let mut signature = [0u8 ;4 ]; + signature.copy_from_slice (&data[0 ..4 ]); + +// Signature should be "PCIR" (0x52494350) or "NPDS" (0x5344504e) +if &signature !=b"PCIR" && &signature !=b"NPDS" { +dev_err! ( + pdev.as_ref (), +"Invalid signature for PcirStruct: {:?}\n" , + signature + ); +return Err (EINVAL ); + } + +let mut class_code = [0u8 ;3 ]; + class_code.copy_from_slice (&data[13 ..16 ]); + +Ok (PcirStruct { + signature, + vendor_id: u16::from_le_bytes ([data[4 ], data[5 ]]), + device_id: u16::from_le_bytes ([data[6 ], data[7 ]]),
drivers/gpu/nova-core/vbios.rs line 344
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ device_list_ptr: u16::from_le_bytes([data[8], data[9]]), + pci_data_struct_len: u16::from_le_bytes([data[10], data[11]]), + pci_data_struct_rev: data[12], + class_code, + image_len: u16::from_le_bytes([data[16], data[17]]), + vendor_rom_rev: u16::from_le_bytes([data[18], data[19]]), + code_type: data[20], + last_image: data[21], + max_runtime_image_len: u16::from_le_bytes([data[22], data[23]]), + })
Quite some of those fields seem unused, do we still want to have them? Same for other structures below.
Quoted 74 lines
drivers/gpu/nova-core/vbios.rs new file +74
-0+ } + +/// Check if this is the last image in the ROM +fn is_last (&self) ->bool { + self.last_image &LAST_IMAGE_BIT_MASK !=0 + } + +/// Calculate image size in bytes +fn image_size_bytes (&self) ->Result <usize > { +if self.image_len >0 { +// Image size is in 512-byte blocks +Ok (self.image_lenas usize *512 ) + }else { +Err (EINVAL ) + } + } +} + +/// BIOS Information Table (BIT) Header +/// This is the head of the BIT table, that is used to locate the Falcon data. +/// The BIT table (with its header) is in the PciAtBiosImage and the falcon data +/// it is pointing to is in the FwSecBiosImage. +#[derive( Debug , Clone , Copy )] +#[expect(dead_code)] +struct BitHeader { +/// 0h: BIT Header Identifier (BMP=0x7FFF/BIT=0xB8FF) +pub id:u16 , +/// 2h: BIT Header Signature ("BIT\0") +pub signature: [u8 ;4 ], +/// 6h: Binary Coded Decimal Version, ex: 0x0100 is 1.00. +pub bcd_version:u16 , +/// 8h: Size of BIT Header (in bytes) +pub header_size:u8 , +/// 9h: Size of BIT Tokens (in bytes) +pub token_size:u8 , +/// 10h: Number of token entries that follow +pub token_entries:u8 , +/// 11h: BIT Header Checksum +pub checksum:u8 , +} + +impl BitHeader { +fn new (data: &[u8 ]) ->Result <Self > { +if data.len () <12 { +return Err (EINVAL ); + } + +let mut signature = [0u8 ;4 ]; + signature.copy_from_slice (&data[2 ..6 ]); + +// Check header ID and signature +let id = u16::from_le_bytes ([data[0 ], data[1 ]]); +if id !=0xB8FF || &signature !=b"BIT\0" { +return Err (EINVAL ); + } + +Ok (BitHeader { + id, + signature, + bcd_version: u16::from_le_bytes ([data[6 ], data[7 ]]), + header_size: data[8 ], + token_size: data[9 ], + token_entries: data[10 ], + checksum: data[11 ], + }) + } +} + +/// BIT Token Entry: Records in the BIT table followed by the BIT header +#[derive( Debug , Clone , Copy )] +#[expect(dead_code)] +struct BitToken { +/// 00h: Token identifier +pub id:u8 ,
drivers/gpu/nova-core/vbios.rs line 428
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+/// 01h: Version of the token data + pub data_version: u8, +/// 02h: Size of token data in bytes + pub data_size: u16, +/// 04h: Offset to the token data + pub data_offset: u16, +} + +// Define the token ID for the Falcon data +pub (in crate ::vbios)const BIT_TOKEN_ID_FALCON_DATA :u8 =0x70 ;
This can just be private.
drivers/gpu/nova-core/vbios.rs line 432
Quoted 4 lines
drivers/gpu/nova-core/vbios.rs new file +4
-0+ +impl BitToken { +/// Find a BIT token entry by BIT ID in a PciAtBiosImage +pub (in crate ::vbios)fn from_id (image: &PciAtBiosImage , token_id:u8 ) ->Result <Self > {
Same here.
<snip>
drivers/gpu/nova-core/vbios.rs line 724
Quoted 4 lines
drivers/gpu/nova-core/vbios.rs new file +4
-0+struct PciAtBiosImage { + base:BiosImageBase , + bit_header:Option <BitHeader >, + bit_offset:Option <usize >,
Why are those Options? AFAICS, this structure is only ever created from
impl TryFrom<BiosImageBase> for PciAtBiosImage
and there you fail if you can't find the bit header anyways.
Also BitToken::from_id fails if bit_header == None, and it doesn't seem to be used anywhere else.
I think we should remove the Option wrapper for both.
<snip>
Quoted 30 lines
drivers/gpu/nova-core/vbios.rs new file +30
-0+/// The PmuLookupTableEntry structure is used to find the PmuLookupTableEntry +/// for a given application ID. The table of entries is pointed to by the falcon +/// data pointer in the BIT table, and is used to locate the Falcon Ucode. +#[expect(dead_code)] +struct PmuLookupTable { + version:u8 , + header_len:u8 , + entry_len:u8 , + entry_count:u8 , + table_data:KVec <u8 >, +} + +impl PmuLookupTable { +fn new (data: &[u8 ]) ->Result <Self > { +if data.len () <4 { +return Err (EINVAL ); + } + +let header_len = data[1 ]as usize ; +let entry_len = data[2 ]as usize ; +let entry_count = data[3 ]as usize ; + +let required_bytes = header_len + (entry_count * entry_len); + +if data.len () < required_bytes { +return Err (EINVAL ); + } + +// Create a copy of only the table data +let mut table_data =KVec ::new ();
drivers/gpu/nova-core/vbios.rs line 973
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ +// "last_entry_bytes" is a debugging aid. +let mut last_entry_bytes:Option <KVec <u8 >> =if cfg! (debug_assertions) { +Some (KVec ::new ()) + }else { +None + }; + +for &bytein &data[header_len..required_bytes] { + table_data.push (byte,GFP_KERNEL )?;
This should just be
table_data.extend_from_slice(&data[header_len..required_bytes], GFP_KERNEL)?;
so you don't need the loop and potentially lots of re-allocations.
Subsequently you can implement the debugging stuff as
if cfg!(debug_assertions) {
let mut last_entry_bytes = KVec::new();
for &byte in &data[header_len..required_bytes] {
// Debugging (dumps the table data to dmesg):
last_entry_bytes.push(byte, GFP_KERNEL)?;
let last_entry_bytes_len = last_entry_bytes.len();
if last_entry_bytes_len == entry_len {
pr_info!("Last entry bytes: {:02x?}\n", &last_entry_bytes[..]);
last_entry_bytes = KVec::new();
}
}
}In general, I feel like this patch utilizes the Option type way too much and often without actual need. Can you please also double check?
drivers/gpu/nova-core/vbios.rs line 982
Quoted 9 lines
drivers/gpu/nova-core/vbios.rs new file +9
-0+ +if cfg! (debug_assertions) { +// Debugging (dumps the table data to dmesg): +if let Some (ref mut last_entry_bytes) = last_entry_bytes { + last_entry_bytes.push (byte,GFP_KERNEL )?; + +let last_entry_bytes_len = last_entry_bytes.len (); +if last_entry_bytes_len == entry_len { +pr_info! ("Last entry bytes: {:02x?}\n" , &last_entry_bytes[..]);
Please use dev_dbg!().
Quoted 37 lines
drivers/gpu/nova-core/vbios.rs new file +37
-0+ *last_entry_bytes =KVec ::new (); + } + } + } + } + +Ok (PmuLookupTable { + version: data[0 ], + header_len: header_lenas u8 , + entry_len: entry_lenas u8 , + entry_count: entry_countas u8 , + table_data, + }) + } + +fn lookup_index (&self, idx:u8 ) ->Result <PmuLookupTableEntry > { +if idx >= self.entry_count { +return Err (EINVAL ); + } + +let index = (idxas usize ) * self.entry_lenas usize ; +PmuLookupTableEntry ::new (&self.table_data[index..]) + } + +// find entry by type value +fn find_entry_by_type (&self, entry_type:u8 ) ->Result <PmuLookupTableEntry > { +for iin 0 ..self.entry_count { +let entry = self.lookup_index (i)?; +if entry.application_id == entry_type { +return Ok (entry); + } + } + +Err (EINVAL ) + } +} +
drivers/gpu/nova-core/vbios.rs line 1029
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+/// The FwSecBiosImage structure contains the PMU table and the Falcon Ucode. +/// The PMU table contains voltage/frequency tables as well as a pointer to the +/// Falcon Ucode. +impl FwSecBiosImage { +fn setup_falcon_data ( + &mut self, + pdev: &pci::Device , + pci_at_image: &PciAtBiosImage , + first_fwsec_image: &FwSecBiosImage , + ) ->Result <()> {
Just Result will do.
Quoted 40 lines
drivers/gpu/nova-core/vbios.rs new file +40
-0+let mut offset = pci_at_image.falcon_data_ptr (pdev)?as usize ; + +// The falcon data pointer assumes that the PciAt and FWSEC images +// are contiguous in memory. However, testing shows the EFI image sits in +// between them. So calculate the offset from the end of the PciAt image +// rather than the start of it. Compensate. + offset -= pci_at_image.base.data.len (); + +// The offset is now from the start of the first Fwsec image, however +// the offset points to a location in the second Fwsec image. Since +// the fwsec images are contiguous, subtract the length of the first Fwsec +// image from the offset to get the offset to the start of the second +// Fwsec image. + offset -= first_fwsec_image.base.data.len (); + + self.falcon_data_offset =Some (offset); + +// The PmuLookupTable starts at the offset of the falcon data pointer + self.pmu_lookup_table =Some (PmuLookupTable ::new (&self.base.data[offset..])?); + +match self + .pmu_lookup_table + .as_ref () + .ok_or (EINVAL )? + .find_entry_by_type (FALCON_UCODE_ENTRY_APPID_FWSEC_PROD ) + { +Ok (entry) => { +let mut ucode_offset = entry.dataas usize ; + ucode_offset -= pci_at_image.base.data.len (); + ucode_offset -= first_fwsec_image.base.data.len (); + self.falcon_ucode_offset =Some (ucode_offset); +if cfg! (debug_assertions) { +// Print the v3_desc header for debugging +let v3_desc = self.fwsec_header (pdev.as_ref ())?; +pr_info! ("PmuLookupTableEntry v3_desc: {:#?}\n" , v3_desc); + } + } +Err (e) => { +dev_err! ( + pdev.as_ref (),
drivers/gpu/nova-core/vbios.rs line 1079
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ "PmuLookupTableEntry not found, error: {:?}\n", + e + ); + } + } +Ok (()) + } + +/// TODO: These were borrowed from the old code for integrating this module +/// with the outside world. They should be cleaned up and integrated properly.
Okay, won't review for now then. :)
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Hi Danilo,
Quoted 25 lines · Danilo Krummrich
Quoted 24 lines · Alexandre Courbot
drivers/gpu/nova-core/vbios.rs new file +24
-0+/// PCI Data Structure as defined in PCI Firmware Specification +#[derive( Debug , Clone )] +#[repr( C )] +struct PcirStruct { +/// PCI Data Structure signature ("PCIR" or "NPDS") +pub signature: [u8 ;4 ], +/// PCI Vendor ID (e.g., 0x10DE for NVIDIA) +pub vendor_id:u16 , +/// PCI Device ID +pub device_id:u16 , +/// Device List Pointer +pub device_list_ptr:u16 , +/// PCI Data Structure Length +pub pci_data_struct_len:u16 , +/// PCI Data Structure Revision +pub pci_data_struct_rev:u8 , +/// Class code (3 bytes, 0x03 for display controller) +pub class_code: [u8 ;3 ], +/// Size of this image in 512-byte blocks +pub image_len:u16 , +/// Revision Level of the Vendor's ROM +pub vendor_rom_rev:u16 , +/// ROM image type (0x00 = PC-AT compatible, 0x03 = EFI, 0x70 = NBSI) +pub code_type:u8 ,
Quoted 8 lines
drivers/gpu/nova-core/vbios.rs line 306
Quoted 5 lines
drivers/gpu/nova-core/vbios.rs new file +5
-0+/// Last image indicator (0x00 = Not last image, 0x80 = Last image) + pub last_image: u8, +/// Maximum Run-time Image Length (units of 512 bytes) + pub max_runtime_image_len: u16, +}Here and in a couple more cases below, please don't use pub for fields of private structures.
Fixed thanks.
Quoted 33 lines
Quoted 33 lines
drivers/gpu/nova-core/vbios.rs new file +33
-0+ +impl PcirStruct { +fn new (pdev: &pci::Device , data: &[u8 ]) ->Result <Self > { +if data.len () < core::mem::size_of ::<PcirStruct >() { +dev_err! (pdev.as_ref (),"Not enough data for PcirStruct\n" ); +return Err (EINVAL ); + } + +let mut signature = [0u8 ;4 ]; + signature.copy_from_slice (&data[0 ..4 ]); + +// Signature should be "PCIR" (0x52494350) or "NPDS" (0x5344504e) +if &signature !=b"PCIR" && &signature !=b"NPDS" { +dev_err! ( + pdev.as_ref (), +"Invalid signature for PcirStruct: {:?}\n" , + signature + ); +return Err (EINVAL ); + } + +let mut class_code = [0u8 ;3 ]; + class_code.copy_from_slice (&data[13 ..16 ]); + +Ok (PcirStruct { + signature, + vendor_id: u16::from_le_bytes ([data[4 ], data[5 ]]), + device_id: u16::from_le_bytes ([data[6 ], data[7 ]]), + device_list_ptr: u16::from_le_bytes ([data[8 ], data[9 ]]), + pci_data_struct_len: u16::from_le_bytes ([data[10 ], data[11 ]]), + pci_data_struct_rev: data[12 ], + class_code, + image_len: u16::from_le_bytes ([data[16 ], data[17 ]]),
Quoted 8 lines
drivers/gpu/nova-core/vbios.rs line 344
Quoted 5 lines
drivers/gpu/nova-core/vbios.rs new file +5
-0+ vendor_rom_rev: u16::from_le_bytes([data[18], data[19]]), + code_type: data[20], + last_image: data[21], + max_runtime_image_len: u16::from_le_bytes([data[22], data[23]]), + })Quite some of those fields seem unused, do we still want to have them? Same for other structures below.
I think we discussed this in the previous posting as well. As such, I am not keen on removing unused fields of structures part of 'standard' specifications since I only see drawbacks of doing so:
1. Obfuscation
2. Replacement of the fields with some kind of padding so that size_of() works.
3. Goes in the opposite direction of documentation and transparency in how the structures work.
4. Partially filling structures.
Quoted 77 lines
Quoted 77 lines
drivers/gpu/nova-core/vbios.rs new file +77
-0+ +/// Check if this is the last image in the ROM +fn is_last (&self) ->bool { + self.last_image &LAST_IMAGE_BIT_MASK !=0 + } + +/// Calculate image size in bytes +fn image_size_bytes (&self) ->Result <usize > { +if self.image_len >0 { +// Image size is in 512-byte blocks +Ok (self.image_lenas usize *512 ) + }else { +Err (EINVAL ) + } + } +} + +/// BIOS Information Table (BIT) Header +/// This is the head of the BIT table, that is used to locate the Falcon data. +/// The BIT table (with its header) is in the PciAtBiosImage and the falcon data +/// it is pointing to is in the FwSecBiosImage. +#[derive( Debug , Clone , Copy )] +#[expect(dead_code)] +struct BitHeader { +/// 0h: BIT Header Identifier (BMP=0x7FFF/BIT=0xB8FF) +pub id:u16 , +/// 2h: BIT Header Signature ("BIT\0") +pub signature: [u8 ;4 ], +/// 6h: Binary Coded Decimal Version, ex: 0x0100 is 1.00. +pub bcd_version:u16 , +/// 8h: Size of BIT Header (in bytes) +pub header_size:u8 , +/// 9h: Size of BIT Tokens (in bytes) +pub token_size:u8 , +/// 10h: Number of token entries that follow +pub token_entries:u8 , +/// 11h: BIT Header Checksum +pub checksum:u8 , +} + +impl BitHeader { +fn new (data: &[u8 ]) ->Result <Self > { +if data.len () <12 { +return Err (EINVAL ); + } + +let mut signature = [0u8 ;4 ]; + signature.copy_from_slice (&data[2 ..6 ]); + +// Check header ID and signature +let id = u16::from_le_bytes ([data[0 ], data[1 ]]); +if id !=0xB8FF || &signature !=b"BIT\0" { +return Err (EINVAL ); + } + +Ok (BitHeader { + id, + signature, + bcd_version: u16::from_le_bytes ([data[6 ], data[7 ]]), + header_size: data[8 ], + token_size: data[9 ], + token_entries: data[10 ], + checksum: data[11 ], + }) + } +} + +/// BIT Token Entry: Records in the BIT table followed by the BIT header +#[derive( Debug , Clone , Copy )] +#[expect(dead_code)] +struct BitToken { +/// 00h: Token identifier +pub id:u8 , +/// 01h: Version of the token data +pub data_version:u8 , +/// 02h: Size of token data in bytes +pub data_size:u16 ,
Quoted 8 lines
drivers/gpu/nova-core/vbios.rs line 428
Quoted 6 lines
drivers/gpu/nova-core/vbios.rs new file +6
-0+/// 04h: Offset to the token data + pub data_offset: u16, +} + +// Define the token ID for the Falcon data +pub (in crate ::vbios)const BIT_TOKEN_ID_FALCON_DATA :u8 =0x70 ;This can just be private.
Yep, fixed.
Quoted 6 lines
drivers/gpu/nova-core/vbios.rs line 432
Quoted 4 lines
drivers/gpu/nova-core/vbios.rs new file +4
-0+ +impl BitToken { +/// Find a BIT token entry by BIT ID in a PciAtBiosImage +pub (in crate ::vbios)fn from_id (image: &PciAtBiosImage , token_id:u8 ) ->Result <Self > {Same here.
Fixed.
Quoted 9 lines
<snip>
drivers/gpu/nova-core/vbios.rs line 724
Quoted 4 lines
drivers/gpu/nova-core/vbios.rs new file +4
-0+struct PciAtBiosImage { + base:BiosImageBase , + bit_header:Option <BitHeader >, + bit_offset:Option <usize >,Why are those Options? AFAICS, this structure is only ever created from
Quoted 8 lines
impl TryFrom<BiosImageBase> for PciAtBiosImage
and there you fail if you can't find the bit header anyways.
Also
BitToken::from_idfails ifbit_header== None, and it doesn't seem to be used anywhere else.I think we should remove the Option wrapper for both.
Yes, thanks. That does simplify the code, I made the change and it works.
Quoted 50 lines
Quoted 31 lines
drivers/gpu/nova-core/vbios.rs new file +30
-0+/// The PmuLookupTableEntry structure is used to find the PmuLookupTableEntry +/// for a given application ID. The table of entries is pointed to by the falcon +/// data pointer in the BIT table, and is used to locate the Falcon Ucode. +#[expect(dead_code)] +struct PmuLookupTable { + version:u8 , + header_len:u8 , + entry_len:u8 , + entry_count:u8 , + table_data:KVec <u8 >, +} + +impl PmuLookupTable { +fn new (data: &[u8 ]) ->Result <Self > { +if data.len () <4 { +return Err (EINVAL ); + } + +let header_len = data[1 ]as usize ; +let entry_len = data[2 ]as usize ; +let entry_count = data[3 ]as usize ; + +let required_bytes = header_len + (entry_count * entry_len); + +if data.len () < required_bytes { +return Err (EINVAL ); + } + +// Create a copy of only the table data +let mut table_data =KVec ::new ();drivers/gpu/nova-core/vbios.rs line 973
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ +// "last_entry_bytes" is a debugging aid. +let mut last_entry_bytes:Option <KVec <u8 >> =if cfg! (debug_assertions) { +Some (KVec ::new ()) + }else { +None + }; + +for &bytein &data[header_len..required_bytes] { + table_data.push (byte,GFP_KERNEL )?;This should just be
table_data.extend_from_slice(&data[header_len..required_bytes], GFP_KERNEL)?;so you don't need the loop and potentially lots of re-allocations.
Subsequently you can implement the debugging stuff as
Quoted 14 lines
if cfg!(debug_assertions) {
let mut last_entry_bytes = KVec::new();
for &byte in &data[header_len..required_bytes] {
// Debugging (dumps the table data to dmesg):
last_entry_bytes.push(byte, GFP_KERNEL)?;
let last_entry_bytes_len = last_entry_bytes.len();
if last_entry_bytes_len == entry_len {
pr_info!("Last entry bytes: {:02x?}\n", &last_entry_bytes[..]);
last_entry_bytes = KVec::new();
}
}
}Ok, that's better, I took the opportunity to replace this code with:
(Sorry for wrapping)
// Create a copy of only the table data
let data_entries = &data[header_len..required_bytes];
let table_data = {
let mut ret = KVec::new();
ret.extend_from_slice(&data_entries, GFP_KERNEL)?;
ret
};
// Debug logging of entries (dumps the table data to dmesg)
if cfg!(debug_assertions) {
for i in 0..entry_count {
pr_info!("PMU entry: {:02x?}\n", &data_entries[i * entry_len..(i
+ 1) * entry_len]);
}
}Quoted 2 lines
In general, I feel like this patch utilizes the Option type way too much and often without actual need. Can you please also double check?
Yeah, sorry, I'm somewhat new to rust. :-D. I am going through all my Options now.
I will continue addressing the rest of the comments and those in the other email and will reply soon. Thanks!
- Joel
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
In reply to Danilo KrummrichHi Danilo,
Quoted 3 lines · Danilo Krummrich
I feel like this patch utilizes the Option type way too much and
often without actual need. Can you please also double check?
I found one other instance (vbios.fwsec_image). Other than that, all others are required AFAICS.
Quoted 12 lines
drivers/gpu/nova-core/vbios.rs line 982
Quoted 9 lines
drivers/gpu/nova-core/vbios.rs new file +9
-0+ +if cfg! (debug_assertions) { +// Debugging (dumps the table data to dmesg): +if let Some (ref mut last_entry_bytes) = last_entry_bytes { + last_entry_bytes.push (byte,GFP_KERNEL )?; + +let last_entry_bytes_len = last_entry_bytes.len (); +if last_entry_bytes_len == entry_len { +pr_info! ("Last entry bytes: {:02x?}\n" , &last_entry_bytes[..]);Please use dev_dbg!().
This required passing down the pdev here, but did that, thanks.
Quoted 41 lines
Quoted 41 lines
drivers/gpu/nova-core/vbios.rs new file +41
-0+ *last_entry_bytes =KVec ::new (); + } + } + } + } + +Ok (PmuLookupTable { + version: data[0 ], + header_len: header_lenas u8 , + entry_len: entry_lenas u8 , + entry_count: entry_countas u8 , + table_data, + }) + } + +fn lookup_index (&self, idx:u8 ) ->Result <PmuLookupTableEntry > { +if idx >= self.entry_count { +return Err (EINVAL ); + } + +let index = (idxas usize ) * self.entry_lenas usize ; +PmuLookupTableEntry ::new (&self.table_data[index..]) + } + +// find entry by type value +fn find_entry_by_type (&self, entry_type:u8 ) ->Result <PmuLookupTableEntry > { +for iin 0 ..self.entry_count { +let entry = self.lookup_index (i)?; +if entry.application_id == entry_type { +return Ok (entry); + } + } + +Err (EINVAL ) + } +} + +/// The FwSecBiosImage structure contains the PMU table and the Falcon Ucode. +/// The PMU table contains voltage/frequency tables as well as a pointer to the +/// Falcon Ucode. +impl FwSecBiosImage {
Quoted 9 lines
drivers/gpu/nova-core/vbios.rs line 1029
Quoted 6 lines
drivers/gpu/nova-core/vbios.rs new file +6
-0+fn setup_falcon_data ( + &mut self, + pdev: &pci::Device , + pci_at_image: &PciAtBiosImage , + first_fwsec_image: &FwSecBiosImage , + ) ->Result <()> {Just Result will do.
Fixed.
Quoted 44 lines
Quoted 44 lines
drivers/gpu/nova-core/vbios.rs new file +44
-0+let mut offset = pci_at_image.falcon_data_ptr (pdev)?as usize ; + +// The falcon data pointer assumes that the PciAt and FWSEC images +// are contiguous in memory. However, testing shows the EFI image sits in +// between them. So calculate the offset from the end of the PciAt image +// rather than the start of it. Compensate. + offset -= pci_at_image.base.data.len (); + +// The offset is now from the start of the first Fwsec image, however +// the offset points to a location in the second Fwsec image. Since +// the fwsec images are contiguous, subtract the length of the first Fwsec +// image from the offset to get the offset to the start of the second +// Fwsec image. + offset -= first_fwsec_image.base.data.len (); + + self.falcon_data_offset =Some (offset); + +// The PmuLookupTable starts at the offset of the falcon data pointer + self.pmu_lookup_table =Some (PmuLookupTable ::new (&self.base.data[offset..])?); + +match self + .pmu_lookup_table + .as_ref () + .ok_or (EINVAL )? + .find_entry_by_type (FALCON_UCODE_ENTRY_APPID_FWSEC_PROD ) + { +Ok (entry) => { +let mut ucode_offset = entry.dataas usize ; + ucode_offset -= pci_at_image.base.data.len (); + ucode_offset -= first_fwsec_image.base.data.len (); + self.falcon_ucode_offset =Some (ucode_offset); +if cfg! (debug_assertions) { +// Print the v3_desc header for debugging +let v3_desc = self.fwsec_header (pdev.as_ref ())?; +pr_info! ("PmuLookupTableEntry v3_desc: {:#?}\n" , v3_desc); + } + } +Err (e) => { +dev_err! ( + pdev.as_ref (), +"PmuLookupTableEntry not found, error: {:?}\n" , + e + ); + }
Quoted 8 lines
drivers/gpu/nova-core/vbios.rs line 1079
Quoted 6 lines
drivers/gpu/nova-core/vbios.rs new file +6
-0+ } +Ok (()) + } + +/// TODO: These were borrowed from the old code for integrating this module +/// with the outside world. They should be cleaned up and integrated properly. Okay, won't review for now then. 🙂
In the next revision, we are removing this TODO and can continue review. :)
thanks,
- Joel
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
In reply to Alexandre CourbotQuoted 40 lines · Alexandre Courbot
drivers/gpu/nova-core/vbios.rs new file +40
-0+impl FwSecBiosImage { + fn setup_falcon_data( + &mut self, + pdev: &pci::Device, + pci_at_image: &PciAtBiosImage, + first_fwsec_image: &FwSecBiosImage, + ) -> Result<()> { + let mut offset = pci_at_image.falcon_data_ptr(pdev)? as usize; + + // The falcon data pointer assumes that the PciAt and FWSEC images + // are contiguous in memory. However, testing shows the EFI image sits in + // between them. So calculate the offset from the end of the PciAt image + // rather than the start of it. Compensate. + offset -= pci_at_image.base.data.len(); + + // The offset is now from the start of the first Fwsec image, however + // the offset points to a location in the second Fwsec image. Since + // the fwsec images are contiguous, subtract the length of the first Fwsec + // image from the offset to get the offset to the start of the second + // Fwsec image. + offset -= first_fwsec_image.base.data.len(); + + self.falcon_data_offset = Some(offset); + + // The PmuLookupTable starts at the offset of the falcon data pointer + self.pmu_lookup_table = Some(PmuLookupTable::new(&self.base.data[offset..])?); + + match self + .pmu_lookup_table + .as_ref() + .ok_or(EINVAL)? + .find_entry_by_type(FALCON_UCODE_ENTRY_APPID_FWSEC_PROD) + { + Ok(entry) => { + let mut ucode_offset = entry.data as usize; + ucode_offset -= pci_at_image.base.data.len(); + ucode_offset -= first_fwsec_image.base.data.len(); + self.falcon_ucode_offset = Some(ucode_offset); + if cfg!(debug_assertions) { + // Print the v3_desc header for debugging
drivers/gpu/nova-core/vbios.rs line 1072
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ let v3_desc = self.fwsec_header(pdev.as_ref())?; + pr_info!("PmuLookupTableEntry v3_desc: {:#?}\n", v3_desc); + } + } + Err(e) => { + dev_err!( + pdev.as_ref(), + "PmuLookupTableEntry not found, error: {:?}\n", + e + );
Shouldn't you return an error here?
If not, then maybe this should probably be dev_warn.
Re: [PATCH v3 16/19] nova-core: Add support for VBIOS ucode extraction for boot
Quoted 53 lines · Timur Tabi
Quoted 40 lines · Alexandre Courbot
drivers/gpu/nova-core/vbios.rs new file +40
-0+impl FwSecBiosImage { + fn setup_falcon_data( + &mut self, + pdev: &pci::Device, + pci_at_image: &PciAtBiosImage, + first_fwsec_image: &FwSecBiosImage, + ) -> Result<()> { + let mut offset = pci_at_image.falcon_data_ptr(pdev)? as usize; + + // The falcon data pointer assumes that the PciAt and FWSEC images + // are contiguous in memory. However, testing shows the EFI image sits in + // between them. So calculate the offset from the end of the PciAt image + // rather than the start of it. Compensate. + offset -= pci_at_image.base.data.len(); + + // The offset is now from the start of the first Fwsec image, however + // the offset points to a location in the second Fwsec image. Since + // the fwsec images are contiguous, subtract the length of the first Fwsec + // image from the offset to get the offset to the start of the second + // Fwsec image. + offset -= first_fwsec_image.base.data.len(); + + self.falcon_data_offset = Some(offset); + + // The PmuLookupTable starts at the offset of the falcon data pointer + self.pmu_lookup_table = Some(PmuLookupTable::new(&self.base.data[offset..])?); + + match self + .pmu_lookup_table + .as_ref() + .ok_or(EINVAL)? + .find_entry_by_type(FALCON_UCODE_ENTRY_APPID_FWSEC_PROD) + { + Ok(entry) => { + let mut ucode_offset = entry.data as usize; + ucode_offset -= pci_at_image.base.data.len(); + ucode_offset -= first_fwsec_image.base.data.len(); + self.falcon_ucode_offset = Some(ucode_offset); + if cfg!(debug_assertions) { + // Print the v3_desc header for debuggingdrivers/gpu/nova-core/vbios.rs line 1072
Quoted 10 lines
drivers/gpu/nova-core/vbios.rs new file +10
-0+ let v3_desc = self.fwsec_header(pdev.as_ref())?; + pr_info!("PmuLookupTableEntry v3_desc: {:#?}\n", v3_desc); + } + } + Err(e) => { + dev_err!( + pdev.as_ref(), + "PmuLookupTableEntry not found, error: {:?}\n", + e + );Shouldn't you return an error here?
Quoted 1 line
If not, then maybe this should probably be
dev_warn.
Good catch, fixed! Thanks,
- Joel
Alexandre Courbot
maintainerPATCH v3 17/19 gpu: nova-core: compute layout of the FRTS region
In reply to Alexandre CourbotFWSEC-FRTS is run with the desired address of the FRTS region as parameter, which we need to compute depending on some hardware parameters.
Do this in a FbLayout structure, that will be later extended to describe more memory regions used to boot the GSP.
--- drivers/gpu/nova-core/gpu.rs | 4 ++ drivers/gpu/nova-core/gsp.rs | 3 ++ drivers/gpu/nova-core/gsp/fb.rs | 108 +++++++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 27 ++++++++++ 5 files changed, 143 insertions(+)
drivers/gpu/nova-core/gpu.rs +4 -0
@@ -7,6 +7,7 @@use crate ::driver::Bar0 ;use crate ::falcon::{gsp::Gsp , sec2::Sec2 ,Falcon };use crate ::firmware::Firmware ; +use crate ::gsp::fb::FbLayout ;use crate ::regs;use crate ::util;use crate ::vbios::Vbios ;
@@ -239,6 +240,9 @@ pub(crate) fn new(let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?; +let fb_layout =FbLayout ::new (spec.chipset, bar)?; +dev_dbg! (pdev.as_ref (),"{:#x?}\n" , fb_layout); +let _bios =Vbios ::new (pdev, bar)?; Ok(pin_init!(Self {
drivers/gpu/nova-core/gsp.rs new file +3 -0
@@ -0,0 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0 + +pub (crate )mod fb;
drivers/gpu/nova-core/gsp/fb.rs new file +108 -0
108 lines not on this page · raw message
drivers/gpu/nova-core/nova_core.rs +1 -0
@@ -8,6 +8,7 @@mod falcon;mod firmware;mod gpu; +mod gsp;mod regs;mod util;mod vbios;
drivers/gpu/nova-core/regs.rs +27 -0
@@ -53,6 +53,12 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> { 23:0 adr_63_40 as u32; }); +register! (NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE @0x00100ce0 { +3 :0 lower_scaleas u8 ; +9 :4 lower_magas u8 ; +30 :30 ecc_mode_enabledas bool ; +}); +/* PGC6 */ register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK @0x00118128 {
@@ -64,6 +70,27 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> { 31:0 value as u32; }); +register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_42 @0x001183a4 { +31 :0 valueas u32 ; +}); + +/* PDISP */ + +register! (NV_PDISP_VGA_WORKSPACE_BASE @0x00625f04 { +3 :3 status_validas bool ; +31 :8 addras u32 ; +}); + +/* FUSE */ + +register! (NV_FUSE_STATUS_OPT_DISPLAY_MAXWELL @0x00021c04 { +0 :0 display_disabledas bool ; +}); + +register! (NV_FUSE_STATUS_OPT_DISPLAY_AMPERE @0x00820c04 { +0 :0 display_disabledas bool ; +}); +/* PFALCON */ register! (NV_PFALCON_FALCON_IRQSCLR @ +0x00000004 {
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 17/19] gpu: nova-core: compute layout of the FRTS region
Quoted 56 lines · Alexandre Courbot
FWSEC-FRTS is run with the desired address of the FRTS region as parameter, which we need to compute depending on some hardware parameters.
Do this in a
FbLayoutstructure, that will be later extended to describe more memory regions used to boot the GSP.Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/gpu.rs | 4 ++ drivers/gpu/nova-core/gsp.rs | 3 ++ drivers/gpu/nova-core/gsp/fb.rs | 108 +++++++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/nova_core.rs | 1 + drivers/gpu/nova-core/regs.rs | 27 ++++++++++ 5 files changed, 143 insertions(+)drivers/gpu/nova-core/gpu.rs +4
-0@@ -7,6 +7,7 @@use crate ::driver::Bar0 ;use crate ::falcon::{gsp::Gsp , sec2::Sec2 ,Falcon };use crate ::firmware::Firmware ; +use crate ::gsp::fb::FbLayout ;use crate ::regs;use crate ::util;use crate ::vbios::Vbios ;@@ -239,6 +240,9 @@ pub(crate) fn new(let _sec2_falcon =Falcon ::<Sec2 >::new (pdev.as_ref (), spec.chipset, bar,true )?; +let fb_layout =FbLayout ::new (spec.chipset, bar)?; +dev_dbg! (pdev.as_ref (),"{:#x?}\n" , fb_layout); +let _bios =Vbios ::new (pdev, bar)?; Ok(pin_init!(Self {drivers/gpu/nova-core/gsp.rs new file +3
-0@@ -0,0 +1,3 @@ +// SPDX-License-Identifier: GPL-2.0 + +pub (crate )mod fb;drivers/gpu/nova-core/gsp/fb.rs new file +3
-0@@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0 + +use core::ops::Range ;
drivers/gpu/nova-core/gsp/fb.rs line 13
Quoted 10 lines
drivers/gpu/nova-core/gsp/fb.rs new file +10
-0+ +use kernel::prelude::*; + +use crate ::driver::Bar0 ; +use crate ::gpu::Chipset ; +use crate ::regs; + +fn align_down (value:u64 , align:u64 ) ->u64 { + value & !(align -1 ) +}
Can this go in the previous patch, i.e. "rust: num: Add an upward alignment helper for usize"?
drivers/gpu/nova-core/gsp/fb.rs line 26
Quoted 13 lines
drivers/gpu/nova-core/gsp/fb.rs new file +13
-0+ +/// Layout of the GPU framebuffer memory. +/// +/// Contains ranges of GPU memory reserved for a given purpose during the GSP bootup process. +#[derive( Debug )] +#[expect(dead_code)] +pub (crate )struct FbLayout { +pub fb:Range <u64 >, + +pub vga_workspace:Range <u64 >, +pub bios:Range <u64 >, + +pub frts:Range <u64 >,
Please remove the empty lines.
drivers/gpu/nova-core/gsp/fb.rs line 36
Quoted 10 lines
drivers/gpu/nova-core/gsp/fb.rs new file +10
-0+} + +impl FbLayout { +pub (crate )fn new (chipset:Chipset , bar: &Bar0 ) ->Result <Self > { +let fb = { +let fb_size =vidmem_size (bar, chipset); + +0 ..fb_size + }; +let fb_len = fb.end - fb.start;
Isn't this the same as fb_size? Why not just write it as
let fb_size = vidmem_size(bar, chipset); let fb = 0..fb_size;
drivers/gpu/nova-core/gsp/fb.rs line 44
Quoted 8 lines
drivers/gpu/nova-core/gsp/fb.rs new file +8
-0+ +let vga_workspace = { +let vga_base =vga_workspace_addr (bar, fb_len, chipset); + + vga_base..fb.end + }; + +let bios = vga_workspace.clone ();
Why? And why store the same thing twice in FbLayout? If it's really needed, clone it in the constructor below and add a comment why it's the same.
Quoted 7 lines
drivers/gpu/nova-core/gsp/fb.rs new file +7
-0+ +let frts = { +const FRTS_DOWN_ALIGN :u64 =0x20000 ; +const FRTS_SIZE :u64 =0x100000 ; +let frts_base =align_down (vga_workspace.start,FRTS_DOWN_ALIGN ) -FRTS_SIZE ; + + frts_base..frts_base +FRTS_SIZE
drivers/gpu/nova-core/gsp/fb.rs line 61
Quoted 10 lines
drivers/gpu/nova-core/gsp/fb.rs new file +10
-0+ }; + +Ok (Self { + fb, + vga_workspace, + bios, + frts, + }) + } +}
I'd probably wrap those helpers below in
mod hal { ... }or even a new file fb/hal.rs to make their purpose obvious.
drivers/gpu/nova-core/gsp/fb.rs line 74
Quoted 12 lines
drivers/gpu/nova-core/gsp/fb.rs new file +12
-0+/// Returns `true` if the display is disabled. +fn display_disabled (bar: &Bar0 , chipset:Chipset ) ->bool { +if chipset >=Chipset ::GA100 { + regs::NV_FUSE_STATUS_OPT_DISPLAY_MAXWELL ::read (bar).display_disabled () + }else { + regs::NV_FUSE_STATUS_OPT_DISPLAY_AMPERE ::read (bar).display_disabled () + } +} + +/// Returns the video memory size in bytes. +fn vidmem_size (bar: &Bar0 , chipset:Chipset ) ->u64 { +if chipset >=Chipset ::GA102 {
Is GA102 intentional or should this also be GA100?
drivers/gpu/nova-core/gsp/fb.rs line 75
Quoted 1 line
drivers/gpu/nova-core/gsp/fb.rs new file +1
-0+ (regs::NV_PGC6_AON_SECURE_SCRATCH_GROUP_42 ::read (bar).value ()as u64 ) <<20
Why isn't this shift part of the register abstraction?
drivers/gpu/nova-core/gsp/fb.rs line 79
Quoted 4 lines
drivers/gpu/nova-core/gsp/fb.rs new file +4
-0+ }else { +let local_mem_range = regs::NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE ::read (bar); +let size = + (local_mem_range.lower_mag ()as u64 ) << ((local_mem_range.lower_scale ()as u64 ) +20 );
Same here. Please make this part of the register abstraction as it is done in previous patches.
drivers/gpu/nova-core/gsp/fb.rs line 91
Quoted 12 lines
drivers/gpu/nova-core/gsp/fb.rs new file +12
-0+ +if local_mem_range.ecc_mode_enabled () { + size /16 *15 + }else { + size + } + } +} + +/// Returns the vga workspace address. +fn vga_workspace_addr (bar: &Bar0 , fb_size:u64 , chipset:Chipset ) ->u64 { +let base = fb_size -0x100000 ;
What's this offset? How do you guarantee that this never underflows?
drivers/gpu/nova-core/gsp/fb.rs line 102
Quoted 11 lines
drivers/gpu/nova-core/gsp/fb.rs new file +11
-0+let vga_workspace_base =if display_disabled (bar, chipset) { + regs::NV_PDISP_VGA_WORKSPACE_BASE ::read (bar) + }else { +return base; + }; + +if !vga_workspace_base.status_valid () { +return base; + } + +let addr = (vga_workspace_base.addr ()as u64 ) <<16 ;
Where does this shift come from? Shouldn't this be part of the register abstraction?
drivers/gpu/nova-core/gsp/fb.rs line 104
Quoted 2 lines
drivers/gpu/nova-core/gsp/fb.rs new file +2
-0+if addr < base { + fb_size -0x20000
What's this offset? Can this ever underflow?
Quoted 65 lines
drivers/gpu/nova-core/gsp/fb.rs new file +4
-0+ }else { + addr + } +}drivers/gpu/nova-core/nova_core.rs +1
-0@@ -8,6 +8,7 @@mod falcon;mod firmware;mod gpu; +mod gsp;mod regs;mod util;mod vbios;drivers/gpu/nova-core/regs.rs +27
-0@@ -53,6 +53,12 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> { 23:0 adr_63_40 as u32; }); +register! (NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE @0x00100ce0 { +3 :0 lower_scaleas u8 ; +9 :4 lower_magas u8 ; +30 :30 ecc_mode_enabledas bool ; +}); +/* PGC6 */ register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK @0x00118128 {@@ -64,6 +70,27 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> { 31:0 value as u32; }); +register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_42 @0x001183a4 { +31 :0 valueas u32 ; +}); + +/* PDISP */ + +register! (NV_PDISP_VGA_WORKSPACE_BASE @0x00625f04 { +3 :3 status_validas bool ; +31 :8 addras u32 ; +}); + +/* FUSE */ + +register! (NV_FUSE_STATUS_OPT_DISPLAY_MAXWELL @0x00021c04 { +0 :0 display_disabledas bool ; +}); + +register! (NV_FUSE_STATUS_OPT_DISPLAY_AMPERE @0x00820c04 { +0 :0 display_disabledas bool ; +}); +/* PFALCON */ register! (NV_PFALCON_FALCON_IRQSCLR @ +0x00000004 {Signature · 2 lines
-- 2.49.0
Alexandre Courbot
maintainerRe: [PATCH v3 17/19] gpu: nova-core: compute layout of the FRTS region
On Wed May 14, 2025 at 1:41 AM JST, Danilo Krummrich wrote:
<snip>
Quoted 14 lines
Quoted 14 lines
drivers/gpu/nova-core/gsp/fb.rs new file +8
-0@@ -0,0 +1,108 @@ +// SPDX-License-Identifier: GPL-2.0 + +use core::ops::Range ; + +use kernel::prelude::*; + +use crate ::driver::Bar0 ; +use crate ::gpu::Chipset ;
Quoted 8 lines
drivers/gpu/nova-core/gsp/fb.rs line 13
Quoted 5 lines
drivers/gpu/nova-core/gsp/fb.rs new file +5
-0+use crate ::regs; + +fn align_down (value:u64 , align:u64 ) ->u64 { + value & !(align -1 ) +}Can this go in the previous patch, i.e. "rust: num: Add an upward alignment helper for usize"?
Yes, let me try to consolidate things around the num module. Not sure why it didn't occur to me to add that one there.
Quoted 30 lines
Quoted 4 lines
drivers/gpu/nova-core/gsp/fb.rs new file +3
-0+ +/// Layout of the GPU framebuffer memory. +/// drivers/gpu/nova-core/gsp/fb.rs line 26
Quoted 10 lines
drivers/gpu/nova-core/gsp/fb.rs new file +10
-0+/// Contains ranges of GPU memory reserved for a given purpose during the GSP bootup process. +#[derive( Debug )] +#[expect(dead_code)] +pub (crate )struct FbLayout { +pub fb:Range <u64 >, + +pub vga_workspace:Range <u64 >, +pub bios:Range <u64 >, + +pub frts:Range <u64 >,Please remove the empty lines.
drivers/gpu/nova-core/gsp/fb.rs line 36
Quoted 10 lines
drivers/gpu/nova-core/gsp/fb.rs new file +10
-0+} + +impl FbLayout { +pub (crate )fn new (chipset:Chipset , bar: &Bar0 ) ->Result <Self > { +let fb = { +let fb_size =vidmem_size (bar, chipset); + +0 ..fb_size + }; +let fb_len = fb.end - fb.start;Isn't this the same as
fb_size? Why not just write it as
Quoted 2 lines
let fb_size = vidmem_size(bar, chipset); let fb = 0..fb_size;
It is the same indeed, and fb_size and fb_len are semantically the same thing so no need to have both.
Quoted 12 lines
drivers/gpu/nova-core/gsp/fb.rs line 44
Quoted 9 lines
drivers/gpu/nova-core/gsp/fb.rs new file +8
-0+ +let vga_workspace = { +let vga_base =vga_workspace_addr (bar, fb_len, chipset); + + vga_base..fb.end + }; + +let bios = vga_workspace.clone ();Why? And why store the same thing twice in FbLayout? If it's really needed, clone it in the constructor below and add a comment why it's the same.
The bios field does not seem to be used at the moment anyway, so let me remove it.
Quoted 21 lines
Quoted 8 lines
drivers/gpu/nova-core/gsp/fb.rs new file +7
-0+ +let frts = { +const FRTS_DOWN_ALIGN :u64 =0x20000 ; +const FRTS_SIZE :u64 =0x100000 ; +let frts_base =align_down (vga_workspace.start,FRTS_DOWN_ALIGN ) -FRTS_SIZE ; + + frts_base..frts_base +FRTS_SIZE drivers/gpu/nova-core/gsp/fb.rs line 61
Quoted 10 lines
drivers/gpu/nova-core/gsp/fb.rs new file +10
-0+ }; + +Ok (Self { + fb, + vga_workspace, + bios, + frts, + }) + } +}I'd probably wrap those helpers below in
Quoted 3 lines
mod hal { ... }or even a new file
fb/hal.rsto make their purpose obvious.
Do we want a module here? I'm fine with it, but these methods are already private anyway and putting them under a module would require them to have pub(super) visibility.
... or maybe we should have an actual HAL here with dynamic dispatch, similar to what we have in the falcon module. That's what OpenRM does actually. Let me look into that.
Quoted 15 lines
drivers/gpu/nova-core/gsp/fb.rs line 74
Quoted 13 lines
drivers/gpu/nova-core/gsp/fb.rs new file +12
-0+/// Returns `true` if the display is disabled. +fn display_disabled (bar: &Bar0 , chipset:Chipset ) ->bool { +if chipset >=Chipset ::GA100 { + regs::NV_FUSE_STATUS_OPT_DISPLAY_MAXWELL ::read (bar).display_disabled () + }else { + regs::NV_FUSE_STATUS_OPT_DISPLAY_AMPERE ::read (bar).display_disabled () + } +} + +/// Returns the video memory size in bytes. +fn vidmem_size (bar: &Bar0 , chipset:Chipset ) ->u64 { +if chipset >=Chipset ::GA102 {Is GA102 intentional or should this also be GA100?
After double-checking with OpenRM GA102 is indeed correct.
Quoted 4 lines
drivers/gpu/nova-core/gsp/fb.rs line 75
Quoted 2 lines
drivers/gpu/nova-core/gsp/fb.rs new file +1
-0+ (regs::NV_PGC6_AON_SECURE_SCRATCH_GROUP_42 ::read (bar).value ()as u64 ) <<20 Why isn't this shift part of the register abstraction?
This value came from a scratch register, which interpretation depends on context so I did not abstract the values. But OpenRM does, so let me add a way to create alias registers so we can get adequate definitions here as well.
Quoted 8 lines
drivers/gpu/nova-core/gsp/fb.rs line 79
Quoted 5 lines
drivers/gpu/nova-core/gsp/fb.rs new file +4
-0+ }else { +let local_mem_range = regs::NV_PFB_PRI_MMU_LOCAL_MEMORY_RANGE ::read (bar); +let size = + (local_mem_range.lower_mag ()as u64 ) << ((local_mem_range.lower_scale ()as u64 ) +20 );Same here. Please make this part of the register abstraction as it is done in previous patches.
Ack.
Quoted 15 lines
drivers/gpu/nova-core/gsp/fb.rs line 91
Quoted 13 lines
drivers/gpu/nova-core/gsp/fb.rs new file +12
-0+ +if local_mem_range.ecc_mode_enabled () { + size /16 *15 + }else { + size + } + } +} + +/// Returns the vga workspace address. +fn vga_workspace_addr (bar: &Bar0 , fb_size:u64 , chipset:Chipset ) ->u64 { +let base = fb_size -0x100000 ;What's this offset? How do you guarantee that this never underflows?
Looked it up in OpenRM, it is the size of PRAMIN. I'll add a constant.
Quoted 15 lines
drivers/gpu/nova-core/gsp/fb.rs line 102
Quoted 12 lines
drivers/gpu/nova-core/gsp/fb.rs new file +11
-0+let vga_workspace_base =if display_disabled (bar, chipset) { + regs::NV_PDISP_VGA_WORKSPACE_BASE ::read (bar) + }else { +return base; + }; + +if !vga_workspace_base.status_valid () { +return base; + } + +let addr = (vga_workspace_base.addr ()as u64 ) <<16 ;Where does this shift come from? Shouldn't this be part of the register abstraction?
Yes. Also added documentation in the field to explain that the field's lowest 16 bits are truncated.
Quoted 5 lines
drivers/gpu/nova-core/gsp/fb.rs line 104
Quoted 3 lines
drivers/gpu/nova-core/gsp/fb.rs new file +2
-0+if addr < base { + fb_size -0x20000 What's this offset? Can this ever underflow?
This one is also defined as a constant in OpenRM. Let me replicate it here.
Alexandre Courbot
maintainerPATCH v3 18/19 gpu: nova-core: extract FWSEC from BIOS and patch it to run FWSEC-FRTS
In reply to Alexandre CourbotThe FWSEC firmware needs to be extracted from the VBIOS and patched with the desired command, as well as the right signature. Do this so we are ready to load and run this firmware into the GSP falcon and create the FRTS region.
--- drivers/gpu/nova-core/dma.rs | 3 - drivers/gpu/nova-core/firmware.rs | 18 ++ drivers/gpu/nova-core/firmware/fwsec.rs | 359 ++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/gpu.rs | 20 +- drivers/gpu/nova-core/vbios.rs | 3 - 5 files changed, 395 insertions(+), 8 deletions(-)
drivers/gpu/nova-core/dma.rs +0 -3
@@ -2,9 +2,6 @@//! Simple DMA object wrapper. -// To be removed when all code is used. -#![expect(dead_code)] -use core::ops::{Deref ,DerefMut };use kernel::device;
drivers/gpu/nova-core/firmware.rs +18 -0
@@ -8,9 +8,12 @@use kernel::prelude::*;use kernel::str::CString ; +use crate ::dma::DmaObject ;use crate ::gpu;use crate ::gpu::Chipset ; +pub (crate )mod fwsec; +pub (crate )const FIRMWARE_VERSION : &str ="535.113.01" ;/// Structure encapsulating the firmware blobs required for the GPU to operate.
@@ -86,6 +89,21 @@ pub(crate) fn size(&self) -> usize { } } +/// Patch the `ucode_dma` firmware at offset `sig_base_img` with `signature`. +fn patch_signature (ucode_dma: &mut DmaObject , signature: &[u8 ], sig_base_img:usize ) ->Result <()> { +if sig_base_img + signature.len () > ucode_dma.size () { +return Err (EINVAL ); + } + +// SAFETY: we are the only user of this object, so there cannot be any race. +let dst =unsafe { ucode_dma.start_ptr_mut ().add (sig_base_img) }; + +// SAFETY: `signature` and `dst` are valid, properly aligned, and do not overlap. +unsafe { core::ptr::copy_nonoverlapping (signature.as_ptr (), dst, signature.len ()) }; + +Ok (()) +} +pub (crate )struct ModInfoBuilder <const N :usize >(firmware::ModInfoBuilder <N >);impl <const N :usize >ModInfoBuilder <N > {
drivers/gpu/nova-core/firmware/fwsec.rs new file +359 -0
359 lines not on this page · raw message
drivers/gpu/nova-core/gpu.rs +18 -2
@@ -6,6 +6,7 @@use crate ::dma::DmaObject ;use crate ::driver::Bar0 ;use crate ::falcon::{gsp::Gsp , sec2::Sec2 ,Falcon }; +use crate ::firmware::fwsec::{FwsecCommand ,FwsecFirmware };use crate ::firmware::Firmware ;use crate ::gsp::fb::FbLayout ;use crate ::regs;
@@ -197,7 +198,11 @@ pub(crate) fn new( ) -> Result<impl PinInit<Self>> {let bar = devres_bar.access (pdev.as_ref ())?;let spec =Spec ::new (bar)?; -let fw =Firmware ::new (pdev.as_ref (), spec.chipset,"535.113.01" )?; +let fw =Firmware ::new ( + pdev.as_ref (), + spec.chipset, +crate ::firmware::FIRMWARE_VERSION , + )?; dev_info!( pdev.as_ref(),
@@ -243,7 +248,18 @@ pub(crate) fn new(let fb_layout =FbLayout ::new (spec.chipset, bar)?;dev_dbg! (pdev.as_ref (),"{:#x?}\n" , fb_layout); -let _bios =Vbios ::new (pdev, bar)?; +let bios =Vbios ::new (pdev, bar)?; + +let _fwsec_frts =FwsecFirmware ::new ( + &gsp_falcon, + pdev.as_ref (), + bar, + &bios, +FwsecCommand ::Frts { + frts_addr: fb_layout.frts.start, + frts_size: fb_layout.frts.end - fb_layout.frts.start, + }, + )?; Ok(pin_init!(Self { spec,
drivers/gpu/nova-core/vbios.rs +0 -3
@@ -2,9 +2,6 @@//! VBIOS extraction and parsing. -// To be removed when all code is used. -#![expect(dead_code)] -use crate ::driver::Bar0 ;use crate ::firmware::FalconUCodeDescV3 ;use core::convert::TryFrom ;
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 18/19] gpu: nova-core: extract FWSEC from BIOS and patch it to run FWSEC-FRTS
Quoted 54 lines · Alexandre Courbot
The FWSEC firmware needs to be extracted from the VBIOS and patched with the desired command, as well as the right signature. Do this so we are ready to load and run this firmware into the GSP falcon and create the FRTS region.
[[email protected]: give better names to FalconAppifHdrV1's fields]Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/dma.rs | 3 - drivers/gpu/nova-core/firmware.rs | 18 ++ drivers/gpu/nova-core/firmware/fwsec.rs | 359 ++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/gpu.rs | 20 +- drivers/gpu/nova-core/vbios.rs | 3 - 5 files changed, 395 insertions(+), 8 deletions(-)drivers/gpu/nova-core/dma.rs +0
-3@@ -2,9 +2,6 @@//! Simple DMA object wrapper. -// To be removed when all code is used. -#![expect(dead_code)] -use core::ops::{Deref ,DerefMut };use kernel::device;drivers/gpu/nova-core/firmware.rs +7
-0@@ -8,9 +8,12 @@use kernel::prelude::*;use kernel::str::CString ; +use crate ::dma::DmaObject ;use crate ::gpu;use crate ::gpu::Chipset ; +pub (crate )mod fwsec; +pub (crate )const FIRMWARE_VERSION : &str ="535.113.01" ;/// Structure encapsulating the firmware blobs required for the GPU to operate. @@ -86,6 +89,21 @@ pub(crate) fn size(&self) -> usize { } } +/// Patch the `ucode_dma` firmware at offset `sig_base_img` with `signature`. +fn patch_signature (ucode_dma: &mut DmaObject , signature: &[u8 ], sig_base_img:usize ) ->Result <()> { +if sig_base_img + signature.len () > ucode_dma.size () { +return Err (EINVAL );
drivers/gpu/nova-core/firmware.rs pub(crate) fn size(&self) -> usize { · line 105
Quoted 10 lines
drivers/gpu/nova-core/firmware.rs +10
-0+ } + +// SAFETY: we are the only user of this object, so there cannot be any race. +let dst =unsafe { ucode_dma.start_ptr_mut ().add (sig_base_img) }; + +// SAFETY: `signature` and `dst` are valid, properly aligned, and do not overlap. +unsafe { core::ptr::copy_nonoverlapping (signature.as_ptr (), dst, signature.len ()) }; + +Ok (()) +}
Why is this not in firmware/fwsec.rs, like patch_command()?
Also, please wrap the ucode DmaObject in its own type, i.e. struct UcodeDma(DmaObject) and make the patch_*() functions methods of this type. They're only applicable for the ucode DmaObject.
Quoted 306 lines
drivers/gpu/nova-core/firmware.rs +1
-0+pub (crate )struct ModInfoBuilder <const N :usize >(firmware::ModInfoBuilder <N >);impl <const N :usize >ModInfoBuilder <N > {drivers/gpu/nova-core/firmware/fwsec.rs new file +296
-0@@ -0,0 +1,359 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! FWSEC is a High Secure firmware that is extracted from the BIOS and performs the first step of +//! the GSP startup by creating the WPR2 memory region and copying critical areas of the VBIOS into +//! it after authenticating them, ensuring they haven't been tampered with. It runs on the GSP +//! falcon. +//! +//! Before being run, it needs to be patched in two areas: +//! +//! - The command to be run, as this firmware can perform several tasks ; +//! - The ucode signature, so the GSP falcon can run FWSEC in HS mode. + +use core::alloc::Layout ; + +use kernel::bindings; +use kernel::device::{self,Device }; +use kernel::prelude::*; +use kernel::transmute::FromBytes ; + +use crate ::dma::DmaObject ; +use crate ::driver::Bar0 ; +use crate ::falcon::gsp::Gsp ; +use crate ::falcon::{Falcon ,FalconBromParams ,FalconFirmware ,FalconLoadTarget }; +use crate ::firmware::FalconUCodeDescV3 ; +use crate ::vbios::Vbios ; + +const NVFW_FALCON_APPIF_ID_DMEMMAPPER :u32 =0x4 ; + +#[repr( C )] +#[derive( Debug )] +struct FalconAppifHdrV1 { + version:u8 , + header_size:u8 , + entry_size:u8 , + entry_count:u8 , +} +// SAFETY: any byte sequence is valid for this struct. +unsafe impl FromBytes for FalconAppifHdrV1 {} + +#[repr( C , packed)] +#[derive( Debug )] +struct FalconAppifV1 { + id:u32 , + dmem_base:u32 , +} +// SAFETY: any byte sequence is valid for this struct. +unsafe impl FromBytes for FalconAppifV1 {} + +#[derive( Debug )] +#[repr( C , packed)] +struct FalconAppifDmemmapperV3 { + signature:u32 , + version:u16 , + size:u16 , + cmd_in_buffer_offset:u32 , + cmd_in_buffer_size:u32 , + cmd_out_buffer_offset:u32 , + cmd_out_buffer_size:u32 , + nvf_img_data_buffer_offset:u32 , + nvf_img_data_buffer_size:u32 , + printf_buffer_hdr:u32 , + ucode_build_time_stamp:u32 , + ucode_signature:u32 , + init_cmd:u32 , + ucode_feature:u32 , + ucode_cmd_mask0:u32 , + ucode_cmd_mask1:u32 , + multi_tgt_tbl:u32 , +} +// SAFETY: any byte sequence is valid for this struct. +unsafe impl FromBytes for FalconAppifDmemmapperV3 {} + +#[derive( Debug )] +#[repr( C , packed)] +struct ReadVbios { + ver:u32 , + hdr:u32 , + addr:u64 , + size:u32 , + flags:u32 , +} +// SAFETY: any byte sequence is valid for this struct. +unsafe impl FromBytes for ReadVbios {} + +#[derive( Debug )] +#[repr( C , packed)] +struct FrtsRegion { + ver:u32 , + hdr:u32 , + addr:u32 , + size:u32 , + ftype:u32 , +} +// SAFETY: any byte sequence is valid for this struct. +unsafe impl FromBytes for FrtsRegion {} + +const NVFW_FRTS_CMD_REGION_TYPE_FB :u32 =2 ; + +#[repr( C , packed)] +struct FrtsCmd { + read_vbios:ReadVbios , + frts_region:FrtsRegion , +} +// SAFETY: any byte sequence is valid for this struct. +unsafe impl FromBytes for FrtsCmd {} + +const NVFW_FALCON_APPIF_DMEMMAPPER_CMD_FRTS :u32 =0x15 ; +const NVFW_FALCON_APPIF_DMEMMAPPER_CMD_SB :u32 =0x19 ; + +/// Command for the [`FwsecFirmware`] to execute. +pub (crate )enum FwsecCommand { +/// Asks [`FwsecFirmware`] to carve out the WPR2 area and place a verified copy of the VBIOS +/// image into it. +Frts { frts_addr:u64 , frts_size:u64 }, +/// Asks [`FwsecFirmware`] to load pre-OS apps on the PMU. +#[expect(dead_code)] +Sb , +} + +/// Reinterpret the area starting from `offset` in `fw` as an instance of `T` (which must implement +/// [`FromBytes`]) and return a reference to it. +/// +/// # Safety +/// +/// Callers must ensure that the region of memory returned is not written for as long as the +/// returned reference is alive. +/// +/// TODO: Remove this and `transmute_mut` once we have a way to transmute objects implementing +/// FromBytes, e.g.: +/// https://lore.kernel.org/lkml/[email protected]/ +unsafe fn transmute <'a, 'b,T :Sized +FromBytes >( + fw: &'aDmaObject , + offset:usize , +) ->Result <&'bT > { +if offset + core::mem::size_of ::<T >() > fw.size () { +return Err (EINVAL ); + } +if (fw.start_ptr ()as usize + offset) % core::mem::align_of ::<T >() !=0 { +return Err (EINVAL ); + } + +// SAFETY: we have checked that the pointer is properly aligned that its pointed memory is +// large enough the contains an instance of `T`, which implements `FromBytes`. +Ok (unsafe { &*(fw.start_ptr ().add (offset)as *const T ) }) +} + +/// Reinterpret the area starting from `offset` in `fw` as a mutable instance of `T` (which must +/// implement [`FromBytes`]) and return a reference to it. +/// +/// # Safety +/// +/// Callers must ensure that the region of memory returned is not read or written for as long as +/// the returned reference is alive. +unsafe fn transmute_mut <'a, 'b,T :Sized +FromBytes >( + fw: &'amut DmaObject , + offset:usize , +) ->Result <&'bmut T > { +if offset + core::mem::size_of ::<T >() > fw.size () { +return Err (EINVAL ); + } +if (fw.start_ptr_mut ()as usize + offset) % core::mem::align_of ::<T >() !=0 { +return Err (EINVAL ); + } + +// SAFETY: we have checked that the pointer is properly aligned that its pointed memory is +// large enough the contains an instance of `T`, which implements `FromBytes`. +Ok (unsafe { &mut *(fw.start_ptr_mut ().add (offset)as *mut T ) }) +} + +/// Patch the Fwsec firmware image in `fw` to run the command `cmd`. +fn patch_command (fw: &mut DmaObject , v3_desc: &FalconUCodeDescV3 , cmd:FwsecCommand ) ->Result <()> { +let hdr_offset = (v3_desc.imem_load_size + v3_desc.interface_offset)as usize ; +// SAFETY: we have an exclusive reference to `fw`, and no caller should have shared `fw` with +// the hardware yet. +let hdr: &FalconAppifHdrV1 =unsafe {transmute (fw, hdr_offset) }?; + +if hdr.version !=1 { +return Err (EINVAL ); + } + +// Find the DMEM mapper section in the firmware. +for iin 0 ..hdr.entry_countas usize { +let app: &FalconAppifV1 = +// SAFETY: we have an exclusive reference to `fw`, and no caller should have shared +// `fw` with the hardware yet. +unsafe { +transmute ( + fw, + hdr_offset + hdr.header_sizeas usize + i * hdr.entry_sizeas usize + ) + }?; + +if app.id !=NVFW_FALCON_APPIF_ID_DMEMMAPPER { +continue ; + } + +let dmem_mapper: &mut FalconAppifDmemmapperV3 = +// SAFETY: we have an exclusive reference to `fw`, and no caller should have shared +// `fw` with the hardware yet. +unsafe {transmute_mut (fw, (v3_desc.imem_load_size + app.dmem_base)as usize ) }?; + +// SAFETY: we have an exclusive reference to `fw`, and no caller should have shared `fw` +// with the hardware yet. +let frts_cmd: &mut FrtsCmd =unsafe { +transmute_mut ( + fw, + (v3_desc.imem_load_size + dmem_mapper.cmd_in_buffer_offset)as usize , + ) + }?; + + frts_cmd.read_vbios =ReadVbios { + ver:1 , + hdr: core::mem::size_of ::<ReadVbios >()as u32 , + addr:0 , + size:0 , + flags:2 , + }; + + dmem_mapper.init_cmd =match cmd { +FwsecCommand ::Frts { + frts_addr, + frts_size, + } => { + frts_cmd.frts_region =FrtsRegion { + ver:1 , + hdr: core::mem::size_of ::<FrtsRegion >()as u32 , + addr: (frts_addr >>12 )as u32 , + size: (frts_size >>12 )as u32 , + ftype:NVFW_FRTS_CMD_REGION_TYPE_FB , + }; + +NVFW_FALCON_APPIF_DMEMMAPPER_CMD_FRTS + } +FwsecCommand ::Sb =>NVFW_FALCON_APPIF_DMEMMAPPER_CMD_SB , + }; + +// Return early as we found and patched the DMEMMAPPER region. +return Ok (()); + } + +Err (ENOTSUPP ) +} + +/// Firmware extracted from the VBIOS and responsible for e.g. carving out the WPR2 region as the +/// first step of the GSP bootflow. +pub (crate )struct FwsecFirmware { + desc:FalconUCodeDescV3 , + ucode:DmaObject , +} + +impl FalconFirmware for FwsecFirmware { +type Target =Gsp ; + +fn dma_handle (&self) -> bindings::dma_addr_t { + self.ucode.dma_handle () + } + +fn imem_load (&self) ->FalconLoadTarget { +FalconLoadTarget { + src_start:0 , + dst_start: self.desc.imem_phys_base, + len: self.desc.imem_load_size, + } + } + +fn dmem_load (&self) ->FalconLoadTarget { +FalconLoadTarget { + src_start: self.desc.imem_load_size, + dst_start: self.desc.dmem_phys_base, + len:Layout ::from_size_align (self.desc.dmem_load_sizeas usize ,256 ) +// Cannot panic, as 256 is non-zero and a power of 2. + .unwrap () + .pad_to_align () + .size ()as u32 , + } + } + +fn brom_params (&self) ->FalconBromParams { +FalconBromParams { + pkc_data_offset: self.desc.pkc_data_offset, + engine_id_mask: self.desc.engine_id_mask, + ucode_id: self.desc.ucode_id, + } + } + +fn boot_addr (&self) ->u32 { +0 + } +} + +impl FwsecFirmware { +/// Extract the Fwsec firmware from `bios` and patch it to run with the `cmd` command. +pub (crate )fn new ( + falcon: &Falcon <Gsp >, + dev: &Device <device::Bound >, + bar: &Bar0,
drivers/gpu/nova-core/firmware/fwsec.rs line 306
Quoted 10 lines
drivers/gpu/nova-core/firmware/fwsec.rs new file +10
-0+ bios: &Vbios, + cmd: FwsecCommand, + ) -> Result<Self> { +let v3_desc = bios.fwsec_header (dev)?; +let ucode = bios.fwsec_ucode (dev)?; + +let mut ucode_dma =DmaObject ::from_data (dev, ucode)?; +patch_command (&mut ucode_dma, v3_desc, cmd)?; + +const SIG_SIZE :usize =96 *4 ;
96 * 4? :-)
Quoted 18 lines
drivers/gpu/nova-core/firmware/fwsec.rs new file +18
-0+let signatures = bios.fwsec_sigs (dev)?; +let sig_base_img = (v3_desc.imem_load_size + v3_desc.pkc_data_offset)as usize ; + +if v3_desc.signature_count !=0 { +// Patch signature. +let desc_sig_versions = v3_desc.signature_versionsas u32 ; +let reg_fuse_version = falcon.get_signature_reg_fuse_version ( + bar, + v3_desc.engine_id_mask, + v3_desc.ucode_id, + )?; +dev_dbg! ( + dev, +"desc_sig_versions: {:#x}, reg_fuse_version: {}\n" , + desc_sig_versions, + reg_fuse_version + ); +let signature_idx = {
drivers/gpu/nova-core/firmware/fwsec.rs line 334
Quoted 10 lines
drivers/gpu/nova-core/firmware/fwsec.rs new file +10
-0+let reg_fuse_version_bit =1 << reg_fuse_version; + +// Check if the fuse version is supported by the firmware. +if desc_sig_versions & reg_fuse_version_bit ==0 { +dev_warn! ( + dev, +"no matching signature: {:#x} {:#x}\n" , + reg_fuse_version_bit, + v3_desc.signature_versions + );
Looks like this should be dev_err!().
Quoted 88 lines
drivers/gpu/nova-core/firmware/fwsec.rs new file +25
-0+return Err (EINVAL ); + } + +// `desc_sig_versions` has one bit set per included signature. Thus, the index of +// the signature to patch is the number of bits in `desc_sig_versions` set to `1` +// before `reg_fuse_version_bit`. + +// Mask of the bits of `desc_sig_versions` to preserve. +let reg_fuse_version_mask = reg_fuse_version_bit.wrapping_sub (1 ); + + (desc_sig_versions & reg_fuse_version_mask).count_ones () + }; + +dev_dbg! (dev,"patching signature with index {}\n" , signature_idx); +let signature_start = signature_idxas usize *SIG_SIZE ; +let signature = &signatures[signature_start..signature_start +SIG_SIZE ]; +super ::patch_signature (&mut ucode_dma, signature, sig_base_img)?; + } + +Ok (FwsecFirmware { + desc: v3_desc.clone (), + ucode: ucode_dma, + }) + } +}drivers/gpu/nova-core/gpu.rs +18
-2@@ -6,6 +6,7 @@use crate ::dma::DmaObject ;use crate ::driver::Bar0 ;use crate ::falcon::{gsp::Gsp , sec2::Sec2 ,Falcon }; +use crate ::firmware::fwsec::{FwsecCommand ,FwsecFirmware };use crate ::firmware::Firmware ;use crate ::gsp::fb::FbLayout ;use crate ::regs;@@ -197,7 +198,11 @@ pub(crate) fn new( ) -> Result<impl PinInit<Self>> {let bar = devres_bar.access (pdev.as_ref ())?;let spec =Spec ::new (bar)?; -let fw =Firmware ::new (pdev.as_ref (), spec.chipset,"535.113.01" )?; +let fw =Firmware ::new ( + pdev.as_ref (), + spec.chipset, +crate ::firmware::FIRMWARE_VERSION , + )?; dev_info!( pdev.as_ref(),@@ -243,7 +248,18 @@ pub(crate) fn new(let fb_layout =FbLayout ::new (spec.chipset, bar)?;dev_dbg! (pdev.as_ref (),"{:#x?}\n" , fb_layout); -let _bios =Vbios ::new (pdev, bar)?; +let bios =Vbios ::new (pdev, bar)?; + +let _fwsec_frts =FwsecFirmware ::new ( + &gsp_falcon, + pdev.as_ref (), + bar, + &bios, +FwsecCommand ::Frts { + frts_addr: fb_layout.frts.start, + frts_size: fb_layout.frts.end - fb_layout.frts.start, + }, + )?; Ok(pin_init!(Self { spec,drivers/gpu/nova-core/vbios.rs +0
-3@@ -2,9 +2,6 @@//! VBIOS extraction and parsing. -// To be removed when all code is used. -#![expect(dead_code)] -use crate ::driver::Bar0 ;use crate ::firmware::FalconUCodeDescV3 ;use core::convert::TryFrom ;Signature · 2 lines
-- 2.49.0
Alexandre Courbot
maintainerRe: [PATCH v3 18/19] gpu: nova-core: extract FWSEC from BIOS and patch it to run FWSEC-FRTS
Quoted 51 lines · Danilo Krummrich
Quoted 50 lines · Alexandre Courbot
The FWSEC firmware needs to be extracted from the VBIOS and patched with the desired command, as well as the right signature. Do this so we are ready to load and run this firmware into the GSP falcon and create the FRTS region.
[[email protected]: give better names to FalconAppifHdrV1's fields]Signed-off-by Alexandre Courbot <[email protected]>--- drivers/gpu/nova-core/dma.rs | 3 - drivers/gpu/nova-core/firmware.rs | 18 ++ drivers/gpu/nova-core/firmware/fwsec.rs | 359 ++++++++++++++++++++++++++++++++ drivers/gpu/nova-core/gpu.rs | 20 +- drivers/gpu/nova-core/vbios.rs | 3 - 5 files changed, 395 insertions(+), 8 deletions(-)drivers/gpu/nova-core/dma.rs +0
-3@@ -2,9 +2,6 @@//! Simple DMA object wrapper. -// To be removed when all code is used. -#![expect(dead_code)] -use core::ops::{Deref ,DerefMut };use kernel::device;drivers/gpu/nova-core/firmware.rs +3
-0@@ -8,9 +8,12 @@use kernel::prelude::*;use kernel::str::CString ; +use crate ::dma::DmaObject ;use crate ::gpu;use crate ::gpu::Chipset ; +pub (crate )mod fwsec; +pub (crate )const FIRMWARE_VERSION : &str ="535.113.01" ;/// Structure encapsulating the firmware blobs required for the GPU to operate. @@ -86,6 +89,21 @@ pub(crate) fn size(&self) -> usize { } }
Quoted 16 lines
Quoted 4 lines
drivers/gpu/nova-core/firmware.rs +4
-0+/// Patch the `ucode_dma` firmware at offset `sig_base_img` with `signature`. +fn patch_signature (ucode_dma: &mut DmaObject , signature: &[u8 ], sig_base_img:usize ) ->Result <()> { +if sig_base_img + signature.len () > ucode_dma.size () { +return Err (EINVAL );drivers/gpu/nova-core/firmware.rs pub(crate) fn size(&self) -> usize { · line 105
Quoted 10 lines
drivers/gpu/nova-core/firmware.rs +10
-0+ } + +// SAFETY: we are the only user of this object, so there cannot be any race. +let dst =unsafe { ucode_dma.start_ptr_mut ().add (sig_base_img) }; + +// SAFETY: `signature` and `dst` are valid, properly aligned, and do not overlap. +unsafe { core::ptr::copy_nonoverlapping (signature.as_ptr (), dst, signature.len ()) }; + +Ok (()) +}Why is this not in
firmware/fwsec.rs, likepatch_command()?
Ah, there is no way to know it now, but this function will also be used to patch the booter firmware that runs on sec2, so having it here makes it available to both sub-modules. I'm fine with moving it into the fwsec module temporarily if you prefer though.
Quoted 4 lines
Also, please wrap the ucode DmaObject in its own type, i.e.
struct UcodeDma(DmaObject)and make the patch_*() functions methods of this type. They're only applicable for the ucode DmaObject.
Indeed, good idea. We will event want to specialize that type against the kind of firmware as not all patching methods may be applicable depending on the firmware.
<snip>
Quoted 10 lines
Quoted 10 lines
drivers/gpu/nova-core/firmware.rs +10
-0+impl FwsecFirmware { +/// Extract the Fwsec firmware from `bios` and patch it to run with the `cmd` command. +pub (crate )fn new ( + falcon: &Falcon <Gsp >, + dev: &Device <device::Bound >, + bar: &Bar0 , + bios: &Vbios , + cmd:FwsecCommand , + ) ->Result <Self > { +let v3_desc = bios.fwsec_header (dev)?;
Quoted 8 lines
drivers/gpu/nova-core/firmware.rs
Quoted 6 lines
drivers/gpu/nova-core/firmware.rs +6
-0+let ucode = bios.fwsec_ucode (dev)?; + +let mut ucode_dma =DmaObject ::from_data (dev, ucode)?; +patch_command (&mut ucode_dma, v3_desc, cmd)?; + +const SIG_SIZE :usize =96 *4 ;96 * 4? :-)
Mmmm let me look that up. ^_^; But I think it means that a signature is made of 96 32-bit integers.
Quoted 23 lines
Quoted 23 lines
drivers/gpu/nova-core/firmware.rs +22
-0+let signatures = bios.fwsec_sigs (dev)?; +let sig_base_img = (v3_desc.imem_load_size + v3_desc.pkc_data_offset)as usize ; + +if v3_desc.signature_count !=0 { +// Patch signature. +let desc_sig_versions = v3_desc.signature_versionsas u32 ; +let reg_fuse_version = falcon.get_signature_reg_fuse_version ( + bar, + v3_desc.engine_id_mask, + v3_desc.ucode_id, + )?; +dev_dbg! ( + dev, +"desc_sig_versions: {:#x}, reg_fuse_version: {}\n" , + desc_sig_versions, + reg_fuse_version + ); +let signature_idx = { +let reg_fuse_version_bit =1 << reg_fuse_version; + +// Check if the fuse version is supported by the firmware. +if desc_sig_versions & reg_fuse_version_bit ==0 {
Quoted 8 lines
drivers/gpu/nova-core/firmware.rs
Quoted 6 lines
drivers/gpu/nova-core/firmware.rs +6
-0+dev_warn! ( + dev, +"no matching signature: {:#x} {:#x}\n" , + reg_fuse_version_bit, + v3_desc.signature_versions + );Looks like this should be dev_err!().
Indeed, fixed.
Alexandre Courbot
maintainerPATCH v3 19/19 gpu: nova-core: load and run FWSEC-FRTS
In reply to Alexandre CourbotWith all the required pieces in place, load FWSEC-FRTS onto the GSP falcon, run it, and check that it successfully carved out the WPR2 region out of framebuffer memory.
--- drivers/gpu/nova-core/falcon.rs | 3 --- drivers/gpu/nova-core/gpu.rs | 57 ++++++++++++++++++++++++++++++++++++++++- drivers/gpu/nova-core/regs.rs | 15 +++++++++++ 3 files changed, 71 insertions(+), 4 deletions(-)
drivers/gpu/nova-core/falcon.rs +0 -3
@@ -2,9 +2,6 @@//! Falcon microprocessor base support -// To be removed when all code is used. -#![expect(dead_code)] -use core::time::Duration ;use hal::FalconHal ;use kernel::bindings;
drivers/gpu/nova-core/gpu.rs +56 -1
69 lines not on this page · raw message
drivers/gpu/nova-core/regs.rs +15 -0
@@ -43,6 +43,13 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> { } } +/* PBUS */ + +// TODO: this is an array of registers. +register! (NV_PBUS_SW_SCRATCH_0E @0x00001438 { +31 :16 frts_err_codeas u16 ; +}); +/* PFB */ register! (NV_PFB_NISO_FLUSH_SYSMEM_ADDR @0x00100c10 {
@@ -59,6 +66,14 @@ pub(crate) fn chipset(self) -> Result<Chipset, Error> { 30:30 ecc_mode_enabled as bool; }); +register! (NV_PFB_PRI_MMU_WPR2_ADDR_LO @0x001fa824 { +31 :4 lo_valas u32 ; +}); + +register! (NV_PFB_PRI_MMU_WPR2_ADDR_HI @0x001fa828 { +31 :4 hi_valas u32 ; +}); +/* PGC6 */ register! (NV_PGC6_AON_SECURE_SCRATCH_GROUP_05_PRIV_LEVEL_MASK @0x00118128 {
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerRe: [PATCH v3 19/19] gpu: nova-core: load and run FWSEC-FRTS
drivers/gpu/nova-core/gpu.rs pub(crate) fn new( · line 317
Quoted 3 lines · Alexandre Courbot
drivers/gpu/nova-core/gpu.rs +3
-0+ +dev_info! (pdev.as_ref (),"WPR2: {:#x}-{:#x}\n" , wpr2_lo, wpr2_hi); +dev_info! (pdev.as_ref (),"GPU instance built\n" );
Please use dev_dbg!().
Quoted 4 lines · Alexandre Courbot
gpu: nova-core: derive useful traits for Chipset
gpu: nova-core: add missing GA100 definition
gpu: nova-core: take bound device inGpu::new
gpu: nova-core: define registers layout using helper macro
[ Fix typo in commit message. - Danilo ]
Quoted 1 line
gpu: nova-core: fix layout of
NV_PMC_BOOT_0
[ Use Result from kernel::prelude. - Danilo ]
Quoted 1 line
gpu: nova-core: move Firmware to firmware module
[ Don't require a bound device, remove pub visibility from Firmware
fields, use FIRMWARE_VERSION consistently. - Danilo ]
Applied the patches above to nova-next, thanks!