Skip to contentSkip to conversation

gpu: nova-core: add GA100 support

PATCH Superseded Timur Tabi 4 participants 29 messages Apr 14 – Apr 17

Subsystems (4)

v2 of 5 Superseded Latest is v5 →

lore
Share & tools
lore

Timur Tabi

author
#0 lore raw

v2 cover gpu: nova-core: add GA100 support

GA100 is an odd GPU. Architecturally, it's an Ampere. However, it uses the same GSP-RM boot process as a Turing. Its VBIOS, for whatever reason, has an IFR header that precedes the PCI ROM header and must be skipped. In addition, the FRTS window size must be set to 0 just because it's a GA100.

So add all the missing code that GA100 needs to be supported. There was some infrastructure already for GA100 (e.g. the FbHal), but it was incomplete.

Changes in v2:
1) Created frts_size_tu102()
2) Added bitfields and more macros for IFR header parsing
3) Added code comments for the IFR header
4) Removed language that said that the FRTS region "does not exist", instead stating that its size is 0.

Timur Tabi (6):
gpu: nova-core: use correct fwsignature for GA100
gpu: nova-core: do not consider 0xBB77 as a valid PCI ROM header
signature
gpu: nova-core: only boot FRTS if its region is allocated
gpu: nova-core: add FbHal::frts_size() for GA100 support
gpu: nova-core: skip the IFR header if present
gpu: nova-core: enable GA100

 drivers/gpu/nova-core/falcon/hal.rs   |  3 +-
 drivers/gpu/nova-core/fb.rs           |  6 +--
 drivers/gpu/nova-core/fb/hal.rs       |  3 ++
 drivers/gpu/nova-core/fb/hal/ga100.rs |  5 ++
 drivers/gpu/nova-core/fb/hal/ga102.rs |  4 ++
 drivers/gpu/nova-core/fb/hal/tu102.rs | 12 ++++-
 drivers/gpu/nova-core/firmware/gsp.rs |  2 +-
 drivers/gpu/nova-core/gsp/boot.rs     |  5 +-
 drivers/gpu/nova-core/vbios.rs        | 69 ++++++++++++++++++++++++++-
 9 files changed, 99 insertions(+), 10 deletions(-)

base-commit: 8884dbc1fd174315b406027375bb7a4b211a8fae

Signature · 1 line
2.53.0
author
#1 lore raw

PATCH v2 1/6 gpu: nova-core: use correct fwsignature for GA100

Although GA100 uses the same GSP-RM firmware as Turing, it has a different signature specifically for it.

Fixes 121ea04cd9f2 ("gpu: nova-core: add support for Turing/GA100 fwsignature")
---
 drivers/gpu/nova-core/firmware/gsp.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/nova-core/firmware/gsp.rs +1 -1
index 2fcc255c3bc8..63e464788c0b 100644
@@ -139,7 +139,7 @@ pub(crate) fn new<'a>(
                         }
                         Architecture::Turing => ".fwsignature_tu10x",
                         // GA100 uses the same firmware as Turing
-                        Architecture::Ampere if chipset == Chipset::GA100 => ".fwsignature_tu10x",
+                        Architecture::Ampere if chipset == Chipset::GA100 => ".fwsignature_ga100",
                         Architecture::Ampere => ".fwsignature_ga10x",
                         Architecture::Ada => ".fwsignature_ad10x",
                     };
Signature · 1 line
2.53.0
reviewer
#2 lore raw

Re: [PATCH v2 1/6] gpu: nova-core: use correct fwsignature for GA100

Quoted 15 lines · Timur Tabi

Although GA100 uses the same GSP-RM firmware as Turing, it has a different signature specifically for it.

Fixes 121ea04cd9f2 ("gpu: nova-core: add support for Turing/GA100 fwsignature")
---
 drivers/gpu/nova-core/firmware/gsp.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/nova-core/firmware/gsp.rs +0 -0
index 2fcc255c3bc8..63e464788c0b 100644

drivers/gpu/nova-core/firmware/gsp.rs pub(crate) fn new<'a>( · line 141

Quoted 3 lines
drivers/gpu/nova-core/firmware/gsp.rs +0 -0
@@ -139,7 +139,7 @@ pub(crate) fn new<'a>(
                         }
                         Architecture::Turing => ".fwsignature_tu10x",
                         // GA100 uses the same firmware as Turing

This comment should be removed.

Best,
Gary

Quoted 5 lines
drivers/gpu/nova-core/firmware/gsp.rs +1 -1
-                        Architecture::Ampere if chipset == Chipset::GA100 => ".fwsignature_tu10x",
+                        Architecture::Ampere if chipset == Chipset::GA100 => ".fwsignature_ga100",
                         Architecture::Ampere => ".fwsignature_ga10x",
                         Architecture::Ada => ".fwsignature_ad10x",
                     };
author
#3 lore raw

Re: [PATCH v2 1/6] gpu: nova-core: use correct fwsignature for GA100

Quoted 4 lines · Gary Guo
Quoted 2 lines

                          Architecture::Turing => ".fwsignature_tu10x", // GA100 uses the same firmware as Turing

This comment should be removed.

This is relevant because GA102 does not, even though both GA100 and GA102 are Amperes.

I could word it better, but it's an important factoid.

reviewer
#4 lore raw

Re: [PATCH v2 1/6] gpu: nova-core: use correct fwsignature for GA100

Quoted 9 lines · Timur Tabi
Quoted 4 lines · Gary Guo
Quoted 2 lines

                          Architecture::Turing => ".fwsignature_tu10x", // GA100 uses the same firmware as Turing

This comment should be removed.

This is relevant because GA102 does not, even though both GA100 and GA102 are Amperes.

I could word it better, but it's an important factoid.

It's an important factoid, but it's not relevant here for fwsignature, right? It doesn't use fwsignature_tu... so the factoid does not belong here.

Best,
Gary

author
#5 lore raw

Re: [PATCH v2 1/6] gpu: nova-core: use correct fwsignature for GA100

Quoted 6 lines · Gary Guo
Quoted 3 lines

This is relevant because GA102 does not, even though both GA100 and GA102 are Amperes.

I could word it better, but it's an important factoid.

It's an important factoid, but it's not relevant here for fwsignature, right? It doesn't use fwsignature_tu... so the factoid does not belong here.

Well, maybe. The point is that if you want to find out where that signature actually exists, even though it's an Ampere, the signature exists in Turing's firmware file.

author
#6 lore raw

PATCH v2 2/6 gpu: nova-core: do not consider 0xBB77 as a valid PCI ROM header signature

In reply to Timur Tabi

Nvidia GPUs have some PCI expansion ROM sections that have an Nvidia- specific signature instead of 0xAA55. Signature 0xBB77 is actually an internal-only value that has been deprecated for over a decade. Nova-core will never encounter a GPU with that signature, so don't look for it.

---
 drivers/gpu/nova-core/vbios.rs | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
drivers/gpu/nova-core/vbios.rs +1 -1
index ebda28e596c5..e726594eb130 100644
@@ -491,7 +491,7 @@ fn new(dev: &device::Device, data: &[u8]) -> Result<Self> {
 
         // Check for valid ROM signatures.
         match signature {
-            0xAA55 | 0xBB77 | 0x4E56 => {}
+            0xAA55 | 0x4E56 => {}
             _ => {
                 dev_err!(dev, "ROM signature unknown {:#x}\n", signature);
                 return Err(EINVAL);
Signature · 1 line
2.53.0
author
#7 lore raw

PATCH v2 3/6 gpu: nova-core: only boot FRTS if its region is allocated

In reply to Timur Tabi

On some Nvidia GPUs (i.e. GA100), the FRTS region is not allocated (its size is set to 0). In such cases, FWSEC-FRTS should not be run.

---
 drivers/gpu/nova-core/gsp/boot.rs | 5 ++++-
 1 file changed, 4 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/gsp/boot.rs +4 -1
index 18f356c9178e..5c56f0539dd7 100644
@@ -155,7 +155,10 @@ pub(crate) fn boot(
         let fb_layout = FbLayout::new(chipset, bar, &gsp_fw)?;
         dev_dbg!(dev, "{:#x?}\n", fb_layout);
 
-        Self::run_fwsec_frts(dev, chipset, gsp_falcon, bar, &bios, &fb_layout)?;
+        // FWSEC-FRTS is not executed on chips where the FRTS region size is 0 (e.g. GA100).
+        if !fb_layout.frts.is_empty() {
+            Self::run_fwsec_frts(dev, chipset, gsp_falcon, bar, &bios, &fb_layout)?;
+        }
 
         let booter_loader = BooterFirmware::new(
             dev,
Signature · 1 line
2.53.0
author
#9 lore raw

PATCH v2 4/6 gpu: nova-core: add FbHal::frts_size() for GA100 support

In reply to Timur Tabi

Introduce FbHal method frts_size() to return the size of the FRTS window. GA100 is a special case in that there is no FRTS, and so the size must be set to 0.

Note that we cannot use supports_display() to determine the FRTS size because there are other GPUs (e.g. GA102GL) that have display disabled (and so supports_display() returns False), but the FRTS window size still needs to be 1MB.

---
 drivers/gpu/nova-core/fb.rs           |  6 +++---
 drivers/gpu/nova-core/fb/hal.rs       |  3 +++
 drivers/gpu/nova-core/fb/hal/ga100.rs |  5 +++++
 drivers/gpu/nova-core/fb/hal/ga102.rs |  4 ++++
 drivers/gpu/nova-core/fb/hal/tu102.rs | 12 +++++++++++-
 5 files changed, 26 insertions(+), 4 deletions(-)
drivers/gpu/nova-core/fb.rs +3 -3
index f357fb28b22c..a305a6dac758 100644
@@ -216,10 +216,10 @@ pub(crate) fn new(chipset: Chipset, bar: &Bar0, gsp_fw: &GspFirmware) -> Result<
 
         let frts = {
             const FRTS_DOWN_ALIGN: Alignment = Alignment::new::<SZ_128K>();
-            const FRTS_SIZE: u64 = usize_as_u64(SZ_1M);
-            let frts_base = vga_workspace.start.align_down(FRTS_DOWN_ALIGN) - FRTS_SIZE;
+            let frts_size: u64 = hal.frts_size();
+            let frts_base = vga_workspace.start.align_down(FRTS_DOWN_ALIGN) - frts_size;
 
-            FbRange(frts_base..frts_base + FRTS_SIZE)
+            FbRange(frts_base..frts_base + frts_size)
         };
 
         let boot = {
drivers/gpu/nova-core/fb/hal.rs +3 -0
index aba0abd8ee00..1c01a6cbed65 100644
@@ -25,6 +25,9 @@ pub(crate) trait FbHal {
 
     /// Returns the VRAM size, in bytes.
     fn vidmem_size(&self, bar: &Bar0) -> u64;
+
+    /// Returns the FRTS size, in bytes.
+    fn frts_size(&self) -> u64;
 }
 
 /// Returns the HAL corresponding to `chipset`.
drivers/gpu/nova-core/fb/hal/ga100.rs +5 -0
index 1c03783cddef..0e7d91fbd130 100644
@@ -66,6 +66,11 @@ fn supports_display(&self, bar: &Bar0) -> bool {
     fn vidmem_size(&self, bar: &Bar0) -> u64 {
         super::tu102::vidmem_size_gp102(bar)
     }
+
+    // GA100 is a special case where it has no FRTS.
+    fn frts_size(&self) -> u64 {
+        0
+    }
 }
 
 const GA100: Ga100 = Ga100;
drivers/gpu/nova-core/fb/hal/ga102.rs +4 -0
index 4b9f0f74d0e7..3bb66f64bef7 100644
@@ -35,6 +35,10 @@ fn supports_display(&self, bar: &Bar0) -> bool {
     fn vidmem_size(&self, bar: &Bar0) -> u64 {
         vidmem_size_ga102(bar)
     }
+
+    fn frts_size(&self) -> u64 {
+        super::tu102::frts_size_tu102()
+    }
 }
 
 const GA102: Ga102 = Ga102;
drivers/gpu/nova-core/fb/hal/tu102.rs +11 -1
index 281bb796e198..3c85bf1b627b 100644
@@ -2,12 +2,14 @@
 
 use kernel::{
     io::Io,
-    prelude::*, //
+    prelude::*,
+    sizes::*, //
 };
 
 use crate::{
     driver::Bar0,
     fb::hal::FbHal,
+    num::*,
     regs, //
 };
 
@@ -38,6 +40,10 @@ pub(super) fn vidmem_size_gp102(bar: &Bar0) -> u64 {
         .usable_fb_size()
 }
 
+pub(super) const fn frts_size_tu102() -> u64 {
+    usize_as_u64(SZ_1M)
+}
+
 struct Tu102;
 
 impl FbHal for Tu102 {
@@ -56,6 +62,10 @@ fn supports_display(&self, bar: &Bar0) -> bool {
     fn vidmem_size(&self, bar: &Bar0) -> u64 {
         vidmem_size_gp102(bar)
     }
+
+    fn frts_size(&self) -> u64 {
+        frts_size_tu102()
+    }
 }
 
 const TU102: Tu102 = Tu102;
Signature · 1 line
2.53.0
Reviewed-by on v2 4/6
#10 lore raw

Re: [PATCH v2 4/6] gpu: nova-core: add FbHal::frts_size() for GA100 support

Quoted 11 lines · Timur Tabi

Introduce FbHal method frts_size() to return the size of the FRTS window. GA100 is a special case in that there is no FRTS, and so the size must be set to 0.

Note that we cannot use supports_display() to determine the FRTS size because there are other GPUs (e.g. GA102GL) that have display disabled (and so supports_display() returns False), but the FRTS window size still needs to be 1MB.

---

#11 lore raw

Re: [PATCH v2 4/6] gpu: nova-core: add FbHal::frts_size() for GA100 support

In reply to Timur Tabi
Quoted 55 lines · Timur Tabi

Introduce FbHal method frts_size() to return the size of the FRTS window. GA100 is a special case in that there is no FRTS, and so the size must be set to 0.

Note that we cannot use supports_display() to determine the FRTS size because there are other GPUs (e.g. GA102GL) that have display disabled (and so supports_display() returns False), but the FRTS window size still needs to be 1MB.

---
 drivers/gpu/nova-core/fb.rs           |  6 +++---
 drivers/gpu/nova-core/fb/hal.rs       |  3 +++
 drivers/gpu/nova-core/fb/hal/ga100.rs |  5 +++++
 drivers/gpu/nova-core/fb/hal/ga102.rs |  4 ++++
 drivers/gpu/nova-core/fb/hal/tu102.rs | 12 +++++++++++-
 5 files changed, 26 insertions(+), 4 deletions(-)
drivers/gpu/nova-core/fb.rs +3 -3
index f357fb28b22c..a305a6dac758 100644
@@ -216,10 +216,10 @@ pub(crate) fn new(chipset: Chipset, bar: &Bar0, gsp_fw: &GspFirmware) -> Result<
 
         let frts = {
             const FRTS_DOWN_ALIGN: Alignment = Alignment::new::<SZ_128K>();
-            const FRTS_SIZE: u64 = usize_as_u64(SZ_1M);
-            let frts_base = vga_workspace.start.align_down(FRTS_DOWN_ALIGN) - FRTS_SIZE;
+            let frts_size: u64 = hal.frts_size();
+            let frts_base = vga_workspace.start.align_down(FRTS_DOWN_ALIGN) - frts_size;
 
-            FbRange(frts_base..frts_base + FRTS_SIZE)
+            FbRange(frts_base..frts_base + frts_size)
         };
 
         let boot = {
drivers/gpu/nova-core/fb/hal.rs +3 -0
index aba0abd8ee00..1c01a6cbed65 100644
@@ -25,6 +25,9 @@ pub(crate) trait FbHal {
 
     /// Returns the VRAM size, in bytes.
     fn vidmem_size(&self, bar: &Bar0) -> u64;
+
+    /// Returns the FRTS size, in bytes.
+    fn frts_size(&self) -> u64;
 }
 
 /// Returns the HAL corresponding to `chipset`.
drivers/gpu/nova-core/fb/hal/ga100.rs +0 -0
index 1c03783cddef..0e7d91fbd130 100644

drivers/gpu/nova-core/fb/hal/ga100.rs fn supports_display(&self, bar: &Bar0) -> bool { · line 70

Quoted 5 lines
drivers/gpu/nova-core/fb/hal/ga100.rs +2 -0
@@ -66,6 +66,11 @@ fn supports_display(&self, bar: &Bar0) -> bool {
     fn vidmem_size(&self, bar: &Bar0) -> u64 {
         super::tu102::vidmem_size_gp102(bar)
     }
+
+    // GA100 is a special case where it has no FRTS.

From your reply to the previous revision [1] I understood that this should rather say that FRTS is empty. Should this comment be updated?

[1] https://lore.kernel.org/[email protected]

Quoted 42 lines
drivers/gpu/nova-core/fb/hal/ga100.rs +3 -0
+    fn frts_size(&self) -> u64 {
+        0
+    }
 }
 
 const GA100: Ga100 = Ga100;
drivers/gpu/nova-core/fb/hal/ga102.rs +4 -0
index 4b9f0f74d0e7..3bb66f64bef7 100644
@@ -35,6 +35,10 @@ fn supports_display(&self, bar: &Bar0) -> bool {
     fn vidmem_size(&self, bar: &Bar0) -> u64 {
         vidmem_size_ga102(bar)
     }
+
+    fn frts_size(&self) -> u64 {
+        super::tu102::frts_size_tu102()
+    }
 }
 
 const GA102: Ga102 = Ga102;
drivers/gpu/nova-core/fb/hal/tu102.rs +3 -1
index 281bb796e198..3c85bf1b627b 100644
@@ -2,12 +2,14 @@
 
 use kernel::{
     io::Io,
-    prelude::*, //
+    prelude::*,
+    sizes::*, //
 };
 
 use crate::{
     driver::Bar0,
     fb::hal::FbHal,
+    num::*,
     regs, //
 };
 

drivers/gpu/nova-core/fb/hal/tu102.rs pub(super) fn vidmem_size_gp102(bar: &Bar0) -> u64 { · line 44

Quoted 5 lines
drivers/gpu/nova-core/fb/hal/tu102.rs +2 -0
@@ -38,6 +40,10 @@ pub(super) fn vidmem_size_gp102(bar: &Bar0) -> u64 {
         .usable_fb_size()
 }
 
+pub(super) const fn frts_size_tu102() -> u64 {
+    usize_as_u64(SZ_1M)

Since we will apply this on top of -rc1, we can use John's u64::SZ_1M instead. It is available in master so feel free to base your series on that.

reviewer
#12 lore raw

Re: [PATCH v2 4/6] gpu: nova-core: add FbHal::frts_size() for GA100 support

In reply to Timur Tabi
Quoted 8 lines · Timur Tabi

Introduce FbHal method frts_size() to return the size of the FRTS window. GA100 is a special case in that there is no FRTS, and so the size must be set to 0.

Note that we cannot use supports_display() to determine the FRTS size because there are other GPUs (e.g. GA102GL) that have display disabled (and so supports_display() returns False), but the FRTS window size still needs to be 1MB.

Do we want to special case 0, or shoul we use None instead to represent that FRTS region dopes not exist?

Best,
Gary

Quoted 115 lines
---
 drivers/gpu/nova-core/fb.rs           |  6 +++---
 drivers/gpu/nova-core/fb/hal.rs       |  3 +++
 drivers/gpu/nova-core/fb/hal/ga100.rs |  5 +++++
 drivers/gpu/nova-core/fb/hal/ga102.rs |  4 ++++
 drivers/gpu/nova-core/fb/hal/tu102.rs | 12 +++++++++++-
 5 files changed, 26 insertions(+), 4 deletions(-)
drivers/gpu/nova-core/fb.rs +3 -3
index f357fb28b22c..a305a6dac758 100644
@@ -216,10 +216,10 @@ pub(crate) fn new(chipset: Chipset, bar: &Bar0, gsp_fw: &GspFirmware) -> Result<
 
         let frts = {
             const FRTS_DOWN_ALIGN: Alignment = Alignment::new::<SZ_128K>();
-            const FRTS_SIZE: u64 = usize_as_u64(SZ_1M);
-            let frts_base = vga_workspace.start.align_down(FRTS_DOWN_ALIGN) - FRTS_SIZE;
+            let frts_size: u64 = hal.frts_size();
+            let frts_base = vga_workspace.start.align_down(FRTS_DOWN_ALIGN) - frts_size;
 
-            FbRange(frts_base..frts_base + FRTS_SIZE)
+            FbRange(frts_base..frts_base + frts_size)
         };
 
         let boot = {
drivers/gpu/nova-core/fb/hal.rs +3 -0
index aba0abd8ee00..1c01a6cbed65 100644
@@ -25,6 +25,9 @@ pub(crate) trait FbHal {
 
     /// Returns the VRAM size, in bytes.
     fn vidmem_size(&self, bar: &Bar0) -> u64;
+
+    /// Returns the FRTS size, in bytes.
+    fn frts_size(&self) -> u64;
 }
 
 /// Returns the HAL corresponding to `chipset`.
drivers/gpu/nova-core/fb/hal/ga100.rs +5 -0
index 1c03783cddef..0e7d91fbd130 100644
@@ -66,6 +66,11 @@ fn supports_display(&self, bar: &Bar0) -> bool {
     fn vidmem_size(&self, bar: &Bar0) -> u64 {
         super::tu102::vidmem_size_gp102(bar)
     }
+
+    // GA100 is a special case where it has no FRTS.
+    fn frts_size(&self) -> u64 {
+        0
+    }
 }
 
 const GA100: Ga100 = Ga100;
drivers/gpu/nova-core/fb/hal/ga102.rs +4 -0
index 4b9f0f74d0e7..3bb66f64bef7 100644
@@ -35,6 +35,10 @@ fn supports_display(&self, bar: &Bar0) -> bool {
     fn vidmem_size(&self, bar: &Bar0) -> u64 {
         vidmem_size_ga102(bar)
     }
+
+    fn frts_size(&self) -> u64 {
+        super::tu102::frts_size_tu102()
+    }
 }
 
 const GA102: Ga102 = Ga102;
drivers/gpu/nova-core/fb/hal/tu102.rs +11 -1
index 281bb796e198..3c85bf1b627b 100644
@@ -2,12 +2,14 @@
 
 use kernel::{
     io::Io,
-    prelude::*, //
+    prelude::*,
+    sizes::*, //
 };
 
 use crate::{
     driver::Bar0,
     fb::hal::FbHal,
+    num::*,
     regs, //
 };
 
@@ -38,6 +40,10 @@ pub(super) fn vidmem_size_gp102(bar: &Bar0) -> u64 {
         .usable_fb_size()
 }
 
+pub(super) const fn frts_size_tu102() -> u64 {
+    usize_as_u64(SZ_1M)
+}
+
 struct Tu102;
 
 impl FbHal for Tu102 {
@@ -56,6 +62,10 @@ fn supports_display(&self, bar: &Bar0) -> bool {
     fn vidmem_size(&self, bar: &Bar0) -> u64 {
         vidmem_size_gp102(bar)
     }
+
+    fn frts_size(&self) -> u64 {
+        frts_size_tu102()
+    }
 }
 
 const TU102: Tu102 = Tu102;
#13 lore raw

Re: [PATCH v2 4/6] gpu: nova-core: add FbHal::frts_size() for GA100 support

Quoted 5 lines · Gary Guo
Quoted 4 lines · Timur Tabi

Introduce FbHal method frts_size() to return the size of the FRTS window. GA100 is a special case in that there is no FRTS, and so the size must be set to 0.

Quoted 7 lines
Quoted 4 lines

Note that we cannot use supports_display() to determine the FRTS size because there are other GPUs (e.g. GA102GL) that have display disabled (and so supports_display() returns False), but the FRTS window size still needs to be 1MB.

Do we want to special case 0, or shoul we use None instead to represent that FRTS region dopes not exist?

This was discussed in [1] - using 0 seems to be the correct approach.

[1] https://lore.kernel.org/all/[email protected]/

reviewer
#14 lore raw

Re: [PATCH v2 4/6] gpu: nova-core: add FbHal::frts_size() for GA100 support

Quoted 11 lines · Alexandre Courbot
Quoted 10 lines · Gary Guo
Quoted 8 lines · Timur Tabi

Introduce FbHal method frts_size() to return the size of the FRTS window. GA100 is a special case in that there is no FRTS, and so the size must be set to 0.

Note that we cannot use supports_display() to determine the FRTS size because there are other GPUs (e.g. GA102GL) that have display disabled (and so supports_display() returns False), but the FRTS window size still needs to be 1MB.

Quoted 6 lines
Quoted 2 lines

Do we want to special case 0, or shoul we use None instead to represent that FRTS region dopes not exist?

This was discussed in [1] - using 0 seems to be the correct approach.

[1] https://lore.kernel.org/all/[email protected]/

Okay, so thye important part is frts_start still matters. Perhaps this should be written down somewhere, in the commit message or as comment to HAL.

Best,
Gary

author
#15 lore raw

Re: [PATCH v2 4/6] gpu: nova-core: add FbHal::frts_size() for GA100 support

Quoted 2 lines · Gary Guo

Okay, so thye important part is frts_start still matters. Perhaps this should be written down somewhere, in the commit message or as comment to HAL.

It's in the commit message(s) already, but I can add a comment in the code as well.

author
#16 lore raw

PATCH v2 5/6 gpu: nova-core: skip the IFR header if present

In reply to Timur Tabi

The GPU's ROM may begin with an Init-from-ROM (IFR) header that precedes the PCI Expansion ROM images (VBIOS). When present, the PROM shadow method must parse this header to determine the offset where the PCI ROM images actually begin, and adjust all subsequent reads accordingly.

On most GPUs this is not needed because the IFR microcode has already applied the ROM offset so that PROM reads transparently skip the header. On GA100, for whatever reason, the IFR offset is not applied to PROM reads. Therefore, the search for the PCI expansion must skip the IFR header, if found.

---
 drivers/gpu/nova-core/vbios.rs | 67 +++++++++++++++++++++++++++++++++-
 1 file changed, 66 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/vbios.rs +66 -1
index e726594eb130..000e823b9e47 100644
@@ -89,13 +89,78 @@ struct VbiosIterator<'a> {
     last_found: bool,
 }
 
+/// Offset of FIXED0 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED0: usize = 0x00000000;
+/// Offset of FIXED1 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED1: usize = 0x00000004;
+/// Offset of FIXED2 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED2: usize = 0x00000008;
+/// IFR signature: ASCII "NVGI" as a little-endian u32.
+const NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE: u32 = 0x4947564E;
+/// ROM directory signature: ASCII "RFRD" as a little-endian u32.
+const NV_ROM_DIRECTORY_IDENTIFIER: u32 = 0x44524652;
+/// Offset of the NV_PMGR_ROM_ADDR_OFFSET register in IFR Extended section
+const IFR_SW_EXT_ROM_ADDR_OFFSET: usize = 4;
+/// Size of Redundant Firmware Flash Status section
+const RFW_FLASH_STATUS_SIZE: usize = 4096;
+
+bitfield! {
+    struct IfrFixed1(u32) {
+        15:8 version as u8;
+        30:16 fixed_data_size as u16;
+    }
+}
+
+bitfield! {
+    struct IfrFixed2(u32) {
+        19:0 total_data_size as u32;
+    }
+}
+
 impl<'a> VbiosIterator<'a> {
     fn new(dev: &'a device::Device, bar0: &'a Bar0) -> Result<Self> {
+        let signature = bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED0)?;
+
+        // The ROM may start with an Init-from-ROM (IFR) header before the PCI
+        // Expansion ROM images. Most GPUs apply the IFR offset transparently, but
+        // GA100 does not, so we must skip the header manually if present.
+
+        let current_offset = if signature == NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE {
+            let fixed1 = IfrFixed1(bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED1)?);
+
+            match fixed1.version() {
+                // Note: We do not actually expect to see v1 or v2 on these GPUs
+                1 | 2 => {
+                    let fixed_data_size = fixed1.fixed_data_size() as usize;
+                    let pmgr_rom_addr_offset = fixed_data_size + IFR_SW_EXT_ROM_ADDR_OFFSET;
+                    bar0.try_read32(ROM_OFFSET + pmgr_rom_addr_offset)? as usize
+                }
+                3 => {
+                    let fixed2 = IfrFixed2(bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED2)?);
+                    let total_data_size = fixed2.total_data_size() as usize;
+                    let dir_offset = bar0.try_read32(ROM_OFFSET + total_data_size)? as usize
+                        + RFW_FLASH_STATUS_SIZE;
+                    let dir_sig = bar0.try_read32(ROM_OFFSET + dir_offset)?;
+                    if dir_sig != NV_ROM_DIRECTORY_IDENTIFIER {
+                        dev_err!(dev, "could not find IFR ROM directory\n");
+                        return Err(EINVAL);
+                    }
+                    bar0.try_read32(ROM_OFFSET + dir_offset + 8)? as usize
+                }
+                _ => {
+                    dev_err!(dev, "unsupported IFR header version {}\n", fixed1.version());
+                    return Err(EINVAL);
+                }
+            }
+        } else {
+            0
+        };
+
         Ok(Self {
             dev,
             bar0,
             data: KVec::new(),
-            current_offset: 0,
+            current_offset,
             last_found: false,
         })
     }
Signature · 1 line
2.53.0
#17 lore raw

Re: [PATCH v2 5/6] gpu: nova-core: skip the IFR header if present

Quoted 22 lines · Timur Tabi

The GPU's ROM may begin with an Init-from-ROM (IFR) header that precedes the PCI Expansion ROM images (VBIOS). When present, the PROM shadow method must parse this header to determine the offset where the PCI ROM images actually begin, and adjust all subsequent reads accordingly.

On most GPUs this is not needed because the IFR microcode has already applied the ROM offset so that PROM reads transparently skip the header. On GA100, for whatever reason, the IFR offset is not applied to PROM reads. Therefore, the search for the PCI expansion must skip the IFR header, if found.

---
 drivers/gpu/nova-core/vbios.rs | 67 +++++++++++++++++++++++++++++++++-
 1 file changed, 66 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/vbios.rs +0 -0
index e726594eb130..000e823b9e47 100644
@@ -89,13 +89,78 @@ struct VbiosIterator<'a> {
     last_found: bool,

drivers/gpu/nova-core/vbios.rs struct VbiosIterator<'a> { · line 99

Quoted 10 lines
drivers/gpu/nova-core/vbios.rs +8 -0
 }
 
+/// Offset of FIXED0 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED0: usize = 0x00000000;
+/// Offset of FIXED1 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED1: usize = 0x00000004;
+/// Offset of FIXED2 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED2: usize = 0x00000008;
+/// IFR signature: ASCII "NVGI" as a little-endian u32.
+const NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE: u32 = 0x4947564E;

These ones are available in tu102/dev_bus.h in OpenRM and shouldn't be too hard to add. As a general convention for the driver IMO it is nice to get as many constants as we can via bindings.

The diff in nova-gsp-binding-generator looks like this:

nvidia-open-gpu-symbols.txt +4 -0
index cd6835fc00b6..9f08d842a3b2 100644
@@ -91,6 +91,10 @@ var:BLACKWELL_DMA_COPY_A
 var:BLACKWELL_DMA_COPY_B
 var:NV2080_ENGINE_TYPE_.*
 var:NV2080_CTRL_CMD_INTERNAL_DEVICE_INFO_MAX_ENTRIES
+var:NV_PBUS_IFR_FMT_FIXED0
+var:NV_PBUS_IFR_FMT_FIXED1
+var:NV_PBUS_IFR_FMT_FIXED2
+var:NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE
 var:NV_RAMIN_ALLOC_SIZE
 var:NV_RAMUSERD_CHAN_SIZE
 var:NVA06F_GP_ENTRY__SIZE
r570_144_bindings.h +3 -0
index cb7b62a446e9..1c1024da60f8 100644
@@ -42,6 +42,9 @@
 /* GP entry size */
 #include <class/cla06f.h>
 
+/* IFR header format constants */
+#include <turing/tu102/dev_bus.h>
+
 /* Instance memory and USERD sizes */
 #include <ampere/ga100/dev_ram.h>
 #include <maxwell/gm107/dev_ram.h>
Quoted 49 lines
drivers/gpu/nova-core/vbios.rs +47 -0
+/// ROM directory signature: ASCII "RFRD" as a little-endian u32.
+const NV_ROM_DIRECTORY_IDENTIFIER: u32 = 0x44524652;
+/// Offset of the NV_PMGR_ROM_ADDR_OFFSET register in IFR Extended section
+const IFR_SW_EXT_ROM_ADDR_OFFSET: usize = 4;
+/// Size of Redundant Firmware Flash Status section
+const RFW_FLASH_STATUS_SIZE: usize = 4096;
+
+bitfield! {
+    struct IfrFixed1(u32) {
+        15:8 version as u8;
+        30:16 fixed_data_size as u16;
+    }
+}
+
+bitfield! {
+    struct IfrFixed2(u32) {
+        19:0 total_data_size as u32;
+    }
+}
+
 impl<'a> VbiosIterator<'a> {
     fn new(dev: &'a device::Device, bar0: &'a Bar0) -> Result<Self> {
+        let signature = bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED0)?;
+
+        // The ROM may start with an Init-from-ROM (IFR) header before the PCI
+        // Expansion ROM images. Most GPUs apply the IFR offset transparently, but
+        // GA100 does not, so we must skip the header manually if present.
+
+        let current_offset = if signature == NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE {
+            let fixed1 = IfrFixed1(bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED1)?);
+
+            match fixed1.version() {
+                // Note: We do not actually expect to see v1 or v2 on these GPUs
+                1 | 2 => {
+                    let fixed_data_size = fixed1.fixed_data_size() as usize;
+                    let pmgr_rom_addr_offset = fixed_data_size + IFR_SW_EXT_ROM_ADDR_OFFSET;
+                    bar0.try_read32(ROM_OFFSET + pmgr_rom_addr_offset)? as usize
+                }
+                3 => {
+                    let fixed2 = IfrFixed2(bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED2)?);
+                    let total_data_size = fixed2.total_data_size() as usize;
+                    let dir_offset = bar0.try_read32(ROM_OFFSET + total_data_size)? as usize
+                        + RFW_FLASH_STATUS_SIZE;
+                    let dir_sig = bar0.try_read32(ROM_OFFSET + dir_offset)?;
+                    if dir_sig != NV_ROM_DIRECTORY_IDENTIFIER {
+                        dev_err!(dev, "could not find IFR ROM directory\n");
+                        return Err(EINVAL);
+                    }
+                    bar0.try_read32(ROM_OFFSET + dir_offset + 8)? as usize

drivers/gpu/nova-core/vbios.rs struct VbiosIterator<'a> { · line 158

Quoted 10 lines
drivers/gpu/nova-core/vbios.rs +10 -0
+                }
+                _ => {
+                    dev_err!(dev, "unsupported IFR header version {}\n", fixed1.version());
+                    return Err(EINVAL);
+                }
+            }
+        } else {
+            0
+        };
+

This (and the constants) is quite specific to GA100. What about putting this into a VbiosIterator::ifr_offset helper function? You can move the constants and bitfield definitions inside of that function as well, which would be nice since they aren't used outside of this area of the code at all.

Quoted 9 lines
drivers/gpu/nova-core/vbios.rs +1 -1
         Ok(Self {
             dev,
             bar0,
             data: KVec::new(),
-            current_offset: 0,
+            current_offset,
             last_found: false,
         })
     }
author
#18 lore raw

Re: [PATCH v2 5/6] gpu: nova-core: skip the IFR header if present

nvidia-open-gpu-symbols.txt var:BLACKWELL_DMA_COPY_A · line 97

Quoted 5 lines · Eliot Courtney

 var:NV2080_CTRL_CMD_INTERNAL_DEVICE_INFO_MAX_ENTRIES

nvidia-open-gpu-symbols.txt +4 -0
+var:NV_PBUS_IFR_FMT_FIXED0
+var:NV_PBUS_IFR_FMT_FIXED1
+var:NV_PBUS_IFR_FMT_FIXED2
+var:NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE

I converted the _FIXED macros to register! entries, and now the only thing I need now is:

/// IFR signature: ASCII "NVGI" as a little-endian u32.
const NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE: u32 = 0x4947564E; /// ROM directory signature: ASCII "RFRD" as a little-endian u32. const NV_ROM_DIRECTORY_IDENTIFIER: u32 = 0x44524652;
/// Offset of the NV_PMGR_ROM_ADDR_OFFSET register in IFR Extended section const IFR_SW_EXT_ROM_ADDR_OFFSET: usize = 4;
/// Size of Redundant Firmware Flash Status section
const RFW_FLASH_STATUS_SIZE: usize = 4096;

The second two I made up, so they can't be imported. NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE comes from dev_bus.h as you said, but NV_ROM_DIRECTORY_IDENTIFIER comes from kernel_gsp_vbios_tu102.c, so I don't think it can be imported either.

Also, what tree is your DIFF for?

author
#19 lore raw

Re: [PATCH v2 5/6] gpu: nova-core: skip the IFR header if present

Quoted 6 lines · Timur Tabi

The second two I made up, so they can't be imported.  NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE comes from dev_bus.h as you said, but NV_ROM_DIRECTORY_IDENTIFIER comes from kernel_gsp_vbios_tu102.c, so
I don't think it can be imported either.

Also, what tree is your DIFF for?

I've given this some thought, and I don't think it's the right approach. It has the side effect of having vbios.rs depend on constants that are defined in GSP version-specific header files. It's also a convoluted way to add just one constant.

So I think I will just keep

/// IFR signature: ASCII "NVGI" as a little-endian u32. const NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE: u32 = 0x4947564E;

in vbios.rs

#20 lore raw

Re: [PATCH v2 5/6] gpu: nova-core: skip the IFR header if present

Quoted 12 lines · Timur Tabi
Quoted 6 lines · Timur Tabi

The second two I made up, so they can't be imported.  NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE comes from dev_bus.h as you said, but NV_ROM_DIRECTORY_IDENTIFIER comes from kernel_gsp_vbios_tu102.c, so
I don't think it can be imported either.

Also, what tree is your DIFF for?

I've given this some thought, and I don't think it's the right approach. It has the side effect of having vbios.rs depend on constants that are defined in GSP version-specific header files. It's also a convoluted way to add just one constant.

Quoted 6 lines

So I think I will just keep

/// IFR signature: ASCII "NVGI" as a little-endian u32. const NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE: u32 = 0x4947564E;

in vbios.rs

Yeah, thinking again about it, these values are from the VBIOS - thus they are fixed irrespective of the GSP firmware version. We use the bindings to shield us against subtle changes between GSP firmware releases, but that's not going to happen here.

Getting our values from the bindings would also extend them from being pure-GSP to covering other stuff, which is something we should try to avoid.

#21 lore raw

Re: [PATCH v2 5/6] gpu: nova-core: skip the IFR header if present

In reply to Timur Tabi
Quoted 52 lines · Timur Tabi

The GPU's ROM may begin with an Init-from-ROM (IFR) header that precedes the PCI Expansion ROM images (VBIOS). When present, the PROM shadow method must parse this header to determine the offset where the PCI ROM images actually begin, and adjust all subsequent reads accordingly.

On most GPUs this is not needed because the IFR microcode has already applied the ROM offset so that PROM reads transparently skip the header. On GA100, for whatever reason, the IFR offset is not applied to PROM reads. Therefore, the search for the PCI expansion must skip the IFR header, if found.

---
 drivers/gpu/nova-core/vbios.rs | 67 +++++++++++++++++++++++++++++++++-
 1 file changed, 66 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/vbios.rs +28 -0
index e726594eb130..000e823b9e47 100644
@@ -89,13 +89,78 @@ struct VbiosIterator<'a> {
     last_found: bool,
 }
 
+/// Offset of FIXED0 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED0: usize = 0x00000000;
+/// Offset of FIXED1 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED1: usize = 0x00000004;
+/// Offset of FIXED2 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED2: usize = 0x00000008;
+/// IFR signature: ASCII "NVGI" as a little-endian u32.
+const NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE: u32 = 0x4947564E;
+/// ROM directory signature: ASCII "RFRD" as a little-endian u32.
+const NV_ROM_DIRECTORY_IDENTIFIER: u32 = 0x44524652;
+/// Offset of the NV_PMGR_ROM_ADDR_OFFSET register in IFR Extended section
+const IFR_SW_EXT_ROM_ADDR_OFFSET: usize = 4;
+/// Size of Redundant Firmware Flash Status section
+const RFW_FLASH_STATUS_SIZE: usize = 4096;
+
+bitfield! {
+    struct IfrFixed1(u32) {
+        15:8 version as u8;
+        30:16 fixed_data_size as u16;
+    }
+}
+
+bitfield! {
+    struct IfrFixed2(u32) {
+        19:0 total_data_size as u32;
+    }
+}
+

drivers/gpu/nova-core/vbios.rs struct VbiosIterator<'a> { · line 129

Quoted 10 lines
drivers/gpu/nova-core/vbios.rs +8 -0
 impl<'a> VbiosIterator<'a> {
     fn new(dev: &'a device::Device, bar0: &'a Bar0) -> Result<Self> {
+        let signature = bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED0)?;
+
+        // The ROM may start with an Init-from-ROM (IFR) header before the PCI
+        // Expansion ROM images. Most GPUs apply the IFR offset transparently, but
+        // GA100 does not, so we must skip the header manually if present.
+
+        let current_offset = if signature == NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE {
+            let fixed1 = IfrFixed1(bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED1)?);

Given that we are reading signature, IfrFixed1 and IfrFixed2 from a fixed offset in the BAR, how about turning these into registers? While they are technically not from a hardware perspective, for the driver we still access them the same way.

Since they are local to the bios, I would declare them in this file instead of regs.rs.

Also I'd move the code parsing the IFR header into a dedicated method so its name informs us about what it does (and provide an anchor to attach the documentation to). That way there can be no confusion of new keeps growing for other reasons.

reviewer
#22 lore raw

Re: [PATCH v2 5/6] gpu: nova-core: skip the IFR header if present

In reply to Timur Tabi
Quoted 41 lines · Timur Tabi

The GPU's ROM may begin with an Init-from-ROM (IFR) header that precedes the PCI Expansion ROM images (VBIOS). When present, the PROM shadow method must parse this header to determine the offset where the PCI ROM images actually begin, and adjust all subsequent reads accordingly.

On most GPUs this is not needed because the IFR microcode has already applied the ROM offset so that PROM reads transparently skip the header. On GA100, for whatever reason, the IFR offset is not applied to PROM reads. Therefore, the search for the PCI expansion must skip the IFR header, if found.

---
 drivers/gpu/nova-core/vbios.rs | 67 +++++++++++++++++++++++++++++++++-
 1 file changed, 66 insertions(+), 1 deletion(-)
drivers/gpu/nova-core/vbios.rs +17 -0
index e726594eb130..000e823b9e47 100644
@@ -89,13 +89,78 @@ struct VbiosIterator<'a> {
     last_found: bool,
 }
 
+/// Offset of FIXED0 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED0: usize = 0x00000000;
+/// Offset of FIXED1 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED1: usize = 0x00000004;
+/// Offset of FIXED2 field in IFR header
+const NV_PBUS_IFR_FMT_FIXED2: usize = 0x00000008;
+/// IFR signature: ASCII "NVGI" as a little-endian u32.
+const NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE: u32 = 0x4947564E;
+/// ROM directory signature: ASCII "RFRD" as a little-endian u32.
+const NV_ROM_DIRECTORY_IDENTIFIER: u32 = 0x44524652;
+/// Offset of the NV_PMGR_ROM_ADDR_OFFSET register in IFR Extended section
+const IFR_SW_EXT_ROM_ADDR_OFFSET: usize = 4;
+/// Size of Redundant Firmware Flash Status section
+const RFW_FLASH_STATUS_SIZE: usize = 4096;
+
+bitfield! {
+    struct IfrFixed1(u32) {

drivers/gpu/nova-core/vbios.rs struct VbiosIterator<'a> { · line 118

Quoted 10 lines
drivers/gpu/nova-core/vbios.rs +10 -0
+        15:8 version as u8;
+        30:16 fixed_data_size as u16;
+    }
+}
+
+bitfield! {
+    struct IfrFixed2(u32) {
+        19:0 total_data_size as u32;
+    }
+}

Shouldn't this be using the register! macro instead?

Best,
Gary

Quoted 49 lines
drivers/gpu/nova-core/vbios.rs +39 -1
+
 impl<'a> VbiosIterator<'a> {
     fn new(dev: &'a device::Device, bar0: &'a Bar0) -> Result<Self> {
+        let signature = bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED0)?;
+
+        // The ROM may start with an Init-from-ROM (IFR) header before the PCI
+        // Expansion ROM images. Most GPUs apply the IFR offset transparently, but
+        // GA100 does not, so we must skip the header manually if present.
+
+        let current_offset = if signature == NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE {
+            let fixed1 = IfrFixed1(bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED1)?);
+
+            match fixed1.version() {
+                // Note: We do not actually expect to see v1 or v2 on these GPUs
+                1 | 2 => {
+                    let fixed_data_size = fixed1.fixed_data_size() as usize;
+                    let pmgr_rom_addr_offset = fixed_data_size + IFR_SW_EXT_ROM_ADDR_OFFSET;
+                    bar0.try_read32(ROM_OFFSET + pmgr_rom_addr_offset)? as usize
+                }
+                3 => {
+                    let fixed2 = IfrFixed2(bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED2)?);
+                    let total_data_size = fixed2.total_data_size() as usize;
+                    let dir_offset = bar0.try_read32(ROM_OFFSET + total_data_size)? as usize
+                        + RFW_FLASH_STATUS_SIZE;
+                    let dir_sig = bar0.try_read32(ROM_OFFSET + dir_offset)?;
+                    if dir_sig != NV_ROM_DIRECTORY_IDENTIFIER {
+                        dev_err!(dev, "could not find IFR ROM directory\n");
+                        return Err(EINVAL);
+                    }
+                    bar0.try_read32(ROM_OFFSET + dir_offset + 8)? as usize
+                }
+                _ => {
+                    dev_err!(dev, "unsupported IFR header version {}\n", fixed1.version());
+                    return Err(EINVAL);
+                }
+            }
+        } else {
+            0
+        };
+
         Ok(Self {
             dev,
             bar0,
             data: KVec::new(),
-            current_offset: 0,
+            current_offset,
             last_found: false,
         })
     }
#23 lore raw

Re: [PATCH v2 5/6] gpu: nova-core: skip the IFR header if present

In reply to Timur Tabi

On Wed Apr 15, 2026 at 8:50 AM JST, Timur Tabi wrote:
<snip>

Quoted 5 lines
 impl<'a> VbiosIterator<'a> {
     fn new(dev: &'a device::Device, bar0: &'a Bar0) -> Result<Self> {
drivers/gpu/nova-core/vbios.rs +3 -0
+        let signature = bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED0)?;
+
+        // The ROM may start with an Init-from-ROM (IFR) header before the PCI

drivers/gpu/nova-core/vbios.rs struct VbiosIterator<'a> { · line 134

Quoted 10 lines
drivers/gpu/nova-core/vbios.rs +10 -0
+        // Expansion ROM images. Most GPUs apply the IFR offset transparently, but
+        // GA100 does not, so we must skip the header manually if present.
+
+        let current_offset = if signature == NV_PBUS_IFR_FMT_FIXED0_SIGNATURE_VALUE {
+            let fixed1 = IfrFixed1(bar0.try_read32(ROM_OFFSET + NV_PBUS_IFR_FMT_FIXED1)?);
+
+            match fixed1.version() {
+                // Note: We do not actually expect to see v1 or v2 on these GPUs
+                1 | 2 => {
+                    let fixed_data_size = fixed1.fixed_data_size() as usize;

Forgot this in my previous review: please use u32_as_usize and the other relevant functions of nova-core's num module to avoid the as keyword as much as possible.

author
#24 lore raw

PATCH v2 6/6 gpu: nova-core: enable GA100

In reply to Timur Tabi

GA100 is a compute-only variant of GA102 that boots GSP-RM like a Turing, although it also has its own unique requirements.

Now that all the pieces are in place, we can enable GA100 support. Although architecturally like an Ampere, GA100 uses the same GSP-RM firmware files as Turing, and therefore must boot it like Turing does. However, as a compute-only part, GA100 has no display engine.

---
 drivers/gpu/nova-core/falcon/hal.rs | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/gpu/nova-core/falcon/hal.rs +1 -2
index 71df33c79884..510046d4a577 100644
@@ -77,13 +77,12 @@ pub(super) fn falcon_hal<E: FalconEngine + 'static>(
     use Chipset::*;
 
     let hal = match chipset {
-        TU102 | TU104 | TU106 | TU116 | TU117 => {
+        TU102 | TU104 | TU106 | TU116 | TU117 | GA100 => {
             KBox::new(tu102::Tu102::<E>::new(), GFP_KERNEL)? as KBox<dyn FalconHal<E>>
         }
         GA102 | GA103 | GA104 | GA106 | GA107 | AD102 | AD103 | AD104 | AD106 | AD107 => {
             KBox::new(ga102::Ga102::<E>::new(), GFP_KERNEL)? as KBox<dyn FalconHal<E>>
         }
-        _ => return Err(ENOTSUPP),
     };
 
     Ok(hal)
Signature · 1 line
2.53.0
reviewer
#25 lore raw

Re: [PATCH v2 6/6] gpu: nova-core: enable GA100

Quoted 7 lines · Timur Tabi

GA100 is a compute-only variant of GA102 that boots GSP-RM like a Turing, although it also has its own unique requirements.

Now that all the pieces are in place, we can enable GA100 support. Although architecturally like an Ampere, GA100 uses the same GSP-RM firmware files as Turing, and therefore must boot it like Turing does. However, as a compute-only part, GA100 has no display engine.

This should go into the code as a comment.

Best,
Gary

Quoted 26 lines
---
 drivers/gpu/nova-core/falcon/hal.rs | 3 +--
 1 file changed, 1 insertion(+), 2 deletions(-)
drivers/gpu/nova-core/falcon/hal.rs +1 -2
index 71df33c79884..510046d4a577 100644
@@ -77,13 +77,12 @@ pub(super) fn falcon_hal<E: FalconEngine + 'static>(
     use Chipset::*;
 
     let hal = match chipset {
-        TU102 | TU104 | TU106 | TU116 | TU117 => {
+        TU102 | TU104 | TU106 | TU116 | TU117 | GA100 => {
             KBox::new(tu102::Tu102::<E>::new(), GFP_KERNEL)? as KBox<dyn FalconHal<E>>
         }
         GA102 | GA103 | GA104 | GA106 | GA107 | AD102 | AD103 | AD104 | AD106 | AD107 => {
             KBox::new(ga102::Ga102::<E>::new(), GFP_KERNEL)? as KBox<dyn FalconHal<E>>
         }
-        _ => return Err(ENOTSUPP),
     };
 
     Ok(hal)
author
#26 lore raw

Re: [PATCH v2 6/6] gpu: nova-core: enable GA100

Quoted 9 lines · Gary Guo
Quoted 7 lines

GA100 is a compute-only variant of GA102 that boots GSP-RM like a Turing, although it also has its own unique requirements.

Now that all the pieces are in place, we can enable GA100 support. Although architecturally like an Ampere, GA100 uses the same GSP-RM firmware files as Turing, and therefore must boot it like Turing does. However, as a compute-only part, GA100 has no display engine.

This should go into the code as a comment.

We don't describe any other GPU in the code. Where exactly do you think this comment should go?

reviewer
#27 lore raw

Re: [PATCH v2 6/6] gpu: nova-core: enable GA100

Quoted 4 lines · Timur Tabi
Quoted 3 lines · Gary Guo
Quoted 3 lines

GA100 is a compute-only variant of GA102 that boots GSP-RM like a Turing, although it also has its own unique requirements.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

Now that all the pieces are in place, we can enable GA100 support. Although architecturally like an Ampere, GA100 uses the same GSP-RM firmware files as Turing, and therefore must boot it like Turing does. However, as a compute-only part, GA100 has no display engine.

This should go into the code as a comment.

We don't describe any other GPU in the code. Where exactly do you think this comment should go?

Something like

GA100 | // GA100 boots like Turing so use Turing HAL TU102 | TU104 | TU106 | TU116 | TU117 => {

? Otherwise it just looks like broken code for those unaware of the details, given it goes and use TU HAL despite being GA.

Best,
Gary

author
#28 lore raw

Re: [PATCH v2 6/6] gpu: nova-core: enable GA100

Quoted 10 lines · Gary Guo
Quoted 2 lines

We don't describe any other GPU in the code.  Where exactly do you think this comment should go?

Something like

    GA100 | // GA100 boots like Turing so use Turing HAL TU102 | TU104 | TU106 | TU116 | TU117 => {

? Otherwise it just looks like broken code for those unaware of the details, given it goes and use TU HAL despite being GA.

Ok, that's fair.