Skip to contentSkip to conversation

gpu: nova-core: fix aux device registration for multi-GPU systems

PATCH Applied John Hubbard 3 participants 3 messages Feb 5 – Feb 13

Subsystems (4)
lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 gpu: nova-core: fix aux device registration for multi-GPU systems

The auxiliary device registration was using a hardcoded ID of 0, which caused probe() to fail on multi-GPU systems with:

sysfs: cannot create duplicate filename '/bus/auxiliary/devices/NovaCore.nova-drm.0'

Fix this by using an LKMM atomic counter to generate unique IDs for each GPU's aux device registration. The TODO item to eventually use XArray for recycling aux device IDs is retained (and modified slightly: IDA might be better) but for now, this works very nicely.

This has the side effect of making debugfs[1] work on multi-GPU systems.

[1] https://lore.kernel.org/[email protected]

Cc Danilo Krummrich <[email protected]>
Cc Gary Guo <[email protected]>
---
 drivers/gpu/nova-core/driver.rs | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

Changes in v2:

* Use LKMM atomics (kernel::sync::atomic::Atomic<u32>) instead of Rust standard library atomics (core::sync::atomic::AtomicU32).

* Fix vertical import formatting (add rustfmt guard comment).

* Remove stray "we" from TODO comment.

drivers/gpu/nova-core/driver.rs +13 -1
index 5a4cc047bcfc..eec8871aa6a5 100644
@@ -14,11 +14,18 @@
     },
     prelude::*,
     sizes::SZ_16M,
+    sync::atomic::{
+        Atomic,
+        Relaxed, //
+    },
     sync::Arc, //
 };
 
 use crate::gpu::Gpu;
 
+/// Counter for generating unique auxiliary device IDs.
+static AUXILIARY_ID_COUNTER: Atomic<u32> = Atomic::new(0);
+
 #[pin_data]
 pub(crate) struct NovaCore {
     #[pin]
@@ -85,12 +92,17 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> impl PinInit<Self, E
                 GFP_KERNEL,
             )?;
 
+            // TODO[XARR]: Use XArray or perhaps IDA for proper ID allocation/recycling. For now,
+            // use a simple atomic counter that never recycles IDs. A unique ID is required for
+            // multi-GPU systems; without it, probe() fails for all but the first GPU.
+            let aux_id = AUXILIARY_ID_COUNTER.fetch_add(1, Relaxed);
+
             Ok(try_pin_init!(Self {
                 gpu <- Gpu::new(pdev, bar.clone(), bar.access(pdev.as_ref())?),
                 _reg <- auxiliary::Registration::new(
                     pdev.as_ref(),
                     c"nova-drm",
-                    0, // TODO[XARR]: Once it lands, use XArray; for now we don't use the ID.
+                    aux_id,
                     crate::MODULE_NAME
                 ),
             }))

base-commit: 9845cf73f7db6094c0d8419d6adb848028f4a921

Signature · 1 line
2.53.0
reviewer Reviewed-by on v2 1/1
#1 lore raw
Quoted 10 lines · John Hubbard

The auxiliary device registration was using a hardcoded ID of 0, which caused probe() to fail on multi-GPU systems with:

sysfs: cannot create duplicate filename '/bus/auxiliary/devices/NovaCore.nova-drm.0'

Fix this by using an LKMM atomic counter to generate unique IDs for each GPU's aux device registration. The TODO item to eventually use XArray for recycling aux device IDs is retained (and modified slightly: IDA might be better) but for now, this works very nicely.

Quoted 7 lines

This has the side effect of making debugfs[1] work on multi-GPU systems.

[1] https://lore.kernel.org/[email protected]

Cc Danilo Krummrich <[email protected]>
Cc Gary Guo <[email protected]>
Quoted 13 lines
---
 drivers/gpu/nova-core/driver.rs | 14 +++++++++++++-
 1 file changed, 13 insertions(+), 1 deletion(-)

Changes in v2:

* Use LKMM atomics (kernel::sync::atomic::Atomic<u32>) instead of Rust standard library atomics (core::sync::atomic::AtomicU32).

* Fix vertical import formatting (add rustfmt guard comment).

* Remove stray "we" from TODO comment.

maintainer Applied
#2 lore raw
In reply to John Hubbard
Quoted 10 lines · John Hubbard

The auxiliary device registration was using a hardcoded ID of 0, which caused probe() to fail on multi-GPU systems with:

sysfs: cannot create duplicate filename '/bus/auxiliary/devices/NovaCore.nova-drm.0'

Fix this by using an LKMM atomic counter to generate unique IDs for each GPU's aux device registration. The TODO item to eventually use XArray for recycling aux device IDs is retained (and modified slightly: IDA might be better) but for now, this works very nicely.

Quoted 7 lines

This has the side effect of making debugfs[1] work on multi-GPU systems.

[1] https://lore.kernel.org/[email protected]

Cc Danilo Krummrich <[email protected]>
Cc Gary Guo <[email protected]>

Queued for merging into drm-rust-next when it repoens - thanks for the fix!