Skip to contentSkip to conversation

rust/drm: Misc. gem bindings cleanup

PATCH Applied Lyude Paul 3 participants 8 messages Sep 8, 2025

Subsystems (4)
lore
Share & tools
lore

Lyude Paul

author
#0 lore raw

v4 cover rust/drm: Misc. gem bindings cleanup

This is the first few patches that were originally part of the series to introduce gem shmem bindings for rust into the Linux kernel, which can be found here:

https://lkml.org/lkml/2025/8/29/1533

These patches don't have any dependencies besides needing to be applied on top of drm-rust-next.

Lyude Paul (3):
rust: drm: gem: Simplify use of generics
rust: drm: gem: Add DriverFile type alias
rust: drm: gem: Drop Object::SIZE

 drivers/gpu/drm/nova/gem.rs |  8 ++--
 rust/kernel/drm/driver.rs   |  3 ++
 rust/kernel/drm/gem/mod.rs  | 93 ++++++++++++++++---------------------
 3 files changed, 47 insertions(+), 57 deletions(-)

base-commit: e2580413a83680f679904ad2f2c1aa6969876469

Signature · 1 line
2.51.0
author
#1 lore raw

PATCH v4 1/3 rust: drm: gem: Simplify use of generics

Now that my rust skills have been honed, I noticed that there's a lot of generics in our gem bindings that don't actually need to be here. Currently the hierarchy of traits in our gem bindings looks like this:

* Drivers implement:
* BaseDriverObject<T: DriverObject> (has the callbacks)
* DriverObject (has the drm::Driver type)
* Crate implements:
* IntoGEMObject for Object<T> where T: DriverObject Handles conversion to/from raw object pointers
* BaseObject for T where T: IntoGEMObject Provides methods common to all gem interfaces

Also of note, this leaves us with two different drm::Driver associated types:
* DriverObject::Driver
* IntoGEMObject::Driver

I'm not entirely sure of the original intent here unfortunately (if anyone is, please let me know!), but my guess is that the idea would be that some objects can implement IntoGEMObject using a different ::Driver than DriverObject - presumably to enable the usage of gem objects from different drivers. A reasonable usecase of course.

However - if I'm not mistaken, I don't think that this is actually how things would go in practice. Driver implementations are of course implemented by their associated drivers, and generally drivers are not linked to each-other when building the kernel. Which is to say that even in a situation where we would theoretically deal with gem objects from another driver, we still wouldn't have access to its drm::driver::Driver implementation. It's more likely we would simply want a variant of gem objects in such a situation that have no association with a drm::driver::Driver type.

Taking that into consideration, we can assume the following:
* Anything that implements BaseDriverObject will implement DriverObject In other words, all BaseDriverObjects indirectly have an associated ::Driver type - so the two traits can be combined into one with no generics.
* Not everything that implements IntoGEMObject will have an associated ::Driver, and that's OK.

And with this, we now can do quite a bit of cleanup with the use of generics here. As such, this commit:

* Removes the generics on BaseDriverObject
* Moves DriverObject::Driver into BaseDriverObject
* Removes DriverObject
* Removes IntoGEMObject::Driver
* Add AllocImpl::Driver, which we can use as a binding to figure out the correct File type for BaseObject

Leaving us with a simpler trait hierarchy that now looks like this:

* Drivers implement: BaseDriverObject
* Crate implements:
* IntoGEMObject for Object<T> where T: DriverObject
* BaseObject for T where T: IntoGEMObject

Which makes the code a lot easier to understand and build on :).

---
V2:
* Don't refer to Object<T> in callbacks, as this would result in drivers getting the wrong gem object type for shmem gem objects once we add support for those. Instead, we'll just add a type alias to clean this part up.
V3:
* Fix nova compilation
* Also, add an associated driver type to AllocImpl - as we still need the current driver accessible from BaseObject so that we can use the driver's various associated types, like File
* Add missing Object = Self constraint to type bounds for create_handle, lookup_handle. I forgot that if drivers can have private gem objects with a different data layout, we can only guarantee gem objects with handles are of the same gem object type as the main one in use by the driver.
V4:
* Add <Driver = D> to DriverFile bounds for create_handle(), lookup_handle()

 drivers/gpu/drm/nova/gem.rs |  8 ++--
 rust/kernel/drm/driver.rs   |  3 ++
 rust/kernel/drm/gem/mod.rs  | 77 ++++++++++++++++---------------------
 3 files changed, 40 insertions(+), 48 deletions(-)
drivers/gpu/drm/nova/gem.rs +3 -5
index cd82773dab92c..2760ba4f3450b 100644
@@ -16,16 +16,14 @@
 #[pin_data]
 pub(crate) struct NovaObject {}
 
-impl gem::BaseDriverObject<gem::Object<NovaObject>> for NovaObject {
+impl gem::DriverObject for NovaObject {
+    type Driver = NovaDriver;
+
     fn new(_dev: &NovaDevice, _size: usize) -> impl PinInit<Self, Error> {
         try_pin_init!(NovaObject {})
     }
 }
 
-impl gem::DriverObject for NovaObject {
-    type Driver = NovaDriver;
-}
-
 impl NovaObject {
     /// Create a new DRM GEM object.
     pub(crate) fn new(dev: &NovaDevice, size: usize) -> Result<ARef<gem::Object<Self>>> {
rust/kernel/drm/driver.rs +3 -0
index fe7e8d06961aa..dae0f4d1bbe3c 100644
@@ -86,6 +86,9 @@ pub struct AllocOps {
 
 /// Trait for memory manager implementations. Implemented internally.
 pub trait AllocImpl: super::private::Sealed + drm::gem::IntoGEMObject {
+    /// The [`Driver`] implementation for this [`AllocImpl`].
+    type Driver: drm::Driver;
+
     /// The C callback operations for this memory manager.
     const ALLOC_OPS: AllocOps;
 }
rust/kernel/drm/gem/mod.rs +34 -43

150 lines not on this page · raw message

Signature · 1 line
2.51.0
maintainer Acked-by on v4 1/3
#2 lore raw

Re: [PATCH v4 1/3] rust: drm: gem: Simplify use of generics

Quoted 59 lines · Lyude Paul

Now that my rust skills have been honed, I noticed that there's a lot of generics in our gem bindings that don't actually need to be here. Currently the hierarchy of traits in our gem bindings looks like this:

* Drivers implement:
* BaseDriverObject<T: DriverObject> (has the callbacks)
* DriverObject (has the drm::Driver type)
* Crate implements:
* IntoGEMObject for Object<T> where T: DriverObject Handles conversion to/from raw object pointers
* BaseObject for T where T: IntoGEMObject Provides methods common to all gem interfaces

Also of note, this leaves us with two different drm::Driver associated types:
* DriverObject::Driver
* IntoGEMObject::Driver

I'm not entirely sure of the original intent here unfortunately (if anyone is, please let me know!), but my guess is that the idea would be that some objects can implement IntoGEMObject using a different ::Driver than DriverObject - presumably to enable the usage of gem objects from different drivers. A reasonable usecase of course.

However - if I'm not mistaken, I don't think that this is actually how things would go in practice. Driver implementations are of course implemented by their associated drivers, and generally drivers are not linked to each-other when building the kernel. Which is to say that even in a situation where we would theoretically deal with gem objects from another driver, we still wouldn't have access to its drm::driver::Driver implementation. It's more likely we would simply want a variant of gem objects in such a situation that have no association with a drm::driver::Driver type.

Taking that into consideration, we can assume the following:
* Anything that implements BaseDriverObject will implement DriverObject In other words, all BaseDriverObjects indirectly have an associated ::Driver type - so the two traits can be combined into one with no generics.
* Not everything that implements IntoGEMObject will have an associated ::Driver, and that's OK.

And with this, we now can do quite a bit of cleanup with the use of generics here. As such, this commit:

* Removes the generics on BaseDriverObject
* Moves DriverObject::Driver into BaseDriverObject
* Removes DriverObject
* Removes IntoGEMObject::Driver
* Add AllocImpl::Driver, which we can use as a binding to figure out the correct File type for BaseObject

Leaving us with a simpler trait hierarchy that now looks like this:

* Drivers implement: BaseDriverObject
* Crate implements:
* IntoGEMObject for Object<T> where T: DriverObject
* BaseObject for T where T: IntoGEMObject

Quoted 4 lines

Which makes the code a lot easier to understand and build on :).

author
#3 lore raw

PATCH v4 2/3 rust: drm: gem: Add DriverFile type alias

In reply to Lyude Paul

Just to reduce the clutter with the File<…> types in gem.rs.

---
V3:
* Rename ObjectFile to DriverFile

 rust/kernel/drm/gem/mod.rs | 23 ++++++++++++-----------
 1 file changed, 12 insertions(+), 11 deletions(-)
rust/kernel/drm/gem/mod.rs +12 -11
index f5b19865d7453..ead723980b276 100644
@@ -14,6 +14,13 @@
 };
 use core::{mem, ops::Deref, ptr::NonNull};
 
+/// A type alias for retrieving a [`Driver`]s [`DriverFile`] implementation from its
+/// [`DriverObject`] implementation.
+///
+/// [`Driver`]: drm::Driver
+/// [`DriverFile`]: drm::file::DriverFile
+pub type DriverFile<T> = drm::File<<<T as DriverObject>::Driver as drm::Driver>::File>;
+
 /// GEM object functions, which must be implemented by drivers.
 pub trait DriverObject: Sync + Send + Sized {
     /// Parent `Driver` for this object.
@@ -23,19 +30,12 @@ pub trait DriverObject: Sync + Send + Sized {
     fn new(dev: &drm::Device<Self::Driver>, size: usize) -> impl PinInit<Self, Error>;
 
     /// Open a new handle to an existing object, associated with a File.
-    fn open(
-        _obj: &<Self::Driver as drm::Driver>::Object,
-        _file: &drm::File<<Self::Driver as drm::Driver>::File>,
-    ) -> Result {
+    fn open(_obj: &<Self::Driver as drm::Driver>::Object, _file: &DriverFile<Self>) -> Result {
         Ok(())
     }
 
     /// Close a handle to an existing object, associated with a File.
-    fn close(
-        _obj: &<Self::Driver as drm::Driver>::Object,
-        _file: &drm::File<<Self::Driver as drm::Driver>::File>,
-    ) {
-    }
+    fn close(_obj: &<Self::Driver as drm::Driver>::Object, _file: &DriverFile<Self>) {}
 }
 
 /// Trait that represents a GEM object subtype
@@ -79,7 +79,8 @@ extern "C" fn open_callback<T: DriverObject>(
     raw_file: *mut bindings::drm_file,
 ) -> core::ffi::c_int {
     // SAFETY: `open_callback` is only ever called with a valid pointer to a `struct drm_file`.
-    let file = unsafe { drm::File::<<T::Driver as drm::Driver>::File>::from_raw(raw_file) };
+    let file = unsafe { DriverFile::<T>::from_raw(raw_file) };
+
     // SAFETY: `open_callback` is specified in the AllocOps structure for `DriverObject<T>`,
     // ensuring that `raw_obj` is contained within a `DriverObject<T>`
     let obj = unsafe { <<T::Driver as drm::Driver>::Object as IntoGEMObject>::from_raw(raw_obj) };
@@ -95,7 +96,7 @@ extern "C" fn close_callback<T: DriverObject>(
     raw_file: *mut bindings::drm_file,
 ) {
     // SAFETY: `open_callback` is only ever called with a valid pointer to a `struct drm_file`.
-    let file = unsafe { drm::File::<<T::Driver as drm::Driver>::File>::from_raw(raw_file) };
+    let file = unsafe { DriverFile::<T>::from_raw(raw_file) };
 
     // SAFETY: `close_callback` is specified in the AllocOps structure for `Object<T>`, ensuring
     // that `raw_obj` is indeed contained within a `Object<T>`.
Signature · 1 line
2.51.0
author
#5 lore raw

PATCH v4 3/3 rust: drm: gem: Drop Object::SIZE

In reply to Lyude Paul

Drive-by fix, it doesn't seem like anything actually uses this constant anymore.

---
 rust/kernel/drm/gem/mod.rs | 5 +----
 1 file changed, 1 insertion(+), 4 deletions(-)
rust/kernel/drm/gem/mod.rs +1 -4
index ead723980b276..fd872de3b6695 100644
@@ -12,7 +12,7 @@
     prelude::*,
     types::{ARef, AlwaysRefCounted, Opaque},
 };
-use core::{mem, ops::Deref, ptr::NonNull};
+use core::{ops::Deref, ptr::NonNull};
 
 /// A type alias for retrieving a [`Driver`]s [`DriverFile`] implementation from its
 /// [`DriverObject`] implementation.
@@ -197,9 +197,6 @@ pub struct Object<T: DriverObject + Send + Sync> {
 }
 
 impl<T: DriverObject> Object<T> {
-    /// The size of this object's structure.
-    pub const SIZE: usize = mem::size_of::<Self>();
-
     const OBJECT_FUNCS: bindings::drm_gem_object_funcs = bindings::drm_gem_object_funcs {
         free: Some(Self::free_callback),
         open: Some(open_callback::<T>),
Signature · 1 line
2.51.0
maintainer Reviewed-by on the v4 series
#6 lore raw
In reply to Lyude Paul
Quoted 15 lines · Lyude Paul

This is the first few patches that were originally part of the series to introduce gem shmem bindings for rust into the Linux kernel, which can be found here:

https://lkml.org/lkml/2025/8/29/1533

These patches don't have any dependencies besides needing to be applied on top of drm-rust-next.

Lyude Paul (3):
rust: drm: gem: Simplify use of generics
rust: drm: gem: Add DriverFile type alias
rust: drm: gem: Drop Object::SIZE

Quoted 4 lines
 drivers/gpu/drm/nova/gem.rs |  8 ++--
 rust/kernel/drm/driver.rs   |  3 ++
 rust/kernel/drm/gem/mod.rs  | 93 ++++++++++++++++---------------------
 3 files changed, 47 insertions(+), 57 deletions(-)

For all 3 patches:

maintainer Applied
#7 lore raw
In reply to Lyude Paul
Quoted 6 lines · Lyude Paul

This is the first few patches that were originally part of the series to introduce gem shmem bindings for rust into the Linux kernel, which can be found here:

https://lkml.org/lkml/2025/8/29/1533

Quoted 7 lines

These patches don't have any dependencies besides needing to be applied on top of drm-rust-next.

Lyude Paul (3):
rust: drm: gem: Simplify use of generics
rust: drm: gem: Add DriverFile type alias
rust: drm: gem: Drop Object::SIZE

Applied to drm-rust-next. Thanks!