Skip to contentSkip to conversation

rust: drm: gem: clean up GEM state in init failure case

PATCH Applied Eliot Courtney 4 participants 4 messages Apr 23 – May 1

Subsystems (3)
lore
Share & tools
lore
#0 lore raw

PATCH 1/1 rust: drm: gem: clean up GEM state in init failure case

Currently, if drm_gem_object_init fails, the object is freed without any cleanup. Perform the cleanup in that case.

---
I looked at drm_gem_shmem_init for an example, and it seems like the correct cleanup here is to do drm_gem_private_object_fini if drm_gem_object_init fails. Other C drivers do different things, but looking at the implementation of drm_gem_object_init, this looks like the only thing we need to clean up.

---
 rust/kernel/drm/gem/mod.rs | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
rust/kernel/drm/gem/mod.rs +8 -1
index 75acda7ba500..7b6a085ace27 100644
@@ -278,7 +278,14 @@ pub fn new(dev: &drm::Device<T::Driver>, size: usize, args: T::Args) -> Result<A
         unsafe { (*obj.as_raw()).funcs = &Self::OBJECT_FUNCS };
 
         // SAFETY: The arguments are all valid per the type invariants.
-        to_result(unsafe { bindings::drm_gem_object_init(dev.as_raw(), obj.obj.get(), size) })?;
+        if let Err(err) =
+            to_result(unsafe { bindings::drm_gem_object_init(dev.as_raw(), obj.obj.get(), size) })
+        {
+            // SAFETY: `drm_gem_object_init()` initializes the private GEM object state before
+            // failing, so `drm_gem_private_object_fini()` is the matching cleanup.
+            unsafe { bindings::drm_gem_private_object_fini(obj.obj.get()) };
+            return Err(err);
+        }
 
         // SAFETY: We will never move out of `Self` as `ARef<Self>` is always treated as pinned.
         let ptr = KBox::into_raw(unsafe { Pin::into_inner_unchecked(obj) });

---
base-commit: a7a080bb4236ebe577b6776d940d1717912ff6dd
change-id: 20260423-fix-gem-1-6c68b9fa0972

Best regards,

Signature · 1 line
Eliot Courtney <[email protected]>
reviewer Reviewed-by on v1 1/1
#1 lore raw
Quoted 21 lines · On Thu, 23 Apr 2026 21:36:52 +0900 Eliot Courtney <[email protected]> wrote:

Currently, if drm_gem_object_init fails, the object is freed without any cleanup. Perform the cleanup in that case.

---
I looked at drm_gem_shmem_init for an example, and it seems like the correct cleanup here is to do drm_gem_private_object_fini if drm_gem_object_init fails. Other C drivers do different things, but looking at the implementation of drm_gem_object_init, this looks like the only thing we need to clean up.

---
 rust/kernel/drm/gem/mod.rs | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
rust/kernel/drm/gem/mod.rs +0 -0
index 75acda7ba500..7b6a085ace27 100644
@@ -278,7 +278,14 @@ pub fn new(dev: &drm::Device<T::Driver>, size: usize, args: T::Args) -> Result<A
         unsafe { (*obj.as_raw()).funcs = &Self::OBJECT_FUNCS };
 

rust/kernel/drm/gem/mod.rs pub fn new(dev: &drm::Device<T::Driver>, size: usize, args: T::Args) -> Result<A · line 288

Quoted 10 lines
rust/kernel/drm/gem/mod.rs +8 -1
         // SAFETY: The arguments are all valid per the type invariants.
-        to_result(unsafe { bindings::drm_gem_object_init(dev.as_raw(), obj.obj.get(), size) })?;
+        if let Err(err) =
+            to_result(unsafe { bindings::drm_gem_object_init(dev.as_raw(), obj.obj.get(), size) })
+        {
+            // SAFETY: `drm_gem_object_init()` initializes the private GEM object state before
+            // failing, so `drm_gem_private_object_fini()` is the matching cleanup.
+            unsafe { bindings::drm_gem_private_object_fini(obj.obj.get()) };
+            return Err(err);
+        }

I can confirm that this matches the usage on the C side. I think the API could be improved to avoid relying on developers remembering this pattern. I don't expect that to happen in this patch, just saying it :).

Quoted 12 lines
rust/kernel/drm/gem/mod.rs +0 -1
         // SAFETY: We will never move out of `Self` as `ARef<Self>` is always treated as pinned.
         let ptr = KBox::into_raw(unsafe { Pin::into_inner_unchecked(obj) });
 
---

base-commit: a7a080bb4236ebe577b6776d940d1717912ff6dd
change-id: 20260423-fix-gem-1-6c68b9fa0972

Best regards,

Signature · 1 line
Eliot Courtney <[email protected]>
maintainer Reviewed-by on v1 1/1
#2 lore raw
In reply to Eliot Courtney
Quoted 5 lines · Eliot Courtney

Currently, if drm_gem_object_init fails, the object is freed without any cleanup. Perform the cleanup in that case.

Tricky stuff. I also concluded that this is correct.

This is a bugfix, so:

Fixes c284d3e42338 ("rust: drm: gem: Add GEM object abstraction")

Alice

maintainer Applied
#3 lore raw
In reply to Eliot Courtney
Quoted 1 line · Eliot Courtney

[PATCH] rust: drm: gem: clean up GEM state in init failure case

Applied, thanks!

Branch: drm-rust-fixes
Tree: https://gitlab.freedesktop.org/drm/rust/kernel.git

[1/1] rust: drm: gem: clean up GEM state in init failure case
commit: 2e42a17b8f6b

[ Move safety comment closer to unsafe block to avoid a clippy warning.
- Danilo ]

The patch will appear in the next linux-next integration (typically within 24 hours on weekdays).

The patch is queued up for Linus's tree and should land in the next -rc release.