Skip to contentSkip to conversation

workqueue: OwnedQueue and ScopedWork

PATCH In review Alice Ryhl 2 participants 9 messages Nov 19

Subsystems (3)
lore
Share & tools
lore

Alice Ryhl

author reviewer
#0 lore raw

v4 cover workqueue: OwnedQueue and ScopedWork

This series extends the Rust workqueue bindings with the ability to create custom workqueues (OwnedQueue), and introduces ScopedWork, a work item type that cancels synchronously on drop, allowing drivers to use workqueues with lifetime-bound data.

This is a continuation of Danilo's v2 [4], which picked up my earlier series [1] and Onur's ScopedQueue work [2].

Patch 1/6 of v2, "rust: workqueue: replace deprecated system_wq with system_{percpu,dfl}_wq", is a fix that was sent separately [3] and landed upstream in v7.4-rc1.

The soundness of this series depends on the ordered workqueue cancellation fix series [5].

[1] https://lore.kernel.org/all/[email protected]
[2] https://lore.kernel.org/all/[email protected]
[3] https://lore.kernel.org/all/[email protected]
[4] https://lore.kernel.org/all/[email protected]
[5] https://lore.kernel.org/r/[email protected]

---
Changes in v4:
- Rebase on v7.4-rc3
- ScopedWork: bind ScopedWork<'wq, T> to &'wq Queue at construction and replace Queue::enqueue_scoped() with ScopedWork::enqueue() so a work
item cannot be migrated across workqueues
- Add patch to pass object_is_on_stack() in init_work_with_key() and
call destroy_work_on_stack() when Work is dropped
- ScopedWork: call destroy_work_on_stack() on drop for CONFIG_DEBUG_OBJECTS_WORK support
- ScopedWork: initialize data before work_struct in ScopedWork::new() so init_work_with_key() is not called if the data initializer fails
- ScopedWork: add INVARIANT comments in ScopedWork::new()
- ScopedWork: require Sync on ScopedWorkItem
- Add Queue::as_raw()
- Pick up Reviewed-by tag from Philipp
- Link to v3: https://lore.kernel.org/r/[email protected]

Changes in v3:
- Rebase on v7.3-rc7
- Split out the system_wq patch and send it separately as a fix [3]; this series no longer depends on it
- Drop ScopedQueue: with the ScopedWork changes below it is no longer needed to enqueue non-'static work items, and it has no other user
- ScopedWork: make ScopedWork::new() unsafe instead of Queue::enqueue_scoped(), which is now a safe method accepting &ScopedWork<T> on any queue
- ScopedWork: drop the WorkItem/WorkItemPointer/HasWork impls and the public ScopedWorkRef type; embed the work_struct directly with a dedicated callback, and keep the Opaque-wrapped payload private
- ScopedWork: drop T on drop (Opaque does not run destructors)
- ScopedWork: propagate the initializer's error type from new()
- Builder: new_power_efficient() uses WQ_PERCPU | WQ_POWER_EFFICIENT; remove the now pointless percpu() modifier
- Pick up Reviewed-by tags from Daniel
- Link to v2: https://lore.kernel.org/all/[email protected]

Changes in v2:
- Merge scoped_work.rs and scoped_queue.rs into scoped.rs
- Introduce ScopedWorkRef<T> and wrap it in Opaque inside ScopedWork to avoid &mut aliasing in drop
- Pass &ScopedWorkRef<Self> to ScopedWorkItem::run to enable self-re-enqueue
- Add Deref<Target = Queue> for ScopedQueue; remove duplicate enqueue_scoped()
- Add Send + Sync for OwnedQueue
- Fix stale workqueue::system() references in docs
- Drop Work::cancel_work_sync()
- Make sysfs() available on ordered workqueues
- Add #[inline] annotations throughout

---
Alice Ryhl (4):
rust: workqueue: support stack-allocated work items
rust: workqueue: restrict delayed work to global wqs
rust: workqueue: create workqueue subdirectory
rust: workqueue: add creation of workqueues

Danilo Krummrich (1):
rust: workqueue: add ScopedWork for non-'static work items

 MAINTAINERS                                    |   1 +
 rust/helpers/workqueue.c                       |  16 +-
 rust/kernel/workqueue/builder.rs               | 370 ++++++++++++++++++++++
 rust/kernel/{workqueue.rs => workqueue/mod.rs} |  86 +++++-
 rust/kernel/workqueue/scoped.rs                | 410 +++++++++++++++++++++++++
 5 files changed, 874 insertions(+), 9 deletions(-)

---
base-commit: aebb7274476403040d582c61478df8f4a7bf9cd3
change-id: 20261012-rust-workqueue-658cd0fbe770

Best regards,

Signature · 1 line
Alice Ryhl <[email protected]>
author reviewer
#1 lore raw

PATCH v4 1/5 rust: workqueue: support stack-allocated work items

Pass object_is_on_stack(work) to __init_work() in rust_helper_init_work_with_key() instead of taking an explicit onstack parameter, and call destroy_work_on_stack() when Work is dropped. This ensures CONFIG_DEBUG_OBJECTS_WORK properly tracks and cleans up stack-allocated work items without requiring separate constructors.

Fixes 7324b88975c5 ("rust: workqueue: add helper for defining work_struct fields")
---
 rust/helpers/workqueue.c |  9 +++++++--
 rust/kernel/workqueue.rs | 13 ++++++++++---
 2 files changed, 17 insertions(+), 5 deletions(-)
rust/helpers/workqueue.c +7 -2
index ce1c3a5b2150..337382acfbf5 100644
@@ -1,16 +1,21 @@
 // SPDX-License-Identifier: GPL-2.0
 
+#include <linux/sched/task_stack.h>
 #include <linux/workqueue.h>
 
 __rust_helper void rust_helper_init_work_with_key(struct work_struct *work,
 						  work_func_t func,
-						  bool onstack,
 						  const char *name,
 						  struct lock_class_key *key)
 {
-	__init_work(work, onstack);
+	__init_work(work, object_is_on_stack(work));
 	work->data = (atomic_long_t)WORK_DATA_INIT();
 	lockdep_init_map(&work->lockdep_map, name, key, 0);
 	INIT_LIST_HEAD(&work->entry);
 	work->func = func;
 }
+
+__rust_helper void rust_helper_destroy_work_on_stack(struct work_struct *work)
+{
+	destroy_work_on_stack(work);
+}
rust/kernel/workqueue.rs +10 -3
index a872f9fffb90..6eb6c40138f2 100644
@@ -481,7 +481,7 @@ pub trait WorkItem<const ID: u64 = 0> {
 /// This is a helper type used to associate a `work_struct` with the [`WorkItem`] that uses it.
 ///
 /// [`run`]: WorkItemPointer::run
-#[pin_data]
+#[pin_data(PinnedDrop)]
 #[repr(transparent)]
 pub struct Work<T: ?Sized, const ID: u64 = 0> {
     #[pin]
@@ -513,7 +513,6 @@ pub fn new(name: &'static CStr, key: Pin<&'static LockClassKey>) -> impl PinInit
                     bindings::init_work_with_key(
                         slot,
                         Some(T::Pointer::run),
-                        false,
                         name.as_char_ptr(),
                         key.as_ptr(),
                     )
@@ -539,6 +538,15 @@ pub unsafe fn raw_get(ptr: *const Self) -> *mut bindings::work_struct {
     }
 }
 
+#[pinned_drop]
+impl<T: ?Sized, const ID: u64> PinnedDrop for Work<T, ID> {
+    fn drop(self: Pin<&mut Self>) {
+        // SAFETY: `self.work.get()` points to a valid `work_struct` initialized in `Work::new`,
+        // and the work item is not currently queued because `Self` is being dropped.
+        unsafe { bindings::destroy_work_on_stack(self.work.get()) }
+    }
+}
+
 /// Declares that a type contains a [`Work<T, ID>`].
 ///
 /// The intended way of using this trait is via the [`impl_has_work!`] macro. You can use the macro
@@ -686,7 +694,6 @@ pub fn new(
                     bindings::init_work_with_key(
                         core::ptr::addr_of_mut!((*slot).work),
                         Some(T::Pointer::run),
-                        false,
                         work_name.as_char_ptr(),
                         work_key.as_ptr(),
                     )
Signature · 1 line
2.56.1.708.g77570a6a1d-goog
author reviewer
#4 lore raw

PATCH v4 2/5 rust: workqueue: restrict delayed work to global wqs

In reply to Alice Ryhl

When a workqueue is shut down, delayed work that is pending but not scheduled does not get properly cleaned up, so it's not safe to use enqueue_delayed on a workqueue that might be destroyed. To fix this, restricted enqueue_delayed to static queues.

This may be fixed in the future by an approach along the lines of [1].

---
 rust/kernel/workqueue.rs | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
rust/kernel/workqueue.rs +8 -1
index 6eb6c40138f2..5ef807ed6a33 100644
@@ -302,8 +302,15 @@ pub fn enqueue<W, const ID: u64>(&self, w: W) -> W::EnqueueOutput
     ///
     /// This may fail if the work item is already enqueued in a workqueue.
     ///
+    /// This is only valid for global workqueues (with static lifetimes) because those are the only
+    /// ones that outlive all possible delayed work items.
+    ///
     /// The work item will be submitted using `WORK_CPU_UNBOUND`.
-    pub fn enqueue_delayed<W, const ID: u64>(&self, w: W, delay: Jiffies) -> W::EnqueueOutput
+    pub fn enqueue_delayed<W, const ID: u64>(
+        &'static self,
+        w: W,
+        delay: Jiffies,
+    ) -> W::EnqueueOutput
     where
         W: RawDelayedWorkItem<ID> + Send + 'static,
     {
Signature · 1 line
2.56.1.708.g77570a6a1d-goog
author reviewer
#5 lore raw

PATCH v4 3/5 rust: workqueue: create workqueue subdirectory

In reply to Alice Ryhl

The following patch will implement a workqueue builder in a separate file. To prepare for that, create a rust/kernel/workqueue subdirectory and move the existing file.

---
 MAINTAINERS                                    | 1 +
 rust/kernel/{workqueue.rs => workqueue/mod.rs} | 0
 2 files changed, 1 insertion(+)
MAINTAINERS +1 -0
index ccbcdc0799b9..9d5e2278f1f6 100644
@@ -30299,6 +30299,7 @@ F:	Documentation/core-api/workqueue.rst
 F:	include/linux/workqueue.h
 F:	kernel/workqueue.c
 F:	kernel/workqueue_internal.h
+F:	rust/kernel/workqueue/
 
 WWAN DRIVERS
 M:	Loic Poulain <[email protected]>
rust/kernel/workqueue/mod.rs renamed from rust/kernel/workqueue.rs +0 -0
similarity index 100%
rename from rust/kernel/workqueue.rs
rename to rust/kernel/workqueue/mod.rs
Signature · 1 line
2.56.1.708.g77570a6a1d-goog
author reviewer
#7 lore raw

PATCH v4 4/5 rust: workqueue: add creation of workqueues

In reply to Alice Ryhl

Creating workqueues is needed by various GPU drivers. Not only does it give you better control over execution, it also allows devices to ensure that all tasks have exited before the device is unbound (or similar) by running the workqueue destructor.

---
 rust/helpers/workqueue.c         |   7 +
 rust/kernel/workqueue/builder.rs | 370 +++++++++++++++++++++++++++++++++++++++
 rust/kernel/workqueue/mod.rs     |  51 +++++-
 3 files changed, 425 insertions(+), 3 deletions(-)
rust/helpers/workqueue.c +7 -0
index 337382acfbf5..7699dcdd5e05 100644
@@ -19,3 +19,10 @@ __rust_helper void rust_helper_destroy_work_on_stack(struct work_struct *work)
 {
 	destroy_work_on_stack(work);
 }
+
+__rust_helper
+struct workqueue_struct *rust_helper_alloc_workqueue(const char *fmt, unsigned int flags,
+						     int max_active, const void *data)
+{
+	return alloc_workqueue(fmt, flags, max_active, data);
+}
rust/kernel/workqueue/builder.rs new file +370 -0

370 lines not on this page · raw message

rust/kernel/workqueue/mod.rs +48 -3

75 lines not on this page · raw message

Signature · 1 line
2.56.1.708.g77570a6a1d-goog
author reviewer
#8 lore raw

PATCH v4 5/5 rust: workqueue: add ScopedWork for non-'static work items

In reply to Alice Ryhl

From: Danilo Krummrich <[email protected]>

Add ScopedWork<'wq, T>, a work item wrapper bound to a specific &'wq Queue whose destructor calls cancel_work_sync(), allowing T to carry non-'static lifetimes. Ownership of the data is not transferred to the workqueue; instead, the synchronous cancellation on drop guarantees the work function is not running when the data is freed.

Each ScopedWork<'wq, T> is bound to a single Queue at construction and enqueued via ScopedWork::enqueue() to uphold the non-reentrance conditions documented in Documentation/core-api/workqueue.rst: queuing an in-flight work item onto a different workqueue violates those conditions, in which case cancel_work_sync() only waits for execution on the last queued workqueue and may return while an earlier execution is still running.

Because the cancellation guarantee only holds if the destructor runs before the bound Queue or any lifetime in T expires, ScopedWork::new() (and thus new_scoped_work!) is unsafe and requires that the resulting value is never forgotten and is dropped before 'wq or T's lifetimes end. In return, ScopedWork::enqueue() is a safe method.

ScopedWork does not use the Work/WorkItem machinery: it embeds the work_struct directly and registers its own C callback, which recovers the &ScopedWork<'_, T> via container_of!() and passes it to ScopedWorkItem::run(). The work_struct, queue pointer, and user data live in a private Inner<T> that is wrapped in Opaque as a whole, so that the &mut Self held by the destructor while cancel_work_sync() waits does not claim uniqueness over memory (including padding) that a worker thread may still be reading through &ScopedWork<'_, T>.

---
 rust/kernel/workqueue/mod.rs    |  13 ++
 rust/kernel/workqueue/scoped.rs | 410 ++++++++++++++++++++++++++++++++++++++++
 2 files changed, 423 insertions(+)
rust/kernel/workqueue/mod.rs +13 -0
index 4f334dbf256a..d80fced7f333 100644
@@ -212,6 +212,13 @@
 mod builder;
 pub use self::builder::Builder;
 
+mod scoped;
+pub use self::scoped::{
+    new_scoped_work,
+    ScopedWork,
+    ScopedWorkItem, //
+};
+
 /// Creates a [`Work`] initialiser with the given name and a newly-created lock class.
 #[macro_export]
 macro_rules! new_work {
@@ -275,6 +282,12 @@ pub unsafe fn from_raw<'a>(ptr: *const bindings::workqueue_struct) -> &'a Queue
         unsafe { &*ptr.cast::<Queue>() }
     }
 
+    /// Obtain the raw `struct workqueue_struct` pointer.
+    #[inline]
+    pub fn as_raw(&self) -> *mut bindings::workqueue_struct {
+        self.0.get()
+    }
+
     /// Enqueues a work item.
     ///
     /// This may fail if the work item is already enqueued in a workqueue.
rust/kernel/workqueue/scoped.rs new file +410 -0

410 lines not on this page · raw message

Signature · 1 line
2.56.1.708.g77570a6a1d-goog
reviewer
#2 lore raw

Re: [PATCH v4 1/5] rust: workqueue: support stack-allocated work items

In reply to Alice Ryhl
Quoted 22 lines · Alice Ryhl

Pass object_is_on_stack(work) to __init_work() in rust_helper_init_work_with_key() instead of taking an explicit onstack parameter, and call destroy_work_on_stack() when Work is dropped. This ensures CONFIG_DEBUG_OBJECTS_WORK properly tracks and cleans up stack-allocated work items without requiring separate constructors.

Fixes 7324b88975c5 ("rust: workqueue: add helper for defining work_struct fields")
---
 rust/helpers/workqueue.c |  9 +++++++--
 rust/kernel/workqueue.rs | 13 ++++++++++---
 2 files changed, 17 insertions(+), 5 deletions(-)
rust/helpers/workqueue.c +1 -0
index ce1c3a5b2150..337382acfbf5 100644
@@ -1,16 +1,21 @@
 // SPDX-License-Identifier: GPL-2.0
 
+#include <linux/sched/task_stack.h>

rust/helpers/workqueue.c line 11

Quoted 10 lines
rust/helpers/workqueue.c +1 -2
 #include <linux/workqueue.h>
 
 __rust_helper void rust_helper_init_work_with_key(struct work_struct *work,
 						  work_func_t func,
-						  bool onstack,
 						  const char *name,
 						  struct lock_class_key *key)
 {
-	__init_work(work, onstack);
+	__init_work(work, object_is_on_stack(work));

Is this fully optimized out if !CONFIG_DEBUG_OBJECTS_WORK?

Best,
Gary

Quoted 55 lines
rust/helpers/workqueue.c +5 -0
 	work->data = (atomic_long_t)WORK_DATA_INIT();
 	lockdep_init_map(&work->lockdep_map, name, key, 0);
 	INIT_LIST_HEAD(&work->entry);
 	work->func = func;
 }
+
+__rust_helper void rust_helper_destroy_work_on_stack(struct work_struct *work)
+{
+	destroy_work_on_stack(work);
+}
rust/kernel/workqueue.rs +10 -3
index a872f9fffb90..6eb6c40138f2 100644
@@ -481,7 +481,7 @@ pub trait WorkItem<const ID: u64 = 0> {
 /// This is a helper type used to associate a `work_struct` with the [`WorkItem`] that uses it.
 ///
 /// [`run`]: WorkItemPointer::run
-#[pin_data]
+#[pin_data(PinnedDrop)]
 #[repr(transparent)]
 pub struct Work<T: ?Sized, const ID: u64 = 0> {
     #[pin]
@@ -513,7 +513,6 @@ pub fn new(name: &'static CStr, key: Pin<&'static LockClassKey>) -> impl PinInit
                     bindings::init_work_with_key(
                         slot,
                         Some(T::Pointer::run),
-                        false,
                         name.as_char_ptr(),
                         key.as_ptr(),
                     )
@@ -539,6 +538,15 @@ pub unsafe fn raw_get(ptr: *const Self) -> *mut bindings::work_struct {
     }
 }
 
+#[pinned_drop]
+impl<T: ?Sized, const ID: u64> PinnedDrop for Work<T, ID> {
+    fn drop(self: Pin<&mut Self>) {
+        // SAFETY: `self.work.get()` points to a valid `work_struct` initialized in `Work::new`,
+        // and the work item is not currently queued because `Self` is being dropped.
+        unsafe { bindings::destroy_work_on_stack(self.work.get()) }
+    }
+}
+
 /// Declares that a type contains a [`Work<T, ID>`].
 ///
 /// The intended way of using this trait is via the [`impl_has_work!`] macro. You can use the macro
@@ -686,7 +694,6 @@ pub fn new(
                     bindings::init_work_with_key(
                         core::ptr::addr_of_mut!((*slot).work),
                         Some(T::Pointer::run),
-                        false,
                         work_name.as_char_ptr(),
                         work_key.as_ptr(),
                     )
reviewer Reviewed-by on v4 3/5
#6 lore raw

Re: [PATCH v4 3/5] rust: workqueue: create workqueue subdirectory

In reply to Alice Ryhl
Quoted 7 lines · Alice Ryhl

The following patch will implement a workqueue builder in a separate file. To prepare for that, create a rust/kernel/workqueue subdirectory and move the existing file.

Quoted 4 lines
---
 MAINTAINERS                                    | 1 +
 rust/kernel/{workqueue.rs => workqueue/mod.rs} | 0
 2 files changed, 1 insertion(+)
author reviewer
#3 lore raw

Re: [PATCH v4 1/5] rust: workqueue: support stack-allocated work items

In reply to Gary Guo
Quoted 11 lines · Gary Guo

rust/helpers/workqueue.c line 11

Quoted 8 lines · Alice Ryhl

__rust_helper void rust_helper_init_work_with_key(struct work_struct *work,

rust/helpers/workqueue.c +1 -2
 						  work_func_t func,
-						  bool onstack,
 						  const char *name,
 						  struct lock_class_key *key)
 {
-	__init_work(work, onstack);
+	__init_work(work, object_is_on_stack(work));

Is this fully optimized out if !CONFIG_DEBUG_OBJECTS_WORK?

Yes, on normal architectures (checked x86_64 and arm64 with Clang and GCC): get_current() omits volatile and current->stack is a plain load, so DCE removes all of object_is_on_stack(work) once the empty __init_work() stub is inlined.

The one exception is ARCH=um with CONFIG_SMP=y, where get_current() calls an external uml_curr_cpu() function without __attribute_const__ (and under CONFIG_RUST_INLINE_HELPERS, the dead mrs/movq %gs:... inline asm stays in helpers.bc until backend codegen drops it). I can wrap it in if (IS_ENABLED(CONFIG_DEBUG_OBJECTS_WORK)) if I send a v5.

Alice