Skip to contentSkip to conversation

Creation of workqueues in Rust

PATCH In review Alice Ryhl 5 participants 16 messages Mar 12 – Jun 9

Subsystems (2)
lore
Share & tools
lore

Alice Ryhl

author reviewer
#0 lore raw

v4 cover Creation of workqueues in Rust

GPU drivers often need to create their own workqueues for various reasons. Add the ability to do so.

---
Changes in v4:
- Add link to delayed work fix.
- Redo workqueue creation to prevent invalid configurations.
- Introduce a directory as workqueue.rs was getting really large.
- Link to v3: https://lore.kernel.org/r/[email protected]

Changes in v3:
- Switch to builder pattern.
- Drop BH workqueues for now.
- Mark delayed wq change as fix.
- Link to v2: https://lore.kernel.org/r/[email protected]

Changes in v2:
- Redo how flagging works.
- Restrict delayed work to not be usable on custom workqueues.
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Alice Ryhl (3):
rust: workqueue: restrict delayed work to global wqs
rust: workqueue: create workqueue subdirectory
rust: workqueue: add creation of workqueues

 MAINTAINERS                                    |   1 +
 rust/helpers/workqueue.c                       |   7 +
 rust/kernel/workqueue/builder.rs               | 380 +++++++++++++++++++++++++
 rust/kernel/{workqueue.rs => workqueue/mod.rs} |  53 +++-
 4 files changed, 437 insertions(+), 4 deletions(-)

---
base-commit: df9c51269a5e2a6fbca2884a756a4011a5e78748
change-id: 20250411-create-workqueue-d053158c7a4b

Best regards,

Signature · 1 line
Alice Ryhl <[email protected]>
author reviewer
#1 lore raw

PATCH v4 1/3 rust: workqueue: restrict delayed work to global wqs

When a workqueue is shut down, delayed work that is pending but not scheduled does not get properly cleaned up, so it's not safe to use enqueue_delayed on a workqueue that might be destroyed. To fix this, restricted enqueue_delayed to static queues.

This may be fixed in the future by an approach along the lines of [1].

---
 rust/kernel/workqueue.rs | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
rust/kernel/workqueue.rs +8 -1
index 706e833e9702..1acd113c04ee 100644
@@ -296,8 +296,15 @@ pub fn enqueue<W, const ID: u64>(&self, w: W) -> W::EnqueueOutput
     ///
     /// This may fail if the work item is already enqueued in a workqueue.
     ///
+    /// This is only valid for global workqueues (with static lifetimes) because those are the only
+    /// ones that outlive all possible delayed work items.
+    ///
     /// The work item will be submitted using `WORK_CPU_UNBOUND`.
-    pub fn enqueue_delayed<W, const ID: u64>(&self, w: W, delay: Jiffies) -> W::EnqueueOutput
+    pub fn enqueue_delayed<W, const ID: u64>(
+        &'static self,
+        w: W,
+        delay: Jiffies,
+    ) -> W::EnqueueOutput
     where
         W: RawDelayedWorkItem<ID> + Send + 'static,
     {
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
reviewer Reviewed-by on v4 1/3
#2 lore raw

Re: [PATCH v4 1/3] rust: workqueue: restrict delayed work to global wqs

Quoted 7 lines · Alice Ryhl

When a workqueue is shut down, delayed work that is pending but not scheduled does not get properly cleaned up, so it's not safe to use enqueue_delayed on a workqueue that might be destroyed. To fix this, restricted enqueue_delayed to static queues.

This may be fixed in the future by an approach along the lines of [1].

Quoted 7 lines

Best regards,
Andreas Hindborg

author reviewer
#3 lore raw

PATCH v4 2/3 rust: workqueue: create workqueue subdirectory

In reply to Alice Ryhl

The following patch will implement a workqueue builder in a separate file. To prepare for that, create a rust/kernel/workqueue subdirectory and move the existing file.

---
 MAINTAINERS                                    | 1 +
 rust/kernel/{workqueue.rs => workqueue/mod.rs} | 0
 2 files changed, 1 insertion(+)
MAINTAINERS +1 -0
index 8fb7b70dfb47..8a12db6bd127 100644
@@ -28511,6 +28511,7 @@ F:	Documentation/core-api/workqueue.rst
 F:	include/linux/workqueue.h
 F:	kernel/workqueue.c
 F:	kernel/workqueue_internal.h
+F:	rust/kernel/workqueue/
 
 WWAN DRIVERS
 M:	Loic Poulain <[email protected]>
rust/kernel/workqueue/mod.rs renamed from rust/kernel/workqueue.rs +0 -0
similarity index 100%
rename from rust/kernel/workqueue.rs
rename to rust/kernel/workqueue/mod.rs
Signature · 1 line
2.53.0.473.g4a7958ca14-goog
author reviewer
#4 lore raw

PATCH v4 3/3 rust: workqueue: add creation of workqueues

In reply to Alice Ryhl

Creating workqueues is needed by various GPU drivers. Not only does it give you better control over execution, it also allows devices to ensure that all tasks have exited before the device is unbound (or similar) by running the workqueue destructor.

---
 rust/helpers/workqueue.c         |   7 +
 rust/kernel/workqueue/builder.rs | 380 +++++++++++++++++++++++++++++++++++++++
 rust/kernel/workqueue/mod.rs     |  44 ++++-
 3 files changed, 428 insertions(+), 3 deletions(-)
rust/helpers/workqueue.c +7 -0
index ce1c3a5b2150..e4b9d1b3d6bf 100644
@@ -14,3 +14,10 @@ __rust_helper void rust_helper_init_work_with_key(struct work_struct *work,
 	INIT_LIST_HEAD(&work->entry);
 	work->func = func;
 }
+
+__rust_helper
+struct workqueue_struct *rust_helper_alloc_workqueue(const char *fmt, unsigned int flags,
+						     int max_active, const void *data)
+{
+	return alloc_workqueue(fmt, flags, max_active, data);
+}
rust/kernel/workqueue/builder.rs new file +380 -0

380 lines not on this page · raw message

rust/kernel/workqueue/mod.rs +41 -3

68 lines not on this page · raw message

Signature · 1 line
2.53.0.473.g4a7958ca14-goog
reviewer
#5 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

Quoted 39 lines · Alice Ryhl

Creating workqueues is needed by various GPU drivers. Not only does it give you better control over execution, it also allows devices to ensure that all tasks have exited before the device is unbound (or similar) by running the workqueue destructor.

---
 rust/helpers/workqueue.c         |   7 +
 rust/kernel/workqueue/builder.rs | 380 +++++++++++++++++++++++++++++++++++++++
 rust/kernel/workqueue/mod.rs     |  44 ++++-
 3 files changed, 428 insertions(+), 3 deletions(-)
rust/helpers/workqueue.c +7 -0
index ce1c3a5b2150..e4b9d1b3d6bf 100644
@@ -14,3 +14,10 @@ __rust_helper void rust_helper_init_work_with_key(struct work_struct *work,
 	INIT_LIST_HEAD(&work->entry);
 	work->func = func;
 }
+
+__rust_helper
+struct workqueue_struct *rust_helper_alloc_workqueue(const char *fmt, unsigned int flags,
+						     int max_active, const void *data)
+{
+	return alloc_workqueue(fmt, flags, max_active, data);
+}
rust/kernel/workqueue/builder.rs new file +6 -0
new file mode 100644
index 000000000000..d4d77b96f9c4
@@ -0,0 +1,380 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Workqueue builders.
+
+use kernel::{
+    alloc::AllocError,

rust/kernel/workqueue/builder.rs line 16

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    prelude::*,
+    workqueue::{
+        OwnedQueue, //
+        Queue,
+    }, //
+};
+
+use core::{
+    marker::PhantomData, //
+    ptr::{self, NonNull},

This is formatted incorrectly.

rust/kernel/workqueue/builder.rs line 26

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+};
+
+/// Workqueue builder.
+///
+/// A valid combination of workqueue flags contains one of the base flags (`WQ_UNBOUND`, `WQ_BH`,
+/// or `WQ_PERCPU`) and a combination of modifier flags that are compatible with the selected base
+/// flag.
+///
+/// For details, please refer to `Documentation/core-api/workqueue.rst`.
+pub struct Builder<T> {

I would name the generic parameter Kind rather than T. T is too generic to indicate it's for specific kind/type of workqueue.

rust/kernel/workqueue/builder.rs line 29

Quoted 3 lines
rust/kernel/workqueue/builder.rs new file +3 -0
+    flags: bindings::wq_flags,
+    max_active: i32,
+    _type: PhantomData<T>,

Hmm, it is somewhat awkward to me that we are having a PhantomData<T> here, as PhantomData is documented to "behave as it it owns a T", but all the possible Ts that we use here are uninhabited.

Quoted 81 lines
rust/kernel/workqueue/builder.rs new file +81 -0
+}
+
+pub enum TypeUnbound {}
+pub enum TypePercpu {}
+pub enum TypePowerEfficient {}
+pub enum TypeBH {}
+pub enum TypeOrdered {}
+
+/// Entry-points to the builder API.
+impl Queue {
+    /// Build a workqueue whose work may execute on any cpu.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from unbound wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_UNBOUND")]
+    pub fn new_unbound() -> Builder<TypeUnbound> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+
+    /// Build a workqueue whose work is bound to a specific cpu.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_percpu().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from percpu wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_PERCPU")]
+    pub fn new_percpu() -> Builder<TypePercpu> {
+        Builder {
+            flags: bindings::wq_flags_WQ_PERCPU,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+
+    /// Build a power-efficient workqueue.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_power_efficient().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from power-efficient wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_POWER_EFFICIENT")]
+    pub fn new_power_efficient() -> Builder<TypePowerEfficient> {
+        Builder {
+            flags: bindings::wq_flags_WQ_POWER_EFFICIENT,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+
+    /// Build a single-threaded workqueue that executes jobs in order.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_ordered().build(c"my-wq")?;

rust/kernel/workqueue/builder.rs line 120

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from ordered wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "alloc_ordered_workqueue")]
+    #[doc(alias = "__WQ_ORDERED")]
+    pub fn new_ordered() -> Builder<TypeOrdered> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND | bindings::wq_flags___WQ_ORDERED,
+            max_active: 0,

This should be 1 instead of 0.

Quoted 74 lines
rust/kernel/workqueue/builder.rs new file +26 -0
+            _type: PhantomData,
+        }
+    }
+
+    /// Build a workqueue that executes in bottom-half (softirq) context.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_bh().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from BH wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_BH")]
+    pub fn new_bh() -> Builder<TypeBH> {
+        Builder {
+            flags: bindings::wq_flags_WQ_BH,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+}
+
 
[snip]
rust/kernel/workqueue/mod.rs +13 -3
index 1acd113c04ee..6049c0e8e4b6 100644
@@ -186,7 +186,10 @@
 //! C header: [`include/linux/workqueue.h`](srctree/include/linux/workqueue.h)
 
 use crate::{
-    alloc::{AllocError, Flags},
+    alloc::{
+        self,
+        AllocError, //
+    },
     container_of,
     prelude::*,
     sync::Arc,
@@ -194,7 +197,14 @@
     time::Jiffies,
     types::Opaque,
 };
-use core::marker::PhantomData;
+use core::{
+    marker::PhantomData,
+    ops::Deref,
+    ptr::NonNull, //
+};
+
+mod builder;
+pub use self::builder::Builder;
 
 /// Creates a [`Work`] initialiser with the given name and a newly-created lock class.
 #[macro_export]
@@ -340,7 +350,7 @@ pub fn enqueue_delayed<W, const ID: u64>(
     /// This method can fail because it allocates memory to store the work item.
     pub fn try_spawn<T: 'static + Send + FnOnce()>(
         &self,
-        flags: Flags,
+        flags: alloc::Flags,
         func: T,
     ) -> Result<(), AllocError> {
         let init = pin_init!(ClosureWork {
@@ -353,6 +363,34 @@ pub fn try_spawn<T: 'static + Send + FnOnce()>(
     }
 }
 

rust/kernel/workqueue/mod.rs pub fn try_spawn<T: 'static + Send + FnOnce()>( · line 375

Quoted 10 lines
rust/kernel/workqueue/mod.rs +10 -0
+/// An owned kernel work queue.
+///
+/// Dropping a workqueue blocks on all pending work.
+///
+/// # Invariants
+///
+/// `queue` points at a valid workqueue that is owned by this `OwnedQueue`.
+pub struct OwnedQueue {
+    queue: NonNull<Queue>,
+}

This looks like something that can become Owned<Queue> when Andreas' series land?

rust/kernel/workqueue/mod.rs pub fn try_spawn<T: 'static + Send + FnOnce()>( · line 378

Quoted 3 lines
rust/kernel/workqueue/mod.rs +3 -0
+
+impl Deref for OwnedQueue {
+    type Target = Queue;

#[inline]

Best,
Gary

Quoted 18 lines
rust/kernel/workqueue/mod.rs +15 -0
+    fn deref(&self) -> &Queue {
+        // SAFETY: By the type invariants, this pointer references a valid queue.
+        unsafe { &*self.queue.as_ptr() }
+    }
+}
+
+impl Drop for OwnedQueue {
+    fn drop(&mut self) {
+        // SAFETY: This `OwnedQueue` owns a valid workqueue, so we can destroy it. There is no
+        // delayed work scheduled on this queue that may attempt to use it after this call, as
+        // scheduling delayed work requires a 'static reference.
+        unsafe { bindings::destroy_workqueue(self.queue.as_ptr().cast()) }
+    }
+}
+
 /// A helper type used in [`try_spawn`].
 ///
 /// [`try_spawn`]: Queue::try_spawn
author reviewer
#6 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

Quoted 6 lines · Gary Guo
Quoted 5 lines · Alice Ryhl

Creating workqueues is needed by various GPU drivers. Not only does it give you better control over execution, it also allows devices to ensure that all tasks have exited before the device is unbound (or similar) by running the workqueue destructor.

Quoted 7 lines

rust/kernel/workqueue/builder.rs line 16

Quoted 5 lines
rust/kernel/workqueue/builder.rs new file +3 -0
+use core::{
+    marker::PhantomData, //
+    ptr::{self, NonNull},

This is formatted incorrectly.

Sigh. Ok.

Quoted 5 lines
Quoted 5 lines
rust/kernel/workqueue/builder.rs new file +5 -0
+};
+
+/// Workqueue builder.
+///
+/// A valid combination of workqueue flags contains one of the base flags (`WQ_UNBOUND`, `WQ_BH`,
Quoted 8 lines

rust/kernel/workqueue/builder.rs line 26

Quoted 5 lines
rust/kernel/workqueue/builder.rs new file +5 -0
+/// or `WQ_PERCPU`) and a combination of modifier flags that are compatible with the selected base
+/// flag.
+///
+/// For details, please refer to `Documentation/core-api/workqueue.rst`.
+pub struct Builder<T> {

I would name the generic parameter Kind rather than T. T is too generic to indicate it's for specific kind/type of workqueue.

I guess 'Kind' works here, yeah.

Quoted 7 lines

rust/kernel/workqueue/builder.rs line 29

Quoted 3 lines
rust/kernel/workqueue/builder.rs new file +3 -0
+    flags: bindings::wq_flags,
+    max_active: i32,
+    _type: PhantomData<T>,

Hmm, it is somewhat awkward to me that we are having a PhantomData<T> here, as PhantomData is documented to "behave as it it owns a T", but all the possible Ts that we use here are uninhabited.

I don't think this is an issue.

Quoted 19 lines
Quoted 8 lines
rust/kernel/workqueue/builder.rs new file +8 -0
+    /// Build a single-threaded workqueue that executes jobs in order.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_ordered().build(c"my-wq")?;

rust/kernel/workqueue/builder.rs line 120

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from ordered wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "alloc_ordered_workqueue")]
+    #[doc(alias = "__WQ_ORDERED")]
+    pub fn new_ordered() -> Builder<TypeOrdered> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND | bindings::wq_flags___WQ_ORDERED,
+            max_active: 0,
Quoted 1 line

This should be 1 instead of 0.

Hmm, it's weird that the doc-test didn't catch this.

Quoted 5 lines
Quoted 5 lines
rust/kernel/workqueue/mod.rs +5 -0
+/// An owned kernel work queue.
+///
+/// Dropping a workqueue blocks on all pending work.
+///
+/// # Invariants
Quoted 8 lines

rust/kernel/workqueue/mod.rs pub fn try_spawn<T: 'static + Send + FnOnce()>( · line 375

Quoted 5 lines
rust/kernel/workqueue/mod.rs +5 -0
+///
+/// `queue` points at a valid workqueue that is owned by this `OwnedQueue`.
+pub struct OwnedQueue {
+    queue: NonNull<Queue>,
+}

This looks like something that can become Owned<Queue> when Andreas' series land?

That's right.

Alice

reviewer
#7 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

Quoted 10 lines · Alice Ryhl
Quoted 7 lines · Gary Guo

rust/kernel/workqueue/builder.rs line 29

Quoted 3 lines
rust/kernel/workqueue/builder.rs new file +3 -0
+    flags: bindings::wq_flags,
+    max_active: i32,
+    _type: PhantomData<T>,

Hmm, it is somewhat awkward to me that we are having a PhantomData<T> here, as PhantomData is documented to "behave as it it owns a T", but all the possible Ts that we use here are uninhabited.

I don't think this is an issue.

Fair. I also don't think it's an issue, just awkward.

I would rather make the uninhabited enums be just unit structs, though (like what we do for atomic ordering and device contexts).

Quoted 20 lines
Quoted 20 lines
Quoted 19 lines
rust/kernel/workqueue/builder.rs new file +18 -0
+    /// Build a single-threaded workqueue that executes jobs in order.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_ordered().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from ordered wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "alloc_ordered_workqueue")]
+    #[doc(alias = "__WQ_ORDERED")]
+    pub fn new_ordered() -> Builder<TypeOrdered> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND | bindings::wq_flags___WQ_ORDERED,
+            max_active: 0,
Quoted 3 lines
Quoted 1 line

This should be 1 instead of 0.

Hmm, it's weird that the doc-test didn't catch this.

It seems that there's no check for this in the __alloc_workqueue (which I guess makes sense, because there's a C macro for allocating ordered workqueue which doesn't take max_active and pass 1 along).

The only assertion is in workqueue_set_max_active to prevent changing it after creation.

Best,
Gary

Quoted 18 lines
Quoted 14 lines

rust/kernel/workqueue/mod.rs pub fn try_spawn<T: 'static + Send + FnOnce()>( · line 375

Quoted 11 lines
rust/kernel/workqueue/mod.rs +10 -0
+/// An owned kernel work queue.
+///
+/// Dropping a workqueue blocks on all pending work.
+///
+/// # Invariants
+///
+/// `queue` points at a valid workqueue that is owned by this `OwnedQueue`.
+pub struct OwnedQueue {
+    queue: NonNull<Queue>,
+}

This looks like something that can become Owned<Queue> when Andreas' series land?

That's right.

Alice

reviewer
#8 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

Quoted 10 lines · Gary Guo
Quoted 9 lines · Alice Ryhl
Quoted 7 lines · Gary Guo

rust/kernel/workqueue/builder.rs line 29

Quoted 3 lines
rust/kernel/workqueue/builder.rs new file +3 -0
+    flags: bindings::wq_flags,
+    max_active: i32,
+    _type: PhantomData<T>,

Hmm, it is somewhat awkward to me that we are having a PhantomData<T> here, as PhantomData is documented to "behave as it it owns a T", but all the possible Ts that we use here are uninhabited.

Quoted 6 lines
Quoted 1 line

I don't think this is an issue.

Fair. I also don't think it's an issue, just awkward.

I would rather make the uninhabited enums be just unit structs, though (like what we do for atomic ordering and device contexts).

One interesting idea with unit structs, is to allow the type to be passed in as argument rather than having multiple new_ variants:

    Queue::builder(wq::TypeUnbound).max_active(..).build(...)

This is the trick that we use to make atomic API looks nice, although in this case it seems that it's not worth the hassle as it needs an extra import...

Best,
Gary

Quoted 53 lines
Quoted 21 lines
Quoted 21 lines
Quoted 20 lines
rust/kernel/workqueue/builder.rs new file +18 -0
+    /// Build a single-threaded workqueue that executes jobs in order.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_ordered().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from ordered wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "alloc_ordered_workqueue")]
+    #[doc(alias = "__WQ_ORDERED")]
+    pub fn new_ordered() -> Builder<TypeOrdered> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND | bindings::wq_flags___WQ_ORDERED,
+            max_active: 0,
Quoted 3 lines
Quoted 1 line

This should be 1 instead of 0.

Hmm, it's weird that the doc-test didn't catch this.

It seems that there's no check for this in the __alloc_workqueue (which I guess makes sense, because there's a C macro for allocating ordered workqueue which doesn't take max_active and pass 1 along).

The only assertion is in workqueue_set_max_active to prevent changing it after creation.

Best,
Gary

Quoted 18 lines
Quoted 14 lines

rust/kernel/workqueue/mod.rs pub fn try_spawn<T: 'static + Send + FnOnce()>( · line 375

Quoted 11 lines
rust/kernel/workqueue/mod.rs +10 -0
+/// An owned kernel work queue.
+///
+/// Dropping a workqueue blocks on all pending work.
+///
+/// # Invariants
+///
+/// `queue` points at a valid workqueue that is owned by this `OwnedQueue`.
+pub struct OwnedQueue {
+    queue: NonNull<Queue>,
+}

This looks like something that can become Owned<Queue> when Andreas' series land?

That's right.

Alice

author reviewer
#9 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

Quoted 18 lines · Gary Guo
Quoted 10 lines · Gary Guo
Quoted 9 lines · Alice Ryhl
Quoted 7 lines · Gary Guo

rust/kernel/workqueue/builder.rs line 29

Quoted 3 lines
rust/kernel/workqueue/builder.rs new file +3 -0
+    flags: bindings::wq_flags,
+    max_active: i32,
+    _type: PhantomData<T>,

Hmm, it is somewhat awkward to me that we are having a PhantomData<T> here, as PhantomData is documented to "behave as it it owns a T", but all the possible Ts that we use here are uninhabited.

Quoted 6 lines
Quoted 1 line

I don't think this is an issue.

Fair. I also don't think it's an issue, just awkward.

I would rather make the uninhabited enums be just unit structs, though (like what we do for atomic ordering and device contexts).

Quoted 7 lines

One interesting idea with unit structs, is to allow the type to be passed in as argument rather than having multiple new_ variants:

    Queue::builder(wq::TypeUnbound).max_active(..).build(...)

This is the trick that we use to make atomic API looks nice, although in this case it seems that it's not worth the hassle as it needs an extra import...

Yeah, I don't think this is worth it at all.

Alice

#10 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

In reply to Alice Ryhl
Quoted 5 lines · Alice Ryhl
rust/kernel/workqueue/builder.rs new file +5 -0
+impl Queue {
+    /// Build a workqueue whose work may execute on any cpu.
+    ///
+    /// # Examples
+    ///

rust/kernel/workqueue/builder.rs line 53

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from unbound wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_UNBOUND")]
+    pub fn new_unbound() -> Builder<TypeUnbound> {

Wouldn't it be more straight forward if those where constructors of Builder?

	let wq = wq::Builder::new_unbound().build(c"my-wq")?;

Queue::new_unbound() suggests that it returns a new Queue object, but instead it returns a new Builder object.

author reviewer
#11 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

Quoted 17 lines · Danilo Krummrich
Quoted 5 lines · Alice Ryhl
rust/kernel/workqueue/builder.rs new file +5 -0
+impl Queue {
+    /// Build a workqueue whose work may execute on any cpu.
+    ///
+    /// # Examples
+    ///

rust/kernel/workqueue/builder.rs line 53

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from unbound wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_UNBOUND")]
+    pub fn new_unbound() -> Builder<TypeUnbound> {
Quoted 3 lines

Wouldn't it be more straight forward if those where constructors of Builder?

	let wq = wq::Builder::new_unbound().build(c"my-wq")?;

Crates out there do both, but I actually like this pattern quite a lot. After all, given 'wq::Builder::new_unbound()' and 'Queue::new_unbound()' I think the latter is the one that makes it more clear that you are building a 'Queue'.

Also, it avoids an extra import.

Quoted 2 lines

Queue::new_unbound() suggests that it returns a new Queue object, but instead it returns a new Builder object.

I'm only really concerned about confusion if it falls into one of these categories:

* You write the wrong code, it compiles, and silently does the wrong thing.
* You read the code, and misunderstand what it does.

I don't think this falls into either of those cases. If you forget the .build() call, all you get is a compiler error, and it's not hard to figure out what to do when you see the error.

Alice

#12 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

Quoted 20 lines · Alice Ryhl
Quoted 19 lines · Danilo Krummrich
Quoted 5 lines · Alice Ryhl
rust/kernel/workqueue/builder.rs new file +5 -0
+impl Queue {
+    /// Build a workqueue whose work may execute on any cpu.
+    ///
+    /// # Examples
+    ///

rust/kernel/workqueue/builder.rs line 53

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from unbound wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_UNBOUND")]
+    pub fn new_unbound() -> Builder<TypeUnbound> {

Wouldn't it be more straight forward if those where constructors of Builder?

Quoted 8 lines
Quoted 1 line
	let wq = wq::Builder::new_unbound().build(c"my-wq")?;

Crates out there do both, but I actually like this pattern quite a lot. After all, given 'wq::Builder::new_unbound()' and 'Queue::new_unbound()' I think the latter is the one that makes it more clear that you are building a 'Queue'.

Also, it avoids an extra import.

If it is about avoiding an extra import, what about

Queue::builder_unbound()

this makes the name less misleading and avoids the import.

Quoted 15 lines
Quoted 2 lines

Queue::new_unbound() suggests that it returns a new Queue object, but instead it returns a new Builder object.

I'm only really concerned about confusion if it falls into one of these categories:

* You write the wrong code, it compiles, and silently does the wrong thing.
* You read the code, and misunderstand what it does.

I don't think this falls into either of those cases. If you forget the .build() call, all you get is a compiler error, and it's not hard to figure out what to do when you see the error.

Alice

reviewer Reviewed-by on v4 3/3
#13 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

In reply to Alice Ryhl
Quoted 6 lines · Alice Ryhl

Creating workqueues is needed by various GPU drivers. Not only does it give you better control over execution, it also allows devices to ensure that all tasks have exited before the device is unbound (or similar) by running the workqueue destructor.

Best regards,
Andreas Hindborg

#14 lore raw

Re: [PATCH v4 3/3] rust: workqueue: add creation of workqueues

In reply to Alice Ryhl
Quoted 58 lines · Alice Ryhl

Creating workqueues is needed by various GPU drivers. Not only does it give you better control over execution, it also allows devices to ensure that all tasks have exited before the device is unbound (or similar) by running the workqueue destructor.

---
 rust/helpers/workqueue.c         |   7 +
 rust/kernel/workqueue/builder.rs | 380 +++++++++++++++++++++++++++++++++++++++
 rust/kernel/workqueue/mod.rs     |  44 ++++-
 3 files changed, 428 insertions(+), 3 deletions(-)
rust/helpers/workqueue.c +7 -0
index ce1c3a5b2150..e4b9d1b3d6bf 100644
@@ -14,3 +14,10 @@ __rust_helper void rust_helper_init_work_with_key(struct work_struct *work,
 	INIT_LIST_HEAD(&work->entry);
 	work->func = func;
 }
+
+__rust_helper
+struct workqueue_struct *rust_helper_alloc_workqueue(const char *fmt, unsigned int flags,
+						     int max_active, const void *data)
+{
+	return alloc_workqueue(fmt, flags, max_active, data);
+}
rust/kernel/workqueue/builder.rs new file +25 -0
new file mode 100644
index 000000000000..d4d77b96f9c4
@@ -0,0 +1,380 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Workqueue builders.
+
+use kernel::{
+    alloc::AllocError,
+    prelude::*,
+    workqueue::{
+        OwnedQueue, //
+        Queue,
+    }, //
+};
+
+use core::{
+    marker::PhantomData, //
+    ptr::{self, NonNull},
+};
+
+/// Workqueue builder.
+///
+/// A valid combination of workqueue flags contains one of the base flags (`WQ_UNBOUND`, `WQ_BH`,
+/// or `WQ_PERCPU`) and a combination of modifier flags that are compatible with the selected base
+/// flag.
+///
+/// For details, please refer to `Documentation/core-api/workqueue.rst`.

rust/kernel/workqueue/builder.rs line 35

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+pub struct Builder<T> {
+    flags: bindings::wq_flags,
+    max_active: i32,
+    _type: PhantomData<T>,
+}
+
+pub enum TypeUnbound {}
+pub enum TypePercpu {}
+pub enum TypePowerEfficient {}
+pub enum TypeBH {}

Could you expand the name here?

Quoted 16 lines
rust/kernel/workqueue/builder.rs new file +16 -0
+pub enum TypeOrdered {}
+
+/// Entry-points to the builder API.
+impl Queue {
+    /// Build a workqueue whose work may execute on any cpu.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from unbound wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]

rust/kernel/workqueue/builder.rs line 61

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    #[doc(alias = "WQ_UNBOUND")]
+    pub fn new_unbound() -> Builder<TypeUnbound> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+
+    /// Build a workqueue whose work is bound to a specific cpu.

Could you elaborate what this means? When are work items moved between CPUs normally? If it is a specific CPU, how is the CPU specified?

Quoted 54 lines
rust/kernel/workqueue/builder.rs new file +54 -0
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_percpu().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from percpu wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_PERCPU")]
+    pub fn new_percpu() -> Builder<TypePercpu> {
+        Builder {
+            flags: bindings::wq_flags_WQ_PERCPU,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+
+    /// Build a power-efficient workqueue.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_power_efficient().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from power-efficient wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_POWER_EFFICIENT")]
+    pub fn new_power_efficient() -> Builder<TypePowerEfficient> {
+        Builder {
+            flags: bindings::wq_flags_WQ_POWER_EFFICIENT,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+
+    /// Build a single-threaded workqueue that executes jobs in order.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_ordered().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from ordered wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "alloc_ordered_workqueue")]

rust/kernel/workqueue/builder.rs line 125

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    #[doc(alias = "__WQ_ORDERED")]
+    pub fn new_ordered() -> Builder<TypeOrdered> {
+        Builder {
+            flags: bindings::wq_flags_WQ_UNBOUND | bindings::wq_flags___WQ_ORDERED,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+
+    /// Build a workqueue that executes in bottom-half (softirq) context.

Is bottom-half and softirq context the same?

Quoted 14 lines
rust/kernel/workqueue/builder.rs new file +14 -0
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_bh().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from BH wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_BH")]
+    pub fn new_bh() -> Builder<TypeBH> {
+        Builder {

rust/kernel/workqueue/builder.rs line 149

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+            flags: bindings::wq_flags_WQ_BH,
+            max_active: 0,
+            _type: PhantomData,
+        }
+    }
+}
+
+/// Options that may be used with all workqueue types.
+impl<T> Builder<T> {
+    /// Mark this workqueue high priority.

Could you elaborate the meaning of "high priority"?

Quoted 91 lines
rust/kernel/workqueue/builder.rs new file +91 -0
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().highpri().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from highpri wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_HIGHPRI")]
+    pub fn highpri(mut self) -> Self {
+        self.flags |= bindings::wq_flags_WQ_HIGHPRI;
+        self
+    }
+
+    /// Creates the workqueue.
+    ///
+    /// The provided name is used verbatim as the workqueue name.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// // create an unbound workqueue registered with sysfs
+    /// let wq = Queue::new_unbound().sysfs().build(c"my-wq")?;
+    ///
+    /// // spawn a work item on it
+    /// wq.try_spawn(
+    ///     GFP_KERNEL,
+    ///     || pr_warn!("Printing from my-wq"),
+    /// )?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "alloc_workqueue")]
+    pub fn build(self, name: &CStr) -> Result<OwnedQueue, AllocError> {
+        // SAFETY:
+        // * c"%s" is compatible with passing the name as a c-string.
+        // * the builder only permits valid flag combinations
+        let ptr = unsafe {
+            bindings::alloc_workqueue(
+                c"%s".as_char_ptr(),
+                self.flags,
+                self.max_active,
+                name.as_char_ptr().cast::<c_void>(),
+            )
+        };
+
+        // INVARIANT: We successfully created the workqueue, so we can return ownership to the
+        // caller.
+        Ok(OwnedQueue {
+            queue: NonNull::new(ptr).ok_or(AllocError)?.cast(),
+        })
+    }
+
+    /// Creates the workqueue.
+    ///
+    /// # Examples
+    ///
+    /// This example shows how to pass a Rust string formatter to the workqueue name, creating
+    /// workqueues with names such as `my-wq-1` and `my-wq-2`.
+    ///
+    /// ```
+    /// use kernel::workqueue::{Queue, OwnedQueue};
+    ///
+    /// fn my_wq(num: u32) -> Result<OwnedQueue> {
+    ///     // create a percpu workqueue called my-wq-{num}
+    ///     let wq = Queue::new_percpu().build_fmt(fmt!("my-wq-{num}"))?;
+    ///     Ok(wq)
+    /// }
+    /// ```
+    #[inline]
+    pub fn build_fmt(self, name: kernel::fmt::Arguments<'_>) -> Result<OwnedQueue, AllocError> {
+        // SAFETY:
+        // * c"%pA" is compatible with passing an `Arguments` pointer.
+        // * the builder only permits valid flag combinations
+        let ptr = unsafe {
+            bindings::alloc_workqueue(
+                c"%pA".as_char_ptr(),
+                self.flags,
+                self.max_active,
+                ptr::from_ref(&name).cast::<c_void>(),
+            )
+        };
+
+        // INVARIANT: We successfully created the workqueue, so we can return ownership to the
+        // caller.
+        Ok(OwnedQueue {

rust/kernel/workqueue/builder.rs line 250

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+            queue: NonNull::new(ptr).ok_or(AllocError)?.cast(),
+        })
+    }
+}
+
+/// Indicates that this workqueue is threaded.
+pub trait TypeThreaded {}
+impl TypeThreaded for TypeUnbound {}
+impl TypeThreaded for TypePercpu {}
+impl TypeThreaded for TypePowerEfficient {}

Could implementing one of these traits for the wrong type cause unsoundness? If so, should they be unsafe?

rust/kernel/workqueue/builder.rs line 254

Quoted 4 lines
rust/kernel/workqueue/builder.rs new file +4 -0
+
+/// Options that are not available on BH or ordered workqueues.
+impl<T: TypeThreaded> Builder<T> {
+    /// Set the maximum number of active cpus.

Please clarify more clearly what "maximum number of active cpus" mean for a workqueue.

Quoted 12 lines
rust/kernel/workqueue/builder.rs new file +12 -0
+    ///
+    /// If not set, a default value of `WQ_DFL_ACTIVE` is used. The maximum value is
+    /// `WQ_MAX_ACTIVE`.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().max_active(16).build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from wq with max_active=16"))?;
+    /// # Ok::<(), Error>(())

rust/kernel/workqueue/builder.rs line 276

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+    /// ```
+    #[inline]
+    pub fn max_active(mut self, max_active: u32) -> Self {
+        // If provided `max_active` is greater than `i32::MAX`, then we need to trigger the C-side
+        // comparison with `WQ_MAX_ACTIVE`, which we can do by clamping to `i32::MAX`.
+        self.max_active = i32::try_from(max_active).unwrap_or(i32::MAX);
+        self
+    }
+
+    /// Mark this workqueue as cpu intensive.

What does the kernel do to a queue when it is marked cpu intensive?

Quoted 36 lines
rust/kernel/workqueue/builder.rs new file +36 -0
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().cpu_intensive().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from cpu-intensive wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_CPU_INTENSIVE")]
+    pub fn cpu_intensive(mut self) -> Self {
+        self.flags |= bindings::wq_flags_WQ_CPU_INTENSIVE;
+        self
+    }
+
+    /// Make this workqueue visible in sysfs.
+    ///
+    /// # Examples
+    ///
+    /// ```
+    /// use kernel::workqueue::Queue;
+    ///
+    /// let wq = Queue::new_unbound().sysfs().build(c"my-wq")?;
+    /// wq.try_spawn(GFP_KERNEL, || pr_info!("Hello from sysfs wq"))?;
+    /// # Ok::<(), Error>(())
+    /// ```
+    #[inline]
+    #[doc(alias = "WQ_SYSFS")]
+    pub fn sysfs(mut self) -> Self {
+        self.flags |= bindings::wq_flags_WQ_SYSFS;
+        self
+    }
+}
+

rust/kernel/workqueue/builder.rs line 322

Quoted 10 lines
rust/kernel/workqueue/builder.rs new file +10 -0
+/// Indicates that this workqueue runs in a normal context (as opposed to softirq context).
+pub trait TypeNormal {}
+impl TypeNormal for TypeUnbound {}
+impl TypeNormal for TypePercpu {}
+impl TypeNormal for TypePowerEfficient {}
+impl TypeNormal for TypeOrdered {}
+
+/// Options that are not available on BH workqueues.
+impl<T: TypeNormal> Builder<T> {
+    /// Allow this workqueue to be frozen during suspend.

What happens to queues that are not frozen during suspend? They just continue executing tasks until the system is powered off?

Best regards,
Andreas Hindborg

reviewer
#15 lore raw
In reply to Alice Ryhl
Quoted 33 lines · On Thu, 12 Mar 2026 09:23:01 +0000 Alice Ryhl <[email protected]> wrote:

GPU drivers often need to create their own workqueues for various reasons. Add the ability to do so.

---
Changes in v4:
- Add link to delayed work fix.
- Redo workqueue creation to prevent invalid configurations.
- Introduce a directory as workqueue.rs was getting really large.
- Link to v3: https://lore.kernel.org/r/[email protected]

Changes in v3:
- Switch to builder pattern.
- Drop BH workqueues for now.
- Mark delayed wq change as fix.
- Link to v2: https://lore.kernel.org/r/[email protected]

Changes in v2:
- Redo how flagging works.
- Restrict delayed work to not be usable on custom workqueues.
- Link to v1: https://lore.kernel.org/r/[email protected]

---
Alice Ryhl (3):
rust: workqueue: restrict delayed work to global wqs
rust: workqueue: create workqueue subdirectory
rust: workqueue: add creation of workqueues

 MAINTAINERS                                    |   1 +
 rust/helpers/workqueue.c                       |   7 +
 rust/kernel/workqueue/builder.rs               | 380 +++++++++++++++++++++++++
 rust/kernel/{workqueue.rs => workqueue/mod.rs} |  53 +++-
 4 files changed, 437 insertions(+), 4 deletions(-)
Quoted 8 lines

---
base-commit: df9c51269a5e2a6fbca2884a756a4011a5e78748
change-id: 20250411-create-workqueue-d053158c7a4b

Best regards,

Signature · 1 line
Alice Ryhl <[email protected]>

Hi Alice,

What's the status of this series? Do you have plans to continue working on this anytime soon? I would like to take it over and continue the work otherwise.

Thanks,
Onur