rust: sync: add WaitQueue infrastructure
Danilo Krummrich
reviewerv3 cover rust: sync: add WaitQueue infrastructure
Add a WaitQueue abstraction wrapping struct wait_queue_head, providing wait_event()-style methods that take a condition closure directly.
This is needed for a (subsequent) DRM patch series to wait for in-flight file close operations during driver unbind. The Tyr driver also needs it for "CSF firmware responses and other GPU-driven events" [1].
Convert CondVar (and PollCondVar) to use WaitQueue internally, removing duplicate code using WaitQueue as the base primitive and add wait_interruptible_freezable_nonexclusive() for subsequent binder usage.
This series is based on FUJITA Tomonori's CondVar timeout conversion [2].
Changes in v3:
- Use Delta<Jiffy> for timeouts and treat negative durations as zero.
- Add might_sleep() and caller tracking to the wait_event() methods.
- Gate the WaitQueue KUnit tests behind a dedicated Kconfig option.
Changes in v2:
- Drop the already merged workqueue patch.
- Make as_raw() public.
- Split error types and use bindings constants for direct C calls.
- Avoid condition rechecks after success.
- Check the condition before preparing a waiter.
- Add non-exclusive wait support, including a CondVar method for subsequent Binder conversion.
- Handle zero and expired timeouts before interruption; extend KUnit coverage.
[1] https://lore.kernel.org/dri-devel/[email protected]/
[2] https://lore.kernel.org/rust-for-linux/[email protected]/
Danilo Krummrich (5):
rust: task: add safe schedule_timeout() wrapper
rust: sync: add WaitQueue infrastructure
rust: sync: convert CondVar and PollCondVar to use WaitQueue
rust: sync: condvar: use task::schedule_timeout()
rust: sync: condvar: add non-exclusive freezable waiting
rust/kernel/Kconfig.test | 10 + rust/kernel/sync.rs | 8 + rust/kernel/sync/condvar.rs | 154 +++++----- rust/kernel/sync/poll.rs | 19 +- rust/kernel/sync/wait.rs | 546 ++++++++++++++++++++++++++++++++++++ rust/kernel/task.rs | 23 ++ 6 files changed, 664 insertions(+), 96 deletions(-) create mode 100644 rust/kernel/sync/wait.rs
base-commit: 60fff02883397e17439c16f0e72b2f8c5f6eb791
Signature · 1 line
2.56.1
Danilo Krummrich
reviewerPATCH v3 1/5 rust: task: add safe schedule_timeout() wrapper
Add a safe wrapper around schedule_timeout() that takes and returns Delta<Jiffy>, treating negative durations as zero.
--- rust/kernel/task.rs | 23 +++++++++++++++++++++++ 1 file changed, 23 insertions(+)
rust/kernel/task.rs +23 -0
@@ -10,6 +10,10 @@ pid_namespace::PidNamespace, prelude::*, sync::aref::ARef, + time::{ + Delta, + Jiffy, // + }, types::{NotThreadSafe, Opaque}, };use core::{
@@ -20,6 +24,25 @@/// A sentinel value used for infinite timeouts. pub const MAX_SCHEDULE_TIMEOUT :c_long = c_long::MAX ; +/// Sleeps for the given timeout, or until woken. +/// +/// Returns the remaining time, or zero if the timeout expired. The task state should be set before +/// calling this. +/// +/// A negative timeout is treated as zero. +/// +/// A timeout of [`MAX_SCHEDULE_TIMEOUT`] jiffies means an indefinite wait. In that case, the +/// returned duration is also [`MAX_SCHEDULE_TIMEOUT`] jiffies. +#[inline] +pub fn schedule_timeout (timeout:Delta <Jiffy >) ->Delta <Jiffy > { +let timeout = timeout.as_jiffies ().max (0 ); + +// SAFETY: `schedule_timeout()` is always safe to call. +let remaining =unsafe { bindings::schedule_timeout (timeout) }; + +Delta ::from_jiffies (remaining) +} +/// Bitmask for tasks that are sleeping in an interruptible state. pub const TASK_INTERRUPTIBLE :c_int = bindings::TASK_INTERRUPTIBLE as c_int ;/// Bitmask for tasks that are sleeping in an uninterruptible state.
Signature · 1 line
2.56.1
Re: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
rust/kernel/task.rs line 38
Quoted 2 lines · On Tue, 17 Nov 2026 01:14:42 +0100 Danilo Krummrich <[email protected]> wrote:
rust/kernel/task.rs +2
-0+pub fn schedule_timeout (timeout:Delta <Jiffy >) ->Delta <Jiffy > { +let timeout = timeout.as_jiffies ().max (0 );
In earlier reviews of my patches, Gary asked me to use the function syntax or clamp() for this instead of the method syntax [1][2]:
let timeout = isize::max(timeout.as_jiffies(), 0);
[1] https://lore.kernel.org/rust-for-linux/[email protected]/
[2] https://lore.kernel.org/rust-for-linux/[email protected]/
Gary Guo
maintainerRe: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
Quoted 10 lines · FUJITA Tomonori
rust/kernel/task.rs line 38
Quoted 2 lines · On Tue, 17 Nov 2026 01:14:42 +0100 Danilo Krummrich <[email protected]> wrote:
rust/kernel/task.rs +2
-0+pub fn schedule_timeout (timeout:Delta <Jiffy >) ->Delta <Jiffy > { +let timeout = timeout.as_jiffies ().max (0 );In earlier reviews of my patches, Gary asked me to use the function syntax or
clamp()for this instead of the method syntax [1][2]:let timeout = isize::max(timeout.as_jiffies(), 0);
Yes, please do. I'm okay with either isize::max or cmp::max.
Best,
Gary
Danilo Krummrich
reviewerRe: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
Quoted 7 lines · Gary Guo
Quoted 6 lines · FUJITA Tomonori
Quoted 2 lines · On Tue, 17 Nov 2026 01:14:42 +0100 Danilo Krummrich <[email protected]> wrote:
rust/kernel/task.rs +2
-0+pub fn schedule_timeout (timeout:Delta <Jiffy >) ->Delta <Jiffy > { +let timeout = timeout.as_jiffies ().max (0 );
Quoted 6 lines
Quoted 4 lines
In earlier reviews of my patches, Gary asked me to use the function syntax or
clamp()for this instead of the method syntax [1][2]:let timeout = isize::max(timeout.as_jiffies(), 0);Yes, please do. I'm okay with either
isize::maxorcmp::max.
The examples in [1][2] both seemed to be able to avoid a cast with this, which isn't the case for this patch. There also doesn't seem to be any ambiguity in the extression used in this patch. So I don't see why changing it would be better.
Thanks,
Danilo
Quoted 2 lines
Gary Guo
maintainerRe: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
Quoted 11 lines · Danilo Krummrich
Quoted 10 lines · Gary Guo
Quoted 9 lines · FUJITA Tomonori
rust/kernel/task.rs line 38
Quoted 2 lines · On Tue, 17 Nov 2026 01:14:42 +0100 Danilo Krummrich <[email protected]> wrote:
rust/kernel/task.rs +2
-0+pub fn schedule_timeout (timeout:Delta <Jiffy >) ->Delta <Jiffy > { +let timeout = timeout.as_jiffies ().max (0 );In earlier reviews of my patches, Gary asked me to use the function syntax or
clamp()for this instead of the method syntax [1][2]:
Quoted 8 lines
Quoted 3 lines
Quoted 1 line
let timeout = isize::max(timeout.as_jiffies(), 0);Yes, please do. I'm okay with either
isize::maxorcmp::max.The examples in [1][2] both seemed to be able to avoid a cast with this, which isn't the case for this patch. There also doesn't seem to be any ambiguity in the extression used in this patch. So I don't see why changing it would be better.
The ambiguity is "the maximum value returned will be 0" and "the minimum value returned will be 0". The answer is always the latter but I'd avoid the mental gymnastics.
Best,
Gary
Danilo Krummrich
reviewerRe: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
Quoted 16 lines · Gary Guo
Quoted 15 lines · Danilo Krummrich
Quoted 13 lines · Gary Guo
Quoted 10 lines · FUJITA Tomonori
rust/kernel/task.rs line 38
Quoted 2 lines · On Tue, 17 Nov 2026 01:14:42 +0100 Danilo Krummrich <[email protected]> wrote:
rust/kernel/task.rs +2
-0+pub fn schedule_timeout (timeout:Delta <Jiffy >) ->Delta <Jiffy > { +let timeout = timeout.as_jiffies ().max (0 );In earlier reviews of my patches, Gary asked me to use the function syntax or
clamp()for this instead of the method syntax [1][2]:let timeout = isize::max(timeout.as_jiffies(), 0);Yes, please do. I'm okay with either
isize::maxorcmp::max.
Quoted 8 lines
Quoted 4 lines
The examples in [1][2] both seemed to be able to avoid a cast with this, which isn't the case for this patch. There also doesn't seem to be any ambiguity in the extression used in this patch. So I don't see why changing it would be better.
The ambiguity is "the maximum value returned will be 0" and "the minimum value returned will be 0". The answer is always the latter but I'd avoid the mental gymnastics.
To me clearly both
a.max(b) (1) max(a, b) (2)
read as the maximum of a and b.
IIUC, what you are saying is that people may read (1) as
a.at_max(b)
which would indeed be equivalent to min(a, b).
If this is indeed commonly the case I'd suggest to raise this to become a common rule (ideally with a lint), such that we avoid this to be discussed in reviews.
Personally, I prefer (1) for the expression above, but I don't care too much, so I'm fine to change it. (In case I don't need to respin I'll change it on apply.)
Thanks,
Danilo
Re: [PATCH v3 1/5] rust: task: add safe schedule_timeout() wrapper
In reply to Danilo KrummrichQuoted 5 lines · Danilo Krummrich
Add a safe wrapper around
schedule_timeout()that takes and returns Delta<Jiffy>, treating negative durations as zero.Acked-by Miguel Ojeda <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>
Danilo Krummrich
reviewerPATCH v3 2/5 rust: sync: add WaitQueue infrastructure
In reply to Danilo KrummrichImplement a wait queue wrapping the kernel's struct wait_queue_head, with wait_event()-style methods that take a condition closure directly.
The API mirrors the C wait_event() family:
- wait_event()
- wait_event_interruptible()
- wait_event_timeout()
- wait_event_interruptible_timeout()
- wake_up() / wake_up_all() / wake_up_sync()
The interruptible variant returns Result<(), Interrupted>, the timeout variant Result<(), TimedOut>, and the combined variant Result<(), WaitError>. These errors map to ERESTARTSYS (signal) or ETIMEDOUT (timeout).
--- rust/kernel/Kconfig.test | 10 + rust/kernel/sync.rs | 8 + rust/kernel/sync/wait.rs | 549 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 567 insertions(+) create mode 100644 rust/kernel/sync/wait.rs
rust/kernel/Kconfig.test +10 -0
@@ -104,4 +104,14 @@ config RUST_TIME_KUNIT_TEST If unsure, say N. +config RUST_WAITQUEUE_KUNIT_TEST + bool "KUnit tests for the Rust wait queue API" if !KUNIT_ALL_TESTS + default KUNIT_ALL_TESTS + help + This option enables KUnit tests for the Rust wait queue API. + These are only for development and testing, not for regular + kernel use cases. + + If unsure, say N. + endif
rust/kernel/sync.rs +8 -0
@@ -19,6 +19,7 @@mod refcount;mod set_once;pub mod srcu; +mod wait;pub use arc::{Arc ,ArcBorrow ,UniqueArc };pub use completion::Completion ;
@@ -41,3 +42,10 @@pub use refcount::Refcount ;pub use set_once::SetOnce ;pub use srcu::Srcu ; +pub use wait::{ + new_waitqueue, +Interrupted , +TimedOut , +WaitError , +WaitQueue ,// +};
rust/kernel/sync/wait.rs new file +549 -0
549 lines not on this page · raw message
Signature · 1 line
2.56.1
Gary Guo
maintainerQuoted 99 lines · Danilo Krummrich
Implement a wait queue wrapping the kernel's struct
wait_queue_head, with wait_event()-style methods that take a condition closure directly.The API mirrors the C
wait_event()family:-
wait_event()
-wait_event_interruptible()
-wait_event_timeout()
-wait_event_interruptible_timeout()
-wake_up()/wake_up_all()/wake_up_sync()The interruptible variant returns Result<(), Interrupted>, the timeout variant Result<(), TimedOut>, and the combined variant Result<(), WaitError>. These errors map to ERESTARTSYS (signal) or ETIMEDOUT (timeout).
Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/Kconfig.test | 10 + rust/kernel/sync.rs | 8 + rust/kernel/sync/wait.rs | 549 +++++++++++++++++++++++++++++++++++++++ 3 files changed, 567 insertions(+) create mode 100644 rust/kernel/sync/wait.rsrust/kernel/Kconfig.test +10
-0@@ -104,4 +104,14 @@ config RUST_TIME_KUNIT_TEST If unsure, say N. +config RUST_WAITQUEUE_KUNIT_TEST + bool "KUnit tests for the Rust wait queue API" if !KUNIT_ALL_TESTS + default KUNIT_ALL_TESTS + help + This option enables KUnit tests for the Rust wait queue API. + These are only for development and testing, not for regular + kernel use cases. + + If unsure, say N. + endifrust/kernel/sync.rs +8
-0@@ -19,6 +19,7 @@mod refcount;mod set_once;pub mod srcu; +mod wait;pub use arc::{Arc ,ArcBorrow ,UniqueArc };pub use completion::Completion ;@@ -41,3 +42,10 @@pub use refcount::Refcount ;pub use set_once::SetOnce ;pub use srcu::Srcu ; +pub use wait::{ + new_waitqueue, +Interrupted , +TimedOut , +WaitError , +WaitQueue ,// +};rust/kernel/sync/wait.rs new file +27
-0@@ -0,0 +1,549 @@ +// SPDX-License-Identifier: GPL-2.0 +// SPDX-FileCopyrightText: Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved. + +//! Wait queue. +//! +//! C header: [`include/linux/wait.h`](srctree/include/linux/wait.h) + +use super ::LockClassKey ; +use crate ::{ + prelude::*, + str::CStr , + task, + time::{ +Delta , +Jiffy ,// + }, + types::Opaque ,// +}; + +use core::{ + pin::Pin , + ptr,// +}; + +/// Creates a [`WaitQueue`] initialiser with the given name and a newly-created lock class. +#[macro_export] +macro_rules! new_waitqueue {
rust/kernel/sync/wait.rs line 37
Quoted 10 lines
rust/kernel/sync/wait.rs new file +10
-0+ ($($name:literal)?) => { + $crate::sync::WaitQueue ::new ( + $crate::optional_name!($($name)?), + $crate::static_lock_class!(), + ) + }; +} +pub use new_waitqueue; + +/// Exposes the kernel's [`struct wait_queue_head`] as a Rust wait queue.
The first paragraph is rendered as rustdoc short item description so please have something more descriptive and leave the C equivalence to other pragraphs.
Best,
Gary
Quoted 45 lines
rust/kernel/sync/wait.rs new file +45
-0+/// +/// A `WaitQueue` allows a thread to sleep until a caller-supplied condition becomes true, +/// re-checking the condition on each wake-up. This matches the C `wait_event()` family of macros. +/// +/// For waiting with a lock guard (the condition variable pattern), use [`CondVar`](super::CondVar) +/// instead. +/// +/// Instances of `WaitQueue` need a lock class and to be pinned. The recommended way to create such +/// instances is with the [`pin_init!`] and [`new_waitqueue!`] macros. +/// +/// # Examples +/// +/// ``` +/// use kernel::sync::{ +/// atomic::{ +/// Atomic, +/// Relaxed, +/// }, +/// new_waitqueue, +/// WaitQueue, +/// }; +/// +/// #[pin_data] +/// pub struct Example { +/// value: Atomic<i32>, +/// #[pin] +/// queue: WaitQueue, +/// } +/// +/// fn wait_for_value(e: &Example, v: i32) { +/// e.queue.wait_event(|| e.value.load(Relaxed) == v); +/// } +/// +/// fn set_value(e: &Example, v: i32) { +/// e.value.store(v, Relaxed); +/// e.queue.wake_up(); +/// } +/// ``` +/// +/// [`struct wait_queue_head`]: srctree/include/linux/wait.h +#[pin_data] +pub struct WaitQueue { +#[pin] + wait_queue_head:Opaque <bindings::wait_queue_head >, +}
Quoted 11 lines · Danilo Krummrich
Implement a wait queue wrapping the kernel's struct
wait_queue_head, with wait_event()-style methods that take a condition closure directly.The API mirrors the C
wait_event()family:-
wait_event()
-wait_event_interruptible()
-wait_event_timeout()
-wait_event_interruptible_timeout()
-wake_up()/wake_up_all()/wake_up_sync()
Quoted 6 lines
The interruptible variant returns Result<(), Interrupted>, the timeout variant Result<(), TimedOut>, and the combined variant Result<(), WaitError>. These errors map to ERESTARTSYS (signal) or ETIMEDOUT (timeout).
Signed-off-by Danilo Krummrich <[email protected]>
Thanks for addressing the feedback from v1!
Alice
Danilo Krummrich
reviewerPATCH v3 3/5 rust: sync: convert CondVar and PollCondVar to use WaitQueue
In reply to Danilo KrummrichCondVar currently wraps a raw wait_queue_head directly. Now that WaitQueue provides the same primitive, convert CondVar to use it instead. This removes duplicate init, pinning, Send/Sync, and wake-up code from CondVar.
Keep the wait logic in CondVar since it needs to release and reacquire a lock guard around the sleep, which WaitQueue's condition-based wait_event() methods do not handle. Use wait_once_exclusive() to perform a single prepare_to_wait_exclusive / finish_wait cycle with a caller-provided schedule function to implement the lock-aware sleep.
Adapt PollCondVar accordingly.
--- rust/kernel/sync/condvar.rs | 89 +++++++++++-------------------------- rust/kernel/sync/poll.rs | 19 ++++---- rust/kernel/sync/wait.rs | 2 - 3 files changed, 33 insertions(+), 77 deletions(-)
rust/kernel/sync/condvar.rs +25 -64
@@ -5,21 +5,29 @@//! This module allows Rust code to use the kernel's [`struct wait_queue_head`] as a condition //! variable. -use super ::{lock::Backend , lock::Guard ,LockClassKey }; +use super ::{ + lock::Backend , + lock::Guard , +LockClassKey , +WaitQueue ,// +}; +use crate ::{ - ffi::{c_int, c_long}, - str::{CStr ,CStrExt as _}, + prelude::*, + str::CStr , task::{ -MAX_SCHEDULE_TIMEOUT ,TASK_FREEZABLE ,TASK_INTERRUPTIBLE ,TASK_NORMAL ,TASK_UNINTERRUPTIBLE , +MAX_SCHEDULE_TIMEOUT , +TASK_FREEZABLE , +TASK_INTERRUPTIBLE , +TASK_UNINTERRUPTIBLE ,// }, time::{Delta ,Jiffy ,// }, - types::Opaque , }; -use core::{marker::PhantomPinned , pin::Pin , ptr}; -use pin_init::{pin_data, pin_init,PinInit }; + +use core::pin::Pin ;/// Creates a [`CondVar`] initialiser with the given name and a newly-created lock class. #[macro_export]
@@ -84,33 +92,14 @@ macro_rules! new_condvar {#[pin_data] pub struct CondVar {#[pin] -pub (crate ) wait_queue_head:Opaque <bindings::wait_queue_head >, - -/// A condvar needs to be pinned because it contains a [`struct list_head`] that is -/// self-referential, so it cannot be safely moved once it is initialised. -/// -/// [`struct list_head`]: srctree/include/linux/types.h -#[pin] - _pin:PhantomPinned , +pub (crate ) wq:WaitQueue , } -// SAFETY: `CondVar` only uses a `struct wait_queue_head`, which is safe to use on any thread. -unsafe impl Send for CondVar {} - -// SAFETY: `CondVar` only uses a `struct wait_queue_head`, which is safe to use on multiple threads -// concurrently. -unsafe impl Sync for CondVar {} -impl CondVar {/// Constructs a new condvar initialiser. pub fn new (name: &'staticCStr , key:Pin <&'staticLockClassKey >) ->impl PinInit <Self > {pin_init! (Self { - _pin:PhantomPinned , -// SAFETY: `slot` is valid while the closure is called and both `name` and `key` have -// static lifetimes so they live indefinitely. - wait_queue_head <-Opaque ::ffi_init (|slot|unsafe { - bindings::__init_waitqueue_head (slot, name.as_char_ptr (), key.as_ptr ()) - }), + wq <-WaitQueue ::new (name, key), }) }
@@ -120,23 +109,10 @@ fn wait_internal<T: ?Sized, B: Backend>( guard: &mut Guard<'_, T, B>, timeout_in_jiffies: c_long, ) -> c_long { -let wait =Opaque ::<bindings::wait_queue_entry >::uninit (); - -// SAFETY: `wait` points to valid memory. -unsafe { bindings::init_wait (wait.get ()) }; - -// SAFETY: Both `wait` and `wait_queue_head` point to valid memory. -unsafe { - bindings::prepare_to_wait_exclusive (self.wait_queue_head.get (), wait.get (), wait_state) - }; - -// SAFETY: Switches to another thread. The timeout can be any number. -let ret = guard.do_unlocked (||unsafe { bindings::schedule_timeout (timeout_in_jiffies) }); - -// SAFETY: Both `wait` and `wait_queue_head` point to valid memory. -unsafe { bindings::finish_wait (self.wait_queue_head.get (), wait.get ()) }; - - ret + self.wq.wait_once_exclusive (wait_state, || { +// SAFETY: Switches to another thread. The timeout can be any number. + guard.do_unlocked (||unsafe { bindings::schedule_timeout (timeout_in_jiffies) }) + }) }/// Releases the lock and waits for a notification in uninterruptible mode.
@@ -211,28 +187,13 @@ pub fn wait_interruptible_timeout<T: ?Sized, B: Backend>( } } -/// Calls the kernel function to notify the appropriate number of threads. -fn notify (&self, count:c_int ) { -// SAFETY: `wait_queue_head` points to valid memory. -unsafe { - bindings::__wake_up ( - self.wait_queue_head.get (), -TASK_NORMAL , - count, - ptr::null_mut (), - ) - }; - } -/// Calls the kernel function to notify one thread synchronously. /// /// This method behaves like `notify_one`, except that it hints to the scheduler that the -/// current thread is about to go to sleep, so it should schedule the target thread on the same -/// CPU. +/// current thread expects to sleep soon. #[inline] pub fn notify_sync (&self) { -// SAFETY: `wait_queue_head` points to valid memory. -unsafe { bindings::__wake_up_sync (self.wait_queue_head.get (),TASK_NORMAL ) }; + self.wq.wake_up_sync (); }/// Wakes a single waiter up, if any.
@@ -241,7 +202,7 @@ pub fn notify_sync(&self) {/// completely (as opposed to automatically waking up the next waiter). #[inline] pub fn notify_one (&self) { - self.notify (1 ); + self.wq.wake_up (); }/// Wakes all waiters up, if any.
@@ -250,7 +211,7 @@ pub fn notify_one(&self) {/// completely (as opposed to automatically waking up the next waiter). #[inline] pub fn notify_all (&self) { - self.notify (0 ); + self.wq.wake_up_all (); } }
rust/kernel/sync/poll.rs +8 -11
@@ -64,11 +64,11 @@ pub fn register_wait(&self, file: &File, cv: &PollCondVar) {// * `file.as_ptr()` references a valid file for the duration of this call. // * `self.table` is null or references a valid poll_table for the duration of this call. // * Since `PollCondVar` is pinned, its destructor is guaranteed to run before the memory -// containing `cv.wait_queue_head` is invalidated. Since the destructor clears all -// waiters and then waits for an rcu grace period, it's guaranteed that -// `cv.wait_queue_head` remains valid for at least an rcu grace period after the removal -// of the last waiter. -unsafe { bindings::poll_wait (file.as_ptr (), cv.wait_queue_head.get (), self.table) } +// containing the wait queue head is invalidated. Since the destructor clears all +// waiters and then waits for an rcu grace period, it's guaranteed that the wait queue +// head remains valid for at least an rcu grace period after the removal of the last +// waiter. +unsafe { bindings::poll_wait (file.as_ptr (), cv.wq.as_raw (), self.table) } } }
@@ -105,9 +105,7 @@ impl PinnedDrop for PollCondVar {#[inline] fn drop (self:Pin <&mut Self >) {// Clear anything registered using `register_wait`. -// -// SAFETY: The pointer points at a valid `wait_queue_head`. -unsafe { bindings::__wake_up_pollfree (self.inner.wait_queue_head.get ()) }; + self.inner.wq.wake_up_pollfree ();// Wait for epoll items to be properly removed. synchronize_rcu ();
@@ -165,14 +163,13 @@ fn drop(&mut self) {// SAFETY: ManuallyDrop::take ok because not already taken. let boxed =unsafe {ManuallyDrop ::take (&mut self.inner) }; + boxed.inner.inner.wq.wake_up_pollfree (); +// SAFETY: The code below frees the box without calling the actual destructor of the type, // but it's okay because it re-implements the destructor using `kfree_rcu()` in place of // `synchronize_rcu()`. let ptr =KBox ::into_raw (unsafe {Pin ::into_inner_unchecked (boxed) }); -// SAFETY: The pointer points at a valid `wait_queue_head`. -unsafe { bindings::__wake_up_pollfree ((*ptr).inner.inner.wait_queue_head.get ()) }; -// SAFETY: This was allocated using `KBox::pin_init`, so it can be freed with `kvfree`. unsafe { bindings::kvfree_call_rcu ((*ptr).rcu.get (), ptr.cast ::<ffi::c_void >()) }; }
rust/kernel/sync/wait.rs +0 -2
@@ -238,7 +238,6 @@ pub(super) fn wait_once<F, R>(&self, wait_state: c_int, schedule_fn: F) -> R/// Performs a single exclusive prepare-to-wait / finish-wait cycle, calling `schedule_fn` /// in between. #[inline] -#[expect(unused)] pub (super )fn wait_once_exclusive <F ,R >(&self, wait_state:c_int , schedule_fn:F ) ->R where F :FnOnce () ->R ,
@@ -322,7 +321,6 @@ pub fn wake_up_sync(&self) {/// Used when a wait queue is about to be freed, to ensure epoll items are properly removed. /// Matches C's `wake_up_pollfree()`. #[inline] -#[expect(unused)] pub (super )fn wake_up_pollfree (&self) {// SAFETY: `wait_queue_head` points to valid memory. unsafe { bindings::__wake_up_pollfree (self.wait_queue_head.get ()) };
Signature · 1 line
2.56.1
Re: [PATCH v3 3/5] rust: sync: convert CondVar and PollCondVar to use WaitQueue
Quoted 11 lines · Danilo Krummrich
CondVar currently wraps a raw
wait_queue_headdirectly. Now that WaitQueue provides the same primitive, convert CondVar to use it instead. This removes duplicate init, pinning, Send/Sync, and wake-up code from CondVar.Keep the wait logic in CondVar since it needs to release and reacquire a lock guard around the sleep, which WaitQueue's condition-based
wait_event()methods do not handle. Usewait_once_exclusive()to perform a singleprepare_to_wait_exclusive/finish_waitcycle with a caller-provided schedule function to implement the lock-aware sleep.
Quoted 3 lines
Adapt PollCondVar accordingly.
Signed-off-by Danilo Krummrich <[email protected]>
Danilo Krummrich
reviewerPATCH v3 4/5 rust: sync: condvar: use task::schedule_timeout()
In reply to Danilo KrummrichConvert CondVar::wait_internal() from calling bindings::schedule_timeout() directly to using the safe task::schedule_timeout() wrapper. This removes the unsafe call from CondVar.
--- rust/kernel/sync/condvar.rs | 42 ++++++++++++++++++------------------- 1 file changed, 20 insertions(+), 22 deletions(-)
rust/kernel/sync/condvar.rs +20 -22
@@ -16,7 +16,7 @@ prelude::*, str::CStr, task::{ -MAX_SCHEDULE_TIMEOUT , + self,TASK_FREEZABLE ,TASK_INTERRUPTIBLE ,TASK_UNINTERRUPTIBLE ,//
@@ -107,11 +107,10 @@ fn wait_internal<T: ?Sized, B: Backend>( &self, wait_state: c_int, guard: &mut Guard<'_, T, B>, - timeout_in_jiffies: c_long, - ) -> c_long { + timeout: Delta<Jiffy>, + ) -> Delta<Jiffy> { self.wq.wait_once_exclusive (wait_state, || { -// SAFETY: Switches to another thread. The timeout can be any number. - guard.do_unlocked (||unsafe { bindings::schedule_timeout (timeout_in_jiffies) }) + guard.do_unlocked (|| task::schedule_timeout (timeout)) }) }
@@ -122,7 +121,11 @@ fn wait_internal<T: ?Sized, B: Backend>(/// [`CondVar::notify_one`] or [`CondVar::notify_all`]. Note that it may also wake up /// spuriously. pub fn wait <T : ?Sized ,B :Backend >(&self, guard: &mut Guard <'_,T ,B >) { - self.wait_internal (TASK_UNINTERRUPTIBLE , guard,MAX_SCHEDULE_TIMEOUT ); + self.wait_internal ( +TASK_UNINTERRUPTIBLE , + guard, +Delta ::from_jiffies (task::MAX_SCHEDULE_TIMEOUT ), + ); }/// Releases the lock and waits for a notification in interruptible mode.
@@ -133,7 +136,11 @@ pub fn wait<T: ?Sized, B: Backend>(&self, guard: &mut Guard<'_, T, B>) {/// Returns whether there is a signal pending. #[must_use = "wait_interruptible returns if a signal is pending, so the caller must check the return value" ] pub fn wait_interruptible <T : ?Sized ,B :Backend >(&self, guard: &mut Guard <'_,T ,B >) ->bool { - self.wait_internal (TASK_INTERRUPTIBLE , guard,MAX_SCHEDULE_TIMEOUT ); + self.wait_internal ( +TASK_INTERRUPTIBLE , + guard, +Delta ::from_jiffies (task::MAX_SCHEDULE_TIMEOUT ), + );crate ::current!().signal_pending () }
@@ -151,7 +158,7 @@ pub fn wait_interruptible_freezable<T: ?Sized, B: Backend>( self.wait_internal (TASK_INTERRUPTIBLE |TASK_FREEZABLE , guard, -MAX_SCHEDULE_TIMEOUT , +Delta ::from_jiffies (task::MAX_SCHEDULE_TIMEOUT ), );crate ::current!().signal_pending () }
@@ -169,21 +176,12 @@ pub fn wait_interruptible_timeout<T: ?Sized, B: Backend>( guard: &mut Guard<'_, T, B>, timeout: Delta<Jiffy>, ) -> CondVarTimeoutResult { - let jiffies = timeout.as_jiffies(); -let res = self.wait_internal ( -TASK_INTERRUPTIBLE , - guard, - jiffies.clamp (0 ,MAX_SCHEDULE_TIMEOUT ), - ); + let remaining = self.wait_internal(TASK_INTERRUPTIBLE, guard, timeout); -match (res,crate ::current!().signal_pending ()) { - (remainder,true ) =>CondVarTimeoutResult ::Signal { - remainder:Delta ::from_jiffies (remainder), - }, - (0 ,false ) =>CondVarTimeoutResult ::Timeout , - (remainder,false ) =>CondVarTimeoutResult ::Woken { - remainder:Delta ::from_jiffies (remainder), - }, +match (remaining,current! ().signal_pending ()) { + (remainder,true ) =>CondVarTimeoutResult ::Signal { remainder }, + (Delta ::ZERO ,false ) =>CondVarTimeoutResult ::Timeout , + (remainder,false ) =>CondVarTimeoutResult ::Woken { remainder }, } }
Signature · 1 line
2.56.1
Re: [PATCH v3 4/5] rust: sync: condvar: use task::schedule_timeout()
rust/kernel/sync/condvar.rs fn wait_internal<T: ?Sized, B: Backend>( · line 129
Quoted 11 lines · Danilo Krummrich
rust/kernel/sync/condvar.rs +5
-1@@ -122,7 +121,11 @@ fn wait_internal<T: ?Sized, B: Backend>(/// [`CondVar::notify_one`] or [`CondVar::notify_all`]. Note that it may also wake up /// spuriously. pub fn wait <T : ?Sized ,B :Backend >(&self, guard: &mut Guard <'_,T ,B >) { - self.wait_internal (TASK_UNINTERRUPTIBLE , guard,MAX_SCHEDULE_TIMEOUT ); + self.wait_internal ( +TASK_UNINTERRUPTIBLE , + guard, +Delta ::from_jiffies (task::MAX_SCHEDULE_TIMEOUT ), + ); }
Now that CondVar uses task::schedule_timeout(), all remaining uses of task::MAX_SCHEDULE_TIMEOUT wrap it in Delta::from_jiffies(...), so we could also change MAX_SCHEDULE_TIMEOUT itself to Delta<Jiffy> (either if you spin a v4 or as a follow-up). Either way:
Alice
Re: [PATCH v3 4/5] rust: sync: condvar: use task::schedule_timeout()
Quoted 18 lines · On Tue, 17 Nov 2026 14:16:52 +0000 Alice Ryhl <[email protected]> wrote:
rust/kernel/sync/condvar.rs fn wait_internal<T: ?Sized, B: Backend>( · line 129
Quoted 11 lines · Danilo Krummrich
rust/kernel/sync/condvar.rs +5
-1@@ -122,7 +121,11 @@ fn wait_internal<T: ?Sized, B: Backend>(/// [`CondVar::notify_one`] or [`CondVar::notify_all`]. Note that it may also wake up /// spuriously. pub fn wait <T : ?Sized ,B :Backend >(&self, guard: &mut Guard <'_,T ,B >) { - self.wait_internal (TASK_UNINTERRUPTIBLE , guard,MAX_SCHEDULE_TIMEOUT ); + self.wait_internal ( +TASK_UNINTERRUPTIBLE , + guard, +Delta ::from_jiffies (task::MAX_SCHEDULE_TIMEOUT ), + ); }Now that
CondVarusestask::schedule_timeout(), all remaining uses oftask::MAX_SCHEDULE_TIMEOUTwrap it inDelta::from_jiffies(...), so we could also changeMAX_SCHEDULE_TIMEOUTitself toDelta<Jiffy>(either if you spin a v4 or as a follow-up). Either way:
Quoted 1 line
Reviewed-by Alice Ryhl <[email protected]>
serdev users need this value too. I was going to add Delta::<Jiffy>::MAX for it, but this looks better.
Danilo Krummrich
reviewerPATCH v3 5/5 rust: sync: condvar: add non-exclusive freezable waiting
In reply to Danilo KrummrichCondVar currently registers all sleeping tasks as exclusive waiters. Add wait_interruptible_freezable_nonexclusive() for callers that need non-exclusive waits, using WaitQueue::wait_once().
Document that notify_one() and notify_sync() also wake all non-exclusive waiters.
This provides non-exclusive waiting support for a subsequent Binder conversion.
--- rust/kernel/sync/condvar.rs | 29 +++++++++++++++++++++++++++-- rust/kernel/sync/wait.rs | 1 - 2 files changed, 27 insertions(+), 3 deletions(-)
rust/kernel/sync/condvar.rs +27 -2
@@ -163,6 +163,31 @@ pub fn wait_interruptible_freezable<T: ?Sized, B: Backend>(crate ::current!().signal_pending () } +/// Releases the lock and waits non-exclusively in interruptible and freezable mode. +/// +/// Like [`CondVar::wait_interruptible_freezable`], but registers the current task as a +/// non-exclusive waiter. +/// +/// The lock is reacquired before returning. The wait may also end due to a signal or a +/// spurious wakeup. +/// +/// No lock other than the one released through `guard` should be held, since the task may +/// be frozen while sleeping. +/// +/// Returns whether there is a signal pending. +#[must_use = "the caller must check whether a signal is pending" ] +pub fn wait_interruptible_freezable_nonexclusive <T : ?Sized ,B :Backend >( + &self, + guard: &mut Guard <'_,T ,B >, + ) ->bool { + self.wq.wait_once (TASK_INTERRUPTIBLE |TASK_FREEZABLE , || { + guard.do_unlocked (|| { + task::schedule_timeout (Delta ::from_jiffies (task::MAX_SCHEDULE_TIMEOUT )) + }) + }); +crate ::current!().signal_pending () + } +/// Releases the lock and waits for a notification in interruptible mode. /// /// Atomically releases the given lock (whose ownership is proven by the guard) and puts the
@@ -185,7 +210,7 @@ pub fn wait_interruptible_timeout<T: ?Sized, B: Backend>( } } -/// Calls the kernel function to notify one thread synchronously. +/// Wakes all non-exclusive waiters and one exclusive waiter synchronously. /// /// This method behaves like `notify_one`, except that it hints to the scheduler that the /// current thread expects to sleep soon.
@@ -194,7 +219,7 @@ pub fn notify_sync(&self) { self.wq.wake_up_sync (); } -/// Wakes a single waiter up, if any. +/// Wakes all non-exclusive waiters and one exclusive waiter, if any. /// /// This is not 'sticky' in the sense that if no thread is waiting, the notification is lost /// completely (as opposed to automatically waking up the next waiter).
rust/kernel/sync/wait.rs +0 -1
@@ -227,7 +227,6 @@ fn wait_event_timeout_internal<F: FnMut() -> bool>(/// Performs a single non-exclusive prepare-to-wait / finish-wait cycle, calling `schedule_fn` /// in between. Non-exclusive waiters are all woken by [`wake_up()`](Self::wake_up). #[inline] -#[expect(unused)] pub (super )fn wait_once <F ,R >(&self, wait_state:c_int , schedule_fn:F ) ->R where F :FnOnce () ->R ,
Signature · 1 line
2.56.1
Re: [PATCH v3 5/5] rust: sync: condvar: add non-exclusive freezable waiting
Quoted 11 lines · Danilo Krummrich
CondVar currently registers all sleeping tasks as exclusive waiters. Add
wait_interruptible_freezable_nonexclusive()for callers that need non-exclusive waits, usingWaitQueue::wait_once().Document that
notify_one()andnotify_sync()also wake all non-exclusive waiters.This provides non-exclusive waiting support for a subsequent Binder conversion.
Signed-off-by Danilo Krummrich <[email protected]>