Skip to contentSkip to conversation

Fix forward()/expires() racing with concurrent arming

PATCH In review FUJITA Tomonori 4 participants 29 messages Aug 13 – Sep 22

Subsystems (2)
lore
Share & tools
lore
#0 lore raw

cover Fix forward()/expires() racing with concurrent arming

From: FUJITA Tomonori <[email protected]>

This series started from the review of patches 3 and 4 [1]: a hrtimer can be armed from any CPU at any time, including while its callback runs, so restricting HrTimer::expires() to the callback context is not by itself enough to remove the race.

It turned out that expires() is not the only problem. A callback may also change its expiry time with hrtimer_forward(), which is sound only because __run_hrtimer() dequeues the timer for the duration of the callback. Arming the same timer from another CPU puts it back into the rbtree while the callback runs, so hrtimer_forward() then changes the expiry of a timer that is queued, without the base lock and without re-checking the ordering, which leaves the tree unsorted.

Two of the four pointer types cannot construct that situation. Starting a Pin<Box<T, A>> moves the box into the handle, and starting a Pin<&mut T> consumes the exclusive borrow, so in both cases nothing is left to arm the timer with. Arc<T> is Clone and Pin<&T> is Copy, and both of their start functions are reachable from safe code, so safe Rust could arm a timer whose callback was running.

"No arming while the callback runs" cannot be expressed in the type system, because the callback begins when the timer expires rather than at any point in the Rust program, so patches 1 and 2 use the stronger "no arming while armed" instead. hrtimer_cancel() waits for the handler to return, which makes that the point where the right to arm can be handed back. The right to arm is split out of Arc<T> into HrTimerArc<T> and out of Pin<&T> into HrTimerPin<'a, T>, both non-clonable and consumed by start, modelled on ListArc; the object itself stays shareable through plain Arc references and shared pinned references respectively.

Patches 3 and 4 are the previously posted expires() and repr(transparent) patches, unchanged. With patches 1 and 2 in place, the callback context has no concurrent writer of node.expires. So HrTimerCallbackContext::expires() is sound.

[1]: https://lore.kernel.org/rust-for-linux/[email protected]/

FUJITA Tomonori (4):
rust: hrtimer: Introduce HrTimerArc to make arming exclusive
rust: hrtimer: Introduce HrTimerPin to make arming exclusive
rust: hrtimer: Restrict expires() to safe contexts
rust: hrtimer: Make HrTimer repr(transparent)

 rust/helpers/time.c                 |   6 ++
 rust/kernel/time/hrtimer.rs         | 135 ++++++++++++++++------------
 rust/kernel/time/hrtimer/arc.rs     | 113 +++++++++++++++++------
 rust/kernel/time/hrtimer/pin.rs     | 107 +++++++++++++++-------
 rust/kernel/time/hrtimer/pin_mut.rs |   2 +-
 rust/kernel/time/hrtimer/tbox.rs    |   2 +-
 6 files changed, 249 insertions(+), 116 deletions(-)

base-commit: 643a7c306b8ce32743d4f94dd700c8588be37e66

Signature · 1 line
2.43.0
#1 lore raw

PATCH 1/4 rust: hrtimer: Introduce HrTimerArc to make arming exclusive

From: FUJITA Tomonori <[email protected]>

A hrtimer callback may change its own expiry time with hrtimer_forward(), which is sound only because __run_hrtimer() dequeues the timer for the duration of the callback. Arming the same timer from another CPU puts it back into the rbtree while the callback runs, so that expiry update lands on a queued timer and the tree ordering goes stale. Arc<T> is Clone and Arc<T>::start() is safe, so safe Rust can arm a timer on any CPU, including while its callback is running.

"No arming while the callback runs" cannot be expressed in the type system, because the callback begins when the timer expires rather than at any point in the Rust program, so use the stronger "no arming while armed" instead. hrtimer_cancel() waits for the handler to return, which makes that the point where the right to arm can be handed back. Pin<Box<T, A>> and Pin<&mut T> already work this way through ownership and exclusive borrow.

Split the right to arm out of Arc<T> into HrTimerArc<T>, a non-clonable wrapper modelled on ListArc that is created from a Pin<UniqueArc<T>> and consumed by start(); the object stays shareable through plain Arc references.

Fixes 3f2a5ba784b8 ("rust: hrtimer: Add HrTimerCallbackContext and ::forward()")
---
 rust/kernel/time/hrtimer.rs     |  33 +++++-----
 rust/kernel/time/hrtimer/arc.rs | 111 +++++++++++++++++++++++++-------
 2 files changed, 103 insertions(+), 41 deletions(-)
rust/kernel/time/hrtimer.rs +16 -17
index 2d7f1131a813..a7587db1d552 100644
@@ -172,12 +172,12 @@
 //! #     sync::{
 //! #         atomic::{ordering, Atomic},
 //! #         completion::Completion,
-//! #         Arc, ArcBorrow,
+//! #         ArcBorrow,
 //! #     },
 //! #     time::{
 //! #         hrtimer::{
-//! #             RelativeMode, HrTimer, HrTimerCallback, HrTimerPointer, HrTimerRestart,
-//! #             HasHrTimer, HrTimerCallbackContext
+//! #             RelativeMode, HrTimer, HrTimerArc, HrTimerCallback, HrTimerPointer,
+//! #             HrTimerRestart, HasHrTimer, HrTimerCallbackContext
 //! #         },
 //! #         Delta, Monotonic,
 //! #     },
@@ -204,7 +204,7 @@
 //! }
 //!
 //! impl HrTimerCallback for ArcIntrusiveHrTimer {
-//!     type Pointer<'a> = Arc<Self>;
+//!     type Pointer<'a> = HrTimerArc<Self>;
 //!
 //!     fn run(
 //!         this: ArcBorrow<'_, Self>,
@@ -229,11 +229,12 @@
 //!     }
 //! }
 //!
-//! let has_timer = Arc::pin_init(ArcIntrusiveHrTimer::new(), GFP_KERNEL)?;
-//! let _handle = has_timer.clone().start(Delta::from_micros(200));
+//! let has_timer = HrTimerArc::pin_init(ArcIntrusiveHrTimer::new(), GFP_KERNEL)?;
+//! let shared = has_timer.clone_arc();
+//! let _handle = has_timer.start(Delta::from_micros(200));
 //!
-//! while has_timer.flag.load(ordering::Relaxed) != 5 {
-//!     has_timer.cond.wait_for_completion();
+//! while shared.flag.load(ordering::Relaxed) != 5 {
+//!     shared.cond.wait_for_completion();
 //! }
 //!
 //! pr_info!("Counted to 5\n");
@@ -589,18 +590,17 @@ pub fn expires(&self) -> HrTimerInstant<T>
 /// `Self` must be [`Sync`] because it is passed to timer callbacks in another
 /// thread of execution (hard or soft interrupt context).
 ///
-/// Starting a timer returns a [`HrTimerHandle`] that can be used to manipulate
-/// the timer. Note that it is OK to call the start function repeatedly, and
-/// that more than one [`HrTimerHandle`] associated with a [`HrTimerPointer`] may
-/// exist. A timer can be manipulated through any of the handles, and a handle
-/// may represent a cancelled timer.
+/// Starting a timer consumes `Self` and returns a [`HrTimerHandle`] that can be
+/// used to manipulate the timer. As a timer in the **started** or **running**
+/// state cannot be started again, at most one [`HrTimerHandle`] for a timer
+/// exists at a time. A handle may represent a cancelled timer.
 pub trait HrTimerPointer: Sync + Sized {
     /// The operational mode associated with this timer.
     ///
     /// This defines how the expiration value is interpreted.
     type TimerMode: HrTimerMode;
 
-    /// A handle representing a started or restarted timer.
+    /// A handle representing a started timer.
     ///
     /// If the timer is running or if the timer callback is executing when the
     /// handle is dropped, the drop method of [`HrTimerHandle`] should not return
@@ -610,8 +610,7 @@ pub trait HrTimerPointer: Sync + Sized {
     /// leak the handle.
     type TimerHandle: HrTimerHandle;
 
-    /// Start the timer with expiry after `expires` time units. If the timer was
-    /// already running, it is restarted with the new expiry time.
+    /// Start the timer with expiry after `expires` time units.
     fn start(self, expires: <Self::TimerMode as HrTimerMode>::Expires) -> Self::TimerHandle;
 }
 
@@ -1110,7 +1109,7 @@ unsafe fn timer_container_of(
 }
 
 mod arc;
-pub use arc::ArcHrTimerHandle;
+pub use arc::{ArcHrTimerHandle, HrTimerArc};
 mod pin;
 pub use pin::PinHrTimerHandle;
 mod pin_mut;
rust/kernel/time/hrtimer/arc.rs +87 -24

146 lines not on this page · raw message

Signature · 1 line
2.43.0
#2 lore raw

PATCH 2/4 rust: hrtimer: Introduce HrTimerPin to make arming exclusive

In reply to FUJITA Tomonori

From: FUJITA Tomonori <[email protected]>

Pin<&T> has the same hole that Arc<T> had: it is Copy and ScopedHrTimerPointer::start_scoped() is safe, so a copy captured by the closure can arm a timer while it is already armed and its callback may be running.

Split the right to arm out of Pin<&T> into HrTimerPin<'a, T>, which is created from a Pin<&'a mut T> and consumed by start_scoped(). The borrow checker supplies the exclusivity here, and the closure keeps reading the object through the shared pinned reference returned by HrTimerPin::as_ref().

All four pointer types now separate sharing an object from arming its timer, so the restart operation no longer exists in the safe API. Drop it from the documentation.

Fixes 3f2a5ba784b8 ("rust: hrtimer: Add HrTimerCallbackContext and ::forward()")
---
 rust/kernel/time/hrtimer.rs     |  50 +++++++--------
 rust/kernel/time/hrtimer/pin.rs | 105 +++++++++++++++++++++++---------
 2 files changed, 98 insertions(+), 57 deletions(-)
rust/kernel/time/hrtimer.rs +23 -27
index a7587db1d552..d94275f2e93f 100644
@@ -9,15 +9,15 @@
 //!
 //! States:
 //!
-//! - Stopped: initialized but not started, or cancelled, or not restarted.
-//! - Started: initialized and started or restarted.
+//! - Stopped: initialized but not started, cancelled, or the callback returned
+//!   `NoRestart`.
+//! - Started: initialized and started, or the callback returned `Restart`.
 //! - Running: executing the callback.
 //!
 //! Operations:
 //!
 //! * Start
 //! * Cancel
-//! * Restart
 //!
 //! Events:
 //!
@@ -42,11 +42,7 @@
 //! --------->|    Stopped      |                 |      Started     +---------->|     Running     |
 //!           |                 |     Cancel      |                  |           |                 |
 //!           |                 |<----------------+                  |           |                 |
-//!           +-----------------+                 +---------------+--+           +-----------------+
-//!                                                     ^         |
-//!                                                     |         |
-//!                                                     +---------+
-//!                                                      Restart
+//!           +-----------------+                 +------------------+           +-----------------+
 //! ```
 //!
 //!
@@ -60,16 +56,13 @@
 //! by the `cancel` operation. A timer that is cancelled enters the **stopped**
 //! state.
 //!
-//! A `cancel` or `restart` operation on a timer in the **running** state takes
-//! effect after the handler has returned and the timer has transitioned
-//! out of the **running** state.
+//! A `cancel` operation on a timer in the **running** state takes effect after
+//! the handler has returned and the timer has transitioned out of the
+//! **running** state.
 //!
-//! A `restart` operation on a timer in the **stopped** state is equivalent to a
-//! `start` operation.
-//!
-//! When a type implements both `HrTimerPointer` and `Clone`, it is possible to
-//! issue the `start` operation while the timer is in the **started** state. In
-//! this case the `start` operation is equivalent to the `restart` operation.
+//! The `start` operation consumes the pointer it is called on, so a timer in the
+//! **started** or **running** state cannot be started again. It has to be
+//! **cancelled** first.
 //!
 //! # Examples
 //!
@@ -253,8 +246,8 @@
 //! #     },
 //! #     time::{
 //! #         hrtimer::{
-//! #             ScopedHrTimerPointer, HrTimer, HrTimerCallback, HrTimerPointer, HrTimerRestart,
-//! #             HasHrTimer, RelativeMode, HrTimerCallbackContext
+//! #             ScopedHrTimerPointer, HrTimer, HrTimerCallback, HrTimerPin, HrTimerPointer,
+//! #             HrTimerRestart, HasHrTimer, RelativeMode, HrTimerCallbackContext
 //! #         },
 //! #         Delta, Monotonic,
 //! #     },
@@ -282,7 +275,7 @@
 //! }
 //!
 //! impl HrTimerCallback for IntrusiveHrTimer {
-//!     type Pointer<'a> = Pin<&'a Self>;
+//!     type Pointer<'a> = HrTimerPin<'a, Self>;
 //!
 //!     fn run(this: Pin<&Self>, _ctx: HrTimerCallbackContext<'_, Self>) -> HrTimerRestart {
 //!         pr_info!("Timer called\n");
@@ -301,9 +294,12 @@
 //! }
 //!
 //! stack_pin_init!( let has_timer = IntrusiveHrTimer::new() );
-//! has_timer.as_ref().start_scoped(Delta::from_micros(200), || {
-//!     while has_timer.flag.load(ordering::Relaxed) != 1 {
-//!         has_timer.cond.wait_for_completion();
+//! let timer_pin = HrTimerPin::new(has_timer);
+//! let shared = timer_pin.as_ref();
+//!
+//! timer_pin.start_scoped(Delta::from_micros(200), || {
+//!     while shared.flag.load(ordering::Relaxed) != 1 {
+//!         shared.cond.wait_for_completion();
 //!     }
 //! });
 //!
@@ -618,7 +614,8 @@ pub trait HrTimerPointer: Sync + Sized {
 /// [`HrTimerHandle`] returned by `start` would be unsound. This is the case for
 /// stack allocated timers.
 ///
-/// Typical implementers are pinned references such as [`Pin<&T>`].
+/// Typical implementers are [`HrTimerPin`] and pinned references such as
+/// [`Pin<&mut T>`].
 ///
 /// # Safety
 ///
@@ -640,8 +637,7 @@ pub unsafe trait UnsafeHrTimerPointer: Sync + Sized {
     /// until the timer is stopped and the callback has completed.
     type TimerHandle: HrTimerHandle;
 
-    /// Start the timer after `expires` time units. If the timer was already
-    /// running, it is restarted at the new expiry time.
+    /// Start the timer after `expires` time units.
     ///
     /// # Safety
     ///
@@ -1111,7 +1107,7 @@ unsafe fn timer_container_of(
 mod arc;
 pub use arc::{ArcHrTimerHandle, HrTimerArc};
 mod pin;
-pub use pin::PinHrTimerHandle;
+pub use pin::{HrTimerPin, PinHrTimerHandle};
 mod pin_mut;
 pub use pin_mut::PinMutHrTimerHandle;
 // `box` is a reserved keyword, so prefix with `t` for timer
rust/kernel/time/hrtimer/pin.rs +75 -30

148 lines not on this page · raw message

Signature · 1 line
2.43.0
#3 lore raw

PATCH 3/4 rust: hrtimer: Restrict expires() to safe contexts

In reply to FUJITA Tomonori

From: FUJITA Tomonori <[email protected]>

HrTimer::expires() previously read node.expires via a volatile load, which can race with C-side updates. Rework the API so it is only callable with exclusive access or from the callback context.

Introduce expires_unchecked() with an explicit safety contract, switch HrTimer::expires() to Pin<&mut Self>, add HrTimerCallbackContext::expires(), and route the read through hrtimer_get_expires() via a Rust helper.

---
 rust/helpers/time.c         |  6 +++++
 rust/kernel/time/hrtimer.rs | 46 ++++++++++++++++++++++++++-----------
 2 files changed, 39 insertions(+), 13 deletions(-)
rust/helpers/time.c +6 -0
index 32f495970493..ef8999621399 100644
@@ -2,6 +2,7 @@
 
 #include <linux/delay.h>
 #include <linux/ktime.h>
+#include <linux/hrtimer.h>
 #include <linux/timekeeping.h>
 
 __rust_helper void rust_helper_fsleep(unsigned long usecs)
@@ -38,3 +39,8 @@ __rust_helper void rust_helper_udelay(unsigned long usec)
 {
 	udelay(usec);
 }
+
+__rust_helper ktime_t rust_helper_hrtimer_get_expires(const struct hrtimer *timer)
+{
+	return hrtimer_get_expires(timer);
+}
rust/kernel/time/hrtimer.rs +33 -13

66 lines not on this page · raw message

Signature · 1 line
2.43.0
#4 lore raw

PATCH 4/4 rust: hrtimer: Make HrTimer repr(transparent)

In reply to FUJITA Tomonori

From: FUJITA Tomonori <[email protected]>

HrTimerCallbackContext acquires a &HrTimer<T> from a NonNull<HrTimer<T>> while a &mut HrTimer<T> can exist at the same time. This is sound only because HrTimer's sole field is Opaque<bindings::hrtimer>, which puts every byte behind an UnsafeCell. Adding a field to HrTimer that is not Opaque would make acquiring that shared reference unsound.

Make HrTimer repr(transparent), which prevents multiple fields, so that such a refactor fails to compile instead of silently introducing unsoundness. This does not guarantee the remaining field stays behind Opaque, but it rules out the likely way of getting there.

repr(transparent) cannot be combined with repr(C), so drop the latter.

---
 rust/kernel/time/hrtimer.rs         | 6 +++++-
 rust/kernel/time/hrtimer/arc.rs     | 2 +-
 rust/kernel/time/hrtimer/pin.rs     | 2 +-
 rust/kernel/time/hrtimer/pin_mut.rs | 2 +-
 rust/kernel/time/hrtimer/tbox.rs    | 2 +-
 5 files changed, 9 insertions(+), 5 deletions(-)
rust/kernel/time/hrtimer.rs +5 -1
index 59e9559e7099..2130dd24cccb 100644
@@ -415,8 +415,12 @@
 /// # Invariants
 ///
 /// * `self.timer` is initialized by `bindings::hrtimer_setup`.
+// `repr(transparent)` is not merely about layout. `HrTimerCallbackContext` acquires a
+// `&HrTimer<T>` while a `&mut HrTimer<T>` may exist, which is sound only because every byte of
+// this type sits inside `Opaque`. Being transparent rejects a second field at compile time,
+// but it does not enforce that the remaining field stays `Opaque`.
 #[pin_data]
-#[repr(C)]
+#[repr(transparent)]
 pub struct HrTimer<T> {
     #[pin]
     timer: Opaque<bindings::hrtimer>,
rust/kernel/time/hrtimer/arc.rs +1 -1
index 2134d12d558c..ce7cff7efe29 100644
@@ -143,7 +143,7 @@ impl<T> RawHrTimerCallback for HrTimerArc<T>
     type CallbackTarget<'a> = ArcBorrow<'a, T>;
 
     unsafe extern "C" fn run(ptr: *mut bindings::hrtimer) -> bindings::hrtimer_restart {
-        // `HrTimer` is `repr(C)`
+        // `HrTimer` is `repr(transparent)`
         let timer_ptr = ptr.cast::<super::HrTimer<T>>();
 
         // SAFETY: By C API contract `ptr` is the pointer we passed when
rust/kernel/time/hrtimer/pin.rs +1 -1
index f44ac07cb722..6a0ac4d7dedf 100644
@@ -128,7 +128,7 @@ impl<'a, T> RawHrTimerCallback for HrTimerPin<'a, T>
     type CallbackTarget<'b> = Pin<&'a T>;
 
     unsafe extern "C" fn run(ptr: *mut bindings::hrtimer) -> bindings::hrtimer_restart {
-        // `HrTimer` is `repr(C)`
+        // `HrTimer` is `repr(transparent)`
         let timer_ptr = ptr.cast::<HrTimer<T>>();
 
         // SAFETY: By the safety requirement of this function, `timer_ptr`
rust/kernel/time/hrtimer/pin_mut.rs +1 -1
index 9d9447d4d57e..65172c9e55e9 100644
@@ -86,7 +86,7 @@ impl<'a, T> RawHrTimerCallback for Pin<&'a mut T>
     type CallbackTarget<'b> = Self;
 
     unsafe extern "C" fn run(ptr: *mut bindings::hrtimer) -> bindings::hrtimer_restart {
-        // `HrTimer` is `repr(C)`
+        // `HrTimer` is `repr(transparent)`
         let timer_ptr = ptr.cast::<HrTimer<T>>();
 
         // SAFETY: By the safety requirement of this function, `timer_ptr`
rust/kernel/time/hrtimer/tbox.rs +1 -1
index aa1ee31a7195..1dd68fcf2bd6 100644
@@ -103,7 +103,7 @@ impl<T, A> RawHrTimerCallback for Pin<Box<T, A>>
     type CallbackTarget<'a> = Pin<&'a mut T>;
 
     unsafe extern "C" fn run(ptr: *mut bindings::hrtimer) -> bindings::hrtimer_restart {
-        // `HrTimer` is `repr(C)`
+        // `HrTimer` is `repr(transparent)`
         let timer_ptr = ptr.cast::<super::HrTimer<T>>();
 
         // SAFETY: By C API contract `ptr` is the pointer we passed when
Signature · 1 line
2.43.0
reviewer
#5 lore raw
In reply to FUJITA Tomonori
Quoted 33 lines · FUJITA Tomonori

From: FUJITA Tomonori <[email protected]>

This series started from the review of patches 3 and 4 [1]: a hrtimer can be armed from any CPU at any time, including while its callback runs, so restricting HrTimer::expires() to the callback context is not by itself enough to remove the race.

It turned out that expires() is not the only problem. A callback may also change its expiry time with hrtimer_forward(), which is sound only because __run_hrtimer() dequeues the timer for the duration of the callback. Arming the same timer from another CPU puts it back into the rbtree while the callback runs, so hrtimer_forward() then changes the expiry of a timer that is queued, without the base lock and without re-checking the ordering, which leaves the tree unsorted.

Two of the four pointer types cannot construct that situation. Starting a Pin<Box<T, A>> moves the box into the handle, and starting a Pin<&mut T> consumes the exclusive borrow, so in both cases nothing is left to arm the timer with. Arc<T> is Clone and Pin<&T> is Copy, and both of their start functions are reachable from safe code, so safe Rust could arm a timer whose callback was running.

"No arming while the callback runs" cannot be expressed in the type system, because the callback begins when the timer expires rather than at any point in the Rust program, so patches 1 and 2 use the stronger "no arming while armed" instead. hrtimer_cancel() waits for the handler to return, which makes that the point where the right to arm can be handed back. The right to arm is split out of Arc<T> into HrTimerArc<T> and out of Pin<&T> into HrTimerPin<'a, T>, both non-clonable and consumed by start, modelled on ListArc; the object itself stays shareable through plain Arc references and shared pinned references respectively.

Quoted 4 lines

Patches 3 and 4 are the previously posted expires() and repr(transparent) patches, unchanged. With patches 1 and 2 in place, the callback context has no concurrent writer of node.expires. So HrTimerCallbackContext::expires() is sound.

I am thinking about this and I wonder about a different approach: the only reason that we're having this issue, is that expires() call and forward/forward_now is executed outside the protection of the base lock.

The fix is easy -- to ensure that they are executed with the base lock held. The callback wants either:
* Do not restart the timer
* Call hrtimer_forward[_now] and restart the timer

So, if we change the order from

    unlock base
    restart = fn(timer)
    lock base
    if restart {
        queue
    }

to

    get expires
    unlock base
    restart = fn(timer, expires)
    lock base
    match restart {
        Restart(now, interval) => {
            hrtimer_forward(timer, now, interval);
            queue
        }
        NoRestart => (),
    }

then we completely eradicate this issue.

Alternatively, we can add another spinlock to protect expires from race condition from within callback and concurrent restart -- that is what perf core
does: perf_mux_hrtimer_handler and perf_mux_hrtimer_restart uses the same hrtimer_lock to prevent race.

But further complicating the type system to prevent concurrent restart sounds like a bad approach to me.

Best,
Gary

Quoted 19 lines

[1]: https://lore.kernel.org/rust-for-linux/[email protected]/

FUJITA Tomonori (4):
rust: hrtimer: Introduce HrTimerArc to make arming exclusive
rust: hrtimer: Introduce HrTimerPin to make arming exclusive
rust: hrtimer: Restrict expires() to safe contexts
rust: hrtimer: Make HrTimer repr(transparent)

 rust/helpers/time.c                 |   6 ++
 rust/kernel/time/hrtimer.rs         | 135 ++++++++++++++++------------
 rust/kernel/time/hrtimer/arc.rs     | 113 +++++++++++++++++------
 rust/kernel/time/hrtimer/pin.rs     | 107 +++++++++++++++-------
 rust/kernel/time/hrtimer/pin_mut.rs |   2 +-
 rust/kernel/time/hrtimer/tbox.rs    |   2 +-
 6 files changed, 249 insertions(+), 116 deletions(-)

base-commit: 643a7c306b8ce32743d4f94dd700c8588be37e66

#6 lore raw
Quoted 71 lines · On Thu, 13 Aug 2026 15:16:38 +0100 "Gary Guo" <[email protected]> wrote:
Quoted 33 lines · FUJITA Tomonori

From: FUJITA Tomonori <[email protected]>

This series started from the review of patches 3 and 4 [1]: a hrtimer can be armed from any CPU at any time, including while its callback runs, so restricting HrTimer::expires() to the callback context is not by itself enough to remove the race.

It turned out that expires() is not the only problem. A callback may also change its expiry time with hrtimer_forward(), which is sound only because __run_hrtimer() dequeues the timer for the duration of the callback. Arming the same timer from another CPU puts it back into the rbtree while the callback runs, so hrtimer_forward() then changes the expiry of a timer that is queued, without the base lock and without re-checking the ordering, which leaves the tree unsorted.

Two of the four pointer types cannot construct that situation. Starting a Pin<Box<T, A>> moves the box into the handle, and starting a Pin<&mut T> consumes the exclusive borrow, so in both cases nothing is left to arm the timer with. Arc<T> is Clone and Pin<&T> is Copy, and both of their start functions are reachable from safe code, so safe Rust could arm a timer whose callback was running.

"No arming while the callback runs" cannot be expressed in the type system, because the callback begins when the timer expires rather than at any point in the Rust program, so patches 1 and 2 use the stronger "no arming while armed" instead. hrtimer_cancel() waits for the handler to return, which makes that the point where the right to arm can be handed back. The right to arm is split out of Arc<T> into HrTimerArc<T> and out of Pin<&T> into HrTimerPin<'a, T>, both non-clonable and consumed by start, modelled on ListArc; the object itself stays shareable through plain Arc references and shared pinned references respectively.

Quoted 4 lines

Patches 3 and 4 are the previously posted expires() and repr(transparent) patches, unchanged. With patches 1 and 2 in place, the callback context has no concurrent writer of node.expires. So HrTimerCallbackContext::expires() is sound.

I am thinking about this and I wonder about a different approach: the only reason that we're having this issue, is that expires() call and forward/forward_now is executed outside the protection of the base lock.

The fix is easy -- to ensure that they are executed with the base lock held. The callback wants either:
* Do not restart the timer
* Call hrtimer_forward[_now] and restart the timer

So, if we change the order from

    unlock base
    restart = fn(timer)
    lock base
    if restart {
        queue
    }

to

    get expires
    unlock base
    restart = fn(timer, expires)
    lock base
    match restart {
        Restart(now, interval) => {
            hrtimer_forward(timer, now, interval);
            queue
        }
        NoRestart => (),
    }
Quoted 1 line

then we completely eradicate this issue.

If I understood the proposal correctly: hrtimer_forward[_now]() would no longer be called by drivers at all. It becomes internal to the core and runs with the base lock held, and every hrtimer callback in the tree is updated to take the expiry as an argument and return the interval, with the ones that use the overrun computing it from what they were passed.

That could remove the need for the rule on the Rust side.

Anna-Maria, Frederic, Thomas: does this direction look reasonable to you?

Quoted 4 lines

Alternatively, we can add another spinlock to protect expires from race condition from within callback and concurrent restart -- that is what perf core
does: perf_mux_hrtimer_handler and perf_mux_hrtimer_restart uses the same hrtimer_lock to prevent race.

Right, with the lock and cpc->hrtimer_active flag together, perf implements the same "do not arm while armed" rule that these patches implement in the type system.

Quoted 2 lines

But further complicating the type system to prevent concurrent restart sounds like a bad approach to me.

My intent is the opposite: I think this makes the design simpler.

All four implementations of start() already take self by value. For Pin<Box<T, A>> and Pin<&mut T> that means what it says -- the box is moved into the handle, the exclusive borrow is consumed -- so "no arming while armed" is already the design there. For Arc<T> and Pin<&T> the same signature meant nothing, because Clone and Copy let you build another pointer and call start() again.

So the contract depended on which pointer type you picked, and the module documentation had to spell that out: "When a type implements both HrTimerPointer and Clone, it is possible to issue the start operation while the timer is in the started state." After the series there is one rule for all four types, and that paragraph is gone together with the restart operation it described.

reviewer
#7 lore raw
Quoted 35 lines · FUJITA Tomonori
Quoted 32 lines · On Thu, 13 Aug 2026 15:16:38 +0100 "Gary Guo" <[email protected]> wrote:

I am thinking about this and I wonder about a different approach: the only reason that we're having this issue, is that expires() call and forward/forward_now is executed outside the protection of the base lock.

The fix is easy -- to ensure that they are executed with the base lock held. The callback wants either:
* Do not restart the timer
* Call hrtimer_forward[_now] and restart the timer

So, if we change the order from

    unlock base
    restart = fn(timer)
    lock base
    if restart {
        queue
    }

to

    get expires
    unlock base
    restart = fn(timer, expires)
    lock base
    match restart {
        Restart(now, interval) => {
            hrtimer_forward(timer, now, interval);
            queue
        }
        NoRestart => (),
    }
Quoted 7 lines
Quoted 1 line

then we completely eradicate this issue.

If I understood the proposal correctly: hrtimer_forward[_now]() would no longer be called by drivers at all. It becomes internal to the core and runs with the base lock held, and every hrtimer callback in the tree is updated to take the expiry as an argument and return the interval, with the ones that use the overrun computing it from what they were passed.

Right, that the idea. I think patching all hrtimer callback is probably a bit excessive, but one way would be add a mode where cpu_base->lock is not unlocked, and the Rust hrtimer abstraction would read the expiry, unlock it, run the callback and re-lock the base lock.

Quoted 28 lines

That could remove the need for the rule on the Rust side.

Anna-Maria, Frederic, Thomas: does this direction look reasonable to you?

Quoted 4 lines

Alternatively, we can add another spinlock to protect expires from race condition from within callback and concurrent restart -- that is what perf core
does: perf_mux_hrtimer_handler and perf_mux_hrtimer_restart uses the same hrtimer_lock to prevent race.

Right, with the lock and cpc->hrtimer_active flag together, perf implements the same "do not arm while armed" rule that these patches implement in the type system.

Quoted 2 lines

But further complicating the type system to prevent concurrent restart sounds like a bad approach to me.

My intent is the opposite: I think this makes the design simpler.

All four implementations of start() already take self by value. For Pin<Box<T, A>> and Pin<&mut T> that means what it says -- the box is moved into the handle, the exclusive borrow is consumed -- so "no arming while armed" is already the design there. For Arc<T> and Pin<&T> the same signature meant nothing, because Clone and Copy let you build another pointer and call start() again.

Quoted 6 lines

So the contract depended on which pointer type you picked, and the module documentation had to spell that out: "When a type implements both HrTimerPointer and Clone, it is possible to issue the start operation while the timer is in the started state." After the series there is one rule for all four types, and that paragraph is gone together with the restart operation it described.

Let's ignore the implementation detail of all various Rust pointers. It is something that I plan to overhaul and doesn't matter to the core issue here.

The change you're making is to remove the ability to concurrently start a timer in Rust. So if you have a timer might be running, you'd need to first cancel it before you can arm it again.

I do think it is conceptually cleaner -- however given this is explicitly added in
https://lore.kernel.org/all/[email protected]/
and the pattern is what perf core uses; so I wouldn't just dismiss the existence of this pattern. Perhaps cancelling before restarting is considered too expensive and has to be avoided?

Best,
Gary

#8 lore raw
Quoted 26 lines · On Fri, 14 Aug 2026 01:54:25 +0100 "Gary Guo" <[email protected]> wrote:
Quoted 12 lines
Quoted 2 lines

But further complicating the type system to prevent concurrent restart sounds like a bad approach to me.

My intent is the opposite: I think this makes the design simpler.

All four implementations of start() already take self by value. For Pin<Box<T, A>> and Pin<&mut T> that means what it says -- the box is moved into the handle, the exclusive borrow is consumed -- so "no arming while armed" is already the design there. For Arc<T> and Pin<&T> the same signature meant nothing, because Clone and Copy let you build another pointer and call start() again.

Quoted 6 lines

So the contract depended on which pointer type you picked, and the module documentation had to spell that out: "When a type implements both HrTimerPointer and Clone, it is possible to issue the start operation while the timer is in the started state." After the series there is one rule for all four types, and that paragraph is gone together with the restart operation it described.

Let's ignore the implementation detail of all various Rust pointers. It is something that I plan to overhaul and doesn't matter to the core issue here.

The change you're making is to remove the ability to concurrently start a timer in Rust. So if you have a timer might be running, you'd need to first cancel it before you can arm it again.

Quoted 6 lines

I do think it is conceptually cleaner -- however given this is explicitly added in
https://lore.kernel.org/all/[email protected]/
and the pattern is what perf core uses; so I wouldn't just dismiss the existence of this pattern. Perhaps cancelling before restarting is considered too expensive and has to be avoided?

The pattern perf core uses is "no arming while armed".

While perf_mux_hrtimer_handler() returns HRTIMER_RESTART -- while the timer is active -- perf_mux_hrtimer_restart() does nothing. Only once the handler has cleared cpc->hrtimer_active and returned HRTIMER_NORESTART does perf_mux_hrtimer_restart() arm it again.

That flag was added precisely to implement "no arming while armed", in 4cfafd3082af ("sched,perf: Fix periodic timers"):

We do not want to race such that the handler has already decided to stop, but the (external) restart sees the timer still active and we end up with a 'lost' timer.

The problem with the current code is that the re-start can come before the callback does the forward, at which point the forward from the callback will WARN about forwarding an enqueued timer.

With cpc->hrtimer_active in place, neither of the two conditions that 5de2755c8c8b touches is reachable in perf's usage.

v1 is missing the ability to restart a stopped timer: once the callback has returned NoRestart, the handle owns the right to arm and never gives it back. I'll add it in v2, including a non-blocking variant built on hrtimer_try_to_cancel(), so that a caller which cannot sleep can re-arm the way perf does. That makes perf's model expressible in the Rust abstraction.

reviewer
#9 lore raw
Quoted 30 lines · FUJITA Tomonori
Quoted 7 lines · On Fri, 14 Aug 2026 01:54:25 +0100 "Gary Guo" <[email protected]> wrote:

Let's ignore the implementation detail of all various Rust pointers. It is something that I plan to overhaul and doesn't matter to the core issue here.

The change you're making is to remove the ability to concurrently start a timer in Rust. So if you have a timer might be running, you'd need to first cancel it before you can arm it again.

Quoted 6 lines

I do think it is conceptually cleaner -- however given this is explicitly added in
https://lore.kernel.org/all/[email protected]/
and the pattern is what perf core uses; so I wouldn't just dismiss the existence of this pattern. Perhaps cancelling before restarting is considered too expensive and has to be avoided?

The pattern perf core uses is "no arming while armed".

While perf_mux_hrtimer_handler() returns HRTIMER_RESTART -- while the timer is active -- perf_mux_hrtimer_restart() does nothing. Only once the handler has cleared cpc->hrtimer_active and returned HRTIMER_NORESTART does perf_mux_hrtimer_restart() arm it again.

That flag was added precisely to implement "no arming while armed", in 4cfafd3082af ("sched,perf: Fix periodic timers"):

We do not want to race such that the handler has already decided to stop, but the (external) restart sees the timer still active and we end up with a 'lost' timer.

Quoted 7 lines

The problem with the current code is that the re-start can come before the callback does the forward, at which point the forward from the callback will WARN about forwarding an enqueued timer.

With cpc->hrtimer_active in place, neither of the two conditions that 5de2755c8c8b touches is reachable in perf's usage.

So are we okay saying that concurrent restart is problematic because apparently it cannot be used correctly (because you don't have a way to synchronize it)? Perhaps we should just revert 5de2755c8c8b or at least do

    if (restart != HRTIMER_NORESTART) {
        WARN_ON(timer->state != HRTIMER_STATE_CALLBACK);
        if (!(timer->state & HRTIMER_STATE_ENQUEUED))
            enqueue_hrtimer(timer, base);
    }

?

That said, the perf core's pattern is still different from the API that you're designing. When perf_mux_hrtimer_handler unlocks cpc->hrtimer_lock at that point perf_mux_hrtimer_restart can already kick in and restart the timer. From hrtimer core's perspective, it is starting a timer that is still have running callback -- but that callback shall only return NORESTART.

Best,
Gary

Quoted 7 lines

v1 is missing the ability to restart a stopped timer: once the callback has returned NoRestart, the handle owns the right to arm and never gives it back. I'll add it in v2, including a non-blocking variant built on hrtimer_try_to_cancel(), so that a caller which cannot sleep can re-arm the way perf does. That makes perf's model expressible in the Rust abstraction.

#10 lore raw
Quoted 43 lines · On Fri, 14 Aug 2026 15:24:10 +0100 "Gary Guo" <[email protected]> wrote:
Quoted 30 lines · FUJITA Tomonori
Quoted 7 lines · On Fri, 14 Aug 2026 01:54:25 +0100 "Gary Guo" <[email protected]> wrote:

Let's ignore the implementation detail of all various Rust pointers. It is something that I plan to overhaul and doesn't matter to the core issue here.

The change you're making is to remove the ability to concurrently start a timer in Rust. So if you have a timer might be running, you'd need to first cancel it before you can arm it again.

Quoted 6 lines

I do think it is conceptually cleaner -- however given this is explicitly added in
https://lore.kernel.org/all/[email protected]/
and the pattern is what perf core uses; so I wouldn't just dismiss the existence of this pattern. Perhaps cancelling before restarting is considered too expensive and has to be avoided?

The pattern perf core uses is "no arming while armed".

While perf_mux_hrtimer_handler() returns HRTIMER_RESTART -- while the timer is active -- perf_mux_hrtimer_restart() does nothing. Only once the handler has cleared cpc->hrtimer_active and returned HRTIMER_NORESTART does perf_mux_hrtimer_restart() arm it again.

That flag was added precisely to implement "no arming while armed", in 4cfafd3082af ("sched,perf: Fix periodic timers"):

We do not want to race such that the handler has already decided to stop, but the (external) restart sees the timer still active and we end up with a 'lost' timer.

Quoted 7 lines

The problem with the current code is that the re-start can come before the callback does the forward, at which point the forward from the callback will WARN about forwarding an enqueued timer.

With cpc->hrtimer_active in place, neither of the two conditions that 5de2755c8c8b touches is reachable in perf's usage.

So are we okay saying that concurrent restart is problematic because apparently it cannot be used correctly (because you don't have a way to synchronize it)? Perhaps we should just revert 5de2755c8c8b or at least do

Quoted 7 lines
    if (restart != HRTIMER_NORESTART) {
        WARN_ON(timer->state != HRTIMER_STATE_CALLBACK);
        if (!(timer->state & HRTIMER_STATE_ENQUEUED))
            enqueue_hrtimer(timer, base);
    }

?

HRTIMER_STATE_CALLBACK is gone; the state machine was reworked, so that WARN is now WARN_ON(timer->is_queued): the timer was armed again while the callback was running. And the

if (!(timer->state & HRTIMER_STATE_ENQUEUED))

part is what the tree does already; __run_hrtimer() checks !timer->is_queued. So against current mainline the proposal is just the WARN.

That is only a diagnostic and does not change behaviour, so I'm not sure whether it is worth it.

Quoted 5 lines

That said, the perf core's pattern is still different from the API that you're designing. When perf_mux_hrtimer_handler unlocks cpc->hrtimer_lock at that point perf_mux_hrtimer_restart can already kick in and restart the timer. From hrtimer core's perspective, it is starting a timer that is still have running callback -- but that callback shall only return NORESTART.

Yes. The difference is that perf can restart a timer that is stopped, and also one that is still in its callback, before the callback returns HRTIMER_NORESTART.

The second case is a short window, but if it matters, I think that there are two ways to cover it. One is an unsafe restart, where the user of the hrtimer synchronizes with a spinlock and a flag as perf does. The other is to add a type that does that synchronization.

reviewer
#11 lore raw
In reply to FUJITA Tomonori
Quoted 58 lines · FUJITA Tomonori
Quoted 26 lines · On Fri, 14 Aug 2026 01:54:25 +0100 "Gary Guo" <[email protected]> wrote:
Quoted 12 lines
Quoted 2 lines

But further complicating the type system to prevent concurrent restart sounds like a bad approach to me.

My intent is the opposite: I think this makes the design simpler.

All four implementations of start() already take self by value. For Pin<Box<T, A>> and Pin<&mut T> that means what it says -- the box is moved into the handle, the exclusive borrow is consumed -- so "no arming while armed" is already the design there. For Arc<T> and Pin<&T> the same signature meant nothing, because Clone and Copy let you build another pointer and call start() again.

Quoted 6 lines

So the contract depended on which pointer type you picked, and the module documentation had to spell that out: "When a type implements both HrTimerPointer and Clone, it is possible to issue the start operation while the timer is in the started state." After the series there is one rule for all four types, and that paragraph is gone together with the restart operation it described.

Let's ignore the implementation detail of all various Rust pointers. It is something that I plan to overhaul and doesn't matter to the core issue here.

The change you're making is to remove the ability to concurrently start a timer in Rust. So if you have a timer might be running, you'd need to first cancel it before you can arm it again.

Quoted 6 lines

I do think it is conceptually cleaner -- however given this is explicitly added in
https://lore.kernel.org/all/[email protected]/
and the pattern is what perf core uses; so I wouldn't just dismiss the existence of this pattern. Perhaps cancelling before restarting is considered too expensive and has to be avoided?

The pattern perf core uses is "no arming while armed".

While perf_mux_hrtimer_handler() returns HRTIMER_RESTART -- while the timer is active -- perf_mux_hrtimer_restart() does nothing. Only once the handler has cleared cpc->hrtimer_active and returned HRTIMER_NORESTART does perf_mux_hrtimer_restart() arm it again.

That flag was added precisely to implement "no arming while armed", in 4cfafd3082af ("sched,perf: Fix periodic timers"):

We do not want to race such that the handler has already decided to stop, but the (external) restart sees the timer still active and we end up with a 'lost' timer.

The problem with the current code is that the re-start can come before the callback does the forward, at which point the forward from the callback will WARN about forwarding an enqueued timer.

With cpc->hrtimer_active in place, neither of the two conditions that 5de2755c8c8b touches is reachable in perf's usage.

Quoted 6 lines

v1 is missing the ability to restart a stopped timer: once the callback has returned NoRestart, the handle owns the right to arm and never gives it back. I'll add it in v2, including a non-blocking variant built on hrtimer_try_to_cancel(), so that a caller which cannot sleep can re-arm the way perf does. That makes perf's model expressible in the Rust abstraction.

I should also add that cancel and then restart can be problematic from a lock order POV.

If there is a shared lock between the handler and the restart like the perf core's case, this lock cannot be held when cancelling the hrtimer. Otherwise you can have a deadlock by having

    restart (with lock held) --wait--> handler --wait--> lock

which can be quite subtle.

Of course, there is an argument that by cancelling and restart, locks as seen in perf core's use case wouldn't be necessary as the cancellation becomes the synnchronization mechanism.

Best,
Gary

#12 lore raw
Quoted 73 lines · On Mon, 17 Aug 2026 16:40:56 +0100 "Gary Guo" <[email protected]> wrote:
Quoted 58 lines · FUJITA Tomonori
Quoted 26 lines · On Fri, 14 Aug 2026 01:54:25 +0100 "Gary Guo" <[email protected]> wrote:
Quoted 12 lines
Quoted 2 lines

But further complicating the type system to prevent concurrent restart sounds like a bad approach to me.

My intent is the opposite: I think this makes the design simpler.

All four implementations of start() already take self by value. For Pin<Box<T, A>> and Pin<&mut T> that means what it says -- the box is moved into the handle, the exclusive borrow is consumed -- so "no arming while armed" is already the design there. For Arc<T> and Pin<&T> the same signature meant nothing, because Clone and Copy let you build another pointer and call start() again.

Quoted 6 lines

So the contract depended on which pointer type you picked, and the module documentation had to spell that out: "When a type implements both HrTimerPointer and Clone, it is possible to issue the start operation while the timer is in the started state." After the series there is one rule for all four types, and that paragraph is gone together with the restart operation it described.

Let's ignore the implementation detail of all various Rust pointers. It is something that I plan to overhaul and doesn't matter to the core issue here.

The change you're making is to remove the ability to concurrently start a timer in Rust. So if you have a timer might be running, you'd need to first cancel it before you can arm it again.

Quoted 6 lines

I do think it is conceptually cleaner -- however given this is explicitly added in
https://lore.kernel.org/all/[email protected]/
and the pattern is what perf core uses; so I wouldn't just dismiss the existence of this pattern. Perhaps cancelling before restarting is considered too expensive and has to be avoided?

The pattern perf core uses is "no arming while armed".

While perf_mux_hrtimer_handler() returns HRTIMER_RESTART -- while the timer is active -- perf_mux_hrtimer_restart() does nothing. Only once the handler has cleared cpc->hrtimer_active and returned HRTIMER_NORESTART does perf_mux_hrtimer_restart() arm it again.

That flag was added precisely to implement "no arming while armed", in 4cfafd3082af ("sched,perf: Fix periodic timers"):

We do not want to race such that the handler has already decided to stop, but the (external) restart sees the timer still active and we end up with a 'lost' timer.

The problem with the current code is that the re-start can come before the callback does the forward, at which point the forward from the callback will WARN about forwarding an enqueued timer.

With cpc->hrtimer_active in place, neither of the two conditions that 5de2755c8c8b touches is reachable in perf's usage.

Quoted 6 lines

v1 is missing the ability to restart a stopped timer: once the callback has returned NoRestart, the handle owns the right to arm and never gives it back. I'll add it in v2, including a non-blocking variant built on hrtimer_try_to_cancel(), so that a caller which cannot sleep can re-arm the way perf does. That makes perf's model expressible in the Rust abstraction.

I should also add that cancel and then restart can be problematic from a lock order POV.

If there is a shared lock between the handler and the restart like the perf core's case, this lock cannot be held when cancelling the hrtimer. Otherwise you can have a deadlock by having

Quoted 7 lines
    restart (with lock held) --wait--> handler --wait--> lock

which can be quite subtle.

Of course, there is an argument that by cancelling and restart, locks as seen in perf core's use case wouldn't be necessary as the cancellation becomes the synnchronization mechanism.

Right, hrtimer_try_to_cancel() does not have that restriction but hrtimer_cancel() must not be called while holding a lock that the callback takes.

maintainer
#13 lore raw
In reply to Gary Guo
Quoted 71 lines · Gary Guo
Quoted 33 lines · FUJITA Tomonori

From: FUJITA Tomonori <[email protected]>

This series started from the review of patches 3 and 4 [1]: a hrtimer can be armed from any CPU at any time, including while its callback runs, so restricting HrTimer::expires() to the callback context is not by itself enough to remove the race.

It turned out that expires() is not the only problem. A callback may also change its expiry time with hrtimer_forward(), which is sound only because __run_hrtimer() dequeues the timer for the duration of the callback. Arming the same timer from another CPU puts it back into the rbtree while the callback runs, so hrtimer_forward() then changes the expiry of a timer that is queued, without the base lock and without re-checking the ordering, which leaves the tree unsorted.

Two of the four pointer types cannot construct that situation. Starting a Pin<Box<T, A>> moves the box into the handle, and starting a Pin<&mut T> consumes the exclusive borrow, so in both cases nothing is left to arm the timer with. Arc<T> is Clone and Pin<&T> is Copy, and both of their start functions are reachable from safe code, so safe Rust could arm a timer whose callback was running.

"No arming while the callback runs" cannot be expressed in the type system, because the callback begins when the timer expires rather than at any point in the Rust program, so patches 1 and 2 use the stronger "no arming while armed" instead. hrtimer_cancel() waits for the handler to return, which makes that the point where the right to arm can be handed back. The right to arm is split out of Arc<T> into HrTimerArc<T> and out of Pin<&T> into HrTimerPin<'a, T>, both non-clonable and consumed by start, modelled on ListArc; the object itself stays shareable through plain Arc references and shared pinned references respectively.

Quoted 4 lines

Patches 3 and 4 are the previously posted expires() and repr(transparent) patches, unchanged. With patches 1 and 2 in place, the callback context has no concurrent writer of node.expires. So HrTimerCallbackContext::expires() is sound.

I am thinking about this and I wonder about a different approach: the only reason that we're having this issue, is that expires() call and forward/forward_now is executed outside the protection of the base lock.

The fix is easy -- to ensure that they are executed with the base lock held. The callback wants either:
* Do not restart the timer
* Call hrtimer_forward[_now] and restart the timer

So, if we change the order from

    unlock base
    restart = fn(timer)
    lock base
    if restart {
        queue
    }

to

    get expires
    unlock base
    restart = fn(timer, expires)
    lock base
    match restart {
        Restart(now, interval) => {
            hrtimer_forward(timer, now, interval);
            queue
        }
        NoRestart => (),
    }
Quoted 6 lines

then we completely eradicate this issue.

Alternatively, we can add another spinlock to protect expires from race condition from within callback and concurrent restart -- that is what perf core
does: perf_mux_hrtimer_handler and perf_mux_hrtimer_restart uses the same hrtimer_lock to prevent race.

With this solution we would have to restrict calls to forward and expires. Maybe that would be OK, but it would be restricting the API further.

As I understand the problem space, we have (on Rust side):

- start and forward may race. forward is callable on exclusive reference to HrTimer or in callback context, but otherwise lacks synchronization. start is serialized on the base lock but is callable at any time.
- start and expires may race because start writes the expiration and expires reads it. The latter has no synchronization and is callable on shared reference to HrTimer.
- forward and expires may race because HrTimer::expires takes a shared reference and is callable at any time concurrently.

I think the solution suggested by Tomo is OK, but we could also add synchronization to start, forward and expires on the rust side. Would that not solve the problem for us?

This way we can still run the handler without lock. Only if we call forward or read the expiry in the handler would we take the lock.

This would allow the API as originally described on the rust side.

Best regards,
Andreas Hindborg

#14 lore raw
Quoted 96 lines · On Mon, 17 Aug 2026 17:26:41 +0200 Andreas Hindborg <[email protected]>…
Quoted 71 lines · Gary Guo
Quoted 33 lines · FUJITA Tomonori

From: FUJITA Tomonori <[email protected]>

This series started from the review of patches 3 and 4 [1]: a hrtimer can be armed from any CPU at any time, including while its callback runs, so restricting HrTimer::expires() to the callback context is not by itself enough to remove the race.

It turned out that expires() is not the only problem. A callback may also change its expiry time with hrtimer_forward(), which is sound only because __run_hrtimer() dequeues the timer for the duration of the callback. Arming the same timer from another CPU puts it back into the rbtree while the callback runs, so hrtimer_forward() then changes the expiry of a timer that is queued, without the base lock and without re-checking the ordering, which leaves the tree unsorted.

Two of the four pointer types cannot construct that situation. Starting a Pin<Box<T, A>> moves the box into the handle, and starting a Pin<&mut T> consumes the exclusive borrow, so in both cases nothing is left to arm the timer with. Arc<T> is Clone and Pin<&T> is Copy, and both of their start functions are reachable from safe code, so safe Rust could arm a timer whose callback was running.

"No arming while the callback runs" cannot be expressed in the type system, because the callback begins when the timer expires rather than at any point in the Rust program, so patches 1 and 2 use the stronger "no arming while armed" instead. hrtimer_cancel() waits for the handler to return, which makes that the point where the right to arm can be handed back. The right to arm is split out of Arc<T> into HrTimerArc<T> and out of Pin<&T> into HrTimerPin<'a, T>, both non-clonable and consumed by start, modelled on ListArc; the object itself stays shareable through plain Arc references and shared pinned references respectively.

Quoted 4 lines

Patches 3 and 4 are the previously posted expires() and repr(transparent) patches, unchanged. With patches 1 and 2 in place, the callback context has no concurrent writer of node.expires. So HrTimerCallbackContext::expires() is sound.

I am thinking about this and I wonder about a different approach: the only reason that we're having this issue, is that expires() call and forward/forward_now is executed outside the protection of the base lock.

The fix is easy -- to ensure that they are executed with the base lock held. The callback wants either:
* Do not restart the timer
* Call hrtimer_forward[_now] and restart the timer

So, if we change the order from

    unlock base
    restart = fn(timer)
    lock base
    if restart {
        queue
    }

to

    get expires
    unlock base
    restart = fn(timer, expires)
    lock base
    match restart {
        Restart(now, interval) => {
            hrtimer_forward(timer, now, interval);
            queue
        }
        NoRestart => (),
    }
Quoted 6 lines

then we completely eradicate this issue.

Alternatively, we can add another spinlock to protect expires from race condition from within callback and concurrent restart -- that is what perf core
does: perf_mux_hrtimer_handler and perf_mux_hrtimer_restart uses the same hrtimer_lock to prevent race.

With this solution we would have to restrict calls to forward and expires. Maybe that would be OK, but it would be restricting the API further.

As I understand the problem space, we have (on Rust side):

- start and forward may race. forward is callable on exclusive reference to HrTimer or in callback context, but otherwise lacks synchronization. start is serialized on the base lock but is callable at any time.
- start and expires may race because start writes the expiration and expires reads it. The latter has no synchronization and is callable on shared reference to HrTimer.
- forward and expires may race because HrTimer::expires takes a shared reference and is callable at any time concurrently.

Quoted 8 lines

I think the solution suggested by Tomo is OK, but we could also add synchronization to start, forward and expires on the rust side. Would that not solve the problem for us?

This way we can still run the handler without lock. Only if we call forward or read the expiry in the handler would we take the lock.

This would allow the API as originally described on the rust side.

That would work, but I think it needs more than the lock. The lock makes start and forward safe against each other, but one of them still loses. If start runs first, hrtimer_forward() returns 0 and does nothing, so the overrun it would have returned is lost. Some callers use that return value.

perf and CFS bandwidth have a flag as well as a lock. The flag is "do not arm while armed", which is the same rule the types enforce here. rtc and the softlockup watchdog look like they cancel first and then start instead. None of them arms a timer that is active, so I would rather the abstraction did not allow it either. Does that seem reasonable?

maintainer
#15 lore raw
Quoted 106 lines · FUJITA Tomonori
Quoted 103 lines · On Mon, 17 Aug 2026 17:26:41 +0200 Andreas Hindborg <[email protected]>…
Quoted 71 lines · Gary Guo
Quoted 33 lines · FUJITA Tomonori

From: FUJITA Tomonori <[email protected]>

This series started from the review of patches 3 and 4 [1]: a hrtimer can be armed from any CPU at any time, including while its callback runs, so restricting HrTimer::expires() to the callback context is not by itself enough to remove the race.

It turned out that expires() is not the only problem. A callback may also change its expiry time with hrtimer_forward(), which is sound only because __run_hrtimer() dequeues the timer for the duration of the callback. Arming the same timer from another CPU puts it back into the rbtree while the callback runs, so hrtimer_forward() then changes the expiry of a timer that is queued, without the base lock and without re-checking the ordering, which leaves the tree unsorted.

Two of the four pointer types cannot construct that situation. Starting a Pin<Box<T, A>> moves the box into the handle, and starting a Pin<&mut T> consumes the exclusive borrow, so in both cases nothing is left to arm the timer with. Arc<T> is Clone and Pin<&T> is Copy, and both of their start functions are reachable from safe code, so safe Rust could arm a timer whose callback was running.

"No arming while the callback runs" cannot be expressed in the type system, because the callback begins when the timer expires rather than at any point in the Rust program, so patches 1 and 2 use the stronger "no arming while armed" instead. hrtimer_cancel() waits for the handler to return, which makes that the point where the right to arm can be handed back. The right to arm is split out of Arc<T> into HrTimerArc<T> and out of Pin<&T> into HrTimerPin<'a, T>, both non-clonable and consumed by start, modelled on ListArc; the object itself stays shareable through plain Arc references and shared pinned references respectively.

Quoted 4 lines

Patches 3 and 4 are the previously posted expires() and repr(transparent) patches, unchanged. With patches 1 and 2 in place, the callback context has no concurrent writer of node.expires. So HrTimerCallbackContext::expires() is sound.

I am thinking about this and I wonder about a different approach: the only reason that we're having this issue, is that expires() call and forward/forward_now is executed outside the protection of the base lock.

The fix is easy -- to ensure that they are executed with the base lock held. The callback wants either:
* Do not restart the timer
* Call hrtimer_forward[_now] and restart the timer

So, if we change the order from

    unlock base
    restart = fn(timer)
    lock base
    if restart {
        queue
    }

to

    get expires
    unlock base
    restart = fn(timer, expires)
    lock base
    match restart {
        Restart(now, interval) => {
            hrtimer_forward(timer, now, interval);
            queue
        }
        NoRestart => (),
    }
Quoted 6 lines

then we completely eradicate this issue.

Alternatively, we can add another spinlock to protect expires from race condition from within callback and concurrent restart -- that is what perf core
does: perf_mux_hrtimer_handler and perf_mux_hrtimer_restart uses the same hrtimer_lock to prevent race.

With this solution we would have to restrict calls to forward and expires. Maybe that would be OK, but it would be restricting the API further.

As I understand the problem space, we have (on Rust side):

- start and forward may race. forward is callable on exclusive reference to HrTimer or in callback context, but otherwise lacks synchronization. start is serialized on the base lock but is callable at any time.
- start and expires may race because start writes the expiration and expires reads it. The latter has no synchronization and is callable on shared reference to HrTimer.
- forward and expires may race because HrTimer::expires takes a shared reference and is callable at any time concurrently.

I think the solution suggested by Tomo is OK, but we could also add synchronization to start, forward and expires on the rust side. Would that not solve the problem for us?

This way we can still run the handler without lock. Only if we call forward or read the expiry in the handler would we take the lock.

Quoted 7 lines
Quoted 1 line

This would allow the API as originally described on the rust side.

That would work, but I think it needs more than the lock. The lock makes start and forward safe against each other, but one of them still loses. If start runs first, hrtimer_forward() returns 0 and does nothing, so the overrun it would have returned is lost. Some callers use that return value.

We can put the lock on the rust side of things. Existing C callers would not be affected. Calling forward from the handler and racing a start from outside needs handling anyway. The zero return value would be an indicator.

Quoted 6 lines

perf and CFS bandwidth have a flag as well as a lock. The flag is "do not arm while armed", which is the same rule the types enforce here. rtc and the softlockup watchdog look like they cancel first and then start instead. None of them arms a timer that is active, so I would rather the abstraction did not allow it either. Does that seem reasonable?

I am fine with preventing starting a timer that is Started or Running, but I am not liking the UniqueArc requirement.

I have a use case in rnull where I have to start a timer behind an Arc with no way to obtain a UniqueArc, so I would prefer if that use case keeps on working. Without this, I would have to allocate a box and put it behind a lock, leading to double indirection.

If we can fold in this logic into the API, callers can be simpler.

Best regards,
Andreas Hindborg

#16 lore raw
Quoted 7 lines · On Tue, 18 Aug 2026 11:02:27 +0200 Andreas Hindborg <[email protected]>…
Quoted 6 lines

perf and CFS bandwidth have a flag as well as a lock. The flag is "do not arm while armed", which is the same rule the types enforce here. rtc and the softlockup watchdog look like they cancel first and then start instead. None of them arms a timer that is active, so I would rather the abstraction did not allow it either. Does that seem reasonable?

Quoted 7 lines

I am fine with preventing starting a timer that is Started or Running, but I am not liking the UniqueArc requirement.

I have a use case in rnull where I have to start a timer behind an Arc with no way to obtain a UniqueArc, so I would prefer if that use case keeps on working. Without this, I would have to allocate a box and put it behind a lock, leading to double indirection.

Before the UniqueArc requirement, I would like to check which timer you have in mind? The bandwidth timer, the per-command timer, or something else? The two seem to need different things, so I would rather not guess.

For the bandwidth timer I do not see where the handle would live, and that is independent of UniqueArc. start() returns a handle that cancels the timer when it is dropped, so it has to be kept somewhere, and the current hrtimer API is the same. queue_rq() only gets a shared borrow of the queue data, and the handle owns an Arc<T>, so putting it inside T means T holds a refcount on itself and is never freed.

maintainer
#17 lore raw
Quoted 18 lines · FUJITA Tomonori
Quoted 7 lines · On Tue, 18 Aug 2026 11:02:27 +0200 Andreas Hindborg <[email protected]>…
Quoted 6 lines

perf and CFS bandwidth have a flag as well as a lock. The flag is "do not arm while armed", which is the same rule the types enforce here. rtc and the softlockup watchdog look like they cancel first and then start instead. None of them arms a timer that is active, so I would rather the abstraction did not allow it either. Does that seem reasonable?

Quoted 7 lines

I am fine with preventing starting a timer that is Started or Running, but I am not liking the UniqueArc requirement.

I have a use case in rnull where I have to start a timer behind an Arc with no way to obtain a UniqueArc, so I would prefer if that use case keeps on working. Without this, I would have to allocate a box and put it behind a lock, leading to double indirection.

Quoted 4 lines

Before the UniqueArc requirement, I would like to check which timer you have in mind? The bandwidth timer, the per-command timer, or something else? The two seem to need different things, so I would rather not guess.

The bandwidth timer. It is started from an `Arc<NullBlkDevice>:
HasHrTimer. If we make the suggested change to ArcTimerHandle, I think I would need to change the NullBlkDevice::bandwidth_timer from an embedded HrTimer to a SpinLock<Box<HrTimer>>` or something like that.

Maybe this is fine. I don't think it will affect performance for rnull
- this is already a throttled path. But it gives slightly more convoluted code in the caller by reducing the way we can use the API.

Quoted 7 lines

For the bandwidth timer I do not see where the handle would live, and that is independent of UniqueArc. start() returns a handle that cancels the timer when it is dropped, so it has to be kept somewhere, and the current hrtimer API is the same. queue_rq() only gets a shared borrow of the queue data, and the handle owns an Arc<T>, so putting it inside T means T holds a refcount on itself and is never freed.

The storage of the handle would not change.

Circular references to keep things alive can be fine.

For the completion timer, the change you propose would work fine I think. I would just start the timer via the unique request reference rather than the shared one. This is probably a better way to do it anyway.

Best regards,
Andreas Hindborg

#18 lore raw
Quoted 31 lines · On Tue, 18 Aug 2026 13:59:30 +0200 Andreas Hindborg <[email protected]>…
Quoted 18 lines · FUJITA Tomonori
Quoted 7 lines · On Tue, 18 Aug 2026 11:02:27 +0200 Andreas Hindborg <[email protected]>…
Quoted 6 lines

perf and CFS bandwidth have a flag as well as a lock. The flag is "do not arm while armed", which is the same rule the types enforce here. rtc and the softlockup watchdog look like they cancel first and then start instead. None of them arms a timer that is active, so I would rather the abstraction did not allow it either. Does that seem reasonable?

Quoted 7 lines

I am fine with preventing starting a timer that is Started or Running, but I am not liking the UniqueArc requirement.

I have a use case in rnull where I have to start a timer behind an Arc with no way to obtain a UniqueArc, so I would prefer if that use case keeps on working. Without this, I would have to allocate a box and put it behind a lock, leading to double indirection.

Quoted 4 lines

Before the UniqueArc requirement, I would like to check which timer you have in mind? The bandwidth timer, the per-command timer, or something else? The two seem to need different things, so I would rather not guess.

The bandwidth timer. It is started from an `Arc<NullBlkDevice>:
HasHrTimer. If we make the suggested change to ArcTimerHandle, I think I would need to change the NullBlkDevice::bandwidth_timer from an embedded HrTimer to a SpinLock<Box<HrTimer>>` or something like that.

Quoted 3 lines

Maybe this is fine. I don't think it will affect performance for rnull
- this is already a throttled path. But it gives slightly more convoluted code in the caller by reducing the way we can use the API.

I think we can allow creating HrTimerArc<T> from Arc<T>, so that UniqueArc is not required. ListArc does the same with AtomicTracker, an atomic bool in the object that records whether a ListArc exists:

    HrTimerArc::try_from_arc(Arc<T>) -> Result<HrTimerArc<T>, Arc<T>>

It fails when there is already another HrTimerArc for the object.

Quoted 4 lines

For the completion timer, the change you propose would work fine I think. I would just start the timer via the unique request reference rather than the shared one. This is probably a better way to do it anyway.

Agreed.

maintainer
#19 lore raw
Quoted 44 lines · FUJITA Tomonori
Quoted 31 lines · On Tue, 18 Aug 2026 13:59:30 +0200 Andreas Hindborg <[email protected]>…
Quoted 18 lines · FUJITA Tomonori
Quoted 7 lines · On Tue, 18 Aug 2026 11:02:27 +0200 Andreas Hindborg <[email protected]>…
Quoted 6 lines

perf and CFS bandwidth have a flag as well as a lock. The flag is "do not arm while armed", which is the same rule the types enforce here. rtc and the softlockup watchdog look like they cancel first and then start instead. None of them arms a timer that is active, so I would rather the abstraction did not allow it either. Does that seem reasonable?

Quoted 7 lines

I am fine with preventing starting a timer that is Started or Running, but I am not liking the UniqueArc requirement.

I have a use case in rnull where I have to start a timer behind an Arc with no way to obtain a UniqueArc, so I would prefer if that use case keeps on working. Without this, I would have to allocate a box and put it behind a lock, leading to double indirection.

Quoted 4 lines

Before the UniqueArc requirement, I would like to check which timer you have in mind? The bandwidth timer, the per-command timer, or something else? The two seem to need different things, so I would rather not guess.

The bandwidth timer. It is started from an `Arc<NullBlkDevice>:
HasHrTimer. If we make the suggested change to ArcTimerHandle, I think I would need to change the NullBlkDevice::bandwidth_timer from an embedded HrTimer to a SpinLock<Box<HrTimer>>` or something like that.

Quoted 3 lines

Maybe this is fine. I don't think it will affect performance for rnull
- this is already a throttled path. But it gives slightly more convoluted code in the caller by reducing the way we can use the API.

I think we can allow creating HrTimerArc<T> from Arc<T>, so that UniqueArc is not required. ListArc does the same with AtomicTracker, an atomic bool in the object that records whether a ListArc exists:

    HrTimerArc::try_from_arc(Arc<T>) -> Result<HrTimerArc<T>, Arc<T>>
Quoted 8 lines

It fails when there is already another HrTimerArc for the object.

Quoted 4 lines

For the completion timer, the change you propose would work fine I think. I would just start the timer via the unique request reference rather than the shared one. This is probably a better way to do it anyway.

Agreed.

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Do you want to send a patch based on this solution Tomo?

Best regards,
Andreas Hindborg

#20 lore raw
Quoted 61 lines · On Thu, 20 Aug 2026 11:00:21 +0200 Andreas Hindborg <[email protected]>…
Quoted 44 lines · FUJITA Tomonori
Quoted 31 lines · On Tue, 18 Aug 2026 13:59:30 +0200 Andreas Hindborg <[email protected]>…
Quoted 18 lines · FUJITA Tomonori
Quoted 7 lines · On Tue, 18 Aug 2026 11:02:27 +0200 Andreas Hindborg <[email protected]>…
Quoted 6 lines

perf and CFS bandwidth have a flag as well as a lock. The flag is "do not arm while armed", which is the same rule the types enforce here. rtc and the softlockup watchdog look like they cancel first and then start instead. None of them arms a timer that is active, so I would rather the abstraction did not allow it either. Does that seem reasonable?

Quoted 7 lines

I am fine with preventing starting a timer that is Started or Running, but I am not liking the UniqueArc requirement.

I have a use case in rnull where I have to start a timer behind an Arc with no way to obtain a UniqueArc, so I would prefer if that use case keeps on working. Without this, I would have to allocate a box and put it behind a lock, leading to double indirection.

Quoted 4 lines

Before the UniqueArc requirement, I would like to check which timer you have in mind? The bandwidth timer, the per-command timer, or something else? The two seem to need different things, so I would rather not guess.

The bandwidth timer. It is started from an `Arc<NullBlkDevice>:
HasHrTimer. If we make the suggested change to ArcTimerHandle, I think I would need to change the NullBlkDevice::bandwidth_timer from an embedded HrTimer to a SpinLock<Box<HrTimer>>` or something like that.

Quoted 3 lines

Maybe this is fine. I don't think it will affect performance for rnull
- this is already a throttled path. But it gives slightly more convoluted code in the caller by reducing the way we can use the API.

I think we can allow creating HrTimerArc<T> from Arc<T>, so that UniqueArc is not required. ListArc does the same with AtomicTracker, an atomic bool in the object that records whether a ListArc exists:

    HrTimerArc::try_from_arc(Arc<T>) -> Result<HrTimerArc<T>, Arc<T>>
Quoted 8 lines

It fails when there is already another HrTimerArc for the object.

Quoted 4 lines

For the completion timer, the change you propose would work fine I think. I would just start the timer via the unique request reference rather than the shared one. This is probably a better way to do it anyway.

Agreed.

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

Quoted 6 lines

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Do you want to send a patch based on this solution Tomo?

https://lore.kernel.org/rust-for-linux/[email protected]/

The solution that we discussed before, right? It changes how the hrtimer core calls the callback. If the C maintainers take that, I will do the Rust side for it.

So we leave this race as it is until we know whether the solution is accepted?

maintainer
#21 lore raw
Quoted 69 lines · FUJITA Tomonori
Quoted 66 lines · On Thu, 20 Aug 2026 11:00:21 +0200 Andreas Hindborg <[email protected]>…
Quoted 44 lines · FUJITA Tomonori
Quoted 31 lines · On Tue, 18 Aug 2026 13:59:30 +0200 Andreas Hindborg <[email protected]>…
Quoted 18 lines · FUJITA Tomonori
Quoted 7 lines · On Tue, 18 Aug 2026 11:02:27 +0200 Andreas Hindborg <[email protected]>…
Quoted 6 lines

perf and CFS bandwidth have a flag as well as a lock. The flag is "do not arm while armed", which is the same rule the types enforce here. rtc and the softlockup watchdog look like they cancel first and then start instead. None of them arms a timer that is active, so I would rather the abstraction did not allow it either. Does that seem reasonable?

Quoted 7 lines

I am fine with preventing starting a timer that is Started or Running, but I am not liking the UniqueArc requirement.

I have a use case in rnull where I have to start a timer behind an Arc with no way to obtain a UniqueArc, so I would prefer if that use case keeps on working. Without this, I would have to allocate a box and put it behind a lock, leading to double indirection.

Quoted 4 lines

Before the UniqueArc requirement, I would like to check which timer you have in mind? The bandwidth timer, the per-command timer, or something else? The two seem to need different things, so I would rather not guess.

The bandwidth timer. It is started from an `Arc<NullBlkDevice>:
HasHrTimer. If we make the suggested change to ArcTimerHandle, I think I would need to change the NullBlkDevice::bandwidth_timer from an embedded HrTimer to a SpinLock<Box<HrTimer>>` or something like that.

Quoted 3 lines

Maybe this is fine. I don't think it will affect performance for rnull
- this is already a throttled path. But it gives slightly more convoluted code in the caller by reducing the way we can use the API.

I think we can allow creating HrTimerArc<T> from Arc<T>, so that UniqueArc is not required. ListArc does the same with AtomicTracker, an atomic bool in the object that records whether a ListArc exists:

    HrTimerArc::try_from_arc(Arc<T>) -> Result<HrTimerArc<T>, Arc<T>>
Quoted 8 lines

It fails when there is already another HrTimerArc for the object.

Quoted 4 lines

For the completion timer, the change you propose would work fine I think. I would just start the timer via the unique request reference rather than the shared one. This is probably a better way to do it anyway.

Agreed.

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Quoted 7 lines
Quoted 1 line

Do you want to send a patch based on this solution Tomo?

https://lore.kernel.org/rust-for-linux/[email protected]/

The solution that we discussed before, right? It changes how the hrtimer core calls the callback. If the C maintainers take that, I will do the Rust side for it.

Yes this one. We don't know if C maintainers will like it. We were discussing having a separate path on the C side just for just, alternatively converting C side callers.

I think we should be able to reach some kind of agreement with C timekeeping. But if not, we can solve it on rust side only, but less efficient. We can grab the base lock again, read expires, then drop the lock. But better to do it in the C code.

Quoted 2 lines

So we leave this race as it is until we know whether the solution is accepted?

I guess so. There are no in-tree users that trigger this anyway.

Best regards,
Andreas Hindborg

#22 lore raw
Quoted 19 lines · On Thu, 20 Aug 2026 14:34:16 +0200 Andreas Hindborg <[email protected]>…
Quoted 11 lines
Quoted 11 lines

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Quoted 7 lines
Quoted 1 line

Do you want to send a patch based on this solution Tomo?

https://lore.kernel.org/rust-for-linux/[email protected]/

The solution that we discussed before, right? It changes how the hrtimer core calls the callback. If the C maintainers take that, I will do the Rust side for it.

Quoted 8 lines

Yes this one. We don't know if C maintainers will like it. We were discussing having a separate path on the C side just for just, alternatively converting C side callers.

I think we should be able to reach some kind of agreement with C timekeeping. But if not, we can solve it on rust side only, but less efficient. We can grab the base lock again, read expires, then drop the lock. But better to do it in the C code.

The Rust side cannot take the base lock: lock_hrtimer_base() is static in kernel/time/hrtimer.c, internal to the core. So that way needs some agreement with the C maintainers as well.

maintainer
#23 lore raw
Quoted 26 lines · FUJITA Tomonori
Quoted 23 lines · On Thu, 20 Aug 2026 14:34:16 +0200 Andreas Hindborg <[email protected]>…
Quoted 11 lines
Quoted 11 lines

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Quoted 7 lines
Quoted 1 line

Do you want to send a patch based on this solution Tomo?

https://lore.kernel.org/rust-for-linux/[email protected]/

The solution that we discussed before, right? It changes how the hrtimer core calls the callback. If the C maintainers take that, I will do the Rust side for it.

Yes this one. We don't know if C maintainers will like it. We were discussing having a separate path on the C side just for just, alternatively converting C side callers.

Quoted 8 lines
Quoted 4 lines

I think we should be able to reach some kind of agreement with C timekeeping. But if not, we can solve it on rust side only, but less efficient. We can grab the base lock again, read expires, then drop the lock. But better to do it in the C code.

The Rust side cannot take the base lock: lock_hrtimer_base() is static in kernel/time/hrtimer.c, internal to the core. So that way needs some agreement with the C maintainers as well.

Right, we would have to export the symbol, or a function for this purpose. But it is a smaller change to C code. I would prefer we solve it properly though, not with hacks.

Best regards,
Andreas Hindborg

r

#24 lore raw
Quoted 33 lines · On Fri, 21 Aug 2026 09:13:01 +0200 Andreas Hindborg <[email protected]>…
Quoted 31 lines · FUJITA Tomonori
Quoted 27 lines · On Thu, 20 Aug 2026 14:34:16 +0200 Andreas Hindborg <[email protected]>…
Quoted 11 lines
Quoted 11 lines

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Quoted 7 lines
Quoted 1 line

Do you want to send a patch based on this solution Tomo?

https://lore.kernel.org/rust-for-linux/[email protected]/

The solution that we discussed before, right? It changes how the hrtimer core calls the callback. If the C maintainers take that, I will do the Rust side for it.

Yes this one. We don't know if C maintainers will like it. We were discussing having a separate path on the C side just for just, alternatively converting C side callers.

I think we should be able to reach some kind of agreement with C timekeeping. But if not, we can solve it on rust side only, but less efficient. We can grab the base lock again, read expires, then drop the lock. But better to do it in the C code.

Quoted 7 lines
Quoted 3 lines

The Rust side cannot take the base lock: lock_hrtimer_base() is static in kernel/time/hrtimer.c, internal to the core. So that way needs some agreement with the C maintainers as well.

Right, we would have to export the symbol, or a function for this purpose. But it is a smaller change to C code. I would prefer we solve it properly though, not with hacks.

Could you propose the solution to the C maintainers? I will do the Rust side, whichever way it goes.

maintainer
#25 lore raw
Quoted 40 lines · FUJITA Tomonori
Quoted 37 lines · On Fri, 21 Aug 2026 09:13:01 +0200 Andreas Hindborg <[email protected]>…
Quoted 34 lines · FUJITA Tomonori
Quoted 19 lines · On Thu, 20 Aug 2026 14:34:16 +0200 Andreas Hindborg <[email protected]>…
Quoted 11 lines
Quoted 11 lines

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Quoted 7 lines
Quoted 1 line

Do you want to send a patch based on this solution Tomo?

https://lore.kernel.org/rust-for-linux/[email protected]/

The solution that we discussed before, right? It changes how the hrtimer core calls the callback. If the C maintainers take that, I will do the Rust side for it.

Quoted 8 lines

Yes this one. We don't know if C maintainers will like it. We were discussing having a separate path on the C side just for just, alternatively converting C side callers.

I think we should be able to reach some kind of agreement with C timekeeping. But if not, we can solve it on rust side only, but less efficient. We can grab the base lock again, read expires, then drop the lock. But better to do it in the C code.

The Rust side cannot take the base lock: lock_hrtimer_base() is static in kernel/time/hrtimer.c, internal to the core. So that way needs some agreement with the C maintainers as well.

Quoted 6 lines
Quoted 3 lines

Right, we would have to export the symbol, or a function for this purpose. But it is a smaller change to C code. I would prefer we solve it properly though, not with hacks.

Could you propose the solution to the C maintainers? I will do the Rust side, whichever way it goes.

Not sure what you are asking. C people are CC in this thread. I think the best way to discuss is to write a patch that can be the basis of discussion. Do you want me to draft up something, or do you want to do that?

After we submit the patch, if they do not respond, I can see if I can find one of them in person at LPC. But I think they usually respond.

Best regards,
Andreas Hindborg

#26 lore raw
Quoted 3 lines · Andreas Hindborg

After we submit the patch, if they do not respond, I can see if I can find one of them in person at LPC. But I think they usually respond.

If it helps, maybe bring them to the Rust MC? We could have a small slot for discussing that :)

Cheers,
Miguel

#27 lore raw
In reply to Andreas Hindborg
Quoted 46 lines · On Mon, 24 Aug 2026 12:44:44 +0200 Andreas Hindborg <[email protected]>…
Quoted 44 lines · FUJITA Tomonori
Quoted 40 lines · On Fri, 21 Aug 2026 09:13:01 +0200 Andreas Hindborg <[email protected]>…
Quoted 26 lines · FUJITA Tomonori
Quoted 23 lines · On Thu, 20 Aug 2026 14:34:16 +0200 Andreas Hindborg <[email protected]>…
Quoted 11 lines
Quoted 11 lines

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Quoted 7 lines
Quoted 1 line

Do you want to send a patch based on this solution Tomo?

https://lore.kernel.org/rust-for-linux/[email protected]/

The solution that we discussed before, right? It changes how the hrtimer core calls the callback. If the C maintainers take that, I will do the Rust side for it.

Yes this one. We don't know if C maintainers will like it. We were discussing having a separate path on the C side just for just, alternatively converting C side callers.

Quoted 8 lines
Quoted 4 lines

I think we should be able to reach some kind of agreement with C timekeeping. But if not, we can solve it on rust side only, but less efficient. We can grab the base lock again, read expires, then drop the lock. But better to do it in the C code.

The Rust side cannot take the base lock: lock_hrtimer_base() is static in kernel/time/hrtimer.c, internal to the core. So that way needs some agreement with the C maintainers as well.

Right, we would have to export the symbol, or a function for this purpose. But it is a smaller change to C code. I would prefer we solve it properly though, not with hacks.

Quoted 7 lines
Quoted 2 lines

Could you propose the solution to the C maintainers? I will do the Rust side, whichever way it goes.

Not sure what you are asking. C people are CC in this thread. I think the best way to discuss is to write a patch that can be the basis of discussion. Do you want me to draft up something, or do you want to do that?

The former, please.

This thread has a lot of Rust-specific detail, so I think a new thread with the patch would be a better basis for the discussion.

maintainer
#28 lore raw
Quoted 57 lines · FUJITA Tomonori
Quoted 46 lines · On Mon, 24 Aug 2026 12:44:44 +0200 Andreas Hindborg <[email protected]>…
Quoted 44 lines · FUJITA Tomonori
Quoted 40 lines · On Fri, 21 Aug 2026 09:13:01 +0200 Andreas Hindborg <[email protected]>…
Quoted 26 lines · FUJITA Tomonori
Quoted 23 lines · On Thu, 20 Aug 2026 14:34:16 +0200 Andreas Hindborg <[email protected]>…
Quoted 11 lines
Quoted 11 lines

We discussed this at the call last night. We came to the conclusion that we would like to experiment with the solution outlined by Gary, where we inject expires into the callback handler, and the callback handler returns a forward duration in addition to a restart value. Because with that approach, we can avoid adding complexity to the Arc end of the API.

For the best implementation of this scheme, we probably need to change some bits in the C code, add an additional path. Down the line, we could also see how man callers of the C code can be changed to use this pattern.

Quoted 7 lines
Quoted 1 line

Do you want to send a patch based on this solution Tomo?

https://lore.kernel.org/rust-for-linux/[email protected]/

The solution that we discussed before, right? It changes how the hrtimer core calls the callback. If the C maintainers take that, I will do the Rust side for it.

Yes this one. We don't know if C maintainers will like it. We were discussing having a separate path on the C side just for just, alternatively converting C side callers.

Quoted 8 lines
Quoted 4 lines

I think we should be able to reach some kind of agreement with C timekeeping. But if not, we can solve it on rust side only, but less efficient. We can grab the base lock again, read expires, then drop the lock. But better to do it in the C code.

The Rust side cannot take the base lock: lock_hrtimer_base() is static in kernel/time/hrtimer.c, internal to the core. So that way needs some agreement with the C maintainers as well.

Right, we would have to export the symbol, or a function for this purpose. But it is a smaller change to C code. I would prefer we solve it properly though, not with hacks.

Quoted 7 lines
Quoted 2 lines

Could you propose the solution to the C maintainers? I will do the Rust side, whichever way it goes.

Not sure what you are asking. C people are CC in this thread. I think the best way to discuss is to write a patch that can be the basis of discussion. Do you want me to draft up something, or do you want to do that?

Quoted 4 lines

The former, please.

This thread has a lot of Rust-specific detail, so I think a new thread with the patch would be a better basis for the discussion.

For tracking purposes, the new thread is here [1].

Best regards,
Andreas Hindborg

[1] https://lore.kernel.org/all/[email protected]/