Skip to contentSkip to conversation

hrtimer Rust API

PATCH Superseded Andreas Hindborg 9 participants 59 messages Sep 17, 2024 – Oct 14, 2024

Subsystems (3)

v2 of 12 Superseded Latest is v12 →

lore
Share & tools
lore

Andreas Hindborg

author maintainer
#0 lore raw

v2 cover hrtimer Rust API

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

The major change in this series is the use of a handle to carry ownership of the callback target. In v1, we used the armed timer to carry ownership of the callback target. This caused issues when the live timer became the last owner of the callback target, because the target would be dropped in timer callback context. That is solved by using a handle instead.

A request from Thomas on v1 was to add a more complete API. While I did add more features, we are still missing some. In the interest of getting the patches on list prior to LPC 2024, I am leaving out the following planned features:

- hrtimer_sleeper, schedule_hrtimeout, hrtimer_nanosleep and friends
- introspection functions:
- try_cancel
- get_remaining
- active
- queued
- callback_running
- hrtimer_forward
- access to timer callback target through timer handle

I plan to add these features in the comming months. Adding the above features should not cause much churn, and pending positive review, I see no reason to not pick up this series first.

To make it absolutely clear that I am willing to maintain the code I submit, I added a mantainer entry in the last patch. Feel free to drop it, if you want to make other arrangements.

---

Changes from v1:
- use a handle to own the timer callback target
- add ability to for callback to reschedule timer
- improve impl_has_timer to allow generics
- add support for stack allocated timers
- add support for scheduling closures
- use Ktime for setting expiration
- use CondVar instead of AtomicBool in examples
- rebase on 6.11
- improve documentation

This series is a dependency for unmerged features of the Rust null block driver [1], and for rkvms [2].

---

Andreas Hindborg (13):
rust: hrtimer: introduce hrtimer support
rust: sync: add Arc::as_ptr
rust: sync: add Arc::clone_from_raw
rust: hrtimer: implement TimerPointer for Arc
rust: hrtimer: allow timer restart from timer handler
rust: hrtimer: add UnsafeTimerPointer
rust: hrtimer: implement UnsafeTimerPointer for Pin<&T>
rust: hrtimer: implement UnsafeTimerPointer for Pin<&mut T>
rust: hrtimer: add hrtimer::ScopedTimerPointer
rust: hrtimer: allow specifying a distinct callback parameter
rust: hrtimer: implement TimerPointer for Pin<Box<T>>
rust: hrtimer: add schedule_function to schedule closures
rust: hrtimer: add maintainer entry

Lyude Paul (1):
rust: time: Add Ktime::from_ns()

 MAINTAINERS                    |  10 +
 rust/kernel/hrtimer.rs         | 550 +++++++++++++++++++++++++++++++++
 rust/kernel/hrtimer/arc.rs     |  86 ++++++
 rust/kernel/hrtimer/closure.rs |  72 +++++
 rust/kernel/hrtimer/pin.rs     |  97 ++++++
 rust/kernel/hrtimer/pin_mut.rs |  99 ++++++
 rust/kernel/hrtimer/tbox.rs    |  95 ++++++
 rust/kernel/lib.rs             |   1 +
 rust/kernel/sync/arc.rs        |  28 ++
 rust/kernel/time.rs            |   8 +
 10 files changed, 1046 insertions(+)
 create mode 100644 rust/kernel/hrtimer.rs
 create mode 100644 rust/kernel/hrtimer/arc.rs
 create mode 100644 rust/kernel/hrtimer/closure.rs
 create mode 100644 rust/kernel/hrtimer/pin.rs
 create mode 100644 rust/kernel/hrtimer/pin_mut.rs
 create mode 100644 rust/kernel/hrtimer/tbox.rs

base-commit: 98f7e32f20d28ec452afb208f9cffc08448a2652

Signature · 1 line
2.46.0
author maintainer
#1 lore raw

PATCH v2 1/14 rust: time: Add Ktime::from_ns()

From: Lyude Paul <[email protected]>

A simple function to turn the provided value in nanoseconds into a Ktime value. We allow any type which implements Into<bindings::ktime_t>, which resolves to Into<i64>.

This is useful for some of the older DRM APIs that never got moved to Ktime

---
 rust/kernel/time.rs | 8 ++++++++
 1 file changed, 8 insertions(+)
rust/kernel/time.rs +8 -0
index e3bb5e89f88d..1e02eee09f22 100644
@@ -8,6 +8,8 @@
 //! C header: [`include/linux/jiffies.h`](srctree/include/linux/jiffies.h).
 //! C header: [`include/linux/ktime.h`](srctree/include/linux/ktime.h).
 
+use core::convert::Into;
+
 /// The number of nanoseconds per millisecond.
 pub const NSEC_PER_MSEC: i64 = bindings::NSEC_PER_MSEC as i64;
 
@@ -63,6 +65,12 @@ pub fn to_ns(self) -> i64 {
     pub fn to_ms(self) -> i64 {
         self.divns_constant::<NSEC_PER_MSEC>()
     }
+
+    /// Creates a new Ktime from the given duration in nanoseconds
+    #[inline]
+    pub fn from_ns(ns: impl Into<bindings::ktime_t>) -> Self {
+        Self { inner: ns.into() }
+    }
 }
 
 /// Returns the number of milliseconds between two ktimes.
Signature · 1 line
2.46.0
author maintainer
#2 lore raw

PATCH v2 2/14 rust: hrtimer: introduce hrtimer support

In reply to Andreas Hindborg

This patch adds support for intrusive use of the hrtimer system. For now, only one timer can be embedded in a Rust struct.

The hrtimer Rust API is based on the intrusive style pattern introduced by the Rust workqueue API.

---
 rust/kernel/hrtimer.rs | 231 +++++++++++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs     |   1 +
 2 files changed, 232 insertions(+)
 create mode 100644 rust/kernel/hrtimer.rs
rust/kernel/hrtimer.rs new file +231 -0
new file mode 100644
index 000000000000..5c92afd8eb2c
@@ -0,0 +1,231 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Intrusive high resolution timers.
+//!
+//! Allows scheduling timer callbacks without doing allocations at the time of
+//! scheduling. For now, only one timer per type is allowed.
+
+use crate::{init::PinInit, prelude::*, time::Ktime, types::Opaque};
+use core::marker::PhantomData;
+
+/// A timer backed by a C `struct hrtimer`.
+///
+/// # Invariants
+///
+/// * `self.timer` is initialized by `bindings::hrtimer_init`.
+#[repr(transparent)]
+#[pin_data]
+pub struct Timer<U> {
+    #[pin]
+    timer: Opaque<bindings::hrtimer>,
+    _t: PhantomData<U>,
+}
+
+// SAFETY: A `Timer` can be moved to other threads and used/dropped from there.
+unsafe impl<U> Send for Timer<U> {}
+
+// SAFETY: Timer operations are locked on C side, so it is safe to operate on a
+// timer from multiple threads
+unsafe impl<U> Sync for Timer<U> {}
+
+impl<T> Timer<T> {
+    /// Return an initializer for a new timer instance.
+    pub fn new() -> impl PinInit<Self>
+    where
+        T: TimerCallback,
+    {
+        pin_init!( Self {
+            // INVARIANTS: We initialize `timer` with `hrtimer_init` below.
+            timer <- Opaque::ffi_init(move |place: *mut bindings::hrtimer| {
+                // SAFETY: By design of `pin_init!`, `place` is a pointer live
+                // allocation. hrtimer_init will initialize `place` and does not
+                // require `place` to be initialized prior to the call.
+                unsafe {
+                    bindings::hrtimer_init(
+                        place,
+                        bindings::CLOCK_MONOTONIC as i32,
+                        bindings::hrtimer_mode_HRTIMER_MODE_REL,
+                    );
+                }
+
+                // SAFETY: `place` is pointing to a live allocation, so the deref
+                // is safe.
+                let function: *mut Option<_> =
+                    unsafe { core::ptr::addr_of_mut!((*place).function) };
+
+                // SAFETY: `function` points to a valid allocation and we have
+                // exclusive access.
+                unsafe { core::ptr::write(function, Some(T::CallbackTarget::run)) };
+            }),
+            _t: PhantomData,
+        })
+    }
+
+    /// Get a pointer to the contained `bindings::hrtimer`.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to a live allocation of at least the size of `Self`.
+    unsafe fn raw_get(ptr: *const Self) -> *mut bindings::hrtimer {
+        // SAFETY: The field projection to `timer` does not go out of bounds,
+        // because the caller of this function promises that `ptr` points to an
+        // allocation of at least the size of `Self`.
+        unsafe { Opaque::raw_get(core::ptr::addr_of!((*ptr).timer)) }
+    }
+}
+
+/// Implemented by pointer types that point to structs that embed a [`Timer`].
+///
+/// Typical implementers would be [`Box<T>`], [`Arc<T>`], [`ARef<T>`] where `T`
+/// has a field of type `Timer`.
+///
+/// Target must be [`Sync`] because timer callbacks happen in another thread of
+/// execution (hard or soft interrupt context).
+///
+/// Scheduling a timer returns a [`TimerHandle`] that can be used to manipulate
+/// the timer. Note that it is OK to call the schedule function repeatedly, and
+/// that more than one [`TimerHandle`] associated with a `TimerPointer` may
+/// exist. A timer can be manipulated through any of the handles, and a handle
+/// may represent a cancelled timer.
+///
+/// [`Box<T>`]: Box
+/// [`Arc<T>`]: crate::sync::Arc
+/// [`ARef<T>`]: crate::types::ARef
+pub trait TimerPointer: Sync + Sized {
+    /// A handle representing a scheduled timer.
+    ///
+    /// If the timer is armed or if the timer callback is running when the
+    /// handle is dropped, the drop method of `TimerHandle` should not return
+    /// until the timer is unarmed and the callback has completed.
+    ///
+    /// Note: It must be safe to leak the handle.
+    type TimerHandle: TimerHandle;
+
+    /// Schedule the timer after `expires` time units. If the timer was already
+    /// scheduled, it is rescheduled at the new expiry time.
+    fn schedule(self, expires: Ktime) -> Self::TimerHandle;
+}
+
+/// Implemented by [`TimerPointer`] implementers to give the C timer callback a
+/// function to call.
+// This is split from `TimerPointer` to make it easier to specify trait bounds.
+pub trait RawTimerCallback {
+    /// Callback to be called from C when timer fires.
+    ///
+    /// # Safety
+    ///
+    /// Only to be called by C code in `hrtimer` subsystem. `ptr` must point to
+    /// the `bindings::hrtimer` structure that was used to schedule the timer.
+    unsafe extern "C" fn run(ptr: *mut bindings::hrtimer) -> bindings::hrtimer_restart;
+}
+
+/// Implemented by structs that can the target of a timer callback.
+pub trait TimerCallback {
+    /// The type that was used for scheduling the timer.
+    type CallbackTarget<'a>: RawTimerCallback;
+
+    /// Called by the timer logic when the timer fires.
+    fn run(this: Self::CallbackTarget<'_>)
+    where
+        Self: Sized;
+}
+
+/// A handle representing a potentially armed timer.
+///
+/// More than one handle representing the same timer might exist.
+///
+/// # Safety
+///
+/// When dropped, the timer represented by this handle must be cancelled, if it
+/// is armed. If the timer handler is running when the handle is dropped, the
+/// drop method must wait for the handler to finish before returning.
+pub unsafe trait TimerHandle {}
+
+/// Implemented by structs that contain timer nodes.
+///
+/// Clients of the timer API would usually safely implement this trait by using
+/// the [`impl_has_timer`] macro.
+///
+/// # Safety
+///
+/// Implementers of this trait must ensure that the implementer has a [`Timer`]
+/// field at the offset specified by `OFFSET` and that all trait methods are
+/// implemented according to their documentation.
+///
+/// [`impl_has_timer`]: crate::impl_has_timer
+pub unsafe trait HasTimer<U> {
+    /// Offset of the [`Timer`] field within `Self`
+    const OFFSET: usize;
+
+    /// Return a pointer to the [`Timer`] within `Self`.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to a valid struct of type `Self`.
+    unsafe fn raw_get_timer(ptr: *const Self) -> *const Timer<U> {
+        // SAFETY: By the safety requirement of this trait, the trait
+        // implementor will have a `Timer` field at the specified offset.
+        unsafe { ptr.cast::<u8>().add(Self::OFFSET).cast::<Timer<U>>() }
+    }
+
+    /// Return a pointer to the struct that is embedding the [`Timer`] pointed
+    /// to by `ptr`.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to a [`Timer<U>`] field in a struct of type `Self`.
+    unsafe fn timer_container_of(ptr: *mut Timer<U>) -> *mut Self
+    where
+        Self: Sized,
+    {
+        // SAFETY: By the safety requirement of this function and the `HasTimer`
+        // trait, the following expression will yield a pointer to the `Self`
+        // containing the timer addressed by `ptr`.
+        unsafe { ptr.cast::<u8>().sub(Self::OFFSET).cast::<Self>() }
+    }
+
+    /// Get pointer to embedded `bindings::hrtimer` struct.
+    ///
+    /// # Safety
+    ///
+    /// `self_ptr` must point to a valid `Self`.
+    unsafe fn c_timer_ptr(self_ptr: *const Self) -> *const bindings::hrtimer {
+        // SAFETY: `self_ptr` is a valid pointer to a `Self`.
+        let timer_ptr = unsafe { Self::raw_get_timer(self_ptr) };
+
+        // SAFETY: timer_ptr points to an allocation of at least `Timer` size.
+        unsafe { Timer::raw_get(timer_ptr) }
+    }
+}
+
+/// Use to implement the [`HasTimer<T>`] trait.
+///
+/// See [`module`] documentation for an example.
+///
+/// [`module`]: crate::hrtimer
+#[macro_export]
+macro_rules! impl_has_timer {
+    (
+        impl$({$($generics:tt)*})?
+            HasTimer<$timer_type:ty>
+            for $self:ty
+        { self.$field:ident }
+        $($rest:tt)*
+    ) => {
+        // SAFETY: This implementation of `raw_get_timer` only compiles if the
+        // field has the right type.
+        unsafe impl$(<$($generics)*>)? $crate::hrtimer::HasTimer<$timer_type>  for $self {
+            const OFFSET: usize = ::core::mem::offset_of!(Self, $field) as usize;
+
+            #[inline]
+            unsafe fn raw_get_timer(ptr: *const Self) ->
+                *const $crate::hrtimer::Timer<$timer_type>
+            {
+                // SAFETY: The caller promises that the pointer is not dangling.
+                unsafe {
+                    ::core::ptr::addr_of!((*ptr).$field)
+                }
+            }
+        }
+    }
+}
rust/kernel/lib.rs +1 -0
index 274bdc1b0a82..55f846c5a849 100644
@@ -34,6 +34,7 @@
 pub mod error;
 #[cfg(CONFIG_RUST_FW_LOADER_ABSTRACTIONS)]
 pub mod firmware;
+pub mod hrtimer;
 pub mod init;
 pub mod ioctl;
 #[cfg(CONFIG_KUNIT)]
Signature · 1 line
2.46.0
#3 lore raw

Re: [PATCH v2 02/14] rust: hrtimer: introduce hrtimer support

Quoted 46 lines · Andreas Hindborg

This patch adds support for intrusive use of the hrtimer system. For now, only one timer can be embedded in a Rust struct.

The hrtimer Rust API is based on the intrusive style pattern introduced by the Rust workqueue API.

---
 rust/kernel/hrtimer.rs | 231 +++++++++++++++++++++++++++++++++++++++++
 rust/kernel/lib.rs     |   1 +
 2 files changed, 232 insertions(+)
 create mode 100644 rust/kernel/hrtimer.rs
rust/kernel/hrtimer.rs new file +27 -0
new file mode 100644
index 000000000000..5c92afd8eb2c
@@ -0,0 +1,231 @@
+// SPDX-License-Identifier: GPL-2.0
+
+//! Intrusive high resolution timers.
+//!
+//! Allows scheduling timer callbacks without doing allocations at the time of
+//! scheduling. For now, only one timer per type is allowed.
+
+use crate::{init::PinInit, prelude::*, time::Ktime, types::Opaque};
+use core::marker::PhantomData;
+
+/// A timer backed by a C `struct hrtimer`.
+///
+/// # Invariants
+///
+/// * `self.timer` is initialized by `bindings::hrtimer_init`.
+#[repr(transparent)]
+#[pin_data]
+pub struct Timer<U> {
+    #[pin]
+    timer: Opaque<bindings::hrtimer>,
+    _t: PhantomData<U>,
+}
+
+// SAFETY: A `Timer` can be moved to other threads and used/dropped from there.
+unsafe impl<U> Send for Timer<U> {}
+
+// SAFETY: Timer operations are locked on C side, so it is safe to operate on a

rust/kernel/hrtimer.rs line 37

Quoted 10 lines
rust/kernel/hrtimer.rs new file +10 -0
+// timer from multiple threads
+unsafe impl<U> Sync for Timer<U> {}
+
+impl<T> Timer<T> {
+    /// Return an initializer for a new timer instance.
+    pub fn new() -> impl PinInit<Self>
+    where
+        T: TimerCallback,
+    {
+        pin_init!( Self {

I would remove the space after the (.
Would be great if we had rustfmt support for custom macros.

Quoted 6 lines
rust/kernel/hrtimer.rs new file +6 -0
+            // INVARIANTS: We initialize `timer` with `hrtimer_init` below.
+            timer <- Opaque::ffi_init(move |place: *mut bindings::hrtimer| {
+                // SAFETY: By design of `pin_init!`, `place` is a pointer live
+                // allocation. hrtimer_init will initialize `place` and does not
+                // require `place` to be initialized prior to the call.
+                unsafe {

rust/kernel/hrtimer.rs line 53

Quoted 10 lines
rust/kernel/hrtimer.rs new file +10 -0
+                    bindings::hrtimer_init(
+                        place,
+                        bindings::CLOCK_MONOTONIC as i32,
+                        bindings::hrtimer_mode_HRTIMER_MODE_REL,
+                    );
+                }
+
+                // SAFETY: `place` is pointing to a live allocation, so the deref
+                // is safe.
+                let function: *mut Option<_> =

Do you really need this type hint?

Quoted 39 lines
rust/kernel/hrtimer.rs new file +39 -0
+                    unsafe { core::ptr::addr_of_mut!((*place).function) };
+
+                // SAFETY: `function` points to a valid allocation and we have
+                // exclusive access.
+                unsafe { core::ptr::write(function, Some(T::CallbackTarget::run)) };
+            }),
+            _t: PhantomData,
+        })
+    }
+
+    /// Get a pointer to the contained `bindings::hrtimer`.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to a live allocation of at least the size of `Self`.
+    unsafe fn raw_get(ptr: *const Self) -> *mut bindings::hrtimer {
+        // SAFETY: The field projection to `timer` does not go out of bounds,
+        // because the caller of this function promises that `ptr` points to an
+        // allocation of at least the size of `Self`.
+        unsafe { Opaque::raw_get(core::ptr::addr_of!((*ptr).timer)) }
+    }
+}
+
+/// Implemented by pointer types that point to structs that embed a [`Timer`].
+///
+/// Typical implementers would be [`Box<T>`], [`Arc<T>`], [`ARef<T>`] where `T`
+/// has a field of type `Timer`.
+///
+/// Target must be [`Sync`] because timer callbacks happen in another thread of
+/// execution (hard or soft interrupt context).
+///
+/// Scheduling a timer returns a [`TimerHandle`] that can be used to manipulate
+/// the timer. Note that it is OK to call the schedule function repeatedly, and
+/// that more than one [`TimerHandle`] associated with a `TimerPointer` may
+/// exist. A timer can be manipulated through any of the handles, and a handle
+/// may represent a cancelled timer.
+///
+/// [`Box<T>`]: Box
+/// [`Arc<T>`]: crate::sync::Arc

rust/kernel/hrtimer.rs line 102

Quoted 10 lines
rust/kernel/hrtimer.rs new file +10 -0
+/// [`ARef<T>`]: crate::types::ARef
+pub trait TimerPointer: Sync + Sized {
+    /// A handle representing a scheduled timer.
+    ///
+    /// If the timer is armed or if the timer callback is running when the
+    /// handle is dropped, the drop method of `TimerHandle` should not return
+    /// until the timer is unarmed and the callback has completed.
+    ///
+    /// Note: It must be safe to leak the handle.
+    type TimerHandle: TimerHandle;

Why does this need to be an associated type? Couldn't we have a TimerHandle<T> struct? The schedule function below could then return TimerHandle<Self>.

Quoted 15 lines
rust/kernel/hrtimer.rs new file +12 -0
+
+    /// Schedule the timer after `expires` time units. If the timer was already
+    /// scheduled, it is rescheduled at the new expiry time.
+    fn schedule(self, expires: Ktime) -> Self::TimerHandle;
+}
 
 
 
+/// Use to implement the [`HasTimer<T>`] trait.
+///
+/// See [`module`] documentation for an example.
+///
+/// [`module`]: crate::hrtimer
+#[macro_export]
+macro_rules! impl_has_timer {

rust/kernel/hrtimer.rs line 217

Quoted 10 lines
rust/kernel/hrtimer.rs new file +10 -0
+    (
+        impl$({$($generics:tt)*})?
+            HasTimer<$timer_type:ty>
+            for $self:ty
+        { self.$field:ident }
+        $($rest:tt)*
+    ) => {
+        // SAFETY: This implementation of `raw_get_timer` only compiles if the
+        // field has the right type.
+        unsafe impl$(<$($generics)*>)? $crate::hrtimer::HasTimer<$timer_type>  for $self {

There is a double space in front of for.

Quoted 30 lines
rust/kernel/hrtimer.rs new file +14 -0
+            const OFFSET: usize = ::core::mem::offset_of!(Self, $field) as usize;
+
+            #[inline]
+            unsafe fn raw_get_timer(ptr: *const Self) ->
+                *const $crate::hrtimer::Timer<$timer_type>
+            {
+                // SAFETY: The caller promises that the pointer is not dangling.
+                unsafe {
+                    ::core::ptr::addr_of!((*ptr).$field)
+                }
+            }
+        }
+    }
+}
rust/kernel/lib.rs +1 -1
index 274bdc1b0a82..55f846c5a849 100644
@@ -34,6 +34,7 @@
 pub mod error;
 #[cfg(CONFIG_RUST_FW_LOADER_ABSTRACTIONS)]
 pub mod firmware;
+pub mod hrtimer;
 pub mod init;
 pub mod ioctl;
 #[cfg(CONFIG_KUNIT)]
--

2.46.0

author maintainer
#4 lore raw

Re: [PATCH v2 02/14] rust: hrtimer: introduce hrtimer support

Hi Benno,

Thanks for the feedback. I will incorporate all the whitespace suggestions you have =F0=9F=91=8D

Quoted 1 line · Benno Lossin

On 18.09.24 00:27, Andreas Hindborg wrote:

[...]

Quoted 11 lines

rust/kernel/hrtimer.rs line 37

Quoted 8 lines
rust/kernel/hrtimer.rs new file +8 -0
+
+impl<T> Timer<T> {
+    /// Return an initializer for a new timer instance.
+    pub fn new() -> impl PinInit<Self>
+    where
+        T: TimerCallback,
+    {
+        pin_init!( Self {

I would remove the space after the (.
Would be great if we had rustfmt support for custom macros.

Yes, that would be great!

Quoted 2 lines
Quoted 2 lines

+ // INVARIANTS: We initialize timer with hrtimer_init be=

low.

Quoted 1 line
Quoted 1 line

+ timer <- Opaque::ffi_init(move |place: *mut bindings::hrtim=

er| {

Quoted 1 line
Quoted 1 line

+ // SAFETY: By design of pin_init!, place is a point=

er live

Quoted 1 line
Quoted 1 line

+ // allocation. hrtimer_init will initialize place and=

does not

Quoted 10 lines

rust/kernel/hrtimer.rs

Quoted 10 lines
rust/kernel/hrtimer.rs new file +10 -0
+                // require `place` to be initialized prior to the call.
+                unsafe {
+                    bindings::hrtimer_init(
+                        place,
+                        bindings::CLOCK_MONOTONIC as i32,
+                        bindings::hrtimer_mode_HRTIMER_MODE_REL,
+                    );
+                }
+
+                // SAFETY: `place` is pointing to a live allocation, so=

the deref

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+                // is safe.
+                let function: *mut Option<_> =3D

Do you really need this type hint?

Apparently not!

[...]

Quoted 4 lines

rust/kernel/hrtimer.rs

Quoted 4 lines
rust/kernel/hrtimer.rs new file +4 -0
+pub trait TimerPointer: Sync + Sized {
+    /// A handle representing a scheduled timer.
+    ///
+    /// If the timer is armed or if the timer callback is running when =

the

Quoted 1 line
Quoted 1 line

+ /// handle is dropped, the drop method of TimerHandle should not =

return

Quoted 8 lines

rust/kernel/hrtimer.rs line 102

Quoted 4 lines
rust/kernel/hrtimer.rs new file +4 -0
+    /// until the timer is unarmed and the callback has completed.
+    ///
+    /// Note: It must be safe to leak the handle.
+    type TimerHandle: TimerHandle;

Why does this need to be an associated type? Couldn't we have a TimerHandle<T> struct? The schedule function below could then return TimerHandle<Self>.

At one point, I had some cycles in trait resolution. Moving generics to associated types solved that issue. Maybe this can be changed to a generic. But are generics preferred over associated types for some reason?

Best regards,
Andreas

#5 lore raw

Re: [PATCH v2 02/14] rust: hrtimer: introduce hrtimer support

Quoted 4 lines · Andreas Hindborg

Hi Benno,

Thanks for the feedback. I will incorporate all the whitespace suggestions you have =F0=9F=91=8D

There seems to be something wrong with the encoding in this email, is that an issue on my side or yours?

Quoted 58 lines
Quoted 1 line · Benno Lossin

On 18.09.24 00:27, Andreas Hindborg wrote:

[...]

Quoted 11 lines

rust/kernel/hrtimer.rs line 37

Quoted 8 lines
rust/kernel/hrtimer.rs new file +8 -0
+
+impl<T> Timer<T> {
+    /// Return an initializer for a new timer instance.
+    pub fn new() -> impl PinInit<Self>
+    where
+        T: TimerCallback,
+    {
+        pin_init!( Self {

I would remove the space after the (.
Would be great if we had rustfmt support for custom macros.

Yes, that would be great!

Quoted 2 lines
Quoted 2 lines

+ // INVARIANTS: We initialize timer with hrtimer_init be=

low.

Quoted 1 line
Quoted 1 line

+ timer <- Opaque::ffi_init(move |place: *mut bindings::hrtim=

er| {

Quoted 1 line
Quoted 1 line

+ // SAFETY: By design of pin_init!, place is a point=

er live

Quoted 1 line
Quoted 1 line

+ // allocation. hrtimer_init will initialize place and=

does not

Quoted 10 lines

rust/kernel/hrtimer.rs

Quoted 10 lines
rust/kernel/hrtimer.rs new file +10 -0
+                // require `place` to be initialized prior to the call.
+                unsafe {
+                    bindings::hrtimer_init(
+                        place,
+                        bindings::CLOCK_MONOTONIC as i32,
+                        bindings::hrtimer_mode_HRTIMER_MODE_REL,
+                    );
+                }
+
+                // SAFETY: `place` is pointing to a live allocation, so=

the deref

Quoted 4 lines

Review

Quoted 2 lines
(quoted diff) +2 -0
+                // is safe.
+                let function: *mut Option<_> =3D

Do you really need this type hint?

Apparently not!

[...]

Quoted 4 lines

rust/kernel/hrtimer.rs

Quoted 4 lines
rust/kernel/hrtimer.rs new file +4 -0
+pub trait TimerPointer: Sync + Sized {
+    /// A handle representing a scheduled timer.
+    ///
+    /// If the timer is armed or if the timer callback is running when =

the

Quoted 1 line
Quoted 1 line

+ /// handle is dropped, the drop method of TimerHandle should not =

return

Quoted 5 lines
Quoted 4 lines
rust/kernel/hrtimer.rs new file +4 -0
+    /// until the timer is unarmed and the callback has completed.
+    ///
+    /// Note: It must be safe to leak the handle.
+    type TimerHandle: TimerHandle;
Quoted 8 lines
Quoted 3 lines

Why does this need to be an associated type? Couldn't we have a TimerHandle<T> struct? The schedule function below could then return TimerHandle<Self>.

At one point, I had some cycles in trait resolution. Moving generics to associated types solved that issue. Maybe this can be changed to a generic. But are generics preferred over associated types for some reason?

The associated type is more complicated IMO, because then every implementer of the trait needs to create one. If we can avoid that, I would prefer a generic type.

---
Cheers,
Benno

author maintainer
#6 lore raw

Re: [PATCH v2 02/14] rust: hrtimer: introduce hrtimer support

Quoted 8 lines · Benno Lossin
Quoted 4 lines · Andreas Hindborg

Hi Benno,

Thanks for the feedback. I will incorporate all the whitespace suggestions you have =F0=9F=91=8D

There seems to be something wrong with the encoding in this email, is that an issue on my side or yours?

It seems to be on my end. It also showed up on lore with errors. I will check my tools.

[...]

Quoted 9 lines
Quoted 9 lines
Quoted 8 lines

rust/kernel/hrtimer.rs line 102

Quoted 4 lines
rust/kernel/hrtimer.rs new file +4 -0
+    /// until the timer is unarmed and the callback has completed.
+    ///
+    /// Note: It must be safe to leak the handle.
+    type TimerHandle: TimerHandle;

Why does this need to be an associated type? Couldn't we have a TimerHandle<T> struct? The schedule function below could then return TimerHandle<Self>.

Quoted 8 lines
Quoted 4 lines

At one point, I had some cycles in trait resolution. Moving generics to associated types solved that issue. Maybe this can be changed to a generic. But are generics preferred over associated types for some reason?

The associated type is more complicated IMO, because then every implementer of the trait needs to create one. If we can avoid that, I would prefer a generic type.

When you say create, you mean specify? Users would not invent their own type to put here, they would use the types defined by the hrtimer module in later patches.

BR Andreas

#7 lore raw

Re: [PATCH v2 02/14] rust: hrtimer: introduce hrtimer support

Quoted 16 lines · Andreas Hindborg
Quoted 15 lines · Benno Lossin
Quoted 13 lines · Andreas Hindborg
Quoted 8 lines

rust/kernel/hrtimer.rs line 102

Quoted 4 lines
rust/kernel/hrtimer.rs new file +4 -0
+    /// until the timer is unarmed and the callback has completed.
+    ///
+    /// Note: It must be safe to leak the handle.
+    type TimerHandle: TimerHandle;

Why does this need to be an associated type? Couldn't we have a TimerHandle<T> struct? The schedule function below could then return TimerHandle<Self>.

At one point, I had some cycles in trait resolution. Moving generics to associated types solved that issue. Maybe this can be changed to a generic. But are generics preferred over associated types for some reason?

Quoted 5 lines
Quoted 3 lines

The associated type is more complicated IMO, because then every implementer of the trait needs to create one. If we can avoid that, I would prefer a generic type.

When you say create, you mean specify?

Yes.

Quoted 2 lines

Users would not invent their own type to put here, they would use the types defined by the hrtimer module in later patches.

I mean the implementers of this trait, not the users of the trait. You define an ArcTimerHandle, PinTimerHandle and a PinMutTimerHandle in this series. I think we can avoid that using a single generic struct.

---
Cheers,
Benno

author maintainer
#8 lore raw

Re: [PATCH v2 02/14] rust: hrtimer: introduce hrtimer support

Quoted 23 lines · Benno Lossin
Quoted 16 lines · Andreas Hindborg
Quoted 15 lines · Benno Lossin
Quoted 13 lines · Andreas Hindborg
Quoted 8 lines

rust/kernel/hrtimer.rs line 102

Quoted 4 lines
rust/kernel/hrtimer.rs new file +4 -0
+    /// until the timer is unarmed and the callback has completed.
+    ///
+    /// Note: It must be safe to leak the handle.
+    type TimerHandle: TimerHandle;

Why does this need to be an associated type? Couldn't we have a TimerHandle<T> struct? The schedule function below could then return TimerHandle<Self>.

At one point, I had some cycles in trait resolution. Moving generics to associated types solved that issue. Maybe this can be changed to a generic. But are generics preferred over associated types for some reason?

Quoted 5 lines
Quoted 3 lines

The associated type is more complicated IMO, because then every implementer of the trait needs to create one. If we can avoid that, I would prefer a generic type.

When you say create, you mean specify?

Quoted 8 lines

Yes.

Quoted 2 lines

Users would not invent their own type to put here, they would use the types defined by the hrtimer module in later patches.

I mean the implementers of this trait, not the users of the trait. You define an ArcTimerHandle, PinTimerHandle and a PinMutTimerHandle in this series. I think we can avoid that using a single generic struct.

It is not immediately clear for me how to do this. For Box we have:

pub struct BoxTimerHandle<U>
where

    U: HasTimer<U>,

{
pub(crate) inner: *mut U,
}

but for Pin<&U> we have

pub struct PinTimerHandle<'a, U>
where

    U: HasTimer<U>,
{
    pub(crate) inner: Pin<&'a U>,
}

How can these be combined to a single generic struct?

Best regards,
Andreas

author maintainer
#9 lore raw

PATCH v2 3/14 rust: sync: add `Arc::as_ptr`

In reply to Andreas Hindborg

Add a method to get a pointer to the data contained in an Arc.

---
 rust/kernel/sync/arc.rs | 8 ++++++++
 1 file changed, 8 insertions(+)
rust/kernel/sync/arc.rs +8 -0
index 3673496c2363..a57ea3e2b44c 100644
@@ -258,6 +258,14 @@ pub fn into_raw(self) -> *const T {
         unsafe { core::ptr::addr_of!((*ptr).data) }
     }
 
+    /// Return a raw pointer to the data in this arc.
+    pub fn as_ptr(&self) -> *const T {
+        let ptr = self.ptr.as_ptr();
+        // SAFETY: As we derive the pointer from a reference above, the pointer
+        // must be valid.
+        unsafe { core::ptr::addr_of!((*ptr).data) }
+    }
+
     /// Recreates an [`Arc`] instance previously deconstructed via [`Arc::into_raw`].
     ///
     /// # Safety
Signature · 1 line
2.46.0
#10 lore raw

Re: [PATCH v2 03/14] rust: sync: add `Arc::as_ptr`

Quoted 12 lines · Andreas Hindborg

Add a method to get a pointer to the data contained in an Arc.

---
 rust/kernel/sync/arc.rs | 8 ++++++++
 1 file changed, 8 insertions(+)
rust/kernel/sync/arc.rs +0 -0
index 3673496c2363..a57ea3e2b44c 100644

rust/kernel/sync/arc.rs pub fn into_raw(self) -> *const T { · line 262

Quoted 5 lines
rust/kernel/sync/arc.rs +2 -0
@@ -258,6 +258,14 @@ pub fn into_raw(self) -> *const T {
         unsafe { core::ptr::addr_of!((*ptr).data) }
     }
 
+    /// Return a raw pointer to the data in this arc.
+    pub fn as_ptr(&self) -> *const T {

I don't know if we have a convention for this, but shouldn't this be an associated function? Because if T also has an as_ptr function, it will be shadowed by this one.

---
Cheers,
Benno

Quoted 13 lines
rust/kernel/sync/arc.rs +6 -1
+        let ptr = self.ptr.as_ptr();
+        // SAFETY: As we derive the pointer from a reference above, the pointer
+        // must be valid.
+        unsafe { core::ptr::addr_of!((*ptr).data) }
+    }
+
     /// Recreates an [`Arc`] instance previously deconstructed via [`Arc::into_raw`].
     ///
     /// # Safety
--

2.46.0

maintainer
#11 lore raw

Re: [PATCH v2 03/14] rust: sync: add `Arc::as_ptr`

Quoted 16 lines · On Thu, 19 Sep 2024 14:03:50 +0000 Benno Lossin <[email protected]> wrote:
Quoted 15 lines · Andreas Hindborg

Add a method to get a pointer to the data contained in an Arc.

---
 rust/kernel/sync/arc.rs | 8 ++++++++
 1 file changed, 8 insertions(+)
rust/kernel/sync/arc.rs +0 -0
index 3673496c2363..a57ea3e2b44c 100644
@@ -258,6 +258,14 @@ pub fn into_raw(self) -> *const T {
         unsafe { core::ptr::addr_of!((*ptr).data) }
     }
 
Quoted 6 lines

rust/kernel/sync/arc.rs pub fn into_raw(self) -> *const T { · line 262

Quoted 2 lines
rust/kernel/sync/arc.rs +2 -0
+    /// Return a raw pointer to the data in this arc.
+    pub fn as_ptr(&self) -> *const T {  

I don't know if we have a convention for this, but shouldn't this be an associated function? Because if T also has an as_ptr function, it will be shadowed by this one.

The usual Rust convention is usually that if Deref is implemented, then unless there's a good reason to do otherwise, associated function should be used.

Best,
Gary

Quoted 19 lines

---
Cheers,
Benno

Quoted 13 lines
rust/kernel/sync/arc.rs +6 -1
+        let ptr = self.ptr.as_ptr();
+        // SAFETY: As we derive the pointer from a reference above, the pointer
+        // must be valid.
+        unsafe { core::ptr::addr_of!((*ptr).data) }
+    }
+
     /// Recreates an [`Arc`] instance previously deconstructed via [`Arc::into_raw`].
     ///
     /// # Safety
--

2.46.0

#12 lore raw

Re: [PATCH v2 03/14] rust: sync: add `Arc::as_ptr`

Quoted 26 lines · Gary Guo
Quoted 16 lines · On Thu, 19 Sep 2024 14:03:50 +0000 Benno Lossin <[email protected]> wrote:
Quoted 15 lines · Andreas Hindborg

Add a method to get a pointer to the data contained in an Arc.

---
 rust/kernel/sync/arc.rs | 8 ++++++++
 1 file changed, 8 insertions(+)
rust/kernel/sync/arc.rs +0 -0
index 3673496c2363..a57ea3e2b44c 100644
@@ -258,6 +258,14 @@ pub fn into_raw(self) -> *const T {
         unsafe { core::ptr::addr_of!((*ptr).data) }
     }
 
Quoted 6 lines

rust/kernel/sync/arc.rs pub fn into_raw(self) -> *const T { · line 262

Quoted 2 lines
rust/kernel/sync/arc.rs +2 -0
+    /// Return a raw pointer to the data in this arc.
+    pub fn as_ptr(&self) -> *const T {

I don't know if we have a convention for this, but shouldn't this be an associated function? Because if T also has an as_ptr function, it will be shadowed by this one.

Quoted 3 lines

The usual Rust convention is usually that if Deref is implemented, then unless there's a good reason to do otherwise, associated function should be used.

We don't do this for into_raw (ie it takes self), I will open a good-first-issue to fix that then.

---
Cheers,
Benno

maintainer
#13 lore raw

Re: [PATCH v2 03/14] rust: sync: add `Arc::as_ptr`

In reply to Gary Guo
Quoted 27 lines · Gary Guo
Quoted 16 lines · On Thu, 19 Sep 2024 14:03:50 +0000 Benno Lossin <[email protected]> wrote:
Quoted 15 lines · Andreas Hindborg

Add a method to get a pointer to the data contained in an Arc.

---
 rust/kernel/sync/arc.rs | 8 ++++++++
 1 file changed, 8 insertions(+)
rust/kernel/sync/arc.rs +0 -0
index 3673496c2363..a57ea3e2b44c 100644
@@ -258,6 +258,14 @@ pub fn into_raw(self) -> *const T {
         unsafe { core::ptr::addr_of!((*ptr).data) }
     }
 
Quoted 6 lines

rust/kernel/sync/arc.rs pub fn into_raw(self) -> *const T { · line 262

Quoted 2 lines
rust/kernel/sync/arc.rs +2 -0
+    /// Return a raw pointer to the data in this arc.
+    pub fn as_ptr(&self) -> *const T {

I don't know if we have a convention for this, but shouldn't this be an associated function? Because if T also has an as_ptr function, it will be shadowed by this one.

Quoted 3 lines

The usual Rust convention is usually that if Deref is implemented, then unless there's a good reason to do otherwise, associated function should be used.

The reason for this convention is that adding new &self methods on a smart pointer is a breaking change, but we don't care about breaking changes, so it doesn't fully apply to us.

Not that we shouldn't follow the convention, but it's not completely clear to me either.

Alice

#14 lore raw

Re: [PATCH v2 03/14] rust: sync: add `Arc::as_ptr`

In reply to Benno Lossin
Quoted 16 lines · Benno Lossin
Quoted 15 lines · Andreas Hindborg

Add a method to get a pointer to the data contained in an Arc.

---
  rust/kernel/sync/arc.rs | 8 ++++++++
  1 file changed, 8 insertions(+)
rust/kernel/sync/arc.rs +0 -0
index 3673496c2363..a57ea3e2b44c 100644
@@ -258,6 +258,14 @@ pub fn into_raw(self) -> *const T {
          unsafe { core::ptr::addr_of!((*ptr).data) }
      }
 
Quoted 6 lines

rust/kernel/sync/arc.rs pub fn into_raw(self) -> *const T { · line 262

Quoted 2 lines
rust/kernel/sync/arc.rs +2 -0
+    /// Return a raw pointer to the data in this arc.
+    pub fn as_ptr(&self) -> *const T {

I don't know if we have a convention for this, but shouldn't this be an associated function? Because if T also has an as_ptr function, it will be shadowed by this one.

Yes. In Fabien's out of tree regmap we have an as_ptr() for Regmap [1] which operates on &Arc<Regmap> [2]. Once this patch is applied to arc.rs the compilation fails as then Arc.as_ptr() is used, not the Regmap.as_ptr() any more [3]. Switching this to something like [4] makes the compiler happy.

Thanks,

Dirk

P.S.: Just to learn something: For the unmodified, failing case: Is there a rule when which as_ptr() will be used? Is there an order rule for the shadowing? Any documentation link?

[1] https://github.com/Fabo/linux/blob/fparent/rust-ncv6336/rust/kernel/regmap.rs#L71

[2] https://github.com/Fabo/linux/blob/fparent/rust-ncv6336/rust/kernel/regulator/driver.rs#L418

[3]

error[E0308]: mismatched types
--> rust/kernel/regulator/driver.rs:420:33

    |
420 |             config.cfg.regmap = regmap.as_ptr();
    |                                 ^^^^^^^^^^^^^^^ types differ in 

mutability
|
= note: expected raw pointer *mut bindings::regmap
found raw pointer *const Regmap

error: aborting due to 1 previous error

[4]

rust/kernel/hrtimer/arc.rs +1 -1
index ff04b0b75bb39..7c39ab440e1c6 100644
@@ -25,7 +25,7 @@ unsafe impl<U> TimerHandle for ArcTimerHandle<U>
     U: HasTimer<U>,
 {
     fn cancel(&mut self) -> bool {
-        let self_ptr = self.inner.as_ptr();
+        let self_ptr = Arc::as_ptr(&self.inner);
 
         // SAFETY: As we obtained `self_ptr` from a valid reference 

above, it

         // must point to a valid `U`.
@@ -57,7 +57,7 @@ impl<U> TimerPointer for Arc<U>
     fn schedule(self, expires: Ktime) -> ArcTimerHandle<U> {
         // SAFETY: Since we generate the pointer passed to `schedule` 
from a
         // valid reference, it is a valid pointer.

- unsafe { U::schedule(self.as_ptr(), expires) };
+ unsafe { U::schedule(Arc::as_ptr(&self), expires) };

         ArcTimerHandle { inner: self }
     }
rust/kernel/sync/arc.rs +2 -2
index 1466d9cd41652..0a314c2f4c5ea 100644
@@ -259,8 +259,8 @@ pub fn into_raw(self) -> *const T {
     }
 
     /// Return a raw pointer to the data in this arc.
-    pub fn as_ptr(&self) -> *const T {
-        let ptr = self.ptr.as_ptr();
+    pub fn as_ptr(arc: &Self) -> *const T {
+        let ptr = arc.ptr.as_ptr();
         // SAFETY: As we derive the pointer from a reference above, 

the pointer

         // must be valid.
         unsafe { core::ptr::addr_of!((*ptr).data) }
#15 lore raw

Re: [PATCH v2 03/14] rust: sync: add `Arc::as_ptr`

Quoted 24 lines · Dirk Behme
Quoted 16 lines · Benno Lossin
Quoted 15 lines · Andreas Hindborg

Add a method to get a pointer to the data contained in an Arc.

---
  rust/kernel/sync/arc.rs | 8 ++++++++
  1 file changed, 8 insertions(+)
rust/kernel/sync/arc.rs +0 -0
index 3673496c2363..a57ea3e2b44c 100644
@@ -258,6 +258,14 @@ pub fn into_raw(self) -> *const T {
          unsafe { core::ptr::addr_of!((*ptr).data) }
      }
 
Quoted 6 lines

rust/kernel/sync/arc.rs pub fn into_raw(self) -> *const T { · line 262

Quoted 2 lines
rust/kernel/sync/arc.rs +2 -0
+    /// Return a raw pointer to the data in this arc.
+    pub fn as_ptr(&self) -> *const T {

I don't know if we have a convention for this, but shouldn't this be an associated function? Because if T also has an as_ptr function, it will be shadowed by this one.

Quoted 5 lines

Yes. In Fabien's out of tree regmap we have an as_ptr() for Regmap [1] which operates on &Arc<Regmap> [2]. Once this patch is applied to arc.rs the compilation fails as then Arc.as_ptr() is used, not the Regmap.as_ptr() any more [3]. Switching this to something like [4] makes the compiler happy.

Yeah then we should switch to that.

Quoted 7 lines

Thanks,

Dirk

P.S.: Just to learn something: For the unmodified, failing case: Is there a rule when which as_ptr() will be used? Is there an order rule for the shadowing? Any documentation link?

Yes, there is a page about the method call expression in the reference:
https://doc.rust-lang.org/reference/expressions/method-call-expr.html

---
Cheers,
Benno

Quoted 64 lines

[1]
https://github.com/Fabo/linux/blob/fparent/rust-ncv6336/rust/kernel/regmap.rs#L71

[2]
https://github.com/Fabo/linux/blob/fparent/rust-ncv6336/rust/kernel/regulator/driver.rs#L418

[3]

error[E0308]: mismatched types

    --> rust/kernel/regulator/driver.rs:420:33
     |

420 | config.cfg.regmap = regmap.as_ptr();
| ^^^^^^^^^^^^^^^ types differ in
mutability
|
= note: expected raw pointer *mut bindings::regmap
found raw pointer *const Regmap

error: aborting due to 1 previous error

[4]

rust/kernel/hrtimer/arc.rs +1 -1
index ff04b0b75bb39..7c39ab440e1c6 100644
@@ -25,7 +25,7 @@ unsafe impl<U> TimerHandle for ArcTimerHandle<U>
      U: HasTimer<U>,
  {
      fn cancel(&mut self) -> bool {
-        let self_ptr = self.inner.as_ptr();
+        let self_ptr = Arc::as_ptr(&self.inner);
 
          // SAFETY: As we obtained `self_ptr` from a valid reference

above, it

          // must point to a valid `U`.
rust/kernel/hrtimer/arc.rs +0 -0
@@ -57,7 +57,7 @@ impl<U> TimerPointer for Arc<U>
      fn schedule(self, expires: Ktime) -> ArcTimerHandle<U> {
          // SAFETY: Since we generate the pointer passed to `schedule`

from a

          // valid reference, it is a valid pointer.
rust/kernel/hrtimer/arc.rs +1 -1
-        unsafe { U::schedule(self.as_ptr(), expires) };
+        unsafe { U::schedule(Arc::as_ptr(&self), expires) };
 
          ArcTimerHandle { inner: self }
      }
rust/kernel/sync/arc.rs +2 -2
index 1466d9cd41652..0a314c2f4c5ea 100644
@@ -259,8 +259,8 @@ pub fn into_raw(self) -> *const T {
      }
 
      /// Return a raw pointer to the data in this arc.
-    pub fn as_ptr(&self) -> *const T {
-        let ptr = self.ptr.as_ptr();
+    pub fn as_ptr(arc: &Self) -> *const T {
+        let ptr = arc.ptr.as_ptr();
          // SAFETY: As we derive the pointer from a reference above,

the pointer

          // must be valid.
          unsafe { core::ptr::addr_of!((*ptr).data) }
author maintainer
#16 lore raw

PATCH v2 4/14 rust: sync: add `Arc::clone_from_raw`

In reply to Andreas Hindborg

Add a method to clone an arc from a pointer to the data managed by the Arc.

---
 rust/kernel/sync/arc.rs | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
rust/kernel/sync/arc.rs +20 -0
index a57ea3e2b44c..2c95712d12a2 100644
@@ -282,6 +282,26 @@ pub unsafe fn from_raw(ptr: *const T) -> Self {
         unsafe { Self::from_inner(ptr) }
     }
 
+    /// Clones an [`Arc`] instance from a pointer to the contained data.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an allocation that is contained within a live [`Arc<T>`].
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }
+    }
+
     /// Returns an [`ArcBorrow`] from the given [`Arc`].
     ///
     /// This is useful when the argument of a function call is an [`ArcBorrow`] (e.g., in a method
Signature · 1 line
2.46.0
#17 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

Quoted 24 lines · Andreas Hindborg

Add a method to clone an arc from a pointer to the data managed by the Arc.

---
 rust/kernel/sync/arc.rs | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
rust/kernel/sync/arc.rs +8 -0
index a57ea3e2b44c..2c95712d12a2 100644
@@ -282,6 +282,26 @@ pub unsafe fn from_raw(ptr: *const T) -> Self {
         unsafe { Self::from_inner(ptr) }
     }
 
+    /// Clones an [`Arc`] instance from a pointer to the contained data.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an allocation that is contained within a live [`Arc<T>`].
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 10 lines
rust/kernel/sync/arc.rs +10 -0
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }

The implementation of this function looks a bit strange to me, how about this?:

    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone

(of course you would need to change the safety requirements of clone_from_raw to point to from_raw)

---
Cheers,
Benno

Quoted 9 lines
rust/kernel/sync/arc.rs +2 -1
+    }
+
     /// Returns an [`ArcBorrow`] from the given [`Arc`].
     ///
     /// This is useful when the argument of a function call is an [`ArcBorrow`] (e.g., in a method
--

2.46.0

maintainer
#18 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

Quoted 46 lines · On Wed, 18 Sep 2024 18:19:20 +0000 Benno Lossin <[email protected]> wrote:
Quoted 24 lines · Andreas Hindborg

Add a method to clone an arc from a pointer to the data managed by the Arc.

---
 rust/kernel/sync/arc.rs | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
rust/kernel/sync/arc.rs +8 -0
index a57ea3e2b44c..2c95712d12a2 100644
@@ -282,6 +282,26 @@ pub unsafe fn from_raw(ptr: *const T) -> Self {
         unsafe { Self::from_inner(ptr) }
     }
 
+    /// Clones an [`Arc`] instance from a pointer to the contained data.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an allocation that is contained within a live [`Arc<T>`].
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 10 lines
rust/kernel/sync/arc.rs +10 -0
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }  

The implementation of this function looks a bit strange to me, how about this?:

    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone
Quoted 2 lines

(of course you would need to change the safety requirements of clone_from_raw to point to from_raw)

Wouldn't this function simply be

	// SAFETY: ...
	let borrow = unsafe { ArcBorrow::from_raw(ptr) }

borrow.into()

?

Maybe this function doesn't even need to exist...

Best,
Gary

#19 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

Quoted 52 lines · Gary Guo
Quoted 46 lines · On Wed, 18 Sep 2024 18:19:20 +0000 Benno Lossin <[email protected]> wrote:
Quoted 24 lines · Andreas Hindborg

Add a method to clone an arc from a pointer to the data managed by the Arc.

---
 rust/kernel/sync/arc.rs | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
rust/kernel/sync/arc.rs +8 -0
index a57ea3e2b44c..2c95712d12a2 100644
@@ -282,6 +282,26 @@ pub unsafe fn from_raw(ptr: *const T) -> Self {
         unsafe { Self::from_inner(ptr) }
     }
 
+    /// Clones an [`Arc`] instance from a pointer to the contained data.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an allocation that is contained within a live [`Arc<T>`].
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 10 lines
rust/kernel/sync/arc.rs +10 -0
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }

The implementation of this function looks a bit strange to me, how about this?:

    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone
Quoted 2 lines

(of course you would need to change the safety requirements of clone_from_raw to point to from_raw)

Quoted 7 lines

Wouldn't this function simply be

	// SAFETY: ...
	let borrow = unsafe { ArcBorrow::from_raw(ptr) }

borrow.into()

?

Oh yeah we do have ArcBorrow...

Quoted 1 line

Maybe this function doesn't even need to exist...

Depends on how often the above has to be written, but I think yeah, it might not be necessary.

---
Cheers,
Benno

author maintainer
#20 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

In reply to Gary Guo
Quoted 4 lines · Gary Guo
Quoted 1 line · On Wed, 18 Sep 2024 18:19:20 +0000 Benno Lossin <[email protected]> wrote:

On 18.09.24 00:27, Andreas Hindborg wrote:

[...]

Quoted 29 lines
Quoted 24 lines

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 13 lines
rust/kernel/sync/arc.rs +13 -0
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }

The implementation of this function looks a bit strange to me, how about this?:

    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone
Quoted 2 lines

(of course you would need to change the safety requirements of clone_from_raw to point to from_raw)

Wouldn't this function simply be

Quoted 7 lines
	// SAFETY: ...
	let borrow = unsafe { ArcBorrow::from_raw(ptr) }

borrow.into()

?

Maybe this function doesn't even need to exist...

Maybe that could work. But my use case does not satisfy the safety requirements on ArcBorrow::from_raw. TheArc::into_raw was not called. Perhaps we can update the requirements for that function?

Best regards,
Andreas

#21 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

Quoted 40 lines · Andreas Hindborg
Quoted 4 lines · Gary Guo
Quoted 1 line · On Wed, 18 Sep 2024 18:19:20 +0000 Benno Lossin <[email protected]> wrote:

On 18.09.24 00:27, Andreas Hindborg wrote:

[...]

Quoted 33 lines
Quoted 24 lines

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 13 lines
rust/kernel/sync/arc.rs +13 -0
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }

The implementation of this function looks a bit strange to me, how about this?:

    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone
Quoted 2 lines

(of course you would need to change the safety requirements of clone_from_raw to point to from_raw)

Wouldn't this function simply be

	// SAFETY: ...
	let borrow = unsafe { ArcBorrow::from_raw(ptr) }

borrow.into()

Quoted 7 lines
Quoted 3 lines

?

Maybe this function doesn't even need to exist...

Maybe that could work. But my use case does not satisfy the safety requirements on ArcBorrow::from_raw. TheArc::into_raw was not called. Perhaps we can update the requirements for that function?

If I understood the code correctly, you are essentially doing this:

    let arc = Arc::<T>::new(..);
    let ptr = Arc::as_ptr(&arc);
    let ptr = T::raw_get_timer(ptr);
    let ptr = Timer::raw_get(ptr);

    // ptr is now used by the timer subsystem to fire the timer

    let ptr = ptr.cast::<Timer>();
    let ptr = T::timer_container_of(ptr);
    let borrow = ArcBorrow::from_raw(ptr);
    let arc = borrow.into();

The only thing that we would have to change would be adding Arc::as_ptr as a source in the ArcBorrow::from_raw safety requirements.

---
Cheers,
Benno

author maintainer
#22 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

Quoted 63 lines · Benno Lossin
Quoted 40 lines · Andreas Hindborg
Quoted 4 lines · Gary Guo
Quoted 1 line · On Wed, 18 Sep 2024 18:19:20 +0000 Benno Lossin <[email protected]> wrote:

On 18.09.24 00:27, Andreas Hindborg wrote:

[...]

Quoted 33 lines
Quoted 24 lines

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 13 lines
rust/kernel/sync/arc.rs +13 -0
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }

The implementation of this function looks a bit strange to me, how about this?:

    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone
Quoted 2 lines

(of course you would need to change the safety requirements of clone_from_raw to point to from_raw)

Wouldn't this function simply be

	// SAFETY: ...
	let borrow = unsafe { ArcBorrow::from_raw(ptr) }

borrow.into()

Quoted 7 lines
Quoted 3 lines

?

Maybe this function doesn't even need to exist...

Maybe that could work. But my use case does not satisfy the safety requirements on ArcBorrow::from_raw. TheArc::into_raw was not called. Perhaps we can update the requirements for that function?

If I understood the code correctly, you are essentially doing this:

    let arc = Arc::<T>::new(..);
    let ptr = Arc::as_ptr(&arc);
    let ptr = T::raw_get_timer(ptr);
    let ptr = Timer::raw_get(ptr);

    // ptr is now used by the timer subsystem to fire the timer

    let ptr = ptr.cast::<Timer>();
    let ptr = T::timer_container_of(ptr);
    let borrow = ArcBorrow::from_raw(ptr);
    let arc = borrow.into();
Quoted 3 lines

The only thing that we would have to change would be adding Arc::as_ptr as a source in the ArcBorrow::from_raw safety requirements.

Yes, I think so. I agree that ArcBorrow is the right way forward here. Patch 11 of this series extends TimerCallback to be able to have a different type for the implementer of RawCallback and the type passed to run. This is to be able to specify Pin<Box<_> as the pointer but & _ as the parameter. Those names should probably be changed to CallbackTarget and CallbackTargetBorrow, but they could cover this use of ArcBorrow as well.

This use is very similar to ForeignOwnable, but a slightly different use case, since I want to implement for pinned references too. Is there a way of aligning this and sharing some trait here?

Best regards,
Andreas

author maintainer
#23 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

In reply to Benno Lossin
Quoted 39 lines · Benno Lossin
Quoted 24 lines · Andreas Hindborg

Add a method to clone an arc from a pointer to the data managed by the Arc.

---
 rust/kernel/sync/arc.rs | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
rust/kernel/sync/arc.rs +8 -0
index a57ea3e2b44c..2c95712d12a2 100644
@@ -282,6 +282,26 @@ pub unsafe fn from_raw(ptr: *const T) -> Self {
         unsafe { Self::from_inner(ptr) }
     }
 
+    /// Clones an [`Arc`] instance from a pointer to the contained data.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an allocation that is contained within a live [`Arc<T>`].
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 10 lines
rust/kernel/sync/arc.rs +10 -0
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }

The implementation of this function looks a bit strange to me, how about this?:

Quoted 7 lines
    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone

We do not own
a refcount on the Arc. For a short duration you will have a wrong refcount. If you have two Arcs and the refcount is 1, the ArcInner might be dropped after the first line of this suggestion, before you do clone, and then this is not sound.

Best regards,
Andreas

author maintainer
#24 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

Quoted 49 lines · Andreas Hindborg
Quoted 39 lines · Benno Lossin
Quoted 24 lines · Andreas Hindborg

Add a method to clone an arc from a pointer to the data managed by the Arc.

---
 rust/kernel/sync/arc.rs | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
rust/kernel/sync/arc.rs +8 -0
index a57ea3e2b44c..2c95712d12a2 100644
@@ -282,6 +282,26 @@ pub unsafe fn from_raw(ptr: *const T) -> Self {
         unsafe { Self::from_inner(ptr) }
     }
 
+    /// Clones an [`Arc`] instance from a pointer to the contained data.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an allocation that is contained within a live [`Arc<T>`].
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 10 lines
rust/kernel/sync/arc.rs +10 -0
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }

The implementation of this function looks a bit strange to me, how about this?:

Quoted 7 lines
    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone
Quoted 5 lines

We do not own
a refcount on the Arc. For a short duration you will have a wrong refcount. If you have two Arcs and the refcount is 1, the ArcInner might be dropped after the first line of this suggestion, before you do clone, and then this is not sound.

Well, disregard that. This is why one should not reply to emails before coffee in the morning.

Of course, a precondition for calling this function is that the arc containing the data pointed to by ptr is live for the duration. So what you wrote would work. But I still do not like having two Arcs in existence with the wrong refcount.

BR Andreas

maintainer
#25 lore raw

Re: [PATCH v2 04/14] rust: sync: add `Arc::clone_from_raw`

Quoted 58 lines · Andreas Hindborg
Quoted 49 lines · Andreas Hindborg
Quoted 39 lines · Benno Lossin
Quoted 24 lines · Andreas Hindborg

Add a method to clone an arc from a pointer to the data managed by the Arc.

---
 rust/kernel/sync/arc.rs | 20 ++++++++++++++++++++
 1 file changed, 20 insertions(+)
rust/kernel/sync/arc.rs +8 -0
index a57ea3e2b44c..2c95712d12a2 100644
@@ -282,6 +282,26 @@ pub unsafe fn from_raw(ptr: *const T) -> Self {
         unsafe { Self::from_inner(ptr) }
     }
 
+    /// Clones an [`Arc`] instance from a pointer to the contained data.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an allocation that is contained within a live [`Arc<T>`].
+    pub unsafe fn clone_from_raw(ptr: *const T) -> Self {
+        // SAFETY: The caller promises that this pointer points to data
+        // contained in an `Arc` that is still valid.

rust/kernel/sync/arc.rs pub unsafe fn from_raw(ptr: *const T) -> Self { · line 302

Quoted 10 lines
rust/kernel/sync/arc.rs +10 -0
+        let inner = unsafe { ArcInner::container_of(ptr).as_ref() };
+
+        // INVARIANT: C `refcount_inc` saturates the refcount, so it cannot
+        // overflow to zero. SAFETY: By the function safety requirement, there
+        // is necessarily a reference to the object, so it is safe to increment
+        // the refcount.
+        unsafe { bindings::refcount_inc(inner.refcount.get()) };
+
+        // SAFETY: We just incremented the refcount. This increment is now owned by the new `Arc`.
+        unsafe { Self::from_inner(inner.into()) }

The implementation of this function looks a bit strange to me, how about this?:

Quoted 7 lines
    // SAFETY: this function has the same safety requirements as `from_raw`.
    let arc = unsafe { Self::from_raw(ptr) };
    let clone = arc.clone();
    // Prevent decrementing the refcount.
    mem::forget(arc);
    clone
Quoted 5 lines

We do not own
a refcount on the Arc. For a short duration you will have a wrong refcount. If you have two Arcs and the refcount is 1, the ArcInner might be dropped after the first line of this suggestion, before you do clone, and then this is not sound.

Quoted 7 lines

Well, disregard that. This is why one should not reply to emails before coffee in the morning.

Of course, a precondition for calling this function is that the arc containing the data pointed to by ptr is live for the duration. So what you wrote would work. But I still do not like having two Arcs in existence with the wrong refcount.

Doing it this way has been pretty standard with std for a long time, until the Arc::increment_strong_count and similar methods were added. I think it's fine, though I would have used ManuallyDrop instead of mem::forget.

Of course, in this particular case, using ArcBorrow is even better.

Alice

author maintainer
#26 lore raw

PATCH v2 5/14 rust: hrtimer: implement `TimerPointer` for `Arc`

In reply to Andreas Hindborg

This patch allows the use of intrusive hrtimer fields in structs that are managed by an Arc.

---
 rust/kernel/hrtimer.rs     | 102 ++++++++++++++++++++++++++++++++++++-
 rust/kernel/hrtimer/arc.rs |  87 +++++++++++++++++++++++++++++++
 2 files changed, 188 insertions(+), 1 deletion(-)
 create mode 100644 rust/kernel/hrtimer/arc.rs
rust/kernel/hrtimer.rs +101 -1
index 5c92afd8eb2c..fd1520ba9fba 100644
@@ -4,6 +4,64 @@
 //!
 //! Allows scheduling timer callbacks without doing allocations at the time of
 //! scheduling. For now, only one timer per type is allowed.
+//!
+//! # Example
+//!
+//! ```
+//! use kernel::{
+//!     hrtimer::{Timer, TimerCallback, TimerPointer},
+//!     impl_has_timer, new_condvar, new_mutex,
+//!     prelude::*,
+//!     sync::{Arc, CondVar, Mutex},
+//!     time::Ktime,
+//! };
+//!
+//! #[pin_data]
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }
+//!
+//! impl ArcIntrusiveTimer {
+//!     fn new() -> impl PinInit<Self, kernel::error::Error> {
+//!         try_pin_init!(Self {
+//!             timer <- Timer::new(),
+//!             flag <- new_mutex!(false),
+//!             cond <- new_condvar!(),
+//!         })
+//!     }
+//! }
+//!
+//! impl TimerCallback for ArcIntrusiveTimer {
+//!     type CallbackTarget<'a> = Arc<Self>;
+//!
+//!     fn run(this: Self::CallbackTarget<'_>) {
+//!         pr_info!("Timer called\n");
+//!         *this.flag.lock() = true;
+//!         this.cond.notify_all();
+//!     }
+//! }
+//!
+//! impl_has_timer! {
+//!     impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
+//! }
+//!
+//!
+//! let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
+//! let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));
+//! let mut guard = has_timer.flag.lock();
+//!
+//! while !*guard {
+//!     has_timer.cond.wait(&mut guard);
+//! }
+//!
+//! pr_info!("Flag raised\n");
+//! # Ok::<(), kernel::error::Error>(())
+//! ```
 
 use crate::{init::PinInit, prelude::*, time::Ktime, types::Opaque};
 use core::marker::PhantomData;
@@ -72,6 +130,25 @@ unsafe fn raw_get(ptr: *const Self) -> *mut bindings::hrtimer {
         // allocation of at least the size of `Self`.
         unsafe { Opaque::raw_get(core::ptr::addr_of!((*ptr).timer)) }
     }
+
+    /// Cancel an initialized and potentially armed timer.
+    ///
+    /// If the timer handler is running, this will block until the handler is
+    /// finished.
+    ///
+    /// # Safety
+    ///
+    /// `self_ptr` must point to a valid `Self`.
+    unsafe fn raw_cancel(self_ptr: *const Self) -> bool {
+        // SAFETY: timer_ptr points to an allocation of at least `Timer` size.
+        let c_timer_ptr = unsafe { Timer::raw_get(self_ptr) };
+
+        // If handler is running, this will wait for handler to finish before
+        // returning.
+        // SAFETY: `c_timer_ptr` is initialized and valid. Synchronization is
+        // handled on C side.
+        unsafe { bindings::hrtimer_cancel(c_timer_ptr) != 0 }
+    }
 }
 
 /// Implemented by pointer types that point to structs that embed a [`Timer`].
@@ -139,7 +216,11 @@ fn run(this: Self::CallbackTarget<'_>)
 /// When dropped, the timer represented by this handle must be cancelled, if it
 /// is armed. If the timer handler is running when the handle is dropped, the
 /// drop method must wait for the handler to finish before returning.
-pub unsafe trait TimerHandle {}
+pub unsafe trait TimerHandle {
+    /// Cancel the timer, if it is armed. If the timer handler is running, block
+    /// till the handler has finished.
+    fn cancel(&mut self) -> bool;
+}
 
 /// Implemented by structs that contain timer nodes.
 ///
@@ -196,6 +277,23 @@ unsafe fn c_timer_ptr(self_ptr: *const Self) -> *const bindings::hrtimer {
         // SAFETY: timer_ptr points to an allocation of at least `Timer` size.
         unsafe { Timer::raw_get(timer_ptr) }
     }
+
+    /// Schedule the timer contained in the `Self` pointed to by `self_ptr`. If
+    /// it is already scheduled it is removed and inserted.
+    ///
+    /// # Safety
+    ///
+    /// `self_ptr` must point to a valid `Self`.
+    unsafe fn schedule(self_ptr: *const Self, expires: Ktime) {
+        unsafe {
+            bindings::hrtimer_start_range_ns(
+                Self::c_timer_ptr(self_ptr).cast_mut(),
+                expires.to_ns(),
+                0,
+                bindings::hrtimer_mode_HRTIMER_MODE_REL,
+            );
+        }
+    }
 }
 
 /// Use to implement the [`HasTimer<T>`] trait.
@@ -229,3 +327,5 @@ unsafe fn raw_get_timer(ptr: *const Self) ->
         }
     }
 }
+
+mod arc;
rust/kernel/hrtimer/arc.rs new file +87 -0

87 lines not on this page · raw message

Signature · 1 line
2.46.0
author maintainer
#27 lore raw

PATCH v2 6/14 rust: hrtimer: allow timer restart from timer handler

In reply to Andreas Hindborg

This patch allows timer handlers to report that they want a timer to be restarted after the timer handler has finished executing.

Also update the hrtimer documentation to showcase the new feature.

---
 rust/kernel/hrtimer.rs     | 50 ++++++++++++++++++++++++++++++++------
 rust/kernel/hrtimer/arc.rs |  4 +--
 2 files changed, 43 insertions(+), 11 deletions(-)
rust/kernel/hrtimer.rs +42 -8
index fd1520ba9fba..d6c3fa89f77e 100644
@@ -9,7 +9,7 @@
 //!
 //! ```
 //! use kernel::{
-//!     hrtimer::{Timer, TimerCallback, TimerPointer},
+//!     hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
 //!     impl_has_timer, new_condvar, new_mutex,
 //!     prelude::*,
 //!     sync::{Arc, CondVar, Mutex},
@@ -21,7 +21,7 @@
 //!     #[pin]
 //!     timer: Timer<Self>,
 //!     #[pin]
-//!     flag: Mutex<bool>,
+//!     flag: Mutex<u64>,
 //!     #[pin]
 //!     cond: CondVar,
 //! }
@@ -30,7 +30,7 @@
 //!     fn new() -> impl PinInit<Self, kernel::error::Error> {
 //!         try_pin_init!(Self {
 //!             timer <- Timer::new(),
-//!             flag <- new_mutex!(false),
+//!             flag <- new_mutex!(0),
 //!             cond <- new_condvar!(),
 //!         })
 //!     }
@@ -39,10 +39,18 @@
 //! impl TimerCallback for ArcIntrusiveTimer {
 //!     type CallbackTarget<'a> = Arc<Self>;
 //!
-//!     fn run(this: Self::CallbackTarget<'_>) {
+//!     fn run(this: Self::CallbackTarget<'_>) -> TimerRestart {
 //!         pr_info!("Timer called\n");
-//!         *this.flag.lock() = true;
+//!         let mut guard = this.flag.lock();
+//!         *guard += 1;
 //!         this.cond.notify_all();
+//!         if *guard == 5 {
+//!             TimerRestart::NoRestart
+//!         }
+//!         else {
+//!             TimerRestart::Restart
+//!
+//!         }
 //!     }
 //! }
 //!
@@ -55,11 +63,11 @@
 //! let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));
 //! let mut guard = has_timer.flag.lock();
 //!
-//! while !*guard {
+//! while *guard != 5 {
 //!     has_timer.cond.wait(&mut guard);
 //! }
 //!
-//! pr_info!("Flag raised\n");
+//! pr_info!("Counted to 5\n");
 //! # Ok::<(), kernel::error::Error>(())
 //! ```
 
@@ -202,7 +210,7 @@ pub trait TimerCallback {
     type CallbackTarget<'a>: RawTimerCallback;
 
     /// Called by the timer logic when the timer fires.
-    fn run(this: Self::CallbackTarget<'_>)
+    fn run(this: Self::CallbackTarget<'_>) -> TimerRestart
     where
         Self: Sized;
 }
@@ -296,6 +304,32 @@ unsafe fn schedule(self_ptr: *const Self, expires: Ktime) {
     }
 }
 
+/// Restart policy for timers.
+pub enum TimerRestart {
+    /// Timer should not be restarted.
+    NoRestart,
+    /// Timer should be restarted.
+    Restart,
+}
+
+impl From<u32> for TimerRestart {
+    fn from(value: bindings::hrtimer_restart) -> Self {
+        match value {
+            0 => Self::NoRestart,
+            _ => Self::Restart,
+        }
+    }
+}
+
+impl From<TimerRestart> for bindings::hrtimer_restart {
+    fn from(value: TimerRestart) -> Self {
+        match value {
+            TimerRestart::NoRestart => bindings::hrtimer_restart_HRTIMER_NORESTART,
+            TimerRestart::Restart => bindings::hrtimer_restart_HRTIMER_RESTART,
+        }
+    }
+}
+
 /// Use to implement the [`HasTimer<T>`] trait.
 ///
 /// See [`module`] documentation for an example.
rust/kernel/hrtimer/arc.rs +1 -3
index 80f6c20f95a9..fb8a40484add 100644
@@ -80,8 +80,6 @@ impl<U> RawTimerCallback for Arc<U>
         // timer. This `U` is contained in an `Arc`.
         let receiver = unsafe { Arc::clone_from_raw(data_ptr) };
 
-        U::run(receiver);
-
-        bindings::hrtimer_restart_HRTIMER_NORESTART
+        U::run(receiver).into()
     }
 }
Signature · 1 line
2.46.0
#28 lore raw

Re: [PATCH v2 06/14] rust: hrtimer: allow timer restart from timer handler

kernel test robot noticed the following build errors:

Hi Andreas,

kernel test robot noticed the following build errors:

[auto build test ERROR on 98f7e32f20d28ec452afb208f9cffc08448a2652]

url:    https://github.com/intel-lab-lkp/linux/commits/Andreas-Hindborg/rust-time-Add-Ktime-from_ns/20240918-063405
base:   98f7e32f20d28ec452afb208f9cffc08448a2652
patch link:    https://lore.kernel.org/r/20240917222739.1298275-7-a.hindborg%40kernel.org
patch subject: [PATCH v2 06/14] rust: hrtimer: allow timer restart from timer handler
config: riscv-randconfig-002-20240920 (https://download.01.org/0day-ci/archive/20240920/[email protected]/config)
compiler: clang version 20.0.0git (https://github.com/llvm/llvm-project 8663a75fa2f31299ab8d1d90288d9df92aadee88)
reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240920/[email protected]/reproduce)

If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/

All errors (new ones prefixed by >>):

In file included from arch/riscv/kernel/asm-offsets.c:10:
In file included from include/linux/mm.h:2232:
include/linux/vmstat.h:517:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]
517 | return node_stat_name(NR_LRU_BASE + lru) + 3; // skip "nr_"
| ~~~~~~~~~~~ ^ ~~~
1 warning generated.
clang diag: include/linux/vmstat.h:517:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion] clang diag: include/linux/vmstat.h:517:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion] clang diag: include/linux/vmstat.h:517:36: warning: arithmetic between different enumeration types ('enum node_stat_item' and 'enum lru_list') [-Wenum-enum-conversion]

>> error[E0053]: method `from` has an incompatible type for trait
   --> rust/kernel/hrtimer.rs:316:20
   |
   316 |     fn from(value: bindings::hrtimer_restart) -> Self {
   |                    ^^^^^^^^^^^^^^^^^^^^^^^^^
   |                    |
   |                    expected `u32`, found `i32`
   |                    help: change the parameter type to match the trait: `u32`
   |

= note: expected signature fn(u32) -> TimerRestart found signature fn(i32) -> TimerRestart

Signature · 2 lines
0-DAY CI Kernel Test Service
https://github.com/intel/lkp-tests/wiki
author maintainer
#29 lore raw

PATCH v2 7/14 rust: hrtimer: add `UnsafeTimerPointer`

In reply to Andreas Hindborg

Add a trait to allow unsafely queuing stack allocated timers.

---
 rust/kernel/hrtimer.rs | 33 +++++++++++++++++++++++++++++++++
 1 file changed, 33 insertions(+)
rust/kernel/hrtimer.rs +33 -0
index d6c3fa89f77e..bb6349f440e2 100644
@@ -191,6 +191,39 @@ pub trait TimerPointer: Sync + Sized {
     fn schedule(self, expires: Ktime) -> Self::TimerHandle;
 }
 
+/// Unsafe version of [`TimerPointer`] for situations where leaking the
+/// `TimerHandle` returned by `schedule` would be unsound. This is the case for
+/// stack allocated timers.
+///
+/// Typical implementers are pinned references such as [`Pin<&T>].
+///
+/// # Safety
+///
+/// Implementers of this trait must ensure that instances of types implementing
+/// [`UnsafeTimerPointer`] outlives any associated [`TimerPointer::TimerHandle`]
+/// instances.
+///
+/// [`Pin<&T>`]: Box
+pub unsafe trait UnsafeTimerPointer: Sync + Sized {
+    /// A handle representing a scheduled timer.
+    ///
+    /// # Safety
+    ///
+    /// If the timer is armed, or if the timer callback is running when the
+    /// handle is dropped, the drop method of `TimerHandle` must not return
+    /// until the timer is unarmed and the callback has completed.
+    type TimerHandle: TimerHandle;
+
+    /// Schedule the timer after `expires` time units. If the timer was already
+    /// scheduled, it is rescheduled at the new expiry time.
+    ///
+    /// # Safety
+    ///
+    /// Caller promises keep the timer structure alive until the timer is dead.
+    /// Caller can ensure this by not leaking the returned `Self::TimerHandle`.
+    unsafe fn schedule(self, expires: Ktime) -> Self::TimerHandle;
+}
+
 /// Implemented by [`TimerPointer`] implementers to give the C timer callback a
 /// function to call.
 // This is split from `TimerPointer` to make it easier to specify trait bounds.
Signature · 1 line
2.46.0
author maintainer
#30 lore raw

PATCH v2 8/14 rust: hrtimer: implement `UnsafeTimerPointer` for `Pin<&T>`

In reply to Andreas Hindborg

Allow pinned references to structs that contain a Timer node to be scheduled with the hrtimer subsystem.

---
 rust/kernel/hrtimer.rs     |  1 +
 rust/kernel/hrtimer/pin.rs | 96 ++++++++++++++++++++++++++++++++++++++
 2 files changed, 97 insertions(+)
 create mode 100644 rust/kernel/hrtimer/pin.rs
rust/kernel/hrtimer.rs +1 -0
index bb6349f440e2..25d3702d0d05 100644
@@ -396,3 +396,4 @@ unsafe fn raw_get_timer(ptr: *const Self) ->
 }
 
 mod arc;
+mod pin;
rust/kernel/hrtimer/pin.rs new file +96 -0

96 lines not on this page · raw message

Signature · 1 line
2.46.0
author maintainer
#31 lore raw

PATCH v2 9/14 rust: hrtimer: implement `UnsafeTimerPointer` for `Pin<&mut T>`

In reply to Andreas Hindborg

Allow pinned mutable references to structs that contain a Timer node to be scheduled with the hrtimer subsystem.

---
 rust/kernel/hrtimer.rs         |  1 +
 rust/kernel/hrtimer/pin_mut.rs | 98 ++++++++++++++++++++++++++++++++++
 2 files changed, 99 insertions(+)
 create mode 100644 rust/kernel/hrtimer/pin_mut.rs
rust/kernel/hrtimer.rs +1 -0
index 25d3702d0d05..09fb674993c2 100644
@@ -397,3 +397,4 @@ unsafe fn raw_get_timer(ptr: *const Self) ->
 
 mod arc;
 mod pin;
+mod pin_mut;
rust/kernel/hrtimer/pin_mut.rs new file +98 -0

98 lines not on this page · raw message

Signature · 1 line
2.46.0
author maintainer
#32 lore raw

PATCH v2 10/14 rust: hrtimer: add `hrtimer::ScopedTimerPointer`

In reply to Andreas Hindborg

Add the trait ScopedTimerPointer to allow safe use of stack allocated timers. Safety is achieved by pinning the stack in place while timers are running.

Implement the trait for all types that implement UnsafeTimerPointer.

---
 rust/kernel/hrtimer.rs | 93 ++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 93 insertions(+)
rust/kernel/hrtimer.rs +93 -0
index 09fb674993c2..6254fa584464 100644
@@ -70,6 +70,66 @@
 //! pr_info!("Counted to 5\n");
 //! # Ok::<(), kernel::error::Error>(())
 //! ```
+//!
+//! Using a stack based timer:
+//! ```
+//! use kernel::{
+//!     hrtimer::{Timer, TimerCallback, ScopedTimerPointer, TimerRestart},
+//!     impl_has_timer, new_condvar, new_mutex,
+//!     prelude::*,
+//!     stack_try_pin_init,
+//!     sync::{CondVar, Mutex},
+//!     time::Ktime,
+//! };
+//!
+//! #[pin_data]
+//! struct IntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }
+//!
+//! impl IntrusiveTimer {
+//!     fn new() -> impl PinInit<Self, kernel::error::Error> {
+//!         try_pin_init!(Self {
+//!             timer <- Timer::new(),
+//!             flag <- new_mutex!(false),
+//!             cond <- new_condvar!(),
+//!         })
+//!     }
+//! }
+//!
+//! impl TimerCallback for IntrusiveTimer {
+//!     type CallbackTarget<'a> = Pin<&'a Self>;
+//!
+//!     fn run(this: Self::CallbackTarget<'_>) -> TimerRestart {
+//!         pr_info!("Timer called\n");
+//!         *this.flag.lock() = true;
+//!         this.cond.notify_all();
+//!         TimerRestart::NoRestart
+//!     }
+//! }
+//!
+//! impl_has_timer! {
+//!     impl HasTimer<Self> for IntrusiveTimer { self.timer }
+//! }
+//!
+//!
+//! stack_try_pin_init!( let has_timer =? IntrusiveTimer::new() );
+//! has_timer.as_ref().schedule_scoped(Ktime::from_ns(200_000_000), || {
+//!     let mut guard = has_timer.flag.lock();
+//!
+//!     while !*guard {
+//!         has_timer.cond.wait(&mut guard);
+//!     }
+//! });
+//!
+//! pr_info!("Flag raised\n");
+//! # Ok::<(), kernel::error::Error>(())
+//! ```
 
 use crate::{init::PinInit, prelude::*, time::Ktime, types::Opaque};
 use core::marker::PhantomData;
@@ -224,6 +284,39 @@ pub unsafe trait UnsafeTimerPointer: Sync + Sized {
     unsafe fn schedule(self, expires: Ktime) -> Self::TimerHandle;
 }
 
+/// A trait for stack allocated timers.
+///
+/// # Safety
+///
+/// Implementers must ensure that `schedule_scoped` does not until the timer is
+/// dead and the timer handler is not running.
+pub unsafe trait ScopedTimerPointer {
+    /// Schedule the timer to run after `expires` time units and immediately
+    /// after call `f`. When `f` returns, the timer is cancelled.
+    fn schedule_scoped<T, F>(self, expires: Ktime, f: F) -> T
+    where
+        F: FnOnce() -> T;
+}
+
+// SAFETY: By the safety requirement of `UnsafeTimerPointer`, dropping the
+// handle returned by `UnsafeTimerPointer::schedule` ensures that the timer is
+// killed.
+unsafe impl<U> ScopedTimerPointer for U
+where
+    U: UnsafeTimerPointer,
+{
+    fn schedule_scoped<T, F>(self, expires: Ktime, f: F) -> T
+    where
+        F: FnOnce() -> T,
+    {
+        // SAFETY: We drop the timer handle below before returning.
+        let handle = unsafe { UnsafeTimerPointer::schedule(self, expires) };
+        let t = f();
+        drop(handle);
+        t
+    }
+}
+
 /// Implemented by [`TimerPointer`] implementers to give the C timer callback a
 /// function to call.
 // This is split from `TimerPointer` to make it easier to specify trait bounds.
Signature · 1 line
2.46.0
author maintainer
#33 lore raw

PATCH v2 11/14 rust: hrtimer: allow specifying a distinct callback parameter

In reply to Andreas Hindborg

For some pointer types it is helpful to be able to differentiate between the TimerPointer and the type passed to the timer callback.

---
 rust/kernel/hrtimer.rs         | 7 ++++++-
 rust/kernel/hrtimer/arc.rs     | 1 +
 rust/kernel/hrtimer/pin.rs     | 1 +
 rust/kernel/hrtimer/pin_mut.rs | 1 +
 4 files changed, 9 insertions(+), 1 deletion(-)
rust/kernel/hrtimer.rs +6 -1
index 6254fa584464..38160221f93e 100644
@@ -38,6 +38,7 @@
 //!
 //! impl TimerCallback for ArcIntrusiveTimer {
 //!     type CallbackTarget<'a> = Arc<Self>;
+//!     type CallbackPointer<'a> = Arc<Self>;
 //!
 //!     fn run(this: Self::CallbackTarget<'_>) -> TimerRestart {
 //!         pr_info!("Timer called\n");
@@ -104,6 +105,7 @@
 //!
 //! impl TimerCallback for IntrusiveTimer {
 //!     type CallbackTarget<'a> = Pin<&'a Self>;
+//!     type CallbackPointer<'a> = Pin<&'a Self>;
 //!
 //!     fn run(this: Self::CallbackTarget<'_>) -> TimerRestart {
 //!         pr_info!("Timer called\n");
@@ -335,8 +337,11 @@ pub trait TimerCallback {
     /// The type that was used for scheduling the timer.
     type CallbackTarget<'a>: RawTimerCallback;
 
+    /// The type passed to the timer callback function.
+    type CallbackPointer<'a>;
+
     /// Called by the timer logic when the timer fires.
-    fn run(this: Self::CallbackTarget<'_>) -> TimerRestart
+    fn run(this: Self::CallbackPointer<'_>) -> TimerRestart
     where
         Self: Sized;
 }
rust/kernel/hrtimer/arc.rs +1 -0
index fb8a40484add..ff04b0b75bb3 100644
@@ -67,6 +67,7 @@ impl<U> RawTimerCallback for Arc<U>
 where
     U: HasTimer<U>,
     U: for<'a> TimerCallback<CallbackTarget<'a> = Self>,
+    U: for<'a> TimerCallback<CallbackPointer<'a> = Self>,
 {
     unsafe extern "C" fn run(ptr: *mut bindings::hrtimer) -> bindings::hrtimer_restart {
         // `Timer` is `repr(transparent)`
rust/kernel/hrtimer/pin.rs +1 -0
index f9ce0498a0d2..d34e0885f0f6 100644
@@ -74,6 +74,7 @@ impl<'a, U> RawTimerCallback for Pin<&'a U>
 where
     U: HasTimer<U>,
     U: TimerCallback<CallbackTarget<'a> = Self>,
+    U: TimerCallback<CallbackPointer<'a> = Self>,
 {
     unsafe extern "C" fn run(ptr: *mut bindings::hrtimer) -> bindings::hrtimer_restart {
         // `Timer` is `repr(transparent)`
rust/kernel/hrtimer/pin_mut.rs +1 -0
index e25c7158ae4f..2589720df233 100644
@@ -76,6 +76,7 @@ impl<'a, U> RawTimerCallback for Pin<&'a mut U>
 where
     U: HasTimer<U>,
     U: TimerCallback<CallbackTarget<'a> = Self>,
+    U: TimerCallback<CallbackPointer<'a> = Self>,
 {
     unsafe extern "C" fn run(ptr: *mut bindings::hrtimer) -> bindings::hrtimer_restart {
         // `Timer` is `repr(transparent)`
Signature · 1 line
2.46.0
author maintainer
#34 lore raw

PATCH v2 12/14 rust: hrtimer: implement `TimerPointer` for `Pin<Box<T>>`

In reply to Andreas Hindborg

Allow Pin<Box<T>> to be the target of a timer callback.

---
 rust/kernel/hrtimer.rs      |  3 ++
 rust/kernel/hrtimer/tbox.rs | 95 +++++++++++++++++++++++++++++++++++++
 2 files changed, 98 insertions(+)
 create mode 100644 rust/kernel/hrtimer/tbox.rs
rust/kernel/hrtimer.rs +3 -0
index 38160221f93e..1750016b2b22 100644
@@ -493,6 +493,9 @@ unsafe fn raw_get_timer(ptr: *const Self) ->
     }
 }
 
+// `box` is a reserved keyword, so prefix with `t` for timer
+mod tbox;
+
 mod arc;
 mod pin;
 mod pin_mut;
rust/kernel/hrtimer/tbox.rs new file +95 -0

95 lines not on this page · raw message

Signature · 1 line
2.46.0
author maintainer
#35 lore raw

PATCH v2 13/14 rust: hrtimer: add `schedule_function` to schedule closures

In reply to Andreas Hindborg

Add a way to quickly schedule functions for execution after a certain duration has elapsed.

---
 rust/kernel/hrtimer.rs         | 49 +++++++++++++++++++++++
 rust/kernel/hrtimer/closure.rs | 72 ++++++++++++++++++++++++++++++++++
 2 files changed, 121 insertions(+)
 create mode 100644 rust/kernel/hrtimer/closure.rs
rust/kernel/hrtimer.rs +49 -0
index 1750016b2b22..2fd3e68368da 100644
@@ -132,6 +132,52 @@
 //! pr_info!("Flag raised\n");
 //! # Ok::<(), kernel::error::Error>(())
 //! ```
+//!
+//! Using a helper:
+//! ```
+//! use kernel::{
+//!     hrtimer::schedule_function,
+//!     impl_has_timer, new_condvar, new_mutex,
+//!     prelude::*,
+//!     stack_try_pin_init,
+//!     sync::{Arc, CondVar, Mutex},
+//!     time::Ktime,
+//! };
+//!
+//! #[pin_data]
+//! struct Data {
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }
+//!
+//! impl Data {
+//!     fn new() -> impl PinInit<Self, kernel::error::Error> {
+//!         try_pin_init!(Self {
+//!             flag <- new_mutex!(false),
+//!             cond <- new_condvar!(),
+//!         })
+//!     }
+//! }
+//!
+//! let data = Arc::pin_init(Data::new(), GFP_KERNEL)?;
+//! let data2 = data.clone();
+//!
+//! let handle = schedule_function(Ktime::from_ns(200_000_000), move || {
+//!     pr_info!("Hello from the future");
+//!     *data2.flag.lock() = true;
+//!     data2.cond.notify_all();
+//! });
+//!
+//! let mut guard = data.flag.lock();
+//! while !*guard {
+//!     data.cond.wait(&mut guard);
+//! }
+//!
+//! pr_info!("Flag raised\n");
+//! # Ok::<(), kernel::error::Error>(())
+//! ```
 
 use crate::{init::PinInit, prelude::*, time::Ktime, types::Opaque};
 use core::marker::PhantomData;
@@ -497,5 +543,8 @@ unsafe fn raw_get_timer(ptr: *const Self) ->
 mod tbox;
 
 mod arc;
+mod closure;
 mod pin;
 mod pin_mut;
+
+pub use closure::schedule_function;
rust/kernel/hrtimer/closure.rs new file +72 -0

72 lines not on this page · raw message

Signature · 1 line
2.46.0
author maintainer
#36 lore raw

PATCH v2 14/14 rust: hrtimer: add maintainer entry

In reply to Andreas Hindborg

Add Andreas Hindborg as maintainer for Rust hrtimer abstractions. Also add Boqun Feng as reviewer.

---
 MAINTAINERS | 10 ++++++++++
 1 file changed, 10 insertions(+)
MAINTAINERS +10 -0
index cc40a9d9b8cd..018847269dd3 100644
@@ -10035,6 +10035,16 @@ F:	kernel/time/timer_list.c
 F:	kernel/time/timer_migration.*
 F:	tools/testing/selftests/timers/
 
+HIGH-RESOLUTION TIMERS [RUST]
+M:	Andreas Hindborg <[email protected]>
+R:	Boqun Feng <[email protected]>
+L:	[email protected]
+S:	Supported
+W:	https://rust-for-linux.com
+B:	https://github.com/Rust-for-Linux/linux/issues
+F:	rust/kernel/hrtimer.rs
+F:	rust/kernel/hrtimer/
+
 HIGH-SPEED SCC DRIVER FOR AX.25
 L:	[email protected]
 S:	Orphan
Signature · 1 line
2.46.0
Acked-by on v2 14/14
#37 lore raw

Re: [PATCH v2 14/14] rust: hrtimer: add maintainer entry

Quoted 4 lines · Andreas Hindborg

Add Andreas Hindborg as maintainer for Rust hrtimer abstractions. Also add Boqun Feng as reviewer.

Although, I would like to suggest we move hrtimer under rust/kernel/time.

Regards,
Boqun

Quoted 30 lines
---
 MAINTAINERS | 10 ++++++++++
 1 file changed, 10 insertions(+)
MAINTAINERS +10 -0
index cc40a9d9b8cd..018847269dd3 100644
@@ -10035,6 +10035,16 @@ F:	kernel/time/timer_list.c
 F:	kernel/time/timer_migration.*
 F:	tools/testing/selftests/timers/
 
+HIGH-RESOLUTION TIMERS [RUST]
+M:	Andreas Hindborg <[email protected]>
+R:	Boqun Feng <[email protected]>
+L:	[email protected]
+S:	Supported
+W:	https://rust-for-linux.com
+B:	https://github.com/Rust-for-Linux/linux/issues
+F:	rust/kernel/hrtimer.rs
+F:	rust/kernel/hrtimer/
+
 HIGH-SPEED SCC DRIVER FOR AX.25
 L:	[email protected]
 S:	Orphan
Signature · 1 line
2.46.0
#38 lore raw
In reply to Andreas Hindborg

Hi Andreas,

Quoted 18 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

The major change in this series is the use of a handle to carry ownership of the callback target. In v1, we used the armed timer to carry ownership of the callback target. This caused issues when the live timer became the last owner of the callback target, because the target would be dropped in timer callback context. That is solved by using a handle instead.

A request from Thomas on v1 was to add a more complete API. While I did add more features, we are still missing some. In the interest of getting the patches on list prior to LPC 2024, I am leaving out the following planned features:

Quoted 9 lines

- hrtimer_sleeper, schedule_hrtimeout, hrtimer_nanosleep and friends
- introspection functions:
- try_cancel
- get_remaining
- active
- queued
- callback_running
- hrtimer_forward
- access to timer callback target through timer handle

Regarding the API: I had a closer look at it after the discussion during LPC. It's possible to change the API (prevent setting the mode in start as well), but it is not as straight forward, as it originally seems to be. So this will take some time to be changed completely.

But what we will do in short term is to create htimer_setup(). This will do the job of hrtimer_init() but expand it by the argument of the hrtimer function callback.

This is just an information update for you. So you can proceed right now with the current API and we keep you in the loop for further changes.

Thanks,

Anna-Maria

author maintainer
#39 lore raw
Quoted 37 lines · Anna-Maria Behnsen

Hi Andreas,

Quoted 18 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

The major change in this series is the use of a handle to carry ownership of the callback target. In v1, we used the armed timer to carry ownership of the callback target. This caused issues when the live timer became the last owner of the callback target, because the target would be dropped in timer callback context. That is solved by using a handle instead.

A request from Thomas on v1 was to add a more complete API. While I did add more features, we are still missing some. In the interest of getting the patches on list prior to LPC 2024, I am leaving out the following planned features:

Quoted 9 lines

- hrtimer_sleeper, schedule_hrtimeout, hrtimer_nanosleep and friends
- introspection functions:
- try_cancel
- get_remaining
- active
- queued
- callback_running
- hrtimer_forward
- access to timer callback target through timer handle

Regarding the API: I had a closer look at it after the discussion during LPC. It's possible to change the API (prevent setting the mode in start as well), but it is not as straight forward, as it originally seems to be. So this will take some time to be changed completely.

Quoted 6 lines

But what we will do in short term is to create htimer_setup(). This will do the job of hrtimer_init() but expand it by the argument of the hrtimer function callback.

This is just an information update for you. So you can proceed right now with the current API and we keep you in the loop for further changes.

Thanks! I think we talked about something similar for v1 as well.

BR Andreas

#40 lore raw
In reply to Andreas Hindborg
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

Best regards

Dirk

[1]

# rust_doctest_kernel_hrtimer_rs_0.location: rust/kernel/hrtimer.rs:10
rust_doctests_kernel: Timer called

=============================
[ BUG: Invalid wait context ]
6.11.0-rc1-arm64 #28 Tainted: G N
-----------------------------
swapper/5/0 is trying to lock:
ffff0004409ab900 (rust/doctests_kernel_generated.rs:1238){+.+.}-{3:3}, at: rust_helper_mutex_lock+0x10/0x18 other info that might help us debug this:
context-{2:2}
no locks held by swapper/5/0.

stack backtrace:
CPU: 5 UID: 0 PID: 0 Comm: swapper/5 Tainted: G N 6.11.0-rc1-arm64 #28
Tainted: [N]=TEST
Hardware name: ARM64 based board (DT)
Call trace:
 $x.11+0x98/0xb4
 show_stack+0x14/0x1c
 $x.3+0x3c/0x94
 dump_stack+0x14/0x1c
 $x.205+0x538/0x594
 $x.179+0xd0/0x18c
 __mutex_lock+0xa0/0xa4
 mutex_lock_nested+0x20/0x28
 rust_helper_mutex_lock+0x10/0x18

_RNvXs_NvNvNvCslTRHJHclVGW_25doctests_kernel_generated32rust_doctest_kernel_hrtimer_rs_04main41__doctest_main_rust_kernel_hrtimer_rs_10_0NtB4_17ArcIntrusiveTimerNtNtCsclYTRz49wqv_6kernel7hrtimer13TimerCallback3run+0x5c/0xd0

_RNvXs1_NtNtCsclYTRz49wqv_6kernel7hrtimer3arcINtNtNtB9_4sync3arc3ArcNtNvNvNvCslTRHJHclVGW_25doctests_kernel_generated32rust_doctest_kernel_hrtimer_rs_04main41__doctest_main_rust_kernel_hrtimer_rs_10_017ArcIntrusiveTimerENtB7_16RawTimerCallback3runB1b_+0x20/0x2c
 $x.90+0x64/0x70
 hrtimer_interrupt+0x1d4/0x2ac
 arch_timer_handler_phys+0x34/0x40
 $x.62+0x50/0x54
 generic_handle_domain_irq+0x28/0x40
 $x.154+0x58/0x6c
 $x.471+0x10/0x20
 el1_interrupt+0x70/0x94
 el1h_64_irq_handler+0x14/0x1c
 el1h_64_irq+0x64/0x68
 arch_local_irq_enable+0x4/0x8
 cpuidle_enter+0x34/0x48
 $x.37+0x58/0xe4
 cpu_startup_entry+0x30/0x34
 $x.2+0xf8/0x118
 $x.13+0x0/0x4

rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Counted to 5
ok 22 rust_doctest_kernel_hrtimer_rs_0
# rust_doctest_kernel_hrtimer_rs_1.location: rust/kernel/hrtimer.rs:137
rust_doctests_kernel: Hello from the future
rust_doctests_kernel: Flag raised
ok 23 rust_doctest_kernel_hrtimer_rs_1
# rust_doctest_kernel_hrtimer_rs_2.location: rust/kernel/hrtimer.rs:76
rust_doctests_kernel: Timer called
rust_doctests_kernel: Flag raised
ok 24 rust_doctest_kernel_hrtimer_rs_2

#41 lore raw
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

+//! struct ArcIntrusiveTimer {

+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,

+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

[*]: I vaguely remember we still have some driver code for slow devices that will enable interrupts during an irq handler, but these are going to be gone, we shouldn't really care about this in Rust code.

Regards,
Boqun

[1]: https://lore.kernel.org/rust-for-linux/[email protected]/

Quoted 68 lines

Best regards

Dirk

[1]

# rust_doctest_kernel_hrtimer_rs_0.location: rust/kernel/hrtimer.rs:10
rust_doctests_kernel: Timer called

=============================
[ BUG: Invalid wait context ]
6.11.0-rc1-arm64 #28 Tainted: G N
-----------------------------
swapper/5/0 is trying to lock:
ffff0004409ab900 (rust/doctests_kernel_generated.rs:1238){+.+.}-{3:3}, at:
rust_helper_mutex_lock+0x10/0x18
other info that might help us debug this:
context-{2:2}
no locks held by swapper/5/0.

stack backtrace:
CPU: 5 UID: 0 PID: 0 Comm: swapper/5 Tainted: G N 6.11.0-rc1-arm64 #28
Tainted: [N]=TEST
Hardware name: ARM64 based board (DT)
Call trace:
 $x.11+0x98/0xb4
 show_stack+0x14/0x1c
 $x.3+0x3c/0x94
 dump_stack+0x14/0x1c
 $x.205+0x538/0x594
 $x.179+0xd0/0x18c
 __mutex_lock+0xa0/0xa4
 mutex_lock_nested+0x20/0x28
 rust_helper_mutex_lock+0x10/0x18

_RNvXs_NvNvNvCslTRHJHclVGW_25doctests_kernel_generated32rust_doctest_kernel_hrtimer_rs_04main41__doctest_main_rust_kernel_hrtimer_rs_10_0NtB4_17ArcIntrusiveTimerNtNtCsclYTRz49wqv_6kernel7hrtimer13TimerCallback3run+0x5c/0xd0

_RNvXs1_NtNtCsclYTRz49wqv_6kernel7hrtimer3arcINtNtNtB9_4sync3arc3ArcNtNvNvNvCslTRHJHclVGW_25doctests_kernel_generated32rust_doctest_kernel_hrtimer_rs_04main41__doctest_main_rust_kernel_hrtimer_rs_10_017ArcIntrusiveTimerENtB7_16RawTimerCallback3runB1b_+0x20/0x2c
 $x.90+0x64/0x70
 hrtimer_interrupt+0x1d4/0x2ac
 arch_timer_handler_phys+0x34/0x40
 $x.62+0x50/0x54
 generic_handle_domain_irq+0x28/0x40
 $x.154+0x58/0x6c
 $x.471+0x10/0x20
 el1_interrupt+0x70/0x94
 el1h_64_irq_handler+0x14/0x1c
 el1h_64_irq+0x64/0x68
 arch_local_irq_enable+0x4/0x8
 cpuidle_enter+0x34/0x48
 $x.37+0x58/0xe4
 cpu_startup_entry+0x30/0x34
 $x.2+0xf8/0x118
 $x.13+0x0/0x4

rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Counted to 5
ok 22 rust_doctest_kernel_hrtimer_rs_0
# rust_doctest_kernel_hrtimer_rs_1.location: rust/kernel/hrtimer.rs:137
rust_doctests_kernel: Hello from the future
rust_doctests_kernel: Flag raised
ok 23 rust_doctest_kernel_hrtimer_rs_1
# rust_doctest_kernel_hrtimer_rs_2.location: rust/kernel/hrtimer.rs:76
rust_doctests_kernel: Timer called
rust_doctests_kernel: Flag raised
ok 24 rust_doctest_kernel_hrtimer_rs_2

#42 lore raw

Am 01.10.24 um 16:42 schrieb Boqun Feng:

Quoted 37 lines · Dirk Behme
Quoted 11 lines · Andreas Hindborg
Quoted 6 lines

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }
Quoted 1 line

has a Mutex<bool>, which actually should be a SpinLockIrq [1].

Two understanding questions:

1. In the main thread (full example for reference below [2]) where is the lock released? After the while loop? I.e. is the lock held until guard reaches 5?

let mut guard = has_timer.flag.lock(); // <= lock taken here?

while *guard != 5 {
     has_timer.cond.wait(&mut guard);
}                                                           // <= lock released here?

I wonder what this would mean for the interrupt TimerCallback in case we would use an irq-off SpinLock instead here?

Or maybe:

2. The only place where the guard is modified (*guard += 1;) is in the TimerCallback which runs in interrupt context as we learned. With that writing the guard value can't be interrupted. Couldn't we drop the whole lock, then?

Best regards

Dirk

[2]

//! #[pin_data]
//! struct ArcIntrusiveTimer {

//!     #[pin]
//!     timer: Timer<Self>,
//!     #[pin]
//!     flag: Mutex<u64>,
//!     #[pin]
//!     cond: CondVar,

//! }
//!
//! impl ArcIntrusiveTimer {
//! fn new() -> impl PinInit<Self, kernel::error::Error> {
//! try_pin_init!(Self {

//!             timer <- Timer::new(),
//!             flag <- new_mutex!(0),
//!             cond <- new_condvar!(),

//! })
//! }
//! }
//!
//! impl TimerCallback for ArcIntrusiveTimer {
//! type CallbackTarget<'a> = Arc<Self>;
//! type CallbackPointer<'a> = Arc<Self>;
//!
//! fn run(this: Self::CallbackTarget<'_>) -> TimerRestart {

//!         pr_info!("Timer called\n");
//!         let mut guard = this.flag.lock();
//!         *guard += 1;
//!         this.cond.notify_all();
//!         if *guard == 5 {

//! TimerRestart::NoRestart
//! }
//! else {
//! TimerRestart::Restart
//!
//! }
//! }
//! }
//!
//! impl_has_timer! {
//! impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
//! }
//!
//!
//! let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?; //! let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000)); //! let mut guard = has_timer.flag.lock();
//!
//! while *guard != 5 {
//! has_timer.cond.wait(&mut guard);
//! }
//!
//! pr_info!("Counted to 5\n");
//! # Ok::<(), kernel::error::Error>(())

Quoted 91 lines

Note that
irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

[*]: I vaguely remember we still have some driver code for slow devices that will enable interrupts during an irq handler, but these are going to be gone, we shouldn't really care about this in Rust code.

Regards,
Boqun

[1]: https://lore.kernel.org/rust-for-linux/[email protected]/

Quoted 68 lines

Best regards

Dirk

[1]

# rust_doctest_kernel_hrtimer_rs_0.location: rust/kernel/hrtimer.rs:10
rust_doctests_kernel: Timer called

=============================
[ BUG: Invalid wait context ]
6.11.0-rc1-arm64 #28 Tainted: G N
-----------------------------
swapper/5/0 is trying to lock:
ffff0004409ab900 (rust/doctests_kernel_generated.rs:1238){+.+.}-{3:3}, at:
rust_helper_mutex_lock+0x10/0x18
other info that might help us debug this:
context-{2:2}
no locks held by swapper/5/0.

stack backtrace:
CPU: 5 UID: 0 PID: 0 Comm: swapper/5 Tainted: G N 6.11.0-rc1-arm64 #28
Tainted: [N]=TEST
Hardware name: ARM64 based board (DT)
Call trace:
  $x.11+0x98/0xb4
  show_stack+0x14/0x1c
  $x.3+0x3c/0x94
  dump_stack+0x14/0x1c
  $x.205+0x538/0x594
  $x.179+0xd0/0x18c
  __mutex_lock+0xa0/0xa4
  mutex_lock_nested+0x20/0x28
  rust_helper_mutex_lock+0x10/0x18

_RNvXs_NvNvNvCslTRHJHclVGW_25doctests_kernel_generated32rust_doctest_kernel_hrtimer_rs_04main41__doctest_main_rust_kernel_hrtimer_rs_10_0NtB4_17ArcIntrusiveTimerNtNtCsclYTRz49wqv_6kernel7hrtimer13TimerCallback3run+0x5c/0xd0

_RNvXs1_NtNtCsclYTRz49wqv_6kernel7hrtimer3arcINtNtNtB9_4sync3arc3ArcNtNvNvNvCslTRHJHclVGW_25doctests_kernel_generated32rust_doctest_kernel_hrtimer_rs_04main41__doctest_main_rust_kernel_hrtimer_rs_10_017ArcIntrusiveTimerENtB7_16RawTimerCallback3runB1b_+0x20/0x2c
  $x.90+0x64/0x70
  hrtimer_interrupt+0x1d4/0x2ac
  arch_timer_handler_phys+0x34/0x40
  $x.62+0x50/0x54
  generic_handle_domain_irq+0x28/0x40
  $x.154+0x58/0x6c
  $x.471+0x10/0x20
  el1_interrupt+0x70/0x94
  el1h_64_irq_handler+0x14/0x1c
  el1h_64_irq+0x64/0x68
  arch_local_irq_enable+0x4/0x8
  cpuidle_enter+0x34/0x48
  $x.37+0x58/0xe4
  cpu_startup_entry+0x30/0x34
  $x.2+0xf8/0x118
  $x.13+0x0/0x4

rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Timer called
rust_doctests_kernel: Counted to 5
ok 22 rust_doctest_kernel_hrtimer_rs_0
# rust_doctest_kernel_hrtimer_rs_1.location: rust/kernel/hrtimer.rs:137
rust_doctests_kernel: Hello from the future
rust_doctests_kernel: Flag raised
ok 23 rust_doctest_kernel_hrtimer_rs_1
# rust_doctest_kernel_hrtimer_rs_2.location: rust/kernel/hrtimer.rs:76
rust_doctests_kernel: Timer called
rust_doctests_kernel: Flag raised
ok 24 rust_doctest_kernel_hrtimer_rs_2

#43 lore raw
Quoted 38 lines · Dirk Behme

Am 01.10.24 um 16:42 schrieb Boqun Feng:

Quoted 37 lines · Dirk Behme
Quoted 11 lines · Andreas Hindborg
Quoted 6 lines

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }
Quoted 5 lines
Quoted 1 line

has a Mutex<bool>, which actually should be a SpinLockIrq [1].

Two understanding questions:

Good questions. ;-)

Quoted 2 lines

1. In the main thread (full example for reference below [2]) where is the lock released? After the while loop? I.e. is the lock held until guard

With the current implementation, there are two places the lock will be
released: 1) inside CondVar::wait() and 2) after guard is eventually drop after the loop.

Quoted 12 lines

reaches 5?

let mut guard = has_timer.flag.lock(); // <= lock taken here?

while *guard != 5 {
     has_timer.cond.wait(&mut guard);
}                                                           // <= lock

released here?

I wonder what this would mean for the interrupt TimerCallback in case we would use an irq-off SpinLock instead here?

Quoted 7 lines

Or maybe:

2. The only place where the guard is modified (*guard += 1;) is in the TimerCallback which runs in interrupt context as we learned. With that writing the guard value can't be interrupted. Couldn't we drop the whole lock, then?

No, because the main thread can run on another CPU, so disabling interrupts (because of the interrupt handlers) doesn't mean exclusive access to value.

Regards,
Boqun

Quoted 64 lines

Best regards

Dirk

[2]

//! #[pin_data]
//! struct ArcIntrusiveTimer {

//!     #[pin]
//!     timer: Timer<Self>,
//!     #[pin]
//!     flag: Mutex<u64>,
//!     #[pin]
//!     cond: CondVar,

//! }
//!
//! impl ArcIntrusiveTimer {
//! fn new() -> impl PinInit<Self, kernel::error::Error> {
//! try_pin_init!(Self {

//!             timer <- Timer::new(),
//!             flag <- new_mutex!(0),
//!             cond <- new_condvar!(),

//! })
//! }
//! }
//!
//! impl TimerCallback for ArcIntrusiveTimer {
//! type CallbackTarget<'a> = Arc<Self>;
//! type CallbackPointer<'a> = Arc<Self>;
//!
//! fn run(this: Self::CallbackTarget<'_>) -> TimerRestart {

//!         pr_info!("Timer called\n");
//!         let mut guard = this.flag.lock();
//!         *guard += 1;
//!         this.cond.notify_all();
//!         if *guard == 5 {

//! TimerRestart::NoRestart
//! }
//! else {
//! TimerRestart::Restart
//!
//! }
//! }
//! }
//!
//! impl_has_timer! {
//! impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
//! }
//!
//!
//! let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?; //! let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000)); //! let mut guard = has_timer.flag.lock();
//!
//! while *guard != 5 {
//! has_timer.cond.wait(&mut guard);
//! }
//!
//! pr_info!("Counted to 5\n");
//! # Ok::<(), kernel::error::Error>(())

[...]

#44 lore raw

Am 03.10.24 um 15:03 schrieb Boqun Feng:

Quoted 39 lines · Dirk Behme
Quoted 38 lines

Am 01.10.24 um 16:42 schrieb Boqun Feng:

Quoted 37 lines · Dirk Behme
Quoted 11 lines · Andreas Hindborg
Quoted 6 lines

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }
Quoted 7 lines
Quoted 5 lines
Quoted 1 line

has a Mutex<bool>, which actually should be a SpinLockIrq [1].

Two understanding questions:

Good questions. ;-)

:-)

Quoted 5 lines
Quoted 2 lines

1. In the main thread (full example for reference below [2]) where is the lock released? After the while loop? I.e. is the lock held until guard

With the current implementation, there are two places the lock will be
released: 1) inside CondVar::wait() and

CondVar::wait() releases *and* reaquires, the lock then? So that outside of CondVar::wait() but inside the while() loop the lock is held until the while loop is exit?

Would that lock handling inside CondVar::wait() handle the irq stuff (irq enable and disable) of SpinLockIrq correctly, then?

Quoted 23 lines

2) after guard is eventually
drop after the loop.

Quoted 12 lines

reaches 5?

let mut guard = has_timer.flag.lock(); // <= lock taken here?

while *guard != 5 {
      has_timer.cond.wait(&mut guard);
}                                                           // <= lock

released here?

I wonder what this would mean for the interrupt TimerCallback in case we would use an irq-off SpinLock instead here?

Quoted 7 lines

Or maybe:

2. The only place where the guard is modified (*guard += 1;) is in the TimerCallback which runs in interrupt context as we learned. With that writing the guard value can't be interrupted. Couldn't we drop the whole lock, then?

Quoted 3 lines

No, because the main thread can run on another CPU, so disabling interrupts (because of the interrupt handlers) doesn't mean exclusive access to value.

Yes. I agree if the main thread would write. But that main thread does read-only accesses, only? So it reads either the old or the new value, indepenent on the locking? Only the interrupt handler does read/modify/write. But thats protected by the interrupt context, already.

Dirk

Quoted 65 lines
Quoted 64 lines

Best regards

Dirk

[2]

//! #[pin_data]
//! struct ArcIntrusiveTimer {

//!     #[pin]
//!     timer: Timer<Self>,
//!     #[pin]
//!     flag: Mutex<u64>,
//!     #[pin]
//!     cond: CondVar,

//! }
//!
//! impl ArcIntrusiveTimer {
//! fn new() -> impl PinInit<Self, kernel::error::Error> {
//! try_pin_init!(Self {

//!             timer <- Timer::new(),
//!             flag <- new_mutex!(0),
//!             cond <- new_condvar!(),

//! })
//! }
//! }
//!
//! impl TimerCallback for ArcIntrusiveTimer {
//! type CallbackTarget<'a> = Arc<Self>;
//! type CallbackPointer<'a> = Arc<Self>;
//!
//! fn run(this: Self::CallbackTarget<'_>) -> TimerRestart {

//!         pr_info!("Timer called\n");
//!         let mut guard = this.flag.lock();
//!         *guard += 1;
//!         this.cond.notify_all();
//!         if *guard == 5 {

//! TimerRestart::NoRestart
//! }
//! else {
//! TimerRestart::Restart
//!
//! }
//! }
//! }
//!
//! impl_has_timer! {
//! impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
//! }
//!
//!
//! let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?; //! let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000)); //! let mut guard = has_timer.flag.lock();
//!
//! while *guard != 5 {
//! has_timer.cond.wait(&mut guard);
//! }
//!
//! pr_info!("Counted to 5\n");
//! # Ok::<(), kernel::error::Error>(())

[...]

author maintainer
#45 lore raw
In reply to Boqun Feng

Dirk, thanks for reporting!

Quoted 41 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

Quoted 7 lines
	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled. Without this, when we get back from bindings::schedule_timeout in CondVar::wait_internal, interrupts are enabled:

```rust
use kernel::{
    hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
    impl_has_timer, new_condvar, new_spinlock, new_spinlock_irq,
    irq::IrqDisabled,
    prelude::*,
    sync::{Arc, ArcBorrow, CondVar, SpinLock, SpinLockIrq},
    time::Ktime,
};

#[pin_data]
struct ArcIntrusiveTimer {
    #[pin]
    timer: Timer<Self>,
    #[pin]
    flag: SpinLockIrq<u64>,
    #[pin]
    cond: CondVar,
}

impl ArcIntrusiveTimer {
    fn new() -> impl PinInit<Self, kernel::error::Error> {
        try_pin_init!(Self {
            timer <- Timer::new(),
            flag <- new_spinlock_irq!(0),
            cond <- new_condvar!(),
        })
    }
}

impl TimerCallback for ArcIntrusiveTimer {
    type CallbackTarget<'a> = Arc<Self>;
    type CallbackTargetParameter<'a> = ArcBorrow<'a, Self>;

    fn run(this: Self::CallbackTargetParameter<'_>, irq: IrqDisabled<'_>) -> TimerRestart {
        pr_info!("Timer called\n");
        let mut guard = this.flag.lock_with(irq);
        *guard += 1;
        this.cond.notify_all();
        if *guard == 5 {
            TimerRestart::NoRestart
        }
        else {
            TimerRestart::Restart

        }
    }
}

impl_has_timer! {
    impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
}


let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));

kernel::irq::with_irqs_disabled(|irq| {
  let mut guard = has_timer.flag.lock_with(irq);

  while *guard != 5 {
      pr_info!("Not 5 yet, waiting\n");
      has_timer.cond.wait(&mut guard); // <-- we arrive back here with interrupts enabled!
  }
});
```

I think an update of CondVar::wait should be part of the patch set [1].

Best regards,
Andreas

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#46 lore raw

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

What do you think?

Thanks

Dirk

Quoted 81 lines

Without
this, when we get back from bindings::schedule_timeout in CondVar::wait_internal, interrupts are enabled:

```rust
use kernel::{
     hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
     impl_has_timer, new_condvar, new_spinlock, new_spinlock_irq,
     irq::IrqDisabled,
     prelude::*,
     sync::{Arc, ArcBorrow, CondVar, SpinLock, SpinLockIrq},
     time::Ktime,
};

#[pin_data]
struct ArcIntrusiveTimer {
     #[pin]
     timer: Timer<Self>,
     #[pin]
     flag: SpinLockIrq<u64>,
     #[pin]
     cond: CondVar,
}

impl ArcIntrusiveTimer {
     fn new() -> impl PinInit<Self, kernel::error::Error> {
         try_pin_init!(Self {
             timer <- Timer::new(),
             flag <- new_spinlock_irq!(0),
             cond <- new_condvar!(),
         })
     }
}

impl TimerCallback for ArcIntrusiveTimer {
     type CallbackTarget<'a> = Arc<Self>;
     type CallbackTargetParameter<'a> = ArcBorrow<'a, Self>;

     fn run(this: Self::CallbackTargetParameter<'_>, irq: IrqDisabled<'_>) -> TimerRestart {
         pr_info!("Timer called\n");
         let mut guard = this.flag.lock_with(irq);
         *guard += 1;
         this.cond.notify_all();
         if *guard == 5 {
             TimerRestart::NoRestart
         }
         else {
             TimerRestart::Restart

         }
     }
}

impl_has_timer! {
     impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
}


let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));

kernel::irq::with_irqs_disabled(|irq| {
   let mut guard = has_timer.flag.lock_with(irq);

   while *guard != 5 {
       pr_info!("Not 5 yet, waiting\n");
       has_timer.cond.wait(&mut guard); // <-- we arrive back here with interrupts enabled!
   }
});
```

I think an update of CondVar::wait should be part of the patch set [1].

Best regards,
Andreas

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#47 lore raw
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Regards,
Boqun

Quoted 87 lines

Thanks

Dirk

Quoted 81 lines

Without
this, when we get back from bindings::schedule_timeout in CondVar::wait_internal, interrupts are enabled:

```rust
use kernel::{
     hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
     impl_has_timer, new_condvar, new_spinlock, new_spinlock_irq,
     irq::IrqDisabled,
     prelude::*,
     sync::{Arc, ArcBorrow, CondVar, SpinLock, SpinLockIrq},
     time::Ktime,
};

#[pin_data]
struct ArcIntrusiveTimer {
     #[pin]
     timer: Timer<Self>,
     #[pin]
     flag: SpinLockIrq<u64>,
     #[pin]
     cond: CondVar,
}

impl ArcIntrusiveTimer {
     fn new() -> impl PinInit<Self, kernel::error::Error> {
         try_pin_init!(Self {
             timer <- Timer::new(),
             flag <- new_spinlock_irq!(0),
             cond <- new_condvar!(),
         })
     }
}

impl TimerCallback for ArcIntrusiveTimer {
     type CallbackTarget<'a> = Arc<Self>;
     type CallbackTargetParameter<'a> = ArcBorrow<'a, Self>;

     fn run(this: Self::CallbackTargetParameter<'_>, irq: IrqDisabled<'_>) -> TimerRestart {
         pr_info!("Timer called\n");
         let mut guard = this.flag.lock_with(irq);
         *guard += 1;
         this.cond.notify_all();
         if *guard == 5 {
             TimerRestart::NoRestart
         }
         else {
             TimerRestart::Restart

         }
     }
}

impl_has_timer! {
     impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
}


let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));

kernel::irq::with_irqs_disabled(|irq| {
   let mut guard = has_timer.flag.lock_with(irq);

   while *guard != 5 {
       pr_info!("Not 5 yet, waiting\n");
       has_timer.cond.wait(&mut guard); // <-- we arrive back here with interrupts enabled!
   }
});
```

I think an update of CondVar::wait should be part of the patch set [1].

Best regards,
Andreas

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#48 lore raw
Quoted 76 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 4 lines
Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the impression that it doesn't matter if the old or new guard value is read in this special case.

Best regards

Dirk

Quoted 90 lines

Regards,
Boqun

Quoted 87 lines

Thanks

Dirk

Quoted 81 lines

Without
this, when we get back from bindings::schedule_timeout in CondVar::wait_internal, interrupts are enabled:

```rust
use kernel::{
      hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
      impl_has_timer, new_condvar, new_spinlock, new_spinlock_irq,
      irq::IrqDisabled,
      prelude::*,
      sync::{Arc, ArcBorrow, CondVar, SpinLock, SpinLockIrq},
      time::Ktime,
};

#[pin_data]
struct ArcIntrusiveTimer {
      #[pin]
      timer: Timer<Self>,
      #[pin]
      flag: SpinLockIrq<u64>,
      #[pin]
      cond: CondVar,
}

impl ArcIntrusiveTimer {
      fn new() -> impl PinInit<Self, kernel::error::Error> {
          try_pin_init!(Self {
              timer <- Timer::new(),
              flag <- new_spinlock_irq!(0),
              cond <- new_condvar!(),
          })
      }
}

impl TimerCallback for ArcIntrusiveTimer {
      type CallbackTarget<'a> = Arc<Self>;
      type CallbackTargetParameter<'a> = ArcBorrow<'a, Self>;

      fn run(this: Self::CallbackTargetParameter<'_>, irq: IrqDisabled<'_>) -> TimerRestart {
          pr_info!("Timer called\n");
          let mut guard = this.flag.lock_with(irq);
          *guard += 1;
          this.cond.notify_all();
          if *guard == 5 {
              TimerRestart::NoRestart
          }
          else {
              TimerRestart::Restart

          }
      }
}

impl_has_timer! {
      impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
}


let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));

kernel::irq::with_irqs_disabled(|irq| {
    let mut guard = has_timer.flag.lock_with(irq);

    while *guard != 5 {
        pr_info!("Not 5 yet, waiting\n");
        has_timer.cond.wait(&mut guard); // <-- we arrive back here with interrupts enabled!
    }
});
```

I think an update of CondVar::wait should be part of the patch set [1].

Best regards,
Andreas

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#49 lore raw
Quoted 77 lines · Dirk Behme
Quoted 76 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being UB).

Quoted 3 lines

impression that it doesn't matter if the old or new guard value is read in this special case.

For one thing, if the compiler believes no one is accessing the value because the code uses an immutable reference, it can "optimize" the loop away:

	while *var != 5 {
	    do_something();
	}

into

	if *var != 5 {
	    loop { do_something(); }
	}

But as I said, I need to see the exact code to suggest a relevant mis-compile, and note that sometimes, even mis-compile seems impossible at the moment, a UB is a UB, compilers are free to do anything they want (or don't want). So "mis-compile" is only helping we understand the potential result of a UB.

Regards,
Boqun

Quoted 96 lines

Best regards

Dirk

Quoted 90 lines

Regards,
Boqun

Quoted 87 lines

Thanks

Dirk

Quoted 81 lines

Without
this, when we get back from bindings::schedule_timeout in CondVar::wait_internal, interrupts are enabled:

```rust
use kernel::{
      hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
      impl_has_timer, new_condvar, new_spinlock, new_spinlock_irq,
      irq::IrqDisabled,
      prelude::*,
      sync::{Arc, ArcBorrow, CondVar, SpinLock, SpinLockIrq},
      time::Ktime,
};

#[pin_data]
struct ArcIntrusiveTimer {
      #[pin]
      timer: Timer<Self>,
      #[pin]
      flag: SpinLockIrq<u64>,
      #[pin]
      cond: CondVar,
}

impl ArcIntrusiveTimer {
      fn new() -> impl PinInit<Self, kernel::error::Error> {
          try_pin_init!(Self {
              timer <- Timer::new(),
              flag <- new_spinlock_irq!(0),
              cond <- new_condvar!(),
          })
      }
}

impl TimerCallback for ArcIntrusiveTimer {
      type CallbackTarget<'a> = Arc<Self>;
      type CallbackTargetParameter<'a> = ArcBorrow<'a, Self>;

      fn run(this: Self::CallbackTargetParameter<'_>, irq: IrqDisabled<'_>) -> TimerRestart {
          pr_info!("Timer called\n");
          let mut guard = this.flag.lock_with(irq);
          *guard += 1;
          this.cond.notify_all();
          if *guard == 5 {
              TimerRestart::NoRestart
          }
          else {
              TimerRestart::Restart

          }
      }
}

impl_has_timer! {
      impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
}


let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));

kernel::irq::with_irqs_disabled(|irq| {
    let mut guard = has_timer.flag.lock_with(irq);

    while *guard != 5 {
        pr_info!("Not 5 yet, waiting\n");
        has_timer.cond.wait(&mut guard); // <-- we arrive back here with interrupts enabled!
    }
});
```

I think an update of CondVar::wait should be part of the patch set [1].

Best regards,
Andreas

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#50 lore raw
Quoted 83 lines · Boqun Feng
Quoted 82 lines · Dirk Behme
Quoted 80 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Quoted 7 lines
Quoted 2 lines

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being UB).

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
      #[pin]
      timer: Timer<Self>,
      #[pin]
-      flag: SpinLockIrq<u64>,
+      flag: u64,
      #[pin]
      cond: CondVar,
}

?

Best regards

Dirk

Quoted 123 lines
Quoted 3 lines

impression that it doesn't matter if the old or new guard value is read in this special case.

For one thing, if the compiler believes no one is accessing the value because the code uses an immutable reference, it can "optimize" the loop away:

	while *var != 5 {
	    do_something();
	}

into

	if *var != 5 {
	    loop { do_something(); }
	}

But as I said, I need to see the exact code to suggest a relevant mis-compile, and note that sometimes, even mis-compile seems impossible at the moment, a UB is a UB, compilers are free to do anything they want (or don't want). So "mis-compile" is only helping we understand the potential result of a UB.

Regards,
Boqun

Quoted 96 lines

Best regards

Dirk

Quoted 90 lines

Regards,
Boqun

Quoted 87 lines

Thanks

Dirk

Quoted 81 lines

Without
this, when we get back from bindings::schedule_timeout in CondVar::wait_internal, interrupts are enabled:

```rust
use kernel::{
       hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
       impl_has_timer, new_condvar, new_spinlock, new_spinlock_irq,
       irq::IrqDisabled,
       prelude::*,
       sync::{Arc, ArcBorrow, CondVar, SpinLock, SpinLockIrq},
       time::Ktime,
};

#[pin_data]
struct ArcIntrusiveTimer {
       #[pin]
       timer: Timer<Self>,
       #[pin]
       flag: SpinLockIrq<u64>,
       #[pin]
       cond: CondVar,
}

impl ArcIntrusiveTimer {
       fn new() -> impl PinInit<Self, kernel::error::Error> {
           try_pin_init!(Self {
               timer <- Timer::new(),
               flag <- new_spinlock_irq!(0),
               cond <- new_condvar!(),
           })
       }
}

impl TimerCallback for ArcIntrusiveTimer {
       type CallbackTarget<'a> = Arc<Self>;
       type CallbackTargetParameter<'a> = ArcBorrow<'a, Self>;

       fn run(this: Self::CallbackTargetParameter<'_>, irq: IrqDisabled<'_>) -> TimerRestart {
           pr_info!("Timer called\n");
           let mut guard = this.flag.lock_with(irq);
           *guard += 1;
           this.cond.notify_all();
           if *guard == 5 {
               TimerRestart::NoRestart
           }
           else {
               TimerRestart::Restart

           }
       }
}

impl_has_timer! {
       impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
}


let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));

kernel::irq::with_irqs_disabled(|irq| {
     let mut guard = has_timer.flag.lock_with(irq);

     while *guard != 5 {
         pr_info!("Not 5 yet, waiting\n");
         has_timer.cond.wait(&mut guard); // <-- we arrive back here with interrupts enabled!
     }
});
```

I think an update of CondVar::wait should be part of the patch set [1].

Best regards,
Andreas

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#51 lore raw
Quoted 106 lines · Dirk Behme
Quoted 83 lines · Boqun Feng
Quoted 82 lines · Dirk Behme
Quoted 80 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Quoted 7 lines
Quoted 2 lines

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being UB).

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
      #[pin]
      timer: Timer<Self>,
      #[pin]
(quoted diff) +1 -1
-      flag: SpinLockIrq<u64>,
+      flag: u64,
      #[pin]
      cond: CondVar,

}

Quoted 2 lines

?

Yes, but have you tried to actually use that for the example from Andreas? I think you will find that you cannot write to flag inside the timer callback, because you only has a Arc<ArcIntrusiveTimer>, so not mutable reference for ArcIntrusiveTimer. You can of course use unsafe to create a mutable reference to flag, but it won't be sound, since you are getting a mutable reference from an immutable reference.

Regards,
Boqun

Quoted 129 lines

Best regards

Dirk

Quoted 123 lines
Quoted 3 lines

impression that it doesn't matter if the old or new guard value is read in this special case.

For one thing, if the compiler believes no one is accessing the value because the code uses an immutable reference, it can "optimize" the loop away:

	while *var != 5 {
	    do_something();
	}

into

	if *var != 5 {
	    loop { do_something(); }
	}

But as I said, I need to see the exact code to suggest a relevant mis-compile, and note that sometimes, even mis-compile seems impossible at the moment, a UB is a UB, compilers are free to do anything they want (or don't want). So "mis-compile" is only helping we understand the potential result of a UB.

Regards,
Boqun

Quoted 96 lines

Best regards

Dirk

Quoted 90 lines

Regards,
Boqun

Quoted 87 lines

Thanks

Dirk

Quoted 81 lines

Without
this, when we get back from bindings::schedule_timeout in CondVar::wait_internal, interrupts are enabled:

```rust
use kernel::{
       hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
       impl_has_timer, new_condvar, new_spinlock, new_spinlock_irq,
       irq::IrqDisabled,
       prelude::*,
       sync::{Arc, ArcBorrow, CondVar, SpinLock, SpinLockIrq},
       time::Ktime,
};

#[pin_data]
struct ArcIntrusiveTimer {
       #[pin]
       timer: Timer<Self>,
       #[pin]
       flag: SpinLockIrq<u64>,
       #[pin]
       cond: CondVar,
}

impl ArcIntrusiveTimer {
       fn new() -> impl PinInit<Self, kernel::error::Error> {
           try_pin_init!(Self {
               timer <- Timer::new(),
               flag <- new_spinlock_irq!(0),
               cond <- new_condvar!(),
           })
       }
}

impl TimerCallback for ArcIntrusiveTimer {
       type CallbackTarget<'a> = Arc<Self>;
       type CallbackTargetParameter<'a> = ArcBorrow<'a, Self>;

       fn run(this: Self::CallbackTargetParameter<'_>, irq: IrqDisabled<'_>) -> TimerRestart {
           pr_info!("Timer called\n");
           let mut guard = this.flag.lock_with(irq);
           *guard += 1;
           this.cond.notify_all();
           if *guard == 5 {
               TimerRestart::NoRestart
           }
           else {
               TimerRestart::Restart

           }
       }
}

impl_has_timer! {
       impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
}


let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));

kernel::irq::with_irqs_disabled(|irq| {
     let mut guard = has_timer.flag.lock_with(irq);

     while *guard != 5 {
         pr_info!("Not 5 yet, waiting\n");
         has_timer.cond.wait(&mut guard); // <-- we arrive back here with interrupts enabled!
     }
});
```

I think an update of CondVar::wait should be part of the patch set [1].

Best regards,
Andreas

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#52 lore raw
Quoted 110 lines · Boqun Feng
Quoted 106 lines · Dirk Behme
Quoted 83 lines · Boqun Feng
Quoted 82 lines · Dirk Behme
Quoted 80 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Quoted 7 lines
Quoted 2 lines

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being UB).

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
       #[pin]
       timer: Timer<Self>,
       #[pin]
(quoted diff) +1 -1
-      flag: SpinLockIrq<u64>,
+      flag: u64,
       #[pin]
       cond: CondVar,

}

Quoted 2 lines

?

Quoted 6 lines

Yes, but have you tried to actually use that for the example from Andreas? I think you will find that you cannot write to flag inside the timer callback, because you only has a Arc<ArcIntrusiveTimer>, so not mutable reference for ArcIntrusiveTimer. You can of course use unsafe to create a mutable reference to flag, but it won't be sound, since you are getting a mutable reference from an immutable reference.

Yes, of course. But, hmm, wouldn't that unsoundness be independent on the topic we discuss here? I mean we are talking about getting the compiler to read/modify/write 'flag' in the TimerCallback. *How* we tell him to do so should be independent on the result what we want to look at regarding the locking requirements of 'flag'?

Anyhow, my root motivation was to simplify Andreas example to not use a lock where not strictly required. And with this make Andreas example independent on mutex lockdep issues, SpinLockIrq changes and possible required CondVar updates. But maybe we find an other way to simplify it and decrease the dependencies. In the end its just example code ;)

Best regards

Dirk

Quoted 132 lines

Regards,
Boqun

Quoted 129 lines

Best regards

Dirk

Quoted 123 lines
Quoted 3 lines

impression that it doesn't matter if the old or new guard value is read in this special case.

For one thing, if the compiler believes no one is accessing the value because the code uses an immutable reference, it can "optimize" the loop away:

	while *var != 5 {
	    do_something();
	}

into

	if *var != 5 {
	    loop { do_something(); }
	}

But as I said, I need to see the exact code to suggest a relevant mis-compile, and note that sometimes, even mis-compile seems impossible at the moment, a UB is a UB, compilers are free to do anything they want (or don't want). So "mis-compile" is only helping we understand the potential result of a UB.

Regards,
Boqun

Quoted 96 lines

Best regards

Dirk

Quoted 90 lines

Regards,
Boqun

Quoted 87 lines

Thanks

Dirk

Quoted 81 lines

Without
this, when we get back from bindings::schedule_timeout in CondVar::wait_internal, interrupts are enabled:

```rust
use kernel::{
        hrtimer::{Timer, TimerCallback, TimerPointer, TimerRestart},
        impl_has_timer, new_condvar, new_spinlock, new_spinlock_irq,
        irq::IrqDisabled,
        prelude::*,
        sync::{Arc, ArcBorrow, CondVar, SpinLock, SpinLockIrq},
        time::Ktime,
};

#[pin_data]
struct ArcIntrusiveTimer {
        #[pin]
        timer: Timer<Self>,
        #[pin]
        flag: SpinLockIrq<u64>,
        #[pin]
        cond: CondVar,
}

impl ArcIntrusiveTimer {
        fn new() -> impl PinInit<Self, kernel::error::Error> {
            try_pin_init!(Self {
                timer <- Timer::new(),
                flag <- new_spinlock_irq!(0),
                cond <- new_condvar!(),
            })
        }
}

impl TimerCallback for ArcIntrusiveTimer {
        type CallbackTarget<'a> = Arc<Self>;
        type CallbackTargetParameter<'a> = ArcBorrow<'a, Self>;

        fn run(this: Self::CallbackTargetParameter<'_>, irq: IrqDisabled<'_>) -> TimerRestart {
            pr_info!("Timer called\n");
            let mut guard = this.flag.lock_with(irq);
            *guard += 1;
            this.cond.notify_all();
            if *guard == 5 {
                TimerRestart::NoRestart
            }
            else {
                TimerRestart::Restart

            }
        }
}

impl_has_timer! {
        impl HasTimer<Self> for ArcIntrusiveTimer { self.timer }
}


let has_timer = Arc::pin_init(ArcIntrusiveTimer::new(), GFP_KERNEL)?;
let _handle = has_timer.clone().schedule(Ktime::from_ns(200_000_000));

kernel::irq::with_irqs_disabled(|irq| {
      let mut guard = has_timer.flag.lock_with(irq);

      while *guard != 5 {
          pr_info!("Not 5 yet, waiting\n");
          has_timer.cond.wait(&mut guard); // <-- we arrive back here with interrupts enabled!
      }
});
```

I think an update of CondVar::wait should be part of the patch set [1].

Best regards,
Andreas

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#53 lore raw
Quoted 92 lines · Dirk Behme
Quoted 91 lines · Boqun Feng
Quoted 90 lines · Dirk Behme
Quoted 89 lines · Boqun Feng
Quoted 77 lines · Dirk Behme
Quoted 76 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being

[...]

Quoted 23 lines
Quoted 16 lines
Quoted 13 lines

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
       #[pin]
       timer: Timer<Self>,
       #[pin]
(quoted diff) +1 -1
-      flag: SpinLockIrq<u64>,
+      flag: u64,
       #[pin]
       cond: CondVar,

}

Quoted 2 lines

?

Quoted 6 lines

Yes, but have you tried to actually use that for the example from Andreas? I think you will find that you cannot write to flag inside the timer callback, because you only has a Arc<ArcIntrusiveTimer>, so not mutable reference for ArcIntrusiveTimer. You can of course use unsafe to create a mutable reference to flag, but it won't be sound, since you are getting a mutable reference from an immutable reference.

Quoted 2 lines

Yes, of course. But, hmm, wouldn't that unsoundness be independent on the topic we discuss here? I mean we are talking about getting the compiler to

What do you mean? If the code is unsound, you won't want to use it in an example, right?

Quoted 6 lines

read/modify/write 'flag' in the TimerCallback. *How* we tell him to do so should be independent on the result what we want to look at regarding the locking requirements of 'flag'?

Anyhow, my root motivation was to simplify Andreas example to not use a lock where not strictly required. And with this make Andreas example independent

Well, if you don't want to use a lock then you need to use atomics, otherwise it's likely a UB, but atomics are still WIP, so that why I suggested Andreas to use a lock first. But I guess I didn't realise the lock needs to be irq-safe when I suggested that.

Regards,
Boqun

Quoted 12 lines

on mutex lockdep issues, SpinLockIrq changes and possible required CondVar updates. But maybe we find an other way to simplify it and decrease the dependencies. In the end its just example code ;)

Best regards

Dirk

Quoted 3 lines

Regards,
Boqun

[...]

#54 lore raw
Quoted 117 lines · Boqun Feng
Quoted 92 lines · Dirk Behme
Quoted 91 lines · Boqun Feng
Quoted 90 lines · Dirk Behme
Quoted 89 lines · Boqun Feng
Quoted 77 lines · Dirk Behme
Quoted 76 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the hrtimer subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being

[...]

Quoted 23 lines
Quoted 22 lines
Quoted 13 lines

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
        #[pin]
        timer: Timer<Self>,
        #[pin]
(quoted diff) +1 -1
-      flag: SpinLockIrq<u64>,
+      flag: u64,
        #[pin]
        cond: CondVar,

}

Quoted 2 lines

?

Yes, but have you tried to actually use that for the example from Andreas? I think you will find that you cannot write to flag inside the timer callback, because you only has a Arc<ArcIntrusiveTimer>, so not mutable reference for ArcIntrusiveTimer. You can of course use unsafe to create a mutable reference to flag, but it won't be sound, since you are getting a mutable reference from an immutable reference.

Quoted 5 lines
Quoted 2 lines

Yes, of course. But, hmm, wouldn't that unsoundness be independent on the topic we discuss here? I mean we are talking about getting the compiler to

What do you mean? If the code is unsound, you won't want to use it in an example, right?

Yes, sure. But ;)

In a first step I just wanted to answer the question if we do need a lock at all in this special example. And that we could do even with unsound read/modify/write I would guess. And then, in a second step, if the answer would be "we don't need the lock", then we could think about how to make the flag handling sound. So I'm talking just about answering that question, not about the final example code. Step by step :)

Quoted 9 lines
Quoted 6 lines

read/modify/write 'flag' in the TimerCallback. *How* we tell him to do so should be independent on the result what we want to look at regarding the locking requirements of 'flag'?

Anyhow, my root motivation was to simplify Andreas example to not use a lock where not strictly required. And with this make Andreas example independent

Well, if you don't want to use a lock then you need to use atomics, otherwise it's likely a UB,

And here we are back to the initial question :) Why would it be UB without lock (and atomics)?

Some (pseudo) assembly:

Lets start with the main thread:

ldr x1, [x0]
<work with x1>

x0 and x1 are registers. x0 contains the address of flag in the main memory. I.e. that instruction reads (ldr == load) the content of that memory location (flag) into x1. x1 then contains flag which can be used then. This is what I mean with "the main thread is read only". If flag, i.e. x1, does contain the old or new flag value doesn't matter. I.e. for the read only operation it doesn't matter if it is protected by a lock as the load (ldr) can't be interrupted.

Now to the TimerCallback:

ldr x1, [x0]
add x1, x1, #1
str x1, [x0]

This is what I mean with read/modify/write. And this needs to be ensured that it is not interruptable. I.e. that we are scheduled between ldr and add or between add and str. Yes, I *totally* agree that for this a lock is needed:

<lock>
ldr x1, [x0]
add x1, x1, #1
str x1, [x0]
<unlock>

But:

In this this special example we know that we are executing this code in interrupt context. I.e.:

<interrupts are disabled>
ldr x1, [x0]
add x1, x1, #1
str x1, [x0]
<interrupts are still disabled>

So this read/modify/write can't be interrupted because the interrupts are off. I.e. the interrupt off prevents the scheduling here. And in this sense replaces the lock. And as mentioned, which value is read by the main thread doesn't matter.

To summarize: I totally agree that usually a lock would be needed. But in this special case with (a) read/modify/write in interrupt context *and* (b) read only in main thread I'm unclear.

So with this back to the main question: What is my misunderstanding here? I.e. what is UB in this special case? :)

Best regards

Dirk

Quoted 20 lines

but atomics are still WIP, so that why I
suggested Andreas to use a lock first. But I guess I didn't realise the lock needs to be irq-safe when I suggested that.

Regards,
Boqun

Quoted 12 lines

on mutex lockdep issues, SpinLockIrq changes and possible required CondVar updates. But maybe we find an other way to simplify it and decrease the dependencies. In the end its just example code ;)

Best regards

Dirk

Quoted 3 lines

Regards,
Boqun

[...]

author maintainer
#55 lore raw
Quoted 138 lines · Dirk Behme
Quoted 117 lines · Boqun Feng
Quoted 92 lines · Dirk Behme
Quoted 91 lines · Boqun Feng
Quoted 90 lines · Dirk Behme
Quoted 89 lines · Boqun Feng
Quoted 77 lines · Dirk Behme
Quoted 76 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the `hrtimer` subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will
ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

	spin_lock(&a);
	> timer interrupt
	  spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being

[...]

Quoted 23 lines
Quoted 22 lines
Quoted 13 lines

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
        #[pin]
        timer: Timer<Self>,
        #[pin]
(quoted diff) +1 -1
-      flag: SpinLockIrq<u64>,
+      flag: u64,
        #[pin]
        cond: CondVar,

}

Quoted 2 lines

?

Yes, but have you tried to actually use that for the example from Andreas? I think you will find that you cannot write to flag inside the timer callback, because you only has a Arc<ArcIntrusiveTimer>, so not mutable reference for ArcIntrusiveTimer. You can of course use unsafe to create a mutable reference to flag, but it won't be sound, since you are getting a mutable reference from an immutable reference.

Quoted 5 lines
Quoted 2 lines

Yes, of course. But, hmm, wouldn't that unsoundness be independent on the topic we discuss here? I mean we are talking about getting the compiler to

What do you mean? If the code is unsound, you won't want to use it in an example, right?

Yes, sure. But ;)

In a first step I just wanted to answer the question if we do need a lock at all in this special example. And that we could do even with unsound read/modify/write I would guess. And then, in a second step, if the answer would be "we don't need the lock", then we could think about how to make the flag handling sound. So I'm talking just about answering that question, not about the final example code. Step by step :)

Quoted 4 lines
Quoted 4 lines

read/modify/write 'flag' in the TimerCallback. *How* we tell him to do so should be independent on the result what we want to look at regarding the locking requirements of 'flag'?

Quoted 8 lines
Quoted 5 lines
Quoted 2 lines

Anyhow, my root motivation was to simplify Andreas example to not use a lock where not strictly required. And with this make Andreas example independent

Well, if you don't want to use a lock then you need to use atomics, otherwise it's likely a UB,

And here we are back to the initial question :) Why would it be UB without lock (and atomics)?

It is UB at the language level. Miri will yell at you. If you do this, the compiler will give you zero guarantees.

Quoted 43 lines

Some (pseudo) assembly:

Lets start with the main thread:

ldr x1, [x0]
<work with x1>

x0 and x1 are registers. x0 contains the address of flag in the main memory. I.e. that instruction reads (ldr == load) the content of that memory location (flag) into x1. x1 then contains flag which can be used then. This is what I mean with "the main thread is read only". If flag, i.e. x1, does contain the old or new flag value doesn't matter. I.e. for the read only operation it doesn't matter if it is protected by a lock as the load (ldr) can't be interrupted.

Now to the TimerCallback:

ldr x1, [x0]
add x1, x1, #1
str x1, [x0]

This is what I mean with read/modify/write. And this needs to be ensured that it is not interruptable. I.e. that we are scheduled between ldr and add or between add and str. Yes, I *totally* agree that for this a lock is needed:

<lock>
ldr x1, [x0]
add x1, x1, #1
str x1, [x0]
<unlock>

But:

In this this special example we know that we are executing this code in interrupt context. I.e.:

<interrupts are disabled>
ldr x1, [x0]
add x1, x1, #1
str x1, [x0]
<interrupts are still disabled>

Quoted 4 lines

So this read/modify/write can't be interrupted because the interrupts are off. I.e. the interrupt off prevents the scheduling here. And in this sense replaces the lock. And as mentioned, which value is read by the main thread doesn't matter.

You can have the interrupt handler running on one core and the process on another core. For uni-processor systems you are right. I actually think spinlock operations collapse to no-ops on non-SMP configurations. Bur for SMP configurations, this would be broken.

I don't think the rust language cares that though. Doing this kind of modification from multiple execution contexts without synchronization is always UB in rust.

BR Andreas

maintainer
#56 lore raw
In reply to Dirk Behme
Quoted 155 lines · Dirk Behme
Quoted 117 lines · Boqun Feng
Quoted 92 lines · Dirk Behme
Quoted 91 lines · Boqun Feng
Quoted 90 lines · Dirk Behme
Quoted 89 lines · Boqun Feng
Quoted 77 lines · Dirk Behme
Quoted 76 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 11 lines · Dirk Behme
Quoted 6 lines · Andreas Hindborg

Hi!

This series adds support for using the `hrtimer` subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will
ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest) Example from hrtimer.rs.

Quoted 6 lines

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

     spin_lock(&a);
     > timer interrupt
       spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being

[...]

Quoted 23 lines
Quoted 22 lines
Quoted 13 lines

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
        #[pin]
        timer: Timer<Self>,
        #[pin]
(quoted diff) +1 -1
-      flag: SpinLockIrq<u64>,
+      flag: u64,
        #[pin]
        cond: CondVar,

}

Quoted 2 lines

?

Yes, but have you tried to actually use that for the example from Andreas? I think you will find that you cannot write to flag inside the timer callback, because you only has a Arc<ArcIntrusiveTimer>, so not mutable reference for ArcIntrusiveTimer. You can of course use unsafe to create a mutable reference to flag, but it won't be sound, since you are getting a mutable reference from an immutable reference.

Quoted 5 lines
Quoted 2 lines

Yes, of course. But, hmm, wouldn't that unsoundness be independent on the topic we discuss here? I mean we are talking about getting the compiler to

What do you mean? If the code is unsound, you won't want to use it in an example, right?

Yes, sure. But ;)

In a first step I just wanted to answer the question if we do need a lock at all in this special example. And that we could do even with unsound read/modify/write I would guess. And then, in a second step, if the answer would be "we don't need the lock", then we could think about how to make the flag handling sound. So I'm talking just about answering that question, not about the final example code. Step by step :)

Quoted 9 lines
Quoted 6 lines

read/modify/write 'flag' in the TimerCallback. *How* we tell him to do so should be independent on the result what we want to look at regarding the locking requirements of 'flag'?

Anyhow, my root motivation was to simplify Andreas example to not use a lock where not strictly required. And with this make Andreas example independent

Well, if you don't want to use a lock then you need to use atomics, otherwise it's likely a UB,

And here we are back to the initial question :) Why would it be UB without lock (and atomics)?

Some (pseudo) assembly:

Lets start with the main thread:

ldr x1, [x0]
<work with x1>

Quoted 7 lines

x0 and x1 are registers. x0 contains the address of flag in the main memory. I.e. that instruction reads (ldr == load) the content of that memory location (flag) into x1. x1 then contains flag which can be used then. This is what I mean with "the main thread is read only". If flag, i.e. x1, does contain the old or new flag value doesn't matter. I.e. for the read only operation it doesn't matter if it is protected by a lock as the load (ldr) can't be interrupted.

If the compiler generates a single load, then sure. But for an unsynchronized load, the compiler may generate two separate load instructions and assume that both loads read the same value.

Alice

#57 lore raw

Hi Alice,

Quoted 164 lines · Alice Ryhl
Quoted 155 lines · Dirk Behme
Quoted 117 lines · Boqun Feng
Quoted 92 lines · Dirk Behme
Quoted 91 lines · Boqun Feng
Quoted 90 lines · Dirk Behme
Quoted 89 lines · Boqun Feng
Quoted 77 lines · Dirk Behme
Quoted 76 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng
Quoted 17 lines · Dirk Behme

On 18.09.2024 00:27, Andreas Hindborg wrote:
Hi!

This series adds support for using the `hrtimer` subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will
ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest)
Example from hrtimer.rs.

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in
interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

(quoted diff) +8 -0
+//! struct ArcIntrusiveTimer {
+//!     #[pin]
+//!     timer: Timer<Self>,
+//!     #[pin]
+//!     flag: Mutex<bool>,
+//!     #[pin]
+//!     cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that irq-off is needed for the lock, because otherwise we will hit a self deadlock due to interrupts:

      spin_lock(&a);
      > timer interrupt
        spin_lock(&a);
Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by ::new(), because irq contexts should guarantee interrupt disabled (i.e. we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being

[...]

Quoted 23 lines
Quoted 22 lines
Quoted 13 lines

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
         #[pin]
         timer: Timer<Self>,
         #[pin]
(quoted diff) +1 -1
-      flag: SpinLockIrq<u64>,
+      flag: u64,
         #[pin]
         cond: CondVar,

}

Quoted 2 lines

?

Yes, but have you tried to actually use that for the example from Andreas? I think you will find that you cannot write to flag inside the timer callback, because you only has a Arc<ArcIntrusiveTimer>, so not mutable reference for ArcIntrusiveTimer. You can of course use unsafe to create a mutable reference to flag, but it won't be sound, since you are getting a mutable reference from an immutable reference.

Quoted 5 lines
Quoted 2 lines

Yes, of course. But, hmm, wouldn't that unsoundness be independent on the topic we discuss here? I mean we are talking about getting the compiler to

What do you mean? If the code is unsound, you won't want to use it in an example, right?

Yes, sure. But ;)

In a first step I just wanted to answer the question if we do need a lock at all in this special example. And that we could do even with unsound read/modify/write I would guess. And then, in a second step, if the answer would be "we don't need the lock", then we could think about how to make the flag handling sound. So I'm talking just about answering that question, not about the final example code. Step by step :)

Quoted 9 lines
Quoted 6 lines

read/modify/write 'flag' in the TimerCallback. *How* we tell him to do so should be independent on the result what we want to look at regarding the locking requirements of 'flag'?

Anyhow, my root motivation was to simplify Andreas example to not use a lock where not strictly required. And with this make Andreas example independent

Well, if you don't want to use a lock then you need to use atomics, otherwise it's likely a UB,

And here we are back to the initial question :) Why would it be UB without lock (and atomics)?

Some (pseudo) assembly:

Lets start with the main thread:

ldr x1, [x0]
<work with x1>

Quoted 7 lines

x0 and x1 are registers. x0 contains the address of flag in the main memory. I.e. that instruction reads (ldr == load) the content of that memory location (flag) into x1. x1 then contains flag which can be used then. This is what I mean with "the main thread is read only". If flag, i.e. x1, does contain the old or new flag value doesn't matter. I.e. for the read only operation it doesn't matter if it is protected by a lock as the load (ldr) can't be interrupted.

Quoted 1 line

If the compiler generates a single load, then sure.

Yes :)

Quoted 3 lines

But for an
unsynchronized load, the compiler may generate two separate load instructions and assume that both loads read the same value.

Ok, yes, if we get this from the compiler I agree that we need the lock, even if its just for the read. If I get the chance the next time I will try to have a look to the compiler's result to get a better idea of this.

Many thanks

Dirk

maintainer
#58 lore raw
Quoted 172 lines · Dirk Behme

Hi Alice,

Quoted 164 lines · Alice Ryhl
Quoted 155 lines · Dirk Behme
Quoted 117 lines · Boqun Feng
Quoted 92 lines · Dirk Behme
Quoted 91 lines · Boqun Feng
Quoted 90 lines · Dirk Behme
Quoted 89 lines · Boqun Feng
Quoted 77 lines · Dirk Behme
Quoted 76 lines · Boqun Feng
Quoted 75 lines · Dirk Behme

Hi Andreas,

Am 11.10.24 um 16:52 schrieb Andreas Hindborg:

Quoted 2 lines

Dirk, thanks for reporting!

:)

Quoted 47 lines
Quoted 45 lines · Boqun Feng

On Tue, Oct 01, 2024 at 02:37:46PM +0200, Dirk Behme wrote:
On 18.09.2024 00:27, Andreas Hindborg wrote:
Hi!

This series adds support for using the `hrtimer` subsystem from Rust code.

I tried breaking up the code in some smaller patches, hopefully that will
ease the review process a bit.

Just fyi, having all 14 patches applied I get [1] on the first (doctest)
Example from hrtimer.rs.

This is from lockdep:

https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/kernel/locking/lockdep.c#n4785

Having just a quick look I'm not sure what the root cause is. Maybe mutex in
interrupt context? Or a more subtle one?

I think it's calling mutex inside an interrupt context as shown by the
callstack:

] __mutex_lock+0xa0/0xa4
] ...
] hrtimer_interrupt+0x1d4/0x2ac

, it is because:

+//! struct ArcIntrusiveTimer {
+//! #[pin]
+//! timer: Timer<Self>,
+//! #[pin]
+//! flag: Mutex<bool>,
+//! #[pin]
+//! cond: CondVar,
+//! }

has a Mutex<bool>, which actually should be a SpinLockIrq [1]. Note that
irq-off is needed for the lock, because otherwise we will hit a self
deadlock due to interrupts:

spin_lock(&a);
> timer interrupt
spin_lock(&a);

Quoted 6 lines
Quoted 3 lines

Also notice that the IrqDisabled<'_> token can be simply created by
::new(), because irq contexts should guarantee interrupt disabled (i.e.
we don't support nested interrupts*).

I updated the example based on the work in [1]. I think we need to update CondVar::wait to support waiting with irq disabled.

Yes, I agree. This answers one of the open questions I had in the discussion with Boqun :)

What do you think regarding the other open question: In this *special* case here, what do you think to go *without* any lock? I mean the 'while *guard != 5' loop in the main thread is read only regarding guard. So it doesn't matter if it *reads* the old or the new value. And the read/modify/write of guard in the callback is done with interrupts disabled anyhow as it runs in interrupt context. And with this can't be interrupted (excluding nested interrupts). So this modification of guard doesn't need to be protected from being interrupted by a lock if there is no modifcation of guard "outside" the interupt locked context.

Quoted 7 lines
Quoted 4 lines
Quoted 2 lines

What do you think?

Reading while there is another CPU is writing is data-race, which is UB.

Could you help to understand where exactly you see UB in Andreas' 'while *guard != 5' loop in case no locking is used? As mentioned I'm under the

Sure, but could you provide the code of what you mean exactly, if you don't use a lock here, you cannot have a guard. I need to the exact code to point out where the compiler may "mis-compile" (a result of being

[...]

Quoted 23 lines
Quoted 22 lines
Quoted 13 lines

I thought we are talking about anything like

#[pin_data]

struct ArcIntrusiveTimer {
         #[pin]
         timer: Timer<Self>,
         #[pin]
(quoted diff) +1 -1
-      flag: SpinLockIrq<u64>,
+      flag: u64,
         #[pin]
         cond: CondVar,

}

Quoted 2 lines

?

Yes, but have you tried to actually use that for the example from Andreas? I think you will find that you cannot write to flag inside the timer callback, because you only has a Arc<ArcIntrusiveTimer>, so not mutable reference for ArcIntrusiveTimer. You can of course use unsafe to create a mutable reference to flag, but it won't be sound, since you are getting a mutable reference from an immutable reference.

Quoted 5 lines
Quoted 2 lines

Yes, of course. But, hmm, wouldn't that unsoundness be independent on the topic we discuss here? I mean we are talking about getting the compiler to

What do you mean? If the code is unsound, you won't want to use it in an example, right?

Yes, sure. But ;)

In a first step I just wanted to answer the question if we do need a lock at all in this special example. And that we could do even with unsound read/modify/write I would guess. And then, in a second step, if the answer would be "we don't need the lock", then we could think about how to make the flag handling sound. So I'm talking just about answering that question, not about the final example code. Step by step :)

Quoted 9 lines
Quoted 6 lines

read/modify/write 'flag' in the TimerCallback. *How* we tell him to do so should be independent on the result what we want to look at regarding the locking requirements of 'flag'?

Anyhow, my root motivation was to simplify Andreas example to not use a lock where not strictly required. And with this make Andreas example independent

Well, if you don't want to use a lock then you need to use atomics, otherwise it's likely a UB,

And here we are back to the initial question :) Why would it be UB without lock (and atomics)?

Some (pseudo) assembly:

Lets start with the main thread:

ldr x1, [x0]
<work with x1>

Quoted 7 lines

x0 and x1 are registers. x0 contains the address of flag in the main memory. I.e. that instruction reads (ldr == load) the content of that memory location (flag) into x1. x1 then contains flag which can be used then. This is what I mean with "the main thread is read only". If flag, i.e. x1, does contain the old or new flag value doesn't matter. I.e. for the read only operation it doesn't matter if it is protected by a lock as the load (ldr) can't be interrupted.

Quoted 1 line

If the compiler generates a single load, then sure.

Yes :)

Quoted 8 lines
Quoted 3 lines

But for an
unsynchronized load, the compiler may generate two separate load instructions and assume that both loads read the same value.

Ok, yes, if we get this from the compiler I agree that we need the lock, even if its just for the read. If I get the chance the next time I will try to have a look to the compiler's result to get a better idea of this.

Usually I would say that for cases like this, the correct approach is to use relaxed atomic loads and stores. That compiles down to ordinary load/store instructions as desired without letting the compiler split the load.

Alice