rust: sync: Introduce Rcu*Box
Subsystems (7)
Boqun Feng
maintainerv2 cover rust: sync: Introduce Rcu*Box
v1: https://lore.kernel.org/rust-for-linux/[email protected]/
Changes since v1:
* Rebased on 7.2-rc1
* As Alice pointed out [5], RcuFreeBox::with_rcu() should return a type that disallows extending the RCU read-side critical section to outlive the guard. As for now, no user needs with_rcu(), hence remove it.
* Includes a not-for-merge example based on Alice's patch.
v1's cover letter
-----------------
(this series is based on Alice's RFC [1] and discussion around Philipp's patches [2], [3])
As an easy way to provide RCU-protected allocation, two major types are provided:
- RcuBox<T, A>, inner T will be dropped and after a grace period.
Users: binder with maple_tree and dma_fence.
- RcuFreeBox<T, A>, inner T will be cleaned up immmediately and freed after a grace period. (Name suggestion is welcome). This is an attempt to consolidate Alice's PollCondVarBox [4] into a generic implementation, InPlaceInit support is still missing, but I want to get some feedback on the trait RcuFreeSafe.
[1]: https://lore.kernel.org/rust-for-linux/[email protected]/
[2]: https://lore.kernel.org/rust-for-linux/[email protected]/
[3]: https://lore.kernel.org/rust-for-linux/[email protected]/
[4]: https://lore.kernel.org/rust-for-linux/[email protected]/
[5]: https://lore.kernel.org/rust-for-linux/[email protected]/
Signature · 1 line
2.50.1 (Apple Git-155)
Boqun Feng
maintainerRFC PATCH v2 1/4 rust: rcu: Add RcuBox type
From: Alice Ryhl <[email protected]>
This adds an RcuBox container, which is like Box except that the value is freed after waiting for one grace period (via {kvfree_,}call_rcu()).
To allow containers to rely on the RCU properties of RcuBox, an extension of ForeignOwnable is added.
--- rust/bindings/bindings_helper.h | 1 + rust/kernel/sync/rcu.rs | 36 ++++- rust/kernel/sync/rcu/rcu_box.rs | 248 ++++++++++++++++++++++++++++++++ 3 files changed, 284 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/sync/rcu/rcu_box.rs
rust/bindings/bindings_helper.h +1 -0
@@ -81,6 +81,7 @@#include <linux/property.h> #include <linux/pwm.h> #include <linux/random.h> +#include <linux/rcupdate.h> #include <linux/refcount.h> #include <linux/regulator/consumer.h> #include <linux/sched.h>
rust/kernel/sync/rcu.rs +35 -1
@@ -4,7 +4,19 @@//! //! C header: [`include/linux/rcupdate.h`](srctree/include/linux/rcupdate.h) -use crate ::{bindings, types::NotThreadSafe }; +use crate ::{ + bindings, + types::{ +ForeignOwnable , +NotThreadSafe ,// + },// +}; + +mod rcu_box; +pub use self::rcu_box::RcuBox ; +pub use self::rcu_box::RcuKBox ; +pub use self::rcu_box::RcuKVBox ; +pub use self::rcu_box::RcuVBox ;/// Evidence that the RCU read side lock is held on the current thread/CPU. ///
@@ -66,3 +78,25 @@ pub fn synchronize_rcu() {// SAFETY: `synchronize_rcu()` is always safe to be called from process context. unsafe { bindings::synchronize_rcu () }; } + +/// Declares that a pointer type is rcu safe. +pub trait ForeignOwnableRcu :ForeignOwnable { +/// Type used to immutably borrow an rcu-safe value that is currently foreign-owned. +type RcuBorrowed <'a> +where +Self : 'a; + +/// Borrows a foreign-owned object immutably for an rcu grace period. +/// +/// This method provides a way to access a foreign-owned rcu-safe value from Rust immutably. +/// +/// # Safety +/// +/// * The provided pointer must have been returned by a previous call to [`into_foreign`]. +/// * If [`from_foreign`] is called, then `'a` must not end after the call to `from_foreign` +/// plus one rcu grace period. +/// +/// [`into_foreign`]: ForeignOwnable::into_foreign +/// [`from_foreign`]: ForeignOwnable::from_foreign +unsafe fn rcu_borrow <'a>(ptr: *mut ffi::c_void ) ->Self ::RcuBorrowed <'a>; +}
rust/kernel/sync/rcu/rcu_box.rs new file +248 -0
248 lines not on this page · raw message
Signature · 1 line
2.50.1 (Apple Git-155)
Boqun Feng
maintainerRFC PATCH v2 2/4 rust: maple_tree: Add load_rcu()
In reply to Boqun FengFrom: Alice Ryhl <[email protected]>
Now that we have a concept of rcu-safe containers, we may add a load_rcu() method to MapleTree that does not take the spinlock.
--- rust/kernel/maple_tree.rs | 52 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 52 insertions(+)
rust/kernel/maple_tree.rs +52 -0
@@ -16,6 +16,10 @@ alloc::Flags, error::to_result, prelude::*, + sync::rcu::{ + self, + ForeignOwnableRcu, // + }, types::{ForeignOwnable, Opaque}, };
@@ -233,6 +237,54 @@ pub fn erase(&self, index: usize) -> Option<T> {unsafe {T ::try_from_foreign (ret) } } +/// Load the value at the given index with rcu. +/// +/// # Examples +/// +/// Read the value under an rcu read lock. Even if the value is removed, it remains accessible +/// for one rcu grace period. +/// +/// ```ignore +/// use kernel::{ +/// maple_tree::MapleTree, +/// sync::rcu::{self, RcuBox}, +/// }; +/// +/// let tree = KBox::pin_init(MapleTree::<RcuBox<i32>>::new(), GFP_KERNEL)?; +/// +/// let ten = RcuBox::new(10, GFP_KERNEL)?; +/// tree.insert(100, ten, GFP_KERNEL)?; +/// +/// let rcu_read_lock = rcu::Guard::new(); +/// let ten = tree.load_rcu(100, &rcu_read_lock); +/// assert_eq!(ten, Some(&10)); +/// +/// // Even if the value gets removed, we may continue to access it for one rcu grace period. +/// tree.erase(100); +/// assert_eq!(ten, Some(&10)); +/// # Ok::<_, Error>(()) +/// ``` +#[inline] +pub fn load_rcu <'rcu>( + &self, + index:usize , + _rcu: &'rcu rcu::Guard , + ) ->Option <T ::RcuBorrowed <'rcu>> +where +T :ForeignOwnableRcu , + { +// SAFETY: `self.tree` contains a valid maple tree. +let ret =unsafe { bindings::mtree_load (self.tree.get (), index) }; +if ret.is_null () { +return None ; + } + +// SAFETY: If the pointer is not null, then it references a valid instance of `T`. It is +// safe to borrow the instance for 'rcu because the signature of this function enforces that +// the borrow does not outlive an rcu grace period. +Some (unsafe {T ::rcu_borrow (ret) }) + } +/// Lock the internal spinlock. #[inline] pub fn lock (&self) ->MapleGuard <'_,T > {
Signature · 1 line
2.50.1 (Apple Git-155)
Boqun Feng
maintainerRFC PATCH v2 3/4 rust: rcu: Introduce RcuFreeBox
In reply to Boqun FengThe current RcuBox will call the drop() function after a grace period inside an RCU callback. This suffices for maintaining a RCU-protected object:
RcuBox::drop():
call_rcu(
|..| { // <- call back after one grace period.
T::drop(); // <- call the destructor of the inner object.
}
)However, to support a different RCU usage pattern as below we need to extend RcuBox:
1. clean up the object, and unshare it from future RCU readers.
2. wait for an RCU grace period.
3. no other RCU readers, we can free the memory.
An RcuFreeBox<T: RcuFreeSafe> is introduced to provide support for this:
RcuFreeBox::drop():
T::drop_before_gp(); // clean up and ushare. kfree_call_rcu(..); // free it after one grace period.
--- rust/kernel/sync/rcu.rs | 34 ++++++++++++++++ rust/kernel/sync/rcu/rcu_box.rs | 72 +++++++++++++++++++++++++++++++-- 2 files changed, 102 insertions(+), 4 deletions(-)
rust/kernel/sync/rcu.rs +34 -0
@@ -4,6 +4,8 @@//! //! C header: [`include/linux/rcupdate.h`](srctree/include/linux/rcupdate.h) +use core::pin::Pin ; +use crate ::{ bindings, types::{
@@ -18,6 +20,8 @@pub use self::rcu_box::RcuKVBox ;pub use self::rcu_box::RcuVBox ; +pub use self::rcu_box::RcuFreeBox ; +/// Evidence that the RCU read side lock is held on the current thread/CPU. /// /// The type is explicitly not `Send` because this property is per-thread/CPU.
@@ -100,3 +104,33 @@ pub trait ForeignOwnableRcu: ForeignOwnable {/// [`from_foreign`]: ForeignOwnable::from_foreign unsafe fn rcu_borrow <'a>(ptr: *mut ffi::c_void ) ->Self ::RcuBorrowed <'a>; } + +/// Declares a struct is safe to free after a grace period if all readers are guarded by RCU. +/// +/// # Safety +/// +/// Implementation must guarantee `drop_before_gp()` makes sure no future RCU reader will access +/// any part of [`Self`], as a result, after `drop_before_gp()` return + one grace period, no RCU +/// reader will be on the object, and it's safe to free it. +/// +/// Notes for implementators: implementing this trait in general requires `Self` being a +/// [`UnsafePinned`], i.e. a `&mut Self` is not a noalias reference if `Self` has non-trivial +/// `drop()` function. +pub unsafe trait RcuFreeSafe { +/// Clean up `Self` and make it ready to be RCU freed. +fn drop_before_gp (self:Pin <&mut Self >); +} + +macro_rules! impl_not_drop { + ($($t:ty, )*) => { +// SAFETY: Dropping `T` has no side effect means `T` is always ready to be freed. And an +// empty `drop_before_gp()` suffices. + $(unsafe impl RcuFreeSafe for $t { +fn drop_before_gp (self:Pin <&mut Self >) { + $crate::const_assert!(!core::mem::needs_drop ::<$t>()); + } + })* + } +} + +impl_not_drop! {i8 ,u8 ,i16 ,u16 ,i32 ,u32 ,isize ,usize ,i64 ,u64 ,}
rust/kernel/sync/rcu/rcu_box.rs +68 -4
109 lines not on this page · raw message
Signature · 1 line
2.50.1 (Apple Git-155)
Boqun Feng
maintainerRFC PATCH v2 4/4 rust: poll: use kfree_rcu() for PollCondVar
In reply to Boqun FengFrom: Alice Ryhl <[email protected]>
Rust Binder currently uses PollCondVar, but it calls synchronize_rcu() in the destructor, which we would like to avoid. Add a variation of PollCondVar that kfree_rcu() instead.
One could avoid the rcu field and allocate the rcu_head on drop using a fallback to synchronize_rcu() on ENOMEM. However, I'd prefer to avoid the potential for synchronize_rcu(), and Binder will only use this for a small fraction of processes, so even if it changes which kmalloc bucket it falls into, the extra memory is not a problem.
--- rust/kernel/sync/poll.rs | 26 +++++++++++++++++++++----- 1 file changed, 21 insertions(+), 5 deletions(-)
rust/kernel/sync/poll.rs +21 -5
@@ -9,12 +9,19 @@ fs::File, prelude::*, sync::{ - rcu::synchronize_rcu, + rcu::{ + synchronize_rcu,// +RcuFreeSafe , + }, CondVar, LockClassKey, // }, // }; -use core::{marker::PhantomData , ops::Deref }; + +use core::{ + marker::PhantomData , + ops::Deref ,// +};/// Creates a [`PollCondVar`] initialiser with the given name and a newly-created lock class. #[macro_export]
@@ -70,6 +77,7 @@ pub fn register_wait(&self, file: &File, cv: &PollCondVar) {/// /// [`CondVar`]: crate::sync::CondVar #[pin_data( PinnedDrop )] +#[repr(transparent)] pub struct PollCondVar {#[pin] inner:CondVar ,
@@ -97,12 +105,20 @@ fn deref(&self) -> &CondVar {impl PinnedDrop for PollCondVar {#[inline] fn drop (self:Pin <&mut Self >) { + self.drop_before_gp (); + +// Wait for epoll items to be properly removed. +synchronize_rcu (); + } +} + +// SAFETY: __wake_up_pollfree() guarantees all the epoll items on the wait list will be gone after +// one grace period. +unsafe impl RcuFreeSafe for PollCondVar { +fn drop_before_gp (self:Pin <&mut Self >) {// Clear anything registered using `register_wait`. // // SAFETY: The pointer points at a valid `wait_queue_head`. unsafe { bindings::__wake_up_pollfree (self.inner.wait_queue_head.get ()) }; - -// Wait for epoll items to be properly removed. -synchronize_rcu (); } }
Signature · 1 line
2.50.1 (Apple Git-155)
[email protected]
AI reviewRe: [RFC PATCH v2 3/4] rust: rcu: Introduce RcuFreeBox
In reply to Boqun FengThank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:
Thank you for your contribution! Sashiko AI review found 7 potential issue(s) to consider:
New issues:
- [High] RcuFreeBox::drop creates an exclusive mutable reference (&mut T) while concurrent RCU readers may exist, causing Undefined Behavior (Mutable Aliasing).
- [High] RcuFreeBox permanently skips T::drop(), silently leaking inner resources or forcing developers into Use-After-Free traps.
- [High] Casting between *mut RcuBoxInner<ManuallyDrop<T>> and *mut RcuBoxInner<T> relies on unguaranteed memory layout, resulting in Undefined Behavior.
- [High] If memory allocation fails in RcuFreeBox::new(), the inner object is silently leaked.
- [Medium] RcuFreeBox lacks implementations for ForeignOwnable and ForeignOwnableRcu, making it completely unusable in RCU data structures.
- [Low] Missing #[inline] annotations on small forwarding functions in an abstraction.
Pre-existing issues:
- [High] RcuBox::drop unconditionally invokes kvfree_call_rcu, bypassing custom generic allocators and causing kernel memory corruption. --
--- Patch [3]: [RFC PATCH v2 3/4] rust: rcu: Introduce RcuFreeBox --- commit a85d6377ca5df1e0c96d95e2dd50fb0b8a721195
Author: Boqun Feng <[email protected]>
Subject: rust: rcu: Introduce RcuFreeBox
This commit introduces RcuFreeBox, which extends RcuBox to support an RCU usage pattern where objects are cleaned up before a grace period and freed afterwards. It provides methods to unshare an object from future RCU readers and wait for a grace period before safely freeing the memory.
Quoted 4 lines
rust/kernel/sync/rcu.rs +0
-0
[ ... ]
Quoted 9 lines
rust/kernel/sync/rcu.rs +5
-0@@ -100,3 +104,33 @@ pub trait ForeignOwnableRcu: ForeignOwnable {/// [`from_foreign`]: ForeignOwnable::from_foreign unsafe fn rcu_borrow <'a>(ptr: *mut ffi::c_void ) ->Self ::RcuBorrowed <'a>; } + +/// Declares a struct is safe to free after a grace period if all readers are guarded by RCU. +/// +/// # Safety +///
rust/kernel/sync/rcu.rs pub trait ForeignOwnableRcu: ForeignOwnable { · line 121
Quoted 10 lines
rust/kernel/sync/rcu.rs +10
-0+/// Implementation must guarantee `drop_before_gp()` makes sure no future RCU reader will access +/// any part of [`Self`], as a result, after `drop_before_gp()` return + one grace period, no RCU +/// reader will be on the object, and it's safe to free it. +/// +/// Notes for implementators: implementing this trait in general requires `Self` being a +/// [`UnsafePinned`], i.e. a `&mut Self` is not a noalias reference if `Self` has non-trivial +/// `drop()` function. +pub unsafe trait RcuFreeSafe { +/// Clean up `Self` and make it ready to be RCU freed. +fn drop_before_gp (self:Pin <&mut Self >);
[Severity: High]
Does this trait signature inherently force a mutable aliasing violation?
By requiring a mutable reference here, does this force the caller to create an exclusive reference while concurrent RCU readers may still hold shared references to the object?
Quoted 1 line
rust/kernel/sync/rcu.rs +1
-0+}
[ ... ]
Quoted 4 lines
rust/kernel/sync/rcu/rcu_box.rs +0
-0
[ ... ]
rust/kernel/sync/rcu/rcu_box.rs fn drop(&mut self) { · line 233
Quoted 10 lines
rust/kernel/sync/rcu/rcu_box.rs +6
-0@@ -223,6 +225,56 @@ fn drop(&mut self) {drop (unsafe {Box ::<_ ,A >::from_raw (box_inner) }); } +/// A box that is freed with RCU. +/// +/// Currently we require `T` being `Send` because of an implementation limitation. In theory we can +/// support `T` being `!Send`, since the RCU callback is only used to free the memory, not dropping +/// `T`. +pub struct RcuFreeBox <T :Send +RcuFreeSafe ,A :Allocator >(RcuBox <ManuallyDrop <T >,A >);
[Severity: Medium]
How can this struct be used in RCU data structures?
Since it lacks implementations for ForeignOwnable and ForeignOwnableRcu, and its inner RcuBox field is private, is it structurally possible to convert this to a foreign pointer or share it with RCU readers?
rust/kernel/sync/rcu/rcu_box.rs fn drop(&mut self) { · line 237
Quoted 4 lines
rust/kernel/sync/rcu/rcu_box.rs +4
-0+ +impl <T :Send +RcuFreeSafe ,A :Allocator >RcuFreeBox <T ,A > { +/// Create a new `RcuFreeBox`. +pub fn new (x:T , flags: alloc::Flags ) ->Result <Self ,AllocError > {
[Severity: Low]
Should this small forwarding function have an inline annotation?
rust/kernel/sync/rcu/rcu_box.rs fn drop(&mut self) { · line 238
Quoted 1 line
rust/kernel/sync/rcu/rcu_box.rs +1
-0+Ok (Self (RcuBox ::new (ManuallyDrop ::new (x), flags)?))
[Severity: High]
Does this silently leak the inner object on allocation failure?
If RcuBox::new() returns an AllocError, the ? operator will abort the function. Because ManuallyDrop intentionally prevents the Drop implementation of its inner value from running, does this leak the object and any dynamic resources it holds?
rust/kernel/sync/rcu/rcu_box.rs fn drop(&mut self) { · line 245
Quoted 7 lines
rust/kernel/sync/rcu/rcu_box.rs +7
-0+ } +} + +impl <T :Send +RcuFreeSafe ,A :Allocator >Deref for RcuFreeBox <T ,A > { +type Target =T ; + +fn deref (&self) -> &T {
[Severity: Low]
Should this trivial deref forwarder have an inline annotation?
rust/kernel/sync/rcu/rcu_box.rs fn drop(&mut self) { · line 251
Quoted 6 lines
rust/kernel/sync/rcu/rcu_box.rs +6
-0+ self.0 .deref () + } +} + +impl <T :Send +RcuFreeSafe ,A :Allocator >Drop for RcuFreeBox <T ,A > { +fn drop (&mut self) {
[Severity: High]
This isn't a bug introduced by this patch, but does the underlying RcuBox::drop unconditionally bypass custom allocators?
When core::mem::needs_drop::<T>() evaluates to false, RcuBox::drop bypasses the allocator A and calls bindings::kvfree_call_rcu():
RcuBox::drop() {
...
unsafe { bindings::kvfree_call_rcu(rcu_head, self.0.as_ptr().cast()) };
...
}If a custom allocator (such as a module-specific slab cache) is used, will passing its pointer to kvfree result in memory corruption?
rust/kernel/sync/rcu/rcu_box.rs fn drop(&mut self) { · line 254
Quoted 3 lines
rust/kernel/sync/rcu/rcu_box.rs +3
-0+// CAST: `ManuallyDrop<RcuBoxInner<T>>` is transparent to `RcuBoxInner<T>`, and `RcuBox` +// owns the object per type invariants. +let inner: *mut RcuBoxInner <T > = self.0 .0 .as_ptr ().cast ();
[Severity: High]
Does casting between these types rely on unguaranteed memory layout?
Both RcuBoxInner<ManuallyDrop<T>> and RcuBoxInner<T> use the default repr(Rust) layout.
Since the Rust compiler provides no layout guarantees for generic repr(Rust) structs across different type parameters, can fields be arbitrarily reordered, making this cast undefined behavior?
rust/kernel/sync/rcu/rcu_box.rs fn drop(&mut self) { · line 258
Quoted 4 lines
rust/kernel/sync/rcu/rcu_box.rs +4
-0+ +// SAFETY: Per the invariants of `RcuBox`, `inner` owns the pointed object. And we are not +// going to move it. +let pin =unsafe {Pin ::new_unchecked (&mut (*inner).value) };
[Severity: High]
Does creating this exclusive mutable reference cause undefined behavior?
When RcuFreeBox::drop() executes before the RCU grace period, pre-existing RCU readers may still be traversing the object and holding shared references.
Does forming a mutable reference while shared references are active violate Rust aliasing rules?
rust/kernel/sync/rcu/rcu_box.rs fn drop(&mut self) { · line 263
Quoted 5 lines
rust/kernel/sync/rcu/rcu_box.rs +5
-0+ + pin.drop_before_gp (); + +// `needs_drop::<ManuallyDrop>()` returns `false`, hence `kvfree_call_rcu()` will be called +// and free the underlying data after a grace period.
[Severity: High]
Does permanently skipping T::drop() force a memory leak or a use-after-free?
Because ManuallyDrop evaluates needs_drop to false, kvfree_call_rcu() will only free the outermost struct after the grace period, bypassing T::drop().
If T owns dynamic resources like a Box or Vec, will they permanently leak?
If a developer attempts to fix the leak by manually dropping the inner fields inside drop_before_gp(), will they trigger a use-after-free because concurrent RCU readers can still access those fields before the grace period ends?
Quoted 2 lines
rust/kernel/sync/rcu/rcu_box.rs +2
-0+ } +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=3
[email protected]
AI reviewRe: [RFC PATCH v2 1/4] rust: rcu: Add RcuBox type
In reply to Boqun FengThank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Critical] `RcuBox` lacks a `'static` bound on `T`, which allows asynchronous Use-After-Free for types…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] RcuBox lacks a 'static bound on T, which allows asynchronous Use-After-Free for types with custom Drop implementations that contain non-'static references.
- [High] RcuBox<T, A> bypasses the generic Allocator's free method in its Drop implementation when needs_drop::<T>() is false, leading to invalid frees for custom allocators.
- [High] RcuBox safely uses call_rcu() with a monomorphized callback drop_rcu_box, but Rust does not expose rcu_barrier(), making safe module unload impossible. --
--- Patch [1]: [RFC PATCH v2 1/4] rust: rcu: Add RcuBox type --- commit b7ba3ecd614c74b985c54d7b0785315fa2c066c8
Author: Alice Ryhl <[email protected]>
rust: rcu: Add RcuBox type
This adds an RcuBox container, which is like Box except that the value is freed after waiting for one grace period (via {kvfree_,}call_rcu()). To allow containers to rely on the RCU properties of RcuBox, an extension of ForeignOwnable is added.
Quoted 5 lines
rust/kernel/sync/rcu/rcu_box.rs new file +0
-0
[ ... ]
rust/kernel/sync/rcu/rcu_box.rs line 43
Quoted 9 lines
rust/kernel/sync/rcu/rcu_box.rs new file +9
-0+/// A box that is freed with rcu. +/// +/// The value must be Send, as rcu may drop it on another thread. +/// +/// # Invariants +/// +/// * The pointer is valid and references a pinned RcuBoxInner<T> allocated with A. +/// * This RcuBox holds exclusive permissions to rcu free the allocation. +pub struct RcuBox <T :Send ,A :Allocator >(NonNull <RcuBoxInner <T >>,PhantomData <A >);
[Severity: Critical]
Since the drop implementation can delegate the actual drop to an asynchronous background task via call_rcu, could a type with a non-static lifetime trigger a use-after-free if its custom drop implementation accesses a scoped reference after it goes out of scope?
Would it be safer to require T: 'static here?
[ ... ]
rust/kernel/sync/rcu/rcu_box.rs line 202
Quoted 9 lines
rust/kernel/sync/rcu/rcu_box.rs new file +9
-0+impl <T :Send ,A :Allocator >Drop for RcuBox <T ,A > { +fn drop (&mut self) { +// SAFETY: The rcu_head field is in-bounds of a valid allocation. +let rcu_head =unsafe { &raw mut (*self.0 .as_ptr ()).rcu_head }; +if core::mem::needs_drop ::<T >() { +// SAFETY: rcu_head is the rcu_head field of RcuBoxInner<T>. All users will be +// gone in an rcu grace period. This is the destructor, so we may pass ownership of the +// allocation. +unsafe { bindings::call_rcu (rcu_head,Some (drop_rcu_box ::<T ,A >)) };
[Severity: High]
Does this monomorphized callback require modules to use rcu_barrier() during unload to prevent executing unloaded memory?
Since rcu_barrier() isn't exposed in the Rust bindings yet, is there a way for loadable modules using RcuBox to safely unload without risking a regression if the RCU grace period hasn't completed?
rust/kernel/sync/rcu/rcu_box.rs line 207
Quoted 5 lines
rust/kernel/sync/rcu/rcu_box.rs new file +5
-0+ }else { +// SAFETY: All users will be gone in an rcu grace period. +// TODO: We are luckily since kvfree_call_rcu() works on both kmalloc and vmalloc, +// maybe a new Allocator method is needed. +unsafe { bindings::kvfree_call_rcu (rcu_head, self.0 .as_ptr ().cast ()) };
[Severity: High]
If RcuBox is instantiated with a custom allocator, will hardcoding kvfree_call_rcu cause an invalid free when needs_drop is false?
This branch appears to completely bypass the generic allocator's free method.
Quoted 3 lines
rust/kernel/sync/rcu/rcu_box.rs new file +3
-0+ } + } +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=1
[email protected]
AI reviewRe: [RFC PATCH v2 2/4] rust: maple_tree: Add load_rcu()
In reply to Boqun FengThank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider: - [Critical] Lockless `load_rcu()` leads to use-after-free on maple tree internal nodes because the tree is…
Thank you for your contribution! Sashiko AI review found 3 potential issue(s) to consider:
- [Critical] Lockless load_rcu() leads to use-after-free on maple tree internal nodes because the tree is not initialized with MT_FLAGS_USE_RCU.
- [High] The ForeignOwnableRcu trait is missing the unsafe keyword, allowing safe code to violate RCU memory safety guarantees.
- [High] RcuBox's Drop implementation bypasses the custom Allocator parameter when dropping types that do not require drop logic. --
--- Patch [2]: [RFC PATCH v2 2/4] rust: maple_tree: Add load_rcu() --- commit fcc71907e345f6a007c4675e0956efaca9ebf22c
Author: Alice Ryhl <[email protected]>
rust: maple_tree: Add load_rcu()
This patch introduces a lockless load_rcu() method to the MapleTree, assuming RCU-safe containers are used. The patch series also adds an RcuBox type and a ForeignOwnableRcu trait to support RCU-based freeing of box contents.
Quoted 4 lines
rust/kernel/maple_tree.rs +0
-0
[ ... ]
Quoted 5 lines
rust/kernel/maple_tree.rs +1
-0@@ -233,6 +237,54 @@ pub fn erase(&self, index: usize) -> Option<T> {unsafe {T ::try_from_foreign (ret) } } +/// Load the value at the given index with rcu.
[ ... ]
rust/kernel/maple_tree.rs pub fn erase(&self, index: usize) -> Option<T> { · line 277
Quoted 11 lines
rust/kernel/maple_tree.rs +11
-0+#[inline] +pub fn load_rcu <'rcu>( + &self, + index:usize , + _rcu: &'rcu rcu::Guard , + ) ->Option <T ::RcuBorrowed <'rcu>> +where +T :ForeignOwnableRcu , + { +// SAFETY: `self.tree` contains a valid maple tree. +let ret =unsafe { bindings::mtree_load (self.tree.get (), index) };
[Severity: Critical]
Does this code cause a use-after-free on maple tree internal nodes?
Because load_rcu() executes bindings::mtree_load() without acquiring the tree's spinlock, the tree must be initialized with bindings::MT_FLAGS_USE_RCU to be safe against concurrent writers.
However, MapleTree::new() initializes the tree without this flag:
rust/kernel/maple_tree.rs:MapleTree::new() {
...
tree <- Opaque::ffi_init(|slot| unsafe { bindings::mt_init_flags(slot, 0) }),
...
}Because the tree is in non-RCU mode, writers will synchronously free internal nodes via kfree(), causing load_rcu() to dereference freed memory during its lockless traversal.
Quoted 17 lines
rust/kernel/maple_tree.rs +10
-0+if ret.is_null () { +return None ; + } + +// SAFETY: If the pointer is not null, then it references a valid instance of `T`. It is +// safe to borrow the instance for 'rcu because the signature of this function enforces that +// the borrow does not outlive an rcu grace period. +Some (unsafe {T ::rcu_borrow (ret) }) + } +/// Lock the internal spinlock. #[inline] pub fn lock (&self) ->MapleGuard <'_,T > {rust/kernel/sync/rcu.rs +0
-0
[ ... ]
rust/kernel/sync/rcu.rs pub fn synchronize_rcu() { · line 83
Quoted 7 lines
rust/kernel/sync/rcu.rs +3
-0@@ -66,3 +78,25 @@ pub fn synchronize_rcu() {// SAFETY: `synchronize_rcu()` is always safe to be called from process context. unsafe { bindings::synchronize_rcu () }; } + +/// Declares that a pointer type is rcu safe. +pub trait ForeignOwnableRcu :ForeignOwnable {
[Severity: High]
Should this trait be marked unsafe to enforce the RCU deferral contract?
The ForeignOwnableRcu trait promises that removing the node will not invalidate the object until an RCU grace period elapses, allowing load_rcu() to safely borrow it.
Because ForeignOwnableRcu is a safe trait, developers can implement it for types that drop synchronously (like Box<T>, which already implements the unsafe trait ForeignOwnable).
Doing so violates the safety assumptions of load_rcu(), causing a use-after-free without any unsafe block in the user's code.
Quoted 5 lines
rust/kernel/sync/rcu.rs +5
-0+/// Type used to immutably borrow an rcu-safe value that is currently foreign-owned. +type RcuBorrowed <'a> +where +Self : 'a; +
[ ... ]
Quoted 5 lines
rust/kernel/sync/rcu/rcu_box.rs new file +0
-0
[ ... ]
Quoted 4 lines
rust/kernel/sync/rcu/rcu_box.rs new file +4
-0+impl <T :Send ,A :Allocator >Drop for RcuBox <T ,A > { +fn drop (&mut self) { +// SAFETY: The `rcu_head` field is in-bounds of a valid allocation. +let rcu_head =unsafe { &raw mut (*self.0 .as_ptr ()).rcu_head };
rust/kernel/sync/rcu/rcu_box.rs
Quoted 10 lines
rust/kernel/sync/rcu/rcu_box.rs new file +10
-0+if core::mem::needs_drop ::<T >() { +// SAFETY: `rcu_head` is the `rcu_head` field of `RcuBoxInner<T>`. All users will be +// gone in an rcu grace period. This is the destructor, so we may pass ownership of the +// allocation. +unsafe { bindings::call_rcu (rcu_head,Some (drop_rcu_box ::<T ,A >)) }; + }else { +// SAFETY: All users will be gone in an rcu grace period. +// TODO: We are luckily since `kvfree_call_rcu()` works on both kmalloc and vmalloc, +// maybe a new `Allocator` method is needed. +unsafe { bindings::kvfree_call_rcu (rcu_head, self.0 .as_ptr ().cast ()) };
[Severity: High]
Does this bypass the custom Allocator parameter when dropping types that do not require drop logic?
If an RcuBox<T, A> is created with a custom allocator A (e.g., a DMA or memory pool allocator) where T does not implement Drop, the code directly branches to kvfree_call_rcu().
This hardcodes the assumption that the memory belongs to kmalloc or vmalloc. When the RcuBox is freed, kvfree_call_rcu() will attempt to free memory it does not own, resulting in allocator corruption.
Quoted 3 lines
rust/kernel/sync/rcu/rcu_box.rs new file +3
-0+ } + } +}
Signature · 1 line
Sashiko AI review · https://sashiko.dev/#/patchset/[email protected]?part=2