Generic `Allocator` support for Rust
Subsystems (9)
Danilo Krummrich
maintainerv8 cover Generic `Allocator` support for Rust
Hi,
This patch series adds generic kernel allocator support for Rust, which so far is limited to kmalloc allocations.
In order to abstain from (re-)adding unstable Rust features to the kernel, this patch series does not extend the Allocator trait from Rust's alloc crate, nor does it extend the BoxExt and VecExt extensions.
Instead, this series introduces a kernel specific Allocator trait, which is implemented by the Kmalloc, Vmalloc and KVmalloc allocators, also implemented in the context of this series.
As a consequence we need our own kernel Box<T, A> and Vec<T, A> types. Additionally, this series adds the following type aliases:
``` pub type KBox<T> = Box<T, Kmalloc>; pub type VBox<T> = Box<T, Vmalloc>; pub type KVBox<T> = Box<T, KVmalloc>; pub type KVec<T> = Vec<T, Kmalloc>; pub type VVec<T> = Vec<T, Vmalloc>; pub type KVVec<T> = Vec<T, KVmalloc>; ```
With that, we can start using the kernel Box and Vec types throughout the tree and remove the now obolete extensions BoxExt and VecExt.
For a final cleanup, this series removes the last minor dependencies to Rust's alloc crate and removes it from the entire kernel build.
The series ensures not to break the rusttest make target by implementing the allocator_test module providing a stub implementation for all kernel Allocators.
This patch series passes all KUnit tests, including the ones added by this series. Additionally, the tests were run with kmemleak and KASAN enabled, without any issues.
This series is based on [1], which hit -mm/mm-stable, and is also available in [2].
[1] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=mm/krealloc
[2] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=rust/mm
Changes in v8:
- rebase onto rust-next (v6.12-rc1), including the -mm dependencies from [1]
- introduce ArrayLayout and rework the Allocator trait to take an old_layout argument
- split up "rust: alloc: implement Allocator for Kmalloc" in three separate patches (introduce ReallocFunc, make allocator module public and implement Kmalloc)
- wrap impl_slice_eq! pattern in $()*
- couple of style changes suggested by Benno and Gary
Changes in v7:
- rebase onto rust-next resolving recent conflicts (93dc3be19450447a3a7090bd1dfb9f3daac3e8d2)
- documentation / safety comment changes suggested by Benno and Boqun
- rename ptr to current in IntoIter::next (Alice)
- remove unnecessary braces in kvec! (Benno)
- add debug_assert! in Vec::set_len (Benno)
- remove unused args in impl_slice_eq! (Benno)
- simplify if statement in Cmalloc::realloc (Benno)
Changes in v6:
- rebase onto rust-dev
- keep compiler annotations for {k,v,kv}realloc()
- documentation changes suggested by Alice, Benno
- remove Box::into_pin
- fix typo in Send and Sync for Box and Vec
- kvec! changes suggested by Alice
- free src after copy in Cmalloc
- handle n == 0 in Vec::extend_with
Changes in v5:
- (safety) comment / documentation fixes suggested by Alice, Benno and Gary
- remove Unique<T> and implement Send and Sync for Box and Vec
- use KMALLOC_MAX_SIZE for KVmalloc test and add a Kmalloc test that expects to fail for KMALLOC_MAX_SIZE
- create use constants KREALLOC, VREALLOC and KVREALLOC for ReallocFuncs
- drop Box::drop_contents for now, will add it again, once I actually rebase on the original patch that introduces it
- improve usage of size_of_val in Box
- move InPlaceInit and ForeignOwnable impls into kbox.rs
- fix missing Box conversions in rnull.rs
- reworked Cmalloc to keep track of the size of memory allocations itself
- remove GlobalAlloc together with the alloc crate to avoid a linker error
- remove alloc from scripts/generate_rust_analyzer.py
Changes in v4:
- (safety) comment fixes suggested by Alice and Boqun
- remove Box::from_raw_alloc and Box::into_raw_alloc, we don't need them
- in Box::drop call size_of_val before drop_in_place
- implement ForeignOwnable for Pin<Box<T>> as suggested by Alice
- in Vec::extend_with, iterate over n instead of spare.len()
- for Vmalloc and KVmalloc fail allocation for alignments larger than PAGE_SIZE for now (will add support for larger alignments in a separate series)
- implement Cmalloc in allocator_test and type alias all kernel allocator types to it, such that we can use the kernel's Box and Vec types in userspace tests (rusttest)
- this makes patch "rust: str: test: replace alloc::format" rather trivial
Changes in v3:
- Box:
- minor documentation fixes
- removed unnecessary imports in doc tests
- dropeed self argument from some remaining Box methods
- implement InPlaceInit for Box<T, A> rather than specifically for KBox<T>
- Vec:
- minor documentation fixes
- removed useless Vec::allocator method
- in Vec::extend_with use Vec::spare_capacity_mut instead of raw pointer operations
- added a few missing safety comments
- pass GFP flags to Vec::collect
- fixed a rustdoc warning in alloc.rs
- fixed the allocator_test module to implement the Allocator trait correctly
- rebased to rust-next
Changes in v2:
- preserve impl GlobalAlloc for Kmalloc and remove it at the end (Benno)
- remove &self parameter from all Allocator functions (Benno)
- various documentation fixes for Allocator (Benno)
- use NonNull<u8> for Allocator::free and Option<NonNull<u8>> for Allocator::realloc (Benno)
- fix leak of IntoIter in Vec::collect (Boqun)
- always realloc (try to shrink) in Vec::collect, it's up the the Allocator to provide a heuristic whether it makes sense to actually shrink
- rename KBox<T, A> -> Box<T, A> and KVec<T, A> -> Vec<T, A> and provide type aliases KBox<T>, VBox<T>, KVBox<T>, etc.
- This allows for much cleaner code and, in combination with removing &self parameters from Allocators, gets us rid of the need for Box::new and Box::new_alloc and all other "_alloc" postfixed functions.
- Before: KBox::new_alloc(foo, Vmalloc)?
- After: VBox::new(foo)?, which resolves to
`Box::<Foo, Vmalloc>::new(foo)?;
Benno Lossin (1):
rust: alloc: introduce ArrayLayout
Danilo Krummrich (28):
rust: alloc: add Allocator trait
rust: alloc: separate aligned_size from krealloc_aligned
rust: alloc: rename KernelAllocator to Kmalloc
rust: alloc: implement ReallocFunc
rust: alloc: make allocator module public
rust: alloc: implement Allocator for Kmalloc
rust: alloc: add module allocator_test
rust: alloc: implement Vmalloc allocator
rust: alloc: implement KVmalloc allocator
rust: alloc: add __GFP_NOWARN to Flags
rust: alloc: implement kernel Box
rust: treewide: switch to our kernel Box type
rust: alloc: remove extension of std's Box
rust: alloc: add Box to prelude
rust: alloc: implement kernel Vec type
rust: alloc: implement IntoIterator for Vec
rust: alloc: implement collect for IntoIter
rust: treewide: switch to the kernel Vec type
rust: alloc: remove VecExt extension
rust: alloc: add Vec to prelude
rust: error: use core::alloc::LayoutError
rust: error: check for config test in Error::name
rust: alloc: implement contains for Flags
rust: alloc: implement Cmalloc in module allocator_test
rust: str: test: replace alloc::format
rust: alloc: update module comment of alloc.rs
kbuild: rust: remove the alloc crate and GlobalAlloc
MAINTAINERS: add entry for the Rust alloc module
MAINTAINERS | 7 + drivers/block/rnull.rs | 4 +- rust/Makefile | 43 +- rust/bindings/bindings_helper.h | 1 + rust/exports.c | 1 - rust/helpers/helpers.c | 1 + rust/helpers/slab.c | 6 + rust/helpers/vmalloc.c | 9 + rust/kernel/alloc.rs | 150 ++++- rust/kernel/alloc/allocator.rs | 208 +++++-- rust/kernel/alloc/allocator_test.rs | 95 +++ rust/kernel/alloc/box_ext.rs | 89 --- rust/kernel/alloc/kbox.rs | 455 ++++++++++++++ rust/kernel/alloc/kvec.rs | 901 ++++++++++++++++++++++++++++ rust/kernel/alloc/layout.rs | 91 +++ rust/kernel/alloc/vec_ext.rs | 185 ------ rust/kernel/error.rs | 6 +- rust/kernel/init.rs | 95 +-- rust/kernel/init/__internal.rs | 2 +- rust/kernel/lib.rs | 1 - rust/kernel/prelude.rs | 5 +- rust/kernel/rbtree.rs | 49 +- rust/kernel/str.rs | 35 +- rust/kernel/sync/arc.rs | 17 +- rust/kernel/sync/condvar.rs | 4 +- rust/kernel/sync/lock/mutex.rs | 2 +- rust/kernel/sync/lock/spinlock.rs | 2 +- rust/kernel/sync/locked_by.rs | 2 +- rust/kernel/types.rs | 52 +- rust/kernel/uaccess.rs | 17 +- rust/kernel/workqueue.rs | 20 +- rust/macros/lib.rs | 12 +- samples/rust/rust_minimal.rs | 4 +- scripts/Makefile.build | 7 +- scripts/generate_rust_analyzer.py | 11 +- 35 files changed, 2009 insertions(+), 580 deletions(-) create mode 100644 rust/helpers/vmalloc.c create mode 100644 rust/kernel/alloc/allocator_test.rs delete mode 100644 rust/kernel/alloc/box_ext.rs create mode 100644 rust/kernel/alloc/kbox.rs create mode 100644 rust/kernel/alloc/kvec.rs create mode 100644 rust/kernel/alloc/layout.rs delete mode 100644 rust/kernel/alloc/vec_ext.rs
base-commit: 9852d85ec9d492ebef56dc5f229416c925758edc
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 1/29 rust: alloc: add `Allocator` trait
Add a kernel specific Allocator trait, that in contrast to the one in Rust's core library doesn't require unstable features and supports GFP flags.
Subsequent patches add the following trait implementors: Kmalloc, Vmalloc and KVmalloc.
--- rust/kernel/alloc.rs | 101 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+)
rust/kernel/alloc.rs +101 -0
@@ -11,6 +11,7 @@/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ; +use core::{alloc::Layout , ptr::NonNull };/// Flags to be used when allocating memory. ///
@@ -86,3 +87,103 @@ pub mod flags {/// small allocations. pub const GFP_NOWAIT :Flags =Flags (bindings::GFP_NOWAIT ); } + +/// The kernel's [`Allocator`] trait. +/// +/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffers described +/// via [`Layout`]. +/// +/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on +/// an object instance. +/// +/// In order to be able to support `#[derive(SmartPointer)]` later on, we need to avoid a design +/// that requires an `Allocator` to be instantiated, hence its functions must not contain any kind +/// of `self` parameter. +/// +/// # Safety +/// +/// - A memory allocation returned from an allocator must remain valid until it is explicitly freed. +/// +/// - Any pointer to a valid memory allocation must be valid to be passed to any other [`Allocator`] +/// function of the same type. +/// +/// - Implementers must ensure that all trait functions abide by the guarantees documented in the +/// `# Guarantees` sections. +pub unsafe trait Allocator { +/// Allocate memory based on `layout` and `flags`. +/// +/// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout +/// constraints (i.e. minimum size and alignment as specified by `layout`). +/// +/// This function is equivalent to `realloc` when called with `None`. +/// +/// # Guarantees +/// +/// When the return value is `Ok(ptr)`, then `ptr` is +/// - valid for reads and writes for `layout.size()` bytes, until it is passed to +/// [`Allocator::free`] or [`Allocator::realloc`], +/// - aligned to `layout.align()`, +/// +/// Additionally, `Flags` are honored as documented in +/// <https://docs.kernel.org/core-api/mm-api.html#mm-api-gfp-flags>. +fn alloc (layout:Layout , flags:Flags ) ->Result <NonNull <[u8 ]>,AllocError > { +// SAFETY: Passing `None` to `realloc` is valid by it's safety requirements and asks for a +// new memory allocation. +unsafe {Self ::realloc (None , layout,Layout ::new ::<()>(), flags) } + } + +/// Re-allocate an existing memory allocation to satisfy the requested `layout`. +/// +/// If the requested size is zero, `realloc` behaves equivalent to `free`. +/// +/// If the requested size is larger than the size of the existing allocation, a successful call +/// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but +/// may also be larger. +/// +/// If the requested size is smaller than the size of the existing allocation, `realloc` may or +/// may not shrink the buffer; this is implementation specific to the allocator. +/// +/// On allocation failure, the existing buffer, if any, remains valid. +/// +/// The buffer is represented as `NonNull<[u8]>`. +/// +/// # Safety +/// +/// - If `ptr == Some(p)`, then `p` must point to an existing and valid memory allocation +/// created by this [`Allocator`]; if `old_layout` is zero-sized `p` does not need to be a +/// pointer returned by this [`Allocator`]. +/// - `ptr` is allowed to be `None`; in this case a new memory allocation is created and +/// `old_layout` is ignored. +/// - `old_layout` must match the `Layout` the allocation has been created with. +/// +/// # Guarantees +/// +/// This function has the same guarantees as [`Allocator::alloc`]. When `ptr == Some(p)`, then +/// it additionally guarantees that: +/// - the contents of the memory pointed to by `p` are preserved up to the lesser of the new +/// and old size, i.e. `ret_ptr[0..min(layout.size(), old_layout.size())] == +/// p[0..min(layout.size(), old_layout.size())]`. +/// - when the return value is `Err(AllocError)`, then `ptr` is still valid. +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError >; + +/// Free an existing memory allocation. +/// +/// # Safety +/// +/// - `ptr` must point to an existing and valid memory allocation created by this [`Allocator`]; +/// if `old_layout` is zero-sized `p` does not need to be a pointer returned by this +/// [`Allocator`]. +/// - `layout` must match the `Layout` the allocation has been created with. +/// - The memory allocation at `ptr` must never again be read from or written to. +unsafe fn free (ptr:NonNull <u8 >, layout:Layout ) { +// SAFETY: The caller guarantees that `ptr` points at a valid allocation created by this +// allocator. We are passing a `Layout` with the smallest possible alignment, so it is +// smaller than or equal to the alignment previously used with this allocation. +let _ =unsafe {Self ::realloc (Some (ptr),Layout ::new ::<()>(), layout,Flags (0 )) }; + } +}
Signature · 1 line
2.46.1
Re: [PATCH v8 01/29] rust: alloc: add `Allocator` trait
Quoted 9 lines · On Tue, 1 Oct 2024 16:59:36 +0200 Danilo Krummrich <[email protected]> wrote:
Add a kernel specific
Allocatortrait, that in contrast to the one in Rust's core library doesn't require unstable features and supports GFP flags.Subsequent patches add the following trait implementors:
Kmalloc,VmallocandKVmalloc.Reviewed-by Alice Ryhl <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>
Quoted 3 lines
--- rust/kernel/alloc.rs | 101 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+)
Re: [PATCH v8 01/29] rust: alloc: add `Allocator` trait
In reply to Danilo KrummrichQuoted 4 lines · Danilo Krummrich
Add a kernel specific
Allocatortrait, that in contrast to the one in Rust's core library doesn't require unstable features and supports GFP flags.
Quoted 8 lines
Subsequent patches add the following trait implementors:
Kmalloc,VmallocandKVmalloc.Reviewed-by Alice Ryhl <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc.rs | 101 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+)
---
Cheers,
Benno
Danilo Krummrich
maintainerPATCH v8 2/29 rust: alloc: separate `aligned_size` from `krealloc_aligned`
In reply to Danilo KrummrichSeparate aligned_size from krealloc_aligned.
Subsequent patches implement Allocator derivates, such as Kmalloc, that require aligned_size and replace the original krealloc_aligned.
--- rust/kernel/alloc/allocator.rs | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
rust/kernel/alloc/allocator.rs +12 -7
@@ -8,6 +8,17 @@struct KernelAllocator ; +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. +fn aligned_size (new_layout:Layout ) ->usize { +// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first. +let layout = new_layout.pad_to_align (); + +// Note that `layout.size()` (after padding) is guaranteed to be a multiple of `layout.align()` +// which together with the slab guarantees means the `krealloc` will return a properly aligned +// object (see comments in `kmalloc()` for more information). + layout.size () +} +/// Calls `krealloc` with a proper size to alloc a new object aligned to `new_layout`'s alignment. /// /// # Safety
@@ -15,13 +26,7 @@/// - `ptr` can be either null or a pointer which has been allocated by this allocator. /// - `new_layout` must have a non-zero size. pub (crate )unsafe fn krealloc_aligned (ptr: *mut u8 , new_layout:Layout , flags:Flags ) -> *mut u8 { -// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first. -let layout = new_layout.pad_to_align (); - -// Note that `layout.size()` (after padding) is guaranteed to be a multiple of `layout.align()` -// which together with the slab guarantees means the `krealloc` will return a properly aligned -// object (see comments in `kmalloc()` for more information). -let size = layout.size (); +let size =aligned_size (new_layout);// SAFETY: // - `ptr` is either null or a pointer returned from a previous `k{re}alloc()` by the
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 3/29 rust: alloc: rename `KernelAllocator` to `Kmalloc`
In reply to Danilo KrummrichSubsequent patches implement Vmalloc and KVmalloc allocators, hence align KernelAllocator to this naming scheme.
--- rust/kernel/alloc/allocator.rs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
rust/kernel/alloc/allocator.rs +3 -3
@@ -6,7 +6,7 @@use core::alloc::{GlobalAlloc ,Layout };use core::ptr; -struct KernelAllocator ; +struct Kmalloc ;/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {
@@ -36,7 +36,7 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: Funsafe { bindings::krealloc (ptras *const core::ffi::c_void , size, flags.0 )as *mut u8 } } -unsafe impl GlobalAlloc for KernelAllocator { +unsafe impl GlobalAlloc for Kmalloc {unsafe fn alloc (&self, layout:Layout ) -> *mut u8 {// SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety // requirement.
@@ -72,7 +72,7 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { }#[global_allocator] -static ALLOCATOR :KernelAllocator =KernelAllocator ; +static ALLOCATOR :Kmalloc =Kmalloc ;// See <https://github.com/rust-lang/rust/pull/86844>. #[no_mangle]
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 4/29 rust: alloc: implement `ReallocFunc`
In reply to Danilo KrummrichReallocFunc is an abstraction for the kernel's realloc derivates, such as krealloc, vrealloc and kvrealloc.
All of the named functions share the same function signature and implement the same semantics. The ReallocFunc abstractions provides a generalized wrapper around those, to trivialize the implementation of Kmalloc, Vmalloc and KVmalloc in subsequent patches.
--- rust/kernel/alloc.rs | 8 ++++ rust/kernel/alloc/allocator.rs | 70 ++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+)
rust/kernel/alloc.rs +8 -0
@@ -187,3 +187,11 @@ unsafe fn free(ptr: NonNull<u8>, layout: Layout) {let _ =unsafe {Self ::realloc (Some (ptr),Layout ::new ::<()>(), layout,Flags (0 )) }; } } + +/// Returns a properly aligned dangling pointer from the given `layout`. +pub (crate )fn dangling_from_layout (layout:Layout ) ->NonNull <u8 > { +let ptr = layout.align ()as *mut u8 ; + +// SAFETY: `layout.align()` (and hence `ptr`) is guaranteed to be non-zero. +unsafe {NonNull ::new_unchecked (ptr) } +}
rust/kernel/alloc/allocator.rs +70 -0
86 lines not on this page · raw message
Signature · 1 line
2.46.1
Re: [PATCH v8 04/29] rust: alloc: implement `ReallocFunc`
Quoted 8 lines · Danilo Krummrich
ReallocFuncis an abstraction for the kernel's realloc derivates, such askrealloc,vreallocandkvrealloc.All of the named functions share the same function signature and implement the same semantics. The
ReallocFuncabstractions provides a generalized wrapper around those, to trivialize the implementation ofKmalloc,VmallocandKVmallocin subsequent patches.
Quoted 5 lines
Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc.rs | 8 ++++ rust/kernel/alloc/allocator.rs | 70 ++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+)
---
Cheers,
Benno
Re: [PATCH v8 04/29] rust: alloc: implement `ReallocFunc`
In reply to Danilo KrummrichQuoted 9 lines · On Tue, 1 Oct 2024 16:59:39 +0200 Danilo Krummrich <[email protected]> wrote:
ReallocFuncis an abstraction for the kernel's realloc derivates, such askrealloc,vreallocandkvrealloc.All of the named functions share the same function signature and implement the same semantics. The
ReallocFuncabstractions provides a generalized wrapper around those, to trivialize the implementation ofKmalloc,VmallocandKVmallocin subsequent patches.Signed-off-by Danilo Krummrich <[email protected]>
Quoted 4 lines
--- rust/kernel/alloc.rs | 8 ++++ rust/kernel/alloc/allocator.rs | 70 ++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+)
Danilo Krummrich
maintainerPATCH v8 5/29 rust: alloc: make `allocator` module public
In reply to Danilo KrummrichSubsequent patches implement allocators such as Kmalloc, Vmalloc, KVmalloc; we need them to be available outside of the kernel crate as well.
--- rust/kernel/alloc.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/alloc.rs +1 -1
@@ -4,7 +4,7 @@#[cfg(not(test))] #[cfg(not(testlib))] -mod allocator; +pub mod allocator;pub mod box_ext;pub mod vec_ext;
Signature · 1 line
2.46.1
Re: [PATCH v8 05/29] rust: alloc: make `allocator` module public
Quoted 5 lines · Danilo Krummrich
Subsequent patches implement allocators such as
Kmalloc,Vmalloc,KVmalloc; we need them to be available outside of the kernel crate as well.Signed-off-by Danilo Krummrich <[email protected]>
---
Cheers,
Benno
Quoted 20 lines
--- rust/kernel/alloc.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)rust/kernel/alloc.rs +1
-2@@ -4,7 +4,7 @@#[cfg(not(test))] #[cfg(not(testlib))] -mod allocator; +pub mod allocator;pub mod box_ext;pub mod vec_ext; --2.46.1
Re: [PATCH v8 05/29] rust: alloc: make `allocator` module public
In reply to Danilo KrummrichQuoted 5 lines · On Tue, 1 Oct 2024 16:59:40 +0200 Danilo Krummrich <[email protected]> wrote:
Subsequent patches implement allocators such as
Kmalloc,Vmalloc,KVmalloc; we need them to be available outside of the kernel crate as well.Signed-off-by Danilo Krummrich <[email protected]>
Quoted 17 lines
--- rust/kernel/alloc.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)rust/kernel/alloc.rs +1
-1@@ -4,7 +4,7 @@#[cfg(not(test))] #[cfg(not(testlib))] -mod allocator; +pub mod allocator;pub mod box_ext;pub mod vec_ext;
Danilo Krummrich
maintainerPATCH v8 6/29 rust: alloc: implement `Allocator` for `Kmalloc`
In reply to Danilo KrummrichImplement Allocator for Kmalloc, the kernel's default allocator, typically used for objects smaller than page size.
All memory allocations made with Kmalloc end up in krealloc().
It serves as allocator for the subsequently introduced types KBox and KVec.
--- rust/kernel/alloc/allocator.rs | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)
rust/kernel/alloc/allocator.rs +28 -3
@@ -13,10 +13,16 @@use core::ptr;use core::ptr::NonNull ; -use crate ::alloc::AllocError ; +use crate ::alloc::{AllocError ,Allocator };use crate ::bindings; -struct Kmalloc ; +/// The contiguous kernel allocator. +/// +/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also +/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific. +/// +/// For more details see [self]. +pub struct Kmalloc ;/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {
@@ -53,8 +59,10 @@ struct ReallocFunc( unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void, ); -#[expect(dead_code)] impl ReallocFunc { +// INVARIANT: `krealloc` satisfies the type invariants. +const KREALLOC :Self =Self (bindings::krealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`].
@@ -106,6 +114,23 @@ unsafe fn call( } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for Kmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`. +unsafe {ReallocFunc ::KREALLOC .call (ptr, layout, old_layout, flags) } + } +} +unsafe impl GlobalAlloc for Kmalloc {unsafe fn alloc (&self, layout:Layout ) -> *mut u8 {// SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
Signature · 1 line
2.46.1
Re: [PATCH v8 06/29] rust: alloc: implement `Allocator` for `Kmalloc`
Quoted 5 lines · Danilo Krummrich
Implement
AllocatorforKmalloc, the kernel's default allocator, typically used for objects smaller than page size.All memory allocations made with
Kmallocend up inkrealloc().
Quoted 7 lines
It serves as allocator for the subsequently introduced types
KBoxandKVec.Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc/allocator.rs | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)
---
Cheers,
Benno
Re: [PATCH v8 06/29] rust: alloc: implement `Allocator` for `Kmalloc`
In reply to Danilo KrummrichQuoted 9 lines · On Tue, 1 Oct 2024 16:59:41 +0200 Danilo Krummrich <[email protected]> wrote:
Implement
AllocatorforKmalloc, the kernel's default allocator, typically used for objects smaller than page size.All memory allocations made with
Kmallocend up inkrealloc().It serves as allocator for the subsequently introduced types
KBoxandKVec.Signed-off-by Danilo Krummrich <[email protected]>
Quoted 63 lines
--- rust/kernel/alloc/allocator.rs | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)rust/kernel/alloc/allocator.rs +28
-3@@ -13,10 +13,16 @@use core::ptr;use core::ptr::NonNull ; -use crate ::alloc::AllocError ; +use crate ::alloc::{AllocError ,Allocator };use crate ::bindings; -struct Kmalloc ; +/// The contiguous kernel allocator. +/// +/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also +/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific. +/// +/// For more details see [self]. +pub struct Kmalloc ;/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {@@ -53,8 +59,10 @@ struct ReallocFunc( unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void, ); -#[expect(dead_code)] impl ReallocFunc { +// INVARIANT: `krealloc` satisfies the type invariants. +const KREALLOC :Self =Self (bindings::krealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`]. @@ -106,6 +114,23 @@ unsafe fn call( } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for Kmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`. +unsafe {ReallocFunc ::KREALLOC .call (ptr, layout, old_layout, flags) } + } +} +unsafe impl GlobalAlloc for Kmalloc {unsafe fn alloc (&self, layout:Layout ) -> *mut u8 {// SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
Danilo Krummrich
maintainerPATCH v8 7/29 rust: alloc: add module `allocator_test`
In reply to Danilo KrummrichAllocators, such as Kmalloc, will be used by e.g. Box and Vec in subsequent patches, and hence this dependency propagates throughout the whole kernel.
Add the allocator_test module that provides an empty implementation for all Allocators in the kernel, such that we don't break the rusttest make target in subsequent patches.
--- rust/kernel/alloc.rs | 9 +++++++-- rust/kernel/alloc/allocator_test.rs | 19 +++++++++++++++++++ 2 files changed, 26 insertions(+), 2 deletions(-) create mode 100644 rust/kernel/alloc/allocator_test.rs
rust/kernel/alloc.rs +7 -2
@@ -2,12 +2,17 @@//! Extensions to the [`alloc`] crate. -#[cfg(not(test))] -#[cfg(not(testlib))] +#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod box_ext;pub mod vec_ext; +#[cfg(any(test, testlib))] +pub mod allocator_test; + +#[cfg(any(test, testlib))] +pub use self::allocator_testas allocator; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;
rust/kernel/alloc/allocator_test.rs new file +19 -0
@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 + +#![allow(missing_docs)] + +use super ::{AllocError ,Allocator ,Flags }; +use core::alloc::Layout ; +use core::ptr::NonNull ; + +pub struct Kmalloc ; + +unsafe impl Allocator for Kmalloc { +unsafe fn realloc ( + _ptr:Option <NonNull <u8 >>, + _layout:Layout , + _flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +panic! (); + } +}
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 8/29 rust: alloc: implement `Vmalloc` allocator
In reply to Danilo KrummrichImplement Allocator for Vmalloc, the kernel's virtually contiguous allocator, typically used for larger objects, (much) larger than page size.
All memory allocations made with Vmalloc end up in vrealloc().
--- rust/helpers/helpers.c | 1 + rust/helpers/vmalloc.c | 9 +++++++ rust/kernel/alloc/allocator.rs | 37 +++++++++++++++++++++++++++++ rust/kernel/alloc/allocator_test.rs | 1 + 4 files changed, 48 insertions(+) create mode 100644 rust/helpers/vmalloc.c
rust/helpers/helpers.c +1 -0
@@ -22,5 +22,6 @@#include "spinlock.c" #include "task.c" #include "uaccess.c" +#include "vmalloc.c" #include "wait.c" #include "workqueue.c"
rust/helpers/vmalloc.c new file +9 -0
@@ -0,0 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/vmalloc.h> + +void * __must_check __realloc_size(2) +rust_helper_vrealloc(const void *p, size_t size, gfp_t flags) +{ +return vrealloc (p, size, flags); +}
rust/kernel/alloc/allocator.rs +37 -0
@@ -15,6 +15,7 @@use crate ::alloc::{AllocError ,Allocator };use crate ::bindings; +use crate ::pr_warn;/// The contiguous kernel allocator. ///
@@ -24,6 +25,15 @@/// For more details see [self]. pub struct Kmalloc ; +/// The virtually contiguous kernel allocator. +/// +/// `Vmalloc` allocates pages from the page level allocator and maps them into the contiguous kernel +/// virtual space. It is typically used for large allocations. The memory allocated with this +/// allocator is not physically contiguous. +/// +/// For more details see [self]. +pub struct Vmalloc ; +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
@@ -63,6 +73,9 @@ impl ReallocFunc {// INVARIANT: `krealloc` satisfies the type invariants. const KREALLOC :Self =Self (bindings::krealloc); +// INVARIANT: `vrealloc` satisfies the type invariants. +const VREALLOC :Self =Self (bindings::vrealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`].
@@ -166,6 +179,30 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for Vmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// TODO: Support alignments larger than PAGE_SIZE. +if layout.align () > bindings::PAGE_SIZE { +pr_warn! ("Vmalloc does not support alignments larger than PAGE_SIZE yet.\n" ); +return Err (AllocError ); + } + +// SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously +// allocated with this `Allocator`. +unsafe {ReallocFunc ::VREALLOC .call (ptr, layout, old_layout, flags) } + } +} +#[global_allocator] static ALLOCATOR :Kmalloc =Kmalloc ;
rust/kernel/alloc/allocator_test.rs +1 -0
@@ -7,6 +7,7 @@use core::ptr::NonNull ;pub struct Kmalloc ; +pub type Vmalloc =Kmalloc ;unsafe impl Allocator for Kmalloc { unsafe fn realloc(
Signature · 1 line
2.46.1
Re: [PATCH v8 08/29] rust: alloc: implement `Vmalloc` allocator
Quoted 9 lines · On Tue, 1 Oct 2024 16:59:43 +0200 Danilo Krummrich <[email protected]> wrote:
Implement
AllocatorforVmalloc, the kernel's virtually contiguous allocator, typically used for larger objects, (much) larger than page size.All memory allocations made with
Vmallocend up invrealloc().Reviewed-by Alice Ryhl <[email protected]>Reviewed-by Benno Lossin <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>
Quoted 115 lines
--- rust/helpers/helpers.c | 1 + rust/helpers/vmalloc.c | 9 +++++++ rust/kernel/alloc/allocator.rs | 37 +++++++++++++++++++++++++++++ rust/kernel/alloc/allocator_test.rs | 1 + 4 files changed, 48 insertions(+) create mode 100644 rust/helpers/vmalloc.crust/helpers/helpers.c +1
-0@@ -22,5 +22,6 @@#include "spinlock.c" #include "task.c" #include "uaccess.c" +#include "vmalloc.c" #include "wait.c" #include "workqueue.c" rust/helpers/vmalloc.c new file +9
-0@@ -0,0 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/vmalloc.h> + +void * __must_check __realloc_size(2) +rust_helper_vrealloc(const void *p, size_t size, gfp_t flags) +{ +return vrealloc (p, size, flags); +}rust/kernel/alloc/allocator.rs +37
-0@@ -15,6 +15,7 @@use crate ::alloc::{AllocError ,Allocator };use crate ::bindings; +use crate ::pr_warn;/// The contiguous kernel allocator. /// @@ -24,6 +25,15 @@/// For more details see [self]. pub struct Kmalloc ; +/// The virtually contiguous kernel allocator. +/// +/// `Vmalloc` allocates pages from the page level allocator and maps them into the contiguous kernel +/// virtual space. It is typically used for large allocations. The memory allocated with this +/// allocator is not physically contiguous. +/// +/// For more details see [self]. +pub struct Vmalloc ; +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first. @@ -63,6 +73,9 @@ impl ReallocFunc {// INVARIANT: `krealloc` satisfies the type invariants. const KREALLOC :Self =Self (bindings::krealloc); +// INVARIANT: `vrealloc` satisfies the type invariants. +const VREALLOC :Self =Self (bindings::vrealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`]. @@ -166,6 +179,30 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for Vmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// TODO: Support alignments larger than PAGE_SIZE. +if layout.align () > bindings::PAGE_SIZE { +pr_warn! ("Vmalloc does not support alignments larger than PAGE_SIZE yet.\n" ); +return Err (AllocError ); + } + +// SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously +// allocated with this `Allocator`. +unsafe {ReallocFunc ::VREALLOC .call (ptr, layout, old_layout, flags) } + } +} +#[global_allocator] static ALLOCATOR :Kmalloc =Kmalloc ;rust/kernel/alloc/allocator_test.rs +1
-0@@ -7,6 +7,7 @@use core::ptr::NonNull ;pub struct Kmalloc ; +pub type Vmalloc =Kmalloc ;unsafe impl Allocator for Kmalloc { unsafe fn realloc(
Danilo Krummrich
maintainerPATCH v8 9/29 rust: alloc: implement `KVmalloc` allocator
In reply to Danilo KrummrichImplement Allocator for KVmalloc, an Allocator that tries to allocate memory wth kmalloc first and, on failure, falls back to vmalloc.
All memory allocations made with KVmalloc end up in kvrealloc_noprof(); all frees in kvfree().
--- rust/helpers/slab.c | 6 +++++ rust/kernel/alloc/allocator.rs | 36 +++++++++++++++++++++++++++++ rust/kernel/alloc/allocator_test.rs | 1 + 3 files changed, 43 insertions(+)
rust/helpers/slab.c +6 -0
@@ -7,3 +7,9 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) {return krealloc (objp, new_size, flags); } + +void * __must_check __realloc_size(2) +rust_helper_kvrealloc(const void *p, size_t size, gfp_t flags) +{ +return kvrealloc (p, size, flags); +}
rust/kernel/alloc/allocator.rs +36 -0
@@ -34,6 +34,15 @@/// For more details see [self]. pub struct Vmalloc ; +/// The kvmalloc kernel allocator. +/// +/// `KVmalloc` attempts to allocate memory with `Kmalloc` first, but falls back to `Vmalloc` upon +/// failure. This allocator is typically used when the size for the requested allocation is not +/// known and may exceed the capabilities of `Kmalloc`. +/// +/// For more details see [self]. +pub struct KVmalloc ; +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
@@ -76,6 +85,9 @@ impl ReallocFunc {// INVARIANT: `vrealloc` satisfies the type invariants. const VREALLOC :Self =Self (bindings::vrealloc); +// INVARIANT: `kvrealloc` satisfies the type invariants. +const KVREALLOC :Self =Self (bindings::kvrealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`].
@@ -203,6 +215,30 @@ unsafe fn realloc( } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for KVmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// TODO: Support alignments larger than PAGE_SIZE. +if layout.align () > bindings::PAGE_SIZE { +pr_warn! ("KVmalloc does not support alignments larger than PAGE_SIZE yet.\n" ); +return Err (AllocError ); + } + +// SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously +// allocated with this `Allocator`. +unsafe {ReallocFunc ::KVREALLOC .call (ptr, layout, old_layout, flags) } + } +} +#[global_allocator] static ALLOCATOR :Kmalloc =Kmalloc ;
rust/kernel/alloc/allocator_test.rs +1 -0
@@ -8,6 +8,7 @@pub struct Kmalloc ;pub type Vmalloc =Kmalloc ; +pub type KVmalloc =Kmalloc ;unsafe impl Allocator for Kmalloc { unsafe fn realloc(
Signature · 1 line
2.46.1
Re: [PATCH v8 09/29] rust: alloc: implement `KVmalloc` allocator
Quoted 10 lines · On Tue, 1 Oct 2024 16:59:44 +0200 Danilo Krummrich <[email protected]> wrote:
Implement
AllocatorforKVmalloc, anAllocatorthat tries to allocate memory wthkmallocfirst and, on failure, falls back tovmalloc.All memory allocations made with
KVmallocend up inkvrealloc_noprof(); all frees inkvfree().Reviewed-by Alice Ryhl <[email protected]>Reviewed-by Benno Lossin <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>
Quoted 93 lines
--- rust/helpers/slab.c | 6 +++++ rust/kernel/alloc/allocator.rs | 36 +++++++++++++++++++++++++++++ rust/kernel/alloc/allocator_test.rs | 1 + 3 files changed, 43 insertions(+)rust/helpers/slab.c +6
-0@@ -7,3 +7,9 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) {return krealloc (objp, new_size, flags); } + +void * __must_check __realloc_size(2) +rust_helper_kvrealloc(const void *p, size_t size, gfp_t flags) +{ +return kvrealloc (p, size, flags); +}rust/kernel/alloc/allocator.rs +36
-0@@ -34,6 +34,15 @@/// For more details see [self]. pub struct Vmalloc ; +/// The kvmalloc kernel allocator. +/// +/// `KVmalloc` attempts to allocate memory with `Kmalloc` first, but falls back to `Vmalloc` upon +/// failure. This allocator is typically used when the size for the requested allocation is not +/// known and may exceed the capabilities of `Kmalloc`. +/// +/// For more details see [self]. +pub struct KVmalloc ; +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first. @@ -76,6 +85,9 @@ impl ReallocFunc {// INVARIANT: `vrealloc` satisfies the type invariants. const VREALLOC :Self =Self (bindings::vrealloc); +// INVARIANT: `kvrealloc` satisfies the type invariants. +const KVREALLOC :Self =Self (bindings::kvrealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`]. @@ -203,6 +215,30 @@ unsafe fn realloc( } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for KVmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// TODO: Support alignments larger than PAGE_SIZE. +if layout.align () > bindings::PAGE_SIZE { +pr_warn! ("KVmalloc does not support alignments larger than PAGE_SIZE yet.\n" ); +return Err (AllocError ); + } + +// SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously +// allocated with this `Allocator`. +unsafe {ReallocFunc ::KVREALLOC .call (ptr, layout, old_layout, flags) } + } +} +#[global_allocator] static ALLOCATOR :Kmalloc =Kmalloc ;rust/kernel/alloc/allocator_test.rs +1
-0@@ -8,6 +8,7 @@pub struct Kmalloc ;pub type Vmalloc =Kmalloc ; +pub type KVmalloc =Kmalloc ;unsafe impl Allocator for Kmalloc { unsafe fn realloc(
Danilo Krummrich
maintainerPATCH v8 10/29 rust: alloc: add __GFP_NOWARN to `Flags`
In reply to Danilo KrummrichSome test cases in subsequent patches provoke allocation failures. Add __GFP_NOWARN to enable test cases to silence unpleasant warnings.
--- rust/bindings/bindings_helper.h | 1 + rust/kernel/alloc.rs | 5 +++++ 2 files changed, 6 insertions(+)
rust/bindings/bindings_helper.h +1 -0
@@ -31,4 +31,5 @@ const gfp_t RUST_CONST_HELPER_GFP_KERNEL_ACCOUNT = GFP_KERNEL_ACCOUNT;const gfp_t RUST_CONST_HELPER_GFP_NOWAIT =GFP_NOWAIT ;const gfp_t RUST_CONST_HELPER___GFP_ZERO = __GFP_ZERO;const gfp_t RUST_CONST_HELPER___GFP_HIGHMEM = ___GFP_HIGHMEM; +const gfp_t RUST_CONST_HELPER___GFP_NOWARN = ___GFP_NOWARN;const blk_features_t RUST_CONST_HELPER_BLK_FEAT_ROTATIONAL =BLK_FEAT_ROTATIONAL ;
rust/kernel/alloc.rs +5 -0
@@ -91,6 +91,11 @@ pub mod flags {/// use any filesystem callback. It is very likely to fail to allocate memory, even for very /// small allocations. pub const GFP_NOWAIT :Flags =Flags (bindings::GFP_NOWAIT ); + +/// Suppresses allocation failure reports. +/// +/// This is normally or'd with other flags. +pub const __GFP_NOWARN:Flags =Flags (bindings::__GFP_NOWARN); }/// The kernel's [`Allocator`] trait.
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 11/29 rust: alloc: implement kernel `Box`
In reply to Danilo KrummrichBox provides the simplest way to allocate memory for a generic type with one of the kernel's allocators, e.g. Kmalloc, Vmalloc or KVmalloc.
In contrast to Rust's Box type, the kernel Box type considers the kernel's GFP flags for all appropriate functions, always reports allocation failures through Result<_, AllocError> and remains independent from unstable features.
--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kbox.rs | 455 ++++++++++++++++++++++++++++++++++++++ rust/kernel/prelude.rs | 2 +- 3 files changed, 462 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/alloc/kbox.rs
rust/kernel/alloc.rs +6 -0
@@ -5,6 +5,7 @@#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod box_ext; +pub mod kbox;pub mod vec_ext;#[cfg(any(test, testlib))]
@@ -13,6 +14,11 @@#[cfg(any(test, testlib))] pub use self::allocator_testas allocator; +pub use self::kbox::Box ; +pub use self::kbox::KBox ; +pub use self::kbox::KVBox ; +pub use self::kbox::VBox ; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;
rust/kernel/alloc/kbox.rs new file +455 -0
455 lines not on this page · raw message
rust/kernel/prelude.rs +1 -1
@@ -14,7 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{box_ext::BoxExt , flags::*, vec_ext::VecExt }; +pub use crate ::alloc::{box_ext::BoxExt , flags::*, vec_ext::VecExt ,KBox ,KVBox ,VBox };#[doc(no_inline)] pub use alloc::{boxed::Box , vec::Vec };
Signature · 1 line
2.46.1
Re: [PATCH v8 11/29] rust: alloc: implement kernel `Box`
Quoted 11 lines · On Tue, 1 Oct 2024 16:59:46 +0200 Danilo Krummrich <[email protected]> wrote:
Boxprovides the simplest way to allocate memory for a generic type with one of the kernel's allocators, e.g.Kmalloc,VmallocorKVmalloc.In contrast to Rust's
Boxtype, the kernelBoxtype considers the kernel's GFP flags for all appropriate functions, always reports allocation failures throughResult<_, AllocError>and remains independent from unstable features.Reviewed-by Benno Lossin <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>
Quoted 6 lines
--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kbox.rs | 455 ++++++++++++++++++++++++++++++++++++++ rust/kernel/prelude.rs | 2 +- 3 files changed, 462 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/alloc/kbox.rs
Re: [PATCH v8 11/29] rust: alloc: implement kernel `Box`
In reply to Danilo Krummrichrust/kernel/alloc/kbox.rs line 150
Quoted 7 lines · Danilo Krummrich
rust/kernel/alloc/kbox.rs new file +6
-0+/// # Examples +/// +/// ``` +/// let x = KBox::new(24, GFP_KERNEL)?; +/// let ptr = KBox::into_raw(x); +/// let x = unsafe { KBox::from_raw(ptr) };
This is the only place that, after applying your series on top of the lints one (i.e. current rust-next), triggered a missing // SAFETY comment lint :)
If there is no new version, what do you want me to write when I apply it?
Thanks!
Cheers,
Miguel
Danilo Krummrich
maintainerRe: [PATCH v8 11/29] rust: alloc: implement kernel `Box`
Quoted 9 lines · Miguel Ojeda
Review
Quoted 7 lines · Danilo Krummrich
(quoted diff) +6
-0+ /// # Examples + /// + /// ``` + /// let x = KBox::new(24, GFP_KERNEL)?; + /// let ptr = KBox::into_raw(x); + /// let x = unsafe { KBox::from_raw(ptr) };
Quoted 5 lines
This is the only place that, after applying your series on top of the lints one (i.e. current
rust-next), triggered a missing// SAFETYcomment lint :)If there is no new version, what do you want me to write when I apply it?
rust/kernel/alloc/kbox.rs +1 -0
@@ -147,6 +147,7 @@ impl<T, A> Box<T, A>/// ``` /// let x = KBox::new(24, GFP_KERNEL)?; /// let ptr = KBox::into_raw(x); +/// // SAFETY: `ptr` comes from a previous call to `KBox::into_raw`. /// let x = unsafe { KBox::from_raw(ptr) }; /// /// assert_eq!(*x, 24);
But maybe it's better to just send a new version, to also fix the missing #![feature(inline_const)] for Vec and missing import for size_of in ArrayLayout.
Quoted 6 lines
Thanks!
Cheers,
Miguel
Danilo Krummrich
maintainerPATCH v8 12/29 rust: treewide: switch to our kernel `Box` type
In reply to Danilo KrummrichNow that we got the kernel Box type in place, convert all existing Box users to make use of it.
--- drivers/block/rnull.rs | 4 +-- rust/kernel/init.rs | 51 ++++++++++++++++--------------- rust/kernel/init/__internal.rs | 2 +- rust/kernel/rbtree.rs | 49 ++++++++++++++++------------- rust/kernel/sync/arc.rs | 17 +++++------ rust/kernel/sync/condvar.rs | 4 +-- rust/kernel/sync/lock/mutex.rs | 2 +- rust/kernel/sync/lock/spinlock.rs | 2 +- rust/kernel/workqueue.rs | 20 ++++++------ rust/macros/lib.rs | 6 ++-- 10 files changed, 81 insertions(+), 76 deletions(-)
drivers/block/rnull.rs +2 -2
@@ -32,7 +32,7 @@ }struct NullBlkModule { - _disk:Pin <Box <Mutex <GenDisk <NullBlkDevice >>>>, + _disk:Pin <KBox <Mutex <GenDisk <NullBlkDevice >>>>, }impl kernel::Module for NullBlkModule {
@@ -47,7 +47,7 @@ fn init(_module: &'static ThisModule) -> Result<Self> { .rotational(false) .build (format_args! ("rnullb{}" ,0 ), tagset)?; -let disk =Box ::pin_init (new_mutex! (disk,"nullb:disk" ), flags::GFP_KERNEL )?; +let disk =KBox ::pin_init (new_mutex! (disk,"nullb:disk" ), flags::GFP_KERNEL )?;Ok (Self { _disk: disk }) }
rust/kernel/init.rs +26 -25
178 lines not on this page · raw message
rust/kernel/init/__internal.rs +1 -1
@@ -102,7 +102,7 @@ fn make_closure<F, O, E>(self, f: F) -> F } } -pub struct AllData <T : ?Sized >(PhantomData <fn (Box <T >) ->Box <T >>); +pub struct AllData <T : ?Sized >(PhantomData <fn (KBox <T >) ->KBox <T >>);impl <T : ?Sized >Clone for AllData <T > {fn clone (&self) ->Self {
rust/kernel/rbtree.rs +27 -22
139 lines not on this page · raw message
rust/kernel/sync/arc.rs +8 -9
@@ -17,13 +17,12 @@//! [`Arc`]: https://doc.rust-lang.org/std/sync/struct.Arc.html use crate ::{ - alloc::{box_ext::BoxExt ,AllocError ,Flags }, + alloc::{AllocError ,Flags ,KBox }, bindings, init::{self,InPlaceInit ,Init ,PinInit }, try_init, types::{ForeignOwnable ,Opaque }, }; -use alloc::boxed::Box ;use core::{ alloc::Layout , fmt,
@@ -204,11 +203,11 @@ pub fn new(contents: T, flags: Flags) -> Result<Self, AllocError> { data: contents, }; -let inner = <Box <_ >as BoxExt <_ >>::new (value, flags)?; +let inner =KBox ::new (value, flags)?;// SAFETY: We just created `inner` with a reference count of 1, which is owned by the new // `Arc` object. -Ok (unsafe {Self ::from_inner (Box ::leak (inner).into ()) }) +Ok (unsafe {Self ::from_inner (KBox ::leak (inner).into ()) }) } }
@@ -401,8 +400,8 @@ fn drop(&mut self) {if is_zero {// The count reached zero, we must free the memory. // -// SAFETY: The pointer was initialised from the result of `Box::leak`. -unsafe {drop (Box ::from_raw (self.ptr.as_ptr ())) }; +// SAFETY: The pointer was initialised from the result of `KBox::leak`. +unsafe {drop (KBox ::from_raw (self.ptr.as_ptr ())) }; } } }
@@ -647,7 +646,7 @@ pub fn new(value: T, flags: Flags) -> Result<Self, AllocError> {/// Tries to allocate a new [`UniqueArc`] instance whose contents are not initialised yet. pub fn new_uninit (flags:Flags ) ->Result <UniqueArc <MaybeUninit <T >>,AllocError > {// INVARIANT: The refcount is initialised to a non-zero value. - let inner = Box::try_init::<AllocError>( + let inner = KBox::try_init::<AllocError>( try_init!(ArcInner { // SAFETY: There are no safety requirements for this FFI call. refcount: Opaque::new(unsafe { bindings::REFCOUNT_INIT(1) }),
@@ -657,8 +656,8 @@ pub fn new_uninit(flags: Flags) -> Result<UniqueArc<MaybeUninit<T>>, AllocError> )?;Ok (UniqueArc {// INVARIANT: The newly-created object has a refcount of 1. -// SAFETY: The pointer from the `Box` is valid. - inner:unsafe {Arc ::from_inner (Box ::leak (inner).into ()) }, +// SAFETY: The pointer from the `KBox` is valid. + inner:unsafe {Arc ::from_inner (KBox ::leak (inner).into ()) }, }) } }
rust/kernel/sync/condvar.rs +2 -2
@@ -70,8 +70,8 @@ macro_rules! new_condvar {/// } /// /// /// Allocates a new boxed `Example`. -/// fn new_example() -> Result<Pin<Box<Example>>> { -/// Box::pin_init(pin_init!(Example { +/// fn new_example() -> Result<Pin<KBox<Example>>> { +/// KBox::pin_init(pin_init!(Example { /// value <- new_mutex!(0), /// value_changed <- new_condvar!(), /// }), GFP_KERNEL)
rust/kernel/sync/lock/mutex.rs +1 -1
@@ -58,7 +58,7 @@ macro_rules! new_mutex {/// } /// /// // Allocate a boxed `Example`. -/// let e = Box::pin_init(Example::new(), GFP_KERNEL)?; +/// let e = KBox::pin_init(Example::new(), GFP_KERNEL)?; /// assert_eq!(e.c, 10); /// assert_eq!(e.d.lock().a, 20); /// assert_eq!(e.d.lock().b, 30);
rust/kernel/sync/lock/spinlock.rs +1 -1
@@ -56,7 +56,7 @@ macro_rules! new_spinlock {/// } /// /// // Allocate a boxed `Example`. -/// let e = Box::pin_init(Example::new(), GFP_KERNEL)?; +/// let e = KBox::pin_init(Example::new(), GFP_KERNEL)?; /// assert_eq!(e.c, 10); /// assert_eq!(e.d.lock().a, 20); /// assert_eq!(e.d.lock().b, 30);
rust/kernel/workqueue.rs +10 -10
@@ -216,7 +216,7 @@ pub fn try_spawn<T: 'static + Send + FnOnce()>( func: Some(func), }); - self.enqueue (Box ::pin_init (init, flags).map_err (|_|AllocError )?); + self.enqueue (KBox ::pin_init (init, flags).map_err (|_|AllocError )?);Ok (()) } }
@@ -239,9 +239,9 @@ fn project(self: Pin<&mut Self>) -> &mut Option<T> { }impl <T :FnOnce ()>WorkItem for ClosureWork <T > { -type Pointer =Pin <Box <Self >>; +type Pointer =Pin <KBox <Self >>; -fn run (mut this:Pin <Box <Self >>) { +fn run (mut this:Pin <KBox <Self >>) {if let Some (func) = this.as_mut ().project ().take () { (func)() }
@@ -297,7 +297,7 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput/// Defines the method that should be called directly when a work item is executed. /// -/// This trait is implemented by `Pin<Box<T>>` and [`Arc<T>`], and is mainly intended to be +/// This trait is implemented by `Pin<KBox<T>>` and [`Arc<T>`], and is mainly intended to be /// implemented for smart pointer types. For your own structs, you would implement [`WorkItem`] /// instead. The [`run`] method on this trait will usually just perform the appropriate /// `container_of` translation and then call into the [`run`][WorkItem::run] method from the
@@ -329,7 +329,7 @@ pub unsafe trait WorkItemPointer<const ID: u64>: RawWorkItem<ID> {/// This trait is used when the `work_struct` field is defined using the [`Work`] helper. pub trait WorkItem <const ID :u64 =0 > {/// The pointer type that this struct is wrapped in. This will typically be `Arc<Self>` or -/// `Pin<Box<Self>>`. +/// `Pin<KBox<Self>>`. type Pointer :WorkItemPointer <ID >;/// The method that should be called when this work item is executed.
@@ -565,7 +565,7 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput } } -unsafe impl <T ,const ID :u64 >WorkItemPointer <ID >for Pin <Box <T >> +unsafe impl <T ,const ID :u64 >WorkItemPointer <ID >for Pin <KBox <T >>where T :WorkItem <ID ,Pointer =Self >,T :HasWork <T ,ID >,
@@ -576,7 +576,7 @@ unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Pin<Box<T>>// SAFETY: This computes the pointer that `__enqueue` got from `Arc::into_raw`. let ptr =unsafe {T ::work_container_of (ptr) };// SAFETY: This pointer comes from `Arc::into_raw` and we've been given back ownership. -let boxed =unsafe {Box ::from_raw (ptr) }; +let boxed =unsafe {KBox ::from_raw (ptr) };// SAFETY: The box was already pinned when it was enqueued. let pinned =unsafe {Pin ::new_unchecked (boxed) };
@@ -584,7 +584,7 @@ unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Pin<Box<T>> } } -unsafe impl <T ,const ID :u64 >RawWorkItem <ID >for Pin <Box <T >> +unsafe impl <T ,const ID :u64 >RawWorkItem <ID >for Pin <KBox <T >>where T :WorkItem <ID ,Pointer =Self >,T :HasWork <T ,ID >,
@@ -598,9 +598,9 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput// SAFETY: We're not going to move `self` or any of its fields, so its okay to temporarily // remove the `Pin` wrapper. let boxed =unsafe {Pin ::into_inner_unchecked (self) }; -let ptr =Box ::into_raw (boxed); +let ptr =KBox ::into_raw (boxed); -// SAFETY: Pointers into a `Box` point at a valid value. +// SAFETY: Pointers into a `KBox` point at a valid value. let work_ptr =unsafe {T ::raw_get_work (ptr) };// SAFETY: `raw_get_work` returns a pointer to a valid value. let work_ptr =unsafe {Work ::raw_get (work_ptr) };
rust/macros/lib.rs +3 -3
@@ -243,7 +243,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// struct DriverData { /// #[pin] /// queue: Mutex<Vec<Command>>, -/// buf: Box<[u8; 1024 * 1024]>, +/// buf: KBox<[u8; 1024 * 1024]>, /// } /// ``` ///
@@ -252,7 +252,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// struct DriverData { /// #[pin] /// queue: Mutex<Vec<Command>>, -/// buf: Box<[u8; 1024 * 1024]>, +/// buf: KBox<[u8; 1024 * 1024]>, /// raw_info: *mut Info, /// } ///
@@ -282,7 +282,7 @@ pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream {/// struct DriverData { /// #[pin] /// queue: Mutex<Vec<Command>>, -/// buf: Box<[u8; 1024 * 1024]>, +/// buf: KBox<[u8; 1024 * 1024]>, /// raw_info: *mut Info, /// } ///
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 13/29 rust: alloc: remove extension of std's `Box`
In reply to Danilo KrummrichNow that all existing Box users were moved to the kernel Box type, remove the BoxExt extension and all other related extensions.
--- rust/kernel/alloc.rs | 1 - rust/kernel/alloc/box_ext.rs | 89 ------------------------------------ rust/kernel/init.rs | 46 +------------------ rust/kernel/lib.rs | 1 - rust/kernel/prelude.rs | 4 +- rust/kernel/types.rs | 50 -------------------- 6 files changed, 3 insertions(+), 188 deletions(-) delete mode 100644 rust/kernel/alloc/box_ext.rs
rust/kernel/alloc.rs +0 -1
@@ -4,7 +4,6 @@#[cfg(not(any(test, testlib)))] pub mod allocator; -pub mod box_ext;pub mod kbox;pub mod vec_ext;
rust/kernel/alloc/box_ext.rs deleted +0 -89
89 lines not on this page · raw message
rust/kernel/init.rs +1 -45
75 lines not on this page · raw message
rust/kernel/lib.rs +0 -1
@@ -14,7 +14,6 @@#![no_std] #![feature(coerce_unsized)] #![feature(dispatch_from_dyn)] -#![feature(new_uninit)] #![feature(receiver_trait)] #![feature(unsize)]
rust/kernel/prelude.rs +2 -2
@@ -14,10 +14,10 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{box_ext::BoxExt , flags::*, vec_ext::VecExt ,KBox ,KVBox ,VBox }; +pub use crate ::alloc::{flags::*, vec_ext::VecExt ,KBox ,KVBox ,VBox };#[doc(no_inline)] -pub use alloc::{boxed::Box , vec::Vec }; +pub use alloc::vec::Vec ;#[doc(no_inline)] pub use macros::{module, pin_data, pinned_drop, vtable,Zeroable };
rust/kernel/types.rs +0 -50
67 lines not on this page · raw message
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 14/29 rust: alloc: add `Box` to prelude
In reply to Danilo KrummrichNow that we removed BoxExt and the corresponding includes in prelude.rs, add the new kernel Box type instead.
--- rust/kernel/prelude.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/prelude.rs +1 -1
@@ -14,7 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{flags::*, vec_ext::VecExt ,KBox ,KVBox ,VBox }; +pub use crate ::alloc::{flags::*, vec_ext::VecExt ,Box ,KBox ,KVBox ,VBox };#[doc(no_inline)] pub use alloc::vec::Vec ;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 15/29 rust: alloc: introduce `ArrayLayout`
In reply to Danilo KrummrichFrom: Benno Lossin <[email protected]>
When allocating memory for arrays using allocators, the Layout::array function is typically used. It returns a result, since the given size might be too big. However, Vec and its iterators store their allocated capacity and thus they already did check that the size is not too big.
The ArrayLayout type provides this exact behavior, as it can be infallibly converted into a Layout. Instead of a usize capacity, Vec and other similar array-storing types can use ArrayLayout instead.
--- rust/kernel/alloc.rs | 1 + rust/kernel/alloc/layout.rs | 91 +++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100644 rust/kernel/alloc/layout.rs
rust/kernel/alloc.rs +1 -0
@@ -5,6 +5,7 @@#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod kbox; +pub mod layout;pub mod vec_ext;#[cfg(any(test, testlib))]
rust/kernel/alloc/layout.rs new file +91 -0
91 lines not on this page · raw message
Signature · 1 line
2.46.1
Re: [PATCH v8 15/29] rust: alloc: introduce `ArrayLayout`
Quoted 7 lines · On Tue, 1 Oct 2024 16:59:50 +0200 Danilo Krummrich <[email protected]> wrote:
From: Benno Lossin <[email protected]>
When allocating memory for arrays using allocators, the
Layout::arrayfunction is typically used. It returns a result, since the given size might be too big. However,Vecand its iterators store their allocated capacity and thus they already did check that the size is not too big.
Quoted 7 lines
The
ArrayLayouttype provides this exact behavior, as it can be infallibly converted into aLayout. Instead of ausizecapacity,Vecand other similar array-storing types can useArrayLayoutinstead.Signed-off-by Benno Lossin <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>
Quoted 78 lines
--- rust/kernel/alloc.rs | 1 + rust/kernel/alloc/layout.rs | 91 +++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100644 rust/kernel/alloc/layout.rsrust/kernel/alloc.rs +1
-0@@ -5,6 +5,7 @@#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod kbox; +pub mod layout;pub mod vec_ext;#[cfg(any(test, testlib))] rust/kernel/alloc/layout.rs new file +54
-0@@ -0,0 +1,91 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Memory layout. +//! +//! Custom layout types extending or improving [`Layout`]. + +use core::{alloc::Layout , marker::PhantomData }; + +/// Error when constructing an [`ArrayLayout`]. +pub struct LayoutError ; + +/// A layout for an array `[T; n]`. +/// +/// # Invariants +/// +/// - `len * size_of::<T>() <= isize::MAX` +pub struct ArrayLayout <T > { + len:usize , + _phantom:PhantomData <fn () ->T >, +} + +impl <T >Clone for ArrayLayout <T > { +fn clone (&self) ->Self { + *self + } +} +impl <T >Copy for ArrayLayout <T > {} + +const ISIZE_MAX :usize = isize::MAX as usize ; + +impl <T >ArrayLayout <T > { +/// Creates a new layout for `[T; 0]`. +pub const fn empty () ->Self { +// INVARIANT: `0 * size_of::<T>() <= isize::MAX` +Self { + len:0 , + _phantom:PhantomData , + } + } + +/// Creates a new layout for `[T; len]`. +/// +/// # Errors +/// +/// When `len * size_of::<T>()` overflows or when `len * size_of::<T>() > isize::MAX`. +pub const fn new (len:usize ) ->Result <Self ,LayoutError > { +match len.checked_mul (size_of ::<T >()) { +Some (len)if len <=ISIZE_MAX => { +// INVARIANT: we checked above that `len * size_of::<T>() <= isize::MAX` +Ok (Self { + len, + _phantom:PhantomData , + }) + }
rust/kernel/alloc/layout.rs line 64
Quoted 10 lines
rust/kernel/alloc/layout.rs new file +10
-0+ _ => Err(LayoutError), + } + } + +/// Creates a new layout for `[T; len]`. +/// +/// # Safety +/// +/// `len` must be a value, for which `len * size_of::<T>() <= isize::MAX` is true. +pub unsafe fn new_unchecked (len:usize ) ->Self {
nit: this can also be const, although this can be added when need arise (or one may use new(...).unwrap() in const context, since it doesn't actually generate a call to BUG).
Quoted 27 lines
rust/kernel/alloc/layout.rs new file +27
-0+// INVARIANT: By the safety requirements of this function +// `len * size_of::<T>() <= isize::MAX`. +Self { + len, + _phantom:PhantomData , + } + } + +/// Returns the number of array elements represented by this layout. +pub const fn len (&self) ->usize { + self.len + } + +/// Returns `true` when no array elements are represented by this layout. +pub const fn is_empty (&self) ->bool { + self.len ==0 + } +} + +impl <T >From <ArrayLayout <T >>for Layout { +fn from (value:ArrayLayout <T >) ->Self { +let res =Layout ::array ::<T >(value.len); +// SAFETY: by the type invariant of `ArrayLayout` we have +// `len * size_of::<T>() <= isize::MAX` and thus the result must be `Ok`. +unsafe { res.unwrap_unchecked () } + } +}
kernel test robot
build botRe: [PATCH v8 15/29] rust: alloc: introduce `ArrayLayout`
In reply to Danilo Krummrichkernel test robot noticed the following build errors:
Hi Danilo,
kernel test robot noticed the following build errors:
[auto build test ERROR on 9852d85ec9d492ebef56dc5f229416c925758edc]
url: https://github.com/intel-lab-lkp/linux/commits/Danilo-Krummrich/rust-alloc-add-Allocator-trait/20241001-230558 base: 9852d85ec9d492ebef56dc5f229416c925758edc patch link: https://lore.kernel.org/r/20241001150008.183102-16-dakr%40kernel.org patch subject: [PATCH v8 15/29] rust: alloc: introduce `ArrayLayout` config: riscv-randconfig-001-20241002 (https://download.01.org/0day-ci/archive/20241003/[email protected]/config) compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241003/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> error[E0425]: cannot find function `size_of` in this scope
--> rust/kernel/alloc/layout.rs:47:31
|
47 | match len.checked_mul(size_of::<T>()) {
| ^^^^^^^ not found in this scope
|
help: consider importing this function
|
7 + use core::mem::size_of;
|
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Danilo Krummrich
maintainerPATCH v8 16/29 rust: alloc: implement kernel `Vec` type
In reply to Danilo KrummrichVec provides a contiguous growable array type with contents allocated with the kernel's allocators (e.g. Kmalloc, Vmalloc or KVmalloc).
In contrast to Rust's stdlib Vec type, the kernel Vec type considers the kernel's GFP flags for all appropriate functions, always reports allocation failures through Result<_, AllocError> and remains independent from unstable features.
--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kvec.rs | 637 ++++++++++++++++++++++++++++++++++++++ rust/kernel/prelude.rs | 2 +- 3 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/alloc/kvec.rs
rust/kernel/alloc.rs +6 -0
@@ -5,6 +5,7 @@#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod kbox; +pub mod kvec;pub mod layout;pub mod vec_ext;
@@ -19,6 +20,11 @@pub use self::kbox::KVBox ;pub use self::kbox::VBox ; +pub use self::kvec::KVVec ; +pub use self::kvec::KVec ; +pub use self::kvec::VVec ; +pub use self::kvec::Vec ; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;
rust/kernel/alloc/kvec.rs new file +637 -0
637 lines not on this page · raw message
rust/kernel/prelude.rs +1 -1
@@ -14,7 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{flags::*, vec_ext::VecExt ,Box ,KBox ,KVBox ,VBox }; +pub use crate ::alloc::{flags::*, vec_ext::VecExt ,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec };#[doc(no_inline)] pub use alloc::vec::Vec ;
Signature · 1 line
2.46.1
Re: [PATCH v8 16/29] rust: alloc: implement kernel `Vec` type
Quoted 9 lines · On Tue, 1 Oct 2024 16:59:51 +0200 Danilo Krummrich <[email protected]> wrote:
Vecprovides a contiguous growable array type with contents allocated with the kernel's allocators (e.g.Kmalloc,VmallocorKVmalloc).In contrast to Rust's stdlib
Vectype, the kernelVectype considers the kernel's GFP flags for all appropriate functions, always reports allocation failures throughResult<_, AllocError>and remains independent from unstable features.Signed-off-by Danilo Krummrich <[email protected]>
Thanks a lot for your work. I think this is much cleaner with the ZST handling offloaded to Allocator and overload checking offloaded to ArrayLayout.
Some nits below, although I think we should apply this and then address the nits with follow up patches (some may also be good first issues).
Therefore:
Best,
Gary
Quoted 51 lines
--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kvec.rs | 637 ++++++++++++++++++++++++++++++++++++++ rust/kernel/prelude.rs | 2 +- 3 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/alloc/kvec.rsrust/kernel/alloc.rs +6
-0@@ -5,6 +5,7 @@#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod kbox; +pub mod kvec;pub mod layout;pub mod vec_ext;@@ -19,6 +20,11 @@pub use self::kbox::KVBox ;pub use self::kbox::VBox ; +pub use self::kvec::KVVec ; +pub use self::kvec::KVec ; +pub use self::kvec::VVec ; +pub use self::kvec::Vec ; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;rust/kernel/alloc/kvec.rs new file +14
-0@@ -0,0 +1,637 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Implementation of [`Vec`]. + +use super ::{ + allocator::{KVmalloc ,Kmalloc ,Vmalloc }, + layout::ArrayLayout , +AllocError ,Allocator ,Box ,Flags , +}; +use core::{ + fmt, + marker::PhantomData , + mem::{ManuallyDrop ,MaybeUninit }, + ops::Deref ,
rust/kernel/alloc/kvec.rs line 24
Quoted 10 lines
rust/kernel/alloc/kvec.rs new file +10
-0+ ops::DerefMut, + ops::Index, + ops::IndexMut, + ptr, + ptr::NonNull, + slice, + slice::SliceIndex, +}; + +/// Create a [`KVec`] containing the arguments.
This should mention that it allocates using GFP_KERNEL.
Quoted 208 lines
rust/kernel/alloc/kvec.rs new file +208
-0+/// +/// # Examples +/// +/// ``` +/// let mut v = kernel::kvec![]; +/// v.push(1, GFP_KERNEL)?; +/// assert_eq!(v, [1]); +/// +/// let mut v = kernel::kvec![1; 3]?; +/// v.push(4, GFP_KERNEL)?; +/// assert_eq!(v, [1, 1, 1, 4]); +/// +/// let mut v = kernel::kvec![1, 2, 3]?; +/// v.push(4, GFP_KERNEL)?; +/// assert_eq!(v, [1, 2, 3, 4]); +/// +/// # Ok::<(), Error>(()) +/// ``` +#[macro_export] +macro_rules! kvec { + () => ( + $crate::alloc::KVec ::new () + ); + ($elem:expr; $n:expr) => ( + $crate::alloc::KVec ::from_elem ($elem, $n,GFP_KERNEL ) + ); + ($($x:expr),+ $(,)?) => ( +match $crate::alloc::KBox ::new_uninit (GFP_KERNEL ) { +Ok (b) =>Ok ($crate::alloc::KVec ::from ($crate::alloc::KBox ::write (b, [$($x),+]))), +Err (e) =>Err (e), + } + ); +} + +/// The kernel's [`Vec`] type. +/// +/// A contiguous growable array type with contents allocated with the kernel's allocators (e.g. +/// [`Kmalloc`], [`Vmalloc`] or [`KVmalloc`]), written `Vec<T, A>`. +/// +/// For non-zero-sized values, a [`Vec`] will use the given allocator `A` for its allocation. For +/// the most common allocators the type aliases [`KVec`], [`VVec`] and [`KVVec`] exist. +/// +/// For zero-sized types the [`Vec`]'s pointer must be `dangling_mut::<T>`; no memory is allocated. +/// +/// Generally, [`Vec`] consists of a pointer that represents the vector's backing buffer, the +/// capacity of the vector (the number of elements that currently fit into the vector), it's length +/// (the number of elements that are currently stored in the vector) and the `Allocator` type used +/// to allocate (and free) the backing buffer. +/// +/// A [`Vec`] can be deconstructed into and (re-)constructed from it's previously named raw parts +/// and manually modified. +/// +/// [`Vec`]'s backing buffer gets, if required, automatically increased (re-allocated) when elements +/// are added to the vector. +/// +/// # Invariants +/// +/// - `self.ptr` is always properly aligned and either points to memory allocated with `A` or, for +/// zero-sized types, is a dangling, well aligned pointer. +/// +/// - `self.len` always represents the exact number of elements stored in the vector. +/// +/// - `self.layout` represents the absolute number of elements that can be stored within the vector +/// without re-allocation. However, it is legal for the backing buffer to be larger than `layout`. +/// +/// - The `Allocator` type `A` of the vector is the exact same `Allocator` type the backing buffer +/// was allocated with (and must be freed with). +pub struct Vec <T ,A :Allocator > { + ptr:NonNull <T >, +/// Represents the actual buffer size as `cap` times `size_of::<T>` bytes. +/// +/// Note: This isn't quite the same as `Self::capacity`, which in contrast returns the number of +/// elements we can still store without reallocating. + layout:ArrayLayout <T >, + len:usize , + _p:PhantomData <A >, +} + +/// Type alias for [`Vec`] with a [`Kmalloc`] allocator. +/// +/// # Examples +/// +/// ``` +/// let mut v = KVec::new(); +/// v.push(1, GFP_KERNEL)?; +/// assert_eq!(&v, &[1]); +/// +/// # Ok::<(), Error>(()) +/// ``` +pub type KVec <T > =Vec <T ,Kmalloc >; + +/// Type alias for [`Vec`] with a [`Vmalloc`] allocator. +/// +/// # Examples +/// +/// ``` +/// let mut v = VVec::new(); +/// v.push(1, GFP_KERNEL)?; +/// assert_eq!(&v, &[1]); +/// +/// # Ok::<(), Error>(()) +/// ``` +pub type VVec <T > =Vec <T ,Vmalloc >; + +/// Type alias for [`Vec`] with a [`KVmalloc`] allocator. +/// +/// # Examples +/// +/// ``` +/// let mut v = KVVec::new(); +/// v.push(1, GFP_KERNEL)?; +/// assert_eq!(&v, &[1]); +/// +/// # Ok::<(), Error>(()) +/// ``` +pub type KVVec <T > =Vec <T ,KVmalloc >; + +// SAFETY: `Vec` is `Send` if `T` is `Send` because `Vec` owns its elements. +unsafe impl <T ,A >Send for Vec <T ,A > +where +T :Send , +A :Allocator , +{ +} + +// SAFETY: `Vec` is `Sync` if `T` is `Sync` because `Vec` owns its elements. +unsafe impl <T ,A >Sync for Vec <T ,A > +where +T :Sync , +A :Allocator , +{ +} + +impl <T ,A >Vec <T ,A > +where +A :Allocator , +{ +#[inline] +const fn is_zst () ->bool { + core::mem::size_of ::<T >() ==0 + } + +/// Returns the number of elements that can be stored within the vector without allocating +/// additional memory. +pub fn capacity (&self) ->usize { +if const {Self ::is_zst () } { + usize::MAX + }else { + self.layout.len () + } + } + +/// Returns the number of elements stored within the vector. +#[inline] +pub fn len (&self) ->usize { + self.len + } + +/// Forcefully sets `self.len` to `new_len`. +/// +/// # Safety +/// +/// - `new_len` must be less than or equal to [`Self::capacity`]. +/// - If `new_len` is greater than `self.len`, all elements within the interval +/// [`self.len`,`new_len`) must be initialized. +#[inline] +pub unsafe fn set_len (&mut self, new_len:usize ) { +debug_assert! (new_len <= self.capacity ()); + self.len = new_len; + } + +/// Returns a slice of the entire vector. +#[inline] +pub fn as_slice (&self) -> &[T ] { + self + } + +/// Returns a mutable slice of the entire vector. +#[inline] +pub fn as_mut_slice (&mut self) -> &mut [T ] { + self + } + +/// Returns a mutable raw pointer to the vector's backing buffer, or, if `T` is a ZST, a +/// dangling raw pointer. +#[inline] +pub fn as_mut_ptr (&mut self) -> *mut T { + self.ptr.as_ptr () + } + +/// Returns a raw pointer to the vector's backing buffer, or, if `T` is a ZST, a dangling raw +/// pointer. +#[inline] +pub fn as_ptr (&self) -> *const T { + self.ptr.as_ptr () + } + +/// Returns `true` if the vector contains no elements, `false` otherwise. +/// +/// # Examples +/// +/// ``` +/// let mut v = KVec::new(); +/// assert!(v.is_empty()); +/// +/// v.push(1, GFP_KERNEL); +/// assert!(!v.is_empty()); +/// ```
rust/kernel/alloc/kvec.rs line 242
Quoted 10 lines
rust/kernel/alloc/kvec.rs new file +10
-0+#[inline] +pub fn is_empty (&self) ->bool { + self.len () ==0 + } + +/// Creates a new, empty Vec<T, A>. +/// +/// This method does not allocate by itself. +#[inline] +pub const fn new () ->Self {
Missing // INVARIANT here.
Quoted 112 lines
rust/kernel/alloc/kvec.rs new file +112
-0+Self { + ptr:NonNull ::dangling (), + layout:ArrayLayout ::empty (), + len:0 , + _p:PhantomData ::<A >, + } + } + +/// Returns a slice of `MaybeUninit<T>` for the remaining spare capacity of the vector. +pub fn spare_capacity_mut (&mut self) -> &mut [MaybeUninit <T >] { +// SAFETY: +// - `self.len` is smaller than `self.capacity` and hence, the resulting pointer is +// guaranteed to be part of the same allocated object. +// - `self.len` can not overflow `isize`. +let ptr =unsafe { self.as_mut_ptr ().add (self.len) }as *mut MaybeUninit <T >; + +// SAFETY: The memory between `self.len` and `self.capacity` is guaranteed to be allocated +// and valid, but uninitialized. +unsafe { slice::from_raw_parts_mut (ptr, self.capacity () - self.len) } + } + +/// Appends an element to the back of the [`Vec`] instance. +/// +/// # Examples +/// +/// ``` +/// let mut v = KVec::new(); +/// v.push(1, GFP_KERNEL)?; +/// assert_eq!(&v, &[1]); +/// +/// v.push(2, GFP_KERNEL)?; +/// assert_eq!(&v, &[1, 2]); +/// # Ok::<(), Error>(()) +/// ``` +pub fn push (&mut self, v:T , flags:Flags ) ->Result <(),AllocError > { + self.reserve (1 , flags)?; + +// SAFETY: +// - `self.len` is smaller than `self.capacity` and hence, the resulting pointer is +// guaranteed to be part of the same allocated object. +// - `self.len` can not overflow `isize`. +let ptr =unsafe { self.as_mut_ptr ().add (self.len) }; + +// SAFETY: +// - `ptr` is properly aligned and valid for writes. +unsafe { core::ptr::write (ptr, v) }; + +// SAFETY: We just initialised the first spare entry, so it is safe to increase the length +// by 1. We also know that the new length is <= capacity because of the previous call to +// `reserve` above. +unsafe { self.set_len (self.len () +1 ) }; +Ok (()) + } + +/// Creates a new [`Vec`] instance with at least the given capacity. +/// +/// # Examples +/// +/// ``` +/// let v = KVec::<u32>::with_capacity(20, GFP_KERNEL)?; +/// +/// assert!(v.capacity() >= 20); +/// # Ok::<(), Error>(()) +/// ``` +pub fn with_capacity (capacity:usize , flags:Flags ) ->Result <Self ,AllocError > { +let mut v =Vec ::new (); + + v.reserve (capacity, flags)?; + +Ok (v) + } + +/// Creates a Vec<T, A> from a pointer, a length and a capacity using the allocator `A`. +/// +/// # Examples +/// +/// ``` +/// let mut v = kernel::kvec![1, 2, 3]?; +/// v.reserve(1, GFP_KERNEL)?; +/// +/// let (mut ptr, mut len, cap) = v.into_raw_parts(); +/// +/// // SAFETY: We've just reserved memory for another element. +/// unsafe { ptr.add(len).write(4) }; +/// len += 1; +/// +/// // SAFETY: We only wrote an additional element at the end of the `KVec`'s buffer and +/// // correspondingly increased the length of the `KVec` by one. Otherwise, we construct it +/// // from the exact same raw parts. +/// let v = unsafe { KVec::from_raw_parts(ptr, len, cap) }; +/// +/// assert_eq!(v, [1, 2, 3, 4]); +/// +/// # Ok::<(), Error>(()) +/// ``` +/// +/// # Safety +/// +/// If `T` is a ZST: +/// +/// - `ptr` must be a dangling, well aligned pointer. +/// +/// Otherwise: +/// +/// - `ptr` must have been allocated with the allocator `A`. +/// - `ptr` must satisfy or exceed the alignment requirements of `T`. +/// - `ptr` must point to memory with a size of at least `size_of::<T>() * capacity`. +/// bytes. +/// - The allocated size in bytes must not be larger than `isize::MAX`. +/// - `length` must be less than or equal to `capacity`. +/// - The first `length` elements must be initialized values of type `T`. +///
rust/kernel/alloc/kvec.rs line 364
Quoted 10 lines
rust/kernel/alloc/kvec.rs new file +10
-0+/// It is also valid to create an empty `Vec` passing a dangling pointer for `ptr` and zero for +/// `cap` and `len`. +pub unsafe fn from_raw_parts (ptr: *mut T , length:usize , capacity:usize ) ->Self { +let layout =if Self ::is_zst () { +ArrayLayout ::empty () + }else { +// SAFETY: By the safety requirements of this function, `capacity * size_of::<T>()` is +// smaller than `isize::MAX`. +unsafe {ArrayLayout ::new_unchecked (capacity) } + };
Missing // INVARIANT here.
Quoted 63 lines
rust/kernel/alloc/kvec.rs new file +63
-0+ +Self { +// SAFETY: By the safety requirements, `ptr` is either dangling or pointing to a valid +// memory allocation, allocated with `A`. + ptr:unsafe {NonNull ::new_unchecked (ptr) }, + layout, + len: length, + _p:PhantomData ::<A >, + } + } + +/// Consumes the `Vec<T, A>` and returns its raw components `pointer`, `length` and `capacity`. +/// +/// This will not run the destructor of the contained elements and for non-ZSTs the allocation +/// will stay alive indefinitely. Use [`Vec::from_raw_parts`] to recover the [`Vec`], drop the +/// elements and free the allocation, if any. +pub fn into_raw_parts (self) -> (*mut T ,usize ,usize ) { +let mut me =ManuallyDrop ::new (self); +let len = me.len (); +let capacity = me.capacity (); +let ptr = me.as_mut_ptr (); + (ptr, len, capacity) + } + +/// Ensures that the capacity exceeds the length by at least `additional` +/// elements. +/// +/// # Examples +/// +/// ``` +/// let mut v = KVec::new(); +/// v.push(1, GFP_KERNEL)?; +/// +/// v.reserve(10, GFP_KERNEL)?; +/// let cap = v.capacity(); +/// assert!(cap >= 10); +/// +/// v.reserve(10, GFP_KERNEL)?; +/// let new_cap = v.capacity(); +/// assert_eq!(new_cap, cap); +/// +/// # Ok::<(), Error>(()) +/// ``` +pub fn reserve (&mut self, additional:usize , flags:Flags ) ->Result <(),AllocError > { +let len = self.len (); +let cap = self.capacity (); + +if cap - len >= additional { +return Ok (()); + } + +if Self ::is_zst () { +// The capacity is already `usize::MAX` for ZSTs, we can't go higher. +return Err (AllocError ); + } + +// We know that `cap <= isize::MAX` because of the type invariants of `Self`. So the +// multiplication by two won't overflow. +let new_cap = core::cmp::max (cap *2 , len.checked_add (additional).ok_or (AllocError )?); +let layout =ArrayLayout ::new (new_cap).map_err (|_|AllocError )?; + +// SAFETY: +// - `ptr` is valid because it's either `None` or comes from a previous call to
rust/kernel/alloc/kvec.rs line 437
Quoted 10 lines
rust/kernel/alloc/kvec.rs new file +10
-0+// `A::realloc`. +// - `self.layout` matches the `ArrayLayout` of the preceeding allocation. +let ptr =unsafe { +A ::realloc ( +Some (self.ptr.cast ()), + layout.into (), + self.layout.into (), + flags, + )? + };
Missing // INVARIANT here.
Quoted 7 lines
rust/kernel/alloc/kvec.rs new file +7
-0+ + self.ptr = ptr.cast (); + self.layout = layout; + +Ok (()) + } +}
Re: [PATCH v8 16/29] rust: alloc: implement kernel `Vec` type
In reply to Danilo KrummrichQuoted 8 lines · Danilo Krummrich
Vecprovides a contiguous growable array type with contents allocated with the kernel's allocators (e.g.Kmalloc,VmallocorKVmalloc).In contrast to Rust's stdlib
Vectype, the kernelVectype considers the kernel's GFP flags for all appropriate functions, always reports allocation failures throughResult<_, AllocError>and remains independent from unstable features.
Quoted 4 lines
Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kvec.rs | 637 ++++++++++++++++++++++++++++++++++++++
Any reason against naming this vec.rs?
Quoted 3 lines
rust/kernel/prelude.rs | 2 +- 3 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/alloc/kvec.rs
I share Gary's opinion, let's do future improvements via good-first-issues/patches in the current cycle. But get this version merged now.
---
Cheers,
Benno
Danilo Krummrich
maintainerRe: [PATCH v8 16/29] rust: alloc: implement kernel `Vec` type
Quoted 9 lines · Benno Lossin
Quoted 8 lines · Danilo Krummrich
Vecprovides a contiguous growable array type with contents allocated with the kernel's allocators (e.g.Kmalloc,VmallocorKVmalloc).In contrast to Rust's stdlib
Vectype, the kernelVectype considers the kernel's GFP flags for all appropriate functions, always reports allocation failures throughResult<_, AllocError>and remains independent from unstable features.
Quoted 6 lines
Quoted 4 lines
Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kvec.rs | 637 ++++++++++++++++++++++++++++++++++++++Any reason against naming this
vec.rs?
I think I didn't rename kvec.rs and kbox.rs because we'd need to escape "box" in various places, e.g. in rust/kernel/alloc.rs:
``` pub mod r#box; pub use self::r#box::Box; pub use self::r#box::KBox; pub use self::r#box::VBox; pub use self::r#box::KVBox; ```
And it's a kernel specific implementation, so kbox.rs and kvec.rs still makes sense. :)
Quoted 15 lines
Quoted 4 lines
rust/kernel/prelude.rs | 2 +- 3 files changed, 644 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/alloc/kvec.rsI share Gary's opinion, let's do future improvements via good-first-issues/patches in the current cycle. But get this version merged now.
Reviewed-by Benno Lossin <[email protected]>---
Cheers,
Benno
Re: [PATCH v8 16/29] rust: alloc: implement kernel `Vec` type
Quoted 20 lines · Danilo Krummrich
Quoted 9 lines · Benno Lossin
Quoted 8 lines · Danilo Krummrich
Vecprovides a contiguous growable array type with contents allocated with the kernel's allocators (e.g.Kmalloc,VmallocorKVmalloc).In contrast to Rust's stdlib
Vectype, the kernelVectype considers the kernel's GFP flags for all appropriate functions, always reports allocation failures throughResult<_, AllocError>and remains independent from unstable features.Quoted 6 lines
Quoted 4 lines
Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kvec.rs | 637 ++++++++++++++++++++++++++++++++++++++Any reason against naming this
vec.rs?I think I didn't rename
kvec.rsandkbox.rsbecause we'd need to escape "box" in various places, e.g. inrust/kernel/alloc.rs:
Quoted 8 lines
``` pub mod r#box; pub use self::r#box::Box; pub use self::r#box::KBox; pub use self::r#box::VBox; pub use self::r#box::KVBox; ```
Yeah, that's why std uses boxed.rs.
Quoted 2 lines
And it's a kernel specific implementation, so
kbox.rsandkvec.rsstill makes sense. :)
Fair.
---
Cheers,
Benno
kernel test robot
build botRe: [PATCH v8 16/29] rust: alloc: implement kernel `Vec` type
In reply to Danilo Krummrichkernel test robot noticed the following build errors:
Hi Danilo,
kernel test robot noticed the following build errors:
[auto build test ERROR on 9852d85ec9d492ebef56dc5f229416c925758edc]
url: https://github.com/intel-lab-lkp/linux/commits/Danilo-Krummrich/rust-alloc-add-Allocator-trait/20241001-230558 base: 9852d85ec9d492ebef56dc5f229416c925758edc patch link: https://lore.kernel.org/r/20241001150008.183102-17-dakr%40kernel.org patch subject: [PATCH v8 16/29] rust: alloc: implement kernel `Vec` type config: riscv-randconfig-001-20241002 (https://download.01.org/0day-ci/archive/20241003/[email protected]/config) compiler: clang version 14.0.6 (https://github.com/llvm/llvm-project f28c006a5895fc0e329fe15fead81e37457cb1d1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20241003/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> error[E0658]: inline-const is experimental
--> rust/kernel/alloc/kvec.rs:170:12
|
170 | if const { Self::is_zst() } {
| ^^^^^
|
= note: see issue #76001 <https://github.com/rust-lang/rust/issues/76001> for more information = help: add #![feature(inline_const)] to the crate attributes to enable
= note: this compiler was built on 2024-04-29; consider upgrading it if it is out of date
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
Danilo Krummrich
maintainerPATCH v8 17/29 rust: alloc: implement `IntoIterator` for `Vec`
In reply to Danilo KrummrichImplement IntoIterator for Vec, Vec's IntoIter type, as well as Iterator for IntoIter.
Vec::into_iter disassembles the Vec into its raw parts; additionally, IntoIter keeps track of a separate pointer, which is incremented correspondingsly as the iterator advances, while the length, or the count of elements, is decremented.
This also means that IntoIter takes the ownership of the backing buffer and is responsible to drop the remaining elements and free the backing buffer, if it's dropped.
--- rust/kernel/alloc.rs | 1 + rust/kernel/alloc/kvec.rs | 170 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 171 insertions(+)
rust/kernel/alloc.rs +1 -0
@@ -20,6 +20,7 @@pub use self::kbox::KVBox ;pub use self::kbox::VBox ; +pub use self::kvec::IntoIter ;pub use self::kvec::KVVec ;pub use self::kvec::KVec ;pub use self::kvec::VVec ;
rust/kernel/alloc/kvec.rs +170 -0
173 lines not on this page · raw message
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 18/29 rust: alloc: implement `collect` for `IntoIter`
In reply to Danilo KrummrichCurrently, we can't implement FromIterator. There are a couple of issues with this trait in the kernel, namely:
- Rust's specialization feature is unstable. This prevents us to optimze for the special case where I::IntoIter equals Vec's IntoIter type.
- We also can't use I::IntoIter's type ID either to work around this, since FromIterator doesn't require this type to be 'static.
- FromIterator::from_iter does return Self instead of Result<Self, AllocError>, hence we can't properly handle allocation failures.
- Neither Iterator::collect nor FromIterator::from_iter can handle additional allocation flags.
Instead, provide IntoIter::collect, such that we can at least convert IntoIter into a Vec again.
--- rust/kernel/alloc/kvec.rs | 94 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+)
rust/kernel/alloc/kvec.rs +94 -0
@@ -681,6 +681,100 @@ pub struct IntoIter<T, A: Allocator> { _p: PhantomData<A>, } +impl <T ,A >IntoIter <T ,A > +where +A :Allocator , +{ +fn into_raw_parts (self) -> (*mut T ,NonNull <T >,usize ,usize ) { +let me =ManuallyDrop ::new (self); +let ptr = me.ptr; +let buf = me.buf; +let len = me.len; +let cap = me.layout.len (); + (ptr, buf, len, cap) + } + +/// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`. +/// +/// # Examples +/// +/// ``` +/// let v = kernel::kvec![1, 2, 3]?; +/// let mut it = v.into_iter(); +/// +/// assert_eq!(it.next(), Some(1)); +/// +/// let v = it.collect(GFP_KERNEL); +/// assert_eq!(v, [2, 3]); +/// +/// # Ok::<(), Error>(()) +/// ``` +/// # Implementation Details +/// +/// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait +/// in the kernel, namely: +/// +/// - Rust's specialization feature is unstable. This prevents us to optimze for the special +/// case where `I::IntoIter` equals `Vec`'s `IntoIter` type. +/// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` +/// doesn't require this type to be `'static`. +/// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence +/// we can't properly handle allocation failures. +/// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation +/// flags. +/// +/// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a +/// `Vec` again. +/// +/// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing +/// buffer. However, this backing buffer may be shrunk to the actual count of elements. +pub fn collect (self, flags:Flags ) ->Vec <T ,A > { +let old_layout = self.layout; +let (mut ptr, buf, len,mut cap) = self.into_raw_parts (); +let has_advanced = ptr != buf.as_ptr (); + +if has_advanced { +// Copy the contents we have advanced to at the beginning of the buffer. +// +// SAFETY: +// - `ptr` is valid for reads of `len * size_of::<T>()` bytes, +// - `buf.as_ptr()` is valid for writes of `len * size_of::<T>()` bytes, +// - `ptr` and `buf.as_ptr()` are not be subject to aliasing restrictions relative to +// each other, +// - both `ptr` and `buf.ptr()` are properly aligned. +unsafe { ptr::copy (ptr, buf.as_ptr (), len) }; + ptr = buf.as_ptr (); + +// SAFETY: `len` is guaranteed to be smaller than `self.layout.len()`. +let layout =unsafe {ArrayLayout ::<T >::new_unchecked (len) }; + +// SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be +// smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves +// it as it is. + ptr =match unsafe { +A ::realloc (Some (buf.cast ()), layout.into (), old_layout.into (), flags) + } { +// If we fail to shrink, which likely can't even happen, continue with the existing +// buffer. +Err (_) => ptr, +Ok (ptr) => { + cap = len; + ptr.as_ptr ().cast () + } + }; + } + +// SAFETY: If the iterator has been advanced, the advanced elements have been copied to +// the beginning of the buffer and `len` has been adjusted accordingly. +// +// - `ptr` is guaranteed to point to the start of the backing buffer. +// - `cap` is either the original capacity or, after shrinking the buffer, equal to `len`. +// - `alloc` is guaranteed to be unchanged since `into_iter` has been called on the original +// `Vec`. +unsafe {Vec ::from_raw_parts (ptr, len, cap) } + } +} +impl <T ,A >Iterator for IntoIter <T ,A >where A :Allocator ,
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 19/29 rust: treewide: switch to the kernel `Vec` type
In reply to Danilo KrummrichNow that we got the kernel Vec in place, convert all existing Vec users to make use of it.
--- rust/kernel/str.rs | 12 +++++------- rust/kernel/sync/locked_by.rs | 2 +- rust/kernel/types.rs | 2 +- rust/kernel/uaccess.rs | 17 +++++++---------- rust/macros/lib.rs | 6 +++--- samples/rust/rust_minimal.rs | 4 ++-- 6 files changed, 19 insertions(+), 24 deletions(-)
rust/kernel/str.rs +5 -7
@@ -2,8 +2,7 @@//! String representations. -use crate ::alloc::{flags::*, vec_ext::VecExt ,AllocError }; -use alloc::vec::Vec ; +use crate ::alloc::{flags::*,AllocError ,KVec };use core::fmt::{self,Write };use core::ops::{self,Deref ,DerefMut ,Index };
@@ -790,7 +789,7 @@ fn write_str(&mut self, s: &str) -> fmt::Result {/// assert_eq!(s.is_ok(), false); /// ``` pub struct CString { - buf:Vec <u8 >, + buf:KVec <u8 >, }impl CString {
@@ -803,7 +802,7 @@ pub fn try_from_fmt(args: fmt::Arguments<'_>) -> Result<Self, Error> {let size = f.bytes_written ();// Allocate a vector with the required number of bytes, and write to it. -let mut buf = <Vec <_ >as VecExt <_ >>::with_capacity (size,GFP_KERNEL )?; +let mut buf =KVec ::with_capacity (size,GFP_KERNEL )?;// SAFETY: The buffer stored in `buf` is at least of size `size` and is valid for writes. let mut f =unsafe {Formatter ::from_buffer (buf.as_mut_ptr (), size) }; f.write_fmt (args)?;
@@ -850,10 +849,9 @@ impl<'a> TryFrom<&'a CStr> for CString {type Error =AllocError ;fn try_from (cstr: &'aCStr ) ->Result <CString ,AllocError > { -let mut buf =Vec ::new (); +let mut buf =KVec ::new (); - <Vec <_ >as VecExt <_ >>::extend_from_slice (&mut buf, cstr.as_bytes_with_nul (),GFP_KERNEL ) - .map_err (|_|AllocError )?; + buf.extend_from_slice (cstr.as_bytes_with_nul (),GFP_KERNEL )?;// INVARIANT: The `CStr` and `CString` types have the same invariants for // the string data, and we copied it over without changes.
rust/kernel/sync/locked_by.rs +1 -1
@@ -43,7 +43,7 @@/// struct InnerDirectory { /// /// The sum of the bytes used by all files. /// bytes_used: u64, -/// _files: Vec<File>, +/// _files: KVec<File>, /// } /// /// struct Directory {
rust/kernel/types.rs +1 -1
@@ -135,7 +135,7 @@ unsafe fn from_foreign(_: *const core::ffi::c_void) -> Self {}/// # use kernel::types::ScopeGuard; /// fn example3(arg: bool) -> Result { /// let mut vec = -/// ScopeGuard::new_with_data(Vec::new(), |v| pr_info!("vec had {} elements\n", v.len())); +/// ScopeGuard::new_with_data(KVec::new(), |v| pr_info!("vec had {} elements\n", v.len())); /// /// vec.push(10u8, GFP_KERNEL)?; /// if arg {
rust/kernel/uaccess.rs +7 -10
@@ -11,7 +11,6 @@ prelude::*, types::{AsBytes, FromBytes}, }; -use alloc::vec::Vec ;use core::ffi::{c_ulong, c_void};use core::mem::{size_of,MaybeUninit };
@@ -46,7 +45,6 @@/// every byte in the region. /// /// ```no_run -/// use alloc::vec::Vec; /// use core::ffi::c_void; /// use kernel::error::Result; /// use kernel::uaccess::{UserPtr, UserSlice};
@@ -54,7 +52,7 @@/// fn bytes_add_one(uptr: UserPtr, len: usize) -> Result<()> { /// let (read, mut write) = UserSlice::new(uptr, len).reader_writer(); /// -/// let mut buf = Vec::new(); +/// let mut buf = KVec::new(); /// read.read_all(&mut buf, GFP_KERNEL)?; /// /// for b in &mut buf {
@@ -69,7 +67,6 @@/// Example illustrating a TOCTOU (time-of-check to time-of-use) bug. /// /// ```no_run -/// use alloc::vec::Vec; /// use core::ffi::c_void; /// use kernel::error::{code::EINVAL, Result}; /// use kernel::uaccess::{UserPtr, UserSlice};
@@ -78,21 +75,21 @@/// fn is_valid(uptr: UserPtr, len: usize) -> Result<bool> { /// let read = UserSlice::new(uptr, len).reader(); /// -/// let mut buf = Vec::new(); +/// let mut buf = KVec::new(); /// read.read_all(&mut buf, GFP_KERNEL)?; /// /// todo!() /// } /// /// /// Returns the bytes behind this user pointer if they are valid. -/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<Vec<u8>> { +/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<KVec<u8>> { /// if !is_valid(uptr, len)? { /// return Err(EINVAL); /// } /// /// let read = UserSlice::new(uptr, len).reader(); /// -/// let mut buf = Vec::new(); +/// let mut buf = KVec::new(); /// read.read_all(&mut buf, GFP_KERNEL)?; /// /// // THIS IS A BUG! The bytes could have changed since we checked them.
@@ -130,7 +127,7 @@ pub fn new(ptr: UserPtr, length: usize) -> Self {/// Reads the entirety of the user slice, appending it to the end of the provided buffer. /// /// Fails with [`EFAULT`] if the read happens on a bad address. -pub fn read_all (self, buf: &mut Vec <u8 >, flags:Flags ) ->Result { +pub fn read_all (self, buf: &mut KVec <u8 >, flags:Flags ) ->Result { self.reader ().read_all (buf, flags) }
@@ -291,9 +288,9 @@ pub fn read<T: FromBytes>(&mut self) -> Result<T> {/// Reads the entirety of the user slice, appending it to the end of the provided buffer. /// /// Fails with [`EFAULT`] if the read happens on a bad address. -pub fn read_all (mut self, buf: &mut Vec <u8 >, flags:Flags ) ->Result { +pub fn read_all (mut self, buf: &mut KVec <u8 >, flags:Flags ) ->Result {let len = self.length; -VecExt ::<u8 >::reserve (buf, len, flags)?; + buf.reserve (len, flags)?;// The call to `try_reserve` was successful, so the spare capacity is at least `len` bytes // long.
rust/macros/lib.rs +3 -3
@@ -242,7 +242,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// #[pin_data] /// struct DriverData { /// #[pin] -/// queue: Mutex<Vec<Command>>, +/// queue: Mutex<KVec<Command>>, /// buf: KBox<[u8; 1024 * 1024]>, /// } /// ```
@@ -251,7 +251,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// #[pin_data(PinnedDrop)] /// struct DriverData { /// #[pin] -/// queue: Mutex<Vec<Command>>, +/// queue: Mutex<KVec<Command>>, /// buf: KBox<[u8; 1024 * 1024]>, /// raw_info: *mut Info, /// }
@@ -281,7 +281,7 @@ pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream {/// #[pin_data(PinnedDrop)] /// struct DriverData { /// #[pin] -/// queue: Mutex<Vec<Command>>, +/// queue: Mutex<KVec<Command>>, /// buf: KBox<[u8; 1024 * 1024]>, /// raw_info: *mut Info, /// }
samples/rust/rust_minimal.rs +2 -2
@@ -13,7 +13,7 @@ }struct RustMinimal { - numbers:Vec <i32 >, + numbers:KVec <i32 >, }impl kernel::Module for RustMinimal {
@@ -21,7 +21,7 @@ fn init(_module: &'static ThisModule) -> Result<Self> {pr_info! ("Rust minimal sample (init)\n" );pr_info! ("Am I built-in? {}\n" , !cfg! (MODULE )); -let mut numbers =Vec ::new (); +let mut numbers =KVec ::new (); numbers.push (72 ,GFP_KERNEL )?; numbers.push (108 ,GFP_KERNEL )?; numbers.push (200 ,GFP_KERNEL )?;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 20/29 rust: alloc: remove `VecExt` extension
In reply to Danilo KrummrichNow that all existing Vec users were moved to the kernel Vec type, remove the VecExt extension.
--- rust/kernel/alloc.rs | 1 - rust/kernel/alloc/vec_ext.rs | 185 ----------------------------------- rust/kernel/prelude.rs | 5 +- 3 files changed, 1 insertion(+), 190 deletions(-) delete mode 100644 rust/kernel/alloc/vec_ext.rs
rust/kernel/alloc.rs +0 -1
@@ -7,7 +7,6 @@pub mod kbox;pub mod kvec;pub mod layout; -pub mod vec_ext;#[cfg(any(test, testlib))] pub mod allocator_test;
rust/kernel/alloc/vec_ext.rs deleted +0 -185
185 lines not on this page · raw message
rust/kernel/prelude.rs +1 -4
@@ -14,10 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{flags::*, vec_ext::VecExt ,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec }; - -#[doc(no_inline)] -pub use alloc::vec::Vec ; +pub use crate ::alloc::{flags::*,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec };#[doc(no_inline)] pub use macros::{module, pin_data, pinned_drop, vtable,Zeroable };
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 21/29 rust: alloc: add `Vec` to prelude
In reply to Danilo KrummrichNow that we removed VecExt and the corresponding includes in prelude.rs, add the new kernel Vec type instead.
--- rust/kernel/prelude.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/prelude.rs +1 -1
@@ -14,7 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{flags::*,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec }; +pub use crate ::alloc::{flags::*,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec ,Vec };#[doc(no_inline)] pub use macros::{module, pin_data, pinned_drop, vtable,Zeroable };
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 22/29 rust: error: use `core::alloc::LayoutError`
In reply to Danilo KrummrichUse core::alloc::LayoutError instead of alloc::alloc::LayoutError in preparation to get rid of Rust's alloc crate.
--- rust/kernel/error.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/error.rs +1 -1
@@ -6,7 +6,7 @@use crate ::{alloc::AllocError , str::CStr }; -use alloc::alloc::LayoutError ; +use core::alloc::LayoutError ;use core::fmt;use core::num::TryFromIntError ;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 23/29 rust: error: check for config `test` in `Error::name`
In reply to Danilo KrummrichAdditional to testlib also check for test in Error::name. This is required by a subsequent patch that (indirectly) uses Error in test cases.
--- rust/kernel/error.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
rust/kernel/error.rs +2 -2
@@ -143,7 +143,7 @@ pub(crate) fn to_ptr<T>(self) -> *mut T { }/// Returns a string representing the error, if one exists. -#[cfg(not(testlib))] +#[cfg(not(any(test, testlib)))] pub fn name (&self) ->Option <&'staticCStr > {// SAFETY: Just an FFI call, there are no extra safety requirements. let ptr =unsafe { bindings::errname (-self.0 ) };
@@ -160,7 +160,7 @@ pub fn name(&self) -> Option<&'static CStr> {/// When `testlib` is configured, this always returns `None` to avoid the dependency on a /// kernel function so that tests that use this (e.g., by calling [`Result::unwrap`]) can still /// run in userspace. -#[cfg(testlib)] +#[cfg(any(test, testlib))] pub fn name (&self) ->Option <&'staticCStr > {None }
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 24/29 rust: alloc: implement `contains` for `Flags`
In reply to Danilo KrummrichProvide a simple helper function to check whether given flags do contain one or multiple other flags.
This is used by a subsequent patch implementing the Cmalloc Allocator to check for __GFP_ZERO.
--- rust/kernel/alloc.rs | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
rust/kernel/alloc.rs +6 -1
@@ -35,7 +35,7 @@/// They can be combined with the operators `|`, `&`, and `!`. /// /// Values can be used from the [`flags`] module. -#[derive( Clone , Copy )] +#[derive( Clone , Copy , PartialEq )] pub struct Flags (u32 );impl Flags {
@@ -43,6 +43,11 @@ impl Flags {pub (crate )fn as_raw (self) ->u32 { self.0 } + +/// Check whether `flags` is contained in `self`. +pub fn contains (self, flags:Flags ) ->bool { + (self & flags) == flags + } }impl core::ops::BitOr for Flags {
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 25/29 rust: alloc: implement `Cmalloc` in module allocator_test
In reply to Danilo KrummrichSo far the kernel's Box and Vec types can't be used by userspace test cases, since all users of those types (e.g. CString) use kernel allocators for instantiation.
In order to allow userspace test cases to make use of such types as well, implement the Cmalloc allocator within the allocator_test module and type alias all kernel allocators to Cmalloc. The Cmalloc allocator uses libc's realloc() function as allocator backend.
--- rust/kernel/alloc/allocator_test.rs | 88 ++++++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 7 deletions(-)
rust/kernel/alloc/allocator_test.rs +81 -7
@@ -1,21 +1,95 @@// SPDX-License-Identifier: GPL-2.0 +//! So far the kernel's `Box` and `Vec` types can't be used by userspace test cases, since all users +//! of those types (e.g. `CString`) use kernel allocators for instantiation. +//! +//! In order to allow userspace test cases to make use of such types as well, implement the +//! `Cmalloc` allocator within the allocator_test module and type alias all kernel allocators to +//! `Cmalloc`. The `Cmalloc` allocator uses libc's realloc() function as allocator backend. +#![allow(missing_docs)] -use super ::{AllocError ,Allocator ,Flags }; +use super ::{flags::*,AllocError ,Allocator ,Flags };use core::alloc::Layout ; +use core::cmp; +use core::ptr;use core::ptr::NonNull ; -pub struct Kmalloc ; +/// The userspace allocator based on libc. +pub struct Cmalloc ; + +pub type Kmalloc =Cmalloc ;pub type Vmalloc =Kmalloc ;pub type KVmalloc =Kmalloc ; -unsafe impl Allocator for Kmalloc { +extern "C" { +#[link_name = "aligned_alloc" ] +fn libc_aligned_alloc (align:usize , size:usize ) -> *mut core::ffi::c_void ; + +#[link_name = "free" ] +fn libc_free (ptr: *mut core::ffi::c_void ); +} + +// SAFETY: +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation created by this `Allocator` is always OK, +// - `realloc` provides the guarantees as provided in the `# Guarantees` section. +unsafe impl Allocator for Cmalloc {unsafe fn realloc ( - _ptr:Option <NonNull <u8 >>, - _layout:Layout , - _flags:Flags , + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , ) ->Result <NonNull <[u8 ]>,AllocError > { -panic! (); +let src =match ptr { +Some (src) => { +if old_layout.size () ==0 { + ptr::null_mut () + }else { + src.as_ptr () + } + } +None => ptr::null_mut (), + }; + +if layout.size () ==0 { +// SAFETY: `src` is either NULL or was previously allocated with this `Allocator` +unsafe {libc_free (src.cast ()) }; + +return Ok (NonNull ::slice_from_raw_parts ( +crate ::alloc::dangling_from_layout (layout), +0 , + )); + } + +// SAFETY: Returns either NULL or a pointer to a memory allocation that satisfies or +// exceeds the given size and alignment requirements. +let dst =unsafe {libc_aligned_alloc (layout.align (), layout.size ()) }as *mut u8 ; +let dst =NonNull ::new (dst).ok_or (AllocError )?; + +if flags.contains (__GFP_ZERO) { +// SAFETY: The preceeding calls to `libc_aligned_alloc` and `NonNull::new` +// guarantee that `dst` points to memory of at least `layout.size()` bytes. +unsafe { dst.as_ptr ().write_bytes (0 , layout.size ()) }; + } + +if !src.is_null () { +// SAFETY: +// - `src` has previously been allocated with this `Allocator`; `dst` has just been +// newly allocated, hence the memory regions do not overlap. +// - both` src` and `dst` are properly aligned and valid for reads and writes +unsafe { + ptr::copy_nonoverlapping ( + src, + dst.as_ptr (), + cmp::min (layout.size (), old_layout.size ()), + ) + }; + } + +// SAFETY: `src` is either NULL or was previously allocated with this `Allocator` +unsafe {libc_free (src.cast ()) }; + +Ok (NonNull ::slice_from_raw_parts (dst, layout.size ())) } }
Signature · 1 line
2.46.1
Re: [PATCH v8 25/29] rust: alloc: implement `Cmalloc` in module allocator_test
Quoted 9 lines · Danilo Krummrich
So far the kernel's
BoxandVectypes can't be used by userspace test cases, since all users of those types (e.g.CString) use kernel allocators for instantiation.In order to allow userspace test cases to make use of such types as well, implement the
Cmallocallocator within theallocator_testmodule and type alias all kernel allocators toCmalloc. TheCmallocallocator uses libc'srealloc()function as allocator backend.
Quoted 4 lines
Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc/allocator_test.rs | 88 ++++++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 7 deletions(-)
---
Cheers,
Benno
Re: [PATCH v8 25/29] rust: alloc: implement `Cmalloc` in module allocator_test
In reply to Danilo KrummrichQuoted 10 lines · On Tue, 1 Oct 2024 17:00:00 +0200 Danilo Krummrich <[email protected]> wrote:
So far the kernel's
BoxandVectypes can't be used by userspace test cases, since all users of those types (e.g.CString) use kernel allocators for instantiation.In order to allow userspace test cases to make use of such types as well, implement the
Cmallocallocator within theallocator_testmodule and type alias all kernel allocators toCmalloc. TheCmallocallocator uses libc'srealloc()function as allocator backend.Signed-off-by Danilo Krummrich <[email protected]>
Quoted 3 lines
--- rust/kernel/alloc/allocator_test.rs | 88 ++++++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 7 deletions(-)
Danilo Krummrich
maintainerPATCH v8 26/29 rust: str: test: replace `alloc::format`
In reply to Danilo KrummrichThe current implementation of tests in str.rs use format! to format strings for comparison, which, internally, creates a new String.
In order to prepare for getting rid of Rust's alloc crate, we have to cut this dependency. Instead, implement format! for CString.
Note that for userspace tests, Kmalloc, which is backing CString's memory, is just a type alias to Cmalloc.
--- rust/kernel/str.rs | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
rust/kernel/str.rs +22 -1
@@ -523,7 +523,28 @@ macro_rules! c_str {#[cfg(test)] mod tests {use super ::*; -use alloc::format; + +struct String (CString ); + +impl String { +fn from_fmt (args: fmt::Arguments <'_>) ->Self { +String (CString ::try_from_fmt (args).unwrap ()) + } + } + +impl Deref for String { +type Target =str ; + +fn deref (&self) -> &str { + self.0 .to_str ().unwrap () + } + } + +macro_rules! format { + ($($f:tt)*) => ({ + &*String ::from_fmt (kernel::fmt!($($f)*)) + }) + }const ALL_ASCII_CHARS : &'staticstr ="\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x09\\x0a\\x0b\\x0c\\x0d\\x0e\\x0f\
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 27/29 rust: alloc: update module comment of alloc.rs
In reply to Danilo KrummrichBefore we remove Rust's alloc crate, rewrite the module comment in alloc.rs to avoid a rustdoc warning.
Besides that, the module comment in alloc.rs isn't correct anymore, we're no longer extending Rust's alloc crate.
--- rust/kernel/alloc.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/alloc.rs +1 -1
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -//! Extensions to the [`alloc`] crate. +//! Implementation of the kernel's memory allocation infrastructure. #[cfg(not(any(test, testlib)))] pub mod allocator;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 28/29 kbuild: rust: remove the `alloc` crate and `GlobalAlloc`
In reply to Danilo KrummrichNow that we have our own Allocator, Box and Vec types we can remove Rust's alloc crate and the new_uninit unstable feature.
Also remove Kmalloc's GlobalAlloc implementation -- we can't remove this in a separate patch, since the alloc crate requires a #[global_allocator] to set, that implements GlobalAlloc.
--- rust/Makefile | 43 +++++---------------- rust/exports.c | 1 - rust/kernel/alloc/allocator.rs | 63 +------------------------------ scripts/Makefile.build | 7 +--- scripts/generate_rust_analyzer.py | 11 +----- 5 files changed, 15 insertions(+), 110 deletions(-)
rust/Makefile +10 -33
@@ -15,8 +15,8 @@ always-$(CONFIG_RUST) += libmacros.so no-clean-files += libmacros.so always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs -obj-$(CONFIG_RUST) += alloc.o bindings.o kernel.o -always-$(CONFIG_RUST) += exports_alloc_generated.h exports_helpers_generated.h \ +obj-$(CONFIG_RUST) += bindings.o kernel.o +always-$(CONFIG_RUST) += exports_helpers_generated.h \ exports_bindings_generated.h exports_kernel_generated.h always-$(CONFIG_RUST) += uapi/uapi_generated.rs
@@ -53,11 +53,6 @@ endif core-cfgs = \ --cfg no_fp_fmt_parse -alloc-cfgs = \ - --cfg no_global_oom_handling \ - --cfg no_rc \ - --cfg no_sync - quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $< cmd_rustdoc = \ OBJTREE=$(abspath $(objtree)) \
@@ -81,7 +76,7 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $< # command-like flags to solve the issue. Meanwhile, we use the non-custom case # and then retouch the generated files. rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \ - rustdoc-alloc rustdoc-kernel + rustdoc-kernel $(Q)cp $(srctree)/Documentation/images/logo.svg $(rustdoc_output)/static.files/ $(Q)cp $(srctree)/Documentation/images/COPYING-logo $(rustdoc_output)/static.files/ $(Q)find $(rustdoc_output) -name '*.html' -type f -print0 | xargs -0 sed -Ei \
@@ -105,20 +100,11 @@ rustdoc-core: $(RUST_LIB_SRC)/core/src/lib.rs FORCE rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE +$(call if_changed,rustdoc) -# We need to allow `rustdoc::broken_intra_doc_links` because some -# `no_global_oom_handling` functions refer to non-`no_global_oom_handling` -# functions. Ideally `rustdoc` would have a way to distinguish broken links -# due to things that are "configured out" vs. entirely non-existing ones. -rustdoc-alloc: private rustc_target_flags = $(alloc-cfgs) \ - -Arustdoc::broken_intra_doc_links -rustdoc-alloc: $(RUST_LIB_SRC)/alloc/src/lib.rs rustdoc-core rustdoc-compiler_builtins FORCE - +$(call if_changed,rustdoc) - -rustdoc-kernel: private rustc_target_flags = --extern alloc \ +rustdoc-kernel: private rustc_target_flags = \ --extern build_error --extern macros=$(objtree)/$(obj)/libmacros.so \ --extern bindings --extern uapi rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-macros \ - rustdoc-compiler_builtins rustdoc-alloc $(obj)/libmacros.so \ + rustdoc-compiler_builtins $(obj)/libmacros.so \ $(obj)/bindings.o FORCE +$(call if_changed,rustdoc)
@@ -162,7 +148,7 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $< mkdir -p $(objtree)/$(obj)/test/doctests/kernel; \ OBJTREE=$(abspath $(objtree)) \ $(RUSTDOC) --test $(rust_flags) \ - -L$(objtree)/$(obj) --extern alloc --extern kernel \ + -L$(objtree)/$(obj) --extern kernel \ --extern build_error --extern macros \ --extern bindings --extern uapi \ --no-run --crate-name kernel -Zunstable-options \
@@ -198,7 +184,7 @@ rusttest-macros: $(src)/macros/lib.rs FORCE +$(call if_changed,rustc_test) +$(call if_changed,rustdoc_test) -rusttest-kernel: private rustc_target_flags = --extern alloc \ +rusttest-kernel: private rustc_target_flags = \ --extern build_error --extern macros --extern bindings --extern uapi rusttest-kernel: $(src)/kernel/lib.rs \ rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
@@ -311,9 +297,6 @@ quiet_cmd_exports = EXPORTS $@ $(obj)/exports_core_generated.h: $(obj)/core.o FORCE $(call if_changed,exports) -$(obj)/exports_alloc_generated.h: $(obj)/alloc.o FORCE - $(call if_changed,exports) - # Even though Rust kernel modules should never use the bindings directly, # symbols from the `bindings` crate and the C helpers need to be exported # because Rust generics and inlined functions may not get their code generated
@@ -360,7 +343,7 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L rust-analyzer: $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ - --cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \ + --cfgs='core=$(core-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ $(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json
@@ -398,12 +381,6 @@ $(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*' $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE +$(call if_changed_rule,rustc_library) -$(obj)/alloc.o: private skip_clippy = 1 -$(obj)/alloc.o: private skip_flags = -Wunreachable_pub -$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs) -$(obj)/alloc.o: $(RUST_LIB_SRC)/alloc/src/lib.rs $(obj)/compiler_builtins.o FORCE - +$(call if_changed_rule,rustc_library) - $(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE +$(call if_changed_rule,rustc_library)
@@ -418,9 +395,9 @@ $(obj)/uapi.o: $(src)/uapi/lib.rs \ $(obj)/uapi/uapi_generated.rs FORCE +$(call if_changed_rule,rustc_library) -$(obj)/kernel.o: private rustc_target_flags = --extern alloc \ +$(obj)/kernel.o: private rustc_target_flags = \ --extern build_error --extern macros --extern bindings --extern uapi -$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \ +$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/build_error.o \ $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE +$(call if_changed_rule,rustc_library)
rust/exports.c +0 -1
@@ -16,7 +16,6 @@#define EXPORT_SYMBOL_RUST_GPL (sym) extern int sym; EXPORT_SYMBOL_GPL(sym)#include "exports_core_generated.h" -#include "exports_alloc_generated.h" #include "exports_helpers_generated.h" #include "exports_bindings_generated.h" #include "exports_kernel_generated.h"
rust/kernel/alloc/allocator.rs +2 -61
84 lines not on this page · raw message
scripts/Makefile.build +1 -6
@@ -247,18 +247,13 @@ $(obj)/%.lst: $(obj)/%.c FORCE # Compile Rust sources (.rs) # --------------------------------------------------------------------------- - -rust_allowed_features := new_uninit - # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. rust_common_cmd = \ RUST_MODFILE=$(modfile) $(RUSTC_OR_CLIPPY) $(rust_flags) \ - -Zallow-features=$(rust_allowed_features) \ -Zcrate-attr=no_std \ - -Zcrate-attr='feature($(rust_allowed_features))' \ - -Zunstable-options --extern force:alloc --extern kernel \ + -Zunstable-options --extern kernel \ --crate-type rlib -L $(objtree)/rust/ \ --crate-name $(basename $(notdir $@)) \ --sysroot=/dev/null \
scripts/generate_rust_analyzer.py +2 -9
@@ -64,13 +64,6 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): [], ) - append_crate( - "alloc", - sysroot_src / "alloc" / "src" / "lib.rs", - ["core", "compiler_builtins"], - cfg=crates_cfgs.get("alloc", []), - ) - append_crate( "macros", srctree / "rust" / "macros" / "lib.rs",
@@ -96,7 +89,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): append_crate( "kernel", srctree / "rust" / "kernel" / "lib.rs", - ["core", "alloc", "macros", "build_error", "bindings"], + ["core", "macros", "build_error", "bindings"], cfg=cfg, ) crates[-1]["source"] = {
@@ -133,7 +126,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): append_crate( name, path, - ["core", "alloc", "kernel"], + ["core", "kernel"], cfg=cfg, )
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v8 29/29 MAINTAINERS: add entry for the Rust `alloc` module
In reply to Danilo KrummrichAdd maintainers entry for the Rust alloc module.
Currently, this includes the Allocator API itself, Allocator implementations, such as Kmalloc or Vmalloc, as well as the kernel's implementation of the primary memory allocation data structures, Box and Vec.
--- MAINTAINERS | 7 +++++++ 1 file changed, 7 insertions(+)
MAINTAINERS +7 -0
@@ -20225,6 +20225,13 @@ F: scripts/*rust* F: tools/testing/selftests/rust/ K: \b(?i:rust)\b +RUST [ALLOC] +M: Danilo Krummrich <[email protected]> +L: [email protected] +S: Maintained +F: rust/kernel/alloc.rs +F: rust/kernel/alloc/ + RXRPC SOCKETS (AF_RXRPC) M: David Howells <[email protected]> M: Marc Dionne <[email protected]>
Signature · 1 line
2.46.1