Generic `Allocator` support for Rust
Subsystems (10)
Danilo Krummrich
maintainerv9 cover Generic `Allocator` support for Rust
Hi,
This patch series adds generic kernel allocator support for Rust, which so far is limited to kmalloc allocations.
In order to abstain from (re-)adding unstable Rust features to the kernel, this patch series does not extend the Allocator trait from Rust's alloc crate, nor does it extend the BoxExt and VecExt extensions.
Instead, this series introduces a kernel specific Allocator trait, which is implemented by the Kmalloc, Vmalloc and KVmalloc allocators, also implemented in the context of this series.
As a consequence we need our own kernel Box<T, A> and Vec<T, A> types. Additionally, this series adds the following type aliases:
``` pub type KBox<T> = Box<T, Kmalloc>; pub type VBox<T> = Box<T, Vmalloc>; pub type KVBox<T> = Box<T, KVmalloc>; pub type KVec<T> = Vec<T, Kmalloc>; pub type VVec<T> = Vec<T, Vmalloc>; pub type KVVec<T> = Vec<T, KVmalloc>; ```
With that, we can start using the kernel Box and Vec types throughout the tree and remove the now obolete extensions BoxExt and VecExt.
For a final cleanup, this series removes the last minor dependencies to Rust's alloc crate and removes it from the entire kernel build.
The series ensures not to break the rusttest make target by implementing the allocator_test module providing a stub implementation for all kernel Allocators.
This patch series passes all KUnit tests, including the ones added by this series. Additionally, the tests were run with kmemleak and KASAN enabled, without any issues.
This series is based on [1], which hit -mm/mm-stable, and is also available in [2].
[1] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=mm/krealloc
[2] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=rust/mm
Changes in v9:
- rebase onto rust-next (lint series)
- Vec:
- add missing invariant comments
- fix missing #![feature(inline_const)]
- add note for kvec! that GFP_KERNEL is used for allocation
- ArrayLayout: fix missing import for size_of::<T>
- Box: fix missing safety comment
Changes in v8:
- rebase onto rust-next (v6.12-rc1), including the -mm dependencies from [1]
- introduce ArrayLayout and rework the Allocator trait to take an old_layout argument
- split up "rust: alloc: implement Allocator for Kmalloc" in three separate patches (introduce ReallocFunc, make allocator module public and implement Kmalloc)
- wrap impl_slice_eq! pattern in $()*
- couple of style changes suggested by Benno and Gary
Changes in v7:
- rebase onto rust-next resolving recent conflicts (93dc3be19450447a3a7090bd1dfb9f3daac3e8d2)
- documentation / safety comment changes suggested by Benno and Boqun
- rename ptr to current in IntoIter::next (Alice)
- remove unnecessary braces in kvec! (Benno)
- add debug_assert! in Vec::set_len (Benno)
- remove unused args in impl_slice_eq! (Benno)
- simplify if statement in Cmalloc::realloc (Benno)
Changes in v6:
- rebase onto rust-dev
- keep compiler annotations for {k,v,kv}realloc()
- documentation changes suggested by Alice, Benno
- remove Box::into_pin
- fix typo in Send and Sync for Box and Vec
- kvec! changes suggested by Alice
- free src after copy in Cmalloc
- handle n == 0 in Vec::extend_with
Changes in v5:
- (safety) comment / documentation fixes suggested by Alice, Benno and Gary
- remove Unique<T> and implement Send and Sync for Box and Vec
- use KMALLOC_MAX_SIZE for KVmalloc test and add a Kmalloc test that expects to fail for KMALLOC_MAX_SIZE
- create use constants KREALLOC, VREALLOC and KVREALLOC for ReallocFuncs
- drop Box::drop_contents for now, will add it again, once I actually rebase on the original patch that introduces it
- improve usage of size_of_val in Box
- move InPlaceInit and ForeignOwnable impls into kbox.rs
- fix missing Box conversions in rnull.rs
- reworked Cmalloc to keep track of the size of memory allocations itself
- remove GlobalAlloc together with the alloc crate to avoid a linker error
- remove alloc from scripts/generate_rust_analyzer.py
Changes in v4:
- (safety) comment fixes suggested by Alice and Boqun
- remove Box::from_raw_alloc and Box::into_raw_alloc, we don't need them
- in Box::drop call size_of_val before drop_in_place
- implement ForeignOwnable for Pin<Box<T>> as suggested by Alice
- in Vec::extend_with, iterate over n instead of spare.len()
- for Vmalloc and KVmalloc fail allocation for alignments larger than PAGE_SIZE for now (will add support for larger alignments in a separate series)
- implement Cmalloc in allocator_test and type alias all kernel allocator types to it, such that we can use the kernel's Box and Vec types in userspace tests (rusttest)
- this makes patch "rust: str: test: replace alloc::format" rather trivial
Changes in v3:
- Box:
- minor documentation fixes
- removed unnecessary imports in doc tests
- dropeed self argument from some remaining Box methods
- implement InPlaceInit for Box<T, A> rather than specifically for KBox<T>
- Vec:
- minor documentation fixes
- removed useless Vec::allocator method
- in Vec::extend_with use Vec::spare_capacity_mut instead of raw pointer operations
- added a few missing safety comments
- pass GFP flags to Vec::collect
- fixed a rustdoc warning in alloc.rs
- fixed the allocator_test module to implement the Allocator trait correctly
- rebased to rust-next
Changes in v2:
- preserve impl GlobalAlloc for Kmalloc and remove it at the end (Benno)
- remove &self parameter from all Allocator functions (Benno)
- various documentation fixes for Allocator (Benno)
- use NonNull<u8> for Allocator::free and Option<NonNull<u8>> for Allocator::realloc (Benno)
- fix leak of IntoIter in Vec::collect (Boqun)
- always realloc (try to shrink) in Vec::collect, it's up the the Allocator to provide a heuristic whether it makes sense to actually shrink
- rename KBox<T, A> -> Box<T, A> and KVec<T, A> -> Vec<T, A> and provide type aliases KBox<T>, VBox<T>, KVBox<T>, etc.
- This allows for much cleaner code and, in combination with removing &self parameters from Allocators, gets us rid of the need for Box::new and Box::new_alloc and all other "_alloc" postfixed functions.
- Before: KBox::new_alloc(foo, Vmalloc)?
- After: VBox::new(foo)?, which resolves to
`Box::<Foo, Vmalloc>::new(foo)?;
Benno Lossin (1):
rust: alloc: introduce ArrayLayout
Danilo Krummrich (28):
rust: alloc: add Allocator trait
rust: alloc: separate aligned_size from krealloc_aligned
rust: alloc: rename KernelAllocator to Kmalloc
rust: alloc: implement ReallocFunc
rust: alloc: make allocator module public
rust: alloc: implement Allocator for Kmalloc
rust: alloc: add module allocator_test
rust: alloc: implement Vmalloc allocator
rust: alloc: implement KVmalloc allocator
rust: alloc: add __GFP_NOWARN to Flags
rust: alloc: implement kernel Box
rust: treewide: switch to our kernel Box type
rust: alloc: remove extension of std's Box
rust: alloc: add Box to prelude
rust: alloc: implement kernel Vec type
rust: alloc: implement IntoIterator for Vec
rust: alloc: implement collect for IntoIter
rust: treewide: switch to the kernel Vec type
rust: alloc: remove VecExt extension
rust: alloc: add Vec to prelude
rust: error: use core::alloc::LayoutError
rust: error: check for config test in Error::name
rust: alloc: implement contains for Flags
rust: alloc: implement Cmalloc in module allocator_test
rust: str: test: replace alloc::format
rust: alloc: update module comment of alloc.rs
kbuild: rust: remove the alloc crate and GlobalAlloc
MAINTAINERS: add entry for the Rust alloc module
MAINTAINERS | 7 + drivers/block/rnull.rs | 4 +- rust/Makefile | 43 +- rust/bindings/bindings_helper.h | 1 + rust/exports.c | 1 - rust/helpers/helpers.c | 1 + rust/helpers/slab.c | 6 + rust/helpers/vmalloc.c | 9 + rust/kernel/alloc.rs | 150 ++++- rust/kernel/alloc/allocator.rs | 210 +++++-- rust/kernel/alloc/allocator_test.rs | 95 +++ rust/kernel/alloc/box_ext.rs | 89 --- rust/kernel/alloc/kbox.rs | 456 ++++++++++++++ rust/kernel/alloc/kvec.rs | 914 ++++++++++++++++++++++++++++ rust/kernel/alloc/layout.rs | 91 +++ rust/kernel/alloc/vec_ext.rs | 185 ------ rust/kernel/error.rs | 6 +- rust/kernel/init.rs | 95 +-- rust/kernel/init/__internal.rs | 2 +- rust/kernel/lib.rs | 2 +- rust/kernel/prelude.rs | 5 +- rust/kernel/rbtree.rs | 49 +- rust/kernel/str.rs | 35 +- rust/kernel/sync/arc.rs | 17 +- rust/kernel/sync/condvar.rs | 4 +- rust/kernel/sync/lock/mutex.rs | 2 +- rust/kernel/sync/lock/spinlock.rs | 2 +- rust/kernel/sync/locked_by.rs | 2 +- rust/kernel/types.rs | 52 +- rust/kernel/uaccess.rs | 17 +- rust/kernel/workqueue.rs | 20 +- rust/macros/lib.rs | 12 +- samples/rust/rust_minimal.rs | 4 +- scripts/Makefile.build | 4 +- scripts/generate_rust_analyzer.py | 11 +- 35 files changed, 2025 insertions(+), 578 deletions(-) create mode 100644 rust/helpers/vmalloc.c create mode 100644 rust/kernel/alloc/allocator_test.rs delete mode 100644 rust/kernel/alloc/box_ext.rs create mode 100644 rust/kernel/alloc/kbox.rs create mode 100644 rust/kernel/alloc/kvec.rs create mode 100644 rust/kernel/alloc/layout.rs delete mode 100644 rust/kernel/alloc/vec_ext.rs
base-commit: 7f4a2c6cacb0efad2bd9ae5062ab1b75e13c6c1f
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 1/29 rust: alloc: add `Allocator` trait
Add a kernel specific Allocator trait, that in contrast to the one in Rust's core library doesn't require unstable features and supports GFP flags.
Subsequent patches add the following trait implementors: Kmalloc, Vmalloc and KVmalloc.
--- rust/kernel/alloc.rs | 101 +++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 101 insertions(+)
rust/kernel/alloc.rs +101 -0
@@ -11,6 +11,7 @@/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ; +use core::{alloc::Layout , ptr::NonNull };/// Flags to be used when allocating memory. ///
@@ -86,3 +87,103 @@ pub mod flags {/// small allocations. pub const GFP_NOWAIT :Flags =Flags (bindings::GFP_NOWAIT ); } + +/// The kernel's [`Allocator`] trait. +/// +/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffers described +/// via [`Layout`]. +/// +/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on +/// an object instance. +/// +/// In order to be able to support `#[derive(SmartPointer)]` later on, we need to avoid a design +/// that requires an `Allocator` to be instantiated, hence its functions must not contain any kind +/// of `self` parameter. +/// +/// # Safety +/// +/// - A memory allocation returned from an allocator must remain valid until it is explicitly freed. +/// +/// - Any pointer to a valid memory allocation must be valid to be passed to any other [`Allocator`] +/// function of the same type. +/// +/// - Implementers must ensure that all trait functions abide by the guarantees documented in the +/// `# Guarantees` sections. +pub unsafe trait Allocator { +/// Allocate memory based on `layout` and `flags`. +/// +/// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout +/// constraints (i.e. minimum size and alignment as specified by `layout`). +/// +/// This function is equivalent to `realloc` when called with `None`. +/// +/// # Guarantees +/// +/// When the return value is `Ok(ptr)`, then `ptr` is +/// - valid for reads and writes for `layout.size()` bytes, until it is passed to +/// [`Allocator::free`] or [`Allocator::realloc`], +/// - aligned to `layout.align()`, +/// +/// Additionally, `Flags` are honored as documented in +/// <https://docs.kernel.org/core-api/mm-api.html#mm-api-gfp-flags>. +fn alloc (layout:Layout , flags:Flags ) ->Result <NonNull <[u8 ]>,AllocError > { +// SAFETY: Passing `None` to `realloc` is valid by it's safety requirements and asks for a +// new memory allocation. +unsafe {Self ::realloc (None , layout,Layout ::new ::<()>(), flags) } + } + +/// Re-allocate an existing memory allocation to satisfy the requested `layout`. +/// +/// If the requested size is zero, `realloc` behaves equivalent to `free`. +/// +/// If the requested size is larger than the size of the existing allocation, a successful call +/// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but +/// may also be larger. +/// +/// If the requested size is smaller than the size of the existing allocation, `realloc` may or +/// may not shrink the buffer; this is implementation specific to the allocator. +/// +/// On allocation failure, the existing buffer, if any, remains valid. +/// +/// The buffer is represented as `NonNull<[u8]>`. +/// +/// # Safety +/// +/// - If `ptr == Some(p)`, then `p` must point to an existing and valid memory allocation +/// created by this [`Allocator`]; if `old_layout` is zero-sized `p` does not need to be a +/// pointer returned by this [`Allocator`]. +/// - `ptr` is allowed to be `None`; in this case a new memory allocation is created and +/// `old_layout` is ignored. +/// - `old_layout` must match the `Layout` the allocation has been created with. +/// +/// # Guarantees +/// +/// This function has the same guarantees as [`Allocator::alloc`]. When `ptr == Some(p)`, then +/// it additionally guarantees that: +/// - the contents of the memory pointed to by `p` are preserved up to the lesser of the new +/// and old size, i.e. `ret_ptr[0..min(layout.size(), old_layout.size())] == +/// p[0..min(layout.size(), old_layout.size())]`. +/// - when the return value is `Err(AllocError)`, then `ptr` is still valid. +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError >; + +/// Free an existing memory allocation. +/// +/// # Safety +/// +/// - `ptr` must point to an existing and valid memory allocation created by this [`Allocator`]; +/// if `old_layout` is zero-sized `p` does not need to be a pointer returned by this +/// [`Allocator`]. +/// - `layout` must match the `Layout` the allocation has been created with. +/// - The memory allocation at `ptr` must never again be read from or written to. +unsafe fn free (ptr:NonNull <u8 >, layout:Layout ) { +// SAFETY: The caller guarantees that `ptr` points at a valid allocation created by this +// allocator. We are passing a `Layout` with the smallest possible alignment, so it is +// smaller than or equal to the alignment previously used with this allocation. +let _ =unsafe {Self ::realloc (Some (ptr),Layout ::new ::<()>(), layout,Flags (0 )) }; + } +}
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 2/29 rust: alloc: separate `aligned_size` from `krealloc_aligned`
In reply to Danilo KrummrichSeparate aligned_size from krealloc_aligned.
Subsequent patches implement Allocator derivates, such as Kmalloc, that require aligned_size and replace the original krealloc_aligned.
--- rust/kernel/alloc/allocator.rs | 19 ++++++++++++------- 1 file changed, 12 insertions(+), 7 deletions(-)
rust/kernel/alloc/allocator.rs +12 -7
@@ -8,6 +8,17 @@struct KernelAllocator ; +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. +fn aligned_size (new_layout:Layout ) ->usize { +// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first. +let layout = new_layout.pad_to_align (); + +// Note that `layout.size()` (after padding) is guaranteed to be a multiple of `layout.align()` +// which together with the slab guarantees means the `krealloc` will return a properly aligned +// object (see comments in `kmalloc()` for more information). + layout.size () +} +/// Calls `krealloc` with a proper size to alloc a new object aligned to `new_layout`'s alignment. /// /// # Safety
@@ -15,13 +26,7 @@/// - `ptr` can be either null or a pointer which has been allocated by this allocator. /// - `new_layout` must have a non-zero size. pub (crate )unsafe fn krealloc_aligned (ptr: *mut u8 , new_layout:Layout , flags:Flags ) -> *mut u8 { -// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first. -let layout = new_layout.pad_to_align (); - -// Note that `layout.size()` (after padding) is guaranteed to be a multiple of `layout.align()` -// which together with the slab guarantees means the `krealloc` will return a properly aligned -// object (see comments in `kmalloc()` for more information). -let size = layout.size (); +let size =aligned_size (new_layout);// SAFETY: // - `ptr` is either null or a pointer returned from a previous `k{re}alloc()` by the
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 3/29 rust: alloc: rename `KernelAllocator` to `Kmalloc`
In reply to Danilo KrummrichSubsequent patches implement Vmalloc and KVmalloc allocators, hence align KernelAllocator to this naming scheme.
--- rust/kernel/alloc/allocator.rs | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-)
rust/kernel/alloc/allocator.rs +3 -3
@@ -6,7 +6,7 @@use core::alloc::{GlobalAlloc ,Layout };use core::ptr; -struct KernelAllocator ; +struct Kmalloc ;/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {
@@ -37,7 +37,7 @@ pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F }// SAFETY: TODO. -unsafe impl GlobalAlloc for KernelAllocator { +unsafe impl GlobalAlloc for Kmalloc {unsafe fn alloc (&self, layout:Layout ) -> *mut u8 {// SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety // requirement.
@@ -74,7 +74,7 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { }#[global_allocator] -static ALLOCATOR :KernelAllocator =KernelAllocator ; +static ALLOCATOR :Kmalloc =Kmalloc ;// See <https://github.com/rust-lang/rust/pull/86844>. #[no_mangle]
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 4/29 rust: alloc: implement `ReallocFunc`
In reply to Danilo KrummrichReallocFunc is an abstraction for the kernel's realloc derivates, such as krealloc, vrealloc and kvrealloc.
All of the named functions share the same function signature and implement the same semantics. The ReallocFunc abstractions provides a generalized wrapper around those, to trivialize the implementation of Kmalloc, Vmalloc and KVmalloc in subsequent patches.
--- rust/kernel/alloc.rs | 8 ++++ rust/kernel/alloc/allocator.rs | 70 ++++++++++++++++++++++++++++++++++ 2 files changed, 78 insertions(+)
rust/kernel/alloc.rs +8 -0
@@ -187,3 +187,11 @@ unsafe fn free(ptr: NonNull<u8>, layout: Layout) {let _ =unsafe {Self ::realloc (Some (ptr),Layout ::new ::<()>(), layout,Flags (0 )) }; } } + +/// Returns a properly aligned dangling pointer from the given `layout`. +pub (crate )fn dangling_from_layout (layout:Layout ) ->NonNull <u8 > { +let ptr = layout.align ()as *mut u8 ; + +// SAFETY: `layout.align()` (and hence `ptr`) is guaranteed to be non-zero. +unsafe {NonNull ::new_unchecked (ptr) } +}
rust/kernel/alloc/allocator.rs +70 -0
86 lines not on this page · raw message
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 5/29 rust: alloc: make `allocator` module public
In reply to Danilo KrummrichSubsequent patches implement allocators such as Kmalloc, Vmalloc, KVmalloc; we need them to be available outside of the kernel crate as well.
--- rust/kernel/alloc.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/alloc.rs +1 -1
@@ -4,7 +4,7 @@#[cfg(not(test))] #[cfg(not(testlib))] -mod allocator; +pub mod allocator;pub mod box_ext;pub mod vec_ext;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 6/29 rust: alloc: implement `Allocator` for `Kmalloc`
In reply to Danilo KrummrichImplement Allocator for Kmalloc, the kernel's default allocator, typically used for objects smaller than page size.
All memory allocations made with Kmalloc end up in krealloc().
It serves as allocator for the subsequently introduced types KBox and KVec.
--- rust/kernel/alloc/allocator.rs | 31 ++++++++++++++++++++++++++++--- 1 file changed, 28 insertions(+), 3 deletions(-)
rust/kernel/alloc/allocator.rs +28 -3
@@ -13,10 +13,16 @@use core::ptr;use core::ptr::NonNull ; -use crate ::alloc::AllocError ; +use crate ::alloc::{AllocError ,Allocator };use crate ::bindings; -struct Kmalloc ; +/// The contiguous kernel allocator. +/// +/// `Kmalloc` is typically used for physically contiguous allocations up to page size, but also +/// supports larger allocations up to `bindings::KMALLOC_MAX_SIZE`, which is hardware specific. +/// +/// For more details see [self]. +pub struct Kmalloc ;/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {
@@ -53,8 +59,10 @@ struct ReallocFunc( unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void, ); -#[expect(dead_code)] impl ReallocFunc { +// INVARIANT: `krealloc` satisfies the type invariants. +const KREALLOC :Self =Self (bindings::krealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`].
@@ -106,6 +114,23 @@ unsafe fn call( } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for Kmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// SAFETY: `ReallocFunc::call` has the same safety requirements as `Allocator::realloc`. +unsafe {ReallocFunc ::KREALLOC .call (ptr, layout, old_layout, flags) } + } +} +// SAFETY: TODO. unsafe impl GlobalAlloc for Kmalloc {unsafe fn alloc (&self, layout:Layout ) -> *mut u8 {
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 7/29 rust: alloc: add module `allocator_test`
In reply to Danilo KrummrichAllocators, such as Kmalloc, will be used by e.g. Box and Vec in subsequent patches, and hence this dependency propagates throughout the whole kernel.
Add the allocator_test module that provides an empty implementation for all Allocators in the kernel, such that we don't break the rusttest make target in subsequent patches.
--- rust/kernel/alloc.rs | 9 +++++++-- rust/kernel/alloc/allocator_test.rs | 19 +++++++++++++++++++ 2 files changed, 26 insertions(+), 2 deletions(-) create mode 100644 rust/kernel/alloc/allocator_test.rs
rust/kernel/alloc.rs +7 -2
@@ -2,12 +2,17 @@//! Extensions to the [`alloc`] crate. -#[cfg(not(test))] -#[cfg(not(testlib))] +#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod box_ext;pub mod vec_ext; +#[cfg(any(test, testlib))] +pub mod allocator_test; + +#[cfg(any(test, testlib))] +pub use self::allocator_testas allocator; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;
rust/kernel/alloc/allocator_test.rs new file +19 -0
@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 + +#![allow(missing_docs)] + +use super ::{AllocError ,Allocator ,Flags }; +use core::alloc::Layout ; +use core::ptr::NonNull ; + +pub struct Kmalloc ; + +unsafe impl Allocator for Kmalloc { +unsafe fn realloc ( + _ptr:Option <NonNull <u8 >>, + _layout:Layout , + _flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +panic! (); + } +}
Signature · 1 line
2.46.1
Re: [PATCH v9 07/29] rust: alloc: add module `allocator_test`
Quoted 52 lines · Danilo Krummrich
Allocators, such asKmalloc, will be used by e.g.BoxandVecin subsequent patches, and hence this dependency propagates throughout the whole kernel.Add the
allocator_testmodule that provides an empty implementation for allAllocators in the kernel, such that we don't break therusttestmake target in subsequent patches.Reviewed-by Alice Ryhl <[email protected]>Reviewed-by Benno Lossin <[email protected]>Reviewed-by Gary Guo <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc.rs | 9 +++++++-- rust/kernel/alloc/allocator_test.rs | 19 +++++++++++++++++++ 2 files changed, 26 insertions(+), 2 deletions(-) create mode 100644 rust/kernel/alloc/allocator_test.rsrust/kernel/alloc.rs +7
-2@@ -2,12 +2,17 @@//! Extensions to the [`alloc`] crate. -#[cfg(not(test))] -#[cfg(not(testlib))] +#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod box_ext;pub mod vec_ext; +#[cfg(any(test, testlib))] +pub mod allocator_test; + +#[cfg(any(test, testlib))] +pub use self::allocator_testas allocator; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;rust/kernel/alloc/allocator_test.rs new file +4
-0@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 + +#![allow(missing_docs)] +
rust/kernel/alloc/allocator_test.rs line 14
Quoted 10 lines
rust/kernel/alloc/allocator_test.rs new file +10
-0+use super ::{AllocError ,Allocator ,Flags }; +use core::alloc::Layout ; +use core::ptr::NonNull ; + +pub struct Kmalloc ; + +unsafe impl Allocator for Kmalloc { +unsafe fn realloc ( + _ptr:Option <NonNull <u8 >>, + _layout: Layout,
The old_layout parameter is missing here.
Regards,
Boqun
Quoted 8 lines
rust/kernel/alloc/allocator_test.rs new file +5
-0+ _flags: Flags, + ) -> Result<NonNull<[u8]>, AllocError> { +panic! (); + } +}Signature · 1 line
2.46.1
Danilo Krummrich
maintainerRe: [PATCH v9 07/29] rust: alloc: add module `allocator_test`
Quoted 57 lines · Boqun Feng
Quoted 56 lines · Danilo Krummrich
Allocators, such asKmalloc, will be used by e.g.BoxandVecin subsequent patches, and hence this dependency propagates throughout the whole kernel.Add the
allocator_testmodule that provides an empty implementation for allAllocators in the kernel, such that we don't break therusttestmake target in subsequent patches.Reviewed-by Alice Ryhl <[email protected]>Reviewed-by Benno Lossin <[email protected]>Reviewed-by Gary Guo <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc.rs | 9 +++++++-- rust/kernel/alloc/allocator_test.rs | 19 +++++++++++++++++++ 2 files changed, 26 insertions(+), 2 deletions(-) create mode 100644 rust/kernel/alloc/allocator_test.rsrust/kernel/alloc.rs +7
-2@@ -2,12 +2,17 @@//! Extensions to the [`alloc`] crate. -#[cfg(not(test))] -#[cfg(not(testlib))] +#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod box_ext;pub mod vec_ext; +#[cfg(any(test, testlib))] +pub mod allocator_test; + +#[cfg(any(test, testlib))] +pub use self::allocator_testas allocator; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;rust/kernel/alloc/allocator_test.rs new file +8
-0@@ -0,0 +1,19 @@ +// SPDX-License-Identifier: GPL-2.0 + +#![allow(missing_docs)] + +use super ::{AllocError ,Allocator ,Flags }; +use core::alloc::Layout ; +use core::ptr::NonNull ; +
Quoted 8 lines
rust/kernel/alloc/allocator_test.rs line 14
Quoted 6 lines
rust/kernel/alloc/allocator_test.rs new file +6
-0+pub struct Kmalloc ; + +unsafe impl Allocator for Kmalloc { +unsafe fn realloc ( + _ptr:Option <NonNull <u8 >>, + _layout: Layout,The
old_layoutparameter is missing here.
Thanks, good catch.
rust/kernel/alloc.rs +1 -0
@@ -193,6 +193,7 @@ unsafe fn free(ptr: NonNull<u8>, layout: Layout) { } } +#[allow(dead_code)] /// Returns a properly aligned dangling pointer from the given `layout`. pub (crate )fn dangling_from_layout (layout:Layout ) ->NonNull <u8 > {let ptr = layout.align ()as *mut u8 ;
rust/kernel/alloc/allocator_test.rs +1 -0
@@ -12,6 +12,7 @@ unsafe impl Allocator for Kmalloc {unsafe fn realloc ( _ptr:Option <NonNull <u8 >>, _layout:Layout , + _old_layout:Layout , _flags:Flags , ) ->Result <NonNull <[u8 ]>,AllocError > {panic! ();
We also have to fix an intermediate dead code warning for dangling_from_layout in the rusttest target, until we start using it when we implement Cmalloc in this module.
Danilo Krummrich
maintainerPATCH v9 8/29 rust: alloc: implement `Vmalloc` allocator
In reply to Danilo KrummrichImplement Allocator for Vmalloc, the kernel's virtually contiguous allocator, typically used for larger objects, (much) larger than page size.
All memory allocations made with Vmalloc end up in vrealloc().
--- rust/helpers/helpers.c | 1 + rust/helpers/vmalloc.c | 9 +++++++ rust/kernel/alloc/allocator.rs | 37 +++++++++++++++++++++++++++++ rust/kernel/alloc/allocator_test.rs | 1 + 4 files changed, 48 insertions(+) create mode 100644 rust/helpers/vmalloc.c
rust/helpers/helpers.c +1 -0
@@ -22,5 +22,6 @@#include "spinlock.c" #include "task.c" #include "uaccess.c" +#include "vmalloc.c" #include "wait.c" #include "workqueue.c"
rust/helpers/vmalloc.c new file +9 -0
@@ -0,0 +1,9 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/vmalloc.h> + +void * __must_check __realloc_size(2) +rust_helper_vrealloc(const void *p, size_t size, gfp_t flags) +{ +return vrealloc (p, size, flags); +}
rust/kernel/alloc/allocator.rs +37 -0
@@ -15,6 +15,7 @@use crate ::alloc::{AllocError ,Allocator };use crate ::bindings; +use crate ::pr_warn;/// The contiguous kernel allocator. ///
@@ -24,6 +25,15 @@/// For more details see [self]. pub struct Kmalloc ; +/// The virtually contiguous kernel allocator. +/// +/// `Vmalloc` allocates pages from the page level allocator and maps them into the contiguous kernel +/// virtual space. It is typically used for large allocations. The memory allocated with this +/// allocator is not physically contiguous. +/// +/// For more details see [self]. +pub struct Vmalloc ; +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
@@ -63,6 +73,9 @@ impl ReallocFunc {// INVARIANT: `krealloc` satisfies the type invariants. const KREALLOC :Self =Self (bindings::krealloc); +// INVARIANT: `vrealloc` satisfies the type invariants. +const VREALLOC :Self =Self (bindings::vrealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`].
@@ -168,6 +181,30 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for Vmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// TODO: Support alignments larger than PAGE_SIZE. +if layout.align () > bindings::PAGE_SIZE { +pr_warn! ("Vmalloc does not support alignments larger than PAGE_SIZE yet.\n" ); +return Err (AllocError ); + } + +// SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously +// allocated with this `Allocator`. +unsafe {ReallocFunc ::VREALLOC .call (ptr, layout, old_layout, flags) } + } +} +#[global_allocator] static ALLOCATOR :Kmalloc =Kmalloc ;
rust/kernel/alloc/allocator_test.rs +1 -0
@@ -7,6 +7,7 @@use core::ptr::NonNull ;pub struct Kmalloc ; +pub type Vmalloc =Kmalloc ;unsafe impl Allocator for Kmalloc { unsafe fn realloc(
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 9/29 rust: alloc: implement `KVmalloc` allocator
In reply to Danilo KrummrichImplement Allocator for KVmalloc, an Allocator that tries to allocate memory wth kmalloc first and, on failure, falls back to vmalloc.
All memory allocations made with KVmalloc end up in kvrealloc_noprof(); all frees in kvfree().
--- rust/helpers/slab.c | 6 +++++ rust/kernel/alloc/allocator.rs | 36 +++++++++++++++++++++++++++++ rust/kernel/alloc/allocator_test.rs | 1 + 3 files changed, 43 insertions(+)
rust/helpers/slab.c +6 -0
@@ -7,3 +7,9 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags) {return krealloc (objp, new_size, flags); } + +void * __must_check __realloc_size(2) +rust_helper_kvrealloc(const void *p, size_t size, gfp_t flags) +{ +return kvrealloc (p, size, flags); +}
rust/kernel/alloc/allocator.rs +36 -0
@@ -34,6 +34,15 @@/// For more details see [self]. pub struct Vmalloc ; +/// The kvmalloc kernel allocator. +/// +/// `KVmalloc` attempts to allocate memory with `Kmalloc` first, but falls back to `Vmalloc` upon +/// failure. This allocator is typically used when the size for the requested allocation is not +/// known and may exceed the capabilities of `Kmalloc`. +/// +/// For more details see [self]. +pub struct KVmalloc ; +/// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment. fn aligned_size (new_layout:Layout ) ->usize {// Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.
@@ -76,6 +85,9 @@ impl ReallocFunc {// INVARIANT: `vrealloc` satisfies the type invariants. const VREALLOC :Self =Self (bindings::vrealloc); +// INVARIANT: `kvrealloc` satisfies the type invariants. +const KVREALLOC :Self =Self (bindings::kvrealloc); +/// # Safety /// /// This method has the same safety requirements as [`Allocator::realloc`].
@@ -205,6 +217,30 @@ unsafe fn realloc( } } +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation is OK, +// - `realloc` satisfies the guarantees, since `ReallocFunc::call` has the same. +unsafe impl Allocator for KVmalloc { +#[inline] +unsafe fn realloc ( + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , + ) ->Result <NonNull <[u8 ]>,AllocError > { +// TODO: Support alignments larger than PAGE_SIZE. +if layout.align () > bindings::PAGE_SIZE { +pr_warn! ("KVmalloc does not support alignments larger than PAGE_SIZE yet.\n" ); +return Err (AllocError ); + } + +// SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously +// allocated with this `Allocator`. +unsafe {ReallocFunc ::KVREALLOC .call (ptr, layout, old_layout, flags) } + } +} +#[global_allocator] static ALLOCATOR :Kmalloc =Kmalloc ;
rust/kernel/alloc/allocator_test.rs +1 -0
@@ -8,6 +8,7 @@pub struct Kmalloc ;pub type Vmalloc =Kmalloc ; +pub type KVmalloc =Kmalloc ;unsafe impl Allocator for Kmalloc { unsafe fn realloc(
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 10/29 rust: alloc: add __GFP_NOWARN to `Flags`
In reply to Danilo KrummrichSome test cases in subsequent patches provoke allocation failures. Add __GFP_NOWARN to enable test cases to silence unpleasant warnings.
--- rust/bindings/bindings_helper.h | 1 + rust/kernel/alloc.rs | 5 +++++ 2 files changed, 6 insertions(+)
rust/bindings/bindings_helper.h +1 -0
@@ -31,4 +31,5 @@ const gfp_t RUST_CONST_HELPER_GFP_KERNEL_ACCOUNT = GFP_KERNEL_ACCOUNT;const gfp_t RUST_CONST_HELPER_GFP_NOWAIT =GFP_NOWAIT ;const gfp_t RUST_CONST_HELPER___GFP_ZERO = __GFP_ZERO;const gfp_t RUST_CONST_HELPER___GFP_HIGHMEM = ___GFP_HIGHMEM; +const gfp_t RUST_CONST_HELPER___GFP_NOWARN = ___GFP_NOWARN;const blk_features_t RUST_CONST_HELPER_BLK_FEAT_ROTATIONAL =BLK_FEAT_ROTATIONAL ;
rust/kernel/alloc.rs +5 -0
@@ -91,6 +91,11 @@ pub mod flags {/// use any filesystem callback. It is very likely to fail to allocate memory, even for very /// small allocations. pub const GFP_NOWAIT :Flags =Flags (bindings::GFP_NOWAIT ); + +/// Suppresses allocation failure reports. +/// +/// This is normally or'd with other flags. +pub const __GFP_NOWARN:Flags =Flags (bindings::__GFP_NOWARN); }/// The kernel's [`Allocator`] trait.
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 11/29 rust: alloc: implement kernel `Box`
In reply to Danilo KrummrichBox provides the simplest way to allocate memory for a generic type with one of the kernel's allocators, e.g. Kmalloc, Vmalloc or KVmalloc.
In contrast to Rust's Box type, the kernel Box type considers the kernel's GFP flags for all appropriate functions, always reports allocation failures through Result<_, AllocError> and remains independent from unstable features.
--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kbox.rs | 456 ++++++++++++++++++++++++++++++++++++++ rust/kernel/prelude.rs | 2 +- 3 files changed, 463 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/alloc/kbox.rs
rust/kernel/alloc.rs +6 -0
@@ -5,6 +5,7 @@#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod box_ext; +pub mod kbox;pub mod vec_ext;#[cfg(any(test, testlib))]
@@ -13,6 +14,11 @@#[cfg(any(test, testlib))] pub use self::allocator_testas allocator; +pub use self::kbox::Box ; +pub use self::kbox::KBox ; +pub use self::kbox::KVBox ; +pub use self::kbox::VBox ; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;
rust/kernel/alloc/kbox.rs new file +456 -0
456 lines not on this page · raw message
rust/kernel/prelude.rs +1 -1
@@ -14,7 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{box_ext::BoxExt , flags::*, vec_ext::VecExt }; +pub use crate ::alloc::{box_ext::BoxExt , flags::*, vec_ext::VecExt ,KBox ,KVBox ,VBox };#[doc(no_inline)] pub use alloc::{boxed::Box , vec::Vec };
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 12/29 rust: treewide: switch to our kernel `Box` type
In reply to Danilo KrummrichNow that we got the kernel Box type in place, convert all existing Box users to make use of it.
--- drivers/block/rnull.rs | 4 +-- rust/kernel/init.rs | 51 ++++++++++++++++--------------- rust/kernel/init/__internal.rs | 2 +- rust/kernel/rbtree.rs | 49 ++++++++++++++++------------- rust/kernel/sync/arc.rs | 17 +++++------ rust/kernel/sync/condvar.rs | 4 +-- rust/kernel/sync/lock/mutex.rs | 2 +- rust/kernel/sync/lock/spinlock.rs | 2 +- rust/kernel/workqueue.rs | 20 ++++++------ rust/macros/lib.rs | 6 ++-- 10 files changed, 81 insertions(+), 76 deletions(-)
drivers/block/rnull.rs +2 -2
@@ -32,7 +32,7 @@ }struct NullBlkModule { - _disk:Pin <Box <Mutex <GenDisk <NullBlkDevice >>>>, + _disk:Pin <KBox <Mutex <GenDisk <NullBlkDevice >>>>, }impl kernel::Module for NullBlkModule {
@@ -47,7 +47,7 @@ fn init(_module: &'static ThisModule) -> Result<Self> { .rotational(false) .build (format_args! ("rnullb{}" ,0 ), tagset)?; -let disk =Box ::pin_init (new_mutex! (disk,"nullb:disk" ), flags::GFP_KERNEL )?; +let disk =KBox ::pin_init (new_mutex! (disk,"nullb:disk" ), flags::GFP_KERNEL )?;Ok (Self { _disk: disk }) }
rust/kernel/init.rs +26 -25
178 lines not on this page · raw message
rust/kernel/init/__internal.rs +1 -1
@@ -105,7 +105,7 @@ fn make_closure<F, O, E>(self, f: F) -> F } } -pub struct AllData <T : ?Sized >(PhantomData <fn (Box <T >) ->Box <T >>); +pub struct AllData <T : ?Sized >(PhantomData <fn (KBox <T >) ->KBox <T >>);impl <T : ?Sized >Clone for AllData <T > {fn clone (&self) ->Self {
rust/kernel/rbtree.rs +27 -22
139 lines not on this page · raw message
rust/kernel/sync/arc.rs +8 -9
@@ -17,13 +17,12 @@//! [`Arc`]: https://doc.rust-lang.org/std/sync/struct.Arc.html use crate ::{ - alloc::{box_ext::BoxExt ,AllocError ,Flags }, + alloc::{AllocError ,Flags ,KBox }, bindings, init::{self,InPlaceInit ,Init ,PinInit }, try_init, types::{ForeignOwnable ,Opaque }, }; -use alloc::boxed::Box ;use core::{ alloc::Layout , fmt,
@@ -201,11 +200,11 @@ pub fn new(contents: T, flags: Flags) -> Result<Self, AllocError> { data: contents, }; -let inner = <Box <_ >as BoxExt <_ >>::new (value, flags)?; +let inner =KBox ::new (value, flags)?;// SAFETY: We just created `inner` with a reference count of 1, which is owned by the new // `Arc` object. -Ok (unsafe {Self ::from_inner (Box ::leak (inner).into ()) }) +Ok (unsafe {Self ::from_inner (KBox ::leak (inner).into ()) }) } }
@@ -398,8 +397,8 @@ fn drop(&mut self) {if is_zero {// The count reached zero, we must free the memory. // -// SAFETY: The pointer was initialised from the result of `Box::leak`. -unsafe {drop (Box ::from_raw (self.ptr.as_ptr ())) }; +// SAFETY: The pointer was initialised from the result of `KBox::leak`. +unsafe {drop (KBox ::from_raw (self.ptr.as_ptr ())) }; } } }
@@ -641,7 +640,7 @@ pub fn new(value: T, flags: Flags) -> Result<Self, AllocError> {/// Tries to allocate a new [`UniqueArc`] instance whose contents are not initialised yet. pub fn new_uninit (flags:Flags ) ->Result <UniqueArc <MaybeUninit <T >>,AllocError > {// INVARIANT: The refcount is initialised to a non-zero value. - let inner = Box::try_init::<AllocError>( + let inner = KBox::try_init::<AllocError>( try_init!(ArcInner { // SAFETY: There are no safety requirements for this FFI call. refcount: Opaque::new(unsafe { bindings::REFCOUNT_INIT(1) }),
@@ -651,8 +650,8 @@ pub fn new_uninit(flags: Flags) -> Result<UniqueArc<MaybeUninit<T>>, AllocError> )?;Ok (UniqueArc {// INVARIANT: The newly-created object has a refcount of 1. -// SAFETY: The pointer from the `Box` is valid. - inner:unsafe {Arc ::from_inner (Box ::leak (inner).into ()) }, +// SAFETY: The pointer from the `KBox` is valid. + inner:unsafe {Arc ::from_inner (KBox ::leak (inner).into ()) }, }) } }
rust/kernel/sync/condvar.rs +2 -2
@@ -70,8 +70,8 @@ macro_rules! new_condvar {/// } /// /// /// Allocates a new boxed `Example`. -/// fn new_example() -> Result<Pin<Box<Example>>> { -/// Box::pin_init(pin_init!(Example { +/// fn new_example() -> Result<Pin<KBox<Example>>> { +/// KBox::pin_init(pin_init!(Example { /// value <- new_mutex!(0), /// value_changed <- new_condvar!(), /// }), GFP_KERNEL)
rust/kernel/sync/lock/mutex.rs +1 -1
@@ -58,7 +58,7 @@ macro_rules! new_mutex {/// } /// /// // Allocate a boxed `Example`. -/// let e = Box::pin_init(Example::new(), GFP_KERNEL)?; +/// let e = KBox::pin_init(Example::new(), GFP_KERNEL)?; /// assert_eq!(e.c, 10); /// assert_eq!(e.d.lock().a, 20); /// assert_eq!(e.d.lock().b, 30);
rust/kernel/sync/lock/spinlock.rs +1 -1
@@ -56,7 +56,7 @@ macro_rules! new_spinlock {/// } /// /// // Allocate a boxed `Example`. -/// let e = Box::pin_init(Example::new(), GFP_KERNEL)?; +/// let e = KBox::pin_init(Example::new(), GFP_KERNEL)?; /// assert_eq!(e.c, 10); /// assert_eq!(e.d.lock().a, 20); /// assert_eq!(e.d.lock().b, 30);
rust/kernel/workqueue.rs +10 -10
@@ -216,7 +216,7 @@ pub fn try_spawn<T: 'static + Send + FnOnce()>( func: Some(func), }); - self.enqueue (Box ::pin_init (init, flags).map_err (|_|AllocError )?); + self.enqueue (KBox ::pin_init (init, flags).map_err (|_|AllocError )?);Ok (()) } }
@@ -239,9 +239,9 @@ fn project(self: Pin<&mut Self>) -> &mut Option<T> { }impl <T :FnOnce ()>WorkItem for ClosureWork <T > { -type Pointer =Pin <Box <Self >>; +type Pointer =Pin <KBox <Self >>; -fn run (mut this:Pin <Box <Self >>) { +fn run (mut this:Pin <KBox <Self >>) {if let Some (func) = this.as_mut ().project ().take () { (func)() }
@@ -297,7 +297,7 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput/// Defines the method that should be called directly when a work item is executed. /// -/// This trait is implemented by `Pin<Box<T>>` and [`Arc<T>`], and is mainly intended to be +/// This trait is implemented by `Pin<KBox<T>>` and [`Arc<T>`], and is mainly intended to be /// implemented for smart pointer types. For your own structs, you would implement [`WorkItem`] /// instead. The [`run`] method on this trait will usually just perform the appropriate /// `container_of` translation and then call into the [`run`][WorkItem::run] method from the
@@ -329,7 +329,7 @@ pub unsafe trait WorkItemPointer<const ID: u64>: RawWorkItem<ID> {/// This trait is used when the `work_struct` field is defined using the [`Work`] helper. pub trait WorkItem <const ID :u64 =0 > {/// The pointer type that this struct is wrapped in. This will typically be `Arc<Self>` or -/// `Pin<Box<Self>>`. +/// `Pin<KBox<Self>>`. type Pointer :WorkItemPointer <ID >;/// The method that should be called when this work item is executed.
@@ -567,7 +567,7 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput }// SAFETY: TODO. -unsafe impl <T ,const ID :u64 >WorkItemPointer <ID >for Pin <Box <T >> +unsafe impl <T ,const ID :u64 >WorkItemPointer <ID >for Pin <KBox <T >>where T :WorkItem <ID ,Pointer =Self >,T :HasWork <T ,ID >,
@@ -578,7 +578,7 @@ unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Pin<Box<T>>// SAFETY: This computes the pointer that `__enqueue` got from `Arc::into_raw`. let ptr =unsafe {T ::work_container_of (ptr) };// SAFETY: This pointer comes from `Arc::into_raw` and we've been given back ownership. -let boxed =unsafe {Box ::from_raw (ptr) }; +let boxed =unsafe {KBox ::from_raw (ptr) };// SAFETY: The box was already pinned when it was enqueued. let pinned =unsafe {Pin ::new_unchecked (boxed) };
@@ -587,7 +587,7 @@ unsafe impl<T, const ID: u64> WorkItemPointer<ID> for Pin<Box<T>> }// SAFETY: TODO. -unsafe impl <T ,const ID :u64 >RawWorkItem <ID >for Pin <Box <T >> +unsafe impl <T ,const ID :u64 >RawWorkItem <ID >for Pin <KBox <T >>where T :WorkItem <ID ,Pointer =Self >,T :HasWork <T ,ID >,
@@ -601,9 +601,9 @@ unsafe fn __enqueue<F>(self, queue_work_on: F) -> Self::EnqueueOutput// SAFETY: We're not going to move `self` or any of its fields, so its okay to temporarily // remove the `Pin` wrapper. let boxed =unsafe {Pin ::into_inner_unchecked (self) }; -let ptr =Box ::into_raw (boxed); +let ptr =KBox ::into_raw (boxed); -// SAFETY: Pointers into a `Box` point at a valid value. +// SAFETY: Pointers into a `KBox` point at a valid value. let work_ptr =unsafe {T ::raw_get_work (ptr) };// SAFETY: `raw_get_work` returns a pointer to a valid value. let work_ptr =unsafe {Work ::raw_get (work_ptr) };
rust/macros/lib.rs +3 -3
@@ -243,7 +243,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// struct DriverData { /// #[pin] /// queue: Mutex<Vec<Command>>, -/// buf: Box<[u8; 1024 * 1024]>, +/// buf: KBox<[u8; 1024 * 1024]>, /// } /// ``` ///
@@ -252,7 +252,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// struct DriverData { /// #[pin] /// queue: Mutex<Vec<Command>>, -/// buf: Box<[u8; 1024 * 1024]>, +/// buf: KBox<[u8; 1024 * 1024]>, /// raw_info: *mut Info, /// } ///
@@ -282,7 +282,7 @@ pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream {/// struct DriverData { /// #[pin] /// queue: Mutex<Vec<Command>>, -/// buf: Box<[u8; 1024 * 1024]>, +/// buf: KBox<[u8; 1024 * 1024]>, /// raw_info: *mut Info, /// } ///
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 13/29 rust: alloc: remove extension of std's `Box`
In reply to Danilo KrummrichNow that all existing Box users were moved to the kernel Box type, remove the BoxExt extension and all other related extensions.
--- rust/kernel/alloc.rs | 1 - rust/kernel/alloc/box_ext.rs | 89 ------------------------------------ rust/kernel/init.rs | 46 +------------------ rust/kernel/lib.rs | 1 - rust/kernel/prelude.rs | 4 +- rust/kernel/types.rs | 50 -------------------- 6 files changed, 3 insertions(+), 188 deletions(-) delete mode 100644 rust/kernel/alloc/box_ext.rs
rust/kernel/alloc.rs +0 -1
@@ -4,7 +4,6 @@#[cfg(not(any(test, testlib)))] pub mod allocator; -pub mod box_ext;pub mod kbox;pub mod vec_ext;
rust/kernel/alloc/box_ext.rs deleted +0 -89
89 lines not on this page · raw message
rust/kernel/init.rs +1 -45
75 lines not on this page · raw message
rust/kernel/lib.rs +0 -1
@@ -16,7 +16,6 @@#![feature(coerce_unsized)] #![feature(dispatch_from_dyn)] #![feature(lint_reasons)] -#![feature(new_uninit)] #![feature(unsize)] // Ensure conditional compilation based on the kernel configuration works;
rust/kernel/prelude.rs +2 -2
@@ -14,10 +14,10 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{box_ext::BoxExt , flags::*, vec_ext::VecExt ,KBox ,KVBox ,VBox }; +pub use crate ::alloc::{flags::*, vec_ext::VecExt ,KBox ,KVBox ,VBox };#[doc(no_inline)] -pub use alloc::{boxed::Box , vec::Vec }; +pub use alloc::vec::Vec ;#[doc(no_inline)] pub use macros::{module, pin_data, pinned_drop, vtable,Zeroable };
rust/kernel/types.rs +0 -50
67 lines not on this page · raw message
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 14/29 rust: alloc: add `Box` to prelude
In reply to Danilo KrummrichNow that we removed BoxExt and the corresponding includes in prelude.rs, add the new kernel Box type instead.
--- rust/kernel/prelude.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/prelude.rs +1 -1
@@ -14,7 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{flags::*, vec_ext::VecExt ,KBox ,KVBox ,VBox }; +pub use crate ::alloc::{flags::*, vec_ext::VecExt ,Box ,KBox ,KVBox ,VBox };#[doc(no_inline)] pub use alloc::vec::Vec ;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 15/29 rust: alloc: introduce `ArrayLayout`
In reply to Danilo KrummrichFrom: Benno Lossin <[email protected]>
When allocating memory for arrays using allocators, the Layout::array function is typically used. It returns a result, since the given size might be too big. However, Vec and its iterators store their allocated capacity and thus they already did check that the size is not too big.
The ArrayLayout type provides this exact behavior, as it can be infallibly converted into a Layout. Instead of a usize capacity, Vec and other similar array-storing types can use ArrayLayout instead.
--- rust/kernel/alloc.rs | 1 + rust/kernel/alloc/layout.rs | 91 +++++++++++++++++++++++++++++++++++++ 2 files changed, 92 insertions(+) create mode 100644 rust/kernel/alloc/layout.rs
rust/kernel/alloc.rs +1 -0
@@ -5,6 +5,7 @@#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod kbox; +pub mod layout;pub mod vec_ext;#[cfg(any(test, testlib))]
rust/kernel/alloc/layout.rs new file +91 -0
91 lines not on this page · raw message
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 16/29 rust: alloc: implement kernel `Vec` type
In reply to Danilo KrummrichVec provides a contiguous growable array type with contents allocated with the kernel's allocators (e.g. Kmalloc, Vmalloc or KVmalloc).
In contrast to Rust's stdlib Vec type, the kernel Vec type considers the kernel's GFP flags for all appropriate functions, always reports allocation failures through Result<_, AllocError> and remains independent from unstable features.
--- rust/kernel/alloc.rs | 6 + rust/kernel/alloc/kvec.rs | 650 ++++++++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + rust/kernel/prelude.rs | 2 +- 4 files changed, 658 insertions(+), 1 deletion(-) create mode 100644 rust/kernel/alloc/kvec.rs
rust/kernel/alloc.rs +6 -0
@@ -5,6 +5,7 @@#[cfg(not(any(test, testlib)))] pub mod allocator;pub mod kbox; +pub mod kvec;pub mod layout;pub mod vec_ext;
@@ -19,6 +20,11 @@pub use self::kbox::KVBox ;pub use self::kbox::VBox ; +pub use self::kvec::KVVec ; +pub use self::kvec::KVec ; +pub use self::kvec::VVec ; +pub use self::kvec::Vec ; +/// Indicates an allocation error. #[derive( Copy , Clone , PartialEq , Eq , Debug )] pub struct AllocError ;
rust/kernel/alloc/kvec.rs new file +650 -0
650 lines not on this page · raw message
rust/kernel/lib.rs +1 -0
@@ -17,6 +17,7 @@#![feature(dispatch_from_dyn)] #![feature(lint_reasons)] #![feature(unsize)] +#![feature(inline_const)] // Ensure conditional compilation based on the kernel configuration works; // otherwise we may silently break things like initcall handling.
rust/kernel/prelude.rs +1 -1
@@ -14,7 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{flags::*, vec_ext::VecExt ,Box ,KBox ,KVBox ,VBox }; +pub use crate ::alloc::{flags::*, vec_ext::VecExt ,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec };#[doc(no_inline)] pub use alloc::vec::Vec ;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 17/29 rust: alloc: implement `IntoIterator` for `Vec`
In reply to Danilo KrummrichImplement IntoIterator for Vec, Vec's IntoIter type, as well as Iterator for IntoIter.
Vec::into_iter disassembles the Vec into its raw parts; additionally, IntoIter keeps track of a separate pointer, which is incremented correspondingsly as the iterator advances, while the length, or the count of elements, is decremented.
This also means that IntoIter takes the ownership of the backing buffer and is responsible to drop the remaining elements and free the backing buffer, if it's dropped.
--- rust/kernel/alloc.rs | 1 + rust/kernel/alloc/kvec.rs | 170 ++++++++++++++++++++++++++++++++++++++ 2 files changed, 171 insertions(+)
rust/kernel/alloc.rs +1 -0
@@ -20,6 +20,7 @@pub use self::kbox::KVBox ;pub use self::kbox::VBox ; +pub use self::kvec::IntoIter ;pub use self::kvec::KVVec ;pub use self::kvec::KVec ;pub use self::kvec::VVec ;
rust/kernel/alloc/kvec.rs +170 -0
173 lines not on this page · raw message
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 18/29 rust: alloc: implement `collect` for `IntoIter`
In reply to Danilo KrummrichCurrently, we can't implement FromIterator. There are a couple of issues with this trait in the kernel, namely:
- Rust's specialization feature is unstable. This prevents us to optimze for the special case where I::IntoIter equals Vec's IntoIter type.
- We also can't use I::IntoIter's type ID either to work around this, since FromIterator doesn't require this type to be 'static.
- FromIterator::from_iter does return Self instead of Result<Self, AllocError>, hence we can't properly handle allocation failures.
- Neither Iterator::collect nor FromIterator::from_iter can handle additional allocation flags.
Instead, provide IntoIter::collect, such that we can at least convert IntoIter into a Vec again.
--- rust/kernel/alloc/kvec.rs | 94 +++++++++++++++++++++++++++++++++++++++ 1 file changed, 94 insertions(+)
rust/kernel/alloc/kvec.rs +94 -0
@@ -694,6 +694,100 @@ pub struct IntoIter<T, A: Allocator> { _p: PhantomData<A>, } +impl <T ,A >IntoIter <T ,A > +where +A :Allocator , +{ +fn into_raw_parts (self) -> (*mut T ,NonNull <T >,usize ,usize ) { +let me =ManuallyDrop ::new (self); +let ptr = me.ptr; +let buf = me.buf; +let len = me.len; +let cap = me.layout.len (); + (ptr, buf, len, cap) + } + +/// Same as `Iterator::collect` but specialized for `Vec`'s `IntoIter`. +/// +/// # Examples +/// +/// ``` +/// let v = kernel::kvec![1, 2, 3]?; +/// let mut it = v.into_iter(); +/// +/// assert_eq!(it.next(), Some(1)); +/// +/// let v = it.collect(GFP_KERNEL); +/// assert_eq!(v, [2, 3]); +/// +/// # Ok::<(), Error>(()) +/// ``` +/// # Implementation Details +/// +/// Currently, we can't implement `FromIterator`. There are a couple of issues with this trait +/// in the kernel, namely: +/// +/// - Rust's specialization feature is unstable. This prevents us to optimze for the special +/// case where `I::IntoIter` equals `Vec`'s `IntoIter` type. +/// - We also can't use `I::IntoIter`'s type ID either to work around this, since `FromIterator` +/// doesn't require this type to be `'static`. +/// - `FromIterator::from_iter` does return `Self` instead of `Result<Self, AllocError>`, hence +/// we can't properly handle allocation failures. +/// - Neither `Iterator::collect` nor `FromIterator::from_iter` can handle additional allocation +/// flags. +/// +/// Instead, provide `IntoIter::collect`, such that we can at least convert a `IntoIter` into a +/// `Vec` again. +/// +/// Note that `IntoIter::collect` doesn't require `Flags`, since it re-uses the existing backing +/// buffer. However, this backing buffer may be shrunk to the actual count of elements. +pub fn collect (self, flags:Flags ) ->Vec <T ,A > { +let old_layout = self.layout; +let (mut ptr, buf, len,mut cap) = self.into_raw_parts (); +let has_advanced = ptr != buf.as_ptr (); + +if has_advanced { +// Copy the contents we have advanced to at the beginning of the buffer. +// +// SAFETY: +// - `ptr` is valid for reads of `len * size_of::<T>()` bytes, +// - `buf.as_ptr()` is valid for writes of `len * size_of::<T>()` bytes, +// - `ptr` and `buf.as_ptr()` are not be subject to aliasing restrictions relative to +// each other, +// - both `ptr` and `buf.ptr()` are properly aligned. +unsafe { ptr::copy (ptr, buf.as_ptr (), len) }; + ptr = buf.as_ptr (); + +// SAFETY: `len` is guaranteed to be smaller than `self.layout.len()`. +let layout =unsafe {ArrayLayout ::<T >::new_unchecked (len) }; + +// SAFETY: `buf` points to the start of the backing buffer and `len` is guaranteed to be +// smaller than `cap`. Depending on `alloc` this operation may shrink the buffer or leaves +// it as it is. + ptr =match unsafe { +A ::realloc (Some (buf.cast ()), layout.into (), old_layout.into (), flags) + } { +// If we fail to shrink, which likely can't even happen, continue with the existing +// buffer. +Err (_) => ptr, +Ok (ptr) => { + cap = len; + ptr.as_ptr ().cast () + } + }; + } + +// SAFETY: If the iterator has been advanced, the advanced elements have been copied to +// the beginning of the buffer and `len` has been adjusted accordingly. +// +// - `ptr` is guaranteed to point to the start of the backing buffer. +// - `cap` is either the original capacity or, after shrinking the buffer, equal to `len`. +// - `alloc` is guaranteed to be unchanged since `into_iter` has been called on the original +// `Vec`. +unsafe {Vec ::from_raw_parts (ptr, len, cap) } + } +} +impl <T ,A >Iterator for IntoIter <T ,A >where A :Allocator ,
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 19/29 rust: treewide: switch to the kernel `Vec` type
In reply to Danilo KrummrichNow that we got the kernel Vec in place, convert all existing Vec users to make use of it.
--- rust/kernel/str.rs | 12 +++++------- rust/kernel/sync/locked_by.rs | 2 +- rust/kernel/types.rs | 2 +- rust/kernel/uaccess.rs | 17 +++++++---------- rust/macros/lib.rs | 6 +++--- samples/rust/rust_minimal.rs | 4 ++-- 6 files changed, 19 insertions(+), 24 deletions(-)
rust/kernel/str.rs +5 -7
@@ -2,8 +2,7 @@//! String representations. -use crate ::alloc::{flags::*, vec_ext::VecExt ,AllocError }; -use alloc::vec::Vec ; +use crate ::alloc::{flags::*,AllocError ,KVec };use core::fmt::{self,Write };use core::ops::{self,Deref ,DerefMut ,Index };
@@ -791,7 +790,7 @@ fn write_str(&mut self, s: &str) -> fmt::Result {/// assert_eq!(s.is_ok(), false); /// ``` pub struct CString { - buf:Vec <u8 >, + buf:KVec <u8 >, }impl CString {
@@ -804,7 +803,7 @@ pub fn try_from_fmt(args: fmt::Arguments<'_>) -> Result<Self, Error> {let size = f.bytes_written ();// Allocate a vector with the required number of bytes, and write to it. -let mut buf = <Vec <_ >as VecExt <_ >>::with_capacity (size,GFP_KERNEL )?; +let mut buf =KVec ::with_capacity (size,GFP_KERNEL )?;// SAFETY: The buffer stored in `buf` is at least of size `size` and is valid for writes. let mut f =unsafe {Formatter ::from_buffer (buf.as_mut_ptr (), size) }; f.write_fmt (args)?;
@@ -851,10 +850,9 @@ impl<'a> TryFrom<&'a CStr> for CString {type Error =AllocError ;fn try_from (cstr: &'aCStr ) ->Result <CString ,AllocError > { -let mut buf =Vec ::new (); +let mut buf =KVec ::new (); - <Vec <_ >as VecExt <_ >>::extend_from_slice (&mut buf, cstr.as_bytes_with_nul (),GFP_KERNEL ) - .map_err (|_|AllocError )?; + buf.extend_from_slice (cstr.as_bytes_with_nul (),GFP_KERNEL )?;// INVARIANT: The `CStr` and `CString` types have the same invariants for // the string data, and we copied it over without changes.
rust/kernel/sync/locked_by.rs +1 -1
@@ -43,7 +43,7 @@/// struct InnerDirectory { /// /// The sum of the bytes used by all files. /// bytes_used: u64, -/// _files: Vec<File>, +/// _files: KVec<File>, /// } /// /// struct Directory {
rust/kernel/types.rs +1 -1
@@ -135,7 +135,7 @@ unsafe fn from_foreign(_: *const core::ffi::c_void) -> Self {}/// # use kernel::types::ScopeGuard; /// fn example3(arg: bool) -> Result { /// let mut vec = -/// ScopeGuard::new_with_data(Vec::new(), |v| pr_info!("vec had {} elements\n", v.len())); +/// ScopeGuard::new_with_data(KVec::new(), |v| pr_info!("vec had {} elements\n", v.len())); /// /// vec.push(10u8, GFP_KERNEL)?; /// if arg {
rust/kernel/uaccess.rs +7 -10
@@ -11,7 +11,6 @@ prelude::*, types::{AsBytes, FromBytes}, }; -use alloc::vec::Vec ;use core::ffi::{c_ulong, c_void};use core::mem::{size_of,MaybeUninit };
@@ -46,7 +45,6 @@/// every byte in the region. /// /// ```no_run -/// use alloc::vec::Vec; /// use core::ffi::c_void; /// use kernel::error::Result; /// use kernel::uaccess::{UserPtr, UserSlice};
@@ -54,7 +52,7 @@/// fn bytes_add_one(uptr: UserPtr, len: usize) -> Result<()> { /// let (read, mut write) = UserSlice::new(uptr, len).reader_writer(); /// -/// let mut buf = Vec::new(); +/// let mut buf = KVec::new(); /// read.read_all(&mut buf, GFP_KERNEL)?; /// /// for b in &mut buf {
@@ -69,7 +67,6 @@/// Example illustrating a TOCTOU (time-of-check to time-of-use) bug. /// /// ```no_run -/// use alloc::vec::Vec; /// use core::ffi::c_void; /// use kernel::error::{code::EINVAL, Result}; /// use kernel::uaccess::{UserPtr, UserSlice};
@@ -78,21 +75,21 @@/// fn is_valid(uptr: UserPtr, len: usize) -> Result<bool> { /// let read = UserSlice::new(uptr, len).reader(); /// -/// let mut buf = Vec::new(); +/// let mut buf = KVec::new(); /// read.read_all(&mut buf, GFP_KERNEL)?; /// /// todo!() /// } /// /// /// Returns the bytes behind this user pointer if they are valid. -/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<Vec<u8>> { +/// fn get_bytes_if_valid(uptr: UserPtr, len: usize) -> Result<KVec<u8>> { /// if !is_valid(uptr, len)? { /// return Err(EINVAL); /// } /// /// let read = UserSlice::new(uptr, len).reader(); /// -/// let mut buf = Vec::new(); +/// let mut buf = KVec::new(); /// read.read_all(&mut buf, GFP_KERNEL)?; /// /// // THIS IS A BUG! The bytes could have changed since we checked them.
@@ -130,7 +127,7 @@ pub fn new(ptr: UserPtr, length: usize) -> Self {/// Reads the entirety of the user slice, appending it to the end of the provided buffer. /// /// Fails with [`EFAULT`] if the read happens on a bad address. -pub fn read_all (self, buf: &mut Vec <u8 >, flags:Flags ) ->Result { +pub fn read_all (self, buf: &mut KVec <u8 >, flags:Flags ) ->Result { self.reader ().read_all (buf, flags) }
@@ -291,9 +288,9 @@ pub fn read<T: FromBytes>(&mut self) -> Result<T> {/// Reads the entirety of the user slice, appending it to the end of the provided buffer. /// /// Fails with [`EFAULT`] if the read happens on a bad address. -pub fn read_all (mut self, buf: &mut Vec <u8 >, flags:Flags ) ->Result { +pub fn read_all (mut self, buf: &mut KVec <u8 >, flags:Flags ) ->Result {let len = self.length; -VecExt ::<u8 >::reserve (buf, len, flags)?; + buf.reserve (len, flags)?;// The call to `try_reserve` was successful, so the spare capacity is at least `len` bytes // long.
rust/macros/lib.rs +3 -3
@@ -242,7 +242,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// #[pin_data] /// struct DriverData { /// #[pin] -/// queue: Mutex<Vec<Command>>, +/// queue: Mutex<KVec<Command>>, /// buf: KBox<[u8; 1024 * 1024]>, /// } /// ```
@@ -251,7 +251,7 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// #[pin_data(PinnedDrop)] /// struct DriverData { /// #[pin] -/// queue: Mutex<Vec<Command>>, +/// queue: Mutex<KVec<Command>>, /// buf: KBox<[u8; 1024 * 1024]>, /// raw_info: *mut Info, /// }
@@ -281,7 +281,7 @@ pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream {/// #[pin_data(PinnedDrop)] /// struct DriverData { /// #[pin] -/// queue: Mutex<Vec<Command>>, +/// queue: Mutex<KVec<Command>>, /// buf: KBox<[u8; 1024 * 1024]>, /// raw_info: *mut Info, /// }
samples/rust/rust_minimal.rs +2 -2
@@ -13,7 +13,7 @@ }struct RustMinimal { - numbers:Vec <i32 >, + numbers:KVec <i32 >, }impl kernel::Module for RustMinimal {
@@ -21,7 +21,7 @@ fn init(_module: &'static ThisModule) -> Result<Self> {pr_info! ("Rust minimal sample (init)\n" );pr_info! ("Am I built-in? {}\n" , !cfg! (MODULE )); -let mut numbers =Vec ::new (); +let mut numbers =KVec ::new (); numbers.push (72 ,GFP_KERNEL )?; numbers.push (108 ,GFP_KERNEL )?; numbers.push (200 ,GFP_KERNEL )?;
Signature · 1 line
2.46.1
Alice Ryhl
maintainerRe: [PATCH v9 19/29] rust: treewide: switch to the kernel `Vec` type
Quoted 8 lines · Danilo Krummrich
Now that we got the kernel
Vecin place, convert all existingVecusers to make use of it.Reviewed-by Alice Ryhl <[email protected]>Reviewed-by Benno Lossin <[email protected]>Reviewed-by Gary Guo <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>
This is missing the mm/kasan/kasan_test_rust.rs file, which was added in v6.12-rc1.
Alice
Danilo Krummrich
maintainerRe: [PATCH v9 19/29] rust: treewide: switch to the kernel `Vec` type
Quoted 5 lines · Alice Ryhl
Quoted 4 lines · Danilo Krummrich
Now that we got the kernel
Vecin place, convert all existingVecusers to make use of it.
Quoted 7 lines
Quoted 4 lines
Reviewed-by Alice Ryhl <[email protected]>Reviewed-by Benno Lossin <[email protected]>Reviewed-by Gary Guo <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>This is missing the
mm/kasan/kasan_test_rust.rsfile, which was added in v6.12-rc1.
Thanks Alice for pointing this out.
mm/kasan/kasan_test_rust.rs +1 -1
@@ -11,7 +11,7 @@/// drop the vector, and touch it. #[no_mangle] pub extern "C" fn kasan_test_rust_uaf () ->u8 { -let mut v:Vec <u8 > =Vec ::new (); +let mut v:KVec <u8 > =KVec ::new ();for _in 0 ..4096 { v.push (0x42 ,GFP_KERNEL ).unwrap (); }
@Miguel: Can you please fix this up when you apply the series?
--
Unrelated to this series, but I think there's also a "regression" against the lint series in this file:
warning: unsafe block missing a safety comment
--> mm/kasan/kasan_test_rust.rs:20:5
|
20 | unsafe { *ptr }
| ^^^^^^^^^^^^^^^
| = help: consider adding a safety comment on the preceding line
= help: for further information visit https://rust-lang.github.io/rust-clippy/master/index.html#undocumented_unsafe_blocks
= note: requested on the command line with -W clippy::undocumented-unsafe-blocks
warning: 1 warning emitted
Re: [PATCH v9 19/29] rust: treewide: switch to the kernel `Vec` type
Quoted 2 lines · Danilo Krummrich
@Miguel: Can you please fix this up when you apply the series?
Of course.
Quoted 2 lines
Unrelated to this series, but I think there's also a "regression" against the lint series in this file:
Good catch, fixed in rust-next directly and added to my tests, thanks!
mm/kasan/kasan_test_rust.rs +1 -0
@@ -17,5 +17,6 @@ pub extern "C" fn kasan_test_rust_uaf() -> u8 { }let ptr: *mut u8 =addr_of_mut! (v[2048 ]);drop (v); +// SAFETY: Incorrect, on purpose. unsafe { *ptr } }
Cheers,
Miguel
Danilo Krummrich
maintainerPATCH v9 20/29 rust: alloc: remove `VecExt` extension
In reply to Danilo KrummrichNow that all existing Vec users were moved to the kernel Vec type, remove the VecExt extension.
--- rust/kernel/alloc.rs | 1 - rust/kernel/alloc/vec_ext.rs | 185 ----------------------------------- rust/kernel/prelude.rs | 5 +- 3 files changed, 1 insertion(+), 190 deletions(-) delete mode 100644 rust/kernel/alloc/vec_ext.rs
rust/kernel/alloc.rs +0 -1
@@ -7,7 +7,6 @@pub mod kbox;pub mod kvec;pub mod layout; -pub mod vec_ext;#[cfg(any(test, testlib))] pub mod allocator_test;
rust/kernel/alloc/vec_ext.rs deleted +0 -185
185 lines not on this page · raw message
rust/kernel/prelude.rs +1 -4
@@ -14,10 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{flags::*, vec_ext::VecExt ,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec }; - -#[doc(no_inline)] -pub use alloc::vec::Vec ; +pub use crate ::alloc::{flags::*,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec };#[doc(no_inline)] pub use macros::{module, pin_data, pinned_drop, vtable,Zeroable };
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 21/29 rust: alloc: add `Vec` to prelude
In reply to Danilo KrummrichNow that we removed VecExt and the corresponding includes in prelude.rs, add the new kernel Vec type instead.
--- rust/kernel/prelude.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/prelude.rs +1 -1
@@ -14,7 +14,7 @@#[doc(no_inline)] pub use core::pin::Pin ; -pub use crate ::alloc::{flags::*,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec }; +pub use crate ::alloc::{flags::*,Box ,KBox ,KVBox ,KVVec ,KVec ,VBox ,VVec ,Vec };#[doc(no_inline)] pub use macros::{module, pin_data, pinned_drop, vtable,Zeroable };
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 22/29 rust: error: use `core::alloc::LayoutError`
In reply to Danilo KrummrichUse core::alloc::LayoutError instead of alloc::alloc::LayoutError in preparation to get rid of Rust's alloc crate.
--- rust/kernel/error.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/error.rs +1 -1
@@ -6,7 +6,7 @@use crate ::{alloc::AllocError , str::CStr }; -use alloc::alloc::LayoutError ; +use core::alloc::LayoutError ;use core::fmt;use core::num::TryFromIntError ;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 23/29 rust: error: check for config `test` in `Error::name`
In reply to Danilo KrummrichAdditional to testlib also check for test in Error::name. This is required by a subsequent patch that (indirectly) uses Error in test cases.
--- rust/kernel/error.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
rust/kernel/error.rs +2 -2
@@ -142,7 +142,7 @@ pub fn to_ptr<T>(self) -> *mut T { }/// Returns a string representing the error, if one exists. -#[cfg(not(testlib))] +#[cfg(not(any(test, testlib)))] pub fn name (&self) ->Option <&'staticCStr > {// SAFETY: Just an FFI call, there are no extra safety requirements. let ptr =unsafe { bindings::errname (-self.0 ) };
@@ -159,7 +159,7 @@ pub fn name(&self) -> Option<&'static CStr> {/// When `testlib` is configured, this always returns `None` to avoid the dependency on a /// kernel function so that tests that use this (e.g., by calling [`Result::unwrap`]) can still /// run in userspace. -#[cfg(testlib)] +#[cfg(any(test, testlib))] pub fn name (&self) ->Option <&'staticCStr > {None }
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 24/29 rust: alloc: implement `contains` for `Flags`
In reply to Danilo KrummrichProvide a simple helper function to check whether given flags do contain one or multiple other flags.
This is used by a subsequent patch implementing the Cmalloc Allocator to check for __GFP_ZERO.
--- rust/kernel/alloc.rs | 7 ++++++- 1 file changed, 6 insertions(+), 1 deletion(-)
rust/kernel/alloc.rs +6 -1
@@ -35,7 +35,7 @@/// They can be combined with the operators `|`, `&`, and `!`. /// /// Values can be used from the [`flags`] module. -#[derive( Clone , Copy )] +#[derive( Clone , Copy , PartialEq )] pub struct Flags (u32 );impl Flags {
@@ -43,6 +43,11 @@ impl Flags {pub (crate )fn as_raw (self) ->u32 { self.0 } + +/// Check whether `flags` is contained in `self`. +pub fn contains (self, flags:Flags ) ->bool { + (self & flags) == flags + } }impl core::ops::BitOr for Flags {
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 25/29 rust: alloc: implement `Cmalloc` in module allocator_test
In reply to Danilo KrummrichSo far the kernel's Box and Vec types can't be used by userspace test cases, since all users of those types (e.g. CString) use kernel allocators for instantiation.
In order to allow userspace test cases to make use of such types as well, implement the Cmalloc allocator within the allocator_test module and type alias all kernel allocators to Cmalloc. The Cmalloc allocator uses libc's realloc() function as allocator backend.
--- rust/kernel/alloc/allocator_test.rs | 88 ++++++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 7 deletions(-)
rust/kernel/alloc/allocator_test.rs +81 -7
@@ -1,21 +1,95 @@// SPDX-License-Identifier: GPL-2.0 +//! So far the kernel's `Box` and `Vec` types can't be used by userspace test cases, since all users +//! of those types (e.g. `CString`) use kernel allocators for instantiation. +//! +//! In order to allow userspace test cases to make use of such types as well, implement the +//! `Cmalloc` allocator within the allocator_test module and type alias all kernel allocators to +//! `Cmalloc`. The `Cmalloc` allocator uses libc's realloc() function as allocator backend. +#![allow(missing_docs)] -use super ::{AllocError ,Allocator ,Flags }; +use super ::{flags::*,AllocError ,Allocator ,Flags };use core::alloc::Layout ; +use core::cmp; +use core::ptr;use core::ptr::NonNull ; -pub struct Kmalloc ; +/// The userspace allocator based on libc. +pub struct Cmalloc ; + +pub type Kmalloc =Cmalloc ;pub type Vmalloc =Kmalloc ;pub type KVmalloc =Kmalloc ; -unsafe impl Allocator for Kmalloc { +extern "C" { +#[link_name = "aligned_alloc" ] +fn libc_aligned_alloc (align:usize , size:usize ) -> *mut core::ffi::c_void ; + +#[link_name = "free" ] +fn libc_free (ptr: *mut core::ffi::c_void ); +} + +// SAFETY: +// - memory remains valid until it is explicitly freed, +// - passing a pointer to a valid memory allocation created by this `Allocator` is always OK, +// - `realloc` provides the guarantees as provided in the `# Guarantees` section. +unsafe impl Allocator for Cmalloc {unsafe fn realloc ( - _ptr:Option <NonNull <u8 >>, - _layout:Layout , - _flags:Flags , + ptr:Option <NonNull <u8 >>, + layout:Layout , + old_layout:Layout , + flags:Flags , ) ->Result <NonNull <[u8 ]>,AllocError > { -panic! (); +let src =match ptr { +Some (src) => { +if old_layout.size () ==0 { + ptr::null_mut () + }else { + src.as_ptr () + } + } +None => ptr::null_mut (), + }; + +if layout.size () ==0 { +// SAFETY: `src` is either NULL or was previously allocated with this `Allocator` +unsafe {libc_free (src.cast ()) }; + +return Ok (NonNull ::slice_from_raw_parts ( +crate ::alloc::dangling_from_layout (layout), +0 , + )); + } + +// SAFETY: Returns either NULL or a pointer to a memory allocation that satisfies or +// exceeds the given size and alignment requirements. +let dst =unsafe {libc_aligned_alloc (layout.align (), layout.size ()) }as *mut u8 ; +let dst =NonNull ::new (dst).ok_or (AllocError )?; + +if flags.contains (__GFP_ZERO) { +// SAFETY: The preceeding calls to `libc_aligned_alloc` and `NonNull::new` +// guarantee that `dst` points to memory of at least `layout.size()` bytes. +unsafe { dst.as_ptr ().write_bytes (0 , layout.size ()) }; + } + +if !src.is_null () { +// SAFETY: +// - `src` has previously been allocated with this `Allocator`; `dst` has just been +// newly allocated, hence the memory regions do not overlap. +// - both` src` and `dst` are properly aligned and valid for reads and writes +unsafe { + ptr::copy_nonoverlapping ( + src, + dst.as_ptr (), + cmp::min (layout.size (), old_layout.size ()), + ) + }; + } + +// SAFETY: `src` is either NULL or was previously allocated with this `Allocator` +unsafe {libc_free (src.cast ()) }; + +Ok (NonNull ::slice_from_raw_parts (dst, layout.size ())) } }
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerRe: [PATCH v9 25/29] rust: alloc: implement `Cmalloc` in module allocator_test
Quoted 9 lines · Danilo Krummrich
So far the kernel's
BoxandVectypes can't be used by userspace test cases, since all users of those types (e.g.CString) use kernel allocators for instantiation.In order to allow userspace test cases to make use of such types as well, implement the
Cmallocallocator within theallocator_testmodule and type alias all kernel allocators toCmalloc. TheCmallocallocator uses libc'srealloc()function as allocator backend.
Quoted 7 lines
Reviewed-by Benno Lossin <[email protected]>Reviewed-by Gary Guo <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>--- rust/kernel/alloc/allocator_test.rs | 88 ++++++++++++++++++++++++++--- 1 file changed, 81 insertions(+), 7 deletions(-)
Here we can remove it again:
rust/kernel/alloc.rs +0 -1
@@ -215,7 +215,6 @@ unsafe fn free(ptr: NonNull<u8>, layout: Layout) { } } -#[allow(dead_code)] /// Returns a properly aligned dangling pointer from the given `layout`. pub (crate )fn dangling_from_layout (layout:Layout ) ->NonNull <u8 > {let ptr = layout.align ()as *mut u8 ;
@Miguel: I pushed [1], which is already rebased to rust-next (-rc2) and contains the fixups.
[1] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=rust/mm
Danilo Krummrich
maintainerPATCH v9 26/29 rust: str: test: replace `alloc::format`
In reply to Danilo KrummrichThe current implementation of tests in str.rs use format! to format strings for comparison, which, internally, creates a new String.
In order to prepare for getting rid of Rust's alloc crate, we have to cut this dependency. Instead, implement format! for CString.
Note that for userspace tests, Kmalloc, which is backing CString's memory, is just a type alias to Cmalloc.
--- rust/kernel/str.rs | 23 ++++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-)
rust/kernel/str.rs +22 -1
@@ -524,7 +524,28 @@ macro_rules! c_str {#[cfg(test)] mod tests {use super ::*; -use alloc::format; + +struct String (CString ); + +impl String { +fn from_fmt (args: fmt::Arguments <'_>) ->Self { +String (CString ::try_from_fmt (args).unwrap ()) + } + } + +impl Deref for String { +type Target =str ; + +fn deref (&self) -> &str { + self.0 .to_str ().unwrap () + } + } + +macro_rules! format { + ($($f:tt)*) => ({ + &*String ::from_fmt (kernel::fmt!($($f)*)) + }) + }const ALL_ASCII_CHARS : &'staticstr ="\\x01\\x02\\x03\\x04\\x05\\x06\\x07\\x08\\x09\\x0a\\x0b\\x0c\\x0d\\x0e\\x0f\
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 27/29 rust: alloc: update module comment of alloc.rs
In reply to Danilo KrummrichBefore we remove Rust's alloc crate, rewrite the module comment in alloc.rs to avoid a rustdoc warning.
Besides that, the module comment in alloc.rs isn't correct anymore, we're no longer extending Rust's alloc crate.
--- rust/kernel/alloc.rs | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-)
rust/kernel/alloc.rs +1 -1
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -//! Extensions to the [`alloc`] crate. +//! Implementation of the kernel's memory allocation infrastructure. #[cfg(not(any(test, testlib)))] pub mod allocator;
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 28/29 kbuild: rust: remove the `alloc` crate and `GlobalAlloc`
In reply to Danilo KrummrichNow that we have our own Allocator, Box and Vec types we can remove Rust's alloc crate and the new_uninit unstable feature.
Also remove Kmalloc's GlobalAlloc implementation -- we can't remove this in a separate patch, since the alloc crate requires a #[global_allocator] to set, that implements GlobalAlloc.
--- rust/Makefile | 43 +++++--------------- rust/exports.c | 1 - rust/kernel/alloc/allocator.rs | 65 +------------------------------ scripts/Makefile.build | 4 +- scripts/generate_rust_analyzer.py | 11 +----- 5 files changed, 16 insertions(+), 108 deletions(-)
rust/Makefile +10 -33
@@ -15,8 +15,8 @@ always-$(CONFIG_RUST) += libmacros.so no-clean-files += libmacros.so always-$(CONFIG_RUST) += bindings/bindings_generated.rs bindings/bindings_helpers_generated.rs -obj-$(CONFIG_RUST) += alloc.o bindings.o kernel.o -always-$(CONFIG_RUST) += exports_alloc_generated.h exports_helpers_generated.h \ +obj-$(CONFIG_RUST) += bindings.o kernel.o +always-$(CONFIG_RUST) += exports_helpers_generated.h \ exports_bindings_generated.h exports_kernel_generated.h always-$(CONFIG_RUST) += uapi/uapi_generated.rs
@@ -53,11 +53,6 @@ endif core-cfgs = \ --cfg no_fp_fmt_parse -alloc-cfgs = \ - --cfg no_global_oom_handling \ - --cfg no_rc \ - --cfg no_sync - quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $< cmd_rustdoc = \ OBJTREE=$(abspath $(objtree)) \
@@ -81,7 +76,7 @@ quiet_cmd_rustdoc = RUSTDOC $(if $(rustdoc_host),H, ) $< # command-like flags to solve the issue. Meanwhile, we use the non-custom case # and then retouch the generated files. rustdoc: rustdoc-core rustdoc-macros rustdoc-compiler_builtins \ - rustdoc-alloc rustdoc-kernel + rustdoc-kernel $(Q)cp $(srctree)/Documentation/images/logo.svg $(rustdoc_output)/static.files/ $(Q)cp $(srctree)/Documentation/images/COPYING-logo $(rustdoc_output)/static.files/ $(Q)find $(rustdoc_output) -name '*.html' -type f -print0 | xargs -0 sed -Ei \
@@ -108,20 +103,11 @@ rustdoc-core: $(RUST_LIB_SRC)/core/src/lib.rs FORCE rustdoc-compiler_builtins: $(src)/compiler_builtins.rs rustdoc-core FORCE +$(call if_changed,rustdoc) -# We need to allow `rustdoc::broken_intra_doc_links` because some -# `no_global_oom_handling` functions refer to non-`no_global_oom_handling` -# functions. Ideally `rustdoc` would have a way to distinguish broken links -# due to things that are "configured out" vs. entirely non-existing ones. -rustdoc-alloc: private rustc_target_flags = $(alloc-cfgs) \ - -Arustdoc::broken_intra_doc_links -rustdoc-alloc: $(RUST_LIB_SRC)/alloc/src/lib.rs rustdoc-core rustdoc-compiler_builtins FORCE - +$(call if_changed,rustdoc) - -rustdoc-kernel: private rustc_target_flags = --extern alloc \ +rustdoc-kernel: private rustc_target_flags = \ --extern build_error --extern macros=$(objtree)/$(obj)/libmacros.so \ --extern bindings --extern uapi rustdoc-kernel: $(src)/kernel/lib.rs rustdoc-core rustdoc-macros \ - rustdoc-compiler_builtins rustdoc-alloc $(obj)/libmacros.so \ + rustdoc-compiler_builtins $(obj)/libmacros.so \ $(obj)/bindings.o FORCE +$(call if_changed,rustdoc)
@@ -165,7 +151,7 @@ quiet_cmd_rustdoc_test_kernel = RUSTDOC TK $< mkdir -p $(objtree)/$(obj)/test/doctests/kernel; \ OBJTREE=$(abspath $(objtree)) \ $(RUSTDOC) --test $(rust_flags) \ - -L$(objtree)/$(obj) --extern alloc --extern kernel \ + -L$(objtree)/$(obj) --extern kernel \ --extern build_error --extern macros \ --extern bindings --extern uapi \ --no-run --crate-name kernel -Zunstable-options \
@@ -201,7 +187,7 @@ rusttest-macros: $(src)/macros/lib.rs FORCE +$(call if_changed,rustc_test) +$(call if_changed,rustdoc_test) -rusttest-kernel: private rustc_target_flags = --extern alloc \ +rusttest-kernel: private rustc_target_flags = \ --extern build_error --extern macros --extern bindings --extern uapi rusttest-kernel: $(src)/kernel/lib.rs \ rusttestlib-build_error rusttestlib-macros rusttestlib-bindings \
@@ -314,9 +300,6 @@ quiet_cmd_exports = EXPORTS $@ $(obj)/exports_core_generated.h: $(obj)/core.o FORCE $(call if_changed,exports) -$(obj)/exports_alloc_generated.h: $(obj)/alloc.o FORCE - $(call if_changed,exports) - # Even though Rust kernel modules should never use the bindings directly, # symbols from the `bindings` crate and the C helpers need to be exported # because Rust generics and inlined functions may not get their code generated
@@ -363,7 +346,7 @@ quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L rust-analyzer: $(Q)$(srctree)/scripts/generate_rust_analyzer.py \ - --cfgs='core=$(core-cfgs)' --cfgs='alloc=$(alloc-cfgs)' \ + --cfgs='core=$(core-cfgs)' \ $(realpath $(srctree)) $(realpath $(objtree)) \ $(rustc_sysroot) $(RUST_LIB_SRC) $(KBUILD_EXTMOD) > \ $(if $(KBUILD_EXTMOD),$(extmod_prefix),$(objtree))/rust-project.json
@@ -401,12 +384,6 @@ $(obj)/compiler_builtins.o: private rustc_objcopy = -w -W '__*' $(obj)/compiler_builtins.o: $(src)/compiler_builtins.rs $(obj)/core.o FORCE +$(call if_changed_rule,rustc_library) -$(obj)/alloc.o: private skip_clippy = 1 -$(obj)/alloc.o: private skip_flags = -Wunreachable_pub -$(obj)/alloc.o: private rustc_target_flags = $(alloc-cfgs) -$(obj)/alloc.o: $(RUST_LIB_SRC)/alloc/src/lib.rs $(obj)/compiler_builtins.o FORCE - +$(call if_changed_rule,rustc_library) - $(obj)/build_error.o: $(src)/build_error.rs $(obj)/compiler_builtins.o FORCE +$(call if_changed_rule,rustc_library)
@@ -421,9 +398,9 @@ $(obj)/uapi.o: $(src)/uapi/lib.rs \ $(obj)/uapi/uapi_generated.rs FORCE +$(call if_changed_rule,rustc_library) -$(obj)/kernel.o: private rustc_target_flags = --extern alloc \ +$(obj)/kernel.o: private rustc_target_flags = \ --extern build_error --extern macros --extern bindings --extern uapi -$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/alloc.o $(obj)/build_error.o \ +$(obj)/kernel.o: $(src)/kernel/lib.rs $(obj)/build_error.o \ $(obj)/libmacros.so $(obj)/bindings.o $(obj)/uapi.o FORCE +$(call if_changed_rule,rustc_library)
rust/exports.c +0 -1
@@ -16,7 +16,6 @@#define EXPORT_SYMBOL_RUST_GPL (sym) extern int sym; EXPORT_SYMBOL_GPL(sym)#include "exports_core_generated.h" -#include "exports_alloc_generated.h" #include "exports_helpers_generated.h" #include "exports_bindings_generated.h" #include "exports_kernel_generated.h"
rust/kernel/alloc/allocator.rs +2 -63
86 lines not on this page · raw message
scripts/Makefile.build +2 -2
@@ -248,7 +248,7 @@ $(obj)/%.lst: $(obj)/%.c FORCE # Compile Rust sources (.rs) # --------------------------------------------------------------------------- -rust_allowed_features := arbitrary_self_types,lint_reasons,new_uninit +rust_allowed_features := arbitrary_self_types,lint_reasons # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree
@@ -258,7 +258,7 @@ rust_common_cmd = \ -Zallow-features=$(rust_allowed_features) \ -Zcrate-attr=no_std \ -Zcrate-attr='feature($(rust_allowed_features))' \ - -Zunstable-options --extern force:alloc --extern kernel \ + -Zunstable-options --extern kernel \ --crate-type rlib -L $(objtree)/rust/ \ --crate-name $(basename $(notdir $@)) \ --sysroot=/dev/null \
scripts/generate_rust_analyzer.py +2 -9
@@ -64,13 +64,6 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): [], ) - append_crate( - "alloc", - sysroot_src / "alloc" / "src" / "lib.rs", - ["core", "compiler_builtins"], - cfg=crates_cfgs.get("alloc", []), - ) - append_crate( "macros", srctree / "rust" / "macros" / "lib.rs",
@@ -96,7 +89,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): append_crate( "kernel", srctree / "rust" / "kernel" / "lib.rs", - ["core", "alloc", "macros", "build_error", "bindings"], + ["core", "macros", "build_error", "bindings"], cfg=cfg, ) crates[-1]["source"] = {
@@ -133,7 +126,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs): append_crate( name, path, - ["core", "alloc", "kernel"], + ["core", "kernel"], cfg=cfg, )
Signature · 1 line
2.46.1
Danilo Krummrich
maintainerPATCH v9 29/29 MAINTAINERS: add entry for the Rust `alloc` module
In reply to Danilo KrummrichAdd maintainers entry for the Rust alloc module.
Currently, this includes the Allocator API itself, Allocator implementations, such as Kmalloc or Vmalloc, as well as the kernel's implementation of the primary memory allocation data structures, Box and Vec.
--- MAINTAINERS | 7 +++++++ 1 file changed, 7 insertions(+)
MAINTAINERS +7 -0
@@ -20226,6 +20226,13 @@ F: scripts/*rust* F: tools/testing/selftests/rust/ K: \b(?i:rust)\b +RUST [ALLOC] +M: Danilo Krummrich <[email protected]> +L: [email protected] +S: Maintained +F: rust/kernel/alloc.rs +F: rust/kernel/alloc/ + RXRPC SOCKETS (AF_RXRPC) M: David Howells <[email protected]> M: Marc Dionne <[email protected]>
Signature · 1 line
2.46.1
Miguel Ojeda
AppliedQuoted 3 lines · Danilo Krummrich
This patch series adds generic kernel allocator support for Rust, which so far is limited to
kmallocallocations.
Applied to rust-next -- thanks _a lot_ everyone!
[ Fixed typo. - Miguel ]
[ Added temporary allow(dead_code) for dangling_from_layout to clean
warning in rusttest target as discussed in the list (but it is needed earlier, i.e. in this patch already). Added colon. - Miguel ]
[ Added missing _old_layout parameter as discussed. - Miguel ]
[ Reworded typo. - Miguel ]
[ Added backticks, fixed typos. - Miguel ]
[ Formatted a few comments. - Miguel ]
[ This patch starts using a new unstable feature, inline_const, but
it was stabilized in Rust 1.79.0, i.e. the next version after the minimum one, thus it will not be an issue. - Miguel ]
[ Cleaned rustdoc unescaped backtick warning, added a couple more
backticks elsewhere, fixed typos, sorted features, rewrapped documentation lines. - Miguel ]
[ Fixed typos. - Miguel ]
[ Added newline in documentation, changed case of section to be
consistent with an existing one, fixed typo. - Miguel ]
[ Converted kasan_test_rust.rs too, as discussed. - Miguel ]
[ Removed the temporary allow(dead_code) as discussed in the list and
fixed typo, added backticks. - Miguel ]
Cheers,
Miguel
We (Asahi Linux) noticed that this doesn't support unsized types, which is a regression from the alloc Box. drm/asahi uses boxed trait objects to generalize across certain object types like render commands, so this breaks the driver.
Is there a plan to add this back in? AIUI this would need the coerce_unsized unstable feature, but is there any other blocker?
Quoted 228 lines · Danilo Krummrich
Hi,
This patch series adds generic kernel allocator support for Rust, which so far is limited to
kmallocallocations.In order to abstain from (re-)adding unstable Rust features to the kernel, this patch series does not extend the
Allocatortrait from Rust'salloccrate, nor does it extend theBoxExtandVecExtextensions.Instead, this series introduces a kernel specific
Allocatortrait, which is implemented by theKmalloc,VmallocandKVmallocallocators, also implemented in the context of this series.As a consequence we need our own kernel
Box<T, A>andVec<T, A>types. Additionally, this series adds the following type aliases:``` pub type KBox<T> = Box<T, Kmalloc>; pub type VBox<T> = Box<T, Vmalloc>; pub type KVBox<T> = Box<T, KVmalloc>; pub type KVec<T> = Vec<T, Kmalloc>; pub type VVec<T> = Vec<T, Vmalloc>; pub type KVVec<T> = Vec<T, KVmalloc>; ```With that, we can start using the kernel
BoxandVectypes throughout the tree and remove the now obolete extensionsBoxExtandVecExt.For a final cleanup, this series removes the last minor dependencies to Rust's
alloccrate and removes it from the entire kernel build.The series ensures not to break the
rusttestmake target by implementing theallocator_testmodule providing a stub implementation for all kernelAllocators.This patch series passes all KUnit tests, including the ones added by this series. Additionally, the tests were run with
kmemleakandKASANenabled, without any issues.This series is based on [1], which hit -mm/mm-stable, and is also available in [2].
[1] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=mm/krealloc
[2] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=rust/mmChanges in v9:
- rebase onto rust-next (lint series)
-Vec:
- add missing invariant comments
- fix missing#![feature(inline_const)]
- add note forkvec!thatGFP_KERNELis used for allocation
-ArrayLayout: fix missing import forsize_of::<T>
-Box: fix missing safety commentChanges in v8:
- rebase onto rust-next (v6.12-rc1), including the -mm dependencies from [1]
- introduceArrayLayoutand rework theAllocatortrait to take anold_layoutargument
- split up "rust: alloc: implementAllocatorforKmalloc" in three separate patches (introduceReallocFunc, make allocator module public and implementKmalloc)
- wrapimpl_slice_eq!pattern in$()*
- couple of style changes suggested by Benno and GaryChanges in v7:
- rebase onto rust-next resolving recent conflicts (93dc3be19450447a3a7090bd1dfb9f3daac3e8d2)
- documentation / safety comment changes suggested by Benno and Boqun
- renameptrtocurrentinIntoIter::next(Alice)
- remove unnecessary braces inkvec!(Benno)
- adddebug_assert!inVec::set_len(Benno)
- remove unused args inimpl_slice_eq!(Benno)
- simplifyifstatement inCmalloc::realloc(Benno)Changes in v6:
- rebase onto rust-dev
- keep compiler annotations for {k,v,kv}realloc()
- documentation changes suggested by Alice, Benno
- removeBox::into_pin
- fix typo inSendandSyncforBoxandVec
-kvec!changes suggested by Alice
- freesrcafter copy inCmalloc
- handlen == 0inVec::extend_withChanges in v5:
- (safety) comment / documentation fixes suggested by Alice, Benno and Gary
- removeUnique<T>and implementSendandSyncforBoxandVec
- useKMALLOC_MAX_SIZEforKVmalloctest and add aKmalloctest that expects to fail forKMALLOC_MAX_SIZE
- create use constantsKREALLOC,VREALLOCandKVREALLOCforReallocFuncs
- dropBox::drop_contentsfor now, will add it again, once I actually rebase on the original patch that introduces it
- improve usage ofsize_of_valinBox
- moveInPlaceInitandForeignOwnableimpls intokbox.rs
- fix missingBoxconversions inrnull.rs
- reworkedCmallocto keep track of the size of memory allocations itself
- removeGlobalAlloctogether with thealloccrate to avoid a linker error
- removeallocfromscripts/generate_rust_analyzer.pyChanges in v4:
- (safety) comment fixes suggested by Alice and Boqun
- removeBox::from_raw_allocandBox::into_raw_alloc, we don't need them
- inBox::dropcallsize_of_valbeforedrop_in_place
- implement ForeignOwnable for Pin<Box<T>> as suggested by Alice
- inVec::extend_with, iterate overninstead ofspare.len()
- forVmallocandKVmallocfail allocation for alignments larger thanPAGE_SIZEfor now (will add support for larger alignments in a separate series)
- implementCmallocinallocator_testand type alias all kernel allocator types to it, such that we can use the kernel'sBoxandVectypes in userspace tests (rusttest)
- this makes patch "rust: str: test: replacealloc::format" rather trivialChanges in v3:
- Box:
- minor documentation fixes
- removed unnecessary imports in doc tests
- dropeedselfargument from some remainingBoxmethods
- implementInPlaceInitfor Box<T, A> rather than specifically forKBox<T>
- Vec:
- minor documentation fixes
- removed uselessVec::allocatormethod
- inVec::extend_withuseVec::spare_capacity_mutinstead of raw pointer operations
- added a few missing safety comments
- pass GFP flags toVec::collect
- fixed a rustdoc warning inalloc.rs
- fixed theallocator_testmodule to implement theAllocatortrait correctly
- rebased to rust-nextChanges in v2:
- preserveimpl GlobalAlloc for Kmallocand remove it at the end (Benno)
- remove&selfparameter from allAllocatorfunctions (Benno)
- various documentation fixes forAllocator(Benno)
- useNonNull<u8>forAllocator::freeandOption<NonNull<u8>>forAllocator::realloc(Benno)
- fix leak ofIntoIterinVec::collect(Boqun)
- always realloc (try to shrink) inVec::collect, it's up the theAllocatorto provide a heuristic whether it makes sense to actually shrink
- renameKBox<T, A>->Box<T, A>andKVec<T, A>->Vec<T, A>and provide type aliasesKBox<T>,VBox<T>,KVBox<T>, etc.
- This allows for much cleaner code and, in combination with removing&selfparameters fromAllocators, gets us rid of the need forBox::newandBox::new_allocand all other "_alloc" postfixed functions.
- Before:KBox::new_alloc(foo, Vmalloc)?
- After:VBox::new(foo)?, which resolves to
`Box::<Foo, Vmalloc>::new(foo)?;Benno Lossin (1):
rust: alloc: introduceArrayLayoutDanilo Krummrich (28):
rust: alloc: addAllocatortrait
rust: alloc: separatealigned_sizefromkrealloc_aligned
rust: alloc: renameKernelAllocatortoKmalloc
rust: alloc: implementReallocFunc
rust: alloc: makeallocatormodule public
rust: alloc: implementAllocatorforKmalloc
rust: alloc: add moduleallocator_test
rust: alloc: implementVmallocallocator
rust: alloc: implementKVmallocallocator
rust: alloc: add__GFP_NOWARNtoFlags
rust: alloc: implement kernelBox
rust: treewide: switch to our kernelBoxtype
rust: alloc: remove extension of std'sBox
rust: alloc: addBoxto prelude
rust: alloc: implement kernelVectype
rust: alloc: implementIntoIteratorforVec
rust: alloc: implementcollectforIntoIter
rust: treewide: switch to the kernelVectype
rust: alloc: removeVecExtextension
rust: alloc: addVecto prelude
rust: error: usecore::alloc::LayoutError
rust: error: check for configtestinError::name
rust: alloc: implementcontainsforFlags
rust: alloc: implementCmallocin moduleallocator_test
rust: str: test: replacealloc::format
rust: alloc: update module comment ofalloc.rs
kbuild: rust: remove thealloccrate andGlobalAlloc
MAINTAINERS: add entry for the RustallocmoduleMAINTAINERS | 7 + drivers/block/rnull.rs | 4 +- rust/Makefile | 43 +- rust/bindings/bindings_helper.h | 1 + rust/exports.c | 1 - rust/helpers/helpers.c | 1 + rust/helpers/slab.c | 6 + rust/helpers/vmalloc.c | 9 + rust/kernel/alloc.rs | 150 ++++- rust/kernel/alloc/allocator.rs | 210 +++++-- rust/kernel/alloc/allocator_test.rs | 95 +++ rust/kernel/alloc/box_ext.rs | 89 --- rust/kernel/alloc/kbox.rs | 456 ++++++++++++++ rust/kernel/alloc/kvec.rs | 914 ++++++++++++++++++++++++++++ rust/kernel/alloc/layout.rs | 91 +++ rust/kernel/alloc/vec_ext.rs | 185 ------ rust/kernel/error.rs | 6 +- rust/kernel/init.rs | 95 +-- rust/kernel/init/__internal.rs | 2 +- rust/kernel/lib.rs | 2 +- rust/kernel/prelude.rs | 5 +- rust/kernel/rbtree.rs | 49 +- rust/kernel/str.rs | 35 +- rust/kernel/sync/arc.rs | 17 +- rust/kernel/sync/condvar.rs | 4 +- rust/kernel/sync/lock/mutex.rs | 2 +- rust/kernel/sync/lock/spinlock.rs | 2 +- rust/kernel/sync/locked_by.rs | 2 +- rust/kernel/types.rs | 52 +- rust/kernel/uaccess.rs | 17 +- rust/kernel/workqueue.rs | 20 +- rust/macros/lib.rs | 12 +- samples/rust/rust_minimal.rs | 4 +- scripts/Makefile.build | 4 +- scripts/generate_rust_analyzer.py | 11 +- 35 files changed, 2025 insertions(+), 578 deletions(-) create mode 100644 rust/helpers/vmalloc.c create mode 100644 rust/kernel/alloc/allocator_test.rs delete mode 100644 rust/kernel/alloc/box_ext.rs create mode 100644 rust/kernel/alloc/kbox.rs create mode 100644 rust/kernel/alloc/kvec.rs create mode 100644 rust/kernel/alloc/layout.rs delete mode 100644 rust/kernel/alloc/vec_ext.rs
Quoted 1 line
base-commit:
7f4a2c6cacb0efad2bd9ae5062ab1b75e13c6c1f
~~ Lina
Alice Ryhl
maintainerQuoted 8 lines · Asahi Lina
We (Asahi Linux) noticed that this doesn't support unsized types, which is a regression from the alloc Box. drm/asahi uses boxed trait objects to generalize across certain object types like render commands, so this breaks the driver.
Is there a plan to add this back in? AIUI this would need the
coerce_unsizedunstable feature, but is there any other blocker?
We're already using coerce_unsized for Arc, so I see no reason we can't just add that for KBox too. Soon, we should be able to use the derive macro [1] I proposed to do this without unstable language features. The limitation that the RFC mentions for custom allocators does not apply to KBox because it uses PhantomData<A> instead of A for the allocator field.
Alice
[1]: https://rust-lang.github.io/rfcs/3621-derive-smart-pointer.html
Quoted 10 lines · Alice Ryhl
Quoted 8 lines · Asahi Lina
We (Asahi Linux) noticed that this doesn't support unsized types, which is a regression from the alloc Box. drm/asahi uses boxed trait objects to generalize across certain object types like render commands, so this breaks the driver.
Is there a plan to add this back in? AIUI this would need the
coerce_unsizedunstable feature, but is there any other blocker?
Quoted 6 lines
We're already using
coerce_unsizedfor Arc, so I see no reason we can't just add that for KBox too. Soon, we should be able to use the derive macro [1] I proposed to do this without unstable language features. The limitation that the RFC mentions for custom allocators does not apply to KBox because it uses PhantomData<A> instead of A for the allocator field.
I have branch rebased onto rust-next and Danilo's "Device / Driver PCI / Platform Rust abstractions". In addition to coerce_unsized for kbox kvec misses the unproblematic methods clear and truncate and array_try_from_vec. drain() as taken from the rust library looks like more effort considering kvec's IntoIter.
Janne