Generic `Allocator` support for Rust

Add a kernel specific Allocator trait, that in contrast to the one in Rust's core library doesn't require unstable features and supports GFP flags.

Subsequent patches add the following trait implementors: Kmalloc, Vmalloc and KVmalloc.

Signed-off-by Danilo Krummrich <[email protected]>

 rust/kernel/alloc.rs | 73 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 73 insertions(+)

rust/kernel/alloc.rs +50 -0

index 1966bd407017..b79dd2c49277 100644
@@ -11,6 +11,7 @@
 /// Indicates an allocation error.
 #[derive(Copy, Clone, PartialEq, Eq, Debug)]
 pub struct AllocError;
+use core::{alloc::Layout, ptr::NonNull};
 
 /// Flags to be used when allocating memory.
 ///

@@ -86,3 +87,75 @@ pub mod flags {
     /// small allocations.
     pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);
 }
+
+/// The kernel's [`Allocator`] trait.
+///
+/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffer described
+/// via [`Layout`].
+///
+/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on
+/// an object instance.
+///
+/// # Safety
+///
+/// Memory returned from an allocator must point to a valid memory buffer and remain valid until
+/// it is explicitly freed.
+///
+/// Any pointer to a memory buffer which is currently allocated must be valid to be passed to any
+/// other [`Allocator`] function. The same applies for a NULL pointer.
+///
+/// If `realloc` is called with:
+///   - a size of zero, the given memory allocation, if any, must be freed
+///   - a NULL pointer, a new memory allocation must be created
+pub unsafe trait Allocator {
+    /// Allocate memory based on `layout` and `flags`.
+    ///
+    /// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout
+    /// constraints (i.e. minimum size and alignment as specified by `layout`).
+    ///
+    /// This function is equivalent to `realloc` when called with a NULL pointer.
+    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
+        // SAFETY: Passing a NULL pointer to `realloc` is valid by it's safety requirements and asks
+        // for a new memory allocation.
+        unsafe { Self::realloc(None, layout, flags) }
+    }
+
+    /// Re-allocate an existing memory allocation to satisfy the requested `layout`. If the
+    /// requested size is zero, `realloc` behaves equivalent to `free`.
+    ///
+    /// If the requested size is larger than the size of the existing allocation, a successful call
+    /// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but
+    /// may also be larger.
+    ///
+    /// If the requested size is smaller than the size of the existing allocation, `realloc` may or
+    /// may not shrink the buffer; this is implementation specific to the allocator.
+    ///
+    /// On allocation failure, the existing buffer, if any, remains valid.
+    ///
+    /// The buffer is represented as `NonNull<[u8]>`.
+    ///
+    /// # Safety
+    ///

Quoted 5 lines · Danilo Krummrich

Add a kernel specific Allocator trait, that in contrast to the one in Rust's core library doesn't require unstable features and supports GFP flags.

Quoted 4 lines

Subsequent patches add the following trait implementors: Kmalloc, Vmalloc and KVmalloc.

Signed-off-by Danilo Krummrich <[email protected]>

It could be nice to mention that the design without self parameters is necessary for compatibility with #[derive(SmartPointer)].

Quoted 74 lines

 rust/kernel/alloc.rs | 73 ++++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 73 insertions(+)

rust/kernel/alloc.rs +55 -0

index 1966bd407017..b79dd2c49277 100644
@@ -11,6 +11,7 @@
 /// Indicates an allocation error.
 #[derive(Copy, Clone, PartialEq, Eq, Debug)]
 pub struct AllocError;
+use core::{alloc::Layout, ptr::NonNull};
 
 /// Flags to be used when allocating memory.
 ///

@@ -86,3 +87,75 @@ pub mod flags {
     /// small allocations.
     pub const GFP_NOWAIT: Flags = Flags(bindings::GFP_NOWAIT);
 }
+
+/// The kernel's [`Allocator`] trait.
+///
+/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffer described
+/// via [`Layout`].
+///
+/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on
+/// an object instance.
+///
+/// # Safety
+///
+/// Memory returned from an allocator must point to a valid memory buffer and remain valid until
+/// it is explicitly freed.
+///
+/// Any pointer to a memory buffer which is currently allocated must be valid to be passed to any
+/// other [`Allocator`] function. The same applies for a NULL pointer.
+///
+/// If `realloc` is called with:
+///   - a size of zero, the given memory allocation, if any, must be freed
+///   - a NULL pointer, a new memory allocation must be created
+pub unsafe trait Allocator {
+    /// Allocate memory based on `layout` and `flags`.
+    ///
+    /// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout
+    /// constraints (i.e. minimum size and alignment as specified by `layout`).
+    ///
+    /// This function is equivalent to `realloc` when called with a NULL pointer.
+    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
+        // SAFETY: Passing a NULL pointer to `realloc` is valid by it's safety requirements and asks
+        // for a new memory allocation.
+        unsafe { Self::realloc(None, layout, flags) }
+    }
+
+    /// Re-allocate an existing memory allocation to satisfy the requested `layout`. If the
+    /// requested size is zero, `realloc` behaves equivalent to `free`.
+    ///
+    /// If the requested size is larger than the size of the existing allocation, a successful call
+    /// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but
+    /// may also be larger.
+    ///
+    /// If the requested size is smaller than the size of the existing allocation, `realloc` may or
+    /// may not shrink the buffer; this is implementation specific to the allocator.
+    ///
+    /// On allocation failure, the existing buffer, if any, remains valid.
+    ///
+    /// The buffer is represented as `NonNull<[u8]>`.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this allocator
+    /// instance.
+    ///
+    /// Additionally, `ptr` is allowed to be a NULL pointer; in this case a new memory allocation is
+    /// created.

rust/kernel/alloc.rs pub mod flags { · line 148

Quoted 5 lines

rust/kernel/alloc.rs +5 -0

+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError>;

Is it intentional that this allows you to change the alignment of an allocation? If so, that could use a note in the docs.

rust/kernel/alloc.rs pub mod flags { · line 160

Quoted 12 lines

rust/kernel/alloc.rs +11 -0

+    /// Free an existing memory allocation.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this `Allocator`
+    /// instance.
+    unsafe fn free(ptr: NonNull<u8>) {
+        // SAFETY: `ptr` is guaranteed to be previously allocated with this `Allocator` or NULL.
+        // Calling `realloc` with a buffer size of zero, frees the buffer `ptr` points to.
+        let _ = unsafe { Self::realloc(Some(ptr), Layout::new::<()>(), Flags(0)) };
+    }

At the very least, the provided implementation of free changes the alignment when it calls realloc.

Alice

Quoted 13 lines · Alice Ryhl

Quoted 11 lines · Danilo Krummrich

rust/kernel/alloc.rs +11 -0

+    /// Free an existing memory allocation.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this `Allocator`
+    /// instance.
+    unsafe fn free(ptr: NonNull<u8>) {
+        // SAFETY: `ptr` is guaranteed to be previously allocated with this `Allocator` or NULL.
+        // Calling `realloc` with a buffer size of zero, frees the buffer `ptr` points to.
+        let _ = unsafe { Self::realloc(Some(ptr), Layout::new::<()>(), Flags(0)) };
+    }

Quoted 2 lines

At the very least, the provided implementation of free changes the alignment when it calls realloc.

Yes, I think that's fine though. Hopefully no one attempts to use the memory anymore once free is being called.

Quoted 18 lines · Danilo Krummrich

Quoted 15 lines · Alice Ryhl

rust/kernel/alloc.rs pub mod flags { · line 160

Quoted 11 lines · Danilo Krummrich

rust/kernel/alloc.rs +11 -0

+    /// Free an existing memory allocation.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this `Allocator`
+    /// instance.
+    unsafe fn free(ptr: NonNull<u8>) {
+        // SAFETY: `ptr` is guaranteed to be previously allocated with this `Allocator` or NULL.
+        // Calling `realloc` with a buffer size of zero, frees the buffer `ptr` points to.
+        let _ = unsafe { Self::realloc(Some(ptr), Layout::new::<()>(), Flags(0)) };
+    }

At the very least, the provided implementation of free changes the alignment when it calls realloc.

Quoted 2 lines

Yes, I think that's fine though. Hopefully no one attempts to use the memory anymore once free is being called.

Sure, but if you require the alignment to remain constant throughout calls to realloc, then you have to word it in a way that allows a different alignment when the new size is zero.

Alice

rust/kernel/alloc.rs +6 -0

+/// The kernel's [`Allocator`] trait.
+///
+/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffer described
+/// via [`Layout`].
+///
+/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on

rust/kernel/alloc.rs +55 -0

+/// If `realloc` is called with:
+///   - a size of zero, the given memory allocation, if any, must be freed
+///   - a NULL pointer, a new memory allocation must be created
+pub unsafe trait Allocator {
+    /// Allocate memory based on `layout` and `flags`.
+    ///
+    /// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout
+    /// constraints (i.e. minimum size and alignment as specified by `layout`).
+    ///
+    /// This function is equivalent to `realloc` when called with a NULL pointer.
+    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
+        // SAFETY: Passing a NULL pointer to `realloc` is valid by it's safety requirements and asks
+        // for a new memory allocation.
+        unsafe { Self::realloc(None, layout, flags) }
+    }
+
+    /// Re-allocate an existing memory allocation to satisfy the requested `layout`. If the
+    /// requested size is zero, `realloc` behaves equivalent to `free`.
+    ///
+    /// If the requested size is larger than the size of the existing allocation, a successful call
+    /// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but
+    /// may also be larger.
+    ///
+    /// If the requested size is smaller than the size of the existing allocation, `realloc` may or
+    /// may not shrink the buffer; this is implementation specific to the allocator.
+    ///
+    /// On allocation failure, the existing buffer, if any, remains valid.
+    ///
+    /// The buffer is represented as `NonNull<[u8]>`.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this allocator
+    /// instance.
+    ///
+    /// Additionally, `ptr` is allowed to be a NULL pointer; in this case a new memory allocation is
+    /// created.
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError>;
+
+    /// Free an existing memory allocation.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this `Allocator`
+    /// instance.
+    unsafe fn free(ptr: NonNull<u8>) {
+        // SAFETY: `ptr` is guaranteed to be previously allocated with this `Allocator` or NULL.
+        // Calling `realloc` with a buffer size of zero, frees the buffer `ptr` points to.
+        let _ = unsafe { Self::realloc(Some(ptr), Layout::new::<()>(), Flags(0)) };
+    }
+}

Signature · 1 line

2.45.2

On Thu, Aug 01, 2024 at 02:02:00AM +0200, Danilo Krummrich wrote:
[...]

Quoted 11 lines

rust/kernel/alloc.rs +11 -0

+/// The kernel's [`Allocator`] trait.
+///
+/// An implementation of [`Allocator`] can allocate, re-allocate and free memory buffer described
+/// via [`Layout`].
+///
+/// [`Allocator`] is designed to be implemented as a ZST; [`Allocator`] functions do not operate on
+/// an object instance.
+///
+/// # Safety
+///
+/// Memory returned from an allocator must point to a valid memory buffer and remain valid until

rust/kernel/alloc.rs pub mod flags { · line 106

Quoted 5 lines

rust/kernel/alloc.rs +5 -0

+/// it is explicitly freed.
+///
+/// Any pointer to a memory buffer which is currently allocated must be valid to be passed to any
+/// other [`Allocator`] function. The same applies for a NULL pointer.
+///

Are you saying you could kmalloc() a memory buffer and pass it to a vfree()? Or am I missing something here?

Regards,
Boqun

Quoted 58 lines

rust/kernel/alloc.rs +55 -0

+/// If `realloc` is called with:
+///   - a size of zero, the given memory allocation, if any, must be freed
+///   - a NULL pointer, a new memory allocation must be created
+pub unsafe trait Allocator {
+    /// Allocate memory based on `layout` and `flags`.
+    ///
+    /// On success, returns a buffer represented as `NonNull<[u8]>` that satisfies the layout
+    /// constraints (i.e. minimum size and alignment as specified by `layout`).
+    ///
+    /// This function is equivalent to `realloc` when called with a NULL pointer.
+    fn alloc(layout: Layout, flags: Flags) -> Result<NonNull<[u8]>, AllocError> {
+        // SAFETY: Passing a NULL pointer to `realloc` is valid by it's safety requirements and asks
+        // for a new memory allocation.
+        unsafe { Self::realloc(None, layout, flags) }
+    }
+
+    /// Re-allocate an existing memory allocation to satisfy the requested `layout`. If the
+    /// requested size is zero, `realloc` behaves equivalent to `free`.
+    ///
+    /// If the requested size is larger than the size of the existing allocation, a successful call
+    /// to `realloc` guarantees that the new or grown buffer has at least `Layout::size` bytes, but
+    /// may also be larger.
+    ///
+    /// If the requested size is smaller than the size of the existing allocation, `realloc` may or
+    /// may not shrink the buffer; this is implementation specific to the allocator.
+    ///
+    /// On allocation failure, the existing buffer, if any, remains valid.
+    ///
+    /// The buffer is represented as `NonNull<[u8]>`.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this allocator
+    /// instance.
+    ///
+    /// Additionally, `ptr` is allowed to be a NULL pointer; in this case a new memory allocation is
+    /// created.
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError>;
+
+    /// Free an existing memory allocation.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must point to an existing and valid memory allocation created by this `Allocator`
+    /// instance.
+    unsafe fn free(ptr: NonNull<u8>) {
+        // SAFETY: `ptr` is guaranteed to be previously allocated with this `Allocator` or NULL.
+        // Calling `realloc` with a buffer size of zero, frees the buffer `ptr` points to.
+        let _ = unsafe { Self::realloc(Some(ptr), Layout::new::<()>(), Flags(0)) };
+    }
+}

Signature · 1 line

2.45.2

Separate aligned_size from krealloc_aligned.

Subsequent patches implement Allocator derivates, such as Kmalloc, that require aligned_size and replace the original krealloc_aligned.

Signed-off-by Danilo Krummrich <[email protected]>

Subsequent patches implement Vmalloc and KVmalloc allocators, hence align KernelAllocator to this naming scheme.

Signed-off-by Danilo Krummrich <[email protected]>

+/// Returns a proper size to alloc a new object aligned to new_layout's alignment.
fn aligned_size(new_layout: Layout) -> usize {

rust/kernel/alloc/allocator.rs +41 -1

+        &self,
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let size = aligned_size(layout);
+        let ptr = match ptr {
+            Some(ptr) => ptr.as_ptr(),
+            None => ptr::null(),
+        };
+
+        // SAFETY: `ptr` is valid by the safety requirements of this function.
+        let raw_ptr = unsafe {
+            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
+            self.0(ptr.cast(), size, flags.0).cast()
+        };
+
+        let ptr = if size == 0 {
+            NonNull::dangling()
+        } else {
+            NonNull::new(raw_ptr).ok_or(AllocError)?
+        };
+
+        Ok(NonNull::slice_from_raw_parts(ptr, size))
+    }
+}
+
+unsafe impl Allocator for Kmalloc {
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let realloc = ReallocFunc::krealloc();
+
+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { realloc.call(ptr, layout, flags) }
+    }
+}
+
 unsafe impl GlobalAlloc for Kmalloc {
     unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
         // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
--

2.45.2

Quoted 2 lines · Danilo Krummrich

+/// Returns a proper size to alloc a new object aligned to new_layout's alignment.
fn aligned_size(new_layout: Layout) -> usize {

This comment could potentially be moved to the previous patch that defined the function.

rust/kernel/alloc/allocator.rs pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F · line 54

Quoted 4 lines

rust/kernel/alloc/allocator.rs +4 -0

+struct ReallocFunc(
+    // INVARIANT: One of the following `krealloc`, `vrealloc`, `kvrealloc`.
+    unsafe extern "C" fn(*const core::ffi::c_void, usize, u32) -> *mut core::ffi::c_void,
+);

In this case, the comment would usually be formatted with markdown.

/// # Invariants
///
/// Must contain one of the following: krealloc, vrealloc, kvrealloc.

The // INVARIANT: syntax is used when constructing an instance to argue why the documentented invariants are satisfied.

rust/kernel/alloc/allocator.rs pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F · line 59

Quoted 4 lines

rust/kernel/alloc/allocator.rs +4 -0

+impl ReallocFunc {
+    fn krealloc() -> Self {
+        Self(bindings::krealloc)
+    }

Technically this should have an // INVARIANT: explaining why the invariants are satisfied by this new value.

rust/kernel/alloc/allocator.rs pub(crate) unsafe fn krealloc_aligned(ptr: *mut u8, new_layout: Layout, flags: F · line 62

Quoted 3 lines

rust/kernel/alloc/allocator.rs +3 -0

+
+    // SAFETY: `call` has the exact same safety requirements as `Allocator::realloc`.
+    unsafe fn call(

Similarly to the above, the // SAFETY: syntax is used when arguing why the preconditions are satisfied, but when explaining what the preconditions are, we usually use this syntax instead:

/// # Safety
///
/// This method has the same safety requirements as Allocator::realloc.

Quoted 48 lines

rust/kernel/alloc/allocator.rs +41 -1

+        &self,
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let size = aligned_size(layout);
+        let ptr = match ptr {
+            Some(ptr) => ptr.as_ptr(),
+            None => ptr::null(),
+        };
+
+        // SAFETY: `ptr` is valid by the safety requirements of this function.
+        let raw_ptr = unsafe {
+            // If `size == 0` and `ptr != NULL` the memory behind the pointer is freed.
+            self.0(ptr.cast(), size, flags.0).cast()
+        };
+
+        let ptr = if size == 0 {
+            NonNull::dangling()
+        } else {
+            NonNull::new(raw_ptr).ok_or(AllocError)?
+        };
+
+        Ok(NonNull::slice_from_raw_parts(ptr, size))
+    }
+}
+
+unsafe impl Allocator for Kmalloc {
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let realloc = ReallocFunc::krealloc();
+
+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { realloc.call(ptr, layout, flags) }
+    }
+}
+
 unsafe impl GlobalAlloc for Kmalloc {
     unsafe fn alloc(&self, layout: Layout) -> *mut u8 {
         // SAFETY: `ptr::null_mut()` is null and `layout` has a non-zero size by the function safety
--

2.45.2

Allocators, such as Kmalloc, will be used by e.g. Box and Vec in subsequent patches, and hence this dependency propagates throughout the whole kernel.

Add the allocator_test module that provides an empty implementation for all Allocators in the kernel, such that we don't break the rusttest make target in subsequent patches.

Signed-off-by Danilo Krummrich <[email protected]>

Implement Allocator for Vmalloc, the kernel's virtually contiguous allocator, typically used for larger objects, (much) larger than page size.

All memory allocations made with Vmalloc end up in vrealloc().

Signed-off-by Danilo Krummrich <[email protected]>

 rust/helpers.c                      |  8 ++++++++
 rust/kernel/alloc/allocator.rs      | 24 ++++++++++++++++++++++++
 rust/kernel/alloc/allocator_test.rs |  1 +
 3 files changed, 33 insertions(+)

rust/helpers.c +8 -0

index 92d3c03ae1bd..4c628986f0c9 100644
@@ -33,6 +33,7 @@
 #include <linux/sched/signal.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/vmalloc.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 

@@ -200,6 +201,13 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(rust_helper_krealloc);
 
+void * __must_check __realloc_size(2)
+rust_helper_vrealloc(const void *p, size_t size, gfp_t flags)
+{
+       return vrealloc(p, size, flags);
+}
+EXPORT_SYMBOL_GPL(rust_helper_vrealloc);
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.

rust/kernel/alloc/allocator.rs +6 -0

index 397ae5bcc043..e9a3d0694f41 100644
@@ -16,6 +16,12 @@
 /// `bindings::krealloc`.
 pub struct Kmalloc;
 
+/// The virtually contiguous kernel allocator.
+///
+/// The vmalloc allocator allocates pages from the page level allocator and maps them into the
+/// contiguous kernel virtual space.
+pub struct Vmalloc;
+
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
     // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.

rust/kernel/alloc/allocator.rs +15 -0

+
     // SAFETY: `call` has the exact same safety requirements as `Allocator::realloc`.
     unsafe fn call(
         &self,

@@ -136,6 +146,20 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
     }
 }
 
+unsafe impl Allocator for Vmalloc {
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let realloc = ReallocFunc::vrealloc();
+
+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { realloc.call(ptr, layout, flags) }
+    }
+}
+
 #[global_allocator]
 static ALLOCATOR: Kmalloc = Kmalloc;
 

rust/kernel/alloc/allocator_test.rs +1 -1

index 4785efc474a7..e7bf2982f68f 100644
@@ -7,6 +7,7 @@
 use core::ptr::NonNull;
 
 pub struct Kmalloc;
+pub type Vmalloc = Kmalloc;
 
 unsafe impl Allocator for Kmalloc {
     unsafe fn realloc(
--

2.45.2

Implement Allocator for Vmalloc, the kernel's virtually contiguous allocator, typically used for larger objects, (much) larger than page size.

All memory allocations made with Vmalloc end up in vrealloc().

Signed-off-by Danilo Krummrich <[email protected]>

---
 rust/helpers.c                      |  8 ++++++++
 rust/kernel/alloc/allocator.rs      | 24 ++++++++++++++++++++++++
 rust/kernel/alloc/allocator_test.rs |  1 +
 3 files changed, 33 insertions(+)

rust/helpers.c +8 -0

index 92d3c03ae1bd..4c628986f0c9 100644
@@ -33,6 +33,7 @@
 #include <linux/sched/signal.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/vmalloc.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 

@@ -200,6 +201,13 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(rust_helper_krealloc);
 
+void * __must_check __realloc_size(2)
+rust_helper_vrealloc(const void *p, size_t size, gfp_t flags)
+{
+	return vrealloc(p, size, flags);
+}
+EXPORT_SYMBOL_GPL(rust_helper_vrealloc);
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.

rust/kernel/alloc/allocator.rs +10 -0

index 397ae5bcc043..e9a3d0694f41 100644
@@ -16,6 +16,12 @@
 /// `bindings::krealloc`.
 pub struct Kmalloc;
 
+/// The virtually contiguous kernel allocator.
+///
+/// The vmalloc allocator allocates pages from the page level allocator and maps them into the
+/// contiguous kernel virtual space.
+pub struct Vmalloc;
+
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
     // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.

@@ -58,6 +64,10 @@ fn krealloc() -> Self {
         Self(bindings::krealloc)
     }
 
+    fn vrealloc() -> Self {
+        Self(bindings::vrealloc)
+    }
+
     // SAFETY: `call` has the exact same safety requirements as `Allocator::realloc`.
     unsafe fn call(
         &self,

@@ -136,6 +146,20 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
     }

rust/kernel/alloc/allocator.rs +6 -0

+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { realloc.call(ptr, layout, flags) }
+    }
+}
+
 #[global_allocator]
 static ALLOCATOR: Kmalloc = Kmalloc;
 

rust/kernel/alloc/allocator_test.rs +1 -0

index 4785efc474a7..e7bf2982f68f 100644
@@ -7,6 +7,7 @@
 use core::ptr::NonNull;
 
 pub struct Kmalloc;
+pub type Vmalloc = Kmalloc;
 
 unsafe impl Allocator for Kmalloc {
     unsafe fn realloc(

Signature · 1 line

2.45.2

Quoted 71 lines · Danilo Krummrich

Implement Allocator for Vmalloc, the kernel's virtually contiguous allocator, typically used for larger objects, (much) larger than page size.

All memory allocations made with Vmalloc end up in vrealloc().

Signed-off-by Danilo Krummrich <[email protected]>

---
 rust/helpers.c                      |  8 ++++++++
 rust/kernel/alloc/allocator.rs      | 24 ++++++++++++++++++++++++
 rust/kernel/alloc/allocator_test.rs |  1 +
 3 files changed, 33 insertions(+)

rust/helpers.c +8 -0

index 92d3c03ae1bd..4c628986f0c9 100644
@@ -33,6 +33,7 @@
 #include <linux/sched/signal.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/vmalloc.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 

@@ -200,6 +201,13 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(rust_helper_krealloc);
 
+void * __must_check __realloc_size(2)
+rust_helper_vrealloc(const void *p, size_t size, gfp_t flags)
+{
+	return vrealloc(p, size, flags);
+}
+EXPORT_SYMBOL_GPL(rust_helper_vrealloc);
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.

rust/kernel/alloc/allocator.rs +10 -0

index 397ae5bcc043..e9a3d0694f41 100644
@@ -16,6 +16,12 @@
 /// `bindings::krealloc`.
 pub struct Kmalloc;
 
+/// The virtually contiguous kernel allocator.
+///
+/// The vmalloc allocator allocates pages from the page level allocator and maps them into the
+/// contiguous kernel virtual space.
+pub struct Vmalloc;
+
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
     // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.

@@ -58,6 +64,10 @@ fn krealloc() -> Self {
         Self(bindings::krealloc)
     }
 
+    fn vrealloc() -> Self {
+        Self(bindings::vrealloc)
+    }
+
     // SAFETY: `call` has the exact same safety requirements as `Allocator::realloc`.
     unsafe fn call(
         &self,

@@ -136,6 +146,20 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
     }
 }
 

rust/kernel/alloc/allocator.rs unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { · line 156

Quoted 8 lines

rust/kernel/alloc/allocator.rs +8 -0

+unsafe impl Allocator for Vmalloc {
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let realloc = ReallocFunc::vrealloc();
+

IIUC, vrealloc() calls __vmalloc_noprof() in allocation case, that is calling __vmalloc_node_noprof() with align=1. In such a case, how would vmalloc() guarantee the allocated memory is aligned to layout.align()?

[Cc Vlastimil]

Regards,
Boqun

Quoted 24 lines

rust/kernel/alloc/allocator.rs +6 -0

+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { realloc.call(ptr, layout, flags) }
+    }
+}
+
 #[global_allocator]
 static ALLOCATOR: Kmalloc = Kmalloc;
 

rust/kernel/alloc/allocator_test.rs +1 -0

index 4785efc474a7..e7bf2982f68f 100644
@@ -7,6 +7,7 @@
 use core::ptr::NonNull;
 
 pub struct Kmalloc;
+pub type Vmalloc = Kmalloc;
 
 unsafe impl Allocator for Kmalloc {
     unsafe fn realloc(

Signature · 1 line

2.45.2

Quoted 72 lines · Boqun Feng

Quoted 71 lines · Danilo Krummrich

Implement Allocator for Vmalloc, the kernel's virtually contiguous allocator, typically used for larger objects, (much) larger than page size.

All memory allocations made with Vmalloc end up in vrealloc().

Signed-off-by Danilo Krummrich <[email protected]>

---
 rust/helpers.c                      |  8 ++++++++
 rust/kernel/alloc/allocator.rs      | 24 ++++++++++++++++++++++++
 rust/kernel/alloc/allocator_test.rs |  1 +
 3 files changed, 33 insertions(+)

rust/helpers.c +8 -0

index 92d3c03ae1bd..4c628986f0c9 100644
@@ -33,6 +33,7 @@
 #include <linux/sched/signal.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/vmalloc.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 

@@ -200,6 +201,13 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(rust_helper_krealloc);
 
+void * __must_check __realloc_size(2)
+rust_helper_vrealloc(const void *p, size_t size, gfp_t flags)
+{
+	return vrealloc(p, size, flags);
+}
+EXPORT_SYMBOL_GPL(rust_helper_vrealloc);
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.

rust/kernel/alloc/allocator.rs +10 -0

index 397ae5bcc043..e9a3d0694f41 100644
@@ -16,6 +16,12 @@
 /// `bindings::krealloc`.
 pub struct Kmalloc;
 
+/// The virtually contiguous kernel allocator.
+///
+/// The vmalloc allocator allocates pages from the page level allocator and maps them into the
+/// contiguous kernel virtual space.
+pub struct Vmalloc;
+
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
     // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.

@@ -58,6 +64,10 @@ fn krealloc() -> Self {
         Self(bindings::krealloc)
     }
 
+    fn vrealloc() -> Self {
+        Self(bindings::vrealloc)
+    }
+
     // SAFETY: `call` has the exact same safety requirements as `Allocator::realloc`.
     unsafe fn call(
         &self,

@@ -136,6 +146,20 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
     }
 }
 

Quoted 12 lines

rust/kernel/alloc/allocator.rs unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { · line 156

Quoted 8 lines

rust/kernel/alloc/allocator.rs +8 -0

+unsafe impl Allocator for Vmalloc {
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let realloc = ReallocFunc::vrealloc();
+

IIUC, vrealloc() calls __vmalloc_noprof() in allocation case, that is calling __vmalloc_node_noprof() with align=1. In such a case, how would vmalloc() guarantee the allocated memory is aligned to layout.align()?

True, good catch. I thought of this a while ago and then forgot to fix it.

Quoted 32 lines

[Cc Vlastimil]

Regards,
Boqun

Quoted 24 lines

rust/kernel/alloc/allocator.rs +6 -0

+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { realloc.call(ptr, layout, flags) }
+    }
+}
+
 #[global_allocator]
 static ALLOCATOR: Kmalloc = Kmalloc;
 

rust/kernel/alloc/allocator_test.rs +1 -0

index 4785efc474a7..e7bf2982f68f 100644
@@ -7,6 +7,7 @@
 use core::ptr::NonNull;
 
 pub struct Kmalloc;
+pub type Vmalloc = Kmalloc;
 
 unsafe impl Allocator for Kmalloc {
     unsafe fn realloc(

Signature · 1 line

2.45.2

Quoted 86 lines · Danilo Krummrich

Quoted 84 lines · Boqun Feng

Quoted 69 lines · Danilo Krummrich

Implement Allocator for Vmalloc, the kernel's virtually contiguous allocator, typically used for larger objects, (much) larger than page size.

All memory allocations made with Vmalloc end up in vrealloc().

Signed-off-by Danilo Krummrich <[email protected]>

---
 rust/helpers.c                      |  8 ++++++++
 rust/kernel/alloc/allocator.rs      | 24 ++++++++++++++++++++++++
 rust/kernel/alloc/allocator_test.rs |  1 +
 3 files changed, 33 insertions(+)

rust/helpers.c +8 -0

index 92d3c03ae1bd..4c628986f0c9 100644
@@ -33,6 +33,7 @@
 #include <linux/sched/signal.h>
 #include <linux/slab.h>
 #include <linux/spinlock.h>
+#include <linux/vmalloc.h>
 #include <linux/wait.h>
 #include <linux/workqueue.h>
 

@@ -200,6 +201,13 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(rust_helper_krealloc);
 
+void * __must_check __realloc_size(2)
+rust_helper_vrealloc(const void *p, size_t size, gfp_t flags)
+{
+	return vrealloc(p, size, flags);
+}
+EXPORT_SYMBOL_GPL(rust_helper_vrealloc);
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.

rust/kernel/alloc/allocator.rs +10 -0

index 397ae5bcc043..e9a3d0694f41 100644
@@ -16,6 +16,12 @@
 /// `bindings::krealloc`.
 pub struct Kmalloc;
 
+/// The virtually contiguous kernel allocator.
+///
+/// The vmalloc allocator allocates pages from the page level allocator and maps them into the
+/// contiguous kernel virtual space.
+pub struct Vmalloc;
+
 /// Returns a proper size to alloc a new object aligned to `new_layout`'s alignment.
 fn aligned_size(new_layout: Layout) -> usize {
     // Customized layouts from `Layout::from_size_align()` can have size < align, so pad first.

@@ -58,6 +64,10 @@ fn krealloc() -> Self {
         Self(bindings::krealloc)
     }
 
+    fn vrealloc() -> Self {
+        Self(bindings::vrealloc)
+    }
+
     // SAFETY: `call` has the exact same safety requirements as `Allocator::realloc`.
     unsafe fn call(
         &self,

@@ -136,6 +146,20 @@ unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 {
     }

rust/kernel/alloc/allocator.rs unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { · line 156

Quoted 10 lines

rust/kernel/alloc/allocator.rs +8 -0

 }
 
+unsafe impl Allocator for Vmalloc {
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,
+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let realloc = ReallocFunc::vrealloc();
+

IIUC, vrealloc() calls __vmalloc_noprof() in allocation case, that is calling __vmalloc_node_noprof() with align=1. In such a case, how would vmalloc() guarantee the allocated memory is aligned to layout.align()?

Quoted 1 line

True, good catch. I thought of this a while ago and then forgot to fix it.

Just for clarification, we're always PAGE_SIZE aligned (guaranteed by __alloc_vmap_area()), which probably would always be sufficient. That's why I didn't gave it too much attention in the first place and then forgot about it.

However, we indeed do not honor layout.align() if it's larger than PAGE_SIZE.

Quoted 33 lines

Quoted 33 lines

[Cc Vlastimil]

Regards,
Boqun

Quoted 24 lines

rust/kernel/alloc/allocator.rs +6 -0

+        // SAFETY: If not `None`, `ptr` is guaranteed to point to valid memory, which was previously
+        // allocated with this `Allocator`.
+        unsafe { realloc.call(ptr, layout, flags) }
+    }
+}
+
 #[global_allocator]
 static ALLOCATOR: Kmalloc = Kmalloc;
 

rust/kernel/alloc/allocator_test.rs +1 -0

index 4785efc474a7..e7bf2982f68f 100644
@@ -7,6 +7,7 @@
 use core::ptr::NonNull;
 
 pub struct Kmalloc;
+pub type Vmalloc = Kmalloc;
 
 unsafe impl Allocator for Kmalloc {
     unsafe fn realloc(

Signature · 1 line

2.45.2

Quoted 19 lines

Quoted 13 lines

Quoted 4 lines

Quoted 4 lines

rust/kernel/alloc/allocator.rs +4 -0

+unsafe impl Allocator for Vmalloc {
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,

Quoted 8 lines

rust/kernel/alloc/allocator.rs unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { · line 156

Quoted 4 lines

rust/kernel/alloc/allocator.rs +4 -0

+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let realloc = ReallocFunc::vrealloc();
+

IIUC, vrealloc() calls __vmalloc_noprof() in allocation case, that is calling __vmalloc_node_noprof() with align=1. In such a case, how would vmalloc() guarantee the allocated memory is aligned to layout.align()?

Quoted 1 line

True, good catch. I thought of this a while ago and then forgot to fix it.

Just for clarification, we're always PAGE_SIZE aligned (guaranteed by __alloc_vmap_area()), which probably would always be sufficient. That's why I didn't gave it too much attention in the first place and then forgot about it.

Quoted 1 line

However, we indeed do not honor layout.align() if it's larger than PAGE_SIZE.

Another note on that:

My plan for this series was to just fail allocation for alignment requests larger than PAGE_SIZE. And, if required, address larger alignments in a later

series, since this one is probably big enough already.

However, for Vmalloc we could support it right away, since it's trivial. For KVmalloc though it requires a bit more effort.

For consistancy it would probably be better to support alignments larger than PAGE_SIZE either for Vmalloc and KVmalloc or neither of those though.

My personal tendency goes a bit more into the direction of picking consistancy.

Any other opinions?

On Sun, Aug 04, 2024 at 07:39:52PM +0200, Danilo Krummrich wrote:
[...]

Quoted 19 lines

Quoted 19 lines

Quoted 13 lines

Quoted 4 lines

Quoted 4 lines

rust/kernel/alloc/allocator.rs +4 -0

+unsafe impl Allocator for Vmalloc {
+    unsafe fn realloc(
+        ptr: Option<NonNull<u8>>,
+        layout: Layout,

Quoted 8 lines

rust/kernel/alloc/allocator.rs unsafe fn alloc_zeroed(&self, layout: Layout) -> *mut u8 { · line 156

Quoted 4 lines

rust/kernel/alloc/allocator.rs +4 -0

+        flags: Flags,
+    ) -> Result<NonNull<[u8]>, AllocError> {
+        let realloc = ReallocFunc::vrealloc();
+

IIUC, vrealloc() calls __vmalloc_noprof() in allocation case, that is calling __vmalloc_node_noprof() with align=1. In such a case, how would vmalloc() guarantee the allocated memory is aligned to layout.align()?

Quoted 1 line

True, good catch. I thought of this a while ago and then forgot to fix it.

Just for clarification, we're always PAGE_SIZE aligned (guaranteed by __alloc_vmap_area()), which probably would always be sufficient. That's why I didn't gave it too much attention in the first place and then forgot about it.

Quoted 6 lines

Quoted 1 line

However, we indeed do not honor layout.align() if it's larger than PAGE_SIZE.

Another note on that:

My plan for this series was to just fail allocation for alignment requests larger than PAGE_SIZE. And, if required, address larger alignments in a later

Yeah, this sounds reasonable.

Quoted 2 lines

series, since this one is probably big enough already.

Quoted 3 lines

However, for Vmalloc we could support it right away, since it's trivial. For KVmalloc though it requires a bit more effort.

Could you elaborate why it requires a bit more effort? Because kvrealloc() and kvmalloc() in C don't have a way to specify alignment requirement?

If so, I think a solution to that would be just providing the K-or-V switch in Rust code, i.e. just Vmalloc and Kmalloc to implement KVmalloc, which I don't think is a bad idea.

Regards,
Boqun

Quoted 7 lines

For consistancy it would probably be better to support alignments larger than PAGE_SIZE either for Vmalloc and KVmalloc or neither of those though.

My personal tendency goes a bit more into the direction of picking consistancy.

Any other opinions?

[...]

Implement Allocator for KVmalloc, an Allocator that tries to allocate memory wth kmalloc first and, on failure, falls back to vmalloc.

All memory allocations made with KVmalloc end up in kvrealloc_noprof(); all frees in kvfree().

Signed-off-by Danilo Krummrich <[email protected]>

Quoted 5 lines · Danilo Krummrich

Implement Allocator for KVmalloc, an Allocator that tries to allocate memory wth kmalloc first and, on failure, falls back to vmalloc.

Quoted 4 lines

All memory allocations made with KVmalloc end up in kvrealloc_noprof(); all frees in kvfree().

Signed-off-by Danilo Krummrich <[email protected]>

The nit from vrealloc also applies here, but:

Reviewed-by Alice Ryhl <[email protected]>

Implement the Unique type as a prerequisite for Box and Vec introduced in subsequent patches.

Unique serves as wrapper around a NonNull, but indicates that the possessor of this wrapper owns the referent.

This type already exists in Rust's core library, but, unfortunately, is exposed as unstable API and hence shouldn't be used in the kernel.

This implementation of Unique is almost identical, but mostly stripped down to the functionality we need for Box and Vec. Additionally, all unstable features are removed and / or replaced by stable ones.

Signed-off-by Danilo Krummrich <[email protected]>

Implement the Unique type as a prerequisite for Box and Vec introduced in subsequent patches.

Unique serves as wrapper around a NonNull, but indicates that the possessor of this wrapper owns the referent.

This type already exists in Rust's core library, but, unfortunately, is exposed as unstable API and hence shouldn't be used in the kernel.

This implementation of Unique is almost identical, but mostly stripped down to the functionality we need for Box and Vec. Additionally, all unstable features are removed and / or replaced by stable ones.

Signed-off-by Danilo Krummrich <[email protected]>

---
 rust/kernel/types.rs | 183 +++++++++++++++++++++++++++++++++++++++++++
 1 file changed, 183 insertions(+)

rust/kernel/types.rs +183 -0

index bd189d646adb..7cf89067b5fc 100644
@@ -473,3 +473,186 @@ unsafe impl AsBytes for str {}
 // does not have any uninitialized portions either.
 unsafe impl<T: AsBytes> AsBytes for [T] {}
 unsafe impl<T: AsBytes, const N: usize> AsBytes for [T; N] {}
+
+/// A wrapper around a raw non-null `*mut T` that indicates that the possessor
+/// of this wrapper owns the referent. Useful for building abstractions like
+/// `Box<T>`, `Vec<T>`, `String`, and `HashMap<K, V>`.
+///
+/// Unlike `*mut T`, `Unique<T>` behaves "as if" it were an instance of `T`.
+/// It implements `Send`/`Sync` if `T` is `Send`/`Sync`. It also implies
+/// the kind of strong aliasing guarantees an instance of `T` can expect:
+/// the referent of the pointer should not be modified without a unique path to
+/// its owning Unique.
+///
+/// If you're uncertain of whether it's correct to use `Unique` for your purposes,
+/// consider using `NonNull`, which has weaker semantics.
+///
+/// Unlike `*mut T`, the pointer must always be non-null, even if the pointer
+/// is never dereferenced. This is so that enums may use this forbidden value
+/// as a discriminant -- `Option<Unique<T>>` has the same size as `Unique<T>`.
+/// However the pointer may still dangle if it isn't dereferenced.
+///
+/// Unlike `*mut T`, `Unique<T>` is covariant over `T`. This should always be correct
+/// for any type which upholds Unique's aliasing requirements.
+#[repr(transparent)]
+pub struct Unique<T: ?Sized> {
+    pointer: NonNull<T>,
+    // NOTE: this marker has no consequences for variance, but is necessary
+    // for dropck to understand that we logically own a `T`.
+    //
+    // For details, see:
+    // https://github.com/rust-lang/rfcs/blob/master/text/0769-sound-generic-drop.md#phantom-data
+    _marker: PhantomData<T>,
+}
+
+/// `Unique` pointers are `Send` if `T` is `Send` because the data they
+/// reference is unaliased. Note that this aliasing invariant is
+/// unenforced by the type system; the abstraction using the
+/// `Unique` must enforce it.
+unsafe impl<T: Send + ?Sized> Send for Unique<T> {}
+
+/// `Unique` pointers are `Sync` if `T` is `Sync` because the data they
+/// reference is unaliased. Note that this aliasing invariant is
+/// unenforced by the type system; the abstraction using the
+/// `Unique` must enforce it.
+unsafe impl<T: Sync + ?Sized> Sync for Unique<T> {}
+
+impl<T: Sized> Unique<T> {
+    /// Creates a new `Unique` that is dangling, but well-aligned.
+    ///
+    /// This is useful for initializing types which lazily allocate, like
+    /// `Vec::new` does.
+    ///
+    /// Note that the pointer value may potentially represent a valid pointer to
+    /// a `T`, which means this must not be used as a "not yet initialized"
+    /// sentinel value. Types that lazily allocate must track initialization by
+    /// some other means.
+    #[must_use]
+    #[inline]
+    pub const fn dangling() -> Self {
+        Unique {
+            pointer: NonNull::dangling(),
+            _marker: PhantomData,
+        }
+    }
+}
+
+impl<T: ?Sized> Unique<T> {
+    /// Creates a new `Unique`.
+    ///
+    /// # Safety
+    ///
+    /// `ptr` must be non-null.
+    #[inline]
+    pub const unsafe fn new_unchecked(ptr: *mut T) -> Self {
+        // SAFETY: the caller must guarantee that `ptr` is non-null.
+        unsafe {
+            Unique {
+                pointer: NonNull::new_unchecked(ptr),
+                _marker: PhantomData,
+            }
+        }
+    }
+
+    /// Creates a new `Unique` if `ptr` is non-null.
+    #[allow(clippy::manual_map)]
+    #[inline]
+    pub fn new(ptr: *mut T) -> Option<Self> {
+        if let Some(pointer) = NonNull::new(ptr) {
+            Some(Unique {
+                pointer,
+                _marker: PhantomData,
+            })
+        } else {
+            None
+        }
+    }
+
+    /// Acquires the underlying `*mut` pointer.
+    #[must_use = "`self` will be dropped if the result is not used"]
+    #[inline]
+    pub const fn as_ptr(self) -> *mut T {
+        self.pointer.as_ptr()
+    }
+
+    /// Dereferences the content.
+    ///
+    /// The resulting lifetime is bound to self so this behaves "as if"
+    /// it were actually an instance of T that is getting borrowed. If a longer
+    /// (unbound) lifetime is needed, use `&*my_ptr.as_ptr()`.
+    ///
+    /// # Safety
+    ///
+    /// Safety requirements for this function are inherited from [NonNull::as_ref].
+    ///
+    #[must_use]
+    #[inline]
+    pub const unsafe fn as_ref(&self) -> &T {
+        // SAFETY: the caller must guarantee that `self` meets all the
+        // requirements for a reference.
+        unsafe { self.pointer.as_ref() }
+    }
+
+    /// Mutably dereferences the content.
+    ///
+    /// The resulting lifetime is bound to self so this behaves "as if"
+    /// it were actually an instance of T that is getting borrowed. If a longer
+    /// (unbound) lifetime is needed, use `&mut *my_ptr.as_ptr()`.
+    ///
+    /// # Safety
+    ///
+    /// Safety requirements for this function are inherited from [NonNull::as_mut].
+    #[must_use]
+    #[inline]
+    pub unsafe fn as_mut(&mut self) -> &mut T {
+        // SAFETY: the caller must guarantee that `self` meets all the
+        // requirements for a mutable reference.
+        unsafe { self.pointer.as_mut() }
+    }
+
+    /// Casts to a pointer of another type.
+    #[must_use = "`self` will be dropped if the result is not used"]
+    #[inline]
+    pub fn cast<U>(self) -> Unique<U> {
+        Unique::from(self.pointer.cast())
+    }
+
+    /// Acquires the underlying `*mut` pointer.
+    #[must_use = "`self` will be dropped if the result is not used"]
+    #[inline]
+    pub const fn as_non_null(self) -> NonNull<T> {
+        self.pointer
+    }
+}
+
+impl<T: ?Sized> Clone for Unique<T> {
+    #[inline]
+    fn clone(&self) -> Self {
+        *self
+    }
+}
+
+impl<T: ?Sized> Copy for Unique<T> {}
+
+impl<T: ?Sized> From<&mut T> for Unique<T> {
+    /// Converts a `&mut T` to a `Unique<T>`.
+    ///
+    /// This conversion is infallible since references cannot be null.
+    #[inline]
+    fn from(reference: &mut T) -> Self {
+        Self::from(NonNull::from(reference))
+    }
+}
+
+impl<T: ?Sized> From<NonNull<T>> for Unique<T> {
+    /// Converts a `NonNull<T>` to a `Unique<T>`.
+    ///
+    /// This conversion is infallible since `NonNull` cannot be null.
+    #[inline]
+    fn from(pointer: NonNull<T>) -> Self {
+        Unique {
+            pointer,
+            _marker: PhantomData,
+        }
+    }
+}

Signature · 1 line

2.45.2

Box provides the simplest way to allocate memory for a generic type with one of the kernel's allocators, e.g. Kmalloc, Vmalloc or KVmalloc.

In contrast to Rust's Box type, the kernel Box type considers the kernel's GFP flags for all appropriate functions, always reports allocation failures through Result<_, AllocError> and remains independent from unstable features.

Signed-off-by Danilo Krummrich <[email protected]>

[...]

rust/kernel/alloc/kbox.rs new file +1 -0

+    /// Constructs a `Box<T, A>` from a raw pointer.

rust/kernel/alloc/kbox.rs new file +7 -0

+impl<T, A> Drop for Box<T, A>
+where
+    T: ?Sized,
+    A: Allocator,
+{
+    fn drop(&mut self) {
+        let ptr = self.0.as_ptr();

rust/kernel/types.rs +14 -0

+impl<T: 'static, A> ForeignOwnable for crate::alloc::Box<T, A>
+where
+    A: Allocator,
+{
+    type Borrowed<'a> = &'a T;
+
+    fn into_foreign(self) -> *const core::ffi::c_void {
+        crate::alloc::Box::into_raw(self) as _
+    }
+
+    unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> &'a T {
+        // SAFETY: The safety requirements for this function ensure that the object is still alive,
+        // so it is safe to dereference the raw pointer.
+        // The safety requirements of `from_foreign` also ensure that the object remains alive for

Quoted 15 lines · Danilo Krummrich

Box provides the simplest way to allocate memory for a generic type with one of the kernel's allocators, e.g. Kmalloc, Vmalloc or KVmalloc.

In contrast to Rust's Box type, the kernel Box type considers the kernel's GFP flags for all appropriate functions, always reports allocation failures through Result<_, AllocError> and remains independent from unstable features.

Signed-off-by Danilo Krummrich <[email protected]>

[...]

rust/kernel/alloc/kbox.rs new file +1 -0

+    /// Constructs a `Box<T, A>` from a raw pointer.

rust/kernel/alloc/kbox.rs line 109

Quoted 10 lines

rust/kernel/alloc/kbox.rs new file +10 -0

+    ///
+    /// # Safety
+    ///
+    /// `raw` must point to valid memory, previously allocated with `A`, and at least the size of
+    /// type `T`.
+    #[inline]
+    pub const unsafe fn from_raw_alloc(raw: *mut T, alloc: PhantomData<A>) -> Self {
+        // SAFETY: Safe by the requirements of this function.
+        Box(unsafe { Unique::new_unchecked(raw) }, alloc)
+    }

I don't think it makes sense to take the PhantomData as a parameter. You can always create a PhantomData value out of thin air.

Box(unsafe { Unique::new_unchecked(raw) }, PhantomData)

rust/kernel/alloc/kbox.rs line 117

Quoted 7 lines

rust/kernel/alloc/kbox.rs new file +7 -0

+    /// Consumes the `Box<T, A>`, returning a wrapped raw pointer and `PhantomData` of the allocator
+    /// it was allocated with.
+    pub fn into_raw_alloc(b: Self) -> (*mut T, PhantomData<A>) {
+        let b = ManuallyDrop::new(b);
+        let alloc = unsafe { ptr::read(&b.1) };
+        (b.0.as_ptr(), alloc)
+    }

I don't think there's any need to have this function. The caller can always create the PhantomData themselves. I would just keep into_raw only.

rust/kernel/alloc/kbox.rs line 169

Quoted 11 lines

rust/kernel/alloc/kbox.rs new file +10 -0

+    /// Converts a `Box<T>` into a `Pin<Box<T>>`.
+    #[inline]
+    pub fn into_pin(b: Self) -> Pin<Self>
+    where
+        A: 'static,
+    {
+        // SAFETY: It's not possible to move or replace the insides of a `Pin<Box<T>>` when
+        // `T: !Unpin`, so it's safe to pin it directly without any additional requirements.
+        unsafe { Pin::new_unchecked(b) }
+    }

In the standard library, this functionality is provided using the From trait rather than an inherent method. I think it makes sense to match std here.

Quoted 15 lines

rust/kernel/alloc/kbox.rs new file +14 -0

+impl<T, A> Drop for Box<T, A>
+where
+    T: ?Sized,
+    A: Allocator,
+{
+    fn drop(&mut self) {
+        let ptr = self.0.as_ptr();
+
+        // SAFETY: We need to drop `self.0` in place, before we free the backing memory.
+        unsafe { core::ptr::drop_in_place(ptr) };
+
+        // SAFETY: `ptr` is always properly aligned, dereferenceable and points to an initialized
+        // instance of `T`.
+        if unsafe { core::mem::size_of_val(&*ptr) } != 0 {

rust/kernel/alloc/kbox.rs line 324

Quoted 3 lines

rust/kernel/alloc/kbox.rs new file +3 -0

+            // SAFETY: `ptr` was previously allocated with `A`.
+            unsafe { A::free(self.0.as_non_null().cast()) };
+        }

You just destroyed the value by calling drop_in_place, so ptr no longer points at an initialized instance of T. Please compute whether the allocation has non-zero size before you call drop_in_place.

Also, in normal Rust this code would leak the allocation on panic in the destructor. We may not care, but it's worth taking into account if anybody else copies this code to a different project with a different panic configuration.

Quoted 20 lines

rust/kernel/types.rs +19 -0

+impl<T: 'static, A> ForeignOwnable for crate::alloc::Box<T, A>
+where
+    A: Allocator,
+{
+    type Borrowed<'a> = &'a T;
+
+    fn into_foreign(self) -> *const core::ffi::c_void {
+        crate::alloc::Box::into_raw(self) as _
+    }
+
+    unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> &'a T {
+        // SAFETY: The safety requirements for this function ensure that the object is still alive,
+        // so it is safe to dereference the raw pointer.
+        // The safety requirements of `from_foreign` also ensure that the object remains alive for
+        // the lifetime of the returned value.
+        unsafe { &*ptr.cast() }
+    }
+
+    unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self {

rust/kernel/types.rs unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self { · line 116

Quoted 5 lines

rust/kernel/types.rs +5 -0

+        // SAFETY: The safety requirements of this function ensure that `ptr` comes from a previous
+        // call to `Self::into_foreign`.
+        unsafe { crate::alloc::Box::from_raw(ptr as _) }
+    }
+}

You may want to also implement ForeignOwnable for Pin<Box<T>>. See:
https://lore.kernel.org/all/[email protected]/

Alice

Quoted 12 lines · Alice Ryhl

Quoted 10 lines · Danilo Krummrich

rust/kernel/alloc/kbox.rs new file +10 -0

+    /// Converts a `Box<T>` into a `Pin<Box<T>>`.
+    #[inline]
+    pub fn into_pin(b: Self) -> Pin<Self>
+    where
+        A: 'static,
+    {
+        // SAFETY: It's not possible to move or replace the insides of a `Pin<Box<T>>` when
+        // `T: !Unpin`, so it's safe to pin it directly without any additional requirements.
+        unsafe { Pin::new_unchecked(b) }
+    }

Quoted 3 lines

In the standard library, this functionality is provided using the From trait rather than an inherent method. I think it makes sense to match std here.

I already provide impl<T, A> From<Box<T, A>> for Pin<Box<T, A>> in this patch, which just calls Box::into_pin.

Quoted 19 lines

Quoted 19 lines

rust/kernel/types.rs +19 -0

+impl<T: 'static, A> ForeignOwnable for crate::alloc::Box<T, A>
+where
+    A: Allocator,
+{
+    type Borrowed<'a> = &'a T;
+
+    fn into_foreign(self) -> *const core::ffi::c_void {
+        crate::alloc::Box::into_raw(self) as _
+    }
+
+    unsafe fn borrow<'a>(ptr: *const core::ffi::c_void) -> &'a T {
+        // SAFETY: The safety requirements for this function ensure that the object is still alive,
+        // so it is safe to dereference the raw pointer.
+        // The safety requirements of `from_foreign` also ensure that the object remains alive for
+        // the lifetime of the returned value.
+        unsafe { &*ptr.cast() }
+    }
+
+    unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self {

Quoted 8 lines

rust/kernel/types.rs unsafe fn from_foreign(ptr: *const core::ffi::c_void) -> Self { · line 116

Quoted 5 lines

rust/kernel/types.rs +5 -0

+        // SAFETY: The safety requirements of this function ensure that `ptr` comes from a previous
+        // call to `Self::into_foreign`.
+        unsafe { crate::alloc::Box::from_raw(ptr as _) }
+    }
+}

You may want to also implement ForeignOwnable for Pin<Box<T>>. See:
https://lore.kernel.org/all/[email protected]/

Yeah, I think I've also seen another patch that it about to add a function to convert a Box back into uninit state.

Depending how fast you need ForeignOwnable for Pin<Box<T>>, do you prefer to contribute a corresponding patch to this series?