BorrowedPage, AsPageIter and VmallocPageIter
Subsystems (3)
Danilo Krummrich
maintainerv5 cover BorrowedPage, AsPageIter and VmallocPageIter
This patch series implements the BorrowedPage type, the AsPageIter trait and VmallocPageIter type.
AsPageIter can be implemented by any type that owns pages to allow users to borrow them through a generic interface.
For instance, this is useful to access and borrow the backing pages of allocation primitives, such as Box and Vec, backing a scatterlist.
Hence, implement AsPageIter for VBox and VVec.
Additionally, implement Vmalloc::to_page() and ArrayLayout::size(), which are dependencies of the above.
Changes in v5:
- Avoid repetition in commit message of "rust: alloc: implement VmallocPageIter".
- Change invariants of VmallocPageIter to:
- buf is a valid and [page::PAGE_SIZE] aligned pointer into a [Vmalloc] allocation.
- size is the number of bytes from buf until the end of the [Vmalloc] allocation buf points to.
- Inline some methods of VmallocPageIter.
Changes in v4:
- Rename IntoPageIter into AsPageIter.
- Put lifetime on AsPageIter::Iter instead of AsPageIter itself.
- Implement size_hint() for VmallocPageIter.
- Use PhantomData<page::BorrowedPage<'a>> instead of PhantomData<&'a u8>.
Changes in v3:
- Generalize the previous PageOwner impl of VBox and VVec in VmallocPageIter.
- Correspondingly, replace PageOwner with IntoPageIter.
Changes in v2:
- BorrowedPage
- Add link to Ownable
- Use borrow_page() in the example
- Add PageOwner, Vmalloc::to_page(), ArrayLayout, VBox, VVec patches.
Danilo Krummrich (7):
rust: page: implement BorrowedPage
rust: alloc: vmalloc: implement Vmalloc::to_page()
rust: alloc: implement VmallocPageIter
rust: page: define trait AsPageIter
rust: alloc: kbox: implement AsPageIter for VBox
rust: alloc: layout: implement ArrayLayout::size()
rust: alloc: kvec: implement AsPageIter for VVec
rust/bindings/bindings_helper.h | 1 + rust/kernel/alloc/allocator.rs | 152 ++++++++++++++++++++++++++++++++ rust/kernel/alloc/kbox.rs | 40 ++++++++- rust/kernel/alloc/kvec.rs | 40 ++++++++- rust/kernel/alloc/layout.rs | 5 ++ rust/kernel/page.rs | 87 +++++++++++++++++- 6 files changed, 322 insertions(+), 3 deletions(-)
base-commit: 8f5ae30d69d7543eee0d70083daf4de8fe15d585
Signature · 1 line
2.50.1
Danilo Krummrich
maintainerPATCH v5 1/7 rust: page: implement BorrowedPage
Currently, a Page always owns the underlying struct page.
However, sometimes a struct page may be owned by some other entity, e.g. a vmalloc allocation.
Hence, introduce BorrowedPage to support such cases, until the Ownable solution [1] lands.
This is required by the scatterlist abstractions.
--- rust/bindings/bindings_helper.h | 1 + rust/kernel/page.rs | 75 ++++++++++++++++++++++++++++++++- 2 files changed, 75 insertions(+), 1 deletion(-)
rust/bindings/bindings_helper.h +1 -0
@@ -57,6 +57,7 @@#include <linux/jiffies.h> #include <linux/jump_label.h> #include <linux/mdio.h> +#include <linux/mm.h> #include <linux/miscdevice.h> #include <linux/of_device.h> #include <linux/pci.h>
rust/kernel/page.rs +74 -1
87 lines not on this page · raw message
Signature · 1 line
2.50.1
Danilo Krummrich
maintainerPATCH v5 2/7 rust: alloc: vmalloc: implement Vmalloc::to_page()
In reply to Danilo KrummrichImplement an abstraction of vmalloc_to_page() for subsequent use in the AsPageIter implementation of VBox and VVec.
--- rust/kernel/alloc/allocator.rs | 49 ++++++++++++++++++++++++++++++++++ 1 file changed, 49 insertions(+)
rust/kernel/alloc/allocator.rs +49 -0
@@ -15,6 +15,7 @@use crate ::alloc::{AllocError ,Allocator };use crate ::bindings; +use crate ::page;use crate ::pr_warn;/// The contiguous kernel allocator.
@@ -140,6 +141,54 @@ unsafe fn realloc( } } +impl Vmalloc { +/// Convert a pointer to a [`Vmalloc`] allocation to a [`page::BorrowedPage`]. +/// +/// # Examples +/// +/// ``` +/// # use core::ptr::{NonNull, from_mut}; +/// # use kernel::{page, prelude::*}; +/// use kernel::alloc::allocator::Vmalloc; +/// +/// let mut vbox = VBox::<[u8; page::PAGE_SIZE]>::new_uninit(GFP_KERNEL)?; +/// +/// { +/// // SAFETY: By the type invariant of `Box` the inner pointer of `vbox` is non-null. +/// let ptr = unsafe { NonNull::new_unchecked(from_mut(&mut *vbox)) }; +/// +/// // SAFETY: +/// // `ptr` is a valid pointer to a `Vmalloc` allocation. +/// // `ptr` is valid for the entire lifetime of `page`. +/// let page = unsafe { Vmalloc::to_page(ptr.cast()) }; +/// +/// // SAFETY: There is no concurrent read or write to the same page. +/// unsafe { page.fill_zero_raw(0, page::PAGE_SIZE)? }; +/// } +/// # Ok::<(), Error>(()) +/// ``` +/// +/// # Safety +/// +/// - `ptr` must be a valid pointer to a [`Vmalloc`] allocation. +/// - `ptr` must remain valid for the entire duration of `'a`. +pub unsafe fn to_page <'a>(ptr:NonNull <u8 >) -> page::BorrowedPage <'a> { +// SAFETY: `ptr` is a valid pointer to `Vmalloc` memory. +let page =unsafe { bindings::vmalloc_to_page (ptr.as_ptr ().cast ()) }; + +// SAFETY: `vmalloc_to_page` returns a valid pointer to a `struct page` for a valid pointer +// to `Vmalloc` memory. +let page =unsafe {NonNull ::new_unchecked (page) }; + +// SAFETY: +// - `page` is a valid pointer to a `struct page`, given that by the safety requirements of +// this function `ptr` is a valid pointer to a `Vmalloc` allocation. +// - By the safety requirements of this function `ptr` is valid for the entire lifetime of +// `'a`. +unsafe { page::BorrowedPage ::from_raw (page) } + } +} +// SAFETY: `realloc` delegates to `ReallocFunc::call`, which guarantees that // - memory remains valid until it is explicitly freed, // - passing a pointer to a valid memory allocation is OK,
Signature · 1 line
2.50.1
Danilo Krummrich
maintainerPATCH v5 3/7 rust: alloc: implement VmallocPageIter
In reply to Danilo KrummrichIntroduce the VmallocPageIter type; an instance of VmallocPageIter may be exposed by owners of vmalloc allocations to provide borrowed access to its backing pages.
For instance, this is useful to access and borrow the backing pages of allocation primitives, such as Box and Vec, backing a scatterlist.
--- rust/kernel/alloc/allocator.rs | 103 +++++++++++++++++++++++++++++++++ 1 file changed, 103 insertions(+)
rust/kernel/alloc/allocator.rs +103 -0
@@ -10,6 +10,7 @@use super ::Flags ;use core::alloc::Layout ; +use core::marker::PhantomData ;use core::ptr;use core::ptr::NonNull ;
@@ -236,3 +237,105 @@ unsafe fn realloc(unsafe {ReallocFunc ::KVREALLOC .call (ptr, layout, old_layout, flags) } } } + +/// An [`Iterator`] of [`page::BorrowedPage`] items owned by a [`Vmalloc`] allocation. +/// +/// # Guarantees +/// +/// The pages iterated by the [`Iterator`] appear in the order as they are mapped in the CPU's +/// virtual address space ascendingly. +/// +/// # Invariants +/// +/// - `buf` is a valid and [`page::PAGE_SIZE`] aligned pointer into a [`Vmalloc`] allocation. +/// - `size` is the number of bytes from `buf` until the end of the [`Vmalloc`] allocation `buf` +/// points to. +pub struct VmallocPageIter <'a> { +/// The base address of the [`Vmalloc`] buffer. + buf:NonNull <u8 >, +/// The size of the buffer pointed to by `buf` in bytes. + size:usize , +/// The current page index of the [`Iterator`]. + index:usize , + _p:PhantomData <page::BorrowedPage <'a>>, +} + +impl <'a>Iterator for VmallocPageIter <'a> { +type Item = page::BorrowedPage <'a>; + +fn next (&mut self) ->Option <Self ::Item > { +let offset = self.index.checked_mul (page::PAGE_SIZE )?; + +// Even though `self.size()` may be smaller than `Self::page_count() * page::PAGE_SIZE`, it +// is always a number between `(Self::page_count() - 1) * page::PAGE_SIZE` and +// `Self::page_count() * page::PAGE_SIZE`, hence the check below is sufficient. +if offset < self.size () { + self.index +=1 ; + }else { +return None ; + } + +// TODO: Use `NonNull::add()` instead, once the minimum supported compiler version is +// bumped to 1.80 or later. +// +// SAFETY: `offset` is in the interval `[0, (self.page_count() - 1) * page::PAGE_SIZE]`, +// hence the resulting pointer is guaranteed to be within the same allocation. +let ptr =unsafe { self.buf.as_ptr ().add (offset) }; + +// SAFETY: `ptr` is guaranteed to be non-null given that it is derived from `self.buf`. +let ptr =unsafe {NonNull ::new_unchecked (ptr) }; + +// SAFETY: +// - `ptr` is a valid pointer to a `Vmalloc` allocation. +// - `ptr` is valid for the duration of `'a`. +Some (unsafe {Vmalloc ::to_page (ptr) }) + } + +fn size_hint (&self) -> (usize ,Option <usize >) { +let remaining = self.page_count ().saturating_sub (self.index); + + (remaining,Some (remaining)) + } +} + +impl <'a>VmallocPageIter <'a> { +/// Creates a new [`VmallocPageIter`] instance. +/// +/// # Safety +/// +/// - `buf` must be a [`page::PAGE_SIZE`] aligned pointer into a [`Vmalloc`] allocation. +/// - `buf` must be valid for at least the lifetime of `'a`. +/// - `size` must be the number of bytes from `buf` until the end of the [`Vmalloc`] allocation +/// `buf` points to. +pub unsafe fn new (buf:NonNull <u8 >, size:usize ) ->Self { +// INVARIANT: By the safety requirements, `buf` is a valid and `page::PAGE_SIZE` aligned +// pointer into a [`Vmalloc`] allocation. +Self { + buf, + size, + index:0 , + _p:PhantomData , + } + } + +/// Returns the base address of the backing [`Vmalloc`] allocation. +#[inline] +pub fn base_address (&self) ->NonNull <u8 > { + self.buf + } + +/// Returns the size of the backing [`Vmalloc`] allocation in bytes. +/// +/// Note that this is the size the [`Vmalloc`] allocation has been allocated with. Hence, this +/// number may be smaller than `[`Self::page_count`] * [`page::PAGE_SIZE`]`. +#[inline] +pub fn size (&self) ->usize { + self.size + } + +/// Returns the number of pages owned by the backing [`Vmalloc`] allocation. +#[inline] +pub fn page_count (&self) ->usize { + self.size ().div_ceil (page::PAGE_SIZE ) + } +}
Signature · 1 line
2.50.1
Danilo Krummrich
maintainerPATCH v5 4/7 rust: page: define trait AsPageIter
In reply to Danilo KrummrichThe AsPageIter trait provides a common interface for types that provide a page iterator, such as VmallocPageIter.
Subsequent patches will leverage this to let VBox and VVec provide a VmallocPageIter though this trait.
--- rust/kernel/page.rs | 12 ++++++++++++ 1 file changed, 12 insertions(+)
rust/kernel/page.rs +12 -0
@@ -103,6 +103,18 @@ fn deref(&self) -> &Self::Target { } } +/// Trait to be implemented by types which provide an [`Iterator`] implementation of +/// [`BorrowedPage`] items, such as [`VmallocPageIter`](kernel::alloc::allocator::VmallocPageIter). +pub trait AsPageIter { +/// The [`Iterator`] type, e.g. [`VmallocPageIter`](kernel::alloc::allocator::VmallocPageIter). +type Iter <'a>:Iterator <Item =BorrowedPage <'a>> +where +Self : 'a; + +/// Returns an [`Iterator`] of [`BorrowedPage`] items over all pages owned by `self`. +fn page_iter (&mut self) ->Self ::Iter <'_>; +} +/// A pointer to a page that owns the page allocation. /// /// # Invariants
Signature · 1 line
2.50.1
Danilo Krummrich
maintainerPATCH v5 5/7 rust: alloc: kbox: implement AsPageIter for VBox
In reply to Danilo KrummrichImplement AsPageIter for VBox; this allows to iterate and borrow the backing pages of a VBox. This, for instance, is useful in combination with VBox backing a scatterlist.
--- rust/kernel/alloc/kbox.rs | 40 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-)
rust/kernel/alloc/kbox.rs +39 -1
@@ -3,7 +3,7 @@//! Implementation of [`Box`]. #[allow(unused_imports)] // Used in doc comments. -use super ::allocator::{KVmalloc ,Kmalloc ,Vmalloc }; +use super ::allocator::{KVmalloc ,Kmalloc ,Vmalloc ,VmallocPageIter };use super ::{AllocError ,Allocator ,Flags };use core::alloc::Layout ;use core::borrow::{Borrow ,BorrowMut };
@@ -18,6 +18,7 @@use crate ::ffi::c_void;use crate ::init::InPlaceInit ; +use crate ::page::AsPageIter ;use crate ::types::ForeignOwnable ;use pin_init::{InPlaceWrite ,Init ,PinInit ,ZeroableOption };
@@ -598,3 +599,40 @@ fn drop(&mut self) {unsafe {A ::free (self.0 .cast (), layout) }; } } + +/// # Examples +/// +/// ``` +/// # use kernel::prelude::*; +/// use kernel::alloc::allocator::VmallocPageIter; +/// use kernel::page::{AsPageIter, PAGE_SIZE}; +/// +/// let mut vbox = VBox::new((), GFP_KERNEL)?; +/// +/// assert!(vbox.page_iter().next().is_none()); +/// +/// let mut vbox = VBox::<[u8; PAGE_SIZE]>::new_uninit(GFP_KERNEL)?; +/// +/// let page = vbox.page_iter().next().expect("At least one page should be available.\n"); +/// +/// // SAFETY: There is no concurrent read or write to the same page. +/// unsafe { page.fill_zero_raw(0, PAGE_SIZE)? }; +/// # Ok::<(), Error>(()) +/// ``` +impl <T >AsPageIter for VBox <T > { +type Iter <'a> + =VmallocPageIter <'a> +where +T : 'a; + +fn page_iter (&mut self) ->Self ::Iter <'_> { +let ptr = self.0 .cast (); +let size = core::mem::size_of ::<T >(); + +// SAFETY: +// - `ptr` is a valid pointer to the beginning of a `Vmalloc` allocation. +// - `ptr` is guaranteed to be valid for the lifetime of `'a`. +// - `size` is the size of the `Vmalloc` allocation `ptr` points to. +unsafe {VmallocPageIter ::new (ptr, size) } + } +}
Signature · 1 line
2.50.1
Danilo Krummrich
maintainerPATCH v5 6/7 rust: alloc: layout: implement ArrayLayout::size()
In reply to Danilo KrummrichProvide a convenience method for ArrayLayout to calculate the size of the ArrayLayout in bytes.
--- rust/kernel/alloc/layout.rs | 5 +++++ 1 file changed, 5 insertions(+)
rust/kernel/alloc/layout.rs +5 -0
@@ -98,6 +98,11 @@ pub const fn len(&self) -> usize {pub const fn is_empty (&self) ->bool { self.len ==0 } + +/// Returns the size of the [`ArrayLayout`] in bytes. +pub const fn size (&self) ->usize { + self.len () * core::mem::size_of ::<T >() + } }impl <T >From <ArrayLayout <T >>for Layout {
Signature · 1 line
2.50.1
Danilo Krummrich
maintainerPATCH v5 7/7 rust: alloc: kvec: implement AsPageIter for VVec
In reply to Danilo KrummrichImplement AsPageIter for VVec; this allows to iterate and borrow the backing pages of a VVec. This, for instance, is useful in combination with VVec backing a scatterlist.
--- rust/kernel/alloc/kvec.rs | 40 ++++++++++++++++++++++++++++++++++++++- 1 file changed, 39 insertions(+), 1 deletion(-)
rust/kernel/alloc/kvec.rs +39 -1
@@ -3,10 +3,11 @@//! Implementation of [`Vec`]. use super ::{ - allocator::{KVmalloc ,Kmalloc ,Vmalloc }, + allocator::{KVmalloc ,Kmalloc ,Vmalloc ,VmallocPageIter }, layout::ArrayLayout ,AllocError ,Allocator ,Box ,Flags , }; +use crate ::page::AsPageIter ;use core::{ borrow::{Borrow ,BorrowMut }, fmt,
@@ -1017,6 +1018,43 @@ fn into_iter(self) -> Self::IntoIter { } } +/// # Examples +/// +/// ``` +/// # use kernel::prelude::*; +/// use kernel::alloc::allocator::VmallocPageIter; +/// use kernel::page::{AsPageIter, PAGE_SIZE}; +/// +/// let mut vec = VVec::<u8>::new(); +/// +/// assert!(vec.page_iter().next().is_none()); +/// +/// vec.reserve(PAGE_SIZE, GFP_KERNEL)?; +/// +/// let page = vec.page_iter().next().expect("At least one page should be available.\n"); +/// +/// // SAFETY: There is no concurrent read or write to the same page. +/// unsafe { page.fill_zero_raw(0, PAGE_SIZE)? }; +/// # Ok::<(), Error>(()) +/// ``` +impl <T >AsPageIter for VVec <T > { +type Iter <'a> + =VmallocPageIter <'a> +where +T : 'a; + +fn page_iter (&mut self) ->Self ::Iter <'_> { +let ptr = self.ptr.cast (); +let size = self.layout.size (); + +// SAFETY: +// - `ptr` is a valid pointer to the beginning of a `Vmalloc` allocation. +// - `ptr` is guaranteed to be valid for the lifetime of `'a`. +// - `size` is the size of the `Vmalloc` allocation `ptr` points to. +unsafe {VmallocPageIter ::new (ptr, size) } + } +} +/// An [`Iterator`] implementation for [`Vec`] that moves elements out of a vector. /// /// This structure is created by the [`Vec::into_iter`] method on [`Vec`] (provided by the
Signature · 1 line
2.50.1
Quoted 1 line · Danilo Krummrich
Danilo Krummrich (7):
Applied to drm-rust-next, thanks!
Quoted 3 lines
rust: page: implement BorrowedPage
rust: alloc: vmalloc: implementVmalloc::to_page()
rust: alloc: implement VmallocPageIter
[ Drop VmallocPageIter::base_address(), move to allocator/iter.rs and
stub VmallocPageIter for allocator_test.rs. - Danilo ]
Quoted 4 lines
rust: page: define trait AsPageIter
rust: alloc: kbox: implement AsPageIter for VBox
rust: alloc: layout: implementArrayLayout::size()
rust: alloc: kvec: implement AsPageIter for VVec