Rust infrastructure for sg_table and scatterlist
Subsystems (5)
Danilo Krummrich
maintainerv4 cover Rust infrastructure for sg_table and scatterlist
This patch series provides abstractions for struct sg_table and struct scatterlist.
Abdiel and me agreed for me to take over his previous iterations on this topic. I decided to send my patches as a new series rather than as a subsequent version of Abdiel's previous iterations, since the changes I made turned out to be much closer to a full rewrite.
The most notable differences in design are:
- SGTable utilizes BorrowedPage, AsPageIter and VmallocPageIter from my patch
series in [1].
- SGTable is a transparent wrapper over either struct Owned<P> (where P is the provider of the backing pages) or struct Borrowed, which by itself is a transparent wrapper over Opaque<bindings::sg_table>, i.e. either SGTable<Owned<P>> or just SGTable (which is equivalent to SGTable<Borrowed>.
- SGTable<Owned<P>>: Represents a table whose resources are fully managed by Rust. It takes ownership of a page provider P, allocates the underlying struct sg_table, maps it for DMA, and handles all cleanup automatically upon drop. The DMA mapping's lifetime is tied to the associated device using Devres, ensuring it is correctly unmapped before the device is unbound.
- SGTable<Borrowed> (or just SGTable): A zero-cost representation of an externally managed struct sg_table. It is created from a raw pointer using SGTable::from_raw() and provides a lifetime-bound reference (&'a SGTable) for operations like iteration.
- As a consequence, a borrowed SG table can be created with SGTable::from_raw(), which returns a &'a SGTable, just like similar existing abstractions.
An owned SGTable is created with SGTable::new(), which returns an impl PinInit<SGTable<Owned<P>>, Error>, such that it can be initialized directly within existing private data memory allocations while providing the required pin guarantees.
- SGTable<Owned<P>> uses an inner type Devres<DmaMapSgt> to ensure that the DMA mapping can't out-live device unbind.
- SGTable<Owned<P>> uses pin-init for initialization.
This patch series depends on [1] (branch containing the patches in [2]). A branch containing this series (including dependencies) can be found in [3]; Abdiel's latest series can be found in [4].
[1] https://lore.kernel.org/rust-for-linux/[email protected]/
[2] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=page-iter
[3] https://git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=scatterlist
[4] https://lore.kernel.org/lkml/[email protected]/
Changes in v4:
- Replace "type state" leftovers.
- Add a brief explanation to RawSGTable.
- Mention "no interior mutability" in Sync safety comments.
- Add safety requirement to RawSGTable::new().
- Don't use pin-init for RawSGTable itself.
Changes in v3:
- Beautify max_segment assignment code.
- Rename DmaMapSg to DmaMappedSg and improve documentation.
- Rename SGTable::as_iter() into SGTable::iter() and remove IntoIterator impl.
- Consider struct sg_table::nents in SGTable::iter() and SGTableIter<'_>.
Changes in v2:
- Switch to an enum impl for DmaDirection utilizing compile time boundary checks.
- Add missing Send/ Sync impls.
- Rename as_ref() to from_raw().
- Add a bunch of inline annotations.
- Add a patch to introduce a typedef for dma_addr_t.
- Let dma_len() return ResourceSize.
- Add addional invariant to DmaMapSgt.
- In RawSGTable::new(), pass pages as mutable slice reference.
- Avoid casts when deriving max_segment in Owned::new().
Danilo Krummrich (5):
rust: dma: implement DataDirection
rust: dma: add type alias for bindings::dma_addr_t
rust: scatterlist: Add abstraction for sg_table
samples: rust: dma: add sample code for SGTable
MAINTAINERS: rust: dma: add scatterlist files
MAINTAINERS | 4 +- drivers/gpu/nova-core/falcon.rs | 4 +- rust/bindings/bindings_helper.h | 1 + rust/helpers/helpers.c | 1 + rust/helpers/scatterlist.c | 24 ++ rust/kernel/dma.rs | 86 +++++- rust/kernel/lib.rs | 1 + rust/kernel/scatterlist.rs | 491 ++++++++++++++++++++++++++++++++ samples/rust/rust_dma.rs | 35 ++- 9 files changed, 631 insertions(+), 16 deletions(-) create mode 100644 rust/helpers/scatterlist.c create mode 100644 rust/kernel/scatterlist.rs
base-commit: 27941214d368f3c17ed26a72662fc453bcc81b9d
Signature · 1 line
2.51.0
Danilo Krummrich
maintainerPATCH v4 1/5 rust: dma: implement DataDirection
Add the DataDirection struct, a newtype wrapper around the C enum dma_data_direction.
This provides a type-safe Rust interface for specifying the direction of DMA transfers.
--- rust/bindings/bindings_helper.h | 1 + rust/kernel/dma.rs | 68 +++++++++++++++++++++++++++++++++ 2 files changed, 69 insertions(+)
rust/bindings/bindings_helper.h +1 -0
@@ -47,6 +47,7 @@#include <linux/cpumask.h> #include <linux/cred.h> #include <linux/device/faux.h> +#include <linux/dma-direction.h> #include <linux/dma-mapping.h> #include <linux/errname.h> #include <linux/ethtool.h>
rust/kernel/dma.rs +68 -0
74 lines not on this page · raw message
Signature · 1 line
2.51.0
Danilo Krummrich
maintainerPATCH v4 2/5 rust: dma: add type alias for bindings::dma_addr_t
In reply to Danilo KrummrichAdd a type alias for bindings::dma_addr_t (DmaAddress), such that we do not have to access bindings directly.
--- drivers/gpu/nova-core/falcon.rs | 4 ++-- rust/kernel/dma.rs | 18 ++++++++++++++---- 2 files changed, 16 insertions(+), 6 deletions(-)
drivers/gpu/nova-core/falcon.rs +2 -2
@@ -4,8 +4,8 @@use core::ops::Deref ;use hal::FalconHal ; -use kernel::bindings;use kernel::device; +use kernel::dma::DmaAddress ;use kernel::prelude::*;use kernel::time::Delta ;use kernel::types::ARef ;
@@ -443,7 +443,7 @@ fn dma_wr<F: FalconFirmware<Target = E>>( fw.dma_handle_with_offset(load_offsets.src_start as usize)?, ), }; -if dma_start % bindings::dma_addr_t::from (DMA_LEN ) >0 { +if dma_start %DmaAddress ::from (DMA_LEN ) >0 {dev_err! ( self.dev,"DMA transfer start addresses must be a multiple of {}" ,
rust/kernel/dma.rs +14 -4
@@ -13,6 +13,16 @@ types::ARef, }; +/// DMA address type. +/// +/// Represents a bus address used for Direct Memory Access (DMA) operations. +/// +/// This is an alias of the kernel's `dma_addr_t`, which may be `u32` or `u64` depending on +/// `CONFIG_ARCH_DMA_ADDR_T_64BIT`. +/// +/// Note that this may be `u64` even on 32-bit architectures. +pub type DmaAddress = bindings::dma_addr_t ; +/// Trait to be implemented by DMA capable bus devices. /// /// The [`dma::Device`](Device) trait should be implemented by bus specific device representations,
@@ -343,7 +353,7 @@ fn from(direction: DataDirection) -> Self {// entire `CoherentAllocation` including the allocated memory itself. pub struct CoherentAllocation <T :AsBytes +FromBytes > { dev:ARef <device::Device >, - dma_handle: bindings::dma_addr_t , + dma_handle:DmaAddress , count:usize , cpu_addr: *mut T , dma_attrs:Attrs ,
@@ -444,7 +454,7 @@ pub fn start_ptr_mut(&mut self) -> *mut T {/// Returns a DMA handle which may be given to the device as the DMA address base of /// the region. -pub fn dma_handle (&self) -> bindings::dma_addr_t { +pub fn dma_handle (&self) ->DmaAddress { self.dma_handle }
@@ -452,13 +462,13 @@ pub fn dma_handle(&self) -> bindings::dma_addr_t {/// device as the DMA address base of the region. /// /// Returns `EINVAL` if `offset` is not within the bounds of the allocation. -pub fn dma_handle_with_offset (&self, offset:usize ) ->Result <bindings::dma_addr_t > { +pub fn dma_handle_with_offset (&self, offset:usize ) ->Result <DmaAddress > {if offset >= self.count {Err (EINVAL ) }else {// INVARIANT: The type invariant of `Self` guarantees that `size_of::<T> * count` fits // into a `usize`, and `offset` is inferior to `count`. -Ok (self.dma_handle + (offset * core::mem::size_of ::<T >())as bindings::dma_addr_t ) +Ok (self.dma_handle + (offset * core::mem::size_of ::<T >())as DmaAddress ) } }
Signature · 1 line
2.51.0
Danilo Krummrich
maintainerPATCH v4 3/5 rust: scatterlist: Add abstraction for sg_table
In reply to Danilo KrummrichAdd a safe Rust abstraction for the kernel's scatter-gather list facilities (struct scatterlist and struct sg_table).
This commit introduces SGTable<T>, a wrapper that uses a generic parameter to provide compile-time guarantees about ownership and lifetime.
The abstraction provides two primary states:
- SGTable<Owned<P>>: Represents a table whose resources are fully managed by Rust. It takes ownership of a page provider P, allocates the underlying struct sg_table, maps it for DMA, and handles all cleanup automatically upon drop. The DMA mapping's lifetime is tied to the associated device using Devres, ensuring it is correctly unmapped before the device is unbound.
- SGTable<Borrowed> (or just SGTable): A zero-cost representation of an externally managed struct sg_table. It is created from a raw pointer using SGTable::from_raw() and provides a lifetime-bound reference (&'a SGTable) for operations like iteration.
The API exposes a safe iterator that yields &SGEntry references, allowing drivers to easily access the DMA address and length of each segment in the list.
--- rust/helpers/helpers.c | 1 + rust/helpers/scatterlist.c | 24 ++ rust/kernel/lib.rs | 1 + rust/kernel/scatterlist.rs | 491 +++++++++++++++++++++++++++++++++++++ 4 files changed, 517 insertions(+) create mode 100644 rust/helpers/scatterlist.c create mode 100644 rust/kernel/scatterlist.rs
rust/helpers/helpers.c +1 -0
@@ -39,6 +39,7 @@#include "rcu.c" #include "refcount.c" #include "regulator.c" +#include "scatterlist.c" #include "security.c" #include "signal.c" #include "slab.c"
rust/helpers/scatterlist.c new file +24 -0
@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/dma-direction.h> + +dma_addr_t rust_helper_sg_dma_address (struct scatterlist *sg) +{ +return sg_dma_address (sg); +} + +unsigned int rust_helper_sg_dma_len (struct scatterlist *sg) +{ +return sg_dma_len (sg); +} + +struct scatterlist *rust_helper_sg_next (struct scatterlist *sg) +{ +return sg_next (sg); +} + +void rust_helper_dma_unmap_sgtable (struct device *dev,struct sg_table *sgt, +enum dma_data_direction dir,unsigned long attrs) +{ +return dma_unmap_sgtable (dev, sgt, dir, attrs); +}
rust/kernel/lib.rs +1 -0
@@ -113,6 +113,7 @@pub mod rbtree;pub mod regulator;pub mod revocable; +pub mod scatterlist;pub mod security;pub mod seq_file;pub mod sizes;
rust/kernel/scatterlist.rs new file +491 -0
491 lines not on this page · raw message
Signature · 1 line
2.51.0
Re: [PATCH v4 3/5] rust: scatterlist: Add abstraction for sg_table
Quoted 592 lines · Danilo Krummrich
Add a safe Rust abstraction for the kernel's scatter-gather list facilities (
struct scatterlistandstruct sg_table).This commit introduces
SGTable<T>, a wrapper that uses a generic parameter to provide compile-time guarantees about ownership and lifetime.The abstraction provides two primary states:
-SGTable<Owned<P>>: Represents a table whose resources are fully managed by Rust. It takes ownership of a page providerP, allocates the underlyingstruct sg_table, maps it for DMA, and handles all cleanup automatically upon drop. The DMA mapping's lifetime is tied to the associated device usingDevres, ensuring it is correctly unmapped before the device is unbound.
-SGTable<Borrowed>(or justSGTable): A zero-cost representation of an externally managedstruct sg_table. It is created from a raw pointer usingSGTable::from_raw()and provides a lifetime-bound reference (&'a SGTable) for operations like iteration.The API exposes a safe iterator that yields
&SGEntryreferences, allowing drivers to easily access the DMA address and length of each segment in the list.Reviewed-by Alice Ryhl <[email protected]>Reviewed-by Alexandre Courbot <[email protected]>Tested-by Alexandre Courbot <[email protected]>Reviewed-by Daniel Almeida <[email protected]>Reviewed-by Lyude Paul <[email protected]>Co-developed-by Abdiel Janulgue <[email protected]>Signed-off-by Abdiel Janulgue <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>--- rust/helpers/helpers.c | 1 + rust/helpers/scatterlist.c | 24 ++ rust/kernel/lib.rs | 1 + rust/kernel/scatterlist.rs | 491 +++++++++++++++++++++++++++++++++++++ 4 files changed, 517 insertions(+) create mode 100644 rust/helpers/scatterlist.c create mode 100644 rust/kernel/scatterlist.rsrust/helpers/helpers.c +1
-0@@ -39,6 +39,7 @@#include "rcu.c" #include "refcount.c" #include "regulator.c" +#include "scatterlist.c" #include "security.c" #include "signal.c" #include "slab.c" rust/helpers/scatterlist.c new file +24
-0@@ -0,0 +1,24 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/dma-direction.h> + +dma_addr_t rust_helper_sg_dma_address (struct scatterlist *sg) +{ +return sg_dma_address (sg); +} + +unsigned int rust_helper_sg_dma_len (struct scatterlist *sg) +{ +return sg_dma_len (sg); +} + +struct scatterlist *rust_helper_sg_next (struct scatterlist *sg) +{ +return sg_next (sg); +} + +void rust_helper_dma_unmap_sgtable (struct device *dev,struct sg_table *sgt, +enum dma_data_direction dir,unsigned long attrs) +{ +return dma_unmap_sgtable (dev, sgt, dir, attrs); +}rust/kernel/lib.rs +1
-0@@ -113,6 +113,7 @@pub mod rbtree;pub mod regulator;pub mod revocable; +pub mod scatterlist;pub mod security;pub mod seq_file;pub mod sizes;rust/kernel/scatterlist.rs new file +491
-0@@ -0,0 +1,491 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Abstractions for scatter-gather lists. +//! +//! C header: [`include/linux/scatterlist.h`](srctree/include/linux/scatterlist.h) +//! +//! Scatter-gather (SG) I/O is a memory access technique that allows devices to perform DMA +//! operations on data buffers that are not physically contiguous in memory. It works by creating a +//! "scatter-gather list", an array where each entry specifies the address and length of a +//! physically contiguous memory segment. +//! +//! The device's DMA controller can then read this list and process the segments sequentially as +//! part of one logical I/O request. This avoids the need for a single, large, physically contiguous +//! memory buffer, which can be difficult or impossible to allocate. +//! +//! This module provides safe Rust abstractions over the kernel's `struct scatterlist` and +//! `struct sg_table` types. +//! +//! The main entry point is the [`SGTable`] type, which represents a complete scatter-gather table. +//! It can be either: +//! +//! - An owned table ([`SGTable<Owned<P>>`]), created from a Rust memory buffer (e.g., [`VVec`]). +//! This type manages the allocation of the `struct sg_table`, the DMA mapping of the buffer, and +//! the automatic cleanup of all resources. +//! - A borrowed reference (&[`SGTable`]), which provides safe, read-only access to a table that was +//! allocated by other (e.g., C) code. +//! +//! Individual entries in the table are represented by [`SGEntry`], which can be accessed by +//! iterating over an [`SGTable`]. + +use crate ::{ + alloc, + alloc::allocator::VmallocPageIter , + bindings, + device::{Bound ,Device }, + devres::Devres , + dma, error, + io::resource::ResourceSize , + page, + prelude::*, + types::{ARef ,Opaque }, +}; +use core::{ops::Deref , ptr::NonNull }; + +/// A single entry in a scatter-gather list. +/// +/// An `SGEntry` represents a single, physically contiguous segment of memory that has been mapped +/// for DMA. +/// +/// Instances of this struct are obtained by iterating over an [`SGTable`]. Drivers do not create +/// or own [`SGEntry`] objects directly. +#[repr(transparent)] +pub struct SGEntry (Opaque <bindings::scatterlist >); + +// SAFETY: `SGEntry` can be sent to any task. +unsafe impl Send for SGEntry {} + +// SAFETY: `SGEntry` has no interior mutability and can be accessed concurrently. +unsafe impl Sync for SGEntry {} + +impl SGEntry { +/// Convert a raw `struct scatterlist *` to a `&'a SGEntry`. +/// +/// # Safety +/// +/// Callers must ensure that the `struct scatterlist` pointed to by `ptr` is valid for the +/// lifetime `'a`. +#[inline] +unsafe fn from_raw <'a>(ptr: *mut bindings::scatterlist ) -> &'aSelf { +// SAFETY: The safety requirements of this function guarantee that `ptr` is a valid pointer +// to a `struct scatterlist` for the duration of `'a`. +unsafe { &*ptr.cast () } + } + +/// Obtain the raw `struct scatterlist *`. +#[inline] +fn as_raw (&self) -> *mut bindings::scatterlist { + self.0 .get () + } + +/// Returns the DMA address of this SG entry. +/// +/// This is the address that the device should use to access the memory segment. +#[inline] +pub fn dma_address (&self) -> dma::DmaAddress { +// SAFETY: `self.as_raw()` is a valid pointer to a `struct scatterlist`. +unsafe { bindings::sg_dma_address (self.as_raw ()) } + } + +/// Returns the length of this SG entry in bytes. +#[inline] +pub fn dma_len (&self) ->ResourceSize { +#[allow(clippy::useless_conversion)] +// SAFETY: `self.as_raw()` is a valid pointer to a `struct scatterlist`. +unsafe { bindings::sg_dma_len (self.as_raw ()) }.into () + } +} + +/// The borrowed generic type of an [`SGTable`], representing a borrowed or externally managed +/// table. +#[repr(transparent)] +pub struct Borrowed (Opaque <bindings::sg_table >); + +// SAFETY: `Borrowed` can be sent to any task. +unsafe impl Send for Borrowed {} + +// SAFETY: `Borrowed` has no interior mutability and can be accessed concurrently. +unsafe impl Sync for Borrowed {} + +/// A scatter-gather table. +/// +/// This struct is a wrapper around the kernel's `struct sg_table`. It manages a list of DMA-mapped +/// memory segments that can be passed to a device for I/O operations. +/// +/// The generic parameter `T` is used as a generic type to distinguish between owned and borrowed +/// tables. +/// +/// - [`SGTable<Owned>`]: An owned table created and managed entirely by Rust code. It handles +/// allocation, DMA mapping, and cleanup of all associated resources. See [`SGTable::new`]. +/// - [`SGTable<Borrowed>`} (or simply [`SGTable`]): Represents a table whose lifetime is managed +/// externally. It can be used safely via a borrowed reference `&'a SGTable`, where `'a` is the +/// external lifetime. +/// +/// All [`SGTable`] variants can be iterated over the individual [`SGEntry`]s. +#[repr(transparent)] +#[pin_data] +pub struct SGTable <T : private::Sealed =Borrowed > { +#[pin] + inner:T , +} + +impl SGTable { +/// Creates a borrowed `&'a SGTable` from a raw `struct sg_table` pointer. +/// +/// This allows safe access to an `sg_table` that is managed elsewhere (for example, in C code). +/// +/// # Safety +/// +/// Callers must ensure that: +/// +/// - the `struct sg_table` pointed to by `ptr` is valid for the entire lifetime of `'a`, +/// - the data behind `ptr` is not modified concurrently for the duration of `'a`. +#[inline] +pub unsafe fn from_raw <'a>(ptr: *mut bindings::sg_table ) -> &'aSelf { +// SAFETY: The safety requirements of this function guarantee that `ptr` is a valid pointer +// to a `struct sg_table` for the duration of `'a`. +unsafe { &*ptr.cast () } + } + +#[inline] +fn as_raw (&self) -> *mut bindings::sg_table { + self.inner.0 .get () + } + +/// Returns an [`SGTableIter`] bound to the lifetime of `self`. +pub fn iter (&self) ->SGTableIter <'_> { +// SAFETY: `self.as_raw()` is a valid pointer to a `struct sg_table`. +let nents =unsafe { (*self.as_raw ()).nents }; + +let pos =if nents >0 { +// SAFETY: `self.as_raw()` is a valid pointer to a `struct sg_table`. +let ptr =unsafe { (*self.as_raw ()).sgl }; + +// SAFETY: `ptr` is guaranteed to be a valid pointer to a `struct scatterlist`. +Some (unsafe {SGEntry ::from_raw (ptr) }) + }else { +None + }; + +SGTableIter { pos, nents } + } +} + +/// Represents the DMA mapping state of a `struct sg_table`. +/// +/// This is used as an inner type of [`Owned`] to manage the DMA mapping lifecycle. +/// +/// # Invariants +/// +/// - `sgt` is a valid pointer to a `struct sg_table` for the entire lifetime of the +/// [`DmaMappedSgt`]. +/// - `sgt` is always DMA mapped. +struct DmaMappedSgt { + sgt:NonNull <bindings::sg_table >, + dev:ARef <Device >, + dir: dma::DataDirection , +} + +// SAFETY: `DmaMappedSgt` can be sent to any task. +unsafe impl Send for DmaMappedSgt {} + +// SAFETY: `DmaMappedSgt` has no interior mutability and can be accessed concurrently. +unsafe impl Sync for DmaMappedSgt {} + +impl DmaMappedSgt { +/// # Safety +/// +/// - `sgt` must be a valid pointer to a `struct sg_table` for the entire lifetime of the +/// returned [`DmaMappedSgt`]. +/// - The caller must guarantee that `sgt` remains DMA mapped for the entire lifetime of +/// [`DmaMappedSgt`]. +unsafe fn new ( + sgt:NonNull <bindings::sg_table >, + dev: &Device <Bound >, + dir: dma::DataDirection , + ) ->Result <Self > { +// SAFETY: +// - `dev.as_raw()` is a valid pointer to a `struct device`, which is guaranteed to be +// bound to a driver for the duration of this call. +// - `sgt` is a valid pointer to a `struct sg_table`. + error::to_result (unsafe { + bindings::dma_map_sgtable (dev.as_raw (), sgt.as_ptr (), dir.into (),0 ) + })?; + +// INVARIANT: By the safety requirements of this function it is guaranteed that `sgt` is +// valid for the entire lifetime of this object instance. +Ok (Self { + sgt, + dev: dev.into (), + dir, + }) + } +} + +impl Drop for DmaMappedSgt { +#[inline] +fn drop (&mut self) { +// SAFETY: +// - `self.dev.as_raw()` is a pointer to a valid `struct device`. +// - `self.dev` is the same device the mapping has been created for in `Self::new()`. +// - `self.sgt.as_ptr()` is a valid pointer to a `struct sg_table` by the type invariants +// of `Self`. +// - `self.dir` is the same `dma::DataDirection` the mapping has been created with in +// `Self::new()`. +unsafe { + bindings::dma_unmap_sgtable (self.dev.as_raw (), self.sgt.as_ptr (), self.dir.into (),0 ) + }; + } +} + +/// A transparent wrapper around a `struct sg_table`. +/// +/// While we could also create the `struct sg_table` in the constructor of [`Owned`], we can't tear +/// down the `struct sg_table` in [`Owned::drop`]; the drop order in [`Owned`] matters. +#[repr(transparent)] +struct RawSGTable (Opaque <bindings::sg_table >); + +// SAFETY: `RawSGTable` can be sent to any task. +unsafe impl Send for RawSGTable {} + +// SAFETY: `RawSGTable` has no interior mutability and can be accessed concurrently. +unsafe impl Sync for RawSGTable {} + +impl RawSGTable { +/// # Safety +/// +/// - `pages` must be a slice of valid `struct page *`. +/// - The pages pointed to by `pages` must remain valid for the entire lifetime of the returned +/// [`RawSGTable`]. +unsafe fn new ( + pages: &mut [*mut bindings::page ], + size:usize , + max_segment:u32 , + flags: alloc::Flags , + ) ->Result <Self > { +// `sg_alloc_table_from_pages_segment()` expects at least one page, otherwise it +// produces a NPE. +if pages.is_empty () { +return Err (EINVAL ); + } + +let sgt =Opaque ::zeroed (); +// SAFETY: +// - `sgt.get()` is a valid pointer to uninitialized memory. +// - As by the check above, `pages` is not empty. + error::to_result (unsafe { + bindings::sg_alloc_table_from_pages_segment ( + sgt.get (), + pages.as_mut_ptr (), + pages.len ().try_into ()?, +0 , + size, + max_segment, + flags.as_raw (), + ) + })?; + +Ok (Self (sgt)) + } + +#[inline] +fn as_raw (&self) -> *mut bindings::sg_table { + self.0 .get () + } +} + +impl Drop for RawSGTable { +#[inline] +fn drop (&mut self) { +// SAFETY: `sgt` is a valid and initialized `struct sg_table`. +unsafe { bindings::sg_free_table (self.0 .get ()) }; + } +} + +/// The [`Owned`] generic type of an [`SGTable`]. +/// +/// A [`SGTable<Owned>`] signifies that the [`SGTable`] owns all associated resources: +/// +/// - The backing memory pages. +/// - The `struct sg_table` allocation (`sgt`). +/// - The DMA mapping, managed through a [`Devres`]-managed `DmaMappedSgt`. +/// +/// Users interact with this type through the [`SGTable`] handle and do not need to manage +/// [`Owned`] directly. +#[pin_data] +pub struct Owned <P > { +// Note: The drop order is relevant; we first have to unmap the `struct sg_table`, then free the +// `struct sg_table` and finally free the backing pages. +#[pin] + dma:Devres <DmaMappedSgt >, + sgt:RawSGTable , + _pages:P , +} + +// SAFETY: `Owned` can be sent to any task if `P` can be send to any task. +unsafe impl <P :Send >Send for Owned <P > {} + +// SAFETY: `Owned` has no interior mutability and can be accessed concurrently if `P` can be +// accessed concurrently. +unsafe impl <P :Sync >Sync for Owned <P > {} + +impl <P >Owned <P > +where +for <'a>P : page::AsPageIter <Iter <'a> =VmallocPageIter <'a>> + 'static, +{ +fn new ( + dev: &Device <Bound >, +mut pages:P , + dir: dma::DataDirection , + flags: alloc::Flags , + ) ->Result <impl PinInit <Self ,Error > + '_> { +let page_iter = pages.page_iter (); +let size = page_iter.size (); + +let mut page_vec:KVec <*mut bindings::page > = +KVec ::with_capacity (page_iter.page_count (), flags)?; + +for pagein page_iter { + page_vec.push (page.as_ptr (), flags)?; + } + +// `dma_max_mapping_size` returns `size_t`, but `sg_alloc_table_from_pages_segment()` takes +// an `unsigned int`. +// +// SAFETY: `dev.as_raw()` is a valid pointer to a `struct device`. +let max_segment =match unsafe { bindings::dma_max_mapping_size (dev.as_raw ()) } { +0 => u32::MAX , + max_segment => u32::try_from (max_segment).unwrap_or (u32::MAX ), + }; + +Ok (try_pin_init! (&this inSelf { +// SAFETY: +// - `page_vec` is a `KVec` of valid `struct page *` obtained from `pages`. +// - The pages contained in `pages` remain valid for the entire lifetime of the +// `RawSGTable`. + sgt:unsafe {RawSGTable ::new (&mut page_vec, size, max_segment, flags) }?, + dma <- { +// SAFETY: `this` is a valid pointer to uninitialized memory. +let sgt =unsafe { &raw mut (*this.as_ptr ()).sgt }.cast (); + +// SAFETY: `sgt` is guaranteed to be non-null. +let sgt =unsafe {NonNull ::new_unchecked (sgt) }; + +// SAFETY: +// - It is guaranteed that the object returned by `DmaMappedSgt::new` won't out-live +// `sgt`. +// - `sgt` is never DMA unmapped manually. +Devres ::new (dev,unsafe {DmaMappedSgt ::new (sgt, dev, dir) }) + }, + _pages: pages, + })) + } +} + +impl <P >SGTable <Owned <P >> +where +for <'a>P : page::AsPageIter <Iter <'a> =VmallocPageIter <'a>> + 'static, +{ +/// Allocates a new scatter-gather table from the given pages and maps it for DMA. +/// +/// This constructor creates a new [`SGTable<Owned>`] that takes ownership of `P`. +/// It allocates a `struct sg_table`, populates it with entries corresponding to the physical +/// pages of `P`, and maps the table for DMA with the specified [`Device`] and +/// [`dma::DataDirection`]. +/// +/// The DMA mapping is managed through [`Devres`], ensuring that the DMA mapping is unmapped +/// once the associated [`Device`] is unbound, or when the [`SGTable<Owned>`] is dropped. +/// +/// # Parameters +/// +/// * `dev`: The [`Device`] that will be performing the DMA. +/// * `pages`: The entity providing the backing pages. It must implement [`page::AsPageIter`]. +/// The ownership of this entity is moved into the new [`SGTable<Owned>`]. +/// * `dir`: The [`dma::DataDirection`] of the DMA transfer. +/// * `flags`: Allocation flags for internal allocations (e.g., [`GFP_KERNEL`]). +/// +/// # Examples +/// +/// ``` +/// use kernel::{ +/// device::{Bound, Device}, +/// dma, page, +/// prelude::*, +/// scatterlist::{SGTable, Owned}, +/// }; +/// +/// fn test(dev: &Device<Bound>) -> Result { +/// let size = 4 * page::PAGE_SIZE; +/// let pages = VVec::<u8>::with_capacity(size, GFP_KERNEL)?; +/// +/// let sgt = KBox::pin_init(SGTable::new( +/// dev, +/// pages, +/// dma::DataDirection::ToDevice, +/// GFP_KERNEL, +/// ), GFP_KERNEL)?; +/// +/// Ok(()) +/// } +/// ``` +pub fn new ( + dev: &Device <Bound >, + pages:P , + dir: dma::DataDirection , + flags: alloc::Flags , + ) ->impl PinInit <Self ,Error > + '_ { +try_pin_init! (Self { + inner <-Owned ::new (dev, pages, dir, flags)? + }) + } +} + +impl <P >Deref for SGTable <Owned <P >> { +type Target =SGTable ; + +#[inline] +fn deref (&self) -> &Self ::Target { +// SAFETY: +// - `self.inner.sgt.as_raw()` is a valid pointer to a `struct sg_table` for the entire +// lifetime of `self`. +// - The backing `struct sg_table` is not modified for the entire lifetime of `self`. +unsafe {SGTable ::from_raw (self.inner.sgt.as_raw ()) } + } +} + +mod private { +pub trait Sealed {} + +impl Sealed for super ::Borrowed {} +impl <P >Sealed for super ::Owned <P > {} +} + +/// An [`Iterator`] over the DMA mapped [`SGEntry`] items of an [`SGTable`]. +/// +/// Note that the existence of an [`SGTableIter`] does not guarantee that the [`SGEntry`] items +/// actually remain DMA mapped; they are prone to be unmapped on device unbind. +pub struct SGTableIter <'a> { + pos:Option <&'aSGEntry >, +/// The number of DMA mapped entries in a `struct sg_table`. + nents:c_uint , +} + +impl <'a>Iterator for SGTableIter <'a> { +type Item = &'aSGEntry ; + +fn next (&mut self) ->Option <Self ::Item > { +let entry = self.pos?; + self.nents = self.nents.saturating_sub (1 ); + +// SAFETY: `entry.as_raw()` is a valid pointer to a `struct scatterlist`. +let next =unsafe { bindings::sg_next (entry.as_raw ()) }; + + self.pos = (!next.is_null () && self.nents >0 ).then (|| { +// SAFETY: If `next` is not NULL, `sg_next()` guarantees to return a valid pointer to +// the next `struct scatterlist`. +unsafe {SGEntry ::from_raw (next) } + }); + +Some (entry) + } +}Signature · 1 line
2.51.0
Verified on Tyr, which is relying on SGTable<Borrowed>. (that implies limited scope of testing as far as Tyr is being concerned).
Thank you for the patches!
---
BR
Beata
Quoted 2 lines
Danilo Krummrich
maintainerPATCH v4 4/5 samples: rust: dma: add sample code for SGTable
In reply to Danilo KrummrichAdd sample code for allocating and mapping a scatter-gather table (SGTable).
--- samples/rust/rust_dma.rs | 35 ++++++++++++++++++++++++++--------- 1 file changed, 26 insertions(+), 9 deletions(-)
samples/rust/rust_dma.rs +26 -9
@@ -7,15 +7,19 @@use kernel::{ bindings, device::Core , - dma::{CoherentAllocation ,Device ,DmaMask }, - pci, + dma::{CoherentAllocation ,DataDirection ,Device ,DmaMask }, + page, pci, prelude::*, + scatterlist::{Owned ,SGTable }, types::ARef , }; +#[pin_data( PinnedDrop )] struct DmaSampleDriver { pdev:ARef <pci::Device >, ca:CoherentAllocation <MyStruct >, +#[pin] + sgt:SGTable <Owned <VVec <u8 >>>, } const TEST_VALUES: [(u32, u32); 5] = [
@@ -70,21 +74,30 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Self kernel::dma_write!(ca[i] =MyStruct ::new (value.0 , value.1 ))?; } -let drvdata =KBox ::new ( -Self { +let size =4 * page::PAGE_SIZE ; +let pages =VVec ::with_capacity (size,GFP_KERNEL )?; + +let sgt =SGTable ::new (pdev.as_ref (), pages,DataDirection ::ToDevice ,GFP_KERNEL ); + +let drvdata =KBox ::pin_init ( +try_pin_init! (Self { pdev: pdev.into (), ca, - }, + sgt <- sgt, + }),GFP_KERNEL , )?; -Ok (drvdata.into ()) +Ok (drvdata) } } -impl Drop for DmaSampleDriver { -fn drop (&mut self) { -dev_info! (self.pdev.as_ref (),"Unload DMA test driver.\n" ); +#[pinned_drop] +impl PinnedDrop for DmaSampleDriver { +fn drop (self:Pin <&mut Self >) { +let dev = self.pdev.as_ref (); + +dev_info! (dev,"Unload DMA test driver.\n" );for (i, value)in TEST_VALUES .into_iter ().enumerate () {let val0 = kernel::dma_read!(self.ca[i].h);
@@ -99,6 +112,10 @@ fn drop(&mut self) {assert_eq! (val1, value.1 ); } } + +for (i, entry)in self.sgt.iter ().enumerate () { +dev_info! (dev,"Entry[{}]: DMA address: {:#x}" , i, entry.dma_address ()); + } } }
Signature · 1 line
2.51.0
Re: [PATCH v4 4/5] samples: rust: dma: add sample code for SGTable
Quoted 9 lines · Danilo Krummrich
Add sample code for allocating and mapping a scatter-gather table (
SGTable).Reviewed-by Alexandre Courbot <[email protected]>Reviewed-by Daniel Almeida <[email protected]>Reviewed-by Lyude Paul <[email protected]>Co-developed-by Abdiel Janulgue <[email protected]>Signed-off-by Abdiel Janulgue <[email protected]>Signed-off-by Danilo Krummrich <[email protected]>
Danilo Krummrich
maintainerPATCH v4 5/5 MAINTAINERS: rust: dma: add scatterlist files
In reply to Danilo KrummrichRename the "DMA MAPPING HELPERS DEVICE DRIVER API [RUST]" maintainers entry to "DMA MAPPING & SCATTERLIST API [RUST]" and add the corresponding scatterlist files.
--- MAINTAINERS | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
MAINTAINERS +3 -1
@@ -7238,7 +7238,7 @@ F: include/linux/dma-mapping.h F: include/linux/swiotlb.h F: kernel/dma/ -DMA MAPPING HELPERS DEVICE DRIVER API [RUST] +DMA MAPPING & SCATTERLIST API [RUST] M: Abdiel Janulgue <[email protected]> M: Danilo Krummrich <[email protected]> R: Daniel Almeida <[email protected]>
@@ -7249,7 +7249,9 @@ S: Supported W: https://rust-for-linux.com T: git https://github.com/Rust-for-Linux/linux.git alloc-next F: rust/helpers/dma.c +F: rust/helpers/scatterlist.c F: rust/kernel/dma.rs +F: rust/kernel/scatterlist.rs F: samples/rust/rust_dma.rs DMA-BUF HEAPS FRAMEWORK
Signature · 1 line
2.51.0
Quoted 6 lines · Danilo Krummrich
Danilo Krummrich (5):
rust: dma: implement DataDirection
rust: dma: add type alias forbindings::dma_addr_t
rust: scatterlist: Add abstraction forsg_table
samples: rust: dma: add sample code for SGTable
MAINTAINERS: rust: dma: add scatterlist files
Applied to drm-rust-next, thanks!