Additional improvements for dma coherent allocator
Subsystems (2)
Abdiel Janulgue
reviewerv4 cover Additional improvements for dma coherent allocator
Changes since v3:
- Improve document clarity and move the range checker in as_slice/write
to a common function (Alexandre Courbot).
Link to v3: https://lore.kernel.org/lkml/[email protected]/
Changes since v2:
- Rebase update, add fix from Alexandre Courbot, commit clarifications, minor sample driver improvements in error handling.
Link to v1: https://lore.kernel.org/lkml/[email protected]/
Changes since v1:
- Pull in reviewed-by tags and include links.
- Improve error handling in rust dma sample driver.
- Clarifications in documentation.
Abdiel Janulgue (3):
rust: dma: clarify wording and be consistent in coherent
nomenclature
rust: dma: convert the read/write macros to return Result
rust: dma: add as_slice/write functions for CoherentAllocation
rust/kernel/dma.rs | 150 +++++++++++++++++++++++++++++++-------- samples/rust/rust_dma.rs | 28 ++++---- 2 files changed, 137 insertions(+), 41 deletions(-)
base-commit: 7a17bbc1d952057898cb0739e60665908fbb8c72
Signature · 1 line
2.43.0
Abdiel Janulgue
reviewerPATCH v4 1/3 rust: dma: clarify wording and be consistent in `coherent` nomenclature
In the kernel, consistent and coherent are used interchangeably for the region described in this api. Stick with coherent nomenclature to show that dma_alloc_coherent() is being used, in addition to improving the clarity in the DMA mapping attributes documentation.
--- rust/kernel/dma.rs | 10 +++++----- 1 file changed, 5 insertions(+), 5 deletions(-)
rust/kernel/dma.rs +5 -5
@@ -89,7 +89,7 @@ pub mod attrs {/// Forces contiguous allocation of the buffer in physical memory. pub const DMA_ATTR_FORCE_CONTIGUOUS :Attrs =Attrs (bindings::DMA_ATTR_FORCE_CONTIGUOUS ); -/// This is a hint to the DMA-mapping subsystem that it's probably not worth the time to try +/// Hints DMA-mapping subsystem that it's probably not worth the time to try /// to allocate memory to in a way that gives better TLB efficiency. pub const DMA_ATTR_ALLOC_SINGLE_PAGES :Attrs =Attrs (bindings::DMA_ATTR_ALLOC_SINGLE_PAGES );
@@ -97,7 +97,7 @@ pub mod attrs {/// `__GFP_NOWARN`). pub const DMA_ATTR_NO_WARN :Attrs =Attrs (bindings::DMA_ATTR_NO_WARN ); -/// Used to indicate that the buffer is fully accessible at an elevated privilege level (and +/// Indicates that the buffer is fully accessible at an elevated privilege level (and /// ideally inaccessible or at least read-only at lesser-privileged levels). pub const DMA_ATTR_PRIVILEGED :Attrs =Attrs (bindings::DMA_ATTR_PRIVILEGED ); }
@@ -105,7 +105,7 @@ pub mod attrs {/// An abstraction of the `dma_alloc_coherent` API. /// /// This is an abstraction around the `dma_alloc_coherent` API which is used to allocate and map -/// large consistent DMA regions. +/// large coherent DMA regions. /// /// A [`CoherentAllocation`] instance contains a pointer to the allocated region (in the /// processor's virtual address space) and the device address which can be given to the device
@@ -115,7 +115,7 @@ pub mod attrs {/// # Invariants /// /// For the lifetime of an instance of [`CoherentAllocation`], the `cpu_addr` is a valid pointer -/// to an allocated region of consistent memory and `dma_handle` is the DMA address base of +/// to an allocated region of coherent memory and `dma_handle` is the DMA address base of /// the region. // TODO //
@@ -138,7 +138,7 @@ pub struct CoherentAllocation<T: AsBytes + FromBytes> { }impl <T :AsBytes +FromBytes >CoherentAllocation <T > { -/// Allocates a region of `size_of::<T> * count` of consistent memory. +/// Allocates a region of `size_of::<T> * count` of coherent memory. /// /// # Examples ///
Signature · 1 line
2.43.0
Re: [PATCH v4 1/3] rust: dma: clarify wording and be consistent in `coherent` nomenclature
Quoted 7 lines · Abdiel Janulgue
In the kernel,
consistentandcoherentare used interchangeably for the region described in this api. Stick withcoherentnomenclature to show thatdma_alloc_coherent()is being used, in addition to improving the clarity in the DMA mapping attributes documentation.Reviewed-by Benno Lossin <[email protected]>Signed-off-by Abdiel Janulgue <[email protected]>
Best regards,
Andreas Hindborg
Abdiel Janulgue
reviewerPATCH v4 2/3 rust: dma: convert the read/write macros to return Result
In reply to Abdiel JanulgueWe could do better here by having the macros return Result, so that we don't have to wrap these calls in a closure for validation which is confusing.
--- rust/kernel/dma.rs | 54 +++++++++++++++++++++++----------------- samples/rust/rust_dma.rs | 28 +++++++++++---------- 2 files changed, 46 insertions(+), 36 deletions(-)
rust/kernel/dma.rs +31 -23
@@ -328,20 +328,22 @@ unsafe impl<T: AsBytes + FromBytes + Send> Send for CoherentAllocation<T> {}#[macro_export] macro_rules! dma_read { ($dma:expr, $idx: expr, $($field:tt)*) => {{ -let item = $crate::dma::CoherentAllocation ::item_from_index (&$dma, $idx)?; -// SAFETY: `item_from_index` ensures that `item` is always a valid pointer and can be -// dereferenced. The compiler also further validates the expression on whether `field` -// is a member of `item` when expanded by the macro. -unsafe { -let ptr_field = ::core::ptr::addr_of!((*item) $($field)*); - $crate::dma::CoherentAllocation ::field_read (&$dma, ptr_field) - } + (|| -> ::core::result::Result <_ , $crate::error::Error > { +let item = $crate::dma::CoherentAllocation ::item_from_index (&$dma, $idx)?; +// SAFETY: `item_from_index` ensures that `item` is always a valid pointer and can be +// dereferenced. The compiler also further validates the expression on whether `field` +// is a member of `item` when expanded by the macro. +unsafe { +let ptr_field = ::core::ptr::addr_of!((*item) $($field)*); + ::core::result::Result ::Ok ($crate::dma::CoherentAllocation ::field_read (&$dma, ptr_field)) + } + })() }}; ($dma:ident [ $idx:expr ] $($field:tt)* ) => { - $crate::dma_read!($dma, $idx, $($field)*); + $crate::dma_read!($dma, $idx, $($field)*) }; ($($dma:ident).* [ $idx:expr ] $($field:tt)* ) => { - $crate::dma_read!($($dma).*, $idx, $($field)*); + $crate::dma_read!($($dma).*, $idx, $($field)*) }; }
@@ -368,24 +370,30 @@ macro_rules! dma_read {#[macro_export] macro_rules! dma_write { ($dma:ident [ $idx:expr ] $($field:tt)*) => {{ - $crate::dma_write!($dma, $idx, $($field)*); + $crate::dma_write!($dma, $idx, $($field)*) }}; ($($dma:ident).* [ $idx:expr ] $($field:tt)* ) => {{ - $crate::dma_write!($($dma).*, $idx, $($field)*); + $crate::dma_write!($($dma).*, $idx, $($field)*) }}; ($dma:expr, $idx: expr, = $val:expr) => { -let item = $crate::dma::CoherentAllocation ::item_from_index (&$dma, $idx)?; -// SAFETY: `item_from_index` ensures that `item` is always a valid item. -unsafe { $crate::dma::CoherentAllocation ::field_write (&$dma, item, $val) } + (|| -> ::core::result::Result <_ , $crate::error::Error > { +let item = $crate::dma::CoherentAllocation ::item_from_index (&$dma, $idx)?; +// SAFETY: `item_from_index` ensures that `item` is always a valid item. +unsafe { $crate::dma::CoherentAllocation ::field_write (&$dma, item, $val) } + ::core::result::Result ::Ok (()) + })() }; ($dma:expr, $idx: expr, $(.$field:ident)* = $val:expr) => { -let item = $crate::dma::CoherentAllocation ::item_from_index (&$dma, $idx)?; -// SAFETY: `item_from_index` ensures that `item` is always a valid pointer and can be -// dereferenced. The compiler also further validates the expression on whether `field` -// is a member of `item` when expanded by the macro. -unsafe { -let ptr_field = ::core::ptr::addr_of_mut!((*item) $(.$field)*); - $crate::dma::CoherentAllocation ::field_write (&$dma, ptr_field, $val) - } + (|| -> ::core::result::Result <_ , $crate::error::Error > { +let item = $crate::dma::CoherentAllocation ::item_from_index (&$dma, $idx)?; +// SAFETY: `item_from_index` ensures that `item` is always a valid pointer and can be +// dereferenced. The compiler also further validates the expression on whether `field` +// is a member of `item` when expanded by the macro. +unsafe { +let ptr_field = ::core::ptr::addr_of_mut!((*item) $(.$field)*); + $crate::dma::CoherentAllocation ::field_write (&$dma, ptr_field, $val) + } + ::core::result::Result ::Ok (()) + })() }; }
samples/rust/rust_dma.rs +15 -13
@@ -54,13 +54,9 @@ fn probe(pdev: &pci::Device<Core>, _info: &Self::IdInfo) -> Result<Pin<KBox<Selflet ca:CoherentAllocation <MyStruct > =CoherentAllocation ::alloc_coherent (pdev.as_ref (),TEST_VALUES .len (),GFP_KERNEL )?; - || ->Result { -for (i, value)in TEST_VALUES .into_iter ().enumerate () { - kernel::dma_write!(ca[i] =MyStruct ::new (value.0 , value.1 )); - } - -Ok (()) - }()?; +for (i, value)in TEST_VALUES .into_iter ().enumerate () { + kernel::dma_write!(ca[i] =MyStruct ::new (value.0 , value.1 ))?; + } let drvdata = KBox::new( Self {
@@ -78,13 +74,19 @@ impl Drop for DmaSampleDriver {fn drop (&mut self) {dev_info! (self.pdev.as_ref (),"Unload DMA test driver.\n" ); -let _ = || ->Result { -for (i, value)in TEST_VALUES .into_iter ().enumerate () { -assert_eq! (kernel::dma_read!(self.ca[i].h), value.0 ); -assert_eq! (kernel::dma_read!(self.ca[i].b), value.1 ); +for (i, value)in TEST_VALUES .into_iter ().enumerate () { +let val0 = kernel::dma_read!(self.ca[i].h); +let val1 = kernel::dma_read!(self.ca[i].b); +assert! (val0.is_ok ()); +assert! (val1.is_ok ()); + +if let Ok (val0) = val0 { +assert_eq! (val0, value.0 ); + } +if let Ok (val1) = val1 { +assert_eq! (val1, value.1 ); } -Ok (()) - }(); + } } }
Signature · 1 line
2.43.0
Abdiel Janulgue
reviewerPATCH v4 3/3 rust: dma: add as_slice/write functions for CoherentAllocation
In reply to Abdiel JanulgueAdd unsafe accessors for the region for reading or writing large blocks of data.
--- rust/kernel/dma.rs | 86 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+)
rust/kernel/dma.rs +86 -0
@@ -218,6 +218,92 @@ pub fn dma_handle(&self) -> bindings::dma_addr_t { self.dma_handle } +/// Common helper to validate a range applied from the allocated region in the CPU's virtual +/// address space. +fn validate_range (&self, offset:usize , count:usize ) ->Result + { +if offset.checked_add (count).ok_or (EOVERFLOW )? > self.count { +return Err (EINVAL ); + } +Ok (()) + } + +/// Returns the data from the region starting from `offset` as a slice. +/// `offset` and `count` are in units of `T`, not the number of bytes. +/// +/// For ringbuffer type of r/w access or use-cases where the pointer to the live data is needed, +/// [`CoherentAllocation::start_ptr`] or [`CoherentAllocation::start_ptr_mut`] could be used instead. +/// +/// # Safety +/// +/// * Callers must ensure that the device does not read/write to/from memory while the returned +/// slice is live. +/// * Callers must ensure that this call does not race with a write to the same region while +/// the returned slice is live. +pub unsafe fn as_slice (&self, offset:usize , count:usize ) ->Result <&[T ]> { + self.validate_range (offset, count)?; +// SAFETY: +// - The pointer is valid due to type invariant on `CoherentAllocation`, +// we've just checked that the range and index is within bounds. The immutability of the +// data is also guaranteed by the safety requirements of the function. +// - `offset + count` can't overflow since it is smaller than `self.count` and we've checked +// that `self.count` won't overflow early in the constructor. +Ok (unsafe { core::slice::from_raw_parts (self.cpu_addr.add (offset), count) }) + } + +/// Performs the same functionality as [`CoherentAllocation::as_slice`], except that a mutable +/// slice is returned. +/// +/// # Safety +/// +/// * Callers must ensure that the device does not read/write to/from memory while the returned +/// slice is live. +/// * Callers must ensure that this call does not race with a read or write to the same region +/// while the returned slice is live. +pub unsafe fn as_slice_mut (&self, offset:usize , count:usize ) ->Result <&mut [T ]> { + self.validate_range (offset, count)?; +// SAFETY: +// - The pointer is valid due to type invariant on `CoherentAllocation`, +// we've just checked that the range and index is within bounds. The immutability of the +// data is also guaranteed by the safety requirements of the function. +// - `offset + count` can't overflow since it is smaller than `self.count` and we've checked +// that `self.count` won't overflow early in the constructor. +Ok (unsafe { core::slice::from_raw_parts_mut (self.cpu_addr.add (offset), count) }) + } + +/// Writes data to the region starting from `offset`. `offset` is in units of `T`, not the +/// number of bytes. +/// +/// # Safety +/// +/// * Callers must ensure that the device does not read/write to/from memory while the returned +/// slice is live. +/// * Callers must ensure that this call does not race with a read or write to the same region +/// that overlaps with this write. +/// +/// # Examples +/// +/// ``` +/// # fn test(alloc: &mut kernel::dma::CoherentAllocation<u8>) -> Result { +/// let somedata: [u8; 4] = [0xf; 4]; +/// let buf: &[u8] = &somedata; +/// // SAFETY: No hw operation on the device and no other r/w access to the region at this point. +/// unsafe { alloc.write(buf, 0)?; } +/// # Ok::<(), Error>(()) } +/// ``` +pub unsafe fn write (&self, src: &[T ], offset:usize ) ->Result { + self.validate_range (offset, src.len ())?; +// SAFETY: +// - The pointer is valid due to type invariant on `CoherentAllocation` +// and we've just checked that the range and index is within bounds. +// - `offset + count` can't overflow since it is smaller than `self.count` and we've checked +// that `self.count` won't overflow early in the constructor. +unsafe { + core::ptr::copy_nonoverlapping (src.as_ptr (), self.cpu_addr.add (offset), src.len ()) + }; +Ok (()) + } +/// Returns a pointer to an element from the region with bounds checking. `offset` is in /// units of `T`, not the number of bytes. ///
Signature · 1 line
2.43.0
Re: [PATCH v4 3/3] rust: dma: add as_slice/write functions for CoherentAllocation
Quoted 5 lines · Abdiel Janulgue
Add unsafe accessors for the region for reading or writing large blocks of data.
Reviewed-by Andreas Hindborg <[email protected]>Signed-off-by Abdiel Janulgue <[email protected]>
A couple remaining nits/questions below, but FWIW:
Quoted 57 lines
--- rust/kernel/dma.rs | 86 ++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 86 insertions(+)rust/kernel/dma.rs +45
-0@@ -218,6 +218,92 @@ pub fn dma_handle(&self) -> bindings::dma_addr_t { self.dma_handle } +/// Common helper to validate a range applied from the allocated region in the CPU's virtual +/// address space. +fn validate_range (&self, offset:usize , count:usize ) ->Result + { +if offset.checked_add (count).ok_or (EOVERFLOW )? > self.count { +return Err (EINVAL ); + } +Ok (()) + } + +/// Returns the data from the region starting from `offset` as a slice. +/// `offset` and `count` are in units of `T`, not the number of bytes. +/// +/// For ringbuffer type of r/w access or use-cases where the pointer to the live data is needed, +/// [`CoherentAllocation::start_ptr`] or [`CoherentAllocation::start_ptr_mut`] could be used instead. +/// +/// # Safety +/// +/// * Callers must ensure that the device does not read/write to/from memory while the returned +/// slice is live. +/// * Callers must ensure that this call does not race with a write to the same region while +/// the returned slice is live. +pub unsafe fn as_slice (&self, offset:usize , count:usize ) ->Result <&[T ]> { + self.validate_range (offset, count)?; +// SAFETY: +// - The pointer is valid due to type invariant on `CoherentAllocation`, +// we've just checked that the range and index is within bounds. The immutability of the +// data is also guaranteed by the safety requirements of the function. +// - `offset + count` can't overflow since it is smaller than `self.count` and we've checked +// that `self.count` won't overflow early in the constructor. +Ok (unsafe { core::slice::from_raw_parts (self.cpu_addr.add (offset), count) }) + } + +/// Performs the same functionality as [`CoherentAllocation::as_slice`], except that a mutable +/// slice is returned. +/// +/// # Safety +/// +/// * Callers must ensure that the device does not read/write to/from memory while the returned +/// slice is live. +/// * Callers must ensure that this call does not race with a read or write to the same region +/// while the returned slice is live. +pub unsafe fn as_slice_mut (&self, offset:usize , count:usize ) ->Result <&mut [T ]> { + self.validate_range (offset, count)?; +// SAFETY:
rust/kernel/dma.rs pub fn dma_handle(&self) -> bindings::dma_addr_t { · line 275
Quoted 10 lines
rust/kernel/dma.rs +10
-0+// - The pointer is valid due to type invariant on `CoherentAllocation`, +// we've just checked that the range and index is within bounds. The immutability of the +// data is also guaranteed by the safety requirements of the function. +// - `offset + count` can't overflow since it is smaller than `self.count` and we've checked +// that `self.count` won't overflow early in the constructor. +Ok (unsafe { core::slice::from_raw_parts_mut (self.cpu_addr.add (offset), count) }) + } + +/// Writes data to the region starting from `offset`. `offset` is in units of `T`, not the +/// number of bytes.
Reading this sentence it occured to me that offset may be ambiguous here, as in my mind it rings as being in bytes unit. How about using index throughout the file?
Quoted 9 lines
rust/kernel/dma.rs +9
-0+/// +/// # Safety +/// +/// * Callers must ensure that the device does not read/write to/from memory while the returned +/// slice is live. +/// * Callers must ensure that this call does not race with a read or write to the same region +/// that overlaps with this write. +/// +/// # Examples
rust/kernel/dma.rs pub fn dma_handle(&self) -> bindings::dma_addr_t { · line 294
Quoted 10 lines
rust/kernel/dma.rs +10
-0+/// +/// ``` +/// # fn test(alloc: &mut kernel::dma::CoherentAllocation<u8>) -> Result { +/// let somedata: [u8; 4] = [0xf; 4]; +/// let buf: &[u8] = &somedata; +/// // SAFETY: No hw operation on the device and no other r/w access to the region at this point. +/// unsafe { alloc.write(buf, 0)?; } +/// # Ok::<(), Error>(()) } +/// ``` +pub unsafe fn write (&self, src: &[T ], offset:usize ) ->Result {
Can this function be written by leveraging as_slice_mut and clone_from_slice? But doing so might require T to implement Clone, so maybe not a good idea (OTOH, aren't types implementing AsBytes implicitly Cloneable?)
Abdiel Janulgue
reviewerRe: [PATCH v4 3/3] rust: dma: add as_slice/write functions for CoherentAllocation
Quoted 9 lines · Alexandre Courbot
Quoted 5 lines · Abdiel Janulgue
Add unsafe accessors for the region for reading or writing large blocks of data.
Reviewed-by Andreas Hindborg <[email protected]>Signed-off-by Abdiel Janulgue <[email protected]>A couple remaining nits/questions below, but FWIW:
Quoted 8 lines
Reviewed-by Alexandre Courbot <[email protected]>rust/kernel/dma.rs pub fn dma_handle(&self) -> bindings::dma_addr_t { · line 275
Quoted 3 lines
rust/kernel/dma.rs +3
-0+ +/// Writes data to the region starting from `offset`. `offset` is in units of `T`, not the +/// number of bytes. Reading this sentence it occured to me that
offsetmay be ambiguous here, as in my mind it rings as being in bytes unit. How about usingindexthroughout the file?
Thanks! I don't have any strong opinion about this, I think it's enough that the subsequent paragraph makes it clear that the unit is in bytes unit? In any case, this could this be updated later after the merge?
Quoted 5 lines
Review
Quoted 2 lines
(quoted diff) +2
-0+ /// ``` + pub unsafe fn write(&self, src: &[T], offset: usize) -> Result {Can this function be written by leveraging
as_slice_mutandclone_from_slice?
using slice::clone_from_slice would enforce the length of the coherent allocation to be always the same as src data. Not sure if that is what we want. Also, instead of just a straight memcpy, this would go through a 2-step layer (a call to slice::from_raw_parts_mut and then the slice::clone_from_slice itself)?.
Regards,
Abdiel
Abdiel Janulgue
reviewerRe: [PATCH v4 3/3] rust: dma: add as_slice/write functions for CoherentAllocation
Quoted 1 line · Abdiel Janulgue
that the subsequent paragraph makes it clear that the unit is in bytes unit?
Sorry, I mean *not* in bytes unit.
Alexandre Courbot
reviewerRe: [PATCH v4 3/3] rust: dma: add as_slice/write functions for CoherentAllocation
In reply to Abdiel JanulgueQuoted 15 lines · Abdiel Janulgue
Quoted 14 lines · Alexandre Courbot
Quoted 5 lines · Abdiel Janulgue
Add unsafe accessors for the region for reading or writing large blocks of data.
Reviewed-by Andreas Hindborg <[email protected]>Signed-off-by Abdiel Janulgue <[email protected]>A couple remaining nits/questions below, but FWIW:
Reviewed-by Alexandre Courbot <[email protected]>Quoted 3 lines
rust/kernel/dma.rs +3
-0+ +/// Writes data to the region starting from `offset`. `offset` is in units of `T`, not the +/// number of bytes.
Quoted 7 lines
Quoted 3 lines
Reading this sentence it occured to me that
offsetmay be ambiguous here, as in my mind it rings as being in bytes unit. How about usingindexthroughout the file?Thanks! I don't have any strong opinion about this, I think it's enough that the subsequent paragraph makes it clear that the unit is in bytes unit? In any case, this could this be updated later after the merge?
I agree this can be its own follow-up change, especially since offset is already used elsewhere in the code and this patch is consistent with the existing nomenclature - let's fix them all together as a separate patch.
Quoted 4 lines
Quoted 4 lines
Quoted 3 lines
(quoted diff) +2
-0+ /// ``` + pub unsafe fn write(&self, src: &[T], offset: usize) -> Result {
Quoted 8 lines
Quoted 2 lines
Can this function be written by leveraging
as_slice_mutandclone_from_slice?using
slice::clone_from_slicewould enforce the length of the coherent allocation to be always the same as src data. Not sure if that is what we want. Also, instead of just a straight memcpy, this would go through a 2-step layer (a call toslice::from_raw_parts_mutand then theslice::clone_from_sliceitself)?.
Ack, thanks for the explanation!
On Mon, Jun 02, 2025 at 11:53:10AM +0300, Abdiel Janulgue wrote:
Applied to alloc-next, thanks!
Quoted 4 lines
Abdiel Janulgue (3):
rust: dma: clarify wording and be consistent incoherent
nomenclature
rust: dma: convert the read/write macros to return Result
[ Fix line length in dma_read!(). - Danilo ]
Quoted 1 line
rust: dma: add as_slice/write functions for CoherentAllocation
[ Fix line length and slightly reword safety comment in doc-test of
CoherentAllocation::write(); fix formatting issue. - Danilo ]