rust: allocator: Prevents mis-aligned allocation
Subsystems (3)
PATCH 1/1 rust: allocator: Prevents mis-aligned allocation
Currently the KernelAllocator simply passes the size of the type Layout to krealloc(), and in theory the alignment requirement from the type Layout may be larger than the guarantee provided by SLAB, which means the allocated object is mis-aligned.
Fixes this by adjusting the allocation size to the nearest power of two, which SLAB always guarantees a size-aligned allocation. And because Rust guarantees that original size must be a multiple of alignment and the alignment must be a power of two, then the alignment requirement is satisfied.
---
Some more explanation:
* Layout is a data structure describing a particular memory layout, conceptionally it has two fields: align and size.
* align is guaranteed to be a power of two.
* size can be smaller than align (only when the Layout is created via
Layout::from_align_size())
* After pad_to_align(), the size is guaranteed to be a multiple of align
For more information, please see:
https://doc.rust-lang.org/stable/std/alloc/struct.Layout.html
rust/bindings/bindings_helper.h | 1 + rust/kernel/allocator.rs | 17 ++++++++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)
rust/bindings/bindings_helper.h +1 -0
@@ -15,3 +15,4 @@/* `bindgen` gets confused at certain things. */ const gfp_t BINDINGS_GFP_KERNEL =GFP_KERNEL ;const gfp_t BINDINGS___GFP_ZERO = __GFP_ZERO; +const size_t BINDINGS_ARCH_SLAB_MINALIGN =ARCH_SLAB_MINALIGN ;
rust/kernel/allocator.rs +16 -1
@@ -11,9 +11,24 @@unsafe impl GlobalAlloc for KernelAllocator {unsafe fn alloc (&self, layout:Layout ) -> *mut u8 { +// Customized layouts from `Layout::from_size_align()` can have size < align, so pads first. +let layout = layout.pad_to_align (); + +let mut size = layout.size (); + +if layout.align () > bindings::BINDINGS_ARCH_SLAB_MINALIGN { +// The alignment requirement exceeds the slab guarantee, then tries to enlarges the size +// to use the "power-of-two" size/alignment guarantee (see comments in kmalloc() for +// more information). +// +// Note that `layout.size()` (after padding) is guaranteed to be muliples of +// `layout.align()`, so `next_power_of_two` gives enough alignment guarantee. + size = size.next_power_of_two (); + } +// `krealloc()` is used instead of `kmalloc()` because the latter is // an inline function and cannot be bound to as a result. -unsafe { bindings::krealloc (ptr::null (), layout.size (), bindings::GFP_KERNEL )as *mut u8 } +unsafe { bindings::krealloc (ptr::null (), size, bindings::GFP_KERNEL )as *mut u8 } }unsafe fn dealloc (&self, ptr: *mut u8 , _layout:Layout ) {
Signature · 1 line
2.39.2
Quoted 11 lines · On Tue, 13 Jun 2023 09:42:58 -0700 Boqun Feng <[email protected]> wrote:
Currently the KernelAllocator simply passes the size of the type Layout to
krealloc(), and in theory the alignment requirement from the type Layout may be larger than the guarantee provided by SLAB, which means the allocated object is mis-aligned.Fixes this by adjusting the allocation size to the nearest power of two, which SLAB always guarantees a size-aligned allocation. And because Rust guarantees that original size must be a multiple of alignment and the alignment must be a power of two, then the alignment requirement is satisfied.
Quoted 5 lines
Suggested-by Vlastimil Babka <[email protected]>Co-developed-by Andreas Hindborg (Samsung) <[email protected]>Signed-off-by Andreas Hindborg (Samsung) <[email protected]>Signed-off-by Boqun Feng <[email protected]>Cc [email protected] # v6.1+
Quoted 59 lines
---
Some more explanation:* Layout is a data structure describing a particular memory layout, conceptionally it has two fields: align and size.
* align is guaranteed to be a power of two.
* size can be smaller than align (only when the Layout is created viaLayout::from_align_size())* After
pad_to_align(), the size is guaranteed to be a multiple of alignFor more information, please see:
https://doc.rust-lang.org/stable/std/alloc/struct.Layout.html
rust/bindings/bindings_helper.h | 1 + rust/kernel/allocator.rs | 17 ++++++++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)rust/bindings/bindings_helper.h +1
-0@@ -15,3 +15,4 @@/* `bindgen` gets confused at certain things. */ const gfp_t BINDINGS_GFP_KERNEL =GFP_KERNEL ;const gfp_t BINDINGS___GFP_ZERO = __GFP_ZERO; +const size_t BINDINGS_ARCH_SLAB_MINALIGN =ARCH_SLAB_MINALIGN ;rust/kernel/allocator.rs +16
-1@@ -11,9 +11,24 @@unsafe impl GlobalAlloc for KernelAllocator {unsafe fn alloc (&self, layout:Layout ) -> *mut u8 { +// Customized layouts from `Layout::from_size_align()` can have size < align, so pads first. +let layout = layout.pad_to_align (); + +let mut size = layout.size (); + +if layout.align () > bindings::BINDINGS_ARCH_SLAB_MINALIGN { +// The alignment requirement exceeds the slab guarantee, then tries to enlarges the size +// to use the "power-of-two" size/alignment guarantee (see comments in kmalloc() for +// more information). +// +// Note that `layout.size()` (after padding) is guaranteed to be muliples of +// `layout.align()`, so `next_power_of_two` gives enough alignment guarantee. + size = size.next_power_of_two (); + } +// `krealloc()` is used instead of `kmalloc()` because the latter is // an inline function and cannot be bound to as a result. -unsafe { bindings::krealloc (ptr::null (), layout.size (), bindings::GFP_KERNEL )as *mut u8 } +unsafe { bindings::krealloc (ptr::null (), size, bindings::GFP_KERNEL )as *mut u8 } }unsafe fn dealloc (&self, ptr: *mut u8 , _layout:Layout ) {
Quoted 11 lines · Boqun Feng
Currently the KernelAllocator simply passes the size of the type Layout to
krealloc(), and in theory the alignment requirement from the type Layout may be larger than the guarantee provided by SLAB, which means the allocated object is mis-aligned.Fixes this by adjusting the allocation size to the nearest power of two, which SLAB always guarantees a size-aligned allocation. And because Rust guarantees that original size must be a multiple of alignment and the alignment must be a power of two, then the alignment requirement is satisfied.
Quoted 5 lines
Suggested-by Vlastimil Babka <[email protected]>Co-developed-by Andreas Hindborg (Samsung) <[email protected]>Signed-off-by Andreas Hindborg (Samsung) <[email protected]>Signed-off-by Boqun Feng <[email protected]>Cc [email protected] # v6.1+
Quoted 11 lines · Boqun Feng
Currently the KernelAllocator simply passes the size of the type Layout to
krealloc(), and in theory the alignment requirement from the type Layout may be larger than the guarantee provided by SLAB, which means the allocated object is mis-aligned.Fixes this by adjusting the allocation size to the nearest power of two, which SLAB always guarantees a size-aligned allocation. And because Rust guarantees that original size must be a multiple of alignment and the alignment must be a power of two, then the alignment requirement is satisfied.
Quoted 5 lines
Suggested-by Vlastimil Babka <[email protected]>Co-developed-by Andreas Hindborg (Samsung) <[email protected]>Signed-off-by Andreas Hindborg (Samsung) <[email protected]>Signed-off-by Boqun Feng <[email protected]>Cc [email protected] # v6.1+
Signature · 2 lines
Cheers, Benno
Quoted 62 lines
---
Some more explanation:* Layout is a data structure describing a particular memory layout, conceptionally it has two fields: align and size.
* align is guaranteed to be a power of two.
* size can be smaller than align (only when the Layout is created viaLayout::from_align_size())* After
pad_to_align(), the size is guaranteed to be a multiple of alignFor more information, please see:
https://doc.rust-lang.org/stable/std/alloc/struct.Layout.html
rust/bindings/bindings_helper.h | 1 + rust/kernel/allocator.rs | 17 ++++++++++++++++- 2 files changed, 17 insertions(+), 1 deletion(-)rust/bindings/bindings_helper.h +1
-0@@ -15,3 +15,4 @@/* `bindgen` gets confused at certain things. */ const gfp_t BINDINGS_GFP_KERNEL =GFP_KERNEL ;const gfp_t BINDINGS___GFP_ZERO = __GFP_ZERO; +const size_t BINDINGS_ARCH_SLAB_MINALIGN =ARCH_SLAB_MINALIGN ;rust/kernel/allocator.rs +16
-2@@ -11,9 +11,24 @@unsafe impl GlobalAlloc for KernelAllocator {unsafe fn alloc (&self, layout:Layout ) -> *mut u8 { +// Customized layouts from `Layout::from_size_align()` can have size < align, so pads first. +let layout = layout.pad_to_align (); + +let mut size = layout.size (); + +if layout.align () > bindings::BINDINGS_ARCH_SLAB_MINALIGN { +// The alignment requirement exceeds the slab guarantee, then tries to enlarges the size +// to use the "power-of-two" size/alignment guarantee (see comments in kmalloc() for +// more information). +// +// Note that `layout.size()` (after padding) is guaranteed to be muliples of +// `layout.align()`, so `next_power_of_two` gives enough alignment guarantee. + size = size.next_power_of_two (); + } +// `krealloc()` is used instead of `kmalloc()` because the latter is // an inline function and cannot be bound to as a result. -unsafe { bindings::krealloc (ptr::null (), layout.size (), bindings::GFP_KERNEL )as *mut u8 } +unsafe { bindings::krealloc (ptr::null (), size, bindings::GFP_KERNEL )as *mut u8 } }unsafe fn dealloc (&self, ptr: *mut u8 , _layout:Layout ) { --2.39.2
Quoted 11 lines · Boqun Feng
Currently the KernelAllocator simply passes the size of the type Layout to
krealloc(), and in theory the alignment requirement from the type Layout may be larger than the guarantee provided by SLAB, which means the allocated object is mis-aligned.Fixes this by adjusting the allocation size to the nearest power of two, which SLAB always guarantees a size-aligned allocation. And because Rust guarantees that original size must be a multiple of alignment and the alignment must be a power of two, then the alignment requirement is satisfied.
Quoted 7 lines
Suggested-by Vlastimil Babka <[email protected]>Co-developed-by Andreas Hindborg (Samsung) <[email protected]>Signed-off-by Andreas Hindborg (Samsung) <[email protected]>Signed-off-by Boqun Feng <[email protected]>Cc [email protected] # v6.1+---
[...]
Miguel Ojeda
AppliedQuoted 2 lines · Boqun Feng
Cc [email protected] # v6.1+
Applied to rust-next, thanks!
However, should this go to stable? The actual functions being called are the __rust_* ones (until they get removed in 1.71), no? Thus this is not actually fixing the actual functions being called, right?
If that is correct, then the fix should change the functions below, perhaps adding krealloc_with_flags() from the other patch (it does not need to be a method, by the way), and calling it with a Layout like the generated ones do. Then I can rebase rust-next on top of the fix that adds the krealloc_with_flags().
Cheers,
Miguel
Quoted 8 lines · Miguel Ojeda
Quoted 2 lines · Boqun Feng
Cc [email protected] # v6.1+Applied to
rust-next, thanks!However, should this go to stable? The actual functions being called are the
__rust_*ones (until they get removed in 1.71), no? Thus
Interesting, I wasn't aware of the __rust_* "hack" here, so you are right, this doesn't fix the issue in stable kernels.
Quoted 6 lines
this is not actually fixing the actual functions being called, right?
If that is correct, then the fix should change the functions below, perhaps adding
krealloc_with_flags()from the other patch (it does not need to be a method, by the way), and calling it with aLayoutlike the generated ones do. Then I can rebaserust-nexton top of
Sounds good, however I think it'll be better if I resend this one, and the other one originally from Bjorn based on the introduction of function krealloc_with_flags (I will name it as krealloc_aligned, since it's a function that returns a aligned object with krealloc).
Thoughts?
Regards,
Boqun
Quoted 4 lines
the fix that adds the
krealloc_with_flags().Cheers,
Miguel