Skip to contentSkip to conversation

rust: alloc: fix dangling pointer in VecExt<T>::reserve()

PATCH Superseded Danilo Krummrich 3 participants 6 messages Apr 30, 2024 – May 7, 2024

Subsystems (2)

v2 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 rust: alloc: fix dangling pointer in VecExt<T>::reserve()

Currently, a Vec<T>'s ptr value, after calling Vec<T>::new(), is initialized to Unique::dangling(). Hence, in VecExt<T>::reserve(), we're passing a dangling pointer (instead of NULL) to krealloc() whenever a new Vec<T> is created through VecExt<T> extension functions.

This only works as long as align_of::<T>(), used by Unique::dangling() to derive the dangling pointer, resolves to a value between 0x0 and ZERO_SIZE_PTR (0x10) and krealloc() hence treats it the same as a NULL pointer however.

This isn't a case we should rely on, since there may be types whose alignment may exceed the range still covered by krealloc(), plus other kernel allocators are not as tolerant either.

Instead, pass a real NULL pointer to krealloc_aligned() if Vec<T>'s capacity is zero.

Fixes 5ab560ce12ed ("rust: alloc: update `VecExt` to take allocation flags")

---
Changes in v2:
- correct the description of the pointer value produced by Unique::dangling() (Alice)
- add RB tags (Alice, Boqun)

---
 rust/kernel/alloc/vec_ext.rs | 10 ++++++++++
 1 file changed, 10 insertions(+)
rust/kernel/alloc/vec_ext.rs +10 -0
index 6a916fcf8bf1..ffcf8a19f715 100644
@@ -4,6 +4,7 @@
 
 use super::{AllocError, Flags};
 use alloc::vec::Vec;
+use core::ptr;
 use core::result::Result;
 
 /// Extensions to [`Vec`].
@@ -137,6 +138,15 @@ fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError>
 
         let (ptr, len, cap) = destructure(self);
 
+        // We need to make sure that ptr is either NULL or comes from a previous call to
+        // `krealloc_aligned`. A `Vec<T>`'s `ptr` value is not guaranteed to be NULL and might be
+        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T>'s capacity
+        // to be zero if no memory has been allocated yet.
+        let ptr = match cap {
+            0 => ptr::null_mut(),
+            _ => ptr,
+        };
+
         // SAFETY: `ptr` is valid because it's either NULL or comes from a previous call to
         // `krealloc_aligned`. We also verified that the type is not a ZST.
         let new_ptr = unsafe { super::allocator::krealloc_aligned(ptr.cast(), layout, flags) };

base-commit: 2c1092853f163762ef0aabc551a630ef233e1be3

Signature · 1 line
2.44.0
#1 lore raw
Quoted 5 lines · Danilo Krummrich

Currently, a Vec<T>'s ptr value, after calling Vec<T>::new(), is initialized to Unique::dangling(). Hence, in VecExt<T>::reserve(), we're passing a dangling pointer (instead of NULL) to krealloc() whenever a new Vec<T> is created through VecExt<T> extension functions.

Nice catch, thanks!

A small nit on the wording above: this applies to any Vec<T>, not just those created via VecExt<T>. For example, if we create with Vec::new(), then use VecExt::push or VecExt::reserve, we'll run into this. (IOW, which function is used to create the Vec is not a factor, using VecExt to extend it from 0 to >0 is.)

Quoted 9 lines

This only works as long as align_of::<T>(), used by Unique::dangling() to derive the dangling pointer, resolves to a value between 0x0 and ZERO_SIZE_PTR (0x10) and krealloc() hence treats it the same as a NULL pointer however.

This isn't a case we should rely on, since there may be types whose alignment may exceed the range still covered by krealloc(), plus other kernel allocators are not as tolerant either.

Perhaps it would make sense to add a sample with such a type?

It would serve as a test as well when kunit and doctests are enabled.

rust/kernel/alloc/vec_ext.rs fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError> · line 149

Quoted 11 lines
         let (ptr, len, cap) = destructure(self);
rust/kernel/alloc/vec_ext.rs +9 -0
+        // We need to make sure that ptr is either NULL or comes from a previous call to
+        // `krealloc_aligned`. A `Vec<T>`'s `ptr` value is not guaranteed to be NULL and might be
+        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T>'s capacity
+        // to be zero if no memory has been allocated yet.
+        let ptr = match cap {
+            0 => ptr::null_mut(),
+            _ => ptr,
+        };
+

nit: why did you choose to use a match here? I don't think C programmers would use a switch to determine if a value is zero or non-zero.

I would have written:

let ptr = if cap != 0 { ptr } else { ptr::null_mut() };

Quoted 3 lines
         // SAFETY: `ptr` is valid because it's either NULL or comes from a previous call to
         // `krealloc_aligned`. We also verified that the type is not a ZST.
         let new_ptr = unsafe { super::allocator::krealloc_aligned(ptr.cast(), layout, flags) };

In the case when reallocation fails, we need to rebuild with the original pointer (the dangling one if cap is zero). This patch is rebuilding it with null when cap is zero, which is incorrect.

#2 lore raw
Quoted 11 lines · Wedson Almeida Filho
Quoted 5 lines · Danilo Krummrich

Currently, a Vec<T>'s ptr value, after calling Vec<T>::new(), is initialized to Unique::dangling(). Hence, in VecExt<T>::reserve(), we're passing a dangling pointer (instead of NULL) to krealloc() whenever a new Vec<T> is created through VecExt<T> extension functions.

Nice catch, thanks!

A small nit on the wording above: this applies to any Vec<T>, not just those created via VecExt<T>. For example, if we create with

True, I think I wanted to say "whenever a new Vec<T>'s backing storage is allocated through VecExt<T> extension functions".

Quoted 10 lines

Vec::new(), then use VecExt::push or VecExt::reserve, we'll run into this. (IOW, which function is used to create the Vec is not a factor, using VecExt to extend it from 0 to >0 is.)

Quoted 6 lines

This only works as long as align_of::<T>(), used by Unique::dangling() to derive the dangling pointer, resolves to a value between 0x0 and ZERO_SIZE_PTR (0x10) and krealloc() hence treats it the same as a NULL pointer however.

Quoted 7 lines
Quoted 3 lines

This isn't a case we should rely on, since there may be types whose alignment may exceed the range still covered by krealloc(), plus other kernel allocators are not as tolerant either.

Perhaps it would make sense to add a sample with such a type?

It would serve as a test as well when kunit and doctests are enabled.

Yeah, that sounds reasonable. Let me see whether I can get to that.

Quoted 13 lines

rust/kernel/alloc/vec_ext.rs fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError> · line 149

Quoted 12 lines
         let (ptr, len, cap) = destructure(self);
rust/kernel/alloc/vec_ext.rs +9 -0
+        // We need to make sure that ptr is either NULL or comes from a previous call to
+        // `krealloc_aligned`. A `Vec<T>`'s `ptr` value is not guaranteed to be NULL and might be
+        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T>'s capacity
+        // to be zero if no memory has been allocated yet.
+        let ptr = match cap {
+            0 => ptr::null_mut(),
+            _ => ptr,
+        };
+
Quoted 1 line

nit: why did you choose to use a match here?

I felt like it reads nicely.

Quoted 3 lines

I don't think C
programmers would use a switch to determine if a value is zero or non-zero.

Well, I think for writing Rust code it doesn't matter too much what C programmers would do? ;-)

What is idiomatic in this case?

Quoted 5 lines

I would have written:

let ptr = if cap != 0 { ptr } else { ptr::null_mut() };

Quoted 8 lines
Quoted 3 lines
         // SAFETY: `ptr` is valid because it's either NULL or comes from a previous call to
         // `krealloc_aligned`. We also verified that the type is not a ZST.
         let new_ptr = unsafe { super::allocator::krealloc_aligned(ptr.cast(), layout, flags) };

In the case when reallocation fails, we need to rebuild with the original pointer (the dangling one if cap is zero). This patch is rebuilding it with null when cap is zero, which is incorrect.

Indeed, good catch. Gonna fix that up in a v3.

#3 lore raw
Quoted 20 lines · Danilo Krummrich
Quoted 14 lines · Wedson Almeida Filho

rust/kernel/alloc/vec_ext.rs fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError> · line 149

Quoted 11 lines · Danilo Krummrich
         let (ptr, len, cap) = destructure(self);
rust/kernel/alloc/vec_ext.rs +9 -0
+        // We need to make sure that ptr is either NULL or comes from a previous call to
+        // `krealloc_aligned`. A `Vec<T>`'s `ptr` value is not guaranteed to be NULL and might be
+        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T>'s capacity
+        // to be zero if no memory has been allocated yet.
+        let ptr = match cap {
+            0 => ptr::null_mut(),
+            _ => ptr,
+        };
+
Quoted 1 line

nit: why did you choose to use a match here?

I felt like it reads nicely.

Quoted 8 lines
Quoted 3 lines

I don't think C
programmers would use a switch to determine if a value is zero or non-zero.

Well, I think for writing Rust code it doesn't matter too much what C programmers would do? ;-)

What is idiomatic in this case?

The idiomatic way is to use if. Even in cases where a match was required (e.g., when checking an enum) and only two options are ever used, the languages has the matches!
(https://doc.rust-lang.org/std/macro.matches.html) macro so you'd do:

if matches!(a, A::X(_)) {
} else {
}

instead of

match a {
    A::X(_) => {
    }
    _ => {
    }
}

They even changed the language with if-let constructs
(https://rust-lang.github.io/rfcs/0160-if-let.html) so that we'd do

if let A::X(_) = a {
} else {
}

instead of the match above. Here's the first paragraph for the justification for this feature:

"Introduce a new if let PAT = EXPR { BODY } construct. This allows for refutable pattern matching without the syntactic and semantic overhead of a full match, and without the corresponding extra rightward drift. Informally this is known as an “if-let statement”."

So the language designers and std library implementers provide a macro and a language construct to avoid using matches when needed, but you're going in the opposite direction where you don't need it but want to use it anyway.

Please change this.

Thanks,
-Wedson

#4 lore raw
Quoted 55 lines · Wedson Almeida Filho
Quoted 20 lines · Danilo Krummrich
Quoted 14 lines · Wedson Almeida Filho

rust/kernel/alloc/vec_ext.rs fn reserve(&mut self, additional: usize, flags: Flags) -> Result<(), AllocError> · line 149

Quoted 11 lines · Danilo Krummrich
         let (ptr, len, cap) = destructure(self);
rust/kernel/alloc/vec_ext.rs +9 -0
+        // We need to make sure that ptr is either NULL or comes from a previous call to
+        // `krealloc_aligned`. A `Vec<T>`'s `ptr` value is not guaranteed to be NULL and might be
+        // dangling after being created with `Vec::new`. Instead, we can rely on `Vec<T>'s capacity
+        // to be zero if no memory has been allocated yet.
+        let ptr = match cap {
+            0 => ptr::null_mut(),
+            _ => ptr,
+        };
+
Quoted 1 line

nit: why did you choose to use a match here?

I felt like it reads nicely.

Quoted 8 lines
Quoted 3 lines

I don't think C
programmers would use a switch to determine if a value is zero or non-zero.

Well, I think for writing Rust code it doesn't matter too much what C programmers would do? ;-)

What is idiomatic in this case?

The idiomatic way is to use if. Even in cases where a match was required (e.g., when checking an enum) and only two options are ever used, the languages has the matches!
(https://doc.rust-lang.org/std/macro.matches.html) macro so you'd do:

if matches!(a, A::X(_)) {
} else {
}

instead of

match a {
    A::X(_) => {
    }
    _ => {
    }
}

They even changed the language with if-let constructs
(https://rust-lang.github.io/rfcs/0160-if-let.html) so that we'd do

if let A::X(_) = a {
} else {
}

Quoted 8 lines

instead of the match above. Here's the first paragraph for the justification for this feature:

"Introduce a new if let PAT = EXPR { BODY } construct. This allows for refutable pattern matching without the syntactic and semantic overhead of a full match, and without the corresponding extra rightward drift. Informally this is known as an “if-let statement”."

Very valuable information - thanks for sharing!

This is what I was potentially looking for when I was asking "Is there any benefit using an if here, or is that your personal preference?" in v3.

Quoted 4 lines

So the language designers and std library implementers provide a macro and a language construct to avoid using matches when needed, but you're going in the opposite direction where you don't need it but want to use it anyway.

Please let me clarify this:

It's not really that I want to use it or insist on it. I really just chose it because I felt like it reads nicely. I did not feel like there is an actual request for change. Especially, since you phrased it as "nit: why did you choose to use a match here?", which is a question and even attenuated by the "nit".

When you mentioned in v3 that you "still think this should be an if", I noticed that you might actually want this to be changed and I wondered why.

Me asking "Is there any benefit using an if here, or is that your personal preference?" wasn't meant to read as if I wouldn't want to change it, but honest interest in why it's more than just a question and more than a "nit".

Please also see my reply [1] to Miguel.

Quoted 2 lines

Please change this.

Sure, I sent out another patch changing this, since v3 has been merged already [2]. Feel free to melt this one into the original one though.

[1] https://lore.kernel.org/rust-for-linux/[email protected]/
[2] https://lore.kernel.org/rust-for-linux/CANiq72nSQAE7c8ypWQA__BMoaBYqiEjXvHpuzn-=CsCky9uu7A@mail.gmail.com/

Quoted 4 lines

Thanks,
-Wedson

Reviewed-by on v2 1/1
#5 lore raw
In reply to Danilo Krummrich
Quoted 14 lines · Danilo Krummrich

Currently, a Vec<T>'s ptr value, after calling Vec<T>::new(), is initialized to Unique::dangling(). Hence, in VecExt<T>::reserve(), we're passing a dangling pointer (instead of NULL) to krealloc() whenever a new Vec<T> is created through VecExt<T> extension functions.

This only works as long as align_of::<T>(), used by Unique::dangling() to derive the dangling pointer, resolves to a value between 0x0 and ZERO_SIZE_PTR (0x10) and krealloc() hence treats it the same as a NULL pointer however.

This isn't a case we should rely on, since there may be types whose alignment may exceed the range still covered by krealloc(), plus other kernel allocators are not as tolerant either.

Quoted 7 lines

Instead, pass a real NULL pointer to krealloc_aligned() if Vec<T>'s capacity is zero.

Fixes 5ab560ce12ed ("rust: alloc: update `VecExt` to take allocation flags")

Nice find!

I don't mind the match, but an if is also fine.

Signature · 2 lines
Cheers,
Benno