Skip to contentSkip to conversation

rust: alloc: extend safety requirements of Vec::set_len()

PATCH Applied Danilo Krummrich 5 participants 34 messages Mar 15, 2025 – Apr 7, 2025

Subsystems (2)
lore
Share & tools
lore

Danilo Krummrich

author maintainer
#0 lore raw

PATCH 1/2 rust: alloc: extend safety requirements of Vec::set_len()

Extend the safety requirements of set_len() to consider the case when the new length is smaller than the old length.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")
---
 rust/kernel/alloc/kvec.rs | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
rust/kernel/alloc/kvec.rs +3 -1
index ae9d072741ce..8540d9e2b717 100644
@@ -189,7 +189,9 @@ pub fn len(&self) -> usize {
     ///
     /// - `new_len` must be less than or equal to [`Self::capacity`].
     /// - If `new_len` is greater than `self.len`, all elements within the interval
-    ///   [`self.len`,`new_len`) must be initialized.
+    ///   [`self.len`,`new_len`) must be initialized,
+    /// - if `new_len` is smaller than `self.len`, all elements within the interval
+    ///   [`new_len`, `self.len`) must either be dropped or copied and taken ownership of.
     #[inline]
     pub unsafe fn set_len(&mut self, new_len: usize) {
         debug_assert!(new_len <= self.capacity());

base-commit: 80e54e84911a923c40d7bee33a34c1b4be148d7a

Signature · 1 line
2.48.1
author maintainer
#1 lore raw

PATCH 2/2 rust: alloc: add missing invariant in Vec::set_len()

When setting a new length, we have to justify that the set length represents the exact number of elements stored in the vector.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")
---
 rust/kernel/alloc/kvec.rs | 3 +++
 1 file changed, 3 insertions(+)
rust/kernel/alloc/kvec.rs +3 -0
index 8540d9e2b717..0a4681ad4ce9 100644
@@ -195,6 +195,9 @@ pub fn len(&self) -> usize {
     #[inline]
     pub unsafe fn set_len(&mut self, new_len: usize) {
         debug_assert!(new_len <= self.capacity());
+
+        // INVARIANT: By the safety requirements of this method `new_len` represents the exact
+        // number of elements stored within `self`.
         self.len = new_len;
     }
 
Signature · 1 line
2.48.1
author maintainer
#2 lore raw

Re: [PATCH 2/2] rust: alloc: add missing invariant in Vec::set_len()

Quoted 5 lines · Danilo Krummrich

When setting a new length, we have to justify that the set length represents the exact number of elements stored in the vector.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")

[1] probably justifies

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

Quoted 21 lines
---
 rust/kernel/alloc/kvec.rs | 3 +++
 1 file changed, 3 insertions(+)
rust/kernel/alloc/kvec.rs +3 -0
index 8540d9e2b717..0a4681ad4ce9 100644
@@ -195,6 +195,9 @@ pub fn len(&self) -> usize {
     #[inline]
     pub unsafe fn set_len(&mut self, new_len: usize) {
         debug_assert!(new_len <= self.capacity());
+
+        // INVARIANT: By the safety requirements of this method `new_len` represents the exact
+        // number of elements stored within `self`.
         self.len = new_len;
     }
 
Signature · 1 line
2.48.1
Reviewed-by on v1 2/2
#3 lore raw

Re: [PATCH 2/2] rust: alloc: add missing invariant in Vec::set_len()

In reply to Danilo Krummrich
Quoted 5 lines · Danilo Krummrich

When setting a new length, we have to justify that the set length represents the exact number of elements stored in the vector.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")

---
Cheers,
Benno

Quoted 18 lines
---
 rust/kernel/alloc/kvec.rs | 3 +++
 1 file changed, 3 insertions(+)
rust/kernel/alloc/kvec.rs +3 -0
index 8540d9e2b717..0a4681ad4ce9 100644
@@ -195,6 +195,9 @@ pub fn len(&self) -> usize {
     #[inline]
     pub unsafe fn set_len(&mut self, new_len: usize) {
         debug_assert!(new_len <= self.capacity());
+
+        // INVARIANT: By the safety requirements of this method `new_len` represents the exact
+        // number of elements stored within `self`.
         self.len = new_len;
     }
 
author maintainer Applied
#4 lore raw

Re: [PATCH 2/2] rust: alloc: add missing invariant in Vec::set_len()

In reply to Danilo Krummrich
Quoted 5 lines · Danilo Krummrich

When setting a new length, we have to justify that the set length represents the exact number of elements stored in the vector.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")

Applied to rust/alloc-next, thanks!

Reviewed-by on v1 1/2
#5 lore raw
In reply to Danilo Krummrich
Quoted 15 lines · Danilo Krummrich

Extend the safety requirements of set_len() to consider the case when the new length is smaller than the old length.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")
---
 rust/kernel/alloc/kvec.rs | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
rust/kernel/alloc/kvec.rs +0 -0
index ae9d072741ce..8540d9e2b717 100644

rust/kernel/alloc/kvec.rs pub fn len(&self) -> usize { · line 192

Quoted 5 lines
rust/kernel/alloc/kvec.rs +1 -1
@@ -189,7 +189,9 @@ pub fn len(&self) -> usize {
     ///
     /// - `new_len` must be less than or equal to [`Self::capacity`].
     /// - If `new_len` is greater than `self.len`, all elements within the interval
-    ///   [`self.len`,`new_len`) must be initialized.
+    ///   [`self.len`,`new_len`) must be initialized,

Why a comma here? There's a period on the previous list item, and a period seems more appropriate before the next item.

rust/kernel/alloc/kvec.rs pub fn len(&self) -> usize { · line 194

Quoted 2 lines
rust/kernel/alloc/kvec.rs +2 -0
+    /// - if `new_len` is smaller than `self.len`, all elements within the interval
+    ///   [`new_len`, `self.len`) must either be dropped or copied and taken ownership of.

Nit: I'd argue that dropping is a special case of taking ownership, and that such phrasing would be clearer.

Quoted 9 lines
rust/kernel/alloc/kvec.rs +0 -0
     #[inline]
     pub unsafe fn set_len(&mut self, new_len: usize) {
         debug_assert!(new_len <= self.capacity());

base-commit: 80e54e84911a923c40d7bee33a34c1b4be148d7a --
2.48.1

Minor comments.

It seems that the standard library also lacks this language, it would be good to send them a PR.

Reviewed-by on v1 1/2
#6 lore raw
In reply to Danilo Krummrich
Quoted 14 lines · Danilo Krummrich

Extend the safety requirements of set_len() to consider the case when the new length is smaller than the old length.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")
---
 rust/kernel/alloc/kvec.rs | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
rust/kernel/alloc/kvec.rs +0 -0
index ae9d072741ce..8540d9e2b717 100644

rust/kernel/alloc/kvec.rs pub fn len(&self) -> usize { · line 194

Quoted 7 lines
rust/kernel/alloc/kvec.rs +3 -1
@@ -189,7 +189,9 @@ pub fn len(&self) -> usize {
     ///
     /// - `new_len` must be less than or equal to [`Self::capacity`].
     /// - If `new_len` is greater than `self.len`, all elements within the interval
-    ///   [`self.len`,`new_len`) must be initialized.
+    ///   [`self.len`,`new_len`) must be initialized,
+    /// - if `new_len` is smaller than `self.len`, all elements within the interval
+    ///   [`new_len`, `self.len`) must either be dropped or copied and taken ownership of.

What is meant by "copied"? I agree with Tamir, this can probably just be "taken ownership of".

With that change:

---
Cheers,
Benno

Quoted 5 lines
rust/kernel/alloc/kvec.rs +0 -0
     #[inline]
     pub unsafe fn set_len(&mut self, new_len: usize) {
         debug_assert!(new_len <= self.capacity());

base-commit: 80e54e84911a923c40d7bee33a34c1b4be148d7a

author maintainer
#7 lore raw
Quoted 10 lines · Benno Lossin
Quoted 9 lines · Danilo Krummrich

Extend the safety requirements of set_len() to consider the case when the new length is smaller than the old length.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")
---
 rust/kernel/alloc/kvec.rs | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
Quoted 15 lines

rust/kernel/alloc/kvec.rs pub fn len(&self) -> usize { · line 194

Quoted 12 lines
rust/kernel/alloc/kvec.rs +3 -1
index ae9d072741ce..8540d9e2b717 100644
@@ -189,7 +189,9 @@ pub fn len(&self) -> usize {
     ///
     /// - `new_len` must be less than or equal to [`Self::capacity`].
     /// - If `new_len` is greater than `self.len`, all elements within the interval
-    ///   [`self.len`,`new_len`) must be initialized.
+    ///   [`self.len`,`new_len`) must be initialized,
+    /// - if `new_len` is smaller than `self.len`, all elements within the interval
+    ///   [`new_len`, `self.len`) must either be dropped or copied and taken ownership of.

What is meant by "copied"? I agree with Tamir, this can probably just be "taken ownership of".

If not dropped in place, the only way to take ownership of the object is to copy the corresponding memory, i.e. the caller can not take ownership of the object at its current memory location, since this memory is owned by the vector.

AFAIU, when we speak of ownership, we mean ownership over a value or object, but not ownership over the underlying memory.

Hence, I think it's important to mention that it needs to be either dropped in place or copied in order to take ownership.

Even if we agree that "drop in place" is considered as "take ownership", we need to differentiate between the special case of taking ownership where its dropped in place and the requirement to copy the value and then take ownership.

Quoted 16 lines

With that change:

---
Cheers,
Benno

Quoted 5 lines
rust/kernel/alloc/kvec.rs +0 -0
     #[inline]
     pub unsafe fn set_len(&mut self, new_len: usize) {
         debug_assert!(new_len <= self.capacity());

base-commit: 80e54e84911a923c40d7bee33a34c1b4be148d7a

#8 lore raw
Quoted 9 lines · Danilo Krummrich
Quoted 3 lines · Benno Lossin

What is meant by "copied"? I agree with Tamir, this can probably just be "taken ownership of".

If not dropped in place, the only way to take ownership of the object is to copy the corresponding memory, i.e. the caller can not take ownership of the object at its current memory location, since this memory is owned by the vector.

I think this is the point I was trying to make: the language semantics are such that "taking ownership of" is equivalent to "become responsible for the dropping of", so it doesn't make sense to write that here, as though this requirement is specific to the semantics of this function.

Quoted 9 lines

AFAIU, when we speak of ownership, we mean ownership over a value or object, but not ownership over the underlying memory.

Hence, I think it's important to mention that it needs to be either dropped in place or copied in order to take ownership.

Even if we agree that "drop in place" is considered as "take ownership", we need to differentiate between the special case of taking ownership where its dropped in place and the requirement to copy the value and then take ownership.

Perhaps:

Quoted 1 line

The caller must ensure the destructors of [new_len, self.len) are run exactly once.

Quoted 16 lines · Tamir Duberstein

rust/kernel/alloc/kvec.rs pub fn len(&self) -> usize { · line 192

Quoted 11 lines · Danilo Krummrich
rust/kernel/alloc/kvec.rs +1 -1
index ae9d072741ce..8540d9e2b717 100644
@@ -189,7 +189,9 @@ pub fn len(&self) -> usize {
     ///
     /// - `new_len` must be less than or equal to [`Self::capacity`].
     /// - If `new_len` is greater than `self.len`, all elements within the interval
-    ///   [`self.len`,`new_len`) must be initialized.
+    ///   [`self.len`,`new_len`) must be initialized,

Why a comma here? There's a period on the previous list item, and a period seems more appropriate before the next item.

Did you see this one? Was it intentional?

#9 lore raw
Quoted 11 lines · Tamir Duberstein
Quoted 9 lines · Danilo Krummrich
Quoted 3 lines · Benno Lossin

What is meant by "copied"? I agree with Tamir, this can probably just be "taken ownership of".

If not dropped in place, the only way to take ownership of the object is to copy the corresponding memory, i.e. the caller can not take ownership of the object at its current memory location, since this memory is owned by the vector.

Quoted 15 lines

I think this is the point I was trying to make: the language semantics are such that "taking ownership of" is equivalent to "become responsible for the dropping of", so it doesn't make sense to write that here, as though this requirement is specific to the semantics of this function.

Quoted 9 lines

AFAIU, when we speak of ownership, we mean ownership over a value or object, but not ownership over the underlying memory.

Hence, I think it's important to mention that it needs to be either dropped in place or copied in order to take ownership.

Even if we agree that "drop in place" is considered as "take ownership", we need to differentiate between the special case of taking ownership where its dropped in place and the requirement to copy the value and then take ownership.

I get where you're coming from, but I wouldn't call that copying a value. Copying for me means that there would be two instances of the same object. But since the one in [new_len, len) will get invalidated, I don't think that copy makes sense.

Also why even is this a safety requirement? There is mem::forget, so doing vec.set_len(0) can also be done by safe code:

    while let Some(val) = vec.pop() {
        forget(val);
    }

I don't think this needs to be a safety requirement.

Quoted 3 lines

Perhaps:

Quoted 1 line

The caller must ensure the destructors of [new_len, self.len) are run exactly once.

Hmm that is not correct though, since

    let mut vec = vec![];
    vec.push(x.clone());
    vec.pop();
    vec.push(x.clone());
    vec.pop();

Drops the memory in [0, 1) two times.

---
Cheers,
Benno

author maintainer
#10 lore raw
Quoted 24 lines · Benno Lossin
Quoted 23 lines · Tamir Duberstein
Quoted 9 lines · Danilo Krummrich
Quoted 3 lines · Benno Lossin

What is meant by "copied"? I agree with Tamir, this can probably just be "taken ownership of".

If not dropped in place, the only way to take ownership of the object is to copy the corresponding memory, i.e. the caller can not take ownership of the object at its current memory location, since this memory is owned by the vector.

I think this is the point I was trying to make: the language semantics are such that "taking ownership of" is equivalent to "become responsible for the dropping of", so it doesn't make sense to write that here, as though this requirement is specific to the semantics of this function.

Quoted 6 lines

AFAIU, when we speak of ownership, we mean ownership over a value or object, but not ownership over the underlying memory.

Hence, I think it's important to mention that it needs to be either dropped in place or copied in order to take ownership.

Quoted 8 lines
Quoted 3 lines
Quoted 3 lines

Even if we agree that "drop in place" is considered as "take ownership", we need to differentiate between the special case of taking ownership where its dropped in place and the requirement to copy the value and then take ownership.

I get where you're coming from, but I wouldn't call that copying a value. Copying for me means that there would be two instances of the same object. But since the one in [new_len, len) will get invalidated, I don't think that copy makes sense.

What I meant is "copy the memory where the value is stored".

Quoted 9 lines

Also why even is this a safety requirement? There is mem::forget, so doing vec.set_len(0) can also be done by safe code:

    while let Some(val) = vec.pop() {
        forget(val);
    }

I don't think this needs to be a safety requirement.

Given that mem::forget() is considered safe, you clearly have a point here.

Let's go with just "must be taken ownership of" then. Unless there's subsequent feedback, I won't send a new version for this, since you both already gave your (conditional) RB for this.

(I tend to forget that mem::forget() is formally safe, since it can cause all kinds of bugs, e.g. dead lock the whole machine if called for a lock in the kernel, etc.)

#11 lore raw
Quoted 6 lines · Danilo Krummrich
Quoted 2 lines · Benno Lossin

I don't think this needs to be a safety requirement.

Given that mem::forget() is considered safe, you clearly have a point here.

Yeah, good catch.

Quoted 3 lines

Let's go with just "must be taken ownership of" then. Unless there's subsequent feedback, I won't send a new version for this, since you both already gave your (conditional) RB for this.

What does it mean to take ownership, if not to run the destructor? Given Benno's insight, I think the safety text is correct as it existed before this patch.

author maintainer
#12 lore raw
Quoted 7 lines · Tamir Duberstein
Quoted 3 lines · Danilo Krummrich

Let's go with just "must be taken ownership of" then. Unless there's subsequent feedback, I won't send a new version for this, since you both already gave your (conditional) RB for this.

What does it mean to take ownership, if not to run the destructor?

Taking responsibility over the decision of what should happen to the value, i.e. forget about the value, drop it right away, keep it alive at a different memory location.

Quoted 2 lines

Given Benno's insight, I think the safety text is correct as it existed before this patch.

Without the addition there's no requirement for the caller to take ownership, but that's what we want here. Without the requirement it would be on set_len() to take a decision on what should happen with the value.

#13 lore raw
Quoted 14 lines · Danilo Krummrich
Quoted 7 lines · Tamir Duberstein
Quoted 3 lines · Danilo Krummrich

Let's go with just "must be taken ownership of" then. Unless there's subsequent feedback, I won't send a new version for this, since you both already gave your (conditional) RB for this.

What does it mean to take ownership, if not to run the destructor?

Taking responsibility over the decision of what should happen to the value, i.e. forget about the value, drop it right away, keep it alive at a different memory location.

Quoted 6 lines
Quoted 2 lines

Given Benno's insight, I think the safety text is correct as it existed before this patch.

Without the addition there's no requirement for the caller to take ownership, but that's what we want here. Without the requirement it would be on set_len() to take a decision on what should happen with the value.

It isn't a safety requirement, right? I think it's fine to document, but given that neglecting to run a destructor is safe, it doesn't seem to affect safety.

author maintainer
#14 lore raw
Quoted 18 lines · Tamir Duberstein
Quoted 17 lines · Danilo Krummrich
Quoted 7 lines · Tamir Duberstein
Quoted 3 lines · Danilo Krummrich

Let's go with just "must be taken ownership of" then. Unless there's subsequent feedback, I won't send a new version for this, since you both already gave your (conditional) RB for this.

What does it mean to take ownership, if not to run the destructor?

Taking responsibility over the decision of what should happen to the value, i.e. forget about the value, drop it right away, keep it alive at a different memory location.

Quoted 2 lines

Given Benno's insight, I think the safety text is correct as it existed before this patch.

Quoted 7 lines
Quoted 3 lines

Without the addition there's no requirement for the caller to take ownership, but that's what we want here. Without the requirement it would be on set_len() to take a decision on what should happen with the value.

It isn't a safety requirement, right? I think it's fine to document, but given that neglecting to run a destructor is safe, it doesn't seem to affect safety.

I don't think there's a rule that safety requirements of unsafe functions must only be used to guarantee memory safety, etc.

But you're right, if set_len() guarantees a default of what happens to the value(s) when the caller doesn't take ownership, we could avoid the safety requirement.

But I don't want set_len() to make any guarantees about about a default, instead I want that the caller is forced to make an explicit and documented decision over what happens to the discarded values.

#15 lore raw
Quoted 8 lines · Danilo Krummrich
Quoted 7 lines · Tamir Duberstein
Quoted 6 lines · Danilo Krummrich
Quoted 5 lines · Tamir Duberstein
Quoted 3 lines · Danilo Krummrich

Let's go with just "must be taken ownership of" then. Unless there's subsequent feedback, I won't send a new version for this, since you both already gave your (conditional) RB for this.

I didn't realize the forget argument immediately, so I sadly have to take my RB back.

Quoted 13 lines
Quoted 13 lines
Quoted 12 lines
Quoted 1 line

What does it mean to take ownership, if not to run the destructor?

Taking responsibility over the decision of what should happen to the value, i.e. forget about the value, drop it right away, keep it alive at a different memory location.

Quoted 2 lines

Given Benno's insight, I think the safety text is correct as it existed before this patch.

Without the addition there's no requirement for the caller to take ownership, but that's what we want here. Without the requirement it would be on set_len() to take a decision on what should happen with the value.

Quoted 6 lines
Quoted 3 lines

It isn't a safety requirement, right? I think it's fine to document, but given that neglecting to run a destructor is safe, it doesn't seem to affect safety.

I don't think there's a rule that safety requirements of unsafe functions must only be used to guarantee memory safety, etc.

Well that's their only point. Safety documentation should only be concerned with memory safety.

You can still put it in the normal documentation though.

Quoted 3 lines

But you're right, if set_len() guarantees a default of what happens to the value(s) when the caller doesn't take ownership, we could avoid the safety requirement.

Yeah the default is to forget the value.

Quoted 3 lines

But I don't want set_len() to make any guarantees about about a default, instead I want that the caller is forced to make an explicit and documented decision over what happens to the discarded values.

But it shouldn't be part of the safety docs.

---
Cheers,
Benno

author maintainer
#16 lore raw
Quoted 30 lines · Benno Lossin
Quoted 8 lines · Danilo Krummrich
Quoted 7 lines · Tamir Duberstein
Quoted 6 lines · Danilo Krummrich
Quoted 5 lines · Tamir Duberstein
Quoted 3 lines · Danilo Krummrich

Let's go with just "must be taken ownership of" then. Unless there's subsequent feedback, I won't send a new version for this, since you both already gave your (conditional) RB for this.

I didn't realize the forget argument immediately, so I sadly have to take my RB back.

Quoted 17 lines
Quoted 16 lines
Quoted 6 lines
Quoted 1 line

What does it mean to take ownership, if not to run the destructor?

Taking responsibility over the decision of what should happen to the value, i.e. forget about the value, drop it right away, keep it alive at a different memory location.

Quoted 6 lines
Quoted 2 lines

Given Benno's insight, I think the safety text is correct as it existed before this patch.

Without the addition there's no requirement for the caller to take ownership, but that's what we want here. Without the requirement it would be on set_len() to take a decision on what should happen with the value.

It isn't a safety requirement, right? I think it's fine to document, but given that neglecting to run a destructor is safe, it doesn't seem to affect safety.

Quoted 5 lines
Quoted 2 lines

I don't think there's a rule that safety requirements of unsafe functions must only be used to guarantee memory safety, etc.

Well that's their only point. Safety documentation should only be concerned with memory safety.

Oops, I wanted to say "must not only be used to *directly* guarantee memory safety, etc.".

For instance, it is also used to justify type invariants (that exist to prevent UB).

So, what I wanted to say is that setting self.len by iteself is never directly a safety critical thing.

Quoted 7 lines

You can still put it in the normal documentation though.

Quoted 3 lines

But you're right, if set_len() guarantees a default of what happens to the value(s) when the caller doesn't take ownership, we could avoid the safety requirement.

Yeah the default is to forget the value.

It's not really a defined default though, it's just what ultimately happens as the consequence of how Vec in general and set_len() are implemented.

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

author maintainer
#17 lore raw
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

#18 lore raw
Quoted 17 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

There are other examples; ManuallyDrop and Box::leak are two that immediately come to mind.

But focusing on Vec::set_len again, could we return a mut slice to the tail when new_len < old_len? Something like:

rust/kernel/alloc/kvec.rs +0 -0
index c12844764671..e5f857d723ec 100644
@@ -191,9 +191,16 @@ pub fn len(&self) -> usize {
     /// - If `new_len` is greater than `self.len`, all elements

within the interval

     ///   [`self.len`,`new_len`) must be initialized.
     #[inline]
-    pub unsafe fn set_len(&mut self, new_len: usize) {
+    pub unsafe fn set_len(&mut self, new_len: usize) -> &mut [T] {
         debug_assert!(new_len <= self.capacity());
-        self.len = new_len;
+        let old_len = core::mem::replace(&mut self.len, new_len);
+        match old_len.checked_sub(new_len) {

+ None => &mut [],
+ Some(len) => {
+ // SAFETY: ...
+ unsafe {
slice::from_raw_parts_mut(self.as_mut_ptr().add(new_len), len) }
+ }
+ }

     }

Would that sufficiently communicate to the caller that they should deal with this memory?

author maintainer
#19 lore raw
Quoted 56 lines · Tamir Duberstein
Quoted 17 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

There are other examples; ManuallyDrop and Box::leak are two that immediately come to mind.

But focusing on Vec::set_len again, could we return a mut slice to the tail when new_len < old_len? Something like:

rust/kernel/alloc/kvec.rs +0 -0
index c12844764671..e5f857d723ec 100644
@@ -191,9 +191,16 @@ pub fn len(&self) -> usize {
     /// - If `new_len` is greater than `self.len`, all elements

within the interval

     ///   [`self.len`,`new_len`) must be initialized.
     #[inline]
rust/kernel/alloc/kvec.rs +7 -2
-    pub unsafe fn set_len(&mut self, new_len: usize) {
+    pub unsafe fn set_len(&mut self, new_len: usize) -> &mut [T] {
         debug_assert!(new_len <= self.capacity());
-        self.len = new_len;
+        let old_len = core::mem::replace(&mut self.len, new_len);
+        match old_len.checked_sub(new_len) {
+            None => &mut [],
+            Some(len) => {
+                // SAFETY: ...
+                unsafe {

slice::from_raw_parts_mut(self.as_mut_ptr().add(new_len), len) }

rust/kernel/alloc/kvec.rs +2 -0
+            }
+        }
     }
Quoted 2 lines

Would that sufficiently communicate to the caller that they should deal with this memory?

I think that is a good idea. I'm not sure I like that this is useless when new_len > self.len, but it also doesn't hurt too much, I guess.

Feel free to send the corresponding patch. Also, feel free to add the corresponding comment about ownership while at it.

I'd just drop this patch then.

#20 lore raw
Quoted 64 lines · Danilo Krummrich
Quoted 56 lines · Tamir Duberstein
Quoted 17 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

There are other examples; ManuallyDrop and Box::leak are two that immediately come to mind.

But focusing on Vec::set_len again, could we return a mut slice to the tail when new_len < old_len? Something like:

rust/kernel/alloc/kvec.rs +0 -0
index c12844764671..e5f857d723ec 100644
@@ -191,9 +191,16 @@ pub fn len(&self) -> usize {
     /// - If `new_len` is greater than `self.len`, all elements

within the interval

     ///   [`self.len`,`new_len`) must be initialized.
     #[inline]
rust/kernel/alloc/kvec.rs +7 -2
-    pub unsafe fn set_len(&mut self, new_len: usize) {
+    pub unsafe fn set_len(&mut self, new_len: usize) -> &mut [T] {
         debug_assert!(new_len <= self.capacity());
-        self.len = new_len;
+        let old_len = core::mem::replace(&mut self.len, new_len);
+        match old_len.checked_sub(new_len) {
+            None => &mut [],
+            Some(len) => {
+                // SAFETY: ...
+                unsafe {

slice::from_raw_parts_mut(self.as_mut_ptr().add(new_len), len) }

rust/kernel/alloc/kvec.rs +2 -0
+            }
+        }
     }
Quoted 2 lines

Would that sufficiently communicate to the caller that they should deal with this memory?

I think that is a good idea. I'm not sure I like that this is useless when new_len > self.len, but it also doesn't hurt too much, I guess.

Quoted 4 lines

Feel free to send the corresponding patch. Also, feel free to add the corresponding comment about ownership while at it.

I'd just drop this patch then.

Looking at the usages of set_len that we have today (excluding the pending nova patches for truncate and resize), they all amount to v.set_len(v.len() + n). What do you think about replacing this method with inc_len? In the nova patches the only need for decrementing the length would be in truncate, so dec_len can be introduced there or the length can be decremented there directly.

author maintainer
#21 lore raw
Quoted 68 lines · Tamir Duberstein
Quoted 67 lines · Danilo Krummrich
Quoted 56 lines · Tamir Duberstein
Quoted 17 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

There are other examples; ManuallyDrop and Box::leak are two that immediately come to mind.

But focusing on Vec::set_len again, could we return a mut slice to the tail when new_len < old_len? Something like:

rust/kernel/alloc/kvec.rs +0 -0
index c12844764671..e5f857d723ec 100644
@@ -191,9 +191,16 @@ pub fn len(&self) -> usize {
     /// - If `new_len` is greater than `self.len`, all elements

within the interval

     ///   [`self.len`,`new_len`) must be initialized.
     #[inline]
rust/kernel/alloc/kvec.rs +7 -2
-    pub unsafe fn set_len(&mut self, new_len: usize) {
+    pub unsafe fn set_len(&mut self, new_len: usize) -> &mut [T] {
         debug_assert!(new_len <= self.capacity());
-        self.len = new_len;
+        let old_len = core::mem::replace(&mut self.len, new_len);
+        match old_len.checked_sub(new_len) {
+            None => &mut [],
+            Some(len) => {
+                // SAFETY: ...
+                unsafe {

slice::from_raw_parts_mut(self.as_mut_ptr().add(new_len), len) }

rust/kernel/alloc/kvec.rs +2 -0
+            }
+        }
     }
Quoted 2 lines

Would that sufficiently communicate to the caller that they should deal with this memory?

I think that is a good idea. I'm not sure I like that this is useless when new_len > self.len, but it also doesn't hurt too much, I guess.

Feel free to send the corresponding patch. Also, feel free to add the corresponding comment about ownership while at it.

Quoted 8 lines
Quoted 1 line

I'd just drop this patch then.

Looking at the usages of set_len that we have today (excluding the pending nova patches for truncate and resize), they all amount to v.set_len(v.len() + n). What do you think about replacing this method with inc_len? In the nova patches the only need for decrementing the length would be in truncate, so dec_len can be introduced there or the length can be decremented there directly.

I think splitting it in inc_len() and dec_len() is good. Note that other methods need dec_len() too, e.g. [1].

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

#22 lore raw
Quoted 79 lines · Danilo Krummrich
Quoted 68 lines · Tamir Duberstein
Quoted 67 lines · Danilo Krummrich
Quoted 56 lines · Tamir Duberstein
Quoted 17 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

There are other examples; ManuallyDrop and Box::leak are two that immediately come to mind.

But focusing on Vec::set_len again, could we return a mut slice to the tail when new_len < old_len? Something like:

rust/kernel/alloc/kvec.rs +0 -0
index c12844764671..e5f857d723ec 100644
@@ -191,9 +191,16 @@ pub fn len(&self) -> usize {
     /// - If `new_len` is greater than `self.len`, all elements

within the interval

     ///   [`self.len`,`new_len`) must be initialized.
     #[inline]
rust/kernel/alloc/kvec.rs +7 -2
-    pub unsafe fn set_len(&mut self, new_len: usize) {
+    pub unsafe fn set_len(&mut self, new_len: usize) -> &mut [T] {
         debug_assert!(new_len <= self.capacity());
-        self.len = new_len;
+        let old_len = core::mem::replace(&mut self.len, new_len);
+        match old_len.checked_sub(new_len) {
+            None => &mut [],
+            Some(len) => {
+                // SAFETY: ...
+                unsafe {

slice::from_raw_parts_mut(self.as_mut_ptr().add(new_len), len) }

rust/kernel/alloc/kvec.rs +2 -0
+            }
+        }
     }
Quoted 2 lines

Would that sufficiently communicate to the caller that they should deal with this memory?

I think that is a good idea. I'm not sure I like that this is useless when new_len > self.len, but it also doesn't hurt too much, I guess.

Feel free to send the corresponding patch. Also, feel free to add the corresponding comment about ownership while at it.

Quoted 8 lines
Quoted 1 line

I'd just drop this patch then.

Looking at the usages of set_len that we have today (excluding the pending nova patches for truncate and resize), they all amount to v.set_len(v.len() + n). What do you think about replacing this method with inc_len? In the nova patches the only need for decrementing the length would be in truncate, so dec_len can be introduced there or the length can be decremented there directly.

Quoted 4 lines

I think splitting it in inc_len() and dec_len() is good. Note that other methods need dec_len() too, e.g. [1].

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

Ok, I'll send a 2-patch series that adds dec_len as a private helper. I see no need for it to be pub at this time.

#23 lore raw
Quoted 81 lines · Tamir Duberstein
Quoted 79 lines · Danilo Krummrich
Quoted 76 lines · Tamir Duberstein
Quoted 64 lines · Danilo Krummrich
Quoted 56 lines · Tamir Duberstein
Quoted 17 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

There are other examples; ManuallyDrop and Box::leak are two that immediately come to mind.

But focusing on Vec::set_len again, could we return a mut slice to the tail when new_len < old_len? Something like:

rust/kernel/alloc/kvec.rs +0 -0
index c12844764671..e5f857d723ec 100644
@@ -191,9 +191,16 @@ pub fn len(&self) -> usize {
     /// - If `new_len` is greater than `self.len`, all elements

within the interval

     ///   [`self.len`,`new_len`) must be initialized.
     #[inline]
rust/kernel/alloc/kvec.rs +7 -2
-    pub unsafe fn set_len(&mut self, new_len: usize) {
+    pub unsafe fn set_len(&mut self, new_len: usize) -> &mut [T] {
         debug_assert!(new_len <= self.capacity());
-        self.len = new_len;
+        let old_len = core::mem::replace(&mut self.len, new_len);
+        match old_len.checked_sub(new_len) {
+            None => &mut [],
+            Some(len) => {
+                // SAFETY: ...
+                unsafe {

slice::from_raw_parts_mut(self.as_mut_ptr().add(new_len), len) }

rust/kernel/alloc/kvec.rs +2 -0
+            }
+        }
     }
Quoted 2 lines

Would that sufficiently communicate to the caller that they should deal with this memory?

I think that is a good idea. I'm not sure I like that this is useless when new_len > self.len, but it also doesn't hurt too much, I guess.

Quoted 4 lines

Feel free to send the corresponding patch. Also, feel free to add the corresponding comment about ownership while at it.

I'd just drop this patch then.

Looking at the usages of set_len that we have today (excluding the pending nova patches for truncate and resize), they all amount to v.set_len(v.len() + n). What do you think about replacing this method with inc_len? In the nova patches the only need for decrementing the length would be in truncate, so dec_len can be introduced there or the length can be decremented there directly.

Quoted 7 lines
Quoted 4 lines

I think splitting it in inc_len() and dec_len() is good. Note that other methods need dec_len() too, e.g. [1].

[1] https://lore.kernel.org/rust-for-linux/[email protected]/

Ok, I'll send a 2-patch series that adds dec_len as a private helper. I see no need for it to be pub at this time.

Actually, I don't think we need dec_len after all. clear can be implemented as self.truncate(0). Do you agree?

author maintainer
#24 lore raw
Quoted 2 lines · Tamir Duberstein

Actually, I don't think we need dec_len after all. clear can be implemented as self.truncate(0). Do you agree?

Indeed, but I still think it would be nice to have dec_len(). pop() would be another candidate for instance.

#25 lore raw
In reply to Danilo Krummrich
Quoted 16 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

I get where you're coming from, but this probably will very quickly get out of hand.

For example, I can define forget safely:

    fn forget<T>(value: T) {
        struct Cycle<T> {
            this: RefCell<Option<Arc<Self>>>,
            value: T,
        }
        let cycle = Arc::new(Cycle { this: RefCell::new(None), value });
        *cycle.this.borrow_mut() = Some(cycle.clone());
    }

How would you ensure that this kind of pattern doesn't get written accidentally (or with many indirections)?

---
Cheers,
Benno

author maintainer
#26 lore raw
Quoted 39 lines · Benno Lossin
Quoted 16 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

I get where you're coming from, but this probably will very quickly get out of hand.

For example, I can define forget safely:

    fn forget<T>(value: T) {
        struct Cycle<T> {
            this: RefCell<Option<Arc<Self>>>,
            value: T,
        }
        let cycle = Arc::new(Cycle { this: RefCell::new(None), value });
        *cycle.this.borrow_mut() = Some(cycle.clone());
    }
Quoted 2 lines

How would you ensure that this kind of pattern doesn't get written accidentally (or with many indirections)?

I don't think that the possibility of writing safe (but yet buggy) code is an argument against having the possibility of enforcing that a caller must write a comment for justification on certain things, such as mem::forget().

But there's another reason I think having something like this could be
problematic: It might set the wrong incentive, as in "hey, I can just use a "sanity requirement" in my function rather figuring out how to ensure it through the type system, etc.".

#27 lore raw
Quoted 40 lines · Danilo Krummrich
Quoted 39 lines · Benno Lossin
Quoted 16 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

I get where you're coming from, but this probably will very quickly get out of hand.

For example, I can define forget safely:

    fn forget<T>(value: T) {
        struct Cycle<T> {
            this: RefCell<Option<Arc<Self>>>,
            value: T,
        }
        let cycle = Arc::new(Cycle { this: RefCell::new(None), value });
        *cycle.this.borrow_mut() = Some(cycle.clone());
    }
Quoted 6 lines
Quoted 2 lines

How would you ensure that this kind of pattern doesn't get written accidentally (or with many indirections)?

I don't think that the possibility of writing safe (but yet buggy) code is an argument against having the possibility of enforcing that a caller must write a comment for justification on certain things, such as mem::forget().

My argument is that the problem of forgetting a value is not self-contained like unsafe code is. Even if we were to document all forget or ManuallyDrop::new invocations (which we definitely should) we wouldn't get the security that one can't accidentally forget a lock guard. I'm totally in favor of mandating an explaining comment above forget calls (but not as a SAFETY comment).

Quoted 4 lines

But there's another reason I think having something like this could be
problematic: It might set the wrong incentive, as in "hey, I can just use a "sanity requirement" in my function rather figuring out how to ensure it through the type system, etc.".

I don't understand your point here, can you explain it more?

---
Cheers,
Benno

author maintainer
#28 lore raw
Quoted 48 lines · Benno Lossin
Quoted 40 lines · Danilo Krummrich
Quoted 39 lines · Benno Lossin
Quoted 16 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

I get where you're coming from, but this probably will very quickly get out of hand.

For example, I can define forget safely:

    fn forget<T>(value: T) {
        struct Cycle<T> {
            this: RefCell<Option<Arc<Self>>>,
            value: T,
        }
        let cycle = Arc::new(Cycle { this: RefCell::new(None), value });
        *cycle.this.borrow_mut() = Some(cycle.clone());
    }
Quoted 6 lines
Quoted 2 lines

How would you ensure that this kind of pattern doesn't get written accidentally (or with many indirections)?

I don't think that the possibility of writing safe (but yet buggy) code is an argument against having the possibility of enforcing that a caller must write a comment for justification on certain things, such as mem::forget().

Quoted 6 lines

My argument is that the problem of forgetting a value is not self-contained like unsafe code is. Even if we were to document all forget or ManuallyDrop::new invocations (which we definitely should) we wouldn't get the security that one can't accidentally forget a lock guard. I'm totally in favor of mandating an explaining comment above forget calls (but not as a SAFETY comment).

Oh, I see where the misunderstanding might lie.

Let's take a look at FileDescriptorReservation::fd_install(). My proposal is to have something like:

	// SANITY: `fd_install` consumed file descriptor
	core::mem::forget(self);
	// SANITY: `fd_install` consumed file reference
	core::mem::forget(file);

Where we have e.g. clippy to complain if there is no "SANITY" (or whatever we call it) comment for mem::forget().

I'm not proposing a SAFETY comment.

Quoted 7 lines
Quoted 5 lines

But there's another reason I think having something like this could be
problematic: It might set the wrong incentive, as in "hey, I can just use a "sanity requirement" in my function rather figuring out how to ensure it through the type system, etc.".

I don't understand your point here, can you explain it more?

Does the explanation above make this concern clear for you?

#29 lore raw
Quoted 11 lines · Danilo Krummrich

Oh, I see where the misunderstanding might lie.

Let's take a look at FileDescriptorReservation::fd_install(). My proposal is to have something like:

        // SANITY: `fd_install` consumed file descriptor
        core::mem::forget(self);
        // SANITY: `fd_install` consumed file reference
        core::mem::forget(file);
Quoted 4 lines

Where we have e.g. clippy to complain if there is no "SANITY" (or whatever we call it) comment for mem::forget().

I'm not proposing a SAFETY comment.

Clippy could "easily" do that, but if we are to do something like that, then I would want to the ability to mark which ones need extra comments in the signature/item (e.g. as an attribute) , i.e. instead of in a global list somewhere like .clippy.toml.

It is also related to the "colored unsafe" idea we discussed very early, but without extra blocks on the call, just a comment.

Cheers,
Miguel

#30 lore raw
In reply to Danilo Krummrich
Quoted 66 lines · Danilo Krummrich
Quoted 48 lines · Benno Lossin
Quoted 40 lines · Danilo Krummrich
Quoted 39 lines · Benno Lossin
Quoted 16 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

I get where you're coming from, but this probably will very quickly get out of hand.

For example, I can define forget safely:

    fn forget<T>(value: T) {
        struct Cycle<T> {
            this: RefCell<Option<Arc<Self>>>,
            value: T,
        }
        let cycle = Arc::new(Cycle { this: RefCell::new(None), value });
        *cycle.this.borrow_mut() = Some(cycle.clone());
    }
Quoted 6 lines
Quoted 2 lines

How would you ensure that this kind of pattern doesn't get written accidentally (or with many indirections)?

I don't think that the possibility of writing safe (but yet buggy) code is an argument against having the possibility of enforcing that a caller must write a comment for justification on certain things, such as mem::forget().

Quoted 6 lines

My argument is that the problem of forgetting a value is not self-contained like unsafe code is. Even if we were to document all forget or ManuallyDrop::new invocations (which we definitely should) we wouldn't get the security that one can't accidentally forget a lock guard. I'm totally in favor of mandating an explaining comment above forget calls (but not as a SAFETY comment).

Oh, I see where the misunderstanding might lie.

Let's take a look at FileDescriptorReservation::fd_install(). My proposal is to have something like:

	// SANITY: `fd_install` consumed file descriptor
	core::mem::forget(self);
	// SANITY: `fd_install` consumed file reference
	core::mem::forget(file);
Quoted 4 lines

Where we have e.g. clippy to complain if there is no "SANITY" (or whatever we call it) comment for mem::forget().

I'm not proposing a SAFETY comment.

Ok, that's good :)

As for having a dedicated SANITY comment, I'm not sold yet. I'll think a bit more.

Quoted 8 lines
Quoted 6 lines
Quoted 4 lines

But there's another reason I think having something like this could be
problematic: It might set the wrong incentive, as in "hey, I can just use a "sanity requirement" in my function rather figuring out how to ensure it through the type system, etc.".

I don't understand your point here, can you explain it more?

Does the explanation above make this concern clear for you?

Do you mean having sanity requirements would lead to people putting less effort into designing a type system solution?

---
Cheers,
Benno

author maintainer
#31 lore raw
Quoted 82 lines · Benno Lossin
Quoted 66 lines · Danilo Krummrich
Quoted 48 lines · Benno Lossin
Quoted 40 lines · Danilo Krummrich
Quoted 39 lines · Benno Lossin
Quoted 16 lines · Danilo Krummrich
Quoted 10 lines · Danilo Krummrich

But let's define it then; what about:

"[Vec::set_len] takes (or kepps) ownership of all elements within the range [0; new_len] and abandons ownership of all values outside of this range, if any."

The caller may take ownership of the abandoned elements."

I'd argue that giving up ownership, while offering someone else to take it means that it implies that otherwise we'll just end up forgetting about the value.

Btw. I'd still prefer if we could enforce that the caller has to document what should happen to the abandoned value. But I acknowledge that the safety comment isn't the scope for it.

Quoted 7 lines

It'd be great if e.g. clippy would give us a tool to do something analogous to safety comments.

It think it would be useful to enfoce some additional safety documentation. For instance, I think the kernel would much benefit if we could enforce that mem::forget() must be justified with a comment, since as mentioned ina previous mail, it can cause fatal bugs, for instance when used on lock guards.

I get where you're coming from, but this probably will very quickly get out of hand.

For example, I can define forget safely:

    fn forget<T>(value: T) {
        struct Cycle<T> {
            this: RefCell<Option<Arc<Self>>>,
            value: T,
        }
        let cycle = Arc::new(Cycle { this: RefCell::new(None), value });
        *cycle.this.borrow_mut() = Some(cycle.clone());
    }
Quoted 6 lines
Quoted 2 lines

How would you ensure that this kind of pattern doesn't get written accidentally (or with many indirections)?

I don't think that the possibility of writing safe (but yet buggy) code is an argument against having the possibility of enforcing that a caller must write a comment for justification on certain things, such as mem::forget().

Quoted 6 lines

My argument is that the problem of forgetting a value is not self-contained like unsafe code is. Even if we were to document all forget or ManuallyDrop::new invocations (which we definitely should) we wouldn't get the security that one can't accidentally forget a lock guard. I'm totally in favor of mandating an explaining comment above forget calls (but not as a SAFETY comment).

Oh, I see where the misunderstanding might lie.

Let's take a look at FileDescriptorReservation::fd_install(). My proposal is to have something like:

	// SANITY: `fd_install` consumed file descriptor
	core::mem::forget(self);
	// SANITY: `fd_install` consumed file reference
	core::mem::forget(file);
Quoted 4 lines

Where we have e.g. clippy to complain if there is no "SANITY" (or whatever we call it) comment for mem::forget().

I'm not proposing a SAFETY comment.

Ok, that's good :)

As for having a dedicated SANITY comment, I'm not sold yet. I'll think a bit more.

Quoted 5 lines
Quoted 5 lines
Quoted 4 lines

But there's another reason I think having something like this could be
problematic: It might set the wrong incentive, as in "hey, I can just use a "sanity requirement" in my function rather figuring out how to ensure it through the type system, etc.".

Quoted 6 lines
Quoted 3 lines
Quoted 1 line

I don't understand your point here, can you explain it more?

Does the explanation above make this concern clear for you?

Do you mean having sanity requirements would lead to people putting less effort into designing a type system solution?

That'd be one concern, yes.

author maintainer
#32 lore raw
In reply to Tamir Duberstein
Quoted 18 lines · Tamir Duberstein
Quoted 16 lines · Danilo Krummrich

rust/kernel/alloc/kvec.rs pub fn len(&self) -> usize { · line 192

Quoted 11 lines
rust/kernel/alloc/kvec.rs +1 -1
index ae9d072741ce..8540d9e2b717 100644
@@ -189,7 +189,9 @@ pub fn len(&self) -> usize {
     ///
     /// - `new_len` must be less than or equal to [`Self::capacity`].
     /// - If `new_len` is greater than `self.len`, all elements within the interval
-    ///   [`self.len`,`new_len`) must be initialized.
+    ///   [`self.len`,`new_len`) must be initialized,

Why a comma here? There's a period on the previous list item, and a period seems more appropriate before the next item.

Quoted 1 line

Did you see this one? Was it intentional?

Yes, it's the continuation of the sentence of the previous one.

maintainer
#33 lore raw
In reply to Danilo Krummrich
Quoted 14 lines · Danilo Krummrich

Extend the safety requirements of set_len() to consider the case when the new length is smaller than the old length.

Fixes 2aac4cd7dae3 ("rust: alloc: implement kernel `Vec` type")
---
 rust/kernel/alloc/kvec.rs | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
rust/kernel/alloc/kvec.rs +0 -0
index ae9d072741ce..8540d9e2b717 100644

rust/kernel/alloc/kvec.rs pub fn len(&self) -> usize { · line 194

Quoted 7 lines
rust/kernel/alloc/kvec.rs +3 -1
@@ -189,7 +189,9 @@ pub fn len(&self) -> usize {
     ///
     /// - `new_len` must be less than or equal to [`Self::capacity`].
     /// - If `new_len` is greater than `self.len`, all elements within the interval
-    ///   [`self.len`,`new_len`) must be initialized.
+    ///   [`self.len`,`new_len`) must be initialized,
+    /// - if `new_len` is smaller than `self.len`, all elements within the interval
+    ///   [`new_len`, `self.len`) must either be dropped or copied and taken ownership of.

This should not be a safety requirement. How about instead adding a guarantee to the caller that the caller may safely take ownership of the values?

Alice