Skip to contentSkip to conversation

rust: mm: add abstractions for mm_struct and vm_area_struct

PATCH Superseded Alice Ryhl 2 participants 4 messages Jul 27, 2024 – Jul 30, 2024

Subsystems (3)

v2 of 16 Superseded Latest is v16 →

lore
Share & tools
lore

Alice Ryhl

author maintainer
#0 lore raw

PATCH v2 1/1 rust: mm: add abstractions for mm_struct and vm_area_struct

This is a follow-up to the page abstractions [1] that were recently merged in 6.11. Rust Binder will need these abstractions to manipulate the vma in its implementation of the mmap fop on the Binder file.

This patch is based on Wedson's implementation on the old rust branch, but has been changed significantly. All mistakes are Alice's.

---
Changes in v2:
- mm.rs is redesigned from scratch making use of AsRef
- Add notes about whether destructors may sleep
- Rename Area to VmArea
- Link to v1: https://lore.kernel.org/r/[email protected]

---
 rust/helpers.c         |  61 +++++++++
 rust/kernel/lib.rs     |   1 +
 rust/kernel/mm.rs      | 335 +++++++++++++++++++++++++++++++++++++++++++++++++
 rust/kernel/mm/virt.rs | 199 +++++++++++++++++++++++++++++
 rust/kernel/types.rs   |   9 ++
 5 files changed, 605 insertions(+)
rust/helpers.c +61 -0
index 305f0577fae9..d2205f980112 100644
@@ -199,6 +199,67 @@ rust_helper_krealloc(const void *objp, size_t new_size, gfp_t flags)
 }
 EXPORT_SYMBOL_GPL(rust_helper_krealloc);
 
+void rust_helper_mmgrab(struct mm_struct *mm)
+{
+	mmgrab(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmgrab);
+
+void rust_helper_mmdrop(struct mm_struct *mm)
+{
+	mmdrop(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmdrop);
+
+void rust_helper_mmget(struct mm_struct *mm)
+{
+	mmget(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmget);
+
+bool rust_helper_mmget_not_zero(struct mm_struct *mm)
+{
+	return mmget_not_zero(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmget_not_zero);
+
+void rust_helper_mmap_read_lock(struct mm_struct *mm)
+{
+	mmap_read_lock(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmap_read_lock);
+
+bool rust_helper_mmap_read_trylock(struct mm_struct *mm)
+{
+	return mmap_read_trylock(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmap_read_trylock);
+
+void rust_helper_mmap_read_unlock(struct mm_struct *mm)
+{
+	mmap_read_unlock(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmap_read_unlock);
+
+void rust_helper_mmap_write_lock(struct mm_struct *mm)
+{
+	mmap_write_lock(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmap_write_lock);
+
+void rust_helper_mmap_write_unlock(struct mm_struct *mm)
+{
+	mmap_write_unlock(mm);
+}
+EXPORT_SYMBOL_GPL(rust_helper_mmap_write_unlock);
+
+struct vm_area_struct *rust_helper_vma_lookup(struct mm_struct *mm,
+					      unsigned long addr)
+{
+	return vma_lookup(mm, addr);
+}
+EXPORT_SYMBOL_GPL(rust_helper_vma_lookup);
+
 /*
  * `bindgen` binds the C `size_t` type as the Rust `usize` type, so we can
  * use it in contexts where Rust expects a `usize` like slice (array) indices.
rust/kernel/lib.rs +1 -0
index 5d310e79485f..3cbc4cf847a2 100644
@@ -33,6 +33,7 @@
 pub mod ioctl;
 #[cfg(CONFIG_KUNIT)]
 pub mod kunit;
+pub mod mm;
 #[cfg(CONFIG_NET)]
 pub mod net;
 pub mod page;
rust/kernel/mm.rs new file +335 -0

335 lines not on this page · raw message

rust/kernel/mm/virt.rs new file +199 -0

199 lines not on this page · raw message

rust/kernel/types.rs +9 -0
index bd189d646adb..143a2bf06941 100644
@@ -366,6 +366,15 @@ pub unsafe fn from_raw(ptr: NonNull<T>) -> Self {
             _p: PhantomData,
         }
     }
+
+    /// Pass ownership of the refcount to a raw pointer.
+    pub fn into_raw(self) -> NonNull<T> {
+        let ptr = self.ptr;
+        // Skip the destructor.
+        core::mem::forget(self);
+
+        ptr
+    }
 }
 
 impl<T: AlwaysRefCounted> Clone for ARef<T> {

---
base-commit: b1263411112305acf2af728728591465becb45b0
change-id: 20240723-vma-f80119f9fb35

Best regards,

Signature · 1 line
Alice Ryhl <[email protected]>
#1 lore raw
Quoted 20 lines · Alice Ryhl
rust/kernel/mm.rs new file +20 -0
+/// A wrapper for the kernel's `struct mm_struct`.
+///
+/// Since `mm_users` may be zero, the associated address space may not exist anymore. You must use
+/// [`mmget_not_zero`] before accessing the address space.
+///
+/// The `ARef<Mm>` smart pointer holds an `mmgrab` refcount. Its destructor may sleep.
+///
+/// # Invariants
+///
+/// Values of this type are always refcounted.
+///
+/// [`mmget_not_zero`]: Mm::mmget_not_zero
+pub struct Mm {
+    mm: Opaque<bindings::mm_struct>,
+}
+
+/// A wrapper for the kernel's `struct mm_struct`.
+///
+/// This type is used only when `mm_users` is known to be non-zero at compile-time. It can be used
+/// to access the associated address space.

rust/kernel/mm.rs line 50

Quoted 10 lines
rust/kernel/mm.rs new file +10 -0
+///
+/// The `ARef<MmWithUser>` smart pointer holds an `mmget` refcount. Its destructor may sleep.
+///
+/// # Invariants
+///
+/// Values of this type are always refcounted. The value of `mm_users` is non-zero.
+#[repr(transparent)]
+pub struct MmWithUser {
+    mm: Mm,
+}

I personally wouldn't sort it this way (so struct decls, methods and then AlwaysRefCounted impl), but I would sort it first by the struct. I find it helpful to have the AlwaysRefCounted impl close to the struct declaration (similarly for Drop). But that might just be me.

rust/kernel/mm.rs line 60

Quoted 10 lines
rust/kernel/mm.rs new file +10 -0
+
+/// Equivalent to `ARef<MmWithUser>` but uses `mmput_async` in destructor.
+///
+/// The destructor of this type will never sleep.
+///
+/// # Invariants
+///
+/// `inner` points to a valid `mm_struct` and the `ARefMmWithUserAsync` owns an `mmget` refcount.
+pub struct ARefMmWithUserAsync {
+    inner: NonNull<bindings::mm_struct>,

I am confused, why doesn't mm: MM work here? I.e. also allow usage of ARef<MmWithUserAsync>.

Another approach might be to have the function on MmWithUser:

fn put_async(this: ARef<Self>)

Or do you need it to be done on drop?

rust/kernel/mm.rs line 70

Quoted 10 lines
rust/kernel/mm.rs new file +10 -0
+}
+
+// Make all `Mm` methods available on `MmWithUser`.
+impl Deref for MmWithUser {
+    type Target = Mm;
+
+    #[inline]
+    fn deref(&self) -> &Mm {
+        &self.mm
+    }

Does it really make sense to expose every function? E.g. mmget_not_zero would always succeed, right?

Quoted 3 lines
rust/kernel/mm.rs new file +3 -0
+}
+
+// These methods are safe to call even if `mm_users` is zero.

[...]

Quoted 24 lines
rust/kernel/mm/virt.rs new file +18 -0
new file mode 100644
index 000000000000..2e97ef1eac58
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Copyright (C) 2024 Google LLC.
+
+//! Virtual memory.
+
+use crate::{
+    bindings,
+    error::{to_result, Result},
+    page::Page,
+    types::Opaque,
+};
+
+/// A wrapper for the kernel's `struct vm_area_struct`.
+///
+/// It represents an area of virtual memory.
+#[repr(transparent)]
+pub struct VmArea {

rust/kernel/mm/virt.rs line 28

Quoted 10 lines
rust/kernel/mm/virt.rs new file +10 -0
+    vma: Opaque<bindings::vm_area_struct>,
+}
+
+impl VmArea {
+    /// Access a virtual memory area given a raw pointer.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that `vma` is valid for the duration of 'a, with shared access. The
+    /// caller must ensure that using the pointer for immutable operations is okay.

Nothing here states that the pointee is not allowed to be changed, unless you mean that by "shared access" which would not match my definition.

Quoted 21 lines
rust/kernel/mm/virt.rs new file +21 -0
+    #[inline]
+    pub unsafe fn from_raw_vma<'a>(vma: *const bindings::vm_area_struct) -> &'a Self {
+        // SAFETY: The caller ensures that the pointer is valid.
+        unsafe { &*vma.cast() }
+    }
+
+    /// Access a virtual memory area given a raw pointer.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that `vma` is valid for the duration of 'a, with exclusive access. The
+    /// caller must ensure that using the pointer for immutable and mutable operations is okay.
+    #[inline]
+    pub unsafe fn from_raw_vma_mut<'a>(vma: *mut bindings::vm_area_struct) -> &'a mut Self {
+        // SAFETY: The caller ensures that the pointer is valid.
+        unsafe { &mut *vma.cast() }
+    }
+
+    /// Returns a raw pointer to this area.
+    #[inline]
+    pub fn as_ptr(&self) -> *mut bindings::vm_area_struct {

rust/kernel/mm/virt.rs line 59

Quoted 10 lines
rust/kernel/mm/virt.rs new file +10 -0
+        self.vma.get()
+    }
+
+    /// Returns the flags associated with the virtual memory area.
+    ///
+    /// The possible flags are a combination of the constants in [`flags`].
+    #[inline]
+    pub fn flags(&self) -> usize {
+        // SAFETY: The pointer is valid since self is a reference. The field is valid for reading
+        // given a shared reference.

Why is the field not changed from the C side? Is this part readonly?

---
Cheers,
Benno

author maintainer
#2 lore raw
Quoted 29 lines · Benno Lossin
Quoted 27 lines · Alice Ryhl
rust/kernel/mm.rs new file +27 -0
+/// A wrapper for the kernel's `struct mm_struct`.
+///
+/// Since `mm_users` may be zero, the associated address space may not exist anymore. You must use
+/// [`mmget_not_zero`] before accessing the address space.
+///
+/// The `ARef<Mm>` smart pointer holds an `mmgrab` refcount. Its destructor may sleep.
+///
+/// # Invariants
+///
+/// Values of this type are always refcounted.
+///
+/// [`mmget_not_zero`]: Mm::mmget_not_zero
+pub struct Mm {
+    mm: Opaque<bindings::mm_struct>,
+}
+
+/// A wrapper for the kernel's `struct mm_struct`.
+///
+/// This type is used only when `mm_users` is known to be non-zero at compile-time. It can be used
+/// to access the associated address space.
+///
+/// The `ARef<MmWithUser>` smart pointer holds an `mmget` refcount. Its destructor may sleep.
+///
+/// # Invariants
+///
+/// Values of this type are always refcounted. The value of `mm_users` is non-zero.
+#[repr(transparent)]
Quoted 8 lines

rust/kernel/mm.rs line 50

Quoted 3 lines
rust/kernel/mm.rs new file +3 -0
+pub struct MmWithUser {
+    mm: Mm,
+}

I personally wouldn't sort it this way (so struct decls, methods and then AlwaysRefCounted impl), but I would sort it first by the struct. I find it helpful to have the AlwaysRefCounted impl close to the struct declaration (similarly for Drop). But that might just be me.

I can reorder for next version.

Quoted 5 lines
Quoted 5 lines
rust/kernel/mm.rs new file +5 -0
+
+/// Equivalent to `ARef<MmWithUser>` but uses `mmput_async` in destructor.
+///
+/// The destructor of this type will never sleep.
+///
Quoted 8 lines

rust/kernel/mm.rs line 60

Quoted 5 lines
rust/kernel/mm.rs new file +5 -0
+/// # Invariants
+///
+/// `inner` points to a valid `mm_struct` and the `ARefMmWithUserAsync` owns an `mmget` refcount.
+pub struct ARefMmWithUserAsync {
+    inner: NonNull<bindings::mm_struct>,

I am confused, why doesn't mm: MM work here? I.e. also allow usage of ARef<MmWithUserAsync>.

We could do that, but I don't know how much sense it makes. With Mm and MmWithUser there's a legitimate distinction between them that makes sense regardless of whether it's behind an ARef or &. But with the mmput_async case, the distinction only makes sense for ARef pointers, and &MmWithUser and &MmWithUserAsync would be 100% interchangeable.

That is to say, this is a property of the pointer, not the pointee. I don't think it makes sense semantically to have it be a wrapper around MmWithUser.

Quoted 5 lines

Another approach might be to have the function on MmWithUser:

fn put_async(this: ARef<Self>)

Or do you need it to be done on drop?

This needs to happen in drop so that use of the question mark operation doesn't suddenly result in sleep-in-atomic-ctx bugs.

Quoted 5 lines
Quoted 5 lines
rust/kernel/mm.rs new file +5 -0
+}
+
+// Make all `Mm` methods available on `MmWithUser`.
+impl Deref for MmWithUser {
+    type Target = Mm;
Quoted 8 lines

rust/kernel/mm.rs line 70

Quoted 5 lines
rust/kernel/mm.rs new file +5 -0
+
+    #[inline]
+    fn deref(&self) -> &Mm {
+        &self.mm
+    }

Does it really make sense to expose every function? E.g. mmget_not_zero would always succeed, right?

I don't think it's a problem. Right now it exposes mmget_not_zero, is_same_mm, and as_raw. The only one where it doesn't make much sense is mmget_not_zero, but I don't think it hurts either.

Quoted 36 lines
Quoted 3 lines
rust/kernel/mm.rs new file +3 -0
+}
+
+// These methods are safe to call even if `mm_users` is zero.

[...]

Quoted 30 lines
rust/kernel/mm/virt.rs new file +24 -0
new file mode 100644
index 000000000000..2e97ef1eac58
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Copyright (C) 2024 Google LLC.
+
+//! Virtual memory.
+
+use crate::{
+    bindings,
+    error::{to_result, Result},
+    page::Page,
+    types::Opaque,
+};
+
+/// A wrapper for the kernel's `struct vm_area_struct`.
+///
+/// It represents an area of virtual memory.
+#[repr(transparent)]
+pub struct VmArea {
+    vma: Opaque<bindings::vm_area_struct>,
+}
+
+impl VmArea {
+    /// Access a virtual memory area given a raw pointer.
+    ///
Quoted 8 lines

rust/kernel/mm/virt.rs line 28

Quoted 4 lines
rust/kernel/mm/virt.rs new file +4 -0
+    /// # Safety
+    ///
+    /// Callers must ensure that `vma` is valid for the duration of 'a, with shared access. The
+    /// caller must ensure that using the pointer for immutable operations is okay.

Nothing here states that the pointee is not allowed to be changed, unless you mean that by "shared access" which would not match my definition.

How about this?

Callers must ensure that:
* vma is valid for the duration of 'a.
* the caller holds the mmap read lock for 'a.

And from_raw_vma_mut would instead require the caller to hold the mmap write lock.

Quoted 25 lines
Quoted 25 lines
rust/kernel/mm/virt.rs new file +25 -0
+    #[inline]
+    pub unsafe fn from_raw_vma<'a>(vma: *const bindings::vm_area_struct) -> &'a Self {
+        // SAFETY: The caller ensures that the pointer is valid.
+        unsafe { &*vma.cast() }
+    }
+
+    /// Access a virtual memory area given a raw pointer.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that `vma` is valid for the duration of 'a, with exclusive access. The
+    /// caller must ensure that using the pointer for immutable and mutable operations is okay.
+    #[inline]
+    pub unsafe fn from_raw_vma_mut<'a>(vma: *mut bindings::vm_area_struct) -> &'a mut Self {
+        // SAFETY: The caller ensures that the pointer is valid.
+        unsafe { &mut *vma.cast() }
+    }
+
+    /// Returns a raw pointer to this area.
+    #[inline]
+    pub fn as_ptr(&self) -> *mut bindings::vm_area_struct {
+        self.vma.get()
+    }
+
+    /// Returns the flags associated with the virtual memory area.
Quoted 8 lines

rust/kernel/mm/virt.rs line 59

Quoted 6 lines
rust/kernel/mm/virt.rs new file +6 -0
+    ///
+    /// The possible flags are a combination of the constants in [`flags`].
+    #[inline]
+    pub fn flags(&self) -> usize {
+        // SAFETY: The pointer is valid since self is a reference. The field is valid for reading
+        // given a shared reference.

Why is the field not changed from the C side? Is this part readonly?

Because we hold the mmap read lock. (or the write lock)

Alice

#3 lore raw
Quoted 22 lines · Alice Ryhl
Quoted 13 lines · Benno Lossin

rust/kernel/mm.rs line 60

Quoted 9 lines · Alice Ryhl
rust/kernel/mm.rs new file +9 -0
+/// Equivalent to `ARef<MmWithUser>` but uses `mmput_async` in destructor.
+///
+/// The destructor of this type will never sleep.
+///
+/// # Invariants
+///
+/// `inner` points to a valid `mm_struct` and the `ARefMmWithUserAsync` owns an `mmget` refcount.
+pub struct ARefMmWithUserAsync {
+    inner: NonNull<bindings::mm_struct>,

I am confused, why doesn't mm: MM work here? I.e. also allow usage of ARef<MmWithUserAsync>.

We could do that, but I don't know how much sense it makes. With Mm and MmWithUser there's a legitimate distinction between them that makes sense regardless of whether it's behind an ARef or &. But with the mmput_async case, the distinction only makes sense for ARef pointers, and &MmWithUser and &MmWithUserAsync would be 100% interchangeable.

Quoted 3 lines

That is to say, this is a property of the pointer, not the pointee. I don't think it makes sense semantically to have it be a wrapper around MmWithUser.

Hmm, I don't think that is a problem. We have ARef for the following reasons (quoting myself from the ARef pattern thread):
(1) prevents having to implement multiple abstractions for a single C
object: say there is a struct foo that is both used via reference counting and by-value on the stack. Without ARef, we would have to write two abstractions, one for each use-case. With ARef, we can have one Foo that can be wrapped with ARef to represent a reference-counted object.
(2) ARef<T> always represents a reference counted object, so it helps with understanding the code. If you read Foo, you cannot be sure if it is heap or stack allocated.
(3) generalizes common code of reference-counted objects (ie avoiding code duplication) and concentration of unsafe code.

If you don't use ARef, you
- have to implement Deref, Drop, From<ARef<_>> manually,
- have a rather ugly name,
- don't benefit from the three points above.

I don't really see a downside to just using ARef in this case.

Quoted 72 lines
Quoted 5 lines

Another approach might be to have the function on MmWithUser:

fn put_async(this: ARef<Self>)

Or do you need it to be done on drop?

This needs to happen in drop so that use of the question mark operation doesn't suddenly result in sleep-in-atomic-ctx bugs.

Quoted 5 lines
Quoted 5 lines
rust/kernel/mm.rs new file +5 -0
+}
+
+// Make all `Mm` methods available on `MmWithUser`.
+impl Deref for MmWithUser {
+    type Target = Mm;
Quoted 8 lines

rust/kernel/mm.rs line 70

Quoted 5 lines
rust/kernel/mm.rs new file +5 -0
+
+    #[inline]
+    fn deref(&self) -> &Mm {
+        &self.mm
+    }

Does it really make sense to expose every function? E.g. mmget_not_zero would always succeed, right?

I don't think it's a problem. Right now it exposes mmget_not_zero, is_same_mm, and as_raw. The only one where it doesn't make much sense is mmget_not_zero, but I don't think it hurts either.

Quoted 36 lines
Quoted 3 lines
rust/kernel/mm.rs new file +3 -0
+}
+
+// These methods are safe to call even if `mm_users` is zero.

[...]

Quoted 30 lines
rust/kernel/mm/virt.rs new file +24 -0
new file mode 100644
index 000000000000..2e97ef1eac58
@@ -0,0 +1,199 @@
+// SPDX-License-Identifier: GPL-2.0
+
+// Copyright (C) 2024 Google LLC.
+
+//! Virtual memory.
+
+use crate::{
+    bindings,
+    error::{to_result, Result},
+    page::Page,
+    types::Opaque,
+};
+
+/// A wrapper for the kernel's `struct vm_area_struct`.
+///
+/// It represents an area of virtual memory.
+#[repr(transparent)]
+pub struct VmArea {
+    vma: Opaque<bindings::vm_area_struct>,
+}
+
+impl VmArea {
+    /// Access a virtual memory area given a raw pointer.
+    ///
Quoted 8 lines

rust/kernel/mm/virt.rs line 28

Quoted 4 lines
rust/kernel/mm/virt.rs new file +4 -0
+    /// # Safety
+    ///
+    /// Callers must ensure that `vma` is valid for the duration of 'a, with shared access. The
+    /// caller must ensure that using the pointer for immutable operations is okay.

Nothing here states that the pointee is not allowed to be changed, unless you mean that by "shared access" which would not match my definition.

Quoted 8 lines

How about this?

Callers must ensure that:
* vma is valid for the duration of 'a.
* the caller holds the mmap read lock for 'a.

And from_raw_vma_mut would instead require the caller to hold the mmap write lock.

SGTM.

Quoted 34 lines
Quoted 25 lines
Quoted 25 lines
rust/kernel/mm/virt.rs new file +25 -0
+    #[inline]
+    pub unsafe fn from_raw_vma<'a>(vma: *const bindings::vm_area_struct) -> &'a Self {
+        // SAFETY: The caller ensures that the pointer is valid.
+        unsafe { &*vma.cast() }
+    }
+
+    /// Access a virtual memory area given a raw pointer.
+    ///
+    /// # Safety
+    ///
+    /// Callers must ensure that `vma` is valid for the duration of 'a, with exclusive access. The
+    /// caller must ensure that using the pointer for immutable and mutable operations is okay.
+    #[inline]
+    pub unsafe fn from_raw_vma_mut<'a>(vma: *mut bindings::vm_area_struct) -> &'a mut Self {
+        // SAFETY: The caller ensures that the pointer is valid.
+        unsafe { &mut *vma.cast() }
+    }
+
+    /// Returns a raw pointer to this area.
+    #[inline]
+    pub fn as_ptr(&self) -> *mut bindings::vm_area_struct {
+        self.vma.get()
+    }
+
+    /// Returns the flags associated with the virtual memory area.
Quoted 8 lines

rust/kernel/mm/virt.rs line 59

Quoted 6 lines
rust/kernel/mm/virt.rs new file +6 -0
+    ///
+    /// The possible flags are a combination of the constants in [`flags`].
+    #[inline]
+    pub fn flags(&self) -> usize {
+        // SAFETY: The pointer is valid since self is a reference. The field is valid for reading
+        // given a shared reference.

Why is the field not changed from the C side? Is this part readonly?

Quoted 1 line

Because we hold the mmap read lock. (or the write lock)

Oh, then it would be good to have it be an invariant.

---
Cheers,
Benno