Skip to contentSkip to conversation

Devres optimization with bound devices

PATCH Superseded Danilo Krummrich 5 participants 27 messages Apr 26, 2025 – Apr 27, 2025

Subsystems (4)

v1 of 2 Superseded Latest is v2 →

lore
Share & tools
lore

Danilo Krummrich

author maintainer
#0 lore raw

cover Devres optimization with bound devices

This patch series implements a direct accessor for the data stored within a Devres container for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

The patches of this series are also available in [1].

[1] https://web.git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=rust/devres

Danilo Krummrich (3):
rust: revocable: implement Revocable::access()
rust: devres: implement Devres::access_with()
samples: rust: pci: take advantage of Devres::access_with()

 rust/kernel/devres.rs           | 35 +++++++++++++++++++++++++++++++++
 rust/kernel/revocable.rs        | 12 +++++++++++
 samples/rust/rust_driver_pci.rs | 12 +++++------
 3 files changed, 53 insertions(+), 6 deletions(-)

base-commit: b5cb47f81a2857d270cabbbb3a9feec0e483caed

Signature · 1 line
2.49.0
author maintainer
#1 lore raw

PATCH 1/3 rust: revocable: implement Revocable::access()

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes.

---
 rust/kernel/revocable.rs | 12 ++++++++++++
 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +12 -0
index 971d0dc38d83..33535de141ce 100644
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {
+        // SAFETY: By the safety requirement of this function it is guaranteed that
+        // `self.data.get()` is a valid pointer to an instance of `T`.
+        unsafe { &*self.data.get() }
+    }
+
     /// # Safety
     ///
     /// Callers must ensure that there are no more concurrent users of the revocable object.
Signature · 1 line
2.49.0
author maintainer
#11 lore raw

PATCH 2/3 rust: devres: implement Devres::access_with()

In reply to Danilo Krummrich

Implement a direct accessor for the data stored within the Devres for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

---
 rust/kernel/devres.rs | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
rust/kernel/devres.rs +35 -0
index 1e58f5d22044..ec2cd9cdda8b 100644
@@ -181,6 +181,41 @@ pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result {
 
         Ok(())
     }
+
+    /// Obtain `&'a T`, bypassing the [`Revocable`].
+    ///
+    /// This method allows to directly obtain a `&'a T`, bypassing the [`Revocable`], by presenting
+    /// a `&'a Device<Bound>` of the same [`Device`] this [`Devres`] instance has been created with.
+    ///
+    /// An error is returned if `dev` does not match the same [`Device`] this [`Devres`] instance
+    /// has been created with.
+    ///
+    /// # Example
+    ///
+    /// ```no_run
+    /// # use kernel::{device::Core, devres::Devres, pci};
+    ///
+    /// fn from_core(dev: &pci::Device<Core>, devres: Devres<pci::Bar<0x4>>) -> Result<()> {
+    ///     let bar = devres.access_with(dev.as_ref())?;
+    ///
+    ///     let _ = bar.read32(0x0);
+    ///
+    ///     // might_sleep()
+    ///
+    ///     bar.write32(0x42, 0x0);
+    ///
+    ///     Ok(())
+    /// }
+    pub fn access_with<'s, 'd: 's>(&'s self, dev: &'d Device<Bound>) -> Result<&'s T> {
+        if self.0.dev.as_raw() != dev.as_raw() {
+            return Err(EINVAL);
+        }
+
+        // SAFETY: `dev` being the same device as the device this `Devres` has been created for
+        // proofes that `self.0.data` hasn't been revoked and is guaranteed to not be revoked as
+        // long as `dev` lives; `dev` lives at least as long as `self`.
+        Ok(unsafe { self.deref().access() })
+    }
 }
 
 impl<T> Deref for Devres<T> {
Signature · 1 line
2.49.0
author maintainer
#18 lore raw

PATCH 3/3 samples: rust: pci: take advantage of Devres::access_with()

In reply to Danilo Krummrich

For the I/O operations executed from the probe() method, take advantage of Devres::access_with(), avoiding the atomic check and RCU read lock required otherwise entirely.

---
 samples/rust/rust_driver_pci.rs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
samples/rust/rust_driver_pci.rs +6 -6
index 9ce3a7323a16..3e1569e5096e 100644
@@ -83,12 +83,12 @@ fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self>
             GFP_KERNEL,
         )?;
 
-        let res = drvdata
-            .bar
-            .try_access_with(|b| Self::testdev(info, b))
-            .ok_or(ENXIO)??;
-
-        dev_info!(pdev.as_ref(), "pci-testdev data-match count: {}\n", res);
+        let bar = drvdata.bar.access_with(pdev.as_ref())?;
+        dev_info!(
+            pdev.as_ref(),
+            "pci-testdev data-match count: {}\n",
+            Self::testdev(info, bar)?
+        );
 
         Ok(drvdata.into())
     }
Signature · 1 line
2.49.0
Reviewed-by on v1 1/3
#2 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

In reply to Danilo Krummrich
Quoted 19 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes. ---> rust/kernel/revocable.rs | 12 ++++++++++++

 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 148

Quoted 10 lines
rust/kernel/revocable.rs +7 -0
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {

I'm not sure if the 's lifetime really carries much meaning here. I find just (explicit) 'a on both parameter and return value is clearer to me, but I'm not sure what others (particularly those not very familiar with rust) think of this.

Either way:

Quoted 8 lines
rust/kernel/revocable.rs +5 -0
+        // SAFETY: By the safety requirement of this function it is guaranteed that
+        // `self.data.get()` is a valid pointer to an instance of `T`.
+        unsafe { &*self.data.get() }
+    }
+
     /// # Safety
     ///
     /// Callers must ensure that there are no more concurrent users of the revocable object.
Reviewed-by on v1 2/3
#12 lore raw

Re: [PATCH 2/3] rust: devres: implement Devres::access_with()

In reply to Danilo Krummrich
Quoted 31 lines · Danilo Krummrich

Implement a direct accessor for the data stored within the Devres for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

---
 rust/kernel/devres.rs | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
rust/kernel/devres.rs +0 -0
index 1e58f5d22044..ec2cd9cdda8b 100644
@@ -181,6 +181,41 @@ pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result {
 

rust/kernel/devres.rs pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result { · line 191

Quoted 10 lines
rust/kernel/devres.rs +8 -0
         Ok(())
     }
+
+    /// Obtain `&'a T`, bypassing the [`Revocable`].
+    ///
+    /// This method allows to directly obtain a `&'a T`, bypassing the [`Revocable`], by presenting
+    /// a `&'a Device<Bound>` of the same [`Device`] this [`Devres`] instance has been created with.
+    ///
+    /// An error is returned if `dev` does not match the same [`Device`] this [`Devres`] instance
+    /// has been created with.

I would prefer this as a # Errors section.

Also are there any cases where this is actually wanted as an error? I'm not very familiar with the Revocable infrastructure, but I would assume a mismatch here to be a bug in almost every case, so a panic here might be reasonable.
(I would be fine with a reason for using an error here in the commit message or documentation/comments)

With that:

Quoted 30 lines
rust/kernel/devres.rs +27 -0
+    ///
+    /// # Example
+    ///
+    /// ```no_run
+    /// # use kernel::{device::Core, devres::Devres, pci};
+    ///
+    /// fn from_core(dev: &pci::Device<Core>, devres: Devres<pci::Bar<0x4>>) -> Result<()> {
+    ///     let bar = devres.access_with(dev.as_ref())?;
+    ///
+    ///     let _ = bar.read32(0x0);
+    ///
+    ///     // might_sleep()
+    ///
+    ///     bar.write32(0x42, 0x0);
+    ///
+    ///     Ok(())
+    /// }
+    pub fn access_with<'s, 'd: 's>(&'s self, dev: &'d Device<Bound>) -> Result<&'s T> {
+        if self.0.dev.as_raw() != dev.as_raw() {
+            return Err(EINVAL);
+        }
+
+        // SAFETY: `dev` being the same device as the device this `Devres` has been created for
+        // proofes that `self.0.data` hasn't been revoked and is guaranteed to not be revoked as
+        // long as `dev` lives; `dev` lives at least as long as `self`.
+        Ok(unsafe { self.deref().access() })
+    }
 }
 
 impl<T> Deref for Devres<T> {
#3 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

In reply to Christian Schrefl
Quoted 23 lines · Christian Schrefl
Quoted 22 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes. ---> rust/kernel/revocable.rs | 12 ++++++++++++

 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
Quoted 11 lines

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 148

Quoted 7 lines
rust/kernel/revocable.rs +7 -0
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {

I'm not sure if the 's lifetime really carries much meaning here. I find just (explicit) 'a on both parameter and return value is clearer to me, but I'm not sure what others (particularly those not very familiar with rust) think of this.

Yeah, I don't think we need two lifetimes here, the following version should be fine (with implicit lifetime):

	pub unsafe fn access(&self) -> &T { ... }

, because if you do:

	let revocable: &'1 Revocable = ...;
	...
	let t: &'2 T = unsafe { revocable.access() };

'1 should already outlive '2 (i.e. '1: '2).

Regards,
Boqun

Quoted 14 lines

Either way:

Quoted 8 lines
rust/kernel/revocable.rs +5 -0
+        // SAFETY: By the safety requirement of this function it is guaranteed that
+        // `self.data.get()` is a valid pointer to an instance of `T`.
+        unsafe { &*self.data.get() }
+    }
+
     /// # Safety
     ///
     /// Callers must ensure that there are no more concurrent users of the revocable object.
author maintainer
#4 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

Quoted 41 lines · Boqun Feng
Quoted 23 lines · Christian Schrefl
Quoted 22 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes. ---> rust/kernel/revocable.rs | 12 ++++++++++++

 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
Quoted 11 lines

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 148

Quoted 7 lines
rust/kernel/revocable.rs +7 -0
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {

I'm not sure if the 's lifetime really carries much meaning here. I find just (explicit) 'a on both parameter and return value is clearer to me, but I'm not sure what others (particularly those not very familiar with rust) think of this.

Yeah, I don't think we need two lifetimes here, the following version should be fine (with implicit lifetime):

	pub unsafe fn access(&self) -> &T { ... }
Quoted 7 lines

, because if you do:

	let revocable: &'1 Revocable = ...;
	...
	let t: &'2 T = unsafe { revocable.access() };

'1 should already outlive '2 (i.e. '1: '2).

Yes, this is indeed sufficient, that's why I wrote

"The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes."

below the commit message. :)

Any opinions in terms of documentation purposes?

Quoted 14 lines
Quoted 14 lines

Either way:

Quoted 8 lines
rust/kernel/revocable.rs +5 -0
+        // SAFETY: By the safety requirement of this function it is guaranteed that
+        // `self.data.get()` is a valid pointer to an instance of `T`.
+        unsafe { &*self.data.get() }
+    }
+
     /// # Safety
     ///
     /// Callers must ensure that there are no more concurrent users of the revocable object.
#7 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

In reply to Boqun Feng
Quoted 41 lines · Boqun Feng
Quoted 23 lines · Christian Schrefl
Quoted 22 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes. ---> rust/kernel/revocable.rs | 12 ++++++++++++

 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
Quoted 11 lines

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 148

Quoted 7 lines
rust/kernel/revocable.rs +7 -0
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {

I'm not sure if the 's lifetime really carries much meaning here. I find just (explicit) 'a on both parameter and return value is clearer to me, but I'm not sure what others (particularly those not very familiar with rust) think of this.

Yeah, I don't think we need two lifetimes here, the following version should be fine (with implicit lifetime):

	pub unsafe fn access(&self) -> &T { ... }
Quoted 7 lines

, because if you do:

	let revocable: &'1 Revocable = ...;
	...
	let t: &'2 T = unsafe { revocable.access() };

'1 should already outlive '2 (i.e. '1: '2).

I understand that implicit lifetimes desugars to effectively the same code, I just think that keeping a explicit 'a makes it a bit more obvious that the lifetimes need to be considered here.

But I'm also fine with just implicit lifetimes here.

Cheers
Christian

author maintainer
#13 lore raw

Re: [PATCH 2/3] rust: devres: implement Devres::access_with()

In reply to Christian Schrefl
Quoted 32 lines · Christian Schrefl
Quoted 31 lines · Danilo Krummrich

Implement a direct accessor for the data stored within the Devres for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

---
 rust/kernel/devres.rs | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
rust/kernel/devres.rs +0 -0
index 1e58f5d22044..ec2cd9cdda8b 100644
@@ -181,6 +181,41 @@ pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result {
 
Quoted 12 lines

rust/kernel/devres.rs pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result { · line 191

Quoted 10 lines
rust/kernel/devres.rs +8 -0
         Ok(())
     }
+
+    /// Obtain `&'a T`, bypassing the [`Revocable`].
+    ///
+    /// This method allows to directly obtain a `&'a T`, bypassing the [`Revocable`], by presenting
+    /// a `&'a Device<Bound>` of the same [`Device`] this [`Devres`] instance has been created with.
+    ///
+    /// An error is returned if `dev` does not match the same [`Device`] this [`Devres`] instance
+    /// has been created with.

I would prefer this as a # Errors section.

I can make this an # Errors section.

Quoted 4 lines

Also are there any cases where this is actually wanted as an error? I'm not very familiar with the Revocable infrastructure, but I would assume a mismatch here to be a bug in almost every case, so a panic here might be reasonable.

Passing in a device reference that doesn't match the device the Devres instance was created with would indeed be a bug, but a panic isn't the solution, since we can handle this error just fine.

We never panic the whole kernel unless things go so utterly wrong that we can't can't recover from it; e.g. if otherwise we'd likely corrupt memory, etc.

Quoted 2 lines

(I would be fine with a reason for using an error here in the commit message or documentation/comments)

I don't think we need this in this commit or the method's documentation, it's a general kernel rule not to panic unless there's really no other way.

Quoted 36 lines

With that:

Quoted 30 lines
rust/kernel/devres.rs +27 -0
+    ///
+    /// # Example
+    ///
+    /// ```no_run
+    /// # use kernel::{device::Core, devres::Devres, pci};
+    ///
+    /// fn from_core(dev: &pci::Device<Core>, devres: Devres<pci::Bar<0x4>>) -> Result<()> {
+    ///     let bar = devres.access_with(dev.as_ref())?;
+    ///
+    ///     let _ = bar.read32(0x0);
+    ///
+    ///     // might_sleep()
+    ///
+    ///     bar.write32(0x42, 0x0);
+    ///
+    ///     Ok(())
+    /// }
+    pub fn access_with<'s, 'd: 's>(&'s self, dev: &'d Device<Bound>) -> Result<&'s T> {
+        if self.0.dev.as_raw() != dev.as_raw() {
+            return Err(EINVAL);
+        }
+
+        // SAFETY: `dev` being the same device as the device this `Devres` has been created for
+        // proofes that `self.0.data` hasn't been revoked and is guaranteed to not be revoked as
+        // long as `dev` lives; `dev` lives at least as long as `self`.
+        Ok(unsafe { self.deref().access() })
+    }
 }
 
 impl<T> Deref for Devres<T> {
#24 lore raw
In reply to Danilo Krummrich
Quoted 11 lines · Danilo Krummrich

This patch series implements a direct accessor for the data stored within a Devres container for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

Quoted 9 lines

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

Nice! However, IIUC, if the users use Devres::new() to create a Devres , they will have a Devres they can revoke anytime, which means you can still revoke the Devres even if the device is bound.

Also if a Devres belongs to device A, but someone passes device B's bound reference to access_with(), the compiler won't check for that, and the Devres can be being revoked as the same, no? If so the function is not safe.

Regards,
Boqun

Quoted 19 lines

The patches of this series are also available in [1].

[1] https://web.git.kernel.org/pub/scm/linux/kernel/git/dakr/linux.git/log/?h=rust/devres

Danilo Krummrich (3):
rust: revocable: implement Revocable::access()
rust: devres: implement Devres::access_with()
samples: rust: pci: take advantage of Devres::access_with()

 rust/kernel/devres.rs           | 35 +++++++++++++++++++++++++++++++++
 rust/kernel/revocable.rs        | 12 +++++++++++
 samples/rust/rust_driver_pci.rs | 12 +++++------
 3 files changed, 53 insertions(+), 6 deletions(-)

base-commit: b5cb47f81a2857d270cabbbb3a9feec0e483caed

Signature · 1 line
2.49.0
#5 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

In reply to Danilo Krummrich
Quoted 50 lines · Danilo Krummrich
Quoted 41 lines · Boqun Feng
Quoted 23 lines · Christian Schrefl
Quoted 22 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes. ---> rust/kernel/revocable.rs | 12 ++++++++++++

 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
Quoted 11 lines

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 148

Quoted 7 lines
rust/kernel/revocable.rs +7 -0
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {

I'm not sure if the 's lifetime really carries much meaning here. I find just (explicit) 'a on both parameter and return value is clearer to me, but I'm not sure what others (particularly those not very familiar with rust) think of this.

Yeah, I don't think we need two lifetimes here, the following version should be fine (with implicit lifetime):

	pub unsafe fn access(&self) -> &T { ... }
Quoted 7 lines

, because if you do:

	let revocable: &'1 Revocable = ...;
	...
	let t: &'2 T = unsafe { revocable.access() };

'1 should already outlive '2 (i.e. '1: '2).

Quoted 8 lines

Yes, this is indeed sufficient, that's why I wrote

"The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes."

below the commit message. :)

Any opinions in terms of documentation purposes?

I would prefer just one explicit lifetime, but I'm not sure about others.
But I think either way is fine.

Maybe I should have written it more clearly that I only meant that the second lifetime makes it IMO unnecessarily complicated when reading it.

Cheers
Christian

author maintainer
#25 lore raw
In reply to Boqun Feng
Quoted 18 lines · Boqun Feng
Quoted 17 lines · Danilo Krummrich

This patch series implements a direct accessor for the data stored within a Devres container for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Quoted 7 lines
Quoted 3 lines

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

Nice! However, IIUC, if the users use Devres::new() to create a Devres , they will have a Devres they can revoke anytime, which means you can still revoke the Devres even if the device is bound.

No, a user of Devres can't revoke the inner Revocable itself. A user can only drop the Devres instance, in which case the user also wouldn't be able to call access_with() anymore.

Quoted 4 lines

Also if a Devres belongs to device A, but someone passes device B's bound reference to access_with(), the compiler won't check for that, and the Devres can be being revoked as the same, no? If so the function is not safe.

Devres::access_with() compares the Device<Bound> parameter with its inner ARef<Device>, and just fails if they don't match.

#26 lore raw
Quoted 23 lines · Danilo Krummrich
Quoted 22 lines · Boqun Feng
Quoted 20 lines · Danilo Krummrich

This patch series implements a direct accessor for the data stored within a Devres container for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

Quoted 8 lines
Quoted 3 lines

Nice! However, IIUC, if the users use Devres::new() to create a Devres , they will have a Devres they can revoke anytime, which means you can still revoke the Devres even if the device is bound.

No, a user of Devres can't revoke the inner Revocable itself. A user can only drop the Devres instance, in which case the user also wouldn't be able to call access_with() anymore.

Oh, right, because it's a Devres not Revocable in general.

Quoted 7 lines
Quoted 4 lines

Also if a Devres belongs to device A, but someone passes device B's bound reference to access_with(), the compiler won't check for that, and the Devres can be being revoked as the same, no? If so the function is not safe.

Devres::access_with() compares the Device<Bound> parameter with its inner ARef<Device>, and just fails if they don't match.

I see, I missed that. Thanks!

Regards,
Boqun

#14 lore raw

Re: [PATCH 2/3] rust: devres: implement Devres::access_with()

In reply to Danilo Krummrich
Quoted 57 lines · Danilo Krummrich
Quoted 32 lines · Christian Schrefl
Quoted 31 lines · Danilo Krummrich

Implement a direct accessor for the data stored within the Devres for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

---
 rust/kernel/devres.rs | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
rust/kernel/devres.rs +0 -0
index 1e58f5d22044..ec2cd9cdda8b 100644
@@ -181,6 +181,41 @@ pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result {
 
Quoted 12 lines

rust/kernel/devres.rs pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result { · line 191

Quoted 10 lines
rust/kernel/devres.rs +8 -0
         Ok(())
     }
+
+    /// Obtain `&'a T`, bypassing the [`Revocable`].
+    ///
+    /// This method allows to directly obtain a `&'a T`, bypassing the [`Revocable`], by presenting
+    /// a `&'a Device<Bound>` of the same [`Device`] this [`Devres`] instance has been created with.
+    ///
+    /// An error is returned if `dev` does not match the same [`Device`] this [`Devres`] instance
+    /// has been created with.

I would prefer this as a # Errors section.

I can make this an # Errors section.

Quoted 4 lines

Also are there any cases where this is actually wanted as an error? I'm not very familiar with the Revocable infrastructure, but I would assume a mismatch here to be a bug in almost every case, so a panic here might be reasonable.

Passing in a device reference that doesn't match the device the Devres instance was created with would indeed be a bug, but a panic isn't the solution, since we can handle this error just fine.

Quoted 7 lines

We never panic the whole kernel unless things go so utterly wrong that we can't can't recover from it; e.g. if otherwise we'd likely corrupt memory, etc.

Quoted 2 lines

(I would be fine with a reason for using an error here in the commit message or documentation/comments)

I don't think we need this in this commit or the method's documentation, it's a general kernel rule not to panic unless there's really no other way.

Alright I'm fine with it then.

I just don't think that most users of the function would be able to gracefully recover from that other than failing the probe or whatever, but it makes sense to allow the caller to deal with this.

Cheers
Christian

#6 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

In reply to Danilo Krummrich
Quoted 48 lines · Danilo Krummrich
Quoted 47 lines · Boqun Feng
Quoted 23 lines · Christian Schrefl
Quoted 22 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes. ---> rust/kernel/revocable.rs | 12 ++++++++++++

 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
Quoted 11 lines

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 148

Quoted 7 lines
rust/kernel/revocable.rs +7 -0
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {

I'm not sure if the 's lifetime really carries much meaning here. I find just (explicit) 'a on both parameter and return value is clearer to me, but I'm not sure what others (particularly those not very familiar with rust) think of this.

Yeah, I don't think we need two lifetimes here, the following version should be fine (with implicit lifetime):

	pub unsafe fn access(&self) -> &T { ... }

, because if you do:

	let revocable: &'1 Revocable = ...;
	...
	let t: &'2 T = unsafe { revocable.access() };
Quoted 9 lines
Quoted 1 line

'1 should already outlive '2 (i.e. '1: '2).

Yes, this is indeed sufficient, that's why I wrote

"The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes."

below the commit message. :)

Sorry, I overlooked.

Quoted 2 lines

Any opinions in terms of documentation purposes?

I think for access() the explicit lifetimes is unnecessary, because it's a one-lifetime case, the two explicit lifetimes would make a simple case looking complicated.

For access_with(), that's needed and a good idea.

Just my two cents.

Regards,
Boqun

Quoted 14 lines
Quoted 14 lines
Quoted 14 lines

Either way:

Quoted 8 lines
rust/kernel/revocable.rs +5 -0
+        // SAFETY: By the safety requirement of this function it is guaranteed that
+        // `self.data.get()` is a valid pointer to an instance of `T`.
+        unsafe { &*self.data.get() }
+    }
+
     /// # Safety
     ///
     /// Callers must ensure that there are no more concurrent users of the revocable object.
#8 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

In reply to Christian Schrefl
Quoted 48 lines · Christian Schrefl
Quoted 47 lines · Boqun Feng
Quoted 23 lines · Christian Schrefl
Quoted 22 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes. ---> rust/kernel/revocable.rs | 12 ++++++++++++

 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
Quoted 11 lines

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 148

Quoted 7 lines
rust/kernel/revocable.rs +7 -0
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {

I'm not sure if the 's lifetime really carries much meaning here. I find just (explicit) 'a on both parameter and return value is clearer to me, but I'm not sure what others (particularly those not very familiar with rust) think of this.

Yeah, I don't think we need two lifetimes here, the following version should be fine (with implicit lifetime):

	pub unsafe fn access(&self) -> &T { ... }

, because if you do:

	let revocable: &'1 Revocable = ...;
	...
	let t: &'2 T = unsafe { revocable.access() };
Quoted 8 lines
Quoted 1 line

'1 should already outlive '2 (i.e. '1: '2).

I understand that implicit lifetimes desugars to effectively the same code, I just think that keeping a explicit 'a makes it a bit more obvious that the lifetimes need to be considered here.

But I'm also fine with just implicit lifetimes here.

We elide lifetimes all over the place especially for methods taking &self and returning &T. I don't think that it serves a purpose here.

---
Cheers,
Benno

#15 lore raw

Re: [PATCH 2/3] rust: devres: implement Devres::access_with()

In reply to Christian Schrefl
Quoted 63 lines · Christian Schrefl
Quoted 62 lines · Danilo Krummrich
Quoted 32 lines · Christian Schrefl
Quoted 31 lines · Danilo Krummrich

Implement a direct accessor for the data stored within the Devres for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

---
 rust/kernel/devres.rs | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
rust/kernel/devres.rs +0 -0
index 1e58f5d22044..ec2cd9cdda8b 100644
@@ -181,6 +181,41 @@ pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result {
 
Quoted 12 lines

rust/kernel/devres.rs pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result { · line 191

Quoted 10 lines
rust/kernel/devres.rs +8 -0
         Ok(())
     }
+
+    /// Obtain `&'a T`, bypassing the [`Revocable`].
+    ///
+    /// This method allows to directly obtain a `&'a T`, bypassing the [`Revocable`], by presenting
+    /// a `&'a Device<Bound>` of the same [`Device`] this [`Devres`] instance has been created with.
+    ///
+    /// An error is returned if `dev` does not match the same [`Device`] this [`Devres`] instance
+    /// has been created with.

I would prefer this as a # Errors section.

I can make this an # Errors section.

Quoted 4 lines

Also are there any cases where this is actually wanted as an error? I'm not very familiar with the Revocable infrastructure, but I would assume a mismatch here to be a bug in almost every case, so a panic here might be reasonable.

Passing in a device reference that doesn't match the device the Devres instance was created with would indeed be a bug, but a panic isn't the solution, since we can handle this error just fine.

We never panic the whole kernel unless things go so utterly wrong that we can't can't recover from it; e.g. if otherwise we'd likely corrupt memory, etc.

Quoted 2 lines

(I would be fine with a reason for using an error here in the commit message or documentation/comments)

Quoted 8 lines
Quoted 2 lines

I don't think we need this in this commit or the method's documentation, it's a general kernel rule not to panic unless there's really no other way.

Alright I'm fine with it then.

I just don't think that most users of the function would be able to gracefully recover from that other than failing the probe or whatever, but it makes sense to allow the caller to deal with this.

Failing the probe *is* "gracefully" handling the error. As Danilo said, a panic is the last resort when the whole world is on fire and you want to avoid doing more damage to the system.

---
Cheers,
Benno

#9 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

In reply to Danilo Krummrich
Quoted 20 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes.

---
 rust/kernel/revocable.rs | 12 ++++++++++++
 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 147

Quoted 9 lines
rust/kernel/revocable.rs +6 -0
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.

Ah I missed this in my other email, in case you want to directly refer to the lifetime, you should keep it defined. I would still remove the 's lifetime though.

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 150

Quoted 3 lines
rust/kernel/revocable.rs +3 -0
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {
+        // SAFETY: By the safety requirement of this function it is guaranteed that
+        // `self.data.get()` is a valid pointer to an instance of `T`.

I don't see how the "not-being revoked" state makes the data ptr be valid. Is that an invariant of Revocable? (it's not documented to have any invariants)

---
Cheers,
Benno

Quoted 6 lines
rust/kernel/revocable.rs +3 -0
+        unsafe { &*self.data.get() }
+    }
+
     /// # Safety
     ///
     /// Callers must ensure that there are no more concurrent users of the revocable object.
#19 lore raw

Re: [PATCH 3/3] samples: rust: pci: take advantage of Devres::access_with()

In reply to Danilo Krummrich
Quoted 14 lines · Danilo Krummrich

For the I/O operations executed from the probe() method, take advantage of Devres::access_with(), avoiding the atomic check and RCU read lock required otherwise entirely.

---
 samples/rust/rust_driver_pci.rs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
samples/rust/rust_driver_pci.rs +0 -0
index 9ce3a7323a16..3e1569e5096e 100644

samples/rust/rust_driver_pci.rs fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self> · line 86

Quoted 10 lines
samples/rust/rust_driver_pci.rs +1 -6
@@ -83,12 +83,12 @@ fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self>
             GFP_KERNEL,
         )?;
 
-        let res = drvdata
-            .bar
-            .try_access_with(|b| Self::testdev(info, b))
-            .ok_or(ENXIO)??;
-
-        dev_info!(pdev.as_ref(), "pci-testdev data-match count: {}\n", res);
+        let bar = drvdata.bar.access_with(pdev.as_ref())?;

Since this code might inspire other code, I don't think that we should return EINVAL here (bubbled up from access_with). Not sure what the correct thing here would be though...

---
Cheers,
Benno

Quoted 8 lines
samples/rust/rust_driver_pci.rs +5 -0
+        dev_info!(
+            pdev.as_ref(),
+            "pci-testdev data-match count: {}\n",
+            Self::testdev(info, bar)?
+        );
 
         Ok(drvdata.into())
     }
author maintainer
#10 lore raw

Re: [PATCH 1/3] rust: revocable: implement Revocable::access()

In reply to Benno Lossin
Quoted 34 lines · Benno Lossin
Quoted 20 lines · Danilo Krummrich

Implement an unsafe direct accessor for the data stored within the Revocable.

This is useful for cases where we can proof that the data stored within the Revocable is not and cannot be revoked for the duration of the lifetime of the returned reference.

---
The explicit lifetimes in access() probably don't serve a practical purpose, but I found them to be useful for documentation purposes.

---
 rust/kernel/revocable.rs | 12 ++++++++++++
 1 file changed, 12 insertions(+)
rust/kernel/revocable.rs +0 -0
index 971d0dc38d83..33535de141ce 100644

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 147

Quoted 9 lines
rust/kernel/revocable.rs +6 -0
@@ -139,6 +139,18 @@ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
         self.try_access().map(|t| f(&*t))
     }
 
+    /// Directly access the revocable wrapped object.
+    ///
+    /// # Safety
+    ///
+    /// The caller must ensure this [`Revocable`] instance hasn't been revoked and won't be revoked
+    /// for the duration of `'a`.

Ah I missed this in my other email, in case you want to directly refer to the lifetime, you should keep it defined. I would still remove the 's lifetime though.

Quoted 7 lines

rust/kernel/revocable.rs pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> { · line 150

Quoted 3 lines
rust/kernel/revocable.rs +3 -0
+    pub unsafe fn access<'a, 's: 'a>(&'s self) -> &'a T {
+        // SAFETY: By the safety requirement of this function it is guaranteed that
+        // `self.data.get()` is a valid pointer to an instance of `T`.

I don't see how the "not-being revoked" state makes the data ptr be valid. Is that an invariant of Revocable? (it's not documented to have any invariants)

What else makes it valid?

AFAICS, try_access() and try_access_with_guard() argue the exact same way, except that the reason for not being revoked is the atomic check and the RCU read lock.

author maintainer
#20 lore raw

Re: [PATCH 3/3] samples: rust: pci: take advantage of Devres::access_with()

In reply to Benno Lossin
Quoted 18 lines · Benno Lossin
Quoted 17 lines · Danilo Krummrich

For the I/O operations executed from the probe() method, take advantage of Devres::access_with(), avoiding the atomic check and RCU read lock required otherwise entirely.

---
 samples/rust/rust_driver_pci.rs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
samples/rust/rust_driver_pci.rs +0 -0
index 9ce3a7323a16..3e1569e5096e 100644
@@ -83,12 +83,12 @@ fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self>
             GFP_KERNEL,
         )?;
 
Quoted 11 lines

samples/rust/rust_driver_pci.rs fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self> · line 86

Quoted 7 lines
samples/rust/rust_driver_pci.rs +1 -6
-        let res = drvdata
-            .bar
-            .try_access_with(|b| Self::testdev(info, b))
-            .ok_or(ENXIO)??;
-
-        dev_info!(pdev.as_ref(), "pci-testdev data-match count: {}\n", res);
+        let bar = drvdata.bar.access_with(pdev.as_ref())?;

Since this code might inspire other code, I don't think that we should return EINVAL here (bubbled up from access_with). Not sure what the correct thing here would be though...

I can't think of any other error code that would match better, EINVAL seems to be the correct thing. Maybe one could argue for ENODEV, but I still think EINVAL fits better.

#21 lore raw

Re: [PATCH 3/3] samples: rust: pci: take advantage of Devres::access_with()

Quoted 31 lines · Danilo Krummrich
Quoted 18 lines · Benno Lossin
Quoted 17 lines · Danilo Krummrich

For the I/O operations executed from the probe() method, take advantage of Devres::access_with(), avoiding the atomic check and RCU read lock required otherwise entirely.

---
 samples/rust/rust_driver_pci.rs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
samples/rust/rust_driver_pci.rs +0 -0
index 9ce3a7323a16..3e1569e5096e 100644
@@ -83,12 +83,12 @@ fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self>
             GFP_KERNEL,
         )?;
 
Quoted 11 lines

samples/rust/rust_driver_pci.rs fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self> · line 86

Quoted 7 lines
samples/rust/rust_driver_pci.rs +1 -6
-        let res = drvdata
-            .bar
-            .try_access_with(|b| Self::testdev(info, b))
-            .ok_or(ENXIO)??;
-
-        dev_info!(pdev.as_ref(), "pci-testdev data-match count: {}\n", res);
+        let bar = drvdata.bar.access_with(pdev.as_ref())?;

Since this code might inspire other code, I don't think that we should return EINVAL here (bubbled up from access_with). Not sure what the correct thing here would be though...

Quoted 3 lines

I can't think of any other error code that would match better, EINVAL seems to be the correct thing. Maybe one could argue for ENODEV, but I still think EINVAL fits better.

The previous iteration of the sample used the ENXIO error code.

In this sample it should be impossible to trigger the error path. But others might copy the code into a context where that is not the case and then might have a horrible time debugging where the EINVAL came from. I don't know if our answer to that should be "improve debugging errors in general" or "improve the error handling in this case". I have no idea how the former could look like, maybe something around #[track_caller] and noting the lines where an error was created? For the latter case, we could do:

    let bar = match drvdata.bar.access_with(pdev.as_ref()) {
        Ok(bar) => bar,
        Err(_) => {
            // `bar` was just created using the `pdev` above, so this should never happen.
            return Err(ENXIO);
        }
    };

---
Cheers,
Benno

author maintainer
#22 lore raw

Re: [PATCH 3/3] samples: rust: pci: take advantage of Devres::access_with()

Quoted 32 lines · Benno Lossin
Quoted 31 lines · Danilo Krummrich
Quoted 29 lines · Benno Lossin
Quoted 14 lines · Danilo Krummrich

For the I/O operations executed from the probe() method, take advantage of Devres::access_with(), avoiding the atomic check and RCU read lock required otherwise entirely.

---
 samples/rust/rust_driver_pci.rs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
samples/rust/rust_driver_pci.rs +0 -0
index 9ce3a7323a16..3e1569e5096e 100644

samples/rust/rust_driver_pci.rs fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self> · line 86

Quoted 10 lines
samples/rust/rust_driver_pci.rs +1 -6
@@ -83,12 +83,12 @@ fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self>
             GFP_KERNEL,
         )?;
 
-        let res = drvdata
-            .bar
-            .try_access_with(|b| Self::testdev(info, b))
-            .ok_or(ENXIO)??;
-
-        dev_info!(pdev.as_ref(), "pci-testdev data-match count: {}\n", res);
+        let bar = drvdata.bar.access_with(pdev.as_ref())?;

Since this code might inspire other code, I don't think that we should return EINVAL here (bubbled up from access_with). Not sure what the correct thing here would be though...

Quoted 5 lines
Quoted 3 lines

I can't think of any other error code that would match better, EINVAL seems to be the correct thing. Maybe one could argue for ENODEV, but I still think EINVAL fits better.

The previous iteration of the sample used the ENXIO error code.

Yes, because the cause of error for try_access_with() failing would have been that the device was unbound already, hence a different error code.

Quoted 3 lines

In this sample it should be impossible to trigger the error path. But others might copy the code into a context where that is not the case and then might have a horrible time debugging where the EINVAL came from.

I think it should always pretty unlikely design wise to supply a non-matching device.

Quoted 11 lines

I don't know if our answer to that should be "improve debugging errors in general" or "improve the error handling in this case". I have no idea how the former could look like, maybe something around #[track_caller] and noting the lines where an error was created? For the latter case, we could do:

    let bar = match drvdata.bar.access_with(pdev.as_ref()) {
        Ok(bar) => bar,
        Err(_) => {
            // `bar` was just created using the `pdev` above, so this should never happen.
            return Err(ENXIO);

If the caller really put in a non-matching device we can't say for sure that the cause is ENXIO, the only thing we know is that the code author confused device instances, so I think EINVAL is still the correct thing to return.

The problem that it might be difficult to figure out the source of the error code has always been present in the kernel, and while I'm not saying we shouldn't aim for improving this, I don't think this patch is quite the place for this.

#16 lore raw

Re: [PATCH 2/3] rust: devres: implement Devres::access_with()

In reply to Danilo Krummrich
Quoted 49 lines · Danilo Krummrich

Implement a direct accessor for the data stored within the Devres for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

---
 rust/kernel/devres.rs | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
rust/kernel/devres.rs +16 -0
index 1e58f5d22044..ec2cd9cdda8b 100644
@@ -181,6 +181,41 @@ pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result {
 
         Ok(())
     }
+
+    /// Obtain `&'a T`, bypassing the [`Revocable`].
+    ///
+    /// This method allows to directly obtain a `&'a T`, bypassing the [`Revocable`], by presenting
+    /// a `&'a Device<Bound>` of the same [`Device`] this [`Devres`] instance has been created with.
+    ///
+    /// An error is returned if `dev` does not match the same [`Device`] this [`Devres`] instance
+    /// has been created with.
+    ///
+    /// # Example
+    ///
+    /// ```no_run
+    /// # use kernel::{device::Core, devres::Devres, pci};
+    ///
+    /// fn from_core(dev: &pci::Device<Core>, devres: Devres<pci::Bar<0x4>>) -> Result<()> {
+    ///     let bar = devres.access_with(dev.as_ref())?;

rust/kernel/devres.rs pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result { · line 209

Quoted 10 lines
rust/kernel/devres.rs +10 -0
+    ///
+    ///     let _ = bar.read32(0x0);
+    ///
+    ///     // might_sleep()
+    ///
+    ///     bar.write32(0x42, 0x0);
+    ///
+    ///     Ok(())
+    /// }
+    pub fn access_with<'s, 'd: 's>(&'s self, dev: &'d Device<Bound>) -> Result<&'s T> {

Coming from Revocable::try_access_with (and the standard library in general), the name of this method made me think that it would take a closure to run while the resource is held. Maybe we should differenciate the names a bit more? Maybe just access is fine, or access_with_device?

author maintainer
#17 lore raw

Re: [PATCH 2/3] rust: devres: implement Devres::access_with()

Quoted 58 lines · Alexandre Courbot
Quoted 57 lines · Danilo Krummrich

Implement a direct accessor for the data stored within the Devres for cases where we can proof that we own a reference to a Device<Bound> (i.e. a bound device) of the same device that was used to create the corresponding Devres container.

Usually, when accessing the data stored within a Devres container, it is not clear whether the data has been revoked already due to the device being unbound and, hence, we have to try whether the access is possible and subsequently keep holding the RCU read lock for the duration of the access.

However, when we can proof that we hold a reference to Device<Bound> matching the device the Devres container has been created with, we can guarantee that the device is not unbound for the duration of the lifetime of the Device<Bound> reference and, hence, it is not possible for the data within the Devres container to be revoked.

Therefore, in this case, we can bypass the atomic check and the RCU read lock, which is a great optimization and simplification for drivers.

---
 rust/kernel/devres.rs | 35 +++++++++++++++++++++++++++++++++++
 1 file changed, 35 insertions(+)
rust/kernel/devres.rs +24 -0
index 1e58f5d22044..ec2cd9cdda8b 100644
@@ -181,6 +181,41 @@ pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result {
 
         Ok(())
     }
+
+    /// Obtain `&'a T`, bypassing the [`Revocable`].
+    ///
+    /// This method allows to directly obtain a `&'a T`, bypassing the [`Revocable`], by presenting
+    /// a `&'a Device<Bound>` of the same [`Device`] this [`Devres`] instance has been created with.
+    ///
+    /// An error is returned if `dev` does not match the same [`Device`] this [`Devres`] instance
+    /// has been created with.
+    ///
+    /// # Example
+    ///
+    /// ```no_run
+    /// # use kernel::{device::Core, devres::Devres, pci};
+    ///
+    /// fn from_core(dev: &pci::Device<Core>, devres: Devres<pci::Bar<0x4>>) -> Result<()> {
+    ///     let bar = devres.access_with(dev.as_ref())?;
+    ///
+    ///     let _ = bar.read32(0x0);
+    ///
+    ///     // might_sleep()
+    ///
+    ///     bar.write32(0x42, 0x0);
+    ///
+    ///     Ok(())
Quoted 8 lines

rust/kernel/devres.rs pub fn new_foreign_owned(dev: &Device<Bound>, data: T, flags: Flags) -> Result { · line 209

Quoted 2 lines
rust/kernel/devres.rs +2 -0
+    /// }
+    pub fn access_with<'s, 'd: 's>(&'s self, dev: &'d Device<Bound>) -> Result<&'s T> {

Coming from Revocable::try_access_with (and the standard library in general), the name of this method made me think that it would take a closure to run while the resource is held. Maybe we should differenciate the names a bit more? Maybe just access is fine, or access_with_device?

Seems reasonable -- access() it is then.

#23 lore raw

Re: [PATCH 3/3] samples: rust: pci: take advantage of Devres::access_with()

In reply to Danilo Krummrich
Quoted 33 lines · Danilo Krummrich
Quoted 32 lines · Benno Lossin
Quoted 31 lines · Danilo Krummrich
Quoted 29 lines · Benno Lossin
Quoted 14 lines · Danilo Krummrich

For the I/O operations executed from the probe() method, take advantage of Devres::access_with(), avoiding the atomic check and RCU read lock required otherwise entirely.

---
 samples/rust/rust_driver_pci.rs | 12 ++++++------
 1 file changed, 6 insertions(+), 6 deletions(-)
samples/rust/rust_driver_pci.rs +0 -0
index 9ce3a7323a16..3e1569e5096e 100644

samples/rust/rust_driver_pci.rs fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self> · line 86

Quoted 10 lines
samples/rust/rust_driver_pci.rs +1 -6
@@ -83,12 +83,12 @@ fn probe(pdev: &pci::Device<Core>, info: &Self::IdInfo) -> Result<Pin<KBox<Self>
             GFP_KERNEL,
         )?;
 
-        let res = drvdata
-            .bar
-            .try_access_with(|b| Self::testdev(info, b))
-            .ok_or(ENXIO)??;
-
-        dev_info!(pdev.as_ref(), "pci-testdev data-match count: {}\n", res);
+        let bar = drvdata.bar.access_with(pdev.as_ref())?;

Since this code might inspire other code, I don't think that we should return EINVAL here (bubbled up from access_with). Not sure what the correct thing here would be though...

Quoted 8 lines
Quoted 5 lines
Quoted 3 lines

I can't think of any other error code that would match better, EINVAL seems to be the correct thing. Maybe one could argue for ENODEV, but I still think EINVAL fits better.

The previous iteration of the sample used the ENXIO error code.

Yes, because the cause of error for try_access_with() failing would have been that the device was unbound already, hence a different error code.

Ah I see.

Quoted 19 lines
Quoted 3 lines

In this sample it should be impossible to trigger the error path. But others might copy the code into a context where that is not the case and then might have a horrible time debugging where the EINVAL came from.

I think it should always pretty unlikely design wise to supply a non-matching device.

Quoted 11 lines

I don't know if our answer to that should be "improve debugging errors in general" or "improve the error handling in this case". I have no idea how the former could look like, maybe something around #[track_caller] and noting the lines where an error was created? For the latter case, we could do:

    let bar = match drvdata.bar.access_with(pdev.as_ref()) {
        Ok(bar) => bar,
        Err(_) => {
            // `bar` was just created using the `pdev` above, so this should never happen.
            return Err(ENXIO);
Quoted 8 lines

If the caller really put in a non-matching device we can't say for sure that the cause is ENXIO, the only thing we know is that the code author confused device instances, so I think EINVAL is still the correct thing to return.

The problem that it might be difficult to figure out the source of the error code has always been present in the kernel, and while I'm not saying we shouldn't aim for improving this, I don't think this patch is quite the place for this.

Agreed.

---
Cheers,
Benno