Devres fixes for v6.15.4
Subsystems (4)
Danilo Krummrich
maintainercover Devres fixes for v6.15.4
As discussed in [1], here's the patch series for the Rust Devres fixes for v6.15.4.
[1] https://lore.kernel.org/stable/2025062455-rogue-flagship-54a4@gregkh/
Danilo Krummrich (4):
rust: completion: implement initial abstraction
rust: revocable: indicate whether data has been revoked already
rust: devres: fix race in Devres::drop()
rust: devres: do not dereference to the internal Revocable
rust/bindings/bindings_helper.h | 1 + rust/helpers/completion.c | 8 +++ rust/helpers/helpers.c | 1 + rust/kernel/devres.rs | 53 ++++++++++----- rust/kernel/revocable.rs | 18 +++-- rust/kernel/sync.rs | 2 + rust/kernel/sync/completion.rs | 112 ++++++++++++++++++++++++++++++++ 7 files changed, 175 insertions(+), 20 deletions(-) create mode 100644 rust/helpers/completion.c create mode 100644 rust/kernel/sync/completion.rs
base-commit: a2b47f77e740a21dbdcb12e2f2ca3c840299545a
Signature · 1 line
2.49.0
Danilo Krummrich
maintainerPATCH 1/4 rust: completion: implement initial abstraction
[ Upstream commit 1b56e765bf8990f1f60e124926c11fc4ac63d752 ]
Implement a minimal abstraction for the completion synchronization primitive.
This initial abstraction only adds complete_all() and wait_for_completion(), since that is what is required for the subsequent Devres patch.
--- rust/bindings/bindings_helper.h | 1 + rust/helpers/completion.c | 8 +++ rust/helpers/helpers.c | 1 + rust/kernel/sync.rs | 2 + rust/kernel/sync/completion.rs | 112 ++++++++++++++++++++++++++++++++ 5 files changed, 124 insertions(+) create mode 100644 rust/helpers/completion.c create mode 100644 rust/kernel/sync/completion.rs
rust/bindings/bindings_helper.h +1 -0
@@ -10,6 +10,7 @@#include <linux/blk-mq.h> #include <linux/blk_types.h> #include <linux/blkdev.h> +#include <linux/completion.h> #include <linux/cpumask.h> #include <linux/cred.h> #include <linux/device/faux.h>
rust/helpers/completion.c new file +8 -0
@@ -0,0 +1,8 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/completion.h> + +void rust_helper_init_completion (struct completion *x) +{ +init_completion (x); +}
rust/helpers/helpers.c +1 -0
@@ -11,6 +11,7 @@#include "bug.c" #include "build_assert.c" #include "build_bug.c" +#include "completion.c" #include "cpumask.c" #include "cred.c" #include "device.c"
rust/kernel/sync.rs +2 -0
@@ -10,6 +10,7 @@use pin_init;mod arc; +pub mod completion;mod condvar;pub mod lock;mod locked_by;
@@ -17,6 +18,7 @@pub mod rcu;pub use arc::{Arc ,ArcBorrow ,UniqueArc }; +pub use completion::Completion ;pub use condvar::{new_condvar,CondVar ,CondVarTimeoutResult };pub use lock::global::{global_lock,GlobalGuard ,GlobalLock ,GlobalLockBackend ,GlobalLockedBy };pub use lock::mutex::{new_mutex,Mutex ,MutexGuard };
rust/kernel/sync/completion.rs new file +112 -0
112 lines not on this page · raw message
Signature · 1 line
2.49.0
Sasha Levin
maintainerRe: [PATCH 1/4] rust: completion: implement initial abstraction
[ Sasha's backport helper bot ]
Hi,
✅ All tests passed successfully. No issues detected. No action required from the submitter.
The upstream commit SHA1 provided is correct: 1b56e765bf8990f1f60e124926c11fc4ac63d752
Note: The patch differs from the upstream commit:
---
1: 1b56e765bf899 ! 1: 9d44ce9765ef8 rust: completion: implement initial abstraction
@@ Metadata
## Commit message ##
rust: completion: implement initial abstraction
+ [ Upstream commit 1b56e765bf8990f1f60e124926c11fc4ac63d752 ]
+
Implement a minimal abstraction for the completion synchronization primitive.
@@ Commit message
## rust/bindings/bindings_helper.h ##
@@
+ #include <linux/blk-mq.h>
#include <linux/blk_types.h>
#include <linux/blkdev.h>
- #include <linux/clk.h>
+#include <linux/completion.h>
- #include <linux/configfs.h>
- #include <linux/cpu.h>
- #include <linux/cpufreq.h>
+ #include <linux/cpumask.h>
+ #include <linux/cred.h>
+ #include <linux/device/faux.h>
## rust/helpers/completion.c (new) ##
@@
@@ rust/helpers/completion.c (new)
## rust/helpers/helpers.c ##
@@
+ #include "bug.c"
#include "build_assert.c"
#include "build_bug.c"
- #include "clk.c"
+#include "completion.c"
- #include "cpufreq.c"
#include "cpumask.c"
#include "cred.c"
+ #include "device.c"
## rust/kernel/sync.rs ##
@@
---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-5.4.y | Success | Success |
Danilo Krummrich
maintainerPATCH 2/4 rust: revocable: indicate whether `data` has been revoked already
In reply to Danilo Krummrich[ Upstream commit 4b76fafb20dd4a2becb94949d78e86bc88006509 ]
Return a boolean from Revocable::revoke() and Revocable::revoke_nosync() to indicate whether the data has been revoked already.
Return true if the data hasn't been revoked yet (i.e. this call revoked the data), false otherwise.
This is required by Devres in order to synchronize the completion of the revoke process.
--- rust/kernel/revocable.rs | 18 ++++++++++++++---- 1 file changed, 14 insertions(+), 4 deletions(-)
rust/kernel/revocable.rs +14 -4
@@ -126,8 +126,10 @@ pub fn try_access_with_guard<'a>(&'a self, _guard: &'a rcu::Guard) -> Option<&'a/// # Safety /// /// Callers must ensure that there are no more concurrent users of the revocable object. -unsafe fn revoke_internal <const SYNC :bool >(&self) { -if self.is_available.swap (false ,Ordering ::Relaxed ) { +unsafe fn revoke_internal <const SYNC :bool >(&self) ->bool { +let revoke = self.is_available.swap (false ,Ordering ::Relaxed ); + +if revoke {if SYNC {// SAFETY: Just an FFI call, there are no further requirements. unsafe { bindings::synchronize_rcu () };
@@ -137,6 +139,8 @@ unsafe fn revoke_internal<const SYNC: bool>(&self) {// `compare_exchange` above that takes `is_available` from `true` to `false`. unsafe {drop_in_place (self.data.get ()) }; } + + revoke }/// Revokes access to and drops the wrapped object.
@@ -144,10 +148,13 @@ unsafe fn revoke_internal<const SYNC: bool>(&self) {/// Access to the object is revoked immediately to new callers of [`Revocable::try_access`], /// expecting that there are no concurrent users of the object. /// +/// Returns `true` if `&self` has been revoked with this call, `false` if it was revoked +/// already. +/// /// # Safety /// /// Callers must ensure that there are no more concurrent users of the revocable object. -pub unsafe fn revoke_nosync (&self) { +pub unsafe fn revoke_nosync (&self) ->bool {// SAFETY: By the safety requirement of this function, the caller ensures that nobody is // accessing the data anymore and hence we don't have to wait for the grace period to // finish.
@@ -161,7 +168,10 @@ pub unsafe fn revoke_nosync(&self) {/// If there are concurrent users of the object (i.e., ones that called /// [`Revocable::try_access`] beforehand and still haven't dropped the returned guard), this /// function waits for the concurrent access to complete before dropping the wrapped object. -pub fn revoke (&self) { +/// +/// Returns `true` if `&self` has been revoked with this call, `false` if it was revoked +/// already. +pub fn revoke (&self) ->bool {// SAFETY: By passing `true` we ask `revoke_internal` to wait for the grace period to // finish. unsafe { self.revoke_internal ::<true >() }
Signature · 1 line
2.49.0
Sasha Levin
maintainerRe: [PATCH 2/4] rust: revocable: indicate whether `data` has been revoked already
[ Sasha's backport helper bot ]
Hi,
✅ All tests passed successfully. No issues detected. No action required from the submitter.
The upstream commit SHA1 provided is correct: 4b76fafb20dd4a2becb94949d78e86bc88006509
Note: The patch differs from the upstream commit:
---
1: 4b76fafb20dd4 ! 1: b26ce8d6f1c62 rust: revocable: indicate whether data has been revoked already
@@ Metadata
## Commit message ##
rust: revocable: indicate whether data has been revoked already
+ [ Upstream commit 4b76fafb20dd4a2becb94949d78e86bc88006509 ]
+
Return a boolean from Revocable::revoke() and Revocable::revoke_nosync() to indicate whether the data has been revoked already.
@@ Commit message
## rust/kernel/revocable.rs ##
-@@ rust/kernel/revocable.rs: pub unsafe fn access(&self) -> &T {
+@@ rust/kernel/revocable.rs: pub fn try_access_with_guard<'a>(&'a self, _guard: &'a rcu::Guard) -> Option<&'a
/// # Safety
///
/// Callers must ensure that there are no more concurrent users of the revocable object.
---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-5.4.y | Success | Success |
Danilo Krummrich
maintainerPATCH 3/4 rust: devres: fix race in Devres::drop()
In reply to Danilo Krummrich[ Upstream commit f744201c6159fc7323c40936fd079525f7063598 ]
In Devres::drop() we first remove the devres action and then drop the wrapped device resource.
The design goal is to give the owner of a Devres object control over when the device resource is dropped, but limit the overall scope to the corresponding device being bound to a driver.
However, there's a race that was introduced with commit 8ff656643d30 ("rust: devres: remove action in Devres::drop"), but also has been (partially) present from the initial version on.
In Devres::drop(), the devres action is removed successfully and subsequently the destructor of the wrapped device resource runs. However, there is no guarantee that the destructor of the wrapped device resource completes before the driver core is done unbinding the corresponding device.
If in Devres::drop(), the devres action can't be removed, it means that the devres callback has been executed already, or is still running concurrently. In case of the latter, either Devres::drop() wins revoking the Revocable or the devres callback wins revoking the Revocable. If Devres::drop() wins, we (again) have no guarantee that the destructor of the wrapped device resource completes before the driver core is done unbinding the corresponding device.
CPU0 CPU1
------------------------------------------------------------------------Devres::drop() { Devres::devres_callback() {
self.data.revoke() { this.data.revoke() {
is_available.swap() == true
is_available.swap == false
}
}
// [...]
// device fully unbound
drop_in_place() {
// release device resource
} }
}
Depending on the specific device resource, this can potentially lead to user-after-free bugs.
In order to fix this, implement the following logic.
In the devres callback, we're always good when we get to revoke the device resource ourselves, i.e. Revocable::revoke() returns true.
If Revocable::revoke() returns false, it means that Devres::drop(), concurrently, already drops the device resource and we have to wait for Devres::drop() to signal that it finished dropping the device resource.
Note that if we hit the case where we need to wait for the completion of Devres::drop() in the devres callback, it means that we're actually racing with a concurrent Devres::drop() call, which already started revoking the device resource for us. This is rather unlikely and means that the concurrent Devres::drop() already started doing our work and we just need to wait for it to complete it for us. Hence, there should not be any additional overhead from that.
(Actually, for now it's even better if Devres::drop() does the work for us, since it can bypass the synchronize_rcu() call implied by Revocable::revoke(), but this goes away anyways once I get to implement the split devres callback approach, which allows us to first flip the atomics of all registered Devres objects of a certain device, execute a single synchronize_rcu() and then drop all revocable objects.)
In Devres::drop() we try to revoke the device resource. If that is *not* successful, it means that the devres callback already did and we're good.
Otherwise, we try to remove the devres action, which, if successful, means that we're good, since the device resource has just been revoked by us *before* we removed the devres action successfully.
If the devres action could not be removed, it means that the devres callback must be running concurrently, hence we signal that the device resource has been revoked by us, using the completion.
This makes it safe to drop a Devres object from any task and at any point of time, which is one of the design goals.
--- rust/kernel/devres.rs | 37 +++++++++++++++++++++++++++++-------- 1 file changed, 29 insertions(+), 8 deletions(-)
rust/kernel/devres.rs +29 -8
@@ -13,7 +13,7 @@ ffi::c_void, prelude::*, revocable::Revocable, - sync::Arc, + sync::{Arc, Completion}, types::ARef, };
@@ -25,13 +25,17 @@ struct DevresInner<T> { callback: unsafe extern "C" fn(*mut c_void),#[pin] data: Revocable<T>, +#[pin] + revoke: Completion, }/// This abstraction is meant to be used by subsystems to containerize [`Device`] bound resources to /// manage their lifetime. /// /// [`Device`] bound resources should be freed when either the resource goes out of scope or the -/// [`Device`] is unbound respectively, depending on what happens first. +/// [`Device`] is unbound respectively, depending on what happens first. In any case, it is always +/// guaranteed that revoking the device resource is completed before the corresponding [`Device`] +/// is unbound. /// /// To achieve that [`Devres`] registers a devres callback on creation, which is called once the /// [`Device`] is unbound, revoking access to the encapsulated resource (see also [`Revocable`]).
@@ -105,6 +109,7 @@ fn new(dev: &Device, data: T, flags: Flags) -> Result<Arc<DevresInner<T>>> { dev: dev.into(), callback: Self::devres_callback, data <- Revocable::new(data), + revoke <- Completion::new(), }), flags, )?;
@@ -133,26 +138,28 @@ fn as_ptr(&self) -> *const Self { selfas _ } -fn remove_action (this: &Arc <Self >) { +fn remove_action (this: &Arc <Self >) ->bool {// SAFETY: // - `self.inner.dev` is a valid `Device`, // - the `action` and `data` pointers are the exact same ones as given to devm_add_action() // previously, // - `self` is always valid, even if the action has been released already. -let ret =unsafe { +let success =unsafe { bindings::devm_remove_action_nowarn ( this.dev.as_raw (),Some (this.callback), this.as_ptr ()as _ , ) - }; + } ==0 ; -if ret ==0 { +if success {// SAFETY: We leaked an `Arc` reference to devm_add_action() in `DevresInner::new`; if // devm_remove_action_nowarn() was successful we can (and have to) claim back ownership // of this reference. let _ =unsafe {Arc ::from_raw (this.as_ptr ()) }; } + + success }#[allow(clippy::missing_safety_doc)]
@@ -164,7 +171,12 @@ fn remove_action(this: &Arc<Self>) {// `DevresInner::new`. let inner =unsafe {Arc ::from_raw (ptr) }; - inner.data.revoke (); +if !inner.data.revoke () { +// If `revoke()` returns false, it means that `Devres::drop` already started revoking +// `inner.data` for us. Hence we have to wait until `Devres::drop()` signals that it +// completed revoking `inner.data`. + inner.revoke.wait_for_completion (); + } } }
@@ -196,6 +208,15 @@ fn deref(&self) -> &Self::Target {impl <T >Drop for Devres <T > {fn drop (&mut self) { -DevresInner ::remove_action (&self.0 ); +// SAFETY: When `drop` runs, it is guaranteed that nobody is accessing the revocable data +// anymore, hence it is safe not to wait for the grace period to finish. +if unsafe { self.revoke_nosync () } { +// We revoked `self.0.data` before the devres action did, hence try to remove it. +if !DevresInner ::remove_action (&self.0 ) { +// We could not remove the devres action, which means that it now runs concurrently, +// hence signal that `self.0.data` has been revoked successfully. + self.0 .revoke.complete_all (); + } + } } }
Signature · 1 line
2.49.0
Sasha Levin
maintainerRe: [PATCH 3/4] rust: devres: fix race in Devres::drop()
[ Sasha's backport helper bot ]
Hi,
✅ All tests passed successfully. No issues detected. No action required from the submitter.
The upstream commit SHA1 provided is correct: f744201c6159fc7323c40936fd079525f7063598
Note: The patch differs from the upstream commit:
---
1: f744201c6159f ! 1: 971cc4bebeda2 rust: devres: fix race in Devres::drop()
@@ Metadata
## Commit message ##
rust: devres: fix race in Devres::drop()
+ [ Upstream commit f744201c6159fc7323c40936fd079525f7063598 ]
+
In Devres::drop() we first remove the devres action and then drop the
wrapped device resource.
@@ rust/kernel/devres.rs: struct DevresInner<T> {
///
/// To achieve that [`Devres`] registers a devres callback on creation, which is called once the
/// [`Device`] is unbound, revoking access to the encapsulated resource (see also [`Revocable`]).
-@@ rust/kernel/devres.rs: fn new(dev: &Device<Bound>, data: T, flags: Flags) -> Result<Arc<DevresInner<T>>
+@@ rust/kernel/devres.rs: fn new(dev: &Device, data: T, flags: Flags) -> Result<Arc<DevresInner<T>>> {
dev: dev.into(),
callback: Self::devres_callback,
data <- Revocable::new(data),---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-5.4.y | Success | Success |
Danilo Krummrich
maintainerPATCH 4/4 rust: devres: do not dereference to the internal Revocable
In reply to Danilo Krummrich[ Upstream commit 20c96ed278e362ae4e324ed7d8c69fb48c508d3c ]
We can't expose direct access to the internal Revocable, since this allows users to directly revoke the internal Revocable without Devres having the chance to synchronize with the devres callback -- we have to guarantee that the internal Revocable has been fully revoked before the device is fully unbound.
Hence, remove the corresponding Deref implementation and, instead, provide indirect accessors for the internal Revocable.
Note that we can still support Devres::revoke() by implementing the required synchronization (which would be almost identical to the synchronization in Devres::drop()).
--- rust/kernel/devres.rs | 20 ++++++++++---------- 1 file changed, 10 insertions(+), 10 deletions(-)
rust/kernel/devres.rs +10 -10
@@ -12,13 +12,11 @@ error::{Error, Result}, ffi::c_void, prelude::*, - revocable::Revocable, - sync::{Arc, Completion}, + revocable::{Revocable, RevocableGuard}, + sync::{rcu, Arc, Completion}, types::ARef, }; -use core::ops::Deref ; -#[pin_data] struct DevresInner <T > { dev:ARef <Device >,
@@ -196,13 +194,15 @@ pub fn new_foreign_owned(dev: &Device, data: T, flags: Flags) -> Result {Ok (()) } -} -impl <T >Deref for Devres <T > { -type Target =Revocable <T >; +/// [`Devres`] accessor for [`Revocable::try_access`]. +pub fn try_access (&self) ->Option <RevocableGuard <'_,T >> { + self.0 .data.try_access () + } -fn deref (&self) -> &Self ::Target { - &self.0 .data +/// [`Devres`] accessor for [`Revocable::try_access_with_guard`]. +pub fn try_access_with_guard <'a>(&'a self, guard: &'a rcu::Guard ) ->Option <&'aT > { + self.0 .data.try_access_with_guard (guard) } }
@@ -210,7 +210,7 @@ impl<T> Drop for Devres<T> {fn drop (&mut self) {// SAFETY: When `drop` runs, it is guaranteed that nobody is accessing the revocable data // anymore, hence it is safe not to wait for the grace period to finish. -if unsafe { self.revoke_nosync () } { +if unsafe { self.0 .data.revoke_nosync () } {// We revoked `self.0.data` before the devres action did, hence try to remove it. if !DevresInner ::remove_action (&self.0 ) {// We could not remove the devres action, which means that it now runs concurrently,
Signature · 1 line
2.49.0
Sasha Levin
maintainerRe: [PATCH 4/4] rust: devres: do not dereference to the internal Revocable
[ Sasha's backport helper bot ]
Hi,
✅ All tests passed successfully. No issues detected. No action required from the submitter.
The upstream commit SHA1 provided is correct: 20c96ed278e362ae4e324ed7d8c69fb48c508d3c
Note: The patch differs from the upstream commit:
---
1: 20c96ed278e36 ! 1: c56dfa9d4bad7 rust: devres: do not dereference to the internal Revocable
@@ Metadata
## Commit message ##
rust: devres: do not dereference to the internal Revocable
+ [ Upstream commit 20c96ed278e362ae4e324ed7d8c69fb48c508d3c ]
+
We can't expose direct access to the internal Revocable, since this
allows users to directly revoke the internal Revocable without Devres
having the chance to synchronize with the devres callback -- we have to
@@ rust/kernel/devres.rs
#[pin_data]
struct DevresInner<T> {
dev: ARef<Device>,
-@@ rust/kernel/devres.rs: pub fn access<'a>(&'a self, dev: &'a Device<Bound>) -> Result<&'a T> {
- // SAFETY: dev being the same device as the device this Devres has been created for
- // proves that self.0.data hasn't been revoked and is guaranteed to not be revoked as
- // long as dev lives; dev lives at least as long as self.
-- Ok(unsafe { self.deref().access() })
-+ Ok(unsafe { self.0.data.access() })
+@@ rust/kernel/devres.rs: pub fn new_foreign_owned(dev: &Device, data: T, flags: Flags) -> Result { +
+ Ok(())
}
-}
@@ rust/kernel/devres.rs: pub fn access<'a>(&'a self, dev: &'a Device<Bound>) -> Re
+ pub fn try_access(&self) -> Option<RevocableGuard<'_, T>> {
+ self.0.data.try_access()
+ }
-+
-+ /// [Devres] accessor for [Revocable::try_access_with].
-+ pub fn try_access_with<R, F: FnOnce(&T) -> R>(&self, f: F) -> Option<R> {
-+ self.0.data.try_access_with(f)
-+ }
- fn deref(&self) -> &Self::Target {
- &self.0.data
---
Results of testing on various branches:
| Branch | Patch Apply | Build Test | |---------------------------|-------------|------------| | stable/linux-5.4.y | Success | Success |