Rust abstractions for Device & Firmware
Subsystems (3)
v2 cover Rust abstractions for Device & Firmware
Hi,
as agreed in [1] this is the separate series for the device and firmware abstractions to unblock the inclusion of Fujita's PHY driver.
Originally, those patches were part of the patch series [2][3].
Changes in v2
=============
- use the 'srctree/' notation
- expand the existing documentation and make it more unambiguous
- use NonNull in Firmware
- generalize the Firmware request functions
- add missing safety comments to Firmware
[1] https://lore.kernel.org/rust-for-linux/2024060745-palatable-dragging-32d1@gregkh/
[2] https://lore.kernel.org/rust-for-linux/[email protected]/
[3] https://lore.kernel.org/rust-for-linux/[email protected]/
Danilo Krummrich (2):
rust: add abstraction for struct device
rust: add firmware abstractions
rust/bindings/bindings_helper.h | 1 + rust/helpers.c | 1 + rust/kernel/device.rs | 97 +++++++++++++++++++++++++++++ rust/kernel/firmware.rs | 107 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 2 + 5 files changed, 208 insertions(+) create mode 100644 rust/kernel/device.rs create mode 100644 rust/kernel/firmware.rs
base-commit: 1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0
Signature · 1 line
2.45.1
PATCH v2 1/2 rust: add abstraction for struct device
Add an (always) reference-counted abstraction for a generic C struct device. This abstraction encapsulates existing struct device instances and manages its reference count.
Subsystems may use this abstraction as a base to abstract subsystem specific device instances based on a generic struct device, such as struct pci_dev.
--- rust/helpers.c | 1 + rust/kernel/device.rs | 97 +++++++++++++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + 3 files changed, 99 insertions(+) create mode 100644 rust/kernel/device.rs
rust/helpers.c +1 -0
@@ -23,6 +23,7 @@#include <kunit/test-bug.h> #include <linux/bug.h> #include <linux/build_bug.h> +#include <linux/device.h> #include <linux/err.h> #include <linux/errname.h> #include <linux/mutex.h>
rust/kernel/device.rs new file +97 -0
97 lines not on this page · raw message
rust/kernel/lib.rs +1 -0
@@ -28,6 +28,7 @@pub mod alloc;mod build_assert; +pub mod device;pub mod error;pub mod init;pub mod ioctl;
Signature · 1 line
2.45.1
Re: [PATCH v2 1/2] rust: add abstraction for struct device
On Mon, Jun 10, 2024 at 08:02:27PM +0200, Danilo Krummrich wrote:
[...]
Quoted 25 lines
rust/kernel/device.rs new file +25
-0+/// A reference-counted device. +/// +/// This structure represents the Rust abstraction for a C `struct device`. This implementation +/// abstracts the usage of an already existing C `struct device` within Rust code that we get +/// passed from the C side. +/// +/// An instance of this abstraction can be obtained temporarily or permanent. +/// +/// A temporary one is bound to the lifetime of the C `struct device` pointer used for creation. +/// A permanent instance is always reference-counted and hence not restricted by any lifetime +/// boundaries. +/// +/// For subsystems it is recommended to create a permanent instance to wrap into a subsystem +/// specifc device structure (e.g. `pci::Device`). This is useful for passing it to drivers in +/// `T::probe()`, such that a driver can store the `ARef<Device>` (equivalent to storing a +/// `struct device` pointer in a C driver) for arbitrary purposes, e.g. allocating DMA coherent +/// memory. +/// +/// # Invariants +/// +/// The pointer stored in `Self` is non-null and valid for the lifetime of the `ARef` instance. In +/// particular, the `ARef` instance owns an increment on the underlying object’s reference count. +#[repr(transparent)] +pub struct Device (Opaque <bindings::device >); +
[...]
rust/kernel/device.rs line 97
Quoted 8 lines
rust/kernel/device.rs new file +8
-0+ +// SAFETY: `Device` only holds a pointer to a C `struct device`, which is safe to be used from any +// thread. +unsafe impl Send for Device {} + +// SAFETY: `Device` only holds a pointer to a C `struct device`, references to which are safe to be +// used from any thread. +unsafe impl Sync for Device {}
These comments need some rework, Device is not a pointer to struct device anymore. For the Sync one, how about:
// SAFETY: Device can be shared among threads because all immutable // methods are protected by the synchronization in struct device. unsafe impl Sync for Device {}
and for Send, I actually don't think we can easily say the generic Device is Send: you can create a struct device where ->release requires to be run on the same thread that creates the device, and nothing is wrong about it, I think (e.g. making a thread be the sole owner of some special devices). Unless, in the #Invariants of Device, and the #safety of from_ptr, you mention that Device assume its ->release can be called on any thread.
Regards,
Boqun
Quoted 15 lines
rust/kernel/lib.rs +1
-0@@ -28,6 +28,7 @@pub mod alloc;mod build_assert; +pub mod device;pub mod error;pub mod init;pub mod ioctl;Signature · 1 line
2.45.1
Re: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 28 lines · Boqun Feng
On Mon, Jun 10, 2024 at 08:02:27PM +0200, Danilo Krummrich wrote:
[...]Quoted 25 lines
rust/kernel/device.rs new file +25
-0+/// A reference-counted device. +/// +/// This structure represents the Rust abstraction for a C `struct device`. This implementation +/// abstracts the usage of an already existing C `struct device` within Rust code that we get +/// passed from the C side. +/// +/// An instance of this abstraction can be obtained temporarily or permanent. +/// +/// A temporary one is bound to the lifetime of the C `struct device` pointer used for creation. +/// A permanent instance is always reference-counted and hence not restricted by any lifetime +/// boundaries. +/// +/// For subsystems it is recommended to create a permanent instance to wrap into a subsystem +/// specifc device structure (e.g. `pci::Device`). This is useful for passing it to drivers in +/// `T::probe()`, such that a driver can store the `ARef<Device>` (equivalent to storing a +/// `struct device` pointer in a C driver) for arbitrary purposes, e.g. allocating DMA coherent +/// memory. +/// +/// # Invariants +/// +/// The pointer stored in `Self` is non-null and valid for the lifetime of the `ARef` instance. In +/// particular, the `ARef` instance owns an increment on the underlying object’s reference count. +#[repr(transparent)] +pub struct Device (Opaque <bindings::device >); +[...]
Quoted 11 lines
rust/kernel/device.rs line 97
Quoted 8 lines
rust/kernel/device.rs new file +8
-0+ +// SAFETY: `Device` only holds a pointer to a C `struct device`, which is safe to be used from any +// thread. +unsafe impl Send for Device {} + +// SAFETY: `Device` only holds a pointer to a C `struct device`, references to which are safe to be +// used from any thread. +unsafe impl Sync for Device {}These comments need some rework,
Deviceis not a pointer tostruct deviceanymore. For theSyncone, how about:
Indeed, I forgot to update them.
Quoted 4 lines
// SAFETY:
Devicecan be shared among threads because all immutable // methods are protected by the synchronization instruct device. unsafe impl Sync for Device {}
Sounds good.
Quoted 5 lines
and for
Send, I actually don't think we can easily say the genericDeviceisSend: you can create astruct devicewhere->releaserequires to be run on the same thread that creates thedevice, and nothing is wrong about it, I think (e.g. making a thread be the sole
Hm, I guess in this case we actually can't argue that it's the owners fault to pass a pointer of this device somewhere else. Since it's C the owner can't enforce that someone else is not taking a reference and prevent sharing ownership...
Quoted 3 lines
owner of some special devices). Unless, in the #Invariants of
Device, and the #safety offrom_ptr, you mention thatDeviceassume its->releasecan be called on any thread.
...hence, I agree we should indeed add to the #Invariants and #Safety section that ->release must be callable from any thread.
However, this is just theory, do we actually have cases where device::release is not allowed to be called from any thread? If so, this would be very confusing for a reference counted type from a design point of view...
- Danilo
Quoted 20 lines
Regards,
BoqunQuoted 15 lines
rust/kernel/lib.rs +1
-0@@ -28,6 +28,7 @@pub mod alloc;mod build_assert; +pub mod device;pub mod error;pub mod init;pub mod ioctl;Signature · 1 line
2.45.1
Greg KH
maintainerRe: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 6 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::releaseis not allowed to be called from any thread? If so, this would be very confusing for a reference counted type from a design point of view...
What do you mean exactly "by any thread"? Maybe not from interrupt context, but any other normal thread (i.e. that you can sleep in), it should be fine to call release() in.
thanks,
greg k-h
Re: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release
@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Quoted 4 lines
Quoted 2 lines
is not allowed to be called from any thread? If so, this would be very confusing for a reference counted type from a design point of view...
What do you mean exactly "by any thread"? Maybe not from interrupt
The Send trait here doesn't really differ between interrupt contexts and process contexts, so "by any thread", it includes all the contexts. However, we rely on klint[1] to detect context mismatch in compile time (it's still a WIP though). For this case, we would need to mark the Device::dec_ref function as might sleep.
Regards,
Boqun
[1]: https://rust-for-linux.com/klint
Quoted 6 lines
context, but any other normal thread (i.e. that you can sleep in), it should be fine to call
release()in.thanks,
greg k-h
Re: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 13 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Quoted 3 lines
Quoted 2 lines
is not allowed to be called from any thread? If so, this would be very confusing for a reference counted type from a design point of view...
Quoted 7 lines
Quoted 1 line
What do you mean exactly "by any thread"? Maybe not from interrupt
The
Sendtrait here doesn't really differ between interrupt contexts and process contexts, so "by any thread", it includes all the contexts. However, we rely on klint[1] to detect context mismatch in compile time (it's still a WIP though). For this case, we would need to mark theDevice::dec_reffunction as might sleep.
I think once we can detect it on compile time we should do that.
As for the might_sleep(), if we want to have this check, we should probably do it in device_release() [1], rather than in the Rust code only.
[1] https://elixir.bootlin.com/linux/latest/source/drivers/base/core.c#L2557
Quoted 13 lines
Regards,
Boqun[1]: https://rust-for-linux.com/klint
Quoted 6 lines
context, but any other normal thread (i.e. that you can sleep in), it should be fine to call
release()in.thanks,
greg k-h
Re: [PATCH v2 1/2] rust: add abstraction for struct device
In reply to Boqun FengQuoted 9 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Similar to my previous reply, if we want to call this out as safety requirement in Device::from_raw, we probably want to add it to the documentation of the C struct device, such that we can argue that this is an invariant of C's struct device.
Otherwise we'd have to write something like:
"It must also be ensured that the ->release function of a struct device can be called from any non-atomic context. While not being officially documented this is guaranteed by the invariant of struct device."
Quoted 24 lines
Quoted 5 lines
Quoted 3 lines
is not allowed to be called from any thread? If so, this would be very confusing for a reference counted type from a design point of view...
What do you mean exactly "by any thread"? Maybe not from interrupt
The
Sendtrait here doesn't really differ between interrupt contexts and process contexts, so "by any thread", it includes all the contexts. However, we rely on klint[1] to detect context mismatch in compile time (it's still a WIP though). For this case, we would need to mark theDevice::dec_reffunction as might sleep.Regards,
Boqun[1]: https://rust-for-linux.com/klint
Quoted 6 lines
context, but any other normal thread (i.e. that you can sleep in), it should be fine to call
release()in.thanks,
greg k-h
Greg KH
maintainerRe: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 16 lines · Danilo Krummrich
Quoted 9 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Similar to my previous reply, if we want to call this out as safety requirement in
Device::from_raw, we probably want to add it to the documentation of the Cstruct device, such that we can argue that this is an invariant of C'sstruct device.
Quoted 5 lines
Otherwise we'd have to write something like:
"It must also be ensured that the
->releasefunction of astruct devicecan be called from any non-atomic context. While not being officially documented this is guaranteed by the invariant ofstruct device."
In the 20+ years of the driver model being part of the kernel, I don't think this has come up yet, so maybe you can call the release function in irq context. I don't know, I was just guessing :)
So let's not go adding constraints that we just do not have please. Same goes for the C code, so the rust code is no different here.
thanks,
greg k-h
Re: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 19 lines · Greg KH
Quoted 18 lines · Danilo Krummrich
Quoted 9 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Similar to my previous reply, if we want to call this out as safety requirement in
Device::from_raw, we probably want to add it to the documentation of the Cstruct device, such that we can argue that this is an invariant of C'sstruct device.Otherwise we'd have to write something like:
Quoted 7 lines
Quoted 3 lines
"It must also be ensured that the
->releasefunction of astruct devicecan be called from any non-atomic context. While not being officially documented this is guaranteed by the invariant ofstruct device."In the 20+ years of the driver model being part of the kernel, I don't think this has come up yet, so maybe you can call the release function in irq context. I don't know, I was just guessing :)
Ah, I see. I thought you know and it's defined, but just not documented.
This means it's simply undefined what we expect to happen when the last reference of a device is dropped from atomic context.
Now, I understand (and would even expect) that practically this has never been an issue. You'd need two circumstances, release() actually does something that is not allowed in atomic context plus the last device reference is dropped from atomic context - rather unlikely.
Quoted 3 lines
So let's not go adding constraints that we just do not have please. Same goes for the C code, so the rust code is no different here.
I agree we shouldn't add random constraints, but for writing safe code we also have to rely on defined behavior.
I see two options:
(1) We globally (for struct device) define from which context release() is allowed to be called.
(2) We define it for the Rust abstraction only and just constrain it to non-atomic context to be able to give a safety guarantee. We can't say "might be safe from any context, but we don't know".
But again, this is really just a formality, the C code does it all the way and practically there never was an issue, which means we actually do follow some rules, it's just about writing them down. :)
- Danilo
Quoted 5 lines
thanks,
greg k-h
Greg KH
maintainerRe: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 39 lines · Danilo Krummrich
Quoted 19 lines · Greg KH
Quoted 18 lines · Danilo Krummrich
Quoted 9 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Similar to my previous reply, if we want to call this out as safety requirement in
Device::from_raw, we probably want to add it to the documentation of the Cstruct device, such that we can argue that this is an invariant of C'sstruct device.Otherwise we'd have to write something like:
Quoted 7 lines
Quoted 3 lines
"It must also be ensured that the
->releasefunction of astruct devicecan be called from any non-atomic context. While not being officially documented this is guaranteed by the invariant ofstruct device."In the 20+ years of the driver model being part of the kernel, I don't think this has come up yet, so maybe you can call the release function in irq context. I don't know, I was just guessing :)
Ah, I see. I thought you know and it's defined, but just not documented.
This means it's simply undefined what we expect to happen when the last reference of a device is dropped from atomic context.
Now, I understand (and would even expect) that practically this has never been an issue. You'd need two circumstances,
release()actually does something that is not allowed in atomic context plus the last device reference is dropped from atomic context - rather unlikely.Quoted 1 line
Quoted 5 lines
Quoted 2 lines
So let's not go adding constraints that we just do not have please. Same goes for the C code, so the rust code is no different here.
I agree we shouldn't add random constraints, but for writing safe code we also have to rely on defined behavior.
As the rust code is relying on C code that could change at any point in time, how can that ever be "safe"? :)
Sorry, this type of definition annoys me.
Quoted 4 lines
I see two options:
(1) We globally (for struct device) define from which context
release()is allowed to be called.
If you want, feel free to do that work please. And then find out how to enforce it in the driver core.
Quoted 3 lines
(2) We define it for the Rust abstraction only and just constrain it to non-atomic context to be able to give a safety guarantee. We can't say "might be safe from any context, but we don't know".
Why can't you say that? Your "saftey" barrier ends/begins at the interaction with the rust/c layer. You can't "guarantee" anything on the C side...
Quoted 3 lines
But again, this is really just a formality, the C code does it all the way and practically there never was an issue, which means we actually do follow some rules, it's just about writing them down. :)
Again, this has NEVER come up in 20+ years, so maybe it's just not an issue? Not to say it isn't, but maybe do some tests and see what happens...
thanks,
greg k-h
Re: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 40 lines · Greg KH
Quoted 39 lines · Danilo Krummrich
Quoted 19 lines · Greg KH
Quoted 18 lines · Danilo Krummrich
Quoted 9 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Similar to my previous reply, if we want to call this out as safety requirement in
Device::from_raw, we probably want to add it to the documentation of the Cstruct device, such that we can argue that this is an invariant of C'sstruct device.Otherwise we'd have to write something like:
Quoted 7 lines
Quoted 3 lines
"It must also be ensured that the
->releasefunction of astruct devicecan be called from any non-atomic context. While not being officially documented this is guaranteed by the invariant ofstruct device."In the 20+ years of the driver model being part of the kernel, I don't think this has come up yet, so maybe you can call the release function in irq context. I don't know, I was just guessing :)
Ah, I see. I thought you know and it's defined, but just not documented.
This means it's simply undefined what we expect to happen when the last reference of a device is dropped from atomic context.
Now, I understand (and would even expect) that practically this has never been an issue. You'd need two circumstances,
release()actually does something that is not allowed in atomic context plus the last device reference is dropped from atomic context - rather unlikely.Quoted 1 line
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
So let's not go adding constraints that we just do not have please. Same goes for the C code, so the rust code is no different here.
I agree we shouldn't add random constraints, but for writing safe code we also have to rely on defined behavior.
As the rust code is relying on C code that could change at any point in time, how can that ever be "safe"? :)
That's the same as with any other API. If the logic of an API is changed the users (e.g a Rust abstraction) of the API have to be adjusted.
Quoted 10 lines
Sorry, this type of definition annoys me.
Quoted 4 lines
I see two options:
(1) We globally (for struct device) define from which context
release()is allowed to be called.If you want, feel free to do that work please. And then find out how to enforce it in the driver core.
If we *would* define non-atomic context only, we could enforce it with might_sleep() for instance.
If we *would* define any context, there is nothing to enforce, but we'd need to validate that no implementer of release() voids that.
The former is a constaint you don't want to add, the latter a lot of work. What if we at least define that implementers of release() must *minimally* make sure that it can be call from any non-atomic context.
That'd be something we can rely on in Rust.
Quoted 8 lines
Quoted 4 lines
(2) We define it for the Rust abstraction only and just constrain it to non-atomic context to be able to give a safety guarantee. We can't say "might be safe from any context, but we don't know".
Why can't you say that? Your "saftey" barrier ends/begins at the interaction with the
rust/clayer. You can't "guarantee" anything on the C side...
Not guarantee as in "technically enforce it", but guarantee as in "we have rules that we follow".
The former would be best, but it isn't always possible. The latter we can always do (and should IMHO).
Quoted 8 lines
Quoted 4 lines
But again, this is really just a formality, the C code does it all the way and practically there never was an issue, which means we actually do follow some rules, it's just about writing them down. :)
Again, this has NEVER come up in 20+ years, so maybe it's just not an issue? Not to say it isn't, but maybe do some tests and see what happens...
Oh, I fully agree with that. Let me try to explain a bit what this is about:
In Rust we have the Send and Sync marker traits. If a type (e.g. Device) implements Send it means that it's safe to pass an instance of this type between threads. Which is clearly something we want to do with a Device.
If I don't implement Sync for Device the compiler will prevent me from sending it between threads, e.g. by disallowing me to put an instance of Device into another data structure that is potentially passed between threads.
If I implement Sync I have to add a safety comment on why it is safe to pass Device between threads. And here we have what Boqun pointed out: Device can only be passed between threads when we're allowed to drop the last reference from any thread. In the case of the kernel this can be any non-atomic context, any context or any other subset. But I have to write something here that is a defined rule and can be relied on.
- Danilo
Quoted 5 lines
thanks,
greg k-h
Greg KH
maintainerRe: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 44 lines · Danilo Krummrich
Quoted 43 lines · Greg KH
Quoted 42 lines · Danilo Krummrich
Quoted 19 lines · Greg KH
Quoted 18 lines · Danilo Krummrich
Quoted 9 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Similar to my previous reply, if we want to call this out as safety requirement in
Device::from_raw, we probably want to add it to the documentation of the Cstruct device, such that we can argue that this is an invariant of C'sstruct device.Otherwise we'd have to write something like:
Quoted 7 lines
Quoted 3 lines
"It must also be ensured that the
->releasefunction of astruct devicecan be called from any non-atomic context. While not being officially documented this is guaranteed by the invariant ofstruct device."In the 20+ years of the driver model being part of the kernel, I don't think this has come up yet, so maybe you can call the release function in irq context. I don't know, I was just guessing :)
Ah, I see. I thought you know and it's defined, but just not documented.
This means it's simply undefined what we expect to happen when the last reference of a device is dropped from atomic context.
Now, I understand (and would even expect) that practically this has never been an issue. You'd need two circumstances,
release()actually does something that is not allowed in atomic context plus the last device reference is dropped from atomic context - rather unlikely.Quoted 3 lines
So let's not go adding constraints that we just do not have please. Same goes for the C code, so the rust code is no different here.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
I agree we shouldn't add random constraints, but for writing safe code we also have to rely on defined behavior.
As the rust code is relying on C code that could change at any point in time, how can that ever be "safe"? :)
That's the same as with any other API. If the logic of an API is changed the users (e.g a Rust abstraction) of the API have to be adjusted.
Agreed, just like any other in-kernel code, so there shouldn't be anything special here.
Quoted 4 lines
Quoted 4 lines
Sorry, this type of definition annoys me.
Quoted 2 lines
I see two options:
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
(1) We globally (for struct device) define from which context
release()is allowed to be called.If you want, feel free to do that work please. And then find out how to enforce it in the driver core.
If we *would* define non-atomic context only, we could enforce it with
might_sleep()for instance.
might_sleep() isn't always correct from what I remember.
Quoted 2 lines
If we *would* define any context, there is nothing to enforce, but we'd need to validate that no implementer of
release()voids that.
Trying to validate that might be hard, again, I don't think it's worth it.
Quoted 5 lines
The former is a constaint you don't want to add, the latter a lot of work. What if we at least define that implementers of
release()must *minimally* make sure that it can be call from any non-atomic context.That'd be something we can rely on in Rust.
Determining if you are, or are not, in atomic context is almost impossible in C, I don't know how you are going to do that at build time in Rust. Good luck!
Quoted 10 lines
Oh, I fully agree with that. Let me try to explain a bit what this is about:
In Rust we have the
SendandSyncmarker traits. If a type (e.g.Device) implementsSendit means that it's safe to pass an instance of this type between threads. Which is clearly something we want to do with aDevice.If I don't implement
SyncforDevicethe compiler will prevent me from sending it between threads, e.g. by disallowing me to put an instance ofDeviceinto another data structure that is potentially passed between threads.
Quoted 6 lines
If I implement
SyncI have to add a safety comment on why it is safe to passDevicebetween threads. And here we have what Boqun pointed out:Devicecan only be passed between threads when we're allowed to drop the last reference from any thread. In the case of the kernel this can be any non-atomic context, any context or any other subset. But I have to write something here that is a defined rule and can be relied on.
You really have two things here, a matrix of:
can transfer between threads
can call in irq context
that are independent and not related to each other at all.
Looks like Rust has built in support for the first. And nothing for the second as that is a very kernel-specific thing.
So let's not confuse the two please. Send and Sync should be fine for a device pointer to be passed around, as long as the reference is incremented, as that's what all of the kernel C code does today. Let's not worry about irq context at all, that's independent and can be handled at a later time, if at all, with a different "marking" as it's independent of the current two things.
thanks,
greg k-h
Greg KH
maintainerRe: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 82 lines · Greg KH
Quoted 44 lines · Danilo Krummrich
Quoted 43 lines · Greg KH
Quoted 42 lines · Danilo Krummrich
Quoted 19 lines · Greg KH
Quoted 18 lines · Danilo Krummrich
Quoted 9 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Similar to my previous reply, if we want to call this out as safety requirement in
Device::from_raw, we probably want to add it to the documentation of the Cstruct device, such that we can argue that this is an invariant of C'sstruct device.Otherwise we'd have to write something like:
Quoted 7 lines
Quoted 3 lines
"It must also be ensured that the
->releasefunction of astruct devicecan be called from any non-atomic context. While not being officially documented this is guaranteed by the invariant ofstruct device."In the 20+ years of the driver model being part of the kernel, I don't think this has come up yet, so maybe you can call the release function in irq context. I don't know, I was just guessing :)
Ah, I see. I thought you know and it's defined, but just not documented.
This means it's simply undefined what we expect to happen when the last reference of a device is dropped from atomic context.
Now, I understand (and would even expect) that practically this has never been an issue. You'd need two circumstances,
release()actually does something that is not allowed in atomic context plus the last device reference is dropped from atomic context - rather unlikely.Quoted 3 lines
So let's not go adding constraints that we just do not have please. Same goes for the C code, so the rust code is no different here.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
I agree we shouldn't add random constraints, but for writing safe code we also have to rely on defined behavior.
As the rust code is relying on C code that could change at any point in time, how can that ever be "safe"? :)
That's the same as with any other API. If the logic of an API is changed the users (e.g a Rust abstraction) of the API have to be adjusted.
Agreed, just like any other in-kernel code, so there shouldn't be anything special here.
Quoted 4 lines
Quoted 4 lines
Sorry, this type of definition annoys me.
Quoted 2 lines
I see two options:
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
(1) We globally (for struct device) define from which context
release()is allowed to be called.If you want, feel free to do that work please. And then find out how to enforce it in the driver core.
If we *would* define non-atomic context only, we could enforce it with
might_sleep()for instance.
might_sleep()isn't always correct from what I remember.Quoted 2 lines
If we *would* define any context, there is nothing to enforce, but we'd need to validate that no implementer of
release()voids that.Trying to validate that might be hard, again, I don't think it's worth it.
Quoted 4 lines
The former is a constaint you don't want to add, the latter a lot of work. What if we at least define that implementers of
release()must *minimally* make sure that it can be call from any non-atomic context.
Quoted 5 lines
Quoted 1 line
That'd be something we can rely on in Rust.
Determining if you are, or are not, in atomic context is almost impossible in C, I don't know how you are going to do that at build time in Rust. Good luck!
Note, the real problem with calling release() on a device, is when you do so from within the device's sysfs file. That causes all sorts of "fun" times that people have had to work around over the years. It's not something you probably need to worry about yet, as you are not writing a bus subsystem in Rust yet, but it is something that can get tricky very quickly. Look at the hoops that scsi has to deal with if you want to see the gory details...
But this is all just "logic" bugs, not thread or irq issues at all. Nothing is different, nor should be different, for code doing this in C or in rust.
What I'm trying to get at is that you can't document all of the "constraints" you are dealing with when handling C apis like this, sorry. It's going to be hard, if not impossible, to worry about that at this point in the api. It's better left to be documented where the real call happens (i.e. in the C code), than attempting to keep it up to date in the c<->rust binding point.
thanks,
greg k-h
Re: [PATCH v2 1/2] rust: add abstraction for struct device
In reply to Greg KHQuoted 82 lines · Greg KH
Quoted 44 lines · Danilo Krummrich
Quoted 43 lines · Greg KH
Quoted 42 lines · Danilo Krummrich
Quoted 19 lines · Greg KH
Quoted 18 lines · Danilo Krummrich
Quoted 9 lines · Boqun Feng
Quoted 5 lines · Greg KH
Quoted 4 lines · Danilo Krummrich
...hence, I agree we should indeed add to the #Invariants and #Safety section that
->releasemust be callable from any thread.However, this is just theory, do we actually have cases where
device::release@Danilo, right, it's only theorical, but it's good to call it out since it's the requirement for a safe Rust abstraction.
Similar to my previous reply, if we want to call this out as safety requirement in
Device::from_raw, we probably want to add it to the documentation of the Cstruct device, such that we can argue that this is an invariant of C'sstruct device.Otherwise we'd have to write something like:
Quoted 7 lines
Quoted 3 lines
"It must also be ensured that the
->releasefunction of astruct devicecan be called from any non-atomic context. While not being officially documented this is guaranteed by the invariant ofstruct device."In the 20+ years of the driver model being part of the kernel, I don't think this has come up yet, so maybe you can call the release function in irq context. I don't know, I was just guessing :)
Ah, I see. I thought you know and it's defined, but just not documented.
This means it's simply undefined what we expect to happen when the last reference of a device is dropped from atomic context.
Now, I understand (and would even expect) that practically this has never been an issue. You'd need two circumstances,
release()actually does something that is not allowed in atomic context plus the last device reference is dropped from atomic context - rather unlikely.Quoted 3 lines
So let's not go adding constraints that we just do not have please. Same goes for the C code, so the rust code is no different here.
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
I agree we shouldn't add random constraints, but for writing safe code we also have to rely on defined behavior.
As the rust code is relying on C code that could change at any point in time, how can that ever be "safe"? :)
That's the same as with any other API. If the logic of an API is changed the users (e.g a Rust abstraction) of the API have to be adjusted.
Agreed, just like any other in-kernel code, so there shouldn't be anything special here.
Quoted 4 lines
Quoted 4 lines
Sorry, this type of definition annoys me.
Quoted 2 lines
I see two options:
Quoted 8 lines
Quoted 5 lines
Quoted 2 lines
(1) We globally (for struct device) define from which context
release()is allowed to be called.If you want, feel free to do that work please. And then find out how to enforce it in the driver core.
If we *would* define non-atomic context only, we could enforce it with
might_sleep()for instance.
might_sleep()isn't always correct from what I remember.Quoted 2 lines
If we *would* define any context, there is nothing to enforce, but we'd need to validate that no implementer of
release()voids that.Trying to validate that might be hard, again, I don't think it's worth it.
Quoted 4 lines
The former is a constaint you don't want to add, the latter a lot of work. What if we at least define that implementers of
release()must *minimally* make sure that it can be call from any non-atomic context.
Quoted 5 lines
Quoted 1 line
That'd be something we can rely on in Rust.
Determining if you are, or are not, in atomic context is almost impossible in C, I don't know how you are going to do that at build time in Rust. Good luck!
We can't always enforce things, but we can still follow rules even if we can't enforce them with tools or language features.
In particular, nothing prevents C code to define from which context things are allowed to be called. When I was working on drm_gpuvm [1] this was one of the major aspects I had to get right and documented; keyword dma_fence signalling critical sections.
[1] https://elixir.bootlin.com/linux/latest/source/drivers/gpu/drm/drm_gpuvm.c
Quoted 18 lines
Quoted 11 lines
Oh, I fully agree with that. Let me try to explain a bit what this is about:
In Rust we have the
SendandSyncmarker traits. If a type (e.g.Device) implementsSendit means that it's safe to pass an instance of this type between threads. Which is clearly something we want to do with aDevice.If I don't implement
SyncforDevicethe compiler will prevent me from sending it between threads, e.g. by disallowing me to put an instance ofDeviceinto another data structure that is potentially passed between threads.Quoted 6 lines
If I implement
SyncI have to add a safety comment on why it is safe to passDevicebetween threads. And here we have what Boqun pointed out:Devicecan only be passed between threads when we're allowed to drop the last reference from any thread. In the case of the kernel this can be any non-atomic context, any context or any other subset. But I have to write something here that is a defined rule and can be relied on.
Quoted 7 lines
You really have two things here, a matrix of:
can transfer between threads
can call in irq context
that are independent and not related to each other at all.Looks like Rust has built in support for the first. And nothing for the second as that is a very kernel-specific thing.
The language documentation defines Send as "can be transferred between threads", likely because it's written from a userspace perspective. But in the kernel context it actually means can be transferred between any context, thread, IRQ, etc.
If this isn't true, then we have to add a comment what is allowed (e.g. any non-atomic context) and what's not allowed.
Quoted 7 lines
So let's not confuse the two please.
SendandSyncshould be fine for a device pointer to be passed around, as long as the reference is incremented, as that's what all of the kernel C code does today. Let's not worry about irq context at all, that's independent and can be handled at a later time, if at all, with a different "marking" as it's independent of the current two things.
That'd be great, but as mentioned above, we only have Send, but nothing like SendIrq, hence Send really means any context.
Given your proposal, to just say it's fine to pass between (actual) threads and ignore IRQ context for now, we have to implement Send, but document that IRQ context is not covered.
We can either do that in the Rust abstraction as safety requirement, or we can, as proposed previously, add a comment to the C struct device documentation that implementers of release() must *at least* make sure that it can be called from any non-atomic context. We can then refer to that.
- Danilo
Quoted 5 lines
thanks,
greg k-h
Greg KH
maintainerRe: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 25 lines · Danilo Krummrich
Quoted 17 lines
Quoted 10 lines
Oh, I fully agree with that. Let me try to explain a bit what this is about:
In Rust we have the
SendandSyncmarker traits. If a type (e.g.Device) implementsSendit means that it's safe to pass an instance of this type between threads. Which is clearly something we want to do with aDevice.If I don't implement
SyncforDevicethe compiler will prevent me from sending it between threads, e.g. by disallowing me to put an instance ofDeviceinto another data structure that is potentially passed between threads.Quoted 6 lines
If I implement
SyncI have to add a safety comment on why it is safe to passDevicebetween threads. And here we have what Boqun pointed out:Devicecan only be passed between threads when we're allowed to drop the last reference from any thread. In the case of the kernel this can be any non-atomic context, any context or any other subset. But I have to write something here that is a defined rule and can be relied on.Quoted 7 lines
You really have two things here, a matrix of:
can transfer between threads
can call in irq context
that are independent and not related to each other at all.Looks like Rust has built in support for the first. And nothing for the second as that is a very kernel-specific thing.
Quoted 7 lines
The language documentation defines
Sendas "can be transferred between threads", likely because it's written from a userspace perspective. But in the kernel context it actually means can be transferred between any context, thread, IRQ, etc.If this isn't true, then we have to add a comment what is allowed (e.g. any non-atomic context) and what's not allowed.
That isn't good, you are going to have to change how Send works here if you think it's safe to do stuff in all of those different contexts, as that is almost never going to be true.
Why not just stick with "accessed from another thread" and leave it at that, as again, the combinations here are a matrix, not a boolean yes/no type of thing.
Quoted 10 lines
Quoted 6 lines
So let's not confuse the two please.
SendandSyncshould be fine for a device pointer to be passed around, as long as the reference is incremented, as that's what all of the kernel C code does today. Let's not worry about irq context at all, that's independent and can be handled at a later time, if at all, with a different "marking" as it's independent of the current two things.That'd be great, but as mentioned above, we only have
Send, but nothing likeSendIrq, henceSendreally means any context.
Quoted 8 lines
Given your proposal, to just say it's fine to pass between (actual) threads and ignore IRQ context for now, we have to implement
Send, but document that IRQ context is not covered.We can either do that in the Rust abstraction as safety requirement, or we can, as proposed previously, add a comment to the C
struct devicedocumentation that implementers ofrelease()must *at least* make sure that it can be called from any non-atomic context. We can then refer to that.
As someone who has written comments in code for functions that are always ignored, I am happy to see you think that this is actually going to do anything :)
Heck, I have put huge messages in kernel code for when people implement the api wrong[1], and they still ignore that at runtime. Only way to get it noticed is if you break someone's build.
So you all need to really define what Send means, as it sounds like a userspace thing that isn't going to fit in well within the kernel model.
thanks,
greg k-h
[1] See the pr_debug() calls in kobject_cleanup() as proof, people just create an "empty" release function to shut them up, thinking that is the correct solution when obviously it is not...
Re: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 34 lines · Greg KH
Quoted 25 lines · Danilo Krummrich
Quoted 17 lines
Quoted 10 lines
Oh, I fully agree with that. Let me try to explain a bit what this is about:
In Rust we have the
SendandSyncmarker traits. If a type (e.g.Device) implementsSendit means that it's safe to pass an instance of this type between threads. Which is clearly something we want to do with aDevice.If I don't implement
SyncforDevicethe compiler will prevent me from sending it between threads, e.g. by disallowing me to put an instance ofDeviceinto another data structure that is potentially passed between threads.Quoted 6 lines
If I implement
SyncI have to add a safety comment on why it is safe to passDevicebetween threads. And here we have what Boqun pointed out:Devicecan only be passed between threads when we're allowed to drop the last reference from any thread. In the case of the kernel this can be any non-atomic context, any context or any other subset. But I have to write something here that is a defined rule and can be relied on.Quoted 7 lines
You really have two things here, a matrix of:
can transfer between threads
can call in irq context
that are independent and not related to each other at all.Looks like Rust has built in support for the first. And nothing for the second as that is a very kernel-specific thing.
Quoted 7 lines
The language documentation defines
Sendas "can be transferred between threads", likely because it's written from a userspace perspective. But in the kernel context it actually means can be transferred between any context, thread, IRQ, etc.If this isn't true, then we have to add a comment what is allowed (e.g. any non-atomic context) and what's not allowed.
Quoted 7 lines
That isn't good, you are going to have to change how
Sendworks here if you think it's safe to do stuff in all of those different contexts, as that is almost never going to be true.Why not just stick with "accessed from another thread" and leave it at that, as again, the combinations here are a matrix, not a boolean yes/no type of thing.
I probably didn't phrase this very well, let me try again.
What the compiler can do for us currently is to check whether a data structure is kept only in it's current context or whether it is send to another one - it can't distinguish context types though.
So, the actual definition of Send isn't really "can be send across threads", but "is not restricted to be kept in a single context".
This means that if something is "is not restricted to be kept in a single context", but limited to certain context type, we, unfortunately, just have to make it Send, but document the restriction.
Quoted 15 lines
Quoted 15 lines
Quoted 7 lines
So let's not confuse the two please.
SendandSyncshould be fine for a device pointer to be passed around, as long as the reference is incremented, as that's what all of the kernel C code does today. Let's not worry about irq context at all, that's independent and can be handled at a later time, if at all, with a different "marking" as it's independent of the current two things.That'd be great, but as mentioned above, we only have
Send, but nothing likeSendIrq, henceSendreally means any context.Given your proposal, to just say it's fine to pass between (actual) threads and ignore IRQ context for now, we have to implement
Send, but document that IRQ context is not covered.
Quoted 8 lines
Quoted 4 lines
We can either do that in the Rust abstraction as safety requirement, or we can, as proposed previously, add a comment to the C
struct devicedocumentation that implementers ofrelease()must *at least* make sure that it can be called from any non-atomic context. We can then refer to that.As someone who has written comments in code for functions that are always ignored, I am happy to see you think that this is actually going to do anything :)
One advantage we have in Rust is, that if something has a specific requirement we mark it as unsafe and the user of the API has to put it in an unsafe block, which we require a sfety comment for, where the user of the API has to explain why this is the correct thing to do in this case.
In other words we can technically enforce that people read the comment and explain how they ensure to fulfill what the comment asks for.
Quoted 4 lines
Heck, I have put huge messages in kernel code for when people implement the api wrong[1], and they still ignore that at runtime. Only way to get it noticed is if you break someone's build.
You only see the ones who still do it wrong. You probably don't have visibility of the ones who did it right due to your effort to write it down. :-)
Quoted 3 lines
So you all need to really define what
Sendmeans, as it sounds like a userspace thing that isn't going to fit in well within the kernel model.
Please see the explanation above.
Quoted 9 lines
thanks,
greg k-h
[1] See the
pr_debug()calls inkobject_cleanup()as proof, people just create an "empty" release function to shut them up, thinking that is the correct solution when obviously it is not...
Re: [PATCH v2 1/2] rust: add abstraction for struct device
Quoted 16 lines · Danilo Krummrich
Quoted 6 lines · Greg KH
On Wed, Jun 12, 2024 at 10:56:43PM +0200, Danilo Krummrich wrote:
As someone who has written comments in code for functions that are always ignored, I am happy to see you think that this is actually going
to do anything :)One advantage we have in Rust is, that if something has a specific requirement
we mark it asunsafeand the user of the API has to put it in anunsafe
block, which we require a sfety comment for, where the user of the API has to
explain why this is the correct thing to do in this case.
Quoted 3 lines
In other words we can technically enforce that people read the comment and
explain how they ensure to fulfill what the comment asks for.
Been going through this thread to catch up on the status of the patches
here: I've been working with danilo for a while, but have been busy with kernel modesetting bindings for rust. Anyway, as someone who's spent quite a lot of time writing up really detailed documentation for some of the DisplayPort MST kernel APIs only to keep running into situations where said documentation was ignored (not to criticize anyone! I'm certainly guilty of having misread documentation or not reading something myself :) I like to think about unsafe blocks in the sense of being one thing I'm less likely to have to keep repeating myself about over a mailing list. Because I know at the very least that a contributor has to actively ignore a compiler warning or be aware in some sense that there is some kind of safety constraint they're being held responsible for that will come up during code review. If it's just a comment in C code, I don't even have a guarantee that a contributor's IDE isn't hiding large kdoc comments by default.
Also, in regards to rust code relying on C code not being guaranteed safe even if it is 'safe' code: frankly, sometimes I wish "unsafe" was just named "danger". "un-safe" implies code outside the block is perfectly "safe" - but it is still possible to introduce UB in entirely safe code bases (source: cve-rs) and most rust contributors I've talked to don't see safe code as being perfectly-safe either.
Quoted 32 lines
Quoted 7 lines
Heck, I have put huge messages in kernel code for when people implement
the api wrong[1], and they still ignore that at runtime. Only way to get
it noticed is if you break someone's build.You only see the ones who still do it wrong. You probably don't have visibility
of the ones who did it right due to your effort to write it down. :-)Quoted 5 lines
So you all need to really define what
Sendmeans, as it sounds like a
userspace thing that isn't going to fit in well within the kernel model.Please see the explanation above.
Quoted 11 lines
thanks,
greg k-h
[1] See the
pr_debug()calls inkobject_cleanup()as proof, people just
create an "empty" release function to shut them up, thinking that is
the correct solution when obviously it is not...
Signature · 3 lines
Cheers, Lyude Paul (she/her) Software Engineer at Red Hat
PATCH v2 2/2 rust: add firmware abstractions
In reply to Danilo KrummrichAdd an abstraction around the kernels firmware API to request firmware images. The abstraction provides functions to access the firmware's size and backing buffer.
The firmware is released once the abstraction instance is dropped.
--- rust/bindings/bindings_helper.h | 1 + rust/kernel/firmware.rs | 107 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + 3 files changed, 109 insertions(+) create mode 100644 rust/kernel/firmware.rs
rust/bindings/bindings_helper.h +1 -0
@@ -9,6 +9,7 @@#include <kunit/test.h> #include <linux/errname.h> #include <linux/ethtool.h> +#include <linux/firmware.h> #include <linux/jiffies.h> #include <linux/mdio.h> #include <linux/phy.h>
rust/kernel/firmware.rs new file +107 -0
107 lines not on this page · raw message
rust/kernel/lib.rs +1 -0
@@ -30,6 +30,7 @@mod build_assert;pub mod device;pub mod error; +pub mod firmware;pub mod init;pub mod ioctl;#[cfg( CONFIG_KUNIT )]
Signature · 1 line
2.45.1
Greg KH
maintainerRe: [PATCH v2 2/2] rust: add firmware abstractions
Quoted 77 lines · Danilo Krummrich
Add an abstraction around the kernels firmware API to request firmware images. The abstraction provides functions to access the firmware's size and backing buffer.
The firmware is released once the abstraction instance is dropped.
Signed-off-by Danilo Krummrich <[email protected]>--- rust/bindings/bindings_helper.h | 1 + rust/kernel/firmware.rs | 107 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + 3 files changed, 109 insertions(+) create mode 100644 rust/kernel/firmware.rsrust/bindings/bindings_helper.h +1
-0@@ -9,6 +9,7 @@#include <kunit/test.h> #include <linux/errname.h> #include <linux/ethtool.h> +#include <linux/firmware.h> #include <linux/jiffies.h> #include <linux/mdio.h> #include <linux/phy.h> rust/kernel/firmware.rs new file +45
-0@@ -0,0 +1,107 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Firmware abstraction +//! +//! C header: [`include/linux/firmware.h`](srctree/include/linux/firmware.h") + +use crate ::{bindings, device::Device , error::Error , error::Result , str::CStr }; +use core::ptr::NonNull ; + +// One of the following: `bindings::request_firmware`, `bindings::firmware_request_nowarn`, +// `firmware_request_platform`, `bindings::request_firmware_direct` +type FwFunc = +unsafe extern "C" fn (*mut *const bindings::firmware , *const i8 , *mut bindings::device ) ->i32 ; + +/// Abstraction around a C `struct firmware`. +/// +/// This is a simple abstraction around the C firmware API. Just like with the C API, firmware can +/// be requested. Once requested the abstraction provides direct access to the firmware buffer as +/// `&[u8]`. The firmware is released once [`Firmware`] is dropped. +/// +/// # Invariants +/// +/// The pointer is valid, and has ownership over the instance of `struct firmware`. +/// +/// # Examples +/// +/// ``` +/// use kernel::firmware::Firmware; +/// +/// let fw = Firmware::request("path/to/firmware.bin", dev.as_ref())?; +/// driver_load_firmware(fw.data()); +/// ``` +pub struct Firmware (NonNull <bindings::firmware >); + +impl Firmware { +fn request_internal (name: &CStr , dev: &Device , func:FwFunc ) ->Result <Self > { +let mut fw: *mut bindings::firmware = core::ptr::null_mut (); +let pfw: *mut *mut bindings::firmware = &mut fw; + +// SAFETY: `pfw` is a valid pointer to a NULL initialized `bindings::firmware` pointer. +// `name` and `dev` are valid as by their type invariants. +let ret =unsafe {func (pfwas _ , name.as_char_ptr (), dev.as_raw ()) }; +if ret !=0 { +return Err (Error ::from_errno (ret)); + }
rust/kernel/firmware.rs line 55
Quoted 10 lines
rust/kernel/firmware.rs new file +10
-0+ +// SAFETY: `func` not bailing out with a non-zero error code, guarantees that `fw` is a +// valid pointer to `bindings::firmware`. +Ok (Firmware (unsafe {NonNull ::new_unchecked (fw) })) + } + +/// Send a firmware request and wait for it. See also `bindings::request_firmware`. +pub fn request (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::request_firmware) + }
How does this handle when CONFIG_FW_LOADER is not enabled? Why are you building these bindings if that option is not checked?
Quoted 8 lines
rust/kernel/firmware.rs new file +8
-0+ +/// Send a request for an optional firmware module. See also +/// `bindings::firmware_request_nowarn`. +pub fn request_nowarn (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::firmware_request_nowarn) + } + +/// Send a request for a firmware with platform-fw fallback. See also
rust/kernel/firmware.rs line 73
Quoted 10 lines
rust/kernel/firmware.rs new file +10
-0+/// `bindings::firmware_request_platform`. +pub fn request_platform (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::firmware_request_platform) + } + +/// Send a request for a firmware directly without usermode helper. See also +/// `bindings::request_firmware_direct`. +pub fn request_direct (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::request_firmware_direct) + }
Why just these variants? Why not just add the ones that people actually need instead of a random assortment like you choose here :)
thanks,
greg k-h
Re: [PATCH v2 2/2] rust: add firmware abstractions
Quoted 83 lines · Greg KH
Quoted 82 lines · Danilo Krummrich
Add an abstraction around the kernels firmware API to request firmware images. The abstraction provides functions to access the firmware's size and backing buffer.
The firmware is released once the abstraction instance is dropped.
Signed-off-by Danilo Krummrich <[email protected]>--- rust/bindings/bindings_helper.h | 1 + rust/kernel/firmware.rs | 107 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + 3 files changed, 109 insertions(+) create mode 100644 rust/kernel/firmware.rsrust/bindings/bindings_helper.h +1
-0@@ -9,6 +9,7 @@#include <kunit/test.h> #include <linux/errname.h> #include <linux/ethtool.h> +#include <linux/firmware.h> #include <linux/jiffies.h> #include <linux/mdio.h> #include <linux/phy.h> rust/kernel/firmware.rs new file +50
-0@@ -0,0 +1,107 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Firmware abstraction +//! +//! C header: [`include/linux/firmware.h`](srctree/include/linux/firmware.h") + +use crate ::{bindings, device::Device , error::Error , error::Result , str::CStr }; +use core::ptr::NonNull ; + +// One of the following: `bindings::request_firmware`, `bindings::firmware_request_nowarn`, +// `firmware_request_platform`, `bindings::request_firmware_direct` +type FwFunc = +unsafe extern "C" fn (*mut *const bindings::firmware , *const i8 , *mut bindings::device ) ->i32 ; + +/// Abstraction around a C `struct firmware`. +/// +/// This is a simple abstraction around the C firmware API. Just like with the C API, firmware can +/// be requested. Once requested the abstraction provides direct access to the firmware buffer as +/// `&[u8]`. The firmware is released once [`Firmware`] is dropped. +/// +/// # Invariants +/// +/// The pointer is valid, and has ownership over the instance of `struct firmware`. +/// +/// # Examples +/// +/// ``` +/// use kernel::firmware::Firmware; +/// +/// let fw = Firmware::request("path/to/firmware.bin", dev.as_ref())?; +/// driver_load_firmware(fw.data()); +/// ``` +pub struct Firmware (NonNull <bindings::firmware >); + +impl Firmware { +fn request_internal (name: &CStr , dev: &Device , func:FwFunc ) ->Result <Self > { +let mut fw: *mut bindings::firmware = core::ptr::null_mut (); +let pfw: *mut *mut bindings::firmware = &mut fw; + +// SAFETY: `pfw` is a valid pointer to a NULL initialized `bindings::firmware` pointer. +// `name` and `dev` are valid as by their type invariants. +let ret =unsafe {func (pfwas _ , name.as_char_ptr (), dev.as_raw ()) }; +if ret !=0 { +return Err (Error ::from_errno (ret)); + } + +// SAFETY: `func` not bailing out with a non-zero error code, guarantees that `fw` is a +// valid pointer to `bindings::firmware`. +Ok (Firmware (unsafe {NonNull ::new_unchecked (fw) })) + }
Quoted 8 lines
rust/kernel/firmware.rs line 55
Quoted 5 lines
rust/kernel/firmware.rs new file +5
-0+ +/// Send a firmware request and wait for it. See also `bindings::request_firmware`. +pub fn request (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::request_firmware) + }How does this handle when
CONFIG_FW_LOADERis not enabled? Why are you building these bindings if that option is not checked?
Good catch, gonna fix it.
Quoted 14 lines
Quoted 14 lines
rust/kernel/firmware.rs new file +13
-0+ +/// Send a request for an optional firmware module. See also +/// `bindings::firmware_request_nowarn`. +pub fn request_nowarn (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::firmware_request_nowarn) + } + +/// Send a request for a firmware with platform-fw fallback. See also +/// `bindings::firmware_request_platform`. +pub fn request_platform (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::firmware_request_platform) + } +
Quoted 8 lines
rust/kernel/firmware.rs line 73
Quoted 5 lines
rust/kernel/firmware.rs new file +5
-0+/// Send a request for a firmware directly without usermode helper. See also +/// `bindings::request_firmware_direct`. +pub fn request_direct (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::request_firmware_direct) + }Why just these variants? Why not just add the ones that people actually need instead of a random assortment like you choose here :)
Indeed seems a bit random, not entirely though. I chose request_firmware and firmware_request_nowarn since those are the ones we need in Nova, maybe we can switch some calls to request_firmware_into_buf in the future, once we got the allocator API in place.
I added firmware_request_platform and request_firmware_direct as well, since they share the same function signature as the ones mentioned above and hence all four of them share the same implementation through Firmware::request_internal, just passing a different function pointer.
If you prefer I can drop the latter for now though.
- Danilo
Quoted 5 lines
thanks,
greg k-h
Greg KH
maintainerRe: [PATCH v2 2/2] rust: add firmware abstractions
Quoted 123 lines · Danilo Krummrich
Quoted 83 lines · Greg KH
Quoted 82 lines · Danilo Krummrich
Add an abstraction around the kernels firmware API to request firmware images. The abstraction provides functions to access the firmware's size and backing buffer.
The firmware is released once the abstraction instance is dropped.
Signed-off-by Danilo Krummrich <[email protected]>--- rust/bindings/bindings_helper.h | 1 + rust/kernel/firmware.rs | 107 ++++++++++++++++++++++++++++++++ rust/kernel/lib.rs | 1 + 3 files changed, 109 insertions(+) create mode 100644 rust/kernel/firmware.rsrust/bindings/bindings_helper.h +1
-0@@ -9,6 +9,7 @@#include <kunit/test.h> #include <linux/errname.h> #include <linux/ethtool.h> +#include <linux/firmware.h> #include <linux/jiffies.h> #include <linux/mdio.h> #include <linux/phy.h> rust/kernel/firmware.rs new file +50
-0@@ -0,0 +1,107 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Firmware abstraction +//! +//! C header: [`include/linux/firmware.h`](srctree/include/linux/firmware.h") + +use crate ::{bindings, device::Device , error::Error , error::Result , str::CStr }; +use core::ptr::NonNull ; + +// One of the following: `bindings::request_firmware`, `bindings::firmware_request_nowarn`, +// `firmware_request_platform`, `bindings::request_firmware_direct` +type FwFunc = +unsafe extern "C" fn (*mut *const bindings::firmware , *const i8 , *mut bindings::device ) ->i32 ; + +/// Abstraction around a C `struct firmware`. +/// +/// This is a simple abstraction around the C firmware API. Just like with the C API, firmware can +/// be requested. Once requested the abstraction provides direct access to the firmware buffer as +/// `&[u8]`. The firmware is released once [`Firmware`] is dropped. +/// +/// # Invariants +/// +/// The pointer is valid, and has ownership over the instance of `struct firmware`. +/// +/// # Examples +/// +/// ``` +/// use kernel::firmware::Firmware; +/// +/// let fw = Firmware::request("path/to/firmware.bin", dev.as_ref())?; +/// driver_load_firmware(fw.data()); +/// ``` +pub struct Firmware (NonNull <bindings::firmware >); + +impl Firmware { +fn request_internal (name: &CStr , dev: &Device , func:FwFunc ) ->Result <Self > { +let mut fw: *mut bindings::firmware = core::ptr::null_mut (); +let pfw: *mut *mut bindings::firmware = &mut fw; + +// SAFETY: `pfw` is a valid pointer to a NULL initialized `bindings::firmware` pointer. +// `name` and `dev` are valid as by their type invariants. +let ret =unsafe {func (pfwas _ , name.as_char_ptr (), dev.as_raw ()) }; +if ret !=0 { +return Err (Error ::from_errno (ret)); + } + +// SAFETY: `func` not bailing out with a non-zero error code, guarantees that `fw` is a +// valid pointer to `bindings::firmware`. +Ok (Firmware (unsafe {NonNull ::new_unchecked (fw) })) + }Quoted 8 lines
rust/kernel/firmware.rs line 55
Quoted 5 lines
rust/kernel/firmware.rs new file +5
-0+ +/// Send a firmware request and wait for it. See also `bindings::request_firmware`. +pub fn request (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::request_firmware) + }How does this handle when
CONFIG_FW_LOADERis not enabled? Why are you building these bindings if that option is not checked?Good catch, gonna fix it.
Quoted 14 lines
Quoted 14 lines
rust/kernel/firmware.rs new file +13
-0+ +/// Send a request for an optional firmware module. See also +/// `bindings::firmware_request_nowarn`. +pub fn request_nowarn (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::firmware_request_nowarn) + } + +/// Send a request for a firmware with platform-fw fallback. See also +/// `bindings::firmware_request_platform`. +pub fn request_platform (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::firmware_request_platform) + } +Quoted 8 lines
rust/kernel/firmware.rs line 73
Quoted 5 lines
rust/kernel/firmware.rs new file +5
-0+/// Send a request for a firmware directly without usermode helper. See also +/// `bindings::request_firmware_direct`. +pub fn request_direct (name: &CStr , dev: &Device ) ->Result <Self > { +Self ::request_internal (name, dev, bindings::request_firmware_direct) + }Why just these variants? Why not just add the ones that people actually need instead of a random assortment like you choose here :)
Indeed seems a bit random, not entirely though. I chose
request_firmwareandfirmware_request_nowarnsince those are the ones we need in Nova, maybe we can switch some calls torequest_firmware_into_bufin the future, once we got the allocator API in place.
Quoted 6 lines
I added
firmware_request_platformandrequest_firmware_directas well, since they share the same function signature as the ones mentioned above and hence all four of them share the same implementation throughFirmware::request_internal, just passing a different function pointer.If you prefer I can drop the latter for now though.
Yes, please only add bindings that you "know" will be used. We can always add new ones later.
thanks,
greg k-h
kernel test robot
build botRe: [PATCH v2 2/2] rust: add firmware abstractions
In reply to Danilo Krummrichkernel test robot noticed the following build errors:
Hi Danilo,
kernel test robot noticed the following build errors:
[auto build test ERROR on 1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0]
url: https://github.com/intel-lab-lkp/linux/commits/Danilo-Krummrich/rust-add-abstraction-for-struct-device/20240611-020558 base: 1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0 patch link: https://lore.kernel.org/r/20240610180318.72152-3-dakr%40redhat.com patch subject: [PATCH v2 2/2] rust: add firmware abstractions config: x86_64-randconfig-012-20240612 (https://download.01.org/0day-ci/archive/20240612/[email protected]/config) compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240612/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> error[E0277]: the `?` operator can only be used in a function that returns `Result` or `Option` (or another type that implements `FromResidual`)
--> rust/doctests_kernel_generated.rs:373:65
|
370 | fn main() { #[allow(non_snake_case)] fn _doctest_main__kbuild_src_consumer_rust_kernel_firmware_rs_27_0() {
| -------------------------------------------------------------------- this function should return `Result` or `Option` to accept `?`
...
373 | let fw = Firmware::request("path/to/firmware.bin", dev.as_ref())?;
| ^ cannot use the `?` operator in a function that returns `()`
|
>> = help: the trait `FromResidual<core::result::Result<Infallible, rust_doctest_kernel_build_assert_rs_0::kernel::error::Error>>` is not implemented for `()`
--
>> error[E0425]: cannot find function `driver_load_firmware` in this scope --> rust/doctests_kernel_generated.rs:374:1
|
374 | driver_load_firmware(fw.data());
| ^^^^^^^^^^^^^^^^^^^^ not found in this scope
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki
kernel test robot
build botRe: [PATCH v2 2/2] rust: add firmware abstractions
In reply to Danilo Krummrichkernel test robot noticed the following build errors:
Hi Danilo,
kernel test robot noticed the following build errors:
[auto build test ERROR on 1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0]
url: https://github.com/intel-lab-lkp/linux/commits/Danilo-Krummrich/rust-add-abstraction-for-struct-device/20240611-020558 base: 1613e604df0cd359cf2a7fbd9be7a0bcfacfabd0 patch link: https://lore.kernel.org/r/20240610180318.72152-3-dakr%40redhat.com patch subject: [PATCH v2 2/2] rust: add firmware abstractions config: x86_64-rhel-8.3-rust (https://download.01.org/0day-ci/archive/20240613/[email protected]/config) compiler: clang version 18.1.5 (https://github.com/llvm/llvm-project 617a15a9eac96088ae5e9134248d8236e34b91b1) reproduce (this is a W=1 build): (https://download.01.org/0day-ci/archive/20240613/[email protected]/reproduce)
If you fix the issue in a separate patch/commit (i.e. not just a new version of the same patch/commit), kindly add following tags
| Reported-by: kernel test robot <[email protected]>
| Closes: https://lore.kernel.org/oe-kbuild-all/[email protected]/
All errors (new ones prefixed by >>):
>> error[E0425]: cannot find value `dev` in this scope
--> rust/doctests_kernel_generated.rs:373:52
|
373 | let fw = Firmware::request("path/to/firmware.bin", dev.as_ref())?;
| ^^^ not found in this scope
--
>> error[E0308]: mismatched types
--> rust/doctests_kernel_generated.rs:373:28
|
373 | let fw = Firmware::request("path/to/firmware.bin", dev.as_ref())?;
| ----------------- ^^^^^^^^^^^^^^^^^^^^^^ expected `&CStr`, found `&str`
| |
| arguments to this function are incorrect
| = note: expected reference &rust_doctest_kernel_build_assert_rs_0::kernel::prelude::CStr found reference &'static str
note: associated function defined here
--> rust/kernel/firmware.rs:53:12
|
53 | pub fn request(name: &CStr, dev: &Device) -> Result<Self> {
| ^^^^^^^
--
>> error[E0277]: the `?` operator can only be used in a function that returns `Result` or `Option` (or another type that implements `FromResidual`)
--> rust/doctests_kernel_generated.rs:373:65
|
370 | fn main() { #[allow(non_snake_case)] fn _doctest_main__kbuild_src_consumer_rust_kernel_firmware_rs_27_0() {
| -------------------------------------------------------------------- this function should return `Result` or `Option` to accept `?`
...
373 | let fw = Firmware::request("path/to/firmware.bin", dev.as_ref())?;
| ^ cannot use the `?` operator in a function that returns `()`
|
>> = help: the trait `FromResidual<core::result::Result<Infallible, rust_doctest_kernel_build_assert_rs_0::kernel::error::Error>>` is not implemented for `()`
--
>> error[E0425]: cannot find function `driver_load_firmware` in this scope --> rust/doctests_kernel_generated.rs:374:1
|
374 | driver_load_firmware(fw.data());
| ^^^^^^^^^^^^^^^^^^^^ not found in this scope
Signature · 2 lines
0-DAY CI Kernel Test Service https://github.com/intel/lkp-tests/wiki