Status of Rust in the block subsystem
Subsystems (2)
Andreas Hindborg
reviewerPATCH RESEND 1/1 Status of Rust in the block subsystem
This is a resend. I have been informed that the original email was malformed and might not have been delivered to the intended recipients. I apologize if you receive this email twice. Original content is below the cut.
---
I would like to propose a session to discuss the status of Rust in the block subsystem. I have two distinct points to discuss: 1) The status of Rust in the block layer, and 2) the out-of-tree Rust NVMe driver.
== The status of Rust in the block subsystem ==
Rust has special status in the block subsystem [1]. We are currently at "Stage 1" of a two stage "kind of thing", where enabling Rust code in the block subsystem is essentially an experiment that can be terminated if it does not work out. In order to move on from this stage to "Stage 2", where Rust in the block subsystem is a first class citizen, we need two things. 1) We must observe that Rust is applicable for writing block device drivers and 2) the kernel in general must have fully adopted Rust.
For 1) I just sent a patch series that brings the Rust null block driver to feature parity with the C counterpart [2]. Although the null block driver is not real hardware, I think this submission goes a long way to demonstrate that Rust is in fact applicable for writing block device drivers.
For 2) I would point to the fact that Rust in general is no longer experimental in the kernel as a whole [3]. The "Rust Experiment" has been concluded, successfully. We see adoption in the form of the binder driver, and in the large, ongoing, effort targeting GPU drivers in the DRM subsystem. A few network PHY drivers also made it in. There is a lot of activity on the Rust list though, and I expect we will see Rust in more and more subsystems over the next few years.
On that basis, I suggest we discuss whether or not we can move to "Stage 2" - and what that entails.
== The out-of-tree Rust PCI NVMe driver ==
As you may know, I am maintaining a Rust PCI NVMe driver out of tree. For the development work of the block layer Rust APIs, it has been essential to implement a real driver, in addition to the null block driver. A driver targeting actual hardware has to deal with interrupts, registers, IO memory, DMA, memory mappings, in a way that an emulated device does not. Keeping this driver around has helped shape the Rust block APIs in a way that makes sense for real devices, not just emulated ones.
Rebasing this driver every cycle is a lot of work. Driven by requirements of the DRM drivers, core Rust APIs for DMA, register access, IO memory, etc. are being reshaped and rethought, which is great. Rebasing on top of these changes as a second class citizen all the time - not so great. But I believe that the Rust NVMe driver is a great piece of reference code, so I keep doing it.
I would ask that we discuss merging the Rust NVMe driver so as to have a reference implementation of a real driver in-tree. I think it would have the following benefits:
- It would help ensure that the Rust block layer API actually works with real devices.
- It would help make sure that the changes to core APIs that are made to facilitate other subsystems are done in a way that is compatible with block layer requirements.
- It would serve as a nice reference implementation and a great piece of documentation.
- I would not have to rebase it all the time. Patches changing Rust APIs used by Rust NVMe would have to change the Rust NVMe call sites.
Please note the emphasis on *reference* driver. I am not suggesting to replace the existing C driver. I will maintain the new code if necessary.
If we could establish consensus on this, I would clean up the driver to a state where it can be included in the kernel.
Best regards,
Andreas Hindborg
Andreas Hindborg
reviewerQuoted 68 lines · Andreas Hindborg
I would like to propose a session to discuss the status of Rust in the block subsystem. I have two distinct points to discuss: 1) The status of Rust in the block layer, and 2) the out-of-tree Rust NVMe driver.
== The status of Rust in the block subsystem ==
Rust has special status in the block subsystem [1]. We are currently at "Stage 1" of a two stage "kind of thing", where enabling Rust code in the block subsystem is essentially an experiment that can be terminated if it does not work out. In order to move on from this stage to "Stage 2", where Rust in the block subsystem is a first class citizen, we need two things. 1) We must observe that Rust is applicable for writing block device drivers and 2) the kernel in general must have fully adopted Rust.
For 1) I just sent a patch series that brings the Rust null block driver to feature parity with the C counterpart [2]. Although the null block driver is not real hardware, I think this submission goes a long way to demonstrate that Rust is in fact applicable for writing block device drivers.
For 2) I would point to the fact that Rust in general is no longer experimental in the kernel as a whole [3]. The "Rust Experiment" has been concluded, successfully. We see adoption in the form of the binder driver, and in the large, ongoing, effort targeting GPU drivers in the DRM subsystem. A few network PHY drivers also made it in. There is a lot of activity on the Rust list though, and I expect we will see Rust in more and more subsystems over the next few years.
On that basis, I suggest we discuss whether or not we can move to "Stage 2" - and what that entails.
== The out-of-tree Rust PCI NVMe driver ==
As you may know, I am maintaining a Rust PCI NVMe driver out of tree. For the development work of the block layer Rust APIs, it has been essential to implement a real driver, in addition to the null block driver. A driver targeting actual hardware has to deal with interrupts, registers, IO memory, DMA, memory mappings, in a way that an emulated device does not. Keeping this driver around has helped shape the Rust block APIs in a way that makes sense for real devices, not just emulated ones.
Rebasing this driver every cycle is a lot of work. Driven by requirements of the DRM drivers, core Rust APIs for DMA, register access, IO memory, etc. are being reshaped and rethought, which is great. Rebasing on top of these changes as a second class citizen all the time - not so great. But I believe that the Rust NVMe driver is a great piece of reference code, so I keep doing it.
I would ask that we discuss merging the Rust NVMe driver so as to have a reference implementation of a real driver in-tree. I think it would have the following benefits:
- It would help ensure that the Rust block layer API actually works with real devices.
- It would help make sure that the changes to core APIs that are made to facilitate other subsystems are done in a way that is compatible with block layer requirements.
- It would serve as a nice reference implementation and a great piece of documentation.
- I would not have to rebase it all the time. Patches changing Rust APIs used by Rust NVMe would have to change the Rust NVMe call sites.
Please note the emphasis on *reference* driver. I am not suggesting to replace the existing C driver. I will maintain the new code if necessary.
If we could establish consensus on this, I would clean up the driver to a state where it can be included in the kernel.
Quoted 6 lines
Best regards,
Andreas Hindborg
As this topic was not selected for discussion at LSF, and I did not receive an invitation for LSF this year, I propose that we discuss these two topics on list.
I do believe that these topics need to be discussed, and I would very much appreciate your input.
Best regards,
Andreas Hindborg
Quoted 6 lines · Andreas Hindborg
As this topic was not selected for discussion at LSF, and I did not receive an invitation for LSF this year, I propose that we discuss these two topics on list.
I do believe that these topics need to be discussed, and I would very much appreciate your input.
I can sympathise the difficulty of maintaining external modules.
In terms of this being a reference driver, that implies some future hardware driver may leverage this for its development. Is there anything in mind at this point for production? If so, maybe that use case should take the lead. But either way, I think rust-nvme upstream inclusion would invite confusion. Once it's upstream, it's no longer a reference when distros and users turn it on.
Quoted 8 lines · Keith Busch
Quoted 6 lines · Andreas Hindborg
As this topic was not selected for discussion at LSF, and I did not receive an invitation for LSF this year, I propose that we discuss these two topics on list.
I do believe that these topics need to be discussed, and I would very much appreciate your input.
Quoted 9 lines
I can sympathise the difficulty of maintaining external modules.
In terms of this being a reference driver, that implies some future hardware driver may leverage this for its development. Is there anything in mind at this point for production? If so, maybe that use case should take the lead. But either way, I think rust-nvme upstream inclusion would invite confusion. Once it's upstream, it's no longer a reference when distros and users turn it on.
I wholeheartedly agree.
While I do see the original appeal to have a rust-nvme driver, having one will just lead to confusion on all sides, especially for distros. (Why is it there? should it be preferred to the original one? Do we have to support both of them? Are there features missing in either of these drivers?)
In general we are trying hard to avoid duplication in the linux kernel, especially on the driver side. We constantly have to fight^Wargue with driver vendors why duplicating existing drivers to support new hardware is a bad idea, so we really should not start now just because the driver is written in another language.
(That really might be giving vendors bad ideas :-)
Cheers,
Hannes
Signature · 4 lines
Dr. Hannes Reinecke Kernel Storage Architect [email protected] +49 911 74053 688 SUSE Software Solutions GmbH, Frankenstr. 146, 90461 Nürnberg HRB 36809 (AG Nürnberg), GF: I. Totev, A. McDonald, W. Knoblich
Quoted 12 lines · Hannes Reinecke
While I do see the original appeal to have a rust-nvme driver, having one will just lead to confusion on all sides, especially for distros. (Why is it there? should it be preferred to the original one? Do we have to support both of them? Are there features missing in either of these drivers?)
In general we are trying hard to avoid duplication in the linux kernel, especially on the driver side. We constantly have to fight^Wargue with driver vendors why duplicating existing drivers to support new hardware is a bad idea, so we really should not start now just because the driver is written in another language.
(That really might be giving vendors bad ideas :-)
I think when Andreas refers to "reference driver" he may be referring to the https://rust-for-linux.com/rust-reference-drivers framework. If so, those are meant to be exceptional, temporary and well-justified.
So vendors cannot use it as an excuse to try to duplicate stuff (they may try, but it should be clear it is not the same case).
And, for users, we should make it so that it is pretty clear they are not intended to be used. Perhaps they could be gated behind EXPERT or BROKEN or staging or similar. Perhaps even a new Kconfig option to mark the ones we have (proposed in the list and in-tree) and that allows to easily grep and to tweak the conditions under they can be enabled etc.
Now, with that said, like Keith mentions, it should nevertheless be clear what the final user of the abstractions are, i.e. the goal is to bootstrap other code which will not be dead.
I hope that helps...
Cheers,
Miguel
Andreas Hindborg
reviewerQuoted 20 lines · Hannes Reinecke
Quoted 8 lines · Keith Busch
Quoted 6 lines · Andreas Hindborg
As this topic was not selected for discussion at LSF, and I did not receive an invitation for LSF this year, I propose that we discuss these two topics on list.
I do believe that these topics need to be discussed, and I would very much appreciate your input.
Quoted 9 lines
I can sympathise the difficulty of maintaining external modules.
In terms of this being a reference driver, that implies some future hardware driver may leverage this for its development. Is there anything in mind at this point for production? If so, maybe that use case should take the lead. But either way, I think rust-nvme upstream inclusion would invite confusion. Once it's upstream, it's no longer a reference when distros and users turn it on.
I wholeheartedly agree.
Quoted 11 lines
While I do see the original appeal to have a rust-nvme driver, having one will just lead to confusion on all sides, especially for distros. (Why is it there? should it be preferred to the original one? Do we have to support both of them? Are there features missing in either of these drivers?)
In general we are trying hard to avoid duplication in the linux kernel, especially on the driver side. We constantly have to fight^Wargue with driver vendors why duplicating existing drivers to support new hardware is a bad idea, so we really should not start now just because the driver is written in another language.
(That really might be giving vendors bad ideas :-)
I actually agree to some extent. But I do think we can get around most confusion with loud and clear documentation. We could make the driver not probe by default, requiring a configfs setting to probe. Or leave the PCI identifier table empty, so patching the driver is required to make it probe.
For me, the big benefit would be having the rust nvme driver as part of an allmodconfig or allyesconfig. That would prevent a ton of trouble.
We do plan to utilize the block infrastructure, but I think we are still quite a long way from sending anything. Keeping the rust nvme driver in tree till that point would prevent pci, dma, irq, etc. from developing in ways that would not support a block device use case. As an example, upstream Rust irq APIs are not actually able to support NVMe at the moment. They work fine for GPU drivers though. And I cannot go an fix them without a user. Same for DMA pool.
I could go and find some other piece of unsupported PCI hardware and write a driver for that, use that to keep the APIs in shape upstream. It's just a lot more work and the NVMe driver is already here and 90% ready.
Or we can fix the APIs at a later point, when we have a user for them that is not a duplicated reference driver. My fear is that by then, I have to patch a number of GPU drivers in the process.
Best regards,
Andreas Hindborg
Quoted 51 lines · Andreas Hindborg
Quoted 20 lines · Hannes Reinecke
Quoted 8 lines · Keith Busch
Quoted 6 lines · Andreas Hindborg
As this topic was not selected for discussion at LSF, and I did not receive an invitation for LSF this year, I propose that we discuss these two topics on list.
I do believe that these topics need to be discussed, and I would very much appreciate your input.
Quoted 9 lines
I can sympathise the difficulty of maintaining external modules.
In terms of this being a reference driver, that implies some future hardware driver may leverage this for its development. Is there anything in mind at this point for production? If so, maybe that use case should take the lead. But either way, I think rust-nvme upstream inclusion would invite confusion. Once it's upstream, it's no longer a reference when distros and users turn it on.
I wholeheartedly agree.
Quoted 11 lines
While I do see the original appeal to have a rust-nvme driver, having one will just lead to confusion on all sides, especially for distros. (Why is it there? should it be preferred to the original one? Do we have to support both of them? Are there features missing in either of these drivers?)
In general we are trying hard to avoid duplication in the linux kernel, especially on the driver side. We constantly have to fight^Wargue with driver vendors why duplicating existing drivers to support new hardware is a bad idea, so we really should not start now just because the driver is written in another language.
(That really might be giving vendors bad ideas :-)I actually agree to some extent. But I do think we can get around most confusion with loud and clear documentation. We could make the driver not probe by default, requiring a configfs setting to probe. Or leave the PCI identifier table empty, so patching the driver is required to make it probe.
For me, the big benefit would be having the rust nvme driver as part of an allmodconfig or allyesconfig. That would prevent a ton of trouble.
We do plan to utilize the block infrastructure, but I think we are still quite a long way from sending anything. Keeping the rust nvme driver in tree till that point would prevent pci, dma, irq, etc. from developing in ways that would not support a block device use case. As an example, upstream Rust irq APIs are not actually able to support NVMe at the moment. They work fine for GPU drivers though. And I cannot go an fix them without a user. Same for DMA pool.
Quoted 4 lines
I could go and find some other piece of unsupported PCI hardware and write a driver for that, use that to keep the APIs in shape upstream. It's just a lot more work and the NVMe driver is already here and 90% ready.
This implies that there really is no "need" for these rust bindings at all, if you don't know of, or are planning for, any real driver for them. So why have them at all?
For the PCI and driver core bindings, and the majority of the other ones merged, we have real users (binder, nova-core, etc.) and so we are willing to take them and keep them up to date. For these block bindings, why is it even worth it to have them around if there's never going to be a real user?
thanks,
greg k-h
Andreas Hindborg
reviewerQuoted 61 lines · Greg KH
Quoted 51 lines · Andreas Hindborg
Quoted 20 lines · Hannes Reinecke
Quoted 8 lines · Keith Busch
Quoted 6 lines · Andreas Hindborg
As this topic was not selected for discussion at LSF, and I did not receive an invitation for LSF this year, I propose that we discuss these two topics on list.
I do believe that these topics need to be discussed, and I would very much appreciate your input.
Quoted 9 lines
I can sympathise the difficulty of maintaining external modules.
In terms of this being a reference driver, that implies some future hardware driver may leverage this for its development. Is there anything in mind at this point for production? If so, maybe that use case should take the lead. But either way, I think rust-nvme upstream inclusion would invite confusion. Once it's upstream, it's no longer a reference when distros and users turn it on.
I wholeheartedly agree.
Quoted 11 lines
While I do see the original appeal to have a rust-nvme driver, having one will just lead to confusion on all sides, especially for distros. (Why is it there? should it be preferred to the original one? Do we have to support both of them? Are there features missing in either of these drivers?)
In general we are trying hard to avoid duplication in the linux kernel, especially on the driver side. We constantly have to fight^Wargue with driver vendors why duplicating existing drivers to support new hardware is a bad idea, so we really should not start now just because the driver is written in another language.
(That really might be giving vendors bad ideas :-)I actually agree to some extent. But I do think we can get around most confusion with loud and clear documentation. We could make the driver not probe by default, requiring a configfs setting to probe. Or leave the PCI identifier table empty, so patching the driver is required to make it probe.
For me, the big benefit would be having the rust nvme driver as part of an allmodconfig or allyesconfig. That would prevent a ton of trouble.
We do plan to utilize the block infrastructure, but I think we are still quite a long way from sending anything. Keeping the rust nvme driver in tree till that point would prevent pci, dma, irq, etc. from developing in ways that would not support a block device use case. As an example, upstream Rust irq APIs are not actually able to support NVMe at the moment. They work fine for GPU drivers though. And I cannot go an fix them without a user. Same for DMA pool.
Quoted 4 lines
I could go and find some other piece of unsupported PCI hardware and write a driver for that, use that to keep the APIs in shape upstream. It's just a lot more work and the NVMe driver is already here and 90% ready.
This implies that there really is no "need" for these rust bindings at all, if you don't know of, or are planning for, any real driver for them. So why have them at all?
Quoted 5 lines
For the PCI and driver core bindings, and the majority of the other ones merged, we have real users (binder, nova-core, etc.) and so we are willing to take them and keep them up to date. For these block bindings, why is it even worth it to have them around if there's never going to be a real user?
I'm just going to quote myself in case you missed these few sentences:
We do plan to utilize the block infrastructure, but I think we are still quite a long way from sending anything.
<cut>
My fear is that by then, I have to patch a number of GPU drivers in the process.
Best regards,
Andreas Hindborg
Quoted 71 lines · Andreas Hindborg
Quoted 61 lines · Greg KH
Quoted 51 lines · Andreas Hindborg
Quoted 20 lines · Hannes Reinecke
Quoted 8 lines · Keith Busch
Quoted 6 lines · Andreas Hindborg
As this topic was not selected for discussion at LSF, and I did not receive an invitation for LSF this year, I propose that we discuss these two topics on list.
I do believe that these topics need to be discussed, and I would very much appreciate your input.
Quoted 9 lines
I can sympathise the difficulty of maintaining external modules.
In terms of this being a reference driver, that implies some future hardware driver may leverage this for its development. Is there anything in mind at this point for production? If so, maybe that use case should take the lead. But either way, I think rust-nvme upstream inclusion would invite confusion. Once it's upstream, it's no longer a reference when distros and users turn it on.
I wholeheartedly agree.
Quoted 11 lines
While I do see the original appeal to have a rust-nvme driver, having one will just lead to confusion on all sides, especially for distros. (Why is it there? should it be preferred to the original one? Do we have to support both of them? Are there features missing in either of these drivers?)
In general we are trying hard to avoid duplication in the linux kernel, especially on the driver side. We constantly have to fight^Wargue with driver vendors why duplicating existing drivers to support new hardware is a bad idea, so we really should not start now just because the driver is written in another language.
(That really might be giving vendors bad ideas :-)I actually agree to some extent. But I do think we can get around most confusion with loud and clear documentation. We could make the driver not probe by default, requiring a configfs setting to probe. Or leave the PCI identifier table empty, so patching the driver is required to make it probe.
For me, the big benefit would be having the rust nvme driver as part of an allmodconfig or allyesconfig. That would prevent a ton of trouble.
We do plan to utilize the block infrastructure, but I think we are still quite a long way from sending anything. Keeping the rust nvme driver in tree till that point would prevent pci, dma, irq, etc. from developing in ways that would not support a block device use case. As an example, upstream Rust irq APIs are not actually able to support NVMe at the moment. They work fine for GPU drivers though. And I cannot go an fix them without a user. Same for DMA pool.
Quoted 4 lines
I could go and find some other piece of unsupported PCI hardware and write a driver for that, use that to keep the APIs in shape upstream. It's just a lot more work and the NVMe driver is already here and 90% ready.
This implies that there really is no "need" for these rust bindings at all, if you don't know of, or are planning for, any real driver for them. So why have them at all?
Quoted 5 lines
For the PCI and driver core bindings, and the majority of the other ones merged, we have real users (binder, nova-core, etc.) and so we are willing to take them and keep them up to date. For these block bindings, why is it even worth it to have them around if there's never going to be a real user?
I'm just going to quote myself in case you missed these few sentences:
Quoted 7 lines
We do plan to utilize the block infrastructure, but I think we are still quite a long way from sending anything.
<cut>
My fear is that by then, I have to patch a number of GPU drivers in the process.
That's the "penalty" for not getting your code "ready" sooner :)
Seriously, don't make others do the maintenance work on these apis until you have something that you want to have merged. It's always been that way in Linux, this isn't anything new.
thanks,
greg k-h
Andreas Hindborg
reviewerQuoted 77 lines · Greg KH
Quoted 76 lines · Andreas Hindborg
Quoted 61 lines · Greg KH
Quoted 51 lines · Andreas Hindborg
Quoted 20 lines · Hannes Reinecke
Quoted 8 lines · Keith Busch
Quoted 6 lines · Andreas Hindborg
As this topic was not selected for discussion at LSF, and I did not receive an invitation for LSF this year, I propose that we discuss these two topics on list.
I do believe that these topics need to be discussed, and I would very much appreciate your input.
Quoted 9 lines
I can sympathise the difficulty of maintaining external modules.
In terms of this being a reference driver, that implies some future hardware driver may leverage this for its development. Is there anything in mind at this point for production? If so, maybe that use case should take the lead. But either way, I think rust-nvme upstream inclusion would invite confusion. Once it's upstream, it's no longer a reference when distros and users turn it on.
I wholeheartedly agree.
Quoted 11 lines
While I do see the original appeal to have a rust-nvme driver, having one will just lead to confusion on all sides, especially for distros. (Why is it there? should it be preferred to the original one? Do we have to support both of them? Are there features missing in either of these drivers?)
In general we are trying hard to avoid duplication in the linux kernel, especially on the driver side. We constantly have to fight^Wargue with driver vendors why duplicating existing drivers to support new hardware is a bad idea, so we really should not start now just because the driver is written in another language.
(That really might be giving vendors bad ideas :-)I actually agree to some extent. But I do think we can get around most confusion with loud and clear documentation. We could make the driver not probe by default, requiring a configfs setting to probe. Or leave the PCI identifier table empty, so patching the driver is required to make it probe.
For me, the big benefit would be having the rust nvme driver as part of an allmodconfig or allyesconfig. That would prevent a ton of trouble.
We do plan to utilize the block infrastructure, but I think we are still quite a long way from sending anything. Keeping the rust nvme driver in tree till that point would prevent pci, dma, irq, etc. from developing in ways that would not support a block device use case. As an example, upstream Rust irq APIs are not actually able to support NVMe at the moment. They work fine for GPU drivers though. And I cannot go an fix them without a user. Same for DMA pool.
Quoted 4 lines
I could go and find some other piece of unsupported PCI hardware and write a driver for that, use that to keep the APIs in shape upstream. It's just a lot more work and the NVMe driver is already here and 90% ready.
This implies that there really is no "need" for these rust bindings at all, if you don't know of, or are planning for, any real driver for them. So why have them at all?
Quoted 5 lines
For the PCI and driver core bindings, and the majority of the other ones merged, we have real users (binder, nova-core, etc.) and so we are willing to take them and keep them up to date. For these block bindings, why is it even worth it to have them around if there's never going to be a real user?
I'm just going to quote myself in case you missed these few sentences:
We do plan to utilize the block infrastructure, but I think we are still quite a long way from sending anything.
<cut>
Quoted 8 lines
Quoted 2 lines
My fear is that by then, I have to patch a number of GPU drivers in the process.
That's the "penalty" for not getting your code "ready" sooner :)
Seriously, don't make others do the maintenance work on these apis until you have something that you want to have merged. It's always been that way in Linux, this isn't anything new.
That is fair.
I appreciate you all taking your time to discuss this. I think with the input I have now received (again), I'm going to let go of the rust NVMe driver and focus my energy elsewhere. We'll carry patches for the subsystems we need downstream, until we are ready to send a real user.
Best regards,
Andreas Hindborg