Rust support
Subsystems (22)
[email protected]
maintainercover Rust support
From: Miguel Ojeda <[email protected]>
Rust support
This is the patch series to add support for Rust as a second language to the Linux kernel.
If you are interested in following this effort, please join us in the mailing list at:
and take a look at the project itself at:
https://github.com/Rust-for-Linux
Cheers,
Miguel
--
# Rust support
This cover letter explains the major changes and updates done since the RFC sent in the previous merge window back in April, plus a few extra notes and announcements. For the RFC, please see:
https://lore.kernel.org/lkml/[email protected]/
## Rust infrastructure updates
There have been several major improvements to the overall Rust support. The following subsections cover these.
### Removed panicking allocations
We have removed infallible allocations. In order to do so, we have integrated a subset of the alloc standard library crate, with some additions on top. This allows us to customize things to our needs, while giving upstream the time they need to evaluate our changes.
Eventually, the goal is to have everything the kernel needs in upstream alloc and drop it from the kernel tree. We have already started this process and some changes have been already accepted upstream.
On top of that, alloc is now compiled with panicking allocation methods disabled, thus they cannot be used within the kernel by mistake either.
Moreover, the documentation for this customized alloc crate (as well as for core) is now generated alongside the rest of the Rust kernel documentation. Thus kernel developers can now easily browse the subset that is available within the kernel. Like last time, you can take a look at a preview of the documentation at:
https://rust-for-linux.github.io/docs/alloc/
Note that the compiler_builtins panicking intrinsics are still there, but those will be solved by partitioning core via feature gates. The first one, for disabling floating-point functionality, has just been accepted upstream.
### Beta compiler supported
Up until now, we have been using nightly releases of rustc because we need some of the latest fixes and unstable features.
However, the kernel can now be compiled with beta and stable rustc releases. At the moment, we are using the 1.54-beta1 version as our reference compiler. At the end of this month, rustc 1.54 will be released, and we will move to that version as our reference.
Note that the kernel still requires unstable features, even if it is compiled with a stable rustc release, thus we cannot guarantee that future rustc versions will work without changes to the kernel tree.
Thus, until all the unstable features we need are stabilized, we will support a single rustc version for each kernel release.
### Testing support
Another big addition has been the support for testing. We now support the standard Rust #[test] attribute to easily write tests, e.g.:
#[test]
fn f() {
let a = 20;
let b = 22;
assert_eq!(a + b, 42);
}Furthermore, we are now also supporting Rust documentation tests ("doctests"). These allow us to make sure our examples remain up to date, and also double as tests too, e.g.:
/// ```
/// assert_eq!(foo::f(), 42);
/// ```
pub fn f() -> i32 {
42
}For the moment, both kinds of tests are run in the host only, but the goal is to have them running in kernel space, so that we can test code that depends on kernel features, and to allow any kernel module to declare and use them.
### Architectures and compiler support
arm (i.e. 32-bit) and riscv are now also supported.
On compilers, we would like to mention all the work that has been going in GCC Rust (a GCC frontend for the Rust language) and rustc_codegen_gcc (a rustc backend for GCC). The latter now passes all core tests, and the former is now working on traits. We continue to track their progress as they may become the best way to have GCC-built kernels with Rust support enabled. Their latest reports can be found at:
https://blog.antoyo.xyz/rustc_codegen_gcc-progress-report-1 https://thephilbert.io/2021/06/28/gcc-rust-weekly-status-report-20/
On top of that, we requested Compiler Explorer to add support to all the alternative compilers. At the time of writing, they have already added mrustc and GCC Rust; and rustc_codegen_gcc is coming soon. See a live example at:
https://godbolt.org/z/8o74c57Yj
## Rust abstractions and driver updates
We have developed new Rust abstractions that use the kernel C
implementations: red-black trees, reference-counted objects, file descriptor creation, tasks, files, io vectors...
Additionally, we have improved driver support: improvements to file_operations (more operations supported, arbitrary state), reduced boiler-plate code, improved module! macro, registration macros, rudimentary (probe and remove) platform drivers...
On Binder, there is now support for transferring files descriptors and LSM hooks; and we are working on preliminary performance numbers.
Moreover, there is ongoing work on a Rust example driver, bcm2835-rng. This is the hardware random-number generator present on Raspberry Pi Zero(W), Classic, Two, and Three.
There are other small improvements, such as drivers being restricted on what unstable features they can use.
## Patch series status
Like it was mentioned in the RFC, the Rust support is still to be considered experimental. However, as noted back in April, support is good enough that kernel developers can start working on the Rust abstractions for subsystems and write drivers and other modules.
Please note that the current series have just arrived in linux-next, thus the first run will happen on Tuesday.
## Industry and academia support
We have been in contact with a set of companies and academia members that would like to use Rust as a second language in the kernel. Some of them have already started to evaluate Rust for their needs using the infrastructure we have already in place.
In particular, we have got a few statements from major companies. In no particular order:
Microsoft's Linux Systems Group is interested in contributing to getting Rust into Linux kernel. Hopefully we will be able to submit select Hyper-V drivers written in Rust in the coming months.
Arm recognises the Rust value proposition and is actively working with the Rust community to improve Rust for Arm based systems. A good example is Arm’s RFC contribution to the Rust language which made Linux on 64-bit Arm systems a Tier-1 Rust supported platform. Rustaceans at Arm are excited about the Rust for Linux initiative and look forward to assisting in this effort.
Google supports and contributes directly to the Rust for Linux project. Our Android team is evaluating a new Binder implementation and considering other drivers where Rust could be adopted.
In addition, IBM contributed the Rust kernel support for PowerPC which was already included in the RFC.
In addition, from academia, there are already several projects around Rust for Linux going on. As an example, members of LSE (Systems Research Laboratory) at EPITA (École pour l'informatique et les techniques avancées) are developing an SPI Rust driver.
And, of course, special thanks go to ISRG (Internet Security Research Group) and Google for their financial support on this endeavor.
## Conferences and talks
We have submitted talk proposals for LPC (Linux Plumbers Conference).
The main one will describe the work we have done so far and ideally will also serve as an introduction for other kernel developers interested in using Rust in the kernel. It will cover an introduction of the language within the context of the kernel, how the overall Rust support works, how code is documented, how tests are written, a tour of available tooling, an explanation of coding guidelines, how kernel driver code looks like, etc.
In addition, we would like to announce that we are organizing a new conference that focuses on Rust and the Linux kernel. The first edition will be virtual and will take place before LPC. Details will be announced soon.
## Acknowledgements
The signatures in the main commits correspond to the people that wrote code that has ended up in them at the present time. For details on contributions to code and discussions, please see our repository:
https://github.com/Rust-for-Linux/linux
However, we would like to give credit to everyone that has contributed in one way or another to the Rust for Linux project. Since the RFC:
- bjorn3 for all the input on Rust compiler details and all the reviews and suggestions.
- Arthur Cohen, Esteban Blanc and Martin Schmidt for their ongoing work on the SPI abstractions and driver.
- Dan Robertson for his ongoing work on softdeps in the module! macro and the addition of a few safety comments.
- Paul Römer for his ongoing experiment on using NonNull as much as possible.
- Sladyn Nunes for his ongoing sorting of error constants.
- Jonathan Corbet and the LPC organizers for lending us the Linux Plumbers Conference infrastructure so that we can have an easy time setting up the new conference.
- John Ericson for quickly implementing the no_global_oom_handling cfg feature in upstream alloc to easily disable all functionality that relies on panicking allocations.
- Josh Triplett and John Ericson for their input on alloc which helped us decide what to do with it (i.e. fully custom vs. slightly custom in-tree copy vs. upstream), as well as offering to help moving forward some needed features on the Rust side.
- Mark Rousskov for answering some questions about beta backports and scheduling, as well as working on unsticking 1.54.0-beta.1.
- Philipp Krones for his input on Clippy lints and discussing extensions for developing custom lints.
- Antoni Boucher for his work on rustc_codegen_gcc.
- Philip Herrons (and his supporters Open Source Security and Embecosm) for his work on GCC Rust.
- Marc Poulhiès for his work on Compiler Explorer to add the alternative Rust compilers we requested.
- Many folks that have reported issues, tested the project, helped spread the word, joined discussions and contributed in other ways! In no particular order: Chenguang Wang, Greg Morenz, John Baublitz, Leah Leshchinsky, Caedin Cook, Liam Arzola, Fabio Aiuto, Hanqing Zhao, Robin Randhawa, Michal Rostecki, Wei Liu...
For additional acknowledgements, please see the RFC from April.
Miguel Ojeda (17):
kallsyms: support big kernel symbols (2-byte lengths)
kallsyms: increase maximum kernel symbol length to 512
Makefile: generate CLANG_FLAGS even in GCC builds
vsprintf: add new %pA format specifier
rust: add C helpers
rust: add compiler_builtins crate
rust: add alloc crate
rust: add build_error crate
rust: add macros crate
rust: add kernel crate
rust: export generated symbols
Kbuild: add Rust support
docs: add Rust documentation
samples: add Rust examples
scripts: add generate_rust_analyzer.py
MAINTAINERS: Rust
Android: Binder IPC in Rust (WIP)
.gitignore | 5 + .rustfmt.toml | 12 + Documentation/doc-guide/kernel-doc.rst | 3 + Documentation/index.rst | 1 + Documentation/kbuild/kbuild.rst | 4 + Documentation/process/changes.rst | 13 + Documentation/rust/arch-support.rst | 35 + Documentation/rust/assets/favicon-16x16.png | Bin 0 -> 798 bytes Documentation/rust/assets/favicon-32x32.png | Bin 0 -> 2076 bytes Documentation/rust/assets/rust-logo.png | Bin 0 -> 53976 bytes Documentation/rust/coding.rst | 92 + Documentation/rust/docs.rst | 110 + Documentation/rust/index.rst | 20 + Documentation/rust/quick-start.rst | 222 ++ MAINTAINERS | 14 + Makefile | 176 +- arch/arm/rust/target.json | 28 + arch/arm64/rust/target.json | 35 + arch/powerpc/rust/target.json | 30 + arch/riscv/Makefile | 1 + arch/riscv/rust/rv32ima.json | 37 + arch/riscv/rust/rv32imac.json | 37 + arch/riscv/rust/rv64ima.json | 37 + arch/riscv/rust/rv64imac.json | 37 + arch/x86/rust/target.json | 37 + drivers/android/Kconfig | 7 + drivers/android/Makefile | 2 + drivers/android/allocation.rs | 264 ++ drivers/android/context.rs | 80 + drivers/android/defs.rs | 99 + drivers/android/node.rs | 476 +++ drivers/android/process.rs | 972 ++++++ drivers/android/range_alloc.rs | 189 ++ drivers/android/rust_binder.rs | 114 + drivers/android/thread.rs | 857 +++++ drivers/android/transaction.rs | 328 ++ include/linux/kallsyms.h | 2 +- include/linux/spinlock.h | 17 +- include/uapi/linux/android/binder.h | 28 +- init/Kconfig | 28 + kernel/kallsyms.c | 7 + kernel/livepatch/core.c | 4 +- kernel/printk/printk.c | 5 +- lib/Kconfig.debug | 144 + lib/vsprintf.c | 12 + rust/.gitignore | 6 + rust/Makefile | 316 ++ rust/alloc/README.md | 32 + rust/alloc/alloc.rs | 425 +++ rust/alloc/borrow.rs | 493 +++ rust/alloc/boxed.rs | 1728 ++++++++++ rust/alloc/collections/mod.rs | 116 + rust/alloc/fmt.rs | 587 ++++ rust/alloc/lib.rs | 197 ++ rust/alloc/macros.rs | 128 + rust/alloc/prelude/mod.rs | 17 + rust/alloc/prelude/v1.rs | 16 + rust/alloc/raw_vec.rs | 612 ++++ rust/alloc/rc.rs | 2539 +++++++++++++++ rust/alloc/slice.rs | 1271 ++++++++ rust/alloc/str.rs | 614 ++++ rust/alloc/string.rs | 2847 ++++++++++++++++ rust/alloc/sync.rs | 2631 +++++++++++++++ rust/alloc/vec/drain.rs | 157 + rust/alloc/vec/drain_filter.rs | 145 + rust/alloc/vec/into_iter.rs | 296 ++ rust/alloc/vec/is_zero.rs | 106 + rust/alloc/vec/mod.rs | 3255 +++++++++++++++++++ rust/alloc/vec/partial_eq.rs | 49 + rust/alloc/vec/set_len_on_drop.rs | 30 + rust/alloc/vec/spec_extend.rs | 170 + rust/bindgen_parameters | 13 + rust/build_error.rs | 33 + rust/compiler_builtins.rs | 146 + rust/exports.c | 16 + rust/helpers.c | 235 ++ rust/kernel/allocator.rs | 63 + rust/kernel/bindings.rs | 28 + rust/kernel/bindings_helper.h | 24 + rust/kernel/buffer.rs | 39 + rust/kernel/build_assert.rs | 80 + rust/kernel/c_types.rs | 119 + rust/kernel/chrdev.rs | 212 ++ rust/kernel/error.rs | 272 ++ rust/kernel/file.rs | 130 + rust/kernel/file_operations.rs | 698 ++++ rust/kernel/io_buffer.rs | 153 + rust/kernel/iov_iter.rs | 95 + rust/kernel/lib.rs | 220 ++ rust/kernel/linked_list.rs | 245 ++ rust/kernel/miscdev.rs | 113 + rust/kernel/module_param.rs | 497 +++ rust/kernel/of.rs | 101 + rust/kernel/pages.rs | 176 + rust/kernel/platdev.rs | 166 + rust/kernel/prelude.rs | 28 + rust/kernel/print.rs | 412 +++ rust/kernel/random.rs | 50 + rust/kernel/raw_list.rs | 361 ++ rust/kernel/rbtree.rs | 570 ++++ rust/kernel/security.rs | 79 + rust/kernel/static_assert.rs | 39 + rust/kernel/str.rs | 259 ++ rust/kernel/sync/arc.rs | 227 ++ rust/kernel/sync/condvar.rs | 136 + rust/kernel/sync/guard.rs | 82 + rust/kernel/sync/locked_by.rs | 112 + rust/kernel/sync/mod.rs | 84 + rust/kernel/sync/mutex.rs | 101 + rust/kernel/sync/spinlock.rs | 109 + rust/kernel/sysctl.rs | 198 ++ rust/kernel/task.rs | 193 ++ rust/kernel/traits.rs | 26 + rust/kernel/types.rs | 249 ++ rust/kernel/user_ptr.rs | 191 ++ rust/macros/lib.rs | 127 + rust/macros/module.rs | 754 +++++ samples/Kconfig | 2 + samples/Makefile | 1 + samples/rust/Kconfig | 113 + samples/rust/Makefile | 12 + samples/rust/rust_chrdev.rs | 51 + samples/rust/rust_minimal.rs | 38 + samples/rust/rust_miscdev.rs | 150 + samples/rust/rust_module_parameters.rs | 72 + samples/rust/rust_print.rs | 57 + samples/rust/rust_random.rs | 61 + samples/rust/rust_semaphore.rs | 177 + samples/rust/rust_semaphore_c.c | 212 ++ samples/rust/rust_stack_probing.rs | 40 + samples/rust/rust_sync.rs | 81 + scripts/Makefile.build | 22 + scripts/Makefile.lib | 12 + scripts/generate_rust_analyzer.py | 143 + scripts/kallsyms.c | 33 +- scripts/kconfig/confdata.c | 67 +- scripts/rust-version.sh | 31 + tools/include/linux/kallsyms.h | 2 +- tools/include/linux/lockdep.h | 2 +- tools/lib/perf/include/perf/event.h | 2 +- tools/lib/symbol/kallsyms.h | 2 +- 141 files changed, 33003 insertions(+), 45 deletions(-) create mode 100644 .rustfmt.toml create mode 100644 Documentation/rust/arch-support.rst create mode 100644 Documentation/rust/assets/favicon-16x16.png create mode 100644 Documentation/rust/assets/favicon-32x32.png create mode 100644 Documentation/rust/assets/rust-logo.png create mode 100644 Documentation/rust/coding.rst create mode 100644 Documentation/rust/docs.rst create mode 100644 Documentation/rust/index.rst create mode 100644 Documentation/rust/quick-start.rst create mode 100644 arch/arm/rust/target.json create mode 100644 arch/arm64/rust/target.json create mode 100644 arch/powerpc/rust/target.json create mode 100644 arch/riscv/rust/rv32ima.json create mode 100644 arch/riscv/rust/rv32imac.json create mode 100644 arch/riscv/rust/rv64ima.json create mode 100644 arch/riscv/rust/rv64imac.json create mode 100644 arch/x86/rust/target.json create mode 100644 drivers/android/allocation.rs create mode 100644 drivers/android/context.rs create mode 100644 drivers/android/defs.rs create mode 100644 drivers/android/node.rs create mode 100644 drivers/android/process.rs create mode 100644 drivers/android/range_alloc.rs create mode 100644 drivers/android/rust_binder.rs create mode 100644 drivers/android/thread.rs create mode 100644 drivers/android/transaction.rs create mode 100644 rust/.gitignore create mode 100644 rust/Makefile create mode 100644 rust/alloc/README.md create mode 100644 rust/alloc/alloc.rs create mode 100644 rust/alloc/borrow.rs create mode 100644 rust/alloc/boxed.rs create mode 100644 rust/alloc/collections/mod.rs create mode 100644 rust/alloc/fmt.rs create mode 100644 rust/alloc/lib.rs create mode 100644 rust/alloc/macros.rs create mode 100644 rust/alloc/prelude/mod.rs create mode 100644 rust/alloc/prelude/v1.rs create mode 100644 rust/alloc/raw_vec.rs create mode 100644 rust/alloc/rc.rs create mode 100644 rust/alloc/slice.rs create mode 100644 rust/alloc/str.rs create mode 100644 rust/alloc/string.rs create mode 100644 rust/alloc/sync.rs create mode 100644 rust/alloc/vec/drain.rs create mode 100644 rust/alloc/vec/drain_filter.rs create mode 100644 rust/alloc/vec/into_iter.rs create mode 100644 rust/alloc/vec/is_zero.rs create mode 100644 rust/alloc/vec/mod.rs create mode 100644 rust/alloc/vec/partial_eq.rs create mode 100644 rust/alloc/vec/set_len_on_drop.rs create mode 100644 rust/alloc/vec/spec_extend.rs create mode 100644 rust/bindgen_parameters create mode 100644 rust/build_error.rs create mode 100644 rust/compiler_builtins.rs create mode 100644 rust/exports.c create mode 100644 rust/helpers.c create mode 100644 rust/kernel/allocator.rs create mode 100644 rust/kernel/bindings.rs create mode 100644 rust/kernel/bindings_helper.h create mode 100644 rust/kernel/buffer.rs create mode 100644 rust/kernel/build_assert.rs create mode 100644 rust/kernel/c_types.rs create mode 100644 rust/kernel/chrdev.rs create mode 100644 rust/kernel/error.rs create mode 100644 rust/kernel/file.rs create mode 100644 rust/kernel/file_operations.rs create mode 100644 rust/kernel/io_buffer.rs create mode 100644 rust/kernel/iov_iter.rs create mode 100644 rust/kernel/lib.rs create mode 100644 rust/kernel/linked_list.rs create mode 100644 rust/kernel/miscdev.rs create mode 100644 rust/kernel/module_param.rs create mode 100644 rust/kernel/of.rs create mode 100644 rust/kernel/pages.rs create mode 100644 rust/kernel/platdev.rs create mode 100644 rust/kernel/prelude.rs create mode 100644 rust/kernel/print.rs create mode 100644 rust/kernel/random.rs create mode 100644 rust/kernel/raw_list.rs create mode 100644 rust/kernel/rbtree.rs create mode 100644 rust/kernel/security.rs create mode 100644 rust/kernel/static_assert.rs create mode 100644 rust/kernel/str.rs create mode 100644 rust/kernel/sync/arc.rs create mode 100644 rust/kernel/sync/condvar.rs create mode 100644 rust/kernel/sync/guard.rs create mode 100644 rust/kernel/sync/locked_by.rs create mode 100644 rust/kernel/sync/mod.rs create mode 100644 rust/kernel/sync/mutex.rs create mode 100644 rust/kernel/sync/spinlock.rs create mode 100644 rust/kernel/sysctl.rs create mode 100644 rust/kernel/task.rs create mode 100644 rust/kernel/traits.rs create mode 100644 rust/kernel/types.rs create mode 100644 rust/kernel/user_ptr.rs create mode 100644 rust/macros/lib.rs create mode 100644 rust/macros/module.rs create mode 100644 samples/rust/Kconfig create mode 100644 samples/rust/Makefile create mode 100644 samples/rust/rust_chrdev.rs create mode 100644 samples/rust/rust_minimal.rs create mode 100644 samples/rust/rust_miscdev.rs create mode 100644 samples/rust/rust_module_parameters.rs create mode 100644 samples/rust/rust_print.rs create mode 100644 samples/rust/rust_random.rs create mode 100644 samples/rust/rust_semaphore.rs create mode 100644 samples/rust/rust_semaphore_c.c create mode 100644 samples/rust/rust_stack_probing.rs create mode 100644 samples/rust/rust_sync.rs create mode 100755 scripts/generate_rust_analyzer.py create mode 100755 scripts/rust-version.sh
Signature · 1 line
2.32.0
[email protected]
maintainerPATCH 1/17 kallsyms: support big kernel symbols (2-byte lengths)
From: Miguel Ojeda <[email protected]>
Rust symbols can become quite long due to namespacing introduced by modules, types, traits, generics, etc.
Increasing to 255 is not enough in some cases, and therefore we need to introduce 2-byte lengths to the symbol table. We call these "big" symbols.
In order to avoid increasing all lengths to 2 bytes (since most of them only require 1 byte, including many Rust ones), we use length zero to mark "big" symbols in the table.
--- kernel/kallsyms.c | 7 +++++++ scripts/kallsyms.c | 31 ++++++++++++++++++++++++++++--- 2 files changed, 35 insertions(+), 3 deletions(-)
kernel/kallsyms.c +7 -0
@@ -73,6 +73,13 @@ static unsigned int kallsyms_expand_symbol(unsigned int off,*/ off += len +1 ; +/* If zero, it is a "big" symbol, so a two byte length follows. */ +if (len ==0 ) { + len = (data[0 ] <<8 ) | data[1 ]; + data +=2 ; + off += len +2 ; + } +/* * For every byte on the compressed symbol data, copy the table * entry for that byte.
scripts/kallsyms.c +28 -3
@@ -470,12 +470,37 @@ static void write_src(void)if ((i &0xFF ) ==0 ) markers[i >>8 ] = off; -printf ("\t.byte 0x%02x" , table[i]->len); +/* +* There cannot be any symbol of length zero -- we use that +* to mark a "big" symbol (and it doesn't make sense anyway). +*/ +if (table[i]->len ==0 ) { +fprintf (stderr,"kallsyms failure: " +"unexpected zero symbol length\n" ); +exit (EXIT_FAILURE ); + } + +/* Only lengths that fit in up to two bytes are supported. */ +if (table[i]->len >0xFFFF ) { +fprintf (stderr,"kallsyms failure: " +"unexpected huge symbol length\n" ); +exit (EXIT_FAILURE ); + } + +if (table[i]->len <=0xFF ) { +/* Most symbols use a single byte for the length. */ +printf ("\t.byte 0x%02x" , table[i]->len); + off += table[i]->len +1 ; + }else { +/* "Big" symbols use a zero and then two bytes. */ +printf ("\t.byte 0x00, 0x%02x, 0x%02x" , + (table[i]->len >>8 ) &0xFF , + table[i]->len &0xFF ); + off += table[i]->len +3 ; + }for (k =0 ; k < table[i]->len; k++)printf (", 0x%02x" , table[i]->sym[k]);printf ("\n" ); - - off += table[i]->len +1 ; }printf ("\n" );
Signature · 1 line
2.32.0
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
kernel/kallsyms.c static unsigned int kallsyms_expand_symbol(unsigned int off, · line 81
Quoted 8 lines · On Sun, Jul 4, 2021 at 1:28 PM <[email protected]> wrote:
kernel/kallsyms.c +6
-0+/* If zero, it is a "big" symbol, so a two byte length follows. */ +if (len ==0 ) { + len = (data[0 ] <<8 ) | data[1 ]; + data +=2 ; + off += len +2 ; + }
Side note: it might be a good idea at this point to verify that "len" is >255.
Also, why is this in big-endian order?
Let's just try to kill big-endian data, it's disgusting and should just die already.
BE is practically dead anyway, we shouldn't add new cases. Networking has legacy reasons from the bad old days when byte order wars were still a thing, but those days are gone.
Linus
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 2 lines · Linus Torvalds
Also, why is this in big-endian order?
No particular reason. It makes sense to use LE -- I will change it.
Cheers,
Miguel
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 2 lines · Miguel Ojeda
No particular reason. It makes sense to use LE -- I will change it.
Matthew's suggestion is denser, which is nice.
At that point, it would be neither BE nor LE. But the "LE-like" version would be
len = data[0];
if (len & 128)
len += data[1] << 7;which ends up having a tiny bit more range (it goes to 11^H32895).
Of course, if the range is expected to be just 0-300, I guess that matters not one whit.
Linus
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
In reply to [email protected]Quoted 5 lines · On Sun, Jul 04, 2021 at 10:27:40PM +0200, [email protected] wrote:
From: Miguel Ojeda <[email protected]>
Rust symbols can become quite long due to namespacing introduced by modules, types, traits, generics, etc.
Quoted 7 lines
Increasing to 255 is not enough in some cases, and therefore we need to introduce 2-byte lengths to the symbol table. We call these "big" symbols.
In order to avoid increasing all lengths to 2 bytes (since most of them only require 1 byte, including many Rust ones), we use length zero to mark "big" symbols in the table.
What happened to my suggestion from last time of encoding symbols < 128 as 0-127 and symbols larger than that as (data[0] - 128) * 256 + data[1]) ?
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 4 lines · Matthew Wilcox
What happened to my suggestion from last time of encoding symbols < 128 as 0-127 and symbols larger than that as (data[0] - 128) * 256 + data[1]) ?
Nothing, sorry, we focused on other parts (e.g. the allocation panics) during this iteration. I can take a look for v2.
Cheers,
Miguel
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 8 lines · Miguel Ojeda
Quoted 4 lines · Matthew Wilcox
What happened to my suggestion from last time of encoding symbols < 128 as 0-127 and symbols larger than that as (data[0] - 128) * 256 + data[1]) ?
Nothing, sorry, we focused on other parts (e.g. the allocation panics) during this iteration. I can take a look for v2.
Here's what I have. Build testing now.
kernel/kallsyms.c +8 -0
@@ -67,6 +67,14 @@ static unsigned int kallsyms_expand_symbol(unsigned int off, len = *data; data++; +/* lengths larger than 128 are encoded as two bytes */ +if (len >=128 ) { + len -=128 ; + len *=256 ; + len += *data; + data++; + } +/* * Update the offset to return the offset for the next symbol on * the compressed stream.
scripts/kallsyms.c +7 -1
@@ -467,10 +467,16 @@ static void write_src(void)output_label ("kallsyms_names" ); off =0 ;for (i =0 ; i < table_cnt; i++) { +int len = table[i]->len;if ((i &0xFF ) ==0 ) markers[i >>8 ] = off; -printf ("\t.byte 0x%02x" , table[i]->len); +if (len >=128 ) { +printf ("\t.byte 0x%02x\n" , len /256 +128 ); + len %=256 ; + off++; + } +printf ("\t.byte 0x%02x" , len);for (k =0 ; k < table[i]->len; k++)printf (", 0x%02x" , table[i]->sym[k]);printf ("\n" );
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 11 lines · Matthew Wilcox
Quoted 8 lines · Miguel Ojeda
Quoted 4 lines · Matthew Wilcox
What happened to my suggestion from last time of encoding symbols < 128 as 0-127 and symbols larger than that as (data[0] - 128) * 256 + data[1]) ?
Nothing, sorry, we focused on other parts (e.g. the allocation panics) during this iteration. I can take a look for v2.
Here's what I have. Build testing now.
It seems to work. Sample output from kallsyms with the test forced to 'true':
.byte 0x80
.byte 0x0a, 0x41, 0xff, 0x70, 0xf3, 0xd0, 0xb0, 0xf2, 0xfc, 0x72, 0x74under normal circumstances, it omits that first line.
I'm unfortunately in the middle of ripping apart and fixing up my test infrastructure, so I can't claim to have really tested it, and I don't have a module with a really large symbol anyway. But I'll submit this to you as a proper patch with changelog if that's enough testing for you.
(now that i look at kallsyms, why on earth is it ripping the string apart and turning it into .byte instead of just using .ascii?)
Quoted 42 lines
kernel/kallsyms.c +8
-0@@ -67,6 +67,14 @@ static unsigned int kallsyms_expand_symbol(unsigned int off, len = *data; data++; +/* lengths larger than 128 are encoded as two bytes */ +if (len >=128 ) { + len -=128 ; + len *=256 ; + len += *data; + data++; + } +/* * Update the offset to return the offset for the next symbol on * the compressed stream. scripts/kallsyms.c +7
-1@@ -467,10 +467,16 @@ static void write_src(void)output_label ("kallsyms_names" ); off =0 ;for (i =0 ; i < table_cnt; i++) { +int len = table[i]->len;if ((i &0xFF ) ==0 ) markers[i >>8 ] = off; -printf ("\t.byte 0x%02x" , table[i]->len); +if (len >=128 ) { +printf ("\t.byte 0x%02x\n" , len /256 +128 ); + len %=256 ; + off++; + } +printf ("\t.byte 0x%02x" , len);for (k =0 ; k < table[i]->len; k++)printf (", 0x%02x" , table[i]->sym[k]);printf ("\n" );
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 5 lines · Matthew Wilcox
I'm unfortunately in the middle of ripping apart and fixing up my test infrastructure, so I can't claim to have really tested it, and I don't have a module with a really large symbol anyway. But I'll submit this to you as a proper patch with changelog if that's enough testing for you.
Sure, and thanks! I can give it a spin with our tests.
Cheers,
Miguel
Gary Guo
maintainerRe: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
In reply to Matthew WilcoxQuoted 46 lines · On Sun, 4 Jul 2021 22:33:39 +0100 Matthew Wilcox <[email protected]> wrote:
Quoted 9 lines · Miguel Ojeda
Quoted 4 lines · Matthew Wilcox
What happened to my suggestion from last time of encoding symbols < 128 as 0-127 and symbols larger than that as (data[0] - 128) * 256 + data[1]) ?
Nothing, sorry, we focused on other parts (e.g. the allocation panics) during this iteration. I can take a look for v2.
Here's what I have. Build testing now.
kernel/kallsyms.c +8
-0@@ -67,6 +67,14 @@ static unsigned int kallsyms_expand_symbol(unsigned int off, len = *data; data++; +/* lengths larger than 128 are encoded as two bytes */ +if (len >=128 ) { + len -=128 ; + len *=256 ; + len += *data; + data++; + } +/* * Update the offset to return the offset for the next symbol on
kernel/kallsyms.c +0
-0* the compressed stream. scripts/kallsyms.c +1
-0@@ -467,10 +467,16 @@ static void write_src(void)output_label ("kallsyms_names" ); off =0 ;for (i =0 ; i < table_cnt; i++) { +int len = table[i]->len;if ((i &0xFF ) ==0 ) markers[i >>8 ] = off;
scripts/kallsyms.c static void write_src(void) · line 482
Quoted 10 lines
scripts/kallsyms.c +6
-1-printf ("\t.byte 0x%02x" , table[i]->len); +if (len >=128 ) { +printf ("\t.byte 0x%02x\n" , len /256 +128 ); + len %=256 ; + off++; + } +printf ("\t.byte 0x%02x" , len);for (k =0 ; k < table[i]->len; k++)printf (", 0x%02x" , table[i]->sym[k]);printf ("\n" );
This is big endian.
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 1 line · Gary Guo
This is big endian.
Fundamentally, it doesn't matter whether it's encoded as top-7 + bottom-8 or bottom-7 + top-8. It could just as well be:
if (len >= 128) {
len -= 128;
len += *data * 256;
data++;
}It doesn't matter whether it's compatible with some other encoding. This encoding has one producer and one consumer. As long as they agree, it's fine. If you want to make an argument about extensibiity, then I'm going to suggest that wanting a symbol name more than 32kB in size is a sign you've done something else very, very wrong.
At that point, you should probably switch to comparing hashes of the symbol instead of the symbol. Indeed, I think we're already there at 300 byte symbols; we should probably SipHash the full, unmangled symbol [1]. At 33k symbols in the current kernel, the risk of a collision of a 64-bit value is negligible, and almost every kernel symbol is longer than 7 bytes (thankfully).
[1] ie SipHash("void unlock_page(struct page *)")
Gary Guo
maintainerRe: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 9 lines · On Sun, 4 Jul 2021 23:42:03 +0100 Matthew Wilcox <[email protected]> wrote:
Quoted 1 line · Gary Guo
This is big endian.
Fundamentally, it doesn't matter whether it's encoded as top-7 + bottom-8 or bottom-7 + top-8. It could just as well be:
if (len >= 128) { len -= 128; len += *data * 256;
Do you mean *data * 128?
Quoted 6 lines
data++; }It doesn't matter whether it's compatible with some other encoding. This encoding has one producer and one consumer. As long as they agree, it's fine.
I am aware that this is only for internal tooling so it doesn't really matter. I mentioned that it's big endian to do top-7 + bottom-8 because Linus suggests that big-endian shouldn't be used.
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
In reply to Matthew WilcoxQuoted 18 lines · Matthew Wilcox
Quoted 1 line · Gary Guo
This is big endian.
Fundamentally, it doesn't matter whether it's encoded as top-7 + bottom-8 or bottom-7 + top-8. It could just as well be:
if (len >= 128) { len -= 128; len += *data * 256; data++; }It doesn't matter whether it's compatible with some other encoding. This encoding has one producer and one consumer. As long as they agree, it's fine. If you want to make an argument about extensibiity, then I'm going to suggest that wanting a symbol name more than 32kB in size is a sign you've done something else very, very wrong.
Quoted 6 lines
At that point, you should probably switch to comparing hashes of the symbol instead of the symbol. Indeed, I think we're already there at 300 byte symbols; we should probably SipHash the full, unmangled symbol [1]. At 33k symbols in the current kernel, the risk of a collision of a 64-bit value is negligible, and almost every kernel symbol is longer than 7 bytes (thankfully).
We really should have a better standard varint encoding - open coding varint encodings in 2021 is offensive, and LEB128 is retarded due to using the high bit of _every_ byte. Here's the encoding I did for bcachefs, which I nominate for a standard varint encoding, unless someone knows of a way to do better:
https://evilpiepirate.org/git/bcachefs.git/tree/fs/bcachefs/varint.c
Gary Guo
maintainerRe: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
In reply to Matthew WilcoxQuoted 10 lines · On Sun, 4 Jul 2021 22:04:49 +0100 Matthew Wilcox <[email protected]> wrote:
Quoted 9 lines · On Sun, Jul 04, 2021 at 10:27:40PM +0200, [email protected] wrote:
From: Miguel Ojeda <[email protected]>
Rust symbols can become quite long due to namespacing introduced by modules, types, traits, generics, etc.
Increasing to 255 is not enough in some cases, and therefore we need to introduce 2-byte lengths to the symbol table. We call these "big" symbols.
Quoted 7 lines
Quoted 3 lines
In order to avoid increasing all lengths to 2 bytes (since most of them only require 1 byte, including many Rust ones), we use length zero to mark "big" symbols in the table.
What happened to my suggestion from last time of encoding symbols < 128 as 0-127 and symbols larger than that as (data[0] - 128) * 256 + data[1]) ?
Yeah, I agree ULEB128 or similar encoding scheme would be better than using 0 as an escape byte. If ULEB128 is used and we restrict number of bytes to 2, it will encode numbers up to 2**14 instead of 2**16 like the current scheme, but that should be sufficient anyway.
- Gary
Re: [PATCH 01/17] kallsyms: support big kernel symbols (2-byte lengths)
Quoted 17 lines · Gary Guo
Quoted 14 lines · On Sun, 4 Jul 2021 22:04:49 +0100 Matthew Wilcox <[email protected]> wrote:
Quoted 12 lines · On Sun, Jul 04, 2021 at 10:27:40PM +0200, [email protected] wrote:
From: Miguel Ojeda <[email protected]>
Rust symbols can become quite long due to namespacing introduced by modules, types, traits, generics, etc.
Increasing to 255 is not enough in some cases, and therefore we need to introduce 2-byte lengths to the symbol table. We call these "big" symbols.
In order to avoid increasing all lengths to 2 bytes (since most of them only require 1 byte, including many Rust ones), we use length zero to mark "big" symbols in the table.
Quoted 8 lines
Quoted 3 lines
What happened to my suggestion from last time of encoding symbols < 128 as 0-127 and symbols larger than that as (data[0] - 128) * 256 + data[1]) ?
Yeah, I agree ULEB128 or similar encoding scheme would be better than using 0 as an escape byte. If ULEB128 is used and we restrict number of bytes to 2, it will encode numbers up to 2**14 instead of 2**16 like the current scheme, but that should be sufficient anyway.
Actually plenty of variants of such encodings exist. You can split the first byte on 192 to keep 6 upper bits, 224 for 5, 240 for 4, etc. It all depends how long the maximum string is expected to be and how often we expect to see large strings. For example when splitting around 240, all sizes from 0 to 239 take one byte, and sizes from 240 to 4335 take two bytes.
But if strings >128 are already extremely rare we don't really care about the extra byte needed to encode them.
Willy
[email protected]
maintainerPATCH 2/17 kallsyms: increase maximum kernel symbol length to 512
In reply to [email protected]From: Miguel Ojeda <[email protected]>
Rust symbols can become quite long due to namespacing introduced by modules, types, traits, generics, etc. For instance, for:
pub mod my_module {
pub struct MyType;
pub struct MyGenericType<T>(T);
pub trait MyTrait {
fn my_method() -> u32;
}
impl MyTrait for MyGenericType<MyType> {
fn my_method() -> u32 {
42
}
}
}generates a symbol of length 96 when using the upcoming v0 mangling scheme:
_RNvXNtCshGpAVYOtgW1_7example9my_moduleINtB2_13MyGenericTypeNtB2_6MyTypeENtB2_7MyTrait9my_method
At the moment, Rust symbols may reach up to 300 in length. Setting 512 as the maximum seems like a reasonable choice to keep some headroom.
--- include/linux/kallsyms.h | 2 +- kernel/livepatch/core.c | 4 ++-- scripts/kallsyms.c | 2 +- tools/include/linux/kallsyms.h | 2 +- tools/include/linux/lockdep.h | 2 +- tools/lib/perf/include/perf/event.h | 2 +- tools/lib/symbol/kallsyms.h | 2 +- 7 files changed, 8 insertions(+), 8 deletions(-)
include/linux/kallsyms.h +1 -1
@@ -14,7 +14,7 @@#include <asm/sections.h> -#define KSYM_NAME_LEN 128 +#define KSYM_NAME_LEN 512#define KSYM_SYMBOL_LEN (sizeof("%s+%#lx/%#lx [%s]") + (KSYM_NAME_LEN - 1) + \ 2*(BITS_PER_LONG*3/10) + (MODULE_NAME_LEN - 1) + 1)
kernel/livepatch/core.c +2 -2
@@ -213,7 +213,7 @@ static int klp_resolve_symbols(Elf64_Shdr *sechdrs, const char *strtab,* we use the smallest/strictest upper bound possible (56, based on * the current definition of MODULE_NAME_LEN) to prevent overflows. */ -BUILD_BUG_ON (MODULE_NAME_LEN <56 ||KSYM_NAME_LEN !=128 ); +BUILD_BUG_ON (MODULE_NAME_LEN <56 ||KSYM_NAME_LEN !=512 ); relas = (Elf_Rela *) relasec->sh_addr;/* For each rela in this klp relocation section */
@@ -227,7 +227,7 @@ static int klp_resolve_symbols(Elf64_Shdr *sechdrs, const char *strtab,/* Format: .klp.sym.sym_objname.sym_name,sympos */ cnt =sscanf (strtab + sym->st_name, -".klp.sym.%55[^.].%127[^,],%lu" , +".klp.sym.%55[^.].%511[^,],%lu" , sym_objname, sym_name, &sympos);if (cnt !=3 ) { pr_err("symbol %s has an incorrectly formatted name\n",
scripts/kallsyms.c +1 -1
@@ -27,7 +27,7 @@#define ARRAY_SIZE (arr) (sizeof(arr) / sizeof(arr[0])) -#define KSYM_NAME_LEN 128 +#define KSYM_NAME_LEN 512struct sym_entry {unsigned long long addr;
tools/include/linux/kallsyms.h +1 -1
@@ -6,7 +6,7 @@#include <stdio.h> #include <unistd.h> -#define KSYM_NAME_LEN 128 +#define KSYM_NAME_LEN 512struct module ;
tools/include/linux/lockdep.h +1 -1
@@ -47,7 +47,7 @@ static inline int debug_locks_off(void)#define task_pid_nr (tsk) ((tsk)->pid) -#define KSYM_NAME_LEN 128 +#define KSYM_NAME_LEN 512#define printk (...) dprintf(STDOUT_FILENO, __VA_ARGS__)#define pr_err (format, ...) fprintf (stderr, format, ## __VA_ARGS__)#define pr_warn pr_err
tools/lib/perf/include/perf/event.h +1 -1
@@ -95,7 +95,7 @@ struct perf_record_throttle { };#ifndef KSYM_NAME_LEN -#define KSYM_NAME_LEN 256 +#define KSYM_NAME_LEN 512#endif struct perf_record_ksymbol {
tools/lib/symbol/kallsyms.h +1 -1
@@ -7,7 +7,7 @@#include <linux/types.h> #ifndef KSYM_NAME_LEN -#define KSYM_NAME_LEN 256 +#define KSYM_NAME_LEN 512#endif static inline u8 kallsyms2elf_binding (char type)
Signature · 1 line
2.32.0
Re: [PATCH 02/17] kallsyms: increase maximum kernel symbol length to 512
Quoted 21 lines · On Sun, Jul 4, 2021 at 1:28 PM <[email protected]> wrote:
From: Miguel Ojeda <[email protected]>
Rust symbols can become quite long due to namespacing introduced by modules, types, traits, generics, etc. For instance, for:
pub mod my_module { pub struct MyType; pub struct MyGenericType<T>(T); pub trait MyTrait { fn my_method() -> u32; } impl MyTrait for MyGenericType<MyType> { fn my_method() -> u32 { 42 } } }
Quoted 3 lines
generates a symbol of length 96 when using the upcoming v0 mangling scheme:
_RNvXNtCshGpAVYOtgW1_7example9my_moduleINtB2_13MyGenericTypeNtB2_6MyTypeENtB2_7MyTrait9my_method
For C++ demangling, we have c++filt.
A quick search turned up: https://github.com/luser/rustfilt
but if the v0 mangling scheme is upcoming, I doubt a repo that hasn't been updated supports an upcoming scheme. Is there a more official equivalent?
Do we demangle rust symbols when printing a trace from a warn/panic? That would be nice.
Quoted 129 lines
At the moment, Rust symbols may reach up to 300 in length. Setting 512 as the maximum seems like a reasonable choice to keep some headroom.
Co-developed-by Alex Gaynor <[email protected]>Signed-off-by Alex Gaynor <[email protected]>Co-developed-by Geoffrey Thomas <[email protected]>Signed-off-by Geoffrey Thomas <[email protected]>Co-developed-by Finn Behrens <[email protected]>Signed-off-by Finn Behrens <[email protected]>Co-developed-by Adam Bratschi-Kaye <[email protected]>Signed-off-by Adam Bratschi-Kaye <[email protected]>Co-developed-by Wedson Almeida Filho <[email protected]>Signed-off-by Wedson Almeida Filho <[email protected]>Signed-off-by Miguel Ojeda <[email protected]>--- include/linux/kallsyms.h | 2 +- kernel/livepatch/core.c | 4 ++-- scripts/kallsyms.c | 2 +- tools/include/linux/kallsyms.h | 2 +- tools/include/linux/lockdep.h | 2 +- tools/lib/perf/include/perf/event.h | 2 +- tools/lib/symbol/kallsyms.h | 2 +- 7 files changed, 8 insertions(+), 8 deletions(-)include/linux/kallsyms.h +1
-1@@ -14,7 +14,7 @@#include <asm/sections.h> -#define KSYM_NAME_LEN 128 +#define KSYM_NAME_LEN 512#define KSYM_SYMBOL_LEN (sizeof("%s+%#lx/%#lx [%s]") + (KSYM_NAME_LEN - 1) + \ 2*(BITS_PER_LONG*3/10) + (MODULE_NAME_LEN - 1) + 1)kernel/livepatch/core.c +2
-2@@ -213,7 +213,7 @@ static int klp_resolve_symbols(Elf64_Shdr *sechdrs, const char *strtab,* we use the smallest/strictest upper bound possible (56, based on * the current definition of MODULE_NAME_LEN) to prevent overflows. */ -BUILD_BUG_ON (MODULE_NAME_LEN <56 ||KSYM_NAME_LEN !=128 ); +BUILD_BUG_ON (MODULE_NAME_LEN <56 ||KSYM_NAME_LEN !=512 ); relas = (Elf_Rela *) relasec->sh_addr;/* For each rela in this klp relocation section */ @@ -227,7 +227,7 @@ static int klp_resolve_symbols(Elf64_Shdr *sechdrs, const char *strtab,/* Format: .klp.sym.sym_objname.sym_name,sympos */ cnt =sscanf (strtab + sym->st_name, -".klp.sym.%55[^.].%127[^,],%lu" , +".klp.sym.%55[^.].%511[^,],%lu" , sym_objname, sym_name, &sympos);if (cnt !=3 ) { pr_err("symbol %s has an incorrectly formatted name\n",scripts/kallsyms.c +1
-1@@ -27,7 +27,7 @@#define ARRAY_SIZE (arr) (sizeof(arr) / sizeof(arr[0])) -#define KSYM_NAME_LEN 128 +#define KSYM_NAME_LEN 512struct sym_entry {unsigned long long addr;tools/include/linux/kallsyms.h +1
-1@@ -6,7 +6,7 @@#include <stdio.h> #include <unistd.h> -#define KSYM_NAME_LEN 128 +#define KSYM_NAME_LEN 512struct module ;tools/include/linux/lockdep.h +1
-1@@ -47,7 +47,7 @@ static inline int debug_locks_off(void)#define task_pid_nr (tsk) ((tsk)->pid) -#define KSYM_NAME_LEN 128 +#define KSYM_NAME_LEN 512#define printk (...) dprintf(STDOUT_FILENO, __VA_ARGS__)#define pr_err (format, ...) fprintf (stderr, format, ## __VA_ARGS__)#define pr_warn pr_errtools/lib/perf/include/perf/event.h +1
-1@@ -95,7 +95,7 @@ struct perf_record_throttle { };#ifndef KSYM_NAME_LEN -#define KSYM_NAME_LEN 256 +#define KSYM_NAME_LEN 512#endif struct perf_record_ksymbol {tools/lib/symbol/kallsyms.h +1
-2@@ -7,7 +7,7 @@#include <linux/types.h> #ifndef KSYM_NAME_LEN -#define KSYM_NAME_LEN 256 +#define KSYM_NAME_LEN 512#endif static inline u8 kallsyms2elf_binding (char type) --2.32.0
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH 02/17] kallsyms: increase maximum kernel symbol length to 512
Quoted 3 lines · Nick Desaulniers
Do we demangle rust symbols when printing a trace from a warn/panic? That would be nice.
I think it would be pretty much required. Otherwise stack traces are going to be very very painful.
In fact, I'm starting to think that Willy is right: we should use hashes for the "real" symbol name, and have demangled names for printing, and at no point would the kernel actually want or need to have the nasty mangled names.
(This wouldn't be rust-specific - using hashes for module linking sounds like a good diea for C code as well, even if the de-mangled names for printing are then the regular ones)
Anybody interested in looking into that? It would make this "big kernel symbols" patch immaterial.
Linus
Re: [PATCH 02/17] kallsyms: increase maximum kernel symbol length to 512
Quoted 14 lines · Linus Torvalds
Quoted 3 lines · Nick Desaulniers
Do we demangle rust symbols when printing a trace from a warn/panic? That would be nice.
I think it would be pretty much required. Otherwise stack traces are going to be very very painful.
In fact, I'm starting to think that Willy is right: we should use hashes for the "real" symbol name, and have demangled names for printing, and at no point would the kernel actually want or need to have the nasty mangled names.
Quoted 6 lines
(This wouldn't be rust-specific - using hashes for module linking sounds like a good diea for C code as well, even if the de-mangled names for printing are then the regular ones)
Anybody interested in looking into that? It would make this "big kernel symbols" patch immaterial.
Are you thinking the hashed kernel symbols need to have their types included in the hash? Or is this just a hash to make the names a managable size?
Eric
Re: [PATCH 02/17] kallsyms: increase maximum kernel symbol length to 512
Quoted 3 lines · Eric W. Biederman
Are you thinking the hashed kernel symbols need to have their types included in the hash?
I think the hash should be the whole mangled name. So yes, for Rust symbols they'd have the the type information.
Quoted 1 line
Or is this just a hash to make the names a managable size?
No, if that was the only point of it, the "just use two bytes for length" would be simpler.
But I don't think we want to do run-time de-mangling of names in the kernel when doing stack traces, which implies that the kallsym information in the kernel should be the de-mangled names.
That makes the names nice and readable, and also makes this "maximum symbol length" thing a non-issue.
BUT.
It also means that you can't use those names for linking purposes, so you'd then need to have a "full version" for that. But as Willy pointed out elsewhere, you might as well just use a constant-sized hash for that, rather than have both a manged and a de-mangled name.
Linus
Re: [PATCH 02/17] kallsyms: increase maximum kernel symbol length to 512
Quoted 7 lines · Linus Torvalds
Quoted 3 lines · Eric W. Biederman
Are you thinking the hashed kernel symbols need to have their types included in the hash?
I think the hash should be the whole mangled name. So yes, for Rust symbols they'd have the the type information.
Sorry I was meaning to ask should the C symbols that get hashed include their types in the hash as well? Because if everything is going to be hashed we need to clear definition of what we do with good old C symbols as well.
Quoted 12 lines
Quoted 1 line
Or is this just a hash to make the names a managable size?
No, if that was the only point of it, the "just use two bytes for length" would be simpler.
But I don't think we want to do run-time de-mangling of names in the kernel when doing stack traces, which implies that the kallsym information in the kernel should be the de-mangled names.
That makes the names nice and readable, and also makes this "maximum symbol length" thing a non-issue.
Quoted 6 lines
BUT.
It also means that you can't use those names for linking purposes, so you'd then need to have a "full version" for that. But as Willy pointed out elsewhere, you might as well just use a constant-sized hash for that, rather than have both a manged and a de-mangled name.
Just to clarify. We want this for the C symbols as well?
Eric
Re: [PATCH 02/17] kallsyms: increase maximum kernel symbol length to 512
Quoted 5 lines · Eric W. Biederman
Sorry I was meaning to ask should the C symbols that get hashed include their types in the hash as well? Because if everything is going to be hashed we need to clear definition of what we do with good old C symbols as well.
Oh, I guess if we wanted to be fancy, we could make the modversion information be part of the hash.
Because we already have that, and that's kind of what it is. But I wasn't actually thinking that fancy - I'd leave the C hash as just the name hash with no type info.
Unless somebody really wants to do that modversions thing. It *would* be a very natural thing to do in that "link using hashes" world view..
Linus
Re: [PATCH 02/17] kallsyms: increase maximum kernel symbol length to 512
In reply to Nick DesaulniersQuoted 8 lines · Nick Desaulniers
For C++ demangling, we have c++filt.
A quick search turned up: https://github.com/luser/rustfilt
but if the v0 mangling scheme is upcoming, I doubt a repo that hasn't been updated supports an upcoming scheme. Is there a more official equivalent?
Demangling v0 is already supported by the latest releases of binutils (2.36) / gdb (10.2), e.g.:
echo _RNvXNtCshGpAVYOtgW1_7example9my_moduleINtB2_13MyGenericTypeNtB2_6MyTypeENtB2_7MyTrait9my_method
| c++filt -i
<example::my_module::MyGenericType<example::my_module::MyType> as
example::my_module::MyTrait>::my_method
See the current status at https://github.com/rust-lang/rust/issues/60705.
(which reminds me I have to update the "Hacking" section in the Quick Start guide, because now the releases I backported the support into are not needed anymore)
Cheers,
Miguel
[email protected]
maintainerPATCH 3/17 Makefile: generate `CLANG_FLAGS` even in GCC builds
In reply to [email protected]From: Miguel Ojeda <[email protected]>
To support Rust under GCC-built kernels, we need to save the flags that would have been passed if the kernel was being compiled with Clang.
The reason is that bindgen -- the tool we use to generate Rust bindings to the C side of the kernel -- relies on libclang to parse C. Ideally:
- bindgen would support a GCC backend (requested at [1]),
- or the Clang driver would be perfectly compatible with GCC, including plugins. Unlikely, of course, but perhaps a big subset of configs may be possible to guarantee to be kept compatible nevertheless.
This is also the reason why GCC builds are very experimental and some configurations may not work (e.g. GCC_PLUGIN_RANDSTRUCT). However, we keep GCC builds working (for some example configs) in the CI to avoid diverging/regressing further, so that we are better prepared for the future when a solution might become available.
[1] https://github.com/rust-lang/rust-bindgen/issues/1949
--- Makefile | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)
Makefile +11 -6
@@ -573,18 +573,23 @@ endif # and from include/config/auto.conf.cmd to detect the compiler upgrade. CC_VERSION_TEXT = $(subst $(pound),,$(shell $(CC) --version 2>/dev/null | head -n 1)) -ifneq ($(findstring clang,$(CC_VERSION_TEXT)),) +TENTATIVE_CLANG_FLAGS := -Werror=unknown-warning-option + ifneq ($(CROSS_COMPILE),) -CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%)) +TENTATIVE_CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%)) endif ifeq ($(LLVM_IAS),1) -CLANG_FLAGS += -integrated-as +TENTATIVE_CLANG_FLAGS += -integrated-as else -CLANG_FLAGS += -no-integrated-as +TENTATIVE_CLANG_FLAGS += -no-integrated-as GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit)) -CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) +TENTATIVE_CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) endif -CLANG_FLAGS += -Werror=unknown-warning-option + +export TENTATIVE_CLANG_FLAGS + +ifneq ($(findstring clang,$(CC_VERSION_TEXT)),) +CLANG_FLAGS += $(TENTATIVE_CLANG_FLAGS) KBUILD_CFLAGS += $(CLANG_FLAGS) KBUILD_AFLAGS += $(CLANG_FLAGS) export CLANG_FLAGS
Signature · 1 line
2.32.0
Re: [PATCH 03/17] Makefile: generate `CLANG_FLAGS` even in GCC builds
Quoted 25 lines · On Sun, Jul 4, 2021 at 1:28 PM <[email protected]> wrote:
From: Miguel Ojeda <[email protected]>
To support Rust under GCC-built kernels, we need to save the flags that would have been passed if the kernel was being compiled with Clang.
The reason is that
bindgen-- the tool we use to generate Rust bindings to the C side of the kernel -- relies onlibclangto parse C. Ideally:-
bindgenwould support a GCC backend (requested at [1]),- or the Clang driver would be perfectly compatible with GCC, including plugins. Unlikely, of course, but perhaps a big subset of configs may be possible to guarantee to be kept compatible nevertheless.
This is also the reason why GCC builds are very experimental and some configurations may not work (e.g.
GCC_PLUGIN_RANDSTRUCT). However, we keep GCC builds working (for some example configs) in the CI to avoid diverging/regressing further, so that we are better prepared for the future when a solution might become available.
Quoted 12 lines
Co-developed-by Alex Gaynor <[email protected]>Signed-off-by Alex Gaynor <[email protected]>Co-developed-by Geoffrey Thomas <[email protected]>Signed-off-by Geoffrey Thomas <[email protected]>Co-developed-by Finn Behrens <[email protected]>Signed-off-by Finn Behrens <[email protected]>Co-developed-by Adam Bratschi-Kaye <[email protected]>Signed-off-by Adam Bratschi-Kaye <[email protected]>Co-developed-by Wedson Almeida Filho <[email protected]>Signed-off-by Wedson Almeida Filho <[email protected]>Signed-off-by Miguel Ojeda <[email protected]>
Patch LGTM; please keep an eye on the series:
https://lore.kernel.org/lkml/[email protected]/
If that lands in kbuild before this, this patch will need to be rebased to avoid a conflict in linux-next.
So (tentatively :-P):
If the patch needs to be rebased on the series linked above, please drop my reviewed by tag and I will re-review. Perhaps putting me explicitly on Cc: in the commit message will help notify me if there are successive versions?
Quoted 41 lines
--- Makefile | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)Makefile +11
-7@@ -573,18 +573,23 @@ endif # and from include/config/auto.conf.cmd to detect the compiler upgrade. CC_VERSION_TEXT = $(subst $(pound),,$(shell $(CC) --version 2>/dev/null | head -n 1)) -ifneq ($(findstring clang,$(CC_VERSION_TEXT)),) +TENTATIVE_CLANG_FLAGS := -Werror=unknown-warning-option + ifneq ($(CROSS_COMPILE),) -CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%)) +TENTATIVE_CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%)) endif ifeq ($(LLVM_IAS),1) -CLANG_FLAGS += -integrated-as +TENTATIVE_CLANG_FLAGS += -integrated-as else -CLANG_FLAGS += -no-integrated-as +TENTATIVE_CLANG_FLAGS += -no-integrated-as GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit)) -CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) +TENTATIVE_CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) endif -CLANG_FLAGS += -Werror=unknown-warning-option + +export TENTATIVE_CLANG_FLAGS + +ifneq ($(findstring clang,$(CC_VERSION_TEXT)),) +CLANG_FLAGS += $(TENTATIVE_CLANG_FLAGS) KBUILD_CFLAGS += $(CLANG_FLAGS) KBUILD_AFLAGS += $(CLANG_FLAGS) export CLANG_FLAGS --2.32.0
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH 03/17] Makefile: generate `CLANG_FLAGS` even in GCC builds
Quoted 66 lines · Nick Desaulniers
Quoted 25 lines · On Sun, Jul 4, 2021 at 1:28 PM <[email protected]> wrote:
From: Miguel Ojeda <[email protected]>
To support Rust under GCC-built kernels, we need to save the flags that would have been passed if the kernel was being compiled with Clang.
The reason is that
bindgen-- the tool we use to generate Rust bindings to the C side of the kernel -- relies onlibclangto parse C. Ideally:-
bindgenwould support a GCC backend (requested at [1]),- or the Clang driver would be perfectly compatible with GCC, including plugins. Unlikely, of course, but perhaps a big subset of configs may be possible to guarantee to be kept compatible nevertheless.
This is also the reason why GCC builds are very experimental and some configurations may not work (e.g.
GCC_PLUGIN_RANDSTRUCT). However, we keep GCC builds working (for some example configs) in the CI to avoid diverging/regressing further, so that we are better prepared for the future when a solution might become available.Quoted 12 lines
Co-developed-by Alex Gaynor <[email protected]>Signed-off-by Alex Gaynor <[email protected]>Co-developed-by Geoffrey Thomas <[email protected]>Signed-off-by Geoffrey Thomas <[email protected]>Co-developed-by Finn Behrens <[email protected]>Signed-off-by Finn Behrens <[email protected]>Co-developed-by Adam Bratschi-Kaye <[email protected]>Signed-off-by Adam Bratschi-Kaye <[email protected]>Co-developed-by Wedson Almeida Filho <[email protected]>Signed-off-by Wedson Almeida Filho <[email protected]>Signed-off-by Miguel Ojeda <[email protected]>Patch LGTM; please keep an eye on the series:
https://lore.kernel.org/lkml/[email protected]/If that lands in kbuild before this, this patch will need to be rebased to avoid a conflict in linux-next.
So (tentatively :-P):
Reviewed-by Nick Desaulniers <[email protected]>If the patch needs to be rebased on the series linked above, please drop my reviewed by tag and I will re-review. Perhaps putting me explicitly on Cc: in the commit message will help notify me if there are successive versions?
Quoted 12 lines
--- Makefile | 17 +++++++++++------ 1 file changed, 11 insertions(+), 6 deletions(-)Makefile +0
-0@@ -573,18 +573,23 @@ endif # and from include/config/auto.conf.cmd to detect the compiler upgrade. CC_VERSION_TEXT = $(subst $(pound),,$(shell $(CC) --version 2>/dev/null | head -n 1))
Quoted 2 lines
Makefile endif · line 576
Quoted 2 lines
Makefile +1
-1-ifneq ($(findstring clang,$(CC_VERSION_TEXT)),) +TENTATIVE_CLANG_FLAGS := -Werror=unknown-warning-option
Also, consider whether BINDGEN_FLAGS would be more descriptive (and less verbose) than TENTATIVE_CLANG_FLAGS.
Quoted 32 lines
Quoted 25 lines
Makefile +10
-6+ ifneq ($(CROSS_COMPILE),) -CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%)) +TENTATIVE_CLANG_FLAGS += --target=$(notdir $(CROSS_COMPILE:%-=%)) endif ifeq ($(LLVM_IAS),1) -CLANG_FLAGS += -integrated-as +TENTATIVE_CLANG_FLAGS += -integrated-as else -CLANG_FLAGS += -no-integrated-as +TENTATIVE_CLANG_FLAGS += -no-integrated-as GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit)) -CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) +TENTATIVE_CLANG_FLAGS += --prefix=$(GCC_TOOLCHAIN_DIR)$(notdir $(CROSS_COMPILE)) endif -CLANG_FLAGS += -Werror=unknown-warning-option + +export TENTATIVE_CLANG_FLAGS + +ifneq ($(findstring clang,$(CC_VERSION_TEXT)),) +CLANG_FLAGS += $(TENTATIVE_CLANG_FLAGS) KBUILD_CFLAGS += $(CLANG_FLAGS) KBUILD_AFLAGS += $(CLANG_FLAGS) export CLANG_FLAGS --Quoted 2 lines
2.32.0
--
Thanks,
~Nick Desaulniers
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH 03/17] Makefile: generate `CLANG_FLAGS` even in GCC builds
In reply to Nick DesaulniersQuoted 9 lines · Nick Desaulniers
Patch LGTM; please keep an eye on the series:
https://lore.kernel.org/lkml/[email protected]/If that lands in kbuild before this, this patch will need to be rebased to avoid a conflict in linux-next.
So (tentatively :-P):
Reviewed-by Nick Desaulniers <[email protected]>
Thanks for the heads up!
Given how fast rustc_codegen_gcc is progressing, we may need to prioritize getting
https://github.com/rust-lang/rust-bindgen/issues/1949 fixed so that we can drop this hack altogether and go for proper, full GCC support.
Quoted 4 lines
If the patch needs to be rebased on the series linked above, please drop my reviewed by tag and I will re-review. Perhaps putting me explicitly on Cc: in the commit message will help notify me if there are successive versions?
Ack -- will do that.
Cheers,
Miguel
[email protected]
maintainerPATCH 4/17 vsprintf: add new `%pA` format specifier
In reply to [email protected]From: Miguel Ojeda <[email protected]>
This patch adds a format specifier %pA to vsprintf which formats a pointer as core::fmt::Arguments. Doing so allows us to directly format to the internal buffer of printf, so we do not have to use a temporary buffer on the stack to pre-assemble the message on the Rust side.
This specifier is intended only to be used from Rust and not for C, so checkpatch.pl is intentionally unchanged to catch any misuse.
--- lib/vsprintf.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)
lib/vsprintf.c +12 -0
@@ -2182,6 +2182,10 @@ char *fwnode_string(char *buf, char *end, struct fwnode_handle *fwnode,return widen_string (buf, buf - buf_start, end, spec); } +#ifdef CONFIG_RUST +char *rust_fmt_argument (char * buf,char * end,void *ptr); +#endif +/* Disable pointer hashing if requested */ bool no_hash_pointers__ro_after_init ;EXPORT_SYMBOL_GPL (no_hash_pointers);
@@ -2335,6 +2339,10 @@ early_param("no_hash_pointers", no_hash_pointers_enable);* * Note: The default behaviour (unadorned %p) is to hash the address, * rendering it useful as a unique identifier. +* +* There is also a '%pA' format specifier, but it is only intended to be used +* from Rust code to format core::fmt::Arguments. Do *not* use it from C. +* See rust/kernel/print.rs for details. */ static noinline_for_stack char *pointer(const char *fmt, char *buf, char *end, void *ptr,
@@ -2407,6 +2415,10 @@ char *pointer(const char *fmt, char *buf, char *end, void *ptr,return device_node_string (buf, end, ptr, spec, fmt +1 );case 'f' :return fwnode_string (buf, end, ptr, spec, fmt +1 ); +#ifdef CONFIG_RUST +case 'A' : +return rust_fmt_argument (buf, end, ptr); +#endif case 'x' :return pointer_string (buf, end, ptr, spec);case 'e' :
Signature · 1 line
2.32.0
Re: [PATCH 04/17] vsprintf: add new `%pA` format specifier
Quoted 52 lines · On Sun, Jul 4, 2021 at 1:29 PM <[email protected]> wrote:
From: Miguel Ojeda <[email protected]>
This patch adds a format specifier
%pAtovsprintfwhich formats a pointer ascore::fmt::Arguments. Doing so allows us to directly format to the internal buffer ofprintf, so we do not have to use a temporary buffer on the stack to pre-assemble the message on the Rust side.This specifier is intended only to be used from Rust and not for C, so
checkpatch.plis intentionally unchanged to catch any misuse.Co-developed-by Alex Gaynor <[email protected]>Signed-off-by Alex Gaynor <[email protected]>Co-developed-by Geoffrey Thomas <[email protected]>Signed-off-by Geoffrey Thomas <[email protected]>Co-developed-by Finn Behrens <[email protected]>Signed-off-by Finn Behrens <[email protected]>Co-developed-by Adam Bratschi-Kaye <[email protected]>Signed-off-by Adam Bratschi-Kaye <[email protected]>Co-developed-by Wedson Almeida Filho <[email protected]>Signed-off-by Wedson Almeida Filho <[email protected]>Co-developed-by Boqun Feng <[email protected]>Signed-off-by Boqun Feng <[email protected]>Co-developed-by Sumera Priyadarsini <[email protected]>Signed-off-by Sumera Priyadarsini <[email protected]>Co-developed-by Michael Ellerman <[email protected]>Signed-off-by Michael Ellerman <[email protected]>Co-developed-by Sven Van Asbroeck <[email protected]>Signed-off-by Sven Van Asbroeck <[email protected]>Co-developed-by Gary Guo <[email protected]>Signed-off-by Gary Guo <[email protected]>Co-developed-by Boris-Chengbiao Zhou <[email protected]>Signed-off-by Boris-Chengbiao Zhou <[email protected]>Co-developed-by Fox Chen <[email protected]>Signed-off-by Fox Chen <[email protected]>Co-developed-by Ayaan Zaidi <[email protected]>Signed-off-by Ayaan Zaidi <[email protected]>Co-developed-by Douglas Su <[email protected]>Signed-off-by Douglas Su <[email protected]>Co-developed-by Yuki Okushi <[email protected]>Signed-off-by Yuki Okushi <[email protected]>Signed-off-by Miguel Ojeda <[email protected]>--- lib/vsprintf.c | 12 ++++++++++++ 1 file changed, 12 insertions(+)lib/vsprintf.c +0
-0
lib/vsprintf.c char *fwnode_string(char *buf, char *end, struct fwnode_handle *fwnode, · line 2186
Quoted 5 lines
lib/vsprintf.c +2
-0@@ -2182,6 +2182,10 @@ char *fwnode_string(char *buf, char *end, struct fwnode_handle *fwnode,return widen_string (buf, buf - buf_start, end, spec); } +#ifdef CONFIG_RUST +char *rust_fmt_argument (char * buf,char * end,void *ptr);
Which patch in the series adds the definition of rust_fmt_argument? Sorry, I haven't looked through the entire series yet, but I don't think it was an earlier patch in the series. If it's later in the series, you may want to rebase this to be after (or combine it with the patch that provides the definition). For instance, let's say the first half of this series was accepted/merged, but not the latter half. It would be weird to provide such definitions/calls to undefined symbols.
Quoted 28 lines
lib/vsprintf.c +10
-1+#endif +/* Disable pointer hashing if requested */ bool no_hash_pointers__ro_after_init ;EXPORT_SYMBOL_GPL (no_hash_pointers);@@ -2335,6 +2339,10 @@ early_param("no_hash_pointers", no_hash_pointers_enable);* * Note: The default behaviour (unadorned %p) is to hash the address, * rendering it useful as a unique identifier. +* +* There is also a '%pA' format specifier, but it is only intended to be used +* from Rust code to format core::fmt::Arguments. Do *not* use it from C. +* See rust/kernel/print.rs for details. */ static noinline_for_stack char *pointer(const char *fmt, char *buf, char *end, void *ptr,@@ -2407,6 +2415,10 @@ char *pointer(const char *fmt, char *buf, char *end, void *ptr,return device_node_string (buf, end, ptr, spec, fmt +1 );case 'f' :return fwnode_string (buf, end, ptr, spec, fmt +1 ); +#ifdef CONFIG_RUST +case 'A' : +return rust_fmt_argument (buf, end, ptr); +#endif case 'x' :return pointer_string (buf, end, ptr, spec);case 'e' : --
Quoted 2 lines
2.32.0
--
Thanks,
~Nick Desaulniers
Re: [PATCH 04/17] vsprintf: add new `%pA` format specifier
Quoted 9 lines · Nick Desaulniers
Which patch in the series adds the definition of
rust_fmt_argument? Sorry, I haven't looked through the entire series yet, but I don't think it was an earlier patch in the series. If it's later in the series, you may want to rebase this to be after (or combine it with the patch that provides the definition). For instance, let's say the first half of this series was accepted/merged, but not the latter half. It would be weird to provide such definitions/calls to undefined symbols.
It is in https://lore.kernel.org/lkml/[email protected]/#Z30rust:kernel:print.rs
Yeah, perhaps it would have been better to put it in the kernel crate commit alongside include/linux/spinlock.h and kernel/printk/printk.c.
On the other hand, having C changes on other commits may be easier to read and explain (note that compilation still works, given things are only enabled near the end in the Kbuild commit).
Cheers,
Miguel
[email protected]
maintainerPATCH 5/17 rust: add C helpers
In reply to [email protected]From: Miguel Ojeda <[email protected]>
This source file contains forwarders to C macros and inlined functions.
--- rust/helpers.c | 235 +++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 235 insertions(+) create mode 100644 rust/helpers.c
rust/helpers.c new file +235 -0
@@ -0,0 +1,235 @@ +// SPDX-License-Identifier: GPL-2.0 + +#include <linux/bug.h> +#include <linux/build_bug.h> +#include <linux/uaccess.h> +#include <linux/sched/signal.h> +#include <linux/gfp.h> +#include <linux/highmem.h> +#include <linux/uio.h> +#include <linux/errname.h> +#include <linux/mutex.h> +#include <linux/platform_device.h> +#include <linux/security.h> + +void rust_helper_BUG (void ) +{ +BUG (); +} + +unsigned long rust_helper_copy_from_user (void *to,const void __user *from,unsigned long n) +{ +return copy_from_user (to, from, n); +} + +unsigned long rust_helper_copy_to_user (void __user *to,const void *from,unsigned long n) +{ +return copy_to_user (to, from, n); +} + +unsigned long rust_helper_clear_user (void __user *to,unsigned long n) +{ +return clear_user (to, n); +} + +void rust_helper_spin_lock_init (spinlock_t *lock,const char *name, +struct lock_class_key *key) +{ +#ifdef CONFIG_DEBUG_SPINLOCK +__spin_lock_init (lock, name, key); +#else +spin_lock_init (lock); +#endif +} +EXPORT_SYMBOL_GPL (rust_helper_spin_lock_init); + +void rust_helper_spin_lock (spinlock_t *lock) +{ +spin_lock (lock); +} +EXPORT_SYMBOL_GPL (rust_helper_spin_lock); + +void rust_helper_spin_unlock (spinlock_t *lock) +{ +spin_unlock (lock); +} +EXPORT_SYMBOL_GPL (rust_helper_spin_unlock); + +void rust_helper_init_wait (struct wait_queue_entry *wq_entry) +{ +init_wait (wq_entry); +} +EXPORT_SYMBOL_GPL (rust_helper_init_wait); + +int rust_helper_signal_pending (struct task_struct *t) +{ +return signal_pending (t); +} +EXPORT_SYMBOL_GPL (rust_helper_signal_pending); + +struct page *rust_helper_alloc_pages (gfp_t gfp_mask,unsigned int order) +{ +return alloc_pages (gfp_mask, order); +} +EXPORT_SYMBOL_GPL (rust_helper_alloc_pages); + +void *rust_helper_kmap (struct page *page) +{ +return kmap (page); +} +EXPORT_SYMBOL_GPL (rust_helper_kmap); + +void rust_helper_kunmap (struct page *page) +{ +return kunmap (page); +} +EXPORT_SYMBOL_GPL (rust_helper_kunmap); + +int rust_helper_cond_resched (void ) +{ +return cond_resched (); +} +EXPORT_SYMBOL_GPL (rust_helper_cond_resched); + +size_t rust_helper_copy_from_iter (void *addr,size_t bytes,struct iov_iter *i) +{ +return copy_from_iter (addr, bytes, i); +} +EXPORT_SYMBOL_GPL (rust_helper_copy_from_iter); + +size_t rust_helper_copy_to_iter (const void *addr,size_t bytes,struct iov_iter *i) +{ +return copy_to_iter (addr, bytes, i); +} +EXPORT_SYMBOL_GPL (rust_helper_copy_to_iter); + +bool rust_helper_is_err (__force const void *ptr) +{ +return IS_ERR (ptr); +} +EXPORT_SYMBOL_GPL (rust_helper_is_err); + +long rust_helper_ptr_err (__force const void *ptr) +{ +return PTR_ERR (ptr); +} +EXPORT_SYMBOL_GPL (rust_helper_ptr_err); + +const char *rust_helper_errname (int err) +{ +return errname (err); +} + +void rust_helper_mutex_lock (struct mutex *lock) +{ +mutex_lock (lock); +} +EXPORT_SYMBOL_GPL (rust_helper_mutex_lock); + +void * +rust_helper_platform_get_drvdata (const struct platform_device *pdev) +{ +return platform_get_drvdata (pdev); +} +EXPORT_SYMBOL_GPL (rust_helper_platform_get_drvdata); + +void +rust_helper_platform_set_drvdata (struct platform_device *pdev, +void *data) +{ +return platform_set_drvdata (pdev, data); +} +EXPORT_SYMBOL_GPL (rust_helper_platform_set_drvdata); + +refcount_t rust_helper_refcount_new (void ) +{ +return (refcount_t )REFCOUNT_INIT (1 ); +} +EXPORT_SYMBOL_GPL (rust_helper_refcount_new); + +void rust_helper_refcount_inc (refcount_t *r) +{ +refcount_inc (r); +} +EXPORT_SYMBOL_GPL (rust_helper_refcount_inc); + +bool rust_helper_refcount_dec_and_test (refcount_t *r) +{ +return refcount_dec_and_test (r); +} +EXPORT_SYMBOL_GPL (rust_helper_refcount_dec_and_test); + +void rust_helper_rb_link_node (struct rb_node *node,struct rb_node *parent, +struct rb_node **rb_link) +{ +rb_link_node (node, parent, rb_link); +} +EXPORT_SYMBOL_GPL (rust_helper_rb_link_node); + +struct task_struct *rust_helper_get_current (void ) +{ +return current; +} +EXPORT_SYMBOL_GPL (rust_helper_get_current); + +void rust_helper_get_task_struct (struct task_struct * t) +{ +get_task_struct (t); +} +EXPORT_SYMBOL_GPL (rust_helper_get_task_struct); + +void rust_helper_put_task_struct (struct task_struct * t) +{ +put_task_struct (t); +} +EXPORT_SYMBOL_GPL (rust_helper_put_task_struct); + +int rust_helper_security_binder_set_context_mgr (struct task_struct *mgr) +{ +return security_binder_set_context_mgr (mgr); +} +EXPORT_SYMBOL_GPL (rust_helper_security_binder_set_context_mgr); + +int rust_helper_security_binder_transaction (struct task_struct *from, +struct task_struct *to) +{ +return security_binder_transaction (from, to); +} +EXPORT_SYMBOL_GPL (rust_helper_security_binder_transaction); + +int rust_helper_security_binder_transfer_binder (struct task_struct *from, +struct task_struct *to) +{ +return security_binder_transfer_binder (from, to); +} +EXPORT_SYMBOL_GPL (rust_helper_security_binder_transfer_binder); + +int rust_helper_security_binder_transfer_file (struct task_struct *from, +struct task_struct *to, +struct file *file) +{ +return security_binder_transfer_file (from, to, file); +} +EXPORT_SYMBOL_GPL (rust_helper_security_binder_transfer_file); + +/* We use bindgen's --size_t-is-usize option to bind the C size_t type +* as the Rust usize type, so we can use it in contexts where Rust +* expects a usize like slice (array) indices. usize is defined to be +* the same as C's uintptr_t type (can hold any pointer) but not +* necessarily the same as size_t (can hold the size of any single +* object). Most modern platforms use the same concrete integer type for +* both of them, but in case we find ourselves on a platform where +* that's not true, fail early instead of risking ABI or +* integer-overflow issues. +* +* If your platform fails this assertion, it means that you are in +* danger of integer-overflow bugs (even if you attempt to remove +* --size_t-is-usize). It may be easiest to change the kernel ABI on +* your platform such that size_t matches uintptr_t (i.e., to increase +* size_t, because uintptr_t has to be at least as big as size_t). +*/ +static_assert ( +sizeof (size_t ) ==sizeof (uintptr_t ) && + __alignof__(size_t ) == __alignof__(uintptr_t ), +"Rust code expects C size_t to match Rust usize" +);
Signature · 1 line
2.32.0
Re: [PATCH 05/17] rust: add C helpers
Quoted 4 lines · On Sun, Jul 04, 2021 at 10:27PM +0200, [email protected] wrote:
From: Miguel Ojeda <[email protected]>
This source file contains forwarders to C macros and inlined functions.
What is the story with Rust and LTO? Intuitively, I would expect Rust code to only perform optimally if the kernel is built with LTO (currently only supported via Clang).
Because if calls to every one of these helpers are real calls, I would expect performance to be pretty poor. There's probably a reason these are macros or inlinable functions.
I would almost go so far and suggest that CONFIG_RUST be modified as follows:
init/Kconfig +1 -0
@@ -2028,6 +2028,7 @@ config RUST depends on HAS_RUST depends on !COMPILE_TEST depends on !MODVERSIONS + depends on LTO || EXPERT default n help Enables Rust support in the kernel.
[ I'm sure there are configs that don't yet work with LTO, but could be
useful to enable for debugging or testing purposes, and therefore would make it conditional on CONFIG_EXPERT as well. ]
[...]
Review
Quoted 5 lines
(quoted diff) +5
-0+unsigned long rust_helper_copy_from_user(void *to, const void __user *from, unsigned long n) +{ + return copy_from_user(to, from, n); +} +
[...]
From some local tests, it looks like simply attaching __attribute__((always_inline)) will do what one would expect when compiling with Clang LTO (I checked -flto=thin).
If you confirm this also works across C and Rust TUs when enabling LTO, I would then suggested adding __attribute__((always_inline)) to all these helpers.
Thanks,
-- Marco
Re: [PATCH 05/17] rust: add C helpers
Quoted 9 lines · Marco Elver
Quoted 4 lines · On Sun, Jul 04, 2021 at 10:27PM +0200, [email protected] wrote:
From: Miguel Ojeda <[email protected]>
This source file contains forwarders to C macros and inlined functions.
What is the story with Rust and LTO? Intuitively, I would expect Rust code to only perform optimally if the kernel is built with LTO (currently only supported via Clang).
I'll answer my own question: it looks like Linux Rust code currently does _not_ generate LLVM-LTO compatible object files, but only native object files (which still link fine if LTO is enabled, but doesn't permit the optimizations below we'd want).
rustc already supports playing nicely with LLVM LTO via -C linker-plugin-lto:
https://doc.rust-lang.org/rustc/linker-plugin-lto.html
So, hopefully it should only require kernel work to make it play nicely with CONFIG_LTO_CLANG.
Quoted 40 lines
Because if calls to every one of these helpers are real calls, I would expect performance to be pretty poor. There's probably a reason these are macros or inlinable functions.
I would almost go so far and suggest that
CONFIG_RUSTbe modified as follows:init/Kconfig +1
-0@@ -2028,6 +2028,7 @@ config RUST depends on HAS_RUST depends on !COMPILE_TEST depends on !MODVERSIONS + depends on LTO || EXPERT default n help Enables Rust support in the kernel.[ I'm sure there are configs that don't yet work with LTO, but could be
useful to enable for debugging or testing purposes, and therefore would make it conditional onCONFIG_EXPERTas well. ][...]
Review
Quoted 5 lines
(quoted diff) +5
-0+unsigned long rust_helper_copy_from_user(void *to, const void __user *from, unsigned long n) +{ + return copy_from_user(to, from, n); +} +[...]
From some local tests, it looks like simply attaching __attribute__((always_inline)) will do what one would expect when compiling with Clang LTO (I checked -flto=thin).
If you confirm this also works across C and Rust TUs when enabling LTO, I would then suggested adding __attribute__((always_inline)) to all these helpers.
Thanks,
-- Marco
Re: [PATCH 05/17] rust: add C helpers
In reply to Marco ElverQuoted 4 lines · Marco Elver' via Clang Built Linux
What is the story with Rust and LTO? Intuitively, I would expect Rust code to only perform optimally if the kernel is built with LTO (currently only supported via Clang).
Cross-language LTO is possible and is used in other major projects. For instance, Firefox seems to use it:
https://bugzilla.mozilla.org/show_bug.cgi?id=1486042
However, we have not taken a look at it yet for the kernel.
Quoted 3 lines
Because if calls to every one of these helpers are real calls, I would expect performance to be pretty poor. There's probably a reason these are macros or inlinable functions.
Maybe -- we need to measure.
Quoted 2 lines
I would almost go so far and suggest that
CONFIG_RUSTbe modified as follows:
Even if the performance was unacceptable, there are other ways than LTO to solve this, such as implementing the equivalent macro/function on Rust's side.
Furthermore, we need to consider GCC too.
Thus I would not be premature in making it depend on LTO.
Cheers,
Miguel
[email protected]
maintainerPATCH 6/17 rust: add `compiler_builtins` crate
In reply to [email protected]From: Miguel Ojeda <[email protected]>
Rust provides compiler_builtins as a port of LLVM's compiler-rt. Since we do not need the vast majority of them, we avoid the dependency by providing our own crate.
--- rust/compiler_builtins.rs | 146 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 146 insertions(+) create mode 100644 rust/compiler_builtins.rs
rust/compiler_builtins.rs new file +146 -0
@@ -0,0 +1,146 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Our own `compiler_builtins`. +//! +//! Rust provides [`compiler_builtins`] as a port of LLVM's [`compiler-rt`]. +//! Since we do not need the vast majority of them, we avoid the dependency +//! by providing this file. +//! +//! At the moment, some builtins are required that should not be. For instance, +//! [`core`] has floating-point functionality which we should not be compiling +//! in. We will work with upstream [`core`] to provide feature flags to disable +//! the parts we do not need. For the moment, we define them to [`panic!`] at +//! runtime for simplicity to catch mistakes, instead of performing surgery +//! on `core.o`. +//! +//! In any case, all these symbols are weakened to ensure we do not override +//! those that may be provided by the rest of the kernel. +//! +//! [`compiler_builtins`]: https://github.com/rust-lang/compiler-builtins +//! [`compiler-rt`]: https://compiler-rt.llvm.org/ + +#![feature(compiler_builtins)] +#![compiler_builtins] +#![no_builtins] +#![no_std] + +macro_rules! define_panicking_intrinsics( + ($reason: tt, { $($ident: ident, )* }) => { + $( +#[doc(hidden)] +#[no_mangle] +pub extern"C" fn $ident() { +panic! ($reason); + } + )* + } +); + +define_panicking_intrinsics! ("`f32` should not be used" , { + __addsf3, + __addsf3vfp, + __aeabi_fcmpeq, + __aeabi_ul2f, + __divsf3, + __divsf3vfp, + __eqsf2, + __eqsf2vfp, + __fixsfdi, + __fixsfsi, + __fixsfti, + __fixunssfdi, + __fixunssfsi, + __fixunssfti, + __floatdisf, + __floatsisf, + __floattisf, + __floatundisf, + __floatunsisf, + __floatuntisf, + __gesf2, + __gesf2vfp, + __gtsf2, + __gtsf2vfp, + __lesf2, + __lesf2vfp, + __ltsf2, + __ltsf2vfp, + __mulsf3, + __mulsf3vfp, + __nesf2, + __nesf2vfp, + __powisf2, + __subsf3, + __subsf3vfp, + __unordsf2, +}); + +define_panicking_intrinsics! ("`f64` should not be used" , { + __adddf3, + __adddf3vfp, + __aeabi_dcmpeq, + __aeabi_ul2d, + __divdf3, + __divdf3vfp, + __eqdf2, + __eqdf2vfp, + __fixdfdi, + __fixdfsi, + __fixdfti, + __fixunsdfdi, + __fixunsdfsi, + __fixunsdfti, + __floatdidf, + __floatsidf, + __floattidf, + __floatundidf, + __floatunsidf, + __floatuntidf, + __gedf2, + __gedf2vfp, + __gtdf2, + __gtdf2vfp, + __ledf2, + __ledf2vfp, + __ltdf2, + __ltdf2vfp, + __muldf3, + __muldf3vfp, + __nedf2, + __nedf2vfp, + __powidf2, + __subdf3, + __subdf3vfp, + __unorddf2, +}); + +define_panicking_intrinsics! ("`i128` should not be used" , { + __ashrti3, + __muloti4, + __multi3, +}); + +define_panicking_intrinsics! ("`u128` should not be used" , { + __ashlti3, + __lshrti3, + __udivmodti4, + __udivti3, + __umodti3, +}); + +#[cfg(target_arch = "arm" )] +define_panicking_intrinsics! ("`u64` division/modulo should not be used" , { + __aeabi_uldivmod, + __mulodi4, +}); + +extern "C" { +fn rust_helper_BUG () -> !; +} + +#[panic_handler] +fn panic (_info: &core::panic::PanicInfo <'_>) -> ! { +unsafe { +rust_helper_BUG (); + } +}
Signature · 1 line
2.32.0
[email protected]
maintainerPATCH 7/17 rust: add `alloc` crate
In reply to [email protected]From: Miguel Ojeda <[email protected]>
This crate is a subset of the Rust standard library alloc, with some additions on top.
This is needed because upstream support for fallible allocations is a work in progress (i.e. the try_* versions of methods which return a Result instead of panicking).
Having the library in-tree also gives us a bit more freedom to experiment with new interfaces and allows us to iterate quickly.
Eventually, the goal is to have everything the kernel needs in upstream alloc and drop it from the kernel tree.
--- rust/alloc/README.md | 32 + rust/alloc/alloc.rs | 425 ++++ rust/alloc/borrow.rs | 493 +++++ rust/alloc/boxed.rs | 1728 +++++++++++++++ rust/alloc/collections/mod.rs | 116 + rust/alloc/fmt.rs | 587 ++++++ rust/alloc/lib.rs | 197 ++ rust/alloc/macros.rs | 128 ++ rust/alloc/prelude/mod.rs | 17 + rust/alloc/prelude/v1.rs | 16 + rust/alloc/raw_vec.rs | 612 ++++++ rust/alloc/rc.rs | 2539 ++++++++++++++++++++++ rust/alloc/slice.rs | 1271 +++++++++++ rust/alloc/str.rs | 614 ++++++ rust/alloc/string.rs | 2847 +++++++++++++++++++++++++ rust/alloc/sync.rs | 2631 +++++++++++++++++++++++ rust/alloc/vec/drain.rs | 157 ++ rust/alloc/vec/drain_filter.rs | 145 ++ rust/alloc/vec/into_iter.rs | 296 +++ rust/alloc/vec/is_zero.rs | 106 + rust/alloc/vec/mod.rs | 3255 +++++++++++++++++++++++++++++ rust/alloc/vec/partial_eq.rs | 49 + rust/alloc/vec/set_len_on_drop.rs | 30 + rust/alloc/vec/spec_extend.rs | 170 ++ 24 files changed, 18461 insertions(+) create mode 100644 rust/alloc/README.md create mode 100644 rust/alloc/alloc.rs create mode 100644 rust/alloc/borrow.rs create mode 100644 rust/alloc/boxed.rs create mode 100644 rust/alloc/collections/mod.rs create mode 100644 rust/alloc/fmt.rs create mode 100644 rust/alloc/lib.rs create mode 100644 rust/alloc/macros.rs create mode 100644 rust/alloc/prelude/mod.rs create mode 100644 rust/alloc/prelude/v1.rs create mode 100644 rust/alloc/raw_vec.rs create mode 100644 rust/alloc/rc.rs create mode 100644 rust/alloc/slice.rs create mode 100644 rust/alloc/str.rs create mode 100644 rust/alloc/string.rs create mode 100644 rust/alloc/sync.rs create mode 100644 rust/alloc/vec/drain.rs create mode 100644 rust/alloc/vec/drain_filter.rs create mode 100644 rust/alloc/vec/into_iter.rs create mode 100644 rust/alloc/vec/is_zero.rs create mode 100644 rust/alloc/vec/mod.rs create mode 100644 rust/alloc/vec/partial_eq.rs create mode 100644 rust/alloc/vec/set_len_on_drop.rs create mode 100644 rust/alloc/vec/spec_extend.rs
rust/alloc/README.md new file +32 -0
@@ -0,0 +1,32 @@ +# `alloc` + +These source files come from the Rust standard library, hosted in +the https://github.com/rust-lang/rust repository. For copyright +details, see https://github.com/rust-lang/rust/blob/master/COPYRIGHT. + +Please note that these files should be kept as close as possible to +upstream. In general, only additions should be performed (e.g. new +methods). Eventually, changes should make it into upstream so that, +at some point, this fork can be dropped from the kernel tree. + + +## Rationale + +On one hand, kernel folks wanted to keep `alloc` in-tree to have more +freedom in both workflow and actual features if actually needed +(e.g. receiver types if we ended up using them), which is reasonable. + +On the other hand, Rust folks wanted to keep `alloc` as close as +upstream as possible and avoid as much divergence as possible, which +is also reasonable. + +We agreed on a middle-ground: we would keep a subset of `alloc` +in-tree that would be as small and as close as possible to upstream. +Then, upstream can start adding the functions that we add to `alloc` +etc., until we reach a point where the kernel already knows exactly +what it needs in `alloc` and all the new methods are merged into +upstream, so that we can drop `alloc` from the kernel tree and go back +to using the upstream one. + +By doing this, the kernel can go a bit faster now, and Rust can +slowly incorporate and discuss the changes as needed.
rust/alloc/alloc.rs new file +425 -0
425 lines not on this page · raw message
rust/alloc/borrow.rs new file +493 -0
493 lines not on this page · raw message
rust/alloc/boxed.rs new file +1728 -0
1,728 lines not on this page · raw message
rust/alloc/collections/mod.rs new file +116 -0
116 lines not on this page · raw message
rust/alloc/fmt.rs new file +587 -0
206 lines not on this page · raw message
rust/alloc/lib.rs new file +197 -0
197 lines not on this page · raw message
rust/alloc/macros.rs new file +128 -0
128 lines not on this page · raw message
rust/alloc/raw_vec.rs new file +612 -0
612 lines not on this page · raw message
rust/alloc/rc.rs new file +2539 -0
2,539 lines not on this page · raw message
rust/alloc/slice.rs new file +1271 -0
1,271 lines not on this page · raw message
rust/alloc/str.rs new file +614 -0
614 lines not on this page · raw message
rust/alloc/string.rs new file +2847 -0
2,847 lines not on this page · raw message
rust/alloc/sync.rs new file +2631 -0
2,631 lines not on this page · raw message
rust/alloc/vec/drain.rs new file +157 -0
157 lines not on this page · raw message
rust/alloc/vec/drain_filter.rs new file +145 -0
145 lines not on this page · raw message
rust/alloc/vec/into_iter.rs new file +296 -0
296 lines not on this page · raw message
rust/alloc/vec/is_zero.rs new file +106 -0
106 lines not on this page · raw message
rust/alloc/vec/mod.rs new file +3255 -0
3,255 lines not on this page · raw message
rust/alloc/vec/partial_eq.rs new file +49 -0
49 lines not on this page · raw message
rust/alloc/vec/set_len_on_drop.rs new file +30 -0
Lines of this file are not shown. View the raw message.
rust/alloc/vec/spec_extend.rs new file +170 -0
170 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.32.0
Part of the diff is not shown here. View the raw message.
[email protected]
maintainerPATCH 8/17 rust: add `build_error` crate
In reply to [email protected]From: Miguel Ojeda <[email protected]>
The build_error crate provides the build_error function which is then used to provide the build_error! and the build_assert! macros.
build_assert! is intended to be used when static_assert! cannot be used, e.g. when the condition refers to generic parameters or parameters of an inline function.
--- rust/build_error.rs | 33 +++++++++++++++++++++++++++++++++ 1 file changed, 33 insertions(+) create mode 100644 rust/build_error.rs
rust/build_error.rs new file +33 -0
@@ -0,0 +1,33 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Build-time error. +//! +//! This crate provides a function `build_error`, which will panic in +//! compile-time if executed in const context, and will cause a build error +//! if not executed at compile time and the optimizer does not optimise away the +//! call. +//! +//! It is used by `build_assert!` in the kernel crate, allowing checking of +//! conditions that could be checked statically, but could not be enforced in +//! Rust yet (e.g. perform some checks in const functions, but those +//! functions could still be called in the runtime). + +#![no_std] +#![feature(const_panic, core_panic)] + +/// Panics if executed in const context, or triggers a build error if not. +#[inline(never)] +#[cold] +#[no_mangle] +#[track_caller] +pub const fn build_error (msg: &'staticstr ) -> ! { +// Could also be `panic!(msg)` to avoid using unstable feature `core_panic`, +// but it is not allowed in Rust 2021, while `panic!("{}", msg)` could not +// yet be used in const context. + core::panicking::panic (msg); +} + +#[cfg( CONFIG_RUST_BUILD_ASSERT_WARN )] +#[link_section = ".gnu.warning.build_error" ] +#[used] +static BUILD_ERROR_WARNING : [u8 ;45 ] = *b"call to build_error present after compilation" ;
Signature · 1 line
2.32.0
[email protected]
maintainerPATCH 9/17 rust: add `macros` crate
In reply to [email protected]From: Miguel Ojeda <[email protected]>
This crate contains all the procedural macros ("proc macros") shared by all the kernel.
Procedural macros allow to create syntax extensions. They run at compile-time and can consume as well as produce Rust syntax.
For instance, the module! macro that is used by Rust modules is implemented here. It allows to easily declare the equivalent information to the MODULE_* macros in C modules, e.g.:
module! {
type: RustMinimal,
name: b"rust_minimal",
author: b"Rust for Linux Contributors",
description: b"Rust minimal sample",
license: b"GPL v2",
}--- rust/macros/lib.rs | 127 +++++++ rust/macros/module.rs | 754 ++++++++++++++++++++++++++++++++++++++++++ 2 files changed, 881 insertions(+) create mode 100644 rust/macros/lib.rs create mode 100644 rust/macros/module.rs
rust/macros/lib.rs new file +127 -0
@@ -0,0 +1,127 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Crate for all kernel procedural macros. + +mod module; + +use proc_macro::TokenStream ; + +/// Declares a kernel module. +/// +/// The `type` argument should be a type which implements the [`KernelModule`] +/// trait. Also accepts various forms of kernel metadata. +/// +/// C header: [`include/linux/moduleparam.h`](../../../include/linux/moduleparam.h) +/// +/// [`KernelModule`]: ../kernel/trait.KernelModule.html +/// +/// # Examples +/// +/// ```ignore +/// use kernel::prelude::*; +/// +/// module!{ +/// type: MyKernelModule, +/// name: b"my_kernel_module", +/// author: b"Rust for Linux Contributors", +/// description: b"My very own kernel module!", +/// license: b"GPL v2", +/// params: { +/// my_i32: i32 { +/// default: 42, +/// permissions: 0o000, +/// description: b"Example of i32", +/// }, +/// writeable_i32: i32 { +/// default: 42, +/// permissions: 0o644, +/// description: b"Example of i32", +/// }, +/// }, +/// } +/// +/// struct MyKernelModule; +/// +/// impl KernelModule for MyKernelModule { +/// fn init() -> Result<Self> { +/// // If the parameter is writeable, then the kparam lock must be +/// // taken to read the parameter: +/// { +/// let lock = THIS_MODULE.kernel_param_lock(); +/// pr_info!("i32 param is: {}\n", writeable_i32.read(&lock)); +/// } +/// // If the parameter is read only, it can be read without locking +/// // the kernel parameters: +/// pr_info!("i32 param is: {}\n", my_i32.read()); +/// Ok(MyKernelModule) +/// } +/// } +/// ``` +/// +/// # Supported argument types +/// - `type`: type which implements the [`KernelModule`] trait (required). +/// - `name`: byte array of the name of the kernel module (required). +/// - `author`: byte array of the author of the kernel module. +/// - `description`: byte array of the description of the kernel module. +/// - `license`: byte array of the license of the kernel module (required). +/// - `alias`: byte array of alias name of the kernel module. +/// - `alias_rtnl_link`: byte array of the `rtnl_link_alias` of the kernel module (mutually exclusive with `alias`). +/// - `params`: parameters for the kernel module, as described below. +/// +/// # Supported parameter types +/// +/// - `bool`: Corresponds to C `bool` param type. +/// - `i8`: No equivalent C param type. +/// - `u8`: Corresponds to C `char` param type. +/// - `i16`: Corresponds to C `short` param type. +/// - `u16`: Corresponds to C `ushort` param type. +/// - `i32`: Corresponds to C `int` param type. +/// - `u32`: Corresponds to C `uint` param type. +/// - `i64`: No equivalent C param type. +/// - `u64`: Corresponds to C `ullong` param type. +/// - `isize`: No equivalent C param type. +/// - `usize`: No equivalent C param type. +/// - `str`: Corresponds to C `charp` param type. Reading returns a byte slice. +/// - `ArrayParam<T,N>`: Corresponds to C parameters created using `module_param_array`. An array +/// of `T`'s of length at **most** `N`. +/// +/// `invbool` is unsupported: it was only ever used in a few modules. +/// Consider using a `bool` and inverting the logic instead. +#[proc_macro] +pub fn module (ts:TokenStream ) ->TokenStream { + module::module (ts) +} + +/// Declares a kernel module that exposes a single misc device. +/// +/// The `type` argument should be a type which implements the [`FileOpener`] trait. Also accepts +/// various forms of kernel metadata. +/// +/// C header: [`include/linux/moduleparam.h`](../../../include/linux/moduleparam.h) +/// +/// [`FileOpener`]: ../kernel/file_operations/trait.FileOpener.html +/// +/// # Examples +/// +/// ```ignore +/// use kernel::prelude::*; +/// +/// module_misc_device! { +/// type: MyFile, +/// name: b"my_miscdev_kernel_module", +/// author: b"Rust for Linux Contributors", +/// description: b"My very own misc device kernel module!", +/// license: b"GPL v2", +/// } +/// +/// #[derive(Default)] +/// struct MyFile; +/// +/// impl kernel::file_operations::FileOperations for MyFile { +/// kernel::declare_file_operations!(); +/// } +/// ``` +#[proc_macro] +pub fn module_misc_device (ts:TokenStream ) ->TokenStream { + module::module_misc_device (ts) +}
rust/macros/module.rs new file +754 -0
754 lines not on this page · raw message
Signature · 1 line
2.32.0
[email protected]
maintainerPATCH 10/17 rust: add `kernel` crate
In reply to [email protected]From: Miguel Ojeda <[email protected]>
The kernel crate currently includes all the abstractions that wrap kernel features written in C.
These abstractions call the C side of the kernel via the generated bindings with the bindgen tool. Modules developed in Rust should never call the bindings themselves.
In the future, as the abstractions grow in number, we may need to split this crate into several, possibly following a similar subdivision in subsystems as the kernel itself.
--- include/linux/spinlock.h | 17 +- kernel/printk/printk.c | 5 +- rust/kernel/allocator.rs | 63 +++ rust/kernel/bindings.rs | 28 ++ rust/kernel/bindings_helper.h | 24 ++ rust/kernel/buffer.rs | 39 ++ rust/kernel/build_assert.rs | 80 ++++ rust/kernel/c_types.rs | 119 ++++++ rust/kernel/chrdev.rs | 212 ++++++++++ rust/kernel/error.rs | 272 +++++++++++++ rust/kernel/file.rs | 130 ++++++ rust/kernel/file_operations.rs | 698 +++++++++++++++++++++++++++++++++ rust/kernel/io_buffer.rs | 153 ++++++++ rust/kernel/iov_iter.rs | 95 +++++ rust/kernel/lib.rs | 220 +++++++++++ rust/kernel/linked_list.rs | 245 ++++++++++++ rust/kernel/miscdev.rs | 113 ++++++ rust/kernel/module_param.rs | 497 +++++++++++++++++++++++ rust/kernel/of.rs | 101 +++++ rust/kernel/pages.rs | 176 +++++++++ rust/kernel/platdev.rs | 166 ++++++++ rust/kernel/prelude.rs | 28 ++ rust/kernel/print.rs | 412 +++++++++++++++++++ rust/kernel/random.rs | 50 +++ rust/kernel/raw_list.rs | 361 +++++++++++++++++ rust/kernel/rbtree.rs | 570 +++++++++++++++++++++++++++ rust/kernel/security.rs | 79 ++++ rust/kernel/static_assert.rs | 39 ++ rust/kernel/str.rs | 259 ++++++++++++ rust/kernel/sync/arc.rs | 227 +++++++++++ rust/kernel/sync/condvar.rs | 136 +++++++ rust/kernel/sync/guard.rs | 82 ++++ rust/kernel/sync/locked_by.rs | 112 ++++++ rust/kernel/sync/mod.rs | 84 ++++ rust/kernel/sync/mutex.rs | 101 +++++ rust/kernel/sync/spinlock.rs | 109 +++++ rust/kernel/sysctl.rs | 198 ++++++++++ rust/kernel/task.rs | 193 +++++++++ rust/kernel/traits.rs | 26 ++ rust/kernel/types.rs | 249 ++++++++++++ rust/kernel/user_ptr.rs | 191 +++++++++ 41 files changed, 6952 insertions(+), 7 deletions(-) create mode 100644 rust/kernel/allocator.rs create mode 100644 rust/kernel/bindings.rs create mode 100644 rust/kernel/bindings_helper.h create mode 100644 rust/kernel/buffer.rs create mode 100644 rust/kernel/build_assert.rs create mode 100644 rust/kernel/c_types.rs create mode 100644 rust/kernel/chrdev.rs create mode 100644 rust/kernel/error.rs create mode 100644 rust/kernel/file.rs create mode 100644 rust/kernel/file_operations.rs create mode 100644 rust/kernel/io_buffer.rs create mode 100644 rust/kernel/iov_iter.rs create mode 100644 rust/kernel/lib.rs create mode 100644 rust/kernel/linked_list.rs create mode 100644 rust/kernel/miscdev.rs create mode 100644 rust/kernel/module_param.rs create mode 100644 rust/kernel/of.rs create mode 100644 rust/kernel/pages.rs create mode 100644 rust/kernel/platdev.rs create mode 100644 rust/kernel/prelude.rs create mode 100644 rust/kernel/print.rs create mode 100644 rust/kernel/random.rs create mode 100644 rust/kernel/raw_list.rs create mode 100644 rust/kernel/rbtree.rs create mode 100644 rust/kernel/security.rs create mode 100644 rust/kernel/static_assert.rs create mode 100644 rust/kernel/str.rs create mode 100644 rust/kernel/sync/arc.rs create mode 100644 rust/kernel/sync/condvar.rs create mode 100644 rust/kernel/sync/guard.rs create mode 100644 rust/kernel/sync/locked_by.rs create mode 100644 rust/kernel/sync/mod.rs create mode 100644 rust/kernel/sync/mutex.rs create mode 100644 rust/kernel/sync/spinlock.rs create mode 100644 rust/kernel/sysctl.rs create mode 100644 rust/kernel/task.rs create mode 100644 rust/kernel/traits.rs create mode 100644 rust/kernel/types.rs create mode 100644 rust/kernel/user_ptr.rs
include/linux/spinlock.h +11 -6
@@ -331,12 +331,17 @@ static __always_inline raw_spinlock_t *spinlock_check(spinlock_t *lock)#ifdef CONFIG_DEBUG_SPINLOCK -# define spin_lock_init (lock) \ -do { \ - static struct lock_class_key __key; \ - \ - __raw_spin_lock_init(spinlock_check(lock), \ - #lock, &__key, LD_WAIT_CONFIG); \ +static inline void __spin_lock_init (spinlock_t *lock,const char *name, +struct lock_class_key *key) +{ +__raw_spin_lock_init (spinlock_check (lock), name, key,LD_WAIT_CONFIG ); +} + +# define spin_lock_init (lock) \ +do { \ + static struct lock_class_key __key; \ + \ + __spin_lock_init(lock, #lock, &__key); \ } while (0)#else
kernel/printk/printk.c +4 -1
@@ -397,7 +397,10 @@ static struct latched_seq clear_seq = {/* the maximum size of a formatted record (i.e. with prefix added per line) */ #define CONSOLE_LOG_MAX 1024 -/* the maximum size allowed to be reserved for a record */ +/* +* The maximum size allowed to be reserved for a record. +* Keep in sync with rust/kernel/print.rs. +*/ #define LOG_LINE_MAX (CONSOLE_LOG_MAX - PREFIX_MAX)#define LOG_LEVEL (v) ((v) & 0x07)
rust/kernel/allocator.rs new file +63 -0
63 lines not on this page · raw message
rust/kernel/bindings.rs new file +28 -0
@@ -0,0 +1,28 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Bindings +//! +//! Imports the generated bindings by `bindgen`. + +// See https://github.com/rust-lang/rust-bindgen/issues/1651. +#![cfg_attr(test, allow(deref_nullptr))] +#![cfg_attr(test, allow(unaligned_references))] +#![cfg_attr(test, allow(unsafe_op_in_unsafe_fn))] + +#[allow( +clippy::all, +non_camel_case_types, +non_upper_case_globals, +non_snake_case, +improper_ctypes, +unsafe_op_in_unsafe_fn +)] +mod bindings_raw { +use crate ::c_types; +include! (env! ("RUST_BINDINGS_FILE" )); +} +pub use bindings_raw::*; + +pub const GFP_KERNEL :gfp_t =BINDINGS_GFP_KERNEL ; +pub const __GFP_ZERO:gfp_t =BINDINGS___GFP_ZERO ; +pub const __GFP_HIGHMEM:gfp_t = ___GFP_HIGHMEM;
rust/kernel/bindings_helper.h new file +24 -0
@@ -0,0 +1,24 @@ +/* SPDX-License-Identifier: GPL-2.0 */ + +#include <linux/cdev.h> +#include <linux/errname.h> +#include <linux/fs.h> +#include <linux/module.h> +#include <linux/random.h> +#include <linux/slab.h> +#include <linux/sysctl.h> +#include <linux/uaccess.h> +#include <linux/uio.h> +#include <linux/version.h> +#include <linux/miscdevice.h> +#include <linux/poll.h> +#include <linux/mm.h> +#include <linux/file.h> +#include <uapi/linux/android/binder.h> +#include <linux/platform_device.h> +#include <linux/of_platform.h> +#include <linux/security.h> + +// `bindgen` gets confused at certain things +const gfp_t BINDINGS_GFP_KERNEL =GFP_KERNEL ; +const gfp_t BINDINGS___GFP_ZERO = __GFP_ZERO;
rust/kernel/buffer.rs new file +39 -0
@@ -0,0 +1,39 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Struct for writing to a pre-allocated buffer with the [`write!`] macro. + +use core::fmt; + +/// A pre-allocated buffer that implements [`core::fmt::Write`]. +/// +/// Consecutive writes will append to what has already been written. +/// Writes that don't fit in the buffer will fail. +pub struct Buffer <'a> { + slice: &'amut [u8 ], + pos:usize , +} + +impl <'a>Buffer <'a> { +/// Create a new buffer from an existing array. +pub fn new (slice: &'amut [u8 ]) ->Self { +Buffer { slice, pos:0 } + } + +/// Number of bytes that have already been written to the buffer. +/// This will always be less than the length of the original array. +pub fn bytes_written (&self) ->usize { + self.pos + } +} + +impl <'a> fmt::Write for Buffer <'a> { +fn write_str (&mut self, s: &str ) -> fmt::Result { +if s.len () > self.slice.len () - self.pos { +Err (fmt::Error ) + }else { + self.slice[self.pos..self.pos + s.len ()].copy_from_slice (s.as_bytes ()); + self.pos += s.len (); +Ok (()) + } + } +}
rust/kernel/build_assert.rs new file +80 -0
80 lines not on this page · raw message
rust/kernel/c_types.rs new file +119 -0
119 lines not on this page · raw message
rust/kernel/chrdev.rs new file +212 -0
212 lines not on this page · raw message
rust/kernel/error.rs new file +272 -0
272 lines not on this page · raw message
rust/kernel/file.rs new file +130 -0
130 lines not on this page · raw message
rust/kernel/file_operations.rs new file +698 -0
698 lines not on this page · raw message
rust/kernel/io_buffer.rs new file +153 -0
153 lines not on this page · raw message
rust/kernel/iov_iter.rs new file +95 -0
95 lines not on this page · raw message
rust/kernel/lib.rs new file +220 -0
220 lines not on this page · raw message
rust/kernel/linked_list.rs new file +245 -0
245 lines not on this page · raw message
rust/kernel/miscdev.rs new file +113 -0
113 lines not on this page · raw message
rust/kernel/module_param.rs new file +497 -0
475 lines not on this page · raw message
rust/kernel/of.rs new file +101 -0
101 lines not on this page · raw message
rust/kernel/pages.rs new file +176 -0
176 lines not on this page · raw message
rust/kernel/platdev.rs new file +166 -0
166 lines not on this page · raw message
rust/kernel/print.rs new file +412 -0
412 lines not on this page · raw message
rust/kernel/random.rs new file +50 -0
50 lines not on this page · raw message
rust/kernel/raw_list.rs new file +361 -0
361 lines not on this page · raw message
rust/kernel/rbtree.rs new file +570 -0
570 lines not on this page · raw message
rust/kernel/security.rs new file +79 -0
79 lines not on this page · raw message
rust/kernel/str.rs new file +259 -0
259 lines not on this page · raw message
rust/kernel/sync/arc.rs new file +227 -0
227 lines not on this page · raw message
rust/kernel/sync/condvar.rs new file +136 -0
136 lines not on this page · raw message
rust/kernel/sync/guard.rs new file +82 -0
82 lines not on this page · raw message
rust/kernel/sync/locked_by.rs new file +112 -0
112 lines not on this page · raw message
rust/kernel/sync/mod.rs new file +84 -0
84 lines not on this page · raw message
rust/kernel/sync/mutex.rs new file +101 -0
101 lines not on this page · raw message
rust/kernel/sync/spinlock.rs new file +109 -0
109 lines not on this page · raw message
rust/kernel/sysctl.rs new file +198 -0
198 lines not on this page · raw message
rust/kernel/task.rs new file +193 -0
193 lines not on this page · raw message
rust/kernel/types.rs new file +249 -0
249 lines not on this page · raw message
rust/kernel/user_ptr.rs new file +191 -0
191 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.32.0
Part of the diff is not shown here. View the raw message.
[email protected]
maintainerPATCH 11/17 rust: export generated symbols
In reply to [email protected]From: Miguel Ojeda <[email protected]>
All symbols are reexported reusing the EXPORT_SYMBOL_GPL macro from C. The lists of symbols are generated on the fly.
There are three main sets of symbols to distinguish:
- The ones from the core and alloc crates (from the Rust standard library). The code is licensed as Apache/MIT.
- The ones from our abstractions in the kernel crate.
- The helpers (already exported since they are not generated).
We export everything as GPL. This ensures we do not mistakenly expose GPL kernel symbols/features as non-GPL, even indirectly.
--- rust/exports.c | 16 ++++++++++++++++ 1 file changed, 16 insertions(+) create mode 100644 rust/exports.c
rust/exports.c new file +16 -0
@@ -0,0 +1,16 @@ +// SPDX-License-Identifier: GPL-2.0 +// +// A hack to export Rust symbols for loadable modules without having to redo +// the entire `include/linux/export.h` logic in Rust. +// +// This requires the Rust's new/future `v0` mangling scheme because the default +// one ("legacy") uses invalid characters for C identifiers (thus we cannot use +// the `EXPORT_SYMBOL_*` macros). + +#include <linux/module.h> + +#define EXPORT_SYMBOL_RUST_GPL (sym) extern int sym; EXPORT_SYMBOL_GPL(sym); + +#include "exports_core_generated.h" +#include "exports_alloc_generated.h" +#include "exports_kernel_generated.h"
Signature · 1 line
2.32.0
[email protected]
maintainerPATCH 12/17 Kbuild: add Rust support
In reply to [email protected]From: Miguel Ojeda <[email protected]>
Having all the new files in place, we now enable Rust support in the build system, including Kconfig entries related to Rust, the Rust configuration printer, the target definition files, the version detection script and a few other bits.
In the future, we will likely want to generate the target files on the fly via a script.
--- .gitignore | 5 + .rustfmt.toml | 12 ++ Documentation/process/changes.rst | 13 ++ Makefile | 159 ++++++++++++++- arch/arm/rust/target.json | 28 +++ arch/arm64/rust/target.json | 35 ++++ arch/powerpc/rust/target.json | 30 +++ arch/riscv/Makefile | 1 + arch/riscv/rust/rv32ima.json | 37 ++++ arch/riscv/rust/rv32imac.json | 37 ++++ arch/riscv/rust/rv64ima.json | 37 ++++ arch/riscv/rust/rv64imac.json | 37 ++++ arch/x86/rust/target.json | 37 ++++ init/Kconfig | 28 +++ lib/Kconfig.debug | 144 ++++++++++++++ rust/.gitignore | 6 + rust/Makefile | 316 ++++++++++++++++++++++++++++++ rust/bindgen_parameters | 13 ++ scripts/Makefile.build | 22 +++ scripts/Makefile.lib | 12 ++ scripts/kconfig/confdata.c | 67 ++++++- scripts/rust-version.sh | 31 +++ 22 files changed, 1099 insertions(+), 8 deletions(-) create mode 100644 .rustfmt.toml create mode 100644 arch/arm/rust/target.json create mode 100644 arch/arm64/rust/target.json create mode 100644 arch/powerpc/rust/target.json create mode 100644 arch/riscv/rust/rv32ima.json create mode 100644 arch/riscv/rust/rv32imac.json create mode 100644 arch/riscv/rust/rv64ima.json create mode 100644 arch/riscv/rust/rv64imac.json create mode 100644 arch/x86/rust/target.json create mode 100644 rust/.gitignore create mode 100644 rust/Makefile create mode 100644 rust/bindgen_parameters create mode 100755 scripts/rust-version.sh
.gitignore +5 -0
@@ -37,6 +37,7 @@ *.o *.o.* *.patch +*.rmeta *.s *.so *.so.dbg
@@ -96,6 +97,7 @@ modules.order !.gitattributes !.gitignore !.mailmap +!.rustfmt.toml # # Generated include files
@@ -161,3 +163,6 @@ x509.genkey # Documentation toolchain sphinx_*/ + +# Rust analyzer configuration +/rust-project.json
.rustfmt.toml new file +12 -0
@@ -0,0 +1,12 @@ +edition = "2018" +newline_style = "Unix" + +# Unstable options that help catching some mistakes in formatting and that we may want to enable +# when they become stable. +# +# They are kept here since they are useful to run from time to time. +#format_code_in_doc_comments = true +#reorder_impl_items = true +#comment_width = 100 +#wrap_comments = true +#normalize_comments = true
Documentation/process/changes.rst +13 -0
@@ -26,11 +26,18 @@ running a Linux kernel. Also, not all tools are necessary on all systems; obviously, if you don't have any PC Card hardware, for example, you probably needn't concern yourself with pcmciautils. +Furthermore, note that newer versions of the Rust toolchain may or may not work +because, for the moment, we depend on some unstable features. Thus, unless you +know what you are doing, use the exact version listed here. Please see +:ref:`Documentation/rust/quick-start.rst <rust_quick_start>` for details. + ====================== =============== ======================================== Program Minimal version Command to check the version ====================== =============== ======================================== GNU C 4.9 gcc --version Clang/LLVM (optional) 10.0.1 clang --version +rustc (optional) 1.54.0-beta.1 rustc --version +bindgen (optional) 0.56.0 bindgen --version GNU make 3.81 make --version binutils 2.23 ld -v flex 2.5.35 flex --version
@@ -329,6 +336,12 @@ Sphinx Please see :ref:`sphinx_install` in :ref:`Documentation/doc-guide/sphinx.rst <sphinxdoc>` for details about Sphinx requirements. +rustdoc +------- + +``rustdoc`` is used to generate Rust documentation. Please see +:ref:`Documentation/rust/docs.rst <rust_docs>` for more information. + Getting updated software ========================
Makefile +152 -7
301 lines not on this page · raw message
arch/arm/rust/target.json new file +28 -0
@@ -0,0 +1,28 @@ +{ + "arch": "arm", + "crt-static-respected": true, + "data-layout": "e-m:e-p:32:32-Fi8-i64:64-v128:64:128-a:0:32-n32-S64", + "dynamic-linking": true, + "env": "gnu", + "executables": true, + "features": "+strict-align,+v6", + "function-sections": false, + "has-elf-tls": true, + "has-rpath": true, + "is-builtin": true, + "linker-is-gnu": true, + "llvm-target": "arm-unknown-linux-gnueabi", + "max-atomic-width": 64, + "os": "linux", + "position-independent-executables": true, + "pre-link-args": { + "gcc": [ + "-Wl,--as-needed", + "-Wl,-z,noexecstack" + ] + }, + "relocation-model": "static", + "target-family": "unix", + "target-mcount": "\u0001__gnu_mcount_nc", + "target-pointer-width": "32" +}
arch/arm64/rust/target.json new file +35 -0
@@ -0,0 +1,35 @@ +{ + "arch": "aarch64", + "data-layout": "e-m:e-i8:8:32-i16:16:32-i64:64-i128:128-n32:64-S128", + "disable-redzone": true, + "eliminate-frame-pointer": false, + "emit-debug-gdb-scripts": false, + "env": "gnu", + "features": "+strict-align,+neon,+fp-armv8", + "function-sections": false, + "is-builtin": true, + "linker-flavor": "gcc", + "linker-is-gnu": true, + "llvm-target": "aarch64-unknown-none", + "max-atomic-width": 128, + "needs-plt": true, + "os": "none", + "panic-strategy": "abort", + "position-independent-executables": true, + "pre-link-args": { + "gcc": [ + "-Wl,--as-needed", + "-Wl,-z,noexecstack", + "-m64" + ] + }, + "relocation-model": "static", + "relro-level": "full", + "stack-probes": { + "kind": "none" + }, + "target-c-int-width": "32", + "target-endian": "little", + "target-pointer-width": "64", + "vendor": "" +}
arch/powerpc/rust/target.json new file +30 -0
@@ -0,0 +1,30 @@ +{ + "arch": "powerpc64", + "code-mode": "kernel", + "cpu": "ppc64le", + "data-layout": "e-m:e-i64:64-n32:64", + "env": "gnu", + "features": "-altivec,-vsx,-hard-float", + "function-sections": false, + "is-builtin": true, + "linker-flavor": "gcc", + "linker-is-gnu": true, + "llvm-target": "powerpc64le-elf", + "max-atomic-width": 64, + "os": "none", + "panic-strategy": "abort", + "position-independent-executables": true, + "pre-link-args": { + "gcc": [ + "-Wl,--as-needed", + "-Wl,-z,noexecstack", + "-m64" + ] + }, + "relocation-model": "static", + "relro-level": "full", + "target-family": "unix", + "target-mcount": "_mcount", + "target-endian": "little", + "target-pointer-width": "64" +}
arch/riscv/Makefile +1 -0
@@ -54,6 +54,7 @@ riscv-march-$(CONFIG_FPU) := $(riscv-march-y)fd riscv-march-$(CONFIG_RISCV_ISA_C) := $(riscv-march-y)c KBUILD_CFLAGS += -march=$(subst fd,,$(riscv-march-y)) KBUILD_AFLAGS += -march=$(riscv-march-y) +KBUILD_RUST_TARGET := $(srctree)/arch/riscv/rust/$(subst fd,,$(riscv-march-y)).json KBUILD_CFLAGS += -mno-save-restore KBUILD_CFLAGS += -DCONFIG_PAGE_OFFSET=$(CONFIG_PAGE_OFFSET)
arch/riscv/rust/rv32ima.json new file +37 -0
@@ -0,0 +1,37 @@ +{ + "arch": "riscv32", + "code-model": "medium", + "cpu": "generic-rv32", + "data-layout": "e-m:e-p:32:32-i64:64-n32-S128", + "disable-redzone": true, + "eliminate-frame-pointer": false, + "emit-debug-gdb-scripts": false, + "env": "gnu", + "features": "+m,+a", + "function-sections": false, + "is-builtin": true, + "linker-flavor": "gcc", + "linker-is-gnu": true, + "llvm-target": "riscv32", + "max-atomic-width": 32, + "needs-plt": true, + "os": "none", + "panic-strategy": "abort", + "position-independent-executables": true, + "pre-link-args": { + "gcc": [ + "-Wl,--as-needed", + "-Wl,-z,noexecstack", + "-m32" + ] + }, + "relocation-model": "static", + "relro-level": "full", + "stack-probes": { + "kind": "none" + }, + "target-c-int-width": "32", + "target-endian": "little", + "target-pointer-width": "32", + "vendor": "" +}
arch/riscv/rust/rv32imac.json new file +37 -0
@@ -0,0 +1,37 @@ +{ + "arch": "riscv32", + "code-model": "medium", + "cpu": "generic-rv32", + "data-layout": "e-m:e-p:32:32-i64:64-n32-S128", + "disable-redzone": true, + "eliminate-frame-pointer": false, + "emit-debug-gdb-scripts": false, + "env": "gnu", + "features": "+m,+a,+c", + "function-sections": false, + "is-builtin": true, + "linker-flavor": "gcc", + "linker-is-gnu": true, + "llvm-target": "riscv32", + "max-atomic-width": 32, + "needs-plt": true, + "os": "none", + "panic-strategy": "abort", + "position-independent-executables": true, + "pre-link-args": { + "gcc": [ + "-Wl,--as-needed", + "-Wl,-z,noexecstack", + "-m32" + ] + }, + "relocation-model": "static", + "relro-level": "full", + "stack-probes": { + "kind": "none" + }, + "target-c-int-width": "32", + "target-endian": "little", + "target-pointer-width": "32", + "vendor": "" +}
arch/riscv/rust/rv64ima.json new file +37 -0
@@ -0,0 +1,37 @@ +{ + "arch": "riscv64", + "code-model": "medium", + "cpu": "generic-rv64", + "data-layout": "e-m:e-p:64:64-i64:64-i128:128-n64-S128", + "disable-redzone": true, + "eliminate-frame-pointer": false, + "emit-debug-gdb-scripts": false, + "env": "gnu", + "features": "+m,+a", + "function-sections": false, + "is-builtin": true, + "linker-flavor": "gcc", + "linker-is-gnu": true, + "llvm-target": "riscv64", + "max-atomic-width": 64, + "needs-plt": true, + "os": "none", + "panic-strategy": "abort", + "position-independent-executables": true, + "pre-link-args": { + "gcc": [ + "-Wl,--as-needed", + "-Wl,-z,noexecstack", + "-m64" + ] + }, + "relocation-model": "static", + "relro-level": "full", + "stack-probes": { + "kind": "none" + }, + "target-c-int-width": "32", + "target-endian": "little", + "target-pointer-width": "64", + "vendor": "" +}
arch/riscv/rust/rv64imac.json new file +37 -0
@@ -0,0 +1,37 @@ +{ + "arch": "riscv64", + "code-model": "medium", + "cpu": "generic-rv64", + "data-layout": "e-m:e-p:64:64-i64:64-i128:128-n64-S128", + "disable-redzone": true, + "eliminate-frame-pointer": false, + "emit-debug-gdb-scripts": false, + "env": "gnu", + "features": "+m,+a,+c", + "function-sections": false, + "is-builtin": true, + "linker-flavor": "gcc", + "linker-is-gnu": true, + "llvm-target": "riscv64", + "max-atomic-width": 64, + "needs-plt": true, + "os": "none", + "panic-strategy": "abort", + "position-independent-executables": true, + "pre-link-args": { + "gcc": [ + "-Wl,--as-needed", + "-Wl,-z,noexecstack", + "-m64" + ] + }, + "relocation-model": "static", + "relro-level": "full", + "stack-probes": { + "kind": "none" + }, + "target-c-int-width": "32", + "target-endian": "little", + "target-pointer-width": "64", + "vendor": "" +}
arch/x86/rust/target.json new file +37 -0
@@ -0,0 +1,37 @@ +{ + "arch": "x86_64", + "code-model": "kernel", + "cpu": "x86-64", + "data-layout": "e-m:e-p270:32:32-p271:32:32-p272:64:64-i64:64-f80:128-n8:16:32:64-S128", + "disable-redzone": true, + "eliminate-frame-pointer": false, + "emit-debug-gdb-scripts": false, + "env": "gnu", + "features": "-mmx,-sse,-sse2,-sse3,-ssse3,-sse4.1,-sse4.2,-3dnow,-3dnowa,-avx,-avx2,+soft-float", + "function-sections": false, + "is-builtin": true, + "linker-flavor": "gcc", + "linker-is-gnu": true, + "llvm-target": "x86_64-elf", + "max-atomic-width": 64, + "needs-plt": true, + "os": "none", + "panic-strategy": "abort", + "position-independent-executables": true, + "pre-link-args": { + "gcc": [ + "-Wl,--as-needed", + "-Wl,-z,noexecstack", + "-m64" + ] + }, + "relocation-model": "static", + "relro-level": "full", + "stack-probes": { + "kind": "none" + }, + "target-c-int-width": "32", + "target-endian": "little", + "target-pointer-width": "64", + "vendor": "unknown" +}
init/Kconfig +28 -0
@@ -60,6 +60,15 @@ config LLD_VERSION default $(ld-version) if LD_IS_LLD default 0 +config HAS_RUST + depends on ARM64 || CPU_32v6 || CPU_32v6K || (PPC64 && CPU_LITTLE_ENDIAN) || X86_64 || RISCV + def_bool $(success,$(RUSTC) --version) + +config RUSTC_VERSION + depends on HAS_RUST + int + default $(shell,$(srctree)/scripts/rust-version.sh $(RUSTC)) + config CC_CAN_LINK bool default $(success,$(srctree)/scripts/cc-can-link.sh $(CC) $(CLANG_FLAGS) $(m64-flag)) if 64BIT
@@ -2011,6 +2020,25 @@ config PROFILING Say Y here to enable the extended profiling support mechanisms used by profilers. +config RUST + bool "Rust support" + depends on HAS_RUST + depends on !COMPILE_TEST + depends on !MODVERSIONS + default n + help + Enables Rust support in the kernel. + + This allows other Rust-related options, like drivers written in Rust, + to be selected. + + It is also required to be able to load external kernel modules + written in Rust. + + See Documentation/rust/ for more information. + + If unsure, say N. + # # Place an empty function call at each tracepoint site. Can be # dynamically changed for a probe function.
lib/Kconfig.debug +144 -0
150 lines not on this page · raw message
rust/.gitignore new file +6 -0
@@ -0,0 +1,6 @@ +# SPDX-License-Identifier: GPL-2.0 + +bindings_generated.rs +exports_*_generated.h +doc/ +test/
rust/Makefile new file +316 -0
316 lines not on this page · raw message
rust/bindgen_parameters new file +13 -0
@@ -0,0 +1,13 @@ +# SPDX-License-Identifier: GPL-2.0 + +--opaque-type xregs_state +--opaque-type desc_struct +--opaque-type arch_lbr_state +--opaque-type local_apic + +# If SMP is disabled, `arch_spinlock_t` is defined as a ZST which triggers a Rust +# warning. We don't need to peek into it anyway. +--opaque-type spinlock + +# `seccomp`'s comment gets understood as a doctest +--no-doc-comments
scripts/Makefile.build +22 -0
@@ -26,6 +26,7 @@ EXTRA_CPPFLAGS := EXTRA_LDFLAGS := asflags-y := ccflags-y := +rustflags-y := cppflags-y := ldflags-y :=
@@ -288,6 +289,27 @@ quiet_cmd_cc_lst_c = MKLST $@ $(obj)/%.lst: $(src)/%.c FORCE $(call if_changed_dep,cc_lst_c) +# Compile Rust sources (.rs) +# --------------------------------------------------------------------------- + +# Need to use absolute path here and have symbolic links resolved; +# otherwise rustdoc and rustc compute different hashes for the target. +rust_cross_flags := --target=$(realpath $(KBUILD_RUST_TARGET)) + +quiet_cmd_rustc_o_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ + cmd_rustc_o_rs = \ + RUST_MODFILE=$(modfile) \ + $(RUSTC_OR_CLIPPY) $(rust_flags) $(rust_cross_flags) \ + -Zallow-features=allocator_api,bench_black_box,concat_idents,global_asm,try_reserve \ + --extern alloc --extern kernel \ + --crate-type rlib --out-dir $(obj) -L $(objtree)/rust/ \ + --crate-name $(patsubst %.o,%,$(notdir $@)) $<; \ + mv $(obj)/$(subst .o,,$(notdir $@)).d $(depfile); \ + sed -i '/^\#/d' $(depfile) + +$(obj)/%.o: $(src)/%.rs FORCE + $(call if_changed_dep,rustc_o_rs) + # Compile assembler sources (.S) # ---------------------------------------------------------------------------
scripts/Makefile.lib +12 -0
@@ -8,6 +8,7 @@ ldflags-y += $(EXTRA_LDFLAGS) # flags that take effect in current and sub directories KBUILD_AFLAGS += $(subdir-asflags-y) KBUILD_CFLAGS += $(subdir-ccflags-y) +KBUILD_RUSTFLAGS += $(subdir-rustflags-y) # Figure out what we need to build from the various variables # ===========================================================================
@@ -133,6 +134,10 @@ _c_flags = $(filter-out $(CFLAGS_REMOVE_$(target-stem).o), \ $(filter-out $(ccflags-remove-y), \ $(KBUILD_CPPFLAGS) $(KBUILD_CFLAGS) $(ccflags-y)) \ $(CFLAGS_$(target-stem).o)) +_rust_flags = $(filter-out $(RUSTFLAGS_REMOVE_$(target-stem).o), \ + $(filter-out $(rustflags-remove-y), \ + $(KBUILD_RUSTFLAGS) $(rustflags-y)) \ + $(RUSTFLAGS_$(target-stem).o)) _a_flags = $(filter-out $(AFLAGS_REMOVE_$(target-stem).o), \ $(filter-out $(asflags-remove-y), \ $(KBUILD_CPPFLAGS) $(KBUILD_AFLAGS) $(asflags-y)) \
@@ -202,6 +207,11 @@ modkern_cflags = \ $(KBUILD_CFLAGS_MODULE) $(CFLAGS_MODULE), \ $(KBUILD_CFLAGS_KERNEL) $(CFLAGS_KERNEL) $(modfile_flags)) +modkern_rustflags = \ + $(if $(part-of-module), \ + $(KBUILD_RUSTFLAGS_MODULE) $(RUSTFLAGS_MODULE), \ + $(KBUILD_RUSTFLAGS_KERNEL) $(RUSTFLAGS_KERNEL)) + modkern_aflags = $(if $(part-of-module), \ $(KBUILD_AFLAGS_MODULE) $(AFLAGS_MODULE), \ $(KBUILD_AFLAGS_KERNEL) $(AFLAGS_KERNEL))
@@ -211,6 +221,8 @@ c_flags = -Wp,-MMD,$(depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) \ $(_c_flags) $(modkern_cflags) \ $(basename_flags) $(modname_flags) +rust_flags = $(_rust_flags) $(modkern_rustflags) @$(objtree)/include/generated/rustc_cfg + a_flags = -Wp,-MMD,$(depfile) $(NOSTDINC_FLAGS) $(LINUXINCLUDE) \ $(_a_flags) $(modkern_aflags)
scripts/kconfig/confdata.c +66 -1
100 lines not on this page · raw message
scripts/rust-version.sh new file +31 -0
@@ -0,0 +1,31 @@ +#!/bin/sh +# SPDX-License-Identifier: GPL-2.0 +# +# rust-version rust-command +# +# Print the compiler version of `rust-command' in a 5 or 6-digit form +# such as `14502' for rustc-1.45.2 etc. +# +# Returns 0 if not found (so that Kconfig does not complain) +compiler="$*" + +if [ ${#compiler} -eq 0 ]; then + echo "Error: No compiler specified." >&2 + printf "Usage:\n\t$0 <rust-command>\n" >&2 + exit 1 +fi + +if ! command -v $compiler >/dev/null 2>&1; then + echo 0 + exit 0 +fi + +VERSION=$($compiler --version | cut -f2 -d' ') + +# Cut suffix if any (e.g. `-dev`) +VERSION=$(echo $VERSION | cut -f1 -d'-') + +MAJOR=$(echo $VERSION | cut -f1 -d'.') +MINOR=$(echo $VERSION | cut -f2 -d'.') +PATCHLEVEL=$(echo $VERSION | cut -f3 -d'.') +printf "%d%02d%02d\\n" $MAJOR $MINOR $PATCHLEVEL
Signature · 1 line
2.32.0
[email protected]
maintainerPATCH 13/17 docs: add Rust documentation
In reply to [email protected]From: Miguel Ojeda <[email protected]>
Most of the documentation for Rust is written within the source code itself, as it is idiomatic for Rust projects. This applies to both the shared infrastructure at rust/ as well as any other Rust module (e.g. drivers) written across the kernel.
These documents contain general information that does not fit particularly well in the source code, like the Quick Start guide.
--- Documentation/doc-guide/kernel-doc.rst | 3 + Documentation/index.rst | 1 + Documentation/kbuild/kbuild.rst | 4 + Documentation/rust/arch-support.rst | 35 +++ Documentation/rust/assets/favicon-16x16.png | Bin 0 -> 798 bytes Documentation/rust/assets/favicon-32x32.png | Bin 0 -> 2076 bytes Documentation/rust/assets/rust-logo.png | Bin 0 -> 53976 bytes Documentation/rust/coding.rst | 92 ++++++++ Documentation/rust/docs.rst | 110 ++++++++++ Documentation/rust/index.rst | 20 ++ Documentation/rust/quick-start.rst | 222 ++++++++++++++++++++ 11 files changed, 487 insertions(+) create mode 100644 Documentation/rust/arch-support.rst create mode 100644 Documentation/rust/assets/favicon-16x16.png create mode 100644 Documentation/rust/assets/favicon-32x32.png create mode 100644 Documentation/rust/assets/rust-logo.png create mode 100644 Documentation/rust/coding.rst create mode 100644 Documentation/rust/docs.rst create mode 100644 Documentation/rust/index.rst create mode 100644 Documentation/rust/quick-start.rst
Documentation/doc-guide/kernel-doc.rst +3 -0
@@ -11,6 +11,9 @@ when it is embedded in source files. reasons. The kernel source contains tens of thousands of kernel-doc comments. Please stick to the style described here. +.. note:: kernel-doc does not cover Rust code: please see + Documentation/rust/docs.rst instead. + The kernel-doc structure is extracted from the comments, and proper `Sphinx C Domain`_ function and type descriptions with anchors are generated from them. The descriptions are filtered for special kernel-doc
Documentation/index.rst +1 -0
@@ -82,6 +82,7 @@ merged much easier. maintainer/index fault-injection/index livepatch/index + rust/index Kernel API documentation
Documentation/kbuild/kbuild.rst +4 -0
@@ -57,6 +57,10 @@ CFLAGS_MODULE ------------- Additional module specific options to use for $(CC). +KRUSTFLAGS +---------- +Additional options to the Rust compiler (for built-in and modules). + LDFLAGS_MODULE -------------- Additional options used for $(LD) when linking modules.
Documentation/rust/arch-support.rst new file +35 -0
@@ -0,0 +1,35 @@ +.. _rust_arch_support: + +Arch Support +============ + +Currently, the Rust compiler (``rustc``) uses LLVM for code generation, +which limits the supported architectures we can target. In addition, support +for building the kernel with LLVM/Clang varies (see :ref:`kbuild_llvm`), +which ``bindgen`` relies on through ``libclang``. + +Below is a general summary of architectures that currently work. Level of +support corresponds to ``S`` values in the ``MAINTAINERS`` file. + +.. list-table:: + :widths: 10 10 10 + :header-rows: 1 + + * - Architecture + - Level of support + - Constraints + * - ``arm`` + - Maintained + - ``armv6`` and compatible only, ``RUST_OPT_LEVEL >= 2`` + * - ``arm64`` + - Maintained + - None + * - ``powerpc`` + - Maintained + - ``ppc64le`` only, ``RUST_OPT_LEVEL < 2`` requires ``CONFIG_THREAD_SHIFT=15`` + * - ``riscv`` + - Maintained + - ``riscv64`` only + * - ``x86`` + - Maintained + - ``x86_64`` only
Documentation/rust/assets/favicon-16x16.png new file binary +0 -0
Documentation/rust/assets/favicon-32x32.png new file binary +0 -0
Documentation/rust/assets/rust-logo.png new file binary +0 -0
Documentation/rust/coding.rst new file +92 -0
92 lines not on this page · raw message
Documentation/rust/docs.rst new file +110 -0
110 lines not on this page · raw message
Documentation/rust/index.rst new file +20 -0
@@ -0,0 +1,20 @@ +Rust +==== + +Documentation related to Rust within the kernel. If you are starting out, +read the :ref:`Documentation/rust/quick-start.rst <rust_quick_start>` guide. + +.. toctree:: + :maxdepth: 1 + + quick-start + coding + docs + arch-support + +.. only:: subproject and html + + Indices + ======= + + * :ref:`genindex`
Documentation/rust/quick-start.rst new file +222 -0
222 lines not on this page · raw message
Signature · 1 line
2.32.0
Re: [PATCH 13/17] docs: add Rust documentation
Quoted 6 lines · On Sun, Jul 04, 2021 at 10:27:52PM +0200, [email protected] wrote:
From: Miguel Ojeda <[email protected]>
Most of the documentation for Rust is written within the source code itself, as it is idiomatic for Rust projects. This applies to both the shared infrastructure at
rust/as well as any other Rust module (e.g. drivers) written across the kernel.
(...)
Quoted 9 lines
Documentation/rust/coding.rst new file +3
-0@@ -0,0 +1,92 @@ +.. _rust_coding: + +Coding
Documentation/rust/coding.rst line 13
Quoted 10 lines
Documentation/rust/coding.rst new file +10
-0+====== + +This document describes how to write Rust code in the kernel. + + +Coding style +------------ + +The code is automatically formatted using the ``rustfmt`` tool. This is very +good news!
Miguel, the wording and style in this file is not much welcome, it looks like a copy-paste of an e-mail in the doc. The exclamation above "this is a very good news" doesn't really belong to a doc, and for readers who don't understand why it appears as a good news to the writer, it probably is an even less good news.
Quoted 5 lines
Documentation/rust/coding.rst new file +5
-0+- If you contribute from time to time to the kernel, you do not need to learn + and remember one more style guide. You will also need less patch roundtrips + to land a change. + +- If you are a reviewer or a maintainer, you will not need to spend time on
Documentation/rust/coding.rst line 29
Quoted 10 lines
Documentation/rust/coding.rst new file +10
-0+ pointing out style issues anymore. + +.. note:: Conventions on comments and documentation are not checked by + ``rustfmt``. Thus we still need to take care of those: please see + :ref:`Documentation/rust/docs.rst <rust_docs>`. + +We use the tool's default settings. This means we are following the idiomatic +Rust style. For instance, we use 4 spaces for indentation rather than tabs. + +Typically, you will want to instruct your editor/IDE to format while you type,
In general you should avoid "we" and "you" when writing documentation. Prefer passive forms instead, which do not place a barrier between those who teach and those who learn. It's generally considered more inclusive in that it makes the reader not feel outside of the team who wrote it.
For example the last sentences above could be reworded like this:
The tool's default settings are used, which means the idiomatic Rust style is followed. For instance, 4 spaces are used for indentation rather than tabs.
Typically editors/IDE will have to be instructed to format while typing. (...)
An additional note is that if the language imposes such unusual constraints on the editor, you should probably point to various known settins for most well-known editors.
Documentation/rust/coding.rst line 42
Quoted 13 lines
Documentation/rust/coding.rst new file +13
-0+when you save or at commit time. However, if for some reason you want +to reformat the entire kernel Rust sources at some point, you may run:: + + make LLVM=1 rustfmt + +To check if everything is formatted (printing a diff otherwise), e.g. if you +have configured a CI for a tree as a maintainer, you may run:: + + make LLVM=1 rustfmtcheck + +Like ``clang-format`` for the rest of the kernel, ``rustfmt`` works on +individual files, and does not require a kernel configuration. Sometimes it may +even work with broken code.
You should also clearly indicate how to recheck (or adjust) individual files, not just say that the command supports it.
Regards,
Willy
Re: [PATCH 13/17] docs: add Rust documentation
Quoted 6 lines · Willy Tarreau
Miguel, the wording and style in this file is not much welcome, it looks like a copy-paste of an e-mail in the doc. The exclamation above "this is a very good news" doesn't really belong to a doc, and for readers who don't understand why it appears as a good news to the writer, it probably is an even less good news.
Yes, I can definitely be more formal here.
Quoted 4 lines
In general you should avoid "we" and "you" when writing documentation. Prefer passive forms instead, which do not place a barrier between those who teach and those who learn. It's generally considered more inclusive in that it makes the reader not feel outside of the team who wrote it.
When I was writing this, I wondered the same thing, because in Spanish this does look quite bad (in the sense of being too informal), and we use the passive forms a lot more for things like this. So I am fine rewriting this. Also, mixing we/you is not ideal either.
Having said that, I am not sure about English and whether people prefer to read text with the passive form or not. In Documentation/ there seems to be a lot of "we"s and "you"s, but they could be wrong too, of course.
Quoted 3 lines
An additional note is that if the language imposes such unusual constraints on the editor, you should probably point to various known settins for most well-known editors.
Are you referring about style? If yes, it is possible to write the code with a text editor with no extra features and then format it, so that should not be a problem.
Quoted 2 lines
You should also clearly indicate how to recheck (or adjust) individual files, not just say that the command supports it.
Sounds good -- I will do that.
Thanks a lot for reviewing the docs!
Cheers,
Miguel
Re: [PATCH 13/17] docs: add Rust documentation
Quoted 9 lines · Miguel Ojeda
Quoted 4 lines
In general you should avoid "we" and "you" when writing documentation. Prefer passive forms instead, which do not place a barrier between those who teach and those who learn. It's generally considered more inclusive in that it makes the reader not feel outside of the team who wrote it.
When I was writing this, I wondered the same thing, because in Spanish this does look quite bad (in the sense of being too informal), and we use the passive forms a lot more for things like this. So I am fine rewriting this. Also, mixing we/you is not ideal either.
Indeed, I can imagine how informal it could sound in Spanish.
Quoted 4 lines
Having said that, I am not sure about English and whether people prefer to read text with the passive form or not. In
Documentation/there seems to be a lot of "we"s and "you"s, but they could be wrong too, of course.
It's possible. While I've seen it used a lot in training or step-by-step instructions which aim to guide the reader through a procedure, it's not commonly found in documentation. One principle to keep in mind is to only focus on the subject. If your documentation describes a component or process and does not involve a human, there's no reason for introducing this human there. If it explicitly aims at the human (e.g. instructions), of course it makes sense. But anything that can end up in a script does not require a human and should avoid we/you.
Quoted 7 lines
Quoted 3 lines
An additional note is that if the language imposes such unusual constraints on the editor, you should probably point to various known settins for most well-known editors.
Are you referring about style? If yes, it is possible to write the code with a text editor with no extra features and then format it, so that should not be a problem.
Yes that's my point, it will likely be the first experience for most casual visitors who have no idea how to reconfigure their editor or who don't want to risk to break their existing config.
Quoted 6 lines
Quoted 2 lines
You should also clearly indicate how to recheck (or adjust) individual files, not just say that the command supports it.
Sounds good -- I will do that.
Thanks a lot for reviewing the docs!
You're welcome.
Willy
Re: [PATCH 13/17] docs: add Rust documentation
In reply to [email protected]Quoted 11 lines · On Sun, Jul 4, 2021 at 1:29 PM <[email protected]> wrote:
From: Miguel Ojeda <[email protected]>
Most of the documentation for Rust is written within the source code itself, as it is idiomatic for Rust projects. This applies to both the shared infrastructure at
rust/as well as any other Rust module (e.g. drivers) written across the kernel.These documents contain general information that does not fit particularly well in the source code, like the Quick Start guide.
... snip ...
Documentation/rust/quick-start.rst line 164
Quoted 12 lines
Documentation/rust/quick-start.rst new file +12
-0+rust-analyzer +************* + +The `rust-analyzer <https://rust-analyzer.github.io/>`_ language server can +be used with many editors to enable syntax highlighting, completion, go to +definition, and other features. + +``rust-analyzer`` will need to be +`configured <https://rust-analyzer.github.io/manual.html#non-cargo-based-projects>`_ +to work with the kernel by adding a ``rust-project.json`` file in the root folder. +A ``rust-project.json`` can be generated by building the Make target ``rust-analyzer``, +which will create a ``rust-project.json`` in the root of the output directory.
Is rust-analyzer necessary to talk about? I think most kernel developers aren't using LSP clients, so it doesn't seem necessary to talk about for kernel development IMO. If a developer would like to use an LSP client, they should refer to the documentation of their LSP client.
Should Documentation/process/changes.rst be updated to note which versions of all of these tools are currently supported?
Quoted 1 line
+ - ``armv6`` and compatible only, ``RUST_OPT_LEVEL >= 2``
For the compat table, for 32b ARM, why is v6 supported but not v7? Why only when optimizations are enabled?
Documentation/rust/arch-support.rst line 9
Quoted 4 lines
Documentation/rust/arch-support.rst new file +4
-0+Currently, the Rust compiler (``rustc``) uses LLVM for code generation, +which limits the supported architectures we can target. In addition, support +for building the kernel with LLVM/Clang varies (see :ref:`kbuild_llvm`), +which ``bindgen`` relies on through ``libclang``.
Doesn't this also depend on rustc having support for a given target triple? Just because LLVM has a backend for a given architecture doesn't mean rustc can target it, right?
Signature · 2 lines
Thanks, ~Nick Desaulniers
Re: [PATCH 13/17] docs: add Rust documentation
Quoted 6 lines · Nick Desaulniers
Is rust-analyzer necessary to talk about? I think most kernel developers aren't using LSP clients, so it doesn't seem necessary to talk about for kernel development IMO. If a developer would like to use an LSP client, they should refer to the documentation of their LSP client.
I think it is fair to advertise it given we have support for generating its configuration.
Otherwise, people might miss the script and spend time generating a configuration by hand (which takes quite some time and would not carry the CONFIG_* settings etc.).
And they may start using it if they did not hear about it! :)
Quoted 2 lines
Should
Documentation/process/changes.rstbe updated to note which versions of all of these tools are currently supported?
For the ones required for building, they are already there (i.e. rustc and bindgen).
For the rest, most come with the Rust toolchain, so I think there is no need to list them explicitly. For rust-analyzer, it is "rolling", so it does not make much sense to put it in any case.
Quoted 2 lines
For the compat table, for 32b ARM, why is v6 supported but not v7? Why only when optimizations are enabled?
For arch support, we only have a few fixed Rust target files for the moment as examples (thus we picked v6) -- we will generate those with a script later on.
For particular restrictions (such as the optimization level), we will look into solving those bits later on. In any case, it is not a priority, since less than -O2 is intended only for kernel hacking.
Quoted 3 lines
Doesn't this also depend on rustc having support for a given target triple? Just because LLVM has a backend for a given architecture doesn't mean rustc can target it, right?
No -- currently we use custom target files instead of particular triples (see previous point), which are basically the settings rustc forwards to LLVM for codegen.
Cheers,
Miguel
[email protected]
maintainerPATCH 14/17 samples: add Rust examples
In reply to [email protected]From: Miguel Ojeda <[email protected]>
A set of Rust modules that showcase how Rust modules look like and how to use the abstracted kernel features.
These samples also double as tests in the CI.
The semaphore sample comes with a C version for comparison.
--- samples/Kconfig | 2 + samples/Makefile | 1 + samples/rust/Kconfig | 113 +++++++++++++ samples/rust/Makefile | 12 ++ samples/rust/rust_chrdev.rs | 51 ++++++ samples/rust/rust_minimal.rs | 38 +++++ samples/rust/rust_miscdev.rs | 150 +++++++++++++++++ samples/rust/rust_module_parameters.rs | 72 +++++++++ samples/rust/rust_print.rs | 57 +++++++ samples/rust/rust_random.rs | 61 +++++++ samples/rust/rust_semaphore.rs | 177 +++++++++++++++++++++ samples/rust/rust_semaphore_c.c | 212 +++++++++++++++++++++++++ samples/rust/rust_stack_probing.rs | 40 +++++ samples/rust/rust_sync.rs | 81 ++++++++++ 14 files changed, 1067 insertions(+) create mode 100644 samples/rust/Kconfig create mode 100644 samples/rust/Makefile create mode 100644 samples/rust/rust_chrdev.rs create mode 100644 samples/rust/rust_minimal.rs create mode 100644 samples/rust/rust_miscdev.rs create mode 100644 samples/rust/rust_module_parameters.rs create mode 100644 samples/rust/rust_print.rs create mode 100644 samples/rust/rust_random.rs create mode 100644 samples/rust/rust_semaphore.rs create mode 100644 samples/rust/rust_semaphore_c.c create mode 100644 samples/rust/rust_stack_probing.rs create mode 100644 samples/rust/rust_sync.rs
samples/Kconfig +2 -0
@@ -223,4 +223,6 @@ config SAMPLE_WATCH_QUEUE Build example userspace program to use the new mount_notify(), sb_notify() syscalls and the KEYCTL_WATCH_KEY keyctl() function. +source "samples/rust/Kconfig" + endif # SAMPLES
samples/Makefile +1 -0
@@ -30,3 +30,4 @@ obj-$(CONFIG_SAMPLE_INTEL_MEI) += mei/ subdir-$(CONFIG_SAMPLE_WATCHDOG) += watchdog subdir-$(CONFIG_SAMPLE_WATCH_QUEUE) += watch_queue obj-$(CONFIG_DEBUG_KMEMLEAK_TEST) += kmemleak/ +obj-$(CONFIG_SAMPLES_RUST) += rust/
samples/rust/Kconfig new file +113 -0
113 lines not on this page · raw message
samples/rust/Makefile new file +12 -0
@@ -0,0 +1,12 @@ +# SPDX-License-Identifier: GPL-2.0 + +obj-$(CONFIG_SAMPLE_RUST_MINIMAL) += rust_minimal.o +obj-$(CONFIG_SAMPLE_RUST_PRINT) += rust_print.o +obj-$(CONFIG_SAMPLE_RUST_MODULE_PARAMETERS) += rust_module_parameters.o +obj-$(CONFIG_SAMPLE_RUST_SYNC) += rust_sync.o +obj-$(CONFIG_SAMPLE_RUST_CHRDEV) += rust_chrdev.o +obj-$(CONFIG_SAMPLE_RUST_MISCDEV) += rust_miscdev.o +obj-$(CONFIG_SAMPLE_RUST_STACK_PROBING) += rust_stack_probing.o +obj-$(CONFIG_SAMPLE_RUST_SEMAPHORE) += rust_semaphore.o +obj-$(CONFIG_SAMPLE_RUST_SEMAPHORE_C) += rust_semaphore_c.o +obj-$(CONFIG_SAMPLE_RUST_RANDOM) += rust_random.o
samples/rust/rust_chrdev.rs new file +51 -0
51 lines not on this page · raw message
samples/rust/rust_minimal.rs new file +38 -0
@@ -0,0 +1,38 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Rust minimal sample + +#![no_std] +#![feature(allocator_api, global_asm)] + +use kernel::prelude::*; + +module! { +type :RustMinimal , +name :b"rust_minimal" , + author:b"Rust for Linux Contributors" , + description:b"Rust minimal sample" , + license:b"GPL v2" , +} + +struct RustMinimal { + message:String , +} + +impl KernelModule for RustMinimal { +fn init () ->Result <Self > { +pr_info! ("Rust minimal sample (init)\n" ); +pr_info! ("Am I built-in? {}\n" , !cfg! (MODULE )); + +Ok (RustMinimal { + message:"on the heap!" .try_to_owned ()?, + }) + } +} + +impl Drop for RustMinimal { +fn drop (&mut self) { +pr_info! ("My message is {}\n" , self.message); +pr_info! ("Rust minimal sample (exit)\n" ); + } +}
samples/rust/rust_miscdev.rs new file +150 -0
150 lines not on this page · raw message
samples/rust/rust_module_parameters.rs new file +72 -0
72 lines not on this page · raw message
samples/rust/rust_print.rs new file +57 -0
57 lines not on this page · raw message
samples/rust/rust_random.rs new file +61 -0
61 lines not on this page · raw message
samples/rust/rust_semaphore.rs new file +177 -0
177 lines not on this page · raw message
samples/rust/rust_semaphore_c.c new file +212 -0
212 lines not on this page · raw message
samples/rust/rust_stack_probing.rs new file +40 -0
@@ -0,0 +1,40 @@ +// SPDX-License-Identifier: GPL-2.0 + +//! Rust stack probing sample + +#![no_std] +#![feature(allocator_api, global_asm)] +#![feature(bench_black_box)] + +use kernel::prelude::*; + +module! { +type :RustStackProbing , +name :b"rust_stack_probing" , + author:b"Rust for Linux Contributors" , + description:b"Rust stack probing sample" , + license:b"GPL v2" , +} + +struct RustStackProbing ; + +impl KernelModule for RustStackProbing { +fn init () ->Result <Self > { +pr_info! ("Rust stack probing sample (init)\n" ); + +// Including this large variable on the stack will trigger +// stack probing on the supported archs. +// This will verify that stack probing does not lead to +// any errors if we need to link `__rust_probestack`. +let x: [u64 ;514 ] = core::hint::black_box ([5 ;514 ]); +pr_info! ("Large array has length: {}\n" , x.len ()); + +Ok (RustStackProbing ) + } +} + +impl Drop for RustStackProbing { +fn drop (&mut self) { +pr_info! ("Rust stack probing sample (exit)\n" ); + } +}
samples/rust/rust_sync.rs new file +81 -0
81 lines not on this page · raw message
Signature · 1 line
2.32.0
[email protected]
maintainerPATCH 15/17 scripts: add `generate_rust_analyzer.py`
In reply to [email protected]From: Miguel Ojeda <[email protected]>
The generate_rust_analyzer.py script generates the configuration file (rust-project.json) for rust-analyzer.
rust-analyzer is a modular compiler frontend for the Rust language. It provides an LSP server which can be used in editors such as VS Code, Emacs or Vim.
--- scripts/generate_rust_analyzer.py | 143 ++++++++++++++++++++++++++++++ 1 file changed, 143 insertions(+) create mode 100755 scripts/generate_rust_analyzer.py
scripts/generate_rust_analyzer.py new file +143 -0
@@ -0,0 +1,143 @@ +#!/usr/bin/env python3 +"""generate_rust_analyzer - Generates the `rust-project.json` file for `rust-analyzer`. +""" + +import argparse +import json +import logging +import pathlib +import sys + +def generate_crates(srctree, objtree, sysroot_src, bindings_file): + # Generate the configuration list. + cfg = [] + with open(objtree / "include" / "generated" / "rustc_cfg") as fd: + for line in fd: + line = line.replace("--cfg=", "") + line = line.replace("\n", "") + cfg.append(line) + + # Now fill the crates list -- dependencies need to come first. + # + # Avoid O(n^2) iterations by keeping a map of indexes. + crates = [] + crates_indexes = {} + + def append_crate(display_name, root_module, is_workspace_member, deps, cfg): + crates_indexes[display_name] = len(crates) + crates.append({ + "display_name": display_name, + "root_module": str(root_module), + "is_workspace_member": is_workspace_member, + "deps": [{"crate": crates_indexes[dep], "name": dep} for dep in deps], + "cfg": cfg, + "edition": "2018", + "env": { + "RUST_MODFILE": "This is only for rust-analyzer" + } + }) + + # First, the ones in `rust/` since they are a bit special. + append_crate( + "core", + sysroot_src / "core" / "src" / "lib.rs", + False, + [], + [], + ) + + append_crate( + "compiler_builtins", + srctree / "rust" / "compiler_builtins.rs", + True, + [], + [], + ) + + append_crate( + "alloc", + srctree / "rust" / "alloc" / "lib.rs", + True, + ["core", "compiler_builtins"], + [], + ) + + append_crate( + "macros", + srctree / "rust" / "macros" / "lib.rs", + True, + [], + [], + ) + crates[-1]["proc_macro_dylib_path"] = "rust/libmacros.so" + + append_crate( + "build_error", + srctree / "rust" / "build_error.rs", + True, + ["core", "compiler_builtins"], + [], + ) + + append_crate( + "kernel", + srctree / "rust" / "kernel" / "lib.rs", + True, + ["core", "alloc", "macros", "build_error"], + cfg, + ) + crates[-1]["env"]["RUST_BINDINGS_FILE"] = str(bindings_file.resolve(True)) + crates[-1]["source"] = { + "include_dirs": [ + str(srctree / "rust" / "kernel"), + str(objtree / "rust") + ], + "exclude_dirs": [], + } + + # Then, the rest outside of `rust/`. + # + # We explicitly mention the top-level folders we want to cover. + for folder in ("samples", "drivers"): + for path in (srctree / folder).rglob("*.rs"): + logging.info("Checking %s", path) + name = path.name.replace(".rs", "") + + # Skip those that are not crate roots. + if f"{name}.o" not in open(path.parent / "Makefile").read(): + continue + + logging.info("Adding %s", name) + append_crate( + name, + path, + True, + ["core", "alloc", "kernel"], + cfg, + ) + + return crates + +def main(): + parser = argparse.ArgumentParser() + parser.add_argument('--verbose', '-v', action='store_true') + parser.add_argument("srctree", type=pathlib.Path) + parser.add_argument("objtree", type=pathlib.Path) + parser.add_argument("sysroot_src", type=pathlib.Path) + parser.add_argument("bindings_file", type=pathlib.Path) + args = parser.parse_args() + + logging.basicConfig( + format="[%(asctime)s] [%(levelname)s] %(message)s", + level=logging.INFO if args.verbose else logging.WARNING + ) + + rust_project = { + "crates": generate_crates(args.srctree, args.objtree, args.sysroot_src, args.bindings_file), + "sysroot_src": str(args.sysroot_src), + } + + json.dump(rust_project, sys.stdout, sort_keys=True, indent=4) + +if __name__ == "__main__": + main()
Signature · 1 line
2.32.0
Re: [PATCH 15/17] scripts: add `generate_rust_analyzer.py`
Quoted 8 lines · On Sun, Jul 04, 2021 at 10:27:54PM +0200, [email protected] wrote:
From: Miguel Ojeda <[email protected]>
The
generate_rust_analyzer.pyscript generates the configuration file (rust-project.json) for rust-analyzer.rust-analyzer is a modular compiler frontend for the Rust language. It provides an LSP server which can be used in editors such as VS Code, Emacs or Vim.
You should probably add some doc about this, as I have no idea how I'm supposed to use this from my editor.
Willy
Re: [PATCH 15/17] scripts: add `generate_rust_analyzer.py`
Quoted 3 lines · Willy Tarreau
You should probably add some doc about this, as I have no idea how I'm supposed to use this from my editor.
Yeah, I can add a link to rust-analyzer's docs and/or the Language Server Protocol site, and perhaps a couple steps on how to set it up; but I would try to avoid repeating their documentation.
Cheers,
Miguel
[email protected]
maintainerPATCH 16/17 MAINTAINERS: Rust
In reply to [email protected]From: Miguel Ojeda <[email protected]>
Miguel, Alex and Wedson will be maintaining the Rust support.
--- MAINTAINERS | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
MAINTAINERS +14 -0
@@ -15894,6 +15894,20 @@ L: [email protected] S: Maintained F: drivers/infiniband/ulp/rtrs/ +RUST +M: Miguel Ojeda <[email protected]> +M: Alex Gaynor <[email protected]> +M: Wedson Almeida Filho <[email protected]> +L: [email protected] +S: Supported +W: https://github.com/Rust-for-Linux/linux +B: https://github.com/Rust-for-Linux/linux/issues +T: git https://github.com/Rust-for-Linux/linux.git rust-next +F: rust/ +F: samples/rust/ +F: Documentation/rust/ +K: \b(?i:rust)\b + RXRPC SOCKETS (AF_RXRPC) M: David Howells <[email protected]> M: Marc Dionne <[email protected]>
Signature · 1 line
2.32.0
Re: [PATCH 16/17] MAINTAINERS: Rust
Quoted 16 lines
--- MAINTAINERS | 14 ++++++++++++++ 1 file changed, 14 insertions(+)MAINTAINERS +4
-0@@ -15894,6 +15894,20 @@ L: [email protected] S: Maintained F: drivers/infiniband/ulp/rtrs/ +RUST +M: Miguel Ojeda <[email protected]> +M: Alex Gaynor <[email protected]> +M: Wedson Almeida Filho <[email protected]>
MAINTAINERS L: [email protected] · line 15910
Quoted 10 lines
MAINTAINERS +10
-0+L: [email protected] +S: Supported +W: https://github.com/Rust-for-Linux/linux +B: https://github.com/Rust-for-Linux/linux/issues +T: git https://github.com/Rust-for-Linux/linux.git rust-next +F: rust/ +F: samples/rust/ +F: Documentation/rust/ +K: \b(?i:rust)\b +
Does this mean that development will be done exclusively on GitHub? If affirmative, is there a bot that will replicate the messages in the mailing list?
Quoted 6 lines
MAINTAINERS +0
-1RXRPC SOCKETS (AF_RXRPC) M: David Howells <[email protected]> M: Marc Dionne <[email protected]> --2.32.0
Re: [PATCH 16/17] MAINTAINERS: Rust
Quoted 4 lines · Pedro Tammela
Does this mean that development will be done exclusively on GitHub? If affirmative, is there a bot that will replicate the messages in the mailing list?
Development has happened mainly in GitHub so far, but we have accepted patches from the ML and we can do their reviews there too.
We do not have a bot to replicate reviews into the mailing list nor any fancy bidirectional sync, though.
If this is deemed unacceptable by Linus et. al., we can do development in the ML as usual -- we do not really mind one way or the other (and, in fact, we discussed moving things to a git.kernel.org tree too).
The main point of GitHub/GitLab/etc. for us so far is having an issue tracker and some kind of CI.
Cheers,
Miguel
[email protected]
maintainerRFC PATCH 17/17 Android: Binder IPC in Rust (WIP)
In reply to [email protected]From: Miguel Ojeda <[email protected]>
A port to Rust of the Android Binder IPC mechanism.
This driver is a work in progress and will be sent for review later on, as well as separately from the Rust support.
However, we include it in so that we can show how an actual working module written in Rust may look like.
--- drivers/android/Kconfig | 7 + drivers/android/Makefile | 2 + drivers/android/allocation.rs | 264 ++++++++ drivers/android/context.rs | 80 +++ drivers/android/defs.rs | 99 +++ drivers/android/node.rs | 476 ++++++++++++++ drivers/android/process.rs | 972 ++++++++++++++++++++++++++++ drivers/android/range_alloc.rs | 189 ++++++ drivers/android/rust_binder.rs | 114 ++++ drivers/android/thread.rs | 857 ++++++++++++++++++++++++ drivers/android/transaction.rs | 328 ++++++++++ include/uapi/linux/android/binder.h | 28 +- 12 files changed, 3403 insertions(+), 13 deletions(-) create mode 100644 drivers/android/allocation.rs create mode 100644 drivers/android/context.rs create mode 100644 drivers/android/defs.rs create mode 100644 drivers/android/node.rs create mode 100644 drivers/android/process.rs create mode 100644 drivers/android/range_alloc.rs create mode 100644 drivers/android/rust_binder.rs create mode 100644 drivers/android/thread.rs create mode 100644 drivers/android/transaction.rs
drivers/android/Kconfig +7 -0
@@ -20,6 +20,13 @@ config ANDROID_BINDER_IPC Android process, using Binder to identify, invoke and pass arguments between said processes. +config ANDROID_BINDER_IPC_RUST + bool "Android Binder IPC Driver in Rust" + depends on MMU && RUST + default n + help + Implementation of the Binder IPC in Rust. + config ANDROID_BINDERFS bool "Android Binderfs filesystem" depends on ANDROID_BINDER_IPC
drivers/android/Makefile +2 -0
@@ -4,3 +4,5 @@ ccflags-y += -I$(src) # needed for trace events obj-$(CONFIG_ANDROID_BINDERFS) += binderfs.o obj-$(CONFIG_ANDROID_BINDER_IPC) += binder.o binder_alloc.o obj-$(CONFIG_ANDROID_BINDER_IPC_SELFTEST) += binder_alloc_selftest.o + +obj-$(CONFIG_ANDROID_BINDER_IPC_RUST) += rust_binder.o
drivers/android/allocation.rs new file +264 -0
264 lines not on this page · raw message
drivers/android/context.rs new file +80 -0
80 lines not on this page · raw message
drivers/android/defs.rs new file +99 -0
99 lines not on this page · raw message
drivers/android/node.rs new file +476 -0
476 lines not on this page · raw message
drivers/android/process.rs new file +972 -0
972 lines not on this page · raw message
drivers/android/range_alloc.rs new file +189 -0
189 lines not on this page · raw message
drivers/android/rust_binder.rs new file +114 -0
114 lines not on this page · raw message
drivers/android/thread.rs new file +857 -0
788 lines not on this page · raw message
drivers/android/transaction.rs new file +328 -0
328 lines not on this page · raw message
The rest of this diff is not shown. View the raw message.
Signature · 1 line
2.32.0
Part of the diff is not shown here. View the raw message.
Quoted 3 lines · On Sun, Jul 4, 2021 at 10:28 PM <[email protected]> wrote:
This is the patch series to add support for Rust as a second language to the Linux kernel.
In case somebody wonders: patch 07 was particularly big and lore does not seem to be showing it.
Let me know if you want me to split it; otherwise I will ask Konstantin when he is back from holidays.
Quoted 6 lines
- Many folks that have reported issues, tested the project, helped spread the word, joined discussions and contributed in other ways! In no particular order: Chenguang Wang, Greg Morenz, John Baublitz, Leah Leshchinsky, Caedin Cook, Liam Arzola, Fabio Aiuto, Hanqing Zhao, Robin Randhawa, Michal Rostecki, Wei Liu...
I forgot to mention Samantha Miller, Santosh Sivaraj, Vegard Nossum and Leandro Coutinho.
My apologies!
Cheers,
Miguel
Not sure if this is just a state of the union post or if you actually want to submit it. If the later I strongly disagree with merging it, as you should refined it and prove it actually is useful first. Where useful would be a real-life driver like say nvme or a usb host controller driver that actually works and shows benefits over the existing one. Until it shows such usefulness it will just drag everyone else down.
Quoted 3 lines · Christoph Hellwig
Not sure if this is just a state of the union post or if you actually want to submit it. If the later I strongly disagree with merging it,
We are submitting it.
Quoted 5 lines
as you should refined it and prove it actually is useful first. Where useful would be a real-life driver like say nvme or a usb host controller driver that actually works and shows benefits over the existing one. Until it shows such usefulness it will just drag everyone else down.
It is "proven" in the sense we are already starting to get users downstream and other interested parties have shown support.
However, others are more conservative and will only start investing into it if we are in mainline, i.e. if the decision of having Rust or not is already taken.
But it is fair to ask for a hardware driver to see how it looks like, which is why the cover letter mentions we are working on one.
Now, if you are OK with non-hardware modules, you can take a look at Rust Binder (last patch in the series) which is a non-trivial module and it is already working.
Cheers,
Miguel
Greg Kroah-Hartman
maintainerQuoted 3 lines · Miguel Ojeda
Now, if you are OK with non-hardware modules, you can take a look at Rust Binder (last patch in the series) which is a non-trivial module and it is already working.
Cool, does it actually pass the binder self-tests that the Android systems have for the codebase? Last I looked at this thing, it was not feature-complete compared to the in-kernel binder code, has that been resolved and the needed filesystem changes added?
thanks,
greg k-h
Quoted 7 lines · Greg Kroah-Hartman
Quoted 3 lines · Miguel Ojeda
Now, if you are OK with non-hardware modules, you can take a look at Rust Binder (last patch in the series) which is a non-trivial module and it is already working.
Cool, does it actually pass the binder self-tests that the Android systems have for the codebase?
We haven't run the Android tests yet because they depend on an Android-specific service (servicemanager) running and other Android-specific libraries. What we are doing instead is adding binder tests that don't depend on anything Android-specific; in fact, we are putting them in tools/testing/selftests/binder so that they can run on any vanilla system.
The commit is available here:
https://github.com/wedsonaf/linux/commit/f90ec49be9207fa765f07ad1071210ad871712ac
The tests are written in C and run successfully against both C and Rust drivers. I still have another ~20 tests that I wrote in another harness that I will convert to selftests soon, but the two together I believe have more coverage than the ones in Android.
We also have a trivial latency benchmark (ping with no payload) where the Rust version performs better than the C one.
The benchmark is available here: https://github.com/wedsonaf/linux/commits/ping
Quoted 3 lines
Last I looked at this thing, it was not
feature-complete compared to the in-kernel binder code, has that been resolved and the needed filesystem changes added?
It is not feature-complete in comparison to the C one just yet, it is missing a few things but not for any fundamental reason -- we were mostly focusing on the kernel crate and tests.
Miguel's point is that it does implement the vast majority of binder features and is non-trivial, so it could be used as evidence that useful kernel drivers can be built with Rust; not just "transpiled" from C, but written with the Rust safety guarantees.
Cheers,
-Wedson
Quoted 4 lines · Wedson Almeida Filho
Miguel's point is that it does implement the vast majority of binder features and is non-trivial, so it could be used as evidence that useful kernel drivers can be built with Rust; not just "transpiled" from C, but written with the Rust safety guarantees.
binder is not a kernel driver. It is a ill design IPC mechanism absolutely no representative for other kernel code.
Please write an actual real driver dealing with real (and common) hardware and come back with the results.
Greg Kroah-Hartman
maintainerQuoted 7 lines · Wedson Almeida Filho
Quoted 3 lines
Last I looked at this thing, it was not
feature-complete compared to the in-kernel binder code, has that been resolved and the needed filesystem changes added?It is not feature-complete in comparison to the C one just yet, it is missing a few things but not for any fundamental reason -- we were mostly focusing on the kernel crate and tests.
I love it how you call "binderfs is missing" a "few things" :)
Quoted 4 lines
Miguel's point is that it does implement the vast majority of binder features and is non-trivial, so it could be used as evidence that useful kernel drivers can be built with Rust; not just "transpiled" from C, but written with the Rust safety guarantees.
As Christoph said, and I and others have said before, binder is in no way shape or form anything that resembles any sort of "driver" at all. It is a crazy IPC mechanism that is tacked onto the side of the kernel. Not to say that it doesn't have its usages, but the interactions between binder and the rest of the kernel are very small and specific. Something that almost no one else will ever write again.
Please work on a real driver to help prove, or disprove, that this all is going to be able to work properly. There are huge unanswered questions that need to be resolved that you will run into when you do such a thing.
Good luck!
greg k-h
Quoted 27 lines · Greg Kroah-Hartman
Quoted 7 lines · Wedson Almeida Filho
Quoted 3 lines
Last I looked at this thing, it was not
feature-complete compared to the in-kernel binder code, has that been resolved and the needed filesystem changes added?It is not feature-complete in comparison to the C one just yet, it is missing a few things but not for any fundamental reason -- we were mostly focusing on the kernel crate and tests.
I love it how you call "binderfs is missing" a "few things" :)
Quoted 4 lines
Miguel's point is that it does implement the vast majority of binder features and is non-trivial, so it could be used as evidence that useful kernel drivers can be built with Rust; not just "transpiled" from C, but written with the Rust safety guarantees.
As Christoph said, and I and others have said before, binder is in no way shape or form anything that resembles any sort of "driver" at all. It is a crazy IPC mechanism that is tacked onto the side of the kernel. Not to say that it doesn't have its usages, but the interactions between binder and the rest of the kernel are very small and specific. Something that almost no one else will ever write again.
Quoted 5 lines
Please work on a real driver to help prove, or disprove, that this all is going to be able to work properly. There are huge unanswered questions that need to be resolved that you will run into when you do such a thing.
There is a more general use driver (network dummy) still in the making, It is fully operational, just the documentation of the rust bindings are not finished yet, so it is not merged into the rust tree yet, also I have to rebase it. I testet the network dummy (dummyrs) driver and for stupid tests like Iperf3 or ping it performed slightly better that the C version. See https://github.com/Kloenk/linux/blob/rust-netdevice/drivers/net/dummy_rs.rs for the actually driver.
Yes this driver is still only virtually, but the network apis are probably more general usage than the binder apis.
Quoted 3 lines
Good luck!
greg k-h
CU,
Finn
Quoted 1 line · Finn Behrens
There is a more general use driver (network dummy) still in the making, It is fully operational, just the documentation of the rust bindings are not finished yet, so it is not merged into the rust tree yet, also I have to rebase it.
Why are you so resistant to writing a real driver that deals with actual hardware? A simple NVMe driver is less than a thousand lines of C. I know the one in the kernel now is ridiculously complicated and has been thoroughly messed up with abstractions to support NVMeoF instead of having a separate driver, but it's really a simple interface at heart.
Re: [PATCH 00/17] Rust support, NVMe
Quoted 7 lines · Matthew Wilcox
Quoted 4 lines · Finn Behrens
There is a more general use driver (network dummy) still in the making, It is fully operational, just the documentation of the rust bindings are not finished yet, so it is not merged into the rust tree yet, also I have to rebase it.
Why are you so resistant to writing a real driver that deals with actual hardware?
} Why are we so resistant to writing a real driver that deals with actual hardware?
It is not dodging hardware, it is universal available.
Quoted 4 lines
A simple NVMe driver is less than a thousand lines of C.
I know the one in the kernel now is ridiculously complicated and has been thoroughly messed up with abstractions to support NVMeoF instead of having a separate driver, but it's really a simple interface at heart.
Summary of https://qemu.readthedocs.io/en/latest/system/nvme.html
NVMe hardware is universal available.
Groeten
Geert Stappers
Signature · 1 line
Silence is hard to parse
Re: [PATCH 00/17] Rust support, NVMe
Quoted 18 lines · Geert Stappers
Quoted 7 lines · Matthew Wilcox
Quoted 4 lines · Finn Behrens
There is a more general use driver (network dummy) still in the making, It is fully operational, just the documentation of the rust bindings are not finished yet, so it is not merged into the rust tree yet, also I have to rebase it.
Why are you so resistant to writing a real driver that deals with actual hardware?
} Why are we so resistant to writing a real driver that deals with actual hardware?
It is not dodging hardware, it is universal available.
Quoted 9 lines
Quoted 4 lines
A simple NVMe driver is less than a thousand lines of C.
I know the one in the kernel now is ridiculously complicated and has been thoroughly messed up with abstractions to support NVMeoF instead of having a separate driver, but it's really a simple interface at heart.Summary of https://qemu.readthedocs.io/en/latest/system/nvme.html
NVMe hardware is universal available.
I actually don’t have a clue how nvme works, that is my reason to not write a nvme driver. But, I’m currently writing a rust version of e1000. This is also available in qemu and is real hardware. This is sadly taking quite some time, as I don’t have a good linux kernel development setup currently.
Quoted 7 lines
Groeten
Geert StappersSignature · 1 line
Silence is hard to parse
CU,
Finn
Hey Matthew,
Quoted 2 lines · Matthew Wilcox
Why are you so resistant to writing a real driver that deals with actual hardware?
I don't think it was so much resistance but rather a prioritisation thing. Have you by any chance seen the gpio driver I posted a couple of days ago?
Quoted 4 lines
A simple NVMe driver is less than a thousand lines of C.
I know the one in the kernel now is ridiculously complicated and has been thoroughly messed up with abstractions to support NVMeoF instead of having a separate driver, but it's really a simple interface at heart.
The latest NVMe spec is 452 pages long, which seems to contradict your claim that it's simple. In any case, translating less than 1K lines of C shouldn't be too hard (after I've built the abstractions, of course). Would you mind sharing the simple driver you mention above?
Thanks,
-Wedson
Quoted 8 lines · Wedson Almeida Filho
Hey Matthew,
Quoted 2 lines · Matthew Wilcox
Why are you so resistant to writing a real driver that deals with actual hardware?
I don't think it was so much resistance but rather a prioritisation thing. Have you by any chance seen the gpio driver I posted a couple of days ago?
I haven't seen it, no ...
Quoted 7 lines
Quoted 4 lines
A simple NVMe driver is less than a thousand lines of C.
I know the one in the kernel now is ridiculously complicated and has been thoroughly messed up with abstractions to support NVMeoF instead of having a separate driver, but it's really a simple interface at heart.The latest NVMe spec is 452 pages long, which seems to contradict your claim that it's simple.
As I said, they've put all kinds of crap into NVMe these days. If you look at the 1.0e spec, it's 127 pages.
Quoted 3 lines
In any case, translating less than 1K lines of C shouldn't be
too hard (after I've built the abstractions, of course). Would you mind sharing the simple driver you mention above?
Unfortunately, most of the early versions were lost during the kernel.org breakin. If you check out commit 5da273fe3fd1 and look at drivers/block/nvme.c, you'll see a driver that's about 2000 lines.
Re: [PATCH 00/17] Rust support, GPIO driver
Quoted 11 lines · Matthew Wilcox
Quoted 8 lines · Wedson Almeida Filho
Hey Matthew,
Quoted 2 lines · Matthew Wilcox
Why are you so resistant to writing a real driver that deals with actual hardware?
I don't think it was so much resistance but rather a prioritisation thing. Have you by any chance seen the gpio driver I posted a couple of days ago?
I haven't seen it, no ...
Most likely it is https://lwn.net/Articles/863459/
Groeten
Geert Stappers
Signature · 1 line
Silence is hard to parse
Quoted 10 lines · Wedson Almeida Filho
Quoted 4 lines · Matthew Wilcox
A simple NVMe driver is less than a thousand lines of C.
I know the one in the kernel now is ridiculously complicated and has been thoroughly messed up with abstractions to support NVMeoF instead of having a separate driver, but it's really a simple interface at heart.The latest NVMe spec is 452 pages long, which seems to contradict your claim that it's simple. In any case, translating less than 1K lines of C shouldn't be too hard (after I've built the abstractions, of course). Would you mind sharing the simple driver you mention above?
You can use the 1.0 spec, which is much shorter. A 1.0 capable driver should be forward compatible with newer devices, too.
The current nvme driver became less simple since blk-mq integration and has only gotten more complicated since then with other transports and more advanced features. For a simpler example, you can reference an in kernel version <= 3.16, and ignore the "nvme-scsi" parts.
On Sun, Jul 04, 2021 at 10:27PM +0200, [email protected] wrote:
[...]
Quoted 1 line
rust: add
alloccrate
[...]
I think this patch never made it to the mailing list. b4 also doesn't find it:
ERROR: missing [7/17]
Quoted 5 lines · Marco Elver
I think this patch never made it to the mailing list. b4 also doesn't find it:
ERROR: missing [7/17]
Yeah, thanks -- I mentioned in my reply above:
https://lore.kernel.org/lkml/CANiq72nQq8Y8v9Pyf7JFq6Kf-+doNP+mHAFNzj_cSFBa3KwS5w@mail.gmail.com/
If you want to try the patches, you can pick them from this tree (the one in linux-next):
https://github.com/Rust-for-Linux/linux/tree/rust-next
Cheers,
Miguel