Rust for 6.6

Hi Linus,

This is the next round of the Rust support.

The majority of the commits have been in linux-next for 10+ days. The younger ones have been there for 3 rounds now, and thus for about a week by the time you start merging pulls.

A single line conflict is expected with the kselftest/kunit tree, because it contains a series that adds support for testing example Rust code within documentation as KUnit tests. The correct resolution is now on 'linux-next':

    diff --cc rust/kernel/init.rs
    index 0071b2834b78,e05563aad2ed..4ebb6f23fc2e
    --- a/rust/kernel/init.rs
    +++ b/rust/kernel/init.rs
    @@@ -517,11 -522,9 +526,10 @@@ macro_rules! stack_try_pin_init 
      /// For instance:
      ///
      /// ```rust
    - /// # use kernel::pin_init;
    - /// # use macros::{Zeroable, pin_data};
     -/// # use kernel::{macros::pin_data, pin_init};
    ++/// # use kernel::{macros::{Zeroable, pin_data}, pin_init};
      /// # use core::{ptr::addr_of_mut, marker::PhantomPinned};
      /// #[pin_data]
     +/// #[derive(Zeroable)]
      /// struct Buf {
      ///     // `ptr` points into `buf`.
      ///     ptr: *mut u8,

I have tested this pull together with the KUnit one, and whether you merge one or the other first should keep all examples compiling and passing.

No changes to the C side.

Please pull for v6.6 -- thanks!

Cheers,
Miguel

The following changes since commit 52a93d39b17dc7eb98b6aa3edb93943248e03b2f:

Linux 6.5-rc5 (2023-08-06 15:07:51 -0700)

are available in the Git repository at:

https://github.com/Rust-for-Linux/linux.git tags/rust-6.6

for you to fetch changes up to 4af84c6a85c63bec24611e46bb3de2c0a6602a51:

rust: init: update expanded macro explanation (2023-08-21 14:31:49 +0200)

----------------------------------------------------------------
Rust changes for v6.6

In terms of lines, most changes this time are on the pinned-init API and infrastructure. While we have a Rust version upgrade, and thus a bunch of changes from the vendored 'alloc' crate as usual, this time those do not account for many lines.

Toolchain and infrastructure:

- Upgrade to Rust 1.71.1. This is the second such upgrade, which is a smaller jump compared to the last time.

This version allows us to remove the '__rust_*' allocator functions -- the compiler now generates them as expected, thus now our 'KernelAllocator' is used.

It also introduces the 'offset_of!' macro in the standard library (as an unstable feature) which we will need soon. So far, we were using a declarative macro as a prerequisite in some not-yet-landed patch series, which did not support sub-fields (i.e. nested structs):

       #[repr(C)]
       struct S {
           a: u16,
           b: (u8, u8),
       }

       assert_eq!(offset_of!(S, b.1), 3);

- Upgrade to bindgen 0.65.1. This is the first time we upgrade its version.

Given it is a fairly big jump, it comes with a fair number of improvements/changes that affect us, such as a fix needed to support LLVM 16 as well as proper support for '__noreturn' C functions, which are now mapped to return the '!' type in Rust:

void __noreturn f(void); // C
pub fn f() -> !; // Rust

- 'scripts/rust_is_available.sh' improvements and fixes.

This series takes care of all the issues known so far and adds a few new checks to cover for even more cases, plus adds some more help texts. All this together will hopefully make problematic setups easier to identify and to be solved by users building the kernel.

In addition, it adds a test suite which covers all branches of the shell script, as well as tests for the issues found so far.

- Support rust-analyzer for out-of-tree modules too.

- Give 'cfg's to rust-analyzer for the 'core' and 'alloc' crates.

- Drop 'scripts/is_rust_module.sh' since it is not needed anymore.

Macros crate:

- New 'paste!' proc macro.

This macro is a more flexible version of 'concat_idents!': it allows the resulting identifier to be used to declare new items and it allows to transform the identifiers before concatenating them, e.g.

       let x_1 = 42;
       paste!(let [<x _2>] = [<x _1>];);
       assert!(x_1 == x_2);

The macro is then used for several of the pinned-init API changes in this pull.

Pinned-init API:

- Make '#[pin_data]' compatible with conditional compilation of fields, allowing to write code like:

       #[pin_data]
       pub struct Foo {
           #[cfg(CONFIG_BAR)]
           a: Bar,
           #[cfg(not(CONFIG_BAR))]
           a: Baz,
       }

- New '#[derive(Zeroable)]' proc macro for the 'Zeroable' trait, which allows 'unsafe' implementations for structs where every field implements the 'Zeroable' trait, e.g.:

       #[derive(Zeroable)]
       pub struct DriverData {
           id: i64,
           buf_ptr: *mut u8,
           len: usize,
       }

- Add '..Zeroable::zeroed()' syntax to the 'pin_init!' macro for zeroing all other fields, e.g.:

       pin_init!(Buf {
           buf: [1; 64],
           ..Zeroable::zeroed()
       });

- New '{,pin_}init_array_from_fn()' functions to create array initializers given a generator function, e.g.:

       let b: Box<[usize; 1_000]> = Box::init::<Error>(
           init_array_from_fn(|i| i)
       ).unwrap();

       assert_eq!(b.len(), 1_000);
       assert_eq!(b[123], 123);

- New '{,pin_}chain' methods for '{,Pin}Init<T, E>' that allow to execute a closure on the value directly after initialization, e.g.:

       let foo = init!(Foo {
           buf <- init::zeroed()
       }).chain(|foo| {
           foo.setup();
           Ok(())
       });

- Support arbitrary paths in init macros, instead of just identifiers and generic types.

- Implement the 'Zeroable' trait for the 'UnsafeCell<T>' and 'Opaque<T>' types.

- Make initializer values inaccessible after initialization.

- Make guards in the init macros hygienic.

'allocator' module:

- Use 'krealloc_aligned()' in 'KernelAllocator::alloc' preventing misaligned allocations when the Rust 1.71.1 upgrade is applied later in this pull.

The equivalent fix for the previous compiler version (where 'KernelAllocator' is not yet used) was merged into 6.5 already, which added the 'krealloc_aligned()' function used here.

- Implement 'KernelAllocator::{realloc, alloc_zeroed}' for performance, using 'krealloc_aligned()' too, which forwards the call to the C API.

'types' module:

- Make 'Opaque' be '!Unpin', removing the need to add a 'PhantomPinned' field to Rust structs that contain C structs which must not be moved.

- Make 'Opaque' use 'UnsafeCell' as the outer type, rather than inner.

Documentation:

- Suggest obtaining the source code of the Rust's 'core' library using the tarball instead of the repository.

MAINTAINERS:

- Andreas and Alice, from Samsung and Google respectively, are joining as reviewers of the "RUST" entry.

As well as a few other minor changes and cleanups.

----------------------------------------------------------------
Aakash Sen Sharma (1):
rust: bindgen: upgrade to 0.65.1

Alice Ryhl (1):
rust: make UnsafeCell the outer type in Opaque

Andrea Righi (1):
btf, scripts: rust: drop is_rust_module.sh

Ariel Miculas (1):
rust: helpers: sort includes alphabetically in rust/helpers.c

Ben Gooding (1):
rust: lock: Add intra-doc links to the Backend trait

Benno Lossin (14):
rust: types: make Opaque be !Unpin
rust: init: consolidate init macros
rust: init: make #[pin_data] compatible with conditional compilation of fields
rust: add derive macro for Zeroable
rust: init: make guards in the init macros hygienic
rust: init: wrap type checking struct initializers in a closure
rust: init: make initializer values inaccessible after initializing
rust: init: add ..Zeroable::zeroed() syntax for zeroing all missing fields
rust: init: add functions to create array initializers
rust: init: add support for arbitrary paths in init macros
rust: init: implement Zeroable for UnsafeCell<T> and Opaque<T>
rust: init: make PinInit<T, E> a supertrait of Init<T, E>
rust: init: add {pin_}chain functions to {Pin}Init<T, E>
rust: init: update expanded macro explanation

Björn Roy Baron (1):
rust: alloc: Add realloc and alloc_zeroed to the GlobalAlloc impl

Boqun Feng (1):
rust: allocator: Use krealloc_aligned() in KernelAllocator::alloc

Gary Guo (1):
rust: macros: add paste! proc macro

Guillaume Plourde (1):
docs: rust: add command line to rust-analyzer section

Martin Rodriguez Reboredo (1):
scripts: generate_rust_analyzer: provide cfgs for core and alloc

Masahiro Yamada (1):
kbuild: rust_is_available: remove -v option

Miguel Ojeda (13):
MAINTAINERS: add Andreas Hindborg as Rust reviewer
MAINTAINERS: add Alice Ryhl as Rust reviewer
docs: rust: add paragraph about finding a suitable libclang
kbuild: rust_is_available: print docs reference
kbuild: rust_is_available: add check for bindgen invocation
kbuild: rust_is_available: check that environment variables are set
kbuild: rust_is_available: fix confusion when a version appears in the path
kbuild: rust_is_available: normalize version matching
kbuild: rust_is_available: handle failures calling $RUSTC/$BINDGEN
kbuild: rust_is_available: check that output looks as expected
kbuild: rust_is_available: add test suite
rust: upgrade to Rust 1.71.1
rust: enable no_mangle_with_rust_abi Clippy lint

Russell Currey (1):
kbuild: rust_is_available: fix version check when CC has multiple arguments

Trevor Gross (2):
docs: rust: update instructions for obtaining 'core' source
docs: rust: clarify what 'rustup override' does

Vinay Varma (1):
scripts: make rust-analyzer for out-of-tree modules

 Documentation/process/changes.rst  |   4 +-
 Documentation/rust/quick-start.rst |  42 ++-
 MAINTAINERS                        |   2 +
 Makefile                           |  16 +-
 rust/Makefile                      |  17 +-
 rust/alloc/alloc.rs                |  20 +-
 rust/alloc/boxed.rs                | 131 ++++----
 rust/alloc/lib.rs                  |  50 +--
 rust/alloc/raw_vec.rs              |  18 +-
 rust/alloc/slice.rs                |  43 ++-
 rust/alloc/vec/drain.rs            |   8 +-
 rust/alloc/vec/drain_filter.rs     |   8 +-
 rust/alloc/vec/into_iter.rs        |  35 +-
 rust/alloc/vec/mod.rs              |  84 ++---
 rust/compiler_builtins.rs          |   7 +
 rust/helpers.c                     |  21 +-
 rust/kernel/allocator.rs           |  84 ++---
 rust/kernel/init.rs                | 646 ++++++++++++++++---------------------
 rust/kernel/init/__internal.rs     |  39 +--
 rust/kernel/init/macros.rs         | 519 +++++++++++++++++++++++++----
 rust/kernel/lib.rs                 |   3 -
 rust/kernel/prelude.rs             |   2 +-
 rust/kernel/sync/lock.rs           |   6 +-
 rust/kernel/types.rs               |  21 +-
 rust/macros/lib.rs                 | 117 +++++++
 rust/macros/module.rs              |   2 +-
 rust/macros/paste.rs               |  96 ++++++
 rust/macros/quote.rs               |  12 +
 rust/macros/zeroable.rs            |  72 +++++
 scripts/Makefile.modfinal          |   2 -
 scripts/generate_rust_analyzer.py  |  39 ++-
 scripts/is_rust_module.sh          |  16 -
 scripts/min-tool-version.sh        |   4 +-
 scripts/rust_is_available.sh       | 233 +++++++++----
 scripts/rust_is_available_test.py  | 346 ++++++++++++++++++++
 35 files changed, 1915 insertions(+), 850 deletions(-)
 create mode 100644 rust/macros/paste.rs
 create mode 100644 rust/macros/zeroable.rs
 delete mode 100755 scripts/is_rust_module.sh
 create mode 100755 scripts/rust_is_available_test.py