Skip to contentSkip to conversation

`syn` rewrite of pin-init

PATCH Applied Benno Lossin 2 participants 19 messages Jan 16 – Jan 17

Subsystems (3)
lore
Share & tools
lore

Benno Lossin

author maintainer
#0 lore raw

v4 cover `syn` rewrite of pin-init

Rewrite the proc-macros of pin-init by using the syn crate for Rust syntax parsing. This series has been a long way coming. At the very start of pin-init, I initially implemented everything using syn, since parsing is so much easier with it. Now after several years it is finally time to remove the dreaded 1600 lines of declarative macros required to parse and expand the initializer syntax.

The move to syn is not only a blessing for the maintenance work, but also improves the implementation of new features. This series includes many such improvements.

* Patch 1, 2, 3 and 4 prepare for the rewrite. The first removes the superfluous try_ variants of the initializer macros from pin-init. Note that the kernel defines its own, so no code changes in the kernel are required. The second patch allows using ::pin_init in the pin-init crate. The third adds the syn dependency and cleans up some old workarounds and new clippy warnings. The fourth adds better error handling on top of syn.
* Patch 5, 6, 7, and 9 rewrite the derive macros for Zeroable, #[pinned_drop] attribute macro, #[pin_data] attribute macro, and the initializer macros respectively using syn.
* Patch 8 ensures soundness in the future by fixing generic bounds in generated code by #[pin_data].
* Patch 10 adds the #[default_error(type)] attribute to initializer macros allowing them to specify a default error that is used when no error is manually specified.
* Patch 11 uses #[default_error(type)] in the definition of the kernel's try_ variants of the initializer macros (which defaults to the kernel's Error type).
* Patch 12 allows putting attributes on fields in initializer macros (for example cfg attributes).
* Patch 13 adds #[disable_initialized_field_access] to support packed structs.
* Patch 14 simplifies the code generated by the initializer macros.
* Patch 15 adds Gary as a maintainer.

In addition to the new features, using syn results in much cleaner error messages and more accurate span information. The code is much easier to read as well and hopefully easier to understand as well.

As always, tests that ensure the correctness of the macro output are included and updated in the upstream pin-init repository. Take a look at the pull request on GitHub for the diff in the test output:

    https://github.com/Rust-for-Linux/pin-init/pull/89

I would greatly appreciate Tested-by's from people actively using pin-init weather in the kernel or outside.

The commit adding syn support to pin-init is also used by Gary's series to rewrite the macros crate using syn:

    https://lore.kernel.org/all/[email protected]

My series can be found at:

https://github.com/BennoLossin/linux/commits/pin-init/sync

The rust-dev branch in the RfL repository will be updated to contain both patch series for ease of use & testing.

Link to previous versions:

- v3: https://lore.kernel.org/all/[email protected]
- v2: https://lore.kernel.org/all/[email protected]
- v1: https://lore.kernel.org/all/[email protected]

Cheers,
Benno

Benno Lossin (15):
rust: pin-init: remove try_ versions of the initializer macros
rust: pin-init: allow the crate to refer to itself as pin-init in
doc tests
rust: pin-init: add syn dependency and remove proc-macro[2] and
quote workarounds
rust: pin-init: internal: add utility API for syn error handling
rust: pin-init: rewrite derive(Zeroable) and derive(MaybeZeroable)
using syn
rust: pin-init: rewrite the #[pinned_drop] attribute macro using

    `syn`

rust: pin-init: rewrite #[pin_data] using syn
rust: pin-init: add ?Sized bounds to traits in #[pin_data] macro
rust: pin-init: rewrite the initializer macros using syn
rust: pin-init: add #[default_error(<type>)] attribute to
initializer macros
rust: init: use #[default_error(err)] for the initializer macros
rust: pin-init: internal: init: add support for attributes on
initializer fields
rust: pin-init: internal: init: add escape hatch for referencing
initialized fields
rust: pin-init: internal: init: simplify Zeroable safety check
MAINTAINERS: add Gary Guo to pin-init

 MAINTAINERS                               |    1 +
 rust/Makefile                             |   16 +-
 rust/kernel/init.rs                       |   40 +-
 rust/pin-init/README.md                   |    2 +-
 rust/pin-init/examples/linked_list.rs     |   19 +-
 rust/pin-init/examples/pthread_mutex.rs   |   10 +-
 rust/pin-init/internal/src/diagnostics.rs |   30 +
 rust/pin-init/internal/src/helpers.rs     |  152 --
 rust/pin-init/internal/src/init.rs        |  548 +++++++
 rust/pin-init/internal/src/lib.rs         |   48 +-
 rust/pin-init/internal/src/pin_data.rs    |  615 ++++++--
 rust/pin-init/internal/src/pinned_drop.rs |   88 +-
 rust/pin-init/internal/src/zeroable.rs    |  157 +-
 rust/pin-init/src/lib.rs                  |  173 +--
 rust/pin-init/src/macros.rs               | 1677 ---------------------
 scripts/generate_rust_analyzer.py         |    2 +-
 16 files changed, 1278 insertions(+), 2300 deletions(-)
 create mode 100644 rust/pin-init/internal/src/diagnostics.rs
 delete mode 100644 rust/pin-init/internal/src/helpers.rs
 create mode 100644 rust/pin-init/internal/src/init.rs
 delete mode 100644 rust/pin-init/src/macros.rs

base-commit: 9448598b22c50c8a5bb77a9103e2d49f134c9578

Signature · 1 line
2.52.0
author maintainer
#1 lore raw

PATCH v4 1/15 rust: pin-init: remove `try_` versions of the initializer macros

The try_[pin_]init! versions of the initializer macros are superfluous. Instead of forcing the user to always write an error in try_[pin_]init! and not allowing one in [pin_]init!, combine them into [pin_]init! that defaults the error to core::convert::Infallible, but also allows to specify a custom one.

Projects using pin-init still can provide their own defaulting initializers using the try_ prefix by using the #[default_error] attribute added in a future patch.

[ Adjust the definition of the kernel's version of the try_
initializer macros - Benno]

---
Changes in v4: none
Changes in v3: none
Changes in v2: none

---
 rust/kernel/init.rs                     |   8 +-
 rust/pin-init/README.md                 |   2 +-
 rust/pin-init/examples/linked_list.rs   |  19 ++--
 rust/pin-init/examples/pthread_mutex.rs |  10 +-
 rust/pin-init/src/lib.rs                | 118 ++++--------------------
 5 files changed, 35 insertions(+), 122 deletions(-)
rust/kernel/init.rs +4 -4
index 899b9a962762..917f7ef001fd 100644
@@ -222,14 +222,14 @@ macro_rules! try_init {
     ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? {
         $($fields:tt)*
     }) => {
-        ::pin_init::try_init!($(&$this in)? $t $(::<$($generics),*>)? {
+        ::pin_init::init!($(&$this in)? $t $(::<$($generics),*>)? {
             $($fields)*
         }? $crate::error::Error)
     };
     ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? {
         $($fields:tt)*
     }? $err:ty) => {
-        ::pin_init::try_init!($(&$this in)? $t $(::<$($generics),*>)? {
+        ::pin_init::init!($(&$this in)? $t $(::<$($generics),*>)? {
             $($fields)*
         }? $err)
     };
@@ -282,14 +282,14 @@ macro_rules! try_pin_init {
     ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? {
         $($fields:tt)*
     }) => {
-        ::pin_init::try_pin_init!($(&$this in)? $t $(::<$($generics),*>)? {
+        ::pin_init::pin_init!($(&$this in)? $t $(::<$($generics),*>)? {
             $($fields)*
         }? $crate::error::Error)
     };
     ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? {
         $($fields:tt)*
     }? $err:ty) => {
-        ::pin_init::try_pin_init!($(&$this in)? $t $(::<$($generics),*>)? {
+        ::pin_init::pin_init!($(&$this in)? $t $(::<$($generics),*>)? {
             $($fields)*
         }? $err)
     };
rust/pin-init/README.md +1 -1
index 74bbb4e0a2f7..6cee6ab1eb57 100644
@@ -135,7 +135,7 @@ struct DriverData {
 
 impl DriverData {
     fn new() -> impl PinInit<Self, Error> {
-        try_pin_init!(Self {
+        pin_init!(Self {
             status <- CMutex::new(0),
             buffer: Box::init(pin_init::init_zeroed())?,
         }? Error)
rust/pin-init/examples/linked_list.rs +9 -10
index f9e117c7dfe0..8445a5890cb7 100644
@@ -6,7 +6,6 @@
 
 use core::{
     cell::Cell,
-    convert::Infallible,
     marker::PhantomPinned,
     pin::Pin,
     ptr::{self, NonNull},
@@ -31,31 +30,31 @@ pub struct ListHead {
 
 impl ListHead {
     #[inline]
-    pub fn new() -> impl PinInit<Self, Infallible> {
-        try_pin_init!(&this in Self {
+    pub fn new() -> impl PinInit<Self> {
+        pin_init!(&this in Self {
             next: unsafe { Link::new_unchecked(this) },
             prev: unsafe { Link::new_unchecked(this) },
             pin: PhantomPinned,
-        }? Infallible)
+        })
     }
 
     #[inline]
     #[allow(dead_code)]
-    pub fn insert_next(list: &ListHead) -> impl PinInit<Self, Infallible> + '_ {
-        try_pin_init!(&this in Self {
+    pub fn insert_next(list: &ListHead) -> impl PinInit<Self> + '_ {
+        pin_init!(&this in Self {
             prev: list.next.prev().replace(unsafe { Link::new_unchecked(this)}),
             next: list.next.replace(unsafe { Link::new_unchecked(this)}),
             pin: PhantomPinned,
-        }? Infallible)
+        })
     }
 
     #[inline]
-    pub fn insert_prev(list: &ListHead) -> impl PinInit<Self, Infallible> + '_ {
-        try_pin_init!(&this in Self {
+    pub fn insert_prev(list: &ListHead) -> impl PinInit<Self> + '_ {
+        pin_init!(&this in Self {
             next: list.prev.next().replace(unsafe { Link::new_unchecked(this)}),
             prev: list.prev.replace(unsafe { Link::new_unchecked(this)}),
             pin: PhantomPinned,
-        }? Infallible)
+        })
     }
 
     #[inline]
rust/pin-init/examples/pthread_mutex.rs +5 -5
index 49b004c8c137..4e082ec7d5de 100644
@@ -98,11 +98,11 @@ fn init_raw() -> impl PinInit<UnsafeCell<libc::pthread_mutex_t>, Error> {
                 // SAFETY: mutex has been initialized
                 unsafe { pin_init_from_closure(init) }
             }
-            try_pin_init!(Self {
-            data: UnsafeCell::new(data),
-            raw <- init_raw(),
-            pin: PhantomPinned,
-        }? Error)
+            pin_init!(Self {
+                data: UnsafeCell::new(data),
+                raw <- init_raw(),
+                pin: PhantomPinned,
+            }? Error)
         }
 
         #[allow(dead_code)]
rust/pin-init/src/lib.rs +16 -102

191 lines not on this page · raw message

Signature · 1 line
2.52.0
author maintainer
#2 lore raw

PATCH v4 2/15 rust: pin-init: allow the crate to refer to itself as `pin-init` in doc tests

In reply to Benno Lossin

The syn approach requires use of ::pin_init::... instead of the $crate::... construct available to declarative macros. To be able to use the pin_init crate from itself (which includes doc tests), we have to declare it as such.

---
Changes in v4: none
Changes in v3: none
Changes in v2: none

---
 rust/pin-init/src/lib.rs | 5 +++++
 1 file changed, 5 insertions(+)
rust/pin-init/src/lib.rs +5 -0
index 8673008f45d2..0e707f00061f 100644
@@ -290,6 +290,11 @@
     ptr::{self, NonNull},
 };
 
+// This is used by doc-tests -- the proc-macros expand to `::pin_init::...` and without this the
+// doc-tests wouldn't have an extern crate named `pin_init`.
+#[allow(unused_extern_crates)]
+extern crate self as pin_init;
+
 #[doc(hidden)]
 pub mod __internal;
 #[doc(hidden)]
Signature · 1 line
2.52.0
author maintainer
#3 lore raw

PATCH v4 3/15 rust: pin-init: add `syn` dependency and remove `proc-macro[2]` and `quote` workarounds

In reply to Benno Lossin

syn makes parsing Rust from proc-macros a lot simpler. pin-init has not used syn up until now, because the we did not support it. That changed in commit 54e3eae85562 ("Merge patch series "syn support""), so we can finally utilize the added ergonomics of parsing proc-macro input with syn.

Previously we only had the proc-macro library available, whereas the user-space version also used proc-macro2 and quote. Now both are available, so remove the workarounds.

Due to these changes, clippy emits warnings about unnecessary .to_string() as proc-macro2 provides an additional PartialEq impl on Ident, so the warnings are fixed.

[ Adjusted wording from upstream version and added build system changes
for the kernel - Benno ]

---
This commit is also used in Gary's series to rewrite the macros crate using syn:

    https://lore.kernel.org/all/[email protected]

Changes in v4: none
Changes in v3: none
Changes in v2: none

---
 rust/Makefile                             | 16 ++++++++++------
 rust/pin-init/internal/src/helpers.rs     |  7 ++-----
 rust/pin-init/internal/src/lib.rs         | 16 ----------------
 rust/pin-init/internal/src/pin_data.rs    | 18 ++++++------------
 rust/pin-init/internal/src/pinned_drop.rs | 10 ++++------
 rust/pin-init/internal/src/zeroable.rs    |  6 ++----
 scripts/generate_rust_analyzer.py         |  2 +-
 7 files changed, 25 insertions(+), 50 deletions(-)
rust/Makefile +10 -6
index 5d357dce1704..82bda6ccfe39 100644
@@ -212,9 +212,10 @@ rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE
 
 rustdoc-pin_init_internal: private rustdoc_host = yes
 rustdoc-pin_init_internal: private rustc_target_flags = --cfg kernel \
-    --extern proc_macro --crate-type proc-macro
+    --extern proc_macro --extern proc_macro2 --extern quote --extern syn \
+    --crate-type proc-macro
 rustdoc-pin_init_internal: $(src)/pin-init/internal/src/lib.rs \
-    rustdoc-clean FORCE
+    rustdoc-clean rustdoc-proc_macro2 rustdoc-quote rustdoc-syn FORCE
 	+$(call if_changed,rustdoc)
 
 rustdoc-pin_init: private rustdoc_host = yes
@@ -273,9 +274,10 @@ rusttestlib-macros: $(src)/macros/lib.rs \
 	+$(call if_changed,rustc_test_library)
 
 rusttestlib-pin_init_internal: private rustc_target_flags = --cfg kernel \
-    --extern proc_macro
+    --extern proc_macro --extern proc_macro2 --extern quote --extern syn
 rusttestlib-pin_init_internal: private rustc_test_library_proc = yes
-rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs FORCE
+rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs \
+    rusttestlib-proc_macro2 rusttestlib-quote rusttestlib-syn FORCE
 	+$(call if_changed,rustc_test_library)
 
 rusttestlib-pin_init: private rustc_target_flags = --extern pin_init_internal \
@@ -547,8 +549,10 @@ $(obj)/$(libmacros_name): $(src)/macros/lib.rs $(obj)/libproc_macro2.rlib \
     $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE
 	+$(call if_changed_dep,rustc_procmacro)
 
-$(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel
-$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs FORCE
+$(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel \
+    --extern proc_macro2 --extern quote --extern syn
+$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs \
+    $(obj)/libproc_macro2.rlib $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE
 	+$(call if_changed_dep,rustc_procmacro)
 
 quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@
rust/pin-init/internal/src/helpers.rs +2 -5
index 236f989a50f2..90f85eaa4123 100644
@@ -1,9 +1,6 @@
 // SPDX-License-Identifier: Apache-2.0 OR MIT
 
-#[cfg(not(kernel))]
-use proc_macro2 as proc_macro;
-
-use proc_macro::{TokenStream, TokenTree};
+use proc_macro2::{TokenStream, TokenTree};
 
 /// Parsed generics.
 ///
@@ -101,7 +98,7 @@ pub(crate) fn parse_generics(input: TokenStream) -> (Generics, Vec<TokenTree>) {
                     1 => {
                         // Here depending on the token, it might be a generic variable name.
                         match tt.clone() {
-                            TokenTree::Ident(i) if at_start && i.to_string() == "const" => {
+                            TokenTree::Ident(i) if at_start && i == "const" => {
                                 let Some(name) = toks.next() else {
                                     // Parsing error.
                                     break;
rust/pin-init/internal/src/lib.rs +0 -16
index 297b0129a5bf..4c4dc639ce82 100644
@@ -7,27 +7,11 @@
 //! `pin-init` proc macros.
 
 #![cfg_attr(not(RUSTC_LINT_REASONS_IS_STABLE), feature(lint_reasons))]
-// Allow `.into()` to convert
-// - `proc_macro2::TokenStream` into `proc_macro::TokenStream` in the user-space version.
-// - `proc_macro::TokenStream` into `proc_macro::TokenStream` in the kernel version.
-//   Clippy warns on this conversion, but it's required by the user-space version.
-//
-// Remove once we have `proc_macro2` in the kernel.
-#![allow(clippy::useless_conversion)]
 // Documentation is done in the pin-init crate instead.
 #![allow(missing_docs)]
 
 use proc_macro::TokenStream;
 
-#[cfg(kernel)]
-#[path = "../../../macros/quote.rs"]
-#[macro_use]
-#[cfg_attr(not(kernel), rustfmt::skip)]
-mod quote;
-#[cfg(not(kernel))]
-#[macro_use]
-extern crate quote;
-
 mod helpers;
 mod pin_data;
 mod pinned_drop;
rust/pin-init/internal/src/pin_data.rs +6 -12
index 87d4a7eb1d35..86a53b37cc66 100644
@@ -1,10 +1,8 @@
 // SPDX-License-Identifier: Apache-2.0 OR MIT
 
-#[cfg(not(kernel))]
-use proc_macro2 as proc_macro;
-
 use crate::helpers::{parse_generics, Generics};
-use proc_macro::{Group, Punct, Spacing, TokenStream, TokenTree};
+use proc_macro2::{Group, Punct, Spacing, TokenStream, TokenTree};
+use quote::quote;
 
 pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
     // This proc-macro only does some pre-parsing and then delegates the actual parsing to
@@ -28,7 +26,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
     // The name of the struct with ty_generics.
     let struct_name = rest
         .iter()
-        .skip_while(|tt| !matches!(tt, TokenTree::Ident(i) if i.to_string() == "struct"))
+        .skip_while(|tt| !matches!(tt, TokenTree::Ident(i) if i == "struct"))
         .nth(1)
         .and_then(|tt| match tt {
             TokenTree::Ident(_) => {
@@ -65,7 +63,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
         .into_iter()
         .flat_map(|tt| {
             // We ignore top level `struct` tokens, since they would emit a compile error.
-            if matches!(&tt, TokenTree::Ident(i) if i.to_string() == "struct") {
+            if matches!(&tt, TokenTree::Ident(i) if i == "struct") {
                 vec![tt]
             } else {
                 replace_self_and_deny_type_defs(&struct_name, tt, &mut errs)
@@ -98,11 +96,7 @@ fn replace_self_and_deny_type_defs(
 ) -> Vec<TokenTree> {
     match tt {
         TokenTree::Ident(ref i)
-            if i.to_string() == "enum"
-                || i.to_string() == "trait"
-                || i.to_string() == "struct"
-                || i.to_string() == "union"
-                || i.to_string() == "impl" =>
+            if i == "enum" || i == "trait" || i == "struct" || i == "union" || i == "impl" =>
         {
             errs.extend(
                 format!(
@@ -119,7 +113,7 @@ fn replace_self_and_deny_type_defs(
             );
             vec![tt]
         }
-        TokenTree::Ident(i) if i.to_string() == "Self" => struct_name.clone(),
+        TokenTree::Ident(i) if i == "Self" => struct_name.clone(),
         TokenTree::Literal(_) | TokenTree::Punct(_) | TokenTree::Ident(_) => vec![tt],
         TokenTree::Group(g) => vec![TokenTree::Group(Group::new(
             g.delimiter(),
rust/pin-init/internal/src/pinned_drop.rs +4 -6
index c4ca7a70b726..cf8cd1c42984 100644
@@ -1,15 +1,13 @@
 // SPDX-License-Identifier: Apache-2.0 OR MIT
 
-#[cfg(not(kernel))]
-use proc_macro2 as proc_macro;
-
-use proc_macro::{TokenStream, TokenTree};
+use proc_macro2::{TokenStream, TokenTree};
+use quote::quote;
 
 pub(crate) fn pinned_drop(_args: TokenStream, input: TokenStream) -> TokenStream {
     let mut toks = input.into_iter().collect::<Vec<_>>();
     assert!(!toks.is_empty());
     // Ensure that we have an `impl` item.
-    assert!(matches!(&toks[0], TokenTree::Ident(i) if i.to_string() == "impl"));
+    assert!(matches!(&toks[0], TokenTree::Ident(i) if i == "impl"));
     // Ensure that we are implementing `PinnedDrop`.
     let mut nesting: usize = 0;
     let mut pinned_drop_idx = None;
@@ -27,7 +25,7 @@ pub(crate) fn pinned_drop(_args: TokenStream, input: TokenStream) -> TokenStream
         if i >= 1 && nesting == 0 {
             // Found the end of the generics, this should be `PinnedDrop`.
             assert!(
-                matches!(tt, TokenTree::Ident(i) if i.to_string() == "PinnedDrop"),
+                matches!(tt, TokenTree::Ident(i) if i == "PinnedDrop"),
                 "expected 'PinnedDrop', found: '{tt:?}'"
             );
             pinned_drop_idx = Some(i);
rust/pin-init/internal/src/zeroable.rs +2 -4
index e0ed3998445c..d8a5ef3883f4 100644
@@ -1,10 +1,8 @@
 // SPDX-License-Identifier: GPL-2.0
 
-#[cfg(not(kernel))]
-use proc_macro2 as proc_macro;
-
 use crate::helpers::{parse_generics, Generics};
-use proc_macro::{TokenStream, TokenTree};
+use proc_macro2::{TokenStream, TokenTree};
+use quote::quote;
 
 pub(crate) fn parse_zeroable_derive_input(
     input: TokenStream,
scripts/generate_rust_analyzer.py +1 -1
index 147d0cc94068..d31d93888658 100755
@@ -123,7 +123,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs, core_edit
     append_crate(
         "pin_init_internal",
         srctree / "rust" / "pin-init" / "internal" / "src" / "lib.rs",
-        [],
+        ["std", "proc_macro", "proc_macro2", "quote", "syn"],
         cfg=["kernel"],
         is_proc_macro=True,
     )
Signature · 1 line
2.52.0
author maintainer
#4 lore raw

PATCH v4 4/15 rust: pin-init: internal: add utility API for syn error handling

In reply to Benno Lossin

The API is similar to diagnostics handling in rustc and uses a ErrorGuaranteed value to signify that an error has been emitted. It supports both fatal errors (which abort the macro expansion immediately by returning Err(ErrorGuaranteed)) and non-fatal ones at at generation time. These errors are appended to the token stream after generation has finished normally. This allows giving good errors while still expanding most of the code as expected to avoid the user encountering additional errors (for example missing definitions).

---
Changes in v4: none
Changes in v3: switch to Gary's suggestion
Changes in v2: added this patch

---
 rust/pin-init/internal/src/diagnostics.rs | 32 +++++++++++++++++++++++
 rust/pin-init/internal/src/lib.rs         |  1 +
 2 files changed, 33 insertions(+)
 create mode 100644 rust/pin-init/internal/src/diagnostics.rs
rust/pin-init/internal/src/diagnostics.rs new file +32 -0
new file mode 100644
index 000000000000..555876c01bab
@@ -0,0 +1,32 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use std::fmt::Display;
+
+use proc_macro2::TokenStream;
+use syn::{spanned::Spanned, Error};
+
+pub(crate) struct DiagCtxt(TokenStream);
+pub(crate) struct ErrorGuaranteed(());
+
+impl DiagCtxt {
+    #[expect(dead_code)]
+    pub(crate) fn error(&mut self, span: impl Spanned, msg: impl Display) -> ErrorGuaranteed {
+        let error = Error::new(span.span(), msg);
+        self.0.extend(error.into_compile_error());
+        ErrorGuaranteed(())
+    }
+
+    #[expect(dead_code)]
+    pub(crate) fn with(
+        fun: impl FnOnce(&mut DiagCtxt) -> Result<TokenStream, ErrorGuaranteed>,
+    ) -> TokenStream {
+        let mut dcx = Self(TokenStream::new());
+        match fun(&mut dcx) {
+            Ok(mut stream) => {
+                stream.extend(dcx.0);
+                stream
+            }
+            Err(ErrorGuaranteed(())) => dcx.0,
+        }
+    }
+}
rust/pin-init/internal/src/lib.rs +1 -0
index 4c4dc639ce82..0e1a4724549d 100644
@@ -12,6 +12,7 @@
 
 use proc_macro::TokenStream;
 
+mod diagnostics;
 mod helpers;
 mod pin_data;
 mod pinned_drop;
Signature · 1 line
2.52.0
author maintainer
#5 lore raw

PATCH v4 5/15 rust: pin-init: rewrite `derive(Zeroable)` and `derive(MaybeZeroable)` using `syn`

In reply to Benno Lossin

Rewrite the two derive macros for Zeroable using syn. One positive side effect of this change is that tuple structs are now supported by them. Additionally, syntax errors and the error emitted when trying to use one of the derive macros on an enum are improved. Otherwise no functional changes intended.

For example:

    #[derive(Zeroable)]
    enum Num {
        A(u32),
        B(i32),
    }

Produced this error before this commit:

    error: no rules expected keyword `enum`
     --> tests/ui/compile-fail/zeroable/enum.rs:5:1
      |
    5 | enum Num {
      | ^^^^ no rules expected this token in macro call
      |
    note: while trying to match keyword `struct`
     --> src/macros.rs
      |
      |             $vis:vis struct $name:ident
      |                      ^^^^^^

Now the error is:

    error: cannot derive `Zeroable` for an enum
     --> tests/ui/compile-fail/zeroable/enum.rs:5:1
      |
    5 | enum Num {
      | ^^^^

    error: cannot derive `Zeroable` for an enum

---
Changes in v4: none
Changes in v3:
* use DiagCtxt error handling
Changes in v2:
* improved error handling

---
 rust/pin-init/internal/src/diagnostics.rs |   2 -
 rust/pin-init/internal/src/lib.rs         |   9 +-
 rust/pin-init/internal/src/zeroable.rs    | 155 ++++++++++------------
 rust/pin-init/src/macros.rs               | 124 -----------------
 4 files changed, 74 insertions(+), 216 deletions(-)
rust/pin-init/internal/src/diagnostics.rs +0 -2
index 555876c01bab..3bdb477c2f2b 100644
@@ -9,14 +9,12 @@
 pub(crate) struct ErrorGuaranteed(());
 
 impl DiagCtxt {
-    #[expect(dead_code)]
     pub(crate) fn error(&mut self, span: impl Spanned, msg: impl Display) -> ErrorGuaranteed {
         let error = Error::new(span.span(), msg);
         self.0.extend(error.into_compile_error());
         ErrorGuaranteed(())
     }
 
-    #[expect(dead_code)]
     pub(crate) fn with(
         fun: impl FnOnce(&mut DiagCtxt) -> Result<TokenStream, ErrorGuaranteed>,
     ) -> TokenStream {
rust/pin-init/internal/src/lib.rs +7 -2
index 0e1a4724549d..4cc9b7b0cda1 100644
@@ -11,6 +11,9 @@
 #![allow(missing_docs)]
 
 use proc_macro::TokenStream;
+use syn::parse_macro_input;
+
+use crate::diagnostics::DiagCtxt;
 
 mod diagnostics;
 mod helpers;
@@ -30,10 +33,12 @@ pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream {
 
 #[proc_macro_derive(Zeroable)]
 pub fn derive_zeroable(input: TokenStream) -> TokenStream {
-    zeroable::derive(input.into()).into()
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| zeroable::derive(input, dcx)).into()
 }
 
 #[proc_macro_derive(MaybeZeroable)]
 pub fn maybe_derive_zeroable(input: TokenStream) -> TokenStream {
-    zeroable::maybe_derive(input.into()).into()
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| zeroable::maybe_derive(input, dcx)).into()
 }
rust/pin-init/internal/src/zeroable.rs +67 -88

166 lines not on this page · raw message

rust/pin-init/src/macros.rs +0 -124

127 lines not on this page · raw message

Signature · 1 line
2.52.0
author maintainer
#6 lore raw

PATCH v4 6/15 rust: pin-init: rewrite the `#[pinned_drop]` attribute macro using `syn`

In reply to Benno Lossin

Rewrite the attribute macro for implementing PinnedDrop using syn. Otherwise no functional changes intended aside from improved error messages on syntactic and semantical errors. For example:

When missing the drop function in the implementation, the old error was:

    error: no rules expected `)`
     --> tests/ui/compile-fail/pinned_drop/no_fn.rs:6:1
      |
    6 | #[pinned_drop]
      | ^^^^^^^^^^^^^^ no rules expected this token in macro call
      |
    note: while trying to match keyword `fn`
     --> src/macros.rs
      |
      |             fn drop($($sig:tt)*) {
      |             ^^
      = note: this error originates in the attribute macro `pinned_drop` (in Nightly builds, run with -Z macro-backtrace for more info)

And the new one is:

    error[E0046]: not all trait items implemented, missing: `drop`
     --> tests/ui/compile-fail/pinned_drop/no_fn.rs:7:1
      |
    7 | impl PinnedDrop for Foo {}
      | ^^^^^^^^^^^^^^^^^^^^^^^ missing `drop` in implementation
      |
      = help: implement the missing item: `fn drop(self: Pin<&mut Self>, _: OnlyCallFromDrop) { todo!() }`

---
Changes in v4: none
Changes in v3:
* use DiagCtxt error handling
Changes in v2:
* improved error handling
* improved error span

---
 rust/pin-init/internal/src/lib.rs         |  4 +-
 rust/pin-init/internal/src/pinned_drop.rs | 86 +++++++++++++----------
 rust/pin-init/src/macros.rs               | 28 --------
 3 files changed, 52 insertions(+), 66 deletions(-)
rust/pin-init/internal/src/lib.rs +3 -1
index 4cc9b7b0cda1..a75b99b58189 100644
@@ -28,7 +28,9 @@ pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream {
 
 #[proc_macro_attribute]
 pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream {
-    pinned_drop::pinned_drop(args.into(), input.into()).into()
+    let args = parse_macro_input!(args);
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| pinned_drop::pinned_drop(args, input, dcx)).into()
 }
 
 #[proc_macro_derive(Zeroable)]
rust/pin-init/internal/src/pinned_drop.rs +49 -37

98 lines not on this page · raw message

rust/pin-init/src/macros.rs +0 -28
index 53ed5ce860fc..b80c95612fd6 100644
@@ -503,34 +503,6 @@
 #[cfg(not(kernel))]
 pub use ::paste::paste;
 
-/// Creates a `unsafe impl<...> PinnedDrop for $type` block.
-///
-/// See [`PinnedDrop`] for more information.
-///
-/// [`PinnedDrop`]: crate::PinnedDrop
-#[doc(hidden)]
-#[macro_export]
-macro_rules! __pinned_drop {
-    (
-        @impl_sig($($impl_sig:tt)*),
-        @impl_body(
-            $(#[$($attr:tt)*])*
-            fn drop($($sig:tt)*) {
-                $($inner:tt)*
-            }
-        ),
-    ) => {
-        // SAFETY: TODO.
-        unsafe $($impl_sig)* {
-            // Inherit all attributes and the type/ident tokens for the signature.
-            $(#[$($attr)*])*
-            fn drop($($sig)*, _: $crate::__internal::OnlyCallFromDrop) {
-                $($inner)*
-            }
-        }
-    }
-}
-
 /// This macro first parses the struct definition such that it separates pinned and not pinned
 /// fields. Afterwards it declares the struct and implement the `PinData` trait safely.
 #[doc(hidden)]
Signature · 1 line
2.52.0
author maintainer
#7 lore raw

PATCH v4 7/15 rust: pin-init: rewrite `#[pin_data]` using `syn`

In reply to Benno Lossin

Rewrite the attribute macro #[pin_data] using syn. No functional changes intended aside from improved error messages on syntactic and semantical errors. For example if one forgets a comma at the end of a field:

    #[pin_data]
    struct Foo {
        a: Box<Foo>
        b: Box<Foo>
    }

The declarative macro reports the following errors:

    error: expected `,`, or `}`, found `b`
     --> tests/ui/compile-fail/pin_data/missing_comma.rs:5:16
      |
    5 |     a: Box<Foo>
      |                ^ help: try adding a comma: `,`

    error: recursion limit reached while expanding `$crate::__pin_data!`
     --> tests/ui/compile-fail/pin_data/missing_comma.rs:3:1
      |
    3 | #[pin_data]
      | ^^^^^^^^^^^
      |
      = help: consider increasing the recursion limit by adding a `#![recursion_limit = "256"]` attribute to your crate (`$CRATE`)
      = note: this error originates in the macro `$crate::__pin_data` which comes from the expansion of the attribute macro `pin_data` (in Nightly builds, run with -Z macro-backtrace for more info)

The new syn version reports:

    error: expected `,`, or `}`, found `b`
     --> tests/ui/compile-fail/pin_data/missing_comma.rs:5:16
      |
    5 |     a: Box<Foo>
      |                ^ help: try adding a comma: `,`

    error: expected `,`
     --> tests/ui/compile-fail/pin_data/missing_comma.rs:6:5
      |
    6 |     b: Box<Foo>
      |     ^

---
Changes in v4:
* improve parsing
* consolidate generics handling
* improve handling of #[pin] attributes
Changes in v3:
* use DiagCtxt error handling
Changes in v2:
* improved error handling
* fix clippy warnings
* fix typos and variable names
* collect the information about the pinned/not pinned fields only once at the beginning

---
 rust/pin-init/internal/src/helpers.rs  | 149 ------
 rust/pin-init/internal/src/lib.rs      |   7 +-
 rust/pin-init/internal/src/pin_data.rs | 611 ++++++++++++++++++++-----
 rust/pin-init/src/macros.rs            | 574 -----------------------
 4 files changed, 503 insertions(+), 838 deletions(-)
 delete mode 100644 rust/pin-init/internal/src/helpers.rs
rust/pin-init/internal/src/helpers.rs deleted +0 -149
deleted file mode 100644
index 90f85eaa4123..000000000000
@@ -1,149 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0 OR MIT
-
-use proc_macro2::{TokenStream, TokenTree};
-
-/// Parsed generics.
-///
-/// See the field documentation for an explanation what each of the fields represents.
-///
-/// # Examples
-///
-/// ```rust,ignore
-/// # let input = todo!();
-/// let (Generics { decl_generics, impl_generics, ty_generics }, rest) = parse_generics(input);
-/// quote! {
-///     struct Foo<$($decl_generics)*> {
-///         // ...
-///     }
-///
-///     impl<$impl_generics> Foo<$ty_generics> {
-///         fn foo() {
-///             // ...
-///         }
-///     }
-/// }
-/// ```
-pub(crate) struct Generics {
-    /// The generics with bounds and default values (e.g. `T: Clone, const N: usize = 0`).
-    ///
-    /// Use this on type definitions e.g. `struct Foo<$decl_generics> ...` (or `union`/`enum`).
-    pub(crate) decl_generics: Vec<TokenTree>,
-    /// The generics with bounds (e.g. `T: Clone, const N: usize`).
-    ///
-    /// Use this on `impl` blocks e.g. `impl<$impl_generics> Trait for ...`.
-    pub(crate) impl_generics: Vec<TokenTree>,
-    /// The generics without bounds and without default values (e.g. `T, N`).
-    ///
-    /// Use this when you use the type that is declared with these generics e.g.
-    /// `Foo<$ty_generics>`.
-    pub(crate) ty_generics: Vec<TokenTree>,
-}
-
-/// Parses the given `TokenStream` into `Generics` and the rest.
-///
-/// The generics are not present in the rest, but a where clause might remain.
-pub(crate) fn parse_generics(input: TokenStream) -> (Generics, Vec<TokenTree>) {
-    // The generics with bounds and default values.
-    let mut decl_generics = vec![];
-    // `impl_generics`, the declared generics with their bounds.
-    let mut impl_generics = vec![];
-    // Only the names of the generics, without any bounds.
-    let mut ty_generics = vec![];
-    // Tokens not related to the generics e.g. the `where` token and definition.
-    let mut rest = vec![];
-    // The current level of `<`.
-    let mut nesting = 0;
-    let mut toks = input.into_iter();
-    // If we are at the beginning of a generic parameter.
-    let mut at_start = true;
-    let mut skip_until_comma = false;
-    while let Some(tt) = toks.next() {
-        if nesting == 1 && matches!(&tt, TokenTree::Punct(p) if p.as_char() == '>') {
-            // Found the end of the generics.
-            break;
-        } else if nesting >= 1 {
-            decl_generics.push(tt.clone());
-        }
-        match tt.clone() {
-            TokenTree::Punct(p) if p.as_char() == '<' => {
-                if nesting >= 1 && !skip_until_comma {
-                    // This is inside of the generics and part of some bound.
-                    impl_generics.push(tt);
-                }
-                nesting += 1;
-            }
-            TokenTree::Punct(p) if p.as_char() == '>' => {
-                // This is a parsing error, so we just end it here.
-                if nesting == 0 {
-                    break;
-                } else {
-                    nesting -= 1;
-                    if nesting >= 1 && !skip_until_comma {
-                        // We are still inside of the generics and part of some bound.
-                        impl_generics.push(tt);
-                    }
-                }
-            }
-            TokenTree::Punct(p) if skip_until_comma && p.as_char() == ',' => {
-                if nesting == 1 {
-                    impl_generics.push(tt.clone());
-                    impl_generics.push(tt);
-                    skip_until_comma = false;
-                }
-            }
-            _ if !skip_until_comma => {
-                match nesting {
-                    // If we haven't entered the generics yet, we still want to keep these tokens.
-                    0 => rest.push(tt),
-                    1 => {
-                        // Here depending on the token, it might be a generic variable name.
-                        match tt.clone() {
-                            TokenTree::Ident(i) if at_start && i == "const" => {
-                                let Some(name) = toks.next() else {
-                                    // Parsing error.
-                                    break;
-                                };
-                                impl_generics.push(tt);
-                                impl_generics.push(name.clone());
-                                ty_generics.push(name.clone());
-                                decl_generics.push(name);
-                                at_start = false;
-                            }
-                            TokenTree::Ident(_) if at_start => {
-                                impl_generics.push(tt.clone());
-                                ty_generics.push(tt);
-                                at_start = false;
-                            }
-                            TokenTree::Punct(p) if p.as_char() == ',' => {
-                                impl_generics.push(tt.clone());
-                                ty_generics.push(tt);
-                                at_start = true;
-                            }
-                            // Lifetimes begin with `'`.
-                            TokenTree::Punct(p) if p.as_char() == '\'' && at_start => {
-                                impl_generics.push(tt.clone());
-                                ty_generics.push(tt);
-                            }
-                            // Generics can have default values, we skip these.
-                            TokenTree::Punct(p) if p.as_char() == '=' => {
-                                skip_until_comma = true;
-                            }
-                            _ => impl_generics.push(tt),
-                        }
-                    }
-                    _ => impl_generics.push(tt),
-                }
-            }
-            _ => {}
-        }
-    }
-    rest.extend(toks);
-    (
-        Generics {
-            impl_generics,
-            decl_generics,
-            ty_generics,
-        },
-        rest,
-    )
-}
rust/pin-init/internal/src/lib.rs +4 -3
index a75b99b58189..56dc306e04a9 100644
@@ -16,14 +16,15 @@
 use crate::diagnostics::DiagCtxt;
 
 mod diagnostics;
-mod helpers;
 mod pin_data;
 mod pinned_drop;
 mod zeroable;
 
 #[proc_macro_attribute]
-pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream {
-    pin_data::pin_data(inner.into(), item.into()).into()
+pub fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
+    let args = parse_macro_input!(args);
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| pin_data::pin_data(args, input, dcx)).into()
 }
 
 #[proc_macro_attribute]
rust/pin-init/internal/src/pin_data.rs +499 -112

625 lines not on this page · raw message

rust/pin-init/src/macros.rs +0 -574

580 lines not on this page · raw message

Signature · 1 line
2.52.0
maintainer Reviewed-by on v4 7/15
#8 lore raw

Re: [PATCH v4 07/15] rust: pin-init: rewrite `#[pin_data]` using `syn`

Quoted 42 lines · Benno Lossin

Rewrite the attribute macro #[pin_data] using syn. No functional changes intended aside from improved error messages on syntactic and semantical errors. For example if one forgets a comma at the end of a field:

    #[pin_data]
    struct Foo {
        a: Box<Foo>
        b: Box<Foo>
    }

The declarative macro reports the following errors:

    error: expected `,`, or `}`, found `b`
     --> tests/ui/compile-fail/pin_data/missing_comma.rs:5:16
      |
    5 |     a: Box<Foo>
      |                ^ help: try adding a comma: `,`

    error: recursion limit reached while expanding `$crate::__pin_data!`
     --> tests/ui/compile-fail/pin_data/missing_comma.rs:3:1
      |
    3 | #[pin_data]
      | ^^^^^^^^^^^
      |
      = help: consider increasing the recursion limit by adding a `#![recursion_limit = "256"]` attribute to your crate (`$CRATE`)
      = note: this error originates in the macro `$crate::__pin_data` which comes from the expansion of the attribute macro `pin_data` (in Nightly builds, run with -Z macro-backtrace for more info)

The new syn version reports:

    error: expected `,`, or `}`, found `b`
     --> tests/ui/compile-fail/pin_data/missing_comma.rs:5:16
      |
    5 |     a: Box<Foo>
      |                ^ help: try adding a comma: `,`

    error: expected `,`
     --> tests/ui/compile-fail/pin_data/missing_comma.rs:6:5
      |
    6 |     b: Box<Foo>
      |     ^
Quoted 2 lines
Quoted 20 lines

---
Changes in v4:
* improve parsing
* consolidate generics handling
* improve handling of #[pin] attributes
Changes in v3:
* use DiagCtxt error handling
Changes in v2:
* improved error handling
* fix clippy warnings
* fix typos and variable names
* collect the information about the pinned/not pinned fields only once at the beginning

---
 rust/pin-init/internal/src/helpers.rs  | 149 ------
 rust/pin-init/internal/src/lib.rs      |   7 +-
 rust/pin-init/internal/src/pin_data.rs | 611 ++++++++++++++++++++-----
 rust/pin-init/src/macros.rs            | 574 -----------------------
 4 files changed, 503 insertions(+), 838 deletions(-)
 delete mode 100644 rust/pin-init/internal/src/helpers.rs
author maintainer
#9 lore raw

PATCH v4 8/15 rust: pin-init: add `?Sized` bounds to traits in `#[pin_data]` macro

In reply to Benno Lossin

The #[pin_data] macro uses some auxiliary traits to ensure that a user does not implement Drop for the annotated struct, as that is unsound and can lead to UB. However, if the struct that is annotated is !Sized, the current bounds do not work, because Sized is an implicit bound for generics.

This is *not* a soundness hole of pin-init, as it currently is impossible to construct an unsized struct using pin-init.

---
Changes in v4: none
Changes in v3: none
Changes in v2: none

---
 rust/pin-init/internal/src/pin_data.rs | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
rust/pin-init/internal/src/pin_data.rs +2 -2
index 11ea3f8d8a1b..7d871236b49c 100644
@@ -215,7 +215,7 @@ fn drop(&mut self) {
             // if it also implements `Drop`
             trait MustNotImplDrop {}
             #[expect(drop_bounds)]
-            impl<T: ::core::ops::Drop> MustNotImplDrop for T {}
+            impl<T: ::core::ops::Drop + ?::core::marker::Sized> MustNotImplDrop for T {}
             impl #impl_generics MustNotImplDrop for #ident #ty_generics
                 #whr
             {}
@@ -224,7 +224,7 @@ impl #impl_generics MustNotImplDrop for #ident #ty_generics
             // `PinnedDrop` as the parameter to `#[pin_data]`.
             #[expect(non_camel_case_types)]
             trait UselessPinnedDropImpl_you_need_to_specify_PinnedDrop {}
-            impl<T: ::pin_init::PinnedDrop>
+            impl<T: ::pin_init::PinnedDrop + ?::core::marker::Sized>
                 UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for T {}
             impl #impl_generics
                 UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for #ident #ty_generics
Signature · 1 line
2.52.0
author maintainer
#10 lore raw

PATCH v4 9/15 rust: pin-init: rewrite the initializer macros using `syn`

In reply to Benno Lossin

Rewrite the initializer macros [pin_]init! using syn. No functional changes intended aside from improved error messages on syntactic and semantical errors. For example if one forgets to use <- with an initializer (and instead uses :):

    impl Bar {
        fn new() -> impl PinInit<Self> { ... }
    }

    impl Foo {
        fn new() -> impl PinInit<Self> {
            pin_init!(Self { bar: Bar::new() })
        }
    }

Then the declarative macro would report:

    error[E0308]: mismatched types
      --> tests/ui/compile-fail/init/colon_instead_of_arrow.rs:21:9
       |
    14 |     fn new() -> impl PinInit<Self> {
       |                 ------------------ the found opaque type
    ...
    21 |         pin_init!(Self { bar: Bar::new() })
       |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
       |         |
       |         expected `Bar`, found opaque type
       |         arguments to this function are incorrect
       |
       = note:   expected struct `Bar`
               found opaque type `impl pin_init::PinInit<Bar>`
    note: function defined here
      --> $RUST/core/src/ptr/mod.rs
       |
       | pub const unsafe fn write<T>(dst: *mut T, src: T) {
       |                     ^^^^^
       = note: this error originates in the macro `$crate::__init_internal` which comes from the expansion of the macro `pin_init` (in Nightly builds, run with -Z macro-backtrace for more info)

And the new error is:

    error[E0308]: mismatched types
      --> tests/ui/compile-fail/init/colon_instead_of_arrow.rs:21:31
       |
    14 |     fn new() -> impl PinInit<Self> {
       |                 ------------------ the found opaque type
    ...
    21 |         pin_init!(Self { bar: Bar::new() })
       |                          ---  ^^^^^^^^^^ expected `Bar`, found opaque type
       |                          |
       |                          arguments to this function are incorrect
       |
       = note:   expected struct `Bar`
               found opaque type `impl pin_init::PinInit<Bar>`
    note: function defined here
      --> $RUST/core/src/ptr/mod.rs
       |
       | pub const unsafe fn write<T>(dst: *mut T, src: T) {
       |                     ^^^^^

Importantly, this error gives much more accurate span locations, pointing to the offending field, rather than the entire macro invocation.

---
Changes in v4: none
Changes in v3:
* use DiagCtxt error handling
Changes in v2:
* fix clippy error
* add missing SPDX-License-Identifier
* improve error handling
* improve default error evaluation
* added missing comment from pre-rewrite code

---
 rust/pin-init/internal/src/init.rs | 445 ++++++++++++++
 rust/pin-init/internal/src/lib.rs  |  13 +
 rust/pin-init/src/lib.rs           |  56 +-
 rust/pin-init/src/macros.rs        | 951 -----------------------------
 4 files changed, 460 insertions(+), 1005 deletions(-)
 create mode 100644 rust/pin-init/internal/src/init.rs
 delete mode 100644 rust/pin-init/src/macros.rs
rust/pin-init/internal/src/init.rs new file +445 -0
new file mode 100644
index 000000000000..b0eb66224341
@@ -0,0 +1,445 @@
+// SPDX-License-Identifier: Apache-2.0 OR MIT
+
+use proc_macro2::{Span, TokenStream};
+use quote::{format_ident, quote, quote_spanned};
+use syn::{
+    braced,
+    parse::{End, Parse},
+    parse_quote,
+    punctuated::Punctuated,
+    spanned::Spanned,
+    token, Block, Expr, ExprCall, ExprPath, Ident, Path, Token, Type,
+};
+
+use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
+
+pub(crate) struct Initializer {
+    this: Option<This>,
+    path: Path,
+    brace_token: token::Brace,
+    fields: Punctuated<InitializerField, Token![,]>,
+    rest: Option<(Token![..], Expr)>,
+    error: Option<(Token![?], Type)>,
+}
+
+struct This {
+    _and_token: Token![&],
+    ident: Ident,
+    _in_token: Token![in],
+}
+
+enum InitializerField {
+    Value {
+        ident: Ident,
+        value: Option<(Token![:], Expr)>,
+    },
+    Init {
+        ident: Ident,
+        _left_arrow_token: Token![<-],
+        value: Expr,
+    },
+    Code {
+        _underscore_token: Token![_],
+        _colon_token: Token![:],
+        block: Block,
+    },
+}
+
+impl InitializerField {
+    fn ident(&self) -> Option<&Ident> {
+        match self {
+            Self::Value { ident, .. } | Self::Init { ident, .. } => Some(ident),
+            Self::Code { .. } => None,
+        }
+    }
+}
+
+pub(crate) fn expand(
+    Initializer {
+        this,
+        path,
+        brace_token,
+        fields,
+        rest,
+        error,
+    }: Initializer,
+    default_error: Option<&'static str>,
+    pinned: bool,
+    dcx: &mut DiagCtxt,
+) -> Result<TokenStream, ErrorGuaranteed> {
+    let error = error.map_or_else(
+        || {
+            if let Some(default_error) = default_error {
+                syn::parse_str(default_error).unwrap()
+            } else {
+                dcx.error(brace_token.span.close(), "expected `? <type>` after `}`");
+                parse_quote!(::core::convert::Infallible)
+            }
+        },
+        |(_, err)| err,
+    );
+    let slot = format_ident!("slot");
+    let (has_data_trait, data_trait, get_data, init_from_closure) = if pinned {
+        (
+            format_ident!("HasPinData"),
+            format_ident!("PinData"),
+            format_ident!("__pin_data"),
+            format_ident!("pin_init_from_closure"),
+        )
+    } else {
+        (
+            format_ident!("HasInitData"),
+            format_ident!("InitData"),
+            format_ident!("__init_data"),
+            format_ident!("init_from_closure"),
+        )
+    };
+    let init_kind = get_init_kind(rest, dcx);
+    let zeroable_check = match init_kind {
+        InitKind::Normal => quote!(),
+        InitKind::Zeroing => quote! {
+            // The user specified `..Zeroable::zeroed()` at the end of the list of fields.
+            // Therefore we check if the struct implements `Zeroable` and then zero the memory.
+            // This allows us to also remove the check that all fields are present (since we
+            // already set the memory to zero and that is a valid bit pattern).
+            fn assert_zeroable<T: ?::core::marker::Sized>(_: *mut T)
+            where T: ::pin_init::Zeroable
+            {}
+            // Ensure that the struct is indeed `Zeroable`.
+            assert_zeroable(#slot);
+            // SAFETY: The type implements `Zeroable` by the check above.
+            unsafe { ::core::ptr::write_bytes(#slot, 0, 1) };
+        },
+    };
+    let this = match this {
+        None => quote!(),
+        Some(This { ident, .. }) => quote! {
+            // Create the `this` so it can be referenced by the user inside of the
+            // expressions creating the individual fields.
+            let #ident = unsafe { ::core::ptr::NonNull::new_unchecked(slot) };
+        },
+    };
+    // `mixed_site` ensures that the data is not accessible to the user-controlled code.
+    let data = Ident::new("__data", Span::mixed_site());
+    let init_fields = init_fields(&fields, pinned, &data, &slot);
+    let field_check = make_field_check(&fields, init_kind, &path);
+    Ok(quote! {{
+        // We do not want to allow arbitrary returns, so we declare this type as the `Ok` return
+        // type and shadow it later when we insert the arbitrary user code. That way there will be
+        // no possibility of returning without `unsafe`.
+        struct __InitOk;
+
+        // Get the data about fields from the supplied type.
+        // SAFETY: TODO
+        let #data = unsafe {
+            use ::pin_init::__internal::#has_data_trait;
+            // Can't use `<#path as #has_data_trait>::#get_data`, since the user is able to omit
+            // generics (which need to be present with that syntax).
+            #path::#get_data()
+        };
+        // Ensure that `#data` really is of type `#data` and help with type inference:
+        let init = ::pin_init::__internal::#data_trait::make_closure::<_, __InitOk, #error>(
+            #data,
+            move |slot| {
+                {
+                    // Shadow the structure so it cannot be used to return early.
+                    struct __InitOk;
+                    #zeroable_check
+                    #this
+                    #init_fields
+                    #field_check
+                }
+                Ok(__InitOk)
+            }
+        );
+        let init = move |slot| -> ::core::result::Result<(), #error> {
+            init(slot).map(|__InitOk| ())
+        };
+        // SAFETY: TODO
+        let init = unsafe { ::pin_init::#init_from_closure::<_, #error>(init) };
+        init
+    }})
+}
+
+enum InitKind {
+    Normal,
+    Zeroing,
+}
+
+fn get_init_kind(rest: Option<(Token![..], Expr)>, dcx: &mut DiagCtxt) -> InitKind {
+    let Some((dotdot, expr)) = rest else {
+        return InitKind::Normal;
+    };
+    match &expr {
+        Expr::Call(ExprCall { func, args, .. }) if args.is_empty() => match &**func {
+            Expr::Path(ExprPath {
+                attrs,
+                qself: None,
+                path:
+                    Path {
+                        leading_colon: None,
+                        segments,
+                    },
+            }) if attrs.is_empty()
+                && segments.len() == 2
+                && segments[0].ident == "Zeroable"
+                && segments[0].arguments.is_none()
+                && segments[1].ident == "init_zeroed"
+                && segments[1].arguments.is_none() =>
+            {
+                return InitKind::Zeroing;
+            }
+            _ => {}
+        },
+        _ => {}
+    }
+    dcx.error(
+        dotdot.span().join(expr.span()).unwrap_or(expr.span()),
+        "expected nothing or `..Zeroable::init_zeroed()`.",
+    );
+    InitKind::Normal
+}
+
+/// Generate the code that initializes the fields of the struct using the initializers in `field`.
+fn init_fields(
+    fields: &Punctuated<InitializerField, Token![,]>,
+    pinned: bool,
+    data: &Ident,
+    slot: &Ident,
+) -> TokenStream {
+    let mut guards = vec![];
+    let mut res = TokenStream::new();
+    for field in fields {
+        let init = match field {
+            InitializerField::Value { ident, value } => {
+                let mut value_ident = ident.clone();
+                let value_prep = value.as_ref().map(|value| &value.1).map(|value| {
+                    // Setting the span of `value_ident` to `value`'s span improves error messages
+                    // when the type of `value` is wrong.
+                    value_ident.set_span(value.span());
+                    quote!(let #value_ident = #value;)
+                });
+                // Again span for better diagnostics
+                let write = quote_spanned!(ident.span()=> ::core::ptr::write);
+                let accessor = if pinned {
+                    let project_ident = format_ident!("__project_{ident}");
+                    quote! {
+                        // SAFETY: TODO
+                        unsafe { #data.#project_ident(&mut (*#slot).#ident) }
+                    }
+                } else {
+                    quote! {
+                        // SAFETY: TODO
+                        unsafe { &mut (*#slot).#ident }
+                    }
+                };
+                quote! {
+                    {
+                        #value_prep
+                        // SAFETY: TODO
+                        unsafe { #write(::core::ptr::addr_of_mut!((*#slot).#ident), #value_ident) };
+                    }
+                    #[allow(unused_variables)]
+                    let #ident = #accessor;
+                }
+            }
+            InitializerField::Init { ident, value, .. } => {
+                // Again span for better diagnostics
+                let init = format_ident!("init", span = value.span());
+                if pinned {
+                    let project_ident = format_ident!("__project_{ident}");
+                    quote! {
+                        {
+                            let #init = #value;
+                            // SAFETY:
+                            // - `slot` is valid, because we are inside of an initializer closure, we
+                            //   return when an error/panic occurs.
+                            // - We also use `#data` to require the correct trait (`Init` or `PinInit`)
+                            //   for `#ident`.
+                            unsafe { #data.#ident(::core::ptr::addr_of_mut!((*#slot).#ident), #init)? };
+                        }
+                        // SAFETY: TODO
+                        #[allow(unused_variables)]
+                        let #ident = unsafe { #data.#project_ident(&mut (*#slot).#ident) };
+                    }
+                } else {
+                    quote! {
+                        {
+                            let #init = #value;
+                            // SAFETY: `slot` is valid, because we are inside of an initializer
+                            // closure, we return when an error/panic occurs.
+                            unsafe {
+                                ::pin_init::Init::__init(
+                                    #init,
+                                    ::core::ptr::addr_of_mut!((*#slot).#ident),
+                                )?
+                            };
+                        }
+                        // SAFETY: TODO
+                        #[allow(unused_variables)]
+                        let #ident = unsafe { &mut (*#slot).#ident };
+                    }
+                }
+            }
+            InitializerField::Code { block: value, .. } => quote!(#[allow(unused_braces)] #value),
+        };
+        res.extend(init);
+        if let Some(ident) = field.ident() {
+            // `mixed_site` ensures that the guard is not accessible to the user-controlled code.
+            let guard = format_ident!("__{ident}_guard", span = Span::mixed_site());
+            guards.push(guard.clone());
+            res.extend(quote! {
+                // Create the drop guard:
+                //
+                // We rely on macro hygiene to make it impossible for users to access this local
+                // variable.
+                // SAFETY: We forget the guard later when initialization has succeeded.
+                let #guard = unsafe {
+                    ::pin_init::__internal::DropGuard::new(
+                        ::core::ptr::addr_of_mut!((*slot).#ident)
+                    )
+                };
+            });
+        }
+    }
+    quote! {
+        #res
+        // If execution reaches this point, all fields have been initialized. Therefore we can now
+        // dismiss the guards by forgetting them.
+        #(::core::mem::forget(#guards);)*
+    }
+}
+
+/// Generate the check for ensuring that every field has been initialized.
+fn make_field_check(
+    fields: &Punctuated<InitializerField, Token![,]>,
+    init_kind: InitKind,
+    path: &Path,
+) -> TokenStream {
+    let fields = fields.iter().filter_map(|f| f.ident());
+    match init_kind {
+        InitKind::Normal => quote! {
+            // We use unreachable code to ensure that all fields have been mentioned exactly once,
+            // this struct initializer will still be type-checked and complain with a very natural
+            // error message if a field is forgotten/mentioned more than once.
+            #[allow(unreachable_code, clippy::diverging_sub_expression)]
+            // SAFETY: this code is never executed.
+            let _ = || unsafe {
+                ::core::ptr::write(slot, #path {
+                    #(
+                        #fields: ::core::panic!(),
+                    )*
+                })
+            };
+        },
+        InitKind::Zeroing => quote! {
+            // We use unreachable code to ensure that all fields have been mentioned at most once.
+            // Since the user specified `..Zeroable::zeroed()` at the end, all missing fields will
+            // be zeroed. This struct initializer will still be type-checked and complain with a
+            // very natural error message if a field is mentioned more than once, or doesn't exist.
+            #[allow(unreachable_code, clippy::diverging_sub_expression, unused_assignments)]
+            // SAFETY: this code is never executed.
+            let _ = || unsafe {
+                let mut zeroed = ::core::mem::zeroed();
+                // We have to use type inference here to make zeroed have the correct type. This
+                // does not get executed, so it has no effect.
+                ::core::ptr::write(slot, zeroed);
+                zeroed = ::core::mem::zeroed();
+                ::core::ptr::write(slot, #path {
+                    #(
+                        #fields: ::core::panic!(),
+                    )*
+                    ..zeroed
+                })
+            };
+        },
+    }
+}
+
+impl Parse for Initializer {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let this = input.peek(Token![&]).then(|| input.parse()).transpose()?;
+        let path = input.parse()?;
+        let content;
+        let brace_token = braced!(content in input);
+        let mut fields = Punctuated::new();
+        loop {
+            let lh = content.lookahead1();
+            if lh.peek(End) || lh.peek(Token![..]) {
+                break;
+            } else if lh.peek(Ident) || lh.peek(Token![_]) {
+                fields.push_value(content.parse()?);
+                let lh = content.lookahead1();
+                if lh.peek(End) {
+                    break;
+                } else if lh.peek(Token![,]) {
+                    fields.push_punct(content.parse()?);
+                } else {
+                    return Err(lh.error());
+                }
+            } else {
+                return Err(lh.error());
+            }
+        }
+        let rest = content
+            .peek(Token![..])
+            .then(|| Ok::<_, syn::Error>((content.parse()?, content.parse()?)))
+            .transpose()?;
+        let error = input
+            .peek(Token![?])
+            .then(|| Ok::<_, syn::Error>((input.parse()?, input.parse()?)))
+            .transpose()?;
+        Ok(Self {
+            this,
+            path,
+            brace_token,
+            fields,
+            rest,
+            error,
+        })
+    }
+}
+
+impl Parse for This {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        Ok(Self {
+            _and_token: input.parse()?,
+            ident: input.parse()?,
+            _in_token: input.parse()?,
+        })
+    }
+}
+
+impl Parse for InitializerField {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let lh = input.lookahead1();
+        if lh.peek(Token![_]) {
+            Ok(Self::Code {
+                _underscore_token: input.parse()?,
+                _colon_token: input.parse()?,
+                block: input.parse()?,
+            })
+        } else if lh.peek(Ident) {
+            let ident = input.parse()?;
+            let lh = input.lookahead1();
+            if lh.peek(Token![<-]) {
+                Ok(Self::Init {
+                    ident,
+                    _left_arrow_token: input.parse()?,
+                    value: input.parse()?,
+                })
+            } else if lh.peek(Token![:]) {
+                Ok(Self::Value {
+                    ident,
+                    value: Some((input.parse()?, input.parse()?)),
+                })
+            } else if lh.peek(Token![,]) || lh.peek(End) {
+                Ok(Self::Value { ident, value: None })
+            } else {
+                Err(lh.error())
+            }
+        } else {
+            Err(lh.error())
+        }
+    }
+}
rust/pin-init/internal/src/lib.rs +13 -0
index 56dc306e04a9..08372c8f65f0 100644
@@ -16,6 +16,7 @@
 use crate::diagnostics::DiagCtxt;
 
 mod diagnostics;
+mod init;
 mod pin_data;
 mod pinned_drop;
 mod zeroable;
@@ -45,3 +46,15 @@ pub fn maybe_derive_zeroable(input: TokenStream) -> TokenStream {
     let input = parse_macro_input!(input);
     DiagCtxt::with(|dcx| zeroable::maybe_derive(input, dcx)).into()
 }
+#[proc_macro]
+pub fn init(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| init::expand(input, Some("::core::convert::Infallible"), false, dcx))
+        .into()
+}
+
+#[proc_macro]
+pub fn pin_init(input: TokenStream) -> TokenStream {
+    let input = parse_macro_input!(input);
+    DiagCtxt::with(|dcx| init::expand(input, Some("::core::convert::Infallible"), true, dcx)).into()
+}
rust/pin-init/src/lib.rs +2 -54

74 lines not on this page · raw message

rust/pin-init/src/macros.rs deleted +0 -951

951 lines not on this page · raw message

Signature · 1 line
2.52.0
author maintainer
#11 lore raw

PATCH v4 10/15 rust: pin-init: add `#[default_error(<type>)]` attribute to initializer macros

In reply to Benno Lossin

The #[default_error(<type>)] attribute can be used to supply a default type as the error used for the [pin_]init! macros. This way one can easily define custom try_[pin_]init! variants that default to your project specific error type. Just write the following declarative macro:

    macro_rules! try_init {
        ($($args:tt)*) => {
            ::pin_init::init!(
                #[default_error(YourCustomErrorType)]
                $($args)*
            )
        }
    }

---
Changes in v4:
* box big value instead of later ignoring the clippy warning Changes in v3: none
Changes in v2:
* improve error handling
* improve parsing and default error evaluation

---
 rust/pin-init/internal/src/init.rs | 44 ++++++++++++++++++++++++++++--
 1 file changed, 41 insertions(+), 3 deletions(-)
rust/pin-init/internal/src/init.rs +41 -3
index b0eb66224341..21cddcd9e9f8 100644
@@ -8,12 +8,13 @@
     parse_quote,
     punctuated::Punctuated,
     spanned::Spanned,
-    token, Block, Expr, ExprCall, ExprPath, Ident, Path, Token, Type,
+    token, Attribute, Block, Expr, ExprCall, ExprPath, Ident, Path, Token, Type,
 };
 
 use crate::diagnostics::{DiagCtxt, ErrorGuaranteed};
 
 pub(crate) struct Initializer {
+    attrs: Vec<InitializerAttribute>,
     this: Option<This>,
     path: Path,
     brace_token: token::Brace,
@@ -54,8 +55,17 @@ fn ident(&self) -> Option<&Ident> {
     }
 }
 
+enum InitializerAttribute {
+    DefaultError(DefaultErrorAttribute),
+}
+
+struct DefaultErrorAttribute {
+    ty: Box<Type>,
+}
+
 pub(crate) fn expand(
     Initializer {
+        attrs,
         this,
         path,
         brace_token,
@@ -69,14 +79,23 @@ pub(crate) fn expand(
 ) -> Result<TokenStream, ErrorGuaranteed> {
     let error = error.map_or_else(
         || {
-            if let Some(default_error) = default_error {
+            if let Some(default_error) = attrs.iter().fold(None, |acc, attr| {
+                #[expect(irrefutable_let_patterns)]
+                if let InitializerAttribute::DefaultError(DefaultErrorAttribute { ty }) = attr {
+                    Some(ty.clone())
+                } else {
+                    acc
+                }
+            }) {
+                default_error
+            } else if let Some(default_error) = default_error {
                 syn::parse_str(default_error).unwrap()
             } else {
                 dcx.error(brace_token.span.close(), "expected `? <type>` after `}`");
                 parse_quote!(::core::convert::Infallible)
             }
         },
-        |(_, err)| err,
+        |(_, err)| Box::new(err),
     );
     let slot = format_ident!("slot");
     let (has_data_trait, data_trait, get_data, init_from_closure) = if pinned {
@@ -358,6 +377,7 @@ fn make_field_check(
 
 impl Parse for Initializer {
     fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let attrs = input.call(Attribute::parse_outer)?;
         let this = input.peek(Token![&]).then(|| input.parse()).transpose()?;
         let path = input.parse()?;
         let content;
@@ -389,7 +409,19 @@ fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
             .peek(Token![?])
             .then(|| Ok::<_, syn::Error>((input.parse()?, input.parse()?)))
             .transpose()?;
+        let attrs = attrs
+            .into_iter()
+            .map(|a| {
+                if a.path().is_ident("default_error") {
+                    a.parse_args::<DefaultErrorAttribute>()
+                        .map(InitializerAttribute::DefaultError)
+                } else {
+                    Err(syn::Error::new_spanned(a, "unknown initializer attribute"))
+                }
+            })
+            .collect::<Result<Vec<_>, _>>()?;
         Ok(Self {
+            attrs,
             this,
             path,
             brace_token,
@@ -400,6 +432,12 @@ fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
     }
 }
 
+impl Parse for DefaultErrorAttribute {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        Ok(Self { ty: input.parse()? })
+    }
+}
+
 impl Parse for This {
     fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
         Ok(Self {
Signature · 1 line
2.52.0
author maintainer
#12 lore raw

PATCH v4 11/15 rust: init: use `#[default_error(err)]` for the initializer macros

In reply to Benno Lossin

Initializer macros should use this attribute instead of manually parsing the macro's input. This is because the syntax is now parsed using syn, which permits more complex constructs to be parsed. In addition, this ensures that the kernel's initializer marcos will have the exact same syntax as the ones from pin-init.

---
Changes in v4: none
Changes in v3: none
Changes in v2: none

---
 rust/kernel/init.rs | 40 ++++++++++++----------------------------
 1 file changed, 12 insertions(+), 28 deletions(-)
rust/kernel/init.rs +12 -28
index 917f7ef001fd..7a0d4559d7b5 100644
@@ -219,20 +219,12 @@ fn init<E>(init: impl Init<T, E>, flags: Flags) -> error::Result<Self>
 /// [`Error`]: crate::error::Error
 #[macro_export]
 macro_rules! try_init {
-    ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? {
-        $($fields:tt)*
-    }) => {
-        ::pin_init::init!($(&$this in)? $t $(::<$($generics),*>)? {
-            $($fields)*
-        }? $crate::error::Error)
-    };
-    ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? {
-        $($fields:tt)*
-    }? $err:ty) => {
-        ::pin_init::init!($(&$this in)? $t $(::<$($generics),*>)? {
-            $($fields)*
-        }? $err)
-    };
+    ($($args:tt)*) => {
+        ::pin_init::init!(
+            #[default_error($crate::error::Error)]
+            $($args)*
+        )
+    }
 }
 
 /// Construct an in-place, fallible pinned initializer for `struct`s.
@@ -279,18 +271,10 @@ macro_rules! try_init {
 /// [`Error`]: crate::error::Error
 #[macro_export]
 macro_rules! try_pin_init {
-    ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? {
-        $($fields:tt)*
-    }) => {
-        ::pin_init::pin_init!($(&$this in)? $t $(::<$($generics),*>)? {
-            $($fields)*
-        }? $crate::error::Error)
-    };
-    ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? {
-        $($fields:tt)*
-    }? $err:ty) => {
-        ::pin_init::pin_init!($(&$this in)? $t $(::<$($generics),*>)? {
-            $($fields)*
-        }? $err)
-    };
+    ($($args:tt)*) => {
+        ::pin_init::pin_init!(
+            #[default_error($crate::error::Error)]
+            $($args)*
+        )
+    }
 }
Signature · 1 line
2.52.0
author maintainer
#13 lore raw

PATCH v4 12/15 rust: pin-init: internal: init: add support for attributes on initializer fields

In reply to Benno Lossin

Initializer fields ought to support the same attributes that are allowed in struct initializers on fields. For example, cfg or lint levels such as expect, allow etc. Add parsing support for these attributes using syn to initializer fields and adjust the macro expansion accordingly.

---
Changes in v4:
* remove inclusion of #[cfg_attr(...)]
Changes in v3: none
Changes in v2:
* only attach cfg attributes to non-user controlled code for fields

---
 rust/pin-init/internal/src/init.rs | 69 ++++++++++++++++++++++++------
 1 file changed, 55 insertions(+), 14 deletions(-)
rust/pin-init/internal/src/init.rs +55 -14
index 21cddcd9e9f8..e8b93b444f7e 100644
@@ -29,7 +29,12 @@ struct This {
     _in_token: Token![in],
 }
 
-enum InitializerField {
+struct InitializerField {
+    attrs: Vec<Attribute>,
+    kind: InitializerKind,
+}
+
+enum InitializerKind {
     Value {
         ident: Ident,
         value: Option<(Token![:], Expr)>,
@@ -46,7 +51,7 @@ enum InitializerField {
     },
 }
 
-impl InitializerField {
+impl InitializerKind {
     fn ident(&self) -> Option<&Ident> {
         match self {
             Self::Value { ident, .. } | Self::Init { ident, .. } => Some(ident),
@@ -227,10 +232,16 @@ fn init_fields(
     slot: &Ident,
 ) -> TokenStream {
     let mut guards = vec![];
+    let mut guard_attrs = vec![];
     let mut res = TokenStream::new();
-    for field in fields {
-        let init = match field {
-            InitializerField::Value { ident, value } => {
+    for InitializerField { attrs, kind } in fields {
+        let cfgs = {
+            let mut cfgs = attrs.clone();
+            cfgs.retain(|attr| attr.path().is_ident("cfg"));
+            cfgs
+        };
+        let init = match kind {
+            InitializerKind::Value { ident, value } => {
                 let mut value_ident = ident.clone();
                 let value_prep = value.as_ref().map(|value| &value.1).map(|value| {
                     // Setting the span of `value_ident` to `value`'s span improves error messages
@@ -253,21 +264,24 @@ fn init_fields(
                     }
                 };
                 quote! {
+                    #(#attrs)*
                     {
                         #value_prep
                         // SAFETY: TODO
                         unsafe { #write(::core::ptr::addr_of_mut!((*#slot).#ident), #value_ident) };
                     }
+                    #(#cfgs)*
                     #[allow(unused_variables)]
                     let #ident = #accessor;
                 }
             }
-            InitializerField::Init { ident, value, .. } => {
+            InitializerKind::Init { ident, value, .. } => {
                 // Again span for better diagnostics
                 let init = format_ident!("init", span = value.span());
                 if pinned {
                     let project_ident = format_ident!("__project_{ident}");
                     quote! {
+                        #(#attrs)*
                         {
                             let #init = #value;
                             // SAFETY:
@@ -277,12 +291,14 @@ fn init_fields(
                             //   for `#ident`.
                             unsafe { #data.#ident(::core::ptr::addr_of_mut!((*#slot).#ident), #init)? };
                         }
+                        #(#cfgs)*
                         // SAFETY: TODO
                         #[allow(unused_variables)]
                         let #ident = unsafe { #data.#project_ident(&mut (*#slot).#ident) };
                     }
                 } else {
                     quote! {
+                        #(#attrs)*
                         {
                             let #init = #value;
                             // SAFETY: `slot` is valid, because we are inside of an initializer
@@ -294,20 +310,25 @@ fn init_fields(
                                 )?
                             };
                         }
+                        #(#cfgs)*
                         // SAFETY: TODO
                         #[allow(unused_variables)]
                         let #ident = unsafe { &mut (*#slot).#ident };
                     }
                 }
             }
-            InitializerField::Code { block: value, .. } => quote!(#[allow(unused_braces)] #value),
+            InitializerKind::Code { block: value, .. } => quote! {
+                #(#attrs)*
+                #[allow(unused_braces)]
+                #value
+            },
         };
         res.extend(init);
-        if let Some(ident) = field.ident() {
+        if let Some(ident) = kind.ident() {
             // `mixed_site` ensures that the guard is not accessible to the user-controlled code.
             let guard = format_ident!("__{ident}_guard", span = Span::mixed_site());
-            guards.push(guard.clone());
             res.extend(quote! {
+                #(#cfgs)*
                 // Create the drop guard:
                 //
                 // We rely on macro hygiene to make it impossible for users to access this local
@@ -319,13 +340,18 @@ fn init_fields(
                     )
                 };
             });
+            guards.push(guard);
+            guard_attrs.push(cfgs);
         }
     }
     quote! {
         #res
         // If execution reaches this point, all fields have been initialized. Therefore we can now
         // dismiss the guards by forgetting them.
-        #(::core::mem::forget(#guards);)*
+        #(
+            #(#guard_attrs)*
+            ::core::mem::forget(#guards);
+        )*
     }
 }
 
@@ -335,7 +361,10 @@ fn make_field_check(
     init_kind: InitKind,
     path: &Path,
 ) -> TokenStream {
-    let fields = fields.iter().filter_map(|f| f.ident());
+    let field_attrs = fields
+        .iter()
+        .filter_map(|f| f.kind.ident().map(|_| &f.attrs));
+    let field_name = fields.iter().filter_map(|f| f.kind.ident());
     match init_kind {
         InitKind::Normal => quote! {
             // We use unreachable code to ensure that all fields have been mentioned exactly once,
@@ -346,7 +375,8 @@ fn make_field_check(
             let _ = || unsafe {
                 ::core::ptr::write(slot, #path {
                     #(
-                        #fields: ::core::panic!(),
+                        #(#field_attrs)*
+                        #field_name: ::core::panic!(),
                     )*
                 })
             };
@@ -366,7 +396,8 @@ fn make_field_check(
                 zeroed = ::core::mem::zeroed();
                 ::core::ptr::write(slot, #path {
                     #(
-                        #fields: ::core::panic!(),
+                        #(#field_attrs)*
+                        #field_name: ::core::panic!(),
                     )*
                     ..zeroed
                 })
@@ -387,7 +418,7 @@ fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
             let lh = content.lookahead1();
             if lh.peek(End) || lh.peek(Token![..]) {
                 break;
-            } else if lh.peek(Ident) || lh.peek(Token![_]) {
+            } else if lh.peek(Ident) || lh.peek(Token![_]) || lh.peek(Token![#]) {
                 fields.push_value(content.parse()?);
                 let lh = content.lookahead1();
                 if lh.peek(End) {
@@ -449,6 +480,16 @@ fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
 }
 
 impl Parse for InitializerField {
+    fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
+        let attrs = input.call(Attribute::parse_outer)?;
+        Ok(Self {
+            attrs,
+            kind: input.parse()?,
+        })
+    }
+}
+
+impl Parse for InitializerKind {
     fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
         let lh = input.lookahead1();
         if lh.peek(Token![_]) {
Signature · 1 line
2.52.0
maintainer Reviewed-by on v4 12/15
#14 lore raw

Re: [PATCH v4 12/15] rust: pin-init: internal: init: add support for attributes on initializer fields

Quoted 7 lines · Benno Lossin

Initializer fields ought to support the same attributes that are allowed in struct initializers on fields. For example, cfg or lint levels such as expect, allow etc. Add parsing support for these attributes using syn to initializer fields and adjust the macro expansion accordingly.

Quoted 9 lines

---
Changes in v4:
* remove inclusion of #[cfg_attr(...)]
Changes in v3: none
Changes in v2:
* only attach cfg attributes to non-user controlled code for fields

---
 rust/pin-init/internal/src/init.rs | 69 ++++++++++++++++++++++++------
 1 file changed, 55 insertions(+), 14 deletions(-)
author maintainer
#15 lore raw

PATCH v4 13/15 rust: pin-init: internal: init: add escape hatch for referencing initialized fields

In reply to Benno Lossin

The initializer macro emits mutable references for already initialized fields, which allows modifying or accessing them later in code blocks or when initializing other fields. This behavior results in compiler errors when combining with packed structs, since those do not permit creating references to misaligned fields. For example:

    #[repr(C, packed)]
    struct Foo {
        a: i8,
        b: i32,
    }

    fn main() {
        let _ = init!(Foo { a: -42, b: 42 });
    }

This will lead to an error like this:

error[E0793]: reference to field of packed struct is unaligned
--> tests/ui/compile-fail/init/packed_struct.rs:10:13
|
10 | let _ = init!(Foo { a: -42, b: 42 });
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
|
= note: this struct is 1-byte aligned, but the type of this field may require higher alignment
= note: creating a misaligned reference is undefined behavior (even if that reference is never dereferenced)
= help: copy the field contents to a local variable, or replace the reference with a raw pointer and use read_unaligned/write_unaligned (loads and stores via *p must be properly aligned even when using raw pointers) = note: this error originates in the macro init (in Nightly builds, run with -Z macro-backtrace for more info)

This was requested by Janne Grunau [1] and will most certainly be used by the kernel when we eventually end up with trying to initialize packed structs.

Thus add an initializer attribute #[disable_initialized_field_access] that does what the name suggests: do not generate references to already initialized fields.

There is space for future work: add yet another attribute which can be applied on fields of initializers that ask for said field to be made accessible. We can add that when the need arises.

---
Changes in v4: none
Changes in v3: none
Changes in v2:
* silence clippy warning

---
 rust/pin-init/internal/src/init.rs | 75 +++++++++++++++++++++---------
 1 file changed, 52 insertions(+), 23 deletions(-)
rust/pin-init/internal/src/init.rs +52 -23
index e8b93b444f7e..ed2e1462e176 100644
@@ -62,6 +62,7 @@ fn ident(&self) -> Option<&Ident> {
 
 enum InitializerAttribute {
     DefaultError(DefaultErrorAttribute),
+    DisableInitializedFieldAccess,
 }
 
 struct DefaultErrorAttribute {
@@ -85,7 +86,6 @@ pub(crate) fn expand(
     let error = error.map_or_else(
         || {
             if let Some(default_error) = attrs.iter().fold(None, |acc, attr| {
-                #[expect(irrefutable_let_patterns)]
                 if let InitializerAttribute::DefaultError(DefaultErrorAttribute { ty }) = attr {
                     Some(ty.clone())
                 } else {
@@ -145,7 +145,15 @@ fn assert_zeroable<T: ?::core::marker::Sized>(_: *mut T)
     };
     // `mixed_site` ensures that the data is not accessible to the user-controlled code.
     let data = Ident::new("__data", Span::mixed_site());
-    let init_fields = init_fields(&fields, pinned, &data, &slot);
+    let init_fields = init_fields(
+        &fields,
+        pinned,
+        !attrs
+            .iter()
+            .any(|attr| matches!(attr, InitializerAttribute::DisableInitializedFieldAccess)),
+        &data,
+        &slot,
+    );
     let field_check = make_field_check(&fields, init_kind, &path);
     Ok(quote! {{
         // We do not want to allow arbitrary returns, so we declare this type as the `Ok` return
@@ -228,6 +236,7 @@ fn get_init_kind(rest: Option<(Token![..], Expr)>, dcx: &mut DiagCtxt) -> InitKi
 fn init_fields(
     fields: &Punctuated<InitializerField, Token![,]>,
     pinned: bool,
+    generate_initialized_accessors: bool,
     data: &Ident,
     slot: &Ident,
 ) -> TokenStream {
@@ -263,6 +272,13 @@ fn init_fields(
                         unsafe { &mut (*#slot).#ident }
                     }
                 };
+                let accessor = generate_initialized_accessors.then(|| {
+                    quote! {
+                        #(#cfgs)*
+                        #[allow(unused_variables)]
+                        let #ident = #accessor;
+                    }
+                });
                 quote! {
                     #(#attrs)*
                     {
@@ -270,37 +286,31 @@ fn init_fields(
                         // SAFETY: TODO
                         unsafe { #write(::core::ptr::addr_of_mut!((*#slot).#ident), #value_ident) };
                     }
-                    #(#cfgs)*
-                    #[allow(unused_variables)]
-                    let #ident = #accessor;
+                    #accessor
                 }
             }
             InitializerKind::Init { ident, value, .. } => {
                 // Again span for better diagnostics
                 let init = format_ident!("init", span = value.span());
-                if pinned {
+                let (value_init, accessor) = if pinned {
                     let project_ident = format_ident!("__project_{ident}");
-                    quote! {
-                        #(#attrs)*
-                        {
-                            let #init = #value;
+                    (
+                        quote! {
                             // SAFETY:
                             // - `slot` is valid, because we are inside of an initializer closure, we
                             //   return when an error/panic occurs.
                             // - We also use `#data` to require the correct trait (`Init` or `PinInit`)
                             //   for `#ident`.
                             unsafe { #data.#ident(::core::ptr::addr_of_mut!((*#slot).#ident), #init)? };
-                        }
-                        #(#cfgs)*
-                        // SAFETY: TODO
-                        #[allow(unused_variables)]
-                        let #ident = unsafe { #data.#project_ident(&mut (*#slot).#ident) };
-                    }
+                        },
+                        quote! {
+                            // SAFETY: TODO
+                            unsafe { #data.#project_ident(&mut (*#slot).#ident) }
+                        },
+                    )
                 } else {
-                    quote! {
-                        #(#attrs)*
-                        {
-                            let #init = #value;
+                    (
+                        quote! {
                             // SAFETY: `slot` is valid, because we are inside of an initializer
                             // closure, we return when an error/panic occurs.
                             unsafe {
@@ -309,12 +319,27 @@ fn init_fields(
                                     ::core::ptr::addr_of_mut!((*#slot).#ident),
                                 )?
                             };
-                        }
+                        },
+                        quote! {
+                            // SAFETY: TODO
+                            unsafe { &mut (*#slot).#ident }
+                        },
+                    )
+                };
+                let accessor = generate_initialized_accessors.then(|| {
+                    quote! {
                         #(#cfgs)*
-                        // SAFETY: TODO
                         #[allow(unused_variables)]
-                        let #ident = unsafe { &mut (*#slot).#ident };
+                        let #ident = #accessor;
+                    }
+                });
+                quote! {
+                    #(#attrs)*
+                    {
+                        let #init = #value;
+                        #value_init
                     }
+                    #accessor
                 }
             }
             InitializerKind::Code { block: value, .. } => quote! {
@@ -446,6 +471,10 @@ fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> {
                 if a.path().is_ident("default_error") {
                     a.parse_args::<DefaultErrorAttribute>()
                         .map(InitializerAttribute::DefaultError)
+                } else if a.path().is_ident("disable_initialized_field_access") {
+                    a.meta
+                        .require_path_only()
+                        .map(|_| InitializerAttribute::DisableInitializedFieldAccess)
                 } else {
                     Err(syn::Error::new_spanned(a, "unknown initializer attribute"))
                 }
Signature · 1 line
2.52.0
author maintainer
#16 lore raw

PATCH v4 14/15 rust: pin-init: internal: init: simplify Zeroable safety check

In reply to Benno Lossin

The Zeroable type check uses a small dance with a raw pointer to aid type inference. It turns out that this is not necessary and type inference is powerful enough to resolve any ambiguity. Thus remove it.

---
Changes in v4: none
Changes in v3: none
Changes in v2: added this patch

---
 rust/pin-init/internal/src/init.rs | 7 +------
 1 file changed, 1 insertion(+), 6 deletions(-)
rust/pin-init/internal/src/init.rs +1 -6
index ed2e1462e176..42936f915a07 100644
@@ -414,17 +414,12 @@ fn make_field_check(
             #[allow(unreachable_code, clippy::diverging_sub_expression, unused_assignments)]
             // SAFETY: this code is never executed.
             let _ = || unsafe {
-                let mut zeroed = ::core::mem::zeroed();
-                // We have to use type inference here to make zeroed have the correct type. This
-                // does not get executed, so it has no effect.
-                ::core::ptr::write(slot, zeroed);
-                zeroed = ::core::mem::zeroed();
                 ::core::ptr::write(slot, #path {
                     #(
                         #(#field_attrs)*
                         #field_name: ::core::panic!(),
                     )*
-                    ..zeroed
+                    ..::core::mem::zeroed()
                 })
             };
         },
Signature · 1 line
2.52.0
author maintainer
#17 lore raw

PATCH v4 15/15 MAINTAINERS: add Gary Guo to pin-init

In reply to Benno Lossin

Gary has been involved in pin-init since the very beginning. He created his own version before I even joined Rust-for-Linux, contributed ideas to my solution, and reviewed it. With the removal of the declarative macro mess, he is also going to be a maintainer of pin-init.

---
 MAINTAINERS | 1 +
 1 file changed, 1 insertion(+)
MAINTAINERS +1 -0
index dc731d37c8fe..fb6534339b75 100644
@@ -22916,6 +22916,7 @@ F:	rust/kernel/num/
 
 RUST [PIN-INIT]
 M:	Benno Lossin <[email protected]>
+M:	Gary Guo <[email protected]>
 L:	[email protected]
 S:	Maintained
 W:	https://rust-for-linux.com/pin-init
Signature · 1 line
2.52.0
author maintainer Applied
#18 lore raw
In reply to Benno Lossin
Quoted 82 lines · Benno Lossin

Rewrite the proc-macros of pin-init by using the syn crate for Rust syntax parsing. This series has been a long way coming. At the very start of pin-init, I initially implemented everything using syn, since parsing is so much easier with it. Now after several years it is finally time to remove the dreaded 1600 lines of declarative macros required to parse and expand the initializer syntax.

The move to syn is not only a blessing for the maintenance work, but also improves the implementation of new features. This series includes many such improvements.

* Patch 1, 2, 3 and 4 prepare for the rewrite. The first removes the superfluous try_ variants of the initializer macros from pin-init. Note that the kernel defines its own, so no code changes in the kernel are required. The second patch allows using ::pin_init in the pin-init crate. The third adds the syn dependency and cleans up some old workarounds and new clippy warnings. The fourth adds better error handling on top of syn.
* Patch 5, 6, 7, and 9 rewrite the derive macros for Zeroable, #[pinned_drop] attribute macro, #[pin_data] attribute macro, and the initializer macros respectively using syn.
* Patch 8 ensures soundness in the future by fixing generic bounds in generated code by #[pin_data].
* Patch 10 adds the #[default_error(type)] attribute to initializer macros allowing them to specify a default error that is used when no error is manually specified.
* Patch 11 uses #[default_error(type)] in the definition of the kernel's try_ variants of the initializer macros (which defaults to the kernel's Error type).
* Patch 12 allows putting attributes on fields in initializer macros (for example cfg attributes).
* Patch 13 adds #[disable_initialized_field_access] to support packed structs.
* Patch 14 simplifies the code generated by the initializer macros.
* Patch 15 adds Gary as a maintainer.

In addition to the new features, using syn results in much cleaner error messages and more accurate span information. The code is much easier to read as well and hopefully easier to understand as well.

As always, tests that ensure the correctness of the macro output are included and updated in the upstream pin-init repository. Take a look at the pull request on GitHub for the diff in the test output:

    https://github.com/Rust-for-Linux/pin-init/pull/89

I would greatly appreciate Tested-by's from people actively using pin-init weather in the kernel or outside.

The commit adding syn support to pin-init is also used by Gary's series to rewrite the macros crate using syn:

    https://lore.kernel.org/all/[email protected]

My series can be found at:

https://github.com/BennoLossin/linux/commits/pin-init/sync

The rust-dev branch in the RfL repository will be updated to contain both patch series for ease of use & testing.

Link to previous versions:

- v3: https://lore.kernel.org/all/[email protected]
- v2: https://lore.kernel.org/all/[email protected]
- v1: https://lore.kernel.org/all/[email protected]

Cheers,
Benno

Benno Lossin (15):
rust: pin-init: remove try_ versions of the initializer macros
rust: pin-init: allow the crate to refer to itself as pin-init in
doc tests
rust: pin-init: add syn dependency and remove proc-macro[2] and
quote workarounds
rust: pin-init: internal: add utility API for syn error handling
rust: pin-init: rewrite derive(Zeroable) and derive(MaybeZeroable)
using syn
rust: pin-init: rewrite the #[pinned_drop] attribute macro using

    `syn`
Quoted 12 lines

rust: pin-init: rewrite #[pin_data] using syn
rust: pin-init: add ?Sized bounds to traits in #[pin_data] macro
rust: pin-init: rewrite the initializer macros using syn
rust: pin-init: add #[default_error(<type>)] attribute to
initializer macros
rust: init: use #[default_error(err)] for the initializer macros
rust: pin-init: internal: init: add support for attributes on
initializer fields
rust: pin-init: internal: init: add escape hatch for referencing
initialized fields
rust: pin-init: internal: init: simplify Zeroable safety check
MAINTAINERS: add Gary Guo to pin-init

Applied to pin-init-next, thanks everyone!

Cheers,
Benno

Quoted 24 lines
 MAINTAINERS                               |    1 +
 rust/Makefile                             |   16 +-
 rust/kernel/init.rs                       |   40 +-
 rust/pin-init/README.md                   |    2 +-
 rust/pin-init/examples/linked_list.rs     |   19 +-
 rust/pin-init/examples/pthread_mutex.rs   |   10 +-
 rust/pin-init/internal/src/diagnostics.rs |   30 +
 rust/pin-init/internal/src/helpers.rs     |  152 --
 rust/pin-init/internal/src/init.rs        |  548 +++++++
 rust/pin-init/internal/src/lib.rs         |   48 +-
 rust/pin-init/internal/src/pin_data.rs    |  615 ++++++--
 rust/pin-init/internal/src/pinned_drop.rs |   88 +-
 rust/pin-init/internal/src/zeroable.rs    |  157 +-
 rust/pin-init/src/lib.rs                  |  173 +--
 rust/pin-init/src/macros.rs               | 1677 ---------------------
 scripts/generate_rust_analyzer.py         |    2 +-
 16 files changed, 1278 insertions(+), 2300 deletions(-)
 create mode 100644 rust/pin-init/internal/src/diagnostics.rs
 delete mode 100644 rust/pin-init/internal/src/helpers.rs
 create mode 100644 rust/pin-init/internal/src/init.rs
 delete mode 100644 rust/pin-init/src/macros.rs

base-commit: 9448598b22c50c8a5bb77a9103e2d49f134c9578