Skip to contentSkip to conversation

refactor Rust proc macros with `syn`

PATCH Applied Gary Guo 6 participants 27 messages Jan 12 – Jan 28

Subsystems (6)
Kselftest Rust Module support KUnit +2 more subsystems
lore
Share & tools
lore

Gary Guo

author
#0 lore raw

v3 cover refactor Rust proc macros with `syn`

From: Gary Guo <[email protected]>

This series convert Rust proc macros that we have with syn, and replace the custom quote! macro that we have with the vendored quote! macro. The pin-init macros are not converted syn yet; Benno has a work in progress in converting them. They're however converted to use quote and proc-macro2 crate so our custom quote! macro can be removed.

Overall this improves the robustness of the macros as we have precise parsing of the AST rather than relying on heuristics to extract needed information from there. This is also a quality-of-life improvement to those using language servers (e.g. Rust analyzer) as the span information of the proc macros are now preserved which allows the "jump-to-definition" feature to work, even when used on completely custom macros such as module!.

Miguel gave a very good explaination on why syn is a good idea in the patch series that introduced it [1], which I shall not repeat here.

Changes since v2:
- Changes are limited to #[vtable] and module! macros.
- #[vtable] macro is reworked per Tamir's suggestion. This makes it easy
to support #[cfg] properly, and hence a new patch is added to do that.
- Avoided variable renames related to param_name/param_type between
patches in module! macro
- CString is now interpolated directly without adding extra Literal::c_string.
- Fixed an typo that affects only CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
builds.
- Link to v2: https://lore.kernel.org/rust-for-linux/[email protected]/

Changes since v1:
- Fixed clippy warnings when moving from proc-macro to proc-macro2
(extra trait impl in proc-macro2 causes some .to_string() to be unnecessary).
- A Makefile change to pin-init-internal that is mistakenly included in
patch 5/11 is moved to the correct patch 1/11.
- module! parsing has been completely reworked to support imports_ns
and params. As there're two structs that need to parse ordered fields (ModuleInfo and Parameter), a parse_ordered_field macro is added
and it is used to implement the parsing instead of ModInfoField and
custom keywords.
- Link to v1: https://lore.kernel.org/rust-for-linux/[email protected]/

Benno Lossin (1):
rust: pin-init: internal: remove proc-macro[2] and quote workarounds

Gary Guo (11):
rust: macros: use quote! from vendored crate
rust: macros: convert #[vtable] macro to use syn
rust: macros: use syn to parse module! macro
rust: macros: use quote! for module! macro
rust: macros: convert #[export] to use syn
rust: macros: convert concat_idents! to use syn
rust: macros: convert #[kunit_tests] macro to use syn
rust: macros: allow arbitrary types to be used in module! macro
rust: macros: rearrange #[doc(hidden)] in module! macro
rust: kunit: use pin_init::zeroed instead of custom null value
rust: macros: support #[cfg] properly in #[vtable] macro.

 rust/Makefile                             |  16 +-
 rust/kernel/kunit.rs                      |  26 +-
 rust/macros/concat_idents.rs              |  39 +-
 rust/macros/export.rs                     |  26 +-
 rust/macros/fmt.rs                        |   4 +-
 rust/macros/helpers.rs                    | 130 +---
 rust/macros/kunit.rs                      | 275 +++----
 rust/macros/lib.rs                        |  41 +-
 rust/macros/module.rs                     | 907 ++++++++++++----------
 rust/macros/paste.rs                      |   2 +-
 rust/macros/quote.rs                      | 182 -----
 rust/macros/vtable.rs                     | 165 ++--
 rust/pin-init/internal/src/helpers.rs     |   7 +-
 rust/pin-init/internal/src/lib.rs         |  16 -
 rust/pin-init/internal/src/pin_data.rs    |  18 +-
 rust/pin-init/internal/src/pinned_drop.rs |  10 +-
 rust/pin-init/internal/src/zeroable.rs    |   6 +-
 scripts/generate_rust_analyzer.py         |   2 +-
 18 files changed, 842 insertions(+), 1030 deletions(-)
 delete mode 100644 rust/macros/quote.rs

base-commit: 9ace4753a5202b02191d54e9fdf7f9e3d02b85eb

Signature · 1 line
2.51.2
author
#1 lore raw

PATCH v3 1/12 rust: pin-init: internal: remove proc-macro[2] and quote workarounds

From: Benno Lossin <[email protected]>

The kernel only had the proc-macro library available, whereas the user-space version also used proc-macro2 and quote. Now both are available to the kernel, making it possible to remove the workarounds.

Clippy complains about unnecessary .to_string() as proc-macro2 provides additional PartialEq impl, so they are removed.

---
 rust/Makefile                             | 16 ++++++++++------
 rust/pin-init/internal/src/helpers.rs     |  7 ++-----
 rust/pin-init/internal/src/lib.rs         | 16 ----------------
 rust/pin-init/internal/src/pin_data.rs    | 18 ++++++------------
 rust/pin-init/internal/src/pinned_drop.rs | 10 ++++------
 rust/pin-init/internal/src/zeroable.rs    |  6 ++----
 scripts/generate_rust_analyzer.py         |  2 +-
 7 files changed, 25 insertions(+), 50 deletions(-)
rust/Makefile +10 -6
index 5d357dce1704d..96f0ac53ec0e8 100644
@@ -212,9 +212,10 @@ rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE
 
 rustdoc-pin_init_internal: private rustdoc_host = yes
 rustdoc-pin_init_internal: private rustc_target_flags = --cfg kernel \
-    --extern proc_macro --crate-type proc-macro
+    --extern proc_macro --extern proc_macro2 --extern quote --extern syn \
+    --crate-type proc-macro
 rustdoc-pin_init_internal: $(src)/pin-init/internal/src/lib.rs \
-    rustdoc-clean FORCE
+    rustdoc-clean rustdoc-proc_macro2 rustdoc-quote rustdoc-syn FORCE
 	+$(call if_changed,rustdoc)
 
 rustdoc-pin_init: private rustdoc_host = yes
@@ -273,9 +274,10 @@ rusttestlib-macros: $(src)/macros/lib.rs \
 	+$(call if_changed,rustc_test_library)
 
 rusttestlib-pin_init_internal: private rustc_target_flags = --cfg kernel \
-    --extern proc_macro
+    --extern proc_macro --extern proc_macro2 --extern quote --extern syn
 rusttestlib-pin_init_internal: private rustc_test_library_proc = yes
-rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs FORCE
+rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs \
+    rusttestlib-proc_macro2 rusttestlib-quote rusttestlib-syn FORCE
 	+$(call if_changed,rustc_test_library)
 
 rusttestlib-pin_init: private rustc_target_flags = --extern pin_init_internal \
@@ -547,8 +549,10 @@ $(obj)/$(libmacros_name): $(src)/macros/lib.rs $(obj)/libproc_macro2.rlib \
     $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE
 	+$(call if_changed_dep,rustc_procmacro)
 
-$(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel
-$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs FORCE
+$(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel \
+    --extern proc_macro2 --extern quote --extern syn
+$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs \
+	$(obj)/libproc_macro2.rlib $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE
 	+$(call if_changed_dep,rustc_procmacro)
 
 quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@
rust/pin-init/internal/src/helpers.rs +2 -5
index 236f989a50f2f..90f85eaa41231 100644
@@ -1,9 +1,6 @@
 // SPDX-License-Identifier: Apache-2.0 OR MIT
 
-#[cfg(not(kernel))]
-use proc_macro2 as proc_macro;
-
-use proc_macro::{TokenStream, TokenTree};
+use proc_macro2::{TokenStream, TokenTree};
 
 /// Parsed generics.
 ///
@@ -101,7 +98,7 @@ pub(crate) fn parse_generics(input: TokenStream) -> (Generics, Vec<TokenTree>) {
                     1 => {
                         // Here depending on the token, it might be a generic variable name.
                         match tt.clone() {
-                            TokenTree::Ident(i) if at_start && i.to_string() == "const" => {
+                            TokenTree::Ident(i) if at_start && i == "const" => {
                                 let Some(name) = toks.next() else {
                                     // Parsing error.
                                     break;
rust/pin-init/internal/src/lib.rs +0 -16
index 297b0129a5bfd..4c4dc639ce823 100644
@@ -7,27 +7,11 @@
 //! `pin-init` proc macros.
 
 #![cfg_attr(not(RUSTC_LINT_REASONS_IS_STABLE), feature(lint_reasons))]
-// Allow `.into()` to convert
-// - `proc_macro2::TokenStream` into `proc_macro::TokenStream` in the user-space version.
-// - `proc_macro::TokenStream` into `proc_macro::TokenStream` in the kernel version.
-//   Clippy warns on this conversion, but it's required by the user-space version.
-//
-// Remove once we have `proc_macro2` in the kernel.
-#![allow(clippy::useless_conversion)]
 // Documentation is done in the pin-init crate instead.
 #![allow(missing_docs)]
 
 use proc_macro::TokenStream;
 
-#[cfg(kernel)]
-#[path = "../../../macros/quote.rs"]
-#[macro_use]
-#[cfg_attr(not(kernel), rustfmt::skip)]
-mod quote;
-#[cfg(not(kernel))]
-#[macro_use]
-extern crate quote;
-
 mod helpers;
 mod pin_data;
 mod pinned_drop;
rust/pin-init/internal/src/pin_data.rs +6 -12
index 87d4a7eb1d35e..86a53b37cc660 100644
@@ -1,10 +1,8 @@
 // SPDX-License-Identifier: Apache-2.0 OR MIT
 
-#[cfg(not(kernel))]
-use proc_macro2 as proc_macro;
-
 use crate::helpers::{parse_generics, Generics};
-use proc_macro::{Group, Punct, Spacing, TokenStream, TokenTree};
+use proc_macro2::{Group, Punct, Spacing, TokenStream, TokenTree};
+use quote::quote;
 
 pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
     // This proc-macro only does some pre-parsing and then delegates the actual parsing to
@@ -28,7 +26,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
     // The name of the struct with ty_generics.
     let struct_name = rest
         .iter()
-        .skip_while(|tt| !matches!(tt, TokenTree::Ident(i) if i.to_string() == "struct"))
+        .skip_while(|tt| !matches!(tt, TokenTree::Ident(i) if i == "struct"))
         .nth(1)
         .and_then(|tt| match tt {
             TokenTree::Ident(_) => {
@@ -65,7 +63,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {
         .into_iter()
         .flat_map(|tt| {
             // We ignore top level `struct` tokens, since they would emit a compile error.
-            if matches!(&tt, TokenTree::Ident(i) if i.to_string() == "struct") {
+            if matches!(&tt, TokenTree::Ident(i) if i == "struct") {
                 vec![tt]
             } else {
                 replace_self_and_deny_type_defs(&struct_name, tt, &mut errs)
@@ -98,11 +96,7 @@ fn replace_self_and_deny_type_defs(
 ) -> Vec<TokenTree> {
     match tt {
         TokenTree::Ident(ref i)
-            if i.to_string() == "enum"
-                || i.to_string() == "trait"
-                || i.to_string() == "struct"
-                || i.to_string() == "union"
-                || i.to_string() == "impl" =>
+            if i == "enum" || i == "trait" || i == "struct" || i == "union" || i == "impl" =>
         {
             errs.extend(
                 format!(
@@ -119,7 +113,7 @@ fn replace_self_and_deny_type_defs(
             );
             vec![tt]
         }
-        TokenTree::Ident(i) if i.to_string() == "Self" => struct_name.clone(),
+        TokenTree::Ident(i) if i == "Self" => struct_name.clone(),
         TokenTree::Literal(_) | TokenTree::Punct(_) | TokenTree::Ident(_) => vec![tt],
         TokenTree::Group(g) => vec![TokenTree::Group(Group::new(
             g.delimiter(),
rust/pin-init/internal/src/pinned_drop.rs +4 -6
index c4ca7a70b726a..cf8cd1c429849 100644
@@ -1,15 +1,13 @@
 // SPDX-License-Identifier: Apache-2.0 OR MIT
 
-#[cfg(not(kernel))]
-use proc_macro2 as proc_macro;
-
-use proc_macro::{TokenStream, TokenTree};
+use proc_macro2::{TokenStream, TokenTree};
+use quote::quote;
 
 pub(crate) fn pinned_drop(_args: TokenStream, input: TokenStream) -> TokenStream {
     let mut toks = input.into_iter().collect::<Vec<_>>();
     assert!(!toks.is_empty());
     // Ensure that we have an `impl` item.
-    assert!(matches!(&toks[0], TokenTree::Ident(i) if i.to_string() == "impl"));
+    assert!(matches!(&toks[0], TokenTree::Ident(i) if i == "impl"));
     // Ensure that we are implementing `PinnedDrop`.
     let mut nesting: usize = 0;
     let mut pinned_drop_idx = None;
@@ -27,7 +25,7 @@ pub(crate) fn pinned_drop(_args: TokenStream, input: TokenStream) -> TokenStream
         if i >= 1 && nesting == 0 {
             // Found the end of the generics, this should be `PinnedDrop`.
             assert!(
-                matches!(tt, TokenTree::Ident(i) if i.to_string() == "PinnedDrop"),
+                matches!(tt, TokenTree::Ident(i) if i == "PinnedDrop"),
                 "expected 'PinnedDrop', found: '{tt:?}'"
             );
             pinned_drop_idx = Some(i);
rust/pin-init/internal/src/zeroable.rs +2 -4
index e0ed3998445cf..d8a5ef3883f4b 100644
@@ -1,10 +1,8 @@
 // SPDX-License-Identifier: GPL-2.0
 
-#[cfg(not(kernel))]
-use proc_macro2 as proc_macro;
-
 use crate::helpers::{parse_generics, Generics};
-use proc_macro::{TokenStream, TokenTree};
+use proc_macro2::{TokenStream, TokenTree};
+use quote::quote;
 
 pub(crate) fn parse_zeroable_derive_input(
     input: TokenStream,
scripts/generate_rust_analyzer.py +1 -1
index 147d0cc940681..d31d938886589 100755
@@ -123,7 +123,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs, core_edit
     append_crate(
         "pin_init_internal",
         srctree / "rust" / "pin-init" / "internal" / "src" / "lib.rs",
-        [],
+        ["std", "proc_macro", "proc_macro2", "quote", "syn"],
         cfg=["kernel"],
         is_proc_macro=True,
     )
Signature · 1 line
2.51.2
author
#2 lore raw

PATCH v3 2/12 rust: macros: use `quote!` from vendored crate

In reply to Gary Guo

From: Gary Guo <[email protected]>

With quote crate now vendored in the kernel, we can remove our custom quote! macro implementation and just rely on that crate instead.

The quote crate uses types from the proc-macro2 library so we also update to use that, and perform conversion in the top-level lib.rs.

Clippy complains about unnecessary .to_string() as proc-macro2 provides additional PartialEq impl, so they are removed.

---
 rust/macros/concat_idents.rs |   2 +-
 rust/macros/export.rs        |   4 +-
 rust/macros/fmt.rs           |   4 +-
 rust/macros/helpers.rs       |   4 +-
 rust/macros/kunit.rs         |   5 +-
 rust/macros/lib.rs           |  21 ++--
 rust/macros/module.rs        |   6 +-
 rust/macros/paste.rs         |   2 +-
 rust/macros/quote.rs         | 182 -----------------------------------
 rust/macros/vtable.rs        |   7 +-
 10 files changed, 32 insertions(+), 205 deletions(-)
 delete mode 100644 rust/macros/quote.rs
rust/macros/concat_idents.rs +1 -1
index 7e4b450f3a507..12cb231c3d715 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{token_stream, Ident, TokenStream, TokenTree};
+use proc_macro2::{token_stream, Ident, TokenStream, TokenTree};
 
 use crate::helpers::expect_punct;
 
rust/macros/export.rs +3 -1
index a08f6337d5c8d..92d9b30971929 100644
@@ -1,7 +1,9 @@
 // SPDX-License-Identifier: GPL-2.0
 
+use proc_macro2::TokenStream;
+use quote::quote;
+
 use crate::helpers::function_name;
-use proc_macro::TokenStream;
 
 /// Please see [`crate::export`] for documentation.
 pub(crate) fn export(_attr: TokenStream, ts: TokenStream) -> TokenStream {
rust/macros/fmt.rs +3 -1
index 2f4b9f6e22110..19f709262552b 100644
@@ -1,8 +1,10 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{Ident, TokenStream, TokenTree};
 use std::collections::BTreeSet;
 
+use proc_macro2::{Ident, TokenStream, TokenTree};
+use quote::quote_spanned;
+
 /// Please see [`crate::fmt`] for documentation.
 pub(crate) fn fmt(input: TokenStream) -> TokenStream {
     let mut input = input.into_iter();
rust/macros/helpers.rs +2 -2
index 365d7eb499c08..13fafaba12261 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{token_stream, Group, Ident, TokenStream, TokenTree};
+use proc_macro2::{token_stream, Group, Ident, TokenStream, TokenTree};
 
 pub(crate) fn try_ident(it: &mut token_stream::IntoIter) -> Option<String> {
     if let Some(TokenTree::Ident(ident)) = it.next() {
@@ -86,7 +86,7 @@ pub(crate) fn function_name(input: TokenStream) -> Option<Ident> {
     let mut input = input.into_iter();
     while let Some(token) = input.next() {
         match token {
-            TokenTree::Ident(i) if i.to_string() == "fn" => {
+            TokenTree::Ident(i) if i == "fn" => {
                 if let Some(TokenTree::Ident(i)) = input.next() {
                     return Some(i);
                 }
rust/macros/kunit.rs +3 -2
index b395bb0536959..5cd6aa5eef07d 100644
@@ -4,10 +4,11 @@
 //!
 //! Copyright (c) 2023 José Expósito <[email protected]>
 
-use proc_macro::{Delimiter, Group, TokenStream, TokenTree};
 use std::collections::HashMap;
 use std::fmt::Write;
 
+use proc_macro2::{Delimiter, Group, TokenStream, TokenTree};
+
 pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
     let attr = attr.to_string();
 
@@ -59,7 +60,7 @@ pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
                 }
                 _ => (),
             },
-            TokenTree::Ident(i) if i.to_string() == "fn" && attributes.contains_key("test") => {
+            TokenTree::Ident(i) if i == "fn" && attributes.contains_key("test") => {
                 if let Some(TokenTree::Ident(test_name)) = body_it.next() {
                     tests.push((test_name, attributes.remove("cfg").unwrap_or_default()))
                 }
rust/macros/lib.rs +11 -10
index b38002151871a..945982c21f703 100644
@@ -11,8 +11,6 @@
 // to avoid depending on the full `proc_macro_span` on Rust >= 1.88.0.
 #![cfg_attr(not(CONFIG_RUSTC_HAS_SPAN_FILE), feature(proc_macro_span))]
 
-#[macro_use]
-mod quote;
 mod concat_idents;
 mod export;
 mod fmt;
@@ -132,7 +130,7 @@
 ///     the kernel module.
 #[proc_macro]
 pub fn module(ts: TokenStream) -> TokenStream {
-    module::module(ts)
+    module::module(ts.into()).into()
 }
 
 /// Declares or implements a vtable trait.
@@ -207,7 +205,7 @@ pub fn module(ts: TokenStream) -> TokenStream {
 /// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html
 #[proc_macro_attribute]
 pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    vtable::vtable(attr, ts)
+    vtable::vtable(attr.into(), ts.into()).into()
 }
 
 /// Export a function so that C code can call it via a header file.
@@ -230,7 +228,7 @@ pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream {
 /// automatically exported with `EXPORT_SYMBOL_GPL`.
 #[proc_macro_attribute]
 pub fn export(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    export::export(attr, ts)
+    export::export(attr.into(), ts.into()).into()
 }
 
 /// Like [`core::format_args!`], but automatically wraps arguments in [`kernel::fmt::Adapter`].
@@ -248,7 +246,7 @@ pub fn export(attr: TokenStream, ts: TokenStream) -> TokenStream {
 /// [`pr_info!`]: ../kernel/macro.pr_info.html
 #[proc_macro]
 pub fn fmt(input: TokenStream) -> TokenStream {
-    fmt::fmt(input)
+    fmt::fmt(input.into()).into()
 }
 
 /// Concatenate two identifiers.
@@ -306,7 +304,7 @@ pub fn fmt(input: TokenStream) -> TokenStream {
 /// ```
 #[proc_macro]
 pub fn concat_idents(ts: TokenStream) -> TokenStream {
-    concat_idents::concat_idents(ts)
+    concat_idents::concat_idents(ts.into()).into()
 }
 
 /// Paste identifiers together.
@@ -444,9 +442,12 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {
 /// [`paste`]: https://docs.rs/paste/
 #[proc_macro]
 pub fn paste(input: TokenStream) -> TokenStream {
-    let mut tokens = input.into_iter().collect();
+    let mut tokens = proc_macro2::TokenStream::from(input).into_iter().collect();
     paste::expand(&mut tokens);
-    tokens.into_iter().collect()
+    tokens
+        .into_iter()
+        .collect::<proc_macro2::TokenStream>()
+        .into()
 }
 
 /// Registers a KUnit test suite and its test cases using a user-space like syntax.
@@ -473,5 +474,5 @@ pub fn paste(input: TokenStream) -> TokenStream {
 /// ```
 #[proc_macro_attribute]
 pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    kunit::kunit_tests(attr, ts)
+    kunit::kunit_tests(attr.into(), ts.into()).into()
 }
rust/macros/module.rs +4 -2
index 80cb9b16f5aaf..b855a2b586e18 100644
@@ -1,9 +1,11 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use crate::helpers::*;
-use proc_macro::{token_stream, Delimiter, Literal, TokenStream, TokenTree};
 use std::fmt::Write;
 
+use proc_macro2::{token_stream, Delimiter, Literal, TokenStream, TokenTree};
+
+use crate::helpers::*;
+
 fn expect_string_array(it: &mut token_stream::IntoIter) -> Vec<String> {
     let group = expect_group(it);
     assert_eq!(group.delimiter(), Delimiter::Bracket);
rust/macros/paste.rs +1 -1
index cce712d19855b..2181e312a7d32 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{Delimiter, Group, Ident, Spacing, Span, TokenTree};
+use proc_macro2::{Delimiter, Group, Ident, Spacing, Span, TokenTree};
 
 fn concat_helper(tokens: &[TokenTree]) -> Vec<(String, Span)> {
     let mut tokens = tokens.iter();
rust/macros/quote.rs deleted +0 -182

182 lines not on this page · raw message

rust/macros/vtable.rs +4 -3
index ee06044fcd4f3..a67d1cc81a2d3 100644
@@ -1,9 +1,10 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{Delimiter, Group, TokenStream, TokenTree};
 use std::collections::HashSet;
 use std::fmt::Write;
 
+use proc_macro2::{Delimiter, Group, TokenStream, TokenTree};
+
 pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream {
     let mut tokens: Vec<_> = ts.into_iter().collect();
 
@@ -31,7 +32,7 @@ pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream {
     let mut consts = HashSet::new();
     while let Some(token) = body_it.next() {
         match token {
-            TokenTree::Ident(ident) if ident.to_string() == "fn" => {
+            TokenTree::Ident(ident) if ident == "fn" => {
                 let fn_name = match body_it.next() {
                     Some(TokenTree::Ident(ident)) => ident.to_string(),
                     // Possibly we've encountered a fn pointer type instead.
@@ -39,7 +40,7 @@ pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream {
                 };
                 functions.push(fn_name);
             }
-            TokenTree::Ident(ident) if ident.to_string() == "const" => {
+            TokenTree::Ident(ident) if ident == "const" => {
                 let const_name = match body_it.next() {
                     Some(TokenTree::Ident(ident)) => ident.to_string(),
                     // Possibly we've encountered an inline const block instead.
Signature · 1 line
2.51.2
author
#4 lore raw

PATCH v3 3/12 rust: macros: convert `#[vtable]` macro to use `syn`

In reply to Gary Guo

From: Gary Guo <[email protected]>

#[vtable] is converted to use syn. This is more robust than the previous heuristic-based searching of defined methods and functions.

When doing so, the trait and impl are split into two code paths as the types are distinct when parsed by syn.

---
 rust/macros/lib.rs    |   9 ++-
 rust/macros/vtable.rs | 169 +++++++++++++++++++++++-------------------
 2 files changed, 98 insertions(+), 80 deletions(-)
rust/macros/lib.rs +7 -2
index 945982c21f703..9955c04dbaae3 100644
@@ -22,6 +22,8 @@
 
 use proc_macro::TokenStream;
 
+use syn::parse_macro_input;
+
 /// Declares a kernel module.
 ///
 /// The `type` argument should be a type which implements the [`Module`]
@@ -204,8 +206,11 @@ pub fn module(ts: TokenStream) -> TokenStream {
 ///
 /// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html
 #[proc_macro_attribute]
-pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    vtable::vtable(attr.into(), ts.into()).into()
+pub fn vtable(attr: TokenStream, input: TokenStream) -> TokenStream {
+    parse_macro_input!(attr as syn::parse::Nothing);
+    vtable::vtable(parse_macro_input!(input))
+        .unwrap_or_else(|e| e.into_compile_error())
+        .into()
 }
 
 /// Export a function so that C code can call it via a header file.
rust/macros/vtable.rs +91 -78

188 lines not on this page · raw message

Signature · 1 line
2.51.2
author
#6 lore raw

PATCH v3 4/12 rust: macros: use `syn` to parse `module!` macro

In reply to Gary Guo

From: Gary Guo <[email protected]>

With syn being available in the kernel, use it to parse the complex custom module! macro to replace existing helpers. Only parsing is changed in this commit, the code generation is untouched.

This has the benefit of better error message when the macro is used incorrectly, as it can point to a concrete span on what's going wrong.

For example, if a field is specified twice, previously it reads:

    error: proc macro panicked
      --> samples/rust/rust_minimal.rs:7:1
       |
    7  | / module! {
    8  | |     type: RustMinimal,
    9  | |     name: "rust_minimal",
    10 | |     author: "Rust for Linux Contributors",
    11 | |     description: "Rust minimal sample",
    12 | |     license: "GPL",
    13 | |     license: "GPL",
    14 | | }
       | |_^
       |
       = help: message: Duplicated key "license". Keys can only be specified once.

now it reads:

    error: duplicated key "license". Keys can only be specified once.
      --> samples/rust/rust_minimal.rs:13:5
       |
    13 |     license: "GPL",
       |     ^^^^^^^
---
 rust/macros/helpers.rs | 109 ++++-------
 rust/macros/lib.rs     |   6 +-
 rust/macros/module.rs  | 399 +++++++++++++++++++++++++----------------
 3 files changed, 280 insertions(+), 234 deletions(-)
rust/macros/helpers.rs +32 -77
index 13fafaba12261..fa66ef6eb0f3d 100644
@@ -1,53 +1,21 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro2::{token_stream, Group, Ident, TokenStream, TokenTree};
-
-pub(crate) fn try_ident(it: &mut token_stream::IntoIter) -> Option<String> {
-    if let Some(TokenTree::Ident(ident)) = it.next() {
-        Some(ident.to_string())
-    } else {
-        None
-    }
-}
-
-pub(crate) fn try_sign(it: &mut token_stream::IntoIter) -> Option<char> {
-    let peek = it.clone().next();
-    match peek {
-        Some(TokenTree::Punct(punct)) if punct.as_char() == '-' => {
-            let _ = it.next();
-            Some(punct.as_char())
-        }
-        _ => None,
-    }
-}
-
-pub(crate) fn try_literal(it: &mut token_stream::IntoIter) -> Option<String> {
-    if let Some(TokenTree::Literal(literal)) = it.next() {
-        Some(literal.to_string())
-    } else {
-        None
-    }
-}
-
-pub(crate) fn try_string(it: &mut token_stream::IntoIter) -> Option<String> {
-    try_literal(it).and_then(|string| {
-        if string.starts_with('\"') && string.ends_with('\"') {
-            let content = &string[1..string.len() - 1];
-            if content.contains('\\') {
-                panic!("Escape sequences in string literals not yet handled");
-            }
-            Some(content.to_string())
-        } else if string.starts_with("r\"") {
-            panic!("Raw string literals are not yet handled");
-        } else {
-            None
-        }
-    })
-}
-
-pub(crate) fn expect_ident(it: &mut token_stream::IntoIter) -> String {
-    try_ident(it).expect("Expected Ident")
-}
+use proc_macro2::{
+    token_stream,
+    Ident,
+    TokenStream,
+    TokenTree, //
+};
+use quote::ToTokens;
+use syn::{
+    parse::{
+        Parse,
+        ParseStream, //
+    },
+    Error,
+    LitStr,
+    Result, //
+};
 
 pub(crate) fn expect_punct(it: &mut token_stream::IntoIter) -> char {
     if let TokenTree::Punct(punct) = it.next().expect("Reached end of token stream for Punct") {
@@ -57,27 +25,28 @@ pub(crate) fn expect_punct(it: &mut token_stream::IntoIter) -> char {
     }
 }
 
-pub(crate) fn expect_string(it: &mut token_stream::IntoIter) -> String {
-    try_string(it).expect("Expected string")
-}
+/// A string literal that is required to have ASCII value only.
+pub(crate) struct AsciiLitStr(LitStr);
 
-pub(crate) fn expect_string_ascii(it: &mut token_stream::IntoIter) -> String {
-    let string = try_string(it).expect("Expected string");
-    assert!(string.is_ascii(), "Expected ASCII string");
-    string
+impl Parse for AsciiLitStr {
+    fn parse(input: ParseStream<'_>) -> Result<Self> {
+        let s: LitStr = input.parse()?;
+        if !s.value().is_ascii() {
+            return Err(Error::new_spanned(s, "expected ASCII-only string literal"));
+        }
+        Ok(Self(s))
+    }
 }
 
-pub(crate) fn expect_group(it: &mut token_stream::IntoIter) -> Group {
-    if let TokenTree::Group(group) = it.next().expect("Reached end of token stream for Group") {
-        group
-    } else {
-        panic!("Expected Group");
+impl ToTokens for AsciiLitStr {
+    fn to_tokens(&self, ts: &mut TokenStream) {
+        self.0.to_tokens(ts);
     }
 }
 
-pub(crate) fn expect_end(it: &mut token_stream::IntoIter) {
-    if it.next().is_some() {
-        panic!("Expected end");
+impl AsciiLitStr {
+    pub(crate) fn value(&self) -> String {
+        self.0.value()
     }
 }
 
@@ -114,17 +83,3 @@ pub(crate) fn file() -> String {
         proc_macro::Span::call_site().file()
     }
 }
-
-/// Parse a token stream of the form `expected_name: "value",` and return the
-/// string in the position of "value".
-///
-/// # Panics
-///
-/// - On parse error.
-pub(crate) fn expect_string_field(it: &mut token_stream::IntoIter, expected_name: &str) -> String {
-    assert_eq!(expect_ident(it), expected_name);
-    assert_eq!(expect_punct(it), ':');
-    let string = expect_string(it);
-    assert_eq!(expect_punct(it), ',');
-    string
-}
rust/macros/lib.rs +4 -2
index 9955c04dbaae3..c5347127a3a51 100644
@@ -131,8 +131,10 @@
 ///   - `firmware`: array of ASCII string literals of the firmware files of
 ///     the kernel module.
 #[proc_macro]
-pub fn module(ts: TokenStream) -> TokenStream {
-    module::module(ts.into()).into()
+pub fn module(input: TokenStream) -> TokenStream {
+    module::module(parse_macro_input!(input))
+        .unwrap_or_else(|e| e.into_compile_error())
+        .into()
 }
 
 /// Declares or implements a vtable trait.
rust/macros/module.rs +244 -155

508 lines not on this page · raw message

Signature · 1 line
2.51.2
author
#8 lore raw

PATCH v3 5/12 rust: macros: use `quote!` for `module!` macro

In reply to Gary Guo

From: Gary Guo <[email protected]>

This has no behavioural change, but is good for maintainability. With quote!, we're no longer using string templates, so we don't need to quote " and {} inside the template anymore. Further more, editors can now highlight the code template.

This also improves the robustness as it eliminates the need for string quoting and escaping.

---
 rust/macros/module.rs | 543 ++++++++++++++++++++++--------------------
 1 file changed, 286 insertions(+), 257 deletions(-)
rust/macros/module.rs +286 -257
index 9fdc9ed1faaf3..43ada49525c9d 100644
@@ -1,12 +1,15 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use std::fmt::Write;
+use std::ffi::CString;
 
 use proc_macro2::{
     Literal,
     TokenStream, //
 };
-use quote::ToTokens;
+use quote::{
+    format_ident,
+    quote, //
+};
 use syn::{
     braced,
     bracketed,
@@ -15,11 +18,13 @@
         Parse,
         ParseStream, //
     },
+    parse_quote,
     punctuated::Punctuated,
     Error,
     Expr,
     Ident,
     LitStr,
+    Path,
     Result,
     Token, //
 };
@@ -29,8 +34,8 @@
 struct ModInfoBuilder<'a> {
     module: &'a str,
     counter: usize,
-    buffer: String,
-    param_buffer: String,
+    ts: TokenStream,
+    param_ts: TokenStream,
 }
 
 impl<'a> ModInfoBuilder<'a> {
@@ -38,8 +43,8 @@ fn new(module: &'a str) -> Self {
         ModInfoBuilder {
             module,
             counter: 0,
-            buffer: String::new(),
-            param_buffer: String::new(),
+            ts: TokenStream::new(),
+            param_ts: TokenStream::new(),
         }
     }
 
@@ -56,33 +61,32 @@ fn emit_base(&mut self, field: &str, content: &str, builtin: bool, param: bool)
             // Loadable modules' modinfo strings go as-is.
             format!("{field}={content}\0")
         };
-
-        let buffer = if param {
-            &mut self.param_buffer
+        let length = string.len();
+        let string = Literal::byte_string(string.as_bytes());
+        let cfg = if builtin {
+            quote!(#[cfg(not(MODULE))])
         } else {
-            &mut self.buffer
+            quote!(#[cfg(MODULE)])
         };
 
-        write!(
-            buffer,
-            "
-                {cfg}
-                #[doc(hidden)]
-                #[cfg_attr(not(target_os = \"macos\"), link_section = \".modinfo\")]
-                #[used(compiler)]
-                pub static __{module}_{counter}: [u8; {length}] = *{string};
-            ",
-            cfg = if builtin {
-                "#[cfg(not(MODULE))]"
-            } else {
-                "#[cfg(MODULE)]"
-            },
+        let counter = format_ident!(
+            "__{module}_{counter}",
             module = self.module.to_uppercase(),
-            counter = self.counter,
-            length = string.len(),
-            string = Literal::byte_string(string.as_bytes()),
-        )
-        .unwrap();
+            counter = self.counter
+        );
+        let item = quote! {
+            #cfg
+            #[doc(hidden)]
+            #[cfg_attr(not(target_os = "macos"), link_section = ".modinfo")]
+            #[used(compiler)]
+            pub static #counter: [u8; #length] = *#string;
+        };
+
+        if param {
+            self.param_ts.extend(item);
+        } else {
+            self.ts.extend(item);
+        }
 
         self.counter += 1;
     }
@@ -117,7 +121,6 @@ fn emit_params(&mut self, info: &ModuleInfo) {
         for param in params {
             let param_name_str = param.name.to_string();
             let param_type_str = param.ptype.to_string();
-            let param_default = param.default.to_token_stream().to_string();
 
             let ops = param_ops_path(&param_type_str);
 
@@ -126,66 +129,74 @@ fn emit_params(&mut self, info: &ModuleInfo) {
             self.emit_param("parmtype", &param_name_str, &param_type_str);
             self.emit_param("parm", &param_name_str, &param.description.value());
 
-            write!(
-                self.param_buffer,
-                "
-                pub(crate) static {param_name_str}:
-                    ::kernel::module_param::ModuleParamAccess<{param_type_str}> =
-                        ::kernel::module_param::ModuleParamAccess::new({param_default});
-
-                const _: () = {{
-                    #[link_section = \"__param\"]
-                    #[used]
-                    static __{module_name}_{param_name_str}_struct:
+            let static_name = format_ident!("__{}_{}_struct", self.module, param.name);
+            let param_name_cstr =
+                CString::new(param_name_str).expect("name contains NUL-terminator");
+            let param_name_cstr_with_module =
+                CString::new(format!("{}.{}", self.module, param.name))
+                    .expect("name contains NUL-terminator");
+
+            let param_name = &param.name;
+            let param_type = &param.ptype;
+            let param_default = &param.default;
+
+            self.param_ts.extend(quote! {
+                #[allow(non_upper_case_globals)]
+                pub(crate) static #param_name:
+                    ::kernel::module_param::ModuleParamAccess<#param_type> =
+                        ::kernel::module_param::ModuleParamAccess::new(#param_default);
+
+                const _: () = {
+                    #[allow(non_upper_case_globals)]
+                    #[link_section = "__param"]
+                    #[used(compiler)]
+                    static #static_name:
                         ::kernel::module_param::KernelParam =
                         ::kernel::module_param::KernelParam::new(
-                            ::kernel::bindings::kernel_param {{
-                                name: if ::core::cfg!(MODULE) {{
-                                    ::kernel::c_str!(\"{param_name_str}\").to_bytes_with_nul()
-                                }} else {{
-                                    ::kernel::c_str!(\"{module_name}.{param_name_str}\")
-                                        .to_bytes_with_nul()
-                                }}.as_ptr(),
+                            ::kernel::bindings::kernel_param {
+                                name: kernel::str::as_char_ptr_in_const_context(
+                                    if ::core::cfg!(MODULE) {
+                                        #param_name_cstr
+                                    } else {
+                                        #param_name_cstr_with_module
+                                    }
+                                ),
                                 // SAFETY: `__this_module` is constructed by the kernel at load
                                 // time and will not be freed until the module is unloaded.
                                 #[cfg(MODULE)]
-                                mod_: unsafe {{
+                                mod_: unsafe {
                                     core::ptr::from_ref(&::kernel::bindings::__this_module)
                                         .cast_mut()
-                                }},
+                                },
                                 #[cfg(not(MODULE))]
                                 mod_: ::core::ptr::null_mut(),
-                                ops: core::ptr::from_ref(&{ops}),
+                                ops: core::ptr::from_ref(&#ops),
                                 perm: 0, // Will not appear in sysfs
                                 level: -1,
                                 flags: 0,
-                                __bindgen_anon_1: ::kernel::bindings::kernel_param__bindgen_ty_1 {{
-                                    arg: {param_name_str}.as_void_ptr()
-                                }},
-                            }}
+                                __bindgen_anon_1: ::kernel::bindings::kernel_param__bindgen_ty_1 {
+                                    arg: #param_name.as_void_ptr()
+                                },
+                            }
                         );
-                }};
-                ",
-                module_name = info.name.value(),
-                ops = ops,
-            )
-            .unwrap();
+                };
+            });
         }
     }
 }
 
-fn param_ops_path(param_type: &str) -> &'static str {
+fn param_ops_path(param_type: &str) -> Path {
     match param_type {
-        "i8" => "::kernel::module_param::PARAM_OPS_I8",
-        "u8" => "::kernel::module_param::PARAM_OPS_U8",
-        "i16" => "::kernel::module_param::PARAM_OPS_I16",
-        "u16" => "::kernel::module_param::PARAM_OPS_U16",
-        "i32" => "::kernel::module_param::PARAM_OPS_I32",
-        "u32" => "::kernel::module_param::PARAM_OPS_U32",
-        "i64" => "::kernel::module_param::PARAM_OPS_I64",
-        "u64" => "::kernel::module_param::PARAM_OPS_U64",
-        "isize" => "::kernel::module_param::PARAM_OPS_ISIZE",
-        "usize" => "::kernel::module_param::PARAM_OPS_USIZE",
+        "i8" => parse_quote!(::kernel::module_param::PARAM_OPS_I8),
+        "u8" => parse_quote!(::kernel::module_param::PARAM_OPS_U8),
+        "i16" => parse_quote!(::kernel::module_param::PARAM_OPS_I16),
+        "u16" => parse_quote!(::kernel::module_param::PARAM_OPS_U16),
+        "i32" => parse_quote!(::kernel::module_param::PARAM_OPS_I32),
+        "u32" => parse_quote!(::kernel::module_param::PARAM_OPS_U32),
+        "i64" => parse_quote!(::kernel::module_param::PARAM_OPS_I64),
+        "u64" => parse_quote!(::kernel::module_param::PARAM_OPS_U64),
+        "isize" => parse_quote!(::kernel::module_param::PARAM_OPS_ISIZE),
+        "usize" => parse_quote!(::kernel::module_param::PARAM_OPS_USIZE),
         t => panic!("Unsupported parameter type {}", t),
     }
 }
@@ -420,29 +431,41 @@ fn parse(input: ParseStream<'_>) -> Result<Self> {
 }
 
 pub(crate) fn module(info: ModuleInfo) -> Result<TokenStream> {
+    let ModuleInfo {
+        type_,
+        license,
+        name,
+        authors,
+        description,
+        alias,
+        firmware,
+        imports_ns,
+        params: _,
+    } = &info;
+
     // Rust does not allow hyphens in identifiers, use underscore instead.
-    let ident = info.name.value().replace('-', "_");
+    let ident = name.value().replace('-', "_");
     let mut modinfo = ModInfoBuilder::new(ident.as_ref());
-    if let Some(authors) = &info.authors {
+    if let Some(authors) = authors {
         for author in authors {
             modinfo.emit("author", &author.value());
         }
     }
-    if let Some(description) = &info.description {
+    if let Some(description) = description {
         modinfo.emit("description", &description.value());
     }
-    modinfo.emit("license", &info.license.value());
-    if let Some(aliases) = &info.alias {
+    modinfo.emit("license", &license.value());
+    if let Some(aliases) = alias {
         for alias in aliases {
             modinfo.emit("alias", &alias.value());
         }
     }
-    if let Some(firmware) = &info.firmware {
+    if let Some(firmware) = firmware {
         for fw in firmware {
             modinfo.emit("firmware", &fw.value());
         }
     }
-    if let Some(imports) = &info.imports_ns {
+    if let Some(imports) = imports_ns {
         for ns in imports {
             modinfo.emit("import_ns", &ns.value());
         }
@@ -455,182 +478,188 @@ pub(crate) fn module(info: ModuleInfo) -> Result<TokenStream> {
 
     modinfo.emit_params(&info);
 
-    Ok(format!(
-        "
-            /// The module name.
-            ///
-            /// Used by the printing macros, e.g. [`info!`].
-            const __LOG_PREFIX: &[u8] = b\"{name}\\0\";
-
-            // SAFETY: `__this_module` is constructed by the kernel at load time and will not be
-            // freed until the module is unloaded.
-            #[cfg(MODULE)]
-            static THIS_MODULE: ::kernel::ThisModule = unsafe {{
-                extern \"C\" {{
-                    static __this_module: ::kernel::types::Opaque<::kernel::bindings::module>;
-                }}
-
-                ::kernel::ThisModule::from_ptr(__this_module.get())
-            }};
-            #[cfg(not(MODULE))]
-            static THIS_MODULE: ::kernel::ThisModule = unsafe {{
-                ::kernel::ThisModule::from_ptr(::core::ptr::null_mut())
-            }};
-
-            /// The `LocalModule` type is the type of the module created by `module!`,
-            /// `module_pci_driver!`, `module_platform_driver!`, etc.
-            type LocalModule = {type_};
-
-            impl ::kernel::ModuleMetadata for {type_} {{
-                const NAME: &'static ::kernel::str::CStr = c\"{name}\";
-            }}
-
-            // Double nested modules, since then nobody can access the public items inside.
-            mod __module_init {{
-                mod __module_init {{
-                    use super::super::{type_};
-                    use pin_init::PinInit;
-
-                    /// The \"Rust loadable module\" mark.
-                    //
-                    // This may be best done another way later on, e.g. as a new modinfo
-                    // key or a new section. For the moment, keep it simple.
-                    #[cfg(MODULE)]
-                    #[doc(hidden)]
-                    #[used(compiler)]
-                    static __IS_RUST_MODULE: () = ();
-
-                    static mut __MOD: ::core::mem::MaybeUninit<{type_}> =
-                        ::core::mem::MaybeUninit::uninit();
-
-                    // Loadable modules need to export the `{{init,cleanup}}_module` identifiers.
-                    /// # Safety
-                    ///
-                    /// This function must not be called after module initialization, because it may be
-                    /// freed after that completes.
-                    #[cfg(MODULE)]
-                    #[doc(hidden)]
-                    #[no_mangle]
-                    #[link_section = \".init.text\"]
-                    pub unsafe extern \"C\" fn init_module() -> ::kernel::ffi::c_int {{
-                        // SAFETY: This function is inaccessible to the outside due to the double
-                        // module wrapping it. It is called exactly once by the C side via its
-                        // unique name.
-                        unsafe {{ __init() }}
-                    }}
-
-                    #[cfg(MODULE)]
-                    #[doc(hidden)]
-                    #[used(compiler)]
-                    #[link_section = \".init.data\"]
-                    static __UNIQUE_ID___addressable_init_module: unsafe extern \"C\" fn() -> i32 = init_module;
-
-                    #[cfg(MODULE)]
-                    #[doc(hidden)]
-                    #[no_mangle]
-                    #[link_section = \".exit.text\"]
-                    pub extern \"C\" fn cleanup_module() {{
-                        // SAFETY:
-                        // - This function is inaccessible to the outside due to the double
-                        //   module wrapping it. It is called exactly once by the C side via its
-                        //   unique name,
-                        // - furthermore it is only called after `init_module` has returned `0`
-                        //   (which delegates to `__init`).
-                        unsafe {{ __exit() }}
-                    }}
-
-                    #[cfg(MODULE)]
-                    #[doc(hidden)]
-                    #[used(compiler)]
-                    #[link_section = \".exit.data\"]
-                    static __UNIQUE_ID___addressable_cleanup_module: extern \"C\" fn() = cleanup_module;
-
-                    // Built-in modules are initialized through an initcall pointer
-                    // and the identifiers need to be unique.
-                    #[cfg(not(MODULE))]
-                    #[cfg(not(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS))]
-                    #[doc(hidden)]
-                    #[link_section = \"{initcall_section}\"]
-                    #[used(compiler)]
-                    pub static __{ident}_initcall: extern \"C\" fn() ->
-                        ::kernel::ffi::c_int = __{ident}_init;
-
-                    #[cfg(not(MODULE))]
-                    #[cfg(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)]
-                    ::core::arch::global_asm!(
-                        r#\".section \"{initcall_section}\", \"a\"
-                        __{ident}_initcall:
-                            .long   __{ident}_init - .
-                            .previous
-                        \"#
-                    );
-
-                    #[cfg(not(MODULE))]
-                    #[doc(hidden)]
-                    #[no_mangle]
-                    pub extern \"C\" fn __{ident}_init() -> ::kernel::ffi::c_int {{
-                        // SAFETY: This function is inaccessible to the outside due to the double
-                        // module wrapping it. It is called exactly once by the C side via its
-                        // placement above in the initcall section.
-                        unsafe {{ __init() }}
-                    }}
-
-                    #[cfg(not(MODULE))]
-                    #[doc(hidden)]
-                    #[no_mangle]
-                    pub extern \"C\" fn __{ident}_exit() {{
-                        // SAFETY:
-                        // - This function is inaccessible to the outside due to the double
-                        //   module wrapping it. It is called exactly once by the C side via its
-                        //   unique name,
-                        // - furthermore it is only called after `__{ident}_init` has
-                        //   returned `0` (which delegates to `__init`).
-                        unsafe {{ __exit() }}
-                    }}
-
-                    /// # Safety
-                    ///
-                    /// This function must only be called once.
-                    unsafe fn __init() -> ::kernel::ffi::c_int {{
-                        let initer =
-                            <{type_} as ::kernel::InPlaceModule>::init(&super::super::THIS_MODULE);
-                        // SAFETY: No data race, since `__MOD` can only be accessed by this module
-                        // and there only `__init` and `__exit` access it. These functions are only
-                        // called once and `__exit` cannot be called before or during `__init`.
-                        match unsafe {{ initer.__pinned_init(__MOD.as_mut_ptr()) }} {{
-                            Ok(m) => 0,
-                            Err(e) => e.to_errno(),
-                        }}
-                    }}
-
-                    /// # Safety
-                    ///
-                    /// This function must
-                    /// - only be called once,
-                    /// - be called after `__init` has been called and returned `0`.
-                    unsafe fn __exit() {{
-                        // SAFETY: No data race, since `__MOD` can only be accessed by this module
-                        // and there only `__init` and `__exit` access it. These functions are only
-                        // called once and `__init` was already called.
-                        unsafe {{
-                            // Invokes `drop()` on `__MOD`, which should be used for cleanup.
-                            __MOD.assume_init_drop();
-                        }}
-                    }}
-                    {modinfo}
-                }}
-            }}
-            mod module_parameters {{
-                {params}
-            }}
-        ",
-        type_ = info.type_,
-        name = info.name.value(),
-        ident = ident,
-        modinfo = modinfo.buffer,
-        params = modinfo.param_buffer,
-        initcall_section = ".initcall6.init"
-    )
-    .parse()
-    .expect("Error parsing formatted string into token stream."))
+    let modinfo_ts = modinfo.ts;
+    let params_ts = modinfo.param_ts;
+
+    let ident_init = format_ident!("__{ident}_init");
+    let ident_exit = format_ident!("__{ident}_exit");
+    let ident_initcall = format_ident!("__{ident}_initcall");
+    let initcall_section = ".initcall6.init";
+
+    let global_asm = format!(
+        r#".section "{initcall_section}", "a"
+        __{ident}_initcall:
+            .long   __{ident}_init - .
+            .previous
+        "#
+    );
+
+    let name_cstr = CString::new(name.value()).expect("name contains NUL-terminator");
+
+    Ok(quote! {
+        /// The module name.
+        ///
+        /// Used by the printing macros, e.g. [`info!`].
+        const __LOG_PREFIX: &[u8] = #name_cstr.to_bytes_with_nul();
+
+        // SAFETY: `__this_module` is constructed by the kernel at load time and will not be
+        // freed until the module is unloaded.
+        #[cfg(MODULE)]
+        static THIS_MODULE: ::kernel::ThisModule = unsafe {
+            extern "C" {
+                static __this_module: ::kernel::types::Opaque<::kernel::bindings::module>;
+            };
+
+            ::kernel::ThisModule::from_ptr(__this_module.get())
+        };
+
+        #[cfg(not(MODULE))]
+        static THIS_MODULE: ::kernel::ThisModule = unsafe {
+            ::kernel::ThisModule::from_ptr(::core::ptr::null_mut())
+        };
+
+        /// The `LocalModule` type is the type of the module created by `module!`,
+        /// `module_pci_driver!`, `module_platform_driver!`, etc.
+        type LocalModule = #type_;
+
+        impl ::kernel::ModuleMetadata for #type_ {
+            const NAME: &'static ::kernel::str::CStr = #name_cstr;
+        }
+
+        // Double nested modules, since then nobody can access the public items inside.
+        mod __module_init {
+            mod __module_init {
+                use super::super::#type_;
+                use pin_init::PinInit;
+
+                /// The "Rust loadable module" mark.
+                //
+                // This may be best done another way later on, e.g. as a new modinfo
+                // key or a new section. For the moment, keep it simple.
+                #[cfg(MODULE)]
+                #[doc(hidden)]
+                #[used(compiler)]
+                static __IS_RUST_MODULE: () = ();
+
+                static mut __MOD: ::core::mem::MaybeUninit<#type_> =
+                    ::core::mem::MaybeUninit::uninit();
+
+                // Loadable modules need to export the `{init,cleanup}_module` identifiers.
+                /// # Safety
+                ///
+                /// This function must not be called after module initialization, because it may be
+                /// freed after that completes.
+                #[cfg(MODULE)]
+                #[doc(hidden)]
+                #[no_mangle]
+                #[link_section = ".init.text"]
+                pub unsafe extern "C" fn init_module() -> ::kernel::ffi::c_int {
+                    // SAFETY: This function is inaccessible to the outside due to the double
+                    // module wrapping it. It is called exactly once by the C side via its
+                    // unique name.
+                    unsafe { __init() }
+                }
+
+                #[cfg(MODULE)]
+                #[doc(hidden)]
+                #[used(compiler)]
+                #[link_section = ".init.data"]
+                static __UNIQUE_ID___addressable_init_module: unsafe extern "C" fn() -> i32 =
+                    init_module;
+
+                #[cfg(MODULE)]
+                #[doc(hidden)]
+                #[no_mangle]
+                #[link_section = ".exit.text"]
+                pub extern "C" fn cleanup_module() {
+                    // SAFETY:
+                    // - This function is inaccessible to the outside due to the double
+                    //   module wrapping it. It is called exactly once by the C side via its
+                    //   unique name,
+                    // - furthermore it is only called after `init_module` has returned `0`
+                    //   (which delegates to `__init`).
+                    unsafe { __exit() }
+                }
+
+                #[cfg(MODULE)]
+                #[doc(hidden)]
+                #[used(compiler)]
+                #[link_section = ".exit.data"]
+                static __UNIQUE_ID___addressable_cleanup_module: extern "C" fn() = cleanup_module;
+
+                // Built-in modules are initialized through an initcall pointer
+                // and the identifiers need to be unique.
+                #[cfg(not(MODULE))]
+                #[cfg(not(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS))]
+                #[doc(hidden)]
+                #[link_section = #initcall_section]
+                #[used(compiler)]
+                pub static #ident_initcall: extern "C" fn() ->
+                    ::kernel::ffi::c_int = #ident_init;
+
+                #[cfg(not(MODULE))]
+                #[cfg(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)]
+                ::core::arch::global_asm!(#global_asm);
+
+                #[cfg(not(MODULE))]
+                #[doc(hidden)]
+                #[no_mangle]
+                pub extern "C" fn #ident_init() -> ::kernel::ffi::c_int {
+                    // SAFETY: This function is inaccessible to the outside due to the double
+                    // module wrapping it. It is called exactly once by the C side via its
+                    // placement above in the initcall section.
+                    unsafe { __init() }
+                }
+
+                #[cfg(not(MODULE))]
+                #[doc(hidden)]
+                #[no_mangle]
+                pub extern "C" fn #ident_exit() {
+                    // SAFETY:
+                    // - This function is inaccessible to the outside due to the double
+                    //   module wrapping it. It is called exactly once by the C side via its
+                    //   unique name,
+                    // - furthermore it is only called after `#ident_init` has
+                    //   returned `0` (which delegates to `__init`).
+                    unsafe { __exit() }
+                }
+
+                /// # Safety
+                ///
+                /// This function must only be called once.
+                unsafe fn __init() -> ::kernel::ffi::c_int {
+                    let initer =
+                        <#type_ as ::kernel::InPlaceModule>::init(&super::super::THIS_MODULE);
+                    // SAFETY: No data race, since `__MOD` can only be accessed by this module
+                    // and there only `__init` and `__exit` access it. These functions are only
+                    // called once and `__exit` cannot be called before or during `__init`.
+                    match unsafe { initer.__pinned_init(__MOD.as_mut_ptr()) } {
+                        Ok(m) => 0,
+                        Err(e) => e.to_errno(),
+                    }
+                }
+
+                /// # Safety
+                ///
+                /// This function must
+                /// - only be called once,
+                /// - be called after `__init` has been called and returned `0`.
+                unsafe fn __exit() {
+                    // SAFETY: No data race, since `__MOD` can only be accessed by this module
+                    // and there only `__init` and `__exit` access it. These functions are only
+                    // called once and `__init` was already called.
+                    unsafe {
+                        // Invokes `drop()` on `__MOD`, which should be used for cleanup.
+                        __MOD.assume_init_drop();
+                    }
+                }
+
+                #modinfo_ts
+            }
+        }
+
+        mod module_parameters {
+            #params_ts
+        }
+    })
 }
Signature · 1 line
2.51.2
author
#9 lore raw

PATCH v3 6/12 rust: macros: convert `#[export]` to use `syn`

In reply to Gary Guo

From: Gary Guo <[email protected]>

This eliminates the custom function_name helper.

---
 rust/macros/export.rs  | 24 +++++++++---------------
 rust/macros/helpers.rs | 18 ------------------
 rust/macros/lib.rs     |  5 +++--
 3 files changed, 12 insertions(+), 35 deletions(-)
rust/macros/export.rs +9 -15
index 92d9b30971929..6d53521f62fc7 100644
@@ -3,19 +3,14 @@
 use proc_macro2::TokenStream;
 use quote::quote;
 
-use crate::helpers::function_name;
-
 /// Please see [`crate::export`] for documentation.
-pub(crate) fn export(_attr: TokenStream, ts: TokenStream) -> TokenStream {
-    let Some(name) = function_name(ts.clone()) else {
-        return "::core::compile_error!(\"The #[export] attribute must be used on a function.\");"
-            .parse::<TokenStream>()
-            .unwrap();
-    };
+pub(crate) fn export(f: syn::ItemFn) -> TokenStream {
+    let name = &f.sig.ident;
 
-    // This verifies that the function has the same signature as the declaration generated by
-    // bindgen. It makes use of the fact that all branches of an if/else must have the same type.
-    let signature_check = quote!(
+    quote! {
+        // This verifies that the function has the same signature as the declaration generated by
+        // bindgen. It makes use of the fact that all branches of an if/else must have the same
+        // type.
         const _: () = {
             if true {
                 ::kernel::bindings::#name
@@ -23,9 +18,8 @@ pub(crate) fn export(_attr: TokenStream, ts: TokenStream) -> TokenStream {
                 #name
             };
         };
-    );
-
-    let no_mangle = quote!(#[no_mangle]);
 
-    TokenStream::from_iter([signature_check, no_mangle, ts])
+        #[no_mangle]
+        #f
+    }
 }
rust/macros/helpers.rs +0 -18
index fa66ef6eb0f3d..754c827cc21e1 100644
@@ -2,7 +2,6 @@
 
 use proc_macro2::{
     token_stream,
-    Ident,
     TokenStream,
     TokenTree, //
 };
@@ -50,23 +49,6 @@ pub(crate) fn value(&self) -> String {
     }
 }
 
-/// Given a function declaration, finds the name of the function.
-pub(crate) fn function_name(input: TokenStream) -> Option<Ident> {
-    let mut input = input.into_iter();
-    while let Some(token) = input.next() {
-        match token {
-            TokenTree::Ident(i) if i == "fn" => {
-                if let Some(TokenTree::Ident(i)) = input.next() {
-                    return Some(i);
-                }
-                return None;
-            }
-            _ => continue,
-        }
-    }
-    None
-}
-
 pub(crate) fn file() -> String {
     #[cfg(not(CONFIG_RUSTC_HAS_SPAN_FILE))]
     {
rust/macros/lib.rs +3 -2
index c5347127a3a51..da8239d2474b6 100644
@@ -234,8 +234,9 @@ pub fn vtable(attr: TokenStream, input: TokenStream) -> TokenStream {
 /// This macro is *not* the same as the C macros `EXPORT_SYMBOL_*`. All Rust symbols are currently
 /// automatically exported with `EXPORT_SYMBOL_GPL`.
 #[proc_macro_attribute]
-pub fn export(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    export::export(attr.into(), ts.into()).into()
+pub fn export(attr: TokenStream, input: TokenStream) -> TokenStream {
+    parse_macro_input!(attr as syn::parse::Nothing);
+    export::export(parse_macro_input!(input)).into()
 }
 
 /// Like [`core::format_args!`], but automatically wraps arguments in [`kernel::fmt::Adapter`].
Signature · 1 line
2.51.2
author
#10 lore raw

PATCH v3 7/12 rust: macros: convert `concat_idents!` to use `syn`

In reply to Gary Guo

From: Gary Guo <[email protected]>

This eliminates the need for expect_punct helper.

---
 rust/macros/concat_idents.rs | 39 ++++++++++++++++++++++++------------
 rust/macros/helpers.rs       | 14 +------------
 rust/macros/lib.rs           |  4 ++--
 3 files changed, 29 insertions(+), 28 deletions(-)
rust/macros/concat_idents.rs +26 -13
index 12cb231c3d715..47b6add378d2c 100644
@@ -1,23 +1,36 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro2::{token_stream, Ident, TokenStream, TokenTree};
+use proc_macro2::{
+    Ident,
+    TokenStream,
+    TokenTree, //
+};
+use syn::{
+    parse::{
+        Parse,
+        ParseStream, //
+    },
+    Result,
+    Token, //
+};
 
-use crate::helpers::expect_punct;
+pub(crate) struct Input {
+    a: Ident,
+    _comma: Token![,],
+    b: Ident,
+}
 
-fn expect_ident(it: &mut token_stream::IntoIter) -> Ident {
-    if let Some(TokenTree::Ident(ident)) = it.next() {
-        ident
-    } else {
-        panic!("Expected Ident")
+impl Parse for Input {
+    fn parse(input: ParseStream<'_>) -> Result<Self> {
+        Ok(Self {
+            a: input.parse()?,
+            _comma: input.parse()?,
+            b: input.parse()?,
+        })
     }
 }
 
-pub(crate) fn concat_idents(ts: TokenStream) -> TokenStream {
-    let mut it = ts.into_iter();
-    let a = expect_ident(&mut it);
-    assert_eq!(expect_punct(&mut it), ',');
-    let b = expect_ident(&mut it);
-    assert!(it.next().is_none(), "only two idents can be concatenated");
+pub(crate) fn concat_idents(Input { a, b, .. }: Input) -> TokenStream {
     let res = Ident::new(&format!("{a}{b}"), b.span());
     TokenStream::from_iter([TokenTree::Ident(res)])
 }
rust/macros/helpers.rs +1 -13
index 754c827cc21e1..adfa60d8f42d8 100644
@@ -1,10 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro2::{
-    token_stream,
-    TokenStream,
-    TokenTree, //
-};
+use proc_macro2::TokenStream;
 use quote::ToTokens;
 use syn::{
     parse::{
@@ -16,14 +12,6 @@
     Result, //
 };
 
-pub(crate) fn expect_punct(it: &mut token_stream::IntoIter) -> char {
-    if let TokenTree::Punct(punct) = it.next().expect("Reached end of token stream for Punct") {
-        punct.as_char()
-    } else {
-        panic!("Expected Punct");
-    }
-}
-
 /// A string literal that is required to have ASCII value only.
 pub(crate) struct AsciiLitStr(LitStr);
 
rust/macros/lib.rs +2 -2
index da8239d2474b6..12467bfc703a8 100644
@@ -311,8 +311,8 @@ pub fn fmt(input: TokenStream) -> TokenStream {
 /// assert_eq!(BR_OK, binder_driver_return_protocol_BR_OK);
 /// ```
 #[proc_macro]
-pub fn concat_idents(ts: TokenStream) -> TokenStream {
-    concat_idents::concat_idents(ts.into()).into()
+pub fn concat_idents(input: TokenStream) -> TokenStream {
+    concat_idents::concat_idents(parse_macro_input!(input)).into()
 }
 
 /// Paste identifiers together.
Signature · 1 line
2.51.2
author
#11 lore raw

PATCH v3 8/12 rust: macros: convert `#[kunit_tests]` macro to use `syn`

In reply to Gary Guo

From: Gary Guo <[email protected]>

Make use of syn to parse the module structurally and thus improve the robustness of parsing.

String interpolation is avoided by generating tokens directly using quote!.

---
 rust/macros/kunit.rs | 274 +++++++++++++++++++------------------------
 rust/macros/lib.rs   |   6 +-
 2 files changed, 123 insertions(+), 157 deletions(-)
rust/macros/kunit.rs +119 -155
index 5cd6aa5eef07d..afbc708cbdc50 100644
@@ -4,81 +4,50 @@
 //!
 //! Copyright (c) 2023 José Expósito <[email protected]>
 
-use std::collections::HashMap;
-use std::fmt::Write;
-
-use proc_macro2::{Delimiter, Group, TokenStream, TokenTree};
-
-pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    let attr = attr.to_string();
-
-    if attr.is_empty() {
-        panic!("Missing test name in `#[kunit_tests(test_name)]` macro")
-    }
-
-    if attr.len() > 255 {
-        panic!("The test suite name `{attr}` exceeds the maximum length of 255 bytes")
+use std::ffi::CString;
+
+use proc_macro2::TokenStream;
+use quote::{
+    format_ident,
+    quote,
+    ToTokens, //
+};
+use syn::{
+    parse_quote,
+    Error,
+    Ident,
+    Item,
+    ItemMod,
+    LitCStr,
+    Result, //
+};
+
+pub(crate) fn kunit_tests(test_suite: Ident, mut module: ItemMod) -> Result<TokenStream> {
+    if test_suite.to_string().len() > 255 {
+        return Err(Error::new_spanned(
+            test_suite,
+            "test suite names cannot exceed the maximum length of 255 bytes",
+        ));
     }
 
-    let mut tokens: Vec<_> = ts.into_iter().collect();
-
-    // Scan for the `mod` keyword.
-    tokens
-        .iter()
-        .find_map(|token| match token {
-            TokenTree::Ident(ident) => match ident.to_string().as_str() {
-                "mod" => Some(true),
-                _ => None,
-            },
-            _ => None,
-        })
-        .expect("`#[kunit_tests(test_name)]` attribute should only be applied to modules");
-
-    // Retrieve the main body. The main body should be the last token tree.
-    let body = match tokens.pop() {
-        Some(TokenTree::Group(group)) if group.delimiter() == Delimiter::Brace => group,
-        _ => panic!("Cannot locate main body of module"),
+    // We cannot handle modules that defer to another file (e.g. `mod foo;`).
+    let Some((module_brace, module_items)) = module.content.take() else {
+        Err(Error::new_spanned(
+            module,
+            "`#[kunit_tests(test_name)]` attribute should only be applied to inline modules",
+        ))?
     };
 
-    // Get the functions set as tests. Search for `[test]` -> `fn`.
-    let mut body_it = body.stream().into_iter();
-    let mut tests = Vec::new();
-    let mut attributes: HashMap<String, TokenStream> = HashMap::new();
-    while let Some(token) = body_it.next() {
-        match token {
-            TokenTree::Punct(ref p) if p.as_char() == '#' => match body_it.next() {
-                Some(TokenTree::Group(g)) if g.delimiter() == Delimiter::Bracket => {
-                    if let Some(TokenTree::Ident(name)) = g.stream().into_iter().next() {
-                        // Collect attributes because we need to find which are tests. We also
-                        // need to copy `cfg` attributes so tests can be conditionally enabled.
-                        attributes
-                            .entry(name.to_string())
-                            .or_default()
-                            .extend([token, TokenTree::Group(g)]);
-                    }
-                    continue;
-                }
-                _ => (),
-            },
-            TokenTree::Ident(i) if i == "fn" && attributes.contains_key("test") => {
-                if let Some(TokenTree::Ident(test_name)) = body_it.next() {
-                    tests.push((test_name, attributes.remove("cfg").unwrap_or_default()))
-                }
-            }
-
-            _ => (),
-        }
-        attributes.clear();
-    }
+    // Make the entire module gated behind `CONFIG_KUNIT`.
+    module
+        .attrs
+        .insert(0, parse_quote!(#[cfg(CONFIG_KUNIT="y")]));
 
-    // Add `#[cfg(CONFIG_KUNIT="y")]` before the module declaration.
-    let config_kunit = "#[cfg(CONFIG_KUNIT=\"y\")]".to_owned().parse().unwrap();
-    tokens.insert(
-        0,
-        TokenTree::Group(Group::new(Delimiter::None, config_kunit)),
-    );
+    let mut processed_items = Vec::new();
+    let mut test_cases = Vec::new();
 
     // Generate the test KUnit test suite and a test case for each `#[test]`.
+    //
     // The code generated for the following test module:
     //
     // ```
@@ -110,98 +79,93 @@ pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
     //
     // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES);
     // ```
-    let mut kunit_macros = "".to_owned();
-    let mut test_cases = "".to_owned();
-    let mut assert_macros = "".to_owned();
-    let path = crate::helpers::file();
-    let num_tests = tests.len();
-    for (test, cfg_attr) in tests {
-        let kunit_wrapper_fn_name = format!("kunit_rust_wrapper_{test}");
-        // Append any `cfg` attributes the user might have written on their tests so we don't
-        // attempt to call them when they are `cfg`'d out. An extra `use` is used here to reduce
-        // the length of the assert message.
-        let kunit_wrapper = format!(
-            r#"unsafe extern "C" fn {kunit_wrapper_fn_name}(_test: *mut ::kernel::bindings::kunit)
-            {{
-                (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED;
-                {cfg_attr} {{
-                    (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS;
-                    use ::kernel::kunit::is_test_result_ok;
-                    assert!(is_test_result_ok({test}()));
+    //
+    // Non-function items (e.g. imports) are preserved.
+    for item in module_items {
+        let Item::Fn(mut f) = item else {
+            processed_items.push(item);
+            continue;
+        };
+
+        // TODO: Replace below with `extract_if` when MSRV is bumped above 1.85.
+        let before_len = f.attrs.len();
+        f.attrs.retain(|attr| !attr.path().is_ident("test"));
+        if f.attrs.len() == before_len {
+            processed_items.push(Item::Fn(f));
+            continue;
+        }
+
+        let test = f.sig.ident.clone();
+
+        // Retrieve `#[cfg]` applied on the function which needs to be present on derived items too.
+        let cfg_attrs: Vec<_> = f
+            .attrs
+            .iter()
+            .filter(|attr| attr.path().is_ident("cfg"))
+            .cloned()
+            .collect();
+
+        // Before the test, override usual `assert!` and `assert_eq!` macros with ones that call
+        // KUnit instead.
+        let test_str = test.to_string();
+        let path = crate::helpers::file();
+        processed_items.push(parse_quote! {
+            #[allow(unused)]
+            macro_rules! assert {
+                ($cond:expr $(,)?) => {{
+                    kernel::kunit_assert!(#test_str, #path, 0, $cond);
+                }}
+            }
+        });
+        processed_items.push(parse_quote! {
+            #[allow(unused)]
+            macro_rules! assert_eq {
+                ($left:expr, $right:expr $(,)?) => {{
+                    kernel::kunit_assert_eq!(#test_str, #path, 0, $left, $right);
                 }}
-            }}"#,
+            }
+        });
+
+        // Add back the test item.
+        processed_items.push(Item::Fn(f));
+
+        let kunit_wrapper_fn_name = format_ident!("kunit_rust_wrapper_{test}");
+        let test_cstr = LitCStr::new(
+            &CString::new(test_str.as_str()).expect("identifier cannot contain NUL"),
+            test.span(),
         );
-        writeln!(kunit_macros, "{kunit_wrapper}").unwrap();
-        writeln!(
-            test_cases,
-            "    ::kernel::kunit::kunit_case(::kernel::c_str!(\"{test}\"), {kunit_wrapper_fn_name}),"
-        )
-        .unwrap();
-        writeln!(
-            assert_macros,
-            r#"
-/// Overrides the usual [`assert!`] macro with one that calls KUnit instead.
-#[allow(unused)]
-macro_rules! assert {{
-    ($cond:expr $(,)?) => {{{{
-        kernel::kunit_assert!("{test}", "{path}", 0, $cond);
-    }}}}
-}}
-
-/// Overrides the usual [`assert_eq!`] macro with one that calls KUnit instead.
-#[allow(unused)]
-macro_rules! assert_eq {{
-    ($left:expr, $right:expr $(,)?) => {{{{
-        kernel::kunit_assert_eq!("{test}", "{path}", 0, $left, $right);
-    }}}}
-}}
-        "#
-        )
-        .unwrap();
-    }
+        processed_items.push(parse_quote! {
+            unsafe extern "C" fn #kunit_wrapper_fn_name(_test: *mut ::kernel::bindings::kunit) {
+                (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED;
 
-    writeln!(kunit_macros).unwrap();
-    writeln!(
-        kunit_macros,
-        "static mut TEST_CASES: [::kernel::bindings::kunit_case; {}] = [\n{test_cases}    ::kernel::kunit::kunit_case_null(),\n];",
-        num_tests + 1
-    )
-    .unwrap();
-
-    writeln!(
-        kunit_macros,
-        "::kernel::kunit_unsafe_test_suite!({attr}, TEST_CASES);"
-    )
-    .unwrap();
-
-    // Remove the `#[test]` macros.
-    // We do this at a token level, in order to preserve span information.
-    let mut new_body = vec![];
-    let mut body_it = body.stream().into_iter();
-
-    while let Some(token) = body_it.next() {
-        match token {
-            TokenTree::Punct(ref c) if c.as_char() == '#' => match body_it.next() {
-                Some(TokenTree::Group(group)) if group.to_string() == "[test]" => (),
-                Some(next) => {
-                    new_body.extend([token, next]);
-                }
-                _ => {
-                    new_body.push(token);
+                // Append any `cfg` attributes the user might have written on their tests so we
+                // don't attempt to call them when they are `cfg`'d out. An extra `use` is used
+                // here to reduce the length of the assert message.
+                #(#cfg_attrs)*
+                {
+                    (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS;
+                    use ::kernel::kunit::is_test_result_ok;
+                    assert!(is_test_result_ok(#test()));
                 }
-            },
-            _ => {
-                new_body.push(token);
             }
-        }
-    }
+        });
 
-    let mut final_body = TokenStream::new();
-    final_body.extend::<TokenStream>(assert_macros.parse().unwrap());
-    final_body.extend(new_body);
-    final_body.extend::<TokenStream>(kunit_macros.parse().unwrap());
-
-    tokens.push(TokenTree::Group(Group::new(Delimiter::Brace, final_body)));
+        test_cases.push(quote!(
+            ::kernel::kunit::kunit_case(#test_cstr, #kunit_wrapper_fn_name)
+        ));
+    }
 
-    tokens.into_iter().collect()
+    let num_tests_plus_1 = test_cases.len() + 1;
+    processed_items.push(parse_quote! {
+        static mut TEST_CASES: [::kernel::bindings::kunit_case; #num_tests_plus_1] = [
+            #(#test_cases,)*
+            ::kernel::kunit::kunit_case_null(),
+        ];
+    });
+    processed_items.push(parse_quote! {
+        ::kernel::kunit_unsafe_test_suite!(#test_suite, TEST_CASES);
+    });
+
+    module.content = Some((module_brace, processed_items));
+    Ok(module.to_token_stream())
 }
rust/macros/lib.rs +4 -2
index 12467bfc703a8..75ac60abe6ffa 100644
@@ -481,6 +481,8 @@ pub fn paste(input: TokenStream) -> TokenStream {
 /// }
 /// ```
 #[proc_macro_attribute]
-pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    kunit::kunit_tests(attr.into(), ts.into()).into()
+pub fn kunit_tests(attr: TokenStream, input: TokenStream) -> TokenStream {
+    kunit::kunit_tests(parse_macro_input!(attr), parse_macro_input!(input))
+        .unwrap_or_else(|e| e.into_compile_error())
+        .into()
 }
Signature · 1 line
2.51.2
author
#14 lore raw

PATCH v3 9/12 rust: macros: allow arbitrary types to be used in `module!` macro

In reply to Gary Guo

From: Gary Guo <[email protected]>

Previously this only accepts an identifier, but now with syn it is easy to make it accepts any type.

---
 rust/macros/module.rs | 13 +++++++------
 1 file changed, 7 insertions(+), 6 deletions(-)
rust/macros/module.rs +7 -6
index 43ada49525c9d..9e0242d42d51f 100644
@@ -26,7 +26,8 @@
     LitStr,
     Path,
     Result,
-    Token, //
+    Token,
+    Type, //
 };
 
 use crate::helpers::*;
@@ -370,7 +371,7 @@ fn parse(input: ParseStream<'_>) -> Result<Self> {
 }
 
 pub(crate) struct ModuleInfo {
-    type_: Ident,
+    type_: Type,
     license: AsciiLitStr,
     name: AsciiLitStr,
     authors: Option<Punctuated<AsciiLitStr, Token![,]>>,
@@ -529,7 +530,6 @@ impl ::kernel::ModuleMetadata for #type_ {
         // Double nested modules, since then nobody can access the public items inside.
         mod __module_init {
             mod __module_init {
-                use super::super::#type_;
                 use pin_init::PinInit;
 
                 /// The "Rust loadable module" mark.
@@ -541,7 +541,7 @@ mod __module_init {
                 #[used(compiler)]
                 static __IS_RUST_MODULE: () = ();
 
-                static mut __MOD: ::core::mem::MaybeUninit<#type_> =
+                static mut __MOD: ::core::mem::MaybeUninit<super::super::LocalModule> =
                     ::core::mem::MaybeUninit::uninit();
 
                 // Loadable modules need to export the `{init,cleanup}_module` identifiers.
@@ -628,8 +628,9 @@ pub extern "C" fn #ident_exit() {
                 ///
                 /// This function must only be called once.
                 unsafe fn __init() -> ::kernel::ffi::c_int {
-                    let initer =
-                        <#type_ as ::kernel::InPlaceModule>::init(&super::super::THIS_MODULE);
+                    let initer = <super::super::LocalModule as ::kernel::InPlaceModule>::init(
+                        &super::super::THIS_MODULE
+                    );
                     // SAFETY: No data race, since `__MOD` can only be accessed by this module
                     // and there only `__init` and `__exit` access it. These functions are only
                     // called once and `__exit` cannot be called before or during `__init`.
Signature · 1 line
2.51.2
author
#15 lore raw

PATCH v3 10/12 rust: macros: rearrange `#[doc(hidden)]` in `module!` macro

In reply to Gary Guo

From: Gary Guo <[email protected]>

This #[doc(hidden)] can just be applied on a module to hide anything within.

---
 rust/macros/module.rs | 10 +---------
 1 file changed, 1 insertion(+), 9 deletions(-)
rust/macros/module.rs +1 -9
index 9e0242d42d51f..e16298e520c78 100644
@@ -77,7 +77,6 @@ fn emit_base(&mut self, field: &str, content: &str, builtin: bool, param: bool)
         );
         let item = quote! {
             #cfg
-            #[doc(hidden)]
             #[cfg_attr(not(target_os = "macos"), link_section = ".modinfo")]
             #[used(compiler)]
             pub static #counter: [u8; #length] = *#string;
@@ -528,6 +527,7 @@ impl ::kernel::ModuleMetadata for #type_ {
         }
 
         // Double nested modules, since then nobody can access the public items inside.
+        #[doc(hidden)]
         mod __module_init {
             mod __module_init {
                 use pin_init::PinInit;
@@ -537,7 +537,6 @@ mod __module_init {
                 // This may be best done another way later on, e.g. as a new modinfo
                 // key or a new section. For the moment, keep it simple.
                 #[cfg(MODULE)]
-                #[doc(hidden)]
                 #[used(compiler)]
                 static __IS_RUST_MODULE: () = ();
 
@@ -550,7 +549,6 @@ mod __module_init {
                 /// This function must not be called after module initialization, because it may be
                 /// freed after that completes.
                 #[cfg(MODULE)]
-                #[doc(hidden)]
                 #[no_mangle]
                 #[link_section = ".init.text"]
                 pub unsafe extern "C" fn init_module() -> ::kernel::ffi::c_int {
@@ -561,14 +559,12 @@ mod __module_init {
                 }
 
                 #[cfg(MODULE)]
-                #[doc(hidden)]
                 #[used(compiler)]
                 #[link_section = ".init.data"]
                 static __UNIQUE_ID___addressable_init_module: unsafe extern "C" fn() -> i32 =
                     init_module;
 
                 #[cfg(MODULE)]
-                #[doc(hidden)]
                 #[no_mangle]
                 #[link_section = ".exit.text"]
                 pub extern "C" fn cleanup_module() {
@@ -582,7 +578,6 @@ pub extern "C" fn cleanup_module() {
                 }
 
                 #[cfg(MODULE)]
-                #[doc(hidden)]
                 #[used(compiler)]
                 #[link_section = ".exit.data"]
                 static __UNIQUE_ID___addressable_cleanup_module: extern "C" fn() = cleanup_module;
@@ -591,7 +586,6 @@ pub extern "C" fn cleanup_module() {
                 // and the identifiers need to be unique.
                 #[cfg(not(MODULE))]
                 #[cfg(not(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS))]
-                #[doc(hidden)]
                 #[link_section = #initcall_section]
                 #[used(compiler)]
                 pub static #ident_initcall: extern "C" fn() ->
@@ -602,7 +596,6 @@ pub extern "C" fn cleanup_module() {
                 ::core::arch::global_asm!(#global_asm);
 
                 #[cfg(not(MODULE))]
-                #[doc(hidden)]
                 #[no_mangle]
                 pub extern "C" fn #ident_init() -> ::kernel::ffi::c_int {
                     // SAFETY: This function is inaccessible to the outside due to the double
@@ -612,7 +605,6 @@ pub extern "C" fn #ident_init() -> ::kernel::ffi::c_int {
                 }
 
                 #[cfg(not(MODULE))]
-                #[doc(hidden)]
                 #[no_mangle]
                 pub extern "C" fn #ident_exit() {
                     // SAFETY:
Signature · 1 line
2.51.2
author
#16 lore raw

PATCH v3 11/12 rust: kunit: use `pin_init::zeroed` instead of custom null value

In reply to Gary Guo

From: Gary Guo <[email protected]>

The last null element can be created (constly) using pin_init::zeroed, so prefer to use it instead of adding a custom way of building it.

---
 rust/kernel/kunit.rs | 26 +-------------------------
 rust/macros/kunit.rs |  4 ++--
 2 files changed, 3 insertions(+), 27 deletions(-)
rust/kernel/kunit.rs +1 -25
index 79436509dd73d..034158cdaf06b 100644
@@ -192,9 +192,6 @@ pub fn is_test_result_ok(t: impl TestResult) -> bool {
 }
 
 /// Represents an individual test case.
-///
-/// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of valid test cases.
-/// Use [`kunit_case_null`] to generate such a delimiter.
 #[doc(hidden)]
 pub const fn kunit_case(
     name: &'static kernel::str::CStr,
@@ -215,27 +212,6 @@ pub const fn kunit_case(
     }
 }
 
-/// Represents the NULL test case delimiter.
-///
-/// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of test cases. This
-/// function returns such a delimiter.
-#[doc(hidden)]
-pub const fn kunit_case_null() -> kernel::bindings::kunit_case {
-    kernel::bindings::kunit_case {
-        run_case: None,
-        name: core::ptr::null_mut(),
-        generate_params: None,
-        attr: kernel::bindings::kunit_attributes {
-            speed: kernel::bindings::kunit_speed_KUNIT_SPEED_NORMAL,
-        },
-        status: kernel::bindings::kunit_status_KUNIT_SUCCESS,
-        module_name: core::ptr::null_mut(),
-        log: core::ptr::null_mut(),
-        param_init: None,
-        param_exit: None,
-    }
-}
-
 /// Registers a KUnit test suite.
 ///
 /// # Safety
@@ -254,7 +230,7 @@ pub const fn kunit_case_null() -> kernel::bindings::kunit_case {
 ///
 /// static mut KUNIT_TEST_CASES: [kernel::bindings::kunit_case; 2] = [
 ///     kernel::kunit::kunit_case(kernel::c_str!("name"), test_fn),
-///     kernel::kunit::kunit_case_null(),
+///     pin_init::zeroed(),
 /// ];
 /// kernel::kunit_unsafe_test_suite!(suite_name, KUNIT_TEST_CASES);
 /// ```
rust/macros/kunit.rs +2 -2
index afbc708cbdc50..cb29f350d5b2b 100644
@@ -74,7 +74,7 @@ pub(crate) fn kunit_tests(test_suite: Ident, mut module: ItemMod) -> Result<Toke
     // static mut TEST_CASES: [::kernel::bindings::kunit_case; 3] = [
     //     ::kernel::kunit::kunit_case(::kernel::c_str!("foo"), kunit_rust_wrapper_foo),
     //     ::kernel::kunit::kunit_case(::kernel::c_str!("bar"), kunit_rust_wrapper_bar),
-    //     ::kernel::kunit::kunit_case_null(),
+    //     ::pin_init::zeroed(),
     // ];
     //
     // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES);
@@ -159,7 +159,7 @@ macro_rules! assert_eq {
     processed_items.push(parse_quote! {
         static mut TEST_CASES: [::kernel::bindings::kunit_case; #num_tests_plus_1] = [
             #(#test_cases,)*
-            ::kernel::kunit::kunit_case_null(),
+            ::pin_init::zeroed(),
         ];
     });
     processed_items.push(parse_quote! {
Signature · 1 line
2.51.2
author
#18 lore raw

PATCH v3 12/12 rust: macros: support `#[cfg]` properly in `#[vtable]` macro.

In reply to Gary Guo

From: Gary Guo <[email protected]>

Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.

Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.

With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.

---
 rust/macros/helpers.rs |  9 +++++++++
 rust/macros/vtable.rs  | 13 ++++---------
 2 files changed, 13 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +9 -0
index adfa60d8f42d8..c17287da801dd 100644
@@ -7,6 +7,7 @@
         Parse,
         ParseStream, //
     },
+    Attribute,
     Error,
     LitStr,
     Result, //
@@ -53,3 +54,11 @@ pub(crate) fn file() -> String {
         proc_macro::Span::call_site().file()
     }
 }
+
+/// Obtain all `#[cfg]` attributes.
+pub(crate) fn gather_cfg_attrs(attr: &[Attribute]) -> Vec<Attribute> {
+    attr.iter()
+        .filter(|a| a.path().is_ident("cfg"))
+        .cloned()
+        .collect()
+}
rust/macros/vtable.rs +4 -9
index 72ae0a1816a04..c6510b0c4ea1d 100644
@@ -23,7 +23,6 @@
 
 fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
     let mut gen_items = Vec::new();
-    let mut gen_consts = HashSet::new();
 
     gen_items.push(parse_quote! {
          /// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable)
@@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
                 &format!("HAS_{}", name.to_string().to_uppercase()),
                 name.span(),
             );
-            // Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively
-            // define functions.
-            // FIXME: `#[cfg]` should be copied and propagated to the generated consts.
-            if gen_consts.contains(&gen_const_name) {
-                continue;
-            }
 
             // We don't know on the implementation-site whether a method is required or provided
             // so we have to generate a const for all methods.
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             let comment =
                 format!("Indicates if the `{name}` method is overridden by the implementor.");
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 #[doc = #comment]
                 const #gen_const_name: bool = false;
             });
-            gen_consts.insert(gen_const_name);
         }
     }
 
@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {
             if defined_consts.contains(&gen_const_name) {
                 continue;
             }
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 const #gen_const_name: bool = true;
             });
-            defined_consts.insert(gen_const_name);
         }
     }
 
Signature · 1 line
2.51.2
Reviewed-by on v3 12/12
#19 lore raw

Re: [PATCH v3 12/12] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.

Quoted 33 lines · Gary Guo

From: Gary Guo <[email protected]>

Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.

Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.

With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.

---
 rust/macros/helpers.rs |  9 +++++++++
 rust/macros/vtable.rs  | 13 ++++---------
 2 files changed, 13 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +1 -0
index adfa60d8f42d8..c17287da801dd 100644
@@ -7,6 +7,7 @@
         Parse,
         ParseStream, //
     },
+    Attribute,
     Error,
     LitStr,
     Result, //
@@ -53,3 +54,11 @@ pub(crate) fn file() -> String {
         proc_macro::Span::call_site().file()

rust/macros/helpers.rs pub(crate) fn file() -> String { · line 64

Quoted 10 lines
rust/macros/helpers.rs +8 -0
     }
 }
+
+/// Obtain all `#[cfg]` attributes.
+pub(crate) fn gather_cfg_attrs(attr: &[Attribute]) -> Vec<Attribute> {
+    attr.iter()
+        .filter(|a| a.path().is_ident("cfg"))
+        .cloned()
+        .collect()
+}

I believe there's no need to collect this into a vector, quote supports iterators.

https://docs.rs/quote/latest/quote/macro.quote.html#interpolation

Quoted 51 lines
rust/macros/vtable.rs +4 -10
index 72ae0a1816a04..c6510b0c4ea1d 100644
@@ -23,7 +23,6 @@
 
 fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
     let mut gen_items = Vec::new();
-    let mut gen_consts = HashSet::new();
 
     gen_items.push(parse_quote! {
          /// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable)
@@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
                 &format!("HAS_{}", name.to_string().to_uppercase()),
                 name.span(),
             );
-            // Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively
-            // define functions.
-            // FIXME: `#[cfg]` should be copied and propagated to the generated consts.
-            if gen_consts.contains(&gen_const_name) {
-                continue;
-            }
 
             // We don't know on the implementation-site whether a method is required or provided
             // so we have to generate a const for all methods.
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             let comment =
                 format!("Indicates if the `{name}` method is overridden by the implementor.");
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 #[doc = #comment]
                 const #gen_const_name: bool = false;
             });
-            gen_consts.insert(gen_const_name);
         }
     }
 
@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {
             if defined_consts.contains(&gen_const_name) {
                 continue;
             }
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 const #gen_const_name: bool = true;
             });
-            defined_consts.insert(gen_const_name);
         }
     }
 
--
Quoted 2 lines

2.51.2

author
#20 lore raw

PATCH 1/1 rust: macros: support `#[cfg]` properly in `#[vtable]` macro.

From: Gary Guo <[email protected]>

Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.

Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.

With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.

---
 rust/macros/helpers.rs |  6 ++++++
 rust/macros/vtable.rs  | 13 ++++---------
 2 files changed, 10 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +6 -0
index adfa60d8f42d8..11117b4c95d25 100644
@@ -7,6 +7,7 @@
         Parse,
         ParseStream, //
     },
+    Attribute,
     Error,
     LitStr,
     Result, //
@@ -53,3 +54,8 @@ pub(crate) fn file() -> String {
         proc_macro::Span::call_site().file()
     }
 }
+
+/// Obtain all `#[cfg]` attributes.
+pub(crate) fn gather_cfg_attrs(attr: &[Attribute]) -> impl Iterator<Item = Attribute> + '_ {
+    attr.iter().filter(|a| a.path().is_ident("cfg")).cloned()
+}
rust/macros/vtable.rs +4 -9
index 72ae0a1816a04..c6510b0c4ea1d 100644
@@ -23,7 +23,6 @@
 
 fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
     let mut gen_items = Vec::new();
-    let mut gen_consts = HashSet::new();
 
     gen_items.push(parse_quote! {
          /// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable)
@@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
                 &format!("HAS_{}", name.to_string().to_uppercase()),
                 name.span(),
             );
-            // Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively
-            // define functions.
-            // FIXME: `#[cfg]` should be copied and propagated to the generated consts.
-            if gen_consts.contains(&gen_const_name) {
-                continue;
-            }
 
             // We don't know on the implementation-site whether a method is required or provided
             // so we have to generate a const for all methods.
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             let comment =
                 format!("Indicates if the `{name}` method is overridden by the implementor.");
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 #[doc = #comment]
                 const #gen_const_name: bool = false;
             });
-            gen_consts.insert(gen_const_name);
         }
     }
 
@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {
             if defined_consts.contains(&gen_const_name) {
                 continue;
             }
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 const #gen_const_name: bool = true;
             });
-            defined_consts.insert(gen_const_name);
         }
     }
 

base-commit: 761e1fbc67b122bfe3d58518a9ea9ee668962c92

Signature · 1 line
2.51.2
#21 lore raw

Re: [PATCH] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.

Quoted 32 lines · Gary Guo

From: Gary Guo <[email protected]>

Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.

Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.

With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.

---
 rust/macros/helpers.rs |  6 ++++++
 rust/macros/vtable.rs  | 13 ++++---------
 2 files changed, 10 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +1 -0
index adfa60d8f42d8..11117b4c95d25 100644
@@ -7,6 +7,7 @@
         Parse,
         ParseStream, //
     },
+    Attribute,
     Error,
     LitStr,
     Result, //

rust/macros/helpers.rs pub(crate) fn file() -> String { · line 61

Quoted 8 lines
rust/macros/helpers.rs +5 -0
@@ -53,3 +54,8 @@ pub(crate) fn file() -> String {
         proc_macro::Span::call_site().file()
     }
 }
+
+/// Obtain all `#[cfg]` attributes.
+pub(crate) fn gather_cfg_attrs(attr: &[Attribute]) -> impl Iterator<Item = Attribute> + '_ {
+    attr.iter().filter(|a| a.path().is_ident("cfg")).cloned()
+}

There's also no need to clone; ToTokens::to_tokens takes &self.

https://docs.rs/quote/latest/quote/trait.ToTokens.html#tymethod.to_tokens

Quoted 55 lines
rust/macros/vtable.rs +4 -9
index 72ae0a1816a04..c6510b0c4ea1d 100644
@@ -23,7 +23,6 @@
 
 fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
     let mut gen_items = Vec::new();
-    let mut gen_consts = HashSet::new();
 
     gen_items.push(parse_quote! {
          /// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable)
@@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
                 &format!("HAS_{}", name.to_string().to_uppercase()),
                 name.span(),
             );
-            // Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively
-            // define functions.
-            // FIXME: `#[cfg]` should be copied and propagated to the generated consts.
-            if gen_consts.contains(&gen_const_name) {
-                continue;
-            }
 
             // We don't know on the implementation-site whether a method is required or provided
             // so we have to generate a const for all methods.
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             let comment =
                 format!("Indicates if the `{name}` method is overridden by the implementor.");
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 #[doc = #comment]
                 const #gen_const_name: bool = false;
             });
-            gen_consts.insert(gen_const_name);
         }
     }
 
@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {
             if defined_consts.contains(&gen_const_name) {
                 continue;
             }
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 const #gen_const_name: bool = true;
             });
-            defined_consts.insert(gen_const_name);
         }
     }
 

base-commit: 761e1fbc67b122bfe3d58518a9ea9ee668962c92 --
2.51.2

maintainer
#22 lore raw

Re: [PATCH] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.

Quoted 44 lines · Tamir Duberstein
Quoted 32 lines · Gary Guo

From: Gary Guo <[email protected]>

Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.

Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.

With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.

---
 rust/macros/helpers.rs |  6 ++++++
 rust/macros/vtable.rs  | 13 ++++---------
 2 files changed, 10 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +1 -0
index adfa60d8f42d8..11117b4c95d25 100644
@@ -7,6 +7,7 @@
         Parse,
         ParseStream, //
     },
+    Attribute,
     Error,
     LitStr,
     Result, //

rust/macros/helpers.rs pub(crate) fn file() -> String { · line 61

Quoted 8 lines
rust/macros/helpers.rs +5 -0
@@ -53,3 +54,8 @@ pub(crate) fn file() -> String {
         proc_macro::Span::call_site().file()
     }
 }
+
+/// Obtain all `#[cfg]` attributes.
+pub(crate) fn gather_cfg_attrs(attr: &[Attribute]) -> impl Iterator<Item = Attribute> + '_ {
+    attr.iter().filter(|a| a.path().is_ident("cfg")).cloned()
+}

There's also no need to clone; ToTokens::to_tokens takes &self.

Quoted 1 line

https://docs.rs/quote/latest/quote/trait.ToTokens.html#tymethod.to_tokens

Indeed. In that case I feel like the line is short enough and we can just inline these and skip the helper function?

Best,
Gary

#23 lore raw

Re: [PATCH] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.

Quoted 46 lines · Gary Guo
Quoted 44 lines · Tamir Duberstein
Quoted 32 lines · Gary Guo

From: Gary Guo <[email protected]>

Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.

Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.

With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.

---
 rust/macros/helpers.rs |  6 ++++++
 rust/macros/vtable.rs  | 13 ++++---------
 2 files changed, 10 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +1 -0
index adfa60d8f42d8..11117b4c95d25 100644
@@ -7,6 +7,7 @@
         Parse,
         ParseStream, //
     },
+    Attribute,
     Error,
     LitStr,
     Result, //

rust/macros/helpers.rs pub(crate) fn file() -> String { · line 61

Quoted 8 lines
rust/macros/helpers.rs +5 -0
@@ -53,3 +54,8 @@ pub(crate) fn file() -> String {
         proc_macro::Span::call_site().file()
     }
 }
+
+/// Obtain all `#[cfg]` attributes.
+pub(crate) fn gather_cfg_attrs(attr: &[Attribute]) -> impl Iterator<Item = Attribute> + '_ {
+    attr.iter().filter(|a| a.path().is_ident("cfg")).cloned()
+}

There's also no need to clone; ToTokens::to_tokens takes &self.

Quoted 4 lines
Quoted 1 line

https://docs.rs/quote/latest/quote/trait.ToTokens.html#tymethod.to_tokens

Indeed. In that case I feel like the line is short enough and we can just inline these and skip the helper function?

I think extracting the duplication is still worthwhile, and we might need this elsewhere so I'd leave it in the helpers module. I don't feel strongly though, so go with your preference.

Quoted 3 lines

Best,
Gary

author
#24 lore raw

PATCH v2 1/1 rust: macros: support `#[cfg]` properly in `#[vtable]` macro.

From: Gary Guo <[email protected]>

Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.

Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.

With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.

---
 rust/macros/helpers.rs |  6 ++++++
 rust/macros/vtable.rs  | 13 ++++---------
 2 files changed, 10 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +6 -0
index adfa60d8f42d8..37ef6a6f2c851 100644
@@ -7,6 +7,7 @@
         Parse,
         ParseStream, //
     },
+    Attribute,
     Error,
     LitStr,
     Result, //
@@ -53,3 +54,8 @@ pub(crate) fn file() -> String {
         proc_macro::Span::call_site().file()
     }
 }
+
+/// Obtain all `#[cfg]` attributes.
+pub(crate) fn gather_cfg_attrs(attr: &[Attribute]) -> impl Iterator<Item = &Attribute> + '_ {
+    attr.iter().filter(|a| a.path().is_ident("cfg"))
+}
rust/macros/vtable.rs +4 -9
index 72ae0a1816a04..c6510b0c4ea1d 100644
@@ -23,7 +23,6 @@
 
 fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
     let mut gen_items = Vec::new();
-    let mut gen_consts = HashSet::new();
 
     gen_items.push(parse_quote! {
          /// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable)
@@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> {
                 &format!("HAS_{}", name.to_string().to_uppercase()),
                 name.span(),
             );
-            // Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively
-            // define functions.
-            // FIXME: `#[cfg]` should be copied and propagated to the generated consts.
-            if gen_consts.contains(&gen_const_name) {
-                continue;
-            }
 
             // We don't know on the implementation-site whether a method is required or provided
             // so we have to generate a const for all methods.
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             let comment =
                 format!("Indicates if the `{name}` method is overridden by the implementor.");
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 #[doc = #comment]
                 const #gen_const_name: bool = false;
             });
-            gen_consts.insert(gen_const_name);
         }
     }
 
@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {
             if defined_consts.contains(&gen_const_name) {
                 continue;
             }
+            let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs);
             gen_items.push(parse_quote! {
+                #(#cfg_attrs)*
                 const #gen_const_name: bool = true;
             });
-            defined_consts.insert(gen_const_name);
         }
     }
 

base-commit: 761e1fbc67b122bfe3d58518a9ea9ee668962c92

Signature · 1 line
2.51.2
maintainer Reviewed-by on v3 3/12
#5 lore raw

Re: [PATCH v3 03/12] rust: macros: convert `#[vtable]` macro to use `syn`

In reply to Gary Guo
Quoted 10 lines · Gary Guo

From: Gary Guo <[email protected]>

#[vtable] is converted to use syn. This is more robust than the previous heuristic-based searching of defined methods and functions.

When doing so, the trait and impl are split into two code paths as the types are distinct when parsed by syn.

Cheers,
Benno

Quoted 4 lines
---
 rust/macros/lib.rs    |   9 ++-
 rust/macros/vtable.rs | 169 +++++++++++++++++++++++-------------------
 2 files changed, 98 insertions(+), 80 deletions(-)
maintainer Reviewed-by on v3 8/12
#12 lore raw

Re: [PATCH v3 08/12] rust: macros: convert `#[kunit_tests]` macro to use `syn`

In reply to Gary Guo
Quoted 10 lines · Gary Guo

From: Gary Guo <[email protected]>

Make use of syn to parse the module structurally and thus improve the robustness of parsing.

String interpolation is avoided by generating tokens directly using quote!.

Cheers,
Benno

Quoted 4 lines
---
 rust/macros/kunit.rs | 274 +++++++++++++++++++------------------------
 rust/macros/lib.rs   |   6 +-
 2 files changed, 123 insertions(+), 157 deletions(-)
maintainer Reviewed-by on v2 1/1
#25 lore raw

Re: [PATCH v2] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.

In reply to Gary Guo
Quoted 5 lines · Gary Guo

From: Gary Guo <[email protected]>

Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.

Quoted 8 lines

Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.

With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.

Cheers,
Benno

Quoted 4 lines
---
 rust/macros/helpers.rs |  6 ++++++
 rust/macros/vtable.rs  | 13 ++++---------
 2 files changed, 10 insertions(+), 9 deletions(-)
maintainer Reviewed-by on v3 4/12
#7 lore raw

Re: [PATCH v3 04/12] rust: macros: use `syn` to parse `module!` macro

In reply to Gary Guo
Quoted 28 lines · Gary Guo

From: Gary Guo <[email protected]>

With syn being available in the kernel, use it to parse the complex custom module! macro to replace existing helpers. Only parsing is changed in this commit, the code generation is untouched.

This has the benefit of better error message when the macro is used incorrectly, as it can point to a concrete span on what's going wrong.

For example, if a field is specified twice, previously it reads:

    error: proc macro panicked
      --> samples/rust/rust_minimal.rs:7:1
       |
    7  | / module! {
    8  | |     type: RustMinimal,
    9  | |     name: "rust_minimal",
    10 | |     author: "Rust for Linux Contributors",
    11 | |     description: "Rust minimal sample",
    12 | |     license: "GPL",
    13 | |     license: "GPL",
    14 | | }
       | |_^
       |
       = help: message: Duplicated key "license". Keys can only be specified once.

now it reads:

Quoted 8 lines
    error: duplicated key "license". Keys can only be specified once.
      --> samples/rust/rust_minimal.rs:13:5
       |
    13 |     license: "GPL",
       |     ^^^^^^^

Cheers,
Benno

Quoted 5 lines
---
 rust/macros/helpers.rs | 109 ++++-------
 rust/macros/lib.rs     |   6 +-
 rust/macros/module.rs  | 399 +++++++++++++++++++++++++----------------
 3 files changed, 280 insertions(+), 234 deletions(-)
Acked-by on v3 2/12
#3 lore raw

Re: [PATCH v3 02/12] rust: macros: use `quote!` from vendored crate

In reply to Gary Guo
Quoted 9 lines · Gary Guo

From: Gary Guo <[email protected]>

With quote crate now vendored in the kernel, we can remove our custom quote! macro implementation and just rely on that crate instead.

The quote crate uses types from the proc-macro2 library so we also update to use that, and perform conversion in the top-level lib.rs.

Quoted 7 lines

Clippy complains about unnecessary .to_string() as proc-macro2 provides additional PartialEq impl, so they are removed.

---

Cheers,
-- David

-- David

Quoted 438 lines
 rust/macros/concat_idents.rs |   2 +-
 rust/macros/export.rs        |   4 +-
 rust/macros/fmt.rs           |   4 +-
 rust/macros/helpers.rs       |   4 +-
 rust/macros/kunit.rs         |   5 +-
 rust/macros/lib.rs           |  21 ++--
 rust/macros/module.rs        |   6 +-
 rust/macros/paste.rs         |   2 +-
 rust/macros/quote.rs         | 182 -----------------------------------
 rust/macros/vtable.rs        |   7 +-
 10 files changed, 32 insertions(+), 205 deletions(-)
 delete mode 100644 rust/macros/quote.rs
rust/macros/concat_idents.rs +1 -1
index 7e4b450f3a507..12cb231c3d715 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{token_stream, Ident, TokenStream, TokenTree};
+use proc_macro2::{token_stream, Ident, TokenStream, TokenTree};
 
 use crate::helpers::expect_punct;
 
rust/macros/export.rs +3 -1
index a08f6337d5c8d..92d9b30971929 100644
@@ -1,7 +1,9 @@
 // SPDX-License-Identifier: GPL-2.0
 
+use proc_macro2::TokenStream;
+use quote::quote;
+
 use crate::helpers::function_name;
-use proc_macro::TokenStream;
 
 /// Please see [`crate::export`] for documentation.
 pub(crate) fn export(_attr: TokenStream, ts: TokenStream) -> TokenStream {
rust/macros/fmt.rs +3 -1
index 2f4b9f6e22110..19f709262552b 100644
@@ -1,8 +1,10 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{Ident, TokenStream, TokenTree};
 use std::collections::BTreeSet;
 
+use proc_macro2::{Ident, TokenStream, TokenTree};
+use quote::quote_spanned;
+
 /// Please see [`crate::fmt`] for documentation.
 pub(crate) fn fmt(input: TokenStream) -> TokenStream {
     let mut input = input.into_iter();
rust/macros/helpers.rs +2 -2
index 365d7eb499c08..13fafaba12261 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{token_stream, Group, Ident, TokenStream, TokenTree};
+use proc_macro2::{token_stream, Group, Ident, TokenStream, TokenTree};
 
 pub(crate) fn try_ident(it: &mut token_stream::IntoIter) -> Option<String> {
     if let Some(TokenTree::Ident(ident)) = it.next() {
@@ -86,7 +86,7 @@ pub(crate) fn function_name(input: TokenStream) -> Option<Ident> {
     let mut input = input.into_iter();
     while let Some(token) = input.next() {
         match token {
-            TokenTree::Ident(i) if i.to_string() == "fn" => {
+            TokenTree::Ident(i) if i == "fn" => {
                 if let Some(TokenTree::Ident(i)) = input.next() {
                     return Some(i);
                 }
rust/macros/kunit.rs +3 -2
index b395bb0536959..5cd6aa5eef07d 100644
@@ -4,10 +4,11 @@
 //!
 //! Copyright (c) 2023 José Expósito <[email protected]>
 
-use proc_macro::{Delimiter, Group, TokenStream, TokenTree};
 use std::collections::HashMap;
 use std::fmt::Write;
 
+use proc_macro2::{Delimiter, Group, TokenStream, TokenTree};
+
 pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
     let attr = attr.to_string();
 
@@ -59,7 +60,7 @@ pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
                 }
                 _ => (),
             },
-            TokenTree::Ident(i) if i.to_string() == "fn" && attributes.contains_key("test") => {
+            TokenTree::Ident(i) if i == "fn" && attributes.contains_key("test") => {
                 if let Some(TokenTree::Ident(test_name)) = body_it.next() {
                     tests.push((test_name, attributes.remove("cfg").unwrap_or_default()))
                 }
rust/macros/lib.rs +11 -10
index b38002151871a..945982c21f703 100644
@@ -11,8 +11,6 @@
 // to avoid depending on the full `proc_macro_span` on Rust >= 1.88.0.
 #![cfg_attr(not(CONFIG_RUSTC_HAS_SPAN_FILE), feature(proc_macro_span))]
 
-#[macro_use]
-mod quote;
 mod concat_idents;
 mod export;
 mod fmt;
@@ -132,7 +130,7 @@
 ///     the kernel module.
 #[proc_macro]
 pub fn module(ts: TokenStream) -> TokenStream {
-    module::module(ts)
+    module::module(ts.into()).into()
 }
 
 /// Declares or implements a vtable trait.
@@ -207,7 +205,7 @@ pub fn module(ts: TokenStream) -> TokenStream {
 /// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html
 #[proc_macro_attribute]
 pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    vtable::vtable(attr, ts)
+    vtable::vtable(attr.into(), ts.into()).into()
 }
 
 /// Export a function so that C code can call it via a header file.
@@ -230,7 +228,7 @@ pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream {
 /// automatically exported with `EXPORT_SYMBOL_GPL`.
 #[proc_macro_attribute]
 pub fn export(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    export::export(attr, ts)
+    export::export(attr.into(), ts.into()).into()
 }
 
 /// Like [`core::format_args!`], but automatically wraps arguments in [`kernel::fmt::Adapter`].
@@ -248,7 +246,7 @@ pub fn export(attr: TokenStream, ts: TokenStream) -> TokenStream {
 /// [`pr_info!`]: ../kernel/macro.pr_info.html
 #[proc_macro]
 pub fn fmt(input: TokenStream) -> TokenStream {
-    fmt::fmt(input)
+    fmt::fmt(input.into()).into()
 }
 
 /// Concatenate two identifiers.
@@ -306,7 +304,7 @@ pub fn fmt(input: TokenStream) -> TokenStream {
 /// ```
 #[proc_macro]
 pub fn concat_idents(ts: TokenStream) -> TokenStream {
-    concat_idents::concat_idents(ts)
+    concat_idents::concat_idents(ts.into()).into()
 }
 
 /// Paste identifiers together.
@@ -444,9 +442,12 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {
 /// [`paste`]: https://docs.rs/paste/
 #[proc_macro]
 pub fn paste(input: TokenStream) -> TokenStream {
-    let mut tokens = input.into_iter().collect();
+    let mut tokens = proc_macro2::TokenStream::from(input).into_iter().collect();
     paste::expand(&mut tokens);
-    tokens.into_iter().collect()
+    tokens
+        .into_iter()
+        .collect::<proc_macro2::TokenStream>()
+        .into()
 }
 
 /// Registers a KUnit test suite and its test cases using a user-space like syntax.
@@ -473,5 +474,5 @@ pub fn paste(input: TokenStream) -> TokenStream {
 /// ```
 #[proc_macro_attribute]
 pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    kunit::kunit_tests(attr, ts)
+    kunit::kunit_tests(attr.into(), ts.into()).into()
 }
rust/macros/module.rs +4 -2
index 80cb9b16f5aaf..b855a2b586e18 100644
@@ -1,9 +1,11 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use crate::helpers::*;
-use proc_macro::{token_stream, Delimiter, Literal, TokenStream, TokenTree};
 use std::fmt::Write;
 
+use proc_macro2::{token_stream, Delimiter, Literal, TokenStream, TokenTree};
+
+use crate::helpers::*;
+
 fn expect_string_array(it: &mut token_stream::IntoIter) -> Vec<String> {
     let group = expect_group(it);
     assert_eq!(group.delimiter(), Delimiter::Bracket);
rust/macros/paste.rs +1 -1
index cce712d19855b..2181e312a7d32 100644
@@ -1,6 +1,6 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{Delimiter, Group, Ident, Spacing, Span, TokenTree};
+use proc_macro2::{Delimiter, Group, Ident, Spacing, Span, TokenTree};
 
 fn concat_helper(tokens: &[TokenTree]) -> Vec<(String, Span)> {
     let mut tokens = tokens.iter();
rust/macros/quote.rs deleted +0 -182
deleted file mode 100644
index ddfc21577539c..0000000000000
@@ -1,182 +0,0 @@
-// SPDX-License-Identifier: Apache-2.0 OR MIT
-
-use proc_macro::{TokenStream, TokenTree};
-
-pub(crate) trait ToTokens {
-    fn to_tokens(&self, tokens: &mut TokenStream);
-}
-
-impl<T: ToTokens> ToTokens for Option<T> {
-    fn to_tokens(&self, tokens: &mut TokenStream) {
-        if let Some(v) = self {
-            v.to_tokens(tokens);
-        }
-    }
-}
-
-impl ToTokens for proc_macro::Group {
-    fn to_tokens(&self, tokens: &mut TokenStream) {
-        tokens.extend([TokenTree::from(self.clone())]);
-    }
-}
-
-impl ToTokens for proc_macro::Ident {
-    fn to_tokens(&self, tokens: &mut TokenStream) {
-        tokens.extend([TokenTree::from(self.clone())]);
-    }
-}
-
-impl ToTokens for TokenTree {
-    fn to_tokens(&self, tokens: &mut TokenStream) {
-        tokens.extend([self.clone()]);
-    }
-}
-
-impl ToTokens for TokenStream {
-    fn to_tokens(&self, tokens: &mut TokenStream) {
-        tokens.extend(self.clone());
-    }
-}
-
-/// Converts tokens into [`proc_macro::TokenStream`] and performs variable interpolations with
-/// the given span.
-///
-/// This is a similar to the
-/// [`quote_spanned!`](https://docs.rs/quote/latest/quote/macro.quote_spanned.html) macro from the
-/// `quote` crate but provides only just enough functionality needed by the current `macros` crate.
-macro_rules! quote_spanned {
-    ($span:expr => $($tt:tt)*) => {{
-        let mut tokens = ::proc_macro::TokenStream::new();
-        {
-            #[allow(unused_variables)]
-            let span = $span;
-            quote_spanned!(@proc tokens span $($tt)*);
-        }
-        tokens
-    }};
-    (@proc $v:ident $span:ident) => {};
-    (@proc $v:ident $span:ident #$id:ident $($tt:tt)*) => {
-        $crate::quote::ToTokens::to_tokens(&$id, &mut $v);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident #(#$id:ident)* $($tt:tt)*) => {
-        for token in $id {
-            $crate::quote::ToTokens::to_tokens(&token, &mut $v);
-        }
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident ( $($inner:tt)* ) $($tt:tt)*) => {
-        #[allow(unused_mut)]
-        let mut tokens = ::proc_macro::TokenStream::new();
-        quote_spanned!(@proc tokens $span $($inner)*);
-        $v.extend([::proc_macro::TokenTree::Group(::proc_macro::Group::new(
-            ::proc_macro::Delimiter::Parenthesis,
-            tokens,
-        ))]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident [ $($inner:tt)* ] $($tt:tt)*) => {
-        let mut tokens = ::proc_macro::TokenStream::new();
-        quote_spanned!(@proc tokens $span $($inner)*);
-        $v.extend([::proc_macro::TokenTree::Group(::proc_macro::Group::new(
-            ::proc_macro::Delimiter::Bracket,
-            tokens,
-        ))]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident { $($inner:tt)* } $($tt:tt)*) => {
-        let mut tokens = ::proc_macro::TokenStream::new();
-        quote_spanned!(@proc tokens $span $($inner)*);
-        $v.extend([::proc_macro::TokenTree::Group(::proc_macro::Group::new(
-            ::proc_macro::Delimiter::Brace,
-            tokens,
-        ))]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident :: $($tt:tt)*) => {
-        $v.extend([::proc_macro::Spacing::Joint, ::proc_macro::Spacing::Alone].map(|spacing| {
-            ::proc_macro::TokenTree::Punct(::proc_macro::Punct::new(':', spacing))
-        }));
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident : $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new(':', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident , $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new(',', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident @ $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new('@', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident ! $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new('!', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident ; $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new(';', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident + $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new('+', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident = $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new('=', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident # $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new('#', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident & $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Punct(
-            ::proc_macro::Punct::new('&', ::proc_macro::Spacing::Alone),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident _ $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Ident(
-            ::proc_macro::Ident::new("_", $span),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-    (@proc $v:ident $span:ident $id:ident $($tt:tt)*) => {
-        $v.extend([::proc_macro::TokenTree::Ident(
-            ::proc_macro::Ident::new(stringify!($id), $span),
-        )]);
-        quote_spanned!(@proc $v $span $($tt)*);
-    };
-}
-
-/// Converts tokens into [`proc_macro::TokenStream`] and performs variable interpolations with
-/// mixed site span ([`Span::mixed_site()`]).
-///
-/// This is a similar to the [`quote!`](https://docs.rs/quote/latest/quote/macro.quote.html) macro
-/// from the `quote` crate but provides only just enough functionality needed by the current
-/// `macros` crate.
-///
-/// [`Span::mixed_site()`]: https://doc.rust-lang.org/proc_macro/struct.Span.html#method.mixed_site
-macro_rules! quote {
-    ($($tt:tt)*) => {
-        quote_spanned!(::proc_macro::Span::mixed_site() => $($tt)*)
-    }
-}
rust/macros/vtable.rs +4 -4
index ee06044fcd4f3..a67d1cc81a2d3 100644
@@ -1,9 +1,10 @@
 // SPDX-License-Identifier: GPL-2.0
 
-use proc_macro::{Delimiter, Group, TokenStream, TokenTree};
 use std::collections::HashSet;
 use std::fmt::Write;
 
+use proc_macro2::{Delimiter, Group, TokenStream, TokenTree};
+
 pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream {
     let mut tokens: Vec<_> = ts.into_iter().collect();
 
@@ -31,7 +32,7 @@ pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream {
     let mut consts = HashSet::new();
     while let Some(token) = body_it.next() {
         match token {
-            TokenTree::Ident(ident) if ident.to_string() == "fn" => {
+            TokenTree::Ident(ident) if ident == "fn" => {
                 let fn_name = match body_it.next() {
                     Some(TokenTree::Ident(ident)) => ident.to_string(),
                     // Possibly we've encountered a fn pointer type instead.
@@ -39,7 +40,7 @@ pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream {
                 };
                 functions.push(fn_name);
             }
-            TokenTree::Ident(ident) if ident.to_string() == "const" => {
+            TokenTree::Ident(ident) if ident == "const" => {
                 let const_name = match body_it.next() {
                     Some(TokenTree::Ident(ident)) => ident.to_string(),
                     // Possibly we've encountered an inline const block instead.
--

2.51.2

Reviewed-by on v3 8/12
#13 lore raw

Re: [PATCH v3 08/12] rust: macros: convert `#[kunit_tests]` macro to use `syn`

In reply to Gary Guo
Quoted 6 lines · Gary Guo

From: Gary Guo <[email protected]>

Make use of syn to parse the module structurally and thus improve the robustness of parsing.

Quoted 6 lines

String interpolation is avoided by generating tokens directly using quote!.

---

Sorry this took so long to review: it's taken a while for me to wrap my head around syn.

I'm pretty happy with this overall, though, and think I understand it well enough now.

Cheers,
-- David

Quoted 13 lines
 rust/macros/kunit.rs | 274 +++++++++++++++++++------------------------
 rust/macros/lib.rs   |   6 +-
 2 files changed, 123 insertions(+), 157 deletions(-)
rust/macros/kunit.rs +0 -1
index 5cd6aa5eef07d..afbc708cbdc50 100644
@@ -4,81 +4,50 @@
 //!
 //! Copyright (c) 2023 José Expósito <[email protected]>
 
-use std::collections::HashMap;

rust/macros/kunit.rs line 17

Quoted 10 lines
rust/macros/kunit.rs +0 -10
-use std::fmt::Write;
-
-use proc_macro2::{Delimiter, Group, TokenStream, TokenTree};
-
-pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    let attr = attr.to_string();
-
-    if attr.is_empty() {
-        panic!("Missing test name in `#[kunit_tests(test_name)]` macro")
-    }

I'm assuming it's okay for us to get rid of this as we get an "unexpected end of input" error instead.

Quoted 303 lines
rust/macros/kunit.rs +119 -144
-
-    if attr.len() > 255 {
-        panic!("The test suite name `{attr}` exceeds the maximum length of 255 bytes")
+use std::ffi::CString;
+
+use proc_macro2::TokenStream;
+use quote::{
+    format_ident,
+    quote,
+    ToTokens, //
+};
+use syn::{
+    parse_quote,
+    Error,
+    Ident,
+    Item,
+    ItemMod,
+    LitCStr,
+    Result, //
+};
+
+pub(crate) fn kunit_tests(test_suite: Ident, mut module: ItemMod) -> Result<TokenStream> {
+    if test_suite.to_string().len() > 255 {
+        return Err(Error::new_spanned(
+            test_suite,
+            "test suite names cannot exceed the maximum length of 255 bytes",
+        ));
     }
 
-    let mut tokens: Vec<_> = ts.into_iter().collect();
-
-    // Scan for the `mod` keyword.
-    tokens
-        .iter()
-        .find_map(|token| match token {
-            TokenTree::Ident(ident) => match ident.to_string().as_str() {
-                "mod" => Some(true),
-                _ => None,
-            },
-            _ => None,
-        })
-        .expect("`#[kunit_tests(test_name)]` attribute should only be applied to modules");
-
-    // Retrieve the main body. The main body should be the last token tree.
-    let body = match tokens.pop() {
-        Some(TokenTree::Group(group)) if group.delimiter() == Delimiter::Brace => group,
-        _ => panic!("Cannot locate main body of module"),
+    // We cannot handle modules that defer to another file (e.g. `mod foo;`).
+    let Some((module_brace, module_items)) = module.content.take() else {
+        Err(Error::new_spanned(
+            module,
+            "`#[kunit_tests(test_name)]` attribute should only be applied to inline modules",
+        ))?
     };
 
-    // Get the functions set as tests. Search for `[test]` -> `fn`.
-    let mut body_it = body.stream().into_iter();
-    let mut tests = Vec::new();
-    let mut attributes: HashMap<String, TokenStream> = HashMap::new();
-    while let Some(token) = body_it.next() {
-        match token {
-            TokenTree::Punct(ref p) if p.as_char() == '#' => match body_it.next() {
-                Some(TokenTree::Group(g)) if g.delimiter() == Delimiter::Bracket => {
-                    if let Some(TokenTree::Ident(name)) = g.stream().into_iter().next() {
-                        // Collect attributes because we need to find which are tests. We also
-                        // need to copy `cfg` attributes so tests can be conditionally enabled.
-                        attributes
-                            .entry(name.to_string())
-                            .or_default()
-                            .extend([token, TokenTree::Group(g)]);
-                    }
-                    continue;
-                }
-                _ => (),
-            },
-            TokenTree::Ident(i) if i == "fn" && attributes.contains_key("test") => {
-                if let Some(TokenTree::Ident(test_name)) = body_it.next() {
-                    tests.push((test_name, attributes.remove("cfg").unwrap_or_default()))
-                }
-            }
-
-            _ => (),
-        }
-        attributes.clear();
-    }
+    // Make the entire module gated behind `CONFIG_KUNIT`.
+    module
+        .attrs
+        .insert(0, parse_quote!(#[cfg(CONFIG_KUNIT="y")]));
 
-    // Add `#[cfg(CONFIG_KUNIT="y")]` before the module declaration.
-    let config_kunit = "#[cfg(CONFIG_KUNIT=\"y\")]".to_owned().parse().unwrap();
-    tokens.insert(
-        0,
-        TokenTree::Group(Group::new(Delimiter::None, config_kunit)),
-    );
+    let mut processed_items = Vec::new();
+    let mut test_cases = Vec::new();
 
     // Generate the test KUnit test suite and a test case for each `#[test]`.
+    //
     // The code generated for the following test module:
     //
     // ```
@@ -110,98 +79,93 @@ pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
     //
     // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES);
     // ```
-    let mut kunit_macros = "".to_owned();
-    let mut test_cases = "".to_owned();
-    let mut assert_macros = "".to_owned();
-    let path = crate::helpers::file();
-    let num_tests = tests.len();
-    for (test, cfg_attr) in tests {
-        let kunit_wrapper_fn_name = format!("kunit_rust_wrapper_{test}");
-        // Append any `cfg` attributes the user might have written on their tests so we don't
-        // attempt to call them when they are `cfg`'d out. An extra `use` is used here to reduce
-        // the length of the assert message.
-        let kunit_wrapper = format!(
-            r#"unsafe extern "C" fn {kunit_wrapper_fn_name}(_test: *mut ::kernel::bindings::kunit)
-            {{
-                (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED;
-                {cfg_attr} {{
-                    (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS;
-                    use ::kernel::kunit::is_test_result_ok;
-                    assert!(is_test_result_ok({test}()));
+    //
+    // Non-function items (e.g. imports) are preserved.
+    for item in module_items {
+        let Item::Fn(mut f) = item else {
+            processed_items.push(item);
+            continue;
+        };
+
+        // TODO: Replace below with `extract_if` when MSRV is bumped above 1.85.
+        let before_len = f.attrs.len();
+        f.attrs.retain(|attr| !attr.path().is_ident("test"));
+        if f.attrs.len() == before_len {
+            processed_items.push(Item::Fn(f));
+            continue;
+        }
+
+        let test = f.sig.ident.clone();
+
+        // Retrieve `#[cfg]` applied on the function which needs to be present on derived items too.
+        let cfg_attrs: Vec<_> = f
+            .attrs
+            .iter()
+            .filter(|attr| attr.path().is_ident("cfg"))
+            .cloned()
+            .collect();
+
+        // Before the test, override usual `assert!` and `assert_eq!` macros with ones that call
+        // KUnit instead.
+        let test_str = test.to_string();
+        let path = crate::helpers::file();
+        processed_items.push(parse_quote! {
+            #[allow(unused)]
+            macro_rules! assert {
+                ($cond:expr $(,)?) => {{
+                    kernel::kunit_assert!(#test_str, #path, 0, $cond);
+                }}
+            }
+        });
+        processed_items.push(parse_quote! {
+            #[allow(unused)]
+            macro_rules! assert_eq {
+                ($left:expr, $right:expr $(,)?) => {{
+                    kernel::kunit_assert_eq!(#test_str, #path, 0, $left, $right);
                 }}
-            }}"#,
+            }
+        });
+
+        // Add back the test item.
+        processed_items.push(Item::Fn(f));
+
+        let kunit_wrapper_fn_name = format_ident!("kunit_rust_wrapper_{test}");
+        let test_cstr = LitCStr::new(
+            &CString::new(test_str.as_str()).expect("identifier cannot contain NUL"),
+            test.span(),
         );
-        writeln!(kunit_macros, "{kunit_wrapper}").unwrap();
-        writeln!(
-            test_cases,
-            "    ::kernel::kunit::kunit_case(::kernel::c_str!(\"{test}\"), {kunit_wrapper_fn_name}),"
-        )
-        .unwrap();
-        writeln!(
-            assert_macros,
-            r#"
-/// Overrides the usual [`assert!`] macro with one that calls KUnit instead.
-#[allow(unused)]
-macro_rules! assert {{
-    ($cond:expr $(,)?) => {{{{
-        kernel::kunit_assert!("{test}", "{path}", 0, $cond);
-    }}}}
-}}
-
-/// Overrides the usual [`assert_eq!`] macro with one that calls KUnit instead.
-#[allow(unused)]
-macro_rules! assert_eq {{
-    ($left:expr, $right:expr $(,)?) => {{{{
-        kernel::kunit_assert_eq!("{test}", "{path}", 0, $left, $right);
-    }}}}
-}}
-        "#
-        )
-        .unwrap();
-    }
+        processed_items.push(parse_quote! {
+            unsafe extern "C" fn #kunit_wrapper_fn_name(_test: *mut ::kernel::bindings::kunit) {
+                (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED;
 
-    writeln!(kunit_macros).unwrap();
-    writeln!(
-        kunit_macros,
-        "static mut TEST_CASES: [::kernel::bindings::kunit_case; {}] = [\n{test_cases}    ::kernel::kunit::kunit_case_null(),\n];",
-        num_tests + 1
-    )
-    .unwrap();
-
-    writeln!(
-        kunit_macros,
-        "::kernel::kunit_unsafe_test_suite!({attr}, TEST_CASES);"
-    )
-    .unwrap();
-
-    // Remove the `#[test]` macros.
-    // We do this at a token level, in order to preserve span information.
-    let mut new_body = vec![];
-    let mut body_it = body.stream().into_iter();
-
-    while let Some(token) = body_it.next() {
-        match token {
-            TokenTree::Punct(ref c) if c.as_char() == '#' => match body_it.next() {
-                Some(TokenTree::Group(group)) if group.to_string() == "[test]" => (),
-                Some(next) => {
-                    new_body.extend([token, next]);
-                }
-                _ => {
-                    new_body.push(token);
+                // Append any `cfg` attributes the user might have written on their tests so we
+                // don't attempt to call them when they are `cfg`'d out. An extra `use` is used
+                // here to reduce the length of the assert message.
+                #(#cfg_attrs)*
+                {
+                    (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS;
+                    use ::kernel::kunit::is_test_result_ok;
+                    assert!(is_test_result_ok(#test()));
                 }
-            },
-            _ => {
-                new_body.push(token);
             }
-        }
-    }
+        });
 
-    let mut final_body = TokenStream::new();
-    final_body.extend::<TokenStream>(assert_macros.parse().unwrap());
-    final_body.extend(new_body);
-    final_body.extend::<TokenStream>(kunit_macros.parse().unwrap());
-
-    tokens.push(TokenTree::Group(Group::new(Delimiter::Brace, final_body)));
+        test_cases.push(quote!(
+            ::kernel::kunit::kunit_case(#test_cstr, #kunit_wrapper_fn_name)
+        ));
+    }
 
-    tokens.into_iter().collect()
+    let num_tests_plus_1 = test_cases.len() + 1;
+    processed_items.push(parse_quote! {
+        static mut TEST_CASES: [::kernel::bindings::kunit_case; #num_tests_plus_1] = [
+            #(#test_cases,)*
+            ::kernel::kunit::kunit_case_null(),
+        ];
+    });
+    processed_items.push(parse_quote! {
+        ::kernel::kunit_unsafe_test_suite!(#test_suite, TEST_CASES);
+    });
+
+    module.content = Some((module_brace, processed_items));
+    Ok(module.to_token_stream())
 }
rust/macros/lib.rs +4 -3
index 12467bfc703a8..75ac60abe6ffa 100644
@@ -481,6 +481,8 @@ pub fn paste(input: TokenStream) -> TokenStream {
 /// }
 /// ```
 #[proc_macro_attribute]
-pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {
-    kunit::kunit_tests(attr.into(), ts.into()).into()
+pub fn kunit_tests(attr: TokenStream, input: TokenStream) -> TokenStream {
+    kunit::kunit_tests(parse_macro_input!(attr), parse_macro_input!(input))
+        .unwrap_or_else(|e| e.into_compile_error())
+        .into()
 }
--

2.51.2

Reviewed-by on v3 11/12
#17 lore raw

Re: [PATCH v3 11/12] rust: kunit: use `pin_init::zeroed` instead of custom null value

In reply to Gary Guo
Quoted 10 lines · Gary Guo

From: Gary Guo <[email protected]>

The last null element can be created (constly) using pin_init::zeroed, so prefer to use it instead of adding a custom way of building it.

---

(Do note, however, that this conflicts with
https://lore.kernel.org/rust-for-linux/[email protected]/)

Cheers,
-- David

Quoted 80 lines
 rust/kernel/kunit.rs | 26 +-------------------------
 rust/macros/kunit.rs |  4 ++--
 2 files changed, 3 insertions(+), 27 deletions(-)
rust/kernel/kunit.rs +1 -25
index 79436509dd73d..034158cdaf06b 100644
@@ -192,9 +192,6 @@ pub fn is_test_result_ok(t: impl TestResult) -> bool {
 }
 
 /// Represents an individual test case.
-///
-/// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of valid test cases.
-/// Use [`kunit_case_null`] to generate such a delimiter.
 #[doc(hidden)]
 pub const fn kunit_case(
     name: &'static kernel::str::CStr,
@@ -215,27 +212,6 @@ pub const fn kunit_case(
     }
 }
 
-/// Represents the NULL test case delimiter.
-///
-/// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of test cases. This
-/// function returns such a delimiter.
-#[doc(hidden)]
-pub const fn kunit_case_null() -> kernel::bindings::kunit_case {
-    kernel::bindings::kunit_case {
-        run_case: None,
-        name: core::ptr::null_mut(),
-        generate_params: None,
-        attr: kernel::bindings::kunit_attributes {
-            speed: kernel::bindings::kunit_speed_KUNIT_SPEED_NORMAL,
-        },
-        status: kernel::bindings::kunit_status_KUNIT_SUCCESS,
-        module_name: core::ptr::null_mut(),
-        log: core::ptr::null_mut(),
-        param_init: None,
-        param_exit: None,
-    }
-}
-
 /// Registers a KUnit test suite.
 ///
 /// # Safety
@@ -254,7 +230,7 @@ pub const fn kunit_case_null() -> kernel::bindings::kunit_case {
 ///
 /// static mut KUNIT_TEST_CASES: [kernel::bindings::kunit_case; 2] = [
 ///     kernel::kunit::kunit_case(kernel::c_str!("name"), test_fn),
-///     kernel::kunit::kunit_case_null(),
+///     pin_init::zeroed(),
 /// ];
 /// kernel::kunit_unsafe_test_suite!(suite_name, KUNIT_TEST_CASES);
 /// ```
rust/macros/kunit.rs +2 -3
index afbc708cbdc50..cb29f350d5b2b 100644
@@ -74,7 +74,7 @@ pub(crate) fn kunit_tests(test_suite: Ident, mut module: ItemMod) -> Result<Toke
     // static mut TEST_CASES: [::kernel::bindings::kunit_case; 3] = [
     //     ::kernel::kunit::kunit_case(::kernel::c_str!("foo"), kunit_rust_wrapper_foo),
     //     ::kernel::kunit::kunit_case(::kernel::c_str!("bar"), kunit_rust_wrapper_bar),
-    //     ::kernel::kunit::kunit_case_null(),
+    //     ::pin_init::zeroed(),
     // ];
     //
     // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES);
@@ -159,7 +159,7 @@ macro_rules! assert_eq {
     processed_items.push(parse_quote! {
         static mut TEST_CASES: [::kernel::bindings::kunit_case; #num_tests_plus_1] = [
             #(#test_cases,)*
-            ::kernel::kunit::kunit_case_null(),
+            ::pin_init::zeroed(),
         ];
     });
     processed_items.push(parse_quote! {
--

2.51.2

Applied
#26 lore raw
In reply to Gary Guo
Quoted 14 lines · Gary Guo

This series convert Rust proc macros that we have with syn, and replace the custom quote! macro that we have with the vendored quote! macro. The pin-init macros are not converted syn yet; Benno has a work in progress in converting them. They're however converted to use quote and proc-macro2 crate so our custom quote! macro can be removed.

Overall this improves the robustness of the macros as we have precise parsing of the AST rather than relying on heuristics to extract needed information from there. This is also a quality-of-life improvement to those using language servers (e.g. Rust analyzer) as the span information of the proc macros are now preserved which allows the "jump-to-definition" feature to work, even when used on completely custom macros such as module!.

Quoted 2 lines

Miguel gave a very good explaination on why syn is a good idea in the patch series that introduced it [1], which I shall not repeat here.

Applied to rust-next -- thanks everyone!

[ Pass C string to match commit 6c37b6841a92 ("rust: kunit: replace
kernel::c_str! with C-Strings"). - Miguel ]

I dropped the first patch as it came from Benno's PR, and for the last one I took the v2.

I also took the cover letter and used it as a merge commit to give some context (with some typos/small rewording done -- I pinged Gary privately to double-check).

I may start doing that for larger series and similar, so cover letters that give proper context like this one are great.

Please double-check things look good!

Cheers,
Miguel