refactor Rust proc macros with `syn`
Subsystems (6)
v3 cover refactor Rust proc macros with `syn`
From: Gary Guo <[email protected]>
This series convert Rust proc macros that we have with syn, and replace the custom quote! macro that we have with the vendored quote! macro. The pin-init macros are not converted syn yet; Benno has a work in progress in converting them. They're however converted to use quote and proc-macro2 crate so our custom quote! macro can be removed.
Overall this improves the robustness of the macros as we have precise parsing of the AST rather than relying on heuristics to extract needed information from there. This is also a quality-of-life improvement to those using language servers (e.g. Rust analyzer) as the span information of the proc macros are now preserved which allows the "jump-to-definition" feature to work, even when used on completely custom macros such as module!.
Miguel gave a very good explaination on why syn is a good idea in the patch series that introduced it [1], which I shall not repeat here.
Changes since v2:
- Changes are limited to #[vtable] and module! macros.
- #[vtable] macro is reworked per Tamir's suggestion. This makes it easy
to support #[cfg] properly, and hence a new patch is added to do that.
- Avoided variable renames related to param_name/param_type between
patches in module! macro
- CString is now interpolated directly without adding extra Literal::c_string.
- Fixed an typo that affects only CONFIG_HAVE_ARCH_PREL32_RELOCATIONS
builds.
- Link to v2: https://lore.kernel.org/rust-for-linux/[email protected]/
Changes since v1:
- Fixed clippy warnings when moving from proc-macro to proc-macro2
(extra trait impl in proc-macro2 causes some .to_string() to be unnecessary).
- A Makefile change to pin-init-internal that is mistakenly included in
patch 5/11 is moved to the correct patch 1/11.
- module! parsing has been completely reworked to support imports_ns
and params. As there're two structs that need to parse ordered fields (ModuleInfo and Parameter), a parse_ordered_field macro is added
and it is used to implement the parsing instead of ModInfoField and
custom keywords.
- Link to v1: https://lore.kernel.org/rust-for-linux/[email protected]/
Benno Lossin (1):
rust: pin-init: internal: remove proc-macro[2] and quote workarounds
Gary Guo (11):
rust: macros: use quote! from vendored crate
rust: macros: convert #[vtable] macro to use syn
rust: macros: use syn to parse module! macro
rust: macros: use quote! for module! macro
rust: macros: convert #[export] to use syn
rust: macros: convert concat_idents! to use syn
rust: macros: convert #[kunit_tests] macro to use syn
rust: macros: allow arbitrary types to be used in module! macro
rust: macros: rearrange #[doc(hidden)] in module! macro
rust: kunit: use pin_init::zeroed instead of custom null value
rust: macros: support #[cfg] properly in #[vtable] macro.
rust/Makefile | 16 +- rust/kernel/kunit.rs | 26 +- rust/macros/concat_idents.rs | 39 +- rust/macros/export.rs | 26 +- rust/macros/fmt.rs | 4 +- rust/macros/helpers.rs | 130 +--- rust/macros/kunit.rs | 275 +++---- rust/macros/lib.rs | 41 +- rust/macros/module.rs | 907 ++++++++++++---------- rust/macros/paste.rs | 2 +- rust/macros/quote.rs | 182 ----- rust/macros/vtable.rs | 165 ++-- rust/pin-init/internal/src/helpers.rs | 7 +- rust/pin-init/internal/src/lib.rs | 16 - rust/pin-init/internal/src/pin_data.rs | 18 +- rust/pin-init/internal/src/pinned_drop.rs | 10 +- rust/pin-init/internal/src/zeroable.rs | 6 +- scripts/generate_rust_analyzer.py | 2 +- 18 files changed, 842 insertions(+), 1030 deletions(-) delete mode 100644 rust/macros/quote.rs
base-commit: 9ace4753a5202b02191d54e9fdf7f9e3d02b85eb
Signature · 1 line
2.51.2
PATCH v3 1/12 rust: pin-init: internal: remove proc-macro[2] and quote workarounds
From: Benno Lossin <[email protected]>
The kernel only had the proc-macro library available, whereas the user-space version also used proc-macro2 and quote. Now both are available to the kernel, making it possible to remove the workarounds.
Clippy complains about unnecessary .to_string() as proc-macro2 provides additional PartialEq impl, so they are removed.
--- rust/Makefile | 16 ++++++++++------ rust/pin-init/internal/src/helpers.rs | 7 ++----- rust/pin-init/internal/src/lib.rs | 16 ---------------- rust/pin-init/internal/src/pin_data.rs | 18 ++++++------------ rust/pin-init/internal/src/pinned_drop.rs | 10 ++++------ rust/pin-init/internal/src/zeroable.rs | 6 ++---- scripts/generate_rust_analyzer.py | 2 +- 7 files changed, 25 insertions(+), 50 deletions(-)
rust/Makefile +10 -6
@@ -212,9 +212,10 @@ rustdoc-ffi: $(src)/ffi.rs rustdoc-core FORCE rustdoc-pin_init_internal: private rustdoc_host = yes rustdoc-pin_init_internal: private rustc_target_flags = --cfg kernel \ - --extern proc_macro --crate-type proc-macro + --extern proc_macro --extern proc_macro2 --extern quote --extern syn \ + --crate-type proc-macro rustdoc-pin_init_internal: $(src)/pin-init/internal/src/lib.rs \ - rustdoc-clean FORCE + rustdoc-clean rustdoc-proc_macro2 rustdoc-quote rustdoc-syn FORCE +$(call if_changed,rustdoc) rustdoc-pin_init: private rustdoc_host = yes
@@ -273,9 +274,10 @@ rusttestlib-macros: $(src)/macros/lib.rs \ +$(call if_changed,rustc_test_library) rusttestlib-pin_init_internal: private rustc_target_flags = --cfg kernel \ - --extern proc_macro + --extern proc_macro --extern proc_macro2 --extern quote --extern syn rusttestlib-pin_init_internal: private rustc_test_library_proc = yes -rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs FORCE +rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs \ + rusttestlib-proc_macro2 rusttestlib-quote rusttestlib-syn FORCE +$(call if_changed,rustc_test_library) rusttestlib-pin_init: private rustc_target_flags = --extern pin_init_internal \
@@ -547,8 +549,10 @@ $(obj)/$(libmacros_name): $(src)/macros/lib.rs $(obj)/libproc_macro2.rlib \ $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE +$(call if_changed_dep,rustc_procmacro) -$(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel -$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs FORCE +$(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel \ + --extern proc_macro2 --extern quote --extern syn +$(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs \ + $(obj)/libproc_macro2.rlib $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE +$(call if_changed_dep,rustc_procmacro) quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@
rust/pin-init/internal/src/helpers.rs +2 -5
@@ -1,9 +1,6 @@// SPDX-License-Identifier: Apache-2.0 OR MIT -#[cfg(not(kernel))] -use proc_macro2as proc_macro; - -use proc_macro::{TokenStream ,TokenTree }; +use proc_macro2::{TokenStream ,TokenTree };/// Parsed generics. ///
@@ -101,7 +98,7 @@ pub(crate) fn parse_generics(input: TokenStream) -> (Generics, Vec<TokenTree>) { 1 => {// Here depending on the token, it might be a generic variable name. match tt.clone () { -TokenTree ::Ident (i)if at_start && i.to_string () =="const" => { +TokenTree ::Ident (i)if at_start && i =="const" => {let Some (name) = toks.next ()else {// Parsing error. break ;
rust/pin-init/internal/src/lib.rs +0 -16
@@ -7,27 +7,11 @@//! `pin-init` proc macros. #![cfg_attr(not( RUSTC_LINT_REASONS_IS_STABLE ), feature(lint_reasons))] -// Allow `.into()` to convert -// - `proc_macro2::TokenStream` into `proc_macro::TokenStream` in the user-space version. -// - `proc_macro::TokenStream` into `proc_macro::TokenStream` in the kernel version. -// Clippy warns on this conversion, but it's required by the user-space version. -// -// Remove once we have `proc_macro2` in the kernel. -#![allow(clippy::useless_conversion)] // Documentation is done in the pin-init crate instead. #![allow(missing_docs)] use proc_macro::TokenStream ; -#[cfg(kernel)] -#[path = "../../../macros/quote.rs" ] -#[macro_use] -#[cfg_attr(not(kernel), rustfmt::skip)] -mod quote; -#[cfg(not(kernel))] -#[macro_use] -extern crate quote; -mod helpers;mod pin_data;mod pinned_drop;
rust/pin-init/internal/src/pin_data.rs +6 -12
@@ -1,10 +1,8 @@// SPDX-License-Identifier: Apache-2.0 OR MIT -#[cfg(not(kernel))] -use proc_macro2as proc_macro; -use crate ::helpers::{parse_generics,Generics }; -use proc_macro::{Group ,Punct ,Spacing ,TokenStream ,TokenTree }; +use proc_macro2::{Group ,Punct ,Spacing ,TokenStream ,TokenTree }; +use quote::quote;pub (crate )fn pin_data (args:TokenStream , input:TokenStream ) ->TokenStream {// This proc-macro only does some pre-parsing and then delegates the actual parsing to
@@ -28,7 +26,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream {// The name of the struct with ty_generics. let struct_name = rest .iter () - .skip_while (|tt| !matches! (tt,TokenTree ::Ident (i)if i.to_string () =="struct" )) + .skip_while (|tt| !matches! (tt,TokenTree ::Ident (i)if i =="struct" )) .nth (1 ) .and_then(|tt| match tt { TokenTree::Ident(_) => {
@@ -65,7 +63,7 @@ pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream { .into_iter() .flat_map (|tt| {// We ignore top level `struct` tokens, since they would emit a compile error. -if matches! (&tt,TokenTree ::Ident (i)if i.to_string () =="struct" ) { +if matches! (&tt,TokenTree ::Ident (i)if i =="struct" ) {vec! [tt] }else {replace_self_and_deny_type_defs (&struct_name, tt, &mut errs)
@@ -98,11 +96,7 @@ fn replace_self_and_deny_type_defs( ) -> Vec<TokenTree> { match tt { TokenTree::Ident(ref i) - if i.to_string() == "enum" - || i.to_string() == "trait" - || i.to_string() == "struct" - || i.to_string() == "union" - || i.to_string() == "impl" => + if i == "enum" || i == "trait" || i == "struct" || i == "union" || i == "impl" => { errs.extend( format!(
@@ -119,7 +113,7 @@ fn replace_self_and_deny_type_defs( );vec! [tt] } - TokenTree::Ident(i) if i.to_string() == "Self" => struct_name.clone(), + TokenTree::Ident(i) if i == "Self" => struct_name.clone(), TokenTree::Literal(_) | TokenTree::Punct(_) | TokenTree::Ident(_) => vec![tt], TokenTree::Group(g) => vec![TokenTree::Group(Group::new( g.delimiter(),
rust/pin-init/internal/src/pinned_drop.rs +4 -6
@@ -1,15 +1,13 @@// SPDX-License-Identifier: Apache-2.0 OR MIT -#[cfg(not(kernel))] -use proc_macro2as proc_macro; - -use proc_macro::{TokenStream ,TokenTree }; +use proc_macro2::{TokenStream ,TokenTree }; +use quote::quote;pub (crate )fn pinned_drop (_args:TokenStream , input:TokenStream ) ->TokenStream {let mut toks = input.into_iter ().collect ::<Vec <_ >>();assert! (!toks.is_empty ());// Ensure that we have an `impl` item. -assert! (matches! (&toks[0 ],TokenTree ::Ident (i)if i.to_string () =="impl" )); +assert! (matches! (&toks[0 ],TokenTree ::Ident (i)if i =="impl" ));// Ensure that we are implementing `PinnedDrop`. let mut nesting:usize =0 ;let mut pinned_drop_idx =None ;
@@ -27,7 +25,7 @@ pub(crate) fn pinned_drop(_args: TokenStream, input: TokenStream) -> TokenStreamif i >=1 && nesting ==0 {// Found the end of the generics, this should be `PinnedDrop`. assert! ( -matches! (tt,TokenTree ::Ident (i)if i.to_string () =="PinnedDrop" ), +matches! (tt,TokenTree ::Ident (i)if i =="PinnedDrop" ),"expected 'PinnedDrop', found: '{tt:?}'" ); pinned_drop_idx =Some (i);
rust/pin-init/internal/src/zeroable.rs +2 -4
@@ -1,10 +1,8 @@// SPDX-License-Identifier: GPL-2.0 -#[cfg(not(kernel))] -use proc_macro2as proc_macro; -use crate ::helpers::{parse_generics,Generics }; -use proc_macro::{TokenStream ,TokenTree }; +use proc_macro2::{TokenStream ,TokenTree }; +use quote::quote;pub (crate )fn parse_zeroable_derive_input ( input: TokenStream,
scripts/generate_rust_analyzer.py +1 -1
@@ -123,7 +123,7 @@ def generate_crates(srctree, objtree, sysroot_src, external_src, cfgs, core_edit append_crate( "pin_init_internal", srctree / "rust" / "pin-init" / "internal" / "src" / "lib.rs", - [], + ["std", "proc_macro", "proc_macro2", "quote", "syn"], cfg=["kernel"], is_proc_macro=True, )
Signature · 1 line
2.51.2
PATCH v3 2/12 rust: macros: use `quote!` from vendored crate
In reply to Gary GuoFrom: Gary Guo <[email protected]>
With quote crate now vendored in the kernel, we can remove our custom quote! macro implementation and just rely on that crate instead.
The quote crate uses types from the proc-macro2 library so we also update to use that, and perform conversion in the top-level lib.rs.
Clippy complains about unnecessary .to_string() as proc-macro2 provides additional PartialEq impl, so they are removed.
--- rust/macros/concat_idents.rs | 2 +- rust/macros/export.rs | 4 +- rust/macros/fmt.rs | 4 +- rust/macros/helpers.rs | 4 +- rust/macros/kunit.rs | 5 +- rust/macros/lib.rs | 21 ++-- rust/macros/module.rs | 6 +- rust/macros/paste.rs | 2 +- rust/macros/quote.rs | 182 ----------------------------------- rust/macros/vtable.rs | 7 +- 10 files changed, 32 insertions(+), 205 deletions(-) delete mode 100644 rust/macros/quote.rs
rust/macros/concat_idents.rs +1 -1
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{token_stream,Ident ,TokenStream ,TokenTree }; +use proc_macro2::{token_stream,Ident ,TokenStream ,TokenTree };use crate ::helpers::expect_punct;
rust/macros/export.rs +3 -1
@@ -1,7 +1,9 @@// SPDX-License-Identifier: GPL-2.0 +use proc_macro2::TokenStream ; +use quote::quote; +use crate ::helpers::function_name; -use proc_macro::TokenStream ;/// Please see [`crate::export`] for documentation. pub (crate )fn export (_attr:TokenStream , ts:TokenStream ) ->TokenStream {
rust/macros/fmt.rs +3 -1
@@ -1,8 +1,10 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{Ident ,TokenStream ,TokenTree };use std::collections::BTreeSet ; +use proc_macro2::{Ident ,TokenStream ,TokenTree }; +use quote::quote_spanned; +/// Please see [`crate::fmt`] for documentation. pub (crate )fn fmt (input:TokenStream ) ->TokenStream {let mut input = input.into_iter ();
rust/macros/helpers.rs +2 -2
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{token_stream,Group ,Ident ,TokenStream ,TokenTree }; +use proc_macro2::{token_stream,Group ,Ident ,TokenStream ,TokenTree };pub (crate )fn try_ident (it: &mut token_stream::IntoIter ) ->Option <String > {if let Some (TokenTree ::Ident (ident)) = it.next () {
@@ -86,7 +86,7 @@ pub(crate) fn function_name(input: TokenStream) -> Option<Ident> {let mut input = input.into_iter ();while let Some (token) = input.next () {match token { -TokenTree ::Ident (i)if i.to_string () =="fn" => { +TokenTree ::Ident (i)if i =="fn" => {if let Some (TokenTree ::Ident (i)) = input.next () {return Some (i); }
rust/macros/kunit.rs +3 -2
@@ -4,10 +4,11 @@//! //! Copyright (c) 2023 José Expósito <[email protected]> -use proc_macro::{Delimiter ,Group ,TokenStream ,TokenTree };use std::collections::HashMap ;use std::fmt::Write ; +use proc_macro2::{Delimiter ,Group ,TokenStream ,TokenTree }; +pub (crate )fn kunit_tests (attr:TokenStream , ts:TokenStream ) ->TokenStream {let attr = attr.to_string ();
@@ -59,7 +60,7 @@ pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream { } _ => (), }, - TokenTree::Ident(i) if i.to_string() == "fn" && attributes.contains_key("test") => { + TokenTree::Ident(i) if i == "fn" && attributes.contains_key("test") => {if let Some (TokenTree ::Ident (test_name)) = body_it.next () { tests.push ((test_name, attributes.remove ("cfg" ).unwrap_or_default ())) }
rust/macros/lib.rs +11 -10
@@ -11,8 +11,6 @@// to avoid depending on the full `proc_macro_span` on Rust >= 1.88.0. #![cfg_attr(not( CONFIG_RUSTC_HAS_SPAN_FILE ), feature(proc_macro_span))] -#[macro_use] -mod quote;mod concat_idents;mod export;mod fmt;
@@ -132,7 +130,7 @@/// the kernel module. #[proc_macro] pub fn module (ts:TokenStream ) ->TokenStream { - module::module (ts) + module::module (ts.into ()).into () }/// Declares or implements a vtable trait.
@@ -207,7 +205,7 @@ pub fn module(ts: TokenStream) -> TokenStream {/// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html #[proc_macro_attribute] pub fn vtable (attr:TokenStream , ts:TokenStream ) ->TokenStream { - vtable::vtable (attr, ts) + vtable::vtable (attr.into (), ts.into ()).into () }/// Export a function so that C code can call it via a header file.
@@ -230,7 +228,7 @@ pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream {/// automatically exported with `EXPORT_SYMBOL_GPL`. #[proc_macro_attribute] pub fn export (attr:TokenStream , ts:TokenStream ) ->TokenStream { - export::export (attr, ts) + export::export (attr.into (), ts.into ()).into () }/// Like [`core::format_args!`], but automatically wraps arguments in [`kernel::fmt::Adapter`].
@@ -248,7 +246,7 @@ pub fn export(attr: TokenStream, ts: TokenStream) -> TokenStream {/// [`pr_info!`]: ../kernel/macro.pr_info.html #[proc_macro] pub fn fmt (input:TokenStream ) ->TokenStream { - fmt::fmt (input) + fmt::fmt (input.into ()).into () }/// Concatenate two identifiers.
@@ -306,7 +304,7 @@ pub fn fmt(input: TokenStream) -> TokenStream {/// ``` #[proc_macro] pub fn concat_idents (ts:TokenStream ) ->TokenStream { - concat_idents::concat_idents (ts) + concat_idents::concat_idents (ts.into ()).into () }/// Paste identifiers together.
@@ -444,9 +442,12 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// [`paste`]: https://docs.rs/paste/ #[proc_macro] pub fn paste (input:TokenStream ) ->TokenStream { -let mut tokens = input.into_iter ().collect (); +let mut tokens = proc_macro2::TokenStream ::from (input).into_iter ().collect (); paste::expand (&mut tokens); - tokens.into_iter ().collect () + tokens + .into_iter () + .collect ::<proc_macro2::TokenStream >() + .into () }/// Registers a KUnit test suite and its test cases using a user-space like syntax.
@@ -473,5 +474,5 @@ pub fn paste(input: TokenStream) -> TokenStream {/// ``` #[proc_macro_attribute] pub fn kunit_tests (attr:TokenStream , ts:TokenStream ) ->TokenStream { - kunit::kunit_tests (attr, ts) + kunit::kunit_tests (attr.into (), ts.into ()).into () }
rust/macros/module.rs +4 -2
@@ -1,9 +1,11 @@// SPDX-License-Identifier: GPL-2.0 -use crate ::helpers::*; -use proc_macro::{token_stream,Delimiter ,Literal ,TokenStream ,TokenTree };use std::fmt::Write ; +use proc_macro2::{token_stream,Delimiter ,Literal ,TokenStream ,TokenTree }; + +use crate ::helpers::*; +fn expect_string_array (it: &mut token_stream::IntoIter ) ->Vec <String > {let group =expect_group (it);assert_eq! (group.delimiter (),Delimiter ::Bracket );
rust/macros/paste.rs +1 -1
@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{Delimiter ,Group ,Ident ,Spacing ,Span ,TokenTree }; +use proc_macro2::{Delimiter ,Group ,Ident ,Spacing ,Span ,TokenTree };fn concat_helper (tokens: &[TokenTree ]) ->Vec <(String ,Span )> {let mut tokens = tokens.iter ();
rust/macros/quote.rs deleted +0 -182
182 lines not on this page · raw message
rust/macros/vtable.rs +4 -3
@@ -1,9 +1,10 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{Delimiter ,Group ,TokenStream ,TokenTree };use std::collections::HashSet ;use std::fmt::Write ; +use proc_macro2::{Delimiter ,Group ,TokenStream ,TokenTree }; +pub (crate )fn vtable (_attr:TokenStream , ts:TokenStream ) ->TokenStream {let mut tokens:Vec <_ > = ts.into_iter ().collect ();
@@ -31,7 +32,7 @@ pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream {let mut consts =HashSet ::new ();while let Some (token) = body_it.next () {match token { -TokenTree ::Ident (ident)if ident.to_string () =="fn" => { +TokenTree ::Ident (ident)if ident =="fn" => {let fn_name =match body_it.next () {Some (TokenTree ::Ident (ident)) => ident.to_string (), // Possibly we've encountered a fn pointer type instead.
@@ -39,7 +40,7 @@ pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream { }; functions.push (fn_name); } - TokenTree::Ident(ident) if ident.to_string() == "const" => { + TokenTree::Ident(ident) if ident == "const" => { let const_name = match body_it.next() { Some(TokenTree::Ident(ident)) => ident.to_string(), // Possibly we've encountered an inline const block instead.
Signature · 1 line
2.51.2
PATCH v3 3/12 rust: macros: convert `#[vtable]` macro to use `syn`
In reply to Gary GuoFrom: Gary Guo <[email protected]>
#[vtable] is converted to use syn. This is more robust than the previous heuristic-based searching of defined methods and functions.
When doing so, the trait and impl are split into two code paths as the types are distinct when parsed by syn.
--- rust/macros/lib.rs | 9 ++- rust/macros/vtable.rs | 169 +++++++++++++++++++++++------------------- 2 files changed, 98 insertions(+), 80 deletions(-)
rust/macros/lib.rs +7 -2
@@ -22,6 +22,8 @@use proc_macro::TokenStream ; +use syn::parse_macro_input; +/// Declares a kernel module. /// /// The `type` argument should be a type which implements the [`Module`]
@@ -204,8 +206,11 @@ pub fn module(ts: TokenStream) -> TokenStream {/// /// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html #[proc_macro_attribute] -pub fn vtable (attr:TokenStream , ts:TokenStream ) ->TokenStream { - vtable::vtable (attr.into (), ts.into ()).into () +pub fn vtable (attr:TokenStream , input:TokenStream ) ->TokenStream { +parse_macro_input! (attras syn::parse::Nothing ); + vtable::vtable (parse_macro_input! (input)) + .unwrap_or_else (|e| e.into_compile_error ()) + .into () }/// Export a function so that C code can call it via a header file.
rust/macros/vtable.rs +91 -78
188 lines not on this page · raw message
Signature · 1 line
2.51.2
PATCH v3 4/12 rust: macros: use `syn` to parse `module!` macro
In reply to Gary GuoFrom: Gary Guo <[email protected]>
With syn being available in the kernel, use it to parse the complex custom module! macro to replace existing helpers. Only parsing is changed in this commit, the code generation is untouched.
This has the benefit of better error message when the macro is used incorrectly, as it can point to a concrete span on what's going wrong.
For example, if a field is specified twice, previously it reads:
error: proc macro panicked
--> samples/rust/rust_minimal.rs:7:1
|
7 | / module! {
8 | | type: RustMinimal,
9 | | name: "rust_minimal",
10 | | author: "Rust for Linux Contributors",
11 | | description: "Rust minimal sample",
12 | | license: "GPL",
13 | | license: "GPL",
14 | | }
| |_^
|
= help: message: Duplicated key "license". Keys can only be specified once.now it reads:
error: duplicated key "license". Keys can only be specified once.
--> samples/rust/rust_minimal.rs:13:5
|
13 | license: "GPL",
| ^^^^^^^--- rust/macros/helpers.rs | 109 ++++------- rust/macros/lib.rs | 6 +- rust/macros/module.rs | 399 +++++++++++++++++++++++++---------------- 3 files changed, 280 insertions(+), 234 deletions(-)
rust/macros/helpers.rs +32 -77
@@ -1,53 +1,21 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro2::{token_stream,Group ,Ident ,TokenStream ,TokenTree }; - -pub (crate )fn try_ident (it: &mut token_stream::IntoIter ) ->Option <String > { -if let Some (TokenTree ::Ident (ident)) = it.next () { -Some (ident.to_string ()) - }else { -None - } -} - -pub (crate )fn try_sign (it: &mut token_stream::IntoIter ) ->Option <char > { -let peek = it.clone ().next (); -match peek { -Some (TokenTree ::Punct (punct))if punct.as_char () =='-' => { -let _ = it.next (); -Some (punct.as_char ()) - } - _ =>None , - } -} - -pub (crate )fn try_literal (it: &mut token_stream::IntoIter ) ->Option <String > { -if let Some (TokenTree ::Literal (literal)) = it.next () { -Some (literal.to_string ()) - }else { -None - } -} - -pub (crate )fn try_string (it: &mut token_stream::IntoIter ) ->Option <String > { -try_literal (it).and_then (|string| { -if string.starts_with ('\"' ) && string.ends_with ('\"' ) { -let content = &string[1 ..string.len () -1 ]; -if content.contains ('\\' ) { -panic! ("Escape sequences in string literals not yet handled" ); - } -Some (content.to_string ()) - }else if string.starts_with ("r\"" ) { -panic! ("Raw string literals are not yet handled" ); - }else { -None - } - }) -} - -pub (crate )fn expect_ident (it: &mut token_stream::IntoIter ) ->String { -try_ident (it).expect ("Expected Ident" ) -} +use proc_macro2::{ + token_stream, +Ident , +TokenStream , +TokenTree ,// +}; +use quote::ToTokens ; +use syn::{ + parse::{ +Parse , +ParseStream ,// + }, +Error , +LitStr , +Result ,// +};pub (crate )fn expect_punct (it: &mut token_stream::IntoIter ) ->char {if let TokenTree ::Punct (punct) = it.next ().expect ("Reached end of token stream for Punct" ) {
@@ -57,27 +25,28 @@ pub(crate) fn expect_punct(it: &mut token_stream::IntoIter) -> char { } } -pub (crate )fn expect_string (it: &mut token_stream::IntoIter ) ->String { -try_string (it).expect ("Expected string" ) -} +/// A string literal that is required to have ASCII value only. +pub (crate )struct AsciiLitStr (LitStr ); -pub (crate )fn expect_string_ascii (it: &mut token_stream::IntoIter ) ->String { -let string =try_string (it).expect ("Expected string" ); -assert! (string.is_ascii (),"Expected ASCII string" ); - string +impl Parse for AsciiLitStr { +fn parse (input:ParseStream <'_>) ->Result <Self > { +let s:LitStr = input.parse ()?; +if !s.value ().is_ascii () { +return Err (Error ::new_spanned (s,"expected ASCII-only string literal" )); + } +Ok (Self (s)) + } } -pub (crate )fn expect_group (it: &mut token_stream::IntoIter ) ->Group { -if let TokenTree ::Group (group) = it.next ().expect ("Reached end of token stream for Group" ) { - group - }else { -panic! ("Expected Group" ); +impl ToTokens for AsciiLitStr { +fn to_tokens (&self, ts: &mut TokenStream ) { + self.0 .to_tokens (ts); } } -pub (crate )fn expect_end (it: &mut token_stream::IntoIter ) { -if it.next ().is_some () { -panic! ("Expected end" ); +impl AsciiLitStr { +pub (crate )fn value (&self) ->String { + self.0 .value () } }
@@ -114,17 +83,3 @@ pub(crate) fn file() -> String { proc_macro::Span ::call_site ().file () } } - -/// Parse a token stream of the form `expected_name: "value",` and return the -/// string in the position of "value". -/// -/// # Panics -/// -/// - On parse error. -pub (crate )fn expect_string_field (it: &mut token_stream::IntoIter , expected_name: &str ) ->String { -assert_eq! (expect_ident (it), expected_name); -assert_eq! (expect_punct (it),':' ); -let string =expect_string (it); -assert_eq! (expect_punct (it),',' ); - string -}
rust/macros/lib.rs +4 -2
@@ -131,8 +131,10 @@/// - `firmware`: array of ASCII string literals of the firmware files of /// the kernel module. #[proc_macro] -pub fn module (ts:TokenStream ) ->TokenStream { - module::module (ts.into ()).into () +pub fn module (input:TokenStream ) ->TokenStream { + module::module (parse_macro_input! (input)) + .unwrap_or_else (|e| e.into_compile_error ()) + .into () }/// Declares or implements a vtable trait.
rust/macros/module.rs +244 -155
508 lines not on this page · raw message
Signature · 1 line
2.51.2
PATCH v3 5/12 rust: macros: use `quote!` for `module!` macro
In reply to Gary GuoFrom: Gary Guo <[email protected]>
This has no behavioural change, but is good for maintainability. With quote!, we're no longer using string templates, so we don't need to quote " and {} inside the template anymore. Further more, editors can now highlight the code template.
This also improves the robustness as it eliminates the need for string quoting and escaping.
--- rust/macros/module.rs | 543 ++++++++++++++++++++++-------------------- 1 file changed, 286 insertions(+), 257 deletions(-)
rust/macros/module.rs +286 -257
@@ -1,12 +1,15 @@// SPDX-License-Identifier: GPL-2.0 -use std::fmt::Write ; +use std::ffi::CString ;use proc_macro2::{Literal ,TokenStream ,// }; -use quote::ToTokens ; +use quote::{ + format_ident, + quote,// +};use syn::{ braced, bracketed,
@@ -15,11 +18,13 @@ Parse, ParseStream, // }, + parse_quote, punctuated::Punctuated, Error, Expr, Ident, LitStr, + Path, Result, Token, // };
@@ -29,8 +34,8 @@struct ModInfoBuilder <'a> { module: &'astr , counter:usize , - buffer:String , - param_buffer:String , + ts:TokenStream , + param_ts:TokenStream , }impl <'a>ModInfoBuilder <'a> {
@@ -38,8 +43,8 @@ fn new(module: &'a str) -> Self {ModInfoBuilder { module, counter:0 , - buffer:String ::new (), - param_buffer:String ::new (), + ts:TokenStream ::new (), + param_ts:TokenStream ::new (), } }
@@ -56,33 +61,32 @@ fn emit_base(&mut self, field: &str, content: &str, builtin: bool, param: bool)// Loadable modules' modinfo strings go as-is. format! ("{field}={content}\0" ) }; - -let buffer =if param { - &mut self.param_buffer +let length = string.len (); +let string =Literal ::byte_string (string.as_bytes ()); +let cfg =if builtin { +quote! (#[cfg(not( MODULE ))] ) }else { - &mut self.buffer +quote! (#[cfg( MODULE )] ) }; -write! ( - buffer, -" -{cfg} -#[doc(hidden)] -#[cfg_attr(not(target_os = \"macos\"), link_section = \".modinfo\")] -#[used(compiler)] -pub static __{module}_{counter}: [u8; {length}] = *{string}; -" , - cfg =if builtin { -"#[cfg(not(MODULE))]" - }else { -"#[cfg(MODULE)]" - }, +let counter =format_ident! ( +"__{module}_{counter}" , module = self.module.to_uppercase (), - counter = self.counter, - length = string.len (), - string =Literal ::byte_string (string.as_bytes ()), - ) - .unwrap (); + counter = self.counter + ); +let item =quote! { + #cfg +#[doc(hidden)] +#[cfg_attr(not(target_os = "macos" ), link_section = ".modinfo" )] +#[used(compiler)] +pub static #counter: [u8 ; #length] = *#string; + }; + +if param { + self.param_ts.extend (item); + }else { + self.ts.extend (item); + } self.counter +=1 ; }
@@ -117,7 +121,6 @@ fn emit_params(&mut self, info: &ModuleInfo) {for paramin params {let param_name_str = param.name.to_string ();let param_type_str = param.ptype.to_string (); -let param_default = param.default.to_token_stream ().to_string ();let ops =param_ops_path (¶m_type_str);
@@ -126,66 +129,74 @@ fn emit_params(&mut self, info: &ModuleInfo) { self.emit_param ("parmtype" , ¶m_name_str, ¶m_type_str); self.emit_param ("parm" , ¶m_name_str, ¶m.description.value ()); -write! ( - self.param_buffer, -" -pub(crate) static {param_name_str}: -::kernel::module_param::ModuleParamAccess<{param_type_str}> = -::kernel::module_param::ModuleParamAccess::new({param_default}); - -const _: () = {{ -#[link_section = \"__param\"] -#[used] -static __{module_name}_{param_name_str}_struct: +let static_name =format_ident! ("__{}_{}_struct" , self.module, param.name); +let param_name_cstr = +CString ::new (param_name_str).expect ("name contains NUL-terminator" ); +let param_name_cstr_with_module = +CString ::new (format! ("{}.{}" , self.module, param.name)) + .expect ("name contains NUL-terminator" ); + +let param_name = ¶m.name; +let param_type = ¶m.ptype; +let param_default = ¶m.default; + + self.param_ts.extend (quote! { +#[allow(non_upper_case_globals)] +pub (crate )static #param_name: + ::kernel::module_param::ModuleParamAccess <#param_type > = + ::kernel::module_param::ModuleParamAccess ::new (#param_default); + +const _: () = { +#[allow(non_upper_case_globals)] +#[link_section = "__param" ] +#[used(compiler)] +static #static_name: ::kernel::module_param::KernelParam = ::kernel::module_param::KernelParam ::new ( -::kernel::bindings::kernel_param {{ -name: if ::core::cfg!(MODULE) {{ -::kernel::c_str!(\"{param_name_str}\").to_bytes_with_nul() -}} else {{ -::kernel::c_str!(\"{module_name}.{param_name_str}\") -.to_bytes_with_nul() -}}.as_ptr(), + ::kernel::bindings::kernel_param { + name: kernel::str ::as_char_ptr_in_const_context ( +if ::core::cfg!(MODULE ) { + #param_name_cstr + }else { + #param_name_cstr_with_module + } + ),// SAFETY: `__this_module` is constructed by the kernel at load // time and will not be freed until the module is unloaded. #[cfg( MODULE )] -mod_: unsafe {{ + mod_:unsafe { core::ptr::from_ref (&::kernel::bindings::__this_module) .cast_mut () -}}, + },#[cfg(not( MODULE ))] mod_: ::core::ptr::null_mut (), -ops: core::ptr::from_ref(&{ops}), + ops: core::ptr::from_ref (&#ops), perm:0 ,// Will not appear in sysfs level: -1 , flags:0 , -__bindgen_anon_1: ::kernel::bindings::kernel_param__bindgen_ty_1 {{ -arg: {param_name_str}.as_void_ptr() -}}, -}} + __bindgen_anon_1: ::kernel::bindings::kernel_param__bindgen_ty_1 { + arg: #param_name.as_void_ptr () + }, + } ); -}}; -" , - module_name = info.name.value (), - ops = ops, - ) - .unwrap (); + }; + }); } } } -fn param_ops_path (param_type: &str ) -> &'staticstr { +fn param_ops_path (param_type: &str ) ->Path {match param_type { -"i8" =>"::kernel::module_param::PARAM_OPS_I8" , -"u8" =>"::kernel::module_param::PARAM_OPS_U8" , -"i16" =>"::kernel::module_param::PARAM_OPS_I16" , -"u16" =>"::kernel::module_param::PARAM_OPS_U16" , -"i32" =>"::kernel::module_param::PARAM_OPS_I32" , -"u32" =>"::kernel::module_param::PARAM_OPS_U32" , -"i64" =>"::kernel::module_param::PARAM_OPS_I64" , -"u64" =>"::kernel::module_param::PARAM_OPS_U64" , -"isize" =>"::kernel::module_param::PARAM_OPS_ISIZE" , -"usize" =>"::kernel::module_param::PARAM_OPS_USIZE" , +"i8" =>parse_quote! (::kernel::module_param::PARAM_OPS_I8 ), +"u8" =>parse_quote! (::kernel::module_param::PARAM_OPS_U8 ), +"i16" =>parse_quote! (::kernel::module_param::PARAM_OPS_I16 ), +"u16" =>parse_quote! (::kernel::module_param::PARAM_OPS_U16 ), +"i32" =>parse_quote! (::kernel::module_param::PARAM_OPS_I32 ), +"u32" =>parse_quote! (::kernel::module_param::PARAM_OPS_U32 ), +"i64" =>parse_quote! (::kernel::module_param::PARAM_OPS_I64 ), +"u64" =>parse_quote! (::kernel::module_param::PARAM_OPS_U64 ), +"isize" =>parse_quote! (::kernel::module_param::PARAM_OPS_ISIZE ), +"usize" =>parse_quote! (::kernel::module_param::PARAM_OPS_USIZE ), t =>panic! ("Unsupported parameter type {}" , t), } }
@@ -420,29 +431,41 @@ fn parse(input: ParseStream<'_>) -> Result<Self> { }pub (crate )fn module (info:ModuleInfo ) ->Result <TokenStream > { +let ModuleInfo { + type_, + license, + name, + authors, + description, + alias, + firmware, + imports_ns, + params: _, + } = &info; +// Rust does not allow hyphens in identifiers, use underscore instead. -let ident = info.name.value ().replace ('-' ,"_" ); +let ident = name.value ().replace ('-' ,"_" );let mut modinfo =ModInfoBuilder ::new (ident.as_ref ()); -if let Some (authors) = &info.authors { +if let Some (authors) = authors {for authorin authors { modinfo.emit ("author" , &author.value ()); } } -if let Some (description) = &info.description { +if let Some (description) = description { modinfo.emit ("description" , &description.value ()); } - modinfo.emit ("license" , &info.license.value ()); -if let Some (aliases) = &info.alias { + modinfo.emit ("license" , &license.value ()); +if let Some (aliases) = alias {for aliasin aliases { modinfo.emit ("alias" , &alias.value ()); } } -if let Some (firmware) = &info.firmware { +if let Some (firmware) = firmware {for fwin firmware { modinfo.emit ("firmware" , &fw.value ()); } } -if let Some (imports) = &info.imports_ns { +if let Some (imports) = imports_ns {for nsin imports { modinfo.emit ("import_ns" , &ns.value ()); }
@@ -455,182 +478,188 @@ pub(crate) fn module(info: ModuleInfo) -> Result<TokenStream> { modinfo.emit_params (&info); -Ok (format! ( -" -/// The module name. -/// -/// Used by the printing macros, e.g. [`info!`]. -const __LOG_PREFIX: &[u8] = b\"{name}\\0\"; - -// SAFETY: `__this_module` is constructed by the kernel at load time and will not be -// freed until the module is unloaded. -#[cfg(MODULE)] -static THIS_MODULE: ::kernel::ThisModule = unsafe {{ -extern \"C\" {{ -static __this_module: ::kernel::types::Opaque<::kernel::bindings::module>; -}} - -::kernel::ThisModule::from_ptr(__this_module.get()) -}}; -#[cfg(not(MODULE))] -static THIS_MODULE: ::kernel::ThisModule = unsafe {{ -::kernel::ThisModule::from_ptr(::core::ptr::null_mut()) -}}; - -/// The `LocalModule` type is the type of the module created by `module!`, -/// `module_pci_driver!`, `module_platform_driver!`, etc. -type LocalModule = {type_}; - -impl ::kernel::ModuleMetadata for {type_} {{ -const NAME: &'static ::kernel::str::CStr = c\"{name}\"; -}} - -// Double nested modules, since then nobody can access the public items inside. -mod __module_init {{ -mod __module_init {{ -use super::super::{type_}; -use pin_init::PinInit; - -/// The \"Rust loadable module\" mark. -// -// This may be best done another way later on, e.g. as a new modinfo -// key or a new section. For the moment, keep it simple. -#[cfg(MODULE)] -#[doc(hidden)] -#[used(compiler)] -static __IS_RUST_MODULE: () = (); - -static mut __MOD: ::core::mem::MaybeUninit<{type_}> = -::core::mem::MaybeUninit::uninit(); - -// Loadable modules need to export the `{{init,cleanup}}_module` identifiers. -/// # Safety -/// -/// This function must not be called after module initialization, because it may be -/// freed after that completes. -#[cfg(MODULE)] -#[doc(hidden)] -#[no_mangle] -#[link_section = \".init.text\"] -pub unsafe extern \"C\" fn init_module() -> ::kernel::ffi::c_int {{ -// SAFETY: This function is inaccessible to the outside due to the double -// module wrapping it. It is called exactly once by the C side via its -// unique name. -unsafe {{ __init() }} -}} - -#[cfg(MODULE)] -#[doc(hidden)] -#[used(compiler)] -#[link_section = \".init.data\"] -static __UNIQUE_ID___addressable_init_module: unsafe extern \"C\" fn() -> i32 = init_module; - -#[cfg(MODULE)] -#[doc(hidden)] -#[no_mangle] -#[link_section = \".exit.text\"] -pub extern \"C\" fn cleanup_module() {{ -// SAFETY: -// - This function is inaccessible to the outside due to the double -// module wrapping it. It is called exactly once by the C side via its -// unique name, -// - furthermore it is only called after `init_module` has returned `0` -// (which delegates to `__init`). -unsafe {{ __exit() }} -}} - -#[cfg(MODULE)] -#[doc(hidden)] -#[used(compiler)] -#[link_section = \".exit.data\"] -static __UNIQUE_ID___addressable_cleanup_module: extern \"C\" fn() = cleanup_module; - -// Built-in modules are initialized through an initcall pointer -// and the identifiers need to be unique. -#[cfg(not(MODULE))] -#[cfg(not(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS))] -#[doc(hidden)] -#[link_section = \"{initcall_section}\"] -#[used(compiler)] -pub static __{ident}_initcall: extern \"C\" fn() -> -::kernel::ffi::c_int = __{ident}_init; - -#[cfg(not(MODULE))] -#[cfg(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)] -::core::arch::global_asm!( -r#\".section \"{initcall_section}\", \"a\" -__{ident}_initcall: -.long __{ident}_init - . -.previous -\"# -); - -#[cfg(not(MODULE))] -#[doc(hidden)] -#[no_mangle] -pub extern \"C\" fn __{ident}_init() -> ::kernel::ffi::c_int {{ -// SAFETY: This function is inaccessible to the outside due to the double -// module wrapping it. It is called exactly once by the C side via its -// placement above in the initcall section. -unsafe {{ __init() }} -}} - -#[cfg(not(MODULE))] -#[doc(hidden)] -#[no_mangle] -pub extern \"C\" fn __{ident}_exit() {{ -// SAFETY: -// - This function is inaccessible to the outside due to the double -// module wrapping it. It is called exactly once by the C side via its -// unique name, -// - furthermore it is only called after `__{ident}_init` has -// returned `0` (which delegates to `__init`). -unsafe {{ __exit() }} -}} - -/// # Safety -/// -/// This function must only be called once. -unsafe fn __init() -> ::kernel::ffi::c_int {{ -let initer = -<{type_} as ::kernel::InPlaceModule>::init(&super::super::THIS_MODULE); -// SAFETY: No data race, since `__MOD` can only be accessed by this module -// and there only `__init` and `__exit` access it. These functions are only -// called once and `__exit` cannot be called before or during `__init`. -match unsafe {{ initer.__pinned_init(__MOD.as_mut_ptr()) }} {{ -Ok(m) => 0, -Err(e) => e.to_errno(), -}} -}} - -/// # Safety -/// -/// This function must -/// - only be called once, -/// - be called after `__init` has been called and returned `0`. -unsafe fn __exit() {{ -// SAFETY: No data race, since `__MOD` can only be accessed by this module -// and there only `__init` and `__exit` access it. These functions are only -// called once and `__init` was already called. -unsafe {{ -// Invokes `drop()` on `__MOD`, which should be used for cleanup. -__MOD.assume_init_drop(); -}} -}} -{modinfo} -}} -}} -mod module_parameters {{ -{params} -}} -" , - type_ = info.type_, - name = info.name.value (), - ident = ident, - modinfo = modinfo.buffer, - params = modinfo.param_buffer, - initcall_section =".initcall6.init" - ) - .parse () - .expect ("Error parsing formatted string into token stream." )) +let modinfo_ts = modinfo.ts; +let params_ts = modinfo.param_ts; + +let ident_init =format_ident! ("__{ident}_init" ); +let ident_exit =format_ident! ("__{ident}_exit" ); +let ident_initcall =format_ident! ("__{ident}_initcall" ); +let initcall_section =".initcall6.init" ; + +let global_asm =format! ( +r#".section "{initcall_section}", "a" +__{ident}_initcall: +.long __{ident}_init - . +.previous +"# + ); + +let name_cstr =CString ::new (name.value ()).expect ("name contains NUL-terminator" ); + +Ok (quote! { +/// The module name. +/// +/// Used by the printing macros, e.g. [`info!`]. +const __LOG_PREFIX: &[u8 ] = #name_cstr.to_bytes_with_nul (); + +// SAFETY: `__this_module` is constructed by the kernel at load time and will not be +// freed until the module is unloaded. +#[cfg( MODULE )] +static THIS_MODULE : ::kernel::ThisModule =unsafe { +extern "C" { +static __this_module: ::kernel::types::Opaque <::kernel::bindings::module >; + }; + + ::kernel::ThisModule ::from_ptr (__this_module.get ()) + }; + +#[cfg(not( MODULE ))] +static THIS_MODULE : ::kernel::ThisModule =unsafe { + ::kernel::ThisModule ::from_ptr (::core::ptr::null_mut ()) + }; + +/// The `LocalModule` type is the type of the module created by `module!`, +/// `module_pci_driver!`, `module_platform_driver!`, etc. +type LocalModule = #type_ ; + +impl ::kernel::ModuleMetadata for #type_ { +const NAME : &'static ::kernel::str ::CStr = #name_cstr; + } + +// Double nested modules, since then nobody can access the public items inside. +mod __module_init { +mod __module_init { +use super ::super ::#type_; +use pin_init::PinInit ; + +/// The "Rust loadable module" mark. +// +// This may be best done another way later on, e.g. as a new modinfo +// key or a new section. For the moment, keep it simple. +#[cfg( MODULE )] +#[doc(hidden)] +#[used(compiler)] +static __IS_RUST_MODULE: () = (); + +static mut __MOD: ::core::mem::MaybeUninit <#type_ > = + ::core::mem::MaybeUninit ::uninit (); + +// Loadable modules need to export the `{init,cleanup}_module` identifiers. +/// # Safety +/// +/// This function must not be called after module initialization, because it may be +/// freed after that completes. +#[cfg( MODULE )] +#[doc(hidden)] +#[no_mangle] +#[link_section = ".init.text" ] +pub unsafe extern "C" fn init_module () -> ::kernel::ffi::c_int { +// SAFETY: This function is inaccessible to the outside due to the double +// module wrapping it. It is called exactly once by the C side via its +// unique name. +unsafe {__init () } + } + +#[cfg( MODULE )] +#[doc(hidden)] +#[used(compiler)] +#[link_section = ".init.data" ] +static __UNIQUE_ID___addressable_init_module:unsafe extern "C" fn () ->i32 = + init_module; + +#[cfg( MODULE )] +#[doc(hidden)] +#[no_mangle] +#[link_section = ".exit.text" ] +pub extern "C" fn cleanup_module () { +// SAFETY: +// - This function is inaccessible to the outside due to the double +// module wrapping it. It is called exactly once by the C side via its +// unique name, +// - furthermore it is only called after `init_module` has returned `0` +// (which delegates to `__init`). +unsafe {__exit () } + } + +#[cfg( MODULE )] +#[doc(hidden)] +#[used(compiler)] +#[link_section = ".exit.data" ] +static __UNIQUE_ID___addressable_cleanup_module:extern "C" fn () = cleanup_module; + +// Built-in modules are initialized through an initcall pointer +// and the identifiers need to be unique. +#[cfg(not( MODULE ))] +#[cfg(not( CONFIG_HAVE_ARCH_PREL32_RELOCATIONS ))] +#[doc(hidden)] +#[link_section = #initcall_section] +#[used(compiler)] +pub static #ident_initcall:extern "C" fn () -> + ::kernel::ffi::c_int = #ident_init; + +#[cfg(not( MODULE ))] +#[cfg( CONFIG_HAVE_ARCH_PREL32_RELOCATIONS )] + ::core::arch::global_asm!(#global_asm); + +#[cfg(not( MODULE ))] +#[doc(hidden)] +#[no_mangle] +pub extern "C" fn #ident_init () -> ::kernel::ffi::c_int { +// SAFETY: This function is inaccessible to the outside due to the double +// module wrapping it. It is called exactly once by the C side via its +// placement above in the initcall section. +unsafe {__init () } + } + +#[cfg(not( MODULE ))] +#[doc(hidden)] +#[no_mangle] +pub extern "C" fn #ident_exit () { +// SAFETY: +// - This function is inaccessible to the outside due to the double +// module wrapping it. It is called exactly once by the C side via its +// unique name, +// - furthermore it is only called after `#ident_init` has +// returned `0` (which delegates to `__init`). +unsafe {__exit () } + } + +/// # Safety +/// +/// This function must only be called once. +unsafe fn __init () -> ::kernel::ffi::c_int { +let initer = + <#type_ as ::kernel::InPlaceModule >::init (&super ::super ::THIS_MODULE ); +// SAFETY: No data race, since `__MOD` can only be accessed by this module +// and there only `__init` and `__exit` access it. These functions are only +// called once and `__exit` cannot be called before or during `__init`. +match unsafe { initer.__pinned_init (__MOD.as_mut_ptr ()) } { +Ok (m) =>0 , +Err (e) => e.to_errno (), + } + } + +/// # Safety +/// +/// This function must +/// - only be called once, +/// - be called after `__init` has been called and returned `0`. +unsafe fn __exit () { +// SAFETY: No data race, since `__MOD` can only be accessed by this module +// and there only `__init` and `__exit` access it. These functions are only +// called once and `__init` was already called. +unsafe { +// Invokes `drop()` on `__MOD`, which should be used for cleanup. + __MOD.assume_init_drop (); + } + } + + #modinfo_ts + } + } + +mod module_parameters { + #params_ts + } + }) }
Signature · 1 line
2.51.2
PATCH v3 6/12 rust: macros: convert `#[export]` to use `syn`
In reply to Gary GuoFrom: Gary Guo <[email protected]>
This eliminates the custom function_name helper.
--- rust/macros/export.rs | 24 +++++++++--------------- rust/macros/helpers.rs | 18 ------------------ rust/macros/lib.rs | 5 +++-- 3 files changed, 12 insertions(+), 35 deletions(-)
rust/macros/export.rs +9 -15
@@ -3,19 +3,14 @@use proc_macro2::TokenStream ;use quote::quote; -use crate ::helpers::function_name; -/// Please see [`crate::export`] for documentation. -pub (crate )fn export (_attr:TokenStream , ts:TokenStream ) ->TokenStream { -let Some (name) =function_name (ts.clone ())else { -return "::core::compile_error!(\"The #[export] attribute must be used on a function.\");" - .parse ::<TokenStream >() - .unwrap (); - }; +pub (crate )fn export (f: syn::ItemFn ) ->TokenStream { +let name = &f.sig.ident; -// This verifies that the function has the same signature as the declaration generated by -// bindgen. It makes use of the fact that all branches of an if/else must have the same type. - let signature_check = quote!( +quote! { +// This verifies that the function has the same signature as the declaration generated by +// bindgen. It makes use of the fact that all branches of an if/else must have the same +// type. const _: () = {if true { ::kernel::bindings::#name
@@ -23,9 +18,8 @@ pub(crate) fn export(_attr: TokenStream, ts: TokenStream) -> TokenStream { #name }; }; - ); - -let no_mangle =quote! (#[no_mangle] ); -TokenStream ::from_iter ([signature_check, no_mangle, ts]) +#[no_mangle] + #f + } }
rust/macros/helpers.rs +0 -18
@@ -2,7 +2,6 @@use proc_macro2::{ token_stream, -Ident ,TokenStream ,TokenTree ,// };
@@ -50,23 +49,6 @@ pub(crate) fn value(&self) -> String { } } -/// Given a function declaration, finds the name of the function. -pub (crate )fn function_name (input:TokenStream ) ->Option <Ident > { -let mut input = input.into_iter (); -while let Some (token) = input.next () { -match token { -TokenTree ::Ident (i)if i =="fn" => { -if let Some (TokenTree ::Ident (i)) = input.next () { -return Some (i); - } -return None ; - } - _ =>continue , - } - } -None -} -pub (crate )fn file () ->String {#[cfg(not( CONFIG_RUSTC_HAS_SPAN_FILE ))] {
rust/macros/lib.rs +3 -2
@@ -234,8 +234,9 @@ pub fn vtable(attr: TokenStream, input: TokenStream) -> TokenStream {/// This macro is *not* the same as the C macros `EXPORT_SYMBOL_*`. All Rust symbols are currently /// automatically exported with `EXPORT_SYMBOL_GPL`. #[proc_macro_attribute] -pub fn export (attr:TokenStream , ts:TokenStream ) ->TokenStream { - export::export (attr.into (), ts.into ()).into () +pub fn export (attr:TokenStream , input:TokenStream ) ->TokenStream { +parse_macro_input! (attras syn::parse::Nothing ); + export::export (parse_macro_input! (input)).into () }/// Like [`core::format_args!`], but automatically wraps arguments in [`kernel::fmt::Adapter`].
Signature · 1 line
2.51.2
PATCH v3 7/12 rust: macros: convert `concat_idents!` to use `syn`
In reply to Gary GuoFrom: Gary Guo <[email protected]>
This eliminates the need for expect_punct helper.
--- rust/macros/concat_idents.rs | 39 ++++++++++++++++++++++++------------ rust/macros/helpers.rs | 14 +------------ rust/macros/lib.rs | 4 ++-- 3 files changed, 29 insertions(+), 28 deletions(-)
rust/macros/concat_idents.rs +26 -13
@@ -1,23 +1,36 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro2::{token_stream,Ident ,TokenStream ,TokenTree }; +use proc_macro2::{ +Ident , +TokenStream , +TokenTree ,// +}; +use syn::{ + parse::{ +Parse , +ParseStream ,// + }, +Result , +Token ,// +}; -use crate ::helpers::expect_punct; +pub (crate )struct Input { + a:Ident , + _comma:Token! [,], + b:Ident , +} -fn expect_ident (it: &mut token_stream::IntoIter ) ->Ident { -if let Some (TokenTree ::Ident (ident)) = it.next () { - ident - }else { -panic! ("Expected Ident" ) +impl Parse for Input { +fn parse (input:ParseStream <'_>) ->Result <Self > { +Ok (Self { + a: input.parse ()?, + _comma: input.parse ()?, + b: input.parse ()?, + }) } } -pub (crate )fn concat_idents (ts:TokenStream ) ->TokenStream { -let mut it = ts.into_iter (); -let a =expect_ident (&mut it); -assert_eq! (expect_punct (&mut it),',' ); -let b =expect_ident (&mut it); -assert! (it.next ().is_none (),"only two idents can be concatenated" ); +pub (crate )fn concat_idents (Input { a, b, .. }:Input ) ->TokenStream {let res =Ident ::new (&format! ("{a}{b}" ), b.span ());TokenStream ::from_iter ([TokenTree ::Ident (res)]) }
rust/macros/helpers.rs +1 -13
@@ -1,10 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro2::{ - token_stream, -TokenStream , -TokenTree ,// -}; +use proc_macro2::TokenStream ;use quote::ToTokens ;use syn::{ parse::{
@@ -16,14 +12,6 @@ Result, // }; -pub (crate )fn expect_punct (it: &mut token_stream::IntoIter ) ->char { -if let TokenTree ::Punct (punct) = it.next ().expect ("Reached end of token stream for Punct" ) { - punct.as_char () - }else { -panic! ("Expected Punct" ); - } -} -/// A string literal that is required to have ASCII value only. pub (crate )struct AsciiLitStr (LitStr );
rust/macros/lib.rs +2 -2
@@ -311,8 +311,8 @@ pub fn fmt(input: TokenStream) -> TokenStream {/// assert_eq!(BR_OK, binder_driver_return_protocol_BR_OK); /// ``` #[proc_macro] -pub fn concat_idents (ts:TokenStream ) ->TokenStream { - concat_idents::concat_idents (ts.into ()).into () +pub fn concat_idents (input:TokenStream ) ->TokenStream { + concat_idents::concat_idents (parse_macro_input! (input)).into () }/// Paste identifiers together.
Signature · 1 line
2.51.2
PATCH v3 8/12 rust: macros: convert `#[kunit_tests]` macro to use `syn`
In reply to Gary GuoFrom: Gary Guo <[email protected]>
Make use of syn to parse the module structurally and thus improve the robustness of parsing.
String interpolation is avoided by generating tokens directly using quote!.
--- rust/macros/kunit.rs | 274 +++++++++++++++++++------------------------ rust/macros/lib.rs | 6 +- 2 files changed, 123 insertions(+), 157 deletions(-)
rust/macros/kunit.rs +119 -155
@@ -4,81 +4,50 @@//! //! Copyright (c) 2023 José Expósito <[email protected]> -use std::collections::HashMap ; -use std::fmt::Write ; - -use proc_macro2::{Delimiter ,Group ,TokenStream ,TokenTree }; - -pub (crate )fn kunit_tests (attr:TokenStream , ts:TokenStream ) ->TokenStream { -let attr = attr.to_string (); - -if attr.is_empty () { -panic! ("Missing test name in `#[kunit_tests(test_name)]` macro" ) - } - -if attr.len () >255 { -panic! ("The test suite name `{attr}` exceeds the maximum length of 255 bytes" ) +use std::ffi::CString ; + +use proc_macro2::TokenStream ; +use quote::{ + format_ident, + quote, +ToTokens ,// +}; +use syn::{ + parse_quote, +Error , +Ident , +Item , +ItemMod , +LitCStr , +Result ,// +}; + +pub (crate )fn kunit_tests (test_suite:Ident ,mut module:ItemMod ) ->Result <TokenStream > { +if test_suite.to_string ().len () >255 { +return Err (Error ::new_spanned ( + test_suite, +"test suite names cannot exceed the maximum length of 255 bytes" , + )); } -let mut tokens:Vec <_ > = ts.into_iter ().collect (); - -// Scan for the `mod` keyword. - tokens - .iter () - .find_map (|token|match token { -TokenTree ::Ident (ident) =>match ident.to_string ().as_str () { -"mod" =>Some (true ), - _ =>None , - }, - _ =>None , - }) - .expect ("`#[kunit_tests(test_name)]` attribute should only be applied to modules" ); - -// Retrieve the main body. The main body should be the last token tree. -let body =match tokens.pop () { -Some (TokenTree ::Group (group))if group.delimiter () ==Delimiter ::Brace => group, - _ =>panic! ("Cannot locate main body of module" ), +// We cannot handle modules that defer to another file (e.g. `mod foo;`). +let Some ((module_brace, module_items)) = module.content.take ()else { +Err (Error ::new_spanned ( + module, +"`#[kunit_tests(test_name)]` attribute should only be applied to inline modules" , + ))? }; -// Get the functions set as tests. Search for `[test]` -> `fn`. -let mut body_it = body.stream ().into_iter (); -let mut tests =Vec ::new (); -let mut attributes:HashMap <String ,TokenStream > =HashMap ::new (); -while let Some (token) = body_it.next () { -match token { -TokenTree ::Punct (ref p)if p.as_char () =='#' =>match body_it.next () { -Some (TokenTree ::Group (g))if g.delimiter () ==Delimiter ::Bracket => { -if let Some (TokenTree ::Ident (name)) = g.stream ().into_iter ().next () { -// Collect attributes because we need to find which are tests. We also -// need to copy `cfg` attributes so tests can be conditionally enabled. - attributes - .entry (name.to_string ()) - .or_default () - .extend ([token,TokenTree ::Group (g)]); - } -continue ; - } - _ => (), - }, -TokenTree ::Ident (i)if i =="fn" && attributes.contains_key ("test" ) => { -if let Some (TokenTree ::Ident (test_name)) = body_it.next () { - tests.push ((test_name, attributes.remove ("cfg" ).unwrap_or_default ())) - } - } - - _ => (), - } - attributes.clear (); - } +// Make the entire module gated behind `CONFIG_KUNIT`. + module + .attrs + .insert (0 ,parse_quote! (#[cfg( CONFIG_KUNIT = "y" )] )); -// Add `#[cfg(CONFIG_KUNIT="y")]` before the module declaration. -let config_kunit ="#[cfg(CONFIG_KUNIT=\"y\")]" .to_owned ().parse ().unwrap (); - tokens.insert ( -0 , -TokenTree ::Group (Group ::new (Delimiter ::None , config_kunit)), - ); +let mut processed_items =Vec ::new (); +let mut test_cases =Vec ::new ();// Generate the test KUnit test suite and a test case for each `#[test]`. +// // The code generated for the following test module: // // ```
@@ -110,98 +79,93 @@ pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {// // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES); // ``` -let mut kunit_macros ="" .to_owned (); -let mut test_cases ="" .to_owned (); -let mut assert_macros ="" .to_owned (); -let path =crate ::helpers::file (); -let num_tests = tests.len (); -for (test, cfg_attr)in tests { -let kunit_wrapper_fn_name =format! ("kunit_rust_wrapper_{test}" ); -// Append any `cfg` attributes the user might have written on their tests so we don't -// attempt to call them when they are `cfg`'d out. An extra `use` is used here to reduce -// the length of the assert message. -let kunit_wrapper =format! ( -r#"unsafe extern "C" fn {kunit_wrapper_fn_name}(_test: *mut ::kernel::bindings::kunit) -{{ -(*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED; -{cfg_attr} {{ -(*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS; -use ::kernel::kunit::is_test_result_ok; -assert!(is_test_result_ok({test}())); +// +// Non-function items (e.g. imports) are preserved. +for itemin module_items { +let Item ::Fn (mut f) = itemelse { + processed_items.push (item); +continue ; + }; + +// TODO: Replace below with `extract_if` when MSRV is bumped above 1.85. +let before_len = f.attrs.len (); + f.attrs.retain (|attr| !attr.path ().is_ident ("test" )); +if f.attrs.len () == before_len { + processed_items.push (Item ::Fn (f)); +continue ; + } + +let test = f.sig.ident.clone (); + +// Retrieve `#[cfg]` applied on the function which needs to be present on derived items too. +let cfg_attrs:Vec <_ > = f + .attrs + .iter () + .filter (|attr| attr.path ().is_ident ("cfg" )) + .cloned () + .collect (); + +// Before the test, override usual `assert!` and `assert_eq!` macros with ones that call +// KUnit instead. +let test_str = test.to_string (); +let path =crate ::helpers::file (); + processed_items.push (parse_quote! { +#[allow(unused)] +macro_rules! assert { + ($cond:expr $(,)?) => {{ + kernel::kunit_assert!(#test_str, #path,0 , $cond); + }} + } + }); + processed_items.push (parse_quote! { +#[allow(unused)] +macro_rules! assert_eq { + ($left:expr, $right:expr $(,)?) => {{ + kernel::kunit_assert_eq!(#test_str, #path,0 , $left, $right); }} -}}"# , + } + }); + +// Add back the test item. + processed_items.push (Item ::Fn (f)); + +let kunit_wrapper_fn_name =format_ident! ("kunit_rust_wrapper_{test}" ); +let test_cstr =LitCStr ::new ( + &CString ::new (test_str.as_str ()).expect ("identifier cannot contain NUL" ), + test.span (), ); -writeln! (kunit_macros,"{kunit_wrapper}" ).unwrap (); -writeln! ( - test_cases, -" ::kernel::kunit::kunit_case(::kernel::c_str!(\"{test}\"), {kunit_wrapper_fn_name})," - ) - .unwrap (); -writeln! ( - assert_macros, -r#" -/// Overrides the usual [`assert!`] macro with one that calls KUnit instead. -#[allow(unused)] -macro_rules! assert {{ -($cond:expr $(,)?) => {{{{ -kernel::kunit_assert!("{test}", "{path}", 0, $cond); -}}}} -}} - -/// Overrides the usual [`assert_eq!`] macro with one that calls KUnit instead. -#[allow(unused)] -macro_rules! assert_eq {{ -($left:expr, $right:expr $(,)?) => {{{{ -kernel::kunit_assert_eq!("{test}", "{path}", 0, $left, $right); -}}}} -}} -"# - ) - .unwrap (); - } + processed_items.push (parse_quote! { +unsafe extern "C" fn #kunit_wrapper_fn_name (_test: *mut ::kernel::bindings::kunit ) { + (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED; -writeln! (kunit_macros).unwrap (); -writeln! ( - kunit_macros, -"static mut TEST_CASES: [::kernel::bindings::kunit_case; {}] = [\n{test_cases} ::kernel::kunit::kunit_case_null(),\n];" , - num_tests +1 - ) - .unwrap (); - -writeln! ( - kunit_macros, -"::kernel::kunit_unsafe_test_suite!({attr}, TEST_CASES);" - ) - .unwrap (); - -// Remove the `#[test]` macros. -// We do this at a token level, in order to preserve span information. -let mut new_body =vec! []; -let mut body_it = body.stream ().into_iter (); - -while let Some (token) = body_it.next () { -match token { -TokenTree ::Punct (ref c)if c.as_char () =='#' =>match body_it.next () { -Some (TokenTree ::Group (group))if group.to_string () =="[test]" => (), -Some (next) => { - new_body.extend ([token, next]); - } - _ => { - new_body.push (token); +// Append any `cfg` attributes the user might have written on their tests so we +// don't attempt to call them when they are `cfg`'d out. An extra `use` is used +// here to reduce the length of the assert message. + #(#cfg_attrs)* + { + (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS; +use ::kernel::kunit::is_test_result_ok; +assert! (is_test_result_ok (#test ())); } - }, - _ => { - new_body.push (token); } - } - } + }); -let mut final_body =TokenStream ::new (); - final_body.extend ::<TokenStream >(assert_macros.parse ().unwrap ()); - final_body.extend (new_body); - final_body.extend ::<TokenStream >(kunit_macros.parse ().unwrap ()); - - tokens.push (TokenTree ::Group (Group ::new (Delimiter ::Brace , final_body))); + test_cases.push (quote! ( + ::kernel::kunit::kunit_case (#test_cstr, #kunit_wrapper_fn_name) + )); + } - tokens.into_iter ().collect () +let num_tests_plus_1 = test_cases.len () +1 ; + processed_items.push (parse_quote! { +static mut TEST_CASES : [::kernel::bindings::kunit_case ; #num_tests_plus_1] = [ + #(#test_cases,)* + ::kernel::kunit::kunit_case_null (), + ]; + }); + processed_items.push (parse_quote! { + ::kernel::kunit_unsafe_test_suite!(#test_suite,TEST_CASES ); + }); + + module.content =Some ((module_brace, processed_items)); +Ok (module.to_token_stream ()) }
rust/macros/lib.rs +4 -2
@@ -481,6 +481,8 @@ pub fn paste(input: TokenStream) -> TokenStream {/// } /// ``` #[proc_macro_attribute] -pub fn kunit_tests (attr:TokenStream , ts:TokenStream ) ->TokenStream { - kunit::kunit_tests (attr.into (), ts.into ()).into () +pub fn kunit_tests (attr:TokenStream , input:TokenStream ) ->TokenStream { + kunit::kunit_tests (parse_macro_input! (attr),parse_macro_input! (input)) + .unwrap_or_else (|e| e.into_compile_error ()) + .into () }
Signature · 1 line
2.51.2
PATCH v3 9/12 rust: macros: allow arbitrary types to be used in `module!` macro
In reply to Gary GuoFrom: Gary Guo <[email protected]>
Previously this only accepts an identifier, but now with syn it is easy to make it accepts any type.
--- rust/macros/module.rs | 13 +++++++------ 1 file changed, 7 insertions(+), 6 deletions(-)
rust/macros/module.rs +7 -6
@@ -26,7 +26,8 @@ LitStr, Path, Result, - Token, // + Token, + Type, // };use crate ::helpers::*;
@@ -370,7 +371,7 @@ fn parse(input: ParseStream<'_>) -> Result<Self> { }pub (crate )struct ModuleInfo { - type_:Ident , + type_:Type , license:AsciiLitStr , name:AsciiLitStr , authors:Option <Punctuated <AsciiLitStr ,Token! [,]>>,
@@ -529,7 +530,6 @@ impl ::kernel::ModuleMetadata for #type_ {// Double nested modules, since then nobody can access the public items inside. mod __module_init {mod __module_init { - use super::super::#type_;use pin_init::PinInit ;/// The "Rust loadable module" mark.
@@ -541,7 +541,7 @@ mod __module_init {#[used(compiler)] static __IS_RUST_MODULE: () = (); - static mut __MOD: ::core::mem::MaybeUninit<#type_> = +static mut __MOD: ::core::mem::MaybeUninit <super ::super ::LocalModule > = ::core::mem::MaybeUninit ::uninit ();// Loadable modules need to export the `{init,cleanup}_module` identifiers.
@@ -628,8 +628,9 @@ pub extern "C" fn #ident_exit() {/// /// This function must only be called once. unsafe fn __init () -> ::kernel::ffi::c_int { -let initer = - <#type_ as ::kernel::InPlaceModule>::init(&super::super::THIS_MODULE); +let initer = <super ::super ::LocalModule as ::kernel::InPlaceModule >::init ( + &super ::super ::THIS_MODULE + );// SAFETY: No data race, since `__MOD` can only be accessed by this module // and there only `__init` and `__exit` access it. These functions are only // called once and `__exit` cannot be called before or during `__init`.
Signature · 1 line
2.51.2
PATCH v3 10/12 rust: macros: rearrange `#[doc(hidden)]` in `module!` macro
In reply to Gary GuoFrom: Gary Guo <[email protected]>
This #[doc(hidden)] can just be applied on a module to hide anything within.
--- rust/macros/module.rs | 10 +--------- 1 file changed, 1 insertion(+), 9 deletions(-)
rust/macros/module.rs +1 -9
@@ -77,7 +77,6 @@ fn emit_base(&mut self, field: &str, content: &str, builtin: bool, param: bool) );let item =quote! { #cfg -#[doc(hidden)] #[cfg_attr(not(target_os = "macos" ), link_section = ".modinfo" )] #[used(compiler)] pub static #counter: [u8 ; #length] = *#string;
@@ -528,6 +527,7 @@ impl ::kernel::ModuleMetadata for #type_ { }// Double nested modules, since then nobody can access the public items inside. +#[doc(hidden)] mod __module_init {mod __module_init {use pin_init::PinInit ;
@@ -537,7 +537,6 @@ mod __module_init {// This may be best done another way later on, e.g. as a new modinfo // key or a new section. For the moment, keep it simple. #[cfg( MODULE )] -#[doc(hidden)] #[used(compiler)] static __IS_RUST_MODULE: () = ();
@@ -550,7 +549,6 @@ mod __module_init {/// This function must not be called after module initialization, because it may be /// freed after that completes. #[cfg( MODULE )] -#[doc(hidden)] #[no_mangle] #[link_section = ".init.text" ] pub unsafe extern "C" fn init_module () -> ::kernel::ffi::c_int {
@@ -561,14 +559,12 @@ mod __module_init { }#[cfg( MODULE )] -#[doc(hidden)] #[used(compiler)] #[link_section = ".init.data" ] static __UNIQUE_ID___addressable_init_module:unsafe extern "C" fn () ->i32 = init_module;#[cfg( MODULE )] -#[doc(hidden)] #[no_mangle] #[link_section = ".exit.text" ] pub extern "C" fn cleanup_module () {
@@ -582,7 +578,6 @@ pub extern "C" fn cleanup_module() { }#[cfg( MODULE )] -#[doc(hidden)] #[used(compiler)] #[link_section = ".exit.data" ] static __UNIQUE_ID___addressable_cleanup_module:extern "C" fn () = cleanup_module;
@@ -591,7 +586,6 @@ pub extern "C" fn cleanup_module() {// and the identifiers need to be unique. #[cfg(not( MODULE ))] #[cfg(not( CONFIG_HAVE_ARCH_PREL32_RELOCATIONS ))] -#[doc(hidden)] #[link_section = #initcall_section]#[used(compiler)] pub static #ident_initcall: extern "C" fn() ->
@@ -602,7 +596,6 @@ pub extern "C" fn cleanup_module() { ::core::arch::global_asm!(#global_asm);#[cfg(not( MODULE ))] -#[doc(hidden)] #[no_mangle] pub extern "C" fn #ident_init() -> ::kernel::ffi::c_int {// SAFETY: This function is inaccessible to the outside due to the double
@@ -612,7 +605,6 @@ pub extern "C" fn #ident_init() -> ::kernel::ffi::c_int { }#[cfg(not( MODULE ))] -#[doc(hidden)] #[no_mangle] pub extern "C" fn #ident_exit() {// SAFETY:
Signature · 1 line
2.51.2
PATCH v3 11/12 rust: kunit: use `pin_init::zeroed` instead of custom null value
In reply to Gary GuoFrom: Gary Guo <[email protected]>
The last null element can be created (constly) using pin_init::zeroed, so prefer to use it instead of adding a custom way of building it.
--- rust/kernel/kunit.rs | 26 +------------------------- rust/macros/kunit.rs | 4 ++-- 2 files changed, 3 insertions(+), 27 deletions(-)
rust/kernel/kunit.rs +1 -25
@@ -192,9 +192,6 @@ pub fn is_test_result_ok(t: impl TestResult) -> bool { }/// Represents an individual test case. -/// -/// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of valid test cases. -/// Use [`kunit_case_null`] to generate such a delimiter. #[doc(hidden)] pub const fn kunit_case ( name: &'static kernel::str::CStr,
@@ -215,27 +212,6 @@ pub const fn kunit_case( } } -/// Represents the NULL test case delimiter. -/// -/// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of test cases. This -/// function returns such a delimiter. -#[doc(hidden)] -pub const fn kunit_case_null () -> kernel::bindings::kunit_case { - kernel::bindings::kunit_case { - run_case:None , - name: core::ptr::null_mut (), - generate_params:None , - attr: kernel::bindings::kunit_attributes { - speed: kernel::bindings::kunit_speed_KUNIT_SPEED_NORMAL, - }, - status: kernel::bindings::kunit_status_KUNIT_SUCCESS, - module_name: core::ptr::null_mut (), - log: core::ptr::null_mut (), - param_init:None , - param_exit:None , - } -} -/// Registers a KUnit test suite. /// /// # Safety
@@ -254,7 +230,7 @@ pub const fn kunit_case_null() -> kernel::bindings::kunit_case {/// /// static mut KUNIT_TEST_CASES: [kernel::bindings::kunit_case; 2] = [ /// kernel::kunit::kunit_case(kernel::c_str!("name"), test_fn), -/// kernel::kunit::kunit_case_null(), +/// pin_init::zeroed(), /// ]; /// kernel::kunit_unsafe_test_suite!(suite_name, KUNIT_TEST_CASES); /// ```
rust/macros/kunit.rs +2 -2
@@ -74,7 +74,7 @@ pub(crate) fn kunit_tests(test_suite: Ident, mut module: ItemMod) -> Result<Toke// static mut TEST_CASES: [::kernel::bindings::kunit_case; 3] = [ // ::kernel::kunit::kunit_case(::kernel::c_str!("foo"), kunit_rust_wrapper_foo), // ::kernel::kunit::kunit_case(::kernel::c_str!("bar"), kunit_rust_wrapper_bar), -// ::kernel::kunit::kunit_case_null(), +// ::pin_init::zeroed(), // ]; // // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES);
@@ -159,7 +159,7 @@ macro_rules! assert_eq { processed_items.push (parse_quote! {static mut TEST_CASES : [::kernel::bindings::kunit_case ; #num_tests_plus_1] = [ #(#test_cases,)* - ::kernel::kunit::kunit_case_null (), + ::pin_init::zeroed (), ]; }); processed_items.push (parse_quote! {
Signature · 1 line
2.51.2
PATCH v3 12/12 rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
In reply to Gary GuoFrom: Gary Guo <[email protected]>
Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.
Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.
With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.
--- rust/macros/helpers.rs | 9 +++++++++ rust/macros/vtable.rs | 13 ++++--------- 2 files changed, 13 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +9 -0
@@ -7,6 +7,7 @@ Parse, ParseStream, // }, + Attribute, Error, LitStr, Result, //
@@ -53,3 +54,11 @@ pub(crate) fn file() -> String { proc_macro::Span ::call_site ().file () } } + +/// Obtain all `#[cfg]` attributes. +pub (crate )fn gather_cfg_attrs (attr: &[Attribute ]) ->Vec <Attribute > { + attr.iter () + .filter (|a| a.path ().is_ident ("cfg" )) + .cloned () + .collect () +}
rust/macros/vtable.rs +4 -9
@@ -23,7 +23,6 @@fn handle_trait (mut item:ItemTrait ) ->Result <ItemTrait > {let mut gen_items =Vec ::new (); -let mut gen_consts =HashSet ::new (); gen_items.push (parse_quote! {/// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable)
@@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> { &format!("HAS_{}", name.to_string().to_uppercase()), name.span(), ); -// Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively -// define functions. -// FIXME: `#[cfg]` should be copied and propagated to the generated consts. -if gen_consts.contains (&gen_const_name) { -continue ; - }// We don't know on the implementation-site whether a method is required or provided // so we have to generate a const for all methods. +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs);let comment =format! ("Indicates if the `{name}` method is overridden by the implementor." ); gen_items.push (parse_quote! { + #(#cfg_attrs)*#[doc = #comment] const #gen_const_name:bool =false ; }); - gen_consts.insert (gen_const_name); } }
@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {if defined_consts.contains (&gen_const_name) {continue ; } +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs); gen_items.push (parse_quote! { + #(#cfg_attrs)*const #gen_const_name:bool =true ; }); - defined_consts.insert (gen_const_name); } }
Signature · 1 line
2.51.2
Re: [PATCH v3 12/12] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
Quoted 33 lines · Gary Guo
From: Gary Guo <[email protected]>
Currently, we generate
HAS_constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.Currently, uses of
#[cfg]present in both trait and impl, so it is not a problem; however if only the impl side uses#[cfg]thenHAS_constants will incorrectly be true while it shouldnt't.With
synsupport, we can now implement#[cfg]handling properly by propagating the#[cfg]attributes to the constants.Signed-off-by Gary Guo <[email protected]>--- rust/macros/helpers.rs | 9 +++++++++ rust/macros/vtable.rs | 13 ++++--------- 2 files changed, 13 insertions(+), 9 deletions(-)rust/macros/helpers.rs +1
-0@@ -7,6 +7,7 @@ Parse, ParseStream, // }, + Attribute, Error, LitStr, Result, //@@ -53,3 +54,11 @@ pub(crate) fn file() -> String { proc_macro::Span ::call_site ().file ()
rust/macros/helpers.rs pub(crate) fn file() -> String { · line 64
Quoted 10 lines
rust/macros/helpers.rs +8
-0} } + +/// Obtain all `#[cfg]` attributes. +pub (crate )fn gather_cfg_attrs (attr: &[Attribute ]) ->Vec <Attribute > { + attr.iter () + .filter (|a| a.path ().is_ident ("cfg" )) + .cloned () + .collect () +}
I believe there's no need to collect this into a vector, quote supports iterators.
https://docs.rs/quote/latest/quote/macro.quote.html#interpolation
Quoted 51 lines
rust/macros/vtable.rs +4
-10@@ -23,7 +23,6 @@fn handle_trait (mut item:ItemTrait ) ->Result <ItemTrait > {let mut gen_items =Vec ::new (); -let mut gen_consts =HashSet ::new (); gen_items.push (parse_quote! {/// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable) @@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> { &format!("HAS_{}", name.to_string().to_uppercase()), name.span(), ); -// Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively -// define functions. -// FIXME: `#[cfg]` should be copied and propagated to the generated consts. -if gen_consts.contains (&gen_const_name) { -continue ; - }// We don't know on the implementation-site whether a method is required or provided // so we have to generate a const for all methods. +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs);let comment =format! ("Indicates if the `{name}` method is overridden by the implementor." ); gen_items.push (parse_quote! { + #(#cfg_attrs)*#[doc = #comment] const #gen_const_name:bool =false ; }); - gen_consts.insert (gen_const_name); } }@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {if defined_consts.contains (&gen_const_name) {continue ; } +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs); gen_items.push (parse_quote! { + #(#cfg_attrs)*const #gen_const_name:bool =true ; }); - defined_consts.insert (gen_const_name); } } --
Quoted 2 lines
2.51.2
PATCH 1/1 rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
From: Gary Guo <[email protected]>
Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.
Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.
With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.
--- rust/macros/helpers.rs | 6 ++++++ rust/macros/vtable.rs | 13 ++++--------- 2 files changed, 10 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +6 -0
@@ -7,6 +7,7 @@ Parse, ParseStream, // }, + Attribute, Error, LitStr, Result, //
@@ -53,3 +54,8 @@ pub(crate) fn file() -> String { proc_macro::Span ::call_site ().file () } } + +/// Obtain all `#[cfg]` attributes. +pub (crate )fn gather_cfg_attrs (attr: &[Attribute ]) ->impl Iterator <Item =Attribute > + '_ { + attr.iter ().filter (|a| a.path ().is_ident ("cfg" )).cloned () +}
rust/macros/vtable.rs +4 -9
@@ -23,7 +23,6 @@fn handle_trait (mut item:ItemTrait ) ->Result <ItemTrait > {let mut gen_items =Vec ::new (); -let mut gen_consts =HashSet ::new (); gen_items.push (parse_quote! {/// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable)
@@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> { &format!("HAS_{}", name.to_string().to_uppercase()), name.span(), ); -// Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively -// define functions. -// FIXME: `#[cfg]` should be copied and propagated to the generated consts. -if gen_consts.contains (&gen_const_name) { -continue ; - }// We don't know on the implementation-site whether a method is required or provided // so we have to generate a const for all methods. +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs);let comment =format! ("Indicates if the `{name}` method is overridden by the implementor." ); gen_items.push (parse_quote! { + #(#cfg_attrs)*#[doc = #comment] const #gen_const_name:bool =false ; }); - gen_consts.insert (gen_const_name); } }
@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {if defined_consts.contains (&gen_const_name) {continue ; } +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs); gen_items.push (parse_quote! { + #(#cfg_attrs)*const #gen_const_name:bool =true ; }); - defined_consts.insert (gen_const_name); } }
base-commit: 761e1fbc67b122bfe3d58518a9ea9ee668962c92
Signature · 1 line
2.51.2
Re: [PATCH] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
Quoted 32 lines · Gary Guo
From: Gary Guo <[email protected]>
Currently, we generate
HAS_constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.Currently, uses of
#[cfg]present in both trait and impl, so it is not a problem; however if only the impl side uses#[cfg]thenHAS_constants will incorrectly be true while it shouldnt't.With
synsupport, we can now implement#[cfg]handling properly by propagating the#[cfg]attributes to the constants.Signed-off-by Gary Guo <[email protected]>--- rust/macros/helpers.rs | 6 ++++++ rust/macros/vtable.rs | 13 ++++--------- 2 files changed, 10 insertions(+), 9 deletions(-)rust/macros/helpers.rs +1
-0@@ -7,6 +7,7 @@ Parse, ParseStream, // }, + Attribute, Error, LitStr, Result, //
rust/macros/helpers.rs pub(crate) fn file() -> String { · line 61
Quoted 8 lines
rust/macros/helpers.rs +5
-0@@ -53,3 +54,8 @@ pub(crate) fn file() -> String { proc_macro::Span ::call_site ().file () } } + +/// Obtain all `#[cfg]` attributes. +pub (crate )fn gather_cfg_attrs (attr: &[Attribute ]) ->impl Iterator <Item =Attribute > + '_ { + attr.iter ().filter (|a| a.path ().is_ident ("cfg" )).cloned () +}
There's also no need to clone; ToTokens::to_tokens takes &self.
https://docs.rs/quote/latest/quote/trait.ToTokens.html#tymethod.to_tokens
Quoted 55 lines
rust/macros/vtable.rs +4
-9@@ -23,7 +23,6 @@fn handle_trait (mut item:ItemTrait ) ->Result <ItemTrait > {let mut gen_items =Vec ::new (); -let mut gen_consts =HashSet ::new (); gen_items.push (parse_quote! {/// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable) @@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> { &format!("HAS_{}", name.to_string().to_uppercase()), name.span(), ); -// Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively -// define functions. -// FIXME: `#[cfg]` should be copied and propagated to the generated consts. -if gen_consts.contains (&gen_const_name) { -continue ; - }// We don't know on the implementation-site whether a method is required or provided // so we have to generate a const for all methods. +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs);let comment =format! ("Indicates if the `{name}` method is overridden by the implementor." ); gen_items.push (parse_quote! { + #(#cfg_attrs)*#[doc = #comment] const #gen_const_name:bool =false ; }); - gen_consts.insert (gen_const_name); } }@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {if defined_consts.contains (&gen_const_name) {continue ; } +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs); gen_items.push (parse_quote! { + #(#cfg_attrs)*const #gen_const_name:bool =true ; }); - defined_consts.insert (gen_const_name); } }base-commit:
761e1fbc67b122bfe3d58518a9ea9ee668962c92--
2.51.2
Gary Guo
maintainerRe: [PATCH] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
Quoted 44 lines · Tamir Duberstein
Quoted 32 lines · Gary Guo
From: Gary Guo <[email protected]>
Currently, we generate
HAS_constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.Currently, uses of
#[cfg]present in both trait and impl, so it is not a problem; however if only the impl side uses#[cfg]thenHAS_constants will incorrectly be true while it shouldnt't.With
synsupport, we can now implement#[cfg]handling properly by propagating the#[cfg]attributes to the constants.Signed-off-by Gary Guo <[email protected]>--- rust/macros/helpers.rs | 6 ++++++ rust/macros/vtable.rs | 13 ++++--------- 2 files changed, 10 insertions(+), 9 deletions(-)rust/macros/helpers.rs +1
-0@@ -7,6 +7,7 @@ Parse, ParseStream, // }, + Attribute, Error, LitStr, Result, //rust/macros/helpers.rs pub(crate) fn file() -> String { · line 61
Quoted 8 lines
rust/macros/helpers.rs +5
-0@@ -53,3 +54,8 @@ pub(crate) fn file() -> String { proc_macro::Span ::call_site ().file () } } + +/// Obtain all `#[cfg]` attributes. +pub (crate )fn gather_cfg_attrs (attr: &[Attribute ]) ->impl Iterator <Item =Attribute > + '_ { + attr.iter ().filter (|a| a.path ().is_ident ("cfg" )).cloned () +}There's also no need to clone;
ToTokens::to_tokenstakes&self.
Indeed. In that case I feel like the line is short enough and we can just inline these and skip the helper function?
Best,
Gary
Re: [PATCH] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
Quoted 46 lines · Gary Guo
Quoted 44 lines · Tamir Duberstein
Quoted 32 lines · Gary Guo
From: Gary Guo <[email protected]>
Currently, we generate
HAS_constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.Currently, uses of
#[cfg]present in both trait and impl, so it is not a problem; however if only the impl side uses#[cfg]thenHAS_constants will incorrectly be true while it shouldnt't.With
synsupport, we can now implement#[cfg]handling properly by propagating the#[cfg]attributes to the constants.Signed-off-by Gary Guo <[email protected]>--- rust/macros/helpers.rs | 6 ++++++ rust/macros/vtable.rs | 13 ++++--------- 2 files changed, 10 insertions(+), 9 deletions(-)rust/macros/helpers.rs +1
-0@@ -7,6 +7,7 @@ Parse, ParseStream, // }, + Attribute, Error, LitStr, Result, //rust/macros/helpers.rs pub(crate) fn file() -> String { · line 61
Quoted 8 lines
rust/macros/helpers.rs +5
-0@@ -53,3 +54,8 @@ pub(crate) fn file() -> String { proc_macro::Span ::call_site ().file () } } + +/// Obtain all `#[cfg]` attributes. +pub (crate )fn gather_cfg_attrs (attr: &[Attribute ]) ->impl Iterator <Item =Attribute > + '_ { + attr.iter ().filter (|a| a.path ().is_ident ("cfg" )).cloned () +}There's also no need to clone;
ToTokens::to_tokenstakes&self.
Quoted 4 lines
Indeed. In that case I feel like the line is short enough and we can just inline these and skip the helper function?
I think extracting the duplication is still worthwhile, and we might need this elsewhere so I'd leave it in the helpers module. I don't feel strongly though, so go with your preference.
Quoted 3 lines
Best,
Gary
PATCH v2 1/1 rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
From: Gary Guo <[email protected]>
Currently, we generate HAS_ constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.
Currently, uses of #[cfg] present in both trait and impl, so it is not a problem; however if only the impl side uses #[cfg] then HAS_ constants will incorrectly be true while it shouldnt't.
With syn support, we can now implement #[cfg] handling properly by propagating the #[cfg] attributes to the constants.
--- rust/macros/helpers.rs | 6 ++++++ rust/macros/vtable.rs | 13 ++++--------- 2 files changed, 10 insertions(+), 9 deletions(-)
rust/macros/helpers.rs +6 -0
@@ -7,6 +7,7 @@ Parse, ParseStream, // }, + Attribute, Error, LitStr, Result, //
@@ -53,3 +54,8 @@ pub(crate) fn file() -> String { proc_macro::Span ::call_site ().file () } } + +/// Obtain all `#[cfg]` attributes. +pub (crate )fn gather_cfg_attrs (attr: &[Attribute ]) ->impl Iterator <Item = &Attribute > + '_ { + attr.iter ().filter (|a| a.path ().is_ident ("cfg" )) +}
rust/macros/vtable.rs +4 -9
@@ -23,7 +23,6 @@fn handle_trait (mut item:ItemTrait ) ->Result <ItemTrait > {let mut gen_items =Vec ::new (); -let mut gen_consts =HashSet ::new (); gen_items.push (parse_quote! {/// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable)
@@ -38,22 +37,17 @@ fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> { &format!("HAS_{}", name.to_string().to_uppercase()), name.span(), ); -// Skip if it's declared already -- this can happen if `#[cfg]` is used to selectively -// define functions. -// FIXME: `#[cfg]` should be copied and propagated to the generated consts. -if gen_consts.contains (&gen_const_name) { -continue ; - }// We don't know on the implementation-site whether a method is required or provided // so we have to generate a const for all methods. +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs);let comment =format! ("Indicates if the `{name}` method is overridden by the implementor." ); gen_items.push (parse_quote! { + #(#cfg_attrs)*#[doc = #comment] const #gen_const_name:bool =false ; }); - gen_consts.insert (gen_const_name); } }
@@ -87,10 +81,11 @@ fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> {if defined_consts.contains (&gen_const_name) {continue ; } +let cfg_attrs =crate ::helpers::gather_cfg_attrs (&fn_item.attrs); gen_items.push (parse_quote! { + #(#cfg_attrs)*const #gen_const_name:bool =true ; }); - defined_consts.insert (gen_const_name); } }
base-commit: 761e1fbc67b122bfe3d58518a9ea9ee668962c92
Signature · 1 line
2.51.2
Re: [PATCH v3 03/12] rust: macros: convert `#[vtable]` macro to use `syn`
In reply to Gary GuoQuoted 10 lines · Gary Guo
From: Gary Guo <[email protected]>
#[vtable]is converted to use syn. This is more robust than the previous heuristic-based searching of defined methods and functions.When doing so, the trait and impl are split into two code paths as the types are distinct when parsed by
syn.Reviewed-by Tamir Duberstein <[email protected]>Signed-off-by Gary Guo <[email protected]>
Cheers,
Benno
Quoted 4 lines
--- rust/macros/lib.rs | 9 ++- rust/macros/vtable.rs | 169 +++++++++++++++++++++++------------------- 2 files changed, 98 insertions(+), 80 deletions(-)
Re: [PATCH v3 08/12] rust: macros: convert `#[kunit_tests]` macro to use `syn`
In reply to Gary GuoQuoted 10 lines · Gary Guo
From: Gary Guo <[email protected]>
Make use of
synto parse the module structurally and thus improve the robustness of parsing.String interpolation is avoided by generating tokens directly using
quote!.Reviewed-by Tamir Duberstein <[email protected]>Signed-off-by Gary Guo <[email protected]>
Cheers,
Benno
Quoted 4 lines
--- rust/macros/kunit.rs | 274 +++++++++++++++++++------------------------ rust/macros/lib.rs | 6 +- 2 files changed, 123 insertions(+), 157 deletions(-)
Re: [PATCH v2] rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
In reply to Gary GuoQuoted 5 lines · Gary Guo
From: Gary Guo <[email protected]>
Currently, we generate
HAS_constants as long as the definition exists in the source, regardless if it is cfg-ed out or not.
Quoted 8 lines
Currently, uses of
#[cfg]present in both trait and impl, so it is not a problem; however if only the impl side uses#[cfg]thenHAS_constants will incorrectly be true while it shouldnt't.With
synsupport, we can now implement#[cfg]handling properly by propagating the#[cfg]attributes to the constants.Signed-off-by Gary Guo <[email protected]>
Cheers,
Benno
Quoted 4 lines
--- rust/macros/helpers.rs | 6 ++++++ rust/macros/vtable.rs | 13 ++++--------- 2 files changed, 10 insertions(+), 9 deletions(-)
Re: [PATCH v3 04/12] rust: macros: use `syn` to parse `module!` macro
In reply to Gary GuoQuoted 28 lines · Gary Guo
From: Gary Guo <[email protected]>
With
synbeing available in the kernel, use it to parse the complex custommodule!macro to replace existing helpers. Only parsing is changed in this commit, the code generation is untouched.This has the benefit of better error message when the macro is used incorrectly, as it can point to a concrete span on what's going wrong.
For example, if a field is specified twice, previously it reads:
error: proc macro panicked --> samples/rust/rust_minimal.rs:7:1 | 7 | / module! { 8 | | type: RustMinimal, 9 | | name: "rust_minimal", 10 | | author: "Rust for Linux Contributors", 11 | | description: "Rust minimal sample", 12 | | license: "GPL", 13 | | license: "GPL", 14 | | } | |_^ | = help: message: Duplicated key "license". Keys can only be specified once.now it reads:
Quoted 8 lines
error: duplicated key "license". Keys can only be specified once. --> samples/rust/rust_minimal.rs:13:5 | 13 | license: "GPL", | ^^^^^^^Reviewed-by Tamir Duberstein <[email protected]>Signed-off-by Gary Guo <[email protected]>
Cheers,
Benno
Quoted 5 lines
--- rust/macros/helpers.rs | 109 ++++------- rust/macros/lib.rs | 6 +- rust/macros/module.rs | 399 +++++++++++++++++++++++++---------------- 3 files changed, 280 insertions(+), 234 deletions(-)
Re: [PATCH v3 02/12] rust: macros: use `quote!` from vendored crate
In reply to Gary GuoQuoted 9 lines · Gary Guo
From: Gary Guo <[email protected]>
With
quotecrate now vendored in the kernel, we can remove our customquote!macro implementation and just rely on that crate instead.The
quotecrate uses types from theproc-macro2library so we also update to use that, and perform conversion in the top-levellib.rs.
Quoted 7 lines
Clippy complains about unnecessary
.to_string()asproc-macro2provides additionalPartialEqimpl, so they are removed.Reviewed-by Tamir Duberstein <[email protected]>Reviewed-by Benno Lossin <[email protected]>Signed-off-by Gary Guo <[email protected]>---
Cheers,
-- David
-- David
Quoted 438 lines
rust/macros/concat_idents.rs | 2 +- rust/macros/export.rs | 4 +- rust/macros/fmt.rs | 4 +- rust/macros/helpers.rs | 4 +- rust/macros/kunit.rs | 5 +- rust/macros/lib.rs | 21 ++-- rust/macros/module.rs | 6 +- rust/macros/paste.rs | 2 +- rust/macros/quote.rs | 182 ----------------------------------- rust/macros/vtable.rs | 7 +- 10 files changed, 32 insertions(+), 205 deletions(-) delete mode 100644 rust/macros/quote.rsrust/macros/concat_idents.rs +1
-1@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{token_stream,Ident ,TokenStream ,TokenTree }; +use proc_macro2::{token_stream,Ident ,TokenStream ,TokenTree };use crate ::helpers::expect_punct;rust/macros/export.rs +3
-1@@ -1,7 +1,9 @@// SPDX-License-Identifier: GPL-2.0 +use proc_macro2::TokenStream ; +use quote::quote; +use crate ::helpers::function_name; -use proc_macro::TokenStream ;/// Please see [`crate::export`] for documentation. pub (crate )fn export (_attr:TokenStream , ts:TokenStream ) ->TokenStream {rust/macros/fmt.rs +3
-1@@ -1,8 +1,10 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{Ident ,TokenStream ,TokenTree };use std::collections::BTreeSet ; +use proc_macro2::{Ident ,TokenStream ,TokenTree }; +use quote::quote_spanned; +/// Please see [`crate::fmt`] for documentation. pub (crate )fn fmt (input:TokenStream ) ->TokenStream {let mut input = input.into_iter ();rust/macros/helpers.rs +2
-2@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{token_stream,Group ,Ident ,TokenStream ,TokenTree }; +use proc_macro2::{token_stream,Group ,Ident ,TokenStream ,TokenTree };pub (crate )fn try_ident (it: &mut token_stream::IntoIter ) ->Option <String > {if let Some (TokenTree ::Ident (ident)) = it.next () {@@ -86,7 +86,7 @@ pub(crate) fn function_name(input: TokenStream) -> Option<Ident> {let mut input = input.into_iter ();while let Some (token) = input.next () {match token { -TokenTree ::Ident (i)if i.to_string () =="fn" => { +TokenTree ::Ident (i)if i =="fn" => {if let Some (TokenTree ::Ident (i)) = input.next () {return Some (i); }rust/macros/kunit.rs +3
-2@@ -4,10 +4,11 @@//! //! Copyright (c) 2023 José Expósito <[email protected]> -use proc_macro::{Delimiter ,Group ,TokenStream ,TokenTree };use std::collections::HashMap ;use std::fmt::Write ; +use proc_macro2::{Delimiter ,Group ,TokenStream ,TokenTree }; +pub (crate )fn kunit_tests (attr:TokenStream , ts:TokenStream ) ->TokenStream {let attr = attr.to_string ();@@ -59,7 +60,7 @@ pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream { } _ => (), }, - TokenTree::Ident(i) if i.to_string() == "fn" && attributes.contains_key("test") => { + TokenTree::Ident(i) if i == "fn" && attributes.contains_key("test") => {if let Some (TokenTree ::Ident (test_name)) = body_it.next () { tests.push ((test_name, attributes.remove ("cfg" ).unwrap_or_default ())) }rust/macros/lib.rs +11
-10@@ -11,8 +11,6 @@// to avoid depending on the full `proc_macro_span` on Rust >= 1.88.0. #![cfg_attr(not( CONFIG_RUSTC_HAS_SPAN_FILE ), feature(proc_macro_span))] -#[macro_use] -mod quote;mod concat_idents;mod export;mod fmt;@@ -132,7 +130,7 @@/// the kernel module. #[proc_macro] pub fn module (ts:TokenStream ) ->TokenStream { - module::module (ts) + module::module (ts.into ()).into () }/// Declares or implements a vtable trait. @@ -207,7 +205,7 @@ pub fn module(ts: TokenStream) -> TokenStream {/// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html #[proc_macro_attribute] pub fn vtable (attr:TokenStream , ts:TokenStream ) ->TokenStream { - vtable::vtable (attr, ts) + vtable::vtable (attr.into (), ts.into ()).into () }/// Export a function so that C code can call it via a header file. @@ -230,7 +228,7 @@ pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream {/// automatically exported with `EXPORT_SYMBOL_GPL`. #[proc_macro_attribute] pub fn export (attr:TokenStream , ts:TokenStream ) ->TokenStream { - export::export (attr, ts) + export::export (attr.into (), ts.into ()).into () }/// Like [`core::format_args!`], but automatically wraps arguments in [`kernel::fmt::Adapter`]. @@ -248,7 +246,7 @@ pub fn export(attr: TokenStream, ts: TokenStream) -> TokenStream {/// [`pr_info!`]: ../kernel/macro.pr_info.html #[proc_macro] pub fn fmt (input:TokenStream ) ->TokenStream { - fmt::fmt (input) + fmt::fmt (input.into ()).into () }/// Concatenate two identifiers. @@ -306,7 +304,7 @@ pub fn fmt(input: TokenStream) -> TokenStream {/// ``` #[proc_macro] pub fn concat_idents (ts:TokenStream ) ->TokenStream { - concat_idents::concat_idents (ts) + concat_idents::concat_idents (ts.into ()).into () }/// Paste identifiers together. @@ -444,9 +442,12 @@ pub fn concat_idents(ts: TokenStream) -> TokenStream {/// [`paste`]: https://docs.rs/paste/ #[proc_macro] pub fn paste (input:TokenStream ) ->TokenStream { -let mut tokens = input.into_iter ().collect (); +let mut tokens = proc_macro2::TokenStream ::from (input).into_iter ().collect (); paste::expand (&mut tokens); - tokens.into_iter ().collect () + tokens + .into_iter () + .collect ::<proc_macro2::TokenStream >() + .into () }/// Registers a KUnit test suite and its test cases using a user-space like syntax. @@ -473,5 +474,5 @@ pub fn paste(input: TokenStream) -> TokenStream {/// ``` #[proc_macro_attribute] pub fn kunit_tests (attr:TokenStream , ts:TokenStream ) ->TokenStream { - kunit::kunit_tests (attr, ts) + kunit::kunit_tests (attr.into (), ts.into ()).into () }rust/macros/module.rs +4
-2@@ -1,9 +1,11 @@// SPDX-License-Identifier: GPL-2.0 -use crate ::helpers::*; -use proc_macro::{token_stream,Delimiter ,Literal ,TokenStream ,TokenTree };use std::fmt::Write ; +use proc_macro2::{token_stream,Delimiter ,Literal ,TokenStream ,TokenTree }; + +use crate ::helpers::*; +fn expect_string_array (it: &mut token_stream::IntoIter ) ->Vec <String > {let group =expect_group (it);assert_eq! (group.delimiter (),Delimiter ::Bracket );rust/macros/paste.rs +1
-1@@ -1,6 +1,6 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{Delimiter ,Group ,Ident ,Spacing ,Span ,TokenTree }; +use proc_macro2::{Delimiter ,Group ,Ident ,Spacing ,Span ,TokenTree };fn concat_helper (tokens: &[TokenTree ]) ->Vec <(String ,Span )> {let mut tokens = tokens.iter ();rust/macros/quote.rs deleted +0
-182@@ -1,182 +0,0 @@ -// SPDX-License-Identifier: Apache-2.0 OR MIT - -use proc_macro::{TokenStream ,TokenTree }; - -pub (crate )trait ToTokens { -fn to_tokens (&self, tokens: &mut TokenStream ); -} - -impl <T :ToTokens >ToTokens for Option <T > { -fn to_tokens (&self, tokens: &mut TokenStream ) { -if let Some (v) = self { - v.to_tokens (tokens); - } - } -} - -impl ToTokens for proc_macro::Group { -fn to_tokens (&self, tokens: &mut TokenStream ) { - tokens.extend ([TokenTree ::from (self.clone ())]); - } -} - -impl ToTokens for proc_macro::Ident { -fn to_tokens (&self, tokens: &mut TokenStream ) { - tokens.extend ([TokenTree ::from (self.clone ())]); - } -} - -impl ToTokens for TokenTree { -fn to_tokens (&self, tokens: &mut TokenStream ) { - tokens.extend ([self.clone ()]); - } -} - -impl ToTokens for TokenStream { -fn to_tokens (&self, tokens: &mut TokenStream ) { - tokens.extend (self.clone ()); - } -} - -/// Converts tokens into [`proc_macro::TokenStream`] and performs variable interpolations with -/// the given span. -/// -/// This is a similar to the -/// [`quote_spanned!`](https://docs.rs/quote/latest/quote/macro.quote_spanned.html) macro from the -/// `quote` crate but provides only just enough functionality needed by the current `macros` crate. -macro_rules! quote_spanned { - ($span:expr => $($tt:tt)*) => {{ -let mut tokens = ::proc_macro::TokenStream ::new (); - { -#[allow(unused_variables)] -let span = $span; -quote_spanned! (@proc tokensspan $($tt)*); - } - tokens - }}; - (@proc $v:ident $span:ident) => {}; - (@proc $v:ident $span:ident #$id:ident $($tt:tt)*) => { - $crate::quote::ToTokens ::to_tokens (&$id, &mut $v); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident #(#$id:ident)* $($tt:tt)*) => { -for tokenin $id { - $crate::quote::ToTokens ::to_tokens (&token, &mut $v); - } -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident ( $($inner:tt)* ) $($tt:tt)*) => { -#[allow(unused_mut)] -let mut tokens = ::proc_macro::TokenStream ::new (); -quote_spanned! (@proc tokens $span $($inner)*); - $v.extend ([::proc_macro::TokenTree ::Group (::proc_macro::Group ::new ( - ::proc_macro::Delimiter ::Parenthesis , - tokens, - ))]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident [ $($inner:tt)* ] $($tt:tt)*) => { -let mut tokens = ::proc_macro::TokenStream ::new (); -quote_spanned! (@proc tokens $span $($inner)*); - $v.extend ([::proc_macro::TokenTree ::Group (::proc_macro::Group ::new ( - ::proc_macro::Delimiter ::Bracket , - tokens, - ))]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident { $($inner:tt)* } $($tt:tt)*) => { -let mut tokens = ::proc_macro::TokenStream ::new (); -quote_spanned! (@proc tokens $span $($inner)*); - $v.extend ([::proc_macro::TokenTree ::Group (::proc_macro::Group ::new ( - ::proc_macro::Delimiter ::Brace , - tokens, - ))]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident :: $($tt:tt)*) => { - $v.extend ([::proc_macro::Spacing ::Joint , ::proc_macro::Spacing ::Alone ].map (|spacing| { - ::proc_macro::TokenTree ::Punct (::proc_macro::Punct ::new (':' , spacing)) - })); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident : $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new (':' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident , $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new (',' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident @ $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new ('@' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident ! $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new ('!' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident ; $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new (';' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident + $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new ('+' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident = $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new ('=' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident # $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new ('#' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident & $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Punct ( - ::proc_macro::Punct ::new ('&' , ::proc_macro::Spacing ::Alone ), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident _ $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Ident ( - ::proc_macro::Ident ::new ("_" , $span), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; - (@proc $v:ident $span:ident $id:ident $($tt:tt)*) => { - $v.extend ([::proc_macro::TokenTree ::Ident ( - ::proc_macro::Ident ::new (stringify! ($id), $span), - )]); -quote_spanned! (@proc $v $span $($tt)*); - }; -} - -/// Converts tokens into [`proc_macro::TokenStream`] and performs variable interpolations with -/// mixed site span ([`Span::mixed_site()`]). -/// -/// This is a similar to the [`quote!`](https://docs.rs/quote/latest/quote/macro.quote.html) macro -/// from the `quote` crate but provides only just enough functionality needed by the current -/// `macros` crate. -/// -/// [`Span::mixed_site()`]: https://doc.rust-lang.org/proc_macro/struct.Span.html#method.mixed_site -macro_rules! quote { - ($($tt:tt)*) => { -quote_spanned! (::proc_macro::Span ::mixed_site () => $($tt)*) - } -}rust/macros/vtable.rs +4
-4@@ -1,9 +1,10 @@// SPDX-License-Identifier: GPL-2.0 -use proc_macro::{Delimiter ,Group ,TokenStream ,TokenTree };use std::collections::HashSet ;use std::fmt::Write ; +use proc_macro2::{Delimiter ,Group ,TokenStream ,TokenTree }; +pub (crate )fn vtable (_attr:TokenStream , ts:TokenStream ) ->TokenStream {let mut tokens:Vec <_ > = ts.into_iter ().collect ();@@ -31,7 +32,7 @@ pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream {let mut consts =HashSet ::new ();while let Some (token) = body_it.next () {match token { -TokenTree ::Ident (ident)if ident.to_string () =="fn" => { +TokenTree ::Ident (ident)if ident =="fn" => {let fn_name =match body_it.next () {Some (TokenTree ::Ident (ident)) => ident.to_string (), // Possibly we've encountered a fn pointer type instead.@@ -39,7 +40,7 @@ pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream { }; functions.push (fn_name); } - TokenTree::Ident(ident) if ident.to_string() == "const" => { + TokenTree::Ident(ident) if ident == "const" => { let const_name = match body_it.next() { Some(TokenTree::Ident(ident)) => ident.to_string(), // Possibly we've encountered an inline const block instead. --2.51.2
Re: [PATCH v3 08/12] rust: macros: convert `#[kunit_tests]` macro to use `syn`
In reply to Gary GuoQuoted 6 lines · Gary Guo
From: Gary Guo <[email protected]>
Make use of
synto parse the module structurally and thus improve the robustness of parsing.
Quoted 6 lines
String interpolation is avoided by generating tokens directly using
quote!.Reviewed-by Tamir Duberstein <[email protected]>Signed-off-by Gary Guo <[email protected]>---
Sorry this took so long to review: it's taken a while for me to wrap my head around syn.
I'm pretty happy with this overall, though, and think I understand it well enough now.
Cheers,
-- David
Quoted 13 lines
rust/macros/kunit.rs | 274 +++++++++++++++++++------------------------ rust/macros/lib.rs | 6 +- 2 files changed, 123 insertions(+), 157 deletions(-)rust/macros/kunit.rs +0
-1@@ -4,81 +4,50 @@//! //! Copyright (c) 2023 José Expósito <[email protected]> -use std::collections::HashMap ;
rust/macros/kunit.rs line 17
Quoted 10 lines
rust/macros/kunit.rs +0
-10-use std::fmt::Write ; - -use proc_macro2::{Delimiter ,Group ,TokenStream ,TokenTree }; - -pub (crate )fn kunit_tests (attr:TokenStream , ts:TokenStream ) ->TokenStream { -let attr = attr.to_string (); - -if attr.is_empty () { -panic! ("Missing test name in `#[kunit_tests(test_name)]` macro" ) - }
I'm assuming it's okay for us to get rid of this as we get an "unexpected end of input" error instead.
Quoted 303 lines
rust/macros/kunit.rs +119
-144- -if attr.len () >255 { -panic! ("The test suite name `{attr}` exceeds the maximum length of 255 bytes" ) +use std::ffi::CString ; + +use proc_macro2::TokenStream ; +use quote::{ + format_ident, + quote, +ToTokens ,// +}; +use syn::{ + parse_quote, +Error , +Ident , +Item , +ItemMod , +LitCStr , +Result ,// +}; + +pub (crate )fn kunit_tests (test_suite:Ident ,mut module:ItemMod ) ->Result <TokenStream > { +if test_suite.to_string ().len () >255 { +return Err (Error ::new_spanned ( + test_suite, +"test suite names cannot exceed the maximum length of 255 bytes" , + )); } -let mut tokens:Vec <_ > = ts.into_iter ().collect (); - -// Scan for the `mod` keyword. - tokens - .iter () - .find_map (|token|match token { -TokenTree ::Ident (ident) =>match ident.to_string ().as_str () { -"mod" =>Some (true ), - _ =>None , - }, - _ =>None , - }) - .expect ("`#[kunit_tests(test_name)]` attribute should only be applied to modules" ); - -// Retrieve the main body. The main body should be the last token tree. -let body =match tokens.pop () { -Some (TokenTree ::Group (group))if group.delimiter () ==Delimiter ::Brace => group, - _ =>panic! ("Cannot locate main body of module" ), +// We cannot handle modules that defer to another file (e.g. `mod foo;`). +let Some ((module_brace, module_items)) = module.content.take ()else { +Err (Error ::new_spanned ( + module, +"`#[kunit_tests(test_name)]` attribute should only be applied to inline modules" , + ))? }; -// Get the functions set as tests. Search for `[test]` -> `fn`. -let mut body_it = body.stream ().into_iter (); -let mut tests =Vec ::new (); -let mut attributes:HashMap <String ,TokenStream > =HashMap ::new (); -while let Some (token) = body_it.next () { -match token { -TokenTree ::Punct (ref p)if p.as_char () =='#' =>match body_it.next () { -Some (TokenTree ::Group (g))if g.delimiter () ==Delimiter ::Bracket => { -if let Some (TokenTree ::Ident (name)) = g.stream ().into_iter ().next () { -// Collect attributes because we need to find which are tests. We also -// need to copy `cfg` attributes so tests can be conditionally enabled. - attributes - .entry (name.to_string ()) - .or_default () - .extend ([token,TokenTree ::Group (g)]); - } -continue ; - } - _ => (), - }, -TokenTree ::Ident (i)if i =="fn" && attributes.contains_key ("test" ) => { -if let Some (TokenTree ::Ident (test_name)) = body_it.next () { - tests.push ((test_name, attributes.remove ("cfg" ).unwrap_or_default ())) - } - } - - _ => (), - } - attributes.clear (); - } +// Make the entire module gated behind `CONFIG_KUNIT`. + module + .attrs + .insert (0 ,parse_quote! (#[cfg( CONFIG_KUNIT = "y" )] )); -// Add `#[cfg(CONFIG_KUNIT="y")]` before the module declaration. -let config_kunit ="#[cfg(CONFIG_KUNIT=\"y\")]" .to_owned ().parse ().unwrap (); - tokens.insert ( -0 , -TokenTree ::Group (Group ::new (Delimiter ::None , config_kunit)), - ); +let mut processed_items =Vec ::new (); +let mut test_cases =Vec ::new ();// Generate the test KUnit test suite and a test case for each `#[test]`. +// // The code generated for the following test module: // // ``` @@ -110,98 +79,93 @@ pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream {// // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES); // ``` -let mut kunit_macros ="" .to_owned (); -let mut test_cases ="" .to_owned (); -let mut assert_macros ="" .to_owned (); -let path =crate ::helpers::file (); -let num_tests = tests.len (); -for (test, cfg_attr)in tests { -let kunit_wrapper_fn_name =format! ("kunit_rust_wrapper_{test}" ); -// Append any `cfg` attributes the user might have written on their tests so we don't -// attempt to call them when they are `cfg`'d out. An extra `use` is used here to reduce -// the length of the assert message. -let kunit_wrapper =format! ( -r#"unsafe extern "C" fn {kunit_wrapper_fn_name}(_test: *mut ::kernel::bindings::kunit) -{{ -(*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED; -{cfg_attr} {{ -(*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS; -use ::kernel::kunit::is_test_result_ok; -assert!(is_test_result_ok({test}())); +// +// Non-function items (e.g. imports) are preserved. +for itemin module_items { +let Item ::Fn (mut f) = itemelse { + processed_items.push (item); +continue ; + }; + +// TODO: Replace below with `extract_if` when MSRV is bumped above 1.85. +let before_len = f.attrs.len (); + f.attrs.retain (|attr| !attr.path ().is_ident ("test" )); +if f.attrs.len () == before_len { + processed_items.push (Item ::Fn (f)); +continue ; + } + +let test = f.sig.ident.clone (); + +// Retrieve `#[cfg]` applied on the function which needs to be present on derived items too. +let cfg_attrs:Vec <_ > = f + .attrs + .iter () + .filter (|attr| attr.path ().is_ident ("cfg" )) + .cloned () + .collect (); + +// Before the test, override usual `assert!` and `assert_eq!` macros with ones that call +// KUnit instead. +let test_str = test.to_string (); +let path =crate ::helpers::file (); + processed_items.push (parse_quote! { +#[allow(unused)] +macro_rules! assert { + ($cond:expr $(,)?) => {{ + kernel::kunit_assert!(#test_str, #path,0 , $cond); + }} + } + }); + processed_items.push (parse_quote! { +#[allow(unused)] +macro_rules! assert_eq { + ($left:expr, $right:expr $(,)?) => {{ + kernel::kunit_assert_eq!(#test_str, #path,0 , $left, $right); }} -}}"# , + } + }); + +// Add back the test item. + processed_items.push (Item ::Fn (f)); + +let kunit_wrapper_fn_name =format_ident! ("kunit_rust_wrapper_{test}" ); +let test_cstr =LitCStr ::new ( + &CString ::new (test_str.as_str ()).expect ("identifier cannot contain NUL" ), + test.span (), ); -writeln! (kunit_macros,"{kunit_wrapper}" ).unwrap (); -writeln! ( - test_cases, -" ::kernel::kunit::kunit_case(::kernel::c_str!(\"{test}\"), {kunit_wrapper_fn_name})," - ) - .unwrap (); -writeln! ( - assert_macros, -r#" -/// Overrides the usual [`assert!`] macro with one that calls KUnit instead. -#[allow(unused)] -macro_rules! assert {{ -($cond:expr $(,)?) => {{{{ -kernel::kunit_assert!("{test}", "{path}", 0, $cond); -}}}} -}} - -/// Overrides the usual [`assert_eq!`] macro with one that calls KUnit instead. -#[allow(unused)] -macro_rules! assert_eq {{ -($left:expr, $right:expr $(,)?) => {{{{ -kernel::kunit_assert_eq!("{test}", "{path}", 0, $left, $right); -}}}} -}} -"# - ) - .unwrap (); - } + processed_items.push (parse_quote! { +unsafe extern "C" fn #kunit_wrapper_fn_name (_test: *mut ::kernel::bindings::kunit ) { + (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED; -writeln! (kunit_macros).unwrap (); -writeln! ( - kunit_macros, -"static mut TEST_CASES: [::kernel::bindings::kunit_case; {}] = [\n{test_cases} ::kernel::kunit::kunit_case_null(),\n];" , - num_tests +1 - ) - .unwrap (); - -writeln! ( - kunit_macros, -"::kernel::kunit_unsafe_test_suite!({attr}, TEST_CASES);" - ) - .unwrap (); - -// Remove the `#[test]` macros. -// We do this at a token level, in order to preserve span information. -let mut new_body =vec! []; -let mut body_it = body.stream ().into_iter (); - -while let Some (token) = body_it.next () { -match token { -TokenTree ::Punct (ref c)if c.as_char () =='#' =>match body_it.next () { -Some (TokenTree ::Group (group))if group.to_string () =="[test]" => (), -Some (next) => { - new_body.extend ([token, next]); - } - _ => { - new_body.push (token); +// Append any `cfg` attributes the user might have written on their tests so we +// don't attempt to call them when they are `cfg`'d out. An extra `use` is used +// here to reduce the length of the assert message. + #(#cfg_attrs)* + { + (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS; +use ::kernel::kunit::is_test_result_ok; +assert! (is_test_result_ok (#test ())); } - }, - _ => { - new_body.push (token); } - } - } + }); -let mut final_body =TokenStream ::new (); - final_body.extend ::<TokenStream >(assert_macros.parse ().unwrap ()); - final_body.extend (new_body); - final_body.extend ::<TokenStream >(kunit_macros.parse ().unwrap ()); - - tokens.push (TokenTree ::Group (Group ::new (Delimiter ::Brace , final_body))); + test_cases.push (quote! ( + ::kernel::kunit::kunit_case (#test_cstr, #kunit_wrapper_fn_name) + )); + } - tokens.into_iter ().collect () +let num_tests_plus_1 = test_cases.len () +1 ; + processed_items.push (parse_quote! { +static mut TEST_CASES : [::kernel::bindings::kunit_case ; #num_tests_plus_1] = [ + #(#test_cases,)* + ::kernel::kunit::kunit_case_null (), + ]; + }); + processed_items.push (parse_quote! { + ::kernel::kunit_unsafe_test_suite!(#test_suite,TEST_CASES ); + }); + + module.content =Some ((module_brace, processed_items)); +Ok (module.to_token_stream ()) }rust/macros/lib.rs +4
-3@@ -481,6 +481,8 @@ pub fn paste(input: TokenStream) -> TokenStream {/// } /// ``` #[proc_macro_attribute] -pub fn kunit_tests (attr:TokenStream , ts:TokenStream ) ->TokenStream { - kunit::kunit_tests (attr.into (), ts.into ()).into () +pub fn kunit_tests (attr:TokenStream , input:TokenStream ) ->TokenStream { + kunit::kunit_tests (parse_macro_input! (attr),parse_macro_input! (input)) + .unwrap_or_else (|e| e.into_compile_error ()) + .into () } --2.51.2
Re: [PATCH v3 11/12] rust: kunit: use `pin_init::zeroed` instead of custom null value
In reply to Gary GuoQuoted 10 lines · Gary Guo
From: Gary Guo <[email protected]>
The last null element can be created (constly) using
pin_init::zeroed, so prefer to use it instead of adding a custom way of building it.Reviewed-by Tamir Duberstein <[email protected]>Reviewed-by Benno Lossin <[email protected]>Signed-off-by Gary Guo <[email protected]>---
(Do note, however, that this conflicts with
https://lore.kernel.org/rust-for-linux/[email protected]/)
Cheers,
-- David
Quoted 80 lines
rust/kernel/kunit.rs | 26 +------------------------- rust/macros/kunit.rs | 4 ++-- 2 files changed, 3 insertions(+), 27 deletions(-)rust/kernel/kunit.rs +1
-25@@ -192,9 +192,6 @@ pub fn is_test_result_ok(t: impl TestResult) -> bool { }/// Represents an individual test case. -/// -/// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of valid test cases. -/// Use [`kunit_case_null`] to generate such a delimiter. #[doc(hidden)] pub const fn kunit_case ( name: &'static kernel::str::CStr,@@ -215,27 +212,6 @@ pub const fn kunit_case( } } -/// Represents the NULL test case delimiter. -/// -/// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of test cases. This -/// function returns such a delimiter. -#[doc(hidden)] -pub const fn kunit_case_null () -> kernel::bindings::kunit_case { - kernel::bindings::kunit_case { - run_case:None , - name: core::ptr::null_mut (), - generate_params:None , - attr: kernel::bindings::kunit_attributes { - speed: kernel::bindings::kunit_speed_KUNIT_SPEED_NORMAL, - }, - status: kernel::bindings::kunit_status_KUNIT_SUCCESS, - module_name: core::ptr::null_mut (), - log: core::ptr::null_mut (), - param_init:None , - param_exit:None , - } -} -/// Registers a KUnit test suite. /// /// # Safety @@ -254,7 +230,7 @@ pub const fn kunit_case_null() -> kernel::bindings::kunit_case {/// /// static mut KUNIT_TEST_CASES: [kernel::bindings::kunit_case; 2] = [ /// kernel::kunit::kunit_case(kernel::c_str!("name"), test_fn), -/// kernel::kunit::kunit_case_null(), +/// pin_init::zeroed(), /// ]; /// kernel::kunit_unsafe_test_suite!(suite_name, KUNIT_TEST_CASES); /// ``` rust/macros/kunit.rs +2
-3@@ -74,7 +74,7 @@ pub(crate) fn kunit_tests(test_suite: Ident, mut module: ItemMod) -> Result<Toke// static mut TEST_CASES: [::kernel::bindings::kunit_case; 3] = [ // ::kernel::kunit::kunit_case(::kernel::c_str!("foo"), kunit_rust_wrapper_foo), // ::kernel::kunit::kunit_case(::kernel::c_str!("bar"), kunit_rust_wrapper_bar), -// ::kernel::kunit::kunit_case_null(), +// ::pin_init::zeroed(), // ]; // // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES); @@ -159,7 +159,7 @@ macro_rules! assert_eq { processed_items.push (parse_quote! {static mut TEST_CASES : [::kernel::bindings::kunit_case ; #num_tests_plus_1] = [ #(#test_cases,)* - ::kernel::kunit::kunit_case_null (), + ::pin_init::zeroed (), ]; }); processed_items.push (parse_quote! { --2.51.2
Miguel Ojeda
AppliedQuoted 14 lines · Gary Guo
This series convert Rust proc macros that we have with
syn, and replace the customquote!macro that we have with the vendoredquote!macro. Thepin-initmacros are not convertedsynyet; Benno has a work in progress in converting them. They're however converted to usequoteandproc-macro2crate so our customquote!macro can be removed.Overall this improves the robustness of the macros as we have precise parsing of the AST rather than relying on heuristics to extract needed information from there. This is also a quality-of-life improvement to those using language servers (e.g. Rust analyzer) as the span information of the proc macros are now preserved which allows the "jump-to-definition" feature to work, even when used on completely custom macros such as
module!.
Quoted 2 lines
Miguel gave a very good explaination on why
synis a good idea in the patch series that introduced it [1], which I shall not repeat here.
Applied to rust-next -- thanks everyone!
[ Pass C string to match commit 6c37b6841a92 ("rust: kunit: replace
kernel::c_str! with C-Strings"). - Miguel ]
I dropped the first patch as it came from Benno's PR, and for the last one I took the v2.
I also took the cover letter and used it as a merge commit to give some context (with some typos/small rewording done -- I pinged Gary privately to double-check).
I may start doing that for larger series and similar, so cover letters that give proper context like this one are great.
Please double-check things look good!
Cheers,
Miguel