rust: add `TryFrom` and `Into` derive macros
cover rust: add `TryFrom` and `Into` derive macros
This patch series introduces derive macros for TryFrom and Into traits.
A few enhancements were made to the custom quote!() macro to write the derive macro. These include support for additional punctuation tokens and a fix for an unused variable warning when quoting simple forms. Detailed information about these enhancements is provided in the relevant patches.
This series builds on the previous work [1], where the FromPrimitive trait was considered too heavy for the current use cases. In response to the emerging need for functionality similar to ToPrimitive, this series also implements the Into derive macro.
The original discussion can be found on Zulip [2].
[1] https://lore.kernel.org/rust-for-linux/[email protected]/
[2] https://rust-for-linux.zulipchat.com/#narrow/channel/288089/topic/x/near/524335626
Jesung Yang (4):
rust: macros: extend custom quote!() macro
rust: macros: prefix variable span with underscore
rust: macro: add derive macro for TryFrom
rust: macro: add derive macro for Into
rust/macros/convert.rs | 361 +++++++++++++++++++++++++++++++++++++++++ rust/macros/lib.rs | 239 +++++++++++++++++++++++++++ rust/macros/quote.rs | 40 ++++- 3 files changed, 638 insertions(+), 2 deletions(-) create mode 100644 rust/macros/convert.rs
base-commit: dff64b072708ffef23c117fa1ee1ea59eb417807
Signature · 1 line
2.39.5
PATCH 1/4 rust: macros: extend custom `quote!()` macro
Implement the ToTokens trait for &T where T implements ToTokens. This allows users to use the quote!() macro with references directly, avoiding the need to clone values.
Extend the quote_spanned!() macro to support additional punctuation
tokens: ->, <, >, and ==. This symbols are commonly needed when dealing with functions, generic bounds, and equality comparisons.
--- rust/macros/quote.rs | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
rust/macros/quote.rs +36 -0
@@ -7,6 +7,12 @@ pub(crate) trait ToTokens {fn to_tokens (&self, tokens: &mut TokenStream ); } +impl <T :ToTokens >ToTokens for &T { +fn to_tokens (&self, tokens: &mut TokenStream ) { + (*self).to_tokens (tokens); + } +} +impl <T :ToTokens >ToTokens for Option <T > {fn to_tokens (&self, tokens: &mut TokenStream ) {if let Some (v) = self {
@@ -144,6 +150,36 @@ macro_rules! quote_spanned { ));quote_spanned! (@proc $v $span $($tt)*); }; + (@proc $v:ident $span:ident -> $($tt:tt)*) => { + $v.push (::proc_macro::TokenTree ::Punct ( + ::proc_macro::Punct ::new ('-' , ::proc_macro::Spacing ::Joint ) + )); + $v.push (::proc_macro::TokenTree ::Punct ( + ::proc_macro::Punct ::new ('>' , ::proc_macro::Spacing ::Alone ) + )); +quote_spanned! (@proc $v $span $($tt)*); + }; + (@proc $v:ident $span:ident < $($tt:tt)*) => { + $v.push(::proc_macro::TokenTree::Punct( + ::proc_macro::Punct::new('<', ::proc_macro::Spacing::Alone) + )); + quote_spanned!(@proc $v $span $($tt)*); + }; + (@proc $v:ident $span:ident > $($tt:tt)*) => { + $v.push(::proc_macro::TokenTree::Punct( + ::proc_macro::Punct::new('>', ::proc_macro::Spacing::Alone) + )); + quote_spanned!(@proc $v $span $($tt)*); + }; + (@proc $v:ident $span:ident == $($tt:tt)*) => { + $v.push(::proc_macro::TokenTree::Punct( + ::proc_macro::Punct::new('=', ::proc_macro::Spacing::Joint) + )); + $v.push(::proc_macro::TokenTree::Punct( + ::proc_macro::Punct::new('=', ::proc_macro::Spacing::Alone) + )); + quote_spanned!(@proc $v $span $($tt)*); + }; (@proc $v:ident $span:ident = $($tt:tt)*) => { $v.push(::proc_macro::TokenTree::Punct( ::proc_macro::Punct::new('=', ::proc_macro::Spacing::Alone)
Signature · 1 line
2.39.5
Alexandre Courbot
reviewerRe: [PATCH 1/4] rust: macros: extend custom `quote!()` macro
Quoted 7 lines · Jesung Yang
Implement the
ToTokenstrait for&TwhereTimplementsToTokens. This allows users to use thequote!()macro with references directly, avoiding the need to clone values.Extend the
quote_spanned!()macro to support additional punctuation
tokens:->,<,>, and==. This symbols are commonly needed when dealing with functions, generic bounds, and equality comparisons.
These two paragraphs feel like they should be split into separate patches, as they are not related to one another AFAICT.
PATCH 2/4 rust: macros: prefix variable `span` with underscore
In reply to Jesung YangPrefix the variable span in quote_spanned!() macro with an underscore to silence unused variable warnings.
The warning occurs when the macro is used without any uninterpolated identifiers. For example:
// Triggers a warning: "unused variable: `span`"
quote! { #foo }
// This is fine
quote! { Some(#foo) }There is no good reason to disallow such quoting patterns, so fix the warning instead.
--- rust/macros/quote.rs | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-)
rust/macros/quote.rs +2 -2
@@ -57,8 +57,8 @@ macro_rules! quote_spanned {#[allow(clippy::vec_init_then_push)] { tokens = ::std::vec::Vec ::new (); -let span = $span; -quote_spanned! (@proc tokensspan $($tt)*); +let _span = $span; +quote_spanned! (@proc tokens_span $($tt)*); } ::proc_macro::TokenStream ::from_iter (tokens) }};
Signature · 1 line
2.39.5
PATCH 3/4 rust: macro: add derive macro for `TryFrom`
In reply to Jesung YangIntroduce a procedural macro TryFrom to automatically implement the TryFrom trait for unit-only enums.
This reduces boilerplate in cases where numeric values need to be interpreted as relevant enum variants. This situation often arise when working with low-level data sources. A typical example is the Chipset enum in nova-core, where the value read from a GPU register should be mapped to a corresponding variant.
Since a pending RFC [1] proposes adding the syn crate [2] as a dependency, keep the parsing logic minimal.
--- rust/macros/convert.rs | 337 +++++++++++++++++++++++++++++++++++++++++ rust/macros/lib.rs | 124 +++++++++++++++ 2 files changed, 461 insertions(+) create mode 100644 rust/macros/convert.rs
rust/macros/convert.rs new file +337 -0
@@ -0,0 +1,337 @@ +// SPDX-License-Identifier: GPL-2.0 + +use proc_macro::{token_stream,Delimiter ,Ident ,Span ,TokenStream ,TokenTree }; +use std::iter::Peekable ; + +#[derive( Debug )] +struct TypeArgs { + helper:Vec <Ident >, + repr:Option <Ident >, +} + +const VALID_TYPES : [&str ;12 ] = [ +"u8" ,"u16" ,"u32" ,"u64" ,"u128" ,"usize" ,"i8" ,"i16" ,"i32" ,"i64" ,"i128" ,"isize" , +]; + +pub (crate )fn derive_try_from (input:TokenStream ) ->TokenStream { +derive (input) +} + +fn derive (input:TokenStream ) ->TokenStream { +let derive_target ="TryFrom" ; +let derive_helper ="try_from" ; + +let mut tokens = input.into_iter ().peekable (); + +let type_args =match parse_types (&mut tokens, derive_helper) { +Ok (type_args) => type_args, +Err (errs) =>return errs, + }; + +// Skip until the `enum` keyword, including the `enum` itself. +for ttin tokens.by_ref () { +if matches! (tt,TokenTree ::Ident (ident)if ident.to_string () =="enum" ) { +break ; + } + } + +let Some (TokenTree ::Ident (enum_ident)) = tokens.next ()else { +return format! ( +"::core::compile_error!(\"`#[derive({derive_target})]` can only \ +be applied to an enum\");" + ) + .parse ::<TokenStream >() + .unwrap (); + }; + +let mut errs =TokenStream ::new (); + +if matches! (tokens.peek (),Some (TokenTree ::Punct (p))if p.as_char () =='<' ) { + errs.extend ( +format! ( +"::core::compile_error!(\"`#[derive({derive_target})]` \ +does not support enums with generic parameters\");" + ) + .parse ::<TokenStream >() + .unwrap (), + ); + } + +let Some (variants_group) = tokens.find_map (|tt|match tt { +TokenTree ::Group (g)if g.delimiter () ==Delimiter ::Brace =>Some (g), + _ =>None , + })else { +unreachable! ("Enums have its corresponding body" ) + }; + +let enum_body_tokens = variants_group.stream ().into_iter ().peekable (); +let variants =match parse_enum_variants (enum_body_tokens, &enum_ident, derive_target) { +Ok (variants) => variants, +Err (new_errs) => { + errs.extend (new_errs); +return errs; + } + }; + +if !errs.is_empty () { +return errs; + } + +if type_args.helper.is_empty () { +// Extract the representation passed by `#[repr(...)]` if present. +// If nothing is specified, the default is `Rust` representation, +// which uses `isize` for the discriminant type. +// See: https://doc.rust-lang.org/reference/items/enumerations.html#r-items.enum.discriminant.repr-rust +let ty = type_args + .repr + .unwrap_or_else (||Ident ::new ("isize" ,Span ::mixed_site ())); +impl_try_from (&ty, &enum_ident, &variants) + }else { +let impls = type_args + .helper + .iter () + .map (|ty|impl_try_from (ty, &enum_ident, &variants)); +quote! { #(#impls)* } + } +} + +fn parse_types ( + attrs: &mut Peekable <token_stream::IntoIter >, + helper: &str , +) ->Result <TypeArgs ,TokenStream > { +let mut helper_args =vec! []; +let mut repr_arg =None ; + +// Scan only the attributes. As soon as we see a token that is +// not `#`, we know we have consumed all attributes. +while let Some (TokenTree ::Punct (p)) = attrs.peek () { +if p.as_char () !='#' { +unreachable! ("Outer attributes start with `#`" ); + } + attrs.next (); + +// The next token should be a `Group` delimited by brackets. +// (e.g., #[try_from(u8, u16)]) +// ^^^^^^^^^^^^^^^^^^^ +let Some (TokenTree ::Group (attr_group)) = attrs.next ()else { +unreachable! ("Outer attributes are surrounded by `[` and `]`" ); + }; + +let mut inner = attr_group.stream ().into_iter (); + +// Extract the attribute identifier. +// (e.g., #[try_from(u8, u16)]) +// ^^^^^^^^ +let attr_name =match inner.next () { +Some (TokenTree ::Ident (ident)) => ident.to_string (), + _ =>unreachable! ("Attributes have identifiers" ), + }; + +if attr_name == helper { +match parse_helper_args (inner, helper) { +Ok (v) => helper_args.extend_from_slice (&v), +Err (errs) =>return Err (errs), + } + }else if attr_name =="repr" { + repr_arg =parse_repr_args (inner); + } + } + +Ok (TypeArgs { + helper: helper_args, + repr: repr_arg, + }) +} + +fn parse_repr_args (mut tt_group:impl Iterator <Item =TokenTree >) ->Option <Ident > { +// The next token should be a `Group` delimited by parentheses. +// (e.g., #[repr(C, u8)]) +// ^^^^^^^ +let Some (TokenTree ::Group (args_group)) = tt_group.next ()else { +unreachable! ("`repr` attribute has at least one argument" ) + }; + +for argin args_group.stream () { +if let TokenTree ::Ident (type_ident) = arg { +if VALID_TYPES .contains (&type_ident.to_string ().as_str ()) { +return Some (type_ident); + } + } + } + +None +} + +fn parse_helper_args ( +mut tt_group:impl Iterator <Item =TokenTree >, + helper: &str , +) ->Result <Vec <Ident >,TokenStream > { +let mut errs =TokenStream ::new (); +let mut args =vec! []; + +// The next token should be a `Group`. +// (e.g., #[try_from(u8, u16)]) +// ^^^^^^^^^ +let Some (TokenTree ::Group (args_group)) = tt_group.next ()else { +return Err (format! ( +"::core::compile_error!(\"`{helper}` attribute expects at \ +least one integer type argument (e.g., `#[{helper}(u8)]`)\");" + ) + .parse ::<TokenStream >() + .unwrap ()); + }; + +let raw_args = args_group.stream (); +if raw_args.is_empty () { +return Err (format! ( +"::core::compile_error!(\"`{helper}` attribute expects at \ +least one integer type argument (e.g., `#[{helper}(u8)]`)\");" + ) + .parse ::<TokenStream >() + .unwrap ()); + } + +// Iterate over the attribute argument tokens to collect valid integer +// type identifiers. +let mut raw_args = raw_args.into_iter (); +while let Some (tt) = raw_args.next () { +let TokenTree ::Ident (type_ident) = ttelse { + errs.extend ( +format! ( +"::core::compile_error!(\"`{helper}` attribute expects \ +comma-separated integer type arguments \ +(e.g., `#[{helper}(u8, u16)]`)\");" + ) + .parse ::<TokenStream >() + .unwrap (), + ); +return Err (errs); + }; + +if VALID_TYPES .contains (&type_ident.to_string ().as_str ()) { + args.push (type_ident); + }else { + errs.extend ( +format! ( +"::core::compile_error!(\"`{type_ident}` in `{helper}` \ +attribute is not an integer type\");" + ) + .parse ::<TokenStream >() + .unwrap (), + ); + } + +match raw_args.next () { +Some (TokenTree ::Punct (p))if p.as_char () ==',' =>continue , +None =>break , +Some (_) => { + errs.extend ( +format! ( +"::core::compile_error!(\"`{helper}` attribute expects \ +comma-separated integer type arguments \ +(e.g., `#[{helper}(u8, u16)]`)\");" + ) + .parse ::<TokenStream >() + .unwrap (), + ); +return Err (errs); + } + } + } + +if !errs.is_empty () { +return Err (errs); + } + +Ok (args) +} + +fn parse_enum_variants ( +mut tokens:Peekable <token_stream::IntoIter >, + enum_ident: &Ident , + derive_target: &str , +) ->Result <Vec <Ident >,TokenStream > { +let mut errs =TokenStream ::new (); + +let mut variants =vec! []; + +if tokens.peek ().is_none () { + errs.extend ( +format! ( +"::core::compile_error!(\"`#[derive({derive_target})]` \ +does not support zero-variant enums\");" + ) + .parse ::<TokenStream >() + .unwrap (), + ); + } + +while let Some (tt) = tokens.next () { +// Skip attributes like `#[...]` if present. +if matches! (&tt,TokenTree ::Punct (p)if p.as_char () =='#' ) { + tokens.next (); +continue ; + } + +let TokenTree ::Ident (ident) = ttelse { +unreachable! ("Enum variants have its corresponding identifier" ); + }; + +// Reject tuple-like or struct-like variants. +if let Some (TokenTree ::Group (g)) = tokens.peek () { +let variant_kind =match g.delimiter () { +Delimiter ::Brace =>"struct-like" , +Delimiter ::Parenthesis =>"tuple-like" , + _ =>unreachable! ("Invalid enum variant syntax" ), + }; + errs.extend ( +format! ( +"::core::compile_error!(\"`#[derive({derive_target})]` does not \ +support {variant_kind} variant `{enum_ident}::{ident}`; \ +only unit variants are allowed\");" + ) + .parse ::<TokenStream >() + .unwrap (), + ); + } + +// Skip through the comma. +for ttin tokens.by_ref () { +if matches! (tt,TokenTree ::Punct (p)if p.as_char () ==',' ) { +break ; + } + } + + variants.push (ident); + } + +if !errs.is_empty () { +return Err (errs); + } + +Ok (variants) +} + +fn impl_try_from (ty: &Ident , enum_ident: &Ident , variants: &[Ident ]) ->TokenStream { +let param =Ident ::new ("value" ,Span ::mixed_site ()); + +let clauses = variants.iter ().map (|variant| { +quote! { +if #param ==Self ::#variantas #ty { + ::core::result::Result ::Ok (Self ::#variant ) + }else + } + }); + +quote! { +#[automatically_derived] +impl ::core::convert::TryFrom <#ty>for #enum_ident { +type Error = ::kernel::prelude::Error ; +fn try_from (#param: #ty) ->Result <Self ,Self ::Error > { + #(#clauses)* { + ::core::result::Result ::Err (::kernel::prelude::EINVAL ) + } + } + } + } +}
rust/macros/lib.rs +124 -0
133 lines not on this page · raw message
Signature · 1 line
2.39.5
PATCH 4/4 rust: macro: add derive macro for `Into`
In reply to Jesung YangIntroduce a procedural macro Into to automatically implement the Into trait for unit-only enums.
This reduces boilerplate in cases where enum variants need to be interpreted as relevant numeric values. A concrete example can be found in nova-core, where the register!() macro requires enum types used within it to be convertible via u32::from() [1].
Note that the macro actually generates From<E> for T implementations, where E is an enum identifier and T is an arbitrary integer type. This automatically provides the corresponding Into<T> for E implementations through the blanket implementation.
--- rust/macros/convert.rs | 36 ++++++++++--- rust/macros/lib.rs | 115 +++++++++++++++++++++++++++++++++++++++++ 2 files changed, 145 insertions(+), 6 deletions(-)
rust/macros/convert.rs +30 -6
@@ -3,6 +3,12 @@use proc_macro::{token_stream,Delimiter ,Ident ,Span ,TokenStream ,TokenTree };use std::iter::Peekable ; +#[derive( Debug )] +enum DeriveTarget { +TryFrom , +Into , +} +#[derive( Debug )] struct TypeArgs { helper:Vec <Ident >,
@@ -13,13 +19,20 @@ struct TypeArgs { "u8", "u16", "u32", "u64", "u128", "usize", "i8", "i16", "i32", "i64", "i128", "isize", ]; +pub (crate )fn derive_into (input:TokenStream ) ->TokenStream { +derive (input,DeriveTarget ::Into ) +} +pub (crate )fn derive_try_from (input:TokenStream ) ->TokenStream { -derive (input) +derive (input,DeriveTarget ::TryFrom ) } -fn derive (input:TokenStream ) ->TokenStream { -let derive_target ="TryFrom" ; -let derive_helper ="try_from" ; +fn derive (input:TokenStream , target:DeriveTarget ) ->TokenStream { +type ImplFn =fn (&Ident , &Ident , &[Ident ]) ->TokenStream ; +let (derive_target, derive_helper, impl_trait) =match target { +DeriveTarget ::TryFrom => ("TryFrom" ,"try_from" , impl_try_fromas ImplFn ), +DeriveTarget ::Into => ("Into" ,"into" , impl_intoas ImplFn ), + };let mut tokens = input.into_iter ().peekable ();
@@ -85,12 +98,12 @@ fn derive(input: TokenStream) -> TokenStream {let ty = type_args .repr .unwrap_or_else (||Ident ::new ("isize" ,Span ::mixed_site ())); -impl_try_from (&ty, &enum_ident, &variants) +impl_trait (&ty, &enum_ident, &variants) }else {let impls = type_args .helper .iter () - .map (|ty|impl_try_from (ty, &enum_ident, &variants)); + .map (|ty|impl_trait (ty, &enum_ident, &variants));quote! { #(#impls)* } } }
@@ -335,3 +348,14 @@ fn try_from(#param: #ty) -> Result<Self, Self::Error> { } } } + +fn impl_into (ty: &Ident , enum_ident: &Ident , _: &[Ident ]) ->TokenStream { +quote! { +#[automatically_derived] +impl ::core::convert::From <#enum_ident >for #ty { +fn from (value: #enum_ident ) -> #ty { + valueas #ty + } + } + } +}
rust/macros/lib.rs +115 -0
121 lines not on this page · raw message
Signature · 1 line
2.39.5
Quoted 14 lines · Jesung Yang
This patch series introduces derive macros for
TryFromandIntotraits.A few enhancements were made to the custom
quote!()macro to write the derive macro. These include support for additional punctuation tokens and a fix for an unused variable warning when quoting simple forms. Detailed information about these enhancements is provided in the relevant patches.This series builds on the previous work [1], where the
FromPrimitivetrait was considered too heavy for the current use cases. In response to the emerging need for functionality similar toToPrimitive, this series also implements theIntoderive macro.
Quoted 4 lines
The original discussion can be found on Zulip [2].
[1] https://lore.kernel.org/rust-for-linux/[email protected]/
[2] https://rust-for-linux.zulipchat.com/#narrow/channel/288089/topic/x/near/524335626
I have tried this series on nova-core and it allowed me to remove 150 lines of boiletplate just like that. As far as nova-core is concerned, this is a perfect fit.
For the larger kernel, where using the right type is essential and where we want to avoid error paths whenever possible, I also feel like this is a much better fit than the initial FromPrimitive. But it would be nice to hear other opinions on the matter.
I will try to review as much as my proc-macro knowledge allows me to, but for now:
Quoted 4 lines · Alexandre Courbot
I have tried this series on nova-core and it allowed me to remove 150 lines of boiletplate just like that. As far as nova-core is concerned, this is a perfect fit.
It would be nice to see the (possibly RFC) patch if you have it around!
Cheers,
Miguel
Alexandre Courbot
reviewerQuoted 7 lines · Miguel Ojeda
Quoted 4 lines · Alexandre Courbot
I have tried this series on nova-core and it allowed me to remove 150 lines of boiletplate just like that. As far as nova-core is concerned, this is a perfect fit.
It would be nice to see the (possibly RFC) patch if you have it around!
Posted it here! Hope this can help push the series forward.
https://lore.kernel.org/rust-for-linux/[email protected]/T/#u