Skip to contentSkip to conversation

rust: proc-macro2: enable `proc_macro_span` feature

PATCH Applied Gary Guo 3 participants 6 messages Sep 21 – Sep 30 Rust

lore
Share & tools
lore

Gary Guo

author
#0 lore raw

PATCH 1/1 rust: proc-macro2: enable `proc_macro_span` feature

From: Gary Guo <[email protected]>

The Span::join function is very important in generating useful diagnostics. When invoking syn::Spanned::span, it attempts to return a overall span that covers the whole expression. If the join function is unavailable, however, only the span of the first token is generated.

This causes worse diagnostics quality for all macros, notably pin-init, when compared to the pin-init's own diagnostics test suite with the feature enabled.

For example, this diagnostic refers to only a fragment of path and not the full one:

    error: The field index `1` of type `PhantomPinned` only has an effect if it has the `#[pin]` attribute
     --> tests/ui/compile-fail/pin_data/tuple_struct_missing_pin_phantom.rs:6:20
      |
    6 | struct Tuple<T>(T, core::marker::PhantomPinned);
      |                    ^^^^

The special "<-" syntax gets parsed as two tokens, so they are only partially marked in diagnostics:

    error: `<-` is not supported in tuple constructor syntax; name the fields by index instead, e.g. `Type { 0 <- initializer, 1: value }`
     --> tests/ui/compile-fail/init/no_tuple_paren_arrow.rs:7:29
      |
    7 |     let _ = pin_init!(Tuple(<- 1, 2));
      |                             ^

In the upcoming pin-init self-reference series, there are also custom diagnostics that wish to use the span of the full type instead of its first token.

Thus, enable the proc_macro_span feature. This does make use of the unstable Span::join function, but this is for producing diagnostics only, so we do not have a hard reliance on the feature. A very small tweak of proc-macro2 vendored code is needed to make the code compile for 1.85.

---
Miguel, please let me know if you're okay with the proc-macro2 modification.

I'd like to take this via pin-init-next if possible.

---
 rust/Makefile                             | 1 +
 rust/proc-macro2/README.md                | 3 ++-
 rust/proc-macro2/probe/proc_macro_span.rs | 8 --------
 3 files changed, 3 insertions(+), 9 deletions(-)
rust/Makefile +1 -0
index da1a7409d984..5ee9c30c324a 100644
@@ -100,6 +100,7 @@ zerocopy-envs := \
 proc_macro2-cfgs := \
     feature="proc-macro" \
     wrap_proc_macro \
+    proc_macro_span \
     $(if $(call rustc-min-version,108800),proc_macro_span_file proc_macro_span_location)
 
 proc_macro2-flags := \
rust/proc-macro2/README.md +2 -1
index af044fee4f59..c63a7434217c 100644
@@ -4,7 +4,8 @@ These source files come from the Rust `proc-macro2` crate, version
 1.0.101 (released 2025-08-16), hosted in the
 <https://github.com/dtolnay/proc-macro2> repository, licensed under
 "Apache-2.0 OR MIT" and only modified to add the SPDX license
-identifiers and to remove the `unicode-ident` dependency.
+identifiers, to remove the `unicode-ident` dependency, and to build with 1.85
+with nightly features enabled.
 
 For copyright details, please see:
 
rust/proc-macro2/probe/proc_macro_span.rs +0 -8
index 892a7eb3e5a0..fc7dfb4ac156 100644
@@ -32,14 +32,6 @@ pub fn column(this: &Span) -> usize {
     this.column()
 }
 
-pub fn file(this: &Span) -> String {
-    this.file()
-}
-
-pub fn local_file(this: &Span) -> Option<PathBuf> {
-    this.local_file()
-}
-
 pub fn join(this: &Span, other: Span) -> Option<Span> {
     this.join(other)
 }

base-commit: dfb6a037fd586f1ffcba3b143dab58f5c88294d1

Signature · 1 line
2.54.0
#1 lore raw
Quoted 5 lines · Gary Guo

Miguel, please let me know if you're okay with the proc-macro2 modification.

I'd like to take this via pin-init-next if possible.

If you think the diagnostics improvements are worth it, then I guess it is fine. The ones in the commit message do not seem like a big deal, but I assume the upcoming ones you mention in pin-init are bigger improvements?

(I would also probably had shown the improved example outputs in the commit message for comparison; and especially one of the upcoming ones if those are more important to get right)

If so, then please feel free to pick it up, thanks! The change itself looks good.

I would also wrap the README.md like

identifiers, to remove the unicode-ident dependency, and to build with 1.85 with nightly features enabled.

to keep it like the previous lines.

Nit: "a overall" -> "an overall"

Cheers,
Miguel

reviewer
#2 lore raw
Quoted 11 lines · Miguel Ojeda
Quoted 5 lines · Gary Guo

Miguel, please let me know if you're okay with the proc-macro2 modification.

I'd like to take this via pin-init-next if possible.

If you think the diagnostics improvements are worth it, then I guess it is fine. The ones in the commit message do not seem like a big deal, but I assume the upcoming ones you mention in pin-init are bigger improvements?

Some examples from pin-init's diagnostics test suite (latest development tree, w/ selfref feature):

Suggestion not matching the quoted span:

    error: expected nothing or `..Zeroable::init_zeroed()`.
      --> tests/ui/compile-fail/zeroable/invalid_spread.rs:15:11
       |
    15 |         ..MyZeroable::init_zeroed()
       |           ^^^^^^^^^^

vs:

    error: expected nothing or `..Zeroable::init_zeroed()`.
      --> tests/ui/compile-fail/zeroable/invalid_spread.rs:15:9
       |
    15 |         ..MyZeroable::init_zeroed()
       |         ^^^^^^^^^^^^^^^^^^^^^^^^^^^

Attribute diagnostics confusingly point to the #:

    error: `#[pin]` attribute specified more than once
     --> tests/ui/compile-fail/pin_data/twice_pin.rs:6:5
      |
    6 |     #[pin]
      |     ^

vs:

      error: `#[pin]` attribute specified more than once
     --> tests/ui/compile-fail/pin_data/twice_pin.rs:6:5
      |
    6 |     #[pin]
      |     ^^^^^^

Self-ref pin-init variance check points to the ADT that is not the problem. In this case, the dyn is causing invariance. There's no way to actually point to that specific part, so pin-init wants to point to the whole type and let user decide. Without join, we can only point to the first token, which may confuse user to think that is the issue:

error: lifetime may not live long enough
--> tests/ui/compile-fail/pin_data/selfref_covariant_check.rs:5:14
|
3 | #[pin_data]
| ----------- in this procedural macro expansion
4 | struct SelfRef {
5 | not_cov: Box<dyn Fn(&'str str) -> bool + 'str>,
| ^^^
| |
| lifetime '__short defined here
| lifetime '__long defined here
| function was supposed to return data with lifetime '__long but it is returning data with lifetime '__short
|
= help: consider adding the following bound: '__short: '__long
= note: this error originates in the attribute macro pin_data (in Nightly builds, run with -Z macro-backtrace for more info)

vs:

error: lifetime may not live long enough
--> tests/ui/compile-fail/pin_data/selfref_covariant_check.rs:5:14
|
3 | #[pin_data]
| ----------- in this procedural macro expansion
4 | struct SelfRef {
5 | not_cov: Box<dyn Fn(&'str str) -> bool + 'str>,
| ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
| |
| lifetime '__short defined here
| lifetime '__long defined here
| function was supposed to return data with lifetime '__long but it is returning data with lifetime '__short
|
= help: consider adding the following bound: '__short: '__long
= note: this error originates in the attribute macro pin_data (in Nightly builds, run with -Z macro-backtrace for more info)

For some spans we try to point to the full operation, where first span can be confusing to user:

error[E0277]: ? couldn't convert the error to std::alloc::AllocError
--> tests/ui/compile-fail/init/no_error_coercion.rs:18:15
|
18 | bar <- init!(Bar { b: 42 }),
| --^
| | |
| | the trait From<Infallible> is not implemented for std::alloc::AllocError
| this can't be annotated with ? because it has type Result<_, Infallible>
|
= note: the question mark operation (?) implicitly performs a conversion on the error value using the From trait = note: this error originates in the macro init (in Nightly builds, run with -Z macro-backtrace for more info)

vs:

error[E0277]: ? couldn't convert the error to std::alloc::AllocError
--> tests/ui/compile-fail/init/no_error_coercion.rs:18:15
|
18 | bar <- init!(Bar { b: 42 }),
| --^------------------------
| | |
| | the trait From<Infallible> is not implemented for std::alloc::AllocError
| this can't be annotated with ? because it has type Result<_, Infallible>
|
= note: the question mark operation (?) implicitly performs a conversion on the error value using the From trait = note: this error originates in the macro init (in Nightly builds, run with -Z macro-backtrace for more info)

Another thing is that pin-init's diagnostics test suite is built with Cargo, and the build script of proc-macro2 will probe if the nightly feature is available and enable proc_macro_span automatically. So if this is not enabled on RfL side, the diagnostics can diverge from what we expect.

I think the self-ref one is the main motivating factor, because the diagnostics with that can be confusing in general. That said, I have improved it much over the last week, it seems that it is not *that* needed anymore.

So, none of these are really big deals and that can be said to diagnostics in general. But I think the change needed here is small enough that any improvement can be used to justify that. I also submitted the proc-macro2 change upstream
too: https://github.com/dtolnay/proc-macro2/pull/542; if that is merged and we later update proc-macro2, we should get down to just a single Makefile line change.

Best,
Gary

Quoted 21 lines

(I would also probably had shown the improved example outputs in the commit message for comparison; and especially one of the upcoming ones if those are more important to get right)

If so, then please feel free to pick it up, thanks! The change itself looks good.

I would also wrap the README.md like

identifiers, to remove the unicode-ident dependency, and to build with 1.85 with nightly features enabled.

to keep it like the previous lines.

Nit: "a overall" -> "an overall"

Cheers,
Miguel

#3 lore raw
Quoted 5 lines · Gary Guo

Another thing is that pin-init's diagnostics test suite is built with Cargo, and the build script of proc-macro2 will probe if the nightly feature is available and enable proc_macro_span automatically. So if this is not enabled on RfL side, the diagnostics can diverge from what we expect.

Doesn't sound fun...

Quoted 3 lines

I think the self-ref one is the main motivating factor, because the diagnostics with that can be confusing in general. That said, I have improved it much over the last week, it seems that it is not *that* needed anymore.

Yeah, that one looks more important to have indeed.

Quoted 6 lines

So, none of these are really big deals and that can be said to diagnostics in general. But I think the change needed here is small enough that any improvement can be used to justify that. I also submitted the proc-macro2 change upstream
too: https://github.com/dtolnay/proc-macro2/pull/542; if that is merged and we later update proc-macro2, we should get down to just a single Makefile line change.

Ok, in that case (plus what you mention above and the examples -- thanks for those, by the way), I think it is fine either way.

I added it preemptively to the "nice to have" section in #2:

https://github.com/Rust-for-Linux/linux/issues/2

Cheers,
Miguel

Acked-by
#4 lore raw
Quoted 3 lines · Miguel Ojeda

Ok, in that case (plus what you mention above and the examples -- thanks for those, by the way), I think it is fine either way.

Of course, please feel free to add:

Cheers,
Miguel

reviewer Applied
#5 lore raw
In reply to Gary Guo
Quoted 33 lines · Gary Guo

From: Gary Guo <[email protected]>

The Span::join function is very important in generating useful diagnostics. When invoking syn::Spanned::span, it attempts to return a overall span that covers the whole expression. If the join function is unavailable, however, only the span of the first token is generated.

This causes worse diagnostics quality for all macros, notably pin-init, when compared to the pin-init's own diagnostics test suite with the feature enabled.

For example, this diagnostic refers to only a fragment of path and not the full one:

    error: The field index `1` of type `PhantomPinned` only has an effect if it has the `#[pin]` attribute
     --> tests/ui/compile-fail/pin_data/tuple_struct_missing_pin_phantom.rs:6:20
      |
    6 | struct Tuple<T>(T, core::marker::PhantomPinned);
      |                    ^^^^

The special "<-" syntax gets parsed as two tokens, so they are only partially marked in diagnostics:

    error: `<-` is not supported in tuple constructor syntax; name the fields by index instead, e.g. `Type { 0 <- initializer, 1: value }`
     --> tests/ui/compile-fail/init/no_tuple_paren_arrow.rs:7:29
      |
    7 |     let _ = pin_init!(Tuple(<- 1, 2));
      |                             ^

In the upcoming pin-init self-reference series, there are also custom diagnostics that wish to use the span of the full type instead of its first token.

Quoted 6 lines

Thus, enable the proc_macro_span feature. This does make use of the unstable Span::join function, but this is for producing diagnostics only, so we do not have a hard reliance on the feature. A very small tweak of proc-macro2 vendored code is needed to make the code compile for 1.85.

Applied to pin-init-next. Thanks all!

Best,
Gary

Quoted 10 lines

---
Miguel, please let me know if you're okay with the proc-macro2 modification.

I'd like to take this via pin-init-next if possible.

---
 rust/Makefile                             | 1 +
 rust/proc-macro2/README.md                | 3 ++-
 rust/proc-macro2/probe/proc_macro_span.rs | 8 --------
 3 files changed, 3 insertions(+), 9 deletions(-)