kbuild: mark `rustc` (and others) invocations as recursive
Subsystems (4)
Miguel Ojeda
maintainerPATCH 1/2 kbuild: mark `rustc` (and others) invocations as recursive
rustc (like Cargo) may take advantage of the jobserver at any time (e.g. for backend parallelism, or eventually frontend too). In the kernel, we call rustc with -Ccodegen-units=1 (and -Zthreads is 1 so far), so we do not expect parallelism. However, in the upcoming Rust 1.76.0, a warning is emitted by rustc when it cannot connect to the jobserver it was passed (in many cases, but not all: compiling and --print sysroot do, but --version does not). And given GNU Make always passes the jobserver in the environment variable (even when a line is deemed non-recursive), rustc will end up complaining about it (in particular in Make 4.3 where there is only the simple pipe jobserver style).
One solution is to remove the jobserver from MAKEFLAGS. However, we can mark the lines with calls to rustc (and Cargo) as recursive, which looks simpler. This is what is now documented and recommended by rustc and allows us to be ready for the time we may use parallelism inside rustc (potentially now, if a user passes -Zthreads). Thus do so.
Similarly, do the same for rustdoc and cargo calls.
Finally, there is one case that the solution does not cover, which is the $(shell ...) call we have. Thus, for that one, set an empty MAKEFLAGS environment variable.
--- Makefile | 4 ++-- rust/Makefile | 48 +++++++++++++++++++++--------------------- scripts/Makefile.build | 8 +++---- scripts/Makefile.host | 2 +- 4 files changed, 31 insertions(+), 31 deletions(-)
Makefile +2 -2
@@ -1197,7 +1197,7 @@ prepare0: archprepare # All the preparing.. prepare: prepare0 ifdef CONFIG_RUST - $(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh + +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh $(Q)$(MAKE) $(build)=rust endif
@@ -1707,7 +1707,7 @@ $(DOC_TARGETS): # "Is Rust available?" target PHONY += rustavailable rustavailable: - $(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh && echo "Rust is available!" + +$(Q)$(CONFIG_SHELL) $(srctree)/scripts/rust_is_available.sh && echo "Rust is available!" # Documentation target #
rust/Makefile +24 -24
148 lines not on this page · raw message
scripts/Makefile.build +4 -4
@@ -290,7 +290,7 @@ quiet_cmd_rustc_o_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ cmd_rustc_o_rs = $(rust_common_cmd) --emit=obj=$@ $< $(obj)/%.o: $(src)/%.rs FORCE - $(call if_changed_dep,rustc_o_rs) + +$(call if_changed_dep,rustc_o_rs) quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ cmd_rustc_rsi_rs = \
@@ -298,19 +298,19 @@ quiet_cmd_rustc_rsi_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ command -v $(RUSTFMT) >/dev/null && $(RUSTFMT) $@ $(obj)/%.rsi: $(src)/%.rs FORCE - $(call if_changed_dep,rustc_rsi_rs) + +$(call if_changed_dep,rustc_rsi_rs) quiet_cmd_rustc_s_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ cmd_rustc_s_rs = $(rust_common_cmd) --emit=asm=$@ $< $(obj)/%.s: $(src)/%.rs FORCE - $(call if_changed_dep,rustc_s_rs) + +$(call if_changed_dep,rustc_s_rs) quiet_cmd_rustc_ll_rs = $(RUSTC_OR_CLIPPY_QUIET) $(quiet_modtag) $@ cmd_rustc_ll_rs = $(rust_common_cmd) --emit=llvm-ir=$@ $< $(obj)/%.ll: $(src)/%.rs FORCE - $(call if_changed_dep,rustc_ll_rs) + +$(call if_changed_dep,rustc_ll_rs) # Compile assembler sources (.S) # ---------------------------------------------------------------------------
scripts/Makefile.host +1 -1
@@ -156,7 +156,7 @@ quiet_cmd_host-rust = HOSTRUSTC $@ cmd_host-rust = \ $(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $< $(host-rust): $(obj)/%: $(src)/%.rs FORCE - $(call if_changed_dep,host-rust) + +$(call if_changed_dep,host-rust) targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \ $(host-cxxmulti) $(host-cxxobjs) $(host-rust)
base-commit: f090f0d0eea9666a96702b29bc9a64cbabee85c5 --
2.43.0
Miguel Ojeda
maintainerPATCH 2/2 rust: upgrade to Rust 1.76.0
This is the next upgrade to the Rust toolchain, from 1.75.0 to 1.76.0 (i.e. the latest) [1].
See the upgrade policy [2] and the comments on the first upgrade in commit 3ed03f4da06e ("rust: upgrade to Rust 1.68.2").
# Unstable features
No unstable features that we use were stabilized in Rust 1.76.0.
The only unstable features allowed to be used outside the kernel crate are still new_uninit,offset_of, though other code to be upstreamed may increase the list.
Please see [3] for details.
# Required changes
rustc (and others) now warns when it cannot connect to the Make jobserver, thus mark those invocations as recursive as needed. Please see the previous commit for details.
# Other changes
Rust 1.76.0 does not emit the .debug_pub{names,types} sections anymore for DWARFv4 [4][5]. For instance, in the uncompressed debug info case, this debug information took:
samples/rust/rust_minimal.o ~64 KiB (~18% of total object size)
rust/kernel.o ~92 KiB (~15%)
rust/core.o ~114 KiB ( ~5%)
In the compressed debug info (zlib) case:
samples/rust/rust_minimal.o ~11 KiB (~6%)
rust/kernel.o ~17 KiB (~5%)
rust/core.o ~21 KiB (~1.5%)In addition, the rustc_codegen_gcc backend now does not emit the .eh_frame section when compiling under -Cpanic=abort [6], thus removing the need for the patch in the CI to compile the kernel [7]. Moreover, it also now emits the .comment section too [6].
# alloc upgrade and reviewing
The vast majority of changes are due to our alloc fork being upgraded at once.
There are two kinds of changes to be aware of: the ones coming from upstream, which we should follow as closely as possible, and the updates needed in our added fallible APIs to keep them matching the newer infallible APIs coming from upstream.
Instead of taking a look at the diff of this patch, an alternative approach is reviewing a diff of the changes between upstream alloc and the kernel's. This allows to easily inspect the kernel additions only, especially to check if the fallible methods we already have still match the infallible ones in the new version coming from upstream.
Another approach is reviewing the changes introduced in the additions in the kernel fork between the two versions. This is useful to spot potentially unintended changes to our additions.
To apply these approaches, one may follow steps similar to the following to generate a pair of patches that show the differences between upstream Rust and the kernel (for the subset of alloc we use) before and after applying this patch:
# Get the difference with respect to the old version.
git -C rust checkout $(linux/scripts/min-tool-version.sh rustc) git -C linux ls-tree -r --name-only HEAD -- rust/alloc |
cut -d/ -f3- |
grep -Fv README.md |
xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH
git -C linux diff --patch-with-stat --summary -R > old.patch git -C linux restore rust/alloc
# Apply this patch.
git -C linux am rust-upgrade.patch
# Get the difference with respect to the new version.
git -C rust checkout $(linux/scripts/min-tool-version.sh rustc) git -C linux ls-tree -r --name-only HEAD -- rust/alloc |
cut -d/ -f3- |
grep -Fv README.md |
xargs -IPATH cp rust/library/alloc/src/PATH linux/rust/alloc/PATH
git -C linux diff --patch-with-stat --summary -R > new.patch git -C linux restore rust/alloc
Now one may check the new.patch to take a look at the additions (first approach) or at the difference between those two patches (second approach). For the latter, a side-by-side tool is recommended.
--- Documentation/process/changes.rst | 2 +- rust/alloc/alloc.rs | 3 ++ rust/alloc/boxed.rs | 14 +++++-- rust/alloc/collections/mod.rs | 1 + rust/alloc/lib.rs | 8 ++-- rust/alloc/raw_vec.rs | 58 +++++++++++++++++++-------- rust/alloc/vec/into_iter.rs | 16 +++++--- rust/alloc/vec/mod.rs | 65 ++++++++++++++++++++++++------- scripts/min-tool-version.sh | 2 +- 9 files changed, 125 insertions(+), 44 deletions(-)
Documentation/process/changes.rst +1 -1
@@ -31,7 +31,7 @@ you probably needn't concern yourself with pcmciautils. ====================== =============== ======================================== GNU C 5.1 gcc --version Clang/LLVM (optional) 11.0.0 clang --version -Rust (optional) 1.75.0 rustc --version +Rust (optional) 1.76.0 rustc --version bindgen (optional) 0.65.1 bindgen --version GNU make 3.82 make --version bash 4.2 bash --version
rust/alloc/alloc.rs +3 -0
@@ -425,12 +425,14 @@ pub unsafe fn __rdl_oom(size: usize, _align: usize) -> ! { } } +#[cfg(not(no_global_oom_handling))] /// Specialize clones into pre-allocated, uninitialized memory. /// Used by `Box::clone` and `Rc`/`Arc::make_mut`. pub (crate )trait WriteCloneIntoRaw :Sized {unsafe fn write_clone_into_raw (&self, target: *mut Self ); } +#[cfg(not(no_global_oom_handling))] impl <T :Clone >WriteCloneIntoRaw for T {#[inline] default unsafe fn write_clone_into_raw (&self, target: *mut Self ) {
@@ -440,6 +442,7 @@ impl<T: Clone> WriteCloneIntoRaw for T { } } +#[cfg(not(no_global_oom_handling))] impl <T :Copy >WriteCloneIntoRaw for T {#[inline] unsafe fn write_clone_into_raw (&self, target: *mut Self ) {
rust/alloc/boxed.rs +11 -3
@@ -1042,10 +1042,18 @@ impl<T: ?Sized, A: Allocator> Box<T, A> {/// use std::ptr; /// /// let x = Box::new(String::from("Hello")); -/// let p = Box::into_raw(x); +/// let ptr = Box::into_raw(x); /// unsafe { -/// ptr::drop_in_place(p); -/// dealloc(p as *mut u8, Layout::new::<String>()); +/// ptr::drop_in_place(ptr); +/// dealloc(ptr as *mut u8, Layout::new::<String>()); +/// } +/// ``` +/// Note: This is equivalent to the following: +/// ``` +/// let x = Box::new(String::from("Hello")); +/// let ptr = Box::into_raw(x); +/// unsafe { +/// drop(Box::from_raw(ptr)); /// } /// ``` ///
rust/alloc/collections/mod.rs +1 -0
@@ -150,6 +150,7 @@ fn fmt(/// An intermediate trait for specialization of `Extend`. #[doc(hidden)] +#[cfg(not(no_global_oom_handling))] trait SpecExtend <I :IntoIterator > {/// Extends `self` with the contents of the given iterator. fn spec_extend (&mut self, iter:I );
rust/alloc/lib.rs +4 -4
@@ -80,8 +80,8 @@ not(no_sync), target_has_atomic = "ptr" ))] -#![cfg_attr(not(bootstrap), doc(rust_logo))] -#![cfg_attr(not(bootstrap), feature(rustdoc_internals))] +#![doc(rust_logo)] +#![feature(rustdoc_internals)] #![no_std] #![needs_allocator] // Lints:
@@ -142,7 +142,6 @@#![feature(maybe_uninit_uninit_array)] #![feature(maybe_uninit_uninit_array_transpose)] #![feature(pattern)] -#![feature(ptr_addr_eq)] #![feature(ptr_internals)] #![feature(ptr_metadata)] #![feature(ptr_sub_ptr)]
@@ -157,6 +156,7 @@#![feature(std_internals)] #![feature(str_internals)] #![feature(strict_provenance)] +#![feature(trusted_fused)] #![feature(trusted_len)] #![feature(trusted_random_access)] #![feature(try_trait_v2)]
@@ -277,7 +277,7 @@ pub(crate) mod test_helpers {/// seed not being the same for every RNG invocation too. pub (crate )fn test_rng () -> rand_xorshift::XorShiftRng {use std::hash::{BuildHasher ,Hash ,Hasher }; -let mut hasher = std::collections::hash_map::RandomState ::new ().build_hasher (); +let mut hasher = std::hash::RandomState ::new ().build_hasher (); std::panic::Location ::caller ().hash (&mut hasher);let hc64 = hasher.finish (); let seed_vec =
rust/alloc/raw_vec.rs +42 -16
156 lines not on this page · raw message
rust/alloc/vec/into_iter.rs +11 -5
@@ -9,7 +9,8 @@use core::array;use core::fmt;use core::iter::{ -FusedIterator ,InPlaceIterable ,SourceIter ,TrustedLen ,TrustedRandomAccessNoCoerce , +FusedIterator ,InPlaceIterable ,SourceIter ,TrustedFused ,TrustedLen , +TrustedRandomAccessNoCoerce , };use core::marker::PhantomData ;use core::mem::{self,ManuallyDrop ,MaybeUninit ,SizedTypeProperties };
@@ -287,9 +288,7 @@ unsafe fn __iterator_get_unchecked(&mut self, i: usize) -> Self::Item// Also note the implementation of `Self: TrustedRandomAccess` requires // that `T: Copy` so reading elements from the buffer doesn't invalidate // them for `Drop`. -unsafe { -if T ::IS_ZST { mem::zeroed () }else { ptr::read (self.ptr.add (i)) } - } +unsafe {if T ::IS_ZST { mem::zeroed () }else { ptr::read (self.ptr.add (i)) } } } }
@@ -341,6 +340,10 @@ fn is_empty(&self) -> bool {#[stable(feature = "fused" , since = "1.26.0" )] impl <T ,A :Allocator >FusedIterator for IntoIter <T ,A > {} +#[doc(hidden)] +#[unstable(issue = "none" , feature = "trusted_fused" )] +unsafe impl <T ,A :Allocator >TrustedFused for IntoIter <T ,A > {} +#[unstable(feature = "trusted_len" , issue = "37572" )] unsafe impl <T ,A :Allocator >TrustedLen for IntoIter <T ,A > {}
@@ -425,7 +428,10 @@ fn drop(&mut self) {// also refer to the vec::in_place_collect module documentation to get an overview #[unstable(issue = "none" , feature = "inplace_iteration" )] #[doc(hidden)] -unsafe impl <T ,A :Allocator >InPlaceIterable for IntoIter <T ,A > {} +unsafe impl <T ,A :Allocator >InPlaceIterable for IntoIter <T ,A > { +const EXPAND_BY :Option <NonZeroUsize > =NonZeroUsize ::new (1 ); +const MERGE_BY :Option <NonZeroUsize > =NonZeroUsize ::new (1 ); +}#[unstable(issue = "none" , feature = "inplace_iteration" )] #[doc(hidden)]
rust/alloc/vec/mod.rs +51 -14
127 lines not on this page · raw message
scripts/min-tool-version.sh +1 -1
@@ -33,7 +33,7 @@ llvm) fi ;; rustc) - echo 1.75.0 + echo 1.76.0 ;; bindgen) echo 0.65.1
--
2.43.0
Re: [PATCH 2/2] rust: upgrade to Rust 1.76.0
Quoted 3 lines · Miguel Ojeda
This is the next upgrade to the Rust toolchain, from 1.75.0 to 1.76.0 (i.e. the latest) [1].
I've applied this on the merge point of for-next/rust-for-arm64 in arm64 tree and today's rust-next (patch #1 has a small conflict with the target.json changes in arm64 tree), tested with kunit for both x86 and arm64.
Regards,
Boqun
Quoted 3 lines
See the upgrade policy [2] and the comments on the first upgrade in commit
3ed03f4da06e("rust: upgrade to Rust 1.68.2").
[...]
Re: [PATCH 2/2] rust: upgrade to Rust 1.76.0
In reply to Miguel OjedaQuoted 45 lines · Miguel Ojeda
This is the next upgrade to the Rust toolchain, from 1.75.0 to 1.76.0 (i.e. the latest) [1].
See the upgrade policy [2] and the comments on the first upgrade in commit
3ed03f4da06e("rust: upgrade to Rust 1.68.2").# Unstable features
No unstable features that we use were stabilized in Rust 1.76.0.
The only unstable features allowed to be used outside the
kernelcrate are stillnew_uninit,offset_of, though other code to be upstreamed may increase the list.Please see [3] for details.
# Required changes
rustc(and others) now warns when it cannot connect to the Make jobserver, thus mark those invocations as recursive as needed. Please see the previous commit for details.# Other changes
Rust 1.76.0 does not emit the
.debug_pub{names,types}sections anymore for DWARFv4 [4][5]. For instance, in the uncompressed debug info case, this debug information took:
samples/rust/rust_minimal.o~64 KiB (~18% of total object size)
rust/kernel.o~92 KiB (~15%)
rust/core.o~114 KiB ( ~5%)In the compressed debug info (zlib) case:
samples/rust/rust_minimal.o ~11 KiB (~6%) rust/kernel.o ~17 KiB (~5%) rust/core.o ~21 KiB (~1.5%)In addition, the
rustc_codegen_gccbackend now does not emit the.eh_framesection when compiling under-Cpanic=abort[6], thus removing the need for the patch in the CI to compile the kernel [7]. Moreover, it also now emits the.commentsection too [6].#
allocupgrade and reviewing
[...]
Quoted 8 lines
Signed-off-by Miguel Ojeda <[email protected]>
Quoted 18 lines · Miguel Ojeda
rustc(like Cargo) may take advantage of the jobserver at any time (e.g. for backend parallelism, or eventually frontend too). In the kernel, we callrustcwith-Ccodegen-units=1(and-Zthreadsis 1 so far), so we do not expect parallelism. However, in the upcoming Rust 1.76.0, a warning is emitted byrustcwhen it cannot connect to the jobserver it was passed (in many cases, but not all: compiling and--print sysrootdo, but--versiondoes not). And given GNU Make always passes the jobserver in the environment variable (even when a line is deemed non-recursive),rustcwill end up complaining about it (in particular in Make 4.3 where there is only the simple pipe jobserver style).One solution is to remove the jobserver from
MAKEFLAGS. However, we can mark the lines with calls torustc(and Cargo) as recursive, which looks simpler. This is what is now documented and recommended byrustcand allows us to be ready for the time we may use parallelism insiderustc(potentially now, if a user passes-Zthreads). Thus do so.
Quoted 8 lines
Similarly, do the same for
rustdocandcargocalls.Finally, there is one case that the solution does not cover, which is the
$(shell ...)call we have. Thus, for that one, set an emptyMAKEFLAGSenvironment variable.
Signature · 2 lines
Best Regards Masahiro Yamada
Quoted 18 lines · Miguel Ojeda
rustc(like Cargo) may take advantage of the jobserver at any time (e.g. for backend parallelism, or eventually frontend too). In the kernel, we callrustcwith-Ccodegen-units=1(and-Zthreadsis 1 so far), so we do not expect parallelism. However, in the upcoming Rust 1.76.0, a warning is emitted byrustcwhen it cannot connect to the jobserver it was passed (in many cases, but not all: compiling and--print sysrootdo, but--versiondoes not). And given GNU Make always passes the jobserver in the environment variable (even when a line is deemed non-recursive),rustcwill end up complaining about it (in particular in Make 4.3 where there is only the simple pipe jobserver style).One solution is to remove the jobserver from
MAKEFLAGS. However, we can mark the lines with calls torustc(and Cargo) as recursive, which looks simpler. This is what is now documented and recommended byrustcand allows us to be ready for the time we may use parallelism insiderustc(potentially now, if a user passes-Zthreads). Thus do so.
Quoted 8 lines
Similarly, do the same for
rustdocandcargocalls.Finally, there is one case that the solution does not cover, which is the
$(shell ...)call we have. Thus, for that one, set an emptyMAKEFLAGSenvironment variable.
Applied (i.e. including the upgrade to 1.76.0) to rust-next -- thanks everyone!
[ Reworded to add link to PR documenting the recommendation. ]
Cheers,
Miguel