kbuild: host: use single executable for rustc -C linker
A message that is not in the archive (sent off-list or not yet received): [email protected]/
PATCH v3 1/1 kbuild: host: use single executable for rustc -C linker
rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:
error: multiple input filenames provided ...
Generate a small wrapper script and pass it to -Clinker e.g.
``` \#!/bin/sh exec sh -c 'exec "$0" "$@"' ccache gcc "$@" ```
This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.
This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.
---
v2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/
v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable
v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
--- scripts/Makefile.host | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)
scripts/Makefile.host +19 -2
@@ -87,11 +87,28 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-Clinker=` expects a single executable path, not a command line. +# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which +# would otherwise be split by the shell and mis-parsed by rustc. +# To work around this, we generate a wrapper script that forwards arguments to +# `HOSTRUSTC_LD` so that such commands can be used safely. +# +# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC` +HOSTRUSTC_LD ?= $(HOSTCC) +quiet_cmd_rustc-wrapper = HOSTGEN $@ + cmd_rustc-wrapper = \ + echo '\#!/bin/sh' > $@; \ + echo 'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"' >> $@; \ + chmod +x $@ + +$(obj)/rustc-wrapper: FORCE + $(call if_changed,rustc-wrapper) + # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=$(obj)/rustc-wrapper \ -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))
@@ -153,7 +170,7 @@ $(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE quiet_cmd_host-rust = HOSTRUSTC $@ cmd_host-rust = \ $(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $< -$(host-rust): $(obj)/%: $(src)/%.rs FORCE +$(host-rust): $(obj)/%: $(src)/%.rs $(obj)/rustc-wrapper FORCE +$(call if_changed_dep,host-rust) targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
Signature · 1 line
2.52.0
Nicolas Schier
maintainerQuoted 11 lines · Mohamad Alsadhan
rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing
-Clinker=$(HOSTCC)results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:error: multiple input filenames provided ...
Generate a small wrapper script and pass it to -Clinker e.g. ```
\#!/bin/sh
('\' is just a copy-and-paste left-over)
Quoted 7 lines
exec sh -c 'exec "$0" "$@"' ccache gcc "$@"
```This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.
Quoted 8 lines
This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.
Suggested-by Yoann Congal <[email protected]>Signed-off-by Mohamad Alsadhan <[email protected]>---
thanks for v3 with the wrapper script; it looks cleaner to me. Some comments below.
Quoted 8 lines
scripts/Makefile.host | 21 +++++++++++++++++++-- 1 file changed, 19 insertions(+), 2 deletions(-)scripts/Makefile.host +0
-0
scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 96
Quoted 10 lines
scripts/Makefile.host +7
-0@@ -87,11 +87,28 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-Clinker=` expects a single executable path, not a command line. +# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which +# would otherwise be split by the shell and mis-parsed by rustc. +# To work around this, we generate a wrapper script that forwards arguments to +# `HOSTRUSTC_LD` so that such commands can be used safely. +# +# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC`
(trailing space at EOL)
scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 101
Quoted 5 lines
scripts/Makefile.host +5
-0+HOSTRUSTC_LD ?= $(HOSTCC) +quiet_cmd_rustc-wrapper = HOSTGEN $@ + cmd_rustc-wrapper = \ + echo '\#!/bin/sh' > $@; \ + echo 'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"' >> $@; \
Why can't we just use this?
echo 'exec $(HOSTRUSTC_LD) "$$@"' >> $@; \
scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 105
Quoted 4 lines
scripts/Makefile.host +4
-0+ chmod +x $@ + +$(obj)/rustc-wrapper: FORCE + $(call if_changed,rustc-wrapper)
The if_changed macro is not optimal here, as there are no real prerequisites for the $(obj)/rustc-wrapper rule but only FORCE, cp. Documentation/kbuild/makefiles.rst ("Command change detection"). Thus, the rustc-wrapper will always be rebuilt and so always show the 'HOSTGEN' line.
But you could use filechk instead, e.g.:
define filechk_rustc-wrapper
printf "%s\n" \ '#!/bin/sh' \ 'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"'
endef
$(obj)/rustc-wrapper: FORCE
$(call filechk,rustc-wrapper) $(Q)chmod +x $@
Kind regards
Signature · 1 line
Nicolas
Quoted 6 lines · Nicolas Schier
Why can't we just use this?
echo 'exec $(HOSTRUSTC_LD) "$$@"' >> $@; \
I was being extra safe with quoting and escaping to make this as general as it can be. Just in case a more complex command with some shell quoting is passed e.g.
HOSTRUSTC_LD="env 'CCACHE_DIR=/tmp/my cache' ccache gcc"
sh -c would reparse it with original quoting and avoid potential issues. If you think it's unnecessary, I can change it to the simpler version.
Quoted 7 lines
The
if_changedmacro is not optimal here, as there are no real prerequisites for the $(obj)/rustc-wrapper rule but only FORCE, cp.Documentation/kbuild/makefiles.rst("Command change detection"). Thus, the rustc-wrapper will always be rebuilt and so always show the 'HOSTGEN' line.But you could use filechk instead, e.g.:
Noted, I will update this in v4 to use filechk instead.
Thanks for the helpful comments and suggestions,
Mo
Nicolas Schier
maintainerQuoted 4 lines · Mohamad Alsadhan
Quoted 3 lines · Nicolas Schier
Why can't we just use this?
Quoted 8 lines
Quoted 3 lines
echo 'exec $(HOSTRUSTC_LD) "$$@"' >> $@; \I was being extra safe with quoting and escaping to make this as general as it can be. Just in case a more complex command with some shell quoting is passed e.g.
HOSTRUSTC_LD="env 'CCACHE_DIR=/tmp/my cache' ccache gcc"
This does not work with the 'printf' variant (v4) either, as the space in 'my cache' will split the printf arguments and results in:
#!/bin/sh
exec sh -c 'exec "$0" "$@"' "env CCACHE_DIR=/tmp/my cache ccache gcc" "$@"
Sorry, I frequently forget to handle spaces in subdir names correctly. I'll send another suggestion as reply to v4.
Signature · 1 line
Nicolas
PATCH v4 1/1 kbuild: host: use single executable for rustc -C linker
In reply to Mohamad Alsadhanrustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:
error: multiple input filenames provided ...
Generate a small wrapper script and pass it to -Clinker e.g.
``` exec sh -c 'exec "$0" "$@"' ccache gcc "$@" ```
This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.
This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.
---
v3 -> v4:
- Use filechk instead of if_changed macro to regenerate script
- Remove trailing space at EOL
v2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/
v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable
v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
--- scripts/Makefile.host | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)
scripts/Makefile.host +20 -2
@@ -87,11 +87,29 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-Clinker=` expects a single executable path, not a command line. +# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which +# would otherwise be split by the shell and mis-parsed by rustc. +# To work around this, we generate a wrapper script that forwards arguments to +# `HOSTRUSTC_LD` so that such commands can be used safely. +# +# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC` +HOSTRUSTC_LD ?= $(HOSTCC) +define filechk_rustc-wrapper + printf "%s\n" \ + '#!/bin/sh' \ + 'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"' +endef + +$(obj)/rustc-wrapper: FORCE + $(call filechk,rustc-wrapper) + $(Q)chmod +x $@ + # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=$(obj)/rustc-wrapper \ -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))
@@ -153,7 +171,7 @@ $(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE quiet_cmd_host-rust = HOSTRUSTC $@ cmd_host-rust = \ $(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $< -$(host-rust): $(obj)/%: $(src)/%.rs FORCE +$(host-rust): $(obj)/%: $(src)/%.rs $(obj)/rustc-wrapper FORCE +$(call if_changed_dep,host-rust) targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
Signature · 1 line
2.52.0
Quoted 57 lines · Mohamad Alsadhan
rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing
-Clinker=$(HOSTCC)results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:error: multiple input filenames provided ...
Generate a small wrapper script and pass it to -Clinker e.g.
``` exec sh -c 'exec "$0" "$@"' ccache gcc "$@" ```This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.
This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.
Suggested-by Yoann Congal <[email protected]>Signed-off-by Mohamad Alsadhan <[email protected]>---
v3 -> v4:
- Use filechk instead ofif_changedmacro to regenerate script
- Remove trailing space at EOLv2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/v1 -> v2:
- RenameHOSTRUSTC_LINKERtoHOSTRUSTC_LDfor consistency
- Introduce explicitHOSTRUSTC_LDoverride
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specifiedHOSTRUSTC_LDis not an executablev1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/--- scripts/Makefile.host | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)scripts/Makefile.host +3
-0@@ -87,11 +87,29 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-Clinker=` expects a single executable path, not a command line. +# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which +# would otherwise be split by the shell and mis-parsed by rustc.
scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 102
Quoted 10 lines
scripts/Makefile.host +10
-0+# To work around this, we generate a wrapper script that forwards arguments to +# `HOSTRUSTC_LD` so that such commands can be used safely. +# +# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC` +HOSTRUSTC_LD ?= $(HOSTCC) +define filechk_rustc-wrapper + printf "%s\n" \ + '#!/bin/sh' \ + 'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"' +endef
This printf-based solution needs some more tweaking to also support complex commands in HOSTRUSTC_LD with quoted spaces. A straight forward way could be:
'exec sh -c '\''exec "$$0" "$$@"'\'' $(call escsq,$(HOSTRUSTC_LD)) "$$@"'
to escape single quotes within HOSTRUSTC_LD. Then the complex example from v3 works again (make HOSTRUSTC_LD="env 'CCACHE_DIR=/tmp/my cache' ccache gcc").
With the quoting fixed:
I am still not seeing a benefit in calling usage of 'exec' and 'sh -c'. If we'd skip both, there would be no need to use 'env' for environment updates:
'$(call escsq,$(HOSTRUSTC_LD)) "$$@"'
But I may be missing something here.
Kind regards,
Nicolas
Quoted 9 lines · Nicolas Schier
This printf-based solution needs some more tweaking to also support complex commands in
HOSTRUSTC_LDwith quoted spaces. A straight forward way could be:'exec sh -c '\''exec "$$0" "$$@"'\'' $(call escsq,$(HOSTRUSTC_LD)) "$$@"'to escape single quotes within
HOSTRUSTC_LD. Then the complex example from v3 works again (make HOSTRUSTC_LD="env 'CCACHE_DIR=/tmp/my cache' ccache gcc").
Thanks for pointing it out, I should've retested more thoroughly.
Quoted 7 lines
I am still not seeing a benefit in calling usage of 'exec' and 'sh -c'. If we'd skip both, there would be no need to use 'env' for environment updates:
'$(call escsq,$(HOSTRUSTC_LD)) "$$@"'But I may be missing something here.
I was trying to make it as general as possible, e.g. for commands with shell magic, but I honestly can't make up a plausible use case for that. We can just go with the simple solution as it also doesn't need env.
I'll send the v5 patch with fix for quoted spaces and the simpler command.
Best regards,
Mo
PATCH v5 1/1 kbuild: host: use single executable for rustc -C linker
In reply to Mohamad Alsadhanrustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:
error: multiple input filenames provided ...
Generate a small wrapper script and pass it to -Clinker e.g.
``` #!/bin/sh ccache gcc "$@" ```
This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.
This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.
---
v4 -> v5:
- Fix word splitting issues
- Remove unnecessary exec sh -c and simplify generated script
v3 -> v4:
- Use filechk instead of if_changed macro to regenerate script
- Remove trailing space at EOL
v2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/
v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable
v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/
--- scripts/Makefile.host | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)
scripts/Makefile.host +20 -2
@@ -87,11 +87,29 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-Clinker=` expects a single executable path, not a command line. +# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which +# would otherwise be split by the shell and mis-parsed by rustc. +# To work around this, we generate a wrapper script that forwards arguments to +# `HOSTRUSTC_LD` so that such commands can be used safely. +# +# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC` +HOSTRUSTC_LD ?= $(HOSTCC) +define filechk_rustc-wrapper + printf "%s\n" \ + '#!/bin/sh' \ + '$(call escsq,$(HOSTRUSTC_LD)) "$$@"' +endef + +$(obj)/rustc-wrapper: FORCE + $(call filechk,rustc-wrapper) + $(Q)chmod +x $@ + # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=$(obj)/rustc-wrapper \ -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))
@@ -153,7 +171,7 @@ $(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE quiet_cmd_host-rust = HOSTRUSTC $@ cmd_host-rust = \ $(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $< -$(host-rust): $(obj)/%: $(src)/%.rs FORCE +$(host-rust): $(obj)/%: $(src)/%.rs $(obj)/rustc-wrapper FORCE +$(call if_changed_dep,host-rust) targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
Signature · 1 line
2.52.0
Hi Mohamad,
Quoted 19 lines · Mohamad Alsadhan
rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing
-Clinker=$(HOSTCC)results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:error: multiple input filenames provided ...
Generate a small wrapper script and pass it to -Clinker e.g.
``` #!/bin/sh ccache gcc "$@" ```This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.
Quoted 7 lines
This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.
Suggested-by Yoann Congal <[email protected]>Signed-off-by Mohamad Alsadhan <[email protected]>---
Thanks!
Unfortunately, I overlooked the missing entries for 'clean'/'mrproper' and scripts/.gitignore. Is it ok for you if I the following diff to your patch?
Makefile +2 -1
@@ -1676,7 +1676,8 @@ MRPROPER_FILES += include/config include/generated \ vmlinux-gdb.py \ rpmbuild \ rust/libmacros.so rust/libmacros.dylib \ - rust/libpin_init_internal.so rust/libpin_init_internal.dylib + rust/libpin_init_internal.so rust/libpin_init_internal.dylib \ + scripts/rustc-wrapper # clean - Delete most, but leave enough to build external modules #
scripts/.gitignore +1 -0
@@ -6,6 +6,7 @@ /kallsyms /module.lds /recordmcount +/rustc-wrapper /rustdoc_test_builder /rustdoc_test_gen /sign-file
Kind regards,
Nicolas
Hello Nicolas,
Quoted 3 lines · Nicolas Schier
Unfortunately, I overlooked the missing entries for 'clean'/'mrproper' and
scripts/.gitignore. Is it ok for you if I the following diff to your patch?
Please go ahead, I missed those as well. Thanks for catching that!
I also missed another instance where the linker is passed to RUSTC, in rust/Makefile, as Yoann pointed out. So it's worth considering adding the fix in there too.
Happy to send an updated patch with the missing bits if you'd prefer, but I'm also fine with you folding it in.
Best,
Mo
Nicolas Schier
maintainerQuoted 7 lines · Mohamad Alsadhan
Hello Nicolas,
Quoted 3 lines · Nicolas Schier
Unfortunately, I overlooked the missing entries for 'clean'/'mrproper' and
scripts/.gitignore. Is it ok for you if I the following diff to your patch?
Quoted 8 lines
Please go ahead, I missed those as well. Thanks for catching that!
I also missed another instance where the linker is passed to
RUSTC, inrust/Makefile, as Yoann pointed out. So it's worth considering adding the fix in there too.Happy to send an updated patch with the missing bits if you'd prefer, but I'm also fine with you folding it in.
yeah, then I'd appreciate if you send a v6 patch.
Thanks and kind regards,
Nicolas
Quoted 19 lines · Mohamad Alsadhan
rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing
-Clinker=$(HOSTCC)results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:error: multiple input filenames provided ...
Generate a small wrapper script and pass it to -Clinker e.g.
``` #!/bin/sh ccache gcc "$@" ```This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.
Quoted 6 lines
This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.
Suggested-by Yoann Congal <[email protected]>Signed-off-by Mohamad Alsadhan <[email protected]>
Hello,
First, thank you for working on this. This is appreciated :-)
Quoted 50 lines
---
v4 -> v5:
- Fix word splitting issues
- Remove unnecessaryexec sh -cand simplify generated scriptv3 -> v4:
- Use filechk instead ofif_changedmacro to regenerate script
- Remove trailing space at EOLv2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/v1 -> v2:
- RenameHOSTRUSTC_LINKERtoHOSTRUSTC_LDfor consistency
- Introduce explicitHOSTRUSTC_LDoverride
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specifiedHOSTRUSTC_LDis not an executablev1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/--- scripts/Makefile.host | 22 ++++++++++++++++++++-- 1 file changed, 20 insertions(+), 2 deletions(-)scripts/Makefile.host +14
-0@@ -87,11 +87,29 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-Clinker=` expects a single executable path, not a command line. +# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which +# would otherwise be split by the shell and mis-parsed by rustc. +# To work around this, we generate a wrapper script that forwards arguments to +# `HOSTRUSTC_LD` so that such commands can be used safely. +# +# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC` +HOSTRUSTC_LD ?= $(HOSTCC) +define filechk_rustc-wrapper + printf "%s\n" \ + '#!/bin/sh' \ + '$(call escsq,$(HOSTRUSTC_LD)) "$$@"' +endef +
scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 112
Quoted 10 lines
scripts/Makefile.host +5
-1+$(obj)/rustc-wrapper: FORCE + $(call filechk,rustc-wrapper) + $(Q)chmod +x $@ + # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=$(obj)/rustc-wrapper \
Shouldn't this fix be also applied to rust/Makefile? It also contains a "-Clinker=$(HOSTCC)" argument for rustc[0].
With something like the following (On a 6.18.19), it does compile:
rust/Makefile +1 -1
@@ -422,7 +422,7 @@ $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@ cmd_rustc_procmacro = \ $(RUSTC_OR_CLIPPY) $(rust_common_flags) $(rustc_target_flags) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=scripts/rustc-wrapper \ -Clink-args='$(call escsq,$(KBUILD_PROCMACROLDFLAGS))' \ --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \ --crate-type proc-macro \
Note that I did not use the same patch: scripts/rustc-wrapper instead of $(obj)/rustc-wrapper.
$(obj)/rustc-wrapper did not work (the wrapper was searched in rust/ and not found) when I tried it.
[0]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/rust/Makefile#n553
Regards,
Quoted 12 lines
scripts/Makefile.host +1
-1-Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))@@ -153,7 +171,7 @@ $(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE quiet_cmd_host-rust = HOSTRUSTC $@ cmd_host-rust = \ $(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $< -$(host-rust): $(obj)/%: $(src)/%.rs FORCE +$(host-rust): $(obj)/%: $(src)/%.rs $(obj)/rustc-wrapper FORCE +$(call if_changed_dep,host-rust) targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
Signature · 2 lines
Yoann Congal Smile ECS