kbuild: host: use single executable for rustc -C linker
PATCH 1/1 kbuild: host: use single executable for rustc -C linker
rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:
error: multiple input filenames provided ...
Use the last word of HOSTCC as the linker executable for host Rust tools. This preserves wrapper usage for C host tools while ensuring rustc receives a single executable path.
--- scripts/Makefile.host | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)
scripts/Makefile.host +8 -1
@@ -87,11 +87,18 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-C linker=` expects a single executable path, not a command line. +# HOSTCC may be a multi-word command when wrapped (e.g. "ccache gcc"), +# which would otherwise be split by the shell and mis-parsed by rustc. +# +# Use a dedicated variable for the linker program used by host Rust tools. +HOSTRUSTC_LINKER ?= $(lastword $(HOSTCC)) + # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=$(HOSTRUSTC_LINKER) \ -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))
Signature · 1 line
2.52.0
Quoted 8 lines · Mohamad Alsadhan
rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:
error: multiple input filenames provided ...
Quoted 7 lines
Use the last word of HOSTCC as the linker executable for host Rust tools. This preserves wrapper usage for C host tools while ensuring rustc receives a single executable path.
Signed-off-by Mohamad Alsadhan <[email protected]>
Miguel, as this is a Rust fix, do you want to take it? If so:
Otherwise, I could take it via kbuild-fixes for 7.0.
Quoted 9 lines
--- scripts/Makefile.host | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)scripts/Makefile.host +0
-0
scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 95
Quoted 9 lines
scripts/Makefile.host +6
-0@@ -87,11 +87,18 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-C linker=` expects a single executable path, not a command line. +# HOSTCC may be a multi-word command when wrapped (e.g. "ccache gcc"), +# which would otherwise be split by the shell and mis-parsed by rustc. +# +# Use a dedicated variable for the linker program used by host Rust tools. +HOSTRUSTC_LINKER ?= $(lastword $(HOSTCC))
I would probably use HOSTRUSTC_LD to mirror how the rest of Kbuild refers to the linker.
Quoted 13 lines
scripts/Makefile.host +2
-1+ # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=$(HOSTRUSTC_LINKER) \ -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))Signature · 1 line
2.52.0
Gary Guo
reviewerQuoted 11 lines · Mohamad Alsadhan
rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:
error: multiple input filenames provided ...
Use the last word of HOSTCC as the linker executable for host Rust tools. This preserves wrapper usage for C host tools while ensuring rustc receives a single executable path.
I don't think this is the right fix. Using only the last word is
(1) not what people asks for when they override HOSTCC. When a user ssays "call ccache gcc", they don't want "gcc" to be invoked instead.
(2) not general enough. If instead of a wrapper, it's something with subcommand, then this just breaks.
(3) going to cause surprises to people and be hard to troubleshoot when things do not work.
I think the proper fix should be invoking a helper script with the multi-word command be passed as environment variable which is then expanded by shell.
Best,
Gary
Quoted 32 lines
Signed-off-by Mohamad Alsadhan <[email protected]>--- scripts/Makefile.host | 9 ++++++++- 1 file changed, 8 insertions(+), 1 deletion(-)scripts/Makefile.host +8
-1@@ -87,11 +87,18 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-C linker=` expects a single executable path, not a command line. +# HOSTCC may be a multi-word command when wrapped (e.g. "ccache gcc"), +# which would otherwise be split by the shell and mis-parsed by rustc. +# +# Use a dedicated variable for the linker program used by host Rust tools. +HOSTRUSTC_LINKER ?= $(lastword $(HOSTCC)) + # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=$(HOSTRUSTC_LINKER) \ -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))
PATCH v2 1/1 kbuild: host: use single executable for rustc -C linker
In reply to Mohamad Alsadhanrustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:
error: multiple input filenames provided ...
Introduce HOSTRUSTC_LD to select the linker executable for host Rust tools. When HOSTCC expands to multiple arguments and HOSTRUSTC_LD is not set, fall back to the last word of HOSTCC and warn. Users needing wrapper semantics can point HOSTRUSTC_LD at a wrapper script.
---
v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable
v1: https://lore.kernel.org/all/[email protected]/
--- scripts/Makefile.host | 21 ++++++++++++++++++++- 1 file changed, 20 insertions(+), 1 deletion(-)
scripts/Makefile.host +20 -1
@@ -87,11 +87,30 @@ hostcxx_flags = -Wp,-MMD,$(depfile) \ $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \ $(HOSTCXXFLAGS_$(target-stem).o) +# rustc's `-C linker=` expects a single executable path, not a command line. +# HOSTCC may be a multi-word command when wrapped (e.g. "ccache gcc"), which +# would otherwise be split by the shell and mis-parsed by rustc. +# +# Allow users to override the linker used by host Rust tools via HOSTRUSTC_LD. +# If HOSTCC is multi-word and HOSTRUSTC_LD is not set, default to the last +# word of HOSTCC (typically the underlying compiler) and emit a warning. +ifndef HOSTRUSTC_LD +HOSTRUSTC_LD := $(HOSTCC) +endif + +ifneq ($(words $(HOSTRUSTC_LD)),1) +ifneq ($(filter command\ line environment,$(origin HOSTRUSTC_LD)),) +$(error HOSTRUSTC_LD must be a single executable for rustc -C linker=. Got: "$(HOSTRUSTC_LD)") +endif +HOSTRUSTC_LD := $(lastword $(HOSTCC)) +$(warning HOSTCC expands to multiple arguments. Trying HOSTRUSTC_LD="$(HOSTRUSTC_LD)" for rustc. Set HOSTRUSTC_LD to override.) +endif + # `--out-dir` is required to avoid temporaries being created by `rustc` in the # current working directory, which may be not accessible in the out-of-tree # modules case. hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \ - -Clinker-flavor=gcc -Clinker=$(HOSTCC) \ + -Clinker-flavor=gcc -Clinker=$(HOSTRUSTC_LD) \ -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \ $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \ $(HOSTRUSTFLAGS_$(target-stem))
Signature · 1 line
2.52.0
Quoted 2 lines · Mohamad Alsadhan
Since v2 is fairly different than v1, I would probably have dropped this tag.
scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 107
Quoted 7 lines
scripts/Makefile.host +7
-0+ifneq ($(words $(HOSTRUSTC_LD)),1) +ifneq ($(filter command\ line environment,$(origin HOSTRUSTC_LD)),) +$(error HOSTRUSTC_LD must be a single executable for rustc -C linker=. Got: "$(HOSTRUSTC_LD)") +endif +HOSTRUSTC_LD := $(lastword $(HOSTCC)) +$(warning HOSTCC expands to multiple arguments. Trying HOSTRUSTC_LD="$(HOSTRUSTC_LD)" for rustc. Set HOSTRUSTC_LD to override.) +endif
It is good that this gives diagnostics, but do we want to make the logic so involved?
Did you try what Gary suggested (a wrapper script) and it didn't work?
Thanks!
Cheers,
Miguel
Quoted 2 lines · Miguel Ojeda
Since v2 is fairly different than v1, I would probably have dropped this tag.
Noted. My bad.
Quoted 4 lines
It is good that this gives diagnostics, but do we want to make the logic so involved?
Did you try what Gary suggested (a wrapper script) and it didn't work?
I was trying to take Gary's comments about unexpected/surprising behaviour into account without adding too much complexity (e.g. new script file and mangling commands as env vars) for such an edge case.
Instead if the user wants to pass some complex command, they get a diagnostic to tell them what's going on and to pass their own script. I felt that was a decent compromise.
However, If the logic is already too involved. Then might as well just go with the full proper fix. I can go ahead and do that in V3.
Cheers,
Mo
Quoted 11 lines · Mohamad Alsadhan
I was trying to take Gary's comments about unexpected/surprising behaviour into account without adding too much complexity (e.g. new script file and mangling commands as env vars) for such an edge case.
Instead if the user wants to pass some complex command, they get a diagnostic to tell them what's going on and to pass their own script. I felt that was a decent compromise.
However, If the logic is already too involved. Then might as well just go with the full proper fix. I can go ahead and do that in V3.
I see what you mean. I guess it depends on how complex v3 would look like.
In other words, if the "full fix" is of similar complexity, then we probably want to just go for it. If it turns out it is harder than it sounds like, then the compromise solution may be better.
In any case, I am not sure how much of an edge case it is -- for instance, ccache is mentioned in the Kbuild docs, and some people likely use it for C and may want to use it with Rust enabled sooner or later (we ourselves used it for some time a long time ago too). So a full fix will be wanted sooner or later anyway.
Thanks for detailing why you did v2 the way you did! (by the way, it is often a good idea to mention it in the changelog or to reply to the feedback in the previous version if you took another approach to avoid confusion).
Cheers,
Miguel