Skip to contentSkip to conversation

kbuild: host: use single executable for rustc -C linker

PATCH Superseded Mohamad Alsadhan 4 participants 7 messages Feb 25 – Mar 11

Subsystems (2)

v1 of 8 Superseded Latest is v8 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 kbuild: host: use single executable for rustc -C linker

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Use the last word of HOSTCC as the linker executable for host Rust tools. This preserves wrapper usage for C host tools while ensuring rustc receives a single executable path.

---
 scripts/Makefile.host | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
scripts/Makefile.host +8 -1
index c1dedf646..22eab2734 100644
@@ -87,11 +87,18 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-C linker=` expects a single executable path, not a command line.
+# HOSTCC may be a multi-word command when wrapped (e.g. "ccache gcc"),
+# which would otherwise be split by the shell and mis-parsed by rustc.
+#
+# Use a dedicated variable for the linker program used by host Rust tools.
+HOSTRUSTC_LINKER ?= $(lastword $(HOSTCC))
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
-		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clinker-flavor=gcc -Clinker=$(HOSTRUSTC_LINKER) \
 		 -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
Signature · 1 line
2.52.0
maintainer Acked-by on v1 1/1
#1 lore raw
Quoted 8 lines · Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Quoted 7 lines

Use the last word of HOSTCC as the linker executable for host Rust tools. This preserves wrapper usage for C host tools while ensuring rustc receives a single executable path.

Miguel, as this is a Rust fix, do you want to take it? If so:

Otherwise, I could take it via kbuild-fixes for 7.0.

Quoted 9 lines
---
 scripts/Makefile.host | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
scripts/Makefile.host +0 -0
index c1dedf646..22eab2734 100644

scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 95

Quoted 9 lines
scripts/Makefile.host +6 -0
@@ -87,11 +87,18 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-C linker=` expects a single executable path, not a command line.
+# HOSTCC may be a multi-word command when wrapped (e.g. "ccache gcc"),
+# which would otherwise be split by the shell and mis-parsed by rustc.
+#
+# Use a dedicated variable for the linker program used by host Rust tools.
+HOSTRUSTC_LINKER ?= $(lastword $(HOSTCC))

I would probably use HOSTRUSTC_LD to mirror how the rest of Kbuild refers to the linker.

Quoted 13 lines
scripts/Makefile.host +2 -1
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
-		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clinker-flavor=gcc -Clinker=$(HOSTRUSTC_LINKER) \
 		 -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
Signature · 1 line
2.52.0
reviewer
#2 lore raw
In reply to Mohamad Alsadhan
Quoted 11 lines · Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Use the last word of HOSTCC as the linker executable for host Rust tools. This preserves wrapper usage for C host tools while ensuring rustc receives a single executable path.

I don't think this is the right fix. Using only the last word is

(1) not what people asks for when they override HOSTCC. When a user ssays "call ccache gcc", they don't want "gcc" to be invoked instead.
(2) not general enough. If instead of a wrapper, it's something with subcommand, then this just breaks.
(3) going to cause surprises to people and be hard to troubleshoot when things do not work.

I think the proper fix should be invoking a helper script with the multi-word command be passed as environment variable which is then expanded by shell.

Best,
Gary

Quoted 32 lines
---
 scripts/Makefile.host | 9 ++++++++-
 1 file changed, 8 insertions(+), 1 deletion(-)
scripts/Makefile.host +8 -1
index c1dedf646..22eab2734 100644
@@ -87,11 +87,18 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-C linker=` expects a single executable path, not a command line.
+# HOSTCC may be a multi-word command when wrapped (e.g. "ccache gcc"),
+# which would otherwise be split by the shell and mis-parsed by rustc.
+#
+# Use a dedicated variable for the linker program used by host Rust tools.
+HOSTRUSTC_LINKER ?= $(lastword $(HOSTCC))
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
-		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clinker-flavor=gcc -Clinker=$(HOSTRUSTC_LINKER) \
 		 -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
#3 lore raw

PATCH v2 1/1 kbuild: host: use single executable for rustc -C linker

In reply to Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Introduce HOSTRUSTC_LD to select the linker executable for host Rust tools. When HOSTCC expands to multiple arguments and HOSTRUSTC_LD is not set, fall back to the last word of HOSTCC and warn. Users needing wrapper semantics can point HOSTRUSTC_LD at a wrapper script.

---
v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable

v1: https://lore.kernel.org/all/[email protected]/

---
 scripts/Makefile.host | 21 ++++++++++++++++++++-
 1 file changed, 20 insertions(+), 1 deletion(-)
scripts/Makefile.host +20 -1
index c1dedf646..f029f3f0c 100644
@@ -87,11 +87,30 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-C linker=` expects a single executable path, not a command line.
+# HOSTCC may be a multi-word command when wrapped (e.g. "ccache gcc"), which
+# would otherwise be split by the shell and mis-parsed by rustc.
+#
+# Allow users to override the linker used by host Rust tools via HOSTRUSTC_LD.
+# If HOSTCC is multi-word and HOSTRUSTC_LD is not set, default to the last
+# word of HOSTCC (typically the underlying compiler) and emit a warning.
+ifndef HOSTRUSTC_LD
+HOSTRUSTC_LD := $(HOSTCC)
+endif
+
+ifneq ($(words $(HOSTRUSTC_LD)),1)
+ifneq ($(filter command\ line environment,$(origin HOSTRUSTC_LD)),)
+$(error HOSTRUSTC_LD must be a single executable for rustc -C linker=. Got: "$(HOSTRUSTC_LD)")
+endif
+HOSTRUSTC_LD := $(lastword $(HOSTCC))
+$(warning HOSTCC expands to multiple arguments. Trying HOSTRUSTC_LD="$(HOSTRUSTC_LD)" for rustc. Set HOSTRUSTC_LD to override.)
+endif
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
-		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clinker-flavor=gcc -Clinker=$(HOSTRUSTC_LD) \
 		 -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
Signature · 1 line
2.52.0
#4 lore raw
Quoted 2 lines · Mohamad Alsadhan

Since v2 is fairly different than v1, I would probably have dropped this tag.

scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 107

Quoted 7 lines
scripts/Makefile.host +7 -0
+ifneq ($(words $(HOSTRUSTC_LD)),1)
+ifneq ($(filter command\ line environment,$(origin HOSTRUSTC_LD)),)
+$(error HOSTRUSTC_LD must be a single executable for rustc -C linker=. Got: "$(HOSTRUSTC_LD)")
+endif
+HOSTRUSTC_LD := $(lastword $(HOSTCC))
+$(warning HOSTCC expands to multiple arguments. Trying HOSTRUSTC_LD="$(HOSTRUSTC_LD)" for rustc. Set HOSTRUSTC_LD to override.)
+endif

It is good that this gives diagnostics, but do we want to make the logic so involved?

Did you try what Gary suggested (a wrapper script) and it didn't work?

Thanks!

Cheers,
Miguel

#5 lore raw
Quoted 2 lines · Miguel Ojeda

Since v2 is fairly different than v1, I would probably have dropped this tag.

Noted. My bad.

Quoted 4 lines

It is good that this gives diagnostics, but do we want to make the logic so involved?

Did you try what Gary suggested (a wrapper script) and it didn't work?

I was trying to take Gary's comments about unexpected/surprising behaviour into account without adding too much complexity (e.g. new script file and mangling commands as env vars) for such an edge case.

Instead if the user wants to pass some complex command, they get a diagnostic to tell them what's going on and to pass their own script. I felt that was a decent compromise.

However, If the logic is already too involved. Then might as well just go with the full proper fix. I can go ahead and do that in V3.

Cheers,
Mo

#6 lore raw
Quoted 11 lines · Mohamad Alsadhan

I was trying to take Gary's comments about unexpected/surprising behaviour into account without adding too much complexity (e.g. new script file and mangling commands as env vars) for such an edge case.

Instead if the user wants to pass some complex command, they get a diagnostic to tell them what's going on and to pass their own script. I felt that was a decent compromise.

However, If the logic is already too involved. Then might as well just go with the full proper fix. I can go ahead and do that in V3.

I see what you mean. I guess it depends on how complex v3 would look like.

In other words, if the "full fix" is of similar complexity, then we probably want to just go for it. If it turns out it is harder than it sounds like, then the compromise solution may be better.

In any case, I am not sure how much of an edge case it is -- for instance, ccache is mentioned in the Kbuild docs, and some people likely use it for C and may want to use it with Rust enabled sooner or later (we ourselves used it for some time a long time ago too). So a full fix will be wanted sooner or later anyway.

Thanks for detailing why you did v2 the way you did! (by the way, it is often a good idea to mention it in the changelog or to reply to the feedback in the previous version if you took another approach to avoid confusion).

Cheers,
Miguel