Skip to contentSkip to conversation

kbuild: host: use single executable for rustc -C linker

PATCH Superseded Mohamad Alsadhan 3 participants 12 messages Mar 12 – Mar 29 1 message not in the archive

Subsystems (2)

v3 of 8 Superseded Latest is v8 →

lore
Share & tools
lore

A message that is not in the archive (sent off-list or not yet received): [email protected]/

#1 lore raw

PATCH v3 1/1 kbuild: host: use single executable for rustc -C linker

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Generate a small wrapper script and pass it to -Clinker e.g.

```
\#!/bin/sh
exec sh -c 'exec "$0" "$@"' ccache gcc "$@"
```

This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.

This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.

---
v2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/

v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable

v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/

---
 scripts/Makefile.host | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)
scripts/Makefile.host +19 -2
index c1dedf646..f976a07b7 100644
@@ -87,11 +87,28 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-Clinker=` expects a single executable path, not a command line.
+# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which
+# would otherwise be split by the shell and mis-parsed by rustc.
+# To work around this, we generate a wrapper script that forwards arguments to
+# `HOSTRUSTC_LD` so that such commands can be used safely.
+#
+# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC` 
+HOSTRUSTC_LD ?= $(HOSTCC)
+quiet_cmd_rustc-wrapper = HOSTGEN $@
+      cmd_rustc-wrapper = \
+	echo '\#!/bin/sh' > $@; \
+	echo 'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"' >> $@; \
+	chmod +x $@
+
+$(obj)/rustc-wrapper: FORCE
+	$(call if_changed,rustc-wrapper)
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
-		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clinker-flavor=gcc -Clinker=$(obj)/rustc-wrapper \
 		 -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
@@ -153,7 +170,7 @@ $(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE
 quiet_cmd_host-rust	= HOSTRUSTC $@
       cmd_host-rust	= \
 	$(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $<
-$(host-rust): $(obj)/%: $(src)/%.rs FORCE
+$(host-rust): $(obj)/%: $(src)/%.rs $(obj)/rustc-wrapper FORCE
 	+$(call if_changed_dep,host-rust)
 
 targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
Signature · 1 line
2.52.0
maintainer
#2 lore raw
Quoted 11 lines · Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Generate a small wrapper script and pass it to -Clinker e.g. ```
\#!/bin/sh

('\' is just a copy-and-paste left-over)

Quoted 7 lines

exec sh -c 'exec "$0" "$@"' ccache gcc "$@"
```

This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.

Quoted 8 lines

This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.

---

thanks for v3 with the wrapper script; it looks cleaner to me. Some comments below.

Quoted 8 lines
 scripts/Makefile.host | 21 +++++++++++++++++++--
 1 file changed, 19 insertions(+), 2 deletions(-)
scripts/Makefile.host +0 -0
index c1dedf646a39..f976a07b7b09 100644

scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 96

Quoted 10 lines
scripts/Makefile.host +7 -0
@@ -87,11 +87,28 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-Clinker=` expects a single executable path, not a command line.
+# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which
+# would otherwise be split by the shell and mis-parsed by rustc.
+# To work around this, we generate a wrapper script that forwards arguments to
+# `HOSTRUSTC_LD` so that such commands can be used safely.
+#
+# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC` 

(trailing space at EOL)

scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 101

Quoted 5 lines
scripts/Makefile.host +5 -0
+HOSTRUSTC_LD ?= $(HOSTCC)
+quiet_cmd_rustc-wrapper = HOSTGEN $@
+      cmd_rustc-wrapper = \
+	echo '\#!/bin/sh' > $@; \
+	echo 'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"' >> $@; \

Why can't we just use this?

    echo 'exec $(HOSTRUSTC_LD) "$$@"' >> $@; \

scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 105

Quoted 4 lines
scripts/Makefile.host +4 -0
+	chmod +x $@
+
+$(obj)/rustc-wrapper: FORCE
+	$(call if_changed,rustc-wrapper)

The if_changed macro is not optimal here, as there are no real prerequisites for the $(obj)/rustc-wrapper rule but only FORCE, cp. Documentation/kbuild/makefiles.rst ("Command change detection"). Thus, the rustc-wrapper will always be rebuilt and so always show the 'HOSTGEN' line.

But you could use filechk instead, e.g.:

define filechk_rustc-wrapper

	printf "%s\n" \
		'#!/bin/sh' \
		'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"'

endef

$(obj)/rustc-wrapper: FORCE

	$(call filechk,rustc-wrapper)
	$(Q)chmod +x $@

Kind regards

Signature · 1 line
Nicolas
#3 lore raw
Quoted 6 lines · Nicolas Schier

Why can't we just use this?

    echo 'exec $(HOSTRUSTC_LD) "$$@"' >> $@; \

I was being extra safe with quoting and escaping to make this as general as it can be. Just in case a more complex command with some shell quoting is passed e.g.
HOSTRUSTC_LD="env 'CCACHE_DIR=/tmp/my cache' ccache gcc"

sh -c would reparse it with original quoting and avoid potential issues. If you think it's unnecessary, I can change it to the simpler version.

Quoted 7 lines

The if_changed macro is not optimal here, as there are no real prerequisites for the $(obj)/rustc-wrapper rule but only FORCE, cp. Documentation/kbuild/makefiles.rst ("Command change detection"). Thus, the rustc-wrapper will always be rebuilt and so always show the 'HOSTGEN' line.

But you could use filechk instead, e.g.:

Noted, I will update this in v4 to use filechk instead.

Thanks for the helpful comments and suggestions,
Mo

maintainer
#4 lore raw
Quoted 4 lines · Mohamad Alsadhan
Quoted 3 lines · Nicolas Schier

Why can't we just use this?

Quoted 8 lines
Quoted 3 lines
    echo 'exec $(HOSTRUSTC_LD) "$$@"' >> $@; \

I was being extra safe with quoting and escaping to make this as general as it can be. Just in case a more complex command with some shell quoting is passed e.g.
HOSTRUSTC_LD="env 'CCACHE_DIR=/tmp/my cache' ccache gcc"

This does not work with the 'printf' variant (v4) either, as the space in 'my cache' will split the printf arguments and results in:

#!/bin/sh
exec sh -c 'exec "$0" "$@"' "env CCACHE_DIR=/tmp/my cache ccache gcc" "$@"

Sorry, I frequently forget to handle spaces in subdir names correctly. I'll send another suggestion as reply to v4.

Signature · 1 line
Nicolas
#5 lore raw

PATCH v4 1/1 kbuild: host: use single executable for rustc -C linker

In reply to Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Generate a small wrapper script and pass it to -Clinker e.g.

```
exec sh -c 'exec "$0" "$@"' ccache gcc "$@"
```

This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.

This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.

---
v3 -> v4:
- Use filechk instead of if_changed macro to regenerate script
- Remove trailing space at EOL

v2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/

v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable

v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/

---
 scripts/Makefile.host | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
scripts/Makefile.host +20 -2
index c1dedf646..e41753828 100644
@@ -87,11 +87,29 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-Clinker=` expects a single executable path, not a command line.
+# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which
+# would otherwise be split by the shell and mis-parsed by rustc.
+# To work around this, we generate a wrapper script that forwards arguments to
+# `HOSTRUSTC_LD` so that such commands can be used safely.
+#
+# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC`
+HOSTRUSTC_LD ?= $(HOSTCC)
+define filechk_rustc-wrapper
+	printf "%s\n" \
+		'#!/bin/sh' \
+		'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"'
+endef
+
+$(obj)/rustc-wrapper: FORCE
+	$(call filechk,rustc-wrapper)
+	$(Q)chmod +x $@
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
-		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clinker-flavor=gcc -Clinker=$(obj)/rustc-wrapper \
 		 -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
@@ -153,7 +171,7 @@ $(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE
 quiet_cmd_host-rust	= HOSTRUSTC $@
       cmd_host-rust	= \
 	$(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $<
-$(host-rust): $(obj)/%: $(src)/%.rs FORCE
+$(host-rust): $(obj)/%: $(src)/%.rs $(obj)/rustc-wrapper FORCE
 	+$(call if_changed_dep,host-rust)
 
 targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
Signature · 1 line
2.52.0
maintainer Reviewed-by Tested-by on v4 1/1
#6 lore raw
Quoted 57 lines · Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Generate a small wrapper script and pass it to -Clinker e.g.

```
exec sh -c 'exec "$0" "$@"' ccache gcc "$@"
```

This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.

This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.

---
v3 -> v4:
- Use filechk instead of if_changed macro to regenerate script
- Remove trailing space at EOL

v2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/

v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable

v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/

---
 scripts/Makefile.host | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
scripts/Makefile.host +3 -0
index c1dedf646..e41753828 100644
@@ -87,11 +87,29 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-Clinker=` expects a single executable path, not a command line.
+# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which
+# would otherwise be split by the shell and mis-parsed by rustc.

scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 102

Quoted 10 lines
scripts/Makefile.host +10 -0
+# To work around this, we generate a wrapper script that forwards arguments to
+# `HOSTRUSTC_LD` so that such commands can be used safely.
+#
+# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC`
+HOSTRUSTC_LD ?= $(HOSTCC)
+define filechk_rustc-wrapper
+	printf "%s\n" \
+		'#!/bin/sh' \
+		'exec sh -c '\''exec "$$0" "$$@"'\'' $(HOSTRUSTC_LD) "$$@"'
+endef

This printf-based solution needs some more tweaking to also support complex commands in HOSTRUSTC_LD with quoted spaces. A straight forward way could be:

	'exec sh -c '\''exec "$$0" "$$@"'\'' $(call escsq,$(HOSTRUSTC_LD)) "$$@"'

to escape single quotes within HOSTRUSTC_LD. Then the complex example from v3 works again (make HOSTRUSTC_LD="env 'CCACHE_DIR=/tmp/my cache' ccache gcc").

With the quoting fixed:

I am still not seeing a benefit in calling usage of 'exec' and 'sh -c'. If we'd skip both, there would be no need to use 'env' for environment updates:

	'$(call escsq,$(HOSTRUSTC_LD)) "$$@"'

But I may be missing something here.

Kind regards,
Nicolas

#7 lore raw
Quoted 9 lines · Nicolas Schier

This printf-based solution needs some more tweaking to also support complex commands in HOSTRUSTC_LD with quoted spaces. A straight forward way could be:

	'exec sh -c '\''exec "$$0" "$$@"'\'' $(call escsq,$(HOSTRUSTC_LD)) "$$@"'

to escape single quotes within HOSTRUSTC_LD. Then the complex example from v3 works again (make HOSTRUSTC_LD="env 'CCACHE_DIR=/tmp/my cache' ccache gcc").

Thanks for pointing it out, I should've retested more thoroughly.

Quoted 7 lines

I am still not seeing a benefit in calling usage of 'exec' and 'sh -c'. If we'd skip both, there would be no need to use 'env' for environment updates:

	'$(call escsq,$(HOSTRUSTC_LD)) "$$@"'

But I may be missing something here.

I was trying to make it as general as possible, e.g. for commands with shell magic, but I honestly can't make up a plausible use case for that. We can just go with the simple solution as it also doesn't need env.

I'll send the v5 patch with fix for quoted spaces and the simpler command.

Best regards,
Mo

#8 lore raw

PATCH v5 1/1 kbuild: host: use single executable for rustc -C linker

In reply to Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Generate a small wrapper script and pass it to -Clinker e.g.

  ```
  #!/bin/sh
  ccache gcc "$@"
  ```

This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.

This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.

---
v4 -> v5:
- Fix word splitting issues
- Remove unnecessary exec sh -c and simplify generated script

v3 -> v4:
- Use filechk instead of if_changed macro to regenerate script
- Remove trailing space at EOL

v2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/

v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable

v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/

---
 scripts/Makefile.host | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
scripts/Makefile.host +20 -2
index c1dedf646..2d2429ca0 100644
@@ -87,11 +87,29 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-Clinker=` expects a single executable path, not a command line.
+# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which
+# would otherwise be split by the shell and mis-parsed by rustc.
+# To work around this, we generate a wrapper script that forwards arguments to
+# `HOSTRUSTC_LD` so that such commands can be used safely.
+#
+# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC`
+HOSTRUSTC_LD ?= $(HOSTCC)
+define filechk_rustc-wrapper
+	printf "%s\n" \
+		'#!/bin/sh' \
+		'$(call escsq,$(HOSTRUSTC_LD)) "$$@"'
+endef
+
+$(obj)/rustc-wrapper: FORCE
+	$(call filechk,rustc-wrapper)
+	$(Q)chmod +x $@
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
-		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clinker-flavor=gcc -Clinker=$(obj)/rustc-wrapper \
 		 -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
@@ -153,7 +171,7 @@ $(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE
 quiet_cmd_host-rust	= HOSTRUSTC $@
       cmd_host-rust	= \
 	$(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $<
-$(host-rust): $(obj)/%: $(src)/%.rs FORCE
+$(host-rust): $(obj)/%: $(src)/%.rs $(obj)/rustc-wrapper FORCE
 	+$(call if_changed_dep,host-rust)
 
 targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
Signature · 1 line
2.52.0
maintainer Reviewed-by Tested-by on v5 1/1
#9 lore raw

Hi Mohamad,

Quoted 19 lines · Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Generate a small wrapper script and pass it to -Clinker e.g.

  ```
  #!/bin/sh
  ccache gcc "$@"
  ```

This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.

Quoted 7 lines

This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.

---

Thanks!

Unfortunately, I overlooked the missing entries for 'clean'/'mrproper' and scripts/.gitignore. Is it ok for you if I the following diff to your patch?

Makefile +2 -1
index a8d8ed711f9b..69fa18565219 100644
@@ -1676,7 +1676,8 @@ MRPROPER_FILES += include/config include/generated          \
 		  vmlinux-gdb.py \
 		  rpmbuild \
 		  rust/libmacros.so rust/libmacros.dylib \
-		  rust/libpin_init_internal.so rust/libpin_init_internal.dylib
+		  rust/libpin_init_internal.so rust/libpin_init_internal.dylib \
+		  scripts/rustc-wrapper
 
 # clean - Delete most, but leave enough to build external modules
 #
scripts/.gitignore +1 -0
index 4215c2208f7e..b3948b148c4b 100644
@@ -6,6 +6,7 @@
 /kallsyms
 /module.lds
 /recordmcount
+/rustc-wrapper
 /rustdoc_test_builder
 /rustdoc_test_gen
 /sign-file

Kind regards,
Nicolas

#10 lore raw

Hello Nicolas,

Quoted 3 lines · Nicolas Schier

Unfortunately, I overlooked the missing entries for 'clean'/'mrproper' and scripts/.gitignore. Is it ok for you if I the following diff to your patch?

Please go ahead, I missed those as well. Thanks for catching that!

I also missed another instance where the linker is passed to RUSTC, in rust/Makefile, as Yoann pointed out. So it's worth considering adding the fix in there too.

Happy to send an updated patch with the missing bits if you'd prefer, but I'm also fine with you folding it in.

Best,
Mo

maintainer
#11 lore raw
Quoted 7 lines · Mohamad Alsadhan

Hello Nicolas,

Quoted 3 lines · Nicolas Schier

Unfortunately, I overlooked the missing entries for 'clean'/'mrproper' and scripts/.gitignore. Is it ok for you if I the following diff to your patch?

Quoted 8 lines

Please go ahead, I missed those as well. Thanks for catching that!

I also missed another instance where the linker is passed to RUSTC, in rust/Makefile, as Yoann pointed out. So it's worth considering adding the fix in there too.

Happy to send an updated patch with the missing bits if you'd prefer, but I'm also fine with you folding it in.

yeah, then I'd appreciate if you send a v6 patch.

Thanks and kind regards,
Nicolas

#12 lore raw
In reply to Mohamad Alsadhan
Quoted 19 lines · Mohamad Alsadhan

rustc's -C linker= option expects a single executable path. When HOSTCC contains a wrapper (e.g. "ccache gcc"), passing -Clinker=$(HOSTCC) results in the shell splitting the value into multiple words, and rustc interprets the additional word as an input filename:

error: multiple input filenames provided ...

Generate a small wrapper script and pass it to -Clinker e.g.

  ```
  #!/bin/sh
  ccache gcc "$@"
  ```

This fix should be general enough to address most if not all cases (incl. wrappers or subcommands) and avoids surprises of simpler fixes like just defaulting to gcc.

Quoted 6 lines

This avoids passing the user command as an environment variable as that would be more challenging to trace and debug shell expansions.

Hello,

First, thank you for working on this. This is appreciated :-)

Quoted 50 lines

---
v4 -> v5:
- Fix word splitting issues
- Remove unnecessary exec sh -c and simplify generated script

v3 -> v4:
- Use filechk instead of if_changed macro to regenerate script
- Remove trailing space at EOL

v2 -> v3:
- Scrap previous hacky approaches (e.g. using lastword) and go with a proper fix which turned out not that complex to implement. Apologies Gary, I should have listened to you earlier :/

v1 -> v2:
- Rename HOSTRUSTC_LINKER to HOSTRUSTC_LD for consistency
- Introduce explicit HOSTRUSTC_LD override
- Warn when falling back due to multi-argument HOSTCC
- Error out if a user-specified HOSTRUSTC_LD is not an executable

v1: https://lore.kernel.org/all/[email protected]/
v2: https://lore.kernel.org/all/[email protected]/
v3: https://lore.kernel.org/all/[email protected]/
v4: https://lore.kernel.org/all/[email protected]/

---
 scripts/Makefile.host | 22 ++++++++++++++++++++--
 1 file changed, 20 insertions(+), 2 deletions(-)
scripts/Makefile.host +14 -0
index c1dedf646..2d2429ca0 100644
@@ -87,11 +87,29 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+# rustc's `-Clinker=` expects a single executable path, not a command line.
+# `HOSTCC` may be a multi-word command when wrapped (e.g. "ccache gcc"), which
+# would otherwise be split by the shell and mis-parsed by rustc.
+# To work around this, we generate a wrapper script that forwards arguments to
+# `HOSTRUSTC_LD` so that such commands can be used safely.
+#
+# Set `HOSTRUSTC_LD` for a different rustc linker command than `HOSTCC`
+HOSTRUSTC_LD ?= $(HOSTCC)
+define filechk_rustc-wrapper
+	printf "%s\n" \
+		'#!/bin/sh' \
+		'$(call escsq,$(HOSTRUSTC_LD)) "$$@"'
+endef
+

scripts/Makefile.host hostcxx_flags = -Wp,-MMD,$(depfile) \ · line 112

Quoted 10 lines
scripts/Makefile.host +5 -1
+$(obj)/rustc-wrapper: FORCE
+	$(call filechk,rustc-wrapper)
+	$(Q)chmod +x $@
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
-		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clinker-flavor=gcc -Clinker=$(obj)/rustc-wrapper \

Shouldn't this fix be also applied to rust/Makefile? It also contains a "-Clinker=$(HOSTCC)" argument for rustc[0].

With something like the following (On a 6.18.19), it does compile:

rust/Makefile +1 -1
index d4618f646b05..516fc1ea872d 100644
@@ -422,7 +422,7 @@ $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
 quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
       cmd_rustc_procmacro = \
 	$(RUSTC_OR_CLIPPY) $(rust_common_flags) $(rustc_target_flags) \
-		-Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		-Clinker-flavor=gcc -Clinker=scripts/rustc-wrapper \
 		-Clink-args='$(call escsq,$(KBUILD_PROCMACROLDFLAGS))' \
 		--emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
 		--crate-type proc-macro \

Note that I did not use the same patch: scripts/rustc-wrapper instead of $(obj)/rustc-wrapper.
$(obj)/rustc-wrapper did not work (the wrapper was searched in rust/ and not found) when I tried it.

[0]: https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/tree/rust/Makefile#n553

Regards,

Quoted 12 lines
scripts/Makefile.host +1 -1
 		 -Clink-args='$(call escsq,$(KBUILD_HOSTLDFLAGS))' \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
@@ -153,7 +171,7 @@ $(host-cxxobjs): $(obj)/%.o: $(obj)/%.cc FORCE
 quiet_cmd_host-rust	= HOSTRUSTC $@
       cmd_host-rust	= \
 	$(HOSTRUSTC) $(hostrust_flags) --emit=link=$@ $<
-$(host-rust): $(obj)/%: $(src)/%.rs FORCE
+$(host-rust): $(obj)/%: $(src)/%.rs $(obj)/rustc-wrapper FORCE
 	+$(call if_changed_dep,host-rust)
 
 targets += $(host-csingle) $(host-cmulti) $(host-cobjs) \
Signature · 2 lines
Yoann Congal
Smile ECS