Skip to contentSkip to conversation

rust: Respect HOSTCC when linking for host

PATCH Superseded Matthew Maurer 5 participants 6 messages Sep 18, 2023 – Sep 26, 2023

Subsystems (2)

v2 of 4 Superseded Latest is v4 →

lore
Share & tools
lore
#0 lore raw

PATCH v2 1/1 rust: Respect HOSTCC when linking for host

Currently, rustc defaults to invoking cc, even if HOSTCC is defined, resulting in build failures in hermetic environments where cc does not exist. This includes both hostprogs and proc-macros.

Since we are setting the linker to HOSTCC, we set the linker flavor to gcc explicitly.

---

Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS should be respected as well.

I did not switch it to use HOSTLD for two reasons:
* That variable is not globally defined - it is only available in two subdirectories of tools/
* C host scripts are linked by HOSTCC as well, even when linking a collection of object files. It *prints* HOSTLD, but invokes HOSTCC. See scripts/Makefile.host cmd_host-cmulti for an example.

 rust/Makefile         | 4 ++++
 scripts/Makefile.host | 4 ++++
 2 files changed, 8 insertions(+)
rust/Makefile +4 -0
index 87958e864be0..b60b7eb8c5a0 100644
@@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
 $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
 	$(call if_changed,exports)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
 quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
       cmd_rustc_procmacro = \
 	$(RUSTC_OR_CLIPPY) $(rust_common_flags) \
+		-Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		-Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
 		--emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
 		--crate-type proc-macro \
 		--crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
scripts/Makefile.host +4 -0
index 8f7f842b54f9..dc0410cae5ca 100644
@@ -87,10 +87,14 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
+		 -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+		 -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
 
Signature · 1 line
2.42.0.459.ge4e396fd5e-goog
Reviewed-by on v2 1/1
#1 lore raw
Quoted 13 lines · Matthew Maurer

Currently, rustc defaults to invoking cc, even if HOSTCC is defined, resulting in build failures in hermetic environments where cc does not exist. This includes both hostprogs and proc-macros.

Since we are setting the linker to HOSTCC, we set the linker flavor to gcc explicitly.

---

Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS should be respected as well.

Quoted 7 lines

I did not switch it to use HOSTLD for two reasons:
* That variable is not globally defined - it is only available in two subdirectories of tools/
* C host scripts are linked by HOSTCC as well, even when linking a collection of object files. It *prints* HOSTLD, but invokes HOSTCC. See scripts/Makefile.host cmd_host-cmulti for an example.
[...]

#2 lore raw
In reply to Matthew Maurer
Quoted 14 lines · Matthew Maurer

Currently, rustc defaults to invoking cc, even if HOSTCC is defined, resulting in build failures in hermetic environments where cc does not exist. This includes both hostprogs and proc-macros.

Since we are setting the linker to HOSTCC, we set the linker flavor to gcc explicitly.

---

Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS should be respected as well.

Quoted 6 lines

I did not switch it to use HOSTLD for two reasons:
* That variable is not globally defined - it is only available in two subdirectories of tools/
* C host scripts are linked by HOSTCC as well, even when linking a collection of object files. It *prints* HOSTLD, but invokes HOSTCC. See scripts/Makefile.host cmd_host-cmulti for an example.

Sure, that makes sense to me, thanks for pointing that out.

Quoted 10 lines
 rust/Makefile         | 4 ++++
 scripts/Makefile.host | 4 ++++
 2 files changed, 8 insertions(+)
rust/Makefile +0 -0
index 87958e864be0..b60b7eb8c5a0 100644

rust/Makefile $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE · line 383

Quoted 4 lines
rust/Makefile +1 -0
@@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
 $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
        $(call if_changed,exports)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'

I don't think we need to do this kind of escaping. If a linker flag is passed to the linker directly as the driver has spaces such as -z relro, if it is instead passed to the compiler as the driver will not contain spaces (-Wl,-z,relro). As such, I don't think we need this escaping (famous last words). Mind submitting a v3 without it?

Quoted 31 lines
rust/Makefile +3 -0
+
 quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
       cmd_rustc_procmacro = \
        $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
+               -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+               -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
                --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
                --crate-type proc-macro \
                --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
scripts/Makefile.host +4 -1
index 8f7f842b54f9..dc0410cae5ca 100644
@@ -87,10 +87,14 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
+                -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+                -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
 
--

2.42.0.459.ge4e396fd5e-goog

Signature · 2 lines
Thanks,
~Nick Desaulniers
Reviewed-by
#3 lore raw
Quoted 38 lines · Nick Desaulniers
Quoted 14 lines · Matthew Maurer

Currently, rustc defaults to invoking cc, even if HOSTCC is defined, resulting in build failures in hermetic environments where cc does not exist. This includes both hostprogs and proc-macros.

Since we are setting the linker to HOSTCC, we set the linker flavor to gcc explicitly.

---

Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS should be respected as well.

Quoted 6 lines

I did not switch it to use HOSTLD for two reasons:
* That variable is not globally defined - it is only available in two subdirectories of tools/
* C host scripts are linked by HOSTCC as well, even when linking a collection of object files. It *prints* HOSTLD, but invokes HOSTCC. See scripts/Makefile.host cmd_host-cmulti for an example.

Sure, that makes sense to me, thanks for pointing that out.

Quoted 13 lines
 rust/Makefile         | 4 ++++
 scripts/Makefile.host | 4 ++++
 2 files changed, 8 insertions(+)
rust/Makefile +0 -0
index 87958e864be0..b60b7eb8c5a0 100644
@@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
 $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
        $(call if_changed,exports)
 
Quoted 7 lines

rust/Makefile $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE · line 383

Quoted 1 line
rust/Makefile +1 -0
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'

I don't think we need to do this kind of escaping. If a linker flag is passed to the linker directly as the driver has spaces such as -z relro, if it is instead passed to the compiler as the driver will not contain spaces (-Wl,-z,relro). As such, I don't think we need this escaping (famous last words). Mind submitting a v3 without it?

Thanks for clarifying off list that:

```
It's not escaping the spaces, it's escaping any quotes
example: -Clink-args='-Wl,-z,relro -Wl,-linker-plugin-lto'
Without the quote, the second one would go to rustc rather than the linker
```

Sorry for misunderstanding what exactly we were escaping here and why; thanks for implementing my suggestion of passing along KBUILD_HOSTLDFLAGS though. That will avoid issues for Android downstream.

Quoted 37 lines
Quoted 30 lines
rust/Makefile +3 -0
+
 quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
       cmd_rustc_procmacro = \
        $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
+               -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+               -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
                --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
                --crate-type proc-macro \
                --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
scripts/Makefile.host +4 -1
index 8f7f842b54f9..dc0410cae5ca 100644
@@ -87,10 +87,14 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
+                -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+                -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
 
--
Quoted 2 lines

2.42.0.459.ge4e396fd5e-goog

--
Thanks,
~Nick Desaulniers

Signature · 2 lines
Thanks,
~Nick Desaulniers
reviewer Tested-by on v2 1/1
#4 lore raw
In reply to Matthew Maurer
Quoted 8 lines · Matthew Maurer

Currently, rustc defaults to invoking cc, even if HOSTCC is defined, resulting in build failures in hermetic environments where cc does not exist. This includes both hostprogs and proc-macros.

Since we are setting the linker to HOSTCC, we set the linker flavor to gcc explicitly.

#5 lore raw
In reply to Matthew Maurer
Quoted 7 lines · Matthew Maurer

Currently, rustc defaults to invoking cc, even if HOSTCC is defined, resulting in build failures in hermetic environments where cc does not exist. This includes both hostprogs and proc-macros.

Since we are setting the linker to HOSTCC, we set the linker flavor to gcc explicitly.

This sentence is unclear to me because it does not explain why we need both despite the spec mentions 'linker-flavor' is inferred from the value of 'linker'.

The answer exists in this thread.
"
As mentioned earlier, we could pass $HOSTLD, but if the linker isn't named something accurate (e.g. if the linker is not named lld, but is lld), we need to know how to pass a flavor:
"

Maybe it is worthing menting that HOSTCC may not be properly named.

Quoted 23 lines

---

Updated the patch to reflect Nick's comment that KBUILD_HOSTLDFLAGS should be respected as well.

I did not switch it to use HOSTLD for two reasons:
* That variable is not globally defined - it is only available in two subdirectories of tools/
* C host scripts are linked by HOSTCC as well, even when linking a collection of object files. It *prints* HOSTLD, but invokes HOSTCC. See scripts/Makefile.host cmd_host-cmulti for an example.

 rust/Makefile         | 4 ++++
 scripts/Makefile.host | 4 ++++
 2 files changed, 8 insertions(+)
rust/Makefile +0 -0
index 87958e864be0..b60b7eb8c5a0 100644

rust/Makefile $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE · line 383

Quoted 4 lines
rust/Makefile +1 -0
@@ -380,9 +380,13 @@ $(obj)/exports_bindings_generated.h: $(obj)/bindings.o FORCE
 $(obj)/exports_kernel_generated.h: $(obj)/kernel.o FORCE
        $(call if_changed,exports)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'

This is equivalent to a macro in scripts/Kbuild.include

  escsq = $(subst $(squote),'\$(squote)',$1)

I would write it directly in cmd_rustc_procmacro.

Quoted 31 lines
rust/Makefile +3 -0
+
 quiet_cmd_rustc_procmacro = $(RUSTC_OR_CLIPPY_QUIET) P $@
       cmd_rustc_procmacro = \
        $(RUSTC_OR_CLIPPY) $(rust_common_flags) \
+               -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+               -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
                --emit=dep-info=$(depfile) --emit=link=$@ --extern proc_macro \
                --crate-type proc-macro \
                --crate-name $(patsubst lib%.so,%,$(notdir $@)) $<
scripts/Makefile.host +4 -1
index 8f7f842b54f9..dc0410cae5ca 100644
@@ -87,10 +87,14 @@ hostcxx_flags  = -Wp,-MMD,$(depfile) \
                  $(KBUILD_HOSTCXXFLAGS) $(HOST_EXTRACXXFLAGS) \
                  $(HOSTCXXFLAGS_$(target-stem).o)
 
+KBUILD_HOSTLDFLAGS_SQ = '$(subst ','\'',$(KBUILD_HOSTLDFLAGS))'
+
 # `--out-dir` is required to avoid temporaries being created by `rustc` in the
 # current working directory, which may be not accessible in the out-of-tree
 # modules case.
 hostrust_flags = --out-dir $(dir $@) --emit=dep-info=$(depfile) \
+                -Clinker-flavor=gcc -Clinker=$(HOSTCC) \
+                -Clink-args=$(KBUILD_HOSTLDFLAGS_SQ) \
                  $(KBUILD_HOSTRUSTFLAGS) $(HOST_EXTRARUSTFLAGS) \
                  $(HOSTRUSTFLAGS_$(target-stem))
 
--

2.42.0.459.ge4e396fd5e-goog

Signature · 2 lines
Best Regards
Masahiro Yamada