Skip to contentSkip to conversation

Makefile: Fix GCC_TOOLCHAIN_DIR prefix for Clang cross compilation

PATCH Superseded Fangrui Song 3 participants 4 messages Jul 20, 2020 – Jul 21, 2020 Kbuild

v1 of 3 Superseded Latest is v3 →

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 Makefile: Fix GCC_TOOLCHAIN_DIR prefix for Clang cross compilation

When CROSS_COMPILE is set (e.g. aarch64-linux-gnu-), if $(CROSS_COMPILE)elfedit is found at /usr/bin/aarch64-linux-gnu-, GCC_TOOLCHAIN_DIR will be set to /usr/bin/. --prefix= will be set to /usr/bin/ and Clang as of 11 will search for both $(prefix)aarch64-linux-gnu-$needle and $(prefix)$needle.

GCC searchs for $(prefix)aarch64-linux-gnu/$version/$needle, $(prefix)aarch64-linux-gnu/$needle and $(prefix)$needle. In practice, $(prefix)aarch64-linux-gnu/$needle rarely contains executables.

To better model how GCC's -B/--prefix takes in effect in practice, newer Clang only searches for $(prefix)$needle and for example it will find /usr/bin/as instead of /usr/bin/aarch64-linux-gnu-as.

Set --prefix= to $(GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE) (/usr/bin/aarch64-linux-gnu-) so that newer Clang can find the appropriate cross compiling GNU as (when -no-integrated-as is in effect).

---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
index 0b5f8538bde5..3ac83e375b61 100644
@@ -567,7 +567,7 @@ ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
 ifneq ($(CROSS_COMPILE),)
 CLANG_FLAGS	+= --target=$(notdir $(CROSS_COMPILE:%-=%))
 GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
-CLANG_FLAGS	+= --prefix=$(GCC_TOOLCHAIN_DIR)
+CLANG_FLAGS	+= --prefix=$(GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE)
 GCC_TOOLCHAIN	:= $(realpath $(GCC_TOOLCHAIN_DIR)/..)
 endif
 ifneq ($(GCC_TOOLCHAIN),)
Signature · 1 line
2.28.0.rc0.105.gf9edc3c819-goog
Reviewed-by Tested-by on v1 1/1
#1 lore raw
Quoted 14 lines · Fangrui Song

When CROSS_COMPILE is set (e.g. aarch64-linux-gnu-), if $(CROSS_COMPILE)elfedit is found at /usr/bin/aarch64-linux-gnu-, GCC_TOOLCHAIN_DIR will be set to /usr/bin/. --prefix= will be set to /usr/bin/ and Clang as of 11 will search for both $(prefix)aarch64-linux-gnu-$needle and $(prefix)$needle.

GCC searchs for $(prefix)aarch64-linux-gnu/$version/$needle, $(prefix)aarch64-linux-gnu/$needle and $(prefix)$needle. In practice, $(prefix)aarch64-linux-gnu/$needle rarely contains executables.

To better model how GCC's -B/--prefix takes in effect in practice, newer Clang only searches for $(prefix)$needle and for example it will find /usr/bin/as instead of /usr/bin/aarch64-linux-gnu-as.

Quoted 8 lines

Set --prefix= to $(GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE) (/usr/bin/aarch64-linux-gnu-) so that newer Clang can find the appropriate cross compiling GNU as (when -no-integrated-as is in effect).

Sorry that I did not pay attention before but this needs

in the body so that it gets automatically backported into all of our stable branches. I am not sure if Masahiro is okay with adding that after the fact or if he will want a v2.

I am fine with having my signed-off-by on the patch but I did not really do much :) I am fine with having that downgraded to

if people find it odd.

Thanks for sending this along!

Cheers,
Nathan

Quoted 20 lines
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -1
index 0b5f8538bde5..3ac83e375b61 100644
@@ -567,7 +567,7 @@ ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
 ifneq ($(CROSS_COMPILE),)
 CLANG_FLAGS	+= --target=$(notdir $(CROSS_COMPILE:%-=%))
 GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
-CLANG_FLAGS	+= --prefix=$(GCC_TOOLCHAIN_DIR)
+CLANG_FLAGS	+= --prefix=$(GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE)
 GCC_TOOLCHAIN	:= $(realpath $(GCC_TOOLCHAIN_DIR)/..)
 endif
 ifneq ($(GCC_TOOLCHAIN),)
Signature · 1 line
2.28.0.rc0.105.gf9edc3c819-goog
#2 lore raw
Quoted 8 lines · Nathan Chancellor
Quoted 6 lines · Fangrui Song

When CROSS_COMPILE is set (e.g. aarch64-linux-gnu-), if $(CROSS_COMPILE)elfedit is found at /usr/bin/aarch64-linux-gnu-, GCC_TOOLCHAIN_DIR will be set to /usr/bin/. --prefix= will be set to /usr/bin/ and Clang as of 11 will search for both $(prefix)aarch64-linux-gnu-$needle and $(prefix)$needle.

Quoted 6 lines
Quoted 6 lines

GCC searchs for $(prefix)aarch64-linux-gnu/$version/$needle, $(prefix)aarch64-linux-gnu/$needle and $(prefix)$needle. In practice, $(prefix)aarch64-linux-gnu/$needle rarely contains executables.

To better model how GCC's -B/--prefix takes in effect in practice, newer Clang only searches for $(prefix)$needle and for example it will find

"newer Clang" requires the reader to recall that "Clang as of 11" was the previous frame of reference. I think it would be clearer to:
1. call of clang-12 as having a difference in behavior.
2. explicitly link to the commit, ie:

Quoted 1 line
Quoted 1 line

/usr/bin/as instead of /usr/bin/aarch64-linux-gnu-as.

That's a common source of pain (for example, when cross compiling without having the proper cross binutils installed, it's common to get spooky errors about unsupported configs or host binutils not recognizing flags specific to cross building).

/usr/bin/as: unrecognized option '-EL'

being the most common. So in that case, I'm actually very happy with the llvm change if it solves that particularly common pain point.

Quoted 6 lines
Quoted 6 lines

Set --prefix= to $(GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE) (/usr/bin/aarch64-linux-gnu-) so that newer Clang can find the appropriate cross compiling GNU as (when -no-integrated-as is in effect).

Quoted 7 lines
Quoted 3 lines

Sorry that I did not pay attention before but this needs

Agreed. This change to llvm will blow up all of our CI jobs that cross compile if not backported to stable.

Quoted 7 lines

in the body so that it gets automatically backported into all of our stable branches. I am not sure if Masahiro is okay with adding that after the fact or if he will want a v2.

I am fine with having my signed-off-by on the patch but I did not really do much :) I am fine with having that downgraded to

Not a big deal, but there's only really two cases I can think of where it's appropriate to attach someone else's "SOB" to a patch:
1. It's their patch that you're resending on their behalf, possibly as part of a larger series.
2. You're a maintainer, and...well I guess that's also case 1 above.

Reported-by is more appropriate, and you can include the tags collected from this thread. Please ping me internally for help sending a v2, if needed.

Quoted 3 lines

I tested with this llvm pre- and post-
https://github.com/llvm/llvm-project/commit/3452a0d8c17f7166f479706b293caf6ac76ffd90.
I also tested downstream Android kernel builds with 3452a0d8c17f7166f479706b293caf6ac76ffd90. Builds that don't make use of CROSS_COMPILE (native host targets) are obviously unaffected. We might see this issue pop up a few more times internally if the patch isn't picked up by stable, or if those downstream kernel trees don't rebase on stable kernel trees as often as they refresh their toolchain.

Quoted 30 lines

if people find it odd.

Thanks for sending this along!

Cheers,
Nathan

Quoted 18 lines
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -2
index 0b5f8538bde5..3ac83e375b61 100644
@@ -567,7 +567,7 @@ ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
 ifneq ($(CROSS_COMPILE),)
 CLANG_FLAGS  += --target=$(notdir $(CROSS_COMPILE:%-=%))
 GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
-CLANG_FLAGS  += --prefix=$(GCC_TOOLCHAIN_DIR)
+CLANG_FLAGS  += --prefix=$(GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE)
 GCC_TOOLCHAIN        := $(realpath $(GCC_TOOLCHAIN_DIR)/..)
 endif
 ifneq ($(GCC_TOOLCHAIN),)
--
Quoted 2 lines

2.28.0.rc0.105.gf9edc3c819-goog

--

Signature · 2 lines
Thanks,
~Nick Desaulniers
#3 lore raw
Quoted 45 lines · Nick Desaulniers
Quoted 8 lines · Nathan Chancellor
Quoted 6 lines · Fangrui Song

When CROSS_COMPILE is set (e.g. aarch64-linux-gnu-), if $(CROSS_COMPILE)elfedit is found at /usr/bin/aarch64-linux-gnu-, GCC_TOOLCHAIN_DIR will be set to /usr/bin/. --prefix= will be set to /usr/bin/ and Clang as of 11 will search for both $(prefix)aarch64-linux-gnu-$needle and $(prefix)$needle.

Quoted 6 lines
Quoted 6 lines

GCC searchs for $(prefix)aarch64-linux-gnu/$version/$needle, $(prefix)aarch64-linux-gnu/$needle and $(prefix)$needle. In practice, $(prefix)aarch64-linux-gnu/$needle rarely contains executables.

To better model how GCC's -B/--prefix takes in effect in practice, newer Clang only searches for $(prefix)$needle and for example it will find

"newer Clang" requires the reader to recall that "Clang as of 11" was the previous frame of reference. I think it would be clearer to:
1. call of clang-12 as having a difference in behavior.
2. explicitly link to the commit, ie:

Quoted 1 line
Quoted 1 line

/usr/bin/as instead of /usr/bin/aarch64-linux-gnu-as.

That's a common source of pain (for example, when cross compiling without having the proper cross binutils installed, it's common to get spooky errors about unsupported configs or host binutils not recognizing flags specific to cross building).

/usr/bin/as: unrecognized option '-EL'

being the most common. So in that case, I'm actually very happy with the llvm change if it solves that particularly common pain point.

Quoted 10 lines
Quoted 9 lines

Set --prefix= to $(GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE) (/usr/bin/aarch64-linux-gnu-) so that newer Clang can find the appropriate cross compiling GNU as (when -no-integrated-as is in effect).

Quoted 6 lines
Quoted 3 lines

Sorry that I did not pay attention before but this needs

Agreed. This change to llvm will blow up all of our CI jobs that cross compile if not backported to stable.

Thanks. I did not know this.

Quoted 14 lines
Quoted 7 lines

in the body so that it gets automatically backported into all of our stable branches. I am not sure if Masahiro is okay with adding that after the fact or if he will want a v2.

I am fine with having my signed-off-by on the patch but I did not really do much :) I am fine with having that downgraded to

Not a big deal, but there's only really two cases I can think of where it's appropriate to attach someone else's "SOB" to a patch:
1. It's their patch that you're resending on their behalf, possibly as part of a larger series.
2. You're a maintainer, and...well I guess that's also case 1 above.

Quoted 3 lines

Reported-by is more appropriate, and you can include the tags collected from this thread. Please ping me internally for help sending a v2, if needed.

Nathan's draft patch on
https://github.com/ClangBuiltLinux/linux/issues/1099 actually works. I removed the slash. The words are my own. Since Nathan explicitly requested a downgrade of his tag, I'll do that for V2.

I'll do that anyway because I need to fix a typo in the description:

$(CROSS_COMPILE)elfedit is found at /usr/bin/aarch64-linux-gnu- $(CROSS_COMPILE)elfedit is found at /usr/bin/aarch64-linux-gnu-elfedit

Quoted 14 lines
Quoted 3 lines

I tested with this llvm pre- and post-
https://github.com/llvm/llvm-project/commit/3452a0d8c17f7166f479706b293caf6ac76ffd90.
I also tested downstream Android kernel builds with 3452a0d8c17f7166f479706b293caf6ac76ffd90. Builds that don't make use of CROSS_COMPILE (native host targets) are obviously unaffected. We might see this issue pop up a few more times internally if the patch isn't picked up by stable, or if those downstream kernel trees don't rebase on stable kernel trees as often as they refresh their toolchain.

Quoted 1 line

Thanks for offerring proofreading service! I'm working on the description...

Quoted 34 lines
Quoted 30 lines

if people find it odd.

Thanks for sending this along!

Cheers,
Nathan

Quoted 18 lines
---
 Makefile | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
Makefile +1 -2
index 0b5f8538bde5..3ac83e375b61 100644
@@ -567,7 +567,7 @@ ifneq ($(shell $(CC) --version 2>&1 | head -n 1 | grep clang),)
 ifneq ($(CROSS_COMPILE),)
 CLANG_FLAGS  += --target=$(notdir $(CROSS_COMPILE:%-=%))
 GCC_TOOLCHAIN_DIR := $(dir $(shell which $(CROSS_COMPILE)elfedit))
-CLANG_FLAGS  += --prefix=$(GCC_TOOLCHAIN_DIR)
+CLANG_FLAGS  += --prefix=$(GCC_TOOLCHAIN_DIR)$(CROSS_COMPILE)
 GCC_TOOLCHAIN        := $(realpath $(GCC_TOOLCHAIN_DIR)/..)
 endif
 ifneq ($(GCC_TOOLCHAIN),)
--
Quoted 2 lines

2.28.0.rc0.105.gf9edc3c819-goog

--

Signature · 2 lines
Thanks,
~Nick Desaulniers