Skip to contentSkip to conversation

how can we test the hexagon port in mainline

DISCUSSION Christoph Hellwig 5 participants 14 messages Jun 23, 2021 – Jul 8, 2021 Qualcomm hexagon architecture

lore
Share & tools
lore
#1 lore raw
Quoted 2 lines

-----Original Message-----
From: Christoph Hellwig <[email protected]>

...

Quoted 10 lines

Hi all,

the oldest supported gcc version in mainline is gcc 4.9. But the only hexagon crosscompiler I can find is the one Arnds website points to here:

https://mirrors.edge.kernel.org/pub/tools/crosstool/

which is a non-upstream gcc 4.6.1 port. How are we supposed to even build test hexagon code?

We have provided a clang-12-based toolchain here:

https://codelinaro.jfrog.io/artifactory/codelinaro-qemu/2021-05-12/clang+llv
m-12.0.0-cross-hexagon-unknown-linux-musl.tar.xz

Could we update https://mirrors.edge.kernel.org/pub/tools/crosstool/ to point here?

-Brian

#2 lore raw
Quoted 12 lines · Brian Cain
Quoted 3 lines

-----Original Message-----
From: Christoph Hellwig <[email protected]>

...

Quoted 8 lines

Hi all,

the oldest supported gcc version in mainline is gcc 4.9. But the only hexagon crosscompiler I can find is the one Arnds website points to here:

https://mirrors.edge.kernel.org/pub/tools/crosstool/

Quoted 7 lines
Quoted 2 lines

which is a non-upstream gcc 4.6.1 port. How are we supposed to even build test hexagon code?

We have provided a clang-12-based toolchain here:

https://codelinaro.jfrog.io/artifactory/codelinaro-qemu/2021-05-12/clang+llv
m-12.0.0-cross-hexagon-unknown-linux-musl.tar.xz

How close is this to the mainline clang builds from https://apt.llvm.org/?

The last time we talked, there were still a couple of bugs that prevented me from building an unmodified kernel with an unmodified clang toolchain, but I think you planned to address those. Are any changes left that did not make it into the 12.0 release?

Quoted 2 lines

Could we update https://mirrors.edge.kernel.org/pub/tools/crosstool/ to point here?

I'm happy to edit the index page on
https://mirrors.edge.kernel.org/pub/tools/crosstool,
but could you provide a stable URL that will keep pointing to the latest version in the future so I don't have to update it for each new build?

Arnd

#3 lore raw
Quoted 26 lines

-----Original Message-----
From: Arnd Bergmann <[email protected]>
Sent: Wednesday, June 23, 2021 9:54 AM
To: Brian Cain <[email protected]>
Cc: Christoph Hellwig <[email protected]>; Linus Torvalds <torvalds@linux- foundation.org>; Sid Manning <[email protected]>; open list:QUALCOMM HEXAGON... <[email protected]>; Linux Kernel Mailing List <[email protected]>
Subject: Re: how can we test the hexagon port in mainline

Quoted 15 lines · Brian Cain
Quoted 3 lines

-----Original Message-----
From: Christoph Hellwig <[email protected]>

...

Quoted 10 lines

Hi all,

the oldest supported gcc version in mainline is gcc 4.9. But the only hexagon crosscompiler I can find is the one Arnds website points to here:

https://mirrors.edge.kernel.org/pub/tools/crosstool/

which is a non-upstream gcc 4.6.1 port. How are we supposed to even build test hexagon code?

Quoted 7 lines
Quoted 3 lines

We have provided a clang-12-based toolchain here:

https://codelinaro.jfrog.io/artifactory/codelinaro-qemu/2021-05-

12/clang+llv

Quoted 1 line

m-12.0.0-cross-hexagon-unknown-linux-musl.tar.xz

How close is this to the mainline clang builds from https://apt.llvm.org/?

This is built from llvm-project 12.0.0 but also contains userspace toolchain features like musl and qemu-linux-user for hexagon. I suppose for the purposes of kernel builds they're equivalent.

Quoted 4 lines

The last time we talked, there were still a couple of bugs that prevented me from building an unmodified kernel with an unmodified clang toolchain, but I think you planned to address those. Are any changes left that did not make it into the 12.0 release?

I think those were identified building with clang 12.0.0 or post-12.0.0, so I don't think they'll be addressed. Let me review and follow up on the status of those items. I recall one workaround was to use the integrated assembler to avoid one of the toolchain bugs.

Quoted 8 lines
Quoted 2 lines

Could we update https://mirrors.edge.kernel.org/pub/tools/crosstool/ to point here?

I'm happy to edit the index page on
https://mirrors.edge.kernel.org/pub/tools/crosstool,
but could you provide a stable URL that will keep pointing to the latest version
in the future so I don't have to update it for each new build?

I don't know how to do that with the mechanisms we have so far. But I will see about getting one created and then hopefully you could use that.

-Brian

#4 lore raw

It seem like it still isn't complete enought for a kernel build, though:

$ export CROSS_COMPILE=/opt/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin/hexagon-unknown-linux-musl- 
$ make ARCH=hexagon LLVM=1 oldconfig
...
scripts/Kconfig.include:40: linker 'ld.lld' not found
make[1]: *** [scripts/kconfig/Makefile:77: oldconfig] Error 1
make: *** [Makefile:614: oldconfig] Error 2
#5 lore raw
Quoted 7 lines · Christoph Hellwig

It seem like it still isn't complete enought for a kernel build, though:

$ export CROSS_COMPILE=/opt/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin/hexagon-unknown-linux-musl-
$ make ARCH=hexagon LLVM=1 oldconfig
...
scripts/Kconfig.include:40: linker 'ld.lld' not found

I tried this using the prebuilt binaries from apt.llvm.org:

$ make ARCH=hexagon LLVM=1 O=obj-hexagon CROSS_COMPILE=hexagon-linux-
LLVM_IAS=1 CC=llvm-12 defconfig modules vmlinux
<stdin>:1515:2: warning: syscall clone3 not implemented [-W#warnings]
#warning syscall clone3 not implemented
 ^
1 warning generated.

Doing the same thing with allmodconfig results in an internal error with clang-12
while compiling kernel/locking/lockdep.c. Same thing with clang-13. After turning
off lock debugging, it seems fine.

Arnd

#6 lore raw
Quoted 18 lines · Arnd Bergmann
Quoted 7 lines · Christoph Hellwig

It seem like it still isn't complete enought for a kernel build, though:

$ export CROSS_COMPILE=/opt/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin/hexagon-unknown-linux-musl-
$ make ARCH=hexagon LLVM=1 oldconfig
...
scripts/Kconfig.include:40: linker 'ld.lld' not found

I tried this using the prebuilt binaries from apt.llvm.org:

$ make ARCH=hexagon LLVM=1 O=obj-hexagon CROSS_COMPILE=hexagon-linux-
LLVM_IAS=1 CC=llvm-12 defconfig modules vmlinux
<stdin>:1515:2: warning: syscall clone3 not implemented [-W#warnings]
#warning syscall clone3 not implemented
 ^
1 warning generated.
Quoted 5 lines

Doing the same thing with allmodconfig results in an internal error with clang-12
while compiling kernel/locking/lockdep.c. Same thing with clang-13. After turning
off lock debugging, it seems fine.

I've reported this upstream with you on CC:

https://bugs.llvm.org/show_bug.cgi?id=50838

Christoph, that toolchain should work (I had to install libtinfo5 and libc++1-7 on Debian Buster):

$ export PATH=$HOME/tmp/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin:$PATH
$ make -skj"$(nproc)" ARCH=hexagon CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all

CROSS_COMPILE is really only used to set the '--target=' flag when LLVM=1 LLVM_IAS=1 are used, it can even be just 'hexagon-linux'.

As Arnd points out, https://apt.llvm.org is also an option, that is what we use in our continuous integration:

https://github.com/ClangBuiltLinux/continuous-integration2

Cheers,
Nathan

#7 lore raw
Quoted 25 lines · Nathan Chancellor
Quoted 18 lines · Arnd Bergmann
Quoted 7 lines · Christoph Hellwig

It seem like it still isn't complete enought for a kernel build, though:

$ export CROSS_COMPILE=/opt/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin/hexagon-unknown-linux-musl-
$ make ARCH=hexagon LLVM=1 oldconfig
...
scripts/Kconfig.include:40: linker 'ld.lld' not found

I tried this using the prebuilt binaries from apt.llvm.org:

$ make ARCH=hexagon LLVM=1 O=obj-hexagon CROSS_COMPILE=hexagon-linux-
LLVM_IAS=1 CC=llvm-12 defconfig modules vmlinux
<stdin>:1515:2: warning: syscall clone3 not implemented [-W#warnings]
#warning syscall clone3 not implemented
 ^
1 warning generated.
Quoted 5 lines

Doing the same thing with allmodconfig results in an internal error with clang-12
while compiling kernel/locking/lockdep.c. Same thing with clang-13. After turning
off lock debugging, it seems fine.

Quoted 3 lines

I've reported this upstream with you on CC:

https://bugs.llvm.org/show_bug.cgi?id=50838

I've not actually got any mail from that Cc..

Quoted 6 lines

Christoph, that toolchain should work (I had to install libtinfo5 and libc++1-7 on Debian Buster):

$ export PATH=$HOME/tmp/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin:$PATH
$ make -skj"$(nproc)" ARCH=hexagon CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all
hch@brick:~/work/linux$ make -j4 ARCH=hexagon
CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all
HOSTCC  scripts/basic/fixdep
clang: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory
#8 lore raw

Hi Christoph,

Quoted 6 lines · Christoph Hellwig
Quoted 3 lines · Nathan Chancellor

I've reported this upstream with you on CC:

https://bugs.llvm.org/show_bug.cgi?id=50838

I've not actually got any mail from that Cc..

Sorry, I directed that at Arnd. I should have kept you in the loop too.

Quoted 11 lines
Quoted 6 lines

Christoph, that toolchain should work (I had to install libtinfo5 and libc++1-7 on Debian Buster):

$ export PATH=$HOME/tmp/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin:$PATH
$ make -skj"$(nproc)" ARCH=hexagon CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all
hch@brick:~/work/linux$ make -j4 ARCH=hexagon
CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all
HOSTCC  scripts/basic/fixdep
clang: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

Hmmm, is that with libtinfo5 installed (or whatever the ncurses-compat equivalent is on your distribution installed)? I had that problem on Debian until I insta

Brian/Sid, it might be worth flipping LLVM_ENABLE_TERMINFO to OFF during your cmake configuration so that there are less dynamic dependencies and it is easier for more people to run the toolchain. Android's clang team did the same thing:

https://github.com/android/ndk/issues/574

With https://reviews.llvm.org/D42055, there should not be too much of a sacrifice.

Cheers,
Nathan

#9 lore raw
Quoted 8 lines · Nathan Chancellor
Quoted 4 lines
hch@brick:~/work/linux$ make -j4 ARCH=hexagon
CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all
HOSTCC  scripts/basic/fixdep
clang: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

Hmmm, is that with libtinfo5 installed (or whatever the ncurses-compat equivalent is on your distribution installed)? I had that problem on Debian until I insta

I did install libtinfo5, which just gets me to the next error:

hch@brick:~/work/linux$ export PATH=/opt/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin/:$PATH
hch@brick:~/work/linux$ make -j4 ARCH=hexagon CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all
  HOSTCC  scripts/basic/fixdep
clang: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory
make[2]: *** [scripts/Makefile.host:95: scripts/basic/fixdep] Error 127
make[1]: *** [Makefile:543: scripts_basic] Error 2
make: *** [Makefile:346: __build_one_by_one] Error 2
#10 lore raw
Quoted 11 lines · Christoph Hellwig
Quoted 8 lines · Nathan Chancellor
Quoted 4 lines
hch@brick:~/work/linux$ make -j4 ARCH=hexagon
CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all
HOSTCC  scripts/basic/fixdep
clang: error while loading shared libraries: libtinfo.so.5: cannot open shared object file: No such file or directory

Hmmm, is that with libtinfo5 installed (or whatever the ncurses-compat equivalent is on your distribution installed)? I had that problem on Debian until I insta

Quoted 6 lines

I did install libtinfo5, which just gets me to the next error:

hch@brick:~/work/linux$ export PATH=/opt/clang+llvm-12.0.0-cross-hexagon-unknown-linux-musl/x86_64-linux-gnu/bin/:$PATH
hch@brick:~/work/linux$ make -j4 ARCH=hexagon CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all
  HOSTCC  scripts/basic/fixdep
clang: error while loading shared libraries: libc++.so.1: cannot open shared object file: No such file or directory

^ Nathan did mention earlier in the thread that he "had to install libtinfo5 and libc++1-7 on Debian Buster." Emphasis on the _and libc++_ part.

I'm not sure if that binary distribution came with a libc++.so.1; if so, that path needs to be specified via LD_LIBRARY_PATH so that the runtime loader can find it. Perhaps rpath wasn't set when the clang binary was built.

We're looking into statically linked images of clang to prevent these kinds of games.

Quoted 4 lines

make[2]: *** [scripts/Makefile.host:95: scripts/basic/fixdep] Error 127 make[1]: *** [Makefile:543: scripts_basic] Error 2
make: *** [Makefile:346: __build_one_by_one] Error 2

Signature · 2 lines
Thanks,
~Nick Desaulniers
#11 lore raw
Quoted 35 lines

-----Original Message-----
From: Nick Desaulniers <[email protected]>
Sent: Thursday, July 8, 2021 12:54 PM
To: Christoph Hellwig <[email protected]>
Cc: Nathan Chancellor <[email protected]>; Brian Cain <[email protected]>; Sid Manning <[email protected]>; Arnd Bergmann <[email protected]>; [email protected]; linux- [email protected]; [email protected]
Subject: Re: how can we test the hexagon port in mainline

Quoted 4 lines · Christoph Hellwig
Quoted 2 lines · Nathan Chancellor
Quoted 2 lines
hch@brick:~/work/linux$ make -j4 ARCH=hexagon
CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1

defconfig all

Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

HOSTCC scripts/basic/fixdep
clang: error while loading shared libraries: libtinfo.so.5: cannot open

shared object file: No such file or directory

Quoted 8 lines
Quoted 4 lines

Hmmm, is that with libtinfo5 installed (or whatever the ncurses-compat equivalent is on your distribution installed)? I had that problem on Debian until I insta

I did install libtinfo5, which just gets me to the next error:

hch@brick:~/work/linux$ export PATH=/opt/clang+llvm-12.0.0-cross-

hexagon-unknown-linux-musl/x86_64-linux-gnu/bin/:$PATH

Quoted 1 line
hch@brick:~/work/linux$ make -j4 ARCH=hexagon

CROSS_COMPILE=hexagon-unknown-linux-musl LLVM=1 LLVM_IAS=1 defconfig all

Quoted 2 lines

HOSTCC scripts/basic/fixdep
clang: error while loading shared libraries: libc++.so.1: cannot open shared

object file: No such file or directory

Quoted 8 lines

^ Nathan did mention earlier in the thread that he "had to install libtinfo5 and libc++1-7 on Debian Buster." Emphasis on the _and libc++_ part.

I'm not sure if that binary distribution came with a libc++.so.1; if so, that path needs to be specified via LD_LIBRARY_PATH so that the runtime loader can find it. Perhaps rpath wasn't set when the clang binary was built.

The only libc++ builds in this distribution are the target hexagon ones. I did not include a host x86_64 libc++.so library, but it does seem like it would be more convenient if we did.

Nathan suggested disabling the terminfo dependency, I will include that change for the next release that we produce. Also, the upcoming clang-13 release from releases.llvm.org should contain all the necessary fixes (discussed recently) to build kernel code for hexagon.

Quoted 2 lines

We're looking into statically linked images of clang to prevent these kinds of games.

Statically linking against libc++/libc++abi at least seems like a good idea. Let me know if we can help.

-Brian

#13 lore raw
In reply to Brian Cain
Quoted 11 lines · Brian Cain
Quoted 2 lines

-----Original Message-----
From: Christoph Hellwig <[email protected]>

...

Quoted 8 lines

Hi all,

the oldest supported gcc version in mainline is gcc 4.9. But the only hexagon crosscompiler I can find is the one Arnds website points to here:

https://mirrors.edge.kernel.org/pub/tools/crosstool/

Quoted 7 lines
Quoted 2 lines

which is a non-upstream gcc 4.6.1 port. How are we supposed to even build test hexagon code?

We have provided a clang-12-based toolchain here:

https://codelinaro.jfrog.io/artifactory/codelinaro-qemu/2021-05-12/clang+llv
m-12.0.0-cross-hexagon-unknown-linux-musl.tar.xz

Do you have something that is just a small compiler like Arnd's gcc build? That tarball is already 264MB with xz compression, which does not compare very favorablt to the 7MB for gcc. And I fear I don't really have space for that on my setups..