uclibc-ng build failures on Xtensa

We are seeing build failures of uclibc-ng on Xtensa:

http://autobuild.buildroot.net/?reason=uclibc%&arch=xtensa

A relevant one is:

http://autobuild.buildroot.net/results/38e/38e9d9a76f1877e7952ad5c20a405382642fd227/build-end.log

Which goes like this:

/home/autobuild/autobuild/instance-7/output-1/host/lib/gcc/xtensa-buildroot-linux-uclibc/10.3.0/../../../../xtensa-buildroot-linux-uclibc/bin/ld: ldso/ldso/ld-uClibc_so.a(ldso.oS): compiled for a big endian system and target is little endian /home/autobuild/autobuild/instance-7/output-1/host/lib/gcc/xtensa-buildroot-linux-uclibc/10.3.0/../../../../xtensa-buildroot-linux-uclibc/bin/ld: failed to merge target specific data of file ldso/ldso/ld-uClibc_so.a(ldso.oS) /home/autobuild/autobuild/instance-7/output-1/host/lib/gcc/xtensa-buildroot-linux-uclibc/10.3.0/../../../../xtensa-buildroot-linux-uclibc/bin/ld: ldso/ldso/ld-uClibc_so.a(dl-tlsdesc.oS): compiled for a big endian system and target is little endian /home/autobuild/autobuild/instance-7/output-1/host/lib/gcc/xtensa-buildroot-linux-uclibc/10.3.0/../../../../xtensa-buildroot-linux-uclibc/bin/ld: failed to merge target specific data of file ldso/ldso/ld-uClibc_so.a(dl-tlsdesc.oS) /home/autobuild/autobuild/instance-7/output-1/host/lib/gcc/xtensa-buildroot-linux-uclibc/10.3.0/../../../../xtensa-buildroot-linux-uclibc/bin/ld: ldso/ldso/ld-uClibc_so.a(resolve.oS): compiled for a big endian system and target is little endian /home/autobuild/autobuild/instance-7/output-1/host/lib/gcc/xtensa-buildroot-linux-uclibc/10.3.0/../../../../xtensa-buildroot-linux-uclibc/bin/ld: failed to merge target specific data of file ldso/ldso/ld-uClibc_so.a(resolve.oS) /home/autobuild/autobuild/instance-7/output-1/host/lib/gcc/xtensa-buildroot-linux-uclibc/10.3.0/../../../../xtensa-buildroot-linux-uclibc/bin/ld: little endian output does not match Xtensa configuration

This happens on configurations that have:

BR2_XTENSA_CUSTOM=y

Indeed, in Buildroot we support two Xtensa variants:

config BR2_XTENSA_CUSTOM
bool "Custom Xtensa processor configuration" select BR2_ARCH_HAS_MMU_OPTIONAL

config BR2_xtensa_fsf
bool "fsf - Default configuration"
select BR2_ARCH_HAS_MMU_MANDATORY

So far, all builds were done using BR2_xtensa_fsf but with the full randomization, we've started doing builds with BR2_XTENSA_CUSTOM. However, for BR2_XTENSA_CUSTOM, the endianess is configurable:

choice
prompt "Target Architecture Endianness"
default BR2_XTENSA_LITTLE_ENDIAN
depends on BR2_XTENSA_CUSTOM

config BR2_XTENSA_LITTLE_ENDIAN
bool "Little endian"

config BR2_XTENSA_BIG_ENDIAN
bool "Big endian"

endchoice

And the default is BR2_XTENSA_LITTLE_ENDIAN, while normally for BR2_xtensa_fsf, the endianness is BIG.

Which configurations are supported?

Is there something to fix here, or
we just need to disallow the usage of little endian?

Quoted 4 lines

And the default is BR2_XTENSA_LITTLE_ENDIAN, while normally for BR2_xtensa_fsf, the endianness is BIG.

Which configurations are supported?

When BR2_XTENSA_CUSTOM is selected BR2_XTENSA_OVERLAY_FILE must be specified and selected endianness must match the endianness of the configuration in the overlay file. Empty BR2_XTENSA_OVERLAY_FILE is equivalent to fsf and it is big endian.

Quoted 2 lines

Is there something to fix here, or
we just need to disallow the usage of little endian?

Little endian xtensa configurations are well supported, see e.g. configs/qemu_xtensa_lx60_defconfig that uses the dc233c core.

Quoted 9 lines · On Fri, 22 Jul 2022 04:24:16 -0700 Max Filippov <[email protected]> wrote:

Quoted 4 lines

And the default is BR2_XTENSA_LITTLE_ENDIAN, while normally for BR2_xtensa_fsf, the endianness is BIG.

Which configurations are supported?

When BR2_XTENSA_CUSTOM is selected BR2_XTENSA_OVERLAY_FILE must be specified and selected endianness must match the endianness of the configuration in the overlay file. Empty BR2_XTENSA_OVERLAY_FILE is equivalent to fsf and it is big endian.

Hm, OK. So what should we do from a testing point of view? Disregard any testing of the BR2_XTENSA_CUSTOM case?

It's kind of weird to have BR2_XTENSA_LITTLE_ENDIAN be the default if the default is actually big endian (when there is no overlay file).

What do you think of:

arch/Config.in.xtensa +4 -2

index 45ad2dc487..a69964f515 100644
@@ -31,7 +31,7 @@ config BR2_XTENSA_OVERLAY_FILE
 choice
        prompt "Target Architecture Endianness"
        default BR2_XTENSA_LITTLE_ENDIAN
-       depends on BR2_XTENSA_CUSTOM
+       depends on BR2_XTENSA_CUSTOM && BR2_XTENSA_OVERLAY_FILE != ""
 
 config BR2_XTENSA_LITTLE_ENDIAN
        bool "Little endian"

@@ -43,7 +43,9 @@ endchoice
 
 config BR2_ENDIAN
        default "LITTLE"        if BR2_XTENSA_LITTLE_ENDIAN
-       default "BIG"           if BR2_xtensa_fsf || BR2_XTENSA_BIG_ENDIAN
+       default "BIG"           if BR2_xtensa_fsf
+       default "BIG"           if BR2_XTENSA_CUSTOM && BR2_XTENSA_OVERLAY_FILE = ""
+       default "BIG"           if BR2_XTENSA_BIG_ENDIAN
 
 config BR2_ARCH
        default "xtensa"        if BR2_xtensa

i.e basically: don't choice the endianness choice when the overlay file is empty, and enforce the fact that the endianess is big when BR2_XTENSA_CUSTOM with an empty overlay file.

Quoted 5 lines

Quoted 2 lines

Is there something to fix here, or
we just need to disallow the usage of little endian?

Little endian xtensa configurations are well supported, see e.g. configs/qemu_xtensa_lx60_defconfig that uses the dc233c core.

I see there are quite a few overlays at
https://github.com/jcmvbkbc/xtensa-toolchain-build/tree/master/overlays
for well-known cores. Would it make sense to have explicit support for some of them in Buildroot? At least having support for one of the little endian cores seems useful to increase the testing coverage.