armv6k support in OE for raspberrypi and s3c6410

The topic from http://lists.openembedded.org/pipermail/openembedded-core/2014-August/095830.html is causing trouble again, specifically with meta-raspberrypi and boost-1.56. Here're the details:

For the purposes of defining preprocessor symbols like __GCC_HAVE_SYNC_COMPARE_AND_SWAP_4 gcc 4.9.1 partitions ARM (non-M) architectures into three groups:

1) armv2 armv2a armv3 armv3m armv4 armv4t armv5 armv5e armv5t armv5te iwmmxt iwmmxt2
2) armv6 armv6j armv6t2 armv6z
3) armv6k armv6zk armv7-a armv7-r armv7ve armv8-a armv8-a+crc

These indicate increasing levels of support for atomic instructions, and perhaps other differences. (There are other preprocessor defines like __ARCH_ARM_FOO that distinguish within these groups; here I'm focusing only on a specific known problem using __GCC_-prefixed symbols.)

Unlike normal builds of a gcc toolchain, OE builds the runtime libraries separately in gcc-runtime and using the machine's tuning flags which include the architecture. The flags affect how atomic operations are implemented in the libraries.

Concurrrent applications built to link against the libraries must be compiled to use compatible atomic operations, or they do not work. While this happens naturally for applications built under OE that use the machine-specific tuning flags, it does not occur for on-target builds or for builds using the OE toolchain outside of the OE environment, which instead normally default to the least capable set of features.

The patch I added to gcc-configure-common.inc attempted to fix this by configuring the compiler using --with-arch=foo in cases where the target used -march=foo.

This turns out to be unstable as the rules for inferring defaults are complex. From gcc/config/arm/arm.h:

/* Support for a compile-time default CPU, et cetera. The rules are:
--with-arch is ignored if -march or -mcpu are specified. --with-cpu is ignored if -march or -mcpu are specified, and is
overridden

    by --with-arch.
   --with-tune is ignored if -mtune or -mcpu are specified (but not 
affected
     by -march).

--with-float is ignored if -mfloat-abi is specified. --with-fpu is ignored if -mfpu is specified. --with-abi is ignored if -mabi is specified. --with-tls is ignored if -mtls-dialect is specified. */

As an example, while -mtune=arm1176jzf-s by itself would imply -march=armv6zk for a normally-built gcc, when gcc is built with --with-arch=armv6 the more capable implied architecture is not selected: the --with-arch setting takes precedence.

Which would be fine, except that meta-raspberrypi calls the arm1176jzf-s (which is armv6zk) an armv6 chip so gcc was built targeting a less-capable architecture. You have to explicitly add -march=armv6zk (or -march=armv6k) to make it work.

So it appears that, in order to correctly support the Raspberry PI, OE needs to have meta/conf/machine/include/arm/arch-armv7a.inc refactored to pull out armv6k, and any overrides related to atomic instructions updated to include armv6k as an option.

Or possibly meta-raspberrypi could just claim to be an armv7a platform, but that seems wrong.

Anybody have a better idea? Is there somebody who can step up and bell this cat? I'm stretching the limits of my cross-ARM familiarity here and really don't feel comfortable taking on this one.

Peter

PS: Lest anybody think this is a new issue that's the fault of gcc 4.9: it's not. It was there with gcc-4.8 and maybe even gcc 4.7 as well, but only a few people noticed and none of them told OE about it. https://groups.google.com/forum/#!msg/automatak-dnp3/Jisp_zGhd5I/qJy1jHLFQoUJ

Which are you trying to fix? The on target gcc or the SDK one?

In the SDK case, we have environment files which do explicitly specify the compiler flags.

On target is harder however the on target gcc is compiled to a specific PACKAGE_ARCH so we should be able to put specific tuning into that gcc. It does sound like the changes to gcc-configure-common.inc were not the way to resolve this though, I'd misunderstood what the patches were doing.

Can we fix this by adjusting gcc itself (the on target version)?

Its even a very old bug:

https://bugzilla.yoctoproject.org/show_bug.cgi?id=139

but we've not actually hit issues due to this before or at least they've not been reported in these terms.

Cheers,

Richard

On-target.

I want compiler invocations like "g++ -std=c++1y -pthread test.cc" with the on-target OE-supplied gcc to work at least as well as the same invocation using the same version of gcc built on-target. For this to happen, the OE gcc default architecture must be compatible with flags used to build the OE gcc-runtime library, just as gcc built on-target is compatible with gcc libraries built on-target.

There are at least two ways to make that happen: change the compiler default architecture to match the target-specific flags used for gcc-runtime, or remove the target-specific flags when building gcc-runtime. The latter approach makes things more consistent with how gcc is built and used outside of OE, but is not how OE has traditionally done things so the impact of the change might be significant. We've been trying the former solution, which also makes the on-target compiler generate native-optimized code by default (a bonus, fixing the bug you mention below).

It may still be the right way to solve it. It breaks with meta-raspberrypi because that arm1176jzf-s is not optimally DEFAULTTUNE="armv6" that it's currently using: -mtune=arm1176jzf-s -march=armv6 is not the same as -mtune=arm1176jzf-s alone. It should be DEFAULTTUNE="armv6k", but OE doesn't have that as an option. If it did we could build gcc --with-arch=armv6k and it'd work as well as the armv7a platforms.

Unfortunately, I think this would have to end up with creating new armv6k-vfp-poky-linux-gnueabi packages, which probably wouldn't fly. So it might be necessary to remove the TUNE_CCARGS flags from gcc-runtime after all.

Maybe. It makes me uncomfortable to have the on-target compiler use different target configuration options than the cross-compilers: I'd expect that to produce even more hidden inconsistencies.

I suspect there are issues with statically linked SDK-built applications with libstdc++ because the SDK gcc runtime libraries aren't built with the same TUNE_CCARGS flags applied by gcc-runtime for the on-target libraries, though maybe not because in that case they're built without target-specific assumptions.

Yes, but that bug relates to not generating target-optimized code by default: programs work, but aren't as fast as they could be. Now that people are using concurrent programs and the compilers are better, we're hitting the issues related to inconsistently identifying which instructions are available on the target.

Peter

Sorry; I misunderstood what -mtune was meant to do and made it sound worse than it is. -mtune must be used in conjunction with -march and does not provide a default architecture as I had expected. It's -mcpu that defaults -march.

--with-arch=foo in gcc's configuration switches is doing exactly what it should and I believe it's the best approach with minimal impact on how OE toolchains have historically been built and invoked. I have no concerns about this resolution.

(If others are less trusting, I have patches to revert the --with-arch change and remove the TARGET_CC_ARCH stuff from gcc-runtime, which is an alternative approach that also meets my requirement. But that has a much wider impact than what we're doing now and I don't think it's a good approach.)

On full investigation, the Boost issue is completely unrelated. Boost 1.56 simply will not work with any platform that explicitly uses -march=armv6. That needs to be fixed in Boost; reverting gcc won't help.

Alternatively meta-raspberrypi could set -march=armv6k and deal with the ramifications of doing that (viz making PACKAGE_ARCH reflect the difference from armv6). There's nothing at all wrong about it being armv6 as it is now: it just wouldn't be the best choice for atomic operations if armv6k were a supported option.

Peter