Skip to contentSkip to conversation

lib: oe: utils: always append host gcc version to NATIVELSBSTRING

PATCH In review [email protected] 4 participants 9 messages Jun 7, 2021 – Jun 8, 2021

lore
Share & tools
lore
#0 lore raw

PATCH 1/1 lib: oe: utils: always append host gcc version to NATIVELSBSTRING

By having multiple uninative feeds based on host gcc version (rather than a single uninative feed for all possible host gcc versions), we avoid a general class of problems (as first observed in the bug below) where changes in native gcc don't cause native sstate to regenerate. Below is another example where this problem is encountered across multiple supported host distros for a given poky version.

gcc 6 supports default generation of PIE, which gets enabled between Ubuntu 16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported distros of warrior. Consider a native package dependency chain as follows:

A
/ | \
B C D

If a complete set of native sstate for the above packages is built prior to the default PIE change, but a workstation upgrades its distro to a version that natively compiles PIE by default, the sstate hash is not currently tainted, so the newer compiler version will pull from sstate not compiled for PIE.

For example, if a source change to C causes part of the above chain to rebuild (C -> A), then this may result in non-PIE sstate (B and D) giving link errors with A.

ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
ld: final link failed: Nonrepresentable section on output

This appears to have been considered previously as an option to address the following bug, but appears to have been set aside only because of proximity to the end of a release cycle.

[YOCTO #10441]

---
 meta/lib/oe/utils.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/utils.py +1 -1
index a84039f585..8afe19fd99 100644
@@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):
         bb.fatal("Can't get compiler version from %s --version output" % compiler)
 
     version = match.group(1)
-    return "-%s" % version if version in ("4.8", "4.9") else ""
+    return "-%s" % version
 
 
 def get_multilib_datastore(variant, d):
Signature · 1 line
2.25.1
#1 lore raw
Quoted 24 lines · On Mon, 2021-06-07 at 17:25 -0400, [email protected] wrote:

By having multiple uninative feeds based on host gcc version (rather than a single uninative feed for all possible host gcc versions), we avoid a general class of problems (as first observed in the bug below) where changes in native gcc don't cause native sstate to regenerate. Below is another example where this problem is encountered across multiple supported host distros for a given poky version.

gcc 6 supports default generation of PIE, which gets enabled between Ubuntu 16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported distros of warrior. Consider a native package dependency chain as follows:

     A
   / | \
  B C D

If a complete set of native sstate for the above packages is built prior to the default PIE change, but a workstation upgrades its distro to a version that natively compiles PIE by default, the sstate hash is not currently tainted, so the newer compiler version will pull from sstate not compiled for PIE.

For example, if a source change to C causes part of the above chain to rebuild (C -> A), then this may result in non-PIE sstate (B and D) giving link errors with A.

Quoted 8 lines

ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
ld: final link failed: Nonrepresentable section on output

This appears to have been considered previously as an option to address the following bug, but appears to have been set aside only because of proximity to the end of a release cycle.

[YOCTO #10441]

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441

This doesn't look quite right?

Quoted 20 lines

---
 meta/lib/oe/utils.py | 2 +-

 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/utils.py +0 -0
index a84039f585..8afe19fd99 100644
@@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):
          bb.fatal("Can't get compiler version from %s --version output" % compiler)

     version = match.group(1)

meta/lib/oe/utils.py def host_gcc_version(d, taskcontextonly=False): · line 429

Quoted 2 lines
meta/lib/oe/utils.py +1 -1
-    return "-%s" % version if version in ("4.8", "4.9") else ""
+    return "-%s" % version

This is a huge performance penalty to put onto the build unfortunately and I really don't want to do that by default on the autobuilder. We haven't seen many of these errors on the autobuilder and I'm reluctant to take the performance hit on our testing.

There were specific issues with gcc 4.8 and 4.9 which required separate sstate. Do we know which gcc version switched the default and can we force older gcc's to use the same default or does it not work? I'm wondering if we could split off a "nopic" verison of sstate for example for the older gccs?

Cheers,

Richard

#2 lore raw
Quoted 5 lines · Richard Purdie

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441

This doesn't look quite right?

Yes, that's the one. I assumed this was what you meant by, "Another option would be to have multiple uninative feeds based on gcc
version rather than a single uninative one."

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441#c1

This is a huge performance penalty to put onto the build unfortunately and

Quoted 6 lines

I really
don't want to do that by default on the autobuilder. We haven't seen many of these
errors on the autobuilder and I'm reluctant to take the performance hit on our testing.

I realize this is not ideal, but I guess the trade-off is dealing with these issues as they arise on
a case-by-case basis.

There were specific issues with gcc 4.8 and 4.9 which required separate

Quoted 7 lines

sstate. Do we
know which gcc version switched the default and can we force older gcc's to use the
same default or does it not work? I'm wondering if we could split off a "nopic" verison
of sstate for example for the older gccs?

gcc 6 introduced an --enable-default-pie configure option, but I'm not exactly sure where
between Ubuntu 16.04 and Ubuntu 18.04 they picked that configure option up. Since packages
based on the older gcc already exist in sstate, I think we would have to force the newer
behavior on each affected package.

I realize we can just swizzle the compiler flags on the affected packages to make things happy,
but it seemed like this approach had at least been considered before, so I thought I would at least
get some feedback.

#3 lore raw
Quoted 10 lines · Kyle Russell
Quoted 5 lines · Richard Purdie

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441

This doesn't look quite right?

Yes, that's the one.  I assumed this was what you meant by, "Another option would be to have multiple uninative feeds based on gcc version rather than a single uninative one."

That is effectively what the code ended up doing and why 4.8 and 4.9 are split out. We only wanted to separate out the specific versions with issues as we wanted to allow reuse where we could.

Quoted 9 lines

https://bugzilla.yoctoproject.org/show_bug.cgi?id=10441#c1

Quoted 4 lines

This is a huge performance penalty to put onto the build unfortunately and I really don't want to do that by default on the autobuilder. We haven't seen many of these errors on the autobuilder and I'm reluctant to take the performance hit on our testing.

I realize this is not ideal, but I guess the trade-off is dealing with these issues as they arise on a case-by-case basis.

If they were many issues arriving that might be what we'd have to do but so far I've not seen reports of that many...

Quoted 8 lines
Quoted 6 lines

There were specific issues with gcc 4.8 and 4.9 which required separate sstate. Do we know which gcc version switched the default and can we force older gcc's to use the same default or does it not work? I'm wondering if we could split off a "nopic" verison of sstate for example for the older gccs?

Quoted 8 lines

gcc 6 introduced an --enable-default-pie configure option, but I'm not exactly sure where between Ubuntu 16.04 and Ubuntu 18.04 they picked that configure option up.  Since packages based on the older gcc already exist in sstate, I think we would have to force the newer behavior on each affected package.

I realize we can just swizzle the compiler flags on the affected packages to make things happy, but it seemed like this approach had at least been considered before, so I thought I would at least get some feedback.

As Andrea points out, we did fix this with a global include file which should have handled the issue. Are you using a release which doesn't have that or not using the include through your own custom distro?

Cheers,

Richard

#4 lore raw
Quoted 19 lines · Richard Purdie
Quoted 1 line
Quoted 1 line

There were specific issues with gcc 4.8 and 4.9 which required

separate sstate. Do we

Quoted 1 line
Quoted 1 line

know which gcc version switched the default and can we force older

gcc's to use the

Quoted 1 line
Quoted 1 line

same default or does it not work? I'm wondering if we could split off

a "nopic" verison

Quoted 3 lines
Quoted 2 lines

of sstate for example for the older gccs?

gcc 6 introduced an --enable-default-pie configure option, but I'm not

exactly sure where

Quoted 1 line

between Ubuntu 16.04 and Ubuntu 18.04 they picked that configure option

up. Since packages

Quoted 1 line

based on the older gcc already exist in sstate, I think we would have to

force the newer

Quoted 3 lines

behavior on each affected package.

I realize we can just swizzle the compiler flags on the affected

packages to make things happy,

Quoted 1 line

but it seemed like this approach had at least been considered before, so

Quoted 9 lines

I thought I would at least

Quoted 1 line

get some feedback.

As Andrea points out, we did fix this with a global include file which should have
handled the issue. Are you using a release which doesn't have that or not using
the include through your own custom distro?

Yes indeed, nice find. bitbake -e confirms I'm including uninative-flags.inc. I do think this is exactly the
same issue, but there's a different set of dependencies involved in my chain. As someone suggested earlier,
I can force -fPIE on the packages where this is a problem, so I could of course do something similar for those
packages.

But I still feel like there's an underlying uninative limitation that's not being addressed, or highlighted. For example,
the SSTATE_MIRRORS documentation seems to imply that differentiating between uninative based on native gcc
version is the normal/correct behavior, even though it's clear to me now that the example was chosen specifically
for the 4.8/4.9 issue, not just the general case.

"When pointing to sstate build artifacts on another machine that uses a different GCC version for native builds,
you must configure SSTATE_MIRROR with a regular expression that maps local search paths to server paths.
The paths need to take into account NATIVELSBSTRING set by the uninative class."
-
https://www.yoctoproject.org/docs/current/mega-manual/mega-manual.html#var-SSTATE_MIRRORS

But I suppose even similar versions of gcc across different supported distros could be configured differently.
For example, warrior claims support for Ubuntu 16.04, 18.04, and Fedora 28, but only Ubuntu 18.04 seems to
use --enable-default-pie based on gcc -v. (Even newer Fedoras seem to not use this flag.)

I guess we could have a "nopic" split like you suggested. Or maybe uninative itself should be hashed against
the native gcc configure args and version, since version by itself also doesn't seem to be sufficient based on the
Fedora example above. Neither one of those sounds particularly appealing the more I think about it, so maybe
I'll just opt for fixing the packages like we've done for binutils-native.

#5 lore raw
In reply to [email protected]

Wouldn't it be easier to just force -fPIE on for those users for whom this is a problem? Or force -fPIE off for native everywhere. I'm not sure that doing something based on GCC version is obviously the right plan.

p.

Quoted 68 lines · On 7 June 2021 22:25:58 BST, [email protected] wrote:

By having multiple uninative feeds based on host gcc version (rather than a
single uninative feed for all possible host gcc versions), we avoid a general
class of problems (as first observed in the bug below) where changes in native
gcc don't cause native sstate to regenerate. Below is another example where
this problem is encountered across multiple supported host distros for a given
poky version.

gcc 6 supports default generation of PIE, which gets enabled between Ubuntu
16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported distros
of warrior. Consider a native package dependency chain as follows:

A
/ | \
B C D

If a complete set of native sstate for the above packages is built prior
to the default PIE change, but a workstation upgrades its distro to a version
that natively compiles PIE by default, the sstate hash is not currently tainted,
so the newer compiler version will pull from sstate not compiled for PIE.

For example, if a source change to C causes part of the above chain to rebuild
(C -> A), then this may result in non-PIE sstate (B and D) giving link errors
with A.

ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
ld: final link failed: Nonrepresentable section on output

This appears to have been considered previously as an option to address the following bug, but appears to have been set aside only because of proximity to the end of a release cycle.

[YOCTO #10441]

---
meta/lib/oe/utils.py | 2 +-

1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/utils.py +0 -0
index a84039f585..8afe19fd99 100644
@@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):

bb.fatal("Can't get compiler version from %s --version output" % compiler)

version = match.group(1)

meta/lib/oe/utils.py +1 -1
-    return "-%s" % version if version in ("4.8", "4.9") else ""
+    return "-%s" % version

def get_multilib_datastore(variant, d):

Signature · 1 line
2.25.1
Signature · 1 line
Sent from my Android device with K-9 Mail. Please excuse my brevity.
#6 lore raw
Quoted 5 lines · Phil Blundell via lists.openembedded.org

Wouldn't it be easier to just force -fPIE on for those users for whom this is a problem? Or force -fPIE off for native everywhere. I'm not sure that doing something based on GCC version is obviously the right plan.

p.

I was exactly wondering about distro/include/uninative-flags.inc which at the time was supposed to fix the changes btw ubuntu 16.x and 18.x. But this doesn't seem included nowhere.

A.A.

Quoted 61 lines · On 7 June 2021 22:25:58 BST, [email protected] wrote:
Quoted 44 lines

By having multiple uninative feeds based on host gcc version (rather than a single uninative feed for all possible host gcc versions), we avoid a general class of problems (as first observed in the bug below) where changes in native gcc don't cause native sstate to regenerate. Below is another example where this problem is encountered across multiple supported host distros for a given poky version.

gcc 6 supports default generation of PIE, which gets enabled between Ubuntu 16.04 (default gcc 5.4) and 18.04 (default gcc 7.5), both supported distros of warrior. Consider a native package dependency chain as follows:

A
/ | \
B C D

If a complete set of native sstate for the above packages is built prior to the default PIE change, but a workstation upgrades its distro to a version that natively compiles PIE by default, the sstate hash is not currently tainted, so the newer compiler version will pull from sstate not compiled for PIE.

For example, if a source change to C causes part of the above chain to rebuild (C -> A), then this may result in non-PIE sstate (B and D) giving link errors with A.

ld: relocation R_X86_64_32 against `.rodata.str1.8' can not be used when making a PIE object; recompile with -fPIC
ld: final link failed: Nonrepresentable section on output

This appears to have been considered previously as an option to address the following bug, but appears to have been set aside only because of proximity to the end of a release cycle.

[YOCTO #10441]

________________________________

 meta/lib/oe/utils.py | 2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/lib/oe/utils.py +0 -0
index a84039f585..8afe19fd99 100644

meta/lib/oe/utils.py def host_gcc_version(d, taskcontextonly=False): · line 432

Quoted 8 lines
meta/lib/oe/utils.py +1 -1
@@ -426,7 +426,7 @@ def host_gcc_version(d, taskcontextonly=False):
         bb.fatal("Can't get compiler version from %s --version output" % compiler)
 
     version = match.group(1)
-    return "-%s" % version if version in ("4.8", "4.9") else ""
+    return "-%s" % version
 
 
 def get_multilib_datastore(variant, d):

--
Sent from my Android device with K-9 Mail. Please excuse my brevity.

#7 lore raw
Quoted 11 lines · Andrea Adami
Quoted 5 lines · Phil Blundell via lists.openembedded.org

Wouldn't it be easier to just force -fPIE on for those users for whom this is a problem? Or force -fPIE off for native everywhere. I'm not sure that doing something based on GCC version is obviously the right plan.

p.

I was exactly wondering about distro/include/uninative-flags.inc which at the time was supposed to fix the changes btw ubuntu 16.x and 18.x. But this doesn't seem included nowhere.

Well remembered, I'd fogotten that. It is included by default:

$ grep uninative-flags * -R

meta/conf/distro/defaultsetup.conf:require conf/distro/include/uninative-flags.inc

Cheers,

Richard

#8 lore raw
Quoted 16 lines · Richard Purdie
Quoted 11 lines · Andrea Adami
Quoted 5 lines · Phil Blundell via lists.openembedded.org

Wouldn't it be easier to just force -fPIE on for those users for whom this is a problem? Or force -fPIE off for native everywhere. I'm not sure that doing something based on GCC version is obviously the right plan.

p.

I was exactly wondering about distro/include/uninative-flags.inc which at the time was supposed to fix the changes btw ubuntu 16.x and 18.x. But this doesn't seem included nowhere.

Well remembered, I'd fogotten that. It is included by default:

Quoted 8 lines
$ grep uninative-flags * -R

meta/conf/distro/defaultsetup.conf:require conf/distro/include/uninative-flags.inc

Cheers,

Richard

Oops, right, I was looking at distro includes but this is included by bitbake.conf
A.A.