Skip to contentSkip to conversation

Building older releases on modern distros (and vice versa)

DISCUSSION Richard Purdie 3 participants 6 messages Jan 17, 2020 – Jan 18, 2020

lore
Share & tools
lore
#0 lore raw

We've been discussing how we could:

a) build old releases on newer distros

as well as

b) how we could build new releases on old distros.

Our proposed answer has been "buildtools-extended-tarball" which includes nativesdk-gcc.

We need b) so we can install this on Centos7 and stop having to work around its really old compiler. In general this should improve our ability to support stable branches in general.

I wanted to stress test a) a bit, partly to prove that b) could work longer term.

I did this by attempting to build buildtools-extended-tarball on pyro which has a gcc v6 and then using that to build morty on an Ubuntu 18.04 system.

It wasn't plain sailing, I had to:

* Upgrade the uninative in pyro
* Switch to xz uninative to allow latest uninative to work
* Patch binutils to fix an ld.so.conf relocation issue
* Add an ld.so.conf to buildtools
* Add the buildtools-extended-tarall recipe

Patches are in:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=rpurdie/pyro

(some need forward porting to master)

Having got the output from the pyro build, in the morty build I added buildtools tarball to PATH before running the build. I had to disable uninative since it was going to get in the way and conflict:

$ x86_64-buildtools-extended-nativesdk-standalone-2.3.4.sh 

Extended Build tools installer version 2.3.4
============================================
Enter target directory for SDK (default: /opt/poky/2.3.4): ./buildtools
You are about to install the SDK to "/media/build1/poky-morty/buildtools". Proceed[Y/n]? Extracting SDK...............done
Setting it up...done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.

 $ . /media/build1/poky-morty/buildtools/environment-setup-x86_64-pokysdk-linux
$ .  /media/build1/poky-morty/buildtools/environment-setup-x86_64-pokysdk-linux
$ source oe-init-build-env
$ mkdir classes
$ touch classes/uninative.bbclas
$ bitbake bash
#1 lore raw
Quoted 32 lines · Richard Purdie

We've been discussing how we could:

a) build old releases on newer distros

as well as

b) how we could build new releases on old distros.

Our proposed answer has been "buildtools-extended-tarball" which includes nativesdk-gcc.

We need b) so we can install this on Centos7 and stop having to work around its really old compiler. In general this should improve our ability to support stable branches in general.

I wanted to stress test a) a bit, partly to prove that b) could work longer term.

I did this by attempting to build buildtools-extended-tarball on pyro which has a gcc v6 and then using that to build morty on an Ubuntu 18.04 system.

It wasn't plain sailing, I had to:

* Upgrade the uninative in pyro
* Switch to xz uninative to allow latest uninative to work
* Patch binutils to fix an ld.so.conf relocation issue
* Add an ld.so.conf to buildtools
* Add the buildtools-extended-tarall recipe

Patches are in:

Quoted 8 lines

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=rpurdie/pyro

(some need forward porting to master)

Having got the output from the pyro build, in the morty build I added buildtools tarball to PATH before running the build. I had to disable uninative since it was going to get in the way and conflict:

do we intend to generate extended buildtools tarballs for all releases if it works out well ?

Quoted 24 lines
$ x86_64-buildtools-extended-nativesdk-standalone-2.3.4.sh

Extended Build tools installer version 2.3.4
============================================
Enter target directory for SDK (default: /opt/poky/2.3.4): ./buildtools
You are about to install the SDK to "/media/build1/poky-morty/buildtools". Proceed[Y/n]? Extracting SDK...............done
Setting it up...done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.

  $ . /media/build1/poky-morty/buildtools/environment-setup-x86_64-pokysdk-linux
$ .  /media/build1/poky-morty/buildtools/environment-setup-x86_64-pokysdk-linux
$ source oe-init-build-env
$ mkdir classes
$ touch classes/uninative.bbclas
$ bitbake bash
#2 lore raw
Quoted 47 lines · Khem Raj
Quoted 37 lines · Richard Purdie

We've been discussing how we could:

a) build old releases on newer distros

as well as

b) how we could build new releases on old distros.

Our proposed answer has been "buildtools-extended-tarball" which includes nativesdk-gcc.

We need b) so we can install this on Centos7 and stop having to work
around its really old compiler. In general this should improve our ability to support stable branches in general.

I wanted to stress test a) a bit, partly to prove that b) could work
longer term.

I did this by attempting to build buildtools-extended-tarball on pyro
which has a gcc v6 and then using that to build morty on an Ubuntu 18.04 system.

It wasn't plain sailing, I had to:

* Upgrade the uninative in pyro
* Switch to xz uninative to allow latest uninative to work
* Patch binutils to fix an ld.so.conf relocation issue
* Add an ld.so.conf to buildtools
* Add the buildtools-extended-tarall recipe

Patches are in:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=rpurdie/pyro

Quoted 8 lines

(some need forward porting to master)

Having got the output from the pyro build, in the morty build I added
buildtools tarball to PATH before running the build. I had to disable
uninative since it was going to get in the way and conflict:

Quoted 3 lines

do we intend to generate extended buildtools tarballs for all releases
if it works out well ?

Going forward there will be one generated as part of the release process for each release, they're part of master builds already.

We only really need a couple of older older ones with older gccs to be able to support a range of different older releases but we should look to generate those.

Cheers,

Richard

#3 lore raw
Quoted 52 lines · Richard Purdie
Quoted 47 lines · Khem Raj
Quoted 37 lines · Richard Purdie

We've been discussing how we could:

a) build old releases on newer distros

as well as

b) how we could build new releases on old distros.

Our proposed answer has been "buildtools-extended-tarball" which includes nativesdk-gcc.

We need b) so we can install this on Centos7 and stop having to work
around its really old compiler. In general this should improve our ability to support stable branches in general.

I wanted to stress test a) a bit, partly to prove that b) could work
longer term.

I did this by attempting to build buildtools-extended-tarball on pyro
which has a gcc v6 and then using that to build morty on an Ubuntu 18.04 system.

It wasn't plain sailing, I had to:

* Upgrade the uninative in pyro
* Switch to xz uninative to allow latest uninative to work
* Patch binutils to fix an ld.so.conf relocation issue
* Add an ld.so.conf to buildtools
* Add the buildtools-extended-tarall recipe

Patches are in:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=rpurdie/pyro

Quoted 8 lines

(some need forward porting to master)

Having got the output from the pyro build, in the morty build I added
buildtools tarball to PATH before running the build. I had to disable
uninative since it was going to get in the way and conflict:

Quoted 3 lines

do we intend to generate extended buildtools tarballs for all releases
if it works out well ?

Quoted 7 lines

Going forward there will be one generated as part of the release process for each release, they're part of master builds already.

We only really need a couple of older older ones with older gccs to be able to support a range of different older releases but we should look to generate those.

Perhaps it will be good to have one with gcc 4.8 ( pre c++11 days )

Quoted 4 lines

Cheers,

Richard

#4 lore raw
In reply to Richard Purdie

I would like to thank you and Intel for taking up this task even thought the community seemed to have issues with Centos7 being dropped.

Is there anything community can do at this point?

- armin

Quoted 64 lines · Richard Purdie

We've been discussing how we could:

a) build old releases on newer distros

as well as

b) how we could build new releases on old distros.

Our proposed answer has been "buildtools-extended-tarball" which includes nativesdk-gcc.

We need b) so we can install this on Centos7 and stop having to work around its really old compiler. In general this should improve our ability to support stable branches in general.

I wanted to stress test a) a bit, partly to prove that b) could work longer term.

I did this by attempting to build buildtools-extended-tarball on pyro which has a gcc v6 and then using that to build morty on an Ubuntu 18.04 system.

It wasn't plain sailing, I had to:

* Upgrade the uninative in pyro
* Switch to xz uninative to allow latest uninative to work
* Patch binutils to fix an ld.so.conf relocation issue
* Add an ld.so.conf to buildtools
* Add the buildtools-extended-tarall recipe

Patches are in:

http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=rpurdie/pyro

(some need forward porting to master)

Having got the output from the pyro build, in the morty build I added buildtools tarball to PATH before running the build. I had to disable uninative since it was going to get in the way and conflict:

$ x86_64-buildtools-extended-nativesdk-standalone-2.3.4.sh 

Extended Build tools installer version 2.3.4
============================================
Enter target directory for SDK (default: /opt/poky/2.3.4): ./buildtools
You are about to install the SDK to "/media/build1/poky-morty/buildtools". Proceed[Y/n]? Extracting SDK...............done
Setting it up...done
SDK has been successfully set up and is ready to be used.
Each time you wish to use the SDK in a new shell session, you need to source the environment setup script e.g.

 $ . /media/build1/poky-morty/buildtools/environment-setup-x86_64-pokysdk-linux
$ .  /media/build1/poky-morty/buildtools/environment-setup-x86_64-pokysdk-linux
$ source oe-init-build-env
$ mkdir classes
$ touch classes/uninative.bbclas
$ bitbake bash
#5 lore raw
Quoted 5 lines · On Fri, 2020-01-17 at 14:41 -0800, akuster808 wrote:

I would like to thank you and Intel for taking up this task even thought
the community seemed to have issues with Centos7 being dropped.

Is there anything community can do at this point?

The work isn't complete, we have a proof of concept so far. The remaining work:

* forward port the binutils patch and buildtools-tarball changes to master
* create some mechanism in the autobuilder-helper to download and install (or init if already present) a given buildtools tarball for a given host distro before invoking builds.
* build a set of buildtools tarballs that cover the major gcc verions we need
* release/publish said tarballs
* add config to the autobuilder for the stable releases we want to build using this mechanism.
* potentially test and debug building older releases

Not sure who is going to do this but it isn't trivial and the work certainly isn't done yet.

Cheers,

Richard