python3 on build host

I have a build host that I've used for years. It is not possible to [directly] update python3 on this system to one which is identified as suitable for use with bitbake.

To try and work around this, I created a meta-toolchain SDK for my target. I remember needing to do this years ago when one of my boxes also became out of date. Sadly, this process has not led me to a solution.

First question: is this the proper way to solve this problem? I simply can't update python3 on this box (I've tried), so I need another solution so I can continue to use this workhorse.

Here's a list of the stumbling blocks I've run across so far.

* Using a SDK toolchain ostensibly is incompatible with the normal bitbake flow. Here's what I did for my 'meta-board' BSP:

[gary@thor tmp]$ TEMPLATECONF=meta-board/conf . ./opt/amltd/poky/oe-init-build-env build
Error: The OE SDK/ADT was detected as already being present in this shell environment. Please use a clean shell when
sourcing this environment script.

* I tried to work around this:

[gary@thor tmp]$ unset OECORE_SDK_VERSION
[gary@thor tmp]$ TEMPLATECONF=meta-board/conf . ./opt/amltd/poky/oe-init-build-env build You had no conf/local.conf file. This configuration file has therefore been created for you with some default values. You may wish to edit it to, for example, select a different MACHINE (target hardware). See conf/local.conf for more information as common configuration options are commented.

You had no conf/bblayers.conf file. This configuration file has therefore been created for you with some default values. To add additional metadata layers into your configuration please add entries to conf/bblayers.conf.

The Yocto Project has extensive documentation about OE including a reference manual which can be found at:
http://yoctoproject.org/documentation

For more information about OpenEmbedded see their website:
http://www.openembedded.org/

### Shell environment set up for builds. ###

You can now run 'bitbake <target>'

So, it looked like I was making some progress. Sadly, it was short-lived:

[gary@thor build]$ bitbake production-testing-image Traceback (most recent call last):

    File "/home/gary/tmp/opt/amltd/poky/bitbake/bin/bitbake", line 31, in <module>
      import bb
    File "/home/gary/tmp/opt/amltd/poky/bitbake/lib/bb/__init__.py", line 43, in <module>
      import logging
    File "/opt/amltd/2.1+snapshot/sysroots/i686-amltdsdk-linux/usr/lib/python3.5/logging/__init__.py", line 26, in <module>
      import sys, os, time, io, traceback, warnings, weakref, collections
    File "/opt/amltd/2.1+snapshot/sysroots/i686-amltdsdk-linux/usr/lib/python3.5/traceback.py", line 3, in <module>
      import collections
    File "/opt/amltd/2.1+snapshot/sysroots/i686-amltdsdk-linux/usr/lib/python3.5/collections/__init__.py", line 10, in 

<module>
from operator import itemgetter as _itemgetter, eq as _eq
ImportError: No module named 'operator'

I moved a step further by copying /usr/lib/python3.5/operator.py from one of my working build hosts into the appropriate sysroots in the SDK. That only got me one more step along as I then needed _compat_pickle.py. Then it failed with no module 'fcntl' and at this point, I don't know where to find that.

Second question: even if this is not the correct way to solve my initial problem of providing a suitable python3 on my build host, shouldn't I be able to run bitbake (or indeed any suitably complex python3 program) using the SDK I installed?

Thanks for your time and I truly hope I can find a working answer for question one. I have deliveries upcoming to customers I know will not be happy that they have to update and I've positioned our BSP deliveries solely on Poky/Yocto.

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

The buildtools-tarball is what you want: you can either build your own (bitbake buildtools-tarball) or just download the one we've built for you. This looks like the right one:

http://downloads.yoctoproject.org/releases/yocto/yocto-2.1/buildtools/poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone-2.1.sh

Second question: even if this is not the correct way to solve my initial

Looks like this SDK doesn't actually contain a full copy of Python, just the pieces to make what you have in the SDK work (theoretically, at least). This is why we add python3-modules to the buildtools-tarball, so if you want your SDK to ship its own complete Py3 runtime then add python3-core python3-modules to it.

Ross

Yes, this is just what I needed (except I had to build my own because my box is i686)

My bitbake build is now running and it looks like this will work for me.

Thanks

What would be the best way to do that?

------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------

Oh good, next time we have a bug which only appears on 32-bit build hosts I'll ping you. :)

I think SDK_RDEPENDS += "nativesdk-python3-core nativesdk-python3-modules" should do the trick.

Ross

Not quite - that's the 2.1 buildtools so it contains Python 2, not 3.

We do have the 2.2 M1 version that does contain Python 3:

http://downloads.yoctoproject.org/releases/yocto/milestones/yocto-2.2_M1/buildtools/poky-glibc-x86_64-buildtools-tarball-core2-64-buildtools-nativesdk-standalone-2.1%2bsnapshot-20160621.sh

(Still 64-bit though.)

Cheers,
Paul

Paul Eggleton
Intel Open Source Technology Centre