Skip to contentSkip to conversation

How does python-native work?

DISCUSSION Gary Thomas 2 participants 6 messages Apr 23, 2014

lore
Share & tools
lore
#0 lore raw

How is python-native supposed to work? I can see that it's built and installed in my tmp/sysroots/HOST but it's installed in a sub-directory which is not found by normal means.

In particular, I'm trying to build a package which uses configure to discover python support and if present package a python interface library. In the config.log, I see this:
configure:12109: checking for python
configure:12127: found /usr/bin/python
configure:12140: result: /usr/bin/python
which only makes sense since there is no python in my sysroots (directly).

  $ find tmp/sysroots/x86_64-linux/usr/bin -name "python*"
  tmp/sysroots/x86_64-linux/usr/bin/python-native
  tmp/sysroots/x86_64-linux/usr/bin/python-native/python-config
  tmp/sysroots/x86_64-linux/usr/bin/python-native/python2
  tmp/sysroots/x86_64-linux/usr/bin/python-native/python2-config
  tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7-config
  tmp/sysroots/x86_64-linux/usr/bin/python-native/python
  tmp/sysroots/x86_64-linux/usr/bin/python-native/python2.7

Even if I convince configure where to find python, e.g. adding
--with-python=$OECORE_NATIVE_SYSROOT/usr/bin/python-native/python --with-python-config=$OECORE_NATIVE_SYSROOT/usr/bin/python-native/python-config
this still isn't right since python-config gives incorrect answers:

  $ tmp/sysroots/x86_64-linux/usr/bin/python-native/python-config --includes
  -I/usr/include/python2.7 -I/usr/include/python2.7

I would think that python-native would install python on my path (from tmp/sysroots) and as well provide python-config which gives the answers I can use, e.g. --includes which point to the python includes in tmp/sysroots/HOST, etc.

What am I missing and how do I work with this?

Thanks

Signature · 4 lines
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------
#1 lore raw
Quoted 4 lines · Gary Thomas

How is python-native supposed to work? I can see that it's built and installed in my tmp/sysroots/HOST but it's installed in a sub-directory which is not found by normal means.

inherit pythonnative (or one of the classes that inherit it)

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#2 lore raw
Quoted 8 lines · Chris Larson

On Wed, Apr 23, 2014 at 7:15 AM, Gary Thomas <[email protected] <mailto:[email protected]>> wrote:

How is python-native supposed to work? I can see that it's built and installed in my tmp/sysroots/HOST but it's installed in a sub-directory which is not found by normal means.

inherit pythonnative (or one of the classes that inherit it)

Thanks, I imported 'distutils' and that helped, but I still have problems with it. Looking
at log.do_configure:
checking for python... /local/rpi_2014-04-10/tmp/sysroots/x86_64-linux/usr/bin/python-native/python checking for python-config... /local/rpi_2014-04-10/tmp/sysroots/x86_64-linux/usr/bin/python-native/python-config Traceback (most recent call last):
File "/local/rpi_2014-04-10/tmp/sysroots/x86_64-linux/usr/bin/python-native/python-config", line 6, in <module>
from distutils import sysconfig
File "/local/rpi_2014-04-10/tmp/sysroots/x86_64-linux/usr/lib/python2.7/distutils/sysconfig.py", line 22, in <module>
PREFIX = os.path.normpath(sys.prefix).replace( os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
TypeError: expected a character buffer object

When I ran python-config before (original email), the results were from outside the build environment, i.e. the command line where I run 'bitbake'. When do_configure() runs in my recipe, the environment seems to be quite different. If I try to run python-config from devshell (for my recipe), it fails as above.

I'm sure I'm just missing something...

Thanks for the help

Signature · 4 lines
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------
#3 lore raw
Quoted 27 lines · Gary Thomas

On 2014-04-23 08:21, Chris Larson wrote:

On Wed, Apr 23, 2014 at 7:15 AM, Gary Thomas <[email protected] <mailto:

Quoted 9 lines

[email protected]>> wrote:

How is python-native supposed to work? I can see that it's built and installed in my tmp/sysroots/HOST but it's installed in a sub-directory which is not found by normal means.

inherit pythonnative (or one of the classes that inherit it)

Thanks, I imported 'distutils' and that helped, but I still have problems with it. Looking
at log.do_configure:
checking for python... /local/rpi_2014-04-10/tmp/
sysroots/x86_64-linux/usr/bin/python-native/python
checking for python-config... /local/rpi_2014-04-10/tmp/
sysroots/x86_64-linux/usr/bin/python-native/python-config
Traceback (most recent call last):

    File "/local/rpi_2014-04-10/tmp/sysroots/x86_64-linux/usr/bin/python-native/python-config",

line 6, in <module>
from distutils import sysconfig
File "/local/rpi_2014-04-10/tmp/sysroots/x86_64-linux/usr/lib/
python2.7/distutils/sysconfig.py", line 22, in <module>

      PREFIX = os.path.normpath(sys.prefix).replace(
Quoted 9 lines

os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
TypeError: expected a character buffer object

When I ran python-config before (original email), the results were from outside the build environment, i.e. the command line where I run 'bitbake'. When do_configure() runs in my recipe, the environment seems to be quite different. If I try to run python-config from devshell (for my recipe), it fails as above.

First, it's a bit worrisome that you're inheriting a class for distutils when your build doesn't actually use distutils.. but beyond that, I don't think the classes inherited by distutils assume you'll be running python-config. Add this:

export BUILD_SYS
export HOST_SYS

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics
#4 lore raw
Quoted 33 lines · Chris Larson

On Wed, Apr 23, 2014 at 7:57 AM, Gary Thomas <[email protected] <mailto:[email protected]>> wrote:

On 2014-04-23 08:21, Chris Larson wrote:

On Wed, Apr 23, 2014 at 7:15 AM, Gary Thomas <[email protected] <mailto:[email protected]> <mailto:[email protected] <mailto:[email protected]>>> wrote:

How is python-native supposed to work? I can see that it's built and installed in my tmp/sysroots/HOST but it's installed in a sub-directory which is not found by normal means.

inherit pythonnative (or one of the classes that inherit it)

Thanks, I imported 'distutils' and that helped, but I still have problems with it. Looking
at log.do_configure:
checking for python... /local/rpi_2014-04-10/tmp/__sysroots/x86_64-linux/usr/bin/__python-native/python checking for python-config... /local/rpi_2014-04-10/tmp/__sysroots/x86_64-linux/usr/bin/__python-native/python-config Traceback (most recent call last):
File "/local/rpi_2014-04-10/tmp/__sysroots/x86_64-linux/usr/bin/__python-native/python-config", line 6, in <module>
from distutils import sysconfig
File "/local/rpi_2014-04-10/tmp/__sysroots/x86_64-linux/usr/lib/__python2.7/distutils/sysconfig.__py", line 22, in <module>
PREFIX = os.path.normpath(sys.prefix).__replace( os.getenv("BUILD_SYS"), os.getenv("HOST_SYS") )
TypeError: expected a character buffer object

When I ran python-config before (original email), the results were from outside the build environment, i.e. the command line where I run 'bitbake'. When do_configure() runs in my recipe, the environment seems to be quite different. If I try to run python-config from devshell (for my recipe), it fails as above.

Quoted 5 lines

First, it's a bit worrisome that you're inheriting a class for distutils when your build doesn't actually use distutils.. but beyond that, I don't think the classes inherited by distutils assume you'll be running python-config. Add this:

export BUILD_SYS
export HOST_SYS

I went back to only inheriting pythonnative (I was a bit overzealous to follow your 'or a class that inherits it').

I found that I also needed:
export STAGING_INCDIR
export STAGING_LIBDIR

Perhaps pythonnative should imply those exports, as without them, python-config is useless.

Anyway, now the recipe successfully builds the python interface!!

Thanks for the help

Signature · 4 lines
------------------------------------------------------------
Gary Thomas                 |  Consulting for the
MLB Associates              |    Embedded world
------------------------------------------------------------
#5 lore raw
Quoted 12 lines · Gary Thomas

First, it's a bit worrisome that you're inheriting a class for distutils

Quoted 7 lines

when your build doesn't actually use distutils.. but beyond that, I don't think the classes inherited by
distutils assume you'll be running python-config. Add this:

export BUILD_SYS
export HOST_SYS

I went back to only inheriting pythonnative (I was a bit overzealous to follow your 'or a class that inherits it').

Quoted 8 lines

I found that I also needed:
export STAGING_INCDIR
export STAGING_LIBDIR

Perhaps pythonnative should imply those exports, as without them, python-config is useless.

Anyway, now the recipe successfully builds the python interface!!

Ah! I knew there was exports needed, but apparently I remembered them wrong. Sorry about that. Glad you got it going, though. I don't think pythonnative exporting those would necessarily be appropriate, though. Not everyone inheriting pythonnative needs to do anything more than run python itself :) Perhaps we need a new class.. though the class proliferation is already a bit much.

Signature · 5 lines
Christopher Larson
clarson at kergoth dot com
Founder - BitBake, OpenEmbedded, OpenZaurus
Maintainer - Tslib
Senior Software Engineer, Mentor Graphics