package/dbus-python: Fix build with python 3.8
PATCH 1/1 package/dbus-python: Fix build with python 3.8
test/import-repeatedly.c uses an embedded python interpreter and PYTHON_EXTRA_LIBS is used only for building this test case, so set PYTHON_EXTRA_LIBS with python3-config --libs --embed to build this test case and don't link the module with libpython3.so.
Fixes:
- http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
- http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
and many similar failures.
---
Changes v1 -> v2:
- Set PYTHON_EXTRA_LIBS with python3-config --libs --embed instead of appending --embed to PYTHON_LIBS
--- package/dbus-python/dbus-python.mk | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-)
package/dbus-python/dbus-python.mk +4 -2
@@ -35,14 +35,16 @@ DBUS_PYTHON_DEPENDENCIES += python3 host-python3 DBUS_PYTHON_CONF_ENV += \ PYTHON=$(HOST_DIR)/bin/python3 \ PYTHON_INCLUDES="`$(STAGING_DIR)/usr/bin/python3-config --includes`" \ - PYTHON_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --ldflags`" + PYTHON_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --ldflags`" \ + PYTHON_EXTRA_LIBS="`$(STAGING_DIR)/usr/bin/python3-config --libs --embed`" HOST_DBUS_PYTHON_DEPENDENCIES += host-python3 HOST_DBUS_PYTHON_CONF_ENV += \ PYTHON=$(HOST_DIR)/bin/python3 \ PYTHON_INCLUDES="`$(HOST_DIR)/usr/bin/python3-config --includes`" \ - PYTHON_LIBS="`$(HOST_DIR)/usr/bin/python3-config --ldflags`" + PYTHON_LIBS="`$(HOST_DIR)/usr/bin/python3-config --ldflags`" \ + PYTHON_EXTRA_LIBS="`$(HOST_DIR)/usr/bin/python3-config --libs --embed`" endif $(eval $(autotools-package))
Signature · 1 line
2.23.0
PATCH 1/1 package/dbus-python: Fix build with python 3.8
Quoted 10 lines · On Wed, 23 Oct 2019 21:19:21 +0000 Grzegorz Blach <[email protected]> wrote:
test/import-repeatedly.cuses an embedded python interpreter andPYTHON_EXTRA_LIBSis used only for building this test case, so setPYTHON_EXTRA_LIBSwithpython3-config --libs --embedto build this test case and don't link the module with libpython3.so.Fixes:
- http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
- http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
and many similar failures.
Quoted 5 lines
Signed-off-by Grzegorz Blach <[email protected]>---
Changes v1 -> v2:
- SetPYTHON_EXTRA_LIBSwithpython3-config --libs --embedinstead of appending --embed toPYTHON_LIBS
This doesn't change the fix, which I think still isn't correct (but I haven't checked in details). A Python extension should not need the --embed option.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/dbus-python: Fix build with python 3.8
Quoted 19 lines · On Thu, 24 Oct 2019 10:05:34 +0200 Thomas Petazzoni…
Quoted 10 lines · On Wed, 23 Oct 2019 21:19:21 +0000 Grzegorz Blach <[email protected]> wrote:
test/import-repeatedly.cuses an embedded python interpreter andPYTHON_EXTRA_LIBSis used only for building this test case, so setPYTHON_EXTRA_LIBSwithpython3-config --libs --embedto build this test case and don't link the module with libpython3.so.Fixes:
- http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
- http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
and many similar failures.Quoted 5 lines
Signed-off-by Grzegorz Blach <[email protected]>---
Changes v1 -> v2:
- SetPYTHON_EXTRA_LIBSwithpython3-config --libs --embedinstead of appending --embed toPYTHON_LIBS
Quoted 3 lines
This doesn't change the fix, which I think still isn't correct (but I haven't checked in details). A Python extension should not need the --embed option.
And I replied too fast, without reading your commit log. Reading it, it makes more sense. However, I think it should be fixed in the dbus-python build system itself, so that the fix can be upstreamed. Could you have a look at that ?
Of course, it's not trivial, because you need to pass --embed for Python >= 3.8, but not for older versions of Python.
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com
PATCH 1/1 package/dbus-python: Fix build with python 3.8
Quoted 26 lines · Thomas Petazzoni
Quoted 19 lines · On Thu, 24 Oct 2019 10:05:34 +0200 Thomas Petazzoni…
Quoted 10 lines · On Wed, 23 Oct 2019 21:19:21 +0000 Grzegorz Blach <[email protected]> wrote:
test/import-repeatedly.cuses an embedded python interpreter andPYTHON_EXTRA_LIBSis used only for building this test case, so setPYTHON_EXTRA_LIBSwithpython3-config --libs --embedto build this test case and don't link the module with libpython3.so.Fixes:
- http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
- http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
and many similar failures.Quoted 5 lines
Signed-off-by Grzegorz Blach <[email protected]>---
Changes v1 -> v2:
- SetPYTHON_EXTRA_LIBSwithpython3-config --libs --embedinstead of appending --embed toPYTHON_LIBSQuoted 3 lines
This doesn't change the fix, which I think still isn't correct (but I haven't checked in details). A Python extension should not need the --embed option.
Quoted 7 lines
And I replied too fast, without reading your commit log. Reading it, it makes more sense. However, I think it should be fixed in the dbus-python build system itself, so that the fix can be upstreamed. Could you have a look at that ?
Of course, it's not trivial, because you need to pass --embed for Python >= 3.8, but not for older versions of Python.
Actually, it looks like this should be retrieved from sysconfig. So maybe it's
just a matter of setting the proper environment variables like we do in pkg-python? In other words, I think the entire logic in dbus-python.mk could be replaced with:
DBUS_PYTHON_CONF_ENV = $(PKG_PYTHON_SETUPTOOLS_ENV) HOST_DBUS_PYTHON_CONF_ENV = $(HOST_PKG_PYTHON_SETUPTOOLS_ENV)
We may still need to set PYTHON=$(HOST_DIR)/bin/python[23] as well.
Regards,
Arnout
PATCH v3 1/1 package/dbus-python: Fix build with python 3.8
In reply to Grzegorz BlachPython 3.8 doesn't allow python modules to be linked with libpythonX.Y.so, but applications that embed a python interpreter still need to be linked with libpythonX.Y.so. As a solution -lpythonX.Y need to be moved from PYTHON_LIBS to PYTHON_EXTRA_LIBS, but only for python 3.8 or newer.
---
Changes v2 -> v3:
- Patch m4/ax_python_devel.m4 instead of setting PYTHON_EXTRA_LIBS
Changes v1 -> v2:
- Set PYTHON_EXTRA_LIBS with python3-config --libs --embed instead of appending --embed to PYTHON_LIBS
--- .../0001-Fix-build-with-python-3.8.patch | 95 +++++++++++++++++++ package/dbus-python/dbus-python.mk | 4 + 2 files changed, 99 insertions(+) create mode 100644 package/dbus-python/0001-Fix-build-with-python-3.8.patch
package/dbus-python/0001-Fix-build-with-python-3.8.patch new file +95 -0
@@ -0,0 +1,95 @@ +From 1d7695681f1db3f6e3b487abbffc9665f1422bf4 Mon Sep 17 00:00:00 2001 +From: Grzegorz Blach <[email protected]> +Date: Thu, 24 Oct 2019 15:42:55 +0000 +Subject: [PATCH] Fix build with python 3.8 + +Python 3.8 doesn't allow python modules to be linked with +libpythonX.Y.so, but applications that embed a python interpreter +still need to be linked with libpythonX.Y.so. As a solution +-lpythonX.Y need to be moved from PYTHON_LIBS to PYTHON_EXTRA_LIBS, +but only for python 3.8 or newer. + +Signed-off-by: Grzegorz Blach <[email protected]> +--- + m4/ax_python_devel.m4 | 36 +++++++++++++++++++++++------------- + 1 file changed, 23 insertions(+), 13 deletions(-) + +diff --git a/m4/ax_python_devel.m4 b/m4/ax_python_devel.m4 +index 44dbd83..0de0c6e 100644 +--- a/m4/ax_python_devel.m4 ++++ b/m4/ax_python_devel.m4 +@@ -40,6 +40,7 @@ + # Copyright (c) 2009 Matteo Settenvini <[email protected]> + # Copyright (c) 2009 Horst Knorr <[email protected]> + # Copyright (c) 2013 Daniel Mullner <[email protected]> ++# Copyright (c) 2019 Grzegorz Blach <[email protected]> + # + # This program is free software: you can redistribute it and/or modify it + # under the terms of the GNU General Public License as published by the +@@ -67,7 +68,7 @@ + # modified version of the Autoconf Macro, you may extend this special + # exception to the GPL to apply to your modified version as well. + +-#serial 21 ++#serial 21 + python 3.8 fixes + + AU_ALIAS([AC_PYTHON_DEVEL], [AX_PYTHON_DEVEL]) + AC_DEFUN([AX_PYTHON_DEVEL],[ +@@ -198,6 +199,11 @@ EOD` + AC_DEFINE_UNQUOTED([HAVE_PYTHON], ["$ac_python_version"], + [If available, contains the Python version number currently in use.]) + ++ # Check if we should omit -lpython in PYTHON_LIBS ++ ac_omit_python_library=`$PYTHON -c "import sys; \ ++ ver = sys.version.split ()[[0]]; \ ++ print (ver >= '3.8.0')"` ++ + # First, the library directory: + ac_python_libdir=`cat<<EOD | $PYTHON - + +@@ -223,18 +229,22 @@ EOD` + # credits goes to momjian, I think. I'd like to put the right name + # in the credits, if someone can point me in the right direction... ? + # +- if test -n "$ac_python_libdir" -a -n "$ac_python_library" +- then +- # use the official shared library +- ac_python_library=`echo "$ac_python_library" | sed "s/^lib//"` +- PYTHON_LIBS="-L$ac_python_libdir -l$ac_python_library" ++ if test "$ac_omit_python_library" = "True"; then ++ PYTHON_LIBS="-L$ac_python_libdir" + else +- # old way: use libpython from python_configdir +- ac_python_libdir=`$PYTHON -c \ +- "from distutils.sysconfig import get_python_lib as f; \ +- import os; \ +- print (os.path.join(f(plat_specific=1, standard_lib=1), 'config'));"` +- PYTHON_LIBS="-L$ac_python_libdir -lpython$ac_python_version" ++ if test -n "$ac_python_libdir" -a -n "$ac_python_library" ++ then ++ # use the official shared library ++ ac_python_library=`echo "$ac_python_library" | sed "s/^lib//"` ++ PYTHON_LIBS="-L$ac_python_libdir -l$ac_python_library" ++ else ++ # old way: use libpython from python_configdir ++ ac_python_libdir=`$PYTHON -c \ ++ "from distutils.sysconfig import get_python_lib as f; \ ++ import os; \ ++ print (os.path.join(f(plat_specific=1, standard_lib=1), 'config'));"` ++ PYTHON_LIBS="-L$ac_python_libdir -lpython$ac_python_version" ++ fi + fi + + if test -z "PYTHON_LIBS"; then +@@ -265,7 +275,7 @@ EOD` + if test -z "$PYTHON_EXTRA_LIBS"; then + PYTHON_EXTRA_LIBS=`$PYTHON -c "import distutils.sysconfig; \ + conf = distutils.sysconfig.get_config_var; \ +- print (conf('LIBS') + ' ' + conf('SYSLIBS'))"` ++ print ('-lpython' + (conf('LDVERSION') or conf('VERSION')) + ' ' + conf('LIBS') + ' ' + conf('SYSLIBS'))"` + fi + AC_MSG_RESULT([$PYTHON_EXTRA_LIBS]) + AC_SUBST(PYTHON_EXTRA_LIBS) +-- +2.23.0 +
package/dbus-python/dbus-python.mk +4 -0
@@ -15,6 +15,10 @@ DBUS_PYTHON_CONF_OPTS = --disable-html-docs --disable-api-docs HOST_DBUS_PYTHON_DEPENDENCIES = host-dbus-glib HOST_DBUS_PYTHON_CONF_OPTS = --disable-html-docs --disable-api-docs +# We patch m4/ax_python_devel.m4 +DBUS_PYTHON_AUTORECONF = YES +HOST_DBUS_PYTHON_AUTORECONF = YES + ifeq ($(BR2_PACKAGE_PYTHON),y) DBUS_PYTHON_DEPENDENCIES += python host-python
Signature · 1 line
2.23.0
PATCH 1/1 package/dbus-python: Fix build with python 3.8
In reply to Grzegorz BlachQuoted 11 lines · On Wed, 23 Oct 2019 21:19:21 +0000 Grzegorz Blach <[email protected]> wrote:
test/import-repeatedly.cuses an embedded python interpreter andPYTHON_EXTRA_LIBSis used only for building this test case, so setPYTHON_EXTRA_LIBSwithpython3-config --libs --embedto build this test case and don't link the module with libpython3.so.Fixes:
- http://autobuild.buildroot.org/results/b30/b308eeb5c5d95ab9f1dbfc19f9183f2ba3ba0ce3/
- http://autobuild.buildroot.org/results/0dd/0dd9203f859b97ee5a3b6358644c26f8ab784ed8/
and many similar failures.Signed-off-by Grzegorz Blach <[email protected]>
Guess what, in the end, after investigating, I agree that this patch was the right solution! PYTHON_LIBS is used when building Python extensions and PYTHON_EXTRA_LIBS is used when building code that embeds a Python interpreter.
So this patch made perfect sense. Of course, it would be *much* better for ax_python_devel.m4 to use python-config, rather than do its convoluted logic, but your patch is simple is easy, so I've applied.
Thanks!
Thomas
Signature · 3 lines
Thomas Petazzoni, CTO, Bootlin Embedded Linux and Kernel engineering https://bootlin.com