Skip to contentSkip to conversation

Fixes for python3 and dbus-ptest

PATCH New [email protected] 1 participant 4 messages Nov 1, 2016

lore
Share & tools
lore
#0 lore raw

cover Fixes for python3 and dbus-ptest

From: Kai Kang <[email protected]>

1
when multilib is enabled, python3 fails to start:

root@qemux86-64:~# python3 
Could not find platform independent libraries <prefix>
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Fatal Python error: Py_Initialize: Unable to get the locale encoding
ImportError: No module named 'encodings'

Current thread 0x00007f4d9b268700 (most recent call first):
Aborted

It is library path issue, and fix it. Then check sys.path are right:

root@qemux86-64:~# python3
Python 3.5.2 (default, Nov  1 2016, 15:32:42) 
[GCC 6.2.0] on linux
Type "help", "copyright", "credits" or "license" for more information.
Quoted 2 lines
Quoted 2 lines
Quoted 2 lines

import sys
sys.path

['', '/usr/lib64/python35.zip', '/usr/lib64/python3.5', '/usr/lib64/python3.5/plat-linux', '/usr/lib64/python3.5/lib-dynload']

Quoted 1 line
Quoted 1 line
Quoted 1 line

2
Only 4 cases PASS of dbus-ptest. With the fixes, all ptests cases PASS on qemux86-64. And for qemuarm, only one case fails:

./run-ptest: line 10: 283 Aborted ./$i ./test/data DBUS_TEST_HOMEDIR=./test > /dev/null
FAIL: test/test-refs

It is a multi-threads issue, and need more work to fix it.

---
The following changes since commit 003b053498aea91fdb8c5418d0d8bbda7b23dbc0:

boost: fix the SRC_URI to point to an actual release, and not a master snapshot (2016-10-31 14:23:12 +0000)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib kangkai/python3_and_dbus
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=kangkai/python3_and_dbus

Kai Kang (3):
dbus-test: correct paths in service and conf files
dbus: update build options when enable ptest
python3: fix library search path

 meta/recipes-core/dbus/dbus-test_1.10.10.bb        |  3 ++
 meta/recipes-core/dbus/dbus_1.10.10.bb             |  4 +-
 ...t-python3-lib-pathes-for-multilib-support.patch | 49 ++++++++++++++++++++++
 .../python/python3/python-3.3-multilib.patch       | 13 +++---
 meta/recipes-devtools/python/python3_3.5.2.bb      |  1 +
 5 files changed, 61 insertions(+), 9 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3/correct-python3-lib-pathes-for-multilib-support.patch
Signature · 1 line
2.10.1
#1 lore raw

PATCH 1/3 dbus-test: correct paths in service and conf files

From: Kai Kang <[email protected]>

Build path ${B} exists in some service and conf files of ptest. Replace with right directory.

---
 meta/recipes-core/dbus/dbus-test_1.10.10.bb | 3 +++
 1 file changed, 3 insertions(+)
meta/recipes-core/dbus/dbus-test_1.10.10.bb +3 -0
index 650b7ab..29d7f91 100644
@@ -54,5 +54,8 @@ do_install_ptest() {
 	for i in $l; do install ${B}/bus/.libs/test-$i ${D}${PTEST_PATH}/test; done
 	install ${B}/dbus/.libs/test-dbus ${D}${PTEST_PATH}/test
 	cp -r ${B}/test/data ${D}${PTEST_PATH}/test
+
+	# set right path in service and conf files
+	find ${D} -type f -not -executable -exec sed -i 's#${B}#${PTEST_PATH}#' {} \;
 }
 RDEPENDS_${PN}-ptest += "bash"
Signature · 1 line
2.10.1
#2 lore raw

PATCH 2/3 dbus: update build options when enable ptest

In reply to [email protected]

From: Kai Kang <[email protected]>

It fails to run some dbus-ptest cases that some symbols cannot be found:

| ./test/test-bus: relocation error: ./test/test-bus: symbol
| _dbus_threads_init_debug, version LIBDBUS_PRIVATE_1.10.10 not defined
| in file libdbus-1.so.3 with link time reference
| FAIL: test/test-bus

These missing symbols are controlled by some macros. Update configure options and compile macro to make the symbols visible to ptest cases.

---
 meta/recipes-core/dbus/dbus_1.10.10.bb | 4 +++-
 1 file changed, 3 insertions(+), 1 deletion(-)
meta/recipes-core/dbus/dbus_1.10.10.bb +3 -1
index 4db0b9b..214cc13 100644
@@ -90,7 +90,7 @@ pkg_postinst_dbus() {
 	fi
 }
 
-EXTRA_OECONF = "--disable-tests \
+EXTRA_OECONF = "${@bb.utils.contains('DISTRO_FEATURES', 'ptest', '--enable-embedded-tests --enable-asserts --enable-verbose-mode', '--disable-tests', d)} \
                 --disable-xml-docs \
                 --disable-doxygen-docs \
                 --disable-libaudit \
@@ -98,6 +98,8 @@ EXTRA_OECONF = "--disable-tests \
 
 EXTRA_OECONF_append_class-native = " --disable-selinux"
 
+EXTRA_OEMAKE = "${@bb.utils.contains('DISTRO_FEATURES', 'ptest', 'CFLAG_VISIBILITY=-fvisibility=default', '', d)}"
+
 PACKAGECONFIG ??= "${@bb.utils.contains('DISTRO_FEATURES', 'systemd', 'systemd', '', d)} \
                    ${@bb.utils.contains('DISTRO_FEATURES', 'largefile', 'largefile', '', d)} \
                    ${@bb.utils.contains('DISTRO_FEATURES', 'x11', 'x11', '', d)}"
Signature · 1 line
2.10.1
#3 lore raw

PATCH 3/3 python3: fix library search path

In reply to [email protected]

From: Kai Kang <[email protected]>

When enable multilib, it fails to run python3:

| Could not find platform independent libraries <prefix>
| Could not find platform dependent libraries <exec_prefix>
| Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
| Fatal Python error: Py_Initialize: Unable to get the locale encoding
| ImportError: No module named 'encodings'
|
| Current thread 0x00007f62ea5b2700 (most recent call first):
| Aborted (core dumped)

The root cause is the module search path /usr/lib is wrong for x86-64 when multilib is enabled. So replace fixed path string with right macro.

Clean up python-3.3-multilib.patch for Modules/getpath.c at same time.

---
 ...t-python3-lib-pathes-for-multilib-support.patch | 49 ++++++++++++++++++++++
 .../python/python3/python-3.3-multilib.patch       | 13 +++---
 meta/recipes-devtools/python/python3_3.5.2.bb      |  1 +
 3 files changed, 55 insertions(+), 8 deletions(-)
 create mode 100644 meta/recipes-devtools/python/python3/correct-python3-lib-pathes-for-multilib-support.patch
meta/recipes-devtools/python/python3/correct-python3-lib-pathes-for-multilib-support.patch new file +49 -0
new file mode 100644
index 0000000..8ca5a6a
@@ -0,0 +1,49 @@
+When enable multilib, it fails to run python3:
+
+| Could not find platform independent libraries <prefix>
+| Could not find platform dependent libraries <exec_prefix>
+| Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
+| Fatal Python error: Py_Initialize: Unable to get the locale encoding
+| ImportError: No module named 'encodings'
+|
+| Current thread 0x00007f62ea5b2700 (most recent call first):
+| Aborted (core dumped)
+
+The root cause is the module search path /usr/lib is wrong for x86-64 when
+multilib is enabled. So replace fixed path string with right macro.
+
+Upstream-Status: Pending
+
+Signed-off-by: Kai Kang <[email protected]>
+---
+diff --git a/Modules/getpath.c b/Modules/getpath.c
+index 18deb60..d22af53 100644
+--- a/Modules/getpath.c
++++ b/Modules/getpath.c
+@@ -494,7 +494,7 @@ calculate_path(void)
+     _pythonpath = Py_DecodeLocale(PYTHONPATH, NULL);
+     _prefix = Py_DecodeLocale(PREFIX, NULL);
+     _exec_prefix = Py_DecodeLocale(EXEC_PREFIX, NULL);
+-    lib_python = Py_DecodeLocale("lib/python" VERSION, NULL);
++    lib_python = Py_DecodeLocale(LIB_PYTHON, NULL);
+ 
+     if (!_pythonpath || !_prefix || !_exec_prefix || !lib_python) {
+         Py_FatalError(
+@@ -683,7 +683,7 @@ calculate_path(void)
+     }
+     else
+         wcsncpy(zip_path, _prefix, MAXPATHLEN);
+-    joinpath(zip_path, L"lib/python00.zip");
++    joinpath(zip_path, LIB L"/python00.zip");
+     bufsz = wcslen(zip_path);   /* Replace "00" with version */
+     zip_path[bufsz - 6] = VERSION[0];
+     zip_path[bufsz - 5] = VERSION[2];
+@@ -695,7 +695,7 @@ calculate_path(void)
+             fprintf(stderr,
+                 "Could not find platform dependent libraries <exec_prefix>\n");
+         wcsncpy(exec_prefix, _exec_prefix, MAXPATHLEN);
+-        joinpath(exec_prefix, L"lib/lib-dynload");
++        joinpath(exec_prefix, LIB L"/lib-dynload");
+     }
+     /* If we found EXEC_PREFIX do *not* reduce it!  (Yet.) */
+ 
meta/recipes-devtools/python/python3/python-3.3-multilib.patch +5 -8
index 056e8e7..f7e73ee 100644
@@ -7,6 +7,11 @@ get the sys.lib from python itself and do not use hardcoded value of 'lib'
 Signed-off-by: Khem Raj <[email protected]>
 Signed-off-by: Alejandro Hernandez <[email protected]>
 
+Remove the section to add a global var in Modules/getpath.c that the global var
+has been removed from python3 upstream.
+
+Signed-off-by: Kai Kang <[email protected]>
+
 Index: Python-3.5.2/Include/pythonrun.h
 ===================================================================
 --- Python-3.5.2.orig/Include/pythonrun.h
@@ -130,14 +135,6 @@ Index: Python-3.5.2/Modules/getpath.c
  #ifndef LANDMARK
  #define LANDMARK L"os.py"
  #endif
-@@ -113,6 +120,7 @@ static wchar_t prefix[MAXPATHLEN+1];
- static wchar_t exec_prefix[MAXPATHLEN+1];
- static wchar_t progpath[MAXPATHLEN+1];
- static wchar_t *module_search_path = NULL;
-+static wchar_t *lib_python = L"" LIB_PYTHON;
- 
- /* Get file status. Encode the path to the locale encoding. */
- 
 Index: Python-3.5.2/Python/getplatform.c
 ===================================================================
 --- Python-3.5.2.orig/Python/getplatform.c
meta/recipes-devtools/python/python3_3.5.2.bb +1 -0
index e6cbb9c..7e6f019 100644
@@ -37,6 +37,7 @@ SRC_URI += "\
             file://setup.py-find-libraries-in-staging-dirs.patch \
             file://configure.ac-fix-LIBPL.patch \
             file://python3-fix-CVE-2016-1000110.patch \
+            file://correct-python3-lib-pathes-for-multilib-support.patch \
            "
 SRC_URI[md5sum] = "8906efbacfcdc7c3c9198aeefafd159e"
 SRC_URI[sha256sum] = "0010f56100b9b74259ebcd5d4b295a32324b58b517403a10d1a2aa7cb22bca40"
Signature · 1 line
2.10.1