fix python buildpaths QA issue
cover fix python buildpaths QA issue
- Patch 01-06 has be sent in previous pull request.
- add buildpaths QA ignore pattern, and ignore Makefile by default
- fix .pyc/.pyo buildpath QA issue
- fix python related buildpath QA issue
Test Steps:
vim local.conf
...
MACHINE ?= "qemux86-64"WARN_QA += "buildpaths"
require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "core2-32"
...
bitbake -k world
In oe-core, these patches fixed 40+ recipes' buildpath QA issue, and fixed all python related buildpath QA issue.
There are still 55 recipes left that has buildpath QA issue.
//Hongxu
The following changes since commit 23056103c949b498c23b47579e8dd57ce78e6ed9:
uclibc: Do not use immediate expansion operator (2016-02-22 20:42:48 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contrib hongxu/fix-buildpath
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=hongxu/fix-buildpath
Hongxu Jia (15):
depmodwrapper-cross: nopackages to avoid QA [buildpaths] issue
gcc: use relative path for configure script bison/gettext: add --with-bisonlocaledir to assign BISON_LOCALEDIR
openssl: substitute buildpath in source code
btrfs-tools: fix symlink creation multiple times
git: fix installed-vs-shipped QA Issue
insane:add buildpaths QA ignore pattern
python2/3: fix buildpath QA issue
bbclass distutils/distutils3: fix .pyc/.pyo buildpath python-setuptools/python3-setuptools: use old-style install
python3-pip: use old-style install
bbclass distutils/distutils3/setuptools/setuptools3: clean up
DISTUTILS_INSTALL_ARGS
python-pygobject: fix buildpath QA issue
python-numpy: fix buildpaths QA issue
python-pycairo: fix buildpath QA issue
meta/classes/distutils.bbclass | 8 +-- meta/classes/distutils3.bbclass | 8 +-- meta/classes/insane.bbclass | 13 +++++ meta/classes/setuptools.bbclass | 5 -- meta/classes/setuptools3.bbclass | 5 -- meta/recipes-connectivity/openssl/openssl.inc | 2 +- .../substitute-buildpath-in-source-code.patch | 41 ++++++++++++++ .../recipes-connectivity/openssl/openssl_1.0.2f.bb | 1 + .../gettext-0.19.6/add-with-bisonlocaledir.patch | 58 ++++++++++++++++++++ meta/recipes-core/gettext/gettext_0.19.6.bb | 4 ++ .../bison/bison/add-with-bisonlocaledir.patch | 58 ++++++++++++++++++++ meta/recipes-devtools/bison/bison_3.0.4.bb | 1 + .../fix-symlink-creation-multiple-times.patch | 46 ++++++++++++++++ .../btrfs-tools/btrfs-tools_4.4.bb | 1 + meta/recipes-devtools/gcc/gcc-runtime.inc | 3 +- meta/recipes-devtools/gcc/gcc-sanitizers.inc | 3 +- meta/recipes-devtools/gcc/libgcc-common.inc | 3 +- meta/recipes-devtools/gcc/libgfortran.inc | 3 +- meta/recipes-devtools/git/git.inc | 2 + .../remove-build-path-in-comments.patch | 30 +++++++++++ .../recipes-devtools/python/python-numpy_1.10.4.bb | 1 + ...trip-off-build-path-in-purported-filename.patch | 62 ++++++++++++++++++++++ .../python/python-pycairo_1.10.0.bb | 22 ++++---- .../python/python-pygobject_2.28.3.bb | 5 ++ .../python/python-setuptools_19.4.bb | 2 - .../use-relative-path-to-compile-source-file.patch | 31 +++++++++++ meta/recipes-devtools/python/python3-pip_8.0.0.bb | 12 ----- .../python/python3-setuptools_19.4.bb | 8 +-- .../use-relative-path-to-compile-source-file.patch | 31 +++++++++++ meta/recipes-devtools/python/python3_3.5.1.bb | 10 +++- meta/recipes-devtools/python/python_2.7.11.bb | 8 +++ .../recipes-kernel/kmod/depmodwrapper-cross_1.0.bb | 2 + 32 files changed, 438 insertions(+), 51 deletions(-) create mode 100644 meta/recipes-connectivity/openssl/openssl/substitute-buildpath-in-source-code.patch create mode 100644 meta/recipes-core/gettext/gettext-0.19.6/add-with-bisonlocaledir.patch create mode 100644 meta/recipes-devtools/bison/bison/add-with-bisonlocaledir.patch create mode 100644 meta/recipes-devtools/btrfs-tools/btrfs-tools/fix-symlink-creation-multiple-times.patch create mode 100644 meta/recipes-devtools/python/python-numpy/remove-build-path-in-comments.patch create mode 100644 meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch create mode 100644 meta/recipes-devtools/python/python/use-relative-path-to-compile-source-file.patch create mode 100644 meta/recipes-devtools/python/python3/use-relative-path-to-compile-source-file.patch
Signature · 1 line
1.9.1
PATCH 7/15 insane:add buildpaths QA ignore pattern
Add variable QA_BUILDPATHS_SKIP_PATTERN to ignore buildpaths QA while filename (do not include directory) matches patterns, the patterns are Unix shell style and separated by space.
[YOCTO #9169]
[YOCTO #7058]
--- meta/classes/insane.bbclass | 13 +++++++++++++ 1 file changed, 13 insertions(+)
meta/classes/insane.bbclass +13 -0
@@ -592,11 +592,16 @@ def package_qa_hash_style(path, name, d, elf, messages): package_qa_add_message(messages, "ldflags", "No GNU_HASH in the elf binary: '%s'" % path) +# Ignore buildpaths QA while filename (do not include directory) +# matches patterns, patterns are Unix shell style and separated by space +QA_BUILDPATHS_SKIP_PATTERN ??= "Makefile" QAPATHTEST[buildpaths] = "package_qa_check_buildpaths" def package_qa_check_buildpaths(path, name, d, elf, messages): """ Check for build paths inside target files and error if not found in the whitelist """ + from fnmatch import fnmatch + # Ignore .debug files, not interesting if path.find(".debug") != -1: return
@@ -609,6 +614,14 @@ def package_qa_check_buildpaths(path, name, d, elf, messages): if path.find(name + "/CONTROL/") != -1 or path.find(name + "/DEBIAN/") != -1: return + # Ignore matched pattern file + ignores = d.getVar('QA_BUILDPATHS_SKIP_PATTERN', True).split() + file = os.path.basename(path) + for ignore in ignores: + if fnmatch(file, ignore): + bb.note("Package %s skipping buildpaths QA: %s" % (name, path)) + return + tmpdir = d.getVar('TMPDIR', True) with open(path) as f: file_content = f.read()
Signature · 1 line
1.9.1
PATCH 8/15 python2/3: fix buildpath QA issue
In reply to Hongxu Jia- use relative path to compile source file
- Since _sysconfigdata.py was modified at do_package, so remove _sysconfigdata.pyc is ok (Python2 only).
- Add _sysconfigdata.py and Makefile it to QA_BUILDPATHS_SKIP_PATTERN.
- Remove -fdebug-prefix-map in _sysconfigdata.py, Makefile and python3.5m-config (Python3 only).
[YOCTO #8446]
[YOCTO #7058]
--- .../use-relative-path-to-compile-source-file.patch | 31 ++++++++++++++++++++++ .../use-relative-path-to-compile-source-file.patch | 31 ++++++++++++++++++++++ meta/recipes-devtools/python/python3_3.5.1.bb | 10 ++++++- meta/recipes-devtools/python/python_2.7.11.bb | 8 ++++++ 4 files changed, 79 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-devtools/python/python/use-relative-path-to-compile-source-file.patch create mode 100644 meta/recipes-devtools/python/python3/use-relative-path-to-compile-source-file.patch
meta/recipes-devtools/python/python/use-relative-path-to-compile-source-file.patch new file +31 -0
@@ -0,0 +1,31 @@ +From 048a7fef98cb4b8323c0b144c0cc1cdff91c603c Mon Sep 17 00:00:00 2001 +From: Hongxu Jia <[email protected]> +Date: Tue, 23 Feb 2016 21:14:45 -0500 +Subject: [PATCH] use relative path to compile source file + +Use relative path rather than absolute path to compile source file. +This could avoid build path issues. + +Upstream-Status: Inappropriate [embedded specific] + +Signed-off-by: Hongxu Jia <[email protected]> +--- + Lib/distutils/ccompiler.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/Lib/distutils/ccompiler.py b/Lib/distutils/ccompiler.py +index 82fd7d5..eb86158 100644 +--- a/Lib/distutils/ccompiler.py ++++ b/Lib/distutils/ccompiler.py +@@ -342,7 +342,7 @@ class CCompiler: + + build = {} + for i in range(len(sources)): +- src = sources[i] ++ src = os.path.relpath(sources[i], '.') + obj = objects[i] + ext = os.path.splitext(src)[1] + self.mkpath(os.path.dirname(obj)) +-- +1.9.1 +
meta/recipes-devtools/python/python3/use-relative-path-to-compile-source-file.patch new file +31 -0
@@ -0,0 +1,31 @@ +From 048a7fef98cb4b8323c0b144c0cc1cdff91c603c Mon Sep 17 00:00:00 2001 +From: Hongxu Jia <[email protected]> +Date: Tue, 23 Feb 2016 21:14:45 -0500 +Subject: [PATCH] use relative path to compile source file + +Use relative path rather than absolute path to compile source file. +This could avoid build path issues. + +Upstream-Status: Inappropriate [embedded specific] + +Signed-off-by: Hongxu Jia <[email protected]> +--- + Lib/distutils/ccompiler.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/Lib/distutils/ccompiler.py b/Lib/distutils/ccompiler.py +index 82fd7d5..eb86158 100644 +--- a/Lib/distutils/ccompiler.py ++++ b/Lib/distutils/ccompiler.py +@@ -342,7 +342,7 @@ class CCompiler: + + build = {} + for i in range(len(sources)): +- src = sources[i] ++ src = os.path.relpath(sources[i], '.') + obj = objects[i] + ext = os.path.splitext(src)[1] + self.mkpath(os.path.dirname(obj)) +-- +1.9.1 +
meta/recipes-devtools/python/python3_3.5.1.bb +9 -1
@@ -37,6 +37,7 @@ SRC_URI += "\ file://setup.py-find-libraries-in-staging-dirs.patch \ file://use_packed_importlib.patch \ file://configure.ac-fix-LIBPL.patch \ + file://use-relative-path-to-compile-source-file.patch \ " SRC_URI[md5sum] = "e9ea6f2623fffcdd871b7b19113fde80" SRC_URI[sha256sum] = "c6d57c0c366d9060ab6c0cdf889ebf3d92711d466cc0119c441dbf2746f725c9"
@@ -179,6 +180,7 @@ do_install_append_class-nativesdk () { create_wrapper ${D}${bindir}/python${PYTHON_MAJMIN} TERMINFO_DIRS='${sysconfdir}/terminfo:/etc/terminfo:/usr/share/terminfo:/usr/share/misc/terminfo:/lib/terminfo' } +QA_BUILDPATHS_SKIP_PATTERN ?= "Makefile _sysconfigdata.py" SSTATE_SCAN_FILES += "Makefile" PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess"
@@ -187,8 +189,14 @@ py_package_preprocess () { install -m 0644 ${B}/Makefile.orig ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile # Remove references to buildmachine paths in target Makefile and _sysconfigdata sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \ + -e 's:-fdebug-prefix-map=${B}=/usr/src/${BPN}::g' \ + -e 's:-fdebug-prefix-map=${S}=/usr/src/${BPN}::g' \ + -e 's:-fdebug-prefix-map=${STAGING_DIR_NATIVE}=::g' \ + -e 's:-fdebug-prefix-map=${STAGING_DIR_HOST}=::g' \ ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile \ - ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py + ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py \ + ${PKGD}/${bindir}/python${PYTHON_BINABI}-config + } require python-${PYTHON_MAJMIN}-manifest.inc
meta/recipes-devtools/python/python_2.7.11.bb +8 -0
@@ -27,6 +27,7 @@ SRC_URI += "\ file://use_sysroot_ncurses_instead_of_host.patch \ file://avoid_parallel_make_races_on_pgen.patch \ file://add-CROSSPYTHONPATH-for-PYTHON_FOR_BUILD.patch \ + file://use-relative-path-to-compile-source-file.patch \ " S = "${WORKDIR}/Python-${PV}"
@@ -126,6 +127,7 @@ do_install_append_class-nativesdk () { create_wrapper ${D}${bindir}/python2.7 PYTHONHOME='${prefix}' TERMINFO_DIRS='${sysconfdir}/terminfo:/etc/terminfo:/usr/share/terminfo:/usr/share/misc/terminfo:/lib/terminfo' } +QA_BUILDPATHS_SKIP_PATTERN ?= "Makefile _sysconfigdata.py" SSTATE_SCAN_FILES += "Makefile" PACKAGE_PREPROCESS_FUNCS += "py_package_preprocess"
@@ -135,8 +137,14 @@ py_package_preprocess () { # Remove references to buildmachine paths in target Makefile and _sysconfigdata sed -i -e 's:--sysroot=${STAGING_DIR_TARGET}::g' -e s:'--with-libtool-sysroot=${STAGING_DIR_TARGET}'::g \ + -e 's:-fdebug-prefix-map=${B}=/usr/src/${BPN}::g' \ + -e 's:-fdebug-prefix-map=${S}=/usr/src/${BPN}::g' \ + -e 's:-fdebug-prefix-map=${STAGING_DIR_NATIVE}=::g' \ + -e 's:-fdebug-prefix-map=${STAGING_DIR_HOST}=::g' \ ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/config/Makefile \ ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.py + + rm -f ${PKGD}/${libdir}/python${PYTHON_MAJMIN}/_sysconfigdata.pyc } require python-${PYTHON_MAJMIN}-manifest.inc
Signature · 1 line
1.9.1
PATCH 9/15 bbclass distutils/distutils3: fix .pyc/.pyo buildpath
In reply to Hongxu JiaGet the "--root" directory supplied to the "install" command, and use it as a prefix to strip off the purported filename encoded in bytecode files.
Since --root added, we need to tweak --install-data, --prefix and --install-lib to use relative path.
[YOCTO #8446]
--- meta/classes/distutils.bbclass | 6 +++--- meta/classes/distutils3.bbclass | 6 +++--- 2 files changed, 6 insertions(+), 6 deletions(-)
meta/classes/distutils.bbclass +3 -3
@@ -4,8 +4,8 @@ DISTUTILS_BUILD_ARGS ?= "" DISTUTILS_STAGE_HEADERS_ARGS ?= "--install-dir=${STAGING_INCDIR}/${PYTHON_DIR}" DISTUTILS_STAGE_ALL_ARGS ?= "--prefix=${STAGING_DIR_HOST}${prefix} \ --install-data=${STAGING_DATADIR}" -DISTUTILS_INSTALL_ARGS ?= "--prefix=${D}/${prefix} \ - --install-data=${D}/${datadir}" +DISTUTILS_INSTALL_ARGS ?= "--prefix=${prefix} \ + --install-data=${datadir}" distutils_do_compile() { STAGING_INCDIR=${STAGING_INCDIR} \
@@ -38,7 +38,7 @@ distutils_do_install() { STAGING_LIBDIR=${STAGING_LIBDIR} \ PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \ BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \ - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \ + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --root=${D} --install-lib=${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \ bbfatal "${PYTHON_PN} setup.py install execution failed." # support filenames with *spaces*
meta/classes/distutils3.bbclass +3 -3
@@ -5,8 +5,8 @@ DISTUTILS_BUILD_EXT_ARGS ?= "" DISTUTILS_STAGE_HEADERS_ARGS ?= "--install-dir=${STAGING_INCDIR}/${PYTHON_DIR}" DISTUTILS_STAGE_ALL_ARGS ?= "--prefix=${STAGING_DIR_HOST}${prefix} \ --install-data=${STAGING_DATADIR}" -DISTUTILS_INSTALL_ARGS ?= "--prefix=${D}/${prefix} \ - --install-data=${D}/${datadir}" +DISTUTILS_INSTALL_ARGS ?= "--prefix=${prefix} \ + --install-data=${datadir}" distutils3_do_compile() { if [ ${BUILD_SYS} != ${HOST_SYS} ]; then
@@ -63,7 +63,7 @@ distutils3_do_install() { STAGING_LIBDIR=${STAGING_LIBDIR} \ PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \ BUILD_SYS=${BUILD_SYS} HOST_SYS=${SYS} \ - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --install-lib=${D}/${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \ + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --root=${D} --install-lib=${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \ bbfatal "${PYTHON_PN} setup.py install execution failed." # support filenames with *spaces*
Signature · 1 line
1.9.1
PATCH 10/15 python-setuptools/python3-setuptools: use old-style install
In reply to Hongxu JiaSince '--root' is added to 'setup.py install' to fix .pyc/.pyo buildpaths issue. For python setuptools, the '--root' option means explicit request for old-style install. There will be no egg generated. And it do not need setuptools.pth any more.
Rename easy_install to easy_install3.
Ubuntu or Debian did the same thing like us:
https://packages.debian.org/sid/all/python3-setuptools/filelist
[YOCTO #8446]
[YOCTO #9162]
--- meta/recipes-devtools/python/python-setuptools_19.4.bb | 2 -- meta/recipes-devtools/python/python3-setuptools_19.4.bb | 8 ++------ 2 files changed, 2 insertions(+), 8 deletions(-)
meta/recipes-devtools/python/python-setuptools_19.4.bb +0 -2
@@ -7,8 +7,6 @@ DEPENDS_class-native += "python-native" inherit distutils -DISTUTILS_INSTALL_ARGS += "--install-lib=${D}${libdir}/${PYTHON_DIR}/site-packages" - RDEPENDS_${PN} = "\ python-stringold \ python-email \
meta/recipes-devtools/python/python3-setuptools_19.4.bb +2 -6
@@ -5,13 +5,9 @@ DEPENDS_class-native += "python3-native" inherit distutils3 -DISTUTILS_INSTALL_ARGS += "--install-lib=${D}${libdir}/${PYTHON_DIR}/site-packages" - -# The installer puts the wrong path in the setuptools.pth file. Correct it. +# Rename easy_install to avoid confliction with python2's setuptools do_install_append() { - rm ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth - mv ${D}${bindir}/easy_install ${D}${bindir}/easy3_install - echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/setuptools.pth + mv ${D}${bindir}/easy_install ${D}${bindir}/easy_install3 } RDEPENDS_${PN} = "\
Signature · 1 line
1.9.1
PATCH 11/15 python3-pip: use old-style install
In reply to Hongxu JiaSince '--root' is added to 'setup.py install' to fix .pyc/.pyo buildpaths issue. For python3-pip, the '--root' option means there will be no eggs generated.
Ubuntu or Debian did the same thing like us:
https://packages.debian.org/sid/all/python3-pip/filelist
[YOCTO #8446]
--- meta/recipes-devtools/python/python3-pip_8.0.0.bb | 12 ------------ 1 file changed, 12 deletions(-)
meta/recipes-devtools/python/python3-pip_8.0.0.bb +0 -12
@@ -19,22 +19,10 @@ S = "${WORKDIR}/${SRCNAME}-${PV}" inherit distutils3 -DISTUTILS_INSTALL_ARGS += "--install-lib=${D}${libdir}/${PYTHON_DIR}/site-packages" - -do_install_prepend() { - install -d ${D}/${libdir}/${PYTHON_DIR}/site-packages -} - # Use setuptools site.py instead, avoid shared state issue do_install_append() { - rm ${D}/${libdir}/${PYTHON_DIR}/site-packages/site.py - rm ${D}/${libdir}/${PYTHON_DIR}/site-packages/__pycache__/site.cpython-*.pyc - # Install as pip3 and leave pip2 as default rm ${D}/${bindir}/pip - - # Installed eggs need to be passed directly to the interpreter via a pth file - echo "./${SRCNAME}-${PV}-py${PYTHON_BASEVERSION}.egg" > ${D}${PYTHON_SITEPACKAGES_DIR}/${SRCNAME}-${PV}.pth } RDEPENDS_${PN} = "\
Signature · 1 line
1.9.1
PATCH 12/15 bbclass distutils/distutils3/setuptools/setuptools3: clean up DISTUTILS_INSTALL_ARGS
In reply to Hongxu JiaSince fix .pyc/.pyo buildpaths issue, we have the same DISTUTILS_INSTALL_ARGS in these bbclass files, so clean up the duplicate.
[YOCTO #8446]
--- meta/classes/distutils.bbclass | 4 +++- meta/classes/distutils3.bbclass | 4 +++- meta/classes/setuptools.bbclass | 5 ----- meta/classes/setuptools3.bbclass | 5 ----- 4 files changed, 6 insertions(+), 12 deletions(-)
meta/classes/distutils.bbclass +3 -1
@@ -5,6 +5,8 @@ DISTUTILS_STAGE_HEADERS_ARGS ?= "--install-dir=${STAGING_INCDIR}/${PYTHON_DIR}" DISTUTILS_STAGE_ALL_ARGS ?= "--prefix=${STAGING_DIR_HOST}${prefix} \ --install-data=${STAGING_DATADIR}" DISTUTILS_INSTALL_ARGS ?= "--prefix=${prefix} \ + --root=${D} \ + --install-lib=${PYTHON_SITEPACKAGES_DIR} \ --install-data=${datadir}" distutils_do_compile() {
@@ -38,7 +40,7 @@ distutils_do_install() { STAGING_LIBDIR=${STAGING_LIBDIR} \ PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \ BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} \ - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --root=${D} --install-lib=${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \ + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install ${DISTUTILS_INSTALL_ARGS} || \ bbfatal "${PYTHON_PN} setup.py install execution failed." # support filenames with *spaces*
meta/classes/distutils3.bbclass +3 -1
@@ -6,6 +6,8 @@ DISTUTILS_STAGE_HEADERS_ARGS ?= "--install-dir=${STAGING_INCDIR}/${PYTHON_DIR}" DISTUTILS_STAGE_ALL_ARGS ?= "--prefix=${STAGING_DIR_HOST}${prefix} \ --install-data=${STAGING_DATADIR}" DISTUTILS_INSTALL_ARGS ?= "--prefix=${prefix} \ + --root=${D} \ + --install-lib=${PYTHON_SITEPACKAGES_DIR} \ --install-data=${datadir}" distutils3_do_compile() {
@@ -63,7 +65,7 @@ distutils3_do_install() { STAGING_LIBDIR=${STAGING_LIBDIR} \ PYTHONPATH=${D}${PYTHON_SITEPACKAGES_DIR} \ BUILD_SYS=${BUILD_SYS} HOST_SYS=${SYS} \ - ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install --root=${D} --install-lib=${PYTHON_SITEPACKAGES_DIR} ${DISTUTILS_INSTALL_ARGS} || \ + ${STAGING_BINDIR_NATIVE}/${PYTHON_PN}-native/${PYTHON_PN} setup.py install ${DISTUTILS_INSTALL_ARGS} || \ bbfatal "${PYTHON_PN} setup.py install execution failed." # support filenames with *spaces*
meta/classes/setuptools.bbclass +0 -5
@@ -1,8 +1,3 @@ inherit distutils DEPENDS += "python-distribute-native" - -DISTUTILS_INSTALL_ARGS = "--root=${D} \ - --prefix=${prefix} \ - --install-lib=${PYTHON_SITEPACKAGES_DIR} \ - --install-data=${datadir}"
meta/classes/setuptools3.bbclass +0 -5
@@ -1,8 +1,3 @@ inherit distutils3 DEPENDS += "python3-setuptools-native" - -DISTUTILS_INSTALL_ARGS = "--root=${D} \ - --prefix=${prefix} \ - --install-lib=${PYTHON_SITEPACKAGES_DIR} \ - --install-data=${datadir}"
Signature · 1 line
1.9.1
PATCH 13/15 python-pygobject: fix buildpath QA issue
In reply to Hongxu JiaThe variable PYTHON is assigned with native python. But for target, it caused pygobject-codegen has build path.
vim ${S}/codegen/pygobject-codegen-2.0.in
[snip]...
exec @PYTHON@ $codegendir/codegen.py "$@"
[snip]...
So we replace it by invoking "which python".
--- meta/recipes-devtools/python/python-pygobject_2.28.3.bb | 5 +++++ 1 file changed, 5 insertions(+)
meta/recipes-devtools/python/python-pygobject_2.28.3.bb +5 -0
@@ -51,3 +51,8 @@ FILES_${PN}-lib = "${libdir}/lib*.so.*" FILES_${PN}-dev += "${bindir} ${datadir}" BBCLASSEXTEND = "native" + +do_install_append_class-target() { + sed -i -e "s:${PYTHON}:\`which python\`:g" \ + ${D}${bindir}/pygobject-codegen-2.0 +}
Signature · 1 line
1.9.1
Re: [PATCH 13/15] python-pygobject: fix buildpath QA issue
meta/recipes-devtools/python/python-pygobject_2.28.3.bb FILES_${PN}-lib = "${libdir}/lib*.so.*" · line 58
Quoted 5 lines · Hongxu Jia
meta/recipes-devtools/python/python-pygobject_2.28.3.bb +4
-0+do_install_append_class-target() { + sed -i -e "s:${PYTHON}:\`which python\`:g" \ + ${D}${bindir}/pygobject-codegen-2.0 +}
If you're going to expect the shell to find the right python on the target there's no need to use which python as exec will search $PATH. However, we do know where Python on the target it: ${bindir}/python.
Ross
Re: [PATCH 13/15] python-pygobject: fix buildpath QA issue
Quoted 9 lines · Ross
+do_install_append_class-target() {
+ sed -i -e "s:${PYTHON}:\which python\:g" \
+ ${D}${bindir}/pygobject-codegen-2.0
+}If you're going to expect the shell to find the right python on the target there's no need to use
which pythonas exec will search $PATH. However, we do know where Python on the target it: ${bindir}/python.
pygobject 3.x (which replaces 2.x as a part of introspection series) does not contain this script anymore, so the patch is moot anyway.
Alex
Re: [PATCH 13/15] python-pygobject: fix buildpath QA issue
In reply to Burton, RossQuoted 11 lines · Ross
On 26 February 2016 at 01:45, Hongxu Jia <[email protected] <mailto:[email protected]>> wrote:
+do_install_append_class-target() {
+ sed -i -e "s:${PYTHON}:\which python\:g" \
+ ${D}${bindir}/pygobject-codegen-2.0
+}If you're going to expect the shell to find the right python on the target there's no need to use
which pythonas exec will search $PATH. However, we do know where Python on the target it: ${bindir}/python.
OK, I directly use '${bindir}/python' to instead.
//Hongxu
Quoted 1 line
Ross
PATCH 14/15 python-numpy: fix buildpaths QA issue
In reply to Hongxu JiaRemove build path in comments:
Without the fix:
...
|This file is generated by /buildarea/raid0/hjia/build-20160225-yocto- buildpath/tmp/work/core2-64-poky-linux/python-numpy/1.10.4-r0/numpy -1.10.4/setup.py
...
With the fix:
...
|This file is generated by /numpy-1.10.4/setup.py
...
[YOCTO #7058]
--- .../remove-build-path-in-comments.patch | 30 ++++++++++++++++++++++ .../recipes-devtools/python/python-numpy_1.10.4.bb | 1 + 2 files changed, 31 insertions(+) create mode 100644 meta/recipes-devtools/python/python-numpy/remove-build-path-in-comments.patch
meta/recipes-devtools/python/python-numpy/remove-build-path-in-comments.patch new file +30 -0
@@ -0,0 +1,30 @@ +From c560abff71f98a39a7401f08c2c13dad9ae7f15f Mon Sep 17 00:00:00 2001 +From: Hongxu Jia <[email protected]> +Date: Thu, 25 Feb 2016 01:23:32 -0500 +Subject: [PATCH] remove build path in comments + +It has build path in comments, so remove it. + +Upstream-Status: Inappropriate [openembedded specific] + +Signed-off-by: Hongxu Jia <[email protected]> +--- + numpy/distutils/misc_util.py | 2 +- + 1 file changed, 1 insertion(+), 1 deletion(-) + +diff --git a/numpy/distutils/misc_util.py b/numpy/distutils/misc_util.py +index 345e60f..dafb068 100644 +--- a/numpy/distutils/misc_util.py ++++ b/numpy/distutils/misc_util.py +@@ -2254,7 +2254,7 @@ def generate_config_py(target): + from distutils.dir_util import mkpath + mkpath(os.path.dirname(target)) + f = open(target, 'w') +- f.write('# This file is generated by %s\n' % (os.path.abspath(sys.argv[0]))) ++ f.write('# This file is generated by %s\n' % (os.path.abspath(sys.argv[0]).replace(os.path.abspath('../'),''))) + f.write('# It contains system_info results at the time of building this package.\n') + f.write('__all__ = ["get_info","show"]\n\n') + for k, i in system_info.saved_results.items(): +-- +1.9.1 +
meta/recipes-devtools/python/python-numpy_1.10.4.bb +1 -0
@@ -5,6 +5,7 @@ LIC_FILES_CHKSUM = "file://LICENSE.txt;md5=9f4e88b5748e8313caaf33d081ce65a3" SRC_URI = "${SOURCEFORGE_MIRROR}/numpy/numpy-${PV}.tar.gz \ file://0001-Don-t-search-usr-and-so-on-for-libraries-by-default-.patch \ + file://remove-build-path-in-comments.patch \ ${CONFIGFILESURI} " CONFIGFILESURI ?= ""
Signature · 1 line
1.9.1
PATCH 15/15 python-pycairo: fix buildpath QA issue
In reply to Hongxu Jia- Append function unpack_waf to do_unpack to do self-extracting first, we need to patch on the extracted files.
- Get "--destdir" directory supplied to the "install" command, and use it as a prefix to strip off the purported filename encoded in bytecode files.
- Do not need to generate pycairo.pc manually
[YOCTO #8446]
--- ...trip-off-build-path-in-purported-filename.patch | 62 ++++++++++++++++++++++ .../python/python-pycairo_1.10.0.bb | 22 ++++---- 2 files changed, 75 insertions(+), 9 deletions(-) create mode 100644 meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch
meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patch new file +62 -0
@@ -0,0 +1,62 @@ +From 86faf604fc8a7f17320db48734cda464119da387 Mon Sep 17 00:00:00 2001 +From: Hongxu Jia <[email protected]> +Date: Thu, 25 Feb 2016 04:22:07 -0500 +Subject: [PATCH] strip off build path in purported filename + +Get the "--destdir" directory supplied to the "install" command, +and use it as a prefix to strip off the purported filename +encoded in bytecode files. + +Upstream-Status: Inappropriate [openembedded specific] + +Signed-off-by: Hongxu Jia <[email protected]> +--- +diff --git a/.waf-1.6.3-3c3129a3ec8fb4a5bbc7ba3161463b22/waflib/Tools/python.py b/.waf-1.6.3-3c3129a3ec8fb4a5bbc7ba3161463b22/waflib/Tools/python.py +--- a/.waf-1.6.3-3c3129a3ec8fb4a5bbc7ba3161463b22/waflib/Tools/python.py ++++ b/.waf-1.6.3-3c3129a3ec8fb4a5bbc7ba3161463b22/waflib/Tools/python.py +@@ -26,7 +26,7 @@ int main() + INST=''' + import sys, py_compile + for pyfile in sys.argv[1:]: +- py_compile.compile(pyfile, pyfile + %r) ++ py_compile.compile(pyfile, pyfile + %r, %r) + ''' + def process_py(self,node): + try: +@@ -39,6 +39,16 @@ def process_py(self,node): + def inst_py(ctx): + install_pyfile(self,node) + self.bld.add_post_fun(inst_py) ++ ++def get_dfile(dfile): ++ if getattr(Options.options,'destdir'): ++ destdir = Options.options.destdir ++ if dfile[:len(destdir)] != destdir: ++ raise Errors.WafError("invalid destdir: filename %r doesn't start with %r" % ((dfile, destdir))) ++ dfile = dfile[len(destdir):] ++ ++ return dfile ++ + def install_pyfile(self,node): + tsk=self.bld.install_files(self.install_path,[node],postpone=False) + path=os.path.join(tsk.get_install_path(),node.name) +@@ -53,12 +63,14 @@ def install_pyfile(self,node): + if self.env['PYC']or self.env['PYO']: + info("+ byte compiling %r"%path) + if self.env['PYC']: +- argv=self.env['PYTHON']+['-c',INST%'c',path] ++ dfile = get_dfile(path+'c') ++ argv=self.env['PYTHON']+['-c',INST%('c',dfile),path] + ret=Utils.subprocess.Popen(argv).wait() + if ret: + raise Errors.WafError('pyc compilation failed %r'%path) + if self.env['PYO']: +- argv=self.env['PYTHON']+[self.env['PYFLAGS_OPT'],'-c',INST%'o',path] ++ dfile = get_dfile(path+'o') ++ argv=self.env['PYTHON']+[self.env['PYFLAGS_OPT'],'-c',INST%('o',dfile),path] + ret=Utils.subprocess.Popen(argv).wait() + if ret: + raise Errors.WafError('pyo compilation failed %r'%path) +-- +1.9.1 +
meta/recipes-devtools/python/python-pycairo_1.10.0.bb +13 -9
@@ -11,7 +11,9 @@ LIC_FILES_CHKSUM = "file://COPYING;md5=f2e071ab72978431b294a0d696327421 \ DEPENDS = "cairo" PR = "r2" -SRC_URI = "http://cairographics.org/releases/py2cairo-${PV}.tar.bz2" +SRC_URI = "http://cairographics.org/releases/py2cairo-${PV}.tar.bz2 \ + file://strip-off-build-path-in-purported-filename.patch \ +" SRC_URI[md5sum] = "20337132c4ab06c1146ad384d55372c5" SRC_URI[sha256sum] = "d30439f06c2ec1a39e27464c6c828b6eface3b22ee17b2de05dc409e429a7431"
@@ -22,8 +24,16 @@ inherit distutils pkgconfig BBCLASSEXTEND = "native" +do_unpack[postfuncs] += "unpack_waf" +do_unpack[vardeps] += "unpack_waf" +unpack_waf() { + # Just self extract tarball embedded in waf, do not care help message. + # Do not generate bytecode, becuase we need to patch python file later + PYTHONDONTWRITEBYTECODE=True ./waf --help >/dev/null +} + do_configure() { - BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} ./waf configure --prefix=${D}${prefix} --libdir=${D}${libdir} + BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} ./waf configure --prefix=${prefix} --libdir=${libdir} } do_compile() {
@@ -31,11 +41,5 @@ do_compile() { } do_install() { - ./waf install - sed \ - -e 's:@prefix@:${prefix}:' \ - -e 's:@VERSION@:${PV}:' \ - -e 's:@includedir@:${includedir}:' \ - pycairo.pc.in > pycairo.pc - install -m 0644 pycairo.pc ${D}${libdir}/pkgconfig/ + ./waf install --destdir=${D} }
Signature · 1 line
1.9.1
Re: [PATCH 15/15] python-pycairo: fix buildpath QA issue
meta/recipes-devtools/python/python-pycairo_1.10.0.bb inherit distutils pkgconfig · line 33
Quoted 10 lines · Hongxu Jia
meta/recipes-devtools/python/python-pycairo_1.10.0.bb +4
-0+do_unpack[postfuncs] += "unpack_waf" +do_unpack[vardeps] += "unpack_waf" +unpack_waf() { + # Just self extract tarball embedded in waf, do not care helpmessage.
+ # Do not generate bytecode, becuase we need to patch python file
latermeta/recipes-devtools/python/python-pycairo_1.10.0.bb +2
-0+ PYTHONDONTWRITEBYTECODE=True ./waf --help >/dev/null +}
Presumably this isn't pycairo-specific but impacts every recipe using waf?
Review
Quoted 7 lines
do_configure(){(quoted diff) +1
-3- BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} ./waf configure --prefix=${D}${prefix} --libdir=${D}${libdir} + BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} ./waf configure --prefix=${prefix} --libdir=${libdir} }
Looks like this recipe needs to use the waf class for a start.
Ross
Re: [PATCH 15/15] python-pycairo: fix buildpath QA issue
Quoted 24 lines · Ross
On 26 February 2016 at 01:45, Hongxu Jia <[email protected] <mailto:[email protected]>> wrote:
+do_unpack[postfuncs] += "
unpack_waf"
+do_unpack[vardeps] += "unpack_waf"
+unpack_waf() {
+ # Just self extract tarball embedded in waf, do not care
help message.
+ # Do not generate bytecode, becuase we need to patch
python file later
+ PYTHONDONTWRITEBYTECODE=True ./waf --help >/dev/null
+}Presumably this isn't pycairo-specific but impacts every recipe using waf?
do_configure() { - BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} ./waf configure --prefix=${D}${prefix} --libdir=${D}${libdir} + BUILD_SYS=${BUILD_SYS} HOST_SYS=${HOST_SYS} ./waf configure --prefix=${prefix} --libdir=${libdir} }
Quoted 2 lines
Looks like this recipe needs to use the waf class for a start.
OK, I will try to merge these to waf.bbclass. V2 Incoming
//Hongxu
Quoted 1 line
Ross
As Richard suggested ( see [YOCTO #9169]), I will drop 'insane:add buildpaths QA ignore pattern'
So please drop these, V2 incoming
//Hongxu
Quoted 97 lines · Hongxu Jia
- Patch 01-06 has be sent in previous pull request.
- add buildpaths QA ignore pattern, and ignore
Makefileby default- fix .pyc/.pyo buildpath QA issue
- fix python related buildpath QA issue
Test Steps:
vim local.conf
...
MACHINE ?= "qemux86-64"WARN_QA+= "buildpaths"
requireconf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "core2-32"
...bitbake -k world
In oe-core, these patches fixed 40+ recipes' buildpath QA issue, and fixed all python related buildpath QA issue.
There are still 55 recipes left that has buildpath QA issue.
//Hongxu
The following changes since commit
23056103c949b498c23b47579e8dd57ce78e6ed9:uclibc: Do not use immediate expansion operator (2016-02-22 20:42:48 +0000)
are available in the git repository at:
git://git.openembedded.org/openembedded-core-contribhongxu/fix-buildpath
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=hongxu/fix-buildpathHongxu Jia (15):
depmodwrapper-cross: nopackages to avoid QA [buildpaths] issue
gcc: use relative path for configure script bison/gettext: add --with-bisonlocaledir to assignBISON_LOCALEDIR
openssl: substitute buildpath in source code
btrfs-tools: fix symlink creation multiple times
git: fix installed-vs-shipped QA Issue
insane:add buildpaths QA ignore pattern
python2/3: fix buildpath QA issue
bbclass distutils/distutils3: fix .pyc/.pyo buildpath python-setuptools/python3-setuptools: use old-style install
python3-pip: use old-style install
bbclass distutils/distutils3/setuptools/setuptools3: clean up
DISTUTILS_INSTALL_ARGS
python-pygobject: fix buildpath QA issue
python-numpy: fix buildpaths QA issue
python-pycairo: fix buildpath QA issuemeta/classes/distutils.bbclass | 8 +-- meta/classes/distutils3.bbclass | 8 +-- meta/classes/insane.bbclass | 13 +++++ meta/classes/setuptools.bbclass | 5 -- meta/classes/setuptools3.bbclass | 5 -- meta/recipes-connectivity/openssl/openssl.inc | 2 +- .../substitute-buildpath-in-source-code.patch | 41 ++++++++++++++ .../recipes-connectivity/openssl/openssl_1.0.2f.bb | 1 + .../gettext-0.19.6/add-with-bisonlocaledir.patch | 58 ++++++++++++++++++++ meta/recipes-core/gettext/gettext_0.19.6.bb | 4 ++ .../bison/bison/add-with-bisonlocaledir.patch | 58 ++++++++++++++++++++ meta/recipes-devtools/bison/bison_3.0.4.bb | 1 + .../fix-symlink-creation-multiple-times.patch | 46 ++++++++++++++++ .../btrfs-tools/btrfs-tools_4.4.bb | 1 + meta/recipes-devtools/gcc/gcc-runtime.inc | 3 +- meta/recipes-devtools/gcc/gcc-sanitizers.inc | 3 +- meta/recipes-devtools/gcc/libgcc-common.inc | 3 +- meta/recipes-devtools/gcc/libgfortran.inc | 3 +- meta/recipes-devtools/git/git.inc | 2 + .../remove-build-path-in-comments.patch | 30 +++++++++++ .../recipes-devtools/python/python-numpy_1.10.4.bb | 1 + ...trip-off-build-path-in-purported-filename.patch | 62 ++++++++++++++++++++++ .../python/python-pycairo_1.10.0.bb | 22 ++++---- .../python/python-pygobject_2.28.3.bb | 5 ++ .../python/python-setuptools_19.4.bb | 2 - .../use-relative-path-to-compile-source-file.patch | 31 +++++++++++ meta/recipes-devtools/python/python3-pip_8.0.0.bb | 12 ----- .../python/python3-setuptools_19.4.bb | 8 +-- .../use-relative-path-to-compile-source-file.patch | 31 +++++++++++ meta/recipes-devtools/python/python3_3.5.1.bb | 10 +++- meta/recipes-devtools/python/python_2.7.11.bb | 8 +++ .../recipes-kernel/kmod/depmodwrapper-cross_1.0.bb | 2 + 32 files changed, 438 insertions(+), 51 deletions(-)create mode 100644
meta/recipes-connectivity/openssl/openssl/substitute-buildpath-in-source-code.patchcreate mode 100644meta/recipes-core/gettext/gettext-0.19.6/add-with-bisonlocaledir.patchcreate mode 100644meta/recipes-devtools/bison/bison/add-with-bisonlocaledir.patch
create mode 100644meta/recipes-devtools/btrfs-tools/btrfs-tools/fix-symlink-creation-multiple-times.patchcreate mode 100644meta/recipes-devtools/python/python-numpy/remove-build-path-in-comments.patchcreate mode 100644meta/recipes-devtools/python/python-pycairo/strip-off-build-path-in-purported-filename.patchcreate mode 100644meta/recipes-devtools/python/python/use-relative-path-to-compile-source-file.patchcreate mode 100644meta/recipes-devtools/python/python3/use-relative-path-to-compile-source-file.patch