Fix a number of package installation related items [v2]
cover Fix a number of package installation related items [v2]
This set of patches fixes a number of items related to regular and multilib package installs.
*** Updated the first two patches from Dongxiao. ***
The next 4 patches are required to fix the basic problems with rpm installation. Due to a typo, the provides were not being set properly causing various invalid rootfs's to be created. Fixing this typo showed a small number of additional runtime dependencies were not being accounted for:
* "rpmdeps" checks for GNU_HASH, and adds a requirement. For compatibility we now set rtld(GNU_HASH) as being provided by eglibc.
* the python specific requirements were versioned, however there is no easy way to satisfy the versioning within the OE-Core environment today. This was modified to instead simply require "python", which the python-core package satisfies... a small change to packages.bbclass was required to invalidate the sstate-cache as well.
* Enhance busybox to specific list a per-file runtime dependency for each link that /bin/busybox is capable of satisfying. This per-file dep is available for all packaging backends, but only affects RPM at this time.
Finally the last patch fixes a problem in the multilib_global.bbclass which causes incorrect provide and rprovides to be placed into packages when multilib builds are enabled.
*** Fourth patch was updated to remove debugging messages that were accidently left in ****
---
All of the above has been tested with
MACHINE = "qemux86_64"MULTILIB_IMAGE_INSTALL = "lib32-connman-gnome lib32-task-base-3g lib32-task-base-wifi lib32-task-base-bluetooth" require conf/multilib.conf
MULTILIBS = "multilib:lib32"
DEFAULTTUNE_virtclass-multilib-lib32 = "x86"
I have build core-image-minimal, core-image-core and core-image-sato. Each image was built with and without the MULTILIB_IMAGE_INSTALL being enabled.
The following changes since commit 81274f4488fbc4d68d150870735ec0181b60b451:
freetype: disable bzip2 compressed font support (2011-09-19 13:14:03 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib mhatle/rpm.deps
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/rpm.deps
Dongxiao Xu (2):
rpm: add multilib prefix for archs under deploy/rpm
multilib: install MULTILIB_IMAGE_INSTALL
Mark Hatle (5):
Fix RPM dependencies
Add a run-time dependency that eglibc support GNU_HASH Update python dependencies to be simply to "python"
busybox: Enhance to add dynamic per-file provides multilib_global.bbclass: Fix non-multilib package provides
meta/classes/multilib.bbclass | 5 +++ meta/classes/multilib_global.bbclass | 27 +++++++++++++--- meta/classes/package.bbclass | 6 ++++ meta/classes/package_rpm.bbclass | 44 +++++++++++++++++--------- meta/classes/rootfs_rpm.bbclass | 8 ++++- meta/recipes-core/busybox/busybox.inc | 32 +++++++++++++++++++ meta/recipes-core/busybox/busybox_1.18.5.bb | 2 +- meta/recipes-core/eglibc/eglibc-package.inc | 3 +- meta/recipes-core/eglibc/eglibc_2.12.bb | 2 +- meta/recipes-core/eglibc/eglibc_2.13.bb | 2 +- meta/recipes-devtools/rpm/rpm/pythondeps.sh | 16 ++++++++++ meta/recipes-devtools/rpm/rpm_5.4.0.bb | 4 ++- 12 files changed, 125 insertions(+), 26 deletions(-) create mode 100755 meta/recipes-devtools/rpm/rpm/pythondeps.sh
Signature · 1 line
1.7.3.4
PATCH 1/7 rpm: add multilib prefix for archs under deploy/rpm [v2]
From: Dongxiao Xu <[email protected]>
Currently MACHINE_ARCH deploy folder is unique in multilib system, thus a lib32 version of rpm package will override a normal rpm package if its PACKAGE_ARCH is ${MACHINE_ARCH}.
Define different deploy folder for multilib architectures to avoid the confliction.
--- meta/classes/multilib.bbclass | 5 +++++ meta/classes/rootfs_rpm.bbclass | 4 +++- 2 files changed, 8 insertions(+), 1 deletions(-)
meta/classes/multilib.bbclass +5 -0
@@ -76,4 +76,9 @@ python __anonymous () { multilib_map_variable("PACKAGES_DYNAMIC", variant, d) multilib_map_variable("PACKAGE_INSTALL", variant, d) multilib_map_variable("INITSCRIPT_PACKAGES", variant, d) + + package_arch = d.getVar("PACKAGE_ARCH", True) + machine_arch = d.getVar("MACHINE_ARCH", True) + if package_arch == machine_arch: + d.setVar("PACKAGE_ARCH", variant + "_" + package_arch) }
meta/classes/rootfs_rpm.bbclass +3 -1
@@ -218,7 +218,9 @@ python () { default_tune = localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False) if default_tune: localdata.setVar("DEFAULTTUNE", default_tune) - ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or "" + localdata.setVar("MACHINE_ARCH", eext[1] + "_" + localdata.getVar("MACHINE_ARCH", False)) + package_archs = localdata.getVar("PACKAGE_ARCHS", True) or "" + ml_package_archs += " " + package_archs #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides)) bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d) }
Signature · 1 line
1.7.3.4
PATCH 2/7 multilib: install MULTILIB_IMAGE_INSTALL [v2]
In reply to Mark HatleFrom: Dongxiao Xu <[email protected]>
If user set MULTILIB_IMAGE_INSTALL, we need to install those multitlib packages into the final image.
Also fix the logic in handling multilib prefix. For certain case like a normal image contains several multilib libraries, the image recipe isn't extended with MLPREFIX, therefore we need to enumerate the possible multilib prefixes and compare them with package prefixes.
--- meta/classes/package_rpm.bbclass | 42 +++++++++++++++++++++++++------------ meta/classes/rootfs_rpm.bbclass | 4 +++ 2 files changed, 32 insertions(+), 14 deletions(-)
meta/classes/package_rpm.bbclass +28 -14
@@ -166,7 +166,7 @@ package_install_internal_rpm () { local platform="${INSTALL_PLATFORM_RPM}" local platform_extra="${INSTALL_PLATFORM_EXTRA_RPM}" local confbase="${INSTALL_CONFBASE_RPM}" - local package_to_install="${INSTALL_PACKAGES_NORMAL_RPM}" + local package_to_install="${INSTALL_PACKAGES_NORMAL_RPM} ${INSTALL_PACKAGES_MULTILIB_RPM}" local package_attemptonly="${INSTALL_PACKAGES_ATTEMPTONLY_RPM}" local package_linguas="${INSTALL_PACKAGES_LINGUAS_RPM}" local providename="${INSTALL_PROVIDENAME_RPM}"
@@ -210,10 +210,15 @@ package_install_internal_rpm () { echo "Processing $pkg..." archvar=base_archs - ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,") - if [ "${ml_pkg}" != "${pkg}" ]; then - archvar=ml_archs - fi + ml_prefix=`echo ${pkg} | cut -d'-' -f1` + ml_pkg=$pkg + for i in ${MULTILIB_PREFIX_LIST} ; do + if [ ${ml_prefix} == ${i} ]; then + ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,") + archvar=ml_archs + break + fi + done pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg}) if [ -z "$pkg_name" ]; then
@@ -224,16 +229,20 @@ package_install_internal_rpm () { done fi fi - if [ ! -z "${package_to_install}" ]; then for pkg in ${package_to_install} ; do echo "Processing $pkg..." archvar=base_archs - ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,") - if [ "${ml_pkg}" != "${pkg}" ]; then - archvar=ml_archs - fi + ml_prefix=`echo ${pkg} | cut -d'-' -f1` + ml_pkg=$pkg + for i in ${MULTILIB_PREFIX_LIST} ; do + if [ ${ml_prefix} == ${i} ]; then + ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,") + archvar=ml_archs + break + fi + done pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg}) if [ -z "$pkg_name" ]; then
@@ -258,10 +267,15 @@ package_install_internal_rpm () { for pkg in ${package_attemptonly} ; do echo "Processing $pkg..." archvar=base_archs - ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,") - if [ "${ml_pkg}" != "${pkg}" ]; then - archvar=ml_archs - fi + ml_prefix=`echo ${pkg} | cut -d'-' -f1` + ml_pkg=$pkg + for i in ${MULTILIB_PREFIX_LIST} ; do + if [ ${ml_prefix} == ${i} ]; then + ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,") + archvar=ml_archs + break + fi + done pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg}) if [ -z "$pkg_name" ]; then
meta/classes/rootfs_rpm.bbclass +4 -0
@@ -58,6 +58,7 @@ fakeroot rootfs_rpm_do_rootfs () { export INSTALL_PLATFORM_RPM="${TARGET_ARCH}" export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}" export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}" + export INSTALL_PACKAGES_MULTILIB_RPM="${MULTILIB_PACKAGE_INSTALL}" export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}" export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}" export INSTALL_PROVIDENAME_RPM=""
@@ -210,6 +211,7 @@ python () { bb.data.setVar('RPM_POSTPROCESS_COMMANDS', '', d) ml_package_archs = "" + ml_prefix_list = "" multilibs = d.getVar('MULTILIBS', True) or "" for ext in multilibs.split(): eext = ext.split(':')
@@ -221,6 +223,8 @@ python () { localdata.setVar("MACHINE_ARCH", eext[1] + "_" + localdata.getVar("MACHINE_ARCH", False)) package_archs = localdata.getVar("PACKAGE_ARCHS", True) or "" ml_package_archs += " " + package_archs + ml_prefix_list += " " + eext[1] #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides)) bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d) + bb.data.setVar('MULTILIB_PREFIX_LIST', ml_prefix_list, d) }
Signature · 1 line
1.7.3.4
PATCH 3/7 Fix RPM dependencies
In reply to Mark HatleWhen packaging for RPM, dependencies were not being properly added to the packages. Only the "providing" dependencies were added due to a typo.
--- meta/classes/package_rpm.bbclass | 2 +- 1 files changed, 1 insertions(+), 1 deletions(-)
meta/classes/package_rpm.bbclass +1 -1
@@ -840,7 +840,7 @@ python do_package_rpm () { os.chmod(outdepends, 0755) # Poky / RPM Provides - outprovides = workdir + "/" + srcname + ".requires" + outprovides = workdir + "/" + srcname + ".provides" try: from __builtin__ import file
Signature · 1 line
1.7.3.4
PATCH 4/7 Add a run-time dependency that eglibc support GNU_HASH
In reply to Mark HatleRPM checks to see if binaries require GNU_HASH, if they do it adds an automatic dependency of "rtld(GNU_HASH)". We need to satisfy this dependency, and we do it by providing rtld(GNU_HASH) in the package(s) that provide a runtime-linker.
--- meta/recipes-core/eglibc/eglibc-package.inc | 3 ++- meta/recipes-core/eglibc/eglibc_2.12.bb | 2 +- meta/recipes-core/eglibc/eglibc_2.13.bb | 2 +- 3 files changed, 4 insertions(+), 3 deletions(-)
meta/recipes-core/eglibc/eglibc-package.inc +2 -1
@@ -22,7 +22,8 @@ PKGSUFFIX_virtclass-nativesdk = "-nativesdk" PACKAGES = "${PN}-dbg ${PN} catchsegv${PKGSUFFIX} sln${PKGSUFFIX} nscd${PKGSUFFIX} ldd${PKGSUFFIX} ${PN}-utils eglibc-extra-nss${PKGSUFFIX} eglibc-thread-db${PKGSUFFIX} ${PN}-pic ${PN}-dev ${PN}-doc libcidn${PKGSUFFIX} libmemusage${PKGSUFFIX} libsegfault${PKGSUFFIX} ${PN}-pcprofile libsotruss${PKGSUFFIX}" -RPROVIDES_${PN} = "glibc${PKGSUFFIX}" +# The ld.so in this eglibc supports the GNU_HASH +RPROVIDES_${PN} = "glibc${PKGSUFFIX} rtld(GNU_HASH)" RPROVIDES_${PN}-utils = "glibc${PKGSUFFIX}-utils" RPROVIDES_${PN}-pic = "glibc${PKGSUFFIX}-pic" RPROVIDES_${PN}-dev = "glibc${PKGSUFFIX}-dev"
meta/recipes-core/eglibc/eglibc_2.12.bb +1 -1
@@ -1,7 +1,7 @@ require eglibc.inc DEPENDS += "gperf-native" -PR = "r23" +PR = "r24" SRCREV = "14158"
meta/recipes-core/eglibc/eglibc_2.13.bb +1 -1
@@ -3,7 +3,7 @@ require eglibc.inc SRCREV = "14157" DEPENDS += "gperf-native" -PR = "r14" +PR = "r15" PR_append = "+svnr${SRCPV}" EGLIBC_BRANCH="eglibc-2_13"
Signature · 1 line
1.7.3.4
PATCH 5/7 Update python dependencies to be simply to "python"
In reply to Mark HatlePreviously python dependencies were of the format "python(abi) = ..." This format is not yet supportable within OE, so revert to a form we know we can handle.
Introduce a change to package.bbclass that ensures it will causes the sstate-cache's "package" to invalidate. Since pythondeps changed, the output of rpmdeps changes, which causes the per-file dependency information to change.... thus we need to invalidate the cache!
--- meta/classes/package.bbclass | 6 ++++++ meta/recipes-devtools/rpm/rpm/pythondeps.sh | 16 ++++++++++++++++ meta/recipes-devtools/rpm/rpm_5.4.0.bb | 4 +++- 3 files changed, 25 insertions(+), 1 deletions(-) create mode 100755 meta/recipes-devtools/rpm/rpm/pythondeps.sh
meta/classes/package.bbclass +6 -0
@@ -1,3 +1,9 @@ +# Change the following version to cause sstate to invalidate the package +# cache. This is useful if an item this class depends on changes in a +# way that the output of this class changes. rpmdeps is a good example +# as any change to rpmdeps requires this to be rerun. +PACKAGE_BBCLASS_VERSION = "r1" + # # Packaging process #
meta/recipes-devtools/rpm/rpm/pythondeps.sh new file +16 -0
@@ -0,0 +1,16 @@ +#!/bin/sh + +[ $# -ge 1 ] || { + cat > /dev/null + exit 0 +} + +case $1 in +-R|--requires) + shift + grep "/usr/\(lib[^/]*\|share\)/python[^/]*/" >/dev/null && echo "python" + exit 0 + ;; +esac + +exit 0
meta/recipes-devtools/rpm/rpm_5.4.0.bb +3 -1
@@ -45,7 +45,7 @@ LIC_FILES_CHKSUM = "file://COPYING.LIB;md5=2d5025d4aa3495befef8f17206a5b0a1" DEPENDS = "bzip2 zlib db openssl elfutils expat libpcre attr acl popt ${extrarpmdeps}" extrarpmdeps = "python perl" extrarpmdeps_virtclass-native = "" -PR = "r20" +PR = "r21" # rpm2cpio is a shell script, which is part of the rpm src.rpm. It is needed # in order to extract the distribution SRPM into a format we can extract...
@@ -63,6 +63,7 @@ SRC_URI = "http://www.rpm5.org/files/rpm/rpm-5.4/rpm-5.4.0-0.20101229.src.rpm;ex file://rpm-fileclass.patch \ file://rpm-canonarch.patch \ file://rpm-no-loopmsg.patch \ + file://pythondeps.sh \ " # file://rpm-autoconf.patch \
@@ -352,6 +353,7 @@ do_install_append() { # Enable Debian style arbitrary tags... sed -i -e 's,%_arbitrary_tags[^_].*,%_arbitrary_tags %{_arbitrary_tags_debian},' ${D}/${libdir}/rpm/macros + install -m 0755 ${WORKDIR}/pythondeps.sh ${D}/${libdir}/rpm/pythondeps.sh install -m 0755 ${WORKDIR}/perfile_rpmdeps.sh ${D}/${libdir}/rpm/perfile_rpmdeps.sh # Remove unpackaged files (based on list in rpm.spec)
Signature · 1 line
1.7.3.4
PATCH 6/7 busybox: Enhance to add dynamic per-file provides
In reply to Mark HatleWhen using the RPM package backend, we need a full list of per-file provides (and requires). The busybox package provides a number of command line utilities, such as /usr/bin/env. However, because the utilities are created at post install time via scripting the provide of each of the links was never made. So any programs, such as python, that require /usr/bin/env were unable to resolve the dependency and failed.
This change only affects packaging backends that use per-file dependency data. Currently RPM is the only packaging backend with this ability.
--- meta/recipes-core/busybox/busybox.inc | 32 +++++++++++++++++++++++++++ meta/recipes-core/busybox/busybox_1.18.5.bb | 2 +- 2 files changed, 33 insertions(+), 1 deletions(-)
meta/recipes-core/busybox/busybox.inc +32 -0
@@ -205,6 +205,38 @@ do_install () { install -m 0644 ${S}/busybox.links ${D}${sysconfdir} } +python package_do_filedeps_append () { + # We need to load the full set of busybox provides from the /etc/busybox.links + # The pkg_postinst_ is what creates the actual links + + pkg = d.getVar('PN', True) + f_busybox = "/bin/busybox" + f_busybox_links = "/etc/busybox.links" + + requires_files = [] + provides_files = [] + + # Load/backup original set + filerprovides = d.getVar('FILERPROVIDES_%s_%s' % (f_busybox, pkg), True) or "" + + dep_pipe = os.popen('sed -e "s,^,Provides: ," %s/%s%s' % (pkgdest, pkg, f_busybox_links)) + + process_deps(dep_pipe, pkg, "%s/%s%s" % (pkgdest, pkg, f_busybox), provides_files, requires_files) + + # Add the new set + filerprovides += d.getVar('FILERPROVIDES_%s_%s' % (f_busybox, pkg), True) or "" + + # Make sure there is an entry for this item in the FILERPROVIDESFLIST... + filerprovidesflist = (d.getVar('FILERPROVIDESFLIST_%s' % pkg, True) or "").split() + for file in provides_files: + if file not in filerprovidesflist: + filerprovidesflist.append(file) + d.setVar('FILERPROVIDESFLIST_%s' % pkg, " ".join(filerprovidesflist)) + + # Store the new provides + d.setVar('FILERPROVIDES_%s_%s' % (f_busybox, pkg), filerprovides) +} + pkg_postinst_${PN} () { # If we are not making an image we create links for the utilities that doesn't exist # so the update-alternatives script will get the utilities it needs
meta/recipes-core/busybox/busybox_1.18.5.bb +1 -1
@@ -1,5 +1,5 @@ require busybox.inc -PR = "r0" +PR = "r1" SRC_URI = "http://www.busybox.net/downloads/busybox-${PV}.tar.bz2;name=tarball \ file://udhcpscript.patch \
Signature · 1 line
1.7.3.4
PATCH 7/7 multilib_global.bbclass: Fix non-multilib package provides [v2]
In reply to Mark HatleIn non-multilib packages, configured in a multilib configuration we need to adjust the system provides and rprovides to include the virtual multilib variant.
This resolves a problem introduced in the 329d864f9bbf94ad3aae8df43d63fe10e4237e4f commit. Where "allarch" packages were suddenly providing all variants of an object.
--- meta/classes/multilib_global.bbclass | 27 ++++++++++++++++++++++----- 1 files changed, 22 insertions(+), 5 deletions(-)
meta/classes/multilib_global.bbclass +22 -5
@@ -6,16 +6,33 @@ python multilib_virtclass_handler_global () { if isinstance(e, bb.event.RecipeParsed) and not variant: if bb.data.inherits_class('kernel', e.data) or bb.data.inherits_class('module-base', e.data) or bb.data.inherits_class('allarch', e.data): - origprovs = provs = e.data.getVar("PROVIDES", True) - rprovs = e.data.getVar("RPROVIDES", True) variants = (e.data.getVar("MULTILIB_VARIANTS", True) or "").split() + + # Process PROVIDES + origprovs = provs = e.data.getVar("PROVIDES", True) or "" for variant in variants: provs = provs + " " + multilib_map_variable("PROVIDES", variant, e.data) - for pkg in e.data.getVar("PACKAGES", True).split(): - rprovs = rprovs + " " + variant + "-" + pkg + # Reset to original value so next time around multilib_map_variable works properly e.data.setVar("PROVIDES", origprovs) e.data.setVar("PROVIDES", provs) + + # Process RPROVIDES + origrprovs = rprovs = e.data.getVar("RPROVIDES", True) or "" + for variant in variants: + rprovs = rprovs + " " + multilib_map_variable("RPROVIDES", variant, e.data) + # Reset to original value so next time around multilib_map_variable works properly + e.data.setVar("RPROVIDES", origrprovs) e.data.setVar("RPROVIDES", rprovs) + + # Process RPROVIDES_${PN}... + for pkg in (e.data.getVar("PACKAGES", True) or "").split(): + origrprovs = rprovs = e.data.getVar("RPROVIDES_%s" % pkg, True) or "" + for variant in variants: + rprovs = rprovs + " " + multilib_map_variable("RPROVIDES_%s" % pkg, variant, e.data) + rprovs = rprovs + " " + variant + "-" + pkg + # Reset to original value so next time around multilib_map_variable works properly + e.data.setVar("RPROVIDES_%s" % pkg, origrprovs) + e.data.setVar("RPROVIDES_%s" % pkg, rprovs) } addhandler multilib_virtclass_handler_global
@@ -35,7 +52,7 @@ def multilib_extend_name(variant, name): def multilib_map_variable(varname, variant, d): var = d.getVar(varname, True) if not var: - return + return "" var = var.split() newvar = [] for v in var:
Signature · 1 line
1.7.3.4