rpm: multilib related fixes
v2 cover rpm: multilib related fixes
Hi Richard and Mark,
This is the second version of RFC to fix multilib related issue to RPM, includes:
1) Add architecture information in RPM spec file to fix bug 1457.
2) Define override for MACHINE to avoid confliction with normal recipe in deploy folder.
3) Add MULTILIB_IMAGE_INSTALL in the package installation list.
We have another patch related with local.conf.sample.extended, which will be sent to [email protected].
Change from v1:
1) we define MACHINE override for multilib instead of add MLPREFIX in front of MACHINE_ARCH.
The following changes since commit 8df322abfd3fa8c460fd73c83c1ef0b5e419dc11:
core-image-minimal: Add missing POKY_EXTRA_IMAGE variable (2011-09-14 01:27:08 +0100)
are available in the git repository at:
git://git.pokylinux.org/poky-contrib dxu4/ml2
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/ml2
Dongxiao Xu (3):
package_rpm: add architecture info in rpm spec file
rootfs_rpm: Use specific MACHINE_ARCH for multilib recipes
multilib: install MULTILIB_IMAGE_INSTALL
meta/classes/multilib.bbclass | 1 + meta/classes/package_rpm.bbclass | 78 +++++++++++++++++++++++++++++++------- meta/classes/rootfs_rpm.bbclass | 7 +++ 3 files changed, 72 insertions(+), 14 deletions(-)
PATCH v2 1/3 package_rpm: add architecture info in rpm spec file
For supporting multilib, architecture information is needed in package require/provide/suggest/recommend fields.
Use DEFAULTTUNE value as the postfix.
For "all" arch recipe, it requires "all" arch recipe with no postfix, but provides all possible multilib archs.
For example, qemu-config:
Requires: rsync
Requires: update-rc.d
Requires: task-core-nfs-server
Requires: distcc
Requires: oprofileui-server
Requires: dbus-x11
Requires: bash
Provides: qemu-config.x86
Provides: qemu-config.x86-64
For other recipe like zlib:
Requires: libc6.x86-64 >= 2.13
Provides: zlib.x86-64
[YOCTO #1457]
--- meta/classes/multilib.bbclass | 1 + meta/classes/package_rpm.bbclass | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 0 deletions(-)
meta/classes/multilib.bbclass +1 -0
@@ -23,6 +23,7 @@ python multilib_virtclass_handler () { e.data.setVar("PN", variant + "-" + e.data.getVar("PN", False)) e.data.setVar("SHLIBSDIR_virtclass-multilib-" + variant ,e.data.getVar("SHLIBSDIR", False) + "/" + variant) e.data.setVar("TARGET_VENDOR_virtclass-multilib-" + variant, e.data.getVar("TARGET_VENDOR", False) + "ml" + variant) + e.data.setVar("SAVED_DEFAULTTUNE", e.data.getVar("DEFAULTTUNE", True)) e.data.setVar("OVERRIDES", e.data.getVar("OVERRIDES", False) + override) }
meta/classes/package_rpm.bbclass +36 -0
@@ -350,6 +350,7 @@ package_install_internal_rpm () { python write_specfile () { import textwrap import oe.packagedata + import re # We need a simple way to remove the MLPREFIX from the package name, # and dependency information...
@@ -498,6 +499,8 @@ python write_specfile () { splitname = strip_multilib(pkgname, d) + defaulttune = bb.data.getVar('DEFAULTTUNE', localdata, True) + splitsummary = (bb.data.getVar('SUMMARY', localdata, True) or bb.data.getVar('DESCRIPTION', localdata, True) or ".") splitversion = (bb.data.getVar('PKGV', localdata, True) or "").replace('-', '+') splitrelease = (bb.data.getVar('PKGR', localdata, True) or "")
@@ -528,6 +531,39 @@ python write_specfile () { if pkg == d.getVar("PN", True): splitrprovides = splitrprovides + " " + (d.getVar('ALTERNATIVE_LINK', True) or '') + " " + (d.getVar('ALTERNATIVE_LINKS', True) or '') + package_arch = bb.data.getVar('PACKAGE_ARCH', localdata, True) + + splitrprovides = splitrprovides + " " + splitname + "." + defaulttune + if package_arch != "all": + pattern = '\([^()]*\)' + prog = re.compile(pattern) + + str_list = [splitrdepends, splitrrecommends, splitrsuggests] + for e in range(len(str_list)): + brackets = prog.findall(str_list[e]) + for i in range(len(brackets)): + str_list[e] = str_list[e].replace(brackets[i], "#BRACKETS"+str(i)+"#") + tmp = "" + for i in str_list[e].split(): + if i.startswith("#BRACKETS"): + tmp += " " + str(i) + continue + tmp += " " + str(i) + "." + defaulttune + str_list[e] = tmp + for i in range(len(brackets)): + str_list[e] = str_list[e].replace("#BRACKETS"+str(i)+"#", brackets[i]) + + [splitrdepends, splitrrecommends, splitrsuggests] = str_list + else: + variants = (bb.data.getVar("MULTILIB_VARIANTS", localdata, True) or "").split() + for variant in variants: + tune = bb.data.getVar("DEFAULTTUNE_virtclass-multilib-" + variant, localdata, True) or "" + if tune: + splitrprovides = splitrprovides + " " + splitname + "." + tune + tune = bb.data.getVar("SAVED_DEFAULTTUNE", localdata, True) or "" + if tune: + splitrprovides = splitrprovides + " " + splitname + "." + tune + # Gather special src/first package data if srcname == splitname: srcrdepends = splitrdepends
Signature · 1 line
1.7.1
Re: [PATCH 1/3] package_rpm: add architecture info in rpm spec file
I still don't think this is what we want. Let me look into the library issues that you mentioned in the other thread and I'll see if we can deal with it that way.
(Note, we may need to use something like this in non-RPM cases.. but I'm not sure about that either.)
--Mark
Quoted 106 lines · Dongxiao Xu
For supporting multilib, architecture information is needed in package require/provide/suggest/recommend fields.
Use DEFAULTTUNE value as the postfix.
For "all" arch recipe, it requires "all" arch recipe with no postfix, but provides all possible multilib archs.
For example, qemu-config:
Requires: rsync
Requires: update-rc.d
Requires: task-core-nfs-server
Requires: distcc
Requires: oprofileui-server
Requires: dbus-x11
Requires: bash
Provides: qemu-config.x86
Provides: qemu-config.x86-64For other recipe like zlib:
Requires: libc6.x86-64 >= 2.13
Provides: zlib.x86-64[YOCTO #1457]
Signed-off-by Dongxiao Xu <[email protected]>--- meta/classes/multilib.bbclass | 1 + meta/classes/package_rpm.bbclass | 36 ++++++++++++++++++++++++++++++++++++ 2 files changed, 37 insertions(+), 0 deletions(-)meta/classes/multilib.bbclass +1
-0@@ -23,6 +23,7 @@ python multilib_virtclass_handler () { e.data.setVar("PN", variant + "-" + e.data.getVar("PN", False)) e.data.setVar("SHLIBSDIR_virtclass-multilib-" + variant ,e.data.getVar("SHLIBSDIR", False) + "/" + variant) e.data.setVar("TARGET_VENDOR_virtclass-multilib-" + variant, e.data.getVar("TARGET_VENDOR", False) + "ml" + variant) + e.data.setVar("SAVED_DEFAULTTUNE", e.data.getVar("DEFAULTTUNE", True)) e.data.setVar("OVERRIDES", e.data.getVar("OVERRIDES", False) + override) }meta/classes/package_rpm.bbclass +36
-0@@ -350,6 +350,7 @@ package_install_internal_rpm () { python write_specfile () { import textwrap import oe.packagedata + import re # We need a simple way to remove the MLPREFIX from the package name, # and dependency information...@@ -498,6 +499,8 @@ python write_specfile () { splitname = strip_multilib(pkgname, d) + defaulttune = bb.data.getVar('DEFAULTTUNE', localdata, True) + splitsummary = (bb.data.getVar('SUMMARY', localdata, True) or bb.data.getVar('DESCRIPTION', localdata, True) or ".") splitversion = (bb.data.getVar('PKGV', localdata, True) or "").replace('-', '+') splitrelease = (bb.data.getVar('PKGR', localdata, True) or "")@@ -528,6 +531,39 @@ python write_specfile () { if pkg == d.getVar("PN", True): splitrprovides = splitrprovides + " " + (d.getVar('ALTERNATIVE_LINK', True) or '') + " " + (d.getVar('ALTERNATIVE_LINKS', True) or '') + package_arch = bb.data.getVar('PACKAGE_ARCH', localdata, True) + + splitrprovides = splitrprovides + " " + splitname + "." + defaulttune + if package_arch != "all": + pattern = '\([^()]*\)' + prog = re.compile(pattern) + + str_list = [splitrdepends, splitrrecommends, splitrsuggests] + for e in range(len(str_list)): + brackets = prog.findall(str_list[e]) + for i in range(len(brackets)): + str_list[e] = str_list[e].replace(brackets[i], "#BRACKETS"+str(i)+"#") + tmp = "" + for i in str_list[e].split(): + if i.startswith("#BRACKETS"): + tmp += " " + str(i) + continue + tmp += " " + str(i) + "." + defaulttune + str_list[e] = tmp + for i in range(len(brackets)): + str_list[e] = str_list[e].replace("#BRACKETS"+str(i)+"#", brackets[i]) + + [splitrdepends, splitrrecommends, splitrsuggests] = str_list + else: + variants = (bb.data.getVar("MULTILIB_VARIANTS", localdata, True) or "").split() + for variant in variants: + tune = bb.data.getVar("DEFAULTTUNE_virtclass-multilib-" + variant, localdata, True) or "" + if tune: + splitrprovides = splitrprovides + " " + splitname + "." + tune + tune = bb.data.getVar("SAVED_DEFAULTTUNE", localdata, True) or "" + if tune: + splitrprovides = splitrprovides + " " + splitname + "." + tune + # Gather special src/first package data if srcname == splitname: srcrdepends = splitrdepends
PATCH v2 2/3 rootfs_rpm: Use specific MACHINE_ARCH for multilib recipes
In reply to Dongxiao XuCurrently MACHINE_ARCH deploy folder is unique in multilib system, thus a lib32 version of rpm package will override a normal rpm package if its PACKAGE_ARCH is ${MACHINE_ARCH}.
Take netbase as an example, which the PACKAGE_ARCH = MACHINE_ARCH. Both the normal version of netbase package and the lib32 version are named as "netbase-4.45-r1.qemux86_64.rpm" putting in tmp/deploy/rpm/qemux86-64 directory, so we need to differentiate them.
Here we define spedific MACHINE_virtclass-multilib-lib(xx) to override the default MACHINE value, thus got different MACHINE_ARCH to fix this issue.
--- meta/classes/rootfs_rpm.bbclass | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)
meta/classes/rootfs_rpm.bbclass +3 -0
@@ -218,6 +218,9 @@ python () { default_tune = localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False) if default_tune: localdata.setVar("DEFAULTTUNE", default_tune) + machine = localdata.getVar("MACHINE_virtclass-multilib-" + eext[1], False) + if machine: + localdata.setVar("MACHINE", machine) ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or "" #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides)) bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d)
Signature · 1 line
1.7.1
Re: [PATCH 2/3] rootfs_rpm: Use specific MACHINE_ARCH for multilib recipes
This looks fine. The do_archgen() function within the libzypp_git.bb will need to be updated to look for the multilib machine type as well.
--Mark
Quoted 32 lines · Dongxiao Xu
Currently
MACHINE_ARCHdeploy folder is unique in multilib system, thus a lib32 version of rpm package will override a normal rpm package if itsPACKAGE_ARCHis ${MACHINE_ARCH}.Take netbase as an example, which the
PACKAGE_ARCH=MACHINE_ARCH. Both the normal version of netbase package and the lib32 version are named as "netbase-4.45-r1.qemux86_64.rpm" putting in tmp/deploy/rpm/qemux86-64 directory, so we need to differentiate them.Here we define spedific MACHINE_virtclass-multilib-lib(xx) to override the default MACHINE value, thus got different
MACHINE_ARCHto fix this issue.Signed-off-by Dongxiao Xu <[email protected]>--- meta/classes/rootfs_rpm.bbclass | 3 +++ 1 files changed, 3 insertions(+), 0 deletions(-)meta/classes/rootfs_rpm.bbclass +3
-0@@ -218,6 +218,9 @@ python () { default_tune = localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False) if default_tune: localdata.setVar("DEFAULTTUNE", default_tune) + machine = localdata.getVar("MACHINE_virtclass-multilib-" + eext[1], False) + if machine: + localdata.setVar("MACHINE", machine) ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or "" #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides)) bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d)
PATCH v2 3/3 multilib: install MULTILIB_IMAGE_INSTALL
In reply to Dongxiao XuIf user set MULTILIB_IMAGE_INSTALL, we need to install those multitlib packages into the final image.
Also fix the logic in handling multilib prefix. For certain case like a normal image contains several multilib libraries, the image recipe isn't extended with MLPREFIX, therefore we need to enumerate the possible multilib prefixes and compare them with package prefixes.
--- meta/classes/package_rpm.bbclass | 42 +++++++++++++++++++++++++------------ meta/classes/rootfs_rpm.bbclass | 4 +++ 2 files changed, 32 insertions(+), 14 deletions(-)
meta/classes/package_rpm.bbclass +28 -14
@@ -166,7 +166,7 @@ package_install_internal_rpm () { local platform="${INSTALL_PLATFORM_RPM}" local platform_extra="${INSTALL_PLATFORM_EXTRA_RPM}" local confbase="${INSTALL_CONFBASE_RPM}" - local package_to_install="${INSTALL_PACKAGES_NORMAL_RPM}" + local package_to_install="${INSTALL_PACKAGES_NORMAL_RPM} ${INSTALL_PACKAGES_MULTILIB_RPM}" local package_attemptonly="${INSTALL_PACKAGES_ATTEMPTONLY_RPM}" local package_linguas="${INSTALL_PACKAGES_LINGUAS_RPM}" local providename="${INSTALL_PROVIDENAME_RPM}"
@@ -210,10 +210,15 @@ package_install_internal_rpm () { echo "Processing $pkg..." archvar=base_archs - ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,") - if [ "${ml_pkg}" != "${pkg}" ]; then - archvar=ml_archs - fi + ml_prefix=`echo ${pkg} | cut -d'-' -f1` + ml_pkg=$pkg + for i in ${MULTILIB_PREFIX_LIST} ; do + if [ ${ml_prefix} == ${i} ]; then + ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,") + archvar=ml_archs + break + fi + done pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg}) if [ -z "$pkg_name" ]; then
@@ -224,16 +229,20 @@ package_install_internal_rpm () { done fi fi - if [ ! -z "${package_to_install}" ]; then for pkg in ${package_to_install} ; do echo "Processing $pkg..." archvar=base_archs - ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,") - if [ "${ml_pkg}" != "${pkg}" ]; then - archvar=ml_archs - fi + ml_prefix=`echo ${pkg} | cut -d'-' -f1` + ml_pkg=$pkg + for i in ${MULTILIB_PREFIX_LIST} ; do + if [ ${ml_prefix} == ${i} ]; then + ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,") + archvar=ml_archs + break + fi + done pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg}) if [ -z "$pkg_name" ]; then
@@ -258,10 +267,15 @@ package_install_internal_rpm () { for pkg in ${package_attemptonly} ; do echo "Processing $pkg..." archvar=base_archs - ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,") - if [ "${ml_pkg}" != "${pkg}" ]; then - archvar=ml_archs - fi + ml_prefix=`echo ${pkg} | cut -d'-' -f1` + ml_pkg=$pkg + for i in ${MULTILIB_PREFIX_LIST} ; do + if [ ${ml_prefix} == ${i} ]; then + ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,") + archvar=ml_archs + break + fi + done pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg}) if [ -z "$pkg_name" ]; then
meta/classes/rootfs_rpm.bbclass +4 -0
@@ -58,6 +58,7 @@ fakeroot rootfs_rpm_do_rootfs () { export INSTALL_PLATFORM_RPM="${TARGET_ARCH}" export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}" export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}" + export INSTALL_PACKAGES_MULTILIB_RPM="${MULTILIB_PACKAGE_INSTALL}" export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}" export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}" export INSTALL_PROVIDENAME_RPM=""
@@ -210,6 +211,7 @@ python () { bb.data.setVar('RPM_POSTPROCESS_COMMANDS', '', d) ml_package_archs = "" + ml_prefix_list = "" multilibs = d.getVar('MULTILIBS', True) or "" for ext in multilibs.split(): eext = ext.split(':')
@@ -222,6 +224,8 @@ python () { if machine: localdata.setVar("MACHINE", machine) ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or "" + ml_prefix_list += " " + eext[1] #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides)) bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d) + bb.data.setVar('MULTILIB_PREFIX_LIST', ml_prefix_list, d) }
Signature · 1 line
1.7.1