Skip to contentSkip to conversation

rpm: multilib related fixes

RFC PATCH In review Dongxiao Xu 2 participants 6 messages Sep 14, 2011

lore
Share & tools
lore
#0 lore raw

v2 cover rpm: multilib related fixes

Hi Richard and Mark,

This is the second version of RFC to fix multilib related issue to RPM, includes:

1) Add architecture information in RPM spec file to fix bug 1457.
2) Define override for MACHINE to avoid confliction with normal recipe in deploy folder.
3) Add MULTILIB_IMAGE_INSTALL in the package installation list.

We have another patch related with local.conf.sample.extended, which will be sent to [email protected].

Change from v1:
1) we define MACHINE override for multilib instead of add MLPREFIX in front of MACHINE_ARCH.

The following changes since commit 8df322abfd3fa8c460fd73c83c1ef0b5e419dc11:

core-image-minimal: Add missing POKY_EXTRA_IMAGE variable (2011-09-14 01:27:08 +0100)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib dxu4/ml2
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=dxu4/ml2

Dongxiao Xu (3):
package_rpm: add architecture info in rpm spec file
rootfs_rpm: Use specific MACHINE_ARCH for multilib recipes
multilib: install MULTILIB_IMAGE_INSTALL

 meta/classes/multilib.bbclass    |    1 +
 meta/classes/package_rpm.bbclass |   78 +++++++++++++++++++++++++++++++-------
 meta/classes/rootfs_rpm.bbclass  |    7 +++
 3 files changed, 72 insertions(+), 14 deletions(-)
#1 lore raw

PATCH v2 1/3 package_rpm: add architecture info in rpm spec file

For supporting multilib, architecture information is needed in package require/provide/suggest/recommend fields.

Use DEFAULTTUNE value as the postfix.

For "all" arch recipe, it requires "all" arch recipe with no postfix, but provides all possible multilib archs.

For example, qemu-config:

Requires: rsync
Requires: update-rc.d
Requires: task-core-nfs-server
Requires: distcc
Requires: oprofileui-server
Requires: dbus-x11
Requires: bash
Provides: qemu-config.x86
Provides: qemu-config.x86-64

For other recipe like zlib:

Requires: libc6.x86-64 >= 2.13
Provides: zlib.x86-64

[YOCTO #1457]

---
 meta/classes/multilib.bbclass    |    1 +
 meta/classes/package_rpm.bbclass |   36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 0 deletions(-)
meta/classes/multilib.bbclass +1 -0
index 583d76b..76c86b2 100644
@@ -23,6 +23,7 @@ python multilib_virtclass_handler () {
     e.data.setVar("PN", variant + "-" + e.data.getVar("PN", False))
     e.data.setVar("SHLIBSDIR_virtclass-multilib-" + variant ,e.data.getVar("SHLIBSDIR", False) + "/" + variant)
     e.data.setVar("TARGET_VENDOR_virtclass-multilib-" + variant, e.data.getVar("TARGET_VENDOR", False) + "ml" + variant)
+    e.data.setVar("SAVED_DEFAULTTUNE", e.data.getVar("DEFAULTTUNE", True))
     e.data.setVar("OVERRIDES", e.data.getVar("OVERRIDES", False) + override)
 }
 
meta/classes/package_rpm.bbclass +36 -0
index 9ef1acd..ea0a079 100644
@@ -350,6 +350,7 @@ package_install_internal_rpm () {
 python write_specfile () {
 	import textwrap
 	import oe.packagedata
+	import re
 
 	# We need a simple way to remove the MLPREFIX from the package name,
 	# and dependency information...
@@ -498,6 +499,8 @@ python write_specfile () {
 
 		splitname    = strip_multilib(pkgname, d)
 
+		defaulttune = bb.data.getVar('DEFAULTTUNE', localdata, True)
+
 		splitsummary = (bb.data.getVar('SUMMARY', localdata, True) or bb.data.getVar('DESCRIPTION', localdata, True) or ".")
 		splitversion = (bb.data.getVar('PKGV', localdata, True) or "").replace('-', '+')
 		splitrelease = (bb.data.getVar('PKGR', localdata, True) or "")
@@ -528,6 +531,39 @@ python write_specfile () {
 		if pkg == d.getVar("PN", True):
 			splitrprovides = splitrprovides + " " + (d.getVar('ALTERNATIVE_LINK', True) or '') + " " + (d.getVar('ALTERNATIVE_LINKS', True) or '')
 
+		package_arch = bb.data.getVar('PACKAGE_ARCH', localdata, True)
+
+		splitrprovides = splitrprovides + " " + splitname + "." + defaulttune
+		if package_arch != "all":
+			pattern = '\([^()]*\)'
+			prog = re.compile(pattern)
+
+			str_list = [splitrdepends, splitrrecommends, splitrsuggests]
+			for e in range(len(str_list)):
+				brackets = prog.findall(str_list[e])
+				for i in range(len(brackets)):
+					str_list[e] = str_list[e].replace(brackets[i], "#BRACKETS"+str(i)+"#")
+				tmp = ""
+				for i in str_list[e].split():
+					if i.startswith("#BRACKETS"):
+						tmp += " " + str(i)
+						continue
+					tmp += " " + str(i) + "." + defaulttune
+				str_list[e] = tmp
+				for i in range(len(brackets)):
+					str_list[e] = str_list[e].replace("#BRACKETS"+str(i)+"#", brackets[i])
+
+			[splitrdepends, splitrrecommends, splitrsuggests] = str_list
+		else:
+			variants = (bb.data.getVar("MULTILIB_VARIANTS", localdata, True) or "").split()
+			for variant in variants:
+				tune = bb.data.getVar("DEFAULTTUNE_virtclass-multilib-" + variant, localdata, True) or ""
+				if tune:
+					splitrprovides = splitrprovides + " " + splitname + "." + tune
+			tune = bb.data.getVar("SAVED_DEFAULTTUNE", localdata, True) or ""
+			if tune:
+				splitrprovides = splitrprovides + " " + splitname + "." + tune
+
 		# Gather special src/first package data
 		if srcname == splitname:
 			srcrdepends    = splitrdepends
Signature · 1 line
1.7.1
#2 lore raw

Re: [PATCH 1/3] package_rpm: add architecture info in rpm spec file

I still don't think this is what we want. Let me look into the library issues that you mentioned in the other thread and I'll see if we can deal with it that way.

(Note, we may need to use something like this in non-RPM cases.. but I'm not sure about that either.)

--Mark

Quoted 106 lines · Dongxiao Xu

For supporting multilib, architecture information is needed in package require/provide/suggest/recommend fields.

Use DEFAULTTUNE value as the postfix.

For "all" arch recipe, it requires "all" arch recipe with no postfix, but provides all possible multilib archs.

For example, qemu-config:

Requires: rsync
Requires: update-rc.d
Requires: task-core-nfs-server
Requires: distcc
Requires: oprofileui-server
Requires: dbus-x11
Requires: bash
Provides: qemu-config.x86
Provides: qemu-config.x86-64

For other recipe like zlib:

Requires: libc6.x86-64 >= 2.13
Provides: zlib.x86-64

[YOCTO #1457]

---
 meta/classes/multilib.bbclass    |    1 +
 meta/classes/package_rpm.bbclass |   36 ++++++++++++++++++++++++++++++++++++
 2 files changed, 37 insertions(+), 0 deletions(-)
meta/classes/multilib.bbclass +1 -0
index 583d76b..76c86b2 100644
@@ -23,6 +23,7 @@ python multilib_virtclass_handler () {
     e.data.setVar("PN", variant + "-" + e.data.getVar("PN", False))
     e.data.setVar("SHLIBSDIR_virtclass-multilib-" + variant ,e.data.getVar("SHLIBSDIR", False) + "/" + variant)
     e.data.setVar("TARGET_VENDOR_virtclass-multilib-" + variant, e.data.getVar("TARGET_VENDOR", False) + "ml" + variant)
+    e.data.setVar("SAVED_DEFAULTTUNE", e.data.getVar("DEFAULTTUNE", True))
     e.data.setVar("OVERRIDES", e.data.getVar("OVERRIDES", False) + override)
 }
 
meta/classes/package_rpm.bbclass +36 -0
index 9ef1acd..ea0a079 100644
@@ -350,6 +350,7 @@ package_install_internal_rpm () {
 python write_specfile () {
 	import textwrap
 	import oe.packagedata
+	import re
 
 	# We need a simple way to remove the MLPREFIX from the package name,
 	# and dependency information...
@@ -498,6 +499,8 @@ python write_specfile () {
 
 		splitname    = strip_multilib(pkgname, d)
 
+		defaulttune = bb.data.getVar('DEFAULTTUNE', localdata, True)
+
 		splitsummary = (bb.data.getVar('SUMMARY', localdata, True) or bb.data.getVar('DESCRIPTION', localdata, True) or ".")
 		splitversion = (bb.data.getVar('PKGV', localdata, True) or "").replace('-', '+')
 		splitrelease = (bb.data.getVar('PKGR', localdata, True) or "")
@@ -528,6 +531,39 @@ python write_specfile () {
 		if pkg == d.getVar("PN", True):
 			splitrprovides = splitrprovides + " " + (d.getVar('ALTERNATIVE_LINK', True) or '') + " " + (d.getVar('ALTERNATIVE_LINKS', True) or '')
 
+		package_arch = bb.data.getVar('PACKAGE_ARCH', localdata, True)
+
+		splitrprovides = splitrprovides + " " + splitname + "." + defaulttune
+		if package_arch != "all":
+			pattern = '\([^()]*\)'
+			prog = re.compile(pattern)
+
+			str_list = [splitrdepends, splitrrecommends, splitrsuggests]
+			for e in range(len(str_list)):
+				brackets = prog.findall(str_list[e])
+				for i in range(len(brackets)):
+					str_list[e] = str_list[e].replace(brackets[i], "#BRACKETS"+str(i)+"#")
+				tmp = ""
+				for i in str_list[e].split():
+					if i.startswith("#BRACKETS"):
+						tmp += " " + str(i)
+						continue
+					tmp += " " + str(i) + "." + defaulttune
+				str_list[e] = tmp
+				for i in range(len(brackets)):
+					str_list[e] = str_list[e].replace("#BRACKETS"+str(i)+"#", brackets[i])
+
+			[splitrdepends, splitrrecommends, splitrsuggests] = str_list
+		else:
+			variants = (bb.data.getVar("MULTILIB_VARIANTS", localdata, True) or "").split()
+			for variant in variants:
+				tune = bb.data.getVar("DEFAULTTUNE_virtclass-multilib-" + variant, localdata, True) or ""
+				if tune:
+					splitrprovides = splitrprovides + " " + splitname + "." + tune
+			tune = bb.data.getVar("SAVED_DEFAULTTUNE", localdata, True) or ""
+			if tune:
+				splitrprovides = splitrprovides + " " + splitname + "." + tune
+
 		# Gather special src/first package data
 		if srcname == splitname:
 			srcrdepends    = splitrdepends
#3 lore raw

PATCH v2 2/3 rootfs_rpm: Use specific MACHINE_ARCH for multilib recipes

In reply to Dongxiao Xu

Currently MACHINE_ARCH deploy folder is unique in multilib system, thus a lib32 version of rpm package will override a normal rpm package if its PACKAGE_ARCH is ${MACHINE_ARCH}.

Take netbase as an example, which the PACKAGE_ARCH = MACHINE_ARCH. Both the normal version of netbase package and the lib32 version are named as "netbase-4.45-r1.qemux86_64.rpm" putting in tmp/deploy/rpm/qemux86-64 directory, so we need to differentiate them.

Here we define spedific MACHINE_virtclass-multilib-lib(xx) to override the default MACHINE value, thus got different MACHINE_ARCH to fix this issue.

---
 meta/classes/rootfs_rpm.bbclass |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
meta/classes/rootfs_rpm.bbclass +3 -0
index 135ca75..56c1a85 100644
@@ -218,6 +218,9 @@ python () {
             default_tune = localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False)
             if default_tune:
                 localdata.setVar("DEFAULTTUNE", default_tune)
+            machine = localdata.getVar("MACHINE_virtclass-multilib-" + eext[1], False)
+            if machine:
+                localdata.setVar("MACHINE", machine)
             ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or ""
             #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
     bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d)
Signature · 1 line
1.7.1
#4 lore raw

Re: [PATCH 2/3] rootfs_rpm: Use specific MACHINE_ARCH for multilib recipes

This looks fine. The do_archgen() function within the libzypp_git.bb will need to be updated to look for the multilib machine type as well.

--Mark

Quoted 32 lines · Dongxiao Xu

Currently MACHINE_ARCH deploy folder is unique in multilib system, thus a lib32 version of rpm package will override a normal rpm package if its PACKAGE_ARCH is ${MACHINE_ARCH}.

Take netbase as an example, which the PACKAGE_ARCH = MACHINE_ARCH. Both the normal version of netbase package and the lib32 version are named as "netbase-4.45-r1.qemux86_64.rpm" putting in tmp/deploy/rpm/qemux86-64 directory, so we need to differentiate them.

Here we define spedific MACHINE_virtclass-multilib-lib(xx) to override the default MACHINE value, thus got different MACHINE_ARCH to fix this issue.

---
 meta/classes/rootfs_rpm.bbclass |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)
meta/classes/rootfs_rpm.bbclass +3 -0
index 135ca75..56c1a85 100644
@@ -218,6 +218,9 @@ python () {
             default_tune = localdata.getVar("DEFAULTTUNE_virtclass-multilib-" + eext[1], False)
             if default_tune:
                 localdata.setVar("DEFAULTTUNE", default_tune)
+            machine = localdata.getVar("MACHINE_virtclass-multilib-" + eext[1], False)
+            if machine:
+                localdata.setVar("MACHINE", machine)
             ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or ""
             #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
     bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d)
#5 lore raw

PATCH v2 3/3 multilib: install MULTILIB_IMAGE_INSTALL

In reply to Dongxiao Xu

If user set MULTILIB_IMAGE_INSTALL, we need to install those multitlib packages into the final image.

Also fix the logic in handling multilib prefix. For certain case like a normal image contains several multilib libraries, the image recipe isn't extended with MLPREFIX, therefore we need to enumerate the possible multilib prefixes and compare them with package prefixes.

---
 meta/classes/package_rpm.bbclass |   42 +++++++++++++++++++++++++------------
 meta/classes/rootfs_rpm.bbclass  |    4 +++
 2 files changed, 32 insertions(+), 14 deletions(-)
meta/classes/package_rpm.bbclass +28 -14
index ea0a079..5e5fb6c 100644
@@ -166,7 +166,7 @@ package_install_internal_rpm () {
 	local platform="${INSTALL_PLATFORM_RPM}"
 	local platform_extra="${INSTALL_PLATFORM_EXTRA_RPM}"
 	local confbase="${INSTALL_CONFBASE_RPM}"
-	local package_to_install="${INSTALL_PACKAGES_NORMAL_RPM}"
+	local package_to_install="${INSTALL_PACKAGES_NORMAL_RPM} ${INSTALL_PACKAGES_MULTILIB_RPM}"
 	local package_attemptonly="${INSTALL_PACKAGES_ATTEMPTONLY_RPM}"
 	local package_linguas="${INSTALL_PACKAGES_LINGUAS_RPM}"
 	local providename="${INSTALL_PROVIDENAME_RPM}"
@@ -210,10 +210,15 @@ package_install_internal_rpm () {
 				echo "Processing $pkg..."
 
 				archvar=base_archs
-				ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,")
-				if [ "${ml_pkg}" != "${pkg}" ]; then
-					archvar=ml_archs
-				fi
+				ml_prefix=`echo ${pkg} | cut -d'-' -f1`
+				ml_pkg=$pkg
+				for i in ${MULTILIB_PREFIX_LIST} ; do
+					if [ ${ml_prefix} == ${i} ]; then
+						ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,")
+						archvar=ml_archs
+						break
+					fi
+				done
 
 				pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
 				if [ -z "$pkg_name" ]; then
@@ -224,16 +229,20 @@ package_install_internal_rpm () {
 			done
 		fi
 	fi
-
 	if [ ! -z "${package_to_install}" ]; then
 		for pkg in ${package_to_install} ; do
 			echo "Processing $pkg..."
 
 			archvar=base_archs
-			ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,")
-			if [ "${ml_pkg}" != "${pkg}" ]; then
-				archvar=ml_archs
-			fi
+			ml_prefix=`echo ${pkg} | cut -d'-' -f1`
+			ml_pkg=$pkg
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				if [ ${ml_prefix} == ${i} ]; then
+					ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,")
+					archvar=ml_archs
+					break
+				fi
+			done
 
 			pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
 			if [ -z "$pkg_name" ]; then
@@ -258,10 +267,15 @@ package_install_internal_rpm () {
 		for pkg in ${package_attemptonly} ; do
 			echo "Processing $pkg..."
 			archvar=base_archs
-			ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,")
-			if [ "${ml_pkg}" != "${pkg}" ]; then
-				archvar=ml_archs
-			fi
+			ml_prefix=`echo ${pkg} | cut -d'-' -f1`
+			ml_pkg=$pkg
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				if [ ${ml_prefix} == ${i} ]; then
+					ml_pkg=$(echo ${pkg} | sed "s,^${ml_prefix}-\(.*\),\1,")
+					archvar=ml_archs
+					break
+				fi
+			done
 
 			pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
 			if [ -z "$pkg_name" ]; then
meta/classes/rootfs_rpm.bbclass +4 -0
index 56c1a85..ec58700 100644
@@ -58,6 +58,7 @@ fakeroot rootfs_rpm_do_rootfs () {
 	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_MULTILIB_RPM="${MULTILIB_PACKAGE_INSTALL}"
 	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
 	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
 	export INSTALL_PROVIDENAME_RPM=""
@@ -210,6 +211,7 @@ python () {
         bb.data.setVar('RPM_POSTPROCESS_COMMANDS', '', d)
 
     ml_package_archs = ""
+    ml_prefix_list = ""
     multilibs = d.getVar('MULTILIBS', True) or ""
     for ext in multilibs.split():
         eext = ext.split(':')
@@ -222,6 +224,8 @@ python () {
             if machine:
                 localdata.setVar("MACHINE", machine)
             ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or ""
+            ml_prefix_list += " " + eext[1]
             #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
     bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d)
+    bb.data.setVar('MULTILIB_PREFIX_LIST', ml_prefix_list, d)
 }
Signature · 1 line
1.7.1