Skip to contentSkip to conversation

Fix the SDK generation

PATCH In review Mark Hatle 3 participants 14 messages Aug 2, 2011 – Aug 3, 2011

lore
Share & tools
lore

Mark Hatle

author
#0 lore raw

cover Fix the SDK generation

This is an attempt to fix a number of issues with the SDK generation. It has been tested using bitbake meta-toolchain-sdk.

Note, patches:

4/5: package_ipk: SDK generation workaround
5/5: update-alternatives: Add alternatives as a runtime provide

4/5 might be reasonable to implement immediately to resolve this issue.. However 5/5 is absolutely incorrect and that is noted within the patch header. I'd like comments on the issue to understand how to fix the problem.

The following changes since commit 2a41a311ddda11713296391050f3c2c1b2c1d3d3:

arch-armv7a.inc: fix armv7a-vfp-neon -> armv7a compat case (2011-08-01 16:49:03 +0100)

are available in the git repository at:
git://git.pokylinux.org/poky-contrib mhatle/oecore/sdk-fix-WIP
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=mhatle/oecore/sdk-fix-WIP

Mark Hatle (5):
rootfs_rpm: Cleanup and minor bug fixes bitbake.conf: Add SDK_PACKAGE_ARCHS populate_sdk_*: Sync SDK and regular rootfs functions
package_ipk: SDK generation workaround
update-alternatives: Add alternatives as a runtime provide

 meta/classes/package_deb.bbclass         |    6 +-
 meta/classes/package_ipk.bbclass         |   43 +++++------
 meta/classes/package_rpm.bbclass         |  120 ++++++++++++------------------
 meta/classes/populate_sdk.bbclass        |    9 ++
 meta/classes/populate_sdk_deb.bbclass    |   35 ++++-----
 meta/classes/populate_sdk_rpm.bbclass    |   89 +++++++++++++++++------
 meta/classes/rootfs_rpm.bbclass          |   30 ++++----
 meta/classes/update-alternatives.bbclass |    2 +
 meta/conf/bitbake.conf                   |    1 +
 9 files changed, 179 insertions(+), 156 deletions(-)
Signature · 1 line
1.7.3.4
author
#1 lore raw

PATCH 1/5 rootfs_rpm: Cleanup and minor bug fixes

in packages_rpm.bbclass:

Cleanup the way package_update_index_rpm works. Break this out into two functions, one for common functionality and one for controlling target vs sdk package processing.

Similar change to package_generate_rpm_conf.

Usage of IMAGE_ROOTFS should actually be "target_rootfs", this fixes a problem with the SDK case.

in rootfs_rpm.bbclass:

Minor re-order of the values, and avoid multiple uses of IMAGE_ROOTFS in order to make the merge with the SDK behavior easier.

---
 meta/classes/package_rpm.bbclass |  128 ++++++++++++++++---------------------
 meta/classes/rootfs_rpm.bbclass  |   17 +++---
 2 files changed, 64 insertions(+), 81 deletions(-)
meta/classes/package_rpm.bbclass +55 -73
index 950f665..0f16613 100644
@@ -25,31 +25,41 @@ package_update_index_rpm () {
 		return
 	fi
 
-	base_package_archs="${PACKAGE_ARCHS}"
-	ml_package_archs="${MULTILIB_PACKAGE_ARCHS}"
+	# Update target packages
+	base_archs="${PACKAGE_ARCHS}"
+	ml_archs="${MULTILIB_PACKAGE_ARCHS}"
+	package_update_index_rpm_common "${RPMCONF_TARGET_BASE}" base_archs ml_archs
+
+	# Update SDK packages
+	base_archs=""
+	for arch in ${PACKAGE_ARCHS}; do
+		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
+		extension="-nativesdk"
+		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
+		    extension=""
+		fi
+		base_archs="$base_archs $sdkarch$extension"
+	done
+	package_update_index_rpm_common "${RPMCONF_HOST_BASE}" base_archs
+}
+
+package_update_index_rpm_common () {
+	rpmconf_base="$1"
+	shift
 
-	for archvar in base_package_archs ml_package_archs; do
+	for archvar in "$@"; do
 		eval archs=\${${archvar}}
 		packagedirs=""
-		packagedirs_sdk=""
 		for arch in $archs; do
-			sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-			extension="-nativesdk"
-			if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-			    extension=""
-			fi
 			packagedirs="${DEPLOY_DIR_RPM}/$arch $packagedirs"
-			packagedirs_sdk="${DEPLOY_DIR_RPM}/$sdkarch$extension $packagedirs_sdk"
-
 			rm -rf ${DEPLOY_DIR_RPM}/$arch/solvedb
-			rm -rf ${DEPLOY_DIR_RPM}/$sdkarch$extension/solvedb
 		done
 
-		cat /dev/null > ${RPMCONF_TARGET_BASE}-${archvar}.conf
+		cat /dev/null > ${rpmconf_base}-${archvar}.conf
 		for pkgdir in $packagedirs; do
 			if [ -e $pkgdir/ ]; then
 				echo "Generating solve db for $pkgdir..."
-				echo $pkgdir/solvedb >> ${RPMCONF_TARGET_BASE}-${archvar}.conf
+				echo $pkgdir/solvedb >> ${rpmconf_base}-${archvar}.conf
 				if [ -d $pkgdir/solvedb ]; then
 					# We've already processed this and it's a duplicate
 					continue
@@ -65,27 +75,6 @@ package_update_index_rpm () {
 					$pkgdir/solvedb/manifest
 			fi
 		done
-
-		cat /dev/null > ${RPMCONF_HOST_BASE}-${archvar}.conf
-		for pkgdir in $packagedirs_sdk; do
-			if [ -e $pkgdir/ ]; then
-				echo "Generating solve db for $pkgdir..."
-				echo $pkgdir/solvedb >> ${RPMCONF_HOST_BASE}-${archvar}.conf
-				if [ -d $pkgdir/solvedb ]; then
-					# We've already processed this and it's a duplicate
-					continue
-				fi	
-				mkdir -p $pkgdir/solvedb
-				echo "# Dynamically generated solve manifest" >> $pkgdir/solvedb/manifest
-				find $pkgdir -maxdepth 1 -type f >> $pkgdir/solvedb/manifest
-				${RPM} -i --replacepkgs --replacefiles --oldpackage \
-					-D "_dbpath $pkgdir/solvedb" --justdb \
-					--noaid --nodeps --noorder --noscripts --notriggers --noparentdirs --nolinktos --stats \
-					--ignoresize --nosignature --nodigest \
-					-D "__dbi_txn create nofsync" \
-					$pkgdir/solvedb/manifest
-			fi
-		done
 	done
 }
 
@@ -94,45 +83,38 @@ package_update_index_rpm () {
 # generated depsolver db's...
 #
 package_generate_rpm_conf () {
-	printf "_solve_dbpath " > ${RPMCONF_TARGET_BASE}.macro
-	o_colon_t=false
-	o_colon_h=false
+	# Update target packages
+	package_generate_rpm_conf_common "${RPMCONF_TARGET_BASE}" base_archs ml_archs
 
-	for archvar in base_package_archs ml_package_archs; do
-		printf "_solve_dbpath " > ${RPMCONF_TARGET_BASE}-${archvar}.macro
-		colon=false
-		for each in `cat ${RPMCONF_TARGET_BASE}-${archvar}.conf` ; do
-			if [ "$o_colon_t" == true ]; then
-				printf ":" >> ${RPMCONF_TARGET_BASE}.macro
-			fi
-			if [ "$colon" == true ]; then
-				printf ":" >> ${RPMCONF_TARGET_BASE}-${archvar}.macro
-			fi
-			printf "%s" $each >> ${RPMCONF_TARGET_BASE}.macro
-			o_colon_t=true
-			printf "%s" $each >> ${RPMCONF_TARGET_BASE}-${archvar}.macro
-			colon=true
-		done
-		printf "\n" >> ${RPMCONF_TARGET_BASE}-${archvar}.macro
+	# Update SDK packages
+	package_generate_rpm_conf_common "${RPMCONF_HOST_BASE}" base_archs
+}
+
+package_generate_rpm_conf_common() {
+	rpmconf_base="$1"
+	shift
+
+	printf "_solve_dbpath " > ${rpmconf_base}.macro
+	o_colon=false
 
-		printf "_solve_dbpath " > ${RPMCONF_HOST_BASE}-${archvar}.macro
+	for archvar in "$@"; do
+		printf "_solve_dbpath " > ${rpmconf_base}-${archvar}.macro
 		colon=false
-		for each in `cat ${RPMCONF_HOST_BASE}-${archvar}.conf` ; do
-			if [ "$o_colon_h" == true ]; then
-				printf ":" >> ${RPMCONF_HOST_BASE}.macro
+		for each in `cat ${rpmconf_base}-${archvar}.conf` ; do
+			if [ "$o_colon" == true ]; then
+				printf ":" >> ${rpmconf_base}.macro
 			fi
 			if [ "$colon" == true ]; then
-				printf ":" >> ${RPMCONF_HOST_BASE}-${archvar}.macro
+				printf ":" >> ${rpmconf_base}-${archvar}.macro
 			fi
-			printf "%s" $each >> ${RPMCONF_HOST_BASE}.macro
-			o_colon_h=true
-			printf "%s" $each >> ${RPMCONF_HOST_BASE}-${archvar}.macro
+			printf "%s" $each >> ${rpmconf_base}.macro
+			o_colon=true
+			printf "%s" $each >> ${rpmconf_base}-${archvar}.macro
 			colon=true
 		done
-		printf "\n" >> ${RPMCONF_HOST_BASE}-${archvar}.macro
+		printf "\n" >> ${rpmconf_base}-${archvar}.macro
 	done
-	printf "\n" >> ${RPMCONF_TARGET_BASE}.macro
-	printf "\n" >> ${RPMCONF_HOST_BASE}.macro
+	printf "\n" >> ${rpmconf_base}.macro
 }
 
 rpm_log_check() {
@@ -211,7 +193,7 @@ package_install_internal_rpm () {
 	mkdir -p ${target_rootfs}/etc/rpm/sysinfo
 	echo "/" >${target_rootfs}/etc/rpm/sysinfo/Dirnames
 	if [ ! -z "$providename" ]; then
-		>>${target_rootfs}/etc/rpm/sysinfo/Providename
+		cat /dev/null > ${target_rootfs}/etc/rpm/sysinfo/Providename
 		for provide in $providename ; do
 			echo $provide >> ${target_rootfs}/etc/rpm/sysinfo/Providename
 		done
@@ -227,10 +209,10 @@ package_install_internal_rpm () {
 			for pkg in ${package_linguas}; do
 				echo "Processing $pkg..."
 
-				archvar=base_package_archs
+				archvar=base_archs
 				ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,")
 				if [ "${ml_pkg}" != "${pkg}" ]; then
-					archvar=ml_package_archs
+					archvar=ml_archs
 				fi
 
 				pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
@@ -238,7 +220,7 @@ package_install_internal_rpm () {
 					echo "Unable to find package $pkg ($ml_pkg)!"
 					exit 1
 				fi
-				echo $pkg_name >> ${IMAGE_ROOTFS}/install/install.manifest
+				echo $pkg_name >> ${target_rootfs}/install/install.manifest
 			done
 		fi
 	fi
@@ -247,10 +229,10 @@ package_install_internal_rpm () {
 		for pkg in ${package_to_install} ; do
 			echo "Processing $pkg..."
 
-			archvar=base_package_archs
+			archvar=base_archs
 			ml_pkg=$(echo ${pkg} | sed "s,^${MLPREFIX}\(.*\),\1,")
 			if [ "${ml_pkg}" != "${pkg}" ]; then
-				archvar=ml_package_archs
+				archvar=ml_archs
 			fi
 
 			pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${ml_pkg})
@@ -258,7 +240,7 @@ package_install_internal_rpm () {
 				echo "Unable to find package $pkg ($ml_pkg)!"
 				exit 1
 			fi
-			echo $pkg_name >> ${IMAGE_ROOTFS}/install/install.manifest
+			echo $pkg_name >> ${target_rootfs}/install/install.manifest
 		done
 	fi
 
@@ -303,7 +285,7 @@ package_install_internal_rpm () {
 		# Dump the full set of recommends...
 		${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
 			--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-			-D "_dbpath ${IMAGE_ROOTFS}/install" -D "`cat ${confbase}.macro`" \
+			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
 			-D "__dbi_txn create nofsync private" \
 			-qa --qf "[%{RECOMMENDS}\n]" | sort -u > ${target_rootfs}/install/recommend
 		# Did we add more to the list?
meta/classes/rootfs_rpm.bbclass +9 -8
index dfe4612..4f627a5 100644
@@ -52,12 +52,16 @@ fakeroot rootfs_rpm_do_rootfs () {
 
 	#createrepo "${DEPLOY_DIR_RPM}"
 
+	# install packages
+	# This needs to work in the same way as populate_sdk_rpm.bbclass!
+	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
+
 	# Setup base system configuration
-	mkdir -p ${IMAGE_ROOTFS}/etc/rpm/
+	mkdir -p ${INSTALL_ROOTFS_RPM}/etc/rpm/
 
-	mkdir -p ${IMAGE_ROOTFS}${rpmlibdir}
-	mkdir -p ${IMAGE_ROOTFS}${rpmlibdir}/log
-	cat > ${IMAGE_ROOTFS}${rpmlibdir}/DB_CONFIG << EOF
+	mkdir -p ${INSTALL_ROOTFS_RPM}${rpmlibdir}
+	mkdir -p ${INSTALL_ROOTFS_RPM}${rpmlibdir}/log
+	cat > ${INSTALL_ROOTFS_RPM}${rpmlibdir}/DB_CONFIG << EOF
 # ================ Environment
 set_data_dir            .
 set_create_dir          .
@@ -81,15 +85,13 @@ mutex_set_max           163840
 # ================ Replication
 EOF
 
-	#install pacakges
-	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
 	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}"
 	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
 	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
 	export INSTALL_PROVIDENAME_RPM=""
-	export INSTALL_TASK_RPM="populate_sdk"
+	export INSTALL_TASK_RPM="rootfs_rpm_do_rootfs"
 
 	# List must be prefered to least preferred order
 	INSTALL_PLATFORM_EXTRA_RPM=""
@@ -221,5 +223,4 @@ python () {
             ml_package_archs += localdata.getVar("PACKAGE_ARCHS", True) or ""
             #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
     bb.data.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs, d)
-
 }
Signature · 1 line
1.7.3.4
author
#2 lore raw

PATCH 2/5 bitbake.conf: Add SDK_PACKAGE_ARCHS

In reply to Mark Hatle

Add SDK_PACKAGE_ARCHS to avoid the sed operations we were previously doing inside of the variouns populate_sdk functions and related items.

Also add documentation to populate_sdk to explain when the various functions are expected to be doing.

Finally fix a bug in populate_sdk_rpm where the wrong value was being set, noticed while working on this change.

---
 meta/classes/package_deb.bbclass      |    6 +----
 meta/classes/package_ipk.bbclass      |   37 +++++++++++++++-----------------
 meta/classes/package_rpm.bbclass      |   10 +--------
 meta/classes/populate_sdk.bbclass     |    9 ++++++++
 meta/classes/populate_sdk_deb.bbclass |   13 +----------
 meta/classes/populate_sdk_rpm.bbclass |   23 ++++++++------------
 meta/conf/bitbake.conf                |    1 +
 7 files changed, 39 insertions(+), 60 deletions(-)
meta/classes/package_deb.bbclass +1 -5
index 8e363f8..48aa232 100644
@@ -78,14 +78,10 @@ package_update_index_deb () {
 		return
 	fi
 
-	for arch in ${PACKAGE_ARCHS}; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
+	for arch in ${PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}; do
 		if [ -e ${DEPLOY_DIR_DEB}/$arch ]; then
 			debarchs="$debarchs $arch"
 		fi
-		if [ -e ${DEPLOY_DIR_DEB}/$sdkarch-nativesdk ]; then
-			debarchs="$debarchs $sdkarch-nativesdk"
-		fi
 	done
 
 	for arch in $debarchs; do
meta/classes/package_ipk.bbclass +17 -20
index 16574e9..3768889 100644
@@ -161,7 +161,7 @@ ipk_log_check() {
 package_update_index_ipk () {
 	set -x
 
-	ipkgarchs="${PACKAGE_ARCHS}"
+	ipkgarchs="${PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}"
 
 	if [ ! -z "${DEPLOY_KEEP_PACKAGES}" ]; then
 		return
@@ -169,8 +169,7 @@ package_update_index_ipk () {
 
 	packagedirs="${DEPLOY_DIR_IPK}"
 	for arch in $ipkgarchs; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch ${DEPLOY_DIR_IPK}/$sdkarch-nativesdk"
+		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch"
 	done
 
 	multilib_archs="${MULTILIB_ARCHS}"
@@ -193,21 +192,20 @@ package_update_index_ipk () {
 #
 package_generate_ipkg_conf () {
 	package_generate_archlist
-	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_TARGET}
 	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_SDK}
+	ipkgarchs="${SDK_PACKAGE_ARCHS}"
+	for arch in $ipkgarchs; do
+		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_SDK}
+		fi
+	done
+
+	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_TARGET}
 	ipkgarchs="${PACKAGE_ARCHS}"
 	for arch in $ipkgarchs; do
 		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
 		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_TARGET}
 		fi
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		extension=-nativesdk
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		if [ -e ${DEPLOY_DIR_IPK}/$sdkarch$extension/Packages ] ; then
-		        echo "src oe-$sdkarch$extension file:${DEPLOY_DIR_IPK}/$sdkarch$extension" >> ${IPKGCONF_SDK}
-		fi
 	done
 
 	multilib_archs="${MULTILIB_ARCHS}"
@@ -219,16 +217,17 @@ package_generate_ipkg_conf () {
 }
 
 package_generate_archlist () {
+	ipkgarchs="${SDK_PACKAGE_ARCHS}"
+	priority=1
+	for arch in $ipkgarchs; do
+		echo "arch $arch $priority" >> ${IPKGCONF_SDK}
+		priority=$(expr $priority + 5)
+	done
+
 	ipkgarchs="${PACKAGE_ARCHS}"
 	priority=1
 	for arch in $ipkgarchs; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
 		echo "arch $arch $priority" >> ${IPKGCONF_TARGET}
-		extension=-nativesdk
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		echo "arch $sdkarch$extension $priority" >> ${IPKGCONF_SDK}
 		priority=$(expr $priority + 5)
 	done
 
@@ -237,8 +236,6 @@ package_generate_archlist () {
 		echo "arch $arch $priority" >> ${IPKGCONF_TARGET}
 		priority=$(expr $priority + 5)
 	done
-
-
 }
 
 python do_package_ipk () {
meta/classes/package_rpm.bbclass +1 -9
index 0f16613..abedc68 100644
@@ -31,15 +31,7 @@ package_update_index_rpm () {
 	package_update_index_rpm_common "${RPMCONF_TARGET_BASE}" base_archs ml_archs
 
 	# Update SDK packages
-	base_archs=""
-	for arch in ${PACKAGE_ARCHS}; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		extension="-nativesdk"
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		base_archs="$base_archs $sdkarch$extension"
-	done
+	base_archs="${SDK_PACKAGE_ARCHS}"
 	package_update_index_rpm_common "${RPMCONF_HOST_BASE}" base_archs
 }
 
meta/classes/populate_sdk.bbclass +9 -0
index 089ed9a..0f3591b 100644
@@ -22,6 +22,15 @@ fakeroot do_populate_sdk() {
 	rm -rf ${SDK_OUTPUT}
 	mkdir -p ${SDK_OUTPUT}
 
+	# populate_sdk_<image> is required to construct two images:
+	#  SDK_ARCH-nativesdk - contains the cross compiler and associated tooling
+	#  target - contains a target rootfs configured for the SDK usage
+	#
+	# the output of populate_sdk_<image> should end up in ${SDK_OUTPUT} it is made
+	# up of:
+	#  ${SDK_OUTPUT}/<sdk_arch-nativesdk pkgs>
+	#  ${SDK_OUTPUT}/${SDKTARGETSYSROOT}/<target pkgs>
+
 	populate_sdk_${IMAGE_PKGTYPE}
 
 	# Don't ship any libGL in the SDK
meta/classes/populate_sdk_deb.bbclass +1 -12
index a5b6384..25d90c7 100644
@@ -34,18 +34,7 @@ fakeroot populate_sdk_deb () {
 	export INSTALL_ROOTFS_DEB="${SDK_OUTPUT}"
 	export INSTALL_BASEARCH_DEB="${SDK_ARCH}"
 	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_HOST_TASK}"
-	INSTALL_ARCHS_DEB=""
-	for arch in ${PACKAGE_ARCHS}; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		extension="-nativesdk"
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		if [ -e ${DEPLOY_DIR_DEB}/$sdkarch$extension ]; then
-			INSTALL_ARCHS_DEB="$INSTALL_ARCHS_DEB $sdkarch$extension"
-		fi
-	done
-	export INSTALL_ARCHS_DEB
+	export INSTALL_ARCHS_DEB="${SDK_PACKAGE_ARCHS}"
 
 	package_install_internal_deb
 	populate_sdk_post_deb ${SDK_OUTPUT}/${SDKPATHNATIVE}
meta/classes/populate_sdk_rpm.bbclass +9 -14
index da308c1..560e40c 100644
@@ -34,16 +34,16 @@ fakeroot populate_sdk_rpm () {
 
 	#install target
 	export INSTALL_ROOTFS_RPM="${SDK_OUTPUT}/${SDKTARGETSYSROOT}"
-	export INSTALL_PLATFORM_RPM="${TARGET_ARCHS}"
+	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${TOOLCHAIN_TARGET_TASK}"
 
 	# List must be prefered to least preferred order
-	INSTALL_PLATFORM_RPM=""
+	INSTALL_PLATFORM_EXTRA_RPM=""
 	for each_arch in ${PACKAGE_ARCHS} ; do
-		INSTALL_PLATFORM_RPM="$each_arch $INSTALL_PLATFORM_RPM"
+		INSTALL_PLATFORM_EXTRA_RPM="$each_arch $INSTALL_PLATFORM_EXTRA_RPM"
 	done
-	export INSTALL_PLATFORM_RPM
+	export INSTALL_PLATFORM_EXTRA_RPM
 
 	package_install_internal_rpm
 	populate_sdk_post_rpm ${INSTALL_ROOTFS_RPM}
@@ -53,16 +53,11 @@ fakeroot populate_sdk_rpm () {
 	export INSTALL_PLATFORM_RPM="${SDK_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_HOST_BASE}"
 	export INSTALL_PACKAGES_NORMAL_RPM="${TOOLCHAIN_HOST_TASK}"
-	export INSTALL_PLATFORM_EXTRA_RPM=""
-	for arch in ${PACKAGE_ARCHS}; do
-		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
-		extension="-nativesdk"
-		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
-		    extension=""
-		fi
-		if [ -e ${DEPLOY_DIR_RPM}/$sdkarch$extension ]; then
-			INSTALL_PLATFORM_EXTRA_RPM="$sdkarch $INSTALL_PLATFORM_EXTRA_RPM"
-		fi
+
+	# List must be prefered to least preferred order
+	INSTALL_PLATFORM_EXTRA_RPM=""
+	for each_arch in ${SDK_PACKAGE_ARCHS} ; do
+		INSTALL_PLATFORM_EXTRA_RPM="$each_arch $INSTALL_PLATFORM_EXTRA_RPM"
 	done
 	export INSTALL_PLATFORM_EXTRA_RPM
 
meta/conf/bitbake.conf +1 -0
index 9c80f11..ac5d1f3 100644
@@ -100,6 +100,7 @@ SDK_VENDOR = "-oesdk"
 SDK_SYS = "${SDK_ARCH}${SDK_VENDOR}${@['-' + bb.data.getVar('SDK_OS', d, 1), ''][bb.data.getVar('SDK_OS', d, 1) == ('' or 'custom')]}"
 SDK_PREFIX = "${SDK_SYS}-"
 SDK_CC_ARCH = "${BUILD_CC_ARCH}"
+SDK_PACKAGE_ARCHS = "all any noarch ${SDK_ARCH}-nativesdk"
 
 PACKAGE_ARCH = "${TUNE_PKGARCH}"
 MACHINE_ARCH = "${@[bb.data.getVar('TUNE_PKGARCH', d, 1), bb.data.getVar('MACHINE', d, 1)][bool(bb.data.getVar('MACHINE', d, 1))].replace('-', '_')}"
Signature · 1 line
1.7.3.4
author
#3 lore raw

PATCH 3/5 populate_sdk_*: Sync SDK and regular rootfs functions

In reply to Mark Hatle

In order for things to be easier to maintain in the future, sync up the sdk and rootfs versions.

---
 meta/classes/populate_sdk_deb.bbclass |   24 ++++++++----
 meta/classes/populate_sdk_rpm.bbclass |   68 ++++++++++++++++++++++++++++----
 meta/classes/rootfs_rpm.bbclass       |   15 +++----
 3 files changed, 82 insertions(+), 25 deletions(-)
meta/classes/populate_sdk_deb.bbclass +16 -8
index 25d90c7..1bf1b95 100644
@@ -11,30 +11,38 @@ populate_sdk_post_deb () {
 
 fakeroot populate_sdk_deb () {
 
+	# update index
 	package_update_index_deb
 
-	export INSTALL_TASK_DEB="populate_sdk"
-	export INSTALL_PACKAGES_LINGUAS_DEB=""
-	export INSTALL_PACKAGES_ATTEMPTONLY_DEB=""
-
-	#install target
+	## install target ##
+	# This needs to work in the same way as rootfs_deb.bbclass
 	echo "Installing TARGET packages"
+
+	mkdir -p ${IMAGE_ROOTFS}/var/dpkg/alternatives
+
 	export INSTALL_ROOTFS_DEB="${SDK_OUTPUT}/${SDKTARGETSYSROOT}"
 	export INSTALL_BASEARCH_DEB="${DPKG_ARCH}"
 	export INSTALL_ARCHS_DEB="${PACKAGE_ARCHS}"
 	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_TARGET_TASK}"
+	export INSTALL_PACKAGES_ATTEMPTONLY_DEB=""
+	export PACKAGES_LINGUAS_DEB=""
+	export INSTALL_TASK_DEB="populate_sdk-target"
 
 	package_install_internal_deb
+
 	populate_sdk_post_deb ${INSTALL_ROOTFS_DEB}
 
 	populate_sdk_log_check populate_sdk
 
-	#install host
-	echo "Installing HOST packages"
+	## install nativesdk ##
+	echo "Installing NATIVESDK packages"
 	export INSTALL_ROOTFS_DEB="${SDK_OUTPUT}"
 	export INSTALL_BASEARCH_DEB="${SDK_ARCH}"
-	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_HOST_TASK}"
 	export INSTALL_ARCHS_DEB="${SDK_PACKAGE_ARCHS}"
+	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_HOST_TASK}"
+	export INSTALL_PACKAGES_ATTEMPTONLY_DEB=""
+	export PACKAGES_LINGUAS_DEB=""
+	export INSTALL_TASK_DEB="populate_sdk-nativesdk"
 
 	package_install_internal_deb
 	populate_sdk_post_deb ${SDK_OUTPUT}/${SDKPATHNATIVE}
meta/classes/populate_sdk_rpm.bbclass +59 -9

94 lines not on this page · raw message

meta/classes/rootfs_rpm.bbclass +7 -8
index 4f627a5..7c06064 100644
@@ -55,6 +55,13 @@ fakeroot rootfs_rpm_do_rootfs () {
 	# install packages
 	# This needs to work in the same way as populate_sdk_rpm.bbclass!
 	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
+	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
+	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
+	export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
+	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
+	export INSTALL_PROVIDENAME_RPM=""
+	export INSTALL_TASK_RPM="rootfs_rpm_do_rootfs"
 
 	# Setup base system configuration
 	mkdir -p ${INSTALL_ROOTFS_RPM}/etc/rpm/
@@ -85,14 +92,6 @@ mutex_set_max           163840
 # ================ Replication
 EOF
 
-	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
-	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
-	export INSTALL_PACKAGES_NORMAL_RPM="${PACKAGE_INSTALL}"
-	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
-	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
-	export INSTALL_PROVIDENAME_RPM=""
-	export INSTALL_TASK_RPM="rootfs_rpm_do_rootfs"
-
 	# List must be prefered to least preferred order
 	INSTALL_PLATFORM_EXTRA_RPM=""
 	for each_arch in ${MULTILIB_PACKAGE_ARCHS} ${PACKAGE_ARCHS}; do
Signature · 1 line
1.7.3.4
author
#4 lore raw

PATCH 4/5 package_ipk: SDK generation workaround

In reply to Mark Hatle

During SDK generation the multilib_sanity_check function is being called, however it is not available unless we've been called from the image.bbclass.

Disable the check if MULTILIB_CHECK_FILE (also set in image.bbclass) doesn't exist or is empty.

---
 meta/classes/package_ipk.bbclass |    6 ++++--
 1 files changed, 4 insertions(+), 2 deletions(-)
meta/classes/package_ipk.bbclass +4 -2
index 3768889..8ecf511 100644
@@ -127,8 +127,10 @@ package_install_internal_ipk() {
 	fi
 
 	package_tryout_install_multilib_ipk
-	#sanity check
-	multilib_sanity_check  ${target_rootfs} ${multilib_tryout_dirs}|| exit 1
+	if [ ! -z "${MULTILIB_CHECK_FILE}" ]; then
+		#sanity check
+		multilib_sanity_check ${target_rootfs} ${multilib_tryout_dirs} || exit 1
+	fi
 
 	if [ ! -z "${package_multilib}" ]; then
 		opkg-cl ${ipkg_args} install ${package_multilib}
Signature · 1 line
1.7.3.4
author
#5 lore raw

PATCH 5/5 update-alternatives: Add alternatives as a runtime provide

In reply to Mark Hatle

The following allows RPM to generate the SDK image, however without it we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because they can not have filenames within their RPROVIDES. I'm looking for some type of a resolution to the issue, the only thing I can think of is to add a way to manually add a FILERPROVIDE for the items. This will require changes to the way FILERPROVIDE is currently generated... but I'm not sure how we can automatically generate the FILERPROVIDE values without the use of python...

Any suggestions?

-----

Add the automatic alternatives as runtime provides.

This was discovered when programers were requiring "/bin/sh" and there was nothing within the system providing "/bin/sh", however bash had an alternative for /bin/sh that would satisfy this dependency.

NOTE: THIS BREAKS IPK AND DEB PACKAGE GENERATION! IT SHOULD NOT BE MERGED!

---
 meta/classes/update-alternatives.bbclass |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)
meta/classes/update-alternatives.bbclass +2 -0
index ba81221..4cd680e 100644
@@ -40,6 +40,8 @@
 ALTERNATIVE_PRIORITY = "10"
 ALTERNATIVE_LINK = "${bindir}/${ALTERNATIVE_NAME}"
 
+RPROVIDES_${PN} += "${@(d.getVar('ALTERNATIVE_LINK', True) or '') + ' ' + (d.getVar('ALTERNATIVE_LINKS', True) or '')}"
+
 update_alternatives_postinst() {
 update-alternatives --install ${ALTERNATIVE_LINK} ${ALTERNATIVE_NAME} ${ALTERNATIVE_PATH} ${ALTERNATIVE_PRIORITY}
 }
Signature · 1 line
1.7.3.4
#6 lore raw

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

Quoted 11 lines · Mark Hatle

The following allows RPM to generate the SDK image, however without it we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because they can not have filenames within their RPROVIDES. I'm looking for some type of a resolution to the issue, the only thing I can think of is to add a way to manually add a FILERPROVIDE for the items. This will require changes to the way FILERPROVIDE is currently generated... but I'm not sure how we can automatically generate the FILERPROVIDE values without the use of python...

Quoted 1 line

Any suggestions?

It's never really been the intent that update-alternatives should put the name of the link being provided into RPROVIDES. If you want to solve the specific problem with /bin/sh then just adding RPROVIDES_${PN} += "virtual-bourne-shell" or something to bash and busybox is probably the easiest way of doing that.

I wouldn't be entirely opposed to the concept of what you're proposing here, though. Something like:

RPROVIDES_${PN} += "${@' '.join(map(lambda x:
legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '', [ d.getVar('ALTERNATIVE_LINK', True) or '' ] + (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"

might be what you want, perhaps. I'm not sure that the resulting virtual names will be very pretty though.

p.

author
#7 lore raw

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

Quoted 23 lines · Phil Blundell
Quoted 11 lines · Mark Hatle

The following allows RPM to generate the SDK image, however without it we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because they can not have filenames within their RPROVIDES. I'm looking for some type of a resolution to the issue, the only thing I can think of is to add a way to manually add a FILERPROVIDE for the items. This will require changes to the way FILERPROVIDE is currently generated... but I'm not sure how we can automatically generate the FILERPROVIDE values without the use of python...

Quoted 1 line

Any suggestions?

It's never really been the intent that update-alternatives should put the name of the link being provided into RPROVIDES. If you want to solve the specific problem with /bin/sh then just adding RPROVIDES_${PN} += "virtual-bourne-shell" or something to bash and busybox is probably the easiest way of doing that.

I wouldn't be entirely opposed to the concept of what you're proposing here, though. Something like:

Quoted 7 lines

RPROVIDES_${PN} += "${@' '.join(map(lambda x:
legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '', [ d.getVar('ALTERNATIVE_LINK', True) or '' ] + (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"

might be what you want, perhaps. I'm not sure that the resulting virtual names will be very pretty though.

Hmm.. Coming from the RPM world, the virtual-path- because we can't just "provide" a file in the system seems a bit wonky to me. But it should be able to work. For RPM at least, we'd want a reversing function to turn virtual paths back into real paths.

If I have time today, I'll try to implement a proof of concept and see if I can get it to work reasonably well.

--Mark

Quoted 8 lines

p.

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#8 lore raw

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

Quoted 32 lines · Mark Hatle
Quoted 23 lines · Phil Blundell
Quoted 11 lines · Mark Hatle

The following allows RPM to generate the SDK image, however without it we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because they can not have filenames within their RPROVIDES. I'm looking for some type of a resolution to the issue, the only thing I can think of is to add a way to manually add a FILERPROVIDE for the items. This will require changes to the way FILERPROVIDE is currently generated... but I'm not sure how we can automatically generate the FILERPROVIDE values without the use of python...

Quoted 1 line

Any suggestions?

It's never really been the intent that update-alternatives should put the name of the link being provided into RPROVIDES. If you want to solve the specific problem with /bin/sh then just adding RPROVIDES_${PN} += "virtual-bourne-shell" or something to bash and busybox is probably the easiest way of doing that.

I wouldn't be entirely opposed to the concept of what you're proposing here, though. Something like:

Quoted 7 lines

RPROVIDES_${PN} += "${@' '.join(map(lambda x:
legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '', [ d.getVar('ALTERNATIVE_LINK', True) or '' ] + (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"

might be what you want, perhaps. I'm not sure that the resulting virtual names will be very pretty though.

Quoted 7 lines

Hmm.. Coming from the RPM world, the virtual-path- because we can't just "provide" a file in the system seems a bit wonky to me. But it should be able to work. For RPM at least, we'd want a reversing function to turn virtual paths back into real paths.

If I have time today, I'll try to implement a proof of concept and see if I can get it to work reasonably well.

Just to be clear for Phil's benefit, RPM natively supports file based dependencies, so a dependency of "/bin/sh" is automatically fulfilled by a package which contains "/bin/sh". Some dependencies such as the shebang in scripts are automatically added to packages and resolved by rpm.

I did chase down this bug a bit and it seems that if you "bitbake meta-toolchain-game" you hit an error about /bin/sh being missing but if you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is due to busybox shipping a /bin/sh.

The question is therefore how to handle this on the deb/ipk side and ensure we get consistency between the behaviours of the different backends. I thought with the rpm filedeps code in do_package, we were adding things like /bin/sh dependencies to the other package formats but now I'm not so sure.

Cheers,

Richard

author
#9 lore raw

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

Quoted 52 lines · Richard Purdie
Quoted 32 lines · Mark Hatle
Quoted 23 lines · Phil Blundell
Quoted 11 lines · Mark Hatle

The following allows RPM to generate the SDK image, however without it we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because they can not have filenames within their RPROVIDES. I'm looking for some type of a resolution to the issue, the only thing I can think of is to add a way to manually add a FILERPROVIDE for the items. This will require changes to the way FILERPROVIDE is currently generated... but I'm not sure how we can automatically generate the FILERPROVIDE values without the use of python...

Quoted 1 line

Any suggestions?

It's never really been the intent that update-alternatives should put the name of the link being provided into RPROVIDES. If you want to solve the specific problem with /bin/sh then just adding RPROVIDES_${PN} += "virtual-bourne-shell" or something to bash and busybox is probably the easiest way of doing that.

I wouldn't be entirely opposed to the concept of what you're proposing here, though. Something like:

Quoted 7 lines

RPROVIDES_${PN} += "${@' '.join(map(lambda x:
legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '', [ d.getVar('ALTERNATIVE_LINK', True) or '' ] + (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"

might be what you want, perhaps. I'm not sure that the resulting virtual names will be very pretty though.

Quoted 7 lines

Hmm.. Coming from the RPM world, the virtual-path- because we can't just "provide" a file in the system seems a bit wonky to me. But it should be able to work. For RPM at least, we'd want a reversing function to turn virtual paths back into real paths.

If I have time today, I'll try to implement a proof of concept and see if I can get it to work reasonably well.

Just to be clear for Phil's benefit, RPM natively supports file based dependencies, so a dependency of "/bin/sh" is automatically fulfilled by a package which contains "/bin/sh". Some dependencies such as the shebang in scripts are automatically added to packages and resolved by rpm.

I did chase down this bug a bit and it seems that if you "bitbake meta-toolchain-game" you hit an error about /bin/sh being missing but if you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is due to busybox shipping a /bin/sh.

Quoted 5 lines

The question is therefore how to handle this on the deb/ipk side and ensure we get consistency between the behaviours of the different backends. I thought with the rpm filedeps code in do_package, we were adding things like /bin/sh dependencies to the other package formats but now I'm not so sure.

Due to deb/ipk not handling file based dependencies, they are filtered out on the creation of the deb/ipk packages. The original intention was to use them, but it wasn't possible at the time. Simply adding a ton of file-based dependencies seemed like a huge mistake as well. (We'd have to add virtual provides for all of the virtual requirements....)

We could certainly select a few specific requirements and scan for and use those to catch obvious issues, such as perl, sh, bash, env/python... but it's still only a partial solution to the real issue.

--Mark

Quoted 12 lines

Cheers,

Richard

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#10 lore raw

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

Quoted 65 lines · Mark Hatle
Quoted 52 lines · Richard Purdie
Quoted 32 lines · Mark Hatle
Quoted 23 lines · Phil Blundell
Quoted 11 lines · Mark Hatle

The following allows RPM to generate the SDK image, however without it we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because they can not have filenames within their RPROVIDES. I'm looking for some type of a resolution to the issue, the only thing I can think of is to add a way to manually add a FILERPROVIDE for the items. This will require changes to the way FILERPROVIDE is currently generated... but I'm not sure how we can automatically generate the FILERPROVIDE values without the use of python...

Quoted 1 line

Any suggestions?

It's never really been the intent that update-alternatives should put the name of the link being provided into RPROVIDES. If you want to solve the specific problem with /bin/sh then just adding RPROVIDES_${PN} += "virtual-bourne-shell" or something to bash and busybox is probably the easiest way of doing that.

I wouldn't be entirely opposed to the concept of what you're proposing here, though. Something like:

Quoted 7 lines

RPROVIDES_${PN} += "${@' '.join(map(lambda x:
legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '', [ d.getVar('ALTERNATIVE_LINK', True) or '' ] + (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"

might be what you want, perhaps. I'm not sure that the resulting virtual names will be very pretty though.

Quoted 7 lines

Hmm.. Coming from the RPM world, the virtual-path- because we can't just "provide" a file in the system seems a bit wonky to me. But it should be able to work. For RPM at least, we'd want a reversing function to turn virtual paths back into real paths.

If I have time today, I'll try to implement a proof of concept and see if I can get it to work reasonably well.

Just to be clear for Phil's benefit, RPM natively supports file based dependencies, so a dependency of "/bin/sh" is automatically fulfilled by a package which contains "/bin/sh". Some dependencies such as the shebang in scripts are automatically added to packages and resolved by rpm.

I did chase down this bug a bit and it seems that if you "bitbake meta-toolchain-game" you hit an error about /bin/sh being missing but if you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is due to busybox shipping a /bin/sh.

Quoted 5 lines

The question is therefore how to handle this on the deb/ipk side and ensure we get consistency between the behaviours of the different backends. I thought with the rpm filedeps code in do_package, we were adding things like /bin/sh dependencies to the other package formats but now I'm not so sure.

Due to deb/ipk not handling file based dependencies, they are filtered out on the creation of the deb/ipk packages. The original intention was to use them, but it wasn't possible at the time. Simply adding a ton of file-based dependencies seemed like a huge mistake as well. (We'd have to add virtual provides for all of the virtual requirements....)

Quoted 3 lines

We could certainly select a few specific requirements and scan for and use those to catch obvious issues, such as perl, sh, bash, env/python... but it's still only a partial solution to the real issue.

Short term I'm tempted to buy us some time and do this (rpm specific):

package_rpm: Ensure alternatives links are reflected in rpm package dependencies

Currently, if a file is provided as an alternative link within the package, rpm doesn't see the dependency. This works out badly for dependencies such as /bin/sh which scripts might require.

Since rpm detects and adds these dependencies we do need to ensure the dependency information in the packages is correct. This patch does so for the rpm backend ensuring internal consistency whilst the approach for addressing this problem in the other package backends is considered.

---

meta/classes/package_rpm.bbclass +4 -0
index abedc68..c44fdcc 100644
@@ -501,6 +501,10 @@ python write_specfile () {
 		splitrconflicts  = bb.data.getVar('RCONFLICTS', localdata, True) or ""
 		splitrobsoletes  = []
 
+		# For now we need to manually supplement RPROVIDES with any update-alternatives links
+		if pkg == d.getVar("PN", True):
+			splitrprovides = splitrprovides + (d.getVar('ALTERNATIVE_LINK', True) or '') + (d.getVar('ALTERNATIVE_LINKS', True) or '')
+
 		# Gather special src/first package data
 		if srcname == splitname:
 			srcrdepends    = splitrdepends
author
#11 lore raw

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

Quoted 90 lines · Richard Purdie
Quoted 65 lines · Mark Hatle
Quoted 52 lines · Richard Purdie
Quoted 32 lines · Mark Hatle
Quoted 23 lines · Phil Blundell
Quoted 11 lines · Mark Hatle

The following allows RPM to generate the SDK image, however without it we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because they can not have filenames within their RPROVIDES. I'm looking for some type of a resolution to the issue, the only thing I can think of is to add a way to manually add a FILERPROVIDE for the items. This will require changes to the way FILERPROVIDE is currently generated... but I'm not sure how we can automatically generate the FILERPROVIDE values without the use of python...

Quoted 1 line

Any suggestions?

It's never really been the intent that update-alternatives should put the name of the link being provided into RPROVIDES. If you want to solve the specific problem with /bin/sh then just adding RPROVIDES_${PN} += "virtual-bourne-shell" or something to bash and busybox is probably the easiest way of doing that.

I wouldn't be entirely opposed to the concept of what you're proposing here, though. Something like:

Quoted 7 lines

RPROVIDES_${PN} += "${@' '.join(map(lambda x:
legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '', [ d.getVar('ALTERNATIVE_LINK', True) or '' ] + (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"

might be what you want, perhaps. I'm not sure that the resulting virtual names will be very pretty though.

Quoted 7 lines

Hmm.. Coming from the RPM world, the virtual-path- because we can't just "provide" a file in the system seems a bit wonky to me. But it should be able to work. For RPM at least, we'd want a reversing function to turn virtual paths back into real paths.

If I have time today, I'll try to implement a proof of concept and see if I can get it to work reasonably well.

Just to be clear for Phil's benefit, RPM natively supports file based dependencies, so a dependency of "/bin/sh" is automatically fulfilled by a package which contains "/bin/sh". Some dependencies such as the shebang in scripts are automatically added to packages and resolved by rpm.

I did chase down this bug a bit and it seems that if you "bitbake meta-toolchain-game" you hit an error about /bin/sh being missing but if you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is due to busybox shipping a /bin/sh.

Quoted 5 lines

The question is therefore how to handle this on the deb/ipk side and ensure we get consistency between the behaviours of the different backends. I thought with the rpm filedeps code in do_package, we were adding things like /bin/sh dependencies to the other package formats but now I'm not so sure.

Due to deb/ipk not handling file based dependencies, they are filtered out on the creation of the deb/ipk packages. The original intention was to use them, but it wasn't possible at the time. Simply adding a ton of file-based dependencies seemed like a huge mistake as well. (We'd have to add virtual provides for all of the virtual requirements....)

Quoted 3 lines

We could certainly select a few specific requirements and scan for and use those to catch obvious issues, such as perl, sh, bash, env/python... but it's still only a partial solution to the real issue.

Short term I'm tempted to buy us some time and do this (rpm specific):

package_rpm: Ensure alternatives links are reflected in rpm package dependencies

Currently, if a file is provided as an alternative link within the package, rpm doesn't see the dependency. This works out badly for dependencies such as /bin/sh which scripts might require.

Since rpm detects and adds these dependencies we do need to ensure the dependency information in the packages is correct. This patch does so for the rpm backend ensuring internal consistency whilst the approach for addressing this problem in the other package backends is considered.

---

meta/classes/package_rpm.bbclass +0 -0
index abedc68..c44fdcc 100644

meta/classes/package_rpm.bbclass python write_specfile () { · line 507

Quoted 7 lines
meta/classes/package_rpm.bbclass +4 -0
@@ -501,6 +501,10 @@ python write_specfile () {
 		splitrconflicts  = bb.data.getVar('RCONFLICTS', localdata, True) or ""
 		splitrobsoletes  = []
 
+		# For now we need to manually supplement RPROVIDES with any update-alternatives links
+		if pkg == d.getVar("PN", True):
+			splitrprovides = splitrprovides + (d.getVar('ALTERNATIVE_LINK', True) or '') + (d.getVar('ALTERNATIVE_LINKS', True) or '')
+

Will the "+" add a space in this case, otherwise we need additional spaces added.

But this is a good solution to the issue. It's RPM specific (for now) until we decide if we have to address the other packaging systems.

--Mark

Quoted 9 lines
meta/classes/package_rpm.bbclass +0 -0
 		# Gather special src/first package data
 		if srcname == splitname:
 			srcrdepends    = splitrdepends

_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core

#12 lore raw

Re: [PATCH 5/5] update-alternatives: Add alternatives as a runtime provide

Quoted 94 lines · Mark Hatle
Quoted 93 lines · Richard Purdie
Quoted 65 lines · Mark Hatle
Quoted 52 lines · Richard Purdie
Quoted 32 lines · Mark Hatle
Quoted 23 lines · Phil Blundell
Quoted 11 lines · Mark Hatle

The following allows RPM to generate the SDK image, however without it we get a failure because the system has nothing that provides /bin/sh.

Unfortunately the patch causes failures with ipk and deb packages because they can not have filenames within their RPROVIDES. I'm looking for some type of a resolution to the issue, the only thing I can think of is to add a way to manually add a FILERPROVIDE for the items. This will require changes to the way FILERPROVIDE is currently generated... but I'm not sure how we can automatically generate the FILERPROVIDE values without the use of python...

Quoted 1 line

Any suggestions?

It's never really been the intent that update-alternatives should put the name of the link being provided into RPROVIDES. If you want to solve the specific problem with /bin/sh then just adding RPROVIDES_${PN} += "virtual-bourne-shell" or something to bash and busybox is probably the easiest way of doing that.

I wouldn't be entirely opposed to the concept of what you're proposing here, though. Something like:

Quoted 7 lines

RPROVIDES_${PN} += "${@' '.join(map(lambda x:
legitimize_package_name("virtual-path-" + x), filter(lambda x: x != '', [ d.getVar('ALTERNATIVE_LINK', True) or '' ] + (d.getVar('ALTERNATIVE_LINKS', True) or '').split())))}"

might be what you want, perhaps. I'm not sure that the resulting virtual names will be very pretty though.

Quoted 7 lines

Hmm.. Coming from the RPM world, the virtual-path- because we can't just "provide" a file in the system seems a bit wonky to me. But it should be able to work. For RPM at least, we'd want a reversing function to turn virtual paths back into real paths.

If I have time today, I'll try to implement a proof of concept and see if I can get it to work reasonably well.

Just to be clear for Phil's benefit, RPM natively supports file based dependencies, so a dependency of "/bin/sh" is automatically fulfilled by a package which contains "/bin/sh". Some dependencies such as the shebang in scripts are automatically added to packages and resolved by rpm.

I did chase down this bug a bit and it seems that if you "bitbake meta-toolchain-game" you hit an error about /bin/sh being missing but if you "bitbake busybox; bitbake meta-toolchain-gmae" it will work. This is due to busybox shipping a /bin/sh.

Quoted 5 lines

The question is therefore how to handle this on the deb/ipk side and ensure we get consistency between the behaviours of the different backends. I thought with the rpm filedeps code in do_package, we were adding things like /bin/sh dependencies to the other package formats but now I'm not so sure.

Due to deb/ipk not handling file based dependencies, they are filtered out on the creation of the deb/ipk packages. The original intention was to use them, but it wasn't possible at the time. Simply adding a ton of file-based dependencies seemed like a huge mistake as well. (We'd have to add virtual provides for all of the virtual requirements....)

Quoted 3 lines

We could certainly select a few specific requirements and scan for and use those to catch obvious issues, such as perl, sh, bash, env/python... but it's still only a partial solution to the real issue.

Short term I'm tempted to buy us some time and do this (rpm specific):

package_rpm: Ensure alternatives links are reflected in rpm package dependencies

Currently, if a file is provided as an alternative link within the package, rpm doesn't see the dependency. This works out badly for dependencies such as /bin/sh which scripts might require.

Since rpm detects and adds these dependencies we do need to ensure the dependency information in the packages is correct. This patch does so for the rpm backend ensuring internal consistency whilst the approach for addressing this problem in the other package backends is considered.

---

meta/classes/package_rpm.bbclass +0 -0
index abedc68..c44fdcc 100644
@@ -501,6 +501,10 @@ python write_specfile () {
 		splitrconflicts  = bb.data.getVar('RCONFLICTS', localdata, True) or ""
 		splitrobsoletes  = []
 
Quoted 6 lines

meta/classes/package_rpm.bbclass python write_specfile () { · line 507

Quoted 4 lines
meta/classes/package_rpm.bbclass +4 -0
+		# For now we need to manually supplement RPROVIDES with any update-alternatives links
+		if pkg == d.getVar("PN", True):
+			splitrprovides = splitrprovides + (d.getVar('ALTERNATIVE_LINK', True) or '') + (d.getVar('ALTERNATIVE_LINKS', True) or '')
+

Will the "+" add a space in this case, otherwise we need additional spaces added.

It needs more spaces. I realised that after I posted it.

Quoted 2 lines

But this is a good solution to the issue. It's RPM specific (for now) until we decide if we have to address the other packaging systems.

I think I'll likely merge this to fix the immediate issues and we can think about better solutions to this...

Cheers,

Richard

#13 lore raw
In reply to Mark Hatle
Quoted 11 lines · Mark Hatle

This is an attempt to fix a number of issues with the SDK generation. It has been tested using bitbake meta-toolchain-sdk.

Note, patches:

4/5: package_ipk: SDK generation workaround
5/5: update-alternatives: Add alternatives as a runtime provide

4/5 might be reasonable to implement immediately to resolve this issue.. However 5/5 is absolutely incorrect and that is noted within the patch header. I'd like comments on the issue to understand how to fix the problem.

I've merged 1-4 to master, 4 on the grounds that it improves the situation substantially.

For 5, I suspect we'll need to process the path using python, I can't really see a way to avoid it. I assume / is an illegal character for deb anywhere, not just at the start of the dependency name?

Cheers,

Richard