Skip to contentSkip to conversation

Smart package management integration

PATCH New Paul Eggleton 1 participant 14 messages Dec 12, 2012

lore
Share & tools
lore
#0 lore raw

v3 cover Smart package management integration

This series integrates Smart into the root filesystem construction when rpm packaging is used, and replaces zypper on the target if runtime package management is enabled.

Improvements over previous versions of this series:
* Fix the sanity tests
* Fix runtime relocation issues
* Fix building multilib images

The following changes since commit b8b1b39961332c99d62ee466f7859bd62a0f806f:

bitbake.conf/utils: Drop some OVERRIDES from FILESPATH (2012-12-11 15:58:27 +0000)

are available in the git repository at:

git://git.openembedded.org/openembedded-core-contrib paule/smart2
http://cgit.openembedded.org/cgit.cgi/openembedded-core-contrib/log/?h=paule/smart2

Mark Hatle (10):
rootfs_rpm.bbclass: Revert to default shell logging behavior
package_rpm: Update the way the multilib package names are translated package_rpm.bbclass: Fix the way the RPM platform file is generated package_rpm.bbclass: Add additional logging package_rpm.bbclass: Add support for incremental installs package_rpm.bbclass: Add additional install error detection rpm/smart: Fix runtime-relocation issues w/ RPM and Smart package_rpm.bbclass: Add a simply way to enable RPM debugging coreutils_6.9: Update the older coreutils to no longer provide 'su'
package_rpm: Workaround for 'all' arch multilib package naming

Paul Eggleton (3):
classes/*_rpm: integrate Smart into RPM filesystem construction classes/rootfs_rpm: install smart instead of zypper in rpm-based
images
Update sanity tests to use smart instead of zypper

 meta/classes/imagetest-qemu.bbclass                |    6 +-
 meta/classes/package_rpm.bbclass                   |  739 +++++++++-----------
 meta/classes/populate_sdk_rpm.bbclass              |   79 +--
 meta/classes/rootfs_rpm.bbclass                    |   89 ++-
 meta/recipes-core/coreutils/coreutils_6.9.bb       |    6 +-
 .../python/python-smartpm_1.4.1.bb                 |   17 +-
 .../rpm/rpm/rpm-debug-platform.patch               |   65 ++
 meta/recipes-devtools/rpm/rpm/rpm-platform.patch   |    2 +-
 meta/recipes-devtools/rpm/rpm/rpm-platform2.patch  |   87 ++-
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |   41 +-
 .../sanity/{zypper_help => smart_help}             |   24 +-
 .../sanity/{zypper_search => smart_query}          |   24 +-
 .../scenario/qemuarm/core-image-lsb                |    4 +-
 .../scenario/qemuarm/core-image-sato               |    4 +-
 .../scenario/qemuarm/core-image-sato-sdk           |    4 +-
 .../scenario/qemumips/core-image-lsb               |    4 +-
 .../scenario/qemumips/core-image-sato              |    4 +-
 .../scenario/qemumips/core-image-sato-sdk          |    4 +-
 .../scenario/qemuppc/core-image-lsb                |    4 +-
 .../scenario/qemuppc/core-image-sato               |    4 +-
 .../scenario/qemuppc/core-image-sato-sdk           |    4 +-
 .../scenario/qemux86-64/core-image-lsb             |    4 +-
 .../scenario/qemux86-64/core-image-sato            |    4 +-
 .../scenario/qemux86-64/core-image-sato-sdk        |    4 +-
 .../scenario/qemux86/core-image-lsb                |    4 +-
 .../scenario/qemux86/core-image-sato               |    4 +-
 .../scenario/qemux86/core-image-sato-sdk           |    4 +-
 .../tools/{zypper_test.sh => smart_test.sh}        |   22 +-
 28 files changed, 635 insertions(+), 626 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-debug-platform.patch
 rename scripts/qemuimage-tests/sanity/{zypper_help => smart_help} (52%)
 rename scripts/qemuimage-tests/sanity/{zypper_search => smart_query} (50%)
 rename scripts/qemuimage-tests/tools/{zypper_test.sh => smart_test.sh} (46%)
Signature · 1 line
1.7.10.4
#1 lore raw

PATCH v3 1/13 rootfs_rpm.bbclass: Revert to default shell logging behavior

From: Mark Hatle <[email protected]>

Remove the set +x, and revert to the normal logging behavior.

---
 meta/classes/rootfs_rpm.bbclass |    2 --
 1 file changed, 2 deletions(-)
meta/classes/rootfs_rpm.bbclass +0 -2
index c4fb470..2bb1b08 100644
@@ -40,8 +40,6 @@ RPM="rpm ${RPMOPTS}"
 do_rootfs[lockfiles] += "${DEPLOY_DIR_RPM}/rpm.lock"
 
 fakeroot rootfs_rpm_do_rootfs () {
-	set +x
-
 	${RPM_PREPROCESS_COMMANDS}
 
 	#createrepo "${DEPLOY_DIR_RPM}"
Signature · 1 line
1.7.10.4
#2 lore raw

PATCH v3 2/13 classes/*_rpm: integrate Smart into RPM filesystem construction

In reply to Paul Eggleton

Use Smart to construct the root filesystem for images and the contents of SDKs rather than the custom scripts around rpm we had previously. This ensures the result when producing an updated image will be the same as upgrading to the same package versions from an older image on the target, as well as allowing us to remove a substantial amount of code making the rpm classes much easier to follow.

Some bugfixes from Bogdan Marinescu <[email protected]>. SDK implementation and testing as well as a number of bugfixes from Mark Hatle <[email protected]>.

---
 meta/classes/package_rpm.bbclass      |  531 ++++++---------------------------
 meta/classes/populate_sdk_rpm.bbclass |   51 +---
 meta/classes/rootfs_rpm.bbclass       |   34 ++-
 3 files changed, 121 insertions(+), 495 deletions(-)
meta/classes/package_rpm.bbclass +90 -441
index 293c52c..4b81b68 100644
@@ -16,88 +16,27 @@ python package_rpm_install () {
     bb.fatal("package_rpm_install not implemented!")
 }
 
-RPMCONF_TARGET_BASE = "${DEPLOY_DIR_RPM}/solvedb"
-RPMCONF_HOST_BASE = "${DEPLOY_DIR_RPM}/solvedb-sdk"
 #
-# Update the Packages depsolver db in ${DEPLOY_DIR_RPM}
+# Update the packages indexes ${DEPLOY_DIR_RPM}
 #
 package_update_index_rpm () {
 	if [ ! -z "${DEPLOY_KEEP_PACKAGES}" -o ! -e "${DEPLOY_DIR_RPM}" ]; then
 		return
 	fi
 
-	# Update target packages
 	base_archs="`echo ${PACKAGE_ARCHS} | sed 's/-/_/g'`"
 	ml_archs="`echo ${MULTILIB_PACKAGE_ARCHS} | sed 's/-/_/g'`"
-	package_update_index_rpm_common "${RPMCONF_TARGET_BASE}" base_archs ml_archs
+	sdk_archs="`echo ${SDK_PACKAGE_ARCHS} | sed 's/-/_/g'`"
 
-	# Update SDK packages
-	base_archs="`echo ${SDK_PACKAGE_ARCHS} | sed 's/-/_/g'`"
-	package_update_index_rpm_common "${RPMCONF_HOST_BASE}" base_archs
-}
-
-package_update_index_rpm_common () {
-	rpmconf_base="$1"
-	shift
-
-        createdirs=""
-	for archvar in "$@"; do
-		eval archs=\${${archvar}}
-		packagedirs=""
-		for arch in $archs; do
-			packagedirs="${DEPLOY_DIR_RPM}/$arch $packagedirs"
-			rm -rf ${DEPLOY_DIR_RPM}/$arch/solvedb.done
-		done
-
-		cat /dev/null > ${rpmconf_base}-${archvar}.conf
-		for pkgdir in $packagedirs; do
-			if [ -e $pkgdir/ ]; then
-				echo "Generating solve db for $pkgdir..."
-				echo $pkgdir/solvedb >> ${rpmconf_base}-${archvar}.conf
-                                createdirs="$createdirs $pkgdir"
-			fi
-		done
-	done
-	rpm-createsolvedb.py "${RPM}" $createdirs
-}
-
-#
-# Generate an rpm configuration suitable for use against the
-# generated depsolver db's...
-#
-package_generate_rpm_conf () {
-	# Update target packages
-	package_generate_rpm_conf_common "${RPMCONF_TARGET_BASE}" base_archs ml_archs
-
-	# Update SDK packages
-	package_generate_rpm_conf_common "${RPMCONF_HOST_BASE}" base_archs
-}
-
-package_generate_rpm_conf_common() {
-	rpmconf_base="$1"
-	shift
+	archs=`for arch in $base_archs $ml_archs $sdk_archs ; do
+		echo $arch
+	done | sort | uniq`
 
-	printf "_solve_dbpath " > ${rpmconf_base}.macro
-	o_colon="false"
-
-	for archvar in "$@"; do
-		printf "_solve_dbpath " > ${rpmconf_base}-${archvar}.macro
-		colon="false"
-		for each in `cat ${rpmconf_base}-${archvar}.conf` ; do
-			if [ "$o_colon" = "true" ]; then
-				printf ":" >> ${rpmconf_base}.macro
-			fi
-			if [ "$colon" = "true" ]; then
-				printf ":" >> ${rpmconf_base}-${archvar}.macro
-			fi
-			printf "%s" $each >> ${rpmconf_base}.macro
-			o_colon="true"
-			printf "%s" $each >> ${rpmconf_base}-${archvar}.macro
-			colon="true"
-		done
-		printf "\n" >> ${rpmconf_base}-${archvar}.macro
+	for arch in $archs; do
+		if [ -d ${DEPLOY_DIR_RPM}/$arch ] ; then
+			createrepo --update -q ${DEPLOY_DIR_RPM}/$arch
+		fi
 	done
-	printf "\n" >> ${rpmconf_base}.macro
 }
 
 rpm_log_check() {
@@ -122,150 +61,23 @@ rpm_log_check() {
 
 
 #
-# Resolve package names to filepaths
-# resolve_pacakge <pkgname> <solvdb conffile>
-#
-resolve_package_rpm () {
-	local conffile="$1"
-	shift
-	local pkg_name=""
-	for solve in `cat ${conffile}`; do
-		pkg_name=$(${RPM} -D "_dbpath $solve" -D "__dbi_txn create nofsync" -q --qf "%{packageorigin}\n" "$@" | grep -v "is not installed" || true)
-		if [ -n "$pkg_name" -a "$pkg_name" != "(none)" ]; then
-			echo $pkg_name
-			break;
-		fi
-	done
-}
-
-# rpm common command and options
-rpm_common_comand () {
-
-    local target_rootfs="${INSTALL_ROOTFS_RPM}"
-
-    ${RPM} --root ${target_rootfs} \
-        --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
-        --predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-        -D "_var ${localstatedir}" \
-        -D "_dbpath ${rpmlibdir}" \
-        -D "_tmppath /install/tmp" \
-        --noparentdirs --nolinktos \
-        -D "__dbi_txn create nofsync private" \
-        -D "_cross_scriptlet_wrapper ${WORKDIR}/scriptlet_wrapper" $@
-}
-
-# install or remove the pkg
-rpm_update_pkg () {
-
-    manifest=$1
-    # The manifest filename, e.g. total_solution.manifest
-    m_name=${manifest##/*/}
-    local target_rootfs="${INSTALL_ROOTFS_RPM}"
-    installdir=$target_rootfs/install
-    pre_btmanifest=$installdir/pre_bt.manifest
-    cur_btmanifest=$installdir/cur_bt.manifest
-
-    # Install/remove the different pkgs when total_solution.manifest is
-    # comming and incremental image generation is enabled.
-    if [ "${INC_RPM_IMAGE_GEN}" = "1" -a -d "${target_rootfs}${rpmlibdir}" \
-        -a "$m_name" = "total_solution.manifest" \
-        -a "${INSTALL_COMPLEMENTARY_RPM}" != "1" ]; then
-        # Get the previous installed list
-        rpm --root $target_rootfs --dbpath ${rpmlibdir} \
-            -qa --qf '%{PACKAGEORIGIN} %{BUILDTIME}\n' | sort -u -o $pre_btmanifest
-        # Get the current installed list (based on install/var/lib/rpm)
-        rpm --root $installdir -D "_dbpath $installdir" \
-            -qa --qf '%{PACKAGEORIGIN} %{BUILDTIME}\n' | sort -u -o $cur_btmanifest
-        comm -1 -3 $cur_btmanifest $pre_btmanifest | sed 's#.*/\(.*\)\.rpm .*#\1#' > \
-            $installdir/remove.manifest
-        comm -2 -3 $cur_btmanifest $pre_btmanifest | awk '{print $1}' > \
-            $installdir/incremental.manifest
-
-        # Attempt to remove unwanted pkgs, the scripts(pre, post, etc.) has not
-        # been run by now, so don't have to run them(preun, postun, etc.) when
-        # erase the pkg
-        if [ -s $installdir/remove.manifest ]; then
-            rpm_common_comand --noscripts --nodeps \
-                -e `cat $installdir/remove.manifest`
-        fi
-
-        # Attempt to install the incremental pkgs
-        if [ -s $installdir/incremental.manifest ]; then
-            rpm_common_comand --replacefiles --replacepkgs \
-               -Uvh $installdir/incremental.manifest
-        fi
-    else
-        # Attempt to install
-        rpm_common_comand --replacepkgs -Uhv $manifest
-    fi
-}
-
-process_pkg_list_rpm() {
-	local insttype=$1
-	shift
-	# $@ is special POSIX linear array can not be assigned
-	# to a local variable directly in dash since its separated by
-	# space and dash expands it before assignment
-	# and local x=1 2 3 and not x="1 2 3"
-	local pkgs
-	pkgs="$@"
-	local confbase=${INSTALL_CONFBASE_RPM}
-
-	printf "" > ${target_rootfs}/install/base_archs.pkglist
-	printf "" > ${target_rootfs}/install/ml_archs.pkglist
-
-	for pkg in $pkgs; do
-		echo "Processing $pkg..."
-
-		archvar=base_archs
-		ml_pkg=$pkg
-		for i in ${MULTILIB_PREFIX_LIST} ; do
-				subst=${pkg#${i}-}
-				if [ $subst != $pkg ] ; then
-						ml_pkg=$subst
-						archvar=ml_archs
-						break
-				fi
-		done
-
-		echo $ml_pkg >> ${target_rootfs}/install/$archvar.pkglist
-	done
-
-	local manifestpfx="install"
-	local extraopt=""
-	if [ "$insttype" = "attemptonly" ] ; then
-		manifestpfx="install_attemptonly"
-		extraopt="-i"
-	fi
-
-	rpmresolve $extraopt ${confbase}-base_archs.conf ${target_rootfs}/install/base_archs.pkglist -o ${target_rootfs}/install/${manifestpfx}.manifest
-	if [ -s ${target_rootfs}/install/ml_archs.pkglist ] ; then
-		rpmresolve $extraopt ${confbase}-ml_archs.conf ${target_rootfs}/install/ml_archs.pkglist -o ${target_rootfs}/install/${manifestpfx}_multilib.manifest
-	fi
-}
-
-#
 # Install a bunch of packages using rpm.
-# There are 3 solutions in an image's FRESH generation:
-# 1) initial_solution
-# 2) total_solution
-# 3) COMPLEMENTARY solution
+# There are two solutions in an image's FRESH generation:
+# 1) main package solution
+# 2) complementary solution
 #
-# It is different when incremental image generation is enabled in the
-# SECOND generation:
-# 1) The initial_solution is skipped.
-# 2) The incremental image generation takes action during the total_solution
-#    installation, the previous installed COMPLEMENTARY pkgs usually would be
-#    removed here, the new COMPLEMENTARY ones would be installed in the next
-#    step.
-# 3) The COMPLEMENTARY would always be installed since it is
-#    generated based on the second step's image.
+# It is different when incremental image generation is enabled:
+# 1) The incremental image generation takes action during the main package
+#    installation, the previous installed complementary packages would
+#    usually be removed here, and the new complementary ones would be
+#    installed in the next step.
+# 2) The complementary would always be installed since it is
+#    generated based on the first step's image.
 #
 # the following shell variables needs to be set before calling this func:
 # INSTALL_ROOTFS_RPM - install root dir
 # INSTALL_PLATFORM_RPM - main platform
 # INSTALL_PLATFORM_EXTRA_RPM - extra platform
-# INSTALL_CONFBASE_RPM - configuration file base name
 # INSTALL_PACKAGES_RPM - packages to be installed
 # INSTALL_PACKAGES_ATTEMPTONLY_RPM - packages attemped to be installed only
 # INSTALL_PACKAGES_LINGUAS_RPM - additional packages for uclibc
@@ -275,15 +87,20 @@ process_pkg_list_rpm() {
 
 package_install_internal_rpm () {
 
-	local target_rootfs="${INSTALL_ROOTFS_RPM}"
-	local platform="`echo ${INSTALL_PLATFORM_RPM} | sed 's#-#_#g'`"
-	local platform_extra="`echo ${INSTALL_PLATFORM_EXTRA_RPM} | sed 's#-#_#g'`"
-	local confbase="${INSTALL_CONFBASE_RPM}"
-	local package_to_install="${INSTALL_PACKAGES_RPM}"
-	local package_attemptonly="${INSTALL_PACKAGES_ATTEMPTONLY_RPM}"
-	local package_linguas="${INSTALL_PACKAGES_LINGUAS_RPM}"
-	local providename="${INSTALL_PROVIDENAME_RPM}"
-	local task="${INSTALL_TASK_RPM}"
+	local target_rootfs="$INSTALL_ROOTFS_RPM"
+	local platform="`echo $INSTALL_PLATFORM_RPM | sed 's#-#_#g'`"
+	local platform_extra="`echo $INSTALL_PLATFORM_EXTRA_RPM | sed 's#-#_#g'`"
+	local package_to_install="$INSTALL_PACKAGES_RPM"
+	local package_attemptonly="$INSTALL_PACKAGES_ATTEMPTONLY_RPM"
+	local package_linguas="$INSTALL_PACKAGES_LINGUAS_RPM"
+	local providename="$INSTALL_PROVIDENAME_RPM"
+	local task="$INSTALL_TASK_RPM"
+
+	# Configure internal RPM environment when using Smart
+	export RPM_ETCRPM=${target_rootfs}/etc/rpm
+
+	# Setup temporary directory -- install...
+	mkdir -p ${target_rootfs}/install
 
 	if [ "${INSTALL_COMPLEMENTARY_RPM}" != "1" ] ; then
 		# Setup base system configuration
@@ -306,166 +123,74 @@ package_install_internal_rpm () {
 		# Tell RPM that the "/" directory exist and is available
 		mkdir -p ${target_rootfs}/etc/rpm/sysinfo
 		echo "/" >${target_rootfs}/etc/rpm/sysinfo/Dirnames
+
 		if [ ! -z "$providename" ]; then
 			cat /dev/null > ${target_rootfs}/etc/rpm/sysinfo/Providename
 			for provide in $providename ; do
 				echo $provide >> ${target_rootfs}/etc/rpm/sysinfo/Providename
 			done
 		fi
-	else
-		# We may run through the complementary installs multiple times.  For each time
-		# we should add the previous solution manifest to the full "original" set to
-		# avoid duplicate install steps.
-		echo "Update original solution..."
-		for m in ${target_rootfs}/install/initial_solution.manifest \
-			${target_rootfs}/install/total_solution.manifest; do
-			if [ -s $m ]; then
-				cat $m >> ${target_rootfs}/install/original_solution.manifest
-				rm -f $m
-			fi
-		done
-		sort -u ${target_rootfs}/install/original_solution.manifest -o ${target_rootfs}/install/original_solution.manifest.new
-		mv ${target_rootfs}/install/original_solution.manifest.new ${target_rootfs}/install/original_solution.manifest
-	fi
 
-	# Setup manifest of packages to install...
-	mkdir -p ${target_rootfs}/install
-	rm -f ${target_rootfs}/install/install.manifest
-	rm -f ${target_rootfs}/install/install_multilib.manifest
-	rm -f ${target_rootfs}/install/install_attemptonly.manifest
+		# Configure RPM... we enforce these settings!
+		mkdir -p ${target_rootfs}${rpmlibdir}
+		mkdir -p ${target_rootfs}${rpmlibdir}/log
+		# After change the __db.* cache size, log file will not be generated automatically,
+		# that will raise some warnings, so touch a bare log for rpm write into it.
+		touch ${target_rootfs}${rpmlibdir}/log/log.0000000001
+		if [ ! -e ${target_rootfs}${rpmlibdir}/DB_CONFIG ]; then
+			cat > ${target_rootfs}${rpmlibdir}/DB_CONFIG << EOF
+# ================ Environment
+set_data_dir .
+set_create_dir .
+set_lg_dir ./log
+set_tmp_dir ./tmp
+set_flags db_log_autoremove on
 
-	# Uclibc builds don't provide this stuff...
-	if [ x${TARGET_OS} = "xlinux" ] || [ x${TARGET_OS} = "xlinux-gnueabi" ] ; then
-		if [ ! -z "${package_linguas}" ]; then
-			process_pkg_list_rpm linguas ${package_linguas}
-		fi
-	fi
+# -- thread_count must be >= 8
+set_thread_count 64
 
-	if [ ! -z "${package_to_install}" ]; then
-		process_pkg_list_rpm default ${package_to_install}
-	fi
+# ================ Logging
 
-	# Normal package installation
-
-	# Generate an install solution by doing a --justdb install, then recreate it with
-	# an actual package install!
-	if [ -s ${target_rootfs}/install/install.manifest ]; then
-		echo "# Install manifest padding" >> ${target_rootfs}/install/install.manifest
-		${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
-			--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-			--root "${target_rootfs}/install" \
-			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}-base_archs.macro`" \
-			-D "__dbi_txn create nofsync" \
-			-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
-			${target_rootfs}/install/install.manifest
-	fi
+# ================ Memory Pool
+set_cachesize 0 1048576 0
+set_mp_mmapsize 268435456
 
-	if [ ! -z "${package_attemptonly}" ]; then
-		echo "Adding attempt only packages..."
-		process_pkg_list_rpm attemptonly ${package_attemptonly}
-		cat ${target_rootfs}/install/install_attemptonly.manifest | while read pkg_name
-		do
-			echo "Attempting $pkg_name..." >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}"
-			${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
-				--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-				--root "${target_rootfs}/install" \
-				-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
-				-D "__dbi_txn create nofsync private" \
-				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
-			$pkg_name >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}" 2>&1 || true
-		done
-	fi
+# ================ Locking
+set_lk_max_locks 16384
+set_lk_max_lockers 16384
+set_lk_max_objects 16384
+ mutex_set_max 163840
 
-	#### Note: 'Recommends' is an arbitrary tag that means _SUGGESTS_ in OE-core..
-	# Add any recommended packages to the image
-	# RPM does not solve for recommended packages because they are optional...
-	# So we query them and tree them like the ATTEMPTONLY packages above...
-	# Change the loop to "1" to run this code...
-	loop=0
-	if [ $loop -eq 1 ]; then
-	 echo "Processing recommended packages..."
-	 cat /dev/null >  ${target_rootfs}/install/recommend.list
-	 while [ $loop -eq 1 ]; do
-		# Dump the full set of recommends...
-		${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
-			--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-			--root "${target_rootfs}/install" \
-			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
-			-D "__dbi_txn create nofsync private" \
-			-qa --qf "[%{RECOMMENDS}\n]" | sort -u > ${target_rootfs}/install/recommend
-		# Did we add more to the list?
-		grep -v -x -F -f ${target_rootfs}/install/recommend.list ${target_rootfs}/install/recommend > ${target_rootfs}/install/recommend.new || true
-		# We don't want to loop unless there is a change to the list!
-		loop=0
-		cat ${target_rootfs}/install/recommend.new | \
-		 while read pkg ; do
-			# Ohh there was a new one, we'll need to loop again...
-			loop=1
-			echo "Processing $pkg..."
-			found=0
-			for archvar in base_archs ml_archs ; do
-				pkg_name=$(resolve_package_rpm ${confbase}-${archvar}.conf ${pkg})
-				if [ -n "$pkg_name" ]; then
-					found=1
-					break
-				fi
-			done
+# ================ Replication
+EOF
+		fi
 
-			if [ $found -eq 0 ]; then
-				echo "Note: Unable to find package $pkg -- suggests"
-				echo "Unable to find package $pkg." >> "`dirname ${BB_LOGFILE}`/log.do_${task}_recommend.${PID}"
-				continue
+		# Create database so that smart doesn't complain (lazy init)
+		rpm --root $target_rootfs --dbpath /var/lib/rpm -qa > /dev/null
+
+		# Configure smart
+		rm -rf ${target_rootfs}/var/lib/smart
+		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-root=${target_rootfs}
+		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-dbpath=${rpmlibdir}
+		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-nolinktos=1
+		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-noparentdirs=1
+		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._var=${localstatedir}
+		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._tmppath=/install/tmp
+		smart --data-dir=${target_rootfs}/var/lib/smart channel --add rpmsys type=rpm-sys -y
+
+		for arch in $platform_extra ; do
+			if [ -d ${DEPLOY_DIR_RPM}/$arch -a ! -e ${target_rootfs}/install/channel.$arch.stamp ] ; then
+				smart --data-dir=${target_rootfs}/var/lib/smart channel --add $arch type=rpm-md type=rpm-md baseurl=${DEPLOY_DIR_RPM}/$arch -y
+				touch ${target_rootfs}/install/channel.$arch.stamp
 			fi
-			echo "Attempting $pkg_name..." >> "`dirname ${BB_LOGFILE}`/log.do_{task}_recommend.${PID}"
-			${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
-				--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-				--root "${target_rootfs}/install" \
-				-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}.macro`" \
-				-D "__dbi_txn create nofsync private" \
-				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
-				$pkg_name >> "`dirname ${BB_LOGFILE}`/log.do_${task}_recommend.${PID}" 2>&1 || true
 		done
-		cat ${target_rootfs}/install/recommend.list ${target_rootfs}/install/recommend.new | sort -u > ${target_rootfs}/install/recommend.new.list
-		mv -f ${target_rootfs}/install/recommend.new.list ${target_rootfs}/install/recommend.list
-		rm ${target_rootfs}/install/recommend ${target_rootfs}/install/recommend.new
-	 done
 	fi
 
-	# Now that we have a solution, pull out a list of what to install...
-	echo "Manifest: ${target_rootfs}/install/install_solution.manifest"
-	${RPM} -D "_dbpath ${target_rootfs}/install" -qa --qf "%{packageorigin}\n" \
-		--root "${target_rootfs}/install" \
-		-D "__dbi_txn create nofsync private" \
-		> ${target_rootfs}/install/install_solution.manifest
-
-	touch ${target_rootfs}/install/install_multilib_solution.manifest
-
-	if [ -s "${target_rootfs}/install/install_multilib.manifest" ]; then
-		# multilib package installation
-		echo "# Install multilib manifest padding" >> ${target_rootfs}/install/install_multilib.manifest
-
-		# Generate an install solution by doing a --justdb install, then recreate it with
-		# an actual package install!
-		${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
-			--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-			--root "${target_rootfs}/install" \
-			-D "_dbpath ${target_rootfs}/install" -D "`cat ${confbase}-ml_archs.macro`" \
-			-D "__dbi_txn create nofsync" \
-			-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
-			${target_rootfs}/install/install_multilib.manifest
-
-		# Now that we have a solution, pull out a list of what to install...
-		echo "Manifest: ${target_rootfs}/install/install_multilib.manifest"
-		${RPM} -D "_dbpath ${target_rootfs}/install" -qa --qf "%{packageorigin}\n" \
-			--root "${target_rootfs}/install" \
-			-D "__dbi_txn create nofsync private" \
-			> ${target_rootfs}/install/install_multilib_solution.manifest
-
+	# Uclibc builds don't provide this stuff...
+	if [ x${TARGET_OS} != "xlinux" ] && [ x${TARGET_OS} != "xlinux-gnueabi" ] ; then
+		package_linguas=""
 	fi
 
-	cat ${target_rootfs}/install/install_solution.manifest \
-	    ${target_rootfs}/install/install_multilib_solution.manifest | sort -u > ${target_rootfs}/install/total_solution.manifest
-
 	# Construct install scriptlet wrapper
 	cat << EOF > ${WORKDIR}/scriptlet_wrapper
 #!/bin/bash
@@ -491,93 +216,17 @@ fi
 EOF
 
 	chmod 0755 ${WORKDIR}/scriptlet_wrapper
+	smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._cross_scriptlet_wrapper=${WORKDIR}/scriptlet_wrapper
 
-	# Configure RPM... we enforce these settings!
-	mkdir -p ${target_rootfs}${rpmlibdir}
-	mkdir -p ${target_rootfs}${rpmlibdir}/log
-	# After change the __db.* cache size, log file will not be generated automatically,
-	# that will raise some warnings, so touch a bare log for rpm write into it.
-	touch ${target_rootfs}${rpmlibdir}/log/log.0000000001
-	cat > ${target_rootfs}${rpmlibdir}/DB_CONFIG << EOF
-# ================ Environment
-set_data_dir .
-set_create_dir .
-set_lg_dir ./log
-set_tmp_dir ./tmp
-set_flags db_log_autoremove on
-
-# -- thread_count must be >= 8
-set_thread_count 64
-
-# ================ Logging
+	smart --data-dir=${target_rootfs}/var/lib/smart install -y ${package_to_install} ${package_linguas}
 
-# ================ Memory Pool
-set_cachesize 0 1048576 0
-set_mp_mmapsize 268435456
-
-# ================ Locking
-set_lk_max_locks 16384
-set_lk_max_lockers 16384
-set_lk_max_objects 16384
-mutex_set_max 163840
-
-# ================ Replication
-EOF
-
-	if [ "${INSTALL_COMPLEMENTARY_RPM}" = "1" ] ; then
-		# Only install packages not already installed (dependency calculation will
-		# almost certainly have added some that have been)
-		sort -u ${target_rootfs}/install/original_solution.manifest > ${target_rootfs}/install/original_solution_sorted.manifest
-		sort -u ${target_rootfs}/install/total_solution.manifest > ${target_rootfs}/install/total_solution_sorted.manifest
-		comm -2 -3 ${target_rootfs}/install/total_solution_sorted.manifest \
-			${target_rootfs}/install/original_solution_sorted.manifest > \
-			${target_rootfs}/install/diff.manifest
-		mv ${target_rootfs}/install/diff.manifest ${target_rootfs}/install/total_solution.manifest
-	elif [ "${INC_RPM_IMAGE_GEN}" = "1" -a -f "${target_rootfs}/etc/passwd" ]; then
-		echo "Skipping pre install due to existing image"
-	else
-		# RPM is special. It can't handle dependencies and preinstall scripts correctly. Its
-		# probably a feature. The only way to convince rpm to actually run the preinstall scripts
-		# for base-passwd and shadow first before installing packages that depend on these packages
-		# is to do two image installs, installing one set of packages, then the other.
-		rm -f ${target_rootfs}/install/initial_install.manifest
-		echo "Installing base dependencies first (base-passwd, base-files and shadow) since rpm is special"
-		grep /base-passwd-[0-9] ${target_rootfs}/install/total_solution.manifest >> ${target_rootfs}/install/initial_install.manifest || true
-		grep /base-files-[0-9] ${target_rootfs}/install/total_solution.manifest >> ${target_rootfs}/install/initial_install.manifest || true
-		grep /shadow-[0-9] ${target_rootfs}/install/total_solution.manifest >> ${target_rootfs}/install/initial_install.manifest || true
-
-		if [ -s ${target_rootfs}/install/initial_install.manifest ]; then
-			echo "# Initial Install manifest padding..." >> ${target_rootfs}/install/initial_install.manifest
-
-			# Generate an install solution by doing a --justdb install, then recreate it with
-			# an actual package install!
-			mkdir -p ${target_rootfs}/initial
-
-			${RPM} --predefine "_rpmds_sysinfo_path ${target_rootfs}/etc/rpm/sysinfo" \
-				--predefine "_rpmrc_platform_path ${target_rootfs}/etc/rpm/platform" \
-				--root "${target_rootfs}/install" \
-				-D "_dbpath ${target_rootfs}/initial" -D "`cat ${confbase}.macro`" \
-				-D "__dbi_txn create nofsync" \
-				-U --justdb --replacepkgs --noscripts --notriggers --noparentdirs --nolinktos --ignoresize \
-				${target_rootfs}/install/initial_install.manifest
-
-			${RPM} -D "_dbpath ${target_rootfs}/initial" -qa --qf "%{packageorigin}\n" \
-				-D "__dbi_txn create nofsync private" \
-				--root "${target_rootfs}/install" \
-				> ${target_rootfs}/install/initial_solution.manifest
-
-			rpm_update_pkg ${target_rootfs}/install/initial_solution.manifest
-		
-			grep -Fv -f ${target_rootfs}/install/initial_solution.manifest ${target_rootfs}/install/total_solution.manifest > ${target_rootfs}/install/total_solution.manifest.new
-			mv ${target_rootfs}/install/total_solution.manifest.new ${target_rootfs}/install/total_solution.manifest
-		
-			rm -rf ${target_rootfs}/initial
-		fi
+	if [ ! -z "${package_attemptonly}" ]; then
+		echo "Installing attempt only packages..."
+		for pkg_name in ${package_attemptonly} ; do
+			echo "Attempting $pkg_name..." >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}"
+			smart --data-dir=${target_rootfs}/var/lib/smart install -y $pkg_name >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}" 2>&1 || true
+		done
 	fi
-
-	echo "Installing main solution manifest (${target_rootfs}/install/total_solution.manifest)"
-
-	rpm_update_pkg ${target_rootfs}/install/total_solution.manifest
 }
 
 python write_specfile () {
meta/classes/populate_sdk_rpm.bbclass +13 -38

99 lines not on this page · raw message

meta/classes/rootfs_rpm.bbclass +18 -16
index 2bb1b08..d0b0d57 100644
@@ -7,11 +7,15 @@ ROOTFS_PKGMANAGE = "rpm zypper"
 # Add 50Meg of extra space for zypper database space
 IMAGE_ROOTFS_EXTRA_SPACE_append = "${@base_contains("PACKAGE_INSTALL", "zypper", " + 51200", "" ,d)}"
 
+# Smart is python based, so be sure python-native is available to us.
+EXTRANATIVEPATH += "python-native"
+
 # Postinstalls on device are handled within this class at present
 ROOTFS_PKGMANAGE_BOOTSTRAP = ""
 
 do_rootfs[depends] += "rpm-native:do_populate_sysroot"
 do_rootfs[depends] += "rpmresolve-native:do_populate_sysroot"
+do_rootfs[depends] += "python-smartpm-native:do_populate_sysroot"
 
 # Needed for update-alternatives
 do_rootfs[depends] += "opkg-native:do_populate_sysroot"
@@ -21,8 +25,8 @@ do_rootfs[depends] += "createrepo-native:do_populate_sysroot"
 
 do_rootfs[recrdeptask] += "do_package_write_rpm"
 
-RPM_PREPROCESS_COMMANDS = "package_update_index_rpm; package_generate_rpm_conf; "
-RPM_POSTPROCESS_COMMANDS = ""
+RPM_PREPROCESS_COMMANDS = "package_update_index_rpm; "
+RPM_POSTPROCESS_COMMANDS = "rpm_setup_smart_target_config; "
 
 # 
 # Allow distributions to alter when [postponed] package install scripts are run
@@ -32,7 +36,7 @@ POSTINSTALL_INITPOSITION ?= "98"
 rpmlibdir = "/var/lib/rpm"
 opkglibdir = "${localstatedir}/lib/opkg"
 
-RPMOPTS="--dbpath ${rpmlibdir} --define='_openall_before_chroot 1'"
+RPMOPTS="--dbpath ${rpmlibdir}"
 RPM="rpm ${RPMOPTS}"
 
 # RPM doesn't work with multiple rootfs generation at once due to collisions in the use of files 
@@ -42,13 +46,10 @@ do_rootfs[lockfiles] += "${DEPLOY_DIR_RPM}/rpm.lock"
 fakeroot rootfs_rpm_do_rootfs () {
 	${RPM_PREPROCESS_COMMANDS}
 
-	#createrepo "${DEPLOY_DIR_RPM}"
-
 	# install packages
 	# This needs to work in the same way as populate_sdk_rpm.bbclass!
 	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
 	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
-	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
 	export INSTALL_PACKAGES_RPM="${PACKAGE_INSTALL}"
 	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
 	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
@@ -114,18 +115,10 @@ EOF
 	# remove lock files
 	rm -f ${IMAGE_ROOTFS}${rpmlibdir}/__db.*
 
-	# Move manifests into the directory with the logs
-	mv ${IMAGE_ROOTFS}/install/*.manifest ${T}/
-
 	# Remove all remaining resolver files
 	rm -rf ${IMAGE_ROOTFS}/install
 
 	log_check rootfs
-
-	# Workaround so the parser knows we need the resolve_package function!
-	if false ; then
-		resolve_package_rpm foo ${RPMCONF_TARGET_BASE}.conf || true
-	fi
 }
 
 remove_packaging_data_files() {
@@ -135,10 +128,19 @@ remove_packaging_data_files() {
 	mkdir -p $t
 	mv ${IMAGE_ROOTFS}${rpmlibdir} $t
 	rm -rf ${IMAGE_ROOTFS}${opkglibdir}
+	rm -rf ${IMAGE_ROOTFS}/var/lib/smart
+}
+
+rpm_setup_smart_target_config() {
+	# Set up smart configuration for the target
+	rm -rf ${IMAGE_ROOTFS}/var/lib/smart
+	smart --data-dir=${IMAGE_ROOTFS}/var/lib/smart channel --add rpmsys type=rpm-sys -y
+	smart --data-dir=${IMAGE_ROOTFS}/var/lib/smart config --set rpm-nolinktos=1
+	smart --data-dir=${IMAGE_ROOTFS}/var/lib/smart config --set rpm-noparentdirs=1
+	rm -f ${IMAGE_ROOTFS}/var/lib/smart/config.old
 }
 
-RPM_QUERY_CMD = '${RPM} --root $INSTALL_ROOTFS_RPM -D "_dbpath ${rpmlibdir}" \
-		-D "__dbi_txn create nofsync private"'
+RPM_QUERY_CMD = '${RPM} --root $INSTALL_ROOTFS_RPM -D "_dbpath ${rpmlibdir}"'
 
 list_installed_packages() {
 	GET_LIST=$(${RPM_QUERY_CMD} -qa --qf "[%{NAME} %{ARCH} %{PACKAGEORIGIN} %{Platform}\n]")
Signature · 1 line
1.7.10.4
#3 lore raw

PATCH v3 3/13 classes/rootfs_rpm: install smart instead of zypper in rpm-based images

In reply to Paul Eggleton

Switch over to smart on the target when package-management is enabled.

---
 meta/classes/rootfs_rpm.bbclass |    6 +++---
 1 file changed, 3 insertions(+), 3 deletions(-)
meta/classes/rootfs_rpm.bbclass +3 -3
index d0b0d57..a507ad6 100644
@@ -2,10 +2,10 @@
 # Creates a root filesystem out of rpm packages
 #
 
-ROOTFS_PKGMANAGE = "rpm zypper"
+ROOTFS_PKGMANAGE = "rpm smartpm"
 
-# Add 50Meg of extra space for zypper database space
-IMAGE_ROOTFS_EXTRA_SPACE_append = "${@base_contains("PACKAGE_INSTALL", "zypper", " + 51200", "" ,d)}"
+# Add 50Meg of extra space for Smart
+IMAGE_ROOTFS_EXTRA_SPACE_append = "${@base_contains("PACKAGE_INSTALL", "smartpm", " + 51200", "" ,d)}"
 
 # Smart is python based, so be sure python-native is available to us.
 EXTRANATIVEPATH += "python-native"
Signature · 1 line
1.7.10.4
#4 lore raw

PATCH v3 4/13 Update sanity tests to use smart instead of zypper

In reply to Paul Eggleton
---
 meta/classes/imagetest-qemu.bbclass                |    6 ++---
 .../sanity/{zypper_help => smart_help}             |   24 ++++++++++----------
 .../sanity/{zypper_search => smart_query}          |   24 ++++++++++----------
 .../scenario/qemuarm/core-image-lsb                |    4 ++--
 .../scenario/qemuarm/core-image-sato               |    4 ++--
 .../scenario/qemuarm/core-image-sato-sdk           |    4 ++--
 .../scenario/qemumips/core-image-lsb               |    4 ++--
 .../scenario/qemumips/core-image-sato              |    4 ++--
 .../scenario/qemumips/core-image-sato-sdk          |    4 ++--
 .../scenario/qemuppc/core-image-lsb                |    4 ++--
 .../scenario/qemuppc/core-image-sato               |    4 ++--
 .../scenario/qemuppc/core-image-sato-sdk           |    4 ++--
 .../scenario/qemux86-64/core-image-lsb             |    4 ++--
 .../scenario/qemux86-64/core-image-sato            |    4 ++--
 .../scenario/qemux86-64/core-image-sato-sdk        |    4 ++--
 .../scenario/qemux86/core-image-lsb                |    4 ++--
 .../scenario/qemux86/core-image-sato               |    4 ++--
 .../scenario/qemux86/core-image-sato-sdk           |    4 ++--
 .../tools/{zypper_test.sh => smart_test.sh}        |   22 +++++++++---------
 19 files changed, 68 insertions(+), 68 deletions(-)
 rename scripts/qemuimage-tests/sanity/{zypper_help => smart_help} (52%)
 rename scripts/qemuimage-tests/sanity/{zypper_search => smart_query} (50%)
 rename scripts/qemuimage-tests/tools/{zypper_test.sh => smart_test.sh} (46%)
meta/classes/imagetest-qemu.bbclass +3 -3
index ce20164..06ba3e3 100644
@@ -86,10 +86,10 @@ def qemuimagetest_main(d):
         for test in list:
             (scen, case, fullpath) = test
 
-            """Skip rpm/zypper if package_rpm not set for PACKAGE_CLASSES"""
-            if case.find("zypper") != -1 or case.find("rpm") != -1:
+            """Skip rpm/smart if package_rpm not set for PACKAGE_CLASSES"""
+            if case.find("smart") != -1 or case.find("rpm") != -1:
                 if d.getVar("PACKAGE_CLASSES", True).find("rpm", 0, 11) == -1:
-                    bb.note("skip rpm/zypper cases since package_rpm not set in PACKAGE_CLASSES")
+                    bb.note("skip rpm/smart cases since package_rpm not set in PACKAGE_CLASSES")
                     continue
                 else:
                     final_list.append((scen, case, fullpath))
scripts/qemuimage-tests/sanity/smart_help renamed from scripts/qemuimage-tests/sanity/zypper_help +12 -12
similarity index 52%
rename from scripts/qemuimage-tests/sanity/zypper_help
rename to scripts/qemuimage-tests/sanity/smart_help
index 94f82c4..0eeac26 100755
@@ -1,7 +1,7 @@
 #!/bin/bash
-# Zypper Check Test Case for Sanity Test
+# Smart Check Test Case for Sanity Test
 # The case boot up the Qemu target with `runqemu qemuxxx`.
-# Then check if zypper command can work in target.
+# Then check if smart command can work in target.
 #
 # Author: Jiajun Xu <[email protected]>
 #
@@ -26,13 +26,13 @@ else
 	RET=1
 fi
 
-# Check if zypper help can work in target
-if [ $RET -eq 0 -a -f $TOOLS/zypper_test.sh ]; then
-	# Copy zypper_test.sh into target
-	Test_Target_Pre ${TARGET_IPADDR} $TOOLS/zypper_test.sh
+# Check if smart --help can work in target
+if [ $RET -eq 0 -a -f $TOOLS/smart_test.sh ]; then
+	# Copy smart_test.sh into target
+	Test_Target_Pre ${TARGET_IPADDR} $TOOLS/smart_test.sh
 	if [ $? -eq 0 ]; then
-		# Run zypper_test.sh to check if zypper help can work in target
-		Test_SSH ${TARGET_IPADDR} "sh $TARGET_TEST_DIR/zypper_test.sh help"
+		# Run smart_test.sh to check if smart --help can work in target
+		Test_SSH ${TARGET_IPADDR} "sh $TARGET_TEST_DIR/smart_test.sh --help"
 		RET=$?
 	else
 		RET=1
@@ -40,13 +40,13 @@ if [ $RET -eq 0 -a -f $TOOLS/zypper_test.sh ]; then
 fi
 
 if [ ${RET} -eq 0 ]; then
-	Test_Info "zypper help Test PASS"
+	Test_Info "smart --help Test PASS"
 	Test_Kill_Qemu
-	Test_Print_Result "zypper_help" 0
+	Test_Print_Result "smart_help" 0
 	exit 0
 else
-	Test_Info "zypper help FAIL, Pls. check above error log"
+	Test_Info "smart --help FAIL, Pls. check above error log"
 	Test_Kill_Qemu
-	Test_Print_Result "zypper_help" 1
+	Test_Print_Result "smart_help" 1
 	exit 1
 fi
scripts/qemuimage-tests/sanity/smart_query renamed from scripts/qemuimage-tests/sanity/zypper_search +12 -12
similarity index 50%
rename from scripts/qemuimage-tests/sanity/zypper_search
rename to scripts/qemuimage-tests/sanity/smart_query
index e687ead..779ee63 100755
@@ -1,7 +1,7 @@
 #!/bin/bash
-# Zypper Check Test Case for Sanity Test
+# Smart Check Test Case for Sanity Test
 # The case boot up the Qemu target with `runqemu qemuxxx`.
-# Then check if zypper command can work in target.
+# Then check if smart command can work in target.
 #
 # Author: Jiajun Xu <[email protected]>
 #
@@ -26,13 +26,13 @@ else
 	RET=1
 fi
 
-# Check if zypper search can work in target
-if [ $RET -eq 0 -a -f $TOOLS/zypper_test.sh ]; then
-	# Copy zypper_test.sh into target
-	Test_Target_Pre ${TARGET_IPADDR} $TOOLS/zypper_test.sh
+# Check if smart query can work in target
+if [ $RET -eq 0 -a -f $TOOLS/smart_test.sh ]; then
+	# Copy smart_test.sh into target
+	Test_Target_Pre ${TARGET_IPADDR} $TOOLS/smart_test.sh
 	if [ $? -eq 0 ]; then
-		# Run zypper_test.sh to check if zypper search can work in target
-		Test_SSH ${TARGET_IPADDR} "sh $TARGET_TEST_DIR/zypper_test.sh search avahi"
+		# Run smart_test.sh to check if smart query can work in target
+		Test_SSH ${TARGET_IPADDR} "sh $TARGET_TEST_DIR/smart_test.sh query avahi*"
 		RET=$?
 	else
 		RET=1
@@ -40,13 +40,13 @@ if [ $RET -eq 0 -a -f $TOOLS/zypper_test.sh ]; then
 fi
 
 if [ ${RET} -eq 0 ]; then
-	Test_Info "zypper search package avahi Test PASS"
+	Test_Info "smart query package avahi Test PASS"
 	Test_Kill_Qemu
-	Test_Print_Result "zypper_search" 0
+	Test_Print_Result "smart_query" 0
 	exit 0
 else
-	Test_Info "zypper search package avahi FAIL, Pls. check above error log"
+	Test_Info "smart query package avahi FAIL, Pls. check above error log"
 	Test_Kill_Qemu
-	Test_Print_Result "zypper_search" 1
+	Test_Print_Result "smart_query" 1
 	exit 1
 fi
scripts/qemuimage-tests/scenario/qemuarm/core-image-lsb +2 -2
index 4fa6068..c0fff0e 100644
@@ -1,7 +1,7 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity shutdown
scripts/qemuimage-tests/scenario/qemuarm/core-image-sato +2 -2
index 0e79703..1ddcab4 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity connman
 sanity shutdown
scripts/qemuimage-tests/scenario/qemuarm/core-image-sato-sdk +2 -2
index 726842c..50d7ad0 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity compiler
 sanity connman
scripts/qemuimage-tests/scenario/qemumips/core-image-lsb +2 -2
index 4fa6068..c0fff0e 100644
@@ -1,7 +1,7 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity shutdown
scripts/qemuimage-tests/scenario/qemumips/core-image-sato +2 -2
index 0e79703..1ddcab4 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity connman
 sanity shutdown
scripts/qemuimage-tests/scenario/qemumips/core-image-sato-sdk +2 -2
index 726842c..50d7ad0 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity compiler
 sanity connman
scripts/qemuimage-tests/scenario/qemuppc/core-image-lsb +2 -2
index 4fa6068..c0fff0e 100644
@@ -1,7 +1,7 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity shutdown
scripts/qemuimage-tests/scenario/qemuppc/core-image-sato +2 -2
index 0e79703..1ddcab4 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity connman
 sanity shutdown
scripts/qemuimage-tests/scenario/qemuppc/core-image-sato-sdk +2 -2
index 726842c..50d7ad0 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity compiler
 sanity connman
scripts/qemuimage-tests/scenario/qemux86-64/core-image-lsb +2 -2
index 4fa6068..c0fff0e 100644
@@ -1,7 +1,7 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity shutdown
scripts/qemuimage-tests/scenario/qemux86-64/core-image-sato +2 -2
index 0e79703..1ddcab4 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity connman
 sanity shutdown
scripts/qemuimage-tests/scenario/qemux86-64/core-image-sato-sdk +2 -2
index 726842c..50d7ad0 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity compiler
 sanity connman
scripts/qemuimage-tests/scenario/qemux86/core-image-lsb +2 -2
index 4fa6068..c0fff0e 100644
@@ -1,7 +1,7 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity shutdown
scripts/qemuimage-tests/scenario/qemux86/core-image-sato +2 -2
index 0e79703..1ddcab4 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity connman
 sanity shutdown
scripts/qemuimage-tests/scenario/qemux86/core-image-sato-sdk +2 -2
index 726842c..50d7ad0 100644
@@ -1,8 +1,8 @@
 sanity ssh
 sanity scp
 sanity dmesg
-sanity zypper_help
-sanity zypper_search
+sanity smart_help
+sanity smart_query
 sanity rpm_query
 sanity compiler
 sanity connman
scripts/qemuimage-tests/tools/smart_test.sh renamed from scripts/qemuimage-tests/tools/zypper_test.sh +11 -11
similarity index 46%
rename from scripts/qemuimage-tests/tools/zypper_test.sh
rename to scripts/qemuimage-tests/tools/smart_test.sh
index 5e8e7aa..f278a25 100644
@@ -1,5 +1,5 @@
 #!/bin/bash
-# zypper test script running in target
+# smart test script running in target
 #
 # Author: Jiajun Xu <[email protected]>
 #
@@ -14,32 +14,32 @@ Target_Info()
 
 Target_Err()
 {
-	echo -e "\tTARGET: zypper command has issue when running, Pls. check the error log"
+	echo -e "\tTARGET: smart command has issue when running, Pls. check the error log"
 	echo -e "\tTARGET: ##### Error Log #####"
 	$1
 	echo -e "\tTARGET: #####    End    #####"
 }
 
-which zypper
+which smart
 if [ $? -ne 0 ]; then
-	Target_Info "No zypper command found"
+	Target_Info "No smart command found"
 	exit 1
 fi
 
-if [ zypper > /dev/null 2>&1 ]; then
-	Target_Info "zypper command run without problem"
+if [ smart > /dev/null 2>&1 ]; then
+	Target_Info "smart command run without problem"
 else
-	Target_Err zypper
+	Target_Err smart
 	exit 1
 fi
 
-# run zypper with specific command parsed to zypper_test.sh
-zypper $* > /dev/null 2>&1
+# run smart with specific command parsed to smart_test.sh
+smart $* > /dev/null 2>&1
 
 if [ $? -eq 0 ]; then
-	Target_Info "zypper $* work without problem"
+	Target_Info "smart $* work without problem"
 	exit 0
 else
-	Target_Err zypper $*
+	Target_Err "smart $*"
 	exit 1
 fi
Signature · 1 line
1.7.10.4
#5 lore raw

PATCH v3 5/13 package_rpm: Update the way the multilib package names are translated

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

The variable MULTILIB_PACKAGE_ARCHS has been removed in favor of a repurposed MULTILIB_PREFIX_LIST. The format of this item is now <libid>:<arch>:<arch1>:...:<archN>. This ensures that we can correctly translate the libid to one of the supported archs in a tri-lib system.

All of the users of MULTILIB_PREFIX_LIST and MULTILIB_PACKAGE_ARCHS have been modified accordingly.

Also change the way attempted packages are installed, verify the package exists in the translate functions, then perform the install in one single operation. This results in a significantly faster install time.

---
 meta/classes/package_rpm.bbclass      |  188 ++++++++++++++++++++++++++++++---
 meta/classes/populate_sdk_rpm.bbclass |   28 +++--
 meta/classes/rootfs_rpm.bbclass       |   47 ++++-----
 3 files changed, 216 insertions(+), 47 deletions(-)
meta/classes/package_rpm.bbclass +175 -13
index 4b81b68..4f60daf 100644
@@ -24,11 +24,24 @@ package_update_index_rpm () {
 		return
 	fi
 
-	base_archs="`echo ${PACKAGE_ARCHS} | sed 's/-/_/g'`"
-	ml_archs="`echo ${MULTILIB_PACKAGE_ARCHS} | sed 's/-/_/g'`"
-	sdk_archs="`echo ${SDK_PACKAGE_ARCHS} | sed 's/-/_/g'`"
+	sdk_archs=`echo "${SDK_PACKAGE_ARCHS}" | tr - _`
+
+	target_archs=""
+	for i in ${MULTILIB_PREFIX_LIST} ; do
+		old_IFS="$IFS"
+		IFS=":"
+		set $i
+		IFS="$old_IFS"
+		shift # remove mlib
+		while [ -n "$1" ]; do
+			target_archs="$target_archs $1"
+			shift
+		done
+	done
 
-	archs=`for arch in $base_archs $ml_archs $sdk_archs ; do
+	target_archs=`echo "$target_archs" | tr - _`
+
+	archs=`for arch in $target_archs $sdk_archs ; do
 		echo $arch
 	done | sort | uniq`
 
@@ -59,6 +72,143 @@ rpm_log_check() {
 	true
 }
 
+# Translate the RPM/Smart format names to the OE multilib format names
+# Input via stdin (only the first item per line is converted!)
+# Output via stdout
+translate_smart_to_oe() {
+	arg1="$1"
+
+	# Dump installed packages
+	while read pkg arch other ; do
+		if [ -z "$pkg" ]; then
+			continue
+		fi
+		new_pkg=$pkg
+		fixed_arch=`echo "$arch" | tr _ -`
+		for i in ${MULTILIB_PREFIX_LIST} ; do
+			old_IFS="$IFS"
+			IFS=":"
+			set $i
+			IFS="$old_IFS"
+			mlib="$1"
+			shift
+			while [ -n "$1" ]; do
+				cmp_arch=$1
+				shift
+				if [ "$arch" = "$cmp_arch" -o "$fixed_arch" = "$cmp_arch" ]; then
+					if [ "$mlib" = "default" ]; then
+						new_pkg="$pkg"
+					else
+						new_pkg="$mlib-$pkg"
+					fi
+					break
+				fi
+			done
+			if [ "$arch" = "$cmp_arch" -o "$fixed_arch" = "$cmp_arch" ]; then
+				break
+			fi
+		done
+
+		#echo "$pkg -> $new_pkg" >&2
+		if [ "$arg1" = "arch" ]; then
+			echo $new_pkg $cmp_arch $other
+		else
+			echo $new_pkg $other
+		fi
+	done
+}		
+
+# Translate the OE multilib format names to the RPM/Smart format names
+# Input via arguments
+# Ouput via pkgs_to_install
+translate_oe_to_smart() {
+	default_archs=""
+	sdk_mode=""
+	if [ "$1" = "--sdk" ]; then
+		shift
+		sdk_mode="true"
+		# Need to reverse the order of the SDK_ARCHS highest -> lowest priority
+		archs=`echo "${SDK_PACKAGE_ARCHS}" | tr - _`
+		for arch in $archs ; do
+		        default_archs="$arch $default_archs"
+		done
+	fi
+
+	attemptonly="Error"
+	if [ "$1" = "--attemptonly" ]; then
+		attemptonly="Warning"
+		shift
+	fi
+
+	# Dump a list of all available packages
+	[ ! -e ${target_rootfs}/install/tmp/fullpkglist.query ] && smart --data-dir=${target_rootfs}/var/lib/smart query --output ${target_rootfs}/install/tmp/fullpkglist.query
+
+	pkgs_to_install=""
+	for pkg in "$@" ; do
+		new_pkg="$pkg"
+		if [ -z "$sdk_mode" ]; then
+			for i in ${MULTILIB_PREFIX_LIST} ; do
+				old_IFS="$IFS"
+				IFS=":"
+				set $i
+				IFS="$old_IFS"
+				mlib="$1"
+				shift
+				if [ "$mlib" = "default" ]; then
+					if [ -z "$default_archs" ]; then
+						default_archs=$@
+					fi
+					continue
+				fi
+				subst=${pkg#${mlib}-}
+				if [ "$subst" != "$pkg" ]; then
+					feeds=$@
+					while [ -n "$1" ]; do
+						arch="$1"
+						arch=`echo "$arch" | tr - _`
+						shift
+						if grep -q '^'$subst'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
+							new_pkg="$subst@$arch"
+							# First found is best match
+							break
+						fi
+					done
+					if [ "$pkg" = "$new_pkg" ]; then
+						# Failed to translate, package not found!
+						echo "$attemptonly: $pkg not found in the $mlib feeds ($feeds)." >&2
+						if [ "$attemptonly" = "Error" ]; then
+							exit 1
+						fi
+						continue
+					fi
+				fi
+			done
+		fi
+		# Apparently not a multilib package...
+		if [ "$pkg" = "$new_pkg" ]; then
+			default_archs_fixed=`echo "$default_archs" | tr - _`
+			for arch in $default_archs_fixed ; do
+				if grep -q '^'$pkg'-[^-]*-[^-]*@'$arch'$' ${target_rootfs}/install/tmp/fullpkglist.query ; then
+					new_pkg="$pkg@$arch"
+					# First found is best match
+					break
+				fi
+			done
+			if [ "$pkg" = "$new_pkg" ]; then
+				# Failed to translate, package not found!
+				echo "$attemptonly: $pkg not found in the base feeds ($default_archs)." >&2
+				if [ "$attemptonly" = "Error" ]; then
+					exit 1
+				fi
+				continue
+			fi
+		fi
+		#echo "$pkg -> $new_pkg" >&2
+		pkgs_to_install="${pkgs_to_install} ${new_pkg}"
+	done
+	export pkgs_to_install
+}
+
 
 #
 # Install a bunch of packages using rpm.
@@ -96,18 +246,26 @@ package_install_internal_rpm () {
 	local providename="$INSTALL_PROVIDENAME_RPM"
 	local task="$INSTALL_TASK_RPM"
 
+	local sdk_mode=""
+	if [ "$1" = "--sdk" ]; then
+		sdk_mode="--sdk"
+	fi
+
 	# Configure internal RPM environment when using Smart
 	export RPM_ETCRPM=${target_rootfs}/etc/rpm
 
 	# Setup temporary directory -- install...
-	mkdir -p ${target_rootfs}/install
+	rm -rf ${target_rootfs}/install
+	mkdir -p ${target_rootfs}/install/tmp
 
+	channel_priority=5
 	if [ "${INSTALL_COMPLEMENTARY_RPM}" != "1" ] ; then
 		# Setup base system configuration
 		mkdir -p ${target_rootfs}/etc/rpm/
 		echo "${platform}${TARGET_VENDOR}-${TARGET_OS}" > ${target_rootfs}/etc/rpm/platform
 		if [ ! -z "$platform_extra" ]; then
 			for pt in $platform_extra ; do
+				channel_priority=$(expr $channel_priority + 5)
 				case $pt in
 					noarch | any | all)
 						os="`echo ${TARGET_OS} | sed "s,-.*,,"`.*"
@@ -178,11 +336,14 @@ EOF
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._tmppath=/install/tmp
 		smart --data-dir=${target_rootfs}/var/lib/smart channel --add rpmsys type=rpm-sys -y
 
-		for arch in $platform_extra ; do
+		platform_extra_fixed=`echo "$platform_extra" | tr - _`
+		for arch in $platform_extra_fixed ; do
 			if [ -d ${DEPLOY_DIR_RPM}/$arch -a ! -e ${target_rootfs}/install/channel.$arch.stamp ] ; then
 				smart --data-dir=${target_rootfs}/var/lib/smart channel --add $arch type=rpm-md type=rpm-md baseurl=${DEPLOY_DIR_RPM}/$arch -y
+				smart --data-dir=${target_rootfs}/var/lib/smart channel --set $arch priority=$channel_priority
 				touch ${target_rootfs}/install/channel.$arch.stamp
 			fi
+			channel_priority=$(expr $channel_priority - 5)
 		done
 	fi
 
@@ -218,14 +379,15 @@ EOF
 	chmod 0755 ${WORKDIR}/scriptlet_wrapper
 	smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._cross_scriptlet_wrapper=${WORKDIR}/scriptlet_wrapper
 
-	smart --data-dir=${target_rootfs}/var/lib/smart install -y ${package_to_install} ${package_linguas}
+	# Determine what to install
+	translate_oe_to_smart ${sdk_mode} ${package_to_install} ${package_linguas}
 
-	if [ ! -z "${package_attemptonly}" ]; then
-		echo "Installing attempt only packages..."
-		for pkg_name in ${package_attemptonly} ; do
-			echo "Attempting $pkg_name..." >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}"
-			smart --data-dir=${target_rootfs}/var/lib/smart install -y $pkg_name >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}" 2>&1 || true
-		done
+	[ -n "$pkgs_to_install" ] && smart --data-dir=${target_rootfs}/var/lib/smart install -y ${pkgs_to_install}
+
+	if [ -n "${package_attemptonly}" ]; then
+		translate_oe_to_smart ${sdk_mode} --attemptonly $package_attemptonly
+		echo "Attempting $pkgs_to_install"
+		smart --data-dir=${target_rootfs}/var/lib/smart install -y $pkgs_to_install >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}" 2>&1 || true
 	fi
 }
 
meta/classes/populate_sdk_rpm.bbclass +18 -10
index 1d6b1be..396f1a2 100644
@@ -52,8 +52,16 @@ populate_sdk_rpm () {
 
 	# List must be prefered to least preferred order
 	INSTALL_PLATFORM_EXTRA_RPM=""
-	for each_arch in ${MULTILIB_PACKAGE_ARCHS} ${PACKAGE_ARCHS} ; do
-		INSTALL_PLATFORM_EXTRA_RPM="$each_arch $INSTALL_PLATFORM_EXTRA_RPM"
+	for i in ${MULTILIB_PREFIX_LIST} ; do
+		old_IFS="$IFS"
+		IFS=":"
+		set $i
+		IFS="$old_IFS"
+		shift #remove mlib
+		while [ -n "$1" ]; do
+			INSTALL_PLATFORM_EXTRA_RPM="$INSTALL_PLATFORM_EXTRA_RPM $1"
+			shift
+		done
 	done
 	export INSTALL_PLATFORM_EXTRA_RPM
 
@@ -79,7 +87,7 @@ populate_sdk_rpm () {
 	done
 	export INSTALL_PLATFORM_EXTRA_RPM
 
-	package_install_internal_rpm
+	package_install_internal_rpm --sdk
 	populate_sdk_post_rpm ${INSTALL_ROOTFS_RPM}
 
 	# move host RPM library data
@@ -96,8 +104,11 @@ populate_sdk_rpm () {
 
 python () {
     # The following code should be kept in sync w/ the rootfs_rpm version.
-    ml_package_archs = ""
-    ml_prefix_list = ""
+
+    # package_arch order is reversed.  This ensures the -best- match is listed first!
+    package_archs = d.getVar("PACKAGE_ARCHS", True) or ""
+    package_archs = ":".join(package_archs.split()[::-1])
+    ml_prefix_list = "%s:%s" % ('default', package_archs)
     multilibs = d.getVar('MULTILIBS', True) or ""
     for ext in multilibs.split():
         eext = ext.split(':')
@@ -107,11 +118,8 @@ python () {
             if default_tune:
                 localdata.setVar("DEFAULTTUNE", default_tune)
             package_archs = localdata.getVar("PACKAGE_ARCHS", True) or ""
-            package_archs = " ".join([i in "all noarch any".split() and i or eext[1]+"_"+i for i in package_archs.split()])
-            ml_package_archs += " " + package_archs
-            ml_prefix_list += " " + eext[1]
-            #bb.note("ML_PACKAGE_ARCHS %s %s %s" % (eext[1], localdata.getVar("PACKAGE_ARCHS", True) or "(none)", overrides))
-    d.setVar('MULTILIB_PACKAGE_ARCHS', ml_package_archs)
+            package_archs = ":".join([i in "all noarch any".split() and i or eext[1]+"_"+i for i in package_archs.split()][::-1])
+            ml_prefix_list += " %s:%s" % (eext[1], package_archs)
     d.setVar('MULTILIB_PREFIX_LIST', ml_prefix_list)
 }
 
meta/classes/rootfs_rpm.bbclass +23 -24

70 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#6 lore raw

PATCH v3 6/13 package_rpm.bbclass: Fix the way the RPM platform file is generated

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

When generating an SDK, we need to use either the SDK or TARGET version of the OS. They are not interchangable!

---
 meta/classes/package_rpm.bbclass |   16 +++++++++++++---
 1 file changed, 13 insertions(+), 3 deletions(-)
meta/classes/package_rpm.bbclass +13 -3
index 4f60daf..fdca2c3 100644
@@ -262,16 +262,26 @@ package_install_internal_rpm () {
 	if [ "${INSTALL_COMPLEMENTARY_RPM}" != "1" ] ; then
 		# Setup base system configuration
 		mkdir -p ${target_rootfs}/etc/rpm/
-		echo "${platform}${TARGET_VENDOR}-${TARGET_OS}" > ${target_rootfs}/etc/rpm/platform
+		if [ -n "${sdk_mode}" ]; then
+			platform_vendor="${SDK_VENDOR}"
+			platform_os="${SDK_OS}"
+		else
+			platform_vendor="${TARGET_VENDOR}"
+			platform_os="${TARGET_OS}"
+		fi
+
+		echo "${platform}${platform_vendor}-${platform_os}" > ${target_rootfs}/etc/rpm/platform
+
+
 		if [ ! -z "$platform_extra" ]; then
 			for pt in $platform_extra ; do
 				channel_priority=$(expr $channel_priority + 5)
 				case $pt in
 					noarch | any | all)
-						os="`echo ${TARGET_OS} | sed "s,-.*,,"`.*"
+						os="`echo ${platform_os} | sed "s,-.*,,"`.*"
 						;;
 					*)
-						os="${TARGET_OS}"
+						os="${platform_os}"
 						;;
 				esac
 				echo "$pt-.*-$os" >> ${target_rootfs}/etc/rpm/platform
Signature · 1 line
1.7.10.4
#7 lore raw

PATCH v3 7/13 package_rpm.bbclass: Add additional logging

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

---
 meta/classes/package_rpm.bbclass |    9 +++++++++
 1 file changed, 9 insertions(+)
meta/classes/package_rpm.bbclass +9 -0
index fdca2c3..b80b939 100644
@@ -261,6 +261,7 @@ package_install_internal_rpm () {
 	channel_priority=5
 	if [ "${INSTALL_COMPLEMENTARY_RPM}" != "1" ] ; then
 		# Setup base system configuration
+		echo "Note: configuring RPM platform settings"
 		mkdir -p ${target_rootfs}/etc/rpm/
 		if [ -n "${sdk_mode}" ]; then
 			platform_vendor="${SDK_VENDOR}"
@@ -289,6 +290,7 @@ package_install_internal_rpm () {
 		fi
 
 		# Tell RPM that the "/" directory exist and is available
+		echo "Note: configuring RPM system provides"
 		mkdir -p ${target_rootfs}/etc/rpm/sysinfo
 		echo "/" >${target_rootfs}/etc/rpm/sysinfo/Dirnames
 
@@ -300,6 +302,7 @@ package_install_internal_rpm () {
 		fi
 
 		# Configure RPM... we enforce these settings!
+		echo "Note: configuring RPM DB settings"
 		mkdir -p ${target_rootfs}${rpmlibdir}
 		mkdir -p ${target_rootfs}${rpmlibdir}/log
 		# After change the __db.* cache size, log file will not be generated automatically,
@@ -337,6 +340,7 @@ EOF
 		rpm --root $target_rootfs --dbpath /var/lib/rpm -qa > /dev/null
 
 		# Configure smart
+		echo "Note: configuring Smart settings"
 		rm -rf ${target_rootfs}/var/lib/smart
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-root=${target_rootfs}
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-dbpath=${rpmlibdir}
@@ -349,6 +353,7 @@ EOF
 		platform_extra_fixed=`echo "$platform_extra" | tr - _`
 		for arch in $platform_extra_fixed ; do
 			if [ -d ${DEPLOY_DIR_RPM}/$arch -a ! -e ${target_rootfs}/install/channel.$arch.stamp ] ; then
+				echo "Note: adding Smart channel $arch ($channel_priority)"
 				smart --data-dir=${target_rootfs}/var/lib/smart channel --add $arch type=rpm-md type=rpm-md baseurl=${DEPLOY_DIR_RPM}/$arch -y
 				smart --data-dir=${target_rootfs}/var/lib/smart channel --set $arch priority=$channel_priority
 				touch ${target_rootfs}/install/channel.$arch.stamp
@@ -386,15 +391,19 @@ if [ \$? -ne 0 ]; then
 fi
 EOF
 
+	echo "Note: configuring RPM cross-install scriptlet_wrapper"
 	chmod 0755 ${WORKDIR}/scriptlet_wrapper
 	smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._cross_scriptlet_wrapper=${WORKDIR}/scriptlet_wrapper
 
 	# Determine what to install
 	translate_oe_to_smart ${sdk_mode} ${package_to_install} ${package_linguas}
 
+	echo "Note: to be installed: ${pkgs_to_install}"
 	[ -n "$pkgs_to_install" ] && smart --data-dir=${target_rootfs}/var/lib/smart install -y ${pkgs_to_install}
 
 	if [ -n "${package_attemptonly}" ]; then
+		echo "Note: installing attempt only packages..."
+		echo "Note: see `dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}"
 		translate_oe_to_smart ${sdk_mode} --attemptonly $package_attemptonly
 		echo "Attempting $pkgs_to_install"
 		smart --data-dir=${target_rootfs}/var/lib/smart install -y $pkgs_to_install >> "`dirname ${BB_LOGFILE}`/log.do_${task}_attemptonly.${PID}" 2>&1 || true
Signature · 1 line
1.7.10.4
#8 lore raw

PATCH v3 8/13 package_rpm.bbclass: Add support for incremental installs

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

---
 meta/classes/package_rpm.bbclass |   48 +++++++++++++++++++++++++++++++++++++-
 1 file changed, 47 insertions(+), 1 deletion(-)
meta/classes/package_rpm.bbclass +47 -1
index b80b939..cb8e0f4 100644
@@ -348,7 +348,8 @@ EOF
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-noparentdirs=1
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._var=${localstatedir}
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._tmppath=/install/tmp
-		smart --data-dir=${target_rootfs}/var/lib/smart channel --add rpmsys type=rpm-sys -y
+		# Delay this until later...
+		#smart --data-dir=${target_rootfs}/var/lib/smart channel --add rpmsys type=rpm-sys -y
 
 		platform_extra_fixed=`echo "$platform_extra" | tr - _`
 		for arch in $platform_extra_fixed ; do
@@ -398,6 +399,51 @@ EOF
 	# Determine what to install
 	translate_oe_to_smart ${sdk_mode} ${package_to_install} ${package_linguas}
 
+	# If incremental install, we need to determine what we've got,
+	# what we need to add, and what to remove...
+	if [ "${INC_RPM_IMAGE_GEN}" = "1" -a "${INSTALL_COMPLEMENTARY_RPM}" != "1" ]; then
+		# Dump the new solution
+		echo "Note: creating install solution for incremental install"
+		smart --data-dir=${target_rootfs}/var/lib/smart install -y --dump ${pkgs_to_install} 2> ${target_rootfs}/../solution.manifest
+	fi
+
+	if [ "${INSTALL_COMPLEMENTARY_RPM}" != "1" ]; then
+		echo "Note: adding Smart RPM DB channel"
+		smart --data-dir=${target_rootfs}/var/lib/smart channel --add rpmsys type=rpm-sys -y
+	fi
+
+	# If incremental install, we need to determine what we've got,
+	# what we need to add, and what to remove...
+	if [ "${INC_RPM_IMAGE_GEN}" = "1" -a "${INSTALL_COMPLEMENTARY_RPM}" != "1" ]; then
+		# First upgrade everything that was previously installed to the latest version
+		echo "Note: incremental update -- upgrade packages in place"
+		smart --data-dir=${target_rootfs}/var/lib/smart upgrade
+
+		# Dump what is already installed
+		echo "Note: dump installed packages for incremental update"
+		smart --data-dir=${target_rootfs}/var/lib/smart query --installed --output ${target_rootfs}/../installed.manifest
+
+		sort ${target_rootfs}/../installed.manifest > ${target_rootfs}/../installed.manifest.sorted
+		sort ${target_rootfs}/../solution.manifest > ${target_rootfs}/../solution.manifest.sorted
+		
+		comm -1 -3 ${target_rootfs}/../solution.manifest.sorted ${target_rootfs}/../installed.manifest.sorted \
+			> ${target_rootfs}/../remove.list
+		comm -2 -3 ${target_rootfs}/../solution.manifest.sorted ${target_rootfs}/../installed.manifest.sorted \
+			> ${target_rootfs}/../install.list
+		
+		pkgs_to_remove=`cat ${target_rootfs}/../remove.list | xargs echo`
+		pkgs_to_install=`cat ${target_rootfs}/../install.list | xargs echo`
+		
+		echo "Note: to be removed: ${pkgs_to_remove}"
+
+		for pkg in ${pkgs_to_remove}; do
+			echo "Debug: What required: $pkg"
+			smart --data-dir=${target_rootfs}/var/lib/smart query $pkg --show-requiredby
+		done
+
+		[ -n "$pkgs_to_remove" ] && smart --data-dir=${target_rootfs}/var/lib/smart remove -y ${pkgs_to_remove}
+	fi
+
 	echo "Note: to be installed: ${pkgs_to_install}"
 	[ -n "$pkgs_to_install" ] && smart --data-dir=${target_rootfs}/var/lib/smart install -y ${pkgs_to_install}
 
Signature · 1 line
1.7.10.4
#9 lore raw

PATCH v3 9/13 package_rpm.bbclass: Add additional install error detection

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

There is a condition where the package will fail to install, but smart does not capture the error. Add the error string to the scan list.

---
 meta/classes/package_rpm.bbclass |    2 +-
 1 file changed, 1 insertion(+), 1 deletion(-)
meta/classes/package_rpm.bbclass +1 -1
index cb8e0f4..7af1d6c 100644
@@ -57,7 +57,7 @@ rpm_log_check() {
 	lf_path="$2"
 
 	lf_txt="`cat $lf_path`"
-	for keyword_die in "Cannot find package" "exit 1" ERR Fail
+	for keyword_die in "unpacking of archive failed" "Cannot find package" "exit 1" ERR Fail
 	do
 		if (echo "$lf_txt" | grep -v log_check | grep "$keyword_die") >/dev/null 2>&1
 		then
Signature · 1 line
1.7.10.4
#10 lore raw

PATCH v3 10/13 rpm/smart: Fix runtime-relocation issues w/ RPM and Smart

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

Fix runtime-relocation issues with the RPM and Smart wrappers.

In addition the patches were necessary to fix related problems. The changes to the includes three categories of issues:
*) Incorrect pathname evaluations
*) Incorrect evaluation of the /etc/rpm/platform file contents *) Confusing vendor #define checks

Finally, a simple way to debug the platformScore was added as that is necessary to debug how this works and into the smart system.

---
 .../python/python-smartpm_1.4.1.bb                 |   17 +++-
 .../rpm/rpm/rpm-debug-platform.patch               |   65 +++++++++++++++
 meta/recipes-devtools/rpm/rpm/rpm-platform.patch   |    2 +-
 meta/recipes-devtools/rpm/rpm/rpm-platform2.patch  |   87 +++++++++++++++-----
 meta/recipes-devtools/rpm/rpm_5.4.9.bb             |   41 +++++----
 5 files changed, 173 insertions(+), 39 deletions(-)
 create mode 100644 meta/recipes-devtools/rpm/rpm/rpm-debug-platform.patch
meta/recipes-devtools/python/python-smartpm_1.4.1.bb +16 -1
index 53f232b..04f61f8 100644
@@ -11,7 +11,7 @@ LICENSE = "GPLv2"
 LIC_FILES_CHKSUM = "file://LICENSE;md5=393a5ca445f6965873eca0259a17f833"
 
 DEPENDS = "python rpm"
-PR = "r4"
+PR = "r5"
 SRCNAME = "smart"
 
 SRC_URI = "\
@@ -87,6 +87,21 @@ do_install_append() {
    fi
 }
 
+add_native_wrapper() {
+        create_wrapper ${D}/${bindir}/smart \
+		RPM_USRLIBRPM='`dirname $''realpath`'/${@os.path.relpath(d.getVar('libdir', True), d.getVar('bindir', True))}/rpm \
+		RPM_ETCRPM='$'{RPM_ETCRPM-'`dirname $''realpath`'/${@os.path.relpath(d.getVar('sysconfdir', True), d.getVar('bindir', True))}/rpm} \
+		RPM_LOCALEDIRRPM='`dirname $''realpath`'/${@os.path.relpath(d.getVar('datadir', True), d.getVar('bindir', True))}/locale
+}
+
+do_install_append_class-native() {
+        add_native_wrapper
+}
+
+do_install_append_class-nativesdk() {
+        add_native_wrapper
+}
+
 PACKAGES = "${PN}-dev ${PN}-dbg ${PN}-doc smartpm \
             ${@base_contains('PACKAGECONFIG', 'rpm', '${PN}-backend-rpm', '', d)} \
             ${@base_contains('PACKAGECONFIG', 'qt4', '${PN}-interface-qt4', '', d)} \
meta/recipes-devtools/rpm/rpm/rpm-debug-platform.patch new file +65 -0

65 lines not on this page · raw message

meta/recipes-devtools/rpm/rpm/rpm-platform.patch +1 -1
index 7b81857..9e167d8 100644
@@ -40,7 +40,7 @@ diff -ur rpm-5.4.0.orig/lib/depends.c rpm-5.4.0/lib/depends.c
      return rc;
  }
  
-+#if defined(RPM_VENDOR_WINDRIVER)
++#if defined(RPM_VENDOR_WINDRIVER) || defined(RPM_VENDOR_POKY)
 +#define _ETC_RPM_SYSINFO        "%{_etcrpm}/sysinfo"
 +#else
 +#define _ETC_RPM_SYSINFO        SYSCONFIGDIR "/sysinfo"
meta/recipes-devtools/rpm/rpm/rpm-platform2.patch +68 -19

120 lines not on this page · raw message

meta/recipes-devtools/rpm/rpm_5.4.9.bb +23 -18

80 lines not on this page · raw message

Signature · 1 line
1.7.10.4
#11 lore raw

PATCH v3 11/13 package_rpm.bbclass: Add a simply way to enable RPM debugging

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

---
 meta/classes/package_rpm.bbclass |    3 +++
 1 file changed, 3 insertions(+)
meta/classes/package_rpm.bbclass +3 -0
index 7af1d6c..6a0bb91 100644
@@ -348,6 +348,9 @@ EOF
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-noparentdirs=1
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._var=${localstatedir}
 		smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-extra-macros._tmppath=/install/tmp
+		# Optional debugging
+		#smart --data-dir=${target_rootfs}/var/lib/smart config --set rpm-log-level=debug
+
 		# Delay this until later...
 		#smart --data-dir=${target_rootfs}/var/lib/smart channel --add rpmsys type=rpm-sys -y
 
Signature · 1 line
1.7.10.4
#12 lore raw

PATCH v3 12/13 coreutils_6.9: Update the older coreutils to no longer provide 'su'

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

'su' was removed from newer versions of coreutils, and in OE-Core we pick up 'su' from shadow instead. So simply remove the binary from the install image to match expected behavior and avoid failures due to conflicting 'su' binaries during do_rootfs when building non-GPLv3 images.

---
 meta/recipes-core/coreutils/coreutils_6.9.bb |    6 +++++-
 1 file changed, 5 insertions(+), 1 deletion(-)
meta/recipes-core/coreutils/coreutils_6.9.bb +5 -1
index 1b25320..0ad148b 100644
@@ -8,7 +8,7 @@ BUGTRACKER = "http://debbugs.gnu.org/coreutils"
 LICENSE = "GPLv2+"
 LIC_FILES_CHKSUM = "file://COPYING;md5=751419260aa954499f7abaabaa882bbe \
                     file://src/ls.c;beginline=4;endline=16;md5=15ed60f67b1db5fedd5dbc37cf8a9543"
-PR = "r3"
+PR = "r4"
 DEPENDS = "coreutils-native-${PV}"
 DEPENDS_class-native = "gettext-native"
 
@@ -60,6 +60,10 @@ do_install() {
 	# in update-alternatives to fail, therefore use lbracket - the name used
 	# for the actual source file.
 	mv ${D}${bindir}/[ ${D}${bindir}/lbracket.${BPN}
+
+	# Newer versions of binutils do not include su, to mimic this behavior
+	# we simply remove it.
+	rm -f ${D}${bindir}/su
 }
 
 inherit update-alternatives
Signature · 1 line
1.7.10.4
#13 lore raw

PATCH v3 13/13 package_rpm: Workaround for 'all' arch multilib package naming

In reply to Paul Eggleton

From: Mark Hatle <[email protected]>

[YOCTO #3565]

All packages should not be multilib specific, however when the multilib rules run, they automatically rename the 'all' target recipes. As such there is no way to know if an 'all' package should or should not contain the multilib prefix. We workaround this issue in the translate function by checking to see if the runtime-reverse file exists or not.

The workaround should be removed once a proper solution to the architectural issue is in place.

---
 meta/classes/package_rpm.bbclass |   14 ++++++++++++--
 1 file changed, 12 insertions(+), 2 deletions(-)
meta/classes/package_rpm.bbclass +12 -2
index 6a0bb91..a00b702 100644
@@ -80,6 +80,7 @@ translate_smart_to_oe() {
 
 	# Dump installed packages
 	while read pkg arch other ; do
+		found=0
 		if [ -z "$pkg" ]; then
 			continue
 		fi
@@ -101,10 +102,19 @@ translate_smart_to_oe() {
 					else
 						new_pkg="$mlib-$pkg"
 					fi
-					break
+					# Workaround for bug 3565
+					# Simply look to see if we know of a package with that name, if not try again!
+					filename=`ls ${TMPDIR}/pkgdata/*/runtime-reverse/$new_pkg 2>/dev/null | head -n 1`
+					if [ -n "$filename" ] ; then
+						found=1
+						break
+					fi
+					# 'real' code
+					# found=1
+					# break
 				fi
 			done
-			if [ "$arch" = "$cmp_arch" -o "$fixed_arch" = "$cmp_arch" ]; then
+			if [ "$found" = "1" ] && [ "$arch" = "$cmp_arch" -o "$fixed_arch" = "$cmp_arch" ]; then
 				break
 			fi
 		done
Signature · 1 line
1.7.10.4