rpm/deb support for meta-toolchain.

From: Lianhao Lu <[email protected]>

The purpose of this series of patches is to add rpm/deb support in meta-toolchain creation, and also to fix bug #227.

It makes the task populate_sdk in meta/recipes-core/meta/meta-toolchain.bb more generic to support installing packages from ipk, rpm, and deb files in creating meta-toolchain tarball, and move the populate_sdk task out of meta-toolchain.bb recipe into a bbclass.

It added populate_sdk_xxx.bbclass for different package file format.

Since there are quite many code duplication between relevant popuplage_sdk_xxx and rootfs_xxx bbclasses, it also moved the common code into the package_xxx.bbclass, based on RP's suggestion. The generic function package_install_internal_xxx() may installs a list of packages, based on the information specified by various shell environment variables about where the target directory to be installed to, package architecture information, etc.

It also fixed a bug in cross-canadian.bbclass to enable the package_deb.bbclass generate correct deb package files for cross-canadian recipies.

Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: llu/bug227
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/bug227

Thanks,
Lianhao Lu <[email protected]>
---

Lianhao Lu (5):
cross-canadian.bbclass: Correct deb package arch.
apt-native: Added licence checksum. image/rootfs_xx/package_xx.bbclass: prepare to add populate_sdk task. classes/populate_sdk_xxx.bbclass: pkg format support for populate_sdk
task
meta-toolchain: pkg format independant populate_sdk task.

 meta/classes/cross-canadian.bbclass            |    3 +
 meta/classes/image.bbclass                     |    5 +-
 meta/classes/package_deb.bbclass               |  145 ++++++++++++++
 meta/classes/package_ipk.bbclass               |   63 ++++++
 meta/classes/package_rpm.bbclass               |  243 +++++++++++++++++++++++-
 meta/classes/populate_sdk.bbclass              |   82 ++++++++
 meta/classes/populate_sdk_deb.bbclass          |   60 ++++++
 meta/classes/populate_sdk_ipk.bbclass          |   44 +++++
 meta/classes/populate_sdk_rpm.bbclass          |   67 +++++++
 meta/classes/rootfs_deb.bbclass                |  111 ++---------
 meta/classes/rootfs_ipk.bbclass                |   45 +----
 meta/classes/rootfs_rpm.bbclass                |  157 ++--------------
 meta/recipes-core/meta/meta-toolchain.bb       |  103 +----------
 meta/recipes-devtools/apt/apt-native_0.7.14.bb |    2 +
 14 files changed, 740 insertions(+), 390 deletions(-)
 create mode 100644 meta/classes/populate_sdk.bbclass
 create mode 100644 meta/classes/populate_sdk_deb.bbclass
 create mode 100644 meta/classes/populate_sdk_ipk.bbclass
 create mode 100644 meta/classes/populate_sdk_rpm.bbclass

From: Lianhao Lu <[email protected]>

Set DPKG_ARCH to make debian package to be generated with correct architecture.

---
 meta/classes/cross-canadian.bbclass |    3 +++
 1 files changed, 3 insertions(+), 0 deletions(-)

index 993f831..1a045ba 100644
@@ -43,6 +43,9 @@ HOST_OS = "${SDK_OS}"
 HOST_PREFIX = "${SDK_PREFIX}"
 HOST_CC_ARCH = "${SDK_CC_ARCH}"
 
+#assign DPKG_ARCH
+DPKG_ARCH = "${SDK_ARCH}"
+
 CPPFLAGS = "${BUILDSDK_CPPFLAGS}"
 CFLAGS = "${BUILDSDK_CFLAGS}"
 CXXFLAGS = "${BUILDSDK_CFLAGS}"

1.7.0.4

From: Lianhao Lu <[email protected]>

Added licence checksum for apt-native.

---
 meta/recipes-devtools/apt/apt-native_0.7.14.bb |    2 ++
 1 files changed, 2 insertions(+), 0 deletions(-)

index f77f817..b1bd55b 100644
@@ -9,3 +9,5 @@ SRC_URI += "file://nodoc.patch \
 
 SRC_URI[md5sum] = "19efa18fb1ef20c58b9b44e94258b814"
 SRC_URI[sha256sum] = "8fc06effaf8a4e4333308eedcdc6840f1c8056f2e924210f151dfc076bcd4045"
+
+LIC_FILES_CHKSUM = "file://COPYING.GPL;md5=0636e73ff0215e8d672dc4c32c317bb3"

1.7.0.4

From: Lianhao Lu <[email protected]>

This commit is a preparation work to add a new task populate_sdk. It moves those common code between both the rootfs task and the populate_sdk task in the relevant package_xx bbclass.

Added the new function package_install_internal_xxx() for each kind of package format we support(rpm, ipk, deb). This function installed a bunch of packages using the into a specific root dir. All the informatino needed for the installation process, i.e. root dir, packages to be installed, architecture information is passed by various shell environment variables. See the comment of different package_install_internal_xxx() function for a descriptioin of those shell environment variables.

For package_rpm.bbclass, this commit modified the package_generate_rpm_conf() and package_update_index_rpm to generate seperate configuration files and index for target architecture and "-nativesdk" architecture.

For package_deb.bbclass, this commit added a new function package_update_index_deb to generate index for debian package format.

---
 meta/classes/image.bbclass       |    5 +-
 meta/classes/package_deb.bbclass |  145 +++++++++++++++++++++++
 meta/classes/package_ipk.bbclass |   63 ++++++++++
 meta/classes/package_rpm.bbclass |  243 ++++++++++++++++++++++++++++++++++++--
 meta/classes/rootfs_deb.bbclass  |  111 +++---------------
 meta/classes/rootfs_ipk.bbclass  |   45 ++------
 meta/classes/rootfs_rpm.bbclass  |  157 ++-----------------------
 7 files changed, 479 insertions(+), 290 deletions(-)

index 8e8d92b..14de3e1 100644
@@ -155,7 +155,6 @@ insert_feed_uris () {
 }
 
 log_check() {
-	set +x
 	for target in $*
 	do
 		lf_path="${WORKDIR}/temp/log.do_$target.${PID}"

@@ -164,14 +163,12 @@ log_check() {
 		
 		if test -e "$lf_path"
 		then
-			rootfs_${IMAGE_PKGTYPE}_log_check $target $lf_path
+			${IMAGE_PKGTYPE}_log_check $target $lf_path
 		else
 			echo "Cannot find logfile [$lf_path]"
 		fi
 		echo "Logfile is clean"
 	done
-
-	set -x
 }
 
 # set '*' as the rootpassword so the images

1.7.0.4

From: Lianhao Lu <[email protected]>

This commit added various populate_sdk_xxx.bbclass to support the populate_sdk task based on different package format(rpm, ipk, deb).

It is based on the commit c66b12f138b6e709f9b14cbff9b579634f15cc96.

---
 meta/classes/populate_sdk_deb.bbclass |   60 +++++++++++++++++++++++++++++
 meta/classes/populate_sdk_ipk.bbclass |   44 +++++++++++++++++++++
 meta/classes/populate_sdk_rpm.bbclass |   67 +++++++++++++++++++++++++++++++++
 3 files changed, 171 insertions(+), 0 deletions(-)
 create mode 100644 meta/classes/populate_sdk_deb.bbclass
 create mode 100644 meta/classes/populate_sdk_ipk.bbclass
 create mode 100644 meta/classes/populate_sdk_rpm.bbclass

new file mode 100644
index 0000000..d563c28
@@ -0,0 +1,60 @@
+do_populate_sdk[depends] += "dpkg-native:do_populate_sysroot apt-native:do_populate_sysroot bzip2-native:do_populate_sysroot"
+do_populate_sdk[recrdeptask] += "do_package_write_deb"
+
+
+populate_sdk_post_deb () {
+
+	local target_rootfs=$1
+
+	cp -r ${STAGING_ETCDIR_NATIVE}/apt ${target_rootfs}/etc
+}
+
+fakeroot populate_sdk_deb () {
+
+	package_update_index_deb
+
+	export INSTALL_TASK_DEB="populate_sdk"
+	export INSTALL_PACKAGES_LINGUAS_DEB=""
+	export INSTALL_PACKAGES_ATTEMPTONLY_DEB=""
+
+	#install target
+	echo "Installing TARGET packages"
+	export INSTALL_ROOTFS_DEB="${SDK_OUTPUT}/${SDKTARGETSYSROOT}"
+	export INSTALL_BASEARCH_DEB="${DPKG_ARCH}"
+	export INSTALL_ARCHS_DEB="${PACKAGE_ARCHS}"
+	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_TARGET_TASK}"
+
+	package_install_internal_deb
+	populate_sdk_post_deb ${INSTALL_ROOTFS_DEB}
+
+	populate_sdk_log_check populate_sdk
+
+	#install host
+	echo "Installing HOST packages"
+	export INSTALL_ROOTFS_DEB="${SDK_OUTPUT}"
+	export INSTALL_BASEARCH_DEB="${SDK_ARCH}"
+	export INSTALL_PACKAGES_NORMAL_DEB="${TOOLCHAIN_HOST_TASK}"
+	INSTALL_ARCHS_DEB=""
+	for arch in ${PACKAGE_ARCHS}; do
+		sdkarch=`echo $arch | sed -e 's/${HOST_ARCH}/${SDK_ARCH}/'`
+		extension="-nativesdk"
+		if [ "$sdkarch" = "all" -o "$sdkarch" = "any" -o "$sdkarch" = "noarch" ]; then
+		    extension=""
+		fi
+		if [ -e ${DEPLOY_DIR_DEB}/$sdkarch$extension ]; then
+			INSTALL_ARCHS_DEB="$INSTALL_ARCHS_DEB $sdkarch$extension"
+		fi
+	done
+	export INSTALL_ARCHS_DEB
+
+	package_install_internal_deb
+	populate_sdk_post_deb ${SDK_OUTPUT}/${SDKPATHNATIVE}
+
+	#move remainings
+	install -d ${SDK_OUTPUT}/${SDKPATHNATIVE}/var/dpkg
+	mv ${SDK_OUTPUT}/var/dpkg/* ${SDK_OUTPUT}/${SDKPATHNATIVE}/var/dpkg
+	rm -rf ${SDK_OUTPUT}/var
+
+	populate_sdk_log_check populate_sdk
+}
+

1.7.0.4

From: Lianhao Lu <[email protected]>

[BUGID# 227] fixing bug #227 by make the populate_sdk task independant of ipk, and generic to all package formats supported(rpm, ipk, deb).

This commit is based on the following 2 commits:
c66b12f138b6e709f9b14cbff9b579634f15cc96
1daf63ead393f7ae2fddc778bdd377c1dd58f6ca

---
 meta/classes/populate_sdk.bbclass        |   82 ++++++++++++++++++++++++
 meta/recipes-core/meta/meta-toolchain.bb |  103 +-----------------------------
 2 files changed, 85 insertions(+), 100 deletions(-)
 create mode 100644 meta/classes/populate_sdk.bbclass

new file mode 100644
index 0000000..82386b7
@@ -0,0 +1,82 @@
+inherit meta toolchain-scripts
+inherit populate_sdk_${IMAGE_PKGTYPE}
+
+SDK_DIR = "${WORKDIR}/sdk"
+SDK_OUTPUT = "${SDK_DIR}/image"
+SDK_DEPLOY = "${TMPDIR}/deploy/sdk"
+
+SDKTARGETSYSROOT = "${SDKPATH}/sysroots/${TARGET_SYS}"
+
+TOOLCHAIN_HOST_TASK ?= "task-sdk-host task-cross-canadian-${TRANSLATED_TARGET_ARCH}"
+TOOLCHAIN_TARGET_TASK ?= "task-poky-standalone-sdk-target task-poky-standalone-sdk-target-dbg"
+TOOLCHAIN_OUTPUTNAME ?= "${SDK_NAME}-toolchain-${DISTRO_VERSION}"
+
+RDEPENDS = "${TOOLCHAIN_TARGET_TASK} ${TOOLCHAIN_HOST_TASK}"
+DEPENDS = "virtual/fakeroot-native sed-native"
+
+PID = "${@os.getpid()}"
+
+EXCLUDE_FROM_WORLD = "1"
+
+fakeroot do_populate_sdk() {
+	rm -rf ${SDK_OUTPUT}
+	mkdir -p ${SDK_OUTPUT}
+
+	populate_sdk_${IMAGE_PKGTYPE}
+
+	# Don't ship any libGL in the SDK
+	rm -rf ${SDK_OUTPUT}/${SDKPATHNATIVE}${libdir_nativesdk}/libGL*
+
+	# Can copy pstage files here
+	# target_pkgs=`cat ${SDK_OUTPUT}/${SDKTARGETSYSROOT}/var/lib/opkg/status | grep Package: | cut -f 2 -d ' '`
+
+	# Fix or remove broken .la files
+	#rm -f ${SDK_OUTPUT}/${SDKPATHNATIVE}/lib/*.la
+	rm -f ${SDK_OUTPUT}/${SDKPATHNATIVE}${libdir_nativesdk}/*.la
+
+	# Link the ld.so.cache file into the hosts filesystem
+	ln -s /etc/ld.so.cache ${SDK_OUTPUT}/${SDKPATHNATIVE}/etc/ld.so.cache
+
+	# Setup site file for external use
+	siteconfig=${SDK_OUTPUT}/${SDKPATH}/site-config-${MULTIMACH_TARGET_SYS}
+	touch $siteconfig
+	for sitefile in ${CONFIG_SITE} ; do
+		cat $sitefile >> $siteconfig
+	done
+
+	toolchain_create_sdk_env_script
+
+	# Add version information
+	versionfile=${SDK_OUTPUT}/${SDKPATH}/version-${MULTIMACH_TARGET_SYS}
+	touch $versionfile
+	echo 'Distro: ${DISTRO}' >> $versionfile
+	echo 'Distro Version: ${DISTRO_VERSION}' >> $versionfile
+	echo 'Metadata Revision: ${METADATA_REVISION}' >> $versionfile
+	echo 'Timestamp: ${DATETIME}' >> $versionfile
+
+	# Package it up
+	mkdir -p ${SDK_DEPLOY}
+	cd ${SDK_OUTPUT}
+	tar --owner=root --group=root -cj --file=${SDK_DEPLOY}/${TOOLCHAIN_OUTPUTNAME}.tar.bz2 .
+}
+
+populate_sdk_log_check() {
+	for target in $*
+	do
+		lf_path="${WORKDIR}/temp/log.do_$target.${PID}"
+		
+		echo "log_check: Using $lf_path as logfile"
+		
+		if test -e "$lf_path"
+		then
+			${IMAGE_PKGTYPE}_log_check $target $lf_path
+		else
+			echo "Cannot find logfile [$lf_path]"
+		fi
+		echo "Logfile is clean"
+	done
+}
+
+do_populate_sdk[nostamp] = "1"
+do_populate_sdk[recrdeptask] = "do_package_write"
+addtask populate_sdk before do_build after do_install

1.7.0.4

These patches are good, if you have time could you:

a) rebase these patches against the merged rpm changes from Mark Hatle from the rpm update to 5.4

b) Try hard to split out any logical changes you can from the bigger patches into specific steps. For example, enhancing the log_check function and moving those around could be a separate commit.

The idea is to make changes in clear logical steps where at all possible so anyone reviewing the changes (such as Saul, Mark and myself) can then more quickly "see" the changes are correct and also, if there are problems, we can bisect the commits to track down which change the problem is from.

Cheers,

Richard

Richard Purdie wrote on 2011-01-29:

Hi RP,

Please use my latest branch at
Pull URL: git://git.pokylinux.org/poky-contrib.git
Branch: llu/bug227
Browse: http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=llu/bug227

It's based to Mark's RPM 5.4 upgrade and split the big patches into smaller ones.

- Best Regards,
Lianhao