Skip to contentSkip to conversation

Install external binary packages

PATCH New Robert Yang 1 participant 4 messages Aug 20, 2012

lore
Share & tools
lore
#0 lore raw

v3 cover Install external binary packages

Changes of V3:
* Fix the "bitbake package-index" failure.
# Tested: (with rpm/ipk/deb rootfs):

  $ bitbake core-image-minimal core-image-sato \
	meta-toolchain meta-toolchain-sdk core-image-sato-sdk
  $ bitbake package-index (I had forgot to test this in V2).

Changes of V2:
* Add commit message to each commit.

Changes of V1:
* Initial version

// Robert

The following changes since commit 91ece5d5668ee1adb5c5e3d757426a0d5187bd84:

libpam: Fix missing DESTDIR for a mkdir causing build failures (2012-08-19 13:32:06 +0100)

are available in the git repository at:

git://git.pokylinux.org/poky-contrib robert/external_install
http://git.pokylinux.org/cgit.cgi/poky-contrib/log/?h=robert/external_install

Robert Yang (3):
rpm: install external binary packages
ipk: install external binary packages
deb: install external binary packages

 meta/classes/external_package.bbclass | 91 +++++++++++++++++++++++++++++++++++
 meta/classes/package_deb.bbclass      | 44 ++++++++++-------
 meta/classes/package_ipk.bbclass      | 22 ++++++++-
 meta/classes/package_rpm.bbclass      | 13 +++--
 meta/classes/rootfs_deb.bbclass       |  2 +-
 meta/classes/rootfs_ipk.bbclass       |  2 +-
 meta/classes/rootfs_rpm.bbclass       |  2 +-
 7 files changed, 152 insertions(+), 24 deletions(-)
 create mode 100644 meta/classes/external_package.bbclass
Signature · 1 line
1.7.11.2
#1 lore raw

PATCH v3 1/3 rpm: install external binary packages

It's been suggested that it would be a useful feature to be able to easily take a binary from a 3rd party software vendor and integrate it into an image created by the build system.

* The user can easily use this by:
# Specify where is the external binary pkg dir
#EXTERNAL_PACKAGE_DIR = "<path1> <path2> ..."
# Specify which pkg will be installed
#EXTERNAL_INSTALL_PACKAGE = "<pkg1> <pkg2> ..."

Then the pkg1, pkg2 ... would be installed.

Add an "EXTERNAL_INSTALL_PACKAGE"here since we can't use the existence variable (for example, IMAGE_INSTALL), if we add the pkg to the IMAGE_INSTALL, it would check whether the pkg is in the PROVIDES, and this is an external pkg, it is not in the PROVIDES.

* Main changes:
- Add external_package.bbclass:
Add the external package to the repo, the package would be copied to deploy/rpm/external/<arch>.

Use an "external" directory since we don't want the "internal" pkgs to be mixed, and it would be easy to remove them.

- package_rpm.bbclass:
Create repo for deploy/rpm/external

- rootfs_rpm.bbclass
Add the package that would be installed to INSTALL_PACKAGES_RPM, so that it would be installed.

[YOCTO #1592]

---
 meta/classes/external_package.bbclass | 42 +++++++++++++++++++++++++++++++++++
 meta/classes/package_rpm.bbclass      | 13 ++++++++---
 meta/classes/rootfs_rpm.bbclass       |  2 +-
 3 files changed, 53 insertions(+), 4 deletions(-)
 create mode 100644 meta/classes/external_package.bbclass
meta/classes/external_package.bbclass new file +42 -0
new file mode 100644
index 0000000..e032bec
@@ -0,0 +1,42 @@
+#
+# ex:ts=4:sw=4:sts=4:et
+# -*- tab-width: 4; c-basic-offset: 4; indent-tabs-mode: nil -*-
+#
+# Add external binary pkg to the repo and install them:
+#
+# Specify where are the external binary pkg dir
+#EXTERNAL_PACKAGE_DIR = "<path1> <path2> ..."
+# Specify which pkg will be installed
+#EXTERNAL_INSTALL_PACKAGE = "<pkg1> <pkg2> ..."
+
+#
+# Add the external binary rpm into the repo, copy the binary rpm files
+# from EXTERNAL_PACKAGE_DIR to ${DEPLOY_DIR_RPM}/external, and put them
+# to the relevant arch dir.
+#
+add_external_rpm () {
+    local supported_archs
+    supported_archs="$@"
+
+    # Clean the EXTERNAL_DIR_RPM dir and re-copy
+    [ ! -d "${EXTERNAL_DIR_RPM}" ] || rm -fr ${EXTERNAL_DIR_RPM}
+
+    if [ -n "${EXTERNAL_PACKAGE_DIR}" -a -n "${EXTERNAL_INSTALL_PACKAGE}" ]; then
+        echo "Adding external binary rpms to the repo ..."
+        for f in `find ${EXTERNAL_PACKAGE_DIR} -type f  -name '*.rpm'`; do
+            arch="`echo $f | awk -F\. '{print $(NF-1)}'`"
+            found=""
+            for archvar in $supported_archs; do
+                # Only pick up the supported arch's rpm
+                if [ "$arch" == "$archvar" ]; then
+                    [ -d "${EXTERNAL_DIR_RPM}/$arch" ] || mkdir -p ${EXTERNAL_DIR_RPM}/$arch
+                    cp -f $f ${EXTERNAL_DIR_RPM}/$arch/
+                    found="1"
+                    break
+                fi
+            done
+            [ -n "$found" ] || echo "Uknown arch $arch"
+        done
+    fi
+}
+
meta/classes/package_rpm.bbclass +10 -3
index 4b18cc6..662a589 100644
@@ -1,4 +1,5 @@
 inherit package
+inherit external_package
 
 IMAGE_PKGTYPE ?= "rpm"
 
@@ -7,6 +8,7 @@ RPMBUILD="rpmbuild"
 
 PKGWRITEDIRRPM = "${WORKDIR}/deploy-rpms"
 PKGWRITEDIRSRPM = "${DEPLOY_DIR}/sources/deploy-srpm"
+EXTERNAL_DIR_RPM = "${DEPLOY_DIR_RPM}/external"
 
 python package_rpm_fn () {
     d.setVar('PKGFN', d.getVar('PKG'))
@@ -26,6 +28,9 @@ package_update_index_rpm () {
 		return
 	fi
 
+	# Add external binary pkgs
+	add_external_rpm ${PACKAGE_ARCHS} ${MULTILIB_PACKAGE_ARCHS} ${SDK_PACKAGE_ARCHS}
+
 	# Update target packages
 	base_archs="${PACKAGE_ARCHS}"
 	ml_archs="${MULTILIB_PACKAGE_ARCHS}"
@@ -44,9 +49,11 @@ package_update_index_rpm_common () {
 	for archvar in "$@"; do
 		eval archs=\${${archvar}}
 		packagedirs=""
-		for arch in $archs; do
-			packagedirs="${DEPLOY_DIR_RPM}/$arch $packagedirs"
-			rm -rf ${DEPLOY_DIR_RPM}/$arch/solvedb.done
+		for d in ${DEPLOY_DIR_RPM} ${EXTERNAL_DIR_RPM}; do
+			for arch in $archs; do
+				packagedirs="$d/$arch $packagedirs"
+				rm -rf $d/$arch/solvedb.done
+			done
 		done
 
 		cat /dev/null > ${rpmconf_base}-${archvar}.conf
meta/classes/rootfs_rpm.bbclass +1 -1
index c0207d8..7ce694c 100644
@@ -51,7 +51,7 @@ fakeroot rootfs_rpm_do_rootfs () {
 	export INSTALL_ROOTFS_RPM="${IMAGE_ROOTFS}"
 	export INSTALL_PLATFORM_RPM="${TARGET_ARCH}"
 	export INSTALL_CONFBASE_RPM="${RPMCONF_TARGET_BASE}"
-	export INSTALL_PACKAGES_RPM="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_RPM="${PACKAGE_INSTALL} ${EXTERNAL_INSTALL_PACKAGE}"
 	export INSTALL_PACKAGES_ATTEMPTONLY_RPM="${PACKAGE_INSTALL_ATTEMPTONLY}"
 	export INSTALL_PACKAGES_LINGUAS_RPM="${LINGUAS_INSTALL}"
 	export INSTALL_PROVIDENAME_RPM=""
Signature · 1 line
1.7.11.2
#2 lore raw

PATCH v3 2/3 ipk: install external binary packages

In reply to Robert Yang

It's been suggested that it would be a useful feature to be able to easily take a binary from a 3rd party software vendor and integrate it into an image created by the build system.

* The user can easily use this by:
# Specify where is the external binary pkg dir
#EXTERNAL_PACKAGE_DIR = "<path1> <path2> ..."
# Specify which pkg will be installed
#EXTERNAL_INSTALL_PACKAGE = "<pkg1> <pkg2> ..."

Then the pkg1, pkg2 ... would be installed.

Add an "EXTERNAL_INSTALL_PACKAGE"here since we can't use the existence variable (for example, IMAGE_INSTALL), if we add the pkg to the IMAGE_INSTALL, it would check whether the pkg is in the PROVIDES, and this is an external pkg, it is not in the PROVIDES.

* Main changes:
- Add external_package.bbclass:
Add the external package to the repo, the package would be copied to deploy/ipk/external/<arch>, the add_external_debipk is used by ipk and deb, the add_external_ipk is just a wrapper.

Use an "external" directory since we don't want the "internal" pkgs to be mixed, and it would be easy to remove them.

- package_ipk.bbclass:
Create repo for deploy/ipk/external

- rootfs_ipk.bbclass
Add the package that would be installed to INSTALL_PACKAGES_IPK, so that it would be installed.

[YOCTO #2948]

---
 meta/classes/external_package.bbclass | 45 +++++++++++++++++++++++++++++++++++
 meta/classes/package_ipk.bbclass      | 22 ++++++++++++++++-
 meta/classes/rootfs_ipk.bbclass       |  2 +-
 3 files changed, 67 insertions(+), 2 deletions(-)
meta/classes/external_package.bbclass +45 -0
index e032bec..c21a5c6 100644
@@ -40,3 +40,48 @@ add_external_rpm () {
     fi
 }
 
+#
+# Add the external binary ipk/deb into the repo, copy the binary files
+# from EXTERNAL_PACKAGE_DIR to ${DEPLOY_DIR_IPK}/external, and put them
+# to the relevant arch dir.
+#
+# $1: EXTERNAL_DIR_IPK or EXTERNAL_DIR_DEB
+# $2, $3, $4...: supported archs
+#
+add_external_debipk () {
+    dir="$1"
+
+    if [ "$dir" != "${EXTERNAL_DIR_DEB}" -a "$dir" != "${EXTERNAL_DIR_IPK}" ]; then
+        echo "Unsupported dir $1"
+        return
+    fi
+
+    shift
+    local supported_archs
+    supported_archs="$@"
+
+    # Clean the dir and re-copy
+    [ ! -d "$dir" ] || rm -fr $dir
+
+    if [ -n "${EXTERNAL_PACKAGE_DIR}" -a -n "${EXTERNAL_INSTALL_PACKAGE}" ]; then
+        echo "Adding external binary ${IMAGE_PKGTYPE} to the repo ..."
+        for f in `find ${EXTERNAL_PACKAGE_DIR} -type f  -name "*.${IMAGE_PKGTYPE}"`; do
+            arch="`echo $f | sed 's/.*_\(.*\)\.'"${IMAGE_PKGTYPE}"'$/\1/'`"
+            found=""
+            for archvar in $supported_archs; do
+                # Only pick up the supported arch
+                if [ "$arch" == "$archvar" ]; then
+                    [ -d "$dir/$arch" ] || mkdir -p $dir/$arch
+                    cp -f $f $dir/$arch/
+                    found="1"
+                    break
+                fi
+            done
+            [ -n "$found" ] || echo "Uknown arch $arch"
+        done
+    fi
+}
+
+add_external_ipk () {
+	add_external_debipk $@
+}
meta/classes/package_ipk.bbclass +21 -1
index a297a1f..0dee1d1 100644
@@ -1,4 +1,5 @@
 inherit package
+inherit external_package
 
 IMAGE_PKGTYPE ?= "ipk"
 
@@ -6,6 +7,7 @@ IPKGCONF_TARGET = "${WORKDIR}/opkg.conf"
 IPKGCONF_SDK =  "${WORKDIR}/opkg-sdk.conf"
 
 PKGWRITEDIRIPK = "${WORKDIR}/deploy-ipks"
+EXTERNAL_DIR_IPK = "${DEPLOY_DIR_IPK}/external"
 
 # Program to be used to build opkg packages
 OPKGBUILDCMD ??= "opkg-build"
@@ -203,7 +205,7 @@ package_update_index_ipk () {
 		return
 	fi
 
-	packagedirs="${DEPLOY_DIR_IPK}"
+	packagedirs="${DEPLOY_DIR_IPK} ${EXTERNAL_DIR_IPK}"
 	for arch in $ipkgarchs; do
 		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch"
 	done
@@ -213,6 +215,12 @@ package_update_index_ipk () {
 		packagedirs="$packagedirs ${DEPLOY_DIR_IPK}/$arch"
 	done
 
+	all_archs="$ipkgarchs $multilib_archs"
+	add_external_ipk "${EXTERNAL_DIR_IPK}" $all_archs
+	for arch in $all_archs; do
+		packagedirs="$packagedirs ${EXTERNAL_DIR_IPK}/$arch"
+	done
+
 	for pkgdir in $packagedirs; do
 		if [ -e $pkgdir/ ]; then
 			touch $pkgdir/Packages
@@ -229,19 +237,31 @@ package_update_index_ipk () {
 package_generate_ipkg_conf () {
 	package_generate_archlist
 	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_SDK}
+	if [ -e ${EXTERNAL_DIR_IPK} ]; then
+		echo "src external file:${EXTERNAL_DIR_IPK}" >> ${IPKGCONF_SDK}
+	fi
 	ipkgarchs="${SDK_PACKAGE_ARCHS}"
 	for arch in $ipkgarchs; do
 		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
 		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_SDK}
 		fi
+		if [ -e ${EXTERNAL_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src external-$arch file:${EXTERNAL_DIR_IPK}/$arch" >> ${IPKGCONF_SDK}
+		fi
 	done
 
 	echo "src oe file:${DEPLOY_DIR_IPK}" >> ${IPKGCONF_TARGET}
+	if [ -e ${EXTERNAL_DIR_IPK} ]; then
+		echo "src external file:${EXTERNAL_DIR_IPK}" >> ${IPKGCONF_TARGET}
+	fi
 	ipkgarchs="${ALL_MULTILIB_PACKAGE_ARCHS}"
 	for arch in $ipkgarchs; do
 		if [ -e ${DEPLOY_DIR_IPK}/$arch/Packages ] ; then
 		        echo "src oe-$arch file:${DEPLOY_DIR_IPK}/$arch" >> ${IPKGCONF_TARGET}
 		fi
+		if [ -e ${EXTERNAL_DIR_IPK}/$arch/Packages ] ; then
+		        echo "src external-$arch file:${EXTERNAL_DIR_IPK}/$arch" >> ${IPKGCONF_TARGET}
+		fi
 	done
 }
 
meta/classes/rootfs_ipk.bbclass +1 -1
index 6cdd8f6..edea67f 100644
@@ -62,7 +62,7 @@ fakeroot rootfs_ipk_do_rootfs () {
 
 	export INSTALL_ROOTFS_IPK="${IMAGE_ROOTFS}"
 	export INSTALL_CONF_IPK="${IPKGCONF_TARGET}"
-	export INSTALL_PACKAGES_IPK="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_IPK="${PACKAGE_INSTALL} ${EXTERNAL_INSTALL_PACKAGE}"
 
 	#post install
 	export D=${IMAGE_ROOTFS}
Signature · 1 line
1.7.11.2
#3 lore raw

PATCH v3 3/3 deb: install external binary packages

In reply to Robert Yang

It's been suggested that it would be a useful feature to be able to easily take a binary from a 3rd party software vendor and integrate it into an image created by the build system.

* The user can easily use this by:
# Specify where is the external binary pkg dir
#EXTERNAL_PACKAGE_DIR = "<path1> <path2> ..."
# Specify which pkg will be installed
#EXTERNAL_INSTALL_PACKAGE = "<pkg1> <pkg2> ..."

Then the pkg1, pkg2 ... would be installed.

Add an "EXTERNAL_INSTALL_PACKAGE"here since we can't use the existence variable (for example, IMAGE_INSTALL), if we add the pkg to the IMAGE_INSTALL, it would check whether the pkg is in the PROVIDES, and this is an external pkg, it is not in the PROVIDES.

* Main changes:
- Add external_package.bbclass:
Add the external package to the repo, the package would be copied to deploy/deb/external/<arch>, the add_external_debipk is used by ipk and deb, the add_external_deb is just a wrapper.

Use an "external" directory since we don't want the "internal" pkgs to be mixed, and it would be easy to remove them.

- package_deb.bbclass:
Create repo for deploy/deb/external

- rootfs_deb.bbclass
Add the package that would be installed to INSTALL_PACKAGES_DEB, so that it would be installed.

[YOCTO #2949]

---
 meta/classes/external_package.bbclass |  4 ++++
 meta/classes/package_deb.bbclass      | 44 +++++++++++++++++++++--------------
 meta/classes/rootfs_deb.bbclass       |  2 +-
 3 files changed, 32 insertions(+), 18 deletions(-)
meta/classes/external_package.bbclass +4 -0
index c21a5c6..d971384 100644
@@ -85,3 +85,7 @@ add_external_debipk () {
 add_external_ipk () {
 	add_external_debipk $@
 }
+
+add_external_deb () {
+	add_external_debipk $@
+}
meta/classes/package_deb.bbclass +27 -17

74 lines not on this page · raw message

meta/classes/rootfs_deb.bbclass +1 -1
index 881fdbd..35196df 100644
@@ -34,7 +34,7 @@ fakeroot rootfs_deb_do_rootfs () {
 	export INSTALL_ROOTFS_DEB="${IMAGE_ROOTFS}"
 	export INSTALL_BASEARCH_DEB="${DPKG_ARCH}"
 	export INSTALL_ARCHS_DEB="${PACKAGE_ARCHS}"
-	export INSTALL_PACKAGES_NORMAL_DEB="${PACKAGE_INSTALL}"
+	export INSTALL_PACKAGES_NORMAL_DEB="${PACKAGE_INSTALL} ${EXTERNAL_INSTALL_PACKAGE}"
 	export INSTALL_PACKAGES_ATTEMPTONLY_DEB="${PACKAGE_INSTALL_ATTEMPTONLY}"
 	export INSTALL_PACKAGES_LINGUAS_DEB="${LINGUAS_INSTALL}"
 	export INSTALL_TASK_DEB="rootfs"
Signature · 1 line
1.7.11.2