scripts/contrib/make_ipk_repo.sh: Add new image-specific package feed generator
PATCH 1/1 scripts/contrib/make_ipk_repo.sh: Add new image-specific package feed generator
Adding a new script which generates a reduced package feed directory. The script uses the list of installed packages for a specific build image and creates a new package feed directory with only the packages relevant to that image. This is useful for distributing firmware upgrades.
At this time the script only works for opkg repositories. Further modifications are needed for the script to work with other package management systems.
--- scripts/contrib/make_ipk_repo.sh | 93 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100755 scripts/contrib/make_ipk_repo.sh
scripts/contrib/make_ipk_repo.sh new file +93 -0
@@ -0,0 +1,93 @@ +#!/bin/bash + +# Creates a package feed directory that only contains the packages that were +# installed with the referenced image. This script depends on image build +# history to properly operate. In your conf/local.conf, add the following two +# lines: +# INHERIT += "buildhistory" +# BUILDHISTORY_COMMIT = "1" +# +# At this time this script is only setup to create opkg package feeds. +# Further script modification is needed if you use a different package +# managment system. +# +# This script assumes it is being run from your image build directory. Some +# paths may need to be modified if it is run from a different directory. +# +# Assumes that the relevant image was recently built. If you have done any +# package builds since the last image build, it is suggested that you re-build +# relevant image. +# +# Script input variables: +# $1 - The image name (i.e. "core-image-minimal") +# $2 - The target machine (i.e. "beaglebone") +# $3 - The target architecture (i.e. "arm926ejste") +# $4 - The target directory (i.e. "feed") +# + +# Validate the number of input variables +if [[ "$#" -ne 4 ]]; then + echo "Unexpected number of arguments"; + echo "Usage: make_ipk_repo.sh IMAGE_NAME TARGET_MACHINE TARGET_ARCH TARGET_DIR"; + exit 1; +fi + +# Some packages inexplicably contain a "1:" in the package name that does not +# belong there. Remove this text from these package names. + +sed -i 's/1://g' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt + +# Read in the list of packages used by the image. This includes the +# machine/architecture independent packages ("all"), the architecture specific +# packages, and the machine specific packages. +ALL_PKG_LIST=$(grep '_all\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) +ARCH_PKG_LIST=$(grep '_'"$3"'\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) +MACHINE_PKG_LIST=$(grep '_'"$2"'\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) + +echo "ALL_PKG_LIST is $ALL_PKG_LIST"; +echo "ARCH_PKG_LIST is $ARCH_PKG_LIST"; +echo "MACHINE_PKG_LIST is $MACHINE_PKG_LIST"; + +# Create the new directory for the feeds, making sure there is no previous copy +rm -r "$4"/ +rm "$4".zip +mkdir -p "$4"/ + +# Copy over the "all" packages +mkdir -p "$4"/all/ +echo "Copying packages to 'all' directory" +for i in $ALL_PKG_LIST; do + cp tmp/deploy/ipk/all/"$i" "$4"/all/ +done + +# Copy over the architecture packages +mkdir -p "$4"/$3/ +echo "Copying packages to '$3' directory" +for i in $ARCH_PKG_LIST; do + cp tmp/deploy/ipk/"$3"/"$i" "$4"/"$3"/ +done + +# Copy over the machine packages +mkdir -p "$4"/"$2"/ +echo "Copying packages to '$2' directory" +for i in $MACHINE_PKG_LIST; do + cp tmp/deploy/ipk/"$2"/"$i" "$4"/"$2"/ +done + +# Build the package index files +echo "Building package index files" +touch "$4"/Packages +flock "$4"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/Packages -p $4/Packages -m $4/" +touch "$4"/all/Packages +flock "$4"/all/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/all/Packages -p $4/all/Packages -m $4/all/" +touch "$4"/"$3"/Packages +flock "$4"/"$3"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/$3/Packages -p $4/$3/Packages -m $4/$3/" +touch "$4"/"$2"/Packages +flock "$4"/"$2"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/$2/Packages -p $4/$2/Packages -m $4/$2/" + +#Create a zip file of the directory for archive purposes +cd "$4"/ +zip -r "$4" . + +echo "A reduced package repository has been created at $4" +
Signature · 1 line
1.7.9.5
Quoted 9 lines · Bryan Evenson
Adding a new script which generates a reduced package feed directory. The script uses the list of installed packages for a specific build image and creates a new package feed directory with only the packages relevant to that image. This is useful for distributing firmware upgrades.
At this time the script only works for opkg repositories. Further modifications are needed for the script to work with other package management systems.
This belongs to oe-core ML
Quoted 114 lines
Signed-off-by Bryan Evenson <[email protected]>--- scripts/contrib/make_ipk_repo.sh | 93 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100755 scripts/contrib/make_ipk_repo.shscripts/contrib/make_ipk_repo.sh new file +93
-0@@ -0,0 +1,93 @@ +#!/bin/bash + +# Creates a package feed directory that only contains the packages that were +# installed with the referenced image. This script depends on image build +# history to properly operate. In your conf/local.conf, add the following two +# lines: +# INHERIT += "buildhistory" +# BUILDHISTORY_COMMIT = "1" +# +# At this time this script is only setup to create opkg package feeds. +# Further script modification is needed if you use a different package +# managment system. +# +# This script assumes it is being run from your image build directory. Some +# paths may need to be modified if it is run from a different directory. +# +# Assumes that the relevant image was recently built. If you have done any +# package builds since the last image build, it is suggested that you re-build +# relevant image. +# +# Script input variables: +# $1 - The image name (i.e. "core-image-minimal") +# $2 - The target machine (i.e. "beaglebone") +# $3 - The target architecture (i.e. "arm926ejste") +# $4 - The target directory (i.e. "feed") +# + +# Validate the number of input variables +if [[ "$#" -ne 4 ]]; then + echo "Unexpected number of arguments"; + echo "Usage: make_ipk_repo.sh IMAGE_NAME TARGET_MACHINE TARGET_ARCH TARGET_DIR"; + exit 1; +fi + +# Some packages inexplicably contain a "1:" in the package name that does not +# belong there. Remove this text from these package names. + +sed -i 's/1://g' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt + +# Read in the list of packages used by the image. This includes the +# machine/architecture independent packages ("all"), the architecture specific +# packages, and the machine specific packages. +ALL_PKG_LIST=$(grep '_all\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) +ARCH_PKG_LIST=$(grep '_'"$3"'\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) +MACHINE_PKG_LIST=$(grep '_'"$2"'\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) + +echo "ALL_PKG_LIST is $ALL_PKG_LIST"; +echo "ARCH_PKG_LIST is $ARCH_PKG_LIST"; +echo "MACHINE_PKG_LIST is $MACHINE_PKG_LIST"; + +# Create the new directory for the feeds, making sure there is no previous copy +rm -r "$4"/ +rm "$4".zip +mkdir -p "$4"/ + +# Copy over the "all" packages +mkdir -p "$4"/all/ +echo "Copying packages to 'all' directory" +for i in $ALL_PKG_LIST; do + cp tmp/deploy/ipk/all/"$i" "$4"/all/ +done + +# Copy over the architecture packages +mkdir -p "$4"/$3/ +echo "Copying packages to '$3' directory" +for i in $ARCH_PKG_LIST; do + cp tmp/deploy/ipk/"$3"/"$i" "$4"/"$3"/ +done + +# Copy over the machine packages +mkdir -p "$4"/"$2"/ +echo "Copying packages to '$2' directory" +for i in $MACHINE_PKG_LIST; do + cp tmp/deploy/ipk/"$2"/"$i" "$4"/"$2"/ +done + +# Build the package index files +echo "Building package index files" +touch "$4"/Packages +flock "$4"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/Packages -p $4/Packages -m $4/" +touch "$4"/all/Packages +flock "$4"/all/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/all/Packages -p $4/all/Packages -m $4/all/" +touch "$4"/"$3"/Packages +flock "$4"/"$3"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/$3/Packages -p $4/$3/Packages -m $4/$3/" +touch "$4"/"$2"/Packages +flock "$4"/"$2"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/$2/Packages -p $4/$2/Packages -m $4/$2/" + +#Create a zip file of the directory for archive purposes +cd "$4"/ +zip -r "$4" . + +echo "A reduced package repository has been created at $4" +Signature · 7 lines
1.7.9.5 -- _______________________________________________ poky mailing list [email protected] https://lists.yoctoproject.org/listinfo/poky
Signature · 1 line
Martin 'JaMa' Jansa jabber: [email protected]
Quoted 7 lines · Martin Jansa
Quoted 6 lines · Bryan Evenson
Adding a new script which generates a reduced package feed directory. The script uses the list of installed packages for a specific build image and creates a new package feed directory with only the packages relevant to that image. This is useful for distributing firmware upgrades.
Quoted 5 lines
Quoted 3 lines
At this time the script only works for opkg repositories. Further modifications are needed for the script to work with other package management systems.
This belongs to oe-core ML
Agreed.
Quoted 91 lines
Quoted 91 lines
Signed-off-by Bryan Evenson <[email protected]>--- scripts/contrib/make_ipk_repo.sh | 93 ++++++++++++++++++++++++++++++++++++++ 1 file changed, 93 insertions(+) create mode 100755 scripts/contrib/make_ipk_repo.shscripts/contrib/make_ipk_repo.sh new file +78
-0@@ -0,0 +1,93 @@ +#!/bin/bash + +# Creates a package feed directory that only contains the packages that were +# installed with the referenced image. This script depends on image build +# history to properly operate. In your conf/local.conf, add the following two +# lines: +# INHERIT += "buildhistory" +# BUILDHISTORY_COMMIT = "1" +# +# At this time this script is only setup to create opkg package feeds. +# Further script modification is needed if you use a different package +# managment system. +# +# This script assumes it is being run from your image build directory. Some +# paths may need to be modified if it is run from a different directory. +# +# Assumes that the relevant image was recently built. If you have done any +# package builds since the last image build, it is suggested that you re-build +# relevant image. +# +# Script input variables: +# $1 - The image name (i.e. "core-image-minimal") +# $2 - The target machine (i.e. "beaglebone") +# $3 - The target architecture (i.e. "arm926ejste") +# $4 - The target directory (i.e. "feed") +# + +# Validate the number of input variables +if [[ "$#" -ne 4 ]]; then + echo "Unexpected number of arguments"; + echo "Usage: make_ipk_repo.sh IMAGE_NAME TARGET_MACHINE TARGET_ARCH TARGET_DIR"; + exit 1; +fi + +# Some packages inexplicably contain a "1:" in the package name that does not +# belong there. Remove this text from these package names. + +sed -i 's/1://g' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt + +# Read in the list of packages used by the image. This includes the +# machine/architecture independent packages ("all"), the architecture specific +# packages, and the machine specific packages. +ALL_PKG_LIST=$(grep '_all\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) +ARCH_PKG_LIST=$(grep '_'"$3"'\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) +MACHINE_PKG_LIST=$(grep '_'"$2"'\.ipk' tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) + +echo "ALL_PKG_LIST is $ALL_PKG_LIST"; +echo "ARCH_PKG_LIST is $ARCH_PKG_LIST"; +echo "MACHINE_PKG_LIST is $MACHINE_PKG_LIST"; + +# Create the new directory for the feeds, making sure there is no previous copy +rm -r "$4"/ +rm "$4".zip +mkdir -p "$4"/ + +# Copy over the "all" packages +mkdir -p "$4"/all/ +echo "Copying packages to 'all' directory" +for i in $ALL_PKG_LIST; do + cp tmp/deploy/ipk/all/"$i" "$4"/all/ +done + +# Copy over the architecture packages +mkdir -p "$4"/$3/ +echo "Copying packages to '$3' directory" +for i in $ARCH_PKG_LIST; do + cp tmp/deploy/ipk/"$3"/"$i" "$4"/"$3"/ +done + +# Copy over the machine packages +mkdir -p "$4"/"$2"/ +echo "Copying packages to '$2' directory" +for i in $MACHINE_PKG_LIST; do + cp tmp/deploy/ipk/"$2"/"$i" "$4"/"$2"/ +done + +# Build the package index files +echo "Building package index files"
Quoted 8 lines
scripts/contrib/make_ipk_repo.sh line 86
Quoted 8 lines
scripts/contrib/make_ipk_repo.sh new file +8
-0+touch "$4"/Packages +flock "$4"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/Packages -p $4/Packages -m $4/" +touch "$4"/all/Packages +flock "$4"/all/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/all/Packages -p $4/all/Packages -m $4/all/" +touch "$4"/"$3"/Packages +flock "$4"/"$3"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/$3/Packages -p $4/$3/Packages -m $4/$3/" +touch "$4"/"$2"/Packages +flock "$4"/"$2"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/$2/Packages -p $4/$2/Packages -m $4/$2/"
This is also hardcoded to be 32 bit build systems :/. There are a number of hardcoded assumptions in the script.
Cheers,
Richard
Richard and Martin,
Quoted 16 lines
-----Original Message-----
From: Richard Purdie [mailto:[email protected]]
Sent: Tuesday, August 12, 2014 8:41 AM
To: Martin Jansa
Cc: Bryan Evenson; [email protected]
Subject: Re: [poky] [PATCH]scripts/contrib/make_ipk_repo.sh: Add new image-specific package feed generatorQuoted 7 lines · Martin Jansa
Quoted 6 lines · Bryan Evenson
Adding a new script which generates a reduced package feed directory. The script uses the list of installed packages for a specific build image and creates a new package feed directory with only the packages relevant to that image. This is useful for distributing firmware upgrades.
Quoted 7 lines
Quoted 5 lines
Quoted 3 lines
At this time the script only works for opkg repositories. Further modifications are needed for the script to work with other package management systems.
This belongs to oe-core ML
Agreed.
From browsing the the openembedded repository and the poky repository, I'd thought the poky mailing list was more appropriate on this item. I'll re-submit to the oe-core mailing list.
Quoted 102 lines
Quoted 5 lines
Quoted 5 lines
Signed-off-by Bryan Evenson <[email protected]>--- scripts/contrib/make_ipk_repo.sh | 93++++++++++++++++++++++++++++++++++++++
Quoted 17 lines
Quoted 17 lines
1 file changed, 93 insertions(+) create mode 100755 scripts/contrib/make_ipk_repo.shscripts/contrib/make_ipk_repo.sh new file +7
-0@@ -0,0 +1,93 @@ +#!/bin/bash + +# Creates a package feed directory that only contains the packages +that were # installed with the referenced image. This script +depends on image build # history to properly operate. In your +conf/local.conf, add the following two # lines: +# INHERIT += "buildhistory"Quoted 8 lines
scripts/contrib/make_ipk_repo.sh
Quoted 8 lines
scripts/contrib/make_ipk_repo.sh new file +8
-0+# BUILDHISTORY_COMMIT = "1" +# +# At this time this script is only setup to create opkg package feeds. +# Further script modification is needed if you use a different +package # managment system. +# +# This script assumes it is being run from your image build +directory. Some # paths may need to be modified if it is run from adifferent directory.
Quoted 5 lines
Quoted 5 lines
scripts/contrib/make_ipk_repo.sh new file +5
-0+# +# Assumes that the relevant image was recently built. If you have +done any # package builds since the last image build, it is +suggested that you re-build # relevant image. +#Quoted 8 lines
scripts/contrib/make_ipk_repo.sh line 28
Quoted 8 lines
scripts/contrib/make_ipk_repo.sh new file +8
-0+# Script input variables: +# $1 - The image name (i.e. "core-image-minimal") # $2 - The target +machine (i.e. "beaglebone") # $3 - The target architecture (i.e. +"arm926ejste") # $4 - The target directory (i.e. "feed") # + +# Validate the number of input variables if [[ "$#" -ne 4 ]]; then + echo "Unexpected number of arguments"; + echo "Usage: make_ipk_repo.sh IMAGE_NAME TARGET_MACHINE
TARGET_ARCHTARGET_DIR";Quoted 5 lines
scripts/contrib/make_ipk_repo.sh
Quoted 5 lines
scripts/contrib/make_ipk_repo.sh new file +5
-0+ exit 1; +fi + +# Some packages inexplicably contain a "1:" in the package name +that does not # belong there. Remove this text from these packagenames.
Quoted 8 lines
Quoted 8 lines
scripts/contrib/make_ipk_repo.sh new file +8
-0+ +sed -i 's/1://g' +tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt + +# Read in the list of packages used by the image. This includes +the # machine/architecture independent packages ("all"), the +architecture specific # packages, and the machine specific packages. +ALL_PKG_LIST=$(grep '_all\.ipk'Quoted 8 lines
scripts/contrib/make_ipk_repo.sh line 49
Quoted 8 lines
scripts/contrib/make_ipk_repo.sh new file +8
-0+tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) +ARCH_PKG_LIST=$(grep '_'"$3"'\.ipk' +tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) +MACHINE_PKG_LIST=$(grep '_'"$2"'\.ipk' +tmp/buildhistory/images/"$2"/eglibc/"$1"/installed-packages.txt) + +echo "ALL_PKG_LIST is $ALL_PKG_LIST"; echo "ARCH_PKG_LIST is +$ARCH_PKG_LIST"; echo "MACHINE_PKG_LIST is$MACHINE_PKG_LIST";
Quoted 17 lines
Quoted 17 lines
scripts/contrib/make_ipk_repo.sh new file +17
-0+ +# Create the new directory for the feeds, making sure there is no +previous copy rm -r "$4"/ rm "$4".zip mkdir -p "$4"/ + +# Copy over the "all" packages +mkdir -p "$4"/all/ +echo "Copying packages to 'all' directory" +for i in $ALL_PKG_LIST; do + cp tmp/deploy/ipk/all/"$i" "$4"/all/ done + +# Copy over the architecture packages mkdir -p "$4"/$3/ echo +"Copying packages to '$3' directory" +for i in $ARCH_PKG_LIST; do + cp tmp/deploy/ipk/"$3"/"$i" "$4"/"$3"/ done + +# Copy over the machine packages +mkdir -p "$4"/"$2"/Quoted 8 lines
scripts/contrib/make_ipk_repo.sh
Quoted 8 lines
scripts/contrib/make_ipk_repo.sh new file +8
-0+echo "Copying packages to '$2' directory" +for i in $MACHINE_PKG_LIST; do + cp tmp/deploy/ipk/"$2"/"$i" "$4"/"$2"/ done + +# Build the package index files +echo "Building package index files" +touch "$4"/Packages +flock "$4"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/Packages -p $4/Packages -m $4/"
Quoted 2 lines
scripts/contrib/make_ipk_repo.sh line 76
Quoted 2 lines
scripts/contrib/make_ipk_repo.sh new file +2
-0+touch "$4"/all/Packages +flock "$4"/all/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/all/Packages -p $4/all/Packages -m $4/all/"
Quoted 2 lines
scripts/contrib/make_ipk_repo.sh line 78
Quoted 2 lines
scripts/contrib/make_ipk_repo.sh new file +2
-0+touch "$4"/"$3"/Packages +flock "$4"/"$3"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/$3/Packages -p $4/$3/Packages -m $4/$3/"
Quoted 7 lines
Quoted 2 lines
scripts/contrib/make_ipk_repo.sh line 80
Quoted 2 lines
scripts/contrib/make_ipk_repo.sh new file +2
-0+touch "$4"/"$2"/Packages +flock "$4"/"$2"/Packages.flock -c "tmp/sysroots/i686-linux/usr/bin/opkg-make-index -r $4/$2/Packages -p $4/$2/Packages -m $4/$2/"
This is also hardcoded to be 32 bit build systems :/. There are a number of hardcoded assumptions in the script.
Agreed, I make a number of assumptions in the script and I list the assumptions that I knew I made. But, it’s a script I've found useful for my purposes and wanted to share with others. If you feel there is potential and if you have specific suggestions on what to change, I'll make the modifications before resubmitting.
Regards,
Bryan
Quoted 5 lines
Cheers,
Richard