image_types: Refactor compression code into a generic compression solution
PATCH 1/1 image_types: Refactor compression code into a generic compression solution
The current approach of adding each different compressed image type doesn't scale. This patch changes the code so compressed images for each form are automatically available using the form <type>.<compression type> in IMAGE_FSTYPES. This doesn't change any existing externally visible behaviour and the image generation process becomes more efficient as a result too.
---
meta/classes/image.bbclass +1 -3
@@ -75,9 +75,7 @@ inherit image-${IMAGE_TYPE} python () { deps = d.getVarFlag('do_rootfs', 'depends') or "" - for type in (d.getVar('IMAGE_FSTYPES', True) or "").split(): - for dep in ((d.getVar('IMAGE_DEPENDS_%s' % type) or "").split() or []): - deps += " %s:do_populate_sysroot" % dep + deps += imagetypes_getdepends(d) for dep in (d.getVar('EXTRA_IMAGEDEPENDS', True) or "").split(): deps += " %s:do_populate_sysroot" % dep d.setVarFlag('do_rootfs', 'depends', deps)
meta/classes/image_types.bbclass +67 -64
206 lines not on this page · raw message
Op 27 feb. 2012, om 13:43 heeft Richard Purdie het volgende geschreven:
Quoted 9 lines
The current approach of adding each different compressed image type doesn't scale. This patch changes the code so compressed images for each form are automatically available using the form <type>.<compression type> in
IMAGE_FSTYPES. This doesn't change any existing externally visible behaviour and the image generation process becomes more efficient as a result too.Signed-off-by Richard Purdie <[email protected]>---
meta/classes/image.bbclass
Quoted 10 lines
meta/classes/image.bbclass +9
-0+COMPRESSIONTYPES = "gz bz2 lzma xy" +COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz" +COMPRESS_CMD_bz2 = "bzip2 -k ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_CMD_xy = "xz -k -c ${XZ_COMPRESSION_LEVEL} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_DEPENDS_lzma = "xz-native" +COMPRESS_DEPENDS_gz = "" +COMPRESS_DEPENDS_bz2 = "" +COMPRESS_DEPENDS_xy = "xz-native"
s/xy/xz/g
regrds,
koen
Quoted 232 lines · Richard Purdie
The current approach of adding each different compressed image type doesn't scale. This patch changes the code so compressed images for each form are automatically available using the form <type>.<compression type> in
IMAGE_FSTYPES. This doesn't change any existing externally visible behaviour and the image generation process becomes more efficient as a result too.Signed-off-by Richard Purdie <[email protected]>---
meta/classes/image.bbclass +0
-0@@ -75,9 +75,7 @@ inherit image-${IMAGE_TYPE} python () {deps =
d.getVarFlag('do_rootfs', 'depends')or ""
- for type in (d.getVar('IMAGE_FSTYPES', True)or "").split():
- for dep in ((d.getVar('IMAGE_DEPENDS_%s' % type)or "").split() or []):
- deps += " %s:do_populate_sysroot" % dep
+ deps +=imagetypes_getdepends(d)
for dep in (d.getVar('EXTRA_IMAGEDEPENDS', True)or "").split():
deps += " %s:do_populate_sysroot" % depd.setVarFlag('do_rootfs', 'depends', deps)meta/classes/image_types.bbclass +0
-0@@ -2,7 +2,21 @@ def get_imagecmds(d):cmds = "\n"
old_overrides=d.getVar('OVERRIDES', 0)+ alltypes =
d.getVar('IMAGE_FSTYPES', True).split() types =d.getVar('IMAGE_FSTYPES', True).split()
+ ctypes =d.getVar('COMPRESSIONTYPES', True).split()
+ cimages = {}meta/classes/image_types.bbclass +11
-0+ + # Filter out all the compressed images from types + for type in types: + for ctype in ctypes: + if type.endswith("." + ctype): + basetype = type.rsplit(".", 1)[0] + types[types.index(type)] = basetype + if type not in cimages: + cimages[basetype] = [] + cimages[basetype].append(ctype) +# Live images will be processed via inheriting bbclass and
# does not get processed here. # live images also depend on ext3 so ensure its presentmeta/classes/image_types.bbclass +1
-0@@ -11,11 +25,24 @@ def get_imagecmds(d): types.append("ext3") types.remove("live") + cmds += " rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.*"for type in types:
+ ccmd = []
+ subimages = []
localdata =bb.data.createCopy(d)localdata.setVar('OVERRIDES', '%s:%s' % (type, old_overrides))bb.data.update_data(localdata)
localdata.setVar('type', type)
+ if type in cimages:
+ for ctype in cimages[type]:
+ ccmd.append("\t" +localdata.getVar("COMPRESS_CMD_" + ctype, True))
+ subimages.append(type + "." + ctype)
+ if type not in alltypes:
+ ccmd.append(localdata.expand("\trm ${IMAGE_NAME}.rootfs.${type}"))
+ else:
+ subimages.append(type)
+ localdata.setVar('ccmd', "\n".join(ccmd))
+ localdata.setVar('subimages', " ".join(subimages))
cmd =localdata.getVar("IMAGE_CMD", True)localdata.setVar('cmd', cmd)
cmds +=localdata.getVar("runimagecmd", True)meta/classes/image_types.bbclass +0
-0@@ -25,10 +52,36 @@ runimagecmd () { # Image generation code for image type ${type}ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{
base_size= ($1 * ${IMAGE_OVERHEAD_FACTOR}); OFMT = "%.0f" ; print ((base_size> ${IMAGE_ROOTFS_SIZE} ?base_size: ${IMAGE_ROOTFS_SIZE}) + ${IMAGE_ROOTFS_EXTRA_SPACE}) }'` ${cmd}
+ # Now create the needed compressed versions
cd ${DEPLOY_DIR_IMAGE}/
- rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
- ln -s ${IMAGE_NAME}.rootfs.${type} ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}meta/classes/image_types.bbclass +28
-1-} + ${ccmd} + # And create the symlinks + for type in ${subimages}; do + ln -s ${IMAGE_NAME}.rootfs.$type ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.$type + done +} + +def imagetypes_getdepends(d): + def adddep(depstr, deps): + for i in (depstr or "").split(): + if i not in deps: + deps.append(i) + + deps = [] + ctypes = d.getVar('COMPRESSIONTYPES', True).split() + for type in (d.getVar('IMAGE_FSTYPES', True) or "").split(): + basetype = type + for ctype in ctypes: + if type.endswith("." + ctype): + basetype = type.rsplit(".", 1)[0] + adddep(d.getVar("COMPRESS_DEPENDS_%s" % ctype, True), deps) + break + adddep(d.getVar('IMAGE_DEPENDS_%s' % basetype, True) , deps) + + depstr = "" + for dep in deps: + depstr += " " + dep + ":do_populate_sysroot" + return depstrXZ_COMPRESSION_LEVEL ?= "-e -9"
meta/classes/image_types.bbclass +0
-0@@ -46,40 +99,11 @@ IMAGE_CMD_ext2 () {mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2 ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2 rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}
}meta/classes/image_types.bbclass +0
-21-IMAGE_CMD_ext2.gz () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} - genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2 - gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} -} -IMAGE_CMD_ext2.bz2 () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz - genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2 - bzip2 -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.bz2 ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.bz2 - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz -} -IMAGE_CMD_ext2.lzma () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz - genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2 - lzma -f -7 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.lzma ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.lzma - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz -}IMAGE_CMD_ext3 () {
genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3 tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
}meta/classes/image_types.bbclass +0
-8-IMAGE_CMD_ext3.gz () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} - genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext3 - tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext3 - gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext3 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} -}oe_mkext4fs () {
genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} $1meta/classes/image_types.bbclass +0
-0@@ -97,13 +121,6 @@ oe_mkext4fs () { IMAGE_CMD_ext4 () {oe_mkext4fs ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext4 }
meta/classes/image_types.bbclass +0
-7-IMAGE_CMD_ext4.gz () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} - oe_mkext4fs ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext4 - gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext4 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext4.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext4.gz - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} -}IMAGE_CMD_btrfs () {
mkfs.btrfs -bexpr ${ROOTFS_SIZE} \* 1024${EXTRA_IMAGECMD} -r ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.btrfsmeta/classes/image_types.bbclass +0
-0@@ -112,9 +129,6 @@ IMAGE_CMD_btrfs () {IMAGE_CMD_squashfs = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.squashfs ${EXTRA_IMAGECMD} -noappend" IMAGE_CMD_squashfs-lzma = "mksquashfs-lzma ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.squashfs-lzma ${EXTRA_IMAGECMD} -noappend"
IMAGE_CMD_tar= "cd ${IMAGE_ROOTFS} && tar -cvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar ."meta/classes/image_types.bbclass +0
-3-IMAGE_CMD_tar.gz = "cd ${IMAGE_ROOTFS} && tar -zcvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar.gz ." -IMAGE_CMD_tar.bz2 = "cd ${IMAGE_ROOTFS} && tar -jcvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar.bz2 ." -IMAGE_CMD_tar.xz = "cd ${IMAGE_ROOTFS} && tar --xz -cvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar.xz ."CPIO_TOUCH_INIT () {
if [ ! -L ${IMAGE_ROOTFS}/init ]meta/classes/image_types.bbclass +0
-12@@ -126,18 +140,6 @@ IMAGE_CMD_cpio () { ${CPIO_TOUCH_INIT} cd ${IMAGE_ROOTFS} && (find . | cpio -o -H newc >${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio) } -IMAGE_CMD_cpio.gz () { - ${CPIO_TOUCH_INIT} - cd ${IMAGE_ROOTFS} && (find . | cpio -o -H newc | gzip -c -9 >${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.gz) -} -IMAGE_CMD_cpio.xz () { - ${CPIO_TOUCH_INIT} - cd ${IMAGE_ROOTFS} && (find . | cpio -o -H newc | xz -c ${XZ_COMPRESSION_LEVEL} --check=${XZ_INTEGRITY_CHECK} > ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.xz) ${EXTRA_IMAGECMD} -} -IMAGE_CMD_cpio.lzma () { - ${CPIO_TOUCH_INIT} - cd ${IMAGE_ROOTFS} && (find . | cpio -o -H newc | xz --format=lzma -c ${XZ_COMPRESSION_LEVEL} --check=${XZ_INTEGRITY_CHECK} >${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.lzma) ${EXTRA_IMAGECMD} -}UBI_VOLNAME ?= "${MACHINE}-rootfs"
meta/classes/image_types.bbclass +0
-0@@ -159,9 +161,7 @@ EXTRA_IMAGECMD = ""EXTRA_IMAGECMD_jffs2 ?= "--pad --little-endian --eraseblock=0x40000"
# Change these if you want default genext2fs behavior (i.e. create minimal inode number)EXTRA_IMAGECMD_ext2?= "-i 8192"
-EXTRA_IMAGECMD_ext2.gz ?= "-i 8192"
EXTRA_IMAGECMD_ext3 ?= "-i 8192"
-EXTRA_IMAGECMD_ext3.gz ?= "-i 8192"
EXTRA_IMAGECMD_btrfs ?= ""IMAGE_DEPENDS = ""
meta/classes/image_types.bbclass +0
-0@@ -169,22 +169,25 @@ IMAGE_DEPENDS_jffs2 = "mtd-utils-native"IMAGE_DEPENDS_sum.jffs2 = "mtd-utils-native"
IMAGE_DEPENDS_cramfs = "cramfs-native"
IMAGE_DEPENDS_ext2 = "genext2fs-native"meta/classes/image_types.bbclass +0
-3-IMAGE_DEPENDS_ext2.gz = "genext2fs-native" -IMAGE_DEPENDS_ext2.bz2 = "genext2fs-native" -IMAGE_DEPENDS_ext2.lzma = "genext2fs-native xz-native"IMAGE_DEPENDS_ext3 = "genext2fs-native e2fsprogs-native" -IMAGE_DEPENDS_ext3.gz = "genext2fs-native e2fsprogs-native"
IMAGE_DEPENDS_ext4= "genext2fs-native e2fsprogs-native" -IMAGE_DEPENDS_ext4.gz = "genext2fs-native e2fsprogs-native"IMAGE_DEPENDS_btrfs= "btrfs-tools-native"IMAGE_DEPENDS_squashfs= "squashfs-tools-native" IMAGE_DEPENDS_squashfs-lzma = "squashfs-lzma-tools-native"meta/classes/image_types.bbclass +0
-3-IMAGE_DEPENDS_tar.xz = "tar-native xz-native" -IMAGE_DEPENDS_cpio.lzma = "xz-native" -IMAGE_DEPENDS_cpio.xz = "xz-native"IMAGE_DEPENDS_ubi = "mtd-utils-native"
IMAGE_DEPENDS_ubifs = "mtd-utils-native"
IMAGE_DEPENDS_vmdk = "qemu-native"# This variable is available to request which values are suitable for
IMAGE_FSTYPES
IMAGE_TYPES = "jffs2 sum.jffs2 cramfs ext2 ext2.gz ext2.bz2 ext3 ext3.gz ext2.lzma btrfs live squashfs squashfs-lzma ubi tar tar.gz tar.bz2 tar.xz cpio cpio.gz cpio.xz cpio.lzma vmdk"meta/classes/image_types.bbclass +1
-0+
meta/classes/image_types.bbclass IMAGE_DEPENDS_jffs2 = "mtd-utils-native" · line 193
Quoted 10 lines
meta/classes/image_types.bbclass +10
-0+COMPRESSIONTYPES = "gz bz2 lzma xy" +COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz" +COMPRESS_CMD_bz2 = "bzip2 -k ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_CMD_xy = "xz -k -c ${XZ_COMPRESSION_LEVEL} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_DEPENDS_lzma = "xz-native" +COMPRESS_DEPENDS_gz = "" +COMPRESS_DEPENDS_bz2 = "" +COMPRESS_DEPENDS_xy = "xz-native" +
Nice patch,
I've spotted a minor issue, just needing s/xy/xz/
meta/classes/image_types.bbclass
Quoted 2 lines
meta/classes/image_types.bbclass +2
-0+COMPRESS_CMD_xy = "xz -k -c ${XZ_COMPRESSION_LEVEL} +COMPRESS_DEPENDS_xy = "xz-native"
Cheers
Andrea
Quoted 7 lines
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Quoted 248 lines · Andrea Adami
Quoted 232 lines · Richard Purdie
The current approach of adding each different compressed image type doesn't scale. This patch changes the code so compressed images for each form are automatically available using the form <type>.<compression type> in
IMAGE_FSTYPES. This doesn't change any existing externally visible behaviour and the image generation process becomes more efficient as a result too.Signed-off-by Richard Purdie <[email protected]>---
meta/classes/image.bbclass +0
-0@@ -75,9 +75,7 @@ inherit image-${IMAGE_TYPE} python () {deps =
d.getVarFlag('do_rootfs', 'depends')or ""
- for type in (d.getVar('IMAGE_FSTYPES', True)or "").split():
- for dep in ((d.getVar('IMAGE_DEPENDS_%s' % type)or "").split() or []):
- deps += " %s:do_populate_sysroot" % dep
+ deps +=imagetypes_getdepends(d)
for dep in (d.getVar('EXTRA_IMAGEDEPENDS', True)or "").split():
deps += " %s:do_populate_sysroot" % depd.setVarFlag('do_rootfs', 'depends', deps)meta/classes/image_types.bbclass +0
-0@@ -2,7 +2,21 @@ def get_imagecmds(d):cmds = "\n"
old_overrides=d.getVar('OVERRIDES', 0)+ alltypes =
d.getVar('IMAGE_FSTYPES', True).split() types =d.getVar('IMAGE_FSTYPES', True).split()
+ ctypes =d.getVar('COMPRESSIONTYPES', True).split()
+ cimages = {}meta/classes/image_types.bbclass +11
-0+ + # Filter out all the compressed images from types + for type in types: + for ctype in ctypes: + if type.endswith("." + ctype): + basetype = type.rsplit(".", 1)[0] + types[types.index(type)] = basetype + if type not in cimages: + cimages[basetype] = [] + cimages[basetype].append(ctype) +# Live images will be processed via inheriting bbclass and
# does not get processed here. # live images also depend on ext3 so ensure its presentmeta/classes/image_types.bbclass +1
-0@@ -11,11 +25,24 @@ def get_imagecmds(d): types.append("ext3") types.remove("live") + cmds += " rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.*"for type in types:
+ ccmd = []
+ subimages = []
localdata =bb.data.createCopy(d)localdata.setVar('OVERRIDES', '%s:%s' % (type, old_overrides))bb.data.update_data(localdata)
localdata.setVar('type', type)
+ if type in cimages:
+ for ctype in cimages[type]:
+ ccmd.append("\t" +localdata.getVar("COMPRESS_CMD_" + ctype, True))
+ subimages.append(type + "." + ctype)
+ if type not in alltypes:
+ ccmd.append(localdata.expand("\trm ${IMAGE_NAME}.rootfs.${type}"))
+ else:
+ subimages.append(type)
+ localdata.setVar('ccmd', "\n".join(ccmd))
+ localdata.setVar('subimages', " ".join(subimages))
cmd =localdata.getVar("IMAGE_CMD", True)localdata.setVar('cmd', cmd)
cmds +=localdata.getVar("runimagecmd", True)meta/classes/image_types.bbclass +0
-0@@ -25,10 +52,36 @@ runimagecmd () { # Image generation code for image type ${type}ROOTFS_SIZE=`du -ks ${IMAGE_ROOTFS}|awk '{
base_size= ($1 * ${IMAGE_OVERHEAD_FACTOR}); OFMT = "%.0f" ; print ((base_size> ${IMAGE_ROOTFS_SIZE} ?base_size: ${IMAGE_ROOTFS_SIZE}) + ${IMAGE_ROOTFS_EXTRA_SPACE}) }'` ${cmd}
+ # Now create the needed compressed versions
cd ${DEPLOY_DIR_IMAGE}/
- rm -f ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}
- ln -s ${IMAGE_NAME}.rootfs.${type} ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.${type}meta/classes/image_types.bbclass +28
-1-} + ${ccmd} + # And create the symlinks + for type in ${subimages}; do + ln -s ${IMAGE_NAME}.rootfs.$type ${DEPLOY_DIR_IMAGE}/${IMAGE_LINK_NAME}.$type + done +} + +def imagetypes_getdepends(d): + def adddep(depstr, deps): + for i in (depstr or "").split(): + if i not in deps: + deps.append(i) + + deps = [] + ctypes = d.getVar('COMPRESSIONTYPES', True).split() + for type in (d.getVar('IMAGE_FSTYPES', True) or "").split(): + basetype = type + for ctype in ctypes: + if type.endswith("." + ctype): + basetype = type.rsplit(".", 1)[0] + adddep(d.getVar("COMPRESS_DEPENDS_%s" % ctype, True), deps) + break + adddep(d.getVar('IMAGE_DEPENDS_%s' % basetype, True) , deps) + + depstr = "" + for dep in deps: + depstr += " " + dep + ":do_populate_sysroot" + return depstrXZ_COMPRESSION_LEVEL ?= "-e -9"
meta/classes/image_types.bbclass +0
-0@@ -46,40 +99,11 @@ IMAGE_CMD_ext2 () {mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2 ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2 rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}
}meta/classes/image_types.bbclass +0
-21-IMAGE_CMD_ext2.gz () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} - genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2 - gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext2.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.gz - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} -} -IMAGE_CMD_ext2.bz2 () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz - genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2 - bzip2 -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.bz2 ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.bz2 - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz -} -IMAGE_CMD_ext2.lzma () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz - genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2 - lzma -f -7 ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz/${IMAGE_NAME}.rootfs.ext2.lzma ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext2.lzma - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz -}IMAGE_CMD_ext3 () {
genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3 tune2fs -j ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3
}meta/classes/image_types.bbclass +0
-8-IMAGE_CMD_ext3.gz () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} - genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext3 - tune2fs -j ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext3 - gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext3 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext3.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext3.gz - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} -}oe_mkext4fs () {
genext2fs -b $ROOTFS_SIZE -i 4096 -d ${IMAGE_ROOTFS} ${EXTRA_IMAGECMD} $1meta/classes/image_types.bbclass +0
-0@@ -97,13 +121,6 @@ oe_mkext4fs () { IMAGE_CMD_ext4 () {oe_mkext4fs ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext4 }
meta/classes/image_types.bbclass +0
-7-IMAGE_CMD_ext4.gz () { - rm -rf ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} && mkdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} - oe_mkext4fs ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext4 - gzip -f -9 ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext4 - mv ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN}/${IMAGE_NAME}.rootfs.ext4.gz ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.ext4.gz - rmdir ${DEPLOY_DIR_IMAGE}/tmp.gz-${PN} -}IMAGE_CMD_btrfs () {
mkfs.btrfs -bexpr ${ROOTFS_SIZE} \* 1024${EXTRA_IMAGECMD} -r ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.btrfsmeta/classes/image_types.bbclass +0
-0@@ -112,9 +129,6 @@ IMAGE_CMD_btrfs () {IMAGE_CMD_squashfs = "mksquashfs ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.squashfs ${EXTRA_IMAGECMD} -noappend" IMAGE_CMD_squashfs-lzma = "mksquashfs-lzma ${IMAGE_ROOTFS} ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.squashfs-lzma ${EXTRA_IMAGECMD} -noappend"
IMAGE_CMD_tar= "cd ${IMAGE_ROOTFS} && tar -cvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar ."meta/classes/image_types.bbclass +0
-3-IMAGE_CMD_tar.gz = "cd ${IMAGE_ROOTFS} && tar -zcvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar.gz ." -IMAGE_CMD_tar.bz2 = "cd ${IMAGE_ROOTFS} && tar -jcvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar.bz2 ." -IMAGE_CMD_tar.xz = "cd ${IMAGE_ROOTFS} && tar --xz -cvf ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.tar.xz ."CPIO_TOUCH_INIT () {
if [ ! -L ${IMAGE_ROOTFS}/init ]meta/classes/image_types.bbclass +0
-12@@ -126,18 +140,6 @@ IMAGE_CMD_cpio () { ${CPIO_TOUCH_INIT} cd ${IMAGE_ROOTFS} && (find . | cpio -o -H newc >${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio) } -IMAGE_CMD_cpio.gz () { - ${CPIO_TOUCH_INIT} - cd ${IMAGE_ROOTFS} && (find . | cpio -o -H newc | gzip -c -9 >${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.gz) -} -IMAGE_CMD_cpio.xz () { - ${CPIO_TOUCH_INIT} - cd ${IMAGE_ROOTFS} && (find . | cpio -o -H newc | xz -c ${XZ_COMPRESSION_LEVEL} --check=${XZ_INTEGRITY_CHECK} > ${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.xz) ${EXTRA_IMAGECMD} -} -IMAGE_CMD_cpio.lzma () { - ${CPIO_TOUCH_INIT} - cd ${IMAGE_ROOTFS} && (find . | cpio -o -H newc | xz --format=lzma -c ${XZ_COMPRESSION_LEVEL} --check=${XZ_INTEGRITY_CHECK} >${DEPLOY_DIR_IMAGE}/${IMAGE_NAME}.rootfs.cpio.lzma) ${EXTRA_IMAGECMD} -}UBI_VOLNAME ?= "${MACHINE}-rootfs"
meta/classes/image_types.bbclass +0
-0@@ -159,9 +161,7 @@ EXTRA_IMAGECMD = ""EXTRA_IMAGECMD_jffs2 ?= "--pad --little-endian --eraseblock=0x40000"
# Change these if you want default genext2fs behavior (i.e. create minimal inode number)EXTRA_IMAGECMD_ext2?= "-i 8192"
-EXTRA_IMAGECMD_ext2.gz ?= "-i 8192"
EXTRA_IMAGECMD_ext3 ?= "-i 8192"
-EXTRA_IMAGECMD_ext3.gz ?= "-i 8192"
EXTRA_IMAGECMD_btrfs ?= ""IMAGE_DEPENDS = ""
meta/classes/image_types.bbclass +0
-0@@ -169,22 +169,25 @@ IMAGE_DEPENDS_jffs2 = "mtd-utils-native"IMAGE_DEPENDS_sum.jffs2 = "mtd-utils-native"
IMAGE_DEPENDS_cramfs = "cramfs-native"
IMAGE_DEPENDS_ext2 = "genext2fs-native"meta/classes/image_types.bbclass +0
-3-IMAGE_DEPENDS_ext2.gz = "genext2fs-native" -IMAGE_DEPENDS_ext2.bz2 = "genext2fs-native" -IMAGE_DEPENDS_ext2.lzma = "genext2fs-native xz-native"IMAGE_DEPENDS_ext3 = "genext2fs-native e2fsprogs-native" -IMAGE_DEPENDS_ext3.gz = "genext2fs-native e2fsprogs-native"
IMAGE_DEPENDS_ext4= "genext2fs-native e2fsprogs-native" -IMAGE_DEPENDS_ext4.gz = "genext2fs-native e2fsprogs-native"IMAGE_DEPENDS_btrfs= "btrfs-tools-native"IMAGE_DEPENDS_squashfs= "squashfs-tools-native" IMAGE_DEPENDS_squashfs-lzma = "squashfs-lzma-tools-native"meta/classes/image_types.bbclass +0
-3-IMAGE_DEPENDS_tar.xz = "tar-native xz-native" -IMAGE_DEPENDS_cpio.lzma = "xz-native" -IMAGE_DEPENDS_cpio.xz = "xz-native"IMAGE_DEPENDS_ubi = "mtd-utils-native"
IMAGE_DEPENDS_ubifs = "mtd-utils-native"
IMAGE_DEPENDS_vmdk = "qemu-native"# This variable is available to request which values are suitable for
IMAGE_FSTYPES
IMAGE_TYPES = "jffs2 sum.jffs2 cramfs ext2 ext2.gz ext2.bz2 ext3 ext3.gz ext2.lzma btrfs live squashfs squashfs-lzma ubi tar tar.gz tar.bz2 tar.xz cpio cpio.gz cpio.xz cpio.lzma vmdk"meta/classes/image_types.bbclass +1
-0+meta/classes/image_types.bbclass IMAGE_DEPENDS_jffs2 = "mtd-utils-native" · line 193
Quoted 10 lines
meta/classes/image_types.bbclass +10
-0+COMPRESSIONTYPES = "gz bz2 lzma xy" +COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}.rootfs.${type} > ${IMAGE_NAME}.rootfs.${type}.gz" +COMPRESS_CMD_bz2 = "bzip2 -k ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_CMD_xy = "xz -k -c ${XZ_COMPRESSION_LEVEL} --check=${XZ_INTEGRITY_CHECK} ${IMAGE_NAME}.rootfs.${type}" +COMPRESS_DEPENDS_lzma = "xz-native" +COMPRESS_DEPENDS_gz = "" +COMPRESS_DEPENDS_bz2 = "" +COMPRESS_DEPENDS_xy = "xz-native" +Nice patch,
I've spotted a minor issue, just needing s/xy/xz/
Quoted 3 lines
meta/classes/image_types.bbclass
Quoted 2 lines
meta/classes/image_types.bbclass +2
-0+COMPRESS_CMD_xy = "xz -k -c ${XZ_COMPRESSION_LEVEL} +COMPRESS_DEPENDS_xy = "xz-native"
..and one more: +COMPRESSIONTYPES = "gz bz2 lzma xy"
Quoted 11 lines
Cheers
Andrea
Quoted 7 lines
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Op 27 feb. 2012, om 13:43 heeft Richard Purdie het volgende geschreven:
Quoted 5 lines
The current approach of adding each different compressed image type doesn't scale. This patch changes the code so compressed images for each form are automatically available using the form <type>.<compression type> in
IMAGE_FSTYPES. This doesn't change any existing externally visible behaviour and the image generation process becomes more efficient as a result too.
There's a a few -f's missing in this patch, e.g. bz2 images now fail with:
bzip2: ERROR: Function failed: do_rootfs (see /OE/tentacle/build/tmp-angstrom_2010_x-eglibc/work/beagleboard-angstrom-linux-gnueabi/systemd-image-1.0-r0/temp/log.do_rootfs.3174 for further information) Output file Angstrom-systemd-image-eglibc-ipk-v2012.02-core-beagleboard.rootfs.tar.bz2 already exists.
A quick look suggests at least bz2 and xz images can't get rebuilt.
regards,
Koen
Quoted 9 lines · Koen Kooi
Op 27 feb. 2012, om 13:43 heeft Richard Purdie het volgende geschreven:
Quoted 5 lines
The current approach of adding each different compressed image type doesn't scale. This patch changes the code so compressed images for each form are automatically available using the form <type>.<compression type> in
IMAGE_FSTYPES. This doesn't change any existing externally visible behaviour and the image generation process becomes more efficient as a result too.
Quoted 7 lines
There's a a few -f's missing in this patch, e.g. bz2 images now fail with:
bzip2: ERROR: Function failed:
do_rootfs(see /OE/tentacle/build/tmp-angstrom_2010_x-eglibc/work/beagleboard-angstrom-linux-gnueabi/systemd-image-1.0-r0/temp/log.do_rootfs.3174 for further information) Output file Angstrom-systemd-image-eglibc-ipk-v2012.02-core-beagleboard.rootfs.tar.bz2 already exists.A quick look suggests at least bz2 and xz images can't get rebuilt.
Issues with cpio.lzma as well:
| ln: failed to create symbolic link `/oe/oe-core/build/tmp-eglibc/deploy/images/initramfs-kexecboot-klibc-image-poodle.cpio.lzma':
File exists
Looking at run.do_rootfs I see runimagecmd is called twice.
In the log.do_rootfs there is also a strange
+ sed sERROR: Function failed: do_rootfs (see /oe/oe-core/build/tmp-eglibc/work/poodle-oe-linux-gnueabi/initramfs-kexecboot-klibc-image-1.0-r0/temp/log.do_rootfs.31195 for further information)
/generic_//g
Regards
Andrea
Quoted 7 lines
regards,
Koen
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core
Quoted 7 lines · Andrea Adami
| ln: failed to create symbolic link `/oe/oe-core/build/tmp-eglibc/deploy/images/initramfs-kexecboot-klibc-image-poodle.cpio.lzma':
File existsLooking at run.do_rootfs I see runimagecmd is called twice.
In the log.do_rootfs there is also a strange
I found many issues with shell variable expanding in current bitbake; I described the issue in #yocto but it seems I am the only one to have those problems.
Basically it seems:
export FOO=bar
Parses fine but /don't work/.
export FOO="bar"
Works.
Signature · 3 lines
Otavio Salvador O.S. Systems E-mail: [email protected] http://www.ossystems.com.br Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br
Quoted 7 lines · Otavio Salvador
Quoted 6 lines · Andrea Adami
| ln: failed to create symbolic link `/oe/oe-core/build/tmp-eglibc/deploy/images/initramfs-kexecboot-klibc-image-poodle.cpio.lzma':
File existsLooking at run.do_rootfs I see runimagecmd is called twice.
Note that I set INITRAMFS_FSTYPES ?= "cpio.gz cpio.lzma" If I only have one single fstype (cpio.lzma) all is fine.
Cheers
Andrea
Quoted 26 lines
Quoted 1 line
In the log.do_rootfs there is also a strange
I found many issues with shell variable expanding in current bitbake; I described the issue in #yocto but it seems I am the only one to have those problems.
Basically it seems:
export FOO=bar
Parses fine but /don't work/.
export FOO="bar"
Works.
--
Otavio Salvador O.S. Systems
E-mail: [email protected] http://www.ossystems.com.br
Mobile: +55 53 9981-7854 http://projetos.ossystems.com.br_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.linuxtogo.org/cgi-bin/mailman/listinfo/openembedded-core