bmaptool support
cover bmaptool support
Hi,
This patchset adds ability to use bmaptool to flash images faster than using traditional methods.
Bmaptool is a generic tool for creating the block map (bmap) for a file and and copying files using the block map. The idea is that large file containing unused blocks, like raw system image files, can be copied or flashed a lot faster with bmaptool than with traditional tools like "dd" or "cp".
Here is an example of flashing image using dd and bmaptool showing around 25% performance gain in flashing time:
Quoted 1 line
ls -slh
19M -rw-r--r-- 1 ed ed 26M Apr 26 11:36 core-image-minimal-qemux86-64-20160426083554.rootfs.ext4
4.0K -rw-r--r-- 1 ed ed 2.7K Apr 26 11:36 core-image-minimal-qemux86-64-20160426083554.rootfs.ext4.bmap
Quoted 1 line
time dd if=core-image-minimal-qemux86-64-20160426083554.rootfs.ext4 of=/dev/sdb
52302+0 records in
52302+0 records out
26778624 bytes (27 MB) copied, 2.51183 s, 10.7 MB/s
real 0m2.515s user 0m0.014s sys 0m0.329s
Quoted 1 line
time bmaptool copy core-image-minimal-qemux86-64-20160426083554.rootfs.ext4
/dev/sdb
bmaptool: info: discovered bmap file 'core-image-minimal-qemux86-64-20160426083554.rootfs.ext4.bmap'
bmaptool: info: block map format version 2.0
bmaptool: info: 6538 blocks of size 4096 (25.5 MiB), mapped 4799 blocks (18.7 MiB or 73.4%)
bmaptool: info: copying image 'core-image-minimal-qemux86-64-20160426083554.rootfs.ext4' to block device '/dev/sdb' using bmap file 'core-image-minimal-qemux86-64-20160426083554.rootfs.ext4.bmap'
bmaptool: info: 100% copied
bmaptool: info: synchronizing '/dev/sdb'
bmaptool: info: copying time: 1.8s, copying speed 10.3 MiB/sec
real 0m1.909s user 0m0.226s sys 0m0.052s
The test was repeated 5 times with the same image and the same usb stick device.
The following changes since commit a9b503b268e94d311f892fa00c5d6bd9ffdb228e:
license.bbclass: make sure that image manifest dir exists (2016-04-22 16:28:57 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/oe-core/bmap-tools-9414
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/oe-core/bmap-tools-9414
Alexander D. Kanevskiy (2):
bmap-tools: utility to create block map files
image types: add bmap generation option
Ed Bartosh (1):
selftest: add bmap test
meta/classes/image_types.bbclass | 4 ++- meta/lib/oeqa/selftest/imagefeatures.py | 27 +++++++++++++++ meta/recipes-support/bmap-tools/bmap-tools_3.2.bb | 41 +++++++++++++++++++++++ 3 files changed, 71 insertions(+), 1 deletion(-) create mode 100644 meta/recipes-support/bmap-tools/bmap-tools_3.2.bb
--
Regards,
Ed
PATCH 1/3 bmap-tools: utility to create block map files
From: "Alexander D. Kanevskiy" <[email protected]>
Bmap-tools - tools to generate block map (AKA bmap) and flash images using bmap. Bmaptool is a generic tool for creating the block map (bmap) for a file and copying files using the block map.
The idea is that large file containing unused blocks, like raw system image files, can be copied or flashed a lot faster with bmaptool than with traditional tools like "dd" or "cp".
Standalone version of the tool provided in deploy/tools directory and can be easily used on any Linux system with Python 2.7+ installed.
[YOCTO #9414]
--- meta/recipes-support/bmap-tools/bmap-tools_3.2.bb | 41 +++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 meta/recipes-support/bmap-tools/bmap-tools_3.2.bb
meta/recipes-support/bmap-tools/bmap-tools_3.2.bb new file +41 -0
@@ -0,0 +1,41 @@ +SUMMARY = "Tools to generate block map (AKA bmap) and flash images using bmap" +DESCRIPTION = "Bmap-tools - tools to generate block map (AKA bmap) and flash images using \ +bmap. Bmaptool is a generic tool for creating the block map (bmap) for a file, \ +and copying files using the block map. The idea is that large file containing \ +unused blocks, like raw system image files, can be copied or flashed a lot \ +faster with bmaptool than with traditional tools like "dd" or "cp"." +HOMEPAGE = "http://git.infradead.org/users/dedekind/bmap-tools.git" +SECTION = "devel/python" +LICENSE = "GPLv2" +LIC_FILES_CHKSUM = "file://COPYING;md5=b234ee4d69f5fce4486a80fdaf4a4263" + +SRC_URI = "ftp://ftp.infradead.org/pub/bmap-tools/bmap-tools-${PV}.tgz" +SRC_URI[md5sum] = "92cdad1cb4dfa0cca7176c8e22752616" +SRC_URI[sha256sum] = "cc6c7f7dc0a37e2a32deb127308e24e6c4b80bfb54f3803c308efab02bf2d434" + +inherit setuptools deploy + +RDEPENDS_${PN} += "python-compression python-core python-unittest" + +BBCLASSEXTEND = "nativesdk native" + +do_install_append_class-native() { + sed -i -e 's|^#!.*/usr/bin/env python|#! /usr/bin/env nativepython|' ${D}${bindir}/bmaptool +} + +do_deploy[sstate-outputdirs] = "${DEPLOY_DIR_TOOLS}" +do_deploy_class-native() { + cp bmaptool __main__.py + python -m zipfile -c bmaptool.zip bmaptools __main__.py + echo '#!/usr/bin/env python' | cat - bmaptool.zip > bmaptool-standalone + install -d ${DEPLOYDIR} + install -m 0755 bmaptool-standalone ${DEPLOYDIR}/bmaptool-${PV} + rm -f ${DEPLOYDIR}/bmaptool + ln -sf ./bmaptool-${PV} ${DEPLOYDIR}/bmaptool +} + +do_deploy() { + : +} + +addtask deploy before do_package after do_install
Signature · 1 line
2.1.4
PATCH 2/3 image types: add bmap generation option
In reply to Ed BartoshFrom: "Alexander D. Kanevskiy" <[email protected]>
bmap image conversion type allows to create block map files for sparse images. Bmap file can be used together with bmap-tools for efficiently flash images to raw devices (hdd or usb drive)
[YOCTO #9414]
--- meta/classes/image_types.bbclass | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-)
meta/classes/image_types.bbclass +3 -1
@@ -262,7 +262,7 @@ IMAGE_TYPES = " \ wic wic.gz wic.bz2 wic.lzma \ " -COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum md5sum sha1sum sha224sum sha256sum sha384sum sha512sum" +COMPRESSIONTYPES = "gz bz2 lzma xz lz4 sum md5sum sha1sum sha224sum sha256sum sha384sum sha512sum bmap" COMPRESS_CMD_lzma = "lzma -k -f -7 ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}" COMPRESS_CMD_gz = "gzip -f -9 -c ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.gz" COMPRESS_CMD_bz2 = "pbzip2 -f -k ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}"
@@ -275,12 +275,14 @@ COMPRESS_CMD_sha224sum = "sha224sum ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > COMPRESS_CMD_sha256sum = "sha256sum ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.sha256sum" COMPRESS_CMD_sha384sum = "sha384sum ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.sha384sum" COMPRESS_CMD_sha512sum = "sha512sum ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} > ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.sha512sum" +COMPRESS_CMD_bmap = "bmaptool create ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type} -o ${IMAGE_NAME}${IMAGE_NAME_SUFFIX}.${type}.bmap" COMPRESS_DEPENDS_lzma = "xz-native" COMPRESS_DEPENDS_gz = "" COMPRESS_DEPENDS_bz2 = "pbzip2-native" COMPRESS_DEPENDS_xz = "xz-native" COMPRESS_DEPENDS_lz4 = "lz4-native" COMPRESS_DEPENDS_sum = "mtd-utils-native" +COMPRESS_DEPENDS_bmap = "bmap-tools-native" RUNNABLE_IMAGE_TYPES ?= "ext2 ext3 ext4" RUNNABLE_MACHINE_PATTERNS ?= "qemu"
Signature · 1 line
2.1.4
PATCH 3/3 selftest: add bmap test
In reply to Ed BartoshAdded test_bmap to imagefeatures tests.
It tests if bmap file is generated for the images and if the image is sparse.
--- meta/lib/oeqa/selftest/imagefeatures.py | 27 +++++++++++++++++++++++++++ 1 file changed, 27 insertions(+)
meta/lib/oeqa/selftest/imagefeatures.py +27 -0
@@ -98,3 +98,30 @@ class ImageFeatures(oeSelfTest): # Build a core-image-weston bitbake('core-image-weston') + def test_bmap(self): + """ + Summary: Check bmap support + Expected: 1. core-image-minimal can be build with bmap support + 2. core-image-minimal is sparse + Product: oe-core + Author: Ed Bartosh <[email protected]> + """ + + features = 'IMAGE_FSTYPES += " ext4 ext4.bmap"' + self.write_config(features) + + image_name = 'core-image-minimal' + bitbake(image_name) + + deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') + link_name = get_bb_var('IMAGE_LINK_NAME', image_name) + image_path = os.path.join(deploy_dir, "%s.ext4" % link_name) + bmap_path = "%s.bmap" % image_path + + # check if result image and bmap file are in deploy directory + self.assertTrue(os.path.exists(image_path)) + self.assertTrue(os.path.exists(bmap_path)) + + # check if result image is sparse + image_stat = os.stat(image_path) + self.assertTrue(image_stat.st_size > image_stat.st_blocks * 512)
Signature · 1 line
2.1.4