V4: Build wic images with bitbake
cover V4: Build wic images with bitbake
Hi,
This is hopefully last version of the patchset.
I've changed location of .wks and .bb files as Richard suggested.
The following changes since commit 0f84702f0811cba4b6e6e890d478f6e98937b831:
image_types.bbclass: add wic image type (2015-08-30 21:38:23 +0100)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/wic/wic-image-type
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wic-image-type
Ed Bartosh (5):
image.py: write bitbake variables to .env file
oe-selftest: test generation of <image>.env
wic-image-minimal: add wic image recipe and .wks
oe-selftest: test building wic image by bitbake
wic: use bitbake variable ROOTFS_SIZE
.../recipes-test/images/wic-image-minimal.bb | 14 +++++++++ .../recipes-test/images/wic-image-minimal.wks | 10 +++++++ meta/classes/image_types.bbclass | 4 +++ meta/lib/oe/image.py | 18 ++++++++++++ meta/lib/oeqa/selftest/wic.py | 34 ++++++++++++++++++++++ scripts/lib/wic/imager/direct.py | 14 +++++++++ 6 files changed, 94 insertions(+) create mode 100644 meta-selftest/recipes-test/images/wic-image-minimal.bb create mode 100644 meta-selftest/recipes-test/images/wic-image-minimal.wks
--
Ed
PATCH 1/5 image.py: write bitbake variables to .env file
Write set of bitbake variables used by wic into build/tmp/sysroots/<machine>/imagedata/<image>.env
List of variables is defined in WICVARS variable in meta/classes/image_types.bbclass.
This is needed for wic to be able to get bitbake variables without running 'bitbake -e'.
--- meta/classes/image_types.bbclass | 4 ++++ meta/lib/oe/image.py | 18 ++++++++++++++++++ 2 files changed, 22 insertions(+)
meta/classes/image_types.bbclass +4 -0
@@ -248,3 +248,7 @@ IMAGE_EXTENSION_live = "hddimg iso" # The IMAGE_TYPES_MASKED variable will be used to mask out from the IMAGE_FSTYPES, # images that will not be built at do_rootfs time: vmdk, vdi, qcow2, hddimg, iso, etc. IMAGE_TYPES_MASKED ?= "" + +# The WICVARS variable is used to define list of bitbake variables used in wic code +# variables from this list is written to <image>.env file +WICVARS ?= "BBLAYERS DEPLOY_DIR_IMAGE HDDDIR IMAGE_BASENAME IMAGE_BOOT_FILES IMAGE_LINK_NAME IMAGE_ROOTFS INITRAMFS_FSTYPES INITRD ISODIR MACHINE_ARCH ROOTFS_SIZE STAGING_DATADIR STAGING_DIR_NATIVE STAGING_LIBDIR TARGET_SYS"
meta/lib/oe/image.py +18 -0
@@ -326,6 +326,22 @@ class Image(ImageDepGraph): return image_cmd_groups + def _write_env(self): + """ + Write environment variables used by wic + to tmp/sysroots/<machine>/imgdata/<image>.env + """ + stdir = self.d.getVar('STAGING_DIR_TARGET', True) + outdir = os.path.join(stdir, 'imgdata') + if not os.path.exists(outdir): + os.makedirs(outdir) + basename = self.d.getVar('IMAGE_BASENAME', True) + with open(os.path.join(outdir, basename) + '.env', 'w') as envf: + for var in self.d.getVar('WICVARS', True).split(): + value = self.d.getVar(var, True) + if value: + envf.write('%s="%s"\n' % (var, value.strip())) + def create(self): bb.note("###### Generate images #######") pre_process_cmds = self.d.getVar("IMAGE_PREPROCESS_COMMAND", True)
@@ -337,6 +353,8 @@ class Image(ImageDepGraph): image_cmd_groups = self._get_imagecmds() + self._write_env() + for image_cmds in image_cmd_groups: # create the images in parallel nproc = multiprocessing.cpu_count()
Signature · 1 line
2.1.4
Re: [PATCH 1/5] image.py: write bitbake variables to .env file
Quoted 11 lines · Ed Bartosh
Write set of bitbake variables used by wic into build/tmp/sysroots/<machine>/imagedata/<image>.env
List of variables is defined in WICVARS variable in meta/classes/image_types.bbclass.
This is needed for wic to be able to get bitbake variables without running 'bitbake -e'.
Signed-off-by Ed Bartosh <[email protected]>
The function name (_write_env) sounds generic, as does the commit message, and the filename, yet it relies on a variable which is WIC-specific. This seems rather inconsistent to me, either this thing is for wic alone, or it isn't, and the function name, file name, and variable name should reflect that.
Signature · 5 lines
Christopher Larson clarson at kergoth dot com Founder - BitBake, OpenEmbedded, OpenZaurus Maintainer - Tslib Senior Software Engineer, Mentor Graphics
Re: [PATCH 1/5] image.py: write bitbake variables to .env file
Quoted 12 lines · Christopher Larson
Quoted 9 lines · Ed Bartosh
Write set of bitbake variables used by wic into build/tmp/sysroots/<machine>/imagedata/<image>.env
List of variables is defined in WICVARS variable in meta/classes/image_types.bbclass.
This is needed for wic to be able to get bitbake variables without running 'bitbake -e'.
Quoted 8 lines
Quoted 2 lines
Signed-off-by Ed Bartosh <[email protected]>The function name (
_write_env) sounds generic, as does the commit message, and the filename, yet it relies on a variable which is WIC-specific. This seems rather inconsistent to me, either this thing is for wic alone, or it isn't, and the function name, file name, and variable name should reflect that.
Thanks for pointing this out. I'll rename _write_env to _write_wic_env it in the next patchset.
--
Regards,
Ed
PATCH 2/5 oe-selftest: test generation of <image>.env
In reply to Ed BartoshAdded test case to check if <image>.env file is generated and contains bitbake variables used in wic code.
--- meta/lib/oeqa/selftest/wic.py | 20 ++++++++++++++++++++ 1 file changed, 20 insertions(+)
meta/lib/oeqa/selftest/wic.py +20 -0
@@ -23,6 +23,7 @@ """Test cases for wic.""" +import os import sys from glob import glob
@@ -168,3 +169,22 @@ class Wic(oeSelfTest): self.assertEqual(1, len(glob(self.resultdir + "HYBRID_ISO_IMG-*.direct"))) self.assertEqual(1, len(glob(self.resultdir + "HYBRID_ISO_IMG-*.iso"))) + def test19_image_env(self): + """Test generation of <image>.env files.""" + image = 'core-image-minimal' + stdir = get_bb_var('STAGING_DIR_TARGET', image) + imgdatadir = os.path.join(stdir, 'imgdata') + + basename = get_bb_var('IMAGE_BASENAME', image) + self.assertEqual(basename, image) + path = os.path.join(imgdatadir, basename) + '.env' + self.assertTrue(os.path.isfile(path)) + + wicvars = get_bb_var('WICVARS', image).split() + wicvars.remove('IMAGE_BOOT_FILES') # this variable is optional + with open(path) as envfile: + content = dict(line.split("=", 1) for line in envfile) + # test if variables used by wic present in the .env file + for var in wicvars: + self.assertTrue(var in content, "%s is not in .env file" % var) + self.assertTrue(content[var])
Signature · 1 line
2.1.4
PATCH 3/5 wic-image-minimal: add wic image recipe and .wks
In reply to Ed BartoshAdded example of recipe and .wks file to create partitioned image.
This image is using quite complex partitioning scheme.
It uses its own rootfs to populate two partitions in two different ways. It also uses core-image-minimal rootfs to populate another partition.
This is how wic reports about artifacts used to create this image:
ROOTFS_DIR: tmp/work/qemux86_64-poky-linux/wic-image-minimal/1.0-r0/rootfs ROOTFS_DIR["/core"]: tmp/work/qemux86_64-poky-linux/core-image-minimal/1.0-r0/rootfs ROOTFS_DIR["/backup"]: tmp/work/qemux86_64-poky-linux/wic-image-minimal/1.0-r0/rootfs
BOOTIMG_DIR: tmp/sysroots/qemux86-64/usr/share
KERNEL_DIR: tmp/deploy/images/qemux86-64
NATIVE_SYSROOT: tmp/sysroots/x86_64-linux
--- meta-selftest/recipes-test/images/wic-image-minimal.bb | 14 ++++++++++++++ meta-selftest/recipes-test/images/wic-image-minimal.wks | 10 ++++++++++ 2 files changed, 24 insertions(+) create mode 100644 meta-selftest/recipes-test/images/wic-image-minimal.bb create mode 100644 meta-selftest/recipes-test/images/wic-image-minimal.wks
meta-selftest/recipes-test/images/wic-image-minimal.bb new file +14 -0
@@ -0,0 +1,14 @@ +SUMMARY = "An example of partitioned image." + +IMAGE_INSTALL = "packagegroup-core-boot ${ROOTFS_PKGMANAGE_BOOTSTRAP}" + +IMAGE_FSTYPES = "wic.bz2" +RM_OLD_IMAGE = "1" + +# core-image-minimal is referenced in .wks, so we need its rootfs +# to be ready before our rootfs +do_rootfs[depends] += "core-image-minimal:do_rootfs" + +IMAGE_ROOTFS_EXTRA_SPACE = "2000" + +inherit image
meta-selftest/recipes-test/images/wic-image-minimal.wks new file +10 -0
@@ -0,0 +1,10 @@ +# short-description: Example of partitioned image with complex layout +# long-description: This image contains boot partition and 3 rootfs partitions +# created from core-image-minimal and wic-image-minimal image recipes. + +part /boot --source bootimg-pcbios --ondisk sda --label boot --active --align 1024 +part / --source rootfs --ondisk sda --fstype=ext2 --label platform --align 1024 +part /core --source rootfs --rootfs-dir=core-image-minimal --ondisk sda --fstype=ext2 --label core --align 1024 +part /backup --source rootfs --rootfs-dir=wic-image-minimal --ondisk sda --fstype=ext2 --label backup --align 1024 + +bootloader --timeout=0 --append="rootwait console=tty0"
Signature · 1 line
2.1.4
PATCH 4/5 oe-selftest: test building wic image by bitbake
In reply to Ed BartoshAdded test case to verify building of wic-image-minimal recipe and produced artifacts: manifest and bzipped partitioned image.
--- meta/lib/oeqa/selftest/wic.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
meta/lib/oeqa/selftest/wic.py +14 -0
@@ -188,3 +188,17 @@ class Wic(oeSelfTest): for var in wicvars: self.assertTrue(var in content, "%s is not in .env file" % var) self.assertTrue(content[var]) + + def test20_wic_image_type(self): + """Test building wic images by bitbake""" + self.assertEqual(0, bitbake('wic-image-minimal').status) + + deploy_dir = get_bb_var('DEPLOY_DIR_IMAGE') + machine = get_bb_var('MACHINE') + prefix = os.path.join(deploy_dir, 'wic-image-minimal-%s.' % machine) + # check if we have result image and manifests symlinks + # pointing to existing files + for suffix in ('wic.bz2', 'manifest'): + path = prefix + suffix + self.assertTrue(os.path.islink(path)) + self.assertTrue(os.path.isfile(os.path.realpath(path)))
Signature · 1 line
2.1.4
PATCH 5/5 wic: use bitbake variable ROOTFS_SIZE
In reply to Ed BartoshIf bitbake image is referenced in .ks file and --size is not used there wic uses ROOTFS_SIZE variable to set minimum partition size.
ROOTFS_SIZE is calculated in meta/lib/oe/image.py when rootfs is created. The calculation is done using other image parameters:IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT, IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE.
--- scripts/lib/wic/imager/direct.py | 14 ++++++++++++++ 1 file changed, 14 insertions(+)
scripts/lib/wic/imager/direct.py +14 -0
@@ -29,6 +29,7 @@ import shutil from wic import kickstart, msger from wic.utils import fs_related +from wic.utils.oe.misc import get_bitbake_var from wic.utils.partitionedfs import Image from wic.utils.errors import CreatorError, ImageError from wic.imager.baseimager import BaseImageCreator
@@ -229,6 +230,19 @@ class DirectImageCreator(BaseImageCreator): fstab_path = self._write_fstab(self.rootfs_dir.get("ROOTFS_DIR")) for p in parts: + # get rootfs size from bitbake variable if it's not set in .ks file + if not p.size: + # and if rootfs name is specified for the partition + image_name = p.get_rootfs() + if image_name: + # Bitbake variable ROOTFS_SIZE is calculated in + # Image._get_rootfs_size method from meta/lib/oe/image.py + # using IMAGE_ROOTFS_SIZE, IMAGE_ROOTFS_ALIGNMENT, + # IMAGE_OVERHEAD_FACTOR and IMAGE_ROOTFS_EXTRA_SPACE + rsize_bb = get_bitbake_var('ROOTFS_SIZE', image_name) + if rsize_bb: + # convert from Kb to Mb + p.size = int(rsize_bb) / 1024 # need to create the filesystems in order to get their # sizes before we can add them and do the layout. # Image.create() actually calls __format_disks() to create
Signature · 1 line
2.1.4