#10073: generic EFI for wic
v2 cover #10073: generic EFI for wic
Hi,
This patchset is an implementation of generic EFI approach for wic images.
Instead of introducing yet another wic plugin it uses existing APIs from EFI_PROVIDER classes to populate ${WORKDIR}/bootfs directory with EFI artifacts and bootloader configuration. This directory can be used by wic rootfs plugin to put into boot partition of the image.
Example kickstart file and 2 test cases were added to wic test suite to better illustrate the approach.
I personally like this approach much better than duplicating oe image creation functionality in wic plugins. This way we can have the code in one place and bootloaders can be configured the same way for oe and wic images.
Changes in v2:
- added patch: fixed default value of GRUB_ROOT
- fixed typo in commit message: timage_types_wic > image_types_wic
The following changes since commit cdb9e7aa493d9ed0eb6ffcec37d581ea928b5f2b:
selftest: fix test_unsupported_subcommand test case (2017-05-02 17:24:23 +0300)
are available in the git repository at:
git://git.yoctoproject.org/poky-contrib ed/wic/wip
http://git.yoctoproject.org/cgit.cgi/poky-contrib/log/?h=ed/wic/wip
Ed Bartosh (5):
efi: add efi_bootfs_populate API
image_types_wic: add do_populate_bootfs task
image_types_wic: merged 2 tasks
oe-selftest: add wic tests for generic EFI
grub-efi: fixed default value of GRUB_ROOT
meta-selftest/wic/test_generic_efi.wks.in | 9 ++++ meta/classes/grub-efi.bbclass | 14 ++++-- meta/classes/image_types_wic.bbclass | 83 +++++++++++++++++++++++-------- meta/classes/systemd-boot.bbclass | 6 +++ meta/lib/oeqa/selftest/wic.py | 36 ++++++++++++++ 5 files changed, 122 insertions(+), 26 deletions(-) create mode 100644 meta-selftest/wic/test_generic_efi.wks.in
Signature · 1 line
2.1.4
PATCH v2 1/5 efi: add efi_bootfs_populate API
Added API to populate ${WORKDIR}/bootfs directory with EFI artifacts to both EFI provider classes(grub-efi and systemd-boot).
This API will be used to prepare artifacts for the wic image build.
[YOCTO #10073]
--- meta/classes/grub-efi.bbclass | 6 ++++++ meta/classes/systemd-boot.bbclass | 6 ++++++ 2 files changed, 12 insertions(+)
meta/classes/grub-efi.bbclass +6 -0
@@ -71,6 +71,12 @@ efi_hddimg_populate() { efi_populate $1 } +efi_bootfs_populate() { + bootfs_dir="${WORKDIR}/bootfs" + efi_populate $bootfs_dir + cp --dereference ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE} $bootfs_dir/vmlinuz +} + python build_efi_cfg() { import sys
meta/classes/systemd-boot.bbclass +6 -0
@@ -62,6 +62,12 @@ efi_hddimg_populate() { efi_populate $1 } +efi_bootfs_populate() { + bootfs_dir="${WORKDIR}/bootfs" + efi_populate $bootfs_dir + cp --dereference ${DEPLOY_DIR_IMAGE}/${KERNEL_IMAGETYPE} $bootfs_dir/vmlinuz +} + python build_efi_cfg() { s = d.getVar("S") labels = d.getVar('LABELS')
Signature · 1 line
2.1.4
PATCH v2 2/5 image_types_wic: add do_populate_bootfs task
In reply to Ed BartoshThis task generates root partition UUID and calls efi_bootfs_populate API of current EFI provider to populate bootfs directory with EFI artifacts.
[YOCTO #10073]
--- meta/classes/image_types_wic.bbclass | 36 ++++++++++++++++++++++++++++++++++++ 1 file changed, 36 insertions(+)
meta/classes/image_types_wic.bbclass +36 -0
@@ -115,3 +115,39 @@ python do_rootfs_wicenv () { addtask do_rootfs_wicenv after do_image before do_image_wic do_rootfs_wicenv[vardeps] += "${WICVARS}" do_rootfs_wicenv[prefuncs] = 'set_image_size' + +# Set variables for efi_bootfs_populate +GRUB_CFG ?= "${WORKDIR}/grub_wic.cfg" +SYSTEMD_BOOT_CFG ?= "${WORKDIR}/loader_wic.conf" +GRUB_GFXSERIAL ?= "1" +LABELS_WIC ?= "boot install" +LABELS ?= "${LABELS_WIC}" + +EFI_PROVIDER ?= "grub-efi" +EFI_CLASS = "${@bb.utils.contains("MACHINE_FEATURES", "efi", "${EFI_PROVIDER}", "", d)}" +inherit ${EFI_CLASS} + +python do_populate_bootfs() { + def populate_bootfs(partuuid): + # remove bootfs dir as it may have files from previous build + bootfs = os.path.join(d.getVar("WORKDIR"), 'bootfs') + if os.path.exists(bootfs): + import shutil + shutil.rmtree(bootfs) + + d.setVar("APPEND", "root=PARTUUID=%s" % partuuid) + bb.build.exec_func('build_efi_cfg', d) + + bb.build.exec_func('efi_bootfs_populate', d) + + if d.getVar('USING_WIC'): + # Generate parition UUID + from uuid import uuid4 + partuuid = str(uuid4()) + d.setVar("ROOTFS_PARTUUID", partuuid) + + if d.getVar("EFI_CLASS"): + populate_bootfs(partuuid) +} + +addtask do_populate_bootfs after do_image before do_image_wic
Signature · 1 line
2.1.4
PATCH v2 3/5 image_types_wic: merged 2 tasks
In reply to Ed BartoshMerged do_write_wks_template and do_populate_bootfs into new task do_prepare_wic_build to be able to write the same partition UUID into bootloader configuration files and kickstart file.
[YOCTO #10073]
--- meta/classes/image_types_wic.bbclass | 51 +++++++++++++++++++----------------- 1 file changed, 27 insertions(+), 24 deletions(-)
meta/classes/image_types_wic.bbclass +27 -24
@@ -43,27 +43,6 @@ do_image_wic[depends] += "wic-tools:do_populate_sysroot" WKS_FILE_DEPENDS ??= '' DEPENDS += "${@ '${WKS_FILE_DEPENDS}' if d.getVar('USING_WIC') else '' }" -python do_write_wks_template () { - """Write out expanded template contents to WKS_FULL_PATH.""" - import re - - template_body = d.getVar('_WKS_TEMPLATE') - - # Remove any remnant variable references left behind by the expansion - # due to undefined variables - expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}") - while True: - new_body = re.sub(expand_var_regexp, '', template_body) - if new_body == template_body: - break - else: - template_body = new_body - - wks_file = d.getVar('WKS_FULL_PATH') - with open(wks_file, 'w') as f: - f.write(template_body) -} - python () { if d.getVar('USING_WIC'): wks_file_u = d.getVar('WKS_FULL_PATH', False)
@@ -90,7 +69,6 @@ python () { # file in process_wks_template as well, so just put it in # a variable and let the metadata deal with the deps. d.setVar('_WKS_TEMPLATE', body) - bb.build.addtask('do_write_wks_template', 'do_image_wic', None, d) } #
@@ -127,7 +105,11 @@ EFI_PROVIDER ?= "grub-efi" EFI_CLASS = "${@bb.utils.contains("MACHINE_FEATURES", "efi", "${EFI_PROVIDER}", "", d)}" inherit ${EFI_CLASS} -python do_populate_bootfs() { +python do_prepare_wic_build() { + # Prepare required artifacts for the wic image build: + # - Populate {WORKDIR}/bootfs directory with EFI content + # - Write wks.in template into the .wks file + def populate_bootfs(partuuid): # remove bootfs dir as it may have files from previous build bootfs = os.path.join(d.getVar("WORKDIR"), 'bootfs')
@@ -140,6 +122,23 @@ python do_populate_bootfs() { bb.build.exec_func('efi_bootfs_populate', d) + def write_wks_template(template_body, wks_file): + """Write out expanded template contents to WKS_FULL_PATH.""" + import re + + # Remove any remnant variable references left behind by the expansion + # due to undefined variables + expand_var_regexp = re.compile(r"\${[^{}@\n\t :]+}") + while True: + new_body = re.sub(expand_var_regexp, '', template_body) + if new_body == template_body: + break + else: + template_body = new_body + + with open(wks_file, 'w') as f: + f.write(template_body) + if d.getVar('USING_WIC'): # Generate parition UUID from uuid import uuid4
@@ -148,6 +147,10 @@ python do_populate_bootfs() { if d.getVar("EFI_CLASS"): populate_bootfs(partuuid) + + template = d.getVar("_WKS_TEMPLATE") + if template: + write_wks_template(template, d.getVar('WKS_FULL_PATH')) } -addtask do_populate_bootfs after do_image before do_image_wic +addtask do_prepare_wic_build after do_image before do_image_wic
Signature · 1 line
2.1.4
PATCH v2 4/5 oe-selftest: add wic tests for generic EFI
In reply to Ed BartoshAdded test_generic_efi_grub_qemu and test_generic_efi_systemd_qemu test cases and wks template to build and boot generic EFI images in qemu.
[YOCTO #10073]
--- meta-selftest/wic/test_generic_efi.wks.in | 9 ++++++++ meta/lib/oeqa/selftest/wic.py | 36 +++++++++++++++++++++++++++++++ 2 files changed, 45 insertions(+) create mode 100644 meta-selftest/wic/test_generic_efi.wks.in
meta-selftest/wic/test_generic_efi.wks.in new file +9 -0
@@ -0,0 +1,9 @@ +# short-description: Create an EFI disk image +# long-description: Creates a partitioned EFI disk image that the user +# can directly dd to boot media. + +part /boot --source rootfs --rootfs-dir=${WORKDIR}/bootfs --fstype=vfat --label boot --active --align 1024 + +part / --source rootfs --fstype=ext4 --label root --align 1024 --uuid ${ROOTFS_PARTUUID} + +bootloader --ptable gpt
meta/lib/oeqa/selftest/wic.py +36 -0
@@ -789,3 +789,39 @@ part /etc --source rootfs --ondisk mmcblk0 --fstype=ext4 --exclude-path bin/ --r # 8 blocks is 4K (physical sector size) self.assertEqual(dest_stat.st_blocks, 8) os.unlink(dest) + + @only_for_arch(['i586', 'i686', 'x86_64']) + def test_generic_efi_grub_qemu(self): + """Test generic efi (grub-efi) image in qemu""" + config = 'IMAGE_FSTYPES = "wic"\nMACHINE_FEATURES_append = " efi"\n'\ + 'EFI_PROVIDER = "grub-efi"\n'\ + 'WKS_FILE = "test_generic_efi.wks.in"\n' + + self.append_config(config) + self.assertEqual(0, bitbake('core-image-minimal ovmf').status) + self.remove_config(config) + + with runqemu('core-image-minimal', ssh=False, + runqemuparams='ovmf', image_fstype='wic') as qemu: + cmd = "grep sda. /proc/partitions |wc -l" + status, output = qemu.run_serial(cmd) + self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output)) + self.assertEqual(output, '2') + + @only_for_arch(['i586', 'i686', 'x86_64']) + def test_generic_efi_systemd_qemu(self): + """Test generic efi (systemd-boot) image in qemu""" + config = 'IMAGE_FSTYPES = "wic"\nMACHINE_FEATURES_append = " efi"\n'\ + 'EFI_PROVIDER = "systemd-boot"\n'\ + 'WKS_FILE = "test_generic_efi.wks.in"\n' + + self.append_config(config) + self.assertEqual(0, bitbake('core-image-minimal ovmf').status) + self.remove_config(config) + + with runqemu('core-image-minimal', ssh=False, + runqemuparams='ovmf', image_fstype='wic') as qemu: + cmd = "grep sda. /proc/partitions |wc -l" + status, output = qemu.run_serial(cmd) + self.assertEqual(1, status, 'Failed to run command "%s": %s' % (cmd, output)) + self.assertEqual(output, '2')
Signature · 1 line
2.1.4
PATCH v2 5/5 grub-efi: fixed default value of GRUB_ROOT
In reply to Ed BartoshCurrently GRUB_ROOT assigned a value of literal "${ROOT}" by default if ROOT variable is not set. This causes kernel commandline to look like this: "linux /vmlinuz LABEL=boot {ROOT}"
Used weak assignments of ROOT and GRUB_ROOT variables to make sure GRUB_ROOT is "" by default.
Allowed empty GRUB_ROOT to be able to use only APPEND variable for both EFI bootloaders (systemd-boot and grub-efi).
--- meta/classes/grub-efi.bbclass | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-)
meta/classes/grub-efi.bbclass +4 -4
@@ -27,7 +27,8 @@ GRUB_TIMEOUT ?= "10" GRUB_OPTS ?= "serial --unit=0 --speed=115200 --word=8 --parity=no --stop=1" EFIDIR = "/EFI/BOOT" -GRUB_ROOT ?= "${ROOT}" +ROOT ??= "" +GRUB_ROOT ??= "${ROOT}" APPEND ?= "" # Need UUID utility code.
@@ -121,8 +122,6 @@ python build_efi_cfg() { cfgfile.write('timeout=50\n') root = d.getVar('GRUB_ROOT') - if not root: - bb.fatal('GRUB_ROOT not defined') if gfxserial == "1": btypes = [ [ " graphics console", "" ],
@@ -146,7 +145,8 @@ python build_efi_cfg() { lb = "install-efi" cfgfile.write('linux /vmlinuz LABEL=%s' % (lb)) - cfgfile.write(' %s' % replace_rootfs_uuid(d, root)) + if root: + cfgfile.write(' %s' % replace_rootfs_uuid(d, root)) append = localdata.getVar('APPEND') initrd = localdata.getVar('INITRD')
Signature · 1 line
2.1.4
Re: [PATCH v2 2/5] image_types_wic: add do_populate_bootfs task
In reply to Ed BartoshHi Ed,
meta/classes/image_types_wic.bbclass python do_rootfs_wicenv () { · line 139
Quoted 2 lines · Ed Bartosh
meta/classes/image_types_wic.bbclass +2
-0+ d.setVar("APPEND", "root=PARTUUID=%s" % partuuid) + bb.build.exec_func('build_efi_cfg', d)
The rootfs wic allows you to build multiple rootfs'es in the image. With that, it'd be good to be able to set PARTUUIDs for each of the rootfs partitions and so that the bootloader LABELs give the options to boot from those.
meta/classes/image_types_wic.bbclass python do_rootfs_wicenv () { · line 153
Quoted 3 lines
meta/classes/image_types_wic.bbclass +3
-0+} + +addtask do_populate_bootfs after do_image before do_image_wic
What if user needed the artifacts on rootfs too. For example IMAGE_ROOTFS could be used as is to prepare sw updates (meta-swupd or ostree).
-- Mikko