Skip to contentSkip to conversation

add wic based image installer

PATCH In review [email protected] 2 participants 7 messages May 8, 2019 – May 16, 2019

lore
Share & tools
lore
#0 lore raw

cover add wic based image installer

From: Chee Yang Lee <[email protected]>

Existing installable image (hddimg/ISO) has limit of 4GB size, so to build larger installable image, we can build it in wic based image.

2 partition needed for installable image
- first partition build using source plugin bootimg-efi and configure to run install
- second partition build with root.img, systemd-boot and kernel to be install on target machine (this partition build using new source plugin installer-partition)

These patches:
- add new source plugin for second partition
- add new .wks
- add new wic dependency and set default value for required variable
- allow source plugin bootimg-efi to configure to install

To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf.

Chee Yang Lee (4):
wic: add new installer-partition plugin
wic: bootimg-efi: add label source parameter
wic: add new wic-installer.wks for wic based image installer
image_types_wic: add dependency for wic based image installer

 meta/classes/image_types_wic.bbclass               |   6 +
 scripts/lib/wic/canned-wks/wic-installer.wks.in    |   7 +
 scripts/lib/wic/plugins/source/bootimg-efi.py      |  20 ++-
 .../lib/wic/plugins/source/installer-partition.py  | 192 +++++++++++++++++++++
 4 files changed, 220 insertions(+), 5 deletions(-)
 create mode 100644 scripts/lib/wic/canned-wks/wic-installer.wks.in
 create mode 100644 scripts/lib/wic/plugins/source/installer-partition.py
Signature · 1 line
2.7.4
#1 lore raw

PATCH 1/4 wic: add new installer-partition plugin

From: Chee Yang Lee <[email protected]>

This patch implement 'installer-partition.py' source plugin for wic. The plugin create an image with systemd-boot and rootfs. The generated image can be use to create wic based image installer.

---
 .../lib/wic/plugins/source/installer-partition.py  | 192 +++++++++++++++++++++
 1 file changed, 192 insertions(+)
 create mode 100644 scripts/lib/wic/plugins/source/installer-partition.py
scripts/lib/wic/plugins/source/installer-partition.py new file +192 -0
new file mode 100644
index 0000000..4d8343f
@@ -0,0 +1,192 @@
+# Copyright (c) 2019, Intel Corporation.
+# All rights reserved.
+#
+# This program is free software; you can redistribute it and/or modify
+# it under the terms of the GNU General Public License version 2 as
+# published by the Free Software Foundation.
+#
+# This program is distributed in the hope that it will be useful,
+# but WITHOUT ANY WARRANTY; without even the implied warranty of
+# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
+# GNU General Public License for more details.
+#
+# You should have received a copy of the GNU General Public License along
+# with this program; if not, write to the Free Software Foundation, Inc.,
+# 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
+#
+# DESCRIPTION
+# This implements the 'installer-partition' source plugin class for 'wic'
+# This plugin prepare partition with content to be install on target machine.
+#
+# AUTHORS
+# Lee Chee Yang <Chee.Yang.Lee (at] intel.com>
+#
+
+import logging
+import os
+
+from wic import WicError
+from wic.engine import get_custom_config
+from wic.pluginbase import SourcePlugin
+from wic.misc import (exec_cmd, exec_native_cmd, get_bitbake_var)
+
+logger = logging.getLogger('wic')
+
+class InstallerImagePlugin(SourcePlugin):
+    """
+    Populate content for wic image based installer
+    """
+
+    name = 'installer-partition'
+
+    @classmethod
+    def do_configure_partition(cls, part, source_params, creator, cr_workdir,
+                               oe_builddir, bootimg_dir, kernel_dir,
+                               native_sysroot):
+        """
+        Called before do_prepare_partition(), creates loader-specific config
+        """
+        if not kernel_dir:
+            kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+            if not kernel_dir:
+                raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
+        staging_kernel_dir = kernel_dir
+
+        partition_dir = "%s/%s-%s" % (cr_workdir, part.label, part.lineno)
+
+        install_cmd = "install -d %s/EFI/BOOT" % partition_dir
+        exec_cmd(install_cmd)
+
+        install_cmd = "install -d %s/loader/entries" % partition_dir
+        exec_cmd(install_cmd)
+
+        bootloader = creator.ks.bootloader
+
+        loader_conf = ""
+        loader_conf += "default boot\n"
+        loader_conf += "timeout %d\n" % bootloader.timeout
+
+        initrd = source_params.get('initrd')
+
+        if initrd:
+            cp_cmd = "cp %s/%s %s" % (kernel_dir, initrd, partition_dir)
+            exec_cmd(cp_cmd, True)
+        else:
+            logger.debug("Ignoring missing initrd")
+
+        logger.debug("Writing systemd-boot config "
+                     "%s/loader/loader.conf", partition_dir)
+        cfg = open("%s/loader/loader.conf" % partition_dir, "w")
+        cfg.write(loader_conf)
+        cfg.close()
+
+        kernel = get_bitbake_var("KERNEL_IMAGETYPE")
+        if not kernel:
+            kernel = "bzImage"
+
+        install_cmd = "install -m 0644 %s/%s %s/%s" % \
+            (staging_kernel_dir, kernel, partition_dir, kernel)
+        exec_cmd(install_cmd)
+
+        configfile = creator.ks.bootloader.configfile
+        custom_cfg = None
+        if configfile:
+            custom_cfg = get_custom_config(configfile)
+            if custom_cfg:
+                # Use a custom configuration for systemd-boot
+                boot_conf = custom_cfg
+                logger.debug("Using custom configuration file "
+                             "%s for systemd-boots's boot.conf", configfile)
+            else:
+                raise WicError("configfile is specified but failed to "
+                               "get it from %s.", configfile)
+
+        if not custom_cfg:
+            # Create systemd-boot configuration using parameters from wks file
+            title = source_params.get('title')
+
+            boot_conf = ""
+            boot_conf += "title %s\n" % (title if title else "boot")
+            boot_conf += "linux /%s\n" % kernel
+            boot_conf += "options LABEL=Boot %s\n" % (bootloader.append)
+
+            if initrd:
+                boot_conf += "initrd /%s\n" % initrd
+
+        for mod in [x for x in os.listdir(kernel_dir) if x.startswith("systemd-")]:
+            cp_cmd = "cp %s/%s %s/EFI/BOOT/%s" % (kernel_dir, mod, partition_dir, mod[8:])
+            exec_cmd(cp_cmd, True)
+
+        logger.debug("Writing systemd-boot config "
+                     "%s/loader/entries/boot.conf", partition_dir)
+        cfg = open("%s//loader/entries/boot.conf" % partition_dir, "w")
+        cfg.write(boot_conf)
+        cfg.close()
+
+
+    @classmethod
+    def do_prepare_partition(cls, part, source_params, creator, cr_workdir,
+                             oe_builddir, bootimg_dir, kernel_dir,
+                             rootfs_dir, native_sysroot):
+        """
+        Called to do the actual content population for a partition,
+        prepare systemd bootloader and rootfs image
+        """
+        partition_dir = "%s/%s-%s" % (cr_workdir, part.label, part.lineno)
+
+        rootfs = "%s/fs_%s.%s.%s" % (cr_workdir, part.label, part.lineno, part.fstype)
+
+        image_deploy_dir = get_bitbake_var("IMGDEPLOYDIR")
+        if not image_deploy_dir:
+            raise WicError("Couldn't find IMGDEPLOYDIR, exiting")
+
+        rootfs_img = "%s/%s.%s" % (image_deploy_dir,
+            get_bitbake_var("IMAGE_LINK_NAME"), part.fstype )
+        if not os.path.isfile(rootfs_img):
+            raise WicError("Couldn't find %s, exiting" % rootfs_img)
+
+        install_cmd = "install -m 0644 %s %s/rootfs.img" % \
+            (rootfs_img, partition_dir)
+        exec_cmd(install_cmd)
+
+        #look for actual size required for the partition
+        du_cmd = "du -ks %s" % partition_dir
+        out = exec_cmd(du_cmd)
+        part.size = cls.get_partition_size(part, int(out.split()[0]))
+        logger.debug("Set partition size to %d ", part.size)
+
+        with open(rootfs, 'w') as sparse:
+            os.ftruncate(sparse.fileno(), part.size * 1024)
+
+        extraopts = part.mkfs_extraopts or "-i 8192"
+
+        label_str = ""
+        if part.label:
+            label_str = "-L %s" % part.label
+
+        mkfs_cmd = "mkfs.%s -F %s %s -U %s %s -d %s" % \
+            (part.fstype, extraopts, label_str, part.fsuuid, rootfs, partition_dir)
+        exec_native_cmd(mkfs_cmd, native_sysroot)
+
+        part.source_file = rootfs
+
+
+    def get_partition_size(part, actual_partition_size=0):
+        """
+        Calculate the required size of the partition, taking into consideration
+        --size/--fixed-size/--extra-space flags specified in kickstart file.
+        Raises an error if the `actual_partition_size` is larger than fixed-size.
+        """
+        if part.fixed_size:
+            partition_size = part.fixed_size
+            if actual_partition_size > partition_size:
+                raise WicError("Actual partition size (%d kB) is larger than "
+                               "allowed size %d kB" %
+                               (actual_partition_size, partition_size))
+        else:
+            extra_space = 0
+            if extra_space < part.extra_space:
+                extra_space = part.extra_space
+            partition_size = actual_partition_size + extra_space
+
+        return partition_size
Signature · 1 line
2.7.4
#2 lore raw

PATCH 2/4 wic: bootimg-efi: add label source parameter

In reply to [email protected]

From: Chee Yang Lee <[email protected]>

Add new source parameter label to allow boot.conf/grub.cfg label option to set as "install-efi", so far it's hardcoded to "Boot".

The label remain to "Boot" for systemd-boot or blank for grub-efi whenever label are not "install-efi".

when label set to "install-efi" the image can boot into image installation.

---
 scripts/lib/wic/plugins/source/bootimg-efi.py | 20 +++++++++++++++-----
 1 file changed, 15 insertions(+), 5 deletions(-)
scripts/lib/wic/plugins/source/bootimg-efi.py +15 -5
index 83a7e18..0a0c5bd 100644
@@ -86,9 +86,14 @@ class BootimgEFIPlugin(SourcePlugin):
             grubefi_conf += "menuentry '%s'{\n" % (title if title else "boot")
 
             kernel = "/bzImage"
+            label = source_params.get('label')
 
-            grubefi_conf += "linux %s root=%s rootwait %s\n" \
-                % (kernel, creator.rootdev, bootloader.append)
+            if label == "install-efi":
+                grubefi_conf += "linux %s LABEL=%s rootwait %s\n" \
+                    % (kernel, label, bootloader.append)
+            else:
+                grubefi_conf += "linux %s root=%s rootwait %s\n" \
+                    % (kernel, creator.rootdev, bootloader.append)
 
             if initrd:
                grubefi_conf += "initrd /%s\n" % initrd
@@ -154,12 +159,17 @@ class BootimgEFIPlugin(SourcePlugin):
             # Create systemd-boot configuration using parameters from wks file
             kernel = "/bzImage"
             title = source_params.get('title')
-
+            label = source_params.get('label')
             boot_conf = ""
             boot_conf += "title %s\n" % (title if title else "boot")
             boot_conf += "linux %s\n" % kernel
-            boot_conf += "options LABEL=Boot root=%s %s\n" % \
-                             (creator.rootdev, bootloader.append)
+
+            if label == "install-efi":
+                boot_conf += "options LABEL=%s %s\n" % \
+                                (label, bootloader.append)
+            else:
+                boot_conf += "options LABEL=Boot root=%s %s\n" % \
+                                 (creator.rootdev, bootloader.append)
 
             if initrd:
                 boot_conf += "initrd /%s\n" % initrd
Signature · 1 line
2.7.4
#3 lore raw

PATCH 3/4 wic: add new wic-installer.wks for wic based image installer

In reply to [email protected]

From: Chee Yang Lee <[email protected]>

---
 scripts/lib/wic/canned-wks/wic-installer.wks.in | 7 +++++++
 1 file changed, 7 insertions(+)
 create mode 100644 scripts/lib/wic/canned-wks/wic-installer.wks.in
scripts/lib/wic/canned-wks/wic-installer.wks.in new file +7 -0
new file mode 100644
index 0000000..a7b018d
@@ -0,0 +1,7 @@
+# description: Create wic installable image
+
+part /boot --source bootimg-efi --sourceparams="loader=systemd-boot,title=install,label=install-efi,initrd=${INITRD_IMAGE_LIVE}-${MACHINE}.${INITRAMFS_FSTYPES}" --ondisk sda --label install --active --align 1024 --use-uuid
+
+part / --source installer-partition --ondisk sda --fstype=ext4 --label image --use-uuid
+
+bootloader --ptable gpt --timeout=5 --append="rootwait console=ttyS0,115200 console=tty0"
Signature · 1 line
2.7.4
#4 lore raw

PATCH 4/4 image_types_wic: add dependency for wic based image installer

In reply to [email protected]

From: Chee Yang Lee <[email protected]>

initramfs and rootfs image are required to build wic based image installer, add both as dependency for do_image_wic.

initiate default value for LIVE_ROOTFS_TYPE, INITRD_IMAGE_LIVE and INITRD_LIVE

---
 meta/classes/image_types_wic.bbclass | 6 ++++++
 1 file changed, 6 insertions(+)
meta/classes/image_types_wic.bbclass +6 -0
index 5b40a9e..e4f2b8f 100644
@@ -102,9 +102,15 @@ python () {
                 # a variable and let the metadata deal with the deps.
                 d.setVar('_WKS_TEMPLATE', body)
                 bb.build.addtask('do_write_wks_template', 'do_image_wic', None, d)
+        d.appendVarFlag('do_image_wic', 'depends', ' %s:do_image_complete' % d.getVar('INITRD_IMAGE_LIVE'))
         bb.build.addtask('do_image_wic', 'do_image_complete', None, d)
 }
 
+LIVE_ROOTFS_TYPE ?= "ext4"
+INITRD_IMAGE_LIVE ?= "${MLPREFIX}core-image-minimal-initramfs"
+INITRD_LIVE ?= "${DEPLOY_DIR_IMAGE}/${INITRD_IMAGE_LIVE}-${MACHINE}.${INITRAMFS_FSTYPES}"
+IMAGE_TYPEDEP_wic = "${LIVE_ROOTFS_TYPE}"
+
 #
 # Write environment variables used by wic
 # to tmp/sysroots/<machine>/imgdata/<image>.env
Signature · 1 line
2.7.4
#5 lore raw
In reply to [email protected]
Quoted 8 lines · On Wed, May 08, 2019 at 02:40:58PM +0800, [email protected] wrote:

From: Chee Yang Lee <[email protected]>

Existing installable image (hddimg/ISO) has limit of 4GB size, so to build larger installable image, we can build it in wic based image.

2 partition needed for installable image
- first partition build using source plugin bootimg-efi and configure to run install
- second partition build with root.img, systemd-boot and kernel to be install on target machine (this partition build using new source plugin installer-partition)

Quoted 8 lines

These patches:
- add new source plugin for second partition
- add new .wks
- add new wic dependency and set default value for required variable
- allow source plugin bootimg-efi to configure to install

To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf.

I'm confused as to why we need most of these changes to support the end goal. The enhancement to bootimg-efi makes conceptual sense. I don't see why we need a new plugin to do the rest, however. What's being done here that we don't already do today? I say this having made installer images for EFI machines using wic (in sumo, even). Thanks!

Signature · 1 line
Tom
#6 lore raw
In reply to [email protected]

Any comments on these patches?

-----Original Message-----
From: [email protected] <[email protected]> On Behalf Of [email protected]
Sent: Wednesday, May 8, 2019 2:41 PM
To: [email protected]
Subject: [OE-core] [PATCH 0/4] add wic based image installer

From: Chee Yang Lee <[email protected]>

Existing installable image (hddimg/ISO) has limit of 4GB size, so to build larger installable image, we can build it in wic based image.

2 partition needed for installable image
- first partition build using source plugin bootimg-efi and configure to run install
- second partition build with root.img, systemd-boot and kernel to be install on target machine (this partition build using new source plugin installer-partition)

These patches:
- add new source plugin for second partition
- add new .wks
- add new wic dependency and set default value for required variable
- allow source plugin bootimg-efi to configure to install

To build the image, set WKS_FILE="wic-installer.wks.in" in local.conf.

Chee Yang Lee (4):
wic: add new installer-partition plugin
wic: bootimg-efi: add label source parameter
wic: add new wic-installer.wks for wic based image installer
image_types_wic: add dependency for wic based image installer

 meta/classes/image_types_wic.bbclass               |   6 +
 scripts/lib/wic/canned-wks/wic-installer.wks.in    |   7 +
 scripts/lib/wic/plugins/source/bootimg-efi.py      |  20 ++-
 .../lib/wic/plugins/source/installer-partition.py  | 192 +++++++++++++++++++++
 4 files changed, 220 insertions(+), 5 deletions(-)  create mode 100644 scripts/lib/wic/canned-wks/wic-installer.wks.in
 create mode 100644 scripts/lib/wic/plugins/source/installer-partition.py

--
2.7.4

--
_______________________________________________
Openembedded-core mailing list
[email protected]
http://lists.openembedded.org/mailman/listinfo/openembedded-core