Skip to contentSkip to conversation

wic: bootimg-partition: Create the u-boot specific boot config

PATCH In review Kevin Hao 2 participants 11 messages Aug 14, 2018 – Aug 20, 2018

lore
Share & tools
lore

Kevin Hao

author
#0 lore raw

cover wic: bootimg-partition: Create the u-boot specific boot config

Hi,

So far, we still have to manually setup the correct boot command for the u-boot before booting the wic image. This patch series tries to leverage the distro boot command in u-boot and compose the corresponding extlinux.conf when creating the wic image. Then the wic image could boot automatically without any manual setting up.

Kevin Hao (6):
wic: Remove the unused variable Partition.sourceparams_dict
wic: bootimg-partition: Use the relative path in the install_task
wic: bootimg-partition: Add do_configure_partition() method
wic: bootimg-partition: Add support to create the u-boot boot config
file
meta-yocto-bsp: beaglebone: Add the u-boot boot config file for the
wic image
meta-yocto-bsp: beaglebone: Use the uuid for the block device naming
in wic image

 meta-yocto-bsp/conf/machine/beaglebone-yocto.conf  |   2 +-
 meta-yocto-bsp/wic/beaglebone-yocto.wks            |   4 +-
 scripts/lib/wic/partition.py                       |   1 -
 .../lib/wic/plugins/source/bootimg-partition.py    | 102 ++++++++++++++++-----
 4 files changed, 82 insertions(+), 27 deletions(-)
Signature · 1 line
2.14.4
author
#1 lore raw

PATCH 1/6 wic: Remove the unused variable Partition.sourceparams_dict

We choose to pass the source parameters to the source plugins' hooks via a local variable srcparams_dict. So the Partition.sourceparams_dict is not used by anyone and seems pretty confused. So drop it.

---
 scripts/lib/wic/partition.py | 1 -
 1 file changed, 1 deletion(-)
scripts/lib/wic/partition.py +0 -1
index c73af9d4275e..5054779b1b6b 100644
@@ -66,7 +66,6 @@ class Partition():
 
         self.lineno = lineno
         self.source_file = ""
-        self.sourceparams_dict = {}
 
     def get_extra_block_count(self, current_blocks):
         """
Signature · 1 line
2.14.4
author
#2 lore raw

PATCH 2/6 wic: bootimg-partition: Use the relative path in the install_task

In reply to Kevin Hao
---
 scripts/lib/wic/plugins/source/bootimg-partition.py | 14 ++++++--------
 1 file changed, 6 insertions(+), 8 deletions(-)
scripts/lib/wic/plugins/source/bootimg-partition.py +6 -8
index b239fc0b4cfe..9480eed6d8a3 100644
@@ -111,20 +111,18 @@ class BootimgPartitionPlugin(SourcePlugin):
 
                 logger.debug('Globbed sources: %s', ', '.join(srcs))
                 for entry in srcs:
+                    src = os.path.relpath(entry, kernel_dir)
                     entry_dst_name = entry_name_fn(entry)
-                    install_task.append((entry,
-                                         os.path.join(hdddir,
-                                                      entry_dst_name)))
+                    install_task.append((src, entry_dst_name)))
             else:
-                install_task = [(os.path.join(kernel_dir, src),
-                                 os.path.join(hdddir, dst))]
+                install_task = [(src, dst)]
 
             for task in install_task:
                 src_path, dst_path = task
-                logger.debug('Install %s as %s',
-                             os.path.basename(src_path), dst_path)
+                logger.debug('Install %s as %s', src_path, dst_path)
                 install_cmd = "install -m 0644 -D %s %s" \
-                              % (src_path, dst_path)
+                              % (os.path.join(kernel_dir, src_path),
+                                 os.path.join(hdddir, dst_path))
                 exec_cmd(install_cmd)
 
         logger.debug('Prepare boot partition using rootfs in %s', hdddir)
Signature · 1 line
2.14.4
author
#3 lore raw

PATCH 3/6 wic: bootimg-partition: Add do_configure_partition() method

In reply to Kevin Hao

We want to add some u-boot specific config file. Before doing this, we need know what files will be installed into this partition. So move the codes about parsing the IMAGE_BOOT_FILES into do_configure_partition(). No function change.

---
 .../lib/wic/plugins/source/bootimg-partition.py    | 61 +++++++++++++---------
 1 file changed, 37 insertions(+), 24 deletions(-)
scripts/lib/wic/plugins/source/bootimg-partition.py +37 -24
index 9480eed6d8a3..364b189758eb 100644
@@ -44,27 +44,17 @@ class BootimgPartitionPlugin(SourcePlugin):
     name = 'bootimg-partition'
 
     @classmethod
-    def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
+    def do_configure_partition(cls, part, source_params, cr, cr_workdir,
                              oe_builddir, bootimg_dir, kernel_dir,
-                             rootfs_dir, native_sysroot):
+                             native_sysroot):
         """
-        Called to do the actual content population for a partition i.e. it
-        'prepares' the partition to be incorporated into the image.
-        In this case, does the following:
-        - sets up a vfat partition
-        - copies all files listed in IMAGE_BOOT_FILES variable
+        Called before do_prepare_partition()
         """
-        hdddir = "%s/boot.%d" % (cr_workdir, part.lineno)
-        install_cmd = "install -d %s" % hdddir
-        exec_cmd(install_cmd)
-
         if not kernel_dir:
             kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
             if not kernel_dir:
                 raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
 
-        logger.debug('Kernel dir: %s', bootimg_dir)
-
         boot_files = None
         for (fmt, id) in (("_uuid-%s", part.uuid), ("_label-%s", part.label), (None, None)):
             if fmt:
@@ -94,9 +84,9 @@ class BootimgPartitionPlugin(SourcePlugin):
             logger.debug('Destination entry: %r', dst_entry)
             deploy_files.append(dst_entry)
 
+        cls.install_task = [];
         for deploy_entry in deploy_files:
             src, dst = deploy_entry
-            install_task = []
             if '*' in src:
                 # by default install files under their basename
                 entry_name_fn = os.path.basename
@@ -113,17 +103,40 @@ class BootimgPartitionPlugin(SourcePlugin):
                 for entry in srcs:
                     src = os.path.relpath(entry, kernel_dir)
                     entry_dst_name = entry_name_fn(entry)
-                    install_task.append((src, entry_dst_name)))
+                    cls.install_task.append((src, entry_dst_name))
             else:
-                install_task = [(src, dst)]
-
-            for task in install_task:
-                src_path, dst_path = task
-                logger.debug('Install %s as %s', src_path, dst_path)
-                install_cmd = "install -m 0644 -D %s %s" \
-                              % (os.path.join(kernel_dir, src_path),
-                                 os.path.join(hdddir, dst_path))
-                exec_cmd(install_cmd)
+                cls.install_task.append((src, dst))
+
+    @classmethod
+    def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
+                             oe_builddir, bootimg_dir, kernel_dir,
+                             rootfs_dir, native_sysroot):
+        """
+        Called to do the actual content population for a partition i.e. it
+        'prepares' the partition to be incorporated into the image.
+        In this case, does the following:
+        - sets up a vfat partition
+        - copies all files listed in IMAGE_BOOT_FILES variable
+        """
+        hdddir = "%s/boot.%d" % (cr_workdir, part.lineno)
+        install_cmd = "install -d %s" % hdddir
+        exec_cmd(install_cmd)
+
+        if not kernel_dir:
+            kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
+            if not kernel_dir:
+                raise WicError("Couldn't find DEPLOY_DIR_IMAGE, exiting")
+
+        logger.debug('Kernel dir: %s', bootimg_dir)
+
+
+        for task in cls.install_task:
+            src_path, dst_path = task
+            logger.debug('Install %s as %s', src_path, dst_path)
+            install_cmd = "install -m 0644 -D %s %s" \
+                          % (os.path.join(kernel_dir, src_path),
+                             os.path.join(hdddir, dst_path))
+            exec_cmd(install_cmd)
 
         logger.debug('Prepare boot partition using rootfs in %s', hdddir)
         part.prepare_rootfs(cr_workdir, oe_builddir, hdddir,
Signature · 1 line
2.14.4
author
#4 lore raw

PATCH 4/6 wic: bootimg-partition: Add support to create the u-boot boot config file

In reply to Kevin Hao

By leveraging the distro boot command feature in the u-boot, we can compose the corresponding extlinux.conf when creating the wic image, and let u-boot boot the kernel automatically. For more detail about the u-boot distro boot command feature, please see doc/README.distro in u-boot source files.

---
 .../lib/wic/plugins/source/bootimg-partition.py    | 51 ++++++++++++++++++++--
 1 file changed, 48 insertions(+), 3 deletions(-)
scripts/lib/wic/plugins/source/bootimg-partition.py +48 -3
index 364b189758eb..7d61595d0414 100644
@@ -48,8 +48,12 @@ class BootimgPartitionPlugin(SourcePlugin):
                              oe_builddir, bootimg_dir, kernel_dir,
                              native_sysroot):
         """
-        Called before do_prepare_partition()
+        Called before do_prepare_partition(), create u-boot specific boot config
         """
+        hdddir = "%s/boot.%d" % (cr_workdir, part.lineno)
+        install_cmd = "install -d %s" % hdddir
+        exec_cmd(install_cmd)
+
         if not kernel_dir:
             kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
             if not kernel_dir:
@@ -107,6 +111,49 @@ class BootimgPartitionPlugin(SourcePlugin):
             else:
                 cls.install_task.append((src, dst))
 
+        if source_params.get('loader') != "u-boot":
+            return
+
+        # The kernel types supported by the sysboot of u-boot
+        kernel_types = ["uImage", "zImage", "Image", "vmlinux", "fitImage"]
+        has_dtb = False
+        fdt_dir = '/'
+        kernel_name = None
+        for task in cls.install_task:
+            src, dst = task
+            # Find the kernel image name
+            for image in kernel_types:
+                if re.match(image, src):
+                    if not kernel_name:
+                        kernel_name = os.path.join('/', dst)
+                    else:
+                        raise WicError('Multi kernel file founded')
+
+            # We suppose that all the dtb are in the same directory
+            if re.search(r'\.dtb', src) and fdt_dir == '/':
+                has_dtb = True
+                fdt_dir = os.path.join(fdt_dir, os.path.dirname(dst))
+
+        if not kernel_name:
+            raise WicError('No kernel file founded')
+
+        # Compose the extlinux.conf
+        extlinux_conf = "default Yocto\n"
+        extlinux_conf += "label Yocto\n"
+        extlinux_conf += "   kernel %s\n" % kernel_name
+        if has_dtb:
+            extlinux_conf += "   fdtdir %s\n" % fdt_dir
+        bootloader = cr.ks.bootloader
+        extlinux_conf += "append root=%s rootwait %s\n" \
+                         % (cr.rootdev, bootloader.append if bootloader.append else '')
+
+        install_cmd = "install -d %s/extlinux/" % hdddir
+        exec_cmd(install_cmd)
+        cfg = open("%s/extlinux/extlinux.conf" % hdddir, "w")
+        cfg.write(extlinux_conf)
+        cfg.close()
+
+
     @classmethod
     def do_prepare_partition(cls, part, source_params, cr, cr_workdir,
                              oe_builddir, bootimg_dir, kernel_dir,
@@ -119,8 +166,6 @@ class BootimgPartitionPlugin(SourcePlugin):
         - copies all files listed in IMAGE_BOOT_FILES variable
         """
         hdddir = "%s/boot.%d" % (cr_workdir, part.lineno)
-        install_cmd = "install -d %s" % hdddir
-        exec_cmd(install_cmd)
 
         if not kernel_dir:
             kernel_dir = get_bitbake_var("DEPLOY_DIR_IMAGE")
Signature · 1 line
2.14.4
#5 lore raw

Re: [PATCH 4/6] wic: bootimg-partition: Add support to create the u-boot boot config file

Quoted 7 lines · Kevin Hao

By leveraging the distro boot command feature in the u-boot, we can compose the corresponding extlinux.conf when creating the wic image, and let u-boot boot the kernel automatically. For more detail about the u-boot distro boot command feature, please see doc/README.distro in u-boot source files.

It doesn't look like this has the ability to provide an external extlinux.conf file like is done for other loaders, but should.

[snip]

scripts/lib/wic/plugins/source/bootimg-partition.py class BootimgPartitionPlugin(SourcePlugin): · line 118

Quoted 2 lines
scripts/lib/wic/plugins/source/bootimg-partition.py +2 -0
+        # The kernel types supported by the sysboot of u-boot
+        kernel_types = ["uImage", "zImage", "Image", "vmlinux", "fitImage"]

This order isn't the best order. We should look for zImage (stock arm32 image), Image (stock arm64 image), fitImage (preferred container in U-Boot) then uImage (legacy container in U-Boot). While I'm sure just vmlinux can work in some cases, I don't think it's appropriate here.

Signature · 1 line
Tom
author
#6 lore raw

Re: [PATCH 4/6] wic: bootimg-partition: Add support to create the u-boot boot config file

Quoted 8 lines · Tom Rini
Quoted 6 lines · Kevin Hao

By leveraging the distro boot command feature in the u-boot, we can compose the corresponding extlinux.conf when creating the wic image, and let u-boot boot the kernel automatically. For more detail about the u-boot distro boot command feature, please see doc/README.distro in u-boot source files.

Quoted 4 lines
Quoted 1 line

It doesn't look like this has the ability to provide an external extlinux.conf file like is done for other loaders, but should.

Yes. I noticed the custom conf support in other loaders. But I chose to keep the first version patch as simple as possible. I would make an addon patch to add that function.

Quoted 6 lines

[snip]

scripts/lib/wic/plugins/source/bootimg-partition.py class BootimgPartitionPlugin(SourcePlugin): · line 118

Quoted 2 lines
scripts/lib/wic/plugins/source/bootimg-partition.py +2 -0
+        # The kernel types supported by the sysboot of u-boot
+        kernel_types = ["uImage", "zImage", "Image", "vmlinux", "fitImage"]

This order isn't the best order.

In the current code, only one of these kernel images can be added in the 'IMAGE_BOOT_FILES'. If it find multi kernel images, it just throw an error and exit. So the order really doesn't matter at all.

Quoted 3 lines

We should look for zImage (stock arm32
image), Image (stock arm64 image), fitImage (preferred container in U-Boot) then uImage (legacy container in U-Boot).

OK, I will adjust the order as your suggested and select the preferred kernel image if multi kernel images are found.

Quoted 2 lines

While I'm sure just
vmlinux can work in some cases, I don't think it's appropriate here.

One of the board supported by meta-yocto-bsp (edgerouter) does use the vmlinux image. So I would keep it.

Thanks,
Kevin

Quoted 3 lines
Signature · 2 lines
-- 
Tom
author
#7 lore raw

PATCH 5/6 meta-yocto-bsp: beaglebone: Add the u-boot boot config file for the wic image

In reply to Kevin Hao
---
 meta-yocto-bsp/conf/machine/beaglebone-yocto.conf | 2 +-
 meta-yocto-bsp/wic/beaglebone-yocto.wks           | 2 +-
 2 files changed, 2 insertions(+), 2 deletions(-)
meta-yocto-bsp/conf/machine/beaglebone-yocto.conf +1 -1
index ae682a47d31b..faa4f1bd3627 100644
@@ -37,4 +37,4 @@ UBOOT_LOADADDRESS = "0x80008000"
 
 MACHINE_FEATURES = "usbgadget usbhost vfat alsa"
 
-IMAGE_BOOT_FILES ?= "u-boot.${UBOOT_SUFFIX} MLO"
+IMAGE_BOOT_FILES ?= "u-boot.${UBOOT_SUFFIX} MLO zImage am335x-bone.dtb am335x-boneblack.dtb am335x-bonegreen.dtb"
meta-yocto-bsp/wic/beaglebone-yocto.wks +1 -1
index 967cad78b870..0b5d793b73ec 100644
@@ -2,5 +2,5 @@
 # long-description: Creates a partitioned SD card image for Beaglebone.
 # Boot files are located in the first vfat partition.
 
-part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16
+part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 --sourceparams="loader=u-boot"
 part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4
Signature · 1 line
2.14.4
author
#8 lore raw

PATCH 6/6 meta-yocto-bsp: beaglebone: Use the uuid for the block device naming in wic image

In reply to Kevin Hao
---
 meta-yocto-bsp/wic/beaglebone-yocto.wks | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta-yocto-bsp/wic/beaglebone-yocto.wks +2 -2
index 0b5d793b73ec..b75f238df937 100644
@@ -2,5 +2,5 @@
 # long-description: Creates a partitioned SD card image for Beaglebone.
 # Boot files are located in the first vfat partition.
 
-part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 --sourceparams="loader=u-boot"
-part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4
+part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 --sourceparams="loader=u-boot" --use-uuid
+part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4 --use-uuid
Signature · 1 line
2.14.4
#9 lore raw

Re: [PATCH 6/6] meta-yocto-bsp: beaglebone: Use the uuid for the block device naming in wic image

Quoted 10 lines · Kevin Hao
---
 meta-yocto-bsp/wic/beaglebone-yocto.wks | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta-yocto-bsp/wic/beaglebone-yocto.wks +0 -0
index 0b5d793b73ec..b75f238df937 100644

meta-yocto-bsp/wic/beaglebone-yocto.wks line 6

Quoted 7 lines
meta-yocto-bsp/wic/beaglebone-yocto.wks +2 -2
@@ -2,5 +2,5 @@
 # long-description: Creates a partitioned SD card image for Beaglebone.
 # Boot files are located in the first vfat partition.
 
-part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 --sourceparams="loader=u-boot"
-part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4
+part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 --sourceparams="loader=u-boot" --use-uuid
+part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4 --use-uuid

You should drop the --ondisk part here entirely so that UUID is used.

Signature · 1 line
Tom
author
#10 lore raw

Re: [PATCH 6/6] meta-yocto-bsp: beaglebone: Use the uuid for the block device naming in wic image

Quoted 14 lines · Tom Rini
Quoted 13 lines · Kevin Hao
---
 meta-yocto-bsp/wic/beaglebone-yocto.wks | 4 ++--
 1 file changed, 2 insertions(+), 2 deletions(-)
meta-yocto-bsp/wic/beaglebone-yocto.wks +0 -0
index 0b5d793b73ec..b75f238df937 100644
@@ -2,5 +2,5 @@
 # long-description: Creates a partitioned SD card image for Beaglebone.
 # Boot files are located in the first vfat partition.
 
Quoted 6 lines

meta-yocto-bsp/wic/beaglebone-yocto.wks line 6

Quoted 4 lines
meta-yocto-bsp/wic/beaglebone-yocto.wks +2 -2
-part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 --sourceparams="loader=u-boot"
-part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4
+part /boot --source bootimg-partition --ondisk mmcblk0 --fstype=vfat --label boot --active --align 4 --size 16 --sourceparams="loader=u-boot" --use-uuid
+part / --source rootfs --ondisk mmcblk --fstype=ext4 --label root --align 4 --use-uuid

You should drop the --ondisk part here entirely so that UUID is used.

The '--use-uuid' does take effect even there is a '--ondisk' parameter. Yes, it seems a bit confusion. I will make a patch to drop the '--ondisk' parameter.

Thanks,
Kevin

Quoted 3 lines
Signature · 2 lines
-- 
Tom