image-bootfiles: new class
v5 cover image-bootfiles: new class
The image-bootfiles class is used to put all files listed in IMAGE_BOOT_FILES into the root filesystem.
IMAGE_BOOT_FILES is used by the bootimg-partition wic plugin to put the files into a boot partition.
Be able to list files as "boot files" in e.g. your .conf or image files instead of install those in every recipe is a good thing.
It is not always desired to have a separate boot partition for boot files. Sometimes it could be good to have them as a part of the root filesystem.
For example, if a double copy strategy is used for update the system, then you probably want to update both the boot files and root filesystem at the same time as there may be dependencies.
v2:
- Removed the documentation from the patch series (will be submitted later)
- Break out the parts in bootimg-partition that is used by image-bootfiles to a common library
- Make the destination directory in root filesystem configurable
v3:
- See changelog in patches
v4:
- See changelog in patches
v5:
- See changelog in patches
Marcus Folkesson (2):
bootimg-partition: break out code to a common library. image-bootfiles.bbclass: new class, copy boot files to root filesystem
meta/classes/image-bootfiles.bbclass | 41 +++++++++++++ meta/lib/oe/bootfiles.py | 57 +++++++++++++++++++ .../wic/plugins/source/bootimg-partition.py | 39 +------------ 3 files changed, 100 insertions(+), 37 deletions(-) create mode 100644 meta/classes/image-bootfiles.bbclass create mode 100644 meta/lib/oe/bootfiles.py
Signature · 1 line
2.45.1
PATCH v5 1/2 bootimg-partition: break out code to a common library.
Break out the code that parse IMAGE_BOOT_FILES to a common library.
---
Notes:
v3:
- Removed unnecessary "import glob"
- Explicitely tell that bootfiles_populate() returns a tuple
v4:
- Remove the right "import glob" :-)
v5:
- No changes
meta/lib/oe/bootfiles.py | 57 +++++++++++++++++++ .../wic/plugins/source/bootimg-partition.py | 39 +------------ 2 files changed, 59 insertions(+), 37 deletions(-) create mode 100644 meta/lib/oe/bootfiles.py
meta/lib/oe/bootfiles.py new file +57 -0
@@ -0,0 +1,57 @@ +# +# SPDX-License-Identifier: MIT +# +# Copyright (C) 2024 Marcus Folkesson +# Author: Marcus Folkesson <[email protected]> +# +# Utility functions handling boot files +# +# Look into deploy_dir and search for boot_files. +# Returns a list of tuples with (original filepath relative to +# deploy_dir, desired filepath renaming) +# +# Heavily inspired of bootimg-partition.py +# +def get_boot_files(deploy_dir, boot_files): + import re + import os + from glob import glob + + if boot_files is None: + return None + + # list of tuples (src_name, dst_name) + deploy_files = [] + for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files): + if ';' in src_entry: + dst_entry = tuple(src_entry.split(';')) + if not dst_entry[0] or not dst_entry[1]: + raise ValueError('Malformed boot file entry: %s' % src_entry) + else: + dst_entry = (src_entry, src_entry) + + deploy_files.append(dst_entry) + + install_files = [] + for deploy_entry in deploy_files: + src, dst = deploy_entry + if '*' in src: + # by default install files under their basename + entry_name_fn = os.path.basename + if dst != src: + # unless a target name was given, then treat name + # as a directory and append a basename + entry_name_fn = lambda name: \ + os.path.join(dst, + os.path.basename(name)) + + srcs = glob(os.path.join(deploy_dir, src)) + + for entry in srcs: + src = os.path.relpath(entry, deploy_dir) + entry_dst_name = entry_name_fn(entry) + install_files.append((src, entry_dst_name)) + else: + install_files.append((src, dst)) + + return install_files
scripts/lib/wic/plugins/source/bootimg-partition.py +2 -37
@@ -16,7 +16,7 @@ import logging import os import re -from glob import glob +from oe.bootfiles import get_boot_files from wic import WicError from wic.engine import get_custom_config
@@ -66,42 +66,7 @@ class BootimgPartitionPlugin(SourcePlugin): logger.debug('Boot files: %s', boot_files) - # list of tuples (src_name, dst_name) - deploy_files = [] - for src_entry in re.findall(r'[\w;\-\./\*]+', boot_files): - if ';' in src_entry: - dst_entry = tuple(src_entry.split(';')) - if not dst_entry[0] or not dst_entry[1]: - raise WicError('Malformed boot file entry: %s' % src_entry) - else: - dst_entry = (src_entry, src_entry) - - logger.debug('Destination entry: %r', dst_entry) - deploy_files.append(dst_entry) - - cls.install_task = []; - for deploy_entry in deploy_files: - src, dst = deploy_entry - if '*' in src: - # by default install files under their basename - entry_name_fn = os.path.basename - if dst != src: - # unless a target name was given, then treat name - # as a directory and append a basename - entry_name_fn = lambda name: \ - os.path.join(dst, - os.path.basename(name)) - - srcs = glob(os.path.join(kernel_dir, src)) - - logger.debug('Globbed sources: %s', ', '.join(srcs)) - for entry in srcs: - src = os.path.relpath(entry, kernel_dir) - entry_dst_name = entry_name_fn(entry) - cls.install_task.append((src, entry_dst_name)) - else: - cls.install_task.append((src, dst)) - + cls.install_task = get_boot_files(kernel_dir, boot_files) if source_params.get('loader') != "u-boot": return
Signature · 1 line
2.45.1
PATCH v5 2/2 image-bootfiles.bbclass: new class, copy boot files to root filesystem
In reply to Marcus Folkessonimage-bootfiles class copy files listed in IMAGE_BOOT_FILES to the IMAGE_BOOT_FILES_DIR directory of the root filesystem.
This is useful when there is no explicit boot partition but all boot files should instead reside inside the root filesystem.
---
Notes:
v3:
- Skip the intermediate bootfiles() function
- Rename variable names to be consistent
- Various python optimizations
v4:
- Correct usage description
- Create image_dst before copy
v5:
- Check if install_files is [] rather than None
- Print warning if overwriting files
- run as ROOTFS_POSTPROCESS_COMMAND instead
meta/classes/image-bootfiles.bbclass | 41 ++++++++++++++++++++++++++++ 1 file changed, 41 insertions(+) create mode 100644 meta/classes/image-bootfiles.bbclass
meta/classes/image-bootfiles.bbclass new file +41 -0
@@ -0,0 +1,41 @@ +# +# SPDX-License-Identifier: MIT +# +# Copyright (C) 2024 Marcus Folkesson +# Author: Marcus Folkesson <[email protected]> +# +# Writes IMAGE_BOOT_FILES to the IMAGE_BOOT_FILES_DIR directory. +# +# Usage: add "inherit image-bootfiles" to your image. +# + +IMAGE_BOOT_FILES_DIR ?= "boot" + +python bootfiles_populate() { + import shutil + from oe.bootfiles import get_boot_files + + deploy_image_dir = d.getVar("DEPLOY_DIR_IMAGE") + boot_dir = os.path.join(d.getVar("IMAGE_ROOTFS"), d.getVar("IMAGE_BOOT_FILES_DIR")) + + boot_files = d.getVar("IMAGE_BOOT_FILES") + if boot_files is None: + return + + install_files = get_boot_files(deploy_image_dir, boot_files) + if not install_files: + bb.warn("Could not find any boot files to install even though IMAGE_BOOT_FILES is not empty") + return + + os.makedirs(boot_dir, exist_ok=True) + for src, dst in install_files: + image_src = os.path.join(deploy_image_dir, src) + image_dst = os.path.join(boot_dir, dst) + if os.path.exists(image_dst): + bb.warn("%s does already exist and will be overwritten" % image_dst) + + os.makedirs(os.path.dirname(image_dst), exist_ok=True) + shutil.copyfile(image_src, image_dst) +} + +ROOTFS_POSTPROCESS_COMMAND += "bootfiles_populate;"
Signature · 1 line
2.45.1
A message that is not in the archive (sent off-list or not yet received): [email protected]
Marcus Folkesson
Reviewed-by Quentin SchulzRe: [OE-core] [PATCH v5 2/2] image-bootfiles.bbclass: new class, copy boot files to root filesystem
Quoted 7 lines · Marcus Folkesson via lists.openembedded.org
image-bootfiles class copy files listed in
IMAGE_BOOT_FILESto theIMAGE_BOOT_FILES_DIRdirectory of the root filesystem.This is useful when there is no explicit boot partition but all boot files should instead reside inside the root filesystem.
Signed-off-by Marcus Folkesson <[email protected]>
Sorry, I forgot the Reviewed-by tag from v4:
Hi all,
Quoted 34 lines · Marcus Folkesson
The image-bootfiles class is used to put all files listed in
IMAGE_BOOT_FILESinto the root filesystem.
IMAGE_BOOT_FILESis used by the bootimg-partition wic plugin to put the files into a boot partition.
Be able to list files as "boot files" in e.g. your .conf or image files instead of install those in every recipe is a good thing.It is not always desired to have a separate boot partition for boot files. Sometimes it could be good to have them as a part of the root filesystem.
For example, if a double copy strategy is used for update the system, then you probably want to update both the boot files and root filesystem at the same time as there may be dependencies.
v2:
- Removed the documentation from the patch series (will be submitted later)
- Break out the parts in bootimg-partition that is used by image-bootfiles to a common library
- Make the destination directory in root filesystem configurable
v3:
- See changelog in patchesv4:
- See changelog in patchesv5:
- See changelog in patchesMarcus Folkesson (2):
bootimg-partition: break out code to a common library. image-bootfiles.bbclass: new class, copy boot files to root filesystem
Quoted 10 lines
meta/classes/image-bootfiles.bbclass | 41 +++++++++++++ meta/lib/oe/bootfiles.py | 57 +++++++++++++++++++ .../wic/plugins/source/bootimg-partition.py | 39 +------------ 3 files changed, 100 insertions(+), 37 deletions(-) create mode 100644 meta/classes/image-bootfiles.bbclass create mode 100644 meta/lib/oe/bootfiles.pySignature · 1 line
2.45.1
Any more thoughts about these patches?
Thank you,
Marcus Folkesson
Konrad Weihmann
Reviewed-byQuoted 45 lines · Marcus Folkesson
Hi all,
Quoted 41 lines · Marcus Folkesson
The image-bootfiles class is used to put all files listed in
IMAGE_BOOT_FILESinto the root filesystem.
IMAGE_BOOT_FILESis used by the bootimg-partition wic plugin to put the files into a boot partition.
Be able to list files as "boot files" in e.g. your .conf or image files instead of install those in every recipe is a good thing.It is not always desired to have a separate boot partition for boot files. Sometimes it could be good to have them as a part of the root filesystem.
For example, if a double copy strategy is used for update the system, then you probably want to update both the boot files and root filesystem at the same time as there may be dependencies.
v2:
- Removed the documentation from the patch series (will be submitted later)
- Break out the parts in bootimg-partition that is used by image-bootfiles to a common library
- Make the destination directory in root filesystem configurable
v3:
- See changelog in patchesv4:
- See changelog in patchesv5:
- See changelog in patchesMarcus Folkesson (2):
bootimg-partition: break out code to a common library. image-bootfiles.bbclass: new class, copy boot files to root filesystemmeta/classes/image-bootfiles.bbclass | 41 +++++++++++++ meta/lib/oe/bootfiles.py | 57 +++++++++++++++++++ .../wic/plugins/source/bootimg-partition.py | 39 +------------ 3 files changed, 100 insertions(+), 37 deletions(-)create mode 100644 meta/classes/image-bootfiles.bbclass create mode 100644
meta/lib/oe/bootfiles.py
Quoted 8 lines
Quoted 3 lines
Signature · 1 line
2.45.1Any more thoughts about these patches?
Thank you,
Marcus Folkesson
LGTM