Add multi-kernel FIT image support
v3 cover Add multi-kernel FIT image support
From: Vahid Gharaee <[email protected]>
This patch series extends FIT image generation to support multiple kernel images and custom configurations. It allows different kernels, such as standard and real-time kernels, to be included in a single FIT image and selected at boot time.
Vahid Gharaee (6):
oe.fitimage: Support multiple kernel nodes and configuration selection oe.fitimage: add metadata for multiple kernels oe.selftest: Add tests for multi-kernel FIT config oe.fitimage: Add custom configurations and harden multi-kernel
handling
oe.selftest: Improve FIT DTB alias validation
kernel-fit-image: preserve vendored DTB aliases for external overrides
.../recipes-test/ext-dtb/test-dtbs-as-ext.bb | 11 + meta/classes-recipe/kernel-fit-image.bbclass | 143 ++++++- meta/conf/image-fitimage.conf | 55 +++ meta/lib/oe/fitimage.py | 95 ++++- meta/lib/oeqa/selftest/cases/fitimage.py | 352 ++++++++++++++++-- 5 files changed, 604 insertions(+), 52 deletions(-)
base-commit: 5ffd9e055469165d486048cd1f505058dbc35b63
Signature · 1 line
2.55.0
PATCH v3 1/6 oe.fitimage: Support multiple kernel nodes and configuration selection
From: Vahid Gharaee <[email protected]>
--- meta/lib/oe/fitimage.py | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
meta/lib/oe/fitimage.py +12 -6
@@ -193,6 +193,7 @@ class ItsNodeRootKernel(ItsNode): self._dtbs = [] self._dtb_alias = [] self._kernel = None + self._kernels = {} self._ramdisk = None self._bootscr = None self._setup = None
@@ -268,8 +269,8 @@ class ItsNodeRootKernel(ItsNode): def fitimage_emit_section_kernel(self, kernel_id, kernel_path, compression, load, entrypoint, mkimage_kernel_type, entrysymbol=None): """Emit the fitImage ITS kernel section""" - if self._kernel: - bb.fatal("Kernel section already exists in the ITS file.") + if kernel_id in self._kernels: + bb.fatal("Kernel section '%s' already exists in the ITS file." % kernel_id) if entrysymbol: result = subprocess.run([self._host_prefix + "nm", "vmlinux"], capture_output=True, text=True) for line in result.stdout.splitlines():
@@ -294,7 +295,10 @@ class ItsNodeRootKernel(ItsNode): compression, opt_props, ) - self._kernel = kernel_node + self._kernels[kernel_id] = kernel_node + + if self._kernel is None: + self._kernel = kernel_node def fitimage_emit_section_dtb(self, dtb_id, dtb_path, dtb_loadaddress=None, dtbo_loadaddress=None, add_compatible=False):
@@ -425,7 +429,7 @@ class ItsNodeRootKernel(ItsNode): self._loadables.append(loadable_node) - def _fitimage_emit_one_section_config(self, conf_node_name, dtb=None): + def _fitimage_emit_one_section_config(self, conf_node_name, dtb=None, kernel=None): """Emit the fitImage ITS configuration section""" opt_props = {} conf_desc = []
@@ -436,9 +440,11 @@ class ItsNodeRootKernel(ItsNode): if dtb and os.path.splitext(dtb.name)[1] == ".dtbo": include_kernel = 0 - if self._kernel and include_kernel: + selected_kernel = kernel if kernel is not None else self._kernel + + if selected_kernel and include_kernel: conf_desc.append("Linux kernel") - opt_props["kernel"] = self._kernel.name + opt_props["kernel"] = selected_kernel.name if self._sign_enable: sign_entries.append("kernel")
Signature · 1 line
2.55.0
PATCH v3 2/6 oe.fitimage: add metadata for multiple kernels
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
--- meta/conf/image-fitimage.conf | 55 +++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+)
meta/conf/image-fitimage.conf +55 -0
@@ -102,3 +102,58 @@ FIT_UBOOT_ENV ?= "" # arch, os) if the corresponding flag is not set, while others (load address # and entry point) will be omitted in the final FIT. FIT_LOADABLES ?= "" + +# Allow user to insert additional kernel images, e.g. a second kernel built by +# another recipe or multiconfig, or a prebuilt kernel. The default kernel +# (kernel-1) from virtual/kernel is always included. Each entry is used as the +# image node name and must not collide with other image nodes, e.g. kernel-1. +# Entries can be configured through variable flags: +# FIT_KERNEL_FILENAME[id] - kernel file, relative to FIT_KERNEL_DEPLOYDIR +# (default: linux.bin, mandatory for prebuilt +# kernels) +# FIT_KERNEL_DEPLOYDIR[id] - directory containing the kernel file +# (default: ${DEPLOY_DIR_IMAGE}) +# FIT_KERNEL_RECIPE[id] - recipe whose do_deploy provides the kernel +# (default: none, the file must already exist). +# Without FIT_KERNEL_MULTICONFIG the kernel file +# must be deployed by this recipe. +# FIT_KERNEL_MULTICONFIG[id] - multiconfig building FIT_KERNEL_RECIPE[id] +# (default recipe: virtual/kernel), requires +# FIT_KERNEL_DEPLOYDIR[id] +# FIT_KERNEL_COMPRESSION[id] - default: for linux.bin deployed by +# FIT_KERNEL_RECIPE or FIT_KERNEL_MULTICONFIG the +# content of linux_comp next to it, else none +# FIT_KERNEL_LOADADDRESS[id] - default: ${UBOOT_LOADADDRESS} +# FIT_KERNEL_ENTRYPOINT[id] - default: ${UBOOT_ENTRYPOINT} +# FIT_KERNEL_TYPE[id] - default: ${UBOOT_MKIMAGE_KERNEL_TYPE} +# FIT_KERNEL_DESCRIPTION[id] - default: "Linux kernel" +# Example, a kernel from the "rt" multiconfig with the same MACHINE: +# FIT_KERNELS = "kernel-rt" +# FIT_KERNEL_MULTICONFIG[kernel-rt] = "rt" +# FIT_KERNEL_DEPLOYDIR[kernel-rt] = "${TOPDIR}/tmp-rt/deploy/images/${MACHINE}" +# Example, a second kernel recipe with KERNEL_PACKAGE_NAME = "kernel-rt": +# FIT_KERNELS = "kernel-rt" +# FIT_KERNEL_RECIPE[kernel-rt] = "linux-yocto-rt" +# FIT_KERNEL_DEPLOYDIR[kernel-rt] = "${DEPLOY_DIR_IMAGE}/kernel-rt" +# Additional kernels are only referenced by configurations from FIT_CONFIGS. +FIT_KERNELS ?= "" + +# Allow user to add configurations in addition to the ones generated per DTB +# (or conf-1 without DTBs). Each entry results in a ${FIT_CONF_PREFIX}<id> +# configuration node which can be configured through variable flags: +# FIT_CONFIG_KERNEL[id] - kernel image node, e.g. one of FIT_KERNELS +# (default: kernel-1) +# FIT_CONFIG_DTB[id] - DTB name as in KERNEL_DEVICETREE or a file name in +# EXTERNAL_KERNEL_DEVICETREE (default: none) +# The ramdisk, u-boot script, setup and loadables are referenced by all +# configurations. +# Example: +# FIT_CONFIGS = "rt" +# FIT_CONFIG_KERNEL[rt] = "kernel-rt" +# FIT_CONFIG_DTB[rt] = "am335x-boneblack.dtb" +FIT_CONFIGS ?= "" + +# Allow user to select the default configuration by name without +# FIT_CONF_PREFIX, e.g. "rt" for "conf-rt". Takes precedence over +# FIT_CONF_DEFAULT_DTB. +FIT_CONF_DEFAULT ?= ""
Signature · 1 line
2.55.0
PATCH v3 3/6 oe.selftest: Add tests for multi-kernel FIT config
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
Add selftests for FIT images containing multiple kernel nodes and configurations.
AI-Generated: Uses GitHub copilot (Claude Opus 5.5)
--- meta/lib/oeqa/selftest/cases/fitimage.py | 156 ++++++++++++++++++++++- 1 file changed, 155 insertions(+), 1 deletion(-)
meta/lib/oeqa/selftest/cases/fitimage.py +155 -1
@@ -461,6 +461,30 @@ class FitImageTestCase(OESelftestTestCase): loadables += var_loadables.split() return loadables + @staticmethod + def _get_flag_values(bb_vars, var, items): + """Map items to the per-item values of a synthetic "a ? b" flag variable""" + values = (bb_vars.get(var) or "").split("?") + return {item: values[index].strip() if index < len(values) else "" for index, item in enumerate(items)} + + @staticmethod + def _get_extra_configs(bb_vars): + """Return a list of (conf, kernel, dtb image) tuples for FIT_CONFIGS""" + configs = (bb_vars.get('FIT_CONFIGS') or "").split() + kernels = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_KERNEL', configs) + dtbs = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_DTB', configs) + _, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) + extra_configs = [] + for conf in configs: + dtb = dtbs[conf] + if dtb in dtb_symlinks: + # Symlinks of test-dtbs-as-ext point to the DTB without "-alias" + dtb = dtb.replace("-alias", "") + elif dtb: + dtb = FitImageTestCase._apply_dtbvendored(bb_vars, dtb) + extra_configs.append((conf, kernels[conf] or "kernel-1", dtb)) + return extra_configs + def _is_req_dict_in_dict(self, found_dict, req_dict): """ Check if all key-value pairs in the required dictionary are present in the found dictionary.
@@ -670,11 +694,16 @@ class KernelFitImageBase(FitImageTestCase): internal_used = { 'DEPLOY_DIR_IMAGE', 'FIT_CONF_DEFAULT_DTB', + 'FIT_CONF_DEFAULT', 'FIT_CONF_MAPPINGS', 'FIT_CONF_PREFIX', + 'FIT_CONFIGS', + 'FIT_CONFIG_DTB', + 'FIT_CONFIG_KERNEL', 'FIT_DESC', 'FIT_HASH_ALG', 'FIT_KERNEL_COMP_ALG', + 'FIT_KERNELS', 'FIT_LOADABLES', 'FIT_LOADABLE_ENTRYPOINT', 'FIT_LOADABLE_LOADADDRESS',
@@ -835,6 +864,8 @@ class KernelFitImageBase(FitImageTestCase): if loadables: images += loadables + images += (bb_vars.get('FIT_KERNELS') or "").split() + # configuration nodes (one per DTB, symlink, and mappings) configurations = [] if dtb_files:
@@ -884,6 +915,8 @@ class KernelFitImageBase(FitImageTestCase): else: configurations = [bb_vars['FIT_CONF_PREFIX'] + '1'] + configurations += [bb_vars['FIT_CONF_PREFIX'] + conf for conf, _, _ in FitImageTestCase._get_extra_configs(bb_vars)] + # Create a list of paths for all image and configuration nodes req_its_paths = [] for image in images:
@@ -926,8 +959,12 @@ class KernelFitImageBase(FitImageTestCase): if uboot_rd_entrypoint: its_field_check.append("entry = <%s>;" % uboot_rd_entrypoint) + fit_conf_default = bb_vars.get('FIT_CONF_DEFAULT') fit_conf_default_dtb = bb_vars.get('FIT_CONF_DEFAULT_DTB') - if fit_conf_default_dtb: + if fit_conf_default: + fit_conf_prefix = bb_vars.get('FIT_CONF_PREFIX', "conf-") + its_field_check.append('default = "' + fit_conf_prefix + fit_conf_default + '";') + elif fit_conf_default_dtb: fit_conf_prefix = bb_vars.get('FIT_CONF_PREFIX', "conf-") its_field_check.append('default = "' + fit_conf_prefix + self._apply_dtbvendored(bb_vars, fit_conf_default_dtb) + '";')
@@ -951,6 +988,12 @@ class KernelFitImageBase(FitImageTestCase): if initramfs_image and initramfs_image_bundle != "1": its_field_check.append('ramdisk = "ramdisk-1";') + # Additional configurations are emitted after the generated ones + for _, kernel, dtb in FitImageTestCase._get_extra_configs(bb_vars): + its_field_check.append('kernel = "%s";' % kernel) + if dtb: + its_field_check.append('fdt = "fdt-%s";' % dtb) + return its_field_check def _get_req_sigvalues_config(self, bb_vars):
@@ -1062,6 +1105,15 @@ class KernelFitImageBase(FitImageTestCase): if loadables: req_sections[conf_name]['Loadables'] = ",".join(loadables) + # Additional kernels and configurations + for kernel in (bb_vars.get('FIT_KERNELS') or "").split(): + req_sections[kernel] = { "Type": "Kernel Image" } + for conf, kernel, dtb in FitImageTestCase._get_extra_configs(bb_vars): + conf_name = bb_vars['FIT_CONF_PREFIX'] + conf + req_sections[conf_name] = { "Kernel": kernel } + if dtb: + req_sections[conf_name]['FDT'] = 'fdt-' + dtb + # Add signing related properties if needed if uboot_sign_enable == "1": for section in req_sections:
@@ -1208,6 +1260,32 @@ FIT_LOADABLE_TYPE[loadable2] = "firmware" self._gen_atf_tee_dummy_images(bb_vars) self._test_fitimage(bb_vars) + def test_fit_image_multiple_kernels(self): + """ + Summary: Check FIT_KERNELS, FIT_CONFIGS and FIT_CONF_DEFAULT + Expected: An additional kernel image node and a configuration + referring to it are present and the configuration is + the default one. + Product: oe-core + """ + machine = self._get_machine_or_skip() + config = """ +KERNEL_IMAGETYPE = "Image" +UBOOT_LOADADDRESS = "0x80080000" +UBOOT_ENTRYPOINT = "0x80080000" +# Use the linux.bin of virtual/kernel also as second kernel to avoid building another kernel +FIT_KERNELS = "kernel-2" +FIT_KERNEL_RECIPE[kernel-2] = "virtual/kernel" +FIT_CONFIGS = "second" +FIT_CONFIG_KERNEL[second] = "kernel-2" +FIT_CONF_DEFAULT = "second" +""" + config = FitImageTestCase._config_add_machine_settings(config, machine, keys=["KERNEL_DEVICETREE"]) + config = self._config_add_kernel_classes(config) + self.write_config(config) + bb_vars = self._fit_get_bb_vars() + self._test_fitimage(bb_vars) + def test_get_compatible_from_dtb(self): """Test the oe.fitimage.get_compatible_from_dtb function
@@ -1501,11 +1579,14 @@ class FitImagePyTests(KernelFitImageBase): # image-fitimage.conf 'FIT_ADDRESS_CELLS': "1", 'FIT_CONF_DEFAULT_DTB': "", + 'FIT_CONF_DEFAULT': "", 'FIT_CONF_MAPPINGS': "", 'FIT_CONF_PREFIX': "conf-", + 'FIT_CONFIGS': "", 'FIT_DESC': "Kernel fitImage for a dummy distro", 'FIT_GENERATE_KEYS': "0", 'FIT_HASH_ALG': "sha256", + 'FIT_KERNELS': "", 'FIT_KEY_GENRSA_ARGS': "-F4", 'FIT_KEY_REQ_ARGS': "-batch -new", 'FIT_KEY_SIGN_PKCS': "-x509",
@@ -1566,6 +1647,11 @@ class FitImagePyTests(KernelFitImageBase): bb_vars.get('UBOOT_MKIMAGE_KERNEL_TYPE'), bb_vars.get("UBOOT_ENTRYSYMBOL") ) + for kernel in bb_vars['FIT_KERNELS'].split(): + root_node.fitimage_emit_section_kernel(kernel, "linux-%s.bin" % kernel, "none", + bb_vars.get('UBOOT_LOADADDRESS'), bb_vars.get('UBOOT_ENTRYPOINT'), + bb_vars.get('UBOOT_MKIMAGE_KERNEL_TYPE')) + dtb_files, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) for dtb in dtb_files: root_node.fitimage_emit_section_dtb(dtb, os.path.join("a-dir", dtb),
@@ -1596,9 +1682,19 @@ class FitImagePyTests(KernelFitImageBase): "loadable-type") root_node.fitimage_emit_section_config(bb_vars['FIT_CONF_DEFAULT_DTB'], bb_vars.get('FIT_CONF_MAPPINGS')) + configs = bb_vars['FIT_CONFIGS'].split() + conf_kernels = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_KERNEL', configs) + conf_dtbs = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_DTB', configs) + for conf in configs: + root_node.fitimage_emit_section_config_custom(conf, conf_kernels[conf], conf_dtbs[conf]) + if bb_vars['FIT_CONF_DEFAULT']: + root_node.fitimage_set_default_config(bb_vars['FIT_CONF_DEFAULT']) root_node.write_its_file(fitimage_its_path) self.assertExists(fitimage_its_path, "%s image tree source doesn't exist" % (fitimage_its_path)) + with open(fitimage_its_path) as its_file: + default_lines = [line for line in its_file if line.strip().startswith("default = ")] + self.assertEqual(len(default_lines), 1, "Expected exactly one default configuration: %s" % default_lines) self.logger.debug("Checking its: %s" % fitimage_its_path) self._check_its_file(bb_vars, fitimage_its_path)
@@ -1665,6 +1761,64 @@ class FitImagePyTests(KernelFitImageBase): } self._test_fitimage_py(bb_vars_overrides) + def test_fitimage_py_multiple_kernels(self): + """Test FIT_KERNELS, FIT_CONFIGS and FIT_CONF_DEFAULT""" + bb_vars_overrides = { + 'KERNEL_DEVICETREE': "one.dtb two.dtb", + 'FIT_CONF_DEFAULT_DTB': "one.dtb", + 'FIT_KERNELS': "kernel-rt", + 'FIT_CONFIGS': "rt rt-two default-two", + 'FIT_CONFIG_KERNEL': "kernel-rt ? kernel-rt ? ", + 'FIT_CONFIG_DTB': " ? two.dtb ? two.dtb", + 'FIT_CONF_DEFAULT': "rt-two", + } + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_multiple_kernels_unknown_kernel_error(self): + """Test that FIT_CONFIG_KERNEL referring to an unknown kernel causes a fatal error""" + bb_vars_overrides = { + 'FIT_CONFIGS': "rt", + 'FIT_CONFIG_KERNEL': "kernel-rt", + } + with self.assertRaises(BBHandledException): + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_conf_default_unknown_error(self): + """Test that FIT_CONF_DEFAULT referring to an unknown configuration causes a fatal error""" + bb_vars_overrides = { + 'FIT_CONF_DEFAULT': "nonexistent", + } + with self.assertRaises(BBHandledException): + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_multiple_kernels_dtb_alias(self): + """Test FIT_CONFIG_DTB referring to an external DTB symlink""" + bb_vars_overrides = { + 'PREFERRED_PROVIDER_virtual/dtb': "test-dtbs-as-ext", + 'FIT_KERNELS': "kernel-rt", + 'FIT_CONFIGS': "rt", + 'FIT_CONFIG_KERNEL': "kernel-rt", + 'FIT_CONFIG_DTB': "test-ext-alias.dtb", + } + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_multiple_kernels_kernel_1_error(self): + """Test that a FIT_KERNELS entry colliding with kernel-1 causes a fatal error""" + bb_vars_overrides = { + 'FIT_KERNELS': "kernel-1", + } + with self.assertRaises(BBHandledException): + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_multiple_kernels_image_collision_error(self): + """Test that a FIT_KERNELS entry colliding with another image node causes a fatal error""" + bb_vars_overrides = { + 'FIT_KERNELS': "my-loadable", + 'FIT_LOADABLES': "my-loadable", + } + with self.assertRaises(BBHandledException): + self._test_fitimage_py(bb_vars_overrides) + class UBootFitImageTests(FitImageTestCase): """Test cases for the uboot-sign bbclass"""
Signature · 1 line
2.55.0
PATCH v3 4/6 oe.fitimage: Add custom configurations and harden multi-kernel handling
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
One can now add configs that point to any kernel node and, optionally, a DTB with fitimage_emit_section_config_custom(). The default config still can be chosen by name with fitimage_set_default_config().
The default configuration is now stored and written to the ITS once, in write_its_file(). Before, fitimage_set_default_config() had to overwrite the property that fitimage_emit_section_config() had already set.
DTBs and DTB symlink aliases are now looked up by the id they were added with. Before, the lookup depended on the "fdt-" node name or the alias name.
Reject explicit kernel selection for FIT overlays and validate configuration generation order.
Tested with oe-selftest -r fitimage.FitImagePyTests.
--- meta/classes-recipe/kernel-fit-image.bbclass | 136 ++++++++++++++++--- meta/lib/oe/fitimage.py | 85 ++++++++++-- meta/lib/oeqa/selftest/cases/fitimage.py | 37 +++++ 3 files changed, 231 insertions(+), 27 deletions(-)
meta/classes-recipe/kernel-fit-image.bbclass +120 -16
@@ -29,22 +29,57 @@ python () { d.appendVarFlag('do_compile', 'depends', ' virtual/dtb:do_populate_sysroot') d.setVar('EXTERNAL_KERNEL_DEVICETREE', "${RECIPE_SYSROOT}/boot/devicetree") - # Parse loadables config - loadables = d.getVar('FIT_LOADABLES') - if loadables: - # Forbid direct assignment, set synthetic values for testability and - # add the vars to vardeps so their flags are part of the signature. - loadable_vars = ['FIT_LOADABLE_ARCH', 'FIT_LOADABLE_COMPRESSION', - 'FIT_LOADABLE_DESCRIPTION', 'FIT_LOADABLE_ENTRYPOINT', - 'FIT_LOADABLE_FILENAME', 'FIT_LOADABLE_LOADADDRESS', - 'FIT_LOADABLE_OS', 'FIT_LOADABLE_TYPE'] - for v in loadable_vars: - if d.getVar(v): - raise bb.parse.SkipRecipe("You cannot use %s as a variable, you can only set flags." % v) - - synt_value = " ? ".join([ d.getVarFlag(v, loadable) or "" for loadable in loadables.split() ]) - d.setVar(v, synt_value) - d.appendVarFlag('do_compile', 'vardeps', ' ' + ' '.join(loadable_vars)) + # Parse loadables, additional kernels and additional configurations + for items_var, flag_vars in ( + ('FIT_LOADABLES', ['FIT_LOADABLE_ARCH', 'FIT_LOADABLE_COMPRESSION', + 'FIT_LOADABLE_DESCRIPTION', 'FIT_LOADABLE_ENTRYPOINT', + 'FIT_LOADABLE_FILENAME', 'FIT_LOADABLE_LOADADDRESS', + 'FIT_LOADABLE_OS', 'FIT_LOADABLE_TYPE']), + ('FIT_KERNELS', ['FIT_KERNEL_COMPRESSION', 'FIT_KERNEL_DEPLOYDIR', + 'FIT_KERNEL_DESCRIPTION', 'FIT_KERNEL_ENTRYPOINT', + 'FIT_KERNEL_FILENAME', 'FIT_KERNEL_LOADADDRESS', + 'FIT_KERNEL_MULTICONFIG', 'FIT_KERNEL_RECIPE', + 'FIT_KERNEL_TYPE']), + ('FIT_CONFIGS', ['FIT_CONFIG_DTB', 'FIT_CONFIG_KERNEL'])): + items = d.getVar(items_var) + if items: + # Check that the configuration variables are not set directly, + # then assign them a synthetic value for testability purposes + for v in flag_vars: + if d.getVar(v): + raise bb.parse.SkipRecipe("You cannot use %s as a variable, you can only set flags." % v) + + d.appendVarFlag('do_compile', 'vardeps', ' ' + v) + + synt_value = " ? ".join([ d.getVarFlag(v, item, False) or "" for item in items.split() ]) + d.setVar(v, synt_value) + + # Preserve upstream's task signature tracking + d.appendVarFlag('do_compile', 'vardeps', ' ' + ' '.join(flag_vars)) + + # Additional kernels may be deployed by another recipe, by a recipe of + # another multiconfig or be provided as prebuilt files + kernels = (d.getVar('FIT_KERNELS') or "").split() + if "kernel-1" in kernels: + bb.fatal("FIT_KERNELS must not contain kernel-1, it is reserved for virtual/kernel") + duplicates = sorted(set(k for k in kernels if kernels.count(k) > 1)) + if duplicates: + bb.fatal("FIT_KERNELS contains duplicate entries: %s" % ' '.join(duplicates)) + for kernel in kernels: + recipe = d.getVarFlag('FIT_KERNEL_RECIPE', kernel) + mc = d.getVarFlag('FIT_KERNEL_MULTICONFIG', kernel) + if mc: + if not d.getVarFlag('FIT_KERNEL_DEPLOYDIR', kernel): + bb.fatal("FIT_KERNEL_DEPLOYDIR[%s] must be set if FIT_KERNEL_MULTICONFIG[%s] is set" % (kernel, kernel)) + d.appendVarFlag('do_compile', 'mcdepends', ' mc:%s:%s:%s:do_deploy' % + (d.getVar('BB_CURRENT_MC'), mc, recipe or 'virtual/kernel')) + elif recipe: + d.appendVarFlag('do_compile', 'depends', ' %s:do_deploy' % recipe) + else: + if not d.getVarFlag('FIT_KERNEL_FILENAME', kernel): + bb.fatal("FIT_KERNEL_FILENAME[%s] must be set if neither FIT_KERNEL_RECIPE[%s] nor FIT_KERNEL_MULTICONFIG[%s] is set" % (kernel, kernel, kernel)) + kernel_path = fit_kernel_path(d, kernel) + d.appendVarFlag('do_compile', 'file-checksums', ' %s:%s' % (kernel_path, os.path.exists(kernel_path))) } do_configure[noexec] = "1"
@@ -64,6 +99,30 @@ def apply_dtbvendored(d,dtb): mapped_name = dtb.replace('/', '_') return mapped_name +def fit_kernel_path(d, kernel): + filename = d.getVarFlag('FIT_KERNEL_FILENAME', kernel) or "linux.bin" + deploydir = d.getVarFlag('FIT_KERNEL_DEPLOYDIR', kernel) or d.getVar('DEPLOY_DIR_IMAGE') + return os.path.join(deploydir, filename) + +def fit_kernel_deployed_files(d, recipe): + """Return the real paths of all files deployed by do_deploy of recipe (PN or PROVIDES) in this multiconfig""" + import bb.runqueue + mc = d.getVar('BB_CURRENT_MC') or "" + taskdepdata = d.getVar('BB_TASKDEPDATA', False) + pns = set(dep.pn for tid, dep in taskdepdata.items() + if dep.taskname == "do_deploy" and bb.runqueue.mc_from_tid(tid) == mc and + (dep.pn == recipe or recipe in dep.provides)) + if len(pns) != 1: + bb.fatal("Could not identify the do_deploy dependency providing %s (found: %s)" % (recipe, ' '.join(sorted(pns)) or "none")) + pn = pns.pop() + for arch in d.getVar('SSTATE_ARCHS').split(): + manifest = os.path.join(d.getVar('SSTATE_MANIFESTS'), "manifest-%s-%s.deploy" % (arch, pn)) + if os.path.exists(manifest): + with open(manifest) as f: + return set(os.path.realpath(line) for line in f.read().splitlines() if line and not line.endswith('/')) + bb.fatal("No deploy manifest found for %s" % pn) +fit_kernel_deployed_files[vardepsexclude] = "BB_TASKDEPDATA SSTATE_ARCHS" + python do_compile() { import shutil import oe.fitimage
@@ -97,6 +156,39 @@ python do_compile() { d.getVar('UBOOT_LOADADDRESS'), d.getVar('UBOOT_ENTRYPOINT'), d.getVar('UBOOT_MKIMAGE_KERNEL_TYPE'), d.getVar("UBOOT_ENTRYSYMBOL")) + # Prepare additional kernel image sections + for kernel in d.getVar('FIT_KERNELS').split(): + recipe = d.getVarFlag('FIT_KERNEL_RECIPE', kernel) + mc = d.getVarFlag('FIT_KERNEL_MULTICONFIG', kernel) + kernel_path = fit_kernel_path(d, kernel) + if not os.path.exists(kernel_path): + bb.fatal("File for kernel %s not found at %s" % (kernel, kernel_path)) + # Artifacts of another multiconfig cannot be verified, its FIT_KERNEL_DEPLOYDIR is trusted + deployed = None + if recipe and not mc: + deployed = fit_kernel_deployed_files(d, recipe) + if os.path.realpath(kernel_path) not in deployed: + bb.fatal("File for kernel %s at %s is not deployed by %s" % (kernel, kernel_path, recipe)) + kernel_file = "linux-%s.bin" % kernel + shutil.copyfile(kernel_path, kernel_file) + + kernel_comp = d.getVarFlag('FIT_KERNEL_COMPRESSION', kernel) + if not kernel_comp: + kernel_comp = "none" + # linux_comp only describes linux.bin written by kernel-fit-extra-artifacts of the same recipe + if (recipe or mc) and os.path.basename(kernel_path) == "linux.bin": + kernel_comp_path = os.path.join(os.path.dirname(kernel_path), "linux_comp") + if not os.path.exists(kernel_comp_path) or \ + (deployed is not None and os.path.realpath(kernel_comp_path) not in deployed): + bb.fatal("%s not found or not deployed by the kernel recipe, set FIT_KERNEL_COMPRESSION[%s]" % (kernel_comp_path, kernel)) + with open(kernel_comp_path) as kernel_comp_f: + kernel_comp = kernel_comp_f.read().strip() + root_node.fitimage_emit_section_kernel(kernel, kernel_file, kernel_comp, + d.getVarFlag('FIT_KERNEL_LOADADDRESS', kernel) or d.getVar('UBOOT_LOADADDRESS'), + d.getVarFlag('FIT_KERNEL_ENTRYPOINT', kernel) or d.getVar('UBOOT_ENTRYPOINT'), + d.getVarFlag('FIT_KERNEL_TYPE', kernel) or d.getVar('UBOOT_MKIMAGE_KERNEL_TYPE'), + description=d.getVarFlag('FIT_KERNEL_DESCRIPTION', kernel) or "Linux kernel") + # Prepare a DTB image section kernel_devicetree = d.getVar('KERNEL_DEVICETREE') external_kernel_devicetree = d.getVar("EXTERNAL_KERNEL_DEVICETREE")
@@ -212,6 +304,18 @@ python do_compile() { # Generate the configuration section root_node.fitimage_emit_section_config(default_dtb, d.getVar("FIT_CONF_MAPPINGS")) + # Generate additional configurations, e.g. referring to additional kernels + for conf in d.getVar('FIT_CONFIGS').split(): + conf_dtb = d.getVarFlag('FIT_CONFIG_DTB', conf) + if conf_dtb: + conf_dtb = apply_dtbvendored(d, conf_dtb) + root_node.fitimage_emit_section_config_custom(conf, + d.getVarFlag('FIT_CONFIG_KERNEL', conf), conf_dtb) + + fit_conf_default = d.getVar('FIT_CONF_DEFAULT') + if fit_conf_default: + root_node.fitimage_set_default_config(fit_conf_default) + # Write the its file root_node.write_its_file(itsfile)
meta/lib/oe/fitimage.py +74 -11
173 lines not on this page · raw message
meta/lib/oeqa/selftest/cases/fitimage.py +37 -0
@@ -1569,6 +1569,43 @@ KERNEL_DTBVENDORED = "1" class FitImagePyTests(KernelFitImageBase): """Test cases for the fitimage.py module without calling bitbake""" + def test_custom_config_rejects_kernel_with_overlay(self): + """An overlay configuration must not silently ignore an explicit kernel.""" + root = oe.fitimage.ItsNodeRootKernel( + "Test FIT", "1", "x86_64-poky-linux-", "x86", + "linux", "conf-", False, None, + "uboot-mkimage", "", "uboot-mkimage", "", + "sha256", "rsa2048", "pkcs-1.5", + None, False, None + ) + + root.fitimage_emit_section_kernel( + "kernel-1", "linux.bin", "none", + "0x20008000", "0x20008000", "kernel" + ) + + root.fitimage_emit_section_dtb( + "test-overlay.dtbo", "a-dir/test-overlay.dtbo" + ) + + with self.assertRaises(bb.BBHandledException): + root.fitimage_emit_section_config_custom( + "invalid", "kernel-1", "test-overlay.dtbo" + ) + + # A regular DTB may explicitly select a kernel. + root.fitimage_emit_section_dtb( + "test-board.dtb", "a-dir/test-board.dtb" + ) + + root.fitimage_emit_section_config_custom( + "regular", "kernel-1", "test-board.dtb" + ) + + # An overlay without an explicit kernel is valid. + root.fitimage_emit_section_config_custom( + "overlay", None, "test-overlay.dtbo" + ) def _test_fitimage_py(self, bb_vars_overrides=None): topdir = os.path.join(os.environ['BUILDDIR'])
Signature · 1 line
2.55.0
PATCH v3 5/6 oe.selftest: Improve FIT DTB alias validation
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
Use explicit DTB alias targets instead of relying on FIT node naming conventions.
Discover external DTBs and symlink aliases when generating expected FIT configurations, including vendored DTB names.
--- meta/lib/oeqa/selftest/cases/fitimage.py | 80 ++++++++++++++++-------- 1 file changed, 55 insertions(+), 25 deletions(-)
meta/lib/oeqa/selftest/cases/fitimage.py +55 -25
@@ -432,6 +432,30 @@ class FitImageTestCase(OESelftestTestCase): mapped_name = dtb.replace('/', '_') return mapped_name + @staticmethod + def _get_test_dtb_files(): + return ( + ["test-ext.dtb", "test-overlay.dtbo"], + {"test-ext-alias.dtb": "test-ext.dtb"}, + ) + + @staticmethod + def _get_external_dtb_files(devicetree_dir): + dtbs = [] + aliases = {} + + for name in sorted(os.listdir(devicetree_dir)): + if not name.endswith((".dtb", ".dtbo")): + continue + + target = oe.fitimage.symlink_points_below(name, devicetree_dir) + if target: + aliases[name] = target + else: + dtbs.append(name) + + return dtbs, aliases + @staticmethod def _get_dtb_files(bb_vars): """Return a list of devicetree names
@@ -442,15 +466,22 @@ class FitImageTestCase(OESelftestTestCase): """ kernel_devicetree = bb_vars.get('KERNEL_DEVICETREE') all_dtbs = [] - dtb_symlinks = [] + dtb_aliases = {} + if kernel_devicetree: all_dtbs += [FitImageTestCase._apply_dtbvendored(bb_vars, dtb) for dtb in kernel_devicetree.split()] - # Support only the test recipe which provides 1 devicetree and 1 devicetree overlay - pref_prov_dtb = bb_vars.get('PREFERRED_PROVIDER_virtual/dtb') - if pref_prov_dtb == "test-dtbs-as-ext": - all_dtbs += ["test-ext.dtb", "test-overlay.dtbo"] - dtb_symlinks.append("test-ext-alias.dtb") - return (all_dtbs, dtb_symlinks) + + if bb_vars.get("PREFERRED_PROVIDER_virtual/dtb") == "test-dtbs-as-ext": + if bb_vars.get("_python_only_test"): + external_dtbs, external_aliases = FitImageTestCase._get_test_dtb_files() + else: + devicetree_dir = os.path.join(bb_vars["DEPLOY_DIR_IMAGE"], "devicetree") + external_dtbs, external_aliases = FitImageTestCase._get_external_dtb_files(devicetree_dir) + + all_dtbs.extend(external_dtbs) + dtb_aliases.update(external_aliases) + + return all_dtbs, dtb_aliases @staticmethod def _get_loadables(bb_vars):
@@ -473,13 +504,12 @@ class FitImageTestCase(OESelftestTestCase): configs = (bb_vars.get('FIT_CONFIGS') or "").split() kernels = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_KERNEL', configs) dtbs = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_DTB', configs) - _, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) + _, dtb_aliases = FitImageTestCase._get_dtb_files(bb_vars) extra_configs = [] for conf in configs: dtb = dtbs[conf] - if dtb in dtb_symlinks: - # Symlinks of test-dtbs-as-ext point to the DTB without "-alias" - dtb = dtb.replace("-alias", "") + if dtb in dtb_aliases: + dtb = dtb_aliases[dtb] elif dtb: dtb = FitImageTestCase._apply_dtbvendored(bb_vars, dtb) extra_configs.append((conf, kernels[conf] or "kernel-1", dtb))
@@ -976,10 +1006,10 @@ class KernelFitImageBase(FitImageTestCase): if not dtb.endswith('.dtbo'): its_field_check.append('kernel = "kernel-1";') its_field_check.append('fdt = "fdt-%s";' % dtb) - for dtb in dtb_symlinks: + + for dtb, target in dtb_symlinks.items(): its_field_check.append('kernel = "kernel-1";') - # Works only for tests were the symlink is with -alias suffix - its_field_check.append('fdt = "fdt-%s";' % dtb.replace('-alias', '')) + its_field_check.append('fdt = "fdt-%s";' % target) if initramfs_image and initramfs_image_bundle != "1": its_field_check.append('ramdisk = "ramdisk-1";')
@@ -1076,11 +1106,10 @@ class KernelFitImageBase(FitImageTestCase): req_sections[loadable]['Entry Point'] = entrypoint # Create a configuration section for each DTB if dtb_files: - for dtb in dtb_files + dtb_symlinks: + for dtb in dtb_files + list(dtb_symlinks): conf_name = bb_vars['FIT_CONF_PREFIX'] + dtb - # Assume that DTBs with an "-alias" in its name are symlink DTBs created e.g. by the - # test-dtbs-as-ext test recipe. Make the configuration node pointing to the real DTB. - real_dtb = dtb.replace("-alias", "") + # Resolve DTB aliases using their explicit target mapping. + real_dtb = dtb_symlinks.get(dtb, dtb) # dtb overlays do not refer to a kernel (yet?) if dtb.endswith('.dtbo'): req_sections[conf_name] = {
@@ -1146,7 +1175,7 @@ class KernelFitImageBase(FitImageTestCase): fit_hash_alg_len = FitImageTestCase.MKIMAGE_HASH_LENGTHS[fit_hash_alg] fit_sign_alg_len = FitImageTestCase.MKIMAGE_SIGNATURE_LENGTHS[fit_sign_alg] dtb_files, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) - all_dtb_names = set(dtb_files + dtb_symlinks) + all_dtb_names = set(dtb_files) | set(dtb_symlinks) # The public key is always injected into UBOOT_DTB_IMAGE (-K flag in # uboot-sign.bbclass concat_dtb). For DTB-based configurations, the # same key is also injected into each per-configuration DTB, so either
@@ -1610,7 +1639,7 @@ class FitImagePyTests(KernelFitImageBase): def _test_fitimage_py(self, bb_vars_overrides=None): topdir = os.path.join(os.environ['BUILDDIR']) fitimage_its_path = os.path.join(topdir, self._testMethodName + '.its') - + # Provide variables without calling bitbake bb_vars = { # image-fitimage.conf
@@ -1660,7 +1689,10 @@ class FitImagePyTests(KernelFitImageBase): # x86 kernels produce a setup.bin section; set to the file name to # enable it, or to "" / omit the key to suppress it. 'KERNEL_SETUP_BIN': 'setup1.bin', + # ... existing entries ... + '_python_only_test': True, } + if bb_vars_overrides: bb_vars.update(bb_vars_overrides) if logging.DEBUG >= self.logger.level:
@@ -1689,15 +1721,12 @@ class FitImagePyTests(KernelFitImageBase): bb_vars.get('UBOOT_LOADADDRESS'), bb_vars.get('UBOOT_ENTRYPOINT'), bb_vars.get('UBOOT_MKIMAGE_KERNEL_TYPE')) - dtb_files, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) + dtb_files, dtb_aliases = FitImageTestCase._get_dtb_files(bb_vars) for dtb in dtb_files: root_node.fitimage_emit_section_dtb(dtb, os.path.join("a-dir", dtb), bb_vars.get("UBOOT_DTB_LOADADDRESS"), bb_vars.get("UBOOT_DTBO_LOADADDRESS")) - for dtb_symlink in dtb_symlinks: - # For test purposes, assume each symlink points to a DTB with the same basename minus "-alias" - # In this case, "test-ext-alias.dtb" -> "test-ext.dtb" - dtb_target = dtb_symlink.replace("-alias", "") + for dtb_symlink, dtb_target in dtb_aliases.items(): root_node.fitimage_emit_section_dtb_alias(dtb_symlink, os.path.join("a-dir", dtb_target)) if bb_vars.get('FIT_UBOOT_ENV'):
@@ -1910,6 +1939,7 @@ class UBootFitImageTests(FitImageTestCase): 'UBOOT_SIGN_IMG_KEYNAME', 'UBOOT_SIGN_KEYDIR', 'UBOOT_SIGN_KEYNAME', + 'EXTERNAL_KERNEL_DEVICETREE', } bb_vars = get_bb_vars(list(internal_used | set(additional_vars)), UBootFitImageTests.BOOTLOADER_RECIPE) self.logger.debug("bb_vars: %s" % pprint.pformat(bb_vars, indent=4))
Signature · 1 line
2.55.0
PATCH v3 6/6 kernel-fit-image: preserve vendored DTB aliases for external overrides
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
When an external DTB overrides a kernel-provided DTB, the FIT generation skips the kernel DTB. With KERNEL_DTBVENDORED enabled, this also drops the vendored configuration name.
Preserve the vendored name by creating a configuration alias referencing the external DTB image. Resolve external symlinks to their actual DTB targets.
Add regression tests for regular and symlinked external DTB overrides.
--- .../recipes-test/ext-dtb/test-dtbs-as-ext.bb | 11 ++ meta/classes-recipe/kernel-fit-image.bbclass | 7 ++ meta/lib/oeqa/selftest/cases/fitimage.py | 105 ++++++++++++++++-- 3 files changed, 113 insertions(+), 10 deletions(-)
meta-selftest/recipes-test/ext-dtb/test-dtbs-as-ext.bb +11 -0
@@ -8,6 +8,7 @@ LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda inherit devicetree +TEST_EXTRA_DTB ?= "" COMPATIBLE_MACHINE = ".*" SRC_URI = "\
@@ -18,8 +19,18 @@ SRC_URI = "\ # Sym-links are handled as extra configuration nodes in FIT images. do_install:append() { ln -sf test-ext.dtb "${D}/boot/devicetree/test-ext-alias.dtb" + + if [ -n "${TEST_EXTRA_DTB}" ]; then + cp "${D}/boot/devicetree/test-ext.dtb" \ + "${D}/boot/devicetree/${TEST_EXTRA_DTB}" + fi } do_deploy:append() { ln -sf test-ext.dtb "${DEPLOYDIR}/devicetree/test-ext-alias.dtb" + + if [ -n "${TEST_EXTRA_DTB}" ]; then + cp "${DEPLOYDIR}/devicetree/test-ext.dtb" \ + "${DEPLOYDIR}/devicetree/${TEST_EXTRA_DTB}" + fi }
meta/classes-recipe/kernel-fit-image.bbclass +7 -0
@@ -192,6 +192,7 @@ python do_compile() { # Prepare a DTB image section kernel_devicetree = d.getVar('KERNEL_DEVICETREE') external_kernel_devicetree = d.getVar("EXTERNAL_KERNEL_DEVICETREE") + external_dtb_aliases = {} if kernel_devicetree: for dtb in kernel_devicetree.split(): # In deploy_dir the DTBs are without sub-directories also with KERNEL_DTBVENDORED = "1"
@@ -205,6 +206,8 @@ python do_compile() { if external_kernel_devicetree: ext_dtb_path = os.path.join(external_kernel_devicetree, dtb_name) if os.path.exists(ext_dtb_path) and os.path.getsize(ext_dtb_path) > 0: + if conf_name != dtb_name: + external_dtb_aliases[conf_name] = ext_dtb_path continue # Copy the dtb or dtbo file into the FIT image assembly directory
@@ -229,6 +232,10 @@ python do_compile() { root_node.fitimage_emit_section_dtb(dtb_name, dtb_name, d.getVar("UBOOT_DTB_LOADADDRESS"), d.getVar("UBOOT_DTBO_LOADADDRESS"), True) + for alias, dtb_path in external_dtb_aliases.items(): + symlink_target = oe.fitimage.symlink_points_below(os.path.basename(dtb_path), external_kernel_devicetree) + root_node.fitimage_emit_section_dtb_alias(alias, symlink_target or dtb_path) + # Prepare a u-boot script section fit_uboot_env = d.getVar("FIT_UBOOT_ENV") if fit_uboot_env:
meta/lib/oeqa/selftest/cases/fitimage.py +95 -10
157 lines not on this page · raw message
Signature · 1 line
2.55.0
Quoted 9 lines · Vahid Gharaee via openembedded-core
From: Vahid Gharaee <[email protected]>
This patch series extends FIT image generation to support multiple kernel images and custom configurations. It allows different kernels, such as standard and real-time kernels, to be included in a single FIT image and selected at boot time.
Vahid Gharaee (6):
Hi Vahid,
Sorry, I've been a bit slow to reply: this is conflicting with the "kernel-fit-image: drop the synthetic FIT_LOADABLE_* values" patch that was merged a few days ago. Can you rebase on master and send a new version please?
Thanks,
Mathieu
Signature · 3 lines
Mathieu Dubois-Briand, Bootlin Embedded Linux and Kernel engineering https://bootlin.com