kernel-fit-image: Add support for multiple kernels and FIT configurations
v4 cover kernel-fit-image: Add support for multiple kernels and FIT configurations
From: Vahid Gharaee <[email protected]>
These extends kernel-fit-image.bbclass
to support multiple kernel images in a single FIT image, with custom configurations selecting the required kernel and device tree. The main use case is to package different kernel variants, for example a standard and a real-time kernel, into one FIT image. Additional kernels can be provided by another recipe, through BitBake multiconfig, or as prebuilt files. The serie also adds validation for kernel and configuration references, improves handling of DTB aliases,
and extends the FIT selftests.
Changes since v3:
- Rebased onto current OE-Core master.
- Adapted to the upstream removal of synthetic FIT_LOADABLE_* variable values.
- Updated FIT configuration tests to retrieve BitBake variable flags through get_bb_vars_flags().
- Preserved compatibility with existing Python-only FIT test fixtures.
Validation:
- bitbake -p
- oe-selftest -r fitimage.FitImagePyTests (16 passed)
- oe-selftest -r fitimage.KernelFitImageRecipeTests.test_fit_image_multiple_kernels
- oe-selftest -r fitimage.KernelFitImageRecipeTests.test_fit_image_ext_dtb_vendored_override
All passed.
Vahid Gharaee (6):
oe.fitimage: Support multiple kernel nodes and configuration selection oe.fitimage: add metadata for multiple kernels oe.selftest: Add tests for multi-kernel FIT config oe.fitimage: Add custom configurations and harden multi-kernel
handling
oe.selftest: Improve FIT DTB alias validation
kernel-fit-image: preserve vendored DTB aliases for external overrides
.../recipes-test/ext-dtb/test-dtbs-as-ext.bb | 11 + meta/classes-recipe/kernel-fit-image.bbclass | 136 ++++++- meta/conf/image-fitimage.conf | 55 +++ meta/lib/oe/fitimage.py | 95 ++++- meta/lib/oeqa/selftest/cases/fitimage.py | 359 ++++++++++++++++-- 5 files changed, 607 insertions(+), 49 deletions(-)
base-commit: 35d103be8b1a2346b7d0a7ff83745ad74d6817e2
Signature · 1 line
2.56.0
PATCH v4 1/6 oe.fitimage: Support multiple kernel nodes and configuration selection
From: Vahid Gharaee <[email protected]>
--- meta/lib/oe/fitimage.py | 18 ++++++++++++------ 1 file changed, 12 insertions(+), 6 deletions(-)
meta/lib/oe/fitimage.py +12 -6
@@ -193,6 +193,7 @@ class ItsNodeRootKernel(ItsNode): self._dtbs = [] self._dtb_alias = [] self._kernel = None + self._kernels = {} self._ramdisk = None self._bootscr = None self._setup = None
@@ -268,8 +269,8 @@ class ItsNodeRootKernel(ItsNode): def fitimage_emit_section_kernel(self, kernel_id, kernel_path, compression, load, entrypoint, mkimage_kernel_type, entrysymbol=None): """Emit the fitImage ITS kernel section""" - if self._kernel: - bb.fatal("Kernel section already exists in the ITS file.") + if kernel_id in self._kernels: + bb.fatal("Kernel section '%s' already exists in the ITS file." % kernel_id) if entrysymbol: result = subprocess.run([self._host_prefix + "nm", "vmlinux"], capture_output=True, text=True) for line in result.stdout.splitlines():
@@ -294,7 +295,10 @@ class ItsNodeRootKernel(ItsNode): compression, opt_props, ) - self._kernel = kernel_node + self._kernels[kernel_id] = kernel_node + + if self._kernel is None: + self._kernel = kernel_node def fitimage_emit_section_dtb(self, dtb_id, dtb_path, dtb_loadaddress=None, dtbo_loadaddress=None, add_compatible=False):
@@ -425,7 +429,7 @@ class ItsNodeRootKernel(ItsNode): self._loadables.append(loadable_node) - def _fitimage_emit_one_section_config(self, conf_node_name, dtb=None): + def _fitimage_emit_one_section_config(self, conf_node_name, dtb=None, kernel=None): """Emit the fitImage ITS configuration section""" opt_props = {} conf_desc = []
@@ -436,9 +440,11 @@ class ItsNodeRootKernel(ItsNode): if dtb and os.path.splitext(dtb.name)[1] == ".dtbo": include_kernel = 0 - if self._kernel and include_kernel: + selected_kernel = kernel if kernel is not None else self._kernel + + if selected_kernel and include_kernel: conf_desc.append("Linux kernel") - opt_props["kernel"] = self._kernel.name + opt_props["kernel"] = selected_kernel.name if self._sign_enable: sign_entries.append("kernel")
Signature · 1 line
2.56.0
PATCH v4 2/6 oe.fitimage: add metadata for multiple kernels
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
--- meta/conf/image-fitimage.conf | 55 +++++++++++++++++++++++++++++++++++ 1 file changed, 55 insertions(+)
meta/conf/image-fitimage.conf +55 -0
@@ -102,3 +102,58 @@ FIT_UBOOT_ENV ?= "" # arch, os) if the corresponding flag is not set, while others (load address # and entry point) will be omitted in the final FIT. FIT_LOADABLES ?= "" + +# Allow user to insert additional kernel images, e.g. a second kernel built by +# another recipe or multiconfig, or a prebuilt kernel. The default kernel +# (kernel-1) from virtual/kernel is always included. Each entry is used as the +# image node name and must not collide with other image nodes, e.g. kernel-1. +# Entries can be configured through variable flags: +# FIT_KERNEL_FILENAME[id] - kernel file, relative to FIT_KERNEL_DEPLOYDIR +# (default: linux.bin, mandatory for prebuilt +# kernels) +# FIT_KERNEL_DEPLOYDIR[id] - directory containing the kernel file +# (default: ${DEPLOY_DIR_IMAGE}) +# FIT_KERNEL_RECIPE[id] - recipe whose do_deploy provides the kernel +# (default: none, the file must already exist). +# Without FIT_KERNEL_MULTICONFIG the kernel file +# must be deployed by this recipe. +# FIT_KERNEL_MULTICONFIG[id] - multiconfig building FIT_KERNEL_RECIPE[id] +# (default recipe: virtual/kernel), requires +# FIT_KERNEL_DEPLOYDIR[id] +# FIT_KERNEL_COMPRESSION[id] - default: for linux.bin deployed by +# FIT_KERNEL_RECIPE or FIT_KERNEL_MULTICONFIG the +# content of linux_comp next to it, else none +# FIT_KERNEL_LOADADDRESS[id] - default: ${UBOOT_LOADADDRESS} +# FIT_KERNEL_ENTRYPOINT[id] - default: ${UBOOT_ENTRYPOINT} +# FIT_KERNEL_TYPE[id] - default: ${UBOOT_MKIMAGE_KERNEL_TYPE} +# FIT_KERNEL_DESCRIPTION[id] - default: "Linux kernel" +# Example, a kernel from the "rt" multiconfig with the same MACHINE: +# FIT_KERNELS = "kernel-rt" +# FIT_KERNEL_MULTICONFIG[kernel-rt] = "rt" +# FIT_KERNEL_DEPLOYDIR[kernel-rt] = "${TOPDIR}/tmp-rt/deploy/images/${MACHINE}" +# Example, a second kernel recipe with KERNEL_PACKAGE_NAME = "kernel-rt": +# FIT_KERNELS = "kernel-rt" +# FIT_KERNEL_RECIPE[kernel-rt] = "linux-yocto-rt" +# FIT_KERNEL_DEPLOYDIR[kernel-rt] = "${DEPLOY_DIR_IMAGE}/kernel-rt" +# Additional kernels are only referenced by configurations from FIT_CONFIGS. +FIT_KERNELS ?= "" + +# Allow user to add configurations in addition to the ones generated per DTB +# (or conf-1 without DTBs). Each entry results in a ${FIT_CONF_PREFIX}<id> +# configuration node which can be configured through variable flags: +# FIT_CONFIG_KERNEL[id] - kernel image node, e.g. one of FIT_KERNELS +# (default: kernel-1) +# FIT_CONFIG_DTB[id] - DTB name as in KERNEL_DEVICETREE or a file name in +# EXTERNAL_KERNEL_DEVICETREE (default: none) +# The ramdisk, u-boot script, setup and loadables are referenced by all +# configurations. +# Example: +# FIT_CONFIGS = "rt" +# FIT_CONFIG_KERNEL[rt] = "kernel-rt" +# FIT_CONFIG_DTB[rt] = "am335x-boneblack.dtb" +FIT_CONFIGS ?= "" + +# Allow user to select the default configuration by name without +# FIT_CONF_PREFIX, e.g. "rt" for "conf-rt". Takes precedence over +# FIT_CONF_DEFAULT_DTB. +FIT_CONF_DEFAULT ?= ""
Signature · 1 line
2.56.0
PATCH v4 3/6 oe.selftest: Add tests for multi-kernel FIT config
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
Add selftests for FIT images containing multiple kernel nodes and configurations.
AI-Generated: Uses GitHub copilot (Claude Opus 5.5)
--- meta/lib/oeqa/selftest/cases/fitimage.py | 163 ++++++++++++++++++++++- 1 file changed, 162 insertions(+), 1 deletion(-)
meta/lib/oeqa/selftest/cases/fitimage.py +162 -1
@@ -461,6 +461,39 @@ class FitImageTestCase(OESelftestTestCase): loadables += var_loadables.split() return loadables + @staticmethod + def _get_flag_values(bb_vars, var, items): + """Return per-item values from BitBake flags or synthetic test values.""" + values = bb_vars.get(var) or {} + + if isinstance(values, dict): + return {item: values.get(item) or "" for item in items} + + # Support synthetic values used by the Python-only test fixtures. + values = values.split("?") + return { + item: values[index].strip() if index < len(values) else "" + for index, item in enumerate(items) + } + + @staticmethod + def _get_extra_configs(bb_vars): + """Return a list of (conf, kernel, dtb image) tuples for FIT_CONFIGS""" + configs = (bb_vars.get('FIT_CONFIGS') or "").split() + kernels = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_KERNEL', configs) + dtbs = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_DTB', configs) + _, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) + extra_configs = [] + for conf in configs: + dtb = dtbs[conf] + if dtb in dtb_symlinks: + # Symlinks of test-dtbs-as-ext point to the DTB without "-alias" + dtb = dtb.replace("-alias", "") + elif dtb: + dtb = FitImageTestCase._apply_dtbvendored(bb_vars, dtb) + extra_configs.append((conf, kernels[conf] or "kernel-1", dtb)) + return extra_configs + def _is_req_dict_in_dict(self, found_dict, req_dict): """ Check if all key-value pairs in the required dictionary are present in the found dictionary.
@@ -672,11 +705,14 @@ class KernelFitImageBase(FitImageTestCase): internal_used = { 'DEPLOY_DIR_IMAGE', 'FIT_CONF_DEFAULT_DTB', + 'FIT_CONF_DEFAULT', 'FIT_CONF_MAPPINGS', 'FIT_CONF_PREFIX', + 'FIT_CONFIGS', 'FIT_DESC', 'FIT_HASH_ALG', 'FIT_KERNEL_COMP_ALG', + 'FIT_KERNELS', 'FIT_LOADABLES', 'FIT_OS', 'FIT_SIGN_ALG',
@@ -837,6 +873,8 @@ class KernelFitImageBase(FitImageTestCase): if loadables: images += loadables + images += (bb_vars.get('FIT_KERNELS') or "").split() + # configuration nodes (one per DTB, symlink, and mappings) configurations = [] if dtb_files:
@@ -886,6 +924,8 @@ class KernelFitImageBase(FitImageTestCase): else: configurations = [bb_vars['FIT_CONF_PREFIX'] + '1'] + configurations += [bb_vars['FIT_CONF_PREFIX'] + conf for conf, _, _ in FitImageTestCase._get_extra_configs(bb_vars)] + # Create a list of paths for all image and configuration nodes req_its_paths = [] for image in images:
@@ -928,8 +968,12 @@ class KernelFitImageBase(FitImageTestCase): if uboot_rd_entrypoint: its_field_check.append("entry = <%s>;" % uboot_rd_entrypoint) + fit_conf_default = bb_vars.get('FIT_CONF_DEFAULT') fit_conf_default_dtb = bb_vars.get('FIT_CONF_DEFAULT_DTB') - if fit_conf_default_dtb: + if fit_conf_default: + fit_conf_prefix = bb_vars.get('FIT_CONF_PREFIX', "conf-") + its_field_check.append('default = "' + fit_conf_prefix + fit_conf_default + '";') + elif fit_conf_default_dtb: fit_conf_prefix = bb_vars.get('FIT_CONF_PREFIX', "conf-") its_field_check.append('default = "' + fit_conf_prefix + self._apply_dtbvendored(bb_vars, fit_conf_default_dtb) + '";')
@@ -953,6 +997,12 @@ class KernelFitImageBase(FitImageTestCase): if initramfs_image and initramfs_image_bundle != "1": its_field_check.append('ramdisk = "ramdisk-1";') + # Additional configurations are emitted after the generated ones + for _, kernel, dtb in FitImageTestCase._get_extra_configs(bb_vars): + its_field_check.append('kernel = "%s";' % kernel) + if dtb: + its_field_check.append('fdt = "fdt-%s";' % dtb) + return its_field_check def _get_req_sigvalues_config(self, bb_vars):
@@ -1064,6 +1114,15 @@ class KernelFitImageBase(FitImageTestCase): if loadables: req_sections[conf_name]['Loadables'] = ",".join(loadables) + # Additional kernels and configurations + for kernel in (bb_vars.get('FIT_KERNELS') or "").split(): + req_sections[kernel] = { "Type": "Kernel Image" } + for conf, kernel, dtb in FitImageTestCase._get_extra_configs(bb_vars): + conf_name = bb_vars['FIT_CONF_PREFIX'] + conf + req_sections[conf_name] = { "Kernel": kernel } + if dtb: + req_sections[conf_name]['FDT'] = 'fdt-' + dtb + # Add signing related properties if needed if uboot_sign_enable == "1": for section in req_sections:
@@ -1212,6 +1271,32 @@ FIT_LOADABLE_TYPE[loadable2] = "firmware" self._gen_atf_tee_dummy_images(bb_vars) self._test_fitimage(bb_vars) + def test_fit_image_multiple_kernels(self): + """ + Summary: Check FIT_KERNELS, FIT_CONFIGS and FIT_CONF_DEFAULT + Expected: An additional kernel image node and a configuration + referring to it are present and the configuration is + the default one. + Product: oe-core + """ + machine = self._get_machine_or_skip() + config = """ +KERNEL_IMAGETYPE = "Image" +UBOOT_LOADADDRESS = "0x80080000" +UBOOT_ENTRYPOINT = "0x80080000" +# Use the linux.bin of virtual/kernel also as second kernel to avoid building another kernel +FIT_KERNELS = "kernel-2" +FIT_KERNEL_RECIPE[kernel-2] = "virtual/kernel" +FIT_CONFIGS = "second" +FIT_CONFIG_KERNEL[second] = "kernel-2" +FIT_CONF_DEFAULT = "second" +""" + config = FitImageTestCase._config_add_machine_settings(config, machine, keys=["KERNEL_DEVICETREE"]) + config = self._config_add_kernel_classes(config) + self.write_config(config) + bb_vars = self._fit_get_bb_vars(flags={'FIT_CONFIG_KERNEL': ['second'], 'FIT_CONFIG_DTB': ['second'],}) + self._test_fitimage(bb_vars) + def test_get_compatible_from_dtb(self): """Test the oe.fitimage.get_compatible_from_dtb function
@@ -1505,11 +1590,14 @@ class FitImagePyTests(KernelFitImageBase): # image-fitimage.conf 'FIT_ADDRESS_CELLS': "1", 'FIT_CONF_DEFAULT_DTB': "", + 'FIT_CONF_DEFAULT': "", 'FIT_CONF_MAPPINGS': "", 'FIT_CONF_PREFIX': "conf-", + 'FIT_CONFIGS': "", 'FIT_DESC': "Kernel fitImage for a dummy distro", 'FIT_GENERATE_KEYS': "0", 'FIT_HASH_ALG': "sha256", + 'FIT_KERNELS': "", 'FIT_KEY_GENRSA_ARGS': "-F4", 'FIT_KEY_REQ_ARGS': "-batch -new", 'FIT_KEY_SIGN_PKCS': "-x509",
@@ -1570,6 +1658,11 @@ class FitImagePyTests(KernelFitImageBase): bb_vars.get('UBOOT_MKIMAGE_KERNEL_TYPE'), bb_vars.get("UBOOT_ENTRYSYMBOL") ) + for kernel in bb_vars['FIT_KERNELS'].split(): + root_node.fitimage_emit_section_kernel(kernel, "linux-%s.bin" % kernel, "none", + bb_vars.get('UBOOT_LOADADDRESS'), bb_vars.get('UBOOT_ENTRYPOINT'), + bb_vars.get('UBOOT_MKIMAGE_KERNEL_TYPE')) + dtb_files, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) for dtb in dtb_files: root_node.fitimage_emit_section_dtb(dtb, os.path.join("a-dir", dtb),
@@ -1600,9 +1693,19 @@ class FitImagePyTests(KernelFitImageBase): "loadable-type") root_node.fitimage_emit_section_config(bb_vars['FIT_CONF_DEFAULT_DTB'], bb_vars.get('FIT_CONF_MAPPINGS')) + configs = bb_vars['FIT_CONFIGS'].split() + conf_kernels = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_KERNEL', configs) + conf_dtbs = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_DTB', configs) + for conf in configs: + root_node.fitimage_emit_section_config_custom(conf, conf_kernels[conf], conf_dtbs[conf]) + if bb_vars['FIT_CONF_DEFAULT']: + root_node.fitimage_set_default_config(bb_vars['FIT_CONF_DEFAULT']) root_node.write_its_file(fitimage_its_path) self.assertExists(fitimage_its_path, "%s image tree source doesn't exist" % (fitimage_its_path)) + with open(fitimage_its_path) as its_file: + default_lines = [line for line in its_file if line.strip().startswith("default = ")] + self.assertEqual(len(default_lines), 1, "Expected exactly one default configuration: %s" % default_lines) self.logger.debug("Checking its: %s" % fitimage_its_path) self._check_its_file(bb_vars, fitimage_its_path)
@@ -1669,6 +1772,64 @@ class FitImagePyTests(KernelFitImageBase): } self._test_fitimage_py(bb_vars_overrides) + def test_fitimage_py_multiple_kernels(self): + """Test FIT_KERNELS, FIT_CONFIGS and FIT_CONF_DEFAULT""" + bb_vars_overrides = { + 'KERNEL_DEVICETREE': "one.dtb two.dtb", + 'FIT_CONF_DEFAULT_DTB': "one.dtb", + 'FIT_KERNELS': "kernel-rt", + 'FIT_CONFIGS': "rt rt-two default-two", + 'FIT_CONFIG_KERNEL': "kernel-rt ? kernel-rt ? ", + 'FIT_CONFIG_DTB': " ? two.dtb ? two.dtb", + 'FIT_CONF_DEFAULT': "rt-two", + } + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_multiple_kernels_unknown_kernel_error(self): + """Test that FIT_CONFIG_KERNEL referring to an unknown kernel causes a fatal error""" + bb_vars_overrides = { + 'FIT_CONFIGS': "rt", + 'FIT_CONFIG_KERNEL': "kernel-rt", + } + with self.assertRaises(BBHandledException): + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_conf_default_unknown_error(self): + """Test that FIT_CONF_DEFAULT referring to an unknown configuration causes a fatal error""" + bb_vars_overrides = { + 'FIT_CONF_DEFAULT': "nonexistent", + } + with self.assertRaises(BBHandledException): + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_multiple_kernels_dtb_alias(self): + """Test FIT_CONFIG_DTB referring to an external DTB symlink""" + bb_vars_overrides = { + 'PREFERRED_PROVIDER_virtual/dtb': "test-dtbs-as-ext", + 'FIT_KERNELS': "kernel-rt", + 'FIT_CONFIGS': "rt", + 'FIT_CONFIG_KERNEL': "kernel-rt", + 'FIT_CONFIG_DTB': "test-ext-alias.dtb", + } + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_multiple_kernels_kernel_1_error(self): + """Test that a FIT_KERNELS entry colliding with kernel-1 causes a fatal error""" + bb_vars_overrides = { + 'FIT_KERNELS': "kernel-1", + } + with self.assertRaises(BBHandledException): + self._test_fitimage_py(bb_vars_overrides) + + def test_fitimage_py_multiple_kernels_image_collision_error(self): + """Test that a FIT_KERNELS entry colliding with another image node causes a fatal error""" + bb_vars_overrides = { + 'FIT_KERNELS': "my-loadable", + 'FIT_LOADABLES': "my-loadable", + } + with self.assertRaises(BBHandledException): + self._test_fitimage_py(bb_vars_overrides) + class UBootFitImageTests(FitImageTestCase): """Test cases for the uboot-sign bbclass"""
Signature · 1 line
2.56.0
PATCH v4 4/6 oe.fitimage: Add custom configurations and harden multi-kernel handling
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
One can now add configs that point to any kernel node and, optionally, a DTB with fitimage_emit_section_config_custom(). The default config still can be chosen by name with fitimage_set_default_config().
The default configuration is now stored and written to the ITS once, in write_its_file(). Before, fitimage_set_default_config() had to overwrite the property that fitimage_emit_section_config() had already set.
DTBs and DTB symlink aliases are now looked up by the id they were added with. Before, the lookup depended on the "fdt-" node name or the alias name.
Reject explicit kernel selection for FIT overlays and validate configuration generation order.
Tested with oe-selftest -r fitimage.FitImagePyTests.
--- meta/classes-recipe/kernel-fit-image.bbclass | 129 +++++++++++++++++-- meta/lib/oe/fitimage.py | 85 ++++++++++-- meta/lib/oeqa/selftest/cases/fitimage.py | 37 ++++++ 3 files changed, 227 insertions(+), 24 deletions(-)
meta/classes-recipe/kernel-fit-image.bbclass +116 -13
@@ -29,19 +29,53 @@ python () { d.appendVarFlag('do_compile', 'depends', ' virtual/dtb:do_populate_sysroot') d.setVar('EXTERNAL_KERNEL_DEVICETREE', "${RECIPE_SYSROOT}/boot/devicetree") - # Parse loadables config - loadables = d.getVar('FIT_LOADABLES') - if loadables: - # Only flags are supported. Add the vars to vardeps so their flags are - # part of the signature. - loadable_vars = ['FIT_LOADABLE_ARCH', 'FIT_LOADABLE_COMPRESSION', - 'FIT_LOADABLE_DESCRIPTION', 'FIT_LOADABLE_ENTRYPOINT', - 'FIT_LOADABLE_FILENAME', 'FIT_LOADABLE_LOADADDRESS', - 'FIT_LOADABLE_OS', 'FIT_LOADABLE_TYPE'] - for v in loadable_vars: - if d.getVar(v): - raise bb.parse.SkipRecipe("You cannot use %s as a variable, you can only set flags." % v) - d.appendVarFlag('do_compile', 'vardeps', ' ' + ' '.join(loadable_vars)) + # Parse loadables, additional kernels and additional configurations + for items_var, flag_vars in ( + ('FIT_LOADABLES', ['FIT_LOADABLE_ARCH', 'FIT_LOADABLE_COMPRESSION', + 'FIT_LOADABLE_DESCRIPTION', 'FIT_LOADABLE_ENTRYPOINT', + 'FIT_LOADABLE_FILENAME', 'FIT_LOADABLE_LOADADDRESS', + 'FIT_LOADABLE_OS', 'FIT_LOADABLE_TYPE']), + ('FIT_KERNELS', ['FIT_KERNEL_COMPRESSION', 'FIT_KERNEL_DEPLOYDIR', + 'FIT_KERNEL_DESCRIPTION', 'FIT_KERNEL_ENTRYPOINT', + 'FIT_KERNEL_FILENAME', 'FIT_KERNEL_LOADADDRESS', + 'FIT_KERNEL_MULTICONFIG', 'FIT_KERNEL_RECIPE', + 'FIT_KERNEL_TYPE']), + ('FIT_CONFIGS', ['FIT_CONFIG_DTB', 'FIT_CONFIG_KERNEL'])): + items = d.getVar(items_var) + if items: + # Only variable flags are supported, not direct assignments. + for v in flag_vars: + if d.getVar(v): + raise bb.parse.SkipRecipe( + "You cannot use %s as a variable, you can only set flags." % v + ) + + # Include variable flags in the do_compile task signature. + d.appendVarFlag('do_compile', 'vardeps', ' ' + ' '.join(flag_vars)) + + # Additional kernels may be deployed by another recipe, by a recipe of + # another multiconfig or be provided as prebuilt files + kernels = (d.getVar('FIT_KERNELS') or "").split() + if "kernel-1" in kernels: + bb.fatal("FIT_KERNELS must not contain kernel-1, it is reserved for virtual/kernel") + duplicates = sorted(set(k for k in kernels if kernels.count(k) > 1)) + if duplicates: + bb.fatal("FIT_KERNELS contains duplicate entries: %s" % ' '.join(duplicates)) + for kernel in kernels: + recipe = d.getVarFlag('FIT_KERNEL_RECIPE', kernel) + mc = d.getVarFlag('FIT_KERNEL_MULTICONFIG', kernel) + if mc: + if not d.getVarFlag('FIT_KERNEL_DEPLOYDIR', kernel): + bb.fatal("FIT_KERNEL_DEPLOYDIR[%s] must be set if FIT_KERNEL_MULTICONFIG[%s] is set" % (kernel, kernel)) + d.appendVarFlag('do_compile', 'mcdepends', ' mc:%s:%s:%s:do_deploy' % + (d.getVar('BB_CURRENT_MC'), mc, recipe or 'virtual/kernel')) + elif recipe: + d.appendVarFlag('do_compile', 'depends', ' %s:do_deploy' % recipe) + else: + if not d.getVarFlag('FIT_KERNEL_FILENAME', kernel): + bb.fatal("FIT_KERNEL_FILENAME[%s] must be set if neither FIT_KERNEL_RECIPE[%s] nor FIT_KERNEL_MULTICONFIG[%s] is set" % (kernel, kernel, kernel)) + kernel_path = fit_kernel_path(d, kernel) + d.appendVarFlag('do_compile', 'file-checksums', ' %s:%s' % (kernel_path, os.path.exists(kernel_path))) } do_configure[noexec] = "1"
@@ -61,6 +95,30 @@ def apply_dtbvendored(d,dtb): mapped_name = dtb.replace('/', '_') return mapped_name +def fit_kernel_path(d, kernel): + filename = d.getVarFlag('FIT_KERNEL_FILENAME', kernel) or "linux.bin" + deploydir = d.getVarFlag('FIT_KERNEL_DEPLOYDIR', kernel) or d.getVar('DEPLOY_DIR_IMAGE') + return os.path.join(deploydir, filename) + +def fit_kernel_deployed_files(d, recipe): + """Return the real paths of all files deployed by do_deploy of recipe (PN or PROVIDES) in this multiconfig""" + import bb.runqueue + mc = d.getVar('BB_CURRENT_MC') or "" + taskdepdata = d.getVar('BB_TASKDEPDATA', False) + pns = set(dep.pn for tid, dep in taskdepdata.items() + if dep.taskname == "do_deploy" and bb.runqueue.mc_from_tid(tid) == mc and + (dep.pn == recipe or recipe in dep.provides)) + if len(pns) != 1: + bb.fatal("Could not identify the do_deploy dependency providing %s (found: %s)" % (recipe, ' '.join(sorted(pns)) or "none")) + pn = pns.pop() + for arch in d.getVar('SSTATE_ARCHS').split(): + manifest = os.path.join(d.getVar('SSTATE_MANIFESTS'), "manifest-%s-%s.deploy" % (arch, pn)) + if os.path.exists(manifest): + with open(manifest) as f: + return set(os.path.realpath(line) for line in f.read().splitlines() if line and not line.endswith('/')) + bb.fatal("No deploy manifest found for %s" % pn) +fit_kernel_deployed_files[vardepsexclude] = "BB_TASKDEPDATA SSTATE_ARCHS" + python do_compile() { import shutil import oe.fitimage
@@ -94,6 +152,39 @@ python do_compile() { d.getVar('UBOOT_LOADADDRESS'), d.getVar('UBOOT_ENTRYPOINT'), d.getVar('UBOOT_MKIMAGE_KERNEL_TYPE'), d.getVar("UBOOT_ENTRYSYMBOL")) + # Prepare additional kernel image sections + for kernel in d.getVar('FIT_KERNELS').split(): + recipe = d.getVarFlag('FIT_KERNEL_RECIPE', kernel) + mc = d.getVarFlag('FIT_KERNEL_MULTICONFIG', kernel) + kernel_path = fit_kernel_path(d, kernel) + if not os.path.exists(kernel_path): + bb.fatal("File for kernel %s not found at %s" % (kernel, kernel_path)) + # Artifacts of another multiconfig cannot be verified, its FIT_KERNEL_DEPLOYDIR is trusted + deployed = None + if recipe and not mc: + deployed = fit_kernel_deployed_files(d, recipe) + if os.path.realpath(kernel_path) not in deployed: + bb.fatal("File for kernel %s at %s is not deployed by %s" % (kernel, kernel_path, recipe)) + kernel_file = "linux-%s.bin" % kernel + shutil.copyfile(kernel_path, kernel_file) + + kernel_comp = d.getVarFlag('FIT_KERNEL_COMPRESSION', kernel) + if not kernel_comp: + kernel_comp = "none" + # linux_comp only describes linux.bin written by kernel-fit-extra-artifacts of the same recipe + if (recipe or mc) and os.path.basename(kernel_path) == "linux.bin": + kernel_comp_path = os.path.join(os.path.dirname(kernel_path), "linux_comp") + if not os.path.exists(kernel_comp_path) or \ + (deployed is not None and os.path.realpath(kernel_comp_path) not in deployed): + bb.fatal("%s not found or not deployed by the kernel recipe, set FIT_KERNEL_COMPRESSION[%s]" % (kernel_comp_path, kernel)) + with open(kernel_comp_path) as kernel_comp_f: + kernel_comp = kernel_comp_f.read().strip() + root_node.fitimage_emit_section_kernel(kernel, kernel_file, kernel_comp, + d.getVarFlag('FIT_KERNEL_LOADADDRESS', kernel) or d.getVar('UBOOT_LOADADDRESS'), + d.getVarFlag('FIT_KERNEL_ENTRYPOINT', kernel) or d.getVar('UBOOT_ENTRYPOINT'), + d.getVarFlag('FIT_KERNEL_TYPE', kernel) or d.getVar('UBOOT_MKIMAGE_KERNEL_TYPE'), + description=d.getVarFlag('FIT_KERNEL_DESCRIPTION', kernel) or "Linux kernel") + # Prepare a DTB image section kernel_devicetree = d.getVar('KERNEL_DEVICETREE') external_kernel_devicetree = d.getVar("EXTERNAL_KERNEL_DEVICETREE")
@@ -209,6 +300,18 @@ python do_compile() { # Generate the configuration section root_node.fitimage_emit_section_config(default_dtb, d.getVar("FIT_CONF_MAPPINGS")) + # Generate additional configurations, e.g. referring to additional kernels + for conf in d.getVar('FIT_CONFIGS').split(): + conf_dtb = d.getVarFlag('FIT_CONFIG_DTB', conf) + if conf_dtb: + conf_dtb = apply_dtbvendored(d, conf_dtb) + root_node.fitimage_emit_section_config_custom(conf, + d.getVarFlag('FIT_CONFIG_KERNEL', conf), conf_dtb) + + fit_conf_default = d.getVar('FIT_CONF_DEFAULT') + if fit_conf_default: + root_node.fitimage_set_default_config(fit_conf_default) + # Write the its file root_node.write_its_file(itsfile)
meta/lib/oe/fitimage.py +74 -11
173 lines not on this page · raw message
meta/lib/oeqa/selftest/cases/fitimage.py +37 -0
@@ -1580,6 +1580,43 @@ KERNEL_DTBVENDORED = "1" class FitImagePyTests(KernelFitImageBase): """Test cases for the fitimage.py module without calling bitbake""" + def test_custom_config_rejects_kernel_with_overlay(self): + """An overlay configuration must not silently ignore an explicit kernel.""" + root = oe.fitimage.ItsNodeRootKernel( + "Test FIT", "1", "x86_64-poky-linux-", "x86", + "linux", "conf-", False, None, + "uboot-mkimage", "", "uboot-mkimage", "", + "sha256", "rsa2048", "pkcs-1.5", + None, False, None + ) + + root.fitimage_emit_section_kernel( + "kernel-1", "linux.bin", "none", + "0x20008000", "0x20008000", "kernel" + ) + + root.fitimage_emit_section_dtb( + "test-overlay.dtbo", "a-dir/test-overlay.dtbo" + ) + + with self.assertRaises(bb.BBHandledException): + root.fitimage_emit_section_config_custom( + "invalid", "kernel-1", "test-overlay.dtbo" + ) + + # A regular DTB may explicitly select a kernel. + root.fitimage_emit_section_dtb( + "test-board.dtb", "a-dir/test-board.dtb" + ) + + root.fitimage_emit_section_config_custom( + "regular", "kernel-1", "test-board.dtb" + ) + + # An overlay without an explicit kernel is valid. + root.fitimage_emit_section_config_custom( + "overlay", None, "test-overlay.dtbo" + ) def _test_fitimage_py(self, bb_vars_overrides=None): topdir = os.path.join(os.environ['BUILDDIR'])
Signature · 1 line
2.56.0
PATCH v4 5/6 oe.selftest: Improve FIT DTB alias validation
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
Use explicit DTB alias targets instead of relying on FIT node naming conventions.
Discover external DTBs and symlink aliases when generating expected FIT configurations, including vendored DTB names.
--- meta/lib/oeqa/selftest/cases/fitimage.py | 80 ++++++++++++++++-------- 1 file changed, 55 insertions(+), 25 deletions(-)
meta/lib/oeqa/selftest/cases/fitimage.py +55 -25
@@ -432,6 +432,30 @@ class FitImageTestCase(OESelftestTestCase): mapped_name = dtb.replace('/', '_') return mapped_name + @staticmethod + def _get_test_dtb_files(): + return ( + ["test-ext.dtb", "test-overlay.dtbo"], + {"test-ext-alias.dtb": "test-ext.dtb"}, + ) + + @staticmethod + def _get_external_dtb_files(devicetree_dir): + dtbs = [] + aliases = {} + + for name in sorted(os.listdir(devicetree_dir)): + if not name.endswith((".dtb", ".dtbo")): + continue + + target = oe.fitimage.symlink_points_below(name, devicetree_dir) + if target: + aliases[name] = target + else: + dtbs.append(name) + + return dtbs, aliases + @staticmethod def _get_dtb_files(bb_vars): """Return a list of devicetree names
@@ -442,15 +466,22 @@ class FitImageTestCase(OESelftestTestCase): """ kernel_devicetree = bb_vars.get('KERNEL_DEVICETREE') all_dtbs = [] - dtb_symlinks = [] + dtb_aliases = {} + if kernel_devicetree: all_dtbs += [FitImageTestCase._apply_dtbvendored(bb_vars, dtb) for dtb in kernel_devicetree.split()] - # Support only the test recipe which provides 1 devicetree and 1 devicetree overlay - pref_prov_dtb = bb_vars.get('PREFERRED_PROVIDER_virtual/dtb') - if pref_prov_dtb == "test-dtbs-as-ext": - all_dtbs += ["test-ext.dtb", "test-overlay.dtbo"] - dtb_symlinks.append("test-ext-alias.dtb") - return (all_dtbs, dtb_symlinks) + + if bb_vars.get("PREFERRED_PROVIDER_virtual/dtb") == "test-dtbs-as-ext": + if bb_vars.get("_python_only_test"): + external_dtbs, external_aliases = FitImageTestCase._get_test_dtb_files() + else: + devicetree_dir = os.path.join(bb_vars["DEPLOY_DIR_IMAGE"], "devicetree") + external_dtbs, external_aliases = FitImageTestCase._get_external_dtb_files(devicetree_dir) + + all_dtbs.extend(external_dtbs) + dtb_aliases.update(external_aliases) + + return all_dtbs, dtb_aliases @staticmethod def _get_loadables(bb_vars):
@@ -482,13 +513,12 @@ class FitImageTestCase(OESelftestTestCase): configs = (bb_vars.get('FIT_CONFIGS') or "").split() kernels = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_KERNEL', configs) dtbs = FitImageTestCase._get_flag_values(bb_vars, 'FIT_CONFIG_DTB', configs) - _, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) + _, dtb_aliases = FitImageTestCase._get_dtb_files(bb_vars) extra_configs = [] for conf in configs: dtb = dtbs[conf] - if dtb in dtb_symlinks: - # Symlinks of test-dtbs-as-ext point to the DTB without "-alias" - dtb = dtb.replace("-alias", "") + if dtb in dtb_aliases: + dtb = dtb_aliases[dtb] elif dtb: dtb = FitImageTestCase._apply_dtbvendored(bb_vars, dtb) extra_configs.append((conf, kernels[conf] or "kernel-1", dtb))
@@ -985,10 +1015,10 @@ class KernelFitImageBase(FitImageTestCase): if not dtb.endswith('.dtbo'): its_field_check.append('kernel = "kernel-1";') its_field_check.append('fdt = "fdt-%s";' % dtb) - for dtb in dtb_symlinks: + + for dtb, target in dtb_symlinks.items(): its_field_check.append('kernel = "kernel-1";') - # Works only for tests were the symlink is with -alias suffix - its_field_check.append('fdt = "fdt-%s";' % dtb.replace('-alias', '')) + its_field_check.append('fdt = "fdt-%s";' % target) if initramfs_image and initramfs_image_bundle != "1": its_field_check.append('ramdisk = "ramdisk-1";')
@@ -1085,11 +1115,10 @@ class KernelFitImageBase(FitImageTestCase): req_sections[loadable]['Entry Point'] = entrypoint # Create a configuration section for each DTB if dtb_files: - for dtb in dtb_files + dtb_symlinks: + for dtb in dtb_files + list(dtb_symlinks): conf_name = bb_vars['FIT_CONF_PREFIX'] + dtb - # Assume that DTBs with an "-alias" in its name are symlink DTBs created e.g. by the - # test-dtbs-as-ext test recipe. Make the configuration node pointing to the real DTB. - real_dtb = dtb.replace("-alias", "") + # Resolve DTB aliases using their explicit target mapping. + real_dtb = dtb_symlinks.get(dtb, dtb) # dtb overlays do not refer to a kernel (yet?) if dtb.endswith('.dtbo'): req_sections[conf_name] = {
@@ -1155,7 +1184,7 @@ class KernelFitImageBase(FitImageTestCase): fit_hash_alg_len = FitImageTestCase.MKIMAGE_HASH_LENGTHS[fit_hash_alg] fit_sign_alg_len = FitImageTestCase.MKIMAGE_SIGNATURE_LENGTHS[fit_sign_alg] dtb_files, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) - all_dtb_names = set(dtb_files + dtb_symlinks) + all_dtb_names = set(dtb_files) | set(dtb_symlinks) # The public key is always injected into UBOOT_DTB_IMAGE (-K flag in # uboot-sign.bbclass concat_dtb). For DTB-based configurations, the # same key is also injected into each per-configuration DTB, so either
@@ -1621,7 +1650,7 @@ class FitImagePyTests(KernelFitImageBase): def _test_fitimage_py(self, bb_vars_overrides=None): topdir = os.path.join(os.environ['BUILDDIR']) fitimage_its_path = os.path.join(topdir, self._testMethodName + '.its') - + # Provide variables without calling bitbake bb_vars = { # image-fitimage.conf
@@ -1671,7 +1700,10 @@ class FitImagePyTests(KernelFitImageBase): # x86 kernels produce a setup.bin section; set to the file name to # enable it, or to "" / omit the key to suppress it. 'KERNEL_SETUP_BIN': 'setup1.bin', + # ... existing entries ... + '_python_only_test': True, } + if bb_vars_overrides: bb_vars.update(bb_vars_overrides) if logging.DEBUG >= self.logger.level:
@@ -1700,15 +1732,12 @@ class FitImagePyTests(KernelFitImageBase): bb_vars.get('UBOOT_LOADADDRESS'), bb_vars.get('UBOOT_ENTRYPOINT'), bb_vars.get('UBOOT_MKIMAGE_KERNEL_TYPE')) - dtb_files, dtb_symlinks = FitImageTestCase._get_dtb_files(bb_vars) + dtb_files, dtb_aliases = FitImageTestCase._get_dtb_files(bb_vars) for dtb in dtb_files: root_node.fitimage_emit_section_dtb(dtb, os.path.join("a-dir", dtb), bb_vars.get("UBOOT_DTB_LOADADDRESS"), bb_vars.get("UBOOT_DTBO_LOADADDRESS")) - for dtb_symlink in dtb_symlinks: - # For test purposes, assume each symlink points to a DTB with the same basename minus "-alias" - # In this case, "test-ext-alias.dtb" -> "test-ext.dtb" - dtb_target = dtb_symlink.replace("-alias", "") + for dtb_symlink, dtb_target in dtb_aliases.items(): root_node.fitimage_emit_section_dtb_alias(dtb_symlink, os.path.join("a-dir", dtb_target)) if bb_vars.get('FIT_UBOOT_ENV'):
@@ -1921,6 +1950,7 @@ class UBootFitImageTests(FitImageTestCase): 'UBOOT_SIGN_IMG_KEYNAME', 'UBOOT_SIGN_KEYDIR', 'UBOOT_SIGN_KEYNAME', + 'EXTERNAL_KERNEL_DEVICETREE', } bb_vars = get_bb_vars(list(internal_used | set(additional_vars)), UBootFitImageTests.BOOTLOADER_RECIPE) self.logger.debug("bb_vars: %s" % pprint.pformat(bb_vars, indent=4))
Signature · 1 line
2.56.0
PATCH v4 6/6 kernel-fit-image: preserve vendored DTB aliases for external overrides
In reply to Vahid GharaeeFrom: Vahid Gharaee <[email protected]>
When an external DTB overrides a kernel-provided DTB, the FIT generation skips the kernel DTB. With KERNEL_DTBVENDORED enabled, this also drops the vendored configuration name.
Preserve the vendored name by creating a configuration alias referencing the external DTB image. Resolve external symlinks to their actual DTB targets.
Add regression tests for regular and symlinked external DTB overrides.
--- .../recipes-test/ext-dtb/test-dtbs-as-ext.bb | 11 ++ meta/classes-recipe/kernel-fit-image.bbclass | 7 ++ meta/lib/oeqa/selftest/cases/fitimage.py | 105 ++++++++++++++++-- 3 files changed, 113 insertions(+), 10 deletions(-)
meta-selftest/recipes-test/ext-dtb/test-dtbs-as-ext.bb +11 -0
@@ -8,6 +8,7 @@ LIC_FILES_CHKSUM = "file://${COMMON_LICENSE_DIR}/MIT;md5=0835ade698e0bcf8506ecda inherit devicetree +TEST_EXTRA_DTB ?= "" COMPATIBLE_MACHINE = ".*" SRC_URI = "\
@@ -18,8 +19,18 @@ SRC_URI = "\ # Sym-links are handled as extra configuration nodes in FIT images. do_install:append() { ln -sf test-ext.dtb "${D}/boot/devicetree/test-ext-alias.dtb" + + if [ -n "${TEST_EXTRA_DTB}" ]; then + cp "${D}/boot/devicetree/test-ext.dtb" \ + "${D}/boot/devicetree/${TEST_EXTRA_DTB}" + fi } do_deploy:append() { ln -sf test-ext.dtb "${DEPLOYDIR}/devicetree/test-ext-alias.dtb" + + if [ -n "${TEST_EXTRA_DTB}" ]; then + cp "${DEPLOYDIR}/devicetree/test-ext.dtb" \ + "${DEPLOYDIR}/devicetree/${TEST_EXTRA_DTB}" + fi }
meta/classes-recipe/kernel-fit-image.bbclass +7 -0
@@ -188,6 +188,7 @@ python do_compile() { # Prepare a DTB image section kernel_devicetree = d.getVar('KERNEL_DEVICETREE') external_kernel_devicetree = d.getVar("EXTERNAL_KERNEL_DEVICETREE") + external_dtb_aliases = {} if kernel_devicetree: for dtb in kernel_devicetree.split(): # In deploy_dir the DTBs are without sub-directories also with KERNEL_DTBVENDORED = "1"
@@ -201,6 +202,8 @@ python do_compile() { if external_kernel_devicetree: ext_dtb_path = os.path.join(external_kernel_devicetree, dtb_name) if os.path.exists(ext_dtb_path) and os.path.getsize(ext_dtb_path) > 0: + if conf_name != dtb_name: + external_dtb_aliases[conf_name] = ext_dtb_path continue # Copy the dtb or dtbo file into the FIT image assembly directory
@@ -225,6 +228,10 @@ python do_compile() { root_node.fitimage_emit_section_dtb(dtb_name, dtb_name, d.getVar("UBOOT_DTB_LOADADDRESS"), d.getVar("UBOOT_DTBO_LOADADDRESS"), True) + for alias, dtb_path in external_dtb_aliases.items(): + symlink_target = oe.fitimage.symlink_points_below(os.path.basename(dtb_path), external_kernel_devicetree) + root_node.fitimage_emit_section_dtb_alias(alias, symlink_target or dtb_path) + # Prepare a u-boot script section fit_uboot_env = d.getVar("FIT_UBOOT_ENV") if fit_uboot_env:
meta/lib/oeqa/selftest/cases/fitimage.py +95 -10
157 lines not on this page · raw message
Signature · 1 line
2.56.0