Add a new bbclass that abstracts the generation of FIT blobs
PATCH 1/1 Add a new bbclass that abstracts the generation of FIT blobs
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
---
Notes:
Testing
-------
1. linux-yocto_5.4.bbappend was modified to have the following configuration:
```
inherit fit-image
KERNEL_IMAGE_NODE[name] = "kernel" KERNEL_IMAGE_NODE[description] = "${PF}" KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' KERNEL_IMAGE_NODE[type] = "kernel" KERNEL_IMAGE_NODE[arch] = "${ARCH}" KERNEL_IMAGE_NODE[os] = "linux" KERNEL_IMAGE_NODE[compression] = "none" KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" KERNEL_IMAGE_NODE[hash] = "sha256"
FDT_IMAGE_NODE[name] = "fdt"
FDT_IMAGE_NODE[description] = "FDT blob"
FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' FDT_IMAGE_NODE[type] = "flat_dt"
FDT_IMAGE_NODE[arch] = "${ARCH}"
FDT_IMAGE_NODE[compression] = "none"
FDT_IMAGE_NODE[hash] = "sha256"
CONF1_CONF_NODE[name] = "conf" CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" CONF1_CONF_NODE[kernel] = "kernel" CONF1_CONF_NODE[fdt] = "fdt"
FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" ```
2. Build the kernel: bitbake virtual/kernel
3. Verify that image-fit.itb is present in the build directory: PASS
4. Disassemble the image using the command: dtc -I dtb -O dts image-fit.itb
5. Verify that the FIT source contains the expected configuration: PASS
meta/classes/fit-image.bbclass | 377 +++++++++++++++++++++++++++++++++ 1 file changed, 377 insertions(+) create mode 100644 meta/classes/fit-image.bbclass
meta/classes/fit-image.bbclass new file +377 -0
@@ -0,0 +1,377 @@ +# +# The class will facilitate the generation of FIT blobs. +# +# Glossary +# FIT - Flattened uImage Tree +# +# Requirements: +# +# * The user need to specify the image content using the format specified in the "FIT Image API" section. +# +# FIT Image API +# +# The bbclass is using variable and variable flags to declare the FIT image content. +# +# * Sub-Images and Configuration Nodes +# +# ** A sub-image node content is declared using the format `VAR_NODE[<property-name>] = <value>`. +# * VAR_NODE - freely selected name of the variable representing a node. +# * <property-name> - a sub-image property (e.g. description, type...). +# * <value> - the property value. +# Depending of the property the value can support different formats. +# ** Property Values Formats +# +# string property +# --------------- +# format: "<text>" - in case the property expects a text. +# (e.g. IMAGE_NODE_KERNEL[type] = "kernel") +# +# address property +# ---------------- +# format: "<address>" - in case the property expects an address. +# (e.g. IMAGE_NODE_KERNEL[entry] = "0xABCDABCD") +# +# hash property +# ------------- +# format: "<hash type>" - for hash property the hash type needs to be specified. +# (e.g. IMAGE_NODE_KERNEL[hash] = "sha256") +# +# sub-image signature property +# ---------------------------- +# format: "<algo>;<key-name-hint>;" - for image signature node. +# Both algorithm and key name needs to be provided. +# (e.g. IMAGE_NODE_KERNEL[signature] = "sha256,rsa2048;kernel;" +# +# configuration signature property +# -------------------------------- +# format: "<algo>;<key-name-hint>;<sub-image list>" - for configuration signature properties algorithm, +# key name and sub-image nodes needs to be provided. +# (e.g. CONF_NODE_CONF1[signature] = "sha256,rsa2048;kernel;"kernel","fdt";") +# +# ** Sub-Image and Configuration Nodes Flags +# See the code for supported flags. +# +# * FIT_IMAGES_NODE - contains a list of variables used to declare the sub-images nodes, separated by space. +# (e.g. FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL IMAGE_NODE_FDT") +# +# * FIT_CONFIGURATIONS_NODE - contains a list of variables used to declare the configuration nodes, +# separated by space. (e.g. FIT_CONFIGURATIONS_NODE = "CONF_NODE_CONF1") +# ** Flags +# - "default": used to configure the default configuration node. +# (e.g. FIT_CONFIGURATIONS_NODE[default] = "conf@0") +# +# Example: +# This is part of a linux_%.bbappend recipe. +# +# IMAGE_NODE_KERNEL[name] = "kernel@0" +# IMAGE_NODE_KERNEL[description] = "${PF}" +# IMAGE_NODE_KERNEL[data] = "/incbin/("./arch/${ARCH}/boot/zImage")" +# IMAGE_NODE_KERNEL[type] = "kernel" +# IMAGE_NODE_KERNEL[arch] = "${ARCH}" +# IMAGE_NODE_KERNEL[os] = "linux" +# IMAGE_NODE_KERNEL[compression] = "none" +# IMAGE_NODE_KERNEL[load] = "${UBOOT_LOADADDRESS}" +# IMAGE_NODE_KERNEL[entry] = "${UBOOT_ENTRYPOINT}" +# IMAGE_NODE_KERNEL[hash] = "sha256" +# +# IMAGE_NODE_FDT[name] = "fdt@0" +# IMAGE_NODE_FDT[type] = "flat_dt" +# IMAGE_NODE_FDT[arch] = "${ARCH}" +# IMAGE_NODE_FDT[compression] = "none" +# IMAGE_NODE_FDT[hash] = "sha256" +# +# CONF_NODE_CONF1[name] = "conf@0" +# CONF_NODE_CONF1[description] = "Linux kernel and FDT blob" +# CONF_NODE_CONF1[kernel] = "kernel@0" +# CONF_NODE_CONF1[fdt] = "fdt@0" +# +# FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL IMAGE_NODE_FDT" +# +# FIT_CONFIGURATIONS_NODE = "CONF_NODE_CONF1" +# FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF_NODE_CONF1', 'name') or ""}" +# + +DEPENDS += "\ + u-boot-mkimage-native \ +" + +FIT_IMAGE_DESCRIPTION ??= "Generic FIT image" +FIT_IMAGE_FILENAME ??= "image-fit" +FIT_IMAGE_UBOOT_MKIMAGE_OPTS ??= "" + + +def get_subimage_node_rules(): + """ + Defines the properties format and validation for sub-image nodes. + + :return: Return a dictionary with the format rules. + """ + + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['data'] = [True, '= $value', ["value"]] + rules['type'] = [True, '= "$value"', ["value"]] + rules['arch'] = [False, '= "$value"', ["value"]] + rules['os'] = [False, '= "$value"', ["value"]] + rules['compression'] = [True, '= "$value"', ["value"]] + rules['load'] = [False, '= <$value>', ["value"]] + rules['entry'] = [False, '= <$value>', ["value"]] + rules['hash'] = [False, '{ algo = "$algo"; }', ['algo']] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; }', ['algo', 'key']] + + return rules + + +def get_conf_node_rules(): + """ + Defines the properties format and validation for configuration nodes. + + :return: Return a dictionary with the format rules. + """ + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['kernel'] = [True, '= "$value"', ["value"]] + rules['ramdisk'] = [False, '= "$value"', ["value"]] + rules['fdt'] = [False, '= "$value"', ["value"]] + rules['loadables'] = [False, '= "$value"', ["value"]] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; sign-images = $images; }', + ['algo', 'key', 'images']] + + return rules + + +def generate_node(name, params, rules): + """ + Generates a node. + + :param name: Node name. + :param params: A dictionary containing the properties values. + :param rules: A dictionary containing the properties values validation and format. + + :return: A string containing the node, including the new line characters. + """ + from string import Template + + content = [] + + for rule in rules.keys(): + if rule in params.keys(): + content.append('{param} {value}; '.format( + param=rule, + value=Template(rules[rule][1]).substitute( + dict(zip(rules[rule][2], params[rule].split(';')))))) + elif rules[rule][0]: + bb.fatal('Missing mandatory parameter: {param} from {name} section'.format(param=rule, section=name)) + + content = """ + """.join(content) + node = """ {name} {{ + {content} + }}; + """.format(name=name, content=content) + + return node + + +def get_section_configuration(var, d): + """ + Generates a string build from variable's flags. + + :param var: variable to extract the flags. + :param d: bitbake environment. + + :return: A string with the format '<flag1> = <value1>; <flag2> = <value2>;...'. + """ + flags = d.getVarFlags(var) + if flags is not None: + flags = dict((flag, d.expand(value)) + for flag, value in list(flags.items())) + else: + flags = {} + + configuration = ''.join(( + """{name} = "{value}"; + """.format(name=name, value=value) for name, value in flags.items())) + + return configuration + + +def get_section_properties(var, d): + """ + Extract the nodes and parameters for a section. + + :param var: variable containing the variable names of the nodes that are part of this section. + :param d: bitbake environment. + + :return: a list containing dictionaries with section nodes parameters. + """ + nodes = [] + parameters = {} + + for node in d.getVar(var).split(): + parameters = d.getVarFlags(node) + if parameters is not None: + parameters = dict((parameter, d.expand(value)) + for parameter, value in list(parameters.items())) + nodes.append(parameters) + + return nodes + + +def generate_section(var, rules, d): + """ + Generates a section node (configuration or sub-image). + + :param var: Variable to extract the node names. + :param rules: Rules to use for generating this section. + :param d: bitbake environment. + + :return: A string containing the section, including the new line characters. + """ + + section = get_section_configuration(var, d) + + nodes_parameters = get_section_properties(var, d) + for parameters in nodes_parameters: + name = parameters.pop('name') + node = generate_node(name, parameters, rules) + section += node + + return section + + +def get_fit_image_template(): + """ + Get the FIT format. + + :return: A Template string containing the FIT image format. + """ + from string import Template + + template = Template("""/dts-v1/; + /{ + description = "$description"; + #address-cells = <1>; + images { + $images_section + }; + configurations { + $configurations_section + }; + };""") + return template + + +def generate_image_tree_source(d): + """ + Generates a string containing the image tree source. + + :return: A string representing the image tree. + """ + from string import Template + + values = {} + values['description'] = d.getVar('FIT_IMAGE_DESCRIPTION') + + image_rules = get_subimage_node_rules() + values['images_section'] = generate_section('FIT_IMAGES_NODE', image_rules, d) + + conf_rules = get_conf_node_rules() + values['configurations_section'] = generate_section('FIT_CONFIGURATIONS_NODE', conf_rules, d) + + image_tree_source = get_fit_image_template().substitute(values) + + return image_tree_source + + +def generate_image_blob(file_name, image, d): + """ + Generates a FIT blob. + + :param file_name: FIT blob file name. + :param image: String containing the image tree source. + :param d: Bitbake environment. + """ + import tempfile + import subprocess + + bb.debug(1, "Generated FIT source is:\n {image}".format(image=image)) + + builddir = d.getVar('B') + blob_file_name = file_name + '.itb' + + try: + fd, faux = tempfile.mkstemp(dir=builddir, prefix=file_name, suffix=".its") + with os.fdopen(fd, "w") as f: + f.write(image) + + mkimage_opts = d.getVar('FIT_IMAGE_UBOOT_MKIMAGE_OPTS').split() or "" + cmd = ['mkimage', '-f', faux] + cmd.extend(mkimage_opts) + cmd.append('{output_name}'.format(output_name=blob_file_name)) + + ret = subprocess.run( + cmd, + check=True, + universal_newlines=True, + cwd=builddir, + stderr=subprocess.STDOUT, + stdout=subprocess.PIPE) + + bb.debug(1, "Command for generating the FIT blob is:\n {cmd}".format(cmd=" ".join(ret.args))) + + except subprocess.CalledProcessError as e: + bb.fatal('Failed to generate the FIT blob: {message}: {output}'.format( + message=str(e), output=e.stdout)) + finally: + os.remove(faux) + + +def generate_fit_image(file_name, d): + """ + Create and generate FIT blob. + + :param file_name: FIT blob file name. + :param d: Bitbake environment. + """ + image = generate_image_tree_source(d) + generate_image_blob(file_name, image, d) + + +python fit-image_do_generate_fit_image() { + generate_fit_image(d.getVar('FIT_IMAGE_FILENAME'), d) +} + +do_generate_fit_image[vardeps] += " \ + ${FIT_CONFIGURATIONS_NODE} \ + ${FIT_IMAGES_NODE} \ + FIT_CONFIGURATIONS_NODE \ + FIT_IMAGES_NODE \ + FIT_IMAGE_FILENAME \ +" + +addtask do_generate_fit_image after do_compile before do_deploy + +EXPORT_FUNCTIONS do_generate_fit_image
Signature · 1 line
2.24.1
PATCH v2 1/1 classes: Add a new bbclass that abstracts the generation of FIT blobs
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
---
Notes:
Testing
-------
1. linux-yocto_5.4.bbappend was modified to have the following configuration:
```
inherit fit-image
KERNEL_IMAGE_NODE[name] = "kernel" KERNEL_IMAGE_NODE[description] = "${PF}" KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' KERNEL_IMAGE_NODE[type] = "kernel" KERNEL_IMAGE_NODE[arch] = "${ARCH}" KERNEL_IMAGE_NODE[os] = "linux" KERNEL_IMAGE_NODE[compression] = "none" KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" KERNEL_IMAGE_NODE[hash] = "sha256"
FDT_IMAGE_NODE[name] = "fdt"
FDT_IMAGE_NODE[description] = "FDT blob"
FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' FDT_IMAGE_NODE[type] = "flat_dt"
FDT_IMAGE_NODE[arch] = "${ARCH}"
FDT_IMAGE_NODE[compression] = "none"
FDT_IMAGE_NODE[hash] = "sha256"
CONF1_CONF_NODE[name] = "conf" CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" CONF1_CONF_NODE[kernel] = "kernel" CONF1_CONF_NODE[fdt] = "fdt"
FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" ```
2. Build the kernel: bitbake virtual/kernel
3. Verify that image-fit.itb is present in the build directory: PASS
4. Disassemble the image using the command: dtc -I dtb -O dts image-fit.itb
5. Verify that the FIT source contains the expected configuration: PASS
Changes since v1:
----------------
- Change the format of short-log to "<target>: <summary>"
meta/classes/fit-image.bbclass | 377 +++++++++++++++++++++++++++++++++ 1 file changed, 377 insertions(+) create mode 100644 meta/classes/fit-image.bbclass
meta/classes/fit-image.bbclass new file +377 -0
@@ -0,0 +1,377 @@ +# +# The class will facilitate the generation of FIT blobs. +# +# Glossary +# FIT - Flattened uImage Tree +# +# Requirements: +# +# * The user need to specify the image content using the format specified in the "FIT Image API" section. +# +# FIT Image API +# +# The bbclass is using variable and variable flags to declare the FIT image content. +# +# * Sub-Images and Configuration Nodes +# +# ** A sub-image node content is declared using the format `VAR_NODE[<property-name>] = <value>`. +# * VAR_NODE - freely selected name of the variable representing a node. +# * <property-name> - a sub-image property (e.g. description, type...). +# * <value> - the property value. +# Depending of the property the value can support different formats. +# ** Property Values Formats +# +# string property +# --------------- +# format: "<text>" - in case the property expects a text. +# (e.g. IMAGE_NODE_KERNEL[type] = "kernel") +# +# address property +# ---------------- +# format: "<address>" - in case the property expects an address. +# (e.g. IMAGE_NODE_KERNEL[entry] = "0xABCDABCD") +# +# hash property +# ------------- +# format: "<hash type>" - for hash property the hash type needs to be specified. +# (e.g. IMAGE_NODE_KERNEL[hash] = "sha256") +# +# sub-image signature property +# ---------------------------- +# format: "<algo>;<key-name-hint>;" - for image signature node. +# Both algorithm and key name needs to be provided. +# (e.g. IMAGE_NODE_KERNEL[signature] = "sha256,rsa2048;kernel;" +# +# configuration signature property +# -------------------------------- +# format: "<algo>;<key-name-hint>;<sub-image list>" - for configuration signature properties algorithm, +# key name and sub-image nodes needs to be provided. +# (e.g. CONF_NODE_CONF1[signature] = "sha256,rsa2048;kernel;"kernel","fdt";") +# +# ** Sub-Image and Configuration Nodes Flags +# See the code for supported flags. +# +# * FIT_IMAGES_NODE - contains a list of variables used to declare the sub-images nodes, separated by space. +# (e.g. FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL IMAGE_NODE_FDT") +# +# * FIT_CONFIGURATIONS_NODE - contains a list of variables used to declare the configuration nodes, +# separated by space. (e.g. FIT_CONFIGURATIONS_NODE = "CONF_NODE_CONF1") +# ** Flags +# - "default": used to configure the default configuration node. +# (e.g. FIT_CONFIGURATIONS_NODE[default] = "conf@0") +# +# Example: +# This is part of a linux_%.bbappend recipe. +# +# IMAGE_NODE_KERNEL[name] = "kernel@0" +# IMAGE_NODE_KERNEL[description] = "${PF}" +# IMAGE_NODE_KERNEL[data] = "/incbin/("./arch/${ARCH}/boot/zImage")" +# IMAGE_NODE_KERNEL[type] = "kernel" +# IMAGE_NODE_KERNEL[arch] = "${ARCH}" +# IMAGE_NODE_KERNEL[os] = "linux" +# IMAGE_NODE_KERNEL[compression] = "none" +# IMAGE_NODE_KERNEL[load] = "${UBOOT_LOADADDRESS}" +# IMAGE_NODE_KERNEL[entry] = "${UBOOT_ENTRYPOINT}" +# IMAGE_NODE_KERNEL[hash] = "sha256" +# +# IMAGE_NODE_FDT[name] = "fdt@0" +# IMAGE_NODE_FDT[type] = "flat_dt" +# IMAGE_NODE_FDT[arch] = "${ARCH}" +# IMAGE_NODE_FDT[compression] = "none" +# IMAGE_NODE_FDT[hash] = "sha256" +# +# CONF_NODE_CONF1[name] = "conf@0" +# CONF_NODE_CONF1[description] = "Linux kernel and FDT blob" +# CONF_NODE_CONF1[kernel] = "kernel@0" +# CONF_NODE_CONF1[fdt] = "fdt@0" +# +# FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL IMAGE_NODE_FDT" +# +# FIT_CONFIGURATIONS_NODE = "CONF_NODE_CONF1" +# FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF_NODE_CONF1', 'name') or ""}" +# + +DEPENDS += "\ + u-boot-mkimage-native \ +" + +FIT_IMAGE_DESCRIPTION ??= "Generic FIT image" +FIT_IMAGE_FILENAME ??= "image-fit" +FIT_IMAGE_UBOOT_MKIMAGE_OPTS ??= "" + + +def get_subimage_node_rules(): + """ + Defines the properties format and validation for sub-image nodes. + + :return: Return a dictionary with the format rules. + """ + + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['data'] = [True, '= $value', ["value"]] + rules['type'] = [True, '= "$value"', ["value"]] + rules['arch'] = [False, '= "$value"', ["value"]] + rules['os'] = [False, '= "$value"', ["value"]] + rules['compression'] = [True, '= "$value"', ["value"]] + rules['load'] = [False, '= <$value>', ["value"]] + rules['entry'] = [False, '= <$value>', ["value"]] + rules['hash'] = [False, '{ algo = "$algo"; }', ['algo']] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; }', ['algo', 'key']] + + return rules + + +def get_conf_node_rules(): + """ + Defines the properties format and validation for configuration nodes. + + :return: Return a dictionary with the format rules. + """ + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['kernel'] = [True, '= "$value"', ["value"]] + rules['ramdisk'] = [False, '= "$value"', ["value"]] + rules['fdt'] = [False, '= "$value"', ["value"]] + rules['loadables'] = [False, '= "$value"', ["value"]] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; sign-images = $images; }', + ['algo', 'key', 'images']] + + return rules + + +def generate_node(name, params, rules): + """ + Generates a node. + + :param name: Node name. + :param params: A dictionary containing the properties values. + :param rules: A dictionary containing the properties values validation and format. + + :return: A string containing the node, including the new line characters. + """ + from string import Template + + content = [] + + for rule in rules.keys(): + if rule in params.keys(): + content.append('{param} {value}; '.format( + param=rule, + value=Template(rules[rule][1]).substitute( + dict(zip(rules[rule][2], params[rule].split(';')))))) + elif rules[rule][0]: + bb.fatal('Missing mandatory parameter: {param} from {name} section'.format(param=rule, section=name)) + + content = """ + """.join(content) + node = """ {name} {{ + {content} + }}; + """.format(name=name, content=content) + + return node + + +def get_section_configuration(var, d): + """ + Generates a string build from variable's flags. + + :param var: variable to extract the flags. + :param d: bitbake environment. + + :return: A string with the format '<flag1> = <value1>; <flag2> = <value2>;...'. + """ + flags = d.getVarFlags(var) + if flags is not None: + flags = dict((flag, d.expand(value)) + for flag, value in list(flags.items())) + else: + flags = {} + + configuration = ''.join(( + """{name} = "{value}"; + """.format(name=name, value=value) for name, value in flags.items())) + + return configuration + + +def get_section_properties(var, d): + """ + Extract the nodes and parameters for a section. + + :param var: variable containing the variable names of the nodes that are part of this section. + :param d: bitbake environment. + + :return: a list containing dictionaries with section nodes parameters. + """ + nodes = [] + parameters = {} + + for node in d.getVar(var).split(): + parameters = d.getVarFlags(node) + if parameters is not None: + parameters = dict((parameter, d.expand(value)) + for parameter, value in list(parameters.items())) + nodes.append(parameters) + + return nodes + + +def generate_section(var, rules, d): + """ + Generates a section node (configuration or sub-image). + + :param var: Variable to extract the node names. + :param rules: Rules to use for generating this section. + :param d: bitbake environment. + + :return: A string containing the section, including the new line characters. + """ + + section = get_section_configuration(var, d) + + nodes_parameters = get_section_properties(var, d) + for parameters in nodes_parameters: + name = parameters.pop('name') + node = generate_node(name, parameters, rules) + section += node + + return section + + +def get_fit_image_template(): + """ + Get the FIT format. + + :return: A Template string containing the FIT image format. + """ + from string import Template + + template = Template("""/dts-v1/; + /{ + description = "$description"; + #address-cells = <1>; + images { + $images_section + }; + configurations { + $configurations_section + }; + };""") + return template + + +def generate_image_tree_source(d): + """ + Generates a string containing the image tree source. + + :return: A string representing the image tree. + """ + from string import Template + + values = {} + values['description'] = d.getVar('FIT_IMAGE_DESCRIPTION') + + image_rules = get_subimage_node_rules() + values['images_section'] = generate_section('FIT_IMAGES_NODE', image_rules, d) + + conf_rules = get_conf_node_rules() + values['configurations_section'] = generate_section('FIT_CONFIGURATIONS_NODE', conf_rules, d) + + image_tree_source = get_fit_image_template().substitute(values) + + return image_tree_source + + +def generate_image_blob(file_name, image, d): + """ + Generates a FIT blob. + + :param file_name: FIT blob file name. + :param image: String containing the image tree source. + :param d: Bitbake environment. + """ + import tempfile + import subprocess + + bb.debug(1, "Generated FIT source is:\n {image}".format(image=image)) + + builddir = d.getVar('B') + blob_file_name = file_name + '.itb' + + try: + fd, faux = tempfile.mkstemp(dir=builddir, prefix=file_name, suffix=".its") + with os.fdopen(fd, "w") as f: + f.write(image) + + mkimage_opts = d.getVar('FIT_IMAGE_UBOOT_MKIMAGE_OPTS').split() or "" + cmd = ['mkimage', '-f', faux] + cmd.extend(mkimage_opts) + cmd.append('{output_name}'.format(output_name=blob_file_name)) + + ret = subprocess.run( + cmd, + check=True, + universal_newlines=True, + cwd=builddir, + stderr=subprocess.STDOUT, + stdout=subprocess.PIPE) + + bb.debug(1, "Command for generating the FIT blob is:\n {cmd}".format(cmd=" ".join(ret.args))) + + except subprocess.CalledProcessError as e: + bb.fatal('Failed to generate the FIT blob: {message}: {output}'.format( + message=str(e), output=e.stdout)) + finally: + os.remove(faux) + + +def generate_fit_image(file_name, d): + """ + Create and generate FIT blob. + + :param file_name: FIT blob file name. + :param d: Bitbake environment. + """ + image = generate_image_tree_source(d) + generate_image_blob(file_name, image, d) + + +python fit-image_do_generate_fit_image() { + generate_fit_image(d.getVar('FIT_IMAGE_FILENAME'), d) +} + +do_generate_fit_image[vardeps] += " \ + ${FIT_CONFIGURATIONS_NODE} \ + ${FIT_IMAGES_NODE} \ + FIT_CONFIGURATIONS_NODE \ + FIT_IMAGES_NODE \ + FIT_IMAGE_FILENAME \ +" + +addtask do_generate_fit_image after do_compile before do_deploy + +EXPORT_FUNCTIONS do_generate_fit_image
Signature · 1 line
2.24.1
Re: [OE-core] [PATCH v2] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 50 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Signed-off-by Nandor Han <[email protected]>---
Notes:
Testing
-------1. linux-yocto_5.4.bbappend was modified to have the following configuration:
```
inherit fit-imageKERNEL_IMAGE_NODE[name] = "kernel" KERNEL_IMAGE_NODE[description] = "${PF}" KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' KERNEL_IMAGE_NODE[type] = "kernel" KERNEL_IMAGE_NODE[arch] = "${ARCH}" KERNEL_IMAGE_NODE[os] = "linux" KERNEL_IMAGE_NODE[compression] = "none" KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" KERNEL_IMAGE_NODE[hash] = "sha256"
FDT_IMAGE_NODE[name] = "fdt"
FDT_IMAGE_NODE[description] = "FDT blob"
FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' FDT_IMAGE_NODE[type] = "flat_dt"
FDT_IMAGE_NODE[arch] = "${ARCH}"
FDT_IMAGE_NODE[compression] = "none"
FDT_IMAGE_NODE[hash] = "sha256"CONF1_CONF_NODE[name] = "conf" CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" CONF1_CONF_NODE[kernel] = "kernel" CONF1_CONF_NODE[fdt] = "fdt"
Quoted 9 lines
FIT_IMAGES_NODE= "KERNEL_IMAGE_NODEFDT_IMAGE_NODE"FIT_CONFIGURATIONS_NODE= "CONF1_CONF_NODE" FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" ```
2. Build the kernel:bitbake virtual/kernel
3. Verify thatimage-fit.itbis present in the build directory: PASS
4. Disassemble the image using the command:dtc -I dtb -O dts image-fit.itb
5. Verify that the FIT source contains the expected configuration: PASS
Whilst I'm definitely interested in this, building the its file through dozens of variables is fragile and ugly...
What I had in a class that I'd started, but hadn't got anything like this far was something like a .its.in file which got passed through bitbake's expand.
Signature · 1 line
Alex Kiernan
Re: [OE-core] [PATCH v2] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 17 lines · Alex Kiernan
Quoted 16 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Signed-off-by Nandor Han <[email protected]>---
<snip>
Quoted 6 lines
Quoted 6 lines
```
2. Build the kernel:bitbake virtual/kernel
3. Verify thatimage-fit.itbis present in the build directory: PASS
4. Disassemble the image using the command:dtc -I dtb -O dts image-fit.itb
5. Verify that the FIT source contains the expected configuration: PASS
Hi Alex and thanks for feedback.
Quoted 8 lines
Whilst I'm definitely interested in this, building the its file through dozens of variables is fragile and ugly...
What I had in a class that I'd started, but hadn't got anything like this far was something like a .its.in file which got passed through bitbake's expand.
For most of the users or better say for a normal user, these variables will be probbly almost invisible. Like I mention in other answers, my plan is to refactor kernel-fitimage class to use this class. This way we can very easily customize the FIT source based on the user needs.
I started also by using an its.in template, however it turns out very quickly that is hard to maintain and hard to extend, given the FIT format possibility and user needs. For example in my situation in some conditions I wanted to have kernel as loadable image or my conf to contain or not a signature...
Nandor
PATCH v3 1/1 classes: Add a new bbclass that abstracts the generation of FIT blobs
In reply to [email protected]FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
---
Notes:
Testing
-------
1. linux-yocto_5.4.bbappend was modified to have the following configuration:
```
inherit fit-image
KERNEL_IMAGE_NODE[name] = "kernel" KERNEL_IMAGE_NODE[description] = "${PF}" KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' KERNEL_IMAGE_NODE[type] = "kernel" KERNEL_IMAGE_NODE[arch] = "${ARCH}" KERNEL_IMAGE_NODE[os] = "linux" KERNEL_IMAGE_NODE[compression] = "none" KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" KERNEL_IMAGE_NODE[hash] = "sha256"
FDT_IMAGE_NODE[name] = "fdt"
FDT_IMAGE_NODE[description] = "FDT blob"
FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' FDT_IMAGE_NODE[type] = "flat_dt"
FDT_IMAGE_NODE[arch] = "${ARCH}"
FDT_IMAGE_NODE[compression] = "none"
FDT_IMAGE_NODE[hash] = "sha256"
CONF1_CONF_NODE[name] = "conf" CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" CONF1_CONF_NODE[kernel] = "kernel" CONF1_CONF_NODE[fdt] = "fdt"
FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" ```
2. Build the kernel: bitbake virtual/kernel
3. Verify that image-fit.itb is present in the build directory: PASS
4. Disassemble the image using the command: dtc -I dtb -O dts image-fit.itb
5. Verify that the FIT source contains the expected configuration: PASS
Changes since v1:
----------------
- Change the format of short-log to "<target>: <summary>"
Changes since v2:
----------------
- rename the file from fit-image to fit_image to successfully export the class functions.
- adding new sanity checks.
- add missing dependency.
- fix a variable reference in a debug log.
meta/classes/fit_image.bbclass | 382 +++++++++++++++++++++++++++++++++ 1 file changed, 382 insertions(+) create mode 100644 meta/classes/fit_image.bbclass
meta/classes/fit_image.bbclass new file +382 -0
@@ -0,0 +1,382 @@ +# +# The class will facilitate the generation of FIT blobs. +# +# Glossary +# FIT - Flattened uImage Tree +# +# Requirements: +# +# * The user need to specify the image content using the format specified in the "FIT Image API" section. +# +# FIT Image API +# +# The bbclass is using variable and variable flags to declare the FIT image content. +# +# * Sub-Images and Configuration Nodes +# +# ** A sub-image node content is declared using the format `VAR_NODE[<property-name>] = <value>`. +# * VAR_NODE - freely selected name of the variable representing a node. +# * <property-name> - a sub-image property (e.g. description, type...). +# * <value> - the property value. +# Depending of the property the value can support different formats. +# ** Property Values Formats +# +# string property +# --------------- +# format: "<text>" - in case the property expects a text. +# (e.g. IMAGE_NODE_KERNEL[type] = "kernel") +# +# address property +# ---------------- +# format: "<address>" - in case the property expects an address. +# (e.g. IMAGE_NODE_KERNEL[entry] = "0xABCDABCD") +# +# hash property +# ------------- +# format: "<hash type>" - for hash property the hash type needs to be specified. +# (e.g. IMAGE_NODE_KERNEL[hash] = "sha256") +# +# sub-image signature property +# ---------------------------- +# format: "<algo>;<key-name-hint>;" - for image signature node. +# Both algorithm and key name needs to be provided. +# (e.g. IMAGE_NODE_KERNEL[signature] = "sha256,rsa2048;kernel;" +# +# configuration signature property +# -------------------------------- +# format: "<algo>;<key-name-hint>;<sub-image list>" - for configuration signature properties algorithm, +# key name and sub-image nodes needs to be provided. +# (e.g. CONF_NODE_CONF1[signature] = "sha256,rsa2048;kernel;"kernel","fdt";") +# +# ** Sub-Image and Configuration Nodes Flags +# See the code for supported flags. +# +# * FIT_IMAGES_NODE - contains a list of variables used to declare the sub-images nodes, separated by space. +# (e.g. FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL IMAGE_NODE_FDT") +# +# * FIT_CONFIGURATIONS_NODE - contains a list of variables used to declare the configuration nodes, +# separated by space. (e.g. FIT_CONFIGURATIONS_NODE = "CONF_NODE_CONF1") +# ** Flags +# - "default": used to configure the default configuration node. +# (e.g. FIT_CONFIGURATIONS_NODE[default] = "conf@0") +# +# Example: +# This is part of a linux_%.bbappend recipe. +# +# KERNEL_IMAGE_NODE[name] = "kernel" +# KERNEL_IMAGE_NODE[description] = "${PF}" +# KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' +# KERNEL_IMAGE_NODE[type] = "kernel" +# KERNEL_IMAGE_NODE[arch] = "${ARCH}" +# KERNEL_IMAGE_NODE[os] = "linux" +# KERNEL_IMAGE_NODE[compression] = "none" +# KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" +# KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" +# KERNEL_IMAGE_NODE[hash] = "sha256" +# +# FDT_IMAGE_NODE[name] = "fdt" +# FDT_IMAGE_NODE[description] = "FDT blob" +# FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' +# FDT_IMAGE_NODE[type] = "flat_dt" +# FDT_IMAGE_NODE[arch] = "${ARCH}" +# FDT_IMAGE_NODE[compression] = "none" +# FDT_IMAGE_NODE[hash] = "sha256" +# +# CONF1_CONF_NODE[name] = "conf" +# CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" +# CONF1_CONF_NODE[kernel] = "kernel" +# CONF1_CONF_NODE[fdt] = "fdt" +# +# FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" +# FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" +# FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" +# + +DEPENDS += "\ + dtc-native \ + u-boot-mkimage-native \ +" + +FIT_IMAGE_DESCRIPTION ??= "Generic FIT image" +FIT_IMAGE_FILENAME ??= "image-fit" +FIT_IMAGE_UBOOT_MKIMAGE_OPTS ??= "" + +def get_subimage_node_rules(): + """ + Defines the properties format and validation for sub-image nodes. + + :return: Return a dictionary with the format rules. + """ + + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['data'] = [True, '= $value', ["value"]] + rules['type'] = [True, '= "$value"', ["value"]] + rules['arch'] = [False, '= "$value"', ["value"]] + rules['os'] = [False, '= "$value"', ["value"]] + rules['compression'] = [True, '= "$value"', ["value"]] + rules['load'] = [False, '= <$value>', ["value"]] + rules['entry'] = [False, '= <$value>', ["value"]] + rules['hash'] = [False, '{ algo = "$algo"; }', ['algo']] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; }', ['algo', 'key']] + + return rules + + +def get_conf_node_rules(): + """ + Defines the properties format and validation for configuration nodes. + + :return: Return a dictionary with the format rules. + """ + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['kernel'] = [True, '= "$value"', ["value"]] + rules['ramdisk'] = [False, '= "$value"', ["value"]] + rules['fdt'] = [False, '= "$value"', ["value"]] + rules['loadables'] = [False, '= "$value"', ["value"]] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; sign-images = $images; }', + ['algo', 'key', 'images']] + + return rules + + +def generate_node(name, params, rules): + """ + Generates a node. + + :param name: Node name. + :param params: A dictionary containing the properties values. + :param rules: A dictionary containing the properties values validation and format. + + :return: A string containing the node, including the new line characters. + """ + from string import Template + + content = [] + + for rule in rules.keys(): + if rule in params.keys(): + content.append('{param} {value}; '.format( + param=rule, + value=Template(rules[rule][1]).substitute( + dict(zip(rules[rule][2], params[rule].split(';')))))) + elif rules[rule][0]: + bb.fatal('Missing mandatory parameter "{param}" from "{section}" section'.format(param=rule, section=name)) + + content = """ + """.join(content) + node = """ {name} {{ + {content} + }}; + """.format(name=name, content=content) + + return node + + +def get_section_configuration(var, d): + """ + Generates a string build from variable's flags. + + :param var: variable to extract the flags. + :param d: bitbake environment. + + :return: A string with the format '<flag1> = <value1>; <flag2> = <value2>;...'. + """ + flags = d.getVarFlags(var) + if flags is not None: + flags = dict((flag, d.expand(value)) + for flag, value in list(flags.items())) + else: + flags = {} + + configuration = ''.join(( + """{name} = "{value}"; + """.format(name=name, value=value) for name, value in flags.items())) + + return configuration + + +def get_section_properties(var, d): + """ + Extract the nodes and parameters for a section. + + :param var: variable containing the variable names of the nodes that are part of this section. + :param d: bitbake environment. + + :return: a list containing dictionaries with section nodes parameters. + """ + nodes = [] + parameters = {} + + for node in d.getVar(var).split(): + parameters = d.getVarFlags(node) + if parameters is not None: + parameters = dict((parameter, d.expand(value)) + for parameter, value in list(parameters.items())) + nodes.append(parameters) + + return nodes + + +def generate_section(var, rules, d): + """ + Generates a section node (configuration or sub-image). + + :param var: Variable to extract the node names. + :param rules: Rules to use for generating this section. + :param d: bitbake environment. + + :return: A string containing the section, including the new line characters. + """ + + section = get_section_configuration(var, d) + + nodes_parameters = get_section_properties(var, d) + for parameters in nodes_parameters: + name = parameters.pop('name') + node = generate_node(name, parameters, rules) + section += node + + return section + + +def get_fit_image_template(): + """ + Get the FIT format. + + :return: A Template string containing the FIT image format. + """ + from string import Template + + template = Template("""/dts-v1/; + /{ + description = "$description"; + #address-cells = <1>; + images { + $images_section + }; + configurations { + $configurations_section + }; + };""") + return template + + +def generate_image_tree_source(d): + """ + Generates a string containing the image tree source. + + :return: A string representing the image tree. + """ + from string import Template + + values = {} + values['description'] = d.getVar('FIT_IMAGE_DESCRIPTION') + + image_rules = get_subimage_node_rules() + if d.getVar('FIT_IMAGES_NODE', False) is None: + bb.fatal("Please add the FIT image nodes to FIT_IMAGES_NODE variable.") + values['images_section'] = generate_section('FIT_IMAGES_NODE', image_rules, d) + + conf_rules = get_conf_node_rules() + if d.getVar('FIT_CONFIGURATIONS_NODE', False) is None: + bb.fatal("Please add the FIT configuration nodes to FIT_CONFIGURATIONS_NODE variable.") + values['configurations_section'] = generate_section('FIT_CONFIGURATIONS_NODE', conf_rules, d) + + image_tree_source = get_fit_image_template().substitute(values) + + return image_tree_source + + +def generate_image_blob(file_name, image, d): + """ + Generates a FIT blob. + + :param file_name: FIT blob file name. + :param image: String containing the image tree source. + :param d: Bitbake environment. + """ + import tempfile + import subprocess + + bb.debug(1, "Generated FIT source is:\n {image}".format(image=image)) + + builddir = d.getVar('B') + blob_file_name = file_name + '.itb' + + try: + fd, faux = tempfile.mkstemp(dir=builddir, prefix=file_name, suffix=".its") + with os.fdopen(fd, "w") as f: + f.write(image) + + mkimage_opts = d.getVar('FIT_IMAGE_UBOOT_MKIMAGE_OPTS').split() or "" + cmd = ['mkimage', '-f', faux] + cmd.extend(mkimage_opts) + cmd.append('{output_name}'.format(output_name=blob_file_name)) + + ret = subprocess.run( + cmd, + check=True, + universal_newlines=True, + cwd=builddir, + stderr=subprocess.STDOUT, + stdout=subprocess.PIPE) + + bb.debug(1, "Command for generating the FIT blob is:\n {cmd}".format(cmd=" ".join(ret.args))) + + except subprocess.CalledProcessError as e: + bb.fatal('Failed to generate the FIT blob: {message}: {output}'.format( + message=str(e), output=e.stdout)) + finally: + os.remove(faux) + + +def generate_fit_image(file_name, d): + """ + Create and generate FIT blob. + + :param file_name: FIT blob file name. + :param d: Bitbake environment. + """ + image = generate_image_tree_source(d) + generate_image_blob(file_name, image, d) + + +python fit_image_do_generate_fit_image() { + generate_fit_image(d.getVar('FIT_IMAGE_FILENAME'), d) +} + +do_generate_fit_image[vardeps] += " \ + ${FIT_CONFIGURATIONS_NODE} \ + ${FIT_IMAGES_NODE} \ + FIT_CONFIGURATIONS_NODE \ + FIT_IMAGES_NODE \ + FIT_IMAGE_FILENAME \ +" + +addtask do_generate_fit_image after do_compile before do_deploy + +EXPORT_FUNCTIONS do_generate_fit_image
Signature · 1 line
2.24.1
Re: [OE-core] [PATCH v3] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 9 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
Quoted 6 lines
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Signed-off-by Nandor Han <[email protected]>
This class looks very useful, but I did have a question. How would you account for creating nodes dynamically? One use case of this would be adding a DT node for each reference in KERNEL_DEVICETREE. Would that functionality be expected to go in the recipe including this class or the class itself?
Thanks,
Zach
Quoted 452 lines
---
Notes:
Testing
-------1. linux-yocto_5.4.bbappend was modified to have the following configuration:
```
inherit fit-imageKERNEL_IMAGE_NODE[name] = "kernel" KERNEL_IMAGE_NODE[description] = "${PF}" KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' KERNEL_IMAGE_NODE[type] = "kernel" KERNEL_IMAGE_NODE[arch] = "${ARCH}" KERNEL_IMAGE_NODE[os] = "linux" KERNEL_IMAGE_NODE[compression] = "none" KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" KERNEL_IMAGE_NODE[hash] = "sha256"
FDT_IMAGE_NODE[name] = "fdt"
FDT_IMAGE_NODE[description] = "FDT blob"
FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' FDT_IMAGE_NODE[type] = "flat_dt"
FDT_IMAGE_NODE[arch] = "${ARCH}"
FDT_IMAGE_NODE[compression] = "none"
FDT_IMAGE_NODE[hash] = "sha256"CONF1_CONF_NODE[name] = "conf" CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" CONF1_CONF_NODE[kernel] = "kernel" CONF1_CONF_NODE[fdt] = "fdt"
FIT_IMAGES_NODE= "KERNEL_IMAGE_NODEFDT_IMAGE_NODE"FIT_CONFIGURATIONS_NODE= "CONF1_CONF_NODE" FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" ```
2. Build the kernel:bitbake virtual/kernel
3. Verify thatimage-fit.itbis present in the build directory: PASS
4. Disassemble the image using the command:dtc -I dtb -O dts image-fit.itb
5. Verify that the FIT source contains the expected configuration: PASSChanges since v1:
----------------
- Change the format of short-log to "<target>: <summary>"Changes since v2:
----------------
- rename the file fromfit-imagetofit_imageto successfully export the class functions.
- adding new sanity checks.
- add missing dependency.
- fix a variable reference in a debug log.meta/classes/fit_image.bbclass | 382 +++++++++++++++++++++++++++++++++ 1 file changed, 382 insertions(+) create mode 100644 meta/classes/fit_image.bbclassmeta/classes/fit_image.bbclass new file +382
-1@@ -0,0 +1,382 @@ +# +# The class will facilitate the generation of FIT blobs. +# +# Glossary +# FIT - Flattened uImage Tree +# +# Requirements: +# +# * The user need to specify the image content using the format specified in the "FIT Image API" section. +# +# FIT Image API +# +# The bbclass is using variable and variable flags to declare the FIT image content. +# +# * Sub-Images and Configuration Nodes +# +# ** A sub-image node content is declared using the format `VAR_NODE[<property-name>] = <value>`. +# * VAR_NODE - freely selected name of the variable representing a node. +# * <property-name> - a sub-image property (e.g. description, type...). +# * <value> - the property value. +# Depending of the property the value can support different formats. +# ** Property Values Formats +# +# string property +# --------------- +# format: "<text>" - in case the property expects a text. +# (e.g. IMAGE_NODE_KERNEL[type] = "kernel") +# +# address property +# ---------------- +# format: "<address>" - in case the property expects an address. +# (e.g. IMAGE_NODE_KERNEL[entry] = "0xABCDABCD") +# +# hash property +# ------------- +# format: "<hash type>" - for hash property the hash type needs to be specified. +# (e.g. IMAGE_NODE_KERNEL[hash] = "sha256") +# +# sub-image signature property +# ---------------------------- +# format: "<algo>;<key-name-hint>;" - for image signature node. +# Both algorithm and key name needs to be provided. +# (e.g. IMAGE_NODE_KERNEL[signature] = "sha256,rsa2048;kernel;" +# +# configuration signature property +# -------------------------------- +# format: "<algo>;<key-name-hint>;<sub-image list>" - for configuration signature properties algorithm, +# key name and sub-image nodes needs to be provided. +# (e.g. CONF_NODE_CONF1[signature] = "sha256,rsa2048;kernel;"kernel","fdt";") +# +# ** Sub-Image and Configuration Nodes Flags +# See the code for supported flags. +# +# * FIT_IMAGES_NODE - contains a list of variables used to declare the sub-images nodes, separated by space. +# (e.g. FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL IMAGE_NODE_FDT") +# +# * FIT_CONFIGURATIONS_NODE - contains a list of variables used to declare the configuration nodes, +# separated by space. (e.g. FIT_CONFIGURATIONS_NODE = "CONF_NODE_CONF1") +# ** Flags +# - "default": used to configure the default configuration node. +# (e.g. FIT_CONFIGURATIONS_NODE[default] = "conf@0") +# +# Example: +# This is part of a linux_%.bbappend recipe. +# +# KERNEL_IMAGE_NODE[name] = "kernel" +# KERNEL_IMAGE_NODE[description] = "${PF}" +# KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' +# KERNEL_IMAGE_NODE[type] = "kernel" +# KERNEL_IMAGE_NODE[arch] = "${ARCH}" +# KERNEL_IMAGE_NODE[os] = "linux" +# KERNEL_IMAGE_NODE[compression] = "none" +# KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" +# KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" +# KERNEL_IMAGE_NODE[hash] = "sha256" +# +# FDT_IMAGE_NODE[name] = "fdt" +# FDT_IMAGE_NODE[description] = "FDT blob" +# FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' +# FDT_IMAGE_NODE[type] = "flat_dt" +# FDT_IMAGE_NODE[arch] = "${ARCH}" +# FDT_IMAGE_NODE[compression] = "none" +# FDT_IMAGE_NODE[hash] = "sha256" +# +# CONF1_CONF_NODE[name] = "conf" +# CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" +# CONF1_CONF_NODE[kernel] = "kernel" +# CONF1_CONF_NODE[fdt] = "fdt" +# +# FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" +# FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" +# FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" +# + +DEPENDS += "\ + dtc-native \ + u-boot-mkimage-native \ +" + +FIT_IMAGE_DESCRIPTION ??= "Generic FIT image" +FIT_IMAGE_FILENAME ??= "image-fit" +FIT_IMAGE_UBOOT_MKIMAGE_OPTS ??= "" + +def get_subimage_node_rules(): + """ + Defines the properties format and validation for sub-image nodes. + + :return: Return a dictionary with the format rules. + """ + + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['data'] = [True, '= $value', ["value"]] + rules['type'] = [True, '= "$value"', ["value"]] + rules['arch'] = [False, '= "$value"', ["value"]] + rules['os'] = [False, '= "$value"', ["value"]] + rules['compression'] = [True, '= "$value"', ["value"]] + rules['load'] = [False, '= <$value>', ["value"]] + rules['entry'] = [False, '= <$value>', ["value"]] + rules['hash'] = [False, '{ algo = "$algo"; }', ['algo']] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; }', ['algo', 'key']] + + return rules + + +def get_conf_node_rules(): + """ + Defines the properties format and validation for configuration nodes. + + :return: Return a dictionary with the format rules. + """ + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['kernel'] = [True, '= "$value"', ["value"]] + rules['ramdisk'] = [False, '= "$value"', ["value"]] + rules['fdt'] = [False, '= "$value"', ["value"]] + rules['loadables'] = [False, '= "$value"', ["value"]] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; sign-images = $images; }', + ['algo', 'key', 'images']] + + return rules + + +def generate_node(name, params, rules): + """ + Generates a node. + + :param name: Node name. + :param params: A dictionary containing the properties values. + :param rules: A dictionary containing the properties values validation and format. + + :return: A string containing the node, including the new line characters. + """ + from string import Template + + content = [] + + for rule in rules.keys(): + if rule in params.keys(): + content.append('{param} {value}; '.format( + param=rule, + value=Template(rules[rule][1]).substitute( + dict(zip(rules[rule][2], params[rule].split(';')))))) + elif rules[rule][0]: + bb.fatal('Missing mandatory parameter "{param}" from "{section}" section'.format(param=rule, section=name)) + + content = """ + """.join(content) + node = """ {name} {{ + {content} + }}; + """.format(name=name, content=content) + + return node + + +def get_section_configuration(var, d): + """ + Generates a string build from variable's flags. + + :param var: variable to extract the flags. + :param d: bitbake environment. + + :return: A string with the format '<flag1> = <value1>; <flag2> = <value2>;...'. + """ + flags = d.getVarFlags(var) + if flags is not None: + flags = dict((flag, d.expand(value)) + for flag, value in list(flags.items())) + else: + flags = {} + + configuration = ''.join(( + """{name} = "{value}"; + """.format(name=name, value=value) for name, value in flags.items())) + + return configuration + + +def get_section_properties(var, d): + """ + Extract the nodes and parameters for a section. + + :param var: variable containing the variable names of the nodes that are part of this section. + :param d: bitbake environment. + + :return: a list containing dictionaries with section nodes parameters. + """ + nodes = [] + parameters = {} + + for node in d.getVar(var).split(): + parameters = d.getVarFlags(node) + if parameters is not None: + parameters = dict((parameter, d.expand(value)) + for parameter, value in list(parameters.items())) + nodes.append(parameters) + + return nodes + + +def generate_section(var, rules, d): + """ + Generates a section node (configuration or sub-image). + + :param var: Variable to extract the node names. + :param rules: Rules to use for generating this section. + :param d: bitbake environment. + + :return: A string containing the section, including the new line characters. + """ + + section = get_section_configuration(var, d) + + nodes_parameters = get_section_properties(var, d) + for parameters in nodes_parameters: + name = parameters.pop('name') + node = generate_node(name, parameters, rules) + section += node + + return section + + +def get_fit_image_template(): + """ + Get the FIT format. + + :return: A Template string containing the FIT image format. + """ + from string import Template + + template = Template("""/dts-v1/; + /{ + description = "$description"; + #address-cells = <1>; + images { + $images_section + }; + configurations { + $configurations_section + }; + };""") + return template + + +def generate_image_tree_source(d): + """ + Generates a string containing the image tree source. + + :return: A string representing the image tree. + """ + from string import Template + + values = {} + values['description'] = d.getVar('FIT_IMAGE_DESCRIPTION') + + image_rules = get_subimage_node_rules() + if d.getVar('FIT_IMAGES_NODE', False) is None: + bb.fatal("Please add the FIT image nodes to FIT_IMAGES_NODE variable.") + values['images_section'] = generate_section('FIT_IMAGES_NODE', image_rules, d) + + conf_rules = get_conf_node_rules() + if d.getVar('FIT_CONFIGURATIONS_NODE', False) is None: + bb.fatal("Please add the FIT configuration nodes to FIT_CONFIGURATIONS_NODE variable.") + values['configurations_section'] = generate_section('FIT_CONFIGURATIONS_NODE', conf_rules, d) + + image_tree_source = get_fit_image_template().substitute(values) + + return image_tree_source + + +def generate_image_blob(file_name, image, d): + """ + Generates a FIT blob. + + :param file_name: FIT blob file name. + :param image: String containing the image tree source. + :param d: Bitbake environment. + """ + import tempfile + import subprocess + + bb.debug(1, "Generated FIT source is:\n {image}".format(image=image)) + + builddir = d.getVar('B') + blob_file_name = file_name + '.itb' + + try: + fd, faux = tempfile.mkstemp(dir=builddir, prefix=file_name, suffix=".its") + with os.fdopen(fd, "w") as f: + f.write(image) + + mkimage_opts = d.getVar('FIT_IMAGE_UBOOT_MKIMAGE_OPTS').split() or "" + cmd = ['mkimage', '-f', faux] + cmd.extend(mkimage_opts) + cmd.append('{output_name}'.format(output_name=blob_file_name)) + + ret = subprocess.run( + cmd, + check=True, + universal_newlines=True, + cwd=builddir, + stderr=subprocess.STDOUT, + stdout=subprocess.PIPE) + + bb.debug(1, "Command for generating the FIT blob is:\n {cmd}".format(cmd=" ".join(ret.args))) + + except subprocess.CalledProcessError as e: + bb.fatal('Failed to generate the FIT blob: {message}: {output}'.format( + message=str(e), output=e.stdout)) + finally: + os.remove(faux) + + +def generate_fit_image(file_name, d): + """ + Create and generate FIT blob. + + :param file_name: FIT blob file name. + :param d: Bitbake environment. + """ + image = generate_image_tree_source(d) + generate_image_blob(file_name, image, d) + + +python fit_image_do_generate_fit_image() { + generate_fit_image(d.getVar('FIT_IMAGE_FILENAME'), d) +} + +do_generate_fit_image[vardeps] += " \ + ${FIT_CONFIGURATIONS_NODE} \ + ${FIT_IMAGES_NODE} \ + FIT_CONFIGURATIONS_NODE \ + FIT_IMAGES_NODE \ + FIT_IMAGE_FILENAME \ +" + +addtask do_generate_fit_image after do_compile before do_deploy + +EXPORT_FUNCTIONS do_generate_fit_image --2.24.1
Re: [OE-core] [PATCH v3] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 15 lines · Zach Booth
Quoted 14 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Quoted 8 lines
Quoted 1 line
Signed-off-by Nandor Han <[email protected]>This class looks very useful, but I did have a question. How would you account for creating nodes dynamically? One use case of this would be adding a DT node for each reference in
KERNEL_DEVICETREE. Would that functionality be expected to go in the recipe including this class or the class itself?
Thanks Zack for feedback. Like I mentioned in one of Rickard's answer, I'm planning to update the kernel-fitimage.bbclass to use this class.
In my local repo I have already a class that does what you're saying.
e.g
```
...
24 python do_generate_fit_image() {
25 import os.path
26
27 device_trees = (d.getVar("KERNEL_DEVICETREE").split())
28 kernel_key_path =
'{path}'.format(path=d.getVar("SECURITY_DIR_KEYS_RD") or "")
29 conf_signature = (d.getVar("CONF_NODE_CONF1") or "")
30 image_name = ""
31 mkimage_opts = ""
32
33 for dtb in device_trees:
34 d.setVarFlag("IMAGE_NODE_FDT", "data",
'/incbin/("./arch/{arch}/boot/dts/{dtb}")'.format(
35 arch=d.getVar("ARCH"), dtb=dtb))
36 d.setVarFlag("IMAGE_NODE_FDT", "description", dtb)
37
38 if os.path.exists(kernel_key_path):
39 image_name =
"kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".rdkeys")
40 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_SIGNED")
41 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", ("-k {key} -r
{extra}".format(
42 key=kernel_key_path, extra=mkimage_opts)))
43 generate_fit_image(image_name, d)
44
45 if not conf_signature:
46 d.delVarFlag("CONF_NODE_CONF1", "signature")
47
48 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_UNSIGNED")
49 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS",
"{extra}".format(extra=mkimage_opts))
50 image_name =
"kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".unsigned")
51 generate_fit_image(image_name, d)
52
53 if not conf_signature:
54 d.setVarFlag("CONF_NODE_CONF1", "signature",
conf_signature)
55 }
...
```The code above will generate a separate FIT blob for every device tree declared in KERNEL_DEVICETREE. However this functionality overlaps with kernel-fitimage and my target is to keep this class as simple as possible and refactor kernel-fitimage to use the fit_image class. The code above will go in kernel-fitimage.
Later on I'm planning to add a different class that can be use to generate U-Boot FIT blobs, which can be used by SPL.
So this is only the first step :)
Quoted 3 lines
Thanks,
Zach
<snip>
Nandor
Re: [OE-core] [PATCH v3] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 78 lines · Nandor Han
Quoted 15 lines · Zach Booth
Quoted 14 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Quoted 8 lines
Quoted 1 line
Signed-off-by Nandor Han <[email protected]>This class looks very useful, but I did have a question. How would you account for creating nodes dynamically? One use case of this would be adding a DT node for each reference in
KERNEL_DEVICETREE. Would that functionality be expected to go in the recipe including this class or the class itself?Thanks Zack for feedback. Like I mentioned in one of Rickard's answer, I'm planning to update the
kernel-fitimage.bbclassto use this class.In my local repo I have already a class that does what you're saying.
e.g
``` ... 24 python do_generate_fit_image() { 25 import os.path 26 27 device_trees = (d.getVar("KERNEL_DEVICETREE").split()) 28 kernel_key_path = '{path}'.format(path=d.getVar("SECURITY_DIR_KEYS_RD") or "") 29 conf_signature = (d.getVar("CONF_NODE_CONF1") or "") 30 image_name = "" 31 mkimage_opts = "" 32 33 for dtb in device_trees: 34 d.setVarFlag("IMAGE_NODE_FDT", "data", '/incbin/("./arch/{arch}/boot/dts/{dtb}")'.format( 35 arch=d.getVar("ARCH"), dtb=dtb)) 36 d.setVarFlag("IMAGE_NODE_FDT", "description", dtb) 37 38 if os.path.exists(kernel_key_path): 39 image_name = "kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".rdkeys") 40 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_SIGNED") 41 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", ("-k {key} -r {extra}".format( 42 key=kernel_key_path, extra=mkimage_opts))) 43 generate_fit_image(image_name, d) 44 45 if not conf_signature: 46 d.delVarFlag("CONF_NODE_CONF1", "signature") 47 48 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_UNSIGNED") 49 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", "{extra}".format(extra=mkimage_opts)) 50 image_name = "kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".unsigned") 51 generate_fit_image(image_name, d) 52 53 if not conf_signature: 54 d.setVarFlag("CONF_NODE_CONF1", "signature", conf_signature) 55 } ... ```
Quoted 2 lines
The code above will generate a separate FIT blob for every device tree declared in
KERNEL_DEVICETREE. However this functionality
So, no multiple device trees in a single FIT image? No multiple configurations, ramdisks, etc?
Quoted 18 lines
overlaps with
kernel-fitimageand my target is to keep this class as simple as possible and refactorkernel-fitimageto use thefit_imageclass. The code above will go inkernel-fitimage.Later on I'm planning to add a different class that can be use to generate U-Boot FIT blobs, which can be used by SPL.
So this is only the first step :)
Quoted 3 lines
Thanks,
Zach<snip>
Nandor
Re: [OE-core] [PATCH v3] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 79 lines · Denys Dmytriyenko
Quoted 78 lines · Nandor Han
Quoted 15 lines · Zach Booth
Quoted 14 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Quoted 8 lines
Quoted 1 line
Signed-off-by Nandor Han <[email protected]>This class looks very useful, but I did have a question. How would you account for creating nodes dynamically? One use case of this would be adding a DT node for each reference in
KERNEL_DEVICETREE. Would that functionality be expected to go in the recipe including this class or the class itself?Thanks Zack for feedback. Like I mentioned in one of Rickard's answer, I'm planning to update the
kernel-fitimage.bbclassto use this class.In my local repo I have already a class that does what you're saying.
e.g
``` ... 24 python do_generate_fit_image() { 25 import os.path 26 27 device_trees = (d.getVar("KERNEL_DEVICETREE").split()) 28 kernel_key_path = '{path}'.format(path=d.getVar("SECURITY_DIR_KEYS_RD") or "") 29 conf_signature = (d.getVar("CONF_NODE_CONF1") or "") 30 image_name = "" 31 mkimage_opts = "" 32 33 for dtb in device_trees: 34 d.setVarFlag("IMAGE_NODE_FDT", "data", '/incbin/("./arch/{arch}/boot/dts/{dtb}")'.format( 35 arch=d.getVar("ARCH"), dtb=dtb)) 36 d.setVarFlag("IMAGE_NODE_FDT", "description", dtb) 37 38 if os.path.exists(kernel_key_path): 39 image_name = "kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".rdkeys") 40 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_SIGNED") 41 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", ("-k {key} -r {extra}".format( 42 key=kernel_key_path, extra=mkimage_opts))) 43 generate_fit_image(image_name, d) 44 45 if not conf_signature: 46 d.delVarFlag("CONF_NODE_CONF1", "signature") 47 48 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_UNSIGNED") 49 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", "{extra}".format(extra=mkimage_opts)) 50 image_name = "kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".unsigned") 51 generate_fit_image(image_name, d) 52 53 if not conf_signature: 54 d.setVarFlag("CONF_NODE_CONF1", "signature", conf_signature) 55 } ... ```
Quoted 7 lines
Quoted 2 lines
The code above will generate a separate FIT blob for every device tree declared in
KERNEL_DEVICETREE. However this functionalitySo, no multiple device trees in a single FIT image? No multiple configurations, ramdisks, etc?
Given the class API is possible to configure your FIT source as you want. However, my plan is to refactor kernel-fitimage class to have this kind of features. The above code was only an example, something that I'm working on. I want to get this in and then it's easier to add new functionalities.
Quoted 9 lines
Quoted 9 lines
overlaps with
kernel-fitimageand my target is to keep this class as simple as possible and refactorkernel-fitimageto use thefit_imageclass. The code above will go inkernel-fitimage.Later on I'm planning to add a different class that can be use to generate U-Boot FIT blobs, which can be used by SPL.
So this is only the first step :)
Nandor
Re: [OE-core] [PATCH v3] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 83 lines · Nandor Han
Quoted 82 lines · Denys Dmytriyenko
Quoted 80 lines · Nandor Han
Quoted 15 lines · Zach Booth
Quoted 14 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Quoted 8 lines
Quoted 1 line
Signed-off-by Nandor Han <[email protected]>This class looks very useful, but I did have a question. How would you account for creating nodes dynamically? One use case of this would be adding a DT node for each reference in
KERNEL_DEVICETREE. Would that functionality be expected to go in the recipe including this class or the class itself?Thanks Zack for feedback. Like I mentioned in one of Rickard's answer, I'm planning to update the
kernel-fitimage.bbclassto use this class.In my local repo I have already a class that does what you're saying.
e.g
``` ... 24 python do_generate_fit_image() { 25 import os.path 26 27 device_trees = (d.getVar("KERNEL_DEVICETREE").split()) 28 kernel_key_path = '{path}'.format(path=d.getVar("SECURITY_DIR_KEYS_RD") or "") 29 conf_signature = (d.getVar("CONF_NODE_CONF1") or "") 30 image_name = "" 31 mkimage_opts = "" 32 33 for dtb in device_trees: 34 d.setVarFlag("IMAGE_NODE_FDT", "data", '/incbin/("./arch/{arch}/boot/dts/{dtb}")'.format( 35 arch=d.getVar("ARCH"), dtb=dtb)) 36 d.setVarFlag("IMAGE_NODE_FDT", "description", dtb) 37 38 if os.path.exists(kernel_key_path): 39 image_name = "kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".rdkeys") 40 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_SIGNED") 41 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", ("-k {key} -r {extra}".format( 42 key=kernel_key_path, extra=mkimage_opts))) 43 generate_fit_image(image_name, d) 44 45 if not conf_signature: 46 d.delVarFlag("CONF_NODE_CONF1", "signature") 47 48 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_UNSIGNED") 49 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", "{extra}".format(extra=mkimage_opts)) 50 image_name = "kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".unsigned") 51 generate_fit_image(image_name, d) 52 53 if not conf_signature: 54 d.setVarFlag("CONF_NODE_CONF1", "signature", conf_signature) 55 } ... ```The code above will generate a separate FIT blob for every device tree declared in
KERNEL_DEVICETREE. However this functionality
Quoted 8 lines
Quoted 2 lines
So, no multiple device trees in a single FIT image? No multiple configurations, ramdisks, etc?
Given the class API is possible to configure your FIT source as you want. However, my plan is to refactor
kernel-fitimageclass to have this kind of features. The above code was only an example, something that I'm working on. I want to get this in and then it's easier to add new functionalities.
Considering that fitimage is a critical functionality used by many BSPs (even though it's not used in OE-Core itself), breaking it would upset so many people... Not sure about this gradual approach.
Signature · 1 line
Denys
Quoted 12 lines
Quoted 9 lines
Quoted 9 lines
overlaps with
kernel-fitimageand my target is to keep this class as simple as possible and refactorkernel-fitimageto use thefit_imageclass. The code above will go inkernel-fitimage.Later on I'm planning to add a different class that can be use to generate U-Boot FIT blobs, which can be used by SPL.
So this is only the first step :)
Nandor
Re: [OE-core] [PATCH v3] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 93 lines · Denys Dmytriyenko
Quoted 83 lines · Nandor Han
Quoted 82 lines · Denys Dmytriyenko
Quoted 80 lines · Nandor Han
Quoted 15 lines · Zach Booth
Quoted 14 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Quoted 8 lines
Quoted 1 line
Signed-off-by Nandor Han <[email protected]>This class looks very useful, but I did have a question. How would you account for creating nodes dynamically? One use case of this would be adding a DT node for each reference in
KERNEL_DEVICETREE. Would that functionality be expected to go in the recipe including this class or the class itself?Thanks Zack for feedback. Like I mentioned in one of Rickard's answer, I'm planning to update the
kernel-fitimage.bbclassto use this class.In my local repo I have already a class that does what you're saying.
e.g
``` ... 24 python do_generate_fit_image() { 25 import os.path 26 27 device_trees = (d.getVar("KERNEL_DEVICETREE").split()) 28 kernel_key_path = '{path}'.format(path=d.getVar("SECURITY_DIR_KEYS_RD") or "") 29 conf_signature = (d.getVar("CONF_NODE_CONF1") or "") 30 image_name = "" 31 mkimage_opts = "" 32 33 for dtb in device_trees: 34 d.setVarFlag("IMAGE_NODE_FDT", "data", '/incbin/("./arch/{arch}/boot/dts/{dtb}")'.format( 35 arch=d.getVar("ARCH"), dtb=dtb)) 36 d.setVarFlag("IMAGE_NODE_FDT", "description", dtb) 37 38 if os.path.exists(kernel_key_path): 39 image_name = "kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".rdkeys") 40 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_SIGNED") 41 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", ("-k {key} -r {extra}".format( 42 key=kernel_key_path, extra=mkimage_opts))) 43 generate_fit_image(image_name, d) 44 45 if not conf_signature: 46 d.delVarFlag("CONF_NODE_CONF1", "signature") 47 48 mkimage_opts = d.getVar("FIT_IMAGE_MKIMAGE_OPTS_UNSIGNED") 49 d.setVar("FIT_IMAGE_UBOOT_MKIMAGE_OPTS", "{extra}".format(extra=mkimage_opts)) 50 image_name = "kernel-{dtb_name}{suffix}".format(dtb_name=os.path.splitext(dtb)[0], suffix=".unsigned") 51 generate_fit_image(image_name, d) 52 53 if not conf_signature: 54 d.setVarFlag("CONF_NODE_CONF1", "signature", conf_signature) 55 } ... ```The code above will generate a separate FIT blob for every device tree declared in
KERNEL_DEVICETREE. However this functionalityQuoted 8 lines
Quoted 2 lines
So, no multiple device trees in a single FIT image? No multiple configurations, ramdisks, etc?
Given the class API is possible to configure your FIT source as you want. However, my plan is to refactor
kernel-fitimageclass to have this kind of features. The above code was only an example, something that I'm working on. I want to get this in and then it's easier to add new functionalities.
Quoted 4 lines
Considering that fitimage is a critical functionality used by many BSPs (even though it's not used in OE-Core itself), breaking it would upset so many people... Not sure about this gradual approach.
Totally agree :). That's the beauty of this class...nothing is impacted. My plan is to build gradually on top of this class different classes and impact nobody about this. The users will be able to seamlessly take in use the final solution without any big changes.
Nandor
Re: [OE-core] [PATCH v3] classes: Add a new bbclass that abstracts the generation of FIT blobs
In reply to Nandor HanQuoted 8 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
Quoted 6 lines
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Signed-off-by Nandor Han <[email protected]>
Have you considered adding automated tests for this?
Does adding this class remove or obsolete any of the existing code/support?
Cheers,
Richard
Re: [OE-core] [PATCH v3] classes: Add a new bbclass that abstracts the generation of FIT blobs
Quoted 9 lines · Richard Purdie
Quoted 8 lines · Nandor Han
FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
Quoted 9 lines
Quoted 6 lines
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
Signed-off-by Nandor Han <[email protected]>Have you considered adding automated tests for this?
Thanks for feedback Rickard. I haven't considered adding any extra tests (automate), but I think is a great idea. I'll try to have a look.
Quoted 3 lines
Does adding this class remove or obsolete any of the existing code/support?
Currently no, AFAIK. But this is only the first step and my plan is to go forward and update the kernel-fitimage.bbclass to use this class. This way we can very easily extend the FIT source dynamically and in the same time reduce the complexity of the kernel-fitimage.bbclass. More over I'm planning to create also a U-Boot related class that will be able to generate a FIT blobs that can contain U-Boot... and is used by SPL.
Quoted 4 lines
Cheers,
Richard
Nandor
v4 cover Add a new bbclass that abstracts the generation of FIT blobs
In reply to [email protected]Description
----------
Add a new class and unittest for generating FIT blobs.
Testing
-------
1. linux-yocto_5.4.bbappend was modified to have the following configuration:
```
inherit fit-image
KERNEL_IMAGE_NODE[name] = "kernel" KERNEL_IMAGE_NODE[description] = "${PF}" KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' KERNEL_IMAGE_NODE[type] = "kernel" KERNEL_IMAGE_NODE[arch] = "${ARCH}" KERNEL_IMAGE_NODE[os] = "linux" KERNEL_IMAGE_NODE[compression] = "none" KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" KERNEL_IMAGE_NODE[hash] = "sha256"
FDT_IMAGE_NODE[name] = "fdt"
FDT_IMAGE_NODE[description] = "FDT blob"
FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' FDT_IMAGE_NODE[type] = "flat_dt"
FDT_IMAGE_NODE[arch] = "${ARCH}"
FDT_IMAGE_NODE[compression] = "none"
FDT_IMAGE_NODE[hash] = "sha256"
CONF1_CONF_NODE[name] = "conf" CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" CONF1_CONF_NODE[kernel] = "kernel" CONF1_CONF_NODE[fdt] = "fdt"
FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" ```
2. Build the kernel: bitbake virtual/kernel
3. Verify that image-fit.itb is present in the build directory: PASS
4. Disassemble the image using the command: dtc -I dtb -O dts image-fit.itb
5. Verify that the FIT source contains the expected configuration: PASS
6. Run the unittest using the command: oe-selftest --run-tests fit_image.FitImage
7. Verify that is successfully: PASS
```
2020-05-26 16:54:34,996 - oe-selftest - INFO - SUMMARY:
2020-05-26 16:54:34,996 - oe-selftest - INFO - oe-selftest () - Ran 13 tests in 1956.639s
2020-05-26 16:54:34,997 - oe-selftest - INFO - oe-selftest - OK - All required tests passed (successes=13, skipped=0, failures=0, errors=0) ```
Changes since v1:
----------------
- Change the format of short-log to "<target>: <summary>"
Changes since v2:
----------------
- rename the file from fit-image to fit_image to successfully export the class functions.
- adding new sanity checks.
- add missing dependency.
- fix a variable reference in a debug log.
Changes since v3:
----------------
- unit-test added
- class updated to support also properties for U-Boot image
Nandor Han (3):
Add a recipe for python3-fdt package
classes: Add a new bbclass that abstracts the generation of FIT blobs
selftest: add a unit-test for fit-image bbclass
.../fit-image-test/files/dt-fake.dtb | 3 + .../fit-image-test/files/zImage-fake | 3 + .../fit-image-test/fit-image-test.bb | 17 + meta/classes/fit_image.bbclass | 387 ++++++++++++++++++ meta/lib/oeqa/selftest/cases/fit_image.py | 212 ++++++++++ .../python/python3-fdt_0.2.0.bb | 14 + 6 files changed, 636 insertions(+) create mode 100644 meta-selftest/recipes-test/fit-image-test/files/dt-fake.dtb create mode 100644 meta-selftest/recipes-test/fit-image-test/files/zImage-fake create mode 100644 meta-selftest/recipes-test/fit-image-test/fit-image-test.bb create mode 100644 meta/classes/fit_image.bbclass create mode 100644 meta/lib/oeqa/selftest/cases/fit_image.py create mode 100644 meta/recipes-devtools/python/python3-fdt_0.2.0.bb
Signature · 1 line
2.24.1
PATCH v4 1/3 Add a recipe for `python3-fdt` package
The package python3-fdt is used for parsing and generating DTBs.
--- meta/recipes-devtools/python/python3-fdt_0.2.0.bb | 14 ++++++++++++++ 1 file changed, 14 insertions(+) create mode 100644 meta/recipes-devtools/python/python3-fdt_0.2.0.bb
meta/recipes-devtools/python/python3-fdt_0.2.0.bb new file +14 -0
@@ -0,0 +1,14 @@ +SUMMARY = "Library for manipulation of Device Tree Data" +HOMEPAGE = "https://github.com/molejar/pyFDT" +SECTION = "devel/python" +LICENSE = "Apache-2.0" +LIC_FILES_CHKSUM = "file://PKG-INFO;beginline=8;endline=8;md5=9a6ea5b6c346a830f54cc95f6a2a9245" + +SRC_URI[md5sum] = "a91daa36b3216f54feeac74ea8e5a475" +SRC_URI[sha256sum] = "b675139346946115513e27b5eed9aa882628ab74ed500bd5e25e122ee0afa2f6" + +PYPI_PACKAGE = "fdt" + +inherit pypi setuptools3 + +BBCLASSEXTEND = "native"
Signature · 1 line
2.24.1
PATCH v4 2/3 classes: Add a new bbclass that abstracts the generation of FIT blobs
In reply to Nandor HanFIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
--- meta/classes/fit_image.bbclass | 387 +++++++++++++++++++++++++++++++++ 1 file changed, 387 insertions(+) create mode 100644 meta/classes/fit_image.bbclass
meta/classes/fit_image.bbclass new file +387 -0
@@ -0,0 +1,387 @@ +# +# The class will facilitate the generation of FIT blobs. +# +# Glossary +# FIT - Flattened uImage Tree +# +# Requirements: +# +# * The user need to specify the image content using the format specified in the "FIT Image API" section. +# +# FIT Image API +# +# The bbclass is using variable and variable flags to declare the FIT image content. +# +# * Sub-Images and Configuration Nodes +# +# ** A sub-image node content is declared using the format `VAR_NODE[<property-name>] = <value>`. +# * VAR_NODE - freely selected name of the variable representing a node. +# * <property-name> - a sub-image property (e.g. description, type...). +# * <value> - the property value. +# Depending of the property the value can support different formats. +# ** Property Values Formats +# +# string property +# --------------- +# format: "<text>" - in case the property expects a text. +# (e.g. IMAGE_NODE_KERNEL[type] = "kernel") +# +# address property +# ---------------- +# format: "<address>" - in case the property expects an address. +# (e.g. IMAGE_NODE_KERNEL[entry] = "0xABCDABCD") +# +# hash property +# ------------- +# format: "<hash type>" - for hash property the hash type needs to be specified. +# (e.g. IMAGE_NODE_KERNEL[hash] = "sha256") +# +# sub-image signature property +# ---------------------------- +# format: "<algo>;<key-name-hint>;" - for image signature node. +# Both algorithm and key name needs to be provided. +# (e.g. IMAGE_NODE_KERNEL[signature] = "sha256,rsa2048;kernel;" +# +# configuration signature property +# -------------------------------- +# format: "<algo>;<key-name-hint>;<sub-image list>" - for configuration signature properties algorithm, +# key name and sub-image nodes needs to be provided. +# (e.g. CONF_NODE_CONF1[signature] = "sha256,rsa2048;kernel;"kernel","fdt";") +# +# ** Sub-Image and Configuration Nodes Flags +# See the code for supported flags. +# +# * FIT_IMAGES_NODE - contains a list of variables used to declare the sub-images nodes, separated by space. +# (e.g. FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL IMAGE_NODE_FDT") +# +# * FIT_CONFIGURATIONS_NODE - contains a list of variables used to declare the configuration nodes, +# separated by space. (e.g. FIT_CONFIGURATIONS_NODE = "CONF_NODE_CONF1") +# ** Flags +# - "default": used to configure the default configuration node. +# (e.g. FIT_CONFIGURATIONS_NODE[default] = "conf@0") +# +# Example: +# This is part of a linux_%.bbappend recipe. +# +# KERNEL_IMAGE_NODE[name] = "kernel" +# KERNEL_IMAGE_NODE[description] = "${PF}" +# KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' +# KERNEL_IMAGE_NODE[type] = "kernel" +# KERNEL_IMAGE_NODE[arch] = "${ARCH}" +# KERNEL_IMAGE_NODE[os] = "linux" +# KERNEL_IMAGE_NODE[compression] = "none" +# KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" +# KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" +# KERNEL_IMAGE_NODE[hash] = "sha256" +# +# FDT_IMAGE_NODE[name] = "fdt" +# FDT_IMAGE_NODE[description] = "FDT blob" +# FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' +# FDT_IMAGE_NODE[type] = "flat_dt" +# FDT_IMAGE_NODE[arch] = "${ARCH}" +# FDT_IMAGE_NODE[compression] = "none" +# FDT_IMAGE_NODE[hash] = "sha256" +# +# CONF1_CONF_NODE[name] = "conf" +# CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" +# CONF1_CONF_NODE[kernel] = "kernel" +# CONF1_CONF_NODE[fdt] = "fdt" +# +# FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" +# FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" +# FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" +# + +DEPENDS += "\ + dtc-native \ + u-boot-mkimage-native \ +" + +FIT_IMAGE_DESCRIPTION ??= "Generic FIT image" +FIT_IMAGE_FILENAME ??= "image-fit" +FIT_IMAGE_UBOOT_MKIMAGE_OPTS ??= "" + +def get_subimage_node_rules(): + """ + Defines the properties format and validation for sub-image nodes. + + :return: Return a dictionary with the format rules. + """ + + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['data'] = [True, '= $value', ["value"]] + rules['type'] = [True, '= "$value"', ["value"]] + rules['arch'] = [False, '= "$value"', ["value"]] + rules['os'] = [False, '= "$value"', ["value"]] + rules['compression'] = [True, '= "$value"', ["value"]] + rules['load'] = [False, '= <$value>', ["value"]] + rules['entry'] = [False, '= <$value>', ["value"]] + rules['hash'] = [False, '{ algo = "$algo"; }', ['algo']] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; }', ['algo', 'key']] + + return rules + + +def get_conf_node_rules(): + """ + Defines the properties format and validation for configuration nodes. + + :return: Return a dictionary with the format rules. + """ + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['kernel'] = [False, '= "$value"', ["value"]] + rules['ramdisk'] = [False, '= "$value"', ["value"]] + rules['firmware'] = [False, '= "$value"', ["value"]] + rules['standalone'] = [False, '= "$value"', ["value"]] + rules['fdt'] = [False, '= "$value"', ["value"]] + rules['loadables'] = [False, '= "$value"', ["value"]] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; sign-images = $images; }', + ['algo', 'key', 'images']] + + return rules + + +def generate_node(name, params, rules): + """ + Generates a node. + + :param name: Node name. + :param params: A dictionary containing the properties values. + :param rules: A dictionary containing the properties values validation and format. + + :return: A string containing the node, including the new line characters. + """ + from string import Template + + content = [] + + for rule in rules.keys(): + if rule in params.keys(): + content.append('{param} {value}; '.format( + param=rule, + value=Template(rules[rule][1]).substitute( + dict(zip(rules[rule][2], params[rule].split(';')))))) + elif rules[rule][0]: + bb.fatal('Missing mandatory parameter "{param}" from "{section}" section'.format(param=rule, section=name)) + + content = """ + """.join(content) + node = """ {name} {{ + {content} + }}; + """.format(name=name, content=content) + + return node + + +def get_section_configuration(var, d): + """ + Generates a string build from variable's flags. + + :param var: variable to extract the flags. + :param d: bitbake environment. + + :return: A string with the format '<flag1> = <value1>; <flag2> = <value2>;...'. + """ + flags = d.getVarFlags(var) + if flags is not None: + flags = dict((flag, d.expand(value)) + for flag, value in list(flags.items())) + else: + flags = {} + + configuration = ''.join(( + """{name} = "{value}"; + """.format(name=name, value=value) for name, value in flags.items())) + + return configuration + + +def get_section_properties(var, d): + """ + Extract the nodes and parameters for a section. + + :param var: variable containing the variable names of the nodes that are part of this section. + :param d: bitbake environment. + + :return: a list containing dictionaries with section nodes parameters. + """ + nodes = [] + parameters = {} + + for node in d.getVar(var).split(): + parameters = d.getVarFlags(node) + if parameters is not None: + parameters = dict((parameter, d.expand(value)) + for parameter, value in list(parameters.items())) + nodes.append(parameters) + + return nodes + + +def generate_section(var, rules, d): + """ + Generates a section node (configuration or sub-image). + + :param var: Variable to extract the node names. + :param rules: Rules to use for generating this section. + :param d: bitbake environment. + + :return: A string containing the section, including the new line characters. + """ + + section = get_section_configuration(var, d) + + nodes_parameters = get_section_properties(var, d) + for parameters in nodes_parameters: + try: + name = parameters.pop('name') + node = generate_node(name, parameters, rules) + section += node + except KeyError: + bb.fatal("Missing name property for node: {node}".format(node=var)) + + return section + + +def get_fit_image_template(): + """ + Get the FIT format. + + :return: A Template string containing the FIT image format. + """ + from string import Template + + template = Template("""/dts-v1/; + /{ + description = "$description"; + #address-cells = <1>; + images { + $images_section + }; + configurations { + $configurations_section + }; + };""") + return template + + +def generate_image_tree_source(d): + """ + Generates a string containing the image tree source. + + :return: A string representing the image tree. + """ + from string import Template + + values = {} + values['description'] = d.getVar('FIT_IMAGE_DESCRIPTION') + + image_rules = get_subimage_node_rules() + if d.getVar('FIT_IMAGES_NODE', False) is None: + bb.fatal("Please add the FIT image nodes to FIT_IMAGES_NODE variable.") + values['images_section'] = generate_section('FIT_IMAGES_NODE', image_rules, d) + + conf_rules = get_conf_node_rules() + if d.getVar('FIT_CONFIGURATIONS_NODE', False) is None: + bb.fatal("Please add the FIT configuration nodes to FIT_CONFIGURATIONS_NODE variable.") + values['configurations_section'] = generate_section('FIT_CONFIGURATIONS_NODE', conf_rules, d) + + image_tree_source = get_fit_image_template().substitute(values) + + return image_tree_source + + +def generate_image_blob(file_name, image, d): + """ + Generates a FIT blob. + + :param file_name: FIT blob file name. + :param image: String containing the image tree source. + :param d: Bitbake environment. + """ + import tempfile + import subprocess + + bb.debug(1, "Generated FIT source is:\n {image}".format(image=image)) + + builddir = d.getVar('B') + blob_file_name = file_name + '.itb' + + try: + fd, faux = tempfile.mkstemp(dir=builddir, prefix=file_name, suffix=".its") + with os.fdopen(fd, "w") as f: + f.write(image) + + mkimage_opts = d.getVar('FIT_IMAGE_UBOOT_MKIMAGE_OPTS').split() or "" + cmd = ['mkimage', '-f', faux] + cmd.extend(mkimage_opts) + cmd.append('{output_name}'.format(output_name=blob_file_name)) + + ret = subprocess.run( + cmd, + check=True, + universal_newlines=True, + cwd=builddir, + stderr=subprocess.STDOUT, + stdout=subprocess.PIPE) + + bb.debug(1, "Command for generating the FIT blob is:\n {cmd}".format(cmd=" ".join(ret.args))) + + except subprocess.CalledProcessError as e: + bb.fatal('Failed to generate the FIT blob: {message}: {output}'.format( + message=str(e), output=e.stdout)) + finally: + os.remove(faux) + + +def generate_fit_image(file_name, d): + """ + Create and generate FIT blob. + + :param file_name: FIT blob file name. + :param d: Bitbake environment. + """ + image = generate_image_tree_source(d) + generate_image_blob(file_name, image, d) + + +python fit_image_do_generate_fit_image() { + generate_fit_image(d.getVar('FIT_IMAGE_FILENAME'), d) +} + +do_generate_fit_image[vardeps] += " \ + ${FIT_CONFIGURATIONS_NODE} \ + ${FIT_IMAGES_NODE} \ + FIT_CONFIGURATIONS_NODE \ + FIT_IMAGES_NODE \ + FIT_IMAGE_FILENAME \ +" + +addtask do_generate_fit_image after do_compile before do_deploy + +EXPORT_FUNCTIONS do_generate_fit_image
Signature · 1 line
2.24.1
PATCH v4 3/3 selftest: add a unit-test for fit-image bbclass
In reply to Nandor HanThe unit-test will test the basic functionality of fit-image.bbclass.
--- .../fit-image-test/files/dt-fake.dtb | 3 + .../fit-image-test/files/zImage-fake | 3 + .../fit-image-test/fit-image-test.bb | 17 ++ meta/lib/oeqa/selftest/cases/fit_image.py | 212 ++++++++++++++++++ 4 files changed, 235 insertions(+) create mode 100644 meta-selftest/recipes-test/fit-image-test/files/dt-fake.dtb create mode 100644 meta-selftest/recipes-test/fit-image-test/files/zImage-fake create mode 100644 meta-selftest/recipes-test/fit-image-test/fit-image-test.bb create mode 100644 meta/lib/oeqa/selftest/cases/fit_image.py
meta-selftest/recipes-test/fit-image-test/files/dt-fake.dtb new file +3 -0
@@ -0,0 +1,3 @@ +Z�P�acn�ZWD1\�!΄�l�V +t(��H����z��jb4.���݁�W;I�ƶ����b�e���Ñ�5�x�&�9,��g +�;���!cV
meta-selftest/recipes-test/fit-image-test/files/zImage-fake new file +3 -0
@@ -0,0 +1,3 @@ +Z�P�acn�ZWD1\�!΄�l�V +t(��H����z��jb4.���݁�W;I�ƶ����b�e���Ñ�5�x�&�9,��g +�;���!cV
meta-selftest/recipes-test/fit-image-test/fit-image-test.bb new file +17 -0
@@ -0,0 +1,17 @@ +SUMMARY = "Recipe for testing the fit_image bbclass" +LICENSE = "MIT" +LIC_FILES_CHKSUM = "file://${COREBASE}/meta/COPYING.MIT;md5=3da9cfbcb788c80a0384361b4de20420" + +DEPENDS += "\ + python3-fdt-native \ +" + +SRC_URI = "\ + file://zImage-fake \ + file://dt-fake.dtb \ +" + +inherit fit_image + +include test_recipe.inc +
meta/lib/oeqa/selftest/cases/fit_image.py new file +212 -0
212 lines not on this page · raw message
Signature · 1 line
2.24.1
PATCH v3 1/1 classes: Add a new bbclass that abstracts the generation of FIT blobs
In reply to [email protected]FIT format is very versatile allowing various combination of booting sequences. In the same time different U-Boot boot stages can use FIT blobs to pack various binaries (e.g. SPL supports reading U-Boot from a FIT blob). Because of the allowed level of customization, the generation of a FIT blob using a fixed image tree source, becomes challenging and increase the level of complexity where different configurations and combinations are needed.
This bbclass will know how to generate a FIT blob, leaving the mechanics of the process (dependencies, task order...) to be handled by the users of the bbclass. In the same time will allow to separate the knowledge of the FIT format leaving the user code cleaner and more readable.
---
Notes:
Testing
-------
1. linux-yocto_5.4.bbappend was modified to have the following configuration:
```
inherit fit-image
KERNEL_IMAGE_NODE[name] = "kernel" KERNEL_IMAGE_NODE[description] = "${PF}" KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' KERNEL_IMAGE_NODE[type] = "kernel" KERNEL_IMAGE_NODE[arch] = "${ARCH}" KERNEL_IMAGE_NODE[os] = "linux" KERNEL_IMAGE_NODE[compression] = "none" KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" KERNEL_IMAGE_NODE[hash] = "sha256"
FDT_IMAGE_NODE[name] = "fdt"
FDT_IMAGE_NODE[description] = "FDT blob"
FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' FDT_IMAGE_NODE[type] = "flat_dt"
FDT_IMAGE_NODE[arch] = "${ARCH}"
FDT_IMAGE_NODE[compression] = "none"
FDT_IMAGE_NODE[hash] = "sha256"
CONF1_CONF_NODE[name] = "conf" CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" CONF1_CONF_NODE[kernel] = "kernel" CONF1_CONF_NODE[fdt] = "fdt"
FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" ```
2. Build the kernel: bitbake virtual/kernel
3. Verify that image-fit.itb is present in the build directory: PASS
4. Disassemble the image using the command: dtc -I dtb -O dts image-fit.itb
5. Verify that the FIT source contains the expected configuration: PASS
Changes since v1:
----------------
- Change the format of short-log to "<target>: <summary>"
Changes since v2:
----------------
- rename the file from fit-image to fit_image to successfully export the class functions.
- adding new sanity checks.
- add missing dependency.
- fix a variable reference in a debug log.
meta/classes/fit_image.bbclass | 382 +++++++++++++++++++++++++++++++++ 1 file changed, 382 insertions(+) create mode 100644 meta/classes/fit_image.bbclass
meta/classes/fit_image.bbclass new file +382 -0
@@ -0,0 +1,382 @@ +# +# The class will facilitate the generation of FIT blobs. +# +# Glossary +# FIT - Flattened uImage Tree +# +# Requirements: +# +# * The user need to specify the image content using the format specified in the "FIT Image API" section. +# +# FIT Image API +# +# The bbclass is using variable and variable flags to declare the FIT image content. +# +# * Sub-Images and Configuration Nodes +# +# ** A sub-image node content is declared using the format `VAR_NODE[<property-name>] = <value>`. +# * VAR_NODE - freely selected name of the variable representing a node. +# * <property-name> - a sub-image property (e.g. description, type...). +# * <value> - the property value. +# Depending of the property the value can support different formats. +# ** Property Values Formats +# +# string property +# --------------- +# format: "<text>" - in case the property expects a text. +# (e.g. IMAGE_NODE_KERNEL[type] = "kernel") +# +# address property +# ---------------- +# format: "<address>" - in case the property expects an address. +# (e.g. IMAGE_NODE_KERNEL[entry] = "0xABCDABCD") +# +# hash property +# ------------- +# format: "<hash type>" - for hash property the hash type needs to be specified. +# (e.g. IMAGE_NODE_KERNEL[hash] = "sha256") +# +# sub-image signature property +# ---------------------------- +# format: "<algo>;<key-name-hint>;" - for image signature node. +# Both algorithm and key name needs to be provided. +# (e.g. IMAGE_NODE_KERNEL[signature] = "sha256,rsa2048;kernel;" +# +# configuration signature property +# -------------------------------- +# format: "<algo>;<key-name-hint>;<sub-image list>" - for configuration signature properties algorithm, +# key name and sub-image nodes needs to be provided. +# (e.g. CONF_NODE_CONF1[signature] = "sha256,rsa2048;kernel;"kernel","fdt";") +# +# ** Sub-Image and Configuration Nodes Flags +# See the code for supported flags. +# +# * FIT_IMAGES_NODE - contains a list of variables used to declare the sub-images nodes, separated by space. +# (e.g. FIT_IMAGES_NODE = "IMAGE_NODE_KERNEL IMAGE_NODE_FDT") +# +# * FIT_CONFIGURATIONS_NODE - contains a list of variables used to declare the configuration nodes, +# separated by space. (e.g. FIT_CONFIGURATIONS_NODE = "CONF_NODE_CONF1") +# ** Flags +# - "default": used to configure the default configuration node. +# (e.g. FIT_CONFIGURATIONS_NODE[default] = "conf@0") +# +# Example: +# This is part of a linux_%.bbappend recipe. +# +# KERNEL_IMAGE_NODE[name] = "kernel" +# KERNEL_IMAGE_NODE[description] = "${PF}" +# KERNEL_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/zImage")' +# KERNEL_IMAGE_NODE[type] = "kernel" +# KERNEL_IMAGE_NODE[arch] = "${ARCH}" +# KERNEL_IMAGE_NODE[os] = "linux" +# KERNEL_IMAGE_NODE[compression] = "none" +# KERNEL_IMAGE_NODE[load] = "${UBOOT_LOADADDRESS}" +# KERNEL_IMAGE_NODE[entry] = "${UBOOT_ENTRYPOINT}" +# KERNEL_IMAGE_NODE[hash] = "sha256" +# +# FDT_IMAGE_NODE[name] = "fdt" +# FDT_IMAGE_NODE[description] = "FDT blob" +# FDT_IMAGE_NODE[data] = '/incbin/("./arch/${ARCH}/boot/dts/am335x-bone.dtb")' +# FDT_IMAGE_NODE[type] = "flat_dt" +# FDT_IMAGE_NODE[arch] = "${ARCH}" +# FDT_IMAGE_NODE[compression] = "none" +# FDT_IMAGE_NODE[hash] = "sha256" +# +# CONF1_CONF_NODE[name] = "conf" +# CONF1_CONF_NODE[description] = "Linux kernel and FDT blob" +# CONF1_CONF_NODE[kernel] = "kernel" +# CONF1_CONF_NODE[fdt] = "fdt" +# +# FIT_IMAGES_NODE = "KERNEL_IMAGE_NODE FDT_IMAGE_NODE" +# FIT_CONFIGURATIONS_NODE = "CONF1_CONF_NODE" +# FIT_CONFIGURATIONS_NODE[default] = "${@d.getVarFlag('CONF1_CONF_NODE', 'name') or ""}" +# + +DEPENDS += "\ + dtc-native \ + u-boot-mkimage-native \ +" + +FIT_IMAGE_DESCRIPTION ??= "Generic FIT image" +FIT_IMAGE_FILENAME ??= "image-fit" +FIT_IMAGE_UBOOT_MKIMAGE_OPTS ??= "" + +def get_subimage_node_rules(): + """ + Defines the properties format and validation for sub-image nodes. + + :return: Return a dictionary with the format rules. + """ + + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['data'] = [True, '= $value', ["value"]] + rules['type'] = [True, '= "$value"', ["value"]] + rules['arch'] = [False, '= "$value"', ["value"]] + rules['os'] = [False, '= "$value"', ["value"]] + rules['compression'] = [True, '= "$value"', ["value"]] + rules['load'] = [False, '= <$value>', ["value"]] + rules['entry'] = [False, '= <$value>', ["value"]] + rules['hash'] = [False, '{ algo = "$algo"; }', ['algo']] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; }', ['algo', 'key']] + + return rules + + +def get_conf_node_rules(): + """ + Defines the properties format and validation for configuration nodes. + + :return: Return a dictionary with the format rules. + """ + # + # Rules Format: [Mandatory, Template String, Dictionary keys] + # Mandatory: True, False - used to verify if this property is mandatory for generating the image. + # Note: Doesn't take in consideration the conditionally mandatory property. + # (see: U-Boot/doc/uImage.FIT/source_file_format.txt) + # Template String: Property content format. Used to generate the parameter content. + # Dictionary Keys: Keys used to be replaced in the Template String. + # + from collections import OrderedDict + + rules = OrderedDict() + + rules['description'] = [True, '= "$value"', ["value"]] + rules['kernel'] = [True, '= "$value"', ["value"]] + rules['ramdisk'] = [False, '= "$value"', ["value"]] + rules['fdt'] = [False, '= "$value"', ["value"]] + rules['loadables'] = [False, '= "$value"', ["value"]] + rules['signature'] = [False, '{ algo = "$algo"; key-name-hint = "$key"; sign-images = $images; }', + ['algo', 'key', 'images']] + + return rules + + +def generate_node(name, params, rules): + """ + Generates a node. + + :param name: Node name. + :param params: A dictionary containing the properties values. + :param rules: A dictionary containing the properties values validation and format. + + :return: A string containing the node, including the new line characters. + """ + from string import Template + + content = [] + + for rule in rules.keys(): + if rule in params.keys(): + content.append('{param} {value}; '.format( + param=rule, + value=Template(rules[rule][1]).substitute( + dict(zip(rules[rule][2], params[rule].split(';')))))) + elif rules[rule][0]: + bb.fatal('Missing mandatory parameter "{param}" from "{section}" section'.format(param=rule, section=name)) + + content = """ + """.join(content) + node = """ {name} {{ + {content} + }}; + """.format(name=name, content=content) + + return node + + +def get_section_configuration(var, d): + """ + Generates a string build from variable's flags. + + :param var: variable to extract the flags. + :param d: bitbake environment. + + :return: A string with the format '<flag1> = <value1>; <flag2> = <value2>;...'. + """ + flags = d.getVarFlags(var) + if flags is not None: + flags = dict((flag, d.expand(value)) + for flag, value in list(flags.items())) + else: + flags = {} + + configuration = ''.join(( + """{name} = "{value}"; + """.format(name=name, value=value) for name, value in flags.items())) + + return configuration + + +def get_section_properties(var, d): + """ + Extract the nodes and parameters for a section. + + :param var: variable containing the variable names of the nodes that are part of this section. + :param d: bitbake environment. + + :return: a list containing dictionaries with section nodes parameters. + """ + nodes = [] + parameters = {} + + for node in d.getVar(var).split(): + parameters = d.getVarFlags(node) + if parameters is not None: + parameters = dict((parameter, d.expand(value)) + for parameter, value in list(parameters.items())) + nodes.append(parameters) + + return nodes + + +def generate_section(var, rules, d): + """ + Generates a section node (configuration or sub-image). + + :param var: Variable to extract the node names. + :param rules: Rules to use for generating this section. + :param d: bitbake environment. + + :return: A string containing the section, including the new line characters. + """ + + section = get_section_configuration(var, d) + + nodes_parameters = get_section_properties(var, d) + for parameters in nodes_parameters: + name = parameters.pop('name') + node = generate_node(name, parameters, rules) + section += node + + return section + + +def get_fit_image_template(): + """ + Get the FIT format. + + :return: A Template string containing the FIT image format. + """ + from string import Template + + template = Template("""/dts-v1/; + /{ + description = "$description"; + #address-cells = <1>; + images { + $images_section + }; + configurations { + $configurations_section + }; + };""") + return template + + +def generate_image_tree_source(d): + """ + Generates a string containing the image tree source. + + :return: A string representing the image tree. + """ + from string import Template + + values = {} + values['description'] = d.getVar('FIT_IMAGE_DESCRIPTION') + + image_rules = get_subimage_node_rules() + if d.getVar('FIT_IMAGES_NODE', False) is None: + bb.fatal("Please add the FIT image nodes to FIT_IMAGES_NODE variable.") + values['images_section'] = generate_section('FIT_IMAGES_NODE', image_rules, d) + + conf_rules = get_conf_node_rules() + if d.getVar('FIT_CONFIGURATIONS_NODE', False) is None: + bb.fatal("Please add the FIT configuration nodes to FIT_CONFIGURATIONS_NODE variable.") + values['configurations_section'] = generate_section('FIT_CONFIGURATIONS_NODE', conf_rules, d) + + image_tree_source = get_fit_image_template().substitute(values) + + return image_tree_source + + +def generate_image_blob(file_name, image, d): + """ + Generates a FIT blob. + + :param file_name: FIT blob file name. + :param image: String containing the image tree source. + :param d: Bitbake environment. + """ + import tempfile + import subprocess + + bb.debug(1, "Generated FIT source is:\n {image}".format(image=image)) + + builddir = d.getVar('B') + blob_file_name = file_name + '.itb' + + try: + fd, faux = tempfile.mkstemp(dir=builddir, prefix=file_name, suffix=".its") + with os.fdopen(fd, "w") as f: + f.write(image) + + mkimage_opts = d.getVar('FIT_IMAGE_UBOOT_MKIMAGE_OPTS').split() or "" + cmd = ['mkimage', '-f', faux] + cmd.extend(mkimage_opts) + cmd.append('{output_name}'.format(output_name=blob_file_name)) + + ret = subprocess.run( + cmd, + check=True, + universal_newlines=True, + cwd=builddir, + stderr=subprocess.STDOUT, + stdout=subprocess.PIPE) + + bb.debug(1, "Command for generating the FIT blob is:\n {cmd}".format(cmd=" ".join(ret.args))) + + except subprocess.CalledProcessError as e: + bb.fatal('Failed to generate the FIT blob: {message}: {output}'.format( + message=str(e), output=e.stdout)) + finally: + os.remove(faux) + + +def generate_fit_image(file_name, d): + """ + Create and generate FIT blob. + + :param file_name: FIT blob file name. + :param d: Bitbake environment. + """ + image = generate_image_tree_source(d) + generate_image_blob(file_name, image, d) + + +python fit_image_do_generate_fit_image() { + generate_fit_image(d.getVar('FIT_IMAGE_FILENAME'), d) +} + +do_generate_fit_image[vardeps] += " \ + ${FIT_CONFIGURATIONS_NODE} \ + ${FIT_IMAGES_NODE} \ + FIT_CONFIGURATIONS_NODE \ + FIT_IMAGES_NODE \ + FIT_IMAGE_FILENAME \ +" + +addtask do_generate_fit_image after do_compile before do_deploy + +EXPORT_FUNCTIONS do_generate_fit_image
Signature · 1 line
2.24.1
✗ patchtest: failure for Add a new bbclass that abstracts the generation of FIT blobs (rev6)
In reply to [email protected]== Series Details ==
Series: Add a new bbclass that abstracts the generation of FIT blobs (rev6)
Revision: 6
URL : https://patchwork.openembedded.org/series/23431/
State : failure
== Summary ==
Thank you for submitting this patch series to OpenEmbedded Core. This is an automated response. Several tests have been executed on the proposed series by patchtest resulting in the following failures:
* Issue Series does not apply on top of target branch [test_series_merge_on_head]
Suggested fix Rebase your series on top of targeted branch
Targeted branch master (currently at 9106d0381d)
If you believe any of these test results are incorrect, please reply to the mailing list ([email protected]) raising your concerns. Otherwise we would appreciate you correcting the issues and submitting a new version of the patchset if applicable. Please ensure you add/increment the version number when sending the new version (i.e. [PATCH] -> [PATCH v2] -> [PATCH v3] -> ...).
---
Guidelines: https://www.openembedded.org/wiki/Commit_Patch_Message_Guidelines Test framework: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest Test suite: http://git.yoctoproject.org/cgit/cgit.cgi/patchtest-oe